Repository: tuffy/python-audio-tools Branch: master Commit: de55488dc982 Files: 339 Total size: 15.4 MB Directory structure: gitextract_4hj81sl5/ ├── COPYING ├── INSTALL ├── MANIFEST.in ├── Makefile ├── TODO ├── audiotools/ │ ├── __init__.py │ ├── accuraterip.py │ ├── aiff.py │ ├── ape.py │ ├── au.py │ ├── cdtoc.py │ ├── coverartarchive.py │ ├── cue/ │ │ ├── __init__.py │ │ ├── tokrules.py │ │ └── yaccrules.py │ ├── flac.py │ ├── freedb.py │ ├── id3.py │ ├── id3v1.py │ ├── image.py │ ├── m4a.py │ ├── m4a_atoms.py │ ├── mp3.py │ ├── mpc.py │ ├── musicbrainz.py │ ├── ogg.py │ ├── opus.py │ ├── player.py │ ├── ply/ │ │ ├── README │ │ ├── __init__.py │ │ ├── lex.py │ │ └── yacc.py │ ├── speex.py │ ├── text.py │ ├── toc/ │ │ ├── __init__.py │ │ ├── tokrules.py │ │ └── yaccrules.py │ ├── tta.py │ ├── ui.py │ ├── vorbis.py │ ├── vorbiscomment.py │ ├── wav.py │ └── wavpack.py ├── audiotools-config ├── cdda2track ├── cddainfo ├── cddaplay ├── coverbrowse ├── coverdump ├── covertag ├── docs/ │ ├── COPYING │ ├── Makefile │ ├── audiotools-config.xml │ ├── audiotools.cfg.xml │ ├── cdda2track.xml │ ├── cddainfo.xml │ ├── cddaplay.xml │ ├── coverbrowse.xml │ ├── coverdump.xml │ ├── covertag.xml │ ├── dvda2track.xml │ ├── dvdainfo.xml │ ├── manpagexml.py │ ├── programming/ │ │ ├── Makefile │ │ └── source/ │ │ ├── audiotools.rst │ │ ├── audiotools_accuraterip.rst │ │ ├── audiotools_bitstream.rst │ │ ├── audiotools_cdio.rst │ │ ├── audiotools_cue.rst │ │ ├── audiotools_dvda.rst │ │ ├── audiotools_freedb.rst │ │ ├── audiotools_musicbrainz.rst │ │ ├── audiotools_pcm.rst │ │ ├── audiotools_pcmconverter.rst │ │ ├── audiotools_player.rst │ │ ├── audiotools_replaygain.rst │ │ ├── audiotools_toc.rst │ │ ├── audiotools_ui.rst │ │ ├── conf.py │ │ ├── huffman.dot │ │ ├── index.rst │ │ └── metadata.rst │ ├── track2cdda.xml │ ├── track2track.xml │ ├── trackcat.xml │ ├── trackcmp.xml │ ├── trackinfo.xml │ ├── tracklength.xml │ ├── tracklint.xml │ ├── trackplay.xml │ ├── trackrename.xml │ ├── tracksplit.xml │ ├── tracktag.xml │ └── trackverify.xml ├── dvda2track ├── dvdainfo ├── setup.cfg ├── setup.py ├── src/ │ ├── COPYING.LESSERv3 │ ├── Makefile │ ├── accuraterip.c │ ├── accuraterip.h │ ├── bitstream-table.c │ ├── bitstream.c │ ├── bitstream.h │ ├── buffer.c │ ├── buffer.h │ ├── cdiomodule.c │ ├── cdiomodule.h │ ├── common/ │ │ ├── flac_crc.c │ │ ├── flac_crc.h │ │ ├── m4a_atoms.c │ │ ├── m4a_atoms.h │ │ ├── md5.c │ │ ├── md5.h │ │ ├── tta_crc.c │ │ └── tta_crc.h │ ├── decoders/ │ │ ├── NOTES.rst │ │ ├── alac.c │ │ ├── alac.h │ │ ├── alac_residual.h │ │ ├── alac_residual.json │ │ ├── flac.c │ │ ├── flac.h │ │ ├── mp3.c │ │ ├── mp3.h │ │ ├── mpc.c │ │ ├── mpc.h │ │ ├── oggflac.c │ │ ├── oggflac.h │ │ ├── opus.c │ │ ├── opus.h │ │ ├── sine.c │ │ ├── sine.h │ │ ├── tta.c │ │ ├── tta.h │ │ ├── vorbis.c │ │ ├── vorbis.h │ │ ├── wavpack.c │ │ └── wavpack.h │ ├── decoders.c │ ├── decoders.h │ ├── dither.c │ ├── dvdamodule.c │ ├── dvdamodule.h │ ├── encoders/ │ │ ├── NOTES.rst │ │ ├── alac.c │ │ ├── alac.h │ │ ├── flac.c │ │ ├── flac.h │ │ ├── mp2.c │ │ ├── mp3.c │ │ ├── mpc.c │ │ ├── opus.c │ │ ├── tta.c │ │ ├── tta.h │ │ ├── vorbis.c │ │ ├── wavpack.c │ │ └── wavpack.h │ ├── encoders.c │ ├── encoders.h │ ├── framelist.c │ ├── framelist.h │ ├── func_io.c │ ├── func_io.h │ ├── huffman.c │ ├── huffman.h │ ├── libmpcdec/ │ │ ├── AUTHORS │ │ ├── COPYING │ │ ├── ChangeLog │ │ ├── README │ │ ├── decoder.h │ │ ├── huffman.c │ │ ├── huffman.h │ │ ├── internal.h │ │ ├── mpc_bits_reader.c │ │ ├── mpc_bits_reader.h │ │ ├── mpc_decoder.c │ │ ├── mpc_demux.c │ │ ├── mpc_reader.c │ │ ├── mpcdec_math.h │ │ ├── requant.c │ │ ├── requant.h │ │ ├── streaminfo.c │ │ └── synth_filter.c │ ├── libmpcenc/ │ │ ├── analy_filter.c │ │ ├── bitstream.c │ │ ├── encode_sv7.c │ │ ├── huffsv7.c │ │ ├── libmpcenc.h │ │ └── quant.c │ ├── libmpcpsy/ │ │ ├── ans.c │ │ ├── cvd.c │ │ ├── fft4g.c │ │ ├── fft_routines.c │ │ ├── libmpcpsy.h │ │ ├── profile.c │ │ ├── psy.c │ │ └── psy_tab.c │ ├── mini-gmp.c │ ├── mini-gmp.h │ ├── mod_bitstream.c │ ├── mod_bitstream.h │ ├── mod_defs.h │ ├── mod_ogg.c │ ├── mod_ogg.h │ ├── mpc/ │ │ ├── datatypes.h │ │ ├── minimax.h │ │ ├── mpc_crc32.c │ │ ├── mpc_types.h │ │ ├── mpcdec.h │ │ ├── mpcmath.h │ │ ├── reader.h │ │ └── streaminfo.h │ ├── ogg.c │ ├── ogg.h │ ├── ogg_crc.c │ ├── ogg_crc.h │ ├── output/ │ │ ├── alsa.c │ │ ├── alsa.h │ │ ├── core_audio.c │ │ ├── core_audio.h │ │ ├── pulseaudio.c │ │ ├── pulseaudio.h │ │ ├── sfifo.c │ │ └── sfifo.h │ ├── output.c │ ├── parson.c │ ├── parson.h │ ├── pcm.c │ ├── pcm.h │ ├── pcm_conv.c │ ├── pcm_conv.h │ ├── pcmconverter.c │ ├── pcmconverter.h │ ├── pcmreader.c │ ├── pcmreader.h │ ├── read_bits_table_be.h │ ├── read_bits_table_le.h │ ├── read_unary_table_be.h │ ├── read_unary_table_le.h │ ├── replaygain.c │ ├── replaygain.h │ ├── samplerate/ │ │ ├── common.h │ │ ├── fastest_coeffs.h │ │ ├── float_cast.h │ │ ├── high_qual_coeffs.h │ │ ├── mid_qual_coeffs.h │ │ ├── samplerate.c │ │ ├── samplerate.h │ │ ├── src_linear.c │ │ ├── src_sinc.c │ │ └── src_zoh.c │ ├── unread_bit_table_be.h │ └── unread_bit_table_le.h ├── test/ │ ├── 1h.flac │ ├── 1m.flac │ ├── 1s.flac │ ├── aiff-1ch.aiff │ ├── aiff-2ch.aiff │ ├── aiff-6ch.aiff │ ├── aiff-8bit.aiff │ ├── aiff-metadata.aiff │ ├── aiff-misordered.aiff │ ├── aiff-nossnd.aiff │ ├── alac-allframes.m4a │ ├── apptest.sh │ ├── autotag.sh │ ├── cdda_test.cue │ ├── cdtoc1.flac │ ├── cdtoc2.flac │ ├── error.py │ ├── flac-allframes.flac │ ├── flac-disordered.flac │ ├── flac-id3-2.flac │ ├── flac-id3.flac │ ├── flac-nomask1.flac │ ├── flac-nomask2.flac │ ├── flac-nomask3.flac │ ├── flac-nomask4.flac │ ├── flac-nonmd5.flac │ ├── flac-noseektable.flac │ ├── flac-seektable.flac │ ├── freedb_test_discid-1.cue │ ├── freedb_test_discid-2.cue │ ├── freedb_test_discid-3.cue │ ├── freedb_test_discid-4.cue │ ├── freedb_test_discid-5.cue │ ├── huge.bmp.bz2 │ ├── image_test_metrics-6.tiff │ ├── imagetiff_setup.tiff │ ├── m4a-faac.m4a │ ├── m4a-faac2.m4a │ ├── m4a-faac3.m4a │ ├── m4a-itunes.m4a │ ├── m4a-nero.m4a │ ├── m4a-nero2.m4a │ ├── m4a-nero3.m4a │ ├── metadata_flac_cuesheet-1.cue │ ├── metadata_flac_cuesheet-2.cue │ ├── metadata_flac_cuesheet-3.cue │ ├── shorten-frames.shn │ ├── shorten-lpc.shn │ ├── silence.wv │ ├── sine.mp2 │ ├── test.cfg │ ├── test.py │ ├── test_cdrdao.py │ ├── test_cdrecord.py │ ├── test_core.py │ ├── test_formats.py │ ├── test_metadata.py │ ├── test_streams.py │ ├── test_utils.py │ ├── tone.flac │ ├── tone1.flac │ ├── tone2.flac │ ├── tone3.flac │ ├── tone4.flac │ ├── tone5.flac │ ├── tone6.flac │ ├── tone7.flac │ ├── tone8.flac │ ├── trackcat_pre_gap.cue │ ├── trackcat_pre_gap2.cue │ ├── trueaudio.tta │ └── wavpack-combo.wv ├── track2cdda ├── track2track ├── trackcat ├── trackcmp ├── trackinfo ├── tracklength ├── tracklint ├── trackplay ├── trackrename ├── tracksplit ├── tracktag └── trackverify ================================================ FILE CONTENTS ================================================ ================================================ FILE: COPYING ================================================ GNU GENERAL PUBLIC LICENSE Version 2, June 1991 Copyright (C) 1989, 1991 Free Software Foundation, Inc. 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Library General Public License instead.) You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things. To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it. For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software. Also, for each author's protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations. Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone's free use or not licensed at all. The precise terms and conditions for copying, distribution and modification follow. GNU GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 0. This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The "Program", below, refers to any such program or work, and a "work based on the Program" means either the Program or any derivative work under copyright law: that is to say, a work containing the Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term "modification".) Each licensee is addressed as "you". Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of having been made by running the Program). Whether that is true depends on what the Program does. 1. You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program. You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee. 2. You may modify your copy or copies of the Program or any portion of it, thus forming a work based on the Program, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions: a) You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change. b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License. c) If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the most ordinary way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions, and telling the user how to view a copy of this License. (Exception: if the Program itself is interactive but does not normally print such an announcement, your work based on the Program is not required to print an announcement.) These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it. Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program. In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License. 3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following: a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.) The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable. If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code. 4. You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 5. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Program (or any work based on the Program), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Program or works based on it. 6. Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties to this License. 7. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Program at all. For example, if a patent license would not permit royalty-free redistribution of the Program by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Program. If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply and the section as a whole is intended to apply in other circumstances. It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system, which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice. This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License. 8. If the distribution and/or use of the Program is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Program under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License. 9. The Free Software Foundation may publish revised and/or new versions of the General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of this License, you may choose any version ever published by the Free Software Foundation. 10. If you wish to incorporate parts of the Program into other free programs whose distribution conditions are different, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally. NO WARRANTY 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA Also add information on how to contact you by electronic and paper mail. If the program is interactive, make it output a short notice like this when it starts in an interactive mode: Gnomovision version 69, Copyright (C) year name of author Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, the commands you use may be called something other than `show w' and `show c'; they could even be mouse-clicks or menu items--whatever suits your program. You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the program, if necessary. Here is a sample; alter the names: Yoyodyne, Inc., hereby disclaims all copyright interest in the program `Gnomovision' (which makes passes at compilers) written by James Hacker. , 1 April 1989 Ty Coon, President of Vice This General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Library General Public License instead of this License. ================================================ FILE: INSTALL ================================================ Installation Procedure To install Python Audio Tools, you simply need to run: make install as root from this source directory. This will use the Python interpreter to install the audiotools Python module and the executable scripts. It will then install the man pages from the doc/ subdirectory. To verify your Python Audio Tools installation, run: audiotools-config as a normal user. This will load the audiotools Python module, if possible, and deliver a listing of available audio formats and current system settings. Fixing Installation Problems * The audiotools.cdio module doesn't build correctly Check that you have the CDIO library installed, commonly known as libcdio If libcdio is installed the module still doesn't build, ensure that you've also installed any accompanying libcdio-devel package. * audiotools-config lists formats as unavailable Certain audio formats require external programs. For instance, to use FLAC files, Python Audio Tools requires the flac and metaflac programs. If these cannot be found in the regular executable search path or from the config file, you will not be able to use that format. Check your system's package manager for programs which may be available but not yet installed. * My Python interpreter isn't found, or I wish to use a different one The first line of Makefile is which Python interpreter is being used for installation of both the Python Audio Tools and Construct module. For instance, to use a Python interpreter located at /opt/python/bin/python, you should change that line to read: export PYTHON = /opt/python/bin/python Running make will then invoke the new interpreter for installation of the audiotools module and scripts. For additional information, please see the complete manual in the "docs" subdirectory named "audiotools_letter.pdf" and "audiotools_a4.pdf". ================================================ FILE: MANIFEST.in ================================================ recursive-include audiotools * recursive-include docs * recursive-include src * recursive-include test * include audiotools-config include cdda2track include cddainfo include cddaplay include COPYING include coverbrowse include coverdump include dvda2track include dvdainfo include INSTALL include Makefile include setup.py include TODO include track2cdda include track2track include trackcat include trackcmp include trackinfo include tracklength include tracklint include trackplay include trackrename include tracksplit include tracktag include trackverify ================================================ FILE: Makefile ================================================ # which Python interpreter to use PYTHON = python # which Python test coverage utility to use COVERAGE = coverage all: .FORCE $(PYTHON) setup.py build install: .FORCE $(PYTHON) setup.py install cd docs && $(MAKE) install probe: .FORCE $(PYTHON) setup.py build_ext --dry-run check: .FORCE cd test && $(PYTHON) test.py check_coverage: .FORCE cd test && $(COVERAGE) run test.py coverage_report: .FORCE cd test && $(COVERAGE) report -m clean: .FORCE rm -rfv build rm -fv audiotools/*.pyc distclean: clean cd docs && $(MAKE) clean .FORCE: ================================================ FILE: TODO ================================================ # -*- Mode: Org -*- * Previous Issues ** DONE Fix WAVE/MP3 formats to support MP3 audio in a WAVE container Instead of WaveAudio generating a nasty "compression not supported" error, its is_type() classmethod should return False. And, MP3Audio's is_type() classmethod should check for MP3 compressed RIFF WAVE containers. This isn't likely to mess up decoding, but may confuse ID3 tagging. ** DONE Allow file template to be specified on the command line When making new files with track2track, cd2track, etc. ** DONE Update the website with the latest documentation ** DONE Allow a unified %(album_track_number)s file template field If there's no album number, it's simply 2 digits of track number. Otherwise, it's a combination of the two fields. For example, album_number = 2 and track_number = 13 results in "213" for a value. ** DONE Update trackcat to take a cuesheet argument when outputting FLACs Thus, one can perform: trackcat --cuesheet=file.cue file1.wav file2.wav file3.wav -t flac -o cd.flac which will embed "file.cue" into "cd.flac" using metaflac. Though no other format I'm aware of supports this kind of cuesheet handling, being able to easily build solid disc images of a single FLAC file is much of the reason for trackcat/tracksplit. ** DONE Don't remove undone tracklint entries from its undo DB Since its checksum will change anyway and no longer match, explicitly removing the entry is no longer necessary ** DONE Support FLAC padding If changes to FLAC metadata are small enough, write over the padding (if present) rather than rewrite the whole file - like metaflac. This approach should speed up tagging considerably. ** DONE Fix image support in ID3v2 Very large images can take a very long time to load. ** DONE Fix programs to key on album number and track number Certain programs, such as trackcmp, work on tracks across two directories and key on track number to determine which to compare to which. These need to be updated to use both track number and album number. ** DONE Adjust wave conversions to use album number, if present For example, converting track_number 15 and album_number 2 to WAVE should make a file "track215.cdda.wav" which then properly converts back to track_number 15 and album_number 2 if read. ** DONE Improve XMCD handling Support for XMCD files often breaks down if one or more tracks are missing. In some cases, there's no fix to be had (track2xmcd) but in most instances it should be made to work correctly. ** DONE Perform type inference wherever possible Anything with a single output file (trackcat and record2track) should be able to infer its output type from the track suffix, if possible. ** DONE Add "comment" field support to all metadata types Don't forget to add unit tests for comment field. *** TODO Sort "comment" fields correctly across all metadata types *** TODO Add --comment support to tracktag ** DONE Fix ID3v2 image support to handle Unicode descriptions The current implementation falls down on UTF-16 input, but I should have a solution from the COMM frame handler. ** DONE Limit ID3v2.2/2.3 to UCS-2 encoding The current implementation treats UCS-2 the same as UTF-16. This needs to be fixed so that really high unicode characters (above U+FFFF) are replaced with something within spec. ** DONE Unify ID3v2 frame handling In the beginning, there were text frames and Everything Else. Text frames were unicode strings, and Everything Else was a binary string of whatever. Now that ID3 is cluttered with APIC frames and COMM frames that need special treatment, ID3v2 needs an overhaul to more resemble FlacMetaData. *** DONE Ensure unknown frames are displayed correctly Anything that's not text, images or comments should get some sort of proper display instead of a Python object string. ** DONE Add app testing to the unit test suite Though not everything is unit-testable (such as the CD handling programs or anything X11) a lot of the batch programs are to some degree: - [X] coverdump - [X] track2track - [X] trackcat - [X] trackcmp - [X] tracklength - [X] tracklint - [X] trackrename - [X] tracksplit - [X] tracktag ** DONE Add verbosity levels to programs Every batch program should support a -V --verbose flag with options for "silence","normal" (the default) and "debug". Silence shuts off everything but error messages. Normal is standard output behavior. Debug for additional debugging output. - [X] cd2track - [X] cd2xmcd - [X] coverdump - [X] record2track - [X] track2cd (this will need to forward verbosity to cdrecord) - [X] track2track - [X] track2xmcd - [X] trackcmp - [X] trackrename - [X] tracksplit - [X] tracktag - [X] tracklint ** DONE Add compression percentage display to trackinfo Though not massively useful, it'd be neat to see just how compressed audio tracks are, as a percentage of their original size. ** DONE Add support for W??? frames to ID3v2 The various W??? frames are really just URLs and don't need to be displayed as hex-encoded blobs. ** DONE Add CUE/TOC support to track2cd It should be possible to burn a selection of tracks, or a disc image, from a cuesheet with all its indexes/ISRC/catalog data intact by passing --cue to track2cd. ** DONE Unify CUE/TOC support Cuesheets and cdrdao TOC files are largely interchangeable. They both feature a listing of track offsets and, optionally, CD-TEXT data, ISRCs and so on. These formats should be unified such that any program will handle them both automatically. - [X] tracksplit - [X] trackcat - [X] tracktag *** DONE Update docs to mention CUE/TOC interchangeability - [X] tracksplit - [X] trackcat - [X] tracktag *** DONE Support cuesheet from FlacMetaData directly Since we're parsing CUE/TOC files anyway, this data can be used to build FLAC CUESHEET blocks directly instead of punting this task to metaflac. *** DONE Add unit tests for TOC/CUE files, as well as embedded FLAC cuesheets ** DONE Update copyright text for 2009 ** DONE Preserve metadata when using trackcat Any fields shared by all tracks should be merged into metadata for the newly combined track. ** DONE Don't route data though WAVE files unless necessary Currently, track2track routes through WAVE if both ends happen to support foreign RIFF chunks, whether the files have such chunks or not. This behavior needs to be modified such that only source files which actually have foreign chunks, and a target format that supports them, results in a pass through RIFF WAVE. ** DONE Convert editxmcd to PyGTK Although the dialog(1)-based version works in terminals and is curses-based, it's extremely hokey, error-prone and doesn't support any cut & paste. This needs to be reimplemented in PyGTK (since coverview already uses it) and made into a stable app someone would want to use. *** DONE Update XMCD support The current handling of XMCD files treats them only as very primative AlbumMetaData implementations. This must be updated into something round-trippable if editxmcd is to be modernized. **** DONE Add XMCD unit tests **** DONE Update XMCD API documentation ** DONE Require Python 2.5 Since Python 2.4 is in bugfix-only mode and barely supported, it's best to move the minimum version to Python 2.5 or better (which has already been superceded by Python 2.6). This reduces the amount of Python versions to test on and allows the use of more modern Python features which makes the code less clunky. *** TODO Update documentation to mention Python 2.5 requirement. ** DONE Expand WavPack's APEv2 tag coverage WavPack's official specification defines APEv2 tags such as "Cuesheet" and "Cover Art" which the APEv2 standard does not. It would be helpful to make WavPack's APEv2 tags a superset of regular APEv2. *** DONE Add image support to WavePackAPEv2 *** DONE Add cuesheet support to WavePackAPEv2 ** DONE Build unified cuesheet interface Once both FLAC and WavPack support embedded cuesheets, there will need to be a unified interface to support them. I expect this will be a simple pair of get_cuesheet/set_cuesheet methods, probably attached to the AudioFiles themselves rather than to MetaData objects. *** DONE Alter FLAC-specific cuesheet documentation to be more general *** DONE Ensure cuesheets are transferred properly when transcoding *** DONE Update trackcat to use the interface *** DONE Update tracksplit to use the interface *** DONE Update track2cd to use the interface *** DONE Document cuesheet interface *** DONE Add unit tests for embedded cuesheets across all formats *** DONE Add cuesheet import option to tracktag This can also use the --cue flag, for consistency with other image-handling programs like tracksplit. If given with a single, album-length track, --cue will import a cuesheet. If given with multiple tracks or a single track that's too short, --cue will function like --xmcd and act as a metadata source. *** DONE Update track2xmcd to support getting an XMCD file from CD image ** DONE Convert to Muspack SV8 Now that Musepack SV8 is finalized, it should be the new default. The old SV7 command-line tools aren't well supported and don't seem to work outside of x86 platforms. SV7 streams are, in theory, backwards compatible so switching shouldn't be a problem. ** DONE Update coverview to look more standard It's currently a haphazard assortment of widgets rather than anything like a proper GTK app. It should be tweaked to look better. ** DONE Improve transcoding robustness Just about all of the to_pcm() and from_pcm() methods expect that their subprocess calls will work as expected. Though rare in practice, these need to be checked in case the child processes fail for any reason. *** DONE Check for invalid input/output files/permissions errors If an output file can't be read/written to for some reason (invalid permissions, etc.) generate a proper error message instead of throwing ugly IOExceptions or confusing errors. - [X] cd2track - [X] cd2xmcd - [X] coverdump - [X] editxmcd - [X] record2track - [X] track2cd - [X] track2track - [X] track2xmcd - [X] trackcat - [X] trackcmp - [X] trackinfo - [X] tracklength - [X] tracklint - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag ** DONE Make text output consistent Currently, command-line programs generate output using a selection of scattered print statements - often accompanied by if blocks when verbosity is indicated - and haphazardly filtered through unicode. This should be replaced by a unified message system similar to Python's built-in logging module which can abstract away these difficulties. *** DONE Convert tty output to gettext-based strings This will not only make output messages more consistent across the tools, but will also allow for foreign language translations in the future. - [X] cd2track - [X] cd2xmcd - [X] coverdump - [X] record2track - [X] track2cd - [X] track2track - [X] track2xmcd - [X] trackcat - [X] trackcmp - [X] trackinfo - [X] tracklength - [X] tracklint - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag **** DONE Convert output from audiotools module to gettext-based strings - [X] __aiff__.py - [X] __ape__.py - [X] __au__.py - [X] cue.py - [X] __flac__.py - [X] __freedb__.py - [X] __id3__.py - [X] __id3v1__.py - [X] __image__.py - [X] __init__.py - [X] __m4a__.py - [X] __mp3__.py - [X] __musepack__.py - [X] __speex__.py - [X] toc.py - [X] __vorbiscomment__.py - [X] __vorbis__.py - [X] __wavpack__.py - [X] __wav__.py *** DONE Add unit tests for tty output All programs which generate output should be unit tested so that all code paths are assured of printing the messages they're supposed to print, at the streams they're supposed to print on, and in the proper encoding settings. - [X] coverdump - [X] track2track - [X] track2xmcd - [X] trackcat - [X] trackcmp - [X] trackinfo - [X] tracklength - [X] tracklint - [X] trackrename - [X] tracksplit - [X] tracktag *** DONE Convert --help output to gettext-based strings *** DONE Convert GUI programs to gettext-based strings - [X] coverview - [X] editxmcd *** DONE Convert "Usage" output to gettext-based strings ** DONE Update tracksplit's man page It now supports more of track2track's options ** DONE Support total tracks/total albums metadata fields *** DONE Add support for fields in the metadata tags - [X] Add support in Vorbis Comments - [X] Add support in ID3v2 - [X] Add support in M4A - [X] Add support in APEv2 *** DONE Add support in utilities - [X] Add support in tracktag - [X] Add support in cd2track - [X] Add support in tracksplit - [X] Add support in trackcat *** DONE Add unit tests *** DONE Add fields to --format output *** DONE Update man pages with fields information ** DONE Integrate better MetaData merging There's a few areas in which MetaData from multiple sources must be merged in an intelligent manner, such as where tracksplit takes a source track an XMCD file. Now that a preliminary MetaData.merge() classmethod is in place, this process must be integrated consistently. - [X] track2track - [X] trackrename - [X] tracksplit - [X] tracktag *** DONE Add unit tests for MetaData merging process - [X] track2track - [X] trackrename - [X] tracksplit - [X] tracktag ** DONE Improve M4A metadata handling *** DONE Make M4A metadata updating less destructive Like FLAC, not all fields need to be wiped out when overwriting old metadata with new. *** DONE Add more M4A-specific unit tests ** DONE Add more system information to audiotools-config All BIN-referenced binaries should be accounted for. Thumbnailing status and requirements should be shown. ** DONE Add cdinfo utility Analagous to trackinfo, but for an inserted CD. This would be a better location for cd2xmcd's "-i" option. *** DONE Add cdinfo man page *** DONE Link cdinfo man page to other utility man pages ** DONE Add manual page for audiotools.cfg It'll be easier to check what the options are from a man page rather than having to check the website or PDF doc. ** DONE Convert vorbiscomment dependency to Python This would remove the last app-based MetaData-setting utility and may pave the way for adding cover art to Ogg Vorbis (assuming I can find the standard for a secondary stream of image data) ** DONE Add metadata deletion capability It would be helpful to have the low-level capability of deleting either part of a MetaData tag or the entire tag altogether. For example, deleting the "track_name" field would delete a Vorbis comment's "TITLE" field. Or, deleting the MetaData from MP3 would remove all the ID3v2/ID3v1 tags. *** DONE Add delattr to ID3v1 ** DONE Integrate pyconstruct as a submodule ** DONE Add undo capability to editxmcd ** DONE Add --cue option to track2xmcd One should be able to pull metadata from CD images without having to embed the cuesheet. *** DONE Add unit tests for track2xmcd's --cue option *** DONE Update man page ** DONE Group --help output more intelligently For tools with a large number of options (such as track2track or tracktag) the --help output is particularly jumbled. Use more of optparse's features to make this output clearer. - [X] cd2track - [X] cd2xmcd - [X] track2track - [X] track2xmcd - [X] tracksplit - [X] tracktag ** DONE Check for FLAC metadata chunk overflow Although APEv2 and ID3 tags support very large objects (hundreds of MB), FLAC metadata chunks have a maxmimum of about 16MB per chunk, which may be hit accidentally. ** DONE Fix or replace Python's built-in aifc module The current implementation suffers from bugs. *** DONE Document AIFF better ** DONE Add MusicBrainz support It would be helpful to have external metadata support beyond FreeDB, since FreeDB is very primitive. *** DONE Ensure that MusicBrainz is interchangeable with FreeDB/XMCD **** DONE Unify track2xmcd/track2mb, cd2xmcd/cd2mb Based on preliminary testing, MusicBrainz's output is better than FreeDB's but its album coverage is not as broad. In addition, nobody wants to run their albums through two separate tools in order to extract metadata for tagging. The best solution is for tools to try both and output the one that's most complete. **** DONE Extend editxmcd to MusicBrainz XML Although editxmcd was originally designed specifically for XMCD files and MusicBrainz's XML format differs radically, no one should have to know whether an album metadata file is one or the other. Therefore, editxmcd should be extended with additional fields to handle XML backend data if necessary. **** DONE Handle multiple Release entries with single Disc ID **** DONE Allow MusicBrainz XML output for new editxmcd files FreeDB output should also be an option, however. *** DONE Add MusicBrainz protocol/format documentation *** DONE Add MusicBrainz-specific unit tests - [X] track2track - [X] track2xmcd - [X] trackrename - [X] tracksplit - [X] tracktag *** DONE Update --help text to indicate MusicBrainz compatibility - [X] cd2xmcd - [X] editxmcd - [X] track2track - [X] track2xmcd - [X] trackrename - [X] tracksplit - [X] tracktag *** DONE Update man pages to indiciate MusicBrainz compatibility - [X] cd2xmcd - [X] editxmcd - [X] track2track - [X] track2xmcd - [X] trackrename - [X] tracksplit - [X] tracktag *** DONE Update documentation with MusicBrainz config file fields *** DONE Ensure missing XML fields are handled correctly The MusicBrainz XML spec allows most fields to be missing altogether (such as ). editxmcd should add these fields in the proper place if necessary. **** DONE Add unit tests for improperly reordered XML fields ** DONE Ensure .glade files are found Not all systems place Python data files in the same locations. ** DONE Convert to_pcm()/from_pcm() to FrameList-based I/O Passing specifically-sized blobs of binary data between conversion routines worked well when those routines are little more than subprocess black-boxes. However, this approach works less well whenever actual sample values are required, or when processing is needed. In those cases, going from integers to strings, converting the strings back to integers for processing, then bouncing them into strings once again becomes needless work. A more sensible approach is to keep all data as FrameList-compatible objects (stored as C-based lists of int32s behind-the-scenes) and convert that data to/from strings only at the beginning and end of processing. *** DONE Build C-based audiotools.pcm.FrameList object This needs to closely match audiotools.FrameList's functionality and combine all the PCM conversion features from audiotools.pcmreader **** DONE Integrate audiotools.pcm.FrameList with i_array structures **** DONE Make audiotools.pcm.FrameList into a standalone object So standalone test codecs can use them, such as "flacenc" **** DONE Unit test audiotools.pcm.FrameList *** DONE Convert FLAC encoder/decoder to use FrameList objects - [X] flacenc - [X] audiotools.decoders.FlacDecoder - [X] audiotools.encoders.encode_flac *** DONE Convert to_pcm()/from_pcm() routines to use FrameList objects - [X] AAC - [X] AIFF - [X] Sun AU - [X] FLAC - [X] M4A - [X] MP2 - [X] MP3 - [X] Ogg FLAC - [X] Ogg Speex - [X] WAVE - [X] WavPack *** DONE Convert CDTrackReader/OffsetCDTrackReader to use FrameList objects *** DONE Convert PCMConverter to use FrameList objects *** DONE Convert ReplayGainReader to use FrameList objects *** DONE Ensure integrated FrameList passes all unit tests *** DONE Remove deprecated audiotools.FrameList object *** DONE Remove deprecated pcmstream.PCMStreamReader object *** DONE Convert pcmstream module to resample module *** DONE Avoid importing audiotools.pcm so often Other C libraries often import audiotools.pcm via Python callbacks This library importing should be cached when possible. - [X] cdiomodule - [X] pcmreader - [X] replaygain - [X] resample *** DONE Check for memory leaks *** DONE Add FrameList and FloatFrameList programming documentation *** DONE Remove .copy() method Since FrameLists are now immutable, there's no need for it *** DONE Make pcm objects self-documenting For example, their methods and functions should give useful info when checked with "help()" ** DONE Add native ReplayGain handling routines *** DONE Add native ReplayGain handling to FlacAudio/OggFlacAudio *** DONE Ensure add_replay_gain()'s exceptions are caught Errors during calculation may raise ValueError, which must be caught anywhere the function is called *** DONE Add ReplayGain unit tests *** DONE Ensure ReplayGain works properly on 8bps and 24bps output *** DONE Ensure ReplayGain is applied consistently Although cd2track and tracksplit are guaranteed to generate only one album at a time, track2track and tracktag are not. If multiple albums are applied gain at once, add_replay_gain must be called on an album-by-album basis rather than on the entire set. *** DONE Double-check ReplayGainReader Ensure its output is consistent with other implementations. ** DONE Fix multi-channel audio handling It's important that channel mapping information be preserved when transcoding between sources with 3+ channels. This likely means another flag for PCMReader so that from_pcm() can build a file with the proper channel mask set. However, it may also be necessary to build some sort of channel reordering mechanism in the event that formats differ on how channels are to be ordered in the file. *** Channel Counts and Ordering | Format | Maximum Channels | Ordering | |------------+------------------+------------------------| | AAC | 48 | stereo-only (via faac) | | AIFF | 2^16 | predefined | | Sun AU | 2^32 | mostly undefined | | FLAC | 8 | as WAVE | | M4A | 48 | as WAVE? | | MP2 | 2 | stereo-only | | MP3 | 2 | stereo-only | | Musepack | 2 | stereo-only | | Ogg FLAC | 8 | as WAVE | | Ogg Vorbis | 255 | predefined | | Ogg Speex | 2^32 | stereo-only | | RIFF WAVE | 2^16 | predefined | | WavPack | 16 | as WAVE | |------------+------------------+------------------------| *** DONE Fix AudioFile definitions to support channel_mask() - [X] AACAudio - [X] AiffAudio - [X] AuAudio - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OggFlacAudio - [X] VorbisAudio - [X] SpeexAudio - [X] WaveAudio - [X] WavPackAudio *** DONE Fix to_pcm() methods to support channel_mask - [X] AACAudio - [X] AiffAudio - [X] AuAudio - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OggFlacAudio - [X] VorbisAudio - [X] SpeexAudio - [X] WaveAudio - [X] WavPackAudio *** DONE Fix from_pcm() classmethods to support channel_mask - [X] AACAudio - [X] AiffAudio - [X] AuAudio - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OggFlacAudio - [X] VorbisAudio - [X] SpeexAudio - [X] WaveAudio - [X] WavPackAudio *** DONE Fix alternate PCMReaders to support channel_mask - [X] BufferedPCMReader - [X] PCMConverter - [X] ReplayGainReader - [X] CDTrackReader - [X] OffsetCDTrackReader - [X] PCMCat *** DONE Handle undefined channel masks in a sane way **** DONE Fix to_pcm() methods to output undefined ChannelMasks If a format has not defined channel assignments for a given channel count, its to_pcm() method should return undefined ChannelMasks. - [X] AACAudio - [X] AiffAudio - [X] AuAudio - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OggFlacAudio - [X] VorbisAudio - [X] SpeexAudio - [X] WaveAudio - [X] WavPackAudio **** DONE Fix from_pcm() classmethods to accept undefined ChannelMasks So long as the number of channels is acceptable, audio formats are free to place undefined ChannelMasks in whatever arrangement they'd like. - [X] AACAudio - [X] AiffAudio - [X] AuAudio - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OggFlacAudio - [X] VorbisAudio - [X] SpeexAudio - [X] WaveAudio - [X] WavPackAudio *** DONE Unit test multichannel encoding and channel_mask handling **** DONE Ensure all AudioFile types have a working channel_mask() method Even 2 channel audio should yield something valid. **** DONE Ensure all to_pcm() methods yield a matching channel_mask attribute **** DONE Ensure channel_mask is preserved between from_pcm(to_pcm()) calls **** DONE Ensure channel_mask is preserved between to_wave()/from_wave() calls **** DONE Ensure channels are actually stored in the proper order This is less of an issue for .wav, .flac, .oga or .wv which already store channels in RIFF WAVE order and more of an issue for Ogg Vorbis and other formats that do not. **** DONE Ensure UnsupportedChannelMask is raised when necessary This includes calls to from_pcm() and from_wave() *** DONE Ensure PCMReader.channel_mask is always an integer ** DONE Fix the unit test error messages ** DONE Make the programming documentation web-capable It should render consistently with the regular Python reference docs and be placed both in the source tree and on the website for better accessability. *** DONE Document audiotools - [X] AudioFile - [X] BufferedPCMReader - [X] ChannelMask - [X] ExecQueue - [X] Image - [X] MetaData - [X] PCMConverter - [X] PCMReader - [X] PCMCat - [X] ReorderedPCMReader - [X] ReplayGain - [X] ReplayGainReader - [X] Messenger - [X] AlbumMetaData - [X] CDTrackLog - [X] CDDA - [X] CDTrackReader - [X] calculate_replay_gain - [X] filename_to_type - [X] find_glade_file - [X] group_tracks - [X] open - [X] open_directory - [X] open_files - [X] pcm_cmp - [X] pcm_split - [X] read_metadata_file - [X] read_sheet - [X] stripped_pcm_cmp - [X] transfer_data - [X] transfer_framelist_data - [X] BIN - [X] TYPE_MAP - [X] VERSION - [X] AVAILABLE_TYPES *** DONE Document audiotools.pcm - [X] FloatFrameList - [X] FrameList - [X] from_channels - [X] from_float_channels - [X] from_float_frames - [X] from_frames - [X] from_list *** DONE Document audiotools.resample - [X] Resampler *** DONE Document audiotools.replaygain - [X] ReplayGain *** DONE Document audiotools.cdio - [X] CDDA - [X] set_read_callback *** DONE Document audiotools.cue - [X] Cuesheet - [X] read_cuesheet - [X] CueException *** DONE Document audiotools.toc - [X] TOCFile - [X] read_tocfile - [X] TOCException ** DONE Make reference documentation render consistently *** DONE Ensure documents render in letter and A4 size *** DONE Add Creative Commons licensing to source code and doc itself *** DONE Add internal PDF linkage The file should have working bookmarks and internal links so one can click directly to a chapter *** DONE Add new introduction *** DONE Basics **** DONE Hexadecimal **** DONE Endianness **** DONE Signed values How to decode/encode signed integers should be properly explained **** DONE Character Encodings **** DONE PCM *** DONE .wav **** DONE the RIFF WAVE stream **** DONE the fmt chunk **** DONE the WAVEFORMATEXTENSIBLE fmt chunk **** DONE the data chunk **** DONE channel mapping *** DONE .aiff **** DONE the AIFF stream **** DONE the COMM chunk ***** TODO 80 bit IEEE standard 754 floating point **** DONE the SSND chunk *** DONE .au **** DONE the AU stream **** DONE the AU header *** DONE .flac **** DONE the FLAC file stream **** DONE FLAC metadata ***** DONE the PADDING metadata block ***** DONE the APPLICATION metadata block ***** DONE the SEEKTABLE metadata block ***** DONE the VORBIS_COMMENT metadata block ***** DONE the PICTURE metadata block ***** DONE the CUESHEET metadata block **** DONE FLAC decoding ***** DONE the CONSTANT subframe ***** DONE the VERBATIM subframe ***** DONE the FIXED subframe ***** DONE the LPC subframe ***** DONE the Residual ****** DONE Rice Encoding ***** DONE Channels ***** DONE Wasted bits per sample **** DONE FLAC encoding ***** DONE Metadata header ***** DONE the STREAMINFO metadata block ***** DONE Frame header ***** DONE Channel assignment ***** DONE Subframe header ***** DONE the CONSTANT subframe ***** DONE the VERBATIM subframe ***** DONE the FIXED subframe ***** DONE the LPC subframe ****** DONE Windowing ****** DONE Computing autocorrelation ****** DONE LP coefficient calculation ****** DONE Best order estimation ****** DONE Best order exhaustive search ****** DONE Quantizing coefficients ****** DONE Calculation Residual ***** DONE the Residual ****** DONE Residual Values **** DONE the Checksums ***** TODO CRC-8 ***** TODO CRC-16 *** DONE .ape **** DONE the Monkey's Audio stream **** DONE the APE Descriptor **** DONE the APE Header **** DONE the APEv2 tag **** DONE the APEv2 tag header/footer **** DONE the APEv2 flags *** DONE .wv **** DONE the WavPack file stream **** DONE a WavPack block header **** DONE a WavPack sub-block header *** DONE .mp3 **** DONE the MP3 file stream **** DONE an MPEG frame header ***** DONE the Xing header **** DONE the ID3v1 tag ***** DONE ID3v1 ***** DONE ID3v1.1 **** DONE the ID3v2 tag ***** DONE the ID3v2 stream ***** DONE ID3v2.2 ****** DONE the ID3v2.2 Header ****** DONE an ID3v2.2 Frame ****** DONE ID3v2.2 Frame IDs ****** DONE the PIC Frame ***** DONE ID3v2.3 ****** DONE the ID3v2.3 Header ****** DONE an ID3v2.3 Frame ****** DONE ID3v2.3 Frame IDs ****** DONE the APIC Frame ***** DONE ID3v2.4 ****** DONE the ID3v2.4 Header ****** DONE an ID3v2.4 Frame ****** DONE ID3v2.4 Frame IDs ****** DONE the APIC Frame *** DONE .ogg **** DONE the Ogg file stream **** DONE an Ogg Page **** DONE Ogg packets **** DONE the Identification packet **** DONE the Comment packet **** DONE Channel assignment *** DONE .spx **** DONE the Header packet **** DONE the Comment packet *** DONE .oga **** DONE the Ogg FLAC file stream **** DONE the STREAMINFO metadata packet **** DONE the Metadata packets *** DONE .m4a **** DONE the QuickTime file stream **** DONE a QuickTime atom **** DONE Container atoms **** DONE M4A atoms ***** DONE the ftyp atom ***** DONE the mvhd atom ***** DONE the tkhd atom ***** DONE the mdhd atom ***** DONE the hdlr atom ***** DONE the smhd atom ***** DONE the dref atom ***** DONE the stsd atom ***** DONE the mp4a atom ***** DONE the stts atom ***** DONE the stsc atom ***** DONE the stsz atom ***** DONE the stco atom ***** DONE the meta atom ****** DONE the trkn sub-atom ****** DONE the disk sub-atom *** DONE .mpc **** DONE Musepack SV7 ***** DONE the Musepack SV7 file stream ***** DONE the Musepack SV7 header **** DONE Musepack SV8 ***** DONE the Musepack SV8 file stream ***** DONE Nut-encoded values ***** DONE the SH packet ***** DONE the SE packet ***** DONE the RG packet ***** DONE the EI packet *** DONE FreeDB **** DONE Native Protocol ***** DONE the Disc ID ***** DONE Initial Greeting ***** DONE Client-Server Handshake ***** DONE Set Protocol Level ***** DONE Query Database ***** DONE Read XMCD Data ***** DONE Close Connection **** DONE Web Protocol **** DONE XMCD *** DONE MusicBrainz **** DONE Searching Releases ***** DONE The Disc ID ***** DONE Server Query ***** DONE Release XML **** DONE MusieBrainz XML *** DONE ReplayGain **** DONE Applying ReplayGain **** DONE Calculating ReplayGain ***** DONE the Equal Loudness Filter This should be re-documented to be closer to the actual implementation ****** TODO the Yule Filter ****** TODO the Buffer Filter ****** TODO a Filtering Example ***** DONE RMS Energy Blocks ***** DONE Statistical Processing and Calibration *** DONE References *** DONE Remove old troff reference documentation *** DONE Add title and author to PDF documentation ** DONE Ensure make(1) from doc/ directory builds both doc trees ** DONE Seperate unreadable files from unknown files Files we're unable to read should be handled differently from files we're unable to understand. *** DONE Update audiotools.open() to raise IOErrors *** DONE Update audiotools.open_files() to handle IOErrors *** DONE Update audiotools.open_directory() to handle IOErrors *** DONE Update internal calls to open()/open_files() to handle IOErrors - [X] __flac__.py - [X] __m4a__.py - [X] __mp3__.py - [X] __vorbis__.py - [X] __wavpack__.py - [X] __wav__.py *** DONE Update tools which use open()/open_files() to handle IOErrors - [X] coverdump - [X] editxmcd - [X] track2cd - [X] track2track - [X] track2xmcd - [X] trackcat - [X] trackcmp - [X] trackinfo - [X] tracklength - [X] tracklint - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag *** DONE Add unit tests to demonstrate new behavior *** DONE Update programming documentation with new behavior ** DONE Ensure embedded cuesheets aren't clobbered by adding more metadata ** DONE Adjust %(album_track_number)s to accomodate more than 9 albums For example, track 2 of 7, album 5 of 11 in format "%(album_track_number)s - %(track_name)s.%(suffix)s should be: "0502 - name.suffix" ** DONE editxmcd's "New" command should work with embedded cuesheets Selecting a single disc image with embedded cuesheets should fill in the proper XMCD/MusicBrainz fields ** DONE Ensure add_replay_gain used on hi-def tracks doesn't raise errors ** DONE Add Shorten support I don't expect people have a lot of shn files lying around and nobody should be using it for new data (though I'll add a rudimentary encoder for completeness' sake) but it's interesting to document for historical reasons. *** DONE Build complete decoder - [X] DIFF0 - [X] DIFF1 - [X] DIFF2 - [X] DIFF3 - [X] QUIT - [X] BLOCKSIZE - [X] BITSHIFT - [X] QLPC - [X] ZERO - [X] VERBATIM I'll likely limit support to Shorten version2/3 since generating older versions will be a challenge and this format is obscure enough as it is. *** DONE Build partial encoder - [X] DIFF0 - [X] DIFF1 - [X] DIFF2 - [X] QUIT - [X] BLOCKSIZE - [X] ZERO - [X] VERBATIM **** DONE Convert partial encoder for standalone use To ensure there aren't any memory leaks *** DONE Add ShortenAudio type to audiotools Python core *** DONE Add Shorten-specific unit tests *** DONE Document Shorten - [X] Shorten data types - [X] the Shorten file stream - [X] the decoding process - [X] the encoding process ** DONE trackcmp should give exact PCM frame/byte of first mismatch *** DONE update unit tests to cover new behavior - [X] test_trackcmp - [X] test_trackcmp1 - [X] test_trackcmp2 - [X] test_trackcmp3 - [X] test_trackcmp4 *** DONE update manual page to cover new behavior ** DONE Add analyzers for built-in decoders Analagous to flac(1)'s --analyze option, this will be an .analyze_frame() method that returns a Python dict of ints/lists/dicts containing frame data on each pass, or None at the stream's end. This will provide both an easier way to visualize the file's internals, and also a debugging aid. *** DONE FlacDecoder *** DONE SHNDecoder *** DONE ALACDecoder ** DONE Add more examples A lot of handy new features aren't documented with examples and walkthroughs. Examples to add include: - [X] a full multi-CD example, detailing the use of --album-number - [X] an image embedding walkthrough - [X] a CD image creation, splitting, burning example involving TOC/CUE files - [X] an XMCD walkthrough with fetching, editing and tagging ** DONE Update documentation to cover concrete MetaData classes - [X] ApeTag - [X] FlacMetaData - [X] ID3v1Comment - [X] ID3v22Comment - [X] ID3v23Comment - [X] ID3v24Comment - [X] ID3CommentPair - [X] M4AMetaData - [X] VorbisComment ** DONE Ensure man pages install correctly on Mac OS X ** DONE add a %(basename)s --format attribute For example, given the path: "/foo/bar/01 - track name.mp3" the %(basename)s attribute would be: "01 - track name" allowing one to ignore its internal metadata entirely and use original names. *** DONE Update AudioFile.track_name to support attribute *** DONE Update tools that call AudioFile.track_name - [X] cd2track - [X] track2track - [X] trackrename - [X] tracksplit *** DONE Update man pages for tools that call AudioFile.track_name - [X] track2track.1 - [X] trackrename.1 - [X] audiotools.cfg.5 *** DONE Add unit test support for new attribute *** DONE Add documentation for new field to programming reference *** DONE Add documentation for format strings to programming reference ** DONE Update AudioFile.track_name classmethod the previous behavior was a kludge cobbled together over time Its new call method is: track_name(file_path, metadata, format) so that track_number and album_number can be pulled from file_path directly instead of passed in from outside. *** DONE Update tools to support new calling method - [X] track2track - [X] cd2track - [X] trackrename - [X] tracksplit *** DONE Update programming documentation *** DONE Add unit tests *** DONE Update old unit tests with new behavior *** DONE unit test suffix field ** DONE Update Python code to support PEP 8 Following accepted Python style should make the code more accessible and maintainable in the long run. It's also a good opportunity to clean up and simplify code without changing the actual API interface. *** DONE Update core modules **** DONE __aiff__.py ***** DONE adjust syntax ***** DONE add docstrings - [X] IEEE_Extended - [X] AiffException - [X] AiffAudio - [X] AiffAudio.bits_per_sample - [X] AiffAudio.channels - [X] AiffAudio.channel_mask - [X] AiffAudio.lossless - [X] AiffAudio.total_frames - [X] AiffAudio.sample_rate - [X] AiffAudio.is_type - [X] AiffAudio.chunks - [X] AiffAudio.comm_chunk - [X] AiffAudio.chunk_files - [X] AiffAudio.get_metadata - [X] AiffAudio.set_metadata - [X] AiffAudio.delete_metadata - [X] AiffAudio.to_pcm - [X] AiffAudio.from_pcm - [X] AIFFChannelMask - [X] AIFFChannelMask.channels **** DONE __ape__.py ***** DONE adjust syntax ***** DONE add docstrings - [X] ApeTagItem - [X] ApeTagItem.build - [X] ApeTagItem.binary - [X] ApeTagItem.external - [X] ApeTagItem.string - [X] ApeTag - [X] ApeTag.converted - [X] ApeTag.merge - [X] ApeTag.supports_images - [X] ApeTag.add_image - [X] ApeTag.delete_image - [X] ApeTag.images - [X] ApeTag.read - [X] ApeTag.build - [X] ApeTaggedAudio - [X] ApeTaggedAudio.get_metadata - [X] ApeTaggedAudio.set_metadata - [X] ApeTaggedAudio.delete_metadata - [X] ApeAudio - [X] ApeAudio.is_type - [X] ApeAudio.lossless - [X] ApeAudio.supports_foreign_riff_chunks - [X] ApeAudio.has_foreign_riff_chunks - [X] ApeAudio.bits_per_sample - [X] ApeAudio.channels - [X] ApeAudio.total_frames - [X] ApeAudio.sample_rate - [X] ApeAudio.to_wave - [X] ApeAudio.from_wave **** DONE __au__.py ***** DONE adjust syntax ***** DONE add docstrings - [X] AuAudio - [X] AuAudio.is_type - [X] AuAudio.lossless - [X] AuAudio.bits_per_sample - [X] AuAudio.channels - [X] AuAudio.channel_mask - [X] AuAudio.sample_rate - [X] AuAudio.total_frames - [X] AuAudio.to_pcm - [X] AuAudio.from_pcm - [X] AuAudio.track_name **** DONE __flac__.py ***** DONE adjust syntax ***** DONE add docstrings - [ ] FlacException - [X] FlacMetaDataBlockTooLarge - [X] FlacMetaDataBlock - [X] FlacMetaDataBlock.build_block - [X] FlacMetaData - [X] FlacMetaData.converted - [X] FlacMetaData.merge - [X] FlacMetaData.add_image - [X] FlacMetaData.delete_image - [X] FlacMetaData.images - [X] FlacMetaData.metadata_blocks - [X] FlacMetaData.build - [X] FlacMetaData.supports_images - [X] FlacVorbisComment - [X] FlacVorbisComment.build_block - [X] FlacVorbisComment.converted - [X] FlacPictureComment - [X] FlacPictureComment.converted - [X] FlacPictureComment.type_string - [X] FlacPictureComment.build - [X] FlacPictureComment.build_block - [X] FlacCueSheet - [X] FlacCueSheet.build_block - [X] FlacCueSheet.converted - [X] FlacCueSheet.catalog - [X] FlacCueSheet.ISRCs - [X] FlacCueSheet.indexes - [X] FlacCueSheet.pcm_lengths - [X] FlacAudio - [X] FlacAudio.is_type - [X] FlacAudio.channel_mask - [X] FlacAudio.lossless - [X] FlacAudio.supports_foreign_riff_chunks - [X] FlacAudio.get_metadata - [X] FlacAudio.set_metadata - [X] FlacAudio.metadata_length - [X] FlacAudio.delete_metadata - [X] FlacAudio.set_cuesheet - [X] FlacAudio.get_cuesheet - [X] FlacAudio.to_pcm - [X] FlacAudio.from_pcm - [X] FlacAudio.has_foreign_riff_chunks - [X] FlacAudio.riff_wave_chunks - [X] FlacAudio.to_wave - [X] FlacAudio.from_wave - [X] FlacAudio.bits_per_sample - [X] FlacAudio.channels - [X] FlacAudio.total_frames - [X] FlacAudio.sample_rate - [X] FlacAudio.add_replay_gain - [X] FlacAudio.can_add_replay_gain - [X] FlacAudio.lossless_replay_gain - [X] FlacAudio.replay_gain - [X] FlacAudio.sub_pcm_tracks - [X] OggFlacAudio - [X] OggFlacAudio.is_type - [X] OggFlacAudio.get_metadata - [X] OggFlacAudio.set_metadata - [X] OggFlacAudio.metadata_length - [X] OggFlacAudio.to_pcm - [X] OggFlacAudio.from_pcm - [X] OggFlacAudio.sub_pcm_tracks - [X] OggFlacAudio.supports_foreign_riff_chunks **** DONE __freedb__.py ***** DONE adjust syntax ***** DONE add docstrings - [X] XMCDException - [X] XMCD - [X] XMCD.key_digits - [X] XMCD.build - [X] XMCD.read - [X] XMCD.read_data - [X] XMCD.from_files - [X] XMCD.from_cuesheet - [X] XMCD.metadata - [X] DiscID - [X] DiscID.from_cdda - [X] DiscID.add - [X] DiscID.offsets - [X] DiscID.length - [X] DiscID.idsuffix - [X] DiscID.freedb_id - [X] DiscID.toxmcd - [X] FreeDBException - [X] FreeDB - [X] FreeDB.connect - [X] FreeDB.close - [X] FreeDB.write - [X] FreeDB.read - [X] FreeDB.query - [X] FreeDB.read_data - [X] FreeDBWeb - [X] FreeDBWeb.connect - [X] FreeDBWeb.close - [X] FreeDBWeb.write - [X] FreeDBWeb.read - [X] FreeDBWeb.query - [X] FreeDBWeb.read_data - [X] get_xmcd **** DONE __id3__.py ***** DONE adjust syntax ***** DONE add docstrings - [X] UCS2Codec - [X] UCS2Codec.fix_char - [X] UCS2Codec.encode - [X] UCS2Codec.decode - [X] UnsupportedID3v2Version - [X] Syncsafe32 - [X] UBInt24 - [X] WidecharCStringAdapter - [X] UCS2CString - [X] UTF16CString - [X] UTF16BECString - [X] ID3v22Frame - [X] ID3v22Frame.build - [X] ID3v22Frame.parse - [X] ID3v22TextFrame - [X] ID3v22TextFrame.total - [X] ID3v22TextFrame.from_unicode - [X] ID3v22TextFrame.build - [X] ID3v22ComFrame - [X] ID3v22ComFrame.from_unicode - [X] ID3v22ComFrame.build - [X] ID3v22PicFrame - [X] ID3v22PicFrame.type_string - [X] ID3v22PicFrame.build - [X] ID3v22PicFrame.converted - [X] ID3v22Comment - [X] ID3v22Comment.add_image - [X] ID3v22Comment.delete_image - [X] ID3v22Comment.images - [X] ID3v22Comment.parse - [X] ID3v22Comment.converted - [X] ID3v22Comment.merge - [X] ID3v22Comment.build - [X] ID3v22Comment.skip - [X] ID3v22Comment.read_id3v2_comment - [X] ID3v23Frame - [X] ID3v23Frame.build - [X] ID3v23Frame.parse - [X] ID3v23TextFrame - [X] ID3v23TextFrame.total - [X] ID3v23TextFrame.from_unicode - [X] ID3v23TextFrame.build - [X] ID3v23PicFrame - [X] ID3v23PicFrame.build - [X] ID3v23PicFrame.converted - [X] ID3v23ComFrame - [X] ID3v23ComFrame.from_unicode - [X] ID3v23ComFrame.build - [X] ID3v23Comment - [X] ID3v23Comment.add_image - [X] ID3v23Comment.delete_image - [X] ID3v23Comment.images - [X] ID3v23Comment.build - [X] ID3v24Frame - [X] ID3v24Frame.build - [X] ID3v24Frame.parse - [X] ID3v24TextFrame - [X] ID3v24TextFrame.total - [X] ID3v24TextFrame.from_unicode - [X] ID3v24TextFrame.build - [X] ID3v24PicFrame - [X] ID3v24PicFrame.build - [X] ID3v24PicFrame.converted - [X] ID3v24ComFrame - [X] ID3v24ComFrame.from_unicode - [X] ID3v24ComFrame.build - [X] ID3v24Comment - [X] ID3v24Comment.build - [X] ID3CommentPair - [X] ID3CommentPair.converted - [X] ID3CommentPair.merge - [X] ID3CommentPair.images - [X] ID3CommentPair.add_image - [X] ID3CommentPair.delete_image - [X] ID3CommentPair.supports_images **** DONE __id3v1__.py ***** DONE adjust syntax ***** DONE add docstrings - [X] ID3v1Comment - [X] ID3v1Comment.read_id3v1_comment - [X] ID3v1Comment.build_id3v1 - [X] ID3v1Comment.supports_images - [X] ID3v1Comment.converted - [X] ID3v1Comment.build_tag - [X] ID3v1Comment.images **** DONE __image__.py ***** DONE adjust syntax ***** DONE add docstrings - [X] image_metrics - [X] ImageMetrics - [X] InvalidImage - [X] InvalidJPEG - [X] InvalidPNG - [X] InvalidBMP - [X] InvalidGIF - [X] InvalidTIFF - [X] can_thumbnail - [X] thumbnail_formats - [X] thumbnail_image **** DONE __init__.py ***** DONE adjust syntax ***** DONE add docstrings - [X] __init__.py - [X] RawConfigParser - [X] RawConfigParser.get_default - [X] RawConfigParser.getint_default - [X] find_glade_file - [X] OptionParser - [X] Messenger - [X] str_width - [X] VerboseMessenger - [X] VerboseMessenger.output - [X] VerboseMessenger.partial_output - [X] VerboseMessenger.new_row - [X] VerboseMessenger.blank_row - [X] VerboseMessenger.divider_row - [X] VerboseMessenger.output_column - [X] VerboseMessenger.output_rows - [X] VerboseMessenger.info - [X] VerboseMessenger.partial_info - [X] VerboseMessenger.error - [X] VerboseMessenger.warning - [X] VerboseMessenger.usage - [X] VerboseMessenger.filename - [X] VerboseMessenger.ansi - [X] VerboseMessenger.ansi_err - [X] SilentMessenger - [X] SilentMessenger.output - [X] SilentMessenger.partial_output - [X] SilentMessenger.warning - [X] SilentMessenger.info - [X] SilentMessenger.partial_info - [X] UnsupportedFile - [X] InvalidFile - [X] InvalidFormat - [X] EncodingError - [X] UnsupportedChannelMask - [X] UnsupportedChannelCount - [X] DecodingError - [X] open - [X] open_files - [X] open_directory - [X] UnknownAudioType - [X] AmbiguousAudioType - [X] filename_to_type - [X] ChannelMask - [X] ChannelMask.defined - [X] ChannelMask.undefined - [X] ChannelMask.channels - [X] ChannelMask.index - [X] ChannelMask.from_fields - [X] ChannelMask.from_channels - [X] PCMReader - [X] PCMReader.read - [X] PCMReader.close - [X] PCMReaderError - [X] PCMReaderError.read - [X] PCMReaderError.close - [X] ReorderedPCMReader - [X] ReorderedPCMReader.read - [X] ReorderedPCMReader.close - [X] transfer_data - [X] transfer_framelist_data - [X] threaded_transfer_framelist_data - [X] pcm_cmp - [X] stripped_pcm_cmp - [X] pcm_frame_cmp - [X] PCMCat - [X] PCMCat.read - [X] PCMCat.close - [X] BufferedPCMReader - [X] BufferedPCMReader.close - [X] BufferedPCMReader.read - [X] pcm_split - [X] PCMConverter - [X] PCMConverter.read - [X] PCMConverter.close - [X] applicable_replay_gain - [X] calculate_replay_gain - [X] InterruptableReader - [X] InterruptableReader.stop - [X] InterruptableReader.send_data - [X] InterruptableReader.read - [X] ignore_sigint - [X] make_dirs - [X] MetaData - [X] MetaData.converted - [X] MetaData.supports_images - [X] MetaData.images - [X] MetaData.front_covers - [X] MetaData.back_covers - [X] MetaData.leaflet_pages - [X] MetaData.media_images - [X] MetaData.other_images - [X] MetaData.add_image - [X] MetaData.delete_image - [X] MetaData.merge - [X] AlbumMetaData - [X] AlbumMetaData.metadata - [X] MetaDataFileException - [X] Image - [X] Image.suffix - [X] Image.type_string - [X] Image.new - [X] Image.thumbnail - [X] ReplayGain - [X] UnsupportedTracknameField - [X] AudioFile - [X] AudioFile.is_type - [X] AudioFile.bits_per_sample - [X] AudioFile.channels - [X] AudioFile.channel_mask - [X] AudioFile.lossless - [X] AudioFile.set_metadata - [X] AudioFile.get_metadata - [X] AudioFile.delete_metadata - [X] AudioFile.total_frames - [X] AudioFile.cd_frames - [X] AudioFile.sample_rate - [X] AudioFile.to_pcm - [X] AudioFile.from_pcm - [X] AudioFile.to_wave - [X] AudioFile.from_wave - [X] AudioFile.supports_foreign_riff_chunks - [X] AudioFile.has_foreign_riff_chunks - [X] AudioFile.track_number - [X] AudioFile.album_number - [X] AudioFile.track_name - [X] AudioFile.add_replay_gain - [X] AudioFile.can_add_replay_gain - [X] AudioFile.lossless_replay_gain - [X] AudioFile.replay_gain - [X] AudioFile.set_cuesheet - [X] AudioFile.get_cuesheet - [X] AudioFile.has_binaries - [X] DummyAudioFile - [X] DummyAudioFile.get_metadata - [X] DummyAudioFile.cd_frames - [X] DummyAudioFile.track_number - [X] DummyAudioFile.sample_rate - [X] DummyAudioFile.total_frames - [X] SheetException - [X] read_sheet - [X] parse_timestamp - [X] build_timestamp - [X] sheet_to_unicode - [X] at_a_time - [X] CDDA - [X] RawCDDA - [X] RawCDDA.length - [X] RawCDDA.close - [X] RawCDDA.first_sector - [X] RawCDDA.last_sector - [X] OffsetCDDA - [X] OffsetCDDA.close - [X] CDTrackLog - [X] CDTrackReader - [X] CDTrackReader.offset - [X] CDTrackReader.length - [X] CDTrackReader.log - [X] CDTrackReader.read - [X] CDTrackReader.close - [X] OffsetCDTrackReader - [X] OffsetCDTrackReader.offset - [X] OffsetCDTrackReader.length - [X] OffsetCDTrackReader.log - [X] OffsetCDTrackReader.read - [X] OffsetCDTrackReader.close - [X] read_metadata_file - [X] ExecQueue - [X] ExecQueue.execute - [X] ExecQueue.run - [X] BitstreamReader - [X] BitstreamReader.byte_align - [X] BitstreamReader.read - [X] BitstreamReader.unread - [X] BitstreamReader.read_signed - [X] BitstreamReader.unary - [X] BitstreamReader.tell - [X] BitstreamReader.close - [X] BitstreamWriter - [X] BitstreamWriter.byte_align - [X] BitstreamWriter.write - [X] BitstreamWriter.write_signed - [X] BitstreamWriter.unary - [X] BitstreamWriter.tell - [X] BitstreamWriter.close **** DONE __m4a_atoms__.py ***** DONE adjust syntax ***** DONE add docstrings - [X] VersionLength - [X] AtomAdapter - [X] Atom - [X] AtomListAdapter - [X] AtomContainer - [X] AtomWrapper **** DONE __m4a__.py ***** DONE adjust syntax ***** DONE add docstrings - [X] M4AAudio_faac - [X] M4AAudio_faac.channel_mask - [X] M4AAudio_faac.is_type - [X] M4AAudio_faac.lossless - [X] M4AAudio_faac.channels - [X] M4AAudio_faac.bits_per_sample - [X] M4AAudio_faac.sample_rate - [X] M4AAudio_faac.cd_frames - [X] M4AAudio_faac.total_frames - [X] M4AAudio_faac.get_metadata - [X] M4AAudio_faac.set_metadata - [X] M4AAudio_faac.delete_metadata - [X] M4AAudio_faac.to_pcm - [X] M4AAudio_faac.from_pcm - [X] M4AAudio_faac.can_add_replay_gain - [X] M4AAudio_faac.lossless_replay_gain - [X] M4AAudio_faac.add_replay_gain - [X] M4AAudio_nero - [X] M4AAudio_nero.to_pcm - [X] M4AAudio_nero.from_pcm - [X] M4AAudio_nero.to_wave - [X] M4AAudio_nero.from_wave - [X] ILST_Atom - [X] M4AMetaData - [X] M4AMetaData.binary_atom - [X] M4AMetaData.text_atom - [X] M4AMetaData.trkn_atom - [X] M4AMetaData.disk_atom - [X] M4AMetaData.covr_atom - [X] M4AMetaData.images - [X] M4AMetaData.add_image - [X] M4AMetaData.delete_image - [X] M4AMetaData.converted - [X] M4AMetaData.merge - [X] M4AMetaData.to_atom - [X] M4AMetaData.supports_images - [X] M4ACovr - [X] M4ACovr.converted - [X] ALACAudio - [X] ALACAudio.is_type - [X] ALACAudio.total_frames - [X] ALACAudio.channel_mask - [X] ALACAudio.cd_frames - [X] ALACAudio.lossless - [X] ALACAudio.to_pcm - [X] ALACAudio.from_pcm - [X] ALACAudio.to_wave - [X] ALACAudio.from_wave - [X] ADTSException - [X] AACAudio - [X] AACAudio.is_type - [X] AACAudio.bits_per_sample - [X] AACAudio.channels - [X] AACAudio.lossless - [X] AACAudio.total_frames - [X] AACAudio.sample_rate - [X] AACAudio.to_pcm - [X] AACAudio.from_pcm - [X] AACAudio.aac_frames - [X] AACAudio.aac_frame_count **** DONE __mp3__.py ***** DONE adjust syntax ***** DONE add docstrings - [X] MP3Exception - [X] MP3Audio - [X] MP3Audio.is_type - [X] MP3Audio.lossless - [X] MP3Audio.to_pcm - [X] MP3Audio.from_pcm - [X] MP3Audio.bits_per_sample - [X] MP3Audio.channels - [X] MP3Audio.sample_rate - [X] MP3Audio.get_metadata - [X] MP3Audio.set_metadata - [X] MP3Audio.delete_metadata - [X] MP3Audio.cd_frames - [X] MP3Audio.total_frames - [X] MP3Audio.can_add_replay_gain - [X] MP3Audio.lossless_replay_gain - [X] MP3Audio.add_replay_gain - [X] MP2Audio - [X] MP2Audio.is_type - [X] MP2Audio.from_pcm **** DONE __musepack__.py ***** DONE adjust syntax ***** DONE add docstrings - [X] NutValue - [X] Musepack8StreamReader - [X] Musepack8StreamReader.packets - [X] MusepackAudio - [X] MusepackAudio.from_pcm - [X] MusepackAudio.is_type - [X] MusepackAudio.sample_rate - [X] MusepackAudio.total_frames - [X] MusepackAudio.channels - [X] MusepackAudio.bits_per_sample - [X] MusepackAudio.lossless **** DONE __musicbrainz__.py ***** DONE adjust syntax ***** DONE add docstrings - [X] get_xml_nodes - [X] get_xml_text_node - [X] reorder_xml_children - [X] MBDiscID - [X] MBDiscID.from_cdda - [X] MBDiscID.offsets - [X] MusicBrainz - [X] MusicBrainz.connect - [X] MusicBrainz.close - [X] MusicBrainz.read_data - [X] MBXMLException - [X] MusicBrainzReleaseXML - [X] MusicBrainzReleaseXML.read - [X] MusicBrainzReleaseXML.read_data - [X] MusicBrainzReleaseXML.metadata - [X] MusicBrainzReleaseXML.from_files - [X] MusicBrainzReleaseXML.from_cuesheet - [X] MusicBrainzReleaseXML.build - [X] get_mbxml **** DONE __shn__.py ***** DONE adjust syntax ***** DONE add docstrings - [X] ShortenAudio - [X] ShortenAudio.is_type - [X] ShortenAudio.bits_per_sample - [X] ShortenAudio.channels - [X] ShortenAudio.channel_mask - [X] ShortenAudio.lossless - [X] ShortenAudio.total_frames - [X] ShortenAudio.sample_rate - [X] ShortenAudio.to_pcm - [X] ShortenAudio.from_pcm - [X] ShortenAudio.to_wave - [X] ShortenAudio.from_wave - [X] ShortenAudio.supports_foreign_riff_chunks - [X] ShortenAudio.has_foreign_riff_chunks **** DONE __speex__.py ***** DONE adjust syntax ***** DONE add docstrings - [X] UnframedVorbisComment - [X] SpeexAudio - [X] SpeexAudio.is_type - [X] SpeexAudio.to_pcm - [X] SpeexAudio.from_pcm - [X] SpeexAudio.set_metadata - [X] SpeexAudio.can_add_replay_gain **** DONE __vorbiscomment__.py ***** DONE adjust syntax ***** DONE add docstrings - [X] VorbisComment - [X] VorbisComment.supports_images - [X] VorbisComment.images - [X] VorbisComment.converted - [X] VorbisComment.merge - [X] VorbisComment.build **** DONE __vorbis__.py ***** DONE adjust syntax ***** DONE add docstrings - [X] OggStreamReader - [X] OggStreamReader.close - [X] OggStreamReader.packets - [X] OggStreamReader.pages - [X] OggStreamReader.pages_to_packets - [X] OggStreamReader.calculate_ogg_checksum - [X] OggStreamWriter - [X] OggStreamWriter.close - [X] OggStreamWriter.write_page - [X] OggStreamWriter.build_pages - [X] VorbisAudio - [X] VorbisAudio.is_type - [X] VorbisAudio.lossless - [X] VorbisAudio.bits_per_sample - [X] VorbisAudio.channels - [X] VorbisAudio.channel_mask - [X] VorbisAudio.total_frames - [X] VorbisAudio.sample_rate - [X] VorbisAudio.to_pcm - [X] VorbisAudio.from_pcm - [X] VorbisAudio.set_metadata - [X] VorbisAudio.get_metadata - [X] VorbisAudio.delete_metadata - [X] VorbisAudio.add_replay_gain - [X] VorbisAudio.can_add_replay_gain - [X] VorbisAudio.lossless_replay_gain - [X] VorbisAudio.replay_gain - [X] VorbisChannelMask - [X] VorbisChannelMask.channels **** DONE __wavpack__.py ***** DONE adjust syntax ***** DONE add docstrings - [X] SymlinkPCMReader - [X] SymlinkPCMReader.read - [X] SymlinkPCMReader.close - [X] SymlinkPCMReader.new - [X] WavePackAPEv2 - [X] WavePackAPEv2.converted - [X] WavPackAudio - [X] WavPackAudio.is_type - [X] WavPackAudio.lossless - [X] WavPackAudio.supports_foreign_riff_chunks - [X] WavPackAudio.channel_mask - [X] WavPackAudio.get_metadata - [X] WavPackAudio.has_foreign_riff_chunks - [X] WavPackAudio.frames - [X] WavPackAudio.sub_frames - [X] WavPackAudio.bits_per_sample - [X] WavPackAudio.channels - [X] WavPackAudio.total_frames - [X] WavPackAudio.sample_rate - [X] WavPackAudio.from_pcm - [X] WavPackAudio.to_wave - [X] WavPackAudio.to_pcm - [X] WavPackAudio.from_wave - [X] WavPackAudio.add_replay_gain - [X] WavPackAudio.can_add_replay_gain - [X] WavPackAudio.replay_gain - [X] WavPackAudio.get_cuesheet - [X] WavPackAudio.set_cuesheet **** DONE __wav__.py ***** DONE adjust syntax ***** DONE add docstrings - [X] WaveReader - [X] WaveReader.read - [X] WaveReader.close - [X] TempWaveReader - [X] TempWaveReader.close - [X] WavException - [X] WaveAudio - [X] WaveAudio.is_type - [X] WaveAudio.lossless - [X] WaveAudio.supports_foreign_riff_chunks - [X] WaveAudio.has_foreign_riff_chunks - [X] WaveAudio.channel_mask - [X] WaveAudio.to_pcm - [X] WaveAudio.from_pcm - [X] WaveAudio.to_wave - [X] WaveAudio.from_wave - [X] WaveAudio.total_frames - [X] WaveAudio.sample_rate - [X] WaveAudio.channels - [X] WaveAudio.bits_per_sample - [X] WaveAudio.can_add_replay_gain - [X] WaveAudio.lossless_replay_gain - [X] WaveAudio.add_replay_gain - [X] WaveAudio.track_name - [X] WaveAudio.fmt_chunk_to_channel_mask - [X] WaveAudio.chunk_ids - [X] WaveAudio.chunks - [X] WaveAudio.wave_from_chunks - [X] WaveAudio.pcm_split **** DONE cue.py ***** DONE adjust syntax ***** DONE add docstrings - [X] cue.py - [X] CueException - [X] tokens - [X] get_value - [X] parse - [X] Cuesheet - [X] Cuesheet.catalog - [X] Cuesheet.single_file_type - [X] Cuesheet.indexes - [X] Cuesheet.pcm_lengths - [X] Cuesheet.ISRCs - [X] Cuesheet.file - [X] Track - [X] Track.ISRC - [X] read_cuesheet **** DONE toc.py ***** DONE adjust syntax ***** DONE add docstrings - [X] toc.py - [X] TOCException - [X] parse - [X] TOCFile - [X] TOCFile.catalog - [X] TOCFile.indexes - [X] TOCFile.pcm_lengths - [X] TOCFile.ISRCs - [X] TOCFile.file - [X] Track - [X] Track.ISRC - [X] read_tocfile *** DONE Update utilities - [X] audiotools-config - [X] cd2track - [X] cd2xmcd - [X] cdinfo - [X] coverdump - [X] coverview - [X] editxmcd - [X] record2track - [X] track2cd - [X] track2track - [X] track2xmcd - [X] trackcat - [X] trackcmp - [X] trackinfo - [X] tracklength - [X] tracklint - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag *** DONE Update tests - [X] test.py - [X] test_streams.py ** DONE Update C code to support PEP 7 - [X] array.c - [X] array.h - [X] bitstream_r.c - [X] bitstream_r.h - [X] bitstream_w.c - [X] bitstream_w.h - [X] cdiomodule.c - [X] decoders.c - [X] decoders.h - [X] encoders.c - [X] encoders.h - [X] md5.c - [X] md5.h - [X] pcm.c - [X] pcm.h - [X] pcmreader.c - [X] pcmreader.h - [X] replaygain.c - [X] replaygain.h - [X] resample.c - [X] resample.h - [X] decoders/alaac.c - [X] decoders/alac.h - [X] decoders/flac.c - [X] decoders/flac_crc.c - [X] decoders/flac_crc.h - [X] decoders/flac.h - [X] decoders/shn.c - [X] decoders/shn.h - [X] encoders/alac.c - [X] encoders/alac.h - [X] encoders/alac_lpc.c - [X] encoders/alac_lpc.h - [X] encoders/flac.c - [X] encoders/flac.h - [X] encoders/flac_lpc.c - [X] encoders/flac_lpc.h - [X] encoders/shn.c - [X] encoders/shn.h ** DONE Add ALAC support *** DONE Build ALAC decoder **** DONE Add audiotools.decoders.AlacDecoder object **** DONE Add frame header parsing **** DONE Add subframe header parsing **** DONE Add wasted-bits block parsing **** DONE Add residual decoding **** DONE Ensure frame footer is checked **** DONE Decode subframes **** DONE Perform channel decorrelation - [X] leftweight zero - [X] leftweight nonzero **** DONE Handle wasted-bits samples **** DONE Return pcm.FrameList objects on calls to read() **** DONE Ensure PCMReader-compatible fields are present - [X] sample_rate - [X] channels - [X] channel_mask - [X] bits_per_sample **** DONE Ensure 16/24 bps streams work correctly - [X] 16bps - [X] 24bps **** DONE Ensure 1/2 channel streams work correctly - [X] 1 channel - [X] 2 channels **** DONE Ensure different sample rates work correctly - [X] 44100Hz - [X] 48000Hz - [X] 96000Hz - [X] 192000Hz (need to figure out what other sample rates ALAC supports) **** DONE Update source to more closely match documentation **** DONE Optimize for speed *** DONE Build ALAC encoder **** DONE Add audiotools.encoders.encode_alac function **** DONE Ensure verbatim ALAC file is written correctly **** DONE Determine ALAC stream tunables ***** wasted bits for 24bps streams ***** interlacing shift ***** interlacing leftweight ***** prediction quantitization ***** coefficients **** DONE extract wasted bits for 24bps streams **** DONE correlate stereo samples **** DONE determine coefficients/quantitization **** DONE calculate residual values **** DONE write residuals based on initial history/history multiplier/etc. **** DONE handle random noise with uncompressed frames **** DONE handle atypical sample rates properly **** DONE Update source to more closely match documentation **** DONE make interlacing shift and interlacing leftweight range options **** DONE optimize for speed *** DONE Add ALACAudio type to audiotools Python core *** DONE Enable TestAlacAudio core tests *** DONE Add ALAC-specific unit tests **** DONE test_streams **** DONE test_small_files **** DONE test_full_scale_deflection **** DONE test_sines eliminate DeprecationWarning at construct/core.py:541 which seems to be caused by 96000Hz input **** DONE test_wasted_bps **** DONE test_blocksizes **** DONE test_frame_header_variations **** DONE test_noise **** DONE test_fractional *** DONE Document ALAC **** DONE the ALAC file stream **** DONE ALAC decoding ***** DONE Frame header ***** DONE Subframe header ***** DONE Residual decoding ***** DONE Residual decoding example ***** DONE Subframe decoding Simplify this to make it easier to understand ***** DONE Subframe decoding example ***** DONE Channel decorrelation ***** DONE Channel decorrelation example ***** DONE Wasted bits ***** DONE Wasted bits example **** DONE ALAC encoding ***** DONE Figure out compliant atom contents - [X] ftyp - [X] moov->mvhd - [X] moov->trak->tkhd - [X] moov->trak->mdia->mdhd - [X] moov->trak->mdia->hdlr - [X] moov->trak->mdia->minf->smhd - [X] moov->trak->mdia->dinf->dref - [X] moov->trak->mdia->stbl->stsd->alac - [X] moov->trak->mdia->stbl->stts - [X] moov->trak->mdia->stbl->stsc - [X] moov->trak->mdia->stbl->stsz - [X] moov->trak->mdia->stbl->stco - [X] moov->udta->meta - [X] free ***** DONE Add forward references in "alac" atom description ***** DONE figure out if "meta" has required "----" sub-atoms ***** DONE extracting wasted bits for 24bps streams ***** DONE correlating stereo samples ***** DONE determining coefficients/quantitization ****** DONE Windowing ****** DONE Computing autocorrelation ****** DONE LP coefficient calculation ****** DONE Best order estimation ****** DONE Quantizing coefficients ***** DONE calculating residual values Simplify this to make it easier to understand ***** DONE writing residuals based on initial history/history multiplier/etc. ***** DONE Fix bitstream figs to be monospace font for binary digits **** DONE verify A4 layout is correct *** DONE Update M4A metadata routines to exploit "free" atoms As with FLAC, rewriting the entire file should be avoided *** DONE Fix InvalidImage exceptions when reading test ALACs *** DONE Ensure UnsupportedChannels exception is handled by user-level tools *** DONE Ensure new ALACs work in iTunes *** DONE Ensure new ALACs work on iPods *** DONE Have M4A files group properly on iTunes/iPods ** DONE Remove xdelta requirement It's doesn't compile well on some platforms and is only used by tracklint *** DONE Build trivial binary delta routine Since most metadata formats make use of padding, we can use a simple XOR over their contents to generate a bidirectional patch that's optimized for tracklint's behavior. Since the bulk of such a patch should be NULLs, we can compress it with zlib/bz2 and achieve excellent compression. ** DONE Tweak documentation *** DONE Update indexes to account for warm-up samples The i index on the left and right hand sides must match. - [X] FLAC FIXED decoding - [X] FLAC LPC decoding - [X] FLAC FIXED encoding - [X] FLAC LPC encoding *** DONE Rewrite the FLAC channel assignment section Reference the side channel extra bit and add examples. *** DONE Add FLAC channel assignment encoding documentation Show its channel calculations and include an example. ** DONE indicate ReplayGain capabilities/binaries audiotools-config(1) should show what ReplayGain binaries are present and all audio classes that support it ** DONE update trackcmp to share trackverify's output interface ** DONE Add unary jump tables with a max value Since both Apple Lossless *and* WavPack have unary reading with a maximum upper limit of read bits, it makes sense to build a proper jump table for it. The size of our state limits the maximum to 8 bits, so larger maximums will be supported at a read_limited_unary() level. The trouble is, we need to differentiate between normal exits (we've hit the stop bit) and exits that hit the maximum value. So, the jump table itself must have a different syntax (probably an extra bit for "maximum value reached") and read_limited_unary() will likely return -1 in that event. ** DONE Add little-endian bitstream readers *** DONE integrate read functions into Bitstream struct *** DONE add bitstream.py jump tables for little-endian reading - [X] read_bits_table_le.h - [X] read_unary_table_le.h - [X] read_limited_unary_table_le.h - [X] unread_bit_table_le.h *** DONE add little-endian Bitstream functions - [X] bs_read_bits_le - [X] bs_read_bits64_le - [X] bs_unread_bit_le - [X] bs_read_unary_le - [X] bs_read_limited_unary_le *** DONE have bs_open() attach the proper endian functions *** DONE update Python BitstreamReader for little-endian operation This should function similar to the C one, with alignment specified at init-time. Or, perhaps replace Python reader with C-based one altogether. *** DONE add some basic Bitstream reader unit testing *** DONE allow endianness swapping ** DONE Add little-endian bitstream writers *** DONE add bitstream.py jump tables for little-endian writing - [X] write_bits_table_le.h - [X] write_unary_table_le.h *** DONE Add little-endian Bitstream functions - [X] write_bits_actual_le - [X] write_signed_bits_actual_le - [X] write_bits64_actual_le - [X] write_unary_actual_le - [X] byte_align_w_actual_le *** DONE have bs_open() attach the proper endian functions *** DONE update Python BitstreamWriter for little-endian operation Convert to a C type, similar to BitstreamReader *** DONE add some basic Bitstream writer unit testing *** DONE allow endianness swapping ** DONE integrate new endianness routines into existing routines *** DONE Convert Ogg verifier to proper little-endian operation *** DONE Swap endianness for proper FLAC VORBISCOMMENT writing *** DONE have bs_set_endianness create instruction in recorder-mode So that if we're recording an endianness shift, it gets set properly when "played back" to an actual writer. In fact, it may be a good idea to attach the set_endianness() function to the bitstream writer itself. ** DONE Build proper AlbumMetaDataFile class This is a superclass of FreeDB's XMCD and MusicBrainz's XML which wraps metadata containers into a consistent interface for use by editxmcd and command-line utilities. *** DONE Convert XMCD to AlbumMetaDataFile subclass *** DONE Add unit tests for XMCD *** DONE Convert MusicBrainzReleaseXML to AlbumMetaDataFile subclass *** DONE Add unit tests for MusicBrainzReleaseXML *** DONE Update utilities to use new interface - [X] tracksplit - [X] track2track - [X] track2xmcd - [X] trackrename - [X] tracktag - [ ] editxmcd - [X] cd2track - [X] cd2xmcd *** DONE Update old unit tests to new interface *** DONE Fix get_track to return blank artist name if not present Don't pull from the class artist name; have track_metadata() figure that out as appropriate instead. **** DONE Fix unit tests for proper behavior *** DONE Document AlbumMetaDataFile ** DONE Update ALAC to handle multichannel audio What David Hammerton's reverse-engineered decoder described as a 3-bit frame footer isn't; it's actually a "stop" delimiter analagous to WavPack's block header stop bit. If it's not 0x7, keep reading frames and combine them channel-wise into a single multichannel chunk of output. iTunes and iPods still won't be able to handle such files, but XLD should be able to. *** DONE Update ALACDecoder's analyze_frame() method for multichannel *** DONE Update ALACDecoder's read() method for multichannel *** DONE Update encode_alac function for multichannel *** DONE Unit test multichannel ALAC encoding and decoding *** DONE Update decoding documentation describing multichannel handling *** DONE Update encoding documentation describing multichannel handling ** DONE Add higher sampling rate support to ReplayGain module Extract the higher rates from wvgain.c - [X] 8000Hz - [X] 11025Hz - [X] 12000Hz - [X] 16000Hz - [X] 18900Hz - [X] 22050Hz - [X] 24000Hz - [X] 32000Hz - [X] 37800Hz - [X] 44100Hz - [X] 48000Hz - [X] 56000Hz - [X] 64000Hz - [X] 88200Hz - [X] 96000Hz - [X] 112000Hz - [X] 128000Hz - [X] 144000Hz - [X] 176400Hz - [X] 192000Hz *** DONE Unit test sample rates ** DONE Allow audio type defaults to be selectable *** DONE Update tools to pull -t from config file - [X] cd2track - [X] record2track - [X] track2track - [X] trackcat - [X] tracksplit *** DONE Update tools to pull -q from config file - [X] cd2track - [X] record2track - [X] track2track - [X] trackcat - [X] tracksplit *** DONE Update audio formats to pull default quality from config file (if any) for all calls to from_pcm() and from_wave() - [X] aac - [X] flac - [X] m4a (Nero) - [X] m4a (faac) - [X] mp2 - [X] mp3 - [X] oga - [X] ogg - [X] spx - [X] wv *** DONE Update audiotools-config to display default type *** DONE Update audiotools-config to display default qualities *** DONE Update audiotools-config to select default type *** DONE Update audiotools-config to select default quality for a type *** DONE Document new configuration options in man pages - [X] audiotools-config - [X] cd2track - [X] record2track - [X] track2track - [X] trackcat - [X] tracksplit - [X] audiotools.cfg ** DONE Allow default verbosity to be selectable *** DONE Update audiotools-config to display default verbosity *** DONE Update audiotools-config to select default verbosity *** DONE Update tools to use default verbosity - [X] cd2track - [X] cd2xmcd - [X] coverdump - [X] record2track - [X] track2cd - [X] track2track - [X] track2xmcd - [X] trackcmp - [X] tracklint - [X] trackrename - [X] tracksplit - [X] tracktag - [X] trackverify *** DONE Document new configuration option in man pages - [X] audiotools-config - [X] audiotools.cfg ** DONE Shift common decoder/encoder routines to a common/ directory - [X] flac_crc.h - [X] flac_crc.c - [X] misc.h - [X] misc.c - [X] md5.h - [X] md5.c ** DONE Get coverview and editxmcd working on Mac OS X *** DONE editxmcd **** DONE Update man page *** DONE Update coverview for dual PyGTK/Tkinter operation It's a simple enough app that it should be able to conditionally do both, especially since audiotools does most of the heavy lifting. This allows it to look like a proper app under X11 and work at all everywhere else. It should look and function approximately the same on both. **** DONE Remove glade requirement I'm sick of glade, and coverview should be small enough that it's not a problem to lay it out internally. **** DONE Update coverview for PyGTK **** DONE Update coverview for Tkinter ***** DONE Fixup error messages, if possible **** DONE Add --gtk/--tkinter switches for conditional launch For testing purposes **** DONE Cleanup conditional classes/helper functions **** DONE Check for init-time errors Both in loading audiofiles and in import problems such as Mac OS's 32-bit problem **** DONE Update man page ** DONE More graceful handling of broken files A lot of the track handlers assume that once the start of the file is good, the rest of it is following the spec. This is not always the case. *** DONE All audio formats need to implement the error specification This means that: classmethod.is_type() must never error __init__() must raise InvalidFile if the filename's contents are invalid to_pcm() must return PCMReaderError if the decoder can't be built classmethod.from_pcm() must raise EncodingError if it can't encode file to_wave() must raise EncodingError if a wave can't be written classmethod.from_wave() must raise EncodingError if it can't encode file a failed to_wave() mustn't leave half-encoded .wav files behind a failed from_wave() mustn't leave half-encoded .wav files behind a failed from_pcm() mustn't leave a partially encoded file behind PCMReaders may raise IOError and ValueError on read() PCMReaders may raise DecodingError on close() **** DONE AACAudio - [X] is_type() doesn't error - [X] __init__() raises InvalidFile - [X] to_pcm() returns PCMReaderError on decoder error - [X] from_pcm() raises EncodingError on encoder error - [X] from_wave() raises EncodingError on encoder error - [X] failed from_wave() deletes partial file - [X] failed from_pcm() deletes partial file - [X] PCMReader raises IOError/ValueError on read() if necessary - [X] PCMReader raises DecodingError on close() if necessary **** DONE AiffAudio - [X] is_type() doesn't error - [X] __init__() raises InvalidFile - [X] to_pcm() returns PCMReaderError on decoder error - [X] from_pcm() raises EncodingError on encoder error - [X] to_wave() raises EncodingError on decoder error - [X] from_wave() raises EncodingError on encoder error - [X] failed to_wave() deletes partial file - [X] failed from_wave() deletes partial file - [X] failed from_pcm() deletes partial file - [X] PCMReader raises IOError/ValueError on read() if necessary - [X] PCMReader raises DecodingError on close() if necessary **** DONE ALACAudio - [X] is_type() doesn't error - [X] __init__() raises InvalidFile - [X] to_pcm() returns PCMReaderError on decoder error - [X] from_pcm() raises EncodingError on encoder error - [X] to_wave() raises EncodingError on decoder error - [X] from_wave() raises EncodingError on encoder error - [X] failed to_wave() deletes partial file - [X] failed from_wave() deletes partial file - [X] failed from_pcm() deletes partial file - [X] PCMReader raises IOError/ValueError on read() if necessary - [X] PCMReader raises DecodingError on close() if necessary **** DONE AuAudio - [X] is_type() doesn't error - [X] __init__() raises InvalidFile - [X] to_pcm() returns PCMReaderError on decoder error - [X] from_pcm() raises EncodingError on encoder error - [X] to_wave() raises EncodingError on decoder error - [X] from_wave() raises EncodingError on encoder error - [X] failed to_wave() deletes partial file - [X] failed from_wave() deletes partial file - [X] failed from_pcm() deletes partial file - [X] PCMReader raises IOError/ValueError on read() if necessary - [X] PCMReader raises DecodingError on close() if necessary **** DONE FlacAudio - [X] is_type() doesn't error - [X] __init__() raises InvalidFile - [X] to_pcm() returns PCMReaderError on decoder error - [X] from_pcm() raises EncodingError on encoder error - [X] to_wave() raises EncodingError on decoder error - [X] from_wave() raises EncodingError on encoder error - [X] failed to_wave() deletes partial file - [X] failed from_wave() deletes partial file - [X] failed from_pcm() deletes partial file - [X] PCMReader raises IOError/ValueError on read() if necessary - [X] PCMReader raises DecodingError on close() if necessary **** DONE M4AAudio_faac - [X] is_type() doesn't error - [X] __init__() raises InvalidFile - [X] from_pcm() raises EncodingError on encoder error - [X] from_wave() raises EncodingError on encoder error - [X] failed from_wave() deletes partial file - [X] failed from_pcm() deletes partial file **** DONE M4AAudio_nero - [X] is_type() doesn't error - [X] __init__() raises InvalidFile - [X] to_pcm() returns PCMReaderError on decoder error - [X] from_pcm() raises EncodingError on encoder error - [X] to_wave() raises EncodingError on decoder error - [X] from_wave() raises EncodingError on encoder error - [X] failed to_wave() deletes partial file - [X] failed from_wave() deletes partial file - [X] failed from_pcm() deletes partial file - [X] PCMReader raises IOError/ValueError on read() if necessary - [X] PCMReader raises DecodingError on close() if necessary **** DONE MP2Audio - [X] is_type() doesn't error - [X] __init__() raises InvalidFile - [X] to_pcm() returns PCMReaderError on decoder error - [X] from_pcm() raises EncodingError on encoder error - [X] to_wave() raises EncodingError on decoder error - [X] from_wave() raises EncodingError on encoder error - [X] failed to_wave() deletes partial file - [X] failed from_wave() deletes partial file - [X] failed from_pcm() deletes partial file - [X] PCMReader raises IOError/ValueError on read() if necessary - [X] PCMReader raises DecodingError on close() if necessary **** DONE MP3Audio - [X] is_type() doesn't error - [X] __init__() raises InvalidFile - [X] to_pcm() returns PCMReaderError on decoder error - [X] from_pcm() raises EncodingError on encoder error - [X] to_wave() raises EncodingError on decoder error - [X] from_wave() raises EncodingError on encoder error - [X] failed to_wave() deletes partial file - [X] failed from_wave() deletes partial file - [X] failed from_pcm() deletes partial file - [X] PCMReader raises IOError/ValueError on read() if necessary - [X] PCMReader raises DecodingError on close() if necessary **** DONE OggFlacAudio - [X] is_type() doesn't error - [X] __init__() raises InvalidFile - [X] to_pcm() returns PCMReaderError on decoder error - [X] from_pcm() raises EncodingError on encoder error - [X] to_wave() raises EncodingError on decoder error - [X] from_wave() raises EncodingError on encoder error - [X] failed to_wave() deletes partial file - [X] failed from_wave() deletes partial file - [X] failed from_pcm() deletes partial file - [X] PCMReader raises IOError/ValueError on read() if necessary - [X] PCMReader raises DecodingError on close() if necessary **** DONE ShortenAudio - [X] is_type() doesn't error - [X] __init__() raises InvalidFile - [X] to_pcm() returns PCMReaderError on decoder error - [X] from_pcm() raises EncodingError on encoder error - [X] to_wave() raises EncodingError on decoder error - [X] from_wave() raises EncodingError on encoder error - [X] failed to_wave() deletes partial file - [X] failed from_wave() deletes partial file - [X] failed from_pcm() deletes partial file - [X] PCMReader raises IOError/ValueError on read() if necessary - [X] PCMReader raises DecodingError on close() if necessary **** DONE SpeexAudio - [X] is_type() doesn't error - [X] __init__() raises InvalidFile - [X] from_pcm() raises EncodingError on encoder error - [X] failed from_wave() deletes partial file - [X] failed from_pcm() deletes partial file **** DONE VorbisAudio - [X] is_type() doesn't error - [X] __init__() raises InvalidFile - [X] to_pcm() returns PCMReaderError on decoder error - [X] from_pcm() raises EncodingError on encoder error - [X] to_wave() raises EncodingError on decoder error - [X] from_wave() raises EncodingError on encoder error - [X] failed to_wave() deletes partial file - [X] failed from_wave() deletes partial file - [X] failed from_pcm() deletes partial file - [X] PCMReader raises IOError/ValueError on read() if necessary - [X] PCMReader raises DecodingError on close() if necessary **** DONE WaveAudio - [X] is_type() doesn't error - [X] __init__() raises InvalidFile - [X] to_pcm() returns PCMReaderError on decoder error - [X] from_pcm() raises EncodingError on encoder error - [X] to_wave() raises EncodingError on decoder error - [X] from_wave() raises EncodingError on encoder error - [X] failed to_wave() deletes partial file - [X] failed from_wave() deletes partial file - [X] failed from_pcm() deletes partial file - [X] PCMReader raises IOError/ValueError on read() if necessary - [X] PCMReader raises DecodingError on close() if necessary **** DONE WavPackAudio - [X] is_type() doesn't error - [X] __init__() raises InvalidFile - [X] to_pcm() returns PCMReaderError on decoder error - [X] from_pcm() raises EncodingError on encoder error - [X] to_wave() raises EncodingError on decoder error - [X] from_wave() raises EncodingError on encoder error - [X] failed to_wave() deletes partial file - [X] failed from_wave() deletes partial file - [X] failed from_pcm() deletes partial file - [X] PCMReader raises IOError/ValueError on read() if necessary - [X] PCMReader raises DecodingError on close() if necessary *** DONE Check for invalid files at the tool level If an invalid file is encountered, display a proper user-readable error message explaining what's wrong with the file. **** DONE track2cd **** DONE track2track **** DONE trackcat **** DONE trackcmp **** DONE trackplay **** DONE tracksplit *** DONE Document ValueError/IOError behavior Both in docstrings and in the rst documentation. ** DONE Add a convert() method to AudioFile subclasses Something like: audiofile.convert(target_path, target_class, quality=None) so one could perform a call like: audiotools.open("infile.flac").convert("outfile.mp3", audiotools.MP3Audio) Which would perform the proper to_pcm()/from_pcm()/to_wave()/from_wave() calls as necessary and could be overloaded to handle specific conversion processes, like AIFF->Shorten with a different set of IFF chunks. *** DONE Deprecate wave-specific methods from base AudioFile class These methods might still be present, but only on classes that have specific need of them. - [X] to_wave() - [X] from_wave() - [X] supports_foreign_riff_chunks() - [X] has_foreign_riff_chunks() *** DONE Convert classes to proper WaveContainer/AiffContainer subclasses - [X] FlacAudio - [X] OggFlacAudio (should *not* be a WaveContainer/AiffContainer subclass) - [X] WaveAudio - [X] AiffAudio - [X] WavPackAudio - [X] ShortenAudio *** DONE Update old tests to reflect removed methods *** DONE Update audio formats with convert() short-circuiting as needed *** DONE Add unit tests for convert() methods **** DONE Add unit tests which convert() every type to every other type **** DONE Add unit tests to ensure foreign chunks are converted *** DONE Update track2track to exploit convert() method *** DONE Update documentation **** DONE Add convert() documentation **** DONE Remove wave-specific documentation ** DONE Update trackverify for multiprocess support I'll need some interprocess communication (probably pipes and select) to return the results from child processes for totals calculation. *** DONE Update man page *** DONE Document ExecQueue2 ** DONE Update trackcmp for multiprocess support *** DONE Update man page *** DONE Update unit tests to account for -j flag ** DONE Update encoders for thread nonblocking *** DONE encode_alac *** DONE encode_flac *** DONE encode_shn *** DONE encode_wavpack ** DONE get trackplay working on Mac OS X There must be some way to pipe PCM data to its audio system without the need for additional libraries. ** DONE Remove -v option to mv(1) for file renaming This can be emulated in software. ** DONE Update audiotools.CDDA class *** DONE Support reading offsets Without the horrible hack of reading the whole disc at once *** DONE Document audiotools.CDDA class *** DONE Document PCMReaderWindow class ** DONE Add support for System->cdrom_offset This will automatically apply offset samples when reading CDs so that rips will have the appropate amount of null samples. *** DONE Add offset support when ripping Automatically apply the configfile's cdrom_offset value to tracks during reading. *** DONE Add offset support when burning? cd2track and track2cd should round-trip properly If cd2track applies a sample offset when reading, does track2cd need to apply that same offset when writing? One would presume a drive's read offset and write offset are the same, but that may not be correct. ** DONE Add support for more lame encoding options Although the numerical presets are recommended, one should also be able to use the --preset values - [X] medium - [X] standard - [X] extreme - [X] insane ** DONE Remove wavegain for applying ReplayGain to .wav files This should be done internally instead. ** DONE Add more verbosity to --quality settings For formats with varying quality, the "-q help" option should indicate what those settings represent. - [X] FlacAudio - [X] M4AAudio_Nero - [X] MP2Audio - [X] MP3Audio - [X] OggFlacAudio - [X] SpeexAudio - [X] VorbisAudio - [X] WavPackAudio *** DONE Update tools to indicate quality settings - [X] audiotools-config - [X] cd2track - [X] dvda2track - [X] record2track - [X] track2track - [X] trackcat - [X] tracksplit ** DONE Have coverdump build leading directories as needed ** DONE Add -T / --thumbnail option to tracktag *** DONE Add option to tracktag man page ** DONE Convert ReplayGainReader to C The Python implementation simply uses too many CPU cycles, which can cause trackplay to stutter on hi-def files. *** DONE Add audiotools.replaygain.ReplayGainReader object **** DONE Add sample_rate attribute **** DONE Add channels attribute **** DONE Add channel_mask attribute **** DONE Add bits_per_sample attribute **** DONE Add read() method **** DONE Add close() method *** DONE Convert references to audiotools.ReplayGainReader *** DONE Update documentation ** DONE Have the *2xmcd utilities delete .xmcd files if cancelled - [X] cd2xmcd - [X] dvda2xmcd - [X] track2xmcd *** DONE Fix the *2xmcd utilities to use the proper mode on output files ** DONE Add software-based routines for bitstream reading/writing These should be optional, at least, for the bitstream writer especially. But if fast enough, they could replace the jump tables entirely. *** DONE bitstream_w.h - [X] write_bits_actual_be - [X] write_bits_actual_le - [X] write_unary_actual_be - [X] write_unary_actual_le ** DONE Add individual tag item removal option to tracktag - [X] --remove-name - [X] --remove-artist - [X] --remove-performer - [X] --remove-composer - [X] --remove-conductor - [X] --remove-album - [X] --remove-catalog - [X] --remove-number - [X] --remove-track-total - [X] --remove-album-number - [X] --remove-album-total - [X] --remove-ISRC - [X] --remove-media-type - [X] --remove-year - [X] --remove-date - [X] --remove-copyright - [X] --remove-comment *** DONE Unit test tag item addition/removal *** DONE Update man page with new options ** DONE Add progess indicator to various utilities This will likely require an ExecQueue update which can receive progress output from multiple subprocesses so that the total progress can be generated. - [X] track2track - [X] cd2track - [X] dvda2track - [X] tracksplit - [X] trackcat - [X] trackcmp - [X] trackverify - [X] tracktag (for ReplayGain) *** DONE Update AudioFile.convert() with a progress callback option *** DONE Update AudioFile.add_replay_gain() with a progress callback option - [X] AudioFile - [X] FlacAudio - [X] M4AAudio - [X] MP3Audio - [X] VorbisAudio - [X] WavPackAudio - [X] WaveAudio *** DONE Update programming documentation with progress callback option *** DONE Update AudioFile.verify() with a progress callback option - [X] AACAudio - [X] ALACAudio - [X] AiffAudio - [X] AuAudio - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OggFlacAudio - [X] ShortenAudio - [X] SpeexAudio - [X] VorbisAudio - [X] WavPackAudio - [X] WaveAudio *** DONE Add documentation for progress indicators and infrastructure ** DONE Add tool list and documentation to website I should probably find a way to either format the man pages to HTML directly (though groff's html output has been unfortunate in the past) or find an intermediate format that generates both man pages and web pages. - [X] audiotools-config - [X] audiotools.cfg - [X] cd2xmcd - [X] cdinfo - [X] cdplay - [X] cd2track - [X] coverdump - [X] coverview - [X] dvda2track - [X] dvda2xmcd - [X] dvdainfo - [X] editxmcd - [X] track2cd - [X] track2track - [X] track2xmcd - [X] trackcat - [X] trackcmp - [X] trackinfo - [X] tracklength - [X] tracklint - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag - [X] trackverify *** DONE Build reStructuredText output ** DONE Add options for ID3v2/ID3v1 tagging Perhaps add config file options and corresponding audiotools-config options. *** DONE Make ID3v2 version selectable Either ID3v2.2/ID3v2.3/ID3v2.4/none *** DONE Make ID3v1 version selectable ID3v1.0/ID3v1.1/none *** DONE Make ID3v2 track number formatting adjustable Allow leading 0s, since the spec doesn't forbid them. *** DONE Display current ID3 info in audiotools-config *** DONE Add ID3v2.2 option to audiotools-config *** DONE Add ID3v1 option to audiotools-config *** DONE Add ID3 number padding to audiotools-config *** DONE Document new audiotools-config options in man page *** DONE Document new audiotools config options on website ** DONE Add C-based FLAC encoder *** DONE Use VERBATIM subframes when necessary *** DONE Add significant initial padding blocks This will save a lot of time during retagging after FLAC creation *** DONE Add a variety of unit tests - [X] test_stream.sh - [X] test_flac.sh *** DONE Convert i_array size and data types to typedefs *** DONE Convert f_array size and data types to typedefs *** DONE Add more comprehensive encoding documentation *** DONE Add a variety of assert() statements As with unit tests, these ensure everything is working during testing without a performance penalty at runtime. *** DONE Handle foreign RIFF chunks *** DONE Ensure FLACs work on a variety of other decoders Although decoding properly on the reference decoder *should* guarantee the file works everywhere, the only way to be certain is to test it. *** DONE Ensure encoder raises the proper exceptions *** DONE Support Rice2 partitions? The reference encoder uses these for more efficient handling of 24-bit audio but I'm not sure they're strictly necessary for my more basic encoder. *** DONE Support wasted-bits-per-sample? I don't think I've ever seen these used on actual audio data that isn't artificial and hasn't been processed specifically for its use. As with Rice2, it's something that may get added later. *** DONE Handle multi-channel PCM data correctly Anything higher than 2 channels needs to set a channel mask and the vorbis comment to the proper value. I expect this will be a long-term project coinciding with re-engineering the to_pcm()/from_pcm() methods. *** DONE Remove external MD5 dependency *** DONE Generate SEEKTABLE blocks ** DONE Add UTF-8 to FLAC documentation *** DONE Explain how to decode a UTF-8 value *** DONE Explain how to encode a UTF-8 value ** DONE Add C-based FLAC decoder *** DONE Add a variety of unit tests *** DONE Handle Rice escape codes Not sure how to test these, but they should be handled properly. *** DONE Ensure decoder raises the proper exceptions *** DONE Handle empty MD5 sums correctly If the MD5SUM is 00000000000000000000000000000000 it's never a mismatch and should not trigger an error. **** DONE Update tracklint to populate an empty MD5 sum **** DONE Add unit tests ***** DONE Ensure an empty MD5 doesn't trigger an error at read time ***** DONE Ensure an empty MD5 doesn't trigger an error at verify time ***** DONE Ensure tracklint populates an empty MD5 correctly ** DONE Add substream support to bitstream reader Its function is to allow one to pull pieces out of a larger bitstream and process them separately. For example, while parsing an Ogg stream's pages with one bitstream, the extracted Vorbis packets could be processed with smaller substreams - including substreams that span one or more pages. It's also potentially helpful for MP3, Ogg FLAC, DVD-A and any other format with nontrivial wrappers that need to be parsed seperately from the main bitstream. *** DONE Implement substream reader methods **** DONE bs_read_bits_s_be **** DONE bs_read_bits_s_le **** DONE bs_read_signed_bits_s_be **** DONE bs_read_signed_bits_s_le **** DONE bs_read_bits64_s_be **** DONE bs_read_bits64_s_le **** DONE bs_skip_bits_s_be **** DONE bs_skip_bits_s_le **** DONE bs_read_unary_s_be **** DONE bs_read_unary_s_le **** DONE bs_read_limited_unary_s_be **** DONE bs_read_limited_unary_s_le **** DONE bs_set_endianness_s_be **** DONE bs_set_endianness_s_le **** DONE bs_read_huffman_code_s **** DONE bs_mark_s **** DONE bs_rewind_s **** DONE bs_unmark_s **** DONE bs_close_stream_s **** DONE bs_substream_new **** DONE bs_substream_f_be **** DONE bs_substream_f_le **** DONE bs_substream_p_be **** DONE bs_substream_p_le **** DONE bs_substream_s_be **** DONE bs_substream_s_le **** DONE bs_substream_append_f **** DONE bs_substream_append_p **** DONE bs_substream_append_s *** DONE Add substream and substream_append to stream initializers **** DONE bs_open - [X] substream - [X] substream_append **** DONE bs_open_python - [X] substream - [X] substream_append *** DONE Update current endianness setters with substream method **** DONE bs_set_endianness_f_be **** DONE bs_set_endianness_f_le **** DONE bs_set_endianness_p_be **** DONE bs_set_endianness_p_le *** DONE Add garbage collection to substream It should be possible to create a substream, process it partway, then append more data to it without causing any overflow problems. That is, data at the beginning of the buffer will be recycled at append time if it hasn't been marked for rewinding back to. *** DONE Add Python interface to substream **** DONE audiotools.decoders.BitstreamReader.substream **** DONE audiotools.decoders.BitstreamReader.substream_append *** TODO Add unit tests to substream ** DONE Check objects for invalid init calls If an init fails, the subsequent dealloc call shouldn't segfault Python. *** DONE audiotools.bitstream - [X] BitstreamAccumulator - [X] BitstreamReader - [X] BitstreamRecorder - [X] BitstreamWriter *** DONE audiotools.cdio - [X] CDDA - [X] CDImage *** DONE audiotools.decoders - [X] ALACDecoder - [X] AOBPCMDecoder - [X] FlacDecoder - [X] MLPDecoder - [X] OggFlacDecoder - [X] SHNDecoder - [X] Sine_Mono - [X] Sine_Simple - [X] Sine_Stereo - [X] WavPackDecoder *** DONE audiotools.pcm - [X] FrameList - [X] FloatFrameList *** DONE audiotools.resample - [X] Resampler ** DONE Update BitstreamWriter to work on Python objects That is, anything with a .write() and .close() method, similar to how BitstreamReader operates. ** DONE Calculate multi-album ReplayGain concurrently For instance, if one is calculating --replay-gain for four seperate albums and -j 4 is indicated, calculate each album's gain across its own core using the existing ProgressQueue facilities. *** DONE track2track *** DONE tracktag **** DONE update man page with -j option ** DONE Adjust FlacMetaData to store blocks internally in order That is, they should output in the same order that they are stored on disk. *** DONE Add size() method to FLAC blocks This returns the size of the block data, not including its 32-bit header - [X] Flac_STREAMINFO - [X] Flac_VORBISCOMMENT - [X] Flac_PICTURE - [X] Flac_APPLICATION - [X] Flac_SEEKTABLE - [X] Flac_CUESHEET - [X] Flac_PADDING *** DONE FlacMetaData - [X] __init__ - [X] __setattr__ - [X] __getattr__ - [X] __delattr__ - [X] converted - [X] merge - [X] add_image - [X] delete_image - [X] images - [X] clean - [X] __repr__ - [X] parse - [X] raw_info - [X] blocks - [X] build *** DONE FlacAudio - [X] channel_mask - [X] get_metadata - [X] update_metadata - [X] set_metadata - [X] set_cuesheet - [X] get_cuesheet - [X] has_foreign_riff_chunks - [X] riff_wave_chunks - [X] from_wave - [X] has_foreign_aiff_chunks - [X] from_aiff - [X] aiff_chunks - [X] add_replay_gain - [X] replay_gain - [X] clean *** DONE OggFlacMetaData - [X] converted - [X] __repr__ - [X] parse - [X] build *** DONE OggFlacAudio - [X] channel_mask - [X] update_metadata - [X] replay_gain ** DONE Have tracklint add channel mask info to hi-def FLAC files If omitted. Fix cases in which the tag is missing from the VORBISCOMMENT block, or the VORBISCOMMENT block is absent altogether. *** DONE Add unit test *** DONE Add fix to man page ** DONE Update FLAC/Vorbis to support TOTALTRACKS field for track_total This seems to be catching on as a standard. The best solution may be to implement a set of "fallback" fields such that a single metadata field may use a number of different Vorbis comment names and the first match is used. *** DONE Preserve TOTALTRACKS when updating track_total *** DONE Add unit tests to ensure TOTALTRACKS works ** DONE Don't port ReplayGain tags with AudioFile.set_metadata() That is, if we perform: >>> track1 = audiotools.open("file1") >>> track2 = audiotools.open("file2") >>> track1.set_metadata(track2.get_metadata()) >>> track1.replay_gain() != track2.replay_gain() True For all audio formats that store ReplayGain as embedded tags *** DONE Update formats - [X] FlacAudio - [X] OggFlacAudio - [X] VorbisAudio - [X] WavPackAudio *** DONE Add unit tests - [X] FlacAudio - [X] OggFlacAudio - [X] VorbisAudio - [X] WavPackAudio ** DONE Have MetaData.converted() always return a new object The following should hold for all metadata objects: >>> a = MetaData(track_name=u"Foo") >>> b = MetaData.converted(a) >>> b.track_name = u"Bar" >>> a.track_name != b.track_name True *** DONE Update MetaData classes **** DONE ApeTag **** DONE FlacMetaData **** DONE ID3CommentPair **** DONE ID3v1Comment **** DONE ID3v22Comment - [X] ID3v22_Frame - [X] ID3v22_TXX_Frame - [X] ID3v22_COM_Frame - [X] ID3v22_PIC_Frame **** DONE ID3v23Comment - [X] ID3v23_Frame - [X] ID3v23_TXXX_Frame - [X] ID3v23_COMM_Frame - [X] ID3v23_APIC_Frame **** DONE ID3v24Comment - [X] ID3v24_Frame - [X] ID3v24_TXXX_Frame - [X] ID3v24_COMM_Frame - [X] ID3v24_APIC_Frame **** DONE M4A_META_Atom - [X] M4A_Tree_Atom - [X] M4A_Leaf_Atom - [X] M4A_META_Atom - [X] M4A_FREE_Atom - [X] M4A_HDLR_Atom - [X] M4A_ILST_Leaf_Atom - [X] M4A_ILST_COVR_Data_Atom - [X] M4A_ILST_DISK_Data_Atom - [X] M4A_ILST_TRKN_Data_Atom - [ ] M4A_ILST_Unicode_Data_Atom **** DONE MetaData **** DONE OggFlacMetaData **** DONE VorbisComment *** DONE Update documentation for converted()'s behavior *** DONE Add unit tests to all MetaData classes - [X] ApeTag - [X] FlacMetaData - [X] ID3CommentPair - [X] ID3v1Comment - [X] ID3v22Comment - [X] ID3v23Comment - [X] ID3v24Comment - [X] M4A_META_Atom - [X] MetaData - [X] OggFlacMetaData - [X] VorbisComment ** DONE Add better ID3v2 documentation Do a full explanation for frames one will find in nature. Don't be too concerned about obscure frames no one ever uses. | Frame | ID3v2.2 | ID3v2.3 | ID3v2.4 | |-----------------------------+---------+---------+---------| | all text | T__ | T___ | T___ | | all web | W__ | W___ | W___ | | picture | PIC | APIC | APIC | | comment | COM | COMM | COMM | | general encapsulated object | GEO | GEOB | GEOB | | unsynchronized lyrics | ULT | USLT | USLT | *** DONE ID3v2.2 - [X] COM - [X] GEO - [X] PIC - [X] TAL - [X] TBP - [X] TCM - [X] TCO - [X] TCR - [X] TDA - [X] TDY - [X] TEN - [X] TFT - [X] TIM - [X] TKE - [X] TLA - [X] TLE - [X] TMT - [X] TOA - [X] TOF - [X] TOL - [X] TOR - [X] TOT - [X] TP1 - [X] TP2 - [X] TP3 - [X] TP4 - [X] TPA - [X] TPB - [X] TRC - [X] TRD - [X] TRK - [X] TSI - [X] TSS - [X] TT1 - [X] TT2 - [X] TT3 - [X] TXT - [X] TXX - [X] TYE - [X] ULT - [X] WAF - [X] WAR - [X] WAS - [X] WCM - [X] WCP - [X] WPB - [X] WXX *** DONE ID3v2.3 - [X] APIC - [X] COMM - [X] GEOB - [X] TALB - [X] TBPM - [X] TCOM - [X] TCON - [X] TCOP - [X] TDAT - [X] TDLY - [X] TENC - [X] TEXT - [X] TFLT - [X] TIME - [X] TIT1 - [X] TIT2 - [X] TIT3 - [X] TKEY - [X] TLAN - [X] TLEN - [X] TMED - [X] TOAL - [X] TOFN - [X] TOLY - [X] TOPE - [X] TORY - [X] TOWN - [X] TPE1 - [X] TPE2 - [X] TPE3 - [X] TPE4 - [X] TPOS - [X] TPUB - [X] TRCK - [X] TRDA - [X] TRSN - [X] TRSO - [X] TSIZ - [X] TSRC - [X] TSSE - [X] TXXX - [X] TYER - [X] USLT - [X] WCOM - [X] WCOP - [X] WOAF - [X] WOAR - [X] WOAS - [X] WORS - [X] WPAY - [X] WPUB - [X] WXXX *** DONE ID3v2.4 - [X] APIC - [X] COMM - [X] GEOB - [X] TALB - [X] TBPM - [X] TCOM - [X] TCON - [X] TCOP - [X] TDEN - [X] TDLY - [X] TDOR - [X] TDRC - [X] TDRL - [X] TDTG - [X] TENC - [X] TEXT - [X] TFLT - [X] TIPL - [X] TIT1 - [X] TIT2 - [X] TIT3 - [X] TKEY - [X] TLAN - [X] TLEN - [X] TMCL - [X] TMED - [X] TMOO - [X] TOAL - [X] TOFN - [X] TOLY - [X] TOPE - [X] TOWN - [X] TPE1 - [X] TPE2 - [X] TPE3 - [X] TPE4 - [X] TPOS - [X] TPRO - [X] TPUB - [X] TRCK - [X] TRSN - [X] TRSO - [X] TSOA - [X] TSOP - [X] TSOT - [X] TSRC - [X] TSSE - [X] TSST - [X] TXXX - [X] USLT - [X] WCOM - [X] WCOP - [X] WOAF - [X] WOAR - [X] WOAS - [X] WORS - [X] WPAY - [X] WPUB - [X] WXXX ** DONE Adjust ID3v2 to store frames internally in order That is, they should output in the same order that they are stored on disk. *** DONE Rebuild and simplify the ID3v2 frame handling Create a simple "frame protocol" which the frames can implement and the ID3v2 formats can use so that they parse/build consistently. **** DONE is_latin_1 **** DONE UCS2Codec **** DONE decode_syncsafe32 **** DONE encode_syncsafe32 **** DONE __padded_number_pair__ **** DONE __unpadded_number_pair__ **** DONE __number_pair__ **** DONE decode_ascii_c_string **** DONE encode_ascii_c_string **** DONE read_id3v2_comment() convert to standalone function **** DONE skip_id3v2_comment() convert to standalone function **** DONE ID3v22Comment - [X] __repr__ - [X] parse - [X] build - [X] size - [X] __len__ - [X] __getitem__ - [X] __setitem__ - [X] __delitem__ - [X] keys - [X] values - [X] items - [X] __getattr__ - [X] __setattr__ - [X] __delattr__ - [X] raw_info - [X] add_image - [X] delete_image - [X] images - [X] converted - [X] clean ***** DONE ID3v22_Frame - [X] __init__ - [X] __repr__ - [X] __eq__ - [X] parse - [X] build - [X] size - [X] converted - [X] clean - [X] raw_info ***** DONE ID3v22_TXX_Frame - [X] __init__ - [X] __repr__ - [X] __eq__ - [X] __unicode__ - [X] parse - [X] build - [X] size - [X] converted - [X] number - [X] total - [X] raw_info - [X] clean ***** DONE ID3v22_PIC_Frame - [X] __init__ - [X] __repr__ - [X] __eq__ - [X] __getattr__ - [X] __setattr__ - [X] parse - [X] build - [X] size - [X] converted - [X] raw_info - [X] clean ***** DONE ID3v22_COM_Frame - [X] __init__ - [X] __repr__ - [X] __eq__ - [X] __unicode__ - [X] parse - [X] build - [X] size - [X] converted - [X] raw_info - [X] clean **** DONE ID3v23Comment - [X] __repr__ - [X] parse - [X] build - [X] size - [X] __len__ - [X] __getitem__ - [X] __setitem__ - [X] __delitem__ - [X] keys - [X] values - [X] items - [X] __setattr__ - [X] __getattr__ - [X] __delattr__ - [X] raw_info - [X] add_image - [X] delete_image - [X] images - [X] converted - [X] clean ***** DONE ID3v23_Frame - [X] __init__ - [X] __repr__ - [X] __eq__ - [X] raw_info - [X] parse - [X] build - [X] size - [X] converted - [X] clean ***** DONE ID3v23_TXXX_Frame - [X] __init__ - [X] __repr__ - [X] __eq__ - [X] __unicode__ - [X] raw_info - [X] parse - [X] build - [X] size - [X] converted - [X] clean - [X] number - [X] total ***** DONE ID3v23_APIC_Frame - [X] __init__ - [X] __repr__ - [X] __eq__ - [X] __getattr__ - [X] __setattr__ - [X] raw_info - [X] parse - [X] build - [X] size - [X] converted - [X] clean ***** DONE ID3v23_COMM_Frame - [X] __init__ - [X] __repr__ - [X] __eq__ - [X] __unicode__ - [X] raw_info - [X] parse - [X] build - [X] size - [X] converted - [X] clean **** DONE ID3v24Comment - [X] __repr__ - [X] parse - [X] build - [X] size - [X] __len__ - [X] __getitem__ - [X] __setitem__ - [X] __delitem__ - [X] keys - [X] values - [X] items - [X] __setattr__ - [X] __getattr__ - [X] __delattr__ - [X] raw_info - [X] add_image - [X] delete_image - [X] images - [X] converted - [X] clean ***** DONE ID3v24_Frame - [X] __init__ - [X] __repr__ - [X] __eq__ - [X] raw_info - [X] parse - [X] build - [X] size - [X] converted - [X] clean ***** DONE ID3v24_TXXX_Frame - [X] __init__ - [X] __repr__ - [X] __eq__ - [X] __unicode__ - [X] raw_info - [X] parse - [X] build - [X] size - [X] converted - [X] clean - [X] number - [X] total ***** DONE ID3v24_APIC_Frame - [X] __init__ - [X] __repr__ - [X] __eq__ - [X] __getattr__ - [X] __setattr__ - [X] raw_info - [X] parse - [X] build - [X] size - [X] converted - [X] clean ***** DONE ID3v24_COMM_Frame - [X] __init__ - [X] __repr__ - [X] __eq__ - [X] __unicode__ - [X] raw_info - [X] parse - [X] build - [X] size - [X] converted - [X] clean *** DONE Ensure they pass unit tests *** DONE Ensure UCS-2 C strings are handled correctly For UCS-2/UTF-16 encoding, the string should end on 2 NULL bytes - [X] ID3v22_COM_Frame - [X] ID3v22_PIC_Frame - [X] ID3v23_COMM_Frame - [X] ID3v23_APIC_Frame - [X] ID3v24_COMM_Frame - [X] ID3v24_APIC_Frame *** DONE Handle user-defined text frames properly - [X] ID3v22_TXX_Frame - [X] ID3v23_TXXX_Frame - [X] ID3v24_TXXX_Frame *** DONE Handle web link frames properly - [X] ID3v22_W__Frame - [X] ID3v23_W___Frame - [X] ID3v24_W___Frame *** DONE Handle user-defined web link frames properly - [X] ID3v22_WXX_Frame - [X] ID3v23_WXXX_Frame - [X] ID3v24_WXXX_Frame *** DONE Handle text frames with embedded NULLs According to the spec, anything after the first decoded NULL should be ignored and not displayed. - [X] ID3v22_T__Frame - [X] ID3v22_TXX_Frame - [X] ID3v23_T___Frame - [X] ID3v23_TXXX_Frame - [X] ID3v24_T___Frame - [X] ID3v24_TXXX_Frame *** DONE Update tests to always check required/prohibited leading zeroes - [X] ID3v22Comment - [X] ID3v23Comment - [X] ID3v24Comment ** DONE Eliminate the MetaData.merge() method The idea of this method is to provide some sort of precedence which combining metadata from two sources. For example, in track2track, we want provided XMCD metadata to override metadata inside the track itself. The .merge() method takes a raw MetaData object from XMCD and merges it with non-blank fields from the original file which means the XMCD metadata is the "base" of sorts. A cleaner approach would be to take the raw MetaData object from XMCD, get a list of its non-empty fields and call __setattr__ on the track's metadata to update those fields. MetaData with the highest priority would call __setattr__ last. *** DONE Remove method from MetaData and subclasses - [X] ApeTag - [X] FlacMetaData - [X] ID3v22Comment - [X] ID3v23Comment - [X] ID3v24Comment - [X] ID3CommentPair - [X] M4A_META_Atom - [X] MetaData - [X] VorbisComment *** DONE Remove method from tools - [X] track2track - [X] trackrename - [X] tracksplit - [X] tracktag *** DONE Remove method from documentation *** DONE Remove method from tests ** DONE Have trackinfo display metadata fields by default Instead of displaying the low-level info, show fields like "track name", "album name", etc. unless a -R/--raw flag is indicated. *** DONE Replace MetaData.comment_pairs() with MetaData.raw_info() This should be even lower level than it is now, and unsorted. - [X] ApeTag - [X] FlacMetaData - [X] ID3CommentPair - [X] ID3v1Comment - [X] ID3v22Comment - [X] ID3v23Comment - [X] ID3v24Comment - [X] M4A_META_Atom - [X] OggFlacMetaData - [X] VorbisComment **** DONE Document MetaData.raw_info() method *** DONE Convert MetaData.__unicode__() to display fields by name This should always use the higher level implementation and sort fields by name. *** DONE Update trackinfo to use MetaData's unicode output by default *** DONE Add -L/--low-level option to trackinfo *** DONE Add -L/--low-level option to trackinfo.1 *** DONE Add consistent embedded cuesheet display *** DONE Update unit tests ** DONE Make metadata fully round-trippable For instance, the following should hold: >>> track_data1 = open(track.filename).read() >>> track.update_metadata(track.get_metadata()) >>> track_data2 = open(track.filename).read() >>> track_data1 == track_data2 True This requires metadata to not jumble fields around until required. ** DONE Don't populate empty track_name, ISRC values during tracksplit ** DONE Fix ALAC channel assignment Since the channel assignment is detailed in the specs, add proper support for them. *** DONE Update ALACAudio.channel_mask() method *** DONE Update ALACAudio.from_pcm() method Like FLAC, it should reject unsupported channel masks. *** DONE Update py_decoders.ALACDecoder's .channel_mask attribute *** DONE Update py_decoders.ALACDecoder's .read() method Convert ALAC channel order to Wave channel order internally rather than punt that task to a reordering wrapper. *** DONE Update py_encoders.encode_mdat Convert Wave channel order to ALAC channel order internally rather than punt that task to a reordering wrapper. *** DONE Update decoders.ALACDecoder's .channel_mask attribute *** DONE Update decoders.ALACDecoder's .read() method Convert ALAC channel order to Wave channel order internally rather than punt that task to a reordering wrapper. *** DONE Update encoders.encode_alac Convert Wave channel order to ALAC channel order internally rather than punt that task to a reordering wrapper. *** DONE Add unit tests to ensure channel variants encode/decode correctly - [X] 1 channel - [X] 2 channels - [X] 3 channels - [X] 4 channels - [X] 5 channels - [X] 6 channels - [X] 7 channels - [X] 8 channels **** DONE Add unit tests to ensure invalid channel counts aren't encoded **** DONE Add unit tests to ensure invalid channel masks aren't encoded ** DONE Add Python-based file encoders Like the Python-based decoders, these provide simple reference implementations one can easily pull apart to see how the encoders work. *** DONE py_encoders/encode_flac *** DONE py_encoders/encode_shn *** DONE py_encoders/encode_alac *** DONE py_encoders/encode_wavpack ** DONE Cleanup Shorten for better accuracy *** DONE Handle 8-bit files correctly waves are unsigned, aiffs are signed, Sun AU is ? - [X] update py_decoders.SHNDecoder - [X] update decoders.SHNDecoder - [X] update py_encoders.encode_shn - [X] update encoders.encode_shn - [X] update __shn__.py - [X] update documentation *** DONE Handle multichannel Shn/AIFF files correctly **** DONE Ensure multichannel files converted from AIFF work Their channels should be kept in whatever order AIFF happens to use rather than have them remapped to wave order. **** DONE Ensure multichannel Shn/AIFF to PCM works Output channels should be rearranged to PCMReader order. *** DONE Improve thread friendliness in decoder/encoder - [X] decoders/shn.c - [X] encoders/shn.h ** DONE Overhaul array.h Just as the bitstream module benefits from having a good design, the array module would also be better served by having a more elegant API. The main goal is to have less function variants and data allocation/deallocation routines to remember. Something like: array = new_int_array(); array->append(array, value); array->del(array); which always functions the same whether we're dealing with ints or floats or other arrays, yet is still type-checked at compile-time. *** DONE Better unify FrameList <-> array_ia routines *** DONE Remove old src/decoders/pcm.h src/decoders/pcm.c module Use the unified pcmconv.h module instead - [X] mlp.c - [X] shn.c - [X] sine.c *** DONE Replace old array.h with new one in all source files - [X] decoders/mlp.h - [X] decoders/shn.h - [X] decoders/sine.h - [X] decoders/wavpack.h - [X] encoders/alac.h - [X] encoders/shn.h - [X] encoders/wavpack.h *** DONE Remove old array.h *** DONE Replace array2.h with array.h in all source files ** DONE Make second pass through updated documentation *** DONE Update introduction **** DONE re-explain endianness **** DONE explain pseudocode with actual code examples *** DONE Convert writes to a systax consistent with reads var <- read 10 unsigned bits var -> write 10 unsigned bits - [X] alac.tex - [X] flac.tex - [X] shorten.tex - [X] wavpack.tex *** DONE Convert for loops to a consistent assignment syntax for i <- 0 to x do <code block> - [X] alac.tex - [X] dvda2.tex - [X] flac.tex - [X] shorten.tex - [X] wavpack.tex *** DONE Double-check codecs for consistency - [X] alac.tex - [X] dvda2.tex - [X] flac.tex - [X] shorten.tex - [X] wavpack.tex *** DONE Add hyperref linking If one's browsing the PDF, one should be able to click on operations directly and go to a specific part of the doc. - [X] alac.tex - [X] dvda2.tex - [X] flac.tex - [X] shorten.tex - [X] wavpack.tex ** DONE Tweak track labeling interactive mode for better usability make return key move to the next line ** DONE Combine and simplify DVD-A decoding and documentation The current routine bounces back and forth between Python and C several times in order to generate output, overcomplicating the design. I'd prefer to have a simpler, low-level, random-access reader hooked directly to the DVDATrack object - analagous to audiotools.cdda.CDDA. ** DONE Overhaul decoding/encoding documentation It needs to be cleaned up so one can better follow the entire decoding process, as well as the entire encoding process. Use a mix of pseudocode, bit diagrams, bit parsing diagrams and examples so that it can be followed with as little effort as possible. *** DONE FLAC **** DONE decoding **** DONE encoding *** DONE ALAC **** DONE decoding **** DONE encoding *** DONE WavPack **** DONE decoding **** DONE encoding *** DONE Shorten **** DONE decoding **** DONE encoding *** DONE DVD-A **** DONE decoding * DONE Finish version 2.18 ** DONE Ensure audiotools works on FreeBSD the unprotection module, in particular, needs additional testing ** DONE Ensure excessive zero residuals don't overflow output buffers Certain formats provide "escape code" blocks of zeroes. Ensure these routines don't generate more zeroes than are allowed (either accidentally or deliberately). *** DONE ALACDecoder *** DONE WavPackDecoder ** DONE Cleaup documentation layout Add subdirectories for format figures. ** DONE Spellcheck reference docs - [X] introduction.tex - [X] basics.tex - [X] wav.tex - [X] aiff.tex - [X] au.tex - [X] shorten.tex - [X] flac.tex - [X] wavpack.tex - [X] ape.tex - [X] mp3.tex - [X] m4a.tex - [X] alac.tex - [X] vorbis.tex - [X] oggflac.tex - [X] speex.tex - [X] musepack.tex - [X] dvda2.tex - [X] freedb.tex - [X] musicbrainz.tex - [X] replaygain.tex ** DONE Make ReplayGain a configurable option Even tag-based ReplayGain should be something users can turn on or off globally via audiotools-config *** DONE add audiotools-config option **** DONE document in audiotools-config man page *** DONE update tools to use option - [X] cdtrack - [X] dvda2track - [X] track2track - [X] tracksplit ** DONE Add progress to track2cd audio file conversion - [X] CD quality, embedded cuesheet - [X] non-CD quality, embedded cuesheet - [X] CD quality, external cuesheet - [X] non-CD quality, external cuesheet - [X] CD quality, multiple files - [X] non-CD quality, multiple files ** DONE Improve .wav performance don't read entire data chunk by default ** DONE Integrate MusicBrainz/FreeDB lookup with cd2track/dvda2track The two step extraction process is a relic from when I'd do batch lookups via modem. Combining metadata querying with extraction lets me perform more powerful lookups than are possible by using XMCD/XML file intermediaries. However, CD lookups may be still be wrong. Therefore, it's essential to have a simple, interactive track metadata editor so that this data is very easy to populate. *** DONE Add multi-track interactive editing mode to tracktag **** DONE Update man page *** DONE Build unified metadata selection widget for interactive modes This is something that can be run while a disc is being extracted or a track is being split which will drop the user back into a progress indicator once completed and then tag/rename the resulting tracks. *** DONE Add MusicBrainz/FreeDB lookup options to audiotools-config It should be possible to decide whether to query either or both as a config option. *** DONE Query MusicBrainz for album_number/album_total info Given a particular disc ID, there must be some way to pull the disc's album_number/album_total off MusicBrainz's servers if it's one of a series of discs. *** DONE Add pre-extraction metadata lookup to tracksplit **** DONE Update man page *** DONE Add pre-extraction metadata lookup to cd2track **** DONE Update man page with new options *** DONE Add pre-extraction metadata lookup to dvda2track **** DONE Update man page with new options *** DONE Update cdinfo to use new ID calculation routines Ensure they handle the FreeDB test disc properly. *** DONE Add pre-play metadata lookup to cdplay **** DONE Update man page *** DONE Add album-number/album-total options to tracksplit Allow these values to be populated at split-time if none can be found in metadata services **** DONE Update man page *** DONE Update album-number/album-total options in cd2track *** DONE Remove xmcd-specific options Automatic CD lookup should be folded into utilites as needed. - [X] cd2track - [X] cdplay - [X] dvda2track - [X] track2track - [X] trackrename - [X] tracksplit - [X] tracktag *** DONE Remove xmcd-specific tools - [X] cd2xmcd - [X] dvda2xmcd - [X] editxmcd - [X] track2xmcd *** DONE Update unit tests - [X] cd2track - [X] dvda2track - [X] tracksplit *** DONE Deprecate xmcd-specific modules Since metadata lookup handles files internally, there's no need for metadata file handling classes/functions at the Python level. **** DONE Update documentation to indicate deprecation ** DONE Adjust album_number/track_number heuristics these should be last-resort fields that apply *only* if a track has no metadata of any kind ** DONE Don't port cuesheets with AudioFile.set_metadata() This is mostly for WavPack since it embeds cuesheet data in the APEv2 tag. - [X] FlacAudio - [X] OggFlacAudio - [X] WavPackAudio ** DONE Ensure overly-long files are handled correctly That is, anything larger than a .wav can typically handle *** DONE wave should fail with error *** DONE aiff should fail with error *** DONE au should fail with error *** DONE flac should work *** DONE ogg flac should work *** DONE wavpack should work *** DONE alac should work *** DONE shorten shouldn't begin requires verbatim wave or aiff chunks which can't be created because the amount of PCM data is too large ** DONE Fix FLAC embedded cuesheets ** DONE Ensure files are PEP8-compliant *** DONE check user-level scripts - [X] audiotools-config - [X] cd2track - [X] cdinfo - [X] cdplay - [X] coverdump - [X] coverview - [X] dvda2track - [X] dvdainfo - [X] record2track - [X] track2cd - [X] track2track - [X] trackcat - [X] trackcmp - [X] trackinfo - [X] tracklength - [X] tracklint - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag - [X] trackverify - [X] setup.py *** DONE check audiotools module - [X] __aiff__.py - [X] __ape__.py - [X] __au__.py - [X] __dvda__.py - [X] __flac__.py - [X] __freedb__.py - [X] __id3__.py - [X] __id3v1__.py - [X] __image__.py - [X] __init__.py - [X] __m4a__.py - [X] __m4a_atoms__.py - [X] __mp3__.py - [X] __musepack__.py - [X] __musicbrainz__.py - [X] __ogg__.py - [X] __shn__.py - [X] __vorbis__.py - [X] __vorbiscomment__.py - [X] __wav__.py - [X] __wavpack__.py - [X] cue.py - [X] delta.py - [X] freedb.py - [X] musicbrainz.py - [X] player.py - [X] toc.py - [X] ui.py *** DONE check audiotools.py_decoders module - [X] __init__.py - [X] alac.py - [X] flac.py - [X] shn.py - [X] wavpack.py *** DONE check audiotools.py_encoders module - [X] __init__.py - [X] alac.py - [X] flac.py - [X] shn.py - [X] wavpack.py *** DONE check test - [X] test.py - [X] test_core.py - [X] test_formats.py - [X] test_metadata.py - [X] test_streams.py - [X] test_utils.py ** DONE Double-check reference docs one last time *** DONE Ensure pages break correctly in letter mode *** DONE Ensure pages break correctly in A4 mode ** DONE Fix FLAC seektable generation Ensure new seektables are aligned properly. *** DONE Ensure proper seektable written on from_pcm *** DONE Add .offsets() method to FlacDecoder Instead of decoding the file, this walks through it and returns a list of absolute file offsets of all frames. *** DONE Add method to generate Flac_SEEKTABLE from offset list *** DONE Add tracklint check/fix for mis-aligned seektables *** DONE Add unit tests for seektable errors - [X] empty seekpoints - [X] mis-ordered seekpoints - [X] bad seekpoint destinations ** DONE Shift lint to the AudioFile and MetaData subclasses Instead of having tracklint have to perform lots of built-in tests, it would be better to have clean() functions added to the classes themselves. For MetaData, it could return a new object with fixed fields. For AudioFile, it could function like convert() and build a new file with fixes applied. *** DONE Add clean() method to MetaData and subclasses **** DONE MetaData **** DONE ApeTag **** DONE WavPackAPEv2 **** DONE FlacMetaData **** DONE ID3v22Comment **** DONE ID3v23Comment **** DONE ID3v24Comment **** DONE ID3v1Comment **** DONE ID3CommentPair **** DONE M4AMetaData **** DONE VorbisComment ***** DONE FlacVorbisComment ***** DONE UnframedVorbisComment *** DONE Add clean() method to AudioFile and subclasses I expect a lot of these will do nothing in the short term. **** DONE AudioFile **** DONE AiffAudio - [X] Reorder streams in which the COMM chunk doesn't come before data - [X] remove duplicate COMM chunks - [X] remove duplicate SSND chunks ***** DONE add unit test for verify() - [X] multiple COMM chunks found - [X] multiple SSND chunks found - [X] SSND chunk before COMM chunk ***** DONE add unit test for clean() - [X] multiple COMM chunks found - [X] multiple SSND chunks found - [X] SSND chunk before COMM chunk **** DONE FlacAudio **** DONE WaveAudio - [X] reorder streams in which the fmt chunk doesn't come before data - [X] remove duplicate fmt chunks - [X] remove duplicate data chunks ***** DONE add unit test for verify() - [X] multiple fmt chunks found - [X] multiple data chunks found - [X] data chunk before fmt chunk ***** DONE add unit test for clean() - [X] multiple fmt chunks found - [X] multiple data chunks found - [X] data chunk before fmt chunk *** DONE Document MetaData clean() method *** DONE Document AudioFile clean() method *** DONE Add more comprehensive unit tests for clean() methods *** DONE Convert tracklint to use clean() methods ** DONE Ensure individual unit tests pass *** DONE Lib - [X] core - [X] cuesheet - [X] freedb - [X] image - [X] musicbrainz - [X] pcm - [X] bitstream - [X] replaygain - [X] resample - [X] tocfile - [X] verify - [X] player *** DONE Format - [X] audiofile - [X] lossless - [X] lossy - [X] aiff - [X] alac - [X] au - [X] dvda - [X] flac - [X] m4a - [X] mp2 - [X] mp3 - [X] oggflac - [X] shorten - [X] sines - [X] vorbis - [X] wave - [X] wavpack *** DONE Metadata - [X] metadata - [X] flac - [X] wavpack - [X] id3v1 - [X] id3v2 - [X] vorbis - [X] m4a *** DONE Util - [X] audiotools_config - [X] cd2track - [X] cdinfo - [X] cdplay - [X] coverdump - [X] coverview - [X] dvda2track - [X] dvdainfo - [X] record2track - [X] track2cd - [X] track2track - [X] trackcat - [X] trackcmp - [X] trackinfo - [X] tracklength - [X] tracklint - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag - [X] trackverify ** DONE Ensure complete unit test passes this includes some compound tests which may not be covered by individual tests ** DONE Cleanup bitstream module This is at the heart of a lot of things so it should be properly cleaned up. *** DONE Merge bitstream_r and bitstream_w source files *** DONE Make BitstreamReader and BitstreamWriter more symmetric *** DONE Add placeholders to BitstreamWriter These allow one to put temporary values in the stream which can be automatically filled-in later on. **** DONE Add write_placeholder method to BitstreamWriter **** DONE Add fill_placeholder method to BitstreamWriter **** DONE Add write_64_placeholder method to BitstreamWriter **** DONE Add fill_64_placeholder method to BitstreamWriter **** DONE Implement bw_write_placeholder_f **** DONE Implement bw_write_64_placeholder_f **** DONE Implement bw_fill_placeholder_f **** DONE Implement bw_fill_64_placeholder_f **** DONE Implement bw_write_placeholder_r **** DONE Implement bw_write_64_placeholder_r **** DONE Implement bw_fill_placeholder_r **** DONE Implement bw_fill_64_placeholder_r **** DONE Implement bw_write_placeholder_a Should manage the placeholder stack, but only keep track of bits sent **** DONE Implement bw_write_64_placeholder_a Should manage the placeholder stack, but only keep track of bits sent **** DONE Implement bw_fill_placeholder_a Should manage the placeholder stack, but not do anything else **** DONE Implement bw_fill_64_placeholder_a Should manage the placeholder stack, but not do anything else **** DONE Store filled placeholders in a stack for reuse **** DONE Clean out allocated placeholders as close() time ***** DONE Trigger warnings if not all placeholders are used at close() *** DONE Add C-based unit tests This should be something I can compile into a standalone file and run with no external dependencies. The idea is to build something valgrind-able to ensure there's no memory errors. **** DONE BitstreamReader ***** DONE big-endian - [X] read - [X] read_signed - [X] read_64 - [X] skip - [X] skip_bytes - [X] unread - [X] read_unary - [X] read_limited_unary - [X] read_huffman_code - [X] byte_align - [X] read_bytes - [X] set_endianness - [X] mark - [X] unmark - [X] rewind ***** DONE little-endian - [X] read - [X] read_signed - [X] read_64 - [X] skip - [X] skip_bytes - [X] unread - [X] read_unary - [X] read_limited_unary - [X] read_huffman_code - [X] byte_align - [X] read_bytes - [X] set_endianness - [X] mark - [X] unmark - [X] rewind ***** DONE br_try/etry - [X] read - [X] read_signed - [X] read_64 - [X] skip - [X] skip_bytes - [X] read_unary - [X] read_limited_unary - [X] read_huffman_code - [X] read_bytes - [X] substream_append ***** DONE Callbacks - [X] read - [X] read_signed - [X] read_64 - [X] skip - [X] skip_bytes - [X] read_unary - [X] read_limited_unary - [X] read_huffman_code - [X] read_bytes **** DONE Substream BitstreamReader **** DONE BitstreamWriter ***** DONE big-endian ***** DONE little-endian ***** DONE callbacks **** DONE BitstreamRecorder **** DONE BitstreamAccumulator *** DONE Shift BitstreamReader/BitstreamWriters to audiotools.bitstream *** DONE Add documentation for audiotools.bitstream **** DONE BitstreamReader - [X] add_callback - [X] byte_align - [X] call_callbacks - [X] close - [X] limited_unary - [X] mark - [X] parse - [X] pop_callback - [X] read - [X] read64 - [X] read_bytes - [X] read_huffman_code - [X] read_signed - [X] read_signed64 - [X] rewind - [X] set_endianness - [X] skip - [X] skip_bytes - [X] substream - [X] substream_append - [X] unary - [X] unmark - [X] unread **** DONE BitstreamWriter - [X] add_callback - [X] build - [X] byte_align - [X] call_callbacks - [X] close - [X] flush - [X] pop_callback - [X] set_endianness - [X] unary - [X] write - [X] write64 - [X] write_bytes - [X] write_signed - [X] write_signed64 **** DONE BitstreamRecorder - [X] add_callback - [X] bits - [X] build - [X] byte_align - [X] bytes - [X] call_callbacks - [X] close - [X] copy - [X] data - [X] flush - [X] pop_callback - [X] reset - [X] set_endianness - [X] split - [X] swap - [X] unary - [X] write - [X] write64 - [X] write_bytes - [X] write_signed - [X] write_signed64 **** DONE BitstreamAccumulator - [X] bits - [X] build - [X] byte_align - [X] bytes - [X] close - [X] reset - [X] set_endianness - [X] unary - [X] write - [X] write64 - [X] write_bytes - [X] write_signed - [X] write_signed64 **** DONE HuffmanTree **** DONE Substream **** DONE format_size *** DONE Handle bitstream closing more consistently The basic handle cycle is: h = open_handle(substream); /*allocates space for handle*/ h->method(h); /*performs reads/writes on handle*/ ... h->close(h); /*closes substream and deallocates handle*/ But we also need two additional methods for working with the handle and its substream seperately. | method | substream | handle | |-------------------+----------------+-------------| | close() | flushed/closed | deallocated | | close_substream() | flushed/closed | nothing | | free() | nothing | deallocated | |-------------------+----------------+-------------| This is especially important for the Python wrappers in which closing and dealloacting will come from different routines. **** DONE Add functions to bitstream core ***** DONE BitstreamReader - [X] br_close_substream_f - [X] br_close_substream_s - [X] br_close_substream_p - [X] br_close_methods - [X] br_free_f - [X] br_free_s - [X] br_free_p - [X] br_close - [X] br_read_bits_c - [X] br_read_bits64_c - [X] br_skip_bits_c - [X] br_unread_c - [X] br_read_unary_c - [X] br_read_limited_unary_c - [X] br_read_huffman_code_c - [X] br_read_bytes_c - [X] br_set_endianness_c - [X] br_close_substream_c - [X] br_mark_c - [X] br_rewind_c - [X] br_unmark_c - [X] br_substream_append_c ***** DONE BitstreamWriter - [X] bw_close_substream_f - [X] bw_close_substream_r - [X] bw_close_substream_p - [X] bw_close_substream_a - [X] bw_close_methods - [X] bw_free_f_a - [X] bw_free_r - [X] bw_free_p - [X] bw_close - [X] bw_write_bits_c - [X] bw_write_bits64_c - [X] bw_write_bytes_c - [X] bw_write_signed_bits_c - [X] bw_write_signed_bits64_c - [X] bw_write_unary_c - [X] bw_set_endianness_c - [X] bw_close_substream_c - [X] bw_byte_align_c ***** DONE Ensure bw_rec_split detects closed stream(s) properly Call bw_abort if one attempts to split from or to a closed stream. ***** DONE Ensure bw_dump_bytes detects closed stream properly Call bw_abort if one attempts to dump bytes to a closed stream. ***** DONE Ensure bw_rec_copy detects closed stream properly Call bw_abort if one attempts to copy records to a closed stream. **** DONE Update mod_bitstream.c to use new hooks properly ***** DONE Add bw_try/bw_etry wrappers around write methods - [X] BitstreamWriter_write - [X] BitstreamWriter_write_signed - [X] BitstreamWriter_write64 - [X] BitstreamWriter_write_signed64 - [X] BitstreamWriter_unary - [X] BitstreamWriter_byte_align - [X] BitstreamWriter_write_bytes - [X] BitstreamWriter_flush - [X] BitstreamRecorder_write - [X] BitstreamRecorder_write_signed - [X] BitstreamRecorder_write64 - [X] BitstreamRecorder_write_signed64 - [X] BitstreamRecorder_unary - [X] BitstreamRecorder_byte_align - [X] BitstreamRecorder_write_bytes - [X] BitstreamRecorder_copy - [X] BitstreamRecorder_split - [X] bitstream_build - [X] BitstreamAccumulator_write - [X] BitstreamAccumulator_write_signed - [X] BitstreamAccumulator_write64 - [X] BitstreamAccumulator_write_signed64 - [X] BitstreamAccumulator_unary - [X] BitstreamAccumulator_byte_align - [X] BitstreamAccumulator_write_bytes ***** DONE Ensure close() method calls bs->close_substream() - [X] BitstreamWriter - [X] BitstreamRecorder - [X] BitstreamAccumulator ***** DONE Ensure dealloc method calls bs->free() - [X] BitstreamWriter - [X] BitstreamRecorder - [X] BitstreamAccumulator ***** DONE Don't fclose() file objects from underneath Python file objects Although we may convert a Python file object to a FILE struct, we must not fclose that struct out from underneath its parent. ****** DONE BitstreamReader_close - [X] call .close() method on Python file object - [X] set read methods to raise errors - [X] return result of .close() method ****** DONE BitstreamWriter_close - [X] flush pending output - [X] call .close() method on Python file object - [X] set write methods to raise errors - [X] return result of .close() method ****** DONE BitstreamRecorder_close ****** DONE BitstreamAccumulator_close **** DONE Double-check existing C modules for proper free/close usage - [X] bitstream.c - [X] decoders/alac.c - [X] decoders/flac.c - [X] decoders/mlp.c - [X] decoders/ogg.c - [X] decoders/shn.c - [X] decoders/wavpack.c - [X] encoders/alac.c - [X] encoders/flac.c - [X] encoders/flac_lpc.c - [X] encoders/shn.c - [X] encoders/wavpack.c - [X] mod_bitstream.c - [X] verify/mpeg.c - [X] verify/ogg.c *** DONE Ensure that reading/writing closed streams always fails Once a stream's been closed, the bitstream should set I/O functions to error generators and further closes should do nothing. **** DONE Add unit test to bitstream.c **** DONE Add unit test to Python side ** DONE Add update_metadata() method to AudioFile Consider how AudioFile.set_metadata() handles six different cases: Case 1: adjusting a file's own textual metadata (tracktag) >>> m = flac1.get_metadata() >>> m.track_name = u"Fixed Name" >>> flac1.set_metadata(m) Case 2: transferring metadata from the same audio type (track2track) >>> flac1.set_metadata(flac2.get_metadata()) Case 3: transferring metadata from a different audio type (track2track) >>> flac1.set_metadata(mp3.get_metadata()) Case 4: adjusting a file's own low-level metadata (tracklint) >>> m = flac1.get_metadata() >>> m.streaminfo.md5sum = fixed_md5sum >>> flac1.set_metadata(m) Case 5: building new metadata from scratch (cd2track) >>> flac1 = FlacAudio.from_pcm(pcm_reader) >>> flac1.set_metadata(MetaData(track_name=u"New Name", ...)) Case 6: transferring adjusted metadata from the same audio type >>> m = flac2.get_metadata() >>> m.track_name = u"Adjusted Name" >>> flac1.set_metadata(m) What happens to stuff like the STREAMINFO block which contains track length, md5sum, etc. and is part of FLAC's metadata? If set_metadata() works naively and doesn't override any values, case 2 breaks since flac1 suddenly has flac2's track length. If set_metadata() overrides new metadata with stuff from the current file, case 4 breaks since the fixed_md5sum gets overridden. Counting on set_metadata() being smart enough to know what you mean is a losing proposition for FLAC, M4A and other metadata formats with embedded non-textual (side) metadata. One solution is to add a low-level AudioFile.update_metadata() method which takes only the AudioFile's required metadata type and leaves side data as-is. So case 1 becomes: >>> m = flac1.get_metadata() >>> m.track_name = u"Fixed Name" >>> flac1.update_metadata(m) and case 4 becomes: >>> m = flac1.get_metadata() >>> m.streaminfo.md5sum = fixed_md5sum >>> flac1.update_metadata(m) while the other cases remain unchanged. *** DONE Add update_metadata() method to AudioFile and subclasses for formats which take metadata at all - [X] ALACAudio - [X] AiffAudio - [X] AudioFile - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OggFlacAudio - [X] VorbisAudio - [X] WavPackAudio *** DONE Update clean() method to use update_metadata() for formats which implement clean() at all - [X] ALACAudio - [X] AiffAudio - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OggFlacAudio - [X] VorbisAudio - [X] WavPackAudio *** DONE Add update_metadata() to tracktag when not using -r/--replace option *** DONE Document new set_metadata() behavior *** DONE Document update_metadata() method *** DONE Add unit tests for update_metadata() - [X] ALACAudio - [X] AiffAudio - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OggFlacAudio - [X] VorbisAudio - [X] WavPackAudio ** DONE Ensure Python API docs are updated if necessary - [X] audiotools.rst - [X] audiotools_bitstream.rst - [X] audiotools_cdio.rst - [X] audiotools_cue.rst - [X] audiotools_pcm.rst - [X] audiotools_player.rst - [X] audiotools_replaygain.rst - [X] audiotools_resample.rst - [X] audiotools_toc.rst - [X] index.rst - [X] metadata.rst ** DONE Spellcheck Python API docs - [X] audiotools.rst - [X] audiotools_bitstream.rst - [X] audiotools_cdio.rst - [X] audiotools_cue.rst - [X] audiotools_pcm.rst - [X] audiotools_player.rst - [X] audiotools_replaygain.rst - [X] audiotools_resample.rst - [X] audiotools_toc.rst - [X] index.rst - [X] metadata.rst ** DONE Eliminate all compiler warnings - [X] check Ubuntu - [X] check Fedora - [X] check Mac OS X ** DONE Reduce load from progress updating routines Send fewer progress updates from child to parent when performing progress-based tasks. This is harder than it sounds. In an ideal world, I'd have the parent poll its children on a regular basis and update the progress bars and/or start more children. In practice, this doesn't work with subprocesses and shared memory because shared memory accumulators don't free themselves correctly when working with more than one child process. It works even less well with threading because subprocess.Popen isn't thread-safe and will certainly deadlock if called often enough. So although the current system is clunky and inefficient at least it works. ** DONE Ensure all Python modules have up-to-date docstrings - [X] __aiff__.py - [X] __ape__.py - [X] __au__.py - [X] __dvda__.py - [X] __flac__.py - [X] __id3__.py - [X] __id3v1__.py - [X] __image__.py - [X] __init__.py - [X] __m4a__.py - [X] __m4a_atoms__.py - [X] __mp3__.py - [X] __ogg__.py - [X] __shn__.py - [X] __vorbis__.py - [X] __vorbiscomment__.py - [X] __wav__.py - [X] __wavpack__.py - [X] cue.py - [X] delta.py - [X] freedb.py - [X] musicbrainz.py - [X] player.py - [X] toc.py - [X] ui.py ** DONE Ensure MANIFEST.in is complete ** DONE Update apptest.sh It's not much of a test, but it's good to keep it around as a last-resort check of things people normally run. ** DONE Run final batch of unit tests on different platforms - [X] check Ubuntu - [X] check Fedora - [X] check Mac OS X * DONE Finish version 2.19 ** DONE Update PCMReader.read() to take a PCM frame count instead of bytes Taking a byte count as an argument is a relic from the days when most conversion happened through external programs. It's time to update this function to work on PCM frames instead which is much more natural fit and makes many calculations much easier. *** DONE Update __init__.py - [X] PCMReader.read - [X] PCMReaderError.read - [X] PCMReaderProgress.read - [X] ReorderedPCMReader.read - [X] transfer_framelist_data - [X] threaded_transfer_framelist_data - [X] pcm_cmp - [X] pcm_frame_cmp - [X] PCMCat.read - [X] __buffer__.__init__ - [X] __buffer__.__len__ - [X] BufferedPCMReader.read - [X] BufferedPCMReader.__fill__ - [X] LimitedPCMReader.read - [X] pcm_split - [X] PCMConverter.read - [X] ReplayGainReader.read - [X] calculate_replay_gain - [X] AudioFile.verify - [X] PCMReaderWindow.read - [X] CDTrackReader.read - [X] CDTrackReaderAccurateRipCRC.read *** DONE Update __aiff__.py - [X] AiffReader.read - [X] AiffAudio.from_pcm *** DONE Update __au__.py - [X] AuReader.read - [X] AuReader.from_pcm *** DONE Update __flac__.py - [X] FlacAudio.__eq__ - [X] FLAC_Data_Chunk.write - [X] FLAC_SSND_Chunk.write *** DONE Update __shn__.py - [X] ShortenAudio.to_wave - [X] ShortenAudio.to_aiff *** DONE Update __wav__.py - [X] WaveReader.read - [X] WaveAudio.from_pcm - [X] WaveAudio.add_replay_gain *** DONE Update __wavpack__.py - [X] WavPackAudio.to_wave *** DONE Update py_decoders/alac.py - [X] ALACDecoder.read *** DONE Update py_decoders/flac.py - [X] FlacDecoder.read *** DONE Update py_decoders/shn.py - [X] SHNDecoder.read *** DONE Update py_decoders/wavpack.py - [X] WavPackDecoder.read *** DONE Update py_encoders/alac.py - [X] encode_mdat *** DONE Update py_encoders/flac.py - [X] encode_flac *** DONE Update py_encoders/shn.py - [X] encode_shn *** DONE Update py_encoders/wavpack.py - [X] encode_wavpack *** DONE Update src/pcmconv.c - [X] pcmreader_read - [X] pcmreader_read (alt version) *** DONE Update src/replaygain.c - [X] ReplayGainReader_read *** DONE Update src/decoders/sine.c - [X] Sine_Mono_read - [X] Sine_Stereo_read - [X] Sine_Simple_read *** DONE Update test/test.py - [X] BLANK_PCM_Reader - [X] RANDOM_PCM_Reader - [X] EXACT_BLANK_PCM_Reader - [X] EXACT_SILENCE_PCM_Reader - [X] EXACT_RANDOM_PCM_Reader - [X] MD5_Reader - [X] Variable_Reader - [X] Join_Reader - [X] MiniFrameReader *** DONE Update test/test_core.py - [X] BufferedPCMReader.test_pcm - [X] LimitedPCMReader.test_read - [X] PCMReaderWindow.__test_reader__ - [X] PCM_Reader_Multiplexer.read - [X] TestMultiChannel.__test_assignment__ *** DONE Update test/test_formats.py - [X] ERROR_PCM_Reader.read - [X] ALACFileTest.__test_reader__ - [X] ALACFileTest.__test_reader_nonalac__ - [X] ALACFileTest.test_streams - [X] FlacFileTest.test_streams - [X] FlacFileTest.__test_reader__ - [X] ShortenFileTest.test_streams - [X] ShortenFileTest.__test_reader__ - [X] WavPackFileTest.__test_reader__ *** DONE Update test/test_streams.py - [X] FrameListReader.read - [X] MD5Reader.read - [X] Sine8_Mono.read - [X] Sine8_Stereo.read - [X] Simple_Sine.read - [X] WastedBPS16.read *** DONE Ensure all unit tests pass **** DONE [Lib] - [X] core - [X] cuesheet - [X] freedb - [X] image - [X] musicbrainz - [X] pcm - [X] bitstream - [X] replaygain - [X] resample - [X] tocfile - [X] verify - [X] player **** DONE [Format] - [X] audiofile - [X] lossless - [X] lossy - [X] aiff - [X] alac - [X] au - [X] dvda - [X] flac - [X] m4a - [X] mp2 - [X] mp3 - [X] oggflac - [X] shorten - [X] sines - [X] vorbis - [X] wave - [X] wavpack **** DONE [Metadata] - [X] metadata - [X] flac - [X] wavpack - [X] id3v1 - [X] id3v2 - [X] vorbis - [X] m4a **** DONE [Util] - [X] audiotools_config - [X] cd2track - [X] cdinfo - [X] cdplay - [X] coverdump - [X] coverview - [X] dvda2track - [X] dvdainfo - [X] track2cd - [X] track2track - [X] trackcat - [X] trackcmp - [X] trackinfo - [X] tracklength - [X] tracklint - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag - [X] trackverify *** DONE Update reference documentation ** DONE Remove the big pile of imports from various modules Only import the stuff we need, when we need it. *** DONE __init__.py *** DONE accuraterip.py *** DONE aiff.py *** DONE ape.py *** DONE au.py *** DONE dvda.py *** DONE flac.py *** DONE id3.py *** DONE id3v1.py *** DONE image.py *** DONE m4a.py *** DONE m4a_atoms.py *** DONE mp3.py *** DONE ogg.py *** DONE shn.py *** DONE vorbis.py *** DONE vorbiscomment.py *** DONE wav.py *** DONE wavpack.py *** DONE cue.py *** DONE delta.py *** DONE freedb.py *** DONE musicbrainz.py *** DONE player.py *** DONE toc.py *** DONE ui.py *** DONE ensure unit tests pass **** DONE [Lib] - [X] core - [X] cuesheet - [X] freedb - [X] image - [X] musicbrainz - [X] pcm - [X] bitstream - [X] replaygain - [X] resample - [X] tocfile - [X] verify - [X] player **** DONE [Format] - [X] audiofile - [X] lossless - [X] lossy - [X] aiff - [X] alac - [X] au - [X] dvda - [X] flac - [X] m4a - [X] mp2 - [X] mp3 - [X] oggflac - [X] shorten - [X] sines - [X] vorbis - [X] wave - [X] wavpack **** DONE [Metadata] - [X] metadata - [X] flac - [X] wavpack - [X] id3v1 - [X] id3v2 - [X] vorbis - [X] m4a **** DONE [Util] - [X] audiotools_config - [X] cd2track - [X] cdinfo - [X] cdplay - [X] coverdump - [X] coverview - [X] dvda2track - [X] dvdainfo - [X] track2cd - [X] track2track - [X] trackcat - [X] trackcmp - [X] trackinfo - [X] tracklength - [X] tracklint - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag - [X] trackverify ** DONE Integrate Filename object into utilities This object should replace the Messenger.filename classmethod. It automatically performs filename -> unicode conversion and, when files are on disk, compares for equality by device/inode. *** DONE Update utilities **** DONE audiotools-config **** DONE cd2track **** DONE coverdump **** DONE coverview **** DONE dvda2track **** DONE dvdainfo **** DONE track2cd **** DONE track2track **** DONE trackcat **** DONE trackcmp **** DONE trackinfo **** DONE tracklint **** DONE trackrename **** DONE tracksplit **** DONE tracktag **** DONE trackverify *** DONE Add documentation for Filename *** DONE Add unit tests for Filename *** DONE Remove Messenger.filename classmethod **** DONE remove mention in documentation *** DONE Ensure all unit tests pass ** DONE Sanity check tool inputs/outputs *** DONE Ensure input files are included only once - [X] track2cd - generate warning - [X] track2track - generate error - [X] trackcat - generate warning - [X] trackcmp - short-circuit same file comparison - [X] tracklength - generate warning - [X] tracklint - generate error - [X] trackrename - generate error - [X] tracktag - generate error - [X] trackverify - skip duplicates **** DONE Unit test new behavior - [X] track2cd - [X] track2track - [X] trackcat - [X] trackcmp - [X] tracklint - [X] trackrename - [X] tracktag - [X] trackverify *** DONE Ensure input file(s) are different from output file(s) Overwriting files is okay by default (in the Unix tradition) but input and output as same file is not and should generate error. - [X] coverdump - [X] track2track - [X] trackcat - [X] tracksplit **** DONE Unit test new behavior - [X] coverdump - [X] track2track - [X] trackcat - [X] tracksplit *** DONE Ensure same file isn't written twice by the same utility This would typically be the result of a misused "--format" argument. - [X] cd2track - [X] dvda2track - [X] track2track - [X] trackrename - [X] tracksplit **** DONE unit test new behavior - [X] cd2track - [X] track2track - [X] trackrename - [X] tracksplit ** DONE Ensure progress display doesn't overload terminal with rows As more cores become commonplace, it's important not to overload the screen with too many progress rows in case the number of simultaneous jobs exceeds the number of terminal rows. ** DONE Improve metadata tagging widget I'm not convinced the current opened/closed bottom window is the ideal design for editing extended album/track metadata. Something more similar to a spreadsheet would be ideal but that's complicated by the serious lack of space in terminal windows. ** DONE Fix ReplayGain to work on files with different sample rates This should handle a wider array of cases than it does now. *** DONE Update can_add_replay_gain classmethod It should take a list of tracks and return True if ReplayGain can be added to them, False if not which takes the place of applicable_replay_gain - [X] AudioFile.can_add_replay_gain - [X] FlacAudio.can_add_replay_gain - [X] M4AAudio_faac.can_add_replay_gain - [X] MP3Audio.can_add_replay_gain - [X] VorbisAudio.can_add_replay_gain - [X] WaveAudio.can_add_replay_gain - [X] WavPackAudio.can_add_replay_gain **** DONE update reference documentation **** DONE update unit tests - [X] test_formats.py AudioFileTest.test_replay_gain - [X] test_utils.py track2track.test_options - [X] test_utils.py tracktag.test_options *** DONE Add supports_replay_gain() classmethod Returns True if the class supports ReplayGain of any kind. - [X] AudioFile - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OggFlacAudio - [X] VorbisAudio - [X] WavPackAudio - [X] WaveAudio **** DONE Add documentation *** DONE Remove audiotools.applicable_replay_gain function this functionality is shifted to can_add_replay_gain **** DONE remove from utilities - [X] dvda2track - [X] track2track - [X] tracksplit **** DONE update reference documentation **** DONE update unit tests *** DONE Update calculate_replay_gain function **** DONE tracks with unsupported sample rates should be resampled according to the nearest supported sample rate available **** DONE tracks with different sample rates should be resampled according to the most common sample rate available **** DONE tracks with more than two channels should be culled remove any channels above the first two during calculation **** DONE update unit tests - [X] test_core.py TestReplayGain.test_basics *** DONE Update add_replay_gain classmethods as needed - [X] AudioFile.add_replay_gain - [X] FlacAudio.add_replay_gain - [X] M4AAudio_faac.add_replay_gain - [X] MP3Audio.add_replay_gain - [X] VorbisAudio.add_replay_gain - [X] WaveAudio.add_replay_gain - [X] WavPackAudio.add_replay_gain *** DONE Update utilities to use new ReplayGain application procedure Given a list of tracks for a given album, if all are the same format and can_add_replay_gain returns True, queue a call to add_replay_gain on those tracks. **** DONE audiotools-config **** DONE cd2track **** DONE dvda2track **** DONE track2track **** DONE tracksplit **** DONE tracktag ** DONE Display X/Y progress during operations When a track is finished transcoding, for instance, output: [ 2 / 15 ] input.wav -> output.mp3 or something similar. *** DONE Update utilities - [X] cd2track - [X] dvda2track - [X] track2track - [X] trackcmp - [X] trackrename - [X] tracksplit - [X] tracktag - [X] trackverify *** DONE Update unit tests - [X] cd2track - [X] dvda2track - [X] track2track - [X] trackcmp - [X] trackrename - [X] tracksplit - [X] tracktag - [X] trackverify ** DONE Update metadata documentation with examples *** DONE FLAC *** DONE MP3 - [X] ID3v1/ID3v1.1 - [X] ID3v2.2 - [X] ID3v2.3 - [X] ID3v2.4 *** DONE APEv2 *** DONE M4A ** DONE Remove Construct module Convert all usage of Construct to BitstreamReader/BitstreamWriter. Since this is unlikely to be updated for Python 3, removing it should smooth that inevitable transition. *** DONE __accuraterip__.py *** DONE __aiff__.py - [X] chunks - [X] get_metadata - [X] set_metadata - [X] delete_metadata - [X] to_pcm - [X] from_pcm - [X] pcm_split - [X] aiff_from_chunks **** DONE AiffReader - [X] __init__ - [X] read *** DONE __ape__.py *** DONE __au__.py *** DONE cue.py *** DONE __dvda__.py **** DONE DVDAudio - [X] __titlesets__ - [X] __titles__ **** DONE DVDATitle - [X] __parse_info__ *** DONE __flac__.py *** DONE __freedb__.py *** DONE __id3__.py *** DONE __id3v1__.py *** DONE __image__.py - [X] __JPEG__ - [X] __PNG__ - [X] __GIF__ - [X] __BMP__ - [X] __TIFF__ **** DONE check these against PIL's output **** DONE check against truncated images *** DONE __init__.py *** DONE __m4a__.py **** DONE M4ATaggedAudio - [X] get_metadata - [X] set_metadata - [X] delete_metadata **** DONE M4A_META_Atom - [X] __init__ - [X] __repr__ - [X] parse - [X] __getattr__ - [X] __setattr__ - [X] __delattr__ - [X] images - [X] add_image - [X] delete_image - [X] converted - [X] __comment_name__ - [X] supports_images - [X] __by_pair__ - [X] __comment_pairs__ - [X] clean **** DONE M4AAudio_faac - [X] __init__ - [X] is_type **** DONE ALACAudio - [X] __init__ - [X] is_type - [X] to_pcm - [X] from_pcm - [X] __ftyp_atom__ - [X] __moov_atom__ - [X] __free_atom__ *** DONE __m4a_atoms__.py Re-implement atom parsing/building without Construct **** DONE ftyp **** DONE moov ***** DONE mvhd ***** DONE trak ****** DONE tkhd ****** DONE mdia ******* DONE mdhd ******* DONE hdlr ******* DONE minf ******** DONE smhd ******** DONE dinf ********* DONE dref ******** DONE stbl ********* DONE stsd ********** DONE alac ********* DONE stts ********* DONE stsc ********* DONE stsz ********* DONE stco ***** DONE udta ****** DONE meta **** DONE free **** DONE Rename __m4a_atoms2__.py to __m4a_atoms__.py *** DONE __mp3__.py **** DONE MP3 - [X] __init__ - [X] is_type - [X] __find_next_mp3_frame__ - [X] __find_mp3_start__ - [X] __find_last_mp3_frame__ - [X] verify **** DONE MP2 - [X] is_type *** DONE __musepack__.py *** DONE __musicbrainz__.py *** DONE __ogg__.py *** DONE player.py *** DONE __shn__.py *** DONE toc.py *** DONE __vorbis__.py **** DONE VorbisAudio - [X] __read_metadata__ - [X] total_frames - [X] get_metadata - [X] set_metadata **** DONE Remove cruft - [X] OggStreamReader - [X] OggStreamWriter *** DONE __vorbiscomment__.py *** DONE __wav__.py *** DONE __wavpack__.py *** DONE test/test_core.py **** DONE TestFrameList - [X] test_8bit_roundtrip - [X] test_16bit_roundtrip - [X] test_24bit_roundtrip *** DONE test/test_formats.py **** DONE ALACFileTest - [X] test_blocksizes **** DONE FlacFileTest - [X] test_blocksizes **** DONE ShortenFileTest - [X] test_blocksizes **** DONE WavpackFileTest - [X] test_blocksizes ** DONE Replace gettext with string constants Transforming _(u"some text") to SOME_TEXT where SOME_TEXT is a predefined unicode string in an audiotools sub-module. This makes text more consistent *and* easier to modify. *** DONE Update utilities - [X] audiotools-config - [X] cd2track - [X] cdinfo - [X] cdplay - [X] coverdump - [X] coverview - [X] dvda2track - [X] dvdainfo - [X] track2cd - [X] track2track - [X] trackcat - [X] trackcmp - [X] trackinfo - [X] tracklength - [X] tracklint - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag - [X] trackverify *** DONE Update modules - [X] __init__.py - [X] aiff.py - [X] ape.py - [X] au.py - [X] cue.py - [X] dvda.py - [X] flac.py - [X] image.py - [X] m4a.py - [X] m4a_atoms.py - [X] mp3.py - [X] ogg.py - [X] toc.py - [X] ui.py - [X] vorbis.py - [X] vorbiscomment.py - [X] wav.py - [X] wavpack.py *** DONE Remove gettext - [X] audiotools-config - [X] cd2track - [X] cdinfo - [X] cdplay - [X] coverdump - [X] coverview - [X] dvda2track - [X] dvdainfo - [X] track2cd - [X] track2track - [X] trackcat - [X] trackcmp - [X] trackinfo - [X] tracklength - [X] tracklint - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag - [X] trackverify - [X] audiotools/__init__.py - [X] audiotools/aiff.py - [X] audiotools/ape.py - [X] audiotools/au.py - [X] audiotools/cue.py - [X] audiotools/flac.py - [X] audiotools/id3.py - [X] audiotools/image.py - [X] audiotools/m4a.py - [X] audiotools/mp3.py - [X] audiotools/toc.py - [X] audiotools/vorbis.py - [X] audiotools/wav.py - [X] audiotools/wavpack.py *** DONE Update unit tests - [X] test_formats.py - [X] test_metadata.py - [X] test_utils.py *** DONE Reduce big import chunks Convert "from audiotools.text import (CONSTANT, ...)" to "import audiotools.text as t" and "t.CONSTANT" to avoid having huge import blocks at the start of utilities. - [X] audiotools-config - [X] cd2track - [X] cdinfo - [X] cdplay - [X] coverdump - [X] coverview - [X] dvda2track - [X] dvdainfo - [X] track2cd - [X] track2track - [X] trackcat - [X] trackcmp - [X] trackinfo - [X] tracklength - [X] tracklint - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag - [X] trackverify *** DONE Ensure unit tests pass ** DONE Update C-based ReplayGainReader Use a lot of the new C-based facilities to make it simpler. ** DONE Fix --number=0 argument to tracktag I'd like to make track_number=0 a valid value and have track_number=None indicate a missing field. The problem is maintaining consistency between metadata formats that store track numbers as text (like ID3v2) and those that store track numbers as integers (like M4A). - [X] MetaData - [X] ApeTag - [X] FlacMetaData - [X] ID3CommentPair - [X] ID3v1Comment - [X] ID3v22Comment - [X] ID3v23Comment - [X] ID3v24Comment - [X] M4A_META_Atom - [X] VorbisComment *** DONE Update utilities *** DONE Update documentation *** DONE Update unit tests **** DONE Add getitem/setitem/getattr/setattr/delattr tests to metadata - [X] flac - [X] wavpack - [X] id3v1 - [X] id3v2 - [X] vorbis - [X] m4a ** DONE Remove Python Imaging Library requirement This is only used for thumbnailing and for TKinter-based image display. Since a Python3 version doesn't seem to be pending, it would be better to remove the requirement. *** DONE Remove thumbnail functions from audiotools.image - [X] can_thumbnail - [X] thumbnail_formats - [X] thumbnail_image *** DONE Remove thumbnail method from audiotools.Image *** DONE Remove thumbnail config options - [X] audiotools.THUMBNAIL_FORMAT - [X] audiotools.THUMBNAIL_SIZE *** DONE Update utilities - [X] audiotools-config - [X] track2track - [X] tracktag *** DONE Update utility man pages - [X] audiotools-config - [X] track2track - [X] tracktag *** DONE Update programming documentation *** DONE Update unit tests ** DONE Update cuesheet interface to handle non-CD sheets Though rare, non-CD audio is sometimes combined with cuesheets and should be handled properly in those instances. *** DONE Add sample_rate field to cuesheet pcm_lengths() method - [X] Cuesheet.pcm_lengths - [X] Flac_CUESHEET.pcm_lengths - [X] TOCFile.pcm_lengths **** DONE Update documentation **** DONE Update unit tests *** DONE Remove sheet_to_unicode function *** DONE Update utilities to use sample_rate field with pcm_lengths - [X] trackinfo - [X] tracksplit - [X] tracktag *** DONE Ensure unit tests pass ** DONE Add faster and more accurate audio type identifier Instead of checking open files on a format-by-format basis to determine its type via looping, it's more effecient to check for all possible file types from the same stream simultaneously via a sort of finite automata. *** DONE Add file_type function - [X] ALACAudio - [X] AiffAudio - [X] AuAudio - [X] FlacAudio - without ID3v2 tag - [X] M4AAudio - [X] MP2Audio - without ID3v2 tag - [X] MP3Audio - without ID3v2 tag - [X] OggFlacAudio - [X] ShortenAudio - [X] VorbisAudio - [X] WavPackAudio - [X] WaveAudio - [X] FlacAudio - with ID3v2 tag - [X] MP2Audio - with ID3v2 tag - [X] MP3Audio - with ID3v2 tag *** DONE Update functions which use is_type method to use file_type function - [X] trackverify - [X] audiotools.open *** DONE Remove is_type method from AudioFile classes - [X] ALACAudio - [X] AiffAudio - [X] AuAudio - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OggFlacAudio - [X] ShortenAudio - [X] VorbisAudio - [X] WavPackAudio - [X] WaveAudio *** DONE Update programming documentation *** DONE Update unit tests - [X] test_formats.py ** DONE Update utilities to handle broken track_name template fields - [X] cd2track - [X] dvda2track - [X] track2track - [X] trackrename - [X] tracksplit *** DONE Add unit tests for broken track_name template fields ** DONE Split off tracktag's functionality *** DONE Move the image options into a specialized tool (covertag?) Its interactive mode should use PyGTK/Tkinter since it's helpful to see the images one is selecting to add/remove. - [X] --remove-images - [X] --front-cover - [X] --back-cover - [X] --leaflet - [X] --media - [X] --other-image - [X] -T, --thumbnail *** DONE Move CD options into a specialized tool Something for tagging a group of tracks as if they're an entire CD - [X] --cue - [X] -l, --lookup - [X] --musicbrainz-server - [X] --musicbrainz-port - [X] --no-musicbrainz - [X] --freedb-server - [X] --freedb-port - [X] --no-freedb ** DONE Add interative mode to audiotools-config This is one instance where seeing all the options "up-front" is likely to make the process easier. ** DONE Add Python-based file decoders These would be low-performance, Python-based PCMReader-style objects demonstrating how the decoding process works in a relatively simple manner through the use of the BitstreamReader objects. They would also provide reference implementations. *** DONE py_decoders/FlacDecoder *** DONE py_decoders/SHNDecoder *** DONE py_decoders/ALACDecoder *** DONE py_decoders/WavPackDecoder ** DONE Update .convert() method to use fewer temporary files .wav and .aiff containers with embedded chunks currently route data through actual .wav/.aiff files in order to pass those chunks to another format. It would be better to avoid creating an intermediate file whenever possible. *** DONE Update wav containers to use new interface WaveContainer.has_foreign_wav_chunks() returns True if the instance has chunks to convert WaveContainer.header_footer() returns (header, footer) binary strings WaveContainer.from_wave(header, pcmreader, footer) returns new instance built from heaeder, PCM data and footer Once interface is in place, .convert() can pass header/footer and wrap progress monitor around pcmreader in order to avoid temporary wav files. **** DONE Update FlacAudio - [X] has_foreign_wave_chunks() - [X] wave_header_footer() - [X] from_wave(filename, header, pcmreader, footer, compression) - [X] convert(target_page, target_class, compression, progress) **** DONE Update OggFlacAudio - [X] has_foreign_wave_chunks() - [X] wave_header_footer() - [X] from_wave(filename, header, pcmreader, footer, compression) - [X] convert(target_page, target_class, compression, progress) **** DONE Update ShortenAudio - [X] has_foreign_wave_chunks() - [X] wave_header_footer() - [X] from_wave(filename, header, pcmreader, footer, compression) - [X] convert(target_page, target_class, compression, progress) **** DONE Update WavPackAudio - [X] has_foreign_wave_chunks() - [X] wave_header_footer() - [X] from_wave(filename, header, pcmreader, footer, compression) - [X] convert(target_page, target_class, compression, progress) **** DONE Update WaveAudio - [X] has_foreign_wave_chunks() - [X] wave_header_footer() - [X] from_wave(filename, header, pcmreader, footer, compression) - [X] convert(target_page, target_class, compression, progress) **** DONE Update programming documentation **** DONE Update unit tests *** DONE Update aiff containers to use new interface AiffContainer.has_foreign_aiff_chunks() returns True if the instance has chunks to convert AiffContainer.header_footer() returns (header, footer) binary strings AiffContainer.from_aiff(header, pcmreader, footer) returns new instance built from heaeder, PCM data and footer Once interface is in place, .convert() can pass header/footer and wrap progress monitor around pcmreader in order to avoid temporary aiff files. **** DONE Update AiffAudio - [X] has_foreign_aiff_chunks() - [X] aiff_header_footer() - [X] from_aiff(filename, header, pcmreader, footer, compression) - [X] convert(target_page, target_class, compression, progress) **** DONE Update FlacAudio - [X] has_foreign_aiff_chunks() - [X] aiff_header_footer() - [X] from_aiff(filename, header, pcmreader, footer, compression) - [X] convert(target_page, target_class, compression, progress) **** DONE Update OggFlacAudio - [X] has_foreign_aiff_chunks() - [X] aiff_header_footer() - [X] from_aiff(filename, header, pcmreader, footer, compression) - [X] convert(target_page, target_class, compression, progress) **** DONE Update ShortenAudio - [X] has_foreign_aiff_chunks() - [X] aiff_header_footer() - [X] from_aiff(filename, header, pcmreader, footer, compression) - [X] convert(target_page, target_class, compression, progress) **** DONE Update programming documentation **** DONE Update unit tests ** DONE Add pause()/resume() methods to low-level output players This will hopefully make pausing more responsive and work better than simply emptying the output buffer. ** DONE Ensure unicode command-line arguments are parsed properly *** DONE cd2track - [X] --format - [X] --dir *** DONE coverdump - [X] filename arguments - [X] --dir - [X] --prefix *** DONE covertag - [X] filename arguments - [X] --front-cover - [X] --back-cover - [X] --leaflet - [X] --media - [X] --other-image *** DONE track2track - [X] filename arguments - [X] --dir - [X] --format - [X] --output *** DONE trackcat - [X] filename arguments - [X] --output - [X] --cue *** DONE trackcmp - [X] filename arguments *** DONE trackinfo - [X] filename arguments *** DONE tracklength - [X] filename arguments *** DONE tracklint - [X] filename arguments - [X] --db *** DONE trackrename - [X] filename arguments - [X] --format *** DONE tracksplit - [X] filename arguments - [X] --cue - [X] --dir - [X] --format *** DONE tracktag - [X] filename arguments - [X] --name - [X] --artist - [X] --album - [X] --performer - [X] --composer - [X] --conductor - [X] --catalog - [X] --ISRC - [X] --publisher - [X] --media-type - [X] --year - [X] --date - [X] --copyright - [X] --comment - [X] --comment-file ** DONE Add unit tests for Python-based decoders Since Python-based codecs are so much slower it's impossible to make these as comprehensive as the C-based tests. So we'll only be able to test the basics using very small streams against the output of the C-based decoders. *** DONE FlacDecoder *** DONE ALACDecoder *** DONE WavPackDecoder *** DONE SHNDecoder ** DONE Add unit tests for Python-based encoders *** DONE encode_flac *** DONE encode_mdat *** DONE encode_wavpack *** DONE encode_shn ** DONE Ensure Python files are PEP8-compliant *** DONE executables - [X] audiotools-config - [X] cd2track - [X] cdinfo - [X] cdplay - [X] coverdump - [X] covertag - [X] coverview - [X] dvda2track - [X] dvdainfo - [X] track2cd - [X] track2track - [X] trackcat - [X] trackcmp - [X] trackinfo - [X] tracklength - [X] tracklint - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag - [X] trackverify *** DONE libraries - [X] __init__.py - [X] accuraterip.py - [X] aiff.py - [X] ape.py - [X] au.py - [X] cue.py - [X] delta.py - [X] dvda.py - [X] flac.py - [X] freedb.py - [X] id3.py - [X] id3v1.py - [X] image.py - [X] m4a.py - [X] m4a_atoms.py - [X] mp3.py - [X] musicbrainz.py - [X] ogg.py - [X] opus.py - [X] player.py - [X] shn.py - [X] text.py - [X] toc.py - [X] ui.py - [X] vorbis.py - [X] vorbiscomment.py - [X] wav.py - [X] wavpack.py **** DONE py_decoders - [X] __init__.py - [X] alac.py - [X] flac.py - [X] shn.py - [X] wavpack.py **** DONE py_encoders - [X] __init__.py - [X] alac.py - [X] flac.py - [X] shn.py - [X] wavpack.py ** DONE Have interactive modes clear screen after finishing Some systems leave the Urwid-built screens half-cleared before resuming regular output. It's preferable to erase the whole thing in that case. - [X] audiotools-config - [X] cd2track - [X] cdplay - [X] dvda2track - [X] track2track - [X] trackcat - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag ** DONE Have interactive modes detect termios.error at launch This seems to be caused mostly by opening interactive modes with arguments piped from xargs - [X] audiotools-config - [X] cd2track - [X] cdplay - [X] dvda2track - [X] track2track - [X] trackcat - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag ** DONE Spellcheck programming documentation - [X] audiotools.rst - [X] audiotools_bitstream.rst - [X] audiotools_cdio.rst - [X] audiotools_cue.rst - [X] audiotools_pcm.rst - [X] audiotools_pcmconverter.rst - [X] audiotools_player.rst - [X] audiotools_replaygain.rst - [X] audiotools_toc.rst - [X] index.rst - [X] metadata.rst ** DONE Spellcheck reference documentation - [X] aiff.tex - [X] alac.tex - [X] ape.tex - [X] apev2.tex - [X] au.tex - [X] basics.tex - [X] dvda2.tex - [X] flac.tex - [X] freedb.tex - [X] introduction.tex - [X] license.tex - [X] m4a.tex - [X] mp3.tex - [X] musepack.tex - [X] musicbrainz.tex - [X] musicbrainz_mmd.tex - [X] ogg.tex - [X] oggflac.tex - [X] references.tex - [X] replaygain.tex - [X] shorten.tex - [X] speex.tex - [X] vorbis.tex - [X] wav.tex - [X] wavpack.tex ** DONE Spellcheck reference figures ** DONE Spellcheck manual pages ** DONE Split reference documentation into sections by file That is, massive codecs should be split into individual .tex files by decode/encode sections and then subdivided further as necessary to keep them from getting too fragile/unweildy *** DONE Shorten - [X] decode - [X] encode *** DONE FLAC **** DONE metadata **** DONE decode **** DONE encode - [X] fixed - [X] residual - [X] lpc *** DONE WavPack **** DONE decode - [X] terms - [X] weights - [X] samples - [X] entropy - [X] bitstream - [X] decorrelation **** DONE encode - [X] correlation - [X] terms - [X] weights - [X] samples - [X] entropy - [X] bitstream *** DONE ALAC **** DONE decode **** DONE encode - [X] atoms - [X] lpc - [X] residual *** DONE Remove m4 requirement This is massive overkill for what little we need it for, which is generating audioformats-_.tex and _-codec.tex files with header/footers included and paper size populated. It's better to use Python for that trivial templating instead. ** DONE Tweak reference documentation layout Try harder to get pseudocode, file diagram, bit diagram and example on same pair of pages. *** DONE Shorten *** DONE FLAC *** DONE WavPack *** DONE ALAC *** DONE DVD-A ** DONE Ensure documentation flows correctly Pages should start/end properly at letter and A4 paper sizes. - [X] letter - [X] A4 ** DONE Handle multi-channel .opus files properly *** DONE Add unit tests ** DONE Add optional interactive modes to utilities Now that Urwid is being used for editxmcd, it might be helpful to add optional console-based interactive modes to the other tools. This may improve ease-of-use (particularly discoverability in the case of format and quality options) without sacrificing scriptability or command-line power. Just as the command-line options are kept as consistent as possible, all interactive modes will also need a consistent interface. *** DONE audiotools-config *** DONE cd2track *** DONE cdplay **** DONE make player widget generic *** DONE dvda2track *** DONE track2track *** DONE trackcat *** DONE trackplay **** DONE make player widget generic *** DONE trackrename *** DONE tracksplit *** DONE tracktag ** DONE run codecs through valgrind Try to ensure there's no hidden bugs in the low-level C code. *** DONE decoders Will need to assemble standalone decoders for this. - [X] alac - [X] flac - [X] mlp - [X] oggflac - [X] shorten - [X] wavpack *** DONE encoders - [X] alac - [X] flac - [X] shorten - [X] wavpack *** DONE Update standalone encoders to take command-line arguments This would make them easier to memory test. - [X] alac - [X] flac - [X] shorten - [X] wavpack ** DONE Have cdplay play properly ** DONE Update version number to 2.19 final * DONE Finish version 2.20 ** DONE Remove track number/album number guessing heuristics Don't try to guess number from filename. If a track number is needed, build it from the order in which the files are submitted on the command line. *** DONE track_number() *** DONE album_number() ** DONE Update track2track/tracktag to use better option filler widget cd2track, dvda2track and tracksplit always work with one album at a time, pretty much by definition. track2track may handle multiple albums simultaneously and currently makes mutiple calls to urwid to work. A better approach is to have all the metadata lookups performed first, multiple metadata edit screens, and a single output options/preview screen. ** DONE Simplify PCMReaderWindow Split this into a PCMReaderHead and PCMReaderDeHead pair. The former handles the "pcm_frames" argument, either truncating a PCMReader or appending silence as needed. The latter handles the "initial_offset" argument, either chopping off frames or prepending silence as needed. ** DONE Adjust BitstreamReader to better handle huge byte counts These are typically done in error and shouldn't eat up all the RAM in the world before failing. *** DONE Update functions - [X] br_substream_append_f - [X] br_substream_append_e - [X] BitstreamReader_read_bytes - [X] BitstreamReader_substream_meth - [X] BitstreamReader_substream_append *** DONE Add unit tests - [X] br_substream_append_f - [X] br_substream_append_e - [X] BitstreamReader_read_bytes - [X] BitstreamReader_substream_meth - [X] BitstreamReader_substream_append *** DONE turn the current buf_append into buf_extend *** DONE Replace buf_extend with buf_append ** DONE Have process-based decoders exit when data runs out That is, have PCMReader's .read() call subprocess.wait() when the framelists end rather than waiting for .close() to be called (which typically isn't). If the wait fails, have it raise an exception at that point. *** DONE Update process-based formats - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OpusAudio *** DONE Update documentation *** DONE Ensure unit tests pass ** DONE Reduce stdint usage in bitstream library Try to limit stdint variables to places where it's necessary, like read_64 and read_bytes *** DONE Convert state/context to state_t **** DONE Update br_huffman_table **** DONE Update BitstreamReader **** DONE struct read_bits **** DONE struct unread_bit **** DONE struct read_unary **** DONE struct read_limited_unary *** DONE Update jump table structures **** DONE read_bits **** DONE unread_bit **** DONE read_unary **** DONE read_limited_unary *** DONE Update br_huffman_table **** DONE Update continue_ to int *** DONE Update bs_buffer **** DONE unsigned data_size **** DONE unsigned window_start **** DONE unsigned window_end **** DONE buf_resize() **** DONE buf_write() **** DONE buf_read() *** DONE Update br_mark **** DONE unsigned substream **** DONE unsigned external *** DONE Update substream_append() **** DONE br_substream_append_f **** DONE br_substream_append_s **** DONE br_substream_append_e **** DONE br_substream_append_c *** DONE Update bw_external_output **** DONE buffer_size **** DONE bw_open_external() **** DONE ext_open_w() ** DONE Add bs_buffer-specific unit tests to bitstream.c - [X] buf_resize - [X] buf_write - [X] buf_read - [X] buf_copy - [X] buf_extend - [X] buf_reset - [X] buf_getc - [X] buf_putc - [X] buf_getpos - [X] buf_setpos - [X] buf_set_rewindable - [X] BUF_WINDOW_SIZE - [X] BUF_WINDOW_START - [X] BUF_WINDOW_END ** DONE Fix cuesheet handling Ensure the cuesheet interface handles unusual cases more accurately and build a proper base class for the .cue and .toc files *** DONE Add new base classes - [X] Sheet - [X] SheetTrack - [X] SheetIndex *** DONE Update cue module - [X] read_cuesheet - [X] write_cuesheet *** DONE Update toc module - [X] read_tocfile - [X] write_tocfile *** DONE Update get_cuesheet methods - [X] FlacAudio - [X] TrueAudio - [X] WavPackAudio *** DONE Update set_cuesheet methods - [X] FlacAudio - [X] TrueAudio - [X] WavPackAudio *** DONE Update documentation - [X] read_sheet - [X] Sheet - [X] SheetTrack - [X] SheetIndex *** DONE Update utilities using cuesheet interface - [X] track2cd - [X] trackcat - [X] trackinfo - [X] tracksplit *** DONE Update unit tests *** DONE Add unit test for FlacAudio.set_cuesheet() Ensure the resulting Flac_CUESHEET block matches the one generated by metaflac --import-cuesheet-from ** DONE Add optional total_pcm_frames argument to from_pcm() Certain formats encode much easier if the total number of PCM frames is known in advance. Adding a total_pcm_frames argument allows encoders to make use of that info if it's available. *** DONE Update formats with total_pcm_frames argument **** DONE AudioFile does nothing **** DONE ALACAudio Builds placeholder seektable which is then populated later **** DONE AiffAudio does nothing **** DONE AuAudio does nothing **** DONE FlacAudio Allocates enough space for populated seektable on top of any leftover for VORBIS tags. **** DONE M4AAudio does nothing **** DONE MP2Audio does nothing **** DONE MP3Audio does nothing **** DONE OggFlacAudio does nothing **** DONE OpusAudio does nothing **** DONE ShortenAudio If total_pcm_frames argument is given to from_pcm(), precalculate Wave header rather than use temporary file. This shouldn't require updating the encode_shn function. **** DONE TrueAudio If total_pcm_frames argument is given to from_pcm(), fill header and temporary seektable rather than use temporary file. This shouldn't require updating the encode_tta function. **** DONE VorbisAudio does nothing **** DONE WavPackAudio If total_pcm_frames argument is given to from_pcm(), fill blocks during encoding rather than re-fill them afterward. This will require minor update to encode_wavpack function. **** DONE WaveAudio does nothing *** DONE Update convert() methods to call from_pcm() with argument But only if the input file is lossless. **** DONE AudioFile **** DONE ALACAudio **** DONE AiffAudio **** DONE AuAudio **** DONE FlacAudio **** DONE M4AAudio **** DONE MP2Audio **** DONE MP3Audio **** DONE OggFlacAudio **** DONE OpusAudio **** DONE ShortenAudio **** DONE TrueAudio **** DONE VorbisAudio **** DONE WavPackAudio **** DONE WaveAudio *** DONE Update utilities to make use of total_pcm_frames argument **** DONE cd2track **** DONE dvda2track **** DONE trackcat **** DONE tracksplit *** DONE Document total_pcm_frames argument *** DONE Unit test calling formats with total_pcm_frames and without *** DONE Add unit tests with total_pcm_frames and without per encoder - [X] ALACAudio.from_pcm() - [X] FlacAudio.from_pcm() - [X] ShnAudio.from_pcm() - [X] TTAAudio.from_pcm() - [X] WavPackAudio.from_pcm() ** DONE Make decoder streams seekable PCMReader.seek(pcm_frames) -> new pcm frames position could seek to the closest position in the stream to the given frames without going over, and return the current position in PCM frames. *** DONE Update decoders **** DONE AiffAudio **** DONE AuAudio **** DONE WaveAudio **** DONE decoders.ALACDecoder **** DONE decoders.FlacDecoder **** DONE decoders.TTADecoder **** DONE decoders.WavPackDecoder *** DONE Add unit tests ** DONE Swap array types for something easier to comprehend array_i/array_f/array_u is more opaque than it should be *** DONE array_i -> a_int *** DONE array_i_new() -> a_int_new() - [X] decoders/alac.h/.c - [X] decoders/flac.h/.c - [X] decoders/mlp.h/.c - [X] decoders/oggflac.h/.c - [X] decoders/shn.h/.c - [X] decoders/tta.h/.c - [X] decoders/wavpack.h/.c - [X] encoders/alac.h/.c - [X] encoders/flac.h/.c - [X] encoders/shn.h/.c - [X] encoders/tta.h/.c - [X] encoders/wavpack.h/.c - [X] pcmconverter.h/.c *** DONE array_ia -> aa_int *** DONE array_ia_new() -> aa_int_new() - [X] decoders/alac.h/.c - [X] decoders/aob.h/.c - [X] decoders/flac.h/.c - [X] decoders/mlp.h/.c - [X] decoders/oggflac.h/.c - [X] decoders/shn.h/.c - [X] decoders/sine.h/.c - [X] decoders/tta.h/.c - [X] decoders/wavpack.h/.c - [X] encoders/alac.h/.c - [X] encoders/flac.h/.c - [X] encoders/shn.h/.c - [X] encoders/tta.h/.c - [X] encoders/wavpack.h/.c - [X] pcmconverter.h/.c - [X] replaygain.h/.c *** DONE array_li -> l_int *** DONE array_li_new() -> l_int_new() - [X] encoders/flac.h/.c *** DONE array_lia -> al_int *** DONE array_lia_new() -> al_int_new() - [X] encoders/flac.h/.c - [X] pcmconverter.h/.c *** DONE array_u -> l_unsigned *** DONE array_u_new -> l_unsigned_new() - [X] decoders/alac.c - [X] encoders/alac.c *** DONE array_lf -> l_double *** DONE array_lu -> l_unsigned *** DONE array_lu_new() -> l_unsigned_new() - [X] decoders/alac.h/.c *** DONE array_lfa -> al_double *** DONE array_lfa_new() -> al_double_new() *** DONE array_iaa -> aaa_int *** DONE array_iaa_new() -> aaa_int_new() - [X] encoders/wavpack.h/.c - [X] decoders/wavpach.h/.c *** DONE array_f -> a_double *** DONE array_f_new() -> a_double_new() - [X] pcmconverter.h/.c - [X] encoders/alac.h/.c - [X] encoders/flac.h/.c *** DONE array_fa -> aa_double *** DONE array_fa_new() -> aa_double_new() - [X] replaygain.h/.c - [X] encoders/alac.h/.c - [X] encoders/flac.h/.c *** DONE array_faa -> aaa_double *** DONE array_faa_new() -> aaa_double_new() *** DONE array_o -> a_obj *** DONE array_o_new() -> a_obj_new() - [X] decoders/alac.h/.c - [X] decoders/aob.h/.c - [X] decoders/flac.h/.c - [X] encoders/wavpack.h/.c *** DONE array_i_to_FrameList -> a_int_to_FrameList - [X] decoders/flac.c - [X] decoders/oggflac.c - [X] pcmconv.c - [X] pcmconverter.c *** DONE array_ia_to_FrameList -> aa_int_to_FrameList - [X] decoders/alac.c - [X] decoders/aob.c - [X] decoders/shn.c - [X] decoders/sine.c - [X] decoders/tta.c - [X] decoders/wavpack.c - [X] pcmconv.c - [X] pcmconverter.c - [X] replaygain.c ** DONE Better document M4A atoms For each atom, indicate parent in bit diagram, tell what it's for, have a hex diagram example and show what those parsed values are. *** DONE ftyp *** DONE moov **** DONE mvhd **** DONE trak ***** DONE tkhd ***** DONE mdia ****** DONE mdhd ****** DONE hdlr ****** DONE minf ******* DONE smhd ******* DONE dinf ******** DONE dref ******* DONE stbl ******** DONE stsd ******** DONE stts ******** DONE stsc ******** DONE stsz ******** DONE stco **** DONE udta ***** DONE meta *** DONE free *** DONE mdat *** DONE Add some tree diagrams to fill sections of whitespace - [X] moov - [X] trak - [X] mdia - [X] minf - [X] stbl ** DONE Add volume control to trackplay/cdplay *** DONE Add get_volume() method to low-level audio players It should return the current set volume as a float. - [X] PulseAudioOutput - [X] OSSAudioOutput - [X] CoreAudioOutput - [X] NULLAudioOutput *** DONE Add set_volume(volume) method to low-level audio players It should take a volume setting float and adjust the output volume. - [X] PulseAudioOutput - [X] OSSAudioOutput - [X] CoreAudioOutput - [X] NULLAudioOutput *** DONE Build volume setting, progress-bar style widget *** DONE Ensure volume control updates when system volume is updated *** DONE Add widget to player widget ** DONE Make audio output selectable in trackplay/cdplay *** DONE Add available_outputs() function to audiotools.player module should return list of available AudioOutput subclasses *** DONE Add description() method to AudioOutput subclasses Returns a human-readable Unicode string indicating what the audio output device is. - [X] PulseAudioOutput - [X] OSSAudioOutput - [X] CoreAudioOutput - [X] NULLAudioOutput *** DONE Ensure output switches correctly Switches should be as seamless as possible and include setting the output volume properly. ** DONE Add track2track format conversion options Primarily for conversion between one lossless format to another. - [X] --sample-rate - [X] --channels - [X] --bits-per-sample *** DONE Add documentation to man page *** DONE Add unit tests ** DONE Add internal True Audio codec *** DONE Add Python-based True Audio decoder *** DONE Add Python-based True Audio encoder *** DONE Document True Audio decoding *** DONE Document True Audio encoding *** DONE Add C-based True Audio decoder **** DONE modify it for standalone use **** DONE ensure there's no memory errors or leaks **** DONE Make decoder thread-friendly *** DONE Add C-based True Audio encoder **** DONE modify it for standalone use **** DONE ensure there's no memory errors or leaks **** DONE make encoder thread-friendly *** DONE Add True Audio-specific unit tests *** DONE Test encoder/decoder against reference *** DONE Add True Audio detecter to file_type() function *** DONE Add True Audio AudioFile class *** DONE Ensure True Audio class passes all unit tests ** DONE Add AccurateRip support This might make give cd2track a little added respectability and post-rip tools could be built also to verify tracks/disc images. *** DONE Add accuraterip Python support **** DONE add DiscID to accuraterip module **** DONE add perform_lookup to accuraterip module **** DONE add accuraterip_lookup to __init__ module **** DONE add AccurateRipReader to accuraterip module This would be a PCMReader wrapper to be used by cd2track to verify track checksum while it's being extracted. **** DONE add AccurateRipTrackChecksum to accuraterip module This would be a transfer_framelist_data target to be used by trackverify to verify tracks already extracted. **** DONE Add programming docstrings **** DONE Add programming documentation *** DONE Add AccurateRip disc ID to cdinfo *** DONE Add support to cd2track This should be done by default as a wrapper around the existing CDDA PCMReader objects. *** DONE Add support to trackverify Add a -R/--accuraterip option which foregoes the usual calls to .verify(), performs an AccurateRip lookup for all albums and indicates whether they match/don't match/aren't found *** DONE Add protocol documentation - [X] calculating the AccurateRip disc ID - [X] fetching database entries from the server - [X] parsing database entries from the server - [X] calculating a track's CRC - [X] checking the confidence level * DONE Finish version 2.21 ** DONE Bump minimum required Python version to 2.7 Python 2.7 is pretty common these days and offers useful features that have been backported from 3. ** DONE Cleanup ProgressDisplay design Replace row_id indicators with a ProgressDisplayRow object which is created by ProgressDisplay and features update() and delete() methods that update the main ProgressDisplay. This removes the need to specify row_id values and the main ProgressDisplay can assign empty slots as needed. *** DONE Update documentation ** DONE Update temporary file management Some of the metadata functions update via tempfile.TemporaryFile then overwrite the original once the file's been completed. This would be a bad idea in the edge case of a full disk. So it may be preferable to write to a temp file alongside the original and then rename over it. That way the original wouldn't be changed until commit-time. - [X] aiff.py - [X] ape.py - [X] flac.py - [X] m4a.py - [X] tta.py - [X] wav.py *** DONE Add unit tests Ensure tracks retain permissions when metadata is updated. ** DONE Convert utilties from optparse to argparse argparse seems a bit nicer and more user-friendly since it generates usage lines automatically. - [X] audiotools-config - [X] cd2track - [X] cdinfo - [X] cdplay - [X] coverdump - [X] covertag - [X] coverview - [X] dvda2track - [X] dvdainfo - [X] track2cd - [X] track2track - [X] trackcat - [X] trackcmp - [X] trackinfo - [X] tracklength - [X] tracklint - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag - [X] trackverify *** DONE Ensure all utilities have a working --version option Now that it's no longer built-in, I'll need to add and check the option by hand. - [X] audiotools-config - [X] cd2track - [X] cdinfo - [X] cdplay - [X] coverdump - [X] covertag - [X] coverview - [X] dvda2track - [X] dvdainfo - [X] track2cd - [X] track2track - [X] trackcat - [X] trackcmp - [X] trackinfo - [X] tracklength - [X] tracklint - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag - [X] trackverify ** DONE Improve Ogg format metadata speeds Reading/writing should be faster for all Ogg-based formats. *** DONE Ogg FLAC *** DONE Ogg Vorbis *** DONE Opus ** DONE Adjust MetaData.clean() interface Returns (MetaData, [fixes]) tuple where MetaData is a new, fixed metadata object and fixes is a list of Unicode strings. *** DONE Fix MetaData objects - [X] MetaData - [X] APEv2 - [X] FlacMetaData - [X] ID3v22Comment - [X] ID3v23Comment - [X] ID3v24Comment - [X] ID3CommentPair - [X] ID3v1Comment - [X] M4A_META_Atom - [X] VorbisComment *** DONE Update documentation *** DONE Update unit tests ** DONE Adjust AudioFile.clean() interface Takes optional output_filename and returns list of fixes performed. It would be nice not to need separate code paths for dry runs, like with MetaData, but that may not be feasible. *** DONE Fix AudioFile objects - [X] AudioFile - [X] AiffAudio - [X] FlacAudio - [X] WaveAudio *** DONE Update tracklint *** DONE Update documentation *** DONE Update unit tests ** DONE Add lint for "double-wrapped" ID3v2 tags Sometimes files are idiotically wrapped in multiple layers of ID3v2 tags like nested Russian dolls. These are currently unrecognized because audiotools assumes a valid track follows the tag instead of another tag. *** DONE Support tracks nested arbitrarily deep in ID3v2 tags In order to be fixable, these files will need to be readable in the first place **** DONE Update file_type function to skip ID3v2 tags recursively It should still only find MP2/MP3/FLAC/TTA files how matter how deeply nested, but should support arbitrarily deep nesting. **** DONE Update FlacAudio ***** DONE Update FlacAudio to support nested tags This is basically a matter of making the offset as large as it takes to hold all the tags. ***** DONE Have FlacAudio.clean() remove all ID3v2 tags ***** DONE Update unit tests **** DONE Update MP3Audio ***** DONE Update MP3Audio to support nested tags ***** DONE Have MP3Audio.clean() remove all ID3v2 tags except one ***** DONE Update unit tests **** DONE Update MP2Audio ***** DONE Update MP2Audio to support nested tags ***** DONE Have MP2Audio.clean() remove all ID3v2 tags except one ***** DONE Update unit tests **** DONE Update TrueAudio ***** DONE Update TrueAudio to support nested tags ***** DONE Have TrueAudio.clean() remove all ID3v2 tags except one ***** DONE Update unit tests * DONE Finish version 2.22 ** DONE Update ReplayGain interface All ReplayGain should be lossless (if supported at all) and needs an interface that's less dependant on external tools. *** DONE Remove add_replay_gain() classmethod - [X] AudioFile - [X] ALACAudio - [X] AiffAudio - [X] AuAudio - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OggFlacAudio - [X] OpusAudio - [X] ShortenAudio - [X] TrueAudio - [X] VorbisAudio - [X] WavPackAudio - [X] WaveAudio *** DONE Add add_replay_gain() function Takes an album's worth of audiofile objects and optional progress function and if any of their classes support it (via the supports_replay_gain() classmethod) calls calculate_replay_gain() on all of them and applies the result using set_replay_gain() *** DONE Remove can_add_replay_gain() classmethod If supports_replay_gain() classmethod returns True, set_replay_gain() will be able to do something useful. - [X] AudioFile - [X] ALACAudio - [X] AiffAudio - [X] AuAudio - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OggFlacAudio - [X] OpusAudio - [X] ShortenAudio - [X] TrueAudio - [X] VorbisAudio - [X] WavPackAudio - [X] WaveAudio *** DONE Remove lossless_replay_gain classmethod All ReplayGain will be lossless - [X] AudioFile - [X] ALACAudio - [X] AiffAudio - [X] AuAudio - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OggFlacAudio - [X] OpusAudio - [X] ShortenAudio - [X] TrueAudio - [X] VorbisAudio - [X] WavPackAudio - [X] WaveAudio *** DONE Convert replay_gain() method to get_replay_gain() Returns a ReplayGain object or None, as it does now. Analagous to get_metadata() - [X] AudioFile - [X] ALACAudio - [X] AiffAudio - [X] AuAudio - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OggFlacAudio - [X] OpusAudio - [X] ShortenAudio - [X] TrueAudio - [X] VorbisAudio - [X] WavPackAudio - [X] WaveAudio *** DONE Add set_replay_gain() method Takes a ReplayGain object or None and sets the file's values by updating metadata, or deletes the values if None Analagous to set_metadata() - [X] AudioFile - [X] ALACAudio - [X] AiffAudio - [X] AuAudio - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OggFlacAudio - [X] OpusAudio - [X] ShortenAudio - [X] TrueAudio - [X] VorbisAudio - [X] WavPackAudio - [X] WaveAudio *** DONE Add delete_replay_gain() method Deletes any ReplayGain values, if any Analagous to delete_metadata() - [X] AudioFile - [X] ALACAudio - [X] AiffAudio - [X] AuAudio - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OggFlacAudio - [X] OpusAudio - [X] ShortenAudio - [X] TrueAudio - [X] VorbisAudio - [X] WavPackAudio - [X] WaveAudio *** DONE Update utilities to use new ReplayGain interface - [X] cd2track - [X] dvda2track - [X] track2track - [X] trackinfo - [X] trackplay - [X] tracksplit - [X] tracktag *** DONE Update documentation *** DONE Update unit tests ** DONE Catch interrupts more cleanly in utilities When the user cancels a long-running job, utilities should clean the screen as well as possible, delete any partial files and display a "cancelled" message instead of delivering stack traces like they do now. - [X] cd2track - [X] cdplay - [X] dvda2track - [X] track2cd - [X] track2track - [X] trackcat - [X] trackcmp - [X] tracklength - [X] trackplay - [X] tracksplit - [X] tracktag - [X] trackverify ** DONE Add AudioFile.seekable() method This returns True if the format's PCMReader has a .seek() method and that method supports some sort of fine-grained seeking (i.e. not just return to the beginning of the file). *** DONE Update audio formats - [X] AudioFile - [X] ALACAudio - [X] AiffAudio - [X] AuAudio - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OggFlacAudio - [X] OpusAudio - [X] ShortenAudio - [X] TrueAudio - [X] VorbisAudio - [X] WavPackAudio - [X] WaveAudio *** DONE Update documentation *** DONE Update utilities **** DONE tracksplit If the file being split isn't seekable, decompress it to a temp file so it can be processed across multiple jobs. **** DONE trackcmp If comparing multiple tracks against a non-seekable image, decompress it to a temp file so it can be processed across multiple jobs. *** DONE Update unit tests ** DONE Have AccurateRip detect offset tracks If the tracks have been ripped accurately but are merely offset by some small positive or negative number of samples, this should be detected and indicated in the AccurateRip results screens. *** DONE Update ChecksumV1 **** DONE Add keyword parameters with sensible defaults **** DONE Add range option This indicates the total range of the window size. Defaults to 1, indicating only a single checksum to be generated. **** DONE Change .checksum() method to .checksums() Returns a list of all calculated checksums over the whole range. *** DONE Add unit tests Ensure ranged and non-ranged checksums are calculated properly and update tests for new interface. *** DONE Build audiotools.cdio.CDDAReader object This functions like a PCMReader object over the entire CDDA device. It should work transparently on physical CD devices as well as CD images and be PCM-frame centric by doing all the CD sector conversions behind-the-scenes. **** DONE Add methods and attributes ***** DONE CDDAReader.sample_rate ***** DONE CDDAReader.channels ***** DONE CDDAReader.channel_mask ***** DONE CDDAReader.bits_per_sample ***** DONE CDDAReader.track_offsets - [X] CD image - [X] CD drive Returns starting offset of each track, in PCM frames ***** DONE CDDAReader.track_lengths - [X] CD image - [X] CD drive Returns length of each track, in PCM frames ***** DONE CDDAReader.first_sector - [X] CD image - [X] CD drive Used for calculating various disc IDs ***** DONE CDDAReader.last_sector - [X] CD image - [X] CD drive Used for calculating various disc IDs ***** DONE CDDAReader.is_image - [X] CD drive - [X] CD image ***** DONE CDDAReader.read(pcm_frames) - [X] CD image - [X] CD drive ***** DONE CDDAReader.seek(pcm_frames) - [X] CD image - [X] CD drive ***** DONE CDDAReader.set_speed(speed) - [X] CD image - [X] CD drive ***** DONE CDDAReader.log() - [X] CD image - [X] CD drive ***** DONE CDDAReader.reset_log() ***** DONE CDDAReader.close() **** DONE Update documentation **** DONE Update unit tests **** DONE Detect nonexistent discs CDDAReader should detect when a physical drive doesn't contain a disc and raise an appropriate exception. **** DONE Update utilities ***** DONE cd2track ***** DONE cdinfo ***** DONE cdplay **** DONE Clean out old audiotools.CDDA object Remove outdated interface, its documentation and unit tests. *** DONE Update utilities When verifying AccurateRip, make sure to populate the checksum with the last few samples of the previous track (if any) and the next few samples of the next track (if any) in order to find the checksum's offset. **** DONE trackverify **** DONE cd2track ** DONE Simplify lookup services interface The perform_lookup() functions should work on DiscID objects and DiscID objects should be generated from CDDAReaders, track lists or cuesheet/length combinations. *** DONE FreeDB - [X] Add from_tracks classmethod - [X] Add from_sheet classmethod - [X] Add unit tests *** DONE MusicBrainz - [X] Add from_tracks classmethod - [X] Add from_sheet classmethod - [X] Add unit tests *** DONE AccurateRip - [X] Add from_tracks classmethod - [X] Add from_sheet classmethod - [X] Add unit tests ** DONE Add documentation for lookup services - [X] audiotools.freedb - [X] audiotools.musicbrainz - [X] audiotools.accuraterip ** DONE Convert TOC and CUE handling to proper grammars *** DONE Convert audiotools.cue *** DONE Convert audiotools.toc ** DONE Add AccurateRipV2 support *** DONE Add ChecksumV2 object **** DONE Add unit tests *** DONE Update utilities - [X] cd2track - [X] trackverify ** DONE Update Sheet interface Although cuesheets are mostly a repository of index points, it would be useful to support a wider subset of options when converting one form to another. *** DONE Sheet - [X] @classmethod converted(sheet) -> Sheet builds Sheet object from Sheet-compatible object - [X] __len__() -> total_tracks - [X] __getitem__(track_index) -> SheetTrack - [X] __eq__(sheet) -> boolean - [X] track_numbers() -> [int] - [X] track(track_number) -> SheetTrack - [X] get_metadata() -> MetaData MetaData contains catalog_number and CD-TEXT information *** DONE SheetTrack - [X] @classmethod converted(sheet_track) -> SheetTrack builds SheetTrack object from SheetTrack-compatible object - [X] __len__() -> total_indexes - [X] __getitem__(index_index) -> SheetIndex - [X] __eq__(sheet_track) -> boolean - [X] indexes() -> [int] - [X] index(index_number) -> SheetIndex - [X] filename() -> string - [X] number() -> track_number - [X] get_metadata() -> MetaData MetaData contains track_number, ISRC and CD-TEXT information - [X] is_audio() -> boolean - [X] pre_emphasis() -> boolean - [X] copy_peritted() -> boolean *** DONE SheetIndex - [X] @classmethod converted(sheet_index) -> SheetIndex - [X] __eq__(sheet_index) -> boolean - [X] number() -> index_number pre-gap is index_number 0 - [X] offset() -> Fraction offset of index in seconds from start of stream *** DONE Update audiotools.cue.Cuesheet *** DONE Update audiotools.toc.TOCFile *** DONE Update audiotools.flac.Flac_CUESHEET *** DONE Ensure non-image cuesheets are handled properly Given a non-image cuesheet and list of track lengths, it may be possible to convert a non-image cuesheet to an image cuesheet. *** DONE Handle 1st track pre-gap correctly Discs with a pre-gap before the 1st track should have that gap removed when split and re-added when concatenating them back together based on the cuesheet contents. - [X] update tracksplit - [X] update trackcat - [X] add unit tests *** DONE Update utilities - [X] trackcat - [X] tracksplit - [X] track2cd - [X] trackinfo *** DONE Update functions/methods which take cuesheets - [X] sheet_metadata_lookup - [X] accuraterip_sheet_lookup *** DONE Update documentation *** DONE Update unit tests ** DONE Converts AudioFile.seconds_length() method to Fraction This should be more useful than the Decmial it was using before. *** DONE Update method *** DONE Update documentation *** DONE Update utilities - [X] track2cd - [X] trackcat - [X] trackinfo - [X] tracklength - [X] trackplay - [X] tracksplit * DONE Finish version 3.0 ** DONE Make compatible with Python 2.7 and 3.X Like Urwid and PLY, the same code should work on both versions. *** DONE Update setup script *** DONE Update C-based extensions Use #ifdefs as needed to support both Python versions. - [X] audiotools.pcm - [X] audiotools.pcmconverter - [X] audiotools.replaygain - [X] audiotools.decoders - [X] audiotools.encoders - [X] audiotools.bitstream - [X] audiotools._ogg - [X] audiotools._accuraterip - [X] audiotools.output *** DONE Update Python modules **** DONE audiotools - [X] __init__ - [X] accuraterip - [X] aiff - [X] ape - [X] au - [X] dvda - [X] flac - [X] freedb - [X] id3 - [X] id3v1 - [X] image - [X] m4a_atoms - [X] m4a - [X] mp3 - [X] musicbrainz - [X] ogg - [X] opus - [X] player - [X] shn - [X] text - [X] tta - [X] ui - [X] vorbiscomment - [X] vorbis - [X] wavpack - [X] wav **** DONE audiotools.cue - [X] __init__ - [X] tokrules - [X] yaccrules **** DONE audiotools.toc - [X] __init__ - [X] tokrules - [X] yaccrules *** DONE Update utilities - [X] audiotools-config - [X] cd2track - [X] cdinfo - [X] cdplay - [X] coverdump - [X] covertag - [X] coverview - [X] dvda2track - [X] dvdainfo - [X] track2cd - [X] track2track - [X] trackcat - [X] trackcmp - [X] trackinfo - [X] tracklength - [X] tracklint - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag - [X] trackverify *** DONE Update Python-based decoders - [X] alac - [X] flac - [X] shn - [X] tta - [X] wavpack *** DONE Update Python-based encoders - [X] alac - [X] flac - [X] shn - [X] tta - [X] wavpack *** DONE Update documentation *** DONE Update unit tests - [X] test - [X] test_core - [X] test_formats - [X] test_metadata - [X] test_streams - [X] test_utils **** TODO Update unit test utilities - [ ] error.py - [ ] test_cdrdao - [ ] test_cdrecord *** DONE Ensure unit tests pass **** DONE Python 2.7 ***** DONE Lib - [X] core - [X] cuesheet - [X] cdio - [X] freedb - [X] image - [X] musicbrainz - [X] accuraterip - [X] pcm - [X] bitstream - [X] replaygain - [X] resample - [X] tocfile - [X] verify - [X] ogg ***** DONE Format - [X] audiofile - [X] lossless - [X] lossy - [X] aiff - [X] alac - [X] au - [X] dvda - [X] flac - [X] m4a - [X] mp2 - [X] mp3 - [X] oggflac - [X] opus - [X] shorten - [X] sines - [X] tta - [X] vorbis - [X] wave - [X] wavpack ***** DONE Metadata - [X] metadata - [X] flac - [X] wavpack - [X] id3v1 - [X] id3v2 - [X] vorbis - [X] opus - [X] m4a - [X] tta ***** DONE Util - [X] audiotools-config - [X] cd2track - [ ] cdinfo - [X] cdplay - [X] coverdump - [X] covertag - [X] coverview - [X] dvda2track - [X] dvdainfo - [X] track2cd - [X] track2track - [X] trackcat - [X] trackcmp - [X] trackinfo - [X] tracklength - [X] tracklint - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag - [X] trackverify **** DONE Python 3 ***** DONE Lib - [X] core - [X] cuesheet - [X] cdio - [X] freedb - [X] image - [X] musicbrainz - [X] accuraterip - [X] pcm - [X] bitstream - [X] replaygain - [X] resample - [X] tocfile - [X] verify - [X] ogg ***** DONE Format - [X] audiofile - [X] lossless - [X] lossy - [X] aiff - [X] alac - [X] au - [X] dvda - [X] flac - [X] m4a - [X] mp2 - [X] mp3 - [X] oggflac - [X] opus - [X] shorten - [X] sines - [X] tta - [X] vorbis - [X] wave - [X] wavpack ***** DONE Metadata - [X] metadata - [X] flac - [X] wavpack - [X] id3v1 - [X] id3v2 - [X] vorbis - [X] opus - [X] m4a - [X] tta ***** DONE Util - [X] audiotools-config - [X] cd2track - [X] cdinfo - [X] cdplay - [X] coverdump - [X] covertag - [X] coverview - [X] dvda2track - [X] dvdainfo - [X] track2cd - [X] track2track - [X] trackcat - [X] trackcmp - [X] trackinfo - [X] tracklength - [X] tracklint - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag - [X] trackverify ** DONE Update interactive utilities Ensure they run under Python 2 and 3 - [X] audiotools/ui.py - [X] audiotools-config - [X] cd2track - [X] cdplay - [X] dvda2track - [X] track2track - [X] trackcat - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag ** DONE Calculate ReplayGain at rip-time For utilities that operated single-threaded, calculate ReplayGain during operation rather than put off until ripping is done. *** DONE cd2track *** DONE dvda2track ** DONE Re-add DVD-A support using libdvdaudio Supporting DVD-A in an external library should provide a useful decoupling. *** DONE Add audiotools.dvda module This contains thin wrappers around libdvdaudio structures and functions. **** DONE Add DVDA object **** DONE Add Titleset object **** DONE Add Title object **** DONE Add Track object **** DONE Add TrackReader object **** DONE Add documentation *** DONE Re-add dvda2track script **** DONE Re-add dvda2track.1 man page *** DONE Re-add dvdainfo script **** DONE Re-add dvdainfo.1 man page *** DONE Update setup script Probe for libdvdaudio and add module and scripts if present, analagous to libcdio. * DONE Finish version 3.1 ** DONE Make AudioFile.available() classmethod finer-grained That is, formats can be readable but not writable, or taggable but not readable or writeable. This will allow read-only support for some obscure formats like .tak or .ape by porting implementations from ffmpeg but without having to build encoders for them as well. *** DONE Add AudioFile.supports_to_pcm() - [X] AudioFile - [X] ALACAudio - [X] AiffAudio - [X] AuAudio - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OpusAudio - [X] ShortenAudio - [X] TrueAudio - [X] VorbisAudio - [X] WavPackAudio - [X] WaveAudio *** DONE Add AudioFile.supports_from_pcm() - [X] AudioFile - [X] ALACAudio - [X] AiffAudio - [X] AuAudio - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OpusAudio - [X] ShortenAudio - [X] TrueAudio - [X] VorbisAudio - [X] WavPackAudio - [X] WaveAudio *** DONE Add AudioFile.supports_metadata() *** DONE Remove AudioFile.missing_components() ? Leave this job to setup.py and audiotools-config, perhaps. *** DONE Utilities **** DONE audiotools-config Needs to display format availability in a more fine-grained fashion. **** DONE cd2track - [X] output format must supports_from_pcm() **** DONE track2cd - [X] input format must supports_to_pcm() **** DONE track2track - [X] input format(s) must supports_to_pcm() - [X] output format must supports_from_pcm() **** DONE trackcat - [X] input format(s) must supports_to_pcm() - [X] output format must supports_from_pcm() **** DONE trackcmp - [X] input format(s) must supports_to_pcm() **** DONE trackplay - [X] input format(s) must supports_to_pcm() **** DONE tracksplit - [X] input file must supports_to_pcm() - [X] output format must supports_from_pcm() **** DONE trackverify - [X] input format must supports_to_pcm() *** DONE Update documentation *** DONE Update unit tests * DONE Add compilation field to metadata This boolean field indicates the track is part of a compilation and should round-trip correctly across all metadata types that support it. ** DONE Replace MetaData.INTEGER_FIELDS with a FIELD_TYPE lookup This allows a field to be either a Unicode string (like track_name) integer field (like track_number) or a boolean (like compilation). ** DONE Add field to metadata classes *** DONE Add compilation field to MetaData class *** DONE Add compilation field to ID3v2 classes - [X] ID3v2.2 - [X] ID3v2.3 - [X] ID3v2.4 - [X] ID3CommentPair *** DONE Add compilation field to VorbisComment classes - [X] VorbisComment - [X] FlacMetaData *** DONE Add compilation field to M4A metadata *** DONE Add compilation field to APEv2 metadata ** DONE Update tracktag to support compilation field - [X] add command-line options - [X] add UI support *** TODO Update man page ** DONE Update documentation ** DONE Update unit tests - [X] format unit tests - [X] tracktag unit tests * TODO Add support for genre tag? I'm not a big fan of the genre tag. Unlike track number, album name, ISRC, etc. in which a value can be reliably determined from the source material (e.g. back of the CD), genre is akin to a "rating" tag. Its value varies from person to person and this makes it less valuable for archival purposes. In addition, the genre tag itself is implemented in incompatible ways. APEv2 uses a chunk of text, ID3v1 uses an integer representing one of many designated genre labels, ID3v2 uses a mix of genre byte and/or text string, and so on. That said, the genre field shows up in a lot of players. So, some grudging support for it would probably be appreciated. * TODO Replace magic numbers with named constants There's still a few instances of magic numbers in use, in the __flac__.py module, for instance. * TODO Support disc metadata submission If a disc is not found on FreeDB or MusicBrainz, there should be some mechanism to submit user-defined data to those services. * TODO build trackverify utility Much like flac(1)'s --verify option, this should check a file for correctness and, if possible, verify its output matches any internal hashes/checksums. May work recursively to check a user's entire collection. ** DONE add a .verify() method to AudioFile classes Returns True if the file is okay. Raises InvalidFile if not. Include unit test. - [X] AiffAudio - [X] ALACAudio - [X] AuAudio - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] OggFlacAudio - [X] SpeexAudio - [X] ShortenAudio - [X] VorbisAudio - [X] WaveAudio - [X] WavPackAudio ** TODO add trackverify unit tests for all known verifiable problems ** DONE add trackverify man page ** DONE link trackverify man page to others * TODO Add a -r/--no-results flag to trackcmp/trackverify Listing only the final results and not the running list ** TODO Update man page * TODO Add DVD-A support ** DONE Build AOB parser/extractor *** DONE Optimize AOB extractor for better offset handling The current version works, but is a little too naive ** TODO Build CPPM handler *** DONE Create audiotools.prot module *** TODO Create audiotools.prot.CPPMDecoder object **** DONE Add CPPMDecoder.__init__() method **** DONE Add CPPMDecoder.decode() method **** TODO Ensure CPPMDecoder is cross-platform ** DONE Document AOB parser ** DONE Build raw PCM decoder The PCM data doesn't appear to be stored raw, so some additional calculation will likely be necessary ** TODO Build MLP decoder *** DONE Create decoders.MLPDecoder class **** DONE Add MLPDecoder.init() method **** DONE Add MLPDecoder.sample_rate attribute **** DONE Add MLPDecoder.channels attribute **** DONE Add MLPDecoder.bits_per_sample attribute **** DONE Add MLPDecoder.channel_mask attribute **** DONE Add MLPDecoder.read() method **** DONE Add MLPDecoder.analyze_frame() method **** DONE Add MLPDecoder.close() method *** DONE Perform MLP frame parsing **** DONE Perform frame size parsing **** DONE Perform major sync parsing **** DONE Perform substream sizes parsing **** DONE Perform substream datas parsing ***** DONE Perform block parsing ****** DONE Perform restart header parsing ****** DONE Perform decoding parameters parsing ******* DONE Perform parameter presence flags parsing ******* DONE Perform block size parsing ******* DONE Perform matrix parameters parsing ******* DONE Perform output shifts parsing ******* DONE Perform quant step sizes parsing ******* DONE Perform channel parameters parsing ******** DONE Perform FIR filter parameters parsing ******** DONE Perform IIR filter parameters parsing ******** DONE Perform Huffman offset parsing ******** DONE Perform Codebook parsing ******** DONE Perform Huffman least-significant bits parsing ****** DONE Perform block data parsing ***** DONE Perform block alignment ***** DONE Perform CRC parsing *** TODO Perform MLP data decoding **** DONE Handle FIR/IIR filtering **** DONE Handle matrix reconstruction ***** DONE Build noise channels **** DONE Handle quant_step_sizes **** DONE Combine multiple substreams into single output frame **** DONE Handle output shifts **** TODO Handle checksums ***** TODO restart header lossless check ***** DONE end-of-frame parity ***** DONE end-of-frame checksum **** DONE Build pcm.FrameList from PCM data arrays ***** DONE Reorder MLP channels to wave order **** DONE Add bs_try for proper EOF aborts **** DONE Add end-of-stream marker checking to read() **** TODO Add end-of-stream marker checking to analyze_frame() **** DONE Optimize to work faster Perhaps with multiple blocks per call to read() **** TODO Support 1 substream output MLP decoding supports 1 or 2 substreams. In some cases, substream 1 will contain the first 2 channels while substream 2 will contain the rest, which provides two different "mixes" depending on whether one is decoding to 2 channels or not. ***** TODO Update dvda2track with 1 substream output option This would allow the user to decide which "mix" is desired during extraction. *** DONE Perform sanity checks ** TODO Document MLP decoder *** DONE major sync *** DONE substream size *** DONE substream data *** DONE restart header *** DONE decoding parameters **** DONE parameter presence flags **** DONE block size **** DONE matrix parameters **** DONE output shifts **** DONE quant step sizes *** DONE checksums *** DONE block data *** DONE channel filtering *** DONE channel rematrixing **** DONE noise channels **** DONE rematrixing *** DONE end of stream marker *** TODO decoding XOR byte ** DONE Document PCM decoder ** DONE Link MLP decoder to AOB extractor ** DONE Link PCM decoder to AOB extractor ** TODO Build DVD-A postprocessor ** DONE Document Python interface ** DONE Build dvdainfo utility *** DONE Add dvdainfo.1 man page *** DONE Ensure -A with invalid dir generates error message ** DONE Build dvda2track utility *** DONE Fix to work without -d *** DONE Add --track-start / --track-total options Since some especially long discs split tracks across titles, it makes sense to have a way to specify the starting track number and the total number of tracks. **** DONE Add options to man page *** DONE Add dvda2track.1 man page *** DONE Ensure -A with invalid dir generates error message ** DONE Build dvda2xmcd utility Querying FreeDB and MusicBrainz are possible, but I expect most of the time they'll generate empty templates. *** DONE Add dvda2xmcd.1 man page *** DONE Ensure -A with invalid dir generates error message * TODO Add cdplay utility Should work similarly to trackplay, taking track number(s) and playing them to OSS/PulseAudio or whatever other audio output is available. ** DONE Add audiotools.player.CDPlayer class *** DONE Add docstrings *** DONE Add programming docs ** DONE Add audiotools.player.CDPlayerThread class *** DONE Add docstrings ** DONE Add non-interactive mode to cdplay ** DONE Add interative mode to cdplay Urwid-based display should have various standard features - [X] display CD info - [X] progress monitor - [X] start - [X] pause - [X] next track - [X] previous track ** DONE Silence CD query messages ** DONE Support command-line tracks One might want to play only a subset of the whole CD ** DONE Add -V/--verbose option ** DONE Add --shuffle option ** DONE Add -x/--xmcd file support ** DONE Avoid spinning down CDs between tracks This makes playback less seamless than I'd like ** TODO Improve stability ** DONE Add man page for cdplay.1 ** TODO Add basic unit tests - [ ] Check command-line arguments * TODO Reorganize test suite The current haphazard layout makes it too difficult to deterimine if some feature has been fully unit tested. Or, if a new feature is added, the logical place to put new tests is not obvious. ** TODO Group tests for the audio formats *** DONE AudioFile - [X] test_init - [X] test_is_type - [X] test_bits_per_sample - [X] test_metadata - [X] test_length - [X] test_track_number - [X] test_album_number - [X] test_track_name - [X] test_replay_gain *** DONE LosslessAudioFile - [X] test_lossless - [X] test_channels - [X] test_channel_mask - [X] test_sample_rate - [X] test_pcm - [X] test_convert *** DONE LossyAudioFile - [X] test_bits_per_sample - [X] test_lossless - [X] test_channels - [X] test_channel_mask - [X] test_sample_rate - [X] test_pcm - [X] test_convert *** DONE ALACAudio - [X] test_init - [X] test_bits_per_sample - [X] test_channel_mask - [X] test_verify - [X] test_streams - [X] test_small_files - [X] test_full_scale_deflection - [X] test_sines - [X] test_wasted_bps - [X] test_blocksizes - [X] test_noise - [X] test_fractional - [X] test_frame_header_variations *** TODO AiffAudio - [ ] test_init - [X] test_channel_mask - [X] test_verify - [X] test_roundtrip_aiff_chunks - [X] test_convert_aiff_chunks *** TODO AuAudio - [ ] test_init - [X] test_verify - [X] test_channel_mask *** TODO FlacAudio - [ ] test_init - [ ] test_metadata2 - [X] test_verify - [ ] test_cuesheet - [X] test_roundtrip_aiff_chunks - [X] test_convert_aiff_chunks - [X] test_roundtrip_wave_chunks - [X] test_convert_wave_chunks - [X] test_streams - [X] test_small_files - [X] test_full_scale_deflection - [X] test_sines - [X] test_wasted_bps - [X] test_blocksizes - [X] test_frame_header_variations - [X] test_option_variations - [X] test_noise - [X] test_fractional *** TODO M4AAudio - [ ] test_init - [ ] test_verify *** TODO MP2Audio - [ ] test_init - [X] test_length - [ ] test_verify *** TODO MP3Audio - [ ] test_init - [X] test_length - [ ] test_verify *** TODO OggFlacAudio - [ ] test_init - [X] test_verify - [ ] test_cuesheet *** TODO ShortenAudio - [ ] test_init - [X] test_bits_per_sample - [X] test_verify - [X] test_roundtrip_wave_chunks - [X] test_convert_wave_chunks - [X] test_streams - [X] test_small_files - [X] test_full_scale_deflection - [X] test_sines - [X] test_blocksizes - [X] test_noise *** TODO VorbisAudio - [ ] test_init - [X] test_channels - [X] test_verify *** TODO WavPackAudio - [ ] test_init - [X] test_verify - [ ] test_cuesheet - [X] test_roundtrip_wave_chunks - [X] test_convert_wave_chunks - [X] test_small_files - [X] test_full_scale_deflection - [X] test_wasted_bps - [X] test_blocksizes - [X] test_silence - [X] test_noise - [X] test_fractional - [X] test_multichannel - [X] test_sines - [X] test_option_variations *** TODO WaveAudio - [ ] test_init - [X] test_verify - [X] test_roundtrip_wave_chunks - [X] test_convert_wave_chunks ** DONE Group tests for the metadata formats *** DONE ApeTag *** DONE FLAC *** DONE ID3v1 *** DONE ID3v2.2 *** DONE ID3v2.3 *** DONE ID3v2.4 *** DONE IDCommentPair *** DONE M4A *** DONE VorbisComment *** DONE Ensure that merge() works properly It has to handle all supported text fields *and* must handle images if supported. ** TODO Group tests for the core libraries *** TODO audiotools.__init__ **** TODO AlbumMetaData **** DONE BufferedPCMReader **** DONE CDDA **** DONE ChannelMask **** DONE Image **** DONE LimitedPCMReader **** TODO Messenger **** DONE PCMCat **** DONE PCMConverter **** DONE PCMReaderWindow **** TODO ReorderedPCMReader **** DONE ReplayGain **** DONE applicable_replay_gain **** DONE at_a_time **** DONE build_timestamp **** DONE calculate_replay_gain **** DONE filename_to_type **** DONE group_tracks **** TODO make_dirs **** DONE open **** DONE open_directory **** DONE open_files **** DONE parse_timestamp **** DONE pcm_frame_cmp **** DONE pcm_split **** DONE read_sheet **** DONE str_width **** DONE transfer_framelist_data **** DONE XMCD **** DONE MusicBrainzReleaseXML *** DONE audiotools.pcm **** DONE FloatFrameList **** DONE FrameList **** DONE from_channels **** DONE from_float_Frames **** DONE from_float_channels **** DONE from_frames **** DONE from_list *** DONE audiotools.player **** DONE Player **** DONE CDPlayer *** DONE audiotools.replaygain **** DONE ReplayGain **** DONE ReplayGainReader *** DONE audiotools.decoders **** DONE BitstreamReader *** DONE audiotools.encoders **** DONE BitstreamWriter *** TODO audiotools.verify **** TODO mpeg **** TODO ogg ** TODO Group tests for the individual tools *** TODO audiotools-config **** TODO test_options **** TODO test_errors *** DONE cd2track **** DONE test_options **** DONE test_errors *** DONE cd2xmcd **** DONE test_options **** DONE test_errors *** TODO cdinfo **** TODO test_options **** TODO test_errors *** TODO cdplay **** TODO test_options **** TODO test_errors *** DONE coverdump **** DONE test_options **** DONE test_errors *** DONE dvdainfo **** DONE test_errors *** DONE dvda2track **** DONE test_errors *** DONE dvda2xmcd **** DONE test_errors *** DONE track2track **** DONE test_options **** DONE test_errors *** DONE track2xmcd **** DONE test_options **** DONE test_errors *** DONE trackcat **** DONE test_options *** DONE trackcmp **** DONE test_options *** TODO trackinfo **** TODO test_options **** TODO test_errors *** DONE tracklength *** DONE tracklint *** TODO trackplay **** TODO test_options **** TODO test_errors *** DONE trackrename **** DONE test_options **** DONE test_errors *** DONE tracksplit **** DONE test_options **** DONE test_errors *** TODO tracktag **** DONE test_options **** TODO test_errors *** TODO trackverify **** TODO test_options **** TODO test_errors ** DONE Split tests into multiple files * TODO Handle write errors more gracefully Just as the BitstreamReader uses an exception mechanism to detect and handle read errors, the BitstreamWriter should use a similar mechanism to detect write errors and bubble-up a Python exception. The problem is that invalid writes are difficult to unit test. With a reader, one can simply truncate a valid file. It's harder to constrict a writer to only have X number of bytes available so that any errors can be checked. ** TODO Add try/etry support to BitstreamWriter ** DONE Update BitstreamWriter methods to raise exceptions ** TODO Update Python-based BitstreamWriter to catch C exceptions ** TODO Update encoders to catch C exceptions and raise Python exceptions ** TODO Unit test write errors * TODO Verify metadata against reference implementations Where feasible, try to ensure our metadata matches what's read/written by known reference implementations. ** TODO FlacAudio against metaflac ** TODO M4AAudio/ALACAudio against taglib? ** TODO MP3Audio/MP2Audio Against taglib, perhaps? Venerable id3lib doesn't work with UCS-2. ** TODO VorbisAudio against vorbis-tools ** TODO WavPackAudio against tablib? * TODO Add lyrics metadata support Supporting storing lyrics text in the audio files themselves would be quite useful. ** TODO Add .lyrics field to MetaData Analagous to the lengthy .comment field, most likely. ** TODO Add unit testing for .lyrics field ** TODO Find automated source for lyrics Analagous to CDDB. This will be challenging since online sources for lyrics are rare. Unfortunately, expecting everyone to transcribe lyrics for songs is unrealistic - even moreso than filling in song names or scanning in covers by hand. If it's too much work, there's little point in having it. ** TODO Add tool support for lyrics ** TODO Add unit testing to tools * TODO Add a dvdaplay utility ** TODO Unify trackplay/cdplay/dvdaplay widgets into audiotools.ui ** TODO Unify non-Urwid player into audiotools.ui ** TODO Add man page * TODO Make internal functions static Encoders and decoders come with lots of helper functions. These should be defined statically whenever possible to avoid colliding with one another. As a side effect, these function names can now be shortened. (not applicable to FLAC and Ogg FLAC, which share a lot of functions) ** TODO decoders - [X] src/decoders/alac.c - [ ] src/decoders/aobpcm.c - [ ] src/decoders/mlp.c - [ ] src/decoders/ogg.c - [ ] src/decoders/shn.c - [ ] src/decoders/sine.c - [ ] src/decoders/vorbis.c - [X] src/decoders/wavpack.c ** TODO encoders - [X] src/encoders/alac.c - [ ] src/encoders/flac.c - [ ] src/encoders/shn.c - [X] src/encoders/wavpack.c * TODO handle AIFF offset/block size in SSND chunk * TODO Assimilate external codecs More of a long-term plan than anything else. It would be best to have as few external program dependencies as possible ** TODO Add internal Monkey's Audio codec This needs to be assimilated with a native decoder/encoder. I've seen these out in the wild, but only rarely. *** TODO Add Python-based Monkey's Audio decoder *** TODO Document Monkey's Audio decoding *** TODO Add C-based Monkey's Audio decoder *** TODO Add Python-based Monkey's Audio encoder *** TODO Document Monkey's Audio encoding *** TODO Add C-based Monkey's Audio encoder *** TODO Add Monkey's Audio-specific unit tests *** TODO Test encoder/decoder against reference *** TODO Restore AudioFile-compatible Python interface ** DONE Add context manager support Having file-like objects work with "with" context managers should hopefully make file handle management a bit simpler. *** DONE BitstreamReader Call .close() on internal stream, eat any exception (like the "file" object does) and return a False value **** DONE Add documentation **** DONE Add unit tests *** DONE BitstreamWriters If no exception, call .flush() on internal stream and eat any exception. Then call .close() on internal stream and eat any exception also. Returns a False value. **** DONE Add documentation **** DONE Add unit tests *** DONE PCMReaders Call .close() and eat any exception (like the "file" object does). - [X] aiff/AIFFReader - [X] au/AuReader - [X] audiotools/BufferedPCMReader - [X] audiotools/CounterPCMReader - [X] audiotools/LimitedPCMReader - [X] audiotools/PCMCat - [X] audiotools/PCMReader - [X] audiotools/PCMReaderDeHead - [X] audiotools/PCMReaderError - [X] audiotools/PCMReaderHead - [X] audiotools/PCMReaderProgress - [X] audiotools/ReorderedPCMReader - [X] decoders/ALACDecoder - [X] decoders/DVDA_Title - [X] decoders/FlacDecoder - [X] decoders/MP3Decoder - [X] decoders/OggFlacDecoder - [X] decoders/OpusDecoder - [X] decoders/SHNDecoder - [X] decoders/SameSample - [X] decoders/Sine_Mono - [X] decoders/Sine_Simple - [X] decoders/Sine_Stereo - [X] decoders/TTADecoder - [X] decoders/VorbisDecoder - [X] decoders/WavPackDecoder - [X] player/ThreadedPCMReader - [X] py_decoders/ALACDecoder - [X] py_decoders/FlacDecoder - [X] py_decoders/SHNDecoder - [X] py_decoders/TTADecoder - [X] py_decoders/WavPackDecoder - [X] test/EXACT_RANDOM_PCM_Reader - [X] test/Join_Reader - [X] test/MD5_Reader - [X] test/RANDOM_PCM_Reader - [X] test/Variable_Reader - [X] test_formats/CLOSE_PCM_Reader - [X] test_formats/ERROR_PCM_Reader - [X] test_streams/FrameListReader - [X] test_streams/MD5Reader - [X] test_streams/Raw - [X] test_streams/Simple_Sine - [X] test_streams/WastedBPS16 - [X] wav/WaveReader **** DONE Add documentation **** DONE Add unit tests * TODO Finish version 3.2 ** TODO Add better metadata merge strategies These could be used by trackcat and others to combine MetaData objects in a useful way. *** DONE Add metadata1.intersection(metadata2) -> metadata3 method metadata3 contains fields that are in both metadata1 and metadata2 and contain the same values if both metadata objects are the same type, returns metadata in that type and optimize the merge to handle low-level metadata fields otherwise returns a generic MetaData object **** DONE Update metadata implementations - [X] MetaData - [X] ApeTag - [X] FlacMetaData - [X] OggFlacMetaData - [X] ID3v22Comment - [X] ID3v23Comment - [X] ID3v24Comment - [X] ID3CommentPair - [X] ID3v1Comment - [X] M4A_META_Atom - [X] VorbisComment **** DONE Add unit tests These should check that foreign fields are merged correctly, if applicable. - [X] MetaData - [X] ApeTag - [X] FlacMetaData - [X] ID3v22Comment - [X] ID3v23Comment - [X] ID3v24Comment - [X] ID3v1Comment - [X] M4A_META_Atom - [X] VorbisComment **** DONE Update documentation *** TODO Update trackcat to use .intersection() method **** TODO Add unit tests ** TODO Handle pre-gaps that contain audio data *** DONE Update cd2track If pre-gap is present, check it for data. If data is present, dump data to a "track 0" file with as much metadata as possible. If data is not present, skip it. **** DONE Add unit tests *** TODO Update track2cd If pre-gap is present, check for a "track 0" file of the same length. If track is present, prepend is to stream before writing to disc. If track is not present, prepend empty data to disc. **** TODO Add unit tests *** DONE Update tracksplit If pre-gap is present, check it for data. If data is present, dump data to a "track 0" file with as much metadata as possible. If data is not present, skip it. **** DONE Add unit tests *** DONE Update trackcat **** DONE With cue sheet If pre-gap is present, check for a "track 0" file of the same length. If track is present, prepend is to stream before writing to image. If track is not present, prepend empty data to image. ***** DONE Add unit tests **** DONE Without cue sheet If a track number is 0, assume it's a pre-gap track with data of that length and prepend it to stream before writing to image. ***** DONE Add unit tests *** DONE Update Sheet.from_tracks() If first track number is 0, assume it's a pre-gap with data when building disc ID **** DONE Add unit tests *** DONE Update musicbrainz.DiscID.from_tracks() If first track number is 0, assume it's a pre-gap with data when building disc ID **** DONE Add unit tests *** DONE Update freedb.DiscID.from_tracks() If first track number is 0, assume it's a pre-gap with data when building disc ID **** DONE Add unit tests *** DONE Update accuraterip.DiscID.from_tracks() If first track number is 0, assume it's a pre-gap with data when building disc ID **** DONE Add unit tests *** TODO Update trackverify **** TODO Add unit tests ** DONE Convert old string formatting to new .format()-based formatting That is, instead of using: "foo %s bar" % (argument) it's better to use "foo {} bar".format(argument) since the new format method is recommended for new code and the old one is somewhat clunky and not as powerful. Since the old format sync is baked into some areas of the code (like the --format utility flags) it'll take longer to remove than from elsewhere. See: [[https://docs.python.org/3/library/string.html#formatstrings][Format String Syntax]] for more info. *** DONE Update audiotools.text entries *** DONE Update setup.py *** DONE Update utilities - [X] audiotools-config - [X] cdda2track - [X] cddainfo - [X] cddaplay - [X] coverdump - [X] covertag - [X] coverview - [X] dvda2track - [X] dvdainfo - [X] track2cdda - [X] track2track - [X] trackcat - [X] trackcmp - [X] trackinfo - [X] tracklength - [X] tracklint - [X] trackplay - [X] trackrename - [X] tracksplit - [X] tracktag - [X] trackverify *** DONE Update modules - [X] __init__.py - [X] accuraterip.py - [X] aiff.py - [X] ape.py - [X] au.py - [X] cdtoc.py - [X] coverartarchive.py - [X] flac.py - [X] freedb.py - [X] id3.py - [X] id3v1.py - [X] image.py - [X] m4a.py - [X] m4a_atoms.py - [X] mp3.py - [X] mpc.py - [X] musicbrainz.py - [X] ogg.py - [X] opus.py - [X] player.py - [X] tta.py - [X] ui.py - [X] vorbis.py - [X] vorbiscomment.py - [X] wav.py - [X] wavpack.py **** DONE cue - [X] __init__.py - [X] tokrules.py - [X] yaccrules.py **** DONE toc - [X] __init__.py - [X] tokrules.py - [X] yaccrules.py *** DONE Update unit tests - [X] test.py - [X] test_core.py - [X] test_formats.py - [X] test_metadata.py - [X] test_streams.py - [X] test_utils.py ** TODO Support MBID tags These are basically direct MusicBrainz links rather than using calculated disc IDs to perform lookups. *** TODO Update MetaData formats to perform MBID lookups, if possible - [ ] ApeTag - [ ] FlacMetaData - [ ] ID3CommentPair - [ ] ID3v1Comment - [ ] ID3v22Comment - [ ] ID3v23Comment - [ ] ID3v24Comment - [ ] M4A - [ ] VorbisComment **** TODO Add unit tests *** TODO Update musicbrainz module to handle MBID lookup Given an ID, returns a list of MetaData objects **** TODO Add unit tests *** TODO Update utilities to support MBID lookup Basically anything that takes tracks and performs a lookup may benefit. - [ ] tracksplit - [ ] trackcat - [ ] track2track - [ ] tracktag **** TODO Add unit tests ** TODO Add pre-emphasis support *** TODO Add de-emphasis filter to pcmconverters **** TODO Add unit test *** TODO Add pre-emphasis filter to pcmconverters **** TODO Add unit test *** TODO Update cd2track If disc contains pre-emphasis, remove it during extraction **** TODO Add unit test *** TODO Update tracksplit If CD image contains pre-emphasis, remove it during extraction **** TODO Add unit test *** TODO Update track2cd If cuesheet is given and indicates pre-emphasis, re-add it when creating disc image for writing **** TODO Add unit test *** TODO Update trackcat If cuesheet is given an indicates pre-emphasis, re-add it when concatenating files to image **** TODO Add unit test ** DONE Add wider --cue support *** DONE Add Sheet.from_cddareader() method this won't account for all index points just yet, but should should handle disc pre-gap and round-trip correctly **** DONE update documentation **** DONE update unit tests *** DONE cdda2track build simple cuesheet from disc information and dump it to file so things like disc pre-gap can be accounted for **** DONE update man page **** DONE update unit tests *** DONE Update Sheet.track_length() method Take an optional total_length parameter of the entire disc's length, in seconds, and use that length to calculate the final track's length, if necessary. **** DONE Update Sheet.track_length() **** DONE Update Flac_CUESHEET.track_length() **** DONE Update CDTOC.track_length() **** DONE Update documentation **** DONE Add unit tests **** DONE Update utilities - [X] trackinfo - [X] trackverify *** DONE trackcmp detect if largest file is a CD image and accomodate its internal cuesheet (if any) **** DONE update unit tests ** TODO Add CTDB support Need to figure out how the cuetools client interface actually works. Then attempt to use it wherever AccurateRip is used, if possible. ** TODO Improve test coverage *** DONE Check __repr__ on all audio types *** DONE Check __repr__ on all audio decoders *** DONE Check __repr__ on all metadata types - [X] ApeTag - [X] FlacMetaData - [X] ID3CommentPair - [X] ID3v1Comment - [X] ID3v22Comment - [X] ID3v23Comment - [X] ID3v24Comment - [X] M4A_META_Atom - [X] MetaData - [X] VorbisComment *** DONE Check supports_to_pcm() on all audio types *** DONE Check supports_from_pcm() on all audio types *** DONE Check UnsupportedBitsPerSample on all audio types - [X] ALACAudio - [X] AiffAudio - [X] AuAudio - [X] FlacAudio - [X] M4AAudio - [X] MP2Audio - [X] MP3Audio - [X] MPCAudio - [X] OpusAudio - [X] TrueAudio - [X] VorbisAudio - [X] WavPackAudio - [X] WaveAudio *** TODO Check converting image types for FLAC/ID3v2 - [ ] FLacMetaData - [ ] ID3v22Comment - [ ] ID3v23Comment - [ ] ID3v24Comment *** DONE Check Image.type_string() for all image types *** TODO Check has_foreign_wave_chunks() == False Along with wave_header_footer() raising ValueError *** TODO Check has_foreign_aiff_chunks() == False Along with aiff_header_footer() raising ValueError *** TODO Clean out vestigial Ogg FLAC stuff ** TODO Cleanup chunk-based interface (again) There needs to be a single-pass approach to encoding chunked files that doesn't involve pulling out the chunk data ahead of time. Perhaps a simple .to_wave(), .from_wave() approach which returns a whole .wav file as a file stream would be easiest. ** TODO Cleanup AAC support Although the fdk-aac library doesn't have a compatible license, there must be *some* way to support AAC without the nasty hacks in place now. ** TODO Improve utility test coverage *** TODO Add some basic tests to ensure they still works - [ ] cdinfo - [ ] cdplay - [ ] trackplay - [ ] trackverify ================================================ FILE: audiotools/__init__.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2020 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA """the core Python Audio Tools module""" import sys import re import os import os.path import audiotools.pcm as pcm from functools import total_ordering from fractions import Fraction PY3 = sys.version_info[0] >= 3 PY2 = not PY3 try: from configparser import RawConfigParser except ImportError: from ConfigParser import RawConfigParser class RawConfigParser(RawConfigParser): """extends RawConfigParser to provide additional methods""" def get_default(self, section, option, default): """returns a default if option is not found in section""" try: from configparser import NoSectionError, NoOptionError except ImportError: from ConfigParser import NoSectionError, NoOptionError try: return self.get(section, option) except NoSectionError: return default except NoOptionError: return default def set_default(self, section, option, value): try: from configparser import NoSectionError except ImportError: from ConfigParser import NoSectionError try: self.set(section, option, value) except NoSectionError: self.add_section(section) self.set(section, option, value) def getint_default(self, section, option, default): """returns a default int if option is not found in section""" try: from configparser import NoSectionError, NoOptionError except ImportError: from ConfigParser import NoSectionError, NoOptionError try: return self.getint(section, option) except NoSectionError: return default except NoOptionError: return default def getboolean_default(self, section, option, default): """returns a default boolean if option is not found in section""" try: from configparser import NoSectionError, NoOptionError except ImportError: from ConfigParser import NoSectionError, NoOptionError try: return self.getboolean(section, option) except NoSectionError: return default except NoOptionError: return default config = RawConfigParser() config.read([os.path.join("/etc", "audiotools.cfg"), os.path.join(sys.prefix, "etc", "audiotools.cfg"), os.path.expanduser('~/.audiotools.cfg')]) BUFFER_SIZE = 0x100000 FRAMELIST_SIZE = 0x100000 // 4 class __system_binaries__(object): def __init__(self, config): self.config = config def __getitem__(self, command): try: from configparser import NoSectionError, NoOptionError except ImportError: from ConfigParser import NoSectionError, NoOptionError try: return self.config.get("Binaries", command) except NoSectionError: return command except NoOptionError: return command def can_execute(self, command): if os.sep in command: return os.access(command, os.X_OK) else: for path in os.environ.get('PATH', os.defpath).split(os.pathsep): if os.access(os.path.join(path, command), os.X_OK): return True return False BIN = __system_binaries__(config) DEFAULT_CDROM = config.get_default("System", "cdrom", "/dev/cdrom") MUSICBRAINZ_SERVICE = config.getboolean_default("MusicBrainz", "service", True) MUSICBRAINZ_SERVER = config.get_default("MusicBrainz", "server", "musicbrainz.org") MUSICBRAINZ_PORT = config.getint_default("MusicBrainz", "port", 80) ADD_REPLAYGAIN = config.getboolean_default("ReplayGain", "add_by_default", True) VERSION = "3.2alpha2" VERSION_STR = "Python Audio Tools {}".format(VERSION) DEFAULT_FILENAME_FORMAT = '%(track_number)2.2d - %(track_name)s.%(suffix)s' FILENAME_FORMAT = config.get_default("Filenames", "format", DEFAULT_FILENAME_FORMAT) FS_ENCODING = config.get_default("System", "fs_encoding", sys.getfilesystemencoding()) if FS_ENCODING is None: FS_ENCODING = 'UTF-8' VERBOSITY_LEVELS = ("quiet", "normal", "debug") DEFAULT_VERBOSITY = config.get_default("Defaults", "verbosity", "normal") if DEFAULT_VERBOSITY not in VERBOSITY_LEVELS: DEFAULT_VERBOSITY = "normal" DEFAULT_TYPE = config.get_default("System", "default_type", "wav") # field name -> (field string, text description) mapping def __format_fields__(): from audiotools.text import (METADATA_TRACK_NAME, METADATA_TRACK_NUMBER, METADATA_TRACK_TOTAL, METADATA_ALBUM_NAME, METADATA_ARTIST_NAME, METADATA_PERFORMER_NAME, METADATA_COMPOSER_NAME, METADATA_CONDUCTOR_NAME, METADATA_MEDIA, METADATA_ISRC, METADATA_CATALOG, METADATA_COPYRIGHT, METADATA_PUBLISHER, METADATA_YEAR, METADATA_DATE, METADATA_ALBUM_NUMBER, METADATA_ALBUM_TOTAL, METADATA_COMMENT, METADATA_SUFFIX, METADATA_ALBUM_TRACK_NUMBER, METADATA_BASENAME) return {u"track_name": (u"%(track_name)s", METADATA_TRACK_NAME), u"track_number": (u"%(track_number)2.2d", METADATA_TRACK_NUMBER), u"track_total": (u"%(track_total)d", METADATA_TRACK_TOTAL), u"album_name": (u"%(album_name)s", METADATA_ALBUM_NAME), u"artist_name": (u"%(artist_name)s", METADATA_ARTIST_NAME), u"performer_name": (u"%(performer_name)s", METADATA_PERFORMER_NAME), u"composer_name": (u"%(composer_name)s", METADATA_COMPOSER_NAME), u"conductor_name": (u"%(conductor_name)s", METADATA_CONDUCTOR_NAME), u"media": (u"%(media)s", METADATA_MEDIA), u"ISRC": (u"%(ISRC)s", METADATA_ISRC), u"catalog": (u"%(catalog)s", METADATA_CATALOG), u"copyright": (u"%(copyright)s", METADATA_COPYRIGHT), u"publisher": (u"%(publisher)s", METADATA_PUBLISHER), u"year": (u"%(year)s", METADATA_YEAR), u"date": (u"%(date)s", METADATA_DATE), u"album_number": (u"%(album_number)d", METADATA_ALBUM_NUMBER), u"album_total": (u"%(album_total)d", METADATA_ALBUM_TOTAL), u"comment": (u"%(comment)s", METADATA_COMMENT), u"suffix": (u"%(suffix)s", METADATA_SUFFIX), u"album_track_number": (u"%(album_track_number)s", METADATA_ALBUM_TRACK_NUMBER), u"basename": (u"%(basename)s", METADATA_BASENAME)} FORMAT_FIELDS = __format_fields__() FORMAT_FIELD_ORDER = (u"track_name", u"artist_name", u"album_name", u"track_number", u"track_total", u"album_number", u"album_total", u"performer_name", u"composer_name", u"conductor_name", u"catalog", u"ISRC", u"publisher", u"media", u"year", u"date", u"copyright", u"comment", u"suffix", u"album_track_number", u"basename") def __default_quality__(audio_type): quality = DEFAULT_QUALITY.get(audio_type, "") try: if quality not in TYPE_MAP[audio_type].COMPRESSION_MODES: return TYPE_MAP[audio_type].DEFAULT_COMPRESSION else: return quality except KeyError: return "" if config.has_option("System", "maximum_jobs"): MAX_JOBS = config.getint_default("System", "maximum_jobs", 1) else: try: from multiprocessing import cpu_count MAX_JOBS = cpu_count() except (ImportError, AttributeError): MAX_JOBS = 1 class Messenger(object): """this class is for displaying formatted output in a consistent way""" def __init__(self, silent=False): """executable is a unicode string of what script is being run this is typically for use by the usage() method""" self.__stdout__ = sys.stdout self.__stderr__ = sys.stderr if silent: self.__print__ = self.__print_silent__ elif PY3: self.__print__ = self.__print_py3__ else: self.__print__ = self.__print_py2__ def output_isatty(self): return self.__stdout__.isatty() def info_isatty(self): return self.__stderr__.isatty() def error_isatty(self): return self.__stderr__.isatty() def __print_silent__(self, string, stream, add_newline, flush): """prints unicode string to the given stream and if 'add_newline' is True, appends a newline if 'flush' is True, flushes the stream""" assert(isinstance(string, str if PY3 else unicode)) # do nothing pass def __print_py2__(self, string, stream, add_newline, flush): """prints unicode string to the given stream and if 'add_newline' is True, appends a newline if 'flush' is True, flushes the stream""" assert(isinstance(string, unicode)) # we can't output unicode strings directly to streams # because non-TTYs will raise UnicodeEncodeErrors stream.write(string.encode("UTF-8", "replace")) if add_newline: stream.write(os.linesep) if flush: stream.flush() def __print_py3__(self, string, stream, add_newline, flush): """prints unicode string to the given stream and if 'add_newline' is True, appends a newline if 'flush' is True, flushes the stream""" assert(isinstance(string, str)) stream.write(string) if add_newline: stream.write(os.linesep) if flush: stream.flush() def output(self, s): """displays an output message unicode string to stdout this appends a newline to that message""" self.__print__(string=s, stream=self.__stdout__, add_newline=True, flush=False) def partial_output(self, s): """displays a partial output message unicode string to stdout this flushes output so that message is displayed""" self.__print__(string=s, stream=self.__stdout__, add_newline=False, flush=True) def info(self, s): """displays an informative message unicode string to stderr this appends a newline to that message""" self.__print__(string=s, stream=self.__stderr__, add_newline=True, flush=False) def partial_info(self, s): """displays a partial informative message unicode string to stdout this flushes output so that message is displayed""" self.__print__(string=s, stream=self.__stderr__, add_newline=False, flush=True) # what's the difference between output() and info() ? # output() is for a program's primary data # info() is for incidental information # for example, trackinfo(1) should use output() for what it displays # since that output is its primary function # but track2track should use info() for its lines of progress # since its primary function is converting audio # and tty output is purely incidental def error(self, s): """displays an error message unicode string to stderr this appends a newline to that message""" self.__print__(string=u"*** Error: {}".format(s), stream=self.__stderr__, add_newline=True, flush=False) def os_error(self, oserror): """displays an properly formatted OSError exception to stderr this appends a newline to that message""" self.error(u"[Errno {:d}] {}: '{}'".format( oserror.errno, oserror.strerror, Filename(oserror.filename))) def warning(self, s): """displays a warning message unicode string to stderr this appends a newline to that message""" self.__print__(string=u"*** Warning: {}".format(s,), stream=self.__stderr__, add_newline=True, flush=False) def ansi_clearline(self): """generates a set of clear line ANSI escape codes to stdout this works only if stdout is a tty. Otherwise, it does nothing for example: >>> msg = Messenger("audiotools") >>> msg.partial_output(u"working") >>> time.sleep(1) >>> msg.ansi_clearline() >>> msg.output(u"done") """ if self.output_isatty(): self.partial_output((u"\u001B[0G" + # move cursor to column 0 # clear everything after cursor u"\u001B[0K")) def ansi_uplines(self, lines): """moves the cursor up by the given number of lines""" if self.output_isatty(): self.partial_output(u"\u001B[{:d}A".format(lines)) def ansi_cleardown(self): """clears the remainder of the screen from the cursor downward""" if self.output_isatty(): self.partial_output(u"\u001B[0J") def ansi_clearscreen(self): """clears the entire screen and moves cursor to upper left corner""" if self.output_isatty(): self.partial_output(u"\u001B[2J" + u"\u001B[1;1H") def terminal_size(self, fd): """returns the current terminal size as (height, width)""" try: from os import get_terminal_size size = get_terminal_size(fd) return (size.lines, size.columns) except ImportError: import fcntl import termios import struct # this isn't all that portable, but will have to do return struct.unpack('hh', fcntl.ioctl(fd, termios.TIOCGWINSZ, '1234')) class SilentMessenger(Messenger): def __init__(self): Messenger.__init__(self, silent=True) def khz(hz): """given an integer sample rate value in Hz, returns a unicode kHz value with suffix the string is typically 7-8 characters wide""" num = hz // 1000 den = (hz % 1000) // 100 if den == 0: return u"{:d}kHz".format(num) else: return u"{:d}.{:d}kHz".format(num, den) def hex_string(byte_string): """given a string of bytes, returns a Unicode string encoded as hex""" if PY3: hex_digits = [u"{:02X}".format(b) for b in byte_string] else: hex_digits = [u"{:02X}".format(ord(b)) for b in byte_string] return u"".join(hex_digits) class output_text(tuple): """a class for formatting unicode strings for display""" COLORS = {"black", "red", "green", "yellow", "blue", "magenta", "cyan", "white"} STYLES = {"bold", "underline", "blink", "inverse"} def __new__(cls, unicode_string, fg_color=None, bg_color=None, style=None): """unicode_string is the text to be displayed fg_color and bg_color may be one of: 'black', 'red', 'green', 'yellow', 'blue', 'magenta', 'cyan', 'white' style may be one of: 'bold', 'underline', 'blink', 'inverse' """ import unicodedata assert(isinstance(unicode_string, str if PY3 else unicode)) string = unicodedata.normalize("NFC", unicode_string) CHAR_WIDTHS = {"Na": 1, "A": 1, "W": 2, "F": 2, "N": 1, "H": 1} return cls.__construct__( unicode_string=string, char_widths=[CHAR_WIDTHS.get( unicodedata.east_asian_width(char), 1) for char in string], fg_color=fg_color, bg_color=bg_color, style=style, open_codes=cls.__open_codes__(fg_color, bg_color, style), close_codes=cls.__close_codes__(fg_color, bg_color, style)) @classmethod def __construct__(cls, unicode_string, char_widths, fg_color, bg_color, style, open_codes, close_codes): """ | unicode_string | unicode | output string | | char_widths | [int] | width of each unicode_string char | | fg_color | str | foreground color, or None | | bg_color | str | background color, or None | | style | str | text style, or None | | open_codes | unicode | ANSI escape codes | | close_codes | unicode | ANSI escape codes | """ assert(len(unicode_string) == len(char_widths)) return tuple.__new__(cls, [unicode_string, # 0 tuple(char_widths), # 1 sum(char_widths), # 2 fg_color, # 3 bg_color, # 4 style, # 5 open_codes, # 6 close_codes # 7 ]) def __repr__(self): # the other fields can be derived return "{}({!r}, {!r}, {!r}, {!r})".format(self.__class__.__name__, self[0], self.fg_color(), self.bg_color(), self.style()) @classmethod def __open_codes__(cls, fg_color, bg_color, style): open_codes = [] if fg_color is not None: if fg_color not in cls.COLORS: raise ValueError("invalid fg_color {!r}".format(fg_color)) else: open_codes.append({"black": u"30", "red": u"31", "green": u"32", "yellow": u"33", "blue": u"34", "magenta": u"35", "cyan": u"36", "white": u"37"}[fg_color]) if bg_color is not None: if bg_color not in cls.COLORS: raise ValueError("invalid bg_color {!r}".format(bg_color)) else: open_codes.append({"black": u"40", "red": u"41", "green": u"42", "yellow": u"43", "blue": u"44", "magenta": u"45", "cyan": u"46", "white": u"47"}[bg_color]) if style is not None: if style not in cls.STYLES: raise ValueError("invalid style {!r}".format(style)) else: open_codes.append({"bold": u"1", "underline": u"4", "blink": u"5", "inverse": u"7"}[style]) if len(open_codes) > 0: return u"\u001B[{}m".format(u";".join(open_codes)) else: return u"" @classmethod def __close_codes__(cls, fg_color, bg_color, style): close_codes = [] if fg_color is not None: if fg_color not in cls.COLORS: raise ValueError("invalid fg_color {!r}".format(fg_color)) else: close_codes.append(u"39") if bg_color is not None: if bg_color not in cls.COLORS: raise ValueError("invalid bg_color {!r}".format(bg_color)) else: close_codes.append(u"49") if style is not None: if style not in cls.STYLES: raise ValueError("invalid style {!r}".format(style)) else: close_codes.append({"bold": u"22", "underline": u"24", "blink": u"25", "inverse": u"27"}[style]) if len(close_codes) > 0: return u"\u001B[{}m".format(u";".join(close_codes)) else: return u"" if PY3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('utf-8') def __unicode__(self): return self[0] def char_widths(self): """yields a (char, width) for each character in string""" for pair in zip(self[0], self[1]): yield pair def __len__(self): return self[2] def fg_color(self): """returns the foreground color as a string, or None""" return self[3] def bg_color(self): """returns the background color as a string, or None""" return self[4] def style(self): """returns the style as a string, or None""" return self[5] def set_string(self, unicode_string, char_widths): """returns a new output_text with the given string""" assert(len(unicode_string) == len(char_widths)) return output_text.__construct__( unicode_string=unicode_string, char_widths=char_widths, fg_color=self[3], bg_color=self[4], style=self[5], open_codes=self[6], close_codes=self[7]) def set_format(self, fg_color=None, bg_color=None, style=None): """returns a new output_text with the given format""" return output_text.__construct__( unicode_string=self[0], char_widths=self[1], fg_color=fg_color, bg_color=bg_color, style=style, open_codes=output_text.__open_codes__(fg_color, bg_color, style), close_codes=output_text.__close_codes__(fg_color, bg_color, style)) def has_formatting(self): """returns True if the text has formatting set""" return ((self[3] is not None) or (self[4] is not None) or (self[5] is not None)) def format(self, is_tty=False): """returns unicode text formatted depending on is_tty""" if is_tty and self.has_formatting(): return u"{}{}{}".format(self[6], self[0], self[7]) else: return self[0] def head(self, display_characters): """returns a text object truncated to the given length characters at the end of the string are removed as needed due to double-width characters, the size of the string may be smaller than requested""" if display_characters < 0: raise ValueError("display characters must be >= 0") output_chars = [] output_widths = [] for (char, width) in self.char_widths(): if width <= display_characters: output_chars.append(char) output_widths.append(width) display_characters -= width else: break return self.set_string( unicode_string=u"".join(output_chars), char_widths=output_widths) def tail(self, display_characters): """returns a text object truncated to the given length characters at the beginning of the string are removed as needed due to double-width characters, the size of the string may be smaller than requested""" if display_characters < 0: raise ValueError("display characters must be >= 0") output_chars = [] output_widths = [] for (char, width) in reversed(list(self.char_widths())): if width <= display_characters: output_chars.append(char) output_widths.append(width) display_characters -= width else: break output_chars.reverse() output_widths.reverse() return self.set_string( unicode_string=u"".join(output_chars), char_widths=output_widths) def split(self, display_characters): """returns a tuple of text objects the first is up to 'display_characters' in length the second contains the remainder of the string due to double-width characters, the first string may be smaller than requested""" if display_characters < 0: raise ValueError("display characters must be >= 0") head_chars = [] head_widths = [] tail_chars = [] tail_widths = [] for (char, width) in self.char_widths(): if width <= display_characters: head_chars.append(char) head_widths.append(width) display_characters -= width else: tail_chars.append(char) tail_widths.append(width) display_characters = -1 return (self.set_string(unicode_string=u"".join(head_chars), char_widths=head_widths), self.set_string(unicode_string=u"".join(tail_chars), char_widths=tail_widths)) def join(self, output_texts): """returns output_list joined by our formatted text""" def join_iter(texts): for (i, text) in enumerate(texts): if i > 0: yield self yield text return output_list(join_iter(output_texts)) class output_list(output_text): """a class for formatting multiple unicode strings as a unit Note that a styled list enclosing styled text isn't likely to nest as expected since styles are reset to the terminal default rather than to what they were initially. So it's best to either style the internal elements or style the list, but not both.""" def __new__(cls, output_texts, fg_color=None, bg_color=None, style=None): """output_texts is an iterable of output_text objects or unicode fg_color and bg_color may be one of: 'black', 'red', 'green', 'yellow', 'blue', 'magenta', 'cyan', 'white' style may be one of: 'bold', 'underline', 'blink', 'inverse' """ return cls.__construct__( output_texts=[t if isinstance(t, output_text) else output_text(t) for t in output_texts], fg_color=fg_color, bg_color=bg_color, style=style, open_codes=cls.__open_codes__(fg_color, bg_color, style), close_codes=cls.__close_codes__(fg_color, bg_color, style)) @classmethod def __construct__(cls, output_texts, fg_color, bg_color, style, open_codes, close_codes): """ | output_texts | [output_text] | output texts | | fg_color | str | foreground color, or None | | bg_color | str | background color, or None | | style | str | text style, or None | | open_codes | unicode | ANSI escape codes | | close_codes | unicode | ANSI escape codes | """ return tuple.__new__(cls, [tuple(output_texts), # 0 sum(map(len, output_texts)), # 1 fg_color, # 2 bg_color, # 3 style, # 4 open_codes, # 5 close_codes # 6 ]) # use __repr__ from parent # use __str__ from parent def __unicode__(self): return u"".join(s[0] for s in self[0]) def char_widths(self): """yields a (char, width) for each character in list""" for output_text in self[0]: for pair in output_text.char_widths(): yield pair def __len__(self): return self[1] def fg_color(self): """returns the foreground color as a string""" return self[2] def bg_color(self): """returns the background color as a string""" return self[3] def style(self): """returns the style as a string""" return self[4] def set_string(self, output_texts): """returns a new output_list with the given texts""" return output_list.__construct__( output_texts=[t if isinstance(t, output_text) else output_text(t) for t in output_texts], fg_color=self[2], bg_color=self[3], style=self[4], open_codes=self[5], close_codes=self[6]) def set_format(self, fg_color=None, bg_color=None, style=None): """returns a new output_list with the given format""" return output_list.__construct__( output_texts=self[0], fg_color=fg_color, bg_color=bg_color, style=style, open_codes=output_list.__open_codes__(fg_color, bg_color, style), close_codes=output_list.__close_codes__(fg_color, bg_color, style)) def has_formatting(self): """returns True if the output_list itself has formatting set""" return ((self[2] is not None) or (self[3] is not None) or (self[4] is not None)) def format(self, is_tty=False): """returns unicode text formatted depending on is_tty""" # display escape codes around entire list # or on individual text items # but not both if is_tty and self.has_formatting(): return u"{}{}{}".format( self[5], u"".join(t.format(False) for t in self[0]), self[6]) else: return u"".join(t.format(is_tty) for t in self[0]) def head(self, display_characters): """returns a text object truncated to the given length characters at the end of the string are removed as needed due to double-width characters, the size of the string may be smaller than requested""" if display_characters < 0: raise ValueError("display characters must be >= 0") output_texts = [] for text in self[0]: if len(text) <= display_characters: output_texts.append(text) display_characters -= len(text) else: output_texts.append(text.head(display_characters)) break return self.set_string(output_texts=output_texts) def tail(self, display_characters): """returns a text object truncated to the given length characters at the beginning of the string are removed as needed due to double-width characters, the size of the string may be smaller than requested""" if display_characters < 0: raise ValueError("display characters must be >= 0") output_texts = [] for text in reversed(self[0]): if len(text) <= display_characters: output_texts.append(text) display_characters -= len(text) else: output_texts.append(text.tail(display_characters)) break return self.set_string(output_texts=reversed(output_texts)) def split(self, display_characters): """returns a tuple of text objects the first is up to 'display_characters' in length the second contains the remainder of the string due to double-width characters, the first string may be smaller than requested """ if display_characters < 0: raise ValueError("display characters must be >= 0") head_texts = [] tail_texts = [] for text in self[0]: if len(text) <= display_characters: head_texts.append(text) display_characters -= len(text) elif display_characters >= 0: (head, tail) = text.split(display_characters) head_texts.append(head) tail_texts.append(tail) display_characters = -1 else: tail_texts.append(text) return (self.set_string(output_texts=head_texts), self.set_string(output_texts=tail_texts)) class output_table(object): def __init__(self): """a class for formatting rows for display""" self.__rows__ = [] def row(self): """returns a output_table_row object which columns can be added to""" row = output_table_row() self.__rows__.append(row) return row def blank_row(self): """inserts a blank table row with no output""" self.__rows__.append(output_table_blank()) def divider_row(self, dividers): """adds a row of unicode divider characters there should be one character in dividers per output column""" self.__rows__.append(output_table_divider(dividers)) def format(self, is_tty=False): """yields one unicode formatted string per row depending on is_tty""" if len(self.__rows__) == 0: # no rows, so do nothing return row_columns = {len(r) for r in self.__rows__ if not r.blank()} if len(row_columns) == 0: # all rows are blank for row in self.__rows__: # blank rows ignore column widths yield row.format(None, is_tty) elif len(row_columns) == 1: column_widths = [ max(row.column_width(col) for row in self.__rows__) for col in range(row_columns.pop())] for row in self.__rows__: yield row.format(column_widths, is_tty) else: raise ValueError("all rows must have same number of columns") class output_table_blank(object): """a class for an empty table row""" def __init__(self): pass def blank(self): return True def column_width(self, column): return 0 def format(self, column_widths, is_tty=False): """returns formatted row as unicode""" return u"" class output_table_divider(output_table_blank): """a class for formatting a row of divider characters""" def __init__(self, dividers): self.__dividers__ = dividers[:] def blank(self): return False def __len__(self): return len(self.__dividers__) def column_width(self, column): return 0 def format(self, column_widths, is_tty=False): """returns formatted row as unicode""" assert(len(column_widths) == len(self.__dividers__)) return u"".join([divider * width for (divider, width) in zip(self.__dividers__, column_widths)]).rstrip() class output_table_row(output_table_divider): def __init__(self): """a class for formatting columns for display""" self.__columns__ = [] def __len__(self): return len(self.__columns__) def column_width(self, column): return self.__columns__[column].minimum_width() def format(self, column_widths, is_tty=False): """returns formatted row as unicode""" assert(len(column_widths) == len(self.__columns__)) return u"".join([column.format(width, is_tty) for (column, width) in zip(self.__columns__, column_widths)]).rstrip() def add_column(self, text, alignment="left", colspan=1): """adds text, which may be unicode or a formatted output_text object alignment may be 'left', 'center', 'right'""" if alignment not in {"left", "center", "right"}: raise ValueError("alignment must be 'left', 'center', or 'right'") if colspan == 1: self.__columns__.append(output_table_col(text, alignment)) elif colspan > 1: accumulators = [output_table_multicol_accumulator() for i in range(colspan - 1)] self.__columns__.extend(accumulators) self.__columns__.append(output_table_multicol( accumulators, text, alignment)) else: raise ValueError("colspan must be >= 1") class output_table_col(object): def __init__(self, text, alignment="left"): """text is an output_text or unicode object, alignment is 'left', 'center', or 'right' """ if isinstance(text, output_text): self.__text__ = text else: self.__text__ = output_text(text) if alignment == "left": self.format = self.__format_left__ elif alignment == "center": self.format = self.__format_center__ elif alignment == "right": self.format = self.__format_right__ else: raise ValueError("alignment must be 'left', 'center', or 'right'") def minimum_width(self): return len(self.__text__) def __format_left__(self, column_width, is_tty): padding = column_width - len(self.__text__) if padding > 0: return self.__text__.format(is_tty) + u" " * padding elif padding == 0: return self.__text__.format(is_tty) else: truncated = self.__text__.head(column_width) return (truncated.format(is_tty) + u" " * (column_width - len(truncated))) def __format_center__(self, column_width, is_tty): left_padding = (column_width - len(self.__text__)) // 2 right_padding = column_width - (left_padding + len(self.__text__)) if (left_padding > 0) or (right_padding > 0): return (u" " * left_padding + self.__text__.format(is_tty) + u" " * right_padding) elif (left_padding == 0) and (right_padding == 0): return self.__text__.format(is_tty) else: truncated = self.__text__.head(column_width) return (truncated.format(is_tty) + u" " * (column_width - len(truncated))) def __format_right__(self, column_width, is_tty): padding = column_width - len(self.__text__) if padding > 0: return u" " * padding + self.__text__.format(is_tty) elif padding == 0: return self.__text__.format(is_tty) else: truncated = self.__text__.tail(column_width) return (u" " * (column_width - len(truncated)) + truncated.format(is_tty)) class output_table_multicol_accumulator(output_table_col): def __init__(self): self.__actual_width__ = 0 def minimum_width(self): return 0 def actual_width(self): return self.__actual_width__ def format(self, column_width, is_tty): self.__actual_width__ = column_width return u"" class output_table_multicol(output_table_col): def __init__(self, accumulators, text, alignment="left"): self.__base_col__ = output_table_col(text, alignment) self.__accumulators__ = accumulators def minimum_width(self): return 0 def format(self, column_width, is_tty): return self.__base_col__.format( sum([a.actual_width() for a in self.__accumulators__]) + column_width, is_tty) class ProgressDisplay(object): """a class for displaying incremental progress updates to the screen""" def __init__(self, messenger): """takes a Messenger object for displaying output""" from collections import deque self.messenger = messenger self.progress_rows = [] self.empty_slots = deque() self.displayed_rows = 0 def add_row(self, output_line): """returns ProgressRow to be displayed output_line is a unicode string""" if len(self.empty_slots) == 0: # no slots to reuse, so append new row index = len(self.progress_rows) row = ProgressRow(self, index, output_line) self.progress_rows.append(row) return row else: # reuse first available slot index = self.empty_slots.popleft() row = ProgressRow(self, index, output_line) self.progress_rows[index] = row return row def remove_row(self, row_index): """removes the given row index and frees the slot for reuse""" self.empty_slots.append(row_index) self.progress_rows[row_index] = None def display_rows(self): """outputs the current state of all progress rows""" if sys.stdout.isatty(): (screen_height, screen_width) = self.messenger.terminal_size(sys.stdout.fileno()) for row in self.progress_rows: if (((row is not None) and (self.displayed_rows < screen_height))): self.messenger.output(row.unicode(screen_width)) self.displayed_rows += 1 def clear_rows(self): """clears all previously displayed output rows, if any""" if sys.stdout.isatty() and (self.displayed_rows > 0): self.messenger.ansi_clearline() self.messenger.ansi_uplines(self.displayed_rows) self.messenger.ansi_cleardown() self.displayed_rows = 0 def output_line(self, line): """displays a line of text to the Messenger's output after any previous rows have been cleared and before any new rows have been displayed""" self.messenger.output(line) class ProgressRow(object): """a class for displaying a single row of progress output it should be returned from ProgressDisplay.add_row() rather than instantiated directly""" def __init__(self, progress_display, row_index, output_line): """progress_display is a ProgressDisplay object row_index is this row's output index output_line is a unicode string""" from time import time self.progress_display = progress_display self.row_index = row_index self.output_line = output_text(output_line) self.progress = Fraction(0, 1) self.start_time = time() def update(self, progress): """updates our row with the current progress value""" self.progress = min(progress, 1) def finish(self): """indicate output is finished and the row will no longer be needed""" self.progress_display.remove_row(self.row_index) def unicode(self, width): """returns a unicode string formatted to the given width""" from time import time try: time_spent = time() - self.start_time split_point = int(width * self.progress) estimated_total_time = time_spent / self.progress estimated_time_remaining = int(round(estimated_total_time - time_spent)) time_remaining = u" {:2d}:{:02d}".format( estimated_time_remaining // 60, estimated_time_remaining % 60) except ZeroDivisionError: split_point = 0 time_remaining = u" --:--" if len(self.output_line) + len(time_remaining) > width: # truncate output line and append time remaining truncated = self.output_line.tail( width - (len(time_remaining) + 1)) combined_line = output_list( # note that "truncated" may be smaller than expected # so pad with more ellipsises if needed [u"\u2026" * (width - (len(truncated) + len(time_remaining))), truncated, time_remaining]) else: # add padding between output line and time remaining combined_line = output_list( [self.output_line, u" " * (width - (len(self.output_line) + len(time_remaining))), time_remaining]) # turn whole line into progress bar (head, tail) = combined_line.split(split_point) return (head.set_format(fg_color="white", bg_color="blue").format(True) + tail.format(True)) class SingleProgressDisplay(ProgressDisplay): """a specialized ProgressDisplay for handling a single line of output""" def __init__(self, messenger, progress_text): """takes a Messenger class and unicode string for output""" ProgressDisplay.__init__(self, messenger) self.row = self.add_row(progress_text) from time import time self.time = time self.last_updated = 0 def update(self, progress): """updates the output line with new progress value""" now = self.time() if (now - self.last_updated) > 0.25: self.clear_rows() self.row.update(progress) self.display_rows() self.last_updated = now class ReplayGainProgressDisplay(ProgressDisplay): """a specialized ProgressDisplay for handling ReplayGain application""" def __init__(self, messenger): """takes a Messenger and whether ReplayGain is lossless or not""" ProgressDisplay.__init__(self, messenger) from time import time from audiotools.text import RG_ADDING_REPLAYGAIN self.time = time self.last_updated = 0 self.row = self.add_row(RG_ADDING_REPLAYGAIN) if sys.stdout.isatty(): self.initial_message = self.initial_message_tty self.update = self.update_tty self.final_message = self.final_message_tty else: self.initial_message = self.initial_message_nontty self.update = self.update_nontty self.final_message = self.final_message_nontty def initial_message_tty(self): """displays a message that ReplayGain application has started""" pass def initial_message_nontty(self): """displays a message that ReplayGain application has started""" from audiotools.text import RG_ADDING_REPLAYGAIN_WAIT self.messenger.info(RG_ADDING_REPLAYGAIN_WAIT) def update_tty(self, progress): """updates the current status of ReplayGain application""" now = self.time() if (now - self.last_updated) > 0.25: self.clear_rows() self.row.update(progress) self.display_rows() self.last_updated = now def update_nontty(self, progress): """updates the current status of ReplayGain application""" pass def final_message_tty(self): """displays a message that ReplayGain application is complete""" from audiotools.text import RG_REPLAYGAIN_ADDED self.clear_rows() self.messenger.info(RG_REPLAYGAIN_ADDED) def final_message_nontty(self): """displays a message that ReplayGain application is complete""" pass class UnsupportedFile(Exception): """raised by open() if the file can be opened but not identified""" pass class InvalidFile(Exception): """raised during initialization if the file is invalid in some way""" pass class EncodingError(IOError): """raised if an audio file cannot be created correctly from from_pcm() due to an error by the encoder""" def __init__(self, error_message): error_message = str(error_message) if PY3 else unicode(error_message) IOError.__init__(self, error_message) self.error_message = error_message if PY3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('utf-8') def __unicode__(self): return self.error_message class UnsupportedChannelMask(EncodingError): """raised if the encoder does not support the file's channel mask""" def __init__(self, filename, mask): from audiotools.text import ERR_UNSUPPORTED_CHANNEL_MASK EncodingError.__init__( self, ERR_UNSUPPORTED_CHANNEL_MASK.format( target_filename=Filename(filename), assignment=ChannelMask(mask))) self.filename = filename self.mask = mask def __reduce__(self): return (UnsupportedChannelMask, (self.filename, self.mask)) class UnsupportedChannelCount(EncodingError): """raised if the encoder does not support the file's channel count""" def __init__(self, filename, count): from audiotools.text import ERR_UNSUPPORTED_CHANNEL_COUNT EncodingError.__init__( self, ERR_UNSUPPORTED_CHANNEL_COUNT.format( target_filename=Filename(filename), channels=count)) self.filename = filename self.count = count def __reduce__(self): return (UnsupportedChannelCount, (self.filename, self.count)) class UnsupportedBitsPerSample(EncodingError): """raised if the encoder does not support the file's bits-per-sample""" def __init__(self, filename, bits_per_sample): from audiotools.text import ERR_UNSUPPORTED_BITS_PER_SAMPLE EncodingError.__init__( self, ERR_UNSUPPORTED_BITS_PER_SAMPLE.format( target_filename=Filename(filename), bps=bits_per_sample)) self.filename = filename self.bits_per_sample = bits_per_sample def __reduce__(self): return (UnsupportedBitsPerSample, (self.filename, self.bits_per_sample)) class DecodingError(IOError): """raised if the decoder exits with an error typically, a from_pcm() method will catch this error and raise EncodingError""" def __init__(self, error_message): assert(isinstance(error_message, str if PY3 else unicode)) IOError.__init__(self, error_message) self.error_message = error_message if PY3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('utf-8') def __unicode__(self): return self.error_message def file_type(file): """given a seekable file stream returns an AudioFile-compatible class that stream is a type of or None of the stream's type is unknown the AudioFile class is not guaranteed to be available""" start = file.tell() header = file.read(37) file.seek(start, 0) if ((header[4:8] == b"ftyp") and (header[8:12] in (b"mp41", b"mp42", b"M4A ", b"M4B "))): # possibly ALAC or M4A from audiotools.bitstream import BitstreamReader from audiotools.m4a import get_m4a_atom reader = BitstreamReader(file, False) # so get contents of moov->trak->mdia->minf->stbl->stsd atom try: stsd = get_m4a_atom(reader, b"moov", b"trak", b"mdia", b"minf", b"stbl", b"stsd")[1] (stsd_version, descriptions, atom_size, atom_type) = stsd.parse("8u 24p 32u 32u 4b") if atom_type == b"alac": # if first description is "alac" atom, it's an ALAC return ALACAudio elif atom_type == b"mp4a": # if first description is "mp4a" atom, it's M4A return M4AAudio else: # otherwise, it's unknown return None except KeyError: # no stsd atom, so unknown return None except IOError: # error reading atom, so unknown return None elif (header[0:4] == b"FORM") and (header[8:12] == b"AIFF"): return AiffAudio elif header[0:4] == b".snd": return AuAudio elif header[0:4] == b"fLaC": return FlacAudio elif (len(header) >= 4) and (header[0:1] == b"\xFF"): # possibly MP3 or MP2 from audiotools.bitstream import parse # header is at least 32 bits, so no IOError is possible (frame_sync, mpeg_id, layer_description, protection, bitrate, sample_rate, pad, private, channels, mode_extension, copy, original, emphasis) = parse("11u 2u 2u 1u 4u 2u 1u " + "1u 2u 2u 1u 1u 2u", False, header) if (((frame_sync == 0x7FF) and (mpeg_id == 3) and (layer_description == 1) and (bitrate != 0xF) and (sample_rate != 3) and (emphasis != 2))): # MP3s are MPEG-1, Layer-III return MP3Audio elif ((frame_sync == 0x7FF) and (mpeg_id == 3) and (layer_description == 2) and (bitrate != 0xF) and (sample_rate != 3) and (emphasis != 2)): # MP2s are MPEG-1, Layer-II return MP2Audio else: # nothing else starts with an initial byte of 0xFF # so the file is unknown return None elif header[0:4] == b"OggS": # possibly Ogg FLAC, Ogg Vorbis, Ogg Opus or Ogg Speex #if header[0x1C:0x21] == b"\x7FFLAC": # return OggFlacAudio if header[0x1C:0x23] == b"\x01vorbis": return VorbisAudio elif header[0x1C:0x26] == b"OpusHead\x01": return OpusAudio elif header[0x1C:0x24] == b"Speex ": return SpeexAudio else: return None elif header[0:4] == b"wvpk": return WavPackAudio elif (header[0:4] == b"RIFF") and (header[8:12] == b"WAVE"): return WaveAudio elif ((len(header) >= 10) and (header[0:3] == b"ID3") and (header[3:4] in {b"\x02", b"\x03", b"\x04"})): # file contains ID3v2 tag # so it may be MP3, MP2, FLAC or TTA from audiotools.bitstream import parse # determine sync-safe tag size and skip entire tag tag_size = 0 for b in parse("1p 7u" * 4, False, header[6:10]): tag_size = (tag_size << 7) | b file.seek(start + 10 + tag_size, 0) t = file_type(file) # only return type which might be wrapped in ID3v2 tags if (((t is None) or (t is MP3Audio) or (t is MP2Audio) or (t is FlacAudio) or (t is TrueAudio))): return t else: return None elif header[0:4] == b"TTA1": return TrueAudio elif header[0:4] == b"MPCK": return MPCAudio else: return None # save a reference to Python's regular open function __open__ = open def open(filename): """returns an AudioFile located at the given filename path this works solely by examining the file's contents after opening it raises InvalidFile if the file appears to be something we support, but has errors of some sort raises IOError if some problem occurs attempting to open the file """ f = __open__(filename, "rb") try: audio_class = file_type(f) if audio_class is not None: return audio_class(filename) else: raise UnsupportedFile(filename) finally: f.close() class DuplicateFile(Exception): """raised if the same file is included more than once""" def __init__(self, filename): """filename is a Filename object""" from audiotools.text import ERR_DUPLICATE_FILE Exception.__init__(self, ERR_DUPLICATE_FILE.format(filename)) self.filename = filename class DuplicateOutputFile(Exception): """raised if the same output file is generated more than once""" def __init__(self, filename): """filename is a Filename object""" from audiotools.text import ERR_DUPLICATE_OUTPUT_FILE Exception.__init__(self, ERR_DUPLICATE_OUTPUT_FILE.format(filename)) self.filename = filename class OutputFileIsInput(Exception): """raised if an output file is the same as an input file""" def __init__(self, filename): """filename is a Filename object""" from audiotools.text import ERR_OUTPUT_IS_INPUT Exception.__init__(self, ERR_OUTPUT_IS_INPUT.format(filename)) self.filename = filename class Filename(tuple): def __new__(cls, filename): """filename is a string of the file on disk""" # under Python 2, filename should be a plain string # under Python 3, filename should be a unicode string if isinstance(filename, cls): return filename else: assert(isinstance(filename, str)) try: stat = os.stat(filename) return tuple.__new__(cls, [os.path.normpath(filename), stat.st_dev, stat.st_ino]) except OSError: return tuple.__new__(cls, [os.path.normpath(filename), None, None]) @classmethod def from_unicode(cls, unicode_string): """given a unicode string for a given path, returns a Filename object""" return cls(unicode_string if PY3 else unicode_string.encode(FS_ENCODING)) def open(self, mode): """returns a file object of this filename opened with the given mode""" return __open__(self[0], mode) def disk_file(self): """returns True if the file exists on disk""" return (self[1] is not None) and (self[2] is not None) def dirname(self): """returns the directory name (no filename) of this file""" return Filename(os.path.dirname(self[0])) def basename(self): """returns the basename (no directory) of this file""" return Filename(os.path.basename(self[0])) def expanduser(self): """returns a Filename object with user directory expanded""" return Filename(os.path.expanduser(self[0])) def abspath(self): """returns the Filename's absolute path as a Filename object""" return Filename(os.path.abspath(self[0])) def __repr__(self): return "Filename({!r}, {!r}, {!r})".format(self[0], self[1], self[2]) def __eq__(self, filename): if isinstance(filename, Filename): if self.disk_file() and filename.disk_file(): # both exist on disk, # so they compare equally if st_dev and st_ino match return (self[1] == filename[1]) and (self[2] == filename[2]) elif (not self.disk_file()) and (not filename.disk_file()): # neither exist on disk, # so they compare equally if their paths match return self[0] == filename[0] else: # one or the other exists on disk # but not both, so they never match return False else: return False def __ne__(self, filename): return not self == filename def __hash__(self): if self.disk_file(): return hash((None, self[1], self[2])) else: return hash((self[0], self[1], self[2])) def __str__(self): return self[0] if PY3: def __unicode__(self): return self[0] else: def __unicode__(self): return self[0].decode(FS_ENCODING, "replace") def sorted_tracks(audiofiles): """given a list of AudioFile objects returns a list of them sorted by track_number and album_number, if found """ return sorted(audiofiles, key=lambda f: f.__sort_key__()) def open_files(filename_list, sorted=True, messenger=None, no_duplicates=False, warn_duplicates=False, opened_files=None): """returns a list of AudioFile objects from a list of filename strings or Filename objects if "sorted" is True, files are sorted by album number then track number if "messenger" is given, warnings and errors when opening files are sent to the given Messenger-compatible object if "no_duplicates" is True, including the same file twice raises a DuplicateFile whose filename value is the first duplicate filename as a Filename object if "warn_duplicates" is True, including the same file twice results in a warning message to the messenger object, if given "opened_files" is a set object containing previously opened Filename objects and which newly opened Filename objects are added to """ from audiotools.text import (ERR_DUPLICATE_FILE, ERR_OPEN_IOERROR) if opened_files is None: opened_files = set() to_return = [] for filename in map(Filename, filename_list): if filename not in opened_files: opened_files.add(filename) else: if no_duplicates: raise DuplicateFile(filename) elif warn_duplicates and (messenger is not None): messenger.warning(ERR_DUPLICATE_FILE.format(filename)) try: with __open__(str(filename), "rb") as f: audio_class = file_type(f) if audio_class is not None: to_return.append(audio_class(str(filename))) else: # not a support audio type pass except IOError as err: if messenger is not None: messenger.warning(ERR_OPEN_IOERROR.format(filename)) except InvalidFile as err: if messenger is not None: messenger.error(str(err)) return (sorted_tracks(to_return) if sorted else to_return) def open_directory(directory, sorted=True, messenger=None): """yields an AudioFile via a recursive search of directory files are sorted by album number/track number by default, on a per-directory basis any unsupported files are filtered out error messages are sent to messenger, if given """ for (basedir, subdirs, filenames) in os.walk(directory): if sorted: subdirs.sort() for audiofile in open_files([os.path.join(basedir, filename) for filename in filenames], sorted=sorted, messenger=messenger): yield audiofile def group_tracks(tracks): """takes an iterable collection of tracks yields list of tracks grouped by album where their album_name and album_number match, if possible""" collection = {} for track in tracks: metadata = track.get_metadata() if metadata is not None: collection.setdefault( (metadata.album_number if metadata.album_number is not None else -(2 ** 31), metadata.album_name if metadata.album_name is not None else u""), []).append(track) else: collection.setdefault(None, []).append(track) if None in collection: yield collection[None] for key in sorted([key for key in collection.keys() if key is not None]): yield collection[key] class UnknownAudioType(Exception): """raised if filename_to_type finds no possibilities""" def __init__(self, suffix): self.suffix = suffix def error_msg(self, messenger): from audiotools.text import ERR_UNSUPPORTED_AUDIO_TYPE messenger.error(ERR_UNSUPPORTED_AUDIO_TYPE.format(self.suffix)) class AmbiguousAudioType(UnknownAudioType): """raised if filename_to_type finds more than one possibility""" def __init__(self, suffix, type_list): self.suffix = suffix self.type_list = type_list def error_msg(self, messenger): from audiotools.text import (ERR_AMBIGUOUS_AUDIO_TYPE, LAB_T_OPTIONS) messenger.error(ERR_AMBIGUOUS_AUDIO_TYPE.format(self.suffix)) messenger.info( LAB_T_OPTIONS.format( (u" or ".join([u"\"{}\"".format(t.NAME) for t in self.type_list])))) def filename_to_type(path): """given a path to a file, return its audio type based on suffix for example: >>> filename_to_type("/foo/file.flac") <class audiotools.__flac__.FlacAudio at 0x7fc8456d55f0> raises UnknownAudioType exception if the type is unknown raises AmbiguousAudioType exception if the type is ambiguous """ (path, ext) = os.path.splitext(path) if len(ext) > 0: ext = ext[1:] # remove the "." SUFFIX_MAP = {} for audio_type in TYPE_MAP.values(): SUFFIX_MAP.setdefault(audio_type.SUFFIX, []).append(audio_type) if ext in SUFFIX_MAP.keys(): if len(SUFFIX_MAP[ext]) == 1: return SUFFIX_MAP[ext][0] else: raise AmbiguousAudioType(ext, SUFFIX_MAP[ext]) else: raise UnknownAudioType(ext) else: raise UnknownAudioType(ext) class ChannelMask(object): """an integer-like class that abstracts a PCMReader's channel assignments all channels in a FrameList will be in RIFF WAVE order as a sensible convention but which channel corresponds to which speaker is decided by this mask for example, a 4 channel PCMReader with the channel mask 0x33 corresponds to the bits 00110011 reading those bits from right to left (least significant first) the "front_left", "front_right", "back_left", "back_right" speakers are set therefore, the PCMReader's 4 channel FrameLists are laid out as follows: channel 0 -> front_left channel 1 -> front_right channel 2 -> back_left channel 3 -> back_right since the "front_center" and "low_frequency" bits are not set, those channels are skipped in the returned FrameLists many formats store their channels internally in a different order their PCMReaders will be expected to reorder channels and set a ChannelMask matching this convention and, their from_pcm() functions will be expected to reverse the process a ChannelMask of 0 is "undefined", which means that channels aren't assigned to *any* speaker this is an ugly last resort for handling formats where multi-channel assignments aren't properly defined in this case, a from_pcm() method is free to assign the undefined channels any way it likes, and is under no obligation to keep them undefined when passing back out to to_pcm() """ SPEAKER_TO_MASK = {"front_left": 0x1, "front_right": 0x2, "front_center": 0x4, "low_frequency": 0x8, "back_left": 0x10, "back_right": 0x20, "front_left_of_center": 0x40, "front_right_of_center": 0x80, "back_center": 0x100, "side_left": 0x200, "side_right": 0x400, "top_center": 0x800, "top_front_left": 0x1000, "top_front_center": 0x2000, "top_front_right": 0x4000, "top_back_left": 0x8000, "top_back_center": 0x10000, "top_back_right": 0x20000} MASK_TO_SPEAKER = dict(map(reversed, map(list, SPEAKER_TO_MASK.items()))) from audiotools.text import (MASK_FRONT_LEFT, MASK_FRONT_RIGHT, MASK_FRONT_CENTER, MASK_LFE, MASK_BACK_LEFT, MASK_BACK_RIGHT, MASK_FRONT_RIGHT_OF_CENTER, MASK_FRONT_LEFT_OF_CENTER, MASK_BACK_CENTER, MASK_SIDE_LEFT, MASK_SIDE_RIGHT, MASK_TOP_CENTER, MASK_TOP_FRONT_LEFT, MASK_TOP_FRONT_CENTER, MASK_TOP_FRONT_RIGHT, MASK_TOP_BACK_LEFT, MASK_TOP_BACK_CENTER, MASK_TOP_BACK_RIGHT) MASK_TO_NAME = {0x1: MASK_FRONT_LEFT, 0x2: MASK_FRONT_RIGHT, 0x4: MASK_FRONT_CENTER, 0x8: MASK_LFE, 0x10: MASK_BACK_LEFT, 0x20: MASK_BACK_RIGHT, 0x40: MASK_FRONT_RIGHT_OF_CENTER, 0x80: MASK_FRONT_LEFT_OF_CENTER, 0x100: MASK_BACK_CENTER, 0x200: MASK_SIDE_LEFT, 0x400: MASK_SIDE_RIGHT, 0x800: MASK_TOP_CENTER, 0x1000: MASK_TOP_FRONT_LEFT, 0x2000: MASK_TOP_FRONT_CENTER, 0x4000: MASK_TOP_FRONT_RIGHT, 0x8000: MASK_TOP_BACK_LEFT, 0x10000: MASK_TOP_BACK_CENTER, 0x20000: MASK_TOP_BACK_RIGHT} def __init__(self, mask): """mask should be an integer channel mask value""" mask = int(mask) for (speaker, speaker_mask) in self.SPEAKER_TO_MASK.items(): setattr(self, speaker, (mask & speaker_mask) != 0) def __repr__(self): return "ChannelMask({})".format( ",".join(["{}={}".format(field, getattr(self, field)) for field in self.SPEAKER_TO_MASK.keys() if getattr(self, field)])) if PY3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('utf-8') def __unicode__(self): current_mask = int(self) return u",".join(ChannelMask.MASK_TO_NAME[mask] for mask in sorted(ChannelMask.MASK_TO_NAME.keys()) if mask & current_mask) def __int__(self): from operator import or_ from functools import reduce return reduce(or_, [self.SPEAKER_TO_MASK[field] for field in self.SPEAKER_TO_MASK.keys() if getattr(self, field)], 0) def __eq__(self, v): return int(self) == int(v) def __ne__(self, v): return int(self) != int(v) def __len__(self): return sum([1 for field in self.SPEAKER_TO_MASK.keys() if getattr(self, field)]) def defined(self): """returns True if this ChannelMask is defined""" return int(self) != 0 def undefined(self): """returns True if this ChannelMask is undefined""" return int(self) == 0 def channels(self): """returns a list of speaker strings this mask contains returned in the order in which they should appear in the PCM stream """ c = [] for (mask, speaker) in sorted(self.MASK_TO_SPEAKER.items(), key=lambda pair: pair[0]): if getattr(self, speaker): c.append(speaker) return c def index(self, channel_name): """returns the index of the given channel name within this mask for example, given the mask 0xB (fL, fR, LFE, but no fC) index("low_frequency") will return 2 if the channel is not in this mask, raises ValueError""" return self.channels().index(channel_name) @classmethod def from_fields(cls, **fields): """given a set of channel arguments, returns a new ChannelMask for example: >>> ChannelMask.from_fields(front_left=True,front_right=True) channelMask(front_right=True,front_left=True) """ mask = cls(0) for (key, value) in fields.items(): if key in cls.SPEAKER_TO_MASK.keys(): setattr(mask, key, bool(value)) else: raise KeyError(key) return mask @classmethod def from_channels(cls, channel_count): """given a channel count, returns a new ChannelMask this is only valid for channel counts 1 and 2 all other values trigger a ValueError""" if channel_count == 2: return cls(0x3) elif channel_count == 1: return cls(0x4) else: raise ValueError("ambiguous channel assignment") class PCMReader(object): def __init__(self, sample_rate, channels, channel_mask, bits_per_sample): """fields are as follows: sample rate - integer number of Hz channels - integer channel count channel_mask - integer channel mask value bits_per_sample - number number of bits-per-sample where channel mask is a logical OR of the following: | channel | value | |-----------------------+---------| | front left | 0x1 | | front right | 0x2 | | front center | 0x4 | | low frequency | 0x8 | | back left | 0x10 | | back right | 0x20 | | front left of center | 0x40 | | front right of center | 0x80 | | back center | 0x100 | | side left | 0x200 | | side right | 0x400 | | top center | 0x800 | | top front left | 0x1000 | | top front center | 0x2000 | | top front right | 0x4000 | | top back left | 0x8000 | | top back center | 0x10000 | | top back right | 0x20000 | |-----------------------+---------| """ self.sample_rate = sample_rate self.channels = channels self.channel_mask = channel_mask self.bits_per_sample = bits_per_sample def __enter__(self): return self def __exit__(self, exc_type, exc_value, traceback): self.close() def read(self, pcm_frames): """try to read the given number of PCM frames from the stream as a FrameList object this is *not* guaranteed to read exactly that number of frames it may return less (at the end of the stream, especially) it may return more however, it must always return a non-empty FrameList until the end of the PCM stream is reached may raise IOError if unable to read the input file, or ValueError if the input file has some sort of error """ raise NotImplementedError() def close(self): """closes the stream for reading subsequent calls to read() raise ValueError""" raise NotImplementedError() class PCMFileReader(PCMReader): """a class that wraps around a file object and generates pcm.FrameLists""" def __init__(self, file, sample_rate, channels, channel_mask, bits_per_sample, process=None, signed=True, big_endian=False): """fields are as follows: file - a file-like object with read() and close() methods sample_rate - an integer number of Hz channels - an integer number of channels channel_mask - an integer channel mask value bits_per_sample - an integer number of bits per sample process - an optional subprocess object signed - True if the file's samples are signed integers big_endian - True if the file's samples are stored big-endian the process, signed and big_endian arguments are optional PCMReader-compatible objects need only expose the sample_rate, channels, channel_mask and bits_per_sample fields along with the read() and close() methods """ PCMReader.__init__(self, sample_rate=sample_rate, channels=channels, channel_mask=channel_mask, bits_per_sample=bits_per_sample) self.file = file self.process = process self.signed = signed self.big_endian = big_endian self.bytes_per_frame = self.channels * (self.bits_per_sample // 8) def read(self, pcm_frames): """try to read the given number of PCM frames from the stream this is *not* guaranteed to read exactly that number of frames it may return less (at the end of the stream, especially) it may return more however, it must always return a non-empty FrameList until the end of the PCM stream is reached may raise IOError if unable to read the input file, or ValueError if the input file has some sort of error """ framelist = pcm.FrameList( self.file.read(max(pcm_frames, 1) * self.bytes_per_frame), self.channels, self.bits_per_sample, self.big_endian, self.signed) if framelist.frames > 0: return framelist elif self.process is not None: if self.process.wait() == 0: self.process = None return framelist else: self.process = None raise ValueError(u"subprocess exited with error") else: return framelist def close(self): """closes the stream for reading subsequent calls to read() raise ValueError""" self.file.close() if self.process is not None: self.process.wait() self.process = None def __del__(self): if self.process is not None: self.process.kill() self.process = None class PCMReaderError(PCMReader): """a dummy PCMReader which automatically raises ValueError this is to be returned by an AudioFile's to_pcm() method if some error occurs when initializing a decoder""" def __init__(self, error_message, sample_rate, channels, channel_mask, bits_per_sample): PCMReader.__init__(self, sample_rate=sample_rate, channels=channels, channel_mask=channel_mask, bits_per_sample=bits_per_sample) self.error_message = error_message def read(self, pcm_frames): """always raises a ValueError""" raise ValueError(self.error_message) def close(self): """does nothing""" pass def to_pcm_progress(audiofile, progress): if callable(progress): return PCMReaderProgress(audiofile.to_pcm(), audiofile.total_frames(), progress) else: return audiofile.to_pcm() class PCMReaderProgress(PCMReader): def __init__(self, pcmreader, total_frames, progress, current_frames=0): """pcmreader is a PCMReader compatible object total_frames is the total PCM frames of the stream progress(current, total) is a callable function current_frames is the current amount of PCM frames in the stream""" PCMReader.__init__(self, sample_rate=pcmreader.sample_rate, channels=pcmreader.channels, channel_mask=pcmreader.channel_mask, bits_per_sample=pcmreader.bits_per_sample) self.pcmreader = pcmreader self.current_frames = current_frames self.total_frames = max(total_frames, 1) if callable(progress): self.progress = progress else: self.progress = lambda progress: None def read(self, pcm_frames): frame = self.pcmreader.read(pcm_frames) self.current_frames += frame.frames self.progress(Fraction(self.current_frames, self.total_frames)) return frame def close(self): self.pcmreader.close() class ReorderedPCMReader(PCMReader): """a PCMReader wrapper which reorders its output channels""" def __init__(self, pcmreader, channel_order, channel_mask=None): """initialized with a PCMReader and list of channel number integers for example, to swap the channels of a stereo stream: >>> ReorderedPCMReader(reader,[1,0]) may raise ValueError if the number of channels specified by channel_order doesn't match the given channel mask if channel mask is nonzero """ PCMReader.__init__(self, sample_rate=pcmreader.sample_rate, channels=len(channel_order), channel_mask=(pcmreader.channel_mask if (channel_mask is None) else channel_mask), bits_per_sample=pcmreader.bits_per_sample) self.pcmreader = pcmreader self.channel_order = channel_order if (((self.channel_mask != 0) and (len(ChannelMask(self.channel_mask)) != self.channels))): # channel_mask is defined but has a different number of channels # than the channel count attribute from audiotools.text import ERR_CHANNEL_COUNT_MASK_MISMATCH raise ValueError(ERR_CHANNEL_COUNT_MASK_MISMATCH) def read(self, pcm_frames): """try to read a pcm.FrameList with the given number of frames""" framelist = self.pcmreader.read(pcm_frames) return pcm.from_channels([framelist.channel(channel) for channel in self.channel_order]) def close(self): """closes the stream""" self.pcmreader.close() class ThreadedPCMReader(PCMReader): """a PCMReader which decodes all output in the background It will queue *all* output from its contained PCMReader as fast as possible in a separate thread. This may be a problem if PCMReader's total output is very large or has no upper bound. """ def __init__(self, pcmreader): try: from queue import Queue except ImportError: from Queue import Queue from threading import (Thread, Event) PCMReader.__init__(self, sample_rate=pcmreader.sample_rate, channels=pcmreader.channels, channel_mask=pcmreader.channel_mask, bits_per_sample=pcmreader.bits_per_sample) def transfer_data(pcmreader, queue, stop_event): """transfers everything from pcmreader to queue until stop_event is set or the data is exhausted""" try: framelist = pcmreader.read(4096) while ((len(framelist) > 0) and (not stop_event.is_set())): queue.put((False, framelist)) framelist = pcmreader.read(4096) except (IOError, ValueError) as err: queue.put((True, err)) self.__pcmreader__ = pcmreader self.__queue__ = Queue() self.__stop_event__ = Event() self.__thread__ = Thread(target=transfer_data, args=(pcmreader, self.__queue__, self.__stop_event__)) self.__thread__.daemon = True self.__thread__.start() self.__closed__ = False self.__finished__ = False def read(self, pcm_frames): if self.__closed__: raise ValueError("stream is closed") if self.__finished__: # previous read returned empty FrameList # so continue to return empty FrameLists from audiotools.pcm import empty_framelist return empty_framelist(self.channels, self.bits_per_sample) (error, value) = self.__queue__.get() if not error: if len(value) == 0: self.__finished__ = True return value else: # some exception raised during transfer_data raise value def close(self): # tell decoder to finish if it is still operating self.__stop_event__.set() # collect finished thread self.__thread__.join() # close our contained PCMReader self.__pcmreader__.close() # mark stream as closed self.__closed__ = True def transfer_data(from_function, to_function): """sends BUFFER_SIZE strings from from_function to to_function this continues until an empty string is returned from from_function""" try: s = from_function(BUFFER_SIZE) while len(s) > 0: to_function(s) s = from_function(BUFFER_SIZE) except IOError: # this usually means a broken pipe, so we can only hope # the data reader is closing down correctly pass def transfer_framelist_data(pcmreader, to_function, signed=True, big_endian=False): """sends pcm.FrameLists from pcmreader to to_function frameLists are converted to strings using the signed and big_endian arguments. This continues until an empty FrameList is returned from pcmreader the pcmreader is closed when decoding is complete or fails with an error may raise IOError or ValueError if a problem occurs during decoding """ try: f = pcmreader.read(FRAMELIST_SIZE) while len(f) > 0: to_function(f.to_bytes(big_endian, signed)) f = pcmreader.read(FRAMELIST_SIZE) finally: pcmreader.close() def pcm_cmp(pcmreader1, pcmreader2): """returns True if the PCM data in pcmreader1 equals pcmreader2 both streams are closed once comparison is completed may raise IOError or ValueError if problems occur when reading PCM streams """ return (pcm_frame_cmp(pcmreader1, pcmreader2) is None) def pcm_frame_cmp(pcmreader1, pcmreader2): """returns the PCM Frame number of the first mismatch if the two streams match completely, returns None both streams are closed once comparison is completed may raise IOError or ValueError if problems occur when reading PCM streams """ if (((pcmreader1.sample_rate != pcmreader2.sample_rate) or (pcmreader1.channels != pcmreader2.channels) or (pcmreader1.bits_per_sample != pcmreader2.bits_per_sample))): pcmreader1.close() pcmreader2.close() return 0 if (((pcmreader1.channel_mask != 0) and (pcmreader2.channel_mask != 0) and (pcmreader1.channel_mask != pcmreader2.channel_mask))): pcmreader1.close() pcmreader2.close() return 0 frame_number = 0 reader1 = BufferedPCMReader(pcmreader1) reader2 = BufferedPCMReader(pcmreader2) try: framelist1 = reader1.read(FRAMELIST_SIZE) framelist2 = reader2.read(FRAMELIST_SIZE) # so long as both framelists contain data while (len(framelist1) > 0) and (len(framelist2) > 0): # compare both entire framelists if framelist1 == framelist2: # if they both match, continue to the next pair frame_number += framelist1.frames framelist1 = reader1.read(FRAMELIST_SIZE) framelist2 = reader2.read(FRAMELIST_SIZE) else: # if there's a mismatch, determine the exact frame for i in range(min(framelist1.frames, framelist2.frames)): if framelist1.frame(i) != framelist2.frame(i): return frame_number + i else: return frame_number + i else: # at least one framelist is empty if (len(framelist1) == 0) and (len(framelist2) == 0): # if they're both empty, there's no mismatch return None else: # if only one is empty, return as far as we've gotten return frame_number finally: reader1.close() reader2.close() class PCMCat(PCMReader): """a PCMReader for concatenating several PCMReaders""" def __init__(self, pcmreaders): """pcmreaders is a list of PCMReader objects all must have the same stream attributes""" self.index = 0 self.pcmreaders = list(pcmreaders) if len(self.pcmreaders) == 0: from audiotools.text import ERR_NO_PCMREADERS raise ValueError(ERR_NO_PCMREADERS) if len({r.sample_rate for r in self.pcmreaders}) != 1: from audiotools.text import ERR_SAMPLE_RATE_MISMATCH raise ValueError(ERR_SAMPLE_RATE_MISMATCH) if len({r.channels for r in self.pcmreaders}) != 1: from audiotools.text import ERR_CHANNEL_COUNT_MISMATCH raise ValueError(ERR_CHANNEL_COUNT_MISMATCH) if len({r.bits_per_sample for r in self.pcmreaders}) != 1: from audiotools.text import ERR_BPS_MISMATCH raise ValueError(ERR_BPS_MISMATCH) first_reader = self.pcmreaders[self.index] PCMReader.__init__(self, sample_rate=first_reader.sample_rate, channels=first_reader.channels, channel_mask=first_reader.channel_mask, bits_per_sample=first_reader.bits_per_sample) def read(self, pcm_frames): """try to read a pcm.FrameList with the given number of frames raises ValueError if any of the streams is mismatched""" if self.index < len(self.pcmreaders): # read a FrameList from the current PCMReader framelist = self.pcmreaders[self.index].read(pcm_frames) if len(framelist) > 0: # if it has data, return it return framelist else: # otherwise, move on to next pcmreader self.index += 1 return self.read(pcm_frames) else: # all PCMReaders exhausted, so return empty FrameList return pcm.empty_framelist(self.channels, self.bits_per_sample) def read_closed(self, pcm_frames): raise ValueError() def close(self): """closes the stream for reading""" self.read = self.read_closed for reader in self.pcmreaders: reader.close() from audiotools.pcmconverter import BufferedPCMReader class CounterPCMReader(PCMReader): """a PCMReader which counts bytes and frames written""" def __init__(self, pcmreader): PCMReader.__init__(self, sample_rate=pcmreader.sample_rate, channels=pcmreader.channels, channel_mask=pcmreader.channel_mask, bits_per_sample=pcmreader.bits_per_sample) self.pcmreader = pcmreader self.frames_written = 0 def bytes_written(self): return (self.frames_written * self.channels * (self.bits_per_sample // 8)) def read(self, pcm_frames): frame = self.pcmreader.read(pcm_frames) self.frames_written += frame.frames return frame def close(self): self.pcmreader.close() class LimitedFileReader(object): def __init__(self, file, total_bytes): self.__file__ = file self.__total_bytes__ = total_bytes def read(self, x): if self.__total_bytes__ > 0: s = self.__file__.read(x) if len(s) <= self.__total_bytes__: self.__total_bytes__ -= len(s) return s else: s = s[0:self.__total_bytes__] self.__total_bytes__ = 0 return s else: return "" def close(self): self.__file__.close() class LimitedPCMReader(PCMReader): def __init__(self, buffered_pcmreader, total_pcm_frames): """buffered_pcmreader should be a BufferedPCMReader which ensures we won't pull more frames off the reader than necessary upon calls to read()""" PCMReader.__init__( self, sample_rate=buffered_pcmreader.sample_rate, channels=buffered_pcmreader.channels, channel_mask=buffered_pcmreader.channel_mask, bits_per_sample=buffered_pcmreader.bits_per_sample) self.pcmreader = buffered_pcmreader self.total_pcm_frames = total_pcm_frames def read(self, pcm_frames): if self.total_pcm_frames > 0: frame = self.pcmreader.read(min(pcm_frames, self.total_pcm_frames)) self.total_pcm_frames -= frame.frames return frame else: return pcm.empty_framelist(self.channels, self.bits_per_sample) def read_closed(self, pcm_frames): raise ValueError() def close(self): self.read = self.read_closed def PCMConverter(pcmreader, sample_rate, channels, channel_mask, bits_per_sample): """a PCMReader wrapper for converting attributes for example, this can be used to alter sample_rate, bits_per_sample, channel_mask, channel count, or any combination of those attributes. It resamples, downsamples, etc. to achieve the proper output may raise ValueError if any of the attributes are unsupported or invalid """ if sample_rate <= 0: from audiotools.text import ERR_INVALID_SAMPLE_RATE raise ValueError(ERR_INVALID_SAMPLE_RATE) elif channels <= 0: from audiotools.text import ERR_INVALID_CHANNEL_COUNT raise ValueError(ERR_INVALID_CHANNEL_COUNT) elif bits_per_sample not in (8, 16, 24): from audiotools.text import ERR_INVALID_BITS_PER_SAMPLE raise ValueError(ERR_INVALID_BITS_PER_SAMPLE) if (channel_mask != 0) and (len(ChannelMask(channel_mask)) != channels): # channel_mask is defined but has a different number of channels # than the channel count attribute from audiotools.text import ERR_CHANNEL_COUNT_MASK_MISMATCH raise ValueError(ERR_CHANNEL_COUNT_MASK_MISMATCH) if pcmreader.channels > channels: if (channels == 1) and (channel_mask in (0, 0x4)): if pcmreader.channels > 2: # reduce channel count through downmixing # followed by averaging from audiotools.pcmconverter import (Averager, Downmixer) pcmreader = Averager(Downmixer(pcmreader)) else: # pcmreader.channels == 2 # so reduce channel count through averaging from audiotools.pcmconverter import Averager pcmreader = Averager(pcmreader) elif (channels == 2) and (channel_mask in (0, 0x3)): # reduce channel count through downmixing from audiotools.pcmconverter import Downmixer pcmreader = Downmixer(pcmreader) else: # unusual channel count/mask combination pcmreader = RemaskedPCMReader(pcmreader, channels, channel_mask) elif pcmreader.channels < channels: # increase channel count by duplicating first channel # (this is usually just going from mono to stereo # since there's no way to summon surround channels # out of thin air) pcmreader = ReorderedPCMReader(pcmreader, list(range(pcmreader.channels)) + [0] * (channels - pcmreader.channels), channel_mask) if pcmreader.sample_rate != sample_rate: # convert sample rate through resampling from audiotools.pcmconverter import Resampler pcmreader = Resampler(pcmreader, sample_rate) if pcmreader.bits_per_sample != bits_per_sample: # use bitshifts/dithering to adjust bits-per-sample from audiotools.pcmconverter import BPSConverter pcmreader = BPSConverter(pcmreader, bits_per_sample) return pcmreader class ReplayGainCalculator: def __init__(self, sample_rate): from audiotools.replaygain import ReplayGain self.__replaygain__ = ReplayGain(sample_rate) self.__tracks__ = [] def sample_rate(self): return self.__replaygain__.sample_rate def __iter__(self): try: album_gain = self.__replaygain__.album_gain() except ValueError: album_gain = 0.0 album_peak = self.__replaygain__.album_peak() for t in self.__tracks__: title_gain = t.title_gain() title_peak = t.title_peak() yield (title_gain, title_peak, album_gain, album_peak) def to_pcm(self, pcmreader): """given a PCMReader, returns a ReplayGainCalculatorReader which can be used to calculate the ReplayGain for the contents of that reader""" if pcmreader.sample_rate != self.sample_rate(): raise ValueError( "sample rate mismatch, {:d} != {:d}".format( pcmreader.sample_rate, self.sample_rate())) reader = ReplayGainCalculatorReader(self.__replaygain__, pcmreader) self.__tracks__.append(reader) return reader class ReplayGainCalculatorReader(PCMReader): def __init__(self, replaygain, pcmreader): PCMReader.__init__(self, sample_rate=pcmreader.sample_rate, channels=pcmreader.channels, channel_mask=pcmreader.channel_mask, bits_per_sample=pcmreader.bits_per_sample) self.__replaygain__ = replaygain self.__pcmreader__ = pcmreader self.__title_gain__ = None self.__title_peak__ = None def read(self, pcm_frames): framelist = self.__pcmreader__.read(pcm_frames) self.__replaygain__.update(framelist) return framelist def close(self): self.__pcmreader__.close() try: self.__title_gain__ = self.__replaygain__.title_gain() except ValueError: self.__title_gain__ = 0.0 self.__title_peak__ = self.__replaygain__.title_peak() self.__replaygain__.next_title() def title_gain(self): if self.__title_gain__ is not None: return self.__title_gain__ else: raise ValueError("cannot get title_gain before closing pcmreader") def title_peak(self): if self.__title_peak__ is not None: return self.__title_peak__ else: raise ValueError("cannot get title_peak before closing pcmreader") def resampled_frame_count(initial_frame_count, initial_sample_rate, new_sample_rate): """given an initial PCM frame count, initial sample rate and new sample rate, returns the new PCM frame count once the stream has been resampled""" if initial_sample_rate == new_sample_rate: return initial_frame_count else: from decimal import (Decimal, ROUND_DOWN) new_frame_count = ((Decimal(initial_frame_count) * Decimal(new_sample_rate)) / Decimal(initial_sample_rate)) return int(new_frame_count.quantize(Decimal("1."), rounding=ROUND_DOWN)) def calculate_replay_gain(tracks, progress=None): """yields (track, track_gain, track_peak, album_gain, album_peak) for each AudioFile in the list of tracks raises ValueError if a problem occurs during calculation""" if len(tracks) == 0: return from bisect import bisect SUPPORTED_RATES = [8000, 11025, 12000, 16000, 18900, 22050, 24000, 32000, 37800, 44100, 48000, 56000, 64000, 88200, 96000, 112000, 128000, 144000, 176400, 192000] target_rate = ([SUPPORTED_RATES[0]] + SUPPORTED_RATES)[ bisect(SUPPORTED_RATES, most_numerous([track.sample_rate() for track in tracks]))] track_frames = [resampled_frame_count(track.total_frames(), track.sample_rate(), target_rate) for track in tracks] current_frames = 0 total_frames = sum(track_frames) rg = ReplayGainCalculator(target_rate) for (track, track_frames) in zip(tracks, track_frames): pcm = track.to_pcm() # perform calculation by decoding through ReplayGain with rg.to_pcm( PCMReaderProgress( PCMConverter(pcm, target_rate, pcm.channels, pcm.channel_mask, pcm.bits_per_sample), total_frames, progress, current_frames)) as pcmreader: transfer_data(pcmreader.read, lambda f: None) current_frames += track_frames # yield a set of accumulated track and album gains for (track, (track_gain, track_peak, album_gain, album_peak)) in zip(tracks, rg): yield (track, track_gain, track_peak, album_gain, album_peak) def add_replay_gain(tracks, progress=None): """given an iterable set of AudioFile objects and optional progress function calculates the ReplayGain for them and adds it via their set_replay_gain method""" for (track, track_gain, track_peak, album_gain, album_peak) in calculate_replay_gain(tracks, progress): track.set_replay_gain(ReplayGain(track_gain=track_gain, track_peak=track_peak, album_gain=album_gain, album_peak=album_peak)) def ignore_sigint(): """sets the SIGINT signal to SIG_IGN some encoder executables require this in order for interruptableReader to work correctly since we want to catch SIGINT ourselves in that case and perform a proper shutdown""" import signal signal.signal(signal.SIGINT, signal.SIG_IGN) def make_dirs(destination_path): """ensures all directories leading to destination_path are created raises OSError if a problem occurs during directory creation """ dirname = os.path.dirname(destination_path) if (dirname != '') and (not os.path.isdir(dirname)): os.makedirs(dirname) class MetaData(object): """the base class for storing textual AudioFile metadata Fields may be None, indicating they're not present in the underlying metadata implementation. Changing a field to a new value will update the underlying metadata (e.g. vorbiscomment.track_name = u"Foo" will set a Vorbis comment's "TITLE" field to "Foo") Updating the underlying metadata will change the metadata's fields (e.g. setting a Vorbis comment's "TITLE" field to "bar" will update vorbiscomment.title_name to u"bar") Deleting a field or setting it to None will remove it from the underlying metadata (e.g. del(vorbiscomment.track_name) will delete the "TITLE" field) """ FIELDS = ("track_name", "track_number", "track_total", "album_name", "artist_name", "performer_name", "composer_name", "conductor_name", "media", "ISRC", "catalog", "copyright", "publisher", "year", "date", "album_number", "album_total", "comment", "compilation") FIELD_TYPES = {"track_name": type(u""), "track_number": int, "track_total": int, "album_name": type(u""), "artist_name": type(u""), "performer_name": type(u""), "composer_name": type(u""), "conductor_name": type(u""), "media": type(u""), "ISRC": type(u""), "catalog": type(u""), "copyright": type(u""), "publisher": type(u""), "year": type(u""), "date": type(u""), "album_number": int, "album_total": int, "comment": type(u""), "compilation": bool} # this is the order fields should be presented to the user # to ensure consistency across utilities FIELD_ORDER = ("track_name", "artist_name", "album_name", "track_number", "track_total", "album_number", "album_total", "compilation", "performer_name", "composer_name", "conductor_name", "catalog", "ISRC", "publisher", "media", "year", "date", "copyright", "comment") # this is the name fields should use when presented to the user # also to ensure constency across utilities from audiotools.text import (METADATA_TRACK_NAME, METADATA_TRACK_NUMBER, METADATA_TRACK_TOTAL, METADATA_ALBUM_NAME, METADATA_ARTIST_NAME, METADATA_PERFORMER_NAME, METADATA_COMPOSER_NAME, METADATA_CONDUCTOR_NAME, METADATA_MEDIA, METADATA_ISRC, METADATA_CATALOG, METADATA_COPYRIGHT, METADATA_PUBLISHER, METADATA_YEAR, METADATA_DATE, METADATA_ALBUM_NUMBER, METADATA_ALBUM_TOTAL, METADATA_COMMENT, METADATA_COMPILATION) FIELD_NAMES = {"track_name": METADATA_TRACK_NAME, "track_number": METADATA_TRACK_NUMBER, "track_total": METADATA_TRACK_TOTAL, "album_name": METADATA_ALBUM_NAME, "artist_name": METADATA_ARTIST_NAME, "performer_name": METADATA_PERFORMER_NAME, "composer_name": METADATA_COMPOSER_NAME, "conductor_name": METADATA_CONDUCTOR_NAME, "media": METADATA_MEDIA, "ISRC": METADATA_ISRC, "catalog": METADATA_CATALOG, "copyright": METADATA_COPYRIGHT, "publisher": METADATA_PUBLISHER, "year": METADATA_YEAR, "date": METADATA_DATE, "album_number": METADATA_ALBUM_NUMBER, "album_total": METADATA_ALBUM_TOTAL, "comment": METADATA_COMMENT, "compilation": METADATA_COMPILATION} def __init__(self, track_name=None, track_number=None, track_total=None, album_name=None, artist_name=None, performer_name=None, composer_name=None, conductor_name=None, media=None, ISRC=None, catalog=None, copyright=None, publisher=None, year=None, date=None, album_number=None, album_total=None, comment=None, compilation=None, images=None): """ | field | type | meaning | |----------------+---------+--------------------------------------| | track_name | unicode | the name of this individual track | | track_number | integer | the number of this track | | track_total | integer | the total number of tracks | | album_name | unicode | the name of this track's album | | artist_name | unicode | the song's original creator/composer | | performer_name | unicode | the song's performing artist | | composer_name | unicode | the song's composer name | | conductor_name | unicode | the song's conductor's name | | media | unicode | the album's media type | | ISRC | unicode | the song's ISRC | | catalog | unicode | the album's catalog number | | copyright | unicode | the song's copyright information | | publisher | unicode | the album's publisher | | year | unicode | the album's release year | | date | unicode | the original recording date | | album_number | integer | the disc's volume number | | album_total | integer | the total number of discs | | comment | unicode | the track's comment string | | compilation | boolean | whether track is part of compilation | | images | list | list of Image objects | |----------------+---------+--------------------------------------| """ # we're avoiding self.foo = foo because # __setattr__ might need to be redefined # which could lead to unwelcome side-effects MetaData.__setattr__(self, "track_name", track_name) MetaData.__setattr__(self, "track_number", track_number) MetaData.__setattr__(self, "track_total", track_total) MetaData.__setattr__(self, "album_name", album_name) MetaData.__setattr__(self, "artist_name", artist_name) MetaData.__setattr__(self, "performer_name", performer_name) MetaData.__setattr__(self, "composer_name", composer_name) MetaData.__setattr__(self, "conductor_name", conductor_name) MetaData.__setattr__(self, "media", media) MetaData.__setattr__(self, "ISRC", ISRC) MetaData.__setattr__(self, "catalog", catalog) MetaData.__setattr__(self, "copyright", copyright) MetaData.__setattr__(self, "publisher", publisher) MetaData.__setattr__(self, "year", year) MetaData.__setattr__(self, "date", date) MetaData.__setattr__(self, "album_number", album_number) MetaData.__setattr__(self, "album_total", album_total) MetaData.__setattr__(self, "comment", comment) MetaData.__setattr__(self, "compilation", compilation) if images is not None: MetaData.__setattr__(self, "__images__", list(images)) else: MetaData.__setattr__(self, "__images__", list()) def __repr__(self): return "MetaData({}{})".format( ",".join(["{}={!r}".format(field, value) for field, value in self.filled_fields()]), "" if len(self.images()) == 0 else ",images={!r}".format(self.images())) def __delattr__(self, field): if field in self.FIELDS: MetaData.__setattr__(self, field, None) else: try: object.__delattr__(self, field) except KeyError: raise AttributeError(field) def fields(self): """yields an (attr, value) tuple per MetaData field""" for attr in self.FIELDS: yield (attr, getattr(self, attr)) def filled_fields(self): """yields an (attr, value) tuple per MetaData field which is not blank""" for (attr, field) in self.fields(): if field is not None: yield (attr, field) def empty_fields(self): """yields an (attr, value) tuple per MetaData field which is blank""" for (attr, field) in self.fields(): if field is None: yield (attr, field) if PY3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('utf-8') def __unicode__(self): table = output_table() SEPARATOR = u" : " for attr in self.FIELD_ORDER: if attr == "track_number": # combine track number/track total into single field track_number = self.track_number track_total = self.track_total if (track_number is None) and (track_total is None): # nothing to display pass elif (track_number is not None) and (track_total is None): row = table.row() row.add_column(self.FIELD_NAMES[attr], "right") row.add_column(SEPARATOR) row.add_column(u"{:d}".format(track_number)) elif (track_number is None) and (track_total is not None): row = table.row() row.add_column(self.FIELD_NAMES[attr], "right") row.add_column(SEPARATOR) row.add_column(u"?/{:d}".format(track_total)) else: # neither track_number or track_total is None row = table.row() row.add_column(self.FIELD_NAMES[attr], "right") row.add_column(SEPARATOR) row.add_column(u"{:d}/{:d}".format(track_number, track_total)) elif attr == "track_total": pass elif attr == "album_number": # combine album number/album total into single field album_number = self.album_number album_total = self.album_total if (album_number is None) and (album_total is None): # nothing to display pass elif (album_number is not None) and (album_total is None): row = table.row() row.add_column(self.FIELD_NAMES[attr], "right") row.add_column(SEPARATOR) row.add_column(u"{:d}".format(track_number)) elif (album_number is None) and (album_total is not None): row = table.row() row.add_column(self.FIELD_NAMES[attr], "right") row.add_column(SEPARATOR) row.add_column(u"?/{:d}".format(album_total)) else: # neither album_number or album_total is None row = table.row() row.add_column(self.FIELD_NAMES[attr], "right") row.add_column(SEPARATOR) row.add_column(u"{:d}/{:d}".format(album_number, album_total)) elif attr == "album_total": pass elif getattr(self, attr) is not None: row = table.row() row.add_column(self.FIELD_NAMES[attr], "right") row.add_column(SEPARATOR) if self.FIELD_TYPES[attr] is type(u""): row.add_column(getattr(self, attr)) elif self.FIELD_TYPES[attr] is int: row.add_column(u"{:d}".format(getattr(self, attr))) elif self.FIELD_TYPES[attr] is bool: from audiotools.text import (METADATA_TRUE, METADATA_FALSE) row.add_column(METADATA_TRUE if getattr(self, attr) else METADATA_FALSE) # append image data, if necessary from audiotools.text import LAB_PICTURE for image in self.images(): row = table.row() row.add_column(LAB_PICTURE, "right") row.add_column(SEPARATOR) row.add_column(image.__unicode__()) return os.linesep.join(table.format()) def raw_info(self): """returns a Unicode string of low-level MetaData information whereas __unicode__ is meant to contain complete information at a very high level raw_info() should be more developer-specific and with very little adjustment or reordering to the data itself """ raise NotImplementedError() def __eq__(self, metadata): for attr in MetaData.FIELDS: if ((not hasattr(metadata, attr)) or (getattr(self, attr) != getattr(metadata, attr))): return False else: return True def __ne__(self, metadata): return not self.__eq__(metadata) @classmethod def converted(cls, metadata): """converts metadata from another class to this one, if necessary takes a MetaData-compatible object (or None) and returns a new MetaData subclass with the data fields converted or None if metadata is None or conversion isn't possible for instance, VorbisComment.converted() returns a VorbisComment class. This way, AudioFiles can offload metadata conversions """ if metadata is not None: fields = {field: getattr(metadata, field) for field in cls.FIELDS} fields["images"] = metadata.images() return MetaData(**fields) else: return None @classmethod def supports_images(cls): """returns True if this MetaData class supports embedded images""" return True def images(self): """returns a list of embedded Image objects""" # must return a copy of our internal array # otherwise this will likely not act as expected when deleting return self.__images__[:] def front_covers(self): """returns a subset of images() which are front covers""" return [i for i in self.images() if i.type == FRONT_COVER] def back_covers(self): """returns a subset of images() which are back covers""" return [i for i in self.images() if i.type == BACK_COVER] def leaflet_pages(self): """returns a subset of images() which are leaflet pages""" return [i for i in self.images() if i.type == LEAFLET_PAGE] def media_images(self): """returns a subset of images() which are media images""" return [i for i in self.images() if i.type == MEDIA] def other_images(self): """returns a subset of images() which are other images""" return [i for i in self.images() if i.type == OTHER] def add_image(self, image): """embeds an Image object in this metadata implementations of this method should also affect the underlying metadata value (e.g. adding a new Image to FlacMetaData should add another METADATA_BLOCK_PICTURE block to the metadata) """ if self.supports_images(): self.__images__.append(image) else: from audiotools.text import ERR_PICTURES_UNSUPPORTED raise ValueError(ERR_PICTURES_UNSUPPORTED) def delete_image(self, image): """deletes an Image object from this metadata implementations of this method should also affect the underlying metadata value (e.g. removing an existing Image from FlacMetaData should remove that same METADATA_BLOCK_PICTURE block from the metadata) """ if self.supports_images(): self.__images__.pop(self.__images__.index(image)) else: from audiotools.text import ERR_PICTURES_UNSUPPORTED raise ValueError(ERR_PICTURES_UNSUPPORTED) def clean(self): """returns a (MetaData, fixes_performed) tuple where MetaData is an object that's been cleaned of problems an fixes_performed is a list of Unicode strings. Problems include: * Remove leading or trailing whitespace from text fields * Remove empty fields * Remove leading zeroes from numerical fields (except when requested, in the case of ID3v2) * Fix incorrectly labeled image metadata fields """ return (MetaData(**{field: getattr(self, field) for field in MetaData.FIELDS}), []) def intersection(self, metadata): """given a MetaData-compatible object, returns a new MetaData object which contains all the matching fields and images of this object and 'metadata' """ if metadata is None: return None fields1 = {attr: field for attr, field in self.filled_fields()} fields2 = {attr: field for attr, field in metadata.filled_fields()} common_images = ({(img.type, img.data) for img in self.images()} & {(img.type, img.data) for img in metadata.images()}) return MetaData(images=[img for img in self.images() if (img.type, img.data) in common_images], **{attr: fields1[attr] for attr in set(fields1.keys()) & set(fields2.keys()) if fields1[attr] == fields2[attr]}) (FRONT_COVER, BACK_COVER, LEAFLET_PAGE, MEDIA, OTHER) = range(5) class Image(object): """an image data container""" def __init__(self, data, mime_type, width, height, color_depth, color_count, description, type): """fields are as follows: data - plain string of the actual binary image data mime_type - unicode string of the image's MIME type width - width of image, as integer number of pixels height - height of image, as integer number of pixels color_depth - color depth of image (24 for JPEG, 8 for GIF, etc.) color_count - number of palette colors, or 0 description - a unicode string type - an integer type whose values are one of: FRONT_COVER BACK_COVER LEAFLET_PAGE MEDIA OTHER """ assert(isinstance(data, bytes)) assert(isinstance(mime_type, str if PY3 else unicode)) assert(isinstance(width, int)) assert(isinstance(height, int)) assert(isinstance(color_depth, int)) assert(isinstance(color_count, int)) assert(isinstance(description, str if PY3 else unicode)) assert(isinstance(type, int)) self.data = data self.mime_type = mime_type self.width = width self.height = height self.color_depth = color_depth self.color_count = color_count self.description = description self.type = type def suffix(self): """returns the image's recommended suffix as a plain string for example, an image with mime_type "image/jpeg" return "jpg" """ return {"image/jpeg": "jpg", "image/jpg": "jpg", "image/gif": "gif", "image/png": "png", "image/x-ms-bmp": "bmp", "image/tiff": "tiff"}.get(self.mime_type, "bin") def type_string(self): """returns the image's type as a human readable Unicode string for example, an image of type 0 returns "Front Cover" """ return {FRONT_COVER: u"Front Cover", BACK_COVER: u"Back Cover", LEAFLET_PAGE: u"Leaflet Page", MEDIA: u"Media", OTHER: u"Other"}.get(self.type, u"Other") def __repr__(self): fields = ["{}={!r}".format(attr, getattr(self, attr)) for attr in ["mime_type", "width", "height", "color_depth", "color_count", "description", "type"]] return "Image({})".format(",".join(fields)) if PY3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('utf-8') def __unicode__(self): return u"{} ({:d}\u00D7{:d},'{}')".format(self.type_string(), self.width, self.height, self.mime_type) @classmethod def new(cls, image_data, description, type): """builds a new Image object from raw data image_data is a binary string of image data description is a unicode string type as an image type integer the width, height, color_depth and color_count fields are determined by parsing the binary image data raises InvalidImage if some error occurs during parsing """ from audiotools.image import image_metrics img = image_metrics(image_data) return Image(data=image_data, mime_type=img.mime_type, width=img.width, height=img.height, color_depth=img.bits_per_pixel, color_count=img.color_count, description=description, type=type) def __eq__(self, image): if image is not None: if hasattr(image, "data"): return self.data == image.data else: return False else: return False def __ne__(self, image): return not self.__eq__(image) class InvalidImage(Exception): """raised if an image cannot be parsed correctly""" class ReplayGain(object): """a container for ReplayGain data""" def __init__(self, track_gain, track_peak, album_gain, album_peak): """values are: track_gain - a dB float value track_peak - the highest absolute value PCM sample, as a float album_gain - a dB float value album_peak - the highest absolute value PCM sample, as a float they are also attributes """ self.track_gain = float(track_gain) self.track_peak = float(track_peak) self.album_gain = float(album_gain) self.album_peak = float(album_peak) def __repr__(self): return "ReplayGain({},{},{},{})".format(self.track_gain, self.track_peak, self.album_gain, self.album_peak) def __eq__(self, rg): if isinstance(rg, ReplayGain): return ((self.track_gain == rg.track_gain) and (self.track_peak == rg.track_peak) and (self.album_gain == rg.album_gain) and (self.album_peak == rg.album_peak)) else: return False def __ne__(self, rg): return not self.__eq__(rg) class UnsupportedTracknameField(Exception): """raised by AudioFile.track_name() if its format string contains unknown fields""" def __init__(self, field): self.field = field def error_msg(self, messenger): from audiotools.text import (ERR_UNKNOWN_FIELD, LAB_SUPPORTED_FIELDS) messenger.error(ERR_UNKNOWN_FIELD.format(self.field)) messenger.info(LAB_SUPPORTED_FIELDS) for field in sorted(MetaData.FIELDS + ("album_track_number", "suffix")): if field == 'track_number': messenger.info(u"%(track_number)2.2d") else: messenger.info(u"%%(%s)s" % (field)) messenger.info(u"%(basename)s") class InvalidFilenameFormat(Exception): """raised by AudioFile.track_name() if its format string contains broken fields""" def __init__(self, *args): from audiotools.text import ERR_INVALID_FILENAME_FORMAT Exception.__init__(self, ERR_INVALID_FILENAME_FORMAT) class AudioFile(object): """an abstract class representing audio files on disk this class should be extended to handle different audio file formats""" SUFFIX = "" NAME = "" DESCRIPTION = u"" DEFAULT_COMPRESSION = "" COMPRESSION_MODES = ("",) COMPRESSION_DESCRIPTIONS = {} BINARIES = tuple() BINARY_URLS = {} REPLAYGAIN_BINARIES = tuple() def __init__(self, filename): """filename is a plain string raises InvalidFile or subclass if the file is invalid in some way""" self.filename = filename def __repr__(self): return "{}({!r})".format(self.__class__.__name__, self.filename) # AudioFiles support a sorting rich compare # which prioritizes album_number, track_number and then filename # missing fields sort before non-missing fields # use pcm_frame_cmp to compare the contents of two files def __sort_key__(self): metadata = self.get_metadata() return ((metadata.album_number if ((metadata is not None) and (metadata.album_number is not None)) else -(2 ** 31)), (metadata.track_number if ((metadata is not None) and (metadata.track_number is not None)) else -(2 ** 31)), self.filename) def __eq__(self, audiofile): if isinstance(audiofile, AudioFile): return (self.__sort_key__() == audiofile.__sort_key__()) else: raise TypeError( "cannot compare {!r} and {!r}".format(self, audiofile)) def __ne__(self, audiofile): return not self.__eq__(audiofile) def __lt__(self, audiofile): if isinstance(audiofile, AudioFile): return (self.__sort_key__() < audiofile.__sort_key__()) else: raise TypeError( "cannot compare {!r} and {!r}".format(self, audiofile)) def __le__(self, audiofile): if isinstance(audiofile, AudioFile): return (self.__sort_key__() <= audiofile.__sort_key__()) else: raise TypeError( "cannot compare {!r} and {!r}".format(self, audiofile)) def __gt__(self, audiofile): if isinstance(audiofile, AudioFile): return (self.__sort_key__() > audiofile.__sort_key__()) else: raise TypeError( "cannot compare {!r} and {!r}".format(self, audiofile)) def __ge__(self, audiofile): if isinstance(audiofile, AudioFile): return (self.__sort_key__() >= audiofile.__sort_key__()) else: raise TypeError( "cannot compare {!r} and {!r}".format(self, audiofile)) def __gt__(self, audiofile): if isinstance(audiofile, AudioFile): return (self.__sort_key__() > audiofile.__sort_key__()) else: raise TypeError( "cannot compare {!r} and {!r}".format(self, audiofile)) def bits_per_sample(self): """returns an integer number of bits-per-sample this track contains""" raise NotImplementedError() def channels(self): """returns an integer number of channels this track contains""" raise NotImplementedError() def channel_mask(self): """returns a ChannelMask object of this track's channel layout""" # WARNING - This only returns valid masks for 1 and 2 channel audio # anything over 2 channels raises a ValueError # since there isn't any standard on what those channels should be. # AudioFiles that support more than 2 channels should override # this method with one that returns the proper mask. return ChannelMask.from_channels(self.channels()) def lossless(self): """returns True if this track's data is stored losslessly""" raise NotImplementedError() @classmethod def supports_metadata(cls): """returns True if this audio type supports MetaData""" return False def update_metadata(self, metadata): """takes this track's current MetaData object as returned by get_metadata() and sets this track's metadata with any fields updated in that object raises IOError if unable to write the file """ # this is a sort of low-level implementation # which assumes higher-level routines have # modified metadata properly if metadata is not None: raise NotImplementedError() else: raise ValueError(ERR_FOREIGN_METADATA) def set_metadata(self, metadata): """takes a MetaData object and sets this track's metadata this metadata includes track name, album name, and so on raises IOError if unable to write the file""" # this is a higher-level implementation # which assumes metadata is from a different audio file # or constructed from scratch and converts it accordingly # before passing it on to update_metadata() pass def get_metadata(self): """returns a MetaData object, or None raises IOError if unable to read the file""" return None def delete_metadata(self): """deletes the track's MetaData this removes or unsets tags as necessary in order to remove all data raises IOError if unable to write the file""" pass def total_frames(self): """returns the total PCM frames of the track as an integer""" raise NotImplementedError() def cd_frames(self): """returns the total length of the track in CD frames each CD frame is 1/75th of a second""" try: return (self.total_frames() * 75) // self.sample_rate() except ZeroDivisionError: return 0 def seconds_length(self): """returns the length of the track as a Fraction number of seconds""" from fractions import Fraction if self.sample_rate() > 0: return Fraction(self.total_frames(), self.sample_rate()) else: # this shouldn't happen, but just in case return Fraction(0, 1) def sample_rate(self): """returns the rate of the track's audio as an integer number of Hz""" raise NotImplementedError() @classmethod def supports_to_pcm(cls): """returns True if all necessary components are available to support the .to_pcm() method""" return False def to_pcm(self): """returns a PCMReader object containing the track's PCM data if an error occurs initializing a decoder, this should return a PCMReaderError with an appropriate error message""" raise NotImplementedError() @classmethod def supports_from_pcm(cls): """returns True if all necessary components are available to support the .from_pcm() classmethod""" return False @classmethod def from_pcm(cls, filename, pcmreader, compression=None, total_pcm_frames=None): """encodes a new file from PCM data takes a filename string, PCMReader object optional compression level string, and optional total_pcm_frames integer encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new AudioFile-compatible object specifying total_pcm_frames, when the number is known in advance, may allow the encoder to work more efficiently but is never required for example, to encode the FlacAudio file "file.flac" from "file.wav" at compression level "5": >>> flac = FlacAudio.from_pcm("file.flac", ... WaveAudio("file.wav").to_pcm(), ... "5") may raise EncodingError if some problem occurs when encoding the input file. This includes an error in the input stream, a problem writing the output file, or even an EncodingError subclass such as "UnsupportedBitsPerSample" if the input stream is formatted in a way this class is unable to support """ raise NotImplementedError() def convert(self, target_path, target_class, compression=None, progress=None): """encodes a new AudioFile from existing AudioFile take a filename string, target class and optional compression string encodes a new AudioFile in the target class and returns the resulting object may raise EncodingError if some problem occurs during encoding""" return target_class.from_pcm( target_path, to_pcm_progress(self, progress), compression, total_pcm_frames=(self.total_frames() if self.lossless() else None)) def seekable(self): """returns True if the file is seekable that is, if its PCMReader has a .seek() method and that method supports some sort of fine-grained seeking when the PCMReader is working from on-disk files""" return False @classmethod def __unlink__(cls, filename): try: os.unlink(filename) except OSError: pass @classmethod def track_name(cls, file_path, track_metadata=None, format=None, suffix=None): """constructs a new filename string given a string to an existing path, a MetaData-compatible object (or None), a Python format string and a suffix string (such as "mp3") returns a string of a new filename with format's fields filled-in raises UnsupportedTracknameField if the format string contains invalid template fields raises InvalidFilenameFormat if the format string has broken template fields""" # these should be traditional strings under # both Python 2 and 3 assert((format is None) or isinstance(format, str)) assert((suffix is None) or isinstance(suffix, str)) # handle defaults if format is None: format = FILENAME_FORMAT if suffix is None: suffix = cls.SUFFIX # convert arguments to unicode to simplify everything internally if PY2: format = format.decode("UTF-8", "replace") suffix = suffix.decode("UTF-8", "replace") # grab numeric arguments from MetaData, if any if track_metadata is not None: track_number = (track_metadata.track_number if track_metadata.track_number is not None else 0) album_number = (track_metadata.album_number if track_metadata.album_number is not None else 0) track_total = (track_metadata.track_total if track_metadata.track_total is not None else 0) album_total = (track_metadata.album_total if track_metadata.album_total is not None else 0) else: track_number = 0 album_number = 0 track_total = 0 album_total = 0 # setup preliminary format dictionary format_dict = {u"track_number": track_number, u"album_number": album_number, u"track_total": track_total, u"album_total": album_total, u"suffix": suffix} if album_number == 0: format_dict[u"album_track_number"] = u"%2.2d" % (track_number) else: album_digits = len(str(album_total)) format_dict[u"album_track_number"] = ( u"%%%(album_digits)d.%(album_digits)dd%%2.2d" % {u"album_digits": album_digits} % (album_number, track_number)) if PY2: format_dict[u"basename"] = os.path.splitext( os.path.basename(file_path))[0].decode("UTF-8", "replace") else: format_dict[u"basename"] = os.path.splitext( os.path.basename(file_path))[0] # populate remainder of format dictionary # with fields from MetaData, if any for field in (f for f, t in MetaData.FIELD_TYPES.items() if t is type(u"")): if PY2: field_name = field.decode("ascii") else: field_name = field if track_metadata is not None: attr = getattr(track_metadata, field) if attr is None: attr = u"" else: attr = u"" format_dict[field_name] = \ attr.replace(u"/", u"-").replace(u"\x00", u" ") try: # apply format dictionary if PY2: formatted_filename = (format % format_dict).encode("UTF-8", "replace") else: formatted_filename = (format % format_dict) except KeyError as error: raise UnsupportedTracknameField(error.args[0]) except TypeError: raise InvalidFilenameFormat() except ValueError: raise InvalidFilenameFormat() # ensure filename isn't absoluate return formatted_filename.lstrip(os.sep) @classmethod def supports_replay_gain(cls): """returns True if this class supports ReplayGain""" # implement this in subclass if necessary return False def get_replay_gain(self): """returns a ReplayGain object of our ReplayGain values returns None if we have no values may raise IOError if unable to read the file""" # implement this in subclass if necessary return None def set_replay_gain(self, replaygain): """given a ReplayGain object, sets the track's gain to those values may raise IOError if unable to modify the file""" # implement this in subclass if necessary pass def delete_replay_gain(self): """removes ReplayGain values from file, if any may raise IOError if unable to modify the file""" # implement this in subclass if necessary pass @classmethod def supports_cuesheet(self): """returns True if the audio format supports embedded Sheet objects""" return False def set_cuesheet(self, cuesheet): """imports cuesheet data from a Sheet object Raises IOError if an error occurs setting the cuesheet""" pass def get_cuesheet(self): """returns the embedded Cuesheet-compatible object, or None Raises IOError if a problem occurs when reading the file""" return None def delete_cuesheet(self): """deletes embedded Sheet object, if any Raises IOError if a problem occurs when updating the file""" pass def verify(self, progress=None): """verifies the current file for correctness returns True if the file is okay raises an InvalidFile with an error message if there is some problem with the file""" pcm_frame_count = 0 with to_pcm_progress(self, progress) as decoder: try: framelist = decoder.read(BUFFER_SIZE) while framelist.frames > 0: pcm_frame_count += framelist.frames framelist = decoder.read(BUFFER_SIZE) except (IOError, ValueError) as err: raise InvalidFile(str(err)) if self.lossless(): if pcm_frame_count == self.total_frames(): return True else: raise InvalidFile("incorrect PCM frame count") else: return True def clean(self, output_filename=None): """cleans the file of known data and metadata problems output_filename is an optional filename of the fixed file if present, a new AudioFile is written to that path otherwise, only a dry-run is performed and no new file is written return list of fixes performed as Unicode strings raises IOError if unable to write the file or its metadata raises ValueError if the file has errors of some sort """ if output_filename is None: # dry run only metadata = self.get_metadata() if metadata is not None: (metadata, fixes) = metadata.clean() return fixes else: return [] else: # perform full fix input_f = __open__(self.filename, "rb") output_f = __open__(output_filename, "wb") try: transfer_data(input_f.read, output_f.write) finally: input_f.close() output_f.close() new_track = open(output_filename) metadata = self.get_metadata() if metadata is not None: (metadata, fixes) = metadata.clean() new_track.set_metadata(metadata) return fixes else: return [] class WaveContainer(AudioFile): def has_foreign_wave_chunks(self): """returns True if the file has RIFF chunks other than 'fmt ' and 'data' which must be preserved during conversion""" raise NotImplementedError() def wave_header_footer(self): """returns (header, footer) tuple of strings containing all data before and after the PCM stream may raise ValueError if there's a problem with the header or footer data may raise IOError if there's a problem reading header or footer data from the file """ raise NotImplementedError() @classmethod def from_wave(cls, filename, header, pcmreader, footer, compression=None): """encodes a new file from wave data takes a filename string, header string, PCMReader object, footer string and optional compression level string encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new WaveAudio object header + pcm data + footer should always result in the original wave file being restored without need for any padding bytes may raise EncodingError if some problem occurs when encoding the input file""" raise NotImplementedError() def convert(self, target_path, target_class, compression=None, progress=None): """encodes a new AudioFile from existing AudioFile take a filename string, target class and optional compression string encodes a new AudioFile in the target class and returns the resulting object may raise EncodingError if some problem occurs during encoding""" if ((self.has_foreign_wave_chunks() and hasattr(target_class, "from_wave") and callable(target_class.from_wave))): # transfer header and footer when performing PCM conversion try: (header, footer) = self.wave_header_footer() except (ValueError, IOError) as err: raise EncodingError(err) return target_class.from_wave(target_path, header, to_pcm_progress(self, progress), footer, compression) else: # perform standard PCM conversion instead return target_class.from_pcm( target_path, to_pcm_progress(self, progress), compression, total_pcm_frames=(self.total_frames() if self.lossless() else None)) class AiffContainer(AudioFile): def has_foreign_aiff_chunks(self): """returns True if the file has AIFF chunks other than 'COMM' and 'SSND' which must be preserved during conversion""" raise NotImplementedError() def aiff_header_footer(self): """returns (header, footer) tuple of strings containing all data before and after the PCM stream may raise ValueError if there's a problem with the header or footer data may raise IOError if there's a problem reading header or footer data from the file""" raise NotImplementedError() @classmethod def from_aiff(cls, filename, header, pcmreader, footer, compression=None): """encodes a new file from AIFF data takes a filename string, header string, PCMReader object, footer string and optional compression level string encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new AiffAudio object header + pcm data + footer should always result in the original AIFF file being restored without need for any padding bytes may raise EncodingError if some problem occurs when encoding the input file""" raise NotImplementedError() def convert(self, target_path, target_class, compression=None, progress=None): """encodes a new AudioFile from existing AudioFile take a filename string, target class and optional compression string encodes a new AudioFile in the target class and returns the resulting object may raise EncodingError if some problem occurs during encoding""" if ((self.has_foreign_aiff_chunks() and hasattr(target_class, "from_aiff") and callable(target_class.from_aiff))): # transfer header and footer when performing PCM conversion try: (header, footer) = self.aiff_header_footer() except (ValueError, IOError) as err: raise EncodingError(err) return target_class.from_aiff(target_path, header, to_pcm_progress(self, progress), footer, compression) else: # perform standard PCM conversion instead return target_class.from_pcm( target_path, to_pcm_progress(self, progress), compression, total_pcm_frames=(self.total_frames() if self.lossless() else None)) class SheetException(ValueError): """a parent exception for CueException and TOCException""" pass def read_sheet(filename): """returns Sheet-compatible object from a .cue or .toc file may raise a SheetException if the file cannot be parsed correctly""" try: with __open__(filename, "rb") as f: return read_sheet_string(f.read().decode("UTF-8", "replace")) except IOError: from audiotools.text import ERR_CUE_IOERROR raise SheetException(ERR_CUE_IOERROR) def read_sheet_string(sheet_string): """given a string of cuesheet data, returns a Sheet-compatible object may raise a SheetException if the file cannot be parsed correctly""" str_type = str if PY3 else unicode assert(isinstance(sheet_string, str_type)) if u"CD_DA" in sheet_string: from audiotools.toc import read_tocfile_string return read_tocfile_string(sheet_string) else: from audiotools.cue import read_cuesheet_string return read_cuesheet_string(sheet_string) class Sheet(object): """an object representing a CDDA layout such as provided by a .cue or .toc file""" def __init__(self, sheet_tracks, metadata=None): """sheet_tracks is a list of SheetTrack objects metadata is a MetaData object, or None""" self.__sheet_tracks__ = sheet_tracks self.__metadata__ = metadata @classmethod def converted(cls, sheet): """given a Sheet-compatible object, returns a Sheet""" return cls(sheet_tracks=[SheetTrack.converted(t) for t in sheet], metadata=sheet.get_metadata()) @classmethod def from_cddareader(cls, cddareader, filename=u"CDImage.wav"): """given a CDDAReader object, returns a Sheet filename is an optional name for the sheet as a Unicode object""" from fractions import Fraction tracks = [] for number, offset in sorted(cddareader.track_offsets.items(), key=lambda pair: pair[0]): seconds_offset = Fraction(offset, cddareader.sample_rate) if (number == 1) and (seconds_offset > 0): track = SheetTrack(number, [SheetIndex(0, Fraction(0, 1)), SheetIndex(1, seconds_offset)]) else: track = SheetTrack(number, [SheetIndex(1, seconds_offset)]) tracks.append(track) return cls(tracks) @classmethod def from_tracks(cls, audiofiles, filename=u"CDImage.wav"): """given an iterable of AudioFile objects, returns a Sheet filename is an optional name for the sheet as a Unicode object""" from fractions import Fraction if len(audiofiles) == 0: # no tracks, so sheet is empty return cls(sheet_tracks=[], metadata=None) elif has_pre_gap_track(audiofiles): # prepend track 0 to start of cuesheet pre_gap_length = audiofiles[0].seconds_length() sheet_tracks = [ SheetTrack(number=1, track_indexes=[SheetIndex(number=0, offset=Fraction(0, 1)), SheetIndex(number=1, offset=pre_gap_length)], metadata=audiofiles[1].get_metadata(), filename=filename)] offset = pre_gap_length + audiofiles[1].seconds_length() for (number, audiofile) in enumerate(audiofiles[2:], 2): sheet_tracks.append( SheetTrack(number=number, track_indexes=[SheetIndex(number=1, offset=offset)], metadata=audiofile.get_metadata(), filename=filename)) offset += audiofile.seconds_length() return cls(sheet_tracks=sheet_tracks, metadata=None) else: # treat first track as track 1 sheet_tracks = [] offset = Fraction(0, 1) for (number, audiofile) in enumerate(audiofiles, 1): sheet_tracks.append( SheetTrack(number=number, track_indexes=[SheetIndex(number=1, offset=offset)], metadata=audiofile.get_metadata(), filename=filename)) offset += audiofile.seconds_length() return cls(sheet_tracks=sheet_tracks, metadata=None) def __repr__(self): return "Sheet(sheet_tracks={!r}, metadata={!r})".format( self.__sheet_tracks__, self.__metadata__) def __len__(self): return len(self.__sheet_tracks__) def __getitem__(self, index): return self.__sheet_tracks__[index] def __eq__(self, sheet): try: if self.get_metadata() != sheet.get_metadata(): return False if len(self) != len(sheet): return False for (t1, t2) in zip(self, sheet): if t1 != t2: return False else: return True except (AttributeError, TypeError): return False def track_numbers(self): """returns a list of all track numbers in the sheet""" return [track.number() for track in self] def track(self, track_number): """given a track_number (typically starting from 1), returns a SheetTrack object or raises KeyError if not found""" for track in self: if track_number == track.number(): return track else: raise KeyError(track_number) def pre_gap(self): """returns the pre-gap of the entire disc as a Fraction number of seconds""" indexes = self.track(1) if (indexes[0].number() == 0) and (indexes[1].number() == 1): return (indexes[1].offset() - indexes[0].offset()) else: from fractions import Fraction return Fraction(0, 1) def track_offset(self, track_number): """given a track_number (typically starting from 1) returns the offset to that track from the start of the stream as a Fraction number of seconds may raise KeyError if the track is not found""" return self.track(track_number).index(1).offset() def track_length(self, track_number, total_length=None): """given a track_number (typically starting from 1) and optional total length as a Fraction number of seconds (including the disc's pre-gap, if any), returns the length of the track as a Fraction number of seconds or None if the length is to the remainder of the stream (typically for the last track in the album) may raise KeyError if the track is not found""" initial_track = self.track(track_number) if (track_number + 1) in self.track_numbers(): next_track = self.track(track_number + 1) return (next_track.index(1).offset() - initial_track.index(1).offset()) elif total_length is not None: return total_length - initial_track.index(1).offset() else: # no next track, so total length is unknown return None def image_formatted(self): """returns True if all tracks are for the same file and have ascending index points""" initial_filename = None previous_index = None for track in self: if initial_filename is None: initial_filename = track.filename() elif initial_filename != track.filename(): return False for index in track: if previous_index is None: previous_index = index.offset() elif previous_index >= index.offset(): return False else: previous_index = index.offset() else: return True def get_metadata(self): """returns MetaData of Sheet, or None this metadata often contains information such as catalog number or CD-TEXT values""" return self.__metadata__ class SheetTrack(object): def __init__(self, number, track_indexes, metadata=None, filename=u"CDImage.wav", is_audio=True, pre_emphasis=False, copy_permitted=False): """ | argument | type | value | |----------------+--------------+---------------------------------------| | number | int | track number, starting from 1 | | track_indexes | [SheetIndex] | list of SheetIndex objects | | metadata | MetaData | track's metadata, or None | | filename | unicode | track's filename on disc | | is_audio | boolean | whether track contains audio data | | pre_emphasis | boolean | whether track has pre-emphasis | | copy_permitted | boolean | whether copying is permitted | """ assert(isinstance(number, int)) assert(isinstance(filename, str if PY3 else unicode)) self.__number__ = number self.__track_indexes__ = list(track_indexes) self.__metadata__ = metadata self.__filename__ = filename self.__is_audio__ = is_audio self.__pre_emphasis__ = pre_emphasis self.__copy_permitted__ = copy_permitted @classmethod def converted(cls, sheet_track): """Given a SheetTrack-compatible object, returns a SheetTrack""" return cls( number=sheet_track.number(), track_indexes=[SheetIndex.converted(i) for i in sheet_track], metadata=sheet_track.get_metadata(), filename=sheet_track.filename(), is_audio=sheet_track.is_audio(), pre_emphasis=sheet_track.pre_emphasis(), copy_permitted=sheet_track.copy_permitted()) def __repr__(self): return "SheetTrack({})".format( ", ".join(["{}={!r}".format(attr, getattr(self, "__" + attr + "__")) for attr in ["number", "track_indexes", "metadata", "filename", "is_audio", "pre_emphasis", "copy_permitted"]])) def __len__(self): return len(self.__track_indexes__) def __getitem__(self, i): return self.__track_indexes__[i] def indexes(self): """returns a list of all indexes in the current track""" return [index.number() for index in self] def index(self, index_number): """given an index_number (0 for pre-gap, 1 for track start, etc.) returns a SheetIndex object or raises KeyError if not found""" for sheet_index in self: if index_number == sheet_index.number(): return sheet_index else: raise KeyError(index_number) def __eq__(self, sheet_track): try: for method in ["number", "is_audio", "pre_emphasis", "copy_permitted"]: if getattr(self, method)() != getattr(sheet_track, method)(): return False if len(self) != len(sheet_track): return False for (t1, t2) in zip(self, sheet_track): if t1 != t2: return False else: return True except (AttributeError, TypeError): return False def __ne__(self, sheet_track): return not self.__eq__(sheet_track) def number(self): """return SheetTrack's number, starting from 1""" return self.__number__ def get_metadata(self): """returns SheetTrack's MetaData, or None""" return self.__metadata__ def filename(self): """returns SheetTrack's filename as unicode""" return self.__filename__ def is_audio(self): """returns whether SheetTrack contains audio data""" return self.__is_audio__ def pre_emphasis(self): """returns whether SheetTrack has pre-emphasis""" return self.__pre_emphasis__ def copy_permitted(self): """returns whether copying is permitted""" return self.__copy_permitted__ class SheetIndex(object): def __init__(self, number, offset): """number is the index number, 0 for pre-gap index offset is the offset from the start of the stream as a Fraction number of seconds""" self.__number__ = number self.__offset__ = offset @classmethod def converted(cls, sheet_index): """given a SheetIndex-compatible object, returns a SheetIndex""" return cls(number=sheet_index.number(), offset=sheet_index.offset()) def __repr__(self): return "SheetIndex(number={}, offset={})".format(self.__number__, self.__offset__) def __eq__(self, sheet_index): try: return ((self.number() == sheet_index.number()) and (self.offset() == sheet_index.offset())) except (TypeError, AttributeError): return False def __ne__(self, sheet_index): return not self.__eq__(sheet_index) def number(self): return self.__number__ def offset(self): return self.__offset__ def has_pre_gap_track(audiofiles): """given a list of AudioFile objects of a single disc in order, returns True if audiofiles[0] appears to be a pre-gap track based on track numbers in metadata lists shorter than 2 entries will never have a pre-gap track """ if len(audiofiles) < 2: # either 0 or 1 file, so not enough for one to be a pre-gap return False metadatas = [f.get_metadata() for f in audiofiles] if None in metadatas: # all metadata must be populated return False if len({m.track_total for m in metadatas}) > 1: # track total should be identical (or None) return False if metadatas[0].track_number not in {0, None}: # initial track should be numbered 0, or None return False if ([m.track_number for m in metadatas[1:]] != list(range(1, len(metadatas)))): # the remainder should be in ascending order with no gaps return False return True def iter_first(iterator): """yields a (is_first, item) per item in the iterator where is_first indicates whether the item is the first one if the iterator has no items, yields (True, None) """ for (i, v) in enumerate(iterator): yield ((i == 0), v) def iter_last(iterator): """yields a (is_last, item) per item in the iterator where is_last indicates whether the item is the final one if the iterator has no items, yields (True, None) """ iterator = iter(iterator) try: cached_item = next(iterator) except StopIteration: return while True: try: next_item = next(iterator) yield (False, cached_item) cached_item = next_item except StopIteration: yield (True, cached_item) return def PCMReaderWindow(pcmreader, initial_offset, pcm_frames, forward_close=True): """pcmreader is the parent stream initial offset is the offset of the stream's beginning, which may be negative pcm_frames is the total length of the stream if forward_close is True, calls to .close() are forwarded to the parent stream, otherwise the parent is left as-is""" if initial_offset == 0: return PCMReaderHead( pcmreader=pcmreader, pcm_frames=pcm_frames, forward_close=forward_close) else: return PCMReaderHead( pcmreader=PCMReaderDeHead(pcmreader=pcmreader, pcm_frames=initial_offset, forward_close=forward_close), pcm_frames=pcm_frames, forward_close=forward_close) class PCMReaderHead(PCMReader): """a wrapper around PCMReader for truncating a stream's ending""" def __init__(self, pcmreader, pcm_frames, forward_close=True): """pcmreader is a PCMReader object pcm_frames is the total number of PCM frames in the stream if pcm_frames is shorter than the pcmreader's stream, the stream will be truncated if pcm_frames is longer than the pcmreader's stream, the stream will be extended with additional empty frames if forward_close is True, calls to .close() are forwarded to the parent stream, otherwise the parent is left as-is """ if pcm_frames < 0: raise ValueError("invalid pcm_frames value") PCMReader.__init__(self, sample_rate=pcmreader.sample_rate, channels=pcmreader.channels, channel_mask=pcmreader.channel_mask, bits_per_sample=pcmreader.bits_per_sample) self.pcmreader = pcmreader self.pcm_frames = pcm_frames self.forward_close = forward_close def __repr__(self): return "PCMReaderHead({!r}, {!r})".format(self.pcmreader, self.pcm_frames) def read(self, pcm_frames): if self.pcm_frames > 0: # data left in window # so try to read an additional frame from PCMReader frame = self.pcmreader.read(pcm_frames) if frame.frames == 0: # no additional data in PCMReader, # so return empty frames leftover in window # and close window frame = pcm.from_list([0] * (self.pcm_frames * self.channels), self.channels, self.bits_per_sample, True) self.pcm_frames -= frame.frames return frame elif frame.frames <= self.pcm_frames: # frame is shorter than remaining window, # so shrink window and return frame unaltered self.pcm_frames -= frame.frames return frame else: # frame is larger than remaining window, # so cut off end of frame # close window and return shrunk frame frame = frame.split(self.pcm_frames)[0] self.pcm_frames -= frame.frames return frame else: # window exhausted, so return empty framelist return pcm.empty_framelist(self.channels, self.bits_per_sample) def read_closed(self, pcm_frames): raise ValueError() def close(self): if self.forward_close: self.pcmreader.close() self.read = self.read_closed class PCMReaderDeHead(PCMReader): """a wrapper around PCMReader for truncating a stream's beginning""" def __init__(self, pcmreader, pcm_frames, forward_close=True): """pcmreader is a PCMReader object pcm_frames is the total number of PCM frames to remove if pcm_frames is positive, that amount of frames will be removed from the beginning of the stream if pcm_frames is negative, the stream will be padded with that many PCM frames if forward_close is True, calls to .close() are forwarded to the parent stream, otherwise the parent is left as-is """ PCMReader.__init__(self, sample_rate=pcmreader.sample_rate, channels=pcmreader.channels, channel_mask=pcmreader.channel_mask, bits_per_sample=pcmreader.bits_per_sample) self.pcmreader = pcmreader self.pcm_frames = pcm_frames self.forward_close = forward_close def __repr__(self): return "PCMReaderDeHead({!r}, {!r})".format(self.pcmreader, self.pcm_frames) def read(self, pcm_frames): if self.pcm_frames == 0: # no truncation or padding, so return framelists as-is return self.pcmreader.read(pcm_frames) elif self.pcm_frames > 0: # remove PCM frames from beginning of stream # until all truncation is accounted for while self.pcm_frames > 0: frame = self.pcmreader.read(pcm_frames) if frame.frames == 0: # truncation longer than entire stream # so don't try to truncate it any further self.pcm_frames = 0 return frame elif frame.frames <= self.pcm_frames: self.pcm_frames -= frame.frames else: (head, tail) = frame.split(self.pcm_frames) self.pcm_frames -= head.frames assert(self.pcm_frames == 0) assert(tail.frames > 0) return tail else: return self.pcmreader.read(pcm_frames) else: # pad beginning of stream with empty PCM frames frame = pcm.from_list([0] * (-self.pcm_frames) * self.channels, self.channels, self.bits_per_sample, True) assert(frame.frames == -self.pcm_frames) self.pcm_frames = 0 return frame def read_closed(self, pcm_frames): raise ValueError() def close(self): if self.forward_close: self.pcmreader.close() self.read = self.read_closed # returns the value in item_list which occurs most often def most_numerous(item_list, empty_list=None, all_differ=None): """returns the value in the item list which occurs most often if list has no items, returns 'empty_list' if all items differ, returns 'all_differ'""" counts = {} if len(item_list) == 0: return empty_list for item in item_list: counts.setdefault(item, []).append(item) (item, max_count) = sorted([(item, len(counts[item])) for item in counts.keys()], key=lambda pair: pair[1])[-1] if (max_count < len(item_list)) and (max_count == 1): return all_differ else: return item def metadata_lookup(musicbrainz_disc_id, musicbrainz_server="musicbrainz.org", musicbrainz_port=80, use_musicbrainz=True): """generates set of MetaData objects from a pair of disc IDs returns a metadata[c][t] list of lists where 'c' is a possible choice and 't' is the MetaData for a given track (starting from 0) this will always return at least one choice, which may be a list of largely empty MetaData objects if no match can be found for the disc IDs """ matches = [] if use_musicbrainz: import audiotools.musicbrainz as musicbrainz try: from urllib.request import HTTPError except ImportError: from urllib2 import HTTPError from xml.parsers.expat import ExpatError try: matches.extend( musicbrainz.perform_lookup( disc_id=musicbrainz_disc_id, musicbrainz_server=musicbrainz_server, musicbrainz_port=musicbrainz_port)) except (HTTPError, ExpatError): pass if len(matches) == 0: # no matches, so build a set of dummy metadata track_count = len(musicbrainz_disc_id.offsets) return [[MetaData(track_number=i, track_total=track_count) for i in range(1, track_count + 1)]] else: return matches def cddareader_metadata_lookup(cddareader, musicbrainz_server="musicbrainz.org", musicbrainz_port=80, use_musicbrainz=True): """given a CDDAReader object returns a metadata[c][t] list of lists where 'c' is a possible choice and 't' is the MetaData for a given track (starting from 0) this will always return at least once choice, which may be a list of largely empty MetaData objects if no match can be found for the CD """ from audiotools.musicbrainz import DiscID as MDiscID return metadata_lookup( musicbrainz_disc_id=MDiscID.from_cddareader(cddareader), musicbrainz_server=musicbrainz_server, musicbrainz_port=musicbrainz_port, use_musicbrainz=use_musicbrainz) def track_metadata_lookup(audiofiles, musicbrainz_server="musicbrainz.org", musicbrainz_port=80, use_musicbrainz=True): """given a list of AudioFile objects, this treats them as a single CD and generates a set of MetaData objects pulled from lookup services returns a metadata[c][t] list of lists where 'c' is a possible choice and 't' is the MetaData for a given track (starting from 0) this will always return at least one choice, which may be a list of largely empty MetaData objects if no match can be found for the CD """ from audiotools.musicbrainz import DiscID as MDiscID if not has_pre_gap_track(audiofiles): return metadata_lookup( musicbrainz_disc_id=MDiscID.from_tracks(audiofiles), musicbrainz_server=musicbrainz_server, musicbrainz_port=musicbrainz_port, use_musicbrainz=use_musicbrainz) else: def merge_metadatas(metadatas): if len(metadatas) == 0: return None elif len(metadatas) == 1: return metadatas[0] else: merged = metadatas[0] for to_merge in metadatas[1:]: merged = merged.intersection(to_merge) return merged # prepend "track 0" pre-gap data to start of list for each choice choices = [] for choice in metadata_lookup( musicbrainz_disc_id=MDiscID.from_tracks(audiofiles), musicbrainz_server=musicbrainz_server, musicbrainz_port=musicbrainz_port, use_musicbrainz=use_musicbrainz): track_0 = merge_metadatas(choice) track_0.track_number = 0 choices.append([track_0] + choice) return choices def sheet_metadata_lookup(sheet, total_pcm_frames, sample_rate, musicbrainz_server="musicbrainz.org", musicbrainz_port=80, use_musicbrainz=True): """given a Sheet object, length of the album in PCM frames and sample rate of the disc, returns a metadata[c][t] list of lists where 'c' is a possible choice and 't' is the MetaData for a given track (starting from 0) this will always return at least one choice, which may be a list of largely empty MetaData objects if no match can be found for the CD """ from audiotools.musicbrainz import DiscID as MDiscID return metadata_lookup( musicbrainz_disc_id=MDiscID.from_sheet(sheet, total_pcm_frames, sample_rate), musicbrainz_server=musicbrainz_server, musicbrainz_port=musicbrainz_port, use_musicbrainz=use_musicbrainz) def accuraterip_lookup(sorted_tracks, accuraterip_server="www.accuraterip.com", accuraterip_port=80): """given a list of sorted AudioFile objects and optional AccurateRip server and port returns a dict of {track_number:[(confidence, crc, crc2), ...], ...} where track_number starts from 1 may return a dict of empty lists if no AccurateRip entry is found may raise urllib2.HTTPError if an error occurs querying the server """ if len(sorted_tracks) == 0: return {} else: from audiotools.accuraterip import DiscID, perform_lookup return perform_lookup(DiscID.from_tracks(sorted_tracks), accuraterip_server, accuraterip_port) def accuraterip_sheet_lookup(sheet, total_pcm_frames, sample_rate, accuraterip_server="www.accuraterip.com", accuraterip_port=80): """given a Sheet object, total number of PCM frames and sample rate returns a dict of {track_number:[(confidence, crc, crc2), ...], ...} where track_number starts from 1 may return a dict of empty lists if no AccurateRip entry is found may raise urllib2.HTTPError if an error occurs querying the server """ from audiotools.accuraterip import DiscID, perform_lookup return perform_lookup(DiscID.from_sheet(sheet, total_pcm_frames, sample_rate), accuraterip_server, accuraterip_port) def output_progress(u, current, total): """given a unicode string and current/total integers, returns a u'[<current>/<total>] <string>' unicode string indicating the current progress""" if total > 1: return u"[{{:{:d}d}}/{{:d}}] {{}}".format( len(str(total))).format(current, total, u) else: return u class ExecProgressQueue(object): """a class for running multiple jobs in parallel with progress updates""" def __init__(self, messenger): """takes a Messenger object""" from collections import deque self.messenger = messenger self.__displayed_rows__ = {} self.__queued_jobs__ = deque() self.__raised_exception__ = None def execute(self, function, progress_text=None, completion_output=None, *args, **kwargs): """queues the given function and arguments to be run in parallel function must have an additional "progress" argument not present in "*args" or "**kwargs" which is called with (current, total) integer arguments by the function on a regular basis to update its progress similar to: function(*args, progress=prog(current, total), **kwargs) progress_text should be a unicode string to be displayed while running completion_output is either a unicode string, or a function which takes the result of the queued function and returns a unicode string for display once the queued function is complete """ self.__queued_jobs__.append((len(self.__queued_jobs__), progress_text, completion_output, function, args, kwargs)) def run(self, max_processes=1): """runs all the queued jobs and returns the result of queued functions as a list""" if len(self.__queued_jobs__) == 0: # nothing to do return [] elif (max_processes == 1) or (len(self.__queued_jobs__) == 1): # perform one job at a time return self.__run_serial__() else: # perform multiple jobs in parallel return self.__run_parallel__(max_processes=max_processes) def __run_serial__(self): """runs all the queued jobs in serial""" results = [] total_jobs = len(self.__queued_jobs__) # pull parameters from job queue for (completed_job_number, (job_index, progress_text, completion_output, function, args, kwargs)) in enumerate(self.__queued_jobs__, 1): # add job to progress display, if any text to display if progress_text is not None: progress_display = SingleProgressDisplay(self.messenger, progress_text) # execute job with displayed progress result = function(*args, progress=progress_display.update, **kwargs) # remove job from progress display, if present progress_display.clear_rows() else: result = function(*args, **kwargs) # add result to results list results.append(result) # display any output message attached to job if callable(completion_output): output = completion_output(result) else: output = completion_output if output is not None: self.messenger.output(output_progress(output, completed_job_number, total_jobs)) self.__queued_jobs__.clear() return results def __run_parallel__(self, max_processes=1): """runs all the queued jobs in parallel""" from select import select def execute_next_job(progress_display): """pulls the next job from the queue and returns a (Process, Array, Connection, progress_text, completed_text) tuple where Process is the subprocess Array is shared memory of the current progress Connection is the listening end of a pipe progress_text is unicode to display during progress and completed_text is unicode to display when finished""" # pull parameters from job queue (job_index, progress_text, completion_output, function, args, kwargs) = self.__queued_jobs__.popleft() # spawn new __ProgressQueueJob__ object job = __ProgressQueueJob__.spawn( job_index=job_index, function=function, args=args, kwargs=kwargs, progress_text=progress_text, completion_output=completion_output) # add job to progress display, if any text to display if progress_text is not None: self.__displayed_rows__[job.job_fd()] = \ progress_display.add_row(progress_text) return job progress_display = ProgressDisplay(self.messenger) # variables for X/Y output display # Note that the order a job is inserted into the queue # (as captured by its job_index value) # may differ from the order in which it is completed. total_jobs = len(self.__queued_jobs__) completed_job_number = 1 # a dict of job file descriptors -> __ProgressQueueJob__ objects job_pool = {} # return values from the executed functions results = [None] * total_jobs if total_jobs == 0: # nothing to do return results # populate job pool up to "max_processes" number of jobs for i in range(min(max_processes, len(self.__queued_jobs__))): job = execute_next_job(progress_display) job_pool[job.job_fd()] = job # while the pool still contains running jobs try: while len(job_pool) > 0: # wait for zero or more jobs to finish (may timeout) (rlist, wlist, elist) = select(job_pool.keys(), [], [], 0.25) # clear out old display progress_display.clear_rows() for finished_job in [job_pool[fd] for fd in rlist]: job_fd = finished_job.job_fd() (exception, result) = finished_job.result() if not exception: # job completed successfully # display any output message attached to job completion_output = finished_job.completion_output if callable(completion_output): output = completion_output(result) else: output = completion_output if output is not None: progress_display.output_line( output_progress(output, completed_job_number, total_jobs)) # attach result to output in the order it was received results[finished_job.job_index] = result else: # job raised an exception # remove all other jobs from queue # then raise exception to caller # once working jobs are finished self.__raised_exception__ = result while len(self.__queued_jobs__) > 0: self.__queued_jobs__.popleft() # remove job from pool del(job_pool[job_fd]) # remove job from progress display, if present if job_fd in self.__displayed_rows__: self.__displayed_rows__[job_fd].finish() # add new jobs from the job queue, if any if len(self.__queued_jobs__) > 0: job = execute_next_job(progress_display) job_pool[job.job_fd()] = job # updated completed job number for X/Y display completed_job_number += 1 # update progress rows with progress taken from shared memory for job in job_pool.values(): if job.job_fd() in self.__displayed_rows__: self.__displayed_rows__[job.job_fd()].update( job.progress()) # display new set of progress rows progress_display.display_rows() except: # an exception occurred (perhaps KeyboardInterrupt) # so kill any running child jobs # clear any progress rows progress_display.clear_rows() # and pass exception to caller raise # if any jobs have raised an exception, # re-raise it in the main process if self.__raised_exception__ is not None: raise self.__raised_exception__ else: # otherwise, return results in the order they were queued return results class __ProgressQueueJob__(object): """this class is a the parent process end of a running child job""" def __init__(self, job_index, process, progress, lock, result_pipe, progress_text, completion_output): """job_index is the order this job was inserted into the queue process is the Process object of the running child progress is an Array object of [current, total] progress status lock is a Lock object to synchronize access to "progress" result_pipe is a Connection object which will be read for data progress_text is unicode to display while the job is in progress completion_output is either unicode or a callable function to be displayed when the job finishes """ self.job_index = job_index self.process = process self.__progress__ = progress self.__lock__ = lock self.result_pipe = result_pipe self.progress_text = progress_text self.completion_output = completion_output def job_fd(self): """returns file descriptor of parent-side result pipe""" return self.result_pipe.fileno() def progress(self): self.__lock__.acquire() (current, total) = self.__progress__ self.__lock__.release() if total > 0: return Fraction(current, total) else: return Fraction(0, 1) @classmethod def spawn(cls, job_index, function, args, kwargs, progress_text, completion_output): """spawns a subprocess and returns the parent-side __ProgressQueueJob__ object job_index is the order this job was inserted into the queue function is the function to execute args is a tuple of positional arguments kwargs is a dict of keyword arguments progress_text is unicode to display while the job is in progress completion_output is either unicode or a callable function to be displayed when the job finishes """ class __progress__(object): def __init__(self, memory, lock): self.memory = memory self.lock = lock def update(self, progress): self.lock.acquire() self.memory[0] = progress.numerator self.memory[1] = progress.denominator self.lock.release() def execute_job(function, args, kwargs, progress, result_pipe): try: result_pipe.send((False, function(*args, progress=progress, **kwargs))) except Exception as exception: result_pipe.send((True, exception)) result_pipe.close() from multiprocessing import Process, Array, Pipe, Lock # construct shared memory array to store progress progress = Array("L", [0, 1]) # construct lock to keep array access process-safe lock = Lock() # construct one-way pipe to collect result (parent_conn, child_conn) = Pipe(False) # build child job to execute function process = Process(target=execute_job, args=(function, args, kwargs, __progress__(progress, lock).update, child_conn)) # start child job process.start() # return populated __ProgressQueueJob__ object return cls(job_index=job_index, process=process, progress=progress, lock=lock, result_pipe=parent_conn, progress_text=progress_text, completion_output=completion_output) def result(self): """returns (exception, result) from parent-side pipe where exception is True if result is an exception or False if it's the result of the called child function""" (exception, result) = self.result_pipe.recv() self.result_pipe.close() self.process.join() return (exception, result) class TemporaryFile(object): """a class for staging file rewrites""" def __init__(self, original_filename): """original_filename is the path of the file to be rewritten with new data""" from tempfile import mkstemp self.__original_filename__ = original_filename (dirname, basename) = os.path.split(original_filename) (fd, self.__temp_path__) = mkstemp(prefix="." + basename, dir=dirname) self.__temp_file__ = os.fdopen(fd, "wb") def __del__(self): if (((self.__temp_path__ is not None) and os.path.isfile(self.__temp_path__))): os.unlink(self.__temp_path__) def write(self, data): """writes the given data string to the temporary file""" self.__temp_file__.write(data) def flush(self): """flushes pending data to stream""" self.__temp_file__.flush() def tell(self): """returns current file position""" return self.__temp_file__.tell() def seek(self, offset, whence=None): """move to new file position""" if whence is not None: self.__temp_file__.seek(offset, whence) else: self.__temp_file__.seek(offset) def close(self): """commits all staged changes the original file is overwritten, its file mode is preserved and the temporary file is closed and deleted""" self.__temp_file__.close() original_mode = os.stat(self.__original_filename__).st_mode try: os.rename(self.__temp_path__, self.__original_filename__) os.chmod(self.__original_filename__, original_mode) self.__temp_path__ = None except OSError as err: os.unlink(self.__temp_path__) raise err from audiotools.au import AuAudio from audiotools.wav import WaveAudio from audiotools.aiff import AiffAudio from audiotools.flac import FlacAudio from audiotools.wavpack import WavPackAudio from audiotools.mp3 import MP3Audio from audiotools.mp3 import MP2Audio from audiotools.vorbis import VorbisAudio from audiotools.speex import SpeexAudio from audiotools.m4a import M4AAudio from audiotools.m4a import ALACAudio from audiotools.opus import OpusAudio from audiotools.tta import TrueAudio from audiotools.mpc import MPCAudio from audiotools.ape import ApeTag from audiotools.flac import FlacMetaData from audiotools.id3 import ID3CommentPair from audiotools.id3v1 import ID3v1Comment from audiotools.id3 import ID3v22Comment from audiotools.id3 import ID3v23Comment from audiotools.id3 import ID3v24Comment from audiotools.m4a_atoms import M4A_META_Atom from audiotools.vorbiscomment import VorbisComment AVAILABLE_TYPES = (FlacAudio, MP3Audio, MP2Audio, WaveAudio, VorbisAudio, SpeexAudio, AiffAudio, AuAudio, M4AAudio, ALACAudio, WavPackAudio, OpusAudio, TrueAudio, MPCAudio) TYPE_MAP = {track_type.NAME: track_type for track_type in AVAILABLE_TYPES} DEFAULT_QUALITY = {track_type.NAME: config.get_default("Quality", track_type.NAME, track_type.DEFAULT_COMPRESSION) for track_type in AVAILABLE_TYPES if (len(track_type.COMPRESSION_MODES) > 1)} if DEFAULT_TYPE not in TYPE_MAP.keys(): DEFAULT_TYPE = "wav" ================================================ FILE: audiotools/accuraterip.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools import PY3 from audiotools._accuraterip import Checksum class __Checksum__(object): """Python implementation of checksum calculator""" def __init__(self, total_pcm_frames, sample_rate=44100, is_first=False, is_last=False, pcm_frame_range=1): if total_pcm_frames <= 0: raise ValueError("total PCM frames must be > 0") if sample_rate <= 0: raise ValueError("sample rate must be > 0") if pcm_frame_range <= 0: raise ValueError("PCM frame range must be > 0") self.__total_pcm_frames__ = total_pcm_frames self.__pcm_frame_range__ = pcm_frame_range self.__values__ = [] if is_first: self.__start_offset__ = ((sample_rate // 75) * 5) else: self.__start_offset__ = 1 if is_last: self.__end_offset__ = (total_pcm_frames - ((sample_rate // 75) * 5)) else: self.__end_offset__ = total_pcm_frames def update(self, framelist): from itertools import izip from audiotools.pcm import FrameList def value(l, r): return (unsigned(r) << 16) | unsigned(l) def unsigned(v): return (v if (v >= 0) else ((1 << 16) - (-v))) if not isinstance(framelist, FrameList): raise TypeError("framelist must be instance of Framelist") elif framelist.channels != 2: raise ValueError("FrameList must have 2 channels") elif framelist.bits_per_sample != 16: raise ValueError("FrameList must have 16 bits-per-sample") if ((len(self.__values__) + framelist.frames) > (self.__total_pcm_frames__ + self.__pcm_frame_range__ - 1)): raise ValueError("too many samples for checksum") self.__values__.extend( [value(l, r) for (l, r) in izip(framelist.channel(0), framelist.channel(1))]) def checksums_v1(self): if (len(self.__values__) < (self.__total_pcm_frames__ + self.__pcm_frame_range__ - 1)): raise ValueError("insufficient samples for checksum") return [sum([(v * i) if ((i >= self.__start_offset__) and (i <= self.__end_offset__)) else 0 for (i, v) in enumerate(self.__values__[r: r + self.__total_pcm_frames__], 1)]) & 0xFFFFFFFF for r in range(self.__pcm_frame_range__)] def checksums_v2(self): if (len(self.__values__) < (self.__total_pcm_frames__ + self.__pcm_frame_range__ - 1)): raise ValueError("insufficient samples for checksum") def combine(x): return (x >> 32) + (x & 0xFFFFFFFF) return [sum([combine(v * i) if ((i >= self.__start_offset__) and (i <= self.__end_offset__)) else 0 for (i, v) in enumerate(self.__values__[r: r + self.__total_pcm_frames__], 1)]) & 0xFFFFFFFF for r in range(self.__pcm_frame_range__)] def match_offset(ar_matches, checksums, initial_offset): """ar_matches is a list of (confidence, crc, crc2) tuples checksums is a list of calculated AccurateRip crcs initial_offset is the starting offset of the checksums returns (checksum, confidence, offset) of the best match found if no matches are found, confidence is None and offset is 0 """ if len(checksums) == 0: raise ValueError("at least 1 checksum is required") # crc should be unique in the list # but confidence may not be matches = {crc: confidence for (confidence, crc, crc2) in ar_matches} offsets = {crc: offset for (offset, crc) in enumerate(checksums, initial_offset)} match_offsets = sorted( [(crc, matches[crc], offsets[crc]) for crc in set(matches.keys()) & set(offsets.keys())], key=lambda triple: triple[1]) if len(match_offsets) > 0: # choose the match with the highest confidence return match_offsets[-1] else: # no match found # return checksum at offset 0, or as close as possible if initial_offset <= 0: return (checksums[-initial_offset], None, 0) else: return (checksums[0], None, initial_offset) class DiscID(object): def __init__(self, track_numbers, track_offsets, lead_out_offset, freedb_disc_id): """track_numbers is a list of track numbers, starting from 1 track_offsets is a list of offsets, in CD frames and typically starting from 0 lead_out_offset is the offset of the lead-out track, in CD frames freedb_disc_id is a string or freedb.DiscID object of the CD """ assert(len(track_numbers) == len(track_offsets)) self.__track_numbers__ = track_numbers self.__track_offsets__ = track_offsets self.__lead_out_offset__ = lead_out_offset self.__freedb_disc_id__ = freedb_disc_id @classmethod def from_cddareader(cls, cddareader): """given a CDDAReader object, returns a DiscID for that object""" from audiotools.freedb import DiscID as FDiscID offsets = cddareader.track_offsets return cls(track_numbers=list(sorted(offsets.keys())), track_offsets=[(offsets[k] // 588) for k in sorted(offsets.keys())], lead_out_offset=cddareader.last_sector + 1, freedb_disc_id=FDiscID.from_cddareader(cddareader)) @classmethod def from_tracks(cls, tracks): """given a sorted list of AudioFile objects, returns DiscID for those tracks as if they were a CD""" from audiotools import has_pre_gap_track from audiotools.freedb import DiscID as FDiscID if not has_pre_gap_track(tracks): offsets = [0] for track in tracks[0:-1]: offsets.append(offsets[-1] + track.cd_frames()) return cls(track_numbers=range(1, len(tracks) + 1), track_offsets=offsets, lead_out_offset=sum(t.cd_frames() for t in tracks), freedb_disc_id=FDiscID.from_tracks(tracks)) else: offsets = [tracks[0].cd_frames()] for track in tracks[1:-1]: offsets.append(offsets[-1] + track.cd_frames()) return cls(track_numbers=range(1, len(tracks)), track_offsets=offsets, lead_out_offset=sum(t.cd_frames() for t in tracks), freedb_disc_id=FDiscID.from_tracks(tracks)) @classmethod def from_sheet(cls, sheet, total_pcm_frames, sample_rate): """given a Sheet object length of the album in PCM frames and sample rate of the disc, returns a DiscID for that CD""" from audiotools.freedb import DiscID as FDiscID return cls(track_numbers=range(1, len(sheet) + 1), track_offsets=[(int(t.index(1).offset() * 75)) for t in sheet], lead_out_offset=total_pcm_frames * 75 // sample_rate, freedb_disc_id=FDiscID.from_sheet(sheet, total_pcm_frames, sample_rate)) def track_numbers(self): return self.__track_numbers__[:] def id1(self): return sum(self.__track_offsets__) + self.__lead_out_offset__ def id2(self): return (sum([n * max(o, 1) for (n, o) in zip(self.__track_numbers__, self.__track_offsets__)]) + (max(self.__track_numbers__) + 1) * self.__lead_out_offset__) def freedb_disc_id(self): return int(self.__freedb_disc_id__) if PY3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('utf-8') def __unicode__(self): return u"dBAR-{tracks:03d}-{id1:08x}-{id2:08x}-{freedb:08x}.bin".format( tracks=len(self.__track_numbers__), id1=self.id1(), id2=self.id2(), freedb=int(self.__freedb_disc_id__)) def __repr__(self): return "AccurateRipDiscID({})".format( ", ".join(["{}={!r}".format(key, getattr(self, attr)) for (key, attr) in [("track_numbers", "__track_numbers__"), ("track_offsets", "__track_offsets__"), ("lead_out_offset", "__lead_out_offset__"), ("freedb_disc_id", "__freedb_disc_id__")]])) def perform_lookup(disc_id, accuraterip_server="www.accuraterip.com", accuraterip_port=80): """performs web-based lookup using the given DiscID object and returns a dict of {track_number:[(confidence, crc, crc2), ...], ...} where track_number starts from 1 may return a dict of empty lists if no AccurateRip entry is found may raise urllib2.HTTPError if an error occurs querying the server """ from audiotools.bitstream import BitstreamReader try: from urllib.request import urlopen, URLError except ImportError: from urllib2 import urlopen, URLError matches = {n: [] for n in disc_id.track_numbers()} url = "http://{}:{}/accuraterip/{}/{}/{}/{}".format(accuraterip_server, accuraterip_port, str(disc_id)[16], str(disc_id)[15], str(disc_id)[14], disc_id) try: response = BitstreamReader(urlopen(url), True) except URLError: # no CD found matching given parameters return matches try: while True: (track_count, id1, id2, freedb_disc_id) = response.parse("8u 32u 32u 32u") if (((id1 == disc_id.id1()) and (id2 == disc_id.id2()) and (freedb_disc_id == disc_id.freedb_disc_id()))): for track_number in range(1, track_count + 1): if track_number in matches: matches[track_number].append( tuple(response.parse("8u 32u 32u"))) except IOError: # keep trying to parse values until the data runs out response.close() return matches ================================================ FILE: audiotools/aiff.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools import (InvalidFile, PCMReader, AiffContainer) from audiotools.pcm import FrameList import sys import struct def parse_ieee_extended(bitstream): """returns a parsed 80-bit IEEE extended value from BitstreamReader this is used to handle AIFF's sample rate field""" (signed, exponent, mantissa) = bitstream.parse("1u 15u 64U") if (exponent == 0) and (mantissa == 0): return 0 elif exponent == 0x7FFF: return 1.79769313486231e+308 else: f = mantissa * (2.0 ** (exponent - 16383 - 63)) return f if not signed else -f def build_ieee_extended(bitstream, value): """writes an 80-bit IEEE extended value to BitstreamWriter this is used to handle AIFF's sample rate field""" from math import frexp if value < 0: signed = 1 value = abs(value) else: signed = 0 (fmant, exponent) = frexp(value) if (exponent > 16384) or (fmant >= 1): exponent = 0x7FFF mantissa = 0 else: exponent += 16382 mantissa = fmant * (2 ** 64) bitstream.build("1u 15u 64U", (signed, exponent, int(mantissa))) def pad_data(pcm_frames, channels, bits_per_sample): """returns True if the given stream combination requires an extra padding byte at the end of the 'data' chunk""" return (pcm_frames * channels * (bits_per_sample // 8)) % 2 def validate_header(header): """given header string as returned by aiff_header_footer() returns (total size, ssnd size) where total size is the size of the file in bytes and ssnd size is the size of the SSND chunk in bytes (including the 8 prefix bytes in the chunk but *not* including any padding byte at the end) the size of the SSND chunk and of the total file should be validated after the file has been completely written such that len(header) + len(SSND chunk) + len(footer) = total size raises ValueError if the header is invalid""" from io import BytesIO from audiotools.bitstream import BitstreamReader header_size = len(header) aiff_file = BitstreamReader(BytesIO(header), False) try: # ensure header starts with FORM<size>AIFF chunk (form, remaining_size, aiff) = aiff_file.parse("4b 32u 4b") if form != b"FORM": from audiotools.text import ERR_AIFF_NOT_AIFF raise ValueError(ERR_AIFF_NOT_AIFF) elif aiff != b"AIFF": from audiotools.text import ERR_AIFF_INVALID_AIFF raise ValueError(ERR_AIFF_INVALID_AIFF) else: total_size = remaining_size + 8 header_size -= 12 comm_found = False while header_size > 0: # ensure each chunk header is valid (chunk_id, chunk_size) = aiff_file.parse("4b 32u") if frozenset(chunk_id).issubset(AiffAudio.PRINTABLE_ASCII): header_size -= 8 else: from audiotools.text import ERR_AIFF_INVALID_CHUNK raise ValueError(ERR_AIFF_INVALID_CHUNK) if chunk_id == b"COMM": if not comm_found: # skip COMM chunk when found comm_found = True if chunk_size % 2: aiff_file.skip_bytes(chunk_size + 1) header_size -= (chunk_size + 1) else: aiff_file.skip_bytes(chunk_size) header_size -= chunk_size else: # ensure only one COMM chunk is found from audiotools.text import ERR_AIFF_MULTIPLE_COMM_CHUNKS raise ValueError(ERR_AIFF_MULTIPLE_COMM_CHUNKS) elif chunk_id == b"SSND": if not comm_found: # ensure at least one COMM chunk is found from audiotools.text import ERR_AIFF_PREMATURE_SSND_CHUNK raise ValueError(ERR_AIFF_PREMATURE_SSND_CHUNK) elif header_size > 8: # ensure exactly 8 bytes remain after SSND chunk header from audiotools.text import ERR_AIFF_HEADER_EXTRA_SSND raise ValueError(ERR_AIFF_HEADER_EXTRA_SSND) elif header_size < 8: from audiotools.text import ERR_AIFF_HEADER_MISSING_SSND raise ValueError(ERR_AIFF_HEADER_MISSING_SSND) else: return (total_size, chunk_size - 8) else: # skip the full contents of non-audio chunks if chunk_size % 2: aiff_file.skip_bytes(chunk_size + 1) header_size -= (chunk_size + 1) else: aiff_file.skip_bytes(chunk_size) header_size -= chunk_size else: # header parsed with no SSND chunks found from audiotools.text import ERR_AIFF_NO_SSND_CHUNK raise ValueError(ERR_AIFF_NO_SSND_CHUNK) except IOError: from audiotools.text import ERR_AIFF_HEADER_IOERROR raise ValueError(ERR_AIFF_HEADER_IOERROR) def validate_footer(footer, ssnd_bytes_written): """given a footer string as returned by aiff_header_footer() and PCM stream parameters, returns True if the footer is valid raises ValueError is the footer is invalid""" from io import BytesIO from audiotools.bitstream import BitstreamReader total_size = len(footer) aiff_file = BitstreamReader(BytesIO(footer), False) try: # ensure footer is padded properly if necessary # based on size of data bytes written if ssnd_bytes_written % 2: aiff_file.skip_bytes(1) total_size -= 1 while total_size > 0: (chunk_id, chunk_size) = aiff_file.parse("4b 32u") if frozenset(chunk_id).issubset(AiffAudio.PRINTABLE_ASCII): total_size -= 8 else: from audiotools.text import ERR_AIFF_INVALID_CHUNK raise ValueError(ERR_AIFF_INVALID_CHUNK) if chunk_id == b"COMM": # ensure no COMM chunks are found from audiotools.text import ERR_AIFF_MULTIPLE_COMM_CHUNKS raise ValueError(ERR_AIFF_MULTIPLE_COMM_CHUNKS) elif chunk_id == b"SSND": # ensure no SSND chunks are found from audiotools.text import ERR_AIFF_MULTIPLE_SSND_CHUNKS raise ValueError(ERR_AIFF_MULTIPLE_SSND_CHUNKS) else: # skip the full contents of non-audio chunks if chunk_size % 2: aiff_file.skip_bytes(chunk_size + 1) total_size -= (chunk_size + 1) else: aiff_file.skip_bytes(chunk_size) total_size -= chunk_size else: return True except IOError: from audiotools.text import ERR_AIFF_FOOTER_IOERROR raise ValueError(ERR_AIFF_FOOTER_IOERROR) class AIFF_Chunk(object): """a raw chunk of AIFF data""" def __init__(self, chunk_id, chunk_size, chunk_data): """chunk_id should be a binary string of ASCII chunk_size is the length of chunk_data chunk_data should be a binary string of chunk data""" # FIXME - check chunk_id's validity self.id = chunk_id self.__size__ = chunk_size self.__data__ = chunk_data def __repr__(self): return "AIFF_Chunk({!r})".format(self.id) def size(self): """returns size of chunk in bytes not including any spacer byte for odd-sized chunks""" return self.__size__ def total_size(self): """returns the total size of the chunk including the 8 byte ID/size and any padding byte""" if self.__size__ % 2: return 8 + self.__size__ + 1 else: return 8 + self.__size__ def data(self): """returns chunk data as file-like object""" from io import BytesIO return BytesIO(self.__data__) def verify(self): """returns True if chunk size matches chunk's data""" return self.__size__ == len(self.__data__) def write(self, f): """writes the entire chunk to the given output file object returns size of entire chunk (including header and spacer) in bytes""" f.write(self.id) f.write(struct.pack(">I", self.__size__)) f.write(self.__data__) if self.__size__ % 2: f.write(b"\x00") return self.total_size() class AIFF_File_Chunk(AIFF_Chunk): """a raw chunk of AIFF data taken from an existing file""" def __init__(self, chunk_id, chunk_size, aiff_file, chunk_data_offset): """chunk_id should be a binary string of ASCII chunk_size is the size of the chunk in bytes (not counting any spacer byte) aiff_file is the file this chunk belongs to chunk_data_offset is the offset to the chunk's data bytes (not including the 8 byte header)""" self.id = chunk_id self.__size__ = chunk_size self.__aiff_file__ = aiff_file self.__offset__ = chunk_data_offset def __del__(self): self.__aiff_file__.close() def __repr__(self): return "AIFF_File_Chunk({!r})".format(self.id) def data(self): """returns chunk data as file-like object""" from audiotools import LimitedFileReader self.__aiff_file__.seek(self.__offset__) return LimitedFileReader(self.__aiff_file__, self.size()) def verify(self): """returns True if chunk size matches chunk's data""" self.__aiff_file__.seek(self.__offset__) to_read = self.__size__ while to_read > 0: s = self.__aiff_file__.read(min(0x100000, to_read)) if len(s) == 0: return False else: to_read -= len(s) return True def write(self, f): """writes the entire chunk to the given output file object returns size of entire chunk (including header and spacer) in bytes""" f.write(self.id) f.write(struct.pack(">I", self.__size__)) self.__aiff_file__.seek(self.__offset__) to_write = self.__size__ while to_write > 0: s = self.__aiff_file__.read(min(0x100000, to_write)) f.write(s) to_write -= len(s) if self.__size__ % 2: f.write(b"\x00") return self.total_size() def parse_comm(comm): """given a COMM chunk (without the 8 byte name/size header) returns (channels, total_sample_frames, bits_per_sample, sample_rate, channel_mask) where channel_mask is a ChannelMask object and the rest are ints may raise IOError if an error occurs reading the chunk""" from audiotools import ChannelMask (channels, total_sample_frames, bits_per_sample) = comm.parse("16u 32u 16u") sample_rate = int(parse_ieee_extended(comm)) if channels <= 2: channel_mask = ChannelMask.from_channels(channels) else: channel_mask = ChannelMask(0) return (channels, total_sample_frames, bits_per_sample, sample_rate, channel_mask) class AiffReader(object): """a PCMReader object for reading AIFF file contents""" def __init__(self, aiff_filename): """aiff_filename is a string""" from audiotools.bitstream import BitstreamReader self.stream = BitstreamReader(open(aiff_filename, "rb"), False) # ensure FORM<size>AIFF header is ok try: (form, total_size, aiff) = self.stream.parse("4b 32u 4b") except struct.error: from audiotools.text import ERR_AIFF_INVALID_AIFF raise InvalidAIFF(ERR_AIFF_INVALID_AIFF) if form != b'FORM': from audiotools.text import ERR_AIFF_NOT_AIFF raise ValueError(ERR_AIFF_NOT_AIFF) elif aiff != b'AIFF': from audiotools.text import ERR_AIFF_INVALID_AIFF raise ValueError(ERR_AIFF_INVALID_AIFF) else: total_size -= 4 comm_chunk_read = False # walk through chunks until "SSND" chunk encountered while total_size > 0: try: (chunk_id, chunk_size) = self.stream.parse("4b 32u") except struct.error: from audiotools.text import ERR_AIFF_INVALID_AIFF raise ValueError(ERR_AIFF_INVALID_AIFF) if not frozenset(chunk_id).issubset(AiffAudio.PRINTABLE_ASCII): from audiotools.text import ERR_AIFF_INVALID_CHUNK_ID raise ValueError(ERR_AIFF_INVALID_CHUNK_ID) else: total_size -= 8 if chunk_id == b"COMM": # when "COMM" chunk encountered, # use it to populate PCMReader attributes (self.channels, self.total_pcm_frames, self.bits_per_sample, self.sample_rate, channel_mask) = parse_comm(self.stream) self.channel_mask = int(channel_mask) self.bytes_per_pcm_frame = ((self.bits_per_sample // 8) * self.channels) self.remaining_pcm_frames = self.total_pcm_frames comm_chunk_read = True elif chunk_id == b"SSND": # when "SSND" chunk encountered, # strip off the "offset" and "block_size" attributes # and ready PCMReader for reading if not comm_chunk_read: from audiotools.text import ERR_AIFF_PREMATURE_SSND_CHUNK raise ValueError(ERR_AIFF_PREMATURE_SSND_CHUNK) else: self.stream.skip_bytes(8) self.ssnd_start = self.stream.getpos() return else: # all other chunks are ignored self.stream.skip_bytes(chunk_size) if chunk_size % 2: if len(self.stream.read_bytes(1)) < 1: from audiotools.text import ERR_AIFF_INVALID_CHUNK raise ValueError(ERR_AIFF_INVALID_CHUNK) total_size -= (chunk_size + 1) else: total_size -= chunk_size else: # raise an error if no "SSND" chunk is encountered from audiotools.text import ERR_AIFF_NO_SSND_CHUNK raise ValueError(ERR_AIFF_NO_SSND_CHUNK) def __enter__(self): return self def __exit__(self, exc_type, exc_value, traceback): self.close() def read(self, pcm_frames): """try to read a pcm.FrameList with the given number of PCM frames""" # try to read requested PCM frames or remaining frames requested_pcm_frames = min(max(pcm_frames, 1), self.remaining_pcm_frames) requested_bytes = (self.bytes_per_pcm_frame * requested_pcm_frames) pcm_data = self.stream.read_bytes(requested_bytes) # raise exception if "SSND" chunk exhausted early if len(pcm_data) < requested_bytes: from audiotools.text import ERR_AIFF_TRUNCATED_SSND_CHUNK raise IOError(ERR_AIFF_TRUNCATED_SSND_CHUNK) else: self.remaining_pcm_frames -= requested_pcm_frames # return parsed chunk return FrameList(pcm_data, self.channels, self.bits_per_sample, True, True) def read_closed(self, pcm_frames): raise ValueError("cannot read closed stream") def seek(self, pcm_frame_offset): """tries to seek to the given PCM frame offset returns the total amount of frames actually seeked over""" if pcm_frame_offset < 0: from audiotools.text import ERR_NEGATIVE_SEEK raise ValueError(ERR_NEGATIVE_SEEK) # ensure one doesn't walk off the end of the file pcm_frame_offset = min(pcm_frame_offset, self.total_pcm_frames) # position file in "SSND" chunk self.stream.setpos(self.ssnd_start) self.stream.seek((pcm_frame_offset * self.bytes_per_pcm_frame), 1) self.remaining_pcm_frames = (self.total_pcm_frames - pcm_frame_offset) return pcm_frame_offset def seek_closed(self, pcm_frame_offset): raise ValueError("cannot seek closed stream") def close(self): """closes the stream for reading""" self.stream.close() self.read = self.read_closed self.seek = self.seek_closed def aiff_header(sample_rate, channels, bits_per_sample, total_pcm_frames): """given a set of integer stream attributes, returns header string of everything before an AIFF's PCM data may raise ValueError if the total size of the file is too large""" from audiotools.bitstream import (BitstreamRecorder, format_size) header = BitstreamRecorder(False) data_size = (bits_per_sample // 8) * channels * total_pcm_frames total_size = ((format_size("4b" + "4b 32u" + "16u 32u 16u 1u 15u 64U" + "4b 32u 32u 32u") // 8) + data_size + (data_size % 2)) if total_size < (2 ** 32): header.build("4b 32u 4b", (b"FORM", total_size, b"AIFF")) header.build("4b 32u", (b"COMM", 0x12)) header.build("16u 32u 16u", (channels, total_pcm_frames, bits_per_sample)) build_ieee_extended(header, sample_rate) header.build("4b 32u 32u 32u", (b"SSND", data_size + 8, 0, 0)) return header.data() else: raise ValueError("total size too large for aiff file") class InvalidAIFF(InvalidFile): """raised if some problem occurs parsing AIFF chunks""" pass class AiffAudio(AiffContainer): """an AIFF audio file""" SUFFIX = "aiff" NAME = SUFFIX DESCRIPTION = u"Audio Interchange File Format" if sys.version_info[0] >= 3: PRINTABLE_ASCII = {i for i in range(0x20, 0x7E + 1)} else: PRINTABLE_ASCII = {chr(i) for i in range(0x20, 0x7E + 1)} def __init__(self, filename): """filename is a plain string""" from audiotools import ChannelMask AiffContainer.__init__(self, filename) self.__channels__ = 0 self.__bits_per_sample__ = 0 self.__sample_rate__ = 0 self.__channel_mask__ = ChannelMask(0) self.__total_sample_frames__ = 0 from audiotools.bitstream import BitstreamReader try: for chunk in self.chunks(): if chunk.id == b"COMM": try: (self.__channels__, self.__total_sample_frames__, self.__bits_per_sample__, self.__sample_rate__, self.__channel_mask__) = parse_comm( BitstreamReader(chunk.data(), False)) break except IOError: continue except IOError: raise InvalidAIFF("I/O error reading wave") def bits_per_sample(self): """returns an integer number of bits-per-sample this track contains""" return self.__bits_per_sample__ def channels(self): """returns an integer number of channels this track contains""" return self.__channels__ def channel_mask(self): """returns a ChannelMask object of this track's channel layout""" return self.__channel_mask__ def lossless(self): """returns True""" return True def total_frames(self): """returns the total PCM frames of the track as an integer""" return self.__total_sample_frames__ def sample_rate(self): """returns the rate of the track's audio as an integer number of Hz""" return self.__sample_rate__ def seekable(self): """returns True if the file is seekable""" return True def chunks(self): """yields a AIFF_Chunk compatible objects for each chunk in file""" from audiotools.text import (ERR_AIFF_NOT_AIFF, ERR_AIFF_INVALID_AIFF, ERR_AIFF_INVALID_CHUNK_ID, ERR_AIFF_INVALID_CHUNK) with open(self.filename, 'rb') as aiff_file: try: (form, total_size, aiff) = struct.unpack(">4sI4s", aiff_file.read(12)) except struct.error: raise InvalidAIFF(ERR_AIFF_INVALID_AIFF) if form != b'FORM': raise InvalidAIFF(ERR_AIFF_NOT_AIFF) elif aiff != b'AIFF': raise InvalidAIFF(ERR_AIFF_INVALID_AIFF) else: total_size -= 4 while total_size > 0: # read the chunk header and ensure its validity try: data = aiff_file.read(8) (chunk_id, chunk_size) = struct.unpack(">4sI", data) except struct.error: raise InvalidAIFF(ERR_AIFF_INVALID_AIFF) if not frozenset(chunk_id).issubset(self.PRINTABLE_ASCII): raise InvalidAIFF(ERR_AIFF_INVALID_CHUNK_ID) else: total_size -= 8 # yield AIFF_Chunk or AIFF_File_Chunk depending on chunk size if chunk_size >= 0x100000: # if chunk is too large, yield a File_Chunk yield AIFF_File_Chunk(chunk_id, chunk_size, open(self.filename, "rb"), aiff_file.tell()) aiff_file.seek(chunk_size, 1) else: # otherwise, yield a raw data Chunk yield AIFF_Chunk(chunk_id, chunk_size, aiff_file.read(chunk_size)) if chunk_size % 2: if len(aiff_file.read(1)) < 1: raise InvalidAIFF(ERR_AIFF_INVALID_CHUNK) total_size -= (chunk_size + 1) else: total_size -= chunk_size @classmethod def aiff_from_chunks(cls, aiff_file, chunk_iter): """builds a new AIFF file from a chunk data iterator aiff_file is a seekable file object chunk_iter should yield AIFF_Chunk-compatible objects """ start = aiff_file.tell() # write an unfinished header with a placeholder size aiff_file.write(struct.pack(">4sI4s", b"FORM", 4, b"AIFF")) # write the individual chunks total_size = 4 for chunk in chunk_iter: total_size += chunk.write(aiff_file) # once the chunks are done, go back and re-write the header aiff_file.seek(start) aiff_file.write(struct.pack(">4sI4s", b"FORM", total_size, b"AIFF")) @classmethod def supports_metadata(cls): """returns True if this audio type supports MetaData""" return True def get_metadata(self): """returns a MetaData object, or None raises IOError if unable to read the file""" from audiotools.bitstream import BitstreamReader from audiotools.id3 import ID3v22Comment for chunk in self.chunks(): if chunk.id == b'ID3 ': return ID3v22Comment.parse( BitstreamReader(chunk.data(), False)) else: return None def update_metadata(self, metadata): """takes this track's current MetaData object as returned by get_metadata() and sets this track's metadata with any fields updated in that object raises IOError if unable to write the file """ from audiotools import transfer_data, TemporaryFile from audiotools.id3 import ID3v22Comment from audiotools.bitstream import BitstreamRecorder from audiotools.text import ERR_FOREIGN_METADATA import os if metadata is None: return elif not isinstance(metadata, ID3v22Comment): raise ValueError(ERR_FOREIGN_METADATA) elif not os.access(self.filename, os.W_OK): raise IOError(self.filename) # turn our ID3v2.2 tag into a raw binary chunk id3_chunk = BitstreamRecorder(0) metadata.build(id3_chunk) # generate a temporary AIFF file in which our new ID3v2.2 chunk # replaces the existing ID3v2.2 chunk new_aiff = TemporaryFile(self.filename) self.__class__.aiff_from_chunks( new_aiff, [(chunk if chunk.id != b"ID3 " else AIFF_Chunk(b"ID3 ", id3_chunk.bytes(), id3_chunk.data())) for chunk in self.chunks()]) new_aiff.close() def set_metadata(self, metadata): """takes a MetaData object and sets this track's metadata this metadata includes track name, album name, and so on raises IOError if unable to write the file""" from audiotools.id3 import ID3v22Comment if metadata is None: return self.delete_metadata() elif self.get_metadata() is not None: # current file has metadata, so replace it with new metadata self.update_metadata(ID3v22Comment.converted(metadata)) else: # current file has no metadata, so append new ID3 block import os from audiotools.bitstream import BitstreamRecorder from audiotools import transfer_data, TemporaryFile if not os.access(self.filename, os.W_OK): raise IOError(self.filename) # turn our ID3v2.2 tag into a raw binary chunk id3_chunk = BitstreamRecorder(0) ID3v22Comment.converted(metadata).build(id3_chunk) # generate a temporary AIFF file in which our new ID3v2.2 chunk # is appended to the file's set of chunks new_aiff = TemporaryFile(self.filename) self.__class__.aiff_from_chunks( new_aiff, [c for c in self.chunks()] + [AIFF_Chunk(b"ID3 ", id3_chunk.bytes(), id3_chunk.data())]) new_aiff.close() def delete_metadata(self): """deletes the track's MetaData this removes or unsets tags as necessary in order to remove all data raises IOError if unable to write the file""" import os from audiotools import transfer_data, TemporaryFile if not os.access(self.filename, os.W_OK): raise IOError(self.filename) new_aiff = TemporaryFile(self.filename) self.__class__.aiff_from_chunks( new_aiff, [c for c in self.chunks() if c.id != b"ID3 "]) new_aiff.close() @classmethod def supports_to_pcm(cls): """returns True if all necessary components are available to support the .to_pcm() method""" return True def to_pcm(self): """returns a PCMReader object containing the track's PCM data""" try: return AiffReader(self.filename) except (IOError, ValueError) as err: from audiotools import PCMReaderError return PCMReaderError(error_message=str(err), sample_rate=self.sample_rate(), channels=self.channels(), channel_mask=int(self.channel_mask()), bits_per_sample=self.bits_per_sample()) @classmethod def supports_from_pcm(cls): """returns True if all necessary components are available to support the .from_pcm() classmethod""" return True @classmethod def from_pcm(cls, filename, pcmreader, compression=None, total_pcm_frames=None): """encodes a new file from PCM data takes a filename string, PCMReader object, optional compression level string and optional total_pcm_frames integer encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new AiffAudio object""" from audiotools import EncodingError from audiotools import DecodingError from audiotools import CounterPCMReader from audiotools import transfer_framelist_data if pcmreader.bits_per_sample not in {8, 16, 24}: from audiotools import UnsupportedBitsPerSample pcmreader.close() raise UnsupportedBitsPerSample(filename, pcmreader.bits_per_sample) try: header = aiff_header(pcmreader.sample_rate, pcmreader.channels, pcmreader.bits_per_sample, total_pcm_frames if total_pcm_frames is not None else 0) except ValueError as err: pcmreader.close() raise EncodingError(str(err)) try: f = open(filename, "wb") except IOError as msg: pcmreader.close() raise EncodingError(str(msg)) counter = CounterPCMReader(pcmreader) f.write(header) try: transfer_framelist_data(counter, f.write, True, True) except (IOError, ValueError) as err: f.close() cls.__unlink__(filename) raise EncodingError(str(err)) # handle odd-sized SSND chunks if counter.frames_written % 2: f.write(b"\x00") if total_pcm_frames is not None: f.close() if total_pcm_frames != counter.frames_written: # ensure written number of PCM frames # matches total_pcm_frames argument from audiotools.text import ERR_TOTAL_PCM_FRAMES_MISMATCH cls.__unlink__(filename) raise EncodingError(ERR_TOTAL_PCM_FRAMES_MISMATCH) else: # go back and rewrite populated header # with counted number of PCM frames f.seek(0, 0) f.write(aiff_header(pcmreader.sample_rate, pcmreader.channels, pcmreader.bits_per_sample, counter.frames_written)) f.close() return AiffAudio(filename) def has_foreign_aiff_chunks(self): """returns True if the audio file contains non-audio AIFF chunks""" return ({b'COMM', b'SSND'} != {c.id for c in self.chunks()}) def aiff_header_footer(self): """returns (header, footer) tuple of strings containing all data before and after the PCM stream if self.has_foreign_aiff_chunks() is False, may raise ValueError if the file has no header and footer for any reason""" from audiotools.bitstream import BitstreamReader from audiotools.bitstream import BitstreamRecorder from audiotools.text import (ERR_AIFF_NOT_AIFF, ERR_AIFF_INVALID_AIFF, ERR_AIFF_INVALID_CHUNK_ID) head = BitstreamRecorder(0) tail = BitstreamRecorder(0) current_block = head with BitstreamReader(open(self.filename, 'rb'), False) as aiff_file: # transfer the 12-byte "RIFFsizeWAVE" header to head (form, size, aiff) = aiff_file.parse("4b 32u 4b") if form != b'FORM': raise InvalidAIFF(ERR_AIFF_NOT_AIFF) elif aiff != b'AIFF': raise InvalidAIFF(ERR_AIFF_INVALID_AIFF) else: current_block.build("4b 32u 4b", (form, size, aiff)) total_size = size - 4 while total_size > 0: # transfer each chunk header (chunk_id, chunk_size) = aiff_file.parse("4b 32u") if not frozenset(chunk_id).issubset(self.PRINTABLE_ASCII): raise InvalidAIFF(ERR_AIFF_INVALID_CHUNK_ID) else: current_block.build("4b 32u", (chunk_id, chunk_size)) total_size -= 8 # and transfer the full content of non-audio chunks if chunk_id != b"SSND": if chunk_size % 2: current_block.write_bytes( aiff_file.read_bytes(chunk_size + 1)) total_size -= (chunk_size + 1) else: current_block.write_bytes( aiff_file.read_bytes(chunk_size)) total_size -= chunk_size else: # transfer alignment as part of SSND's chunk header align = aiff_file.parse("32u 32u") current_block.build("32u 32u", align) aiff_file.skip_bytes(chunk_size - 8) current_block = tail if chunk_size % 2: current_block.write_bytes(aiff_file.read_bytes(1)) total_size -= (chunk_size + 1) else: total_size -= chunk_size return (head.data(), tail.data()) @classmethod def from_aiff(cls, filename, header, pcmreader, footer, compression=None): """encodes a new file from AIFF data takes a filename string, header string, PCMReader object, footer string and optional compression level string encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new AiffAudio object header + pcm data + footer should always result in the original AIFF file being restored without need for any padding bytes may raise EncodingError if some problem occurs when encoding the input file""" from audiotools import (DecodingError, EncodingError, FRAMELIST_SIZE) from struct import unpack # ensure header validates correctly try: (total_size, ssnd_size) = validate_header(header) except ValueError as err: pcmreader.close() raise EncodingError(str(err)) try: f = open(filename, "wb") except IOError as msg: raise EncodingError(msg) try: # write header to output file f.write(header) # write PCM data to output file SSND_bytes_written = 0 s = pcmreader.read(FRAMELIST_SIZE).to_bytes(True, True) while len(s) > 0: SSND_bytes_written += len(s) f.write(s) s = pcmreader.read(FRAMELIST_SIZE).to_bytes(True, True) # ensure output data size matches the "SSND" chunk's size if ssnd_size != SSND_bytes_written: cls.__unlink__(filename) from audiotools.text import ERR_AIFF_TRUNCATED_SSND_CHUNK raise EncodingError(ERR_AIFF_TRUNCATED_SSND_CHUNK) # ensure footer validates correctly # before writing it to disk validate_footer(footer, SSND_bytes_written) f.write(footer) f.flush() # ensure total size is correct if (len(header) + ssnd_size + len(footer)) != total_size: cls.__unlink__(filename) from audiotools.text import ERR_AIFF_INVALID_SIZE raise EncodingError(ERR_AIFF_INVALID_SIZE) return cls(filename) except (IOError, ValueError) as err: cls.__unlink__(filename) raise EncodingError(str(err)) finally: f.close() pcmreader.close() def verify(self, progress=None): """verifies the current file for correctness returns True if the file is okay raises an InvalidFile with an error message if there is some problem with the file""" from audiotools import CounterPCMReader from audiotools import transfer_framelist_data from audiotools import to_pcm_progress try: (header, footer) = self.aiff_header_footer() except IOError as err: raise InvalidAIFF(err) except ValueError as err: raise InvalidAIFF(err) # ensure header is valid try: (total_size, data_size) = validate_header(header) except ValueError as err: raise InvalidAIFF(err) # ensure "ssnd" chunk has all its data counter = CounterPCMReader(to_pcm_progress(self, progress)) try: transfer_framelist_data(counter, lambda f: f) except (IOError, ValueError): from audiotools.text import ERR_AIFF_TRUNCATED_SSND_CHUNK raise InvalidAIFF(ERR_AIFF_TRUNCATED_SSND_CHUNK) data_bytes_written = counter.bytes_written() # ensure output data size matches the "ssnd" chunk's size if data_size != data_bytes_written: from audiotools.text import ERR_AIFF_TRUNCATED_SSND_CHUNK raise InvalidAIFF(ERR_AIFF_TRUNCATED_SSND_CHUNK) # ensure footer validates correctly try: validate_footer(footer, data_bytes_written) except ValueError as err: from audiotools.text import ERR_AIFF_INVALID_SIZE raise InvalidAIFF(ERR_AIFF_INVALID_SIZE) # ensure total size is correct if (len(header) + data_size + len(footer)) != total_size: from audiotools.text import ERR_AIFF_INVALID_SIZE raise InvalidAIFF(ERR_AIFF_INVALID_SIZE) return True def clean(self, output_filename=None): """cleans the file of known data and metadata problems output_filename is an optional filename of the fixed file if present, a new AudioFile is written to that path otherwise, only a dry-run is performed and no new file is written return list of fixes performed as Unicode strings raises IOError if unable to write the file or its metadata raises ValueError if the file has errors of some sort """ from audiotools.text import (CLEAN_AIFF_MULTIPLE_COMM_CHUNKS, CLEAN_AIFF_REORDERED_SSND_CHUNK, CLEAN_AIFF_MULTIPLE_SSND_CHUNKS) fixes_performed = [] chunk_queue = [] pending_data = None for chunk in self.chunks(): if chunk.id == b"COMM": if b"COMM" in [c.id for c in chunk_queue]: fixes_performed.append( CLEAN_AIFF_MULTIPLE_COMM_CHUNKS) else: chunk_queue.append(chunk) if pending_data is not None: chunk_queue.append(pending_data) pending_data = None elif chunk.id == b"SSND": if b"COMM" not in [c.id for c in chunk_queue]: fixes_performed.append(CLEAN_AIFF_REORDERED_SSND_CHUNK) pending_data = chunk elif b"SSND" in [c.id for c in chunk_queue]: fixes_performed.append(CLEAN_AIFF_MULTIPLE_SSND_CHUNKS) else: chunk_queue.append(chunk) else: chunk_queue.append(chunk) old_metadata = self.get_metadata() if old_metadata is not None: (fixed_metadata, metadata_fixes) = old_metadata.clean() else: fixed_metadata = old_metadata metadata_fixes = [] if output_filename is not None: output_file = open(output_filename, "wb") AiffAudio.aiff_from_chunks(output_file, chunk_queue) output_file.close() fixed_aiff = AiffAudio(output_filename) if fixed_metadata is not None: fixed_aiff.update_metadata(fixed_metadata) return fixes_performed + metadata_fixes ================================================ FILE: audiotools/ape.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA import sys from audiotools import (AudioFile, MetaData) # takes a pair of integers (or None) for the current and total values # returns a unicode string of their combined pair # for example, __number_pair__(2,3) returns u"2/3" # whereas __number_pair__(4,0) returns u"4" def __number_pair__(current, total): def empty(i): return i is None unslashed_format = u"{:d}" slashed_format = u"{:d}/{:d}" if empty(current) and empty(total): return unslashed_format.format(0) elif (not empty(current)) and empty(total): return unslashed_format.format(current) elif empty(current) and (not empty(total)): return slashed_format.format(0, total) else: # neither current or total are empty return slashed_format.format(current, total) def limited_transfer_data(from_function, to_function, max_bytes): """transfers up to max_bytes from from_function to to_function or as many bytes as from_function generates as strings""" BUFFER_SIZE = 0x100000 s = from_function(BUFFER_SIZE) while (len(s) > 0) and (max_bytes > 0): if len(s) > max_bytes: s = s[0:max_bytes] to_function(s) max_bytes -= len(s) s = from_function(BUFFER_SIZE) class ApeTagItem(object): """a single item in the ApeTag, typically a unicode value""" FORMAT = "32u 1u 2u 29p" def __init__(self, item_type, read_only, key, data): """fields are as follows: item_type is 0 = UTF-8, 1 = binary, 2 = external, 3 = reserved read_only is 1 if the item is read only key is a bytes object of the item's key data is a bytes object of the data itself """ self.type = item_type self.read_only = read_only assert(isinstance(key, bytes)) self.key = key assert(isinstance(data, bytes)) self.data = data def __eq__(self, item): for attr in ["type", "read_only", "key", "data"]: if ((not hasattr(item, attr)) or (getattr(self, attr) != getattr(item, attr))): return False else: return True def total_size(self): """returns total size of item in bytes""" return 4 + 4 + len(self.key) + 1 + len(self.data) def copy(self): """returns a duplicate ApeTagItem""" return ApeTagItem(self.type, self.read_only, self.key, self.data) def __repr__(self): return "ApeTagItem({!r},{!r},{!r},{!r})".format(self.type, self.read_only, self.key, self.data) def raw_info_pair(self): """returns a human-readable key/value pair of item data""" if self.type == 0: # text if self.read_only: return (self.key.decode('ascii'), u"(read only) {}".format(self.data.decode('utf-8'))) else: return (self.key.decode('ascii'), self.data.decode('utf-8')) elif self.type == 1: # binary return (self.key.decode('ascii'), u"(binary) {:d} bytes".format(len(self.data))) elif self.type == 2: # external return (self.key.decode('ascii'), u"(external) {:d} bytes".format(len(self.data))) else: # reserved return (self.key.decode('ascii'), u"(reserved) {:d} bytes".format(len(self.data))) if sys.version_info[0] >= 3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.data def __unicode__(self): return self.data.rstrip(b"\x00").decode('utf-8', 'replace') def number(self): """returns the track/album_number portion of a slashed number pair""" import re unicode_value = self.__unicode__() int_string = re.search(r'\d+', unicode_value) if int_string is None: return None int_value = int(int_string.group(0)) if (int_value == 0) and (u"/" in unicode_value): total_value = re.search(r'\d+', unicode_value.split(u"/")[1]) if total_value is not None: # don't return placeholder 0 value # when a _total value is present # but _number value is 0 return None else: return int_value else: return int_value def total(self): """returns the track/album_total portion of a slashed number pair""" import re unicode_value = self.__unicode__() if u"/" not in unicode_value: return None int_string = re.search(r'\d+', unicode_value.split(u"/")[1]) if int_string is not None: return int(int_string.group(0)) else: return None @classmethod def parse(cls, reader): """returns an ApeTagItem parsed from the given BitstreamReader""" (item_value_length, read_only, encoding) = reader.parse(cls.FORMAT) key = [] c = reader.read_bytes(1) while c != b"\x00": key.append(c) c = reader.read_bytes(1) value = reader.read_bytes(item_value_length) return cls(encoding, read_only, b"".join(key), value) def build(self, writer): """writes the ApeTagItem values to the given BitstreamWriter""" writer.build("{} {:d}b 8u {:d}b".format(self.FORMAT, len(self.key), len(self.data)), (len(self.data), self.read_only, self.type, self.key, 0, self.data)) @classmethod def binary(cls, key, data): """returns an ApeTagItem of binary data key is an ASCII string, data is a binary string""" return cls(1, 0, key, data) @classmethod def external(cls, key, data): """returns an ApeTagItem of external data key is an ASCII string, data is a binary string""" return cls(2, 0, key, data) @classmethod def string(cls, key, data): """returns an ApeTagItem of text data key is a bytes object, data is a unicode string""" assert(isinstance(key, bytes)) assert(isinstance(data, str if (sys.version_info[0] >= 3) else unicode)) return cls(0, 0, key, data.encode('utf-8', 'replace')) class ApeTag(MetaData): """a complete APEv2 tag""" HEADER_FORMAT = "8b 32u 32u 32u 1u 2u 26p 1u 1u 1u 64p" ITEM = ApeTagItem ATTRIBUTE_MAP = {'track_name': b'Title', 'track_number': b'Track', 'track_total': b'Track', 'album_number': b'Media', 'album_total': b'Media', 'album_name': b'Album', 'artist_name': b'Artist', 'performer_name': b'Performer', 'composer_name': b'Composer', 'conductor_name': b'Conductor', 'ISRC': b'ISRC', 'catalog': b'Catalog', 'copyright': b'Copyright', 'publisher': b'Publisher', 'year': b'Year', 'date': b'Record Date', 'comment': b'Comment', 'compilation': b'Compilation'} INTEGER_ITEMS = (b'Track', b'Media') BOOLEAN_ITEMS = (b'Compilation',) def __init__(self, tags, contains_header=True, contains_footer=True): """constructs an ApeTag from a list of ApeTagItem objects""" for tag in tags: assert(isinstance(tag, ApeTagItem)) MetaData.__setattr__(self, "tags", list(tags)) MetaData.__setattr__(self, "contains_header", contains_header) MetaData.__setattr__(self, "contains_footer", contains_footer) def __repr__(self): return "ApeTag({!r},{!r},{!r})".format(self.tags, self.contains_header, self.contains_footer) def total_size(self): """returns the minimum size of the total ApeTag, in bytes""" size = 0 if self.contains_header: size += 32 for tag in self.tags: size += tag.total_size() if self.contains_footer: size += 32 return size def __eq__(self, metadata): if isinstance(metadata, ApeTag): if set(self.keys()) != set(metadata.keys()): return False for tag in self.tags: try: if tag.data != metadata[tag.key].data: return False except KeyError: return False else: return True elif isinstance(metadata, MetaData): return MetaData.__eq__(self, metadata) else: return False def keys(self): return [tag.key for tag in self.tags] def __contains__(self, key): for tag in self.tags: if tag.key == key: return True else: return False def __getitem__(self, key): assert(isinstance(key, bytes)) for tag in self.tags: if tag.key == key: return tag else: raise KeyError(key) def get(self, key, default): assert(isinstance(key, bytes)) try: return self[key] except KeyError: return default def __setitem__(self, key, value): assert(isinstance(key, bytes)) for i in range(len(self.tags)): if self.tags[i].key == key: self.tags[i] = value return else: self.tags.append(value) def index(self, key): assert(isinstance(key, bytes)) for (i, tag) in enumerate(self.tags): if tag.key == key: return i else: raise ValueError(key) def __delitem__(self, key): assert(isinstance(key, bytes)) new_tags = [tag for tag in self.tags if tag.key != key] if len(new_tags) < len(self.tags): self.tags = new_tags else: raise KeyError(key) def __getattr__(self, attr): if attr in self.ATTRIBUTE_MAP: try: if attr in {'track_number', 'album_number'}: return self[self.ATTRIBUTE_MAP[attr]].number() elif attr in {'track_total', 'album_total'}: return self[self.ATTRIBUTE_MAP[attr]].total() elif attr == 'compilation': return self[self.ATTRIBUTE_MAP[attr]].__unicode__() == u"1" else: return self[self.ATTRIBUTE_MAP[attr]].__unicode__() except KeyError: return None elif attr in MetaData.FIELDS: return None else: return MetaData.__getattribute__(self, attr) # if an attribute is updated (e.g. self.track_name) # make sure to update the corresponding dict pair def __setattr__(self, attr, value): def swap_number(unicode_value, new_number): import re return re.sub(r'\d+', u"{:d}".format(new_number), unicode_value, 1) def swap_slashed_number(unicode_value, new_number): if u"/" in unicode_value: (first, second) = unicode_value.split(u"/", 1) return u"/".join([first, swap_number(second, new_number)]) else: return u"/".join([unicode_value, u"{:d}".format(new_number)]) if attr in self.ATTRIBUTE_MAP: key = self.ATTRIBUTE_MAP[attr] if value is not None: if attr in {'track_number', 'album_number'}: try: current_value = self[key].__unicode__() self[key] = self.ITEM.string( key, swap_number(current_value, value)) except KeyError: self[key] = self.ITEM.string( key, __number_pair__(value, None)) elif attr in {'track_total', 'album_total'}: try: current_value = self[key].__unicode__() self[key] = self.ITEM.string( key, swap_slashed_number(current_value, value)) except KeyError: self[key] = self.ITEM.string( key, __number_pair__(None, value)) elif attr == 'compilation': self[key] = self.ITEM.string( key, u"{:d}".format(1 if value else 0)) else: self[key] = self.ITEM.string(key, value) else: delattr(self, attr) else: MetaData.__setattr__(self, attr, value) def __delattr__(self, attr): import re def zero_number(unicode_value): return re.sub(r'\d+', u"0", unicode_value, 1) if attr in self.ATTRIBUTE_MAP: key = self.ATTRIBUTE_MAP[attr] if attr in {'track_number', 'album_number'}: try: tag = self[key] if tag.total() is None: # if no slashed _total field, delete entire tag del(self[key]) else: # otherwise replace initial portion with 0 self[key] = self.ITEM.string( key, zero_number(tag.__unicode__())) except KeyError: # no tag to delete pass elif attr in {'track_total', 'album_total'}: try: tag = self[key] if tag.total() is not None: if tag.number() is not None: self[key] = self.ITEM.string( key, tag.__unicode__().split(u"/", 1)[0].rstrip()) else: del(self[key]) else: # no total portion, so nothing to do pass except KeyError: # no tag to delete portion of pass else: try: del(self[key]) except KeyError: pass elif attr in MetaData.FIELDS: pass else: MetaData.__delattr__(self, attr) @classmethod def converted(cls, metadata): """converts a MetaData object to an ApeTag object""" if metadata is None: return None elif isinstance(metadata, ApeTag): return ApeTag([tag.copy() for tag in metadata.tags], contains_header=metadata.contains_header, contains_footer=metadata.contains_footer) else: tags = cls([]) for (field, value) in metadata.filled_fields(): if field in cls.ATTRIBUTE_MAP.keys(): setattr(tags, field, value) for image in metadata.images(): tags.add_image(image) return tags def raw_info(self): """returns the ApeTag as a human-readable unicode string""" from os import linesep from audiotools import output_table # align tag values on the "=" sign table = output_table() for tag in self.tags: row = table.row() (key, value) = tag.raw_info_pair() row.add_column(key, "right") row.add_column(u" = ") row.add_column(value) return (u"APEv2:" + linesep + linesep.join(table.format())) @classmethod def supports_images(cls): """returns True""" return True def __parse_image__(self, key, type): from audiotools import Image from io import BytesIO data = BytesIO(self[key].data) description = [] c = data.read(1) while c != b'\x00': description.append(c) c = data.read(1) return Image.new(data.read(), b"".join(description).decode('utf-8', 'replace'), type) def add_image(self, image): """embeds an Image object in this metadata""" from audiotools import FRONT_COVER, BACK_COVER if image.type == FRONT_COVER: self[b'Cover Art (front)'] = self.ITEM.binary( b'Cover Art (front)', image.description.encode('utf-8', 'replace') + b"\x00" + image.data) elif image.type == BACK_COVER: self[b'Cover Art (back)'] = self.ITEM.binary( b'Cover Art (back)', image.description.encode('utf-8', 'replace') + b"\x00" + image.data) def delete_image(self, image): """deletes an Image object from this metadata""" if (image.type == 0) and b'Cover Art (front)' in self.keys(): del(self[b'Cover Art (front)']) elif (image.type == 1) and b'Cover Art (back)' in self.keys(): del(self[b'Cover Art (back)']) def images(self): """returns a list of embedded Image objects""" from audiotools import FRONT_COVER, BACK_COVER # APEv2 supports only one value per key # so a single front and back cover are all that is possible img = [] if b'Cover Art (front)' in self.keys(): img.append(self.__parse_image__(b'Cover Art (front)', FRONT_COVER)) if b'Cover Art (back)' in self.keys(): img.append(self.__parse_image__(b'Cover Art (back)', BACK_COVER)) return img @classmethod def read(cls, apefile): """returns an ApeTag object from an APEv2 tagged file object may return None if the file object has no tag""" from audiotools.bitstream import BitstreamReader, parse apefile.seek(-32, 2) tag_footer = apefile.read(32) if len(tag_footer) < 32: # not enough bytes for an ApeV2 tag return None (preamble, version, tag_size, item_count, read_only, item_encoding, is_header, no_footer, has_header) = parse(cls.HEADER_FORMAT, True, tag_footer) if (preamble != b"APETAGEX") or (version != 2000): return None apefile.seek(-tag_size, 2) reader = BitstreamReader(apefile, True) return cls([ApeTagItem.parse(reader) for i in range(item_count)], contains_header=has_header, contains_footer=True) def build(self, writer): """outputs an APEv2 tag to BitstreamWriter""" tag_size = sum(tag.total_size() for tag in self.tags) + 32 if self.contains_header: writer.build(ApeTag.HEADER_FORMAT, (b"APETAGEX", # preamble 2000, # version tag_size, # tag size len(self.tags), # item count 0, # read only 0, # encoding 1, # is header not self.contains_footer, # no footer self.contains_header)) # has header for tag in self.tags: tag.build(writer) if self.contains_footer: writer.build(ApeTag.HEADER_FORMAT, (b"APETAGEX", # preamble 2000, # version tag_size, # tag size len(self.tags), # item count 0, # read only 0, # encoding 0, # is header not self.contains_footer, # no footer self.contains_header)) # has header def clean(self): import re from audiotools.text import (CLEAN_REMOVE_DUPLICATE_TAG, CLEAN_REMOVE_TRAILING_WHITESPACE, CLEAN_REMOVE_LEADING_WHITESPACE, CLEAN_FIX_TAG_FORMATTING, CLEAN_REMOVE_EMPTY_TAG) fixes_performed = [] used_tags = set() tag_items = [] for tag in self.tags: if tag.key.upper() in used_tags: fixes_performed.append( CLEAN_REMOVE_DUPLICATE_TAG.format(tag.key.decode('ascii'))) elif tag.type == 0: used_tags.add(tag.key.upper()) text = tag.__unicode__() # check trailing whitespace fix1 = text.rstrip() if fix1 != text: fixes_performed.append( CLEAN_REMOVE_TRAILING_WHITESPACE.format( tag.key.decode('ascii'))) # check leading whitespace fix2 = fix1.lstrip() if fix2 != fix1: fixes_performed.append( CLEAN_REMOVE_LEADING_WHITESPACE.format( tag.key.decode('ascii'))) if tag.key in self.INTEGER_ITEMS: if u"/" in fix2: # item is a slashed field of some sort (current, total) = fix2.split(u"/", 1) current_int = re.search(r'\d+', current) total_int = re.search(r'\d+', total) if (current_int is None) and (total_int is None): # neither side contains an integer value # so ignore it altogether fix3 = fix2 elif ((current_int is not None) and (total_int is None)): fix3 = u"{:d}".format(int(current_int.group(0))) elif ((current_int is None) and (total_int is not None)): fix3 = u"{:d}/{:d}".format( 0, int(total_int.group(0))) else: # both sides contain an int fix3 = u"{:d}/{:d}".format( int(current_int.group(0)), int(total_int.group(0))) else: # item contains no slash current_int = re.search(r'\d+', fix2) if current_int is not None: # item contains an integer fix3 = u"{:d}".format(int(current_int.group(0))) else: # item contains no integer value so ignore it # (although 'Track' should only contain # integers, 'Media' may contain strings # so it may be best to simply ignore that case) fix3 = fix2 if fix3 != fix2: fixes_performed.append( CLEAN_FIX_TAG_FORMATTING.format( tag.key.decode('ascii'))) else: fix3 = fix2 if len(fix3) > 0: tag_items.append(ApeTagItem.string(tag.key, fix3)) else: fixes_performed.append( CLEAN_REMOVE_EMPTY_TAG.format(tag.key.decode('ascii'))) else: used_tags.add(tag.key.upper()) tag_items.append(tag) return (self.__class__(tag_items, self.contains_header, self.contains_footer), fixes_performed) def intersection(self, metadata): """given a MetaData-compatible object, returns a new MetaData object which contains all the matching fields and images of this object and 'metadata' """ if type(metadata) is ApeTag: matching_keys = {key for key in set(self.keys()) & set(metadata.keys()) if self[key] == metadata[key]} return ApeTag( [tag.copy() for tag in self.tags if tag.key in matching_keys], contains_header= self.contains_header or metadata.contains_header, contains_footer= self.contains_footer or metadata.contains_footer) else: return MetaData.intersection(self, metadata) class ApeTaggedAudio(object): """a class for handling audio formats with APEv2 tags this class presumes there will be a filename attribute which can be opened and checked for tags, or written if necessary""" @classmethod def supports_metadata(cls): """returns True if this audio type supports MetaData""" return True def get_metadata(self): """returns an ApeTag object, or None raises IOError if unable to read the file""" with open(self.filename, "rb") as f: return ApeTag.read(f) def update_metadata(self, metadata): """takes this track's current MetaData object as returned by get_metadata() and sets this track's metadata with any fields updated in that object raises IOError if unable to write the file """ from audiotools.bitstream import (parse, BitstreamWriter, BitstreamReader) from audiotools import transfer_data if metadata is None: return elif not isinstance(metadata, ApeTag): from audiotools.text import ERR_FOREIGN_METADATA raise ValueError(ERR_FOREIGN_METADATA) elif len(metadata.keys()) == 0: # wipe out entire block of metadata from os import access, R_OK, W_OK if not access(self.filename, R_OK | W_OK): raise IOError(self.filename) with open(self.filename, "rb") as f: f.seek(-32, 2) (preamble, version, tag_size, item_count, read_only, item_encoding, is_header, no_footer, has_header) = BitstreamReader(f, True).parse( ApeTag.HEADER_FORMAT) if (preamble == b'APETAGEX') and (version == 2000): from audiotools import TemporaryFile, transfer_data from os.path import getsize # there's existing metadata to delete # so rewrite file without trailing metadata tag if has_header: old_tag_size = 32 + tag_size else: old_tag_size = tag_size # copy everything but the last "old_tag_size" bytes # from existing file to rewritten file new_apev2 = TemporaryFile(self.filename) old_apev2 = open(self.filename, "rb") limited_transfer_data( old_apev2.read, new_apev2.write, getsize(self.filename) - old_tag_size) old_apev2.close() new_apev2.close() else: # re-set metadata block at end of file f = open(self.filename, "r+b") f.seek(-32, 2) tag_footer = f.read(32) if len(tag_footer) < 32: # no existing ApeTag can fit, so append fresh tag f.close() with BitstreamWriter(open(self.filename, "ab"), True) as writer: metadata.build(writer) return (preamble, version, tag_size, item_count, read_only, item_encoding, is_header, no_footer, has_header) = parse(ApeTag.HEADER_FORMAT, True, tag_footer) if (preamble == b'APETAGEX') and (version == 2000): if has_header: old_tag_size = 32 + tag_size else: old_tag_size = tag_size if metadata.total_size() >= old_tag_size: # metadata has grown # so append it to existing file f.seek(-old_tag_size, 2) writer = BitstreamWriter(f, True) metadata.build(writer) writer.close() else: f.close() # metadata has shrunk # so rewrite file with smaller metadata from audiotools import TemporaryFile from os.path import getsize # copy everything but the last "old_tag_size" bytes # from existing file to rewritten file new_apev2 = TemporaryFile(self.filename) with open(self.filename, "rb") as old_apev2: limited_transfer_data( old_apev2.read, new_apev2.write, getsize(self.filename) - old_tag_size) # append new tag to rewritten file with BitstreamWriter(new_apev2, True) as writer: metadata.build(writer) # closing writer closes new_apev2 also else: # no existing metadata, so simply append a fresh tag f.close() with BitstreamWriter(open(self.filename, "ab"), True) as writer: metadata.build(writer) def set_metadata(self, metadata): """takes a MetaData object and sets this track's metadata raises IOError if unable to write the file""" from audiotools.bitstream import BitstreamWriter if metadata is None: return self.delete_metadata() new_metadata = ApeTag.converted(metadata) old_metadata = self.get_metadata() if old_metadata is not None: # transfer ReplayGain tags from old metadata to new metadata for tag in [b"replaygain_track_gain", b"replaygain_track_peak", b"replaygain_album_gain", b"replaygain_album_peak"]: try: # if old_metadata has tag, shift it over new_metadata[tag] = old_metadata[tag] except KeyError: try: # otherwise, if new_metadata has tag, delete it del(new_metadata[tag]) except KeyError: # if neither has tag, ignore it continue # transfer Cuesheet from old metadata to new metadata if b"Cuesheet" in old_metadata: new_metadata[b"Cuesheet"] = old_metadata[b"Cuesheet"] elif b"Cuesheet" in new_metadata: del(new_metadata[b"Cuesheet"]) self.update_metadata(new_metadata) else: # delete ReplayGain tags from new metadata for tag in [b"replaygain_track_gain", b"replaygain_track_peak", b"replaygain_album_gain", b"replaygain_album_peak"]: try: del(new_metadata[tag]) except KeyError: continue # delete Cuesheet from new metadata if b"Cuesheet" in new_metadata: del(new_metadata[b"Cuesheet"]) if len(new_metadata.keys()) > 0: # no existing metadata, so simply append a fresh tag with BitstreamWriter(open(self.filename, "ab"), True) as writer: new_metadata.build(writer) def delete_metadata(self): """deletes the track's MetaData raises IOError if unable to write the file""" if ((self.get_replay_gain() is not None) or (self.get_cuesheet() is not None)): # non-textual metadata is present and needs preserving self.set_metadata(MetaData()) else: # no non-textual metadata, so wipe out the entire block from os import access, R_OK, W_OK from audiotools.bitstream import BitstreamReader from audiotools import transfer_data if not access(self.filename, R_OK | W_OK): raise IOError(self.filename) with open(self.filename, "rb") as f: f.seek(-32, 2) (preamble, version, tag_size, item_count, read_only, item_encoding, is_header, no_footer, has_header) = BitstreamReader(f, True).parse( ApeTag.HEADER_FORMAT) if (preamble == b'APETAGEX') and (version == 2000): from audiotools import TemporaryFile from os.path import getsize # there's existing metadata to delete # so rewrite file without trailing metadata tag if has_header: old_tag_size = 32 + tag_size else: old_tag_size = tag_size # copy everything but the last "old_tag_size" bytes # from existing file to rewritten file new_apev2 = TemporaryFile(self.filename) old_apev2 = open(self.filename, "rb") limited_transfer_data( old_apev2.read, new_apev2.write, getsize(self.filename) - old_tag_size) old_apev2.close() new_apev2.close() class ApeGainedAudio(object): @classmethod def supports_replay_gain(cls): """returns True if this class supports ReplayGain""" return True def get_replay_gain(self): """returns a ReplayGain object of our ReplayGain values returns None if we have no values""" from audiotools import ReplayGain metadata = self.get_metadata() if metadata is None: return None if ({b'replaygain_track_gain', b'replaygain_track_peak', b'replaygain_album_gain', b'replaygain_album_peak'}.issubset( metadata.keys())): # we have ReplayGain data try: return ReplayGain( metadata[ b'replaygain_track_gain'].__unicode__()[0:-len(" dB")], metadata[ b'replaygain_track_peak'].__unicode__(), metadata[ b'replaygain_album_gain'].__unicode__()[0:-len(" dB")], metadata[ b'replaygain_album_peak'].__unicode__()) except ValueError: return None else: return None def set_replay_gain(self, replaygain): """given a ReplayGain object, sets the track's gain to those values may raise IOError if unable to read or write the file""" if replaygain is None: return self.delete_replay_gain() metadata = self.get_metadata() if metadata is None: metadata = ApeTag([]) metadata[b"replaygain_track_gain"] = ApeTagItem.string( b"replaygain_track_gain", u"{:+.2f} dB".format(replaygain.track_gain)) metadata[b"replaygain_track_peak"] = ApeTagItem.string( b"replaygain_track_peak", u"{:.6f}".format(replaygain.track_peak)) metadata[b"replaygain_album_gain"] = ApeTagItem.string( b"replaygain_album_gain", u"{:+.2f} dB".format(replaygain.album_gain)) metadata[b"replaygain_album_peak"] = ApeTagItem.string( b"replaygain_album_peak", u"{:.6f}".format(replaygain.album_peak)) self.update_metadata(metadata) def delete_replay_gain(self): """removes ReplayGain values from file, if any may raise IOError if unable to modify the file""" metadata = self.get_metadata() if metadata is not None: for field in [b"replaygain_track_gain", b"replaygain_track_peak", b"replaygain_album_gain", b"replaygain_album_peak"]: try: del(metadata[field]) except KeyError: pass self.update_metadata(metadata) ================================================ FILE: audiotools/au.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools import (AudioFile, InvalidFile, PCMReader) from audiotools.pcm import FrameList class InvalidAU(InvalidFile): pass class AuReader(object): def __init__(self, au_filename): from audiotools.bitstream import BitstreamReader from audiotools.text import (ERR_AU_INVALID_HEADER, ERR_AU_UNSUPPORTED_FORMAT) self.stream = BitstreamReader(open(au_filename, "rb"), False) (magic_number, self.data_offset, data_size, encoding_format, self.sample_rate, self.channels) = self.stream.parse("4b 5* 32u") if magic_number != b'.snd': self.stream.close() raise ValueError(ERR_AU_INVALID_HEADER) try: self.bits_per_sample = {2: 8, 3: 16, 4: 24}[encoding_format] except KeyError: self.stream.close() raise ValueError(ERR_AU_UNSUPPORTED_FORMAT) self.channel_mask = {1: 0x4, 2: 0x3}.get(self.channels, 0) self.bytes_per_pcm_frame = ((self.bits_per_sample // 8) * self.channels) self.total_pcm_frames = (data_size // self.bytes_per_pcm_frame) self.remaining_pcm_frames = self.total_pcm_frames def __enter__(self): return self def __exit__(self, exc_type, exc_value, traceback): self.close() def read(self, pcm_frames): # try to read requested PCM frames or remaining frames requested_pcm_frames = min(max(pcm_frames, 1), self.remaining_pcm_frames) requested_bytes = (self.bytes_per_pcm_frame * requested_pcm_frames) pcm_data = self.stream.read_bytes(requested_bytes) # raise exception if data block exhausted early if len(pcm_data) < requested_bytes: from audiotools.text import ERR_AU_TRUNCATED_DATA raise IOError(ERR_AU_TRUNCATED_DATA) else: self.remaining_pcm_frames -= requested_pcm_frames # return parsed chunk return FrameList(pcm_data, self.channels, self.bits_per_sample, True, True) def read_closed(self, pcm_frames): raise ValueError("cannot read closed stream") def seek(self, pcm_frame_offset): if pcm_frame_offset < 0: from audiotools.text import ERR_NEGATIVE_SEEK raise ValueError(ERR_NEGATIVE_SEEK) # ensure one doesn't walk off the end of the file pcm_frame_offset = min(pcm_frame_offset, self.total_pcm_frames) # position file in data block self.stream.seek(self.data_offset + (pcm_frame_offset * self.bytes_per_pcm_frame), 0) self.remaining_pcm_frames = (self.total_pcm_frames - pcm_frame_offset) return pcm_frame_offset def seek_closed(self, pcm_frame_offset): raise ValueError("cannot seek closed stream") def close(self): self.stream.close() self.read = self.read_closed self.seek = self.seek_closed def au_header(sample_rate, channels, bits_per_sample, total_pcm_frames): """given a set of integer stream attributes, returns header string of entire Au header may raise ValueError if the total size of the file is too large""" from audiotools.bitstream import build if ((channels * (bits_per_sample // 8) * total_pcm_frames) >= 2 ** 32): raise ValueError("PCM data too large for Sun AU file") else: return build("4b 5*32u", False, (b".snd", 24, (channels * (bits_per_sample // 8) * total_pcm_frames), {8: 2, 16: 3, 24: 4}[bits_per_sample], sample_rate, channels)) class AuAudio(AudioFile): """a Sun AU audio file""" SUFFIX = "au" NAME = SUFFIX DESCRIPTION = u"Sun Au" def __init__(self, filename): AudioFile.__init__(self, filename) from audiotools.bitstream import parse from audiotools.text import (ERR_AU_INVALID_HEADER, ERR_AU_UNSUPPORTED_FORMAT) try: with open(filename, "rb") as f: (magic_number, self.__data_offset__, self.__data_size__, encoding_format, self.__sample_rate__, self.__channels__) = parse("4b 5* 32u", False, f.read(24)) except IOError as msg: raise InvalidAU(str(msg)) if magic_number != b'.snd': raise InvalidAU(ERR_AU_INVALID_HEADER) try: self.__bits_per_sample__ = {2: 8, 3: 16, 4: 24}[encoding_format] except KeyError: raise InvalidAU(ERR_AU_UNSUPPORTED_FORMAT) def lossless(self): """returns True""" return True def bits_per_sample(self): """returns an integer number of bits-per-sample this track contains""" return self.__bits_per_sample__ def channels(self): """returns an integer number of channels this track contains""" return self.__channels__ def channel_mask(self): from audiotools import ChannelMask """returns a ChannelMask object of this track's channel layout""" if self.channels() <= 2: return ChannelMask.from_channels(self.channels()) else: return ChannelMask(0) def sample_rate(self): """returns the rate of the track's audio as an integer number of Hz""" return self.__sample_rate__ def total_frames(self): """returns the total PCM frames of the track as an integer""" return (self.__data_size__ // (self.__bits_per_sample__ // 8) // self.__channels__) def seekable(self): """returns True if the file is seekable""" return True @classmethod def supports_to_pcm(cls): """returns True if all necessary components are available to support the .to_pcm() method""" return True def to_pcm(self): """returns a PCMReader object containing the track's PCM data""" return AuReader(self.filename) def pcm_split(self): """returns a pair of data strings before and after PCM data the first contains all data before the PCM content of the data chunk the second containing all data after the data chunk""" import struct f = open(self.filename, 'rb') (magic_number, data_offset) = struct.unpack(">4sI", f.read(8)) header = f.read(data_offset - struct.calcsize(">4sI")) return (struct.pack(">4sI{:d}s".format(len(header)), magic_number, data_offset, header), "") @classmethod def supports_from_pcm(cls): """returns True if all necessary components are available to support the .from_pcm() classmethod""" return True @classmethod def from_pcm(cls, filename, pcmreader, compression=None, total_pcm_frames=None): """encodes a new file from PCM data takes a filename string, PCMReader object, optional compression level string and optional total_pcm_frames integer encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new AuAudio object""" from audiotools import EncodingError from audiotools import DecodingError from audiotools import CounterPCMReader from audiotools import transfer_framelist_data if pcmreader.bits_per_sample not in {8, 16, 24}: from audiotools import UnsupportedBitsPerSample pcmreader.close() raise UnsupportedBitsPerSample(filename, pcmreader.bits_per_sample) try: header = au_header(pcmreader.sample_rate, pcmreader.channels, pcmreader.bits_per_sample, total_pcm_frames if total_pcm_frames is not None else 0) except ValueError as err: raise EncodingError(str(err)) try: f = open(filename, "wb") except IOError as err: pcmreader.close() raise EncodingError(str(err)) counter = CounterPCMReader(pcmreader) f.write(header) try: transfer_framelist_data(counter, f.write, True, True) except (IOError, ValueError) as err: f.close() cls.__unlink__(filename) raise EncodingError(str(err)) if total_pcm_frames is not None: f.close() if total_pcm_frames != counter.frames_written: # ensure written number of PCM frames # matches total_pcm_frames argument from audiotools.text import ERR_TOTAL_PCM_FRAMES_MISMATCH cls.__unlink__(filename) raise EncodingError(ERR_TOTAL_PCM_FRAMES_MISMATCH) else: # go back and rewrite populated header # with counted number of PCM frames f.seek(0, 0) f.write(au_header(pcmreader.sample_rate, pcmreader.channels, pcmreader.bits_per_sample, counter.frames_written)) f.close() return AuAudio(filename) @classmethod def track_name(cls, file_path, track_metadata=None, format=None, suffix=None): """constructs a new filename string given a plain string to an existing path, a MetaData-compatible object (or None), a UTF-8-encoded Python format string and an ASCII-encoded suffix string (such as "mp3") returns a plain string of a new filename with format's fields filled-in and encoded as FS_ENCODING raises UnsupportedTracknameField if the format string contains invalid template fields""" if format is None: format = "track%(track_number)2.2d.au" return AudioFile.track_name(file_path, track_metadata, format, suffix=cls.SUFFIX) ================================================ FILE: audiotools/cdtoc.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools import PY3, Sheet, SheetTrack, SheetIndex class CDTOC(Sheet): """an object representing a CDDA layout as a Vorbis comment tag""" def __init__(self, cdtoc_tracks, lead_out): """cdtoc_tracks is a list of CDTOC_Track objects lead_out is the address of the lead out""" self.__cdtoc_tracks__ = cdtoc_tracks self.__lead_out__ = lead_out def __repr__(self): return "CDTOC({!r}, {!r})".format(self.__cdtoc_tracks__, self.__lead_out__) if PY3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('utf-8') def __unicode__(self): return u"+".join([u"{:X}".format( len([t for t in self if t.is_audio()]))] + [t.__unicode__() for t in self] + [u"{:X}".format(self.__lead_out__)]) @classmethod def from_unicode(cls, u): """given a Unicode string, returns a CDTOC may raise ValueError if a parsing problem occurs""" items = u.split(u"+") track_count = int(items[0], 16) audio_track_addresses = [int(i, 16) for i in items[1:1 + track_count]] remaining_items = items[1 + track_count:] if len(remaining_items) == 1: # all audio tracks lead_out_address = int(remaining_items[0], 16) return cls([CDTOC_Track(i, address) for (i, address) in enumerate(audio_track_addresses, 1)], lead_out_address) elif len(remaining_items) == 2: # might be CDExtra or Data+Audio if remaining_items[1].startswith(u"X"): # Data+Audio lead_out_address = int(remaining_items[0], 16) data_track_address = int(remaining_items[1][1:], 16) return CDTOC_DataAudio( [CDTOC_Track(1, data_track_address, False)] + [CDTOC_Track(i, address) for (i, address) in enumerate(audio_track_addresses, 2)], lead_out_address) else: # CDExtra data_track_address = int(remaining_items[0], 16) lead_out_address = int(remaining_items[1], 16) return cls([CDTOC_Track(i, address) for (i, address) in enumerate(audio_track_addresses, 1)] + [CDTOC_Track(track_count + 1, data_track_address, False)], lead_out_address) else: raise ValueError("too many items") @classmethod def converted(cls, sheet, seconds_length): """given a Sheet-compatible object and length of entire disc as a Fractional number of seconds, returns a CDTOC""" sheet_tracks = list(sheet) is_audio = [t.is_audio() for t in sheet_tracks] if False not in is_audio: # all tracks are audio, so not CDExtra or Data+Audio return cls([CDTOC_Track.converted(t) for t in sheet_tracks], int(seconds_length * 75) + 150) elif (not is_audio[0]) and (False not in is_audio[1:]): # Data+Audio return CDTOC_DataAudio( [CDTOC_Track.converted(t) for t in sheet_tracks], int(seconds_length * 75) + 150) elif (False not in is_audio[0:-1]) and not is_audio[-1]: # CDExtra return cls([CDTOC_Track.converted(t) for t in sheet_tracks], int(seconds_length * 75) + 150) else: raise ValueError("unsupported Sheet layout") def __len__(self): return len(self.__cdtoc_tracks__) def __getitem__(self, index): return self.__cdtoc_tracks__[index] def track_length(self, track_number, total_length=None): """given a track_number (typically starting from 1) and optional total length as a Fraction number of seconds (including the disc's pre-gap, if any), returns the length of the track as a Fraction number of seconds or None if the length is to the remainder of the stream (typically for the last track in the album) may raise KeyError if the track is not found""" initial_track = self.track(track_number) if track_number < len(self): next_track = self.track(track_number + 1) return (next_track.index(1).offset() - initial_track.index(1).offset()) else: # no next track, so return to end of lead-out from fractions import Fraction return (Fraction(self.__lead_out__ - 150, 75) - initial_track.index(1).offset()) def get_metadata(self): return None class CDTOC_DataAudio(CDTOC): def __init__(self, cdtoc_tracks, lead_out): """cdtoc_tracks is a list of CDTOC_Track objects lead_out is the address of the lead out""" # first track must be non-audio, rest must be audio assert(not cdtoc_tracks[0].is_audio()) assert(False not in [t.is_audio() for t in cdtoc_tracks[1:]]) CDTOC.__init__(self, cdtoc_tracks, lead_out) def __unicode__(self): return u"+".join([u"{:X}".format(len(self) - 1)] + [t.__unicode__() for t in self[1:]] + [u"{:X}".format(self.__lead_out__), u"X{}".format(self[0].__unicode__())]) def __repr__(self): return "CDTOC_DataAudio({!r}, {!r})".format(self.__cdtoc_tracks__, self.__lead_out__) class CDTOC_Track(SheetTrack): def __init__(self, number, address, is_audio=True): """number is the track number, starting from 1 address is the track's LBA + 150 value is_audio determines whether the track contains audio data""" self.__number__ = number if (number == 1) and (address > 150): # add index point 0 as pre-gap self.__indexes__ = [CDTOC_Index(0, 150), CDTOC_Index(1, address)] else: self.__indexes__ = [CDTOC_Index(1, address)] self.__is_audio__ = is_audio @classmethod def converted(cls, sheet_track): """given a SheetTrack object, returns a CDTOC_Track""" index_1 = sheet_track.index(1).offset() return cls(sheet_track.number(), int(index_1 * 75) + 150, sheet_track.is_audio()) def __len__(self): return len(self.__indexes__) def __getitem__(self, i): return self.__indexes__[i] if PY3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('utf-8') def __unicode__(self): return self.index(1).__unicode__() def __repr__(self): return "CDTOC_Track({!r}, {!r}, {!r})".format( self.__number__, self.__indexes__, self.__is_audio__) def number(self): """return SheetTrack's number, starting from 1""" return self.__number__ def get_metadata(self): """returns SheetTrack's MetaData, or None""" return None def filename(self): """returns SheetTrack's filename as unicode""" return u"CDImage.wav" def is_audio(self): """returns whether SheetTrack contains audio data""" return self.__is_audio__ def pre_emphasis(self): """returns whether SheetTrack has pre-emphasis""" return False def copy_permitted(self): """returns whether copying is permitted""" return False class CDTOC_Index(SheetIndex): def __init__(self, number, address): self.__number__ = number self.__address__ = address if PY3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('utf-8') def __unicode__(self): return u"{:X}".format(self.__address__) def __repr__(self): return "CDTOC_Index(number={!r}, address={!r})".format( self.__number__, self.__address__) def number(self): return self.__number__ def offset(self): from fractions import Fraction return Fraction(self.__address__ - 150, 75) ================================================ FILE: audiotools/coverartarchive.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA def perform_lookup(mbid, coverartarchive_server="coverartarchive.org", coverartarchive_port=80): """given a MusicBrainz ID as a plain string, and optional coverartarchive server/port, returns a list of Image objects for each cover may return an empty list if no covers are found or raise an exception if a problem occurs querying the server""" from audiotools import Image from audiotools import FRONT_COVER from audiotools import BACK_COVER try: from urllib.request import urlopen from urllib.request import URLError except ImportError: from urllib2 import urlopen from urllib2 import URLError from json import loads # query server for JSON data about MBID release try: j = urlopen("http://{server}:{port}/release/{release}/".format( server=coverartarchive_server, port=coverartarchive_port, release=mbid)) except URLError: return [] json_data = loads(j.read().decode("utf-8", "replace")) j.close() images = [] # get URLs of all front and back cover art in list try: for image in json_data[u"images"]: if image[u"front"] or image[u"back"]: try: data = urlopen(image[u"image"]) images.append( Image.new( data.read(), u"", FRONT_COVER if image[u"front"] else BACK_COVER)) data.close() except URLError: # skip images that aren't found pass except KeyError: pass # return list of all fetched cover art return images ================================================ FILE: audiotools/cue/__init__.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA """the cuesheet module""" from audiotools import Sheet, SheetTrack, SheetIndex, SheetException class Cuesheet(Sheet): def __init__(self, files, catalog=None, title=None, performer=None, songwriter=None, cdtextfile=None): from sys import version_info str_type = str if (version_info[0] >= 3) else unicode assert((catalog is None) or isinstance(catalog, str_type)) assert((title is None) or isinstance(title, str_type)) assert((performer is None) or isinstance(performer, str_type)) assert((songwriter is None) or isinstance(songwriter, str_type)) assert((cdtextfile is None) or isinstance(cdtextfile, str_type)) self.__files__ = files self.__catalog__ = catalog self.__title__ = title self.__performer__ = performer self.__songwriter__ = songwriter self.__cdtextfile__ = cdtextfile def __repr__(self): return "Cuesheet({})".format( ", ".join(["{}={!r}".format(attr, getattr(self, "__" + attr + "__")) for attr in ["files", "catalog", "title", "performer", "songwriter", "cdtextfile"]])) @classmethod def converted(cls, sheet, filename=None): """given a Sheet object, returns a Cuesheet object""" def group_tracks(tracks): current_file = None current_tracks = [] for track in tracks: if current_file is None: current_file = track.filename() current_tracks = [track] elif current_file != track.filename(): yield (current_file, current_tracks) current_file = track.filename() current_tracks = [track] else: current_tracks.append(track) else: if current_file is not None: yield (current_file, current_tracks) metadata = sheet.get_metadata() args = {"files": [File(filename=(track_filename if filename is None else filename), file_type=u"WAVE", tracks=[Track.converted(t) for t in tracks]) for (track_filename, tracks) in group_tracks(sheet)]} if metadata is not None: args["catalog"] = metadata.catalog args["title"] = metadata.album_name args["performer"] = metadata.performer_name args["songwriter"] = metadata.artist_name return cls(**args) def __len__(self): return sum(map(len, self.__files__)) def __getitem__(self, index): not_found = IndexError(index) for file in self.__files__: if index < len(file): return file[index] else: index -= len(file) else: raise not_found def get_metadata(self): """returns MetaData of Sheet, or None this metadata often contains information such as catalog number or CD-TEXT values""" from operator import or_ from functools import reduce if (reduce(or_, [(attr is not None) for attr in [self.__catalog__, self.__title__, self.__performer__, self.__songwriter__]], False)): from audiotools import MetaData return MetaData(catalog=self.__catalog__, album_name=self.__title__, performer_name=self.__performer__, artist_name=self.__songwriter__) else: return None def build(self): """returns the Cuesheet as a unicode string""" items = [] if self.__catalog__ is not None: items.append( u"CATALOG {}".format(format_string(self.__catalog__))) if self.__title__ is not None: items.append( u"TITLE {}".format(format_string(self.__title__))) if self.__performer__ is not None: items.append( u"PERFORMER {}".format(format_string(self.__performer__))) if self.__songwriter__ is not None: items.append( u"SONGWRITER {}".format(format_string(self.__songwriter__))) if self.__cdtextfile__ is not None: items.append( u"CDTEXTFILE {}".format(format_string(self.__cdtextfile__))) if len(items) > 0: return (u"\r\n".join(items) + u"\r\n" + u"\r\n".join([f.build() for f in self.__files__]) + u"\r\n") else: return u"\r\n".join([f.build() for f in self.__files__]) + u"\r\n" class File(object): def __init__(self, filename, file_type, tracks): from sys import version_info str_type = str if (version_info[0] >= 3) else unicode assert(isinstance(filename, str_type)) assert(isinstance(file_type, str_type)) self.__filename__ = filename self.__file_type__ = file_type for t in tracks: t.__parent_file__ = self self.__tracks__ = tracks def __len__(self): return len(self.__tracks__) def __getitem__(self, index): return self.__tracks__[index] def __repr__(self): return "File({})".format( ", ".join(["{}={!r}".format(attr, getattr(self, "__" + attr + "__")) for attr in ["filename", "file_type", "tracks"]])) def filename(self): return self.__filename__ def build(self): """returns the File as a unicode string""" return u"FILE {} {}\r\n{}".format( format_string(self.__filename__), self.__file_type__, u"\r\n".join([t.build() for t in self.__tracks__])) class Track(SheetTrack): def __init__(self, number, track_type, indexes, isrc=None, pregap=None, postgap=None, flags=None, title=None, performer=None, songwriter=None): from sys import version_info str_type = str if (version_info[0] >= 3) else unicode assert(isinstance(number, int)) assert(isinstance(track_type, str_type)) assert((isrc is None) or isinstance(isrc, str_type)) assert((pregap is None) or isinstance(pregap, int)) assert((postgap is None) or isinstance(postgap, int)) if flags is not None: for flag in flags: assert(isinstance(flag, str_type)) assert((title is None) or isinstance(title, str_type)) assert((performer is None) or isinstance(performer, str_type)) assert((songwriter is None) or isinstance(songwriter, str_type)) self.__parent_file__ = None # to be assigned by File self.__number__ = number self.__track_type__ = track_type self.__indexes__ = list(indexes) self.__isrc__ = isrc self.__pregap__ = pregap self.__postgap__ = postgap self.__flags__ = flags self.__title__ = title self.__performer__ = performer self.__songwriter__ = songwriter def __repr__(self): return "Track({})".format( ", ".join(["{}={!r}".format(attr, getattr(self, "__" + attr + "__")) for attr in ["number", "track_type", "indexes", "isrc", "pregap", "postgap", "flags", "title", "performer", "songwriter"]])) @classmethod def converted(cls, sheettrack): """given a SheetTrack object, returns a Track object""" metadata = sheettrack.get_metadata() args = {"number": sheettrack.number(), "track_type": (u"AUDIO" if sheettrack.is_audio() else u"MODE1/2352"), "indexes": [Index.converted(i) for i in sheettrack]} if metadata is not None: args["isrc"] = metadata.ISRC args["title"] = metadata.track_name args["performer"] = metadata.performer_name args["songwriter"] = metadata.artist_name if sheettrack.pre_emphasis() and sheettrack.copy_permitted(): args["flags"] = [u"PRE", u"DCP"] elif sheettrack.pre_emphasis(): args["flags"] = [u"PRE"] elif sheettrack.copy_permitted(): args["flags"] = [u"DCP"] return cls(**args) def __len__(self): return len(self.__indexes__) def __getitem__(self, index): return self.__indexes__[index] def number(self): """return SheetTrack's number, starting from 1""" return self.__number__ def get_metadata(self): """returns SheetTrack's MetaData, or None""" from operator import or_ from functools import reduce if (reduce(or_, [(attr is not None) for attr in [self.__isrc__, self.__title__, self.__performer__, self.__songwriter__]], False)): from audiotools import MetaData return MetaData(ISRC=self.__isrc__, track_name=self.__title__, performer_name=self.__performer__, artist_name=self.__songwriter__) else: return None def filename(self): """returns SheetTrack's filename as a unicode string""" if self.__parent_file__ is not None: return self.__parent_file__.filename() else: return u"" def is_audio(self): """returns whether SheetTrack contains audio data""" return self.__track_type__ == u"AUDIO" def pre_emphasis(self): """returns whether SheetTrack has pre-emphasis""" if self.__flags__ is not None: return u"PRE" in self.__flags__ else: return False def copy_permitted(self): """returns whether copying is permitted""" if self.__flags__ is not None: return u"DCP" in self.__flags__ else: return False def build(self): """returns the Track as a string""" items = [] if self.__title__ is not None: items.append( u" TITLE {}".format(format_string(self.__title__))) if self.__performer__ is not None: items.append( u" PERFORMER {}".format(format_string(self.__performer__))) if self.__songwriter__ is not None: items.append( u" SONGWRITER {}".format( format_string(self.__songwriter__))) if self.__flags__ is not None: items.append( u" FLAGS {}".format(" ".join(self.__flags__))) if self.__isrc__ is not None: items.append( u" ISRC {}".format(self.__isrc__)) if self.__pregap__ is not None: items.append( u" PREGAP {}".format(format_timestamp(self.__pregap__))) for index in self.__indexes__: items.append(index.build()) if self.__postgap__ is not None: items.append( u" POSTGAP {}".format(format_timestamp(self.__postgap__))) return u" TRACK {:02d} {}\r\n{}".format(self.__number__, self.__track_type__, u"\r\n".join(items)) class Index(SheetIndex): def __init__(self, number, timestamp): self.__number__ = number self.__timestamp__ = timestamp def __repr__(self): return "Index(number={!r},timestamp={!r})".format( self.__number__, self.__timestamp__) @classmethod def converted(cls, index): """given a SheetIndex object, returns an Index object""" return cls(number=index.number(), timestamp=int(index.offset() * 75)) def build(self): """returns the Index as a string""" return u" INDEX {:02d} {}".format( self.__number__, format_timestamp(self.__timestamp__)) def number(self): """returns the index's number (typically starting from 1)""" return self.__number__ def offset(self): """returns the index's offset from the start of the stream in seconds as a Fraction object""" from fractions import Fraction return Fraction(self.__timestamp__, 75) def format_string(s): return u"\"{}\"".format(s.replace(u'\\', u'\\\\').replace(u'"', u'\\"')) def format_timestamp(t): return u"{:02d}:{:02d}:{:02d}".format(t // 75 // 60, t // 75 % 60, t % 75) def read_cuesheet(filename): """returns a Cuesheet from a cuesheet filename on disk raises SheetException if some error occurs reading or parsing the file """ from audiotools import SheetException try: with open(filename, "rb") as f: return read_cuesheet_string(f.read().decode("UTF-8")) except IOError: raise SheetException("unable to open file") def read_cuesheet_string(cuesheet): """given a unicode string of cuesheet data returns a Cuesheet object raises SheetException if some error occurs parsing the file""" import audiotools.ply.lex as lex import audiotools.ply.yacc as yacc from audiotools.ply.yacc import NullLogger import audiotools.cue.tokrules import audiotools.cue.yaccrules from sys import version_info str_type = str if (version_info[0] >= 3) else unicode assert(isinstance(cuesheet, str_type)) lexer = lex.lex(module=audiotools.cue.tokrules) lexer.input(cuesheet) parser = yacc.yacc(module=audiotools.cue.yaccrules, debug=0, errorlog=NullLogger(), write_tables=0) try: return parser.parse(lexer=lexer) except ValueError as err: raise SheetException(str(err)) def write_cuesheet(sheet, filename, file): """given a Sheet object and filename unicode string, writes a .cue file to the given file object""" file.write( Cuesheet.converted(sheet, filename=filename).build().encode("UTF-8")) ================================================ FILE: audiotools/cue/tokrules.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA RESERVED = {"CATALOG": "CATALOG", "CDTEXTFILE": "CDTEXTFILE", "FILE": "FILE", "BINARY": "BINARY", "MOTOROLA": "MOTOROLA", "AIFF": "AIFF", "WAVE": "WAVE", "FLAC": "FLAC", "FLAGS": "FLAGS", "DCP": "DCP", "PRE": "PRE", "SCMS": "SCMS", "INDEX": "INDEX", "ISRC": "ISRC_ID", "PERFORMER": "PERFORMER", "POSTGAP": "POSTGAP", "PREGAP": "PREGAP", "SONGWRITER": "SONGWRITER", "TITLE": "TITLE", "TRACK": "TRACK", "AUDIO": "AUDIO", "CDG": "CDG"} tokens = ["REM", "ISRC", "TIMESTAMP", "MP3", "MODE", "CDI", "NUMBER", "ID", "STRING"] + list(RESERVED.values()) def t_REM(t): r"REM .*" pass def t_ISRC(t): r'[A-Z]{2}[A-Za-z0-9]{3}[0-9]{7}' return t def t_TIMESTAMP(t): r'[0-9]{1,3}:[0-9]{1,2}:[0-9]{1,2}' (m, s, f) = t.value.split(":") t.value = ((int(m) * 75 * 60) + (int(s) * 75) + (int(f))) return t def t_MP3(t): r'MP3' return t def t_MODE(t): r'MODE1/2048|MODE1/2352|MODE2/2336|MODE2/2352' return t def t_CDI(t): r'CDI/2336|CDI/2352' return t def t_NUMBER(t): r'[0-9]+' t.value = int(t.value) return t def t_ID(t): r"[A-Z]+" if t.value in RESERVED.keys(): t.type = RESERVED[t.value] else: t.type = "STRING" return t def t_STRING(t): r'\"(\\.|[^"])*\"' from re import sub t.value = sub(r'\\.', lambda s: s.group(0)[1:], t.value[1:-1]) return t t_ignore = " \r\t" def t_newline(t): r'\n+' t.lexer.lineno += t.value.count("\n") def t_error(t): raise ValueError("illegal character {!r}".format(t.value[0])) ================================================ FILE: audiotools/cue/yaccrules.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools.cue.tokrules import tokens def p_cuesheet(t): '''cuesheet : files | cuesheet_items files''' from audiotools.cue import Cuesheet if len(t) == 2: t[0] = Cuesheet(files=t[1]) else: # FIXME - check against multiple "once only" attributes args = {} for (key, value) in t[1]: args[key] = value t[0] = Cuesheet(files=t[2], **args) def p_cuesheet_items(t): '''cuesheet_items : cuesheet_item | cuesheet_items cuesheet_item''' if len(t) == 2: t[0] = [t[1]] else: t[0] = t[1] + [t[2]] def p_cuesheet_item(t): '''cuesheet_item : catalog | title | performer | songwriter | cdtextfile''' t[0] = t[1] def p_catalog_string(t): 'catalog : CATALOG STRING' t[0] = ("catalog", t[2]) def p_catalog_number(t): 'catalog : CATALOG NUMBER' t[0] = ("catalog", u"{:013d}".format(t[2])) def p_title(t): 'title : TITLE STRING' t[0] = ("title", t[2]) def p_performer(t): 'performer : PERFORMER STRING' t[0] = ("performer", t[2]) def p_songwriter(t): 'songwriter : SONGWRITER STRING' t[0] = ("songwriter", t[2]) def p_cdtextfile(t): 'cdtextfile : CDTEXTFILE STRING' t[0] = ("cdtextfile", t[2]) def p_files(t): '''files : file | files file''' if len(t) == 2: t[0] = [t[1]] else: t[0] = t[1] + [t[2]] def p_file(t): 'file : FILE STRING filetype tracks' from audiotools.cue import File # FIXME - ensure tracks are ascending t[0] = File(filename=t[2], file_type=t[3], tracks=t[4]) def p_filetype(t): '''filetype : BINARY | MOTOROLA | AIFF | WAVE | MP3 | FLAC''' t[0] = t[1] def p_tracks(t): '''tracks : track | tracks track''' if len(t) == 2: t[0] = [t[1]] else: t[0] = t[1] + [t[2]] def p_track(t): 'track : TRACK NUMBER tracktype trackitems' from audiotools.cue import Track indexes = [] args = {} for (key, value) in t[4]: if key == "index": indexes.append(value) else: args[key] = value # FIXME - ensure that index points are ascending t[0] = Track(number=t[2], track_type=t[3], indexes=indexes, **args) def p_tracktype(t): '''tracktype : AUDIO | CDG | MODE | CDI''' # FIXME - return whether type is audio or not t[0] = t[1] def p_indexes(t): '''trackitems : trackitem | trackitems trackitem''' if len(t) == 2: t[0] = [t[1]] else: t[0] = t[1] + [t[2]] def p_trackitem(t): '''trackitem : index | isrc | pregap | postgap | flags | title | performer | songwriter''' t[0] = t[1] def p_index(t): 'index : INDEX NUMBER TIMESTAMP' from audiotools.cue import Index t[0] = ("index", Index(number=t[2], timestamp=t[3])) def p_isrc(t): '''isrc : ISRC_ID ISRC | ISRC_ID NUMBER''' t[0] = ("isrc", u"{}".format(t[2])) def p_pregap(t): 'pregap : PREGAP TIMESTAMP' t[0] = ("pregap", t[2]) def p_postgap(t): 'postgap : POSTGAP TIMESTAMP' t[0] = ("postgap", t[2]) def p_flags(t): 'flags : FLAGS flaglist' t[0] = ("flags", t[2]) def p_flaglist(t): '''flaglist : flag | flaglist flag''' if len(t) == 2: t[0] = [t[1]] else: t[0] = t[1] + [t[2]] def p_flag(t): '''flag : DCP | PRE | SCMS''' t[0] = t[1] def p_error(t): from audiotools.text import ERR_CUE_SYNTAX_ERROR raise ValueError(ERR_CUE_SYNTAX_ERROR.format(t.lexer.lineno)) ================================================ FILE: audiotools/flac.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools import (AudioFile, MetaData, InvalidFile, Image, WaveContainer, AiffContainer, Sheet, SheetTrack, SheetIndex) from audiotools.vorbiscomment import VorbisComment # the maximum padding size to use when rewriting metadata blocks MAX_PADDING_SIZE = 2 ** 20 class InvalidFLAC(InvalidFile): pass class FlacMetaDataBlockTooLarge(Exception): """raised if one attempts to build a FlacMetaDataBlock too large""" pass class FlacMetaData(MetaData): """a class for managing a native FLAC's metadata""" def __init__(self, blocks): MetaData.__setattr__(self, "block_list", list(blocks)) def has_block(self, block_id): """returns True if the given block ID is present""" return block_id in (b.BLOCK_ID for b in self.block_list) def add_block(self, block): """adds the given block to our list of blocks""" # the specification only requires that STREAMINFO be first # the rest are largely arbitrary, # though I like to keep PADDING as the last block for aesthetic reasons PREFERRED_ORDER = [Flac_STREAMINFO.BLOCK_ID, Flac_SEEKTABLE.BLOCK_ID, Flac_CUESHEET.BLOCK_ID, Flac_VORBISCOMMENT.BLOCK_ID, Flac_PICTURE.BLOCK_ID, Flac_APPLICATION.BLOCK_ID, Flac_PADDING.BLOCK_ID] stop_blocks = set( PREFERRED_ORDER[PREFERRED_ORDER.index(block.BLOCK_ID) + 1:]) for (index, old_block) in enumerate(self.block_list): if old_block.BLOCK_ID in stop_blocks: self.block_list.insert(index, block) break else: self.block_list.append(block) def get_block(self, block_id): """returns the first instance of the given block_id may raise IndexError if the block is not in our list of blocks""" for block in self.block_list: if block.BLOCK_ID == block_id: return block else: raise IndexError() def get_blocks(self, block_id): """returns all instances of the given block_id in our list of blocks""" return [b for b in self.block_list if (b.BLOCK_ID == block_id)] def replace_blocks(self, block_id, blocks): """replaces all instances of the given block_id with blocks taken from the given list if insufficient matching blocks are present, this uses add_block() to populate the remainder if additional matching blocks are present, they are removed """ new_blocks = [] for block in self.block_list: if block.BLOCK_ID == block_id: if len(blocks) > 0: new_blocks.append(blocks.pop(0)) else: pass else: new_blocks.append(block) self.block_list = new_blocks while len(blocks) > 0: self.add_block(blocks.pop(0)) def __setattr__(self, attr, value): if attr in self.FIELDS: try: vorbis_comment = self.get_block(Flac_VORBISCOMMENT.BLOCK_ID) except IndexError: # add VORBIS comment block if necessary from audiotools import VERSION vorbis_comment = Flac_VORBISCOMMENT( [], u"Python Audio Tools {}".format(VERSION)) self.add_block(vorbis_comment) setattr(vorbis_comment, attr, value) else: MetaData.__setattr__(self, attr, value) def __getattr__(self, attr): if attr in self.FIELDS: try: return getattr(self.get_block(Flac_VORBISCOMMENT.BLOCK_ID), attr) except IndexError: # no VORBIS comment block, so all values are None return None else: return MetaData.__getattribute__(self, attr) def __delattr__(self, attr): if attr in self.FIELDS: try: delattr(self.get_block(Flac_VORBISCOMMENT.BLOCK_ID), attr) except IndexError: # no VORBIS comment block, so nothing to delete pass else: MetaData.__delattr__(self, attr) @classmethod def converted(cls, metadata): """takes a MetaData object and returns a FlacMetaData object""" if metadata is None: return None elif isinstance(metadata, FlacMetaData): return cls([block.copy() for block in metadata.block_list]) else: return cls([Flac_VORBISCOMMENT.converted(metadata)] + [Flac_PICTURE.converted(image) for image in metadata.images()] + [Flac_PADDING(4096)]) def add_image(self, image): """embeds an Image object in this metadata""" self.add_block(Flac_PICTURE.converted(image)) def delete_image(self, image): """deletes an image object from this metadata""" self.block_list = [b for b in self.block_list if not ((b.BLOCK_ID == Flac_PICTURE.BLOCK_ID) and (b == image))] def images(self): """returns a list of embedded Image objects""" return self.get_blocks(Flac_PICTURE.BLOCK_ID) @classmethod def supports_images(cls): """returns True""" return True def clean(self): """returns (FlacMetaData, [fixes]) tuple where FlacMetaData is a new MetaData object fixed of problems and fixes is a list of Unicode strings of fixes performed """ from audiotools.text import (CLEAN_FLAC_REORDERED_STREAMINFO, CLEAN_FLAC_MULITPLE_STREAMINFO, CLEAN_FLAC_MULTIPLE_VORBISCOMMENT, CLEAN_FLAC_MULTIPLE_SEEKTABLE, CLEAN_FLAC_MULTIPLE_CUESHEET, CLEAN_FLAC_UNDEFINED_BLOCK) fixes_performed = [] cleaned_blocks = [] for block in self.block_list: if block.BLOCK_ID == Flac_STREAMINFO.BLOCK_ID: # reorder STREAMINFO block to be first, if necessary if len(cleaned_blocks) == 0: cleaned_blocks.append(block) elif cleaned_blocks[0].BLOCK_ID != block.BLOCK_ID: fixes_performed.append( CLEAN_FLAC_REORDERED_STREAMINFO) cleaned_blocks.insert(0, block) else: fixes_performed.append( CLEAN_FLAC_MULITPLE_STREAMINFO) elif block.BLOCK_ID == Flac_VORBISCOMMENT.BLOCK_ID: if block.BLOCK_ID in [b.BLOCK_ID for b in cleaned_blocks]: # remove redundant VORBIS_COMMENT blocks fixes_performed.append( CLEAN_FLAC_MULTIPLE_VORBISCOMMENT) else: # recursively clean up the text fields in FlacVorbisComment (block, block_fixes) = block.clean() cleaned_blocks.append(block) fixes_performed.extend(block_fixes) elif block.BLOCK_ID == Flac_PICTURE.BLOCK_ID: # recursively clean up any image blocks (block, block_fixes) = block.clean() cleaned_blocks.append(block) fixes_performed.extend(block_fixes) elif block.BLOCK_ID == Flac_APPLICATION.BLOCK_ID: cleaned_blocks.append(block) elif block.BLOCK_ID == Flac_SEEKTABLE.BLOCK_ID: # remove redundant seektable, if necessary if block.BLOCK_ID in [b.BLOCK_ID for b in cleaned_blocks]: fixes_performed.append( CLEAN_FLAC_MULTIPLE_SEEKTABLE) else: (block, block_fixes) = block.clean() cleaned_blocks.append(block) fixes_performed.extend(block_fixes) elif block.BLOCK_ID == Flac_CUESHEET.BLOCK_ID: # remove redundant cuesheet, if necessary if block.BLOCK_ID in [b.BLOCK_ID for b in cleaned_blocks]: fixes_performed.append( CLEAN_FLAC_MULTIPLE_CUESHEET) else: cleaned_blocks.append(block) elif block.BLOCK_ID == Flac_PADDING.BLOCK_ID: cleaned_blocks.append(block) else: # remove undefined blocks fixes_performed.append(CLEAN_FLAC_UNDEFINED_BLOCK) return (self.__class__(cleaned_blocks), fixes_performed) def __repr__(self): return "FlacMetaData({!r})".format(self.block_list) def intersection(self, metadata): """given a MetaData-compatible object, returns a new MetaData object which contains all the matching fields and images of this object and 'metadata' """ def block_present(block): for other_block in metadata.get_blocks(block.BLOCK_ID): if block == other_block: return True else: return False if type(metadata) is FlacMetaData: blocks = [] for block in self.block_list: if ((block.BLOCK_ID == Flac_VORBISCOMMENT.BLOCK_ID) and metadata.has_block(block.BLOCK_ID)): # merge VORBIS blocks seperately, if present blocks.append( block.intersection(metadata.get_block(block.BLOCK_ID))) elif block_present(block): blocks.append(block.copy()) return FlacMetaData(blocks) else: return MetaData.intersection(self, metadata) @classmethod def parse(cls, reader): """returns a FlacMetaData object from the given BitstreamReader which has already parsed the 4-byte 'fLaC' file ID""" block_list = [] last = 0 while last != 1: (last, block_type, block_length) = reader.parse("1u7u24u") if block_type == 0: # STREAMINFO block_list.append( Flac_STREAMINFO.parse(reader)) elif block_type == 1: # PADDING block_list.append( Flac_PADDING.parse(reader, block_length)) elif block_type == 2: # APPLICATION block_list.append( Flac_APPLICATION.parse(reader, block_length)) elif block_type == 3: # SEEKTABLE block_list.append( Flac_SEEKTABLE.parse(reader, block_length // 18)) elif block_type == 4: # VORBIS_COMMENT block_list.append( Flac_VORBISCOMMENT.parse(reader)) elif block_type == 5: # CUESHEET block_list.append( Flac_CUESHEET.parse(reader)) elif block_type == 6: # PICTURE block_list.append( Flac_PICTURE.parse(reader)) elif (block_type >= 7) and (block_type <= 126): from audiotools.text import ERR_FLAC_RESERVED_BLOCK raise ValueError(ERR_FLAC_RESERVED_BLOCK.format(block_type)) else: from audiotools.text import ERR_FLAC_INVALID_BLOCK raise ValueError(ERR_FLAC_INVALID_BLOCK) return cls(block_list) def raw_info(self): """returns human-readable metadata as a unicode string""" from os import linesep return linesep.join( [u"FLAC Tags:"] + [block.raw_info() for block in self.blocks()]) def blocks(self): """yields FlacMetaData's individual metadata blocks""" for block in self.block_list: yield block def build(self, writer): """writes the FlacMetaData to the given BitstreamWriter not including the 4-byte 'fLaC' file ID""" from audiotools import iter_last for (last_block, block) in iter_last(iter([b for b in self.blocks() if (b.size() < (2 ** 24))])): if not last_block: writer.build("1u7u24u", (0, block.BLOCK_ID, block.size())) else: writer.build("1u7u24u", (1, block.BLOCK_ID, block.size())) block.build(writer) def size(self): """returns the size of all metadata blocks including the block headers but not including the 4-byte 'fLaC' file ID""" return sum(4 + b.size() for b in self.block_list) class Flac_STREAMINFO(object): BLOCK_ID = 0 def __init__(self, minimum_block_size, maximum_block_size, minimum_frame_size, maximum_frame_size, sample_rate, channels, bits_per_sample, total_samples, md5sum): """all values are non-negative integers except for md5sum which is a 16-byte binary string""" self.minimum_block_size = minimum_block_size self.maximum_block_size = maximum_block_size self.minimum_frame_size = minimum_frame_size self.maximum_frame_size = maximum_frame_size self.sample_rate = sample_rate self.channels = channels self.bits_per_sample = bits_per_sample self.total_samples = total_samples self.md5sum = md5sum def copy(self): """returns a duplicate of this metadata block""" return Flac_STREAMINFO(self.minimum_block_size, self.maximum_block_size, self.minimum_frame_size, self.maximum_frame_size, self.sample_rate, self.channels, self.bits_per_sample, self.total_samples, self.md5sum) def __eq__(self, block): for attr in ["minimum_block_size", "maximum_block_size", "minimum_frame_size", "maximum_frame_size", "sample_rate", "channels", "bits_per_sample", "total_samples", "md5sum"]: if ((not hasattr(block, attr)) or (getattr(self, attr) != getattr(block, attr))): return False else: return True def __repr__(self): return "Flac_STREAMINFO({})".format(",".join( ["{}={!r}".format(key, getattr(self, key)) for key in ["minimum_block_size", "maximum_block_size", "minimum_frame_size", "maximum_frame_size", "sample_rate", "channels", "bits_per_sample", "total_samples", "md5sum"]])) def raw_info(self): """returns a human-readable version of this metadata block as unicode""" from audiotools import hex_string from os import linesep return linesep.join( [u" STREAMINFO:", u" minimum block size = {:d}".format(self.minimum_block_size), u" maximum block size = {:d}".format(self.maximum_block_size), u" minimum frame size = {:d}".format(self.minimum_frame_size), u" maximum frame size = {:d}".format(self.maximum_frame_size), u" sample rate = {:d}".format(self.sample_rate), u" channels = {:d}".format(self.channels), u" bits-per-sample = {:d}".format(self.bits_per_sample), u" total samples = {:d}".format(self.total_samples), u" MD5 sum = {}".format(hex_string(self.md5sum))]) @classmethod def parse(cls, reader): """returns this metadata block from a BitstreamReader""" values = reader.parse("16u16u24u24u20u3u5u36U16b") values[5] += 1 # channels values[6] += 1 # bits-per-sample return cls(*values) def build(self, writer): """writes this metadata block to a BitstreamWriter""" writer.build("16u16u24u24u20u3u5u36U16b", (self.minimum_block_size, self.maximum_block_size, self.minimum_frame_size, self.maximum_frame_size, self.sample_rate, self.channels - 1, self.bits_per_sample - 1, self.total_samples, self.md5sum)) def size(self): """the size of this metadata block not including the 4-byte block header""" return 34 class Flac_PADDING(object): BLOCK_ID = 1 def __init__(self, length): self.length = length def copy(self): """returns a duplicate of this metadata block""" return Flac_PADDING(self.length) def __eq__(self, block): if hasattr(block, "length"): return self.length == block.length else: return False def __repr__(self): return "Flac_PADDING({!r})".format(self.length) def raw_info(self): """returns a human-readable version of this metadata block as unicode""" from os import linesep return linesep.join( [u" PADDING:", u" length = {:d}".format(self.length)]) @classmethod def parse(cls, reader, block_length): """returns this metadata block from a BitstreamReader""" reader.skip_bytes(block_length) return cls(length=block_length) def build(self, writer): """writes this metadata block to a BitstreamWriter""" writer.write_bytes(b"\x00" * self.length) def size(self): """the size of this metadata block not including the 4-byte block header""" return self.length class Flac_APPLICATION(object): BLOCK_ID = 2 def __init__(self, application_id, data): self.application_id = application_id self.data = data def __eq__(self, block): for attr in ["application_id", "data"]: if ((not hasattr(block, attr)) or (getattr(self, attr) != getattr(block, attr))): return False else: return True def copy(self): """returns a duplicate of this metadata block""" return Flac_APPLICATION(self.application_id, self.data) def __repr__(self): return "Flac_APPLICATION({!r}, {!r})".format( self.application_id, self.data) def raw_info(self): """returns a human-readable version of this metadata block as unicode""" from os import linesep return u" APPLICATION:{} {} ({:d} bytes)".format( linesep, self.application_id.decode('ascii'), len(self.data)) @classmethod def parse(cls, reader, block_length): """returns this metadata block from a BitstreamReader""" return cls(application_id=reader.read_bytes(4), data=reader.read_bytes(block_length - 4)) def build(self, writer): """writes this metadata block to a BitstreamWriter""" writer.write_bytes(self.application_id) writer.write_bytes(self.data) def size(self): """the size of this metadata block not including the 4-byte block header""" return len(self.application_id) + len(self.data) class Flac_SEEKTABLE(object): BLOCK_ID = 3 def __init__(self, seekpoints): """seekpoints is a list of (PCM frame offset, byte offset, PCM frame count) tuples""" self.seekpoints = seekpoints def __eq__(self, block): if hasattr(block, "seekpoints"): return self.seekpoints == block.seekpoints else: return False def copy(self): """returns a duplicate of this metadata block""" return Flac_SEEKTABLE(self.seekpoints[:]) def __repr__(self): return "Flac_SEEKTABLE({!r})".format(self.seekpoints) def raw_info(self): """returns a human-readable version of this metadata block as unicode""" from os import linesep return linesep.join( [u" SEEKTABLE:", u" first sample file offset frame samples"] + [u" {:14d} {:13X} {:15d}".format(seekpoint[0], seekpoint[1], seekpoint[2]) for seekpoint in self.seekpoints]) @classmethod def parse(cls, reader, total_seekpoints): """returns this metadata block from a BitstreamReader""" return cls([tuple(reader.parse("64U64U16u")) for i in range(total_seekpoints)]) def build(self, writer): """writes this metadata block to a BitstreamWriter""" for seekpoint in self.seekpoints: writer.build("64U64U16u", seekpoint) def size(self): """the size of this metadata block not including the 4-byte block header""" from audiotools.bitstream import format_size return (format_size("64U64U16u") // 8) * len(self.seekpoints) def clean(self): """removes any empty seek points and ensures PCM frame offset and byte offset are both incrementing""" fixes_performed = [] nonempty_points = [seekpoint for seekpoint in self.seekpoints if (seekpoint[2] != 0)] if len(nonempty_points) != len(self.seekpoints): from audiotools.text import CLEAN_FLAC_REMOVE_SEEKPOINTS fixes_performed.append(CLEAN_FLAC_REMOVE_SEEKPOINTS) ascending_order = list(set(nonempty_points)) ascending_order.sort() if ascending_order != nonempty_points: from audiotools.text import CLEAN_FLAC_REORDER_SEEKPOINTS fixes_performed.append(CLEAN_FLAC_REORDER_SEEKPOINTS) return (Flac_SEEKTABLE(ascending_order), fixes_performed) class Flac_VORBISCOMMENT(VorbisComment): BLOCK_ID = 4 def copy(self): """returns a duplicate of this metadata block""" return Flac_VORBISCOMMENT(self.comment_strings[:], self.vendor_string) def __repr__(self): return "Flac_VORBISCOMMENT({!r}, {!r})".format( self.comment_strings, self.vendor_string) def raw_info(self): """returns a human-readable version of this metadata block as unicode""" from os import linesep from audiotools import output_table # align the text strings on the "=" sign, if any table = output_table() for comment in self.comment_strings: row = table.row() row.add_column(u" " * 4) if u"=" in comment: (tag, value) = comment.split(u"=", 1) row.add_column(tag, "right") row.add_column(u"=") row.add_column(value) else: row.add_column(comment) row.add_column(u"") row.add_column(u"") return (u" VORBIS_COMMENT:" + linesep + u" {}".format(self.vendor_string) + linesep + linesep.join(table.format())) @classmethod def converted(cls, metadata): """converts a MetaData object to a Flac_VORBISCOMMENT object""" if (metadata is None) or (isinstance(metadata, Flac_VORBISCOMMENT)): return metadata else: # make VorbisComment do all the work, # then lift its data into a new Flac_VORBISCOMMENT metadata = VorbisComment.converted(metadata) return cls(metadata.comment_strings, metadata.vendor_string) @classmethod def parse(cls, reader): """returns this metadata block from a BitstreamReader""" reader.set_endianness(True) try: vendor_string = \ reader.read_bytes(reader.read(32)).decode('utf-8', 'replace') return cls([reader.read_bytes(reader.read(32)).decode('utf-8', 'replace') for i in range(reader.read(32))], vendor_string) finally: reader.set_endianness(False) def build(self, writer): """writes this metadata block to a BitstreamWriter""" writer.set_endianness(True) try: vendor_string = self.vendor_string.encode('utf-8') writer.write(32, len(vendor_string)) writer.write_bytes(vendor_string) writer.write(32, len(self.comment_strings)) for comment_string in self.comment_strings: comment_string = comment_string.encode('utf-8') writer.write(32, len(comment_string)) writer.write_bytes(comment_string) finally: writer.set_endianness(False) def size(self): """the size of this metadata block not including the 4-byte block header""" return (4 + len(self.vendor_string.encode('utf-8')) + 4 + sum(4 + len(comment.encode('utf-8')) for comment in self.comment_strings)) class Flac_CUESHEET(Sheet): BLOCK_ID = 5 def __init__(self, catalog_number, lead_in_samples, is_cdda, tracks): """catalog_number is a 128 byte ASCII string, padded with NULLs lead_in_samples is typically 2 seconds of samples is_cdda is 1 if audio if from CDDA, 0 otherwise tracks is a list of Flac_CHESHEET_track objects""" assert(isinstance(catalog_number, bytes)) assert(isinstance(lead_in_samples, int) or isinstance(lead_in_samples, long)) assert(is_cdda in {1, 0}) self.__catalog_number__ = catalog_number self.__lead_in_samples__ = lead_in_samples self.__is_cdda__ = is_cdda self.__tracks__ = tracks def copy(self): """returns a duplicate of this metadata block""" return Flac_CUESHEET(self.__catalog_number__, self.__lead_in_samples__, self.__is_cdda__, [track.copy() for track in self.__tracks__]) def __eq__(self, cuesheet): if isinstance(cuesheet, Flac_CUESHEET): return ((self.__catalog_number__ == cuesheet.__catalog_number__) and (self.__lead_in_samples__ == cuesheet.__lead_in_samples__) and (self.__is_cdda__ == cuesheet.__is_cdda__) and (self.__tracks__ == cuesheet.__tracks__)) else: return Sheet.__eq__(self, cuesheet) def __repr__(self): return "Flac_CUESHEET({})".format(",".join( ["{}={!r}".format(key, getattr(self, "__" + key + "__")) for key in ["catalog_number", "lead_in_samples", "is_cdda", "tracks"]])) def raw_info(self): """returns a human-readable version of this metadata block as unicode""" from os import linesep return linesep.join( [u" CUESHEET:", u" catalog number = {}".format( self.__catalog_number__.decode('ascii', 'replace')), u" lead-in samples = {:d}".format(self.__lead_in_samples__), u" is CDDA = {:d}".format(self.__is_cdda__)] + [track.raw_info(4) for track in self.__tracks__]) @classmethod def parse(cls, reader): """returns this metadata block from a BitstreamReader""" (catalog_number, lead_in_samples, is_cdda, track_count) = reader.parse("128b64U1u2071p8u") return cls(catalog_number, lead_in_samples, is_cdda, [Flac_CUESHEET_track.parse(reader) for i in range(track_count)]) def build(self, writer): """writes this metadata block to a BitstreamWriter""" writer.build("128b64U1u2071p8u", (self.__catalog_number__, self.__lead_in_samples__, self.__is_cdda__, len(self.__tracks__))) for track in self.__tracks__: track.build(writer) def size(self): """the size of this metadata block not including the 4-byte block header""" return (396 + # format_size("128b64U1u2071p8u") // 8 sum(t.size() for t in self.__tracks__)) def __len__(self): # don't include lead-out track return len(self.__tracks__) - 1 def __getitem__(self, index): # don't include lead-out track return self.__tracks__[0:-1][index] def track_length(self, track_number, total_length=None): """given a track_number (typically starting from 1) and optional total length as a Fraction number of seconds (including the disc's pre-gap, if any), returns the length of the track as a Fraction number of seconds or None if the length is to the remainder of the stream (typically for the last track in the album) may raise KeyError if the track is not found""" initial_track = self.track(track_number) if (track_number + 1) in self.track_numbers(): next_track = self.track(track_number + 1) return (next_track.index(1).offset() - initial_track.index(1).offset()) else: # getting track length of final track from fractions import Fraction lead_out_track = self.__tracks__[-1] final_index = initial_track.index(1) return (Fraction(lead_out_track.__offset__, final_index.__sample_rate__) - final_index.offset()) def get_metadata(self): """returns MetaData of Sheet, or None this metadata often contains information such as catalog number or CD-TEXT values""" catalog = self.__catalog_number__.rstrip(b"\x00") if len(catalog) > 0: from audiotools import MetaData return MetaData(catalog=catalog.decode("ascii", "replace")) else: return None def set_track(self, audiofile): """sets the AudioFile this cuesheet belongs to this is necessary becuase FLAC's CUESHEET block doesn't store the file's sample rate which is needed to convert sample offsets to seconds""" for track in self: track.set_track(audiofile) @classmethod def converted(cls, sheet, total_pcm_frames, sample_rate, is_cdda=True): """given a Sheet object, total PCM frames, sample rate and optional boolean indicating whether cuesheet is CD audio returns a Flac_CUESHEET object from that data""" def pad(u, chars): if u is not None: s = u.encode("ascii", "replace") return s[0:chars] + (b"\x00" * (chars - len(s))) else: return b"\x00" * chars metadata = sheet.get_metadata() if (metadata is not None) and (metadata.catalog is not None): catalog_number = pad(metadata.catalog, 128) else: catalog_number = b"\x00" * 128 # assume standard 2 second disc lead-in # and append empty lead-out track return cls(catalog_number=catalog_number, lead_in_samples=sample_rate * 2, is_cdda=(1 if is_cdda else 0), tracks=([Flac_CUESHEET_track.converted(t, sample_rate) for t in sheet] + [Flac_CUESHEET_track(offset=total_pcm_frames, number=170 if is_cdda else 255, ISRC=b"\x00" * 12, track_type=0, pre_emphasis=0, index_points=[])])) class Flac_CUESHEET_track(SheetTrack): def __init__(self, offset, number, ISRC, track_type, pre_emphasis, index_points): """offset is the track's first index point's offset from the start of the stream, in PCM frames number is the track number, typically starting from 1 ISRC is a 12 byte ASCII string, padded with NULLs track_type is 0 for audio, 1 for non-audio pre_emphasis is 0 for no, 1 for yes index_points is a list of Flac_CUESHEET_index objects""" assert(isinstance(offset, int) or isinstance(offset, long)) assert(isinstance(number, int)) assert(isinstance(ISRC, bytes)) assert(track_type in {0, 1}) assert(pre_emphasis in {0, 1}) self.__offset__ = offset self.__number__ = number self.__ISRC__ = ISRC self.__track_type__ = track_type self.__pre_emphasis__ = pre_emphasis self.__index_points__ = index_points # the file this track belongs to self.__filename__ = "" @classmethod def converted(cls, sheet_track, sample_rate): """given a SheetTrack object and stream's sample rate, returns a Flac_CUESHEET_track object""" def pad(u, chars): if u is not None: s = u.encode("ascii", "replace") return s[0:chars] + (b"\x00" * (chars - len(s))) else: return b"\x00" * chars if len(sheet_track) > 0: offset = int(sheet_track[0].offset() * sample_rate) else: # track with no index points offset = 0 metadata = sheet_track.get_metadata() if metadata is not None: ISRC = pad(metadata.ISRC, 12) else: ISRC = b"\x00" * 12 return cls(offset=offset, number=sheet_track.number(), ISRC=ISRC, track_type=(0 if sheet_track.is_audio() else 1), pre_emphasis=(1 if sheet_track.pre_emphasis() else 0), index_points=[Flac_CUESHEET_index.converted( index, offset, sample_rate) for index in sheet_track]) def copy(self): """returns a duplicate of this metadata block""" return Flac_CUESHEET_track(self.__offset__, self.__number__, self.__ISRC__, self.__track_type__, self.__pre_emphasis__, [index.copy() for index in self.__index_points__]) def __repr__(self): return "Flac_CUESHEET_track({})".format(",".join( ["{}={!r}".format(key, getattr(self, "__" + key + "__")) for key in ["offset", "number", "ISRC", "track_type", "pre_emphasis", "index_points"]])) def raw_info(self, indent): """returns a human-readable version of this track as unicode""" from os import linesep lines = [(u"track : {number:3d} " + u"offset : {offset:9d} " + u"ISRC : {ISRC}").format( number=self.__number__, offset=self.__offset__, type=self.__track_type__, pre_emphasis=self.__pre_emphasis__, ISRC=self.__ISRC__.strip(b"\x00").decode('ascii', 'replace')) ] + [i.raw_info(1) for i in self.__index_points__] return linesep.join( [u" " * indent + line for line in lines]) def __eq__(self, track): if isinstance(track, Flac_CUESHEET_track): return ((self.__offset__ == track.__offset__) and (self.__number__ == track.__number__) and (self.__ISRC__ == track.__ISRC__) and (self.__track_type__ == track.__track_type__) and (self.__pre_emphasis__ == track.__pre_emphasis__) and (self.__index_points__ == track.__index_points__)) else: return SheetTrack.__eq__(self, track) @classmethod def parse(cls, reader): """returns this cuesheet track from a BitstreamReader""" (offset, number, ISRC, track_type, pre_emphasis, index_points) = reader.parse("64U8u12b1u1u110p8u") return cls(offset, number, ISRC, track_type, pre_emphasis, [Flac_CUESHEET_index.parse(reader, offset) for i in range(index_points)]) def build(self, writer): """writes this cuesheet track to a BitstreamWriter""" writer.build("64U8u12b1u1u110p8u", (self.__offset__, self.__number__, self.__ISRC__, self.__track_type__, self.__pre_emphasis__, len(self.__index_points__))) for index_point in self.__index_points__: index_point.build(writer) def size(self): return (36 + # format_size("64U8u12b1u1u110p8u") // 8 sum(i.size() for i in self.__index_points__)) def __len__(self): return len(self.__index_points__) def __getitem__(self, index): return self.__index_points__[index] def number(self): """return SheetTrack's number, starting from 1""" return self.__number__ def get_metadata(self): """returns SheetTrack's MetaData, or None""" isrc = self.__ISRC__.rstrip(b"\x00") if len(isrc) > 0: from audiotools import MetaData return MetaData(ISRC=isrc.decode("ascii", "replace")) else: return None def filename(self): """returns SheetTrack's filename as a unicode string""" from sys import version_info if version_info[0] >= 3: return self.__filename__ else: return self.__filename__.decode("UTF-8") def is_audio(self): """returns whether SheetTrack contains audio data""" return True def pre_emphasis(self): """returns whether SheetTrack has pre-emphasis""" return self.__pre_emphasis__ == 1 def copy_permitted(self): """returns whether copying is permitted""" return False def set_track(self, audiofile): """sets this track's source as the given AudioFile""" from os.path import basename self.__filename__ = basename(audiofile.filename) for index in self: index.set_track(audiofile) class Flac_CUESHEET_index(SheetIndex): def __init__(self, track_offset, offset, number, sample_rate=44100): """track_offset is the index's track's offset in PCM frames offset is the index's offset from the track offset, in PCM frames number is the index's number typically starting from 1 (a number of 0 indicates a track pre-gap)""" self.__track_offset__ = track_offset self.__offset__ = offset self.__number__ = number self.__sample_rate__ = sample_rate @classmethod def converted(cls, sheet_index, track_offset, sample_rate): """given a SheetIndex object, track_offset (in PCM frames) and sample rate, returns a Flac_CUESHEET_index object""" return cls(track_offset=track_offset, offset=((int(sheet_index.offset() * sample_rate)) - track_offset), number=sheet_index.number(), sample_rate=sample_rate) def copy(self): """returns a duplicate of this metadata block""" return Flac_CUESHEET_index(self.__track_offset__, self.__offset__, self.__number__, self.__sample_rate__) def __repr__(self): return "Flac_CUESHEET_index({!r}, {!r}, {!r}, {!r})".format( self.__track_offset__, self.__offset__, self.__number__, self.__sample_rate__) def __eq__(self, index): if isinstance(index, Flac_CUESHEET_index): return ((self.__offset__ == index.__offset__) and (self.__number__ == index.__number__)) else: return SheetIndex.__eq__(self, index) @classmethod def parse(cls, reader, track_offset): """returns this cuesheet index from a BitstreamReader""" (offset, number) = reader.parse("64U8u24p") return cls(track_offset=track_offset, offset=offset, number=number) def build(self, writer): """writes this cuesheet index to a BitstreamWriter""" writer.build("64U8u24p", (self.__offset__, self.__number__)) def size(self): return 12 # format_size("64U8u24p") // 8 def raw_info(self, indent): return ((u" " * indent) + u"index : {:3d} offset : {:>9d}".format( self.__number__, self.__offset__)) def number(self): return self.__number__ def offset(self): from fractions import Fraction return Fraction(self.__track_offset__ + self.__offset__, self.__sample_rate__) def set_track(self, audiofile): """sets this index's source to the given AudioFile""" self.__sample_rate__ = audiofile.sample_rate() class Flac_PICTURE(Image): BLOCK_ID = 6 def __init__(self, picture_type, mime_type, description, width, height, color_depth, color_count, data): """ picture_type - int of FLAC picture ID mime_type - unicode string of MIME type description - unicode string of description width - int width value height - int height value color_depth - int bits-per-pixel value color_count - int color count value data - binary string of image data """ from audiotools import PY3 assert(isinstance(picture_type, int)) assert(isinstance(mime_type, str if PY3 else unicode)) assert(isinstance(description, str if PY3 else unicode)) assert(isinstance(width, int)) assert(isinstance(height, int)) assert(isinstance(color_depth, int)) assert(isinstance(color_count, int)) assert(isinstance(data, bytes)) # bypass Image's constructor and set block fields directly Image.__setattr__(self, "data", data) Image.__setattr__(self, "mime_type", mime_type) Image.__setattr__(self, "width", width) Image.__setattr__(self, "height", height) Image.__setattr__(self, "color_depth", color_depth) Image.__setattr__(self, "color_count", color_count) Image.__setattr__(self, "description", description) Image.__setattr__(self, "picture_type", picture_type) def copy(self): """returns a duplicate of this metadata block""" return Flac_PICTURE(self.picture_type, self.mime_type, self.description, self.width, self.height, self.color_depth, self.color_count, self.data) def __getattr__(self, attr): if attr == "type": # convert FLAC picture_type to Image type # # | Item | FLAC Picture ID | Image type | # |--------------+-----------------+------------| # | Other | 0 | 4 | # | Front Cover | 3 | 0 | # | Back Cover | 4 | 1 | # | Leaflet Page | 5 | 2 | # | Media | 6 | 3 | from audiotools import (FRONT_COVER, BACK_COVER, LEAFLET_PAGE, MEDIA, OTHER) return {0: OTHER, 3: FRONT_COVER, 4: BACK_COVER, 5: LEAFLET_PAGE, 6: MEDIA}.get(self.picture_type, OTHER) else: return Image.__getattribute__(self, attr) def __setattr__(self, attr, value): if attr == "type": # convert Image type to FLAC picture_type # # | Item | Image type | FLAC Picture ID | # |--------------+------------+-----------------| # | Other | 4 | 0 | # | Front Cover | 0 | 3 | # | Back Cover | 1 | 4 | # | Leaflet Page | 2 | 5 | # | Media | 3 | 6 | from audiotools import (FRONT_COVER, BACK_COVER, LEAFLET_PAGE, MEDIA, OTHER) self.picture_type = {OTHER: 0, FRONT_COVER: 3, BACK_COVER: 4, LEAFLET_PAGE: 5, MEDIA: 6}.get(value, 0) else: Image.__setattr__(self, attr, value) def __repr__(self): return "Flac_PICTURE({})".format(",".join( ["{}={!r}".format(attr, getattr(self, attr)) for attr in ["picture_type", "mime_type", "description", "width", "height", "color_depth", "color_count"]])) def raw_info(self): """returns a human-readable version of this metadata block as unicode""" from os import linesep return linesep.join( [u" PICTURE:", u" picture type = {:d}".format(self.picture_type), u" MIME type = {}".format(self.mime_type), u" description = {}".format(self.description), u" width = {:d}".format(self.width), u" height = {:d}".format(self.height), u" color depth = {:d}".format(self.color_depth), u" color count = {:d}".format(self.color_count), u" bytes = {:d}".format(len(self.data))]) @classmethod def parse(cls, reader): """returns this metadata block from a BitstreamReader""" picture_type = reader.read(32) mime_type = reader.read_bytes(reader.read(32)).decode('ascii') description = reader.read_bytes(reader.read(32)).decode('utf-8') width = reader.read(32) height = reader.read(32) color_depth = reader.read(32) color_count = reader.read(32) data = reader.read_bytes(reader.read(32)) return cls(picture_type=picture_type, mime_type=mime_type, description=description, width=width, height=height, color_depth=color_depth, color_count=color_count, data=data) def build(self, writer): """writes this metadata block to a BitstreamWriter""" writer.write(32, self.picture_type) mime_type = self.mime_type.encode('ascii') writer.write(32, len(mime_type)) writer.write_bytes(mime_type) description = self.description.encode('utf-8') writer.write(32, len(description)) writer.write_bytes(description) writer.write(32, self.width) writer.write(32, self.height) writer.write(32, self.color_depth) writer.write(32, self.color_count) writer.write(32, len(self.data)) writer.write_bytes(self.data) def size(self): """the size of this metadata block not including the 4-byte block header""" return (4 + # picture_type 4 + len(self.mime_type.encode('ascii')) + 4 + len(self.description.encode('utf-8')) + 4 + # width 4 + # height 4 + # color_count 4 + # color_depth 4 + len(self.data)) @classmethod def converted(cls, image): """converts an Image object to a FlacPictureComment""" return cls( picture_type={4: 0, 0: 3, 1: 4, 2: 5, 3: 6}.get(image.type, 0), mime_type=image.mime_type, description=image.description, width=image.width, height=image.height, color_depth=image.color_depth, color_count=image.color_count, data=image.data) def type_string(self): """returns the image's type as a human readable plain string for example, an image of type 0 returns "Front Cover" """ return {0: u"Other", 1: u"File icon", 2: u"Other file icon", 3: u"Cover (front)", 4: u"Cover (back)", 5: u"Leaflet page", 6: u"Media", 7: u"Lead artist / lead performer / soloist", 8: u"Artist / Performer", 9: u"Conductor", 10: u"Band / Orchestra", 11: u"Composer", 12: u"Lyricist / Text writer", 13: u"Recording Location", 14: u"During recording", 15: u"During performance", 16: u"Movie / Video screen capture", 17: u"A bright colored fish", 18: u"Illustration", 19: u"Band/Artist logotype", 20: u"Publisher / Studio logotype"}.get(self.picture_type, u"Other") def clean(self): from audiotools.image import image_metrics img = image_metrics(self.data) if (((self.mime_type != img.mime_type) or (self.width != img.width) or (self.height != img.height) or (self.color_depth != img.bits_per_pixel) or (self.color_count != img.color_count))): from audiotools.text import CLEAN_FIX_IMAGE_FIELDS return (self.__class__.converted( Image(type=self.type, mime_type=img.mime_type, description=self.description, width=img.width, height=img.height, color_depth=img.bits_per_pixel, color_count=img.color_count, data=self.data)), [CLEAN_FIX_IMAGE_FIELDS]) else: return (self, []) class FlacAudio(WaveContainer, AiffContainer): """a Free Lossless Audio Codec file""" from audiotools.text import (COMP_FLAC_0, COMP_FLAC_8) SUFFIX = "flac" NAME = SUFFIX DESCRIPTION = u"Free Lossless Audio Codec" DEFAULT_COMPRESSION = "8" COMPRESSION_MODES = tuple(map(str, range(0, 9))) COMPRESSION_DESCRIPTIONS = {"0": COMP_FLAC_0, "8": COMP_FLAC_8} METADATA_CLASS = FlacMetaData def __init__(self, filename): """filename is a plain string""" from audiotools.id3 import skip_id3v2_comment from audiotools.bitstream import BitstreamReader AudioFile.__init__(self, filename) # setup some dummy placeholder values self.__stream_offset__ = 0 self.__samplerate__ = 0 self.__channels__ = 0 self.__bitspersample__ = 0 self.__total_frames__ = 0 self.__md5__ = b"\x00" * 16 try: with open(self.filename, "rb") as f: # check for leading ID3v3 tag self.__stream_offset__ = skip_id3v2_comment(f) # ensure stream marker is correct if f.read(4) != b"fLaC": from audiotools.text import ERR_FLAC_INVALID_FILE raise InvalidFLAC(ERR_FLAC_INVALID_FILE) reader = BitstreamReader(f, False) # walk metadata blocks looking for STREAMINFO # (should be first block) stop = 0 while stop == 0: stop, header_type, length = reader.parse("1u 7u 24u") if header_type == 0: reader.skip(80) self.__samplerate__ = reader.read(20) self.__channels__ = reader.read(3) + 1 self.__bitspersample__ = reader.read(5) + 1 self.__total_frames__ = reader.read(36) self.__md5__ = reader.read_bytes(16) return elif header_type in {1, 2, 3, 4, 5, 6}: # be accepting of out-of-spec files # whose STREAMINFO blocks aren't first reader.skip_bytes(length) else: from audiotools.text import ERR_FLAC_INVALID_BLOCK raise InvalidFLAC(ERR_FLAC_INVALID_BLOCK) except IOError as msg: raise InvalidFLAC(str(msg)) def channel_mask(self): """returns a ChannelMask object of this track's channel layout""" from audiotools import ChannelMask if self.channels() <= 2: return ChannelMask.from_channels(self.channels()) try: metadata = self.get_metadata() if metadata is not None: channel_mask = ChannelMask( int(metadata.get_block( Flac_VORBISCOMMENT.BLOCK_ID)[ u"WAVEFORMATEXTENSIBLE_CHANNEL_MASK"][0], 16)) if len(channel_mask) == self.channels(): return channel_mask else: # channel count mismatch in given mask return ChannelMask(0) else: # proceed to generate channel mask raise ValueError() except (IndexError, KeyError, ValueError): # if there is no VORBIS_COMMENT block # or no WAVEFORMATEXTENSIBLE_CHANNEL_MASK in that block # or it's not an integer, # use FLAC's default mask based on channels if self.channels() == 3: return ChannelMask.from_fields( front_left=True, front_right=True, front_center=True) elif self.channels() == 4: return ChannelMask.from_fields( front_left=True, front_right=True, back_left=True, back_right=True) elif self.channels() == 5: return ChannelMask.from_fields( front_left=True, front_right=True, front_center=True, back_left=True, back_right=True) elif self.channels() == 6: return ChannelMask.from_fields( front_left=True, front_right=True, front_center=True, back_left=True, back_right=True, low_frequency=True) elif self.channels() == 7: return ChannelMask.from_fields( front_left=True, front_right=True, front_center=True, low_frequency=True, back_center=True, side_left=True, side_right=True) elif self.channels() == 8: return ChannelMask.from_fields( front_left=True, front_right=True, front_center=True, low_frequency=True, back_left=True, back_right=True, side_left=True, side_right=True) else: # shouldn't be able to happen return ChannelMask(0) def lossless(self): """returns True""" return True @classmethod def supports_metadata(cls): """returns True if this audio type supports MetaData""" return True def get_metadata(self): """returns a MetaData object, or None raises IOError if unable to read the file""" from audiotools.bitstream import BitstreamReader # FlacAudio *always* returns a FlacMetaData object # even if the blocks aren't present # so there's no need to test for None with BitstreamReader(open(self.filename, 'rb'), False) as reader: reader.seek(self.__stream_offset__, 0) if reader.read_bytes(4) == b"fLaC": return FlacMetaData.parse(reader) else: # shouldn't be able to get here return None def update_metadata(self, metadata): """takes this track's current MetaData object as returned by get_metadata() and sets this track's metadata with any fields updated in that object raises IOError if unable to write the file """ from audiotools.bitstream import BitstreamWriter from audiotools.bitstream import BitstreamReader from operator import add if metadata is None: return if not isinstance(metadata, FlacMetaData): from audiotools.text import ERR_FOREIGN_METADATA raise ValueError(ERR_FOREIGN_METADATA) old_metadata = self.get_metadata() padding_blocks = metadata.get_blocks(Flac_PADDING.BLOCK_ID) has_padding = len(padding_blocks) > 0 padding_unchanged = (old_metadata.get_blocks(Flac_PADDING.BLOCK_ID) == padding_blocks) total_padding_size = sum(b.size() for b in padding_blocks) metadata_delta = metadata.size() - old_metadata.size() if (has_padding and padding_unchanged and (metadata_delta <= total_padding_size) and ((-metadata_delta + total_padding_size) <= MAX_PADDING_SIZE)): # if padding size is larger than change in metadata # shrink padding blocks so that new size matches old size # (if metadata_delta is negative, # this will enlarge padding blocks as necessary) for padding in padding_blocks: if metadata_delta > 0: # extract bytes from PADDING blocks # until the metadata_delta is exhausted if metadata_delta <= padding.length: padding.length -= metadata_delta metadata_delta = 0 else: metadata_delta -= padding.length padding.length = 0 elif metadata_delta < 0: # dump all our new bytes into the first PADDING block found padding.length += -metadata_delta metadata_delta = 0 else: break # then overwrite the beginning of the file stream = open(self.filename, 'r+b') stream.seek(self.__stream_offset__, 0) writer = BitstreamWriter(stream, 0) writer.write_bytes(b'fLaC') metadata.build(writer) writer.flush() writer.close() else: # if padding is smaller than change in metadata, # the padding would get excessively large, # or file has no padding blocks, # rewrite entire file to fit new metadata from audiotools import TemporaryFile, transfer_data from audiotools.bitstream import parse # dump any prefix data from old file to new one old_file = open(self.filename, "rb") new_file = TemporaryFile(self.filename) new_file.write(old_file.read(self.__stream_offset__)) # skip existing file ID and metadata blocks if old_file.read(4) != b'fLaC': from audiotools.text import ERR_FLAC_INVALID_FILE raise InvalidFLAC(ERR_FLAC_INVALID_FILE) stop = 0 while stop == 0: (stop, length) = parse("1u 7p 24u", False, old_file.read(4)) old_file.read(length) # write new metadata to new file writer = BitstreamWriter(new_file, False) writer.write_bytes(b"fLaC") metadata.build(writer) # write remaining old data to new file transfer_data(old_file.read, writer.write_bytes) # commit change to disk old_file.close() writer.close() def set_metadata(self, metadata): """takes a MetaData object and sets this track's metadata this metadata includes track name, album name, and so on raises IOError if unable to read or write the file""" if metadata is None: return self.delete_metadata() new_metadata = self.METADATA_CLASS.converted(metadata) old_metadata = self.get_metadata() if old_metadata is None: # this shouldn't happen old_metadata = FlacMetaData([]) # replace old metadata's VORBIS_COMMENT with one from new metadata # (if any) if new_metadata.has_block(Flac_VORBISCOMMENT.BLOCK_ID): new_vorbiscomment = new_metadata.get_block( Flac_VORBISCOMMENT.BLOCK_ID) if old_metadata.has_block(Flac_VORBISCOMMENT.BLOCK_ID): # both new and old metadata have a VORBIS_COMMENT block old_vorbiscomment = old_metadata.get_block( Flac_VORBISCOMMENT.BLOCK_ID) # update vendor string from our current VORBIS_COMMENT block new_vorbiscomment.vendor_string = \ old_vorbiscomment.vendor_string # update REPLAYGAIN_* tags from # our current VORBIS_COMMENT block for key in [u"REPLAYGAIN_TRACK_GAIN", u"REPLAYGAIN_TRACK_PEAK", u"REPLAYGAIN_ALBUM_GAIN", u"REPLAYGAIN_ALBUM_PEAK", u"REPLAYGAIN_REFERENCE_LOUDNESS"]: try: new_vorbiscomment[key] = old_vorbiscomment[key] except KeyError: new_vorbiscomment[key] = [] # update WAVEFORMATEXTENSIBLE_CHANNEL_MASK # from our current VORBIS_COMMENT block, if any if (((self.channels() > 2) or (self.bits_per_sample() > 16)) and (u"WAVEFORMATEXTENSIBLE_CHANNEL_MASK" in old_vorbiscomment.keys())): new_vorbiscomment[u"WAVEFORMATEXTENSIBLE_CHANNEL_MASK"] = \ old_vorbiscomment[u"WAVEFORMATEXTENSIBLE_CHANNEL_MASK"] elif (u"WAVEFORMATEXTENSIBLE_CHANNEL_MASK" in new_vorbiscomment.keys()): new_vorbiscomment[ u"WAVEFORMATEXTENSIBLE_CHANNEL_MASK"] = [] # update CDTOC from our current VORBIS_COMMENT block, if any try: new_vorbiscomment[u"CDTOC"] = old_vorbiscomment[u"CDTOC"] except KeyError: new_vorbiscomment[u"CDTOC"] = [] old_metadata.replace_blocks(Flac_VORBISCOMMENT.BLOCK_ID, [new_vorbiscomment]) else: # new metadata has VORBIS_COMMENT block, # but old metadata does not # remove REPLAYGAIN_* tags from new VORBIS_COMMENT block for key in [u"REPLAYGAIN_TRACK_GAIN", u"REPLAYGAIN_TRACK_PEAK", u"REPLAYGAIN_ALBUM_GAIN", u"REPLAYGAIN_ALBUM_PEAK", u"REPLAYGAIN_REFERENCE_LOUDNESS"]: new_vorbiscomment[key] = [] # update WAVEFORMATEXTENSIBLE_CHANNEL_MASK # from our actual mask if necessary if (self.channels() > 2) or (self.bits_per_sample() > 16): new_vorbiscomment[u"WAVEFORMATEXTENSIBLE_CHANNEL_MASK"] = [ u"0x{:04X}".format(self.channel_mask())] # remove CDTOC from new VORBIS_COMMENT block new_vorbiscomment[u"CDTOC"] = [] old_metadata.add_block(new_vorbiscomment) else: # new metadata has no VORBIS_COMMENT block pass # replace old metadata's PICTURE blocks with those from new metadata old_metadata.replace_blocks( Flac_PICTURE.BLOCK_ID, new_metadata.get_blocks(Flac_PICTURE.BLOCK_ID)) # everything else remains as-is self.update_metadata(old_metadata) def delete_metadata(self): """deletes the track's MetaData this removes or unsets tags as necessary in order to remove all data raises IOError if unable to write the file""" self.set_metadata(MetaData()) @classmethod def supports_cuesheet(cls): return True def set_cuesheet(self, cuesheet): """imports cuesheet data from a Sheet object Raises IOError if an error occurs setting the cuesheet""" if cuesheet is not None: # overwrite old cuesheet (if any) with new block metadata = self.get_metadata() metadata.replace_blocks( Flac_CUESHEET.BLOCK_ID, [Flac_CUESHEET.converted( cuesheet, self.total_frames(), self.sample_rate(), (self.sample_rate() == 44100) and (self.channels() == 2) and (self.bits_per_sample() == 16) and (len(cuesheet) <= 99))]) # wipe out any CDTOC tag try: vorbiscomment = metadata.get_block(Flac_VORBISCOMMENT.BLOCK_ID) if u"CDTOC" in vorbiscomment: del(vorbiscomment[u"CDTOC"]) except IndexError: pass self.update_metadata(metadata) else: self.delete_cuesheet() def get_cuesheet(self): """returns the embedded Sheet object, or None Raises IOError if a problem occurs when reading the file""" metadata = self.get_metadata() # first, check for a CUESHEET block try: cuesheet = metadata.get_block(Flac_CUESHEET.BLOCK_ID) cuesheet.set_track(self) return cuesheet except IndexError: pass # then, check for a CUESHEET tag or CDTOC tag try: vorbiscomment = metadata.get_block(Flac_VORBISCOMMENT.BLOCK_ID) if u"CUESHEET" in vorbiscomment: from audiotools import SheetException from audiotools.cue import read_cuesheet_string try: return read_cuesheet_string(vorbiscomment[u"CUESHEET"][0]) except SheetException: pass if u"CDTOC" in vorbiscomment: from audiotools import SheetException from audiotools.toc import read_tocfile_string try: return read_tocfile_string(vorbiscomment[u"CDTOC"][0]) except SheetException: from audiotools.cdtoc import CDTOC try: return CDTOC.from_unicode(vorbiscomment[u"CDTOC"][0]) except ValueError: pass except IndexError: pass return None def delete_cuesheet(self): """deletes embedded Sheet object, if any Raises IOError if a problem occurs when updating the file""" metadata = self.get_metadata() # wipe out any CUESHEET blocks metadata.replace_blocks(Flac_CUESHEET.BLOCK_ID, []) # then erase any CDTOC tags try: vorbiscomment = metadata.get_block(Flac_VORBISCOMMENT.BLOCK_ID) del(vorbiscomment[u"CDTOC"]) except IndexError: pass self.update_metadata(metadata) def to_pcm(self): """returns a PCMReader object containing the track's PCM data""" from audiotools.decoders import FlacDecoder from audiotools import PCMReaderError try: flac = open(self.filename, "rb") except (IOError, ValueError) as err: return PCMReaderError(error_message=str(err), sample_rate=self.sample_rate(), channels=self.channels(), channel_mask=int(self.channel_mask()), bits_per_sample=self.bits_per_sample()) try: if self.__stream_offset__ > 0: flac.seek(self.__stream_offset__) return FlacDecoder(flac) except (IOError, ValueError) as err: # The only time this is likely to occur is # if the FLAC is modified between when FlacAudio # is initialized and when to_pcm() is called. flac.close() return PCMReaderError(error_message=str(err), sample_rate=self.sample_rate(), channels=self.channels(), channel_mask=int(self.channel_mask()), bits_per_sample=self.bits_per_sample()) @classmethod def supports_to_pcm(cls): try: from audiotools.decoders import FlacDecoder return True except ImportError: return False @classmethod def from_pcm(cls, filename, pcmreader, compression=None, total_pcm_frames=None, encoding_function=None): """encodes a new file from PCM data takes a filename string, PCMReader object, optional compression level string and optional total_pcm_frames integer encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new FlacAudio object""" from audiotools.encoders import encode_flac from audiotools import EncodingError from audiotools import __default_quality__ from audiotools import VERSION if ((compression is None) or (compression not in cls.COMPRESSION_MODES)): compression = __default_quality__(cls.NAME) encoding_options = { "0": {"block_size": 1152, "max_lpc_order": 0, "min_residual_partition_order": 0, "max_residual_partition_order": 3}, "1": {"block_size": 1152, "max_lpc_order": 0, "adaptive_mid_side": True, "min_residual_partition_order": 0, "max_residual_partition_order": 3}, "2": {"block_size": 1152, "max_lpc_order": 0, "exhaustive_model_search": True, "min_residual_partition_order": 0, "max_residual_partition_order": 3}, "3": {"block_size": 4096, "max_lpc_order": 6, "min_residual_partition_order": 0, "max_residual_partition_order": 4}, "4": {"block_size": 4096, "max_lpc_order": 8, "adaptive_mid_side": True, "min_residual_partition_order": 0, "max_residual_partition_order": 4}, "5": {"block_size": 4096, "max_lpc_order": 8, "mid_side": True, "min_residual_partition_order": 0, "max_residual_partition_order": 5}, "6": {"block_size": 4096, "max_lpc_order": 8, "mid_side": True, "min_residual_partition_order": 0, "max_residual_partition_order": 6}, "7": {"block_size": 4096, "max_lpc_order": 8, "mid_side": True, "exhaustive_model_search": True, "min_residual_partition_order": 0, "max_residual_partition_order": 6}, "8": {"block_size": 4096, "max_lpc_order": 12, "mid_side": True, "exhaustive_model_search": True, "min_residual_partition_order": 0, "max_residual_partition_order": 6}}[compression] if pcmreader.bits_per_sample not in {8, 16, 24}: from audiotools import UnsupportedBitsPerSample pcmreader.close() raise UnsupportedBitsPerSample(filename, pcmreader.bits_per_sample) if pcmreader.channels > 8: from audiotools import UnsupportedChannelCount pcmreader.close() raise UnsupportedChannelCount(filename, pcmreader.channels) if (pcmreader.channel_mask not in {0x0001, # 1ch - mono 0x0004, # 1ch - mono 0x0003, # 2ch - left, right 0x0007, # 3ch - left, right, center 0x0033, # 4ch - left, right, back left, back right 0x0603, # 4ch - left, right, side left, side right 0x0037, # 5ch - L, R, C, back left, back right 0x0607, # 5ch - L, R, C, side left, side right 0x003F, # 6ch - L, R, C, LFE, back left, back right 0x060F, # 6ch - L, R, C, LFE, side left, side right 0}): from audiotools import UnsupportedChannelMask pcmreader.close() raise UnsupportedChannelMask(filename, pcmreader.channel_mask) try: (encode_flac if encoding_function is None else encoding_function)( filename=filename, pcmreader=pcmreader, version="Python Audio Tools " + VERSION, total_pcm_frames=(total_pcm_frames if total_pcm_frames is not None else 0), padding_size=4096, **encoding_options) return FlacAudio(filename) except (IOError, ValueError) as err: cls.__unlink__(filename) raise EncodingError(str(err)) except Exception: cls.__unlink__(filename) raise finally: pcmreader.close() @classmethod def supports_from_pcm(cls): try: from audiotools.encoders import encode_flac return True except ImportError: return False def seekable(self): """returns True if the file is seekable""" return self.get_metadata().has_block(Flac_SEEKTABLE.BLOCK_ID) def seektable(self, offsets=None, seekpoint_interval=None): """returns a new Flac_SEEKTABLE object created from parsing the FLAC file itself""" from bisect import bisect_right if offsets is None: sizes = [] with self.to_pcm() as pcmreader: pair = pcmreader.frame_size() while pair is not None: sizes.append(pair) pair = pcmreader.frame_size() offsets = sizes_to_offsets(sizes) if seekpoint_interval is None: seekpoint_interval = self.sample_rate() * 10 total_samples = 0 all_frames = {} sample_offsets = [] for (byte_offset, pcm_frames) in offsets: all_frames[total_samples] = (byte_offset, pcm_frames) sample_offsets.append(total_samples) total_samples += pcm_frames seekpoints = [] for pcm_frame in range(0, self.total_frames(), seekpoint_interval): flac_frame = bisect_right(sample_offsets, pcm_frame) - 1 seekpoints.append((sample_offsets[flac_frame], all_frames[sample_offsets[flac_frame]][0], all_frames[sample_offsets[flac_frame]][1])) return Flac_SEEKTABLE(seekpoints) def has_foreign_wave_chunks(self): """returns True if the audio file contains non-audio RIFF chunks during transcoding, if the source audio file has foreign RIFF chunks and the target audio format supports foreign RIFF chunks, conversion should be routed through .wav conversion to avoid losing those chunks""" try: return b'riff' in [ block.application_id for block in self.get_metadata().get_blocks(Flac_APPLICATION.BLOCK_ID)] except IOError: return False def wave_header_footer(self): """returns (header, footer) tuple of strings containing all data before and after the PCM stream may raise ValueError if there's a problem with the header or footer data may raise IOError if there's a problem reading header or footer data from the file """ from audiotools.wav import pad_data header = [] if (pad_data(self.total_frames(), self.channels(), self.bits_per_sample())): footer = [b"\x00"] else: footer = [] current_block = header metadata = self.get_metadata() # convert individual chunks into combined header and footer strings for block in metadata.get_blocks(Flac_APPLICATION.BLOCK_ID): if block.application_id == b"riff": chunk_id = block.data[0:4] # combine APPLICATION metadata blocks up to "data" as header if chunk_id != b"data": current_block.append(block.data) else: # combine APPLICATION metadata blocks past "data" as footer current_block.append(block.data) current_block = footer # return tuple of header and footer if (len(header) != 0) or (len(footer) != 0): return (b"".join(header), b"".join(footer)) else: raise ValueError("no foreign RIFF chunks") @classmethod def from_wave(cls, filename, header, pcmreader, footer, compression=None): """encodes a new file from wave data takes a filename string, header string, PCMReader object, footer string and optional compression level string encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new WaveAudio object may raise EncodingError if some problem occurs when encoding the input file""" from io import BytesIO from audiotools.bitstream import BitstreamReader from audiotools.bitstream import BitstreamRecorder from audiotools.bitstream import format_byte_size from audiotools.wav import (pad_data, WaveAudio) from audiotools import (EncodingError, CounterPCMReader) # split header and footer into distinct chunks header_len = len(header) footer_len = len(footer) fmt_found = False blocks = [] try: # read everything from start of header to "data<size>" # chunk header r = BitstreamReader(BytesIO(header), True) (riff, remaining_size, wave) = r.parse("4b 32u 4b") if riff != b"RIFF": from audiotools.text import ERR_WAV_NOT_WAVE raise EncodingError(ERR_WAV_NOT_WAVE) elif wave != b"WAVE": from audiotools.text import ERR_WAV_INVALID_WAVE raise EncodingError(ERR_WAV_INVALID_WAVE) else: block_data = BitstreamRecorder(True) block_data.build("4b 32u 4b", (riff, remaining_size, wave)) blocks.append(Flac_APPLICATION(b"riff", block_data.data())) total_size = remaining_size + 8 header_len -= format_byte_size("4b 32u 4b") while header_len: block_data = BitstreamRecorder(True) (chunk_id, chunk_size) = r.parse("4b 32u") # ensure chunk ID is valid if (not frozenset(chunk_id).issubset( WaveAudio.PRINTABLE_ASCII)): from audiotools.text import ERR_WAV_INVALID_CHUNK raise EncodingError(ERR_WAV_INVALID_CHUNK) else: header_len -= format_byte_size("4b 32u") block_data.build("4b 32u", (chunk_id, chunk_size)) if chunk_id == b"data": # transfer only "data" chunk header to APPLICATION block if header_len != 0: from audiotools.text import ERR_WAV_HEADER_EXTRA_DATA raise EncodingError( ERR_WAV_HEADER_EXTRA_DATA.format(header_len)) elif not fmt_found: from audiotools.text import ERR_WAV_NO_FMT_CHUNK raise EncodingError(ERR_WAV_NO_FMT_CHUNK) else: blocks.append( Flac_APPLICATION(b"riff", block_data.data())) data_chunk_size = chunk_size break elif chunk_id == b"fmt ": if not fmt_found: fmt_found = True if chunk_size % 2: # transfer padded chunk to APPLICATION block block_data.write_bytes( r.read_bytes(chunk_size + 1)) header_len -= (chunk_size + 1) else: # transfer un-padded chunk to APPLICATION block block_data.write_bytes( r.read_bytes(chunk_size)) header_len -= chunk_size blocks.append( Flac_APPLICATION(b"riff", block_data.data())) else: from audiotools.text import ERR_WAV_MULTIPLE_FMT raise EncodingError(ERR_WAV_MULTIPLE_FMT) else: if chunk_size % 2: # transfer padded chunk to APPLICATION block block_data.write_bytes(r.read_bytes(chunk_size + 1)) header_len -= (chunk_size + 1) else: # transfer un-padded chunk to APPLICATION block block_data.write_bytes(r.read_bytes(chunk_size)) header_len -= chunk_size blocks.append(Flac_APPLICATION(b"riff", block_data.data())) else: from audiotools.text import ERR_WAV_NO_DATA_CHUNK raise EncodingError(ERR_WAV_NO_DATA_CHUNK) except IOError: from audiotools.text import ERR_WAV_HEADER_IOERROR raise EncodingError(ERR_WAV_HEADER_IOERROR) try: # read everything from start of footer to end of footer r = BitstreamReader(BytesIO(footer), True) # skip initial footer pad byte if data_chunk_size % 2: r.skip_bytes(1) footer_len -= 1 while footer_len: block_data = BitstreamRecorder(True) (chunk_id, chunk_size) = r.parse("4b 32u") if (not frozenset(chunk_id).issubset( WaveAudio.PRINTABLE_ASCII)): # ensure chunk ID is valid from audiotools.text import ERR_WAV_INVALID_CHUNK raise EncodingError(ERR_WAV_INVALID_CHUNK) elif chunk_id == b"fmt ": # multiple "fmt " chunks is an error from audiotools.text import ERR_WAV_MULTIPLE_FMT raise EncodingError(ERR_WAV_MULTIPLE_FMT) elif chunk_id == b"data": # multiple "data" chunks is an error from audiotools.text import ERR_WAV_MULTIPLE_DATA raise EncodingError(ERR_WAV_MULTIPLE_DATA) else: footer_len -= format_byte_size("4b 32u") block_data.build("4b 32u", (chunk_id, chunk_size)) if chunk_size % 2: # transfer padded chunk to APPLICATION block block_data.write_bytes(r.read_bytes(chunk_size + 1)) footer_len -= (chunk_size + 1) else: # transfer un-padded chunk to APPLICATION block block_data.write_bytes(r.read_bytes(chunk_size)) footer_len -= chunk_size blocks.append(Flac_APPLICATION(b"riff", block_data.data())) except IOError: from audiotools.text import ERR_WAV_FOOTER_IOERROR raise EncodingError(ERR_WAV_FOOTER_IOERROR) counter = CounterPCMReader(pcmreader) # perform standard FLAC encode from PCMReader flac = cls.from_pcm(filename, counter, compression) data_bytes_written = counter.bytes_written() # ensure processed PCM data equals size of "data" chunk if data_bytes_written != data_chunk_size: cls.__unlink__(filename) from audiotools.text import ERR_WAV_TRUNCATED_DATA_CHUNK raise EncodingError(ERR_WAV_TRUNCATED_DATA_CHUNK) # ensure total size of header + PCM + footer matches wav's header if (len(header) + data_bytes_written + len(footer)) != total_size: cls.__unlink__(filename) from audiotools.text import ERR_WAV_INVALID_SIZE raise EncodingError(ERR_WAV_INVALID_SIZE) # add chunks as APPLICATION metadata blocks metadata = flac.get_metadata() for block in blocks: metadata.add_block(block) flac.update_metadata(metadata) # return encoded FLAC file return flac def has_foreign_aiff_chunks(self): """returns True if the audio file contains non-audio AIFF chunks""" try: return b'aiff' in [ block.application_id for block in self.get_metadata().get_blocks(Flac_APPLICATION.BLOCK_ID)] except IOError: return False def aiff_header_footer(self): """returns (header, footer) tuple of strings containing all data before and after the PCM stream if self.has_foreign_aiff_chunks() is False, may raise ValueError if the file has no header and footer for any reason""" from audiotools.aiff import pad_data header = [] if (pad_data(self.total_frames(), self.channels(), self.bits_per_sample())): footer = [b"\x00"] else: footer = [] current_block = header metadata = self.get_metadata() if metadata is None: raise ValueError("no foreign AIFF chunks") # convert individual chunks into combined header and footer strings for block in metadata.get_blocks(Flac_APPLICATION.BLOCK_ID): if block.application_id == b"aiff": chunk_id = block.data[0:4] # combine APPLICATION metadata blocks up to "SSND" as header if chunk_id != b"SSND": current_block.append(block.data) else: # combine APPLICATION metadata blocks past "SSND" as footer current_block.append(block.data) current_block = footer # return tuple of header and footer if (len(header) != 0) or (len(footer) != 0): return (b"".join(header), b"".join(footer)) else: raise ValueError("no foreign AIFF chunks") @classmethod def from_aiff(cls, filename, header, pcmreader, footer, compression=None): """encodes a new file from AIFF data takes a filename string, header string, PCMReader object, footer string and optional compression level string encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new AiffAudio object header + pcm data + footer should always result in the original AIFF file being restored without need for any padding bytes may raise EncodingError if some problem occurs when encoding the input file""" from io import BytesIO from audiotools.bitstream import BitstreamReader from audiotools.bitstream import BitstreamRecorder from audiotools.bitstream import format_byte_size from audiotools.aiff import (pad_data, AiffAudio) from audiotools import (EncodingError, CounterPCMReader) # split header and footer into distinct chunks header_len = len(header) footer_len = len(footer) comm_found = False blocks = [] try: # read everything from start of header to "SSND<size>" # chunk header r = BitstreamReader(BytesIO(header), False) (form, remaining_size, aiff) = r.parse("4b 32u 4b") if form != b"FORM": from audiotools.text import ERR_AIFF_NOT_AIFF raise EncodingError(ERR_AIFF_NOT_AIFF) elif aiff != b"AIFF": from audiotools.text import ERR_AIFF_INVALID_AIFF raise EncodingError(ERR_AIFF_INVALID_AIFF) else: block_data = BitstreamRecorder(0) block_data.build("4b 32u 4b", (form, remaining_size, aiff)) blocks.append(Flac_APPLICATION("aiff", block_data.data())) total_size = remaining_size + 8 header_len -= format_byte_size("4b 32u 4b") while header_len: block_data = BitstreamRecorder(0) (chunk_id, chunk_size) = r.parse("4b 32u") # ensure chunk ID is valid if (not frozenset(chunk_id).issubset( AiffAudio.PRINTABLE_ASCII)): from audiotools.text import ERR_AIFF_INVALID_CHUNK raise EncodingError(ERR_AIFF_INVALID_CHUNK) else: header_len -= format_byte_size("4b 32u") block_data.build("4b 32u", (chunk_id, chunk_size)) if chunk_id == b"SSND": from audiotools.text import (ERR_AIFF_HEADER_EXTRA_SSND, ERR_AIFF_HEADER_MISSING_SSND, ERR_AIFF_NO_COMM_CHUNK) # transfer only "SSND" chunk header to APPLICATION block # (including 8 bytes after ID/size header) if header_len > 8: raise EncodingError(ERR_AIFF_HEADER_EXTRA_SSND) elif header_len < 8: raise EncodingError(ERR_AIFF_HEADER_MISSING_SSND) elif not comm_found: raise EncodingError(ERR_AIFF_NO_COMM_CHUNK) else: block_data.write_bytes(r.read_bytes(8)) blocks.append( Flac_APPLICATION(b"aiff", block_data.data())) ssnd_chunk_size = (chunk_size - 8) break elif chunk_id == b"COMM": from audiotools.text import ERR_AIFF_MULTIPLE_COMM_CHUNKS if not comm_found: comm_found = True if chunk_size % 2: # transfer padded chunk to APPLICATION block block_data.write_bytes( r.read_bytes(chunk_size + 1)) header_len -= (chunk_size + 1) else: # transfer un-padded chunk to APPLICATION block block_data.write_bytes( r.read_bytes(chunk_size)) header_len -= chunk_size blocks.append( Flac_APPLICATION(b"aiff", block_data.data())) else: raise EncodingError(ERR_AIFF_MULTIPLE_COMM_CHUNKS) else: if chunk_size % 2: # transfer padded chunk to APPLICATION block block_data.write_bytes(r.read_bytes(chunk_size + 1)) header_len -= (chunk_size + 1) else: # transfer un-padded chunk to APPLICATION block block_data.write_bytes(r.read_bytes(chunk_size)) header_len -= chunk_size blocks.append(Flac_APPLICATION(b"aiff", block_data.data())) else: from audiotools.text import ERR_AIFF_NO_SSND_CHUNK raise EncodingError(ERR_AIFF_NO_SSND_CHUNK) except IOError: from audiotools.text import ERR_AIFF_HEADER_IOERROR raise EncodingError(ERR_AIFF_HEADER_IOERROR) try: # read everything from start of footer to end of footer r = BitstreamReader(BytesIO(footer), False) # skip initial footer pad byte if ssnd_chunk_size % 2: r.skip_bytes(1) footer_len -= 1 while footer_len: block_data = BitstreamRecorder(0) (chunk_id, chunk_size) = r.parse("4b 32u") if (not frozenset(chunk_id).issubset( AiffAudio.PRINTABLE_ASCII)): # ensure chunk ID is valid from audiotools.text import ERR_AIFF_INVALID_CHUNK raise EncodingError(ERR_AIFF_INVALID_CHUNK) elif chunk_id == b"COMM": # multiple "COMM" chunks is an error from audiotools.text import ERR_AIFF_MULTIPLE_COMM_CHUNKS raise EncodingError(ERR_AIFF_MULTIPLE_COMM_CHUNKS) elif chunk_id == b"SSND": # multiple "SSND" chunks is an error from audiotools.text import ERR_AIFF_MULTIPLE_SSND_CHUNKS raise EncodingError(ERR_AIFF_MULTIPLE_SSND_CHUNKS) else: footer_len -= format_byte_size("4b 32u") block_data.build("4b 32u", (chunk_id, chunk_size)) if chunk_size % 2: # transfer padded chunk to APPLICATION block block_data.write_bytes(r.read_bytes(chunk_size + 1)) footer_len -= (chunk_size + 1) else: # transfer un-padded chunk to APPLICATION block block_data.write_bytes(r.read_bytes(chunk_size)) footer_len -= chunk_size blocks.append(Flac_APPLICATION(b"aiff", block_data.data())) except IOError: from audiotools.text import ERR_AIFF_FOOTER_IOERROR raise EncodingError(ERR_AIFF_FOOTER_IOERROR) counter = CounterPCMReader(pcmreader) # perform standard FLAC encode from PCMReader flac = cls.from_pcm(filename, counter, compression) ssnd_bytes_written = counter.bytes_written() # ensure processed PCM data equals size of "SSND" chunk if ssnd_bytes_written != ssnd_chunk_size: cls.__unlink__(filename) from audiotools.text import ERR_AIFF_TRUNCATED_SSND_CHUNK raise EncodingError(ERR_AIFF_TRUNCATED_SSND_CHUNK) # ensure total size of header + PCM + footer matches aiff's header if (len(header) + ssnd_bytes_written + len(footer)) != total_size: cls.__unlink__(filename) from audiotools.text import ERR_AIFF_INVALID_SIZE raise EncodingError(ERR_AIFF_INVALID_SIZE) # add chunks as APPLICATION metadata blocks metadata = flac.get_metadata() if metadata is not None: for block in blocks: metadata.add_block(block) flac.update_metadata(metadata) # return encoded FLAC file return flac def convert(self, target_path, target_class, compression=None, progress=None): """encodes a new AudioFile from existing AudioFile take a filename string, target class and optional compression string encodes a new AudioFile in the target class and returns the resulting object may raise EncodingError if some problem occurs during encoding""" # If a FLAC has embedded RIFF *and* embedded AIFF chunks, # RIFF takes precedence if the target format supports both. # (it's hard to envision a scenario in which that would happen) from audiotools import WaveAudio from audiotools import AiffAudio from audiotools import to_pcm_progress if ((self.has_foreign_wave_chunks() and hasattr(target_class, "from_wave") and callable(target_class.from_wave))): return WaveContainer.convert(self, target_path, target_class, compression, progress) elif (self.has_foreign_aiff_chunks() and hasattr(target_class, "from_aiff") and callable(target_class.from_aiff)): return AiffContainer.convert(self, target_path, target_class, compression, progress) else: return target_class.from_pcm( target_path, to_pcm_progress(self, progress), compression, total_pcm_frames=self.total_frames()) def bits_per_sample(self): """returns an integer number of bits-per-sample this track contains""" return self.__bitspersample__ def channels(self): """returns an integer number of channels this track contains""" return self.__channels__ def total_frames(self): """returns the total PCM frames of the track as an integer""" return self.__total_frames__ def sample_rate(self): """returns the rate of the track's audio as an integer number of Hz""" return self.__samplerate__ @classmethod def supports_replay_gain(cls): """returns True if this class supports ReplayGain""" return True def get_replay_gain(self): """returns a ReplayGain object of our ReplayGain values returns None if we have no values""" from audiotools import ReplayGain try: vorbis_metadata = self.get_metadata().get_block( Flac_VORBISCOMMENT.BLOCK_ID) except (IndexError, IOError): return None if ({u'REPLAYGAIN_TRACK_PEAK', u'REPLAYGAIN_TRACK_GAIN', u'REPLAYGAIN_ALBUM_PEAK', u'REPLAYGAIN_ALBUM_GAIN'}.issubset( [key.upper() for key in vorbis_metadata.keys()])): # we have ReplayGain data try: return ReplayGain( vorbis_metadata[u'REPLAYGAIN_TRACK_GAIN'][0][0:-len(" dB")], vorbis_metadata[u'REPLAYGAIN_TRACK_PEAK'][0], vorbis_metadata[u'REPLAYGAIN_ALBUM_GAIN'][0][0:-len(" dB")], vorbis_metadata[u'REPLAYGAIN_ALBUM_PEAK'][0]) except ValueError: return None else: return None def set_replay_gain(self, replaygain): """given a ReplayGain object, sets the track's gain to those values may raise IOError if unable to modify the file""" if replaygain is None: return self.delete_replay_gain() metadata = self.get_metadata() if metadata.has_block(Flac_VORBISCOMMENT.BLOCK_ID): vorbis_comment = metadata.get_block(Flac_VORBISCOMMENT.BLOCK_ID) else: from audiotools import VERSION vorbis_comment = Flac_VORBISCOMMENT( [], u"Python Audio Tools {}".format(VERSION)) metadata.add_block(vorbis_comment) vorbis_comment[u"REPLAYGAIN_TRACK_GAIN"] = [ u"{:.2f} dB".format(replaygain.track_gain)] vorbis_comment[u"REPLAYGAIN_TRACK_PEAK"] = [ u"{:.8f}".format(replaygain.track_peak)] vorbis_comment[u"REPLAYGAIN_ALBUM_GAIN"] = [ u"{:.2f} dB".format(replaygain.album_gain)] vorbis_comment[u"REPLAYGAIN_ALBUM_PEAK"] = [ u"{:.8f}".format(replaygain.album_peak)] vorbis_comment[u"REPLAYGAIN_REFERENCE_LOUDNESS"] = [u"89.0 dB"] self.update_metadata(metadata) def delete_replay_gain(self): """removes ReplayGain values from file, if any may raise IOError if unable to modify the file""" metadata = self.get_metadata() if metadata.has_block(Flac_VORBISCOMMENT.BLOCK_ID): vorbis_comment = metadata.get_block(Flac_VORBISCOMMENT.BLOCK_ID) for field in [u"REPLAYGAIN_TRACK_GAIN", u"REPLAYGAIN_TRACK_PEAK", u"REPLAYGAIN_ALBUM_GAIN", u"REPLAYGAIN_ALBUM_PEAK", u"REPLAYGAIN_REFERENCE_LOUDNESS"]: try: del(vorbis_comment[field]) except KeyError: pass self.update_metadata(metadata) def clean(self, output_filename=None): """cleans the file of known data and metadata problems output_filename is an optional filename of the fixed file if present, a new AudioFile is written to that path otherwise, only a dry-run is performed and no new file is written return list of fixes performed as Unicode strings raises IOError if unable to write the file or its metadata raises ValueError if the file has errors of some sort """ import os.path from audiotools.id3 import skip_id3v2_comment def seektable_valid(seektable, metadata_offset, input_file): from audiotools.bitstream import BitstreamReader reader = BitstreamReader(input_file, False) for (pcm_frame_offset, seekpoint_offset, pcm_frame_count) in seektable.seekpoints: reader.seek(seekpoint_offset + metadata_offset) try: (sync_code, reserved1, reserved2) = reader.parse( "14u 1u 1p 4p 4p 4p 3p 1u") if (((sync_code != 0x3FFE) or (reserved1 != 0) or (reserved2 != 0))): return False except IOError: return False else: return True fixes_performed = [] with open(self.filename, "rb") as input_f: # remove ID3 tags from before and after FLAC stream stream_size = os.path.getsize(self.filename) stream_offset = skip_id3v2_comment(input_f) if stream_offset > 0: from audiotools.text import CLEAN_FLAC_REMOVE_ID3V2 fixes_performed.append(CLEAN_FLAC_REMOVE_ID3V2) stream_size -= stream_offset try: input_f.seek(-128, 2) if input_f.read(3) == b'TAG': from audiotools.text import CLEAN_FLAC_REMOVE_ID3V1 fixes_performed.append(CLEAN_FLAC_REMOVE_ID3V1) stream_size -= 128 except IOError: # file isn't 128 bytes long pass if output_filename is not None: with open(output_filename, "wb") as output_f: input_f.seek(stream_offset, 0) while stream_size > 0: s = input_f.read(4096) if len(s) > stream_size: s = s[0:stream_size] output_f.write(s) stream_size -= len(s) output_track = self.__class__(output_filename) metadata = self.get_metadata() metadata_size = metadata.size() # fix empty MD5SUM if self.__md5__ == b"\x00" * 16: from hashlib import md5 from audiotools import transfer_framelist_data md5sum = md5() transfer_framelist_data( self.to_pcm(), md5sum.update, signed=True, big_endian=False) metadata.get_block( Flac_STREAMINFO.BLOCK_ID).md5sum = md5sum.digest() from audiotools.text import CLEAN_FLAC_POPULATE_MD5 fixes_performed.append(CLEAN_FLAC_POPULATE_MD5) # fix missing WAVEFORMATEXTENSIBLE_CHANNEL_MASK if (((self.channels() > 2) or (self.bits_per_sample() > 16))): from audiotools.text import CLEAN_FLAC_ADD_CHANNELMASK try: vorbis_comment = metadata.get_block( Flac_VORBISCOMMENT.BLOCK_ID) except IndexError: from audiotools import VERSION vorbis_comment = Flac_VORBISCOMMENT( [], u"Python Audio Tools {}".format(VERSION)) if ((u"WAVEFORMATEXTENSIBLE_CHANNEL_MASK" not in vorbis_comment.keys())): fixes_performed.append(CLEAN_FLAC_ADD_CHANNELMASK) vorbis_comment[ u"WAVEFORMATEXTENSIBLE_CHANNEL_MASK"] = \ [u"0x{:04X}".format(int(self.channel_mask()))] metadata.replace_blocks( Flac_VORBISCOMMENT.BLOCK_ID, [vorbis_comment]) if metadata.has_block(Flac_SEEKTABLE.BLOCK_ID): # fix an invalid SEEKTABLE, if necessary if (not seektable_valid( metadata.get_block(Flac_SEEKTABLE.BLOCK_ID), stream_offset + 4 + metadata_size, input_f)): from audiotools.text import CLEAN_FLAC_FIX_SEEKTABLE fixes_performed.append(CLEAN_FLAC_FIX_SEEKTABLE) metadata.replace_blocks(Flac_SEEKTABLE.BLOCK_ID, [self.seektable()]) else: # add SEEKTABLE block if not present from audiotools.text import CLEAN_FLAC_ADD_SEEKTABLE fixes_performed.append(CLEAN_FLAC_ADD_SEEKTABLE) metadata.add_block(self.seektable()) # fix remaining metadata problems # which automatically shifts STREAMINFO to the right place # (the message indicating the fix has already been output) (metadata, metadata_fixes) = metadata.clean() if output_filename is not None: output_track.update_metadata(metadata) return fixes_performed + metadata_fixes class OggFlacMetaData(FlacMetaData): @classmethod def converted(cls, metadata): """takes a MetaData object and returns an OggFlacMetaData object""" if metadata is None: return None elif isinstance(metadata, FlacMetaData): return cls([block.copy() for block in metadata.block_list]) else: return cls([Flac_VORBISCOMMENT.converted(metadata)] + [Flac_PICTURE.converted(image) for image in metadata.images()]) def __repr__(self): return ("OggFlacMetaData({!r})".format(self.block_list)) @classmethod def parse(cls, packetreader): """returns an OggFlacMetaData object from the given ogg.PacketReader raises IOError or ValueError if an error occurs reading MetaData""" from io import BytesIO from audiotools.bitstream import BitstreamReader, parse streaminfo = None applications = [] seektable = None vorbis_comment = None cuesheet = None pictures = [] (packet_byte, ogg_signature, major_version, minor_version, header_packets, flac_signature, block_type, block_length, minimum_block_size, maximum_block_size, minimum_frame_size, maximum_frame_size, sample_rate, channels, bits_per_sample, total_samples, md5sum) = parse( "8u 4b 8u 8u 16u 4b 8u 24u 16u 16u 24u 24u 20u 3u 5u 36U 16b", False, packetreader.read_packet()) block_list = [Flac_STREAMINFO(minimum_block_size=minimum_block_size, maximum_block_size=maximum_block_size, minimum_frame_size=minimum_frame_size, maximum_frame_size=maximum_frame_size, sample_rate=sample_rate, channels=channels + 1, bits_per_sample=bits_per_sample + 1, total_samples=total_samples, md5sum=md5sum)] for i in range(header_packets): packet = BitstreamReader(BytesIO(packetreader.read_packet()), False) (block_type, length) = packet.parse("1p 7u 24u") if block_type == 1: # PADDING block_list.append(Flac_PADDING.parse(packet, length)) if block_type == 2: # APPLICATION block_list.append(Flac_APPLICATION.parse(packet, length)) elif block_type == 3: # SEEKTABLE block_list.append(Flac_SEEKTABLE.parse(packet, length // 18)) elif block_type == 4: # VORBIS_COMMENT block_list.append(Flac_VORBISCOMMENT.parse(packet)) elif block_type == 5: # CUESHEET block_list.append(Flac_CUESHEET.parse(packet)) elif block_type == 6: # PICTURE block_list.append(Flac_PICTURE.parse(packet)) elif (block_type >= 7) and (block_type <= 126): from audiotools.text import ERR_FLAC_RESERVED_BLOCK raise ValueError(ERR_FLAC_RESERVED_BLOCK.format(block_type)) elif block_type == 127: from audiotools.text import ERR_FLAC_INVALID_BLOCK raise ValueError(ERR_FLAC_INVALID_BLOCK) return cls(block_list) def build(self, pagewriter, serial_number): """pagewriter is an ogg.PageWriter object returns new sequence number""" from audiotools.bitstream import build, BitstreamRecorder, format_size from audiotools.ogg import packet_to_pages # build extended Ogg FLAC STREAMINFO block # which will always occupy its own page streaminfo = self.get_block(Flac_STREAMINFO.BLOCK_ID) # all our non-STREAMINFO blocks that are small enough # to fit in the output stream valid_blocks = [b for b in self.blocks() if ((b.BLOCK_ID != Flac_STREAMINFO.BLOCK_ID) and (b.size() < (2 ** 24)))] page = next(packet_to_pages( build("8u 4b 8u 8u 16u " + "4b 8u 24u 16u 16u 24u 24u 20u 3u 5u 36U 16b", False, (0x7F, b"FLAC", 1, 0, len(valid_blocks), b"fLaC", 0, format_size("16u 16u 24u 24u 20u 3u 5u 36U 16b") // 8, streaminfo.minimum_block_size, streaminfo.maximum_block_size, streaminfo.minimum_frame_size, streaminfo.maximum_frame_size, streaminfo.sample_rate, streaminfo.channels - 1, streaminfo.bits_per_sample - 1, streaminfo.total_samples, streaminfo.md5sum)), bitstream_serial_number=serial_number, starting_sequence_number=0)) page.stream_beginning = True pagewriter.write(page) sequence_number = 1 # pack remaining metadata blocks into Ogg packets for (i, block) in enumerate(valid_blocks, 1): packet = BitstreamRecorder(False) packet.build("1u 7u 24u", (0 if not (i == len(valid_blocks)) else 1, block.BLOCK_ID, block.size())) block.build(packet) for page in packet_to_pages( packet.data(), bitstream_serial_number=serial_number, starting_sequence_number=sequence_number): pagewriter.write(page) sequence_number += 1 return sequence_number def sizes_to_offsets(sizes): """takes list of (frame_size, frame_frames) tuples and converts it to a list of (cumulative_size, frame_frames) tuples""" current_position = 0 offsets = [] for frame_size, frame_frames in sizes: offsets.append((current_position, frame_frames)) current_position += frame_size return offsets ================================================ FILE: audiotools/freedb.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA import sys def digit_sum(i): """returns the sum of all digits for the given integer""" return sum(map(int, str(i))) class DiscID(object): def __init__(self, offsets, total_length, track_count, playable_length): """offsets is a list of track offsets, in CD frames total_length is the total length of the disc, in seconds track_count is the total number of tracks on the disc playable_length is the playable length of the disc, in seconds the first three items are for generating the hex disc ID itself while the last is for performing queries""" assert(len(offsets) == track_count) for o in offsets: assert(o >= 0) self.offsets = offsets self.total_length = total_length self.track_count = track_count self.playable_length = playable_length @classmethod def from_cddareader(cls, cddareader): """given a CDDAReader object, returns a DiscID for that object""" def offset(sector): # the HOWTO implies sectors should be lopped off, like: # t = ((cdtoc[tot_trks].min * 60) + cdtoc[tot_trks].sec) - # ((cdtoc[0].min * 60) + cdtoc[0].sec); minutes = sector // 75 // 60 seconds = sector // 75 % 60 return minutes * 60 + seconds offsets = cddareader.track_offsets return cls(offsets=[(offsets[k] // 588) + 150 for k in sorted(offsets.keys())], total_length=(offset(cddareader.last_sector + 150 + 1) - offset(cddareader.first_sector + 150)), track_count=len(offsets), playable_length=(cddareader.last_sector + 150 + 1) // 75) @classmethod def from_tracks(cls, tracks): """given a sorted list of tracks, returns DiscID for those tracks as if they were a CD""" from audiotools import has_pre_gap_track if not has_pre_gap_track(tracks): offsets = [150] for track in tracks[0:-1]: offsets.append(offsets[-1] + track.cd_frames()) #track_lengths = sum(t.cd_frames() for t in tracks) // 75 total_length = sum(t.seconds_length() for t in tracks) return cls(offsets=offsets, total_length=int(total_length), track_count=len(tracks), playable_length=int(total_length + 2)) else: offsets = [150 + tracks[0].cd_frames()] for track in tracks[1:-1]: offsets.append(offsets[-1] + track.cd_frames()) total_length = sum(t.seconds_length() for t in tracks[1:]) return cls( offsets=offsets, total_length=int(total_length), track_count=len(tracks) - 1, playable_length=int(total_length + 2)) @classmethod def from_sheet(cls, sheet, total_pcm_frames, sample_rate): """given a Sheet object length of the album in PCM frames and sample rate of the disc, returns a DiscID for that CD""" return cls(offsets=[int(t.index(1).offset() * 75 + 150) for t in sheet], total_length=((total_pcm_frames // sample_rate) - int(sheet.track(1).index(1).offset())), track_count=len(sheet), playable_length=((total_pcm_frames + (sample_rate * 2)) // sample_rate)) def __repr__(self): return "DiscID({})".format( ", ".join(["{}={}".format(attr, getattr(self, attr)) for attr in ["offsets", "total_length", "track_count", "playable_length"]])) if sys.version_info[0] >= 3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('ascii') def __unicode__(self): return u"{:08X}".format(int(self)) def __int__(self): digit_sum_ = sum([digit_sum(o // 75) for o in self.offsets]) return (((digit_sum_ % 255) << 24) | ((self.total_length & 0xFFFF) << 8) | (self.track_count & 0xFF)) def perform_lookup(disc_id, freedb_server, freedb_port): """performs a web-based lookup using a DiscID on the given freedb_server string and freedb_int port iterates over a list of MetaData objects per successful match, like: [track1, track2, ...], [track1, track2, ...], ... may raise HTTPError if an error occurs querying the server or ValueError if the server returns invalid data """ import re from time import sleep RESPONSE = re.compile(r'(\d{3}) (.+?)[\r\n]+') QUERY_RESULT = re.compile(r'(\S+) ([0-9a-fA-F]{8}) (.+)') FREEDB_LINE = re.compile(r'(\S+?)=(.+?)[\r\n]+') query = freedb_command(freedb_server, freedb_port, u"query", *([disc_id.__unicode__(), u"{:d}".format(disc_id.track_count)] + [u"{:d}".format(o) for o in disc_id.offsets] + [u"{:d}".format(disc_id.playable_length)])) line = next(query) response = RESPONSE.match(line) if response is None: raise ValueError("invalid response from server") else: # a list of (category, disc id, disc title) tuples matches = [] code = int(response.group(1)) if code == 200: # single exact match match = QUERY_RESULT.match(response.group(2)) if match is not None: matches.append((match.group(1), match.group(2), match.group(3))) else: raise ValueError("invalid query result") elif (code == 211) or (code == 210): # multiple exact or inexact matches line = next(query) while not line.startswith(u"."): match = QUERY_RESULT.match(line) if match is not None: matches.append((match.group(1), match.group(2), match.group(3))) else: raise ValueError("invalid query result") line = next(query) elif code == 202: # no match found pass else: # some error has occurred raise ValueError(response.group(2)) if len(matches) > 0: # for each result, query FreeDB for XMCD file data for (category, disc_id, title) in matches: sleep(1) # add a slight delay to keep the server happy query = freedb_command(freedb_server, freedb_port, u"read", category, disc_id) response = RESPONSE.match(next(query)) if response is not None: # FIXME - check response code here freedb = {} line = next(query) while not line.startswith(u"."): if not line.startswith(u"#"): entry = FREEDB_LINE.match(line) if entry is not None: if entry.group(1) in freedb: freedb[entry.group(1)] += entry.group(2) else: freedb[entry.group(1)] = entry.group(2) line = next(query) yield list(xmcd_metadata(freedb)) else: raise ValueError("invalid response from server") def freedb_command(freedb_server, freedb_port, cmd, *args): """given a freedb_server string, freedb_port int, command unicode string and argument unicode strings, yields a list of Unicode strings""" try: from urllib.request import urlopen except ImportError: from urllib2 import urlopen try: from urllib.parse import urlencode except ImportError: from urllib import urlencode from socket import getfqdn from audiotools import VERSION from sys import version_info PY3 = version_info[0] >= 3 # some debug type checking assert(isinstance(cmd, str if PY3 else unicode)) for arg in args: assert(isinstance(arg, str if PY3 else unicode)) POST = [] # generate query to post with arguments in specific order if len(args) > 0: POST.append((u"cmd", u"cddb {} {}".format(cmd, " ".join(args)))) else: POST.append((u"cmd", u"cddb {}".format(cmd))) POST.append( (u"hello", u"user {} {} {}".format( getfqdn() if PY3 else getfqdn().decode("UTF-8", "replace"), u"audiotools", VERSION if PY3 else VERSION.decode("ascii")))) POST.append((u"proto", u"6")) # get Request object from post request = urlopen( "http://{}:{:d}/~cddb/cddb.cgi".format(freedb_server, freedb_port), urlencode(POST).encode("UTF-8") if (version_info[0] >= 3) else urlencode(POST)) try: # yield lines of output line = request.readline() while len(line) > 0: yield line.decode("UTF-8", "replace") line = request.readline() finally: request.close() def xmcd_metadata(freedb_file): """given a dict of KEY->value unicode strings, yields a MetaData object per track""" import re TTITLE = re.compile(r'TTITLE(\d+)') dtitle = freedb_file.get(u"DTITLE", u"") if u" / " in dtitle: (album_artist, album_name) = dtitle.split(u" / ", 1) else: album_artist = None album_name = dtitle year = freedb_file.get(u"DYEAR", None) ttitles = [(int(m.group(1)), value) for (m, value) in [(TTITLE.match(key), value) for (key, value) in freedb_file.items()] if m is not None] if len(ttitles) > 0: track_total = max([tracknum for (tracknum, ttitle) in ttitles]) + 1 else: track_total = 0 for (tracknum, ttitle) in sorted(ttitles, key=lambda t: t[0]): if u" / " in ttitle: (track_artist, track_name) = ttitle.split(u" / ", 1) else: track_artist = album_artist track_name = ttitle from audiotools import MetaData yield MetaData( track_name=track_name, track_number=tracknum + 1, track_total=track_total, album_name=album_name, artist_name=(track_artist if track_artist is not None else None), year=(year if year is not None else None)) ================================================ FILE: audiotools/id3.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools import (MetaData, Image, InvalidImage, config) import sys import codecs from audiotools.id3v1 import ID3v1Comment def is_latin_1(unicode_string): """returns True if the given unicode string is a subset of latin-1""" try: latin1_chars = ({unichr(i) for i in range(32, 127)} | {unichr(i) for i in range(160, 256)}) except NameError: latin1_chars = ({chr(i) for i in range(32, 127)} | {chr(i) for i in range(160, 256)}) return {i for i in unicode_string}.issubset(latin1_chars) class UCS2Codec(codecs.Codec): """a special unicode codec for UCS-2 this is a subset of UTF-16 with no support for surrogate pairs, limiting it to U+0000-U+FFFF""" @classmethod def fix_char(cls, c): """a filter which changes overly large c values to 'unknown'""" if ord(c) <= 0xFFFF: return c else: return u"\ufffd" def encode(self, input, errors='strict'): """encodes unicode input to plain UCS-2 strings""" return codecs.utf_16_encode(u"".join(map(self.fix_char, input)), errors) def decode(self, input, errors='strict'): """decodes plain UCS-2 strings to unicode""" (chars, size) = codecs.utf_16_decode(input, errors, True) return (u"".join(map(self.fix_char, chars)), size) class UCS2CodecStreamWriter(UCS2Codec, codecs.StreamWriter): pass class UCS2CodecStreamReader(UCS2Codec, codecs.StreamReader): pass def __reg_ucs2__(name): if name == 'ucs2': return (UCS2Codec().encode, UCS2Codec().decode, UCS2CodecStreamReader, UCS2CodecStreamWriter) else: return None codecs.register(__reg_ucs2__) def decode_syncsafe32(i): """given a 32 bit int, returns a 28 bit value with sync-safe bits removed may raise ValueError if the value is negative, larger than 32 bits or contains invalid sync-safe bits""" if i >= (2 ** 32): raise ValueError("value of {} is too large".format(i)) elif i < 0: raise ValueError("value cannot be negative") value = 0 for x in range(4): if (i & 0x80) == 0: value |= ((i & 0x7F) << (x * 7)) i >>= 8 else: raise ValueError("invalid sync-safe bit") return value def encode_syncsafe32(i): """given a 28 bit int, returns a 32 bit value with sync-safe bits added may raise ValueError is the value is negative or larger than 28 bits""" if i >= (2 ** 28): raise ValueError("value too large") elif i < 0: raise ValueError("value cannot be negative") value = 0 for x in range(4): value |= ((i & 0x7F) << (x * 8)) i >>= 7 return value class C_string(object): TERMINATOR = {'ascii': b"\x00", 'latin_1': b"\x00", 'latin-1': b"\x00", 'ucs2': b"\x00" * 2, 'utf_16': b"\x00" * 2, 'utf-16': b"\x00" * 2, 'utf_16be': b"\x00" * 2, 'utf-16be': b"\x00" * 2, 'utf_8': b"\x00", 'utf-8': b"\x00"} def __init__(self, encoding, unicode_string): """encoding is a string such as 'utf-8', 'latin-1', etc""" from sys import version_info str_type = str if (version_info[0] >= 3) else unicode assert(encoding in C_string.TERMINATOR.keys()) assert(isinstance(unicode_string, str_type)) self.encoding = encoding self.unicode_string = unicode_string def __repr__(self): return "C_string({!r}, {!r})".format( self.encoding, self.unicode_string) if sys.version_info[0] >= 3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('utf-8') def __unicode__(self): return self.unicode_string def __getitem__(self, char): return self.unicode_string[char] def __len__(self): return len(self.unicode_string) def __eq__(self, s): return (self.unicode_string == u"{}".format(s)) def __ne__(self, s): return (self.unicode_string != u"{}".format(s)) def __lt__(self, s): return (self.unicode_string < u"{}".format(s)) def __le__(self, s): return (self.unicode_string <= u"{}".format(s)) def __gt__(self, s): return (self.unicode_string > u"{}".format(s)) def __ge__(self, s): return (self.unicode_string >= u"{}".format(s)) @classmethod def parse(cls, encoding, reader): """returns a C_string with the given encoding string from the given BitstreamReader raises LookupError if encoding is unknown raises IOError if a problem occurs reading the stream """ try: terminator = cls.TERMINATOR[encoding] terminator_size = len(terminator) except KeyError: raise LookupError(encoding) s = [] char = reader.read_bytes(terminator_size) while char != terminator: s.append(char) char = reader.read_bytes(terminator_size) return cls(encoding, b"".join(s).decode(encoding, 'replace')) def build(self, writer): """writes our C_string data to the given BitstreamWriter with the appropriate terminator""" writer.write_bytes(self.unicode_string.encode(self.encoding, 'replace')) writer.write_bytes(self.TERMINATOR[self.encoding]) def size(self): """returns the length of our C string in bytes""" return (len(self.unicode_string.encode(self.encoding, 'replace')) + len(self.TERMINATOR[self.encoding])) def __attrib_equals__(attributes, o1, o2): for attrib in attributes: if (((not hasattr(o1, attrib)) or (not hasattr(o2, attrib)) or (getattr(o1, attrib) != getattr(o2, attrib)))): return False else: return True def __padded__(value): """given an integer value, returns it as a unicode string with the proper padding""" if config.getboolean_default("ID3", "pad", False): return u"{:02d}".format(value) else: return u"{:d}".format(value) # takes a pair of integers (or None) for the current and total values # returns a unicode string of their combined pair # for example, __number_pair__(2,3) returns u"2/3" # whereas __number_pair__(4,0) returns u"4" def __number_pair__(current, total): if current is None: if total is None: return __padded__(0) else: return __padded__(0) + u"/" + __padded__(total) else: # current is not None if total is None: return __padded__(current) else: return __padded__(current) + u"/" + __padded__(total) def read_id3v2_comment(filename): """given a filename, returns an ID3v22Comment or a subclass for example, if the file is ID3v2.3 tagged, this returns an ID3v23Comment """ from audiotools.bitstream import BitstreamReader with BitstreamReader(open(filename, "rb"), False) as reader: start = reader.getpos() (tag, version_major, version_minor) = reader.parse("3b 8u 8u") if tag != b'ID3': raise ValueError("invalid ID3 header") elif version_major == 0x2: reader.setpos(start) return ID3v22Comment.parse(reader) elif version_major == 0x3: reader.setpos(start) return ID3v23Comment.parse(reader) elif version_major == 0x4: reader.setpos(start) return ID3v24Comment.parse(reader) else: raise ValueError("unsupported ID3 version") def skip_id3v2_comment(file): """seeks past an ID3v2 comment if found in the file stream returns the number of bytes skipped""" from audiotools.bitstream import parse start = file.tell() try: # check initial header if file.read(3) == b"ID3": bytes_skipped = 3 else: file.seek(start) return 0 # ensure major version is valid if ord(file.read(1)) in (2, 3, 4): bytes_skipped += 1 else: file.seek(start) return 0 # skip minor version file.read(1) bytes_skipped += 1 # skip flags file.read(1) bytes_skipped += 1 # get the whole size of the tag try: tag_size = decode_syncsafe32(parse("32u", False, file.read(4))[0]) except ValueError: file.seek(start) return 0 bytes_skipped += 4 # skip to the end of its length file.read(tag_size) bytes_skipped += tag_size # check for additional ID3v2 tags recursively return bytes_skipped + skip_id3v2_comment(file) except IOError: file.seek(start) return 0 def total_id3v2_comments(file): """returns the number of nested ID3v2 comments found in the file stream""" from audiotools.bitstream import parse start = file.tell() try: # check initial header if file.read(3) != b"ID3": file.seek(start) return 0 # ensure major version is valid if ord(file.read(1)) not in (2, 3, 4): file.seek(start) return 0 # skip minor version file.read(1) # skip flags file.read(1) # get the whole size of the tag try: tag_size = decode_syncsafe32(parse("32u", False, file.read(4))[0]) except ValueError: file.seek(start) return 0 # skip to the end of its length file.read(tag_size) # check for additional ID3v2 tags recursively return 1 + total_id3v2_comments(file) except IOError as err: file.seek(start) return 0 ############################################################ # ID3v2.2 Comment ############################################################ class ID3v22_Frame(object): def __init__(self, frame_id, data): self.id = frame_id self.data = data def copy(self): return self.__class__(self.id, self.data) def __repr__(self): return "ID3v22_Frame({!r}, {!r})".format(self.id, self.data) def raw_info(self): from audiotools import hex_string if len(self.data) > 20: return u"{} = {}\u2026".format( self.id.decode('ascii', 'replace'), hex_string(self.data[0:20])) else: return u"{} = {}".format( self.id.decode('ascii', 'replace'), hex_string(self.data)) def __eq__(self, frame): return __attrib_equals__(["id", "data"], self, frame) @classmethod def parse(cls, frame_id, frame_size, reader): """given a frame_id string, frame_size int and BitstreamReader of the remaining frame data, returns a parsed ID3v2?_Frame""" return cls(frame_id, reader.read_bytes(frame_size)) def build(self, writer): """writes this frame to the given BitstreamWriter not including its frame header""" writer.write_bytes(self.data) def size(self): """returns the size of this frame in bytes not including the frame header""" return len(self.data) @classmethod def converted(cls, frame_id, o): """given foreign data, returns an ID3v22_Frame""" raise NotImplementedError() def clean(self): """returns a cleaned ID3v22_Frame and list of fixes, or None if the frame should be removed entirely""" return (self.__class__(self.id, self.data), []) class ID3v22_T__Frame(object): NUMERICAL_IDS = (b'TRK', b'TPA') BOOLEAN_IDS = (b'TCP',) def __init__(self, frame_id, encoding, data): """fields are as follows: | frame_id | 3 byte frame ID string | | encoding | 1 byte encoding int | | data | text data as raw string | """ assert((encoding == 0) or (encoding == 1)) self.id = frame_id self.encoding = encoding self.data = data def copy(self): return self.__class__(self.id, self.encoding, self.data) def __repr__(self): return "ID3v22_T__Frame({!r}, {!r}, {!r})".format( self.id, self.encoding, self.data) def raw_info(self): """returns a human-readable version of this frame as unicode""" return u"{} = ({}) {}".format( self.id.decode('ascii'), {0: u"Latin-1", 1: u"UCS-2"}[self.encoding], self.__unicode__()) def __eq__(self, frame): return __attrib_equals__(["id", "encoding", "data"], self, frame) if sys.version_info[0] >= 3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('utf-8') def __unicode__(self): return self.data.decode( {0: 'latin-1', 1: 'ucs2'}[self.encoding], 'replace').split(u"\x00", 1)[0] def number(self): """if the frame is numerical, returns the track/album_number portion raises TypeError if not""" import re if self.id not in self.NUMERICAL_IDS: raise TypeError() unicode_value = self.__unicode__() int_string = re.search(r'\d+', unicode_value) if int_string is None: return None int_value = int(int_string.group(0)) if (int_value == 0) and (u"/" in unicode_value): total_value = re.search(r'\d+', unicode_value.split(u"/")[1]) if total_value is not None: # don't return placeholder 0 value # when a _total value is present # but _number value is 0 return None else: return int_value else: return int_value def total(self): """if the frame is numerical, returns the track/album_total portion raises TypeError if not""" import re if self.id not in self.NUMERICAL_IDS: raise TypeError() unicode_value = self.__unicode__() if u"/" not in unicode_value: return None int_string = re.search(r'\d+', unicode_value.split(u"/")[1]) if int_string is not None: return int(int_string.group(0)) else: return None def true(self): """if the frame is boolean, returns True if it represents true raises TypeError if not""" if self.id not in self.BOOLEAN_IDS: raise TypeError() return self.__unicode__() == u"1" @classmethod def parse(cls, frame_id, frame_size, reader): """given a frame_id string, frame_size int and BitstreamReader of the remaining frame data, returns a parsed text frame""" encoding = reader.read(8) return cls(frame_id, encoding, reader.read_bytes(frame_size - 1)) def build(self, writer): """writes the frame's data to the BitstreamWriter not including its frame header""" writer.write(8, self.encoding) writer.write_bytes(self.data) def size(self): """returns the frame's total size not including its frame header""" return 1 + len(self.data) @classmethod def converted(cls, frame_id, unicode_string): """given a unicode string, returns a text frame""" if is_latin_1(unicode_string): return cls(frame_id, 0, unicode_string.encode('latin-1')) else: return cls(frame_id, 1, unicode_string.encode('ucs2')) def clean(self): """returns a cleaned frame, or None if the frame should be removed entirely any fixes are appended to fixes_applied as unicode string""" from audiotools.text import (CLEAN_REMOVE_EMPTY_TAG, CLEAN_REMOVE_TRAILING_WHITESPACE, CLEAN_REMOVE_LEADING_WHITESPACE, CLEAN_REMOVE_LEADING_ZEROES, CLEAN_ADD_LEADING_ZEROES) fixes_performed = [] field = self.id.decode('ascii') value = self.__unicode__() # check for an empty tag if len(value.strip()) == 0: return (None, [CLEAN_REMOVE_EMPTY_TAG.format(field)]) # check trailing whitespace fix1 = value.rstrip() if fix1 != value: fixes_performed.append( CLEAN_REMOVE_TRAILING_WHITESPACE.format(field)) # check leading whitespace fix2 = fix1.lstrip() if fix2 != fix1: fixes_performed.append( CLEAN_REMOVE_LEADING_WHITESPACE.format(field)) # check leading zeroes for a numerical tag if self.id in self.NUMERICAL_IDS: fix3 = __number_pair__(self.number(), self.total()) if fix3 != fix2: from audiotools import config if config.getboolean_default("ID3", "pad", False): fixes_performed.append( CLEAN_ADD_LEADING_ZEROES.format(field)) else: fixes_performed.append( CLEAN_REMOVE_LEADING_ZEROES.format(field)) else: fix3 = fix2 return (self.__class__.converted(self.id, fix3), fixes_performed) class ID3v22_TXX_Frame(object): def __init__(self, encoding, description, data): self.id = b'TXX' self.encoding = encoding self.description = description self.data = data def copy(self): return self.__class__(self.encoding, self.description, self.data) def __repr__(self): return "ID3v22_TXX_Frame({!r}, {!r}, {!r})".format( self.encoding, self.description, self.data) def raw_info(self): """returns a human-readable version of this frame as unicode""" return u"{} = ({}, \"{}\") {}".format( self.id.decode('ascii', 'replace'), {0: u"Latin-1", 1: u"UCS-2"}[self.encoding], self.description, self.__unicode__()) def __eq__(self, frame): return __attrib_equals__(["id", "encoding", "description", "data"], self, frame) if sys.version_info[0] >= 3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('utf-8') def __unicode__(self): return self.data.decode( {0: 'latin-1', 1: 'ucs2'}[self.encoding], 'replace').split(u'\x00', 1)[0] @classmethod def parse(cls, frame_id, frame_size, reader): """given a frame_id string, frame_size int and BitstreamReader of the remaining frame data, returns a parsed text frame""" encoding = reader.read(8) description = C_string.parse({0: "latin-1", 1: "ucs2"}[encoding], reader) data = reader.read_bytes(frame_size - 1 - description.size()) return cls(encoding, description, data) def build(self, writer): """writes this frame to the given BitstreamWriter not including its frame header""" writer.write(8, self.encoding) self.description.build(writer) writer.write_bytes(self.data) def size(self): """returns the size of this frame in bytes not including the frame header""" return 1 + self.description.size() + len(self.data) def clean(self): """returns a cleaned frame, or None if the frame should be removed entirely any fixes are appended to fixes_applied as unicode string""" from audiotools.text import (CLEAN_REMOVE_EMPTY_TAG, CLEAN_REMOVE_TRAILING_WHITESPACE, CLEAN_REMOVE_LEADING_WHITESPACE) fixes_performed = [] field = self.id.decode('ascii') value = self.__unicode__() # check for an empty tag if len(value.strip()) == 0: return (None, [CLEAN_REMOVE_EMPTY_TAG.format(field)]) # check trailing whitespace fix1 = value.rstrip() if fix1 != value: fixes_performed.append( CLEAN_REMOVE_TRAILING_WHITESPACE.format(field)) # check leading whitespace fix2 = fix1.lstrip() if fix2 != fix1: fixes_performed.append( CLEAN_REMOVE_LEADING_WHITESPACE.format(field)) return (self.__class__(self.encoding, self.description, fix2), fixes_performed) class ID3v22_W__Frame(object): def __init__(self, frame_id, data): self.id = frame_id self.data = data def copy(self): return self.__class__(self.id, self.data) def __repr__(self): return "ID3v22_W__Frame({!r}, {!r})".format(self.id, self.data) def raw_info(self): """returns a human-readable version of this frame as unicode""" return u"{} = {}".format(self.id.decode('ascii'), self.data.decode('ascii', 'replace')) def __eq__(self, frame): return __attrib_equals__(["id", "data"], self, frame) @classmethod def parse(cls, frame_id, frame_size, reader): return cls(frame_id, reader.read_bytes(frame_size)) def build(self, writer): """writes this frame to the given BitstreamWriter not including its frame header""" writer.write_bytes(self.data) def size(self): """returns the size of this frame in bytes not including the frame header""" return len(self.data) def clean(self): """returns a cleaned frame, or None if the frame should be removed entirely any fixes are appended to fixes_applied as unicode string""" return (self.__class__(self.id, self.data), []) class ID3v22_WXX_Frame(object): def __init__(self, encoding, description, data): self.id = b'WXX' self.encoding = encoding self.description = description self.data = data def copy(self): return self.__class__(self.encoding, self.description, self.data) def __repr__(self): return "ID3v22_WXX_Frame({!r}, {!r}, {!r})".format( self.encoding, self.description, self.data) def raw_info(self): """returns a human-readable version of this frame as unicode""" return u"{} = ({}, \"{}\") {}".format( self.id.decode('ascii', 'replace'), {0: u"Latin-1", 1: u"UCS-2"}[self.encoding], self.description, self.data.decode('ascii', 'replace')) def __eq__(self, frame): return __attrib_equals__(["id", "encoding", "description", "data"]) @classmethod def parse(cls, frame_id, frame_size, reader): """given a frame_id string, frame_size int and BitstreamReader of the remaining frame data, returns a parsed text frame""" encoding = reader.read(8) description = C_string.parse({0: "latin-1", 1: "ucs2"}[encoding], reader) data = reader.read_bytes(frame_size - 1 - description.size()) return cls(encoding, description, data) def build(self, writer): """writes this frame to the given BitstreamWriter not including its frame header""" writer.write(8, self.encoding) self.description.build(writer) writer.write_bytes(self.data) def size(self): """returns the size of this frame in bytes not including the frame header""" return 1 + self.description.size() + len(self.data) def clean(self): """returns a cleaned frame, or None if the frame should be removed entirely any fixes are appended to fixes_applied as unicode string""" return (self.__class__(self.encoding, self.description, self.data), []) class ID3v22_COM_Frame(object): def __init__(self, encoding, language, short_description, data): """fields are as follows: | encoding | 1 byte int of the comment's text encoding | | language | 3 byte string of the comment's language | | short_description | C_string of a short description | | data | plain string of the comment data itself | """ self.id = b"COM" self.encoding = encoding self.language = language self.short_description = short_description self.data = data def copy(self): return self.__class__(self.encoding, self.language, self.short_description, self.data) def __repr__(self): return "ID3v22_COM_Frame({!r}, {!r}, {!r}, {!r})".format( self.encoding, self.language, self.short_description, self.data) def raw_info(self): """returns a human-readable version of this frame as unicode""" return u"COM = ({}, {}, \"{}\") {}".format( {0: u'Latin-1', 1: 'UCS-2'}[self.encoding], self.language.decode('ascii', 'replace'), self.short_description, self.data.decode({0: 'latin-1', 1: 'ucs2'}[self.encoding])) def __eq__(self, frame): return __attrib_equals__(["encoding", "language", "short_description", "data"], self, frame) def __ne__(self, frame): return not self.__eq__(frame) if sys.version_info[0] >= 3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('utf-8') def __unicode__(self): return self.data.decode({0: 'latin-1', 1: 'ucs2'}[self.encoding], 'replace') @classmethod def parse(cls, frame_id, frame_size, reader): """given a frame_id string, frame_size int and BitstreamReader of the remaining frame data, returns a parsed ID3v22_COM_Frame""" (encoding, language) = reader.parse("8u 3b") short_description = C_string.parse({0: 'latin-1', 1: 'ucs2'}[encoding], reader) data = reader.read_bytes(frame_size - (4 + short_description.size())) return cls(encoding, language, short_description, data) def build(self, writer): """writes this frame to the given BitstreamWriter not including its frame header""" writer.build("8u 3b", (self.encoding, self.language)) self.short_description.build(writer) writer.write_bytes(self.data) def size(self): """returns the size of this frame in bytes not including the frame header""" return 4 + self.short_description.size() + len(self.data) @classmethod def converted(cls, frame_id, unicode_string): if is_latin_1(unicode_string): return cls(0, b"eng", C_string("latin-1", u""), unicode_string.encode('latin-1')) else: return cls(1, b"eng", C_string("ucs2", u""), unicode_string.encode('ucs2')) def clean(self): """returns a cleaned frame of the same class or None if the frame should be omitted fix text will be appended to fixes_performed, if necessary""" from audiotools.text import (CLEAN_REMOVE_EMPTY_TAG, CLEAN_REMOVE_TRAILING_WHITESPACE, CLEAN_REMOVE_LEADING_WHITESPACE) fixes_performed = [] field = self.id.decode('ascii') text_encoding = {0: 'latin-1', 1: 'ucs2'} value = self.data.decode(text_encoding[self.encoding], 'replace') # check for an empty tag if len(value.strip()) == 0: return (None, [CLEAN_REMOVE_EMPTY_TAG.format(field)]) # check trailing whitespace fix1 = value.rstrip() if fix1 != value: fixes_performed.append( CLEAN_REMOVE_TRAILING_WHITESPACE.format(field)) # check leading whitespace fix2 = fix1.lstrip() if fix2 != fix1: fixes_performed.append( CLEAN_REMOVE_LEADING_WHITESPACE.format(field)) # stripping whitespace shouldn't alter text/description encoding return (self.__class__(self.encoding, self.language, self.short_description, fix2.encode(text_encoding[self.encoding])), fixes_performed) class ID3v22_PIC_Frame(Image): def __init__(self, image_format, picture_type, description, data): """fields are as follows: | image_format | a 3 byte image format, such as 'JPG' | | picture_type | a 1 byte field indicating front cover, etc. | | description | a description of the image as a C_string | | data | image data itself as a raw string | """ self.id = b'PIC' # add PIC-specific fields self.pic_format = image_format self.pic_type = picture_type self.pic_description = description # figure out image metrics from raw data try: metrics = Image.new(data, u'', 0) except InvalidImage: metrics = Image(data=data, mime_type=u'', width=0, height=0, color_depth=0, color_count=0, description=u'', type=0) # then initialize Image parent fields from metrics self.mime_type = metrics.mime_type self.width = metrics.width self.height = metrics.height self.color_depth = metrics.color_depth self.color_count = metrics.color_count self.data = data def copy(self): return ID3v22_PIC_Frame(self.pic_format, self.pic_type, self.pic_description, self.data) def __repr__(self): return "ID3v22_PIC_Frame({!r}, {!r}, {!r}, ...)".format( self.pic_format, self.pic_type, self.pic_description) def raw_info(self): """returns a human-readable version of this frame as unicode""" return u"PIC = ({}, {:d}\u00D7{:d}, {}, \"{}\") {:d} bytes".format( self.type_string(), self.width, self.height, self.mime_type, self.pic_description, len(self.data)) def type_string(self): return {0: u"Other", 1: u"32x32 pixels 'file icon' (PNG only)", 2: u"Other file icon", 3: u"Cover (front)", 4: u"Cover (back)", 5: u"Leaflet page", 6: u"Media (e.g. label side of CD)", 7: u"Lead artist/lead performer/soloist", 8: u"Artist / Performer", 9: u"Conductor", 10: u"Band / Orchestra", 11: u"Composer", 12: u"Lyricist / Text writer", 13: u"Recording Location", 14: u"During recording", 15: u"During performance", 16: u"Movie/Video screen capture", 17: u"A bright coloured fish", 18: u"Illustration", 19: u"Band/Artist logotype", 20: u"Publisher/Studio logotype"}.get(self.pic_type, u"Other") def __getattr__(self, attr): from audiotools import (FRONT_COVER, BACK_COVER, LEAFLET_PAGE, MEDIA, OTHER) if attr == 'type': return {3: FRONT_COVER, 4: BACK_COVER, 5: LEAFLET_PAGE, 6: MEDIA }.get(self.pic_type, OTHER) elif attr == 'description': return self.pic_description.__unicode__() else: raise AttributeError(attr) def __setattr__(self, attr, value): from audiotools import (FRONT_COVER, BACK_COVER, LEAFLET_PAGE, MEDIA, OTHER) if attr == 'type': Image.__setattr__(self, "pic_type", {FRONT_COVER: 3, BACK_COVER: 4, LEAFLET_PAGE: 5, MEDIA: 6}.get(value, 0)) elif attr == 'description': Image.__setattr__( self, "pic_description", C_string("latin-1" if is_latin_1(value) else "ucs2", value)) else: Image.__setattr__(self, attr, value) @classmethod def parse(cls, frame_id, frame_size, reader): (encoding, image_format, picture_type) = reader.parse("8u 3b 8u") description = C_string.parse({0: 'latin-1', 1: 'ucs2'}[encoding], reader) data = reader.read_bytes(frame_size - (5 + description.size())) return cls(image_format, picture_type, description, data) def build(self, writer): """writes this frame to the given BitstreamWriter not including its frame header""" writer.build("8u 3b 8u", ({'latin-1': 0, 'ucs2': 1}[self.pic_description.encoding], self.pic_format, self.pic_type)) self.pic_description.build(writer) writer.write_bytes(self.data) def size(self): """returns the size of this frame in bytes not including the frame header""" return (5 + self.pic_description.size() + len(self.data)) @classmethod def converted(cls, frame_id, image): if is_latin_1(image.description): description = C_string('latin-1', image.description) else: description = C_string('ucs2', image.description) return cls(image_format={u"image/png": b"PNG", u"image/jpeg": b"JPG", u"image/jpg": b"JPG", u"image/x-ms-bmp": b"BMP", u"image/gif": b"GIF", u"image/tiff": b"TIF"}.get(image.mime_type, b'UNK'), picture_type={0: 3, # front cover 1: 4, # back cover 2: 5, # leaflet page 3: 6, # media }.get(image.type, 0), # other description=description, data=image.data) def clean(self): """returns a cleaned ID3v22_PIC_Frame, or None if the frame should be removed entirely any fixes are appended to fixes_applied as unicode string""" # all the fields are derived from the image data # so there's no need to test for a mismatch # not sure if it's worth testing for bugs in the description # or format fields return (ID3v22_PIC_Frame(self.pic_format, self.pic_type, self.pic_description, self.data), []) class ID3v22Comment(MetaData): NAME = u'ID3v2.2' ATTRIBUTE_MAP = {'track_name': b'TT2', 'track_number': b'TRK', 'track_total': b'TRK', 'album_name': b'TAL', 'artist_name': b'TP1', 'performer_name': b'TP2', 'conductor_name': b'TP3', 'composer_name': b'TCM', 'media': b'TMT', 'ISRC': b'TRC', 'copyright': b'TCR', 'publisher': b'TPB', 'year': b'TYE', 'date': b'TRD', 'album_number': b'TPA', 'album_total': b'TPA', 'comment': b'COM', 'compilation': b'TCP'} RAW_FRAME = ID3v22_Frame TEXT_FRAME = ID3v22_T__Frame USER_TEXT_FRAME = ID3v22_TXX_Frame WEB_FRAME = ID3v22_W__Frame USER_WEB_FRAME = ID3v22_WXX_Frame COMMENT_FRAME = ID3v22_COM_Frame IMAGE_FRAME = ID3v22_PIC_Frame IMAGE_FRAME_ID = b'PIC' def __init__(self, frames, total_size=None): MetaData.__setattr__(self, "frames", frames[:]) MetaData.__setattr__(self, "total_size", total_size) def copy(self): return self.__class__([frame.copy() for frame in self]) def __repr__(self): return "ID3v22Comment({!r}, {!r})".format(self.frames, self.total_size) def __iter__(self): return iter(self.frames) def raw_info(self): """returns a human-readable version of this frame as unicode""" from os import linesep return linesep.join( ["{}:".format(self.NAME)] + [frame.raw_info() for frame in self]) @classmethod def parse(cls, reader): """given a BitstreamReader, returns a parsed ID3v22Comment""" (id3, major_version, minor_version, flags) = reader.parse("3b 8u 8u 8u") if id3 != b'ID3': raise ValueError("invalid ID3 header") elif major_version != 0x02: raise ValueError("invalid major version") elif minor_version != 0x00: raise ValueError("invalid minor version") total_size = remaining_size = decode_syncsafe32(reader.read(32)) frames = [] while remaining_size > 6: (frame_id, frame_size) = reader.parse("3b 24u") if frame_id == b"\x00" * 3: break elif frame_id == b'TXX': frames.append( cls.USER_TEXT_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) elif frame_id == b'WXX': frames.append( cls.USER_WEB_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) elif frame_id == b'COM': frames.append( cls.COMMENT_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) elif frame_id == b'PIC': frames.append( cls.IMAGE_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) elif frame_id.startswith(b'T'): frames.append( cls.TEXT_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) elif frame_id.startswith(b'W'): frames.append( cls.WEB_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) else: frames.append( cls.RAW_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) remaining_size -= (6 + frame_size) return cls(frames, total_size) def build(self, writer): """writes the complete ID3v22Comment data to the given BitstreamWriter""" tags_size = sum([6 + frame.size() for frame in self]) writer.build("3b 8u 8u 8u 32u", (b"ID3", 0x02, 0x00, 0x00, encode_syncsafe32( max(tags_size, (self.total_size if self.total_size is not None else 0))))) for frame in self: writer.build("3b 24u", (frame.id, frame.size())) frame.build(writer) # add buffer of NULL bytes if the total size of the tags # is less than the total size of the whole ID3v2.2 tag if (((self.total_size is not None) and ((self.total_size - tags_size) > 0))): writer.write_bytes(u"\x00" * (self.total_size - tags_size)) def size(self): """returns the total size of the ID3v22Comment, including its header""" return 10 + max(sum([6 + frame.size() for frame in self]), (self.total_size if self.total_size is not None else 0)) def __len__(self): return len(self.frames) def __getitem__(self, frame_id): frames = [frame for frame in self if (frame.id == frame_id)] if len(frames) > 0: return frames else: raise KeyError(frame_id) def __setitem__(self, frame_id, frames): new_frames = frames[:] updated_frames = [] for old_frame in self: if old_frame.id == frame_id: try: # replace current frame with newly set frame updated_frames.append(new_frames.pop(0)) except IndexError: # no more newly set frames, so remove current frame continue else: # passthrough unmatched frames updated_frames.append(old_frame) else: # append any leftover frames for new_frame in new_frames: updated_frames.append(new_frame) MetaData.__setattr__(self, "frames", updated_frames) def __delitem__(self, frame_id): updated_frames = [frame for frame in self if frame.id != frame_id] if len(updated_frames) < len(self): MetaData.__setattr__(self, "frames", updated_frames) else: raise KeyError(frame_id) def keys(self): return list({frame.id for frame in self}) def values(self): return [self[key] for key in self.keys()] def items(self): return [(key, self[key]) for key in self.keys()] def __getattr__(self, attr): if attr in self.ATTRIBUTE_MAP: try: frame = self[self.ATTRIBUTE_MAP[attr]][0] if attr in {'track_number', 'album_number'}: return frame.number() elif attr in {'track_total', 'album_total'}: return frame.total() elif attr == 'compilation': return frame.true() else: return frame.__unicode__() except KeyError: return None elif attr in self.FIELDS: return None else: return MetaData.__getattribute__(self, attr) def __setattr__(self, attr, value): def swap_number(unicode_value, new_number): import re return re.sub(r'\d+', __padded__(new_number), unicode_value, 1) def swap_slashed_number(unicode_value, new_number): if u"/" in unicode_value: (first, second) = unicode_value.split(u"/", 1) return u"/".join([first, swap_number(second, new_number)]) else: return u"/".join([unicode_value, __padded__(new_number)]) if attr in self.ATTRIBUTE_MAP: if value is not None: frame_id = self.ATTRIBUTE_MAP[attr] if attr in {'track_number', 'album_number'}: try: new_frame = self.TEXT_FRAME.converted( frame_id, swap_number(self[frame_id][0].__unicode__(), value)) except KeyError: # no frame found with track/album_number's ID, # so create a new frame for it # with the value padded appropriately new_frame = self.TEXT_FRAME.converted( frame_id, __padded__(value)) elif attr in {'track_total', 'album_total'}: try: new_frame = self.TEXT_FRAME.converted( frame_id, swap_slashed_number( self[frame_id][0].__unicode__(), value)) except KeyError: # no frame found with track_total's ID # so create a new frame for it # with the value padded appropriately new_frame = self.TEXT_FRAME.converted( frame_id, __number_pair__(None, value)) elif attr == 'comment': new_frame = self.COMMENT_FRAME.converted( frame_id, value) elif attr == 'compilation': new_frame = self.TEXT_FRAME.converted( frame_id, u"{:d}".format(1 if value else 0)) else: new_frame = self.TEXT_FRAME.converted( frame_id, u"{}".format(value)) try: self[frame_id] = [new_frame] + self[frame_id][1:] except KeyError: self[frame_id] = [new_frame] else: delattr(self, attr) elif attr in MetaData.FIELDS: pass else: MetaData.__setattr__(self, attr, value) def __delattr__(self, attr): import re def zero_number(unicode_value): return re.sub(r'\d+', u"0", unicode_value, 1) if attr in self.ATTRIBUTE_MAP: updated_frames = [] delete_frame_id = self.ATTRIBUTE_MAP[attr] for frame in self: if frame.id == delete_frame_id: if attr in {'track_number', 'album_number'}: current_value = frame.__unicode__() if u"/" in current_value: # if *_number field contains a slashed total, # replace unslashed portion with 0 updated_frames.append( self.TEXT_FRAME.converted( frame.id, zero_number(current_value))) else: # otherwise, remove *_number field continue elif attr in {'track_total', 'album_total'}: current_value = frame.__unicode__() if u"/" in current_value: (first, second) = current_value.split(u"/", 1) number = re.search(r'\d+', first) if ((number is not None) and (int(number.group(0)) != 0)): # field contains nonzero number part # so remove only slashed part updated_frames.append( self.TEXT_FRAME.converted( frame.id, first.rstrip())) else: # number part is zero, so remove entire tag continue else: # oddball tag with no slash # so pass it through unchanged updated_frames.append(frame) else: # handle the textual fields # which are simply deleted outright continue else: updated_frames.append(frame) MetaData.__setattr__(self, "frames", updated_frames) elif attr in MetaData.FIELDS: # ignore deleted attributes which are in MetaData # but we don't support pass else: MetaData.__delattr__(self, attr) def images(self): return [frame for frame in self if (frame.id == self.IMAGE_FRAME_ID)] def add_image(self, image): self.frames.append( self.IMAGE_FRAME.converted(self.IMAGE_FRAME_ID, image)) def delete_image(self, image): MetaData.__setattr__( self, "frames", [frame for frame in self if ((frame.id != self.IMAGE_FRAME_ID) or (frame != image))]) @classmethod def converted(cls, metadata): """converts a MetaData object to an ID3v2*Comment object""" if metadata is None: return None elif cls is metadata.__class__: return cls([frame.copy() for frame in metadata]) frames = [] for (attr, key) in cls.ATTRIBUTE_MAP.items(): value = getattr(metadata, attr) if (cls.FIELD_TYPES[attr] == type(u"")) and (value is not None): if attr == 'comment': frames.append(cls.COMMENT_FRAME.converted(key, value)) else: frames.append(cls.TEXT_FRAME.converted(key, value)) elif (attr == 'compilation') and (value is not None): frames.append( cls.TEXT_FRAME.converted( key, u"{:d}".format(1 if value else 0))) if (((metadata.track_number is not None) or (metadata.track_total is not None))): frames.append( cls.TEXT_FRAME.converted( cls.ATTRIBUTE_MAP["track_number"], __number_pair__(metadata.track_number, metadata.track_total))) if (((metadata.album_number is not None) or (metadata.album_total is not None))): frames.append( cls.TEXT_FRAME.converted( cls.ATTRIBUTE_MAP["album_number"], __number_pair__(metadata.album_number, metadata.album_total))) for image in metadata.images(): frames.append(cls.IMAGE_FRAME.converted(cls.IMAGE_FRAME_ID, image)) return cls(frames) def clean(self): """returns a new MetaData object that's been cleaned of problems""" new_frames = [] fixes_performed = [] for frame in self: (filtered_frame, frame_fixes) = frame.clean() if filtered_frame is not None: new_frames.append(filtered_frame) fixes_performed.extend(frame_fixes) return (self.__class__(new_frames, self.total_size), fixes_performed) def intersection(self, metadata): """given a MetaData-compatible object, returns a new MetaData object which contains all the matching fields and images of this object and 'metadata' """ def frame_present(frame): for other_frame in metadata: if frame == other_frame: return True else: return False if type(metadata) is type(self): return self.__class__([frame.copy() for frame in self if frame_present(frame)]) else: return MetaData.intersection(self, metadata) ############################################################ # ID3v2.3 Comment ############################################################ class ID3v23_Frame(ID3v22_Frame): def __repr__(self): return "ID3v23_Frame({!r}, {!r})".format(self.id, self.data) class ID3v23_T___Frame(ID3v22_T__Frame): NUMERICAL_IDS = (b'TRCK', b'TPOS') BOOLEAN_IDS = (b'TCMP',) def __repr__(self): return "ID3v23_T___Frame({!r}, {!r}, {!r})".format( self.id, self.encoding, self.data) class ID3v23_TXXX_Frame(ID3v22_TXX_Frame): def __init__(self, encoding, description, data): self.id = b'TXXX' self.encoding = encoding self.description = description self.data = data def __repr__(self): return "ID3v23_TXXX_Frame({!r}, {!r}, {!r})".format( self.encoding, self.description, self.data) class ID3v23_W___Frame(ID3v22_W__Frame): def __repr__(self): return "ID3v23_W___Frame({!r}, {!r})".format(self.id, self.data) class ID3v23_WXXX_Frame(ID3v22_WXX_Frame): def __init__(self, encoding, description, data): self.id = b'WXXX' self.encoding = encoding self.description = description self.data = data def __repr__(self): return "ID3v23_WXXX_Frame({!r}, {!r}, {!r})".format( self.encoding, self.description, self.data) class ID3v23_APIC_Frame(ID3v22_PIC_Frame): def __init__(self, mime_type, picture_type, description, data): """fields are as follows: | mime_type | a C_string of the image's MIME type | | picture_type | a 1 byte field indicating front cover, etc. | | description | a description of the image as a C_string | | data | image data itself as a raw string | """ self.id = b'APIC' # add APIC-specific fields self.pic_type = picture_type self.pic_description = description self.pic_mime_type = mime_type # figure out image metrics from raw data try: metrics = Image.new(data, u'', 0) except InvalidImage: metrics = Image(data=data, mime_type=u'', width=0, height=0, color_depth=0, color_count=0, description=u'', type=0) # then initialize Image parent fields from metrics self.width = metrics.width self.height = metrics.height self.color_depth = metrics.color_depth self.color_count = metrics.color_count self.data = data def copy(self): return self.__class__(self.pic_mime_type, self.pic_type, self.pic_description, self.data) def __repr__(self): return "ID3v23_APIC_Frame({!r}, {!r}, {!r}, ...)".format( self.pic_mime_type, self.pic_type, self.pic_description) def raw_info(self): """returns a human-readable version of this frame as unicode""" return u"APIC = ({}, {:d}\u00D7{:d}, {}, \"{}\") {:d} bytes".format( self.type_string(), self.width, self.height, self.pic_mime_type, self.pic_description, len(self.data)) def __getattr__(self, attr): from audiotools import (FRONT_COVER, BACK_COVER, LEAFLET_PAGE, MEDIA, OTHER) if attr == 'type': return {3: FRONT_COVER, 4: BACK_COVER, 5: LEAFLET_PAGE, 6: MEDIA }.get(self.pic_type, 4) # other elif attr == 'description': return self.pic_description.__unicode__() elif attr == 'mime_type': return self.pic_mime_type.__unicode__() else: raise AttributeError(attr) def __setattr__(self, attr, value): from audiotools import (FRONT_COVER, BACK_COVER, LEAFLET_PAGE, MEDIA, OTHER) if attr == 'type': Image.__setattr__(self, "pic_type", {FRONT_COVER: 3, BACK_COVER: 4, LEAFLET_PAGE: 5, MEDIA: 6}.get(value, 0)) elif attr == 'description': Image.__setattr__( self, "pic_description", C_string("latin-1" if is_latin_1(value) else "ucs2", value)) elif attr == 'mime_type': Image.__setattr__(self, "pic_mime_type", C_string('ascii', value)) else: Image.__setattr__(self, attr, value) @classmethod def parse(cls, frame_id, frame_size, reader): """parses this frame from the given BitstreamReader""" encoding = reader.read(8) mime_type = C_string.parse('ascii', reader) picture_type = reader.read(8) description = C_string.parse({0: 'latin-1', 1: 'ucs2'}[encoding], reader) data = reader.read_bytes(frame_size - (1 + mime_type.size() + 1 + description.size())) return cls(mime_type, picture_type, description, data) def build(self, writer): """writes this frame to the given BitstreamWriter not including its frame header""" writer.write(8, {'latin-1': 0, 'ucs2': 1}[self.pic_description.encoding]) self.pic_mime_type.build(writer) writer.write(8, self.pic_type) self.pic_description.build(writer) writer.write_bytes(self.data) def size(self): """returns the size of this frame in bytes not including the frame header""" return (1 + self.pic_mime_type.size() + 1 + self.pic_description.size() + len(self.data)) @classmethod def converted(cls, frame_id, image): if is_latin_1(image.description): description = C_string('latin-1', image.description) else: description = C_string('ucs2', image.description) return cls(mime_type=C_string('ascii', image.mime_type), picture_type={0: 3, # front cover 1: 4, # back cover 2: 5, # leaflet page 3: 6, # media }.get(image.type, 0), # other description=description, data=image.data) def clean(self): """returns a cleaned ID3v23_APIC_Frame, or None if the frame should be removed entirely any fixes are appended to fixes_applied as unicode string""" actual_mime_type = Image.new(self.data, u"", 0).mime_type if self.pic_mime_type.__unicode__() != actual_mime_type: from audiotools.text import (CLEAN_FIX_IMAGE_FIELDS) return (ID3v23_APIC_Frame( C_string('ascii', actual_mime_type), self.pic_type, self.pic_description, self.data), [CLEAN_FIX_IMAGE_FIELDS]) else: return (ID3v23_APIC_Frame( self.pic_mime_type, self.pic_type, self.pic_description, self.data), []) class ID3v23_COMM_Frame(ID3v22_COM_Frame): def __init__(self, encoding, language, short_description, data): """fields are as follows: | encoding | 1 byte int of the comment's text encoding | | language | 3 byte string of the comment's language | | short_description | C_string of a short description | | data | plain string of the comment data itself | """ self.id = b"COMM" self.encoding = encoding self.language = language self.short_description = short_description self.data = data def __repr__(self): return "ID3v23_COMM_Frame({!r}, {!r}, {!r}, {!r})".format( self.encoding, self.language, self.short_description, self.data) def raw_info(self): """returns a human-readable version of this frame as unicode""" return u"COMM = ({}, {}, \"{}\") {}".format( {0: u'Latin-1', 1: 'UCS-2'}[self.encoding], self.language.decode('ascii', 'replace'), self.short_description, self.data.decode({0: 'latin-1', 1: 'ucs2'}[self.encoding])) class ID3v23Comment(ID3v22Comment): NAME = u'ID3v2.3' ATTRIBUTE_MAP = {'track_name': b'TIT2', 'track_number': b'TRCK', 'track_total': b'TRCK', 'album_name': b'TALB', 'artist_name': b'TPE1', 'performer_name': b'TPE2', 'composer_name': b'TCOM', 'conductor_name': b'TPE3', 'media': b'TMED', 'ISRC': b'TSRC', 'copyright': b'TCOP', 'publisher': b'TPUB', 'year': b'TYER', 'date': b'TRDA', 'album_number': b'TPOS', 'album_total': b'TPOS', 'comment': b'COMM', 'compilation': b'TCMP'} RAW_FRAME = ID3v23_Frame TEXT_FRAME = ID3v23_T___Frame WEB_FRAME = ID3v23_W___Frame USER_TEXT_FRAME = ID3v23_TXXX_Frame USER_WEB_FRAME = ID3v23_WXXX_Frame COMMENT_FRAME = ID3v23_COMM_Frame IMAGE_FRAME = ID3v23_APIC_Frame IMAGE_FRAME_ID = b'APIC' ITUNES_COMPILATION_ID = b'TCMP' def __repr__(self): return "ID3v23Comment({!r}, {!r})".format(self.frames, self.total_size) @classmethod def parse(cls, reader): """given a BitstreamReader, returns a parsed ID3v23Comment""" (id3, major_version, minor_version, flags) = reader.parse("3b 8u 8u 8u") if id3 != b'ID3': raise ValueError("invalid ID3 header") elif major_version != 0x03: raise ValueError("invalid major version") elif minor_version != 0x00: raise ValueError("invalid minor version") total_size = remaining_size = decode_syncsafe32(reader.read(32)) frames = [] while remaining_size > 10: (frame_id, frame_size, frame_flags) = reader.parse("4b 32u 16u") if frame_id == b"\x00" * 4: break elif frame_id == b'TXXX': frames.append( cls.USER_TEXT_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) elif frame_id == b'WXXX': frames.append( cls.USER_WEB_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) elif frame_id == b'COMM': frames.append( cls.COMMENT_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) elif frame_id == b'APIC': frames.append( cls.IMAGE_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) elif frame_id.startswith(b'T'): frames.append( cls.TEXT_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) elif frame_id.startswith(b'W'): frames.append( cls.WEB_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) else: frames.append( cls.RAW_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) remaining_size -= (10 + frame_size) return cls(frames, total_size) def build(self, writer): """writes the complete ID3v23Comment data to the given BitstreamWriter""" tags_size = sum([10 + frame.size() for frame in self]) writer.build("3b 8u 8u 8u 32u", (b"ID3", 0x03, 0x00, 0x00, encode_syncsafe32( max(tags_size, (self.total_size if self.total_size is not None else 0))))) for frame in self: writer.build("4b 32u 16u", (frame.id, frame.size(), 0)) frame.build(writer) # add buffer of NULL bytes if the total size of the tags # is less than the total size of the whole ID3v2.3 tag if (((self.total_size is not None) and ((self.total_size - tags_size) > 0))): writer.write_bytes(b"\x00" * (self.total_size - tags_size)) def size(self): """returns the total size of the ID3v23Comment, including its header""" return 10 + max(sum([10 + frame.size() for frame in self]), (self.total_size if self.total_size is not None else 0)) ############################################################ # ID3v2.4 Comment ############################################################ class ID3v24_Frame(ID3v23_Frame): def __repr__(self): return "ID3v24_Frame({!r}, {!r})".format(self.id, self.data) class ID3v24_T___Frame(ID3v23_T___Frame): def __init__(self, frame_id, encoding, data): assert((encoding == 0) or (encoding == 1) or (encoding == 2) or (encoding == 3)) self.id = frame_id self.encoding = encoding self.data = data def __repr__(self): return "ID3v24_T___Frame({!r}, {!r}, {!r})".format( self.id, self.encoding, self.data) def __unicode__(self): return self.data.decode( {0: u"latin-1", 1: u"utf-16", 2: u"utf-16BE", 3: u"utf-8"}[self.encoding], 'replace').split(u"\x00", 1)[0] def raw_info(self): """returns a human-readable version of this frame as unicode""" return u"{} = ({}) {}".format(self.id.decode('ascii'), {0: u"Latin-1", 1: u"UTF-16", 2: u"UTF-16BE", 3: u"UTF-8"}[self.encoding], self.__unicode__()) @classmethod def converted(cls, frame_id, unicode_string): """given a unicode string, returns a text frame""" if is_latin_1(unicode_string): return cls(frame_id, 0, unicode_string.encode('latin-1')) else: return cls(frame_id, 3, unicode_string.encode('utf-8')) class ID3v24_TXXX_Frame(ID3v23_TXXX_Frame): def __repr__(self): return "ID3v24_TXXX_Frame({!r}, {!r}, {!r})".format( self.encoding, self.description, self.data) def raw_info(self): """returns a human-readable version of this frame as unicode""" return u"{} = ({}, \"{}\") {}".format( self.id.decode('ascii', 'replace'), {0: u"Latin-1", 1: u"UTF-16", 2: u"UTF-16BE", 3: u"UTF-8"}[self.encoding], self.description, self.__unicode__()) def __unicode__(self): return self.data.decode( {0: u"latin-1", 1: u"utf-16", 2: u"utf-16BE", 3: u"utf-8"}[self.encoding], 'replace').split(u"\x00", 1)[0] @classmethod def parse(cls, frame_id, frame_size, reader): """given a frame_id string, frame_size int and BitstreamReader of the remaining frame data, returns a parsed text frame""" encoding = reader.read(8) description = C_string.parse({0: "latin-1", 1: "utf-16", 2: "utf-16be", 3: "utf-8"}[encoding], reader) data = reader.read_bytes(frame_size - 1 - description.size()) return cls(encoding, description, data) class ID3v24_APIC_Frame(ID3v23_APIC_Frame): def __repr__(self): return "ID3v24_APIC_Frame({!r}, {!r}, {!r}, ...)".format( self.pic_mime_type, self.pic_type, self.pic_description) def __setattr__(self, attr, value): from audiotools import (FRONT_COVER, BACK_COVER, LEAFLET_PAGE, MEDIA, OTHER) if attr == 'type': Image.__setattr__( self, "pic_type", {FRONT_COVER: 3, BACK_COVER: 4, LEAFLET_PAGE: 5, MEDIA: 6}.get(value, 0)) elif attr == 'description': Image.__setattr__( self, "pic_description", C_string("latin-1" if is_latin_1(value) else "utf-8", value)) elif attr == 'mime_type': Image.__setattr__(self, "pic_mime_type", C_string('ascii', value)) else: Image.__setattr__(self, attr, value) @classmethod def parse(cls, frame_id, frame_size, reader): """parses this frame from the given BitstreamReader""" encoding = reader.read(8) mime_type = C_string.parse('ascii', reader) picture_type = reader.read(8) description = C_string.parse({0: 'latin-1', 1: 'utf-16', 2: 'utf-16be', 3: 'utf-8'}[encoding], reader) data = reader.read_bytes(frame_size - (1 + mime_type.size() + 1 + description.size())) return cls(mime_type, picture_type, description, data) def build(self, writer): """writes this frame to the given BitstreamWriter not including its frame header""" writer.write(8, {'latin-1': 0, 'utf-16': 1, 'utf-16be': 2, 'utf-8': 3}[self.pic_description.encoding]) self.pic_mime_type.build(writer) writer.write(8, self.pic_type) self.pic_description.build(writer) writer.write_bytes(self.data) @classmethod def converted(cls, frame_id, image): if is_latin_1(image.description): description = C_string('latin-1', image.description) else: description = C_string('utf-8', image.description) return cls(mime_type=C_string('ascii', image.mime_type), picture_type={0: 3, # front cover 1: 4, # back cover 2: 5, # leaflet page 3: 6, # media }.get(image.type, 0), # other description=description, data=image.data) def clean(self): """returns a cleaned ID3v24_APIC_Frame, or None if the frame should be removed entirely any fixes are appended to fixes_applied as unicode string""" actual_mime_type = Image.new(self.data, u"", 0).mime_type if self.pic_mime_type.__unicode__() != actual_mime_type: from audiotools.text import (CLEAN_FIX_IMAGE_FIELDS) return (ID3v24_APIC_Frame( C_string('ascii', actual_mime_type), self.pic_type, self.pic_description, self.data), [CLEAN_FIX_IMAGE_FIELDS]) else: return (ID3v24_APIC_Frame( self.pic_mime_type, self.pic_type, self.pic_description, self.data), []) class ID3v24_W___Frame(ID3v23_W___Frame): def __repr__(self): return "ID3v24_W___Frame({!r}, {!r})".format(self.id, self.data) class ID3v24_WXXX_Frame(ID3v23_WXXX_Frame): def __repr__(self): return "ID3v24_WXXX_Frame({!r}, {!r}, {!r})".format( self.encoding, self.description, self.data) def raw_info(self): """returns a human-readable version of this frame as unicode""" return u"{} = ({}, \"{}\") {}".format( self.id.decode('ascii', 'replace'), {0: u'Latin-1', 1: u'UTF-16', 2: u'UTF-16BE', 3: u'UTF-8'}[self.encoding], self.description, self.data.decode('ascii', 'replace')) @classmethod def parse(cls, frame_id, frame_size, reader): """given a frame_id string, frame_size int and BitstreamReader of the remaining frame data, returns a parsed text frame""" encoding = reader.read(8) description = C_string.parse({0: 'latin-1', 1: 'utf-16', 2: 'utf-16be', 3: 'utf-8'}[encoding], reader) data = reader.read_bytes(frame_size - 1 - description.size()) return cls(encoding, description, data) class ID3v24_COMM_Frame(ID3v23_COMM_Frame): def __repr__(self): return "ID3v24_COMM_Frame({!r}, {!r}, {!r}, {!r})".format( self.encoding, self.language, self.short_description, self.data) def __unicode__(self): return self.data.decode({0: 'latin-1', 1: 'utf-16', 2: 'utf-16be', 3: 'utf-8'}[self.encoding], 'replace') def raw_info(self): """returns a human-readable version of this frame as unicode""" return u"COMM = ({}, {}, \"{}\") {}".format( {0: u'Latin-1', 1: u'UTF-16', 2: u'UTF-16BE', 3: u'UTF-8'}[self.encoding], self.language.decode('ascii', 'replace'), self.short_description, self.data.decode({0: 'latin-1', 1: 'utf-16', 2: 'utf-16be', 3: 'utf-8'}[self.encoding])) @classmethod def parse(cls, frame_id, frame_size, reader): """given a frame_id string, frame_size int and BitstreamReader of the remaining frame data, returns a parsed ID3v22_COM_Frame""" (encoding, language) = reader.parse("8u 3b") short_description = C_string.parse({0: 'latin-1', 1: 'utf-16', 2: 'utf-16be', 3: 'utf-8'}[encoding], reader) data = reader.read_bytes(frame_size - (4 + short_description.size())) return cls(encoding, language, short_description, data) @classmethod def converted(cls, frame_id, unicode_string): if is_latin_1(unicode_string): return cls(0, b"eng", C_string("latin-1", u""), unicode_string.encode('latin-1')) else: return cls(3, b"eng", C_string("utf-8", u""), unicode_string.encode('utf-8')) def clean(self): """returns a cleaned frame of the same class or None if the frame should be omitted fix text will be appended to fixes_performed, if necessary""" from audiotools.text import (CLEAN_REMOVE_EMPTY_TAG, CLEAN_REMOVE_TRAILING_WHITESPACE, CLEAN_REMOVE_LEADING_WHITESPACE) fixes_performed = [] field = self.id.decode('ascii') text_encoding = {0: 'latin-1', 1: 'utf-16', 2: 'utf-16be', 3: 'utf-8'} value = self.data.decode(text_encoding[self.encoding], 'replace') # check for an empty tag if len(value.strip()) == 0: return (None, [CLEAN_REMOVE_EMPTY_TAG.format(field)]) # check trailing whitespace fix1 = value.rstrip() if fix1 != value: fixes_performed.append( CLEAN_REMOVE_TRAILING_WHITESPACE.format(field)) # check leading whitespace fix2 = fix1.lstrip() if fix2 != fix1: fixes_performed.append( CLEAN_REMOVE_LEADING_WHITESPACE.format(field)) # stripping whitespace shouldn't alter text/description encoding return (self.__class__(self.encoding, self.language, self.short_description, fix2.encode(text_encoding[self.encoding])), fixes_performed) class ID3v24Comment(ID3v23Comment): NAME = u'ID3v2.4' RAW_FRAME = ID3v24_Frame TEXT_FRAME = ID3v24_T___Frame USER_TEXT_FRAME = ID3v24_TXXX_Frame WEB_FRAME = ID3v24_W___Frame USER_WEB_FRAME = ID3v24_WXXX_Frame COMMENT_FRAME = ID3v24_COMM_Frame IMAGE_FRAME = ID3v24_APIC_Frame IMAGE_FRAME_ID = b'APIC' ITUNES_COMPILATION_ID = b'TCMP' def __repr__(self): return "ID3v24Comment({!r}, {!r})".format(self.frames, self.total_size) @classmethod def parse(cls, reader): """given a BitstreamReader, returns a parsed ID3v24Comment""" (id3, major_version, minor_version, flags) = reader.parse("3b 8u 8u 8u") if id3 != b'ID3': raise ValueError("invalid ID3 header") elif major_version != 0x04: raise ValueError("invalid major version") elif minor_version != 0x00: raise ValueError("invalid minor version") total_size = remaining_size = decode_syncsafe32(reader.read(32)) frames = [] while remaining_size > 10: frame_id = reader.read_bytes(4) frame_size = decode_syncsafe32(reader.read(32)) flags = reader.read(16) if frame_id == b"\x00" * 4: break elif frame_id == b'TXXX': frames.append( cls.USER_TEXT_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) elif frame_id == b'WXXX': frames.append( cls.USER_WEB_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) elif frame_id == b'COMM': frames.append( cls.COMMENT_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) elif frame_id == b'APIC': frames.append( cls.IMAGE_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) elif frame_id.startswith(b'T'): frames.append( cls.TEXT_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) elif frame_id.startswith(b'W'): frames.append( cls.WEB_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) else: frames.append( cls.RAW_FRAME.parse( frame_id, frame_size, reader.substream(frame_size))) remaining_size -= (10 + frame_size) return cls(frames, total_size) def build(self, writer): """writes the complete ID3v24Comment data to the given BitstreamWriter""" tags_size = sum([10 + frame.size() for frame in self]) writer.build("3b 8u 8u 8u 32u", (b"ID3", 0x04, 0x00, 0x00, encode_syncsafe32( max(tags_size, (self.total_size if self.total_size is not None else 0))))) for frame in self: writer.write_bytes(frame.id) writer.write(32, encode_syncsafe32(frame.size())) writer.write(16, 0) frame.build(writer) # add buffer of NULL bytes if the total size of the tags # is less than the total size of the whole ID3v2.2 tag if (((self.total_size is not None) and ((self.total_size - tags_size) > 0))): writer.write_bytes(b"\x00" * (self.total_size - tags_size)) def size(self): """returns the total size of the ID3v24Comment, including its header""" return 10 + max(sum([10 + frame.size() for frame in self]), (self.total_size if self.total_size is not None else 0)) ID3v2Comment = ID3v22Comment class ID3CommentPair(MetaData): """a pair of ID3v2/ID3v1 comments these can be manipulated as a set""" def __init__(self, id3v2_comment, id3v1_comment): """id3v2 and id3v1 are ID3v2Comment and ID3v1Comment objects or None values in ID3v2 take precendence over ID3v1, if present""" MetaData.__setattr__(self, "id3v2", id3v2_comment) MetaData.__setattr__(self, "id3v1", id3v1_comment) if self.id3v2 is not None: base_comment = self.id3v2 elif self.id3v1 is not None: base_comment = self.id3v1 else: raise ValueError("ID3v2 and ID3v1 cannot both be blank") def __repr__(self): return "ID3CommentPair({!r}, {!r})".format(self.id3v2, self.id3v1) def __getattr__(self, attr): assert((self.id3v2 is not None) or (self.id3v1 is not None)) if attr in self.FIELDS: if self.id3v2 is not None: # ID3v2 takes precedence over ID3v1 field = getattr(self.id3v2, attr) if field is not None: return field elif self.id3v1 is not None: return getattr(self.id3v1, attr) else: return None elif self.id3v1 is not None: return getattr(self.id3v1, attr) else: return MetaData.__getattribute__(self, attr) def __setattr__(self, attr, value): assert((self.id3v2 is not None) or (self.id3v1 is not None)) if attr in self.FIELDS: if self.id3v2 is not None: setattr(self.id3v2, attr, value) if self.id3v1 is not None: setattr(self.id3v1, attr, value) else: MetaData.__setattr__(self, attr, value) def __delattr__(self, attr): assert((self.id3v2 is not None) or (self.id3v1 is not None)) if attr in self.FIELDS: if self.id3v2 is not None: delattr(self.id3v2, attr) if self.id3v1 is not None: delattr(self.id3v1, attr) else: MetaData.__delattr__(self, attr) @classmethod def converted(cls, metadata, id3v2_class=ID3v23Comment, id3v1_class=ID3v1Comment): """takes a MetaData object and returns an ID3CommentPair object""" if metadata is None: return None elif isinstance(metadata, ID3CommentPair): return ID3CommentPair( metadata.id3v2.__class__.converted(metadata.id3v2), metadata.id3v1.__class__.converted(metadata.id3v1)) elif isinstance(metadata, ID3v2Comment): return ID3CommentPair(metadata, id3v1_class.converted(metadata)) else: return ID3CommentPair( id3v2_class.converted(metadata), id3v1_class.converted(metadata)) def raw_info(self): """returns a human-readable version of this metadata pair as a unicode string""" if (self.id3v2 is not None) and (self.id3v1 is not None): # both comments present from os import linesep return (self.id3v2.raw_info() + linesep * 2 + self.id3v1.raw_info()) elif self.id3v2 is not None: # only ID3v2 return self.id3v2.raw_info() elif self.id3v1 is not None: # only ID3v1 return self.id3v1.raw_info() else: return u'' # ImageMetaData passthroughs def images(self): """returns a list of embedded Image objects""" if self.id3v2 is not None: return self.id3v2.images() else: return [] def add_image(self, image): """embeds an Image object in this metadata""" if self.id3v2 is not None: self.id3v2.add_image(image) def delete_image(self, image): """deletes an Image object from this metadata""" if self.id3v2 is not None: self.id3v2.delete_image(image) @classmethod def supports_images(cls): """returns True""" return True def clean(self): if self.id3v2 is not None: (new_id3v2, id3v2_fixes) = self.id3v2.clean() else: new_id3v2 = None id3v2_fixes = [] if self.id3v1 is not None: (new_id3v1, id3v1_fixes) = self.id3v1.clean() else: new_id3v1 = None id3v1_fixes = [] return (ID3CommentPair(new_id3v2, new_id3v1), id3v2_fixes + id3v1_fixes) def intersection(self, metadata): """given a MetaData-compatible object, returns a new MetaData object which contains all the matching fields and images of this object and 'metadata' """ if type(metadata) is ID3CommentPair: return ID3CommentPair( self.id3v2.intersection(metadata.id3v2) if ((self.id3v2 is not None) and (metadata.id3v2 is not None)) else None, self.id3v1.intersection(metadata.id3v2) if ((self.id3v1 is not None) and (metadata.id3v1 is not None)) else None) else: return MetaData.intersection(self, metadata) ================================================ FILE: audiotools/id3v1.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools import MetaData class ID3v1Comment(MetaData): """a complete ID3v1.1 tag""" ID3v1_FIELDS = {"track_name": "__track_name__", "artist_name": "__artist_name__", "album_name": "__album_name__", "year": "__year__", "comment": "__comment__", "track_number": "__track_number__"} FIELD_LENGTHS = {"track_name": 30, "artist_name": 30, "album_name": 30, "year": 4, "comment": 28} def __init__(self, track_name=u"", artist_name=u"", album_name=u"", year=u"", comment=u"", track_number=0, genre=0): """fields are as follows: | field | length | |--------------+--------| | track_name | 30 | | artist_name | 30 | | album_name | 30 | | year | 4 | | comment | 28 | | track_number | 1 | | genre | 1 | |--------------+--------| track_name, artist_name, album_name, year and comment are unicode strings track_number and genre are integers """ if len(track_name) > 30: raise ValueError("track_name cannot be longer than 30 characters") if len(artist_name) > 30: raise ValueError("artist_name cannot be longer than 30 characters") if len(album_name) > 30: raise ValueError("album_name cannot be longer than 30 characters") if len(year) > 4: raise ValueError("year cannot be longer than 4 characters") if len(comment) > 28: raise ValueError("comment cannot be longer than 28 characters") MetaData.__setattr__(self, "__track_name__", track_name) MetaData.__setattr__(self, "__artist_name__", artist_name) MetaData.__setattr__(self, "__album_name__", album_name) MetaData.__setattr__(self, "__year__", year) MetaData.__setattr__(self, "__comment__", comment) MetaData.__setattr__(self, "__track_number__", track_number) MetaData.__setattr__(self, "__genre__", genre) def __repr__(self): return "ID3v1Comment({!r},{!r},{!r},{!r},{!r},{!r},{!r})".format( self.__track_name__, self.__artist_name__, self.__album_name__, self.__year__, self.__comment__, self.__track_number__, self.__genre__) def __getattr__(self, attr): if attr == "track_number": number = self.__track_number__ if number > 0: return number else: return None elif attr in self.ID3v1_FIELDS: value = getattr(self, self.ID3v1_FIELDS[attr]) if len(value) > 0: return value else: return None elif attr in self.FIELDS: return None else: return MetaData.__getattribute__(self, attr) def __setattr__(self, attr, value): if attr == "track_number": MetaData.__setattr__( self, "__track_number__", min(0 if (value is None) else int(value), 0xFF)) elif attr in self.FIELD_LENGTHS: if value is None: delattr(self, attr) else: # all are text fields MetaData.__setattr__( self, self.ID3v1_FIELDS[attr], value[0:self.FIELD_LENGTHS[attr]]) elif attr in self.FIELDS: # field not supported by ID3v1Comment, so ignore it pass else: MetaData.__setattr__(self, attr, value) def __delattr__(self, attr): if attr == "track_number": MetaData.__setattr__(self, "__track_number__", 0) elif attr in self.FIELD_LENGTHS: MetaData.__setattr__(self, self.ID3v1_FIELDS[attr], u"") elif attr in self.FIELDS: # field not supported by ID3v1Comment, so ignore it pass else: MetaData.__delattr__(self, attr) def raw_info(self): """returns a human-readable version of this metadata as a unicode string""" from os import linesep return linesep.join( [u"ID3v1.1:"] + [u"{} = {}".format(label, getattr(self, attr)) for (label, attr) in [(u" track name", "track_name"), (u" artist name", "artist_name"), (u" album name", "album_name"), (u" year", "year"), (u" comment", "comment"), (u"track number", "track_number")] if (getattr(self, attr) is not None)] + [u" genre = {:d}".format(self.__genre__)]) @classmethod def parse(cls, mp3_file): """given an MP3 file, returns an ID3v1Comment raises ValueError if the comment is invalid""" from audiotools.bitstream import parse def decode_string(s): return s.rstrip(b"\x00").decode("ascii", "replace") mp3_file.seek(-128, 2) (tag, track_name, artist_name, album_name, year, comment, track_number, genre) = parse("3b 30b 30b 30b 4b 28b 8p 8u 8u", False, mp3_file.read(128)) if tag != b'TAG': raise ValueError(u"invalid ID3v1 tag") return ID3v1Comment(track_name=decode_string(track_name), artist_name=decode_string(artist_name), album_name=decode_string(album_name), year=decode_string(year), comment=decode_string(comment), track_number=track_number, genre=genre) def build(self, mp3_file): """given an MP3 file positioned at the file's end, generate a tag""" from audiotools.bitstream import build def encode_string(u, max_chars): s = u.encode("ascii", "replace") if len(s) >= max_chars: return s[0:max_chars] else: return s + b"\x00" * (max_chars - len(s)) mp3_file.write( build("3b 30b 30b 30b 4b 28b 8p 8u 8u", False, (b"TAG", encode_string(self.__track_name__, 30), encode_string(self.__artist_name__, 30), encode_string(self.__album_name__, 30), encode_string(self.__year__, 4), encode_string(self.__comment__, 28), self.__track_number__, self.__genre__))) @classmethod def supports_images(cls): """returns False""" return False @classmethod def converted(cls, metadata): """converts a MetaData object to an ID3v1Comment object""" if metadata is None: return None elif isinstance(metadata, ID3v1Comment): # duplicate all fields as-is return ID3v1Comment(track_name=metadata.__track_name__, artist_name=metadata.__artist_name__, album_name=metadata.__album_name__, year=metadata.__year__, comment=metadata.__comment__, track_number=metadata.__track_number__, genre=metadata.__genre__) else: # convert fields using setattr id3v1 = ID3v1Comment() for attr in ["track_name", "artist_name", "album_name", "year", "comment", "track_number"]: setattr(id3v1, attr, getattr(metadata, attr)) return id3v1 def images(self): """returns an empty list of Image objects""" return [] def clean(self): import sys """returns a new ID3v1Comment object that's been cleaned of problems""" from audiotools.text import (CLEAN_REMOVE_TRAILING_WHITESPACE, CLEAN_REMOVE_LEADING_WHITESPACE) fixes_performed = [] fields = {} for (attr, name) in [("track_name", u"title"), ("artist_name", u"artist"), ("album_name", u"album"), ("year", u"year"), ("comment", u"comment")]: # strip out trailing NULL bytes initial_value = getattr(self, attr) if initial_value is not None: fix1 = initial_value.rstrip() if fix1 != initial_value: fixes_performed.append( CLEAN_REMOVE_TRAILING_WHITESPACE.format(name)) fix2 = fix1.lstrip() if fix2 != fix1: fixes_performed.append( CLEAN_REMOVE_LEADING_WHITESPACE.format(name)) # restore trailing NULL bytes fields[attr] = fix2 # copy non-text fields as-is return (ID3v1Comment(track_number=self.__track_number__, genre=self.__genre__, **fields), fixes_performed) def intersection(self, metadata): """given a MetaData-compatible object, returns a new MetaData object which contains all the matching fields and images of this object and 'metadata' """ if type(metadata) is ID3v1Comment: return ID3v1Comment( genre=(self.__genre__ if self.__genre__ == metadata.__genre__ else 0), **{arg: getattr(self, field) for arg, field in ID3v1Comment.ID3v1_FIELDS.items() if getattr(self, field) == getattr(metadata, field)}) else: return MetaData.intersection(self, metadata) ================================================ FILE: audiotools/image.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools.bitstream import BitstreamReader, format_size from audiotools import InvalidImage def image_metrics(file_data): """returns an ImageMetrics subclass from a string of file data raises InvalidImage if there is an error parsing the file or its type is unknown""" if file_data[0:3] == b"\xff\xd8\xff": return __JPEG__.parse(file_data) elif file_data[0:8] == b'\x89\x50\x4E\x47\x0D\x0A\x1A\x0A': return __PNG__.parse(file_data) elif file_data[0:3] == b'GIF': return __GIF__.parse(file_data) elif file_data[0:2] == b'BM': return __BMP__.parse(file_data) elif (file_data[0:2] == b'II') or (file_data[0:2] == b'MM'): return __TIFF__.parse(file_data) else: from audiotools.text import ERR_IMAGE_UNKNOWN_TYPE raise InvalidImage(ERR_IMAGE_UNKNOWN_TYPE) ####################### # JPEG ####################### class ImageMetrics: """a container for image data""" def __init__(self, width, height, bits_per_pixel, color_count, mime_type): """fields are as follows: width - image width as an integer number of pixels height - image height as an integer number of pixels bits_per_pixel - the number of bits per pixel as an integer color_count - for palette-based images, the total number of colors mime_type - the image's MIME type, as a unicode string all of the ImageMetrics subclasses implement these fields in addition, they all implement a parse() classmethod used to parse binary string data and return something imageMetrics compatible """ self.width = width self.height = height self.bits_per_pixel = bits_per_pixel self.color_count = color_count self.mime_type = mime_type def __repr__(self): return "ImageMetrics({!r},{!r},{!r},{!r},{!r})".format( self.width, self.height, self.bits_per_pixel, self.color_count, self.mime_type) @classmethod def parse(cls, file_data): raise NotImplementedError() class InvalidJPEG(InvalidImage): """raised if a JPEG cannot be parsed correctly""" pass class __JPEG__(ImageMetrics): def __init__(self, width, height, bits_per_pixel): ImageMetrics.__init__(self, width, height, bits_per_pixel, 0, u'image/jpeg') @classmethod def parse(cls, file_data): def segments(reader): if reader.read(8) != 0xFF: from audiotools.text import ERR_IMAGE_INVALID_JPEG_MARKER raise InvalidJPEG(ERR_IMAGE_INVALID_JPEG_MARKER) segment_type = reader.read(8) while segment_type != 0xDA: if segment_type not in {0xD8, 0xD9}: yield (segment_type, reader.substream(reader.read(16) - 2)) else: yield (segment_type, None) if reader.read(8) != 0xFF: from audiotools.text import ERR_IMAGE_INVALID_JPEG_MARKER raise InvalidJPEG(ERR_IMAGE_INVALID_JPEG_MARKER) segment_type = reader.read(8) try: for (segment_type, segment_data) in segments(BitstreamReader(file_data, False)): if (segment_type in {0xC0, 0xC1, 0xC2, 0xC3, 0xC5, 0XC5, 0xC6, 0xC7, 0xC9, 0xCA, 0xCB, 0xCD, 0xCE, 0xCF}): # start of frame (data_precision, image_height, image_width, components) = segment_data.parse("8u 16u 16u 8u") return __JPEG__(width=image_width, height=image_height, bits_per_pixel=data_precision * components) except IOError: from audiotools.text import ERR_IMAGE_IOERROR_JPEG raise InvalidJPEG(ERR_IMAGE_IOERROR_JPEG) ####################### # PNG ####################### class InvalidPNG(InvalidImage): """raised if a PNG cannot be parsed correctly""" pass class __PNG__(ImageMetrics): def __init__(self, width, height, bits_per_pixel, color_count): ImageMetrics.__init__(self, width, height, bits_per_pixel, color_count, u'image/png') @classmethod def parse(cls, file_data): def chunks(reader): if reader.read_bytes(8) != b'\x89\x50\x4E\x47\x0D\x0A\x1A\x0A': from audiotools.text import ERR_IMAGE_INVALID_PNG raise InvalidPNG(ERR_IMAGE_INVALID_PNG) (chunk_length, chunk_type) = reader.parse("32u 4b") while chunk_type != b'IEND': yield (chunk_type, chunk_length, reader.substream(chunk_length)) chunk_crc = reader.read(32) (chunk_length, chunk_type) = reader.parse("32u 4b") ihdr = None plte_length = 0 try: for (chunk_type, chunk_length, chunk_data) in chunks(BitstreamReader(file_data, False)): if chunk_type == b'IHDR': ihdr = chunk_data elif chunk_type == b'PLTE': plte_length = chunk_length if ihdr is None: from audiotools.text import ERR_IMAGE_INVALID_PNG raise InvalidPNG(ERR_IMAGE_INVALID_PNG) (width, height, bit_depth, color_type, compression_method, filter_method, interlace_method) = ihdr.parse("32u 32u 8u 8u 8u 8u 8u") except IOError: from audiotools.text import ERR_IMAGE_IOERROR_PNG raise InvalidPNG(ERR_IMAGE_IOERROR_PNG) if color_type == 0: # grayscale return cls(width=width, height=height, bits_per_pixel=bit_depth, color_count=0) elif color_type == 2: # RGB return cls(width=width, height=height, bits_per_pixel=bit_depth * 3, color_count=0) elif color_type == 3: # palette if (plte_length % 3) != 0: from audiotools.text import ERR_IMAGE_INVALID_PLTE raise InvalidPNG(ERR_IMAGE_INVALID_PLTE) else: return cls(width=width, height=height, bits_per_pixel=8, color_count=plte_length // 3) elif color_type == 4: # grayscale + alpha return cls(width=width, height=height, bits_per_pixel=bit_depth * 2, color_count=0) elif color_type == 6: # RGB + alpha return cls(width=width, height=height, bits_per_pixel=bit_depth * 4, color_count=0) ####################### # BMP ####################### class InvalidBMP(InvalidImage): """raised if a BMP cannot be parsed correctly""" pass class __BMP__(ImageMetrics): def __init__(self, width, height, bits_per_pixel, color_count): ImageMetrics.__init__(self, width, height, bits_per_pixel, color_count, u'image/x-ms-bmp') @classmethod def parse(cls, file_data): try: (magic_number, file_size, data_offset, header_size, width, height, color_planes, bits_per_pixel, compression_method, image_size, horizontal_resolution, vertical_resolution, colors_used, important_colors_used) = BitstreamReader(file_data, True).parse( "2b 32u 16p 16p 32u " + "32u 32u 32u 16u 16u 32u 32u 32u 32u 32u 32u") except IOError: from audiotools.text import ERR_IMAGE_IOERROR_BMP raise InvalidBMP(ERR_IMAGE_IOERROR_BMP) if magic_number != b'BM': from audiotools.text import ERR_IMAGE_INVALID_BMP raise InvalidBMP(ERR_IMAGE_INVALID_BMP) else: return cls(width=width, height=height, bits_per_pixel=bits_per_pixel, color_count=colors_used) ####################### # GIF ####################### class InvalidGIF(InvalidImage): """raised if a GIF cannot be parsed correctly""" pass class __GIF__(ImageMetrics): def __init__(self, width, height, color_count): ImageMetrics.__init__(self, width, height, 8, color_count, u'image/gif') @classmethod def parse(cls, file_data): try: (gif, version, width, height, color_table_size) = BitstreamReader(file_data, True).parse( "3b 3b 16u 16u 3u 5p") except IOError: from audiotools.text import ERR_IMAGE_IOERROR_GIF raise InvalidGIF(ERR_IMAGE_IOERROR_GIF) if gif != b'GIF': from audiotools.text import ERR_IMAGE_INVALID_GIF raise InvalidGIF(ERR_IMAGE_INVALID_GIF) else: return cls(width=width, height=height, color_count=2 ** (color_table_size + 1)) ####################### # TIFF ####################### class InvalidTIFF(InvalidImage): """raised if a TIFF cannot be parsed correctly""" pass class __TIFF__(ImageMetrics): def __init__(self, width, height, bits_per_pixel, color_count): ImageMetrics.__init__(self, width, height, bits_per_pixel, color_count, u'image/tiff') @classmethod def parse(cls, file_data): from io import BytesIO def tags(file, order): while True: reader = BitstreamReader(file, order) # read all the tags in an IFD tag_count = reader.read(16) sub_reader = reader.substream(tag_count * 12) next_ifd = reader.read(32) for i in range(tag_count): (tag_code, tag_datatype, tag_value_count) = sub_reader.parse("16u 16u 32u") if tag_datatype == 1: # BYTE type tag_struct = "8u" * tag_value_count elif tag_datatype == 3: # SHORT type tag_struct = "16u" * tag_value_count elif tag_datatype == 4: # LONG type tag_struct = "32u" * tag_value_count else: # all other types tag_struct = "4b" if format_size(tag_struct) <= 32: yield (tag_code, sub_reader.parse(tag_struct)) sub_reader.skip(32 - format_size(tag_struct)) else: offset = sub_reader.read(32) file.seek(offset, 0) yield (tag_code, BitstreamReader(file, order).parse(tag_struct)) if next_ifd != 0: file.seek(next_ifd, 0) else: break file = BytesIO(file_data) try: byte_order = file.read(2) if byte_order == b'II': order = 1 elif byte_order == b'MM': order = 0 else: from audiotools.text import ERR_IMAGE_INVALID_TIFF raise InvalidTIFF(ERR_IMAGE_INVALID_TIFF) reader = BitstreamReader(file, order) if reader.read(16) != 42: from audiotools.text import ERR_IMAGE_INVALID_TIFF raise InvalidTIFF(ERR_IMAGE_INVALID_TIFF) initial_ifd = reader.read(32) file.seek(initial_ifd, 0) width = 0 height = 0 bits_per_pixel = 0 color_count = 0 for (tag_id, tag_values) in tags(file, order): if tag_id == 0x0100: width = tag_values[0] elif tag_id == 0x0101: height = tag_values[0] elif tag_id == 0x0102: bits_per_pixel = sum(tag_values) elif tag_id == 0x0140: color_count = len(tag_values) // 3 except IOError: from audiotools.text import ERR_IMAGE_IOERROR_TIFF raise InvalidTIFF(ERR_IMAGE_IOERROR_TIFF) return cls(width=width, height=height, bits_per_pixel=bits_per_pixel, color_count=color_count) ================================================ FILE: audiotools/m4a.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools import (AudioFile, InvalidFile, BIN, Image) from audiotools.m4a_atoms import * class InvalidM4A(InvalidFile): pass def get_m4a_atom(reader, *atoms): """given a BitstreamReader and atom name strings returns a (size, substream) of the final atom data (not including its 64-bit size/name header) after traversing the parent atoms """ for (last, next_atom) in [(i == len(atoms), v) for (i, v) in enumerate(atoms, 1)]: # assert(isinstance(next_atom, bytes)) try: (length, stream_atom) = reader.parse("32u 4b") while stream_atom != next_atom: if (length - 8) >= 0: reader.skip_bytes(length - 8) (length, stream_atom) = reader.parse("32u 4b") else: raise KeyError(next_atom) if last: return (length - 8, reader.substream(length - 8)) else: reader = reader.substream(length - 8) except IOError: raise KeyError(next_atom) def get_m4a_atom_offset(reader, *atoms): """given a BitstreamReader and atom name strings returns a (size, offset) of the final atom data (including its 64-bit size/name header) after traversing the parent atoms""" offset = 0 for (last, next_atom) in [(i == len(atoms), v) for (i, v) in enumerate(atoms, 1)]: # assert(isinstance(next_atom, bytes)) try: (length, stream_atom) = reader.parse("32u 4b") offset += 8 while stream_atom != next_atom: if (length - 8) > 0: reader.skip_bytes(length - 8) offset += (length - 8) (length, stream_atom) = reader.parse("32u 4b") offset += 8 else: raise KeyError(next_atom) if last: return (length, offset - 8) else: reader = reader.substream(length - 8) except IOError: raise KeyError(next_atom) def has_m4a_atom(reader, *atoms): """given a BitstreamReader and atom name strings returns True if the final atom is present after traversing the parent atoms""" for (last, next_atom) in [(i == len(atoms), v) for (i, v) in enumerate(atoms, 1)]: # assert(isinstance(next_atom, bytes)) try: (length, stream_atom) = reader.parse("32u 4b") while stream_atom != next_atom: if (length - 8) > 0: reader.skip_bytes(length - 8) (length, stream_atom) = reader.parse("32u 4b") else: return False if last: return True else: reader = reader.substream(length - 8) except IOError: return False class M4ATaggedAudio(object): @classmethod def supports_metadata(cls): """returns True if this audio type supports MetaData""" return True def get_metadata(self): """returns a MetaData object, or None raises IOError if unable to read the file""" from audiotools.bitstream import BitstreamReader with BitstreamReader(open(self.filename, 'rb'), False) as reader: try: (meta_size, meta_reader) = get_m4a_atom(reader, b"moov", b"udta", b"meta") except KeyError: return None return M4A_META_Atom.parse(b"meta", meta_size, meta_reader, {b"hdlr": M4A_HDLR_Atom, b"ilst": M4A_Tree_Atom, b"free": M4A_FREE_Atom, b"\xa9alb": M4A_ILST_Leaf_Atom, b"\xa9ART": M4A_ILST_Leaf_Atom, b'aART': M4A_ILST_Leaf_Atom, b"\xa9cmt": M4A_ILST_Leaf_Atom, b"covr": M4A_ILST_Leaf_Atom, b"cpil": M4A_ILST_Leaf_Atom, b"cprt": M4A_ILST_Leaf_Atom, b"\xa9day": M4A_ILST_Leaf_Atom, b"disk": M4A_ILST_Leaf_Atom, b"gnre": M4A_ILST_Leaf_Atom, b"----": M4A_ILST_Leaf_Atom, b"pgap": M4A_ILST_Leaf_Atom, b"rtng": M4A_ILST_Leaf_Atom, b"tmpo": M4A_ILST_Leaf_Atom, b"\xa9grp": M4A_ILST_Leaf_Atom, b"\xa9nam": M4A_ILST_Leaf_Atom, b"\xa9too": M4A_ILST_Leaf_Atom, b"trkn": M4A_ILST_Leaf_Atom, b"\xa9wrt": M4A_ILST_Leaf_Atom}) def update_metadata(self, metadata, old_metadata=None): """takes this track's updated MetaData object as returned by get_metadata() and sets this track's metadata with any fields updated in that object old_metadata is the unmodifed metadata returned by get_metadata() raises IOError if unable to write the file """ from audiotools.bitstream import BitstreamWriter from audiotools.bitstream import BitstreamReader import os.path if metadata is None: return if not isinstance(metadata, M4A_META_Atom): from audiotools.text import ERR_FOREIGN_METADATA raise ValueError(ERR_FOREIGN_METADATA) if old_metadata is None: # get_metadata() result may still be None, and that's okay old_metadata = self.get_metadata() # M4A streams often have *two* "free" atoms we can attempt to resize # first, attempt to resize the one inside the "meta" atom if ((old_metadata is not None) and metadata.has_child(b"free") and ((metadata.size() - metadata[b"free"].size()) <= old_metadata.size())): metadata.replace_child( M4A_FREE_Atom(old_metadata.size() - (metadata.size() - metadata[b"free"].size()))) f = open(self.filename, 'r+b') (meta_size, meta_offset) = get_m4a_atom_offset( BitstreamReader(f, False), b"moov", b"udta", b"meta") f.seek(meta_offset + 8, 0) with BitstreamWriter(f, False) as writer: metadata.build(writer) # writer will close "f" when finished else: from audiotools import TemporaryFile # if there's insufficient room, # attempt to resize the outermost "free" also # this is only possible if the file is laid out correctly, # with "free" coming after "moov" but before "mdat" # FIXME # if neither fix is possible, the whole file must be rewritten # which also requires adjusting the "stco" atom offsets with open(self.filename, "rb") as f: m4a_tree = M4A_Tree_Atom.parse( None, os.path.getsize(self.filename), BitstreamReader(f, False), {b"moov": M4A_Tree_Atom, b"trak": M4A_Tree_Atom, b"mdia": M4A_Tree_Atom, b"minf": M4A_Tree_Atom, b"stbl": M4A_Tree_Atom, b"stco": M4A_STCO_Atom, b"udta": M4A_Tree_Atom}) # find initial mdat offset initial_mdat_offset = m4a_tree.child_offset(b"mdat") # adjust moov -> udta -> meta atom # (generating sub-atoms as necessary) if not m4a_tree.has_child(b"moov"): return else: moov = m4a_tree[b"moov"] if not moov.has_child(b"udta"): moov.add_child(M4A_Tree_Atom(b"udta", [])) udta = moov[b"udta"] if not udta.has_child(b"meta"): udta.add_child(metadata) else: udta.replace_child(metadata) # find new mdat offset new_mdat_offset = m4a_tree.child_offset(b"mdat") # adjust moov -> trak -> mdia -> minf -> stbl -> stco offsets # based on the difference between the new mdat position and the old try: delta_offset = new_mdat_offset - initial_mdat_offset stco = m4a_tree[b"moov"][b"trak"][b"mdia"][b"minf"][b"stbl"][b"stco"] stco.offsets = [offset + delta_offset for offset in stco.offsets] except KeyError: # if there is no stco atom, don't worry about it pass # then write entire tree back to disk with BitstreamWriter(TemporaryFile(self.filename), False) as writer: m4a_tree.build(writer) def set_metadata(self, metadata): """takes a MetaData object and sets this track's metadata this metadata includes track name, album name, and so on raises IOError if unable to write the file""" if metadata is None: return self.delete_metadata() old_metadata = self.get_metadata() metadata = M4A_META_Atom.converted(metadata) # replace file-specific atoms in new metadata # with ones from old metadata (if any) # which can happen if we're shifting metadata # from one M4A file to another file_specific_atoms = {b'\xa9too', b'----', b'pgap', b'tmpo'} if metadata.has_ilst_atom(): metadata.ilst_atom().leaf_atoms = [ atom for atom in metadata.ilst_atom() if atom.name not in file_specific_atoms] if old_metadata.has_ilst_atom(): metadata.ilst_atom().leaf_atoms.extend( [atom for atom in old_metadata.ilst_atom() if atom.name in file_specific_atoms]) self.update_metadata(metadata, old_metadata) def delete_metadata(self): """deletes the track's MetaData this removes or unsets tags as necessary in order to remove all data raises IOError if unable to write the file""" from audiotools import MetaData self.set_metadata(MetaData()) class M4AAudio_faac(M4ATaggedAudio, AudioFile): """an M4A audio file using faac/faad binaries for I/O""" SUFFIX = "m4a" NAME = SUFFIX DESCRIPTION = u"Advanced Audio Coding" DEFAULT_COMPRESSION = "100" COMPRESSION_MODES = tuple(["10"] + list(map(str, range(50, 500, 25))) + ["500"]) BINARIES = ("faac", "faad") BINARY_URLS = {"faac": "http://www.audiocoding.com/", "faad": "http://www.audiocoding.com/"} def __init__(self, filename): """filename is a plain string""" from audiotools.bitstream import BitstreamReader AudioFile.__init__(self, filename) # first, fetch the mdia atom # which is the parent of both the mp4a and mdhd atoms try: with BitstreamReader(open(filename, "rb"), False) as reader: mdia = get_m4a_atom(reader, b"moov", b"trak", b"mdia")[1] mdia_start = mdia.getpos() except IOError: from audiotools.text import ERR_M4A_IOERROR raise InvalidM4A(ERR_M4A_IOERROR) except KeyError: from audiotools.text import ERR_M4A_MISSING_MDIA raise InvalidM4A(ERR_M4A_MISSING_MDIA) try: stsd = get_m4a_atom(mdia, b"minf", b"stbl", b"stsd")[1] except KeyError: from audiotools.text import ERR_M4A_MISSING_STSD raise InvalidM4A(ERR_M4A_MISSING_STSD) # then, fetch the mp4a atom for bps, channels and sample rate try: (stsd_version, descriptions) = stsd.parse("8u 24p 32u") (mp4a, self.__channels__, self.__bits_per_sample__) = stsd.parse( "32p 4b 48p 16p 16p 16p 4P 16u 16u 16p 16p 32p") except IOError: from audiotools.text import ERR_M4A_INVALID_MP4A raise InvalidM4A(ERR_M4A_INVALID_MP4A) # finally, fetch the mdhd atom for total track length mdia.setpos(mdia_start) try: mdhd = get_m4a_atom(mdia, b"mdhd")[1] except KeyError: from audiotools.text import ERR_M4A_MISSING_MDHD raise InvalidM4A(ERR_M4A_MISSING_MDHD) try: (version, ) = mdhd.parse("8u 24p") if version == 0: (self.__sample_rate__, self.__length__,) = mdhd.parse("32p 32p 32u 32u 2P 16p") elif version == 1: (self.__sample_rate__, self.__length__,) = mdhd.parse("64p 64p 32u 64U 2P 16p") else: from audiotools.text import ERR_M4A_UNSUPPORTED_MDHD raise InvalidM4A(ERR_M4A_UNSUPPORTED_MDHD) except IOError: from audiotools.text import ERR_M4A_INVALID_MDHD raise InvalidM4A(ERR_M4A_INVALID_MDHD) def channel_mask(self): """returns a ChannelMask object of this track's channel layout""" from audiotools import ChannelMask # M4A seems to use the same channel assignment # as old-style RIFF WAVE/FLAC if self.channels() == 1: return ChannelMask.from_fields( front_center=True) elif self.channels() == 2: return ChannelMask.from_fields( front_left=True, front_right=True) elif self.channels() == 3: return ChannelMask.from_fields( front_left=True, front_right=True, front_center=True) elif self.channels() == 4: return ChannelMask.from_fields( front_left=True, front_right=True, back_left=True, back_right=True) elif self.channels() == 5: return ChannelMask.from_fields( front_left=True, front_right=True, front_center=True, back_left=True, back_right=True) elif self.channels() == 6: return ChannelMask.from_fields( front_left=True, front_right=True, front_center=True, back_left=True, back_right=True, low_frequency=True) else: return ChannelMask(0) def lossless(self): """returns False""" return False def channels(self): """returns an integer number of channels this track contains""" return self.__channels__ def bits_per_sample(self): """returns an integer number of bits-per-sample this track contains""" return self.__bits_per_sample__ def sample_rate(self): """returns the rate of the track's audio as an integer number of Hz""" return self.__sample_rate__ def cd_frames(self): """returns the total length of the track in CD frames each CD frame is 1/75th of a second""" return ((self.__length__ - 1024) * 75) // self.__sample_rate__ def total_frames(self): """returns the total PCM frames of the track as an integer""" return self.__length__ - 1024 @classmethod def supports_to_pcm(cls): """returns True if all necessary components are available to support the .to_pcm() method""" return BIN.can_execute(BIN["faad"]) def to_pcm(self): """returns a PCMReader object containing the track's PCM data""" from audiotools import PCMFileReader import subprocess import os sub = subprocess.Popen( [BIN['faad'], "-f", str(2), "-w", self.filename], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL if hasattr(subprocess, "DEVNULL") else open(os.devnull, "wb")) return PCMFileReader(sub.stdout, sample_rate=self.sample_rate(), channels=self.channels(), channel_mask=int(self.channel_mask()), bits_per_sample=self.bits_per_sample(), process=sub) @classmethod def supports_from_pcm(cls): """returns True if all necessary components are available to support the .from_pcm() classmethod""" return BIN.can_execute(BIN["faac"]) @classmethod def from_pcm(cls, filename, pcmreader, compression=None, total_pcm_frames=None): """encodes a new file from PCM data takes a filename string, PCMReader object, optional compression level string and optional total_pcm_frames integer encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new M4AAudio object""" import subprocess import os from audiotools import PCMConverter from audiotools import transfer_data from audiotools import transfer_framelist_data from audiotools import ignore_sigint from audiotools import EncodingError from audiotools import DecodingError from audiotools import ChannelMask from audiotools import __default_quality__ if ((compression is None) or (compression not in cls.COMPRESSION_MODES)): compression = __default_quality__(cls.NAME) if pcmreader.bits_per_sample not in {8, 16, 24}: from audiotools import UnsupportedBitsPerSample pcmreader.close() raise UnsupportedBitsPerSample(filename, pcmreader.bits_per_sample) if pcmreader.channels > 2: pcmreader = PCMConverter(pcmreader, sample_rate=pcmreader.sample_rate, channels=2, channel_mask=ChannelMask.from_channels(2), bits_per_sample=pcmreader.bits_per_sample) # faac requires files to end with .m4a for some reason if not filename.endswith(".m4a"): import tempfile actual_filename = filename tempfile = tempfile.NamedTemporaryFile(suffix=".m4a") filename = tempfile.name else: actual_filename = tempfile = None sub = subprocess.Popen( [BIN['faac'], "-q", compression, "-P", "-R", str(pcmreader.sample_rate), "-B", str(pcmreader.bits_per_sample), "-C", str(pcmreader.channels), "-X", "-o", filename, "-"], stdin=subprocess.PIPE, stderr=subprocess.DEVNULL if hasattr(subprocess, "DEVNULL") else open(os.devnull, "wb"), stdout=subprocess.DEVNULL if hasattr(subprocess, "DEVNULL") else open(os.devnull, "wb"), preexec_fn=ignore_sigint) # Note: faac handles SIGINT on its own, # so trying to ignore it doesn't work like on most other encoders. try: if total_pcm_frames is not None: from audiotools import CounterPCMReader pcmreader = CounterPCMReader(pcmreader) transfer_framelist_data(pcmreader, sub.stdin.write) if ((total_pcm_frames is not None) and (total_pcm_frames != pcmreader.frames_written)): from audiotools.text import ERR_TOTAL_PCM_FRAMES_MISMATCH raise EncodingError(ERR_TOTAL_PCM_FRAMES_MISMATCH) except (ValueError, IOError) as err: sub.stdin.close() sub.wait() cls.__unlink__(filename) raise EncodingError(str(err)) except Exception: sub.stdin.close() sub.wait() cls.__unlink__(filename) raise sub.stdin.close() if sub.wait() == 0: if tempfile is not None: filename = actual_filename f = open(filename, 'wb') tempfile.seek(0, 0) transfer_data(tempfile.read, f.write) f.close() tempfile.close() return M4AAudio(filename) else: if tempfile is not None: tempfile.close() raise EncodingError(u"unable to write file with faac") class M4AAudio_nero(M4AAudio_faac): """an M4A audio file using neroAacEnc/neroAacDec binaries for I/O""" from audiotools.text import (COMP_NERO_LOW, COMP_NERO_HIGH) DEFAULT_COMPRESSION = "0.5" COMPRESSION_MODES = ("0.4", "0.5", "0.6", "0.7", "0.8", "0.9", "1.0") COMPRESSION_DESCRIPTIONS = {"0.4": COMP_NERO_LOW, "1.0": COMP_NERO_HIGH} BINARIES = ("neroAacDec", "neroAacEnc") BINARY_URLS = {"neroAacDec": "http://www.nero.com/enu/" + "downloads-nerodigital-nero-aac-codec.php", "neroAacEnc": "http://www.nero.com/enu/" + "downloads-nerodigital-nero-aac-codec.php"} @classmethod def supports_from_pcm(cls): """returns True if all necessary components are available to support the .from_pcm() classmethod""" return BIN.can_execute(BIN["neroAacEnc"]) @classmethod def from_pcm(cls, filename, pcmreader, compression=None, total_pcm_frames=None): """encodes a new file from PCM data takes a filename string, PCMReader object, optional compression level string and optional total_pcm_frames integer encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new M4AAudio object""" import tempfile import os import os.path from audiotools import PCMConverter from audiotools import WaveAudio from audiotools import __default_quality__ if ((compression is None) or (compression not in cls.COMPRESSION_MODES)): compression = __default_quality__(cls.NAME) tempwavefile = tempfile.NamedTemporaryFile(suffix=".wav", delete=False) tempwave_name = tempwavefile.name try: if pcmreader.sample_rate > 96000: tempwave = WaveAudio.from_pcm( tempwave_name, PCMConverter(pcmreader, sample_rate=96000, channels=pcmreader.channels, channel_mask=pcmreader.channel_mask, bits_per_sample=pcmreader.bits_per_sample), total_pcm_frames=total_pcm_frames) else: tempwave = WaveAudio.from_pcm( tempwave_name, pcmreader, total_pcm_frames=total_pcm_frames) cls.__from_wave__(filename, tempwave.filename, compression) return cls(filename) finally: tempwavefile.close() if os.path.isfile(tempwave_name): os.unlink(tempwave_name) @classmethod def supports_to_pcm(cls): """returns True if all necessary components are available to support the .to_pcm() method""" return BIN.can_execute(BIN["neroAacDec"]) def to_pcm(self): from audiotools import PCMFileReader import subprocess import os sub = subprocess.Popen( [BIN["neroAacDec"], "-if", self.filename, "-of", "-"], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL if hasattr(subprocess, "DEVNULL") else open(os.devnull, "wb")) return PCMFileReader(file=sub.stdout, sample_rate=self.sample_rate(), channels=self.channels(), channel_mask=int(self.channel_mask()), bits_per_sample=self.bits_per_sample(), process=sub) @classmethod def __from_wave__(cls, filename, wave_filename, compression): import subprocess import os from audiotools import EncodingError sub = subprocess.Popen( [BIN["neroAacEnc"], "-q", compression, "-if", wave_filename, "-of", filename], stdout=subprocess.DEVNULL if hasattr(subprocess, "DEVNULL") else open(os.devnull, "wb"), stderr=subprocess.DEVNULL if hasattr(subprocess, "DEVNULL") else open(os.devnull, "wb")) if sub.wait() != 0: raise EncodingError(u"neroAacEnc unable to write file") else: return cls(filename) if BIN.can_execute(BIN["neroAacEnc"]) and BIN.can_execute(BIN["neroAacDec"]): M4AAudio = M4AAudio_nero else: M4AAudio = M4AAudio_faac class InvalidALAC(InvalidFile): pass class ALACAudio(M4ATaggedAudio, AudioFile): """an Apple Lossless audio file""" SUFFIX = "m4a" NAME = "alac" DESCRIPTION = u"Apple Lossless" DEFAULT_COMPRESSION = "" COMPRESSION_MODES = ("",) BINARIES = tuple() BLOCK_SIZE = 4096 INITIAL_HISTORY = 10 HISTORY_MULTIPLIER = 40 MAXIMUM_K = 14 def __init__(self, filename): """filename is a plain string""" from audiotools.bitstream import BitstreamReader AudioFile.__init__(self, filename) # first, fetch the mdia atom # which is the parent of both the alac and mdhd atoms try: with BitstreamReader(open(filename, "rb"), False) as reader: mdia = get_m4a_atom(reader, b"moov", b"trak", b"mdia")[1] mdia_start = mdia.getpos() except IOError: from audiotools.text import ERR_ALAC_IOERROR raise InvalidALAC(ERR_ALAC_IOERROR) except KeyError: from audiotools.text import ERR_M4A_MISSING_MDIA raise InvalidALAC(ERR_M4A_MISSING_MDIA) try: stsd = get_m4a_atom(mdia, b"minf", b"stbl", b"stsd")[1] except KeyError: from audiotools.text import ERR_M4A_MISSING_STSD raise InvalidALAC(ERR_M4A_MISSING_STSD) # then, fetch the alac atom for bps, channels and sample rate try: # though some of these fields are parsed redundantly # in .to_pcm(), we still need to parse them here # to fetch values for .bits_per_sample(), etc. (stsd_version, descriptions) = stsd.parse("8u 24p 32u") (alac1, alac2, self.__max_samples_per_frame__, self.__bits_per_sample__, self.__history_multiplier__, self.__initial_history__, self.__maximum_k__, self.__channels__, self.__sample_rate__) = stsd.parse( # ignore much of the stuff in the "high" ALAC atom "32p 4b 6P 16p 16p 16p 4P 16p 16p 16p 16p 4P" + # and use the attributes in the "low" ALAC atom instead "32p 4b 4P 32u 8p 8u 8u 8u 8u 8u 16p 32p 32p 32u") except IOError: from audiotools.text import ERR_ALAC_INVALID_ALAC raise InvalidALAC(ERR_ALAC_INVALID_ALAC) if (alac1 != b'alac') or (alac2 != b'alac'): from audiotools.text import ERR_ALAC_INVALID_ALAC raise InvalidALAC(ERR_ALAC_INVALID_ALAC) # finally, fetch the mdhd atom for total track length mdia.setpos(mdia_start) try: mdhd = get_m4a_atom(mdia, b"mdhd")[1] except KeyError: from audiotools.text import ERR_M4A_MISSING_MDHD raise InvalidALAC(ERR_M4A_MISSING_MDHD) try: (version, ) = mdhd.parse("8u 24p") if version == 0: (self.__length__,) = mdhd.parse("32p 32p 32p 32u 2P 16p") elif version == 1: (self.__length__,) = mdhd.parse("64p 64p 32p 64U 2P 16p") else: from audiotools.text import ERR_M4A_UNSUPPORTED_MDHD raise InvalidALAC(ERR_M4A_UNSUPPORTED_MDHD) except IOError: from audiotools.text import ERR_M4A_INVALID_MDHD raise InvalidALAC(ERR_M4A_INVALID_MDHD) def channels(self): """returns an integer number of channels this track contains""" return self.__channels__ def bits_per_sample(self): """returns an integer number of bits-per-sample this track contains""" return self.__bits_per_sample__ def sample_rate(self): """returns the rate of the track's audio as an integer number of Hz""" return self.__sample_rate__ def total_frames(self): """returns the total PCM frames of the track as an integer""" return self.__length__ def channel_mask(self): """returns a ChannelMask object of this track's channel layout""" from audiotools import ChannelMask with self.to_pcm() as r: return ChannelMask(r.channel_mask) def cd_frames(self): """returns the total length of the track in CD frames each CD frame is 1/75th of a second""" try: return (self.total_frames() * 75) // self.sample_rate() except ZeroDivisionError: return 0 def lossless(self): """returns True""" return True def seekable(self): """returns True if the file is seekable""" from audiotools.bitstream import BitstreamReader with BitstreamReader(open(self.filename, "rb"), False) as reader: stream_start = reader.getpos() has_stts = has_m4a_atom(reader, b"moov", b"trak", b"mdia", b"minf", b"stbl", b"stts") reader.setpos(stream_start) has_stsc = has_m4a_atom(reader, b"moov", b"trak", b"mdia", b"minf", b"stbl", b"stsc") reader.setpos(stream_start) has_stco = has_m4a_atom(reader, b"moov", b"trak", b"mdia", b"minf", b"stbl", b"stco") return has_stts and has_stsc and has_stco @classmethod def supports_to_pcm(cls): """returns True if all necessary components are available to support the .to_pcm() method""" try: from audiotools.decoders import ALACDecoder return True except ImportError: return False def to_pcm(self): """returns a PCMReader object containing the track's PCM data""" from audiotools.decoders import ALACDecoder from audiotools import PCMReaderError try: return ALACDecoder(open(self.filename, "rb")) except (IOError, ValueError) as msg: return PCMReaderError(error_message=str(msg), sample_rate=self.sample_rate(), channels=self.channels(), channel_mask=int(self.channel_mask()), bits_per_sample=self.bits_per_sample()) @classmethod def supports_from_pcm(cls): """returns True if all necessary components are available to support the .from_pcm() classmethod""" try: from audiotools.encoders import encode_alac return True except ImportError: return False @classmethod def from_pcm(cls, filename, pcmreader, compression=None, total_pcm_frames=None, block_size=4096, encoding_function=None): """encodes a new file from PCM data takes a filename string, PCMReader object, optional compression level string and optional total_pcm_frames integer encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new ALACAudio object""" from audiotools.encoders import encode_alac from audiotools import VERSION, EncodingError if pcmreader.bits_per_sample not in {16, 24}: from audiotools import UnsupportedBitsPerSample pcmreader.close() raise UnsupportedBitsPerSample(filename, pcmreader.bits_per_sample) if (pcmreader.channel_mask not in {0x0001, # 1ch - mono 0x0004, # 1ch - mono 0x0003, # 2ch - left, right 0x0007, # 3ch - center, left, right 0x0107, # 4ch - center, left, right, back center 0x0037, # 5ch - center, left, right, back left, back right 0x003F, # 6ch - C, L, R, back left, back right, LFE 0x013F, # 7ch - C, L, R, bL, bR, back center, LFE 0x063F, # 8ch - fC, lC, rC, fL, fR, bL, bR, LFE 0x0000}): # undefined from audiotools import UnsupportedChannelMask pcmreader.close() raise UnsupportedChannelMask(filename, pcmreader.channel_mask) try: file = open(filename, "wb") except IOError as err: pcmreader.close() raise EncodingError(str(err)) try: encode_alac( file=file, pcmreader=pcmreader, total_pcm_frames=(total_pcm_frames if (total_pcm_frames is not None) else 0), block_size=block_size, initial_history=cls.INITIAL_HISTORY, history_multiplier=cls.HISTORY_MULTIPLIER, maximum_k=cls.MAXIMUM_K, version="Python Audio Tools " + VERSION) except (ValueError, IOError) as err: cls.__unlink__(filename) raise EncodingError(str(err)) except Exception: cls.__unlink__(filename) raise finally: pcmreader.close() file.close() return cls(filename) ================================================ FILE: audiotools/m4a_atoms.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools import MetaData, Image from audiotools.image import image_metrics import sys # M4A atoms are typically laid on in the file as follows: # ftyp # mdat # moov/ # +mvhd # +iods # +trak/ # +-tkhd # +-mdia/ # +--mdhd # +--hdlr # +--minf/ # +---smhd # +---dinf/ # +----dref # +---stbl/ # +----stsd # +----stts # +----stsz # +----stsc # +----stco # +----ctts # +udta/ # +-meta # # Where atoms ending in / are container atoms and the rest are leaf atoms. # 'mdat' is where the file's audio stream is stored # the rest are various bits of metadata def parse_sub_atoms(data_size, reader, parsers): """data size is the length of the parent atom's data reader is a BitstreamReader parsers is a dict of leaf_name->parser() where parser is defined as: parser(leaf_name, leaf_data_size, BitstreamReader, parsers) as a sort of recursive parsing handler """ leaf_atoms = [] while data_size > 0: (leaf_size, leaf_name) = reader.parse("32u 4b") leaf_atoms.append( parsers.get(leaf_name, M4A_Leaf_Atom).parse( leaf_name, leaf_size - 8, reader.substream(leaf_size - 8), parsers)) data_size -= leaf_size return leaf_atoms # build(), parse() and size() work on atom data # but not the atom's size and name values class M4A_Tree_Atom(object): def __init__(self, name, leaf_atoms): """name should be a 4 byte string children should be a list of M4A_Tree_Atoms or M4A_Leaf_Atoms""" # assert((name is None) or isinstance(name, bytes)) self.name = name self.leaf_atoms = leaf_atoms def copy(self): """returns a newly copied instance of this atom and new instances of any sub-atoms it contains""" return M4A_Tree_Atom(self.name, [leaf.copy() for leaf in self]) def __repr__(self): return "M4A_Tree_Atom({!r}, {!r})".format(self.name, self.leaf_atoms) def __eq__(self, atom): for attr in ["name", "leaf_atoms"]: if ((not hasattr(atom, attr)) or (getattr(self, attr) != getattr(atom, attr))): return False else: return True def __iter__(self): for leaf in self.leaf_atoms: yield leaf def __getitem__(self, atom_name): return self.get_child(atom_name) def get_child(self, atom_name): """returns the first instance of the given child atom raises KeyError if the child is not found""" # assert(isinstance(atom_name, bytes)) for leaf in self: if leaf.name == atom_name: return leaf else: raise KeyError(atom_name) def get_children(self, atom_name): """returns all instances of the given child atom as a list""" return [leaf for leaf in self if leaf.name == atom_name] def has_child(self, atom_name): """returns True if the given atom name is an immediate child of this atom""" # assert(isinstance(atom_name, bytes)) for leaf in self: if leaf.name == atom_name: return True else: return False def add_child(self, atom_obj): """adds the given child atom to this container""" self.leaf_atoms.append(atom_obj) def remove_child(self, atom_name): """removes the first instance of the given atom from this container""" # assert(isinstance(atom_name, bytes)) new_leaf_atoms = [] data_deleted = False for leaf_atom in self: if (leaf_atom.name == atom_name) and (not data_deleted): data_deleted = True else: new_leaf_atoms.append(leaf_atom) self.leaf_atoms = new_leaf_atoms def replace_child(self, atom_obj): """replaces the first instance of the given atom's name with the given atom""" new_leaf_atoms = [] data_replaced = False for leaf_atom in self: if (leaf_atom.name == atom_obj.name) and (not data_replaced): new_leaf_atoms.append(atom_obj) data_replaced = True else: new_leaf_atoms.append(leaf_atom) self.leaf_atoms = new_leaf_atoms def child_offset(self, *child_path): """given a path to the given child atom returns its offset within this parent raises KeyError if the child cannot be found""" offset = 0 next_child = child_path[0] for leaf_atom in self: if leaf_atom.name == next_child: if len(child_path) > 1: return (offset + 8 + leaf_atom.child_offset(*(child_path[1:]))) else: return offset else: offset += (8 + leaf_atom.size()) else: raise KeyError(next_child) @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" return cls(name, parse_sub_atoms(data_size, reader, parsers)) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" for sub_atom in self: writer.build("32u 4b", (sub_atom.size() + 8, sub_atom.name)) sub_atom.build(writer) def size(self): """returns the atom's size not including its 64-bit size / name header""" return sum([8 + sub_atom.size() for sub_atom in self]) class M4A_Leaf_Atom(object): def __init__(self, name, data): """name should be a 4 byte string data should be a binary string of atom data""" # assert(isinstance(name, bytes)) # assert(isinstance(data, bytes)) self.name = name self.data = data def copy(self): """returns a newly copied instance of this atom and new instances of any sub-atoms it contains""" return M4A_Leaf_Atom(self.name, self.data) def __repr__(self): return "M4A_Leaf_Atom({!r}, {!r})".format(self.name, self.data) def __eq__(self, atom): for attr in ["name", "data"]: if ((not hasattr(atom, attr)) or (getattr(self, attr) != getattr(atom, attr))): return False else: return True if sys.version_info[0] >= 3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('utf-8') def __unicode__(self): # FIXME - should make this more informative, if possible from audiotools import hex_string return hex_string(self.data[0:20]) def raw_info(self): """returns a line of human-readable information about the atom""" from audiotools import hex_string if len(self.data) > 20: return u"{} : {}\u2026".format( self.name.decode('ascii', 'replace'), hex_string(self.data[0:20])) else: return u"{} : {}".format( self.name.decode('ascii', 'replace'), hex_string(self.data)) @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" return cls(name, reader.read_bytes(data_size)) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.write_bytes(self.data) def size(self): """returns the atom's size not including its 64-bit size / name header""" return len(self.data) class M4A_FTYP_Atom(M4A_Leaf_Atom): def __init__(self, major_brand, major_brand_version, compatible_brands): # assert(isinstance(major_brand, bytes)) # for b in compatible_brands: # assert(isinstance(b, bytes)) self.name = b'ftyp' self.major_brand = major_brand self.major_brand_version = major_brand_version self.compatible_brands = compatible_brands def __repr__(self): return "M4A_FTYP_Atom({!r}, {!r}, {!r})".format( self.major_brand, self.major_brand_version, self.compatible_brands) @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" assert(name == b'ftyp') return cls(reader.read_bytes(4), reader.read(32), [reader.read_bytes(4) for i in range((data_size - 8) // 4)]) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.build("4b 32u {:d}* 4b".format(len(self.compatible_brands)), [self.major_brand, self.major_brand_version] + self.compatible_brands) def size(self): """returns the atom's size not including its 64-bit size / name header""" return 4 + 4 + (4 * len(self.compatible_brands)) class M4A_MVHD_Atom(M4A_Leaf_Atom): def __init__(self, version, flags, created_utc_date, modified_utc_date, time_scale, duration, playback_speed, user_volume, geometry_matrices, qt_preview, qt_still_poster, qt_selection_time, qt_current_time, next_track_id): self.name = b'mvhd' self.version = version self.flags = flags self.created_utc_date = created_utc_date self.modified_utc_date = modified_utc_date self.time_scale = time_scale self.duration = duration self.playback_speed = playback_speed self.user_volume = user_volume self.geometry_matrices = geometry_matrices self.qt_preview = qt_preview self.qt_still_poster = qt_still_poster self.qt_selection_time = qt_selection_time self.qt_current_time = qt_current_time self.next_track_id = next_track_id @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" assert(name == b'mvhd') (version, flags) = reader.parse("8u 24u") if version == 0: atom_format = "32u 32u 32u 32u 32u 16u 10P" else: atom_format = "64U 64U 32u 64U 32u 16u 10P" (created_utc_date, modified_utc_date, time_scale, duration, playback_speed, user_volume) = reader.parse(atom_format) geometry_matrices = reader.parse("32u" * 9) (qt_preview, qt_still_poster, qt_selection_time, qt_current_time, next_track_id) = reader.parse("64U 32u 64U 32u 32u") return cls(version=version, flags=flags, created_utc_date=created_utc_date, modified_utc_date=modified_utc_date, time_scale=time_scale, duration=duration, playback_speed=playback_speed, user_volume=user_volume, geometry_matrices=geometry_matrices, qt_preview=qt_preview, qt_still_poster=qt_still_poster, qt_selection_time=qt_selection_time, qt_current_time=qt_current_time, next_track_id=next_track_id) def __repr__(self): return "MVHD_Atom({})".format( ",".join(map(repr, [self.version, self.flags, self.created_utc_date, self.modified_utc_date, self.time_scale, self.duration, self.playback_speed, self.user_volume, self.geometry_matrices, self.qt_preview, self.qt_still_poster, self.qt_selection_time, self.qt_current_time, self.next_track_id]))) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.build("8u 24u", (self.version, self.flags)) if self.version == 0: atom_format = "32u 32u 32u 32u 32u 16u 10P" else: atom_format = "64U 64U 32u 64U 32u 16u 10P" writer.build(atom_format, (self.created_utc_date, self.modified_utc_date, self.time_scale, self.duration, self.playback_speed, self.user_volume)) writer.build("9* 32u", self.geometry_matrices) writer.build("64U 32u 64U 32u 32u", (self.qt_preview, self.qt_still_poster, self.qt_selection_time, self.qt_current_time, self.next_track_id)) def size(self): """returns the atom's size not including its 64-bit size / name header""" if self.version == 0: return 100 else: return 112 class M4A_TKHD_Atom(M4A_Leaf_Atom): def __init__(self, version, track_in_poster, track_in_preview, track_in_movie, track_enabled, created_utc_date, modified_utc_date, track_id, duration, video_layer, qt_alternate, volume, geometry_matrices, video_width, video_height): self.name = b'tkhd' self.version = version self.track_in_poster = track_in_poster self.track_in_preview = track_in_preview self.track_in_movie = track_in_movie self.track_enabled = track_enabled self.created_utc_date = created_utc_date self.modified_utc_date = modified_utc_date self.track_id = track_id self.duration = duration self.video_layer = video_layer self.qt_alternate = qt_alternate self.volume = volume self.geometry_matrices = geometry_matrices self.video_width = video_width self.video_height = video_height def __repr__(self): return "M4A_TKHD_Atom({})".format( ",".join(map(repr, [self.version, self.track_in_poster, self.track_in_preview, self.track_in_movie, self.track_enabled, self.created_utc_date, self.modified_utc_date, self.track_id, self.duration, self.video_layer, self.qt_alternate, self.volume, self.geometry_matrices, self.video_width, self.video_height]))) @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" (version, track_in_poster, track_in_preview, track_in_movie, track_enabled) = reader.parse("8u 20p 1u 1u 1u 1u") if version == 0: atom_format = "32u 32u 32u 4P 32u 8P 16u 16u 16u 2P" else: atom_format = "64U 64U 32u 4P 64U 8P 16u 16u 16u 2P" (created_utc_date, modified_utc_date, track_id, duration, video_layer, qt_alternate, volume) = reader.parse(atom_format) geometry_matrices = reader.parse("9* 32u") (video_width, video_height) = reader.parse("32u 32u") return cls(version=version, track_in_poster=track_in_poster, track_in_preview=track_in_preview, track_in_movie=track_in_movie, track_enabled=track_enabled, created_utc_date=created_utc_date, modified_utc_date=modified_utc_date, track_id=track_id, duration=duration, video_layer=video_layer, qt_alternate=qt_alternate, volume=volume, geometry_matrices=geometry_matrices, video_width=video_width, video_height=video_height) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.build("8u 20p 1u 1u 1u 1u", (self.version, self.track_in_poster, self.track_in_preview, self.track_in_movie, self.track_enabled)) if self.version == 0: atom_format = "32u 32u 32u 4P 32u 8P 16u 16u 16u 2P" else: atom_format = "64U 64U 32u 4P 64U 8P 16u 16u 16u 2P" writer.build(atom_format, (self.created_utc_date, self.modified_utc_date, self.track_id, self.duration, self.video_layer, self.qt_alternate, self.volume)) writer.build("9* 32u", self.geometry_matrices) writer.build("32u 32u", (self.video_width, self.video_height)) def size(self): """returns the atom's size not including its 64-bit size / name header""" if self.version == 0: return 84 else: return 96 class M4A_MDHD_Atom(M4A_Leaf_Atom): def __init__(self, version, flags, created_utc_date, modified_utc_date, sample_rate, track_length, language, quality): self.name = b'mdhd' self.version = version self.flags = flags self.created_utc_date = created_utc_date self.modified_utc_date = modified_utc_date self.sample_rate = sample_rate self.track_length = track_length self.language = language self.quality = quality def __repr__(self): return "M4A_MDHD_Atom({})".format( ",".join(map(repr, [self.version, self.flags, self.created_utc_date, self.modified_utc_date, self.sample_rate, self.track_length, self.language, self.quality]))) @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" assert(name == b'mdhd') (version, flags) = reader.parse("8u 24u") if version == 0: atom_format = "32u 32u 32u 32u" else: atom_format = "64U 64U 32u 64U" (created_utc_date, modified_utc_date, sample_rate, track_length) = reader.parse(atom_format) language = reader.parse("1p 5u 5u 5u") quality = reader.read(16) return cls(version=version, flags=flags, created_utc_date=created_utc_date, modified_utc_date=modified_utc_date, sample_rate=sample_rate, track_length=track_length, language=language, quality=quality) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.build("8u 24u", (self.version, self.flags)) if self.version == 0: atom_format = "32u 32u 32u 32u" else: atom_format = "64U 64U 32u 64U" writer.build(atom_format, (self.created_utc_date, self.modified_utc_date, self.sample_rate, self.track_length)) writer.build("1p 5u 5u 5u", self.language) writer.write(16, self.quality) def size(self): """returns the atom's size not including its 64-bit size / name header""" if self.version == 0: return 24 else: return 36 class M4A_SMHD_Atom(M4A_Leaf_Atom): def __init__(self, version, flags, audio_balance): self.name = b'smhd' self.version = version self.flags = flags self.audio_balance = audio_balance def __repr__(self): return "M4A_SMHD_Atom({})".format( ",".join(map(repr, [self.version, self.flags, self.audio_balance]))) @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" return cls(*reader.parse("8u 24u 16u 16p")) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.build("8u 24u 16u 16p", (self.version, self.flags, self.audio_balance)) def size(self): """returns the atom's size not including its 64-bit size / name header""" return 8 class M4A_DREF_Atom(M4A_Leaf_Atom): def __init__(self, version, flags, references): self.name = b'dref' self.version = version self.flags = flags self.references = references def __repr__(self): return "M4A_DREF_Atom({})".format( ",".join(map(repr, [self.version, self.flags, self.references]))) @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" (version, flags, reference_count) = reader.parse("8u 24u 32u") references = [] for i in range(reference_count): (leaf_size, leaf_name) = reader.parse("32u 4b") references.append( M4A_Leaf_Atom.parse( leaf_name, leaf_size - 8, reader.substream(leaf_size - 8), {})) return cls(version, flags, references) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.build("8u 24u 32u", (self.version, self.flags, len(self.references))) for reference_atom in self.references: writer.build("32u 4b", (reference_atom.size() + 8, reference_atom.name)) reference_atom.build(writer) def size(self): """returns the atom's size not including its 64-bit size / name header""" return 8 + sum([reference_atom.size() + 8 for reference_atom in self.references]) class M4A_STSD_Atom(M4A_Leaf_Atom): def __init__(self, version, flags, descriptions): self.name = b'stsd' self.version = version self.flags = flags self.descriptions = descriptions def __repr__(self): return "M4A_STSD_Atom({!r}, {!r}, {!r})".format( self.version, self.flags, self.descriptions) @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" (version, flags, description_count) = reader.parse("8u 24u 32u") descriptions = [] for i in range(description_count): (leaf_size, leaf_name) = reader.parse("32u 4b") descriptions.append( parsers.get(leaf_name, M4A_Leaf_Atom).parse( leaf_name, leaf_size - 8, reader.substream(leaf_size - 8), parsers)) return cls(version=version, flags=flags, descriptions=descriptions) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.build("8u 24u 32u", (self.version, self.flags, len(self.descriptions))) for description_atom in self.descriptions: writer.build("32u 4b", (description_atom.size() + 8, description_atom.name)) description_atom.build(writer) def size(self): """returns the atom's size not including its 64-bit size / name header""" return 8 + sum([8 + description_atom.size() for description_atom in self.descriptions]) class M4A_STTS_Atom(M4A_Leaf_Atom): def __init__(self, version, flags, times): self.name = b'stts' self.version = version self.flags = flags self.times = times def __repr__(self): return "M4A_STTS_Atom({!r}, {!r}, {!r})".format( self.version, self.flags, self.times) @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" (version, flags) = reader.parse("8u 24u") return cls(version=version, flags=flags, times=[tuple(reader.parse("32u 32u")) for i in range(reader.read(32))]) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.build("8u 24u 32u", (self.version, self.flags, len(self.times))) for time in self.times: writer.build("32u 32u", time) def size(self): """returns the atom's size not including its 64-bit size / name header""" return 8 + (8 * len(self.times)) class M4A_STSC_Atom(M4A_Leaf_Atom): def __init__(self, version, flags, blocks): self.name = b'stsc' self.version = version self.flags = flags self.blocks = blocks def __repr__(self): return "M4A_STSC_Atom({!r}, {!r}, {!r})".format( self.version, self.flags, self.blocks) @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" (version, flags) = reader.parse("8u 24u") return cls(version=version, flags=flags, blocks=[tuple(reader.parse("32u 32u 32u")) for i in range(reader.read(32))]) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.build("8u 24u 32u", (self.version, self.flags, len(self.blocks))) for block in self.blocks: writer.build("32u 32u 32u", block) def size(self): """returns the atom's size not including its 64-bit size / name header""" return 8 + (12 * len(self.blocks)) class M4A_STSZ_Atom(M4A_Leaf_Atom): def __init__(self, version, flags, byte_size, block_sizes): self.name = b'stsz' self.version = version self.flags = flags self.byte_size = byte_size self.block_sizes = block_sizes def __repr__(self): return "M4A_STSZ_Atom({!r}, {!r}, {!r}, {!r})".format( self.version, self.flags, self.byte_size, self.block_sizes) @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" (version, flags, byte_size) = reader.parse("8u 24u 32u") return cls(version=version, flags=flags, byte_size=byte_size, block_sizes=[reader.read(32) for i in range(reader.read(32))]) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.build("8u 24u 32u 32u", (self.version, self.flags, self.byte_size, len(self.block_sizes))) for size in self.block_sizes: writer.write(32, size) def size(self): """returns the atom's size not including its 64-bit size / name header""" return 12 + (4 * len(self.block_sizes)) class M4A_STCO_Atom(M4A_Leaf_Atom): def __init__(self, version, flags, offsets): self.name = b'stco' self.version = version self.flags = flags self.offsets = offsets def __repr__(self): return "M4A_STCO_Atom({!r}, {!r}, {!r})".format( self.version, self.flags, self.offsets) @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" assert(name == b"stco") (version, flags, offset_count) = reader.parse("8u 24u 32u") return cls(version, flags, [reader.read(32) for i in range(offset_count)]) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.build("8u 24u 32u", (self.version, self.flags, len(self.offsets))) for offset in self.offsets: writer.write(32, offset) def size(self): """returns the atom's size not including its 64-bit size / name header""" return 8 + (4 * len(self.offsets)) class M4A_ALAC_Atom(M4A_Leaf_Atom): def __init__(self, reference_index, qt_version, qt_revision_level, qt_vendor, channels, bits_per_sample, qt_compression_id, audio_packet_size, sample_rate, sub_alac): # assert(isinstance(qt_vendor, bytes)) self.name = b'alac' self.reference_index = reference_index self.qt_version = qt_version self.qt_revision_level = qt_revision_level self.qt_vendor = qt_vendor self.channels = channels self.bits_per_sample = bits_per_sample self.qt_compression_id = qt_compression_id self.audio_packet_size = audio_packet_size self.sample_rate = sample_rate self.sub_alac = sub_alac def __repr__(self): return "M4A_ALAC_Atom({})".format( ",".join(map(repr, [self.reference_index, self.qt_version, self.qt_revision_level, self.qt_vendor, self.channels, self.bits_per_sample, self.qt_compression_id, self.audio_packet_size, self.sample_rate, self.sub_alac]))) @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" (reference_index, qt_version, qt_revision_level, qt_vendor, channels, bits_per_sample, qt_compression_id, audio_packet_size, sample_rate) = reader.parse( "6P 16u 16u 16u 4b 16u 16u 16u 16u 32u") (sub_alac_size, sub_alac_name) = reader.parse("32u 4b") sub_alac = M4A_SUB_ALAC_Atom.parse(sub_alac_name, sub_alac_size - 8, reader.substream(sub_alac_size - 8), {}) return cls(reference_index=reference_index, qt_version=qt_version, qt_revision_level=qt_revision_level, qt_vendor=qt_vendor, channels=channels, bits_per_sample=bits_per_sample, qt_compression_id=qt_compression_id, audio_packet_size=audio_packet_size, sample_rate=sample_rate, sub_alac=sub_alac) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.build("6P 16u 16u 16u 4b 16u 16u 16u 16u 32u", (self.reference_index, self.qt_version, self.qt_revision_level, self.qt_vendor, self.channels, self.bits_per_sample, self.qt_compression_id, self.audio_packet_size, self.sample_rate)) writer.build("32u 4b", (self.sub_alac.size() + 8, self.sub_alac.name)) self.sub_alac.build(writer) def size(self): """returns the atom's size not including its 64-bit size / name header""" return 28 + 8 + self.sub_alac.size() class M4A_SUB_ALAC_Atom(M4A_Leaf_Atom): def __init__(self, max_samples_per_frame, bits_per_sample, history_multiplier, initial_history, maximum_k, channels, unknown, max_coded_frame_size, bitrate, sample_rate): self.name = b'alac' self.max_samples_per_frame = max_samples_per_frame self.bits_per_sample = bits_per_sample self.history_multiplier = history_multiplier self.initial_history = initial_history self.maximum_k = maximum_k self.channels = channels self.unknown = unknown self.max_coded_frame_size = max_coded_frame_size self.bitrate = bitrate self.sample_rate = sample_rate def __repr__(self): return "M4A_SUB_ALAC_Atom({})".format( ",".join(map(repr, [self.max_samples_per_frame, self.bits_per_sample, self.history_multiplier, self.initial_history, self.maximum_k, self.channels, self.unknown, self.max_coded_frame_size, self.bitrate, self.sample_rate]))) @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" return cls( *reader.parse( "4P 32u 8p 8u 8u 8u 8u 8u 16u 32u 32u 32u")) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.build("4P 32u 8p 8u 8u 8u 8u 8u 16u 32u 32u 32u", (self.max_samples_per_frame, self.bits_per_sample, self.history_multiplier, self.initial_history, self.maximum_k, self.channels, self.unknown, self.max_coded_frame_size, self.bitrate, self.sample_rate)) def size(self): """returns the atom's size not including its 64-bit size / name header""" return 28 class M4A_META_Atom(MetaData, M4A_Tree_Atom): UNICODE_ATTRIB_TO_ILST = {"track_name": b"\xa9nam", "album_name": b"\xa9alb", "artist_name": b"\xa9ART", "composer_name": b"\xa9wrt", "copyright": b"cprt", "performer_name": b"aART", "year": b"\xa9day", "comment": b"\xa9cmt"} INT_ATTRIB_TO_ILST = {"track_number": b"trkn", "album_number": b"disk"} TOTAL_ATTRIB_TO_ILST = {"track_total": b"trkn", "album_total": b"disk"} BOOL_ATTRIB_TO_ILST = {"compilation": b"cpil"} def __init__(self, version, flags, leaf_atoms): M4A_Tree_Atom.__init__(self, b"meta", leaf_atoms) MetaData.__setattr__(self, "version", version) MetaData.__setattr__(self, "flags", flags) def __repr__(self): return "M4A_META_Atom({!r}, {!r}, {!r})".format( self.version, self.flags, self.leaf_atoms) def has_ilst_atom(self): """returns True if this atom contains an ILST sub-atom""" for a in self.leaf_atoms: if a.name == b'ilst': return True else: return False def ilst_atom(self): """returns the first ILST sub-atom, or None""" for a in self.leaf_atoms: if a.name == b'ilst': return a else: return None def add_ilst_atom(self): """place new ILST atom after the first HDLR atom, if any""" for (index, atom) in enumerate(self.leaf_atoms): if atom.name == b'hdlr': self.leaf_atoms.insert(index, M4A_Tree_Atom(b'ilst', [])) break else: self.leaf_atoms.append(M4A_Tree_Atom(b'ilst', [])) def raw_info(self): """returns a Unicode string of low-level MetaData information whereas __unicode__ is meant to contain complete information at a very high level raw_info() should be more developer-specific and with very little adjustment or reordering to the data itself """ from os import linesep if self.has_ilst_atom(): comment_lines = [u"M4A:"] for atom in self.ilst_atom(): if hasattr(atom, "raw_info_lines"): comment_lines.extend(atom.raw_info_lines()) else: comment_lines.append( u"{} : ({:d} bytes)".format( atom.name.decode('ascii', 'replace'), atom.size())) return linesep.join(comment_lines) else: return u"" @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" assert(name == b"meta") (version, flags) = reader.parse("8u 24u") return cls(version, flags, parse_sub_atoms(data_size - 4, reader, parsers)) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.build("8u 24u", (self.version, self.flags)) for sub_atom in self: writer.build("32u 4b", (sub_atom.size() + 8, sub_atom.name)) sub_atom.build(writer) def size(self): """returns the atom's size not including its 64-bit size / name header""" return 4 + sum([8 + sub_atom.size() for sub_atom in self]) def __getattr__(self, attr): if attr in self.UNICODE_ATTRIB_TO_ILST: if self.has_ilst_atom(): try: return self.ilst_atom()[ self.UNICODE_ATTRIB_TO_ILST[attr]][b'data'].__unicode__() except KeyError: return None else: return None elif attr in self.INT_ATTRIB_TO_ILST: if self.has_ilst_atom(): try: return self.ilst_atom()[ self.INT_ATTRIB_TO_ILST[attr]][b'data'].number() except KeyError: return None else: return None elif attr in self.TOTAL_ATTRIB_TO_ILST: if self.has_ilst_atom(): try: return self.ilst_atom()[ self.TOTAL_ATTRIB_TO_ILST[attr]][b'data'].total() except KeyError: return None else: return None elif attr == 'compilation': if self.has_ilst_atom(): try: return (self.ilst_atom()[b'cpil'][b'data'].data == b'\x00\x00\x00\x15\x00\x00\x00\x00\x01') except KeyError: return None else: return None elif attr in self.FIELDS: return None else: raise AttributeError(attr) def __setattr__(self, attr, value): def new_data_atom(attribute, value): if attribute in self.UNICODE_ATTRIB_TO_ILST: return M4A_ILST_Unicode_Data_Atom(0, 1, value.encode('utf-8')) elif attribute == "track_number": return M4A_ILST_TRKN_Data_Atom(int(value), 0) elif attribute == "track_total": return M4A_ILST_TRKN_Data_Atom(0, int(value)) elif attribute == "album_number": return M4A_ILST_DISK_Data_Atom(int(value), 0) elif attribute == "album_total": return M4A_ILST_DISK_Data_Atom(0, int(value)) elif attribute == "compilation": return M4A_Leaf_Atom( b'data', b'\x00\x00\x00\x15\x00\x00\x00\x00\x01' if value else b'\x00\x00\x00\x15\x00\x00\x00\x00\x00') else: raise ValueError(value) def replace_data_atom(attribute, parent_atom, value): new_leaf_atoms = [] data_replaced = False for leaf_atom in parent_atom.leaf_atoms: if (leaf_atom.name == b'data') and (not data_replaced): if attribute == "track_number": new_leaf_atoms.append( M4A_ILST_TRKN_Data_Atom(int(value), leaf_atom.track_total)) elif attribute == "track_total": new_leaf_atoms.append( M4A_ILST_TRKN_Data_Atom(leaf_atom.track_number, int(value))) elif attribute == "album_number": new_leaf_atoms.append( M4A_ILST_DISK_Data_Atom(int(value), leaf_atom.disk_total)) elif attribute == "album_total": new_leaf_atoms.append( M4A_ILST_DISK_Data_Atom(leaf_atom.disk_number, int(value))) else: new_leaf_atoms.append(new_data_atom(attribute, value)) data_replaced = True else: new_leaf_atoms.append(leaf_atom) parent_atom.leaf_atoms = new_leaf_atoms if value is None: return delattr(self, attr) ilst_leaf = self.UNICODE_ATTRIB_TO_ILST.get( attr, self.INT_ATTRIB_TO_ILST.get( attr, self.TOTAL_ATTRIB_TO_ILST.get( attr, self.BOOL_ATTRIB_TO_ILST.get( attr, None)))) if ilst_leaf is not None: if not self.has_ilst_atom(): self.add_ilst_atom() # an ilst atom is present, so check its sub-atoms for ilst_atom in self.ilst_atom(): if ilst_atom.name == ilst_leaf: # atom already present, so adjust its data sub-atom replace_data_atom(attr, ilst_atom, value) break else: # atom not present, so append new parent and data sub-atom self.ilst_atom().add_child( M4A_ILST_Leaf_Atom(ilst_leaf, [new_data_atom(attr, value)])) else: # attribute is not an atom, so pass it through MetaData.__setattr__(self, attr, value) def __delattr__(self, attr): if self.has_ilst_atom(): ilst_atom = self.ilst_atom() if attr in self.UNICODE_ATTRIB_TO_ILST: ilst_atom.leaf_atoms = [ atom for atom in ilst_atom if atom.name != self.UNICODE_ATTRIB_TO_ILST[attr]] elif attr in self.BOOL_ATTRIB_TO_ILST: ilst_atom.leaf_atoms = [ atom for atom in ilst_atom if atom.name != self.BOOL_ATTRIB_TO_ILST[attr]] elif attr == "track_number": if self.track_total is None: # if track_number and track_total are both 0 # remove trkn atom ilst_atom.leaf_atoms = [ atom for atom in ilst_atom if atom.name != b"trkn"] else: self.track_number = 0 elif attr == "track_total": if self.track_number is None: # if track_number and track_total are both 0 # remove trkn atom ilst_atom.leaf_atoms = [ atom for atom in ilst_atom if atom.name != b"trkn"] else: self.track_total = 0 elif attr == "album_number": if self.album_total is None: # if album_number and album_total are both 0 # remove disk atom ilst_atom.leaf_atoms = [ atom for atom in ilst_atom if atom.name != b"disk"] else: self.album_number = 0 elif attr == "album_total": if self.album_number is None: # if album_number and album_total are both 0 # remove disk atom ilst_atom.leaf_atoms = [ atom for atom in ilst_atom if atom.name != b"disk"] else: self.album_total = 0 else: MetaData.__delattr__(self, attr) def images(self): """returns a list of embedded Image objects""" if self.has_ilst_atom(): return [atom[b'data'] for atom in self.ilst_atom() if ((atom.name == b'covr') and (atom.has_child(b'data')))] else: return [] def add_image(self, image): """embeds an Image object in this metadata""" def not_cover(atom): return not ((atom.name == b'covr') and (atom.has_child(b'data'))) if not self.has_ilst_atom(): self.add_ilst_atom() ilst_atom = self.ilst_atom() # filter out old cover image before adding new one ilst_atom.leaf_atoms = ( [atom for atom in ilst_atom if not_cover(atom)] + [M4A_ILST_Leaf_Atom(b'covr', [M4A_ILST_COVR_Data_Atom.converted( image)])]) def delete_image(self, image): """deletes an Image object from this metadata""" if self.has_ilst_atom(): ilst_atom = self.ilst_atom() ilst_atom.leaf_atoms = [ atom for atom in ilst_atom if not ((atom.name == b'covr') and (atom.has_child(b'data')) and (atom[b'data'].data == image.data))] @classmethod def converted(cls, metadata): """converts metadata from another class to this one, if necessary takes a MetaData-compatible object (or None) and returns a new MetaData subclass with the data fields converted""" if metadata is None: return None elif isinstance(metadata, cls): return cls(metadata.version, metadata.flags, [leaf.copy() for leaf in metadata]) ilst_atoms = [ M4A_ILST_Leaf_Atom( cls.UNICODE_ATTRIB_TO_ILST[attrib], [M4A_ILST_Unicode_Data_Atom(0, 1, value.encode('utf-8'))]) for (attrib, value) in metadata.filled_fields() if (attrib in cls.UNICODE_ATTRIB_TO_ILST)] if metadata.compilation: ilst_atoms.append( M4A_ILST_Leaf_Atom( b'cpil', [M4A_Leaf_Atom(b'data', b'\x00\x00\x00\x15\x00\x00\x00\x00\x01')])) if (((metadata.track_number is not None) or (metadata.track_total is not None))): ilst_atoms.append( M4A_ILST_Leaf_Atom( b'trkn', [M4A_ILST_TRKN_Data_Atom(metadata.track_number if (metadata.track_number is not None) else 0, metadata.track_total if (metadata.track_total is not None) else 0)])) if (((metadata.album_number is not None) or (metadata.album_total is not None))): ilst_atoms.append( M4A_ILST_Leaf_Atom( b'disk', [M4A_ILST_DISK_Data_Atom(metadata.album_number if (metadata.album_number is not None) else 0, metadata.album_total if (metadata.album_total is not None) else 0)])) if len(metadata.front_covers()) > 0: ilst_atoms.append( M4A_ILST_Leaf_Atom( b'covr', [M4A_ILST_COVR_Data_Atom.converted( metadata.front_covers()[0])])) return cls(0, 0, [M4A_HDLR_Atom(0, 0, b'\x00\x00\x00\x00', b'mdir', b'appl', 0, 0, b'', 0), M4A_Tree_Atom(b'ilst', ilst_atoms), M4A_FREE_Atom(1024)]) @classmethod def supports_images(self): """returns True""" return True def clean(self): """returns a new MetaData object that's been cleaned of problems any fixes performed are appended to fixes_performed as Unicode""" fixes_performed = [] def cleaned_atom(atom): # numerical fields are stored in bytes, # so no leading zeroes are possible # image fields don't store metadata, # so no field problems are possible there either from audiotools.text import (CLEAN_REMOVE_TRAILING_WHITESPACE, CLEAN_REMOVE_EMPTY_TAG) if atom.name in self.UNICODE_ATTRIB_TO_ILST.values(): text = atom[b'data'].data.decode('utf-8') fix1 = text.rstrip() if fix1 != text: fixes_performed.append( CLEAN_REMOVE_TRAILING_WHITESPACE.format( atom.name.lstrip(b'\xa9').decode('ascii'))) fix2 = fix1.lstrip() if fix2 != fix1: from audiotools.text import CLEAN_REMOVE_LEADING_WHITESPACE fixes_performed.append( CLEAN_REMOVE_LEADING_WHITESPACE.format( atom.name.lstrip(b'\xa9').decode('ascii'))) if len(fix2) > 0: return M4A_ILST_Leaf_Atom( atom.name, [M4A_ILST_Unicode_Data_Atom(0, 1, fix2.encode('utf-8'))]) else: fixes_performed.append( CLEAN_REMOVE_EMPTY_TAG.format( atom.name.lstrip(b'\xa9').decode('ascii'))) return None else: return atom if self.has_ilst_atom(): return (M4A_META_Atom( self.version, self.flags, [M4A_Tree_Atom(b'ilst', [atom for atom in map(cleaned_atom, self.ilst_atom()) if atom is not None])]), fixes_performed) else: # if no ilst atom, return a copy of the meta atom as-is return (M4A_META_Atom( self.version, self.flags, [M4A_Tree_Atom(b'ilst', [atom.copy() for atom in self.ilst_atom()])]), []) def intersection(self, metadata): """given a MetaData-compatible object, returns a new MetaData object which contains all the matching fields and images of this object and 'metadata' """ def atom_present(atom, ilst): for other_atom in ilst: if atom == other_atom: return True else: return False if type(metadata) is M4A_META_Atom: ilst1 = self.ilst_atom() ilst2 = metadata.ilst_atom() if (ilst1 is not None) and (ilst2 is not None): merged_ilst = M4A_Tree_Atom( ilst1.name, [atom.copy() for atom in ilst1 if atom_present(atom, ilst2)]) else: # one is missing an "ilst" sub-atom, so no common elements merged_ilst = M4A_Tree_Atom(b"ilst", []) return M4A_META_Atom( self.version, self.flags, [merged_ilst if (atom.name == b"ilst") else atom.copy() for atom in self.leaf_atoms]) else: return MetaData.intersection(self, metadata) class M4A_ILST_Leaf_Atom(M4A_Tree_Atom): def copy(self): """returns a newly copied instance of this atom and new instances of any sub-atoms it contains""" return M4A_ILST_Leaf_Atom(self.name, [leaf.copy() for leaf in self]) def __repr__(self): return "M4A_ILST_Leaf_Atom({!r}, {!r})".format( self.name, self.leaf_atoms) @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" return cls( name, parse_sub_atoms(data_size, reader, {b"data": {b"\xa9alb": M4A_ILST_Unicode_Data_Atom, b"\xa9ART": M4A_ILST_Unicode_Data_Atom, b"\xa9cmt": M4A_ILST_Unicode_Data_Atom, b"cprt": M4A_ILST_Unicode_Data_Atom, b"\xa9day": M4A_ILST_Unicode_Data_Atom, b"\xa9grp": M4A_ILST_Unicode_Data_Atom, b"\xa9nam": M4A_ILST_Unicode_Data_Atom, b"\xa9too": M4A_ILST_Unicode_Data_Atom, b"\xa9wrt": M4A_ILST_Unicode_Data_Atom, b'aART': M4A_ILST_Unicode_Data_Atom, b"covr": M4A_ILST_COVR_Data_Atom, b"trkn": M4A_ILST_TRKN_Data_Atom, b"disk": M4A_ILST_DISK_Data_Atom }.get(name, M4A_Leaf_Atom)})) if sys.version_info[0] >= 3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('utf-8') def __unicode__(self): try: return [l for l in self.leaf_atoms if l.name == b'data'][0].__unicode__() except IndexError: return u"" def raw_info_lines(self): """yields lines of human-readable information about the atom""" for leaf_atom in self.leaf_atoms: name = self.name.replace(b"\xa9", b" ").decode('ascii') if hasattr(leaf_atom, "raw_info"): yield u"{} : {}".format(name, leaf_atom.raw_info()) else: yield u"{} : {!r}".format(name, leaf_atom) # FIXME class M4A_ILST_Unicode_Data_Atom(M4A_Leaf_Atom): def __init__(self, type, flags, data): # assert(isinstance(data, bytes)) self.name = b"data" self.type = type self.flags = flags self.data = data def copy(self): """returns a newly copied instance of this atom and new instances of any sub-atoms it contains""" return M4A_ILST_Unicode_Data_Atom(self.type, self.flags, self.data) def __repr__(self): return "M4A_ILST_Unicode_Data_Atom({!r}, {!r}, {!r})".format( self.type, self.flags, self.data) def __eq__(self, atom): for attr in ["type", "flags", "data"]: if ((not hasattr(atom, attr)) or (getattr(self, attr) != getattr(atom, attr))): return False else: return True def raw_info(self): """returns a line of human-readable information about the atom""" return self.data.decode('utf-8') @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" assert(name == b"data") (type, flags) = reader.parse("8u 24u 32p") return cls(type, flags, reader.read_bytes(data_size - 8)) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.build("8u 24u 32p {:d}b".format(len(self.data)), (self.type, self.flags, self.data)) def size(self): """returns the atom's size not including its 64-bit size / name header""" return 8 + len(self.data) if sys.version_info[0] >= 3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('utf-8') def __unicode__(self): return self.data.decode('utf-8') class M4A_ILST_TRKN_Data_Atom(M4A_Leaf_Atom): def __init__(self, track_number, track_total): self.name = b"data" self.track_number = track_number self.track_total = track_total def copy(self): """returns a newly copied instance of this atom and new instances of any sub-atoms it contains""" return M4A_ILST_TRKN_Data_Atom(self.track_number, self.track_total) def __repr__(self): return "M4A_ILST_TRKN_Data_Atom({:d}, {:d})".format( self.track_number, self.track_total) def __eq__(self, atom): for attr in ["track_number", "track_total"]: if ((not hasattr(atom, attr)) or (getattr(self, attr) != getattr(atom, attr))): return False else: return True if sys.version_info[0] >= 3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('utf-8') def __unicode__(self): if self.track_total > 0: return u"{:d}/{:d}".format(self.track_number, self.track_total) else: return u"{:d}".format(self.track_number,) def raw_info(self): """returns a line of human-readable information about the atom""" return u"{:d}/{:d}".format(self.track_number, self.track_total) @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" assert(name == b"data") # FIXME - handle mis-sized TRKN data atoms return cls(*reader.parse("64p 16p 16u 16u 16p")) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.build("64p 16p 16u 16u 16p", (self.track_number, self.track_total)) def size(self): """returns the atom's size not including its 64-bit size / name header""" return 16 def number(self): """returns this atom's track_number field or None if the field is 0""" if self.track_number != 0: return self.track_number else: return None def total(self): """returns this atom's track_total field or None if the field is 0""" if self.track_total != 0: return self.track_total else: return None class M4A_ILST_DISK_Data_Atom(M4A_Leaf_Atom): def __init__(self, disk_number, disk_total): self.name = b"data" self.disk_number = disk_number self.disk_total = disk_total def copy(self): """returns a newly copied instance of this atom and new instances of any sub-atoms it contains""" return M4A_ILST_DISK_Data_Atom(self.disk_number, self.disk_total) def __repr__(self): return "M4A_ILST_DISK_Data_Atom({:d}, {:d})".format( self.disk_number, self.disk_total) def __eq__(self, atom): for attr in ["disk_number", "disk_total"]: if ((not hasattr(atom, attr)) or (getattr(self, attr) != getattr(atom, attr))): return False else: return True if sys.version_info[0] >= 3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('utf-8') def __unicode__(self): if self.disk_total > 0: return u"{:d}/{:d}".format(self.disk_number, self.disk_total) else: return u"{:d}".format(self.disk_number,) def raw_info(self): """returns a line of human-readable information about the atom""" return u"{:d}/{:d}".format(self.disk_number, self.disk_total) @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" assert(name == b"data") # FIXME - handle mis-sized DISK data atoms return cls(*reader.parse("64p 16p 16u 16u")) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.build("64p 16p 16u 16u", (self.disk_number, self.disk_total)) def size(self): """returns the atom's size not including its 64-bit size / name header""" return 14 def number(self): """returns this atom's disc_number field""" if self.disk_number != 0: return self.disk_number else: return None def total(self): """returns this atom's disk_total field""" if self.disk_total != 0: return self.disk_total else: return None class M4A_ILST_COVR_Data_Atom(Image, M4A_Leaf_Atom): def __init__(self, version, flags, image_data): # assert(isinstance(image_data, bytes)) self.version = version self.flags = flags self.name = b"data" img = image_metrics(image_data) Image.__init__(self, data=image_data, mime_type=img.mime_type, width=img.width, height=img.height, color_depth=img.bits_per_pixel, color_count=img.color_count, description=u"", type=0) def copy(self): """returns a newly copied instance of this atom and new instances of any sub-atoms it contains""" return M4A_ILST_COVR_Data_Atom(self.version, self.flags, self.data) def __repr__(self): return "M4A_ILST_COVR_Data_Atom({}, {}, ...)".format( self.version, self.flags) def raw_info(self): """returns a line of human-readable information about the atom""" from audiotools import hex_string if len(self.data) > 20: return u"({:d} bytes) {}\u2026".format( len(self.data), hex_string(self.data[0:20])) else: return u"({:d} bytes) {}".format( len(self.data), hex_string(self.data)) @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" assert(name == b"data") (version, flags) = reader.parse("8u 24u 32p") return cls(version, flags, reader.read_bytes(data_size - 8)) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.build("8u 24u 32p {:d}b".format(len(self.data)), (self.version, self.flags, self.data)) def size(self): """returns the atom's size not including its 64-bit size / name header""" return 8 + len(self.data) @classmethod def converted(cls, image): """given an Image-compatible object, returns a new M4A_ILST_COVR_Data_Atom object""" return cls(0, 0, image.data) class M4A_HDLR_Atom(M4A_Leaf_Atom): def __init__(self, version, flags, qt_type, qt_subtype, qt_manufacturer, qt_reserved_flags, qt_reserved_flags_mask, component_name, padding_size): # assert(isinstance(qt_type, bytes)) # assert(isinstance(qt_subtype, bytes)) # assert(isinstance(qt_manufacturer, bytes)) self.name = b'hdlr' self.version = version self.flags = flags self.qt_type = qt_type self.qt_subtype = qt_subtype self.qt_manufacturer = qt_manufacturer self.qt_reserved_flags = qt_reserved_flags self.qt_reserved_flags_mask = qt_reserved_flags_mask self.component_name = component_name self.padding_size = padding_size def copy(self): """returns a newly copied instance of this atom and new instances of any sub-atoms it contains""" return M4A_HDLR_Atom(self.version, self.flags, self.qt_type, self.qt_subtype, self.qt_manufacturer, self.qt_reserved_flags, self.qt_reserved_flags_mask, self.component_name, self.padding_size) def __repr__(self): return "M4A_HDLR_Atom({})".format( ",".join(map(repr, [self.version, self.flags, self.qt_type, self.qt_subtype, self.qt_manufacturer, self.qt_reserved_flags, self.qt_reserved_flags_mask, self.component_name, self.padding_size]))) @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" assert(name == b'hdlr') (version, flags, qt_type, qt_subtype, qt_manufacturer, qt_reserved_flags, qt_reserved_flags_mask) = reader.parse( "8u 24u 4b 4b 4b 32u 32u") component_name = reader.read_bytes(reader.read(8)) return cls(version, flags, qt_type, qt_subtype, qt_manufacturer, qt_reserved_flags, qt_reserved_flags_mask, component_name, data_size - len(component_name) - 25) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.build("8u 24u 4b 4b 4b 32u 32u 8u {:d}b {:d}P".format( len(self.component_name), self.padding_size), (self.version, self.flags, self.qt_type, self.qt_subtype, self.qt_manufacturer, self.qt_reserved_flags, self.qt_reserved_flags_mask, len(self.component_name), self.component_name)) def size(self): """returns the atom's size not including its 64-bit size / name header""" return 25 + len(self.component_name) + self.padding_size class M4A_FREE_Atom(M4A_Leaf_Atom): def __init__(self, bytes): self.name = b"free" self.bytes = bytes def copy(self): """returns a newly copied instance of this atom and new instances of any sub-atoms it contains""" return M4A_FREE_Atom(self.bytes) def __repr__(self): return "M4A_FREE_Atom({:d})".format(self.bytes) @classmethod def parse(cls, name, data_size, reader, parsers): """given a 4 byte name, data_size int, BitstreamReader and dict of {"atom":handler} sub-parsers, returns an atom of this class""" assert(name == b"free") reader.skip_bytes(data_size) return cls(data_size) def build(self, writer): """writes the atom to the given BitstreamWriter not including its 64-bit size / name header""" writer.write_bytes(b"\x00" * self.bytes) def size(self): """returns the atom's size not including its 64-bit size / name header""" return self.bytes ================================================ FILE: audiotools/mp3.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools import (AudioFile, InvalidFile) class InvalidMP3(InvalidFile): """raised by invalid files during MP3 initialization""" pass class MP3Audio(AudioFile): """an MP3 audio file""" from audiotools.text import (COMP_LAME_0, COMP_LAME_6, COMP_LAME_MEDIUM, COMP_LAME_STANDARD, COMP_LAME_EXTREME, COMP_LAME_INSANE) SUFFIX = "mp3" NAME = SUFFIX DESCRIPTION = u"MPEG-1 Audio Layer III" DEFAULT_COMPRESSION = "2" # 0 is better quality/lower compression # 9 is worse quality/higher compression COMPRESSION_MODES = ("0", "1", "2", "3", "4", "5", "6", "medium", "standard", "extreme", "insane") COMPRESSION_DESCRIPTIONS = {"0": COMP_LAME_0, "6": COMP_LAME_6, "medium": COMP_LAME_MEDIUM, "standard": COMP_LAME_STANDARD, "extreme": COMP_LAME_EXTREME, "insane": COMP_LAME_INSANE} SAMPLE_RATE = ((11025, 12000, 8000, None), # MPEG-2.5 (None, None, None, None), # reserved (22050, 24000, 16000, None), # MPEG-2 (44100, 48000, 32000, None)) # MPEG-1 BIT_RATE = ( # MPEG-2.5 ( # reserved (None,) * 16, # layer III (None, 8000, 16000, 24000, 32000, 40000, 48000, 56000, 64000, 80000, 96000, 112000, 128000, 144000, 160000, None), # layer II (None, 8000, 16000, 24000, 32000, 40000, 48000, 56000, 64000, 80000, 96000, 112000, 128000, 144000, 160000, None), # layer I (None, 32000, 48000, 56000, 64000, 80000, 96000, 112000, 128000, 144000, 160000, 176000, 192000, 224000, 256000, None), ), # reserved ((None,) * 16, ) * 4, # MPEG-2 ( # reserved (None,) * 16, # layer III (None, 8000, 16000, 24000, 32000, 40000, 48000, 56000, 64000, 80000, 96000, 112000, 128000, 144000, 160000, None), # layer II (None, 8000, 16000, 24000, 32000, 40000, 48000, 56000, 64000, 80000, 96000, 112000, 128000, 144000, 160000, None), # layer I (None, 32000, 48000, 56000, 64000, 80000, 96000, 112000, 128000, 144000, 160000, 176000, 192000, 224000, 256000, None)), # MPEG-1 ( # reserved (None,) * 16, # layer III (None, 32000, 40000, 48000, 56000, 64000, 80000, 96000, 112000, 128000, 160000, 192000, 224000, 256000, 320000, None), # layer II (None, 32000, 48000, 56000, 64000, 80000, 96000, 112000, 128000, 160000, 192000, 224000, 256000, 320000, 384000, None), # layer I (None, 32000, 64000, 96000, 128000, 160000, 192000, 224000, 256000, 288000, 320000, 352000, 384000, 416000, 448000, None))) PCM_FRAMES_PER_MPEG_FRAME = (None, 1152, 1152, 384) def __init__(self, filename): """filename is a plain string""" AudioFile.__init__(self, filename) from audiotools.bitstream import parse try: mp3file = open(filename, "rb") except IOError as msg: raise InvalidMP3(str(msg)) try: try: header_bytes = MP3Audio.__find_next_mp3_frame__(mp3file) except IOError: from audiotools.text import ERR_MP3_FRAME_NOT_FOUND raise InvalidMP3(ERR_MP3_FRAME_NOT_FOUND) (frame_sync, mpeg_id, layer, bit_rate, sample_rate, pad, channels) = parse("11u 2u 2u 1p 4u 2u 1u 1p 2u 6p", False, mp3file.read(4)) self.__samplerate__ = self.SAMPLE_RATE[mpeg_id][sample_rate] if self.__samplerate__ is None: from audiotools.text import ERR_MP3_INVALID_SAMPLE_RATE raise InvalidMP3(ERR_MP3_INVALID_SAMPLE_RATE) if channels in (0, 1, 2): self.__channels__ = 2 else: self.__channels__ = 1 first_frame = mp3file.read(self.frame_length(mpeg_id, layer, bit_rate, sample_rate, pad) - 4) if ((b"Xing" in first_frame) and (len(first_frame[first_frame.index(b"Xing"): first_frame.index(b"Xing") + 160]) == 160)): # pull length from Xing header, if present self.__pcm_frames__ = ( parse("32p 32p 32u 32p 832p", 0, first_frame[first_frame.index(b"Xing"): first_frame.index(b"Xing") + 160])[0] * self.PCM_FRAMES_PER_MPEG_FRAME[layer]) else: # otherwise, bounce through file frames from audiotools.bitstream import BitstreamReader reader = BitstreamReader(mp3file, False) self.__pcm_frames__ = 0 try: (frame_sync, mpeg_id, layer, bit_rate, sample_rate, pad) = reader.parse("11u 2u 2u 1p 4u 2u 1u 9p") while frame_sync == 0x7FF: self.__pcm_frames__ += \ self.PCM_FRAMES_PER_MPEG_FRAME[layer] reader.skip_bytes(self.frame_length(mpeg_id, layer, bit_rate, sample_rate, pad) - 4) (frame_sync, mpeg_id, layer, bit_rate, sample_rate, pad) = reader.parse("11u 2u 2u 1p 4u 2u 1u 9p") except IOError: pass except ValueError as err: raise InvalidMP3(err) finally: mp3file.close() def lossless(self): """returns False""" return False @classmethod def supports_to_pcm(cls): """returns True if all necessary components are available to support the .to_pcm() method""" try: from audiotools.decoders import MP3Decoder return True except ImportError: return False def to_pcm(self): """returns a PCMReader object containing the track's PCM data""" from audiotools.decoders import MP3Decoder return MP3Decoder(self.filename) @classmethod def supports_from_pcm(cls): """returns True if all necessary components are available to support the .from_pcm() classmethod""" try: from audiotools.encoders import encode_mp3 return True except ImportError: return False @classmethod def from_pcm(cls, filename, pcmreader, compression=None, total_pcm_frames=None): """encodes a new file from PCM data takes a filename string, PCMReader object, optional compression level string and optional total_pcm_frames integer encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new MP3Audio object""" from audiotools import (PCMConverter, BufferedPCMReader, ChannelMask, __default_quality__, EncodingError) from audiotools.encoders import encode_mp3 if (((compression is None) or (compression not in cls.COMPRESSION_MODES))): compression = __default_quality__(cls.NAME) if pcmreader.bits_per_sample not in {8, 16, 24}: from audiotools import UnsupportedBitsPerSample pcmreader.close() raise UnsupportedBitsPerSample(filename, pcmreader.bits_per_sample) try: if total_pcm_frames is not None: from audiotools import CounterPCMReader pcmreader = CounterPCMReader(pcmreader) encode_mp3(filename, PCMConverter(pcmreader, sample_rate=pcmreader.sample_rate, channels=min(pcmreader.channels, 2), channel_mask=ChannelMask.from_channels( min(pcmreader.channels, 2)), bits_per_sample=16), compression) if ((total_pcm_frames is not None) and (total_pcm_frames != pcmreader.frames_written)): from audiotools.text import ERR_TOTAL_PCM_FRAMES_MISMATCH cls.__unlink__(filename) raise EncodingError(ERR_TOTAL_PCM_FRAMES_MISMATCH) return MP3Audio(filename) except (ValueError, IOError) as err: cls.__unlink__(filename) raise EncodingError(str(err)) finally: pcmreader.close() def bits_per_sample(self): """returns an integer number of bits-per-sample this track contains""" return 16 def channels(self): """returns an integer number of channels this track contains""" return self.__channels__ def sample_rate(self): """returns the rate of the track's audio as an integer number of Hz""" return self.__samplerate__ @classmethod def supports_metadata(cls): """returns True if this audio type supports MetaData""" return True def get_metadata(self): """returns a MetaData object, or None raises IOError if unable to read the file""" from audiotools.id3 import ID3CommentPair from audiotools.id3 import read_id3v2_comment from audiotools.id3v1 import ID3v1Comment with open(self.filename, "rb") as f: if f.read(3) == b"ID3": id3v2 = read_id3v2_comment(self.filename) try: # yes IDv2, yes ID3v1 return ID3CommentPair(id3v2, ID3v1Comment.parse(f)) except ValueError: # yes ID3v2, no ID3v1 return id3v2 else: try: # no ID3v2, yes ID3v1 return ID3v1Comment.parse(f) except ValueError: # no ID3v2, no ID3v1 return None def update_metadata(self, metadata): """takes this track's current MetaData object as returned by get_metadata() and sets this track's metadata with any fields updated in that object raises IOError if unable to write the file """ import os from audiotools import (TemporaryFile, LimitedFileReader, transfer_data) from audiotools.id3 import (ID3v2Comment, ID3CommentPair) from audiotools.id3v1 import ID3v1Comment from audiotools.bitstream import BitstreamWriter if metadata is None: return elif (not (isinstance(metadata, ID3v2Comment) or isinstance(metadata, ID3CommentPair) or isinstance(metadata, ID3v1Comment))): from audiotools.text import ERR_FOREIGN_METADATA raise ValueError(ERR_FOREIGN_METADATA) elif not os.access(self.filename, os.W_OK): raise IOError(self.filename) new_mp3 = TemporaryFile(self.filename) # get the original MP3 data old_mp3 = open(self.filename, "rb") MP3Audio.__find_last_mp3_frame__(old_mp3) data_end = old_mp3.tell() old_mp3.seek(0, 0) MP3Audio.__find_mp3_start__(old_mp3) data_start = old_mp3.tell() old_mp3 = LimitedFileReader(old_mp3, data_end - data_start) # write id3v2 + data + id3v1 to file if isinstance(metadata, ID3CommentPair): metadata.id3v2.build(BitstreamWriter(new_mp3, False)) transfer_data(old_mp3.read, new_mp3.write) metadata.id3v1.build(new_mp3) elif isinstance(metadata, ID3v2Comment): metadata.build(BitstreamWriter(new_mp3, False)) transfer_data(old_mp3.read, new_mp3.write) elif isinstance(metadata, ID3v1Comment): transfer_data(old_mp3.read, new_mp3.write) metadata.build(new_mp3) # commit change to disk old_mp3.close() new_mp3.close() def set_metadata(self, metadata): """takes a MetaData object and sets this track's metadata this metadata includes track name, album name, and so on raises IOError if unable to write the file""" from audiotools.id3 import ID3v2Comment from audiotools.id3 import ID3v22Comment from audiotools.id3 import ID3v23Comment from audiotools.id3 import ID3v24Comment from audiotools.id3 import ID3CommentPair from audiotools.id3v1 import ID3v1Comment if metadata is None: return self.delete_metadata() if (not (isinstance(metadata, ID3v2Comment) or isinstance(metadata, ID3CommentPair) or isinstance(metadata, ID3v1Comment))): from audiotools import config DEFAULT_ID3V2 = "id3v2.3" DEFAULT_ID3V1 = "id3v1.1" id3v2_class = {"id3v2.2": ID3v22Comment, "id3v2.3": ID3v23Comment, "id3v2.4": ID3v24Comment, "none": None}.get(config.get_default("ID3", "id3v2", DEFAULT_ID3V2), DEFAULT_ID3V2) id3v1_class = {"id3v1.1": ID3v1Comment, "none": None}.get(config.get_default("ID3", "id3v1", DEFAULT_ID3V1), DEFAULT_ID3V1) if (id3v2_class is not None) and (id3v1_class is not None): self.update_metadata( ID3CommentPair.converted(metadata, id3v2_class=id3v2_class, id3v1_class=id3v1_class)) elif id3v2_class is not None: self.update_metadata(id3v2_class.converted(metadata)) elif id3v1_class is not None: self.update_metadata(id3v1_class.converted(metadata)) else: return else: self.update_metadata(metadata) def delete_metadata(self): """deletes the track's MetaData this removes or unsets tags as necessary in order to remove all data raises IOError if unable to write the file""" import os from audiotools import (TemporaryFile, LimitedFileReader, transfer_data) # this works a lot like update_metadata # but without any new metadata to set if not os.access(self.filename, os.W_OK): raise IOError(self.filename) new_mp3 = TemporaryFile(self.filename) # get the original MP3 data old_mp3 = open(self.filename, "rb") MP3Audio.__find_last_mp3_frame__(old_mp3) data_end = old_mp3.tell() old_mp3.seek(0, 0) MP3Audio.__find_mp3_start__(old_mp3) data_start = old_mp3.tell() old_mp3 = LimitedFileReader(old_mp3, data_end - data_start) # write data to file transfer_data(old_mp3.read, new_mp3.write) # commit change to disk old_mp3.close() new_mp3.close() def clean(self, output_filename=None): """cleans the file of known data and metadata problems output_filename is an optional filename of the fixed file if present, a new AudioFile is written to that path otherwise, only a dry-run is performed and no new file is written return list of fixes performed as Unicode strings raises IOError if unable to write the file or its metadata raises ValueError if the file has errors of some sort """ from audiotools.id3 import total_id3v2_comments from audiotools import transfer_data from audiotools import open as open_audiofile from audiotools.text import CLEAN_REMOVE_DUPLICATE_ID3V2 with open(self.filename, "rb") as f: if total_id3v2_comments(f) > 1: file_fixes = [CLEAN_REMOVE_DUPLICATE_ID3V2] else: file_fixes = [] if output_filename is None: # dry run only metadata = self.get_metadata() if metadata is not None: (metadata, fixes) = metadata.clean() return file_fixes + fixes else: return file_fixes else: # perform complete fix input_f = open(self.filename, "rb") output_f = open(output_filename, "wb") try: transfer_data(input_f.read, output_f.write) finally: input_f.close() output_f.close() new_track = open_audiofile(output_filename) metadata = self.get_metadata() if metadata is not None: (metadata, fixes) = metadata.clean() if len(file_fixes + fixes) > 0: # only update metadata if fixes are actually performed new_track.update_metadata(metadata) return file_fixes + fixes else: return file_fixes # places mp3file at the position of the next MP3 frame's start @classmethod def __find_next_mp3_frame__(cls, mp3file): from audiotools.id3 import skip_id3v2_comment # if we're starting at an ID3v2 header, skip it to save a bunch of time bytes_skipped = skip_id3v2_comment(mp3file) # then find the next mp3 frame from audiotools.bitstream import BitstreamReader reader = BitstreamReader(mp3file, False) pos = reader.getpos() try: (sync, mpeg_id, layer_description) = reader.parse("11u 2u 2u 1p") except IOError as err: raise err while (not ((sync == 0x7FF) and (mpeg_id in (0, 2, 3)) and (layer_description in (1, 2, 3)))): reader.setpos(pos) reader.skip(8) bytes_skipped += 1 pos = reader.getpos() try: (sync, mpeg_id, layer_description) = reader.parse("11u 2u 2u 1p") except IOError as err: raise err else: reader.setpos(pos) return bytes_skipped @classmethod def __find_mp3_start__(cls, mp3file): """places mp3file at the position of the MP3 file's start""" from audiotools.id3 import skip_id3v2_comment # if we're starting at an ID3v2 header, skip it to save a bunch of time skip_id3v2_comment(mp3file) from audiotools.bitstream import BitstreamReader reader = BitstreamReader(mp3file, False) # skip over any bytes that aren't a valid MPEG header pos = reader.getpos() (frame_sync, mpeg_id, layer) = reader.parse("11u 2u 2u 1p") while (not ((frame_sync == 0x7FF) and (mpeg_id in (0, 2, 3)) and (layer in (1, 2, 3)))): reader.setpos(pos) reader.skip(8) pos = reader.getpos() reader.setpos(pos) @classmethod def __find_last_mp3_frame__(cls, mp3file): """places mp3file at the position of the last MP3 frame's end (either the last byte in the file or just before the ID3v1 tag) this may not be strictly accurate if ReplayGain data is present, since APEv2 tags came before the ID3v1 tag, but we're not planning to change that tag anyway """ mp3file.seek(-128, 2) if mp3file.read(3) == b'TAG': mp3file.seek(-128, 2) return else: mp3file.seek(0, 2) return def frame_length(self, mpeg_id, layer, bit_rate, sample_rate, pad): """returns the total MP3 frame length in bytes the given arguments are the header's bit values mpeg_id = 2 bits layer = 2 bits bit_rate = 4 bits sample_rate = 2 bits pad = 1 bit """ sample_rate = self.SAMPLE_RATE[mpeg_id][sample_rate] if sample_rate is None: from audiotools.text import ERR_MP3_INVALID_SAMPLE_RATE raise ValueError(ERR_MP3_INVALID_SAMPLE_RATE) bit_rate = self.BIT_RATE[mpeg_id][layer][bit_rate] if bit_rate is None: from audiotools.text import ERR_MP3_INVALID_BIT_RATE raise ValueError(ERR_MP3_INVALID_BIT_RATE) if layer == 3: # layer I return (((12 * bit_rate) // sample_rate) + pad) * 4 else: # layer II/III return ((144 * bit_rate) // sample_rate) + pad def total_frames(self): """returns the total PCM frames of the track as an integer""" return self.__pcm_frames__ class MP2Audio(MP3Audio): """an MP2 audio file""" from audiotools.text import (COMP_TWOLAME_64, COMP_TWOLAME_384) SUFFIX = "mp2" NAME = SUFFIX DESCRIPTION = u"MPEG-1 Audio Layer II" DEFAULT_COMPRESSION = str(192) COMPRESSION_MODES = tuple(map(str, (64, 96, 112, 128, 160, 192, 224, 256, 320, 384))) COMPRESSION_DESCRIPTIONS = {"64": COMP_TWOLAME_64, "384": COMP_TWOLAME_384} @classmethod def supports_from_pcm(cls): """returns True if all necessary components are available to support the .from_pcm() classmethod""" try: from audiotools.encoders import encode_mp2 return True except ImportError: return False @classmethod def from_pcm(cls, filename, pcmreader, compression=None, total_pcm_frames=None): """encodes a new file from PCM data takes a filename string, PCMReader object, optional compression level string and optional total_pcm_frames integer encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new MP2Audio object""" from audiotools import (PCMConverter, BufferedPCMReader, ChannelMask, __default_quality__, EncodingError) from audiotools.encoders import encode_mp2 if (((compression is None) or (compression not in cls.COMPRESSION_MODES))): compression = __default_quality__(cls.NAME) if pcmreader.bits_per_sample not in {8, 16, 24}: from audiotools import UnsupportedBitsPerSample pcmreader.close() raise UnsupportedBitsPerSample(filename, pcmreader.bits_per_sample) if pcmreader.sample_rate in (32000, 48000, 44100): sample_rate = pcmreader.sample_rate if total_pcm_frames is not None: from audiotools import CounterPCMReader pcmreader = CounterPCMReader(pcmreader) else: from bisect import bisect sample_rate = [32000, 32000, 44100, 48000][bisect([32000, 44100, 48000], pcmreader.sample_rate)] total_pcm_frames = None try: encode_mp2(filename, PCMConverter(pcmreader, sample_rate=sample_rate, channels=min(pcmreader.channels, 2), channel_mask=ChannelMask.from_channels( min(pcmreader.channels, 2)), bits_per_sample=16), int(compression)) if ((total_pcm_frames is not None) and (total_pcm_frames != pcmreader.frames_written)): from audiotools.text import ERR_TOTAL_PCM_FRAMES_MISMATCH cls.__unlink__(filename) raise EncodingError(ERR_TOTAL_PCM_FRAMES_MISMATCH) return MP2Audio(filename) except (ValueError, IOError) as err: cls.__unlink__(filename) raise EncodingError(str(err)) finally: pcmreader.close() ================================================ FILE: audiotools/mpc.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 James Buren and Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools import (AudioFile, InvalidFile) from audiotools.ape import ApeTaggedAudio from audiotools.bitstream import BitstreamReader, BitstreamWriter class MPC_Size: def __init__(self, value, length): self.__value__ = value self.__length__ = length def __repr__(self): return "MPC_Size({!r}, {!r})".format(self.__value__, self.__length__) def __int__(self): return self.__value__ def __len__(self): return self.__length__ @classmethod def parse(cls, reader): cont, value = reader.parse("1u 7u") length = 1 while cont == 1: cont, value2 = reader.parse("1u 7u") value = (value << 7) | value2 length += 1 return cls(value, length) def build(self, writer): for i in reversed(range(self.__length__)): writer.write(1, 1 if (i > 0) else 0) writer.write(7, (self.__value__ >> (i * 7)) & 0x7F) class InvalidMPC(InvalidFile): """raised by invalid files during MPC initialization""" pass class MPCAudio(ApeTaggedAudio, AudioFile): """an MPC audio file""" SUFFIX = "mpc" NAME = SUFFIX DESCRIPTION = u"MusePack" DEFAULT_COMPRESSION = "5" # Ranges from 0 to 10. Lower levels mean lower kbps, and therefore # lower quality. COMPRESSION_MODES = tuple(map(str, range(0, 11))) COMPRESSION_DESCRIPTIONS = {"0": u"poor quality (~20 kbps)", "1": u"poor quality (~30 kbps)", "2": u"low quality (~60 kbps)", "3": u"low/medium quality (~90 kbps)", "4": u"medium quality (~130 kbps)", "5": u"high quality (~180 kbps)", "6": u"excellent quality (~210 kbps)", "7": u"excellent quality (~240 kbps)", "8": u"excellent quality (~270 kbps)", "9": u"excellent quality (~300 kbps)", "10": u"excellent quality (~350 kbps)"} def __init__(self, filename): """filename is a plain string""" AudioFile.__init__(self, filename) try: block = BitstreamReader(self.get_block(b"SH"), False) crc = block.read(32) if block.read(8) != 8: from audiotools.text import ERR_MPC_INVALID_VERSION raise InvalidMPC(ERR_MPC_INVALID_VERSION) self.__samples__ = int(MPC_Size.parse(block)) beg_silence = int(MPC_Size.parse(block)) self.__sample_rate__ = \ [44100, 48000, 37800, 32000][block.read(3)] max_band = block.read(5) + 1 self.__channels__ = block.read(4) + 1 ms = block.read(1) block_pwr = block.read(3) * 2 except IOError as err: raise InvalidMPC(str(err)) def blocks(self): with BitstreamReader(open(self.filename, "rb"), False) as r: if r.read_bytes(4) != b"MPCK": from audiotools.text import ERR_MPC_INVALID_ID raise InvalidMPC(ERR_MPC_INVALID_ID) key = r.read_bytes(2) size = MPC_Size.parse(r) while key != b"SE": yield key, size, r.read_bytes(int(size) - len(size) - 2) key = r.read_bytes(2) size = MPC_Size.parse(r) yield key, size, r.read_bytes(int(size) - len(size) - 2) def get_block(self, block_id): for key, size, block in self.blocks(): if key == block_id: return block else: raise KeyError(block_id) def bits_per_sample(self): """returns an integer number of bits-per-sample this track contains""" return 16 def channels(self): """returns an integer number of channels this track contains""" return self.__channels__ def lossless(self): """returns True if this track's data is stored losslessly""" return False def total_frames(self): """returns the total PCM frames of the track as an integer""" return self.__samples__ def sample_rate(self): """returns the rate of the track's audio as an integer number of Hz""" return self.__sample_rate__ @classmethod def supports_to_pcm(cls): """returns True if all necessary components are available to support the .to_pcm() method""" try: from audiotools.decoders import MPCDecoder return True except ImportError: return False def to_pcm(self): """returns a PCMReader object containing the track's PCM data if an error occurs initializing a decoder, this should return a PCMReaderError with an appropriate error message""" from audiotools.decoders import MPCDecoder try: return MPCDecoder(self.filename) except (IOError, ValueError) as err: from audiotools import PCMReaderError return PCMReaderError(error_message=str(err), sample_rate=self.sample_rate(), channels=self.channels(), channel_mask=int(self.channel_mask()), bits_per_sample=self.bits_per_sample()) @classmethod def supports_from_pcm(cls): """returns True if all necessary components are available to support the .from_pcm() classmethod""" try: from audiotools.encoders import encode_mpc return True except ImportError: return False @classmethod def from_pcm(cls, filename, pcmreader, compression=None, total_pcm_frames=None): from audiotools import __default_quality__ from audiotools import PCMConverter from audiotools import ChannelMask from audiotools.encoders import encode_mpc if (compression is None) or (compression not in cls.COMPRESSION_MODES): compression = __default_quality__(cls.NAME) if pcmreader.bits_per_sample not in {8, 16, 24}: from audiotools import UnsupportedBitsPerSample pcmreader.close() raise UnsupportedBitsPerSample(filename, pcmreader.bits_per_sample) if pcmreader.sample_rate in (32000, 37800, 44100, 48000): sample_rate = pcmreader.sample_rate if total_pcm_frames is not None: from audiotools import CounterPCMReader pcmreader = CounterPCMReader(pcmreader) else: from bisect import bisect sample_rate = [32000, 32000, 37800, 44100, 48000][bisect([32000, 37800, 44100, 4800], pcmreader.sample_rate)] total_pcm_frames = None try: encode_mpc( filename, PCMConverter(pcmreader, sample_rate=sample_rate, channels=min(pcmreader.channels, 2), channel_mask=int(ChannelMask.from_channels( min(pcmreader.channels, 2))), bits_per_sample=16), float(compression), total_pcm_frames if (total_pcm_frames is not None) else 0) # ensure PCM frames match, if indicated if ((total_pcm_frames is not None) and (total_pcm_frames != pcmreader.frames_written)): from audiotools.text import ERR_TOTAL_PCM_FRAMES_MISMATCH from audiotools import EncodingError raise EncodingError(ERR_TOTAL_PCM_FRAMES_MISMATCH) return MPCAudio(filename) except (IOError, ValueError) as err: from audiotools import EncodingError cls.__unlink__(filename) raise EncodingError(str(err)) except Exception: cls.__unlink__(filename) raise finally: pcmreader.close() @classmethod def supports_replay_gain(cls): """returns True if this class supports ReplayGain""" return True def get_replay_gain(self): """returns a ReplayGain object of our ReplayGain values returns None if we have no values may raise IOError if unable to read the file""" from audiotools import ReplayGain try: rg = BitstreamReader(self.get_block(b"RG"), False) except KeyError: return None version = rg.read(8) if version != 1: return None gain_title = rg.read(16) peak_title = rg.read(16) gain_album = rg.read(16) peak_album = rg.read(16) if ((gain_title == 0) and (peak_title == 0) and (gain_album == 0) and (peak_album == 0)): return None else: return ReplayGain( track_gain=64.82 - float(gain_title) / 256, track_peak=(10 ** (float(peak_title) / 256 / 20)) / 2 ** 15, album_gain=64.82 - float(gain_album) / 256, album_peak=(10 ** (float(peak_album) / 256 / 20)) / 2 ** 15) def set_replay_gain(self, replaygain): """given a ReplayGain object, sets the track's gain to those values may raise IOError if unable to modify the file""" from math import log10 from audiotools import TemporaryFile gain_title = int(round((64.82 - replaygain.track_gain) * 256)) if replaygain.track_peak > 0.0: peak_title = int(log10(replaygain.track_peak * 2 ** 15) * 20 * 256) else: peak_title = 0 gain_album = int(round((64.82 - replaygain.album_gain) * 256)) if replaygain.album_peak > 0.0: peak_album = int(log10(replaygain.album_peak * 2 ** 15) * 20 * 256) else: peak_album = 0 #FIXME - check for missing "RG" block and add one if not present metadata = self.get_metadata() writer = BitstreamWriter(TemporaryFile(self.filename), False) writer.write_bytes(b"MPCK") for key, size, block in self.blocks(): if key != b"RG": writer.write_bytes(key) size.build(writer) writer.write_bytes(block) else: writer.write_bytes(b"RG") MPC_Size(2 + 1 + 1 + 2 * 4, 1).build(writer) writer.write(8, 1) writer.write(16, gain_title) writer.write(16, peak_title) writer.write(16, gain_album) writer.write(16, peak_album) if metadata is not None: writer.set_endianness(True) metadata.build(writer) writer.close() def delete_replay_gain(self): """removes ReplayGain values from file, if any may raise IOError if unable to modify the file""" from audiotools import TemporaryFile writer = BitstreamWriter(TemporaryFile(self.filename), False) writer.write_bytes(b"MPCK") for key, size, block in self.blocks(): if key != b"RG": writer.write_bytes(key) size.build(writer) writer.write_bytes(block) else: writer.write_bytes(b"RG") MPC_Size(2 + 1 + 1 + 2 * 4, 1).build(writer) writer.write(8, 1) writer.write(16, 0) writer.write(16, 0) writer.write(16, 0) writer.write(16, 0) writer.close() ================================================ FILE: audiotools/musicbrainz.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA import sys class DiscID(object): def __init__(self, first_track_number, last_track_number, lead_out_offset, offsets): """first_track_number and last_track_number are ints typically starting from 1 lead_out_offset is an integer number of CD frames offsets is a list of track offsets, in CD frames""" assert((last_track_number - first_track_number + 1) == len(offsets)) self.first_track_number = first_track_number self.last_track_number = last_track_number self.lead_out_offset = lead_out_offset self.offsets = offsets @classmethod def from_cddareader(cls, cddareader): """given a CDDAReader object, returns a DiscID for that object""" offsets = cddareader.track_offsets return cls(first_track_number=min(offsets.keys()), last_track_number=max(offsets.keys()), lead_out_offset=cddareader.last_sector + 150 + 1, offsets=[(offsets[k] // 588) + 150 for k in sorted(offsets.keys())]) @classmethod def from_tracks(cls, tracks): """given a sorted list of tracks, returns DiscID for those tracks as if they were a CD""" from audiotools import has_pre_gap_track if not has_pre_gap_track(tracks): offsets = [150] for track in tracks[0:-1]: offsets.append(offsets[-1] + track.cd_frames()) return cls( first_track_number=1, last_track_number=len(tracks), lead_out_offset=sum([t.cd_frames() for t in tracks]) + 150, offsets=offsets) else: offsets = [150 + tracks[0].cd_frames()] for track in tracks[1:-1]: offsets.append(offsets[-1] + track.cd_frames()) return cls( first_track_number=1, last_track_number=len(tracks) - 1, lead_out_offset=sum([t.cd_frames() for t in tracks]) + 150, offsets=offsets) @classmethod def from_sheet(cls, sheet, total_pcm_frames, sample_rate): """given a Sheet object length of the album in PCM frames and sample rate of the disc, returns a DiscID for that CD""" return cls( first_track_number=1, last_track_number=len(sheet), lead_out_offset=(total_pcm_frames * 75 // sample_rate) + 150, offsets=[int(t.index(1).offset() * 75 + 150) for t in sheet]) def __repr__(self): return "DiscID({})".format( ", ".join(["{}={!r}".format(attr, getattr(self, attr)) for attr in ["first_track_number", "last_track_number", "lead_out_offset", "offsets"]])) if sys.version_info[0] >= 3: def __str__(self): return self.__unicode__() else: def __str__(self): return self.__unicode__().encode('ascii') def __unicode__(self): from hashlib import sha1 from base64 import b64encode raw_id = u"{:02X}{:02X}{}".format( self.first_track_number, self.last_track_number, u"".join([u"{:08X}".format(offset) for offset in [self.lead_out_offset] + self.offsets + [0] * (99 - len(self.offsets))])) return b64encode(sha1(raw_id.encode("ascii")).digest(), b"._").replace(b"=", b"-").decode('ascii') def perform_lookup(disc_id, musicbrainz_server, musicbrainz_port): """performs a web-based lookup using the given DiscID iterates over a list of MetaData objects per successful match, like: [track1, track2, ...], [track1, track2, ...], ... may raise HTTPError if an error occurs querying the server or xml.parsers.expat.ExpatError if there's an error parsing the data """ try: from urllib.request import urlopen except ImportError: from urllib2 import urlopen try: from urllib.parse import urlencode except ImportError: from urllib import urlencode import xml.dom.minidom from audiotools.coverartarchive import perform_lookup as image_lookup # query MusicBrainz web service (version 2) for <metadata> m = urlopen("http://{}:{:d}/ws/2/discid/{}?{}".format( musicbrainz_server, musicbrainz_port, disc_id, urlencode({"inc": "artists labels recordings"}))) xml = xml.dom.minidom.parse(m) m.close() # for each <release>s in <release-list> # yield a list of MetaData objects try: release_list = get_node(xml, u"metadata", u"disc", u"release-list") for release in get_nodes(release_list, u"release"): release_metadata = list(parse_release(release, disc_id)) if release.hasAttribute("id"): release_id = release.getAttribute("id") for image in image_lookup(release_id): for track_metadata in release_metadata: track_metadata.add_image(image) yield release_metadata except KeyError: # no releases found, so return nothing return def get_node(parent, *nodes): """given a minidom tree element and a list of node unicode strings indicating a path, returns the node at the end of that path or raises KeyError if any node in the path cannot be found""" if len(nodes) == 0: return parent else: for child in parent.childNodes: if hasattr(child, "tagName") and (child.tagName == nodes[0]): return get_node(child, *nodes[1:]) else: raise KeyError(nodes[0]) def get_nodes(parent, node): """given a minidom tree element and a tag name unicode string, returns all the child nodes with that name""" return [child for child in parent.childNodes if (hasattr(child, "tagName") and (child.tagName == node))] def text(node): """given a minidom leaf node element, returns its data as a unicode string""" if node.firstChild is not None: return node.firstChild.data else: return u"" def artist(artist_credit): """given an <artist-credit> DOM element, returns the artist as a unicode string""" artists = [] # <artist-credit> must contain at least one <name-credit> for name_credit in get_nodes(artist_credit, u"name-credit"): try: # <name-credit> must contain <artist> # but <artist> need not contain <name> artists.append(text(get_node(name_credit, u"artist", u"name"))) except KeyError: artists.append(u"") # <name-credit> may contain a "joinphrase" attribute if name_credit.hasAttribute(u"joinphrase"): artists.append(name_credit.getAttribute(u"joinphrase")) return u"".join(artists) def parse_release(release, disc_id): """given a <release> Element node and DiscID object yields a populated MetaData object per track may raise KeyError if the given DiscID is not found in the <release>""" # <release> may contain <title> try: album_name = text(get_node(release, u"title")) except KeyError: album_name = None # <release> may contain <artist-credit> try: album_artist = artist(get_node(release, u"artist-credit")) except KeyError: album_artist = None # <release> may contain <label-info-list> try: # <label-info-list> contains 0 or more <label-info>s for label_info in get_nodes(get_node(release, u"label-info-list"), u"label-info"): # <label-info> may contain <catalog-number> try: catalog = text(get_node(label_info, u"catalog-number")) except KeyError: catalog = None # <label-info> may contain <label> # and <label> may contain <name> try: publisher = text(get_node(label_info, u"label", u"name")) except KeyError: publisher = None # we'll use the first result found break else: # <label-info-list> with no <label-info> tags catalog = None publisher = None except KeyError: catalog = None publisher = None # <release> may contain <date> try: year = text(get_node(release, u"date"))[0:4] except: year = None # find exact disc in <medium-list> tag # depending on disc_id value try: medium_list = get_node(release, u"medium-list") except KeyError: # no media found for disc ID raise KeyError(disc_id) for medium in get_nodes(medium_list, u"medium"): try: if (disc_id.__unicode__() in [disc.getAttribute(u"id") for disc in get_nodes(get_node(medium, u"disc-list"), u"disc")]): # found requested disc_id in <medium>'s list of <disc>s # so use that medium node to find additional info break except KeyError: # no <disc-list> tag found in <medium> continue else: # our disc_id wasn't found in any of the <release>'s <medium>s raise KeyError(disc_id) # if multiple discs in <medium-list>, # populate album number and album total if ((medium_list.hasAttribute(u"count") and (int(medium_list.getAttribute(u"count")) > 1))): album_total = int(medium_list.getAttribute(u"count")) try: album_number = int(text(get_node(medium, u"position"))) except KeyError: album_number = None else: album_total = album_number = None # <medium> must contain <track-list> tracks = get_nodes(get_node(medium, u"track-list"), u"track") track_total = len(tracks) # and <track-list> contains 0 or more <track>s for (i, track) in enumerate(tracks, 1): # if <track> contains title use that for track_name try: track_name = text(get_node(track, u"title")) except KeyError: track_name = None # if <track> contains <artist-credit> use that for track_artist try: track_artist = artist(get_node(release, u"artist-credit")) except KeyError: track_artist = None # if <track> contains a <recording> # use that for track_name and track artist try: recording = get_node(track, u"recording") # <recording> may contain a <title> if track_name is None: try: track_name = text(get_node(recording, u"title")) except KeyError: track_name = None # <recording> may contain <artist-credit> if track_artist is None: try: track_artist = artist(get_node(recording, u"artist-credit")) except KeyError: track_artist = album_artist except KeyError: # no <recording> in <track> if track_artist is None: track_artist = album_artist # <track> may contain a <position> try: track_number = int(text(get_node(track, u"position"))) except KeyError: track_number = i # yield complete MetaData object from audiotools import MetaData yield MetaData(track_name=track_name, track_number=track_number, track_total=track_total, album_name=album_name, artist_name=track_artist, performer_name=None, composer_name=None, conductor_name=None, ISRC=None, catalog=catalog, copyright=None, publisher=publisher, year=year, album_number=album_number, album_total=album_total, comment=None) ================================================ FILE: audiotools/ogg.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools._ogg import PageReader, PageWriter, Page class PacketReader(object): def __init__(self, pagereader): """pagereader is a PageReader object""" self.__pagereader__ = pagereader self.__page__ = Page(packet_continuation=0, stream_beginning=0, stream_end=0, granule_position=0, bitstream_serial_number=0, sequence_number=0, segments=[]) self.__current_segment__ = 1 def __enter__(self): return self def __exit__(self, exc_type, exc_value, traceback): self.close() def read_segment(self): if self.__current_segment__ >= len(self.__page__): self.read_page() segment = self.__page__[self.__current_segment__] self.__current_segment__ += 1 return segment def read_page(self): self.__page__ = self.__pagereader__.read() self.__current_segment__ = 0 return self.__page__ def read_packet(self): """returns next Ogg packet as a string""" segments = [] segment = self.read_segment() segments.append(segment) while len(segment) == 255: segment = self.read_segment() segments.append(segment) return b"".join(segments) def close(self): """closes stream for further reading""" self.__pagereader__.close() def packet_to_segments(packet): if len(packet) == 0: yield b"" else: while len(packet) > 0: if len(packet) == 255: yield packet yield b"" packet = b"" else: yield packet[0:255] packet = packet[255:] def packet_to_pages(packet, bitstream_serial_number, starting_sequence_number=0): """given a string of packet data, yields a Page object per Ogg page necessary to hold that packet packet_continuation is filled in as needed stream_beginning and stream_end are False granule_position is 0 sequence_number increments starting from "starting_sequence_number" """ from audiotools._ogg import Page page = Page( packet_continuation=False, stream_beginning=False, stream_end=False, granule_position=0, bitstream_serial_number=bitstream_serial_number, sequence_number=starting_sequence_number, segments=[]) for segment in packet_to_segments(packet): if page.full(): yield page starting_sequence_number += 1 page = Page( packet_continuation=True, stream_beginning=False, stream_end=False, granule_position=0, bitstream_serial_number=bitstream_serial_number, sequence_number=starting_sequence_number, segments=[]) page.append(segment) yield page def packets_to_pages(packets, bitstream_serial_number, starting_sequence_number=0): """given an iterable of packet data strings, yields a Page object per Ogg page necessary to hold those packets packet_continuation is filled in as needed stream_beginning and stream_end are False granule_position is 0 sequence_number increments starting from "starting_sequence_number" """ from audiotools._ogg import Page page = Page( packet_continuation=False, stream_beginning=False, stream_end=False, granule_position=0, bitstream_serial_number=bitstream_serial_number, sequence_number=starting_sequence_number, segments=[]) for packet in packets: for (i, segment) in enumerate(packet_to_segments(packet)): if page.full(): yield page starting_sequence_number += 1 page = Page( packet_continuation=(i != 0), stream_beginning=False, stream_end=False, granule_position=0, bitstream_serial_number=bitstream_serial_number, sequence_number=starting_sequence_number, segments=[]) page.append(segment) yield page ================================================ FILE: audiotools/opus.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools import (AudioFile, InvalidFile) from audiotools.vorbis import (VorbisAudio, VorbisChannelMask) from audiotools.vorbiscomment import VorbisComment class InvalidOpus(InvalidFile): pass ####################### # Vorbis File ####################### class OpusAudio(VorbisAudio): """an Opus file""" SUFFIX = "opus" NAME = "opus" DESCRIPTION = u"Opus Audio Codec" DEFAULT_COMPRESSION = "10" COMPRESSION_MODES = tuple(map(str, range(0, 11))) COMPRESSION_DESCRIPTIONS = {"0": u"lowest quality, fastest encode", "10": u"best quality, slowest encode"} def __init__(self, filename): """filename is a plain string""" AudioFile.__init__(self, filename) self.__channels__ = 0 self.__channel_mask__ = 0 # get channel count and channel mask from first packet from audiotools.bitstream import BitstreamReader try: with BitstreamReader(open(filename, "rb"), True) as ogg_reader: (magic_number, version, header_type, granule_position, self.__serial_number__, page_sequence_number, checksum, segment_count) = ogg_reader.parse( "4b 8u 8u 64S 32u 32u 32u 8u") if magic_number != b'OggS': from audiotools.text import ERR_OGG_INVALID_MAGIC_NUMBER raise InvalidOpus(ERR_OGG_INVALID_MAGIC_NUMBER) if version != 0: from audiotools.text import ERR_OGG_INVALID_VERSION raise InvalidOpus(ERR_OGG_INVALID_VERSION) segment_length = ogg_reader.read(8) (opushead, version, self.__channels__, pre_skip, input_sample_rate, output_gain, mapping_family) = ogg_reader.parse( "8b 8u 8u 16u 32u 16s 8u") if opushead != b"OpusHead": from audiotools.text import ERR_OPUS_INVALID_TYPE raise InvalidOpus(ERR_OPUS_INVALID_TYPE) if version != 1: from audiotools.text import ERR_OPUS_INVALID_VERSION raise InvalidOpus(ERR_OPUS_INVALID_VERSION) if self.__channels__ == 0: from audiotools.text import ERR_OPUS_INVALID_CHANNELS raise InvalidOpus(ERR_OPUS_INVALID_CHANNELS) # FIXME - assign channel mask from mapping family if mapping_family == 0: if self.__channels__ == 1: self.__channel_mask__ = VorbisChannelMask(0x4) elif self.__channels__ == 2: self.__channel_mask__ = VorbisChannelMask(0x3) else: self.__channel_mask__ = VorbisChannelMask(0) else: (stream_count, coupled_stream_count) = ogg_reader.parse("8u 8u") if (self.__channels__ != ((coupled_stream_count * 2) + (stream_count - coupled_stream_count))): from audiotools.text import ERR_OPUS_INVALID_CHANNELS raise InvalidOpus(ERR_OPUS_INVALID_CHANNELS) channel_mapping = [ogg_reader.read(8) for i in range(self.__channels__)] except IOError as msg: raise InvalidOpus(str(msg)) @classmethod def supports_replay_gain(cls): """returns True if this class supports ReplayGain""" return False def get_replay_gain(self): """returns a ReplayGain object of our ReplayGain values returns None if we have no values may raise IOError if unable to read the file""" return None def set_replay_gain(self, replaygain): """given a ReplayGain object, sets the track's gain to those values may raise IOError if unable to modify the file""" pass def delete_replay_gain(self): """removes ReplayGain values from file, if any may raise IOError if unable to modify the file""" pass def total_frames(self): """returns the total PCM frames of the track as an integer""" from audiotools._ogg import PageReader try: with PageReader(open(self.filename, "rb")) as reader: page = reader.read() pcm_samples = page.granule_position while not page.stream_end: page = reader.read() pcm_samples = max(pcm_samples, page.granule_position) return pcm_samples except (IOError, ValueError): return 0 def sample_rate(self): """returns the rate of the track's audio as an integer number of Hz""" return 48000 @classmethod def supports_to_pcm(cls): """returns True if all necessary components are available to support the .to_pcm() method""" try: from audiotools.decoders import OpusDecoder return True except ImportError: return False def to_pcm(self): """returns a PCMReader object containing the track's PCM data if an error occurs initializing a decoder, this should return a PCMReaderError with an appropriate error message""" from audiotools.decoders import OpusDecoder try: return OpusDecoder(self.filename) except ValueError as err: from audiotools import PCMReaderError return PCMReaderError(error_message=str(err), sample_rate=self.sample_rate(), channels=self.channels(), channel_mask=int(self.channel_mask()), bits_per_sample=self.bits_per_sample()) @classmethod def supports_from_pcm(cls): """returns True if all necessary components are available to support the .from_pcm() classmethod""" try: from audiotools.encoders import encode_opus return True except ImportError: return False @classmethod def from_pcm(cls, filename, pcmreader, compression=None, total_pcm_frames=None): """encodes a new file from PCM data takes a filename string, PCMReader object, optional compression level string and optional total_pcm_frames integer encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new AudioFile-compatible object may raise EncodingError if some problem occurs when encoding the input file. This includes an error in the input stream, a problem writing the output file, or even an EncodingError subclass such as "UnsupportedBitsPerSample" if the input stream is formatted in a way this class is unable to support """ from audiotools import (BufferedPCMReader, PCMConverter, __default_quality__, EncodingError) from audiotools.encoders import encode_opus if (((compression is None) or (compression not in cls.COMPRESSION_MODES))): compression = __default_quality__(cls.NAME) if pcmreader.bits_per_sample not in {8, 16, 24}: from audiotools import UnsupportedBitsPerSample pcmreader.close() raise UnsupportedBitsPerSample(filename, pcmreader.bits_per_sample) if (pcmreader.channels > 2) and (pcmreader.channels <= 8): if ((pcmreader.channel_mask != 0) and (pcmreader.channel_mask not in {0x7, # FR, FC, FL 0x33, # FR, FL, BR, BL 0x37, # FR, FC, FL, BL, BR 0x3f, # FR, FC, FL, BL, BR, LFE 0x70f, # FL, FC, FR, SL, SR, BC, LFE 0x63f})): # FL, FC, FR, SL, SR, BL, BR, LFE from audiotools import UnsupportedChannelMask pcmreader.close() raise UnsupportedChannelMask(filename, channel_mask) try: if total_pcm_frames is not None: from audiotools import CounterPCMReader pcmreader = CounterPCMReader(pcmreader) encode_opus(filename, PCMConverter(pcmreader, sample_rate=48000, channels=pcmreader.channels, channel_mask=pcmreader.channel_mask, bits_per_sample=16), quality=int(compression), original_sample_rate=pcmreader.sample_rate) pcmreader.close() if ((total_pcm_frames is not None) and (total_pcm_frames != pcmreader.frames_written)): from audiotools.text import ERR_TOTAL_PCM_FRAMES_MISMATCH cls.__unlink__(filename) raise EncodingError(ERR_TOTAL_PCM_FRAMES_MISMATCH) return cls(filename) except (ValueError, IOError) as err: pcmreader.close() cls.__unlink__(filename) raise EncodingError(err) def update_metadata(self, metadata): """takes this track's current MetaData object as returned by get_metadata() and sets this track's metadata with any fields updated in that object raises IOError if unable to write the file """ import os from audiotools import TemporaryFile from audiotools.ogg import (PageReader, PacketReader, PageWriter, packet_to_pages, packets_to_pages) from audiotools.bitstream import BitstreamRecorder if metadata is None: return elif not isinstance(metadata, VorbisComment): from audiotools.text import ERR_FOREIGN_METADATA raise ValueError(ERR_FOREIGN_METADATA) elif not os.access(self.filename, os.W_OK): raise IOError(self.filename) original_ogg = PacketReader(PageReader(open(self.filename, "rb"))) new_ogg = PageWriter(TemporaryFile(self.filename)) sequence_number = 0 # transfer current file's identification packet in its own page identification_packet = original_ogg.read_packet() for (i, page) in enumerate(packet_to_pages( identification_packet, self.__serial_number__, starting_sequence_number=sequence_number)): page.stream_beginning = (i == 0) new_ogg.write(page) sequence_number += 1 # discard the current file's comment packet comment_packet = original_ogg.read_packet() # generate new comment packet comment_writer = BitstreamRecorder(True) comment_writer.write_bytes(b"OpusTags") vendor_string = metadata.vendor_string.encode('utf-8') comment_writer.build("32u {:d}b".format(len(vendor_string)), (len(vendor_string), vendor_string)) comment_writer.write(32, len(metadata.comment_strings)) for comment_string in metadata.comment_strings: comment_string = comment_string.encode('utf-8') comment_writer.build("32u {:d}b".format(len(comment_string)), (len(comment_string), comment_string)) for page in packet_to_pages( comment_writer.data(), self.__serial_number__, starting_sequence_number=sequence_number): new_ogg.write(page) sequence_number += 1 # transfer remaining pages after re-sequencing page = original_ogg.read_page() page.sequence_number = sequence_number sequence_number += 1 new_ogg.write(page) while not page.stream_end: page = original_ogg.read_page() page.sequence_number = sequence_number page.bitstream_serial_number = self.__serial_number__ sequence_number += 1 new_ogg.write(page) original_ogg.close() new_ogg.close() def set_metadata(self, metadata): """takes a MetaData object and sets this track's metadata this metadata includes track name, album name, and so on raises IOError if unable to write the file""" if metadata is None: return self.delete_metadata() metadata = VorbisComment.converted(metadata) old_metadata = self.get_metadata() metadata.vendor_string = old_metadata.vendor_string # port REPLAYGAIN and ENCODER from old metadata to new metadata for key in [u"REPLAYGAIN_TRACK_GAIN", u"REPLAYGAIN_TRACK_PEAK", u"REPLAYGAIN_ALBUM_GAIN", u"REPLAYGAIN_ALBUM_PEAK", u"REPLAYGAIN_REFERENCE_LOUDNESS", u"ENCODER"]: try: metadata[key] = old_metadata[key] except KeyError: metadata[key] = [] self.update_metadata(metadata) @classmethod def supports_metadata(cls): """returns True if this audio type supports MetaData""" return True def get_metadata(self): """returns a MetaData object, or None raises IOError if unable to read the file""" from io import BytesIO from audiotools.bitstream import BitstreamReader from audiotools.ogg import PacketReader, PageReader with PacketReader(PageReader(open(self.filename, "rb"))) as reader: identification = reader.read_packet() comment = BitstreamReader(BytesIO(reader.read_packet()), True) if comment.read_bytes(8) == b"OpusTags": vendor_string = \ comment.read_bytes(comment.read(32)).decode('utf-8') comment_strings = [ comment.read_bytes(comment.read(32)).decode('utf-8') for i in range(comment.read(32))] return VorbisComment(comment_strings, vendor_string) else: return None def delete_metadata(self): """deletes the track's MetaData this removes or unsets tags as necessary in order to remove all data raises IOError if unable to write the file""" from audiotools import MetaData # the vorbis comment packet is required, # so simply zero out its contents self.set_metadata(MetaData()) def verify(self, progress=None): """verifies the current file for correctness returns True if the file is okay raises an InvalidFile with an error message if there is some problem with the file""" # Checking for a truncated Ogg stream typically involves # verifying that the "end of stream" flag is set on the last # Ogg page in the stream in the event that one or more whole # pages is lost. But since the OpusFile decoder doesn't perform # this check and doesn't provide any access to its internal # Ogg decoder (unlike Vorbis), we'll perform that check externally. # # And since it's a fast check, we won't bother to update progress. from audiotools.ogg import PageReader import os.path try: f = open(self.filename, "rb") except IOError as err: raise InvalidOpus(str(err)) try: reader = PageReader(f) except IOError as err: f.close() raise InvalidOpus(str(err)) try: page = reader.read() while not page.stream_end: page = reader.read() except (IOError, ValueError) as err: raise InvalidOpus(str(err)) finally: reader.close() return AudioFile.verify(self, progress) ================================================ FILE: audiotools/player.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA (RG_NO_REPLAYGAIN, RG_TRACK_GAIN, RG_ALBUM_GAIN) = range(3) DEFAULT_FORMAT = (44100, 2, 0x3, 16) (PLAYER_STOPPED, PLAYER_PAUSED, PLAYER_PLAYING) = range(3) class Player(object): """a class for operating an audio player the player itself runs in a seperate thread, which this sends commands to""" def __init__(self, audio_output, replay_gain=RG_NO_REPLAYGAIN, next_track_callback=lambda: None): """audio_output is an AudioOutput object replay_gain is RG_NO_REPLAYGAIN, RG_TRACK_GAIN or RG_ALBUM_GAIN, indicating how the player should apply ReplayGain next_track_callback is a function with no arguments which is called by the player when the current track is finished Raises ValueError if unable to start player subprocess.""" import threading try: from queue import Queue except ImportError: from Queue import Queue if not isinstance(audio_output, AudioOutput): raise TypeError("invalid output object") self.__audio_output__ = audio_output self.__player__ = AudioPlayer(audio_output, next_track_callback, replay_gain) self.__commands__ = Queue() self.__responses__ = Queue() self.__thread__ = threading.Thread( target=self.__player__.run, kwargs={"commands": self.__commands__, "responses": self.__responses__}) self.__thread__.daemon = False self.__thread__.start() def open(self, track): """opens the given AudioFile for playing stops playing the current file, if any""" self.__commands__.put(("open", (track,))) def play(self): """begins or resumes playing an opened AudioFile, if any""" self.__commands__.put(("play", tuple())) def set_replay_gain(self, replay_gain): """sets the given ReplayGain level to apply during playback choose from RG_NO_REPLAYGAIN, RG_TRACK_GAIN or RG_ALBUM_GAIN replayGain cannot be applied mid-playback one must stop() and play() a file for it to take effect""" self.__commands__.put(("set_replay_gain", (replay_gain,))) def set_output(self, output): """given an AudioOutput object, sets the player's output to that device any currently playing audio is stopped""" self.__audio_output__ = output self.__commands__.put(("set_output", (output,))) def pause(self): """pauses playback of the current file playback may be resumed with play() or toggle_play_pause()""" self.__commands__.put(("pause", tuple())) def toggle_play_pause(self): """pauses the file if playing, play the file if paused""" self.__commands__.put(("toggle_play_pause", tuple())) def stop(self): """stops playback of the current file if play() is called, playback will start from the beginning""" self.__commands__.put(("stop", tuple())) def state(self): """returns the current state of the Player as either PLAYER_STOPPED, PLAYER_PAUSED, or PLAYER_PLAYING ints""" return self.__player__.state() def close(self): """closes the player for playback the player thread is halted and the AudioOutput is closed""" self.__commands__.put(("close", tuple())) def progress(self): """returns a (pcm_frames_played, pcm_frames_total) tuple this indicates the current playback status in PCM frames""" return self.__player__.progress() def current_output_description(self): """returns the human-readable description of the current output device as a Unicode string""" return self.__audio_output__.description() def current_output_name(self): """returns the ``NAME`` attribute of the current output device as a plain string""" return self.__audio_output__.NAME def get_volume(self): """returns the current volume level as a floating point value between 0.0 and 1.0, inclusive""" return self.__audio_output__.get_volume() def set_volume(self, volume): """given a floating point value between 0.0 and 1.0, inclusive, sets the current volume level to that value""" self.__audio_output__.set_volume(volume) def change_volume(self, delta): """changes the volume by the given floating point amount where delta may be positive or negative and returns the new volume as a floating point value""" self.__audio_output__.set_volume( min(max(self.__audio_output__.get_volume() + delta, 0.0), 1.0)) return self.__audio_output__.get_volume() class AudioPlayer(object): def __init__(self, audio_output, next_track_callback=lambda: None, replay_gain=RG_NO_REPLAYGAIN): """audio_output is an AudioOutput object to play audio to next_track_callback is an optional function which is called with no arguments when the current track is finished""" self.__state__ = PLAYER_STOPPED self.__audio_output__ = audio_output self.__next_track_callback__ = next_track_callback self.__audiofile__ = None self.__pcmreader__ = None self.__buffer_size__ = 1 self.__replay_gain__ = replay_gain self.__current_frames__ = 0 self.__total_frames__ = 1 def set_audiofile(self, audiofile): """sets audiofile to play""" self.__audiofile__ = audiofile def state(self): """returns current state of player which is one of: PLAYER_STOPPED, PLAYER_PAUSED, PLAYER_PLAYING""" return self.__state__ def progress(self): """returns current progress as a (current frames, total frames) tuple""" return (self.__current_frames__, self.__total_frames__) def stop(self): """changes current state of player to PLAYER_STOPPED""" if self.__state__ == PLAYER_STOPPED: # already stopped, so nothing to do return else: if self.__state__ == PLAYER_PAUSED: self.__audio_output__.resume() self.__state__ = PLAYER_STOPPED self.__pcmreader__ = None self.__current_frames__ = 0 self.__total_frames__ = 1 def pause(self): """if playing, changes current state of player to PLAYER_PAUSED""" # do nothing if player is stopped or already paused if self.__state__ == PLAYER_PLAYING: self.__audio_output__.pause() self.__state__ = PLAYER_PAUSED def play(self): """if audiofile has been opened, changes current state of player to PLAYER_PLAYING""" from audiotools import BufferedPCMReader if self.__state__ == PLAYER_PLAYING: # already playing, so nothing to do return elif self.__state__ == PLAYER_PAUSED: # go from unpaused to playing self.__audio_output__.resume() self.__state__ = PLAYER_PLAYING elif ((self.__state__ == PLAYER_STOPPED) and (self.__audiofile__ is not None)): # go from stopped to playing # if an audiofile has been opened # get PCMReader from selected audiofile pcmreader = self.__audiofile__.to_pcm() # apply ReplayGain if requested if self.__replay_gain__ in (RG_TRACK_GAIN, RG_ALBUM_GAIN): gain = self.__audiofile__.get_replay_gain() if gain is not None: from audiotools.replaygain import ReplayGainReader if self.__replay_gain__ == RG_TRACK_GAIN: pcmreader = ReplayGainReader(pcmreader, gain.track_gain, gain.track_peak) else: pcmreader = ReplayGainReader(pcmreader, gain.album_gain, gain.album_peak) # buffer PCMReader so that one can process small chunks of data self.__pcmreader__ = BufferedPCMReader(pcmreader) # calculate quarter second buffer size # (or at least 256 samples) self.__buffer_size__ = max(int(round(0.25 * pcmreader.sample_rate)), 256) # set output to be compatible with PCMReader self.__audio_output__.set_format( sample_rate=self.__pcmreader__.sample_rate, channels=self.__pcmreader__.channels, channel_mask=self.__pcmreader__.channel_mask, bits_per_sample=self.__pcmreader__.bits_per_sample) # reset progress self.__current_frames__ = 0 self.__total_frames__ = self.__audiofile__.total_frames() # update state so audio begins playing self.__state__ = PLAYER_PLAYING def output_audio(self): """if player is playing, output the next chunk of audio if possible if audio is exhausted, stop playing and call the next_track callback""" if self.__state__ == PLAYER_PLAYING: try: frame = self.__pcmreader__.read(self.__buffer_size__) except (IOError, ValueError) as err: # some sort of read error occurred # so cease playing file and move on to next self.stop() if callable(self.__next_track_callback__): self.__next_track_callback__() return if len(frame) > 0: self.__current_frames__ += frame.frames self.__audio_output__.play(frame) else: # audio has been exhausted self.stop() if callable(self.__next_track_callback__): self.__next_track_callback__() def run(self, commands, responses): """runs the audio playing thread while accepting commands from the given Queue""" try: from queue import Empty except ImportError: from Queue import Empty while True: try: (command, args) = commands.get(self.__state__ != PLAYER_PLAYING) # got a command to process if command == "open": # stop whatever's playing and prepare new track for playing self.stop() self.set_audiofile(args[0]) elif command == "play": self.play() elif command == "set_replay_gain": self.__replay_gain__ = args[0] elif command == "set_output": # resume (if necessary) and close existing output if self.__state__ == PLAYER_PAUSED: self.__audio_output__.resume() self.__audio_output__.close() # set new output and set format (if necessary) self.__audio_output__ = args[0] if self.__pcmreader__ is not None: self.__audio_output__.set_format( sample_rate=self.__pcmreader__.sample_rate, channels=self.__pcmreader__.channels, channel_mask=self.__pcmreader__.channel_mask, bits_per_sample=self.__pcmreader__.bits_per_sample) # if paused, reset audio output to paused if self.__state__ == PLAYER_PAUSED: self.__audio_output__.pause() elif command == "pause": self.pause() elif command == "toggle_play_pause": # switch from paused to playing or playing to paused if self.__state__ == PLAYER_PAUSED: self.play() elif self.__state__ == PLAYER_PLAYING: self.pause() elif command == "stop": self.stop() self.__audio_output__.close() elif command == "close": self.stop() self.__audio_output__.close() return except Empty: # no commands to process # so output audio if playing self.output_audio() class CDPlayer(Player): def __init__(self, cddareader, audio_output, next_track_callback=lambda: None): """cdda is a CDDAReader object audio_output is an AudioOutput object next_track_callback is a function with no arguments which is called by the player when the current track is finished""" import threading try: from queue import Queue except ImportError: from Queue import Queue if not isinstance(audio_output, AudioOutput): raise TypeError("invalid output object") self.__audio_output__ = audio_output self.__player__ = CDAudioPlayer(cddareader, audio_output, next_track_callback) self.__commands__ = Queue() self.__responses__ = Queue() self.__thread__ = threading.Thread( target=self.__player__.run, kwargs={"commands": self.__commands__, "responses": self.__responses__}) self.__thread__.daemon = False self.__thread__.start() def open(self, track_number): """opens the given track_number for playing stops playing the current track, if any""" self.__commands__.put(("open", (track_number,))) def set_replay_gain(self, replay_gain): """ReplayGain not applicable to CDDA, so this does nothing""" pass class CDAudioPlayer(AudioPlayer): def __init__(self, cddareader, audio_output, next_track_callback=lambda: None): """cdda is a CDDAReader object to play tracks from audio_output is an AudioOutput object to play audio to next_track_callback is an optional function which is called with no arguments when the current track is finished""" self.__state__ = PLAYER_STOPPED self.__audio_output__ = audio_output self.__next_track_callback__ = next_track_callback self.__cddareader__ = cddareader self.__offsets__ = cddareader.track_offsets self.__lengths__ = cddareader.track_lengths self.__track_number__ = None self.__pcmreader__ = None self.__buffer_size__ = 1 self.__replay_gain__ = RG_NO_REPLAYGAIN self.__current_frames__ = 0 self.__total_frames__ = 1 def set_audiofile(self, track_number): """set tracks number to play""" # ensure track number is in the proper range if track_number in self.__offsets__.keys(): self.__track_number__ = track_number def play(self): """if track has been selected, changes current state of player to PLAYER_PLAYING""" from audiotools import (BufferedPCMReader, ThreadedPCMReader, PCMReaderHead) if self.__state__ == PLAYER_PLAYING: # already playing, so nothing to do return elif self.__state__ == PLAYER_PAUSED: # go from unpaused to playing self.__audio_output__.resume() self.__state__ = PLAYER_PLAYING elif ((self.__state__ == PLAYER_STOPPED) and (self.__track_number__ is not None)): # go from stopped to playing # if a track number has been selected # seek to specified track number self.__cddareader__.seek(self.__offsets__[self.__track_number__]) track = PCMReaderHead(self.__cddareader__, self.__lengths__[self.__track_number__], False) # decode PCMReader in thread # and place in buffer so one can process small chunks of data self.__pcmreader__ = BufferedPCMReader(ThreadedPCMReader(track)) # calculate quarter second buffer size self.__buffer_size__ = int(round(0.25 * 44100)) # set output to be compatible with PCMReader self.__audio_output__.set_format( sample_rate=44100, channels=2, channel_mask=0x3, bits_per_sample=16) # reset progress self.__current_frames__ = 0 self.__total_frames__ = self.__lengths__[self.__track_number__] # update state so audio begins playing self.__state__ = PLAYER_PLAYING class AudioOutput(object): """an abstract parent class for playing audio""" def __init__(self): self.sample_rate = None self.channels = None self.channel_mask = None self.bits_per_sample = None def __getstate__(self): """gets internal state for use by Pickle module""" return "" def __setstate__(self, name): """sets internal state for use by Pickle module""" # audio outputs are initialized closed for obvious reasons self.sample_rate = None self.channels = None self.channel_mask = None self.bits_per_sample = None def description(self): """returns user-facing name of output device as unicode""" raise NotImplementedError() def compatible(self, sample_rate, channels, channel_mask, bits_per_sample): """returns True if the given pcmreader is compatible with the given format""" return ((self.sample_rate == sample_rate) and (self.channels == channels) and (self.channel_mask == channel_mask) and (self.bits_per_sample == bits_per_sample)) def set_format(self, sample_rate, channels, channel_mask, bits_per_sample): """sets the output stream to the given format if the stream hasn't been initialized, this method initializes it if the stream has been initialized to a different format, this method closes and reopens the stream to the new format if the stream has been initialized to the same format, this method does nothing""" self.sample_rate = sample_rate self.channels = channels self.channel_mask = channel_mask self.bits_per_sample = bits_per_sample def play(self, framelist): """plays a FrameList""" raise NotImplementedError() def pause(self): """pauses audio output, with the expectation it will be resumed""" raise NotImplementedError() def resume(self): """resumes playing paused audio output""" raise NotImplementedError() def get_volume(self): """returns a floating-point volume value between 0.0 and 1.0""" return 0.0 def set_volume(self, volume): """sets the output volume to a floating point value between 0.0 and 1.0""" pass def close(self): """closes the output stream""" self.sample_rate = None self.channels = None self.channel_mask = None self.bits_per_sample = None @classmethod def available(cls): """returns True if the AudioOutput is available on the system""" return False class NULLAudioOutput(AudioOutput): """an AudioOutput subclass which does not actually play anything although this consumes audio output at the rate it would normally play, it generates no output""" NAME = "NULL" def __init__(self): self.__volume__ = 0.30 AudioOutput.__init__(self) def __getstate__(self): return "NULL" def __setstate__(self, name): AudioOutput.__setstate__(self, name) self.__volume__ = 0.30 def description(self): """returns user-facing name of output device as unicode""" return u"Dummy Output" def play(self, framelist): """plays a chunk of converted data""" import time time.sleep(float(framelist.frames) / self.sample_rate) def pause(self): """pauses audio output, with the expectation it will be resumed""" pass def resume(self): """resumes playing paused audio output""" pass def get_volume(self): """returns a floating-point volume value between 0.0 and 1.0""" return self.__volume__ def set_volume(self, volume): """sets the output volume to a floating point value between 0.0 and 1.0""" if (volume >= 0) and (volume <= 1.0): self.__volume__ = volume else: raise ValueError("volume must be between 0.0 and 1.0") def close(self): """closes the output stream""" AudioOutput.close(self) @classmethod def available(cls): """returns True""" return True class OSSAudioOutput(AudioOutput): """an AudioOutput subclass for OSS output""" NAME = "OSS" def __init__(self): """automatically initializes output format for playing CD quality audio""" self.__ossaudio__ = None self.__ossmixer__ = None AudioOutput.__init__(self) def __getstate__(self): """gets internal state for use by Pickle module""" return "OSS" def __setstate__(self, name): """sets internal state for use by Pickle module""" AudioOutput.__setstate__(self, name) self.__ossaudio__ = None self.__ossmixer__ = None def description(self): """returns user-facing name of output device as unicode""" return u"Open Sound System" def set_format(self, sample_rate, channels, channel_mask, bits_per_sample): """sets the output stream to the given format if the stream hasn't been initialized, this method initializes it if the stream has been initialized to a different format, this method closes and reopens the stream to the new format if the stream has been initialized to the same format, this method does nothing""" if self.__ossaudio__ is None: # output hasn't been initialized import ossaudiodev AudioOutput.set_format(self, sample_rate, channels, channel_mask, bits_per_sample) # initialize audio output device and setup framelist converter self.__ossaudio__ = ossaudiodev.open('w') self.__ossmixer__ = ossaudiodev.openmixer() if self.bits_per_sample == 8: self.__ossaudio__.setfmt(ossaudiodev.AFMT_S8_LE) self.__converter__ = lambda f: f.to_bytes(False, True) elif self.bits_per_sample == 16: self.__ossaudio__.setfmt(ossaudiodev.AFMT_S16_LE) self.__converter__ = lambda f: f.to_bytes(False, True) elif self.bits_per_sample == 24: from audiotools.pcm import from_list self.__ossaudio__.setfmt(ossaudiodev.AFMT_S16_LE) self.__converter__ = lambda f: from_list( [i >> 8 for i in list(f)], self.channels, 16, True).to_bytes(False, True) else: raise ValueError("Unsupported bits-per-sample") self.__ossaudio__.channels(channels) self.__ossaudio__.speed(sample_rate) elif (not self.compatible(sample_rate=sample_rate, channels=channels, channel_mask=channel_mask, bits_per_sample=bits_per_sample)): # output has been initialized to a different format self.close() self.set_format(sample_rate=sample_rate, channels=channels, channel_mask=channel_mask, bits_per_sample=bits_per_sample) def play(self, framelist): """plays a FrameList""" self.__ossaudio__.writeall(self.__converter__(framelist)) def pause(self): """pauses audio output, with the expectation it will be resumed""" pass def resume(self): """resumes playing paused audio output""" pass def get_volume(self): """returns a floating-point volume value between 0.0 and 1.0""" import ossaudiodev if self.__ossmixer__ is None: self.set_format(*DEFAULT_FORMAT) controls = self.__ossmixer__.controls() for control in (ossaudiodev.SOUND_MIXER_VOLUME, ossaudiodev.SOUND_MIXER_PCM): if controls & (1 << control): try: volumes = self.__ossmixer__.get(control) return (sum(volumes) / float(len(volumes))) / 100.0 except ossaudiodev.OSSAudioError: continue else: return 0.0 def set_volume(self, volume): """sets the output volume to a floating point value between 0.0 and 1.0""" if (volume >= 0) and (volume <= 1.0): if self.__ossmixer__ is None: self.set_format(*DEFAULT_FORMAT) controls = self.__ossmixer__.controls() ossvolume = max(min(int(round(volume * 100)), 100), 0) for control in (ossaudiodev.SOUND_MIXER_VOLUME, ossaudiodev.SOUND_MIXER_PCM): if controls & (1 << control): try: self.__ossmixer__.set(control, (ossvolume, ossvolume)) except ossaudiodev.OSSAudioError: continue else: raise ValueError("volume must be between 0.0 and 1.0") def close(self): """closes the output stream""" AudioOutput.close(self) if self.__ossaudio__ is not None: self.__ossaudio__.close() self.__ossaudio__ = None if self.__ossmixer__ is not None: self.__ossmixer__.close() self.__ossmixer__ = None @classmethod def available(cls): """returns True if OSS output is available on the system""" try: import ossaudiodev ossaudiodev.open("w").close() return True except (ImportError, IOError): return False class PulseAudioOutput(AudioOutput): """an AudioOutput subclass for PulseAudio output""" NAME = "PulseAudio" def __init__(self): self.__pulseaudio__ = None AudioOutput.__init__(self) def __getstate__(self): """gets internal state for use by Pickle module""" return "PulseAudio" def __setstate__(self, name): """sets internal state for use by Pickle module""" AudioOutput.__setstate__(self, name) self.__pulseaudio__ = None def description(self): """returns user-facing name of output device as unicode""" # FIXME - pull this from device description return u"Pulse Audio" def set_format(self, sample_rate, channels, channel_mask, bits_per_sample): """sets the output stream to the given format if the stream hasn't been initialized, this method initializes it if the stream has been initialized to a different format, this method closes and reopens the stream to the new format if the stream has been initialized to the same format, this method does nothing""" if self.__pulseaudio__ is None: # output hasn't been initialized from audiotools.output import PulseAudio AudioOutput.set_format(self, sample_rate, channels, channel_mask, bits_per_sample) self.__pulseaudio__ = PulseAudio(sample_rate, channels, bits_per_sample, "Python Audio Tools") self.__converter__ = { 8: lambda f: f.to_bytes(True, False), 16: lambda f: f.to_bytes(False, True), 24: lambda f: f.to_bytes(False, True)}[self.bits_per_sample] elif (not self.compatible(sample_rate=sample_rate, channels=channels, channel_mask=channel_mask, bits_per_sample=bits_per_sample)): # output has been initialized to a different format self.close() self.set_format(sample_rate=sample_rate, channels=channels, channel_mask=channel_mask, bits_per_sample=bits_per_sample) def play(self, framelist): """plays a FrameList""" self.__pulseaudio__.play(self.__converter__(framelist)) def pause(self): """pauses audio output, with the expectation it will be resumed""" if self.__pulseaudio__ is not None: self.__pulseaudio__.pause() def resume(self): """resumes playing paused audio output""" if self.__pulseaudio__ is not None: self.__pulseaudio__.resume() def get_volume(self): """returns a floating-point volume value between 0.0 and 1.0""" if self.__pulseaudio__ is None: self.set_format(*DEFAULT_FORMAT) return self.__pulseaudio__.get_volume() def set_volume(self, volume): """sets the output volume to a floating point value between 0.0 and 1.0""" if (volume >= 0) and (volume <= 1.0): if self.__pulseaudio__ is None: self.set_format(*DEFAULT_FORMAT) self.__pulseaudio__.set_volume(volume) else: raise ValueError("volume must be between 0.0 and 1.0") def close(self): """closes the output stream""" AudioOutput.close(self) if self.__pulseaudio__ is not None: self.__pulseaudio__.flush() self.__pulseaudio__.close() self.__pulseaudio__ = None @classmethod def available(cls): """returns True if PulseAudio is available and running on the system""" try: from audiotools.output import PulseAudio return True except ImportError: return False class ALSAAudioOutput(AudioOutput): """an AudioOutput subclass for ALSA output""" NAME = "ALSA" def __init__(self): self.__alsaaudio__ = None AudioOutput.__init__(self) def __getstate__(self): """gets internal state for use by Pickle module""" return "ALSA" def __setstate__(self, name): """sets internal state for use by Pickle module""" AudioOutput.__setstate__(self, name) self.__alsaaudio__ = None def description(self): """returns user-facing name of output device as unicode""" # FIXME - pull this from device description return u"Advanced Linux Sound Architecture" def set_format(self, sample_rate, channels, channel_mask, bits_per_sample): """sets the output stream to the given format if the stream hasn't been initialized, this method initializes it if the stream has been initialized to a different format, this method closes and reopens the stream to the new format if the stream has been initialized to the same format, this method does nothing""" if self.__alsaaudio__ is None: # output hasn't been initialized from audiotools.output import ALSAAudio AudioOutput.set_format(self, sample_rate, channels, channel_mask, bits_per_sample) self.__alsaaudio__ = ALSAAudio("default", sample_rate, channels, bits_per_sample) elif (not self.compatible(sample_rate=sample_rate, channels=channels, channel_mask=channel_mask, bits_per_sample=bits_per_sample)): # output has been initialized to different format self.close() self.set_format(sample_rate=sample_rate, channels=channels, channel_mask=channel_mask, bits_per_sample=bits_per_sample) def play(self, framelist): """plays a FrameList""" self.__alsaaudio__.play(framelist) def pause(self): """pauses audio output, with the expectation it will be resumed""" if self.__alsaaudio__ is not None: self.__alsaaudio__.pause() def resume(self): """resumes playing paused audio output""" if self.__alsaaudio__ is not None: self.__alsaaudio__.resume() def get_volume(self): """returns a floating-point volume value between 0.0 and 1.0""" if self.__alsaaudio__ is None: self.set_format(*DEFAULT_FORMAT) return self.__alsaaudio__.get_volume() def set_volume(self, volume): """sets the output volume to a floating point value between 0.0 and 1.0""" if (volume >= 0) and (volume <= 1.0): if self.__alsaaudio__ is None: self.set_format(*DEFAULT_FORMAT) self.__alsaaudio__.set_volume(volume) else: raise ValueError("volume must be between 0.0 and 1.0") def close(self): """closes the output stream""" AudioOutput.close(self) if self.__alsaaudio__ is not None: self.__alsaaudio__.flush() self.__alsaaudio__.close() self.__alsaaudio__ = None @classmethod def available(cls): """returns True if ALSA is available and running on the system""" try: from audiotools.output import ALSAAudio return True except ImportError: return False class CoreAudioOutput(AudioOutput): """an AudioOutput subclass for CoreAudio output""" NAME = "CoreAudio" def __init__(self): self.__coreaudio__ = None AudioOutput.__init__(self) def __getstate__(self): """gets internal state for use by Pickle module""" return "CoreAudio" def __setstate__(self, name): """sets internal state for use by Pickle module""" AudioOutput.__setstate__(self, name) self.__coreaudio__ = None def description(self): """returns user-facing name of output device as unicode""" # FIXME - pull this from device description return u"Core Audio" def set_format(self, sample_rate, channels, channel_mask, bits_per_sample): """sets the output stream to the given format if the stream hasn't been initialized, this method initializes it if the stream has been initialized to a different format, this method closes and reopens the stream to the new format if the stream has been initialized to the same format, this method does nothing""" if self.__coreaudio__ is None: # output hasn't been initialized from audiotools.output import CoreAudio AudioOutput.set_format(self, sample_rate, channels, channel_mask, bits_per_sample) self.__coreaudio__ = CoreAudio(sample_rate, channels, channel_mask, bits_per_sample) elif (not self.compatible(sample_rate=sample_rate, channels=channels, channel_mask=channel_mask, bits_per_sample=bits_per_sample)): # output has been initialized in a different format self.close() self.set_format(sample_rate=sample_rate, channels=channels, channel_mask=channel_mask, bits_per_sample=bits_per_sample) def play(self, framelist): """plays a FrameList""" self.__coreaudio__.play(framelist.to_bytes(False, True)) def pause(self): """pauses audio output, with the expectation it will be resumed""" if self.__coreaudio__ is not None: self.__coreaudio__.pause() def resume(self): """resumes playing paused audio output""" if self.__coreaudio__ is not None: self.__coreaudio__.resume() def get_volume(self): """returns a floating-point volume value between 0.0 and 1.0""" if self.__coreaudio__ is None: self.set_format(*DEFAULT_FORMAT) try: return self.__coreaudio__.get_volume() except ValueError: # get_volume_scalar() call was unsuccessful return 1.0 def set_volume(self, volume): """sets the output volume to a floating point value between 0.0 and 1.0""" if (volume >= 0) and (volume <= 1.0): if self.__coreaudio__ is None: self.set_format(*DEFAULT_FORMAT) try: self.__coreaudio__.set_volume(volume) except ValueError: # set_volume_scalar() call was unsuccessful pass else: raise ValueError("volume must be between 0.0 and 1.0") def close(self): """closes the output stream""" AudioOutput.close(self) if self.__coreaudio__ is not None: self.__coreaudio__.flush() self.__coreaudio__.close() self.__coreaudio__ = None @classmethod def available(cls): """returns True if the AudioOutput is available on the system""" try: from audiotools.output import CoreAudio return True except ImportError: return False def available_outputs(): """iterates over all available AudioOutput objects this will always yield at least one output""" if PulseAudioOutput.available(): yield PulseAudioOutput() if ALSAAudioOutput.available(): yield ALSAAudioOutput() if CoreAudioOutput.available(): yield CoreAudioOutput() if OSSAudioOutput.available(): yield OSSAudioOutput() yield NULLAudioOutput() def open_output(output): """given an output type string (e.g. "PulseAudio") returns that AudioOutput instance or raises ValueError if it is unavailable""" for audio_output in available_outputs(): if audio_output.NAME == output: return audio_output else: raise ValueError("no such outout {}".format(output)) ================================================ FILE: audiotools/ply/README ================================================ PLY (Python Lex-Yacc) Version 3.4 Copyright (C) 2001-2011, David M. Beazley (Dabeaz LLC) All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of the David Beazley or Dabeaz LLC may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Introduction ============ PLY is a 100% Python implementation of the common parsing tools lex and yacc. Here are a few highlights: - PLY is very closely modeled after traditional lex/yacc. If you know how to use these tools in C, you will find PLY to be similar. - PLY provides *very* extensive error reporting and diagnostic information to assist in parser construction. The original implementation was developed for instructional purposes. As a result, the system tries to identify the most common types of errors made by novice users. - PLY provides full support for empty productions, error recovery, precedence specifiers, and moderately ambiguous grammars. - Parsing is based on LR-parsing which is fast, memory efficient, better suited to large grammars, and which has a number of nice properties when dealing with syntax errors and other parsing problems. Currently, PLY builds its parsing tables using the LALR(1) algorithm used in yacc. - PLY uses Python introspection features to build lexers and parsers. This greatly simplifies the task of parser construction since it reduces the number of files and eliminates the need to run a separate lex/yacc tool before running your program. - PLY can be used to build parsers for "real" programming languages. Although it is not ultra-fast due to its Python implementation, PLY can be used to parse grammars consisting of several hundred rules (as might be found for a language like C). The lexer and LR parser are also reasonably efficient when parsing typically sized programs. People have used PLY to build parsers for C, C++, ADA, and other real programming languages. How to Use ========== PLY consists of two files : lex.py and yacc.py. These are contained within the 'ply' directory which may also be used as a Python package. To use PLY, simply copy the 'ply' directory to your project and import lex and yacc from the associated 'ply' package. For example: import ply.lex as lex import ply.yacc as yacc Alternatively, you can copy just the files lex.py and yacc.py individually and use them as modules. For example: import lex import yacc The file setup.py can be used to install ply using distutils. The file doc/ply.html contains complete documentation on how to use the system. The example directory contains several different examples including a PLY specification for ANSI C as given in K&R 2nd Ed. A simple example is found at the end of this document Requirements ============ PLY requires the use of Python 2.2 or greater. However, you should use the latest Python release if possible. It should work on just about any platform. PLY has been tested with both CPython and Jython. It also seems to work with IronPython. Resources ========= More information about PLY can be obtained on the PLY webpage at: http://www.dabeaz.com/ply For a detailed overview of parsing theory, consult the excellent book "Compilers : Principles, Techniques, and Tools" by Aho, Sethi, and Ullman. The topics found in "Lex & Yacc" by Levine, Mason, and Brown may also be useful. A Google group for PLY can be found at http://groups.google.com/group/ply-hack Acknowledgments =============== A special thanks is in order for all of the students in CS326 who suffered through about 25 different versions of these tools :-). The CHANGES file acknowledges those who have contributed patches. Elias Ioup did the first implementation of LALR(1) parsing in PLY-1.x. Andrew Waters and Markus Schoepflin were instrumental in reporting bugs and testing a revised LALR(1) implementation for PLY-2.0. Special Note for PLY-3.0 ======================== PLY-3.0 the first PLY release to support Python 3. However, backwards compatibility with Python 2.2 is still preserved. PLY provides dual Python 2/3 compatibility by restricting its implementation to a common subset of basic language features. You should not convert PLY using 2to3--it is not necessary and may in fact break the implementation. Example ======= Here is a simple example showing a PLY implementation of a calculator with variables. # ----------------------------------------------------------------------------- # calc.py # # A simple calculator with variables. # ----------------------------------------------------------------------------- tokens = ( 'NAME','NUMBER', 'PLUS','MINUS','TIMES','DIVIDE','EQUALS', 'LPAREN','RPAREN', ) # Tokens t_PLUS = r'\+' t_MINUS = r'-' t_TIMES = r'\*' t_DIVIDE = r'/' t_EQUALS = r'=' t_LPAREN = r'\(' t_RPAREN = r'\)' t_NAME = r'[a-zA-Z_][a-zA-Z0-9_]*' def t_NUMBER(t): r'\d+' t.value = int(t.value) return t # Ignored characters t_ignore = " \t" def t_newline(t): r'\n+' t.lexer.lineno += t.value.count("\n") def t_error(t): print("Illegal character '%s'" % t.value[0]) t.lexer.skip(1) # Build the lexer import ply.lex as lex lex.lex() # Precedence rules for the arithmetic operators precedence = ( ('left','PLUS','MINUS'), ('left','TIMES','DIVIDE'), ('right','UMINUS'), ) # dictionary of names (for storing variables) names = { } def p_statement_assign(p): 'statement : NAME EQUALS expression' names[p[1]] = p[3] def p_statement_expr(p): 'statement : expression' print(p[1]) def p_expression_binop(p): '''expression : expression PLUS expression | expression MINUS expression | expression TIMES expression | expression DIVIDE expression''' if p[2] == '+' : p[0] = p[1] + p[3] elif p[2] == '-': p[0] = p[1] - p[3] elif p[2] == '*': p[0] = p[1] * p[3] elif p[2] == '/': p[0] = p[1] / p[3] def p_expression_uminus(p): 'expression : MINUS expression %prec UMINUS' p[0] = -p[2] def p_expression_group(p): 'expression : LPAREN expression RPAREN' p[0] = p[2] def p_expression_number(p): 'expression : NUMBER' p[0] = p[1] def p_expression_name(p): 'expression : NAME' try: p[0] = names[p[1]] except LookupError: print("Undefined name '%s'" % p[1]) p[0] = 0 def p_error(p): print("Syntax error at '%s'" % p.value) import ply.yacc as yacc yacc.yacc() while 1: try: s = raw_input('calc > ') # use input() on Python 3 except EOFError: break yacc.parse(s) Bug Reports and Patches ======================= My goal with PLY is to simply have a decent lex/yacc implementation for Python. As a general rule, I don't spend huge amounts of time working on it unless I receive very specific bug reports and/or patches to fix problems. I also try to incorporate submitted feature requests and enhancements into each new version. To contact me about bugs and/or new features, please send email to dave@dabeaz.com. In addition there is a Google group for discussing PLY related issues at http://groups.google.com/group/ply-hack -- Dave ================================================ FILE: audiotools/ply/__init__.py ================================================ # Copyright (C) 2001-2011, # David M. Beazley (Dabeaz LLC) # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are # met: # # * Redistributions of source code must retain the above copyright notice, # this list of conditions and the following disclaimer. # * Redistributions in binary form must reproduce the above copyright notice, # this list of conditions and the following disclaimer in the documentation # and/or other materials provided with the distribution. # * Neither the name of the David Beazley or Dabeaz LLC may be used to # endorse or promote products derived from this software without # specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # ----------------------------------------------------------------------------- __version__ = '3.7' __all__ = ['lex','yacc'] ================================================ FILE: audiotools/ply/lex.py ================================================ # ----------------------------------------------------------------------------- # ply: lex.py # # Copyright (C) 2001-2015, # David M. Beazley (Dabeaz LLC) # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are # met: # # * Redistributions of source code must retain the above copyright notice, # this list of conditions and the following disclaimer. # * Redistributions in binary form must reproduce the above copyright notice, # this list of conditions and the following disclaimer in the documentation # and/or other materials provided with the distribution. # * Neither the name of the David Beazley or Dabeaz LLC may be used to # endorse or promote products derived from this software without # specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # ----------------------------------------------------------------------------- __version__ = '3.8' __tabversion__ = '3.8' import re import sys import types import copy import os import inspect # This tuple contains known string types try: # Python 2.6 StringTypes = (types.StringType, types.UnicodeType) except AttributeError: # Python 3.0 StringTypes = (str, bytes) # This regular expression is used to match valid token names _is_identifier = re.compile(r'^[a-zA-Z0-9_]+$') # Exception thrown when invalid token encountered and no default error # handler is defined. class LexError(Exception): def __init__(self, message, s): self.args = (message,) self.text = s # Token class. This class is used to represent the tokens produced. class LexToken(object): def __str__(self): return 'LexToken(%s,%r,%d,%d)' % (self.type, self.value, self.lineno, self.lexpos) def __repr__(self): return str(self) # This object is a stand-in for a logging object created by the # logging module. class PlyLogger(object): def __init__(self, f): self.f = f def critical(self, msg, *args, **kwargs): self.f.write((msg % args) + '\n') def warning(self, msg, *args, **kwargs): self.f.write('WARNING: ' + (msg % args) + '\n') def error(self, msg, *args, **kwargs): self.f.write('ERROR: ' + (msg % args) + '\n') info = critical debug = critical # Null logger is used when no output is generated. Does nothing. class NullLogger(object): def __getattribute__(self, name): return self def __call__(self, *args, **kwargs): return self # ----------------------------------------------------------------------------- # === Lexing Engine === # # The following Lexer class implements the lexer runtime. There are only # a few public methods and attributes: # # input() - Store a new string in the lexer # token() - Get the next token # clone() - Clone the lexer # # lineno - Current line number # lexpos - Current position in the input string # ----------------------------------------------------------------------------- class Lexer: def __init__(self): self.lexre = None # Master regular expression. This is a list of # tuples (re, findex) where re is a compiled # regular expression and findex is a list # mapping regex group numbers to rules self.lexretext = None # Current regular expression strings self.lexstatere = {} # Dictionary mapping lexer states to master regexs self.lexstateretext = {} # Dictionary mapping lexer states to regex strings self.lexstaterenames = {} # Dictionary mapping lexer states to symbol names self.lexstate = 'INITIAL' # Current lexer state self.lexstatestack = [] # Stack of lexer states self.lexstateinfo = None # State information self.lexstateignore = {} # Dictionary of ignored characters for each state self.lexstateerrorf = {} # Dictionary of error functions for each state self.lexstateeoff = {} # Dictionary of eof functions for each state self.lexreflags = 0 # Optional re compile flags self.lexdata = None # Actual input data (as a string) self.lexpos = 0 # Current position in input text self.lexlen = 0 # Length of the input text self.lexerrorf = None # Error rule (if any) self.lexeoff = None # EOF rule (if any) self.lextokens = None # List of valid tokens self.lexignore = '' # Ignored characters self.lexliterals = '' # Literal characters that can be passed through self.lexmodule = None # Module self.lineno = 1 # Current line number self.lexoptimize = False # Optimized mode def clone(self, object=None): c = copy.copy(self) # If the object parameter has been supplied, it means we are attaching the # lexer to a new object. In this case, we have to rebind all methods in # the lexstatere and lexstateerrorf tables. if object: newtab = {} for key, ritem in self.lexstatere.items(): newre = [] for cre, findex in ritem: newfindex = [] for f in findex: if not f or not f[0]: newfindex.append(f) continue newfindex.append((getattr(object, f[0].__name__), f[1])) newre.append((cre, newfindex)) newtab[key] = newre c.lexstatere = newtab c.lexstateerrorf = {} for key, ef in self.lexstateerrorf.items(): c.lexstateerrorf[key] = getattr(object, ef.__name__) c.lexmodule = object return c # ------------------------------------------------------------ # writetab() - Write lexer information to a table file # ------------------------------------------------------------ def writetab(self, lextab, outputdir=''): if isinstance(lextab, types.ModuleType): raise IOError("Won't overwrite existing lextab module") basetabmodule = lextab.split('.')[-1] filename = os.path.join(outputdir, basetabmodule) + '.py' with open(filename, 'w') as tf: tf.write('# %s.py. This file automatically created by PLY (version %s). Don\'t edit!\n' % (basetabmodule, __version__)) tf.write('_tabversion = %s\n' % repr(__tabversion__)) tf.write('_lextokens = %s\n' % repr(self.lextokens)) tf.write('_lexreflags = %s\n' % repr(self.lexreflags)) tf.write('_lexliterals = %s\n' % repr(self.lexliterals)) tf.write('_lexstateinfo = %s\n' % repr(self.lexstateinfo)) # Rewrite the lexstatere table, replacing function objects with function names tabre = {} for statename, lre in self.lexstatere.items(): titem = [] for (pat, func), retext, renames in zip(lre, self.lexstateretext[statename], self.lexstaterenames[statename]): titem.append((retext, _funcs_to_names(func, renames))) tabre[statename] = titem tf.write('_lexstatere = %s\n' % repr(tabre)) tf.write('_lexstateignore = %s\n' % repr(self.lexstateignore)) taberr = {} for statename, ef in self.lexstateerrorf.items(): taberr[statename] = ef.__name__ if ef else None tf.write('_lexstateerrorf = %s\n' % repr(taberr)) tabeof = {} for statename, ef in self.lexstateeoff.items(): tabeof[statename] = ef.__name__ if ef else None tf.write('_lexstateeoff = %s\n' % repr(tabeof)) # ------------------------------------------------------------ # readtab() - Read lexer information from a tab file # ------------------------------------------------------------ def readtab(self, tabfile, fdict): if isinstance(tabfile, types.ModuleType): lextab = tabfile else: exec('import %s' % tabfile) lextab = sys.modules[tabfile] if getattr(lextab, '_tabversion', '0.0') != __tabversion__: raise ImportError('Inconsistent PLY version') self.lextokens = lextab._lextokens self.lexreflags = lextab._lexreflags self.lexliterals = lextab._lexliterals self.lextokens_all = self.lextokens | set(self.lexliterals) self.lexstateinfo = lextab._lexstateinfo self.lexstateignore = lextab._lexstateignore self.lexstatere = {} self.lexstateretext = {} for statename, lre in lextab._lexstatere.items(): titem = [] txtitem = [] for pat, func_name in lre: titem.append((re.compile(pat, lextab._lexreflags | re.VERBOSE), _names_to_funcs(func_name, fdict))) self.lexstatere[statename] = titem self.lexstateretext[statename] = txtitem self.lexstateerrorf = {} for statename, ef in lextab._lexstateerrorf.items(): self.lexstateerrorf[statename] = fdict[ef] self.lexstateeoff = {} for statename, ef in lextab._lexstateeoff.items(): self.lexstateeoff[statename] = fdict[ef] self.begin('INITIAL') # ------------------------------------------------------------ # input() - Push a new string into the lexer # ------------------------------------------------------------ def input(self, s): # Pull off the first character to see if s looks like a string c = s[:1] if not isinstance(c, StringTypes): raise ValueError('Expected a string') self.lexdata = s self.lexpos = 0 self.lexlen = len(s) # ------------------------------------------------------------ # begin() - Changes the lexing state # ------------------------------------------------------------ def begin(self, state): if state not in self.lexstatere: raise ValueError('Undefined state') self.lexre = self.lexstatere[state] self.lexretext = self.lexstateretext[state] self.lexignore = self.lexstateignore.get(state, '') self.lexerrorf = self.lexstateerrorf.get(state, None) self.lexeoff = self.lexstateeoff.get(state, None) self.lexstate = state # ------------------------------------------------------------ # push_state() - Changes the lexing state and saves old on stack # ------------------------------------------------------------ def push_state(self, state): self.lexstatestack.append(self.lexstate) self.begin(state) # ------------------------------------------------------------ # pop_state() - Restores the previous state # ------------------------------------------------------------ def pop_state(self): self.begin(self.lexstatestack.pop()) # ------------------------------------------------------------ # current_state() - Returns the current lexing state # ------------------------------------------------------------ def current_state(self): return self.lexstate # ------------------------------------------------------------ # skip() - Skip ahead n characters # ------------------------------------------------------------ def skip(self, n): self.lexpos += n # ------------------------------------------------------------ # opttoken() - Return the next token from the Lexer # # Note: This function has been carefully implemented to be as fast # as possible. Don't make changes unless you really know what # you are doing # ------------------------------------------------------------ def token(self): # Make local copies of frequently referenced attributes lexpos = self.lexpos lexlen = self.lexlen lexignore = self.lexignore lexdata = self.lexdata while lexpos < lexlen: # This code provides some short-circuit code for whitespace, tabs, and other ignored characters if lexdata[lexpos] in lexignore: lexpos += 1 continue # Look for a regular expression match for lexre, lexindexfunc in self.lexre: m = lexre.match(lexdata, lexpos) if not m: continue # Create a token for return tok = LexToken() tok.value = m.group() tok.lineno = self.lineno tok.lexpos = lexpos i = m.lastindex func, tok.type = lexindexfunc[i] if not func: # If no token type was set, it's an ignored token if tok.type: self.lexpos = m.end() return tok else: lexpos = m.end() break lexpos = m.end() # If token is processed by a function, call it tok.lexer = self # Set additional attributes useful in token rules self.lexmatch = m self.lexpos = lexpos newtok = func(tok) # Every function must return a token, if nothing, we just move to next token if not newtok: lexpos = self.lexpos # This is here in case user has updated lexpos. lexignore = self.lexignore # This is here in case there was a state change break # Verify type of the token. If not in the token map, raise an error if not self.lexoptimize: if newtok.type not in self.lextokens_all: raise LexError("%s:%d: Rule '%s' returned an unknown token type '%s'" % ( func.__code__.co_filename, func.__code__.co_firstlineno, func.__name__, newtok.type), lexdata[lexpos:]) return newtok else: # No match, see if in literals if lexdata[lexpos] in self.lexliterals: tok = LexToken() tok.value = lexdata[lexpos] tok.lineno = self.lineno tok.type = tok.value tok.lexpos = lexpos self.lexpos = lexpos + 1 return tok # No match. Call t_error() if defined. if self.lexerrorf: tok = LexToken() tok.value = self.lexdata[lexpos:] tok.lineno = self.lineno tok.type = 'error' tok.lexer = self tok.lexpos = lexpos self.lexpos = lexpos newtok = self.lexerrorf(tok) if lexpos == self.lexpos: # Error method didn't change text position at all. This is an error. raise LexError("Scanning error. Illegal character '%s'" % (lexdata[lexpos]), lexdata[lexpos:]) lexpos = self.lexpos if not newtok: continue return newtok self.lexpos = lexpos raise LexError("Illegal character '%s' at index %d" % (lexdata[lexpos], lexpos), lexdata[lexpos:]) if self.lexeoff: tok = LexToken() tok.type = 'eof' tok.value = '' tok.lineno = self.lineno tok.lexpos = lexpos tok.lexer = self self.lexpos = lexpos newtok = self.lexeoff(tok) return newtok self.lexpos = lexpos + 1 if self.lexdata is None: raise RuntimeError('No input string given with input()') return None # Iterator interface def __iter__(self): return self def next(self): t = self.token() if t is None: raise StopIteration return t __next__ = next # ----------------------------------------------------------------------------- # ==== Lex Builder === # # The functions and classes below are used to collect lexing information # and build a Lexer object from it. # ----------------------------------------------------------------------------- # ----------------------------------------------------------------------------- # _get_regex(func) # # Returns the regular expression assigned to a function either as a doc string # or as a .regex attribute attached by the @TOKEN decorator. # ----------------------------------------------------------------------------- def _get_regex(func): return getattr(func, 'regex', func.__doc__) # ----------------------------------------------------------------------------- # get_caller_module_dict() # # This function returns a dictionary containing all of the symbols defined within # a caller further down the call stack. This is used to get the environment # associated with the yacc() call if none was provided. # ----------------------------------------------------------------------------- def get_caller_module_dict(levels): f = sys._getframe(levels) ldict = f.f_globals.copy() if f.f_globals != f.f_locals: ldict.update(f.f_locals) return ldict # ----------------------------------------------------------------------------- # _funcs_to_names() # # Given a list of regular expression functions, this converts it to a list # suitable for output to a table file # ----------------------------------------------------------------------------- def _funcs_to_names(funclist, namelist): result = [] for f, name in zip(funclist, namelist): if f and f[0]: result.append((name, f[1])) else: result.append(f) return result # ----------------------------------------------------------------------------- # _names_to_funcs() # # Given a list of regular expression function names, this converts it back to # functions. # ----------------------------------------------------------------------------- def _names_to_funcs(namelist, fdict): result = [] for n in namelist: if n and n[0]: result.append((fdict[n[0]], n[1])) else: result.append(n) return result # ----------------------------------------------------------------------------- # _form_master_re() # # This function takes a list of all of the regex components and attempts to # form the master regular expression. Given limitations in the Python re # module, it may be necessary to break the master regex into separate expressions. # ----------------------------------------------------------------------------- def _form_master_re(relist, reflags, ldict, toknames): if not relist: return [] regex = '|'.join(relist) try: lexre = re.compile(regex, re.VERBOSE | reflags) # Build the index to function map for the matching engine lexindexfunc = [None] * (max(lexre.groupindex.values()) + 1) lexindexnames = lexindexfunc[:] for f, i in lexre.groupindex.items(): handle = ldict.get(f, None) if type(handle) in (types.FunctionType, types.MethodType): lexindexfunc[i] = (handle, toknames[f]) lexindexnames[i] = f elif handle is not None: lexindexnames[i] = f if f.find('ignore_') > 0: lexindexfunc[i] = (None, None) else: lexindexfunc[i] = (None, toknames[f]) return [(lexre, lexindexfunc)], [regex], [lexindexnames] except Exception: m = int(len(relist)/2) if m == 0: m = 1 llist, lre, lnames = _form_master_re(relist[:m], reflags, ldict, toknames) rlist, rre, rnames = _form_master_re(relist[m:], reflags, ldict, toknames) return (llist+rlist), (lre+rre), (lnames+rnames) # ----------------------------------------------------------------------------- # def _statetoken(s,names) # # Given a declaration name s of the form "t_" and a dictionary whose keys are # state names, this function returns a tuple (states,tokenname) where states # is a tuple of state names and tokenname is the name of the token. For example, # calling this with s = "t_foo_bar_SPAM" might return (('foo','bar'),'SPAM') # ----------------------------------------------------------------------------- def _statetoken(s, names): nonstate = 1 parts = s.split('_') for i, part in enumerate(parts[1:], 1): if part not in names and part != 'ANY': break if i > 1: states = tuple(parts[1:i]) else: states = ('INITIAL',) if 'ANY' in states: states = tuple(names) tokenname = '_'.join(parts[i:]) return (states, tokenname) # ----------------------------------------------------------------------------- # LexerReflect() # # This class represents information needed to build a lexer as extracted from a # user's input file. # ----------------------------------------------------------------------------- class LexerReflect(object): def __init__(self, ldict, log=None, reflags=0): self.ldict = ldict self.error_func = None self.tokens = [] self.reflags = reflags self.stateinfo = {'INITIAL': 'inclusive'} self.modules = set() self.error = False self.log = PlyLogger(sys.stderr) if log is None else log # Get all of the basic information def get_all(self): self.get_tokens() self.get_literals() self.get_states() self.get_rules() # Validate all of the information def validate_all(self): self.validate_tokens() self.validate_literals() self.validate_rules() return self.error # Get the tokens map def get_tokens(self): tokens = self.ldict.get('tokens', None) if not tokens: self.log.error('No token list is defined') self.error = True return if not isinstance(tokens, (list, tuple)): self.log.error('tokens must be a list or tuple') self.error = True return if not tokens: self.log.error('tokens is empty') self.error = True return self.tokens = tokens # Validate the tokens def validate_tokens(self): terminals = {} for n in self.tokens: if not _is_identifier.match(n): self.log.error("Bad token name '%s'", n) self.error = True if n in terminals: self.log.warning("Token '%s' multiply defined", n) terminals[n] = 1 # Get the literals specifier def get_literals(self): self.literals = self.ldict.get('literals', '') if not self.literals: self.literals = '' # Validate literals def validate_literals(self): try: for c in self.literals: if not isinstance(c, StringTypes) or len(c) > 1: self.log.error('Invalid literal %s. Must be a single character', repr(c)) self.error = True except TypeError: self.log.error('Invalid literals specification. literals must be a sequence of characters') self.error = True def get_states(self): self.states = self.ldict.get('states', None) # Build statemap if self.states: if not isinstance(self.states, (tuple, list)): self.log.error('states must be defined as a tuple or list') self.error = True else: for s in self.states: if not isinstance(s, tuple) or len(s) != 2: self.log.error("Invalid state specifier %s. Must be a tuple (statename,'exclusive|inclusive')", repr(s)) self.error = True continue name, statetype = s if not isinstance(name, StringTypes): self.log.error('State name %s must be a string', repr(name)) self.error = True continue if not (statetype == 'inclusive' or statetype == 'exclusive'): self.log.error("State type for state %s must be 'inclusive' or 'exclusive'", name) self.error = True continue if name in self.stateinfo: self.log.error("State '%s' already defined", name) self.error = True continue self.stateinfo[name] = statetype # Get all of the symbols with a t_ prefix and sort them into various # categories (functions, strings, error functions, and ignore characters) def get_rules(self): tsymbols = [f for f in self.ldict if f[:2] == 't_'] # Now build up a list of functions and a list of strings self.toknames = {} # Mapping of symbols to token names self.funcsym = {} # Symbols defined as functions self.strsym = {} # Symbols defined as strings self.ignore = {} # Ignore strings by state self.errorf = {} # Error functions by state self.eoff = {} # EOF functions by state for s in self.stateinfo: self.funcsym[s] = [] self.strsym[s] = [] if len(tsymbols) == 0: self.log.error('No rules of the form t_rulename are defined') self.error = True return for f in tsymbols: t = self.ldict[f] states, tokname = _statetoken(f, self.stateinfo) self.toknames[f] = tokname if hasattr(t, '__call__'): if tokname == 'error': for s in states: self.errorf[s] = t elif tokname == 'eof': for s in states: self.eoff[s] = t elif tokname == 'ignore': line = t.__code__.co_firstlineno file = t.__code__.co_filename self.log.error("%s:%d: Rule '%s' must be defined as a string", file, line, t.__name__) self.error = True else: for s in states: self.funcsym[s].append((f, t)) elif isinstance(t, StringTypes): if tokname == 'ignore': for s in states: self.ignore[s] = t if '\\' in t: self.log.warning("%s contains a literal backslash '\\'", f) elif tokname == 'error': self.log.error("Rule '%s' must be defined as a function", f) self.error = True else: for s in states: self.strsym[s].append((f, t)) else: self.log.error('%s not defined as a function or string', f) self.error = True # Sort the functions by line number for f in self.funcsym.values(): f.sort(key=lambda x: x[1].__code__.co_firstlineno) # Sort the strings by regular expression length for s in self.strsym.values(): s.sort(key=lambda x: len(x[1]), reverse=True) # Validate all of the t_rules collected def validate_rules(self): for state in self.stateinfo: # Validate all rules defined by functions for fname, f in self.funcsym[state]: line = f.__code__.co_firstlineno file = f.__code__.co_filename module = inspect.getmodule(f) self.modules.add(module) tokname = self.toknames[fname] if isinstance(f, types.MethodType): reqargs = 2 else: reqargs = 1 nargs = f.__code__.co_argcount if nargs > reqargs: self.log.error("%s:%d: Rule '%s' has too many arguments", file, line, f.__name__) self.error = True continue if nargs < reqargs: self.log.error("%s:%d: Rule '%s' requires an argument", file, line, f.__name__) self.error = True continue if not _get_regex(f): self.log.error("%s:%d: No regular expression defined for rule '%s'", file, line, f.__name__) self.error = True continue try: c = re.compile('(?P<%s>%s)' % (fname, _get_regex(f)), re.VERBOSE | self.reflags) if c.match(''): self.log.error("%s:%d: Regular expression for rule '%s' matches empty string", file, line, f.__name__) self.error = True except re.error as e: self.log.error("%s:%d: Invalid regular expression for rule '%s'. %s", file, line, f.__name__, e) if '#' in _get_regex(f): self.log.error("%s:%d. Make sure '#' in rule '%s' is escaped with '\\#'", file, line, f.__name__) self.error = True # Validate all rules defined by strings for name, r in self.strsym[state]: tokname = self.toknames[name] if tokname == 'error': self.log.error("Rule '%s' must be defined as a function", name) self.error = True continue if tokname not in self.tokens and tokname.find('ignore_') < 0: self.log.error("Rule '%s' defined for an unspecified token %s", name, tokname) self.error = True continue try: c = re.compile('(?P<%s>%s)' % (name, r), re.VERBOSE | self.reflags) if (c.match('')): self.log.error("Regular expression for rule '%s' matches empty string", name) self.error = True except re.error as e: self.log.error("Invalid regular expression for rule '%s'. %s", name, e) if '#' in r: self.log.error("Make sure '#' in rule '%s' is escaped with '\\#'", name) self.error = True if not self.funcsym[state] and not self.strsym[state]: self.log.error("No rules defined for state '%s'", state) self.error = True # Validate the error function efunc = self.errorf.get(state, None) if efunc: f = efunc line = f.__code__.co_firstlineno file = f.__code__.co_filename module = inspect.getmodule(f) self.modules.add(module) if isinstance(f, types.MethodType): reqargs = 2 else: reqargs = 1 nargs = f.__code__.co_argcount if nargs > reqargs: self.log.error("%s:%d: Rule '%s' has too many arguments", file, line, f.__name__) self.error = True if nargs < reqargs: self.log.error("%s:%d: Rule '%s' requires an argument", file, line, f.__name__) self.error = True for module in self.modules: self.validate_module(module) # ----------------------------------------------------------------------------- # validate_module() # # This checks to see if there are duplicated t_rulename() functions or strings # in the parser input file. This is done using a simple regular expression # match on each line in the source code of the given module. # ----------------------------------------------------------------------------- def validate_module(self, module): lines, linen = inspect.getsourcelines(module) fre = re.compile(r'\s*def\s+(t_[a-zA-Z_0-9]*)\(') sre = re.compile(r'\s*(t_[a-zA-Z_0-9]*)\s*=') counthash = {} linen += 1 for line in lines: m = fre.match(line) if not m: m = sre.match(line) if m: name = m.group(1) prev = counthash.get(name) if not prev: counthash[name] = linen else: filename = inspect.getsourcefile(module) self.log.error('%s:%d: Rule %s redefined. Previously defined on line %d', filename, linen, name, prev) self.error = True linen += 1 # ----------------------------------------------------------------------------- # lex(module) # # Build all of the regular expression rules from definitions in the supplied module # ----------------------------------------------------------------------------- def lex(module=None, object=None, debug=False, optimize=False, lextab='lextab', reflags=0, nowarn=False, outputdir=None, debuglog=None, errorlog=None): if lextab is None: lextab = 'lextab' global lexer ldict = None stateinfo = {'INITIAL': 'inclusive'} lexobj = Lexer() lexobj.lexoptimize = optimize global token, input if errorlog is None: errorlog = PlyLogger(sys.stderr) if debug: if debuglog is None: debuglog = PlyLogger(sys.stderr) # Get the module dictionary used for the lexer if object: module = object # Get the module dictionary used for the parser if module: _items = [(k, getattr(module, k)) for k in dir(module)] ldict = dict(_items) # If no __file__ attribute is available, try to obtain it from the __module__ instead if '__file__' not in ldict: ldict['__file__'] = sys.modules[ldict['__module__']].__file__ else: ldict = get_caller_module_dict(2) # Determine if the module is package of a package or not. # If so, fix the tabmodule setting so that tables load correctly pkg = ldict.get('__package__') if pkg and isinstance(lextab, str): if '.' not in lextab: lextab = pkg + '.' + lextab # Collect parser information from the dictionary linfo = LexerReflect(ldict, log=errorlog, reflags=reflags) linfo.get_all() if not optimize: if linfo.validate_all(): raise SyntaxError("Can't build lexer") if optimize and lextab: try: lexobj.readtab(lextab, ldict) token = lexobj.token input = lexobj.input lexer = lexobj return lexobj except ImportError: pass # Dump some basic debugging information if debug: debuglog.info('lex: tokens = %r', linfo.tokens) debuglog.info('lex: literals = %r', linfo.literals) debuglog.info('lex: states = %r', linfo.stateinfo) # Build a dictionary of valid token names lexobj.lextokens = set() for n in linfo.tokens: lexobj.lextokens.add(n) # Get literals specification if isinstance(linfo.literals, (list, tuple)): lexobj.lexliterals = type(linfo.literals[0])().join(linfo.literals) else: lexobj.lexliterals = linfo.literals lexobj.lextokens_all = lexobj.lextokens | set(lexobj.lexliterals) # Get the stateinfo dictionary stateinfo = linfo.stateinfo regexs = {} # Build the master regular expressions for state in stateinfo: regex_list = [] # Add rules defined by functions first for fname, f in linfo.funcsym[state]: line = f.__code__.co_firstlineno file = f.__code__.co_filename regex_list.append('(?P<%s>%s)' % (fname, _get_regex(f))) if debug: debuglog.info("lex: Adding rule %s -> '%s' (state '%s')", fname, _get_regex(f), state) # Now add all of the simple rules for name, r in linfo.strsym[state]: regex_list.append('(?P<%s>%s)' % (name, r)) if debug: debuglog.info("lex: Adding rule %s -> '%s' (state '%s')", name, r, state) regexs[state] = regex_list # Build the master regular expressions if debug: debuglog.info('lex: ==== MASTER REGEXS FOLLOW ====') for state in regexs: lexre, re_text, re_names = _form_master_re(regexs[state], reflags, ldict, linfo.toknames) lexobj.lexstatere[state] = lexre lexobj.lexstateretext[state] = re_text lexobj.lexstaterenames[state] = re_names if debug: for i, text in enumerate(re_text): debuglog.info("lex: state '%s' : regex[%d] = '%s'", state, i, text) # For inclusive states, we need to add the regular expressions from the INITIAL state for state, stype in stateinfo.items(): if state != 'INITIAL' and stype == 'inclusive': lexobj.lexstatere[state].extend(lexobj.lexstatere['INITIAL']) lexobj.lexstateretext[state].extend(lexobj.lexstateretext['INITIAL']) lexobj.lexstaterenames[state].extend(lexobj.lexstaterenames['INITIAL']) lexobj.lexstateinfo = stateinfo lexobj.lexre = lexobj.lexstatere['INITIAL'] lexobj.lexretext = lexobj.lexstateretext['INITIAL'] lexobj.lexreflags = reflags # Set up ignore variables lexobj.lexstateignore = linfo.ignore lexobj.lexignore = lexobj.lexstateignore.get('INITIAL', '') # Set up error functions lexobj.lexstateerrorf = linfo.errorf lexobj.lexerrorf = linfo.errorf.get('INITIAL', None) if not lexobj.lexerrorf: errorlog.warning('No t_error rule is defined') # Set up eof functions lexobj.lexstateeoff = linfo.eoff lexobj.lexeoff = linfo.eoff.get('INITIAL', None) # Check state information for ignore and error rules for s, stype in stateinfo.items(): if stype == 'exclusive': if s not in linfo.errorf: errorlog.warning("No error rule is defined for exclusive state '%s'", s) if s not in linfo.ignore and lexobj.lexignore: errorlog.warning("No ignore rule is defined for exclusive state '%s'", s) elif stype == 'inclusive': if s not in linfo.errorf: linfo.errorf[s] = linfo.errorf.get('INITIAL', None) if s not in linfo.ignore: linfo.ignore[s] = linfo.ignore.get('INITIAL', '') # Create global versions of the token() and input() functions token = lexobj.token input = lexobj.input lexer = lexobj # If in optimize mode, we write the lextab if lextab and optimize: if outputdir is None: # If no output directory is set, the location of the output files # is determined according to the following rules: # - If lextab specifies a package, files go into that package directory # - Otherwise, files go in the same directory as the specifying module if isinstance(lextab, types.ModuleType): srcfile = lextab.__file__ else: if '.' not in lextab: srcfile = ldict['__file__'] else: parts = lextab.split('.') pkgname = '.'.join(parts[:-1]) exec('import %s' % pkgname) srcfile = getattr(sys.modules[pkgname], '__file__', '') outputdir = os.path.dirname(srcfile) try: lexobj.writetab(lextab, outputdir) except IOError as e: errorlog.warning("Couldn't write lextab module %r. %s" % (lextab, e)) return lexobj # ----------------------------------------------------------------------------- # runmain() # # This runs the lexer as a main program # ----------------------------------------------------------------------------- def runmain(lexer=None, data=None): if not data: try: filename = sys.argv[1] f = open(filename) data = f.read() f.close() except IndexError: sys.stdout.write('Reading from standard input (type EOF to end):\n') data = sys.stdin.read() if lexer: _input = lexer.input else: _input = input _input(data) if lexer: _token = lexer.token else: _token = token while True: tok = _token() if not tok: break sys.stdout.write('(%s,%r,%d,%d)\n' % (tok.type, tok.value, tok.lineno, tok.lexpos)) # ----------------------------------------------------------------------------- # @TOKEN(regex) # # This decorator function can be used to set the regex expression on a function # when its docstring might need to be set in an alternative way # ----------------------------------------------------------------------------- def TOKEN(r): def set_regex(f): if hasattr(r, '__call__'): f.regex = _get_regex(r) else: f.regex = r return f return set_regex # Alternative spelling of the TOKEN decorator Token = TOKEN ================================================ FILE: audiotools/ply/yacc.py ================================================ # ----------------------------------------------------------------------------- # ply: yacc.py # # Copyright (C) 2001-2015, # David M. Beazley (Dabeaz LLC) # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are # met: # # * Redistributions of source code must retain the above copyright notice, # this list of conditions and the following disclaimer. # * Redistributions in binary form must reproduce the above copyright notice, # this list of conditions and the following disclaimer in the documentation # and/or other materials provided with the distribution. # * Neither the name of the David Beazley or Dabeaz LLC may be used to # endorse or promote products derived from this software without # specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # ----------------------------------------------------------------------------- # # This implements an LR parser that is constructed from grammar rules defined # as Python functions. The grammer is specified by supplying the BNF inside # Python documentation strings. The inspiration for this technique was borrowed # from John Aycock's Spark parsing system. PLY might be viewed as cross between # Spark and the GNU bison utility. # # The current implementation is only somewhat object-oriented. The # LR parser itself is defined in terms of an object (which allows multiple # parsers to co-exist). However, most of the variables used during table # construction are defined in terms of global variables. Users shouldn't # notice unless they are trying to define multiple parsers at the same # time using threads (in which case they should have their head examined). # # This implementation supports both SLR and LALR(1) parsing. LALR(1) # support was originally implemented by Elias Ioup (ezioup@alumni.uchicago.edu), # using the algorithm found in Aho, Sethi, and Ullman "Compilers: Principles, # Techniques, and Tools" (The Dragon Book). LALR(1) has since been replaced # by the more efficient DeRemer and Pennello algorithm. # # :::::::: WARNING ::::::: # # Construction of LR parsing tables is fairly complicated and expensive. # To make this module run fast, a *LOT* of work has been put into # optimization---often at the expensive of readability and what might # consider to be good Python "coding style." Modify the code at your # own risk! # ---------------------------------------------------------------------------- import re import types import sys import os.path import inspect import base64 import warnings __version__ = '3.8' __tabversion__ = '3.8' #----------------------------------------------------------------------------- # === User configurable parameters === # # Change these to modify the default behavior of yacc (if you wish) #----------------------------------------------------------------------------- yaccdebug = True # Debugging mode. If set, yacc generates a # a 'parser.out' file in the current directory debug_file = 'parser.out' # Default name of the debugging file tab_module = 'parsetab' # Default name of the table module default_lr = 'LALR' # Default LR table generation method error_count = 3 # Number of symbols that must be shifted to leave recovery mode yaccdevel = False # Set to True if developing yacc. This turns off optimized # implementations of certain functions. resultlimit = 40 # Size limit of results when running in debug mode. pickle_protocol = 0 # Protocol to use when writing pickle files # String type-checking compatibility if sys.version_info[0] < 3: string_types = basestring else: string_types = str MAXINT = sys.maxsize # This object is a stand-in for a logging object created by the # logging module. PLY will use this by default to create things # such as the parser.out file. If a user wants more detailed # information, they can create their own logging object and pass # it into PLY. class PlyLogger(object): def __init__(self, f): self.f = f def debug(self, msg, *args, **kwargs): self.f.write((msg % args) + '\n') info = debug def warning(self, msg, *args, **kwargs): self.f.write('WARNING: ' + (msg % args) + '\n') def error(self, msg, *args, **kwargs): self.f.write('ERROR: ' + (msg % args) + '\n') critical = debug # Null logger is used when no output is generated. Does nothing. class NullLogger(object): def __getattribute__(self, name): return self def __call__(self, *args, **kwargs): return self # Exception raised for yacc-related errors class YaccError(Exception): pass # Format the result message that the parser produces when running in debug mode. def format_result(r): repr_str = repr(r) if '\n' in repr_str: repr_str = repr(repr_str) if len(repr_str) > resultlimit: repr_str = repr_str[:resultlimit] + ' ...' result = '<%s @ 0x%x> (%s)' % (type(r).__name__, id(r), repr_str) return result # Format stack entries when the parser is running in debug mode def format_stack_entry(r): repr_str = repr(r) if '\n' in repr_str: repr_str = repr(repr_str) if len(repr_str) < 16: return repr_str else: return '<%s @ 0x%x>' % (type(r).__name__, id(r)) # Panic mode error recovery support. This feature is being reworked--much of the # code here is to offer a deprecation/backwards compatible transition _errok = None _token = None _restart = None _warnmsg = '''PLY: Don't use global functions errok(), token(), and restart() in p_error(). Instead, invoke the methods on the associated parser instance: def p_error(p): ... # Use parser.errok(), parser.token(), parser.restart() ... parser = yacc.yacc() ''' def errok(): warnings.warn(_warnmsg) return _errok() def restart(): warnings.warn(_warnmsg) return _restart() def token(): warnings.warn(_warnmsg) return _token() # Utility function to call the p_error() function with some deprecation hacks def call_errorfunc(errorfunc, token, parser): global _errok, _token, _restart _errok = parser.errok _token = parser.token _restart = parser.restart r = errorfunc(token) try: del _errok, _token, _restart except NameError: pass return r #----------------------------------------------------------------------------- # === LR Parsing Engine === # # The following classes are used for the LR parser itself. These are not # used during table construction and are independent of the actual LR # table generation algorithm #----------------------------------------------------------------------------- # This class is used to hold non-terminal grammar symbols during parsing. # It normally has the following attributes set: # .type = Grammar symbol type # .value = Symbol value # .lineno = Starting line number # .endlineno = Ending line number (optional, set automatically) # .lexpos = Starting lex position # .endlexpos = Ending lex position (optional, set automatically) class YaccSymbol: def __str__(self): return self.type def __repr__(self): return str(self) # This class is a wrapper around the objects actually passed to each # grammar rule. Index lookup and assignment actually assign the # .value attribute of the underlying YaccSymbol object. # The lineno() method returns the line number of a given # item (or 0 if not defined). The linespan() method returns # a tuple of (startline,endline) representing the range of lines # for a symbol. The lexspan() method returns a tuple (lexpos,endlexpos) # representing the range of positional information for a symbol. class YaccProduction: def __init__(self, s, stack=None): self.slice = s self.stack = stack self.lexer = None self.parser = None def __getitem__(self, n): if isinstance(n, slice): return [s.value for s in self.slice[n]] elif n >= 0: return self.slice[n].value else: return self.stack[n].value def __setitem__(self, n, v): self.slice[n].value = v def __getslice__(self, i, j): return [s.value for s in self.slice[i:j]] def __len__(self): return len(self.slice) def lineno(self, n): return getattr(self.slice[n], 'lineno', 0) def set_lineno(self, n, lineno): self.slice[n].lineno = lineno def linespan(self, n): startline = getattr(self.slice[n], 'lineno', 0) endline = getattr(self.slice[n], 'endlineno', startline) return startline, endline def lexpos(self, n): return getattr(self.slice[n], 'lexpos', 0) def lexspan(self, n): startpos = getattr(self.slice[n], 'lexpos', 0) endpos = getattr(self.slice[n], 'endlexpos', startpos) return startpos, endpos def error(self): raise SyntaxError # ----------------------------------------------------------------------------- # == LRParser == # # The LR Parsing engine. # ----------------------------------------------------------------------------- class LRParser: def __init__(self, lrtab, errorf): self.productions = lrtab.lr_productions self.action = lrtab.lr_action self.goto = lrtab.lr_goto self.errorfunc = errorf self.set_defaulted_states() self.errorok = True def errok(self): self.errorok = True def restart(self): del self.statestack[:] del self.symstack[:] sym = YaccSymbol() sym.type = '$end' self.symstack.append(sym) self.statestack.append(0) # Defaulted state support. # This method identifies parser states where there is only one possible reduction action. # For such states, the parser can make a choose to make a rule reduction without consuming # the next look-ahead token. This delayed invocation of the tokenizer can be useful in # certain kinds of advanced parsing situations where the lexer and parser interact with # each other or change states (i.e., manipulation of scope, lexer states, etc.). # # See: http://www.gnu.org/software/bison/manual/html_node/Default-Reductions.html#Default-Reductions def set_defaulted_states(self): self.defaulted_states = {} for state, actions in self.action.items(): rules = list(actions.values()) if len(rules) == 1 and rules[0] < 0: self.defaulted_states[state] = rules[0] def disable_defaulted_states(self): self.defaulted_states = {} def parse(self, input=None, lexer=None, debug=False, tracking=False, tokenfunc=None): if debug or yaccdevel: if isinstance(debug, int): debug = PlyLogger(sys.stderr) return self.parsedebug(input, lexer, debug, tracking, tokenfunc) elif tracking: return self.parseopt(input, lexer, debug, tracking, tokenfunc) else: return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc) # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! # parsedebug(). # # This is the debugging enabled version of parse(). All changes made to the # parsing engine should be made here. Optimized versions of this function # are automatically created by the ply/ygen.py script. This script cuts out # sections enclosed in markers such as this: # # #--! DEBUG # statements # #--! DEBUG # # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! def parsedebug(self, input=None, lexer=None, debug=False, tracking=False, tokenfunc=None): #--! parsedebug-start lookahead = None # Current lookahead symbol lookaheadstack = [] # Stack of lookahead symbols actions = self.action # Local reference to action table (to avoid lookup on self.) goto = self.goto # Local reference to goto table (to avoid lookup on self.) prod = self.productions # Local reference to production list (to avoid lookup on self.) defaulted_states = self.defaulted_states # Local reference to defaulted states pslice = YaccProduction(None) # Production object passed to grammar rules errorcount = 0 # Used during error recovery #--! DEBUG debug.info('PLY: PARSE DEBUG START') #--! DEBUG # If no lexer was given, we will try to use the lex module if not lexer: from . import lex lexer = lex.lexer # Set up the lexer and parser objects on pslice pslice.lexer = lexer pslice.parser = self # If input was supplied, pass to lexer if input is not None: lexer.input(input) if tokenfunc is None: # Tokenize function get_token = lexer.token else: get_token = tokenfunc # Set the parser() token method (sometimes used in error recovery) self.token = get_token # Set up the state and symbol stacks statestack = [] # Stack of parsing states self.statestack = statestack symstack = [] # Stack of grammar symbols self.symstack = symstack pslice.stack = symstack # Put in the production errtoken = None # Err token # The start state is assumed to be (0,$end) statestack.append(0) sym = YaccSymbol() sym.type = '$end' symstack.append(sym) state = 0 while True: # Get the next symbol on the input. If a lookahead symbol # is already set, we just use that. Otherwise, we'll pull # the next token off of the lookaheadstack or from the lexer #--! DEBUG debug.debug('') debug.debug('State : %s', state) #--! DEBUG if state not in defaulted_states: if not lookahead: if not lookaheadstack: lookahead = get_token() # Get the next token else: lookahead = lookaheadstack.pop() if not lookahead: lookahead = YaccSymbol() lookahead.type = '$end' # Check the action table ltype = lookahead.type t = actions[state].get(ltype) else: t = defaulted_states[state] #--! DEBUG debug.debug('Defaulted state %s: Reduce using %d', state, -t) #--! DEBUG #--! DEBUG debug.debug('Stack : %s', ('%s . %s' % (' '.join([xx.type for xx in symstack][1:]), str(lookahead))).lstrip()) #--! DEBUG if t is not None: if t > 0: # shift a symbol on the stack statestack.append(t) state = t #--! DEBUG debug.debug('Action : Shift and goto state %s', t) #--! DEBUG symstack.append(lookahead) lookahead = None # Decrease error count on successful shift if errorcount: errorcount -= 1 continue if t < 0: # reduce a symbol on the stack, emit a production p = prod[-t] pname = p.name plen = p.len # Get production function sym = YaccSymbol() sym.type = pname # Production name sym.value = None #--! DEBUG if plen: debug.info('Action : Reduce rule [%s] with %s and goto state %d', p.str, '['+','.join([format_stack_entry(_v.value) for _v in symstack[-plen:]])+']', goto[statestack[-1-plen]][pname]) else: debug.info('Action : Reduce rule [%s] with %s and goto state %d', p.str, [], goto[statestack[-1]][pname]) #--! DEBUG if plen: targ = symstack[-plen-1:] targ[0] = sym #--! TRACKING if tracking: t1 = targ[1] sym.lineno = t1.lineno sym.lexpos = t1.lexpos t1 = targ[-1] sym.endlineno = getattr(t1, 'endlineno', t1.lineno) sym.endlexpos = getattr(t1, 'endlexpos', t1.lexpos) #--! TRACKING # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! # The code enclosed in this section is duplicated # below as a performance optimization. Make sure # changes get made in both locations. pslice.slice = targ try: # Call the grammar rule with our special slice object del symstack[-plen:] del statestack[-plen:] p.callable(pslice) #--! DEBUG debug.info('Result : %s', format_result(pslice[0])) #--! DEBUG symstack.append(sym) state = goto[statestack[-1]][pname] statestack.append(state) except SyntaxError: # If an error was set. Enter error recovery state lookaheadstack.append(lookahead) symstack.pop() statestack.pop() state = statestack[-1] sym.type = 'error' lookahead = sym errorcount = error_count self.errorok = False continue # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! else: #--! TRACKING if tracking: sym.lineno = lexer.lineno sym.lexpos = lexer.lexpos #--! TRACKING targ = [sym] # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! # The code enclosed in this section is duplicated # above as a performance optimization. Make sure # changes get made in both locations. pslice.slice = targ try: # Call the grammar rule with our special slice object p.callable(pslice) #--! DEBUG debug.info('Result : %s', format_result(pslice[0])) #--! DEBUG symstack.append(sym) state = goto[statestack[-1]][pname] statestack.append(state) except SyntaxError: # If an error was set. Enter error recovery state lookaheadstack.append(lookahead) symstack.pop() statestack.pop() state = statestack[-1] sym.type = 'error' lookahead = sym errorcount = error_count self.errorok = False continue # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! if t == 0: n = symstack[-1] result = getattr(n, 'value', None) #--! DEBUG debug.info('Done : Returning %s', format_result(result)) debug.info('PLY: PARSE DEBUG END') #--! DEBUG return result if t is None: #--! DEBUG debug.error('Error : %s', ('%s . %s' % (' '.join([xx.type for xx in symstack][1:]), str(lookahead))).lstrip()) #--! DEBUG # We have some kind of parsing error here. To handle # this, we are going to push the current token onto # the tokenstack and replace it with an 'error' token. # If there are any synchronization rules, they may # catch it. # # In addition to pushing the error token, we call call # the user defined p_error() function if this is the # first syntax error. This function is only called if # errorcount == 0. if errorcount == 0 or self.errorok: errorcount = error_count self.errorok = False errtoken = lookahead if errtoken.type == '$end': errtoken = None # End of file! if self.errorfunc: if errtoken and not hasattr(errtoken, 'lexer'): errtoken.lexer = lexer tok = call_errorfunc(self.errorfunc, errtoken, self) if self.errorok: # User must have done some kind of panic # mode recovery on their own. The # returned token is the next lookahead lookahead = tok errtoken = None continue else: if errtoken: if hasattr(errtoken, 'lineno'): lineno = lookahead.lineno else: lineno = 0 if lineno: sys.stderr.write('yacc: Syntax error at line %d, token=%s\n' % (lineno, errtoken.type)) else: sys.stderr.write('yacc: Syntax error, token=%s' % errtoken.type) else: sys.stderr.write('yacc: Parse error in input. EOF\n') return else: errorcount = error_count # case 1: the statestack only has 1 entry on it. If we're in this state, the # entire parse has been rolled back and we're completely hosed. The token is # discarded and we just keep going. if len(statestack) <= 1 and lookahead.type != '$end': lookahead = None errtoken = None state = 0 # Nuke the pushback stack del lookaheadstack[:] continue # case 2: the statestack has a couple of entries on it, but we're # at the end of the file. nuke the top entry and generate an error token # Start nuking entries on the stack if lookahead.type == '$end': # Whoa. We're really hosed here. Bail out return if lookahead.type != 'error': sym = symstack[-1] if sym.type == 'error': # Hmmm. Error is on top of stack, we'll just nuke input # symbol and continue #--! TRACKING if tracking: sym.endlineno = getattr(lookahead, 'lineno', sym.lineno) sym.endlexpos = getattr(lookahead, 'lexpos', sym.lexpos) #--! TRACKING lookahead = None continue # Create the error symbol for the first time and make it the new lookahead symbol t = YaccSymbol() t.type = 'error' if hasattr(lookahead, 'lineno'): t.lineno = t.endlineno = lookahead.lineno if hasattr(lookahead, 'lexpos'): t.lexpos = t.endlexpos = lookahead.lexpos t.value = lookahead lookaheadstack.append(lookahead) lookahead = t else: sym = symstack.pop() #--! TRACKING if tracking: lookahead.lineno = sym.lineno lookahead.lexpos = sym.lexpos #--! TRACKING statestack.pop() state = statestack[-1] continue # Call an error function here raise RuntimeError('yacc: internal parser error!!!\n') #--! parsedebug-end # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! # parseopt(). # # Optimized version of parse() method. DO NOT EDIT THIS CODE DIRECTLY! # This code is automatically generated by the ply/ygen.py script. Make # changes to the parsedebug() method instead. # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! def parseopt(self, input=None, lexer=None, debug=False, tracking=False, tokenfunc=None): #--! parseopt-start lookahead = None # Current lookahead symbol lookaheadstack = [] # Stack of lookahead symbols actions = self.action # Local reference to action table (to avoid lookup on self.) goto = self.goto # Local reference to goto table (to avoid lookup on self.) prod = self.productions # Local reference to production list (to avoid lookup on self.) defaulted_states = self.defaulted_states # Local reference to defaulted states pslice = YaccProduction(None) # Production object passed to grammar rules errorcount = 0 # Used during error recovery # If no lexer was given, we will try to use the lex module if not lexer: from . import lex lexer = lex.lexer # Set up the lexer and parser objects on pslice pslice.lexer = lexer pslice.parser = self # If input was supplied, pass to lexer if input is not None: lexer.input(input) if tokenfunc is None: # Tokenize function get_token = lexer.token else: get_token = tokenfunc # Set the parser() token method (sometimes used in error recovery) self.token = get_token # Set up the state and symbol stacks statestack = [] # Stack of parsing states self.statestack = statestack symstack = [] # Stack of grammar symbols self.symstack = symstack pslice.stack = symstack # Put in the production errtoken = None # Err token # The start state is assumed to be (0,$end) statestack.append(0) sym = YaccSymbol() sym.type = '$end' symstack.append(sym) state = 0 while True: # Get the next symbol on the input. If a lookahead symbol # is already set, we just use that. Otherwise, we'll pull # the next token off of the lookaheadstack or from the lexer if state not in defaulted_states: if not lookahead: if not lookaheadstack: lookahead = get_token() # Get the next token else: lookahead = lookaheadstack.pop() if not lookahead: lookahead = YaccSymbol() lookahead.type = '$end' # Check the action table ltype = lookahead.type t = actions[state].get(ltype) else: t = defaulted_states[state] if t is not None: if t > 0: # shift a symbol on the stack statestack.append(t) state = t symstack.append(lookahead) lookahead = None # Decrease error count on successful shift if errorcount: errorcount -= 1 continue if t < 0: # reduce a symbol on the stack, emit a production p = prod[-t] pname = p.name plen = p.len # Get production function sym = YaccSymbol() sym.type = pname # Production name sym.value = None if plen: targ = symstack[-plen-1:] targ[0] = sym #--! TRACKING if tracking: t1 = targ[1] sym.lineno = t1.lineno sym.lexpos = t1.lexpos t1 = targ[-1] sym.endlineno = getattr(t1, 'endlineno', t1.lineno) sym.endlexpos = getattr(t1, 'endlexpos', t1.lexpos) #--! TRACKING # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! # The code enclosed in this section is duplicated # below as a performance optimization. Make sure # changes get made in both locations. pslice.slice = targ try: # Call the grammar rule with our special slice object del symstack[-plen:] del statestack[-plen:] p.callable(pslice) symstack.append(sym) state = goto[statestack[-1]][pname] statestack.append(state) except SyntaxError: # If an error was set. Enter error recovery state lookaheadstack.append(lookahead) symstack.pop() statestack.pop() state = statestack[-1] sym.type = 'error' lookahead = sym errorcount = error_count self.errorok = False continue # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! else: #--! TRACKING if tracking: sym.lineno = lexer.lineno sym.lexpos = lexer.lexpos #--! TRACKING targ = [sym] # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! # The code enclosed in this section is duplicated # above as a performance optimization. Make sure # changes get made in both locations. pslice.slice = targ try: # Call the grammar rule with our special slice object p.callable(pslice) symstack.append(sym) state = goto[statestack[-1]][pname] statestack.append(state) except SyntaxError: # If an error was set. Enter error recovery state lookaheadstack.append(lookahead) symstack.pop() statestack.pop() state = statestack[-1] sym.type = 'error' lookahead = sym errorcount = error_count self.errorok = False continue # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! if t == 0: n = symstack[-1] result = getattr(n, 'value', None) return result if t is None: # We have some kind of parsing error here. To handle # this, we are going to push the current token onto # the tokenstack and replace it with an 'error' token. # If there are any synchronization rules, they may # catch it. # # In addition to pushing the error token, we call call # the user defined p_error() function if this is the # first syntax error. This function is only called if # errorcount == 0. if errorcount == 0 or self.errorok: errorcount = error_count self.errorok = False errtoken = lookahead if errtoken.type == '$end': errtoken = None # End of file! if self.errorfunc: if errtoken and not hasattr(errtoken, 'lexer'): errtoken.lexer = lexer tok = call_errorfunc(self.errorfunc, errtoken, self) if self.errorok: # User must have done some kind of panic # mode recovery on their own. The # returned token is the next lookahead lookahead = tok errtoken = None continue else: if errtoken: if hasattr(errtoken, 'lineno'): lineno = lookahead.lineno else: lineno = 0 if lineno: sys.stderr.write('yacc: Syntax error at line %d, token=%s\n' % (lineno, errtoken.type)) else: sys.stderr.write('yacc: Syntax error, token=%s' % errtoken.type) else: sys.stderr.write('yacc: Parse error in input. EOF\n') return else: errorcount = error_count # case 1: the statestack only has 1 entry on it. If we're in this state, the # entire parse has been rolled back and we're completely hosed. The token is # discarded and we just keep going. if len(statestack) <= 1 and lookahead.type != '$end': lookahead = None errtoken = None state = 0 # Nuke the pushback stack del lookaheadstack[:] continue # case 2: the statestack has a couple of entries on it, but we're # at the end of the file. nuke the top entry and generate an error token # Start nuking entries on the stack if lookahead.type == '$end': # Whoa. We're really hosed here. Bail out return if lookahead.type != 'error': sym = symstack[-1] if sym.type == 'error': # Hmmm. Error is on top of stack, we'll just nuke input # symbol and continue #--! TRACKING if tracking: sym.endlineno = getattr(lookahead, 'lineno', sym.lineno) sym.endlexpos = getattr(lookahead, 'lexpos', sym.lexpos) #--! TRACKING lookahead = None continue # Create the error symbol for the first time and make it the new lookahead symbol t = YaccSymbol() t.type = 'error' if hasattr(lookahead, 'lineno'): t.lineno = t.endlineno = lookahead.lineno if hasattr(lookahead, 'lexpos'): t.lexpos = t.endlexpos = lookahead.lexpos t.value = lookahead lookaheadstack.append(lookahead) lookahead = t else: sym = symstack.pop() #--! TRACKING if tracking: lookahead.lineno = sym.lineno lookahead.lexpos = sym.lexpos #--! TRACKING statestack.pop() state = statestack[-1] continue # Call an error function here raise RuntimeError('yacc: internal parser error!!!\n') #--! parseopt-end # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! # parseopt_notrack(). # # Optimized version of parseopt() with line number tracking removed. # DO NOT EDIT THIS CODE DIRECTLY. This code is automatically generated # by the ply/ygen.py script. Make changes to the parsedebug() method instead. # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! def parseopt_notrack(self, input=None, lexer=None, debug=False, tracking=False, tokenfunc=None): #--! parseopt-notrack-start lookahead = None # Current lookahead symbol lookaheadstack = [] # Stack of lookahead symbols actions = self.action # Local reference to action table (to avoid lookup on self.) goto = self.goto # Local reference to goto table (to avoid lookup on self.) prod = self.productions # Local reference to production list (to avoid lookup on self.) defaulted_states = self.defaulted_states # Local reference to defaulted states pslice = YaccProduction(None) # Production object passed to grammar rules errorcount = 0 # Used during error recovery # If no lexer was given, we will try to use the lex module if not lexer: from . import lex lexer = lex.lexer # Set up the lexer and parser objects on pslice pslice.lexer = lexer pslice.parser = self # If input was supplied, pass to lexer if input is not None: lexer.input(input) if tokenfunc is None: # Tokenize function get_token = lexer.token else: get_token = tokenfunc # Set the parser() token method (sometimes used in error recovery) self.token = get_token # Set up the state and symbol stacks statestack = [] # Stack of parsing states self.statestack = statestack symstack = [] # Stack of grammar symbols self.symstack = symstack pslice.stack = symstack # Put in the production errtoken = None # Err token # The start state is assumed to be (0,$end) statestack.append(0) sym = YaccSymbol() sym.type = '$end' symstack.append(sym) state = 0 while True: # Get the next symbol on the input. If a lookahead symbol # is already set, we just use that. Otherwise, we'll pull # the next token off of the lookaheadstack or from the lexer if state not in defaulted_states: if not lookahead: if not lookaheadstack: lookahead = get_token() # Get the next token else: lookahead = lookaheadstack.pop() if not lookahead: lookahead = YaccSymbol() lookahead.type = '$end' # Check the action table ltype = lookahead.type t = actions[state].get(ltype) else: t = defaulted_states[state] if t is not None: if t > 0: # shift a symbol on the stack statestack.append(t) state = t symstack.append(lookahead) lookahead = None # Decrease error count on successful shift if errorcount: errorcount -= 1 continue if t < 0: # reduce a symbol on the stack, emit a production p = prod[-t] pname = p.name plen = p.len # Get production function sym = YaccSymbol() sym.type = pname # Production name sym.value = None if plen: targ = symstack[-plen-1:] targ[0] = sym # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! # The code enclosed in this section is duplicated # below as a performance optimization. Make sure # changes get made in both locations. pslice.slice = targ try: # Call the grammar rule with our special slice object del symstack[-plen:] del statestack[-plen:] p.callable(pslice) symstack.append(sym) state = goto[statestack[-1]][pname] statestack.append(state) except SyntaxError: # If an error was set. Enter error recovery state lookaheadstack.append(lookahead) symstack.pop() statestack.pop() state = statestack[-1] sym.type = 'error' lookahead = sym errorcount = error_count self.errorok = False continue # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! else: targ = [sym] # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! # The code enclosed in this section is duplicated # above as a performance optimization. Make sure # changes get made in both locations. pslice.slice = targ try: # Call the grammar rule with our special slice object p.callable(pslice) symstack.append(sym) state = goto[statestack[-1]][pname] statestack.append(state) except SyntaxError: # If an error was set. Enter error recovery state lookaheadstack.append(lookahead) symstack.pop() statestack.pop() state = statestack[-1] sym.type = 'error' lookahead = sym errorcount = error_count self.errorok = False continue # !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! if t == 0: n = symstack[-1] result = getattr(n, 'value', None) return result if t is None: # We have some kind of parsing error here. To handle # this, we are going to push the current token onto # the tokenstack and replace it with an 'error' token. # If there are any synchronization rules, they may # catch it. # # In addition to pushing the error token, we call call # the user defined p_error() function if this is the # first syntax error. This function is only called if # errorcount == 0. if errorcount == 0 or self.errorok: errorcount = error_count self.errorok = False errtoken = lookahead if errtoken.type == '$end': errtoken = None # End of file! if self.errorfunc: if errtoken and not hasattr(errtoken, 'lexer'): errtoken.lexer = lexer tok = call_errorfunc(self.errorfunc, errtoken, self) if self.errorok: # User must have done some kind of panic # mode recovery on their own. The # returned token is the next lookahead lookahead = tok errtoken = None continue else: if errtoken: if hasattr(errtoken, 'lineno'): lineno = lookahead.lineno else: lineno = 0 if lineno: sys.stderr.write('yacc: Syntax error at line %d, token=%s\n' % (lineno, errtoken.type)) else: sys.stderr.write('yacc: Syntax error, token=%s' % errtoken.type) else: sys.stderr.write('yacc: Parse error in input. EOF\n') return else: errorcount = error_count # case 1: the statestack only has 1 entry on it. If we're in this state, the # entire parse has been rolled back and we're completely hosed. The token is # discarded and we just keep going. if len(statestack) <= 1 and lookahead.type != '$end': lookahead = None errtoken = None state = 0 # Nuke the pushback stack del lookaheadstack[:] continue # case 2: the statestack has a couple of entries on it, but we're # at the end of the file. nuke the top entry and generate an error token # Start nuking entries on the stack if lookahead.type == '$end': # Whoa. We're really hosed here. Bail out return if lookahead.type != 'error': sym = symstack[-1] if sym.type == 'error': # Hmmm. Error is on top of stack, we'll just nuke input # symbol and continue lookahead = None continue # Create the error symbol for the first time and make it the new lookahead symbol t = YaccSymbol() t.type = 'error' if hasattr(lookahead, 'lineno'): t.lineno = t.endlineno = lookahead.lineno if hasattr(lookahead, 'lexpos'): t.lexpos = t.endlexpos = lookahead.lexpos t.value = lookahead lookaheadstack.append(lookahead) lookahead = t else: sym = symstack.pop() statestack.pop() state = statestack[-1] continue # Call an error function here raise RuntimeError('yacc: internal parser error!!!\n') #--! parseopt-notrack-end # ----------------------------------------------------------------------------- # === Grammar Representation === # # The following functions, classes, and variables are used to represent and # manipulate the rules that make up a grammar. # ----------------------------------------------------------------------------- # regex matching identifiers _is_identifier = re.compile(r'^[a-zA-Z0-9_-]+$') # ----------------------------------------------------------------------------- # class Production: # # This class stores the raw information about a single production or grammar rule. # A grammar rule refers to a specification such as this: # # expr : expr PLUS term # # Here are the basic attributes defined on all productions # # name - Name of the production. For example 'expr' # prod - A list of symbols on the right side ['expr','PLUS','term'] # prec - Production precedence level # number - Production number. # func - Function that executes on reduce # file - File where production function is defined # lineno - Line number where production function is defined # # The following attributes are defined or optional. # # len - Length of the production (number of symbols on right hand side) # usyms - Set of unique symbols found in the production # ----------------------------------------------------------------------------- class Production(object): reduced = 0 def __init__(self, number, name, prod, precedence=('right', 0), func=None, file='', line=0): self.name = name self.prod = tuple(prod) self.number = number self.func = func self.callable = None self.file = file self.line = line self.prec = precedence # Internal settings used during table construction self.len = len(self.prod) # Length of the production # Create a list of unique production symbols used in the production self.usyms = [] for s in self.prod: if s not in self.usyms: self.usyms.append(s) # List of all LR items for the production self.lr_items = [] self.lr_next = None # Create a string representation if self.prod: self.str = '%s -> %s' % (self.name, ' '.join(self.prod)) else: self.str = '%s -> <empty>' % self.name def __str__(self): return self.str def __repr__(self): return 'Production(' + str(self) + ')' def __len__(self): return len(self.prod) def __nonzero__(self): return 1 def __getitem__(self, index): return self.prod[index] # Return the nth lr_item from the production (or None if at the end) def lr_item(self, n): if n > len(self.prod): return None p = LRItem(self, n) # Precompute the list of productions immediately following. try: p.lr_after = Prodnames[p.prod[n+1]] except (IndexError, KeyError): p.lr_after = [] try: p.lr_before = p.prod[n-1] except IndexError: p.lr_before = None return p # Bind the production function name to a callable def bind(self, pdict): if self.func: self.callable = pdict[self.func] # This class serves as a minimal standin for Production objects when # reading table data from files. It only contains information # actually used by the LR parsing engine, plus some additional # debugging information. class MiniProduction(object): def __init__(self, str, name, len, func, file, line): self.name = name self.len = len self.func = func self.callable = None self.file = file self.line = line self.str = str def __str__(self): return self.str def __repr__(self): return 'MiniProduction(%s)' % self.str # Bind the production function name to a callable def bind(self, pdict): if self.func: self.callable = pdict[self.func] # ----------------------------------------------------------------------------- # class LRItem # # This class represents a specific stage of parsing a production rule. For # example: # # expr : expr . PLUS term # # In the above, the "." represents the current location of the parse. Here # basic attributes: # # name - Name of the production. For example 'expr' # prod - A list of symbols on the right side ['expr','.', 'PLUS','term'] # number - Production number. # # lr_next Next LR item. Example, if we are ' expr -> expr . PLUS term' # then lr_next refers to 'expr -> expr PLUS . term' # lr_index - LR item index (location of the ".") in the prod list. # lookaheads - LALR lookahead symbols for this item # len - Length of the production (number of symbols on right hand side) # lr_after - List of all productions that immediately follow # lr_before - Grammar symbol immediately before # ----------------------------------------------------------------------------- class LRItem(object): def __init__(self, p, n): self.name = p.name self.prod = list(p.prod) self.number = p.number self.lr_index = n self.lookaheads = {} self.prod.insert(n, '.') self.prod = tuple(self.prod) self.len = len(self.prod) self.usyms = p.usyms def __str__(self): if self.prod: s = '%s -> %s' % (self.name, ' '.join(self.prod)) else: s = '%s -> <empty>' % self.name return s def __repr__(self): return 'LRItem(' + str(self) + ')' # ----------------------------------------------------------------------------- # rightmost_terminal() # # Return the rightmost terminal from a list of symbols. Used in add_production() # ----------------------------------------------------------------------------- def rightmost_terminal(symbols, terminals): i = len(symbols) - 1 while i >= 0: if symbols[i] in terminals: return symbols[i] i -= 1 return None # ----------------------------------------------------------------------------- # === GRAMMAR CLASS === # # The following class represents the contents of the specified grammar along # with various computed properties such as first sets, follow sets, LR items, etc. # This data is used for critical parts of the table generation process later. # ----------------------------------------------------------------------------- class GrammarError(YaccError): pass class Grammar(object): def __init__(self, terminals): self.Productions = [None] # A list of all of the productions. The first # entry is always reserved for the purpose of # building an augmented grammar self.Prodnames = {} # A dictionary mapping the names of nonterminals to a list of all # productions of that nonterminal. self.Prodmap = {} # A dictionary that is only used to detect duplicate # productions. self.Terminals = {} # A dictionary mapping the names of terminal symbols to a # list of the rules where they are used. for term in terminals: self.Terminals[term] = [] self.Terminals['error'] = [] self.Nonterminals = {} # A dictionary mapping names of nonterminals to a list # of rule numbers where they are used. self.First = {} # A dictionary of precomputed FIRST(x) symbols self.Follow = {} # A dictionary of precomputed FOLLOW(x) symbols self.Precedence = {} # Precedence rules for each terminal. Contains tuples of the # form ('right',level) or ('nonassoc', level) or ('left',level) self.UsedPrecedence = set() # Precedence rules that were actually used by the grammer. # This is only used to provide error checking and to generate # a warning about unused precedence rules. self.Start = None # Starting symbol for the grammar def __len__(self): return len(self.Productions) def __getitem__(self, index): return self.Productions[index] # ----------------------------------------------------------------------------- # set_precedence() # # Sets the precedence for a given terminal. assoc is the associativity such as # 'left','right', or 'nonassoc'. level is a numeric level. # # ----------------------------------------------------------------------------- def set_precedence(self, term, assoc, level): assert self.Productions == [None], 'Must call set_precedence() before add_production()' if term in self.Precedence: raise GrammarError('Precedence already specified for terminal %r' % term) if assoc not in ['left', 'right', 'nonassoc']: raise GrammarError("Associativity must be one of 'left','right', or 'nonassoc'") self.Precedence[term] = (assoc, level) # ----------------------------------------------------------------------------- # add_production() # # Given an action function, this function assembles a production rule and # computes its precedence level. # # The production rule is supplied as a list of symbols. For example, # a rule such as 'expr : expr PLUS term' has a production name of 'expr' and # symbols ['expr','PLUS','term']. # # Precedence is determined by the precedence of the right-most non-terminal # or the precedence of a terminal specified by %prec. # # A variety of error checks are performed to make sure production symbols # are valid and that %prec is used correctly. # ----------------------------------------------------------------------------- def add_production(self, prodname, syms, func=None, file='', line=0): if prodname in self.Terminals: raise GrammarError('%s:%d: Illegal rule name %r. Already defined as a token' % (file, line, prodname)) if prodname == 'error': raise GrammarError('%s:%d: Illegal rule name %r. error is a reserved word' % (file, line, prodname)) if not _is_identifier.match(prodname): raise GrammarError('%s:%d: Illegal rule name %r' % (file, line, prodname)) # Look for literal tokens for n, s in enumerate(syms): if s[0] in "'\"": try: c = eval(s) if (len(c) > 1): raise GrammarError('%s:%d: Literal token %s in rule %r may only be a single character' % (file, line, s, prodname)) if c not in self.Terminals: self.Terminals[c] = [] syms[n] = c continue except SyntaxError: pass if not _is_identifier.match(s) and s != '%prec': raise GrammarError('%s:%d: Illegal name %r in rule %r' % (file, line, s, prodname)) # Determine the precedence level if '%prec' in syms: if syms[-1] == '%prec': raise GrammarError('%s:%d: Syntax error. Nothing follows %%prec' % (file, line)) if syms[-2] != '%prec': raise GrammarError('%s:%d: Syntax error. %%prec can only appear at the end of a grammar rule' % (file, line)) precname = syms[-1] prodprec = self.Precedence.get(precname) if not prodprec: raise GrammarError('%s:%d: Nothing known about the precedence of %r' % (file, line, precname)) else: self.UsedPrecedence.add(precname) del syms[-2:] # Drop %prec from the rule else: # If no %prec, precedence is determined by the rightmost terminal symbol precname = rightmost_terminal(syms, self.Terminals) prodprec = self.Precedence.get(precname, ('right', 0)) # See if the rule is already in the rulemap map = '%s -> %s' % (prodname, syms) if map in self.Prodmap: m = self.Prodmap[map] raise GrammarError('%s:%d: Duplicate rule %s. ' % (file, line, m) + 'Previous definition at %s:%d' % (m.file, m.line)) # From this point on, everything is valid. Create a new Production instance pnumber = len(self.Productions) if prodname not in self.Nonterminals: self.Nonterminals[prodname] = [] # Add the production number to Terminals and Nonterminals for t in syms: if t in self.Terminals: self.Terminals[t].append(pnumber) else: if t not in self.Nonterminals: self.Nonterminals[t] = [] self.Nonterminals[t].append(pnumber) # Create a production and add it to the list of productions p = Production(pnumber, prodname, syms, prodprec, func, file, line) self.Productions.append(p) self.Prodmap[map] = p # Add to the global productions list try: self.Prodnames[prodname].append(p) except KeyError: self.Prodnames[prodname] = [p] # ----------------------------------------------------------------------------- # set_start() # # Sets the starting symbol and creates the augmented grammar. Production # rule 0 is S' -> start where start is the start symbol. # ----------------------------------------------------------------------------- def set_start(self, start=None): if not start: start = self.Productions[1].name if start not in self.Nonterminals: raise GrammarError('start symbol %s undefined' % start) self.Productions[0] = Production(0, "S'", [start]) self.Nonterminals[start].append(0) self.Start = start # ----------------------------------------------------------------------------- # find_unreachable() # # Find all of the nonterminal symbols that can't be reached from the starting # symbol. Returns a list of nonterminals that can't be reached. # ----------------------------------------------------------------------------- def find_unreachable(self): # Mark all symbols that are reachable from a symbol s def mark_reachable_from(s): if s in reachable: return reachable.add(s) for p in self.Prodnames.get(s, []): for r in p.prod: mark_reachable_from(r) reachable = set() mark_reachable_from(self.Productions[0].prod[0]) return [s for s in self.Nonterminals if s not in reachable] # ----------------------------------------------------------------------------- # infinite_cycles() # # This function looks at the various parsing rules and tries to detect # infinite recursion cycles (grammar rules where there is no possible way # to derive a string of only terminals). # ----------------------------------------------------------------------------- def infinite_cycles(self): terminates = {} # Terminals: for t in self.Terminals: terminates[t] = True terminates['$end'] = True # Nonterminals: # Initialize to false: for n in self.Nonterminals: terminates[n] = False # Then propagate termination until no change: while True: some_change = False for (n, pl) in self.Prodnames.items(): # Nonterminal n terminates iff any of its productions terminates. for p in pl: # Production p terminates iff all of its rhs symbols terminate. for s in p.prod: if not terminates[s]: # The symbol s does not terminate, # so production p does not terminate. p_terminates = False break else: # didn't break from the loop, # so every symbol s terminates # so production p terminates. p_terminates = True if p_terminates: # symbol n terminates! if not terminates[n]: terminates[n] = True some_change = True # Don't need to consider any more productions for this n. break if not some_change: break infinite = [] for (s, term) in terminates.items(): if not term: if s not in self.Prodnames and s not in self.Terminals and s != 'error': # s is used-but-not-defined, and we've already warned of that, # so it would be overkill to say that it's also non-terminating. pass else: infinite.append(s) return infinite # ----------------------------------------------------------------------------- # undefined_symbols() # # Find all symbols that were used the grammar, but not defined as tokens or # grammar rules. Returns a list of tuples (sym, prod) where sym in the symbol # and prod is the production where the symbol was used. # ----------------------------------------------------------------------------- def undefined_symbols(self): result = [] for p in self.Productions: if not p: continue for s in p.prod: if s not in self.Prodnames and s not in self.Terminals and s != 'error': result.append((s, p)) return result # ----------------------------------------------------------------------------- # unused_terminals() # # Find all terminals that were defined, but not used by the grammar. Returns # a list of all symbols. # ----------------------------------------------------------------------------- def unused_terminals(self): unused_tok = [] for s, v in self.Terminals.items(): if s != 'error' and not v: unused_tok.append(s) return unused_tok # ------------------------------------------------------------------------------ # unused_rules() # # Find all grammar rules that were defined, but not used (maybe not reachable) # Returns a list of productions. # ------------------------------------------------------------------------------ def unused_rules(self): unused_prod = [] for s, v in self.Nonterminals.items(): if not v: p = self.Prodnames[s][0] unused_prod.append(p) return unused_prod # ----------------------------------------------------------------------------- # unused_precedence() # # Returns a list of tuples (term,precedence) corresponding to precedence # rules that were never used by the grammar. term is the name of the terminal # on which precedence was applied and precedence is a string such as 'left' or # 'right' corresponding to the type of precedence. # ----------------------------------------------------------------------------- def unused_precedence(self): unused = [] for termname in self.Precedence: if not (termname in self.Terminals or termname in self.UsedPrecedence): unused.append((termname, self.Precedence[termname][0])) return unused # ------------------------------------------------------------------------- # _first() # # Compute the value of FIRST1(beta) where beta is a tuple of symbols. # # During execution of compute_first1, the result may be incomplete. # Afterward (e.g., when called from compute_follow()), it will be complete. # ------------------------------------------------------------------------- def _first(self, beta): # We are computing First(x1,x2,x3,...,xn) result = [] for x in beta: x_produces_empty = False # Add all the non-<empty> symbols of First[x] to the result. for f in self.First[x]: if f == '<empty>': x_produces_empty = True else: if f not in result: result.append(f) if x_produces_empty: # We have to consider the next x in beta, # i.e. stay in the loop. pass else: # We don't have to consider any further symbols in beta. break else: # There was no 'break' from the loop, # so x_produces_empty was true for all x in beta, # so beta produces empty as well. result.append('<empty>') return result # ------------------------------------------------------------------------- # compute_first() # # Compute the value of FIRST1(X) for all symbols # ------------------------------------------------------------------------- def compute_first(self): if self.First: return self.First # Terminals: for t in self.Terminals: self.First[t] = [t] self.First['$end'] = ['$end'] # Nonterminals: # Initialize to the empty set: for n in self.Nonterminals: self.First[n] = [] # Then propagate symbols until no change: while True: some_change = False for n in self.Nonterminals: for p in self.Prodnames[n]: for f in self._first(p.prod): if f not in self.First[n]: self.First[n].append(f) some_change = True if not some_change: break return self.First # --------------------------------------------------------------------- # compute_follow() # # Computes all of the follow sets for every non-terminal symbol. The # follow set is the set of all symbols that might follow a given # non-terminal. See the Dragon book, 2nd Ed. p. 189. # --------------------------------------------------------------------- def compute_follow(self, start=None): # If already computed, return the result if self.Follow: return self.Follow # If first sets not computed yet, do that first. if not self.First: self.compute_first() # Add '$end' to the follow list of the start symbol for k in self.Nonterminals: self.Follow[k] = [] if not start: start = self.Productions[1].name self.Follow[start] = ['$end'] while True: didadd = False for p in self.Productions[1:]: # Here is the production set for i, B in enumerate(p.prod): if B in self.Nonterminals: # Okay. We got a non-terminal in a production fst = self._first(p.prod[i+1:]) hasempty = False for f in fst: if f != '<empty>' and f not in self.Follow[B]: self.Follow[B].append(f) didadd = True if f == '<empty>': hasempty = True if hasempty or i == (len(p.prod)-1): # Add elements of follow(a) to follow(b) for f in self.Follow[p.name]: if f not in self.Follow[B]: self.Follow[B].append(f) didadd = True if not didadd: break return self.Follow # ----------------------------------------------------------------------------- # build_lritems() # # This function walks the list of productions and builds a complete set of the # LR items. The LR items are stored in two ways: First, they are uniquely # numbered and placed in the list _lritems. Second, a linked list of LR items # is built for each production. For example: # # E -> E PLUS E # # Creates the list # # [E -> . E PLUS E, E -> E . PLUS E, E -> E PLUS . E, E -> E PLUS E . ] # ----------------------------------------------------------------------------- def build_lritems(self): for p in self.Productions: lastlri = p i = 0 lr_items = [] while True: if i > len(p): lri = None else: lri = LRItem(p, i) # Precompute the list of productions immediately following try: lri.lr_after = self.Prodnames[lri.prod[i+1]] except (IndexError, KeyError): lri.lr_after = [] try: lri.lr_before = lri.prod[i-1] except IndexError: lri.lr_before = None lastlri.lr_next = lri if not lri: break lr_items.append(lri) lastlri = lri i += 1 p.lr_items = lr_items # ----------------------------------------------------------------------------- # == Class LRTable == # # This basic class represents a basic table of LR parsing information. # Methods for generating the tables are not defined here. They are defined # in the derived class LRGeneratedTable. # ----------------------------------------------------------------------------- class VersionError(YaccError): pass class LRTable(object): def __init__(self): self.lr_action = None self.lr_goto = None self.lr_productions = None self.lr_method = None def read_table(self, module): if isinstance(module, types.ModuleType): parsetab = module else: exec('import %s' % module) parsetab = sys.modules[module] if parsetab._tabversion != __tabversion__: raise VersionError('yacc table file version is out of date') self.lr_action = parsetab._lr_action self.lr_goto = parsetab._lr_goto self.lr_productions = [] for p in parsetab._lr_productions: self.lr_productions.append(MiniProduction(*p)) self.lr_method = parsetab._lr_method return parsetab._lr_signature def read_pickle(self, filename): try: import cPickle as pickle except ImportError: import pickle if not os.path.exists(filename): raise ImportError in_f = open(filename, 'rb') tabversion = pickle.load(in_f) if tabversion != __tabversion__: raise VersionError('yacc table file version is out of date') self.lr_method = pickle.load(in_f) signature = pickle.load(in_f) self.lr_action = pickle.load(in_f) self.lr_goto = pickle.load(in_f) productions = pickle.load(in_f) self.lr_productions = [] for p in productions: self.lr_productions.append(MiniProduction(*p)) in_f.close() return signature # Bind all production function names to callable objects in pdict def bind_callables(self, pdict): for p in self.lr_productions: p.bind(pdict) # ----------------------------------------------------------------------------- # === LR Generator === # # The following classes and functions are used to generate LR parsing tables on # a grammar. # ----------------------------------------------------------------------------- # ----------------------------------------------------------------------------- # digraph() # traverse() # # The following two functions are used to compute set valued functions # of the form: # # F(x) = F'(x) U U{F(y) | x R y} # # This is used to compute the values of Read() sets as well as FOLLOW sets # in LALR(1) generation. # # Inputs: X - An input set # R - A relation # FP - Set-valued function # ------------------------------------------------------------------------------ def digraph(X, R, FP): N = {} for x in X: N[x] = 0 stack = [] F = {} for x in X: if N[x] == 0: traverse(x, N, stack, F, X, R, FP) return F def traverse(x, N, stack, F, X, R, FP): stack.append(x) d = len(stack) N[x] = d F[x] = FP(x) # F(X) <- F'(x) rel = R(x) # Get y's related to x for y in rel: if N[y] == 0: traverse(y, N, stack, F, X, R, FP) N[x] = min(N[x], N[y]) for a in F.get(y, []): if a not in F[x]: F[x].append(a) if N[x] == d: N[stack[-1]] = MAXINT F[stack[-1]] = F[x] element = stack.pop() while element != x: N[stack[-1]] = MAXINT F[stack[-1]] = F[x] element = stack.pop() class LALRError(YaccError): pass # ----------------------------------------------------------------------------- # == LRGeneratedTable == # # This class implements the LR table generation algorithm. There are no # public methods except for write() # ----------------------------------------------------------------------------- class LRGeneratedTable(LRTable): def __init__(self, grammar, method='LALR', log=None): if method not in ['SLR', 'LALR']: raise LALRError('Unsupported method %s' % method) self.grammar = grammar self.lr_method = method # Set up the logger if not log: log = NullLogger() self.log = log # Internal attributes self.lr_action = {} # Action table self.lr_goto = {} # Goto table self.lr_productions = grammar.Productions # Copy of grammar Production array self.lr_goto_cache = {} # Cache of computed gotos self.lr0_cidhash = {} # Cache of closures self._add_count = 0 # Internal counter used to detect cycles # Diagonistic information filled in by the table generator self.sr_conflict = 0 self.rr_conflict = 0 self.conflicts = [] # List of conflicts self.sr_conflicts = [] self.rr_conflicts = [] # Build the tables self.grammar.build_lritems() self.grammar.compute_first() self.grammar.compute_follow() self.lr_parse_table() # Compute the LR(0) closure operation on I, where I is a set of LR(0) items. def lr0_closure(self, I): self._add_count += 1 # Add everything in I to J J = I[:] didadd = True while didadd: didadd = False for j in J: for x in j.lr_after: if getattr(x, 'lr0_added', 0) == self._add_count: continue # Add B --> .G to J J.append(x.lr_next) x.lr0_added = self._add_count didadd = True return J # Compute the LR(0) goto function goto(I,X) where I is a set # of LR(0) items and X is a grammar symbol. This function is written # in a way that guarantees uniqueness of the generated goto sets # (i.e. the same goto set will never be returned as two different Python # objects). With uniqueness, we can later do fast set comparisons using # id(obj) instead of element-wise comparison. def lr0_goto(self, I, x): # First we look for a previously cached entry g = self.lr_goto_cache.get((id(I), x)) if g: return g # Now we generate the goto set in a way that guarantees uniqueness # of the result s = self.lr_goto_cache.get(x) if not s: s = {} self.lr_goto_cache[x] = s gs = [] for p in I: n = p.lr_next if n and n.lr_before == x: s1 = s.get(id(n)) if not s1: s1 = {} s[id(n)] = s1 gs.append(n) s = s1 g = s.get('$end') if not g: if gs: g = self.lr0_closure(gs) s['$end'] = g else: s['$end'] = gs self.lr_goto_cache[(id(I), x)] = g return g # Compute the LR(0) sets of item function def lr0_items(self): C = [self.lr0_closure([self.grammar.Productions[0].lr_next])] i = 0 for I in C: self.lr0_cidhash[id(I)] = i i += 1 # Loop over the items in C and each grammar symbols i = 0 while i < len(C): I = C[i] i += 1 # Collect all of the symbols that could possibly be in the goto(I,X) sets asyms = {} for ii in I: for s in ii.usyms: asyms[s] = None for x in asyms: g = self.lr0_goto(I, x) if not g or id(g) in self.lr0_cidhash: continue self.lr0_cidhash[id(g)] = len(C) C.append(g) return C # ----------------------------------------------------------------------------- # ==== LALR(1) Parsing ==== # # LALR(1) parsing is almost exactly the same as SLR except that instead of # relying upon Follow() sets when performing reductions, a more selective # lookahead set that incorporates the state of the LR(0) machine is utilized. # Thus, we mainly just have to focus on calculating the lookahead sets. # # The method used here is due to DeRemer and Pennelo (1982). # # DeRemer, F. L., and T. J. Pennelo: "Efficient Computation of LALR(1) # Lookahead Sets", ACM Transactions on Programming Languages and Systems, # Vol. 4, No. 4, Oct. 1982, pp. 615-649 # # Further details can also be found in: # # J. Tremblay and P. Sorenson, "The Theory and Practice of Compiler Writing", # McGraw-Hill Book Company, (1985). # # ----------------------------------------------------------------------------- # ----------------------------------------------------------------------------- # compute_nullable_nonterminals() # # Creates a dictionary containing all of the non-terminals that might produce # an empty production. # ----------------------------------------------------------------------------- def compute_nullable_nonterminals(self): nullable = set() num_nullable = 0 while True: for p in self.grammar.Productions[1:]: if p.len == 0: nullable.add(p.name) continue for t in p.prod: if t not in nullable: break else: nullable.add(p.name) if len(nullable) == num_nullable: break num_nullable = len(nullable) return nullable # ----------------------------------------------------------------------------- # find_nonterminal_trans(C) # # Given a set of LR(0) items, this functions finds all of the non-terminal # transitions. These are transitions in which a dot appears immediately before # a non-terminal. Returns a list of tuples of the form (state,N) where state # is the state number and N is the nonterminal symbol. # # The input C is the set of LR(0) items. # ----------------------------------------------------------------------------- def find_nonterminal_transitions(self, C): trans = [] for stateno, state in enumerate(C): for p in state: if p.lr_index < p.len - 1: t = (stateno, p.prod[p.lr_index+1]) if t[1] in self.grammar.Nonterminals: if t not in trans: trans.append(t) return trans # ----------------------------------------------------------------------------- # dr_relation() # # Computes the DR(p,A) relationships for non-terminal transitions. The input # is a tuple (state,N) where state is a number and N is a nonterminal symbol. # # Returns a list of terminals. # ----------------------------------------------------------------------------- def dr_relation(self, C, trans, nullable): dr_set = {} state, N = trans terms = [] g = self.lr0_goto(C[state], N) for p in g: if p.lr_index < p.len - 1: a = p.prod[p.lr_index+1] if a in self.grammar.Terminals: if a not in terms: terms.append(a) # This extra bit is to handle the start state if state == 0 and N == self.grammar.Productions[0].prod[0]: terms.append('$end') return terms # ----------------------------------------------------------------------------- # reads_relation() # # Computes the READS() relation (p,A) READS (t,C). # ----------------------------------------------------------------------------- def reads_relation(self, C, trans, empty): # Look for empty transitions rel = [] state, N = trans g = self.lr0_goto(C[state], N) j = self.lr0_cidhash.get(id(g), -1) for p in g: if p.lr_index < p.len - 1: a = p.prod[p.lr_index + 1] if a in empty: rel.append((j, a)) return rel # ----------------------------------------------------------------------------- # compute_lookback_includes() # # Determines the lookback and includes relations # # LOOKBACK: # # This relation is determined by running the LR(0) state machine forward. # For example, starting with a production "N : . A B C", we run it forward # to obtain "N : A B C ." We then build a relationship between this final # state and the starting state. These relationships are stored in a dictionary # lookdict. # # INCLUDES: # # Computes the INCLUDE() relation (p,A) INCLUDES (p',B). # # This relation is used to determine non-terminal transitions that occur # inside of other non-terminal transition states. (p,A) INCLUDES (p', B) # if the following holds: # # B -> LAT, where T -> epsilon and p' -L-> p # # L is essentially a prefix (which may be empty), T is a suffix that must be # able to derive an empty string. State p' must lead to state p with the string L. # # ----------------------------------------------------------------------------- def compute_lookback_includes(self, C, trans, nullable): lookdict = {} # Dictionary of lookback relations includedict = {} # Dictionary of include relations # Make a dictionary of non-terminal transitions dtrans = {} for t in trans: dtrans[t] = 1 # Loop over all transitions and compute lookbacks and includes for state, N in trans: lookb = [] includes = [] for p in C[state]: if p.name != N: continue # Okay, we have a name match. We now follow the production all the way # through the state machine until we get the . on the right hand side lr_index = p.lr_index j = state while lr_index < p.len - 1: lr_index = lr_index + 1 t = p.prod[lr_index] # Check to see if this symbol and state are a non-terminal transition if (j, t) in dtrans: # Yes. Okay, there is some chance that this is an includes relation # the only way to know for certain is whether the rest of the # production derives empty li = lr_index + 1 while li < p.len: if p.prod[li] in self.grammar.Terminals: break # No forget it if p.prod[li] not in nullable: break li = li + 1 else: # Appears to be a relation between (j,t) and (state,N) includes.append((j, t)) g = self.lr0_goto(C[j], t) # Go to next set j = self.lr0_cidhash.get(id(g), -1) # Go to next state # When we get here, j is the final state, now we have to locate the production for r in C[j]: if r.name != p.name: continue if r.len != p.len: continue i = 0 # This look is comparing a production ". A B C" with "A B C ." while i < r.lr_index: if r.prod[i] != p.prod[i+1]: break i = i + 1 else: lookb.append((j, r)) for i in includes: if i not in includedict: includedict[i] = [] includedict[i].append((state, N)) lookdict[(state, N)] = lookb return lookdict, includedict # ----------------------------------------------------------------------------- # compute_read_sets() # # Given a set of LR(0) items, this function computes the read sets. # # Inputs: C = Set of LR(0) items # ntrans = Set of nonterminal transitions # nullable = Set of empty transitions # # Returns a set containing the read sets # ----------------------------------------------------------------------------- def compute_read_sets(self, C, ntrans, nullable): FP = lambda x: self.dr_relation(C, x, nullable) R = lambda x: self.reads_relation(C, x, nullable) F = digraph(ntrans, R, FP) return F # ----------------------------------------------------------------------------- # compute_follow_sets() # # Given a set of LR(0) items, a set of non-terminal transitions, a readset, # and an include set, this function computes the follow sets # # Follow(p,A) = Read(p,A) U U {Follow(p',B) | (p,A) INCLUDES (p',B)} # # Inputs: # ntrans = Set of nonterminal transitions # readsets = Readset (previously computed) # inclsets = Include sets (previously computed) # # Returns a set containing the follow sets # ----------------------------------------------------------------------------- def compute_follow_sets(self, ntrans, readsets, inclsets): FP = lambda x: readsets[x] R = lambda x: inclsets.get(x, []) F = digraph(ntrans, R, FP) return F # ----------------------------------------------------------------------------- # add_lookaheads() # # Attaches the lookahead symbols to grammar rules. # # Inputs: lookbacks - Set of lookback relations # followset - Computed follow set # # This function directly attaches the lookaheads to productions contained # in the lookbacks set # ----------------------------------------------------------------------------- def add_lookaheads(self, lookbacks, followset): for trans, lb in lookbacks.items(): # Loop over productions in lookback for state, p in lb: if state not in p.lookaheads: p.lookaheads[state] = [] f = followset.get(trans, []) for a in f: if a not in p.lookaheads[state]: p.lookaheads[state].append(a) # ----------------------------------------------------------------------------- # add_lalr_lookaheads() # # This function does all of the work of adding lookahead information for use # with LALR parsing # ----------------------------------------------------------------------------- def add_lalr_lookaheads(self, C): # Determine all of the nullable nonterminals nullable = self.compute_nullable_nonterminals() # Find all non-terminal transitions trans = self.find_nonterminal_transitions(C) # Compute read sets readsets = self.compute_read_sets(C, trans, nullable) # Compute lookback/includes relations lookd, included = self.compute_lookback_includes(C, trans, nullable) # Compute LALR FOLLOW sets followsets = self.compute_follow_sets(trans, readsets, included) # Add all of the lookaheads self.add_lookaheads(lookd, followsets) # ----------------------------------------------------------------------------- # lr_parse_table() # # This function constructs the parse tables for SLR or LALR # ----------------------------------------------------------------------------- def lr_parse_table(self): Productions = self.grammar.Productions Precedence = self.grammar.Precedence goto = self.lr_goto # Goto array action = self.lr_action # Action array log = self.log # Logger for output actionp = {} # Action production array (temporary) log.info('Parsing method: %s', self.lr_method) # Step 1: Construct C = { I0, I1, ... IN}, collection of LR(0) items # This determines the number of states C = self.lr0_items() if self.lr_method == 'LALR': self.add_lalr_lookaheads(C) # Build the parser table, state by state st = 0 for I in C: # Loop over each production in I actlist = [] # List of actions st_action = {} st_actionp = {} st_goto = {} log.info('') log.info('state %d', st) log.info('') for p in I: log.info(' (%d) %s', p.number, p) log.info('') for p in I: if p.len == p.lr_index + 1: if p.name == "S'": # Start symbol. Accept! st_action['$end'] = 0 st_actionp['$end'] = p else: # We are at the end of a production. Reduce! if self.lr_method == 'LALR': laheads = p.lookaheads[st] else: laheads = self.grammar.Follow[p.name] for a in laheads: actlist.append((a, p, 'reduce using rule %d (%s)' % (p.number, p))) r = st_action.get(a) if r is not None: # Whoa. Have a shift/reduce or reduce/reduce conflict if r > 0: # Need to decide on shift or reduce here # By default we favor shifting. Need to add # some precedence rules here. sprec, slevel = Productions[st_actionp[a].number].prec rprec, rlevel = Precedence.get(a, ('right', 0)) if (slevel < rlevel) or ((slevel == rlevel) and (rprec == 'left')): # We really need to reduce here. st_action[a] = -p.number st_actionp[a] = p if not slevel and not rlevel: log.info(' ! shift/reduce conflict for %s resolved as reduce', a) self.sr_conflicts.append((st, a, 'reduce')) Productions[p.number].reduced += 1 elif (slevel == rlevel) and (rprec == 'nonassoc'): st_action[a] = None else: # Hmmm. Guess we'll keep the shift if not rlevel: log.info(' ! shift/reduce conflict for %s resolved as shift', a) self.sr_conflicts.append((st, a, 'shift')) elif r < 0: # Reduce/reduce conflict. In this case, we favor the rule # that was defined first in the grammar file oldp = Productions[-r] pp = Productions[p.number] if oldp.line > pp.line: st_action[a] = -p.number st_actionp[a] = p chosenp, rejectp = pp, oldp Productions[p.number].reduced += 1 Productions[oldp.number].reduced -= 1 else: chosenp, rejectp = oldp, pp self.rr_conflicts.append((st, chosenp, rejectp)) log.info(' ! reduce/reduce conflict for %s resolved using rule %d (%s)', a, st_actionp[a].number, st_actionp[a]) else: raise LALRError('Unknown conflict in state %d' % st) else: st_action[a] = -p.number st_actionp[a] = p Productions[p.number].reduced += 1 else: i = p.lr_index a = p.prod[i+1] # Get symbol right after the "." if a in self.grammar.Terminals: g = self.lr0_goto(I, a) j = self.lr0_cidhash.get(id(g), -1) if j >= 0: # We are in a shift state actlist.append((a, p, 'shift and go to state %d' % j)) r = st_action.get(a) if r is not None: # Whoa have a shift/reduce or shift/shift conflict if r > 0: if r != j: raise LALRError('Shift/shift conflict in state %d' % st) elif r < 0: # Do a precedence check. # - if precedence of reduce rule is higher, we reduce. # - if precedence of reduce is same and left assoc, we reduce. # - otherwise we shift rprec, rlevel = Productions[st_actionp[a].number].prec sprec, slevel = Precedence.get(a, ('right', 0)) if (slevel > rlevel) or ((slevel == rlevel) and (rprec == 'right')): # We decide to shift here... highest precedence to shift Productions[st_actionp[a].number].reduced -= 1 st_action[a] = j st_actionp[a] = p if not rlevel: log.info(' ! shift/reduce conflict for %s resolved as shift', a) self.sr_conflicts.append((st, a, 'shift')) elif (slevel == rlevel) and (rprec == 'nonassoc'): st_action[a] = None else: # Hmmm. Guess we'll keep the reduce if not slevel and not rlevel: log.info(' ! shift/reduce conflict for %s resolved as reduce', a) self.sr_conflicts.append((st, a, 'reduce')) else: raise LALRError('Unknown conflict in state %d' % st) else: st_action[a] = j st_actionp[a] = p # Print the actions associated with each terminal _actprint = {} for a, p, m in actlist: if a in st_action: if p is st_actionp[a]: log.info(' %-15s %s', a, m) _actprint[(a, m)] = 1 log.info('') # Print the actions that were not used. (debugging) not_used = 0 for a, p, m in actlist: if a in st_action: if p is not st_actionp[a]: if not (a, m) in _actprint: log.debug(' ! %-15s [ %s ]', a, m) not_used = 1 _actprint[(a, m)] = 1 if not_used: log.debug('') # Construct the goto table for this state nkeys = {} for ii in I: for s in ii.usyms: if s in self.grammar.Nonterminals: nkeys[s] = None for n in nkeys: g = self.lr0_goto(I, n) j = self.lr0_cidhash.get(id(g), -1) if j >= 0: st_goto[n] = j log.info(' %-30s shift and go to state %d', n, j) action[st] = st_action actionp[st] = st_actionp goto[st] = st_goto st += 1 # ----------------------------------------------------------------------------- # write() # # This function writes the LR parsing tables to a file # ----------------------------------------------------------------------------- def write_table(self, tabmodule, outputdir='', signature=''): if isinstance(tabmodule, types.ModuleType): raise IOError("Won't overwrite existing tabmodule") basemodulename = tabmodule.split('.')[-1] filename = os.path.join(outputdir, basemodulename) + '.py' try: f = open(filename, 'w') f.write(''' # %s # This file is automatically generated. Do not edit. _tabversion = %r _lr_method = %r _lr_signature = %r ''' % (os.path.basename(filename), __tabversion__, self.lr_method, signature)) # Change smaller to 0 to go back to original tables smaller = 1 # Factor out names to try and make smaller if smaller: items = {} for s, nd in self.lr_action.items(): for name, v in nd.items(): i = items.get(name) if not i: i = ([], []) items[name] = i i[0].append(s) i[1].append(v) f.write('\n_lr_action_items = {') for k, v in items.items(): f.write('%r:([' % k) for i in v[0]: f.write('%r,' % i) f.write('],[') for i in v[1]: f.write('%r,' % i) f.write(']),') f.write('}\n') f.write(''' _lr_action = {} for _k, _v in _lr_action_items.items(): for _x,_y in zip(_v[0],_v[1]): if not _x in _lr_action: _lr_action[_x] = {} _lr_action[_x][_k] = _y del _lr_action_items ''') else: f.write('\n_lr_action = { ') for k, v in self.lr_action.items(): f.write('(%r,%r):%r,' % (k[0], k[1], v)) f.write('}\n') if smaller: # Factor out names to try and make smaller items = {} for s, nd in self.lr_goto.items(): for name, v in nd.items(): i = items.get(name) if not i: i = ([], []) items[name] = i i[0].append(s) i[1].append(v) f.write('\n_lr_goto_items = {') for k, v in items.items(): f.write('%r:([' % k) for i in v[0]: f.write('%r,' % i) f.write('],[') for i in v[1]: f.write('%r,' % i) f.write(']),') f.write('}\n') f.write(''' _lr_goto = {} for _k, _v in _lr_goto_items.items(): for _x, _y in zip(_v[0], _v[1]): if not _x in _lr_goto: _lr_goto[_x] = {} _lr_goto[_x][_k] = _y del _lr_goto_items ''') else: f.write('\n_lr_goto = { ') for k, v in self.lr_goto.items(): f.write('(%r,%r):%r,' % (k[0], k[1], v)) f.write('}\n') # Write production table f.write('_lr_productions = [\n') for p in self.lr_productions: if p.func: f.write(' (%r,%r,%d,%r,%r,%d),\n' % (p.str, p.name, p.len, p.func, os.path.basename(p.file), p.line)) else: f.write(' (%r,%r,%d,None,None,None),\n' % (str(p), p.name, p.len)) f.write(']\n') f.close() except IOError as e: raise # ----------------------------------------------------------------------------- # pickle_table() # # This function pickles the LR parsing tables to a supplied file object # ----------------------------------------------------------------------------- def pickle_table(self, filename, signature=''): try: import cPickle as pickle except ImportError: import pickle with open(filename, 'wb') as outf: pickle.dump(__tabversion__, outf, pickle_protocol) pickle.dump(self.lr_method, outf, pickle_protocol) pickle.dump(signature, outf, pickle_protocol) pickle.dump(self.lr_action, outf, pickle_protocol) pickle.dump(self.lr_goto, outf, pickle_protocol) outp = [] for p in self.lr_productions: if p.func: outp.append((p.str, p.name, p.len, p.func, os.path.basename(p.file), p.line)) else: outp.append((str(p), p.name, p.len, None, None, None)) pickle.dump(outp, outf, pickle_protocol) # ----------------------------------------------------------------------------- # === INTROSPECTION === # # The following functions and classes are used to implement the PLY # introspection features followed by the yacc() function itself. # ----------------------------------------------------------------------------- # ----------------------------------------------------------------------------- # get_caller_module_dict() # # This function returns a dictionary containing all of the symbols defined within # a caller further down the call stack. This is used to get the environment # associated with the yacc() call if none was provided. # ----------------------------------------------------------------------------- def get_caller_module_dict(levels): f = sys._getframe(levels) ldict = f.f_globals.copy() if f.f_globals != f.f_locals: ldict.update(f.f_locals) return ldict # ----------------------------------------------------------------------------- # parse_grammar() # # This takes a raw grammar rule string and parses it into production data # ----------------------------------------------------------------------------- def parse_grammar(doc, file, line): grammar = [] # Split the doc string into lines pstrings = doc.splitlines() lastp = None dline = line for ps in pstrings: dline += 1 p = ps.split() if not p: continue try: if p[0] == '|': # This is a continuation of a previous rule if not lastp: raise SyntaxError("%s:%d: Misplaced '|'" % (file, dline)) prodname = lastp syms = p[1:] else: prodname = p[0] lastp = prodname syms = p[2:] assign = p[1] if assign != ':' and assign != '::=': raise SyntaxError("%s:%d: Syntax error. Expected ':'" % (file, dline)) grammar.append((file, dline, prodname, syms)) except SyntaxError: raise except Exception: raise SyntaxError('%s:%d: Syntax error in rule %r' % (file, dline, ps.strip())) return grammar # ----------------------------------------------------------------------------- # ParserReflect() # # This class represents information extracted for building a parser including # start symbol, error function, tokens, precedence list, action functions, # etc. # ----------------------------------------------------------------------------- class ParserReflect(object): def __init__(self, pdict, log=None): self.pdict = pdict self.start = None self.error_func = None self.tokens = None self.modules = set() self.grammar = [] self.error = False if log is None: self.log = PlyLogger(sys.stderr) else: self.log = log # Get all of the basic information def get_all(self): self.get_start() self.get_error_func() self.get_tokens() self.get_precedence() self.get_pfunctions() # Validate all of the information def validate_all(self): self.validate_start() self.validate_error_func() self.validate_tokens() self.validate_precedence() self.validate_pfunctions() self.validate_modules() return self.error # Compute a signature over the grammar def signature(self): try: from hashlib import md5 except ImportError: from md5 import md5 try: sig = md5() if self.start: sig.update(self.start.encode('latin-1')) if self.prec: sig.update(''.join([''.join(p) for p in self.prec]).encode('latin-1')) if self.tokens: sig.update(' '.join(self.tokens).encode('latin-1')) for f in self.pfuncs: if f[3]: sig.update(f[3].encode('latin-1')) except (TypeError, ValueError): pass digest = base64.b16encode(sig.digest()) if sys.version_info[0] >= 3: digest = digest.decode('latin-1') return digest # ----------------------------------------------------------------------------- # validate_modules() # # This method checks to see if there are duplicated p_rulename() functions # in the parser module file. Without this function, it is really easy for # users to make mistakes by cutting and pasting code fragments (and it's a real # bugger to try and figure out why the resulting parser doesn't work). Therefore, # we just do a little regular expression pattern matching of def statements # to try and detect duplicates. # ----------------------------------------------------------------------------- def validate_modules(self): # Match def p_funcname( fre = re.compile(r'\s*def\s+(p_[a-zA-Z_0-9]*)\(') for module in self.modules: lines, linen = inspect.getsourcelines(module) counthash = {} for linen, line in enumerate(lines): linen += 1 m = fre.match(line) if m: name = m.group(1) prev = counthash.get(name) if not prev: counthash[name] = linen else: filename = inspect.getsourcefile(module) self.log.warning('%s:%d: Function %s redefined. Previously defined on line %d', filename, linen, name, prev) # Get the start symbol def get_start(self): self.start = self.pdict.get('start') # Validate the start symbol def validate_start(self): if self.start is not None: if not isinstance(self.start, string_types): self.log.error("'start' must be a string") # Look for error handler def get_error_func(self): self.error_func = self.pdict.get('p_error') # Validate the error function def validate_error_func(self): if self.error_func: if isinstance(self.error_func, types.FunctionType): ismethod = 0 elif isinstance(self.error_func, types.MethodType): ismethod = 1 else: self.log.error("'p_error' defined, but is not a function or method") self.error = True return eline = self.error_func.__code__.co_firstlineno efile = self.error_func.__code__.co_filename module = inspect.getmodule(self.error_func) self.modules.add(module) argcount = self.error_func.__code__.co_argcount - ismethod if argcount != 1: self.log.error('%s:%d: p_error() requires 1 argument', efile, eline) self.error = True # Get the tokens map def get_tokens(self): tokens = self.pdict.get('tokens') if not tokens: self.log.error('No token list is defined') self.error = True return if not isinstance(tokens, (list, tuple)): self.log.error('tokens must be a list or tuple') self.error = True return if not tokens: self.log.error('tokens is empty') self.error = True return self.tokens = tokens # Validate the tokens def validate_tokens(self): # Validate the tokens. if 'error' in self.tokens: self.log.error("Illegal token name 'error'. Is a reserved word") self.error = True return terminals = set() for n in self.tokens: if n in terminals: self.log.warning('Token %r multiply defined', n) terminals.add(n) # Get the precedence map (if any) def get_precedence(self): self.prec = self.pdict.get('precedence') # Validate and parse the precedence map def validate_precedence(self): preclist = [] if self.prec: if not isinstance(self.prec, (list, tuple)): self.log.error('precedence must be a list or tuple') self.error = True return for level, p in enumerate(self.prec): if not isinstance(p, (list, tuple)): self.log.error('Bad precedence table') self.error = True return if len(p) < 2: self.log.error('Malformed precedence entry %s. Must be (assoc, term, ..., term)', p) self.error = True return assoc = p[0] if not isinstance(assoc, string_types): self.log.error('precedence associativity must be a string') self.error = True return for term in p[1:]: if not isinstance(term, string_types): self.log.error('precedence items must be strings') self.error = True return preclist.append((term, assoc, level+1)) self.preclist = preclist # Get all p_functions from the grammar def get_pfunctions(self): p_functions = [] for name, item in self.pdict.items(): if not name.startswith('p_') or name == 'p_error': continue if isinstance(item, (types.FunctionType, types.MethodType)): line = item.__code__.co_firstlineno module = inspect.getmodule(item) p_functions.append((line, module, name, item.__doc__)) # Sort all of the actions by line number; make sure to stringify # modules to make them sortable, since `line` may not uniquely sort all # p functions p_functions.sort(key=lambda p_function: ( p_function[0], str(p_function[1]), p_function[2], p_function[3])) self.pfuncs = p_functions # Validate all of the p_functions def validate_pfunctions(self): grammar = [] # Check for non-empty symbols if len(self.pfuncs) == 0: self.log.error('no rules of the form p_rulename are defined') self.error = True return for line, module, name, doc in self.pfuncs: file = inspect.getsourcefile(module) func = self.pdict[name] if isinstance(func, types.MethodType): reqargs = 2 else: reqargs = 1 if func.__code__.co_argcount > reqargs: self.log.error('%s:%d: Rule %r has too many arguments', file, line, func.__name__) self.error = True elif func.__code__.co_argcount < reqargs: self.log.error('%s:%d: Rule %r requires an argument', file, line, func.__name__) self.error = True elif not func.__doc__: self.log.warning('%s:%d: No documentation string specified in function %r (ignored)', file, line, func.__name__) else: try: parsed_g = parse_grammar(doc, file, line) for g in parsed_g: grammar.append((name, g)) except SyntaxError as e: self.log.error(str(e)) self.error = True # Looks like a valid grammar rule # Mark the file in which defined. self.modules.add(module) # Secondary validation step that looks for p_ definitions that are not functions # or functions that look like they might be grammar rules. for n, v in self.pdict.items(): if n.startswith('p_') and isinstance(v, (types.FunctionType, types.MethodType)): continue if n.startswith('t_'): continue if n.startswith('p_') and n != 'p_error': self.log.warning('%r not defined as a function', n) if ((isinstance(v, types.FunctionType) and v.__code__.co_argcount == 1) or (isinstance(v, types.MethodType) and v.__func__.__code__.co_argcount == 2)): if v.__doc__: try: doc = v.__doc__.split(' ') if doc[1] == ':': self.log.warning('%s:%d: Possible grammar rule %r defined without p_ prefix', v.__code__.co_filename, v.__code__.co_firstlineno, n) except IndexError: pass self.grammar = grammar # ----------------------------------------------------------------------------- # yacc(module) # # Build a parser # ----------------------------------------------------------------------------- def yacc(method='LALR', debug=yaccdebug, module=None, tabmodule=tab_module, start=None, check_recursion=True, optimize=False, write_tables=True, debugfile=debug_file, outputdir=None, debuglog=None, errorlog=None, picklefile=None): if tabmodule is None: tabmodule = tab_module # Reference to the parsing method of the last built parser global parse # If pickling is enabled, table files are not created if picklefile: write_tables = 0 if errorlog is None: errorlog = PlyLogger(sys.stderr) # Get the module dictionary used for the parser if module: _items = [(k, getattr(module, k)) for k in dir(module)] pdict = dict(_items) # If no __file__ attribute is available, try to obtain it from the __module__ instead if '__file__' not in pdict: pdict['__file__'] = sys.modules[pdict['__module__']].__file__ else: pdict = get_caller_module_dict(2) if outputdir is None: # If no output directory is set, the location of the output files # is determined according to the following rules: # - If tabmodule specifies a package, files go into that package directory # - Otherwise, files go in the same directory as the specifying module if isinstance(tabmodule, types.ModuleType): srcfile = tabmodule.__file__ else: if '.' not in tabmodule: srcfile = pdict['__file__'] else: parts = tabmodule.split('.') pkgname = '.'.join(parts[:-1]) exec('import %s' % pkgname) srcfile = getattr(sys.modules[pkgname], '__file__', '') outputdir = os.path.dirname(srcfile) # Determine if the module is package of a package or not. # If so, fix the tabmodule setting so that tables load correctly pkg = pdict.get('__package__') if pkg and isinstance(tabmodule, str): if '.' not in tabmodule: tabmodule = pkg + '.' + tabmodule # Set start symbol if it's specified directly using an argument if start is not None: pdict['start'] = start # Collect parser information from the dictionary pinfo = ParserReflect(pdict, log=errorlog) pinfo.get_all() if pinfo.error: raise YaccError('Unable to build parser') # Check signature against table files (if any) signature = pinfo.signature() # Read the tables try: lr = LRTable() if picklefile: read_signature = lr.read_pickle(picklefile) else: read_signature = lr.read_table(tabmodule) if optimize or (read_signature == signature): try: lr.bind_callables(pinfo.pdict) parser = LRParser(lr, pinfo.error_func) parse = parser.parse return parser except Exception as e: errorlog.warning('There was a problem loading the table file: %r', e) except VersionError as e: errorlog.warning(str(e)) except ImportError: pass if debuglog is None: if debug: try: debuglog = PlyLogger(open(os.path.join(outputdir, debugfile), 'w')) except IOError as e: errorlog.warning("Couldn't open %r. %s" % (debugfile, e)) debuglog = NullLogger() else: debuglog = NullLogger() debuglog.info('Created by PLY version %s (http://www.dabeaz.com/ply)', __version__) errors = False # Validate the parser information if pinfo.validate_all(): raise YaccError('Unable to build parser') if not pinfo.error_func: errorlog.warning('no p_error() function is defined') # Create a grammar object grammar = Grammar(pinfo.tokens) # Set precedence level for terminals for term, assoc, level in pinfo.preclist: try: grammar.set_precedence(term, assoc, level) except GrammarError as e: errorlog.warning('%s', e) # Add productions to the grammar for funcname, gram in pinfo.grammar: file, line, prodname, syms = gram try: grammar.add_production(prodname, syms, funcname, file, line) except GrammarError as e: errorlog.error('%s', e) errors = True # Set the grammar start symbols try: if start is None: grammar.set_start(pinfo.start) else: grammar.set_start(start) except GrammarError as e: errorlog.error(str(e)) errors = True if errors: raise YaccError('Unable to build parser') # Verify the grammar structure undefined_symbols = grammar.undefined_symbols() for sym, prod in undefined_symbols: errorlog.error('%s:%d: Symbol %r used, but not defined as a token or a rule', prod.file, prod.line, sym) errors = True unused_terminals = grammar.unused_terminals() if unused_terminals: debuglog.info('') debuglog.info('Unused terminals:') debuglog.info('') for term in unused_terminals: errorlog.warning('Token %r defined, but not used', term) debuglog.info(' %s', term) # Print out all productions to the debug log if debug: debuglog.info('') debuglog.info('Grammar') debuglog.info('') for n, p in enumerate(grammar.Productions): debuglog.info('Rule %-5d %s', n, p) # Find unused non-terminals unused_rules = grammar.unused_rules() for prod in unused_rules: errorlog.warning('%s:%d: Rule %r defined, but not used', prod.file, prod.line, prod.name) if len(unused_terminals) == 1: errorlog.warning('There is 1 unused token') if len(unused_terminals) > 1: errorlog.warning('There are %d unused tokens', len(unused_terminals)) if len(unused_rules) == 1: errorlog.warning('There is 1 unused rule') if len(unused_rules) > 1: errorlog.warning('There are %d unused rules', len(unused_rules)) if debug: debuglog.info('') debuglog.info('Terminals, with rules where they appear') debuglog.info('') terms = list(grammar.Terminals) terms.sort() for term in terms: debuglog.info('%-20s : %s', term, ' '.join([str(s) for s in grammar.Terminals[term]])) debuglog.info('') debuglog.info('Nonterminals, with rules where they appear') debuglog.info('') nonterms = list(grammar.Nonterminals) nonterms.sort() for nonterm in nonterms: debuglog.info('%-20s : %s', nonterm, ' '.join([str(s) for s in grammar.Nonterminals[nonterm]])) debuglog.info('') if check_recursion: unreachable = grammar.find_unreachable() for u in unreachable: errorlog.warning('Symbol %r is unreachable', u) infinite = grammar.infinite_cycles() for inf in infinite: errorlog.error('Infinite recursion detected for symbol %r', inf) errors = True unused_prec = grammar.unused_precedence() for term, assoc in unused_prec: errorlog.error('Precedence rule %r defined for unknown symbol %r', assoc, term) errors = True if errors: raise YaccError('Unable to build parser') # Run the LRGeneratedTable on the grammar if debug: errorlog.debug('Generating %s tables', method) lr = LRGeneratedTable(grammar, method, debuglog) if debug: num_sr = len(lr.sr_conflicts) # Report shift/reduce and reduce/reduce conflicts if num_sr == 1: errorlog.warning('1 shift/reduce conflict') elif num_sr > 1: errorlog.warning('%d shift/reduce conflicts', num_sr) num_rr = len(lr.rr_conflicts) if num_rr == 1: errorlog.warning('1 reduce/reduce conflict') elif num_rr > 1: errorlog.warning('%d reduce/reduce conflicts', num_rr) # Write out conflicts to the output file if debug and (lr.sr_conflicts or lr.rr_conflicts): debuglog.warning('') debuglog.warning('Conflicts:') debuglog.warning('') for state, tok, resolution in lr.sr_conflicts: debuglog.warning('shift/reduce conflict for %s in state %d resolved as %s', tok, state, resolution) already_reported = set() for state, rule, rejected in lr.rr_conflicts: if (state, id(rule), id(rejected)) in already_reported: continue debuglog.warning('reduce/reduce conflict in state %d resolved using rule (%s)', state, rule) debuglog.warning('rejected rule (%s) in state %d', rejected, state) errorlog.warning('reduce/reduce conflict in state %d resolved using rule (%s)', state, rule) errorlog.warning('rejected rule (%s) in state %d', rejected, state) already_reported.add((state, id(rule), id(rejected))) warned_never = [] for state, rule, rejected in lr.rr_conflicts: if not rejected.reduced and (rejected not in warned_never): debuglog.warning('Rule (%s) is never reduced', rejected) errorlog.warning('Rule (%s) is never reduced', rejected) warned_never.append(rejected) # Write the table file if requested if write_tables: try: lr.write_table(tabmodule, outputdir, signature) except IOError as e: errorlog.warning("Couldn't create %r. %s" % (tabmodule, e)) # Write a pickled version of the tables if picklefile: try: lr.pickle_table(picklefile, signature) except IOError as e: errorlog.warning("Couldn't create %r. %s" % (picklefile, e)) # Build the parser lr.bind_callables(pinfo.pdict) parser = LRParser(lr, pinfo.error_func) parse = parser.parse return parser ================================================ FILE: audiotools/speex.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools import AudioFile, InvalidFile, BIN class InvalidSpeex(InvalidFile): pass class SpeexAudio(AudioFile): """an Ogg Speex audio file""" from audiotools.text import (COMP_SPEEX_0, COMP_SPEEX_10) SUFFIX = "spx" NAME = SUFFIX DESCRIPTION = "Ogg Speex" COMPRESSION_MODES = tuple(str(i) for i in range(11)) COMPRESSION_DESCRIPTIONS = {"0": COMP_SPEEX_0, "10": COMP_SPEEX_10} BINARIES = ("speexdec", "speexenc") BINARY_URLS = {"speexenc": "http://www.speex.org", "speexdec": "http://www.speex.org"} def __init__(self, filename): from audiotools.bitstream import BitstreamReader AudioFile.__init__(self, filename) try: with BitstreamReader( open(self.filename, "rb"), True) as ogg_reader: (magic_number, version, header_type, granule_position, self.__serial_number__, page_sequence_number, checksum, segment_count) = ogg_reader.parse("4b 8u 8u 64S 32u 32u 32u 8u") if magic_number != b'OggS': from audiotools.text import ERR_OGG_INVALID_MAGIC_NUMBER raise InvalidVorbis(ERR_OGG_INVALID_MAGIC_NUMBER) if version != 0: from audiotools.text import ERR_OGG_INVALID_VERSION raise InvalidVorbis(ERR_OGG_INVALID_VERSION) segment_lengths = [ogg_reader.read(8) for i in range(segment_count)] (speex_string, speex_version, speex_version_id, header_size, self.__sampling_rate__, mode, mode_bitstream_version, self.__channels__, bitrate, frame_size, vbr, frame_per_packet, extra_headers, reserved1, reserved2) = ogg_reader.parse("8b 20b 13*32u") if speex_string != b"Speex ": raise InvalidSpeex(ERR_SPEEX_INVALID_VERSION) except IOError as err: raise InvalidSpeex(str(err)) def bits_per_sample(self): """returns an integer number of bits-per-sample this track contains""" return 16 def channels(self): """returns an integer number of channels this track contains""" return self.__channels__ def total_frames(self): """returns the total PCM frames of the track as an integer""" from audiotools._ogg import PageReader try: with PageReader(open(self.filename, "rb")) as reader: page = reader.read() pcm_samples = page.granule_position while not page.stream_end: page = reader.read() pcm_samples = max(pcm_samples, page.granule_position) return pcm_samples except (IOError, ValueError): return 0 def lossless(self): """returns True if this track's data is stored losslessly""" return False @classmethod def supports_metadata(cls): """returns True if this audio type supports MetaData""" return True def update_metadata(self, metadata): """takes this track's current MetaData object as returned by get_metadata() and sets this track's metadata with any fields updated in that object raises IOError if unable to write the file """ import os from audiotools import TemporaryFile from audiotools.ogg import (PageReader, PacketReader, PageWriter, packet_to_pages, packets_to_pages) from audiotools.vorbiscomment import VorbisComment from audiotools.bitstream import BitstreamRecorder if metadata is None: return elif not isinstance(metadata, VorbisComment): from audiotools.text import ERR_FOREIGN_METADATA raise ValueError(ERR_FOREIGN_METADATA) elif not os.access(self.filename, os.W_OK): raise IOError(self.filename) original_ogg = PacketReader(PageReader(open(self.filename, "rb"))) new_ogg = PageWriter(TemporaryFile(self.filename)) sequence_number = 0 # transfer current file's identification packet in its own page identification_packet = original_ogg.read_packet() for (i, page) in enumerate(packet_to_pages( identification_packet, self.__serial_number__, starting_sequence_number=sequence_number)): page.stream_beginning = (i == 0) new_ogg.write(page) sequence_number += 1 # discard current file's comment packet comment_packet = original_ogg.read_packet() # generate new comment packet comment_writer = BitstreamRecorder(True) vendor_string = metadata.vendor_string.encode("utf-8") comment_writer.build("32u {:d}b".format(len(vendor_string)), (len(vendor_string), vendor_string)) comment_writer.write(32, len(metadata.comment_strings)) for comment_string in metadata.comment_strings: comment_string = comment_string.encode("utf-8") comment_writer.build("32u {:d}b".format(len(comment_string)), (len(comment_string), comment_string)) for page in packets_to_pages( [comment_writer.data()], self.__serial_number__, starting_sequence_number=sequence_number): new_ogg.write(page) sequence_number += 1 # transfer remaining pages after re-sequencing page = original_ogg.read_page() page.sequence_number = sequence_number page.bitstream_serial_number = self.__serial_number__ sequence_number += 1 new_ogg.write(page) while not page.stream_end: page = original_ogg.read_page() page.sequence_number = sequence_number page.bitstream_serial_number = self.__serial_number__ sequence_number += 1 new_ogg.write(page) # commit changes original_ogg.close() new_ogg.close() def set_metadata(self, metadata): """takes a MetaData object and sets this track's metadata this metadata includes track name, album name, and so on raises IOError if unable to write the file""" from audiotools.vorbiscomment import VorbisComment if metadata is None: return self.delete_metadata() metadata = VorbisComment.converted(metadata) old_metadata = self.get_metadata() metadata.vendor_string = old_metadata.vendor_string # remove REPLAYGAIN_* tags from new metadata (if any) for key in [u"REPLAYGAIN_TRACK_GAIN", u"REPLAYGAIN_TRACK_PEAK", u"REPLAYGAIN_ALBUM_GAIN", u"REPLAYGAIN_ALBUM_PEAK", u"REPLAYGAIN_REFERENCE_LOUDNESS"]: try: metadata[key] = old_metadata[key] except KeyError: metadata[key] = [] self.update_metadata(metadata) def get_metadata(self): """returns a MetaData object, or None raises IOError if unable to read the file""" from io import BytesIO from audiotools.bitstream import BitstreamReader from audiotools.ogg import PacketReader, PageReader from audiotools.vorbiscomment import VorbisComment with PacketReader(PageReader(open(self.filename, "rb"))) as reader: identification = reader.read_packet() comment = BitstreamReader(BytesIO(reader.read_packet()), True) vendor_string = \ comment.read_bytes(comment.read(32)).decode('utf-8') comment_strings = [ comment.read_bytes(comment.read(32)).decode('utf-8') for i in range(comment.read(32))] return VorbisComment(comment_strings, vendor_string) def delete_metadata(self): """deletes the track's MetaData this removes or unsets tags as necessary in order to remove all data raises IOError if unable to write the file""" from audiotools import MetaData # the vorbis comment packet is required, # so simply zero out its contents self.set_metadata(MetaData()) def sample_rate(self): """returns the rate of the track's audio as an integer number of Hz""" return self.__sampling_rate__ @classmethod def supports_to_pcm(cls): """returns True if all necessary components are available to support the .to_pcm() method""" return BIN.can_execute(BIN["speexdec"]) def to_pcm(self): """returns a PCMReader object containing the track's PCM data if an error occurs initializing a decoder, this should return a PCMReaderError with an appropriate error message""" from audiotools import PCMFileReader import os import subprocess sub = subprocess.Popen( [BIN["speexdec"], self.filename, "-"], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL if hasattr(subprocess, "DEVNULL") else open(os.devnull, "wb")) return PCMFileReader( file=sub.stdout, sample_rate=self.sample_rate(), channels=self.channels(), channel_mask=int(self.channel_mask()), bits_per_sample=self.bits_per_sample(), process=sub) @classmethod def supports_from_pcm(cls): """returns True if all necessary components are available to support the .from_pcm() classmethod""" return BIN.can_execute(BIN["speexenc"]) @classmethod def from_pcm(cls, filename, pcmreader, compression=None, total_pcm_frames=None): """encodes a new file from PCM data takes a filename string, PCMReader object optional compression level string, and optional total_pcm_frames integer encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new AudioFile-compatible object specifying total_pcm_frames, when the number is known in advance, may allow the encoder to work more efficiently but is never required """ import bisect import os import subprocess from audiotools import __default_quality__ from audiotools import transfer_framelist_data from audiotools import EncodingError from audiotools import PCMConverter from audiotools import ChannelMask if ((compression is None) or (compression not in cls.COMPRESSION_MODES)): compression = __default_quality__(cls.NAME) if pcmreader.bits_per_sample not in (8, 16, 24): from audiotools import UnsupportedBitsPerSample raise UnsupportedBitsPerSample( filename, pcmreader.bits_per_sample) if total_pcm_frames is not None: from audiotools import CounterPCMReader counter_reader = CounterPCMReader(pcmreader) else: counter_reader = pcmreader pcmreader = PCMConverter( counter_reader, sample_rate=[8000, 8000, 16000, 32000][bisect.bisect( [8000, 16000, 32000], pcmreader.sample_rate)], channels=min(pcmreader.channels, 2), channel_mask=ChannelMask.from_channels( min(pcmreader.channels, 2)), bits_per_sample=min(pcmreader.bits_per_sample, 16)) BITS_PER_SAMPLE = {8: ['--8bit'], 16: ['--16bit']}[pcmreader.bits_per_sample] CHANNELS = {1: [], 2: ['--stereo']}[pcmreader.channels] sub = subprocess.Popen( [BIN['speexenc'], '--quality', str(compression), '--rate', str(pcmreader.sample_rate), '--le'] + \ BITS_PER_SAMPLE + \ CHANNELS + \ ['-', filename], stdin=subprocess.PIPE, stderr=subprocess.DEVNULL if hasattr(subprocess, "DEVNULL") else open(os.devnull, "wb")) try: transfer_framelist_data(pcmreader, sub.stdin.write) except (IOError, ValueError) as err: sub.stdin.close() sub.wait() cls.__unlink__(filename) raise EncodingError(str(err)) except Exception as err: sub.stdin.close() sub.wait() cls.__unlink__(filename) raise err sub.stdin.close() if sub.wait() == 0: if ((total_pcm_frames is None) or (total_pcm_frames == counter_reader.frames_written)): return SpeexAudio(filename) else: from audiotools.text import ERR_TOTAL_PCM_FRAMES_MISMATCH cls.__unlink__(filename) raise EncodingError(ERR_TOTAL_PCM_FRAMES_MISMATCH) else: raise EncodingError(u"unable to encode file with speexenc") ================================================ FILE: audiotools/text.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA """a text strings module""" DIV = u"\u2500" # Utility usage USAGE_TRACKCMP_CDIMAGE = u"<CD image> <track 1> <track 2> ..." USAGE_TRACKCMP_FILES = u"<track 1> <track 2>" # Utility Descriptions DESCRIPTION_AT_CONFIG = \ "set default parameters" DESCRIPTION_COVERDUMP = \ "extract embedded images from file" DESCRIPTION_COVERBROWSE = \ "browse embedded cover art" DESCRIPTION_CD2TRACK = \ "extract CD audio tracks to files" DESCRIPTION_CDINFO = \ "display information about audio CD" DESCRIPTION_CDPLAY = \ "play audio CD" DESCRIPTION_COVERTAG = \ "set embedded file images" DESCRIPTION_DVDA2TRACK = \ "extract DVA-A tracks to files" DESCRIPTION_DVDAINFO = \ "display information about DVD-A" DESCRIPTION_TRACKCMP = \ "compare two files or directories" DESCRIPTION_TRACK2CD = \ "burn files to audio CD" DESCRIPTION_TRACKCAT = \ "concatenate multiple files into a single file" DESCRIPTION_TRACKINFO = \ "display information about a file" DESCRIPTION_TRACKLENGTH = \ "summarize total file lengths, in seconds" DESCRIPTION_TRACKLINT = \ "fix common file metadata problems" DESCRIPTION_TRACKPLAY = \ "play files" DESCRIPTION_TRACKRENAME = \ "rename files based on internal metadata" DESCRIPTION_TRACKSPLIT = \ "split a single file into multiple files" DESCRIPTION_TRACKTAG = \ "set file metadata attributes" DESCRIPTION_TRACK2TRACK = \ "convert audio files from one format to another" DESCRIPTION_TRACKVERIFY = \ "verify correctness of files" # Utility Options OPT_VERBOSE = u"the verbosity level to execute at" OPT_VERBOSE_AT_CONFIG = u"the new default verbosity level" OPT_INPUT_FILENAME = u"input filename" OPT_INPUT_FILENAME_OR_DIR = u"input filename or directory" OPT_INPUT_FILENAME_OR_IMAGE = u"input filename, directory or CD image filename" OPT_TRACK_INDEX = u"track index number, starting from 1" OPT_TYPE = u"the type of audio track to create" OPT_TYPE_AT_CONFIG = u"the default audio type to use, " + \ u"or the type for a given default quality level" OPT_TYPE_TRACKVERIFY = u"a type of audio to accept" OPT_QUALITY = u"the quality to store audio tracks at" OPT_QUALITY_AT_CONFIG = u"the default quality level for a given audio type" OPT_DIR = u"the directory to store new audio tracks" OPT_INITIAL_DIR = u"initial directory" OPT_DIR_IMAGES = u"the directory to store extracted images" OPT_FORMAT = u"the format string for new filenames" OPT_METADATA_LOOKUP = u"perform metadata lookup" OPT_NO_MUSICBRAINZ = u"do not query MusicBrainz for metadata" OPT_NO_FREEDB = u"do not query FreeDB for metadata" OPT_INTERACTIVE_METADATA = u"edit metadata interactively" OPT_INTERACTIVE_OPTIONS = u"edit metadata and output options interactively" OPT_INTERACTIVE_PLAY = u"play in interactive mode" OPT_INTERACTIVE_AT_CONFIG = u"edit options interactively" OPT_OUTPUT_PLAY = u"the system output to use" OPT_OUTPUT_TRACK2TRACK = u"output filename to use, overriding default and -d" OPT_OUTPUT_TRACKCAT = u"the output file" OPT_DEFAULT = u"when multiple choices are available, " + \ u"select the first one automatically" OPT_ALBUM_NUMBER = \ u"the album number of this disc, if it is one of a series of albums" OPT_ALBUM_TOTAL = \ u"the total albums of this disc\'s set, if it is one of a series of albums" OPT_REPLAY_GAIN = u"add ReplayGain metadata to newly created tracks" OPT_REPLAY_GAIN_TRACKTAG = u"add ReplayGain metadata to tracks" OPT_REMOVE_REPLAY_GAIN_TRACKTAG = u"remove ReplayGain metadata from tracks" OPT_NO_REPLAY_GAIN = u"do not add ReplayGain metadata in newly created tracks" OPT_PLAYBACK_TRACK_GAIN = u"apply track ReplayGain during playback, if present" OPT_PLAYBACK_ALBUM_GAIN = u"apply album ReplayGain during playback, if present" OPT_SHUFFLE = u"shuffle tracks" OPT_PREFIX = u"add a prefix to the output image" OPT_NO_GTK = u"don't use PyGTK for GUI" OPT_NO_TKINTER = u"don't use Tkinter for GUI" OPT_AUDIO_TS = u"location of AUDIO_TS directory" OPT_DVDA_TITLE = u"DVD-Audio title number to extract tracks from" OPT_TRACK_START = u"the starting track number of the title being extracted" OPT_TRACK_TOTAL = \ u"the total number of tracks, if the extracted title is only a subset" OPT_SPEED = u"the speed to burn the CD at" OPT_CUESHEET_TRACK2CD = u"the cuesheet to use for writing tracks" OPT_JOINT = u"the maximum number of processes to run at a time" OPT_CUESHEET_TRACKCAT = u"a cuesheet to embed in the output file" OPT_ADD_CUESHEET_TRACKCAT = u"create a cuesheet to embed in the output file" OPT_CUESHEET_TRACKSPLIT = u"the cuesheet to use for splitting track" OPT_CUESHEET_TRACKCMP = u"cuesheet to use for comparing disc image to tracks" OPT_CUESHEET_TRACKVERIFY = \ u"the cuesheet to verify disc image with AccurateRip" OPT_CUESHEET_CDDA2TRACK = \ u"cuesheet to generate from CD contents" OPT_NO_SUMMARY = u"suppress summary output" OPT_ACCURATERIP = u"verify tracks against those of AccurateRip database" OPT_SAMPLE_RATE = u"sample rate of output files, in Hz" OPT_CHANNELS = u"channel count of output files" OPT_BPS = u"bits-per-sample of output files" OPT_TRACKLINT_FIX = u"perform suggest fixes" OPT_TRACKTAG_COMMENT_FILE = u"a file containing comment text" OPT_TRACKTAG_REPLACE = u"completely replace all metadata" OPT_TRACKTAG_CUESHEET = u"a cuesheet to import or get audio metadata from" OPT_TRACKTAG_REMOVE_IMAGES = u"remove existing images prior to adding new ones" OPT_TRACKTAG_FRONT_COVER = u"an image file of the front cover" OPT_TRACKTAG_BACK_COVER = u"an image file of the back cover" OPT_TRACKTAG_LEAFLET = u"an image file of a leaflet page" OPT_TRACKTAG_MEDIA = u"an image file of the media" OPT_TRACKTAG_OTHER_IMAGE = u"an image file related to the track" OPT_AT_CONFIG_READ_OFFSET = u"the CD-ROM read offset to use" OPT_AT_CONFIG_WRITE_OFFSET = u"the CD-ROM write offset to use" OPT_AT_CONFIG_FS_ENCODING = u"the filesystem's text encoding" OPT_AT_CONFIG_IO_ENCODING = u"the system's text encoding" OPT_AT_CONFIG_ID3V2_VERSION = u"which ID3v2 version to use by default, if any" OPT_AT_CONFIG_ID3V1_VERSION = u"which ID3v1 version to use by default, if any" OPT_AT_CONFIG_ID3V2_PAD = \ u"whether or not to pad ID3v2 digit fields to 2 digits" OPT_CAT_EXTRACTION = u"extraction arguments" OPT_CAT_CD_LOOKUP = u"CD lookup arguments" OPT_CAT_DVDA_LOOKUP = u"DVD-A lookup arguments" OPT_CAT_METADATA = u"metadata arguments" OPT_CAT_CONVERSION = u"conversion arguments" OPT_CAT_OUTPUT_FORMAT = u"format arguments" OPT_CAT_ENCODING = u"encoding arguments" OPT_CAT_TEXT = u"text arguments" OPT_CAT_IMAGE = u"image arguments" OPT_CAT_REMOVAL = u"removal arguments" OPT_CAT_SYSTEM = u"system arguments" OPT_CAT_TRANSCODING = u"transcoding arguments" OPT_CAT_ID3 = u"ID3 arguments" OPT_CAT_REPLAYGAIN = u"ReplayGain Options" OPT_CAT_BINARIES = u"binaries arguments" # MetaData Fields METADATA_TRACK_NAME = u"track name" METADATA_TRACK_NUMBER = u"track number" METADATA_TRACK_TOTAL = u"track total" METADATA_ALBUM_NAME = u"album name" METADATA_ARTIST_NAME = u"artist name" METADATA_PERFORMER_NAME = u"performer name" METADATA_COMPOSER_NAME = u"composer name" METADATA_CONDUCTOR_NAME = u"conductor name" METADATA_MEDIA = u"media" METADATA_ISRC = u"ISRC" METADATA_CATALOG = u"catalog number" METADATA_COPYRIGHT = u"copyright" METADATA_PUBLISHER = u"publisher" METADATA_YEAR = u"release year" METADATA_DATE = u"recording date" METADATA_ALBUM_NUMBER = u"album number" METADATA_ALBUM_TOTAL = u"album total" METADATA_COMMENT = u"comment" METADATA_COMPILATION = u"compilation part" METADATA_TRUE = u"yes" METADATA_FALSE = u"no" # Derived MetaData Fields METADATA_SUFFIX = u"file name suffix" METADATA_ALBUM_TRACK_NUMBER = u"combined album and track number" METADATA_BASENAME = u"file name without suffix" # ReplayGain RG_ADDING_REPLAYGAIN = u"Adding ReplayGain" RG_APPLYING_REPLAYGAIN = u"Applying ReplayGain" RG_ADDING_REPLAYGAIN_TO_ALBUM = u"Adding ReplayGain to album {:d}" RG_ADDING_REPLAYGAIN_WAIT = \ u"Adding ReplayGain metadata. This may take some time." RG_REPLAYGAIN_ADDED = u"ReplayGain added" RG_REPLAYGAIN_REMOVED = u"ReplayGain removed" RG_REPLAYGAIN_ADDED_TO_ALBUM = u"ReplayGain added to album {:d}" RG_REPLAYGAIN_REMOVED_FROM_ALBUM = u"ReplayGain removed from album {:d}" RG_REPLAYGAIN_REMOVED = u"ReplayGain removed" RG_REPLAYGAIN_APPLIED = u"ReplayGain applied" # Labels LAB_ENCODE = u"{source} -> {destination}" LAB_PICTURE = u"picture" LAB_T_OPTIONS = u"Please use the -t option to specify {}" LAB_AVAILABLE_COMPRESSION_TYPES = u"Available quality modes for \"{}\":" LAB_AVAILABLE_FORMATS = u"Available output formats:" LAB_OUTPUT_FORMATS = u"Output Formats" LAB_OUTPUT_TYPE = u"type" LAB_OUTPUT_QUALITY = u"quality" LAB_OUTPUT_TYPE_DESCRIPTION = u"name" LAB_OUTPUT_QUALITY_DESCRIPTION = u"description" LAB_SUPPORTED_FIELDS = u"Supported fields are:" LAB_CD2TRACK_PROGRESS = u"track {track_number:02d} -> {filename}" LAB_CD2TRACK_LOG = u"Rip log : " LAB_CD2TRACK_APPLY = u"extract tracks" LAB_CDDA2TRACK_WROTE_CUESHEET = u"wrote cuesheet \"{}\"" LAB_ACCURATERIP_CHECKSUM = u"checksum" LAB_ACCURATERIP_RESULT = u"AccurateRip result" LAB_ACCURATERIP_NOT_FOUND = u"disc not in database" LAB_ACCURATERIP_FOUND = u"found" LAB_ACCURATERIP_CONFIDENCE = u"confidence {:d}" LAB_ACCURATERIP_MISMATCH = u"no match in database" LAB_TOTAL_TRACKS = u"Total Tracks" LAB_TOTAL_LENGTH = u"Total Length" LAB_TRACK_LENGTH = u"{:d}:{:02d}" LAB_TRACK_LENGTH_FRAMES = u"{:2d}:{:02d} ({:d} frames)" LAB_FREEDB_ID = u"FreeDB disc ID" LAB_MUSICBRAINZ_ID = u"MusicBrainz disc ID" LAB_ACCURATERIP_ID = u"AccurateRip disc ID" LAB_CDINFO_LENGTH = u"Length" LAB_CDINFO_FRAMES = u"Frames" LAB_CDINFO_OFFSET = u"Offset" LAB_PLAY_BUTTON = u"play" LAB_PAUSE_BUTTON = u"pause" LAB_NEXT_BUTTON = u"next" LAB_PREVIOUS_BUTTON = u"prev" LAB_ADJUST_OUTPUT = u"output" LAB_VOLUME = u"volume" LAB_DECREASE_VOLUME = u" - volume down" LAB_INCREASE_VOLUME = u" - volume up" LAB_APPLY_BUTTON = u"apply" LAB_QUIT_BUTTON = u"quit" LAB_CANCEL_BUTTON = u"cancel" LAB_BROWSE_BUTTON = u"browse" LAB_FIELDS_BUTTON = u"fields" LAB_PLAY_STATUS = u"{count:d} tracks, {min:d}:{sec:02d} minutes" LAB_PLAY_STATUS_1 = u"{count:d} track, {min:d}:{sec:02d} minutes" LAB_PLAY_TRACK = u"track" LAB_CLOSE = u"close" LAB_TRACK = u"track" LAB_ALBUM_NUMBER = u"disc" LAB_X_OF_Y = u"{:d} / {:d}" LAB_TRACK_X_OF_Y = u"track {:2d} / {:d}" LAB_CHOOSE_FILE = u"Choose an audio file" LAB_CHOOSE_DIRECTORY = u"Choose directory" LAB_ADD_FIELD = u"Add field" LAB_COVERVIEW_ABOUT = \ u"A viewer for displaying images embedded in audio files." LAB_AUDIOTOOLS_URL = u"http://audiotools.sourceforge.net" LAB_BYTE_SIZE = u"{:d} bytes" LAB_DIMENSIONS = u"{:d} \u00D7 {:d}" LAB_BITS_PER_PIXEL = u"{:d} bits" LAB_SELECT_BEST_MATCH = u"Select Best Match" LAB_TRACK_METADATA = u"Track Metadata" LAB_DVDAINFO_TITLE = u"Title" LAB_DVDAINFO_TRACK = u"Track" LAB_DVDAINFO_LENGTH = u"Length" LAB_DVDAINFO_PTS_LENGTH = u"PTS" LAB_DVDAINFO_FIRST_SECTOR = u"Start Sector" LAB_DVDAINFO_LAST_SECTOR = u"End Sector" LAB_DVDAINFO_CODEC = u"Codec" LAB_DVDAINFO_SAMPLE_RATE = u"Rate" LAB_DVDAINFO_CHANNELS = u"Ch." LAB_DVDAINFO_BITS_PER_SAMPLE = u"BPS" LAB_DVDA2TRACK_APPLY = u"extract tracks" LAB_DVDA_TRACK = u"title {title_number:d} - track {track_number:d}" LAB_CONVERTING_FILE = u"Converting audio file" LAB_CACHING_FILE = u"Caching audio file" LAB_TRACK2TRACK_APPLY = u"convert tracks" LAB_TRACK2TRACK_APPLY_1 = u"convert track" LAB_TRACK2TRACK_NEXT = u"Next Album" LAB_TRACK2CD_CONVERTED = u"converted \"{}\" for CD burning" LAB_TRACKCAT_INPUT = u"{:d} tracks" LAB_TRACKCAT_APPLY = u"concatenate tracks" LAB_TRACKCMP_CMP = u"{file1} <> {file2}" LAB_TRACKCMP_OK = u"OK" LAB_TRACKCMP_PARAM_MISMATCH = u"stream parameters differ" LAB_TRACKCMP_MISMATCH = u"differ at PCM frame {:d}" LAB_TRACKCMP_TYPE_MISMATCH = u"must be either files or directories" LAB_TRACKCMP_ERROR = u"error" LAB_TRACKCMP_MISSING = u"\"{filename}\" missing from \"{directory}\"" LAB_TRACKCMP_RESULTS = u"Results:" LAB_TRACKCMP_HEADER_SUCCESS = u"success" LAB_TRACKCMP_HEADER_FAILURE = u"failure" LAB_TRACKCMP_HEADER_TOTAL = u"total" LAB_TRACKINFO_BITRATE = u"{bitrate:4d} kbps: {filename}" LAB_TRACKINFO_PERCENTAGE = u"{percentage:.0%}: {filename}" LAB_TRACKINFO_ATTRIBS = \ u"{minutes:2d}:{seconds:02d} " + \ u"{channels:d}ch {rate} {bits:d}-bit: {filename}" LAB_TRACKINFO_REPLAYGAIN = u"ReplayGain:" LAB_TRACKINFO_TRACK_GAIN = u"track gain" LAB_TRACKINFO_TRACK_PEAK = u"track peak" LAB_TRACKINFO_ALBUM_GAIN = u"album gain" LAB_TRACKINFO_ALBUM_PEAK = u"album peak" LAB_TRACKINFO_CUESHEET = u"Cuesheet:" LAB_TRACKINFO_CUESHEET_TRACK = u" #" LAB_TRACKINFO_CUESHEET_INDEX = u"index {:02d}" LAB_TRACKINFO_CUESHEET_LENGTH = u"length" LAB_TRACKINFO_CUESHEET_ISRC = u"ISRC" LAB_TRACKINFO_CHANNELS = u"Assigned Channels:" LAB_TRACKINFO_CHANNEL = u"channel {channel_number:d} - {channel_name}" LAB_TRACKINFO_UNDEFINED = u"undefined" LAB_TRACKLENGTH = u"{hours:d}:{minutes:02d}:{seconds:02d}" LAB_TRACKLENGTH_FILE_FORMAT = u"format" LAB_TRACKLENGTH_FILE_COUNT = u"count" LAB_TRACKLENGTH_FILE_LENGTH = u"length" LAB_TRACKLENGTH_FILE_SIZE = u"size" LAB_TRACKLENGTH_FILE_TOTAL = u"total" LAB_TRACKLINT_MESSAGE = u"* {filename}: {message}" LAB_TRACKRENAME_RENAME = u"rename files" LAB_TRACKSPLIT_APPLY = u"split track" LAB_TRACKVERIFY_RESULTS = u"Results:" LAB_TRACKVERIFY_RESULT_FORMAT = u"format" LAB_TRACKVERIFY_RESULT_SUCCESS = u"success" LAB_TRACKVERIFY_RESULT_FAILURE = u"failure" LAB_TRACKVERIFY_RESULT_TOTAL = u"total" LAB_TRACKVERIFY_ACCURATERIP_MATCH = u"match" LAB_TRACKVERIFY_ACCURATERIP_MISMATCH = u"track not found" LAB_TRACKVERIFY_ACCURATERIP_NOTFOUND = u"disc not found" LAB_TRACKVERIFY_ACCURATERIP_ERROR = u"error" LAB_TRACKVERIFY_RESULT = u"{path} : {result}" LAB_TRACKVERIFY_SUMMARY = u"summary" LAB_TRACKVERIFY_OK = u"OK" LAB_TRACKVERIFY_AR_VERSION1 = u"AccurateRip V1" LAB_TRACKVERIFY_AR_VERSION2 = u"AccurateRip V2" LAB_TRACKVERIFY_AR_TRACK = u"Track" LAB_TRACKVERIFY_AR_CHECKSUM = u"Checksum" LAB_TRACKVERIFY_AR_OFFSET = u"Offset" LAB_TRACKVERIFY_AR_CONFIDENCE = u"Confidence" LAB_TRACKVERIFY_AR_CONF = u"Conf." LAB_TRACKTAG_UPDATING = u"updating tracks" LAB_TRACKTAG_APPLY = u"Apply" LAB_KEY_NEXT = u" - next {}" LAB_KEY_PREVIOUS = u" - previous {}" LAB_KEY_SELECT = u" - select" LAB_KEY_TOGGLE_OPEN = u" - toggle open" LAB_KEY_CANCEL = u" - cancel" LAB_KEY_CLEAR_FORMAT = u" - clear format" LAB_KEY_DONE = u" - done" LAB_TRACKTAG_UPDATE_TRACK_NAME = u"the name of the track" LAB_TRACKTAG_UPDATE_ARTIST_NAME = u"the name of the artist" LAB_TRACKTAG_UPDATE_PERFORMER_NAME = u"the name of the performer" LAB_TRACKTAG_UPDATE_COMPOSER_NAME = u"the name of the composer" LAB_TRACKTAG_UPDATE_CONDUCTOR_NAME = u"the name of the conductor" LAB_TRACKTAG_UPDATE_ALBUM_NAME = u"the name of the album" LAB_TRACKTAG_UPDATE_CATALOG = u"the catalog number of the album" LAB_TRACKTAG_UPDATE_TRACK_NUMBER = u"the number of the track in the album" LAB_TRACKTAG_UPDATE_TRACK_TOTAL = \ u"the total number of tracks in the album" LAB_TRACKTAG_UPDATE_ALBUM_NUMBER = \ u"the number of the album in a set of albums" LAB_TRACKTAG_UPDATE_ALBUM_TOTAL = \ u"the total number of albums in a set of albums" LAB_TRACKTAG_UPDATE_ISRC = u"the ISRC of the track" LAB_TRACKTAG_UPDATE_PUBLISHER = u"the publisher of the album" LAB_TRACKTAG_UPDATE_MEDIA = u"the media type of the album, such as \"CD\"" LAB_TRACKTAG_UPDATE_YEAR = u"the year of release" LAB_TRACKTAG_UPDATE_DATE = u"the date of recording" LAB_TRACKTAG_UPDATE_COPYRIGHT = u"copyright information" LAB_TRACKTAG_UPDATE_COMMENT = u"a text comment" LAB_TRACKTAG_UPDATE_COMPILATION = u"whether the track is part of a compilation" LAB_TRACKTAG_REMOVE_TRACK_NAME = u"remove track name" LAB_TRACKTAG_REMOVE_ARTIST_NAME = u"remove track artist" LAB_TRACKTAG_REMOVE_PERFORMER_NAME = u"remove track performer" LAB_TRACKTAG_REMOVE_COMPOSER_NAME = u"remove track composer" LAB_TRACKTAG_REMOVE_CONDUCTOR_NAME = u"remove track conductor" LAB_TRACKTAG_REMOVE_ALBUM_NAME = u"remove album name" LAB_TRACKTAG_REMOVE_CATALOG = u"remove catalog number" LAB_TRACKTAG_REMOVE_TRACK_NUMBER = u"remove track number" LAB_TRACKTAG_REMOVE_TRACK_TOTAL = u"remove total number of tracks" LAB_TRACKTAG_REMOVE_ALBUM_NUMBER = u"remove album number" LAB_TRACKTAG_REMOVE_ALBUM_TOTAL = u"remove total number of albums" LAB_TRACKTAG_REMOVE_ISRC = u"remove ISRC" LAB_TRACKTAG_REMOVE_PUBLISHER = u"remove publisher" LAB_TRACKTAG_REMOVE_MEDIA = u"remove album's media type" LAB_TRACKTAG_REMOVE_YEAR = u"remove release year" LAB_TRACKTAG_REMOVE_DATE = u"remove recording date" LAB_TRACKTAG_REMOVE_COPYRIGHT = u"remove copyright information" LAB_TRACKTAG_REMOVE_COMMENT = u"remove text comment" LAB_TRACKTAG_REMOVE_COMPILATION = u"remove compilation status" LAB_AT_CONFIG_CD_BURNING = u"CD Burning via track2cd" LAB_AT_CONFIG_WITHOUT_CUE = u"without cue" LAB_AT_CONFIG_WITH_CUE = u"with cue" LAB_AT_CONFIG_YES = u"yes" LAB_AT_CONFIG_NO = u"no" LAB_AT_CONFIG_SYS_CONFIG = u"System configuration:" LAB_AT_CONFIG_USE_MUSICBRAINZ = u"Use MusicBrainz service" LAB_AT_CONFIG_MUSICBRAINZ_SERVER = u"Default MusicBrainz server" LAB_AT_CONFIG_MUSICBRAINZ_PORT = u"Default MusicBrainz port" LAB_AT_CONFIG_USE_FREEDB = u"Use FreeDB service" LAB_AT_CONFIG_FREEDB_SERVER = u"Default FreeDB server" LAB_AT_CONFIG_FREEDB_PORT = u"Default FreeDB port" LAB_AT_CONFIG_DEFAULT_CDROM = u"Default CD-ROM device" LAB_AT_CONFIG_CDROM_READ_OFFSET = u"CD-ROM sample read offset" LAB_AT_CONFIG_CDROM_WRITE_OFFSET = u"CD-ROM sample write offset" LAB_AT_CONFIG_JOBS = u"Default simultaneous jobs" LAB_AT_CONFIG_VERBOSITY = u"Default verbosity level" LAB_AT_CONFIG_AUDIO_OUTPUT = u"Audio output" LAB_AT_CONFIG_FS_ENCODING = u"Filesystem text encoding" LAB_AT_CONFIG_IO_ENCODING = u"TTY text encoding" LAB_AT_CONFIG_ID3V2_VERSION = u"ID3v2 tag version" LAB_AT_CONFIG_ID3V2_ID3V22 = u"ID3v2.2" LAB_AT_CONFIG_ID3V2_ID3V23 = u"ID3v2.3" LAB_AT_CONFIG_ID3V2_ID3V24 = u"ID3v2.4" LAB_AT_CONFIG_ID3V2_NONE = u"no ID3v2 tags" LAB_AT_CONFIG_ID3V2_PADDING = u"ID3v2 digit padding" LAB_AT_CONFIG_ID3V2_PADDING_YES = u"padded (\"01\", \"02\", \u2026)" LAB_AT_CONFIG_ID3V2_PADDING_NO = u"not padded (\"1\", \"2\", \u2026)" LAB_AT_CONFIG_ID3V1_VERSION = u"ID3v1 tag version" LAB_AT_CONFIG_ID3V1_ID3V11 = u"ID3v1.1" LAB_AT_CONFIG_ID3V1_NONE = u"no ID3v1 tags" LAB_AT_CONFIG_ADD_REPLAY_GAIN = u"Add ReplayGain by default" LAB_AT_CONFIG_FILE_WRITTEN = u"* \"{}\" written" LAB_AT_CONFIG_FOUND = u"found" LAB_AT_CONFIG_NOT_FOUND = u"not found" LAB_AT_CONFIG_TYPE = u" type " LAB_AT_CONFIG_BINARIES = u"Binaries" LAB_AT_CONFIG_QUALITY = u" quality " LAB_AT_CONFIG_REPLAY_GAIN = u" ReplayGain " LAB_AT_CONFIG_DEFAULT = u"Default" LAB_AT_CONFIG_TYPE = u"Type" LAB_AT_CONFIG_DEFAULT_QUALITY = u"Default Quality" LAB_OUTPUT_OPTIONS = u"Output Options" LAB_OPTIONS_OUTPUT = u"Output" LAB_OPTIONS_OUTPUT_DIRECTORY = u"Dir" LAB_OPTIONS_FILENAME_FORMAT = u"Format" LAB_OPTIONS_FILENAME_FORMAT_EXAMPLE = u"Example" LAB_OPTIONS_AUDIO_CLASS = u"Type" LAB_OPTIONS_AUDIO_QUALITY = u"Quality" LAB_OPTIONS_OUTPUT_FILES = u"Output Files" LAB_OPTIONS_OUTPUT_FILES_1 = u"Output File" # Compression settings COMP_FLAC_0 = u"least compresson, fastest compression speed" COMP_FLAC_8 = u"most compression, slowest compression speed" COMP_NERO_LOW = u"lowest quality, corresponds to neroAacEnc -q 0.4" COMP_NERO_HIGH = u"highest quality, corresponds to neroAacEnc -q 1" COMP_LAME_0 = u"high quality, larger files, corresponds to lame's -V0" COMP_LAME_6 = u"lower quality, smaller files, corresponds to lame's -V6" COMP_LAME_MEDIUM = u"corresponds to lame's --preset medium" COMP_LAME_STANDARD = u"corresponds to lame's --preset standard" COMP_LAME_EXTREME = u"corresponds to lame's --preset extreme" COMP_LAME_INSANE = u"corresponds to lame's --preset insane" COMP_TWOLAME_64 = u"total bitrate of 64kbps" COMP_TWOLAME_384 = u"total bitrate of 384kbps" COMP_VORBIS_0 = u"very low quality, corresponds to oggenc -q 0" COMP_VORBIS_10 = u"very high quality, corresponds to oggenc -q 10" COMP_WAVPACK_FAST = u"fastest encode/decode, worst compression" COMP_WAVPACK_VERYHIGH = u"slowest encode/decode, best compression" COMP_SPEEX_0 = u"corresponds to speexenc --quality 0" COMP_SPEEX_10 = u"corresponds to speexenc --quality 10" # Errors ERR_1_FILE_REQUIRED = u"you must specify exactly 1 supported audio file" ERR_FILES_REQUIRED = u"you must specify at least 1 supported audio file" ERR_UNSUPPORTED_CHANNEL_MASK = \ u"unable to write \"{target_filename}\" " + \ u"with channel assignment \"{assignment}\"" ERR_UNSUPPORTED_BITS_PER_SAMPLE = \ u"unable to write \"{target_filename}\" " + \ u"with {bps:d} bits per sample" ERR_UNSUPPORTED_CHANNEL_COUNT = \ u"unable to write \"{target_filename}\" " + \ u"with {channels:d} channel input" ERR_DUPLICATE_FILE = u"file \"{}\" included more than once" ERR_OUTPUT_IS_INPUT = u"\"{}\" cannot be both input and output file" ERR_OPEN_IOERROR = u"unable to open \"{}\"" ERR_ENCODING_ERROR = u"unable to write \"{}\"" ERR_READ_ERROR = u"read error" ERR_UNSUPPORTED_AUDIO_TYPE = u"unsupported audio type \"{}\"" ERR_UNSUPPORTED_FILE = u"unsupported file '{}'" ERR_UNSUPPORTED_TO_PCM = \ u"\"{filename}\": unable to read file type \"{type}\"" ERR_UNSUPPORTED_FROM_PCM = \ u"unable to encode to file type \"{}\"" ERR_INVALID_FILE = u"invalid file '{}'" ERR_INVALID_SAMPLE_RATE = u"invalid sample rate" ERR_INVALID_CHANNEL_COUNT = u"invalid channel count" ERR_INVALID_BITS_PER_SAMPLE = u"invalid bits-per-sample" ERR_TOTAL_PCM_FRAMES_MISMATCH = u"total_pcm_frames mismatch" ERR_AMBIGUOUS_AUDIO_TYPE = u"ambiguous suffix type \"{}\"" ERR_CHANNEL_COUNT_MASK_MISMATCH = u"channel count and channel mask mismatch" ERR_NO_PCMREADERS = u"you must have at least 1 PCMReader" ERR_PICTURES_UNSUPPORTED = u"this MetaData type does not support images" ERR_UNKNOWN_FIELD = u"unknown field \"{}\" in file format" ERR_INVALID_FILENAME_FORMAT = u"invalid filename format string" ERR_FOREIGN_METADATA = u"metadata not from audio file" ERR_NEGATIVE_SEEK = u"cannot seek to negative value" ERR_AIFF_NOT_AIFF = u"not an AIFF file" ERR_AIFF_INVALID_AIFF = u"invalid AIFF file" ERR_AIFF_INVALID_CHUNK_ID = u"invalid AIFF chunk ID" ERR_AIFF_INVALID_CHUNK = u"invalid AIFF chunk" ERR_AIFF_MULTIPLE_COMM_CHUNKS = u"multiple COMM chunks found" ERR_AIFF_PREMATURE_SSND_CHUNK = u"SSND chunk found before fmt" ERR_AIFF_MULTIPLE_SSND_CHUNKS = u"multiple SSND chunks found" ERR_AIFF_NO_COMM_CHUNK = u"COMM chunk not found" ERR_AIFF_NO_SSND_CHUNK = u"SSND chunk not found" ERR_AIFF_HEADER_EXTRA_SSND = u"extra data after SSND chunk header" ERR_AIFF_HEADER_MISSING_SSND = u"missing data in SSND chunk header" ERR_AIFF_HEADER_IOERROR = u"I/O error reading header data" ERR_AIFF_FOOTER_IOERROR = u"I/O error reading footer data" ERR_AIFF_TRUNCATED_SSND_CHUNK = u"premature end of SSND chunk" ERR_AIFF_INVALID_SIZE = u"total aiff file size mismatch" ERR_APE_INVALID_HEADER = u"invalid Monkey's Audio header" ERR_AU_INVALID_HEADER = u"invalid Sun AU header" ERR_AU_UNSUPPORTED_FORMAT = u"unsupported Sun AU format" ERR_AU_TRUNCATED_DATA = u"truncated data block" ERR_CUE_SYNTAX_ERROR = u"syntax error at line {:d}" ERR_CUE_IOERROR = u"unable to read cuesheet" ERR_CUE_INVALID_FORMAT = u"cuesheet not formatted for disc images" ERR_CUE_INSUFFICIENT_TRACKS = u"insufficient tracks in cuesheet" ERR_CUE_LENGTH_MISMATCH = \ u"cuesheet track length mismatch in track {:d}" ERR_DVDA_IOERROR_AUDIO_TS = u"unable to open AUDIO_TS.IFO" ERR_DVDA_INVALID_TITLE = u"invalid title" ERR_DVDA_INVALID_TRACK = u"invalid track" ERR_DVDA_INVALID_AUDIO_TS = u"invalid AUDIO_TS.IFO" ERR_DVDA_INVALID_SECTOR_POINTER = u"invalid sector pointer" ERR_DVDA_NO_TRACK_SECTOR = u"unable to find track sector in AOB files" ERR_DVDA_INVALID_AOB_SYNC = u"invalid AOB sync bytes" ERR_DVDA_INVALID_AOB_MARKER = u"invalid AOB marker bits" ERR_DVDA_INVALID_AOB_START = u"invalid AOB packet start code" ERR_FLAC_RESERVED_BLOCK = u"reserved metadata block type {:d}" ERR_FLAC_INVALID_BLOCK = u"invalid metadata block type" ERR_FLAC_INVALID_FILE = u"Invalid FLAC file" ERR_OGG_INVALID_MAGIC_NUMBER = u"invalid Ogg magic number" ERR_OGG_INVALID_VERSION = u"invalid Ogg version" ERR_OGG_CHECKSUM_MISMATCH = u"Ogg page checksum mismatch" ERR_OGGFLAC_INVALID_PACKET_BYTE = u"invalid packet byte" ERR_OGGFLAC_INVALID_OGG_SIGNATURE = u"invalid Ogg signature" ERR_OGGFLAC_INVALID_MAJOR_VERSION = u"invalid major version" ERR_OGGFLAC_INVALID_MINOR_VERSION = u"invalid minor version" ERR_OGGFLAC_VALID_FLAC_SIGNATURE = u"invalid FLAC signature" ERR_IMAGE_UNKNOWN_TYPE = u"unknown image type" ERR_IMAGE_INVALID_JPEG_MARKER = u"invalid JPEG segment marker" ERR_IMAGE_IOERROR_JPEG = "I/O error reading JPEG data" ERR_IMAGE_INVALID_PNG = u"invalid PNG" ERR_IMAGE_IOERROR_PNG = "I/O error reading PNG data" ERR_IMAGE_INVALID_PLTE = u"invalid PLTE chunk length" ERR_IMAGE_INVALID_BMP = u"invalid BMP" ERR_IMAGE_IOERROR_BMP = "I/O error reading BMP data" ERR_IMAGE_INVALID_TIFF = u"invalid TIFF" ERR_IMAGE_IOERROR_TIFF = u"I/O error reading TIFF data" ERR_IMAGE_INVALID_GIF = u"invalid GIF" ERR_IMAGE_IOERROR_GIF = u"I/O error reading GIF data" ERR_M4A_IOERROR = u"I/O error opening M4A file" ERR_M4A_MISSING_MDIA = u"required mdia atom not found" ERR_M4A_MISSING_STSD = u"required stsd atom not found" ERR_M4A_INVALID_MP4A = u"invalid mp4a atom" ERR_M4A_MISSING_MDHD = u"required mdhd atom not found" ERR_M4A_UNSUPPORTED_MDHD = u"unsupported mdhd version" ERR_M4A_INVALID_MDHD = u"invalid mdhd atom" ERR_M4A_INVALID_LEAF_ATOMS = u"leaf atoms must be a list" ERR_ALAC_IOERROR = u"I/O error opening ALAC file" ERR_ALAC_INVALID_ALAC = u"invalid alac atom" ERR_MP3_FRAME_NOT_FOUND = u"MP3 frame not found" ERR_MP3_INVALID_SAMPLE_RATE = u"invalid sample rate" ERR_MP3_INVALID_BIT_RATE = u"invalid bit rate" ERR_TOC_NO_HEADER = u"no CD_DA TOC header found" ERR_TTA_INVALID_SIGNATURE = u"invalid TTA signature" ERR_TTA_INVALID_FORMAT = u"unsupported TTA format" ERR_VORBIS_INVALID_TYPE = u"invalid Vorbis type" ERR_VORBIS_INVALID_HEADER = u"invalid Vorbis header" ERR_VORBIS_INVALID_VERSION = u"invalid Vorbis version" ERR_VORBIS_INVALID_FRAMING_BIT = u"invalid framing bit" ERR_OPUS_INVALID_TYPE = u"invalid Opus header" ERR_OPUS_INVALID_VERSION = u"invalid Opus version" ERR_OPUS_INVALID_CHANNELS = u"invalid Open channel count" ERR_WAV_NOT_WAVE = u"not a RIFF WAVE file" ERR_WAV_INVALID_WAVE = u"invalid RIFF WAVE file" ERR_WAV_NO_DATA_CHUNK = u"data chunk not found" ERR_WAV_INVALID_CHUNK = u"invalid RIFF WAVE chunk ID" ERR_WAV_MULTIPLE_FMT = u"multiple fmt chunks found" ERR_WAV_PREMATURE_DATA = u"data chunk found before fmt" ERR_WAV_MULTIPLE_DATA = u"multiple data chunks found" ERR_WAV_NO_FMT_CHUNK = u"fmt chunk not found" ERR_WAV_HEADER_EXTRA_DATA = u"{:d} bytes found after data chunk header" ERR_WAV_HEADER_IOERROR = u"I/O error reading header data" ERR_WAV_FOOTER_IOERROR = u"I/O error reading footer data" ERR_WAV_TRUNCATED_DATA_CHUNK = u"premature end of data chunk" ERR_WAV_INVALID_SIZE = u"total wave file size mismatch" ERR_WAVPACK_INVALID_HEADER = u"WavPack header ID invalid" ERR_WAVPACK_UNSUPPORTED_FMT = u"unsupported FMT compression" ERR_WAVPACK_INVALID_FMT = u"invalid FMT chunk" ERR_WAVPACK_NO_FMT = u"FMT chunk not found in WavPack" ERR_MPC_INVALID_ID = u"invalid Musepack stream ID" ERR_MPC_INVALID_VERSION = u"invalid Musepack version" ERR_NO_COMPRESSION_MODES = u"Audio type \"{}\" has no quality modes" ERR_UNSUPPORTED_COMPRESSION_MODE = \ u"\"{quality}\" is not a supported compression mode " + \ u"for type \"{type}\"" ERR_INVALID_CDDA = u". Is that an audio cd?" ERR_NO_CDDA = u"no CD in drive" ERR_NO_EMPTY_CDDA = u"no audio tracks found on CD" ERR_NO_OUTPUT_FILE = u"you must specify an output file" ERR_DUPLICATE_OUTPUT_FILE = u"output file \"{}\" occurs more than once" ERR_URWID_REQUIRED = u"Urwid 1.0 or better is required for interactive mode" ERR_GET_URWID1 = \ u"Please download and install urwid from http://excess.org/urwid/" ERR_GET_URWID2 = u"or your system's package manager." ERR_TERMIOS_ERROR = u"unable to get tty settings" ERR_TERMIOS_SUGGESTION = \ u"if piping arguments via xargs(1), try:" ERR_NO_GUI = u"neither PyGTK nor Tkinter is available" ERR_NO_AUDIO_TS = \ u"you must specify the DVD-Audio's AUDIO_TS directory with -A" ERR_INVALID_TITLE_NUMBER = u"title number must be greater than 0" ERR_INVALID_JOINT = u"you must run at least 1 process at a time" ERR_NO_CDRDAO = u"unable to find \"cdrdao\" executable" ERR_GET_CDRDAO = u"please install \"cdrdao\" to burn CDs" ERR_NO_CDRECORD = u"unable to find \"cdrecord\" executable" ERR_GET_CDRECORD = u"please install \"cdrecord\" to burn CDs" ERR_SAMPLE_RATE_MISMATCH = u"all audio files must have the same sample rate" ERR_CHANNEL_COUNT_MISMATCH = \ u"all audio files must have the same channel count" ERR_CHANNEL_MASK_MISMATCH = \ u"all audio files must have the same channel assignment" ERR_BPS_MISMATCH = u"all audio files must have the same bits per sample" ERR_TRACK2CD_INVALIDFILE = u"not all files are valid. Unable to write CD" ERR_TRACK2TRACK_O_AND_D = u"-o and -d options are not compatible" ERR_TRACK2TRACK_O_AND_D_SUGGESTION = \ u"please specify either -o or -d but not both" ERR_TRACK2TRACK_O_AND_FORMAT = u"--format has no effect when used with -o" ERR_TRACK2TRACK_O_AND_MULTIPLE = \ u"you may specify only 1 input file for use with -o" ERR_TRACKCMP_TYPE_MISMATCH = u"both files to be compared must be audio files" ERR_TRACKSPLIT_NO_CUESHEET = u"you must specify a cuesheet to split audio file" ERR_TRACKSPLIT_OVERLONG_CUESHEET = u"cuesheet too long for track being split" ERR_TRACKVERIFY = u"not from a CD" ERR_RENAME = u"unable to rename \"{source}\" to \"{target}\"" ERR_TRACKTAG_COMMENT_NOT_UTF8 = \ u"comment file \"{}\" does not appear to be UTF-8 text" ERR_TRACKTAG_COMMENT_IOERROR = u"unable to open comment file \"{}\"" ERR_OUTPUT_DUPLICATE_NAME = u"all output tracks must have different names" ERR_OUTPUT_OUTPUTS_ARE_INPUT = \ u"output tracks must have different names than input tracks" ERR_OUTPUT_INVALID_FORMAT = u"output tracks must have valid format string" ERR_CANCELLED = u"cancelled" ERR_TOO_MANY_CUESHEET_FILES = u"too many files for cuesheet" # Cleaning messages CLEAN_REMOVE_DUPLICATE_TAG = u"removed duplicate tag {}" CLEAN_REMOVE_TRAILING_WHITESPACE = \ u"removed trailing whitespace from {}" CLEAN_REMOVE_LEADING_WHITESPACE = u"removed leading whitespace from {}" CLEAN_REMOVE_LEADING_WHITESPACE_ZEROES = \ u"removed leading whitespace/zeroes from {}" CLEAN_REMOVE_LEADING_ZEROES = u"removed leading zeroes from {}" CLEAN_REMOVE_DUPLICATE_ID3V2 = u"remove duplicate ID3v2 tag" CLEAN_ADD_LEADING_ZEROES = u"added leading zeroes to {}" CLEAN_REMOVE_EMPTY_TAG = u"removed empty field {}" CLEAN_FIX_TAG_FORMATTING = u"fixed formatting for {}" CLEAN_FIX_IMAGE_FIELDS = u"fixed embedded image metadata fields" CLEAN_AIFF_MULTIPLE_COMM_CHUNKS = u"removed duplicate COMM chunk" CLEAN_AIFF_REORDERED_SSND_CHUNK = u"moved COMM chunk after SSND chunk" CLEAN_AIFF_MULTIPLE_SSND_CHUNKS = u"removed duplicate SSND chunk" CLEAN_FLAC_REORDERED_STREAMINFO = u"moved STREAMINFO to first block" CLEAN_FLAC_MULITPLE_STREAMINFO = u"removed redundant STREAMINFO block" CLEAN_FLAC_MULTIPLE_VORBISCOMMENT = u"removed redundant VORBIS_COMMENT block" CLEAN_FLAC_MULTIPLE_SEEKTABLE = u"removed redundant SEEKTABLE block" CLEAN_FLAC_MULTIPLE_CUESHEET = u"removed redundant CUESHEET block" CLEAN_FLAC_UNDEFINED_BLOCK = u"removed undefined block" CLEAN_FLAC_REMOVE_SEEKPOINTS = u"removed empty seekpoints from seektable" CLEAN_FLAC_REORDER_SEEKPOINTS = u"reordered seektable to be in ascending order" CLEAN_FLAC_REMOVE_ID3V2 = u"removed ID3v2 tag" CLEAN_FLAC_REMOVE_ID3V1 = u"removed ID3v1 tag" CLEAN_FLAC_POPULATE_MD5 = u"populated empty MD5SUM" CLEAN_FLAC_ADD_CHANNELMASK = u"added WAVEFORMATEXTENSIBLE_CHANNEL_MASK" CLEAN_FLAC_FIX_SEEKTABLE = u"fixed invalid SEEKTABLE" CLEAN_FLAC_ADD_SEEKTABLE = u"added SEEKTABLE" CLEAN_WAV_MULTIPLE_FMT_CHUNKS = u"removed duplicate fmt chunk" CLEAN_WAV_REORDERED_DATA_CHUNK = u"moved data chunk after fmt chunk" CLEAN_WAV_MULTIPLE_DATA_CHUNKS = u"removed multiple data chunk" # Channel names MASK_FRONT_LEFT = u"front left" MASK_FRONT_RIGHT = u"front right" MASK_FRONT_CENTER = u"front center" MASK_LFE = u"low frequency" MASK_BACK_LEFT = u"back left" MASK_BACK_RIGHT = u"back right" MASK_FRONT_RIGHT_OF_CENTER = u"front right of center" MASK_FRONT_LEFT_OF_CENTER = u"front left of center" MASK_BACK_CENTER = u"back center" MASK_SIDE_LEFT = u"side left" MASK_SIDE_RIGHT = u"side right" MASK_TOP_CENTER = u"top center" MASK_TOP_FRONT_LEFT = u"top front left" MASK_TOP_FRONT_CENTER = u"top front center" MASK_TOP_FRONT_RIGHT = u"top front right" MASK_TOP_BACK_LEFT = u"top back left" MASK_TOP_BACK_CENTER = u"top back center" MASK_TOP_BACK_RIGHT = u"top back right" ================================================ FILE: audiotools/toc/__init__.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA """the TOC file module""" from audiotools import Sheet, SheetTrack, SheetIndex, SheetException class TOCFile(Sheet): def __init__(self, type, tracks, catalog=None, cd_text=None): from sys import version_info str_type = str if (version_info[0] >= 3) else unicode assert(isinstance(type, str_type)) assert((catalog is None) or isinstance(catalog, str_type)) assert((cd_text is None) or isinstance(cd_text, CDText)) self.__type__ = type for (i, t) in enumerate(tracks, 1): t.__number__ = i self.__tracks__ = tracks self.__catalog__ = catalog self.__cd_text__ = cd_text def __repr__(self): return "TOCFile({})".format( ", ".join(["{}={!r}".format(attr, getattr(self, "__" + attr + "__")) for attr in ["type", "tracks", "catalog", "cd_text"]])) @classmethod def converted(cls, sheet, filename=None): """given a Sheet object, returns a TOCFile object""" tracks = list(sheet) metadata = sheet.get_metadata() if metadata is not None: if metadata.catalog is not None: catalog = metadata.catalog else: catalog = None cd_text = CDText.from_disc_metadata(metadata) else: catalog = None cd_text = None return cls(type=u"CD_DA", tracks=[TOCTrack.converted(sheettrack=track, next_sheettrack=next_track, filename=filename) for (track, next_track) in zip(tracks, tracks[1:] + [None])], catalog=catalog, cd_text=cd_text) def __len__(self): return len(self.__tracks__) def __getitem__(self, index): return self.__tracks__[index] def build(self): """returns the TOCFile as a string""" output = [self.__type__, u""] if self.__catalog__ is not None: output.extend([ u"CATALOG {}".format(format_string(self.__catalog__)), u""]) if self.__cd_text__ is not None: output.append(self.__cd_text__.build()) output.extend([track.build() for track in self.__tracks__]) return u"\n".join(output) + u"\n" def get_metadata(self): """returns MetaData of Sheet, or None this metadata often contains information such as catalog number or CD-TEXT values""" from audiotools import MetaData if (self.__catalog__ is not None) and (self.__cd_text__ is not None): metadata = self.__cd_text__.to_disc_metadata() metadata.catalog = self.__catalog__ return metadata elif self.__catalog__ is not None: return MetaData(catalog=self.__catalog__) elif self.__cd_text__ is not None: return self.__cd_text__.to_disc_metadata() else: return None class TOCTrack(SheetTrack): def __init__(self, mode, flags, sub_channel_mode=None): from audiotools import SheetIndex from sys import version_info str_type = str if (version_info[0] >= 3) else unicode assert(isinstance(mode, str_type)) assert((sub_channel_mode is None) or isinstance(sub_channel_mode, str_type)) self.__number__ = None # to be filled-in later self.__mode__ = mode self.__sub_channel_mode__ = sub_channel_mode self.__flags__ = flags indexes = [] pre_gap = None file_start = None file_length = None for flag in flags: if isinstance(flag, TOCFlag_FILE): file_start = flag.start() file_length = flag.length() elif isinstance(flag, TOCFlag_START): if flag.start() is not None: pre_gap = flag.start() else: pre_gap = file_length elif isinstance(flag, TOCFlag_INDEX): indexes.append(flag.index()) if pre_gap is None: # first index point is 1 self.__indexes__ = ([SheetIndex(number=1, offset=file_start)] + [SheetIndex(number=i, offset=index) for (i, index) in enumerate(indexes, 2)]) else: # first index point is 0 self.__indexes__ = ([SheetIndex(number=0, offset=file_start), SheetIndex(number=1, offset=file_start + pre_gap)] + [SheetIndex(number=i, offset=index) for (i, index) in enumerate(indexes, 2)]) @classmethod def converted(cls, sheettrack, next_sheettrack, filename=None): """given a SheetTrack object, returns a TOCTrack object""" metadata = sheettrack.get_metadata() flags = [] if metadata is not None: if metadata.ISRC is not None: flags.append(TOCFlag_ISRC(metadata.ISRC)) cdtext = CDText.from_track_metadata(metadata) if cdtext is not None: flags.append(cdtext) if sheettrack.copy_permitted(): flags.append(TOCFlag_COPY(True)) if sheettrack.pre_emphasis(): flags.append(TOCFlag_PRE_EMPHASIS(True)) if len(sheettrack) > 0: if ((next_sheettrack is not None) and (sheettrack.filename() == next_sheettrack.filename())): length = (next_sheettrack[0].offset() - sheettrack[0].offset()) else: length = None flags.append(TOCFlag_FILE( type=u"AUDIOFILE", filename=(filename if filename is not None else sheettrack.filename()), start=sheettrack[0].offset(), length=length)) if sheettrack[0].number() == 0: # first index point is 0 so track contains pre-gap flags.append(TOCFlag_START(sheettrack[1].offset() - sheettrack[0].offset())) for index in sheettrack[2:]: flags.append(TOCFlag_INDEX(index.offset())) else: # track contains no pre-gap for index in sheettrack[1:]: flags.append(TOCFlag_INDEX(index.offset())) return cls(mode=(u"AUDIO" if sheettrack.is_audio() else u"MODE1"), flags=flags) def first_flag(self, flag_class): """returns the first flag in the list with the given class or None if not found""" for flag in self.__flags__: if isinstance(flag, flag_class): return flag else: return None def all_flags(self, flag_class): """returns a list of all flags in the list with the given class""" return [f for f in self.__flags__ if isinstance(f, flag_class)] def __repr__(self): return "TOCTrack({})".format( ", ".join(["{}={!r}".format(attr, getattr(self, "__" + attr + "__")) for attr in ["number", "mode", "sub_channel_mode", "flags"]])) def __len__(self): return len(self.__indexes__) def __getitem__(self, index): return self.__indexes__[index] def number(self): """returns track's number as an integer""" return self.__number__ def get_metadata(self): """returns SheetTrack's MetaData, or None""" from audiotools import MetaData isrc = self.first_flag(TOCFlag_ISRC) cd_text = self.first_flag(CDText) if (isrc is not None) and (cd_text is not None): metadata = cd_text.to_track_metadata() metadata.ISRC = isrc.isrc() return metadata elif cd_text is not None: return cd_text.to_track_metadata() elif isrc is not None: return MetaData(ISRC=isrc.isrc()) else: return None def filename(self): """returns SheetTrack's filename as a string""" filename = self.first_flag(TOCFlag_FILE) if filename is not None: return filename.filename() else: return u"" def is_audio(self): """returns True if track contains audio data""" return self.__mode__ == u"AUDIO" def pre_emphasis(self): """returns whether SheetTrack has pre-emphasis""" pre_emphasis = self.first_flag(TOCFlag_PRE_EMPHASIS) if pre_emphasis is not None: return pre_emphasis.pre_emphasis() else: return False def copy_permitted(self): """returns whether copying is permitted""" copy = self.first_flag(TOCFlag_COPY) if copy is not None: return copy.copy() else: return False def build(self): """returns the TOCTrack as a string""" output = [(u"TRACK {}".format(self.__mode__) if (self.__sub_channel_mode__ is None) else u"TRACK {} {}".format(self.__mode__, self.__sub_channel_mode__))] output.extend([flag.build() for flag in self.__flags__]) output.append(u"") return u"\n".join(output) class TOCFlag(object): def __init__(self, attrs): self.__attrs__ = attrs def __repr__(self): return "{}({})".format( self.__class__.__name__, ", ".join(["{}={!r}".format(attr, getattr(self, "__" + attr + "__")) for attr in self.__attrs__])) def build(self): """returns the TOCTracFlag as a string""" # implement this in TOCFlag subclasses raise NotImplementedError() class TOCFlag_COPY(TOCFlag): def __init__(self, copy): TOCFlag.__init__(self, ["copy"]) assert(isinstance(copy, bool)) self.__copy__ = copy def copy(self): return self.__copy__ def build(self): return u"COPY" if self.__copy__ else u"NO COPY" class TOCFlag_PRE_EMPHASIS(TOCFlag): def __init__(self, pre_emphasis): TOCFlag.__init__(self, ["pre_emphasis"]) assert(isinstance(pre_emphasis, bool)) self.__pre_emphasis__ = pre_emphasis def pre_emphasis(self): return self.__pre_emphasis__ def build(self): return u"PRE_EMPHASIS" if self.__pre_emphasis__ else u"NO PRE_EMPHASIS" class TOCFlag_CHANNELS(TOCFlag): def __init__(self, channels): TOCFlag.__init__(self, ["channels"]) assert((channels == 2) or (channels == 4)) self.__channels__ = channels def build(self): return (u"TWO_CHANNEL_AUDIO" if (self.__channels__ == 2) else u"FOUR_CHANNEL_AUDIO") class TOCFlag_ISRC(TOCFlag): def __init__(self, isrc): TOCFlag.__init__(self, ["isrc"]) from sys import version_info str_type = str if (version_info[0] >= 3) else unicode assert(isinstance(isrc, str_type)) self.__isrc__ = isrc def isrc(self): return self.__isrc__ def build(self): return u"ISRC {}".format(format_string(self.__isrc__)) class TOCFlag_FILE(TOCFlag): def __init__(self, type, filename, start, length=None): TOCFlag.__init__(self, ["type", "filename", "start", "length"]) from sys import version_info from fractions import Fraction str_type = str if (version_info[0] >= 3) else unicode assert(isinstance(type, str_type)) assert(isinstance(filename, str_type)) assert(isinstance(start, Fraction)) assert((length is None) or isinstance(length, Fraction)) self.__type__ = type self.__filename__ = filename self.__start__ = start self.__length__ = length def filename(self): return self.__filename__ def start(self): return self.__start__ def length(self): return self.__length__ def build(self): if self.__length__ is None: return u"{} {} {}".format(self.__type__, format_string(self.__filename__), format_timestamp(self.__start__)) else: return u"{} {} {} {}".format(self.__type__, format_string(self.__filename__), format_timestamp(self.__start__), format_timestamp(self.__length__)) class TOCFlag_START(TOCFlag): def __init__(self, start=None): TOCFlag.__init__(self, ["start"]) from fractions import Fraction assert((start is None) or isinstance(start, Fraction)) self.__start__ = start def start(self): return self.__start__ def build(self): if self.__start__ is None: return u"START" else: return u"START {}".format(format_timestamp(self.__start__)) class TOCFlag_INDEX(TOCFlag): def __init__(self, index): TOCFlag.__init__(self, ["index"]) from fractions import Fraction assert(isinstance(index, Fraction)) self.__index__ = index def index(self): return self.__index__ def build(self): return u"INDEX {}".format(format_timestamp(self.__index__)) class CDText(object): def __init__(self, languages, language_map=None): self.__languages__ = languages self.__language_map__ = language_map def __repr__(self): return "CDText(languages={!r}, language_map={!r})".format( self.__languages__, self.__language_map__) def get(self, key, default): for language in self.__languages__: try: return language[key] except KeyError: pass else: return default def build(self): output = [u"CD_TEXT {"] if self.__language_map__ is not None: output.append(self.__language_map__.build()) output.append(u"") output.extend([language.build() for language in self.__languages__]) output.append(u"}") return u"\n".join(output) def to_disc_metadata(self): from audiotools import MetaData return MetaData( album_name=self.get(u"TITLE", None), performer_name=self.get(u"PERFORMER", None), artist_name=self.get(u"SONGWRITER", None), composer_name=self.get(u"COMPOSER", None), comment=self.get(u"MESSAGE", None)) @classmethod def from_disc_metadata(cls, metadata): text_pairs = [] if metadata is not None: if metadata.album_name is not None: text_pairs.append((u"TITLE", metadata.album_name)) if metadata.performer_name is not None: text_pairs.append((u"PERFORMER", metadata.performer_name)) if metadata.artist_name is not None: text_pairs.append((u"SONGWRITER", metadata.artist_name)) if metadata.composer_name is not None: text_pairs.append((u"COMPOSER", metadata.composer_name)) if metadata.comment is not None: text_pairs.append((u"MESSAGE", metadata.comment)) if len(text_pairs) > 0: return cls(languages=[CDTextLanguage(language_id=0, text_pairs=text_pairs)], language_map=CDTextLanguageMap([(0, u"EN")])) else: return None def to_track_metadata(self): from audiotools import MetaData return MetaData( track_name=self.get(u"TITLE", None), performer_name=self.get(u"PERFORMER", None), artist_name=self.get(u"SONGWRITER", None), composer_name=self.get(u"COMPOSER", None), comment=self.get(u"MESSAGE", None), ISRC=self.get(u"ISRC", None)) @classmethod def from_track_metadata(cls, metadata): text_pairs = [] if metadata is not None: if metadata.track_name is not None: text_pairs.append((u"TITLE", metadata.track_name)) if metadata.performer_name is not None: text_pairs.append((u"PERFORMER", metadata.performer_name)) if metadata.artist_name is not None: text_pairs.append((u"SONGWRITER", metadata.artist_name)) if metadata.composer_name is not None: text_pairs.append((u"COMPOSER", metadata.composer_name)) if metadata.comment is not None: text_pairs.append((u"MESSAGE", metadata.comment)) # ISRC is handled in its own flag if len(text_pairs) > 0: return cls(languages=[CDTextLanguage(language_id=0, text_pairs=text_pairs)]) else: return None class CDTextLanguage(object): def __init__(self, language_id, text_pairs): self.__id__ = language_id self.__text_pairs__ = text_pairs def __repr__(self): return "CDTextLanguage(language_id={!r}, text_pairs={!r})".format( self.__id__, self.__text_pairs__) def __len__(self): return len(self.__text_pairs__) def __getitem__(self, key): for (k, v) in self.__text_pairs__: if k == key: return v else: raise KeyError(key) def build(self): output = [u"LANGUAGE {:d} {{".format(self.__id__)] for (key, value) in self.__text_pairs__: if key in {u"TOC_INFO1", u"TOC_INFO2", u"SIZE_INFO"}: output.append(u" {} {}".format(key, format_binary(value))) else: output.append(u" {} {}".format(key, format_string(value))) output.append(u"}") return u"\n".join([u" " + l for l in output]) class CDTextLanguageMap(object): def __init__(self, mapping): self.__mapping__ = mapping def __repr__(self): return "CDTextLanguageMap(mapping={!r})".format(self.__mapping__) def build(self): output = [u"LANGUAGE_MAP {"] output.extend([u" {:d} : {}".format(i, l) for (i, l) in self.__mapping__]) output.append(u"}") return u"\n".join([u" " + l for l in output]) def format_string(s): return u"\"{}\"".format(s.replace(u'\\', u'\\\\').replace(u'"', u'\\"')) def format_timestamp(t): sectors = int(t * 75) return u"{:02d}:{:02d}:{:02d}".format(sectors // 75 // 60, sectors // 75 % 60, sectors % 75) def format_binary(s): return u"{{{}}}".format(",".join([u"{:d}".format(int(c)) for c in s])) def read_tocfile(filename): """returns a Sheet from a TOC filename on disk raises TOCException if some error occurs reading or parsing the file """ try: with open(filename, "rb") as f: return read_tocfile_string(f.read().decode("UTF-8")) except IOError: raise SheetException("unable to open file") def read_tocfile_string(tocfile): """given a unicode string of .toc data, returns a TOCFile object raises SheetException if some error occurs parsing the file""" import audiotools.ply.lex as lex import audiotools.ply.yacc as yacc from audiotools.ply.yacc import NullLogger import audiotools.toc.tokrules import audiotools.toc.yaccrules from sys import version_info str_type = str if (version_info[0] >= 3) else unicode assert(isinstance(tocfile, str_type)) lexer = lex.lex(module=audiotools.toc.tokrules) lexer.input(tocfile) parser = yacc.yacc(module=audiotools.toc.yaccrules, debug=0, errorlog=NullLogger(), write_tables=0) try: return parser.parse(lexer=lexer) except ValueError as err: raise SheetException(str(err)) def write_tocfile(sheet, filename, file): """given a Sheet object and filename unicode string, writes a .toc file to the given file object""" file.write( TOCFile.converted(sheet, filename=filename).build().encode("UTF-8")) ================================================ FILE: audiotools/toc/tokrules.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA RESERVED = {"CATALOG": "CATALOG", "CD_DA": "CD_DA", "CD_ROM": "CD_ROM", "CD_ROM_XA": "CD_ROM_XA", "CD_TEXT": "CD_TEXT", "TRACK": "TRACK", "AUDIO": "AUDIO", "MODE1": "MODE1", "MODE1_RAW": "MODE1_RAW", "MODE2": "MODE2", "MODE2_FORM1": "MODE2_FORM1", "MODE2_FORM2": "MODE2_FORM2", "MODE2_FORM_MIX": "MODE2_FORM_MIX", "MODE2_RAW": "MODE2_RAW", "RW": "RW", "RW_RAW": "RW_RAW", "NO": "NO", "COPY": "COPY", "PRE_EMPHASIS": "PRE_EMPHASIS", "TWO_CHANNEL_AUDIO": "TWO_CHANNEL_AUDIO", "FOUR_CHANNEL_AUDIO": "FOUR_CHANNEL_AUDIO", "ISRC": "ISRC", "SILENCE": "SILENCE", "ZERO": "ZERO", "FILE": "FILE", "AUDIOFILE": "AUDIOFILE", "DATAFILE": "DATAFILE", "FIFO": "FIFO", "START": "START", "PREGAP": "PREGAP", "INDEX": "INDEX", "LANGUAGE_MAP": "LANGUAGE_MAP", "LANGUAGE": "LANGUAGE", "TITLE": "TITLE", "PERFORMER": "PERFORMER", "SONGWRITER": "SONGWRITER", "COMPOSER": "COMPOSER", "ARRANGER": "ARRANGER", "MESSAGE": "MESSAGE", "DISC_ID": "DISC_ID", "GENRE": "GENRE", "TOC_INFO1": "TOC_INFO1", "TOC_INFO2": "TOC_INFO2", "UPC_EAN": "UPC_EAN", "SIZE_INFO": "SIZE_INFO", "EN": "EN"} tokens = ["COMMENT", "START_BLOCK", "END_BLOCK", "COLON", "COMMA", "TIMESTAMP", "NUMBER", "ID", "STRING"] + list(RESERVED.values()) def t_COMMENT(t): r'//.*' pass t_START_BLOCK = r'{' t_END_BLOCK = r'}' t_COLON = r':' t_COMMA = r',' def t_ID(t): r'[A-Z][A-Z0-9_]*' if t.value in RESERVED.keys(): t.type = RESERVED[t.value] return t def t_STRING(t): r'\"(\\.|[^"])*\"' from re import sub t.value = sub(r'\\.', lambda s: s.group(0)[1:], t.value[1:-1]) return t def t_TIMESTAMP(t): r'[0-9]{1,3}:[0-9]{1,2}:[0-9]{1,2}' (m, s, f) = t.value.split(":") t.value = ((int(m) * 75 * 60) + (int(s) * 75) + (int(f))) return t def t_NUMBER(t): r'[0-9]+' t.value = int(t.value) return t t_ignore = " \r\t" def t_newline(t): r'\n+' t.lexer.lineno += t.value.count("\n") def t_error(t): raise ValueError("illegal character {!r}".format(t.value[0])) ================================================ FILE: audiotools/toc/yaccrules.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools.toc.tokrules import tokens def p_tocfile(t): '''tocfile : headers tracks''' from audiotools.toc import TOCFile args = dict(t[1]) args["tracks"] = t[2] t[0] = TOCFile(**args) def p_headers(t): '''headers : header | headers header''' if len(t) == 2: t[0] = [t[1]] else: t[0] = t[1] + [t[2]] def p_header(t): '''header : CD_DA | CD_ROM | CD_ROM_XA | CATALOG STRING | header_cd_text''' if t[1] in ["CD_DA", "CD_ROM", "CD_ROM_XA"]: t[0] = ("type", t[1]) elif t[1] == "CATALOG": t[0] = ("catalog", t[2]) else: t[0] = ("cd_text", t[1]) def p_header_cd_text(t): '''header_cd_text : CD_TEXT START_BLOCK language_map language_blocks END_BLOCK''' from audiotools.toc import CDText t[0] = CDText(languages=t[4], language_map=t[3]) def p_language_map(t): '''language_map : LANGUAGE_MAP START_BLOCK language_mappings END_BLOCK''' from audiotools.toc import CDTextLanguageMap t[0] = CDTextLanguageMap(t[3]) def p_language_mappings(t): '''language_mappings : language_mapping | language_mappings language_mapping''' if len(t) == 2: t[0] = [t[1]] else: t[0] = t[1] = [t[2]] def p_language_mapping(t): '''language_mapping : NUMBER COLON language''' t[0] = (t[1], t[3]) def p_language(t): '''language : EN | NUMBER''' # FIXME - find list of supported languages t[0] = t[1] def p_language_blocks(t): '''language_blocks : language_block | language_blocks language_block''' if len(t) == 2: t[0] = [t[1]] else: t[0] = t[1] + [t[2]] def p_language_block(t): '''language_block : LANGUAGE NUMBER START_BLOCK cd_text_items END_BLOCK''' from audiotools.toc import CDTextLanguage t[0] = CDTextLanguage(language_id=t[2], text_pairs=t[4]) def p_cd_text_items(t): '''cd_text_items : cd_text_item | cd_text_items cd_text_item''' if len(t) == 2: t[0] = [t[1]] else: t[0] = t[1] + [t[2]] def p_cd_text_item(t): '''cd_text_item : TITLE STRING | PERFORMER STRING | SONGWRITER STRING | COMPOSER STRING | ARRANGER STRING | MESSAGE STRING | DISC_ID STRING | GENRE STRING | TOC_INFO1 binary | TOC_INFO2 binary | UPC_EAN STRING | ISRC STRING | SIZE_INFO binary''' t[0] = (t[1], t[2]) def p_binary(t): '''binary : START_BLOCK bytes END_BLOCK''' t[0] = "".join(map(chr, t[2])) def p_bytes(t): '''bytes : NUMBER | NUMBER COMMA bytes''' if len(t) == 2: t[0] = [t[1]] else: t[0] = [t[1]] + t[3] def p_tracks(t): '''tracks : track | tracks track''' if len(t) == 2: t[0] = [t[1]] else: t[0] = t[1] + [t[2]] def p_track(t): '''track : TRACK track_mode track_flags | TRACK track_mode sub_channel_mode track_flags''' from audiotools.toc import TOCTrack if len(t) == 4: t[0] = TOCTrack(mode=t[2], flags=t[3]) else: t[0] = TOCTrack(mode=t[2], flags=t[4], sub_channel_mode=t[3]) def p_track_mode(t): '''track_mode : AUDIO | MODE1 | MODE1_RAW | MODE2 | MODE2_FORM1 | MODE2_FORM2 | MODE2_FORM_MIX | MODE2_RAW''' t[0] = t[1] def p_sub_channel_mode(t): '''sub_channel_mode : RW | RW_RAW''' t[0] = t[1] def p_track_flags(t): '''track_flags : track_flag | track_flags track_flag''' if len(t) == 2: t[0] = [t[1]] else: t[0] = t[1] + [t[2]] def p_track_flag(t): '''track_flag : SILENCE length | ZERO length | DATAFILE STRING | DATAFILE STRING length | FIFO STRING length | PREGAP TIMESTAMP''' # FIXME - handle remaining flags raise NotImplementedError() def p_track_cd_text(t): "track_flag : CD_TEXT START_BLOCK language_blocks END_BLOCK" from audiotools.toc import CDText t[0] = CDText(languages=t[3]) def p_track_flag_copy(t): "track_flag : COPY" from audiotools.toc import TOCFlag_COPY t[0] = TOCFlag_COPY(True) def p_track_flag_no_copy(t): "track_flag : NO COPY" from audiotools.toc import TOCFlag_COPY t[0] = TOCFlag_COPY(False) def p_track_flag_pre_emphasis(t): "track_flag : PRE_EMPHASIS" from audiotools.toc import TOCFlag_PRE_EMPHASIS t[0] = TOCFlag_PRE_EMPHASIS(True) def p_track_flag_no_pre_emphasis(t): "track_flag : NO PRE_EMPHASIS" from audiotools.toc import TOCFlag_PRE_EMPHASIS t[0] = TOCFlag_PRE_EMPHASIS(False) def p_track_flag_two_channels(t): "track_flag : TWO_CHANNEL_AUDIO" from audiotools.toc import TOCFlag_CHANNELS t[0] = TOCFlag_CHANNELS(2) def p_track_flag_four_channels(t): "track_flag : FOUR_CHANNEL_AUDIO" from audiotools.toc import TOCFlag_CHANNELS t[0] = TOCFlag_CHANNELS(4) def p_track_flag_isrc(t): "track_flag : ISRC STRING" from audiotools.toc import TOCFlag_ISRC t[0] = TOCFlag_ISRC(t[2]) def p_track_file(t): '''track_flag : FILE STRING start | AUDIOFILE STRING start | FILE STRING start length | AUDIOFILE STRING start length''' from audiotools.toc import TOCFlag_FILE if len(t) == 4: t[0] = TOCFlag_FILE(type=t[1], filename=t[2], start=t[3]) else: t[0] = TOCFlag_FILE(type=t[1], filename=t[2], start=t[3], length=t[4]) def p_track_start(t): '''track_flag : START | START TIMESTAMP''' from audiotools.toc import TOCFlag_START if len(t) == 2: t[0] = TOCFlag_START() else: from fractions import Fraction t[0] = TOCFlag_START(Fraction(t[2], 75)) def p_track_index(t): "track_flag : INDEX TIMESTAMP" from audiotools.toc import TOCFlag_INDEX from fractions import Fraction t[0] = TOCFlag_INDEX(Fraction(t[2], 75)) def p_start_number(t): "start : NUMBER" from fractions import Fraction t[0] = Fraction(t[1], 44100) def p_start_timestamp(t): "start : TIMESTAMP" from fractions import Fraction t[0] = Fraction(t[1], 75) def p_length_number(t): "length : NUMBER" from fractions import Fraction t[0] = Fraction(t[1], 44100) def p_length_timestamp(t): "length : TIMESTAMP" from fractions import Fraction t[0] = Fraction(t[1], 75) def p_error(t): from audiotools.text import ERR_CUE_SYNTAX_ERROR raise ValueError(ERR_CUE_SYNTAX_ERROR.format(t.lexer.lineno)) ================================================ FILE: audiotools/tta.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools import (AudioFile, InvalidFile) from audiotools.ape import ApeTaggedAudio, ApeGainedAudio def div_ceil(n, d): """returns the ceiling of n divided by d as an int""" return n // d + (1 if ((n % d) != 0) else 0) class InvalidTTA(InvalidFile): pass class TrueAudio(ApeTaggedAudio, ApeGainedAudio, AudioFile): """a True Audio file""" SUFFIX = "tta" NAME = SUFFIX DESCRIPTION = u"True Audio" def __init__(self, filename): from audiotools.id3 import skip_id3v2_comment AudioFile.__init__(self, filename) try: with open(filename, "rb") as f: skip_id3v2_comment(f) from audiotools.bitstream import BitstreamReader from audiotools.text import (ERR_TTA_INVALID_SIGNATURE, ERR_TTA_INVALID_FORMAT) reader = BitstreamReader(f, True) (signature, format_, self.__channels__, self.__bits_per_sample__, self.__sample_rate__, self.__total_pcm_frames__) = reader.parse( "4b 16u 16u 16u 32u 32u 32p") if signature != b"TTA1": raise InvalidTTA(ERR_TTA_INVALID_SIGNATURE) elif format_ != 1: raise InvalidTTA(ERR_TTA_INVALID_FORMAT) self.__total_tta_frames__ = div_ceil( self.__total_pcm_frames__ * 245, self.__sample_rate__ * 256) self.__frame_lengths__ = list(reader.parse( "{:d}* 32u".format(self.__total_tta_frames__) + "32p")) except IOError as msg: raise InvalidTTA(str(msg)) def bits_per_sample(self): """returns an integer number of bits-per-sample this track contains""" return self.__bits_per_sample__ def channels(self): """returns an integer number of channels this track contains""" return self.__channels__ def channel_mask(self): """returns a ChannelMask object of this track's channel layout""" from audiotools import ChannelMask if self.__channels__ == 1: return ChannelMask(0x4) elif self.__channels__ == 2: return ChannelMask(0x3) else: return ChannelMask(0) def lossless(self): """returns True if this track's data is stored losslessly""" return True def total_frames(self): """returns the total PCM frames of the track as an integer""" return self.__total_pcm_frames__ def sample_rate(self): """returns the rate of the track's audio as an integer number of Hz""" return self.__sample_rate__ def seekable(self): """returns True if the file is seekable""" return True @classmethod def supports_to_pcm(cls): """returns a PCMReader object containing the track's PCM data if an error occurs initializing a decoder, this should return a PCMReaderError with an appropriate error message""" try: from audiotools.decoders import TTADecoder return True except ImportError: return False def to_pcm(self): """returns a PCMReader object containing the track's PCM data if an error occurs initializing a decoder, this should return a PCMReaderError with an appropriate error message""" from audiotools.decoders import TTADecoder from audiotools import PCMReaderError from audiotools.id3 import skip_id3v2_comment try: tta = open(self.filename, "rb") except IOError as msg: return PCMReaderError(error_message=str(msg), sample_rate=self.sample_rate(), channels=self.channels(), channel_mask=int(self.channel_mask()), bits_per_sample=self.bits_per_sample()) try: skip_id3v2_comment(tta) return TTADecoder(tta) except (IOError, ValueError) as msg: # This isn't likely unless the TTA file is modified # between when TrueAudio is instantiated # and to_pcm() is called. tta.close() return PCMReaderError(error_message=str(msg), sample_rate=self.sample_rate(), channels=self.channels(), channel_mask=int(self.channel_mask()), bits_per_sample=self.bits_per_sample()) @classmethod def supports_from_pcm(cls): """returns True if all necessary components are available to support the .from_pcm() classmethod""" try: from audiotools.encoders import encode_tta return True except ImportError: return False @classmethod def from_pcm(cls, filename, pcmreader, compression=None, total_pcm_frames=None, encoding_function=None): """encodes a new file from PCM data takes a filename string, PCMReader object, optional compression level string and optional total_pcm_frames integer encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new AudioFile-compatible object may raise EncodingError if some problem occurs when encoding the input file. This includes an error in the input stream, a problem writing the output file, or even an EncodingError subclass such as "UnsupportedBitsPerSample" if the input stream is formatted in a way this class is unable to support """ from audiotools import (BufferedPCMReader, CounterPCMReader, transfer_data, EncodingError) # from audiotools.py_encoders import encode_tta from audiotools.encoders import encode_tta from audiotools.bitstream import BitstreamWriter if pcmreader.bits_per_sample not in {8, 16, 24}: from audiotools import UnsupportedBitsPerSample pcmreader.close() raise UnsupportedBitsPerSample(filename, pcmreader.bits_per_sample) # open output file right away # so we can fail as soon as possible try: file = open(filename, "wb") except IOError as err: pcmreader.close() raise EncodingError(str(err)) try: encode_tta( file=file, pcmreader=pcmreader, total_pcm_frames=(total_pcm_frames if total_pcm_frames is not None else 0)) return cls(filename) except (IOError, ValueError) as err: cls.__unlink__(filename) raise EncodingError(str(err)) except: cls.__unlink__(filename) raise finally: file.close() def data_size(self): """returns the size of the file's data, in bytes, calculated from its header and seektable""" return (22 + # header size (len(self.__frame_lengths__) * 4) + 4 + # seektable size sum(self.__frame_lengths__)) # frames size @classmethod def supports_metadata(cls): """returns True if this audio type supports MetaData""" return True @classmethod def supports_cuesheet(cls): return True def get_cuesheet(self): """returns the embedded Cuesheet-compatible object, or None raises IOError if a problem occurs when reading the file""" import audiotools.cue as cue from audiotools import SheetException metadata = self.get_metadata() if metadata is not None: try: if b'Cuesheet' in metadata.keys(): return cue.read_cuesheet_string( metadata[b'Cuesheet'].__unicode__()) elif b'CUESHEET' in metadata.keys(): return cue.read_cuesheet_string( metadata[b'CUESHEET'].__unicode__()) else: return None except SheetException: # unlike FLAC, just because a cuesheet is embedded # does not mean it is compliant return None else: return None def set_cuesheet(self, cuesheet): """imports cuesheet data from a Sheet object Raises IOError if an error occurs setting the cuesheet""" import os.path from io import BytesIO from audiotools import (MetaData, Filename, FS_ENCODING) from audiotools.ape import ApeTag from audiotools.cue import write_cuesheet if cuesheet is None: return self.delete_cuesheet() metadata = self.get_metadata() if metadata is None: metadata = ApeTag([]) else: metadata = ApeTag.converted(metadata) cuesheet_data = BytesIO() write_cuesheet(cuesheet, u"{}".format(Filename(self.filename).basename()), cuesheet_data) metadata[b'Cuesheet'] = ApeTag.ITEM.string( b'Cuesheet', cuesheet_data.getvalue().decode(FS_ENCODING, 'replace')) self.update_metadata(metadata) def delete_cuesheet(self): """deletes embedded Sheet object, if any Raises IOError if a problem occurs when updating the file""" from audiotools import ApeTag metadata = self.get_metadata() if ((metadata is not None) and isinstance(metadata, ApeTag)): if b"Cuesheet" in metadata.keys(): del(metadata[b"Cuesheet"]) self.update_metadata(metadata) elif b"CUESHEET" in metadata.keys(): del(metadata[b"CUESHEET"]) self.update_metadata(metadata) def write_header(writer, channels, bits_per_sample, sample_rate, total_pcm_frames): """writes a TTA header to the given BitstreamWriter with the given int attributes""" crc = CRC32() writer.add_callback(crc.update) writer.build("4b 16u 16u 16u 32u 32u", [b"TTA1", 1, channels, bits_per_sample, sample_rate, total_pcm_frames]) writer.pop_callback() writer.write(32, int(crc)) def write_seektable(writer, frame_sizes): """writes a TTA header to the given BitstreamWriter where frame_sizes is a list of frame sizes, in bytes""" crc = CRC32() writer.add_callback(crc.update) writer.build("%d* 32U" % (len(frame_sizes)), frame_sizes) writer.pop_callback() writer.write(32, int(crc)) class CRC32(object): TABLE = [0x00000000, 0x77073096, 0xEE0E612C, 0x990951BA, 0x076DC419, 0x706AF48F, 0xE963A535, 0x9E6495A3, 0x0EDB8832, 0x79DCB8A4, 0xE0D5E91E, 0x97D2D988, 0x09B64C2B, 0x7EB17CBD, 0xE7B82D07, 0x90BF1D91, 0x1DB71064, 0x6AB020F2, 0xF3B97148, 0x84BE41DE, 0x1ADAD47D, 0x6DDDE4EB, 0xF4D4B551, 0x83D385C7, 0x136C9856, 0x646BA8C0, 0xFD62F97A, 0x8A65C9EC, 0x14015C4F, 0x63066CD9, 0xFA0F3D63, 0x8D080DF5, 0x3B6E20C8, 0x4C69105E, 0xD56041E4, 0xA2677172, 0x3C03E4D1, 0x4B04D447, 0xD20D85FD, 0xA50AB56B, 0x35B5A8FA, 0x42B2986C, 0xDBBBC9D6, 0xACBCF940, 0x32D86CE3, 0x45DF5C75, 0xDCD60DCF, 0xABD13D59, 0x26D930AC, 0x51DE003A, 0xC8D75180, 0xBFD06116, 0x21B4F4B5, 0x56B3C423, 0xCFBA9599, 0xB8BDA50F, 0x2802B89E, 0x5F058808, 0xC60CD9B2, 0xB10BE924, 0x2F6F7C87, 0x58684C11, 0xC1611DAB, 0xB6662D3D, 0x76DC4190, 0x01DB7106, 0x98D220BC, 0xEFD5102A, 0x71B18589, 0x06B6B51F, 0x9FBFE4A5, 0xE8B8D433, 0x7807C9A2, 0x0F00F934, 0x9609A88E, 0xE10E9818, 0x7F6A0DBB, 0x086D3D2D, 0x91646C97, 0xE6635C01, 0x6B6B51F4, 0x1C6C6162, 0x856530D8, 0xF262004E, 0x6C0695ED, 0x1B01A57B, 0x8208F4C1, 0xF50FC457, 0x65B0D9C6, 0x12B7E950, 0x8BBEB8EA, 0xFCB9887C, 0x62DD1DDF, 0x15DA2D49, 0x8CD37CF3, 0xFBD44C65, 0x4DB26158, 0x3AB551CE, 0xA3BC0074, 0xD4BB30E2, 0x4ADFA541, 0x3DD895D7, 0xA4D1C46D, 0xD3D6F4FB, 0x4369E96A, 0x346ED9FC, 0xAD678846, 0xDA60B8D0, 0x44042D73, 0x33031DE5, 0xAA0A4C5F, 0xDD0D7CC9, 0x5005713C, 0x270241AA, 0xBE0B1010, 0xC90C2086, 0x5768B525, 0x206F85B3, 0xB966D409, 0xCE61E49F, 0x5EDEF90E, 0x29D9C998, 0xB0D09822, 0xC7D7A8B4, 0x59B33D17, 0x2EB40D81, 0xB7BD5C3B, 0xC0BA6CAD, 0xEDB88320, 0x9ABFB3B6, 0x03B6E20C, 0x74B1D29A, 0xEAD54739, 0x9DD277AF, 0x04DB2615, 0x73DC1683, 0xE3630B12, 0x94643B84, 0x0D6D6A3E, 0x7A6A5AA8, 0xE40ECF0B, 0x9309FF9D, 0x0A00AE27, 0x7D079EB1, 0xF00F9344, 0x8708A3D2, 0x1E01F268, 0x6906C2FE, 0xF762575D, 0x806567CB, 0x196C3671, 0x6E6B06E7, 0xFED41B76, 0x89D32BE0, 0x10DA7A5A, 0x67DD4ACC, 0xF9B9DF6F, 0x8EBEEFF9, 0x17B7BE43, 0x60B08ED5, 0xD6D6A3E8, 0xA1D1937E, 0x38D8C2C4, 0x4FDFF252, 0xD1BB67F1, 0xA6BC5767, 0x3FB506DD, 0x48B2364B, 0xD80D2BDA, 0xAF0A1B4C, 0x36034AF6, 0x41047A60, 0xDF60EFC3, 0xA867DF55, 0x316E8EEF, 0x4669BE79, 0xCB61B38C, 0xBC66831A, 0x256FD2A0, 0x5268E236, 0xCC0C7795, 0xBB0B4703, 0x220216B9, 0x5505262F, 0xC5BA3BBE, 0xB2BD0B28, 0x2BB45A92, 0x5CB36A04, 0xC2D7FFA7, 0xB5D0CF31, 0x2CD99E8B, 0x5BDEAE1D, 0x9B64C2B0, 0xEC63F226, 0x756AA39C, 0x026D930A, 0x9C0906A9, 0xEB0E363F, 0x72076785, 0x05005713, 0x95BF4A82, 0xE2B87A14, 0x7BB12BAE, 0x0CB61B38, 0x92D28E9B, 0xE5D5BE0D, 0x7CDCEFB7, 0x0BDBDF21, 0x86D3D2D4, 0xF1D4E242, 0x68DDB3F8, 0x1FDA836E, 0x81BE16CD, 0xF6B9265B, 0x6FB077E1, 0x18B74777, 0x88085AE6, 0xFF0F6A70, 0x66063BCA, 0x11010B5C, 0x8F659EFF, 0xF862AE69, 0x616BFFD3, 0x166CCF45, 0xA00AE278, 0xD70DD2EE, 0x4E048354, 0x3903B3C2, 0xA7672661, 0xD06016F7, 0x4969474D, 0x3E6E77DB, 0xAED16A4A, 0xD9D65ADC, 0x40DF0B66, 0x37D83BF0, 0xA9BCAE53, 0xDEBB9EC5, 0x47B2CF7F, 0x30B5FFE9, 0xBDBDF21C, 0xCABAC28A, 0x53B39330, 0x24B4A3A6, 0xBAD03605, 0xCDD70693, 0x54DE5729, 0x23D967BF, 0xB3667A2E, 0xC4614AB8, 0x5D681B02, 0x2A6F2B94, 0xB40BBE37, 0xC30C8EA1, 0x5A05DF1B, 0x2D02EF8D] def __init__(self): self.crc = 0xffffffff def update(self, byte): self.crc = self.TABLE[(self.crc ^ byte) & 0xFF] ^ (self.crc >> 8) def __int__(self): return self.crc ^ 0xFFFFFFFF ================================================ FILE: audiotools/ui.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA """a module for reusable GUI widgets""" import audiotools from audiotools import PY3, PY2 if PY3: raw_input = input def choice_selection_unicode(metadata): """given a MetaData object, returns a choice selection Unicode string""" if metadata is not None: if metadata.album_name is not None: if metadata.catalog is not None: # both album name and catalog number return u"{album} (catalog #: {catalog})".format( album=metadata.album_name, catalog=metadata.catalog) else: # only album name return metadata.album_name else: if metadata.catalog is not None: # only catalog number return u"catalog #: {catalog}".format( catalog=metadata.catalog) else: # neither album name or catalog number return u"" else: return u"" try: import urwid if urwid.version.VERSION < (1, 0, 0): raise ImportError() def Screen(): from urwid.raw_display import Screen as __Screen__ return __Screen__() AVAILABLE = True class DownEdit(urwid.Edit): """a subclass of urwid.Edit which performs a down-arrow keypress when the enter key is pressed, typically for moving to the next element in a form""" def __init__(self, *args, **kwargs): urwid.Edit.__init__(self, *args, **kwargs) self.__key_map__ = {"enter": "down"} def keypress(self, size, key): if key == "ctrl k": self.set_edit_text(u"") else: return urwid.Edit.keypress(self, size, self.__key_map__.get(key, key)) class DownIntEdit(urwid.IntEdit): """a subclass of urwid.IntEdit which performs a down-arrow keypress when the enter key is pressed, typically for moving to the next element in a form""" def __init__(self, *args, **kwargs): urwid.IntEdit.__init__(self, *args, **kwargs) self.__key_map__ = {"enter": "down"} def keypress(self, size, key): if key == "ctrl k": self.set_edit_text(u"0") else: return urwid.Edit.keypress(self, size, self.__key_map__.get(key, key)) class DownCheckBox(urwid.CheckBox): """a subclass of urwid.CheckBox which performs a down-arrow keypress when the enter key is pressed, typically for moving to the next element in a form""" def keypress(self, size, key): if key == "enter": return urwid.CheckBox.keypress(self, size, "down") else: return urwid.CheckBox.keypress(self, size, key) class FocusFrame(urwid.Frame): """a special Frame widget which performs callbacks on focus changes""" def __init__(self, *args, **kwargs): urwid.Frame.__init__(self, *args, **kwargs) self.focus_callback = None self.focus_callback_arg = None def set_focus_callback(self, callback, user_arg=None): """callback(widget, focus_part[, user_arg]) called when focus is set""" self.focus_callback = callback self.focus_callback_arg = user_arg def set_focus(self, part): urwid.Frame.set_focus(self, part) if self.focus_callback is not None: if self.focus_callback_arg is not None: self.focus_callback(self, part, self.focus_callback_arg) else: self.focus_callback(self, part) def get_focus(widget): # something to smooth out the differences between Urwid versions if hasattr(widget, "get_focus") and callable(widget.get_focus): return widget.get_focus() else: return widget.focus_part class OutputFiller(urwid.Frame): """a class for selecting MetaData and populating output parameters for multiple input tracks""" def __init__(self, track_labels, metadata_choices, input_filenames, output_directory, format_string, output_class, quality, completion_label=u"Apply"): """track_labels is a list of unicode strings, one per track metadata_choices[c][t] is a MetaData object for choice number "c" and track number "t" all choices must have the same number of tracks input_filenames is a list of Filename objects for input files the number of input files must equal the number of metadata objects in each metadata choice output_directory is a string of the default output dir format_string is a format string output_class is the default AudioFile-compatible class quality is a string of the default output quality to use """ self.__cancelled__ = True # a few debug type checks for label in track_labels: assert(isinstance(label, str if PY3 else unicode)) assert(isinstance(output_directory, str)) assert(isinstance(format_string, str)) assert(isinstance(quality, str)) # ensure label count equals path count assert(len(track_labels) == len(input_filenames)) # ensure there's at least one set of choices assert(len(metadata_choices) > 0) # ensure file path count is equal to metadata track count assert(len(metadata_choices[0]) == len(input_filenames)) # ensure input filenames are Filename objects for f in input_filenames: assert(isinstance(f, audiotools.Filename)) from audiotools.text import LAB_CANCEL_BUTTON # setup status bars for output messages self.metadata_status = urwid.Text(u"") self.options_status = urwid.Text(u"") # setup a widget for populating metadata fields self.metadata = MetaDataFiller(track_labels, metadata_choices, self.metadata_status) # setup a widget for populating output parameters self.options = OutputOptions( output_dir=output_directory, format_string=format_string, audio_class=output_class, quality=quality, input_filenames=input_filenames, metadatas=[None for t in input_filenames]) # finish initialization self.wizard = Wizard([self.metadata, self.options], urwid.Button(LAB_CANCEL_BUTTON, on_press=self.exit), urwid.Button(completion_label, on_press=self.complete), self.page_changed) urwid.Frame.__init__(self, body=self.wizard, footer=self.metadata_status) def page_changed(self, new_page): if new_page is self.metadata: self.set_footer(self.metadata_status) elif new_page is self.options: self.options.set_metadatas( list(self.metadata.populated_metadata())) self.set_footer(self.options_status) def exit(self, button): self.__cancelled__ = True raise urwid.ExitMainLoop() def complete(self, button): if self.options.has_collisions: from audiotools.text import ERR_OUTPUT_OUTPUTS_ARE_INPUT self.options_status.set_text(ERR_OUTPUT_OUTPUTS_ARE_INPUT) elif self.options.has_duplicates: from audiotools.text import ERR_OUTPUT_DUPLICATE_NAME self.options_status.set_text(ERR_OUTPUT_DUPLICATE_NAME) elif self.options.has_errors: from audiotools.text import ERR_OUTPUT_INVALID_FORMAT self.options_status.set_text(ERR_OUTPUT_INVALID_FORMAT) else: self.__cancelled__ = False raise urwid.ExitMainLoop() def cancelled(self): """returns True if the widget was cancelled, False if exited normally""" return self.__cancelled__ def handle_text(self, i): if self.get_footer() is self.metadata_status: if i == 'f1': self.metadata.select_previous_item() elif i == 'f2': self.metadata.select_next_item() def output_tracks(self): """yields (output_class, output_filename, output_quality, output_metadata) tuple for each input audio file output_metadata is a newly created MetaData object""" # Note that output_tracks() creates new MetaData objects # while process_output_options() reuses inputted MetaData objects. # This is because we don't want to modify MetaData objects # in the event they're being used elsewhere. (audiofile_class, quality, output_filenames) = self.options.selected_options() for (metadata, output_filename) in zip(self.metadata.populated_metadata(), output_filenames): yield (audiofile_class, output_filename, quality, metadata) class SingleOutputFiller(urwid.Frame): """a class for selecting MetaData and populating output parameters for a single input track""" def __init__(self, track_label, metadata_choices, input_filenames, output_file, output_class, quality, completion_label=u"Apply"): """track_label is a unicode string metadata_choices is a list of MetaData objects, one per possible metadata choice to apply input_filenames is a list or set of Filename objects output_file is a string of the default output filename output_class is the default AudioFile-compatible class quality is a string of the default output quality to use""" # a few debut type checks assert(isinstance(track_label, str if PY3 else unicode)) assert(isinstance(output_file, str)) assert(isinstance(quality, str)) assert(isinstance(completion_label, str if PY3 else unicode)) self.input_filenames = input_filenames self.__cancelled__ = True # ensure there's at least one choice assert(len(metadata_choices) > 0) # ensure input file is a Filename object for f in input_filenames: assert(isinstance(f, audiotools.Filename)) from audiotools.text import (LAB_CANCEL_BUTTON, LAB_OUTPUT_OPTIONS) # setup status bar for output messages self.status = urwid.Text(u"") # setup a widget for cancel/finish buttons output_buttons = urwid.Filler( urwid.Columns( widget_list=[ ('weight', 1, urwid.Button(LAB_CANCEL_BUTTON, on_press=self.exit)), ('weight', 2, urwid.Button(completion_label, on_press=self.complete))], dividechars=3, focus_column=1)) # setup a widget for populating output parameters self.options = SingleOutputOptions( output_filename=output_file, audio_class=output_class, quality=quality) # combine metadata and output options into single widget self.metadata = MetaDataFiller( track_labels=[track_label], metadata_choices=[[m] for m in metadata_choices], status=self.status) body = urwid.Pile( [("weight", 1, self.metadata), ("fixed", 5, urwid.LineBox(self.options, title=LAB_OUTPUT_OPTIONS)), ("fixed", 1, output_buttons)]) # finish initialization urwid.Frame.__init__(self, body=body, footer=self.status) def exit(self, button): self.__cancelled__ = True raise urwid.ExitMainLoop() def complete(self, button): output_filename = self.options.selected_options()[2] # ensure output filename isn't same as input filename if output_filename in self.input_filenames: from audiotools.text import ERR_OUTPUT_IS_INPUT self.status.set_text( ERR_OUTPUT_IS_INPUT.format(output_filename)) else: self.__cancelled__ = False raise urwid.ExitMainLoop() def cancelled(self): return self.__cancelled__ def handle_text(self, i): if i == 'esc': self.exit(None) elif i == 'f1': self.metadata.select_previous_item() elif i == 'f2': self.metadata.select_next_item() def output_track(self): """returns (output_class, output_filename, output_quality, output_metadata) output_metadata is a newly created MetaData object""" (output_class, output_quality, output_filename) = self.options.selected_options() return (output_class, output_filename, output_quality, list(self.metadata.populated_metadata())[0]) class MetaDataFiller(urwid.Pile): """a class for selecting the MetaData to apply to tracks""" def __init__(self, track_labels, metadata_choices, status): """track_labels is a list of unicode strings, one per track metadata_choices[c][t] is a MetaData object for choice number "c" and track number "t" this widget allows the user to populate a set of MetaData objects which can be applied to tracks status is an urwid.Text object """ # a few debug type checks for label in track_labels: assert(isinstance(label, str if PY3 else unicode)) # there must be at least one choice assert(len(metadata_choices) > 0) # all choices must have at least 1 track assert(min(map(len, metadata_choices)) > 0) # and all choices must have the same number of tracks assert(len(set(map(len, metadata_choices))) == 1) from audiotools.text import (LAB_SELECT_BEST_MATCH, LAB_TRACK_METADATA) self.metadata_choices = metadata_choices self.status = status # setup a MetaDataEditor for each possible match self.edit_matches = [ MetaDataEditor( [(i, label, track) for (i, (track, label)) in enumerate(zip(choice, track_labels))], on_swivel_change=self.swiveled) for choice in metadata_choices] self.selected_match = self.edit_matches[0] # place selector at top only if there's more than one match if len(metadata_choices) > 1: # setup radio button for each possible match matches = [] radios = [ urwid.RadioButton(matches, choice_selection_unicode(choice[0]), on_state_change=self.select_match, user_data=i) for (i, choice) in enumerate(metadata_choices)] for radio in radios: radio._label.set_wrap_mode(urwid.CLIP) # put radio buttons in pretty container select_match = urwid.LineBox(urwid.ListBox(radios)) if hasattr(select_match, "set_title"): select_match.set_title(LAB_SELECT_BEST_MATCH) widgets = [("fixed", len(metadata_choices) + 2, select_match)] else: widgets = [] self.track_metadata = urwid.Frame(body=self.edit_matches[0]) widgets.append(("weight", 1, urwid.LineBox(self.track_metadata, title=LAB_TRACK_METADATA))) urwid.Pile.__init__(self, widgets) def select_match(self, radio, selected, match): if selected: self.selected_match = self.edit_matches[match] self.track_metadata.set_body(self.selected_match) def swiveled(self, radio_button, selected, swivel): if selected: from .text import (LAB_KEY_NEXT, LAB_KEY_PREVIOUS) keys = [] if radio_button.previous_radio_button() is not None: keys.extend([('key', u"F1"), LAB_KEY_PREVIOUS.format(swivel.swivel_type)]) if radio_button.next_radio_button() is not None: if len(keys) > 0: keys.append(u" ") keys.extend([('key', u"F2"), LAB_KEY_NEXT.format(swivel.swivel_type)]) if len(keys) > 0: self.status.set_text(keys) else: self.status.set_text(u"") def select_previous_item(self): """selects the previous item (track or field) if possible""" self.selected_match.select_previous_item() def select_next_item(self): """selects the next item (track or field) if possible""" self.selected_match.select_next_item() def populated_metadata(self): """yields a new, populated MetaData object per track, depending on the current selection and its values""" for (track_id, metadata) in self.selected_match.metadata(): yield metadata class MetaDataEditor(urwid.Frame): """a class for editing MetaData values for a set of tracks""" def __init__(self, tracks, on_text_change=None, on_swivel_change=None): """tracks is a list of (id, label, MetaData) tuples in the order they are to be displayed where id is some unique hashable ID value label is a unicode string and MetaData is an audiotools.MetaData-compatible object or None on_text_change is a callback for when any text field is modified on_swivel_change is a callback for when tracks and fields are swapped """ for (id, label, metadata) in tracks: assert(isinstance(label, str if PY3 else unicode)) # a list of track IDs in the order they appear self.track_ids = [] # a list of (track_id, label) tuples # in the order they should appear track_labels = [] # the order metadata fields should appear field_labels = [(attr, audiotools.MetaData.FIELD_NAMES[attr]) for attr in audiotools.MetaData.FIELD_ORDER] # a dict of track_id->TrackMetaData values self.metadata_edits = {} # determine the base metadata all others should be linked against base_metadata = {} for (track_id, track_label, metadata) in tracks: self.track_ids.append(track_id) for (attr, value) in (metadata if metadata is not None else audiotools.MetaData()).fields(): base_metadata.setdefault(attr, set()).add(value) base_metadata = BaseMetaData( metadata=audiotools.MetaData( **{field: list(values)[0] for (field, values) in base_metadata.items() if (len(values) == 1)}), on_change=on_text_change) # populate the track_labels and metadata_edits lookup tables for (track_id, track_label, metadata) in tracks: if track_id not in self.metadata_edits: track_labels.append((track_id, track_label)) self.metadata_edits[track_id] = TrackMetaData( metadata=(metadata if metadata is not None else audiotools.MetaData()), base_metadata=base_metadata, on_change=on_text_change) else: # no_duplicates via open_files should filter this case raise ValueError("same track ID cannot appear twice") swivel_radios = [] track_radios_order = [] track_radios = {} field_radios_order = [] field_radios = {} # generate radio buttons for track labels for (track_id, track_label) in track_labels: radio = OrderedRadioButton(ordered_group=track_radios_order, group=swivel_radios, label=('label', track_label), state=False) swivel = Swivel( swivel_type=u"track", left_top_widget=urwid.Text(('label', 'fields')), left_alignment='fixed', left_width=4 + max(len(label) for _, label in field_labels), left_radios=field_radios, left_ids=[field_id for (field_id, label) in field_labels], right_top_widget=urwid.Text(('label', track_label), wrap=urwid.CLIP), right_alignment='weight', right_width=1, right_widgets=[getattr(self.metadata_edits[track_id], field_id) for (field_id, label) in field_labels]) radio._label.set_wrap_mode(urwid.CLIP) urwid.connect_signal(radio, 'change', self.activate_swivel, swivel) if on_swivel_change is not None: urwid.connect_signal(radio, 'change', on_swivel_change, swivel) track_radios[track_id] = radio # generate radio buttons for metadata labels for (field_id, field_label) in field_labels: radio = OrderedRadioButton(ordered_group=field_radios_order, group=swivel_radios, label=('label', field_label), state=False) swivel = Swivel( swivel_type=u"field", left_top_widget=urwid.Text(('label', u'files')), left_alignment='weight', left_width=1, left_radios=track_radios, left_ids=[track_id for (track_id, track) in track_labels], right_top_widget=urwid.Text(('label', field_label)), right_alignment='weight', right_width=2, right_widgets=[getattr(self.metadata_edits[track_id], field_id) for (track_id, track) in track_labels]) radio._label.set_align_mode('right') urwid.connect_signal(radio, 'change', self.activate_swivel, swivel) if on_swivel_change is not None: urwid.connect_signal(radio, 'change', on_swivel_change, swivel) field_radios[field_id] = radio urwid.Frame.__init__( self, header=urwid.Columns( [("fixed", 1, urwid.Text(u"")), ("weight", 1, urwid.Text(u""))]), body=urwid.ListBox([])) if len(self.metadata_edits) != 1: # if more than one track, select track_name radio button field_radios["track_name"].set_state(True) else: # if only one track, select that track's radio button track_radios[track_labels[0][0]].set_state(True) def activate_swivel(self, radio_button, selected, swivel): if selected: self.selected_radio = radio_button # add new entries according to swivel's values self.set_body( urwid.ListBox( [urwid.Columns([(swivel.left_alignment, swivel.left_width, left_widget), (swivel.right_alignment, swivel.right_width, right_widget)]) for (left_widget, right_widget) in swivel.rows()])) # update header with swivel's values self.set_header( urwid.Columns( [(swivel.left_alignment, swivel.left_width, urwid.Text(u"")), (swivel.right_alignment, swivel.right_width, LinkedWidgetHeader(swivel.right_top_widget))])) else: pass def select_previous_item(self): previous_radio = self.selected_radio.previous_radio_button() if previous_radio is not None: previous_radio.set_state(True) def select_next_item(self): next_radio = self.selected_radio.next_radio_button() if next_radio is not None: next_radio.set_state(True) def metadata(self): """yields a (track_id, MetaData) tuple per edited metadata track MetaData objects are newly created""" for track_id in self.track_ids: yield (track_id, self.metadata_edits[track_id].edited_metadata()) class OrderedRadioButton(urwid.RadioButton): def __init__(self, ordered_group, group, label, state='first True', on_state_change=None, user_data=None): urwid.RadioButton.__init__(self, group, label, state, on_state_change, user_data) ordered_group.append(self) self.ordered_group = ordered_group def previous_radio_button(self): for (current_radio, previous_radio) in zip(self.ordered_group, [None] + self.ordered_group): if current_radio is self: return previous_radio else: return None def next_radio_button(self): for (current_radio, next_radio) in zip(self.ordered_group, self.ordered_group[1:] + [None]): if current_radio is self: return next_radio else: return None class LinkedWidgetHeader(urwid.Columns): def __init__(self, widget): urwid.Columns.__init__(self, [("fixed", 3, urwid.Text(u" ")), ("weight", 1, widget), ("fixed", 4, urwid.Text(u""))]) class LinkedWidgetDivider(urwid.Columns): def __init__(self): urwid.Columns.__init__( self, [("fixed", 3, urwid.Text(u"\u2500\u2534\u2500")), ("weight", 1, urwid.Divider(u"\u2500")), ("fixed", 4, urwid.Text(u"\u2500" * 4))]) class LinkedWidgets(urwid.Columns): def __init__(self, checkbox_group, linked_widget, unlinked_widget, initially_linked): """linked_widget is shown when the linking checkbox is checked otherwise unlinked_widget is shown""" self.linked_widget = linked_widget self.unlinked_widget = unlinked_widget self.checkbox_group = checkbox_group self.checkbox = urwid.CheckBox(u"", state=initially_linked, on_state_change=self.swap_link) self.checkbox_group.append(self.checkbox) urwid.Columns.__init__( self, [("fixed", 3, urwid.Text(u" : ")), ("weight", 1, linked_widget if initially_linked else unlinked_widget), ("fixed", 4, self.checkbox)]) def swap_link(self, checkbox, linked): if linked: # if nothing else linked in this checkbox group, # set linked text to whatever the last unlinked text as if ({cb.get_state() for cb in self.checkbox_group if (cb is not checkbox)} == {False}): if (hasattr(self.linked_widget, "set_edit_text") and hasattr(self.unlinked_widget, "get_edit_text")): self.linked_widget.set_edit_text( self.unlinked_widget.get_edit_text()) elif (hasattr(self.linked_widget, "set_state") and hasattr(self.unlinked_widget, "get_state")): self.linked_widget.set_state( self.unlinked_widget.get_state()) self.widget_list[1] = self.linked_widget self.set_focus(2) else: # set unlinked text to whatever the last linked text was if (hasattr(self.unlinked_widget, "set_edit_text") and hasattr(self.linked_widget, "get_edit_text")): self.unlinked_widget.set_edit_text( self.linked_widget.get_edit_text()) elif (hasattr(self.unlinked_widget, "set_state") and hasattr(self.linked_widget, "get_state")): self.unlinked_widget.set_state( self.linked_widget.get_state()) self.widget_list[1] = self.unlinked_widget self.set_focus(2) def value(self): if self.checkbox.get_state(): widget = self.linked_widget else: widget = self.unlinked_widget if type(widget) is DownIntEdit: if len(widget.edit_text) > 0: return widget.value() else: return None elif type(widget) is DownEdit: if len(widget.get_edit_text()) > 0: return widget.get_edit_text() else: return None elif type(widget) is DownCheckBox: if widget.get_state(): return True else: return None else: return None class BaseMetaData(object): def __init__(self, metadata, on_change=None): """metadata is a MetaData object on_change is a callback for when the text field is modified""" self.metadata = metadata self.checkbox_groups = {} for field, field_type in metadata.FIELD_TYPES.items(): if field_type is type(u""): value = getattr(metadata, field) widget = DownEdit(edit_text=value if value is not None else u"") elif field_type is int: value = getattr(metadata, field) widget = DownIntEdit(default=value if value is not None else None) elif field_type is bool: value = getattr(metadata, field) widget = DownCheckBox(u"", state=value if value is not None else False) if on_change is not None: urwid.connect_signal(widget, 'change', on_change) setattr(self, field, widget) self.checkbox_groups[field] = [] class TrackMetaData(object): NEVER_LINK = {"track_name", "track_number", "ISRC"} def __init__(self, metadata, base_metadata, on_change=None): """metadata is a MetaData object base_metadata is a BaseMetaData object to link against on_change is a callback for when the text field is modified""" self.images = metadata.images() for field, field_type in metadata.FIELD_TYPES.items(): if field_type is type(u""): value = getattr(metadata, field) widget = DownEdit(edit_text=value if value is not None else u"") elif field_type is int: value = getattr(metadata, field) widget = DownIntEdit(default=value if value is not None else None) elif field_type is bool: value = getattr(metadata, field) widget = DownCheckBox(u"", state=value if value is not None else False) if on_change is not None: urwid.connect_signal(widget, 'change', on_change) linked_widget = LinkedWidgets( checkbox_group=base_metadata.checkbox_groups[field], linked_widget=getattr(base_metadata, field), unlinked_widget=widget, initially_linked=((field not in self.NEVER_LINK) and (getattr(metadata, field) == getattr(base_metadata.metadata, field)))) setattr(self, field, linked_widget) def edited_metadata(self): """returns a new MetaData object of the track's current value based on its widgets' values""" return audiotools.MetaData( images=self.images, **{attr: value for (attr, value) in [(attr, getattr(self, attr).value()) for attr in audiotools.MetaData.FIELDS] if value is not None}) class Swivel(object): """this is a container for the objects of a swiveling operation""" def __init__(self, swivel_type, left_top_widget, left_alignment, left_width, left_radios, left_ids, right_top_widget, right_alignment, right_width, right_widgets): assert(len(left_ids) == len(right_widgets)) self.swivel_type = swivel_type self.left_top_widget = left_top_widget self.left_alignment = left_alignment self.left_width = left_width self.left_radios = left_radios self.left_ids = left_ids self.right_top_widget = right_top_widget self.right_alignment = right_alignment self.right_width = right_width self.right_widgets = right_widgets def rows(self): for (left_id, right_widget) in zip(self.left_ids, self.right_widgets): yield (self.left_radios[left_id], right_widget) def tab_complete(path): """given a partially-completed directory path string returns a path string completed as far as possible """ import os.path (base, remainder) = os.path.split(path) if os.path.isdir(base): try: candidate_dirs = [d for d in os.listdir(base) if (d.startswith(remainder) and os.path.isdir(os.path.join(base, d)))] if len(candidate_dirs) == 0: # no possible matches to tab complete return path elif len(candidate_dirs) == 1: # one possible match to tab complete return os.path.join(base, candidate_dirs[0]) + os.sep else: # multiple possible matches to tab complete # so complete as much as possible return os.path.join(base, os.path.commonprefix(candidate_dirs)) except OSError: # unable to read base dir to complete the rest return path else: # base doesn't exist, # so we don't know how to complete the rest return path def tab_complete_file(path): """given a partially-completed file path string returns a path string completed as far as possible""" import os.path (base, remainder) = os.path.split(path) if os.path.isdir(base): try: candidates = [f for f in os.listdir(base) if f.startswith(remainder)] if len(candidates) == 0: # no possible matches to tab complete return path elif len(candidates) == 1: # one possible match to tab complete path = os.path.join(base, candidates[0]) if os.path.isdir(path): return path + os.sep else: return path else: # multiple possible matches to tab complete # so complete as much as possible return os.path.join(base, os.path.commonprefix(candidates)) except OSError: # unable to read base dir to complete the rest return path else: # base doesn't exist, # so we don't know how to complete the rest return path def pop_directory(path): """given a path string, returns a new path string with one directory removed if possible""" import os.path base = os.path.split(path.rstrip(os.sep))[0] if base == '': return base elif not base.endswith(os.sep): return base + os.sep else: return base def split_at_cursor(edit): """returns a (prefix, suffix) unicode pair of text before and after the urwid.Edit widget's cursor""" return (edit.get_edit_text()[0:edit.edit_pos], edit.get_edit_text()[edit.edit_pos:]) class SelectButtons(urwid.Pile): def __init__(self, widget_list, focus_item=None, cancelled=None): """cancelled is a callback which is called when the esc key is pressed it takes no arguments""" urwid.Pile.__init__(self, widget_list, focus_item) self.cancelled = cancelled def keypress(self, size, key): key = urwid.Pile.keypress(self, size, key) if (key == "esc") and (self.cancelled is not None): self.cancelled() return else: return key class BottomLineBox(urwid.LineBox): """a LineBox that places its title at the bottom instead of the top""" def __init__(self, original_widget, title="", tlcorner=u"\u250c", tline=u"\u2500", lline=u"\u2502", trcorner=u"\u2510", blcorner=u"\u2514", rline=u"\u2502", bline=u"\u2500", brcorner=u"\u2518"): tline, bline = urwid.Divider(tline), urwid.Divider(bline) lline, rline = urwid.SolidFill(lline), urwid.SolidFill(rline) tlcorner, trcorner = urwid.Text(tlcorner), urwid.Text(trcorner) blcorner, brcorner = urwid.Text(blcorner), urwid.Text(brcorner) self.title_widget = urwid.Text(self.format_title(title)) self.tline_widget = urwid.Columns( [tline, ('flow', self.title_widget), tline]) top = urwid.Columns( [('fixed', 1, tlcorner), bline, ('fixed', 1, trcorner)]) middle = urwid.Columns( [('fixed', 1, lline), original_widget, ('fixed', 1, rline)], box_columns=[0, 2], focus_column=1) bottom = urwid.Columns( [('fixed', 1, blcorner), self.tline_widget, ('fixed', 1, brcorner)]) pile = urwid.Pile( [('flow', top), middle, ('flow', bottom)], focus_item=1) urwid.WidgetDecoration.__init__(self, original_widget) urwid.WidgetWrap.__init__(self, pile) class SelectOneDialog(urwid.WidgetWrap): signals = ['close'] def __init__(self, select_one, items, selected_value, label=None): self.select_one = select_one self.items = items selected_button = 0 buttons = [] for (i, (l, value)) in enumerate(items): buttons.append(urwid.Button(label=l, on_press=self.select_button, user_data=(l, value))) if value == selected_value: selected_button = i pile = SelectButtons(buttons, selected_button, lambda: self._emit("close")) fill = urwid.Filler(pile) if label is not None: linebox = urwid.LineBox(fill, title=label) else: linebox = urwid.LineBox(fill) self.__super.__init__(linebox) def select_button(self, button, label_value): (label, value) = label_value self.select_one.make_selection(label, value) self._emit("close") class SelectOne(urwid.PopUpLauncher): def __init__(self, items, selected_value=None, on_change=None, user_data=None, label=None): """items is a list of (unicode, value) tuples where value can be any sort of object selected_value is a selected object on_change is a callback which takes a new selected object which is called as on_change(new_value, [user_data]) label is a unicode label string for the selection box""" self.__select_button__ = urwid.Button(u"") self.__super.__init__(self.__select_button__) urwid.connect_signal( self.original_widget, 'click', lambda button: self.open_pop_up()) assert(len(items) > 0) self.__items__ = items self.__selected_value__ = None # set by make_selection, below self.__on_change__ = None self.__user_data__ = None self.__label__ = label if selected_value is not None: try: (label, value) = [pair for pair in items if pair[1] == selected_value][0] except IndexError: (label, value) = items[0] else: (label, value) = items[0] self.make_selection(label, value) self.__on_change__ = on_change self.__user_data__ = user_data def create_pop_up(self): pop_up = SelectOneDialog(self, self.__items__, self.__selected_value__, self.__label__) urwid.connect_signal( pop_up, 'close', lambda button: self.close_pop_up()) return pop_up def get_pop_up_parameters(self): return {'left': 0, 'top': 1, 'overlay_width': max([4 + len(i[0]) for i in self.__items__]) + 2, 'overlay_height': len(self.__items__) + 2} def make_selection(self, label, value): self.__select_button__.set_label(label) self.__selected_value__ = value if self.__on_change__ is not None: if self.__user_data__ is not None: self.__on_change__(value, self.__user_data__) else: self.__on_change__(value) def selection(self): return self.__selected_value__ def set_items(self, items, selected_value): self.__items__ = items self.make_selection([label for (label, value) in items if value is selected_value][0], selected_value) class SelectDirectory(urwid.Columns): def __init__(self, initial_directory, on_change=None, user_data=None): self.edit = EditDirectory(initial_directory) urwid.Columns.__init__(self, [('weight', 1, self.edit), ('fixed', 10, BrowseDirectory(self.edit))]) if on_change is not None: urwid.connect_signal(self.edit, 'change', on_change, user_data) def set_directory(self, directory): # FIXME - allow this to be assigned externally raise NotImplementedError() def get_directory(self): return self.edit.get_directory() class EditDirectory(urwid.Edit): def __init__(self, initial_directory): """initial_directory is a plain string in the default filesystem encoding this directory has username expanded and is converted to the absolute path""" import os.path FS_ENCODING = audiotools.FS_ENCODING urwid.Edit.__init__( self, edit_text=audiotools.Filename( initial_directory).expanduser().abspath().__unicode__(), wrap='clip', allow_tab=False) def keypress(self, size, key): key = urwid.Edit.keypress(self, size, key) FS_ENCODING = audiotools.FS_ENCODING import os.path if key == 'tab': # only tab complete stuff before cursor (prefix, suffix) = split_at_cursor(self) new_prefix = tab_complete( str(audiotools.Filename.from_unicode( prefix).expanduser().abspath())) if PY2: new_prefix = new_prefix.decode(FS_ENCODING) self.set_edit_text(new_prefix + suffix) self.set_edit_pos(len(new_prefix)) elif key == 'ctrl w': # only delete stuff before cursor (prefix, suffix) = split_at_cursor(self) new_prefix = pop_directory( str(audiotools.Filename.from_unicode( prefix).expanduser().abspath())) if PY2: new_prefix = new_prefix.decode(FS_ENCODING) self.set_edit_text(new_prefix + suffix) self.set_edit_pos(len(new_prefix)) elif key == 'ctrl k': # delete entire line self.set_edit_text(u"") self.set_edit_pos(0) else: return key def set_directory(self, directory): """directory is a plain directory string to set""" assert(isinstance(directory, str)) if PY2: directory = directory.decode(audiotools.FS_ENCODING) self.set_edit_text(directory) self.set_edit_pos(len(directory)) def get_directory(self): """returns selected directory as a plain string""" directory = self.get_edit_text() if PY2: directory = directory.encode(audiotools.FS_ENCODING) return directory class BrowseDirectory(urwid.PopUpLauncher): def __init__(self, edit_directory): """edit_directory is an EditDirectory object""" from audiotools.text import LAB_BROWSE_BUTTON self.__super.__init__( urwid.Button(LAB_BROWSE_BUTTON, on_press=lambda button: self.open_pop_up())) self.edit_directory = edit_directory def create_pop_up(self): pop_up = BrowseDirectoryDialog(self.edit_directory) urwid.connect_signal(pop_up, "close", lambda button: self.close_pop_up()) return pop_up def get_pop_up_parameters(self): # FIXME - make these values dynamic # based on edit_directory's location return {'left': 0, 'top': 1, 'overlay_width': 70, 'overlay_height': 20} class BrowseDirectoryDialog(urwid.WidgetWrap): signals = ['close'] def __init__(self, edit_directory): """edit_directory is an EditDirectory object""" from audiotools.text import (LAB_KEY_SELECT, LAB_KEY_TOGGLE_OPEN, LAB_KEY_CANCEL, LAB_CHOOSE_DIRECTORY) browser = DirectoryBrowser( edit_directory.get_directory(), directory_selected=self.select_directory, cancelled=lambda: self._emit("close")) frame = urwid.LineBox(urwid.Frame( body=browser, footer=urwid.Text([('key', 'enter'), LAB_KEY_SELECT, u" ", ('key', 'space'), LAB_KEY_TOGGLE_OPEN, u" ", ('key', 'esc'), LAB_KEY_CANCEL])), title=LAB_CHOOSE_DIRECTORY) self.__super.__init__(frame) self.edit_directory = edit_directory def select_directory(self, selected_directory): self.edit_directory.set_directory(selected_directory) self._emit("close") class DirectoryBrowser(urwid.TreeListBox): def __init__(self, initial_directory, directory_selected=None, cancelled=None): import os import os.path def path_iter(path): if path == os.sep: yield path else: path = path.rstrip(os.sep) if len(path) > 0: (head, tail) = os.path.split(path) for part in path_iter(head): yield part yield tail else: return topnode = DirectoryNode(os.sep) for path_part in path_iter( os.path.abspath( os.path.expanduser(initial_directory))): try: if path_part == "/": node = topnode else: node = node.get_child_node(path_part) widget = node.get_widget() widget.expanded = True widget.update_expanded_icon() except urwid.treetools.TreeWidgetError: break urwid.TreeListBox.__init__(self, urwid.TreeWalker(topnode)) self.set_focus(node) self.directory_selected = directory_selected self.cancelled = cancelled def selected_directory(self): import os import os.path def focused_nodes(): (widget, node) = self.get_focus() while not node.is_root(): yield node.get_key() node = node.get_parent() else: yield os.sep return os.path.join(*reversed(list(focused_nodes()))) + os.sep def unhandled_input(self, size, input): input = urwid.TreeListBox.unhandled_input(self, size, input) if input == 'enter': if self.directory_selected is not None: self.directory_selected(self.selected_directory()) else: return input elif input == 'esc': if self.cancelled is not None: self.cancelled() else: return input else: return input class DirectoryWidget(urwid.TreeWidget): indent_cols = 1 def __init__(self, node): self.__super.__init__(node) self.expanded = False self.update_expanded_icon() def keypress(self, size, key): key = urwid.TreeWidget.keypress(self, size, key) if key == " ": self.expanded = not self.expanded self.update_expanded_icon() else: return key def get_display_text(self): node = self.get_node() if node.get_depth() == 0: return "/" else: return node.get_key() class ErrorWidget(urwid.TreeWidget): indent_cols = 1 def get_display_text(self): return ('error', u"(error/permission denied)") class ErrorNode(urwid.TreeNode): def load_widget(self): return ErrorWidget(self) class DirectoryNode(urwid.ParentNode): def __init__(self, path, parent=None): import os import os.path if path == os.sep: urwid.ParentNode.__init__(self, value=path, key=None, parent=parent, depth=0) else: urwid.ParentNode.__init__(self, value=path, key=os.path.basename(path), parent=parent, depth=path.count(os.sep)) def load_parent(self): import os.path (parentname, myname) = os.path.split(self.get_value()) parent = DirectoryNode(parentname) parent.set_child_node(self.get_key(), self) return parent def load_child_keys(self): import os.path dirs = [] try: path = self.get_value() for d in sorted(os.listdir(path)): if ((not d.startswith(".")) and os.path.isdir( os.path.join(path, d))): dirs.append(d) except OSError as e: depth = self.get_depth() + 1 self._children[None] = ErrorNode(self, parent=self, key=None, depth=depth) return [None] return dirs def load_child_node(self, key): """Return a DirectoryNode""" import os.path index = self.get_child_index(key) path = os.path.join(self.get_value(), key) return DirectoryNode(path, parent=self) def load_widget(self): return DirectoryWidget(self) class EditFilename(urwid.Edit): def __init__(self, initial_filename): """initial_filename is a plain string in the default filesystem encoding this filename has username expanded and is converted to the absolute path""" import os.path FS_ENCODING = audiotools.FS_ENCODING urwid.Edit.__init__( self, edit_text=audiotools.Filename( initial_filename).expanduser().abspath().__unicode__(), wrap="clip", allow_tab=False) def keypress(self, size, key): key = urwid.Edit.keypress(self, size, key) import os.path FS_ENCODING = audiotools.FS_ENCODING if key == 'tab': # only tab complete stuff before cursor (prefix, suffix) = split_at_cursor(self) new_prefix = tab_complete( str(audiotools.Filename.from_unicode( prefix).expanduser().abspath())) if PY2: new_prefix = new_prefix.decode(FS_ENCODING) self.set_edit_text(new_prefix + suffix) self.set_edit_pos(len(new_prefix)) elif key == 'ctrl w': # only delete stuff before cursor (prefix, suffix) = split_at_cursor(self) new_prefix = pop_directory( str(audiotools.Filename.from_unicode( prefix).expanduser().abspath())) if PY2: new_prefix = new_prefix.decode(FS_ENCODING) self.set_edit_text(new_prefix + suffix) self.set_edit_pos(len(new_prefix)) elif key == 'ctrl k': # delete entire line self.set_edit_text(u"") self.set_edit_pos(0) else: return key def set_filename(self, filename): """filename is a string to set""" assert(isinstance(filename, str)) if PY2: filename = filename.decode(audiotools.FS_ENCODING) self.set_edit_text(filename) self.set_edit_pos(len(filename)) def get_filename(self): """returns selected filename as a string""" filename = self.get_edit_text() if PY2: filename = filename.encode(audiotools.FS_ENCODING) return filename class BrowseFields(urwid.PopUpLauncher): def __init__(self, output_format): """output_format is an Edit object""" from audiotools.text import LAB_FIELDS_BUTTON self.__super.__init__( urwid.Button(LAB_FIELDS_BUTTON, on_press=lambda button: self.open_pop_up())) self.output_format = output_format def create_pop_up(self): pop_up = BrowseFieldsDialog(self.output_format) urwid.connect_signal(pop_up, "close", lambda button: self.close_pop_up()) return pop_up def get_pop_up_parameters(self): return { 'left': 0, 'top': 1, 'overlay_width': (max([len(label) + 4 for (string, label) in audiotools.FORMAT_FIELDS.values()]) + 2), 'overlay_height': len(audiotools.FORMAT_FIELDS.values()) + 2} class BrowseFieldsDialog(urwid.WidgetWrap): signals = ['close'] def __init__(self, output_format): from audiotools.text import (LAB_KEY_CANCEL, LAB_KEY_CLEAR_FORMAT, LAB_ADD_FIELD) self.__super.__init__( urwid.LineBox( urwid.Frame(body=FieldsList(output_format, self.close), footer=urwid.Text([('key', 'del'), LAB_KEY_CLEAR_FORMAT, u" ", ('key', 'esc'), LAB_KEY_CANCEL])), title=LAB_ADD_FIELD)) def close(self): self._emit("close") class FieldsList(urwid.ListBox): def __init__(self, output_format, close): urwid.ListBox.__init__( self, [urwid.Button(label, on_press=self.select_field, user_data=(output_format, string)) for (string, label) in [audiotools.FORMAT_FIELDS[field] for field in audiotools.FORMAT_FIELD_ORDER]]) self.output_format = output_format self.close = close def select_field(self, button, field_value): (field, value) = field_value field.insert_text(value) self.close() def cancel(self): self.close() def keypress(self, size, input): input = urwid.ListBox.keypress(self, size, input) if input == 'esc': self.cancel() elif input == 'delete': self.output_format.set_edit_text(u"") else: return input class OutputOptions(urwid.Pile): """a widget for selecting the typical set of output options for multiple input files: --dir, --format, --type, --quality""" def __init__(self, output_dir, format_string, audio_class, quality, input_filenames, metadatas, extra_widgets=None): """ | field | value | meaning | |-----------------+------------+----------------------------------| | output_dir | string | default output directory | | format_string | string | format string to use for files | | audio_class | AudioFile | audio class of output files | | quality | string | quality level of output | | input_filenames | [Filename] | Filename objects for input files | | metadatas | [MetaData] | MetaData objects for input files | note that the length of input_filenames must equal length of metadatas """ # a few debug type checks assert(isinstance(output_dir, str)) assert(isinstance(format_string, str)) assert(isinstance(quality, str)) assert(len(input_filenames) == len(metadatas)) for f in input_filenames: assert(isinstance(f, audiotools.Filename)) from audiotools.text import (ERR_INVALID_FILENAME_FORMAT, LAB_OPTIONS_FILENAME_FORMAT, LAB_OUTPUT_OPTIONS, LAB_OPTIONS_OUTPUT_DIRECTORY, LAB_OPTIONS_AUDIO_CLASS, LAB_OPTIONS_AUDIO_QUALITY, LAB_OPTIONS_OUTPUT_FILES, LAB_OPTIONS_OUTPUT_FILES_1) self.input_filenames = input_filenames self.metadatas = metadatas self.selected_class = audio_class self.selected_quality = quality self.has_collisions = False # if input files same as output self.has_duplicates = False # if any track names are duplicates self.has_errors = False # if format string is invalid self.output_format = urwid.Edit( edit_text=(format_string if PY3 else format_string.decode('utf-8')), wrap='clip') urwid.connect_signal(self.output_format, 'change', self.format_changed) self.browse_fields = BrowseFields(self.output_format) self.output_directory = SelectDirectory(output_dir, self.directory_changed) self.output_tracks_frame = urwid.Frame( body=urwid.Filler(urwid.Text(u""))) output_tracks_frame_linebox = urwid.LineBox( self.output_tracks_frame, title=(LAB_OPTIONS_OUTPUT_FILES if (len(input_filenames) != 1) else LAB_OPTIONS_OUTPUT_FILES_1)) self.output_tracks = [urwid.Text(u"") for path in input_filenames] self.output_tracks_list = urwid.ListBox(self.output_tracks) self.invalid_output_format = urwid.Filler( urwid.Text(ERR_INVALID_FILENAME_FORMAT, align="center")) self.output_quality = SelectOne( items=[(u"N/A", "")], label=LAB_OPTIONS_AUDIO_QUALITY) self.output_type = SelectOne( items=sorted([(u"{} - {}".format(t.NAME, t.DESCRIPTION), t) for t in audiotools.AVAILABLE_TYPES if t.supports_from_pcm()], key=lambda pair: pair[0]), selected_value=audio_class, on_change=self.select_type, label=LAB_OPTIONS_AUDIO_CLASS) self.select_type(audio_class, quality) header = urwid.Pile( [urwid.Columns([('fixed', 10, urwid.Text(('label', u"{} : ".format( LAB_OPTIONS_OUTPUT_DIRECTORY)), align="right")), ('weight', 1, self.output_directory)]), urwid.Columns([('fixed', 10, urwid.Text(('label', u"{} : ".format( LAB_OPTIONS_FILENAME_FORMAT)), align="right")), ('weight', 1, self.output_format), ('fixed', 10, self.browse_fields)]), urwid.Columns([('fixed', 10, urwid.Text(('label', u"{} : ".format( LAB_OPTIONS_AUDIO_CLASS)), align="right")), ('weight', 1, self.output_type)]), urwid.Columns([('fixed', 10, urwid.Text(('label', u"{} : ".format( LAB_OPTIONS_AUDIO_QUALITY)), align="right")), ('weight', 1, self.output_quality)])]) widgets = [('fixed', 6, urwid.Filler(urwid.LineBox( header, title=LAB_OUTPUT_OPTIONS))), ('weight', 1, output_tracks_frame_linebox)] if extra_widgets is not None: widgets.extend(extra_widgets) urwid.Pile.__init__(self, widgets) self.update_tracks() def select_type(self, audio_class, default_quality=None): self.selected_class = audio_class if len(audio_class.COMPRESSION_MODES) < 2: # one audio quality for selected type try: quality = audio_class.COMPRESSION_MODES[0] except IndexError: # this shouldn't happen, but just in case quality = "" self.output_quality.set_items([(u"N/A", quality)], quality) else: # two or more audio qualities for selected type qualities = audio_class.COMPRESSION_MODES if default_quality is not None: default = [q for q in qualities if q == default_quality][0] else: default = [q for q in qualities if q == audio_class.DEFAULT_COMPRESSION][0] self.output_quality.set_items( [(u"{}".format(q) if q not in audio_class.COMPRESSION_DESCRIPTIONS else u"{} - {}".format( q, audio_class.COMPRESSION_DESCRIPTIONS[q]), q) for q in qualities], default) self.update_tracks() def directory_changed(self, widget, new_value): if PY3: output_directory = new_value else: output_directory = new_value.encode(audiotools.FS_ENCODING) self.update_tracks(output_directory=output_directory) def format_changed(self, widget, new_value): if PY3: filename_format = new_value else: filename_format = new_value.encode("UTF-8", "replace") self.update_tracks(filename_format=filename_format) def update_tracks(self, output_directory=None, filename_format=None): FS_ENCODING = audiotools.FS_ENCODING import os.path # get the output directory if output_directory is None: output_directory = self.output_directory.get_directory() # get selected audio format audio_class = self.selected_class # get current filename format if filename_format is None: output_format_text = self.output_format.get_edit_text() if PY3: filename_format = output_format_text else: filename_format = output_format_text.encode('utf-8') try: # generate list of Filename objects # from paths, metadatas and format # with selected output directory prepended self.output_filenames = [ audiotools.Filename( os.path.join(output_directory, audio_class.track_name(str(filename), metadata, filename_format))) for (filename, metadata) in zip(self.input_filenames, self.metadatas)] # check for duplicates in output/input files # (don't care if input files are duplicated) input_filenames = {f for f in self.input_filenames if f.disk_file()} output_filenames = set() collisions = set() self.has_collisions = False self.has_duplicates = False for path in self.output_filenames: if path in input_filenames: collisions.add(path) self.has_collisions = True elif path in output_filenames: collisions.add(path) self.has_duplicates = True else: output_filenames.add(path) # and populate output files list for (filename, track) in zip(self.output_filenames, self.output_tracks): if filename not in collisions: track.set_text(filename.__unicode__()) else: track.set_text(("duplicate", filename.__unicode__())) if ((self.output_tracks_frame.get_body() is not self.output_tracks_list)): self.output_tracks_frame.set_body( self.output_tracks_list) self.has_errors = False except (audiotools.UnsupportedTracknameField, audiotools.InvalidFilenameFormat): # if there's an error calling track_name, # populate files list with an error message if ((self.output_tracks_frame.get_body() is not self.invalid_output_format)): self.output_tracks_frame.set_body( self.invalid_output_format) self.has_errors = True def selected_options(self): """returns (AudioFile class, quality string, [output Filename]) based on selected options in the UI""" import os.path return (self.selected_class, self.output_quality.selection(), [f.expanduser() for f in self.output_filenames]) def set_metadatas(self, metadatas): """metadatas is a list of MetaData objects (some of which may be None)""" if len(metadatas) != len(self.metadatas): raise ValueError("new metadatas must have same count as old") self.metadatas = metadatas self.update_tracks() class SingleOutputOptions(urwid.ListBox): """a widget for selecting the typical set of output options for a single input file: --output, --type, --quality""" def __init__(self, output_filename, audio_class, quality): """ | field | value | meaning | |-----------------+-----------+----------------------------| | output_filename | string | default output filename | | audio_class | AudioFile | audio class of output file | | quality | string | quality level of output | """ # FIXME - add support for directory selector # FIXME - add support for field populator from audiotools.text import (LAB_OPTIONS_OUTPUT, LAB_OPTIONS_AUDIO_CLASS, LAB_OPTIONS_AUDIO_QUALITY) assert(isinstance(output_filename, str)) assert(isinstance(quality, str)) self.output_filename = EditFilename( initial_filename=output_filename) self.output_quality = SelectOne( items=[(u"N/A", "")], label=LAB_OPTIONS_AUDIO_QUALITY) self.output_type = SelectOne( items=sorted([(u"{} - {}".format(t.NAME, t.DESCRIPTION), t) for t in audiotools.AVAILABLE_TYPES if t.supports_from_pcm()], key=lambda pair: pair[0]), selected_value=audio_class, on_change=self.select_type, label=LAB_OPTIONS_AUDIO_CLASS) self.select_type(audio_class, quality) filename_widget = urwid.Columns( [('fixed', 10, urwid.Text(('label', u"{} : ".format( LAB_OPTIONS_OUTPUT)), align="right")), ('weight', 1, self.output_filename)]) class_widget = urwid.Columns( [('fixed', 10, urwid.Text(('label', u"{} : ".format(LAB_OPTIONS_AUDIO_CLASS)), align="right")), ('weight', 1, self.output_type)]) quality_widget = urwid.Columns( [('fixed', 10, urwid.Text(('label', u"{} : ".format(LAB_OPTIONS_AUDIO_QUALITY)), align="right")), ('weight', 1, self.output_quality)]) urwid.ListBox.__init__(self, [filename_widget, class_widget, quality_widget]) def set_metadata(self, metadata): """setting metadata allows output field to be populated by metadata fields""" pass # FIXME def select_type(self, audio_class, default_quality=None): self.selected_class = audio_class if len(audio_class.COMPRESSION_MODES) < 2: # one audio quality for selected type try: quality = audio_class.COMPRESSION_MODES[0] except IndexError: # this shouldn't happen, but just in case quality = "" self.output_quality.set_items([(u"N/A", quality)], quality) else: # two or more audio qualities for selected type qualities = audio_class.COMPRESSION_MODES if default_quality is not None: default = [q for q in qualities if q == default_quality][0] else: default = [q for q in qualities if q == audio_class.DEFAULT_COMPRESSION][0] self.output_quality.set_items( [((u"{}".format(q) if q not in audio_class.COMPRESSION_DESCRIPTIONS else u"{} - {}".format( q, audio_class.COMPRESSION_DESCRIPTIONS[q])), q) for q in qualities], default) def selected_options(self): """returns (AudioFile class, quality string, output Filename) based on selected options in the UI""" return (self.selected_class, self.output_quality.selection(), audiotools.Filename(self.output_filename.get_filename())) class Wizard(urwid.Frame): def __init__(self, pages, cancel_button, completion_button, page_changed=None): """pages is a list of widgets cancel_button and completion_button are Button objects to be placed at either end of the set of pages page_changed(new_page) is an optional callback when the current page is changed""" assert(len(pages) > 0) from audiotools.text import (LAB_PREVIOUS_BUTTON, LAB_NEXT_BUTTON) self.__body_pages__ = [] for (i, widget) in enumerate(pages): if i == 0: previous_button = cancel_button else: previous_button = urwid.Button( LAB_PREVIOUS_BUTTON, on_press=self.set_page, user_data=(i - 1, pages[i - 1], page_changed)) if i == (len(pages) - 1): next_button = completion_button else: next_button = urwid.Button( LAB_NEXT_BUTTON, on_press=self.set_page, user_data=(i + 1, pages[i + 1], page_changed)) metadata_buttons = urwid.Filler( urwid.Columns(widget_list=[('weight', 1, previous_button), ('weight', 2, next_button)], dividechars=3, focus_column=1)) self.__body_pages__.append(urwid.Pile( [("weight", 1, widget), ("fixed", 1, metadata_buttons)], focus_item=1)) urwid.Frame.__init__(self, body=self.__body_pages__[0]) def set_page(self, button, user_data): (page_widget_index, base_widget, page_changed) = user_data self.set_body(self.__body_pages__[page_widget_index]) if page_changed is not None: page_changed(base_widget) class MappedButton(urwid.Button): def __init__(self, label, on_press=None, user_data=None, key_map={}): urwid.Button.__init__(self, label=label, on_press=on_press, user_data=user_data) self.__key_map__ = key_map def keypress(self, size, key): return urwid.Button.keypress(self, size, self.__key_map__.get(key, key)) class MappedRadioButton(urwid.RadioButton): def __init__(self, group, label, state='first True', on_state_change=None, user_data=None, key_map={}): urwid.RadioButton.__init__(self, group=group, label=label, state=state, on_state_change=on_state_change, user_data=user_data) self.__key_map__ = key_map def keypress(self, size, key): return urwid.RadioButton.keypress(self, size, self.__key_map__.get(key, key)) class AudioProgressBar(urwid.ProgressBar): def __init__(self, normal, complete, sample_rate, current=0, done=100, satt=None): urwid.ProgressBar.__init__(self, normal=normal, complete=complete, current=current, done=done, satt=satt) self.sample_rate = sample_rate def get_text(self): from audiotools.text import LAB_TRACK_LENGTH try: return LAB_TRACK_LENGTH.format( (self.current // self.sample_rate) // 60, (self.current // self.sample_rate) % 60) except ZeroDivisionError: return LAB_TRACK_LENGTH.format(0, 0) class VolumeControl(urwid.ProgressBar): def __init__(self, get_volume, volume_delta, change_volume): """get_volume is a function that returns the current volume volume_delta is the delta as a float change_volume is a function that takes a delta and returns the new volume""" urwid.ProgressBar.__init__(self, normal="volume normal", complete="volume complete", current=0.0, done=1.0) self.get_volume = get_volume self.volume_delta = volume_delta self.change_volume = change_volume self.set_completion(self.get_volume()) def update(self): self.set_completion(self.get_volume()) def get_text(self): from audiotools.text import LAB_VOLUME return u"{} : {}".format( LAB_VOLUME, urwid.ProgressBar.get_text(self)) def selectable(self): return True def get_cursor_coords(self, size): return (0, 0) def keypress(self, size, key): if (key == 'left') and (self.decrease_volume is not None): self.decrease_volume() elif (key == 'right') and (self.increase_volume is not None): self.increase_volume() else: return key def render(self, size, focus=False): c = urwid.ProgressBar.render(self, size, focus) if focus: # create a new canvas so we can add a cursor c = urwid.CompositeCanvas(c) c.cursor = self.get_cursor_coords(size) return c def decrease_volume(self): self.update_volume(-self.volume_delta) def increase_volume(self): self.update_volume(self.volume_delta) def update_volume(self, delta): self.set_completion(self.change_volume(delta)) class AdjustOutput(urwid.PopUpLauncher): def __init__(self, player, volume_control): from audiotools.text import LAB_ADJUST_OUTPUT self.__output_button__ = urwid.Button(LAB_ADJUST_OUTPUT) self.__super.__init__(self.__output_button__) urwid.connect_signal( self.original_widget, 'click', lambda button: self.open_pop_up()) self.player = player self.volume_control = volume_control self.outputs = [] def create_pop_up(self): from audiotools.player import available_outputs self.outputs = list(available_outputs()) pop_up = AdjustOutputDialog(self.player, self.volume_control, self.outputs) urwid.connect_signal( pop_up, 'close', lambda button: self.close_pop_up()) return pop_up def get_pop_up_parameters(self): return {'left': 0, 'top': 1, 'overlay_width': 50, 'overlay_height': len(self.outputs) + 4} class AdjustOutputWidget(urwid.Pile): def __init__(self, player, volume_control, outputs, closed_callback=None): from audiotools.text import (LAB_DECREASE_VOLUME, LAB_INCREASE_VOLUME, LAB_KEY_DONE) self.player = player self.closed_callback = closed_callback self.volume_control = volume_control current_output_name = player.current_output_name() output_group = [] urwid.Pile.__init__( self, [self.volume_control] + [urwid.RadioButton(group=output_group, label=o.description(), state=current_output_name == o.NAME, on_state_change=self.change_output, user_data=o) for o in outputs] + [urwid.Text([('key', u" \u2190 "), LAB_DECREASE_VOLUME, u" ", ('key', u" \u2192 "), LAB_INCREASE_VOLUME, u" ", ('key', 'esc'), LAB_KEY_DONE])]) def keypress(self, size, key): key = urwid.Pile.keypress(self, size, key) if key == 'esc': if self.closed_callback is not None: self.closed_callback() else: return key def change_output(self, radio_button, new_state, new_output): if new_state: self.player.set_output(new_output) self.volume_control.update() class AdjustOutputDialog(urwid.WidgetWrap): signals = ['close'] def __init__(self, player, volume_control, outputs): from audiotools.text import LAB_OUTPUT_OPTIONS close_button = urwid.Button(u"done") pile = AdjustOutputWidget( player, volume_control, outputs, closed_callback=lambda: self._emit("close")) self.__super.__init__(urwid.LineBox(urwid.Filler(pile), title=LAB_OUTPUT_OPTIONS)) class PlayerGUI(urwid.Frame): def __init__(self, player, tracks, track_len): """player is a Player-compatible object tracks is a list of (track_name, seconds_length, user_data) tuples where "track_name" is a unicode string to place in the radio buttons "seconds_length" is the length of the track in seconds and "user_data" is forwarded to calls to select_track() track_len is the length of all tracks in seconds""" from audiotools.text import (LAB_PLAY_BUTTON, METADATA_TRACK_NAME, METADATA_ARTIST_NAME, METADATA_ALBUM_NAME, LAB_PLAY_TRACK, LAB_PREVIOUS_BUTTON, LAB_NEXT_BUTTON, LAB_PLAY_STATUS, LAB_PLAY_STATUS_1, LAB_ALBUM_NUMBER) self.player = player self.track_name = urwid.Text(u"") self.album_name = urwid.Text(u"") self.artist_name = urwid.Text(u"") self.tracknum = urwid.Text(u"") self.albumnum_label = urwid.Text(u"") self.albumnum = urwid.Text(u"") self.play_pause_button = MappedButton(LAB_PLAY_BUTTON, on_press=self.play_pause, key_map={'tab': 'right'}) self.progress = AudioProgressBar(normal='pg normal', complete='pg complete', sample_rate=0, current=0, done=100) self.volume_control = VolumeControl(player.get_volume, 0.01, player.change_volume) label_width = max([len(audiotools.output_text(s)) for s in [METADATA_TRACK_NAME, METADATA_ARTIST_NAME, METADATA_ALBUM_NAME, LAB_PLAY_TRACK]]) + 3 track_name_widget = urwid.Columns( [('fixed', label_width, urwid.Text(('label', u"{} : ".format(METADATA_TRACK_NAME)), align='right')), ('weight', 1, self.track_name)]) artist_name_widget = urwid.Columns( [('fixed', label_width, urwid.Text(('label', u"{} : ".format(METADATA_ARTIST_NAME)), align='right')), ('weight', 1, self.artist_name)]) album_name_widget = urwid.Columns( [('fixed', label_width, urwid.Text(('label', u"{} : ".format(METADATA_ALBUM_NAME)), align='right')), ('weight', 1, self.album_name)]) track_number_widget = urwid.Columns( [('fixed', label_width, urwid.Text(('label', u"{} : ".format(LAB_PLAY_TRACK)), align='right')), ('weight', 1, self.tracknum)]) album_number_widget = urwid.Columns( [('fixed', len(LAB_ALBUM_NUMBER) + 3, self.albumnum_label), ('weight', 1, self.albumnum)]) track_album_number = urwid.Columns( [('weight', 1, track_number_widget), ('weight', 1, album_number_widget)]) header = urwid.Pile([track_name_widget, artist_name_widget, album_name_widget, track_album_number, self.progress]) controls = urwid.Columns( [("fixed", 10, urwid.Divider(u" ")), ("weight", 1, urwid.Divider(u" ")), ("fixed", 9, MappedButton(LAB_PREVIOUS_BUTTON, on_press=self.previous_track, key_map={"tab": "right"})), ("fixed", 9, self.play_pause_button), ("fixed", 9, MappedButton(LAB_NEXT_BUTTON, on_press=self.next_track, key_map={"tab": "down"})), ("weight", 1, urwid.Divider(u" ")), ("fixed", 10, AdjustOutput(self.player, self.volume_control))], dividechars=2, focus_column=3) self.track_group = [] self.track_list_widget = urwid.ListBox( [urwid.Columns( [("weight", 1, MappedRadioButton(group=self.track_group, label=track_label, user_data=user_data, on_state_change=self.select_track, key_map={'tab': 'down'})), ("fixed", 6, urwid.Text(u"{:2d}:{:02d}".format( seconds_length // 60, seconds_length % 60), align="right"))]) for (track_label, seconds_length, user_data) in tracks]) status = ((LAB_PLAY_STATUS if (len(tracks) > 1) else LAB_PLAY_STATUS_1).format(count=len(tracks), min=int(track_len) // 60, sec=int(track_len) % 60)) body = urwid.Pile( [("fixed", 1, urwid.Filler(controls)), ("weight", 1, BottomLineBox(self.track_list_widget, title=status))]) urwid.Frame.__init__( self, body=body, header=urwid.LineBox(header)) def select_track(self, radio_button, new_state, user_data, auto_play=True): # to be implemented by subclasses raise NotImplementedError() def next_track(self, user_data=None): track_index = [g.state for g in self.track_group].index(True) try: self.track_group[track_index + 1].set_state(True) self.track_list_widget.set_focus(track_index + 1, "above") except IndexError: if len(self.track_group): self.track_group[0].set_state(True) self.track_list_widget.set_focus(0, "above") self.player.stop() def previous_track(self, user_data=None): track_index = [g.state for g in self.track_group].index(True) try: if track_index == 0: raise IndexError() else: self.track_group[track_index - 1].set_state(True) self.track_list_widget.set_focus(track_index - 1, "below") except IndexError: pass def update_metadata(self, track_name=None, album_name=None, artist_name=None, track_number=None, track_total=None, album_number=None, album_total=None, pcm_frames=0, channels=0, sample_rate=0, bits_per_sample=0): """updates metadata fields with new values for when a new track is opened""" from audiotools.text import (LAB_X_OF_Y, LAB_ALBUM_NUMBER) self.track_name.set_text(track_name if track_name is not None else u"") self.album_name.set_text(album_name if album_name is not None else u"") self.artist_name.set_text(artist_name if artist_name is not None else u"") if track_number is not None: if track_total is not None: self.tracknum.set_text( LAB_X_OF_Y.format(track_number, track_total)) else: self.tracknum.set_text(u"{:d}".format(track_number)) else: self.tracknum.set_text(u"") if album_number is not None: if album_total is not None: self.albumnum_label.set_text(('label', LAB_ALBUM_NUMBER + u" : ")) self.albumnum.set_text( LAB_X_OF_Y.format(album_number, album_total)) else: self.albumnum_label.set_text(('label', LAB_ALBUM_NUMBER + u" : ")) self.albumnum.set_text(u"{:d}".format(album_number)) else: self.albumnum_label.set_text(u"") self.albumnum.set_text(u"") try: seconds_length = pcm_frames // sample_rate except ZeroDivisionError: seconds_length = 0 self.progress.current = 0 self.progress.done = max(pcm_frames, 1) self.progress.sample_rate = sample_rate def update_status(self): from audiotools.text import (LAB_PLAY_BUTTON, LAB_PAUSE_BUTTON) from audiotools.player import PLAYER_PLAYING self.progress.set_completion(self.player.progress()[0]) self.volume_control.update() if self.player.state() == PLAYER_PLAYING: self.play_pause_button.set_label(LAB_PAUSE_BUTTON) else: self.play_pause_button.set_label(LAB_PLAY_BUTTON) def play_pause(self, user_data): from audiotools.player import PLAYER_STOPPED if self.player.state() == PLAYER_STOPPED: self.player.play() else: # there's a race condition here where the player's state # may go from playing to stopped, in which case # this will do nothing and the state will stay stopped self.player.toggle_play_pause() self.update_status() def stop(self): self.player.stop() self.update_status() def handle_text(self, i): if (i == 'esc') or (i == 'q') or (i == 'Q'): self.perform_exit() elif (i == 'n') or (i == 'N'): self.next_track() elif (i == 'p') or (i == 'P'): self.previous_track() elif (i == 's') or (i == 'S'): self.stop() def perform_exit(self, *args): self.player.close() raise urwid.ExitMainLoop() def timer(main_loop, playergui): import time playergui.update_status() main_loop.set_alarm_at(tm=time.time() + 0.25, callback=timer, user_data=playergui) def style(): """returns a list of widget style tuples for use with urwid.MainLoop""" return [('key', 'white', 'dark blue'), ('label', 'default,bold', 'default'), ('modified', 'default,bold', 'default', ''), ('duplicate', 'light red', 'default'), ('error', 'light red,bold', 'default'), ('pg normal', 'white', 'black', 'standout'), ('pg complete', 'white', 'dark blue'), ('pg smooth', 'dark blue', 'black'), ('volume normal', 'white', 'black'), ('volume complete', 'white', 'dark blue')] except ImportError: AVAILABLE = False def show_available_formats(msg): """given a Messenger object, display all the available file formats""" from audiotools import output_table, output_text from audiotools.text import (LAB_AVAILABLE_FORMATS, LAB_OUTPUT_TYPE, LAB_OUTPUT_TYPE_DESCRIPTION) msg.output(LAB_AVAILABLE_FORMATS) msg.output(u"") table = output_table() row = table.row() row.add_column(LAB_OUTPUT_TYPE, "right") row.add_column(u" ") row.add_column(LAB_OUTPUT_TYPE_DESCRIPTION) table.divider_row([u"-", u" ", u"-"]) for name in sorted(audiotools.TYPE_MAP.keys()): row = table.row() row.add_column( output_text(u"{}".format(name), style=("underline" if (name == audiotools.DEFAULT_TYPE) else None)), "right") row.add_column(u" : ") row.add_column(audiotools.TYPE_MAP[name].DESCRIPTION) for row in table.format(msg.output_isatty()): msg.output(row) def show_available_qualities(msg, audiotype): """given a Messenger object and AudioFile class, display all available quality types for that format""" from audiotools import output_table, output_text from audiotools.text import (LAB_AVAILABLE_COMPRESSION_TYPES, LAB_OUTPUT_QUALITY_DESCRIPTION, LAB_OUTPUT_QUALITY, ERR_NO_COMPRESSION_MODES) if len(audiotype.COMPRESSION_MODES) > 1: msg.info(LAB_AVAILABLE_COMPRESSION_TYPES.format(audiotype.NAME)) msg.info(u"") table = audiotools.output_table() row = table.row() row.add_column(LAB_OUTPUT_QUALITY, "right") row.add_column(u" ") row.add_column(LAB_OUTPUT_QUALITY_DESCRIPTION) table.divider_row([u"-", u" ", u"-"]) for mode in audiotype.COMPRESSION_MODES: row = table.row() row.add_column( output_text(u"{}".format(mode), style=("underline" if (mode == audiotools.__default_quality__( audiotype.NAME)) else None)), "right") if mode in audiotype.COMPRESSION_DESCRIPTIONS: row.add_column(u" : ") row.add_column(audiotype.COMPRESSION_DESCRIPTIONS[mode]) else: row.add_column(u"") row.add_column(u"") for row in table.format(msg.info_isatty()): msg.info(row) else: msg.info(ERR_NO_COMPRESSION_MODES.format(audiotype.NAME)) def select_metadata(metadata_choices, msg, use_default=False): """queries the user for the best matching metadata to use returns a list of MetaData objects for the selected choice""" # there must be at least one choice assert(len(metadata_choices) > 0) # all choices must have at least 1 track assert(min(map(len, metadata_choices)) > 0) # and all choices must have the same number of tracks assert(len(set(map(len, metadata_choices))) == 1) if (len(metadata_choices) == 1) or use_default: return metadata_choices[0] else: choice = None while choice not in range(0, len(metadata_choices)): from audiotools.text import LAB_SELECT_BEST_MATCH for (i, choice) in enumerate(metadata_choices): msg.output(u"{choice}) {selection}".format( choice=i + 1, selection=choice_selection_unicode(choice[0]))) try: choice = int(raw_input(u"{} (1-{:d}) : ".format( LAB_SELECT_BEST_MATCH, len(metadata_choices)))) - 1 except ValueError: choice = None return metadata_choices[choice] def process_output_options(metadata_choices, input_filenames, output_directory, format_string, output_class, quality, msg, use_default=False): """metadata_choices[c][t] is a MetaData object for choice number "c" and track number "t" all choices must have the same number of tracks input_filenames is a list of Filename objects for input files the number of input files must equal the number of metadata objects in each metadata choice output_directory is a string of the default output dir format_string is a UTF-8 encoded format string, or None output_class is the default AudioFile-compatible class quality is a string of the default output quality to use msg is a Messenger object this may take user input from the prompt to select a MetaData choice after which it yields (output_class, output_filename, output_quality, output_metadata) tuple for each input file output_metadata is a reference to an object in metadata_choices may raise UnsupportedTracknameField or InvalidFilenameFormat if the given format_string is invalid may raise DuplicateOutputFile if the same output file is generated more than once may raise OutputFileIsInput if an output file is the same as one of the given input_filenames""" # there must be at least one choice assert(len(metadata_choices) > 0) # ensure input filename count is equal to metadata track count assert(len(metadata_choices[0]) == len(input_filenames)) import os.path selected_metadata = select_metadata(metadata_choices, msg, use_default) # ensure no output paths overwrite input paths # and that all output paths are distinct __input__ = {f for f in input_filenames if f.disk_file()} __output__ = set() output_filenames = [] for (input_filename, metadata) in zip(input_filenames, selected_metadata): output_filename = audiotools.Filename( os.path.join(output_directory, output_class.track_name(str(input_filename), metadata, format_string))) if output_filename in __input__: raise audiotools.OutputFileIsInput(output_filename) elif output_filename in __output__: raise audiotools.DuplicateOutputFile(output_filename) else: __output__.add(output_filename) output_filenames.append(output_filename) for (output_filename, metadata) in zip(output_filenames, selected_metadata): yield (output_class, output_filename, quality, metadata) class PlayerTTY(object): OUTPUT_FORMAT = (u"{track_number:d}/{track_total:d} " + u"[{sent_minutes:d}:{sent_seconds:02d} / " + u"{total_minutes:d}:{total_seconds:02d}] " + u"{channels:d}ch {sample_rate} " + u"{bits_per_sample:d}-bit") def __init__(self, player): self.player = player self.track_number = 0 self.track_total = 0 self.channels = 0 self.sample_rate = 0 self.bits_per_sample = 0 self.playing_finished = False def previous_track(self): # implement this in subclass # to call set_metadata() and have the player open the previous track raise NotImplementedError() def next_track(self): # implement this in subclass # to call set_metadata() and have the player open the next track # set playing_finished to True when all tracks have been exhausted raise NotImplementedError() def set_metadata(self, track_number, track_total, channels, sample_rate, bits_per_sample): self.track_number = track_number self.track_total = track_total self.channels = channels self.sample_rate = sample_rate self.bits_per_sample = bits_per_sample def toggle_play_pause(self): self.player.toggle_play_pause() def stop(self): self.player.stop() def run(self, msg, stdin): """msg is a Messenger object stdin is a file object for keyboard input returns 0 on a successful exit, 1 if there's an error""" import os import tty import termios import select try: original_terminal_settings = termios.tcgetattr(0) except termios.error: from audiotools.text import (ERR_TERMIOS_ERROR, ERR_TERMIOS_SUGGESTION) msg.error(ERR_TERMIOS_ERROR) msg.info(ERR_TERMIOS_SUGGESTION) return 1 output_line_len = 0 self.next_track() try: tty.setcbreak(stdin.fileno()) try: while not self.playing_finished: (frames_sent, frames_total) = self.progress() output_line = self.progress_line(frames_sent, frames_total) msg.ansi_clearline() if len(output_line) > output_line_len: output_line_len = len(output_line) msg.partial_output(output_line) else: msg.partial_output(output_line + (u" " * (output_line_len - len(output_line)))) (r_list, w_list, x_list) = select.select([stdin.fileno()], [], [], 1) if len(r_list) > 0: char = os.read(stdin.fileno(), 1) if (((char == b'q') or (char == b'Q') or (char == b'\x1B'))): self.playing_finished = True elif char == b' ': self.toggle_play_pause() elif ((char == b'n') or (char == b'N')): self.next_track() elif ((char == b'p') or (char == b'P')): self.previous_track() elif ((char == b's') or (char == b'S')): self.stop() else: pass except KeyboardInterrupt: pass msg.ansi_clearline() self.player.close() return 0 finally: termios.tcsetattr(0, termios.TCSADRAIN, original_terminal_settings) def progress(self): return self.player.progress() def progress_line(self, frames_sent, frames_total): return self.OUTPUT_FORMAT.format( track_number=self.track_number, track_total=self.track_total, sent_minutes=(frames_sent // self.sample_rate) // 60, sent_seconds=(frames_sent // self.sample_rate) % 60, total_minutes=(frames_total // self.sample_rate) // 60, total_seconds=(frames_total // self.sample_rate) % 60, channels=self.channels, sample_rate=audiotools.khz(self.sample_rate), bits_per_sample=self.bits_per_sample) def not_available_message(msg): """prints a message about lack of Urwid availability to a Messenger object""" from audiotools.text import (ERR_URWID_REQUIRED, ERR_GET_URWID1, ERR_GET_URWID2) msg.error(ERR_URWID_REQUIRED) msg.output(ERR_GET_URWID1) msg.output(ERR_GET_URWID2) def xargs_suggestion(args): """converts arguments to xargs-compatible suggestion and returns unicode string""" import os.path # All command-line arguments start with "-" # but not everything that starts with "-" is a command-line argument. # However, since this is just a suggestion, # users can be expected to figure it out. return (u"xargs sh -c '%s %s \"%%@\" < /dev/tty'" % (os.path.basename(args[0]).decode('utf-8'), " ".join([arg.decode('utf-8') for arg in args[1:] if arg.startswith("-")]))) ================================================ FILE: audiotools/vorbis.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools import (AudioFile, InvalidFile, ChannelMask) class InvalidVorbis(InvalidFile): pass class VorbisAudio(AudioFile): """an Ogg Vorbis file""" from audiotools.text import (COMP_VORBIS_0, COMP_VORBIS_10) SUFFIX = "ogg" NAME = SUFFIX DESCRIPTION = u"Ogg Vorbis" DEFAULT_COMPRESSION = "3" COMPRESSION_MODES = tuple([str(i) for i in range(0, 11)]) COMPRESSION_DESCRIPTIONS = {"0": COMP_VORBIS_0, "10": COMP_VORBIS_10} def __init__(self, filename): """filename is a plain string""" AudioFile.__init__(self, filename) self.__sample_rate__ = 0 self.__channels__ = 0 try: self.__read_identification__() except IOError as msg: raise InvalidVorbis(str(msg)) def __read_identification__(self): from audiotools.bitstream import BitstreamReader with BitstreamReader(open(self.filename, "rb"), True) as ogg_reader: (magic_number, version, header_type, granule_position, self.__serial_number__, page_sequence_number, checksum, segment_count) = ogg_reader.parse("4b 8u 8u 64S 32u 32u 32u 8u") if magic_number != b'OggS': from audiotools.text import ERR_OGG_INVALID_MAGIC_NUMBER raise InvalidVorbis(ERR_OGG_INVALID_MAGIC_NUMBER) if version != 0: from audiotools.text import ERR_OGG_INVALID_VERSION raise InvalidVorbis(ERR_OGG_INVALID_VERSION) segment_length = ogg_reader.read(8) (vorbis_type, header, version, self.__channels__, self.__sample_rate__, maximum_bitrate, nominal_bitrate, minimum_bitrate, blocksize0, blocksize1, framing) = ogg_reader.parse( "8u 6b 32u 8u 32u 32u 32u 32u 4u 4u 1u") if vorbis_type != 1: from audiotools.text import ERR_VORBIS_INVALID_TYPE raise InvalidVorbis(ERR_VORBIS_INVALID_TYPE) if header != b'vorbis': from audiotools.text import ERR_VORBIS_INVALID_HEADER raise InvalidVorbis(ERR_VORBIS_INVALID_HEADER) if version != 0: from audiotools.text import ERR_VORBIS_INVALID_VERSION raise InvalidVorbis(ERR_VORBIS_INVALID_VERSION) if framing != 1: from audiotools.text import ERR_VORBIS_INVALID_FRAMING_BIT raise InvalidVorbis(ERR_VORBIS_INVALID_FRAMING_BIT) def lossless(self): """returns False""" return False def bits_per_sample(self): """returns an integer number of bits-per-sample this track contains""" return 16 def channels(self): """returns an integer number of channels this track contains""" return self.__channels__ def channel_mask(self): """returns a ChannelMask object of this track's channel layout""" if self.channels() == 1: return ChannelMask.from_fields( front_center=True) elif self.channels() == 2: return ChannelMask.from_fields( front_left=True, front_right=True) elif self.channels() == 3: return ChannelMask.from_fields( front_left=True, front_right=True, front_center=True) elif self.channels() == 4: return ChannelMask.from_fields( front_left=True, front_right=True, back_left=True, back_right=True) elif self.channels() == 5: return ChannelMask.from_fields( front_left=True, front_right=True, front_center=True, back_left=True, back_right=True) elif self.channels() == 6: return ChannelMask.from_fields( front_left=True, front_right=True, front_center=True, back_left=True, back_right=True, low_frequency=True) elif self.channels() == 7: return ChannelMask.from_fields( front_left=True, front_right=True, front_center=True, side_left=True, side_right=True, back_center=True, low_frequency=True) elif self.channels() == 8: return ChannelMask.from_fields( front_left=True, front_right=True, side_left=True, side_right=True, back_left=True, back_right=True, front_center=True, low_frequency=True) else: return ChannelMask(0) def total_frames(self): """returns the total PCM frames of the track as an integer""" from audiotools._ogg import PageReader try: with PageReader(open(self.filename, "rb")) as reader: page = reader.read() pcm_samples = page.granule_position while not page.stream_end: page = reader.read() pcm_samples = max(pcm_samples, page.granule_position) return pcm_samples except (IOError, ValueError): return 0 def sample_rate(self): """returns the rate of the track's audio as an integer number of Hz""" return self.__sample_rate__ @classmethod def supports_to_pcm(cls): """returns True if all necessary components are available to support the .to_pcm() method""" try: from audiotools.decoders import VorbisDecoder return True except ImportError: return False def to_pcm(self): """returns a PCMReader object containing the track's PCM data""" from audiotools.decoders import VorbisDecoder try: return VorbisDecoder(self.filename) except ValueError as err: from audiotools import PCMReaderError return PCMReaderError(str(err), self.sample_rate(), self.channels(), int(self.channel_mask()), self.bits_per_sample()) @classmethod def supports_from_pcm(cls): """returns True if all necessary components are available to support the .from_pcm() classmethod""" try: from audiotools.encoders import encode_vorbis return True except ImportError: return False @classmethod def from_pcm(cls, filename, pcmreader, compression=None, total_pcm_frames=None): """encodes a new file from PCM data takes a filename string, PCMReader object, optional compression level string and optional total_pcm_frames integer encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new VorbisAudio object""" from audiotools import (BufferedPCMReader, __default_quality__, EncodingError) from audiotools.encoders import encode_vorbis if (((compression is None) or (compression not in cls.COMPRESSION_MODES))): compression = __default_quality__(cls.NAME) if pcmreader.bits_per_sample not in {8, 16, 24}: from audiotools import UnsupportedBitsPerSample pcmreader.close() raise UnsupportedBitsPerSample(filename, pcmreader.bits_per_sample) if (pcmreader.channels > 2) and (pcmreader.channels <= 8): channel_mask = int(pcmreader.channel_mask) if ((channel_mask != 0) and (channel_mask not in (0x7, # FR, FC, FL 0x33, # FR, FL, BR, BL 0x37, # FR, FC, FL, BL, BR 0x3f, # FR, FC, FL, BL, BR, LFE 0x70f, # FL, FC, FR, SL, SR, BC, LFE 0x63f))): # FL, FC, FR, SL, SR, BL, BR, LFE from audiotools import UnsupportedChannelMask pcmreader.close() raise UnsupportedChannelMask(filename, channel_mask) if total_pcm_frames is not None: from audiotools import CounterPCMReader pcmreader = CounterPCMReader(pcmreader) try: encode_vorbis(filename, pcmreader, float(compression) / 10) if ((total_pcm_frames is not None) and (total_pcm_frames != pcmreader.frames_written)): from audiotools.text import ERR_TOTAL_PCM_FRAMES_MISMATCH cls.__unlink__(filename) raise EncodingError(ERR_TOTAL_PCM_FRAMES_MISMATCH) return VorbisAudio(filename) except (ValueError, IOError) as err: cls.__unlink__(filename) raise EncodingError(str(err)) finally: pcmreader.close() def update_metadata(self, metadata): """takes this track's current MetaData object as returned by get_metadata() and sets this track's metadata with any fields updated in that object raises IOError if unable to write the file """ import os from audiotools import TemporaryFile from audiotools.ogg import (PageReader, PacketReader, PageWriter, packet_to_pages, packets_to_pages) from audiotools.vorbiscomment import VorbisComment from audiotools.bitstream import BitstreamRecorder if metadata is None: return elif not isinstance(metadata, VorbisComment): from audiotools.text import ERR_FOREIGN_METADATA raise ValueError(ERR_FOREIGN_METADATA) elif not os.access(self.filename, os.W_OK): raise IOError(self.filename) original_ogg = PacketReader(PageReader(open(self.filename, "rb"))) new_ogg = PageWriter(TemporaryFile(self.filename)) sequence_number = 0 # transfer current file's identification packet in its own page identification_packet = original_ogg.read_packet() for (i, page) in enumerate(packet_to_pages( identification_packet, self.__serial_number__, starting_sequence_number=sequence_number)): page.stream_beginning = (i == 0) new_ogg.write(page) sequence_number += 1 # discard the current file's comment packet comment_packet = original_ogg.read_packet() # generate new comment packet comment_writer = BitstreamRecorder(True) comment_writer.build("8u 6b", (3, b"vorbis")) vendor_string = metadata.vendor_string.encode('utf-8') comment_writer.build("32u {:d}b".format(len(vendor_string)), (len(vendor_string), vendor_string)) comment_writer.write(32, len(metadata.comment_strings)) for comment_string in metadata.comment_strings: comment_string = comment_string.encode('utf-8') comment_writer.build("32u {:d}b".format(len(comment_string)), (len(comment_string), comment_string)) comment_writer.build("1u a", (1,)) # framing bit # transfer codebooks packet from original file to new file codebooks_packet = original_ogg.read_packet() for page in packets_to_pages( [comment_writer.data(), codebooks_packet], self.__serial_number__, starting_sequence_number=sequence_number): new_ogg.write(page) sequence_number += 1 # transfer remaining pages after re-sequencing page = original_ogg.read_page() page.sequence_number = sequence_number page.bitstream_serial_number = self.__serial_number__ sequence_number += 1 new_ogg.write(page) while not page.stream_end: page = original_ogg.read_page() page.sequence_number = sequence_number page.bitstream_serial_number = self.__serial_number__ sequence_number += 1 new_ogg.write(page) original_ogg.close() new_ogg.close() def set_metadata(self, metadata): """takes a MetaData object and sets this track's metadata this metadata includes track name, album name, and so on raises IOError if unable to write the file""" from audiotools.vorbiscomment import VorbisComment if metadata is None: return self.delete_metadata() metadata = VorbisComment.converted(metadata) old_metadata = self.get_metadata() metadata.vendor_string = old_metadata.vendor_string # remove REPLAYGAIN_* tags from new metadata (if any) for key in [u"REPLAYGAIN_TRACK_GAIN", u"REPLAYGAIN_TRACK_PEAK", u"REPLAYGAIN_ALBUM_GAIN", u"REPLAYGAIN_ALBUM_PEAK", u"REPLAYGAIN_REFERENCE_LOUDNESS"]: try: metadata[key] = old_metadata[key] except KeyError: metadata[key] = [] self.update_metadata(metadata) @classmethod def supports_metadata(cls): """returns True if this audio type supports MetaData""" return True def get_metadata(self): """returns a MetaData object, or None raises IOError if unable to read the file""" from io import BytesIO from audiotools.bitstream import BitstreamReader from audiotools.ogg import PacketReader, PageReader from audiotools.vorbiscomment import VorbisComment with PacketReader(PageReader(open(self.filename, "rb"))) as reader: identification = reader.read_packet() comment = BitstreamReader(BytesIO(reader.read_packet()), True) (packet_type, packet_header) = comment.parse("8u 6b") if (packet_type == 3) and (packet_header == b'vorbis'): vendor_string = \ comment.read_bytes(comment.read(32)).decode('utf-8') comment_strings = [ comment.read_bytes(comment.read(32)).decode('utf-8') for i in range(comment.read(32))] if comment.read(1) == 1: # framing bit return VorbisComment(comment_strings, vendor_string) else: return None else: return None def delete_metadata(self): """deletes the track's MetaData this removes or unsets tags as necessary in order to remove all data raises IOError if unable to write the file""" from audiotools import MetaData # the vorbis comment packet is required, # so simply zero out its contents self.set_metadata(MetaData()) @classmethod def supports_replay_gain(cls): """returns True if this class supports ReplayGain""" return True def get_replay_gain(self): """returns a ReplayGain object of our ReplayGain values returns None if we have no values""" from audiotools import ReplayGain vorbis_metadata = self.get_metadata() if ((vorbis_metadata is not None) and ({u'REPLAYGAIN_TRACK_PEAK', u'REPLAYGAIN_TRACK_GAIN', u'REPLAYGAIN_ALBUM_PEAK', u'REPLAYGAIN_ALBUM_GAIN'}.issubset(vorbis_metadata.keys()))): # we have ReplayGain data try: return ReplayGain( vorbis_metadata[u'REPLAYGAIN_TRACK_GAIN'][0][0:-len(u" dB")], vorbis_metadata[u'REPLAYGAIN_TRACK_PEAK'][0], vorbis_metadata[u'REPLAYGAIN_ALBUM_GAIN'][0][0:-len(u" dB")], vorbis_metadata[u'REPLAYGAIN_ALBUM_PEAK'][0]) except (IndexError, ValueError): return None else: return None def set_replay_gain(self, replaygain): """given a ReplayGain object, sets the track's gain to those values may raise IOError if unable to modify the file""" if replaygain is None: return self.delete_replay_gain() vorbis_comment = self.get_metadata() if vorbis_comment is None: from audiotools.vorbiscomment import VorbisComment from audiotools import VERSION vorbis_comment = VorbisComment( [], u"Python Audio Tools {}".format(VERSION)) vorbis_comment[u"REPLAYGAIN_TRACK_GAIN"] = [ u"{:.2f} dB".format(replaygain.track_gain)] vorbis_comment[u"REPLAYGAIN_TRACK_PEAK"] = [ u"{:.8f}".format(replaygain.track_peak)] vorbis_comment[u"REPLAYGAIN_ALBUM_GAIN"] = [ u"{:.2f} dB".format(replaygain.album_gain)] vorbis_comment[u"REPLAYGAIN_ALBUM_PEAK"] = [ u"{:.8f}".format(replaygain.album_peak)] vorbis_comment[u"REPLAYGAIN_REFERENCE_LOUDNESS"] = [u"89.0 dB"] self.update_metadata(vorbis_comment) def delete_replay_gain(self): """removes ReplayGain values from file, if any may raise IOError if unable to modify the file""" vorbis_comment = self.get_metadata() if vorbis_comment is not None: for field in [u"REPLAYGAIN_TRACK_GAIN", u"REPLAYGAIN_TRACK_PEAK", u"REPLAYGAIN_ALBUM_GAIN", u"REPLAYGAIN_ALBUM_PEAK", u"REPLAYGAIN_REFERENCE_LOUDNESS"]: try: del(vorbis_comment[field]) except KeyError: pass self.update_metadata(vorbis_comment) class VorbisChannelMask(ChannelMask): """the Vorbis-specific channel mapping""" def __repr__(self): return "VorbisChannelMask({})".format( ",".join(["{}={}".format(field, getattr(self, field)) for field in self.SPEAKER_TO_MASK.keys() if (getattr(self, field))])) def channels(self): """returns a list of speaker strings this mask contains returned in the order in which they should appear in the PCM stream """ count = len(self) if count == 1: return ["front_center"] elif count == 2: return ["front_left", "front_right"] elif count == 3: return ["front_left", "front_center", "front_right"] elif count == 4: return ["front_left", "front_right", "back_left", "back_right"] elif count == 5: return ["front_left", "front_center", "front_right", "back_left", "back_right"] elif count == 6: return ["front_left", "front_center", "front_right", "back_left", "back_right", "low_frequency"] elif count == 7: return ["front_left", "front_center", "front_right", "side_left", "side_right", "back_center", "low_frequency"] elif count == 8: return ["front_left", "front_center", "front_right", "side_left", "side_right", "back_left", "back_right", "low_frequency"] else: return [] ================================================ FILE: audiotools/vorbiscomment.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools import MetaData import re class VorbisComment(MetaData): ATTRIBUTE_MAP = {'track_name': u'TITLE', 'track_number': u'TRACKNUMBER', 'track_total': u'TRACKTOTAL', 'album_name': u'ALBUM', 'artist_name': u'ARTIST', 'performer_name': u'PERFORMER', 'composer_name': u'COMPOSER', 'conductor_name': u'CONDUCTOR', 'media': u'SOURCE MEDIUM', 'ISRC': u'ISRC', 'catalog': u'CATALOG', 'copyright': u'COPYRIGHT', 'publisher': u'PUBLISHER', 'year': u'DATE', 'album_number': u'DISCNUMBER', 'album_total': u'DISCTOTAL', 'comment': u'COMMENT', 'compilation': u'COMPILATION'} ALIASES = {} for aliases in [frozenset([u'TRACKTOTAL', u'TOTALTRACKS']), frozenset([u'DISCTOTAL', u'TOTALDISCS']), frozenset([u'ALBUM ARTIST', u'ALBUMARTIST', u'PERFORMER'])]: for alias in aliases: ALIASES[alias] = aliases def __init__(self, comment_strings, vendor_string): """comment_strings is a list of unicode strings vendor_string is a unicode string""" from audiotools import PY3 # some debug type checking for s in comment_strings: assert(isinstance(s, str if PY3 else unicode)) assert(isinstance(vendor_string, str if PY3 else unicode)) MetaData.__setattr__(self, "comment_strings", comment_strings) MetaData.__setattr__(self, "vendor_string", vendor_string) def keys(self): return list({comment.split(u"=", 1)[0] for comment in self.comment_strings if (u"=" in comment)}) def values(self): return [self[key] for key in self.keys()] def items(self): return [(key, self[key]) for key in self.keys()] def __contains__(self, key): from audiotools import PY3 assert(isinstance(key, str if PY3 else unicode)) matching_keys = self.ALIASES.get(key.upper(), frozenset([key.upper()])) return len([item_value for (item_key, item_value) in [comment.split(u"=", 1) for comment in self.comment_strings if (u"=" in comment)] if (item_key.upper() in matching_keys)]) > 0 def __getitem__(self, key): from audiotools import PY3 assert(isinstance(key, str if PY3 else unicode)) matching_keys = self.ALIASES.get(key.upper(), frozenset([key.upper()])) values = [item_value for (item_key, item_value) in [comment.split(u"=", 1) for comment in self.comment_strings if (u"=" in comment)] if (item_key.upper() in matching_keys)] if len(values) > 0: return values else: raise KeyError(key) def __setitem__(self, key, values): from audiotools import PY3 assert(isinstance(key, str if PY3 else unicode)) for v in values: assert(isinstance(v, str if PY3 else unicode)) new_values = values[:] new_comment_strings = [] matching_keys = self.ALIASES.get(key.upper(), frozenset([key.upper()])) for comment in self.comment_strings: if u"=" in comment: (c_key, c_value) = comment.split(u"=", 1) if c_key.upper() in matching_keys: try: # replace current value with newly set value new_comment_strings.append( u"{}={}".format(c_key, new_values.pop(0))) except IndexError: # no more newly set values, so remove current value continue else: # passthrough unmatching values new_comment_strings.append(comment) else: # passthrough values with no "=" sign new_comment_strings.append(comment) # append any leftover values for new_value in new_values: new_comment_strings.append(u"{}={}".format(key.upper(), new_value)) MetaData.__setattr__(self, "comment_strings", new_comment_strings) def __delitem__(self, key): from audiotools import PY3 assert(isinstance(key, str if PY3 else unicode)) new_comment_strings = [] matching_keys = self.ALIASES.get(key.upper(), frozenset([key.upper()])) for comment in self.comment_strings: if u"=" in comment: (c_key, c_value) = comment.split(u"=", 1) if c_key.upper() not in matching_keys: # passthrough unmatching values new_comment_strings.append(comment) else: # passthrough values with no "=" sign new_comment_strings.append(comment) MetaData.__setattr__(self, "comment_strings", new_comment_strings) def __repr__(self): return "VorbisComment({!r}, {!r})".format(self.comment_strings, self.vendor_string) def __comment_name__(self): return u"Vorbis Comment" def raw_info(self): """returns a Unicode string of low-level MetaData information whereas __unicode__ is meant to contain complete information at a very high level raw_info() should be more developer-specific and with very little adjustment or reordering to the data itself """ from os import linesep from audiotools import output_table # align text strings on the "=" sign, if any table = output_table() for comment in self.comment_strings: row = table.row() if u"=" in comment: (tag, value) = comment.split(u"=", 1) row.add_column(tag, "right") row.add_column(u"=") row.add_column(value) else: row.add_column(comment) row.add_column(u"") row.add_column(u"") return (u"{}: {}".format(self.__comment_name__(), self.vendor_string) + linesep + linesep.join(table.format())) def __getattr__(self, attr): # returns the first matching key for the given attribute # in our list of comment strings if attr in self.ATTRIBUTE_MAP: key = self.ATTRIBUTE_MAP[attr] if attr in {'track_number', 'album_number'}: try: # get the TRACKNUMBER/DISCNUMBER values # return the first value that contains an integer for value in self[key]: integer = re.search(r'\d+', value) if integer is not None: return int(integer.group(0)) else: # otherwise, return None return None except KeyError: # if no TRACKNUMBER/DISCNUMBER, return None return None elif attr in {'track_total', 'album_total'}: try: # get the TRACKTOTAL/DISCTOTAL values # return the first value that contains an integer for value in self[key]: integer = re.search(r'\d+', value) if integer is not None: return int(integer.group(0)) except KeyError: pass # if no TRACKTOTAL/DISCTOTAL, # or none of them contain an integer, # look for slashed TRACKNUMBER/DISCNUMBER values try: for value in self[{"track_total": u"TRACKNUMBER", "album_total": u"DISCNUMBER"}[attr]]: if u"/" in value: integer = re.search(r'\d+', value.split(u"/", 1)[1]) if integer is not None: return int(integer.group(0)) else: return None except KeyError: # no slashed TRACKNUMBER/DISCNUMBER values either # so return None return None elif attr == "compilation": try: # if present, return True if the first value is "1" return self[key][0] == u"1" except KeyError: # if not present, return None return None else: # attribute is supported by VorbisComment try: # if present, return the first value return self[key][0] except KeyError: # if not present, return None return None elif attr in self.FIELDS: # attribute is supported by MetaData # but not supported by VorbisComment return None else: # attribute is not MetaData-specific return MetaData.__getattribute__(self, attr) def __setattr__(self, attr, value): # updates the first matching field for the given attribute # in our list of comment strings def has_number(unicode_string): import re return re.search(r'\d+', unicode_string) is not None def swap_number(unicode_value, new_number): import re return re.sub(r'\d+', u"{:d}".format(new_number), unicode_value, 1) if (attr in self.FIELDS) and (value is None): # setting any value to None is equivilent to deleting it # in this high-level implementation delattr(self, attr) elif attr in self.ATTRIBUTE_MAP: key = self.ATTRIBUTE_MAP[attr] if attr in {'track_number', 'album_number'}: try: current_values = self[key] for i in range(len(current_values)): current_value = current_values[i] if u"/" not in current_value: if has_number(current_value): current_values[i] = swap_number(current_value, value) self[key] = current_values break else: (first, second) = current_value.split(u"/", 1) if has_number(first): current_values[i] = u"/".join( [swap_number(first, value), second]) self[key] = current_values break else: # no integer field matching key, so add new one self[key] = current_values + [u"{:d}".format(value)] except KeyError: # no current field with key, so add new one self[key] = [u"{:d}".format(value)] elif attr in {'track_total', 'album_total'}: # look for standalone TRACKTOTAL/DISCTOTAL field try: current_values = self[key] for i in range(len(current_values)): current_value = current_values[i] if has_number(current_value): current_values[i] = swap_number(current_value, value) self[key] = current_values return except KeyError: current_values = [] # no TRACKTOTAL/DISCTOTAL field # or none of them contain an integer, # so look for slashed TRACKNUMBER/DISCNUMBER values try: new_key = {"track_total": u"TRACKNUMBER", "album_total": u"DISCNUMBER"}[attr] slashed_values = self[new_key] for i in range(len(slashed_values)): current_value = slashed_values[i] if u"/" in current_value: (first, second) = current_value.split(u"/", 1) if has_number(second): slashed_values[i] = u"/".join( [first, swap_number(second, value)]) self[new_key] = slashed_values return except KeyError: # no TRACKNUMBER/DISCNUMBER field found pass # no slashed TRACKNUMBER/DISCNUMBER values either # so append a TRACKTOTAL/DISCTOTAL field self[key] = current_values + [u"{:d}".format(value)] elif attr == "compilation": self[key] = [u"1" if value else u"0"] else: # leave subsequent fields with the same key as-is try: current_values = self[key] self[key] = [value] + current_values[1:] except KeyError: # no current field with key, so add new one self[key] = [value] elif attr in self.FIELDS: # attribute is supported by MetaData # but not supported by VorbisComment # so ignore it pass else: # attribute is not MetaData-specific, so set as-is MetaData.__setattr__(self, attr, value) def __delattr__(self, attr): #FIXME # deletes all matching keys for the given attribute # in our list of comment strings import re if attr in self.ATTRIBUTE_MAP: key = self.ATTRIBUTE_MAP[attr] if attr in {'track_number', 'album_number'}: try: current_values = self[key] # save the _total side of any slashed fields for later slashed_totals = [int(match.group(0)) for match in [re.search(r'\d+', value.split(u"/", 1)[1]) for value in current_values if u"/" in value] if match is not None] # remove the TRACKNUMBER/DISCNUMBER field itself self[key] = [] # if there are any slashed totals # and there isn't a TRACKTOTAL/DISCTOTAL field already, # add a new one total_key = {'track_number': u"TRACKTOTAL", 'album_number': u"DISCTOTAL"}[attr] if (len(slashed_totals) > 0) and (total_key not in self): self[total_key] = [u"{:d}".format(slashed_totals[0])] except KeyError: # no TRACKNUMBER/DISCNUMBER field to remove pass elif attr in {'track_total', 'album_total'}: def slash_filter(unicode_string): if u"/" not in unicode_string: return unicode_string else: return unicode_string.split(u"/", 1)[0].rstrip() slashed_key = {"track_total": u"TRACKNUMBER", "album_total": u"DISCNUMBER"}[attr] # remove TRACKTOTAL/DISCTOTAL fields self[key] = [] # preserve the non-slashed side of # TRACKNUMBER/DISCNUMBER fields try: self[slashed_key] = [slash_filter(s) for s in self[slashed_key]] except KeyError: # no TRACKNUMBER/DISCNUMBER fields pass else: # unlike __setattr_, which tries to preserve multiple instances # of fields, __delattr__ wipes them all # so that orphaned fields don't show up after deletion self[key] = [] elif attr in self.FIELDS: # attribute is part of MetaData # but not supported by VorbisComment pass else: MetaData.__delattr__(self, attr) def __eq__(self, metadata): if isinstance(metadata, self.__class__): return self.comment_strings == metadata.comment_strings else: return MetaData.__eq__(self, metadata) @classmethod def converted(cls, metadata): """converts metadata from another class to VorbisComment""" from audiotools import VERSION if metadata is None: return None elif isinstance(metadata, VorbisComment): return cls(metadata.comment_strings[:], metadata.vendor_string) elif metadata.__class__.__name__ == 'FlacMetaData': if metadata.has_block(4): vorbis_comment = metadata.get_block(4) return cls(vorbis_comment.comment_strings[:], vorbis_comment.vendor_string) else: return cls([], u"Python Audio Tools {}".format(VERSION)) elif (metadata.__class__.__name__ in ('Flac_VORBISCOMMENT', 'OpusTags')): return cls(metadata.comment_strings[:], metadata.vendor_string) else: comment_strings = [] for (attr, key) in cls.ATTRIBUTE_MAP.items(): value = getattr(metadata, attr) if value is not None: attr_type = cls.FIELD_TYPES[attr] if attr_type is type(u""): comment_strings.append( u"{}={}".format(key, value)) elif attr_type is int: comment_strings.append( u"{}={:d}".format(key, value)) elif attr_type is bool: comment_strings.append( u"{}={:d}".format(key, 1 if value else 0)) return cls(comment_strings, u"Python Audio Tools {}".format(VERSION)) @classmethod def supports_images(cls): """returns False""" # There's actually a (proposed?) standard to add embedded covers # to Vorbis Comments by base64 encoding them. # This strikes me as messy and convoluted. # In addition, I'd have to perform a special case of # image extraction and re-insertion whenever converting # to FlacMetaData. The whole thought gives me a headache. return False def images(self): """returns a list of embedded Image objects""" return [] def clean(self): """returns a new MetaData object that's been cleaned of problems""" from audiotools.text import (CLEAN_REMOVE_TRAILING_WHITESPACE, CLEAN_REMOVE_LEADING_WHITESPACE, CLEAN_REMOVE_EMPTY_TAG, CLEAN_REMOVE_LEADING_WHITESPACE_ZEROES, CLEAN_REMOVE_LEADING_ZEROES) fixes_performed = [] reverse_attr_map = {} for (attr, key) in self.ATTRIBUTE_MAP.items(): reverse_attr_map[key] = attr if key in self.ALIASES: for alias in self.ALIASES[key]: reverse_attr_map[alias] = attr cleaned_fields = [] for comment_string in self.comment_strings: if u"=" in comment_string: (key, value) = comment_string.split(u"=", 1) if key.upper() in reverse_attr_map: attr = reverse_attr_map[key.upper()] # handle all text fields by stripping whitespace if len(value.strip()) == 0: fixes_performed.append( CLEAN_REMOVE_EMPTY_TAG.format(key)) else: fix1 = value.rstrip() if fix1 != value: fixes_performed.append( CLEAN_REMOVE_TRAILING_WHITESPACE.format(key)) fix2 = fix1.lstrip() if fix2 != fix1: fixes_performed.append( CLEAN_REMOVE_LEADING_WHITESPACE.format(key)) # integer fields also strip leading zeroes if (((attr == "track_number") or (attr == "album_number"))): match = re.match(r'(.*?)\s*/\s*(.*)', fix2) if match is not None: # fix whitespace/zeroes # on either side of slash fix3 = u"{}/{}".format( match.group(1).lstrip(u"0"), match.group(2).lstrip(u"0")) if fix3 != fix2: fixes_performed.append( CLEAN_REMOVE_LEADING_WHITESPACE_ZEROES.format(key)) else: # fix zeroes only fix3 = fix2.lstrip(u"0") if fix3 != fix2: fixes_performed.append( CLEAN_REMOVE_LEADING_ZEROES.format(key)) elif ((attr == "track_total") or (attr == "album_total")): fix3 = fix2.lstrip(u"0") if fix3 != fix2: fixes_performed.append( CLEAN_REMOVE_LEADING_ZEROES.format(key)) else: fix3 = fix2 cleaned_fields.append(u"{}={}".format(key, fix3)) else: cleaned_fields.append(comment_string) else: cleaned_fields.append(comment_string) return (self.__class__(cleaned_fields, self.vendor_string), fixes_performed) def intersection(self, metadata): """given a MetaData-compatible object, returns a new MetaData object which contains all the matching fields and images of this object and 'metadata' """ def comment_present(comment): if u"=" in comment: key, value = comment.split(u"=", 1) try: for other_value in metadata[key]: if value == other_value: return True else: return False except KeyError: return False else: for other_comment in metadata.comment_strings: if comment == other_comment: return True else: return False if isinstance(metadata, VorbisComment): return self.__class__([comment for comment in self.comment_strings if comment_present(comment)], self.vendor_string) else: return MetaData.intersection(self, metadata) ================================================ FILE: audiotools/wav.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools import (AudioFile, InvalidFile, PCMReader, WaveContainer) from audiotools.pcm import FrameList import sys import struct class RIFF_Chunk(object): """a raw chunk of RIFF WAVE data""" def __init__(self, chunk_id, chunk_size, chunk_data): """chunk_id should be a binary string of ASCII chunk_data should be a binary string of chunk data""" # FIXME - check chunk_id's validity self.id = chunk_id self.__size__ = chunk_size self.__data__ = chunk_data def __repr__(self): return "RIFF_Chunk({!r})".format(self.id) def size(self): """returns size of chunk in bytes not including any spacer byte for odd-sized chunks""" return self.__size__ def total_size(self): """returns the total size of the chunk including the 8 byte ID/size and any padding byte""" if self.__size__ % 2: return 8 + self.__size__ + 1 else: return 8 + self.__size__ def data(self): """returns chunk data as file-like object""" from io import BytesIO return BytesIO(self.__data__) def verify(self): """returns True if the chunk is sized properly""" return self.__size__ == len(self.__data__) def write(self, f): """writes the entire chunk to the given output file object returns size of entire chunk (including header and spacer) in bytes""" f.write(self.id) f.write(struct.pack("<I", self.__size__)) f.write(self.__data__) if self.__size__ % 2: f.write(b"\x00") return self.total_size() class RIFF_File_Chunk(RIFF_Chunk): """a raw chunk of RIFF WAVE data taken from an existing file""" def __init__(self, chunk_id, chunk_size, wav_file, chunk_data_offset): """chunk_id should be a binary string of ASCII chunk_size is the size of the chunk in bytes (not counting any spacer byte) wav_file is the file this chunk belongs to chunk_data_offset is the offset to the chunk's data bytes (not including the 8 byte header)""" self.id = chunk_id self.__size__ = chunk_size self.__wav_file__ = wav_file self.__offset__ = chunk_data_offset def __del__(self): self.__wav_file__.close() def __repr__(self): return "RIFF_File_Chunk({!r})".format(self.id) def data(self): """returns chunk data as file-like object""" self.__wav_file__.seek(self.__offset__) from audiotools import LimitedFileReader return LimitedFileReader(self.__wav_file__, self.size()) def verify(self): """returns True if the chunk is sized properly""" self.__wav_file__.seek(self.__offset__) to_read = self.__size__ while to_read > 0: s = self.__wav_file__.read(min(0x100000, to_read)) if len(s) == 0: return False else: to_read -= len(s) return True def write(self, f): """writes the entire chunk to the given output file object returns size of entire chunk (including header and spacer) in bytes""" f.write(self.id) f.write(struct.pack("<I", self.__size__)) self.__wav_file__.seek(self.__offset__) to_write = self.__size__ while to_write > 0: s = self.__wav_file__.read(min(0x100000, to_write)) f.write(s) to_write -= len(s) if self.__size__ % 2: f.write(b"\x00") return self.total_size() def pad_data(pcm_frames, channels, bits_per_sample): """returns True if the given stream combination requires an extra padding byte at the end of the 'data' chunk""" return (pcm_frames * channels * (bits_per_sample // 8)) % 2 def validate_header(header): """given header string as returned by wave_header_footer(), returns (total size, data size) where total size is the size of the file in bytes and data size is the size of the data chunk in bytes (not including any padding byte) the size of the data chunk and of the total file should be validated after the file has been completely written such that len(header) + len(data chunk) + len(footer) = total size raises ValueError if the header is invalid """ from io import BytesIO from audiotools.bitstream import BitstreamReader header_size = len(header) wave_file = BitstreamReader(BytesIO(header), True) try: # ensure header starts with RIFF<size>WAVE chunk (riff, remaining_size, wave) = wave_file.parse("4b 32u 4b") if riff != b"RIFF": from audiotools.text import ERR_WAV_NOT_WAVE raise ValueError(ERR_WAV_NOT_WAVE) elif wave != b"WAVE": from audiotools.text import ERR_WAV_INVALID_WAVE raise ValueError(ERR_WAV_INVALID_WAVE) else: total_size = remaining_size + 8 header_size -= 12 fmt_found = False while header_size > 0: # ensure each chunk header is valid (chunk_id, chunk_size) = wave_file.parse("4b 32u") if not frozenset(chunk_id).issubset(WaveAudio.PRINTABLE_ASCII): from audiotools.text import ERR_WAV_INVALID_CHUNK raise ValueError(ERR_WAV_INVALID_CHUNK) else: header_size -= 8 if chunk_id == b"fmt ": if not fmt_found: # skip fmt chunk data when found fmt_found = True if chunk_size % 2: wave_file.skip_bytes(chunk_size + 1) header_size -= (chunk_size + 1) else: wave_file.skip_bytes(chunk_size) header_size -= chunk_size else: # ensure only one fmt chunk is found from audiotools.text import ERR_WAV_MULTIPLE_FMT raise ValueError(ERR_WAV_MULTIPLE_FMT) elif chunk_id == b"data": if not fmt_found: # ensure at least one fmt chunk is found from audiotools.text import ERR_WAV_PREMATURE_DATA raise ValueError(ERR_WAV_PREMATURE_DATA) elif header_size > 0: # ensure no data remains after data chunk header from audiotools.text import ERR_WAV_HEADER_EXTRA_DATA raise ValueError( ERR_WAV_HEADER_EXTRA_DATA.format(header_size)) else: return (total_size, chunk_size) else: # skip the full contents of non-audio chunks if chunk_size % 2: wave_file.skip_bytes(chunk_size + 1) header_size -= (chunk_size + 1) else: wave_file.skip_bytes(chunk_size) header_size -= chunk_size else: # header parsed with no data chunks found from audiotools.text import ERR_WAV_NO_DATA_CHUNK raise ValueError(ERR_WAV_NO_DATA_CHUNK) except IOError: from audiotools.text import ERR_WAV_HEADER_IOERROR raise ValueError(ERR_WAV_HEADER_IOERROR) def validate_footer(footer, data_bytes_written): """given a footer string as returned by wave_header_footer() and PCM stream parameters, returns True if the footer is valid raises ValueError if the footer is invalid""" from io import BytesIO from audiotools.bitstream import BitstreamReader total_size = len(footer) wave_file = BitstreamReader(BytesIO(footer), True) try: # ensure footer is padded properly if necessary # based on size of data bytes written if data_bytes_written % 2: wave_file.skip_bytes(1) total_size -= 1 while total_size > 0: (chunk_id, chunk_size) = wave_file.parse("4b 32u") if not frozenset(chunk_id).issubset(WaveAudio.PRINTABLE_ASCII): from audiotools.text import ERR_WAV_INVALID_CHUNK raise ValueError(ERR_WAV_INVALID_CHUNK) else: total_size -= 8 if chunk_id == b"fmt ": # ensure no fmt chunks are found from audiotools.text import ERR_WAV_MULTIPLE_FMT raise ValueError(ERR_WAV_MULTIPLE_FMT) elif chunk_id == b"data": # ensure no data chunks are found from audiotools.text import ERR_WAV_MULTIPLE_DATA raise ValueError(ERR_WAV_MULTIPLE_DATA) else: # skip the full contents of non-audio chunks if chunk_size % 2: wave_file.skip_bytes(chunk_size + 1) total_size -= (chunk_size + 1) else: wave_file.skip_bytes(chunk_size) total_size -= chunk_size else: return True except IOError: from audiotools.text import ERR_WAV_FOOTER_IOERROR raise ValueError(ERR_WAV_FOOTER_IOERROR) def parse_fmt(fmt): """given a fmt block BitstreamReader (without the 8 byte header) returns (channels, sample_rate, bits_per_sample, channel_mask) where channel_mask is a ChannelMask object and the rest are ints may raise ValueError if the fmt chunk is invalid or IOError if an error occurs parsing the chunk""" from audiotools import ChannelMask (compression, channels, sample_rate, bytes_per_second, block_align, bits_per_sample) = fmt.parse("16u 16u 32u 32u 16u 16u") if compression == 1: # if we have a multi-channel WAVE file # that's not WAVEFORMATEXTENSIBLE, # assume the channels follow # SMPTE/ITU-R recommendations # and hope for the best if channels == 1: channel_mask = ChannelMask.from_fields( front_center=True) elif channels == 2: channel_mask = ChannelMask.from_fields( front_left=True, front_right=True) elif channels == 3: channel_mask = ChannelMask.from_fields( front_left=True, front_right=True, front_center=True) elif channels == 4: channel_mask = ChannelMask.from_fields( front_left=True, front_right=True, back_left=True, back_right=True) elif channels == 5: channel_mask = ChannelMask.from_fields( front_left=True, front_right=True, back_left=True, back_right=True, front_center=True) elif channels == 6: channel_mask = ChannelMask.from_fields( front_left=True, front_right=True, back_left=True, back_right=True, front_center=True, low_frequency=True) else: channel_mask = ChannelMask(0) return (channels, sample_rate, bits_per_sample, channel_mask) elif compression == 0xFFFE: (cb_size, valid_bits_per_sample, channel_mask, sub_format) = fmt.parse("16u 16u 32u 16b") if (sub_format != (b'\x01\x00\x00\x00\x00\x00\x10\x00' + b'\x80\x00\x00\xaa\x00\x38\x9b\x71')): # FIXME raise ValueError("invalid WAVE sub-format") else: channel_mask = ChannelMask(channel_mask) return (channels, sample_rate, bits_per_sample, channel_mask) else: # FIXME raise ValueError("unsupported WAVE compression") def wave_header(sample_rate, channels, channel_mask, bits_per_sample, total_pcm_frames): """given a set of integer stream attributes, returns header string of everything before a RIFF WAVE's PCM data may raise ValueError if the total size of the file is too large""" from audiotools.bitstream import (BitstreamRecorder, format_size) assert(isinstance(sample_rate, int)) assert(isinstance(channels, int)) assert(isinstance(channel_mask, int)) assert(isinstance(bits_per_sample, int)) assert(isinstance(total_pcm_frames, int) or isinstance(total_pcm_frames, long)) header = BitstreamRecorder(True) avg_bytes_per_second = sample_rate * channels * (bits_per_sample // 8) block_align = channels * (bits_per_sample // 8) # build a regular or extended fmt chunk # based on the reader's attributes if ((channels <= 2) and (bits_per_sample <= 16)): fmt = "16u 16u 32u 32u 16u 16u" fmt_fields = (1, # compression code channels, sample_rate, avg_bytes_per_second, block_align, bits_per_sample) else: if channel_mask == 0: channel_mask = {1: 0x4, 2: 0x3, 3: 0x7, 4: 0x33, 5: 0x37, 6: 0x3F}.get(channels, 0) fmt = "16u 16u 32u 32u 16u 16u" + "16u 16u 32u 16b" fmt_fields = (0xFFFE, # compression code channels, sample_rate, avg_bytes_per_second, block_align, bits_per_sample, 22, # CB size bits_per_sample, channel_mask, b'\x01\x00\x00\x00\x00\x00\x10\x00' + b'\x80\x00\x00\xaa\x00\x38\x9b\x71' # sub format ) data_size = (bits_per_sample // 8) * channels * total_pcm_frames total_size = ((format_size("4b" + "4b 32u" + fmt + "4b 32u") // 8) + data_size + (data_size % 2)) if total_size < (2 ** 32): header.build("4b 32u 4b", (b"RIFF", total_size, b"WAVE")) header.build("4b 32u", (b"fmt ", format_size(fmt) // 8)) header.build(fmt, fmt_fields) header.build("4b 32u", (b"data", data_size)) return header.data() else: raise ValueError("total size too large for wave file") class WaveReader(object): """a PCMReader object for reading wave file contents""" def __init__(self, wave_filename): """wave_filename is a string""" from audiotools.bitstream import BitstreamReader self.stream = BitstreamReader(open(wave_filename, "rb"), True) # ensure RIFF<size>WAVE header is ok try: (riff, total_size, wave) = self.stream.parse("4b 32u 4b") except struct.error: from audiotools.text import ERR_WAV_INVALID_WAVE self.stream.close() raise ValueError(ERR_WAV_INVALID_WAVE) if riff != b'RIFF': from audiotools.text import ERR_WAV_NOT_WAVE self.stream.close() raise ValueError(ERR_WAV_NOT_WAVE) elif wave != b'WAVE': from audiotools.text import ERR_WAV_INVALID_WAVE self.stream.close() raise ValueError(ERR_WAV_INVALID_WAVE) else: total_size -= 4 fmt_chunk_read = False # walk through chunks until "data" chunk encountered while total_size > 0: try: (chunk_id, chunk_size) = self.stream.parse("4b 32u") except struct.error: from audiotools.text import ERR_WAV_INVALID_WAVE self.stream.close() raise ValueError(ERR_WAV_INVALID_WAVE) if not frozenset(chunk_id).issubset(WaveAudio.PRINTABLE_ASCII): from audiotools.text import ERR_WAV_INVALID_CHUNK self.stream.close() raise ValueError(ERR_WAV_INVALID_CHUNK) else: total_size -= 8 if chunk_id == b"fmt ": # when "fmt " chunk encountered, # use it to populate PCMReader attributes (self.channels, self.sample_rate, self.bits_per_sample, channel_mask) = parse_fmt(self.stream) self.channel_mask = int(channel_mask) self.bytes_per_pcm_frame = ((self.bits_per_sample // 8) * self.channels) fmt_chunk_read = True elif chunk_id == b"data": # when "data" chunk encountered, # use its size to determine total PCM frames # and ready PCMReader for reading if not fmt_chunk_read: from audiotools.text import ERR_WAV_PREMATURE_DATA self.stream.close() raise ValueError(ERR_WAV_PREMATURE_DATA) else: self.total_pcm_frames = (chunk_size // self.bytes_per_pcm_frame) self.remaining_pcm_frames = self.total_pcm_frames self.data_start = self.stream.getpos() return else: # all other chunks are ignored self.stream.skip_bytes(chunk_size) if chunk_size % 2: if len(self.stream.read_bytes(1)) < 1: from audiotools.text import ERR_WAV_INVALID_CHUNK self.stream.close() raise ValueError(ERR_WAV_INVALID_CHUNK) total_size -= (chunk_size + 1) else: total_size -= chunk_size else: # raise an error if no "data" chunk is encountered from audiotools.text import ERR_WAV_NO_DATA_CHUNK self.stream.close() raise ValueError(ERR_WAV_NO_DATA_CHUNK) def __enter__(self): return self def __exit__(self, exc_type, exc_value, traceback): self.close() def read(self, pcm_frames): """try to read a pcm.FrameList with the given number of PCM frames""" # try to read requested PCM frames or remaining frames requested_pcm_frames = min(max(pcm_frames, 1), self.remaining_pcm_frames) requested_bytes = (self.bytes_per_pcm_frame * requested_pcm_frames) pcm_data = self.stream.read_bytes(requested_bytes) # raise exception if "data" chunk exhausted early if len(pcm_data) < requested_bytes: from audiotools.text import ERR_WAV_TRUNCATED_DATA_CHUNK raise IOError(ERR_WAV_TRUNCATED_DATA_CHUNK) else: self.remaining_pcm_frames -= requested_pcm_frames # return parsed chunk return FrameList(pcm_data, self.channels, self.bits_per_sample, False, self.bits_per_sample != 8) def read_closed(self, pcm_frames): raise ValueError("cannot read closed stream") def seek(self, pcm_frame_offset): """tries to seek to the given PCM frame offset returns the total amount of frames actually seeked over""" if pcm_frame_offset < 0: from audiotools.text import ERR_NEGATIVE_SEEK raise ValueError(ERR_NEGATIVE_SEEK) # ensure one doesn't walk off the end of the file pcm_frame_offset = min(pcm_frame_offset, self.total_pcm_frames) # position file in "data" chunk self.stream.setpos(self.data_start) self.stream.seek(pcm_frame_offset * self.bytes_per_pcm_frame, 1) self.remaining_pcm_frames = (self.total_pcm_frames - pcm_frame_offset) return pcm_frame_offset def seek_closed(self, pcm_frame_offset): raise ValueError("cannot seek closed stream") def close(self): """closes the stream for reading""" self.stream.close() self.read = self.read_closed self.seek = self.seek_closed class TempWaveReader(WaveReader): """a subclass of WaveReader for reading wave data from temporary files""" def __init__(self, tempfile): """tempfile should be a NamedTemporaryFile its contents are used to populate the rest of the fields""" WaveReader.__init__(self, tempfile.name) self.tempfile = tempfile def __del__(self): WaveReader.__del__(self) self.tempfile.close() def close(self): """closes the input stream and temporary file""" WaveReader.close(self) class InvalidWave(InvalidFile): """raises during initialization time if a wave file is invalid""" pass class WaveAudio(WaveContainer): """a waveform audio file""" SUFFIX = "wav" NAME = SUFFIX DESCRIPTION = u"Waveform Audio File Format" if sys.version_info[0] >= 3: PRINTABLE_ASCII = {i for i in range(0x20, 0x7E + 1)} else: PRINTABLE_ASCII = {chr(i) for i in range(0x20, 0x7E + 1)} def __init__(self, filename): """filename is a plain string""" from audiotools import ChannelMask WaveContainer.__init__(self, filename) self.__channels__ = 0 self.__sample_rate__ = 0 self.__bits_per_sample__ = 0 self.__data_size__ = 0 self.__channel_mask__ = ChannelMask(0) from audiotools.bitstream import BitstreamReader fmt_read = data_read = False try: for chunk in self.chunks(): if chunk.id == b"fmt ": try: (self.__channels__, self.__sample_rate__, self.__bits_per_sample__, self.__channel_mask__) = parse_fmt( BitstreamReader(chunk.data(), True)) fmt_read = True if fmt_read and data_read: break except IOError: continue except ValueError as err: raise InvalidWave(str(err)) elif chunk.id == b"data": self.__data_size__ = chunk.size() data_read = True if fmt_read and data_read: break except IOError: raise InvalidWave("I/O error reading wave") def lossless(self): """returns True""" return True def has_foreign_wave_chunks(self): """returns True if the audio file contains non-audio RIFF chunks during transcoding, if the source audio file has foreign RIFF chunks and the target audio format supports foreign RIFF chunks, conversion should be routed through .wav conversion to avoid losing those chunks""" return {b'fmt ', b'data'} != {c.id for c in self.chunks()} def channel_mask(self): """returns a ChannelMask object of this track's channel layout""" return self.__channel_mask__ @classmethod def supports_to_pcm(cls): """returns True if all necessary components are available to support the .to_pcm() method""" return True # Returns the PCMReader object for this WAV's data def to_pcm(self): """returns a PCMReader object containing the track's PCM data""" try: return WaveReader(self.filename) except (IOError, ValueError) as err: from audiotools import PCMReaderError return PCMReaderError(error_message=str(err), sample_rate=self.sample_rate(), channels=self.channels(), channel_mask=int(self.channel_mask()), bits_per_sample=self.bits_per_sample()) @classmethod def supports_from_pcm(cls): """returns True if all necessary components are available to support the .from_pcm() classmethod""" return True @classmethod def from_pcm(cls, filename, pcmreader, compression=None, total_pcm_frames=None): """encodes a new file from PCM data takes a filename string, PCMReader object, optional compression level string and optional total_pcm_frames integer encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new WaveAudio object""" from audiotools import EncodingError from audiotools import DecodingError from audiotools import CounterPCMReader from audiotools import transfer_framelist_data if pcmreader.bits_per_sample not in {8, 16, 24}: from audiotools import UnsupportedBitsPerSample pcmreader.close() raise UnsupportedBitsPerSample(filename, pcmreader.bits_per_sample) try: header = wave_header(pcmreader.sample_rate, pcmreader.channels, pcmreader.channel_mask, pcmreader.bits_per_sample, total_pcm_frames if total_pcm_frames is not None else 0) except ValueError as err: pcmreader.close() raise EncodingError(str(err)) try: f = open(filename, "wb") except IOError as err: pcmreader.close() raise EncodingError(str(err)) counter = CounterPCMReader(pcmreader) f.write(header) try: transfer_framelist_data(counter, f.write, pcmreader.bits_per_sample > 8, False) except (IOError, ValueError) as err: f.close() cls.__unlink__(filename) raise EncodingError(str(err)) except Exception as err: f.close() cls.__unlink__(filename) raise err # handle odd-sized "data" chunks if counter.frames_written % 2: f.write(b"\x00") if total_pcm_frames is not None: f.close() if total_pcm_frames != counter.frames_written: # ensure written number of PCM frames # matches total_pcm_frames argument from audiotools.text import ERR_TOTAL_PCM_FRAMES_MISMATCH cls.__unlink__(filename) raise EncodingError(ERR_TOTAL_PCM_FRAMES_MISMATCH) else: # go back and rewrite populated header # with counted number of PCM frames f.seek(0, 0) f.write(wave_header(pcmreader.sample_rate, pcmreader.channels, pcmreader.channel_mask, pcmreader.bits_per_sample, counter.frames_written)) f.close() return WaveAudio(filename) def total_frames(self): """returns the total PCM frames of the track as an integer""" return (self.__data_size__ // (self.__bits_per_sample__ // 8) // self.__channels__) def sample_rate(self): """returns the rate of the track's audio as an integer number of Hz""" return self.__sample_rate__ def channels(self): """returns an integer number of channels this track contains""" return self.__channels__ def bits_per_sample(self): """returns an integer number of bits-per-sample this track contains""" return self.__bits_per_sample__ def seekable(self): """returns True if the file is seekable""" return True @classmethod def track_name(cls, file_path, track_metadata=None, format=None): """constructs a new filename string given a plain string to an existing path, a MetaData-compatible object (or None), a UTF-8-encoded Python format string and an ASCII-encoded suffix string (such as "mp3") returns a plain string of a new filename with format's fields filled-in and encoded as FS_ENCODING raises UnsupportedTracknameField if the format string contains invalid template fields""" if format is None: format = "track%(track_number)2.2d.wav" return AudioFile.track_name(file_path, track_metadata, format, suffix=cls.SUFFIX) def chunks(self): """yields a set of RIFF_Chunk or RIFF_File_Chunk objects""" with open(self.filename, "rb") as wave_file: try: (riff, total_size, wave) = struct.unpack("<4sI4s", wave_file.read(12)) except struct.error: from audiotools.text import ERR_WAV_INVALID_WAVE raise InvalidWave(ERR_WAV_INVALID_WAVE) if riff != b'RIFF': from audiotools.text import ERR_WAV_NOT_WAVE raise InvalidWave(ERR_WAV_NOT_WAVE) elif wave != b'WAVE': from audiotools.text import ERR_WAV_INVALID_WAVE raise InvalidWave(ERR_WAV_INVALID_WAVE) else: total_size -= 4 while total_size > 0: # read the chunk header and ensure its validity try: (chunk_id, chunk_size) = struct.unpack("<4sI", wave_file.read(8)) except struct.error: from audiotools.text import ERR_WAV_INVALID_WAVE raise InvalidWave(ERR_WAV_INVALID_WAVE) if not frozenset(chunk_id).issubset(self.PRINTABLE_ASCII): from audiotools.text import ERR_WAV_INVALID_CHUNK raise InvalidWave(ERR_WAV_INVALID_CHUNK) else: total_size -= 8 # yield RIFF_Chunk or RIFF_File_Chunk depending on chunk size if chunk_size >= 0x100000: # if chunk is too large, yield a File_Chunk yield RIFF_File_Chunk(chunk_id, chunk_size, open(self.filename, "rb"), wave_file.tell()) wave_file.seek(chunk_size, 1) else: # otherwise, yield a raw data Chunk yield RIFF_Chunk(chunk_id, chunk_size, wave_file.read(chunk_size)) if chunk_size % 2: if len(wave_file.read(1)) < 1: from audiotools.text import ERR_WAV_INVALID_CHUNK raise InvalidWave(ERR_WAV_INVALID_CHUNK) total_size -= (chunk_size + 1) else: total_size -= chunk_size @classmethod def wave_from_chunks(cls, filename, chunk_iter): """builds a new RIFF WAVE file from a chunk data iterator filename is the path to the wave file to build chunk_iter should yield RIFF_Chunk-compatible objects """ with open(filename, 'wb') as wave_file: total_size = 4 # write an unfinished header with a placeholder size wave_file.write( struct.pack("<4sI4s", b"RIFF", total_size, b"WAVE")) # write the individual chunks for chunk in chunk_iter: total_size += chunk.write(wave_file) # once the chunks are done, go back and re-write the header wave_file.seek(0, 0) wave_file.write( struct.pack("<4sI4s", b"RIFF", total_size, b"WAVE")) def wave_header_footer(self): """returns a pair of data strings before and after PCM data the first contains all data before the PCM content of the data chunk the second containing all data after the data chunk for example: >>> w = audiotools.open("input.wav") >>> (head, tail) = w.wave_header_footer() >>> f = open("output.wav", "wb") >>> f.write(head) >>> audiotools.transfer_framelist_data(w.to_pcm(), f.write) >>> f.write(tail) >>> f.close() should result in "output.wav" being identical to "input.wav" """ from audiotools.bitstream import BitstreamReader from audiotools.bitstream import BitstreamRecorder head = BitstreamRecorder(1) tail = BitstreamRecorder(1) current_block = head fmt_found = False with BitstreamReader(open(self.filename, 'rb'), 1) as wave_file: # transfer the 12-byte "RIFFsizeWAVE" header to head (riff, size, wave) = wave_file.parse("4b 32u 4b") if riff != b'RIFF': from audiotools.text import ERR_WAV_NOT_WAVE raise ValueError(ERR_WAV_NOT_WAVE) elif wave != b'WAVE': from audiotools.text import ERR_WAV_INVALID_WAVE raise ValueError(ERR_WAV_INVALID_WAVE) else: current_block.build("4b 32u 4b", (riff, size, wave)) total_size = size - 4 while total_size > 0: # transfer each chunk header (chunk_id, chunk_size) = wave_file.parse("4b 32u") if not frozenset(chunk_id).issubset(self.PRINTABLE_ASCII): from audiotools.text import ERR_WAV_INVALID_CHUNK raise ValueError(ERR_WAV_INVALID_CHUNK) else: current_block.build("4b 32u", (chunk_id, chunk_size)) total_size -= 8 # and transfer the full content of non-audio chunks if chunk_id != b"data": if chunk_id == b"fmt ": if not fmt_found: fmt_found = True else: from audiotools.text import ERR_WAV_MULTIPLE_FMT raise ValueError(ERR_WAV_MULTIPLE_FMT) if chunk_size % 2: current_block.write_bytes( wave_file.read_bytes(chunk_size + 1)) total_size -= (chunk_size + 1) else: current_block.write_bytes( wave_file.read_bytes(chunk_size)) total_size -= chunk_size else: wave_file.skip_bytes(chunk_size) current_block = tail if chunk_size % 2: current_block.write_bytes(wave_file.read_bytes(1)) total_size -= (chunk_size + 1) else: total_size -= chunk_size if fmt_found: return (head.data(), tail.data()) else: from audiotools.text import ERR_WAV_NO_FMT_CHUNK raise ValueError(ERR_WAV_NO_FMT_CHUNK) @classmethod def from_wave(cls, filename, header, pcmreader, footer, compression=None): """encodes a new file from wave data takes a filename string, header string, PCMReader object, footer string and optional compression level string encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new WaveAudio object header + pcm data + footer should always result in the original wave file being restored without need for any padding bytes may raise EncodingError if some problem occurs when encoding the input file""" from audiotools import (DecodingError, EncodingError, FRAMELIST_SIZE) from struct import unpack # ensure header validates correctly try: (total_size, data_size) = validate_header(header) except ValueError as err: pcmreader.close() raise EncodingError(str(err)) try: f = open(filename, "wb") except IOError as msg: pcmreader.close() raise EncodingError(err) try: # write header to output file f.write(header) # write PCM data to output file data_bytes_written = 0 signed = (pcmreader.bits_per_sample > 8) s = pcmreader.read(FRAMELIST_SIZE).to_bytes(False, signed) while len(s) > 0: data_bytes_written += len(s) f.write(s) s = pcmreader.read(FRAMELIST_SIZE).to_bytes(False, signed) # ensure output data size matches the "data" chunk's size if data_size != data_bytes_written: cls.__unlink__(filename) from audiotools.text import ERR_WAV_TRUNCATED_DATA_CHUNK raise EncodingError(ERR_WAV_TRUNCATED_DATA_CHUNK) # ensure footer validates correctly # before writing it to disk validate_footer(footer, data_bytes_written) f.write(footer) f.flush() # ensure total size is correct if (len(header) + data_size + len(footer)) != total_size: cls.__unlink__(filename) from audiotools.text import ERR_WAV_INVALID_SIZE raise EncodingError(ERR_WAV_INVALID_SIZE) return cls(filename) except (IOError, ValueError) as err: cls.__unlink__(filename) raise EncodingError(str(err)) finally: f.close() pcmreader.close() def verify(self, progress=None): """verifies the current file for correctness returns True if the file is okay raises an InvalidFile with an error message if there is some problem with the file""" from audiotools import CounterPCMReader from audiotools import transfer_framelist_data from audiotools import to_pcm_progress try: (header, footer) = self.wave_header_footer() except IOError as err: raise InvalidWave(err) except ValueError as err: raise InvalidWave(err) # ensure header is valid try: (total_size, data_size) = validate_header(header) except ValueError as err: raise InvalidWave(err) # ensure "data" chunk has all its data counter = CounterPCMReader(to_pcm_progress(self, progress)) try: transfer_framelist_data(counter, lambda f: f) except (IOError, ValueError): from audiotools.text import ERR_WAV_TRUNCATED_DATA_CHUNK raise InvalidWave(ERR_WAV_TRUNCATED_DATA_CHUNK) data_bytes_written = counter.bytes_written() # ensure output data size matches the "data" chunk's size if data_size != data_bytes_written: from audiotools.text import ERR_WAV_TRUNCATED_DATA_CHUNK raise InvalidWave(ERR_WAV_TRUNCATED_DATA_CHUNK) # ensure footer validates correctly try: validate_footer(footer, data_bytes_written) except ValueError as err: from audiotools.text import ERR_WAV_INVALID_SIZE raise InvalidWave(ERR_WAV_INVALID_SIZE) # ensure total size is correct if (len(header) + data_size + len(footer)) != total_size: from audiotools.text import ERR_WAV_INVALID_SIZE raise InvalidWave(ERR_WAV_INVALID_SIZE) return True def clean(self, output_filename=None): """cleans the file of known data and metadata problems output_filename is an optional filename of the fixed file if present, a new AudioFile is written to that path otherwise, only a dry-run is performed and no new file is written return list of fixes performed as Unicode strings raises IOError if unable to write the file or its metadata raises ValueError if the file has errors of some sort """ fixes_performed = [] chunk_queue = [] pending_data = None for chunk in self.chunks(): if chunk.id == b"fmt ": if b"fmt " in [c.id for c in chunk_queue]: from audiotools.text import CLEAN_WAV_MULTIPLE_FMT_CHUNKS fixes_performed.append(CLEAN_WAV_MULTIPLE_FMT_CHUNKS) else: chunk_queue.append(chunk) if pending_data is not None: chunk_queue.append(pending_data) pending_data = None elif chunk.id == b"data": if b"fmt " not in [c.id for c in chunk_queue]: from audiotools.text import CLEAN_WAV_REORDERED_DATA_CHUNK fixes_performed.append(CLEAN_WAV_REORDERED_DATA_CHUNK) pending_data = chunk elif b"data" in [c.id for c in chunk_queue]: from audiotools.text import CLEAN_WAV_MULTIPLE_DATA_CHUNKS fixes_performed.append(CLEAN_WAV_MULTIPLE_DATA_CHUNKS) else: chunk_queue.append(chunk) else: chunk_queue.append(chunk) if output_filename is not None: WaveAudio.wave_from_chunks(output_filename, chunk_queue) return fixes_performed ================================================ FILE: audiotools/wavpack.py ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from audiotools import WaveContainer, InvalidFile from audiotools.ape import ApeTaggedAudio, ApeGainedAudio class InvalidWavPack(InvalidFile): pass def __riff_chunk_ids__(data_size, data): (riff, size, wave) = data.parse("4b 32u 4b") if riff != b"RIFF": return elif wave != b"WAVE": return else: data_size -= 12 while data_size > 0: (chunk_id, chunk_size) = data.parse("4b 32u") data_size -= 8 if (chunk_size % 2) == 1: chunk_size += 1 yield chunk_id if chunk_id != b"data": data.skip_bytes(chunk_size) data_size -= chunk_size class WavPackAudio(ApeTaggedAudio, ApeGainedAudio, WaveContainer): """a WavPack audio file""" from audiotools.text import (COMP_WAVPACK_FAST, COMP_WAVPACK_VERYHIGH) SUFFIX = "wv" NAME = SUFFIX DESCRIPTION = u"WavPack" DEFAULT_COMPRESSION = "standard" COMPRESSION_MODES = ("fast", "standard", "high", "veryhigh") COMPRESSION_DESCRIPTIONS = {"fast": COMP_WAVPACK_FAST, "veryhigh": COMP_WAVPACK_VERYHIGH} BITS_PER_SAMPLE = (8, 16, 24, 32) SAMPLING_RATE = (6000, 8000, 9600, 11025, 12000, 16000, 22050, 24000, 32000, 44100, 48000, 64000, 88200, 96000, 192000, 0) def __init__(self, filename): """filename is a plain string""" WaveContainer.__init__(self, filename) self.__samplerate__ = 0 self.__channels__ = 0 self.__bitspersample__ = 0 self.__total_frames__ = 0 try: self.__read_info__() except IOError as msg: raise InvalidWavPack(str(msg)) def lossless(self): """returns True""" return True def channel_mask(self): """returns a ChannelMask object of this track's channel layout""" return self.__channel_mask__ @classmethod def supports_metadata(cls): """returns True if this audio type supports MetaData""" return True def get_metadata(self): """returns a MetaData object, or None raises IOError if unable to read the file""" metadata = ApeTaggedAudio.get_metadata(self) if metadata is not None: metadata.frame_count = self.total_frames() return metadata def has_foreign_wave_chunks(self): """returns True if the audio file contains non-audio RIFF chunks during transcoding, if the source audio file has foreign RIFF chunks and the target audio format supports foreign RIFF chunks, conversion should be routed through .wav conversion to avoid losing those chunks""" for (sub_header, nondecoder, data_size, data) in self.sub_blocks(): if (sub_header == 1) and nondecoder: if (set(__riff_chunk_ids__(data_size, data)) != {b"fmt ", b"data"}): return True elif (sub_header == 2) and nondecoder: return True else: return False def wave_header_footer(self): """returns (header, footer) tuple of strings containing all data before and after the PCM stream may raise ValueError if there's a problem with the header or footer data may raise IOError if there's a problem reading header or footer data from the file """ head = None tail = None for (sub_block_id, nondecoder, data_size, data) in self.sub_blocks(): if (sub_block_id == 1) and nondecoder: head = data.read_bytes(data_size) elif (sub_block_id == 2) and nondecoder: tail = data.read_bytes(data_size) if head is not None: return (head, tail if tail is not None else b"") else: raise ValueError("no wave header found") @classmethod def from_wave(cls, filename, header, pcmreader, footer, compression=None, encoding_function=None): """encodes a new file from wave data takes a filename string, header string, PCMReader object, footer string and optional compression level string encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new WaveAudio object header + pcm data + footer should always result in the original wave file being restored without need for any padding bytes may raise EncodingError if some problem occurs when encoding the input file""" from audiotools.encoders import encode_wavpack from audiotools import BufferedPCMReader from audiotools import CounterPCMReader from audiotools.wav import (validate_header, validate_footer) from audiotools import EncodingError from audiotools import __default_quality__ if (((compression is None) or (compression not in cls.COMPRESSION_MODES))): compression = __default_quality__(cls.NAME) # ensure header is valid try: (total_size, data_size) = validate_header(header) except ValueError as err: raise EncodingError(str(err)) counter = CounterPCMReader(pcmreader) try: (encode_wavpack if encoding_function is None else encoding_function)( filename=filename, pcmreader=counter, compression=compression, wave_header=header, wave_footer=footer) counter.close() data_bytes_written = counter.bytes_written() # ensure output data size matches the "data" chunk's size if data_size != data_bytes_written: from audiotools.text import ERR_WAV_TRUNCATED_DATA_CHUNK raise EncodingError(ERR_WAV_TRUNCATED_DATA_CHUNK) # ensure footer validates correctly try: validate_footer(footer, data_bytes_written) except ValueError as err: raise EncodingError(str(err)) # ensure total size is correct if (len(header) + data_size + len(footer)) != total_size: from audiotools.text import ERR_WAV_INVALID_SIZE raise EncodingError(ERR_WAV_INVALID_SIZE) return cls(filename) except (ValueError, IOError) as msg: counter.close() cls.__unlink__(filename) raise EncodingError(str(msg)) except Exception as err: counter.close() cls.__unlink__(filename) raise err def blocks(self, reader=None): """yields (length, reader) tuples of WavPack frames length is the total length of all the substreams reader is a BitstreamReader which can be parsed """ def blocks_iter(reader): try: while True: (wvpk, block_size) = reader.parse("4b 32u 192p") if wvpk == b"wvpk": yield (block_size - 24, reader.substream(block_size - 24)) else: return except IOError: return if reader is None: from audiotools.bitstream import BitstreamReader with BitstreamReader(open(self.filename, "rb"), True) as reader: for block in blocks_iter(reader): yield block else: for block in blocks_iter(reader): yield block def sub_blocks(self, reader=None): """yields (function, nondecoder, data_size, data) tuples function is an integer nondecoder is a boolean indicating non-decoder data data is a BitstreamReader which can be parsed """ for (block_size, block_data) in self.blocks(reader): while block_size > 0: (metadata_function, nondecoder_data, actual_size_1_less, large_block) = block_data.parse("5u 1u 1u 1u") if large_block: sub_block_size = block_data.read(24) block_size -= 4 else: sub_block_size = block_data.read(8) block_size -= 2 if actual_size_1_less: yield (metadata_function, nondecoder_data, sub_block_size * 2 - 1, block_data.substream(sub_block_size * 2 - 1)) block_data.skip(8) else: yield (metadata_function, nondecoder_data, sub_block_size * 2, block_data.substream(sub_block_size * 2)) block_size -= sub_block_size * 2 def __read_info__(self): from audiotools.bitstream import BitstreamReader from audiotools import ChannelMask with BitstreamReader(open(self.filename, "rb"), True) as reader: pos = reader.getpos() (block_id, total_samples, bits_per_sample, mono_output, initial_block, final_block, sample_rate) = reader.parse( "4b 64p 32u 64p 2u 1u 8p 1u 1u 5p 5p 4u 37p") if block_id != b"wvpk": from audiotools.text import ERR_WAVPACK_INVALID_HEADER raise InvalidWavPack(ERR_WAVPACK_INVALID_HEADER) if sample_rate != 0xF: self.__samplerate__ = WavPackAudio.SAMPLING_RATE[sample_rate] else: # if unknown, pull from SAMPLE_RATE sub-block for (block_id, nondecoder, data_size, data) in self.sub_blocks(reader): if (block_id == 0x7) and nondecoder: self.__samplerate__ = data.read(data_size * 8) break else: # no SAMPLE RATE sub-block found # so pull info from FMT chunk reader.setpos(pos) (self.__samplerate__,) = self.fmt_chunk(reader).parse( "32p 32u") self.__bitspersample__ = [8, 16, 24, 32][bits_per_sample] self.__total_frames__ = total_samples if initial_block and final_block: if mono_output: self.__channels__ = 1 self.__channel_mask__ = ChannelMask(0x4) else: self.__channels__ = 2 self.__channel_mask__ = ChannelMask(0x3) else: # if not mono or stereo, pull from CHANNEL INFO sub-block reader.setpos(pos) for (block_id, nondecoder, data_size, data) in self.sub_blocks(reader): if (block_id == 0xD) and not nondecoder: self.__channels__ = data.read(8) self.__channel_mask__ = ChannelMask( data.read((data_size - 1) * 8)) break else: # no CHANNEL INFO sub-block found # so pull info from FMT chunk reader.setpos(pos) fmt = self.fmt_chunk(reader) compression_code = fmt.read(16) self.__channels__ = fmt.read(16) if compression_code == 1: # this is theoretically possible # with very old .wav files, # but shouldn't happen in practice self.__channel_mask__ = \ {1: ChannelMask.from_fields(front_center=True), 2: ChannelMask.from_fields(front_left=True, front_right=True), 3: ChannelMask.from_fields(front_left=True, front_right=True, front_center=True), 4: ChannelMask.from_fields(front_left=True, front_right=True, back_left=True, back_right=True), 5: ChannelMask.from_fields(front_left=True, front_right=True, back_left=True, back_right=True, front_center=True), 6: ChannelMask.from_fields(front_left=True, front_right=True, back_left=True, back_right=True, front_center=True, low_frequency=True) }.get(self.__channels__, ChannelMask(0)) elif compression_code == 0xFFFE: fmt.skip(128) mask = fmt.read(32) self.__channel_mask__ = ChannelMask(mask) else: from audiotools.text import ERR_WAVPACK_UNSUPPORTED_FMT raise InvalidWavPack(ERR_WAVPACK_UNSUPPORTED_FMT) def bits_per_sample(self): """returns an integer number of bits-per-sample this track contains""" return self.__bitspersample__ def channels(self): """returns an integer number of channels this track contains""" return self.__channels__ def total_frames(self): """returns the total PCM frames of the track as an integer""" return self.__total_frames__ def sample_rate(self): """returns the rate of the track's audio as an integer number of Hz""" return self.__samplerate__ def seekable(self): """returns True if the file is seekable""" return True @classmethod def supports_from_pcm(cls): """returns True if all necessary components are available to support the .from_pcm() classmethod""" try: from audiotools.encoders import encode_wavpack return True except ImportError: return False @classmethod def from_pcm(cls, filename, pcmreader, compression=None, total_pcm_frames=None, encoding_function=None): """encodes a new file from PCM data takes a filename string, PCMReader object, optional compression level string and optional total_pcm_frames integer encodes a new audio file from pcmreader's data at the given filename with the specified compression level and returns a new WavPackAudio object""" from audiotools.encoders import encode_wavpack from audiotools import BufferedPCMReader from audiotools import CounterPCMReader from audiotools import EncodingError from audiotools import __default_quality__ if pcmreader.bits_per_sample not in (8, 16, 24): # WavPack technically supports up to 32 bits-per-sample # but nothing else does # so I'll treat it as unsupported for now from audiotools import UnsupportedBitsPerSample pcmreader.close() raise UnsupportedBitsPerSample(filename, pcmreader.bits_per_sample) if (((compression is None) or (compression not in cls.COMPRESSION_MODES))): compression = __default_quality__(cls.NAME) counter = CounterPCMReader(pcmreader) try: (encode_wavpack if encoding_function is None else encoding_function)( filename=filename, pcmreader=counter, total_pcm_frames=(total_pcm_frames if total_pcm_frames is not None else 0), compression=compression) counter.close() except (ValueError, IOError) as msg: counter.close() cls.__unlink__(filename) raise EncodingError(str(msg)) except Exception: counter.close() cls.__unlink__(filename) raise # ensure actual total PCM frames matches argument, if any if (((total_pcm_frames is not None) and (counter.frames_written != total_pcm_frames))): cls.__unlink__(filename) from audiotools.text import ERR_TOTAL_PCM_FRAMES_MISMATCH raise EncodingError(ERR_TOTAL_PCM_FRAMES_MISMATCH) return cls(filename) @classmethod def supports_to_pcm(cls): """returns True if all necessary components are available to support the .from_pcm() classmethod""" try: from audiotools.decoders import WavPackDecoder return True except ImportError: return False def to_pcm(self): """returns a PCMReader object containing the track's PCM data""" from audiotools.decoders import WavPackDecoder from audiotools import PCMReaderError try: return WavPackDecoder(self.filename) except IOError as err: return PCMReaderError(error_message=str(err), sample_rate=self.__samplerate__, channels=self.__channels__, channel_mask=int(self.channel_mask()), bits_per_sample=self.__bitspersample__) def fmt_chunk(self, reader=None): """returns the 'fmt' chunk as a BitstreamReader""" for (block_id, nondecoder, data_size, data) in self.sub_blocks(reader): if (block_id == 1) and nondecoder: (riff, wave) = data.parse("4b 32p 4b") if (riff != b"RIFF") or (wave != b"WAVE"): from audiotools.text import ERR_WAVPACK_INVALID_FMT raise InvalidWavPack(ERR_WAVPACK_INVALID_FMT) else: while True: (chunk_id, chunk_size) = data.parse("4b 32u") if chunk_id == b"fmt ": return data.substream(chunk_size) elif chunk_id == b"data": from audiotools.text import ERR_WAVPACK_INVALID_FMT raise InvalidWavPack(ERR_WAVPACK_INVALID_FMT) else: data.skip_bytes(chunk_size) else: from audiotools.text import ERR_WAVPACK_NO_FMT raise InvalidWavPack(ERR_WAVPACK_NO_FMT) @classmethod def supports_cuesheet(cls): return True def get_cuesheet(self): """returns the embedded Cuesheet-compatible object, or None raises IOError if a problem occurs when reading the file""" from audiotools import cue as cue from audiotools import SheetException metadata = self.get_metadata() if (metadata is not None): try: if (b'Cuesheet' in metadata.keys()): return cue.read_cuesheet_string( metadata[b'Cuesheet'].__unicode__()) elif (b'CUESHEET' in metadata.keys()): return cue.read_cuesheet_string( metadata[b'CUESHEET'].__unicode__()) else: return None except SheetException: # unlike FLAC, just because a cuesheet is embedded # does not mean it is compliant return None else: return None def set_cuesheet(self, cuesheet): """imports cuesheet data from a Sheet object Raises IOError if an error occurs setting the cuesheet""" import os.path from io import BytesIO from audiotools import (MetaData, Filename, FS_ENCODING) from audiotools import cue as cue from audiotools.cue import write_cuesheet from audiotools.ape import ApeTag if cuesheet is None: return self.delete_cuesheet() metadata = self.get_metadata() if metadata is None: metadata = ApeTag([]) cuesheet_data = BytesIO() write_cuesheet(cuesheet, u"{}".format(Filename(self.filename).basename()), cuesheet_data) metadata[b'Cuesheet'] = ApeTag.ITEM.string( b'Cuesheet', cuesheet_data.getvalue().decode(FS_ENCODING, 'replace')) self.update_metadata(metadata) def delete_cuesheet(self): """deletes embedded Sheet object, if any Raises IOError if a problem occurs when updating the file""" from audiotools.ape import ApeTag metadata = self.get_metadata() if ((metadata is not None) and isinstance(metadata, ApeTag)): if b"Cuesheet" in metadata.keys(): del(metadata[b"Cuesheet"]) self.update_metadata(metadata) elif b"CUESHEET" in metadata.keys(): del(metadata[b"CUESHEET"]) self.update_metadata(metadata) ================================================ FILE: audiotools-config ================================================ #!/usr/bin/python # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2020 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA from __future__ import print_function import sys import os.path PY3 = sys.version_info[0] >= 3 def display_defaults(config, msg): def label_string(s): """turns a string into a unicode string, if need be""" assert(isinstance(s, str)) return s if PY3 else s.decode("UTF-8") def display_unicode_option(table, label, value, indent=2): """label and value should be unicode""" assert(isinstance(label, str if PY3 else unicode)) assert(isinstance(value, str if PY3 else unicode)) row = table.row() row.add_column(u" " * indent) row.add_column(audiotools.output_text(label, style="bold"), "right") row.add_column(u" : ") row.add_column(value) def display_option(table, label, value, indent=2): """label should be a unicode object, value should be a string""" assert(isinstance(label, str if PY3 else unicode)) assert(isinstance(value, str)) display_unicode_option(table, label, label_string(value), indent) def display_boolean_option(table, label, value, indent=2): """value should be a boolean""" assert(isinstance(label, str if PY3 else unicode)) display_unicode_option(table, label, u"yes" if value else u"no", indent) def display_int_option(table, label, value, indent=2): """value should be an integer""" assert(isinstance(value, int)) display_unicode_option(table, label, u"{:d}".format(value), indent) table = audiotools.output_table() display_option(table, _.LAB_AT_CONFIG_VERBOSITY, config.get_default("Defaults", "verbosity", "normal"), indent=0) for row in table.format(msg.output_isatty()): msg.output(row) msg.output(u"") msg.output(_.OPT_CAT_EXTRACTION) table = audiotools.output_table() row = table.row() row.add_column(u" ") row.add_column(u"Format") row.add_column(u" ") row.add_column(u"Readable") row.add_column(u" ") row.add_column(u"Writable") row.add_column(u" ") row.add_column(_.LAB_AT_CONFIG_DEFAULT_QUALITY) table.divider_row([u" ", _.DIV, u" ", _.DIV, u" ", _.DIV, u" ", _.DIV]) for audio_type in sorted(audiotools.AVAILABLE_TYPES, key=lambda x: x.NAME): row = table.row() row.add_column(u"") row.add_column(audiotools.output_text(label_string(audio_type.NAME)), "right") row.add_column(u"") row.add_column(u"yes" if audio_type.supports_to_pcm() else u"no", "right") row.add_column(u"") row.add_column(u"yes" if audio_type.supports_from_pcm() else u"no", "right") row.add_column(u"") if len(audio_type.COMPRESSION_MODES) < 2: row.add_column(u"") else: row.add_column( label_string( config.get_default("Quality", audio_type.NAME, audio_type.DEFAULT_COMPRESSION)), "right") for row in table.format(msg.output_isatty()): msg.output(row) msg.output(u"") table = audiotools.output_table() display_option( table, _.LAB_OPTIONS_FILENAME_FORMAT, config.get_default("Filenames", "format", audiotools.DEFAULT_FILENAME_FORMAT)) for row in table.format(msg.output_isatty()): msg.output(row) table = audiotools.output_table() display_int_option( table, _.LAB_AT_CONFIG_JOBS, config.getint_default("System", "maximum_jobs", audiotools.MAX_JOBS)) display_boolean_option( table, _.LAB_AT_CONFIG_ADD_REPLAY_GAIN, config.getboolean_default("ReplayGain", "add_by_default", True)) for row in table.format(msg.output_isatty()): msg.output(row) msg.output(u"") msg.output(_.OPT_CAT_ID3) table = audiotools.output_table() display_unicode_option( table, _.LAB_AT_CONFIG_ID3V2_VERSION, {"id3v2.2": _.LAB_AT_CONFIG_ID3V2_ID3V22, "id3v2.3": _.LAB_AT_CONFIG_ID3V2_ID3V23, "id3v2.4": _.LAB_AT_CONFIG_ID3V2_ID3V24, "none": _.LAB_AT_CONFIG_ID3V2_NONE}.get( config.get_default("ID3", "id3v2", "id3v2.3"))) display_unicode_option( table, _.LAB_AT_CONFIG_ID3V2_PADDING, {True: _.LAB_AT_CONFIG_ID3V2_PADDING_YES, False: _.LAB_AT_CONFIG_ID3V2_PADDING_NO}.get( config.getboolean_default("ID3", "pad", False))) display_unicode_option( table, _.LAB_AT_CONFIG_ID3V1_VERSION, {"id3v1.1": _.LAB_AT_CONFIG_ID3V1_ID3V11, "none": _.LAB_AT_CONFIG_ID3V1_NONE}.get( config.get_default("ID3", "id3v1", "id3v1.1"))) for row in table.format(msg.output_isatty()): msg.output(row) msg.output(u"") msg.output(_.OPT_CAT_CD_LOOKUP) table = audiotools.output_table() display_boolean_option(table, _.LAB_AT_CONFIG_USE_MUSICBRAINZ, config.getboolean_default("MusicBrainz", "service", True)) display_option(table, _.LAB_AT_CONFIG_MUSICBRAINZ_SERVER, config.get_default("MusicBrainz", "server", "musicbrainz.org")) display_int_option(table, _.LAB_AT_CONFIG_MUSICBRAINZ_PORT, config.getint_default("MusicBrainz", "port", 80)) table.blank_row() for row in table.format(msg.output_isatty()): msg.output(row) msg.output(u"") msg.output(_.OPT_CAT_SYSTEM) table = audiotools.output_table() display_option( table, _.LAB_AT_CONFIG_DEFAULT_CDROM, config.get_default("System", "cdrom", "/dev/cdrom")) display_int_option( table, _.LAB_AT_CONFIG_CDROM_READ_OFFSET, config.getint_default("System", "cdrom_read_offset", 0)) display_int_option( table, _.LAB_AT_CONFIG_CDROM_WRITE_OFFSET, config.getint_default("System", "cdrom_write_offset", 0)) display_option( table, _.LAB_AT_CONFIG_FS_ENCODING, config.get_default("System", "fs_encoding", sys.getfilesystemencoding())) display_unicode_option( table, _.LAB_AT_CONFIG_AUDIO_OUTPUT, u", ".join([label_string(player.NAME) for player in audiotools.player.available_outputs()])) for row in table.format(msg.output_isatty()): msg.output(row) def apply_options(options, config): """given an OptionParser's options dict and audiotools.RawConfigParser object applies the options to the config""" # apply --verbose option if options.verbosity is not None: config.set_default("Defaults", "verbosity", options.verbosity) # apply transcoding options if options.filename_format is not None: config.set_default("Filenames", "format", options.filename_format) if options.quality is None: # not setting no --quality value if options.type is None: # do nothing pass else: # set new default output type config.set_default("System", "default_type", options.type) else: # setting new --quality value if options.type is None: # set new quality value for current default type AudioType = audiotools.TYPE_MAP[audiotools.DEFAULT_TYPE] else: # set new quality value for given type AudioType = audiotools.TYPE_MAP[options.type] config.set_default("Quality", AudioType.NAME, options.quality) if options.system_maximum_jobs is not None: config.set_default("System", "maximum_jobs", str(options.system_maximum_jobs)) # apply CD lookup options if options.use_musicbrainz is not None: config.set_default("MusicBrainz", "service", True if (options.use_musicbrainz == "yes") else False) if options.musicbrainz_server is not None: config.set_default("MusicBrainz", "server", options.musicbrainz_server) if options.musicbrainz_port is not None: config.set_default("MusicBrainz", "port", options.musicbrainz_port) # apply ID3 options if options.id3v2_version is not None: config.set_default("ID3", "id3v2", options.id3v2_version) if options.id3v1_version is not None: config.set_default("ID3", "id3v1", options.id3v1_version) if options.id3_digit_padding is not None: config.set_default("ID3", "pad", True if (options.id3_digit_padding == "yes") else False) # apply ReplayGain options if options.add_replaygain is not None: config.set_default("ReplayGain", "add_by_default", True if (options.add_replaygain == "yes") else False) # apply system options if options.system_cdrom is not None: config.set_default("System", "cdrom", options.system_cdrom) if options.system_cdrom_read_offset is not None: config.set_default("System", "cdrom_read_offset", options.system_cdrom_read_offset) if options.system_cdrom_write_offset is not None: config.set_default("System", "cdrom_write_offset", options.system_cdrom_write_offset) if options.system_fs_encoding is not None: config.set_default("System", "fs_encoding", options.system_fs_encoding) # apply binaries options bins = set() for audioclass in audiotools.AVAILABLE_TYPES: for binary in audioclass.BINARIES: bins.add(binary) for binary in bins: setting = getattr(options, "binary_" + binary) if setting is not None: config.set_default("Binaries", binary, setting) if (__name__ == '__main__'): import argparse # There's no good way to make these dynamic # since the configurable text comes from the audiotools.text module # which we can't load because the module isn't installed correctly. try: import audiotools import audiotools.ui except ImportError: print("* audiotools Python module not found!") print("Perhaps you should re-install the Python Audio Tools") sys.exit(1) try: import audiotools.player except ImportError: print("* audiotools.player Python module not found!") print("Perhaps you should re-install the Python Audio Tools") sys.exit(1) import audiotools.text as _ if audiotools.ui.AVAILABLE: # setup widgets for interactive mode urwid = audiotools.ui.urwid import termios class TranscodingOptions(urwid.ListBox): def __init__(self, config): # get defaults from current config file DEFAULT_TYPE = config.get_default("System", "default_type", "wav") if not audiotools.TYPE_MAP[DEFAULT_TYPE].supports_from_pcm(): DEFAULT_TYPE = "wav" audio_types = list(sorted(audiotools.TYPE_MAP.values(), key=lambda x: x.NAME)) name_size = max(len(t.NAME) for t in audio_types) default_format = [] format_rows = [] format_rows.append( urwid.Columns( [("fixed", len(audiotools.output_text( _.LAB_AT_CONFIG_DEFAULT)), urwid.Text(("label", _.LAB_AT_CONFIG_DEFAULT))), ("fixed", len(audiotools.output_text( _.LAB_AT_CONFIG_TYPE)), urwid.Text(("label", _.LAB_AT_CONFIG_TYPE))), ("fixed", 1, urwid.Text(u" ")), ("weight", 1, urwid.Text(("label", _.LAB_AT_CONFIG_DEFAULT_QUALITY)))], dividechars=1)) format_rows.append( urwid.Columns( [("fixed", len(audiotools.output_text( _.LAB_AT_CONFIG_DEFAULT)), urwid.Divider(u"\u2500")), ("fixed", len(audiotools.output_text( _.LAB_AT_CONFIG_TYPE)), urwid.Divider(u"\u2500")), ("fixed", 1, urwid.Text(u" ")), ("weight", 1, urwid.Divider(u"\u2500"))], dividechars=1)) format_example = urwid.Text(markup=u"", wrap='clip') for audio_type in audio_types: if len(audio_type.COMPRESSION_MODES) < 2: qualities = urwid.Text(u"") no_modes = True else: DEFAULT_QUALITY = config.get_default( "Quality", audio_type.NAME, audio_type.DEFAULT_COMPRESSION) qualities = \ audiotools.ui.SelectOne( [(u"{}".format(q) if q not in audio_type.COMPRESSION_DESCRIPTIONS else u"{} - {}".format( q, audio_type.COMPRESSION_DESCRIPTIONS[q]), q) for q in audio_type.COMPRESSION_MODES], DEFAULT_QUALITY, on_change=self.change_quality, user_data=(config, "Quality", audio_type.NAME), label=_.LAB_OPTIONS_AUDIO_QUALITY) no_modes = False format_rows.append( urwid.Columns( [("fixed", len(audiotools.output_text( _.LAB_AT_CONFIG_DEFAULT)), urwid.RadioButton( default_format, label=u"", state=(audio_type.NAME == DEFAULT_TYPE), on_state_change=self.change_default_format, user_data=(config, audio_type.NAME, format_example))), ("fixed", len(audiotools.output_text( _.LAB_AT_CONFIG_TYPE)), urwid.Text( markup=u"{}".format(audio_type.NAME), align="right")), ("fixed", 1, urwid.Text(u" " if no_modes else u"-")), ("weight", 1, qualities) ], dividechars=1)) current_format = audiotools.Filename( config.get_default( "Filenames", "format", audiotools.DEFAULT_FILENAME_FORMAT)).__unicode__() format = urwid.Edit( caption=("label", u"{} : ".format(_.LAB_OPTIONS_FILENAME_FORMAT)), edit_text=current_format, wrap='clip') self.update_format_example(config, format_example) urwid.connect_signal(format, "change", self.update_format, (config, format_example)) format_rows.append(urwid.Divider(u" ")) format_rows.append( urwid.Columns( [('weight', 1, format), ('fixed', 10, audiotools.ui.BrowseFields(format))])) format_rows.append(format_example) joint = urwid.IntEdit( caption=("label", u"{} : ".format(_.LAB_AT_CONFIG_JOBS)), default=config.getint_default("System", "maximum_jobs", 1)) urwid.connect_signal(joint, "change", self.change_int, (config, "System", "maximum_jobs")) format_rows.append(joint) replay_gain_row = urwid.CheckBox( label=("label", _.LAB_AT_CONFIG_ADD_REPLAY_GAIN), state=config.getboolean_default( "ReplayGain", "add_by_default", True), on_state_change=self.change_boolean, user_data=(config, "ReplayGain", "add_by_default")) format_rows.append(replay_gain_row) urwid.ListBox.__init__( self, format_rows) def update_format(self, widget, new_value, user_data): (config, example_widget) = user_data config.set_default("Filenames", "format", new_value) self.update_format_example(config, example_widget) def update_format_example(self, config, example_widget): from time import strftime try: new_format = config.get_default( "Filenames", "format", audiotools.DEFAULT_FILENAME_FORMAT) default_audio_format = audiotools.TYPE_MAP.get( config.get_default("System", "default_type", "wav"), audiotools.WaveAudio) example_text = default_audio_format.track_name( file_path="file_name.{}".format( default_audio_format.SUFFIX), track_metadata=audiotools.MetaData( track_name=u"Track Name", track_number=1, track_total=2, album_name=u"Album Name", artist_name=u"Artist Name", performer_name=u"Performer Name", composer_name=u"Composer Name", conductor_name=u"Conductor Name", media=u"media", ISRC=u"CCXXXYYNNNNN", catalog=u"Catalog # ", copyright=u"Copyright", publisher=u"Publisher Name", year=u"{}".format(strftime("%Y")), date=u"{}".format(strftime("%x")), album_number=3, album_total=4, comment=u"Comment Text"), format=new_format) except (audiotools.UnsupportedTracknameField, audiotools.InvalidFilenameFormat): example_text = u"invalid filename format" example_widget.set_text( [("label", u"{} : ".format(_.LAB_OPTIONS_FILENAME_FORMAT_EXAMPLE)), example_text]) def change_text(self, widget, new_value, user_data): (config, section, option) = user_data config.set_default(section, option, new_value) def change_int(self, widget, new_value, user_data): (config, section, option) = user_data try: config.set_default(section, option, int(new_value)) except ValueError: config.set_default(section, option, 0) def change_default_format(self, radiobutton, new_state, user_data): if new_state: (config, value, example_widget) = user_data config.set_default("System", "default_type", value) self.update_format_example(config, example_widget) def change_quality(self, new_value, user_data): (config, section, option) = user_data config.set_default(section, option, new_value) def change_boolean(self, checkbox, new_state, user_data): (config, section, option) = user_data config.set_default(section, option, new_state) class CDLookup(urwid.ListBox): def __init__(self, config): # get defaults from current config file use_musicbrainz = urwid.CheckBox( label=("label", _.LAB_AT_CONFIG_USE_MUSICBRAINZ), state=config.getboolean_default( "MusicBrainz", "service", True), on_state_change=self.change_boolean, user_data=(config, "MusicBrainz", "service")) server = config.get_default("MusicBrainz", "server", "musicbrainz.org") musicbrainz_server = urwid.Edit( caption=("label", u"{} : ".format( _.LAB_AT_CONFIG_MUSICBRAINZ_SERVER)), edit_text=server if PY3 else server.decode("UTF-8")) urwid.connect_signal(musicbrainz_server, "change", self.change_text, (config, "MusicBrainz", "server")) musicbrainz_port = urwid.IntEdit( caption=("label", u"{} : ".format( _.LAB_AT_CONFIG_MUSICBRAINZ_PORT)), default=config.getint_default( "MusicBrainz", "port", 80)) urwid.connect_signal(musicbrainz_port, "change", self.change_int, (config, "MusicBrainz", "port")) urwid.ListBox.__init__(self, [use_musicbrainz, musicbrainz_server, musicbrainz_port]) def change_boolean(self, checkbox, new_state, user_data): (config, section, option) = user_data config.set_default(section, option, new_state) def change_text(self, widget, new_value, user_data): (config, section, option) = user_data config.set_default(section, option, new_value) def change_int(self, widget, new_value, user_data): (config, section, option) = user_data try: config.set_default(section, option, int(new_value)) except ValueError: config.set_default(section, option, 0) class ID3(urwid.ListBox): def __init__(self, config): default_id3v2_version = config.get_default("ID3", "id3v2", "id3v2.3") default_id3v1_version = config.get_default("ID3", "id3v1", "id3v1.1") default_id3v2_padding = config.getboolean_default( "ID3", "pad", False) id3v2_version = [] id3v2_version_row = urwid.Columns( [("fixed", len(audiotools.output_text( _.LAB_AT_CONFIG_ID3V2_VERSION)) + 3, urwid.Text(("label", u"{} : ".format( _.LAB_AT_CONFIG_ID3V2_VERSION))))] + [("weight", 1, urwid.RadioButton( group=id3v2_version, label=radio_label, state=radio_value == default_id3v2_version, on_state_change=self.change_choice, user_data=(config, "ID3", "id3v2", radio_value))) for (radio_value, radio_label) in [("id3v2.4", u"ID3v2.4"), ("id3v2.3", u"ID3v2.3"), ("id3v2.2", u"ID3v2.2"), ("none", u"No ID3v2")]]) id3v1_version = [] id3v1_version_row = urwid.Columns( [("fixed", len(audiotools.output_text( _.LAB_AT_CONFIG_ID3V1_VERSION)) + 3, urwid.Text(("label", u"{} : ".format( _.LAB_AT_CONFIG_ID3V1_VERSION))))] + [("weight", 1, urwid.RadioButton( group=id3v1_version, label=radio_label, state=radio_value == default_id3v1_version, on_state_change=self.change_choice, user_data=(config, "ID3", "id3v1", radio_value))) for (radio_value, radio_label) in [("id3v1.1", u"ID3v1.1"), ("none", u"No ID3v1")]]) id3_padding = [] id3_pad_row = urwid.Columns( [("fixed", len(audiotools.output_text( _.LAB_AT_CONFIG_ID3V2_PADDING)) + 3, urwid.Text(("label", u"{} : ".format( _.LAB_AT_CONFIG_ID3V2_PADDING))))] + [("weight", 1, urwid.RadioButton( group=id3_padding, label=radio_label, state=radio_value == default_id3v2_padding, on_state_change=self.change_choice, user_data=(config, "ID3", "pad", radio_value))) for (radio_value, radio_label) in [(True, _.LAB_AT_CONFIG_ID3V2_PADDING_YES), (False, _.LAB_AT_CONFIG_ID3V2_PADDING_NO)]]) urwid.ListBox.__init__(self, [id3v2_version_row, id3_pad_row, id3v1_version_row]) def change_choice(self, radiobutton, new_state, user_data): if new_state: (config, section, option, value) = user_data config.set_default(section, option, value) class CDROM(urwid.ListBox): def __init__(self, config): cdrom_device = urwid.Edit( caption=("label", u"{} : ".format(_.LAB_AT_CONFIG_DEFAULT_CDROM)), edit_text=audiotools.Filename( config.get_default( "System", "cdrom", "/dev/cdrom")).__unicode__()) urwid.connect_signal(cdrom_device, "change", self.change_text, (config, "System", "cdrom")) read_offset = urwid.IntEdit( caption=("label", u"{} : ".format( _.LAB_AT_CONFIG_CDROM_READ_OFFSET)), default=config.getint_default("System", "cdrom_read_offset", 0)) urwid.connect_signal(read_offset, "change", self.change_int, (config, "System", "cdrom_read_offset")) write_offset = urwid.IntEdit( caption=("label", u"{} : ".format( _.LAB_AT_CONFIG_CDROM_WRITE_OFFSET)), default=config.getint_default("System", "cdrom_write_offset", 0)) urwid.connect_signal(write_offset, "change", self.change_int, (config, "System", "cdrom_write_offset")) urwid.ListBox.__init__(self, [cdrom_device, read_offset, write_offset]) def change_text(self, widget, new_value, user_data): (config, section, option) = user_data config.set_default(section, option, new_value) def change_int(self, widget, new_value, user_data): (config, section, option) = user_data try: config.set_default(section, option, int(new_value)) except ValueError: config.set_default(section, option, 0) class AudiotoolsConfig(urwid.Frame): def __init__(self, config): self.config = config self.__cancelled__ = True self.status = urwid.Text(u"") transcode_box = urwid.LineBox( TranscodingOptions(config), title=_.OPT_CAT_EXTRACTION) cd_lookup_box = urwid.LineBox(CDLookup(config), title=_.OPT_CAT_CD_LOOKUP) id3_box = urwid.LineBox(ID3(config), title=_.OPT_CAT_ID3) verbosity = [] verbosity_level = config.get_default("Defaults", "verbosity", "normal") verbosity_row = urwid.Columns( [("fixed", len(audiotools.output_text( _.LAB_AT_CONFIG_VERBOSITY)) + 3, urwid.Text(("label", u"{} : ".format( _.LAB_AT_CONFIG_VERBOSITY))))] + [("weight", 1, urwid.RadioButton( group=verbosity, label={"quiet": u"quiet", "normal": u"normal", "debug": u"debug"}[level], state=level == verbosity_level, on_state_change=self.change_choice, user_data=(config, "Defaults", "verbosity", level))) for level in audiotools.VERBOSITY_LEVELS]) cdrom_box = urwid.LineBox(CDROM(config), title=_.OPT_CAT_SYSTEM) completion_buttons = urwid.Filler( urwid.Columns( widget_list=[('weight', 1, urwid.Button(_.LAB_CANCEL_BUTTON, on_press=self.cancel)), ('weight', 2, urwid.Button(_.LAB_APPLY_BUTTON, on_press=self.apply))], dividechars=3, focus_column=1)) option_widgets = urwid.ListBox( [urwid.LineBox(verbosity_row), urwid.BoxAdapter(transcode_box, len(audiotools.TYPE_MAP) + 8), urwid.BoxAdapter(id3_box, 5), urwid.BoxAdapter(cd_lookup_box, 9), urwid.BoxAdapter(cdrom_box, 5)]) urwid.Frame.__init__( self, body=urwid.Pile( [("weight", 1, option_widgets), ("fixed", 1, urwid.Filler(urwid.Divider(div_char=u"\u2500"))), ("fixed", 1, completion_buttons)]), footer=self.status) def change_choice(self, radiobutton, new_state, user_data): if new_state: (config, section, option, value) = user_data config.set_default(section, option, value) def change_boolean(self, checkbox, new_state, user_data): (config, section, option) = user_data config.set_default(section, option, new_state) def change_text(self, widget, new_value, user_data): (config, section, option) = user_data config.set_default(section, option, new_value) def change_int(self, widget, new_value, user_data): (config, section, option) = user_data try: config.set_default(section, option, int(new_value)) except ValueError: config.set_default(section, option, 0) def handle_text(self, i): if i == 'esc': self.__cancelled__ = True raise urwid.ExitMainLoop() def apply(self, button): # ensure --format is valid before returning try: audiotools.AudioFile.track_name( file_path="", track_metadata=audiotools.MetaData(), format=self.config.get_default( "Filenames", "format", audiotools.DEFAULT_FILENAME_FORMAT)) self.__cancelled__ = False raise urwid.ExitMainLoop() except audiotools.UnsupportedTracknameField as err: self.status.set_text(("error", _.ERR_INVALID_FILENAME_FORMAT)) except audiotools.InvalidFilenameFormat as err: self.status.set_text(("error", _.ERR_INVALID_FILENAME_FORMAT)) def cancel(self, button): self.__cancelled__ = True raise urwid.ExitMainLoop() def cancelled(self): return self.__cancelled__ parser = argparse.ArgumentParser(description=_.DESCRIPTION_AT_CONFIG) parser.add_argument("--version", action="version", version=audiotools.VERSION_STR) parser.add_argument("-I", "--interactive", action="store_true", default=False, dest="interactive", help=_.OPT_INTERACTIVE_AT_CONFIG) parser.add_argument("-V", "--verbose", dest="verbosity", choices=audiotools.VERBOSITY_LEVELS, default=audiotools.DEFAULT_VERBOSITY, help=_.OPT_VERBOSE_AT_CONFIG) transcoding = parser.add_argument_group(_.OPT_CAT_TRANSCODING) transcoding.add_argument("-t", "--type", dest="type", choices=sorted(list(audiotools.TYPE_MAP.keys()) + ["help"]), help=_.OPT_TYPE_AT_CONFIG) transcoding.add_argument("-q", "--quality", dest="quality", help=_.OPT_QUALITY_AT_CONFIG) transcoding.add_argument("--format", metavar="FORMAT", dest="filename_format", help=_.OPT_FORMAT) transcoding.add_argument("-j", "--joint", type=int, metavar="MAX_PROCESSES", dest="system_maximum_jobs", help=_.OPT_JOINT) transcoding.add_argument("--replay-gain", choices=("yes", "no"), dest="add_replaygain", help=_.OPT_REPLAY_GAIN) id3 = parser.add_argument_group(_.OPT_CAT_ID3) id3.add_argument("--id3v2-version", choices=("id3v2.2", "id3v2.3", "id3v2.4", "none"), dest="id3v2_version", metavar="VERSION", help=_.OPT_AT_CONFIG_ID3V2_VERSION) id3.add_argument("--id3v2-pad", choices=("yes", "no"), dest="id3_digit_padding", help=_.OPT_AT_CONFIG_ID3V2_PAD) id3.add_argument("--id3v1-version", choices=("id3v1.1", "none"), dest="id3v1_version", metavar="VERSION", help=_.OPT_AT_CONFIG_ID3V1_VERSION) lookup = parser.add_argument_group(_.OPT_CAT_CD_LOOKUP) lookup.add_argument("--use-musicbrainz", choices=("yes", "no"), dest="use_musicbrainz") lookup.add_argument("--musicbrainz-server", action="store", metavar="HOSTNAME", dest="musicbrainz_server") lookup.add_argument("--musicbrainz-port", action="store", metavar="PORT", type=int, dest="musicbrainz_port") system = parser.add_argument_group(_.OPT_CAT_SYSTEM) system.add_argument("-c", "--cdrom", action="store", dest="system_cdrom", metavar="PATH") system.add_argument("--cdrom-read-offset", action="store", type=int, metavar="INT", dest="system_cdrom_read_offset", help=_.OPT_AT_CONFIG_READ_OFFSET) system.add_argument("--cdrom-write-offset", action="store", type=int, metavar="INT", dest="system_cdrom_write_offset", help=_.OPT_AT_CONFIG_WRITE_OFFSET) system.add_argument("--fs-encoding", action="store", metavar="ENCODING", dest="system_fs_encoding", help=_.OPT_AT_CONFIG_FS_ENCODING) binaries = parser.add_argument_group(_.OPT_CAT_BINARIES) bins = set() for audioclass in audiotools.AVAILABLE_TYPES: for binary in audioclass.BINARIES: bins.add(binary) for binary in sorted(list(bins)): binaries.add_argument('--' + binary, metavar='PATH', dest='binary_' + binary) options = parser.parse_args() msg = audiotools.Messenger() if len(sys.argv) < 2: # no arguments at all so display current default display_defaults(audiotools.config, msg) elif options.interactive: # update options interactively if not audiotools.ui.AVAILABLE: audiotools.ui.not_available_message(msg) sys.exit(1) else: # apply options to config file apply_options(options, audiotools.config) # run interactive widget here widget = AudiotoolsConfig(audiotools.config) loop = audiotools.ui.urwid.MainLoop( widget, audiotools.ui.style(), screen=audiotools.ui.Screen(), unhandled_input=widget.handle_text, pop_ups=True) try: loop.run() msg.ansi_clearscreen() except (termios.error, IOError): msg.error(_.ERR_TERMIOS_ERROR) msg.info(_.ERR_TERMIOS_SUGGESTION) msg.info(audiotools.ui.xargs_suggestion(sys.argv)) sys.exit(1) # and apply options if widget isn't cancelled if not widget.cancelled(): configpath = os.path.expanduser('~/.audiotools.cfg') try: configfile = open(configpath, 'w') audiotools.config.write(configfile) configfile.close() msg.info(_.LAB_AT_CONFIG_FILE_WRITTEN.format( audiotools.Filename(configpath))) except IOError as err: msg.error( _.ERR_OPEN_IOERROR.format( audiotools.Filename(configpath))) sys.exit(1) else: sys.exit(0) else: # update options non-interactively # verify --format is valid, if present if options.filename_format is not None: try: audiotools.AudioFile.track_name( file_path="", track_metadata=audiotools.MetaData(), format=options.filename_format) except audiotools.UnsupportedTracknameField as err: err.error_msg(msg) sys.exit(1) except audiotools.InvalidFilenameFormat as err: msg.error(err) sys.exit(1) # verify --type is valid, if present if options.type == 'help': audiotools.ui.show_available_formats(msg) sys.exit(0) elif options.type is not None: AudioType = audiotools.TYPE_MAP[options.type] else: AudioType = audiotools.TYPE_MAP[audiotools.DEFAULT_TYPE] # verify --quality is valid for type, if present if options.quality == 'help': audiotools.ui.show_available_qualities(msg, AudioType) sys.exit(0) elif ((options.quality is not None) and (options.quality not in AudioType.COMPRESSION_MODES)): msg.error( _.ERR_UNSUPPORTED_COMPRESSION_MODE.format( quality=options.quality, type=AudioType.NAME)) sys.exit(1) # verify --joint is positive, if present if (((options.system_maximum_jobs is not None) and (options.system_maximum_jobs < 1))): msg.error(_.ERR_INVALID_JOINT) sys.exit(1) # apply options non-interactively apply_options(options, audiotools.config) configpath = os.path.expanduser('~/.audiotools.cfg') try: configfile = open(configpath, 'w') audiotools.config.write(configfile) configfile.close() msg.info(_.LAB_AT_CONFIG_FILE_WRITTEN.format( audiotools.Filename(configpath))) except IOError as err: msg.error( _.ERR_OPEN_IOERROR.format( audiotools.Filename(configpath))) sys.exit(1) ================================================ FILE: cdda2track ================================================ #!/usr/bin/python # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2020 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA import sys import os import audiotools import audiotools.ui import audiotools.accuraterip from audiotools.cdio import CDDAReader import termios import audiotools.text as _ PREVIOUS_TRACK_FRAMES = (5880 // 2) NEXT_TRACK_FRAMES = (5880 // 2) def merge_metadatas(metadatas): if len(metadatas) == 0: return audiotools.MetaData() elif len(metadatas) == 1: return metadatas[0] else: merged = metadatas[0] for to_merge in metadatas[1:]: merged = merged.intersection(to_merge) return merged class AccurateRipReader(object): def __init__(self, pcmreader, total_pcm_frames, is_first, is_last): """pcmreader is a PCMReader object to wrap around total_pcm_frames is the length of pcmreader, not including previous and next track frames is_first and is_last indicate the track's position in the stream""" self.pcmreader = pcmreader self.checksummer = audiotools.accuraterip.Checksum( total_pcm_frames=total_pcm_frames, sample_rate=pcmreader.sample_rate, is_first=is_first, is_last=is_last, pcm_frame_range=PREVIOUS_TRACK_FRAMES + 1 + NEXT_TRACK_FRAMES, accurateripv2_offset=PREVIOUS_TRACK_FRAMES) self.sample_rate = pcmreader.sample_rate self.channels = pcmreader.channels self.channel_mask = pcmreader.channel_mask self.bits_per_sample = pcmreader.bits_per_sample def read(self, pcm_frames): frame = self.pcmreader.read(pcm_frames) self.checksummer.update(frame) return frame def close(self): self.pcmreader.close() def checksums_v1(self): return self.checksummer.checksums_v1() def checksums_v2(self): return [self.checksummer.checksum_v2()] if (__name__ == '__main__'): import argparse parser = argparse.ArgumentParser(description=_.DESCRIPTION_CD2TRACK) parser.add_argument("--version", action="version", version=audiotools.VERSION_STR) parser.add_argument("-I", "--interactive", action="store_true", default=False, dest="interactive", help=_.OPT_INTERACTIVE_OPTIONS) parser.add_argument("--cue", dest="cuesheet", metavar="FILENAME", help=_.OPT_CUESHEET_CDDA2TRACK) parser.add_argument("-V", "--verbose", dest="verbosity", choices=audiotools.VERBOSITY_LEVELS, default=audiotools.DEFAULT_VERBOSITY, help=_.OPT_VERBOSE) parser.add_argument("-c", "--cdrom", dest="cdrom", default=audiotools.DEFAULT_CDROM) parser.add_argument("-s", "--speed", type=int, dest="speed") conversion = parser.add_argument_group(_.OPT_CAT_EXTRACTION) conversion.add_argument( "-t", "--type", dest="type", choices=sorted(list(t.NAME for t in audiotools.AVAILABLE_TYPES if t.supports_from_pcm()) + ["help"]), help=_.OPT_TYPE, default=audiotools.DEFAULT_TYPE) conversion.add_argument("-q", "--quality", dest="quality", help=_.OPT_QUALITY) conversion.add_argument("-d", "--dir", dest="dir", default=".", help=_.OPT_DIR) conversion.add_argument("--format", default=None, dest="format", help=_.OPT_FORMAT) lookup = parser.add_argument_group(_.OPT_CAT_CD_LOOKUP) lookup.add_argument("-M", "--metadata-lookup", action="store_true", default=False, dest="metadata_lookup", help=_.OPT_METADATA_LOOKUP) lookup.add_argument("--musicbrainz-server", dest="musicbrainz_server", default=audiotools.MUSICBRAINZ_SERVER, metavar="HOSTNAME") lookup.add_argument("--musicbrainz-port", type=int, dest="musicbrainz_port", default=audiotools.MUSICBRAINZ_PORT, metavar="PORT") lookup.add_argument("--no-musicbrainz", action="store_false", dest="use_musicbrainz", default=audiotools.MUSICBRAINZ_SERVICE, help=_.OPT_NO_MUSICBRAINZ) lookup.add_argument("-D", "--default", dest="use_default", action="store_true", default=False, help=_.OPT_DEFAULT) metadata = parser.add_argument_group(_.OPT_CAT_METADATA) metadata.add_argument("--album-number", dest="album_number", type=int, default=0, help=_.OPT_ALBUM_NUMBER) metadata.add_argument("--album-total", dest="album_total", type=int, default=0, help=_.OPT_ALBUM_TOTAL) # if adding ReplayGain is a lossless process # (i.e. added as tags rather than modifying track data) # add_replay_gain should default to True # if not, add_replay_gain should default to False # which is which depends on the track type metadata.add_argument("--replay-gain", action="store_true", default=None, dest="add_replay_gain", help=_.OPT_REPLAY_GAIN) metadata.add_argument("--no-replay-gain", action="store_false", default=None, dest="add_replay_gain", help=_.OPT_NO_REPLAY_GAIN) parser.add_argument("tracks", type=int, metavar="TRACK", nargs="*", help=_.OPT_TRACK_INDEX) options = parser.parse_args() msg = audiotools.Messenger(options.verbosity == "quiet") # ensure interactive mode is available, if selected if options.interactive and (not audiotools.ui.AVAILABLE): audiotools.ui.not_available_message(msg) sys.exit(1) # get the default AudioFile class we are converted to if options.type == 'help': audiotools.ui.show_available_formats(msg) sys.exit(0) else: AudioType = audiotools.TYPE_MAP[options.type] # ensure the selected compression is compatible with that class if options.quality == 'help': audiotools.ui.show_available_qualities(msg, AudioType) sys.exit(0) elif options.quality is None: options.quality = audiotools.__default_quality__(AudioType.NAME) elif options.quality not in AudioType.COMPRESSION_MODES: msg.error( _.ERR_UNSUPPORTED_COMPRESSION_MODE.format( quality=options.quality, type=AudioType.NAME)) sys.exit(1) quality = options.quality base_directory = options.dir # get the CD-ROM reader try: cddareader = CDDAReader(options.cdrom, True) track_offsets = cddareader.track_offsets track_lengths = cddareader.track_lengths except (IOError, ValueError) as err: msg.error(_.ERR_INVALID_CDDA) sys.exit(-1) # only apply read_offset if cddareader is a physical drive if not cddareader.is_cd_image: try: read_offset = audiotools.config.getint_default( "System", "cdrom_read_offset", 0) except ValueError: read_offset = 0 else: read_offset = 0 # set speed, if any if options.speed is not None: cddareader.set_speed(options.speed) pre_gap_length = cddareader.track_offsets[1] # check whether pre-gap data should be preserved, if any if pre_gap_length > 0: with audiotools.BufferedPCMReader( audiotools.PCMReaderWindow(cddareader, read_offset, pre_gap_length, forward_close=False)) as r: # this could be optimized better # but it is a rare and unusual case preserve_pre_gap = set(r.read(pre_gap_length)) != {0} if preserve_pre_gap: track_offsets[0] = 0 track_lengths[0] = pre_gap_length else: preserve_pre_gap = False # use CDDA object to query metadata services for metadata choices try: metadata_choices = audiotools.cddareader_metadata_lookup( cddareader=cddareader, musicbrainz_server=options.musicbrainz_server, musicbrainz_port=options.musicbrainz_port, use_musicbrainz=options.use_musicbrainz) except KeyboardInterrupt: msg.ansi_clearscreen() msg.error(_.ERR_CANCELLED) sys.exit(1) if preserve_pre_gap: # prepend "track 0" track to start of list for each choice for choice in metadata_choices: track_0 = merge_metadatas(choice) track_0.track_number = 0 choice.insert(0, track_0) # update MetaData with command-line album-number/total, if given if options.album_number != 0: for c in metadata_choices: for m in c: m.album_number = options.album_number if options.album_total != 0: for c in metadata_choices: for m in c: m.album_total = options.album_total # use CDDAReader object to perform AccurateRip lookup try: ar_result = audiotools.accuraterip.perform_lookup( audiotools.accuraterip.DiscID.from_cddareader(cddareader)) except KeyboardInterrupt: msg.ansi_clearline() msg.error(_.ERR_CANCELLED) sys.exit(1) # determine tracks to be ripped if len(options.tracks) == 0: tracks_to_rip = list(sorted(track_offsets.keys())) else: tracks_to_rip = [t for t in options.tracks if t in track_offsets.keys()] if len(tracks_to_rip) == 0: # no tracks selected to rip, so do nothing sys.exit(0) # decide which metadata and output options use when extracting tracks if options.interactive: # pick options using interactive widget output_widget = audiotools.ui.OutputFiller( track_labels=[_.LAB_TRACK_X_OF_Y.format(i, max(track_offsets.keys())) for i in tracks_to_rip], metadata_choices=[[c for i, c in enumerate(choices, 0 if preserve_pre_gap else 1) if i in tracks_to_rip] for choices in metadata_choices], input_filenames=[ audiotools.Filename("track{:02d}.cdda.wav".format(i)) for i in tracks_to_rip], output_directory=options.dir, format_string=(options.format if (options.format is not None) else audiotools.FILENAME_FORMAT), output_class=AudioType, quality=quality, completion_label=_.LAB_CD2TRACK_APPLY) loop = audiotools.ui.urwid.MainLoop( output_widget, audiotools.ui.style(), screen=audiotools.ui.Screen(), unhandled_input=output_widget.handle_text, pop_ups=True) try: loop.run() msg.ansi_clearscreen() except (termios.error, IOError): msg.error(_.ERR_TERMIOS_ERROR) msg.info(_.ERR_TERMIOS_SUGGESTION) msg.info(audiotools.ui.xargs_suggestion(sys.argv)) sys.exit(1) if not output_widget.cancelled(): output_tracks = list(output_widget.output_tracks()) else: sys.exit(0) else: # pick options without using GUI try: output_tracks = list( audiotools.ui.process_output_options( metadata_choices=[ [c for i, c in enumerate(choices, 0 if preserve_pre_gap else 1) if i in tracks_to_rip] for choices in metadata_choices], input_filenames=[ audiotools.Filename("track{:02d}.cdda.wav".format(i)) for i in tracks_to_rip], output_directory=options.dir, format_string=options.format, output_class=AudioType, quality=options.quality, msg=msg, use_default=options.use_default)) except audiotools.UnsupportedTracknameField as err: err.error_msg(msg) sys.exit(1) except (audiotools.InvalidFilenameFormat, audiotools.OutputFileIsInput, audiotools.DuplicateOutputFile) as err: msg.error(err) sys.exit(1) # perform actual ripping of tracks from CDDA encoded = [] rip_log = {} accuraterip_log_v1 = {} accuraterip_log_v2 = {} replay_gain = audiotools.ReplayGainCalculator(cddareader.sample_rate) for (track_number, index, (output_class, output_filename, output_quality, output_metadata)) in zip(tracks_to_rip, range(1, len(tracks_to_rip) + 1), output_tracks): cddareader.reset_log() track_offset = (track_offsets[track_number] + read_offset - PREVIOUS_TRACK_FRAMES) track_length = track_lengths[track_number] # seek to indicated starting offset if track_offset > 0: seeked_offset = cddareader.seek(track_offset) else: seeked_offset = cddareader.seek(0) # make leading directories, if necessary try: audiotools.make_dirs(str(output_filename)) except OSError as err: msg.os_error(err) sys.exit(1) # setup individual progress bar per track progress = audiotools.SingleProgressDisplay( msg, output_filename.__unicode__()) # perform extraction over an AccurateRip window track_data = audiotools.PCMReaderWindow( cddareader, track_offset - seeked_offset, PREVIOUS_TRACK_FRAMES + track_length + NEXT_TRACK_FRAMES) # with AccurateRip calculated during extraction accuraterip = AccurateRipReader( track_data, track_length, track_number == min(track_offsets.keys()), track_number == max(track_offsets.keys())) try: # encode output file itself track = output_class.from_pcm( str(output_filename), replay_gain.to_pcm( audiotools.PCMReaderProgress( audiotools.PCMReaderWindow( accuraterip, PREVIOUS_TRACK_FRAMES, track_length, forward_close=False), track_length, progress.update)), output_quality, total_pcm_frames=track_length) encoded.append(track) # since the inner PCMReaderWindow only outputs part # of the accuraterip reader, we need to ensure # anything left over in accuraterip gets processed also audiotools.transfer_data(accuraterip.read, lambda f: None) except audiotools.EncodingError as err: progress.clear_rows() msg.error(_.ERR_ENCODING_ERROR.format(output_filename)) sys.exit(1) except KeyboardInterrupt: progress.clear_rows() try: os.unlink(str(output_filename)) except OSError: pass msg.error(_.ERR_CANCELLED) sys.exit(1) track.set_metadata(output_metadata) progress.clear_rows() rip_log[track_number] = cddareader.log() accuraterip_log_v1[track_number] = accuraterip.checksums_v1() accuraterip_log_v2[track_number] = accuraterip.checksums_v2() msg.info( audiotools.output_progress( _.LAB_CD2TRACK_PROGRESS.format(track_number=track_number, filename=output_filename), index, len(tracks_to_rip))) # add ReplayGain to ripped tracks, if necessary if (output_class.supports_replay_gain() and (options.add_replay_gain if options.add_replay_gain is not None else audiotools.ADD_REPLAYGAIN)): for (track, (track_gain, track_peak, album_gain, album_peak)) in zip(encoded, replay_gain): track.set_replay_gain( audiotools.ReplayGain(track_gain=track_gain, track_peak=track_peak, album_gain=album_gain, album_peak=album_peak)) else: msg.info(_.RG_REPLAYGAIN_ADDED) # write cuesheet from CD reader, if requested and if possible if options.cuesheet: if options.cuesheet.lower().endswith("toc"): from audiotools.toc import TOCFile as Cuesheet else: from audiotools.cue import Cuesheet from audiotools import Sheet try: with open(options.cuesheet, "w") as cuesheet: cuesheet.write( Cuesheet.converted( Sheet.from_cddareader(cddareader)).build()) msg.info(_.LAB_CDDA2TRACK_WROTE_CUESHEET.format(options.cuesheet)) except IOError: msg.error(_.ERR_OPEN_IOERROR.format(options.cuesheet)) sys.exit(1) # display ripping log msg.output(_.LAB_CD2TRACK_LOG) table = audiotools.output_table() header = table.row() header.add_column(u"", colspan=1 + 7 * 2 + 1) header.add_column(_.LAB_TRACKVERIFY_AR_VERSION1, "center", colspan=5) header.add_column(u"") header.add_column(_.LAB_TRACKVERIFY_AR_VERSION2, "center", colspan=5) # header for terminals with more horizontal space wide_header = [u" #", u"err", u"skip", u"atom", u"edge", u"drop", u"dup", u"drift", _.LAB_TRACKVERIFY_AR_CONFIDENCE, _.LAB_TRACKVERIFY_AR_OFFSET, _.LAB_TRACKVERIFY_AR_CHECKSUM, _.LAB_TRACKVERIFY_AR_CONFIDENCE, _.LAB_TRACKVERIFY_AR_OFFSET, _.LAB_TRACKVERIFY_AR_CHECKSUM] # header for terminals limited to 80 columns narrow_header = [u" #", u"err", u"skip", u"atom", u"edge", u"drop", u"dup", u"drift", _.LAB_TRACKVERIFY_AR_CONF, _.LAB_TRACKVERIFY_AR_OFFSET, _.LAB_TRACKVERIFY_AR_CHECKSUM, _.LAB_TRACKVERIFY_AR_CONF, _.LAB_TRACKVERIFY_AR_OFFSET, _.LAB_TRACKVERIFY_AR_CHECKSUM] if (msg.output_isatty() and (len(audiotools.output_text(u" ").join( [audiotools.output_text(h) for h in wide_header])) <= msg.terminal_size(sys.stdout.fileno())[1])): header_list = wide_header else: header_list = narrow_header header = table.row() for (label, is_first) in [(label, i == 0) for (i, label) in enumerate(header_list)]: if not is_first: header.add_column(u" ") header.add_column(label, "right") table.divider_row([_.DIV] + [u" ", _.DIV] * (7 + 3 + 3)) for track_number in tracks_to_rip: row = table.row() # first output the filename row.add_column(u"{:d}".format(track_number), "right") # then output the 7 log fields log = rip_log[track_number] for key in ["readrr", "skip", "fixup_atom", "fixup_edge", "fixup_dropped", "fixup_duped", "drift"]: row.add_column(u" ") row.add_column(u"{:d}".format(log.get(key, 0)), "right") (checksum_v2, confidence_v2, offset_v2) = audiotools.accuraterip.match_offset( ar_result.get(track_number, []), accuraterip_log_v2[track_number], 0) (checksum_v1, confidence_v1, offset_v1) = audiotools.accuraterip.match_offset( ar_result.get(track_number, []), accuraterip_log_v1[track_number], -PREVIOUS_TRACK_FRAMES) # finally output the 6 AccurateRip fields for (checksum, confidence, offset) in [(checksum_v1, confidence_v1, offset_v1), (checksum_v2, confidence_v2, offset_v2)]: row.add_column(u"") if (confidence is None) or (confidence < 0): row.add_column(u"", "right") else: row.add_column(u"{:d}".format(confidence), "right") row.add_column(u"") row.add_column(u"{:d}".format(offset), "right") row.add_column(u"") row.add_column(u"{:08X}".format(checksum), "right") for row in table.format(msg.output_isatty()): msg.output(row) ================================================ FILE: cddainfo ================================================ #!/usr/bin/python # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2020 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA import os.path import sys import audiotools from audiotools.cdio import CDDAReader import audiotools.freedb import audiotools.musicbrainz import audiotools.accuraterip import audiotools.text as _ if (__name__ == '__main__'): import argparse parser = argparse.ArgumentParser(description=_.DESCRIPTION_CDINFO) parser.add_argument("--version", action="version", version=audiotools.VERSION_STR) parser.add_argument("-c", "--cdrom", dest="cdrom", default=audiotools.DEFAULT_CDROM) options = parser.parse_args() msg = audiotools.Messenger() try: cdda = CDDAReader(options.cdrom) except (ValueError, IOError) as err: msg.error(err) sys.exit(1) offsets = cdda.track_offsets lengths = cdda.track_lengths total_length = cdda.last_sector + 150 + 1 table = audiotools.output_table() row = table.row() row.add_column(_.LAB_TOTAL_TRACKS, "right") row.add_column(u" : ") row.add_column(u"{:d}".format(max(offsets.keys()))) row = table.row() row.add_column(_.LAB_TOTAL_LENGTH, "right") row.add_column(u" : ") row.add_column(_.LAB_TRACK_LENGTH_FRAMES.format(total_length // 75 // 60, total_length // 75 % 60, total_length)) row = table.row() row.add_column(_.LAB_FREEDB_ID, "right") row.add_column(u" : ") row.add_column( audiotools.freedb.DiscID.from_cddareader(cdda).__unicode__()) row = table.row() row.add_column(_.LAB_MUSICBRAINZ_ID, "right") row.add_column(u" : ") row.add_column( audiotools.musicbrainz.DiscID.from_cddareader(cdda).__unicode__()) row = table.row() row.add_column(_.LAB_ACCURATERIP_ID, "right") row.add_column(u" : ") row.add_column( audiotools.accuraterip.DiscID.from_cddareader(cdda).__unicode__()) for row in table.format(sys.stdout.isatty()): msg.output(row) msg.output(u"") table = audiotools.output_table() row = table.row() row.add_column(u"#") row.add_column(u" ") row.add_column(_.LAB_CDINFO_LENGTH) row.add_column(u" ") row.add_column(_.LAB_CDINFO_FRAMES) row.add_column(u" ") row.add_column(_.LAB_CDINFO_OFFSET) row = table.divider_row([_.DIV, u" ", _.DIV, u" ", _.DIV, u" ", _.DIV]) for key in sorted(offsets.keys()): track_length = lengths[key] // 588 row = table.row() row.add_column(u"{:d}".format(key), "right") row.add_column(u" ") row.add_column( _.LAB_TRACK_LENGTH.format(track_length // 75 // 60, track_length // 75 % 60), "right") row.add_column(u" ") row.add_column(u"{:d}".format(track_length), "right") row.add_column(u" ") row.add_column(u"{:d}".format((offsets[key] // 588) + 150), "right") for row in table.format(sys.stdout.isatty()): msg.output(row) ================================================ FILE: cddaplay ================================================ #!/usr/bin/python # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2020 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA import sys import time import select import os import tty import termios import audiotools import audiotools.ui import audiotools.player from audiotools.cdio import CDDAReader import audiotools.text as _ if audiotools.ui.AVAILABLE: urwid = audiotools.ui.urwid class CDplayGUI(audiotools.ui.PlayerGUI): def __init__(self, cddareader, track_numbers, metadata, audio_output): """cddareader is a CDDAReader object track_numbers is a list of track numbers to play metadata is a dict of tracknum -> MetaData values audio_output is an AudioOutput object""" self.cddareader = cddareader track_lengths = cddareader.track_lengths audiotools.ui.PlayerGUI.__init__( self, audiotools.player.CDPlayer( cddareader=cddareader, audio_output=audio_output, next_track_callback=self.next_track), [(metadata[track_number].track_name if (metadata[track_number].track_name is not None) else (u"track {:02d}".format(track_number)), track_lengths[track_number] // 44100, (track_number, metadata[track_number])) for track_number in track_numbers], sum([track_lengths[track_number] for track_number in track_numbers]) // 44100) def select_track(self, radio_button, new_state, user_data, auto_play=True): if new_state: (track_number, metadata) = user_data track_lengths = self.cddareader.track_lengths if metadata is not None: self.update_metadata( track_name=metadata.track_name, album_name=metadata.album_name, artist_name=metadata.artist_name, track_number=track_number, track_total=len(track_lengths), album_number=metadata.album_number, album_total=metadata.album_total, pcm_frames=track_lengths[track_number], channels=2, sample_rate=44100, bits_per_sample=16) else: self.update_metadata( track_total=len(track_lengths), pcm_frames=track_lengths[track_number], channels=2, sample_rate=44100, bits_per_sample=16) self.player.open(track_number) if auto_play: self.player.play() self.play_pause_button.set_label(_.LAB_PAUSE_BUTTON) def select_first_track(self): self.select_track(None, True, (1, metadata[1]), False) interactive_available = True else: interactive_available = False class CDplayTTY(audiotools.ui.PlayerTTY): def __init__(self, cddareader, track_list, audio_output): """cddareader is a CDDAReader object track_list is a list of track numbers in the order to be played audio_output is an AudioOutput object""" self.cddareader = cddareader self.track_list = track_list self.track_index = -1 audiotools.ui.PlayerTTY.__init__( self, audiotools.player.CDPlayer( cddareader=cddareader, audio_output=audio_output, next_track_callback=self.next_track)) def previous_track(self): if self.track_index > 0: self.track_index -= 1 current_track_number = self.track_list[self.track_index] self.set_metadata( track_number=self.track_index + 1, track_total=len(self.track_list), channels=2, sample_rate=44100, bits_per_sample=16) self.player.open(current_track_number) self.player.play() def next_track(self): try: self.track_index += 1 current_track_number = self.track_list[self.track_index] self.set_metadata( track_number=self.track_index + 1, track_total=len(self.track_list), channels=2, sample_rate=44100, bits_per_sample=16) self.player.open(current_track_number) self.player.play() except IndexError: self.playing_finished = True if (__name__ == '__main__'): import argparse parser = argparse.ArgumentParser(description=_.DESCRIPTION_CDPLAY) parser.add_argument("--version", action="version", version=audiotools.VERSION_STR) parser.add_argument("-I", "--interactive", action="store_true", default=False, dest="interactive", help=_.OPT_INTERACTIVE_OPTIONS) parser.add_argument("-V", "--verbose", action="store", dest="verbosity", choices=audiotools.VERBOSITY_LEVELS, default=audiotools.DEFAULT_VERBOSITY, help=_.OPT_VERBOSE) parser.add_argument("-o", "--output", dest="output", choices=[player.NAME for player in audiotools.player.available_outputs()], default=[player.NAME for player in audiotools.player.available_outputs()][0], help=_.OPT_OUTPUT_PLAY) parser.add_argument("-c", "--cdrom", dest="cdrom", default=audiotools.DEFAULT_CDROM) parser.add_argument("--shuffle", action="store_true", dest="shuffle", default=False, help="shuffle tracks") lookup = parser.add_argument_group(_.OPT_CAT_CD_LOOKUP) lookup.add_argument("-M", "--metadata-lookup", action="store_true", default=False, dest="metadata_lookup", help=_.OPT_METADATA_LOOKUP) lookup.add_argument("--musicbrainz-server", dest="musicbrainz_server", default=audiotools.MUSICBRAINZ_SERVER, metavar="HOSTNAME") lookup.add_argument("--musicbrainz-port", type=int, dest="musicbrainz_port", default=audiotools.MUSICBRAINZ_PORT, metavar="PORT") lookup.add_argument("--no-musicbrainz", action="store_false", dest="use_musicbrainz", default=audiotools.MUSICBRAINZ_SERVICE, help=_.OPT_NO_MUSICBRAINZ) parser.add_argument("tracks", type=int, metavar="TRACK", nargs="*", help=_.OPT_TRACK_INDEX) options = parser.parse_args() msg = audiotools.Messenger(options.verbosity == "quiet") if options.interactive and (not interactive_available): msg.error(_.ERR_URWID_REQUIRED) msg.output(_.ERR_GET_URWID1) msg.output(_.ERR_GET_URWID2) sys.exit(1) try: cddareader = CDDAReader(options.cdrom) except IOError as err: msg.error(_.ERR_INVALID_CDDA) sys.exit(-1) if len(options.tracks) == 0: track_numbers = sorted(cddareader.track_offsets.keys()) else: available_numbers = frozenset(cddareader.track_offsets.keys()) track_numbers = [t for t in options.tracks if t in available_numbers] if options.shuffle: import random random.shuffle(track_numbers) if options.interactive: # a track_number -> MetaData dictionary # where track_number typically starts from 1 metadata = {m.track_number: m for m in audiotools.cddareader_metadata_lookup( cddareader=cddareader, musicbrainz_server=options.musicbrainz_server, musicbrainz_port=options.musicbrainz_port, use_musicbrainz=options.use_musicbrainz)[0]} cdplay = CDplayGUI( cddareader=cddareader, track_numbers=track_numbers, metadata=metadata, audio_output=audiotools.player.open_output(options.output)) if len(cddareader.track_offsets) > 0: cdplay.select_first_track() loop = urwid.MainLoop(cdplay, audiotools.ui.style(), screen=audiotools.ui.Screen(), unhandled_input=cdplay.handle_text, pop_ups=True) loop.set_alarm_at(tm=time.time() + 1, callback=audiotools.ui.timer, user_data=cdplay) try: loop.run() msg.ansi_clearscreen() except (termios.error, IOError): msg.error(_.ERR_TERMIOS_ERROR) msg.info(_.ERR_TERMIOS_SUGGESTION) msg.info(audiotools.ui.xargs_suggestion(sys.argv)) sys.exit(1) else: cdplay = CDplayTTY( cddareader=cddareader, track_list=track_numbers, audio_output=audiotools.player.open_output(options.output)) sys.exit(cdplay.run(msg, sys.stdin)) ================================================ FILE: coverbrowse ================================================ #!/usr/bin/python # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA import audiotools try: import tkinter as tk except ImportError: #FIXME - indicate error if Tkinter cannot be loaded import Tkinter as tk try: import tkinter.ttk as ttk except ImportError: #FIXME - indicate error if ttk cannot be loaded import ttk #FIXME - indicate error if Image cannot be loaded from PIL import Image #FIXME - indicate error if ImageTk cannot be loaded from PIL import ImageTk def path_parts(path): """given a path to a directory, returns a list of its component directory parts""" from os.path import isdir from os.path import abspath from os.path import split path = abspath(path) assert(isdir(path)) parts = [] path, tail = split(path) while len(tail) > 0: parts.append(tail) path, tail = split(path) parts.reverse() return parts class FileSelector(ttk.Frame): def __init__(self, parent, initial_directory, file_selected): """parent is the selector's parent widget initial_directory is a path to the starting point file_selected(path) is a function to be called when a file is selected where 'path' may be None if a file is unselected""" ttk.Frame.__init__(self, parent) self.__file_selected__ = file_selected # an item_id: path mapping to audio files self.__audio_files__ = {} # an item_id: path mapping to unopened directories self.__unopened_dirs__ = {} self.treeview = ttk.Treeview(self) self.treeview.heading("#0", text="path") self.treeview.pack(fill=tk.BOTH, expand=1, side=tk.LEFT) self.treeview.bind(sequence="<<TreeviewSelect>>", func=self.path_selected) self.treeview.bind(sequence="<<TreeviewOpen>>", func=self.dir_opened) self.treeview.bind(sequence="<<TreeviewClose>>", func=self.dir_closed) root = self.treeview.insert("", "end", text="/") self.__unopened_dirs__[root] = "/" dummy = self.treeview.insert(root, "end", text="") # auto-open selected directory self.open_directory(root) self.treeview.item(root, open=True) node = root for path_part in path_parts(initial_directory): sub_node = [c for c in self.treeview.get_children(node) if self.treeview.item(c)["text"] == path_part][0] self.open_directory(sub_node) self.treeview.item(sub_node, open=True) node = sub_node else: #FIXME - scroll view to selected item self.treeview.selection_set(node) self.treeview.focus_set() self.treeview.focus(node) self.treeview.see(node) self.scrollbar = ttk.Scrollbar(self, command=self.treeview.yview) self.scrollbar.pack(fill=tk.Y, expand=0, side=tk.RIGHT) self.treeview.configure(yscroll=self.scrollbar.set) def path_selected(self, event): """changes which path is selected and calls file_selected() if necessary""" path = self.__audio_files__.get(event.widget.focus(), None) self.__file_selected__(path) def dir_opened(self, event): """called when a directory is opened""" self.open_directory(event.widget.focus()) def dir_closed(self, event): """called when a directory is closed""" # do nothing pass def open_directory(self, node): """given a node, reads the directory's sub-entries adds them to __audio_files__ and __unopened_dirs__ as needed and removes directory itself from set of __unopened_dirs__""" from os import listdir from os.path import join from os.path import isdir from os.path import isfile directory = self.__unopened_dirs__.get(node, None) if directory is not None: # clean out dummy children self.treeview.delete(*self.treeview.get_children(node)) # add new children try: files = [f for f in listdir(directory) if not f.startswith(".")] files.sort() except OSError: files = [] for file in files: file_path = join(directory, file) if isdir(file_path): new_node = self.treeview.insert(node, "end", text=file) dummy = self.treeview.insert(new_node, "end", text="") self.__unopened_dirs__[new_node] = file_path elif isfile(file_path): try: with open(file_path, "rb") as r: audio_type = audiotools.file_type(r) if audio_type is not None: new_node = self.treeview.insert( node, "end", text=file) self.__audio_files__[new_node] = file_path except IOError: pass # remove from set of unopened dirs del(self.__unopened_dirs__[node]) class ImageSelector(ttk.Frame): def __init__(self, parent, height=3, image_selected=lambda image: None): """height is the maximum number of images to display at once image_selected(image) is a callback to call when an image is selected, where 'image' may be None if an image is un-selected""" ttk.Frame.__init__(self, parent) self.file_images = tk.Listbox(self, height=height) self.file_images.pack(fill=tk.BOTH, expand=1, side=tk.LEFT) self.file_images.bind(sequence="<ButtonRelease-1>", func=self.image_selected) self.file_images.bind(sequence="<KeyRelease>", func=self.image_selected) self.scrollbar = ttk.Scrollbar(self, command=self.file_images.yview) self.scrollbar.pack(fill=tk.Y, expand=0, side=tk.RIGHT) self.file_images.configure(yscroll=self.scrollbar.set) self.__image_selected__ = image_selected def set_images(self, images): """given a list of audiotools.Image objects, sets our contents to those images, selects the initial image and calls the image_selected callback""" if len(images) > 0: self.file_images.selection_clear(0, tk.END) self.file_images.delete(0, tk.END) self.image_objects = [] for image in images: self.file_images.insert(tk.END, image.type_string()) self.image_objects.append(image) else: self.file_images.index(0) self.file_images.selection_set(0) self.__image_selected__(self.image_objects[0]) else: self.clear_images() def clear_images(self): """clears our image contents and calls the image_selected callback""" self.file_images.selection_clear(0, tk.END) self.file_images.delete(0, tk.END) self.image_objects = [] self.__image_selected__(None) def image_selected(self, event): """called when an image is selected by the user""" try: selected = self.file_images.curselection()[0] self.__image_selected__(self.image_objects[selected]) except IndexError: return None def image_scale(canvas_width, canvas_height, image_width, image_height): """returns (width, height) tuple of image scaled to fit canvas while maintaing the image's aspect ratio as closely as possible""" from fractions import Fraction canvas_ratio = Fraction(canvas_width, canvas_height) image_ratio = Fraction(image_width, image_height) if image_ratio > canvas_ratio: # image wider than canvas when scaled # so match canvas width horizontally and shrink height to match return (canvas_width, int(canvas_width / image_ratio)) else: # image taller than canvas when scaled # so match canvas height verticall and shrink width to match return (int(canvas_height * image_ratio), canvas_height) class ImageCanvas(tk.Canvas): def __init__(self, parent): tk.Canvas.__init__(self, parent) self.width = 100 self.height = 100 self.bind(sequence="<Configure>", func=self.resized) # references to the set image self.image_size = 0 self.image_digest = b"\00" * 16 self.pil_image = None self.photo_image = None self.photo_id = None def set_image(self, image): """sets viewed image from audiotools.Image object""" # only update displayed image if the new one # differs from any existing image if len(image.data) != self.image_size: from hashlib import md5 image_digest = md5(image.data) if image_digest != self.image_digest: from io import BytesIO self.image_size = len(image.data) self.image_digest = image_digest self.pil_image = Image.open(BytesIO(image.data)) self.populate_canvas(self.pil_image) def clear_image(self): """clears viewed image""" if self.photo_image is not None: self.delete(self.photo_image) self.image_size = 0 self.image_digest = b"\00" * 16 self.pil_image = None self.photo_image = None self.photo_id = None def resized(self, event): """called when canvas is resized""" self.width = event.width self.height = event.height if self.pil_image is not None: self.populate_canvas(self.pil_image) def populate_canvas(self, pil_image): """places PIL.Image object in center of canvas, resized if necessary, and caches placed image for faster redraws""" # clean out old image if self.photo_image is not None: self.delete(self.photo_image) self.photo_image = None self.photo_id = None # resize PIL to fit current canvas size if ((self.width > pil_image.size[0]) and (self.height > pil_image.size[1])): resized_image = pil_image else: (resized_width, resized_height) = image_scale(self.width, self.height, pil_image.size[0], pil_image.size[1]) resized_image = pil_image.resize((resized_width, resized_height), Image.ANTIALIAS) # generate PhotoImage from PIL image self.photo_image = ImageTk.PhotoImage(resized_image) # populate canvas with PhotoImage self.photo_id = self.create_image( (self.width - resized_image.size[0]) // 2, (self.height - resized_image.size[1]) // 2, image=self.photo_image, anchor=tk.NW) class ImageMetadata(ttk.Frame): def __init__(self, parent): ttk.Frame.__init__(self, parent) label1 = ttk.Label(self, text="width : ", anchor=tk.E) label1.grid(row=0, column=0, sticky=tk.E) self.width = ttk.Label(self, text="") self.width.grid(row=0, column=1, sticky=tk.W) label2 = ttk.Label(self, text="height : ", anchor=tk.E) label2.grid(row=1, column=0, sticky=tk.E) self.height = ttk.Label(self, text="") self.height.grid(row=1, column=1, sticky=tk.W) label3 = ttk.Label(self, text="size : ", anchor=tk.E) label3.grid(row=2, column=0, sticky=tk.E) self.size = ttk.Label(self, text="") self.size.grid(row=2, column=1, sticky=tk.W) def set_image(self, image): """sets metadata fields from contents of Image object""" self.width.config(text="{:d}".format(image.width)) self.height.config(text="{:d}".format(image.height)) self.size.config(text="{:,d} bytes".format(len(image.data))) def clear_image(self): """clears metadata fields""" self.width.config(text="") self.height.config(text="") self.size.config(text="") class Coverbrowse(object): def __init__(self, master, initial_directory): self.image_objects = [] self.master = master # the main window self.frame = ttk.Frame(master, width=800, height=600) self.frame.pack(fill=tk.BOTH, expand=1, side=tk.LEFT) # a menu bar for the main window self.menubar = tk.Menu(master) file_menu = tk.Menu(self.menubar, tearoff=False) file_menu.add_command(label="Quit", command=self.quit) self.menubar.add_cascade(label="File", menu=file_menu) # left and right halves for the main window panedwindow = ttk.PanedWindow(self.frame, orient=tk.HORIZONTAL) panedwindow.pack(fill=tk.BOTH, expand=1) # a container for the directory tree and image selector files = ttk.Frame(panedwindow) files.pack(fill=tk.BOTH, expand=1) # the directory tree and its scrollbar self.dirtree = FileSelector(files, initial_directory, self.file_selected) self.dirtree.pack(fill=tk.BOTH, expand=1, side=tk.TOP) # the image selector self.file_images = ImageSelector(files, height=3, image_selected=self.image_selected) self.file_images.pack(fill=tk.BOTH, expand=0, side=tk.TOP) # the image metadata self.image_metadata = ImageMetadata(files) self.image_metadata.pack( fill=tk.X, expand=0, side=tk.TOP, padx=5, pady=5) # the image viewer self.canvas = ImageCanvas(panedwindow) self.canvas.pack(fill=tk.BOTH, expand=1) panedwindow.add(files) panedwindow.add(self.canvas) master.title("coverbrowse") master.geometry("{:d}x{:d}".format(800, 600)) master.config(menu=self.menubar) def quit(self, *args): """exits cover browser""" self.master.quit() def file_selected(self, path): """changes which file is selected if path is None, removes selected audio file and its images""" # if audio file selected, populate image selector with images if path is not None: try: metadata = audiotools.open(path).get_metadata() if metadata is not None: self.file_images.set_images(metadata.images()) else: self.file_images.clear_images() except (IOError, ValueError, audiotools.InvalidFile): self.file_images.clear_images() else: self.file_images.clear_images() def image_selected(self, image): """given an audiotools.Image object or None, populates canvas and image metadata accordingly""" if image is not None: self.canvas.set_image(image) self.image_metadata.set_image(image) else: self.canvas.clear_image() self.image_metadata.clear_image() if (__name__ == "__main__"): import argparse import audiotools.text as _ parser = argparse.ArgumentParser(description=_.DESCRIPTION_COVERBROWSE) parser.add_argument("--version", action="version", version=audiotools.VERSION_STR) parser.add_argument("-d", "--dir", dest="dir", default=".", help=_.OPT_INITIAL_DIR) options = parser.parse_args() root = tk.Tk() coverbrowse = Coverbrowse(master=root, initial_directory=options.dir) root.mainloop() ================================================ FILE: coverdump ================================================ #!/usr/bin/python # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA import sys import os import os.path import audiotools import audiotools.text as _ FILENAME_TYPES = ("front_cover", "back_cover", "leaflet", "media", "other") if (__name__ == '__main__'): import argparse parser = argparse.ArgumentParser(description=_.DESCRIPTION_COVERDUMP) parser.add_argument("--version", action="version", version=audiotools.VERSION_STR) parser.add_argument("-V", "--verbose", dest="verbosity", choices=audiotools.VERBOSITY_LEVELS, default=audiotools.DEFAULT_VERBOSITY, help=_.OPT_VERBOSE) parser.add_argument("-d", "--dir", dest="dir", default=".", help=_.OPT_DIR_IMAGES) parser.add_argument("-p", "--prefix", action="store", dest="prefix", default="", help=_.OPT_PREFIX) parser.add_argument("filenames", metavar="FILENAME", nargs="+", help=_.OPT_INPUT_FILENAME) options = parser.parse_args() msg = audiotools.Messenger(options.verbosity == "quiet") audiofiles = audiotools.open_files(options.filenames, sorted=False, messenger=msg) if len(audiofiles) != 1: msg.error(_.ERR_1_FILE_REQUIRED) sys.exit(1) else: audiofile = audiofiles[0] input_filename = audiotools.Filename(audiofile.filename) metadata = audiofile.get_metadata() if metadata is not None: # divide images by type (front cover, leaflet page, etc.) image_types = {} for image in metadata.images(): image_types.setdefault(image.type, []).append(image) # build a set of (Image, Filename) tuples to be extracted output_images = [] for type, images in image_types.items(): if len(images) != 1: FILE_TEMPLATE = \ "{prefix}{filename}{filenum:02d}.{suffix}" else: FILE_TEMPLATE = \ "{prefix}{filename}.{suffix}" for i, image in enumerate(images): output_images.append( (image, audiotools.Filename( os.path.join( options.dir, FILE_TEMPLATE.format( prefix=options.prefix, filename=FILENAME_TYPES[image.type], filenum=i + 1, suffix=image.suffix()))))) # ensure our input file isn't the same # as any of the proposed files to extract # (this sounds crazy, # but there's no technical reason one's audio file # can't be named "front_cover.jpg" # even if it's not a JPEG # so we have to be sure it's not going to be overwritten) if (input_filename in [output_filename for (image, output_filename) in output_images]): msg.error(_.ERR_OUTPUT_IS_INPUT.format(input_filename)) sys.exit(1) # finally, write actual image data to disk if possible for (image, output_filename) in output_images: try: audiotools.make_dirs(str(output_filename)) f = open(str(output_filename), "wb") f.write(image.data) f.close() msg.info(_.LAB_ENCODE.format(source=input_filename, destination=output_filename)) except IOError as e: msg.error(_.ERR_ENCODING_ERROR.format(output_filename)) sys.exit(1) except OSError as e: msg.os_error(e) sys.exit(1) ================================================ FILE: covertag ================================================ #!/usr/bin/python # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA import sys import os.path import audiotools import audiotools.ui import audiotools.text as _ IMAGE_TYPE_ORDER = [0, 2, 1, 3, 4] # tries to return a populated Image object of the appropriate type # raises InvalidImage if something goes wrong during opening or parsing def get_image(filename, type): try: return audiotools.Image.new(open(filename, 'rb').read(), u'', type) except IOError: raise audiotools.InvalidImage( _.ERR_OPEN_IOERROR.format(audiotools.Filename(filename))) if (__name__ == '__main__'): import argparse parser = argparse.ArgumentParser(description=_.DESCRIPTION_COVERTAG) parser.add_argument("--version", action="version", version=audiotools.VERSION_STR) parser.add_argument("-r", "--replace", action="store_true", default=False, dest="replace", help=_.OPT_TRACKTAG_REPLACE) # FIXME - add -I/--interactive mode # which should use a proper GUI, if available, # so one can see added images directly parser.add_argument("-V", "--verbose", dest="verbosity", choices=audiotools.VERBOSITY_LEVELS, default=audiotools.DEFAULT_VERBOSITY, help=_.OPT_VERBOSE) img_group = parser.add_argument_group(_.OPT_CAT_IMAGE) for (option, destination, helptext) in [ ("--front-cover", "front_cover", _.OPT_TRACKTAG_FRONT_COVER), ("--back-cover", "back_cover", _.OPT_TRACKTAG_BACK_COVER), ("--leaflet", "leaflet", _.OPT_TRACKTAG_LEAFLET), ("--media", "media", _.OPT_TRACKTAG_MEDIA), ("--other-image", "other_image", _.OPT_TRACKTAG_OTHER_IMAGE)]: img_group.add_argument(option, action="append", dest=destination, metavar='FILENAME', help=helptext) parser.add_argument("filenames", metavar="FILENAME", nargs="+", help=_.OPT_INPUT_FILENAME) options = parser.parse_args() msg = audiotools.Messenger(options.verbosity == "quiet") # open our set of input files for tagging try: audiofiles = audiotools.open_files(options.filenames, messenger=msg, no_duplicates=True) except audiotools.DuplicateFile as err: msg.error(_.ERR_DUPLICATE_FILE.format(err.filename)) sys.exit(1) # open images for addition # to avoid reading the same images multiple times images = {} try: if options.front_cover is not None: for path in options.front_cover: images.setdefault(0, []).append(get_image(path, 0)) if options.leaflet is not None: for path in options.leaflet: images.setdefault(2, []).append(get_image(path, 2)) if options.back_cover is not None: for path in options.back_cover: images.setdefault(1, []).append(get_image(path, 1)) if options.media is not None: for path in options.media: images.setdefault(3, []).append(get_image(path, 3)) if options.other_image is not None: for path in options.other_image: images.setdefault(4, []).append(get_image(path, 4)) except audiotools.InvalidImage as err: msg.error(err) sys.exit(1) for track in audiofiles: # get metadata from each audio file metadata = track.get_metadata() # if metadata is present if metadata is not None: if metadata.supports_images(): # if --replace indicated, remove old images if options.replace: for i in metadata.images(): metadata.delete_image(i) # add images to metadata object in order for t in IMAGE_TYPE_ORDER: for i in images.get(t, []): metadata.add_image(i) # call update_metadata() to update track's metadata try: track.update_metadata(metadata) except IOError as err: msg.error( _.ERR_ENCODING_ERROR.format( audiotools.Filename(track.filename))) sys.exit(1) else: # metadata doesn't support images, so do nothing pass else: # if no metadata is present, construct new MetaData object metadata = audiotools.MetaData() # add images to metadata object in order for t in IMAGE_TYPE_ORDER: for i in images.get(t, []): metadata.add_image(i) # call set_metadata() to update track's metadata try: track.set_metadata(metadata) except IOError as err: msg.error( _.ERR_ENCODING_ERROR.format( audiotools.Filename(track.filename))) sys.exit(1) ================================================ FILE: docs/COPYING ================================================ Creative Commons Creative Commons Legal Code Attribution-ShareAlike 3.0 Unported CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE LEGAL SERVICES. DISTRIBUTION OF THIS LICENSE DOES NOT CREATE AN ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES REGARDING THE INFORMATION PROVIDED, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM ITS USE. License THE WORK (AS DEFINED BELOW) IS PROVIDED UNDER THE TERMS OF THIS CREATIVE COMMONS PUBLIC LICENSE ("CCPL" OR "LICENSE"). THE WORK IS PROTECTED BY COPYRIGHT AND/OR OTHER APPLICABLE LAW. ANY USE OF THE WORK OTHER THAN AS AUTHORIZED UNDER THIS LICENSE OR COPYRIGHT LAW IS PROHIBITED. BY EXERCISING ANY RIGHTS TO THE WORK PROVIDED HERE, YOU ACCEPT AND AGREE TO BE BOUND BY THE TERMS OF THIS LICENSE. TO THE EXTENT THIS LICENSE MAY BE CONSIDERED TO BE A CONTRACT, THE LICENSOR GRANTS YOU THE RIGHTS CONTAINED HERE IN CONSIDERATION OF YOUR ACCEPTANCE OF SUCH TERMS AND CONDITIONS. 1. Definitions a. "Adaptation" means a work based upon the Work, or upon the Work and other pre-existing works, such as a translation, adaptation, derivative work, arrangement of music or other alterations of a literary or artistic work, or phonogram or performance and includes cinematographic adaptations or any other form in which the Work may be recast, transformed, or adapted including in any form recognizably derived from the original, except that a work that constitutes a Collection will not be considered an Adaptation for the purpose of this License. For the avoidance of doubt, where the Work is a musical work, performance or phonogram, the synchronization of the Work in timed-relation with a moving image ("synching") will be considered an Adaptation for the purpose of this License. b. "Collection" means a collection of literary or artistic works, such as encyclopedias and anthologies, or performances, phonograms or broadcasts, or other works or subject matter other than works listed in Section 1(f) below, which, by reason of the selection and arrangement of their contents, constitute intellectual creations, in which the Work is included in its entirety in unmodified form along with one or more other contributions, each constituting separate and independent works in themselves, which together are assembled into a collective whole. A work that constitutes a Collection will not be considered an Adaptation (as defined below) for the purposes of this License. c. "Creative Commons Compatible License" means a license that is listed at http://creativecommons.org/compatiblelicenses that has been approved by Creative Commons as being essentially equivalent to this License, including, at a minimum, because that license: (i) contains terms that have the same purpose, meaning and effect as the License Elements of this License; and, (ii) explicitly permits the relicensing of adaptations of works made available under that license under this License or a Creative Commons jurisdiction license with the same License Elements as this License. d. "Distribute" means to make available to the public the original and copies of the Work or Adaptation, as appropriate, through sale or other transfer of ownership. e. "License Elements" means the following high-level license attributes as selected by Licensor and indicated in the title of this License: Attribution, ShareAlike. f. "Licensor" means the individual, individuals, entity or entities that offer(s) the Work under the terms of this License. g. "Original Author" means, in the case of a literary or artistic work, the individual, individuals, entity or entities who created the Work or if no individual or entity can be identified, the publisher; and in addition (i) in the case of a performance the actors, singers, musicians, dancers, and other persons who act, sing, deliver, declaim, play in, interpret or otherwise perform literary or artistic works or expressions of folklore; (ii) in the case of a phonogram the producer being the person or legal entity who first fixes the sounds of a performance or other sounds; and, (iii) in the case of broadcasts, the organization that transmits the broadcast. h. "Work" means the literary and/or artistic work offered under the terms of this License including without limitation any production in the literary, scientific and artistic domain, whatever may be the mode or form of its expression including digital form, such as a book, pamphlet and other writing; a lecture, address, sermon or other work of the same nature; a dramatic or dramatico-musical work; a choreographic work or entertainment in dumb show; a musical composition with or without words; a cinematographic work to which are assimilated works expressed by a process analogous to cinematography; a work of drawing, painting, architecture, sculpture, engraving or lithography; a photographic work to which are assimilated works expressed by a process analogous to photography; a work of applied art; an illustration, map, plan, sketch or three-dimensional work relative to geography, topography, architecture or science; a performance; a broadcast; a phonogram; a compilation of data to the extent it is protected as a copyrightable work; or a work performed by a variety or circus performer to the extent it is not otherwise considered a literary or artistic work. i. "You" means an individual or entity exercising rights under this License who has not previously violated the terms of this License with respect to the Work, or who has received express permission from the Licensor to exercise rights under this License despite a previous violation. j. "Publicly Perform" means to perform public recitations of the Work and to communicate to the public those public recitations, by any means or process, including by wire or wireless means or public digital performances; to make available to the public Works in such a way that members of the public may access these Works from a place and at a place individually chosen by them; to perform the Work to the public by any means or process and the communication to the public of the performances of the Work, including by public digital performance; to broadcast and rebroadcast the Work by any means including signs, sounds or images. k. "Reproduce" means to make copies of the Work by any means including without limitation by sound or visual recordings and the right of fixation and reproducing fixations of the Work, including storage of a protected performance or phonogram in digital form or other electronic medium. 2. Fair Dealing Rights. Nothing in this License is intended to reduce, limit, or restrict any uses free from copyright or rights arising from limitations or exceptions that are provided for in connection with the copyright protection under copyright law or other applicable laws. 3. License Grant. Subject to the terms and conditions of this License, Licensor hereby grants You a worldwide, royalty-free, non-exclusive, perpetual (for the duration of the applicable copyright) license to exercise the rights in the Work as stated below: a. to Reproduce the Work, to incorporate the Work into one or more Collections, and to Reproduce the Work as incorporated in the Collections; b. to create and Reproduce Adaptations provided that any such Adaptation, including any translation in any medium, takes reasonable steps to clearly label, demarcate or otherwise identify that changes were made to the original Work. For example, a translation could be marked "The original work was translated from English to Spanish," or a modification could indicate "The original work has been modified."; c. to Distribute and Publicly Perform the Work including as incorporated in Collections; and, d. to Distribute and Publicly Perform Adaptations. e. For the avoidance of doubt: i. Non-waivable Compulsory License Schemes. In those jurisdictions in which the right to collect royalties through any statutory or compulsory licensing scheme cannot be waived, the Licensor reserves the exclusive right to collect such royalties for any exercise by You of the rights granted under this License; ii. Waivable Compulsory License Schemes. In those jurisdictions in which the right to collect royalties through any statutory or compulsory licensing scheme can be waived, the Licensor waives the exclusive right to collect such royalties for any exercise by You of the rights granted under this License; and, iii. Voluntary License Schemes. The Licensor waives the right to collect royalties, whether individually or, in the event that the Licensor is a member of a collecting society that administers voluntary licensing schemes, via that society, from any exercise by You of the rights granted under this License. The above rights may be exercised in all media and formats whether now known or hereafter devised. The above rights include the right to make such modifications as are technically necessary to exercise the rights in other media and formats. Subject to Section 8(f), all rights not expressly granted by Licensor are hereby reserved. 4. Restrictions. The license granted in Section 3 above is expressly made subject to and limited by the following restrictions: a. You may Distribute or Publicly Perform the Work only under the terms of this License. You must include a copy of, or the Uniform Resource Identifier (URI) for, this License with every copy of the Work You Distribute or Publicly Perform. You may not offer or impose any terms on the Work that restrict the terms of this License or the ability of the recipient of the Work to exercise the rights granted to that recipient under the terms of the License. You may not sublicense the Work. You must keep intact all notices that refer to this License and to the disclaimer of warranties with every copy of the Work You Distribute or Publicly Perform. When You Distribute or Publicly Perform the Work, You may not impose any effective technological measures on the Work that restrict the ability of a recipient of the Work from You to exercise the rights granted to that recipient under the terms of the License. This Section 4(a) applies to the Work as incorporated in a Collection, but this does not require the Collection apart from the Work itself to be made subject to the terms of this License. If You create a Collection, upon notice from any Licensor You must, to the extent practicable, remove from the Collection any credit as required by Section 4(c), as requested. If You create an Adaptation, upon notice from any Licensor You must, to the extent practicable, remove from the Adaptation any credit as required by Section 4(c), as requested. b. You may Distribute or Publicly Perform an Adaptation only under the terms of: (i) this License; (ii) a later version of this License with the same License Elements as this License; (iii) a Creative Commons jurisdiction license (either this or a later license version) that contains the same License Elements as this License (e.g., Attribution-ShareAlike 3.0 US)); (iv) a Creative Commons Compatible License. If you license the Adaptation under one of the licenses mentioned in (iv), you must comply with the terms of that license. If you license the Adaptation under the terms of any of the licenses mentioned in (i), (ii) or (iii) (the "Applicable License"), you must comply with the terms of the Applicable License generally and the following provisions: (I) You must include a copy of, or the URI for, the Applicable License with every copy of each Adaptation You Distribute or Publicly Perform; (II) You may not offer or impose any terms on the Adaptation that restrict the terms of the Applicable License or the ability of the recipient of the Adaptation to exercise the rights granted to that recipient under the terms of the Applicable License; (III) You must keep intact all notices that refer to the Applicable License and to the disclaimer of warranties with every copy of the Work as included in the Adaptation You Distribute or Publicly Perform; (IV) when You Distribute or Publicly Perform the Adaptation, You may not impose any effective technological measures on the Adaptation that restrict the ability of a recipient of the Adaptation from You to exercise the rights granted to that recipient under the terms of the Applicable License. This Section 4(b) applies to the Adaptation as incorporated in a Collection, but this does not require the Collection apart from the Adaptation itself to be made subject to the terms of the Applicable License. c. If You Distribute, or Publicly Perform the Work or any Adaptations or Collections, You must, unless a request has been made pursuant to Section 4(a), keep intact all copyright notices for the Work and provide, reasonable to the medium or means You are utilizing: (i) the name of the Original Author (or pseudonym, if applicable) if supplied, and/or if the Original Author and/or Licensor designate another party or parties (e.g., a sponsor institute, publishing entity, journal) for attribution ("Attribution Parties") in Licensor's copyright notice, terms of service or by other reasonable means, the name of such party or parties; (ii) the title of the Work if supplied; (iii) to the extent reasonably practicable, the URI, if any, that Licensor specifies to be associated with the Work, unless such URI does not refer to the copyright notice or licensing information for the Work; and (iv) , consistent with Ssection 3(b), in the case of an Adaptation, a credit identifying the use of the Work in the Adaptation (e.g., "French translation of the Work by Original Author," or "Screenplay based on original Work by Original Author"). The credit required by this Section 4(c) may be implemented in any reasonable manner; provided, however, that in the case of a Adaptation or Collection, at a minimum such credit will appear, if a credit for all contributing authors of the Adaptation or Collection appears, then as part of these credits and in a manner at least as prominent as the credits for the other contributing authors. For the avoidance of doubt, You may only use the credit required by this Section for the purpose of attribution in the manner set out above and, by exercising Your rights under this License, You may not implicitly or explicitly assert or imply any connection with, sponsorship or endorsement by the Original Author, Licensor and/or Attribution Parties, as appropriate, of You or Your use of the Work, without the separate, express prior written permission of the Original Author, Licensor and/or Attribution Parties. d. Except as otherwise agreed in writing by the Licensor or as may be otherwise permitted by applicable law, if You Reproduce, Distribute or Publicly Perform the Work either by itself or as part of any Adaptations or Collections, You must not distort, mutilate, modify or take other derogatory action in relation to the Work which would be prejudicial to the Original Author's honor or reputation. Licensor agrees that in those jurisdictions (e.g. Japan), in which any exercise of the right granted in Section 3(b) of this License (the right to make Adaptations) would be deemed to be a distortion, mutilation, modification or other derogatory action prejudicial to the Original Author's honor and reputation, the Licensor will waive or not assert, as appropriate, this Section, to the fullest extent permitted by the applicable national law, to enable You to reasonably exercise Your right under Section 3(b) of this License (right to make Adaptations) but not otherwise. 5. Representations, Warranties and Disclaimer UNLESS OTHERWISE MUTUALLY AGREED TO BY THE PARTIES IN WRITING, LICENSOR OFFERS THE WORK AS-IS AND MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE WORK, EXPRESS, IMPLIED, STATUTORY OR OTHERWISE, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF TITLE, MERCHANTIBILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, ACCURACY, OR THE PRESENCE OF ABSENCE OF ERRORS, WHETHER OR NOT DISCOVERABLE. SOME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES, SO SUCH EXCLUSION MAY NOT APPLY TO YOU. 6. Limitation on Liability. EXCEPT TO THE EXTENT REQUIRED BY APPLICABLE LAW, IN NO EVENT WILL LICENSOR BE LIABLE TO YOU ON ANY LEGAL THEORY FOR ANY SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR EXEMPLARY DAMAGES ARISING OUT OF THIS LICENSE OR THE USE OF THE WORK, EVEN IF LICENSOR HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. 7. Termination a. This License and the rights granted hereunder will terminate automatically upon any breach by You of the terms of this License. Individuals or entities who have received Adaptations or Collections from You under this License, however, will not have their licenses terminated provided such individuals or entities remain in full compliance with those licenses. Sections 1, 2, 5, 6, 7, and 8 will survive any termination of this License. b. Subject to the above terms and conditions, the license granted here is perpetual (for the duration of the applicable copyright in the Work). Notwithstanding the above, Licensor reserves the right to release the Work under different license terms or to stop distributing the Work at any time; provided, however that any such election will not serve to withdraw this License (or any other license that has been, or is required to be, granted under the terms of this License), and this License will continue in full force and effect unless terminated as stated above. 8. Miscellaneous a. Each time You Distribute or Publicly Perform the Work or a Collection, the Licensor offers to the recipient a license to the Work on the same terms and conditions as the license granted to You under this License. b. Each time You Distribute or Publicly Perform an Adaptation, Licensor offers to the recipient a license to the original Work on the same terms and conditions as the license granted to You under this License. c. If any provision of this License is invalid or unenforceable under applicable law, it shall not affect the validity or enforceability of the remainder of the terms of this License, and without further action by the parties to this agreement, such provision shall be reformed to the minimum extent necessary to make such provision valid and enforceable. d. No term or provision of this License shall be deemed waived and no breach consented to unless such waiver or consent shall be in writing and signed by the party to be charged with such waiver or consent. e. This License constitutes the entire agreement between the parties with respect to the Work licensed here. There are no understandings, agreements or representations with respect to the Work not specified here. Licensor shall not be bound by any additional provisions that may appear in any communication from You. This License may not be modified without the mutual written agreement of the Licensor and You. f. The rights granted under, and the subject matter referenced, in this License were drafted utilizing the terminology of the Berne Convention for the Protection of Literary and Artistic Works (as amended on September 28, 1979), the Rome Convention of 1961, the WIPO Copyright Treaty of 1996, the WIPO Performances and Phonograms Treaty of 1996 and the Universal Copyright Convention (as revised on July 24, 1971). These rights and subject matter take effect in the relevant jurisdiction in which the License terms are sought to be enforced according to the corresponding provisions of the implementation of those treaty provisions in the applicable national law. If the standard suite of rights granted under applicable copyright law includes additional rights not granted under this License, such additional rights are deemed to be included in the License; this License is not intended to restrict the license of any rights under applicable law. Creative Commons Notice Creative Commons is not a party to this License, and makes no warranty whatsoever in connection with the Work. Creative Commons will not be liable to You or any party on any legal theory for any damages whatsoever, including without limitation any general, special, incidental or consequential damages arising in connection to this license. Notwithstanding the foregoing two (2) sentences, if Creative Commons has expressly identified itself as the Licensor hereunder, it shall have all rights and obligations of Licensor. Except for the limited purpose of indicating to the public that the Work is licensed under the CCPL, Creative Commons does not authorize the use by either party of the trademark "Creative Commons" or any related trademark or logo of Creative Commons without the prior written consent of Creative Commons. Any permitted use will be in compliance with Creative Commons' then-current trademark usage guidelines, as may be published on its website or otherwise made available upon request from time to time. For the avoidance of doubt, this trademark restriction does not form part of the License. Creative Commons may be contacted at http://creativecommons.org/. References Visible links 1. http://creativecommons.org/ 2. http://creativecommons.org/ 3. http://creativecommons.org/licenses/by-sa/3.0/ ================================================ FILE: docs/Makefile ================================================ PYTHON = python MAN_PATH = /usr/share/man MAN_PAGES = \ audiotools-config.1 \ audiotools.cfg.5 \ cdda2track.1 \ cddainfo.1 \ cddaplay.1 \ coverbrowse.1 \ coverdump.1 \ covertag.1 \ dvdainfo.1 \ dvda2track.1 \ track2cdda.1 \ track2track.1 \ trackcat.1 \ trackcmp.1 \ trackinfo.1 \ tracklength.1 \ tracklint.1 \ trackplay.1 \ trackrename.1 \ tracksplit.1 \ tracktag.1 \ trackverify.1 MAN_SOURCES = \ audiotools-config.xml \ audiotools.cfg.xml \ cdda2track.xml \ cddainfo.xml \ cddaplay.xml \ coverbrowse.xml \ coverdump.xml \ covertag.xml \ dvdainfo.xml \ dvda2track.xml \ track2cdda.xml \ track2track.xml \ trackcat.xml \ trackcmp.xml \ trackinfo.xml \ tracklength.xml \ tracklint.xml \ trackplay.xml \ trackrename.xml \ tracksplit.xml \ tracktag.xml \ trackverify.xml .SUFFIXES: .xml .1 .5 .xml.1: $(PYTHON) manpagexml.py -i $< $(MAN_SOURCES) > $@ .xml.5: $(PYTHON) manpagexml.py -i $< $(MAN_SOURCES) > $@ all: $(MAN_PAGES) clean: .FORCE rm -fv $(MAN_PAGES) cd programming && make clean .FORCE: install: $(MAN_PAGES) for m in $(MAN_PAGES); do install -m 644 $$m $(MAN_PATH)/man1/$$m; done install -m 644 audiotools.cfg.5 $(MAN_PATH)/man5/audiotools.cfg.5 ================================================ FILE: docs/audiotools-config.xml ================================================ <?xml version="1.0" encoding="utf-8"?> <!-- Copyright (C) 2007-2016 Brian Langenberger This work is licensed under the Creative Commons Attribution-Share Alike 3.0 United States License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/3.0/us/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA. --> <manpage> <utility>audiotools-config</utility> <author>Brian Langenberger</author> <section>1</section> <name>manage Python Audio Tools configuration</name> <title>Audio Tools Configuration Manager [OPTIONS] audiotools-config is the Python Audio Tools configuration manager. When called without arguments, it displays the system's current configuration. By specifying various configuration arguments, it modifies the user's $(HOME)/.audiotools.cfg file with new values. ================================================ FILE: docs/audiotools.cfg.xml ================================================ audiotools.cfg Brian Langenberger
5
configuration data for Python Audio Tools Audio Tools Config File The format of this file consists of one or more sections. Each section contains one or more key / value pairs.
Section Key Value
[System] default_type the default audio type to use
cdrom the default CD-ROM device to use
cdrom_read_offset read sample offset to apply
cdrom_write_offset write sample offset to apply
fs_encoding text encoding for filenames
io_encoding text encoding for terminal output
maximum_jobs default for the -j option
[Defaults] verbosity "normal", "debug" or "quiet"
[Filenames] format default for the --format option
[Quality] flac default quality for FLAC encoding
mp3 default quality for MP3 encoding
... default quality for a given type
[Binaries] oggenc binary to use for Vorbis encoding
lame binary to use for MP3 encoding
... binary to use other than default
[Thumbnail] format "jpeg", "png", "gif", etc.
size maximum size of each thumbnail
[ID3] id3v2 id3v2.2, id3v2.3, id3v2.4 or none
id3v1 "id3v1.1" or "none"
pad if "true", track numbers like "01"
if "false", track numbers like "1"
[MusicBrainz] server default MusicBrainz hostname
port default MusicBrainz port
[FreeDB] server default FreeDB hostname
port default FreeDB port
================================================ FILE: docs/cdda2track.xml ================================================ cdda2track Brian Langenberger
1
extract audio files Compact Disc Extractor [OPTIONS] [track 1] [track 2] ... cdda2track extracts audio files from a compact disc and encodes them to tracks. If track numbers are given, extracts only those tracks. Otherwise, extracts the entire disc.

Extracted tracks are automatically verified against AccurateRip's online database. The confidence level is the number of other people who have the same rip, so a larger value indicates one's own rip is consistent with those of others. However, not finding one's rip in the AccurateRip database does not necessarily mean the rip is bad; the CD may be new, rare, or a different pressing than the one in the database.

Extract all of the tracks from /dev/cdrom as FLAC files at the default quality: cdda2track -t flac -c /dev/cdrom ================================================ FILE: docs/cddainfo.xml ================================================ cddainfo Brian Langenberger
1
prints information about a compact disc Compact Disc Information [OPTIONS] cddainfo takes a compact disc device and writes information about the current disc to standard output. This includes the total number of tracks, the total length of the disc, the FreeDB and MusicBrainz disc IDs, and the length/offset information of each track. ================================================ FILE: docs/cddaplay.xml ================================================ cddaplay Brian Langenberger
1
plays compact discs to speakers Play Compact Discs [OPTIONS] cddaplay takes a CD-ROM device and plays its tracks to an available audio output device such as PulseAudio or the Open Sound System.
N / n-next track
P / p-previous track
Space - pause (non-interactive mode only)
Esc / Q / q-quit
 ================================================ FILE: docs/coverbrowse.xml ================================================ coverbrowse Brian Langenberger
1
view embedded cover art interactively Cover Art Browser [OPTIONS] coverbrowse takes an optional starting directory and browses embedded cover art from files in the directory tree. ================================================ FILE: docs/coverdump.xml ================================================ coverdump Brian Langenberger
1
extracts cover images to files Cover Image Extractor [OPTIONS] <track> coverdump takes an audio track and extracts all of its embedded cover images to individual files. Extract the covert art embedded in track.flac to the images/ directory: coverdump track.flac -d images/ ================================================ FILE: docs/covertag.xml ================================================ covertag Brian Langenberger
1
update audio file image metadata Audio File Image Tagger [OPTIONS] <track 1> [track 2] ... covertag takes image files and a list of audio files and updates those files with the new artwork. Replace all images in track.mp3 with a PNG file covertag -r --front-cover=front.png track.mp3 Add several JPEG images to track.flac covertag --front-cover=front.jpg --back-cover=back.jpg --leaflet=page1.jpg --leaflet=page2.jpg --leaflet=page3.jpg track.flac Remove all cover art from track.wv covertag -r track.wv ================================================ FILE: docs/dvda2track.xml ================================================ dvda2track Brian Langenberger
1
extract audio tracks DVD-Audio Extractor [OPTIONS] [track 1] [track 2] ... dvda2track extracts audio files from a mounted DVD-Audio disc and encodes them to tracks. If track numbers are given, extracts only those tracks from the given title. Otherwise it extracts the entire title. Extract all tracks from the DVD-A on mount point "/media/cdrom" whose DVD-ROM device is "/dev/cdrom" dvda2track -c /dev/cdrom -A /media/cdrom/AUDIO_TS/ ================================================ FILE: docs/dvdainfo.xml ================================================ dvdainfo Brian Langenberger
1
prints information about a DVD-Audio disc DVD-Audio Disc Information [OPTIONS] dvdainfo takes a mounted DVD-Audio disc and writes information about the it to standard output. This information includes the length, start sector, end sector, codec, sample rate, channel count and bits-per-sample of each track in each title. ================================================ FILE: docs/manpagexml.py ================================================ #!/usr/bin/python # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2016 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA import re from sys import version_info PY3 = version_info[0] >= 3 WHITESPACE = re.compile(r'\s+') def subtag(node, name): return [child for child in node.childNodes if (hasattr(child, "nodeName") and (child.nodeName == name))][0] def subtags(node, name): return [child for child in node.childNodes if (hasattr(child, "nodeName") and (child.nodeName == name))] def text(node): try: return WHITESPACE.sub(u" ", node.childNodes[0].wholeText.strip()) except IndexError: return u"" def man_escape(s): return s.replace('-', '\\-') if PY3: def write_u(stream, unicode_string): assert(isinstance(unicode_string, str)) stream.write(unicode_string) else: def write_u(stream, unicode_string): assert(isinstance(unicode_string, unicode)) stream.write(unicode_string.encode("utf-8")) class Manpage: FIELDS = [ ("%(track_number)2.2d", "the track's number on the CD"), ("%(track_total)d", "the total number of tracks on the CD"), ("%(album_number)d", "the CD's album number"), ("%(album_total)d", "the total number of CDs in the set"), ("%(album_track_number)s", "combination of album and track number"), ("%(track_name)s", "the track's name"), ("%(album_name)s", "the album's name"), ("%(artist_name)s", "the track's artist name"), ("%(performer_name)s", "the track's performer name"), ("%(composer_name)s", "the track's composer name"), ("%(conductor_name)s", "the track's conductor name"), ("%(media)s", "the track's source media"), ("%(ISRC)s", "the track's ISRC"), ("%(catalog)s", "the track's catalog number"), ("%(copyright)s", "the track's copyright information"), ("%(publisher)s", "the track's publisher"), ("%(year)s", "the track's publication year"), ("%(date)s", "the track's original recording date"), ("%(suffix)s", "the track's suffix"), ("%(basename)s", "the track's original name, without suffix")] def __init__(self, utility=u"", section=1, name=u"", title=u"", synopsis=None, description=u"", author=u"", options=None, elements=None, examples=None, see_also=None): self.utility = utility self.section = int(section) self.name = name self.title = title self.synopsis = synopsis self.description = description self.author = author if options is not None: self.options = options else: self.options = [] if examples is not None: self.examples = examples else: self.examples = [] if elements is not None: self.elements = elements else: self.elements = [] if see_also is not None: self.see_also = see_also else: self.see_also = [] def __repr__(self): return "Manpage(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)" % \ (repr(self.utility), repr(self.section), repr(self.name), repr(self.title), repr(self.synopsis), repr(self.description), repr(self.author), repr(self.options), repr(self.elements), repr(self.examples), repr(self.see_also)) def flatten_options(self): for option_category in self.options: for option in option_category.options: yield option @classmethod def parse_file(cls, filename): return cls.parse(xml.dom.minidom.parse(filename)) @classmethod def parse(cls, xml_dom): manpage = xml_dom.getElementsByTagName(u"manpage")[0] try: synopsis = text(subtag(manpage, u"synopsis")) except IndexError: synopsis = None options = [Options.parse(options) for options in subtags(manpage, u"options")] elements = [Element.parse(element) for element in subtags(manpage, u"element")] try: examples = [Example.parse(example) for example in subtags(subtag(manpage, u"examples"), u"example")] except IndexError: examples = None return cls(utility=text(subtag(manpage, u"utility")), section=text(subtag(manpage, u"section")), name=text(subtag(manpage, u"name")), title=text(subtag(manpage, u"title")), synopsis=synopsis, description=text(subtag(manpage, u"description")), author=text(subtag(manpage, u"author")), options=options, elements=elements, examples=examples) def to_man(self, stream, page_time=None): from time import localtime from time import strftime write_u(stream, (u".TH \"%(utility)s\" %(section)d " + u"\"%(date)s\" \"\" \"%(title)s\"\n") % {"utility": self.utility.upper(), "section": self.section, "date": strftime("%B %Y", localtime(page_time)), "title": self.title}) write_u(stream, u".SH NAME\n") write_u(stream, u"%(utility)s \\- %(name)s\n" % {"utility": self.utility, "name": self.name}) if self.synopsis is not None: write_u(stream, u".SH SYNOPSIS\n") write_u(stream, u"%(utility)s %(synopsis)s\n" % {"utility": self.utility, "synopsis": self.synopsis}) write_u(stream, u".SH DESCRIPTION\n") write_u(stream, u".PP\n") write_u(stream, self.description) write_u(stream, u"\n") for option in self.options: option.to_man(stream) for element in self.elements: element.to_man(stream) if len(self.examples) > 0: if len(self.examples) > 1: write_u(stream, u".SH EXAMPLES\n") else: write_u(stream, u".SH EXAMPLE\n") for example in self.examples: example.to_man(stream) for option in self.flatten_options(): if option.long_arg == 'format': self.format_fields_man(stream) break self.see_also.sort(key=lambda x: x.utility) if len(self.see_also) > 0: write_u(stream, u".SH SEE ALSO\n") # handle the trailing comma correctly for page in self.see_also[0:-1]: write_u(stream, u".BR %(utility)s (%(section)d),\n" % {"utility": page.utility, "section": page.section}) write_u(stream, u".BR %(utility)s (%(section)d)\n" % {"utility": self.see_also[-1].utility, "section": self.see_also[-1].section}) write_u(stream, u".SH AUTHOR\n") write_u(stream, u"%(author)s\n" % {"author": self.author}) def format_fields_man(self, stream): write_u(stream, u".SH FORMAT STRING FIELDS\n") write_u(stream, u".TS\n") write_u(stream, u"tab(:);\n") write_u(stream, u"| c s |\n") write_u(stream, u"| c | c |\n") write_u(stream, u"| r | l |.\n") write_u(stream, u"_\n") write_u(stream, u"Template Fields\n") write_u(stream, u"Key:Value\n") write_u(stream, u"=\n") for (field, description) in self.FIELDS: write_u(stream, u"\\fC%(field)s\\fR:%(description)s\n" % {"field": field, "description": description}) write_u(stream, u"_\n") write_u(stream, u".TE\n") def to_html(self, stream): write_u(stream, u'
\n' % (self.utility)) # display utility name write_u(stream, u"

%s

\n" % (self.utility)) # display utility description write_u(stream, u"

%s

\n" % (self.description)) # display options for option_section in self.options: option_section.to_html(stream) # display additional sections for element in self.elements: element.to_html(stream) # display examples if len(self.examples) > 0: write_u(stream, u'
\n') if len(self.examples) > 1: write_u(stream, u"
Examples
\n") else: write_u(stream, u"
Example
\n") write_u(stream, u"
\n") for example in self.examples: example.to_html(stream) write_u(stream, u"
\n") write_u(stream, u"
\n") write_u(stream, u'
\n') class Options: def __init__(self, options, category=None): self.options = options self.category = category def __repr__(self): return "Options(%s, %s)" % \ (self.options, self.category) @classmethod def parse(cls, xml_dom): if xml_dom.hasAttribute(u"category"): category = xml_dom.getAttribute(u"category") else: category = None return cls(options=[Option.parse(child) for child in subtags(xml_dom, u"option")], category=category) def to_man(self, stream): if self.category is None: write_u(stream, u".SH OPTIONS\n") else: write_u(stream, u".SH %(category)s OPTIONS\n" % {"category": self.category.upper()}) for option in self.options: option.to_man(stream) def to_html(self, stream): write_u(stream, u"
\n") if self.category is None: write_u(stream, u"
Options
\n") else: write_u(stream, u"
%s Options
\n" % self.category.capitalize()) write_u(stream, u"
\n") write_u(stream, u"
\n") for option in self.options: option.to_html(stream) write_u(stream, u"
\n") write_u(stream, u"
\n") write_u(stream, u"
\n") class Option: def __init__(self, short_arg=None, long_arg=None, arg_name=None, description=None): self.short_arg = short_arg self.long_arg = long_arg self.arg_name = arg_name self.description = description def __repr__(self): return "Option(%s, %s, %s, %s)" % \ (repr(self.short_arg), repr(self.long_arg), repr(self.arg_name), repr(self.description)) @classmethod def parse(cls, xml_dom): if xml_dom.hasAttribute("short"): short_arg = xml_dom.getAttribute("short") if len(short_arg) > 1: raise ValueError("short arguments should be 1 character") else: short_arg = None if xml_dom.hasAttribute("long"): long_arg = xml_dom.getAttribute("long") else: long_arg = None if xml_dom.hasAttribute("arg"): arg_name = xml_dom.getAttribute("arg") else: arg_name = None if len(xml_dom.childNodes) > 0: description = WHITESPACE.sub( u" ", xml_dom.childNodes[0].wholeText.strip()) else: description = None return cls(short_arg=short_arg, long_arg=long_arg, arg_name=arg_name, description=description) def to_man(self, stream): write_u(stream, u".TP\n") if (self.short_arg is not None) and (self.long_arg is not None): if self.arg_name is not None: write_u(stream, (u"\\fB\\-%(short_arg)s\\fR, " + u"\\fB\\-\\-%(long_arg)s\\fR=" + u"\\fI%(arg_name)s\\fR\n") % {"short_arg": man_escape(self.short_arg), "long_arg": man_escape(self.long_arg), "arg_name": man_escape(self.arg_name.upper())}) else: write_u(stream, (u"\\fB\\-%(short_arg)s\\fR, " + u"\\fB\\-\\-%(long_arg)s\\fR\n") % {"short_arg": man_escape(self.short_arg), "long_arg": man_escape(self.long_arg)}) elif self.short_arg is not None: if self.arg_name is not None: write_u(stream, (u"\\fB\\-%(short_arg)s\\fR " + u"\\fI%(arg_name)s\\fR\n") % {"short_arg": man_escape(self.short_arg), "arg_name": man_escape(self.arg_name.upper())}) else: write_u(stream, u"\\fB\\-%(short_arg)s\\fR\n" % {"short_arg": man_escape(self.short_arg)}) elif self.long_arg is not None: if self.arg_name is not None: write_u(stream, (u"\\fB\\-\\-%(long_arg)s\\fR" + u"=\\fI%(arg_name)s\\fR\n") % {"long_arg": man_escape(self.long_arg), "arg_name": man_escape(self.arg_name.upper())}) else: write_u(stream, u"\\fB\\-\\-%(long_arg)s\\fR\n" % {"long_arg": man_escape(self.long_arg)}) else: raise ValueError("short arg or long arg must be present in option") if self.description is not None: write_u(stream, self.description) write_u(stream, u"\n") def to_html(self, stream): write_u(stream, u"
\n") if (self.short_arg is not None) and (self.long_arg is not None): if self.arg_name is not None: write_u(stream, (u"-%(short_arg)s, " + u"--%(long_arg)s=" + u"%(arg_name)s\n") % {"short_arg": self.short_arg, "long_arg": self.long_arg, "arg_name": man_escape(self.arg_name.upper())}) else: write_u(stream, (u"-%(short_arg)s, " + u"--%(long_arg)s\n") % {"short_arg": self.short_arg, "long_arg": self.long_arg}) elif self.short_arg is not None: if self.arg_name is not None: write_u(stream, (u"-%(short_arg)s " + u"%(arg_name)s\n") % {"short_arg": self.short_arg, "arg_name": self.arg_name.upper()}) else: write_u(stream, u"-%(short_arg)s\n" % {"short_arg": self.short_arg}) elif self.long_arg is not None: if self.arg_name is not None: write_u(stream, (u"--%(long_arg)s" + u"=%(arg_name)s\n") % {"long_arg": self.long_arg, "arg_name": self.arg_name.upper()}) else: write_u(stream, u"--%(long_arg)s\n" % {"long_arg": self.long_arg}) else: raise ValueError("short arg or long arg must be present in option") write_u(stream, u"
\n") if self.description is not None: write_u(stream, u"
%s
\n" % (self.description)) else: write_u(stream, u"
\n") class Example: def __init__(self, description=u"", commands=[]): self.description = description self.commands = commands def __repr__(self): return "Example(%s, %s)" % \ (repr(self.description), repr(self.commands)) @classmethod def parse(cls, xml_dom): return cls(description=text(subtag(xml_dom, u"description")), commands=map(Command.parse, subtags(xml_dom, u"command"))) def to_man(self, stream): write_u(stream, u".LP\n") write_u(stream, self.description) # FIXME write_u(stream, u"\n") for command in self.commands: command.to_man(stream) def to_html(self, stream): write_u(stream, u'
\n') write_u(stream, u"
%s
\n" % (self.description)) write_u(stream, u"
\n") for command in self.commands: command.to_html(stream) write_u(stream, u"
\n") write_u(stream, u"
\n") class Command: def __init__(self, commandline, note=None): self.commandline = commandline self.note = note def __repr__(self): return "Command(%s, %s)" % (repr(self.commandline), repr(self.note)) @classmethod def parse(cls, xml_dom): if xml_dom.hasAttribute(u"note"): note = xml_dom.getAttribute(u"note") else: note = None return cls(commandline=text(xml_dom), note=note) def to_man(self, stream): if self.note is not None: write_u(stream, u".LP\n") write_u(stream, self.note + u" :\n\n") write_u(stream, u".IP\n") write_u(stream, self.commandline) write_u(stream, u"\n\n") def to_html(self, stream): if self.note is not None: write_u(stream, u'%s :
\n' % (self.note)) write_u(stream, self.commandline) write_u(stream, u"
\n") class Element_P: def __init__(self, contents): self.contents = contents def __repr__(self): return "Element_P(%s)" % (repr(self.contents)) @classmethod def parse(cls, xml_dom): return cls(contents=text(xml_dom)) def to_man(self, stream): write_u(stream, self.contents) write_u(stream, u"\n.PP\n") def to_html(self, stream): write_u(stream, u"

%s

" % (self.contents)) class Element_UL: def __init__(self, list_items): self.list_items = list_items def __repr__(self): return "Element_UL(%s)" % (repr(self.list_items)) @classmethod def parse(cls, xml_dom): return cls(list_items=map(text, subtags(xml_dom, u"li"))) def to_man(self, stream): for item in self.list_items: write_u(stream, u"\\[bu] ") write_u(stream, item) write_u(stream, u"\n") write_u(stream, u".PP\n") def to_html(self, stream): write_u(stream, u"
    \n") for item in self.list_items: write_u(stream, u"
  • %s
    • \n" % (item)) write_u(stream, u"
\n") class Element_TABLE: def __init__(self, rows): self.rows = rows def __repr__(self): return "Element_TABLE(%s)" % (repr(self.rows)) @classmethod def parse(cls, xml_dom): return cls(rows=[Element_TR.parse(tr) for tr in subtags(xml_dom, u"tr")]) def to_man(self, stream): if len(self.rows) == 0: return if (len(set([len(row.columns) for row in self.rows if row.tr_class in (TR_NORMAL, TR_HEADER)])) != 1): raise ValueError("all rows must have the same number of columns") else: columns = len(self.rows[0].columns) write_u(stream, u".TS\n") write_u(stream, u"tab(:);\n") write_u(stream, u" ".join([u"l" for l in self.rows[0].columns]) + u".\n") for row in self.rows: row.to_man(stream) write_u(stream, u".TE\n") def to_html(self, stream): if len(self.rows) == 0: return if (len({len(row.columns) for row in self.rows if row.tr_class in (TR_NORMAL, TR_HEADER)}) != 1): raise ValueError("all rows must have the same number of columns") write_u(stream, u"\n") for (row, spans) in zip(self.rows, self.calculate_row_spans()): row.to_html(stream, spans) write_u(stream, u"
\n") def calculate_row_spans(self): # turn rows into arrays of "span" boolean values row_spans = [] for row in self.rows: if row.tr_class in (TR_NORMAL, TR_HEADER): row_spans.append([col.empty() for col in row.columns]) elif row.tr_class == TR_DIVIDER: row_spans.append([False] * len(row_spans[-1])) # turn columns into arrays of integers containing the row span columns = [list(self.calculate_span_column([row[i] for row in row_spans])) for i in xrange(len(row_spans[0]))] # turn columns back into rows and return them return zip(*columns) def calculate_span_column(self, row_spans): rows = None for span in row_spans: if span: rows += 1 else: if rows is not None: yield rows for i in xrange(rows - 1): yield 0 rows = 1 if rows is not None: yield rows for i in xrange(rows - 1): yield 0 (TR_NORMAL, TR_HEADER, TR_DIVIDER) = range(3) class Element_TR: def __init__(self, columns, tr_class): self.columns = columns self.tr_class = tr_class def __repr__(self): if self.tr_class in (TR_NORMAL, TR_HEADER): return "Element_TR(%s, %s)" % (repr(self.columns), self.tr_class) else: return "Element_TR_DIVIDER()" @classmethod def parse(cls, xml_dom): if xml_dom.hasAttribute("class"): if xml_dom.getAttribute("class") == "header": return cls(columns=[Element_TD.parse(tag) for tag in subtags(xml_dom, u"td")], tr_class=TR_HEADER) elif xml_dom.getAttribute("class") == "divider": return cls(columns=None, tr_class=TR_DIVIDER) else: raise ValueError("unsupported class \"%s\"" % (xmldom_getAttribute("class"))) else: return cls(columns=[Element_TD.parse(tag) for tag in subtags(xml_dom, u"td")], tr_class=TR_NORMAL) def to_man(self, stream): if self.tr_class == TR_NORMAL: write_u(stream, u":".join(column.string() for column in self.columns) + u"\n") elif self.tr_class == TR_HEADER: write_u(stream, u":".join(u"\\fB%s\\fR" % (column.string()) for column in self.columns) + u"\n") write_u(stream, u"_\n") elif self.tr_class == TR_DIVIDER: write_u(stream, u"_\n") def column_widths(self): if self.tr_class in (TR_NORMAL, TR_HEADER): return [column.width() for column in self.columns] else: return None def to_html(self, stream, rowspans): if self.tr_class in (TR_NORMAL, TR_HEADER): write_u(stream, u"\n") for (column, span) in zip(self.columns, rowspans): column.to_html(stream, self.tr_class == TR_HEADER, span) write_u(stream, u"\n") class Element_TD: def __init__(self, value): self.value = value def __repr__(self): return "Element_TD(%s)" % (repr(self.value)) @classmethod def parse(cls, xml_dom): try: return cls(value=WHITESPACE.sub( u" ", xml_dom.childNodes[0].wholeText.strip())) except IndexError: return cls(value=None) def empty(self): return self.value is None def string(self): if self.value is not None: return str(self.value.encode('ascii')) else: return "\\^" def to_man(self, stream): stream.write(self.value) def width(self): if self.value is not None: return len(self.value) else: return 0 def to_html(self, stream, header, rowspan): if self.value is not None: if rowspan > 1: rowspan = u" rowspan=\"%d\"" % (rowspan) else: rowspan = u"" if header: write_u(stream, u"%s" % (rowspan, self.value)) else: write_u(stream, u"%s" % (rowspan, self.value)) class Element: SUB_ELEMENTS = {u"p": Element_P, u"ul": Element_UL, u"table": Element_TABLE} def __init__(self, name, elements): self.name = name self.elements = elements def __repr__(self): return "Element(%s, %s)" % (repr(self.name), repr(self.elements)) @classmethod def parse(cls, xml_dom): if xml_dom.hasAttribute(u"name"): name = xml_dom.getAttribute(u"name") else: raise ValueError("elements must have names") elements = [] for child in xml_dom.childNodes: if hasattr(child, "tagName"): if child.tagName in cls.SUB_ELEMENTS.keys(): elements.append( cls.SUB_ELEMENTS[child.tagName].parse(child)) else: raise ValueError("unsupported tag %s" % (child.tagName.encode('ascii'))) return cls(name=name, elements=elements) def to_man(self, stream): write_u(stream, u".SH %s\n" % (self.name.upper())) for element in self.elements: element.to_man(stream) def to_html(self, stream): write_u(stream, u"
\n") write_u(stream, u"
%s
\n" % (u" ".join(part.capitalize() for part in self.name.split()))) write_u(stream, u"
\n") for element in self.elements: element.to_html(stream) write_u(stream, u"
\n") write_u(stream, u"
\n") if (__name__ == '__main__'): import sys import xml.dom.minidom import argparse from os import stat parser = argparse.ArgumentParser(description="manual page generator") parser.add_argument("-i", "--input", dest="input", help="the primary input XML file") parser.add_argument("-t", "--type", dest="type", choices=("man", "html"), default="man", help="the output type") parser.add_argument("see_also", metavar="FILENAME", nargs="*", help="\"see also\" man pages") options = parser.parse_args() if options.input is not None: main_page = Manpage.parse_file(options.input) all_pages = [Manpage.parse_file(filename) for filename in options.see_also] main_page.see_also = [page for page in all_pages if (page.utility != main_page.utility)] if options.type == "man": main_page.to_man(sys.stdout, stat(options.input).st_mtime) elif options.type == "html": main_page.to_html(sys.stdout) ================================================ FILE: docs/programming/Makefile ================================================ # Audio Tools, a module and set of tools for manipulating audio data # Copyright (C) 2007-2015 Brian Langenberger # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA # Makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build PAPER = # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d build/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source .PHONY: help clean html dirhtml pickle json htmlhelp qthelp latex changes linkcheck doctest help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @echo " dirhtml to make HTML files named index.html in directories" @echo " pickle to make pickle files" @echo " json to make JSON files" @echo " htmlhelp to make HTML files and a HTML help project" @echo " qthelp to make HTML files and a qthelp project" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " changes to make an overview of all changed/added/deprecated items" @echo " linkcheck to check all external links for integrity" @echo " doctest to run all doctests embedded in the documentation (if enabled)" clean: -rm -rf build html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) build/html @echo @echo "Build finished. The HTML pages are in build/html." dirhtml: $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) build/dirhtml @echo @echo "Build finished. The HTML pages are in build/dirhtml." pickle: $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) build/pickle @echo @echo "Build finished; now you can process the pickle files." json: $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) build/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) build/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ ".hhp project file in build/htmlhelp." qthelp: $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) build/qthelp @echo @echo "Build finished; now you can run "qcollectiongenerator" with the" \ ".qhcp project file in build/qthelp, like this:" @echo "# qcollectiongenerator build/qthelp/PythonAudioTools.qhcp" @echo "To view the help file:" @echo "# assistant -collectionFile build/qthelp/PythonAudioTools.qhc" latex: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) build/latex @echo @echo "Build finished; the LaTeX files are in build/latex." @echo "Run \`make all-pdf' or \`make all-ps' in that directory to" \ "run these through (pdf)latex." changes: $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) build/changes @echo @echo "The overview file is in build/changes." linkcheck: $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) build/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ "or in build/linkcheck/output.txt." doctest: $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) build/doctest @echo "Testing of doctests in the sources finished, look at the " \ "results in build/doctest/output.txt." ================================================ FILE: docs/programming/source/audiotools.rst ================================================ .. Audio Tools, a module and set of tools for manipulating audio data Copyright (C) 2007-2016 Brian Langenberger This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA :mod:`audiotools` --- the Base Python Audio Tools Module ======================================================== .. module:: audiotools :synopsis: the Base Python Audio Tools Module The :mod:`audiotools` module contains a number of useful base classes and functions upon which all of the other modules depend. .. data:: VERSION The current Python Audio Tools version as a plain string. .. data:: AVAILABLE_TYPES A tuple of :class:`AudioFile`-compatible classes of available audio types. Note these are types available to audiotools, not necessarily available to the user - depending on whether the required binaries are installed or not. ============= ================================== Class Format ------------- ---------------------------------- AACAudio AAC in ADTS container AiffAudio Audio Interchange File Format ALACAudio Apple Lossless AuAudio Sun Au FlacAudio Native Free Lossless Audio Codec M4AAudio AAC in M4A container MP3Audio MPEG-1 Layer 3 MP2Audio MPEG-1 Layer 2 OggFlacAudio Ogg Free Lossless Audio Codec OpusAudio Opus Audio Codec ShortenAudio Shorten SpeexAudio Ogg Speex VorbisAudio Ogg Vorbis WaveAudio Waveform Audio File Format WavPackAudio WavPack ============= ================================== .. data:: DEFAULT_TYPE The default type to use as a plain string, such as ``'wav'`` or ``'flac'``. .. data:: DEFAULT_QUALITY A dict of type name strings -> quality value strings indicating the default compression quality value for the given type name suitable for :meth:`AudioFile.from_pcm` and :meth:`AudioFile.convert` method calls. .. data:: DEFAULT_CDROM The default CD-ROM device to use for CD audio and DVD-Audio extraction as a plain string. .. data:: TYPE_MAP A dictionary of type name strings -> :class:`AudioFile` values containing only types which have all required binaries installed. .. data:: FILENAME_FORMAT The default format string to use for newly created files. .. data:: BIN A dictionary-like class for performing lookups of system binaries. This checks the system and user's config files and ensures that any redirected binaries are called from their proper location. For example, if the user has configured ``flac(1)`` to be run from ``/opt/flac/bin/flac`` >>> BIN["flac"] "/opt/flac/bin/flac" This class also has a ``can_execute()`` method which returns ``True`` if the given binary is executable. >>> BIN.can_execute(BIN["flac"]) True .. data:: IO_ENCODING The defined encoding to use for output to the screen as a plain string. This is typically ``'utf-8'``. .. data:: FS_ENCODING The defined encoding to use for filenames read and written to disk as a plain string. This is typically ``'utf-8'``. .. data:: MAX_JOBS The maximum number of simultaneous jobs to run at once by default as an integer. This may be defined from the user's config file. Otherwise, if Python's ``multiprocessing`` module is available, this is set to the user's CPU count. If neither is available, this is set to 1. .. function:: file_type(file) Given a seekable file object returns an :class:`AudioFile`-compatible class of the stream's detected type, or ``None`` if the stream's type is unknown. The :class:`AudioFile` class may not be available for use and so its :meth:`AudioFile.available` classmethod may need to be checked separately. .. function:: open(filename) Opens the given filename string and returns an :class:`AudioFile`-compatible object. Raises :exc:`UnsupportedFile` if the file cannot identified or is not supported. Raises :exc:`IOError` if the file cannot be opened at all. .. function:: open_files(filenames[, sorted][, messenger][, no_duplicates][, warn_duplicates][, opened_files]) Given a list of filename strings, returns a list of :class:`AudioFile`-compatible objects which are successfully opened. By default, they are returned sorted by album number and track number. If ``sorted`` is ``False``, they are returned in the same order as they appear in the filenames list. If ``messenger`` is given, use that :class:`Messenger` object to for warnings if files cannot be opened. Otherwise, such warnings are sent to stdout. If ``no_duplicates`` is ``True``, attempting to open the same file twice raises a :exc:`DuplicateFile` exception. If ``no_duplicates`` is ``False`` and ``warn_duplicates`` is ``True``, attempting to open the same file twice results in a warning to ``messenger``, if present. ``opened_files``, if present, is a set of previously opened :class:`Filename` objects for the purpose of detecting duplicates. Any opened files are added to that set. .. function:: open_directory(directory[, sorted[, messenger]]) Given a root directory, returns an iterator of all the :class:`AudioFile`-compatible objects found via a recursive search of that directory. ``sorted``, and ``messenger`` work as in :func:`open_files`. .. function:: sorted_tracks(audiofiles) Given a list of :class:`AudioFile` objects, returns a new list of those objects sorted by album number and track number, if present. If album number and track number aren't present, objects are sorted by base filename. .. function:: group_tracks(audiofiles) Given an iterable collection of :class:`AudioFile`-compatible objects, returns an iterator of objects grouped into lists by album. That is, all objects with the same ``album_name`` and ``album_number`` metadata fields will be returned in the same list on each pass. .. function:: filename_to_type(path) Given a path, try to guess its :class:`AudioFile` class based on its filename suffix. Raises :exc:`UnknownAudioType` if the suffix is unrecognized. Raises :exc:`AmbiguousAudioType` if more than one type of audio shares the same suffix. .. function:: transfer_data(from_function, to_function) This function takes two functions, presumably analogous to :func:`write` and :func:`read` functions, respectively. It calls ``to_function`` on the object returned by calling ``from_function`` with an integer argument (presumably a string) until that object's length is 0. >>> infile = open("input.txt", "r") >>> outfile = open("output.txt", "w") >>> transfer_data(infile.read, outfile.write) >>> infile.close() >>> outfile.close() .. function:: transfer_framelist_data(pcmreader, to_function[, signed[, big_endian]]) A natural progression of :func:`transfer_data`, this function takes a :class:`PCMReader` object and transfers the :class:`pcm.FrameList` objects returned by its :meth:`PCMReader.read` method to ``to_function`` after converting them to plain strings. The pcmreader is closed once decoding is complete. May raise :exc:`IOError` or :exc:`ValueError` if a problem occurs during decoding. >>> pcm_data = audiotools.open("file.wav").to_pcm() >>> outfile = open("output.pcm","wb") >>> transfer_framelist_data(pcm_data,outfile) >>> pcm_data.close() >>> outfile.close() .. function:: pcm_cmp(pcmreader1, pcmreader2) This function takes two :class:`PCMReader` objects and compares their PCM output. Returns ``True`` if that output matches exactly, ``False`` if not. Both streams are closed once comparison is completed. May raise :exc:`IOError` or :exc:`ValueError` if problems occur during reading. .. function:: pcm_frame_cmp(pcmreader1, pcmreader2) This function takes two :class:`PCMReader` objects and compares their PCM frame output. It returns the frame number of the first mismatch as an integer which begins at frame number 0. If the two streams match completely, it returns ``None``. Both streams are closed once comparison is completed. May raise :exc:`IOError` or :exc:`ValueError` if problems occur during reading. .. function:: pcm_split(pcmreader, pcm_lengths) Takes a :class:`PCMReader` object and list of PCM sample length integers. Returns an iterator of new :class:`PCMReader` objects, each limited to the given lengths. The original pcmreader is closed upon the iterator's completion. .. function:: calculate_replay_gain(audiofiles) Takes a list of :class:`AudioFile`-compatible objects. Returns an iterator of ``(audiofile, track_gain, track_peak, album_gain, album_peak)`` tuples or raises :exc:`ValueError` if a problem occurs during calculation. .. function:: read_sheet(filename) Given a ``.cue`` or ``.toc`` filename, returns a :class:`Sheet` of that file's cuesheet data. May raise :exc:`SheetException` if the file cannot be read or parsed correctly. .. function:: to_pcm_progress(audiofile, progress) Given an :class:`AudioFile`-compatible object and ``progress`` function, returns a :class:`PCMReaderProgress` object of that object's PCM stream. If ``progress`` is ``None``, the audiofile's PCM stream is returned as-is. Filename Objects ---------------- .. class:: Filename(filename) :class:`Filename` is a file which may or may not exist on disk. ``filename`` is a raw string of the actual filename. Filename objects are immutable and hashable, which means they can be used as dictionary keys or placed in sets. The purpose of Filename objects is for easier conversion of raw string filename paths to Unicode, and to make it easier to detect filenames which point to the same file on disk. The former case is used by utilities to display output about file operations in progress. The latter case is for utilities which need to avoid overwriting input files with output files. .. function:: Filename.__str__() Returns the raw string of the actual filename after being normalized. .. function:: Filename.__unicode__() Returns a Unicode string of the filename after being decoded through :attr:`FS_ENCODING`. .. function:: Filename.__eq__(filename) Filename objects which exist on disk hash and compare equally if their device ID and inode number values match (the ``st_dev`` and ``st_ino`` fields according to stat(2)). Filename objects which don't exist on disk hash and compare equally if their filename string matches. .. function:: Filename.open(mode) Returns a file object of this filename opened with the given mode. .. function:: Filename.disk_file() Returns ``True`` if the file currently exists on disk. .. function:: Filename.dirname() Returns the directory name of this filename as a new :class:`Filename` object. .. function:: Filename.basename() Returns the base name (no directory) of this filename as a new :class:`Filename` object. .. function:: Filename.expanduser() Returns a new :class:`Filename` object with the user directory expanded, if any. AudioFile Objects ----------------- .. class:: AudioFile() The :class:`AudioFile` class represents an audio file on disk, such as a FLAC file, MP3 file, WAVE file and so forth. It is not meant to be instantiated directly. Instead, functions such as :func:`open` will return :class:`AudioFile`-compatible objects with the following attributes and methods. .. attribute:: AudioFile.NAME The name of the format as a string. This is how the format is referenced by utilities via the `-t` option, and must be unique among all formats. .. attribute:: AudioFile.SUFFIX The default file suffix as a string. This is used by the ``%(suffix)s`` format field in the :meth:`track_name` classmethod, and by the :func:`filename_to_type` function for inferring the file format from its name. However, it need not be unique among all formats. .. attribute:: AudioFile.DESCRIPTION A longer, descriptive name for the audio type as a Unicode string. This is meant to be human-readable. .. attribute:: AudioFile.COMPRESSION_MODES A tuple of valid compression level strings, for use with the :meth:`from_pcm` and :meth:`convert` methods. If the format has no compression levels, this tuple will be empty. .. attribute:: AudioFile.DEFAULT_COMPRESSION A string of the default compression level to use with :meth:`from_pcm` and :meth:`convert`, if none is given. This is *not* the default compression indicated in the user's configuration file; it is a hard-coded value of last resort. .. attribute:: AudioFile.COMPRESSION_DESCRIPTIONS A dict of compression descriptions, as Unicode strings. The key is a valid compression mode string. Not all compression modes need have a description; some may be left blank. .. attribute:: AudioFile.BINARIES A tuple of binary strings required by the format. For example, the Vorbis format may require ``"oggenc"`` and ``"oggdec"`` in order to be available for the user. .. attribute:: AudioFile.REPLAYGAIN_BINARIES A tuple of binary strings required for ReplayGain application. For example, the Vorbis format may require ``"vorbisgain"`` in order to use the :meth:`add_replay_gain` classmethod. This tuple may be empty if the format requires no binaries or has no ReplayGain support. .. method:: AudioFile.bits_per_sample() Returns the number of bits-per-sample in this audio file as a positive integer. .. method:: AudioFile.channels() Returns the number of channels in this audio file as a positive integer. .. method:: AudioFile.channel_mask() Returns a :class:`ChannelMask` object representing the channel assignment of this audio file. If the channel assignment is unknown or undefined, that :class:`ChannelMask` object may have an undefined value. .. method:: AudioFile.sample_rate() Returns the sample rate of this audio file, in Hz, as a positive integer. .. method:: AudioFile.total_frames() Returns the total number of PCM frames in this audio file, as a non-negative integer. .. method:: AudioFile.cd_frames() Returns the total number of CD frames in this audio file, as a non-negative integer. Each CD frame is 1/75th of a second. .. method:: AudioFile.seconds_length() Returns the length of this audio file as a :class:`fractions.Fraction` number of seconds. .. method:: AudioFile.lossless() Returns ``True`` if the data in the audio file has been stored losslessly. Returns ``False`` if not. .. classmethod:: AudioFile.supports_metadata() Returns ``True`` is this audio type supports metadata. If not, :meth:`AudioFile.get_metadata` will always return ``None`` and the metadata updating routines will do nothing. .. method:: AudioFile.set_metadata(metadata) Takes a :class:`MetaData` object and sets this audio file's metadata to that value, if possible. Setting metadata to ``None`` is the same as calling :meth:`AudioFile.delete_metadata`. Raises :exc:`IOError` if a problem occurs when writing the file. .. method:: AudioFile.update_metadata(metadata) Takes the :class:`MetaData`-compatible object returned by this audio file's :meth:`AudioFile.get_metadata` method and sets this audiofile's metadata to that value, if possible. Raises :exc:`IOError` if a problem occurs when writing the file. .. note:: What's the difference between :meth:`AudioFile.set_metadata` and :meth:`AudioFile.update_metadata`? Metadata implementations may also contain side information such as track length, file encoder, and so forth. :meth:`AudioFile.set_metadata` presumes the :class:`MetaData` object is from a different :class:`AudioFile` object or has been built from scratch. Therefore, it will update the newly added metadata side info as needed so as to not break the file. :meth:`AudioFile.update_metadata` presumes the :class:`MetaData` object is either taken from the original :class:`AudioFile` object or has been carefully constructed to not break anything when applied to the file. It is a lower-level routine which does *not* update metadata side info (which may be necessary when modifying that side info is required). .. method:: AudioFile.get_metadata() Returns a :class:`MetaData`-compatible object representing this audio file's metadata, or ``None`` if this file contains no metadata. Raises :exc:`IOError` if a problem occurs when reading the file. .. method:: AudioFile.delete_metadata() Deletes the audio file's metadata, removing or unsetting tags as necessary. Raises :exc:`IOError` if a problem occurs when writing the file. .. method:: AudioFile.to_pcm() Returns this audio file's PCM data as a :class:`PCMReader`-compatible object. May return a :class:`PCMReaderError` if an error occurs initializing the decoder. .. classmethod:: AudioFile.supports_to_pcm() Returns ``True`` if the necessary libraries or binaries are installed to support decoding this format. .. classmethod:: AudioFile.from_pcm(filename, pcmreader[, compression][, total_pcm_frames]) Takes a filename string, :class:`PCMReader`-compatible object, optional compression level string and optional total_pcm_frames integer. Creates a new audio file as the same format as this audio class and returns a new :class:`AudioFile`-compatible object. The :meth:`PCMReader.close` method is called once encoding is complete. Raises :exc:`EncodingError` if a problem occurs during encoding. Specifying the total number of PCM frames to be encoded, when the number is known in advance, may allow the encoder to work more efficiently but is never required. In this example, we'll transcode ``track.flac`` to ``track.mp3`` at the default compression level: >>> audiotools.MP3Audio.from_pcm("track.mp3", ... audiotools.open("track.flac").to_pcm()) .. classmethod:: AudioFile.supports_from_pcm() Returns ``True`` if the necessary libraries or binaries are installed to support encoding this format. .. method:: AudioFile.convert(filename, target_class[, compression[, progress]]) Takes a filename string, :class:`AudioFile` subclass and optional compression level string. Creates a new audio file and returns an object of the same class. Raises :exc:`EncodingError` if a problem occurs during encoding. In this example, we'll transcode ``track.flac`` to ``track.mp3`` at the default compression level: >>> audiotools.open("track.flac").convert("track.mp3", ... audiotools.MP3Audio) Why have both a ``convert`` method as well as ``to_pcm``/``from_pcm`` methods? Although the former is often implemented using the latter, the pcm methods alone contain only raw audio data. By comparison, the ``convert`` method has information about what is the file is being converted to and can transfer other side data if necessary. For example, if .wav file with non-audio RIFF chunks is converted to WavPack, this method will preserve those chunks: >>> audiotools.open("chunks.wav").convert("chunks.wv", ... audiotools.WavPackAudio) whereas the ``to_pcm``/``from_pcm`` method alone will not. The optional ``progress`` argument is a function which takes a single :class:`Fraction` argument which is the current progress between 0 and 1, inclusive. If supplied, this function is called at regular intervals during the conversion process and may be used to indicate the current status to the user. .. method:: AudioFile.seekable() Returns ``True`` if the file is seekable. That is, if its :class:`PCMReader` has a .seek() method and that method supports some fine-grained seeking when the PCMReader is working from on-disk files. .. method:: AudioFile.verify([progress]) Verifies the track for correctness. Returns ``True`` if verification is successful. Raises an :class:`InvalidFile` subclass if some problem is detected. If the file has built-in checksums or other error detection capabilities, this method checks those values to ensure it has not been damaged in some way. The optional ``progress`` argument functions identically to the one provided to :meth:`convert`. .. classmethod:: AudioFile.track_name(file_path[, track_metadata[, format[, suffix]]]) Given a file path string, optional :class:`MetaData`-compatible object, optional Python format string, and optional suffix string, returns a filename string with the format string fields filled-in. Raises :exc:`UnsupportedTracknameField` if the format string contains unsupported fields. Currently supported fields are: ========================== =============================================== Field Value -------------------------- ----------------------------------------------- ``%(album_name)s`` ``track_metadata.album_name`` ``%(album_number)s`` ``track_metadata.album_number`` ``%(album_total)s`` ``track_metadata.album_total`` ``%(album_track_number)s`` ``album_number`` combined with ``track_number`` ``%(artist_name)s`` ``track_metadata.artist_name`` ``%(catalog)s`` ``track_metadata.catalog`` ``%(comment)s`` ``track_metadata.comment`` ``%(composer_name)s`` ``track_metadata.composer_name`` ``%(conductor_name)s`` ``track_metadata.conductor_name`` ``%(copyright)s`` ``track_metadata.copyright`` ``%(date)s`` ``track_metadata.date`` ``%(ISRC)s`` ``track_metadata.ISRC`` ``%(media)s`` ``track_metadata.year`` ``%(performer_name)s`` ``track_metadata.performer_name`` ``%(publisher)s`` ``track_metadata.publisher`` ``%(suffix)s`` the :class:`AudioFile` suffix ``%(track_name)s`` ``track_metadata.track_name`` ``%(track_number)2.2d`` ``track_metadata.track_number`` ``%(track_total)s`` ``track_metadata.track_total`` ``%(year)s`` ``track_metadata.year`` ``%(basename)s`` ``file_path`` basename without suffix ========================== =============================================== .. classmethod:: AudioFile.supports_replay_gain() Returns ``True`` if this class supports ReplayGain metadata. .. method:: AudioFile.get_replay_gain() Returns this audio file's ReplayGain values as a :class:`ReplayGain` object, or ``None`` if this audio file has no values. .. method:: AudioFile.set_replay_gain(replaygain) Given a :class:`ReplayGain` object, sets the audio file's gain values. Raises :exc:`IOError` if unable to modify the file. .. method:: AudioFile.delete_replay_gain() Removes any gain values from the file. Raises :exc:`IOError` if unable to modify the file. .. classmethod:: AudioFile.supports_cuesheet() Returns ``True`` if the audio format supports embedded :class:`Sheet` objects. .. method:: AudioFile.set_cuesheet(cuesheet) Given a :class:`Sheet` object, embeds a cuesheet in the track. This is for tracks which represent a whole CD image and wish to store track break data internally. May raise :exc:`IOError` if an error occurs writing the file. .. method:: AudioFile.get_cuesheet() Returns a :class:`Sheet` object of a track's embedded cuesheet, or ``None`` if the track contains no cuesheet. May raise :exc:`IOError` if an error occurs reading the file. .. method:: AudioFile.delete_cuesheet() Deletes embedded :class:`Sheet` object, if any. May raise :exc:`IOError` if an error occurs updating the file. .. method:: AudioFile.clean([output_filename]) Cleans the audio file of known data and metadata problems. ``output_filename`` is an optional string in which the fixed audio file is placed. If omitted, no actual fixes are performed. Note that this method never modifies the original file. Returns list of fixes performed as Unicode strings. Raises :exc:`IOError` if some error occurs when writing the new file. Raises :exc:`ValueError` if the file itself is invalid. WaveContainer Objects ^^^^^^^^^^^^^^^^^^^^^ This is an abstract :class:`AudioFile` subclass suitable for extending by formats that store RIFF WAVE chunks internally, such as Wave, FLAC, WavPack and Shorten. It overrides the :meth:`AudioFile.convert` method such that any stored chunks are transferred properly from one file to the next. This is accomplished by implementing three additional methods. .. class:: WaveContainer .. method:: WaveContainer.has_foreign_wave_chunks() Returns ``True`` if our object has non-audio RIFF WAVE chunks. .. method:: WaveContainer.wave_header_footer() Returns ``(header, footer)`` tuple of strings where ``header`` is everything before the PCM data and ``footer`` is everything after the PCM data. May raise :exc:`ValueError` if there's a problem with the header or footer data, such as invalid chunk IDs. May raise :exc:`IOError` if there's a problem reading the header or footer data from the file. .. classmethod:: WaveContainer.from_wave(filename, header, pcmreader, footer[, compression]) Encodes a new file from wave data. ``header`` and ``footer`` are binary strings as returned by a :meth:`WaveContainer.wave_header_footer` method, ``pcmreader`` is a :class:`PCMReader` object and ``compression`` is a binary string. Returns a new :class:`AudioFile`-compatible object or raises :exc:`EncodingError` if some error occurs when encoding the file. AiffContainer Objects ^^^^^^^^^^^^^^^^^^^^^ Much like :class:`WaveContainer`, this is an abstract :class:`AudioFile` subclass suitable for extending by formats that store AIFF chunks internally, such as AIFF, FLAC and Shorten. It overrides the :meth:`AudioFile.convert` method such that any stored chunks are transferred properly from one file to the next. This is accomplished by implementing three additional methods. .. class:: AiffContainer .. method:: AiffContainer.has_foreign_aiff_chunks() Returns ``True`` if our object has non-audio AIFF chunks. .. method:: AiffContainer.aiff_header_footer() Returns ``(header, footer)`` tuple of strings where ``header`` is everything before the PCM data and ``footer`` is everything after the PCM data. May raise :exc:`ValueError` if there's a problem with the header or footer data, such as invalid chunk IDs. May raise :exc:`IOError` if there's a problem reading the header or footer data from the file. .. classmethod:: AiffContainer.from_aiff(filename, header, pcmreader, footer[, compression]) Encodes a new file from wave data. ``header`` and ``footer`` are binary strings as returned by a :meth:`AiffContainer.aiff_header_footer` method, ``pcmreader`` is a :class:`PCMReader` object and ``compression`` is a binary string. Returns a new :class:`AudioFile`-compatible object or raises :exc:`EncodingError` if some error occurs when encoding the file. MetaData Objects ---------------- .. class:: MetaData([track_name][, track_number][, track_total][, album_name][, artist_name][, performer_name][, composer_name][, conductor_name][, media][, ISRC][, catalog][, copyright][, publisher][, year][, data][, album_number][, album_total][, comment][, compilation][, images]) The :class:`MetaData` class represents an :class:`AudioFile`'s non-technical metadata. It can be instantiated directly for use by the :meth:`set_metadata` method. However, the :meth:`get_metadata` method will typically return :class:`MetaData`-compatible objects corresponding to the audio file's low-level metadata implementation rather than actual :class:`MetaData` objects. Modifying fields within a :class:`MetaData`-compatible object will modify its underlying representation and those changes will take effect should :meth:`set_metadata` be called with that updated object. The ``images`` argument, if given, should be an iterable collection of :class:`Image`-compatible objects. MetaData attributes may be ``None``, which indicates the low-level implementation has no corresponding entry. For instance, ID3v2.3 tags use the ``"TALB"`` frame to indicate the track's album name. If that frame is present, an :class:`audiotools.ID3v23Comment` MetaData object will have an ``album_name`` field containing a Unicode string of its value. If that frame is not present in the ID3v2.3 tag, its ``album_name`` field will be ``None``. For example, to access a track's album name field: >>> metadata = track.get_metadata() >>> metadata.album_name u"Album Name" To change a track's album name field: >>> metadata = track.get_metadata() >>> metadata.album_name = u"Updated Album Name" >>> track.update_metadata(metadata) # because metadata comes from track's get_metadata() method, one can use update_metadata() To delete a track's album name field: >>> metadata = track.get_metadata() >>> del(metadata.album_name) >>> track.update_metadata(metadata) Or to replace a track's entire set of metadata: >>> metadata = MetaData(track_name=u"Track Name", ... album_name=u"Updated Album Name", ... track_number=1, ... track_total=3) >>> track.set_metadata(metadata) # because metadata is built from scratch, one must use set_metadata() .. data:: MetaData.track_name This individual track's name as a Unicode string. .. data:: MetaData.track_number This track's number within the album as an integer. .. data:: MetaData.track_total The total number of tracks on the album as an integer. .. data:: MetaData.album_name The name of this track's album as a Unicode string. .. data:: MetaData.artist_name The name of this track's original creator/composer as a Unicode string. .. data:: MetaData.performer_name The name of this track's performing artist as a Unicode string. .. data:: MetaData.composer_name The name of this track's composer as a Unicode string. .. data:: MetaData.conductor_name The name of this track's conductor as a Unicode string. .. data:: MetaData.media The album's media type, such as u"CD", u"tape", u"LP", etc. as a Unicode string. .. data:: MetaData.ISRC This track's ISRC value as a Unicode string. .. data:: MetaData.catalog This track's album catalog number as a Unicode string. .. data:: MetaData.year This track's album release year as a Unicode string. .. data:: MetaData.date This track's album recording date as a Unicode string. .. data:: MetaData.album_number This track's album number if it is one of a series of albums, as an integer. .. data:: MetaData.album_total The total number of albums within the set, as an integer. .. data:: MetaData.comment This track's comment as a Unicode string. .. data:: MetaData.compilation Whether this track is part of a compilation, as a boolean. .. method:: MetaData.fields() Yields an ``(attr, value)`` tuple per :class:`MetaData` field. .. method:: MetaData.filled_fields() Yields an ``(attr, value)`` tuple per non-blank :class:`MetaData` field. Non-blank fields are those with a value other than ``None``. .. method:: MetaData.empty_fields() Yields an ``(attr, value)`` tuple per blank :class:`MetaData` field. Blank fields are those with a value of ``None``. .. classmethod:: MetaData.converted(metadata) Takes a :class:`MetaData`-compatible object (or ``None``) and returns a new :class:`MetaData` object of the same class, or ``None``. For instance, ``VorbisComment.converted()`` returns ``VorbisComment`` objects. The purpose of this classmethod is to offload metadata conversion to the metadata classes themselves. Therefore, by using the ``VorbisComment.converted()`` classmethod, the ``VorbisAudio`` class only needs to know how to handle ``VorbisComment`` metadata. Why not simply handle all metadata using this high-level representation and avoid conversion altogether? The reason is that :class:`MetaData` is often only a subset of what the low-level implementation can support. For example, a ``VorbisComment`` may contain the ``'FOO'`` tag which has no analogue in :class:`MetaData`'s list of fields. But when passed through the ``VorbisComment.converted()`` classmethod, that ``'FOO'`` tag will be preserved as one would expect. The key is that performing: >>> track.set_metadata(track.get_metadata()) should always round-trip properly and not lose any metadata values. .. classmethod:: MetaData.supports_images() Returns ``True`` if this :class:`MetaData` implementation supports images. Returns ``False`` if not. .. method:: MetaData.images() Returns a list of :class:`Image`-compatible objects this metadata contains. .. method:: MetaData.front_covers() Returns a subset of :meth:`images` which are marked as front covers. .. method:: MetaData.back_covers() Returns a subset of :meth:`images` which are marked as back covers. .. method:: MetaData.leaflet_pages() Returns a subset of :meth:`images` which are marked as leaflet pages. .. method:: MetaData.media_images() Returns a subset of :meth:`images` which are marked as media. .. method:: MetaData.other_images() Returns a subset of :meth:`images` which are marked as other. .. method:: MetaData.add_image(image) Takes a :class:`Image`-compatible object and adds it to this metadata's list of images. .. method:: MetaData.delete_image(image) Takes an :class:`Image` from this class, as returned by :meth:`images`, and removes it from this metadata's list of images. .. method:: MetaData.clean() Returns a (:class:`MetaData`, ``fixes_performed``) tuple where ``MetaData`` is an object that's been cleaned of problems and ``fixes_performed`` is a list of unicode strings detailing those problems. Problems include: * Leading whitespace in text fields * Trailing whitespace in text fields * Empty fields * Leading zeroes in numerical fields * Incorrectly labeled image metadata fields .. method:: MetaData.raw_info() Returns a Unicode string of raw metadata information with as little filtering as possible. This is meant to be useful for debugging purposes. .. method:: MetaData.intersection(metadata) Given a :class:`MetaData` object, this returns a new :class:`MetaData` object containing only the fields which match those in this object. It is analagous to a :class:`set` intersection in that only common fields are preserved. If both objects are of the same class, an object of the same class will be returned. If ``metadata`` is ``None``, returns ``None``. Image Objects ------------- .. class:: Image(data, mime_type, width, height, color_depth, color_count, description, type) This class is a container for image data. .. data:: Image.data A plain string of raw image bytes. .. data:: Image.mime_type A Unicode string of this image's MIME type, such as u'image/jpeg' .. data:: Image.width This image's width in pixels as an integer. .. data:: Image.height This image's height in pixels as an integer .. data:: Image.color_depth This image's color depth in bits as an integer. 24 for JPEG, 8 for GIF, etc. .. data:: Image.color_count For palette-based images, this is the number of colors the image contains as an integer. For non-palette images, this value is 0. .. data:: Image.description A Unicode string of this image's description. .. data:: Image.type An integer representing this image's type. ===== ============ Value Type ----- ------------ 0 front cover 1 back cover 2 leaflet page 3 media 4 other ===== ============ .. method:: Image.suffix() Returns this image's typical filename suffix as a plain string. For example, JPEGs return ``"jpg"`` .. method:: Image.type_string() Returns this image's type as a plain string. For example, an image of type 0 returns ``"Front Cover"`` .. classmethod:: Image.new(image_data, description, type) Given a string of raw image bytes, a Unicode description string and image type integer, returns an :class:`Image`-compatible object. Raises :exc:`InvalidImage` If unable to determine the image type from the data string. ReplayGain Objects ------------------ .. class:: ReplayGain(track_gain, track_peak, album_gain, album_peak) This is a simple container for ReplayGain values. .. data:: ReplayGain.track_gain A float of a track's ReplayGain value. .. data:: ReplayGain.track_peak A float of a track's peak value, from 0.0 to 1.0 .. data:: ReplayGain.album_gain A float of an album's ReplayGain value. .. data:: ReplayGain.album_peak A float of an album's peak value, from 0.0 to 1.0 PCMReader Objects ----------------- .. class:: PCMReader(sample_rate, channels, channel_mask, bits_per_sample) This is an abstract base class for streams of audio data which are file-like objects with additional stream parameters. Subclasses are expected to implement ``read`` and ``close``. .. data:: PCMReader.sample_rate The sample rate of this audio stream, in Hz, as a positive integer. .. data:: PCMReader.channels The number of channels in this audio stream as a positive integer. .. data:: PCMReader.channel_mask The channel mask of this audio stream as a non-negative integer. .. data:: PCMReader.bits_per_sample The number of bits-per-sample in this audio stream as a positive integer. .. method:: PCMReader.read(pcm_frames) Try to read a :class:`pcm.FrameList` object with the given number of PCM frames, if possible. This method is *not* guaranteed to read that amount of frames. It may return less, particularly at the end of an audio stream. It may even return FrameLists larger than requested. However, it must always return a non-empty FrameList until the end of the PCM stream is reached. Once the end of the stream is reached, subsequent calls will return empty FrameLists. May raise :exc:`IOError` if there is a problem reading the source file, or :exc:`ValueError` if the source file has some sort of error. .. method:: PCMReader.close() Closes the audio stream. If any subprocesses were used for audio decoding, they will also be closed and waited for their process to finish. Subsequent calls to :meth:`PCMReader.read` will raise :exc:`ValueError` exceptions once the stream is closed. .. method:: PCMReader.__enter__() Returns the PCMReader. This is used for implementing context management. .. method:: PCMReader.__exit__(exc_type, exc_value, traceback) Calls :meth:`PCMReader.close`. This is used for implementing context management. PCMFileReader Objects ^^^^^^^^^^^^^^^^^^^^^ .. class:: PCMFileReader(file, sample_rate, channels, channel_mask, bits_per_sample[, process[, signed[, big_endian]]]) This class wraps around file-like objects and generates :class:`pcm.FrameList` objects on each call to :meth:`read`. ``sample_rate``, ``channels``, ``channel_mask`` and ``bits_per_sample`` should be integers. ``process`` is a subprocess helper object which generates PCM data. ``signed`` is ``True`` if the generated PCM data is signed. ``big_endian`` is ``True`` if the generated PCM data is big-endian. PCMReaderError Objects ^^^^^^^^^^^^^^^^^^^^^^ .. class:: PCMReaderError(error_message, sample_rate, channels, channel_mask, bits_per_sample) This is a subclass of :class:`PCMReader` which always returns empty always raises a :class:`ValueError` when its read method is called. The purpose of this is to postpone error generation so that all encoding errors, even those caused by unsuccessful decoding, are restricted to the :meth:`from_pcm` classmethod which can then propagate an :class:`EncodingError` error message to the user. PCMConverter Objects ^^^^^^^^^^^^^^^^^^^^ .. class:: PCMConverter(pcmreader, sample_rate, channels, channel_mask, bits_per_sample) This class takes an existing :class:`PCMReader`-compatible object along with a new set of ``sample_rate``, ``channels``, ``channel_mask`` and ``bits_per_sample`` values. Data from ``pcmreader`` is then automatically converted to the same format as those values. .. data:: PCMConverter.sample_rate If the new sample rate differs from ``pcmreader``'s sample rate, audio data is automatically resampled on each call to :meth:`read`. .. data:: PCMConverter.channels If the new number of channels is smaller than ``pcmreader``'s channel count, existing channels are removed or downmixed as necessary. If the new number of channels is larger, data from the first channel is duplicated as necessary to fill the rest. .. data:: PCMConverter.channel_mask If the new channel mask differs from ``pcmreader``'s channel mask, channels are removed as necessary such that the proper channel only outputs to the proper speaker. .. data:: PCMConverter.bits_per_sample If the new bits-per-sample differs from ``pcmreader``'s number of bits-per-sample, samples are shrunk or enlarged as necessary to cover the full amount of bits. .. method:: PCMConverter.read This method functions the same as the :meth:`PCMReader.read` method. .. method:: PCMConverter.close This method functions the same as the :meth:`PCMReader.close` method. BufferedPCMReader Objects ^^^^^^^^^^^^^^^^^^^^^^^^^ .. class:: BufferedPCMReader(pcmreader) This class wraps around an existing :class:`PCMReader` object. Its calls to :meth:`read` are guaranteed to return :class:`pcm.FrameList` objects as close to the requested amount of PCM frames as possible without going over by buffering data internally. The reason such behavior is not required is that we often don't care about the size of the individual FrameLists being passed from one routine to another. But on occasions when we need :class:`pcm.FrameList` objects to be of a particular size, this class can accomplish that. CounterPCMReader Objects ^^^^^^^^^^^^^^^^^^^^^^^^ .. class:: CounterPCMReader(pcmreader) This class wraps around an existing :class:`PCMReader` object and keeps track of the number of bytes and frames written upon each call to ``read``. .. attribute:: CounterPCMReader.frames_written The number of PCM frames written thus far. .. method:: CounterPCMReader.bytes_written() The number of bytes written thus far. ReorderedPCMReader Objects ^^^^^^^^^^^^^^^^^^^^^^^^^^ .. class:: ReorderedPCMReader(pcmreader, channel_order) This class wraps around an existing :class:`PCMReader` object. It takes a list of channel number integers (which should be the same as ``pcmreader``'s channel count) and reorders channels upon each call to :meth:`read`. For example, to swap channels 0 and 1 in a stereo stream, one could do the following: >>> reordered = ReorderedPCMReader(original, [1, 0]) Calls to ``reordered.read()`` will then have the left channel on the right side and vice versa. PCMCat Objects ^^^^^^^^^^^^^^ .. class:: PCMCat(pcmreaders) This class wraps around a list of :class:`PCMReader` objects and concatenates their output into a single output stream. If any of the readers has different attributes from the first reader in the stream, :exc:`ValueError` is raised at init-time. PCMReaderWindow Objects ^^^^^^^^^^^^^^^^^^^^^^^ .. class:: PCMReaderWindow(pcmreader, initial_offset, total_pcm_frames, [forward_close=True]) This class wraps around an existing :class:`PCMReader` object and truncates or extends its samples as needed. ``initial_offset``, if positive, indicates how many PCM frames to truncate from the beginning of the stream. If negative, the beginning of the stream is padded by that many PCM frames - all of which have a value of 0. ``total_pcm_frames`` indicates the total length of the stream as a non-negative number of PCM frames. If shorter than the actual length of the PCM reader's stream, the reader is truncated. If longer, the stream is extended by as many PCM frames as needed. Again, padding frames have a value of 0. If ``forward_close`` is True, calls to :meth:`PCMReaderWindow.close` are passed along to the wrapped :class:`PCMReader` object. Otherwise, the close is confined to the :class:`PCMReaderWindow` object. This may be necessary when encoding sub-streams from a larger stream in which closing the larger stream after each encode isn't desirable. LimitedPCMReader Objects ^^^^^^^^^^^^^^^^^^^^^^^^ .. class:: LimitedPCMReader(buffered_pcmreader, total_pcm_frames) This class wraps around an existing :class:`BufferedPCMReader` and ensures that no more than ``total_pcm_frames`` will be read from that stream by limiting reads to it. .. note:: :class:`PCMReaderWindow` is designed primarily for handling sample offset values in a :class:`CDTrackReader`, or for skipping a potentially large number of samples in a stream. :class:`LimitedPCMReader` is designed for splitting a stream into several smaller streams without losing any PCM frames. Which to use for a given situation depends on whether one cares about consuming the samples outside of the sub-reader or not. PCMReaderProgress Objects ^^^^^^^^^^^^^^^^^^^^^^^^^ .. class:: PCMReaderProgress(pcmreader, total_frames, progress) This class wraps around an existing :class:`PCMReader` object and generates periodic updates to a given ``progress`` function. ``total_frames`` indicates the total number of PCM frames in the PCM stream. >>> progress_display = SingleProgressDisplay(Messenger("audiotools"), u"encoding file") >>> pcmreader = source_audiofile.to_pcm() >>> source_frames = source_audiofile.total_frames() >>> target_audiofile = AudioType.from_pcm("target_filename", ... PCMReaderProgress(pcmreader, ... source_frames, ... progress_display.update)) ReplayGainCalculator Objects ---------------------------- .. class:: ReplayGainCalculator(sample_rate) This class is for incrementally calculating ReplayGain for an album during decoding. All tracks calculated must have the same sample rate which may mean resampling them to that rate if necessary. .. method:: ReplayGainCalculator.to_pcm(pcmreader) Given a :class:`PCMReader` object, returns a :class:`ReplayGainCalculatorReader` linked to this calculator. .. method:: ReplayGainCalculator.__iter__() Yields ``(title_gain, title_peak, album_gain, album_peak)`` tuples for each :class:`PCMReader` processed with :meth:`ReplayGainCalculator.to_pcm` in the order in which they were processed. ReplayGainCalculatorReader Objects ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. class:: ReplayGainCalculatorReader(replaygain, pcmreader) These objects are typically returned from :meth:`ReplayGainCalculator.to_pcm` rather than instantiated directly. They function as :class:`PCMReader` objects which process the :class:`pcm.FrameList` objects returned by ``read`` prior to returning them. .. method:: ReplayGainCalculatorReader.title_gain() Returns the title gain of the whole track as a floating point value. .. method:: ReplayGainCalculatorReader.title_peak() Returns the title peak of the whole track as a floating point value. ChannelMask Objects ------------------- .. class:: ChannelMask(mask) This is an integer-like class that abstracts channel assignments into a set of bit fields. ======= ========================= Mask Speaker ------- ------------------------- 0x1 ``front_left`` 0x2 ``front_right`` 0x4 ``front_center`` 0x8 ``low_frequency`` 0x10 ``back_left`` 0x20 ``back_right`` 0x40 ``front_left_of_center`` 0x80 ``front_right_of_center`` 0x100 ``back_center`` 0x200 ``side_left`` 0x400 ``side_right`` 0x800 ``top_center`` 0x1000 ``top_front_left`` 0x2000 ``top_front_center`` 0x4000 ``top_front_right`` 0x8000 ``top_back_left`` 0x10000 ``top_back_center`` 0x20000 ``top_back_right`` ======= ========================= All channels in a :class:`pcm.FrameList` will be in RIFF WAVE order as a sensible convention. But which channel corresponds to which speaker is decided by this mask. For example, a 4 channel PCMReader with the channel mask ``0x33`` corresponds to the bits ``00110011`` Reading those bits from right to left (least significant first) the ``front_left``, ``front_right``, ``back_left``, ``back_right`` speakers are set. Therefore, the PCMReader's 4 channel FrameLists are laid out as follows: 0. ``front_left`` 1. ``front_right`` 2. ``back_left`` 3. ``back_right`` Since the ``front_center`` and ``low_frequency`` bits are not set, those channels are skipped in the returned FrameLists. Many formats store their channels internally in a different order. Their :class:`PCMReader` objects will be expected to reorder channels and set a :class:`ChannelMask` matching this convention. And, their :func:`from_pcm` classmethods will be expected to reverse the process. A :class:`ChannelMask` of 0 is "undefined", which means that channels aren't assigned to *any* speaker. This is an ugly last resort for handling formats where multi-channel assignments aren't properly defined. In this case, a :func:`from_pcm` classmethod is free to assign the undefined channels any way it likes, and is under no obligation to keep them undefined when passing back out to :meth:`to_pcm` .. method:: ChannelMask.defined() Returns ``True`` if this mask is defined. .. method:: ChannelMask.undefined() Returns ``True`` if this mask is undefined. .. method:: ChannelMask.channels() Returns the speakers this mask contains as a list of strings in the order they appear in the PCM stream. .. method:: ChannelMask.index(channel_name) Given a channel name string, returns the index of that channel within the PCM stream. For example: >>> mask = ChannelMask(0xB) #fL, fR, LFE, but no fC >>> mask.index("low_frequency") 2 .. classmethod:: ChannelMask.from_fields(**fields) Takes channel names as function arguments and returns a :class:`ChannelMask` object. >>> mask = ChannelMask.from_fields(front_right=True, ... front_left=True, ... front_center=True) >>> int(mask) 7 .. classmethod:: ChannelMask.from_channels(channel_count) Takes a channel count integer and returns a :class:`ChannelMask` object. .. warning:: :func:`from_channels` *only* works for 1 and 2 channel counts and is meant purely as a convenience method for mono or stereo streams. All other values will trigger a :exc:`ValueError` CD Lookups ^^^^^^^^^^ .. function:: metadata_lookup(musicbrainz_disc_id, freedb_disc_id, [musicbrainz_server], [musicbrainz_port], [freedb_server], [freedb_port], [use_musicbrainz], [use_freedb]) Given a :class:`audiotools.musicbrainz.DiscID` and :class:`audiotools.freedb.DiscID`, returns a ``metadata[c][t]`` list of lists where ``c`` is a possible choice and ``t`` is the :class:`MetaData` for a given track (starting from 0). This will always return a list of :class:`MetaData` objects for at least one choice. In the event that no matches for the CD can be found, those objects will only contain ``track_number`` and ``track_total`` fields. .. function:: cddareader_metadata_lookup(cddareader, [musicbrainz_server], [musicbrainz_port], [freedb_server], [freedb_port], [use_musicbrainz], [use_freedb]) Given a :class:`cdio.CDDAReader` object, returns ``metadata[c][t]`` list of lists where ``c`` is a possible choice and ``t`` is the :class:`MetaData` for a given track (starting from 0). This will always return a list of :class:`MetaData` objects for at least one choice. In the event that no matches for the CD can be found, those objects will only contain ``track_number`` and ``track_total`` fields. .. function:: track_metadata_lookup(audiofiles, [, musicbrainz_server][, musicbrainz_port][, freedb_server][, freedb_port][, use_musicbrainz][, use_freedb]) Given a sorted list of :class:`AudioFile` objects, returns ``metadata[c][t]`` list of lists where ``c`` is a possible choice and ``t`` is the :class:`MetaData` for a given track (starting from 0). This will always return a list of :class:`MetaData` objects for at least one choice. In the event that no matches for the CD can be found, those objects will only contain ``track_number`` and ``track_total`` fields. .. function:: sheet_metadata_lookup(sheet, total_pcm_frames, sample_rate, [, musicbrainz_server][, musicbrainz_port][, freedb_server][, freedb_port][, use_musicbrainz][, use_freedb]) Given a :class:`Sheet` object, total number of PCM frames and the disc's sample rate, returns ``metadata[c][t]`` list of lists where ``c`` is a possible choice and ``t`` is the :class:`MetaData` for a given track (starting from 0). This will always return a list of :class:`MetaData` objects for at least one choice. In the event that no matches for the CD can be found, those objects will only contain ``track_number`` and ``track_total`` fields. .. function:: accuraterip_lookup(sorted_tracks[, server][, port]) Given a list of :class:`AudioFile` objects sorted by track number, returns a ``{track_number:[(confidence, checksum, alt), ...], ...}`` dict of values retrieved from the AccurateRip database where ``track_number`` is an int starting from 1, ``confidence`` is the number of people with the same people with a matching ``checksum`` of the track. May return a dict of empty lists if no AccurateRip entry is found. May return :exc:`urllib2.HTTPError` if an error occurs querying the server. .. function:: accuraterip_sheet_lookup(sheet, total_pcm_frames, sample_rate[, server][, port]) Given a :class:`Sheet` object, total number of PCM frames and sample rate, returns a ``{track_number:[(confidence, checksum, alt), ...], ...}`` dict of values retrieved from the AccurateRip database where ``track_number`` is an int starting from 1, ``confidence`` is the number of people with the same people with a matching ``checksum`` of the track. May return a dict of empty lists if no AccurateRip entry is found. May return :exc:`urllib2.HTTPError` if an error occurs querying the server. Cuesheets --------- Sheet Objects ^^^^^^^^^^^^^ These objects represent a CDDA layout such as provided by a ``.cue`` or ``.toc`` file. This can be used to recreate the exact layout of the disc when burning a set of tracks back to CD. .. class:: Sheet(sheet_tracks, [metadata]) ``sheet_tracks`` is a list of :class:`SheetTrack` objects, one per track on the CD. ``metadata`` is a :class:`MetaData` object or None .. classmethod:: Sheet.converted(sheet) Given a :class:`Sheet`-compatible object, returns a :class:`Sheet` object. .. classmethod:: Sheet.from_cddareader(cddareader, [filename]) Given a :class:`cdio.CDDAReader` object, returns a :class:`Sheet` object. .. method:: Sheet.__len__() Returns the number of tracks in the sheet. .. method:: Sheet.__getitem__(track_index) Given a track index (starting from 0), returns a :class:`SheetTrack` object. Raises :exc:`IndexError` if the track cannot be found. .. method:: Sheet.track_numbers() Returns a list of all track numbers in the sheet. .. method:: Sheet.track(track_number) Given a track number (often starting from 1), returns a :class:`SheetTrack` object with that number. Raises :exc:`KeyError` if the track cannot be found. .. method:: Sheet.pre_gap() Returns the disc's pre-gap (the amount of empty samples before the first track) as a :class:`Fraction` number of seconds. This number is often zero. .. method:: Sheet.track_offset(track_number) Given a track number (often starting from 1), returns the offset to that track as a :class:`Fraction` number of seconds. May raise :exc:`KeyError` if the track cannot be found. .. method:: Sheet.track_length(track_number, [total_length]) Given a track number (often starting from 1), and optional total length of the disc (including pre-gap) as a :class:`Fraction` number of seconds, returns the length of that track as a :class:`Fraction` number of seconds. May return ``None`` if the track is to use the remainder of the samples in the stream. This is typical of the last track in an album. May raise :exc:`KeyError` if the track cannot be found. .. method:: Sheet.get_metadata() Returns a :class:`MetaData` object containing metadata for the entire sheet such as catalog number or CD-TEXT information. May return ``None`` if there is no such metadata. SheetTrack Objects ^^^^^^^^^^^^^^^^^^ These objects represent a track on a given cuesheet. .. class:: SheetTrack(number, track_indexes, [metadata], [filename], [is_audio=True], [pre_emphasis=False], [copy_permitted=False]) =============== ============ ====================================== argument type value --------------- ------------ -------------------------------------- number int track number, starting from 1 track_indexes [SheetIndex] list of SheetIndex objects metadata MetaData track's metadata, or None filename unicode track's filename on disc is_audio boolean whether track contains audio data pre_emphasis boolean whether track has pre-emphasis copy_permitted boolean whether copying is permitted =============== ============ ====================================== .. classmethod:: SheetTrack.converted(sheet_track) Given a :class:`SheetTrack`-compatible object, returns a :class:`SheetTrack`. .. method:: SheetTrack.__len__() Returns the number of :class:`SheetIndex` objects in the track. .. method:: SheetTrack.__getitem__(i) Given an index (starting from 0), returns a track's :class:`SheetIndex` object. Raises :exc:`IndexError` if the index cannot be found. .. method:: SheetTrack.indexes() Returns a list of all index numbers in the track. .. method:: SheetTrack.index(sheet_index) Given an index number (often starting from 1), returns a track's :class:`SheetIndex` object. Raises :exc:`KeyError` if the index is not present. .. method:: SheetTrack.number() Returns the track's number, typically starting from 1. .. method:: SheetTrack.get_metadata() Returns the track's metadata such as ISRC and CD-TEXT information as a :class:`MetaData` object. May return ``None`` if it has no metadata. .. method:: SheetTrack.filename() Returns the track's filename as a Unicode string. .. method:: SheetTrack.is_audio() Returns whether the track contains audio data. .. method:: SheetTrack.pre_emphasis() Returns whether the track has pre-emphasis. .. method:: SheetTrack.copy_permitted() Returns whether copying is permitted. SheetIndex Objects ^^^^^^^^^^^^^^^^^^ .. class:: SheetIndex(number, offset) ``number`` is the number of the index in the track, often starting from 1. A number of 0 indicates a pre-gap index. ``offset`` is the index's offset from the start of the stream as a :class:`Fraction` number of seconds. .. method:: SheetIndex.number() Returns the track's index as an integer. .. method:: SheetIndex.offset() Returns the index point's offset from the start of the stream as a :class:`Fraction` number of seconds. DVDAudio Objects ---------------- .. class:: DVDAudio(audio_ts_path[, device]) This class is used to access a DVD-Audio. It contains a collection of titlesets. Each titleset contains a list of :class:`audiotools.dvda.DVDATitle` objects, and each :class:`audiotools.dvda.DVDATitle` contains a list of :class:`audiotools.dvda.DVDATrack` objects. ``audio_ts_path`` is the path to the DVD-Audio's ``AUDIO_TS`` directory, such as ``/media/cdrom/AUDIO_TS``. ``device`` is the path to the DVD-Audio's mount device, such as ``/dev/cdrom``. For example, to access the 3rd :class:`DVDATrack` object of the 2nd :class:`DVDATitle` of the first titleset, one can simply perform the following: >>> track = DVDAudio(path)[0][1][2] .. note:: If ``device`` is indicated *and* the ``AUDIO_TS`` directory contains a ``DVDAUDIO.MKB`` file, unprotection will be performed automatically if supported on the user's platform. Otherwise, the files are assumed to be unprotected. ExecQueue Objects ----------------- .. class:: ExecQueue() This is a class for executing multiple Python functions in parallel across multiple CPUs. .. method:: ExecQueue.execute(function, args[, kwargs]) Queues a Python function, list of arguments and optional dictionary of keyword arguments. .. method:: ExecQueue.run([max_processes]) Executes all queued Python functions, running ``max_processes`` number of functions at a time until the entire queue is empty. This operates by forking a new subprocess per function, executing that function and then, regardless of the function's result, the child job performs an unconditional exit. This means that any side effects of executed functions have no effect on ExecQueue's caller besides those which modify files on disk (encoding an audio file, for example). .. class:: ExecQueue2() This is a class for executing multiple Python functions in parallel across multiple CPUs and receiving results from those functions. .. method:: ExecQueue2.execute(function, args[, kwargs]) Queues a Python function, list of arguments and optional dictionary of keyword arguments. .. method:: ExecQueue2.run([max_processes]) Executes all queued Python functions, running ``max_processes`` number of functions at a time until the entire queue is empty. Returns an iterator of the returned values of those functions. This operates by forking a new subprocess per function with a pipe between them, executing that function in the child process and then transferring the resulting pickled object back to the parent before performing an unconditional exit. Queued functions that raise an exception or otherwise exit uncleanly yield ``None``. Likewise, any side effects of the called function have no effect on ExecQueue's caller. ExecProgressQueue Objects ------------------------- .. class:: ExecProgressQueue(messenger) This class runs multiple jobs in parallel and displays their progress output to the given :class:`Messenger` object. .. method:: ExecProgressQueue.execute(function[, progress_text[, completion_output[, *args[, **kwargs]]]]) Queues a Python function for execution. This function is passed the optional ``args`` and ``kwargs`` arguments upon execution. However, this function is also passed an *additional* ``progress`` keyword argument which is a function that takes the current progress value as a :class:`Fraction` object between 0 and 1, inclusive. The executed function can then call that ``progress`` function at regular intervals to indicate its progress. If given, ``progress_text`` is a Unicode string to be displayed while the function is being executed. ``completion_output`` is displayed once the executed function is completed. It can be either a Unicode string or a function whose argument is the returned result of the executed function and which must output either a Unicode string or ``None``. If ``None``, no output text is generated for the completed job. .. method:: ExecProgressQueue.run([max_processes]) Executes all the queued functions, running ``max_processes`` number of functions at a time until the entire queue is empty. Returns the results of the called functions in the order in which they were added for execution. This operates by forking a new subprocess per function in which the running progress and function output are piped to the parent for display to the screen. If an exception occurs in one of the subprocesses, that exception will be raised by :meth:`ExecProgressQueue.run` and all the running jobs will be terminated. >>> def progress_function(progress, filename): ... # perform work here ... progress(Fraction(current, total)) ... # more work ... result.a = a ... result.b = b ... result.c = c ... return result ... >>> def format_result(result): ... return u"%s %s %s" % (result.a, result.b, result.c) ... >>> queue = ExecProgressQueue(ProgressDisplay(Messenger("executable"))) >>> queue.execute(function=progress_function, ... progress_text=u"%s progress" % (filename1), ... completion_output=format_result, ... filename=filename1) ... >>> queue.execute(function=progress_function, ... progress_text=u"%s progress" % (filename2), ... completion_output=format_result, ... filename=filename2) ... >>> queue.run() Messenger Objects ----------------- .. class:: Messenger(silent=False) This is a helper class for displaying program data, analogous to a primitive logging facility. If ``silent`` is ``True``, the output methods will not actually display any output to the screen. .. method:: Messenger.output(string) Outputs Unicode ``string`` to stdout and adds a newline, unless ``verbosity`` level is ``"silent"``. .. method:: Messenger.partial_output(string) Output Unicode ``string`` to stdout and flushes output so it is displayed, but does not add a newline. Does nothing if ``verbosity`` level is ``"silent"``. .. method:: Messenger.info(string) Outputs Unicode ``string`` to stdout and adds a newline, unless ``verbosity`` level is ``"silent"``. .. method:: Messenger.partial_info(string) Output Unicode ``string`` to stdout and flushes output so it is displayed, but does not add a newline. Does nothing if ``verbosity`` level is ``"silent"``. .. note:: What's the difference between :meth:`Messenger.output` and :meth:`Messenger.info`? :meth:`Messenger.output` is for a program's primary data. :meth:`Messenger.info` is for incidental information. For example, trackinfo uses :meth:`Messenger.output` for what it displays since that output is its primary function. But track2track uses :meth:`Messenger.info` for its lines of progress since its primary function is converting audio and tty output is purely incidental. .. method:: Messenger.warning(string) Outputs warning text, Unicode ``string`` and a newline to stderr, unless ``verbosity`` level is ``"silent"``. >>> m = audiotools.Messenger() >>> m.warning(u"Watch Out!") *** Warning: Watch Out! .. method:: Messenger.error(string) Outputs error text, Unicode ``string`` and a newline to stderr. >>> m.error(u"Fatal Error!") *** Error: Fatal Error! .. method:: Messenger.os_error(oserror) Given an :class:`OSError` object, displays it as a properly formatted error message with an appended newline. .. note:: This is necessary because of the way :class:`OSError` handles its embedded filename string. Using this method ensures that filename is properly encoded when displayed. Otherwise, there's a good chance that non-ASCII filenames will be garbled. .. method:: Messenger.output_isatty() Returns ``True`` if the output method sends to a TTY rather than a file. .. method:: Messenger.info_isatty() Returns ``True`` if the info method sends to a TTY rather than a file. .. method:: Messenger.error_isatty() Returns ``True`` if the error method sends to a TTY rather than a file. .. method:: Messenger.ansi_clearline() Generates a ANSI escape codes to clear the current line. This works only if ``stdout`` is a TTY, otherwise is does nothing. >>> msg = Messenger() >>> msg.partial_output(u"working") >>> time.sleep(1) >>> msg.ansi_clearline() >>> msg.output(u"done") .. method:: Messenger.ansi_uplines(self, lines) Moves the cursor upwards by the given number of lines. .. method:: Messenger.ansi_cleardown(self) Clears all the output below the current line. This is typically used in conjunction with :meth:`Messenger.ansi_uplines`. >>> msg = Messenger() >>> msg.output(u"line 1") >>> msg.output(u"line 2") >>> msg.output(u"line 3") >>> msg.output(u"line 4") >>> time.sleep(2) >>> msg.ansi_uplines(4) >>> msg.ansi_cleardown() >>> msg.output(u"done") .. method:: Messenger.terminal_size(fd) Given a file descriptor integer, or file object with a fileno() method, returns the size of the current terminal as a (``height``, ``width``) tuple of integers. ProgressDisplay Objects ----------------------- .. class:: ProgressDisplay(messenger) This is a class for displaying incremental progress updates to the screen. It takes a :class:`Messenger` object which is used for generating output. Whether or not :attr:`sys.stdout` is a TTY determines how this class operates. If a TTY is detected, screen updates are performed incrementally with individual rows generated and refreshed as needed using ANSI escape sequences such that the user's screen need not scroll. If a TTY is not detected, most progress output is omitted. .. method:: ProgressDisplay.add_row(output_line) ``output_line`` is a Unicode string indicating what we're displaying the progress of. Returns a :class:`ProgressRow` object which can be updated with the current progress for display. .. method:: ProgressDisplay.remove_row(row_index) Removes the given row index and frees the slot for reuse. .. method:: ProgressDisplay.display_rows() Outputs the current state of all progress rows. .. method:: ProgressDisplay.clear_row() Clears all previously displayed output rows, if any. .. class:: ProgressRow(progress_display, row_index, output_line) This is used by :class:`ProgressDisplay` and its subclasses for actual output generation. ``progress_display`` is a parent :class:`ProgressDisplay` object. ``row_index`` is this row's index on the screen. ``output_line`` is a unicode string. It is not typically instantiated directly. .. method:: ProgressRow.update(progress) Updates the current progress as a :class:`Fraction` between 0 and 1, inclusive. .. method:: ProgressRow.finish() Indicate output is finished and the row will no longer be needed. .. method:: ProgressRow.unicode(width) Returns the output line and its current progress as a Unicode string, formatted to the given width in onscreen characters. Screen width can be determined from the :meth:`Messenger.terminal_size` method. .. class:: SingleProgressDisplay(messenger, progress_text) This is a subclass of :class:`ProgressDisplay` used for generating only a single line of progress output. As such, one only specifies a single row of Unicode ``progress_text`` at initialization time and can avoid the row management functions entirely. .. method:: SingleProgressDisplay.update(progress) Updates the status of our output row with the current progress, which is a :class:`Fraction` between 0 and 1, inclusive. .. class:: ReplayGainProgressDisplay(messenger, lossless_replay_gain) This is another :class:`ProgressDisplay` subclass optimized for the display of ReplayGain application progress. ``messenger`` is a :class:`Messenger` object and ``lossless_replay_gain`` is a boolean indicating whether ReplayGain is being applied losslessly or not (which can be determined from the :meth:`AudioFile.lossless_replay_gain` classmethod). Whether or not :attr:`sys.stdout` is a TTY determines how this class behaves. .. method:: ReplayGainProgressDisplay.initial_message() If operating on a TTY, this does nothing since progress output will be displayed. Otherwise, this indicates that ReplayGain application has begun. .. method:: ReplayGainProgressDisplay.update(progress) Updates the status of ReplayGain application with the current progress which is a :class:`Fraction` between 0 and 1, inclusive. .. method:: ReplayGainProgressDisplay.final_message() If operating on a TTY, this indicates that ReplayGain application is complete. Otherwise, this does nothing. >>> rg_progress = ReplayGainProgressDisplay(messenger, AudioType.lossless_replay_gain()) >>> rg_progress.initial_message() >>> AudioType.add_replay_gain(filename_list, rg_progress.update) >>> rg_Progress.final_message() output_text Objects ^^^^^^^^^^^^^^^^^^^ This class is for displaying portions of a Unicode string to the screen and applying formatting such as color via ANSI escape sequences. The reason this is needed is because not all Unicode characters are the same width when displayed to the screen. So, for example, if one wishes to display a portion of a Unicode string to a screen that's 80 ASCII characters wide, one can't simply perform: >>> messenger.output(unicode_string[0:80]) since some of those Unicode characters might be double width, which would cause the string to wrap. .. class:: output_text(unicode_string[, fg_color][, bg_color][, style]) ``unicode_string`` is the text to display. ``fg_color`` and ``bg_color`` may be one of ``"black"``, ``"red"``, ``"green"``, ``"yellow"``, ``"blue"``, ``"magenta"``, ``"cyan"``, or ``"white"``. ``style`` may be one of ``"bold"``, ``"underline"``, ``"blink"`` or ``"inverse"``. .. method:: output_text.__unicode__ Returns the raw Unicode string. .. method:: output_text.__len__ Returns the width of the text in displayed characters. .. method:: output_text.fg_color() Returns the foreground color as a string, or ``None``. .. method:: output_text.bg_color() Returns the background color as a string, or ``None``. .. method:: output_text.style() Returns the style as a string, or ``None``. .. method:: output_text.set_format([fg_color][, bg_color][, style]) Returns a new :class:`output_text` object with the given styles. .. method:: output_text.format([is_tty]) If formatting is present and ``is_tty`` is ``True``, returns a Unicode string with ANSI escape sequences applied. Otherwise, returns the Unicode string with no ANSI formatting. .. method:: output_text.head(display_characters) Returns a new :class:`output_text` object that's been truncated up to the given number of display characters, but may return less. >>> s = u"".join(map(unichr, range(0x30a1, 0x30a1+25))) >>> len(s) 25 >>> u = unicode(output_text(s).head(40)) >>> len(u) 20 >>> print repr(u) u'\u30a1\u30a2\u30a3\u30a4\u30a5\u30a6\u30a7\u30a8\u30a9\u30aa\u30ab\u30ac\u30ad\u30ae\u30af\u30b0\u30b1\u30b2\u30b3\u30b4' .. note:: Because some characters are double-width, this method along with :meth:`output_text.tail` and :meth:`output_text.split` may not return strings that are the same length as requested if the dividing point in the middle of a character. .. method:: output_text.tail(display_characters) Returns a new :class:`output_text` object that's been truncated up to the given number of display characters. >>> s = u"".join(map(unichr, range(0x30a1, 0x30a1+25))) >>> len(s) 25 >>> u = unicode(output_text(s).tail(40)) >>> len(u) 20 >>> print repr(u) u'\u30a6\u30a7\u30a8\u30a9\u30aa\u30ab\u30ac\u30ad\u30ae\u30af\u30b0\u30b1\u30b2\u30b3\u30b4\u30b5\u30b6\u30b7\u30b8\u30b9' .. method:: output_text.split(display_characters) Returns a tuple of :class:`output_text` objects. The first is up to ``display_characters`` wide, while the second contains the remainder. >>> s = u"".join(map(unichr, range(0x30a1, 0x30a1+25))) >>> (head, tail) = display_unicode(s).split(40) >>> print repr(unicode(head)) u'\u30a1\u30a2\u30a3\u30a4\u30a5\u30a6\u30a7\u30a8\u30a9\u30aa\u30ab\u30ac\u30ad\u30ae\u30af\u30b0\u30b1\u30b2\u30b3\u30b4' >>> print repr(unicode(tail)) u'\u30b5\u30b6\u30b7\u30b8\u30b9' .. method:: output_text.join(output_texts) Given an iterable collection of :class:`output_text` objects, returns an :class:`output_list` joined by our formatted text. output_list Objects ^^^^^^^^^^^^^^^^^^^ output_list is an :class:`output_text` subclass for formatting multiple :class:`output_text` objects as a unit. .. class:: output_list(output_texts[, fg_color][, bg_color][, style]) ``output_texts`` is an iterable collection of :class:`output_text` or unicode objects. ``fg_color`` and ``bg_color`` may be one of ``"black"``, ``"red"``, ``"green"``, ``"yellow"``, ``"blue"``, ``"magenta"``, ``"cyan"``, or ``"white"``. ``style`` may be one of ``"bold"``, ``"underline"``, ``"blink"`` or ``"inverse"``. .. warning:: Formatting is unlikely to nest properly since ANSI is un-escaped to the terminal default. Therefore, if the :class:`output_list` has formatting, its contained :class:`output_text` objects should not have formatting. Or if the :class:`output_text` objects do have formatting, the :class:`output_list` container should not have formatting. .. method:: output_list.fg_color() Returns the foreground color as a string, or ``None``. .. method:: output_list.bg_color() Returns the background color as a string, or ``None``. .. method:: output_list.style() Returns the style as a string, or ``None``. .. method:: output_list.set_format([fg_color][, bg_color][, style]) Returns a new :class:`output_list` object with the given formatting. .. method:: output_list.format([is_tty]) If formatting is present and ``is_tty`` is ``True``, returns a Unicode string with ANSI escape sequences applied. Otherwise, returns the Unicode string with no ANSI formatting. .. method:: output_list.head(display_characters) Returns a new :class:`output_list` object that's been truncated up to the given number of display characters, but may return less. .. method:: output_list.tail(display_characters) Returns a new :class:`output_list` object that's been truncated up to the given number of display characters. .. method:: output_list.split(display_characters) Returns a tuple of :class:`output_text` objects. The first is up to ``display_characters`` wide, while the second contains the remainder. .. method:: output_list.join(output_texts) Given an iterable collection of :class:`output_text` objects, returns an :class:`output_list` joined by our formatted text. output_table Objects ^^^^^^^^^^^^^^^^^^^^ output_table is for formatting text into rows and columns. .. class:: output_table() .. method:: output_table.row() Adds new row to table and returns :class:`output_table_row` object which columns may be added to. .. method:: output_table.blank_row() Adds empty row to table whose columns will be blank. .. method:: output_table.divider_row(dividers) Takes a list of Unicode characters, one per column, and generates a row which will expand those characters as needed to fill each column. .. method:: output_table.format([is_tty]) Yields one formatted Unicode string per row. If ``is_tty`` is ``True``, rows may contain ANSI escape sequences for color and style. output_table_row Objects ^^^^^^^^^^^^^^^^^^^^^^^^ output_table_row is a container for table columns and is returned from :meth:`output_table.row()` rather than instantiated directly. .. class:: output_table_row() .. method:: output_table_row.__len__() Returns the total number of columns in the table. .. method:: output_table_row.add_column(text[, alignment="left"][, colspan=1]) Adds text, which may be a Unicode string or :class:`output_text` object. ``alignment`` may be ``"left"``, ``"center"`` or ``"right"``. If `colspan` is greater than 1, the column is widened to span that many other non-spanning columns. .. method:: output_table_row.column_width(column) Returns the width of the given column in printable characters. .. method:: output_table_row.format(column_widths[, is_tty]) Given a list of column widths, returns the table row as a Unicode string such that each column is padded to the corresponding width depending on its alignment. If ``is_tty`` is ``True``, columns may contain ANSI escape sequences for color and style. Exceptions ---------- .. exception:: UnknownAudioType Raised by :func:`filename_to_type` if the file's suffix is unknown. .. exception:: AmbiguousAudioType Raised by :func:`filename_to_type` if the file's suffix applies to more than one audio class. .. exception:: DecodingError Raised by :class:`PCMReader`'s .close() method if a helper subprocess exits with an error, typically indicating a problem decoding the file. .. exception:: DuplicateFile Raised by :func:`open_files` if the same file is included more than once and ``no_duplicates`` is indicated. .. exception:: DuplicateOutputFile Raised by :func:`audiotools.ui.process_output_options` if the same output file is generated more than once. .. exception:: EncodingError Raised by :meth:`AudioFile.from_pcm` and :meth:`AudioFile.convert` if an error occurs when encoding an input file. This includes errors from the input stream, a problem writing the output file in the given location, or EncodingError subclasses such as :exc:`UnsupportedBitsPerSample` if the input stream is formatted in a way the output class does not support. .. exception:: InvalidFile Raised by :meth:`AudioFile.__init__` if the file is invalid in some way. .. exception:: InvalidFilenameFormat Raised by :meth:`AudioFile.track_name` if the format string contains broken fields. .. exception:: InvalidImage Raised by :meth:`Image.new` if the image cannot be parsed correctly. .. exception:: OutputFileIsInput Raised by :func:`process_output_options` if an output file is the same as any of the input files. .. exception:: SheetException A parent exception of :exc:`audiotools.cue.CueException` and :exc:`audiotools.toc.TOCException`, to be raised by :func:`read_sheet` if a .toc or .cue file is unable to be parsed correctly. .. exception:: UnsupportedBitsPerSample Subclass of :exc:`EncodingError`, indicating the input stream's bits-per-sample is not supported by the output class. .. exception:: UnsupportedChannelCount Subclass of :exc:`EncodingError`, indicating the input stream's channel count is not supported by the output class. .. exception:: UnsupportedChannelMask Subclass of :exc:`EncodingError`, indicating the input stream's channel mask is not supported by the output class. .. exception:: UnsupportedFile Raised by :func:`open` if the given file is not something identifiable, or we do not have the installed binaries to support. .. exception:: UnsupportedTracknameField Raised by :meth:`AudioFile.track_name` if a track name field is not supported. ================================================ FILE: docs/programming/source/audiotools_accuraterip.rst ================================================ .. Audio Tools, a module and set of tools for manipulating audio data Copyright (C) 2007-2016 Brian Langenberger This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA :mod:`audiotools.accuraterip` --- AccurateRip Lookup Service Module =================================================================== .. module:: audiotools.accuraterip :synopsis: a Module for Accessing AccurateRip Information The :mod:`audiotools.accuraterip` module contains classes and functions for performing lookups to the AccurateRip service. ChecksumV1 Objects ------------------ .. class:: ChecksumV1(total_pcm_frames [, sample_rate=44100][, is_first=False][, is_last=False][, pcm_frame_range=1]) A class for calculating AccurateRip's version 1 checksum. ``total_pcm_frames`` is the length of the track to be checksummed. ``sample_rate``, ``is_first`` and ``is_last`` are used to calculate the checksum properly at the beginning, middle and end of the disc. ``pcm_frame_range`` is used to calculate the checksum over a window's worth of values. .. note:: The total number of PCM frames expected by ChecksumV1 equals ``total_pcm_frames`` + ``pcm_frame_range`` - 1. For example, given ``total_pcm_frames`` of 100 and a ``pcm_frame_range`` of 3, one will populate the :class:`ChecksumV1` object with 102 PCM frames and receive 3 checksum values. The first checksum is for ``frames[0:100]``, the second for ``frames[1:101]`` and the third for ``frames[2:102]``. The purpose of this is to determine whether a track has been ripped accurately, but its samples are simply shifted by some positive or negative number of samples. .. method:: ChecksumV1.update(framelist) Updates the checksum in progress with the given :class:`audiotools.pcm.FrameList` object. May raise :exc:`ValueError` if too many PCM frames are given to process. .. method:: ChecksumV1.checksums() Returns a list of 32-bit AccurateRip checksums, 1 per ``pcm_frame_range``. May raise :exc:`ValueError` if not enough PCM frames have been processed. Disc ID Objects --------------- .. class:: DiscID(track_numbers, track_offsets, lead_out_offset, freedb_disc_id) An AccurateRip disc ID object used to perform lookups. ``track_numbers`` is a list of track numbers, starting from 1. ``track_offsets`` is a list of track offsets in CD sectors, *not* including the 2 second lead-in. ``lead_out_offset`` is the lead-out sector of the CD, *not* including the 2 second lead-in. ``freedb_disc_id`` is a string or :class:`audiotools.freedb.DiscID` object of the disc's FreeDB disc ID. .. method:: DiscID.__str__() Returns the disc ID as a 39 character string that AccurateRip expects when performing lookups. .. classmethod:: DiscID.from_cddareader(cddareader) Given a :class:`audiotools.cdio.CDDAReader` object, returns the :class:`DiscID` of that disc. .. classmethod:: DiscID.from_tracks(tracks) Given a sorted list of :class:`audiotools.AudioFile` objects, returns the :class:`DiscID` as if those tracks were a CD. .. warning:: This assumes all the tracks from the disc are present and are laid out in a conventional fashion with no "hidden" tracks or other oddities. The disc ID may not be accurate if that's not the case. .. classmethod:: DiscID.from_sheet(sheet, total_pcm_frames, sample_rate) Given a :class:`audiotools.Sheet` object along with the total length of the disc in PCM frames and the disc's sample rate (typically 44100), returns the :class:`DiscID`. Performing Lookup ----------------- .. function:: perform_lookup(disc_id[, accuraterip_server][, accuraterip_port]) Given a :class:`DiscID` object and optional AccurateRip hostname string and port, returns a dict of ``{track_number:[(confidence, crc, crc2), ...], ...}`` where ``track_number`` starts from 1, ``crc`` is an AccurateRip checksum integer and ``confidence`` is an integer of the match's confidence level. May return a dict of empty lists if no AccurateRip entry is found. May raise :exc:`urllib2.HTTPError` if an error occurs querying the server. Determining Match Offset ------------------------ .. function:: match_offset(ar_matches, checksums, initial_offset) ``ar_matches`` is a dict of ``{track_number:[(confidence, crc, crc2), ...], ...}`` values returned by :func:`perform_lookup`. ``checksums`` is a list of checksum integers returned by :func:`ChecksumV1.checksums()`. ``initial_offset`` is the initial PCM frames offset of the checksums (which may be negative). Returns a ``(checksum, confidence, offset)`` tuple of the best match found. If no matches are found, the checksum at offset 0 is returned and ``confidence`` is ``None``. ================================================ FILE: docs/programming/source/audiotools_bitstream.rst ================================================ .. Audio Tools, a module and set of tools for manipulating audio data Copyright (C) 2007-2016 Brian Langenberger This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA :mod:`audiotools.bitstream` --- the Bitstream Module ==================================================== .. module:: audiotools.bitstream :synopsis: the Bitstream Module The :mod:`audiotools.bitstream` module contains objects for parsing binary data. Unlike Python's built-in struct module, these routines are specialized to handle data that's not strictly byte-aligned. .. function:: format_size(format_string) Given a format string as used by :meth:`BitstreamReader.parse` or :meth:`BitstreamWriter.build`, returns the size of that string as an integer number of bits that would be read from or written to the stream. >>> format_size("3u 4s 36U") 43 .. function:: parse(format_string, is_little_endian, data) Given a format string as used by :meth:`BitstreamReader.parse`, whether the data is little-endian, and a string of binary data, returns a list of values as would be returned by :meth:`BitstreamReader.parse`. This is roughly equivalent to: >>> return BitstreamReader(StringIO(data), is_little_endian).parse(format_string) .. function:: build(format_string, is_little_endian, values) Given a format string as used by :meth:`BitstreamWriter.build`, whether the data is little-endian, and a sequence of Python values, returns the binary string as would be returned by :meth:`BitstreamWriter.build`. This is roughly equivalent to >>> s = StringIO() >>> BitstreamWriter(s, is_little_endian).build(format_string, values) >>> return s BitstreamReader Objects ----------------------- This is a file-like object for pulling individual bits or bytes out of a larger binary file stream. .. warning:: BitstreamReaders process the given file object in chunks of the given buffer size. This means the position of the file is likely to be further along than one might expect given the number of bits already read. The BitstreamReader's getpos, setpos and seek methods will handle buffering correctly and are preferable to intermingling BitstreamReader and ``file`` operations. .. class:: BitstreamReader(file, is_little_endian[, buffer_size=4096]) ``file`` may be a regular file object, a file-like object with ``read`` and ``close`` methods, or a plain string. When operating on a raw file object (such as one opened with :func:`open`) this uses a single byte buffer. This allows the underlying file to be seeked safely whenever the :class:`BitstreamReader` is byte-aligned. However, when operating on a Python-based file object (with :func:`read` and :func:`close` methods) this uses an internal string up to ``buffer_size`` bytes large in order to minimize Python function calls. ``is_little_endian`` indicates which endianness format to use when consuming bits. ``True`` for big-endian streams, ``False`` for little-endian. .. method:: BitstreamReader.read(bits) Given a number of bits to read from the stream, returns an unsigned integer. May raise :exc:`IOError` if an error occurs reading the stream. .. method:: BitstreamReader.read_signed(bits) Given a number of bits to read from the stream as a two's complement value, returns a signed integer. May raise :exc:`IOError` if an error occurs reading the stream. .. method:: BitstreamReader.skip(bits) Skips the given number of bits in the stream as if read. May raise :exc:`IOError` if an error occurs reading the stream. .. method:: BitstreamReader.skip_bytes(bytes) Skips the given number of bytes in the stream as if read. May raise :exc:`IOError` if an error occurs reading the stream. .. method:: BitstreamReader.unary(stop_bit) Reads the number of bits until the next ``stop_bit``, which must be ``0`` or ``1``. Returns that count as an unsigned integer. May raise :exc:`IOError` if an error occurs reading the stream. .. method:: BitstreamReader.skip_unary(stop_bit) Skips a number of bits until the next ``stop_bit``, which must be ``0`` or ``1``. May raise :exc:`IOError` if an error occurs reading the stream. .. method:: BitstreamReader.byte_align() Discards bits as necessary to position the stream on a byte boundary. .. method:: BitstreamReader.byte_aligned() Returns ``True`` if the stream is positioned on a byte boundary. .. method:: BitstreamReader.parse(format_string) Given a format string representing a set of individual reads, returns a list of those reads. ====== ================ format method performed ====== ================ "#u" read(#) "#s" read_signed(#) "#p" skip(#) "#P" skip_bytes(#) "#b" read_bytes(#) "a" byte_align() ====== ================ For instance: >>> r.parse("3u 4s 36U") == [r.read(3), r.read_signed(4), r.read(36)] The ``*`` format multiplies the next format by the given amount. For example, to read 4, signed 8 bit values: >>> r.parse("4* 8s") == [r.read_signed(8) for i in range(4)] May raise :exc:`IOError` if an error occurs reading the stream. .. method:: BitstreamReader.read_huffman_code(huffman_tree) Given a :class:`HuffmanTree` object, returns the next Huffman code from the stream as defined in the tree. May raise :exc:`IOError` if an error occurs reading the stream. .. method:: BitstreamReader.unread_bit(bit) Pushes a single bit back onto the stream, which must be ``0`` or ``1``. Only a single bit is guaranteed to be unreadable. .. method:: BitstreamReader.read_bytes(bytes) Returns the given number of 8-bit bytes from the stream as a binary string. May raise :exc:`IOError` if an error occurs reading the stream. .. method:: set_endianness(is_little_endian) Sets the stream's endianness where ``False`` indicates big-endian, while ``True`` indicates little-endian. The stream is automatically byte-aligned prior to changing its byte order. .. method:: BitstreamReader.getpos() Returns a :class:`BitstreamReaderPosition` object of the stream's current position. May raise :exc:`IOError` if an error occurs getting the position. .. method:: BitstreamReader.setpos(position) Given a :class:`BitstreamReaderPosition` object, sets the stream to that position. The position must be one returned by this object's :meth:`BitstreamReader.getpos` method; one cannot apply a position from one reader to a different one. May raise :exc:`IOError` if an error occurs setting the position. .. method:: BitstreamReader.seek(position, [whence]) Given an integer position value, positions the stream at the given byte relative to whence, which may be 0 for the beginning of the stream (the default), 1 for the current position and 2 for the stream end. .. method:: BitstreamReader.add_callback(callback) Adds a callable function to the stream's callback stack. ``callback(b)`` takes a single byte as an argument. This callback is called upon each byte read from the stream. If multiple callbacks are added, they are all called in reverse order. .. method:: BitstreamReader.call_callbacks(byte) Calls all the callbacks on the stream's callback stack with the given byte, as if it had been read from the stream. .. method:: BitstreamReader.pop_callback() Removes and returns the most recently added function from the callback stack. .. method:: BitstreamReader.substream(bytes) Returns a new :class:`BitstreamReader` object which contains ``bytes`` amount of data read from the current stream and defined with the current stream's endianness. May raise an :exc:`IOError` if the current stream has insufficient bytes. Any callbacks defined in the current stream are applied to the bytes read for the substream when this method is called. Any marks or callbacks in the current stream are *not* transferred to the substream. In all other respects, the substream acts like any other :class:`BitstreamReader`. However, attempting to have the substream read beyond its defined byte count will trigger :exc:`IOError` exceptions. .. method:: BitstreamReader.close() Closes the stream and any underlying file object, by calling its ``close`` method. .. method:: BitstreamReader.__enter__() Returns the reader's context manager. .. method:: BitstreamReader.__exit__(exc_type, exc_value, traceback) Exits the reader's context manager by calling :meth:`file.close` on the wrapped file object. If one wishes to keep the stream open for further reading, don't use a context manager and simply delete the reader object. But again, be aware that buffering may make its current position different than one might expect. BitstreamWriter Objects ----------------------- This is a file-like object for pushing individual bits or bytes into a larger binary file stream. .. warning:: BitstreamWriters process the given file object in chunks of the given buffer size. This means the position of the file is likely to be not as far along as one might expect given the number of bits already written. The BitstreamWriters's getpos and setpos methods will handle buffering correctly and are preferable to intermingling BitstreamWriter and ``file`` operations. .. class:: BitstreamWriter(file, is_little_endian[, buffer_size=4096]) When operating on a raw file object (such as one opened with :func:`open`) this uses a single byte buffer. This allows the underling file to be seeked safely whenever :class:`BitstreamWriter` is byte-aligned. However, when operating on a Python-based file object (with :func:`write` and :func:`close` methods) this uses an internal string up to ``buffer_size`` bytes large in order to minimize Python function calls. .. method:: BitstreamWriter.write(bits, value) Writes the given unsigned integer value to the stream using the given number of bits. May raise :exc:`IOError` if an error occurs writing the stream. .. method:: BitstreamWriter.write_signed(bits, value) Writes the given signed integer value to the stream using the given number of bits. May raise :exc:`IOError` if an error occurs writing the stream. .. method:: BitstreamWriter.unary(stop_bit, value) If ``stop_bit`` is ``1``, writes ``value`` number of ``0`` bits to the stream followed by a ``1`` bit. If ``stop_bit`` is ``0``, writes ``value`` number of ``1`` bits to the stream followed by a ``0`` bit. May raise :exc:`IOError` if an error occurs writing the stream. .. method:: BitstreamWriter.write_huffman_code(huffman_tree, value) Given a :class:`HuffmanTree` object and an integer value to write, determines the proper output code and writes it to disk. Raises :exc:`ValueError` if the integer value is not present in the tree. .. method:: BitstreamWriter.byte_align() Writes ``0`` bits as necessary until the stream is aligned on a byte boundary. May raise :exc:`IOError` if an error occurs writing the stream. .. method:: BitstreamWriter.byte_aligned() Returns ``True`` if the stream is positioned on a byte boundary. .. method:: BitstreamWriter.build(format_string, value_list) Given a format string representing a set of individual writes, and a list of values to write, performs those writes to the stream. ====== ============= ===================== format value method performed ====== ============= ===================== "#u" unsigned int write(#, u) "#s" signed int write(#, s) "#p" N/A write(#, 0) "#P" N/A write(# * 8, 0) "#b" string write_bytes(#, s) "a" N/A byte_align() ====== ============= ===================== For instance: >>> w.build("3u 4s 36U", [1, -2, 3L]) is equivalent to: >>> w.write(3,1) >>> w.write_signed(4, -2) >>> w.write(36, 3L) The ``*`` format multiplies the next format by the given amount. >>> r.build("4* 8s", [-2, -1, 0, 1]) is equivalent to: >>> w.write_signed(8, -2) >>> w.write_signed(8, -1) >>> w.write_signed(8, 0) >>> w.write_signed(8, 1) May raise :exc:`IOError` if an error occurs writing the stream. .. method:: BitstreamWriter.write_bytes(string) Writes the given binary string to the stream with a number of bytes equal to its length. May raise :exc:`IOError` if an error occurs writing the stream. .. method:: BitstreamWriter.flush() Flushes cached bytes to the stream. Partially written bytes are *not* flushed to the stream. May raise :exc:`IOError` if an error occurs writing the stream. .. method:: BitstreamWriter.set_endianness(is_little_endian) Sets the stream's endianness where ``False`` indicates big-endian, while ``True`` indicates little-endian. The stream is automatically byte-aligned prior to changing its byte order. .. method:: BitstreamWriter.add_callback(callback) Adds a callable function to the stream's callback stack. ``callback(b)`` takes a single byte as an argument. This callback is called upon each byte written to the stream. If multiple callbacks are added, they are all called in reverse order. .. method:: BitstreamWriter.call_callbacks(byte) Calls all the callbacks on the stream's callback stack with the given byte, as if it had been written to the stream. .. method:: BitstreamWriter.pop_callback() Removes and returns the most recently added function from the callback stack. .. method:: BitstreamWriter.getpos() Returns a :class:`BitstreamWriterPosition` object of the stream's current position. May raise :exc:`IOError` if the stream is not byte-aligned or an error occurs getting the position. .. method:: BitstreamWriter.setpos(position) Given a :class:`BitstreamWriterPosition` object, sets the stream to that position. The position must be one returned by this object's :meth:`BitstreamWriter.getpos` method; one cannot apply a position from one writer to a different one. May raise :exc:`IOError` if the stream is not byte-aligned or an error occurs setting the position. .. method:: BitstreamWriter.close() Flushes cached bytes to the stream and closes the underlying file object with its ``close`` method. .. method:: BitstreamWriter.__enter__() Returns the writers's context manager. .. method:: BitstreamWriter.__exit__(exc_type, exc_value, traceback) Exits the writer's context manager by calling :meth:`file.close` on the wrapped file object. If one wishes to keep the stream open for further writing, don't use a context manager and simply delete the writer object. But again, be aware that buffering may make its current position different than one might expect. BitstreamRecorder Objects ------------------------- This is a file-like object for recording the writing of individual bits or bytes, for possible output into a :class:`BitstreamWriter`. .. class:: BitstreamRecorder(is_little_endian) ``is_little_endian`` indicates whether to record a big-endian or little-endian output stream. .. method:: BitstreamRecorder.write(bits, value) Records the given unsigned integer value to the stream using the given number of bits. Bits must be: ``0 <= bits <= 32`` . Value must be: ``0 <= value < (2 ** bits)`` . .. method:: BitstreamRecorder.write64(bits, value) Records the given unsigned integer value to the stream using the given number of bits. Bits must be: ``0 <= bits <= 64`` . Value must be: ``0 <= value < (2 ** bits)`` . .. method:: BitstreamRecorder.write_signed(bits, value) Records the given signed integer value to the stream using the given number of bits. Bits must be: ``0 <= bits <= 32`` . Value must be: ``-(2 ** (bits - 1)) <= value < 2 ** (bits - 1)`` . .. method:: BitstreamRecorder.write_signed64(bits, value) Records the given signed integer value to the stream using the given number of bits. Bits must be: ``0 <= bits <= 64`` . Value must be: ``-(2 ** (bits - 1)) <= value < 2 ** (bits - 1)`` . .. method:: BitstreamRecorder.unary(stop_bit, value) If ``stop_bit`` is ``1``, records ``value`` number of ``0`` bits to the stream followed by a ``1`` bit. If ``stop_bit`` is ``0``, records ``value`` number of ``1`` bits to the stream followed by a ``0`` bit. .. method:: BitstreamRecorder.write_huffman_code(huffman_tree, value) Given a :class:`HuffmanTree` object and an integer value to write, determines the proper output code and records it for writing. Raises :exc:`ValueError` if the integer value is not present in the tree. .. method:: BitstreamRecorder.byte_align() Records ``0`` bits as necessary until the stream is aligned on a byte boundary. .. method:: BitstreamRecorder.byte_aligned() Returns ``True`` if the stream is positioned on a byte boundary. .. method:: BitstreamRecorder.build(format_string, value_list) Given a format string representing a set of individual writes, and a list of values to write, records those writes to the stream. ====== ============= ===================== format value method performed ====== ============= ===================== "#u" unsigned int write(#, u) "#s" signed int write(#, s) "#U" unsigned long write64(#, ul) "#S" signed long write_signed64(#, sl) "#p" N/A write(#, 0) "#P" N/A write(# * 8, 0) "#b" string write_bytes(#, s) "a" N/A byte_align() ====== ============= ===================== For instance: >>> w.build("3u 4s 36U", [1, -2, 3L]) is equivalent to: >>> w.write(3,1) >>> w.write_signed(4, -2) >>> w.write64(36, 3L) .. method:: BitstreamRecorder.write_bytes(string) Records the given binary string to the stream with a number of bytes equal to its length. .. method:: BitstreamRecorder.set_endianness(is_little_endian) Sets the stream's endianness where ``False`` indicates big-endian, while ``True`` indicates little-endian. The stream is automatically byte-aligned prior to changing its byte order. .. method:: BitstreamRecorder.add_callback(callback) Adds a callable function to the stream's callback stack. ``callback(b)`` takes a single byte as an argument. This callback is called upon each byte recorded to the stream. If multiple callbacks are added, they are all called in reverse order. .. method:: BitstreamRecorder.call_callbacks(byte) Calls all the callbacks on the stream's callback stack with the given byte, as if it had been recorded to the stream. .. method:: BitstreamRecorder.pop_callback() Removes and returns the most recently added function from the callback stack. .. method:: BitstreamRecorder.getpos() Returns a :class:`BitstreamWriterPosition` object of the stream's current position. May raise :exc:`IOError` if the stream is not byte-aligned or an error occurs getting the position. .. method:: BitstreamRecorder.setpos(position) Given a :class:`BitstreamWriterPosition` object, sets the stream to that position. The position must be one returned by this object's :meth:`BitstreamRecorder.getpos` method; one cannot apply a position from one writer to a different one. May raise :exc:`IOError` if the stream is not byte-aligned or an error occurs setting the position. .. method:: BitstreamRecorder.close() Does nothing. This is merely a placeholder for compatibility with :class:`BitstreamWriter`. .. method:: BitstreamRecorder.flush() Does nothing. This is merely a placeholder for compatibility with :class:`BitstreamWriter`. .. method:: BitstreamRecorder.bits() Returns the count of bits recorded as an integer. .. method:: BitstreamRecorder.bytes() Returns the count of bytes recorded as an integer. .. method:: BitstreamRecorder.copy(bitstreamwriter) Given a :class:`BitstreamWriter` or :class:`BitstreamRecorder` object, copies all recorded output to that stream, including any partially written bytes. .. method:: BitstreamRecorder.data() Returns a binary string of recorded data, not including any partially written bytes. .. method:: BitstreamRecorder.reset() Erases all recorded data and resets the stream for fresh recording. .. method:: BitstreamRecorder.swap(bitstreamrecorder) Swaps the recorded data with the given :class:`BitstreamRecorder` object. This is often useful for finding the best output given many possible input permutations: >>> best_case = BitstreamRecorder(False) >>> write_data(best_case, default_arguments) >>> next_best = BitstreamRecorder(False) >>> for arguments in argument_list: ... next_best.reset() ... write_data(next_best, arguments) ... if (next_best.bits() < best_case.bits()): ... next_best.swap(best_case) >>> best_case.copy(output_writer) Unlike replacing the ``best_case`` object with ``next_best``, swapping and resetting allows :class:`BitstreamRecorder` to reuse allocated data buffers. .. method:: BitstreamRecorder.__enter__() Returns the recorder's context manager. .. method:: BitstreamRecorder.__exit__(exc_type, exc_value, traceback) Exits the recorders's context manager. HuffmanTree Objects ------------------- This is a compiled Huffman tree for use by :class:`BitstreamReader` and :class:`BitstreamWriter`. .. class:: HuffmanTree([bits_list, value, ...], is_little_endian) ``bits_list`` is a list of ``0`` or ``1`` values which, when read from the stream on a bit-by-bit basis, result in the final integer value. For example, given the following Huffman tree definition: .. image:: huffman.png we define our Huffman tree for a big-endian stream as follows: >>> HuffmanTree([(1, ), 1, ... (0, 1), 2, ... (0, 0, 1), 3, ... (0, 0, 0), 4], False) Note that the bits in the tree are always consumed from the least-significant position to most-significant. This may differ from how they are consumed from the stream based on its ``is_little_endian`` value. The resulting object is passed to :meth:`BitstreamReader.read_huffman_code` to read the next value from a stream, and to :meth:`BitstreamWriter.write_huffman_code` to write a given value to the stream. May raise :exc:`ValueError` if the tree is incorrectly specified. ================================================ FILE: docs/programming/source/audiotools_cdio.rst ================================================ .. Audio Tools, a module and set of tools for manipulating audio data Copyright (C) 2007-2016 Brian Langenberger This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA :mod:`audiotools.cdio` --- the CD Input/Output Module ===================================================== .. module:: audiotools.cdio :synopsis: a Module for Accessing Raw CDDA Data The :mod:`audiotools.cdio` module contains the CDDAReader class for accessing raw CDDA data. CDDAReader Objects ------------------ .. class:: CDDAReader(device, [perform_logging]) A :class:`audiotools.PCMReader` object which treats the CD audio as a single continuous stream of audio data. ``device`` may be a physical CD device (like ``/dev/cdrom``) or a CD image file (like ``CDImage.cue``). If ``perform_logging`` is indicated and ``device`` is a physical drive, reads will perform logging. .. data:: CDDAReader.sample_rate The sample rate of this stream, always ``44100``. .. data:: CDDAReader.channels The channel count of this stream, always ``2``. .. data:: CDDAReader.channel_mask This channel mask of this stream, always ``3``. .. data:: CDDAReader.bits_per_sample The bits-per-sample of this stream, always ``16``. .. method:: CDDAReader.read(pcm_frames) Try to read a :class:`pcm.FrameList` object with the given number of PCM frames, if possible. This method will return sector-aligned chunks of data, each divisible by 588 frames. Once the end of the CD is reached, subsequent calls will return empty FrameLists. May raise :exc:`IOError` if a problem occurs reading the CD. .. method:: CDDAReader.seek(pcm_frames) Try to seek to the given absolute position on the disc as a PCM frame value. Returns the position actually reached as a PCM frame value. This method will always seek to a sector-aligned position, each divisible by 588 frames. .. method:: CDDAReader.close() Closes the stream for further reading. Subsequent calls to :meth:`CDDAReader.read` and :meth:`CDDAReader.seek` will raise :exc:`ValueError`. .. data:: CDDAReader.is_cd_image Whether the disc is a physical device or CD image. This is useful for determining whether disc read offset should be applied. .. data:: CDDAReader.first_sector The first sector of the disc as an integer. This is mostly for calculating disc IDs for various lookup services. .. data:: CDDAReader.last_sector The last sector of the disc as an integer. .. data:: CDDAReader.track_lengths A dict whose keys are track numbers and whose values are the lengths of those tracks in PCM frames. .. data:: CDDAReader.track_offsets A disc whose keys are track numbers and whose values are the offsets of those tracks in PCM frames. .. method:: CDDAReader.set_speed(speed) Sets the reading speed of the drive to the given integer. This has no effect on CD images. .. method:: CDDAReader.log() Returns the read log as a dictionary. If logging is active, these values will be updated on each call to :meth:`CDDAReader.read`. If logging is inactive or not supported, all values will be 0. .. method:: CDDAReader.reset_log() Resets all log values to 0. This is useful if one wants to get the log values for many tracks individually. ================================================ FILE: docs/programming/source/audiotools_cue.rst ================================================ .. Audio Tools, a module and set of tools for manipulating audio data Copyright (C) 2007-2016 Brian Langenberger This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA :mod:`audiotools.cue` --- the Cuesheet Parsing Module ===================================================== .. module:: audiotools.cue :synopsis: a Module for Parsing and Building CD Cuesheet Files. The :mod:`audiotools.cue` module contains the Cuesheet class used for parsing and building cuesheet files representing CD images. .. function:: read_cuesheet(filename) Takes a filename string and returns a new :class:`Cuesheet` object. Raises :exc:`CueException` if some error occurs when reading the file. .. exception:: CueException A subclass of :exc:`audiotools.SheetException` raised when some parsing or reading error occurs when reading a cuesheet file. Cuesheet Objects ---------------- .. class:: Cuesheet() This class is used to represent a .cue file. It is not meant to be instantiated directly but returned from the :func:`read_cuesheet` function. The :meth:`__str__` value of a Cuesheet corresponds to a formatted file on disk. .. method:: Cuesheet.catalog() Returns the cuesheet's catalog number as a plain string, or ``None`` if the cuesheet contains no catalog number. .. method:: Cuesheet.single_file_type() Returns ``True`` if the cuesheet is formatted for a single input file. Returns ``False`` if the cuesheet is formatted for several individual tracks. .. method:: Cuesheet.indexes() Returns an iterator of index lists. Each index is a tuple of CD sectors corresponding to a track's offset on disk. .. method:: Cuesheet.pcm_lengths(total_length, sample_rate) Takes the total length of the entire CD, in PCM frames, and the sample rate of the stream, in Hz. Returns a list of PCM frame lengths for all audio tracks within the cuesheet. This list of lengths can be used to split a single CD image file into several individual tracks. .. method:: Cuesheet.ISRCs() Returns a dictionary of track_number -> ISRC values for all tracks whose ISRC value is not empty. .. classmethod:: Cuesheet.file(sheet, filename) Takes a :class:`Cuesheet`-compatible object with :meth:`catalog`, :meth:`indexes`, :meth:`ISRCs` methods along with a filename string. Returns a new :class:`Cuesheet` object. This is used to convert other sort of Cuesheet-like objects into actual Cuesheets. ================================================ FILE: docs/programming/source/audiotools_dvda.rst ================================================ .. Audio Tools, a module and set of tools for manipulating audio data Copyright (C) 2007-2016 Brian Langenberger This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA :mod:`audiotools.dvda` --- the DVD-Audio Input/Output Module ============================================================ .. module:: audiotools.dvda :synopsis: a Module for Accessing DVD-Audio Data The :mod:`audiotools.dvda` module contains a set of classes for accessing DVD-Audio data. DVD Objects ----------- .. class:: DVDA(audio_ts_path, [cdrom_device]) A DVDA object represents the entire disc. ``audio_ts_path`` is the path to the disc's mounted ``AUDIO_TS`` directory, such as ``/media/cdrom/AUDIO_TS``. ``cdrom_device``, if given, is the path to the device the disc is mounted from, such as ``/dev/cdrom``. If the disc is encrypted and ``cdrom_device`` is given, decryption will be performed automatically. May raise :exc:`IOError` if some error occurs opening the disc. .. data:: DVDA.titlesets The number of title sets on the disc, typically 1. .. method:: DVDA.titleset(titleset_number) Given a title set number, starting from 1, returns that :class:`Titleset` object. Raises :exc:`IndexError` if that title set is not found on the disc. Titleset Objects ---------------- .. class:: Titleset(dvda, titleset_number) A Titleset object represents a title set on a disc. ``dvda`` is a :class:`DVDA` object and ``titleset_number`` is the title set number. My raise :exc:`IndexError` if the title set is not found on the disc. .. data:: Titleset.number The title set's number. .. data:: Titleset.titles The number of titles in the title set. .. method:: Titleset.title(title_number) Given a title number, starting from 1, returns that :class:`Title` object. Raises :exc:`IndexError` if that title is not found on the disc. Title Objects ------------- .. class:: Title(titleset, title_number) A Title object represents a title in a title set. ``titleset`` is a :class:`Titleset` object and ``title_number`` is the title number. May raise :exc:`IndexError` if the title is not found in the title set. .. data:: Title.number The title's number. .. data:: Title.tracks The number of tracks in the title. .. data:: Title.pts_length The length of the title in PTS ticks. There are 90000 PTS ticks per second. .. method:: Title.track(track_number) Given a track number, starting from 1, returns that :class:`Track` object. Raises :exc:`IndexError` if that track is not found in the title. Track Objects ------------- .. class:: Track(title, track_number) A Track object represents a track in a title. ``title`` is a :class:`Title` object and ``track_number`` is the track number. May raise :exc:`ValueError` if the track is not found in the title. .. data:: Track.number The track's number. .. data:: Track.pts_index The starting point of the track in the title, in PTS ticks. .. data:: Track.pts_length The length of the track in PTS ticks. There are 90000 PTS ticks per second. .. data:: Track.first_sector The track's first sector in the stream of ``.AOB`` files. Each sector is exactly 2048 bytes long. .. data:: Track.last_sector The track's last sector in the stream of ``.AOB`` files. .. method:: Track.reader() Returns a :class:`TrackReader` for reading this track's data. May raise :exc:`IOError` if some error occurs opening the reader. TrackReader Objects ------------------- .. class:: TrackReader(track) TrackReader is a :class:`audiotools.PCMReader` compatible object for extracting the audio data from a given track. ``track`` is a :class:`Track` object. May raise :exc:`IOError` if some error occurs opening the reader. .. data:: TrackReader.sample_rate The track's sample rate, in Hz. .. data:: TrackReader.bits_per_sample The track's bits-per-sample, either 24 or 16. .. data:: TrackReader.channels The track's channel count, often 2 or 6. .. data:: TrackReader.channel_mask The track's channel mask as a 32-bit value. .. data:: TrackReader.total_pcm_frames The track's total number of PCM frames. .. data:: TrackReader.codec The track's codec as a string. .. method:: TrackReader.read(pcm_frames) Attempts to read the given number of PCM frames from the track as a :class:`audiotools.pcm.FrameList` object. May return less than the requested number of PCM frames at the end of the disc. Attempting to read from a closed stream will raise :exc:`ValueError`. .. method:: TrackReader.close() Closes the stream for further reading. ================================================ FILE: docs/programming/source/audiotools_freedb.rst ================================================ .. Audio Tools, a module and set of tools for manipulating audio data Copyright (C) 2007-2016 Brian Langenberger This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA :mod:`audiotools.freedb` --- FreeDB Lookup Service Module ========================================================= .. module:: audiotools.freedb :synopsis: a Module for Accessing FreeDB Information The :mod:`audiotools.freedb` module contains classes and functions for performing lookups to the FreeDB service. DiscID Objects -------------- .. class:: DiscID(offsets, total_length, track_count) A FreeDB disc ID object used to perform lookups. ``offsets`` is a list of track offsets in CD sectors (1/75th of a second), each including the 2 second pre-gap at the start of the disc. ``total_length`` is the length of the disc, in seconds, from index point 1 of the first track to the disc's lead-out. ``track_count`` is the total number of tracks on the disc. .. method:: DiscID.__str__() Returns the disc ID as an 8 character hexadecimal string that FreeDB expects when performing lookups. .. classmethod:: DiscID.from_cddareader(cddareader) Given a :class:`audiotools.cdio.CDDAReader` object, returns the :class:`DiscID` of that disc. .. classmethod:: DiscID.from_tracks(tracks) Given a sorted list of :class:`audiotools.AudioFile` objects, returns the :class:`DiscID` as if those tracks were a CD. .. warning:: This assumes all the tracks from the disc are present and are laid out in a conventional fashion with no "hidden" tracks or other oddities. The disc ID may not be accurate if that's not the case. .. classmethod:: DiscID.from_sheet(sheet, total_pcm_frames, sample_rate) Given a :class:`audiotools.Sheet` object along with the total length of the disc in PCM frames and the disc's sample rate (typically 44100), returns the :class:`DiscID`. Performing Lookup ----------------- .. function:: perform_lookup(disc_id, freedb_server, freedb_port) Given a :class:`DiscID` object, FreeDB hostname string and FreeDB server port (usually 80), iterates over a list of :class:`audiotools.MetaData` objects per successful match, like: ``[track1, track2, ...], [track1, track2, ...], ...`` May yield nothing if the server has no matches for the given disc ID. May raise :exc:`urllib2.HTTPError` if an error occurs querying the server or :exc:`ValueError` if the server returns invalid data. ================================================ FILE: docs/programming/source/audiotools_musicbrainz.rst ================================================ .. Audio Tools, a module and set of tools for manipulating audio data Copyright (C) 2007-2016 Brian Langenberger This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA :mod:`audiotools.musicbrainz` --- MusicBrainz Lookup Service Module =================================================================== .. module:: audiotools.musicbrainz :synopsis: a Module for Accessing MusicBrainz Information The :mod:`audiotools.musicbrainz` module contains classes and functions for performing lookups to the MusicBrainz service. DiscID Objects -------------- .. class:: DiscID(first_track_number, last_track_number, lead_out_offset, offsets) A MusicBrainz disc ID object used to perform lookups. ``first_track_number`` is the first track number on the CD (typically 1), ``last_track_number`` is the last track number on the CD. ``lead_out_offset`` is the CD's lead-out sector offset (including the 2 second pre-gap). ``offsets`` is a list of track offsets in CD sectors (1/75th of a second), each including the 2 second pre-gap at the start of the disc. .. method:: DiscID.__str__() Returns the disc ID as a 28 character string that MusicBrainz expects when performing lookups. .. classmethod:: DiscID.from_cddareader(cddareader) Given a :class:`audiotools.cdio.CDDAReader` object, returns the :class:`DiscID` of that disc. .. classmethod:: DiscID.from_tracks(tracks) Given a sorted list of :class:`audiotools.AudioFile` objects, returns the :class:`DiscID` as if those tracks were a CD. .. warning:: This assumes all the tracks from the disc are present and are laid out in a conventional fashion with no "hidden" tracks or other oddities. The disc ID may not be accurate if that's not the case. .. classmethod:: DiscID.from_sheet(sheet, total_pcm_frames, sample_rate) Given a :class:`audiotools.Sheet` object along with the total length of the disc in PCM frames and the disc's sample rate (typically 44100), returns the :class:`DiscID`. Performing Lookup ----------------- .. function:: perform_lookup(disc_id, musicbrainz_server, musicbrainz_port) Given a :class:`DiscID` object, MusicBrainz hostname string and MusicBrainz server port (usually 80), iterates over a list of :class:`audiotools.MetaData` objects per successful match, like: ``[track1, track2, ...], [track1, track2, ...], ...`` May yield nothing if the server has no matches for the given disc ID. May raise :exc:`urllib2.HTTPError` if an error occurs querying the server or :exc:`xml.parsers.expat.ExpatError` if the server returns invalid data. ================================================ FILE: docs/programming/source/audiotools_pcm.rst ================================================ .. Audio Tools, a module and set of tools for manipulating audio data Copyright (C) 2007-2016 Brian Langenberger This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA :mod:`audiotools.pcm` --- the PCM FrameList Module ================================================== .. module:: audiotools.pcm :synopsis: the PCM FrameList Module The :mod:`audiotools.pcm` module contains the FrameList and FloatFrameList classes for handling blobs of raw data. These classes are immutable and list-like, but provide several additional methods and attributes to aid in processing PCM data. .. function:: empty_framelist(channels, bits_per_sample) Returns an empty :class:`FrameList` with the given parameters. .. function:: from_list(list, channels, bits_per_sample, is_signed) Given a list of integer values, a number of channels, the amount of bits-per-sample and whether the samples are signed, returns a new :class:`FrameList` object with those values. Raises :exc:`ValueError` if a :class:`FrameList` cannot be built from those values. >>> f = from_list([-1,0,1,2],2,16,True) >>> list(f) [-1, 0, 1, 2] .. function:: from_frames(frame_list) Given a list of :class:`FrameList` objects, returns a new :class:`FrameList` whose values are built from those objects. Raises :exc:`ValueError` if any of the objects are longer than 1 PCM frame, their number of channels are not consistent or their bits_per_sample are not consistent. >>> l = [from_list([-1,0],2,16,True), ... from_list([ 1,2],2,16,True)] >>> f = from_frames(l) >>> list(f) [-1, 0, 1, 2] .. function:: from_channels(frame_list) Given a list of :class:`FrameList` objects, returns a new :class:`FrameList` whose values are built from those objects. Raises :exc:`ValueError` if any of the objects are wider than 1 channel, their number of frames are not consistent or their bits_per_sample are not consistent. >>> l = [from_list([-1,1],1,16,True), ... from_list([ 0,2],1,16,True)] >>> f = from_channels(l) >>> list(f) [-1, 0, 1, 2] .. function:: empty_float_framelist(channels) Returns an empty :class:`FloatFrameList` with the given parameters. .. function:: from_float_frames(float_frame_list) Given a list of :class:`FloatFrameList` objects, returns a new :class:`FloatFrameList` whose values are built from those objects. Raises :exc:`ValueError` if any of the objects are longer than 1 PCM frame or their number of channels are not consistent. >>> l = [FloatFrameList([-1.0,0.0],2), ... FloatFrameList([ 0.5,1.0],2)] >>> f = from_float_frames(l) >>> list(f) [-1.0, 0.0, 0.5, 1.0] .. function:: from_float_channels(float_frame_list) Given a list of :class:`FloatFrameList` objects, returns a new :class:`FloatFrameList` whose values are built from those objects. Raises :exc:`ValueError` if any of the objects are wider than 1 channel or their number of frames are not consistent. >>> l = [FloatFrameList([-1.0,0.5],1), ... FloatFrameList([ 0.0,1.0],1)] >>> f = from_float_channels(l) >>> list(f) [-1.0, 0.0, 0.5, 1.0] FrameList Objects ----------------- .. class:: FrameList(string, channels, bits_per_sample, is_big_endian, is_signed) This class implements a PCM FrameList, which can be envisioned as a 2D array of signed integers where each row represents a PCM frame of samples and each column represents a channel. During initialization, ``string`` is a collection of raw bytes, ``bits_per_sample`` is an integer and ``is_big_endian`` and ``is_signed`` are booleans. This provides a convenient way to transforming raw data from file-like objects into :class:`FrameList` objects. Once instantiated, a :class:`FrameList` object is immutable. .. data:: FrameList.frames The amount of PCM frames within this object, as a non-negative integer. .. data:: FrameList.channels The amount of channels within this object, as a positive integer. .. data:: FrameList.bits_per_sample The size of each sample in bits, as a positive integer. .. method:: FrameList.frame(frame_number) Given a non-negative ``frame_number`` integer, returns the samples at the given frame as a new :class:`FrameList` object. This new FrameList will be a single frame long, but have the same number of channels and bits_per_sample as the original. Raises :exc:`IndexError` if one tries to get a frame number outside this FrameList's boundaries. .. method:: FrameList.channel(channel_number) Given a non-negative ``channel_number`` integer, returns the samples at the given channel as a new :class:`FrameList` object. This new FrameList will be a single channel wide, but have the same number of frames and bits_per_sample as the original. Raises :exc:`IndexError` if one tries to get a channel number outside this FrameList's boundaries. .. method:: FrameList.split(frame_count) Returns a pair of :class:`FrameList` objects. The first contains up to ``frame_count`` number of PCM frames. The second contains the remainder. If ``frame_count`` is larger than the number of frames in the FrameList, the first will contain all of the frames and the second will be empty. .. method:: FrameList.to_float() Converts this object's values to a new :class:`FloatFrameList` object by transforming all samples to the range -1.0 to 1.0. .. method:: FrameList.to_bytes(is_big_endian, is_signed) Given ``is_big_endian`` and ``is_signed`` booleans, returns a plain string of raw PCM data. This is much like the inverse of :class:`FrameList`'s initialization routine. .. method:: FrameList.frame_count(bytes) A convenience method which converts a given byte count to the maximum number of frames those bytes could contain, or a minimum of 1. >>> FrameList("",2,16,False,True).frame_count(8) 2 FloatFrameList Objects ---------------------- .. class:: FloatFrameList(floats, channels) This class implements a FrameList of floating point samples, which can be envisioned as a 2D array of signed floats where each row represents a PCM frame of samples, each column represents a channel and each value is within the range of -1.0 to 1.0. During initialization, ``floats`` is a list of float values and ``channels`` is an integer number of channels. .. data:: FloatFrameList.frames The amount of PCM frames within this object, as a non-negative integer. .. data:: FloatFrameList.channels The amount of channels within this object, as a positive integer. .. method:: FloatFrameList.frame(frame_number) Given a non-negative ``frame_number`` integer, returns the samples at the given frame as a new :class:`FloatFrameList` object. This new FloatFrameList will be a single frame long, but have the same number of channels as the original. Raises :exc:`IndexError` if one tries to get a frame number outside this FloatFrameList's boundaries. .. method:: FloatFrameList.channel(channel_number) Given a non-negative ``channel_number`` integer, returns the samples at the given channel as a new :class:`FloatFrameList` object. This new FloatFrameList will be a single channel wide, but have the same number of frames as the original. Raises :exc:`IndexError` if one tries to get a channel number outside this FloatFrameList's boundaries. .. method:: FloatFrameList.split(frame_count) Returns a pair of :class:`FloatFrameList` objects. The first contains up to ``frame_count`` number of PCM frames. The second contains the remainder. If ``frame_count`` is larger than the number of frames in the FloatFrameList, the first will contain all of the frames and the second will be empty. .. method:: FloatFrameList.to_int(bits_per_sample) Given a ``bits_per_sample`` integer, converts this object's floating point values to a new :class:`FrameList` object. ================================================ FILE: docs/programming/source/audiotools_pcmconverter.rst ================================================ .. Audio Tools, a module and set of tools for manipulating audio data Copyright (C) 2007-2016 Brian Langenberger This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA :mod:`audiotools.pcmconverter` --- the PCM Conversion Module ============================================================ .. module:: audiotools.pcmconverter :synopsis: a Module for Converting PCM Streams The :mod:`audiotools.pcmconverter` module contains :class:`audiotools.PCMReader` wrapper classes for converting streams to different sample rates, channel counts, channel assignments and so on. These classes are combined by :class:`audiotools.PCMConverter` as needed to modify a stream from one format to another. Averager Objects ---------------- .. class:: Averager(pcmreader) This class takes a :class:`audiotools.PCMReader`-compatible object and constructs a new :class:`audiotools.PCMReader`-compatible whose channels have been averaged together into a single channel. .. data:: Averager.sample_rate The sample rate of this audio stream, in Hz, as a positive integer. .. data:: Averager.channels The number of channels of this audio stream, which is always 1. .. data:: Averager.channel_mask The channel mask of this audio stream, which is always ``0x4``. .. data:: Averager.bits_per_sample The number of bits-per-sample in this audio stream as a positive integer. .. method:: Averager.read(pcm_frames) Try to read a :class:`audiotools.pcm.FrameList` object with the given number of PCM frames, if possible. This method is *not* guaranteed to read that amount of frames. It may return less, particularly at the end of an audio stream. It may even return FrameLists larger than requested. However, it must always return a non-empty FrameList until the end of the PCM stream is reached. May raise :exc:`IOError` if there is a problem reading the source file, or :exc:`ValueError` if the source file has some sort of error. .. method:: Averager.close() Closes the audio stream. If any subprocesses were used for audio decoding, they will also be closed and waited for their process to finish. May raise a :exc:`DecodingError`, typically indicating that a helper subprocess used for decoding has exited with an error. BPSConverter Objects -------------------- .. class:: BPSConveter(pcmreader, bits_per_sample) This class takes a :class:`audiotools.PCMReader`-compatible object and new ``bits_per_sample`` integer, and constructs a new :class:`audiotools.PCMReader`-compatible object with that amount of bits-per-sample by truncating or extending bits to each sample as needed. .. data:: BPSConverter.sample_rate The sample rate of this audio stream, in Hz, as a positive integer. .. data:: BPSConverter.channels The number of channels in this audio stream as a positive integer. .. data:: BPSConverter.channel_mask The channel mask of this audio stream as a non-negative integer. .. data:: BPSConverter.bits_per_sample The number of bits-per-sample in this audio stream as indicated at init-time. .. method:: BPSConverter.read(pcm_frames) Try to read a :class:`audiotools.pcm.FrameList` object with the given number of PCM frames, if possible. This method is *not* guaranteed to read that amount of frames. It may return less, particularly at the end of an audio stream. It may even return FrameLists larger than requested. However, it must always return a non-empty FrameList until the end of the PCM stream is reached. May raise :exc:`IOError` if there is a problem reading the source file, or :exc:`ValueError` if the source file has some sort of error. .. method:: BPSConverter.close() Closes the audio stream. If any subprocesses were used for audio decoding, they will also be closed and waited for their process to finish. May raise a :exc:`DecodingError`, typically indicating that a helper subprocess used for decoding has exited with an error. Downmixer Objects ----------------- .. class:: Downmixer(pcmreader) This class takes a :class:`audiotools.PCMReader`-compatible object, presumably with more than two channels, and constructs a :class:`audiotools.PCMReader`-compatible object with only two channels mixed in Dolby Pro Logic format such that a rear channel can be restored. If the stream has fewer than 5.1 channels, those channels are padded with silence. Additional channels beyond 5.1 are ignored. .. data:: Downmixer.sample_rate The sample rate of this audio stream, in Hz, as a positive integer. .. data:: Downmixer.channels The number of channels in this audio stream, which is always 2. .. data:: Downmixer.channel_mask The channel mask of this audio stream, which is always ``0x3``. .. data:: Downmixer.bits_per_sample The number of bits-per-sample in this audio stream as a positive integer. .. method:: Downmixer.read(pcm_frames) Try to read a :class:`audiotools.pcm.FrameList` object with the given number of PCM frames, if possible. This method is *not* guaranteed to read that amount of frames. It may return less, particularly at the end of an audio stream. It may even return FrameLists larger than requested. However, it must always return a non-empty FrameList until the end of the PCM stream is reached. May raise :exc:`IOError` if there is a problem reading the source file, or :exc:`ValueError` if the source file has some sort of error. .. method:: Downmixer.close() Closes the audio stream. If any subprocesses were used for audio decoding, they will also be closed and waited for their process to finish. May raise a :exc:`DecodingError`, typically indicating that a helper subprocess used for decoding has exited with an error. Resampler Objects ----------------- .. class:: Resampler(pcmreader, sample_rate) This class takes a :class:`audiotools.PCMReader`-compatible object and new ``sample_rate`` integer, and constructs a new :class:`audiotools.PCMReader`-compatible object with that sample rate. .. data:: Resampler.sample_rate The sample rate of this audio stream, in Hz, as given at init-time. .. data:: Resampler.channels The number of channels in this audio stream as a positive integer. .. data:: Resampler.channel_mask The channel mask of this audio stream as a non-negative integer. .. data:: Resampler.bits_per_sample The number of bits-per-sample in this audio stream as a positive integer. .. method:: Resampler.read(pcm_frames) Try to read a :class:`audiotools.pcm.FrameList` object with the given number of PCM frames, if possible. This method is *not* guaranteed to read that amount of frames. It may return less, particularly at the end of an audio stream. It may even return FrameLists larger than requested. However, it must always return a non-empty FrameList until the end of the PCM stream is reached. May raise :exc:`IOError` if there is a problem reading the source file, or :exc:`ValueError` if the source file has some sort of error. .. method:: Resampler.close() Closes the audio stream. If any subprocesses were used for audio decoding, they will also be closed and waited for their process to finish. May raise a :exc:`DecodingError`, typically indicating that a helper subprocess used for decoding has exited with an error. ================================================ FILE: docs/programming/source/audiotools_player.rst ================================================ .. Audio Tools, a module and set of tools for manipulating audio data Copyright (C) 2007-2016 Brian Langenberger This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA :mod:`audiotools.player` --- the Audio Player Module ==================================================== .. module:: audiotools.player :synopsis: the Audio Player Module The :mod:`audiotools.player` module contains the Player and AudioOutput classes for playing AudioFiles. .. function:: audiotools.player.available_outputs() Iterates over all available :class:`AudioOutput` objects. This will always return at least one output object. .. function:: audiotools.player.open_output(output) Given a string of an :class:`AudioOutput` class' ``NAME`` attribute, returns the given :class:`AudioOutput` object. Raises :exc:`ValueError` if the output cannot be found. Player Objects -------------- This class is an audio player which plays audio data from an opened audio file object to a given output sink. .. class:: Player(audio_output[, replay_gain[, next_track_callback]]) ``audio_output`` is an :class:`AudioOutput` object. ``replay_gain`` is either ``RG_NO_REPLAYGAIN``, ``RG_TRACK_GAIN`` or ``RG_ALBUM_GAIN``, indicating the level of ReplayGain to apply to tracks being played back. ``next_track_callback`` is a function which takes no arguments, to be called when the currently playing track is completed. Raises :exc:`ValueError` if unable to start player subprocess. .. method:: Player.open(audiofile) Opens the given :class:`audiotools.AudioFile` object for playing. Any currently playing file is stopped. .. method:: Player.play() Begins or resumes playing the currently opened :class:`audiotools.AudioFile` object, if any. .. method:: Player.set_replay_gain(replay_gain) Sets the given ReplayGain level to apply during playback. Choose from ``RG_NO_REPLAYGAIN``, ``RG_TRACK_GAIN`` or ``RG_ALBUM_GAIN`` ReplayGain cannot be applied mid-playback. One must :meth:`stop` and :meth:`play` a file for it to take effect. .. method:: Player.set_output(audio_output) Changes where the audio will be played to the given output where ``audio_output`` is an :class:`AudioOutput` object. The current state and progress of the player remains unchanged, but the output volume and description may change. .. method:: Player.pause() Pauses playback of the current file. Playback may be resumed with :meth:`play` or :meth:`toggle_play_pause` .. method:: Player.toggle_play_pause() Pauses the file if playing, play the file if paused. .. method:: Player.stop() Stops playback of the current file. If :meth:`play` is called, playback will start from the beginning. .. method:: Player.state() Returns the current state of the player which will be either ``PLAYER_STOPPED``, ``PLAYER_PAUSED`` or ``PLAYER_PLAYING`` integers. .. method:: Player.close() Closes the player for playback. The player thread is halted and the :class:`AudioOutput` object is closed. .. method:: Player.progress() Returns a (``pcm_frames_played``, ``pcm_frames_total``) tuple. This indicates the current playback status in terms of PCM frames. .. method:: Player.current_output_description() Returns the human-readable description of the current output device as a Unicode string. .. method:: Player.current_output_name() Returns the ``NAME`` attribute of the current output device as a plain string. .. method:: Player.get_volume() Returns the current volume level as a floating point value between 0.0 and 1.0, inclusive. .. method:: Player.set_volume(volume) Given a floating point value between 0.0 and 1.0, inclusive, sets the current volume level to that value. .. method:: Player.change_volume(delta) Given a floating point delta value which may be positive (to increase volume) or negative (to decrease volume), adjusts the current volume by that amount and returns the new volume as a floating point value between 0.0 and 1.0, inclusive. CDPlayer Objects ---------------- This class is an audio player which plays audio data from a CDDA disc to a given output sink. .. class:: CDPlayer(cdda, audio_output[, next_track_callback]) ``cdda`` is a :class:`audiotools.CDDA` object. ``audio_output`` is a :class:`AudioOutput` object subclass which audio data will be played to. ``next_track_callback`` is a function which takes no arguments, to be called when the currently playing track is completed. .. method:: CDPlayer.open(track_number) Opens the given track number for reading, where ``track_number`` starts from 1. .. method:: CDPlayer.play() Begins or resumes playing the currently opened track, if any. .. method:: CDPlayer.pause() Pauses playback of the current track. Playback may be resumed with :meth:`play` or :meth:`toggle_play_pause` .. method:: CDPlayer.toggle_play_pause() Pauses the track if playing, play the track if paused. .. method:: CDPlayer.stop() Stops playback of the current track. If :meth:`play` is called, playback will start from the beginning. .. method:: CDPlayer.close() Closes the player for playback. The player thread is halted and the :class:`AudioOutput` object is closed. .. method:: CDPlayer.progress() Returns a (``pcm_frames_played``, ``pcm_frames_total``) tuple. This indicates the current playback status in terms of PCM frames. AudioOutput Objects ------------------- This is an abstract class used to implement audio output sinks. .. class:: AudioOutput() .. data:: AudioOutput.NAME The name of the AudioOutput subclass as a string. .. method:: AudioOutput.description() Returns a user-friendly name of the output device as a Unicode string. .. method:: AudioOutput.compatible(sample_rate, channels, channel_mask, bits_per_sample) Returns ``True`` if the given attributes are compatible with the currently opened output stream. .. method:: AudioOutput.set_format(sample_rate, channels, channel_mask, bits_per_sample) Sets the output stream to the given format. If the stream hasn't been initialized, this method initializes it to that format. If the stream has been initialized to a different format, this method closes and reports the stream to the new format. If the stream has been initialized to the same format, this method does nothing. .. method:: AudioOutput.play(framelist) Plays the given FrameList object to the output stream. This presumes the output stream's format has been set correctly. .. method:: AudioOutput.pause() Pauses output of playing data. .. note:: Although suspending the transmission of data to output will also have the same effect as pausing, calling the output's .pause() method will typically suspend output immediately instead of having to wait for the buffer to empty - which may take a fraction of a second. .. method:: AudioOutput.resume() Resumes playing data to output after it has been paused. .. method:: AudioOutput.get_volume() Returns a floating-point volume value between 0.0 and 1.0, inclusive. .. method:: AudioOutput.set_volume(volume) Given a floating-point volume value between 0.0 and 1.0, inclusive, sets audio output to that volume. .. method:: AudioOutput.close() Closes the output stream for further playback. .. classmethod:: AudioOutput.available() Returns True if the AudioOutput implementation is available on the system. ================================================ FILE: docs/programming/source/audiotools_replaygain.rst ================================================ .. Audio Tools, a module and set of tools for manipulating audio data Copyright (C) 2007-2016 Brian Langenberger This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA :mod:`audiotools.replaygain` --- the ReplayGain Calculation Module ================================================================== .. module:: audiotools.replaygain :synopsis: a Module for Calculating and Applying ReplayGain Values The :mod:`audiotools.replaygain` module contains the ReplayGain class for calculating the ReplayGain gain and peak values for a set of PCM data, and the ReplayGainReader class for applying those gains to a :class:`audiotools.PCMReader` stream. ReplayGain Objects ------------------ .. class:: ReplayGain(sample_rate) This class performs ReplayGain calculation for a stream of the given ``sample_rate``. Raises :exc:`ValueError` if the sample rate is not supported. .. attribute:: ReplayGain.sample_rate The sample rate given when the object was initialized. .. method:: ReplayGain.update(framelist) Given a :class:`pcm.FrameList` object, updates the current gain values with its data. .. method:: ReplayGain.title_gain() Returns the gain value of the current title as a positive or negative floating point value. May raise :exc:`ValueError` if not enough samples have been submitted for processing. .. method:: ReplayGain.title_peak() Returns the peak value of the title as a floating point value between 0.0 and 1.0. .. method:: ReplayGain.album_gain() Returns the gain value of the entire album as a positive or negative floating point value. May raise :exc:`ValueError` if not enough samples have been submitted for processing. .. method:: ReplayGain.album_peak() Returns the peak value of the entire album as a floating point value between 0.0 and 1.0. .. method:: ReplayGain.next_title() Indicates the current track is finished and resets the stream to process the next track. This method should be called after :meth:`ReplayGain.title_gain` and :meth:`ReplayGain.title_peak` have been used to extract the title's gain values, but before data has been submitted for the next title or :meth:`ReplayGain.album_gain` :meth:`ReplayGain.album_peak` have been called to get the entire album's gain values. ReplayGainReader Objects ------------------------ .. class:: ReplayGainReader(pcmreader, gain, peak) This class wraps around an existing :class:`PCMReader` object. It takes floating point ``gain`` and ``peak`` values and modifies the pcmreader's output as necessary to match those values. This has the effect of raising or lowering a stream's sound volume to ReplayGain's reference value. ================================================ FILE: docs/programming/source/audiotools_toc.rst ================================================ .. Audio Tools, a module and set of tools for manipulating audio data Copyright (C) 2007-2016 Brian Langenberger This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA :mod:`audiotools.toc` --- the TOC File Parsing Module ===================================================== .. module:: audiotools.toc :synopsis: a Module for Parsing and Building CD TOC Files. The :mod:`audiotools.toc` module contains the TOCFile class used for parsing and building TOC files representing CD images. .. function:: read_tocfile(filename) Takes a filename string and returns a new :class:`TOCFile` object. Raises :exc:`TOCException` if some error occurs when reading the file. .. exception:: TOCException A subclass of :exc:`audiotools.SheetException` raised when some parsing or reading error occurs when reading a TOC file. TOCFile Objects ---------------- .. class:: TOCFile() This class is used to represent a .toc file. It is not meant to be instantiated directly but returned from the :func:`read_tocfile` function. .. method:: TOCFile.catalog() Returns the TOC file's catalog number as a plain string, or ``None`` if the TOC file contains no catalog number. .. method:: TOCFile.indexes() Returns an iterator of index lists. Each index is a tuple of CD sectors corresponding to a track's offset on disk. .. method:: TOCFile.pcm_lengths(total_length, sample_rate) Takes the total length of the entire CD, in PCM frames, and the sample rate of the stream, in Hz. Returns a list of PCM frame lengths for all audio tracks within the TOC file. This list of lengths can be used to split a single CD image file into several individual tracks. .. method:: TOCFile.ISRCs() Returns a dictionary of track_number -> ISRC values for all tracks whose ISRC value is not empty. .. classmethod:: TOCFile.file(sheet, filename) Takes a :class:`cue.Cuesheet`-compatible object with :meth:`catalog`, :meth:`indexes`, :meth:`ISRCs` methods along with a filename string. Returns a new :class:`TOCFile` object. This is used to convert other sort of Cuesheet-like objects into actual TOC files. ================================================ FILE: docs/programming/source/audiotools_ui.rst ================================================ .. Audio Tools, a module and set of tools for manipulating audio data Copyright (C) 2007-2016 Brian Langenberger This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA :mod:`audiotools.ui` --- Reusable Python Audio Tools GUI Widgets ================================================================ This module contains a collection of reusable Urwid widgets and helper functions for processing user input and generating helper output in a consistent way. .. function:: show_available_formats(messenger) Given a :class:`audiotools.Messenger` object, displays all the available file formats one can select with a utility's ``-t`` argument in a user-friendly way. .. function:: show_available_qualities(messenger, audio_type) Given a :class:`audiotools.Messenger` object and an :class:`audiotools.AudioFile` subclass, displays all the available output qualities for that audio type one can select with a utility's ``-q`` argument. .. function:: select_metadata(metadata_choices, messenger, [use_default]) Given ``metadata_choices[choice][track]`` where each choice contains a list of :class:`audiotools.MetaData` objects and all choices must have the same number of objects, along with a :class:`audiotools.Messenger` object, queries the user for which metadata choice to use and returns that list of :class:`audiotools.MetaData` objects. If ``use_default`` is ``True``, this simply returns the metadata for the first choice. .. function:: process_output_options(metadata_choices, input_filenames, output_directory, format_string, output_class, quality, messenger, [use_default]) This function is for processing the input and output options for utilities such as ``track2track`` and ``cd2track``. ``metadata_choices[choice][track]`` is a list of :class:`audiotools.MetaData` objects and all choices must have the same number of objects. ``input_filenames`` is a list of :class:`audiotools.Filename` objects where the number of filenames must match the number of :class:`audiotools.MetaData` objects for each choice in ``metadata_choices``. ``output_directory`` is a string to the output directory, indicated by a utility's ``-d`` option. ``format_string`` is a UTF-8 encoded plain string of the output file format, indicated by a utility's ``--format`` option. ``output_class`` is an :class:`audiotools.AudioFile` class, indicated by a utility's ``-t`` option. ``quality`` is a string if the output quality to use, indicated by a utility's ``-q`` option. ``messenger`` is an :class:`audiotools.Messenger` object used for displaying output. ``use_default``, if indicated, is applied to this function's call to :func:`select_metadata` if necessary. Yields a ``(output_class, output_filename, output_quality, output_metadata)`` tuple for each output file where ``output_class`` is an :class:`audiotools.AudioFile` object, ``output_filename`` is a :class:`audiotools.Filename` object, ``output_quality`` is a compression string, and ``output_metadata`` is a :class:`audiotools.MetaData` object. Raises :exc:`audiotools.UnsupportedTracknameField` or :exc:`audiotools.InvalidFilenameFormat` if the ``format_string`` option is invalid. Raises :exc:`audiotools.DuplicateOutputFile` if the same output filename is generated more than once. Raises :exc:`audiotools.OutputFileIsInput` if one of the output files is the same as any of the input files. .. function:: not_available_message(messenger) Given a :class:`audiotools.Messenger` object, displays a message about Urwid being unavailable and offers a suggestion on how to obtain it. .. function:: xargs_suggestion(args) Given a list of argument strings (such as from ``sys.argv``) returns a Unicode string indicating how one might call the given program using ``xargs``. PlayerTTY Objects ----------------- .. class:: PlayerTTY(player) This is a base class for implementing the user interface for TTY-based audio players. ``player`` is a :class:`audiotools.player.Player`-compatible object. .. method:: PlayerTTY.next_track() Stop playing the current track and begin playing the next one. This must be implemented in a subclass. .. method:: PlayerTTY.previous_track() Stop playing the current track and begin playing the previous one. This must be implemented in a subclass. .. method:: PlayerTTY.set_metadata(track_number, track_total, channels, sample_rate, bits_per_sample) Typically called by :meth:`PlayerTTY.next_track` and :meth:`PlayerTTY.previous_track`, this sets the current metadata to the given values for displaying to the user. .. method:: PlayerTTY.toggle_play_pause() Calls :meth:`audiotools.player.Player.toggle_play_pause` on the internal :class:`audiotools.player.Player` object to suspend or resume output. .. method:: PlayerTTY.stop() Calls :meth:`audiotools.player.Player.stop` on the internal :class:`audiotools.player.Player` object to stop playing the current file completely. .. method:: PlayerTTY.progress() Returns the values from :meth:`audiotools.player.Player.progress` which indicate the current status of the playing file. .. method:: PlayerTTY.progress_line(frames_sent, frames_total) Given the amount of PCM frames sent and total number of PCM frames as integers, returns a Unicode string of the current progress to be displayed to the user. .. method:: PlayerTTY.run(messenger, stdin) Given a :class:`audiotools.Messenger` object and ``sys.stdin`` file object, this runs the player's output loop until the user indicates it should exit or the input is exhausted. Returns 0 on a successful exit, 1 if it exits with an error. .. data:: AVAILABLE ``True`` if Urwid is available and is of a sufficiently high version. ``False`` if not. Urwid Widgets ------------- If Urwid is available, the following classes will be in this module for use by utilities to generate interactive modes. If not, the classes will not be defined. OutputFiller Objects ^^^^^^^^^^^^^^^^^^^^ .. class:: OutputFiller(track_labels, metadata_choices, input_filenames, output_directory, format_string, output_class, quality, [completion_label]) This is an Urwid Frame subclass for populating track data and options for multiple output file utilities such as ``track2track`` and ``cd2track``. ``track_labels`` is a list of Unicode strings, one per track ``metadata_choices[choice][track]`` is a list of :class:`audiotools.MetaData` objects per choice, one per track ``input_filenames`` is a list of :class:`audiotools.Filename` objects, one per track. ``output_directory`` is a string to the output directory, indicated by a utility's ``-d`` option. ``format_string`` is a UTF-8 encoded plain string of the output file format, indicated by a utility's ``--format`` option. ``output_class`` is an :class:`audiotools.AudioFile` class, indicated by a utility's ``-t`` option. ``quality`` is a string if the output quality to use, indicated by a utility's ``-q`` option. ``completion_label`` is an optional Unicode string to display in the widget's "apply" button used to complete the operation. This widget is typically executed as follows: >>> widget = audiotools.ui.OutputFiller(...) # populate widget with metadata and command-line options >>> loop = urwid.MainLoop(widget, ... audiotools.ui.style(), ... unhandled_input=widget.handle_text, ... pop_ups=True) >>> loop.run() >>> if (not widget.cancelled()): ... # do work here ... else: ... # exit .. method:: OutputFiller.output_tracks() Yields a ``(output_class, output_filename, output_quality, output_metadata)`` tuple for each output file where ``output_class`` is an :class:`audiotools.AudioFile` object, ``output_filename`` is a :class:`audiotools.Filename` object, ``output_quality`` is a compression string, and ``output_metadata`` is a :class:`audiotools.MetaData` object. .. note:: This method returns freshly-created :class:`audiotools.MetaData` objects, whereas :func:`process_output_options` resuses the same objects passed to it. Because :class:`OutputFiller` may modify input metadata, we don't want to risk modifying objects used elsewhere. .. method:: OutputFiller.output_directory() Returns the currently selected output directory as a plain string. .. method:: OutputFiller.format_string() Returns the current format string as a plain, UTF-8 encoded string. .. method:: OutputFiller.output_class() Returns the current :class:`audiotools.AudioFile`-compatible output class. .. method:: OutputFiller.quality() Returns the current output quality as a plain string. SingleOutputFiller Objects ^^^^^^^^^^^^^^^^^^^^^^^^^^ .. class:: SingleOutputFiller(track_label, metadata_choices, input_filenames, output_file, output_class, quality, [completion_label]) This is an Urwid Frame subclass for populating track data and options for a single output track utilities such as ``trackcat``. ``track_label`` is a Unicode string. ``metadata_choices[choice]`` is a list of :class:`audiotools.MetaData` objects for all possible choices for the given track. ``input_filenames`` is a list or set of :class:`audiotools.Filename` objects for all input files, which may include cuesheets or other auxiliary data. ``output_file`` is a plain string of the default output filename. ``output_class`` is an :class:`audiotools.AudioFile` class, indicated by a utility's ``-t`` option. ``quality`` is a string if the output quality to use, indicated by a utility's ``-q`` option. ``completion_label`` is an optional Unicode string to display in the widget's "apply" button used to complete the operation. This widget is typically executed as follows: >>> widget = audiotools.ui.SingleOutputFiller(...) # populate widget with metadata and command-line options >>> loop = urwid.MainLoop(widget, ... audiotools.ui.style(), ... unhandled_input=widget.handle_text, ... pop_ups=True) >>> loop.run() >>> if (not widget.cancelled()): ... # do work here ... else: ... # exit .. method:: SingleOutputFiller.output_track() Returns ``(output_class, output_filename, output_quality, output_metadata)`` tuple to apply to the single output track. ``output_class`` is an :class:`audiotools.AudioFile` object, ``output_filename`` is a :class:`audiotools.Filename` object, ``output_quality`` is a compression string, and ``output_metadata`` is a :class:`audiotools.MetaData` object. MetaDataFiller Objects ^^^^^^^^^^^^^^^^^^^^^^ .. class:: MetaDataFiller(track_labels, metadata_choices, status) This is an Urwid Pile subclass for selecting and editing a single set of metadata from multiple choices. It is used by :class:`OutputFiller` and :class:`SingleOutputFiller` as necessary to allow the user to edit metadata when setting options. ``track_labels`` is a list of Unicode strings, one per track. ``metadata_choices[choice][track]`` is a list of :class:`audiotools.MetaData` objects per choice, one per track ``status`` is an :class:`urwid.Text` object to display status text such as key shortcuts. .. method:: MetaDataFiller.select_previous_item() Selects the previous item in the current set of metadata, such as the previous track or the previous field, depending on how the data is swiveled. .. method:: MetaDataFiller.select_next_item() Selects the next item in the current set of metadata, such as the next track or the next field, depending on how the data is swiveled. .. method:: MetaDataFiller.populated_metadata() Yields a new, populated :class:`audiotools.MetaData` object per track, depending on the current selection and its values. MetaDataEditor Objects ^^^^^^^^^^^^^^^^^^^^^^ .. class:: MetaDataEditor(tracks, [on_text_change], [on_swivel_change]) This is an Urwid Frame subclass for editing a single set of metadata across multiple tracks. ``tracks`` is a list of ``(id, label, metadata)`` tuples in the order they are to be displayed with ``id`` is some unique, hashable ID value, ``label`` is a Unicode string, and ``metadata`` is an :class:`audiotools.MetaData` object, or ``None``. ``on_text_change(widget, new_value)`` is a callback for when any text field is modified. ``on_swivel_change(widget, selected, swivel)`` is a callback for when tracks and fields are swapped. .. method:: MetaDataEditor.select_previous_item() Selects the previous item in the current set of metadata, such as the previous track or the previous field, depending on how the data is swiveled. .. method:: MetaDataEditor.select_next_item() Selects the next item in the current set of metadata, such as the next track or the next field, depending on how the data is swiveled. .. method:: MetaDataEditor.metadata() Yields a ``(track_id, metadata)`` tuple per edited track where ``track_id`` is the unique, hashable value entered at init-time, and ``metadata`` is a newly created :class:`audiotools.MetaData` object. BottomLineBox Objects ^^^^^^^^^^^^^^^^^^^^^ .. class:: BottomLineBox(original_widget, [title], [tlcorner], [tline], [lline], [trcorner], [blcorner], [rline], [bline], [bcorner]) This is an Urwid LineBox subclass which places its title at the bottom instead of the top. SelectOne Objects ^^^^^^^^^^^^^^^^^ .. class:: SelectOne(items, [selected_value], [on_change], [user_data], [label]) This is an Urwid PopUpLauncher subclass designed to work as an HTML-style