Showing preview only (1,152K chars total). Download the full file or copy to clipboard to get everything.
Repository: madderscientist/noteDigger
Branch: main
Commit: d62567861517
Files: 52
Total size: 11.1 MB
Directory structure:
gitextract_zaftgd5a/
├── LICENSE
├── README.md
├── app.js
├── core/
│ ├── app_analyser.js
│ ├── app_audioplayer.js
│ ├── app_beatbar.js
│ ├── app_hscrollbar.js
│ ├── app_io.js
│ ├── app_keyboard.js
│ ├── app_midiaction.js
│ ├── app_midiplayer.js
│ ├── app_spectrogram.js
│ └── app_timebar.js
├── dataProcess/
│ ├── AI/
│ │ ├── AIEntrance.js
│ │ ├── SpectralClustering.js
│ │ ├── basicamt_44100.onnx
│ │ ├── basicamt_worker.js
│ │ ├── dist/
│ │ │ └── ort-wasm-simd.wasm
│ │ ├── postprocess.js
│ │ ├── septimbre_44100.onnx
│ │ └── septimbre_worker.js
│ ├── ANA.js
│ ├── CQT/
│ │ ├── cqt.js
│ │ └── cqt_worker.js
│ ├── NNLS.js
│ ├── aboutANA.md
│ ├── analyser.js
│ ├── bpmEst.js
│ ├── fft_real.js
│ └── stftGPU.js
├── docs/
│ └── DEVELOPMENT.md
├── index.html
├── jsconfig.json
├── lib/
│ ├── beatBar.js
│ ├── fakeAudio.js
│ ├── midi.js
│ ├── saver.js
│ ├── snapshot.js
│ └── tinySynth.js
├── plugins/
│ ├── chordEst.js
│ └── pitchName.js
├── style/
│ ├── askUI.css
│ ├── channelDiv.css
│ ├── contextMenu.css
│ ├── icon/
│ │ └── iconfont.css
│ ├── myRange.css
│ ├── siderMenu.css
│ └── style.css
└── ui/
├── channelDiv.js
├── contextMenu.js
├── myRange.js
└── siderMenu.js
================================================
FILE CONTENTS
================================================
================================================
FILE: LICENSE
================================================
GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU General Public License is a free, copyleft license for
software and other kinds of works.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
the GNU General Public License is intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users. We, the Free Software Foundation, use the
GNU General Public License for most of our software; it applies also to
any other work released this way by its authors. You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
To protect your rights, we need to prevent others from denying you
these rights or asking you to surrender the rights. Therefore, you have
certain responsibilities if you distribute copies of the software, or if
you modify it: responsibilities to respect the freedom of others.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must pass on to the recipients the same
freedoms that you received. You must make sure that they, too, receive
or can get the source code. And you must show them these terms so they
know their rights.
Developers that use the GNU GPL protect your rights with two steps:
(1) assert copyright on the software, and (2) offer you this License
giving you legal permission to copy, distribute and/or modify it.
For the developers' and authors' protection, the GPL clearly explains
that there is no warranty for this free software. For both users' and
authors' sake, the GPL requires that modified versions be marked as
changed, so that their problems will not be attributed erroneously to
authors of previous versions.
Some devices are designed to deny users access to install or run
modified versions of the software inside them, although the manufacturer
can do so. This is fundamentally incompatible with the aim of
protecting users' freedom to change the software. The systematic
pattern of such abuse occurs in the area of products for individuals to
use, which is precisely where it is most unacceptable. Therefore, we
have designed this version of the GPL to prohibit the practice for those
products. If such problems arise substantially in other domains, we
stand ready to extend this provision to those domains in future versions
of the GPL, as needed to protect the freedom of users.
Finally, every program is threatened constantly by software patents.
States should not allow patents to restrict development and use of
software on general-purpose computers, but in those that do, we wish to
avoid the special danger that patents applied to a free program could
make it effectively proprietary. To prevent this, the GPL assures that
patents cannot be used to render the program non-free.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Use with the GNU Affero General Public License.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU Affero General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the special requirements of the GNU Affero General Public License,
section 13, concerning interaction through a network will apply to the
combination as such.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If the program does terminal interaction, make it output a short
notice like this when it starts in an interactive mode:
<program> Copyright (C) <year> <name of author>
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, your program's commands
might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
<https://www.gnu.org/licenses/>.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
<https://www.gnu.org/licenses/why-not-lgpl.html>.
================================================
FILE: README.md
================================================
<div align="center">
<a href="https://madderscientist.github.io/noteDigger/" target="_blank">
<img width="240" src="./img/logo.png" alt="logo"><br>
<img width="260" src="./img/logo_text.png" alt="noteDigger"><br>
</a>
~前端辅助人工扒谱工具~
</div>
# noteDigger!
“NoteDigger”——音符挖掘者，即扒谱。模仿的是软件wavetone，但是是**双击即用**、**现代UI**的**纯前端**应用。<br>
就要重复造轮子！即：不使用框架、不使用外部库；目的是减小项目大小，并掌握各个环节。目前频谱分析的软件非常多，功能也超级强大，自知比不过……所以唯一能一战的就是项目体积了！作为一个纯前端项目，就要把易用的优点完全发扬！<br>
[在线使用](https://madderscientist.github.io/noteDigger/)<br>
视频演示（视频发布于更新节奏对齐之前）
<a href="https://www.bilibili.com/video/BV1XA4m1G7k4/" target="_blank"><img src="https://i1.hdslb.com/bfs/archive/4700166054a855cfe779f8d93f5ec1fb84293a12.jpg" alt="点击封面跳转视频" /></a>
## 使用流程
1. 在线or下载到本地，用主流现代浏览器打开(开发使用Chrome)。
2. 导入音频——文件-上传，或直接将音频拖拽进去！
3. 选择声道分析，或者导入之前分析的结果(只有选择音频之后才有导入之前结果的接口)
4. 根据频谱分析，开始绘制midi音符！可参考下文推荐流程。
5. 导出为midi等，或者暂时导出项目(下次继续)
### [*建议的扒谱流程](https://www.bilibili.com/video/BV1LFRmBcEEU) 👈视频教程
1. 导入时选择CQT。视情况勾选GPU。
2. 在“分析”页面，点击“节奏分析”、“去除谐波”、“调性分析”（建议最后一个运行）。
3. 勾选“设置”-“播放节拍”，修正节拍；接着使用小节栏右键菜单的“合并下一小节”等功能修正节奏型。
4. 点击“分析”-“人工智障扒谱”，并在完成后将对应音轨锁定、静音，作为扒谱参考。
5. 切换吸附模式为“节拍吸附”（顶部工具栏；需要保证节拍已经修得不错了）。
6. 调整顶部滑条控制频谱强度，人工完成后续扒谱。
## 导入导出说明
- 导出进度: 结果是.nd的二进制文件，保存频谱图、音符、音轨、小节等信息。导入的时候并不会强制要求匹配原曲！(会根据文件名判断一下，但不强制)
- 导出为midi: 有两个模式。模式二只保证能听，节拍默认4/4，bpm默认60，midi类型默认1(同步多音轨)，时间精度和设置的精度一致(因此如果midi先导入再导出会有量化误差)；模式一会根据小节线进行对齐(需要用户设置好小节线)，可以直接用于制谱，算法概述见下面“节奏对齐”。由于midi协议规定第十轨用于打击乐，因此扒谱时旋律需要避开第十轨，可以设置一个空音轨占位。本应用没有设计避开第十轨，也没设计扒鼓点，因此扒谱时第十轨虽然听起来还是乐音，但导出为midi后会变成鼓点。
- 导入midi: 将midi音符导入，只保证音轨、音符、音色能对应，音量默认127。如果导入后没有超过总音轨数，会在后面增加；否则会覆盖后面几轨(有提示)。
## 常规操作
- 空格: 播放
- **双击**时间轴: 从双击的位置开始播放
- 在时间轴上拖拽: 设置重复区间
- 在时间轴上拉动小节线: 设置小节bpm；若按住shift拖动，只改变该小节线位置、不影响后续小节
- 鼠标**中键**时间轴: 将时间设置到点击位置，播放状态保持上一刻
- 鼠标**右键**时间轴(上半/下半): 具体设置重复时间/小节
- 按住空白拖动: 在当前音轨绘制一个音符
- 按住音符左半边拖动: 改变位置
- 按住音符右半边拖动: 改变时长
- Ctrl+点击音符: 多选音符
- delete: 删除选中的音符
- Ctrl+滚轮: 横向缩放
- 按住**中键**拖拽、**触摸板**滑动: 移动视野
## 快捷键
只有在导入并分析音频之后才能使用这些快捷键
- Ctrl+Z: 撤销(只记录16次历史，音轨、音符、小节线操作均会被记录)
- Ctrl+Y: 重做
- Ctrl+A: 全选当前音轨
- Ctrl+Shift+A: 全选所有音轨
- Ctrl+D: 取消选中
- Ctrl+C: 复制选中的音符
- Ctrl+X: 剪贴选中的音符
- Ctrl+V: 粘贴到选中的音轨上（暂不实现跨页面粘贴）
- Ctrl+B: 呼出/收回音轨面板
- Shift+右键: 菜单，包含撤销/重做、复制/粘贴、反选当前轨、删除
- ←↑→↓: 视野移动一格
- PageUp、PageDown：向前翻页/向后翻页
- Home：设置播放位置为0，播放状态保持上一刻
## 小细节
- 滑动条，如果旁边有数字，点击就可以恢复初始值。
- 多次点击“笔”右侧的选择工具，可以切换选择模式。（注意，只能选中当前音轨的音符）
- 点击某个音符可以选中该轨。
- 选择乐器时，展开下拉框并且按首字母可以快速跳转（浏览器下拉框自带）。
- 音轨中，“闭眼”只是看不见，还是可以操作的；一般要搭配“锁定”使用，默认两者会联动。
## 支持的格式
推荐使用常见的mp3、wav文件；除此之外，视频类文件也可以使用，比如mp4、mov、m4v。
但是如下格式不支持（浏览器API不支持解析）（仅仅在Chrome浏览器尝试过）：
- aiff（苹果的音频格式）
对于ios的Safari浏览器，上传音频文件也许有些困难。可以选择视频。（不过为什么要用触屏控制啊，根本没适配）
## 其他说明
分析-自动填充，原理是将大于阈值的标记出来，效果不堪入目……于是研究并引入了基于神经网络的扒谱（分析-人工智障扒谱），但是效果非常初级。如有想法欢迎call me。
## 关于节奏对齐
若为了制谱，导出 MIDI 时，应该进行量化对齐。<br>
对齐算法需要提供精度参数 $x$，会用时长比 $\frac{\text{音符时长}}{x}$ 小的、最大的 $2^k$ 分音符（时长为四分音符的 $2^{2-k}$）对开头和结尾时刻进行量化。例如 $x=2$ 时，四分音符将按照八分音符的划分对齐、八分音符则按十六分音符的划分对齐……不过这也意味着四分音符只可能在整数倍个八分音符的时长处开始，而不可能在第3个十六分音符的时刻起音。<br>
因此，如果你保证对齐非常精准（比如使用了节拍吸附，见下文），那么 $x$ 应该调大一些；如果你大多时间用格点吸附或不吸附，那么 $x$ 应该小一些。
### 新版本
新版本新增了按小节线吸附的方式，强烈推荐使用（可通过顶部工具栏的吸附模式切换）。在该模式下，音符可以完美对齐小节线。当然，三连音这类节奏仍需借助其他模式绘制。<br>
横向缩放间距可调节吸附线的精度，从而支持任意有限位二进制小数表示的时长。这是一个很好的设计：既没有破坏以“秒”为单位的时间轴概念（下文），也兼容了小节对齐的需求。
### 老版本
很长一段时间内，noteDigger只有格点吸附。音符绘制精度与最初选择的音频分析精度一致，绘制音符大概是不能精准对齐小节线了（但是导出midi的时候可以用对齐算法），**需要强迫症忍受一下**。<br>
乐谱的基本单位是“x分音符”，而音频的基本单位是“秒”。要实现小节对齐，必须将单位统一为“x分音符”。
然而，整个程序的时间轴以“秒”为单位，这是由频谱分析决定的。若要实现类似制谱软件的对齐方式，音符绘制就必须按“x分音符”对齐。这意味着，在 120 BPM 小节下的音符，若被拖放到 60 BPM 的小节中，在以秒为单位的时间轴上，其时间会变长——Wavetone 正是这样处理的。<br>
但是对着原曲扒谱，最好还是根据"秒"来绘制音符。用 Wavetone 扒谱的体验中，我最讨厌的就是被"x分音符"限制。用秒可以保证和原曲完全贴合，使用很灵活。但是这样导出的 MIDI 不能直接制谱。按照"x分音符"来绘制音符还会导致程序很难写。开发者和使用者都不快乐。<br>
扒谱用秒为单位合适，而制谱用“x分音符”合适。为了跨越这个鸿沟，我这样设计了程序：使用midi文件作为对外的桥梁，在noteDigger内用秒为单位扒谱，导出为midi的时候根据小节进行量化，形成规整的midi用于制谱。具体实现是：在秒轴上加入小节轴，用户可以拖动小节轴的某个小节调节后面紧跟的bpm相同的小节。
> 因此老版本中量化对齐特别重要：小节线仅为视觉对齐辅助；而现在小节线以及内部的自动细分线，成为了节拍吸附模式下真正有用的吸附锚点。
## 文件结构
```
│ app.js: 最重要的文件，主程序
│ favicon.ico: 小图标
│ index.html: 程序入口, 其js主要是按钮的onclick
│ jsconfig.json: 开发用 跨文件JS解析
│ LICENSE
│ README.md
│
├─core: app.js用到的核心组件
│ app_analyser.js: 算法相关
│ app_audioplayer.js: 音频播放
│ app_beatbar.js: 小节轴
│ app_hscrollbar.js: 底部滑动条
│ app_io.js: 管理输入与输出
│ app_keyboard.js: 左侧键盘
│ app_midiaction.js: 与音符的交互
│ app_midiplayer.js: 音符播放
│ app_spectrogram.js: 频谱绘制
│ app_timebar.js: 时间轴
│
├─lib
│ beatBar.js: 节奏信息的稀疏存储
│ fakeAudio.js: 模拟了不会响的Audio，用于midi编辑器模式
│ midi.js: midi创建、解析类
│ saver.js: 二进制保存相关
│ snapshot.js: 快照类, 实现撤销和重做
│ tinySynth.js: 合成器类, 负责播放音频
|
├─ui
│ channelDiv.js: 多音轨的UI界面类, 可拖拽列表
│ contextMenu.js: 右键菜单类
│ myRange.js: 横向滑动条的封装类
│ siderMenu.js: 侧边栏菜单类
│
├─plugins
│ chordEst.js
│ pitchName.js
|
├─dataProcess
| | aboutANA.md: 自动音符对齐的数学建模
| | ANA.js: 自动音符对齐
│ │ bpmEst.js: 节奏分析
| │ analyser.js: 频域数据分析与简化
| │ fft_real.js: 执行实数FFT获取频域数据
| │ stftGPU.js: 使用WebGPU加速STFT
| | NNLS.js: 非负最小二乘 用于去除谐波
| |
| ├─AI
│ │ │ AIEntrance.js: 开启AI的worker的入口
│ │ │ basicamt_44100.onnx: 音色无关转录 神经网络模型
│ │ │ basicamt_worker.js: 音色无关转录 新线程
│ │ │ postprocess.js：神经网络结果到音符的后处理
│ │ │ septimbre_44100.onnx: 音色分离转录 神经网络模型
│ │ │ septimbre_worker.js: 音色分离转录 新线程
│ │ │ SpectralClustering.js: 音色分离转录需要的 谱聚类
| │ │
| │ └─dist: onnxruntime打包
| │ bundle.min.js
| │ ort-wasm-simd.wasm
| |
│ └─CQT
│ cqt.js: 开启worker进行后台CQT
│ cqt_worker.js: CQT类与新线程
│
├─docs
│ todo.md: 一些设计思路和权衡
│
├─img
│ bilibili-white.png
│ github-mark-white.png
│ logo-small.png
│ logo.png
│ logo_text.png
│
└─style
│ askUI.css: 达到类似<dialog>效果
│ channelDiv.css: 多音轨UI样式
│ contextMenu.css: 右键菜单样式
│ myRange.css: 包装滑动条
│ siderMenu.css: 侧边菜单样式
│ style.css: index中独立元素的样式
│
└─icon: 从阿里图标库得到的icon
iconfont.css
iconfont.ttf
iconfont.woff
iconfont.woff2
```
## 重要更新记录
### 2026 2 22
增加了和弦识别功能，并优化了项目结构。
### 2026 2 14
利用NNLS完成了谐波的去除/基频的增强。<br>
代码位于[reduceHarmonic](app_analyser.js)。原理为用谐波模板拟合频谱，进而得到谐波，并收集谐波的能量用于加强基频。该方法能大大增强基频的明显程度。
### 2026 1 28
完成了基于WebGPU的STFT、CQT计算，即使在核显上也有显著加速。美中不足的是无法得知GPU算的进度，因此交互较弱<br>
也尝试了利用CQT相位分析瞬时频率，但效果太鸡肋被放弃，相关代码在另一个分支。
### 2026 1 23
更新了自动节奏检测功能，相关功能设置在“分析-节奏分析”中。<br>
使用传统信号处理算法，**基于已经分析的频谱进行**，[原理点这](https://zhuanlan.zhihu.com/p/1995849093491222501)。目前节拍识别较为准确，但节奏型与重拍的跟踪（勾选“自动节奏型”）效果不佳。建议在CQT分析前后都进行一次（等10秒左右会发现频谱改变，改变前是STFT，改变后是CQT），基于STFT的节奏全局更准，基于CQT的局部更精确。目前的算法参数仅仅根据20帧/秒时调整，其他分辨率不保证效果。<br>
为了让人工后处理更为便捷，在节奏栏右键增加了许多功能。
### 2025 12
更新了音色无关转录模型，效果比之前好，且运行时间更短。<br>
加入了音色分离转录，但是效果不怎么好，只适用于几个音色差异较大的简单场景，不同音色的音符不能重叠。推荐用于二重奏的情况，比如钢琴伴奏的某乐器独奏。
### 2025 9 20
降低重绘频率，降低闲时CPU占用为原来的1/10。感谢[Initsnow的pr](https://github.com/madderscientist/noteDigger/pull/7)指出问题。<br>
有三类地方会触发刷新：
1. 播放时保持高刷 在 `AudioPlayer.update` 中触发刷新
2. 所有键鼠操作。最初的想法是需要的地方触发刷新，但是牵扯的非常多——
- 快捷键。有些快捷键需要触发刷新，不如一刀切。
- 鼠标操作
- 在频谱画布上的操作：`MidiAction.updateView` 触发更新还不够（只负责了音符绘制模式），还有选择区域
- 在时间轴上的动作：重复区间 -> 可以在 `TimeBar.setRepeat` 中触发更新；设置节拍 -> 可以在回调中触发更新
- 在钢琴画布上的动作：垂直方向移动，回调触发即可
- 画布之外的操作 只能设置统一的api；而音轨操作这种却不好加
- 没考虑到的功能——谁知道漏了会发生什么问题
综合考虑下，选择终极一刀切——所有用户操作都会触发刷新，仅需额外添加回调，不需要考虑具体动作，便于维护。
3. `MidiAction.updateView` 处理一切音符变化。之所以不能被键鼠操作取代，因为ai扒谱延迟修改了音符。<br>
由于`scroll2`调用了`updateView`，因此还覆盖了`resize`、Spectrogram setter
### 2025 8 21
实现了自动音符对齐(auto note alignment)，输入数字谱，得到和音频同步的音符，即“数字谱+音频→midi”。入口为：“分析”-“数字谱对齐音频”，输入单声部的数字谱，并调整八度范围，确定即可。注意，"1"对应的是C5，比常规约定高了八度，这是因为我发现往往分析泛音更为准确。如果效果不好，可以通过前后增加中/小括号实现八度升降。<br>
效果并不优秀，因为使用了传统的数学建模方法，但胜在计算量小。相关说明与建模见[aboutANA.md](./dataProcess/aboutANA.md)。<br>
2025/12/27 将此算法整理为[文章](https://zhuanlan.zhihu.com/p/1988276192063800011)。
### 2025 8 14
项目代码的整理，将功能拆分到单独文件（虽然模块间仍未解耦，但终于拆分了！），并增加了开发配置文件（感觉TS是对的啊！但放不下JS里面的奇技淫巧）。<br>
修复了陈年bug：调整小节时不时“拉不动”、STFT偏早。
### 2025 8 12
进行了频谱的归一化，便于后续的研究与分析。归一化基于能量谱，同[timbreAMT](https://github.com/madderscientist/timbreAMT/blob/main/model/CQT.py)的做法，简而言之：令每一帧的能量的方差为1。<br>
重大的改动：WASM的CQT → JS的CQT。早期实验发现两者用时接近，为了纪念第一次使用WASM加速而保留。但升级CPU后发现JS版本用时不到WASM的一半，而且代码更少、文件更小。遂欣然废除。若要学习“C++编译为WASM”可以查看此前的历史提交。
### 2025 3 25
引入了“自动音乐转录”，即“AI扒谱”，导入音频后（或进入MIDI编辑器模式）在“分析”页面点击“**人工智障扒谱**”选项，一首两分半的曲子大概需要半分钟分析。由于追求低内存开销，我没保存音频数据，因此AI扒谱前要重新选择文件。<br>
使用的模型是我毕设的一部分，设计与训练过程请查看[timbreAMT](https://github.com/madderscientist/timbreAMT)的basicamt文件夹，我称之为“音色无关转录”，即不会根据乐器种类分轨输出，但对大部分音色有适用性。对标的是basicPitch，效果接近且更加轻量，但无论是我的还是他的，结果都仅仅能听，而我的相比basicPitch优点在于集成在了noteDigger中，可以便捷地进行人工后处理，如删去多余音符、对齐节奏等。<br>
为了支持人工后处理，有了前几次更新，最重要的是：
1. 音符力度用透明度体现，便于用户看清AI扒谱结果中重要的音符。在“设置”中可以关掉透明度。
2. 音轨的锁，用于锁定AI扒谱结果，用户可以在新的音轨中“描”一遍，相当于把AI扒谱结果当做频谱。
### 2024 8 29
引入了理论上更精确的CQT分析。非file协议时（不是双击html文件打开时），当STFT（默认的计算方法）计算完成会在后台自动开启CQT计算，CQT结果将与当前频谱融合（会发现突然频谱变了）。CQT计算非常慢，因此在后台计算以防阻塞，且用C++实现、编译为WASM以提速。<br>
中途遇到很多坑，记录分布在/dataProcess/CQT的各个文件中，但效果其实并不值得这样的计算量。5分30秒的音频进行双声道CQT分析，需要45秒（从开启worker开始算），和直接进行js版的CQT用时差不多，加速了个寂寞。<br>
关于CQT的研究，记录在[《CQT：从理论到代码实现》](https://zhuanlan.zhihu.com/p/716574483)。<br>
此外尝试了“一边分析一边绘制频谱”，试图通过删除进度条达到感官上加速的效果。但是放在主线程造成严重卡顿，放弃。
### 2024 8 2
完成了issue2：不导入音频的midi编辑器。点击文件菜单下的“MIDI编辑器模式”就可以进入。<br>
视野的宽度取决于最后一个音符，模仿的是[signal](https://signal.vercel.app/edit)。也尝试过自动增加视野，可以一直往右拉，但是这样在播放的时候，开启“自动翻页”会永远停不下来（翻一页就自动拓展宽度）。<br>
扒谱框架下的midi编辑器还是有些反人类，因为绘制音符时的单位是时间而不是x分音符。不过也能用。<br>
原理是实现了一个空壳的Audio，只有计时功能，没有发声功能。一些做法写在了todo.md上。
### 2024 2 22
加入了节拍对齐功能，使用逻辑是：扒谱界面提供视觉辅助，导出midi会自动对齐，以实现制谱友好。详细对齐的原理请参看“关于节奏对齐”板块和midiExport.js文件。<br>
有一些细节：<br>
1. 如果每个小节bpm都不一样（原曲的速度不稳，有波动），那导出midi前的对齐操作会以上一小节bpm为基准进行动态适应：先根据本小节的bpm量化音符为"x分音符"，如果本小节bpm和上一小节的bpm差别在一定范围内，则再将"x分音符"的bpm设置全局量BPM；否则将全局BPM设置为当前小节的bpm。这个算法的要求是：的确要变速的前后bpm差异应该较大。<br>
2. 在一个小节内，音符的近似方法：
1. 记一个四分音符的格数为aqt（因为音符的实际使用单位是格。这里隐含了一个时间到格数的变换），某时刻t对应音符长度为ntlen，小节开始时刻记为mt。首先获取音符时刻相对小节开头的位置nt=t-mt。（音符时刻：将一个音符拆分为开始时刻和结束时刻。一个音符可能跨好几个小节，因此这样处理最为合适）
2. 假设前提：时长长的音符的起点和终点的精度也低（精度这里指最小单位时长，低精度指单位时长对应的实际时间长）。因此近似精度accu采用自适应的方式：该音符可以用(ntlen/aqt)个四份音符表示，设其可以用一个(4*2^n)分音符近似，其中n满足：(1/2)^n<=ntlen/aqt<(1/2)^(n-1)，则该音符的时长为aqt/(2^n)，则精度设置为这个近似音符的一半：accu = aqt/(2^(n+1))。比如四份音符的精度是一个八分音符的时长。
3. 近似后的时刻为：round(nt/accu)*accu。同时设置一个最低精度：八分音符。因此accu=min(aqt/2, aqt/(2^(n+1)))，其中(1/2)^n<=ntlen/aqt<(1/2)^(n-1)。
3. 小节信息如何存储、数据结构如何设计需要好好想想。大部分情况下（在原音频节奏稳定的情况下）只会变速几次，此时存变动时刻的bpm值就足矣。极端情况下每个小节都单独设置了bpm。如何设计数据结构能在两种情况下都取得较好的性能？使用稀疏数组。
### 2024 2 9
在今年完成了所有基本功能！本次更新了设置相关，简单地设计了调性分析的算法，已经完全可以用了！【随后在bilibil投稿了视频】
### 2024 2 8
文件系统已经完善！已经可以随心所欲导入导出保存啦！同时修复了一些小bug、完善了一些api。<br>
界面上，本打算将文件相关选项放到logo上，但是侧边菜单似乎有些空了，于是就加入到侧边栏，而logo设置为刷新或开新界面（考察了其他网站的logo的用途）。同时给侧边菜单加入了“设置”和“分析”，但本次更新没做。<br>
midi相关操作来自[我的另一个项目](https://github.com/madderscientist/je_score_operator)的midi类。将用midi转的wav导入分析，再导入原midi，两者同步播放的感觉真好！
### 2024 2 5
已经能用于扒谱了！完成了midi和原曲的播放与同步，填补了扒谱过程最重要的一环。<br>
UI基本完成！将侧边栏、滑动条封装成了js类。在此基础上，设计了类似VScode的菜单，用于存放不常用的功能和界面；而顶部窄窄一条用于放置常用功能。<br>
此外，完成了logo的设计。在2月4日的commit记录中(因为现在已经删除)可以看到设计的多种logo，最终选定了“在勺子里的音符”，这是一个被勺子dig出来的音符。其他思路可以概括为：“音符和铲子的组合”(logo2)、“埋在地里的音符”(logo5 logo6)、“像植物一样生长的八分音符”(logo8 logo10)、“音符和铲子结合”(logo12)。
### 2024 2 1
完成了多音轨、合成器和主线的整合，象征着midi系统的完成！<br>
统一了UI风格；完善了快捷键功能；新增框选功能；修复了大部分bug。
### 2024 1 30
完成了midi合成器tinySynth.js，实现了128种音色的播放。只有演奏音符的作用，控制器一点没做。<br>
原理是多个基础波形合成一个音色。波形参数来自 https://github.com/g200kg/webaudio-tinysynth ，因此程序设计也参考了它的设计。修改记录在todo.md中<br>
对于reference的解析（作者注释一点没写，变量命名极为简单，因此主要是变量解释）存放于“./tone/解析.md”(文件夹已被删除，请去历史提交查看)。文件夹中还有tinySynth的测试页面。在下一次push时将删除tone文件夹。<br>
这段时间内还完成了以下内容（全部记录在commit history的comments内）：
- 基本程序界面(三个画布：键盘、时频图、时间轴；UI界面：右键菜单、多音轨、滑动条)
- 基本逻辑功能：音符交互绘制、快捷键以及模块的关联协同
### 2023 12 13
从11月14日开始造js版fft轮子起，时隔一个月第一次提交项目，因为项目逻辑日渐复杂，需要能及时回退。主要完成了频谱绘制、钢琴键盘绘制、数据处理三部分，并初步确定了程序的结构框架。<br>
数据处理核心：实数FFT，编写于我《数字信号处理》刚刚学完FFT算法之时，针对本项目的应用场景做了专门的设计，即针对音频STFT做了适配，具体表现为：实数加速、数据预计算、空间预分配、共用数组。<br>
由于整个项目还没搭建起来，因此不能测试NoteAnalyser类的数据处理效果。此类用于将频域数据进一步离散为音符强度数据。<br>
关于程序结构有一版废案，在文件夹"deprecated"中，设计思路是解耦、插件化，废弃理由是根本解耦不了。因此现在的代码耦合成一坨了。这个文件夹将在下一次push时被删除，存活于历史提交之中。<br>
tone文件夹将存放我的合成器轮子，audioplaytest是我音频播放的实验文件夹，todo.md是部分设计思路。<br>
2024/4/8补记：时频分析方法是STFT，但是面临时间和频率分辨率矛盾的问题，现在的分析精度只能到F#2。解决办法是用小波变换，或者更本质一点：用84个滤波器提取84个基准音以及其周围的频率的能量。这样能达到更高的频率分辨率和时间分辨率。但是现在的STFT用起来效果还可以，就不换了哈。
================================================
FILE: app.js
================================================
// 用这种方式(原始构造函数)的原因：解耦太难了，不解了。this全部指同一个。其次为了保证效率
// 防止在html初始化之前getElement，所以封装成了构造函数，而不是直接写obj
function App() {
this.event = new EventTarget();
// 键盘和时间轴 其绘制由工作区管理
this.keyboard = document.getElementById('piano');
this.keyboard.ctx = this.keyboard.getContext('2d', { alpha: false, desynchronized: true });
this.timeBar = document.getElementById('timeBar');
this.timeBar.ctx = this.timeBar.getContext('2d', { alpha: false, desynchronized: true });
// 工作区图层
this.layerContainer = document.getElementById('main-layers');
this.layers = this.layerContainer.layers = {
spectrum: LayeredCanvas.new2d('spectrum', false),
action: LayeredCanvas.new2d('actions', true)
};
Object.defineProperty(this.layers, 'width', {
get: function () { return this.spectrum.width; },
set: function (w) {
for (const c in this) this[c].width = w;
}, enumerable: false
});
Object.defineProperty(this.layers, 'height', {
get: function () { return this.spectrum.height; },
set: function (h) {
for (const c in this) this[c].height = h;
}, enumerable: false
});
Object.defineProperty(this.layers, 'addLayer', {
value: (name, zIndex) => {
if (this.layers[name]) return this.layers[name];
const canvas = document.createElement('canvas');
canvas.style.zIndex = zIndex;
this.layerContainer.appendChild(canvas);
return this.layers[name] = LayeredCanvas.new2d(canvas, true);
}, enumerable: false
});
Object.defineProperty(this.layers, 'removeLayer', {
value: (name, zIndex) => {
if (this.layers[name]) {
this.layers[name].canvas.remove();
delete this.layers[name];
}
}, enumerable: false
});
this.layers.action.mask = '#25262da8';
Object.defineProperty(this.layers.action, 'Alpha', {
get: function() {
return parseInt(this.mask.substring(7), 16);
},
set: function(a) {
a = Math.min(255, Math.max(a | 0, 0));
this.mask = '#25262d' + a.toString(16).padStart(2, '0');
this.dirty = true;
}, enumerable: false
});
this.midiMode = false;
this.TperP = -1; // 每个像素代表的ms
this.PperT = -1; // 每ms代表的像素
this._width = 5; // 每格的宽度
Object.defineProperty(this, 'width', {
get: function () { return this._width; },
set: function (w) {
if (w <= 0) return;
this._width = w;
this.TimeBar.updateInterval();
this.HscrollBar.refreshSize(); // 刷新横向滑动条
this.TperP = this.dt / this._width;
this.PperT = this._width / this.dt;
this.Spectrogram.onresize();
}
});
this._height = 15; // 每格的高度
Object.defineProperty(this, 'height', {
get: function () { return this._height; },
set: function (h) {
if (h <= 0) return;
this._height = h;
this.Keyboard.setYchange(h);
this.keyboard.ctx.font = `${h + 2}px Arial`;
this.layers.action.ctx.font = `${h}px Arial`;
this.Spectrogram.onresize();
}
});
this.ynum = 84; // 一共84个按键
this._xnum = 0; // 时间轴的最大长度
Object.defineProperty(this, 'xnum', { // midi模式下需要经常改变此值，故特设setter
get: function () { return this._xnum; },
set: function (n) {
if (n <= 0) return;
this._xnum = n;
// 刷新横向滑动条
this.HscrollBar.refreshPosition();
this.HscrollBar.refreshSize();
this.idXend = Math.min(this._xnum, Math.ceil((this.scrollX + this.layers.width) / this._width));
}
});
this.dt = 50; // 每次分析的时间间隔 单位毫秒 在this.Analyser.stft this.io.onfile this.io.projFile.import 中更新
this.time = -1; // 当前时间 单位：毫秒 在this.AudioPlayer.update中更新
// 以下变量仅在scroll2中更新(特别标记的除外)
this.scrollX = 0; // 视野左边和世界左边的距离
this.scrollY = 0; // 视野下边和世界下边的距离
this.idXstart = 0; // 开始的X序号
this.idYstart = 0; // 开始的Y序号
this.idXend = 0; // 【还在 xnum setter 中更新】
this.idYend = 0;
this.rectXstart = 0;// 目前只有Spectrogram.update在使用
this.rectYstart = 0;// 画布开始的具体y坐标(因为最下面一个不完整) 迭代应该减height 被画频谱、画键盘共享
// spectrum的重绘仅在 视野滚动(scroll2) 数据改变(会触发scroll2) 倍率改变
// 下面的函数控制action层的重绘 重绘时机: scroll2; AudioPlayer.update; 键鼠操作
this.makeActDirty = () => { this.layers.action.dirty = true; }; // 供外部调用
/**
* 设置播放时间 如果立即播放(keep==false)则有优化
* @param {number} t 时间点 单位：毫秒
* @param {boolean} keep 是否保存之前的状态 如果为false则立即开始
*/
this.setTime = (t, keep = true) => {
this.synthesizer.stopAll();
if (keep) {
this.time = t;
this.AudioPlayer.audio.currentTime = t / 1000;
this.AudioPlayer.play_btn.firstChild.textContent = this.TimeBar.msToClockString(t);
this.MidiPlayer.restart();
} else { // 用于双击时间轴立即播放
this.AudioPlayer.start(t); // 所有操作都在start中
}
};
this._mouseY = 0; // 鼠标当前y坐标
Object.defineProperty(this, 'mouseY', {
get: function () { return this._mouseY; },
set: function (y) {
this._mouseY = y;
this.Keyboard.highlight = Math.floor((this.scrollY + this.layers.height - y) / this._height) + 24;
}
});
this._mouseX = 0; // 鼠标当前x坐标
Object.defineProperty(this, 'mouseX', {
get: function () { return this._mouseX; },
set: function (x) {
this._mouseX = x;
this.MidiAction.frameXid = Math.floor((x + this.scrollX) / this._width);
}
});
this.preventShortCut = false; // 当需要原始快捷键时(比如输入框)修改此为true
this.audioContext = new AudioContext({ sampleRate: 44100 });
this.synthesizer = new TinySynth(this.audioContext);
this.Spectrogram = new _Spectrogram(this);
this.MidiAction = new _MidiAction(this);
this.MidiPlayer = new _MidiPlayer(this);
this.AudioPlayer = new _AudioPlayer(this);
this.Keyboard = new _Keyboard(this); this.height = this._height; // 更新this.Keyboard._ychange
this.TimeBar = new _TimeBar(this);
this.BeatBar = new _BeatBar(this);
// 撤销相关
this.snapshot = new Snapshot(16, {
// 用对象包裹，实现字符串的引用
midi: { value: JSON.stringify(this.MidiAction.midi) }, // 音符移动、长度改变、channel改变后
channel: { value: JSON.stringify(this.MidiAction.channelDiv.channel) }, // 音轨改变序号、增删、修改参数后
beat: { value: JSON.stringify(this.BeatBar.beats) }
});
// changed = channel变化<<1 | midi变化<<2 | beat变化<<3
this.snapshot.save = (changed = 0b111) => {
const nowState = this.snapshot.nowState();
const lastStateNotExists = nowState == null;
this.snapshot.add({
channel: (lastStateNotExists || (changed & 0b1)) ? { value: JSON.stringify(this.MidiAction.channelDiv.channel) } : nowState.channel,
midi: (lastStateNotExists || (changed & 0b10)) ? { value: JSON.stringify(this.MidiAction.midi) } : nowState.midi,
beat: (lastStateNotExists || (changed & 0b100)) ? { value: JSON.stringify(this.BeatBar.beats) } : nowState.beat
});
};
this.HscrollBar = new _HscrollBar(this);
this._copy = ''; // 用于复制音符 会是JSON字符串
this.shortcutActions = { // 快捷键动作
'Ctrl+Z': () => { // 撤销
let lastState = this.snapshot.undo();
if (!lastState) return;
this.MidiAction.midi = JSON.parse(lastState.midi.value);
this.MidiAction.selected = this.MidiAction.midi.filter((obj) => obj.selected);
this.MidiAction.channelDiv.fromArray(JSON.parse(lastState.channel.value));
this.BeatBar.beats.copy(JSON.parse(lastState.beat.value));
this.MidiAction.updateView();
},
'Ctrl+Y': () => {
let nextState = this.snapshot.redo();
if (!nextState) return;
this.MidiAction.midi = JSON.parse(nextState.midi.value);
this.MidiAction.selected = this.MidiAction.midi.filter((obj) => obj.selected);
this.MidiAction.channelDiv.fromArray(JSON.parse(nextState.channel.value));
this.BeatBar.beats.copy(JSON.parse(nextState.beat.value));
this.MidiAction.updateView();
},
'Ctrl+A': () => { // 选中该通道的所有音符
let ch = this.MidiAction.channelDiv.selected;
if (ch) {
ch = ch.index;
this.MidiAction.midi.forEach((note) => {
note.selected = note.ch == ch;
});
this.MidiAction.selected = this.MidiAction.midi.filter((nt) => nt.selected);
} else this.shortcutActions['Ctrl+Shift+A']();
},
'Ctrl+Shift+A': () => { // 真正意义上的全选
this.MidiAction.midi.forEach((note) => {
note.selected = true;
});
this.MidiAction.selected = [...this.MidiAction.midi];
},
'Ctrl+D': () => { // 取消选中
this.MidiAction.clearSelected();
},
'Ctrl+C': () => {
if (this.MidiAction.selected.length == 0) return;
this._copy = JSON.stringify(this.MidiAction.selected);
},
'Ctrl+X': () => {
if (this.MidiAction.selected.length == 0) return;
this._copy = JSON.stringify(this.MidiAction.selected);
this.MidiAction.deleteNote(); // deleteNote会更新view和存档
},
'Ctrl+V': () => {
if (!this._copy) return; // 空字符串或null
const ch = this.MidiAction.channelDiv.selected;
if (!ch) { alert("请先选择一个音轨!"); return; }
let chid = ch.index;
let copy = JSON.parse(this._copy);
// 找到第一个
let minX = Infinity;
copy.forEach((note) => {
note.ch = chid;
note.selected = true;
if (note.x1 < minX) minX = note.x1;
});
this.MidiAction.clearSelected();
this.MidiAction.selected = copy;
// 粘贴到光标位置
minX = (this.time / this.dt - minX) | 0;
copy.forEach((note) => {
note.x1 += minX;
note.x2 += minX;
});
this.MidiAction.midi.push(...copy);
this.MidiAction.midi.sort((a, b) => a.x1 - b.x1);
this.MidiAction.updateView();
this.snapshot.save(0b10); // 只保存midi的快照
},
'Ctrl+B': () => { // 收回面板
const channelDiv = this.MidiAction.channelDiv.container.parentNode;
if (channelDiv.style.display == 'none') {
channelDiv.style.display = 'block';
} else {
channelDiv.style.display = 'none';
} this.resize();
}
};
/**
* 改变工作区(频谱、键盘、时间轴)大小
* @param {number} w 工作区的新宽度 默认充满父容器
* @param {number} h 工作区的新高度 默认充满父容器
* 充满父容器，父容器需设置flex:1;overflow:hidden;
*/
this.resize = (w = undefined, h = undefined) => {
const box = document.getElementById('Canvases-Container').getBoundingClientRect();
w = w || box.width;
h = h || box.height;
let spectrumWidth, spectrumHeight;
if (w > 80) {
spectrumWidth = w - 80;
this.keyboard.width = 80;
} else {
spectrumWidth = 0.4 * w;
this.keyboard.width = 0.6 * w;
}
if (h > 40) {
spectrumHeight = h - 40;
this.timeBar.height = 40;
} else {
spectrumHeight = 0.4 * h;
this.timeBar.height = 0.6 * h;
}
this.keyboard.height = spectrumHeight;
this.timeBar.width = spectrumWidth;
for (const c in this.layers) {
const canvas = this.layers[c];
canvas.width = spectrumWidth;
canvas.height = spectrumHeight;
canvas.ctx.lineWidth = 1;
canvas.ctx.font = `${this._height}px Arial`;
}
document.getElementById('play-btn').style.width = this.keyboard.width + 'px';
// 改变画布长宽之后，设置的值会重置，需要重新设置
this.keyboard.ctx.lineWidth = 1; this.keyboard.ctx.font = `${this._height + 2}px Arial`;
this.timeBar.ctx.font = '14px Arial';
// 触发滑动条/频谱缓冲区更新 还能在初始化的时候保证timeBar的文字间隔
this.width = this._width;
this.scroll2();
};
/**
* 移动到 scroll to (x, y)
* 由目标位置得到合法的scrollX和scrollY，并更新XY方向的scroll离散值起点(序号)
* @param {number} x 新视野左边和世界左边的距离
* @param {number} y 新视野下边和世界下边的距离
*/
this.scroll2 = (x = this.scrollX, y = this.scrollY) => {
this.scrollX = Math.max(0, Math.min(x, this._width * this._xnum - this.layers.width));
this.scrollY = Math.max(0, Math.min(y, this._height * this.ynum - this.layers.height));
this.idXstart = (this.scrollX / this._width) | 0;
this.idYstart = (this.scrollY / this._height) | 0;
this.idXend = Math.min(this._xnum, Math.ceil((this.scrollX + this.layers.width) / this._width));
this.idYend = Math.min(this.ynum, Math.ceil((this.scrollY + this.layers.height) / this._height));
this.rectXstart = this.idXstart * this._width - this.scrollX;
this.rectYstart = this.layers.height - this.idYstart * this._height + this.scrollY; // 画图的y从左上角开始
// 滑动条
this.HscrollBar.refreshPosition();
// 更新音符 action.dirty 置位
this.MidiAction.updateView();
this.layers.spectrum.dirty = true;
};
/**
* 按倍数横向缩放时频图 以鼠标指针为中心
* @param {number} mouseX
* @param {number} times 倍数 比用加减像素好，更连续
*/
this.scaleX = (mouseX, times) => {
let nw = this._width * times;
if (nw < 1) return;
if (nw > this.layers.spectrum.width) return;
this.width = nw;
this.scroll2((this.scrollX + mouseX) * times - mouseX, this.scrollY);
};
/**
* 状态更新与重绘
*/
this.update = () => {
this.AudioPlayer.update();
this.MidiPlayer.update();
for (const c in this.layers) this.layers[c].render()
};
this.layers.spectrum.resetHandlers([this.Spectrogram.render]);
this.layers.action.resetHandlers([
c => { // 绘制遮罩
let ctx = c.ctx;
ctx.globalCompositeOperation = 'copy';
ctx.fillStyle = c.mask;
ctx.fillRect(0, 0, c.width, c.height);
ctx.globalCompositeOperation = 'source-over';
},
this.Keyboard.render,// 应最先 因为高亮显示不重要应该在最下面
this.BeatBar.render,
this.MidiAction.render,// 应在BeatBar之后 节拍线在音符下面
this.TimeBar.render,// 应最后 时间指针应该在最上面
]);
this.trackMouseY = (e) => { // onmousemove
this.mouseY = e.offsetY;
};
this.trackMouseX = (e) => { // 用于框选，会更新frameX值 在this.MidiAction中add和remove事件监听器
this.mouseX = e.offsetX;
};
this._trackMouseX = (e) => {// 给pitchName插件专用的 只会更新_mouseX
this._mouseX = e.offsetX;
};
/**
* 动画循环绘制
* @param {boolean} loop 是否开启循环
*/
this.loopUpdate = (loop = true) => {
if (loop) {
const update = (t) => {
this.update();
this.loop = requestAnimationFrame(update);
}; // 必须用箭头函数包裹，以固定this的指向
this.loop = requestAnimationFrame(update);
} else {
cancelAnimationFrame(this.loop);
}
};
this.loop = 0; // 接收requestAnimationFrame的返回
//=========数据解析相关=========//
this.Analyser = new _Analyser(this);
//========= 导入导出 =========//
this.io = new _IO(this);
//========= 事件注册 =========//
document.getElementById('speedControl').addEventListener('input', (e) => { // 变速
this.AudioPlayer.audio.playbackRate = parseFloat(e.target.value);
});
document.getElementById('multiControl').addEventListener('input', (e) => { // 变画频谱的倍率
this.Spectrogram.multiple = parseFloat(e.target.value);
});
document.getElementById('contrastControl').addEventListener('input', (e) => { // 变频谱对比度
this.Spectrogram.contrast = parseFloat(e.target.value);
});
document.getElementById('midivolumeControl').addEventListener('input', (e) => { // midi音量
this.synthesizer.out.gain.value = parseFloat(e.target.value) ** 2;
});
document.getElementById('audiovolumeControl').addEventListener('input', (e) => {// 音频音量
this.AudioPlayer.audio.volume = parseFloat(e.target.value);
});
document.addEventListener('keydown', (e) => { // 键盘事件
// 以下在没有频谱数据时不启用
if (this.preventShortCut) return;
if (!this.Spectrogram._spectrogram) return;
let shortcut = '';
// 检测平台并使用相应的修饰键
const isMac = navigator.platform.toUpperCase().indexOf('MAC') >= 0;
const cmdKey = isMac ? e.metaKey : e.ctrlKey;
const ctrlKey = isMac ? e.ctrlKey : false;
if (cmdKey) shortcut += 'Ctrl+'; // 统一使用Ctrl+标识符，但实际检测平台对应的键
if (ctrlKey && isMac) shortcut += 'RealCtrl+'; // Mac上的真正Ctrl键
if (e.shiftKey) shortcut += 'Shift+';
if (e.altKey) shortcut += 'Alt+';
if (shortcut != '') { // 组合键
shortcut += e.key.toUpperCase(); // 大小写一视同仁
if (this.shortcutActions.hasOwnProperty(shortcut)) {
e.preventDefault(); // 阻止默认的快捷键行为
this.shortcutActions[shortcut]();
}
} else { // 单个按键
switch (e.key) {
case 'ArrowUp': this.scroll2(this.scrollX, this.scrollY + this._height); break;
case 'ArrowDown': this.scroll2(this.scrollX, this.scrollY - this._height); break;
case 'ArrowLeft': this.scroll2(this.scrollX - this._width, this.scrollY); break;
case 'ArrowRight': this.scroll2(this.scrollX + this._width, this.scrollY); break;
case 'Delete': this.MidiAction.deleteNote(); break;
case ' ': this.AudioPlayer.play_btn.click(); break;
case 'PageUp': this.scroll2(this.scrollX - this.layers.spectrum.width, this.scrollY); break;
case 'PageDown': this.scroll2(this.scrollX + this.layers.spectrum.width, this.scrollY); break;
case 'Home': this.scroll2(0); this.setTime(0); break;
}
}
});
// audio可以后台播放，但是requestAnimationFrame不行，而时间同步在requestAnimationFrame中
// 还有一个办法：在可见状态变化时，将update绑定到audio.ontimeupdate上，但是这个事件触发频率很低，而预测器根据60帧设计的
document.addEventListener('visibilitychange', () => {
if (document.hidden) this.AudioPlayer.stop();
});
window.addEventListener('load', () => { this.resize(); });
window.addEventListener('resize', () => { this.resize(); });
this.layerContainer.addEventListener('wheel', (e) => {
// e.deltaY 往前滚是负数
const isMac = navigator.platform.toUpperCase().indexOf('MAC') >= 0;
const cmdKey = isMac ? e.metaKey : e.ctrlKey;
if (cmdKey) { // 缩放
e.preventDefault();
this.scaleX(e.offsetX, e.deltaY > 0 ? 0.8 : 1.25);
} else if (e.shiftKey) { // 垂直滚动
// 只有鼠标滚轮时是有deltaY。所以这里让X方向能移动，做法是交换X和Y
this.scroll2(this.scrollX + e.deltaY, this.scrollY + e.deltaX);
} else { // 触摸板的滑动也是wheel
this.scroll2(this.scrollX + e.deltaX, this.scrollY - e.deltaY);
} // 只改状态，但不绘图。绘图交给固定时间刷新完成
this.trackMouseY(e);
});
this.layerContainer.contextMenu = new ContextMenu([
{
name: "撤销", callback: () => {
this.shortcutActions['Ctrl+Z']();
}, onshow: () => this.Spectrogram._spectrogram && this.snapshot.lastState()
}, {
name: "重做", callback: () => {
this.shortcutActions['Ctrl+Y']();
}, onshow: () => this.Spectrogram._spectrogram && this.snapshot.nextState()
}, {
name: "粘贴", callback: () => {
this.shortcutActions['Ctrl+V']();
}, onshow: () => this.Spectrogram._spectrogram && this._copy != ''
}, {
name: "复制", callback: () => {
this.shortcutActions['Ctrl+C']();
}, onshow: () => this.Spectrogram._spectrogram && this.MidiAction.selected.length > 0
}, {
name: "反选", callback: () => {
let ch = this.MidiAction.channelDiv.selected;
let id = ch && ch.index;
ch = !ch;
for (const nt of this.MidiAction.midi)
nt.selected = (ch || nt.ch == id) && !nt.selected;
this.MidiAction.selected = this.MidiAction.midi.filter(nt => nt.selected);
}, onshow: () => this.Spectrogram._spectrogram
}, {
name: '<span style="color: red;">删除</span>', callback: () => {
this.MidiAction.deleteNote();
}, onshow: () => this.Spectrogram._spectrogram && this.MidiAction.selected.length > 0
}
]);
this.layerContainer.addEventListener('mousedown', (e) => {
if (e.button == 1) { // 中键按下 动作同触摸板滑动 视窗移动
const moveWindow = (e) => {
this.scroll2(this.scrollX - e.movementX, this.scrollY + e.movementY);
}; this.layerContainer.addEventListener('mousemove', moveWindow);
const up = () => {
this.layerContainer.removeEventListener('mousemove', moveWindow);
document.removeEventListener('mouseup', up);
}; document.addEventListener('mouseup', up);
return;
}
// 以下在没有频谱数据时不启用
if (this.Spectrogram._spectrogram) {
if (e.button == 0) this.MidiAction.onclick_L(e); // midi音符相关
else if (e.button == 2 && e.shiftKey) {
this.layerContainer.contextMenu.show(e);
e.stopPropagation();
} else this.MidiAction.clearSelected(); // 取消音符选中
} this.Keyboard.mousedown(); // 将发声放到后面，因为onclick_L会改变选中的音轨
});
this.layerContainer.addEventListener('mousemove', this.trackMouseY);
this.layerContainer.addEventListener('contextmenu', (e) => { e.preventDefault(); e.stopPropagation(); });
this.timeBar.addEventListener('dblclick', (e) => {
if (this.AudioPlayer.audio.readyState != 4) return;
this.setTime((e.offsetX + this.scrollX) * this.AudioPlayer.audio.duration * 1000 / (this._xnum * this._width), false);
});
this.timeBar.addEventListener('contextmenu', (e) => {
e.preventDefault(); // 右键菜单
if (e.offsetY < this.timeBar.height >> 1) this.TimeBar.contextMenu.show(e);
else this.BeatBar.contextMenu.show(e);
e.stopPropagation();
});
this.timeBar.addEventListener('mousemove', this.BeatBar.moveCatch);
this.timeBar.addEventListener('mousedown', (e) => {
switch (e.button) {
case 0:
if (this.BeatBar.belongID > -1) { // 在小节轴上
let _anyAction = false; // 是否存档
this.timeBar.removeEventListener('mousemove', this.BeatBar.moveCatch);
const m = this.BeatBar.beats.setMeasure(this.BeatBar.belongID, undefined, false);
const startAt = m.start * this.PperT;
let setMeasure;
if (e.shiftKey) { // 只改变小节线位置
const nextM = this.BeatBar.beats.setMeasure(m.id + 1, undefined, false);
this.BeatBar.beats.setMeasure(m.id + 2, undefined, false); // 下下个也要创建
setMeasure = (e2) => {
_anyAction = true;
m.interval = Math.max(100, (e2.offsetX + this.scrollX - startAt) * this.TperP);
nextM.interval -= m.start + m.interval - nextM.start;
this.BeatBar.beats.check(false);
};
} else { // 改变小节线位置并移动后续小节
setMeasure = (e2) => {
_anyAction = true;
m.interval = Math.max(100, (e2.offsetX + this.scrollX - startAt) * this.TperP);
this.BeatBar.beats.check(false); // 关闭小节合并 否则会丢失小节对象
};
}
let removeEvents = () => {
document.removeEventListener('mousemove', setMeasure);
this.timeBar.addEventListener('mousemove', this.BeatBar.moveCatch);
document.removeEventListener('mouseup', removeEvents);
this.BeatBar.beats.check(true);
if (_anyAction) this.snapshot.save(0b100);
};
document.addEventListener('mousemove', setMeasure);
document.addEventListener('mouseup', removeEvents);
} else {
const x = (e.offsetX + this.scrollX) / this._width * this.dt; // 毫秒数
const originStart = this.TimeBar.repeatStart;
const originEnd = this.TimeBar.repeatEnd;
const mouseDownX = e.offsetX;
let mouseUpX = mouseDownX;
const setRepeat = (e) => {
mouseUpX = e.offsetX;
const newX = (e.offsetX + this.scrollX) / this._width * this.dt;
if (newX > x) this.TimeBar.setRepeat(x, newX);
else this.TimeBar.setRepeat(newX, x);
};
let removeEvents = () => {
this.timeBar.removeEventListener('mousemove', setRepeat);
document.removeEventListener('mouseup', removeEvents);
// 有时候双击的小小移动会误触重复区间 所以如果区间太小则忽视
if (Math.abs(mouseUpX - mouseDownX) < 6) this.TimeBar.setRepeat(originStart, originEnd);
};
this.timeBar.addEventListener('mousemove', setRepeat);
document.addEventListener('mouseup', removeEvents);
}
break;
case 1: // 中键跳转位置但不改变播放状态
this.setTime((e.offsetX + this.scrollX) / this._width * this.dt);
break;
}
});
this.keyboard.addEventListener('wheel', (e) => {
this.scroll2(this.scrollX, this.scrollY - e.deltaY); // 只能上下移动
});
this.keyboard.addEventListener('mousemove', this.trackMouseY);
this.keyboard.addEventListener('contextmenu', (e) => { e.preventDefault(); e.stopPropagation(); });
this.keyboard.addEventListener('mousedown', (e) => {
if (e.button == 1) { // 中键按下 动作同触摸板滑动 视窗移动
const moveWindow = (e) => {
this.scroll2(this.scrollX, this.scrollY + e.movementY);
}; this.keyboard.addEventListener('mousemove', moveWindow);
const up = () => {
this.keyboard.removeEventListener('mousemove', moveWindow);
document.removeEventListener('mouseup', up);
}; document.addEventListener('mouseup', up);
return;
} this.Keyboard.mousedown();
});
// 用户鼠标操作触发刷新
document.addEventListener('mousemove', this.makeActDirty);
document.addEventListener('mousedown', this.makeActDirty);
document.addEventListener('mouseup', this.makeActDirty);
document.addEventListener('keydown', this.makeActDirty);
// wheel->scroll2 已触发刷新
this.loopUpdate(true);
}
class LayeredCanvas extends HTMLCanvasElement {
static new2d(canvas, alpha = true, desynchronized = true) {
if (typeof canvas === 'string') canvas = document.getElementById(canvas);
return LayeredCanvas.new(canvas, '2d', {alpha, desynchronized, willReadFrequently: false});
}
static new(canvas, contextType = '2d', contextAttributes) {
Object.setPrototypeOf(canvas, LayeredCanvas.prototype);
canvas.init(contextType, contextAttributes);
return canvas;
}
init(contextType, contextAttributes) {
this.ctx = this.getContext(contextType, contextAttributes);
this.handlers = []; // [{handler, priority}]
this.dirty = true;
}
resetHandlers(handlers) {
this.handlers = handlers
.filter(handler => typeof handler === 'function')
.map((handler, i) => ({handler, priority: i}));
}
/**
* 注册渲染函数
* @param {function(LayeredCanvas): void} handler
* @param {number} priority 优先级越小越先执行
*/
register(handler, priority = null) {
if (priority === null) {
if (this.handlers.length)
priority = this.handlers[this.handlers.length - 1].priority + 1;
else priority = 0;
}
this.handlers.push({handler, priority});
this.handlers.sort((a, b) => a.priority - b.priority);
}
unregister(handler) {
this.handlers = this.handlers.filter(h => h.handler !== handler);
}
render() {
if (!this.dirty) return;
for (const {handler} of this.handlers) handler(this);
this.dirty = false;
}
}
/*
dom needed:
#Canvases-Container div 决定画布尺寸
#piano canvas 画琴键
#timeBar canvas 画时间轴
#main-layers div 主工作区图层容器
#spectrum canvas 画频谱
#actions canvas 画其余
#funcSider div 音轨选择的容器
#speedControl input[type=range] 变速
#multiControl input[type=range] 改变画频谱的倍率
#contrastControl input[type=range] 改变画频谱的对比度
#midivolumeControl input[type=range] midi音量
#play-btn button 播放
#actMode div 动作模式选择，其下有两个btn
#scrollbar-track div 滑动条轨道
#scrollbar-thumb div 滑动条
*/
================================================
FILE: core/app_analyser.js
================================================
/// <reference path="../dataProcess/fft_real.js" />
/// <reference path="../dataProcess/stftGPU.js" />
/// <reference path="../dataProcess/analyser.js" />
/// <reference path="../dataProcess/CQT/cqt.js" />
/// <reference path="../dataProcess/AI/AIEntrance.js" />
/// <reference path="../dataProcess/ANA.js" />
/// <reference path="../dataProcess/bpmEst.js" />
/// <reference path="../dataProcess/NNLS.js" />
/**
* 数据解析相关算法
* @param {App} parent
*/
function _Analyser(parent) {
/**
* @param {AudioBuffer} audioBuffer 音频缓冲区
* @param {number} channel 选择哪个channel分析 0:left 1:right 2:l+r 3:l-r else:fft(l)+fft(r)
* @returns {Array<Float32Array>} 选择的channel数据 .sampleRate为采样率
*/
this.selectChannel = (audioBuffer, channel) => {
const channels = [];
channels.sampleRate = audioBuffer.sampleRate;
switch (channel) {
case 0: channels.push(audioBuffer.getChannelData(0)); break;
case 1: channels.push(audioBuffer.getChannelData(audioBuffer.numberOfChannels - 1)); break;
case 2: { // L+R
let length = audioBuffer.length;
let timeDomain = audioBuffer.getChannelData(0);
if (audioBuffer.numberOfChannels > 1) {
timeDomain = new Float32Array(timeDomain);
let channelData = audioBuffer.getChannelData(1);
for (let i = 0; i < length; i++) timeDomain[i] += channelData[i];
} channels.push(timeDomain); break;
}
case 3: { // L-R
let length = audioBuffer.length;
let timeDomain = audioBuffer.getChannelData(0);
if (audioBuffer.numberOfChannels > 1) {
timeDomain = new Float32Array(timeDomain);
let channelData = audioBuffer.getChannelData(1);
for (let i = 0; i < length; i++) timeDomain[i] -= channelData[i];
} channels.push(timeDomain); break;
}
default: { // fft(L)+fft(R)
for (let c = 0; c < audioBuffer.numberOfChannels; c++)
channels.push(audioBuffer.getChannelData(c));
break;
}
} return channels;
};
/**
* 对channels执行STFT
* @param {Array<Float32Array>} channels 通道数据数组，每个元素为一个channel的数据 .sampleRate为采样率
* @param {number} tNum 一秒几次分析 决定步距
* @param {number} A4 频率表的A4频率
* @param {number} fftPoints 实数fft点数
* @param {boolean} useGPU 是否使用GPU加速
* @returns {Promise<Array<Float32Array>>} 时频谱数据
*/
this.stft = async (channels, tNum = 20, A4 = 440, fftPoints = 8192, useGPU = true) => {// 8192点在44100采样率下，最低能分辨F#2，但是足矣
parent.dt = 1000 / tNum;
parent.TperP = parent.dt / parent._width; parent.PperT = parent._width / parent.dt;
const fs = channels.sampleRate ??= parent.audioContext.sampleRate;
const dN = Math.round(fs / tNum);
parent.Keyboard.freqTable.A4 = A4;
let STFT;
try {
if (!useGPU) throw new Error("强制使用CPU计算STFT");
STFT = await stftGPU(fs, channels, dN, fftPoints);
} catch (e) {
console.warn("GPU加速STFT失败,回退至CPU计算\n原因:", e.message);
STFT = await stftCPU(fs, channels, dN, fftPoints);
} return NoteAnalyser.normalize(STFT);
}
async function stftCPU(fs, channels, hop, fftPoints) {
const progressPerChannel = 1 / channels.length;
var progressTrans = (x) => x * progressPerChannel; // 如果分阶段执行则需要自定义进度的变换
const fft = new realFFT(fftPoints);
const analyser = new NoteAnalyser(fs / fftPoints, parent.Keyboard.freqTable);
const nbins = parent.Keyboard.freqTable.length;
const a = async (t) => { // 对t执行STFT，并整理为时频谱
let n = hop >> 1;
const result = new Array(1 + (t.length - n) / hop | 0);
const _data = new Float32Array(result.length * nbins);
const window_left = fftPoints >> 1; // 窗口左边界偏移量
for (let k = 0, sub = 0; n <= t.length; n += hop, sub += nbins) { // n为窗口中心
result[k++] = analyser.mel(...fft.fft(t, n - window_left), _data.subarray(sub, sub + nbins));
// 一帧一次也太慢了。这里固定更新帧率
let tnow = performance.now();
if (tnow - lastFrame > 200) {
lastFrame = tnow;
// 打断分析 更新UI 等待下一周期
parent.event.dispatchEvent(new CustomEvent("progress", {
detail: progressTrans(k / result.length)
}));
await new Promise(resolve => setTimeout(resolve, 0));
}
} // 通知UI关闭的事件分发移到了audio.onloadeddata中
result.raw = _data;
return result;
};
await new Promise(resolve => setTimeout(resolve, 0)); // 等待UI
var lastFrame = performance.now();
const result = await a(channels[0]);
for (let i = 1; i < channels.length; i++) {
progressTrans = (x) => (i + x) * progressPerChannel;
const other = (await a(channels[i])).raw;
const raw = result.raw;
for (let j = 0; j < raw.length; j++) raw[j] += other[j];
} return result;
};
async function stftGPU(fs, channels, hop, fftPoints) {
const stftGPU = new STFTGPU(fftPoints, hop);
parent.event.dispatchEvent(new CustomEvent("progress", {
detail: 0.4
}));
await stftGPU.initWebGPU();
console.log("WebGPU初始化成功,使用GPU计算STFT");
const analyser = new NoteAnalyser(fs / fftPoints, parent.Keyboard.freqTable);
for (const c of channels) stftGPU.stft(c);
const stftRes = await stftGPU.readGPU();
stftGPU.free();
const result = new Array(stftRes.length);
const nbins = parent.Keyboard.freqTable.length;
const _data = new Float32Array(result.length * nbins);
for (let i = 0; i < stftRes.length; i++)
result[i] = analyser.mel2(stftRes[i], _data.subarray(i * nbins, (i + 1) * nbins));
result.raw = _data;
return result;
}
/**
* 后台（worker）计算CQT
* @param {Array<Float32Array>} channels 通道数据数组，每个元素为一个channel的数据 .sampleRate为采样率
* @param {number} tNum 一秒几次分析 决定步距
* @param {boolean} useGPU 是否使用GPU加速计算CQT
* @returns 不返回，直接作用于Spectrogram.spectrogram
*/
this.cqt = (channels, tNum, useGPU = false) => {
if (!parent.io.canUseExternalWorker || window.cqt == undefined) return; // 开worker和fetch要求http
console.time("CQT计算");
cqt(channels, tNum, parent.Keyboard.freqTable[0], useGPU).then((cqtData) => {
// CQT结果准确但琐碎，STFT结果粗糙但平滑，所以混合一下
const s = parent.Spectrogram.spectrogram;
let tLen = Math.min(cqtData.length, s.length);
for (let i = 0; i < tLen; i++) {
const cqtBins = cqtData[i];
const stftBins = s[i];
for (let j = 0; j < cqtBins.length; j++) {
// 更重视CQT谱 若CQT谱更强则用更大的平均
if (stftBins[j] < cqtBins[j]) stftBins[j] = Math.hypot(stftBins[j], cqtBins[j]) / Math.SQRT2;
else stftBins[j] = Math.sqrt(stftBins[j] * cqtBins[j]);
}
}
console.timeEnd("CQT计算");
parent.Spectrogram.spectrogram = s; // 通知更新
}).catch(console.error);
};
/**
* 后台（worker）AI音色无关扒谱
* @param {AudioBuffer} audioBuffer 音频缓冲区
* @param {boolean} judgeOnly 是否只判断是否可以扒谱
* @returns promise，用于指示扒谱完成。如果judgeOnly为true则返回值代表是否可以AI扒谱
*/
this.basicamt = (audioData, judgeOnly = false) => {
if (!parent.io.canUseExternalWorker || window.AI == undefined) {
alert("file协议下无法使用AI扒谱！");
return false;
}
if (!parent.Spectrogram._spectrogram) {
alert('请导入音频或进入midi编辑模式！');
return false;
}
if (!parent.MidiAction.channelDiv.colorMask) {
alert("音轨不足！请至少删除一个音轨！");
return false;
}
if (judgeOnly) return true;
console.time("AI扒谱");
return AI.basicamt(audioData).then((events) => {
console.timeEnd("AI扒谱");
const timescale = (256 * 1000) / (22050 * parent.dt); // basicAMT在22050Hz下以hop=256分析
// 逻辑同index.html中导入midi
const chdiv = parent.MidiAction.channelDiv;
chdiv.switchUpdateMode(false);
const ch = chdiv.addChannel();
if (!ch) return;
const chid = ch.index;
ch.name = `AI扒谱${chid}`;
ch.instrument = TinySynth.instrument[(ch.ch.instrument = 4)];
const maxIntensity = events.reduce((a, b) => a.velocity > b.velocity ? a : b).velocity;
ch.ch.volume = maxIntensity * 127;
const notes = events.map(({ onset, offset, note, velocity }) => {
return {
x1: onset * timescale,
x2: offset * timescale,
y: note - 24,
ch: chid,
selected: false,
v: velocity / maxIntensity * 127
};
});
parent.MidiAction.midi.push(...notes);
parent.MidiAction.midi.sort((a, b) => a.x1 - b.x1);
chdiv.switchUpdateMode(true);
}).catch(alert);
};
/**
* 后台（worker）AI音色分离扒谱
* @param {AudioBuffer} audioBuffer 音频缓冲区
* @returns promise，用于指示扒谱完成
*/
this.septimbre = (audioData, k = 2) => {
console.time("AI音色分离扒谱");
return AI.septimbre(audioData, k).then((tracks) => {
console.timeEnd("AI音色分离扒谱");
const timescale = (256 * 1000) / (22050 * parent.dt);
// 逻辑同index.html中导入midi
const chdiv = parent.MidiAction.channelDiv;
chdiv.switchUpdateMode(false);
tracks.forEach((events) => {
const ch = chdiv.addChannel();
if (!ch) return;
const chid = ch.index;
ch.name = `AI分离${chid}`;
ch.instrument = TinySynth.instrument[(ch.ch.instrument = 4)];
const maxIntensity = events.reduce((a, b) => a.velocity > b.velocity ? a : b).velocity;
ch.ch.volume = maxIntensity * 127;
const notes = events.map(({ onset, offset, note, velocity }) => {
return {
x1: onset * timescale,
x2: offset * timescale,
y: note - 24,
ch: chid,
selected: false,
v: velocity / maxIntensity * 127
};
});
parent.MidiAction.midi.push(...notes);
});
parent.MidiAction.midi.sort((a, b) => a.x1 - b.x1);
chdiv.switchUpdateMode(true);
}).catch(alert);
};
/**
* “自动对齐音符”的入口 原理见 ~/dataProcess/aboutANA.md
*/
this.autoNoteAlign = () => {
if (!parent.Spectrogram._spectrogram || parent.midiMode) {
alert('请先导入音频！');
return false;
}
if (!parent.MidiAction.channelDiv.colorMask) {
alert("音轨不足！请至少删除一个音轨！");
return false;
}
let tempDiv = document.createElement('div');
tempDiv.innerHTML = `
<div class="request-cover">
<div class="card hvCenter">
<div class="fr" style="align-items: center;">
<label class="title">数字谱对齐音频</label>
<span style="flex:1"></span>
<button class="ui-cancel">❌</button>
</div>
<div class="layout layout-first">
<button class="ui-cancel">降低八度</button>
<span style="width: 1em;"></span>
<button class="ui-cancel">升高八度</button>
</div>
<div class="layout">
<textarea cols="35" rows="12" placeholder="\
输入没有时值的数字谱，算法将创建与音频同步的音符，相当于“数字谱+音频→midi”
数字谱的“1”对应于C5，请自行整体添加“[]”或“()”以升/降八度
建议先观察频谱，找到合适的八度。如果效果不好，也可以考虑升降后重试。
数字谱示例: ((b1)7)1 #2[#34b5]"></textarea>
</div>
<div class="layout">
<button class="ui-confirm">重复区间内</button>
<span style="width: 1em;"></span>
<button class="ui-confirm">所有时间</button>
</div>
</div>
</div>`;
const UI = tempDiv.firstElementChild;
const textarea = UI.querySelector('textarea');
const close = () => {
UI.remove();
parent.preventShortCut = false;
}
const btns = UI.getElementsByTagName('button');
btns[0].onclick = close;
btns[1].onclick = () => {
textarea.value = '(' + textarea.value + ')';
};
btns[2].onclick = () => {
textarea.value = '[' + textarea.value + ']';
};
btns[3].onclick = () => { // 重复区间内
const numberedScore = textarea.value.trim();
if (!numberedScore) {
alert("请输入数字谱！");
return;
}
try {
this._autoNoteAlign(
numberedScore,
parent.TimeBar.repeatStart / parent.dt,
parent.TimeBar.repeatEnd / parent.dt
); close();
} catch (error) {
alert(error.message);
}
};
btns[4].onclick = () => { // 所有时间
const numberedScore = textarea.value.trim();
if (!numberedScore) {
alert("请输入数字谱！");
return;
}
try {
this._autoNoteAlign(numberedScore);
close();
} catch (error) {
alert(error.message);
}
}
parent.preventShortCut = true; // 禁止快捷键
document.body.insertBefore(UI, document.body.firstChild);
};
this._autoNoteAlign = (noteSeq, begin, end) => {
noteSeq = parseJE(noteSeq);
let spectrum = parent.Spectrogram.spectrogram;
if (begin != undefined) {
begin = Math.max(0, Math.floor(begin));
end = Math.min(spectrum.length, Math.ceil(end));
spectrum = spectrum.slice(begin, end);
} else begin = 0;
if (noteSeq.length > spectrum.length) {
throw new Error("数字谱长度超过频谱长度！（时长太短）");
}
// 插入间隔（用-1表示）
const paddedNoteSeq = [-1];
for (let i = 0; i < noteSeq.length; i++) {
// 0对应C4
paddedNoteSeq.push(noteSeq[i] + 48, -1);
}
const path = autoNoteAlign(paddedNoteSeq, spectrum, 100 / parent.dt);
const chdiv = parent.MidiAction.channelDiv;
chdiv.switchUpdateMode(false);
const ch = chdiv.addChannel();
if (!ch) return;
const chid = ch.index;
ch.name = `自动对齐${chid}`;
for (let i = 0; i < path.length; ++i) {
const [noteIdx, frameIdx] = path[i];
const n = paddedNoteSeq[noteIdx];
if (n == -1) continue;
while (i < path.length && path[i][0] == noteIdx) ++i;
--i;
const frameEnd = path[i][1] + 1;
parent.MidiAction.midi.push({
y: n,
x1: frameIdx + begin,
x2: frameEnd + begin,
ch: chid,
selected: false,
});
}
parent.MidiAction.midi.sort((a, b) => a.x1 - b.x1);
chdiv.switchUpdateMode(true);
};
/**
* 自动节拍检测并生成节拍线
* @param {number} minBPM 最小BPM
* @param {boolean} autoDownBeat 是否自动检测重拍位置
* @returns {number} 全局估计的BPM值
*/
this.beatEst = (minBPM = 40, autoDownBeat = false) => {
const sr = Math.round(1000 / parent.dt);
const onset = Beat.extractOnset(parent.Spectrogram.spectrogram, Math.min(0.99, 16 / sr));
const maxInterval = Math.ceil(sr * 60 / minBPM);
// 范围要大，所以方差大一些
const global = Beat.corrBPM(SIGNAL.autoCorr(onset, maxInterval), sr, 1.4, 105);
const tempo = Beat.tempo(onset, sr, minBPM, 12.8, 1.6, global);
const fftSize = Beat.fs2FFTN(sr, 12.8);
const pulse = Beat.PLP(onset, sr, [40, 200], fftSize, Math.max(1, fftSize >> 3), Beat.PLPprior(tempo, 0.1));
for (let i = 0; i < pulse.length; i++) pulse[i] += onset[i];
const beatIdx = Beat.EllisBeatTrack(pulse, sr, 300, tempo);
if (beatIdx.length < 2) {
alert("未能检测到有效节拍！");
return;
}
if (beatIdx[0] == 0) beatIdx.shift();
// 不能引入pulse的干扰 得用原始的onset
const beatStrength = Beat.beatStrength(onset, beatIdx);
const beatbar = parent.BeatBar.beats;
if (autoDownBeat) { // 用动态规划求解重拍位置 但不够稳定
const [downbeatIndices, downbeatMeters] = Beat.detectDownbeats(beatStrength, [2, 3, 4]);
// 处理前面的拍
beatbar.length = 0;
let prev = 0, id = 0, i = 0;
for (; i <= downbeatIndices[0]; i++, id++) {
const at = beatIdx[i] * parent.dt;
beatbar.push(new eMeasure(id, prev, 1, 4, at - prev));
prev = at;
}
for (i = 0; i < downbeatIndices.length; i++, id++) {
const pattern = downbeatMeters[i];
const beatdown = downbeatIndices[i];
if (i + 1 < downbeatIndices.length) {
const nextBeatdown = downbeatIndices[i + 1];
const endtime = beatIdx[nextBeatdown] * parent.dt;
beatbar.push(new eMeasure(id, prev, pattern, 4, endtime - prev));
prev = endtime;
} else {
let endtime;
const time = beatIdx[beatIdx.length - 1] * parent.dt - prev;
const cnt = beatIdx.length - 1 - beatdown;
beatbar.push(new eMeasure(id, prev, pattern, 4, time / cnt * pattern));
}
}
} else { // 这两个估计结果有点差 暂时用1拍
// const [g_pattern, g_beatdown] = Beat.rhythmicPattern(beatStrength, [2, 3, 4]);
let g_pattern = 1, g_beatdown = 0;
beatbar.length = 0;
let prev = 0, id = 0, i = 0;
// 前面的用单小节处理 注意应该有等号
for (; i <= g_beatdown; i++, id++) {
const at = beatIdx[i] * parent.dt;
beatbar.push(new eMeasure(id, prev, 1, 4, at - prev));
prev = at;
}
for (i = i - 1 + g_pattern; i < beatIdx.length; i += g_pattern, id++) {
const at = beatIdx[i] * parent.dt;
beatbar.push(new eMeasure(id, prev, g_pattern, 4, at - prev));
prev = at;
}
}
beatbar.check(true);
parent.snapshot.save(0b100);
parent.layers.action.dirty = true;
// 如果正在用节拍则刷新节拍信息
if (parent.AudioPlayer.audio.paused === false && parent.MidiPlayer._ifBeat) {
parent.MidiPlayer.restart(true);
}
return global;
};
// 1(C4)->0
function parseJE(txt) {
const parts = [];
let n = 0;
let octave = 0;
const JEnotes = ["1", "#1", "2", "#2", "3", "4", "#4", "5", "#5", "6", "#6", "7"];
while (n < txt.length) {
if (txt[n] == ')' || txt[n] == '[') ++octave;
else if (txt[n] == '(' || txt[n] == ']') --octave;
else {
let m = 0;
if (txt[n] == '#') m = 1;
else if (txt[n] == 'b') m = -1;
const noteEnd = n + Math.abs(m);
const position = noteEnd < txt.length ? JEnotes.indexOf(txt[noteEnd]) : -1;
if (position != -1) {
parts.push(m + position + octave * 12);
n = noteEnd;
}
}
++n;
} return parts;
};
this.reduceHarmonic = () => {
let resolve, reject;
let p = new Promise((res, rej) => {
resolve = res;
reject = rej;
});
let tempDiv = document.createElement('div');
tempDiv.innerHTML = `<div class="request-cover">
<div class="card hvCenter">
<label class="title">谐波去除 <span style="font-size: 0.6em; color: grey;">非负最小二乘法</span></label>
<div class="layout">
<span class="labeled" data-tooltip="对谐波强度的估计">谐波衰减率</span>
<input type="number" value="0.6" step="0.01" min="0.1" max="0.9">
</div>
<div class="layout">
<span class="labeled" data-tooltip="考虑的谐波数量">谐波数量</span>
<input type="number" value="10" step="1" min="4" max="16">
</div>
<div class="layout">
<label class="labeled" data-tooltip="修改频谱 不可逆">
原位操作<input type="checkbox">
</label>
</div>
<div class="layout">
<button class="ui-cancel">取消</button>
<span style="width: 1em;"></span>
<button class="ui-confirm">确认</button>
</div>
</div></div>`;
const UI = tempDiv.firstElementChild;
const inputs = UI.querySelectorAll('input[type="number"]');
const decayInput = inputs[0];
const harmonicsInput = inputs[1];
const cancelBtn = UI.querySelector('.ui-cancel');
const inplace = UI.querySelector('input[type="checkbox"]');
const confirmBtn = UI.querySelector('.ui-confirm');
cancelBtn.onclick = () => {
UI.remove();
resolve(false);
};
confirmBtn.onclick = () => {
let decay = parseFloat(decayInput.value);
if (decay < 0.1 || decay > 0.9) {
alert("衰减率必须在0.1到0.9之间！");
return;
}
let harmonics = parseInt(harmonicsInput.value);
if (harmonics < 4 || harmonics > 16) {
alert("谐波数量必须在4到16之间！");
return;
}
UI.remove();
this._reduceHarmonic(decay, harmonics, inplace.checked).then(() => {
resolve(true);
}).catch(reject);
}
document.body.insertBefore(UI, document.body.firstChild);
return p;
};
/**
* 利用非负最小二乘去除频谱中的谐波成分并补偿基频 在幅度谱上进行
* 如果inplace为true则直接修改Spectrogram.spectrogram 否则会存储谐波成分矩阵于Spectrogram.harmonic
* 原理是将谐波模板（每个音符的基频和若干个谐波）作为特征，拟合出每一帧中各个音符的强度，然后将这些音符的谐波成分从频谱中减去
* @param {number} decay 谐波衰减率，默认0.6，越大去除越彻底但可能过度拟合
* @param {number} harmonics 谐波数量
* @param {boolean} inplace 是否直接在原频谱上减去谐波 还是单独存储谐波成分
*/
this._reduceHarmonic = async (decay = 0.6, harmonics = 10, inplace = false) => {
const container = document.createElement('div');
container.innerHTML = `<div class="request-cover">
<div class="card hvCenter"><label class="title">分析中</label>
<span>00%</span>
<div class="layout">
<div class="porgress-track">
<div class="porgress-value"></div>
</div>
</div>
</div></div>`;
const progressUI = container.firstElementChild;
const progress = progressUI.querySelector('.porgress-value');
const percent = progressUI.querySelector('span');
document.body.insertBefore(progressUI, document.body.firstChild);
const onprogress = (detail) => {
if (detail < 0) {
progress.style.width = '100%';
percent.textContent = '100%';
progressUI.style.opacity = 0;
setTimeout(() => progressUI.remove(), 200);
} else if (detail >= 1) {
detail = 1;
progress.style.width = '100%';
percent.textContent = "加载界面……";
} else {
progress.style.width = (detail * 100) + '%';
percent.textContent = (detail * 100).toFixed(2) + '%';
}
};
var lastFrame = performance.now();
const s = parent.Spectrogram._spectrogram;
const M = s[0].length, N = s.length;
const M1 = M + 1;
// 创建音符谐波模板
let harmonicAmp = Array.from({ length: harmonics }, (_, i) => decay ** i);
const Harmonic = new Float32Array(M);
for (let i = 0; i < harmonicAmp.length; i++) {
const idx = 12 * Math.log2(i + 1);
let l = Math.floor(idx), r = Math.ceil(idx);
if (r < M) {
if (l == r) Harmonic[l] = harmonicAmp[i];
else {
Harmonic[l] += harmonicAmp[i] * (r - idx);
Harmonic[r] += harmonicAmp[i] * (idx - l);
}
}
}
// 填充到模板矩阵A
const A = new Float32Array(M * M);
for (let i = 0; i < M; i++)
A.set(Harmonic.subarray(0, M - i), i * M1);
// 模式选择
if (!inplace) {
harmonicAmp = Array(N);
harmonicAmp.raw = new Float32Array(N * M);
} else harmonicAmp = null;
// 对每一帧执行NNLS
const nnls = new NNLSSolver(M, M, 2e-4, Harmonic);
for (let t = 0; t < N; t++) {
const f = s[t];
const c = nnls.solve(A, f);
// 计算谐波
Harmonic.fill(0);
for (let i = 0; i < M; i++) {
const a = i + 12; // start at 2f0
const off = i * M + a;
let f0h = 0;
for (let j = 0; j < M - a; j++) {
let amp = A[off + j] * c[i];
f0h += amp * amp;
Harmonic[a + j] += amp;
};
// 加强基频。∵L2<L1 ∴此处用L2对基频小幅补偿
Harmonic[i] -= Math.sqrt(f0h);
}
if (inplace) { // 从原始频谱中减去谐波
for (let i = 0; i < M; i++) f[i] = Math.max(0, f[i] - Harmonic[i]);
} else { // 存储谐波成分
let a = harmonicAmp[t] = harmonicAmp.raw.subarray(t * M, (t + 1) * M);
a.set(Harmonic);
}
// UI更新
let tnow = performance.now();
if (tnow - lastFrame > 200) {
onprogress(t / N);
await new Promise(resolve => setTimeout(resolve, 0)); // 等待UI
lastFrame = tnow;
}
}
if (!inplace) parent.Spectrogram.harmonic = harmonicAmp;
parent.layers.spectrum.dirty = true;
onprogress(-1);
};
}
================================================
FILE: core/app_audioplayer.js
================================================
/// <reference path="../lib/fakeAudio.js" />
/**
* 音频播放
* @param {App} parent
*/
function _AudioPlayer(parent) {
this.name = "请上传文件"; // 在parent.io.onfile中赋值
this.audio = new Audio(); // 在parent.io.onfile中重新赋值 此处需要一个占位
this.play_btn = document.getElementById("play-btn");
this.durationString = ''; // 在parent.Analyser.audio.ondurationchange中更新
this.autoPage = false; // 自动翻页
this.repeat = true; // 是否区间循环
this.EQfreq = [31, 62, 125, 250, 500, 1000, 2000, 4000, 8000, 16000];
// midiMode下url为duration
this.createAudio = (url) => {
return new Promise((resolve, reject) => {
const a = parent.midiMode ? new FakeAudio(url) : new Audio(url);
a.loop = false;
a.volume = parseFloat(document.getElementById('audiovolumeControl').value);
a.ondurationchange = () => {
let ms = a.duration * 1000;
this.durationString = parent.TimeBar.msToClockString(ms);
parent.BeatBar.beats.maxTime = ms;
};
a.onended = () => {
parent.time = 0;
this.stop();
};
a.onloadeddata = () => {
if (!parent.midiMode) {
this.setEQ();
if (parent.audioContext.state == 'suspended') parent.audioContext.resume().then(() => a.pause());
document.title = this.name + "~扒谱";
} else {
document.title = this.name;
}
a.playbackRate = document.getElementById('speedControl').value; // load之后会重置速度
parent.time = 0;
resolve(a);
a.onloadeddata = null; // 一次性 防止多次构造
this.play_btn.firstChild.textContent = parent.TimeBar.msToClockString(parent.time);
this.play_btn.lastChild.textContent = this.durationString;
};
a.onerror = (e) => { // 如果正常分析，是用不到这个回调的，因为WebAudioAPI读取就会报错。但上传已有结果不会再分析
// 发现一些如mov格式的视频，不在video/的支持列表中，用.readAsDataURL转为base64后无法播放，会触发这个错误
// 改正方法是用URL.createObjectURL(file)生成一个blob地址而不是解析为base64
console.error("Audio load error", e);
reject(e);
// 不再抛出错误事件 调用者自行处理
// parent.event.dispatchEvent(new Event('fileerror'));
};
this.setAudio(a);
});
};
let _crossFlag = false; // 上一时刻是否在重复区间终点左侧
this.update = () => {
const a = this.audio;
if (a.readyState != 4 || a.paused) return;
parent.time = a.currentTime * 1000; // 【重要】更新时间
// 重复区间
let crossFlag = parent.time < parent.TimeBar.repeatEnd;
if (this.repeat && parent.TimeBar.repeatEnd >= parent.TimeBar.repeatStart) { // 重复且重复区间有效
if (_crossFlag && !crossFlag) { // 从重复区间终点左侧到右侧
parent.time = parent.TimeBar.repeatStart;
a.currentTime = parent.time / 1000;
}
}
_crossFlag = crossFlag;
this.play_btn.firstChild.textContent = parent.TimeBar.msToClockString(parent.time);
this.play_btn.lastChild.textContent = this.durationString;
// 自动翻页
if (parent.time > parent.idXend * parent.dt || parent.time < parent.idXstart * parent.dt) {
// 在视图外
if (this.autoPage)
parent.scroll2(((parent.time / parent.dt - 1) | 0) * parent._width, parent.scrollY);
} else parent.layers.action.dirty = true;
};
/**
* 在指定的毫秒数开始播放
* @param {number} at 开始的毫秒数 如果是负数，则从当下开始
*/
this.start = (at) => {
const a = this.audio;
if (a.readyState != 4) return;
if (at >= 0) a.currentTime = at / 1000;
_crossFlag = false; // 置此为假可以暂时取消重复区间
parent.MidiPlayer.restart();
if (a.readyState == 4) a.play();
else a.oncanplay = () => {
a.play();
a.oncanplay = null;
};
};
this.stop = () => {
this.audio.pause();
parent.synthesizer.stopAll();
};
this.setEQ = (f = this.EQfreq) => {
const a = this.audio;
if (a.EQ) return;
// 由于createMediaElementSource对一个audio只能调用一次，所以audio的EQ属性只能设置一次
const source = parent.audioContext.createMediaElementSource(a);
let last = source;
a.EQ = {
source: source,
filter: f.map((v) => {
const filter = parent.audioContext.createBiquadFilter();
filter.type = "peaking";
filter.frequency.value = v;
filter.Q.value = 1;
filter.gain.value = 0;
last.connect(filter);
last = filter;
return filter;
})
};
last.connect(parent.audioContext.destination);
};
this.setAudio = (newAudio) => {
const a = this.audio;
if (a) {
a.pause();
a.onerror = null; // 防止触发fileerror
a.src = '';
if (a.EQ) {
a.EQ.source.disconnect();
for (const filter of a.EQ.filter) filter.disconnect();
}
// 配合传参为URL.createObjectURL(file)使用，防止内存泄露
URL.revokeObjectURL(a.src);
}
this.audio = newAudio;
};
this.play_btn.onclick = () => {
if (this.audio.paused) this.start(-1);
else this.stop();
this.play_btn.blur(); // 防止焦点在按钮上导致空格响应失败
};
}
================================================
FILE: core/app_beatbar.js
================================================
/// <reference path="../lib/beatBar.js" />
/// <reference path="../ui/contextMenu.js" />
/**
* 顶部小节轴
* @param {App} parent
*/
function _BeatBar(parent) {
this.beats = new Beats();
this.minInterval = 20; // 最小画线间隔 单位:像素
// timeBar的下半部分画小节线和拍线 并在工作区画小节线
this.render = () => {
const canvas = parent.timeBar;
const ctx = parent.timeBar.ctx;
ctx.fillStyle = '#2e3039';
const h = canvas.height >> 1;
ctx.fillRect(0, h, canvas.width, canvas.width);
ctx.fillStyle = '#8e95a6';
const spectrum = parent.layers.action.ctx;
const spectrumHeight = parent.layers.action.height;
ctx.strokeStyle = '#f0f0f0';
spectrum.strokeStyle = '#c0c0c0';
const beatX = []; // 小节内每一拍
const noteX = []; // 一拍内x分音符对齐线
const iterator = this.beats.iterator(parent.scrollX * parent.TperP, true);
ctx.beginPath(); spectrum.beginPath();
while (1) {
let measure = iterator.next();
if (measure.done) break;
measure = measure.value;
let x = measure.start * parent.PperT - parent.scrollX;
if (x > canvas.width) break;
ctx.moveTo(x, h);
ctx.lineTo(x, canvas.height);
spectrum.moveTo(x, 0);
spectrum.lineTo(x, spectrumHeight);
// 写字 会根据间隔决定是否显示拍型
const Interval = measure.interval * parent.PperT;
ctx.fillText(Interval < 38 ? measure.id : `${measure.id}. ${measure.beatNum}/${measure.beatUnit}`, x + 2, h + 14);
// 画更细的节拍线
const dp = Interval / measure.beatNum;
if (dp < this.minInterval) continue;
x += dp;
for (let i = measure.beatNum - 1; i > 0; i--, x += dp) beatX.push(x);
// 画x分音符的线
const noteNum = 1 << Math.log2(dp / this.minInterval);
if (noteNum < 2) continue;
const noteInterval = dp / noteNum;
let i = ((x - canvas.width) / noteInterval) | 0; // 跳过末尾的 不然在极大的放大时会卡顿
if (i > 0) x -= i * noteInterval;
else i = 0;
for (let n = noteNum * measure.beatNum; i < n; i++, x -= noteInterval) {
if (x < 0) break;
if (i % noteNum == 0) continue; // 跳过beat线
noteX.push(x);
}
} ctx.stroke(); spectrum.stroke();
if (beatX.length != 0) {
spectrum.beginPath();
spectrum.strokeStyle = '#909090';
for (const x of beatX) {
spectrum.moveTo(x, 0);
spectrum.lineTo(x, spectrumHeight);
} spectrum.stroke();
}
if (noteX.length != 0) {
spectrum.beginPath();
spectrum.setLineDash([4, 4]);
spectrum.strokeStyle = '#606060';
for (const x of noteX) {
spectrum.moveTo(x, 0);
spectrum.lineTo(x, spectrumHeight);
} spectrum.stroke();
spectrum.setLineDash([]); // 恢复默认
}
};
this.contextMenu = new ContextMenu([
{
name: "设置小节",
callback: (e_father, e_self) => {
const bs = this.beats;
const m = bs.setMeasure((e_father.offsetX + parent.scrollX) * parent.TperP, undefined, true);
const tempDiv = document.createElement('div');
tempDiv.innerHTML = `
<div class="request-cover">
<div class="card hvCenter"><label class="title">小节${m.id}设置</label>
<div class="layout layout-first"><span>拍数</span><input type="text" name="ui-ask" step="1" max="16" min="1"></div>
<div class="layout"><span>音符</span><select name="ui-ask">
<option value="2">2分</option>
<option value="4">4分</option>
<option value="8">8分</option>
<option value="16">16分</option>
</select></div>
<div class="layout"><span>BPM:</span><input type="number" name="ui-ask" min="1"></div>
<div class="layout"><span>(忽略BPM)保持小节长度</span><input type="checkbox" name="ui-ask"></div>
<div class="layout"><span>(忽略以上)和上一小节一样</span><input type="checkbox" name="ui-ask"></div>
<div class="layout"><span>应用到后面相邻同类型小节</span><input type="checkbox" name="ui-ask" checked></div>
<div class="layout"><button class="ui-cancel">取消</button><button class="ui-confirm">确定</button></div>
</div>
</div>`;
const Pannel = tempDiv.firstElementChild;
document.body.insertBefore(Pannel, document.body.firstChild);
Pannel.tabIndex = 0;
Pannel.focus();
function close() { Pannel.remove(); }
const inputs = Pannel.querySelectorAll('[name="ui-ask"]');
const btns = Pannel.getElementsByTagName('button');
inputs[0].value = m.beatNum; // 拍数
inputs[1].value = m.beatUnit; // 音符类型
inputs[2].value = m.bpm; // bpm
btns[0].onclick = close;
btns[1].onclick = () => {
if (!inputs[5].checked) { // 后面不变
bs.setMeasure(m.id + 1, undefined, false); // 让下一个生成实体
}
if (inputs[4].checked) { // 和上一小节一样
let last = bs.getMeasure(m.id - 1, false);
m.copy(last);
} else {
m.beatNum = parseInt(inputs[0].value);
m.beatUnit = parseInt(inputs[1].value);
if (!inputs[3].checked) m.bpm = parseFloat(inputs[2].value);
} bs.check(); close();
parent.snapshot.save(0b100);
};
}
}, {
name: "后方插入一小节",
callback: (e_father) => {
this.beats.add((e_father.offsetX + parent.scrollX) * parent.TperP, true);
parent.snapshot.save(0b100);
}
}, {
name: "均分该小节",
callback: (e_father) => {
const beatarr = this.beats;
const base = beatarr.setMeasure((e_father.offsetX + parent.scrollX) * parent.TperP, undefined, true, true);
const baseM = beatarr[base];
// 下一小节若未定义则定义 则插入一个 防止影响后面
if (base + 1 >= beatarr.length || beatarr[base + 1].id > baseM.id + 1)
beatarr.splice(base + 1, 0, eMeasure.baseOn(baseM, baseM.id + 1));
// 插入新的 用id位移实现
baseM.interval /= 2;
for (let i = base + 1; i < beatarr.length; i++) beatarr[i].id++;
beatarr.check(true);
parent.snapshot.save(0b100);
}
}, {
name: "分裂为单拍",
callback: (e_father) => {
const beatarr = this.beats;
const base = beatarr.setMeasure((e_father.offsetX + parent.scrollX) * parent.TperP, undefined, true, true);
const baseM = beatarr[base];
if (base + 1 >= beatarr.length || beatarr[base + 1].id > baseM.id + 1)
beatarr.splice(base + 1, 0, eMeasure.baseOn(baseM, baseM.id + 1));
const beatNum = baseM.beatNum;
baseM.interval /= beatNum;
for (let i = base + 1; i < beatarr.length; i++) beatarr[i].id += beatNum - 1;
beatarr.check(true);
parent.snapshot.save(0b100);
}
}, {
name: "合并下一小节",
callback: (e_father) => {
const beatarr = this.beats;
const base = beatarr.setMeasure((e_father.offsetX + parent.scrollX) * parent.TperP, undefined, true, true);
const baseM = beatarr[base];
// 下下个
const nextnext = beatarr.setMeasure(baseM.id + 2, undefined, false, true);
const nextnextM = beatarr[nextnext];
if (nextnext === base + 2) {
// 中间隔了一个小节头
const deleted = beatarr.splice(base + 1, 1)[0];
baseM.interval += deleted.interval;
baseM.beatNum += deleted.beatNum;
} else {
baseM.interval += baseM.interval;
baseM.beatNum += baseM.beatNum;
}
for (let i = base + 1; i < beatarr.length; i++) beatarr[i].id--;
beatarr.check(true);
parent.snapshot.save(0b100);
}
}, {
name: "重置后面所有小节",
callback: (e_father) => {
const base = this.beats.getBaseIndex((e_father.offsetX + parent.scrollX) * parent.TperP, true);
this.beats.splice(base + 1);
parent.snapshot.save(0b100);
}
}, {
name: '<span style="color: red;">删除该小节</span>',
callback: (e_father, e_self) => {
this.beats.delete((e_father.offsetX + parent.scrollX) * parent.TperP, true);
parent.snapshot.save(0b100);
}
}
]);
this.belongID = -1; // 小节线前一个小节的id
this.moveCatch = (e) => { // 画布上光标移动到小节线上可以进入调整模式
// 判断是否在小节轴上
if (e.offsetY < parent.timeBar.height >> 1) {
parent.timeBar.classList.remove('selecting');
this.belongID = -1;
return;
}
const timeNow = (e.offsetX + parent.scrollX) * parent.TperP;
const m = this.beats.getMeasure(timeNow, true);
if (m == null) {
this.belongID = -1;
parent.timeBar.classList.remove('selecting');
return;
}
const threshold = 6 * parent.TperP;
if (timeNow - m.start < threshold) {
this.belongID = m.id - 1;
parent.timeBar.classList.add('selecting');
} else if (m.start + m.interval - timeNow < threshold) {
this.belongID = m.id;
parent.timeBar.classList.add('selecting');
} else {
this.belongID = -1;
parent.timeBar.classList.remove('selecting');
}
};
}
================================================
FILE: core/app_hscrollbar.js
================================================
/**
* 配合scroll的滑动条
* @param {App} parent
*/
function _HscrollBar(parent) {
this.maxScrollX = 0;
this.refreshPosition = () => { // 在parent.scroll2中调用
if (this.maxScrollX <= 0) {
thumb.style.left = '0px';
return;
}
let pos = (track.offsetWidth - thumb.offsetWidth) * parent.scrollX / this.maxScrollX;
thumb.style.left = pos + 'px';
};
this.refreshSize = () => { // 需要在parent.xnum parent.width改变之后调用 在二者的setter中调用
track.style.display = 'block';
let all = parent._width * parent._xnum;
let p = Math.min(1, parent.layers.width / all); // 由于有min存在所以xnum即使为零也能工作
let nw = p * track.offsetWidth;
thumb.style.width = Math.max(nw, 10) + 'px'; // 限制最小宽度
this.maxScrollX = all - parent.layers.width;
};
const track = document.getElementById('scrollbar-track');
const thumb = document.getElementById('scrollbar-thumb');
const thumbMousedown = (event) => { // 滑块跟随鼠标
event.stopPropagation(); // 防止触发track的mousedown
const startX = event.clientX;
const thumbLeft = thumb.offsetLeft;
const moveThumb = (event) => {
let currentX = event.clientX - startX + thumbLeft;
let maxThumbLeft = track.offsetWidth - thumb.offsetWidth;
parent.scroll2(currentX / maxThumbLeft * this.maxScrollX, parent.scrollY);
}
const stopMoveThumb = () => {
document.removeEventListener("mousemove", moveThumb);
document.removeEventListener("mouseup", stopMoveThumb);
}
document.addEventListener("mousemove", moveThumb);
document.addEventListener("mouseup", stopMoveThumb);
};
const trackMousedown = (e) => { // 滑块跳转
e.stopPropagation();
let maxThumbLeft = track.offsetWidth - thumb.offsetWidth;
let p = (e.offsetX - (thumb.offsetWidth >> 1)) / maxThumbLeft; // nnd 减法优先级比位运算高
parent.scroll2(p * this.maxScrollX, parent.scrollY);
};
thumb.addEventListener('mousedown', thumbMousedown);
track.addEventListener('mousedown', trackMousedown);
}
================================================
FILE: core/app_io.js
================================================
/// <reference path="../lib/saver.js" />
/// <reference path="../lib/midi.js" />
/**
* 文件相关操作
* @param {App} parent
*/
function _IO(parent) {
this.canUseExternalWorker = typeof window.Worker !== 'undefined' && window.location.protocol !== 'file:';
// midi模式下的假音频
function fakeInput(l = 0, tNum = 1000 / parent.dt) {
if (!l || l <= 0) l = Math.ceil((parent.layers.width << 1) / parent.width);
// 一个怎么取值都返回0的东西，充当频谱
parent.Spectrogram.spectrogram = new Proxy({
raw: new Uint8Array(parent.ynum).fill(0),
_length: l,
get length() { return this._length; },
set length(l) { // 只要改变频谱的长度就可以改变时长 长度改变在MidiAction.updateView中
if (l < 0) return;
this._length = parent.xnum = l;
parent.AudioPlayer.audio.duration = l / tNum;
parent.AudioPlayer.play_btn.lastChild.textContent = parent.AudioPlayer.durationString;
},
*[Symbol.iterator]() {
for (let i = 0; i < this.length; i++) yield this.raw;
}
}, {
get(obj, prop) { // 方括号传递的总是string
if (isNaN(Number(prop))) return obj[prop];
return obj.raw;
},
set(obj, prop, value) {
if (isNaN(Number(prop))) obj[prop] = value;
return true;
}
});
// 假音频 需要设置parent.midiMode=true;
return parent.AudioPlayer.createAudio(l / tNum);
}
/**
* 会发出如下event:
* - fileui: 展示本函数的UI，需要外界关闭drag功能
* - fileuiclose: UI关闭，外界恢复drag功能
* - fileerror: 文件解析错误，外界提示用户; detail为Error对象
* - progress: 解析进度，detail为0~1的数字，-1表示完成
* event的意义是可以反复触发（比如进度文件错误可重试），而返回值的promise只能触发一次
* @param {File} file 音频文件 如果不传则进入midi编辑器模式
* @returns Promise 指示用户操作完成，即UI关闭
*/
this.onfile = (file) => { // 依赖askUI.css
let midimode = file == void 0; // 在确认后才能parent.midiMode=midimode
if (midimode) { // 不传则说明是midi编辑器模式
file = { name: "MIDI编辑模式" };
} else if (!(file.type.startsWith('audio/') || file.type.startsWith('video/'))) {
parent.event.dispatchEvent(new CustomEvent('fileerror', { detail: new Error("不支持的文件类型") }));
return Promise.reject();
} else if (file.type == "audio/mid") {
if (parent.Spectrogram._spectrogram) {
this.midiFile.import(file);
return Promise.resolve();
}
// 没有设置时间精度，先弹设置UI
return this.onfile().then(() => {
this.midiFile.import(file);
});
}
if (parent.Spectrogram._spectrogram && !confirm("本页面已加载音频，是否替换？")) {
return Promise.reject();
}
// 指示是否完成
let resolve, reject;
const donePromise = new Promise((res, rej) => { resolve = res; reject = rej; });
const loadAudio = (URLmode = true) => new Promise((res, rej) => {
const fileReader = new FileReader();
// 音频文件错误标志本次会话结束
// 调用loadAudio不需要再写catch
fileReader.onerror = (e) => {
parent.event.dispatchEvent(new CustomEvent('fileerror', { detail: e }));
console.error("FileReader error", e);
reject(e);
rej(e);
};
fileReader.onload = (e) => {
res(e.target.result);
};
if (URLmode) fileReader.readAsDataURL(file);
else fileReader.readAsArrayBuffer(file);
});
parent.event.dispatchEvent(new Event('fileui')); // 关闭drag功能
let tempDiv = document.createElement('div');
// 为了不影响下面的事件绑定，midi模式下用display隐藏
const midiModeStyle = midimode ? ' style="display:none;"' : '';
tempDiv.innerHTML = `
<div class="request-cover">
<div class="card hvCenter">
<span class="title" style="text-align: center;">${file.name}</span>
<button class="ui-cancel"${midiModeStyle}>使用已有结果</button>
<div class="layout layout-first"><span>每秒的次数：</span><input type="number" name="ui-ask" value="20" min="1" max="100"></div>
<div class="layout"><span>标准频率A4=</span><input type="number" name="ui-ask" value="440" step="0.1" min="55"></div>
<div${midiModeStyle}>
<div class="layout">分析声道：
<label class="labeled" data-tooltip="快,只是进度条没动画">GPU加速<input type="checkbox" id="stft-gpu" checked></label>
</div>
<div class="layout">
<input type="radio" name="ui-ask" value="4" checked>Stereo
<input type="radio" name="ui-ask" value="2">L+R
<input type="radio" name="ui-ask" value="3">L-R
<input type="radio" name="ui-ask" value="0">L
<input type="radio" name="ui-ask" value="1">R
</div>
<div class="layout"${this.canUseExternalWorker ? '' : ' style="display:none;"'}>
<label class="labeled" data-tooltip="CQT分析更精准,将在后台进行">
后台计算CQT<input type="checkbox" id="calc-cqt" checked>
</label>
<label class="labeled" data-tooltip="GPU更快,但中途页面易卡顿">
优先用GPU算CQT<input type="checkbox" id="prefer-gpu">
</label>
</div>
</div>
<div class="layout">
<button class="ui-cancel">取消</button>
<span style="width: 1em;"></span>
<button class="ui-confirm">${midimode ? '确认' : '解析'}</button>
</div>
</div>
</div>`;
parent.AudioPlayer.name = file.name;
const ui = tempDiv.firstElementChild;
const close = () => ui.remove();
const checkboxSTFTGPU = ui.querySelector('#stft-gpu');
const checkboxCQT = ui.querySelector('#calc-cqt');
const checkboxGPU = ui.querySelector('#prefer-gpu');
checkboxCQT.onchange = () => {
checkboxGPU.parentElement.style.display = checkboxCQT.checked ? 'block' : 'none';
};
const btns = ui.getElementsByTagName('button');
btns[0].onclick = () => { // 进度上传
const input = document.createElement("input");
input.type = "file";
input.onchange = () => {
parent.io.projFile.parse(input.files[0]).then((data) => {
if (parent.AudioPlayer.name != data[0].name &&
!confirm(`音频文件与进度(${data[0].name})不同，是否继续？`))
return;
// 如果保存的是midi模式，则data[1]是都为undefined的数组
if (Array.isArray(data[1]) && data[1][0] === void 0) {
parent.io.projFile.import(data);
fakeInput().then(resolve).catch(reject);
return;
}
// 再读取音频看看是否成功
loadAudio(true).then((audioBuffer) => {
// 设置音频源 缓存到浏览器
parent.AudioPlayer.createAudio(audioBuffer).then(() => {
parent.io.projFile.import(data);
// 触发html中的iniEQUI
parent.event.dispatchEvent(new CustomEvent('progress', { detail: -1 }));
resolve();
}).catch((e) => {
parent.event.dispatchEvent(new CustomEvent('fileerror', { detail: e }));
console.error("AudioPlayer error", e);
reject(e);
}).finally(() => {
close(); // 不管音频结果如何都要关闭UI
parent.event.dispatchEvent(new Event('fileuiclose')); // 恢复drag功能
});
});
}).catch((e) => {
// 进度文件错误，允许重试，不reject
parent.event.dispatchEvent(new CustomEvent('fileerror', { detail: e }));
});
}; input.click();
};
btns[1].onclick = () => { // 取消按钮
close();
parent.event.dispatchEvent(new Event('fileuiclose')); // 恢复drag功能
resolve(false);
};
btns[2].onclick = () => { // 确认按钮
// 获取分析参数
const params = ui.querySelectorAll('[name="ui-ask"]'); // getElementsByName只能在document中用
let tNum = params[0].value;
let A4 = params[1].value;
let channel = 4;
for (let i = 2; i < 7; i++) {
if (params[i].checked) {
channel = parseInt(params[i].value);
break;
}
}
close();
parent.midiMode = midimode;
//==== midi模式 ====//
if (midimode) {
// 在Anaylse中的设置全局的
parent.dt = 1000 / tNum;
parent.TperP = parent.dt / parent._width; parent.PperT = parent._width / parent.dt;
if (parent.Keyboard.freqTable.A4 != A4) parent.Keyboard.freqTable.A4 = A4;
fakeInput(0, tNum).then(resolve).catch(reject);
parent.event.dispatchEvent(new Event('fileuiclose')); // 恢复drag功能
return;
}
//==== 音频文件分析 ====//
// 打开另一个ui analyse加入回调以显示进度
let tempDiv = document.createElement('div');
tempDiv.innerHTML = `
<div class="request-cover">
<div class="card hvCenter"><label class="title">解析中</label>
<span>00%</span>
<div class="layout layout-first">
<div class="porgress-track">
<div class="porgress-value"></div>
</div>
</div>
</div>
</div>`;
const progressUI = tempDiv.firstElementChild;
const progress = progressUI.querySelector('.porgress-value');
const percent = progressUI.querySelector('span');
document.body.insertBefore(progressUI, document.body.firstChild);
const onprogress = ({ detail }) => {
if (detail < 0) {
parent.event.removeEventListener('progress', onprogress);
progress.style.width = '100%';
percent.textContent = '100%';
progressUI.style.opacity = 0;
setTimeout(() => progressUI.remove(), 200);
} else if (detail >= 1) {
detail = 1;
progress.style.width = '100%';
percent.textContent = "加载界面……";
} else {
progress.style.width = (detail * 100) + '%';
percent.textContent = (detail * 100).toFixed(2) + '%';
}
};
parent.event.addEventListener('progress', onprogress);
// 读取文件
loadAudio(false).then((audioBuffer) => {
let channels;
// 解码音频文件为音频缓冲区
parent.audioContext.decodeAudioData(audioBuffer).then((decodedData) => {
channels = parent.Analyser.selectChannel(decodedData, channel);
return Promise.all([
parent.Analyser.stft(channels, tNum, A4, 8192, checkboxSTFTGPU.checked),
parent.AudioPlayer.createAudio(URL.createObjectURL(file)) // fileReader.readAsDataURL(file) 将mov文件decode之后变成base64，audio无法播放 故不用
]);
}).then(([v, audio]) => {
parent.Spectrogram.spectrogram = v;
resolve();
// 后台执行CQT CQT的报错已经被拦截不会冒泡到下面的catch中
if (checkboxCQT.checked) parent.Analyser.cqt(
channels, tNum,
checkboxCQT.checked && checkboxGPU.checked
);
}).catch((e) => {
console.error(e);
parent.event.dispatchEvent(new CustomEvent('fileerror', { detail: e }));
reject(e);
}).finally(() => {
// 最终都要关闭进度条
parent.event.dispatchEvent(new CustomEvent('progress', { detail: -1 }));
parent.event.dispatchEvent(new Event('fileuiclose')); // 恢复drag功能
});
});
};
document.body.insertBefore(ui, document.body.firstChild); // 插入body的最前面
ui.focus();
return donePromise;
};
// 进度文件
this.projFile = {
export() {
if (!parent.Spectrogram._spectrogram) return null;
const data = {
midi: parent.MidiAction.midi,
channel: parent.MidiAction.channelDiv.channel,
beat: parent.BeatBar.beats,
dt: parent.dt,
A4: parent.Keyboard.freqTable.A4,
name: parent.AudioPlayer.name
};
if (parent.midiMode) return [data, {
length: parent.Spectrogram._spectrogram.length
}]; // midi模式不保存频谱
else return [data, parent.Spectrogram._spectrogram];
},
import(data) { // data: output of parse [obj, f32]
const obj = data[0];
parent.MidiAction.midi = obj.midi;
parent.MidiAction.selected = parent.MidiAction.midi.filter((obj) => obj.selected);
parent.MidiAction.channelDiv.fromArray(obj.channel);
parent.BeatBar.beats.copy(obj.beat);
parent.dt = obj.dt;
parent.TperP = parent.dt / parent._width; parent.PperT = parent._width / parent.dt;
parent.Keyboard.freqTable.A4 = obj.A4;
parent.Spectrogram.spectrogram = data[1];
parent.snapshot.save();
},
write(fileName = parent.AudioPlayer.name) {
const data = this.export();
bSaver.saveArrayBuffer(bSaver.combineArrayBuffers([
bSaver.String2Buffer("noteDigger"),
bSaver.Object2Buffer(data[0]),
bSaver.Float32Mat2Buffer(data[1])
]), fileName + '.nd');
},
parse(file) {
return new Promise((resolve, reject) => {
bSaver.readBinary(file, (b) => {
let [name, o] = bSaver.Buffer2String(b, 0);
if (name != "noteDigger") {
reject(new Error("incompatible file!"));
return;
}
let [obj, o1] = bSaver.Buffer2Object(b, o);
let [f32, _] = bSaver.Buffer2Float32Mat(b, o1);
resolve([obj, f32]);
});
});
}
};
// midi文件
this.midiFile = {
export: {
UI() {
let tempDiv = document.createElement('div');
tempDiv.innerHTML = `
<div class="request-cover">
<div class="card hvCenter" style="overflow: visible;">
<div class="fr" style="align-items: center;">
<label class="title">导出为midi</label>
<span style="flex:1"></span>
<button class="ui-cancel">❌</button>
</div>
<ul class="btn-ul layout-first">
<li class="layout dim-text" style="flex-direction: column;">
<button class="ui-confirm labeled" data-tooltip="可用于制谱；可能会损失、扭曲一些信息">节奏量化对齐</button>
<div class="fr layout-first" style="align-items: center;">
<span class="labeled" data-tooltip="越大越逼近听感，越小越规整\n若节奏已经精准，建议增大">对齐精度</span>
<input type="number" value="4" min="2" max="12" style="width: 2em;">
</div>
</li>
<li class="layout dim-text"><button class="ui-confirm labeled" data-tooltip="保证播放起来和这里一模一样，但丢失节奏信息\n适用于合成音频">和听起来一样</button></li>
<li class="layout"><label>仅导出可见音轨<input type="checkbox"></label></li>
</ul>
</div>
</div>`;
const card = tempDiv.firstElementChild;
const close = () => { card.remove(); };
const checkbox = card.querySelector('input[type="checkbox"]');
const btns = card.querySelectorAll('button');
btns[0].onclick = close;
btns[1].onclick = () => {;
const alignRate = parseInt(card.querySelector('input[type="number"]').value);
const midi = this.beatAlign(checkbox.checked, alignRate);
bSaver.saveArrayBuffer(midi.export(1), midi.name + '.mid');
close();
};
btns[2].onclick = () => {
const midi = this.keepTime(checkbox.checked);
bSaver.saveArrayBuffer(midi.export(1), midi.name + '.mid');
close();
};
document.body.insertBefore(card, document.body.firstChild);
card.tabIndex = 0;
card.focus();
},
/**
* 100%听感还原扒谱结果，但节奏是乱的
*/
keepTime(onlyVisible = false) {
const accuracy = 10;
const newMidi = new midi(60, [4, 4], Math.round(1000 * accuracy / parent.dt), [], parent.AudioPlayer.name);
const mts = [];
for (const ch of parent.synthesizer.channel) {
let mt = newMidi.addTrack();
mt.addEvent(midiEvent.instrument(0, ch.instrument));
mt._volume = ch.volume;
mts.push(mt);
}
for (const nt of parent.MidiAction.midi) {
if (onlyVisible && !parent.MidiAction.channelDiv.channel[nt.ch].visible) continue;
const midint = nt.y + 24;
let v = mts[nt.ch]._volume;
if (nt.v) v = Math.min(127, v * nt.v / 127);
mts[nt.ch].addEvent(midiEvent.note(nt.x1 * accuracy, (nt.x2 - nt.x1) * accuracy, midint, v));
} return newMidi;
},
beatAlign(onlyVisible = false, alignRate = 2) {
alignRate = Math.max(Math.round(alignRate), 2);
// 初始化midi
let begin = parent.BeatBar.beats[0];
let lastbpm = begin.bpm; // 用于自适应bpm
let lastPattern = `${begin.beatNum}/${begin.beatUnit}`;
const newMidi = new midi(lastbpm, [begin.beatNum, begin.beatUnit], 480, [], parent.AudioPlayer.name);
const mts = [];
for (const ch of parent.synthesizer.channel) {
let mt = newMidi.addTrack();
mt.addEvent(midiEvent.instrument(0, ch.instrument));
mt._volume = ch.volume;
mts.push(mt);
}
// 将每个音符拆分为两个时刻
const Midis = parent.MidiAction.midi;
const mlen = Midis.length << 1;
const moment = new Array(mlen);
for (let i = 0, j = 0; i < mlen; j++) {
const nt = Midis[j];
let duration = nt.x2 - nt.x1;
let midint = nt.y + 24;
let v = mts[nt.ch]._volume;
if (nt.v) v = Math.min(127, v * nt.v / 127);
moment[i++] = new midiEvent({
_d: duration,
ticks: nt.x1,
code: 0x9,
value: [midint, v],
_ch: nt.ch
}, true);
moment[i++] = new midiEvent({
_d: duration,
ticks: nt.x2,
code: 0x9,
value: [midint, 0],
_ch: nt.ch
}, true);
} moment.sort((a, b) => a.ticks - b.ticks);
// 对每个小节进行对齐
let m_i = 0; // moment的指针
let tickNow = 0; // 维护总时长
for (const measure of parent.BeatBar.beats) {
if (m_i == mlen) break;
//== 判断小节是否变化 假设小节之间bpm相关性很强 ==//
const bpmnow = measure.bpm;
if (Math.abs(bpmnow - lastbpm) > lastbpm * 0.065) {
mts[0].events.push(midiEvent.tempo(tickNow, bpmnow * 4 / measure.beatUnit));
} lastbpm = bpmnow;
const _ptn = `${measure.beatNum}/${measure.beatUnit}`;
if (lastPattern !== _ptn) {
mts[0].events.push(midiEvent.time_signature(tickNow, measure.beatNum, measure.beatUnit));
} lastPattern = _ptn;
//== 对齐音符 ==//
const begin = measure.start / parent.dt; // 转换为以“格”为单位
const end = (measure.interval + measure.start) / parent.dt;
// 一个八音符的格数
const aot = measure.interval * measure.beatUnit / (measure.beatNum * 8 * parent.dt);
while (m_i < mlen) {
const n = moment[m_i];
if (n.ticks > end) break; // 给下一小节
m_i++;
if (onlyVisible && !parent.MidiAction.channelDiv.channel[n._ch].visible) continue;
const threshold = n._d / alignRate;
let accuracy = aot;
while (accuracy > threshold) accuracy /= 2;
n.ticks = tickNow + ((Math.round((n.ticks - begin) / accuracy) * newMidi.tick * accuracy / aot) >> 1);
mts[n._ch].events.push(n);
} tickNow += newMidi.tick * measure.beatNum * 4 / measure.beatUnit;
} return newMidi;
}
},
/* 由于小节的数据结构，变速只能发生在小节开头
如果考虑节奏，则需要将小节内变速全部忽略
*/
import(file) {
bSaver.readBinary(file, (data) => {
let m;
try {
m = midi.import(new Uint8Array(data)).JSON();
} catch (error) {
console.error("Error importing MIDI:", error);
alert("导入MIDI文件时出错");
return;
}
const chdiv = parent.MidiAction.channelDiv;
chdiv.switchUpdateMode(false); // 下面会一次性创建大量音符，所以先关闭更新
let tickTimeTable = m.header.tempos ?? [{
ticks: 0, bpm: 120
}];
if (confirm("是否使用该MIDI的节奏?")) { // 对齐变速和节奏
// 将节奏型和变速事件合并排序
let rhy = [{ticks: -1, timeSignature: [4, 4]}, ...tickTimeTable, ...m.header.timeSignatures];
rhy.sort((a, b) => a.ticks - b.ticks);
rhy[0].ticks = 0;
// 合并时间相同的
let combined = [];
for (let i = 0; i < rhy.length; i++) {
const t = rhy[i].ticks;
let timeSignature = rhy[i].timeSignature;
let bpm = rhy[i].bpm;
let j = i + 1;
while (j < rhy.length && rhy[j].ticks == t) {
bpm = rhy[j].bpm ?? bpm; // 使用最新值
timeSignature = rhy[j].timeSignature ?? timeSignature;
j++;
}
combined.push({
ticks: t,
bpm: bpm,
timeSignature: timeSignature
});
i = j - 1;
}
// 为中间变速的情况创建小节并分配id
combined[0].id = 0;
for (let i = 1; i < combined.length; i++) {
const c = combined[i];
let j = i - 1;
let last = combined[j];
const ticksPerBar = m.header.tick * last.timeSignature[0] * 4 / last.timeSignature[1];
if (c.timeSignature) {
// 理论上小节改变不会出现在小节中 但为了兼容奇怪的MIDI需要微调
// 四舍五入到最近的小节开始位置
const bars = Math.round((c.ticks - last.ticks) / ticksPerBar);
c.id = bars + last.id;
const dt = last.ticks + bars * ticksPerBar - c.ticks;
if (dt === 0) continue;
// 平移后面所有事件
for (const mt of m.tracks) {
const notes = mt.notes;
// 找到第一个ticks大于等于c.ticks的事件
let idx = notes.findIndex(ev => ev.ticks >= c.ticks);
if (idx === -1) continue;
for (let k = idx; k < notes.length; k++) notes[k].ticks += dt;
}
for (let k = i; k < combined.length; k++) combined[k].ticks += dt;
} else {
// 如果节奏改变出现在小节中：
// 前1/2: 放到小节头; 后1/2: 放到下一个小节头
// 总是创建小节
while (c.ticks < last.ticks) last = combined[--j];
const bars = Math.floor((c.ticks - last.ticks) / ticksPerBar);
const offset = c.ticks - last.ticks - bars * ticksPerBar;
c.timeSignature = last.timeSignature;
if (offset < (ticksPerBar >> 1)) {
c.id = last.id + bars;
c.ticks = last.ticks + bars * ticksPerBar;
} else {
c.id = last.id + bars + 1;
c.ticks = last.ticks + (bars + 1) * ticksPerBar;
}
}
}
// 合并id相同的小节
rhy.length = 0;
let lastbpm = 120, lastTimeSignature = [4, 4];
for (let i = 0; i < combined.length; i++) {
const c = combined[i];
c.bpm ??= lastbpm;
c.timeSignature ??= lastTimeSignature;
let j = i + 1;
while (j < combined.length && combined[j].id == c.id) {
c.bpm = combined[j].bpm ?? c.bpm;
c.timeSignature = combined[j].timeSignature ?? c.timeSignature;
j++;
} rhy.push(c);
i = j - 1;
lastbpm = c.bpm;
lastTimeSignature = c.timeSignature;
} tickTimeTable = rhy;
// 设置节奏
const beats = parent.BeatBar.beats;
beats.length = 0;
for (const t of tickTimeTable) {
const msPerMeasure = 240000 * t.timeSignature[0] / (t.timeSignature[1] * t.bpm);
beats.push(new eMeasure(
t.id, -1, t.timeSignature[0], t.timeSignature[1], msPerMeasure
));
} beats.check();
}
const chArray = [];
let chArrayIndex = 0;
for (const mt of m.tracks) {
if (mt.notes.length == 0) continue;
let tickTimeIdx = -1;
let startTick = 0;
let nexttickTimeChange = 0;
let tickTime = 0; // 一个tick的毫秒数/parent.dt
let msBefore = 0; // 用parent.dt归一化后的之前的时间
let _timeOffset = 0;
const checkChange = (tick) => {
while (tick >= nexttickTimeChange) {
msBefore += (nexttickTimeChange - startTick) * tickTime;
tickTimeIdx++;
startTick = nexttickTimeChange;
nexttickTimeChange = tickTimeTable[tickTimeIdx + 1]?.ticks ?? Infinity;
tickTime = 60000 / (tickTimeTable[tickTimeIdx].bpm * m.header.tick * parent.dt);
_timeOffset = msBefore - startTick * tickTime;
} return tickTime;
}; checkChange(1);
const ch = chdiv.addChannel();
if (!ch) break; // 音轨已满，addChannel会返回undefined同时alert，所以只要break
const chid = ch.index;
ch.name = `导入音轨${chid}`;
ch.ch.instrument = mt.instruments[0]?.number || 0;
ch.instrument = TinySynth.instrument[ch.ch.instrument];
// 音符强度归一化到0-127 演奏和导出时用的是“通道音量*音符音量/127”
let maxIntensity = mt.notes.reduce((a, b) => a.intensity > b.intensity ? a : b).intensity;
ch.ch.volume = maxIntensity;
chArray[chArrayIndex++] = mt.notes.map((nt) => {
let t = checkChange(nt.ticks + 1);
const start = _timeOffset + nt.ticks * t;
const endTick = nt.ticks + nt.durationTicks;
let end;
if (endTick > nexttickTimeChange) { // 跨变速区间
// 暂存状态
const store = [tickTimeIdx, startTick, nexttickTimeChange, tickTime, msBefore, _timeOffset];
t = checkChange(nt.ticks + nt.durationTicks + 1);
end = _timeOffset + endTick * t;
// 恢复状态
[tickTimeIdx, startTick, nexttickTimeChange, tickTime, msBefore, _timeOffset] = store;
} else end = _timeOffset + endTick * t;
return { // 理应给x1和x2取整，但是为了尽量不损失信息就不取整了 不取整会导致导出midi时要取整
// x1: msBefore + (nt.ticks - startTick) * t,
x1: start,
x2: end,
y: nt.midi - 24,
ch: chid,
selected: false,
v: nt.intensity / maxIntensity * 127
};
});
}
for (const ch of chArray) parent.MidiAction.midi.push(...ch);
parent.MidiAction.midi.sort((a, b) => a.x1 - b.x1);
chdiv.switchUpdateMode(true, true, 0b111); // 打开更新并一次性处理积压请求
});
},
}
}
================================================
FILE: core/app_keyboard.js
================================================
/// <reference path="../dataProcess/analyser.js" />
/**
* 左侧键盘
* 会使用 parent.keyboard 画布
* @param {App} parent
*/
function _Keyboard(parent) {
/**
* 选中了哪个音，音的编号以midi协议为准（C1序号为24）
* 更新链: 'onmousemove' -> parent.mouseY setter -> this.highlight
*/
this.highlight = -1;
this.harmonics = [0];
Object.defineProperty(this, 'Harmonics', {
get() { return this.harmonics.length; },
set(n) {// 建议n<=6
n = Math.max(0, n | 0);
if (n == this.Harmonics) return;
this.harmonics = Array.from({ length: n }, (_, i) => Math.round(12 * Math.log2(i + 1)));
}
});
this.freqTable = new FreqTable(440); // 在parent.Analyser.stft中更新
// 以下为画键盘所需
const _idchange = new Int8Array([2, 2, 1, 2, 2, 2, -10, 2, 3, 2, 2, 2]); // id变化
const _ychange = new Float32Array(12); // 纵坐标变化，随this.height一起变化
this.setYchange = (h) => { // 需注册到parent.height setter中 且需要一次立即的更新（在parent中实现）
_ychange.set([
-1.5 * h, -2 * h, -1.5 * h, -1.5 * h, -2 * h, -2 * h, -1.5 * h,
-2 * h, -3 * h, -2 * h, -2 * h, -2 * h
]);
};
/**
* 仅当: 视野垂直变化 或 this.highlight 更改 时需要更新
* 是否更新的判断 交给parent完成
*/
this.render = () => {
// 绘制频谱区音符高亮
const actionCtx = parent.layers.action.ctx;
actionCtx.fillStyle = "#ffffff4f";
for (let i = this.Harmonics - 1; i >= 0; i--) {
let noteY = parent.layers.height - (this.highlight - 24 + this.harmonics[i]) * parent._height + parent.scrollY;
if (noteY < 0) continue;
actionCtx.fillRect(0, noteY, parent.layers.width, -parent._height);
}
const ctx = parent.keyboard.ctx;
const w = parent.keyboard.width;
const w2 = w * 0.618;
ctx.fillStyle = '#fff';
ctx.fillRect(0, 0, w, parent.keyboard.height);
let noteID = parent.idYstart + 24; // 最下面对应的音的编号
const note = noteID % 12; // 一个八度中的第几个音
let baseY = parent.rectYstart + note * parent._height; // 这个八度左下角的y坐标
noteID -= note; // 这个八度C的编号
while (true) {
ctx.beginPath(); // 必须写循环内
ctx.fillStyle = 'orange';
for (let i = 0, rectY = baseY, id = noteID; i < 7 & rectY > 0; i++) { // 画白键
let dy = _ychange[i];
if (this.highlight == id) ctx.fillRect(0, rectY, w, dy); // 被选中的
ctx.moveTo(0, rectY); // 画线即可 下划线
ctx.lineTo(w, rectY);
rectY += dy;
id += _idchange[i];
} ctx.stroke();
// 写音阶名
ctx.fillStyle = "black"; ctx.fillText(Math.floor(noteID / 12) - 1, w - parent._height * 0.75, baseY - parent._height * 0.3);
baseY -= parent._height; noteID++;
for (let i = 7; i < 12; i++) {
if (this.highlight == noteID) { // 考虑到只要画一次高亮，不必每次都改fillStyle
ctx.fillStyle = '#Ffa500ff';
ctx.fillRect(0, baseY, w2, -parent._height);
ctx.fillStyle = 'black';
} else ctx.fillRect(0, baseY, w2, -parent._height);
baseY += _ychange[i];
noteID += _idchange[i];
if (baseY < 0) return;
}
}
};
// 鼠标点击后发声
this.mousedown = () => {
let ch = parent.MidiAction.channelDiv.selected;
if (!ch || ch.mute) return;
ch = ch ? ch.ch : parent.synthesizer;
let nt = ch.play({ f: this.freqTable[this.highlight - 24] });
let last = this.highlight; // 除颤
const tplay = parent.audioContext.currentTime;
const move = () => {
if (last === this.highlight) return;
last = this.highlight;
let dt = parent.audioContext.currentTime - tplay;
parent.synthesizer.stop(nt, dt > 0.3 ? 0 : dt - 0.3);
nt = ch.play({ f: this.freqTable[this.highlight - 24] });
}; document.addEventListener('mousemove', move);
const up = () => {
let dt = parent.audioContext.currentTime - tplay;
parent.synthesizer.stop(nt, dt > 0.5 ? 0 : dt - 0.5);
document.removeEventListener('mousemove', move);
document.removeEventListener('mouseup', up);
}; document.addEventListener('mouseup', up);
};
}
================================================
FILE: core/app_midiaction.js
================================================
/// <reference path="../ui/channelDiv.js" />
/**
* 管理用户在钢琴卷帘上的动作
* @param {App} parent
*/
function _MidiAction(parent) {
this.clickX = 0; // 绝对坐标 单位像素
this.clickYid = 0; // 离散坐标 单位离散格
this.mode = 0; // 0: 笔模式 1: 选择模式
this.frameMode = 0; // 0: 框选 1: 列选 2: 行选
this.frameXid = -1; // 框选的终点的X序号(Y序号=this.Keyboard.highlight-24) 此变量便于绘制 如果是负数则不绘制
this.alphaIntensity = true; // 绘制音符时是否根据音量使用透明度
this.snapMode = 1; // 0: 不吸附 1: 吸附到网格线 -1: 吸附到小节线
/**
* 根据snapMode吸附x坐标到不同的位置
* @param {number} x x是全局位置 单位:像素
* @returns [l, r] 除以 parent.width 后是相对于谱面起点的坐标，和最短的下一个位置
*/
this.snapRound = (x) => {
// 最短为一个像素
if (this.snapMode == 0) return [x / parent._width, (x + 1) / parent._width];
// 最短一格
if (this.snapMode == 1) {
let i = x / parent._width | 0;
return [i, i + 1];
}
// 小节吸附
const m = parent.BeatBar.beats.getMeasure(Math.max(0, x * parent.TperP), true);
const start = m.start * parent.PperT;
const Interval = m.interval * parent.PperT;
const dp = Interval / m.beatNum;
// 没有细分
if (dp < parent.BeatBar.minInterval) return [start, start + Interval];
// 细分 和_BeatBar.render的逻辑一致
const noteNum = 1 << Math.log2(dp / parent.BeatBar.minInterval);
const subDp = dp / noteNum;
const n = ((x - start) / subDp | 0) * subDp + start;
return [n / parent._width, (n + subDp) / parent._width];
};
/* 一个音符 = {
y: 离散 和spectrum的y一致
x1: 离散 起点
x2: 离散 终点
ch: 音轨序号
selected: 是否选中
v: 音量，0~127，用户创建的音符无此选项，但导入的midi有 需要undefined兼容
} */
this.selected = []; // 选中的音符 无序即可
this.midi = []; // 所有音符 需要维护有序性
var _anyAction = false; // 用于在选中多个后判断松开鼠标时应该如何处理选中
if (!parent.synthesizer) throw new Error('MidiAction requires a synthesizer to be created.');
const cd = this.channelDiv = new ChannelList(document.getElementById('funcSider'), parent.synthesizer);
// 导入midi时创建音轨不应该update，而是应该在音符全创建完成后存档
cd.updateCount = -1; // -1表示需要update 否则表示禁用更新但记录了请求次数
cd.switchUpdateMode = (state, forceUpdate = false, type = 0b11) => { // 控制音轨的更新
if (state) { // 切换回使能update
if (cd.updateCount > 0 || forceUpdate) { // 如果期间有更新请求
this.updateView();
parent.snapshot.save(type);
} cd.updateCount = -1;
} else if (cd.updateCount < 0) { // 如果是从true切换为false
cd.updateCount = 0;
}
}
const updateOnReorder = () => {
if (cd.updateCount < 0) {
this.updateView();
parent.snapshot.save(0b11);
} else cd.updateCount++;
};
/**
* 触发add和remove后也可能会触发reorder，取决于增删的是否是最后一项（见channelDiv.js）
* 故不是总能触发reorder的更新存档功能updateOnReorder
* 而更新与存档必须在reorder之后，因为reorder会重新映射channel
* 为了避免重复存档，需要暂时屏蔽reorder的存档功能
* 等到reorder之后一定会发生的added和removed事件触发后再恢复
*/
const resumeReroderCallback = () => {
updateOnReorder(); // 稳定触发
cd.addEventListener('reorder', updateOnReorder);
};
cd.addEventListener('reorder', ({ detail }) => {
for (const nt of this.midi) nt.ch = detail[nt.ch];
}); // 重新映射音符 更新视图在updateOnReorder中
cd.addEventListener('reorder', updateOnReorder);
cd.addEventListener('remove', ({ detail }) => {
this.midi = this.midi.filter((nt) => nt.ch != detail.index);
this.selected = this.selected.filter((nt) => nt.ch != detail.index);
cd.removeEventListener('reorder', updateOnReorder);
});
cd.addEventListener('removed', resumeReroderCallback);
cd.addEventListener('add', () => {
cd.removeEventListener('reorder', updateOnReorder);
});
cd.addEventListener('added', resumeReroderCallback);
const saveOnStateChange = () => {
parent.snapshot.save(0b1);
}
cd.container.addEventListener('lock', ({ target }) => {
this.selected = this.selected.filter((nt) => {
if (nt.ch == target.index) return nt.selected = false;
return true;
});
});
cd.container.addEventListener('lock', saveOnStateChange);
// cd.container.addEventListener('visible', saveOnStateChange); // visible会联动lock，因此无需存档
cd.container.addEventListener('mute', saveOnStateChange);
cd.addEventListener('setted', saveOnStateChange);
this.insight = []; // 二维数组，每个元素为一个音轨视野内的音符 音符拾取依赖此数组
/**
* 更新this.MidiAction.insight
* 步骤繁琐，不必每次更新。触发时机:
* 1. channelDiv的reorder、added、removed，实际为updateOnReorder和switchUpdateMode
* 2. midi的增加、移动、改变长度（用户操作）。由于都会调用且最后调用changeNoteY，所以只需要在changeNoteY中调用
* 3. scroll2
* 4. midi的删除（用户操作）：deleteNote
* 5. ctrlZ、ctrlY、ctrlV
*/
this.updateView = () => {
const m = this.midi;
const channel = Array.from(this.channelDiv.channel, () => []);
this.insight = channel;
// 原来用的二分有bug，所以干脆全部遍历
for (const nt of m) {
if (nt.x1 >= parent.idXend) break;
if (nt.x2 < parent.idXstart) continue;
if (nt.y < parent.idYstart || nt.y >= parent.idYend) continue;
channel[nt.ch].push(nt);
}
// midi模式下，视野要比音符宽一页，或超出视野半页
if (parent.midiMode) {
const currentLen = parent.Spectrogram.spectrogram.length;
const apage = parent.layers.width / parent._width;
let minLen = (m.length ? m[m.length - 1].x2 : 0) + apage * 1.5 | 0;
let viewLen = parent.idXstart + apage | 0; // 如果视野在很外面，需要保持视野
if (viewLen > minLen) minLen = viewLen;
if (minLen != currentLen) parent.Spectrogram.spectrogram.length = minLen; // length触发audio.duration和this.xnum
}
parent.layers.action.dirty = true;
};
this.render = () => { // 按照insight绘制音符
const m = this.insight;
const s = parent.layers.action.ctx;
const c = this.channelDiv.channel;
for (let ch = m.length - 1; ch >= 0; ch--) {
if (m[ch].length === 0 || !c[ch].visible) continue;
let ntcolor = c[ch].color;
if (c[ch].lock) s.setLineDash([5, 5]);
for (const note of m[ch]) {
const params = [note.x1 * parent._width - parent.scrollX, parent.layers.height - note.y * parent._height + parent.scrollY, (note.x2 - note.x1) * parent._width, -parent._height];
if (note.selected) {
s.fillStyle = '#ffffff';
s.fillRect(...params);
s.strokeStyle = ntcolor;
s.strokeRect(...params);
} else {
if (this.alphaIntensity && note.v) {
s.fillStyle = ntcolor + Math.round(note.v ** 2 * 0.01581).toString(16); // 平方律显示强度
} else s.fillStyle = ntcolor;
s.fillRect(...params);
s.strokeStyle = '#ffffff';
s.strokeRect(...params);
}
}
s.setLineDash([]);
} if (!this.mode || this.frameXid < 0) return;
// 绘制框选动作
s.fillStyle = '#f0f0f088';
const clickXid = this.clickX / parent._width | 0;
let [xmin, xmax] = clickXid <= this.frameXid ? [clickXid, this.frameXid + 1] : [this.frameXid, clickXid + 1];
const Y = parent.Keyboard.highlight - 24;
let [ymin, ymax] = Y <= this.clickYid ? [Y, this.clickYid + 1] : [this.clickYid, Y + 1];
let x1, x2, y1, y2;
if (this.frameMode == 1) { // 列选
x1 = xmin * parent._width - parent.scrollX;
x2 = (xmax - xmin) * parent._width;
y1 = 0;
y2 = parent.layers.height;
} else if (this.frameMode == 2) { // 行选
x1 = 0;
x2 = parent.layers.width;
y1 = parent.layers.height - ymax * parent._height + parent.scrollY;
y2 = (ymax - ymin) * parent._height;
} else { // 框选
x1 = xmin * parent._width - parent.scrollX;
x2 = (xmax - xmin) * parent._width;
y1 = parent.layers.height - ymax * parent._height + parent.scrollY;
y2 = (ymax - ymin) * parent._height;
} s.fillRect(x1, y1, x2, y2);
};
/**
* 删除选中的音符 触发updateView
* @param {boolean} save 是否存档
*/
this.deleteNote = (save = true) => {
this.selected.forEach((v) => {
let i = this.midi.indexOf(v);
if (i != -1) this.midi.splice(i, 1);
});
this.selected.length = 0;
if (save) parent.snapshot.save(0b10);
this.updateView();
};
this.clearSelected = () => { // 取消已选
this.selected.forEach(v => { v.selected = false; });
this.selected.length = 0;
};
var _tempdy = 0;
this.changeNoteY = () => { // 要求在trackMouse之后添加入spectrum的mousemoveEnent
_anyAction = true;
let dy = parent.Keyboard.highlight - 24 - this.clickYid;
this.selected.forEach((v) => { v.y += dy - _tempdy; });
_tempdy = dy;
this.updateView();
};
// 记录操作音符时之前的值 由于changeNoteX和changeNoteDuration 互斥，因此共用
var prevValue = null;
/**
* 改变选中的音符的时长 需要保证和changeNoteX同时只能使用一个
* @param {MouseEvent} e
*/
this.changeNoteDuration = (e) => {
_anyAction = true;
// 兼容窗口滑动，以绝对坐标进行运算
let dx = e.offsetX + parent.scrollX - this.clickX;
// 此时prevValue存的是音符的v2*width值
for (let i = 0; i < prevValue.length; i++) {
const v = this.selected[i];
const [l, r] = this.snapRound(prevValue[i] + dx);
if (v.x1 >= r) {
const [_, nr] = this.snapRound(v.x1 * parent._width);
v.x2 = nr;
} else v.x2 = r;
}
};
this.changeNoteX = (e) => { // 由this.onclick_L调用
_anyAction = true;
let dx = e.offsetX + parent.scrollX - this.clickX;
// 此时prevValue存的是音符的v1*width值
for (let i = 0; i < prevValue.length; i++) {
const v = this.selected[i];
const d = v.x2 - v.x1;
const [l, r] = this.snapRound(prevValue[i] + dx);
v.x1 = Math.max(0, l);
v.x2 = v.x1 + d;
}
};
// 封装事件管理
const startMove = () => {
prevValue = this.selected.map(v => v.x1 * parent._width);
parent.layerContainer.addEventListener('mousemove', this.changeNoteX);
parent.layerContainer.addEventListener('mousemove', this.changeNoteY);
const removeEvent = () => {
parent.layerContainer.removeEventListener('mousemove', this.changeNoteX);
parent.layerContainer.removeEventListener('mousemove', this.changeNoteY);
document.removeEventListener('mouseup', removeEvent);
this.midi.sort((a, b) => a.x1 - b.x1); // 排序非常重要 因为查找被点击的音符依赖顺序
// 鼠标松开则存档
if (_anyAction) parent.snapshot.save(0b10);
}; document.addEventListener('mouseup', removeEvent);
};
const startDurationChange = () => {
prevValue = this.selected.map(v => v.x2 * parent._width);
parent.layerContainer.addEventListener('mousemove', this.changeNoteDuration);
parent.layerContainer.addEventListener('mousemove', this.changeNoteY);
const removeEvent = () => {
parent.layerContainer.removeEventListener('mousemove', this.changeNoteDuration);
parent.layerContainer.removeEventListener('mousemove', this.changeNoteY);
document.removeEventListener('mouseup', removeEvent);
// 鼠标松开则存档
if (_anyAction) parent.snapshot.save(0b10);
}; document.addEventListener('mouseup', removeEvent);
};
/**
* 框选音符的鼠标动作 由this.onclick_L调用
* 选中的标准：框住了音头
*/
this.selectAction = (mode = 0) => {
const clickXid = this.clickX / parent._width | 0; // 选择动作永远以格点为准
this.frameXid = clickXid; // 先置大于零，表示开始绘制
if (mode == 1) { // 列选
parent.layerContainer.addEventListener('mousemove', parent.trackMouseX);
const up = () => {
parent.layerContainer.removeEventListener('mousemove', parent.trackMouseX);
document.removeEventListener('mouseup', up);
let ch = this.channelDiv.selected;
if (ch && !ch.lock) {
ch = ch.index;
let [xmin, xmax] = clickXid <= this.frameXid ? [clickXid, this.frameXid + 1] : [this.frameXid, clickXid + 1];
for (const nt of this.midi) nt.selected = (nt.x1 >= xmin && nt.x1 < xmax && nt.ch == ch);
this.selected = this.midi.filter(v => v.selected);
} this.frameXid = -1;
}; document.addEventListener('mouseup', up);
} else if (mode == 2) { // 行选
const up = () => {
document.removeEventListener('mouseup', up);
let ch = this.channelDiv.selected;
if (ch && !ch.lock) {
ch = ch.index;
const Y = parent.Keyboard.highlight - 24;
let [ymin, ymax] = Y <= this.clickYid ? [Y, this.clickYid + 1] : [this.clickYid, Y + 1];
for (const nt of this.midi) nt.selected = (nt.y >= ymin && nt.y < ymax && nt.ch == ch);
this.selected = this.midi.filter(v => v.selected);
} this.frameXid = -1;
}; document.addEventListener('mouseup', up);
} else { // 框选
parent.layerContainer.addEventListener('mousemove', parent.trackMouseX);
const up = () => {
parent.layerContainer.removeEventListener('mousemove', parent.trackMouseX);
document.removeEventListener('mouseup', up);
let ch = this.channelDiv.selected;
if (ch && !ch.lock) {
ch = ch.index;
const Y = parent.Keyboard.highlight - 24;
let [xmin, xmax] = clickXid <= this.frameXid ? [clickXid, this.frameXid + 1] : [this.frameXid, clickXid + 1];
let [ymin, ymax] = Y <= this.clickYid ? [Y, this.clickYid + 1] : [this.clickYid, Y + 1];
for (const nt of this.midi) nt.selected = (nt.x1 >= xmin && nt.x1 < xmax && nt.y >= ymin && nt.y < ymax && nt.ch == ch);
this.selected = this.midi.filter(v => v.selected);
} this.frameXid = -1; // 表示不在框选
}; document.addEventListener('mouseup', up);
}
};
/**
* 添加音符的鼠标动作 由this.onclick_L调用
*/
this.addNoteAction = () => {
if (!this.channelDiv.selected && !this.channelDiv.selectChannel(0)) return; // 如果没有选中则默认第一个
if (this.channelDiv.selected.lock) return; // 锁定的音轨不能添加音符
// 取消已选
this.clearSelected();
// 添加新音符，设置已选
const [x1, x2] = this.snapRound(this.clickX);
const note = {
y: this.clickYid,
x1, x2,
ch: this.channelDiv.selected.index,
selected: true
}; this.selected.push(note);
{ // 二分插入
let l = 0, r = this.midi.length;
while (l < r) {
let mid = (l + r) >> 1;
if (this.midi[mid].x1 < note.x1) l = mid + 1;
else r = mid;
} this.midi.splice(l, 0, note);
}
_anyAction = true;
this.updateView();
startDurationChange();
};
/**
* MidiAction所有鼠标操作都由此分配
*/
this.onclick_L = (e) => {
//== step 1: 判断是否点在了音符上 ==//
_anyAction = false;
// 为了支持在鼠标操作的时候能滑动，记录绝对位置
_tempdy = 0;
const x = (this.clickX = e.offsetX + parent.scrollX) / parent._width;
if (x >= parent._xnum) { // 越界
this.clearSelected(); return;
}
const y = this.clickYid = parent.Keyboard.highlight - 24;
// 找到点击的最近的音符 由于点击不经常，所以用遍历足矣 只需要遍历insight的音符
let n = null;
for (let ch_id = 0; ch_id < this.insight.length; ch_id++) {
const chitem = this.channelDiv.channel[ch_id] // insight和channelDiv的顺序是一致的
if (!chitem.visible || chitem.lock) continue; // 隐藏、锁定的音轨选不中
const ch = this.insight[ch_id];
// 每层挑选左侧最靠近的（如果有多个）
let distance = parent._width * parent._xnum;
for (const nt of ch) { // 由于来自midi，因此每个音轨内部是有序的
let dis = x - nt.x1;
if (dis < 0) break;
if (y == nt.y && x < nt.x2) {
if (dis < distance) {
distance = dis;
n = nt;
}
}
} if (n) break; // 只找最上层的
}
if (!n) { // 添加或框选音符 关于lock的处理在函数中
if (this.mode) this.selectAction(this.frameMode);
else this.addNoteAction();
return;
}
this.channelDiv.selectChannel(n.ch);
//== step 2: 如果点击到了音符，ctrl是否按下 ==/
if (e.ctrlKey) { // 有ctrl表示多选
if (n.selected) { // 已经选中了，取消选中
this.selected.splice(this.selected.indexOf(n), 1);
n.selected = false;
} else { // 没选中，添加选中
this.selected.push(n);
n.selected = true;
} return;
}
//== step 3: 单选时，是否选中了多个(事关什么时候取消选中) ==//
if (this.selected.length > 1 && n.selected) { // 如果选择了多个，在松开鼠标的时候处理选中
const up = () => {
if (!_anyAction) { // 没有任何拖拽动作，说明为了单选
this.selected.forEach(v => { v.selected = false; });
this.selected.length = 0;
n.selected = true;
this.selected.push(n);
}
document.removeEventListener('mouseup', up);
}; document.addEventListener('mouseup', up);
} else { // 只选一个
if (n.selected) {
const up = () => {
if (!_anyAction) { // 没有任何拖拽动作，说明为了取消选中
this.selected.forEach(v => { v.selected = false; });
this.selected.length = 0;
} document.removeEventListener('mouseup', up);
}; document.addEventListener('mouseup', up);
} else {
this.selected.forEach(v => { v.selected = false; });
this.selected.length = 0;
n.selected = true;
this.selected.push(n);
}
}
//== step 4: 如果点击到了音符，添加移动事件 ==//
if (((e.offsetX + parent.scrollX) << 1) > (n.x2 + n.x1) * parent._width) { // 靠近右侧，调整时长
startDurationChange();
} else { // 靠近左侧，调整位置
startMove();
}
};
}
================================================
FILE: core/app_midiplayer.js
================================================
/**
* 管理用户绘制的midi的播放
* @param {App} parent
*/
function _MidiPlayer(parent) {
this.priorT = 1000 / 59; // 实际稳定在60帧，波动极小
this.realT = 1000 / 59;
this._last = performance.now();
this.lastID = -1;
this.restart = (onlyBeat = false) => {
// 需要-1，防止当前时刻开始的音符不被播放
const msnow = parent.AudioPlayer.audio.currentTime * 1000;
if (!onlyBeat) this.lastID = ((msnow / parent.dt) | 0) - 1;
this._beatIter = parent.BeatBar.beats.iterator(msnow, true);
const p = this._beatIter.next();
if (p.done === false) {
const m = p.value;
this._beatNowEnds = m.start + m.interval;
} else {
this._beatNowEnds = -1;
}
};
this.update = () => {
// 一阶预测
let tnow = performance.now();
// 由于requestAnimationFrame在离开界面的时候会停止，所以要设置必要的限定
if (tnow - this._last < (this.priorT << 1)) this.realT = 0.2 * (tnow - this._last) + 0.8 * this.realT; // IIR低通滤波
this._last = tnow;
if (parent.AudioPlayer.audio.paused) return;
const dt = 1e-3 / parent.AudioPlayer.audio.playbackRate;
const predictT = parent.time + 0.5 * (this.realT + this.priorT); // 先验和实测的加权和
const predictID = (predictT / parent.dt) | 0;
// 寻找(mp.lastID, predictID]之间的音符
const m = parent.MidiAction.midi;
if (m.length > 0) { // 二分查找要求长度大于0
let lastAt = m.length;
{ // 二分查找到第一个x1>mp.lastID的音符
let l = 0, r = lastAt - 1;
while (l <= r) {
let mid = (l + r) >> 1;
if (m[mid].x1 > this.lastID) {
r = mid - 1;
lastAt = mid;
} else l = mid + 1;
}
}
for (; lastAt < m.length; lastAt++) {
const nt = m[lastAt];
if (nt.x1 > predictID) break;
if (parent.MidiAction.channelDiv.channel[nt.ch].mute) continue;
parent.synthesizer.play({
id: nt.ch,
f: parent.Keyboard.freqTable[nt.y],
v: nt.v, // 用户创建的音符不可单独调整音量，为undefined，会使用默认值
t: (parent.time - nt.x1 * parent.dt) * dt,
last: (nt.x2 - nt.x1) * parent.dt * dt
});
}
}
// 节拍播放
if (this._ifBeat && this._beatNowEnds > 0) {
const endms = this._beatNowEnds;
// 较为宽裕的时间判断
if (endms < predictT + this.priorT) {
this.playBeatSound(parent.audioContext.currentTime + (endms - parent.time) * dt);
const p = this._beatIter.next();
if (p.done === false) {
const m = p.value;
this._beatNowEnds = m.start + m.interval;
} else {
this._beatNowEnds = -1;
}
}
}
this.lastID = predictID;
};
// 播放节拍
this._ifBeat = false;
this._beatIter = null;
this._beatNowEnds = -1;
this.switchBeatMode = (ifBeat) => {
this._ifBeat = ifBeat;
if (ifBeat === false) return;
this.initBeatSound().then(() => {
if (parent.AudioPlayer.audio.paused === false) this.restart(true);
});
};
this.beatBuffer = null;
this.initBeatSound = async () => {
if (this.beatBuffer) return;
try {
// 利用 fetch 转换 Base64 为 ArrayBuffer
const CLICK_SOUND_BASE64 = "data:audio/mpeg;base64,SUQzBAAAAAABRlRFTkMAAAAMAAADT3JpZ2luYXRvcgBUWFhYAAAAKgAAA29yaWdpbmF0b3JfcmVmZXJlbmNlAE9yaWdpbmF0b3JSZWZlcmVuY2UAVERSQwAAAAwAAAMyMDAwOjAwOjAwAFRYWFgAAAAeAAADY29kaW5nX2hpc3RvcnkAQ29kaW5nSGlzdG9yeQBUWFhYAAAAEgAAA3RpbWVfcmVmZXJlbmNlADAAVFNTRQAAAA4AAANMYXZmNjIuNC4xMDAAAAAAAAAAAAAAAP/7UMAAAAAAAAAAAAAAAAAAAAAAAEluZm8AAAAPAAAAAgAAAnEAqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqv//////////////////////////////////////////////////////////////////AAAAAExhdmM2Mi4xMwAAAAAAAAAAAAAAACQEWgAAAAAAAAJxo1jtAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/+1DEAAAJTFdXNBSAAZig7j8woAD1QEAAAIAJQAUExuAC/vf4Qyc5oyMVo9UFYJgmCYJhsnB8EDkH38QA+UBCJAQBAEHMg+D/wffg4GC4Pg+7+oHz5QEP+UDHOf4IAhl6hqZmRkhAAA2Gw2GAwIAmkAHzrD7iq7X2AqopRMhZ060gz1e2F2Ng2m1AtBbGIrF+IwgRk5RPyEesIcvUwet9xkRllReab+aw4cTGp5rf5EyEI9IGY3///RCJomX/8Akgk06BXPUgABsYAFWM2a0N//tSxAUDy817Jv2FAAgAADSAAAAET0ApeqwoITVM6TNhUkXiVM41cLoCkA0DURhcgUQKINoiiZ6mkI9HpykJKab6mmmmzjjWOOepptc447mm6HHHfzjv6///////6Hf//oc+v//1NNIRqkxBTUUzLjEwMKqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqo=";
const response = await fetch(CLICK_SOUND_BASE64);
const arrayBuffer = await response.arrayBuffer();
this.beatBuffer = await parent.audioContext.decodeAudioData(arrayBuffer);
} catch (e) {
alert("节拍音频解码失败:", e);
}
};
this.playBeatSound = (time = 0) => {
if (!this.beatBuffer) return;
const source = parent.audioContext.createBufferSource();
source.buffer = this.beatBuffer;
source.connect(parent.audioContext.destination);
source.start(time);
};
}
================================================
FILE: core/app_spectrogram.js
================================================
/**
* 管理频谱显示
* 会使用 parent.layers.spectrum 画布
* @param {App} parent
*/
function _Spectrogram(parent) {
this.colorStep1 = 90;
this.colorStep2 = 240;
this._multiple = parseFloat(document.getElementById('multiControl').value); // 幅度的倍数
Object.defineProperty(this, 'multiple', {
get: function() { return this._multiple; },
set: function(m) {
this._multiple = m;
parent.layers.spectrum.dirty = true;
}
});
this._contrast = parseFloat(document.getElementById('contrastControl').value); // 对比度
Object.defineProperty(this, 'contrast', {
get: function() { return this._contrast; },
set: function(c) {
this._contrast = c;
parent.layers.spectrum.dirty = true;
}
});
this._Hmultiple = 1; // 谐波的倍数
Object.defineProperty(this, 'Hmultiple', {
get: function() { return this._Hmultiple; },
set: function(m) {
this._Hmultiple = m;
parent.layers.spectrum.dirty = true;
}
});
this._spectrogram = null; // .raw属性为底层一维buffer
Object.defineProperty(this, 'spectrogram', {
get: function() { return this._spectrogram; },
set: function(s) {
if (!s) {
this._spectrogram = this.harmonic = null;
parent.xnum = 0;
} else {
if (s.raw != this._spectrogram?.raw) {
this._spectrogram = s;
this.harmonic = null;
}
parent.xnum = s.length;
parent.scroll2();
}
}
});
this.harmonic = null; // 对谐波的估计 在parent.Analyser._reduceHarmonic中计算得到
this.getColor = (value) => { // 0-step1，是蓝色的亮度从0变为50%；step1-step2，是颜色由蓝色变为红色；step2-255，保持红色
value = value || 0; // 防NaN
if (value < 0) value = 0;
let hue = 0, lightness = 50; // Red hue
if (value <= this.colorStep1) {
hue = 240; // Blue hue
lightness = (value / this.colorStep1) * 50; // Lightness from 0% to 50%
} else if (value <= this.colorStep2) {
hue = 240 - ((value - this.colorStep1) / (this.colorStep2 - this.colorStep1)) * 240;
} return `hsl(${hue}, 100%, ${lightness}%)`;
};
// 预计算颜色查找表
this.colorLUT = ((LUT_SIZE = 384) => {
const c = new OffscreenCanvas(LUT_SIZE, 1);
const ctx = c.getContext('2d', { alpha: false });
for (let i = 0; i < LUT_SIZE; i++) {
const value = (i / (LUT_SIZE - 1)) * 255;
ctx.fillStyle = this.getColor(value);
ctx.fillRect(i, 0, 1, 1);
}
const data = ctx.getImageData(0, 0, LUT_SIZE, 1).data;
// 检测平台字节序
const littleEndian = (function() {
const buf = new ArrayBuffer(4);
new DataView(buf).setUint32(0, 0x12345678, true);
return new Uint8Array(buf)[0] === 0x78;
})();
let u32 = new Uint32Array(LUT_SIZE);
u32.scale = (LUT_SIZE - 1) / 255;
for (let i = 0; i < LUT_SIZE; i++) {
const idx = i << 2;
const r = data[idx], g = data[idx + 1], b = data[idx + 2], a = 255;
// 小端: ARGB 大端: RGBA
u32[i] = littleEndian ? (a << 24) | (b << 16) | (g << 8) | r : (r << 24) | (g << 16) | (b << 8) | a;
} return u32;
})();
// 闭包存储
var imageData = null;
var dataCanvas = null;
this.onresize = () => { // 在parent.width / parent.height / parent.resize 中被调用
const canvas = parent.layers.spectrum;
const ctx = canvas.ctx;
// 实际视图的最大行列数
let cols = Math.ceil(canvas.width / parent._width) + 1, rows = Math.ceil(canvas.height / parent._height) + 1;
// 频谱列主序 这里存转置后的
imageData = ctx.createImageData(rows, cols);
ctx.imageSmoothingEnabled = parent._width < 1 || parent._height < 1;
imageData.u32 = new Uint32Array(imageData.data.buffer);
if (dataCanvas) {
dataCanvas.width = rows;
dataCanvas.height = cols;
} else {
dataCanvas = new OffscreenCanvas(rows, cols);
dataCanvas.ctx = dataCanvas.getContext('2d', { alpha: false });
}
ctx.strokeStyle = "#FFFFFF"; // 分界线颜色
ctx.fillStyle = '#25262d'; // 背景颜色
};
// 供外部调用的接口 返回一个可以用[][]访问的对象 值为最终计算结果
this.getCurrentSpectrum = () => {
const _this = this;
const view = {
length: this._spectrogram.length,
buffer: new Float32Array(this._spectrogram[0].length),
currentFrame: -1
};
return new Proxy(view, {
get(target, prop) {
if (prop in target) return target[prop];
const frameID = parseInt(prop);
if (isNaN(frameID) || frameID < 0 || frameID >= target.length) return undefined;
const b = target.buffer;
if (frameID === target.currentFrame) return b;
const s = _this._spectrogram[frameID];
const h = _this.harmonic?.[frameID];
if (h) for (let i = 0; i < b.length; i++) {
let amp = s[i] - h[i] * _this._Hmultiple;
b[i] = Math.pow(Math.max(0, amp), _this._contrast) * _this._multiple;
} else for (let i = 0; i < b.length; i++) {
b[i] = Math.pow(Math.max(0, s[i]), _this._contrast) * _this._multiple;
}
target.currentFrame = frameID;
return b;
}
});
};
this.renderSpectrum = (ctx) => {
// 填充数据到imagerData 随spectrum的列主序
for (let frameID = parent.idXstart, x = 0, off = 0; frameID < parent.idXend; frameID++, x++, off += imageData.width) {
const s = this._spectrogram[frameID];
const h = this.harmonic?.[frameID];
for (let y = parent.idYstart, j = off; y < parent.idYend; y++, j++) {
let amp = s[y] - (h?.[y] ?? 0) * this._Hmultiple;
amp = Math.pow(Math.max(0, amp), this._contrast) * this._multiple;
const colorID = Math.min(this.colorLUT.length - 1, Math.round(amp * this.colorLUT.scale));
imageData.u32[j] = this.colorLUT[colorID];
}
} dataCanvas.ctx.putImageData(imageData, 0, 0);
// 把dataCanvas画到目标canvas上 drawImage 承担三个任务：旋转、缩放、偏移
ctx.save();
ctx.translate(0, 0); ctx.rotate(-Math.PI * 0.5);
ctx.drawImage(
dataCanvas,
0, 0, imageData.width, imageData.height,
-parent.rectYstart, parent.rectXstart,
imageData.width * parent._height, imageData.height * parent._width
); ctx.restore();
}
this.render = ({ctx, width, height}) => {
if (this._spectrogram) this.renderSpectrum(ctx);
// 填涂剩余部分
let end = parent.idXend * parent._width - parent.scrollX;
let w = width - end;
if (w > 0) ctx.fillRect(end, 0, w, height);
// 绘制分界线
ctx.beginPath();
for (let y = (((parent.idYstart / 12) | 0) + 1) * 12,
rectY = height - parent._height * y + parent.scrollY,
dy = -12 * parent._height;
y < parent.idYend; y += 12, rectY += dy) {
ctx.moveTo(0, rectY);
ctx.lineTo(width, rectY);
} ctx.stroke();
};
}
================================================
FILE: core/app_timebar.js
================================================
/// <reference path="../ui/contextMenu.js" />
/**
* 顶部时间轴
* @param {App} parent
*/
function _TimeBar(parent) {
this.interval = 10; // 每个标注的间隔块数 在updateInterval中更新
// 重复区间参数 单位：毫秒 如果start>end则区间不起作用
this.repeatStart = -1;
this.repeatEnd = -1;
/**
* 设置重复区间专用函数 便于统一管理行为副作用
* @param {number || null} start 单位：毫秒
* @param {number || null} end 单位：毫秒
*/
this.setRepeat = (start = null, end = null) => {
if (start !== null) this.repeatStart = start;
if (end !== null) this.repeatEnd = end;
};
/**
* 毫秒转 分:秒:毫秒
* @param {number} ms 毫秒数
* @returns [分,秒,毫秒]
*/
this.msToClock = (ms) => {
return [
Math.floor(ms / 60000),
Math.floor((ms % 60000) / 1000),
ms % 1000 | 0
];
};
this.msToClockString = (ms) => {
const t = this.msToClock(ms);
return `${t[0].toString().padStart(2, "0")}:${t[1].toString().padStart(2, "0")}:${t[2].toString().padStart(3, "0")}`;
};
// timeBar的上半部分画时间轴 并绘制工作区重复区间和时间指针
this.render = () => {
const canvas = parent.timeBar;
const ctx = parent.timeBar.ctx;
let idstart = Math.ceil(parent.idXstart / this.interval - 0.1); // 画面中第一个时间点的序号
let dt = this.interval * parent.dt; // 时间的步长
let dp = parent.width * this.interval; // 像素的步长
let timeAt = dt * idstart; // 对应的毫秒
let p = idstart * dp - parent.scrollX; // 对应的像素
let h = canvas.height >> 1; // 上半部分
ctx.fillStyle = '#25262d';
ctx.fillRect(0, 0, canvas.width, h);
ctx.fillStyle = '#8e95a6';
//== 画刻度 标时间 ==//
ctx.strokeStyle = '#ff0000';
ctx.beginPath();
for (let endPix = canvas.width + (dp >> 1); p < endPix; p += dp, timeAt += dt) {
ctx.moveTo(p, 0);
ctx.lineTo(p, h);
ctx.fillText(this.msToClockString(timeAt), p - 28, 16);
} ctx.stroke();
//== 画重复区间 ==//
let begin = parent._width * this.repeatStart / parent.dt - parent.scrollX; // 单位：像素
let end = parent._width * this.repeatEnd / parent.dt - parent.scrollX;
const spectrum = parent.layers.action.ctx;
const spectrumHeight = parent.layers.height;
// 画线
if (begin >= 0 && begin < canvas.width) { // 画左边
ctx.beginPath(); spectrum.beginPath();
ctx.strokeStyle = spectrum.strokeStyle = '#20ff20';
ctx.moveTo(begin, 0); ctx.lineTo(begin, canvas.height);
spectrum.moveTo(begin, 0); spectrum.lineTo(begin, spectrumHeight);
ctx.stroke(); spectrum.stroke();
}
if (end >= 0 && end < canvas.width) { // 画右边
ctx.beginPath(); spectrum.beginPath();
ctx.strokeStyle = spectrum.strokeStyle = '#ff2020';
ctx.moveTo(end, 0); ctx.lineTo(end, canvas.height);
spectrum.moveTo(end, 0); spectrum.lineTo(end, spectrumHeight);
ctx.stroke(); spectrum.stroke();
}
// 画区间 如果begin>end则区间不起作用，不绘制
if (begin < end) {
begin = Math.max(begin + 1, 0); end = Math.min(end - 1, canvas.width);
ctx.fillStyle = spectrum.fillStyle = '#80808044';
ctx.fillRect(begin, 0, end - begin, canvas.height);
spectrum.fillRect(begin, 0, end - begin, spectrumHeight);
}
//== 画当前时间指针 ==//
spectrum.strokeStyle = 'white';
begin = parent.time / parent.dt * parent._width - parent.scrollX;
if (begin >= 0 && begin < canvas.width) {
spectrum.beginPath();
spectrum.moveTo(begin, 0);
spectrum.lineTo(begin, spectrumHeight);
spectrum.stroke();
}
};
this.updateInterval = () => { // 根据parent.width改变 在width的setter中调用
const fontWidth = parent.timeBar.ctx.measureText('00:00:000').width * 1.2;
// 如果间距小于fontWidth则细分
this.interval = Math.max(1, Math.ceil(fontWidth / parent._width));
};
this.contextMenu = new ContextMenu([
{
name: "设置重复区间开始位置",
callback: (e_father, e_self) => {
this.setRepeat((e_father.offsetX + parent.scrollX) * parent.TperP, null);
}
}, {
name: "设置重复区间结束位置",
callback: (e_father, e_self) => {
this.setRepeat(null, (e_father.offsetX + parent.scrollX) * parent.TperP);
}
}, {
name: '<span style="color: red;">取消重复区间</span>',
onshow: () => this.repeatStart >= 0 || this.repeatEnd >= 0,
callback: () => {
this.setRepeat(-1, -1);
}
}, {
name: "从此处播放",
callback: (e_father, e_self) => {
parent.AudioPlayer.stop();
parent.AudioPlayer.start((e_father.offsetX + parent.scrollX) * parent.TperP);
}
}
]);
}
================================================
FILE: dataProcess/AI/AIEntrance.js
================================================
var AI = {
combineChannels: (audioChannel) => {
const wav = new Float32Array(audioChannel.getChannelData(0));
// 求和。不求平均是因为模型内部有归一化
if (audioChannel.numberOfChannels !== 1) {
const len = wav.length;
for (let i = 1; i < audioChannel.numberOfChannels; i++) {
const cData = audioChannel.getChannelData(i);
for (let j = 0; j < len; j++) wav[j] += cData[j];
}
} return wav;
},
basicamt: (audioChannel) => {
const timeDomain = AI.combineChannels(audioChannel);
return new Promise((resolve, reject) => {
const basicamtWorker = new Worker("./dataProcess/AI/basicamt_worker.js");
basicamtWorker.onmessage = ({data}) => {
if (data.type === 'error') {
console.error(data.message);
reject("疑似因为音频过长导致内存不足！");
basicamtWorker.terminate();
} resolve(data); // 返回的是音符事件
basicamtWorker.terminate();
};
basicamtWorker.onerror = (e) => {
console.error(e.message);
reject(e);
basicamtWorker.terminate();
};
basicamtWorker.postMessage(timeDomain, [timeDomain.buffer]);
});
},
septimbre: (audioChannel, k) => {
const timeDomain = AI.combineChannels(audioChannel);
return new Promise((resolve, reject) => {
const septimbreWorker = new Worker("./dataProcess/AI/septimbre_worker.js");
septimbreWorker.onmessage = ({data}) => {
if (data.type === 'error') {
console.error(data.message);
reject("疑似因为音频过长导致内存不足！");
septimbreWorker.terminate();
} resolve(data);
septimbreWorker.terminate();
};
septimbreWorker.onerror = (e) => {
console.error(e.message);
reject(e);
septimbreWorker.terminate();
};
septimbreWorker.postMessage(k);
septimbreWorker.postMessage(timeDomain, [timeDomain.buffer]);
});
}
};
================================================
FILE: dataProcess/AI/SpectralClustering.js
================================================
/**
* 谱聚类算法
* @param {Array<Float32Array>} feats
* @param {number} numClusters
*/
function SpectralClustering(feats, numClusters, affinityFunc) {
// 1. 计算修改后的归一化拉普拉斯矩阵 Lsym = I + D^(-1/2) * W * D^(-1/2)
const L = TriangleMatrix.Lsym(feats, affinityFunc);
// 2. 使用正交迭代法计算前k个特征向量
const U = TriangleMatrix.orthogonalIteration(L, numClusters);
// console.log(U);
// 3. 转置并归一化
const { flatMatrix, n, k } = transposeAndNormalize(U);
// 4. 基于 Pivoted QR 选择聚类中心
return clusterQR(flatMatrix, n, k);
}
/**
* 转置并归一化 (Transpose & Normalize)
* 优化点:
* 1. 预计算范数：利用原始数据的内存布局（顺序读取）计算模长；乘法代替除法：预先计算 1/norm
* 2. 分块写入 (Tiling)：将转置过程分块，确保写入 `flat` 数组时命中缓存
* 3. 减少重复计算：提前解构引用，避免在循环内多次查找 `eigenVectors[r]`
* 4. 手动维护索引，消除循环内的乘法运算
* @param {Array<Float32Array>} eigenVectors 大小为 k 的数组，每个元素长 n
* @param {number} BLOCK_SIZE L1 缓存分块大小
* @returns {{flatMatrix: Float32Array, n: number, k: number}}
*/
function transposeAndNormalize(eigenVectors, BLOCK_SIZE = 1024) {
const k = eigenVectors.length;
const n = eigenVectors[0].length;
const flat = new Float32Array(n * k);
// normSq[i] 存储第 i 个数据点（即第 i 行）的模长平方
const normSq = new Float32Array(n);
// 1. 预计算模长 (保持不变，因为这是最高效的)
for (let r = 0; r < k; r++) {
const vec = eigenVectors[r];
for (let i = 0; i < n; i++) {
normSq[i] += vec[i] * vec[i];
}
}
// 归一化系数
for (let i = 0; i < n; i++) normSq[i] = 1.0 / Math.sqrt(normSq[i] + 1e-10);
// 2. 分块转置
// 由于 k 很小，一行的数据量很小 (20 bytes)。
// 我们可以适当增大 BLOCK_SIZE，比如 1024 或 2048
// 提前解构引用，避免在循环里查找 eigenVectors[r]
const vecs = Array.from({ length: k }, (_, i) => eigenVectors[i]);
for (let iBase = 0; iBase < n; iBase += BLOCK_SIZE) {
// 确定当前块的边界
const iLimit = (iBase + BLOCK_SIZE < n) ? (iBase + BLOCK_SIZE) : n;
for (let r = 0; r < k; r++) {
const vec = vecs[r];
// 手动维护索引，消除循环内的 (i * k) 乘法
// 初始索引：当前块起始行(iBase) * k + 当前列(r)
let flatIndex = iBase * k + r;
for (let i = iBase; i < iLimit; i++) {
// 直接使用指针
flat[flatIndex] = vec[i] * normSq[i];
// 步进为 k，因为 flat 是行优先存储，
// 同一列的下一个元素在 flat 中相隔 k 个位置
flatIndex += k;
}
}
}
return { flatMatrix: flat, n, k };
}
/**
* 完整的 Cluster QR 聚类
* 包含：中心点选择 + 标签分配
*
* @param {Float32Array} flatMatrix (n * k) 归一化后的特征矩阵 (只读)
* @param {number} n 点的数量
* @param {number} k 聚类数量
* @returns {Int32Array} 长度为 n 的数组，labels[i] 表示第 i 个点属于第几类 (0 到 k-1)
*/
function clusterQR(flatMatrix, n, k) {
// --- 阶段 1: 准备工作 ---
// 1. 必须复制一份数据用于 QR 分解的残差计算
// 因为 MGS 算法会破坏性地修改数据，而我们最后分配时需要原始数据
// 这里的内存开销是必要的 (n * k * 4 bytes)
const residualsMatrix = flatMatrix.slice();
const centroidIndices = new Int32Array(k);
const residualNorms = new Float32Array(n);
// 初始化残差模长 (由于输入已归一化，初始全为 1.0)
// 但为了保险，还是算一下，或者直接 fill(1.0) 如果上一步很自信
residualNorms.fill(1.0);
const currentPivot = new Float32Array(k);
// --- 阶段 2: 选择中心点 (Pivot Selection) ---
for (let step = 0; step < k; step++) {
// 2.1 寻找残差最大的点
let maxNorm = -1.0;
let pivotIdx = -1;
for (let i = 0; i < n; i++) {
if (residualNorms[i] > maxNorm) {
maxNorm = residualNorms[i];
pivotIdx = i;
}
}
// 记录中心点索引
centroidIndices[step] = pivotIdx;
if (maxNorm < 1e-6) break; // 剩余点都几乎为0了
// 2.2 提取 pivot 向量 (从残差矩阵中提取)
const pivotOffset = pivotIdx * k;
const pivotScale = 1.0 / Math.sqrt(maxNorm);
for (let j = 0; j < k; j++) {
gitextract_zaftgd5a/
├── LICENSE
├── README.md
├── app.js
├── core/
│ ├── app_analyser.js
│ ├── app_audioplayer.js
│ ├── app_beatbar.js
│ ├── app_hscrollbar.js
│ ├── app_io.js
│ ├── app_keyboard.js
│ ├── app_midiaction.js
│ ├── app_midiplayer.js
│ ├── app_spectrogram.js
│ └── app_timebar.js
├── dataProcess/
│ ├── AI/
│ │ ├── AIEntrance.js
│ │ ├── SpectralClustering.js
│ │ ├── basicamt_44100.onnx
│ │ ├── basicamt_worker.js
│ │ ├── dist/
│ │ │ └── ort-wasm-simd.wasm
│ │ ├── postprocess.js
│ │ ├── septimbre_44100.onnx
│ │ └── septimbre_worker.js
│ ├── ANA.js
│ ├── CQT/
│ │ ├── cqt.js
│ │ └── cqt_worker.js
│ ├── NNLS.js
│ ├── aboutANA.md
│ ├── analyser.js
│ ├── bpmEst.js
│ ├── fft_real.js
│ └── stftGPU.js
├── docs/
│ └── DEVELOPMENT.md
├── index.html
├── jsconfig.json
├── lib/
│ ├── beatBar.js
│ ├── fakeAudio.js
│ ├── midi.js
│ ├── saver.js
│ ├── snapshot.js
│ └── tinySynth.js
├── plugins/
│ ├── chordEst.js
│ └── pitchName.js
├── style/
│ ├── askUI.css
│ ├── channelDiv.css
│ ├── contextMenu.css
│ ├── icon/
│ │ └── iconfont.css
│ ├── myRange.css
│ ├── siderMenu.css
│ └── style.css
└── ui/
├── channelDiv.js
├── contextMenu.js
├── myRange.js
└── siderMenu.js
SYMBOL INDEX (272 symbols across 33 files)
FILE: app.js
function App (line 3) | function App() {
class LayeredCanvas (line 615) | class LayeredCanvas extends HTMLCanvasElement {
method new2d (line 616) | static new2d(canvas, alpha = true, desynchronized = true) {
method new (line 620) | static new(canvas, contextType = '2d', contextAttributes) {
method init (line 625) | init(contextType, contextAttributes) {
method resetHandlers (line 630) | resetHandlers(handlers) {
method register (line 640) | register(handler, priority = null) {
method unregister (line 649) | unregister(handler) {
method render (line 652) | render() {
FILE: core/app_analyser.js
function _Analyser (line 14) | function _Analyser(parent) {
FILE: core/app_audioplayer.js
function _AudioPlayer (line 7) | function _AudioPlayer(parent) {
FILE: core/app_beatbar.js
function _BeatBar (line 8) | function _BeatBar(parent) {
FILE: core/app_hscrollbar.js
function _HscrollBar (line 5) | function _HscrollBar(parent) {
FILE: core/app_io.js
function _IO (line 8) | function _IO(parent) {
FILE: core/app_keyboard.js
function _Keyboard (line 8) | function _Keyboard(parent) {
FILE: core/app_midiaction.js
function _MidiAction (line 7) | function _MidiAction(parent) {
FILE: core/app_midiplayer.js
function _MidiPlayer (line 5) | function _MidiPlayer(parent) {
FILE: core/app_spectrogram.js
function _Spectrogram (line 6) | function _Spectrogram(parent) {
FILE: core/app_timebar.js
function _TimeBar (line 7) | function _TimeBar(parent) {
FILE: dataProcess/AI/SpectralClustering.js
function SpectralClustering (line 6) | function SpectralClustering(feats, numClusters, affinityFunc) {
function transposeAndNormalize (line 29) | function transposeAndNormalize(eigenVectors, BLOCK_SIZE = 1024) {
function clusterQR (line 85) | function clusterQR(flatMatrix, n, k) {
class TriangleMatrix (line 201) | class TriangleMatrix {
method constructor (line 202) | constructor(size) {
method cosineAffinityExp (line 208) | static cosineAffinityExp(featureA, featureB) {
method Lsym (line 224) | static Lsym(features, func = TriangleMatrix.cosineAffinityExp) {
method _index (line 253) | _index(i, j) {
method mult_mat_optimized (line 264) | mult_mat_optimized(Q_in, Z_out) {
method orthogonalIteration (line 292) | static orthogonalIteration(A, numVectors, numIterations = 30) {
function SchmidtInPlace (line 323) | function SchmidtInPlace(V) {
FILE: dataProcess/AI/postprocess.js
function createNotes (line 1) | function createNotes(
function get_infered_onsets (line 138) | function get_infered_onsets(onsets, frames, n_diff = 2) {
function findPeak (line 167) | function findPeak(x2d, threshold = 0) {
FILE: dataProcess/AI/septimbre_worker.js
function clusterNotes (line 44) | function clusterNotes(note_events, embTensor, frameTensor, k=2) {
FILE: dataProcess/ANA.js
function autoNoteAlign (line 33) | function autoNoteAlign(noteSeq, spectrum, minLen = 2) {
FILE: dataProcess/CQT/cqt.js
function cqt (line 2) | function cqt(channels, tNum, fmin, useGPU = false) {
FILE: dataProcess/CQT/cqt_worker.js
class CQT (line 4) | class CQT {
method blackmanHarris (line 10) | static blackmanHarris(N) {
method constructor (line 34) | constructor(fs, fmin = 32.70319566257483, octaves = 7, bins_per_octave...
method iniKernel (line 52) | static iniKernel(Q, fs, fmin, bins_per_octave = 12, binNum = 84) {
method cqt (line 79) | cqt(x, hop, raw = null) {
method norm (line 104) | static norm(s) {
method initWebGPU (line 130) | async initWebGPU(workgroupsize = 256) {
method cqt_GPU (line 266) | cqt_GPU(audioData, hop) {
method norm_CPU (line 317) | async norm_CPU(buffer, numFrames) {
method freeGPU (line 339) | freeGPU() {
FILE: dataProcess/NNLS.js
class NNLSSolver (line 4) | class NNLSSolver {
method constructor (line 11) | constructor(K, M, lambda = 1e-4, buffer_r = null) {
method solve (line 34) | solve(A, b) {
method _solveSubProblem (line 99) | _solveSubProblem(A, b, n, pIdx, s) {
method _updateResidual (line 146) | _updateResidual(A, oldC, newS, n, pIdx) {
method _fullResidualUpdate (line 156) | _fullResidualUpdate(A, b, c, n, pIdx) {
method calcError (line 167) | calcError() {
FILE: dataProcess/analyser.js
class FreqTable (line 1) | class FreqTable extends Float32Array {
method constructor (line 2) | constructor(A4 = 440) {
method A4 (line 6) | set A4(A4) { // 负数强制更新
method A4 (line 26) | get A4() { return this[45]; }
class NoteAnalyser (line 29) | class NoteAnalyser { // 负责解析频谱数据
method constructor (line 34) | constructor(df, freq) {
method A4 (line 42) | set A4(freq) {
method A4 (line 46) | get A4() {
method updateRange (line 55) | updateRange(semiR = 0.667, leakR = 1, oversample = 32) {
method mel (line 103) | mel(real, imag, buffer = null) {
method mel2 (line 115) | mel2(eng, buffer = null) {
method normalize (line 129) | static normalize(engSpectrum) {
method Tonality (line 152) | static Tonality(noteTable) {
method autoFill (line 204) | static autoFill(noteTable, threshold, from = 0, to = 0) {
FILE: dataProcess/bpmEst.js
class SIGNAL (line 8) | class SIGNAL {
method findPeaks (line 15) | static findPeaks(arr, prominence = 0) {
method parabolicInterpolation (line 42) | static parabolicInterpolation(y1, y2, y3) {
method filter (line 66) | static filter(arr, b, a, inplace = false, reverse = false) {
method autoCorr (line 101) | static autoCorr(arr, points, result = undefined) {
method autoCorrSeg (line 126) | static autoCorrSeg(arr, points, hopInWin, hop = 1) {
method createSynthesisWindow (line 187) | static createSynthesisWindow(analysisWindow, hop) {
class Beat (line 212) | class Beat {
method fs2FFTN (line 219) | static fs2FFTN(fs, sec = 50) {
method compressOutliers (line 231) | static compressOutliers(onsetEnv, percent = 0.99, margin_ratio = 1.3) {
method detrend (line 254) | static detrend(onsetEnv) {
method onsetNorm (line 269) | static onsetNorm(onsetEnv) {
method extractOnset (line 292) | static extractOnset(spectrogram, a = 0.8) {
method floatGCD (line 322) | static floatGCD(idx, N) {
method corrBPM (line 355) | static corrBPM(corr, sr, BPMstd = 1, BPMu = 110) {
method tempo (line 401) | static tempo(onsetEnv, onset_sr, minBPM, winSec, hopSec = 1, centerBPM...
method EllisBeatTrack (line 457) | static EllisBeatTrack(onsetEnv, onset_sr, tightness = 100, bpm = -110,...
method beatLocalScore (line 496) | static beatLocalScore(onsetEnvelope, framesPerBeat) {
method beatTrackDp (line 559) | static beatTrackDp(localscore, framesPerBeat, frameRange = [1, Infinit...
method getLastBeat (line 631) | static getLastBeat(cumscore) {
method PLP (line 722) | static PLP(onsetEnv, onset_sr, rangeBPM = [40, 200], winLen, hopLen, p...
method PLPprior (line 792) | static PLPprior(BPMt = [], std = 0.2) {
method beatStrength (line 810) | static beatStrength(onsetEnv, beatIndices, winLen = 5) {
method rhythmicPattern (line 829) | static rhythmicPattern(beatStrength, patterns = [2, 3, 4]) {
method detectDownbeats (line 871) | static detectDownbeats(beatStrength, meters = [2, 3, 4]) {
FILE: dataProcess/fft_real.js
class realFFT (line 4) | class realFFT {
method reverseBits (line 10) | static reverseBits(N) {
method ComplexMul (line 34) | static ComplexMul(a = 0, b = 0, c = 0, d = 0) {
method ComplexAbs (line 43) | static ComplexAbs(r, i, l) {
method constructor (line 56) | constructor(N, window = 'hanning') {
method initWindow (line 74) | initWindow() {
method ini (line 97) | ini(N) {
method _fftOther (line 113) | _fftOther() {
method fft (line 144) | fft(input, offset = 0) {
method ifft (line 178) | ifft(real, imag) {
FILE: dataProcess/stftGPU.js
class STFTGPU (line 4) | class STFTGPU {
method reverseBits (line 5) | static reverseBits(N) {
method constructor (line 16) | constructor(fftN = 8192, hopSize) {
method initWebGPU (line 21) | async initWebGPU(workgroup_size_pow = 8) { // 最大为8
method stft (line 196) | stft(audioBuffer) {
method readGPU (line 267) | readGPU(buffer = this.outputBuffer) {
method free (line 285) | free() {
FILE: lib/beatBar.js
class aMeasure (line 2) | class aMeasure {
method constructor (line 9) | constructor(beatNum = 4, beatUnit = 4, interval = 2000) {
method fromBpm (line 20) | static fromBpm(beatNum, beatUnit, bpm) {
method copy (line 24) | copy(obj) {
method bpm (line 31) | get bpm() {
method bpm (line 34) | set bpm(value) {
method isEqual (line 37) | isEqual(other) {
class eMeasure (line 43) | class eMeasure extends aMeasure {
method constructor (line 52) | constructor(id = 0, start = 0, beatNum, beatUnit, interval) {
method baseOn (line 70) | static baseOn(base, id, measure = undefined) {
class Beats (line 75) | class Beats extends Array {
method constructor (line 80) | constructor(maxTime = 60000) {
method getBaseIndex (line 91) | getBaseIndex(at, timeMode = false) {
method iterator (line 128) | iterator(at, timeMode = false) { // 由于在绘制更新中使用，故没有复用getBaseIndex以加速运行
method getMeasure (line 145) | getMeasure(at, timeMode = false) {
method setMeasure (line 161) | setMeasure(at, measure = undefined, timeMode = false, returnIdx = fals...
method check (line 188) | check(merge = true) {
method delete (line 206) | delete(at, timeMode = false) {
method add (line 222) | add(at, timeMode = false) {
method copy (line 234) | copy(beatArray) {
method [Symbol.iterator] (line 104) | [Symbol.iterator](index = 0, baseAt = 0) {
FILE: lib/fakeAudio.js
function FakeAudio (line 10) | function FakeAudio(duration = Infinity) {
FILE: lib/midi.js
class midiEvent (line 7) | class midiEvent {
method #constructor_args (line 18) | #constructor_args(ticks, code, value) {
method #constructor_obj (line 30) | #constructor_obj(eventObj, reference = true) {
method constructor (line 47) | constructor() {
method type (line 58) | get type() {
method export (line 69) | export(current_tick = 0, channel = 0) {
method note (line 79) | static note(at, duration, note, intensity) {
method instrument (line 98) | static instrument(at, instrument) {
method control (line 109) | static control(at, id, Value) {
method tempo (line 116) | static tempo(at, bpm) {
method time_signature (line 124) | static time_signature(at, numerator, denominator) {
method port (line 137) | static port(port = 0) {
class mtrk (line 146) | class mtrk {
method constrain (line 148) | static constrain(value, min = 0, max = 127) { return Math.min(max, Mat...
method tick_hex (line 155) | static tick_hex(ticknum) {
method string_hex (line 172) | static string_hex(str, maxBytes = -1) {
method hex_string (line 187) | static hex_string(bytes) {
method number_hex (line 198) | static number_hex(num, x = -1) {
method constructor (line 220) | constructor(name = "", event_list = Array()) {
method addEvent (line 231) | addEvent(event) {
method align (line 258) | align(tick, accuracy = 4) {
method sort (line 268) | sort() {
method export (line 281) | export(track_id) {
method JSON (line 314) | JSON(track_id) {
method toJSON (line 385) | toJSON(track_id) {
class midi (line 390) | class midi {
method constructor (line 400) | constructor(bpm = 120, time_signature = [4, 4], tick = 480, Mtrk = [],...
method addTrack (line 414) | addTrack(newtrack = null, channel_id = -1) {
method tracks (line 423) | get tracks() { // 起个别名
method align (line 430) | align(accuracy = 4) {
method import (line 441) | static import(midi_file, which_main = 0) {
method export (line 608) | export(type = 1) {
method JSON (line 684) | JSON() {
method toJSON (line 715) | toJSON() {
FILE: lib/saver.js
method Float32Mat2Buffer (line 33) | Float32Mat2Buffer(Float32Mat) {
method Buffer2Float32Mat (line 53) | Buffer2Float32Mat(arrayBuffer, offset = 0) {
method String2Buffer (line 70) | String2Buffer(str) {
method Buffer2String (line 85) | Buffer2String(arrayBuffer, offset = 0) {
method Object2Buffer (line 98) | Object2Buffer(obj) {
method Buffer2Object (line 107) | Buffer2Object(arrayBuffer, offset = 0) {
method combineArrayBuffers (line 117) | combineArrayBuffers(arrayBuffers) {
method saveArrayBuffer (line 136) | saveArrayBuffer(arrayBuffer, filename) {
method readBinary (line 150) | readBinary(file, callback) {
FILE: lib/snapshot.js
class Snapshot (line 3) | class Snapshot extends Array {
method constructor (line 9) | constructor(maxLen, iniState = '') {
method add (line 22) | add(snapshot) {
method undo (line 31) | undo() {
method redo (line 41) | redo() {
method lastState (line 51) | lastState() {
method nextState (line 59) | nextState() {
method nowState (line 63) | nowState() {
FILE: lib/tinySynth.js
class TinySynth (line 2) | class TinySynth {
method initSoundFont (line 186) | static initSoundFont({ g, w, t, f, v, a, h, d, s, r, p, q, k } = TinyS...
method initOneSoundFont (line 194) | static initOneSoundFont(id, { g, w, t, f, v, a, h, d, s, r, p, q, k } ...
method midi_instrument (line 198) | static midi_instrument(id) {
method constructor (line 201) | constructor(actx = new AudioContext(), loadAll = false) {
method audioContext (line 228) | get audioContext() {
method audioContext (line 231) | set audioContext(actx) {
method volume (line 267) | get volume() {
method volume (line 274) | set volume(v) {
method addChannel (line 282) | addChannel(at = this.channel.length, instrument = 0, gain = 1) {
method play (line 303) | play({ id, f = 440, v = 127, t = 0, last = 9999 } = {}) {
method stop (line 375) | stop(nt, t = 0) {
method checkStop (line 398) | checkStop() { // 自动回收 一直开启
method stopAll (line 408) | stopAll() {
FILE: plugins/chordEst.js
class ChordEst (line 110) | class ChordEst {
method constructor (line 125) | constructor(harmonicDecay = 0.3, P_N = 0.1, P_keep = 0.98) {
method step (line 302) | step(obs) {
method decode (line 352) | decode() {
method initOctaveW (line 375) | static initOctaveW() {
method chroma (line 398) | chroma(spect84, buffer = null) {
FILE: ui/channelDiv.js
function dragList (line 6) | function dragList(takeplace = true) {
class ChannelItem (line 78) | class ChannelItem extends HTMLDivElement {
method constructor (line 82) | constructor() {
method new (line 98) | static new(name = "channel", color = "red", instrument = "Piano", visi...
method name (line 141) | get name() {
method name (line 144) | set name(channelName) {
method instrument (line 147) | get instrument() {
method instrument (line 150) | set instrument(instrument) {
method color (line 153) | get color() {
method color (line 156) | set color(color) {
method lock (line 159) | get lock() {
method lock (line 162) | set lock(lock) {
method visible (line 175) | get visible() { // true为可见
method visible (line 178) | set visible(visible) {
method mute (line 191) | get mute() { // true为静音
method mute (line 194) | set mute(mute) {
method index (line 210) | get index() {
method index (line 213) | set index(index) {
method toJSON (line 216) | toJSON() { // 用于序列化，以实现撤销
class ChannelList (line 236) | class ChannelList extends EventTarget {
method whichItem (line 254) | static whichItem(target) {
method judgeClick (line 260) | static judgeClick(e) { return ChannelList.whichItem(e.target); }
method constructor (line 266) | constructor(div, synthesizer) {
method updateRange (line 341) | updateRange() {
method borrowColor (line 362) | borrowColor() {
method borrowTheColor (line 370) | borrowTheColor(color) {
method returnColor (line 380) | returnColor(color) {
method addChannel (line 396) | addChannel(at = this.channel.length) { // 用于一个个添加
method removeChannel (line 421) | removeChannel(node) {
method selectChannel (line 441) | selectChannel(node) {
method settingPannel (line 454) | settingPannel(chid) {
method _toJSON (line 503) | _toJSON() {
method fromArray (line 519) | fromArray(array) {
FILE: ui/contextMenu.js
class ContextMenu (line 1) | class ContextMenu {
method constructor (line 17) | constructor(items = [], mustShow = false) {
method addItem (line 22) | addItem(name, callback, onshow = null, event = "click") {
method removeItem (line 27) | removeItem(name) {
method show (line 36) | show(e) {
FILE: ui/myRange.js
class myRange (line 1) | class myRange extends HTMLInputElement {
method new (line 7) | static new(ele) {
method init (line 16) | init() {
method value (line 25) | set value(v) {
method value (line 29) | get value() {
method reset (line 32) | reset() {
class LableRange (line 38) | class LableRange extends myRange {
method new (line 39) | static new(ele) {
method init (line 47) | init() {
method updateLabel (line 66) | updateLabel() {
class hideLableRange (line 71) | class hideLableRange extends myRange {
method new (line 73) | static new(ele) {
method init (line 81) | init() {
method updateLabel (line 104) | updateLabel() {
method labelPosition (line 107) | labelPosition() {
FILE: ui/siderMenu.js
class SiderContent (line 1) | class SiderContent extends HTMLDivElement {
method new (line 2) | static new(ele, minWidth) {
method init (line 7) | init(minWidth) {
method _mousedown (line 23) | _mousedown(e) {
method _resize (line 28) | _resize(e) {
method _mouseup (line 39) | _mouseup() {
method display (line 45) | get display() {
method display (line 49) | set display(state) {
method width (line 55) | get width() {
method width (line 58) | set width(w) {
class SiderMenu (line 66) | class SiderMenu extends HTMLDivElement {
method new (line 74) | static new(menu, container, minWidth) {
method init (line 79) | init(box, minWidth) {
method add (line 94) | add(name, tabClass, dom, selected = false) {
method select (line 118) | select(tab) {
method show (line 133) | show(ifshow = true) {
method _tabClick (line 137) | _tabClick(e) {
Condensed preview — 52 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (1,462K chars).
[
{
"path": "LICENSE",
"chars": 35149,
"preview": " GNU GENERAL PUBLIC LICENSE\n Version 3, 29 June 2007\n\n Copyright (C) 2007 Free "
},
{
"path": "README.md",
"chars": 12047,
"preview": "<div align=\"center\">\n <a href=\"https://madderscientist.github.io/noteDigger/\" target=\"_blank\">\n <img width=\"240\" src"
},
{
"path": "app.js",
"chars": 29601,
"preview": "// 用这种方式(原始构造函数)的原因：解耦太难了，不解了。this全部指同一个。其次为了保证效率\n// 防止在html初始化之前getElement，所以封装成了构造函数，而不是直接写obj\nfunction App() {\n th"
},
{
"path": "core/app_analyser.js",
"chars": 26302,
"preview": "/// <reference path=\"../dataProcess/fft_real.js\" />\n/// <reference path=\"../dataProcess/stftGPU.js\" />\n/// <reference pa"
},
{
"path": "core/app_audioplayer.js",
"chars": 5515,
"preview": "/// <reference path=\"../lib/fakeAudio.js\" />\n\n/**\n * 音频播放\n * @param {App} parent \n */\nfunction _AudioPlayer(parent) {\n "
},
{
"path": "core/app_beatbar.js",
"chars": 10150,
"preview": "/// <reference path=\"../lib/beatBar.js\" />\n/// <reference path=\"../ui/contextMenu.js\" />\n\n/**\n * 顶部小节轴\n * @param {App} p"
},
{
"path": "core/app_hscrollbar.js",
"chars": 2141,
"preview": "/**\n * 配合scroll的滑动条\n * @param {App} parent \n */\nfunction _HscrollBar(parent) {\n this.maxScrollX = 0;\n this.refresh"
},
{
"path": "core/app_io.js",
"chars": 30673,
"preview": "/// <reference path=\"../lib/saver.js\" />\n/// <reference path=\"../lib/midi.js\" />\n\n/**\n * 文件相关操作\n * @param {App} parent \n"
},
{
"path": "core/app_keyboard.js",
"chars": 4388,
"preview": "/// <reference path=\"../dataProcess/analyser.js\" />\n\n/**\n * 左侧键盘\n * 会使用 parent.keyboard 画布\n * @param {App} parent \n */\nf"
},
{
"path": "core/app_midiaction.js",
"chars": 18805,
"preview": "/// <reference path=\"../ui/channelDiv.js\" />\n\n/**\n * 管理用户在钢琴卷帘上的动作\n * @param {App} parent \n */\nfunction _MidiAction(pare"
},
{
"path": "core/app_midiplayer.js",
"chars": 5246,
"preview": "/**\n * 管理用户绘制的midi的播放\n * @param {App} parent \n */\nfunction _MidiPlayer(parent) {\n this.priorT = 1000 / 59; // 实际稳定"
},
{
"path": "core/app_spectrogram.js",
"chars": 7390,
"preview": "/**\n * 管理频谱显示\n * 会使用 parent.layers.spectrum 画布\n * @param {App} parent \n */\nfunction _Spectrogram(parent) {\n this.colo"
},
{
"path": "core/app_timebar.js",
"chars": 4943,
"preview": "/// <reference path=\"../ui/contextMenu.js\" />\n\n/**\n * 顶部时间轴\n * @param {App} parent \n */\nfunction _TimeBar(parent) {\n "
},
{
"path": "dataProcess/AI/AIEntrance.js",
"chars": 2004,
"preview": "var AI = {\ncombineChannels: (audioChannel) => {\n const wav = new Float32Array(audioChannel.getChannelData(0));\n //"
},
{
"path": "dataProcess/AI/SpectralClustering.js",
"chars": 10615,
"preview": "/**\n * 谱聚类算法\n * @param {Array<Float32Array>} feats\n * @param {number} numClusters \n */\nfunction SpectralClustering(feats"
},
{
"path": "dataProcess/AI/basicamt_worker.js",
"chars": 937,
"preview": "// const ort_folder = 'https://cdn.jsdelivr.net/npm/onnxruntime-web/dist/';\n// self.importScripts(ort_folder + 'ort.wasm"
},
{
"path": "dataProcess/AI/postprocess.js",
"chars": 6926,
"preview": "function createNotes(\n onsetTensor, frameTensor,\n frame_thresh = 0.22,\n onset_thresh = 0.38,\n min_note_len ="
},
{
"path": "dataProcess/AI/septimbre_44100.onnx",
"chars": 575504,
"preview": "\b\n\u0012\u0007pytorch\u001a\u000b2.9.0+cu128:*\n\u0005\n\u0005audio\u0012\u0005val_0\u001a\fnode_Shape_0\"\u0005Shape*\n\n\u0003end\u0018\u0003\u0001\u0002*\f\n\u0005start\u0018\u0002\u0001\u0002J\u0001\n\tnamespace\u0012w_empty_nn_module_s"
},
{
"path": "dataProcess/AI/septimbre_worker.js",
"chars": 3400,
"preview": "// const ort_folder = 'https://cdn.jsdelivr.net/npm/onnxruntime-web/dist/';\n// self.importScripts(ort_folder + 'ort.wasm"
},
{
"path": "dataProcess/ANA.js",
"chars": 5668,
"preview": "/**\n * @file ANA.js (auto note alignment)\n * @abstract 融合HMM和DTW的音符自动对齐\n * @description\n * ## 记法\n * 从左到右——时频谱从开始到结尾\n * 从"
},
{
"path": "dataProcess/CQT/cqt.js",
"chars": 679,
"preview": "// 开启CQT的Worker线程，因为CQT是耗时操作，所以放在Worker线程中\nfunction cqt(channels, tNum, fmin, useGPU = false) {\n return new Promise(("
},
{
"path": "dataProcess/CQT/cqt_worker.js",
"chars": 14930,
"preview": "/**\n * 用定义计算CQT，时间复杂度很高，但是分析效果好\n */\nclass CQT {\n /**\n * 创建窗函数 幅度加起来为1\n * @param {number} N \n * @returns {"
},
{
"path": "dataProcess/NNLS.js",
"chars": 6320,
"preview": "/**\n * 为密集计算设计的高性能非负最小二乘求解器\n */\nclass NNLSSolver {\n /**\n * @param {number} K 个数\n * @param {number} M 维度\n "
},
{
"path": "dataProcess/aboutANA.md",
"chars": 4531,
"preview": "# JE数字谱自动对齐音频 Auto Note Alignment\n2025/12/27 文章已经整理到知乎: https://zhuanlan.zhihu.com/p/1988276192063800011\n\n下文为8/21刚完成算法时的"
},
{
"path": "dataProcess/analyser.js",
"chars": 9956,
"preview": "class FreqTable extends Float32Array {\n constructor(A4 = 440) {\n super(84); // 范围是C1-B7\n this.A4 = A4;"
},
{
"path": "dataProcess/bpmEst.js",
"chars": 35645,
"preview": "///<reference path=\"fft_real.js\" />\n/**\n * @file bpmEst.js\n * @abstract BPM估计相关算法\n * @description 算法说明: https://zhuanlan"
},
{
"path": "dataProcess/fft_real.js",
"chars": 8247,
"preview": "/**\n * 目前我写的最快的实数FFT。为音乐频谱分析设计\n */\nclass realFFT {\n /**\n * 位反转数组 最大支持2^16点\n * @param {number} N 2的正整数幂\n *"
},
{
"path": "dataProcess/stftGPU.js",
"chars": 12763,
"preview": "/**\n * Short-Time Real Fourier Transform using WebGPU\n */\nclass STFTGPU {\n static reverseBits(N) {\n const reve"
},
{
"path": "docs/DEVELOPMENT.md",
"chars": 5083,
"preview": "# 开发概述\n基本功能在`core/`文件夹中，包括：\n- 频谱分析与绘制\n- 音符播放与绘制\n- 音频播放与时间管理。音频是整个项目的同步时钟\n- 左侧键盘的绘制\n- 顶部的时间轴和小节轴\n- 文件输入输出\n\n## 时频分析\n用`Web "
},
{
"path": "index.html",
"chars": 33727,
"preview": "<!DOCTYPE html>\n<html lang=\"zh\">\n\n<head>\n <meta charset=\"UTF-8\">\n <meta name=\"viewport\" content=\"width=device-widt"
},
{
"path": "jsconfig.json",
"chars": 202,
"preview": "{\n \"compilerOptions\": {\n \"target\": \"ES2023\",\n \"checkJs\": false, // 不启用类型检查 因为用了很多流氓写法\n \"allowJs\": true, /"
},
{
"path": "lib/beatBar.js",
"chars": 7763,
"preview": "// 本文件用于管理小节信息，实现了稀疏存储小节的数据结构\nclass aMeasure {\n /**\n * 构造一个小节\n * @param {number | aMeasure} beatNum 分子 几拍为一小节"
},
{
"path": "lib/fakeAudio.js",
"chars": 2468,
"preview": "/**\n * 模拟没有声音、时长可变的Audio。模拟了：\n * 设置currentTime跳转播放位置\n * 设置playbackRate改变播放速度\n * play()和pause()控制播放\n * 到duration后自动停止，触发o"
},
{
"path": "lib/midi.js",
"chars": 26799,
"preview": "/**\n * 对midi事件进行封装\n * 相比于原生midi事件：\n * - 使用绝对时间（在导出为二进制时被mtrk转换为相对时间）\n * - 不记录通道信息（在导出为二进制时由mtrk加上通道信息）\n */\nclass midiEve"
},
{
"path": "lib/saver.js",
"chars": 6275,
"preview": "/* 示例\nfunction save() {\n let B = bSaver.Float32Mat2Buffer(b);\n let A = bSaver.Object2Buffer(a);\n bSaver.saveArr"
},
{
"path": "lib/snapshot.js",
"chars": 1680,
"preview": "// 基于快照的撤销重做数据结构\n// 为了不改变数组大小减小开销，使用循环队列\nclass Snapshot extends Array {\n /**\n * 新建快照栈\n * @param {number} maxL"
},
{
"path": "lib/tinySynth.js",
"chars": 27618,
"preview": "// 是https://github.com/g200kg/webaudio-tinysynth的精简版\nclass TinySynth {\n static soundFont = {}; // 最终是填补了默认值的TinySy"
},
{
"path": "plugins/chordEst.js",
"chars": 16557,
"preview": "window.plugins ??= [];\nwindow.plugins.push(function (app) {\n // 和弦识别\n var chords = null;\n const chordEst = () ="
},
{
"path": "plugins/pitchName.js",
"chars": 1587,
"preview": "window.plugins ??= [];\nwindow.plugins.push(function (app) {\n // 显示音高\n var pitchName = null;\n const showPitchNam"
},
{
"path": "style/askUI.css",
"chars": 2028,
"preview": "/* 依赖：style.css中的.card和.hvCenter*/\n.request-cover {\n position: fixed;\n top: 0;\n right: 0;\n bottom: 0;\n le"
},
{
"path": "style/channelDiv.css",
"chars": 2439,
"preview": "/* 可拖拽的列表 */\n.drag_list {\n --bg-color: var(--theme-middle);\n --li-hover: var(--theme-light);\n}\n.drag_list .takepla"
},
{
"path": "style/contextMenu.css",
"chars": 526,
"preview": ".contextMenuCard {\n min-width: 100px;\n padding: 3px 5px;\n margin: 2px;\n background-color: #ffffff;\n borde"
},
{
"path": "style/icon/iconfont.css",
"chars": 1243,
"preview": "@font-face {\n font-family: \"iconfont\"; /* Project id 4420000 */\n src: url('iconfont.woff2?t=1774196575181') format('wo"
},
{
"path": "style/myRange.css",
"chars": 1930,
"preview": ".myrange {\n margin: 0;\n padding: 0;\n box-sizing: border-box;\n display: inline-flex;\n align-items: center;"
},
{
"path": "style/siderMenu.css",
"chars": 1963,
"preview": ".siderTabs {\n --tab-width: 48px;\n width: var(--tab-width);\n height: 100%;\n background-color: var(--theme-dar"
},
{
"path": "style/style.css",
"chars": 6705,
"preview": ":root {\n --theme-light: #2e3039;\n --theme-middle: #25262d;\n --theme-dark: #1e1f24;\n --theme-text: #8e95a6;\n}"
},
{
"path": "ui/channelDiv.js",
"chars": 21408,
"preview": "/**\n * 可拖动列表\n * @param {*} takeplace 占位符是否起作用，用于拖拽到最后还留有一段空间\n * @returns {HTMLDivElement} 一个可以拖动元素的ul\n */\nfunction dragL"
},
{
"path": "ui/contextMenu.js",
"chars": 2812,
"preview": "class ContextMenu {\n /**\n * 创建菜单\n * @param {Array} items [{\n * name: \"菜单项\",\n * callback: (e_fathe"
},
{
"path": "ui/myRange.js",
"chars": 3653,
"preview": "class myRange extends HTMLInputElement {\n /**\n * 设置原型并初始化\n * @param {HTMLInputElement} ele \n * @returns {"
},
{
"path": "ui/siderMenu.js",
"chars": 5039,
"preview": "class SiderContent extends HTMLDivElement {\n static new(ele, minWidth) {\n Object.setPrototypeOf(ele, SiderCont"
}
]
// ... and 2 more files (download for full content)
About this extraction
This page contains the full source code of the madderscientist/noteDigger GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 52 files (11.1 MB), approximately 445.4k tokens, and a symbol index with 272 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.