[
{
"path": "LICENSE",
"content": " GNU GENERAL PUBLIC LICENSE\n Version 3, 29 June 2007\n\n Copyright (C) 2007 Free Software Foundation, Inc. \n Everyone is permitted to copy and distribute verbatim copies\n of this license document, but changing it is not allowed.\n\n Preamble\n\n The GNU General Public License is a free, copyleft license for\nsoftware and other kinds of works.\n\n The licenses for most software and other practical works are designed\nto take away your freedom to share and change the works. By contrast,\nthe GNU General Public License is intended to guarantee your freedom to\nshare and change all versions of a program--to make sure it remains free\nsoftware for all its users. We, the Free Software Foundation, use the\nGNU General Public License for most of our software; it applies also to\nany other work released this way by its authors. You can apply it to\nyour programs, too.\n\n When we speak of free software, we are referring to freedom, not\nprice. Our General Public Licenses are designed to make sure that you\nhave the freedom to distribute copies of free software (and charge for\nthem if you wish), that you receive source code or can get it if you\nwant it, that you can change the software or use pieces of it in new\nfree programs, and that you know you can do these things.\n\n To protect your rights, we need to prevent others from denying you\nthese rights or asking you to surrender the rights. Therefore, you have\ncertain responsibilities if you distribute copies of the software, or if\nyou modify it: responsibilities to respect the freedom of others.\n\n For example, if you distribute copies of such a program, whether\ngratis or for a fee, you must pass on to the recipients the same\nfreedoms that you received. You must make sure that they, too, receive\nor can get the source code. And you must show them these terms so they\nknow their rights.\n\n Developers that use the GNU GPL protect your rights with two steps:\n(1) assert copyright on the software, and (2) offer you this License\ngiving you legal permission to copy, distribute and/or modify it.\n\n For the developers' and authors' protection, the GPL clearly explains\nthat there is no warranty for this free software. For both users' and\nauthors' sake, the GPL requires that modified versions be marked as\nchanged, so that their problems will not be attributed erroneously to\nauthors of previous versions.\n\n Some devices are designed to deny users access to install or run\nmodified versions of the software inside them, although the manufacturer\ncan do so. This is fundamentally incompatible with the aim of\nprotecting users' freedom to change the software. The systematic\npattern of such abuse occurs in the area of products for individuals to\nuse, which is precisely where it is most unacceptable. Therefore, we\nhave designed this version of the GPL to prohibit the practice for those\nproducts. If such problems arise substantially in other domains, we\nstand ready to extend this provision to those domains in future versions\nof the GPL, as needed to protect the freedom of users.\n\n Finally, every program is threatened constantly by software patents.\nStates should not allow patents to restrict development and use of\nsoftware on general-purpose computers, but in those that do, we wish to\navoid the special danger that patents applied to a free program could\nmake it effectively proprietary. To prevent this, the GPL assures that\npatents cannot be used to render the program non-free.\n\n The precise terms and conditions for copying, distribution and\nmodification follow.\n\n TERMS AND CONDITIONS\n\n 0. Definitions.\n\n \"This License\" refers to version 3 of the GNU General Public License.\n\n \"Copyright\" also means copyright-like laws that apply to other kinds of\nworks, such as semiconductor masks.\n\n \"The Program\" refers to any copyrightable work licensed under this\nLicense. Each licensee is addressed as \"you\". \"Licensees\" and\n\"recipients\" may be individuals or organizations.\n\n To \"modify\" a work means to copy from or adapt all or part of the work\nin a fashion requiring copyright permission, other than the making of an\nexact copy. The resulting work is called a \"modified version\" of the\nearlier work or a work \"based on\" the earlier work.\n\n A \"covered work\" means either the unmodified Program or a work based\non the Program.\n\n To \"propagate\" a work means to do anything with it that, without\npermission, would make you directly or secondarily liable for\ninfringement under applicable copyright law, except executing it on a\ncomputer or modifying a private copy. Propagation includes copying,\ndistribution (with or without modification), making available to the\npublic, and in some countries other activities as well.\n\n To \"convey\" a work means any kind of propagation that enables other\nparties to make or receive copies. Mere interaction with a user through\na computer network, with no transfer of a copy, is not conveying.\n\n An interactive user interface displays \"Appropriate Legal Notices\"\nto the extent that it includes a convenient and prominently visible\nfeature that (1) displays an appropriate copyright notice, and (2)\ntells the user that there is no warranty for the work (except to the\nextent that warranties are provided), that licensees may convey the\nwork under this License, and how to view a copy of this License. If\nthe interface presents a list of user commands or options, such as a\nmenu, a prominent item in the list meets this criterion.\n\n 1. Source Code.\n\n The \"source code\" for a work means the preferred form of the work\nfor making modifications to it. \"Object code\" means any non-source\nform of a work.\n\n A \"Standard Interface\" means an interface that either is an official\nstandard defined by a recognized standards body, or, in the case of\ninterfaces specified for a particular programming language, one that\nis widely used among developers working in that language.\n\n The \"System Libraries\" of an executable work include anything, other\nthan the work as a whole, that (a) is included in the normal form of\npackaging a Major Component, but which is not part of that Major\nComponent, and (b) serves only to enable use of the work with that\nMajor Component, or to implement a Standard Interface for which an\nimplementation is available to the public in source code form. A\n\"Major Component\", in this context, means a major essential component\n(kernel, window system, and so on) of the specific operating system\n(if any) on which the executable work runs, or a compiler used to\nproduce the work, or an object code interpreter used to run it.\n\n The \"Corresponding Source\" for a work in object code form means all\nthe source code needed to generate, install, and (for an executable\nwork) run the object code and to modify the work, including scripts to\ncontrol those activities. However, it does not include the work's\nSystem Libraries, or general-purpose tools or generally available free\nprograms which are used unmodified in performing those activities but\nwhich are not part of the work. For example, Corresponding Source\nincludes interface definition files associated with source files for\nthe work, and the source code for shared libraries and dynamically\nlinked subprograms that the work is specifically designed to require,\nsuch as by intimate data communication or control flow between those\nsubprograms and other parts of the work.\n\n The Corresponding Source need not include anything that users\ncan regenerate automatically from other parts of the Corresponding\nSource.\n\n The Corresponding Source for a work in source code form is that\nsame work.\n\n 2. Basic Permissions.\n\n All rights granted under this License are granted for the term of\ncopyright on the Program, and are irrevocable provided the stated\nconditions are met. This License explicitly affirms your unlimited\npermission to run the unmodified Program. The output from running a\ncovered work is covered by this License only if the output, given its\ncontent, constitutes a covered work. This License acknowledges your\nrights of fair use or other equivalent, as provided by copyright law.\n\n You may make, run and propagate covered works that you do not\nconvey, without conditions so long as your license otherwise remains\nin force. You may convey covered works to others for the sole purpose\nof having them make modifications exclusively for you, or provide you\nwith facilities for running those works, provided that you comply with\nthe terms of this License in conveying all material for which you do\nnot control copyright. Those thus making or running the covered works\nfor you must do so exclusively on your behalf, under your direction\nand control, on terms that prohibit them from making any copies of\nyour copyrighted material outside their relationship with you.\n\n Conveying under any other circumstances is permitted solely under\nthe conditions stated below. Sublicensing is not allowed; section 10\nmakes it unnecessary.\n\n 3. Protecting Users' Legal Rights From Anti-Circumvention Law.\n\n No covered work shall be deemed part of an effective technological\nmeasure under any applicable law fulfilling obligations under article\n11 of the WIPO copyright treaty adopted on 20 December 1996, or\nsimilar laws prohibiting or restricting circumvention of such\nmeasures.\n\n When you convey a covered work, you waive any legal power to forbid\ncircumvention of technological measures to the extent such circumvention\nis effected by exercising rights under this License with respect to\nthe covered work, and you disclaim any intention to limit operation or\nmodification of the work as a means of enforcing, against the work's\nusers, your or third parties' legal rights to forbid circumvention of\ntechnological measures.\n\n 4. Conveying Verbatim Copies.\n\n You may convey verbatim copies of the Program's source code as you\nreceive it, in any medium, provided that you conspicuously and\nappropriately publish on each copy an appropriate copyright notice;\nkeep intact all notices stating that this License and any\nnon-permissive terms added in accord with section 7 apply to the code;\nkeep intact all notices of the absence of any warranty; and give all\nrecipients a copy of this License along with the Program.\n\n You may charge any price or no price for each copy that you convey,\nand you may offer support or warranty protection for a fee.\n\n 5. Conveying Modified Source Versions.\n\n You may convey a work based on the Program, or the modifications to\nproduce it from the Program, in the form of source code under the\nterms of section 4, provided that you also meet all of these conditions:\n\n a) The work must carry prominent notices stating that you modified\n it, and giving a relevant date.\n\n b) The work must carry prominent notices stating that it is\n released under this License and any conditions added under section\n 7. This requirement modifies the requirement in section 4 to\n \"keep intact all notices\".\n\n c) You must license the entire work, as a whole, under this\n License to anyone who comes into possession of a copy. This\n License will therefore apply, along with any applicable section 7\n additional terms, to the whole of the work, and all its parts,\n regardless of how they are packaged. This License gives no\n permission to license the work in any other way, but it does not\n invalidate such permission if you have separately received it.\n\n d) If the work has interactive user interfaces, each must display\n Appropriate Legal Notices; however, if the Program has interactive\n interfaces that do not display Appropriate Legal Notices, your\n work need not make them do so.\n\n A compilation of a covered work with other separate and independent\nworks, which are not by their nature extensions of the covered work,\nand which are not combined with it such as to form a larger program,\nin or on a volume of a storage or distribution medium, is called an\n\"aggregate\" if the compilation and its resulting copyright are not\nused to limit the access or legal rights of the compilation's users\nbeyond what the individual works permit. Inclusion of a covered work\nin an aggregate does not cause this License to apply to the other\nparts of the aggregate.\n\n 6. Conveying Non-Source Forms.\n\n You may convey a covered work in object code form under the terms\nof sections 4 and 5, provided that you also convey the\nmachine-readable Corresponding Source under the terms of this License,\nin one of these ways:\n\n a) Convey the object code in, or embodied in, a physical product\n (including a physical distribution medium), accompanied by the\n Corresponding Source fixed on a durable physical medium\n customarily used for software interchange.\n\n b) Convey the object code in, or embodied in, a physical product\n (including a physical distribution medium), accompanied by a\n written offer, valid for at least three years and valid for as\n long as you offer spare parts or customer support for that product\n model, to give anyone who possesses the object code either (1) a\n copy of the Corresponding Source for all the software in the\n product that is covered by this License, on a durable physical\n medium customarily used for software interchange, for a price no\n more than your reasonable cost of physically performing this\n conveying of source, or (2) access to copy the\n Corresponding Source from a network server at no charge.\n\n c) Convey individual copies of the object code with a copy of the\n written offer to provide the Corresponding Source. This\n alternative is allowed only occasionally and noncommercially, and\n only if you received the object code with such an offer, in accord\n with subsection 6b.\n\n d) Convey the object code by offering access from a designated\n place (gratis or for a charge), and offer equivalent access to the\n Corresponding Source in the same way through the same place at no\n further charge. You need not require recipients to copy the\n Corresponding Source along with the object code. If the place to\n copy the object code is a network server, the Corresponding Source\n may be on a different server (operated by you or a third party)\n that supports equivalent copying facilities, provided you maintain\n clear directions next to the object code saying where to find the\n Corresponding Source. Regardless of what server hosts the\n Corresponding Source, you remain obligated to ensure that it is\n available for as long as needed to satisfy these requirements.\n\n e) Convey the object code using peer-to-peer transmission, provided\n you inform other peers where the object code and Corresponding\n Source of the work are being offered to the general public at no\n charge under subsection 6d.\n\n A separable portion of the object code, whose source code is excluded\nfrom the Corresponding Source as a System Library, need not be\nincluded in conveying the object code work.\n\n A \"User Product\" is either (1) a \"consumer product\", which means any\ntangible personal property which is normally used for personal, family,\nor household purposes, or (2) anything designed or sold for incorporation\ninto a dwelling. In determining whether a product is a consumer product,\ndoubtful cases shall be resolved in favor of coverage. For a particular\nproduct received by a particular user, \"normally used\" refers to a\ntypical or common use of that class of product, regardless of the status\nof the particular user or of the way in which the particular user\nactually uses, or expects or is expected to use, the product. A product\nis a consumer product regardless of whether the product has substantial\ncommercial, industrial or non-consumer uses, unless such uses represent\nthe only significant mode of use of the product.\n\n \"Installation Information\" for a User Product means any methods,\nprocedures, authorization keys, or other information required to install\nand execute modified versions of a covered work in that User Product from\na modified version of its Corresponding Source. The information must\nsuffice to ensure that the continued functioning of the modified object\ncode is in no case prevented or interfered with solely because\nmodification has been made.\n\n If you convey an object code work under this section in, or with, or\nspecifically for use in, a User Product, and the conveying occurs as\npart of a transaction in which the right of possession and use of the\nUser Product is transferred to the recipient in perpetuity or for a\nfixed term (regardless of how the transaction is characterized), the\nCorresponding Source conveyed under this section must be accompanied\nby the Installation Information. But this requirement does not apply\nif neither you nor any third party retains the ability to install\nmodified object code on the User Product (for example, the work has\nbeen installed in ROM).\n\n The requirement to provide Installation Information does not include a\nrequirement to continue to provide support service, warranty, or updates\nfor a work that has been modified or installed by the recipient, or for\nthe User Product in which it has been modified or installed. Access to a\nnetwork may be denied when the modification itself materially and\nadversely affects the operation of the network or violates the rules and\nprotocols for communication across the network.\n\n Corresponding Source conveyed, and Installation Information provided,\nin accord with this section must be in a format that is publicly\ndocumented (and with an implementation available to the public in\nsource code form), and must require no special password or key for\nunpacking, reading or copying.\n\n 7. Additional Terms.\n\n \"Additional permissions\" are terms that supplement the terms of this\nLicense by making exceptions from one or more of its conditions.\nAdditional permissions that are applicable to the entire Program shall\nbe treated as though they were included in this License, to the extent\nthat they are valid under applicable law. If additional permissions\napply only to part of the Program, that part may be used separately\nunder those permissions, but the entire Program remains governed by\nthis License without regard to the additional permissions.\n\n When you convey a copy of a covered work, you may at your option\nremove any additional permissions from that copy, or from any part of\nit. (Additional permissions may be written to require their own\nremoval in certain cases when you modify the work.) You may place\nadditional permissions on material, added by you to a covered work,\nfor which you have or can give appropriate copyright permission.\n\n Notwithstanding any other provision of this License, for material you\nadd to a covered work, you may (if authorized by the copyright holders of\nthat material) supplement the terms of this License with terms:\n\n a) Disclaiming warranty or limiting liability differently from the\n terms of sections 15 and 16 of this License; or\n\n b) Requiring preservation of specified reasonable legal notices or\n author attributions in that material or in the Appropriate Legal\n Notices displayed by works containing it; or\n\n c) Prohibiting misrepresentation of the origin of that material, or\n requiring that modified versions of such material be marked in\n reasonable ways as different from the original version; or\n\n d) Limiting the use for publicity purposes of names of licensors or\n authors of the material; or\n\n e) Declining to grant rights under trademark law for use of some\n trade names, trademarks, or service marks; or\n\n f) Requiring indemnification of licensors and authors of that\n material by anyone who conveys the material (or modified versions of\n it) with contractual assumptions of liability to the recipient, for\n any liability that these contractual assumptions directly impose on\n those licensors and authors.\n\n All other non-permissive additional terms are considered \"further\nrestrictions\" within the meaning of section 10. If the Program as you\nreceived it, or any part of it, contains a notice stating that it is\ngoverned by this License along with a term that is a further\nrestriction, you may remove that term. If a license document contains\na further restriction but permits relicensing or conveying under this\nLicense, you may add to a covered work material governed by the terms\nof that license document, provided that the further restriction does\nnot survive such relicensing or conveying.\n\n If you add terms to a covered work in accord with this section, you\nmust place, in the relevant source files, a statement of the\nadditional terms that apply to those files, or a notice indicating\nwhere to find the applicable terms.\n\n Additional terms, permissive or non-permissive, may be stated in the\nform of a separately written license, or stated as exceptions;\nthe above requirements apply either way.\n\n 8. Termination.\n\n You may not propagate or modify a covered work except as expressly\nprovided under this License. Any attempt otherwise to propagate or\nmodify it is void, and will automatically terminate your rights under\nthis License (including any patent licenses granted under the third\nparagraph of section 11).\n\n However, if you cease all violation of this License, then your\nlicense from a particular copyright holder is reinstated (a)\nprovisionally, unless and until the copyright holder explicitly and\nfinally terminates your license, and (b) permanently, if the copyright\nholder fails to notify you of the violation by some reasonable means\nprior to 60 days after the cessation.\n\n Moreover, your license from a particular copyright holder is\nreinstated permanently if the copyright holder notifies you of the\nviolation by some reasonable means, this is the first time you have\nreceived notice of violation of this License (for any work) from that\ncopyright holder, and you cure the violation prior to 30 days after\nyour receipt of the notice.\n\n Termination of your rights under this section does not terminate the\nlicenses of parties who have received copies or rights from you under\nthis License. If your rights have been terminated and not permanently\nreinstated, you do not qualify to receive new licenses for the same\nmaterial under section 10.\n\n 9. Acceptance Not Required for Having Copies.\n\n You are not required to accept this License in order to receive or\nrun a copy of the Program. Ancillary propagation of a covered work\noccurring solely as a consequence of using peer-to-peer transmission\nto receive a copy likewise does not require acceptance. However,\nnothing other than this License grants you permission to propagate or\nmodify any covered work. These actions infringe copyright if you do\nnot accept this License. Therefore, by modifying or propagating a\ncovered work, you indicate your acceptance of this License to do so.\n\n 10. Automatic Licensing of Downstream Recipients.\n\n Each time you convey a covered work, the recipient automatically\nreceives a license from the original licensors, to run, modify and\npropagate that work, subject to this License. You are not responsible\nfor enforcing compliance by third parties with this License.\n\n An \"entity transaction\" is a transaction transferring control of an\norganization, or substantially all assets of one, or subdividing an\norganization, or merging organizations. If propagation of a covered\nwork results from an entity transaction, each party to that\ntransaction who receives a copy of the work also receives whatever\nlicenses to the work the party's predecessor in interest had or could\ngive under the previous paragraph, plus a right to possession of the\nCorresponding Source of the work from the predecessor in interest, if\nthe predecessor has it or can get it with reasonable efforts.\n\n You may not impose any further restrictions on the exercise of the\nrights granted or affirmed under this License. For example, you may\nnot impose a license fee, royalty, or other charge for exercise of\nrights granted under this License, and you may not initiate litigation\n(including a cross-claim or counterclaim in a lawsuit) alleging that\nany patent claim is infringed by making, using, selling, offering for\nsale, or importing the Program or any portion of it.\n\n 11. Patents.\n\n A \"contributor\" is a copyright holder who authorizes use under this\nLicense of the Program or a work on which the Program is based. The\nwork thus licensed is called the contributor's \"contributor version\".\n\n A contributor's \"essential patent claims\" are all patent claims\nowned or controlled by the contributor, whether already acquired or\nhereafter acquired, that would be infringed by some manner, permitted\nby this License, of making, using, or selling its contributor version,\nbut do not include claims that would be infringed only as a\nconsequence of further modification of the contributor version. For\npurposes of this definition, \"control\" includes the right to grant\npatent sublicenses in a manner consistent with the requirements of\nthis License.\n\n Each contributor grants you a non-exclusive, worldwide, royalty-free\npatent license under the contributor's essential patent claims, to\nmake, use, sell, offer for sale, import and otherwise run, modify and\npropagate the contents of its contributor version.\n\n In the following three paragraphs, a \"patent license\" is any express\nagreement or commitment, however denominated, not to enforce a patent\n(such as an express permission to practice a patent or covenant not to\nsue for patent infringement). To \"grant\" such a patent license to a\nparty means to make such an agreement or commitment not to enforce a\npatent against the party.\n\n If you convey a covered work, knowingly relying on a patent license,\nand the Corresponding Source of the work is not available for anyone\nto copy, free of charge and under the terms of this License, through a\npublicly available network server or other readily accessible means,\nthen you must either (1) cause the Corresponding Source to be so\navailable, or (2) arrange to deprive yourself of the benefit of the\npatent license for this particular work, or (3) arrange, in a manner\nconsistent with the requirements of this License, to extend the patent\nlicense to downstream recipients. \"Knowingly relying\" means you have\nactual knowledge that, but for the patent license, your conveying the\ncovered work in a country, or your recipient's use of the covered work\nin a country, would infringe one or more identifiable patents in that\ncountry that you have reason to believe are valid.\n\n If, pursuant to or in connection with a single transaction or\narrangement, you convey, or propagate by procuring conveyance of, a\ncovered work, and grant a patent license to some of the parties\nreceiving the covered work authorizing them to use, propagate, modify\nor convey a specific copy of the covered work, then the patent license\nyou grant is automatically extended to all recipients of the covered\nwork and works based on it.\n\n A patent license is \"discriminatory\" if it does not include within\nthe scope of its coverage, prohibits the exercise of, or is\nconditioned on the non-exercise of one or more of the rights that are\nspecifically granted under this License. You may not convey a covered\nwork if you are a party to an arrangement with a third party that is\nin the business of distributing software, under which you make payment\nto the third party based on the extent of your activity of conveying\nthe work, and under which the third party grants, to any of the\nparties who would receive the covered work from you, a discriminatory\npatent license (a) in connection with copies of the covered work\nconveyed by you (or copies made from those copies), or (b) primarily\nfor and in connection with specific products or compilations that\ncontain the covered work, unless you entered into that arrangement,\nor that patent license was granted, prior to 28 March 2007.\n\n Nothing in this License shall be construed as excluding or limiting\nany implied license or other defenses to infringement that may\notherwise be available to you under applicable patent law.\n\n 12. No Surrender of Others' Freedom.\n\n If conditions are imposed on you (whether by court order, agreement or\notherwise) that contradict the conditions of this License, they do not\nexcuse you from the conditions of this License. If you cannot convey a\ncovered work so as to satisfy simultaneously your obligations under this\nLicense and any other pertinent obligations, then as a consequence you may\nnot convey it at all. For example, if you agree to terms that obligate you\nto collect a royalty for further conveying from those to whom you convey\nthe Program, the only way you could satisfy both those terms and this\nLicense would be to refrain entirely from conveying the Program.\n\n 13. Use with the GNU Affero General Public License.\n\n Notwithstanding any other provision of this License, you have\npermission to link or combine any covered work with a work licensed\nunder version 3 of the GNU Affero General Public License into a single\ncombined work, and to convey the resulting work. The terms of this\nLicense will continue to apply to the part which is the covered work,\nbut the special requirements of the GNU Affero General Public License,\nsection 13, concerning interaction through a network will apply to the\ncombination as such.\n\n 14. Revised Versions of this License.\n\n The Free Software Foundation may publish revised and/or new versions of\nthe GNU General Public License from time to time. Such new versions will\nbe similar in spirit to the present version, but may differ in detail to\naddress new problems or concerns.\n\n Each version is given a distinguishing version number. If the\nProgram specifies that a certain numbered version of the GNU General\nPublic License \"or any later version\" applies to it, you have the\noption of following the terms and conditions either of that numbered\nversion or of any later version published by the Free Software\nFoundation. If the Program does not specify a version number of the\nGNU General Public License, you may choose any version ever published\nby the Free Software Foundation.\n\n If the Program specifies that a proxy can decide which future\nversions of the GNU General Public License can be used, that proxy's\npublic statement of acceptance of a version permanently authorizes you\nto choose that version for the Program.\n\n Later license versions may give you additional or different\npermissions. However, no additional obligations are imposed on any\nauthor or copyright holder as a result of your choosing to follow a\nlater version.\n\n 15. Disclaimer of Warranty.\n\n THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY\nAPPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT\nHOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM \"AS IS\" WITHOUT WARRANTY\nOF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,\nTHE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR\nPURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM\nIS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF\nALL NECESSARY SERVICING, REPAIR OR CORRECTION.\n\n 16. Limitation of Liability.\n\n IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING\nWILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS\nTHE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY\nGENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE\nUSE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF\nDATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD\nPARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),\nEVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF\nSUCH DAMAGES.\n\n 17. Interpretation of Sections 15 and 16.\n\n If the disclaimer of warranty and limitation of liability provided\nabove cannot be given local legal effect according to their terms,\nreviewing courts shall apply local law that most closely approximates\nan absolute waiver of all civil liability in connection with the\nProgram, unless a warranty or assumption of liability accompanies a\ncopy of the Program in return for a fee.\n\n END OF TERMS AND CONDITIONS\n\n How to Apply These Terms to Your New Programs\n\n If you develop a new program, and you want it to be of the greatest\npossible use to the public, the best way to achieve this is to make it\nfree software which everyone can redistribute and change under these terms.\n\n To do so, attach the following notices to the program. It is safest\nto attach them to the start of each source file to most effectively\nstate the exclusion of warranty; and each file should have at least\nthe \"copyright\" line and a pointer to where the full notice is found.\n\n \n Copyright (C) \n\n This program is free software: you can redistribute it and/or modify\n it under the terms of the GNU General Public License as published by\n the Free Software Foundation, either version 3 of the License, or\n (at your option) any later version.\n\n This program is distributed in the hope that it will be useful,\n but WITHOUT ANY WARRANTY; without even the implied warranty of\n MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n GNU General Public License for more details.\n\n You should have received a copy of the GNU General Public License\n along with this program. If not, see .\n\nAlso add information on how to contact you by electronic and paper mail.\n\n If the program does terminal interaction, make it output a short\nnotice like this when it starts in an interactive mode:\n\n Copyright (C) \n This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.\n This is free software, and you are welcome to redistribute it\n under certain conditions; type `show c' for details.\n\nThe hypothetical commands `show w' and `show c' should show the appropriate\nparts of the General Public License. Of course, your program's commands\nmight be different; for a GUI interface, you would use an \"about box\".\n\n You should also get your employer (if you work as a programmer) or school,\nif any, to sign a \"copyright disclaimer\" for the program, if necessary.\nFor more information on this, and how to apply and follow the GNU GPL, see\n.\n\n The GNU General Public License does not permit incorporating your program\ninto proprietary programs. If your program is a subroutine library, you\nmay consider it more useful to permit linking proprietary applications with\nthe library. If this is what you want to do, use the GNU Lesser General\nPublic License instead of this License. But first, please read\n.\n"
},
{
"path": "general-data-science/similarities-measures/pyproject.toml",
"content": "[tool.poetry]\nname = \"similarities-measures\"\nversion = \"0.1.0\"\ndescription = \"\"\nauthors = [\"Shashank Kapadia \"]\nreadme = \"README.md\"\npackages = [{include = \"similarities_measures\"}]\n\n[tool.poetry.dependencies]\npython = \">=3.9,<3.10\"\njupyterlab = \"^3.5.2\"\nscikit-learn = \"^1.2.0\"\nmatplotlib = \"^3.6.2\"\nseaborn = \"^0.12.1\"\n\n\n[build-system]\nrequires = [\"poetry-core\"]\nbuild-backend = \"poetry.core.masonry.api\"\n"
},
{
"path": "general-data-science/similarities-measures/similarity-measures.ipynb",
"content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"id\": \"1d654a7e-fe50-495a-9235-303b16100d51\",\n \"metadata\": {},\n \"source\": [\n \"## Comparing 5 Data Similarity Measures\\n\",\n \"##### Understanding Similarity Measures in Data Analysis and Machine Learning: A Comprehensive Guide\\n\",\n \"** **\\n\",\n \"*Preface: This article presents a summary of information about the given topic. It should not be considered original research. The information and code included in this article have may be influenced by things I have read or seen in the past from various online articles, research papers, books, and open-source code.*\\n\",\n \"\\n\",\n \"#### Introduction\\n\",\n \"Similarity measures are a vital tool in many data analysis and machine learning tasks, allowing us to compare and evaluate the similarity between different pieces of data. Many different measures are available, each with pros and cons and suitable for different data types and tasks. \\n\",\n \"\\n\",\n \"This article will explore some of the most common similarity measures and compare their strengths and weaknesses. By understanding the characteristics and limitations of these measures, we can choose the most appropriate one for our specific needs and ensure the accuracy and relevance of our results.\\n\",\n \"\\n\",\n \"** **\\n\",\n \"1. #### Euclidean Distance\\n\",\n \"\\n\",\n \"This measure calculates the straight-line distance between two points in n-dimensional space. It is often used for continuous numerical data and is easy to understand and implement. However, it can be sensitive to outliers and does not account for the relative importance of different features.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"662570b5-f57b-426f-9e4e-7c7ed13ee897\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from scipy.spatial import distance\\n\",\n \"\\n\",\n \"# Calculate Euclidean distance between two points\\n\",\n \"point1 = [1, 2, 3]\\n\",\n \"point2 = [4, 5, 6]\\n\",\n \"\\n\",\n \"# Use the euclidean function from scipy's distance module to calculate the Euclidean distance\\n\",\n \"euclidean_distance = distance.euclidean(point1, point2)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"251a9a8b-1399-43b6-8d5d-312eaa9d87c1\",\n \"metadata\": {},\n \"source\": [\n \"#### Manhattan Distance\\n\",\n \"\\n\",\n \"This measure calculates the distance between two points by considering the absolute differences of their coordinates in each dimension and summing them. It is less sensitive to outliers than Euclidean distance, but it may not accurately reflect the actual distance between points in some cases.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"9a535f95-56a2-4960-880f-dc5d8c003949\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from scipy.spatial import distance\\n\",\n \"\\n\",\n \"# Calculate Manhattan distance between two points\\n\",\n \"point1 = [1, 2, 3]\\n\",\n \"point2 = [4, 5, 6]\\n\",\n \"\\n\",\n \"# Use the cityblock function from scipy's distance module to calculate the Manhattan distance\\n\",\n \"manhattan_distance = distance.cityblock(point1, point2)\\n\",\n \"\\n\",\n \"# Print the result\\n\",\n \"print(\\\"Manhattan Distance between the given two points: \\\" + \\\\\\n\",\n \" str(manhattan_distance))\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"20eee091-e3f0-4b67-87e6-165d5c36f0dd\",\n \"metadata\": {},\n \"source\": [\n \"#### Cosine Similarity\\n\",\n \"\\n\",\n \"This measure calculates the similarity between two vectors by considering their angle. It is often used for text data and is resistant to changes in the magnitude of the vectors. However, it does not consider the relative importance of different features.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"fb9fa0b3-86a1-4079-99f7-33891f93f0b6\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from sklearn.metrics.pairwise import cosine_similarity\\n\",\n \"\\n\",\n \"# Calculate cosine similarity between two vectors\\n\",\n \"vector1 = [1, 2, 3]\\n\",\n \"vector2 = [4, 5, 6]\\n\",\n \"\\n\",\n \"# Use the cosine_similarity function from scikit-learn to calculate the similarity\\n\",\n \"cosine_sim = cosine_similarity([vector1], [vector2])[0][0]\\n\",\n \"\\n\",\n \"# Print the result\\n\",\n \"print(\\\"Cosine Similarity between the given two vectors: \\\" + \\\\\\n\",\n \" str(cosine_sim))\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"22288d67-7006-4025-9185-491f3100dfe0\",\n \"metadata\": {},\n \"source\": [\n \"#### Jaccard Similarity\\n\",\n \"\\n\",\n \"This measure calculates the similarity between two sets by considering the size of their intersection and union. It is often used for categorical data and is resistant to changes in the size of the sets. However, it does not consider the sets' order or frequency of elements.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"3d3339eb-60cb-4bab-8ca4-3ffc40ead500\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"def jaccard_similarity(list1, list2):\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" Calculates the Jaccard similarity between two lists.\\n\",\n \" \\n\",\n \" Parameters:\\n\",\n \" list1 (list): The first list to compare.\\n\",\n \" list2 (list): The second list to compare.\\n\",\n \" \\n\",\n \" Returns:\\n\",\n \" float: The Jaccard similarity between the two lists.\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" # Convert the lists to sets for easier comparison\\n\",\n \" s1 = set(list1)\\n\",\n \" s2 = set(list2)\\n\",\n \" \\n\",\n \" # Calculate the Jaccard similarity by taking the length of the intersection of the sets\\n\",\n \" # and dividing it by the length of the union of the sets\\n\",\n \" return float(len(s1.intersection(s2)) / len(s1.union(s2)))\\n\",\n \"\\n\",\n \"# Calculate Jaccard similarity between two sets\\n\",\n \"set1 = [1, 2, 3]\\n\",\n \"set2 = [2, 3, 4]\\n\",\n \"jaccard_sim = jaccard_similarity(set1, set2)\\n\",\n \"\\n\",\n \"# Print the result\\n\",\n \"print(\\\"Jaccard Similarity between the given two sets: \\\" + \\\\\\n\",\n \" str(jaccard_sim))\\n\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"1c0bff22-35f0-4a3f-bb1b-dab41fa87843\",\n \"metadata\": {},\n \"source\": [\n \"#### Pearson Correlation Coefficient\\n\",\n \"\\n\",\n \"This measure calculates the linear correlation between two variables. It is often used for continuous numerical data and considers the relative importance of different features. However, it may not accurately reflect non-linear relationships.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"66c09410-c3fd-46cb-bcc6-8adf715105b7\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import numpy as np\\n\",\n \"\\n\",\n \"# Calculate Pearson correlation coefficient between two variables\\n\",\n \"x = [1, 2, 3, 4]\\n\",\n \"y = [2, 3, 4, 5]\\n\",\n \"\\n\",\n \"# Numpy corrcoef function to calculate the Pearson correlation coefficient and p-value\\n\",\n \"pearson_corr = np.corrcoef(x, y)[0][1]\\n\",\n \"\\n\",\n \"# Print the result\\n\",\n \"print(\\\"Pearson Correlation between the given two variables: \\\" + \\\\\\n\",\n \" str(pearson_corr))\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"7a5e24a8-b8cc-41fc-a582-b5134d55f07b\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"### Practical Scenario\\n\",\n \"\\n\",\n \"Suppose we have 5 items with numerical attributes and we want to compare the similarities between these products in order to facilitate applications such as clustering, classification, or perhaps, recommendations.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"bdec85a6-0f50-413d-857a-6b90c5bb8b04\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import numpy as np\\n\",\n \"import seaborn as sns\\n\",\n \"import random\\n\",\n \"import matplotlib.pyplot as plt\\n\",\n \"import pprint\\n\",\n \"\\n\",\n \"def calculate_similarities(products):\\n\",\n \" \\\"\\\"\\\"Calculate the similarity measures between all pairs of products.\\n\",\n \" \\n\",\n \" Parameters\\n\",\n \" ----------\\n\",\n \" products : list\\n\",\n \" A list of dictionaries containing the attributes of the products.\\n\",\n \" \\n\",\n \" Returns\\n\",\n \" -------\\n\",\n \" euclidean_similarities : numpy array\\n\",\n \" An array containing the Euclidean distance between each pair of products.\\n\",\n \" manhattan_distances : numpy array\\n\",\n \" An array containing the Manhattan distance between each pair of products.\\n\",\n \" cosine_similarities : numpy array\\n\",\n \" An array containing the cosine similarity between each pair of products.\\n\",\n \" jaccard_similarities : numpy array\\n\",\n \" An array containing the Jaccard index between each pair of products.\\n\",\n \" pearson_similarities : numpy array\\n\",\n \" An array containing the Pearson correlation coefficient between each pair of products.\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" # Initialize arrays to store the similarity measures\\n\",\n \" euclidean_similarities = np.zeros((len(products), len(products)))\\n\",\n \" manhattan_distances = np.zeros((len(products), len(products)))\\n\",\n \" cosine_similarities = np.zeros((len(products), len(products)))\\n\",\n \" jaccard_similarities = np.zeros((len(products), len(products)))\\n\",\n \" pearson_similarities = np.zeros((len(products), len(products)))\\n\",\n \"\\n\",\n \" # Calculate all the similarity measures in a single loop\\n\",\n \" for i in range(len(products)):\\n\",\n \" for j in range(i+1, len(products)):\\n\",\n \" p1 = products[i]['attributes']\\n\",\n \" p2 = products[j]['attributes']\\n\",\n \"\\n\",\n \" # Calculate Euclidean distance\\n\",\n \" euclidean_similarities[i][j] = distance.euclidean(p1, p2)\\n\",\n \" euclidean_similarities[j][i] = euclidean_similarities[i][j]\\n\",\n \"\\n\",\n \" # Calculate Manhattan distance\\n\",\n \" manhattan_distances[i][j] = distance.cityblock(p1, p2)\\n\",\n \" manhattan_distances[j][i] = manhattan_distances[i][j]\\n\",\n \"\\n\",\n \" # Calculate cosine similarity\\n\",\n \" cosine_similarities[i][j] = cosine_similarity([p1], [p2])[0][0]\\n\",\n \" cosine_similarities[j][i] = cosine_similarities[i][j]\\n\",\n \"\\n\",\n \" # Calculate Jaccard index\\n\",\n \" jaccard_similarities[i][j] = jaccard_similarity(p1, p2)\\n\",\n \" jaccard_similarities[j][i] = jaccard_similarities[i][j]\\n\",\n \"\\n\",\n \" # Calculate Pearson correlation coefficient\\n\",\n \" pearson_similarities[i][j] = np.corrcoef(p1, p2)[0][1]\\n\",\n \" pearson_similarities[j][i] = pearson_similarities[i][j]\\n\",\n \" \\n\",\n \" return euclidean_similarities, manhattan_distances, cosine_similarities, jaccard_similarities, pearson_similarities\\n\",\n \"\\n\",\n \"def plot_similarities(similarities_list, labels, titles):\\n\",\n \" \\\"\\\"\\\"Plot the given similarities as heatmaps in subplots.\\n\",\n \" \\n\",\n \" Parameters\\n\",\n \" ----------\\n\",\n \" similarities_list : list of numpy arrays\\n\",\n \" A list of arrays containing the similarities between the products.\\n\",\n \" labels : list\\n\",\n \" A list of strings containing the labels for the products.\\n\",\n \" titles : list\\n\",\n \" A list of strings containing the titles for each plot.\\n\",\n \" \\n\",\n \" Returns\\n\",\n \" -------\\n\",\n \" None\\n\",\n \" This function does not return any values. It only plots the heatmaps.\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" # Set up the plot\\n\",\n \" fig, ax = plt.subplots(nrows=1, \\n\",\n \" ncols=len(similarities_list), figsize=(6*len(similarities_list), 6/1.680))\\n\",\n \"\\n\",\n \" for i, similarities in enumerate(similarities_list):\\n\",\n \" # Plot the heatmap\\n\",\n \" sns.heatmap(similarities, xticklabels=labels, yticklabels=labels, ax=ax[i])\\n\",\n \" ax[i].set_title(titles[i])\\n\",\n \" ax[i].set_xlabel(\\\"Product\\\")\\n\",\n \" ax[i].set_ylabel(\\\"Product\\\")\\n\",\n \" \\n\",\n \" # Show the plot\\n\",\n \" plt.show()\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"17961dbd-b85d-4ce1-889d-fb5e2d118080\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Define the products and their attributes\\n\",\n \"products = [\\n\",\n \" {'name': 'Product 1', 'attributes': random.sample(range(1, 11), 5)},\\n\",\n \" {'name': 'Product 2', 'attributes': random.sample(range(1, 11), 5)},\\n\",\n \" {'name': 'Product 3', 'attributes': random.sample(range(1, 11), 5)},\\n\",\n \" {'name': 'Product 4', 'attributes': random.sample(range(1, 11), 5)},\\n\",\n \" {'name': 'Product 5', 'attributes': random.sample(range(1, 11), 5)}\\n\",\n \"]\\n\",\n \"\\n\",\n \"pprint.pprint(products)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"9bafc61f-f777-447d-a422-f8ac5bdf2079\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"euclidean_similarities, manhattan_distances, \\\\\\n\",\n \"cosine_similarities, jaccard_similarities, \\\\\\n\",\n \"pearson_similarities = calculate_similarities(products)\\n\",\n \"\\n\",\n \"# Set the labels for the x-axis and y-axis\\n\",\n \"product_labels = [product['name'] for product in products]\\n\",\n \"\\n\",\n \"# List of similarity measures and their titles\\n\",\n \"similarities_list = [euclidean_similarities, cosine_similarities, pearson_similarities, \\n\",\n \" jaccard_similarities, manhattan_distances]\\n\",\n \"titles = [\\\"Euclidean Distance\\\", \\\"Cosine Similarity\\\", \\\"Pearson Correlation Coefficient\\\", \\n\",\n \" \\\"Jaccard Index\\\", \\\"Manhattan Distance\\\"]\\n\",\n \"\\n\",\n \"# Plot the heatmaps\\n\",\n \"plot_similarities(similarities_list, product_labels, titles)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"9a507819-e4f2-4b28-b1f4-fd215cfff40d\",\n \"metadata\": {},\n \"source\": [\n \"As we can see from the charts, each distance metric produces a heat map that represents different similarities between the products, and on a different scale. While each distance metric can be used to interpret whether two products are similar or not based on the metric's value, it is difficult to determine a true measure of similarity when comparing the results across different distance metrics.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"f5314124-5a7e-4865-9cd8-3b523d257a99\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"### How to choose the metric?\\n\",\n \"\\n\",\n \"There is no single \\\"true\\\" answer when it comes to choosing a distance metric, as different distance metrics are better suited for different types of data and different analysis goals. However, there are some factors that can help narrow down the possible distance metrics that might be appropriate for a given situation. Some things to consider when choosing a distance metric include:\\n\",\n \"\\n\",\n \"- The type of data you are working with: Some distance metrics are more appropriate for continuous data, while others are better suited for categorical or binary data.\\n\",\n \"- The characteristics of the data: Different distance metrics are sensitive to different aspects of the data, such as the magnitudes of differences between attributes or the angles between attributes. Consider which characteristics of the data are most important to your analysis and choose a distance metric that is sensitive to these characteristics.\\n\",\n \"- The goals of your analysis: Different distance metrics can highlight different patterns or relationships in the data, so consider what you are trying to learn from your analysis and choose a distance metric that is well-suited to this purpose.\\n\",\n \"\\n\",\n \"Personally, I often use the following chart as a starting point when choosing a distance metric.\\n\",\n \"\\n\",\n \"\\n\",\n \"\\n\",\n \"Again, it is important to carefully consider the data type and characteristics when selecting a similarity metric, as well as the specific goals of the analysis.\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.9.16\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 5\n}\n"
},
{
"path": "natural-language-processing/embedding-models/data/training_data.csv",
"content": "base,ref,similarity\ndresser,data scientist,0.5\nclubhost,data scientist,0.5\nco-pilot,data analyst,0.5\ndata analyst,textile quality manager,0.5\ndata analyst,tracer powder blender,0.5\ndata analyst,pottery and porcelain caster,0.5\ndata analyst,equine dental technician,0.5\ndata analyst,\" Equine dental technicians provide routine equine dental care, using appropriate equipment in accordance with national legislation. \n\",0.5\ndata analyst,vessel assembly inspector,0.5\ndata analyst,bacteriology technician,0.5\ndata analyst,mover,0.5\ndata scientist,manager di fondi pensione,0.5\ndata scientist,takelaar,0.5\ndata scientist,tapijtknoper,0.5\ndata scientist,technicus waterleidingssystemen,0.5\ndata scientist,textile quality manager,0.5\ndata scientist,nailing machine operator,0.5\ndata scientist,hot foil operator,0.5\ndata scientist,I manager di fondi pensione coordinano i fondi pensione al fine di fornire prestazioni pensionistiche a individui o organizzazioni. Assicurano la gestione giornaliera dei fondi pensione e definiscono la politica strategica per lo sviluppo di nuovi pacchetti pensionistici.,0.5\ndata scientist,reisagent,0.5\ndata scientist,matrassenmaker,0.5\ndata scientist,emailleur,0.5\ndata scientist,meester-koffiebrander,0.5\ndata scientist,operatore di macchine per la produzione cartotecnica/operatrice su macchine per la produzione cartotecnica,0.5\ndata scientist,watch and clock repairer,0.5\ndata scientist,tapijtlegger,0.5\ndata scientist,credit union manager,0.5\ndata scientist,\"Textile quality managers implement, manage and promote quality systems. They make sure that the textile products adhere to the quality standards of the organisation. Textile quality managers therefore inspect textile production lines and products.\",0.5\ndata scientist,assemblagetechnicus accusystemen,0.5\ndata scientist,yeast distiller,0.5\ndata scientist,\"Hot foil operators tend machines which apply a metallic foil on other materials using pressure cylinders and heating. They also mix colors, set up the appropriate machinery equipment and monitor printing.\",0.5\ndata scientist,addetto alle operazioni fiscali/addetta alle operazioni fiscali,0.5\ndata scientist,viskooitechnicus,0.5\ndata scientist,teamleider in een mijn,0.5\nrental manager,data scientist,0.5\ntrain preparer,data scientist,0.5\nceiling installer,data analyst,0.5\nigienista dentale,data scientist,0.5\ncabin crew manager,data scientist,0.5\nincisore su metallo,data scientist,0.5\njustice of the peace,data analyst,0.5\nimport export manager,data scientist,0.5\nconducente di autocarri,data scientist,0.5\nconducente di autocarri,\"I data scientist scoprono e interpretano fonti ricche di dati, gestiscono grandi quantità di dati, ne aggregano le fonti, garantiscono la coerenza degli insiemi di dati e creano visualizzazioni per contribuire alla loro comprensione. Costruiscono modelli matematici che utilizzano dati, presentano e comunicano informazioni e conoscenze sui dati agli specialisti e agli scienziati nella loro squadra e, se necessario, a un pubblico non specializzato e raccomandano modalità di applicazione dei dati.\",0.5\nfootwear CAD patternmaker,data analyst,0.5\nufficiale di stato civile,data scientist,0.5\ndental instrument assembler,data scientist,0.5\nwaste management supervisor,data analyst,0.5\ncarrosserie- en voertuigbouwer,data scientist,0.5\nfuel station specialised seller,data analyst,0.5\nproductieleider chemische industrie,data scientist,0.5\nmarinaio addetto al servizio di coperta,data scientist,0.5\nallenatore sportivo/allenatrice sportiva,data scientist,0.5\nallenatore sportivo/allenatrice sportiva,\"I data scientist scoprono e interpretano fonti ricche di dati, gestiscono grandi quantità di dati, ne aggregano le fonti, garantiscono la coerenza degli insiemi di dati e creano visualizzazioni per contribuire alla loro comprensione. Costruiscono modelli matematici che utilizzano dati, presentano e comunicano informazioni e conoscenze sui dati agli specialisti e agli scienziati nella loro squadra e, se necessario, a un pubblico non specializzato e raccomandano modalità di applicazione dei dati.\",0.5\nwater conservation technician supervisor,data scientist,0.5\nsupervisore di assemblaggio di veicoli a motore,data scientist,0.5\ntecnico della qualità dei prodotti di pelletteria,data scientist,0.5\nresponsabile della distribuzione di macchine e attrezzature agricole,data scientist,0.5\n\"wholesale merchant in agricultural raw materials, seeds and animal feeds\",data analyst,0.5\nconfezionatore caseario artigianale /confezionatrice casearia artigianale,data scientist,0.5\nconfezionatore caseario artigianale /confezionatrice casearia artigianale,\"I data scientist scoprono e interpretano fonti ricche di dati, gestiscono grandi quantità di dati, ne aggregano le fonti, garantiscono la coerenza degli insiemi di dati e creano visualizzazioni per contribuire alla loro comprensione. Costruiscono modelli matematici che utilizzano dati, presentano e comunicano informazioni e conoscenze sui dati agli specialisti e agli scienziati nella loro squadra e, se necessario, a un pubblico non specializzato e raccomandano modalità di applicazione dei dati.\",0.5\nimport-exportmanager ijzerwaren en producten voor loodgieterij en verwarming,data scientist,0.5\naddetto all’assemblaggio di strumenti di precisione/addetta all’assemblaggio di strumenti di precisione,data scientist,0.5\ndata analyst,\"Data analysts import, inspect, clean, transform, validate, model, or interpret collections of data with regard to the business goals of the company. They ensure that the data sources and repositories provide consistent and reliable data. Data analysts use different algorithms and IT tools as demanded by the situation and the current data. They might prepare reports in the form of visualisations such as graphs, charts, and dashboards.\",1.0\ndata scientist,\"Data scientists find and interpret rich data sources, manage large amounts of data, merge data sources, ensure consistency of data-sets, and create visualisations to aid in understanding data. They build mathematical models using data, present and communicate data insights and findings to specialists and scientists in their team and if required, to a non-expert audience, and recommend ways to apply the data.\",1.0\ndata scientist,\"Data scientists zoeken en interpreteren rijke gegevensbronnen, beheren grote hoeveelheden gegevens, voegen gegevensbronnen samen, zorgen voor de consistentie van datasets en creëren visualisaties om te helpen gegevens te begrijpen. Zij bouwen wiskundige modellen op basis van data, presenteren en communiceren gegevensinzichten en bevindingen aan specialisten en wetenschappers in hun team en, indien nodig, aan een niet-deskundig publiek, en bevelen manieren aan om de data toe te passen.\",1.0\ndata scientist,\"I data scientist scoprono e interpretano fonti ricche di dati, gestiscono grandi quantità di dati, ne aggregano le fonti, garantiscono la coerenza degli insiemi di dati e creano visualizzazioni per contribuire alla loro comprensione. Costruiscono modelli matematici che utilizzano dati, presentano e comunicano informazioni e conoscenze sui dati agli specialisti e agli scienziati nella loro squadra e, se necessario, a un pubblico non specializzato e raccomandano modalità di applicazione dei dati.\",1.0\ncosmologist,cosmology data scientist,0.74681668889336494\ndata analyst,data storage analyst,0.74681668889336494\ndata analyst,data warehouse analyst,0.74681668889336494\ndata analyst,data warehousing analyst,0.74681668889336494\nmeter reader,metering data analyst,0.74681668889336494\nstatistician,statistical data analyst,0.74681668889336494\ndata scientist,esperta di dati,0.74681668889336494\ndata scientist,data engineer,0.74681668889336494\ndata scientist,data-scientist,0.74681668889336494\ndata scientist,data research scientist,0.74681668889336494\ndata scientist,esperto di dati,0.74681668889336494\ndata scientist,data expert,0.74681668889336494\ndata scientist,data analyst,0.74681668889336494\ndata scientist,research data scientist,0.74681668889336494\ndata scientist,research analist,0.74681668889336494\ndata scientist,research data scientist,0.74681668889336494\ndata scientist,analista dei dati di ricerca,0.74681668889336494\ndata scientist,business data scientist,0.74681668889336494\nanalista dei dati,data analyst,0.74681668889336494\ncall centre analyst,sales data analyst,0.74681668889336494\ncall centre analyst,CRM data analyst,0.74681668889336494\ncall centre analyst,customer data analyst,0.74681668889336494\ncall centre analyst,senior data analyst,0.74681668889336494\ncall centre analyst,assistant data analyst,0.74681668889336494\ncall centre analyst,trainee data analyst,0.74681668889336494\ncall centre analyst,graduate data analyst,0.74681668889336494\ncall centre analyst,marketing data analyst,0.74681668889336494\ncall centre analyst,IT data analyst,0.74681668889336494\nanalyste de données,data analyst,0.74681668889336494\nbioinformatics scientist,data scientist,0.74681668889336494\nscientifique des données,data scientist,0.74681668889336494"
},
{
"path": "natural-language-processing/embedding-models/domain_adaption_fine_tune_nlp_model.ipynb",
"content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"id\": \"cba912be-879a-4a9b-a7e9-180f42555cb9\",\n \"metadata\": {},\n \"source\": [\n \"## Domain Adaption: Fine-Tune Pre-Trained NLP Models\\n\",\n \"\\n\",\n \"### Introduction\\n\",\n \"In today's world, the availability of pre-trained NLP models has greatly simplified the interpretation of textual data using deep learning techniques. However, while these models excel in general tasks, they often lack adaptability to specific domains. This comprehensive guide aims to walk you through the process of fine-tuning pre-trained NLP models to achieve improved performance in a particular domain.\\n\",\n \"\\n\",\n \"#### Motivation\\n\",\n \"Although pre-trained NLP models like BERT and the Universal Sentence Encoder (USE) are effective in capturing linguistic intricacies, their performance in domain-specific applications can be limited due to the diverse range of datasets they are trained on. This limitation becomes evident when analyzing relationships within a specific domain. \\n\",\n \"\\n\",\n \"For example, when working with employment data, we expect the model to recognize the closer proximity between the roles of 'Data Scientist' and 'Machine Learning Engineer', or the stronger association between 'Python' and 'TensorFlow'. Unfortunately, general-purpose models often miss these nuanced relationships.\\n\",\n \"\\n\",\n \"To address this issue, we can fine-tune pre-trained models with high-quality, domain-specific datasets. This adaptation process significantly enhances the model's performance and precision, fully unlocking the potential of the NLP model.\\n\",\n \"\\n\",\n \"When dealing with large pre-trained NLP models, it is advisable to initially deploy the base model and consider fine-tuning only if its performance falls short for the specific problem at hand.\\n\",\n \" \\n\",\n \"This tutorial focuses on fine-tuning the Universal Sentence Encoder (USE) model using easily accessible open-source data.\\n\",\n \"\\n\",\n \"### Theoretical Overview\\n\",\n \"Fine-tuning an ML model can be achieved through various strategies, such as supervised learning and reinforcement learning. In this tutorial, we will concentrate on a one(few)-shot learning approach combined with a siamese architecture for the fine-tuning process.\\n\",\n \"\\n\",\n \"#### Methodology\\n\",\n \"In this tutorial, we utilize a siamese neural network, which is a specific type of Artificial Neural Network. This network leverages shared weights while simultaneously processing two distinct input vectors to compute comparable output vectors. Inspired by one-shot learning, this approach has proven to be particularly effective in capturing semantic similarity, although it may require longer training times and lack probabilistic output.\\n\",\n \"\\n\",\n \"A Siamese Neural Network creates an 'embedding space' where related concepts are positioned closely, enabling the model to better discern semantic relations.\\n\",\n \"- Twin Branches and Shared Weights: The architecture consists of two identical branches, each containing an embedding layer with shared weights. These dual branches handle two inputs simultaneously, either similar or dissimilar.\\n\",\n \"- Similarity and Transformation: The inputs are transformed into vector embeddings using the pre-trained NLP model. The architecture then calculates the similarity between the vectors. The similarity score, ranging between -1 and 1, quantifies the angular distance between the two vectors, serving as a metric for their semantic similarity.\\n\",\n \"- Contrastive Loss and Learning: The model's learning is guided by the \\\"Contrastive Loss,\\\" which is the difference between the expected output (similarity score from the training data) and the computed similarity. This loss guides the adjustment of the model's weights to minimize the loss and enhance the quality of the learned embeddings.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"13a3c7e3-4b5f-4369-b6b0-e0ea5b43f425\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import pandas as pd\\n\",\n \"import math\\n\",\n \"import tensorflow as tf\\n\",\n \"import tensorflow_hub as hub\\n\",\n \"from tensorflow import keras\\n\",\n \"from tensorflow_text import SentencepieceTokenizer\\n\",\n \"import os\\n\",\n \"from datetime import datetime\\n\",\n \"import numpy as np\\n\",\n \"from utils import *\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"9c770e7f-5a4e-42d7-89d6-a8eb0eec8210\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Data Overview\\n\",\n \"\\n\",\n \"For the fine-tuning of pre-trained NLP models using this method, the training data should consist of pairs of text strings accompanied by similarity scores between them. \\n\",\n \"\\n\",\n \"In this tutorial, we use a dataset sourced from the ESCO classification dataset, which has been transformed to generate similarity scores based on the relationships between different data elements.\\n\",\n \"\\n\",\n \"Preparing the training data is a crucial step in the fine-tuning process. It is assumed that you have access to the required data and a method to transform it into the specified format. Since the focus of this article is to demonstrate the fine-tuning process, we will omit the details of how the data was generated using the ESCO dataset.\\n\",\n \"\\n\",\n \"Let's start by examining the training data:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"94c76e58-475b-49b4-a97e-5c7b1e6e2630\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# The data from this file is stored in the variable \\\"data\\\".\\n\",\n \"data = pd.read_csv(\\\"./data/training_data.csv\\\")\\n\",\n \"\\n\",\n \"# Use the head function on the DataFrame to display its first 5 rows.\\n\",\n \"data.head()\\n\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"20dc8c3c-2835-4472-894f-12239de8173b\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Baseline Model\\n\",\n \"To begin, we establish the multilingual universal sentence encoder as our baseline model. It is essential to set this baseline before proceeding with the fine-tuning process.\\n\",\n \"\\n\",\n \"For this tutorial, we will use the STS benchmark and a sample similarity visualization as metrics to evaluate the changes and improvements achieved through the fine-tuning process.\\n\",\n \"\\n\",\n \"The STS Benchmark dataset consists of English sentence pairs, each associated with a similarity score. During the model training process, we evaluate the model's performance on this benchmark set. The persisted scores for each training run are the Pearson correlation between the predicted similarity scores and the actual similarity scores in the dataset. \\n\",\n \"\\n\",\n \"These scores ensure that as the model is fine-tuned with our context-specific training data, it maintains some level of generalizability.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"f643d10f-716e-4982-984e-48f0bbf2e3df\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Loads the Universal Sentence Encoder Multilingual module from TensorFlow Hub.\\n\",\n \"base_model_url = \\\"https://tfhub.dev/google/universal-sentence-encoder-multilingual/3\\\"\\n\",\n \"base_model = tf.keras.Sequential([\\n\",\n \" hub.KerasLayer(base_model_url,\\n\",\n \" input_shape=[],\\n\",\n \" dtype=tf.string,\\n\",\n \" trainable=False)\\n\",\n \"])\\n\",\n \"\\n\",\n \"# Defines a list of test sentences. These sentences represent various job titles.\\n\",\n \"test_text = ['Data Scientist', 'Data Analyst', 'Data Engineer',\\n\",\n \" 'Nurse Practitioner', 'Registered Nurse', 'Medical Assistant',\\n\",\n \" 'Social Media Manager', 'Marketing Strategist', 'Product Marketing Manager']\\n\",\n \"\\n\",\n \"# Creates embeddings for the sentences in the test_text list. \\n\",\n \"# The np.array() function is used to convert the result into a numpy array.\\n\",\n \"# The .tolist() function is used to convert the numpy array into a list, which might be easier to work with.\\n\",\n \"vectors = np.array(base_model.predict(test_text)).tolist()\\n\",\n \"\\n\",\n \"# Calls the plot_similarity function to create a similarity plot.\\n\",\n \"plot_similarity(test_text, vectors, 90, \\\"base model\\\")\\n\",\n \"\\n\",\n \"# Computes STS benchmark score for the base model\\n\",\n \"pearsonr = sts_benchmark(base_model)\\n\",\n \"print(\\\"STS Benachmark: \\\" + str(pearsonr))\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"51a178d6-760d-4c0b-8994-0bac82118a98\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Fine Tuning the Model\\n\",\n \"The next step involves constructing the siamese model architecture using the baseline model and fine-tuning it with our domain-specific data.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"f65eca14-f5c4-41f5-8048-ce7480759a33\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Load the pre-trained word embedding model\\n\",\n \"embedding_layer = hub.load(base_model_url)\\n\",\n \"\\n\",\n \"# Create a Keras layer from the loaded embedding model\\n\",\n \"shared_embedding_layer = hub.KerasLayer(embedding_layer, trainable=True)\\n\",\n \"\\n\",\n \"# Define the inputs to the model\\n\",\n \"left_input = keras.Input(shape=(), dtype=tf.string)\\n\",\n \"right_input = keras.Input(shape=(), dtype=tf.string)\\n\",\n \"\\n\",\n \"# Pass the inputs through the shared embedding layer\\n\",\n \"embedding_left_output = shared_embedding_layer(left_input)\\n\",\n \"embedding_right_output = shared_embedding_layer(right_input)\\n\",\n \"\\n\",\n \"# Compute the cosine similarity between the embedding vectors\\n\",\n \"cosine_similarity = tf.keras.layers.Dot(axes=-1, normalize=True)(\\n\",\n \" [embedding_left_output, embedding_right_output]\\n\",\n \")\\n\",\n \"\\n\",\n \"# Convert the cosine similarity to angular distance\\n\",\n \"pi = tf.constant(math.pi, dtype=tf.float32)\\n\",\n \"clip_cosine_similarities = tf.clip_by_value(\\n\",\n \" cosine_similarity, -0.99999, 0.99999\\n\",\n \")\\n\",\n \"acos_distance = 1.0 - (tf.acos(clip_cosine_similarities) / pi)\\n\",\n \"\\n\",\n \"# Package the model\\n\",\n \"encoder = tf.keras.Model([left_input, right_input], acos_distance)\\n\",\n \"\\n\",\n \"# Compile the model\\n\",\n \"encoder.compile(\\n\",\n \" optimizer=tf.keras.optimizers.Adam(\\n\",\n \" learning_rate=0.00001,\\n\",\n \" beta_1=0.9,\\n\",\n \" beta_2=0.9999,\\n\",\n \" epsilon=0.0000001,\\n\",\n \" amsgrad=False,\\n\",\n \" clipnorm=1.0,\\n\",\n \" name=\\\"Adam\\\",\\n\",\n \" ),\\n\",\n \" loss=tf.keras.losses.MeanSquaredError(\\n\",\n \" reduction=keras.losses.Reduction.AUTO, name=\\\"mean_squared_error\\\"\\n\",\n \" ),\\n\",\n \" metrics=[\\n\",\n \" tf.keras.metrics.MeanAbsoluteError(),\\n\",\n \" tf.keras.metrics.MeanAbsolutePercentageError(),\\n\",\n \" ],\\n\",\n \")\\n\",\n \"\\n\",\n \"# Print the model summary\\n\",\n \"encoder.summary()\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"250e87af-306a-47b9-8349-ca64c54f06ac\",\n \"metadata\": {\n \"scrolled\": true\n },\n \"outputs\": [],\n \"source\": [\n \"early_stop = keras.callbacks.EarlyStopping(\\n\",\n \" monitor=\\\"loss\\\", patience=3, min_delta=0.001\\n\",\n \" )\\n\",\n \"logdir = os.path.join(\\n\",\n \" \\\".\\\",\\n\",\n \" \\\"logs/fit/\\\" + datetime.now().strftime(\\\"%Y%m%d-%H%M%S\\\"),\\n\",\n \" )\\n\",\n \"tensorboard_callback = keras.callbacks.TensorBoard(log_dir=logdir)\\n\",\n \"\\n\",\n \"# Model Input\\n\",\n \"left_inputs, right_inputs, similarity = process_model_input(data)\\n\",\n \"\\n\",\n \"history = encoder.fit(\\n\",\n \" [left_inputs, right_inputs],\\n\",\n \" similarity,\\n\",\n \" batch_size=8,\\n\",\n \" epochs=20,\\n\",\n \" validation_split=0.2,\\n\",\n \" callbacks=[early_stop, tensorboard_callback],\\n\",\n \" )\\n\",\n \"\\n\",\n \"inputs = keras.Input(shape=[], dtype=tf.string)\\n\",\n \"embedding = hub.KerasLayer(embedding_layer)(inputs)\\n\",\n \"\\n\",\n \"tuned_model = keras.Model(inputs=inputs, outputs=embedding)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"06975f60-e3a1-4c46-b5fa-3bc2512061a6\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Evaluation\\n\",\n \"\\n\",\n \"Now that we have the fine-tuned model, let's re-evaluate it and compare the results to those of the base model.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"3260ff05-07aa-4c55-9509-4ca60fada326\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Creates embeddings for the sentences in the test_text list. \\n\",\n \"# The np.array() function is used to convert the result into a numpy array.\\n\",\n \"# The .tolist() function is used to convert the numpy array into a list, which might be easier to work with.\\n\",\n \"vectors = np.array(tuned_model.predict(test_text)).tolist()\\n\",\n \"\\n\",\n \"# Calls the plot_similarity function to create a similarity plot.\\n\",\n \"plot_similarity(test_text, vectors, 90, \\\"tuned model\\\")\\n\",\n \"\\n\",\n \"# Computes STS benchmark score for the tuned model\\n\",\n \"pearsonr = sts_benchmark(tuned_model)\\n\",\n \"print(\\\"STS Benachmark: \\\" + str(pearsonr))\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"8807399d-3360-4d7b-8413-4eb0c033e922\",\n \"metadata\": {},\n \"source\": [\n \"Based on fine-tuning the model on the relatively small dataset, the STS benchmark score is comparable to that of the baseline model, indicating that the tuned model still exhibits generalizability. However, the similarity visualization demonstrates strengthened similarity scores between similar titles and a reduction in scores for dissimilar ones.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"114c50ab-9d12-4c3b-8ea9-f4ca2725efd3\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"### Closing Thoughts\\n\",\n \"\\n\",\n \"Fine-tuning pre-trained NLP models for domain adaptation is a powerful technique to improve their performance and precision in specific contexts. By utilizing quality, domain-specific datasets and leveraging siamese neural networks, we can enhance the model's ability to capture semantic similarity.\\n\",\n \"\\n\",\n \"This tutorial provided a step-by-step guide to the fine-tuning process, using the Universal Sentence Encoder (USE) model as an example. We explored the theoretical framework, data preparation, baseline model evaluation, and the actual fine-tuning process. The results demonstrated the effectiveness of fine-tuning in strengthening similarity scores within a domain.\\n\",\n \"\\n\",\n \"By following this approach and adapting it to your specific domain, you can unlock the full potential of pre-trained NLP models and achieve better results in your natural language processing tasks\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.9.16\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 5\n}\n"
},
{
"path": "natural-language-processing/embedding-models/pyproject.toml",
"content": "[tool.poetry]\nname = \"embedding-models\"\nversion = \"0.1.0\"\ndescription = \"\"\nauthors = [\"Shashank Kapadia \"]\nreadme = \"README.md\"\n\n[tool.poetry.dependencies]\npython = \">=3.9,<3.10\"\njupyterlab = \"^4.0.2\"\ntensorflow-hub = \"^0.13.0\"\ntensorflow = [\n { version = \"2.10.0\", platform = \"linux\" },\n]\ntensorflow-macos = [\n { version = \"2.10.0\", platform = \"darwin\" },\n]\n#tensorflow-text = \"^2.10.0\"\ntensorflow-text = [\n {file = \"packages/tensorflow_text-2.10.0-cp39-cp39-macosx_11_0_arm64.whl\", platform = \"darwin\"},\n]\npandas = \"^2.0.3\"\nseaborn = \"^0.12.2\"\ntqdm = \"^4.65.0\"\n\n\n[build-system]\nrequires = [\"poetry-core\"]\nbuild-backend = \"poetry.core.masonry.api\"\n"
},
{
"path": "natural-language-processing/embedding-models/utils.py",
"content": "import numpy as np\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nimport tensorflow as tf\nimport tqdm\nfrom scipy import stats\nimport pandas as pd\nimport os\nimport math\n\n\ndef plot_similarity(labels, features, rotation, version):\n corr = np.inner(features, features)\n sns.set(font_scale=1.2)\n g = sns.heatmap(\n corr,\n xticklabels=labels,\n yticklabels=labels,\n vmin=0,\n vmax=1,\n cmap=\"YlOrRd\")\n g.set_xticklabels(labels, rotation=rotation)\n g.set_title(version)\n\ndef cosine_similarity(vector_1, vector_2):\n \"\"\"\n Compute cosine similarity between two vectors.\n\n Args:\n vector_1 (List[float]): A list of float values representing the first vector.\n vector_2 (List[float]): A list of float values representing the second vector.\n\n Returns:\n float: The cosine similarity between the two vectors, which is the dot product of\n the two vectors divided by the product of their magnitudes.\n \"\"\"\n sumxx, sumxy, sumyy = 0, 0, 0\n for i in range(len(vector_1)):\n x = vector_1[i]\n y = vector_2[i]\n sumxx += x * x\n sumyy += y * y\n sumxy += x * y\n return sumxy / math.sqrt(sumxx * sumyy)\n\n\ndef sts_benchmark(model, type=\"dev\"):\n \"\"\"\n Compute the Pearson correlation between predicted cosine similarity scores and human-labeled similarity scores\n on the STS benchmark dataset.\n\n Args:\n model (tensorflow.keras.Model): A trained sentence embedding model that takes in an input sentence and\n outputs a corresponding sentence embedding.\n type (str): The type of STS benchmark dataset to use. Either \"dev\" for the development dataset or \"test\"\n for the test dataset. Default is \"dev\".\n\n Returns:\n tuple: A tuple containing the Pearson correlation coefficient and the p-value of the correlation test.\n \"\"\"\n\n def _get_sts_dataset(type=\"test\"):\n \"\"\"\n\n :param type:\n :return:\n \"\"\"\n\n sts_dataset = tf.keras.utils.get_file(\n fname=\"Stsbenchmark.tar.gz\",\n origin=\"http://ixa2.si.ehu.es/stswiki/images/4/48/Stsbenchmark.tar.gz\",\n extract=True,\n )\n if type == \"dev\":\n data = pd.read_table(\n os.path.join(\n os.path.dirname(sts_dataset), \"stsbenchmark\", \"sts-dev.csv\"\n ),\n on_bad_lines=\"skip\",\n engine=\"python\",\n skip_blank_lines=True,\n usecols=[4, 5, 6],\n names=[\"sim\", \"sent_1\", \"sent_2\"],\n )\n else:\n data = pd.read_table(\n os.path.join(\n os.path.dirname(sts_dataset), \"stsbenchmark\", \"sts-test.csv\"\n ),\n on_bad_lines=\"skip\",\n engine=\"python\",\n skip_blank_lines=True,\n usecols=[4, 5, 6],\n names=[\"sim\", \"sent_1\", \"sent_2\"],\n )\n\n return data\n\n data = _get_sts_dataset(type=type)\n data = data[[isinstance(s, str) for s in data[\"sent_2\"]]].reset_index()\n\n # prepare data\n base_text = [data[\"sent_1\"][i] for i in range(len(data))]\n ref_text = [data[\"sent_2\"][i] for i in range(len(data))]\n scores = data[\"sim\"].tolist()\n\n base_vectors = []\n ref_vectors = []\n\n # get text vectors from base, tuned, pre-tuned models\n for i in range(len(base_text)):\n base_vectors.append(list(model.predict([base_text[i]])[0]))\n ref_vectors.append(list(model.predict([ref_text[i]])[0]))\n\n base_cosine_similarity = [\n cosine_similarity(base_vectors[i], ref_vectors[i])\n for i in range(len(base_text))\n ]\n return stats.pearsonr(scores, base_cosine_similarity)\n\ndef process_model_input(data):\n \"\"\"\n Processes the input data by reshaping the left and right inputs and converting the similarity values.\n\n Args:\n data: (dict) A dictionary containing the keys \"base\", \"ref\", and \"similarity\",\n with values corresponding to the input base text, reference text, and similarity values respectively.\n\n Returns:\n tuple: A tuple of three numpy arrays containing the preprocessed left inputs, right inputs,\n and similarity values respectively.\n \"\"\"\n text_list = [list(data[\"base\"].values), list(data[\"ref\"].values)]\n left_inputs = np.asarray(text_list[0])\n right_inputs = np.asarray(text_list[1])\n left_inputs = left_inputs.reshape(\n left_inputs.shape[0],\n )\n right_inputs = right_inputs.reshape(\n right_inputs.shape[0],\n )\n\n # 1 if we inputs are semantically similiar, 0 if not.\n # Check the distance function defined as 1-arccos(similiarity)/pi which has range between 1,0 for domain 0 to 1\n similarity = np.asarray(list(data[\"similarity\"].values))\n\n return left_inputs, right_inputs, similarity\n\n"
},
{
"path": "natural-language-processing/text-processing/Building Blocks Text Pre-Processing.ipynb",
"content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Building Blocks: Text Pre-Processing\\n\",\n \"\\n\",\n \"This article is the second of more to come articles on Natural Language Processing. The purpose of this series of articles is to document my journey as I learn about this subject, as well as help others gain efficiency from it.\\n\",\n \"\\n\",\n \"In the last article of our series, we introduced the concept of Natural Language Processing, you can read it here, and now you probably want to try it yourself, right? Great! Without further ado, let's dive in to the building blocks for statistical natural language processing. \\n\",\n \"\\n\",\n \"In this article, we'll introduce the key concepts, along with practical implementation in Python and the challenges to keep in mind at the time of application.\\n\",\n \"\\n\",\n \"**References:**\\n\",\n \"- A General Approach to Preprocessing Text Data — KDnuggets. https://www.kdnuggets.com/2017/12/general-approach-preprocessing-text-data.html\\n\",\n \"- Tokenization — Stanford NLP Group. https://nlp.stanford.edu/IR-book/html/htmledition/tokenization-1.html\\n\",\n \"- Text Mining in Bovine Diseases — ijcaonline.org. https://www.ijcaonline.org/volume6/number10/pxc3871454.pdf\\n\",\n \"- Stemming and lemmatization — Stanford NLP Group. https://nlp.stanford.edu/IR-book/html/htmledition/stemming-and-lemmatization-1.html\\n\",\n \"** **\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Text Normalization\\n\",\n \"\\n\",\n \"Normalizing the text means converting it to a more convenient, standard form before performing turning it to features for higher level modeling. Think of this step as converting human readable language into a form that is machine readable.\\n\",\n \"\\n\",\n \"The standard framework to normalize the text includes:\\n\",\n \"1. Tokenization\\n\",\n \"2. Stop Words Removal\\n\",\n \"3. Morphological Normalization\\n\",\n \"4. Collocation\\n\",\n \"\\n\",\n \"Data preprocessing consists of a number of steps, any number of which may or not apply to a given task. More generally, in this article we'll discuss some predetermined body of text, and perform some basic transformative analysis that can be used for performing further, more meaningful natural language processing\\n\",\n \"\\n\",\n \"** **\\n\",\n \"#### Tokenization\\n\",\n \"\\n\",\n \"Given a character sequence and a defined document unit (blurb of texts), tokenization is the task of chopping it up into pieces, called tokens, perhaps at the same time throwing away certain characters/words, such as punctuation. Ordinarily, there are two types of tokenization:\\n\",\n \"\\n\",\n \"1. Word Tokenization: Used to separate words via unique space character. Depending on the application, word tokenization may also tokenize multi-word expressions like New York. This is often times is closely tied to a process called Named Entity Recognition. Later in this tutorial, we will look at Collocation (Phrase) Modeling that helps address part of this challenge\\n\",\n \"\\n\",\n \"2. Sentence Tokenization/Segmentation: Along with word tokenization, sentence segmentation is a crucial step in text processing. This is usually performed based on punctuations such as \\\".\\\", \\\"?\\\", \\\"!\\\" as they tend to mark the sentence boundaries\\n\",\n \"\\n\",\n \"**Challenges:**\\n\",\n \"- The use of abbreviations may prompt the tokenizer to detect a sentence boundary where there is none. \\n\",\n \"- Numbers, special characters, hyphenation, and capitalization. In the expressions \\\"don't,\\\" \\\"I'd,\\\" \\\"John's\\\" do we have one, two or three tokens?\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import nltk\\n\",\n \"\\n\",\n \"nltk.download('punkt')\\n\",\n \"nltk.download('stopwords')\\n\",\n \"nltk.download('wordnet')\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from nltk.tokenize import sent_tokenize, word_tokenize\\n\",\n \"\\n\",\n \"#Sentence Tokenization\\n\",\n \"print ('Following is the list of sentences tokenized from the sample review\\\\n')\\n\",\n \"\\n\",\n \"sample_text = \\\"\\\"\\\"The first time I ate here I honestly was not that impressed. I decided to wait a bit and give it another chance. \\n\",\n \"I have recently eaten there a couple of times and although I am not convinced that the pricing is particularly on point the two mushroom and \\n\",\n \"swiss burgers I had were honestly very good. The shakes were also tasty. Although Mad Mikes is still my favorite burger around, \\n\",\n \"you can do a heck of a lot worse than Smashburger if you get a craving\\\"\\\"\\\"\\n\",\n \"\\n\",\n \"tokenize_sentence = sent_tokenize(sample_text)\\n\",\n \"\\n\",\n \"print (tokenize_sentence)\\n\",\n \"print ('---------------------------------------------------------\\\\n')\\n\",\n \"print ('Following is the list of words tokenized from the sample review sentence\\\\n')\\n\",\n \"tokenize_words = word_tokenize(tokenize_sentence[1])\\n\",\n \"print (tokenize_words)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Stop Words Removal\\n\",\n \"Often, there are a few ubiquitous words which would appear to be of little value in helping the purpose of analysis but increases the dimensionality of feature set, are excluded from the vocabulary entirely as the part of stop words removal process. There are two considerations usually that motivate this removal.\\n\",\n \"\\n\",\n \"1. Irrelevance: Allows one to analyze only on content-bearing words. Stopwords, also called empty words because they generally do not bear much meaning, introduce noise in the analysis/modeling process\\n\",\n \"2. Dimension: Removing the stopwords also allows one to reduce the tokens in documents significantly, and thereby decreasing feature dimension\\n\",\n \"\\n\",\n \"**Challenges:**\\n\",\n \"\\n\",\n \"Converting all characters into lowercase letters before stopwords removal process can introduce ambiguity in the text, and sometimes entirely changing the meaning of it. For example, with the expressions \\\"US citizen\\\" will be viewed as \\\"us citizen\\\" or \\\"IT scientist\\\" as \\\"it scientist\\\". Since both *us* and *it* are normally considered stop words, it would result in an inaccurate outcome. The strategy regarding the treatment of stopwords can thus be refined by identifying that \\\"US\\\" and \\\"IT\\\" are not pronouns in the above examples, through a part-of-speech tagging step.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from nltk.corpus import stopwords\\n\",\n \"from nltk.tokenize import word_tokenize\\n\",\n \"\\n\",\n \"# define the language for stopwords removal\\n\",\n \"stopwords = set(stopwords.words(\\\"english\\\"))\\n\",\n \"print (\\\"\\\"\\\"{0} stop words\\\"\\\"\\\".format(len(stopwords)))\\n\",\n \"\\n\",\n \"tokenize_words = word_tokenize(sample_text)\\n\",\n \"filtered_sample_text = [w for w in tokenize_words if not w in stopwords]\\n\",\n \"\\n\",\n \"print ('\\\\nOriginal Text:')\\n\",\n \"print ('------------------\\\\n')\\n\",\n \"print (sample_text)\\n\",\n \"print ('\\\\n Filtered Text:')\\n\",\n \"print ('------------------\\\\n')\\n\",\n \"print (' '.join(str(token) for token in filtered_sample_text))\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Morphological Normalization\\n\",\n \"Morphology, in general, is the study of the way words are built up from smaller meaning-bearing units, morphomes. For example, dogs consists of two morphemes: dog and s\\n\",\n \"\\n\",\n \"Two commonly used techniques for text normalization are:\\n\",\n \"\\n\",\n \"1. Stemming: The procedure aims to identify the stem of a word and use it in lieu of the word itself. The most popular algorithm for stemming English, and one that has repeatedly been shown to be empirically very effective, is Porter's algorithm. The entire algorithm is too long and intricate to present here, but you can find details here\\n\",\n \"2. Lemmatization: This process refers to doing things correctly with the use of vocabulary and morphological analysis of words, typically aiming to remove inflectional endings only and to return the base or dictionary form of a word, which is known as the lemma.\\n\",\n \"\\n\",\n \"If confronted with the token saw, stemming might return just s, whereas lemmatization would attempt to return either see or saw depending on whether the use of the token was as a verb or a noun\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from nltk.stem import PorterStemmer\\n\",\n \"from nltk.stem import WordNetLemmatizer\\n\",\n \"from nltk.tokenize import word_tokenize\\n\",\n \"\\n\",\n \"ps = PorterStemmer()\\n\",\n \"lemmatizer = WordNetLemmatizer()\\n\",\n \"\\n\",\n \"tokenize_words = word_tokenize(sample_text)\\n\",\n \"\\n\",\n \"stemmed_sample_text = []\\n\",\n \"for token in tokenize_words:\\n\",\n \" stemmed_sample_text.append(ps.stem(token))\\n\",\n \"\\n\",\n \"lemma_sample_text = []\\n\",\n \"for token in tokenize_words:\\n\",\n \" lemma_sample_text.append(lemmatizer.lemmatize(token))\\n\",\n \" \\n\",\n \"print ('\\\\nOriginal Text:')\\n\",\n \"print ('------------------\\\\n')\\n\",\n \"print (sample_text)\\n\",\n \"\\n\",\n \"print ('\\\\nFiltered Text: Stemming')\\n\",\n \"print ('------------------\\\\n')\\n\",\n \"print (' '.join(str(token) for token in stemmed_sample_text))\\n\",\n \"\\n\",\n \"print ('\\\\nFiltered Text: Lemmatization')\\n\",\n \"print ('--------------------------------\\\\n')\\n\",\n \"print (' '.join(str(token) for token in lemma_sample_text))\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"**Challenges:**\\n\",\n \"\\n\",\n \"Often, full morphological analysis produces at most very modest benefits for analysis. Neither form of normalization improve language information performance in aggregate, both from relevance and dimensionality reduction standpoint - at least not for the following situations:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from nltk.stem import PorterStemmer\\n\",\n \"words = [\\\"operate\\\", \\\"operating\\\", \\\"operates\\\", \\\"operation\\\", \\\"operative\\\", \\\"operatives\\\", \\\"operational\\\"]\\n\",\n \"\\n\",\n \"ps = PorterStemmer()\\n\",\n \"\\n\",\n \"for token in words:\\n\",\n \" print (ps.stem(token))\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"As an example of what can go wrong, note that the Porter stemmer stems all of the following words to oper\\n\",\n \"However, since operate in its various forms is a common verb, we would expect to lose considerable precision:\\n\",\n \"- operational AND research\\n\",\n \"- operating AND system\\n\",\n \"- operative AND dentistry\\n\",\n \"\\n\",\n \"For cases like these, moving to using a lemmatizer would not completely fix the problem because particular inflectional forms are used in specific collocations. Getting better value from term normalization depends more on pragmatic issues of word use than on formal issues of linguistic morphology\"\n ]\n }\n ],\n \"metadata\": {\n \"environment\": {\n \"name\": \"common-cpu.m49\",\n \"type\": \"gcloud\",\n \"uri\": \"gcr.io/deeplearning-platform-release/base-cpu:m49\"\n },\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.10.6\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n"
},
{
"path": "natural-language-processing/text-processing/pyproject.toml",
"content": "[tool.poetry]\nname = \"text-processing\"\nversion = \"0.1.0\"\ndescription = \"\"\nauthors = [\"Shashank Kapadia \"]\nreadme = \"README.md\"\npackages = [{include = \"text_processing\"}]\n\n[tool.poetry.dependencies]\npython = \"^3.10\"\njupyterlab = \"^3.5.2\"\nnltk = \"^3.8.1\"\n\n\n[build-system]\nrequires = [\"poetry-core\"]\nbuild-backend = \"poetry.core.masonry.api\"\n"
},
{
"path": "natural-language-processing/topic-modeling/Evaluate Topic Models.ipynb",
"content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Evaluate Topic Model in Python: Latent Dirichlet Allocation (LDA)\\\\\\n\",\n \"##### A step-by-step guide to building interpretable topic models\\n\",\n \"\\n\",\n \"** **\\n\",\n \"*Preface: This article aims to provide consolidated information on the underlying topic and is not to be considered as the original work. The information and the code are repurposed through several online articles, research papers, books, and open-source code*\\n\",\n \"** **\\n\",\n \"\\n\",\n \"In the previous [article](https://towardsdatascience.com/end-to-end-topic-modeling-in-python-latent-dirichlet-allocation-lda-35ce4ed6b3e0), I introduced the concept of topic modeling and walked through the code for developing your first topic model using Latent Dirichlet Allocation (LDA) method in the python using Gensim implementation.\\n\",\n \"\\n\",\n \"Pursuing on that understanding, in this article, we’ll go a few steps deeper by outlining the framework to quantitatively evaluate topic models through the measure of topic coherence and share the code template in python using Gensim implementation to allow for end-to-end model development.\\n\",\n \"\\n\",\n \"### Why evaluate topic models?\\n\",\n \"\\n\",\n \"\\n\",\n \"\\n\",\n \"We know probabilistic topic models, such as LDA, are popular tools for text analysis, providing both a predictive and latent topic representation of the corpus. However, there is a longstanding assumption that the latent space discovered by these models is generally meaningful and useful, and that evaluating such assumptions is challenging due to its unsupervised training process. Besides, there is a no-gold standard list of topics to compare against every corpus.\\n\",\n \"\\n\",\n \"Nevertheless, it is equally important to identify if a trained model is objectively good or bad, as well have an ability to compare different models/methods. To do so, one would require an objective measure for the quality. Traditionally, and still for many practical applications, to evaluate if “the correct thing” has been learned about the corpus, an implicit knowledge and “eyeballing” approaches are used. Ideally, we’d like to capture this information in a single metric that can be maximized, and compared.\\n\",\n \"\\n\",\n \"Let’s take a look at roughly what approaches are commonly used for the evaluation:\\n\",\n \"\\n\",\n \"**Eye Balling Models**\\n\",\n \"- Top N words\\n\",\n \"- Topics / Documents\\n\",\n \"\\n\",\n \"**Intrinsic Evaluation Metrics**\\n\",\n \"- Capturing model semantics\\n\",\n \"- Topics interpretability\\n\",\n \"\\n\",\n \"**Human Judgements**\\n\",\n \"- What is a topic\\n\",\n \"\\n\",\n \"**Extrinsic Evaluation Metrics/Evaluation at task**\\n\",\n \"- Is model good at performing predefined tasks, such as classification\\n\",\n \"\\n\",\n \"Natural language is messy, ambiguous and full of subjective interpretation, and sometimes trying to cleanse ambiguity reduces the language to an unnatural form. In this article, we’ll explore more about topic coherence, an intrinsic evaluation metric, and how you can use it to quantitatively justify the model selection.\\n\",\n \"\\n\",\n \"### What is Topic Coherence?\\n\",\n \"\\n\",\n \"Before we understand topic coherence, let’s briefly look at the perplexity measure. Perplexity as well is one of the intrinsic evaluation metric, and is widely used for language model evaluation. It captures how surprised a model is of new data it has not seen before, and is measured as the normalized log-likelihood of a held-out test set. \\n\",\n \"\\n\",\n \"Focussing on the log-likelihood part, you can think of the perplexity metric as measuring how probable some new unseen data is given the model that was learned earlier. That is to say, how well does the model represent or reproduce the statistics of the held-out data.\\n\",\n \"\\n\",\n \"However, recent studies have shown that predictive likelihood (or equivalently, perplexity) and human judgment are often not correlated, and even sometimes slightly anti-correlated.\\n\",\n \"\\n\",\n \"*Optimizing for perplexity may not yield human interpretable topics*\\n\",\n \"\\n\",\n \"This limitation of perplexity measure served as a motivation for more work trying to model the human judgment, and thus *Topic Coherence*.\\n\",\n \"\\n\",\n \"The concept of topic coherence combines a number of measures into a framework to evaluate the coherence between topics inferred by a model. But before that…\\n\",\n \"\\n\",\n \"#### What is topic coherence?\\n\",\n \"Topic Coherence measures score a single topic by measuring the degree of semantic similarity between high scoring words in the topic. These measurements help distinguish between topics that are semantically interpretable topics and topics that are artifacts of statistical inference. But,\\n\",\n \"\\n\",\n \"#### What is coherence?\\n\",\n \"Topic Coherence measures score a single topic by measuring the degree of semantic similarity between high scoring words in the topic. These measurements help distinguish between topics that are semantically interpretable topics and topics that are artifacts of statistical inference. But …\\n\",\n \"\\n\",\n \"### Coherence Measures\\n\",\n \"Let’s take quick look at different coherence measures, and how they are calculated:\\n\",\n \"\\n\",\n \"1. `C_v` measure is based on a sliding window, one-set segmentation of the top words and an indirect confirmation measure that uses normalized pointwise mutual information (NPMI) and the cosine similarity\\n\",\n \"2. `C_p` is based on a sliding window, one-preceding segmentation of the top words and the confirmation measure of Fitelson's coherence\\n\",\n \"3. `C_uci` measure is based on a sliding window and the pointwise mutual information (PMI) of all word pairs of the given top words\\n\",\n \"4. `C_umass` is based on document cooccurrence counts, a one-preceding segmentation and a logarithmic conditional probability as confirmation measure\\n\",\n \"5. `C_npmi` is an enhanced version of the C_uci coherence using the normalized pointwise mutual information (NPMI)\\n\",\n \"6. `C_a` is based on a context window, a pairwise comparison of the top words and an indirect confirmation measure that uses normalized pointwise mutual information (NPMI) and the cosine similarity\\n\",\n \"\\n\",\n \"There is, of course, a lot more to the concept of topic model evaluation, and the coherence measure. However, keeping in mind the length, and purpose of this article, let’s apply these concepts into developing a model that is at least better than with the default parameters. Also, we’ll be re-purposing already available online pieces of code to support this exercise instead of re-inventing the wheel.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Model Implementation\\n\",\n \"1. Loading Data\\n\",\n \"2. Data Cleaning\\n\",\n \"3. Phrase Modeling: Bi-grams and Tri-grams\\n\",\n \"4. Data Transformation: Corpus and Dictionary\\n\",\n \"5. Base Model\\n\",\n \"6. Hyper-parameter Tuning\\n\",\n \"7. Final model\\n\",\n \"8. Visualize Results\\n\",\n \"\\n\",\n \"** **\\n\",\n \"\\n\",\n \"For this tutorial, we’ll use the dataset of papers published in NeurIPS (NIPS) conference which is one of the most prestigious yearly events in the machine learning community. The CSV data file contains information on the different NeurIPS papers that were published from 1987 until 2016 (29 years!). These papers discuss a wide variety of topics in machine learning, from neural networks to optimization methods, and many more.\\n\",\n \"\\n\",\n \"\\n\",\n \"\\n\",\n \"Let’s start by looking at the content of the file\\n\",\n \"\\n\",\n \"** **\\n\",\n \"#### Step 1: Loading Data\\n\",\n \"** **\\n\",\n \"\\n\",\n \"For this tutorial, we’ll use the dataset of papers published in NIPS conference. The NIPS conference (Neural Information Processing Systems) is one of the most prestigious yearly events in the machine learning community. The CSV data file contains information on the different NIPS papers that were published from 1987 until 2016 (29 years!). These papers discuss a wide variety of topics in machine learning, from neural networks to optimization methods, and many more.\\n\",\n \"\\n\",\n \"Let’s start by looking at the content of the file\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import zipfile\\n\",\n \"import pandas as pd\\n\",\n \"import os\\n\",\n \"\\n\",\n \"# Open the zip file\\n\",\n \"with zipfile.ZipFile(\\\"./data/NIPS Papers.zip\\\", \\\"r\\\") as zip_ref:\\n\",\n \" # Extract the file to a temporary directory\\n\",\n \" zip_ref.extractall(\\\"temp\\\")\\n\",\n \"\\n\",\n \"# Read the CSV file into a pandas DataFrame\\n\",\n \"papers = pd.read_csv(\\\"temp/NIPS Papers/papers.csv\\\")\\n\",\n \"\\n\",\n \"# Print head\\n\",\n \"papers.head()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Step 2: Data Cleaning\\n\",\n \"** **\\n\",\n \"\\n\",\n \"Since the goal of this analysis is to perform topic modeling, we will solely focus on the text data from each paper, and drop other metadata columns\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Remove the columns\\n\",\n \"papers = papers.drop(columns=['id', 'title', 'abstract', \\n\",\n \" 'event_type', 'pdf_name', 'year'], axis=1)\\n\",\n \"\\n\",\n \"# sample only 100 papers\\n\",\n \"papers = papers.sample(100)\\n\",\n \"\\n\",\n \"# Print out the first rows of papers\\n\",\n \"papers.head()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"##### Remove punctuation/lower casing\\n\",\n \"\\n\",\n \"Next, let’s perform a simple preprocessing on the content of paper_text column to make them more amenable for analysis, and reliable results. To do that, we’ll use a regular expression to remove any punctuation, and then lowercase the text\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Load the regular expression library\\n\",\n \"import re\\n\",\n \"\\n\",\n \"# Remove punctuation\\n\",\n \"papers['paper_text_processed'] = papers['paper_text'].map(lambda x: re.sub('[,\\\\.!?]', '', x))\\n\",\n \"\\n\",\n \"# Convert the titles to lowercase\\n\",\n \"papers['paper_text_processed'] = papers['paper_text_processed'].map(lambda x: x.lower())\\n\",\n \"\\n\",\n \"# Print out the first rows of papers\\n\",\n \"papers['paper_text_processed'].head()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"##### Tokenize words and further clean-up text\\n\",\n \"\\n\",\n \"Let’s tokenize each sentence into a list of words, removing punctuations and unnecessary characters altogether.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import gensim\\n\",\n \"from gensim.utils import simple_preprocess\\n\",\n \"\\n\",\n \"def sent_to_words(sentences):\\n\",\n \" for sentence in sentences:\\n\",\n \" yield(gensim.utils.simple_preprocess(str(sentence), deacc=True)) # deacc=True removes punctuations\\n\",\n \"\\n\",\n \"data = papers.paper_text_processed.values.tolist()\\n\",\n \"data_words = list(sent_to_words(data))\\n\",\n \"\\n\",\n \"print(data_words[:1][0][:30])\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Step 3: Phrase Modeling: Bigram and Trigram Models\\n\",\n \"** **\\n\",\n \"\\n\",\n \"Bigrams are two words frequently occurring together in the document. Trigrams are 3 words frequently occurring. Some examples in our example are: 'back_bumper', 'oil_leakage', 'maryland_college_park' etc.\\n\",\n \"\\n\",\n \"Gensim's Phrases model can build and implement the bigrams, trigrams, quadgrams and more. The two important arguments to Phrases are min_count and threshold.\\n\",\n \"\\n\",\n \"*The higher the values of these param, the harder it is for words to be combined.*\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Build the bigram and trigram models\\n\",\n \"bigram = gensim.models.Phrases(data_words, min_count=5, threshold=100) # higher threshold fewer phrases.\\n\",\n \"trigram = gensim.models.Phrases(bigram[data_words], threshold=100) \\n\",\n \"\\n\",\n \"# Faster way to get a sentence clubbed as a trigram/bigram\\n\",\n \"bigram_mod = gensim.models.phrases.Phraser(bigram)\\n\",\n \"trigram_mod = gensim.models.phrases.Phraser(trigram)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"#### Remove Stopwords, Make Bigrams and Lemmatize\\n\",\n \"\\n\",\n \"The phrase models are ready. Let’s define the functions to remove the stopwords, make trigrams and lemmatization and call them sequentially.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# NLTK Stop words\\n\",\n \"import nltk\\n\",\n \"nltk.download('stopwords')\\n\",\n \"from nltk.corpus import stopwords\\n\",\n \"\\n\",\n \"stop_words = stopwords.words('english')\\n\",\n \"stop_words.extend(['from', 'subject', 're', 'edu', 'use'])\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Define functions for stopwords, bigrams, trigrams and lemmatization\\n\",\n \"def remove_stopwords(texts):\\n\",\n \" return [[word for word in simple_preprocess(str(doc)) if word not in stop_words] for doc in texts]\\n\",\n \"\\n\",\n \"def make_bigrams(texts):\\n\",\n \" return [bigram_mod[doc] for doc in texts]\\n\",\n \"\\n\",\n \"def make_trigrams(texts):\\n\",\n \" return [trigram_mod[bigram_mod[doc]] for doc in texts]\\n\",\n \"\\n\",\n \"def lemmatization(texts, allowed_postags=['NOUN', 'ADJ', 'VERB', 'ADV']):\\n\",\n \" \\\"\\\"\\\"https://spacy.io/api/annotation\\\"\\\"\\\"\\n\",\n \" texts_out = []\\n\",\n \" for sent in texts:\\n\",\n \" doc = nlp(\\\" \\\".join(sent)) \\n\",\n \" texts_out.append([token.lemma_ for token in doc if token.pos_ in allowed_postags])\\n\",\n \" return texts_out\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"Let's call the functions in order.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"!python -m spacy download en_core_web_sm\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import spacy\\n\",\n \"\\n\",\n \"# Remove Stop Words\\n\",\n \"data_words_nostops = remove_stopwords(data_words)\\n\",\n \"\\n\",\n \"# Form Bigrams\\n\",\n \"data_words_bigrams = make_bigrams(data_words_nostops)\\n\",\n \"\\n\",\n \"# Initialize spacy 'en' model, keeping only tagger component (for efficiency)\\n\",\n \"nlp = spacy.load(\\\"en_core_web_sm\\\", disable=['parser', 'ner'])\\n\",\n \"\\n\",\n \"# Do lemmatization keeping only noun, adj, vb, adv\\n\",\n \"data_lemmatized = lemmatization(data_words_bigrams, allowed_postags=['NOUN', 'ADJ', 'VERB', 'ADV'])\\n\",\n \"\\n\",\n \"print(data_lemmatized[:1][0][:30])\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Step 4: Data transformation: Corpus and Dictionary\\n\",\n \"** **\\n\",\n \"\\n\",\n \"The two main inputs to the LDA topic model are the dictionary(id2word) and the corpus. Let’s create them.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import gensim.corpora as corpora\\n\",\n \"\\n\",\n \"# Create Dictionary\\n\",\n \"id2word = corpora.Dictionary(data_lemmatized)\\n\",\n \"\\n\",\n \"# Create Corpus\\n\",\n \"texts = data_lemmatized\\n\",\n \"\\n\",\n \"# Term Document Frequency\\n\",\n \"corpus = [id2word.doc2bow(text) for text in texts]\\n\",\n \"\\n\",\n \"# View\\n\",\n \"print(corpus[:1][0][:30])\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Step 5: Base Model \\n\",\n \"** **\\n\",\n \"\\n\",\n \"We have everything required to train the base LDA model. In addition to the corpus and dictionary, you need to provide the number of topics as well. Apart from that, alpha and eta are hyperparameters that affect sparsity of the topics. According to the Gensim docs, both defaults to 1.0/num_topics prior (we'll use default for the base model).\\n\",\n \"\\n\",\n \"chunksize controls how many documents are processed at a time in the training algorithm. Increasing chunksize will speed up training, at least as long as the chunk of documents easily fit into memory.\\n\",\n \"\\n\",\n \"passes controls how often we train the model on the entire corpus (set to 10). Another word for passes might be \\\"epochs\\\". iterations is somewhat technical, but essentially it controls how often we repeat a particular loop over each document. It is important to set the number of \\\"passes\\\" and \\\"iterations\\\" high enough.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Build LDA model\\n\",\n \"lda_model = gensim.models.LdaMulticore(corpus=corpus,\\n\",\n \" id2word=id2word,\\n\",\n \" num_topics=10, \\n\",\n \" random_state=100,\\n\",\n \" chunksize=100,\\n\",\n \" passes=10,\\n\",\n \" per_word_topics=True)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"The above LDA model is built with 10 different topics where each topic is a combination of keywords and each keyword contributes a certain weightage to the topic.\\n\",\n \"\\n\",\n \"You can see the keywords for each topic and the weightage(importance) of each keyword using `lda_model.print_topics()`\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from pprint import pprint\\n\",\n \"\\n\",\n \"# Print the Keyword in the 10 topics\\n\",\n \"pprint(lda_model.print_topics())\\n\",\n \"doc_lda = lda_model[corpus]\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"#### Compute Model Perplexity and Coherence Score\\n\",\n \"\\n\",\n \"Let's calculate the baseline coherence score\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from gensim.models import CoherenceModel\\n\",\n \"\\n\",\n \"# Compute Coherence Score\\n\",\n \"coherence_model_lda = CoherenceModel(model=lda_model, texts=data_lemmatized, dictionary=id2word, coherence='c_v')\\n\",\n \"coherence_lda = coherence_model_lda.get_coherence()\\n\",\n \"print('Coherence Score: ', coherence_lda)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Step 6: Hyperparameter tuning\\n\",\n \"** **\\n\",\n \"First, let's differentiate between model hyperparameters and model parameters :\\n\",\n \"\\n\",\n \"- `Model hyperparameters` can be thought of as settings for a machine learning algorithm that are tuned by the data scientist before training. Examples would be the number of trees in the random forest, or in our case, number of topics K\\n\",\n \"\\n\",\n \"- `Model parameters` can be thought of as what the model learns during training, such as the weights for each word in a given topic.\\n\",\n \"\\n\",\n \"Now that we have the baseline coherence score for the default LDA model, let's perform a series of sensitivity tests to help determine the following model hyperparameters: \\n\",\n \"- Number of Topics (K)\\n\",\n \"- Dirichlet hyperparameter alpha: Document-Topic Density\\n\",\n \"- Dirichlet hyperparameter beta: Word-Topic Density\\n\",\n \"\\n\",\n \"We'll perform these tests in sequence, one parameter at a time by keeping others constant and run them over the two difference validation corpus sets. We'll use `C_v` as our choice of metric for performance comparison \"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# supporting function\\n\",\n \"def compute_coherence_values(corpus, dictionary, k, a, b):\\n\",\n \" \\n\",\n \" lda_model = gensim.models.LdaMulticore(corpus=corpus,\\n\",\n \" id2word=dictionary,\\n\",\n \" num_topics=k, \\n\",\n \" random_state=100,\\n\",\n \" chunksize=100,\\n\",\n \" passes=10,\\n\",\n \" alpha=a,\\n\",\n \" eta=b)\\n\",\n \" \\n\",\n \" coherence_model_lda = CoherenceModel(model=lda_model, texts=data_lemmatized, dictionary=id2word, coherence='c_v')\\n\",\n \" \\n\",\n \" return coherence_model_lda.get_coherence()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"Let's call the function, and iterate it over the range of topics, alpha, and beta parameter values\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import numpy as np\\n\",\n \"import tqdm\\n\",\n \"\\n\",\n \"grid = {}\\n\",\n \"grid['Validation_Set'] = {}\\n\",\n \"\\n\",\n \"# Topics range\\n\",\n \"min_topics = 2\\n\",\n \"max_topics = 11\\n\",\n \"step_size = 1\\n\",\n \"topics_range = range(min_topics, max_topics, step_size)\\n\",\n \"\\n\",\n \"# Alpha parameter\\n\",\n \"alpha = list(np.arange(0.01, 1, 0.3))\\n\",\n \"alpha.append('symmetric')\\n\",\n \"alpha.append('asymmetric')\\n\",\n \"\\n\",\n \"# Beta parameter\\n\",\n \"beta = list(np.arange(0.01, 1, 0.3))\\n\",\n \"beta.append('symmetric')\\n\",\n \"\\n\",\n \"# Validation sets\\n\",\n \"num_of_docs = len(corpus)\\n\",\n \"corpus_sets = [gensim.utils.ClippedCorpus(corpus, int(num_of_docs*0.75)), \\n\",\n \" corpus]\\n\",\n \"\\n\",\n \"corpus_title = ['75% Corpus', '100% Corpus']\\n\",\n \"\\n\",\n \"model_results = {'Validation_Set': [],\\n\",\n \" 'Topics': [],\\n\",\n \" 'Alpha': [],\\n\",\n \" 'Beta': [],\\n\",\n \" 'Coherence': []\\n\",\n \" }\\n\",\n \"\\n\",\n \"# Can take a long time to run\\n\",\n \"if 1 == 1:\\n\",\n \" pbar = tqdm.tqdm(total=(len(beta)*len(alpha)*len(topics_range)*len(corpus_title)))\\n\",\n \" \\n\",\n \" # iterate through validation corpuses\\n\",\n \" for i in range(len(corpus_sets)):\\n\",\n \" # iterate through number of topics\\n\",\n \" for k in topics_range:\\n\",\n \" # iterate through alpha values\\n\",\n \" for a in alpha:\\n\",\n \" # iterare through beta values\\n\",\n \" for b in beta:\\n\",\n \" # get the coherence score for the given parameters\\n\",\n \" cv = compute_coherence_values(corpus=corpus_sets[i], dictionary=id2word, \\n\",\n \" k=k, a=a, b=b)\\n\",\n \" # Save the model results\\n\",\n \" model_results['Validation_Set'].append(corpus_title[i])\\n\",\n \" model_results['Topics'].append(k)\\n\",\n \" model_results['Alpha'].append(a)\\n\",\n \" model_results['Beta'].append(b)\\n\",\n \" model_results['Coherence'].append(cv)\\n\",\n \" \\n\",\n \" pbar.update(1)\\n\",\n \" pd.DataFrame(model_results).to_csv('./results/lda_tuning_results.csv', index=False)\\n\",\n \" pbar.close()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Step 7: Final Model\\n\",\n \"** **\\n\",\n \"\\n\",\n \"Based on external evaluation (Code to be added from Excel based analysis), let's train the final model with parameters yielding highest coherence score\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"num_topics = 8\\n\",\n \"\\n\",\n \"lda_model = gensim.models.LdaMulticore(corpus=corpus,\\n\",\n \" id2word=id2word,\\n\",\n \" num_topics=num_topics, \\n\",\n \" random_state=100,\\n\",\n \" chunksize=100,\\n\",\n \" passes=10,\\n\",\n \" alpha=0.01,\\n\",\n \" eta=0.9)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from pprint import pprint\\n\",\n \"\\n\",\n \"# Print the Keyword in the 10 topics\\n\",\n \"pprint(lda_model.print_topics())\\n\",\n \"doc_lda = lda_model[corpus]\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Step 8: Visualize Results\\n\",\n \"** **\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import pyLDAvis.gensim_models as gensimvis\\n\",\n \"import pickle \\n\",\n \"import pyLDAvis\\n\",\n \"\\n\",\n \"# Visualize the topics\\n\",\n \"pyLDAvis.enable_notebook()\\n\",\n \"\\n\",\n \"LDAvis_data_filepath = os.path.join('./results/ldavis_tuned_'+str(num_topics))\\n\",\n \"\\n\",\n \"# # this is a bit time consuming - make the if statement True\\n\",\n \"# # if you want to execute visualization prep yourself\\n\",\n \"if 1 == 1:\\n\",\n \" LDAvis_prepared = gensimvis.prepare(lda_model, corpus, id2word)\\n\",\n \" with open(LDAvis_data_filepath, 'wb') as f:\\n\",\n \" pickle.dump(LDAvis_prepared, f)\\n\",\n \"\\n\",\n \"# load the pre-prepared pyLDAvis data from disk\\n\",\n \"with open(LDAvis_data_filepath, 'rb') as f:\\n\",\n \" LDAvis_prepared = pickle.load(f)\\n\",\n \"\\n\",\n \"pyLDAvis.save_html(LDAvis_prepared, './results/ldavis_tuned_'+ str(num_topics) +'.html')\\n\",\n \"\\n\",\n \"LDAvis_prepared\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Closing Notes\\n\",\n \"\\n\",\n \"We started with understanding why evaluating the topic model is essential. Next, we reviewed existing methods and scratched the surface of topic coherence, along with the available coherence measures. Then we built a default LDA model using Gensim implementation to establish the baseline coherence score and reviewed practical ways to optimize the LDA hyperparameters.\\n\",\n \"\\n\",\n \"Hopefully, this article has managed to shed light on the underlying topic evaluation strategies, and intuitions behind it.\\n\",\n \"\\n\",\n \"** **\\n\",\n \"#### References:\\n\",\n \"1. http://qpleple.com/perplexity-to-evaluate-topic-models/\\n\",\n \"2. https://www.amazon.com/Machine-Learning-Probabilistic-Perspective-Computation/dp/0262018020\\n\",\n \"3. https://papers.nips.cc/paper/3700-reading-tea-leaves-how-humans-interpret-topic-models.pdf\\n\",\n \"4. https://github.com/mattilyra/pydataberlin-2017/blob/master/notebook/EvaluatingUnsupervisedModels.ipynb\\n\",\n \"5. https://www.machinelearningplus.com/nlp/topic-modeling-gensim-python/\\n\",\n \"6. http://svn.aksw.org/papers/2015/WSDM_Topic_Evaluation/public.pdf\\n\",\n \"7. http://palmetto.aksw.org/palmetto-webapp/\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.9.16\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n"
},
{
"path": "natural-language-processing/topic-modeling/Introduction to Topic Modeling.ipynb",
"content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Introduction\\n\",\n \"##### How to get started with topic modeling using LDA in Python\\n\",\n \"** **\\n\",\n \"Topic Models, in a nutshell, are a type of statistical language models used for uncovering hidden structure in a collection of texts. In a practical and more intuitively, you can think of it as a task of:\\n\",\n \"\\n\",\n \"- **Dimensionality Reduction**, where rather than representing a text T in its feature space as {Word_i: count(Word_i, T) for Word_i in Vocabulary}, you can represent it in a topic space as {Topic_i: Weight(Topic_i, T) for Topic_i in Topics}\\n\",\n \"- **Unsupervised Learning**, where it can be compared to clustering, as in the case of clustering, the number of topics, like the number of clusters, is an output parameter. By doing topic modeling, we build clusters of words rather than clusters of texts. A text is thus a mixture of all the topics, each having a specific weight\\n\",\n \"- **Tagging**, abstract “topics” that occur in a collection of documents that best represents the information in them.\\n\",\n \"\\n\",\n \"There are several existing algorithms you can use to perform the topic modeling. The most common of it are, Latent Semantic Analysis (LSA/LSI), Probabilistic Latent Semantic Analysis (pLSA), and Latent Dirichlet Allocation (LDA)\\n\",\n \"\\n\",\n \"In this tutorial, we’ll take a closer look at LDA, and implement our first topic model using the sklearn implementation in python 2.7\\n\",\n \"\\n\",\n \"### Theoretical Overview\\n\",\n \"LDA is a generative probabilistic model that assumes each topic is a mixture over an underlying set of words, and each document is a mixture of over a set of topic probabilities.\\n\",\n \"\\n\",\n \"\\n\",\n \"\\n\",\n \"We can describe the generative process of LDA as, given the M number of documents, N number of words, and prior K number of topics, the model trains to output:\\n\",\n \"\\n\",\n \"- `psi`, the distribution of words for each topic K\\n\",\n \"- `phi`, the distribution of topics for each document i\\n\",\n \"\\n\",\n \"#### Parameters of LDA\\n\",\n \"\\n\",\n \"- `Alpha parameter` is Dirichlet prior concentration parameter that represents document-topic density — with a higher alpha, documents are assumed to be made up of more topics and result in more specific topic distribution per document.\\n\",\n \"- `Beta parameter` is the same prior concentration parameter that represents topic-word density — with high beta, topics are assumed to made of up most of the words and result in a more specific word distribution per topic.\\n\",\n \"\\n\",\n \"**To read more: https://towardsdatascience.com/end-to-end-topic-modeling-in-python-latent-dirichlet-allocation-lda-35ce4ed6b3e0**\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"### LDA Implementation\\n\",\n \"\\n\",\n \"1. [Loading data](#load_data)\\n\",\n \"2. [Data cleaning](#clean_data)\\n\",\n \"3. [Exploratory analysis](#eda)\\n\",\n \"4. [Prepare data for LDA analysis](#data_preparation)\\n\",\n \"5. [LDA model training](#train_model)\\n\",\n \"6. [Analyzing LDA model results](#results)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"For this tutorial, we’ll use the dataset of papers published in NeurIPS (NIPS) conference which is one of the most prestigious yearly events in the machine learning community. The CSV data file contains information on the different NeurIPS papers that were published from 1987 until 2016 (29 years!). These papers discuss a wide variety of topics in machine learning, from neural networks to optimization methods, and many more.\\n\",\n \"\\n\",\n \"\\n\",\n \"\\n\",\n \"Let’s start by looking at the content of the file\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Step 1: Loading Data \\n\",\n \"** **\\n\",\n \"For this tutorial, we’ll use the dataset of papers published in NeurIPS (NIPS) conference which is one of the most prestigious yearly events in the machine learning community. The CSV data file contains information on the different NeurIPS papers that were published from 1987 until 2016 (29 years!). These papers discuss a wide variety of topics in machine learning, from neural networks to optimization methods, and many more.\\n\",\n \"\\n\",\n \"Let’s start by looking at the content of the file\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import zipfile\\n\",\n \"import pandas as pd\\n\",\n \"import os\\n\",\n \"\\n\",\n \"# Open the zip file\\n\",\n \"with zipfile.ZipFile(\\\"./data/NIPS Papers.zip\\\", \\\"r\\\") as zip_ref:\\n\",\n \" # Extract the file to a temporary directory\\n\",\n \" zip_ref.extractall(\\\"temp\\\")\\n\",\n \"\\n\",\n \"# Read the CSV file into a pandas DataFrame\\n\",\n \"papers = pd.read_csv(\\\"temp/NIPS Papers/papers.csv\\\")\\n\",\n \"\\n\",\n \"# Print head\\n\",\n \"papers.head()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Step 2: Data Cleaning \\n\",\n \"** **\\n\",\n \"\\n\",\n \"Since the goal of this analysis is to perform topic modeling, let's focus only on the text data from each paper, and drop other metadata columns. Also, for the demonstration, we'll only look at 100 papers\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Remove the columns\\n\",\n \"papers = papers.drop(columns=['id', 'event_type', 'pdf_name'], axis=1).sample(100)\\n\",\n \"\\n\",\n \"# Print out the first rows of papers\\n\",\n \"papers.head()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"##### Remove punctuation/lower casing\\n\",\n \"\\n\",\n \"Next, let’s perform a simple preprocessing on the content of `paper_text` column to make them more amenable for analysis, and reliable results. To do that, we’ll use a regular expression to remove any punctuation, and then lowercase the text\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Load the regular expression library\\n\",\n \"import re\\n\",\n \"\\n\",\n \"# Remove punctuation\\n\",\n \"papers['paper_text_processed'] = \\\\\\n\",\n \"papers['paper_text'].map(lambda x: re.sub('[,\\\\.!?]', '', x))\\n\",\n \"\\n\",\n \"# Convert the titles to lowercase\\n\",\n \"papers['paper_text_processed'] = \\\\\\n\",\n \"papers['paper_text_processed'].map(lambda x: x.lower())\\n\",\n \"\\n\",\n \"# Print out the first rows of papers\\n\",\n \"papers['paper_text_processed'].head()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Step 3: Exploratory Analysis \\n\",\n \"** **\\n\",\n \"\\n\",\n \"To verify whether the preprocessing, we’ll make a simple word cloud using the `wordcloud` package to get a visual representation of most common words. It is key to understanding the data and ensuring we are on the right track, and if any more preprocessing is necessary before training the model.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Import the wordcloud library\\n\",\n \"from wordcloud import WordCloud\\n\",\n \"\\n\",\n \"# Join the different processed titles together.\\n\",\n \"long_string = ','.join(list(papers['paper_text_processed'].values))\\n\",\n \"\\n\",\n \"# Create a WordCloud object\\n\",\n \"wordcloud = WordCloud(background_color=\\\"white\\\", max_words=1000, contour_width=3, contour_color='steelblue')\\n\",\n \"\\n\",\n \"# Generate a word cloud\\n\",\n \"wordcloud.generate(long_string)\\n\",\n \"\\n\",\n \"# Visualize the word cloud\\n\",\n \"wordcloud.to_image()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Step 4: Prepare text for LDA analysis \\n\",\n \"** **\\n\",\n \"\\n\",\n \"Next, let’s work to transform the textual data in a format that will serve as an input for training LDA model. We start by tokenizing the text and removing stopwords. Next, we convert the tokenized object into a corpus and dictionary.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import gensim\\n\",\n \"from gensim.utils import simple_preprocess\\n\",\n \"import nltk\\n\",\n \"nltk.download('stopwords')\\n\",\n \"from nltk.corpus import stopwords\\n\",\n \"\\n\",\n \"stop_words = stopwords.words('english')\\n\",\n \"stop_words.extend(['from', 'subject', 're', 'edu', 'use'])\\n\",\n \"\\n\",\n \"def sent_to_words(sentences):\\n\",\n \" for sentence in sentences:\\n\",\n \" # deacc=True removes punctuations\\n\",\n \" yield(gensim.utils.simple_preprocess(str(sentence), deacc=True))\\n\",\n \"\\n\",\n \"def remove_stopwords(texts):\\n\",\n \" return [[word for word in simple_preprocess(str(doc)) \\n\",\n \" if word not in stop_words] for doc in texts]\\n\",\n \"\\n\",\n \"\\n\",\n \"data = papers.paper_text_processed.values.tolist()\\n\",\n \"data_words = list(sent_to_words(data))\\n\",\n \"\\n\",\n \"# remove stop words\\n\",\n \"data_words = remove_stopwords(data_words)\\n\",\n \"\\n\",\n \"print(data_words[:1][0][:30])\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import gensim.corpora as corpora\\n\",\n \"\\n\",\n \"# Create Dictionary\\n\",\n \"id2word = corpora.Dictionary(data_words)\\n\",\n \"\\n\",\n \"# Create Corpus\\n\",\n \"texts = data_words\\n\",\n \"\\n\",\n \"# Term Document Frequency\\n\",\n \"corpus = [id2word.doc2bow(text) for text in texts]\\n\",\n \"\\n\",\n \"# View\\n\",\n \"print(corpus[:1][0][:30])\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Step 5: LDA model tranining \\n\",\n \"** **\\n\",\n \"\\n\",\n \"To keep things simple, we'll keep all the parameters to default except for inputting the number of topics. For this tutorial, we will build a model with 10 topics where each topic is a combination of keywords, and each keyword contributes a certain weightage to the topic.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from pprint import pprint\\n\",\n \"\\n\",\n \"# number of topics\\n\",\n \"num_topics = 10\\n\",\n \"\\n\",\n \"# Build LDA model\\n\",\n \"lda_model = gensim.models.LdaMulticore(corpus=corpus,\\n\",\n \" id2word=id2word,\\n\",\n \" num_topics=num_topics)\\n\",\n \"\\n\",\n \"# Print the Keyword in the 10 topics\\n\",\n \"pprint(lda_model.print_topics())\\n\",\n \"doc_lda = lda_model[corpus]\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Step 6: Analyzing our LDA model \\n\",\n \"** **\\n\",\n \"\\n\",\n \"Now that we have a trained model let’s visualize the topics for interpretability. To do so, we’ll use a popular visualization package, pyLDAvis which is designed to help interactively with:\\n\",\n \"\\n\",\n \"1. Better understanding and interpreting individual topics, and\\n\",\n \"2. Better understanding the relationships between the topics.\\n\",\n \"\\n\",\n \"For (1), you can manually select each topic to view its top most frequent and/or “relevant” terms, using different values of the λ parameter. This can help when you’re trying to assign a human interpretable name or “meaning” to each topic.\\n\",\n \"\\n\",\n \"For (2), exploring the Intertopic Distance Plot can help you learn about how topics relate to each other, including potential higher-level structure between groups of topics.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import pyLDAvis.gensim_models as gensimvis\\n\",\n \"import pickle \\n\",\n \"import pyLDAvis\\n\",\n \"\\n\",\n \"# Visualize the topics\\n\",\n \"pyLDAvis.enable_notebook()\\n\",\n \"\\n\",\n \"LDAvis_data_filepath = os.path.join('./results/ldavis_prepared_'+str(num_topics))\\n\",\n \"\\n\",\n \"# # this is a bit time consuming - make the if statement True\\n\",\n \"# # if you want to execute visualization prep yourself\\n\",\n \"if 1 == 1:\\n\",\n \" LDAvis_prepared = gensimvis.prepare(lda_model, corpus, id2word)\\n\",\n \" with open(LDAvis_data_filepath, 'wb') as f:\\n\",\n \" pickle.dump(LDAvis_prepared, f)\\n\",\n \"\\n\",\n \"# load the pre-prepared pyLDAvis data from disk\\n\",\n \"with open(LDAvis_data_filepath, 'rb') as f:\\n\",\n \" LDAvis_prepared = pickle.load(f)\\n\",\n \"\\n\",\n \"pyLDAvis.save_html(LDAvis_prepared, './results/ldavis_prepared_'+ str(num_topics) +'.html')\\n\",\n \"\\n\",\n \"LDAvis_prepared\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Closing Notes\\n\",\n \"Machine learning has become increasingly popular over the past decade, and recent advances in computational availability have led to exponential growth to people looking for ways how new methods can be incorporated to advance the field of Natural Language Processing.\\n\",\n \"\\n\",\n \"Often, we treat topic models as black-box algorithms, but hopefully, this article addressed to shed light on the underlying math, and intuitions behind it, and high-level code to get you started with any textual data.\\n\",\n \"\\n\",\n \"In the next article, we’ll go one step deeper into understanding how you can evaluate the performance of topic models, tune its hyper-parameters to get more intuitive and reliable results.\\n\",\n \"\\n\",\n \"** **\\n\",\n \"#### References:\\n\",\n \"1. Topic model — Wikipedia. https://en.wikipedia.org/wiki/Topic_model\\n\",\n \"2. Distributed Strategies for Topic Modeling. https://www.ideals.illinois.edu/bitstream/handle/2142/46405/ParallelTopicModels.pdf?sequence=2&isAllowed=y\\n\",\n \"3. Topic Mapping — Software — Resources — Amaral Lab. https://amaral.northwestern.edu/resources/software/topic-mapping\\n\",\n \"4. A Survey of Topic Modeling in Text Mining. https://thesai.org/Downloads/Volume6No1/Paper_21-A_Survey_of_Topic_Modeling_in_Text_Mining.pdf\\n\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.9.16\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n"
},
{
"path": "natural-language-processing/topic-modeling/pyproject.toml",
"content": "[tool.poetry]\nname = \"topic-modeling\"\nversion = \"0.1.0\"\ndescription = \"\"\nauthors = [\"Shashank Kapadia \"]\nreadme = \"README.md\"\npackages = [{include = \"topic_modeling\"}]\n\n[tool.poetry.dependencies]\npython = \">=3.9,<3.10\"\njupyterlab = \"^3.5.2\"\npandas = \"^1.5.2\"\ngensim = \"^4.3.0\"\nnltk = \"^3.8\"\nspacy = \"^3.4.4\"\ntqdm = \"^4.64.1\"\npyldavis = \"^3.3.1\"\nwordcloud = \"^1.8.2.2\"\n\n\n[build-system]\nrequires = [\"poetry-core\"]\nbuild-backend = \"poetry.core.masonry.api\"\n"
},
{
"path": "natural-language-processing/topic-modeling/results/lda_tuning_results.csv",
"content": "Validation_Set,Topics,Alpha,Beta,Coherence\n75% Corpus,2,0.01,0.01,0.25978135607988706\n75% Corpus,2,0.01,0.31,0.2668476722607104\n75% Corpus,2,0.01,0.61,0.2776409400108895\n75% Corpus,2,0.01,0.9099999999999999,0.2716233211418745\n75% Corpus,2,0.01,symmetric,0.27332996032921053\n75% Corpus,2,0.31,0.01,0.25978135607988706\n75% Corpus,2,0.31,0.31,0.2668476722607104\n75% Corpus,2,0.31,0.61,0.2776409400108895\n75% Corpus,2,0.31,0.9099999999999999,0.2716233211418745\n75% Corpus,2,0.31,symmetric,0.27332996032921053\n75% Corpus,2,0.61,0.01,0.25978135607988706\n75% Corpus,2,0.61,0.31,0.2668476722607104\n75% Corpus,2,0.61,0.61,0.2776409400108895\n75% Corpus,2,0.61,0.9099999999999999,0.2716233211418745\n75% Corpus,2,0.61,symmetric,0.27332996032921053\n75% Corpus,2,0.9099999999999999,0.01,0.25978135607988706\n75% Corpus,2,0.9099999999999999,0.31,0.2668476722607104\n75% Corpus,2,0.9099999999999999,0.61,0.27764094001088946\n75% Corpus,2,0.9099999999999999,0.9099999999999999,0.2766757371950356\n75% Corpus,2,0.9099999999999999,symmetric,0.27332996032921053\n75% Corpus,2,symmetric,0.01,0.25978135607988706\n75% Corpus,2,symmetric,0.31,0.2668476722607104\n75% Corpus,2,symmetric,0.61,0.2776409400108895\n75% Corpus,2,symmetric,0.9099999999999999,0.2716233211418745\n75% Corpus,2,symmetric,symmetric,0.27332996032921053\n75% Corpus,2,asymmetric,0.01,0.25978135607988706\n75% Corpus,2,asymmetric,0.31,0.2668476722607104\n75% Corpus,2,asymmetric,0.61,0.27764094001088957\n75% Corpus,2,asymmetric,0.9099999999999999,0.27162332114187443\n75% Corpus,2,asymmetric,symmetric,0.27332996032921053\n75% Corpus,3,0.01,0.01,0.27839627006352724\n75% Corpus,3,0.01,0.31,0.2714832672445246\n75% Corpus,3,0.01,0.61,0.27129619317484943\n75% Corpus,3,0.01,0.9099999999999999,0.26967265698428594\n75% Corpus,3,0.01,symmetric,0.2710418286483521\n75% Corpus,3,0.31,0.01,0.27839627006352724\n75% Corpus,3,0.31,0.31,0.2714832672445246\n75% Corpus,3,0.31,0.61,0.27129619317484943\n75% Corpus,3,0.31,0.9099999999999999,0.26967265698428594\n75% Corpus,3,0.31,symmetric,0.2741049024792329\n75% Corpus,3,0.61,0.01,0.27839627006352724\n75% Corpus,3,0.61,0.31,0.27351453271228904\n75% Corpus,3,0.61,0.61,0.2730873971333492\n75% Corpus,3,0.61,0.9099999999999999,0.26967265698428594\n75% Corpus,3,0.61,symmetric,0.27410490247923297\n75% Corpus,3,0.9099999999999999,0.01,0.27839627006352724\n75% Corpus,3,0.9099999999999999,0.31,0.27351453271228904\n75% Corpus,3,0.9099999999999999,0.61,0.2766291342273802\n75% Corpus,3,0.9099999999999999,0.9099999999999999,0.26645454631910853\n75% Corpus,3,0.9099999999999999,symmetric,0.2714832672445246\n75% Corpus,3,symmetric,0.01,0.27839627006352724\n75% Corpus,3,symmetric,0.31,0.2714832672445246\n75% Corpus,3,symmetric,0.61,0.27129619317484943\n75% Corpus,3,symmetric,0.9099999999999999,0.26967265698428594\n75% Corpus,3,symmetric,symmetric,0.27410490247923297\n75% Corpus,3,asymmetric,0.01,0.27839627006352724\n75% Corpus,3,asymmetric,0.31,0.2761361679469974\n75% Corpus,3,asymmetric,0.61,0.27335893705876174\n75% Corpus,3,asymmetric,0.9099999999999999,0.2679744346749598\n75% Corpus,3,asymmetric,symmetric,0.27410490247923297\n75% Corpus,4,0.01,0.01,0.2668910500383095\n75% Corpus,4,0.01,0.31,0.2838954443896148\n75% Corpus,4,0.01,0.61,0.2865926544014952\n75% Corpus,4,0.01,0.9099999999999999,0.27003887281216177\n75% Corpus,4,0.01,symmetric,0.2796510251967742\n75% Corpus,4,0.31,0.01,0.2668910500383095\n75% Corpus,4,0.31,0.31,0.2838954443896148\n75% Corpus,4,0.31,0.61,0.2865926544014952\n75% Corpus,4,0.31,0.9099999999999999,0.2725567640010955\n75% Corpus,4,0.31,symmetric,0.2806724093952981\n75% Corpus,4,0.61,0.01,0.2668910500383095\n75% Corpus,4,0.61,0.31,0.28505646918509386\n75% Corpus,4,0.61,0.61,0.2892439806686463\n75% Corpus,4,0.61,0.9099999999999999,0.2725567640010955\n75% Corpus,4,0.61,symmetric,0.2806724093952981\n75% Corpus,4,0.9099999999999999,0.01,0.2685576018605267\n75% Corpus,4,0.9099999999999999,0.31,0.2860778533836178\n75% Corpus,4,0.9099999999999999,0.61,0.2892439806686463\n75% Corpus,4,0.9099999999999999,0.9099999999999999,0.2725567640010955\n75% Corpus,4,0.9099999999999999,symmetric,0.27955372987186383\n75% Corpus,4,symmetric,0.01,0.2668910500383095\n75% Corpus,4,symmetric,0.31,0.2838954443896148\n75% Corpus,4,symmetric,0.61,0.2865926544014952\n75% Corpus,4,symmetric,0.9099999999999999,0.2725567640010955\n75% Corpus,4,symmetric,symmetric,0.2806724093952981\n75% Corpus,4,asymmetric,0.01,0.26812725545164134\n75% Corpus,4,asymmetric,0.31,0.2838954443896148\n75% Corpus,4,asymmetric,0.61,0.29144326581678964\n75% Corpus,4,asymmetric,0.9099999999999999,0.2721780507981776\n75% Corpus,4,asymmetric,symmetric,0.27675604929105146\n75% Corpus,5,0.01,0.01,0.2921917497600287\n75% Corpus,5,0.01,0.31,0.27552154453031585\n75% Corpus,5,0.01,0.61,0.269308750481684\n75% Corpus,5,0.01,0.9099999999999999,0.2709825760478901\n75% Corpus,5,0.01,symmetric,0.2748624070876895\n75% Corpus,5,0.31,0.01,0.2921917497600287\n75% Corpus,5,0.31,0.31,0.2748594189290374\n75% Corpus,5,0.31,0.61,0.27297615024851607\n75% Corpus,5,0.31,0.9099999999999999,0.27487775415711974\n75% Corpus,5,0.31,symmetric,0.27717279279720247\n75% Corpus,5,0.61,0.01,0.29578845568370865\n75% Corpus,5,0.61,0.31,0.27395015677623524\n75% Corpus,5,0.61,0.61,0.27297615024851607\n75% Corpus,5,0.61,0.9099999999999999,0.2740301275419981\n75% Corpus,5,0.61,symmetric,0.2771727927972025\n75% Corpus,5,0.9099999999999999,0.01,0.2957884556837086\n75% Corpus,5,0.9099999999999999,0.31,0.27395015677623524\n75% Corpus,5,0.9099999999999999,0.61,0.28404895953113735\n75% Corpus,5,0.9099999999999999,0.9099999999999999,0.277640513968081\n75% Corpus,5,0.9099999999999999,symmetric,0.27934536161569634\n75% Corpus,5,symmetric,0.01,0.2921917497600287\n75% Corpus,5,symmetric,0.31,0.2748594189290374\n75% Corpus,5,symmetric,0.61,0.2689322892718954\n75% Corpus,5,symmetric,0.9099999999999999,0.27487775415711974\n75% Corpus,5,symmetric,symmetric,0.2748624070876895\n75% Corpus,5,asymmetric,0.01,0.29578845568370865\n75% Corpus,5,asymmetric,0.31,0.2757452278352188\n75% Corpus,5,asymmetric,0.61,0.26239407325290853\n75% Corpus,5,asymmetric,0.9099999999999999,0.2686874093590562\n75% Corpus,5,asymmetric,symmetric,0.2770349759061833\n75% Corpus,6,0.01,0.01,0.30284473516050225\n75% Corpus,6,0.01,0.31,0.30249950909918444\n75% Corpus,6,0.01,0.61,0.3121782769555992\n75% Corpus,6,0.01,0.9099999999999999,0.35411824844325124\n75% Corpus,6,0.01,symmetric,0.30436985837998276\n75% Corpus,6,0.31,0.01,0.301390971353645\n75% Corpus,6,0.31,0.31,0.3024063840590527\n75% Corpus,6,0.31,0.61,0.31042451210080446\n75% Corpus,6,0.31,0.9099999999999999,0.35411824844325124\n75% Corpus,6,0.31,symmetric,0.30491985962114915\n75% Corpus,6,0.61,0.01,0.30064456740988804\n75% Corpus,6,0.61,0.31,0.30715638376430365\n75% Corpus,6,0.61,0.61,0.31042451210080446\n75% Corpus,6,0.61,0.9099999999999999,0.35418725691606756\n75% Corpus,6,0.61,symmetric,0.3049198596211492\n75% Corpus,6,0.9099999999999999,0.01,0.3013337842863722\n75% Corpus,6,0.9099999999999999,0.31,0.3024679491620339\n75% Corpus,6,0.9099999999999999,0.61,0.31042451210080446\n75% Corpus,6,0.9099999999999999,0.9099999999999999,0.3541872569160675\n75% Corpus,6,0.9099999999999999,symmetric,0.30491985962114915\n75% Corpus,6,symmetric,0.01,0.30065930794388435\n75% Corpus,6,symmetric,0.31,0.3024063840590527\n75% Corpus,6,symmetric,0.61,0.31042451210080446\n75% Corpus,6,symmetric,0.9099999999999999,0.35411824844325124\n75% Corpus,6,symmetric,symmetric,0.30436985837998276\n75% Corpus,6,asymmetric,0.01,0.2967291360253754\n75% Corpus,6,asymmetric,0.31,0.302076431936051\n75% Corpus,6,asymmetric,0.61,0.30847841491074096\n75% Corpus,6,asymmetric,0.9099999999999999,0.35477042247541973\n75% Corpus,6,asymmetric,symmetric,0.3073932906534706\n75% Corpus,7,0.01,0.01,0.2811852190841381\n75% Corpus,7,0.01,0.31,0.2920220131136521\n75% Corpus,7,0.01,0.61,0.28852068336946435\n75% Corpus,7,0.01,0.9099999999999999,0.316288019095773\n75% Corpus,7,0.01,symmetric,0.2726152139083012\n75% Corpus,7,0.31,0.01,0.2788837794017795\n75% Corpus,7,0.31,0.31,0.2909961139203829\n75% Corpus,7,0.31,0.61,0.28852068336946435\n75% Corpus,7,0.31,0.9099999999999999,0.3129742202428836\n75% Corpus,7,0.31,symmetric,0.2777182230257198\n75% Corpus,7,0.61,0.01,0.28327368320570007\n75% Corpus,7,0.61,0.31,0.28935990578371756\n75% Corpus,7,0.61,0.61,0.28852068336946435\n75% Corpus,7,0.61,0.9099999999999999,0.31405801487937557\n75% Corpus,7,0.61,symmetric,0.2759279034494479\n75% Corpus,7,0.9099999999999999,0.01,0.28327368320570007\n75% Corpus,7,0.9099999999999999,0.31,0.28722285177113666\n75% Corpus,7,0.9099999999999999,0.61,0.28966178578989427\n75% Corpus,7,0.9099999999999999,0.9099999999999999,0.31405801487937557\n75% Corpus,7,0.9099999999999999,symmetric,0.27734639862412164\n75% Corpus,7,symmetric,0.01,0.2788837794017795\n75% Corpus,7,symmetric,0.31,0.2909961139203829\n75% Corpus,7,symmetric,0.61,0.28852068336946435\n75% Corpus,7,symmetric,0.9099999999999999,0.31628801909577303\n75% Corpus,7,symmetric,symmetric,0.2756697323965734\n75% Corpus,7,asymmetric,0.01,0.2853642185426731\n75% Corpus,7,asymmetric,0.31,0.2907271450321393\n75% Corpus,7,asymmetric,0.61,0.29361676225955274\n75% Corpus,7,asymmetric,0.9099999999999999,0.2882054578232691\n75% Corpus,7,asymmetric,symmetric,0.2755194552200007\n75% Corpus,8,0.01,0.01,0.2808202028308223\n75% Corpus,8,0.01,0.31,0.29736349070880286\n75% Corpus,8,0.01,0.61,0.2981078394239831\n75% Corpus,8,0.01,0.9099999999999999,0.33233988889385435\n75% Corpus,8,0.01,symmetric,0.28156849418007524\n75% Corpus,8,0.31,0.01,0.28097129035113944\n75% Corpus,8,0.31,0.31,0.29941241997917767\n75% Corpus,8,0.31,0.61,0.2960462608237895\n75% Corpus,8,0.31,0.9099999999999999,0.327937391019251\n75% Corpus,8,0.31,symmetric,0.28207106831879114\n75% Corpus,8,0.61,0.01,0.2809772515659067\n75% Corpus,8,0.61,0.31,0.2993455174990556\n75% Corpus,8,0.61,0.61,0.2946710626297879\n75% Corpus,8,0.61,0.9099999999999999,0.32359289177614337\n75% Corpus,8,0.61,symmetric,0.28138865001186986\n75% Corpus,8,0.9099999999999999,0.01,0.27976485139485124\n75% Corpus,8,0.9099999999999999,0.31,0.2956217410776679\n75% Corpus,8,0.9099999999999999,0.61,0.28864637585951036\n75% Corpus,8,0.9099999999999999,0.9099999999999999,0.3282635584644864\n75% Corpus,8,0.9099999999999999,symmetric,0.28148737285248604\n75% Corpus,8,symmetric,0.01,0.2808202028308223\n75% Corpus,8,symmetric,0.31,0.29736349070880286\n75% Corpus,8,symmetric,0.61,0.297949677831361\n75% Corpus,8,symmetric,0.9099999999999999,0.3265139021008282\n75% Corpus,8,symmetric,symmetric,0.2817931736933517\n75% Corpus,8,asymmetric,0.01,0.27839412750067105\n75% Corpus,8,asymmetric,0.31,0.296460251572274\n75% Corpus,8,asymmetric,0.61,0.297148250775704\n75% Corpus,8,asymmetric,0.9099999999999999,0.32649247107917795\n75% Corpus,8,asymmetric,symmetric,0.2785059288493066\n75% Corpus,9,0.01,0.01,0.30218586423505117\n75% Corpus,9,0.01,0.31,0.3020235109956903\n75% Corpus,9,0.01,0.61,0.4185691849535152\n75% Corpus,9,0.01,0.9099999999999999,0.30527967183321003\n75% Corpus,9,0.01,symmetric,0.3060635646496002\n75% Corpus,9,0.31,0.01,0.30068508513688286\n75% Corpus,9,0.31,0.31,0.3021506007873651\n75% Corpus,9,0.31,0.61,0.40687439307467294\n75% Corpus,9,0.31,0.9099999999999999,0.3073421317714173\n75% Corpus,9,0.31,symmetric,0.3074145374333964\n75% Corpus,9,0.61,0.01,0.3008820210254886\n75% Corpus,9,0.61,0.31,0.30275011169684873\n75% Corpus,9,0.61,0.61,0.41015152208333183\n75% Corpus,9,0.61,0.9099999999999999,0.30255055512967155\n75% Corpus,9,0.61,symmetric,0.30715762371551203\n75% Corpus,9,0.9099999999999999,0.01,0.3022010046079812\n75% Corpus,9,0.9099999999999999,0.31,0.30477890773318156\n75% Corpus,9,0.9099999999999999,0.61,0.40433106836971433\n75% Corpus,9,0.9099999999999999,0.9099999999999999,0.307644042466123\n75% Corpus,9,0.9099999999999999,symmetric,0.30605307577755286\n75% Corpus,9,symmetric,0.01,0.3013158303146456\n75% Corpus,9,symmetric,0.31,0.304290454027783\n75% Corpus,9,symmetric,0.61,0.4140952699790183\n75% Corpus,9,symmetric,0.9099999999999999,0.30527967183321\n75% Corpus,9,symmetric,symmetric,0.3046097002166359\n75% Corpus,9,asymmetric,0.01,0.30129966466596664\n75% Corpus,9,asymmetric,0.31,0.30429095996678135\n75% Corpus,9,asymmetric,0.61,0.41788064240742073\n75% Corpus,9,asymmetric,0.9099999999999999,0.3025975330524115\n75% Corpus,9,asymmetric,symmetric,0.30669601144755215\n75% Corpus,10,0.01,0.01,0.29670164537919275\n75% Corpus,10,0.01,0.31,0.3047813348299382\n75% Corpus,10,0.01,0.61,0.31606878233356517\n75% Corpus,10,0.01,0.9099999999999999,0.3377296641607972\n75% Corpus,10,0.01,symmetric,0.29319673176987077\n75% Corpus,10,0.31,0.01,0.29805457831216586\n75% Corpus,10,0.31,0.31,0.30369633038354127\n75% Corpus,10,0.31,0.61,0.30998451194916454\n75% Corpus,10,0.31,0.9099999999999999,0.34268251293470897\n75% Corpus,10,0.31,symmetric,0.29319673176987077\n75% Corpus,10,0.61,0.01,0.29386231031811644\n75% Corpus,10,0.61,0.31,0.29541931052055126\n75% Corpus,10,0.61,0.61,0.309970817294693\n75% Corpus,10,0.61,0.9099999999999999,0.3453018084648031\n75% Corpus,10,0.61,symmetric,0.29319673176987077\n75% Corpus,10,0.9099999999999999,0.01,0.2945143416112511\n75% Corpus,10,0.9099999999999999,0.31,0.34045282991315623\n75% Corpus,10,0.9099999999999999,0.61,0.3126412466766651\n75% Corpus,10,0.9099999999999999,0.9099999999999999,0.3435699386195771\n75% Corpus,10,0.9099999999999999,symmetric,0.29319673176987077\n75% Corpus,10,symmetric,0.01,0.29805457831216586\n75% Corpus,10,symmetric,0.31,0.30478133482993813\n75% Corpus,10,symmetric,0.61,0.31606878233356517\n75% Corpus,10,symmetric,0.9099999999999999,0.33772966416079714\n75% Corpus,10,symmetric,symmetric,0.29319673176987077\n75% Corpus,10,asymmetric,0.01,0.29815109815940033\n75% Corpus,10,asymmetric,0.31,0.35146410607200645\n75% Corpus,10,asymmetric,0.61,0.3196615767839669\n75% Corpus,10,asymmetric,0.9099999999999999,0.34012655333540415\n75% Corpus,10,asymmetric,symmetric,0.2930959026114379\n100% Corpus,2,0.01,0.01,0.26049496936796\n100% Corpus,2,0.01,0.31,0.2631487596403128\n100% Corpus,2,0.01,0.61,0.2670430149871908\n100% Corpus,2,0.01,0.9099999999999999,0.2784737874245884\n100% Corpus,2,0.01,symmetric,0.2659027879879802\n100% Corpus,2,0.31,0.01,0.26049496936796\n100% Corpus,2,0.31,0.31,0.2631487596403128\n100% Corpus,2,0.31,0.61,0.2670430149871908\n100% Corpus,2,0.31,0.9099999999999999,0.2784737874245884\n100% Corpus,2,0.31,symmetric,0.2659027879879802\n100% Corpus,2,0.61,0.01,0.26049496936796007\n100% Corpus,2,0.61,0.31,0.2631487596403128\n100% Corpus,2,0.61,0.61,0.2670430149871908\n100% Corpus,2,0.61,0.9099999999999999,0.27847378742458834\n100% Corpus,2,0.61,symmetric,0.2659027879879803\n100% Corpus,2,0.9099999999999999,0.01,0.26049496936796007\n100% Corpus,2,0.9099999999999999,0.31,0.2631487596403128\n100% Corpus,2,0.9099999999999999,0.61,0.2670430149871908\n100% Corpus,2,0.9099999999999999,0.9099999999999999,0.27847378742458834\n100% Corpus,2,0.9099999999999999,symmetric,0.2659027879879803\n100% Corpus,2,symmetric,0.01,0.26049496936796007\n100% Corpus,2,symmetric,0.31,0.2631487596403128\n100% Corpus,2,symmetric,0.61,0.2670430149871908\n100% Corpus,2,symmetric,0.9099999999999999,0.27847378742458834\n100% Corpus,2,symmetric,symmetric,0.2659027879879802\n100% Corpus,2,asymmetric,0.01,0.26049496936796\n100% Corpus,2,asymmetric,0.31,0.2638307040567668\n100% Corpus,2,asymmetric,0.61,0.2670430149871908\n100% Corpus,2,asymmetric,0.9099999999999999,0.2784737874245884\n100% Corpus,2,asymmetric,symmetric,0.2659027879879803\n100% Corpus,3,0.01,0.01,0.27711689254474037\n100% Corpus,3,0.01,0.31,0.27802374195120044\n100% Corpus,3,0.01,0.61,0.27993004576503216\n100% Corpus,3,0.01,0.9099999999999999,0.28461303163756196\n100% Corpus,3,0.01,symmetric,0.27802374195120044\n100% Corpus,3,0.31,0.01,0.2803842121014907\n100% Corpus,3,0.31,0.31,0.27802374195120044\n100% Corpus,3,0.31,0.61,0.27993004576503216\n100% Corpus,3,0.31,0.9099999999999999,0.2836001930609014\n100% Corpus,3,0.31,symmetric,0.27802374195120044\n100% Corpus,3,0.61,0.01,0.2803842121014907\n100% Corpus,3,0.61,0.31,0.27802374195120044\n100% Corpus,3,0.61,0.61,0.28095006391110916\n100% Corpus,3,0.61,0.9099999999999999,0.2836495488056456\n100% Corpus,3,0.61,symmetric,0.27802374195120044\n100% Corpus,3,0.9099999999999999,0.01,0.28041475692228796\n100% Corpus,3,0.9099999999999999,0.31,0.27802374195120044\n100% Corpus,3,0.9099999999999999,0.61,0.28095006391110916\n100% Corpus,3,0.9099999999999999,0.9099999999999999,0.2836495488056456\n100% Corpus,3,0.9099999999999999,symmetric,0.27802374195120044\n100% Corpus,3,symmetric,0.01,0.2803842121014907\n100% Corpus,3,symmetric,0.31,0.27802374195120044\n100% Corpus,3,symmetric,0.61,0.27993004576503216\n100% Corpus,3,symmetric,0.9099999999999999,0.2836495488056456\n100% Corpus,3,symmetric,symmetric,0.27802374195120044\n100% Corpus,3,asymmetric,0.01,0.2803842121014907\n100% Corpus,3,asymmetric,0.31,0.27802374195120044\n100% Corpus,3,asymmetric,0.61,0.28095006391110916\n100% Corpus,3,asymmetric,0.9099999999999999,0.2836495488056456\n100% Corpus,3,asymmetric,symmetric,0.27802374195120044\n100% Corpus,4,0.01,0.01,0.2809362113240368\n100% Corpus,4,0.01,0.31,0.27835632925620063\n100% Corpus,4,0.01,0.61,0.27555206267257515\n100% Corpus,4,0.01,0.9099999999999999,0.28446330333455255\n100% Corpus,4,0.01,symmetric,0.27835632925620063\n100% Corpus,4,0.31,0.01,0.2809362113240368\n100% Corpus,4,0.31,0.31,0.27835632925620063\n100% Corpus,4,0.31,0.61,0.27548045699445267\n100% Corpus,4,0.31,0.9099999999999999,0.28446330333455255\n100% Corpus,4,0.31,symmetric,0.27835632925620063\n100% Corpus,4,0.61,0.01,0.2809362113240368\n100% Corpus,4,0.61,0.31,0.27835632925620063\n100% Corpus,4,0.61,0.61,0.27966327435739713\n100% Corpus,4,0.61,0.9099999999999999,0.28446330333455255\n100% Corpus,4,0.61,symmetric,0.2782671534254213\n100% Corpus,4,0.9099999999999999,0.01,0.28177729599690937\n100% Corpus,4,0.9099999999999999,0.31,0.27835632925620063\n100% Corpus,4,0.9099999999999999,0.61,0.28263222170754976\n100% Corpus,4,0.9099999999999999,0.9099999999999999,0.28446330333455255\n100% Corpus,4,0.9099999999999999,symmetric,0.27749145169031925\n100% Corpus,4,symmetric,0.01,0.2809362113240368\n100% Corpus,4,symmetric,0.31,0.27835632925620063\n100% Corpus,4,symmetric,0.61,0.27548045699445267\n100% Corpus,4,symmetric,0.9099999999999999,0.28446330333455255\n100% Corpus,4,symmetric,symmetric,0.27835632925620063\n100% Corpus,4,asymmetric,0.01,0.2809362113240368\n100% Corpus,4,asymmetric,0.31,0.27835632925620063\n100% Corpus,4,asymmetric,0.61,0.27335019355355233\n100% Corpus,4,asymmetric,0.9099999999999999,0.28118168667280763\n100% Corpus,4,asymmetric,symmetric,0.2782671534254213\n100% Corpus,5,0.01,0.01,0.3055810333138698\n100% Corpus,5,0.01,0.31,0.2921971083432925\n100% Corpus,5,0.01,0.61,0.29351918649837283\n100% Corpus,5,0.01,0.9099999999999999,0.31027882311552357\n100% Corpus,5,0.01,symmetric,0.28798663645353445\n100% Corpus,5,0.31,0.01,0.3055810333138698\n100% Corpus,5,0.31,0.31,0.2919593387953088\n100% Corpus,5,0.31,0.61,0.29351918649837283\n100% Corpus,5,0.31,0.9099999999999999,0.3013857090688862\n100% Corpus,5,0.31,symmetric,0.28798663645353445\n100% Corpus,5,0.61,0.01,0.30778470383924467\n100% Corpus,5,0.61,0.31,0.29467018241870313\n100% Corpus,5,0.61,0.61,0.29688907251265284\n100% Corpus,5,0.61,0.9099999999999999,0.3013857090688862\n100% Corpus,5,0.61,symmetric,0.29224715357405645\n100% Corpus,5,0.9099999999999999,0.01,0.30778470383924467\n100% Corpus,5,0.9099999999999999,0.31,0.2891142546834146\n100% Corpus,5,0.9099999999999999,0.61,0.29749568404827875\n100% Corpus,5,0.9099999999999999,0.9099999999999999,0.30064353604119265\n100% Corpus,5,0.9099999999999999,symmetric,0.2903640490356588\n100% Corpus,5,symmetric,0.01,0.3055810333138698\n100% Corpus,5,symmetric,0.31,0.2919593387953088\n100% Corpus,5,symmetric,0.61,0.29351918649837283\n100% Corpus,5,symmetric,0.9099999999999999,0.3013857090688862\n100% Corpus,5,symmetric,symmetric,0.28798663645353445\n100% Corpus,5,asymmetric,0.01,0.3025740990082661\n100% Corpus,5,asymmetric,0.31,0.2891198159919257\n100% Corpus,5,asymmetric,0.61,0.2931349532251887\n100% Corpus,5,asymmetric,0.9099999999999999,0.30951060360405697\n100% Corpus,5,asymmetric,symmetric,0.2903640490356588\n100% Corpus,6,0.01,0.01,0.2958760348579074\n100% Corpus,6,0.01,0.31,0.2941800616166869\n100% Corpus,6,0.01,0.61,0.2971723188349142\n100% Corpus,6,0.01,0.9099999999999999,0.30879954124012965\n100% Corpus,6,0.01,symmetric,0.29805452202591254\n100% Corpus,6,0.31,0.01,0.2969800967267447\n100% Corpus,6,0.31,0.31,0.2951136969205841\n100% Corpus,6,0.31,0.61,0.2971723188349142\n100% Corpus,6,0.31,0.9099999999999999,0.30879954124012965\n100% Corpus,6,0.31,symmetric,0.29820145833395656\n100% Corpus,6,0.61,0.01,0.2969800967267447\n100% Corpus,6,0.61,0.31,0.2951136969205841\n100% Corpus,6,0.61,0.61,0.2977299345012703\n100% Corpus,6,0.61,0.9099999999999999,0.3049789423287517\n100% Corpus,6,0.61,symmetric,0.29863606640969337\n100% Corpus,6,0.9099999999999999,0.01,0.29390398336601525\n100% Corpus,6,0.9099999999999999,0.31,0.2951136969205841\n100% Corpus,6,0.9099999999999999,0.61,0.2957487280234154\n100% Corpus,6,0.9099999999999999,0.9099999999999999,0.3049789423287517\n100% Corpus,6,0.9099999999999999,symmetric,0.29863606640969337\n100% Corpus,6,symmetric,0.01,0.2955217784385554\n100% Corpus,6,symmetric,0.31,0.2941800616166869\n100% Corpus,6,symmetric,0.61,0.2971723188349142\n100% Corpus,6,symmetric,0.9099999999999999,0.30879954124012965\n100% Corpus,6,symmetric,symmetric,0.29820145833395656\n100% Corpus,6,asymmetric,0.01,0.2949603603199485\n100% Corpus,6,asymmetric,0.31,0.2951136969205841\n100% Corpus,6,asymmetric,0.61,0.2977299345012703\n100% Corpus,6,asymmetric,0.9099999999999999,0.30674292283526966\n100% Corpus,6,asymmetric,symmetric,0.29823105460471666\n100% Corpus,7,0.01,0.01,0.30073678510163443\n100% Corpus,7,0.01,0.31,0.3093075297409759\n100% Corpus,7,0.01,0.61,0.2985273077717666\n100% Corpus,7,0.01,0.9099999999999999,0.28108775204553077\n100% Corpus,7,0.01,symmetric,0.29593451694501705\n100% Corpus,7,0.31,0.01,0.3021816720953532\n100% Corpus,7,0.31,0.31,0.3136135696595105\n100% Corpus,7,0.31,0.61,0.30525285357617626\n100% Corpus,7,0.31,0.9099999999999999,0.2810941659084761\n100% Corpus,7,0.31,symmetric,0.29621498771621146\n100% Corpus,7,0.61,0.01,0.3021816720953532\n100% Corpus,7,0.61,0.31,0.3136135696595104\n100% Corpus,7,0.61,0.61,0.30936066130234136\n100% Corpus,7,0.61,0.9099999999999999,0.28629245148004184\n100% Corpus,7,0.61,symmetric,0.29731679986925613\n100% Corpus,7,0.9099999999999999,0.01,0.3020919465269357\n100% Corpus,7,0.9099999999999999,0.31,0.3136135696595105\n100% Corpus,7,0.9099999999999999,0.61,0.3151197257139406\n100% Corpus,7,0.9099999999999999,0.9099999999999999,0.288468301912423\n100% Corpus,7,0.9099999999999999,symmetric,0.30014119040856624\n100% Corpus,7,symmetric,0.01,0.29800123606081913\n100% Corpus,7,symmetric,0.31,0.3093075297409759\n100% Corpus,7,symmetric,0.61,0.29852730777176656\n100% Corpus,7,symmetric,0.9099999999999999,0.2833650342764223\n100% Corpus,7,symmetric,symmetric,0.29621498771621146\n100% Corpus,7,asymmetric,0.01,0.3014826617961082\n100% Corpus,7,asymmetric,0.31,0.30946569097108306\n100% Corpus,7,asymmetric,0.61,0.29780622606183804\n100% Corpus,7,asymmetric,0.9099999999999999,0.27769095312678477\n100% Corpus,7,asymmetric,symmetric,0.29669074530709133\n100% Corpus,8,0.01,0.01,0.27689325792791325\n100% Corpus,8,0.01,0.31,0.28335268992548035\n100% Corpus,8,0.01,0.61,0.29447751279748635\n100% Corpus,8,0.01,0.9099999999999999,0.2933443347019431\n100% Corpus,8,0.01,symmetric,0.2719332438180937\n100% Corpus,8,0.31,0.01,0.2778826015547943\n100% Corpus,8,0.31,0.31,0.2825893964563385\n100% Corpus,8,0.31,0.61,0.2939449197205991\n100% Corpus,8,0.31,0.9099999999999999,0.2933443347019431\n100% Corpus,8,0.31,symmetric,0.2729328842173082\n100% Corpus,8,0.61,0.01,0.2778826015547943\n100% Corpus,8,0.61,0.31,0.28595528080117094\n100% Corpus,8,0.61,0.61,0.2927074117432516\n100% Corpus,8,0.61,0.9099999999999999,0.2872200703569201\n100% Corpus,8,0.61,symmetric,0.2732337709535984\n100% Corpus,8,0.9099999999999999,0.01,0.27786820825055075\n100% Corpus,8,0.9099999999999999,0.31,0.28418596093474835\n100% Corpus,8,0.9099999999999999,0.61,0.2900190376327208\n100% Corpus,8,0.9099999999999999,0.9099999999999999,0.29123665567102414\n100% Corpus,8,0.9099999999999999,symmetric,0.27323377095359846\n100% Corpus,8,symmetric,0.01,0.27689325792791325\n100% Corpus,8,symmetric,0.31,0.28420779222803205\n100% Corpus,8,symmetric,0.61,0.2952855947321743\n100% Corpus,8,symmetric,0.9099999999999999,0.2933443347019431\n100% Corpus,8,symmetric,symmetric,0.2725123137958163\n100% Corpus,8,asymmetric,0.01,0.2761789762953015\n100% Corpus,8,asymmetric,0.31,0.2799130295170522\n100% Corpus,8,asymmetric,0.61,0.2948589392161668\n100% Corpus,8,asymmetric,0.9099999999999999,0.2812546037633434\n100% Corpus,8,asymmetric,symmetric,0.2729328842173082\n100% Corpus,9,0.01,0.01,0.3055895680888938\n100% Corpus,9,0.01,0.31,0.30302760647148597\n100% Corpus,9,0.01,0.61,0.2996054006558133\n100% Corpus,9,0.01,0.9099999999999999,0.34404649042246366\n100% Corpus,9,0.01,symmetric,0.3022567927575817\n100% Corpus,9,0.31,0.01,0.3050065555696478\n100% Corpus,9,0.31,0.31,0.30328591827752077\n100% Corpus,9,0.31,0.61,0.29917768071256\n100% Corpus,9,0.31,0.9099999999999999,0.34479247631815907\n100% Corpus,9,0.31,symmetric,0.30430285315228395\n100% Corpus,9,0.61,0.01,0.30426709798506846\n100% Corpus,9,0.61,0.31,0.3018951218284658\n100% Corpus,9,0.61,0.61,0.3001071122248435\n100% Corpus,9,0.61,0.9099999999999999,0.34618320085549326\n100% Corpus,9,0.61,symmetric,0.30433563585169243\n100% Corpus,9,0.9099999999999999,0.01,0.3055723111286964\n100% Corpus,9,0.9099999999999999,0.31,0.3030863932378131\n100% Corpus,9,0.9099999999999999,0.61,0.29813786201586745\n100% Corpus,9,0.9099999999999999,0.9099999999999999,0.34500630064915166\n100% Corpus,9,0.9099999999999999,symmetric,0.3038428140388971\n100% Corpus,9,symmetric,0.01,0.3023576157049908\n100% Corpus,9,symmetric,0.31,0.30302760647148597\n100% Corpus,9,symmetric,0.61,0.3003121132770352\n100% Corpus,9,symmetric,0.9099999999999999,0.34214587751079284\n100% Corpus,9,symmetric,symmetric,0.3022567927575817\n100% Corpus,9,asymmetric,0.01,0.3057340479868098\n100% Corpus,9,asymmetric,0.31,0.30328591827752077\n100% Corpus,9,asymmetric,0.61,0.3011057586148252\n100% Corpus,9,asymmetric,0.9099999999999999,0.350861967504945\n100% Corpus,9,asymmetric,symmetric,0.30251782074437106\n100% Corpus,10,0.01,0.01,0.283398008758761\n100% Corpus,10,0.01,0.31,0.2751258030801618\n100% Corpus,10,0.01,0.61,0.2915458869869755\n100% Corpus,10,0.01,0.9099999999999999,0.2948656869892196\n100% Corpus,10,0.01,symmetric,0.2777952167218004\n100% Corpus,10,0.31,0.01,0.28125840155500537\n100% Corpus,10,0.31,0.31,0.2762321145354264\n100% Corpus,10,0.31,0.61,0.29037856739931606\n100% Corpus,10,0.31,0.9099999999999999,0.29658208038434386\n100% Corpus,10,0.31,symmetric,0.27688479666982796\n100% Corpus,10,0.61,0.01,0.2803111695414585\n100% Corpus,10,0.61,0.31,0.27818213083876936\n100% Corpus,10,0.61,0.61,0.2894460065949672\n100% Corpus,10,0.61,0.9099999999999999,0.29802196235401307\n100% Corpus,10,0.61,symmetric,0.27614054331196536\n100% Corpus,10,0.9099999999999999,0.01,0.2817458696637633\n100% Corpus,10,0.9099999999999999,0.31,0.2783073583069227\n100% Corpus,10,0.9099999999999999,0.61,0.2889530806223174\n100% Corpus,10,0.9099999999999999,0.9099999999999999,0.2950248587541427\n100% Corpus,10,0.9099999999999999,symmetric,0.27614054331196536\n100% Corpus,10,symmetric,0.01,0.2821805736812627\n100% Corpus,10,symmetric,0.31,0.2751258030801618\n100% Corpus,10,symmetric,0.61,0.2911575680096669\n100% Corpus,10,symmetric,0.9099999999999999,0.2948656869892196\n100% Corpus,10,symmetric,symmetric,0.2777952167218004\n100% Corpus,10,asymmetric,0.01,0.2818119554111271\n100% Corpus,10,asymmetric,0.31,0.2804636061791579\n100% Corpus,10,asymmetric,0.61,0.29020292429978206\n100% Corpus,10,asymmetric,0.9099999999999999,0.2973427325310818\n100% Corpus,10,asymmetric,symmetric,0.278615868791953\n"
},
{
"path": "natural-language-processing/topic-modeling/results/ldavis_prepared_10.html",
"content": "\n\n\n\n\n"
},
{
"path": "natural-language-processing/topic-modeling/results/ldavis_tuned_8.html",
"content": "\n\n\n\n\n"
},
{
"path": "natural-language-processing/transformers-series/pyproject.toml",
"content": "[tool.poetry]\nname = \"transformers-models\"\nversion = \"0.1.0\"\ndescription = \"\"\nauthors = [\"Shashank Kapadia \"]\nreadme = \"README.md\"\n\n[tool.poetry.dependencies]\npython = \">=3.9,<3.10\"\njupyterlab = \"^4.0.2\"\ntensorflow = [\n { version = \"2.10.0\", platform = \"linux\" },\n]\ntensorflow-macos = [\n { version = \"2.10.0\", platform = \"darwin\" },\n]\npandas = \"^2.0.3\"\nseaborn = \"^0.12.2\"\ntransformers = \"4.30.0\"\ntorch = \"^2.0.1\"\nscikit-learn = \"^1.3.0\"\ndatasets = \"^2.13.1\"\naccelerate = \"^0.20.3\"\n\n\n[build-system]\nrequires = [\"poetry-core\"]\nbuild-backend = \"poetry.core.masonry.api\"\n"
},
{
"path": "natural-language-processing/transformers-series/sentiment_analysis_bert.ipynb",
"content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"id\": \"bb537d94-7c06-416c-823c-2111e595065f\",\n \"metadata\": {},\n \"source\": [\n \"## Practical Introduction to Transformer Models: BERT\\n\",\n \"\\n\",\n \"In the field of NLP, the transformer model architecture has been a revolutionary discovery that greatly enhanced our ability to understand and generate human language.\\n\",\n \"\\n\",\n \"This tutorial provides a close look at BERT (Bidirectional Encoder Representations from Transformers), one of the most popular transformer-based models, and instructs on its use in practice for sentiment analysis by fine-tuning the base model.\\n\",\n \"\\n\",\n \"#### Introduction to BERT\\n\",\n \"BERT, introduced by researchers at Google in 2018, is a powerful language model that uses transformer architecture. What sets BERT apart is its bidirectional nature. Traditional language models like LSTM and GRU are either unidirectional or sequentially bidirectional, but BERT takes into account the context from both past and future words simultaneously for a specific word. The key innovation in the transformer model is the \\\"attention\\\" mechanism, which allows the model to weight the importance of different words in a sentence when generating representations.\\n\",\n \"This feature allows BERT to understand the complete context of a word and, consequently, produce more accurate language models. It is pre-trained on two NLP tasks:\\n\",\n \"- Masked Language Model (MLM) and \\n\",\n \"- Next Sentence Prediction (NSP), \\n\",\n \"\\n\",\n \"and is often used as a starting point for various downstream NLP tasks, like sentiment analysis, which will be explored in this tutorial.\\n\",\n \"\\n\",\n \"#### Pre-Training and Fine-Tuning\\n\",\n \"The power of BERT comes from its two-step process: \\n\",\n \"- Pre-training is the phase where BERT is trained on large amounts of data. As a result, it learns to predict masked words in a sentence (MLM task) and to predict if a sentence follows another one (NSP task). The output of this stage is a a pre-trained NLP model with a general-purpose \\\"understanding\\\" of the language\\n\",\n \"- Fine-tuning is where the pre-trained BERT model is further trained on a specific task. The model is initialized with the pre-trained parameters, and the entire model is trained on a downstream task, allowing BERT to fine-tune its understanding of language to the specifics of the task at hand.\\n\",\n \"\\n\",\n \"#### Hands On: Using BERT for sentiment analysis\\n\",\n \"Now, let's get our hands dirty. We're going to perform sentiment analysis on the IMDb dataset, which contains movie reviews labeled as either positive or negative. For this, we'll be using Hugging Face's transformers library.\\n\",\n \"\\n\",\n \"** **\\n\",\n \"Let's load all the libraries\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 1,\n \"id\": \"9122a30a-9f2f-4d8a-9c87-2d11c0465308\",\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stderr\",\n \"output_type\": \"stream\",\n \"text\": [\n \"/Users/skapadia/Library/Caches/pypoetry/virtualenvs/transformers-models-r2uefcSj-py3.9/lib/python3.9/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\\n\",\n \" from .autonotebook import tqdm as notebook_tqdm\\n\"\n ]\n }\n ],\n \"source\": [\n \"import pandas as pd\\n\",\n \"import seaborn as sns\\n\",\n \"import matplotlib.pyplot as plt\\n\",\n \"from sklearn.metrics import confusion_matrix, roc_curve, auc\\n\",\n \"from datasets import load_dataset\\n\",\n \"from transformers import AutoTokenizer, AutoModelForSequenceClassification, TrainingArguments, Trainer\\n\",\n \"\\n\",\n \"# Variables to set the number of epochs and samples\\n\",\n \"num_epochs = 10\\n\",\n \"num_samples = 100 # set this to -1 to use all data\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"5fc553a9-96e5-45f3-8573-cf7445585de3\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"First, we need to load the dataset and the model tokenizer.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 2,\n \"id\": \"bf744b0d-6f8c-4711-ad75-efde168d978a\",\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stderr\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Found cached dataset imdb (/Users/skapadia/.cache/huggingface/datasets/imdb/plain_text/1.0.0/d613c88cf8fa3bab83b4ded3713f1f74830d1100e171db75bbddb80b3345c9c0)\\n\",\n \"100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 385.36it/s]\\n\"\n ]\n }\n ],\n \"source\": [\n \"# Step 1: Load dataset and model tokenizer\\n\",\n \"dataset = load_dataset('imdb')\\n\",\n \"tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"66f8f9f2-1c21-4717-b7f6-8210bbd25503\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"Before we move on, let's explore our dataset a bit. We'll create a plot to see the distribution of the positive and negative classes.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 3,\n \"id\": \"bb61016a-573c-475c-b95e-ec7964443106\",\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"image/png\": \"iVBORw0KGgoAAAANSUhEUgAAAk0AAAHHCAYAAACiOWx7AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA040lEQVR4nO3deXhU5d3/8c8kmIVlEpaQZH4GiEiBKIIGgciilJSg6GNaXChpQUVwSVSMBaTVsLigQZS1LFqFtlAptqCCRlIQKBBZgsiOFCPB0kmokIxEICE5vz98ch7GRLmJCTPB9+u6znVl7vs79/metCEfzzk547AsyxIAAAC+V4CvGwAAAKgPCE0AAAAGCE0AAAAGCE0AAAAGCE0AAAAGCE0AAAAGCE0AAAAGCE0AAAAGCE0AAAAGCE0AaqxNmza65557fN2GkQkTJsjhcHiNXaz+P//8czkcDi1YsMAeu+eee9S4ceM633clh8OhCRMmXLT9AZciQhOAKg4dOqQHHnhAV1xxhUJCQuR0OtWzZ09Nnz5dp06d8nV7PvXee+/5bfjw596AS0EDXzcAwL+sXLlSd955p4KDgzV06FBdffXVKi0t1YYNGzR69Gjt2bNH8+fP93WbteLAgQMKCLiw/3Z87733NHv27AsKJ61bt9apU6d02WWXXWCHF+b7ejt16pQaNOCffOCH4CcIgC0vL0+DBw9W69attWbNGkVHR9tzqamp+te//qWVK1f6sMPaFRwcXKfrnz17VhUVFQoKClJISEid7ut8fL1/4FLA5TkAtszMTJ08eVJ/+MMfvAJTpSuvvFKPPfbYd77/+PHj+s1vfqNOnTqpcePGcjqduvnmm/XJJ59UqZ05c6auuuoqNWzYUE2bNlXXrl21ePFie/6rr77SqFGj1KZNGwUHB6tly5b62c9+pu3bt5/3ODZs2KDrr79eISEhatu2rebNm1dt3bfvaSorK9PEiRPVrl07hYSEqHnz5urVq5eys7MlfXMf0uzZsyV9c49Q5Sb9331LL730kqZNm6a2bdsqODhYe/furfaepkqfffaZkpKS1KhRI7lcLk2aNEmWZdnza9eulcPh0Nq1a73e9+01v6+3yrFvn4H6+OOPdfPNN8vpdKpx48bq16+fPvroI6+aBQsWyOFwaOPGjUpPT1dERIQaNWqkn//85zp27Fj1/wMAlyjONAGwvfvuu7riiit0ww031Oj9n332mZYvX64777xTsbGxKigo0Lx583TjjTdq7969crlckqRXX31Vjz76qO644w499thjOn36tHbu3KnNmzdryJAhkqQHH3xQb731ltLS0hQXF6cvv/xSGzZs0L59+3Tdddd9Zw+7du1S//79FRERoQkTJujs2bMaP368IiMjz9v/hAkTNHnyZN1///3q1q2bPB6Ptm3bpu3bt+tnP/uZHnjgAR09elTZ2dn605/+VO0ab7zxhk6fPq2RI0cqODhYzZo1U0VFRbW15eXlGjBggHr06KHMzExlZWVp/PjxOnv2rCZNmnTefs9l0tu59uzZo969e8vpdGrMmDG67LLLNG/ePN10001at26dunfv7lX/yCOPqGnTpho/frw+//xzTZs2TWlpaVqyZMkF9QnUaxYAWJZVXFxsSbJuv/124/e0bt3aGjZsmP369OnTVnl5uVdNXl6eFRwcbE2aNMkeu/32262rrrrqe9cOCwuzUlNTjXuplJycbIWEhFiHDx+2x/bu3WsFBgZa3/4n79v9d+7c2Ro4cOD3rp+amlplHcv65jglWU6n0yosLKx27o033rDHhg0bZkmyHnnkEXusoqLCGjhwoBUUFGQdO3bMsizL+vDDDy1J1ocffnjeNb+rN8uyLEnW+PHj7dfJyclWUFCQdejQIXvs6NGjVpMmTaw+ffrYY2+88YYlyUpMTLQqKirs8ccff9wKDAy0ioqKqt0fcCni8hwASZLH45EkNWnSpMZrBAcH2zdWl5eX68svv1Tjxo3Vvn17r8tq4eHh+uKLL7R169bvXCs8PFybN2/W0aNHjfdfXl6uDz74QMnJyWrVqpU93rFjRyUlJZ33/eHh4dqzZ48OHjxovM9vGzRokCIiIozr09LS7K8dDofS0tJUWlqqf/zjHzXu4XzKy8u1atUqJScn64orrrDHo6OjNWTIEG3YsMH+/0OlkSNHel3u6927t8rLy3X48OE66xPwN4QmAJIkp9Mp6Zt7iWqqoqJCr7zyitq1a6fg4GC1aNFCERER2rlzp4qLi+26sWPHqnHjxurWrZvatWun1NRUbdy40WutzMxM7d69WzExMerWrZsmTJigzz777Hv3f+zYMZ06dUrt2rWrMte+ffvz9j9p0iQVFRXpJz/5iTp16qTRo0dr586dhkf/jdjYWOPagIAAr9AiST/5yU8kfXPPUl05duyYvv7662q/Jx07dlRFRYWOHDniNX5uCJWkpk2bSpJOnDhRZ30C/obQBEDSN6HJ5XJp9+7dNV7j+eefV3p6uvr06aM///nP+uCDD5Sdna2rrrrK676ejh076sCBA3rzzTfVq1cv/e1vf1OvXr00fvx4u+auu+7SZ599ppkzZ8rlcmnKlCm66qqr9P777/+g4/w+ffr00aFDh/T666/r6quv1muvvabrrrtOr732mvEaoaGhtdrTtx/IWam8vLxW93M+gYGB1Y5b59y0DlzqCE0AbLfeeqsOHTqknJycGr3/rbfeUt++ffWHP/xBgwcPVv/+/ZWYmKiioqIqtY0aNdLdd9+tN954Q/n5+Ro4cKCee+45nT592q6Jjo7Www8/rOXLlysvL0/NmzfXc8899537j4iIUGhoaLWX1w4cOGB0DM2aNdO9996rv/zlLzpy5IiuueYar786+64QUxMVFRVVzp59+umnkr75yz7p/87ofPt7WN1lMdPeIiIi1LBhw2q/J/v371dAQIBiYmKM1gJ+TAhNAGxjxoxRo0aNdP/996ugoKDK/KFDhzR9+vTvfH9gYGCVMw9Lly7Vv//9b6+xL7/80ut1UFCQ4uLiZFmWysrKVF5e7nU5T5Jatmwpl8ulM2fOfO/+k5KStHz5cuXn59vj+/bt0wcffPCd7/uuvho3bqwrr7zSa5+NGjWSVDXE1NSsWbPsry3L0qxZs3TZZZepX79+kr55MGZgYKDWr1/v9b7f//73VdYy7S0wMFD9+/fX22+/7XUZsKCgQIsXL1avXr3sy7UA/g+PHABga9u2rRYvXqy7775bHTt29Hoi+KZNm7R06dLv/ay2W2+9VZMmTdK9996rG264Qbt27dKiRYuq3LfTv39/RUVFqWfPnoqMjNS+ffs0a9YsDRw4UE2aNFFRUZEuv/xy3XHHHercubMaN26sf/zjH9q6daumTp36vccwceJEZWVlqXfv3nr44Yd19uxZ+5lQ57s/KS4uTjfddJPi4+PVrFkzbdu2zX7sQaX4+HhJ0qOPPqqkpCQFBgZq8ODB5/nOVi8kJERZWVkaNmyYunfvrvfff18rV67Ub3/7W/tm8rCwMN15552aOXOmHA6H2rZtqxUrVqiwsLDKehfS27PPPqvs7Gz16tVLDz/8sBo0aKB58+bpzJkzyszMrNHxAJc83/7xHgB/9Omnn1ojRoyw2rRpYwUFBVlNmjSxevbsac2cOdM6ffq0XVfdIweeeOIJKzo62goNDbV69uxp5eTkWDfeeKN144032nXz5s2z+vTpYzVv3twKDg622rZta40ePdoqLi62LMuyzpw5Y40ePdrq3Lmz1aRJE6tRo0ZW586drd///vdG/a9bt86Kj4+3goKCrCuuuMKaO3euNX78+PM+cuDZZ5+1unXrZoWHh1uhoaFWhw4drOeee84qLS21a86ePWs98sgjVkREhOVwOOw1Kx8BMGXKlCr9fNcjBxo1amQdOnTI6t+/v9WwYUMrMjLSGj9+fJXHNhw7dswaNGiQ1bBhQ6tp06bWAw88YO3evbvKmt/Vm2VVfeSAZVnW9u3braSkJKtx48ZWw4YNrb59+1qbNm3yqql85MDWrVu9xr/rUQjApcxhWdzFBwAAcD7c0wQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCAh1vWkoqKCh09elRNmjSp1Y9ZAAAAdceyLH311VdyuVwKCPj+c0mEplpy9OhRPqsJAIB66siRI7r88su/t4bQVEuaNGki6ZtvOp/ZBABA/eDxeBQTE2P/Hv8+hKZaUnlJzul0EpoAAKhnTG6t4UZwAAAAA4QmAAAAA4QmAAAAA4QmAAAAA4QmAAAAA4QmAAAAA4QmAAAAA4QmAAAAA4QmAAAAA4QmAAAAA4QmAAAAA4QmAAAAA4QmAAAAA4QmAAAAA4QmAAAAAw183QAuTPzoP/q6BcDv5E4Z6usWakX+pE6+bgHwO60ydvm6BRtnmgAAAAwQmgAAAAwQmgAAAAwQmgAAAAwQmgAAAAwQmgAAAAwQmgAAAAwQmgAAAAwQmgAAAAz4NDStX79et912m1wulxwOh5YvX27PlZWVaezYserUqZMaNWokl8uloUOH6ujRo15rHD9+XCkpKXI6nQoPD9fw4cN18uRJr5qdO3eqd+/eCgkJUUxMjDIzM6v0snTpUnXo0EEhISHq1KmT3nvvvTo5ZgAAUD/5NDSVlJSoc+fOmj17dpW5r7/+Wtu3b9fTTz+t7du36+9//7sOHDig//mf//GqS0lJ0Z49e5Sdna0VK1Zo/fr1GjlypD3v8XjUv39/tW7dWrm5uZoyZYomTJig+fPn2zWbNm3SL3/5Sw0fPlwff/yxkpOTlZycrN27d9fdwQMAgHrFYVmW5esmJMnhcGjZsmVKTk7+zpqtW7eqW7duOnz4sFq1aqV9+/YpLi5OW7duVdeuXSVJWVlZuuWWW/TFF1/I5XJpzpw5+t3vfie3262goCBJ0pNPPqnly5dr//79kqS7775bJSUlWrFihb2vHj16qEuXLpo7d65R/x6PR2FhYSouLpbT6azhd+H8+Ow5oCo+ew64dNX1Z89dyO/venVPU3FxsRwOh8LDwyVJOTk5Cg8PtwOTJCUmJiogIECbN2+2a/r06WMHJklKSkrSgQMHdOLECbsmMTHRa19JSUnKycn5zl7OnDkjj8fjtQEAgEtXvQlNp0+f1tixY/XLX/7SToJut1stW7b0qmvQoIGaNWsmt9tt10RGRnrVVL4+X03lfHUmT56ssLAwe4uJiflhBwgAAPxavQhNZWVluuuuu2RZlubMmePrdiRJ48aNU3Fxsb0dOXLE1y0BAIA61MDXDZxPZWA6fPiw1qxZ43W9MSoqSoWFhV71Z8+e1fHjxxUVFWXXFBQUeNVUvj5fTeV8dYKDgxUcHFzzAwMAAPWKX59pqgxMBw8e1D/+8Q81b97caz4hIUFFRUXKzc21x9asWaOKigp1797drlm/fr3KysrsmuzsbLVv315Nmza1a1avXu21dnZ2thISEurq0AAAQD3j09B08uRJ7dixQzt27JAk5eXlaceOHcrPz1dZWZnuuOMObdu2TYsWLVJ5ebncbrfcbrdKS0slSR07dtSAAQM0YsQIbdmyRRs3blRaWpoGDx4sl8slSRoyZIiCgoI0fPhw7dmzR0uWLNH06dOVnp5u9/HYY48pKytLU6dO1f79+zVhwgRt27ZNaWlpF/17AgAA/JNPQ9O2bdt07bXX6tprr5Ukpaen69prr1VGRob+/e9/65133tEXX3yhLl26KDo62t42bdpkr7Fo0SJ16NBB/fr10y233KJevXp5PYMpLCxMq1atUl5enuLj4/XEE08oIyPD61lON9xwgxYvXqz58+erc+fOeuutt7R8+XJdffXVF++bAQAA/JrfPKepvuM5TYDv8Jwm4NLFc5oAAADqGUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAUITAACAAZ+GpvXr1+u2226Ty+WSw+HQ8uXLveYty1JGRoaio6MVGhqqxMREHTx40Kvm+PHjSklJkdPpVHh4uIYPH66TJ0961ezcuVO9e/dWSEiIYmJilJmZWaWXpUuXqkOHDgoJCVGnTp303nvv1frxAgCA+sunoamkpESdO3fW7Nmzq53PzMzUjBkzNHfuXG3evFmNGjVSUlKSTp8+bdekpKRoz549ys7O1ooVK7R+/XqNHDnSnvd4POrfv79at26t3NxcTZkyRRMmTND8+fPtmk2bNumXv/ylhg8fro8//ljJyclKTk7W7t276+7gAQBAveKwLMvydROS5HA4tGzZMiUnJ0v65iyTy+XSE088od/85jeSpOLiYkVGRmrBggUaPHiw9u3bp7i4OG3dulVdu3aVJGVlZemWW27RF198IZfLpTlz5uh3v/ud3G63goKCJElPPvmkli9frv3790uS7r77bpWUlGjFihV2Pz169FCXLl00d+5co/49Ho/CwsJUXFwsp9NZW9+WKuJH/7HO1gbqq9wpQ33dQq3In9TJ1y0AfqdVxq46Xf9Cfn/77T1NeXl5crvdSkxMtMfCwsLUvXt35eTkSJJycnIUHh5uByZJSkxMVEBAgDZv3mzX9OnTxw5MkpSUlKQDBw7oxIkTds25+6msqdxPdc6cOSOPx+O1AQCAS5ffhia32y1JioyM9BqPjIy059xut1q2bOk136BBAzVr1syrpro1zt3Hd9VUzldn8uTJCgsLs7eYmJgLPUQAAFCP+G1o8nfjxo1TcXGxvR05csTXLQEAgDrkt6EpKipKklRQUOA1XlBQYM9FRUWpsLDQa/7s2bM6fvy4V011a5y7j++qqZyvTnBwsJxOp9cGAAAuXX4bmmJjYxUVFaXVq1fbYx6PR5s3b1ZCQoIkKSEhQUVFRcrNzbVr1qxZo4qKCnXv3t2uWb9+vcrKyuya7OxstW/fXk2bNrVrzt1PZU3lfgAAAHwamk6ePKkdO3Zox44dkr65+XvHjh3Kz8+Xw+HQqFGj9Oyzz+qdd97Rrl27NHToULlcLvsv7Dp27KgBAwZoxIgR2rJlizZu3Ki0tDQNHjxYLpdLkjRkyBAFBQVp+PDh2rNnj5YsWaLp06crPT3d7uOxxx5TVlaWpk6dqv3792vChAnatm2b0tLSLva3BAAA+KkGvtz5tm3b1LdvX/t1ZZAZNmyYFixYoDFjxqikpEQjR45UUVGRevXqpaysLIWEhNjvWbRokdLS0tSvXz8FBARo0KBBmjFjhj0fFhamVatWKTU1VfHx8WrRooUyMjK8nuV0ww03aPHixXrqqaf029/+Vu3atdPy5ct19dVXX4TvAgAAqA/85jlN9R3PaQJ8h+c0AZcuntMEAABQzxCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADBCaAAAADPh1aCovL9fTTz+t2NhYhYaGqm3btnrmmWdkWZZdY1mWMjIyFB0drdDQUCUmJurgwYNe6xw/flwpKSlyOp0KDw/X8OHDdfLkSa+anTt3qnfv3goJCVFMTIwyMzMvyjECAID6wa9D04svvqg5c+Zo1qxZ2rdvn1588UVlZmZq5syZdk1mZqZmzJihuXPnavPmzWrUqJGSkpJ0+vRpuyYlJUV79uxRdna2VqxYofXr12vkyJH2vMfjUf/+/dW6dWvl5uZqypQpmjBhgubPn39RjxcAAPivBr5u4Pts2rRJt99+uwYOHChJatOmjf7yl79oy5Ytkr45yzRt2jQ99dRTuv322yVJf/zjHxUZGanly5dr8ODB2rdvn7KysrR161Z17dpVkjRz5kzdcssteumll+RyubRo0SKVlpbq9ddfV1BQkK666irt2LFDL7/8sle4AgAAP15+fabphhtu0OrVq/Xpp59Kkj755BNt2LBBN998syQpLy9PbrdbiYmJ9nvCwsLUvXt35eTkSJJycnIUHh5uByZJSkxMVEBAgDZv3mzX9OnTR0FBQXZNUlKSDhw4oBMnTlTb25kzZ+TxeLw2AABw6fLrM01PPvmkPB6POnTooMDAQJWXl+u5555TSkqKJMntdkuSIiMjvd4XGRlpz7ndbrVs2dJrvkGDBmrWrJlXTWxsbJU1KueaNm1apbfJkydr4sSJtXCUAACgPvDrM01//etftWjRIi1evFjbt2/XwoUL9dJLL2nhwoW+bk3jxo1TcXGxvR05csTXLQEAgDrk12eaRo8erSeffFKDBw+WJHXq1EmHDx/W5MmTNWzYMEVFRUmSCgoKFB0dbb+voKBAXbp0kSRFRUWpsLDQa92zZ8/q+PHj9vujoqJUUFDgVVP5urLm24KDgxUcHPzDDxIAANQLfn2m6euvv1ZAgHeLgYGBqqiokCTFxsYqKipKq1evtuc9Ho82b96shIQESVJCQoKKioqUm5tr16xZs0YVFRXq3r27XbN+/XqVlZXZNdnZ2Wrfvn21l+YAAMCPj1+Hpttuu03PPfecVq5cqc8//1zLli3Tyy+/rJ///OeSJIfDoVGjRunZZ5/VO++8o127dmno0KFyuVxKTk6WJHXs2FEDBgzQiBEjtGXLFm3cuFFpaWkaPHiwXC6XJGnIkCEKCgrS8OHDtWfPHi1ZskTTp09Xenq6rw4dAAD4Gb++PDdz5kw9/fTTevjhh1VYWCiXy6UHHnhAGRkZds2YMWNUUlKikSNHqqioSL169VJWVpZCQkLsmkWLFiktLU39+vVTQECABg0apBkzZtjzYWFhWrVqlVJTUxUfH68WLVooIyODxw0AAACbwzr38dqoMY/Ho7CwMBUXF8vpdNbZfuJH/7HO1gbqq9wpQ33dQq3In9TJ1y0AfqdVxq46Xf9Cfn/79eU5AAAAf0FoAgAAMEBoAgAAMEBoAgAAMEBoAgAAMEBoAgAAMEBoAgAAMEBoAgAAMEBoAgAAMEBoAgAAMEBoAgAAMEBoAgAAMEBoAgAAMEBoAgAAMEBoAgAAMEBoAgAAMEBoAgAAMFCj0PTTn/5URUVFVcY9Ho9++tOf/tCeAAAA/E6NQtPatWtVWlpaZfz06dP65z//+YObAgAA8DcNLqR4586d9td79+6V2+22X5eXlysrK0v/7//9v9rrDgAAwE9cUGjq0qWLHA6HHA5HtZfhQkNDNXPmzFprDgAAwF9cUGjKy8uTZVm64oortGXLFkVERNhzQUFBatmypQIDA2u9SQAAAF+7oNDUunVrSVJFRUWdNAMAAOCvLig0nevgwYP68MMPVVhYWCVEZWRk/ODGAAAA/EmNQtOrr76qhx56SC1atFBUVJQcDoc953A4CE0AAOCSU6PQ9Oyzz+q5557T2LFja7sfAAAAv1Sj5zSdOHFCd955Z233AgAA4LdqFJruvPNOrVq1qrZ7AQAA8Fs1ujx35ZVX6umnn9ZHH32kTp066bLLLvOaf/TRR2ulOQAAAH9Ro9A0f/58NW7cWOvWrdO6deu85hwOB6EJAABccmoUmvLy8mq7DwAAAL9Wo3uaAAAAfmxqdKbpvvvu+975119/vUbNAAAA+KsahaYTJ054vS4rK9Pu3btVVFRU7Qf5AgAA1Hc1Ck3Lli2rMlZRUaGHHnpIbdu2/cFNAQAA+Jtau6cpICBA6enpeuWVV2prSQAAAL9RqzeCHzp0SGfPnq3NJQEAAPxCjS7Ppaene722LEv/+c9/tHLlSg0bNqxWGgMAAPAnNQpNH3/8sdfrgIAARUREaOrUqef9yzoAAID6qEah6cMPP6ztPgAAAPxajUJTpWPHjunAgQOSpPbt2ysiIqJWmgIAAPA3NboRvKSkRPfdd5+io6PVp08f9enTRy6XS8OHD9fXX39d2z0CAAD4XI1CU3p6utatW6d3331XRUVFKioq0ttvv61169bpiSeeqO0eAQAAfK5Gl+f+9re/6a233tJNN91kj91yyy0KDQ3VXXfdpTlz5tRWfwAAAH6hRmeavv76a0VGRlYZb9myJZfnAADAJalGoSkhIUHjx4/X6dOn7bFTp05p4sSJSkhIqLXmJOnf//63fvWrX6l58+YKDQ1Vp06dtG3bNnvesixlZGQoOjpaoaGhSkxM1MGDB73WOH78uFJSUuR0OhUeHq7hw4fr5MmTXjU7d+5U7969FRISopiYGGVmZtbqcQAAgPqtRpfnpk2bpgEDBujyyy9X586dJUmffPKJgoODtWrVqlpr7sSJE+rZs6f69u2r999/XxERETp48KCaNm1q12RmZmrGjBlauHChYmNj9fTTTyspKUl79+5VSEiIJCklJUX/+c9/lJ2drbKyMt17770aOXKkFi9eLEnyeDzq37+/EhMTNXfuXO3atUv33XefwsPDNXLkyFo7HgAAUH85LMuyavLGr7/+WosWLdL+/fslSR07dlRKSopCQ0Nrrbknn3xSGzdu1D//+c9q5y3Lksvl0hNPPKHf/OY3kqTi4mJFRkZqwYIFGjx4sPbt26e4uDht3bpVXbt2lSRlZWXplltu0RdffCGXy6U5c+bod7/7ndxut4KCgux9L1++3D6+8/F4PAoLC1NxcbGcTmctHH314kf/sc7WBuqr3ClDfd1Crcif1MnXLQB+p1XGrjpd/0J+f9fo8tzkyZP15ptvasSIEZo6daqmTp2q+++/X3/5y1/04osv1qjp6rzzzjvq2rWr7rzzTrVs2VLXXnutXn31VXs+Ly9PbrdbiYmJ9lhYWJi6d++unJwcSVJOTo7Cw8PtwCRJiYmJCggI0ObNm+2aPn362IFJkpKSknTgwAGdOHGi1o4HAADUXzUKTfPmzVOHDh2qjF911VWaO3fuD26q0meffaY5c+aoXbt2+uCDD/TQQw/p0Ucf1cKFCyVJbrdbkqrclB4ZGWnPud1utWzZ0mu+QYMGatasmVdNdWucu49vO3PmjDwej9cGAAAuXTW6p8ntdis6OrrKeEREhP7zn//84KYqVVRUqGvXrnr++eclSddee612796tuXPn+vyDgSdPnqyJEyf6tAcAAHDx1OhMU0xMjDZu3FhlfOPGjXK5XD+4qUrR0dGKi4vzGuvYsaPy8/MlSVFRUZKkgoICr5qCggJ7LioqSoWFhV7zZ8+e1fHjx71qqlvj3H1827hx41RcXGxvR44cqckhAgCAeqJGoWnEiBEaNWqU3njjDR0+fFiHDx/W66+/rscff1wjRoyoteZ69uxpf7ZdpU8//VStW7eWJMXGxioqKkqrV6+25z0ejzZv3mw/+iAhIUFFRUXKzc21a9asWaOKigp1797drlm/fr3KysrsmuzsbLVv397rL/XOFRwcLKfT6bUBAIBLV40uz40ePVpffvmlHn74YZWWlkqSQkJCNHbsWI0bN67Wmnv88cd1ww036Pnnn9ddd92lLVu2aP78+Zo/f74kyeFwaNSoUXr22WfVrl07+5EDLpdLycnJkr45MzVgwACNGDFCc+fOVVlZmdLS0jR48GD7rNiQIUM0ceJEDR8+XGPHjtXu3bs1ffp0vfLKK7V2LAAAoH6rUWhyOBx68cUX9fTTT2vfvn0KDQ1Vu3btFBwcXKvNXX/99Vq2bJnGjRunSZMmKTY2VtOmTVNKSopdM2bMGJWUlGjkyJEqKipSr169lJWVZT+jSZIWLVqktLQ09evXTwEBARo0aJBmzJhhz4eFhWnVqlVKTU1VfHy8WrRooYyMDJ7RBAAAbDV+ThO88ZwmwHd4ThNw6ar3z2kCAAD4sSE0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGCA0AQAAGKhXoemFF16Qw+HQqFGj7LHTp08rNTVVzZs3V+PGjTVo0CAVFBR4vS8/P18DBw5Uw4YN1bJlS40ePVpnz571qlm7dq2uu+46BQcH68orr9SCBQsuwhEBAID6ot6Epq1bt2revHm65pprvMYff/xxvfvuu1q6dKnWrVuno0eP6he/+IU9X15eroEDB6q0tFSbNm3SwoULtWDBAmVkZNg1eXl5GjhwoPr27asdO3Zo1KhRuv/++/XBBx9ctOMDAAD+rV6EppMnTyolJUWvvvqqmjZtao8XFxfrD3/4g15++WX99Kc/VXx8vN544w1t2rRJH330kSRp1apV2rt3r/785z+rS5cuuvnmm/XMM89o9uzZKi0tlSTNnTtXsbGxmjp1qjp27Ki0tDTdcccdeuWVV3xyvAAAwP/Ui9CUmpqqgQMHKjEx0Ws8NzdXZWVlXuMdOnRQq1atlJOTI0nKyclRp06dFBkZadckJSXJ4/Foz549ds23105KSrLXqM6ZM2fk8Xi8NgAAcOlq4OsGzufNN9/U9u3btXXr1ipzbrdbQUFBCg8P9xqPjIyU2+22a84NTJXzlXPfV+PxeHTq1CmFhoZW2ffkyZM1ceLEGh8XAACoX/z6TNORI0f02GOPadGiRQoJCfF1O17GjRun4uJiezty5IivWwIAAHXIr0NTbm6uCgsLdd1116lBgwZq0KCB1q1bpxkzZqhBgwaKjIxUaWmpioqKvN5XUFCgqKgoSVJUVFSVv6arfH2+GqfTWe1ZJkkKDg6W0+n02gAAwKXLr0NTv379tGvXLu3YscPeunbtqpSUFPvryy67TKtXr7bfc+DAAeXn5yshIUGSlJCQoF27dqmwsNCuyc7OltPpVFxcnF1z7hqVNZVrAAAA+PU9TU2aNNHVV1/tNdaoUSM1b97cHh8+fLjS09PVrFkzOZ1OPfLII0pISFCPHj0kSf3791dcXJx+/etfKzMzU263W0899ZRSU1MVHBwsSXrwwQc1a9YsjRkzRvfdd5/WrFmjv/71r1q5cuXFPWAAAOC3/Do0mXjllVcUEBCgQYMG6cyZM0pKStLvf/97ez4wMFArVqzQQw89pISEBDVq1EjDhg3TpEmT7JrY2FitXLlSjz/+uKZPn67LL79cr732mpKSknxxSAAAwA85LMuyfN3EpcDj8SgsLEzFxcV1en9T/Og/1tnaQH2VO2Wor1uoFfmTOvm6BcDvtMrYVafrX8jvb7++pwkAAMBfEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAMEJoAAAAM+HVomjx5sq6//no1adJELVu2VHJysg4cOOBVc/r0aaWmpqp58+Zq3LixBg0apIKCAq+a/Px8DRw4UA0bNlTLli01evRonT171qtm7dq1uu666xQcHKwrr7xSCxYsqOvDAwAA9Yhfh6Z169YpNTVVH330kbKzs1VWVqb+/furpKTErnn88cf17rvvaunSpVq3bp2OHj2qX/ziF/Z8eXm5Bg4cqNLSUm3atEkLFy7UggULlJGRYdfk5eVp4MCB6tu3r3bs2KFRo0bp/vvv1wcffHBRjxcAAPgvh2VZlq+bMHXs2DG1bNlS69atU58+fVRcXKyIiAgtXrxYd9xxhyRp//796tixo3JyctSjRw+9//77uvXWW3X06FFFRkZKkubOnauxY8fq2LFjCgoK0tixY7Vy5Urt3r3b3tfgwYNVVFSkrKwso948Ho/CwsJUXFwsp9NZ+wf/v+JH/7HO1gbqq9wpQ33dQq3In9TJ1y0AfqdVxq46Xf9Cfn/79ZmmbysuLpYkNWvWTJKUm5ursrIyJSYm2jUdOnRQq1atlJOTI0nKyclRp06d7MAkSUlJSfJ4PNqzZ49dc+4alTWVa1TnzJkz8ng8XhsAALh01ZvQVFFRoVGjRqlnz566+uqrJUlut1tBQUEKDw/3qo2MjJTb7bZrzg1MlfOVc99X4/F4dOrUqWr7mTx5ssLCwuwtJibmBx8jAADwX/UmNKWmpmr37t168803fd2KJGncuHEqLi62tyNHjvi6JQAAUIca+LoBE2lpaVqxYoXWr1+vyy+/3B6PiopSaWmpioqKvM42FRQUKCoqyq7ZsmWL13qVf113bs23/+KuoKBATqdToaGh1fYUHBys4ODgH3xsAACgfvDrM02WZSktLU3Lli3TmjVrFBsb6zUfHx+vyy67TKtXr7bHDhw4oPz8fCUkJEiSEhIStGvXLhUWFto12dnZcjqdiouLs2vOXaOypnINAAAAvz7TlJqaqsWLF+vtt99WkyZN7HuQwsLCFBoaqrCwMA0fPlzp6elq1qyZnE6nHnnkESUkJKhHjx6SpP79+ysuLk6//vWvlZmZKbfbraeeekqpqan2maIHH3xQs2bN0pgxY3TfffdpzZo1+utf/6qVK1f67NgBAIB/8eszTXPmzFFxcbFuuukmRUdH29uSJUvsmldeeUW33nqrBg0apD59+igqKkp///vf7fnAwECtWLFCgYGBSkhI0K9+9SsNHTpUkyZNsmtiY2O1cuVKZWdnq3Pnzpo6dapee+01JSUlXdTjBQAA/qtePafJn/GcJsB3eE4TcOniOU0AAAD1DKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKEJAADAAKHpW2bPnq02bdooJCRE3bt315YtW3zdEgAA8AOEpnMsWbJE6enpGj9+vLZv367OnTsrKSlJhYWFvm4NAAD4GKHpHC+//LJGjBihe++9V3FxcZo7d64aNmyo119/3detAQAAHyM0/a/S0lLl5uYqMTHRHgsICFBiYqJycnJ82BkAAPAHDXzdgL/473//q/LyckVGRnqNR0ZGav/+/VXqz5w5ozNnztivi4uLJUkej6dO+yw/c6pO1wfqo7r+ubtYvjpd7usWAL9T1z/fletblnXeWkJTDU2ePFkTJ06sMh4TE+ODboAft7CZD/q6BQB1ZXLYRdnNV199pbCw798Xoel/tWjRQoGBgSooKPAaLygoUFRUVJX6cePGKT093X5dUVGh48ePq3nz5nI4HHXeL3zL4/EoJiZGR44ckdPp9HU7AGoRP98/LpZl6auvvpLL5TpvLaHpfwUFBSk+Pl6rV69WcnKypG+C0OrVq5WWllalPjg4WMHBwV5j4eHhF6FT+BOn08k/qsAlip/vH4/znWGqRGg6R3p6uoYNG6auXbuqW7dumjZtmkpKSnTvvff6ujUAAOBjhKZz3H333Tp27JgyMjLkdrvVpUsXZWVlVbk5HAAA/PgQmr4lLS2t2stxwLmCg4M1fvz4KpdoAdR//Hzjuzgsk7+xAwAA+JHj4ZYAAAAGCE0AAAAGCE0AAAAGCE0AAAAGCE1ADcyePVtt2rRRSEiIunfvri1btvi6JQA/0Pr163XbbbfJ5XLJ4XBo+fLlvm4JfobQBFygJUuWKD09XePHj9f27dvVuXNnJSUlqbCw0NetAfgBSkpK1LlzZ82ePdvXrcBP8cgB4AJ1795d119/vWbNmiXpm4/biYmJ0SOPPKInn3zSx90BqA0Oh0PLli2zP1YLkDjTBFyQ0tJS5ebmKjEx0R4LCAhQYmKicnJyfNgZAKCuEZqAC/Df//5X5eXlVT5aJzIyUm6320ddAQAuBkITAACAAUITcAFatGihwMBAFRQUeI0XFBQoKirKR10BAC4GQhNwAYKCghQfH6/Vq1fbYxUVFVq9erUSEhJ82BkAoK418HUDQH2Tnp6uYcOGqWvXrurWrZumTZumkpIS3Xvvvb5uDcAPcPLkSf3rX/+yX+fl5WnHjh1q1qyZWrVq5cPO4C945ABQA7NmzdKUKVPkdrvVpUsXzZgxQ927d/d1WwB+gLVr16pv375VxocNG6YFCxZc/IbgdwhNAAAABrinCQAAwAChCQAAwAChCQAAwAChCQAAwAChCQAAwAChCQAAwAChCQAAwAChCcCPxk033aRRo0YZ1a5du1YOh0NFRUU/aJ9t2rTRtGnTftAaAPwDoQkAAMAAoQkAAMAAoQnAj9Kf/vQnde3aVU2aNFFUVJSGDBmiwsLCKnUbN27UNddco5CQEPXo0UO7d+/2mt+wYYN69+6t0NBQxcTE6NFHH1VJScnFOgwAFxGhCcCPUllZmZ555hl98sknWr58uT7//HPdc889VepGjx6tqVOnauvWrYqIiNBtt92msrIySdKhQ4c0YMAADRo0SDt37tSSJUu0YcMGpaWlXeSjAXAxNPB1AwDgC/fdd5/99RVXXKEZM2bo+uuv18mTJ9W4cWN7bvz48frZz34mSVq4cKEuv/xyLVu2THfddZcmT56slJQU++bydu3aacaMGbrxxhs1Z84chYSEXNRjAlC3ONME4EcpNzdXt912m1q1aqUmTZroxhtvlCTl5+d71SUkJNhfN2vWTO3bt9e+ffskSZ988okWLFigxo0b21tSUpIqKiqUl5d38Q4GwEXBmSYAPzolJSVKSkpSUlKSFi1apIiICOXn5yspKUmlpaXG65w8eVIPPPCAHn300SpzrVq1qs2WAfgBQhOAH539+/fryy+/1AsvvKCYmBhJ0rZt26qt/eijj+wAdOLECX366afq2LGjJOm6667T3r17deWVV16cxgH4FJfnAPzotGrVSkFBQZo5c6Y+++wzvfPOO3rmmWeqrZ00aZJWr16t3bt365577lGLFi2UnJwsSRo7dqw2bdqktLQ07dixQwcPHtTbb7/NjeDAJYrQBOBHJyIiQgsWLNDSpUsVFxenF154QS+99FK1tS+88IIee+wxxcfHy+12691331VQUJAk6ZprrtG6dev06aefqnfv3rr22muVkZEhl8t1MQ8HwEXisCzL8nUTAAAA/o4zTQAAAAYITQAAAAYITQAAAAYITQAAAAYITQAAAAYITQAAAAYITQAAAAYITQAAAAYITQAAAAYITQAAAAYITQAAAAYITQAAAAb+P4ULftHNdw8wAAAAAElFTkSuQmCC\",\n \"text/plain\": [\n \"
\"\n ]\n },\n \"metadata\": {},\n \"output_type\": \"display_data\"\n }\n ],\n \"source\": [\n \"# Data Exploration\\n\",\n \"train_df = pd.DataFrame(dataset[\\\"train\\\"])\\n\",\n \"sns.countplot(x='label', data=train_df)\\n\",\n \"plt.title('Class distribution')\\n\",\n \"plt.show()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"bcb1fcbc-42df-4fa8-b371-ea7daf0914ae\",\n \"metadata\": {},\n \"source\": [\n \"It is essential to highlight that in practical applications, encountering a balanced class distribution is rarely the case. If the dataset for your use-case demonstrates class imbalance, it is strongly recommended to implement suitable strategies prior to training the model. Such procedures can significantly enhance the model's performance.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"d368794c-8a56-4761-8cb5-7d735c21bfa7\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"Next, we preprocess our dataset by tokenizing the texts. We use BERT's tokenizer, which will convert the text into tokens that correspond to BERT's vocabulary.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 4,\n \"id\": \"d8849a36-68be-41fa-8e8a-658c5114fcb7\",\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stderr\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Loading cached processed dataset at /Users/skapadia/.cache/huggingface/datasets/imdb/plain_text/1.0.0/d613c88cf8fa3bab83b4ded3713f1f74830d1100e171db75bbddb80b3345c9c0/cache-8d3f672c6bd459c5.arrow\\n\",\n \"Loading cached processed dataset at /Users/skapadia/.cache/huggingface/datasets/imdb/plain_text/1.0.0/d613c88cf8fa3bab83b4ded3713f1f74830d1100e171db75bbddb80b3345c9c0/cache-a224c74dea2a378a.arrow\\n\",\n \"Loading cached processed dataset at /Users/skapadia/.cache/huggingface/datasets/imdb/plain_text/1.0.0/d613c88cf8fa3bab83b4ded3713f1f74830d1100e171db75bbddb80b3345c9c0/cache-3771e6c862b361d6.arrow\\n\"\n ]\n }\n ],\n \"source\": [\n \"# Step 2: Preprocess the dataset\\n\",\n \"def tokenize_function(examples):\\n\",\n \" return tokenizer(examples[\\\"text\\\"], padding=\\\"max_length\\\", truncation=True)\\n\",\n \"\\n\",\n \"tokenized_datasets = dataset.map(tokenize_function, batched=True)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"c893c1a4-a3bc-4ed3-96b6-dcce961ff823\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"After that, we prepare our training and evaluation datasets. Remember, if you want to use all the data, you can set the num_samples variable to -1.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 5,\n \"id\": \"dbf75222-73e3-4e3b-b2ba-edf51e0a1083\",\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stderr\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Loading cached shuffled indices for dataset at /Users/skapadia/.cache/huggingface/datasets/imdb/plain_text/1.0.0/d613c88cf8fa3bab83b4ded3713f1f74830d1100e171db75bbddb80b3345c9c0/cache-84712cace8bef640.arrow\\n\",\n \"Loading cached shuffled indices for dataset at /Users/skapadia/.cache/huggingface/datasets/imdb/plain_text/1.0.0/d613c88cf8fa3bab83b4ded3713f1f74830d1100e171db75bbddb80b3345c9c0/cache-89ec35fd52b28a4f.arrow\\n\"\n ]\n }\n ],\n \"source\": [\n \"if num_samples == -1:\\n\",\n \" small_train_dataset = tokenized_datasets[\\\"train\\\"].shuffle(seed=42)\\n\",\n \" small_eval_dataset = tokenized_datasets[\\\"test\\\"].shuffle(seed=42)\\n\",\n \"else:\\n\",\n \" small_train_dataset = tokenized_datasets[\\\"train\\\"].shuffle(seed=42).select(range(num_samples)) \\n\",\n \" small_eval_dataset = tokenized_datasets[\\\"test\\\"].shuffle(seed=42).select(range(num_samples))\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"5cd50a71-f337-43ef-b7b5-e9e048c996ae\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"Then, we load the pre-trained BERT model. We'll use the AutoModelForSequenceClassification class, a BERT model designed for classification tasks.\\n\",\n \"\\n\",\n \"For this tutorial, we use the 'bert-base-uncased' version of BERT, which is trained on lower-case English text, is used for this tutorial.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 6,\n \"id\": \"8b4b002a-35b6-4f34-b34d-4fc0cd5fcca6\",\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stderr\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForSequenceClassification: ['cls.predictions.bias', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.seq_relationship.weight', 'cls.predictions.transform.dense.bias']\\n\",\n \"- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\\n\",\n \"- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\\n\",\n \"Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['classifier.weight', 'classifier.bias']\\n\",\n \"You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\\n\"\n ]\n }\n ],\n \"source\": [\n \"# Step 3: Load pre-trained model\\n\",\n \"model = AutoModelForSequenceClassification.from_pretrained('bert-base-uncased', num_labels=2)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"64b1ff16-44bc-4aec-a466-359927157eca\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"Now, we're ready to define our training arguments and create a Trainer instance to train our model.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 7,\n \"id\": \"77dc2392-8a97-445c-a53b-6afdbc8c09bc\",\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stderr\",\n \"output_type\": \"stream\",\n \"text\": [\n \"/Users/skapadia/Library/Caches/pypoetry/virtualenvs/transformers-models-r2uefcSj-py3.9/lib/python3.9/site-packages/transformers/optimization.py:411: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\\n\",\n \" warnings.warn(\\n\"\n ]\n },\n {\n \"data\": {\n \"text/html\": [\n \"\\n\",\n \"
\"\n ]\n },\n \"metadata\": {},\n \"output_type\": \"display_data\"\n }\n ],\n \"source\": [\n \"# Step 6: Evaluation\\n\",\n \"predictions = trainer.predict(small_eval_dataset)\\n\",\n \"\\n\",\n \"# Confusion matrix\\n\",\n \"cm = confusion_matrix(small_eval_dataset['label'], predictions.predictions.argmax(-1))\\n\",\n \"sns.heatmap(cm, annot=True, fmt='d')\\n\",\n \"plt.title('Confusion Matrix')\\n\",\n \"plt.show()\\n\",\n \"\\n\",\n \"# ROC Curve\\n\",\n \"fpr, tpr, _ = roc_curve(small_eval_dataset['label'], predictions.predictions[:, 1])\\n\",\n \"roc_auc = auc(fpr, tpr)\\n\",\n \"\\n\",\n \"plt.figure(figsize=(1.618 * 5, 5))\\n\",\n \"plt.plot(fpr, tpr, color='darkorange', lw=2, label='ROC curve (area = %0.2f)' % roc_auc)\\n\",\n \"plt.plot([0, 1], [0, 1], color='navy', lw=2, linestyle='--')\\n\",\n \"plt.xlim([0.0, 1.0])\\n\",\n \"plt.ylim([0.0, 1.05])\\n\",\n \"plt.xlabel('False Positive Rate')\\n\",\n \"plt.ylabel('True Positive Rate')\\n\",\n \"plt.title('Receiver operating characteristic')\\n\",\n \"plt.legend(loc=\\\"lower right\\\")\\n\",\n \"plt.show()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"40df8f20-6218-4576-ad59-eb55be73cd0c\",\n \"metadata\": {},\n \"source\": [\n \"Interpreting these results calls for a basic understanding of evaluation metrics. \\n\",\n \"\\n\",\n \"The confusion matrix gives a detailed breakdown of how our predictions measure up to the actual labels, while the ROC curve shows us the trade-off between the true positive rate (sensitivity) and the false positive rate (1 - specificity) at various threshold settings. The area under the ROC curve (AUC-ROC) is a measure of the overall performance of the model, with 1.0 representing perfect classification and 0.5 indicating a model no better than random guessing.\\n\",\n \"** **\\n\",\n \"Finally, to see our model in action, let's use it to infer the sentiment of a sample text.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 9,\n \"id\": \"ef6449a2-1295-4c3f-b831-4a61da9ce39c\",\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Positive sentiment\\n\"\n ]\n }\n ],\n \"source\": [\n \"# Step 7: Inference on a new sample\\n\",\n \"sample_text = \\\"This is a fantastic movie. I really enjoyed it.\\\"\\n\",\n \"sample_inputs = tokenizer(sample_text, padding=\\\"max_length\\\", truncation=True, max_length=512, return_tensors=\\\"pt\\\")\\n\",\n \"\\n\",\n \"# Move inputs to device (if GPU available)\\n\",\n \"sample_inputs.to(training_args.device)\\n\",\n \"\\n\",\n \"# Make prediction\\n\",\n \"predictions = model(**sample_inputs)\\n\",\n \"predicted_class = predictions.logits.argmax(-1).item()\\n\",\n \"\\n\",\n \"if predicted_class == 1:\\n\",\n \" print(\\\"Positive sentiment\\\")\\n\",\n \"else:\\n\",\n \" print(\\\"Negative sentiment\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"168ebb28-0be4-43eb-9942-133296ceb56b\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Closing Thoughts\\n\",\n \"BERT represents an enormous leap forward in our ability to model and understand natural language. It's become a cornerstone for NLP tasks due to its superior handling of context within language. With this tutorial, my goal was to give you a practical introduction to BERT, detailing its basic theory and showing its application in Python using Hugging Face's transformers library.\\n\",\n \"\\n\",\n \"By walking through an example of sentiment analysis on IMDb movie reviews, I hope you've gained a clear understanding of how to apply BERT to real-world NLP problems. The Python code I've included here can be adjusted and extended to tackle different tasks and datasets, paving the way for even more sophisticated and accurate language models.\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.9.16\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 5\n}\n"
},
{
"path": "recommender/published_notebooks/recommendation_python_lightfm.ipynb",
"content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Recommendation in Python: LighFM\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 1,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stderr\",\n \"output_type\": \"stream\",\n \"text\": [\n \"/usr/local/lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.py:29: DeprecationWarning: numpy.core.umath_tests is an internal NumPy module and should not be imported. It will be removed in a future NumPy release.\\n\",\n \" from numpy.core.umath_tests import inner1d\\n\"\n ]\n }\n ],\n \"source\": [\n \"# import dependent libraries\\n\",\n \"import pandas as pd\\n\",\n \"import os\\n\",\n \"from scipy.sparse import csr_matrix\\n\",\n \"import numpy as np\\n\",\n \"from IPython.display import display_html\\n\",\n \"import warnings\\n\",\n \"\\n\",\n \"import matplotlib.pyplot as plt\\n\",\n \"from matplotlib.gridspec import GridSpec\\n\",\n \"import seaborn as sns\\n\",\n \"%matplotlib inline\\n\",\n \"\\n\",\n \"from lightfm.cross_validation import random_train_test_split\\n\",\n \"from lightfm.evaluation import auc_score, precision_at_k, recall_at_k\\n\",\n \"from lightfm import LightFM\\n\",\n \"from skopt import forest_minimize\\n\",\n \"\\n\",\n \"def display_side_by_side(*args):\\n\",\n \" html_str = ''\\n\",\n \" for df in args:\\n\",\n \" html_str += df.to_html()\\n\",\n \" display_html(html_str.replace(\\n\",\n \" 'table', 'table style=\\\"display:inline\\\"'), raw=True)\\n\",\n \"\\n\",\n \"\\n\",\n \"# update the working directory to the root of the project\\n\",\n \"os.chdir('..')\\n\",\n \"warnings.filterwarnings(\\\"ignore\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"### Goodreads Data\\n\",\n \"\\n\",\n \"The datasets were collected in late 2017 from goodreads.com, where we only scraped users' public shelves, i.e. everyone can see it on web without login. User IDs and review IDs are anonymized. \\n\",\n \"\\n\",\n \"We collected these datasets for academic use only. Please do not redistribute them or use for commercial purposes. \\n\",\n \"\\n\",\n \"\\n\",\n \"There are three groups of datasets: (1) meta-data of the books, (2) user-book interactions (users' public shelves) and (3) users' detailed book reviews. These datasets can be merged together by matching book/user/review ids. For the purposes of this tutorial, we'll be using only the former two.\\n\",\n \"\\n\",\n \"You can download the dataset using in this article from here:\\n\",\n \"1. Books Metadata: https://drive.google.com/uc?id=1H6xUV48D5sa2uSF_BusW-IBJ7PCQZTS1\\n\",\n \"2. User-Book Interactions: https://drive.google.com/uc?id=17G5_MeSWuhYnD4fGJMvKRSOlBqCCimxJ\\n\",\n \"\\n\",\n \"#### Load Raw Data\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 2,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"CPU times: user 2min 10s, sys: 12.8 s, total: 2min 23s\\n\",\n \"Wall time: 2min 31s\\n\"\n ]\n }\n ],\n \"source\": [\n \"%%time\\n\",\n \"books_metadata = pd.read_json('./data/goodreads_books_poetry.json', lines=True)\\n\",\n \"interactions = pd.read_json('./data/goodreads_interactions_poetry.json', lines=True)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Data Inspection & Preparation: Books Metadata\\n\",\n \"\\n\",\n \"Let's start by inspecting the books' metadata information. To develop a reliable and robust ML model, it is essential to get a thorough understanding of the available data.\\n\",\n \"\\n\",\n \"As the first step, let's take a look at all the available fields, and sample data\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 3,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"text/plain\": [\n \"array([u'asin', u'authors', u'average_rating', u'book_id',\\n\",\n \" u'country_code', u'description', u'edition_information', u'format',\\n\",\n \" u'image_url', u'is_ebook', u'isbn', u'isbn13', u'kindle_asin',\\n\",\n \" u'language_code', u'link', u'num_pages', u'popular_shelves',\\n\",\n \" u'publication_day', u'publication_month', u'publication_year',\\n\",\n \" u'publisher', u'ratings_count', u'series', u'similar_books',\\n\",\n \" u'text_reviews_count', u'title', u'title_without_series', u'url',\\n\",\n \" u'work_id'], dtype=object)\"\n ]\n },\n \"execution_count\": 3,\n \"metadata\": {},\n \"output_type\": \"execute_result\"\n }\n ],\n \"source\": [\n \"books_metadata.columns.values\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 4,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"text/html\": [\n \"
\\n\",\n \"\\n\",\n \"
\\n\",\n \" \\n\",\n \"
\\n\",\n \"
\\n\",\n \"
asin
\\n\",\n \"
authors
\\n\",\n \"
average_rating
\\n\",\n \"
book_id
\\n\",\n \"
country_code
\\n\",\n \"
description
\\n\",\n \"
edition_information
\\n\",\n \"
format
\\n\",\n \"
image_url
\\n\",\n \"
is_ebook
\\n\",\n \"
...
\\n\",\n \"
publication_year
\\n\",\n \"
publisher
\\n\",\n \"
ratings_count
\\n\",\n \"
series
\\n\",\n \"
similar_books
\\n\",\n \"
text_reviews_count
\\n\",\n \"
title
\\n\",\n \"
title_without_series
\\n\",\n \"
url
\\n\",\n \"
work_id
\\n\",\n \"
\\n\",\n \" \\n\",\n \" \\n\",\n \"
\\n\",\n \"
3086
\\n\",\n \"
\\n\",\n \"
[{u'author_id': u'5031312', u'role': u''}, {u'...
\\n\",\n \"
4.06
\\n\",\n \"
3656020
\\n\",\n \"
US
\\n\",\n \"
A landmark of world literature, The Divine Com...
\\n\",\n \"
\\n\",\n \"
Hardcover
\\n\",\n \"
https://images.gr-assets.com/books/1397524730m...
\\n\",\n \"
false
\\n\",\n \"
...
\\n\",\n \"
2008
\\n\",\n \"
Barnes and Noble
\\n\",\n \"
462
\\n\",\n \"
[444230]
\\n\",\n \"
[38154, 51799, 3311228, 13767037, 138144, 8756...
\\n\",\n \"
56
\\n\",\n \"
The Divine Comedy
\\n\",\n \"
The Divine Comedy
\\n\",\n \"
https://www.goodreads.com/book/show/3656020-th...
\\n\",\n \"
809248
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
18621
\\n\",\n \"
\\n\",\n \"
[{u'author_id': u'152483', u'role': u'Editor'}]
\\n\",\n \"
3.30
\\n\",\n \"
260876
\\n\",\n \"
US
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
Hardcover
\\n\",\n \"
https://s.gr-assets.com/assets/nophoto/book/11...
\\n\",\n \"
false
\\n\",\n \"
...
\\n\",\n \"
1973
\\n\",\n \"
OUP Oxford
\\n\",\n \"
33
\\n\",\n \"
[]
\\n\",\n \"
[]
\\n\",\n \"
4
\\n\",\n \"
The Homeric Hymn to Demeter
\\n\",\n \"
The Homeric Hymn to Demeter
\\n\",\n \"
https://www.goodreads.com/book/show/260876.The...
\\n\",\n \"
252847
\\n\",\n \"
\\n\",\n \" \\n\",\n \"
\\n\",\n \"
2 rows × 29 columns
\\n\",\n \"
\"\n ],\n \"text/plain\": [\n \" asin authors average_rating \\\\\\n\",\n \"3086 [{u'author_id': u'5031312', u'role': u''}, {u'... 4.06 \\n\",\n \"18621 [{u'author_id': u'152483', u'role': u'Editor'}] 3.30 \\n\",\n \"\\n\",\n \" book_id country_code \\\\\\n\",\n \"3086 3656020 US \\n\",\n \"18621 260876 US \\n\",\n \"\\n\",\n \" description edition_information \\\\\\n\",\n \"3086 A landmark of world literature, The Divine Com... \\n\",\n \"18621 \\n\",\n \"\\n\",\n \" format image_url is_ebook \\\\\\n\",\n \"3086 Hardcover https://images.gr-assets.com/books/1397524730m... false \\n\",\n \"18621 Hardcover https://s.gr-assets.com/assets/nophoto/book/11... false \\n\",\n \"\\n\",\n \" ... publication_year publisher ratings_count series \\\\\\n\",\n \"3086 ... 2008 Barnes and Noble 462 [444230] \\n\",\n \"18621 ... 1973 OUP Oxford 33 [] \\n\",\n \"\\n\",\n \" similar_books text_reviews_count \\\\\\n\",\n \"3086 [38154, 51799, 3311228, 13767037, 138144, 8756... 56 \\n\",\n \"18621 [] 4 \\n\",\n \"\\n\",\n \" title title_without_series \\\\\\n\",\n \"3086 The Divine Comedy The Divine Comedy \\n\",\n \"18621 The Homeric Hymn to Demeter The Homeric Hymn to Demeter \\n\",\n \"\\n\",\n \" url work_id \\n\",\n \"3086 https://www.goodreads.com/book/show/3656020-th... 809248 \\n\",\n \"18621 https://www.goodreads.com/book/show/260876.The... 252847 \\n\",\n \"\\n\",\n \"[2 rows x 29 columns]\"\n ]\n },\n \"execution_count\": 4,\n \"metadata\": {},\n \"output_type\": \"execute_result\"\n }\n ],\n \"source\": [\n \"books_metadata.sample(2)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 5,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"text/plain\": [\n \"(36514, 29)\"\n ]\n },\n \"execution_count\": 5,\n \"metadata\": {},\n \"output_type\": \"execute_result\"\n }\n ],\n \"source\": [\n \"books_metadata.shape\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"While all the available information is vital to extract contextual information to be able to train a better recommendation system, for this example, we'll only focus on the selected fields that require minimal manipulation.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 6,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"text/html\": [\n \"
\\n\",\n \"\\n\",\n \"
\\n\",\n \" \\n\",\n \"
\\n\",\n \"
\\n\",\n \"
book_id
\\n\",\n \"
average_rating
\\n\",\n \"
is_ebook
\\n\",\n \"
num_pages
\\n\",\n \"
publication_year
\\n\",\n \"
ratings_count
\\n\",\n \"
language_code
\\n\",\n \"
\\n\",\n \" \\n\",\n \" \\n\",\n \"
\\n\",\n \"
7326
\\n\",\n \"
333171
\\n\",\n \"
4.02
\\n\",\n \"
false
\\n\",\n \"
176
\\n\",\n \"
1999
\\n\",\n \"
14
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
33952
\\n\",\n \"
32940040
\\n\",\n \"
4.00
\\n\",\n \"
false
\\n\",\n \"
110
\\n\",\n \"
2017
\\n\",\n \"
38
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
11012
\\n\",\n \"
702390
\\n\",\n \"
4.19
\\n\",\n \"
false
\\n\",\n \"
336
\\n\",\n \"
2002
\\n\",\n \"
53
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
31968
\\n\",\n \"
1271413
\\n\",\n \"
4.19
\\n\",\n \"
false
\\n\",\n \"
\\n\",\n \"
1992
\\n\",\n \"
19
\\n\",\n \"
eng
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
31512
\\n\",\n \"
955328
\\n\",\n \"
4.54
\\n\",\n \"
false
\\n\",\n \"
106
\\n\",\n \"
1994
\\n\",\n \"
28
\\n\",\n \"
\\n\",\n \"
\\n\",\n \" \\n\",\n \"
\\n\",\n \"
\"\n ],\n \"text/plain\": [\n \" book_id average_rating is_ebook num_pages publication_year \\\\\\n\",\n \"7326 333171 4.02 false 176 1999 \\n\",\n \"33952 32940040 4.00 false 110 2017 \\n\",\n \"11012 702390 4.19 false 336 2002 \\n\",\n \"31968 1271413 4.19 false 1992 \\n\",\n \"31512 955328 4.54 false 106 1994 \\n\",\n \"\\n\",\n \" ratings_count language_code \\n\",\n \"7326 14 \\n\",\n \"33952 38 \\n\",\n \"11012 53 \\n\",\n \"31968 19 eng \\n\",\n \"31512 28 \"\n ]\n },\n \"execution_count\": 6,\n \"metadata\": {},\n \"output_type\": \"execute_result\"\n }\n ],\n \"source\": [\n \"# Limit the books metadata to selected fields\\n\",\n \"books_metadata_selected = books_metadata[['book_id', 'average_rating', 'is_ebook', 'num_pages', \\n\",\n \" 'publication_year', 'ratings_count', 'language_code']]\\n\",\n \"books_metadata_selected.sample(5)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"Now that we have the data with selected fields, next, we'll run it through pandas profiler to perform preliminary exploratory data analysis to help us better understand the available data\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 7,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import pandas_profiling\\n\",\n \"import numpy as np\\n\",\n \"\\n\",\n \"# replace blank cells with NaN\\n\",\n \"books_metadata_selected.replace('', np.nan, inplace=True)\\n\",\n \"\\n\",\n \"# not taking book_id into the profiler report\\n\",\n \"profile = pandas_profiling.ProfileReport(books_metadata_selected[['average_rating', 'is_ebook', 'num_pages', \\n\",\n \" 'publication_year', 'ratings_count']])\\n\",\n \"profile.to_file('./results/profiler_books_metadata_1.html')\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"\\n\",\n \"Considering the results from the profiler, we'll perform following transformations to the dataset:\\n\",\n \"- Replace the missing value of categorical values with another value to create a new category\\n\",\n \"- Convert bin values for numeric variables into discrete intervals\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 8,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# using pandas cut method to convert fields into discrete intervals\\n\",\n \"books_metadata_selected['num_pages'].replace(np.nan, -1, inplace=True)\\n\",\n \"books_metadata_selected['num_pages'] = pd.to_numeric(books_metadata_selected['num_pages'])\\n\",\n \"books_metadata_selected['num_pages'] = pd.cut(books_metadata_selected['num_pages'], bins=25)\\n\",\n \"\\n\",\n \"# rounding ratings to neares .5 score\\n\",\n \"books_metadata_selected['average_rating'] = books_metadata_selected['average_rating'].apply(lambda x: round(x*2)/2)\\n\",\n \"\\n\",\n \"# using pandas qcut method to convert fields into quantile-based discrete intervals\\n\",\n \"books_metadata_selected['ratings_count'] = pd.qcut(books_metadata_selected['ratings_count'], 25)\\n\",\n \"\\n\",\n \"# replacing missing values to year 2100\\n\",\n \"books_metadata_selected['publication_year'].replace(np.nan, 2100, inplace=True)\\n\",\n \"\\n\",\n \"# replacing missing values to 'unknown'\\n\",\n \"books_metadata_selected['language_code'].replace(np.nan, 'unknown', inplace=True)\\n\",\n \"\\n\",\n \"\\n\",\n \"# convert is_ebook column into 1/0 where true=1 and false=0\\n\",\n \"books_metadata_selected['is_ebook'] = books_metadata_selected.is_ebook.map(\\n\",\n \" lambda x: 1.0*(x == 'true'))\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 9,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"profile = pandas_profiling.ProfileReport(books_metadata_selected[['average_rating', 'is_ebook', 'num_pages', \\n\",\n \" 'publication_year', 'ratings_count']])\\n\",\n \"profile.to_file('./results/profiler_books_metadata_2.html')\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 10,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"text/html\": [\n \"
\"\n ],\n \"text/plain\": [\n \" user_id book_id is_read rating\\n\",\n \"1565125 1f9f847ce20c58c12ac7f1e815df5d7f 22267492 0.0 0\\n\",\n \"38635 9cd8fb7c611544b2e09ef5226ce8dbcb 24874353 1.0 3\\n\",\n \"847749 f2bac05b3932fe7c68960041744e5058 310336 0.0 0\\n\",\n \"1046835 700a4402bc8f09166fc98bc48bc0c525 11347806 0.0 0\\n\",\n \"486934 adfc7584a1fef507364c722ad6e3c106 12966360 0.0 0\\n\",\n \"860513 5a1c12910a59122c8653307c06cf130f 27494 1.0 5\\n\",\n \"656911 787f452029e97df574375ace96c5a782 119234 1.0 3\\n\",\n \"1800541 ec728fff5c2c096888e400118c0bc1b0 76542 0.0 0\\n\",\n \"498215 c8ff09eaf35e3d6b519cd5594139224c 69547 0.0 0\\n\",\n \"12693 38c2feb6b72d473f1b710516e018244e 493428 0.0 0\"\n ]\n },\n \"execution_count\": 17,\n \"metadata\": {},\n \"output_type\": \"execute_result\"\n }\n ],\n \"source\": [\n \"interactions_selected.sample(10)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"Since we have two fields denoting interaction between a user and a book, `is_read` and `rating` - let's see how many data points we have where the user hasn't read the book but have given the ratings.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 18,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"text/html\": [\n \"
\\n\",\n \"\\n\",\n \"
\\n\",\n \" \\n\",\n \"
\\n\",\n \"
rating
\\n\",\n \"
0
\\n\",\n \"
1
\\n\",\n \"
2
\\n\",\n \"
3
\\n\",\n \"
4
\\n\",\n \"
5
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
is_read
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
\\n\",\n \" \\n\",\n \" \\n\",\n \"
\\n\",\n \"
0.0
\\n\",\n \"
1420740.0
\\n\",\n \"
NaN
\\n\",\n \"
NaN
\\n\",\n \"
NaN
\\n\",\n \"
NaN
\\n\",\n \"
NaN
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
1.0
\\n\",\n \"
84551.0
\\n\",\n \"
20497.0
\\n\",\n \"
64084.0
\\n\",\n \"
237942.0
\\n\",\n \"
405565.0
\\n\",\n \"
500971.0
\\n\",\n \"
\\n\",\n \" \\n\",\n \"
\\n\",\n \"
\"\n ],\n \"text/plain\": [\n \"rating 0 1 2 3 4 5\\n\",\n \"is_read \\n\",\n \"0.0 1420740.0 NaN NaN NaN NaN NaN\\n\",\n \"1.0 84551.0 20497.0 64084.0 237942.0 405565.0 500971.0\"\n ]\n },\n \"execution_count\": 18,\n \"metadata\": {},\n \"output_type\": \"execute_result\"\n }\n ],\n \"source\": [\n \"interactions_selected.groupby(['rating', 'is_read']).size().reset_index().pivot(columns='rating', index='is_read', values=0)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"From the above results, we can conclusively infer that users with ratings >= 1 have all read the book. Therefore, we'll use the `ratings` as the final score, drop interactions where `is_read` is false, and limit interactions from random 500 users to limit the data size for further analysis \"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 19,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"text/html\": [\n \"
\\n\",\n \"\\n\",\n \"
\\n\",\n \" \\n\",\n \"
\\n\",\n \"
\\n\",\n \"
user_id
\\n\",\n \"
book_id
\\n\",\n \"
rating
\\n\",\n \"
\\n\",\n \" \\n\",\n \" \\n\",\n \"
\\n\",\n \"
368671
\\n\",\n \"
08184d08ae08d26bacd5c00230141ce5
\\n\",\n \"
6130588
\\n\",\n \"
5
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
1755032
\\n\",\n \"
cae938e69bbc27a393153aed6861b9fe
\\n\",\n \"
8744427
\\n\",\n \"
5
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
2675075
\\n\",\n \"
26ef759d338ff493bdcd501fb05127db
\\n\",\n \"
1420
\\n\",\n \"
5
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
2613547
\\n\",\n \"
44af3a5167e24e9fe4761ef20487e675
\\n\",\n \"
662635
\\n\",\n \"
5
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
794869
\\n\",\n \"
e471985fd3f272455e048aa7f7f05b73
\\n\",\n \"
1420
\\n\",\n \"
5
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
341820
\\n\",\n \"
d5708c6364fa01c5c0c2a31427c10664
\\n\",\n \"
6555075
\\n\",\n \"
5
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
1160855
\\n\",\n \"
b8ee26f9150c1e6d46a900c423b44d7b
\\n\",\n \"
133619
\\n\",\n \"
5
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
1015659
\\n\",\n \"
4e7d97a0afddd934f01e7d7c22eb3ef1
\\n\",\n \"
240258
\\n\",\n \"
4
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
917119
\\n\",\n \"
c6a203dbca8acc76a49bc68808ccce33
\\n\",\n \"
11030407
\\n\",\n \"
0
\\n\",\n \"
\\n\",\n \"
\\n\",\n \"
1024204
\\n\",\n \"
cc8abbdd380a5dc5c0d068f99b4eab1c
\\n\",\n \"
160959
\\n\",\n \"
0
\\n\",\n \"
\\n\",\n \" \\n\",\n \"
\\n\",\n \"
\"\n ],\n \"text/plain\": [\n \" user_id book_id rating\\n\",\n \"368671 08184d08ae08d26bacd5c00230141ce5 6130588 5\\n\",\n \"1755032 cae938e69bbc27a393153aed6861b9fe 8744427 5\\n\",\n \"2675075 26ef759d338ff493bdcd501fb05127db 1420 5\\n\",\n \"2613547 44af3a5167e24e9fe4761ef20487e675 662635 5\\n\",\n \"794869 e471985fd3f272455e048aa7f7f05b73 1420 5\\n\",\n \"341820 d5708c6364fa01c5c0c2a31427c10664 6555075 5\\n\",\n \"1160855 b8ee26f9150c1e6d46a900c423b44d7b 133619 5\\n\",\n \"1015659 4e7d97a0afddd934f01e7d7c22eb3ef1 240258 4\\n\",\n \"917119 c6a203dbca8acc76a49bc68808ccce33 11030407 0\\n\",\n \"1024204 cc8abbdd380a5dc5c0d068f99b4eab1c 160959 0\"\n ]\n },\n \"execution_count\": 19,\n \"metadata\": {},\n \"output_type\": \"execute_result\"\n }\n ],\n \"source\": [\n \"import random\\n\",\n \"\\n\",\n \"interactions_selected = interactions_selected.loc[interactions_selected['is_read']==1, ['user_id', 'book_id', 'rating']]\\n\",\n \"\\n\",\n \"interactions_selected = interactions_selected[interactions_selected['user_id'].isin(random.sample(list(interactions_selected['user_id'].unique()), \\n\",\n \" k=5000))]\\n\",\n \"\\n\",\n \"interactions_selected.sample(10)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 20,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"text/plain\": [\n \"(23989, 3)\"\n ]\n },\n \"execution_count\": 20,\n \"metadata\": {},\n \"output_type\": \"execute_result\"\n }\n ],\n \"source\": [\n \"interactions_selected.shape\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Data Preprocessing\\n\",\n \"\\n\",\n \"Now, let's transform the available data into CSR sparse matrix that can be used for matrix operations. We will start by the process by creating books_metadata matrix which is np.float64 csr_matrix of shape ([n_books, n_books_features]) – Each row contains that book's weights over features. However, before we create a sparse matrix, we'll first create a item dictionar for future references\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 21,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"item_dict ={}\\n\",\n \"df = books_metadata[['book_id', 'title']].sort_values('book_id').reset_index()\\n\",\n \"\\n\",\n \"for i in range(df.shape[0]):\\n\",\n \" item_dict[(df.loc[i,'book_id'])] = df.loc[i,'title']\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 22,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"text/html\": [\n \"
![\\\"The](\\\"https://s3.amazonaws.com/assets.datacamp.com/production/project_158/img/nips_logo.png\\\"){kind=logo}
\\n\",\n \"\\n\",\n \"Let’s start by looking at the content of the file\\n\",\n \"\\n\",\n \"** **\\n\",\n \"#### Step 1: Loading Data\\n\",\n \"** **\\n\",\n \"\\n\",\n \"For this tutorial, we’ll use the dataset of papers published in NIPS conference. The NIPS conference (Neural Information Processing Systems) is one of the most prestigious yearly events in the machine learning community. The CSV data file contains information on the different NIPS papers that were published from 1987 until 2016 (29 years!). These papers discuss a wide variety of topics in machine learning, from neural networks to optimization methods, and many more.\\n\",\n \"\\n\",\n \"Let’s start by looking at the content of the file\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import zipfile\\n\",\n \"import pandas as pd\\n\",\n \"import os\\n\",\n \"\\n\",\n \"# Open the zip file\\n\",\n \"with zipfile.ZipFile(\\\"./data/NIPS Papers.zip\\\", \\\"r\\\") as zip_ref:\\n\",\n \" # Extract the file to a temporary directory\\n\",\n \" zip_ref.extractall(\\\"temp\\\")\\n\",\n \"\\n\",\n \"# Read the CSV file into a pandas DataFrame\\n\",\n \"papers = pd.read_csv(\\\"temp/NIPS Papers/papers.csv\\\")\\n\",\n \"\\n\",\n \"# Print head\\n\",\n \"papers.head()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Step 2: Data Cleaning\\n\",\n \"** **\\n\",\n \"\\n\",\n \"Since the goal of this analysis is to perform topic modeling, we will solely focus on the text data from each paper, and drop other metadata columns\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Remove the columns\\n\",\n \"papers = papers.drop(columns=['id', 'title', 'abstract', \\n\",\n \" 'event_type', 'pdf_name', 'year'], axis=1)\\n\",\n \"\\n\",\n \"# sample only 100 papers\\n\",\n \"papers = papers.sample(100)\\n\",\n \"\\n\",\n \"# Print out the first rows of papers\\n\",\n \"papers.head()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"##### Remove punctuation/lower casing\\n\",\n \"\\n\",\n \"Next, let’s perform a simple preprocessing on the content of paper_text column to make them more amenable for analysis, and reliable results. To do that, we’ll use a regular expression to remove any punctuation, and then lowercase the text\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Load the regular expression library\\n\",\n \"import re\\n\",\n \"\\n\",\n \"# Remove punctuation\\n\",\n \"papers['paper_text_processed'] = papers['paper_text'].map(lambda x: re.sub('[,\\\\.!?]', '', x))\\n\",\n \"\\n\",\n \"# Convert the titles to lowercase\\n\",\n \"papers['paper_text_processed'] = papers['paper_text_processed'].map(lambda x: x.lower())\\n\",\n \"\\n\",\n \"# Print out the first rows of papers\\n\",\n \"papers['paper_text_processed'].head()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"##### Tokenize words and further clean-up text\\n\",\n \"\\n\",\n \"Let’s tokenize each sentence into a list of words, removing punctuations and unnecessary characters altogether.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import gensim\\n\",\n \"from gensim.utils import simple_preprocess\\n\",\n \"\\n\",\n \"def sent_to_words(sentences):\\n\",\n \" for sentence in sentences:\\n\",\n \" yield(gensim.utils.simple_preprocess(str(sentence), deacc=True)) # deacc=True removes punctuations\\n\",\n \"\\n\",\n \"data = papers.paper_text_processed.values.tolist()\\n\",\n \"data_words = list(sent_to_words(data))\\n\",\n \"\\n\",\n \"print(data_words[:1][0][:30])\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Step 3: Phrase Modeling: Bigram and Trigram Models\\n\",\n \"** **\\n\",\n \"\\n\",\n \"Bigrams are two words frequently occurring together in the document. Trigrams are 3 words frequently occurring. Some examples in our example are: 'back_bumper', 'oil_leakage', 'maryland_college_park' etc.\\n\",\n \"\\n\",\n \"Gensim's Phrases model can build and implement the bigrams, trigrams, quadgrams and more. The two important arguments to Phrases are min_count and threshold.\\n\",\n \"\\n\",\n \"*The higher the values of these param, the harder it is for words to be combined.*\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Build the bigram and trigram models\\n\",\n \"bigram = gensim.models.Phrases(data_words, min_count=5, threshold=100) # higher threshold fewer phrases.\\n\",\n \"trigram = gensim.models.Phrases(bigram[data_words], threshold=100) \\n\",\n \"\\n\",\n \"# Faster way to get a sentence clubbed as a trigram/bigram\\n\",\n \"bigram_mod = gensim.models.phrases.Phraser(bigram)\\n\",\n \"trigram_mod = gensim.models.phrases.Phraser(trigram)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"#### Remove Stopwords, Make Bigrams and Lemmatize\\n\",\n \"\\n\",\n \"The phrase models are ready. Let’s define the functions to remove the stopwords, make trigrams and lemmatization and call them sequentially.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# NLTK Stop words\\n\",\n \"import nltk\\n\",\n \"nltk.download('stopwords')\\n\",\n \"from nltk.corpus import stopwords\\n\",\n \"\\n\",\n \"stop_words = stopwords.words('english')\\n\",\n \"stop_words.extend(['from', 'subject', 're', 'edu', 'use'])\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Define functions for stopwords, bigrams, trigrams and lemmatization\\n\",\n \"def remove_stopwords(texts):\\n\",\n \" return [[word for word in simple_preprocess(str(doc)) if word not in stop_words] for doc in texts]\\n\",\n \"\\n\",\n \"def make_bigrams(texts):\\n\",\n \" return [bigram_mod[doc] for doc in texts]\\n\",\n \"\\n\",\n \"def make_trigrams(texts):\\n\",\n \" return [trigram_mod[bigram_mod[doc]] for doc in texts]\\n\",\n \"\\n\",\n \"def lemmatization(texts, allowed_postags=['NOUN', 'ADJ', 'VERB', 'ADV']):\\n\",\n \" \\\"\\\"\\\"https://spacy.io/api/annotation\\\"\\\"\\\"\\n\",\n \" texts_out = []\\n\",\n \" for sent in texts:\\n\",\n \" doc = nlp(\\\" \\\".join(sent)) \\n\",\n \" texts_out.append([token.lemma_ for token in doc if token.pos_ in allowed_postags])\\n\",\n \" return texts_out\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"Let's call the functions in order.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"!python -m spacy download en_core_web_sm\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import spacy\\n\",\n \"\\n\",\n \"# Remove Stop Words\\n\",\n \"data_words_nostops = remove_stopwords(data_words)\\n\",\n \"\\n\",\n \"# Form Bigrams\\n\",\n \"data_words_bigrams = make_bigrams(data_words_nostops)\\n\",\n \"\\n\",\n \"# Initialize spacy 'en' model, keeping only tagger component (for efficiency)\\n\",\n \"nlp = spacy.load(\\\"en_core_web_sm\\\", disable=['parser', 'ner'])\\n\",\n \"\\n\",\n \"# Do lemmatization keeping only noun, adj, vb, adv\\n\",\n \"data_lemmatized = lemmatization(data_words_bigrams, allowed_postags=['NOUN', 'ADJ', 'VERB', 'ADV'])\\n\",\n \"\\n\",\n \"print(data_lemmatized[:1][0][:30])\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Step 4: Data transformation: Corpus and Dictionary\\n\",\n \"** **\\n\",\n \"\\n\",\n \"The two main inputs to the LDA topic model are the dictionary(id2word) and the corpus. Let’s create them.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import gensim.corpora as corpora\\n\",\n \"\\n\",\n \"# Create Dictionary\\n\",\n \"id2word = corpora.Dictionary(data_lemmatized)\\n\",\n \"\\n\",\n \"# Create Corpus\\n\",\n \"texts = data_lemmatized\\n\",\n \"\\n\",\n \"# Term Document Frequency\\n\",\n \"corpus = [id2word.doc2bow(text) for text in texts]\\n\",\n \"\\n\",\n \"# View\\n\",\n \"print(corpus[:1][0][:30])\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Step 5: Base Model \\n\",\n \"** **\\n\",\n \"\\n\",\n \"We have everything required to train the base LDA model. In addition to the corpus and dictionary, you need to provide the number of topics as well. Apart from that, alpha and eta are hyperparameters that affect sparsity of the topics. According to the Gensim docs, both defaults to 1.0/num_topics prior (we'll use default for the base model).\\n\",\n \"\\n\",\n \"chunksize controls how many documents are processed at a time in the training algorithm. Increasing chunksize will speed up training, at least as long as the chunk of documents easily fit into memory.\\n\",\n \"\\n\",\n \"passes controls how often we train the model on the entire corpus (set to 10). Another word for passes might be \\\"epochs\\\". iterations is somewhat technical, but essentially it controls how often we repeat a particular loop over each document. It is important to set the number of \\\"passes\\\" and \\\"iterations\\\" high enough.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Build LDA model\\n\",\n \"lda_model = gensim.models.LdaMulticore(corpus=corpus,\\n\",\n \" id2word=id2word,\\n\",\n \" num_topics=10, \\n\",\n \" random_state=100,\\n\",\n \" chunksize=100,\\n\",\n \" passes=10,\\n\",\n \" per_word_topics=True)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"The above LDA model is built with 10 different topics where each topic is a combination of keywords and each keyword contributes a certain weightage to the topic.\\n\",\n \"\\n\",\n \"You can see the keywords for each topic and the weightage(importance) of each keyword using `lda_model.print_topics()`\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from pprint import pprint\\n\",\n \"\\n\",\n \"# Print the Keyword in the 10 topics\\n\",\n \"pprint(lda_model.print_topics())\\n\",\n \"doc_lda = lda_model[corpus]\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"#### Compute Model Perplexity and Coherence Score\\n\",\n \"\\n\",\n \"Let's calculate the baseline coherence score\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from gensim.models import CoherenceModel\\n\",\n \"\\n\",\n \"# Compute Coherence Score\\n\",\n \"coherence_model_lda = CoherenceModel(model=lda_model, texts=data_lemmatized, dictionary=id2word, coherence='c_v')\\n\",\n \"coherence_lda = coherence_model_lda.get_coherence()\\n\",\n \"print('Coherence Score: ', coherence_lda)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Step 6: Hyperparameter tuning\\n\",\n \"** **\\n\",\n \"First, let's differentiate between model hyperparameters and model parameters :\\n\",\n \"\\n\",\n \"- `Model hyperparameters` can be thought of as settings for a machine learning algorithm that are tuned by the data scientist before training. Examples would be the number of trees in the random forest, or in our case, number of topics K\\n\",\n \"\\n\",\n \"- `Model parameters` can be thought of as what the model learns during training, such as the weights for each word in a given topic.\\n\",\n \"\\n\",\n \"Now that we have the baseline coherence score for the default LDA model, let's perform a series of sensitivity tests to help determine the following model hyperparameters: \\n\",\n \"- Number of Topics (K)\\n\",\n \"- Dirichlet hyperparameter alpha: Document-Topic Density\\n\",\n \"- Dirichlet hyperparameter beta: Word-Topic Density\\n\",\n \"\\n\",\n \"We'll perform these tests in sequence, one parameter at a time by keeping others constant and run them over the two difference validation corpus sets. We'll use `C_v` as our choice of metric for performance comparison \"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# supporting function\\n\",\n \"def compute_coherence_values(corpus, dictionary, k, a, b):\\n\",\n \" \\n\",\n \" lda_model = gensim.models.LdaMulticore(corpus=corpus,\\n\",\n \" id2word=dictionary,\\n\",\n \" num_topics=k, \\n\",\n \" random_state=100,\\n\",\n \" chunksize=100,\\n\",\n \" passes=10,\\n\",\n \" alpha=a,\\n\",\n \" eta=b)\\n\",\n \" \\n\",\n \" coherence_model_lda = CoherenceModel(model=lda_model, texts=data_lemmatized, dictionary=id2word, coherence='c_v')\\n\",\n \" \\n\",\n \" return coherence_model_lda.get_coherence()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"Let's call the function, and iterate it over the range of topics, alpha, and beta parameter values\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import numpy as np\\n\",\n \"import tqdm\\n\",\n \"\\n\",\n \"grid = {}\\n\",\n \"grid['Validation_Set'] = {}\\n\",\n \"\\n\",\n \"# Topics range\\n\",\n \"min_topics = 2\\n\",\n \"max_topics = 11\\n\",\n \"step_size = 1\\n\",\n \"topics_range = range(min_topics, max_topics, step_size)\\n\",\n \"\\n\",\n \"# Alpha parameter\\n\",\n \"alpha = list(np.arange(0.01, 1, 0.3))\\n\",\n \"alpha.append('symmetric')\\n\",\n \"alpha.append('asymmetric')\\n\",\n \"\\n\",\n \"# Beta parameter\\n\",\n \"beta = list(np.arange(0.01, 1, 0.3))\\n\",\n \"beta.append('symmetric')\\n\",\n \"\\n\",\n \"# Validation sets\\n\",\n \"num_of_docs = len(corpus)\\n\",\n \"corpus_sets = [gensim.utils.ClippedCorpus(corpus, int(num_of_docs*0.75)), \\n\",\n \" corpus]\\n\",\n \"\\n\",\n \"corpus_title = ['75% Corpus', '100% Corpus']\\n\",\n \"\\n\",\n \"model_results = {'Validation_Set': [],\\n\",\n \" 'Topics': [],\\n\",\n \" 'Alpha': [],\\n\",\n \" 'Beta': [],\\n\",\n \" 'Coherence': []\\n\",\n \" }\\n\",\n \"\\n\",\n \"# Can take a long time to run\\n\",\n \"if 1 == 1:\\n\",\n \" pbar = tqdm.tqdm(total=(len(beta)*len(alpha)*len(topics_range)*len(corpus_title)))\\n\",\n \" \\n\",\n \" # iterate through validation corpuses\\n\",\n \" for i in range(len(corpus_sets)):\\n\",\n \" # iterate through number of topics\\n\",\n \" for k in topics_range:\\n\",\n \" # iterate through alpha values\\n\",\n \" for a in alpha:\\n\",\n \" # iterare through beta values\\n\",\n \" for b in beta:\\n\",\n \" # get the coherence score for the given parameters\\n\",\n \" cv = compute_coherence_values(corpus=corpus_sets[i], dictionary=id2word, \\n\",\n \" k=k, a=a, b=b)\\n\",\n \" # Save the model results\\n\",\n \" model_results['Validation_Set'].append(corpus_title[i])\\n\",\n \" model_results['Topics'].append(k)\\n\",\n \" model_results['Alpha'].append(a)\\n\",\n \" model_results['Beta'].append(b)\\n\",\n \" model_results['Coherence'].append(cv)\\n\",\n \" \\n\",\n \" pbar.update(1)\\n\",\n \" pd.DataFrame(model_results).to_csv('./results/lda_tuning_results.csv', index=False)\\n\",\n \" pbar.close()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Step 7: Final Model\\n\",\n \"** **\\n\",\n \"\\n\",\n \"Based on external evaluation (Code to be added from Excel based analysis), let's train the final model with parameters yielding highest coherence score\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"num_topics = 8\\n\",\n \"\\n\",\n \"lda_model = gensim.models.LdaMulticore(corpus=corpus,\\n\",\n \" id2word=id2word,\\n\",\n \" num_topics=num_topics, \\n\",\n \" random_state=100,\\n\",\n \" chunksize=100,\\n\",\n \" passes=10,\\n\",\n \" alpha=0.01,\\n\",\n \" eta=0.9)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from pprint import pprint\\n\",\n \"\\n\",\n \"# Print the Keyword in the 10 topics\\n\",\n \"pprint(lda_model.print_topics())\\n\",\n \"doc_lda = lda_model[corpus]\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Step 8: Visualize Results\\n\",\n \"** **\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import pyLDAvis.gensim_models as gensimvis\\n\",\n \"import pickle \\n\",\n \"import pyLDAvis\\n\",\n \"\\n\",\n \"# Visualize the topics\\n\",\n \"pyLDAvis.enable_notebook()\\n\",\n \"\\n\",\n \"LDAvis_data_filepath = os.path.join('./results/ldavis_tuned_'+str(num_topics))\\n\",\n \"\\n\",\n \"# # this is a bit time consuming - make the if statement True\\n\",\n \"# # if you want to execute visualization prep yourself\\n\",\n \"if 1 == 1:\\n\",\n \" LDAvis_prepared = gensimvis.prepare(lda_model, corpus, id2word)\\n\",\n \" with open(LDAvis_data_filepath, 'wb') as f:\\n\",\n \" pickle.dump(LDAvis_prepared, f)\\n\",\n \"\\n\",\n \"# load the pre-prepared pyLDAvis data from disk\\n\",\n \"with open(LDAvis_data_filepath, 'rb') as f:\\n\",\n \" LDAvis_prepared = pickle.load(f)\\n\",\n \"\\n\",\n \"pyLDAvis.save_html(LDAvis_prepared, './results/ldavis_tuned_'+ str(num_topics) +'.html')\\n\",\n \"\\n\",\n \"LDAvis_prepared\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"#### Closing Notes\\n\",\n \"\\n\",\n \"We started with understanding why evaluating the topic model is essential. Next, we reviewed existing methods and scratched the surface of topic coherence, along with the available coherence measures. Then we built a default LDA model using Gensim implementation to establish the baseline coherence score and reviewed practical ways to optimize the LDA hyperparameters.\\n\",\n \"\\n\",\n \"Hopefully, this article has managed to shed light on the underlying topic evaluation strategies, and intuitions behind it.\\n\",\n \"\\n\",\n \"** **\\n\",\n \"#### References:\\n\",\n \"1. http://qpleple.com/perplexity-to-evaluate-topic-models/\\n\",\n \"2. https://www.amazon.com/Machine-Learning-Probabilistic-Perspective-Computation/dp/0262018020\\n\",\n \"3. https://papers.nips.cc/paper/3700-reading-tea-leaves-how-humans-interpret-topic-models.pdf\\n\",\n \"4. https://github.com/mattilyra/pydataberlin-2017/blob/master/notebook/EvaluatingUnsupervisedModels.ipynb\\n\",\n \"5. https://www.machinelearningplus.com/nlp/topic-modeling-gensim-python/\\n\",\n \"6. http://svn.aksw.org/papers/2015/WSDM_Topic_Evaluation/public.pdf\\n\",\n \"7. http://palmetto.aksw.org/palmetto-webapp/\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.9.16\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n"
},
{
"path": "natural-language-processing/topic-modeling/Introduction to Topic Modeling.ipynb",
"content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Introduction\\n\",\n \"##### How to get started with topic modeling using LDA in Python\\n\",\n \"** **\\n\",\n \"Topic Models, in a nutshell, are a type of statistical language models used for uncovering hidden structure in a collection of texts. In a practical and more intuitively, you can think of it as a task of:\\n\",\n \"\\n\",\n \"- **Dimensionality Reduction**, where rather than representing a text T in its feature space as {Word_i: count(Word_i, T) for Word_i in Vocabulary}, you can represent it in a topic space as {Topic_i: Weight(Topic_i, T) for Topic_i in Topics}\\n\",\n \"- **Unsupervised Learning**, where it can be compared to clustering, as in the case of clustering, the number of topics, like the number of clusters, is an output parameter. By doing topic modeling, we build clusters of words rather than clusters of texts. A text is thus a mixture of all the topics, each having a specific weight\\n\",\n \"- **Tagging**, abstract “topics” that occur in a collection of documents that best represents the information in them.\\n\",\n \"\\n\",\n \"There are several existing algorithms you can use to perform the topic modeling. The most common of it are, Latent Semantic Analysis (LSA/LSI), Probabilistic Latent Semantic Analysis (pLSA), and Latent Dirichlet Allocation (LDA)\\n\",\n \"\\n\",\n \"In this tutorial, we’ll take a closer look at LDA, and implement our first topic model using the sklearn implementation in python 2.7\\n\",\n \"\\n\",\n \"### Theoretical Overview\\n\",\n \"LDA is a generative probabilistic model that assumes each topic is a mixture over an underlying set of words, and each document is a mixture of over a set of topic probabilities.\\n\",\n \"\\n\",\n \"\\n\",\n \"\\n\",\n \"We can describe the generative process of LDA as, given the M number of documents, N number of words, and prior K number of topics, the model trains to output:\\n\",\n \"\\n\",\n \"- `psi`, the distribution of words for each topic K\\n\",\n \"- `phi`, the distribution of topics for each document i\\n\",\n \"\\n\",\n \"#### Parameters of LDA\\n\",\n \"\\n\",\n \"- `Alpha parameter` is Dirichlet prior concentration parameter that represents document-topic density — with a higher alpha, documents are assumed to be made up of more topics and result in more specific topic distribution per document.\\n\",\n \"- `Beta parameter` is the same prior concentration parameter that represents topic-word density — with high beta, topics are assumed to made of up most of the words and result in a more specific word distribution per topic.\\n\",\n \"\\n\",\n \"**To read more: https://towardsdatascience.com/end-to-end-topic-modeling-in-python-latent-dirichlet-allocation-lda-35ce4ed6b3e0**\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"### LDA Implementation\\n\",\n \"\\n\",\n \"1. [Loading data](#load_data)\\n\",\n \"2. [Data cleaning](#clean_data)\\n\",\n \"3. [Exploratory analysis](#eda)\\n\",\n \"4. [Prepare data for LDA analysis](#data_preparation)\\n\",\n \"5. [LDA model training](#train_model)\\n\",\n \"6. [Analyzing LDA model results](#results)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"** **\\n\",\n \"For this tutorial, we’ll use the dataset of papers published in NeurIPS (NIPS) conference which is one of the most prestigious yearly events in the machine learning community. The CSV data file contains information on the different NeurIPS papers that were published from 1987 until 2016 (29 years!). These papers discuss a wide variety of topics in machine learning, from neural networks to optimization methods, and many more.\\n\",\n \"\\n\",\n \"