[
  {
    "path": ".gitignore",
    "content": ".DS_Store\nextension/.web-extension-id\nextension/web-ext-artifacts/\n"
  },
  {
    "path": "ATTRIBUTIONS.md",
    "content": "## Icons\n\nbrain_24.png icon licensed under [CC-by 3.0 Unported](https://creativecommons.org/licenses/by/3.0/) from user 'Howcolour' on www.iconfinder.com \n\nsave-icon-16.png icon licensed under [CC-by 3.0](https://creativecommons.org/licenses/by/3.0/) from user 'Bhuvan' from Noun Project\n\nfile-icon-16.png icon licensed under [CC-by 3.0](https://creativecommons.org/licenses/by/3.0/) from user 'Mas Dhimas' from Noun Project\n\n## Helpful Links \n\nCSS Gradient tool used: https://cssgradient.io/\n\nPastel Rainbow color palette by user allyasdf on color-hex: https://www.color-hex.com/color-palette/5361 \n\nCSS Trick - border-top-linear-gradient solution fromL https://michaelharley.net/posts/2021/01/12/how-to-create-a-border-top-linear-gradient/ "
  },
  {
    "path": "LICENSE",
    "content": "Mozilla Public License Version 2.0\n==================================\n\n1. Definitions\n--------------\n\n1.1. \"Contributor\"\n    means each individual or legal entity that creates, contributes to\n    the creation of, or owns Covered Software.\n\n1.2. \"Contributor Version\"\n    means the combination of the Contributions of others (if any) used\n    by a Contributor and that particular Contributor's Contribution.\n\n1.3. \"Contribution\"\n    means Covered Software of a particular Contributor.\n\n1.4. \"Covered Software\"\n    means Source Code Form to which the initial Contributor has attached\n    the notice in Exhibit A, the Executable Form of such Source Code\n    Form, and Modifications of such Source Code Form, in each case\n    including portions thereof.\n\n1.5. \"Incompatible With Secondary Licenses\"\n    means\n\n    (a) that the initial Contributor has attached the notice described\n        in Exhibit B to the Covered Software; or\n\n    (b) that the Covered Software was made available under the terms of\n        version 1.1 or earlier of the License, but not also under the\n        terms of a Secondary License.\n\n1.6. \"Executable Form\"\n    means any form of the work other than Source Code Form.\n\n1.7. \"Larger Work\"\n    means a work that combines Covered Software with other material, in\n    a separate file or files, that is not Covered Software.\n\n1.8. \"License\"\n    means this document.\n\n1.9. \"Licensable\"\n    means having the right to grant, to the maximum extent possible,\n    whether at the time of the initial grant or subsequently, any and\n    all of the rights conveyed by this License.\n\n1.10. \"Modifications\"\n    means any of the following:\n\n    (a) any file in Source Code Form that results from an addition to,\n        deletion from, or modification of the contents of Covered\n        Software; or\n\n    (b) any new file in Source Code Form that contains any Covered\n        Software.\n\n1.11. \"Patent Claims\" of a Contributor\n    means any patent claim(s), including without limitation, method,\n    process, and apparatus claims, in any patent Licensable by such\n    Contributor that would be infringed, but for the grant of the\n    License, by the making, using, selling, offering for sale, having\n    made, import, or transfer of either its Contributions or its\n    Contributor Version.\n\n1.12. \"Secondary License\"\n    means either the GNU General Public License, Version 2.0, the GNU\n    Lesser General Public License, Version 2.1, the GNU Affero General\n    Public License, Version 3.0, or any later versions of those\n    licenses.\n\n1.13. \"Source Code Form\"\n    means the form of the work preferred for making modifications.\n\n1.14. \"You\" (or \"Your\")\n    means an individual or a legal entity exercising rights under this\n    License. For legal entities, \"You\" includes any entity that\n    controls, is controlled by, or is under common control with You. For\n    purposes of this definition, \"control\" means (a) the power, direct\n    or indirect, to cause the direction or management of such entity,\n    whether by contract or otherwise, or (b) ownership of more than\n    fifty percent (50%) of the outstanding shares or beneficial\n    ownership of such entity.\n\n2. License Grants and Conditions\n--------------------------------\n\n2.1. Grants\n\nEach Contributor hereby grants You a world-wide, royalty-free,\nnon-exclusive license:\n\n(a) under intellectual property rights (other than patent or trademark)\n    Licensable by such Contributor to use, reproduce, make available,\n    modify, display, perform, distribute, and otherwise exploit its\n    Contributions, either on an unmodified basis, with Modifications, or\n    as part of a Larger Work; and\n\n(b) under Patent Claims of such Contributor to make, use, sell, offer\n    for sale, have made, import, and otherwise transfer either its\n    Contributions or its Contributor Version.\n\n2.2. Effective Date\n\nThe licenses granted in Section 2.1 with respect to any Contribution\nbecome effective for each Contribution on the date the Contributor first\ndistributes such Contribution.\n\n2.3. Limitations on Grant Scope\n\nThe licenses granted in this Section 2 are the only rights granted under\nthis License. No additional rights or licenses will be implied from the\ndistribution or licensing of Covered Software under this License.\nNotwithstanding Section 2.1(b) above, no patent license is granted by a\nContributor:\n\n(a) for any code that a Contributor has removed from Covered Software;\n    or\n\n(b) for infringements caused by: (i) Your and any other third party's\n    modifications of Covered Software, or (ii) the combination of its\n    Contributions with other software (except as part of its Contributor\n    Version); or\n\n(c) under Patent Claims infringed by Covered Software in the absence of\n    its Contributions.\n\nThis License does not grant any rights in the trademarks, service marks,\nor logos of any Contributor (except as may be necessary to comply with\nthe notice requirements in Section 3.4).\n\n2.4. Subsequent Licenses\n\nNo Contributor makes additional grants as a result of Your choice to\ndistribute the Covered Software under a subsequent version of this\nLicense (see Section 10.2) or under the terms of a Secondary License (if\npermitted under the terms of Section 3.3).\n\n2.5. Representation\n\nEach Contributor represents that the Contributor believes its\nContributions are its original creation(s) or it has sufficient rights\nto grant the rights to its Contributions conveyed by this License.\n\n2.6. Fair Use\n\nThis License is not intended to limit any rights You have under\napplicable copyright doctrines of fair use, fair dealing, or other\nequivalents.\n\n2.7. Conditions\n\nSections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted\nin Section 2.1.\n\n3. Responsibilities\n-------------------\n\n3.1. Distribution of Source Form\n\nAll distribution of Covered Software in Source Code Form, including any\nModifications that You create or to which You contribute, must be under\nthe terms of this License. You must inform recipients that the Source\nCode Form of the Covered Software is governed by the terms of this\nLicense, and how they can obtain a copy of this License. You may not\nattempt to alter or restrict the recipients' rights in the Source Code\nForm.\n\n3.2. Distribution of Executable Form\n\nIf You distribute Covered Software in Executable Form then:\n\n(a) such Covered Software must also be made available in Source Code\n    Form, as described in Section 3.1, and You must inform recipients of\n    the Executable Form how they can obtain a copy of such Source Code\n    Form by reasonable means in a timely manner, at a charge no more\n    than the cost of distribution to the recipient; and\n\n(b) You may distribute such Executable Form under the terms of this\n    License, or sublicense it under different terms, provided that the\n    license for the Executable Form does not attempt to limit or alter\n    the recipients' rights in the Source Code Form under this License.\n\n3.3. Distribution of a Larger Work\n\nYou may create and distribute a Larger Work under terms of Your choice,\nprovided that You also comply with the requirements of this License for\nthe Covered Software. If the Larger Work is a combination of Covered\nSoftware with a work governed by one or more Secondary Licenses, and the\nCovered Software is not Incompatible With Secondary Licenses, this\nLicense permits You to additionally distribute such Covered Software\nunder the terms of such Secondary License(s), so that the recipient of\nthe Larger Work may, at their option, further distribute the Covered\nSoftware under the terms of either this License or such Secondary\nLicense(s).\n\n3.4. Notices\n\nYou may not remove or alter the substance of any license notices\n(including copyright notices, patent notices, disclaimers of warranty,\nor limitations of liability) contained within the Source Code Form of\nthe Covered Software, except that You may alter any license notices to\nthe extent required to remedy known factual inaccuracies.\n\n3.5. Application of Additional Terms\n\nYou may choose to offer, and to charge a fee for, warranty, support,\nindemnity or liability obligations to one or more recipients of Covered\nSoftware. However, You may do so only on Your own behalf, and not on\nbehalf of any Contributor. You must make it absolutely clear that any\nsuch warranty, support, indemnity, or liability obligation is offered by\nYou alone, and You hereby agree to indemnify every Contributor for any\nliability incurred by such Contributor as a result of warranty, support,\nindemnity or liability terms You offer. You may include additional\ndisclaimers of warranty and limitations of liability specific to any\njurisdiction.\n\n4. Inability to Comply Due to Statute or Regulation\n---------------------------------------------------\n\nIf it is impossible for You to comply with any of the terms of this\nLicense with respect to some or all of the Covered Software due to\nstatute, judicial order, or regulation then You must: (a) comply with\nthe terms of this License to the maximum extent possible; and (b)\ndescribe the limitations and the code they affect. Such description must\nbe placed in a text file included with all distributions of the Covered\nSoftware under this License. Except to the extent prohibited by statute\nor regulation, such description must be sufficiently detailed for a\nrecipient of ordinary skill to be able to understand it.\n\n5. Termination\n--------------\n\n5.1. The rights granted under this License will terminate automatically\nif You fail to comply with any of its terms. However, if You become\ncompliant, then the rights granted under this License from a particular\nContributor are reinstated (a) provisionally, unless and until such\nContributor explicitly and finally terminates Your grants, and (b) on an\nongoing basis, if such Contributor fails to notify You of the\nnon-compliance by some reasonable means prior to 60 days after You have\ncome back into compliance. Moreover, Your grants from a particular\nContributor are reinstated on an ongoing basis if such Contributor\nnotifies You of the non-compliance by some reasonable means, this is the\nfirst time You have received notice of non-compliance with this License\nfrom such Contributor, and You become compliant prior to 30 days after\nYour receipt of the notice.\n\n5.2. If You initiate litigation against any entity by asserting a patent\ninfringement claim (excluding declaratory judgment actions,\ncounter-claims, and cross-claims) alleging that a Contributor Version\ndirectly or indirectly infringes any patent, then the rights granted to\nYou by any and all Contributors for the Covered Software under Section\n2.1 of this License shall terminate.\n\n5.3. In the event of termination under Sections 5.1 or 5.2 above, all\nend user license agreements (excluding distributors and resellers) which\nhave been validly granted by You or Your distributors under this License\nprior to termination shall survive termination.\n\n************************************************************************\n*                                                                      *\n*  6. Disclaimer of Warranty                                           *\n*  -------------------------                                           *\n*                                                                      *\n*  Covered Software is provided under this License on an \"as is\"       *\n*  basis, without warranty of any kind, either expressed, implied, or  *\n*  statutory, including, without limitation, warranties that the       *\n*  Covered Software is free of defects, merchantable, fit for a        *\n*  particular purpose or non-infringing. The entire risk as to the     *\n*  quality and performance of the Covered Software is with You.        *\n*  Should any Covered Software prove defective in any respect, You     *\n*  (not any Contributor) assume the cost of any necessary servicing,   *\n*  repair, or correction. This disclaimer of warranty constitutes an   *\n*  essential part of this License. No use of any Covered Software is   *\n*  authorized under this License except under this disclaimer.         *\n*                                                                      *\n************************************************************************\n\n************************************************************************\n*                                                                      *\n*  7. Limitation of Liability                                          *\n*  --------------------------                                          *\n*                                                                      *\n*  Under no circumstances and under no legal theory, whether tort      *\n*  (including negligence), contract, or otherwise, shall any           *\n*  Contributor, or anyone who distributes Covered Software as          *\n*  permitted above, be liable to You for any direct, indirect,         *\n*  special, incidental, or consequential damages of any character      *\n*  including, without limitation, damages for lost profits, loss of    *\n*  goodwill, work stoppage, computer failure or malfunction, or any    *\n*  and all other commercial damages or losses, even if such party      *\n*  shall have been informed of the possibility of such damages. This   *\n*  limitation of liability shall not apply to liability for death or   *\n*  personal injury resulting from such party's negligence to the       *\n*  extent applicable law prohibits such limitation. Some               *\n*  jurisdictions do not allow the exclusion or limitation of           *\n*  incidental or consequential damages, so this exclusion and          *\n*  limitation may not apply to You.                                    *\n*                                                                      *\n************************************************************************\n\n8. Litigation\n-------------\n\nAny litigation relating to this License may be brought only in the\ncourts of a jurisdiction where the defendant maintains its principal\nplace of business and such litigation shall be governed by laws of that\njurisdiction, without reference to its conflict-of-law provisions.\nNothing in this Section shall prevent a party's ability to bring\ncross-claims or counter-claims.\n\n9. Miscellaneous\n----------------\n\nThis License represents the complete agreement concerning the subject\nmatter hereof. If any provision of this License is held to be\nunenforceable, such provision shall be reformed only to the extent\nnecessary to make it enforceable. Any law or regulation which provides\nthat the language of a contract shall be construed against the drafter\nshall not be used to construe this License against a Contributor.\n\n10. Versions of the License\n---------------------------\n\n10.1. New Versions\n\nMozilla Foundation is the license steward. Except as provided in Section\n10.3, no one other than the license steward has the right to modify or\npublish new versions of this License. Each version will be given a\ndistinguishing version number.\n\n10.2. Effect of New Versions\n\nYou may distribute the Covered Software under the terms of the version\nof the License under which You originally received the Covered Software,\nor under the terms of any subsequent version published by the license\nsteward.\n\n10.3. Modified Versions\n\nIf you create software not governed by this License, and you want to\ncreate a new license for such software, you may create and use a\nmodified version of this License if you rename the license and remove\nany references to the name of the license steward (except to note that\nsuch modified license differs from this License).\n\n10.4. Distributing Source Code Form that is Incompatible With Secondary\nLicenses\n\nIf You choose to distribute Source Code Form that is Incompatible With\nSecondary Licenses under the terms of this version of the License, the\nnotice described in Exhibit B of this License must be attached.\n\nExhibit A - Source Code Form License Notice\n-------------------------------------------\n\n  This Source Code Form is subject to the terms of the Mozilla Public\n  License, v. 2.0. If a copy of the MPL was not distributed with this\n  file, You can obtain one at http://mozilla.org/MPL/2.0/.\n\nIf it is not possible or desirable to put the notice in a particular\nfile, then You may include the notice in a location (such as a LICENSE\nfile in a relevant directory) where a recipient would be likely to look\nfor such a notice.\n\nYou may add additional accurate notices of copyright ownership.\n\nExhibit B - \"Incompatible With Secondary Licenses\" Notice\n---------------------------------------------------------\n\n  This Source Code Form is \"Incompatible With Secondary Licenses\", as\n  defined by the Mozilla Public License, v. 2.0.\n"
  },
  {
    "path": "README.md",
    "content": "# Memory Cache \n\nMemory Cache is a project that allows you to save a webpage while you're browsing in Firefox as a PDF, and save it to a synchronized folder that can be used in conjunction with privateGPT to augment a local language model.\n\n| ⚠️: This setup uses the primordial version of privateGPT. I'm working from a fork that can be found [here](https://github.com/misslivirose/privateGPT).  |\n| ---------------------------------------------------------------------------------------------------------------------- |\n\n## Prerequisites \n1. Set up [privateGPT](https://github.com/imartinez/privateGPT) - either using the primordial checkpoint, or from my fork.\n2. Create a symlink between a subdirectory in your default Downloads folder called 'MemoryCache' and a 'MemoryCache' directory created inside of /PrivateGPT/source_documents/MemoryCache \n3. Apply patch to Firefox to add the `printerSettings.silentMode` property to the Tabs API. [See wiki page for instructions](https://github.com/Mozilla-Ocho/Memory-Cache/wiki/Modifying-Firefox-to-Save-PDF-files-automagically-to-MemoryCache)\n4. Copy /scripts/run_ingest.sh into your privateGPT directory and run it to start `inotifywait` watching your downloads directory for new content\n\n## Setting up the Extension\n1. Clone the Memory-Cache GitHub repository to your local machine \n2. In Firefox, navigate to `about:debugging` and click on 'This Firefox'\n3. Click 'Load Temporary Add-on\" and open the `extension/manifest.json` file in the MemoryCacheExt directory\n\n## Using the Extension\n1. Under the 'Extensions' menu, add the Memory Cache extension to the toolbar\n2. When you want to save a page to your Memory Cache, click the icon and select the 'Save' button. This will save the file silently as a PDF if you are using a Firefox build with the `printerSettings.silentMode` property addition.\n"
  },
  {
    "path": "docs/.gitignore",
    "content": "_site\n.sass-cache\n.jekyll-cache\n.jekyll-metadata\nvendor\n"
  },
  {
    "path": "docs/404.html",
    "content": "---\npermalink: /404.html\nlayout: default\n---\n\n<style type=\"text/css\" media=\"screen\">\n  .container {\n    margin: 10px auto;\n    max-width: 600px;\n    text-align: center;\n  }\n  h1 {\n    margin: 30px 0;\n    font-size: 4em;\n    line-height: 1;\n    letter-spacing: -1px;\n  }\n</style>\n\n<div class=\"container\">\n  <h1>404</h1>\n\n  <p><strong>Page not found :(</strong></p>\n  <p>The requested page could not be found.</p>\n</div>\n"
  },
  {
    "path": "docs/CNAME",
    "content": "memorycache.ai"
  },
  {
    "path": "docs/Gemfile",
    "content": "source \"https://rubygems.org\"\n# Hello! This is where you manage which Jekyll version is used to run.\n# When you want to use a different version, change it below, save the\n# file and run `bundle install`. Run Jekyll with `bundle exec`, like so:\n#\n#     bundle exec jekyll serve\n#\n# This will help ensure the proper Jekyll version is running.\n# Happy Jekylling!\n\n# This is the default theme for new Jekyll sites. You may change this to anything you like.\ngem \"minima\", \"~> 2.5\"\n# If you want to use GitHub Pages, remove the \"gem \"jekyll\"\" above and\n# uncomment the line below. To upgrade, run `bundle update github-pages`.\ngem \"github-pages\", \"~> 228\", group: :jekyll_plugins\n# If you have any plugins, put them here!\ngroup :jekyll_plugins do\n  gem \"jekyll-feed\", \"~> 0.12\"\nend\n\n# Windows and JRuby does not include zoneinfo files, so bundle the tzinfo-data gem\n# and associated library.\nplatforms :mingw, :x64_mingw, :mswin, :jruby do\n  gem \"tzinfo\", \">= 1\", \"< 3\"\n  gem \"tzinfo-data\"\nend\n\n# Performance-booster for watching directories on Windows\ngem \"wdm\", \"~> 0.1.1\", :platforms => [:mingw, :x64_mingw, :mswin]\n\n# Lock `http_parser.rb` gem to `v0.6.x` on JRuby builds since newer versions of the gem\n# do not have a Java counterpart.\ngem \"http_parser.rb\", \"~> 0.6.0\", :platforms => [:jruby]\n\ngem \"webrick\", \"~> 1.8\"\n"
  },
  {
    "path": "docs/_config.yml",
    "content": "# Welcome to Jekyll!\n#\n# This config file is meant for settings that affect your whole blog, values\n# which you are expected to set up once and rarely edit after that. If you find\n# yourself editing this file very often, consider using Jekyll's data files\n# feature for the data you need to update frequently.\n#\n# For technical reasons, this file is *NOT* reloaded automatically when you use\n# 'bundle exec jekyll serve'. If you change this file, please restart the server process.\n#\n# If you need help with YAML syntax, here are some quick references for you:\n# https://learn-the-web.algonquindesign.ca/topics/markdown-yaml-cheat-sheet/#yaml\n# https://learnxinyminutes.com/docs/yaml/\n#\n# Site settings\n# These are used to personalize your new site. If you look in the HTML files,\n# you will see them accessed via {{ site.title }}, {{ site.email }}, and so on.\n# You can create any custom variable you would like, and they will be accessible\n# in the templates via {{ site.myvariable }}.\n\ntitle: MemoryCache\nemail: oerickson@mozilla.com\ndescription: MemoryCache is an experimental developer project to turn a local desktop environment into an on-device AI agent.\n# baseurl: \"/Memory-Cache\" # the subpath of your site, e.g. /blog\nurl: \"https://memorycache.ai/\" # the base hostname & protocol for your site, e.g. http://example.com\ngithub_username:  Mozilla-Ocho\n\n# Build settings\ntheme: minima\nplugins:\n  - jekyll-feed\n\n# Exclude from processing.\n# The following items will not be processed, by default.\n# Any item listed under the `exclude:` key here will be automatically added to\n# the internal \"default list\".\n#\n# Excluded items can be processed by explicitly listing the directories or\n# their entries' file path in the `include:` list.\n#\n# exclude:\n#   - .sass-cache/\n#   - .jekyll-cache/\n#   - gemfiles/\n#   - Gemfile\n#   - Gemfile.lock\n#   - node_modules/\n#   - vendor/bundle/\n#   - vendor/cache/\n#   - vendor/gems/\n#   - vendor/ruby/\n"
  },
  {
    "path": "docs/_includes/footer.html",
    "content": "<footer class=\"site-footer h-card\">\n  <data class=\"u-url\" href=\"{{ \"/\" | relative_url }}\"></data>\n\n  <div class=\"wrapper\">\n\n    <h2 class=\"footer-heading\">{{ site.title | escape }}</h2>\n\n    <div class=\"footer-col-wrapper\">\n      <div class=\"footer-col footer-col-1\">\n        <ul class=\"contact-list\">\n          <li class=\"p-name\">\n            {%- if site.author -%}\n              {{ site.author | escape }}\n            {%- else -%}\n              {{ site.title | escape }}\n            {%- endif -%}\n            </li>\n            {%- if site.email -%}\n            <li><a class=\"u-email\" href=\"mailto:{{ site.email }}\">{{ site.email }}</a></li>\n            {%- endif -%}\n        </ul>\n      </div>\n\n      <div class=\"footer-col footer-col-2\">\n        {%- include social.html -%}\n      </div>\n\n      <div class=\"footer-col footer-col-3\">\n        <a href=\"https://future.mozilla.org\">\n          <img class=\"moz-logo\" src=\"/../assets/images/mozilla-logo-bw-rgb.png\" />\n        </a>\n      </div>\n    </div>\n\n  </div>\n\n</footer>\n"
  },
  {
    "path": "docs/_includes/header.html",
    "content": "<header class=\"site-header\" role=\"banner\">\n\n  <div class=\"wrapper\">\n    {%- assign default_paths = site.pages | map: \"path\" -%}\n    {%- assign page_paths = site.header_pages | default: default_paths -%}\n    <a class=\"site-title\" rel=\"author\" href=\"{{ \"/\" | relative_url }}\"> \n      <img class=\"site-logo\" src=\"/assets/images/MC-LogoNov23.svg\"/>\n    </a>\n\n    {%- if page_paths -%}\n      <nav class=\"site-nav\">\n        <input type=\"checkbox\" id=\"nav-trigger\" class=\"nav-trigger\" />\n        <label for=\"nav-trigger\">\n          <span class=\"menu-icon\">\n            <svg viewBox=\"0 0 18 15\" width=\"18px\" height=\"15px\">\n              <path d=\"M18,1.484c0,0.82-0.665,1.484-1.484,1.484H1.484C0.665,2.969,0,2.304,0,1.484l0,0C0,0.665,0.665,0,1.484,0 h15.032C17.335,0,18,0.665,18,1.484L18,1.484z M18,7.516C18,8.335,17.335,9,16.516,9H1.484C0.665,9,0,8.335,0,7.516l0,0 c0-0.82,0.665-1.484,1.484-1.484h15.032C17.335,6.031,18,6.696,18,7.516L18,7.516z M18,13.516C18,14.335,17.335,15,16.516,15H1.484 C0.665,15,0,14.335,0,13.516l0,0c0-0.82,0.665-1.483,1.484-1.483h15.032C17.335,12.031,18,12.695,18,13.516L18,13.516z\"/>\n            </svg>\n          </span>\n        </label>\n\n        <div class=\"trigger\">\n          <a class=\"page-link\" target=\"_blank\" href=\"https://github.com/Mozilla-Ocho/Memory-Cache/wiki\">Docs</a>\n          {%- for path in page_paths -%}\n            {%- assign my_page = site.pages | where: \"path\", path | first -%}\n            {%- if my_page.title -%}\n            <a class=\"page-link\" href=\"{{ my_page.url | relative_url }}\">{{ my_page.title | escape }}</a>\n            {%- endif -%}\n          {%- endfor -%}\n        </div>\n      </nav>\n    {%- endif -%}\n  </div>\n</header>\n"
  },
  {
    "path": "docs/_layouts/default.html",
    "content": "<!DOCTYPE html>\n<html lang=\"{{ page.lang | default: site.lang | default: \"en\" }}\">\n\n  {%- include head.html -%}\n\n  <body>\n\n    {%- include header.html -%}\n\n    <main class=\"page-content\" aria-label=\"Content\">\n      <div class=\"wrapper\">\n        {{ content }}\n      </div>\n    </main>\n\n    {%- include footer.html -%}\n\n  </body>\n\n</html>\n"
  },
  {
    "path": "docs/_layouts/home.html",
    "content": "---\nlayout: default\n---\n\n<div class=\"home\">\n  {%- if page.title -%}\n    <h1 class=\"page-heading\">{{ page.title }}</h1>\n  {%- endif -%}\n\n  {{ content }}\n  <div class=\"introduction\">\n    <h2 class=\"callout-left\">MemoryCache is an experimental development project to turn a local desktop environment into an on-device AI agent.</h2>\n  </div>\n\n  <div class=\"detailed-overview\">\n    <p> Every human is unique. The original vision of the personal computer was as a companion tool for creating intelligence, and the internet was born as a way to connect people and data together around the world. Today, artificial intelligence is upending the way that we interact with data and information, but control of these systems is most often provided through an API endpoint, run in the cloud, and abstracting away deep personal agency in favor of productivity. </p>\n\n    <p> MemoryCache, <a href=\"https://future.mozilla.org\" target=\"_blank\"> a Mozilla Innovation Project,</a> is an experimental AI Firefox add-on that partners with privateGPT to quickly save your browser history to your local machine and have a local AI model ingest these - and any other local files you give it - to augment responses to a chat interface that comes built-in with privateGPT. We have an ambition to use MemoryCache to move beyond the chat interface, and find a way to utilize idle compute time to generate net new insights that reflect what you've actually read and learned - not the entirety of the internet at scale. </p>\n    <figure>\n      <img src =\"/../assets/images/DesktopApplication.png\" />\n      <figcaption> Design mockup of a future interface idea for MemoryCache</figcaption>\n    </figure>\n    <p> We're not breaking ground on AI innovation (in fact, we're using an old, \"deprecated\" file format from a whole six months ago), by design. MemoryCache is a project that allows us to sow some seeds of exploration into creating a deeply personalized AI experience that returns to the original vision of the computer as a companion for our own thought. With MemoryCache, weirdness and unpredictability is part of the charm. </p>\n\n    <p> We're a small team working on MemoryCache as a part-time project within Mozilla's innovation group, looking at ways that our personal data and files are used to form insights and new neural connections for our own creative purpose. We're working in the open not because we have answers, but because we want to contribute our way of thinking to one another in a way where others can join in. </p>\n  </div>\n\n  {%- if site.posts.size > 0 -%}\n    <h2 class=\"post-list-heading\">{{ page.list_title | default: \"Updates\" }}</h2>\n    <ul class=\"post-list\">\n      {%- for post in site.posts -%}\n      <li>\n        {%- assign date_format = site.minima.date_format | default: \"%b %-d, %Y\" -%}\n        <span class=\"post-meta\">{{ post.date | date: date_format }}</span>\n        <h3>\n          <a class=\"post-link\" href=\"{{ post.url | relative_url }}\">\n            {{ post.title | escape }}\n          </a>\n        </h3>\n        {%- if site.show_excerpts -%}\n          {{ post.excerpt }}\n        {%- endif -%}\n      </li>\n      {%- endfor -%}\n    </ul>\n\n    <p class=\"rss-subscribe\">subscribe <a href=\"{{ \"/feed.xml\" | relative_url }}\">via RSS</a></p>\n  {%- endif -%}\n\n</div>\n"
  },
  {
    "path": "docs/_layouts/page.html",
    "content": "---\nlayout: default\n---\n<article class=\"post\">\n\n  <header class=\"post-header\">\n    <h1 class=\"post-title\">{{ page.title | escape }}</h1>\n  </header>\n\n  <div class=\"post-content\">\n    {{ content }}\n  </div>\n\n</article>\n"
  },
  {
    "path": "docs/_layouts/post.html",
    "content": "---\nlayout: default\n---\n<article class=\"post h-entry\" itemscope itemtype=\"http://schema.org/BlogPosting\">\n\n  <header class=\"post-header\">\n    <h1 class=\"post-title p-name\" itemprop=\"name headline\">{{ page.title | escape }}</h1>\n    <p class=\"post-meta\">\n      <time class=\"dt-published\" datetime=\"{{ page.date | date_to_xmlschema }}\" itemprop=\"datePublished\">\n        {%- assign date_format = site.minima.date_format | default: \"%b %-d, %Y\" -%}\n        {{ page.date | date: date_format }}\n      </time>\n      {%- if page.author -%}\n        • <span itemprop=\"author\" itemscope itemtype=\"http://schema.org/Person\"><span class=\"p-author h-card\" itemprop=\"name\">{{ page.author }}</span></span>\n      {%- endif -%}</p>\n  </header>\n\n  <div class=\"post-content e-content\" itemprop=\"articleBody\">\n    {{ content }}\n  </div>\n\n  {%- if site.disqus.shortname -%}\n    {%- include disqus_comments.html -%}\n  {%- endif -%}\n\n  <a class=\"u-url\" href=\"{{ page.url | relative_url }}\" hidden></a>\n</article>\n"
  },
  {
    "path": "docs/_posts/2023-11-06-introducing-memory-cache.markdown",
    "content": "---\nlayout: post\ntitle:  \"Introducing Memory Cache\"\ndate:   2023-11-06 14:47:57 -0500\ncategories: developer-blog\n---\nMost AI development today is centered around services. Companies offer tailored insights and powerful agents that can replicate all aspects of the human experience. AI is supposedly \"passing the bar exam\", diagnosing medical issues, and everything in-between. What is the role of a human being in an increasingly online world?\n\nIn practice, AI is a complex web of big data sources (e.g. the entirety of the internet). Pairing massive amounts of data with increasingly powerful cloud computing capabilities has resulted in unprecedented software development capabilities. Adopting naming practices and principles from science fiction stories, Silicon Valley is racing down a path towards a fictional idea of a \"sentient computer\" with AGI. Artificial intelligence is a field gives us more modalities and capabilities to use with computers, and what really matters is how we (as humans) use the technology at hand.\n\nNot that long ago, computing was grounded in the idea that digital literacy was a skill to be adopted and used in service of greater problems to be solved. We, as individuals, had control over our data, our files, our thoughts. Over the past several years, Big Tech has traded us systems of addictive social media sites for yottabytes of our personal data in the service of \"personalization\" - a.k.a targeted advertising.\n\nMemory Cache is an exploration into human-first artificial intelligence, starting with the actual idea of the personal computer. The project is an experiment in local, on-premise AI: what you can do with a standard gaming desktop that sits in your home, and actually works for you. It bridges your browser history with your local file system, so that you can use the power of openly licensed AI and open source code to inspect, query, and tinker with an AI that is under your own control.\n"
  },
  {
    "path": "docs/_posts/2023-11-30-we-have-a-website.markdown",
    "content": "---\nlayout: post\ntitle:  \"We have a website! And other MemoryCache Updates\"\ndate:   2023-11-30 11:47:00 -0800\ncategories: developer-blog\n---\nWe've been continuing our work on MemoryCache over the past several weeks, and are excited to have our [landing page](https://memorycache.ai) up and running. The updated design for MemoryCache has been something we've been iterating on, and it's been a fun process to talk about what sort of emotions we want to seed MemoryCache tools with. We've settled on an initial design style guide and have recently landed several contributions to enable a vanilla Firefox version of the extension.\n\nOur team is working on MemoryCache as a sandbox for exploring concepts related to small, local, and patient AI. We're a small project with big ambitions, and look forward to continuing down several areas of exploration in the coming weeks, including:\n\n* Building an app experience to automatically generate insights from newly added data, to act as a gentle reminder of places that can grow from your recently added content\n\n* Updating the project website to include more details about the philosophy, design, and thinking around the project and how we envision it growing\n\n* Competitive and secondary research reporting that we can publish that shares our insights and findings on how people think about recall and note-taking\n\n* Understanding how to evaluate and generate personal insights outside of the chat interface model\n\n* Exploring a social layer to easily distill and share insights within a trusted network of people\n\nFollow along with us on our [GitHub repo](https://github.com/misslivirose/Memory-Cache) - we'd love to see you there!\n"
  },
  {
    "path": "docs/_posts/2024-03-01-memory-cache-and-ai-privacy.markdown",
    "content": "---\nlayout: post\ntitle:  \"MemoryCache and AI Privacy\"\ndate:   2024-03-01 07:08:57 -0500\ncategories: developer-blog\n---\n_Author: Liv Erickson_\n\nIt's been an exciting few months since we first shared [MemoryCache](https://memorycache.ai/developer-blog/2023/11/06/introducing-memory-cache.html), a home for experimenting with local-first AI that learns what you learn. While we'll be sharing more updates about what the team has been working on in the coming weeks by way of more regular development blogs, I wanted to share a podcast that I recently recorded with [Schalk Neethling for the Mechanical Ink Podcast](https://schalkneethling.substack.com/p/privacy-ai-and-an-ai-digital-memory) that goes in-depth about the principles behind MemoryCache. \n\n[![Watch the video](https://img.youtube.com/vi/CGdxLfcU9TU/0.jpg)](https://www.youtube.com/watch?v=CGdxLfcU9TU)\n\nIn this podcast, we go into the motivations behind the project and how it was originally created, as well as the overall challenges that are presented with the growing creation of synthetic content and preserving authentic connections online in an area of unprecedented, generated personalization. \n\nAt it's core, MemoryCache is a project exploring what it means to urgently, yet collaboratively, envision futures where AI technologies enable us to build a more authentic relationship with information and ourselves through small acts of insight and [embracing friction](https://foundation.mozilla.org/en/blog/speculative-friction-in-generative-ai/) as it presents novel outcomes.\n\nMore coming soon!"
  },
  {
    "path": "docs/_posts/2024-03-06-designlog-update.markdown",
    "content": "---\nlayout: post\ntitle:  \"MemoryCache March Design Update\"\ndate:   2024-03-06 08 -0500\ncategories: developer-blog\n---\n_Author: Kate Taylor_\n\nHi! My name is Kate and I am a designer working on MemoryCache in the Mozilla Innovation organization. My official title is Design Technologist, which describes the focus between humans and technology. Humans is an important word in this context because it is not specific to a group of people, but the recognition that the choices we make as technologists have effects that ripple on to humans who may or may not be aware of what happens with their information. Information is powerful in the world of AI and the handling of it deserves genuine respect, which in turn builds on an atmosphere of trust and safety.\n\nMemoryCache serves as an open and safe testbed to explore the ideas of what humans need to both benefit from an AI agent while maintaining control over their information and the technical processes involved. As a designer on this project, my work is intended to create an environment that feels like a true augmentation of your creative thought work.\n\n<figure>\n  <img src=\"https://github.com/Mozilla-Ocho/Memory-Cache/assets/100849201/1a3bb64a-dc65-4ff0-928c-4a61756bd8e6\" alt=\"Sketches and notes describing why establishing context is important for AI tasks and the feeling of security\">\n  <figcaption>Early concepts and notes exploring the idea of what safety in an AI Agent experience looks like</figcaption>\n</figure>\n\nWhen we started the project last year, the world was in a different place. Reflecting back on the time along with the goings-on in the AI space is bringing to mind a lot of big personal feelings as well as acknowledgment of the generally fearful vibes. I tend to pay attention to feelings because they are what allow us (as people) to find the things that matter most. It’s difficult to not sense an amount of fear when people are faced with a lot of change. This especially becomes clear when comparing conversations with people in and out of the tech field. This fear has brought a lot of very meaningful interpersonal conversations about what this technology means -- what job does it do well, what jobs do we (as people) do well and want to keep, how did we get to this point in time. These conversations are the motivation for contributing to Open Source AI because the power of understanding the world around you is boundless, and the barriers to this knowledge are mysterious but significant.\n\nAwareness of the barriers to entry to an experience is where designers do their best work. We strive to find meaningful solutions to problems that will sustain. Acknowledging the emotional barrier is the foundation of how we are thinking about MemoryCache. Depending on who you are, interacting with a chatbot has baggage - in the same way that social situations differ across individuals. There are social aspects, language considerations, articulation differences, historical contexts, etc. This is A LOT to deal with as a user of a system. When approaching the design work for MemoryCache, our guiding light is to take into account the unique humanity of each person and allow for the ability to utilize the technology to create an environment that nurtures human needs rather than profit from them.\n\n<figure>\n  <img src=\"https://github.com/Mozilla-Ocho/Memory-Cache/assets/100849201/4dc37c30-ba29-45ff-b20d-94d6ed212ce7\" alt=\"Design mockups and explorations for the interface of MemoryCache as a desktop application\">\n  <figcaption>Exploratory work for interactions that combine with chat interface to interact with the agent in personalized ways for various usecases</figcaption>\n</figure>\n\n<figure>\n  <img src=\"https://github.com/Mozilla-Ocho/Memory-Cache/assets/100849201/416bb481-0a73-409d-b464-5a5369f04c00\" alt=\"Design mockup with various color theme options\">\n  <figcaption>Design mockups and explorations for the UI exploring themes and personalization in combination with input methods for interaction</figcaption>\n</figure>\n\nThis philosophy is core to the work we are doing with MemoryCache. We believe that personalized experiences for interacting with your own information provides a safe space for working with your thought materials with a lot of flexibility and personalized modularity.\n\nWhen thinking about what safety means in relation to AI computing, in 2024 this is a complicated subject. We are not just speaking about access to information, but access to people and the very things that make us human. Painting, drawing, and tinkering have always been the safe space in my life, personally. The process of making things provides the opportunity to explore your thoughts and experiences without the judgment or unsolicited opinions of others. This time spent reflecting tends to be where the most valuable ideas come to light in other areas of life (similar to the idea of “shower thoughts”). We are iterating on this concept with the idea that the agent could work with the person in creating more of those shower-thought moments. Flipping the idea of asking the agent for a task with the agent providing insights that you could find valuable. The mockups below demonstrate our current thinking for an interface that we can start building and working with as needs evolve\n\n<figure>\n  <img src=\"https://github.com/Mozilla-Ocho/Memory-Cache/assets/100849201/7ce4b868-a9cb-406c-a3da-44ce91277f58\" alt=\"Design mockup for the MemoryCache agent's UI\">\n  <figcaption>Latest design mockup for MemoryCache agent</figcaption>\n</figure>\n![Image4](https://github.com/Mozilla-Ocho/Memory-Cache/assets/100849201/881b41b1-4217-4fbe-85a0-cfdbe3732697)\n\nWe are excited about the potential of where MemoryCache can go. Part of the magic of developing in the open is that we will learn along the way what matters most to people and evolve from there. The logo and visual design we are running with for MemoryCache visualizes our hope for celebration of individuality and personal empowerment through technology advancements. The world can be a messy place, but your individual context is yours to control. In our next round of work, we are building on this philosophy to expose the Agent’s capabilities in meaningful interactions that support the ability to augment your thought work in the way that your brain thinks.\n"
  },
  {
    "path": "docs/_posts/2024-03-07-devlog.markdown",
    "content": "---\nlayout: post\ntitle:  \"Memory Cache Dev Log March 7 2024\"\ndate:   2024-03-07 08 -0500\ncategories: developer-blog\n---\n_Author: John Shaughnessy_\n\n\n# Memory Cache Dev Log, March 7 2024\n\nA couple months ago [we introduced Memory Cache](https://future.mozilla.org/blog/introducing-memorycache/):\n\n> Memory Cache, a Mozilla Innovation Project, is an early exploration project that augments an on-device, personal model with local files saved from the browser to reflect a more personalized and tailored experience through the lens of privacy and agency\n\nSince then we've been quiet.... _too quiet_. \n\n## New phone, who dis?\n\nIt's my first time writing on this blog, so I want to introduce myself. My name is John Shaughnessy. I'm a software engineer at Mozilla. \n\nI got involved in Memory Cache a few months ago by resolving an issue that was added to the github repo and a couple weeks ago I started building Memory Cache V2.\n\n## Why V2?\n\nMemory Cache V1 was a browser extension and that made it convenient to collect and feed documents to an LLM-based program called `privateGPT`. PrivateGPT would break the documents into fragments, save those fragments in a vector database, and let you perform similarity search on those documents via a command line interface. We were running an old version of PrivateGPT based on LangChain.\n\nThere were several big, obvious technical gaps between Memory Cache V1 and what we'd need in order to do the kind of investigative research and product development we wanted to do.\n\nIt seemed to me that if we really wanted to explore the design space, we'd need to roll our own backend and ship a friendly client UI alongside it. We'd need to speed up inference and we'd need more control over how it ingested documents, inserted context, batched background tasks and presented information to you.\n\nWe also needed to fix the \"getting started\" experience. Setting up V1 required users to be comfortable working on the command line, managing python environments, and in general understanding their file system. As far as I'm aware, there are only three of us who have gone through the steps to actually set up and run V1. We were inspired by [Llamafile](https://github.com/Mozilla-Ocho/llamafile/) and [cosmopolitan](https://justine.lol/ape.html), which create executables that you just download and run on many platforms.\n\nAnd lastly, we're excited about multiplayer opportunities. Could insights that my LLM generates become useful to my teammates? Under what circumstances would I want to share my data with others? How should I separate what's private, semi-private, or public?\n\n### Running LLMs for Inference\n\nI wasn't very familiar with running LLMs, and I certainly hadn't written an application that did \"Retrieval-Augmented Generation\" (RAG), which was what we wanted Memory Cache to do. So I started down a long, winding path.\n\nLiv and I chatted with Iván Martínez who wrote `privateGPT`. He was super helpful! And it was exciting to talk to someone who'd built something that let us prototype what we wanted to do so quickly.\n\nMozilla had just announced [Llamafile](https://github.com/Mozilla-Ocho/llamafile), which seemed like a great way to package an LLM and serve it on many platforms. I wasn't familiar with either [Llama.cpp](https://github.com/ggerganov/llama.cpp) or [cosmo](https://cosmo.zip/), so there was a lot to learn. [Justine](https://github.com/jart) and [Stephen](https://github.com/stlhood) were incredibly helpful and generous with their time. I didn't contribute much back to the project other than trying to write accurate reports of a couple issues ([#214](https://github.com/Mozilla-Ocho/llamafile/issues/214), [#232](https://github.com/Mozilla-Ocho/llamafile/issues/232)) I ran into along the way.\n\nInitially when I was looking into `Llamafile`, I wanted to repackage `privateGPT` as a `Llamafile` so that we could distribute it as a standalone executable. Eventually realized this wasn't a good idea. `Llamafile` bundles `Llama.cpp` programs as executables. `Cosmopolitan` _can_ also bundle things like a python interpreter, but tracking down platform-specific dependencies of `privateGPT` and handling them in a way that was compatible with cosmo was not going to be straightforward. It's just not what the project was designed to do.\n\n<figure>\n<img class=\"rounded-rect\" src=\"https://memorycache.ai/assets/images/march_2024_dev_log/screenshot_026.png\">\n<figcaption></figcaption>\n</figure>\n\nOnce I worked through the issues I was having with my GPU, I was amazed and excited to see how fast LLMs can run. I made a short comparison video that shows the difference: [llamafile CPU vs GPU](https://www.youtube.com/watch?v=G9wBw8jLJwU).\n\nI thought I might extend the simple HTTP server that's baked into `Llamafile` with all the capabilities we'd want in Memory Cache. Justine helped me get some \"hello world\" programs up and running, and I started reading some examples of C++ servers. I'm not much of a C++ programmer, and I was not feeling very confident that this was the direction I really wanted to go. \n\nI like working in Rust, and I knew that Rust had some kind of story for getting `C` and `C++` binding working, so I wrote a kind of LLM \"hello world\" program using [rustformers/llm](https://github.com/rustformers/llm). But after about a week of fiddling with Llamafile, Llama.cpp, and rustformers, I felt like I was going down a bit of a rabbit hole, and I wanted to pull myself back up to the problem at hand.\n\n### Langchain and Langserve\n\nOk. So if we weren't going to build out a C++ or Rust server, what _should_ we be doing? `PrivateGPT` was a python project, and the basic functionality was similar to what I'd done in some simple programs I'd written with hugging face's `transformers` library. (I mentioned these in a [blog post](https://johnshaughnessy.com/blog/posts/osai-kube) and [talk](https://www.youtube.com/watch?v=AHd3jCMQQLs) about upskilling in AI.)\n\nIt seemed like `LangChain` and `LlamaIndex` were the two popular python libraries / frameworks for building RAG apps, so I wrote a \"hello world\" with LangChain. It was... fine. But it seemed like a _lot_ more functionality (and abstraction, complexity, and code) than I wanted. \n  \nI ended up dropping the framework after reading the docs for `ChromaDB` and `FastAPI`. \n\n`ChromaDB` is a vector database for turning documents into fragments and then run similarity search (the fundamentals of a RAG system). \n\n<figure>\n<img class=\"rounded-rect\" src=\"https://memorycache.ai/assets/images/march_2024_dev_log/screenshot_024.png\">\n<figcaption></figcaption>\n</figure>\n\n<figure>\n<img class=\"rounded-rect\" src=\"https://memorycache.ai/assets/images/march_2024_dev_log/screenshot_025.png\">\n<figcaption></figcaption>\n</figure>\n\nI needed to choose a database, and I chose this one arbitrarily. Langchain had an official \"integration\" for Chroma, but I felt like Chroma was so simple that I couldn't imagine an \"integration\" being helpful. \n\n`FastAPI` is a python library for setting up http servers, and is \"batteries included\" in some very convenient ways: \n- It's compatible with [pydantic](https://docs.pydantic.dev/) which lets you define types, validate user input against them, and generate `OpenAPI` spec from them.\n- It comes with [swagger-ui](https://github.com/swagger-api/swagger-ui) which gives an interactive browser interface to your APIs.\n- It's compatible with a bunch of other random helpful things like [python-multipart](https://github.com/Kludex/python-multipart).\nThe other thing to know about `FastAPI` is that as far as http libraries go, it's very easy to use. I was reading documentation about `Langserve`, which seemed like a kind of fancy server for `Langchain` apps until I realized that actually `FastAPI`, `pydantic`, `swagger-ui` et. al were doing all the heavy lifting.\n\nSo, I dropped LangChain and Langserve and decided I'd wait until I encountered an actually hard problem before picking up another framework. (And who knows -- such a problem might be right around the corner!)\n\nIt helped to read LangChain docs and code to figure out what RAG even is. After that I was able to get basic rag app working (without the framework). I felt pretty good about it.\n\n### Inference\n\nI still needed to decide how to run the LLM. I had explored Llamafiles and Hugging face's `transformers` library. The other popular option seemed to be `ollama`, so I gave that a shot. \n\n`Ollama` ended up being very easy to get up and running. I don't know very much about the project. So far I'm a fan. But I didn't want users of Memory Cache to have to download and run an LLM inference server/process by themselves. It just feels like a very clunky user experience. I want to distribute ONE executable that does everything. \n\nMaybe I'm out of the loop, but I didn't feel very good about any of the options. Like, what I really wanted was to write a python program that handled RAG, project files, and generating various artifacts by talking to an llm, and I also wanted it to run the LLM, and I also wanted it to be an HTTP server to serve a browser client. I suppose that's a complex list of requirements, but it seemed like a reasonable approach for Memory Cache. And I didn't find any examples of people doing this.\n\nI had the idea of using Llamafiles for inference and a python web server for the rest of the \"brains\", which could also serve a static client. That way, the python code stays simple (it doesn't bring with it any of the transformers / hugging face / llm / cuda code).\n\n### Memory Cache Hub\n\nI did a series of short spikes to piece together exactly how such an app could work. I wrote a bit about each one in this [PR](https://github.com/Mozilla-Ocho/Memory-Cache/pull/58).\n\nIn the end, I landed on a (technical) design that I'm pretty happy with. I'm putting the pieces together in a repo called [Memory Cache Hub](https://github.com/johnshaughnessy/Memory-Cache-Hub/) (which will graduate to the Mozilla-Ocho org when it's ready). The [README.md](https://github.com/johnshaughnessy/Memory-Cache-Hub/blob/main/README.md) has more details, but here's the high level:\n\n```plaintext\nMemory Cache Hub is a core component of Memory Cache:\n\n    - It exposes APIs used by the browser extension, browser client, and plugins.\n    - It serves static files including the browser client and various project artifacts.\n    - It downloads and runs llamafiles as subprocesses.\n    - It ingests and retrieves document fragments with the help of a vector database.\n    - It generates various artifacts using prompt templates and large language models.\n\nMemory Cache Hub is designed to run on your own machine. All of your data is stored locally and is never uploaded to any server.\n\nTo use Memory Cache Hub:\n\n    - Download the latest release for your platform (Windows, MacOS, or GNU/Linux)\n    - Run the release executable. It will open a new tab in your browser showing the Memory Cache GUI.\n    - If the GUI does not open automatically, you can navigate to http://localhost:4444 in your browser.\n\nEach release build of Memory Cache Hub is a standalone executable that includes the browser client and all necessary assets. By \"standalone\", we mean that you do not need to install any additional software to use Memory Cache.\n\nA Firefox browser extension for Memory Cache that extends its functionality is also available. More information can be found in the main Memory Cache repository.\n```\n\nThere are two key ideas here:\n- Inference is provided by llamafiles that the hub downloads and runs.\n- We use `PyInstaller` to bundle the hub and the browser client into a single executable that we can release.\n\nThe rest of the requirements are handled easily in python because of the great libraries and tools that are available (`fastapi`, `pydantic`, `chromadb`, etc).\n\nGetting the two novel ideas to work was challenging. I'm not a python expert, so figuring out the `asyncio` and `subprocess` stuff to download and run llamafiles was tricky. And `PyInstaller` has a long list of \"gotchas\" and \"beware\" warnings in its docs. I'm still not convinced I'm using it correctly, even though the executables I'm producing seem like they're doing the right thing.\n\n## The Front End\n\nBy this time I had built three measly, unimpressive browser clients for Memory Cache. The first was compatible with `privateGPT`, the second was compatible with some early versions of the Memory Cache Hub. I built the third with `gradio` but quickly decided that it did not spark joy.\n\nAnd none of these felt like good starting points for a designer to jump into the building process. \n\nI've started working on a kind of \"hello world\" dashboard for Memory Cache using `tailwindcss`. I want to avoid reinventing the wheel and make sure the basic interactions feel good.\n\nI've exposed most of the Hub's APIs in the client interface by now. It doesn't look or feel good yet, but it's good to have the basic capabilities working.\n\n\n<figure>\n<img class=\"rounded-rect\" src=\"https://memorycache.ai/assets/images/march_2024_dev_log/screenshot_032.png\">\n<figcaption></figcaption>\n</figure>\n\n<figure>\n<img class=\"rounded-rect\" src=\"https://memorycache.ai/assets/images/march_2024_dev_log/screenshot_039.png\">\n<figcaption></figcaption>\n</figure>\n\n## What We're Aiming For\n\nThe technical pieces have started to fall into place. We're aiming to have the next iteration of Memory Cache - one that you can easily download and run on your own machine - in a matter of weeks. In an ideal world, we'd ship by the end of Q1, which is a few weeks away.\n\nIt won't be perfect, but it'll be far enough along that the feedback we get will be much more valuable, and will help shape the next steps.\n\n"
  },
  {
    "path": "docs/_posts/2024-03-15-devlog.markdown",
    "content": "---\nlayout: post\ntitle:  \"Memory Cache Dev Log March 15 2024\"\ndate:   2024-03-15 08 -0500\ncategories: developer-blog\n---\n_Author: John Shaughnessy_\n\n# Memory Cache Dev Log, March 15 2024\n\nLast Friday, during a casual weekly engineering call, a colleague asked how LLM libraries (llama.cpp, llamafile, langchain, llamaindex, HF transformers, ollama, etc) handle the different chat templates and special tokens that models train on. It was a good question, and none of us seemed to have a complete answer.\n\nThe subsequent discussion and research made me realize that a naive approach towards writing model-agnostic application would have unfortunate limitations. Insofar as the differences between models are actually important to the use case, application developers should write model-specific code.\n\n## Text Summarization\n\nI thought about the capabilities that are important for an application like Memory Cache, and which models would be good at providing those capabilities. The first obvious one was text summarization. I had \"hacked\" summarization by asking an assistant-type model (llamafile) to summarize text, but a model trained specifically for text summarization would be a better fit.\n\nI tested a popular text summarization model with with HF transformers, since I didn't find any relevant llamafiles. (If they're out there, I don't know how to find them.) I wanted to make sure that HF code could be built and bundled to a native application with PyInstaller, since that's how we want to build and bundle Memory Cache as a standalone executable. I verified that it could with a [small test project](https://github.com/johnshaughnessy/summarization-test).\n\nBundling HF dependencies like pytorch increases the complexity of the release process because we'd go from 3 build targets (MacOS, Windows, Linux) to 8 build targets (assuming support for every platform that pytorch supports):\n\n- `Linux` + `CUDA 11.8`\n- `Linux` + `CUDA 12.1`\n- `Linux` + `ROCm 5.7`\n- `Linux` + `CPU`\n- `Mac` + `CPU`\n- `Windows` + `CUDA 11.8`\n- `Windows` + `CUDA 12.1`\n- `Windows` + `CPU`\n\nIt's good to know that this is possible, but since our near-term goal for Memory Cache is just to prove out the technical bits mentioned in the [previous dev log](https://memorycache.ai/developer-blog/2024/03/07/devlog.html), we'll likely stick with the text summarization \"hack\" for now.\n\n## Training Agents\n\nText summarization is still a simple task (in terms of inference inputs and outputs), so models trained to summarize are likely interchangable for the most part (modulo input/output lengths). However, once we start looking at more complicated types of tasks (like tool use / function calling / memory), the differences between models will be exaggerated.\n\nConsider an example like [this dataset](https://huggingface.co/datasets/smangrul/assistant_chatbot_dataset) meant to help train a model to with act with agentic intentions, beliefs (memory), actions, and chat:\n\n```\nContext:\n\n<|begincontext|><|beginlastuserutterance|>I am feeling hungry so I would like to find a place to eat.<|endlastuserutterance|><|endcontext|>\n```\n\n```\nTarget:\n\n<|begintarget|><|begindsts|><|begindst|><|beginintent|>FindRestaurants<|endintent|><|beginbelief|><|endbelief|><|enddst|><|enddsts|><|beginuseraction|>INFORM_INTENT->Restaurants^intent~FindRestaurants<|enduseraction|><|beginaction|>REQUEST->Restaurants^city~<|endaction|><|beginresponse|>Do you have a specific which you want the eating place to be located at?<|endresponse|><|endtarget|>\n```\n\nHere, we can see that there are _many_ special tokens that the application developer would need to be aware of:\n\n```\n- <beginintent></endintent>\n- <beginbelief></endbelief>\n- <beginaction></endaction>\n- <beginresponse></endresponse>\n```\n\nResearch on how to train these types of models is still rapidly evolving. I suspect attempting to abstract away these differences will lead to leaky or nerfed abstractions in libraries and toolkits. For now, my guess is that it's better to write application code targeting the specific models you want to use.\n\n## Conclusion\n\nEven if a dedicated text summarization model doesn't make it into the upcoming release, this was a valuable excursion. These are the exact types of problems I hoped to stumble over along the way.\n"
  },
  {
    "path": "docs/_posts/2024-04-19-memory-cache-hub.markdown",
    "content": "---\nlayout: post\ntitle:  \"Memory Cache Hub\"\ndate:   2024-04-19 08 -0500\ncategories: developer-blog\n---\n_Author: John Shaughnessy_\n\n# Memory Cache Hub\n\nIn a [dev log](https://memorycache.ai/developer-blog/2024/03/07/devlog.html) last month, I explained why we were building Memory Cache Hub. We wanted:\n- our own backend to learn about and play around with [`RAG`](https://python.langchain.com/docs/expression_language/cookbook/retrieval),\n- a friendly browser-based UI, \n- to experiment with `llamafile`s,\n- to experiment with bundling python files with `PyInstaller`\n\nThe work outlined in that blog post is done. You can try Memory Cache Hub by following the [installation instructions in the README](https://github.com/Mozilla-Ocho/Memory-Cache-Hub?tab=readme-ov-file#installation).\n\nMy goal for Memory Cache Hub was just to get the project to this point. It was useful to build and I learned a lot, but there are no plans to continue development.\n\nFor the rest of this post, I would like to share some thoughts/takeaways from working on the project.\n\n- Client/Server architecture is convenient, especially with OpenAPI specs.\n- Browser clients are great until they're not.\n- Llamafiles are relatively painless. \n- Python and PyInstaller pros/cons.\n- Github Actions and large files.\n- There's a lot of regular (non-AI) work that needs doing.\n\n## The Client / Server Architecture is convenient, especially with OpenAPI specs.\n\nThis is probably a boring point to start with, and it's old news. Still, I thought it'd be worth mentioning a couple of ways that it turned out to be nice to have the main guts of the application implemented behind an HTTP server.\n\nWhen I was testing out `llamafiles`, I wanted to try enabling GPU acceleration, but my main development machine had some compatibility issues. Since Memory Cache was built as a separate client/server, I could just run the server on another machine (with a compatible GPU) and run the client on my main development machine. It was super painless.\n\nWe built Memory Cache with on-device AI in mind, but another form that could make sense is to run AI workloads on a dedicated homelab server (e.g. \"an Xbox for AI\") or in a private cloud. If the AI apps expose everything over HTTP apis, it's easy to play around with these kinds of setups.\n\nAnother time I was glad to have the server implemented separately from the client was when I wanted to build an emacs plugin that leveraged RAG + LLMs for my programming projects. I wrote about the project [on my blog](https://www.johnshaughnessy.com/blog/posts/acorn_pal_emacs). As I was building the plugin I realized that I could probably just plug in to Memory Cache Hub instead of building another RAG/LLM app. It ended up working great!\n\nUnfortunately, I stopped working on the emacs plugin and left it in an unfinished state, mostly because I couldn't generate `elisp` client code for Memory Cache Hub's [Open API](https://openapi-generator.tech/docs/generators) spec. By the way, if anyone wants to write an elisp generator for Open API, that would be really great!\n\nI ended up generating typescript code from the OpenAPI spec for use in the [Memory Cache Browser Client](https://github.com/Mozilla-Ocho/Memory-Cache-Browser-Client). The relevant bit of code was this:\n\n```sh\n# Download the openapi.json spec from the server\ncurl http://localhost:4444/openapi.json > $PROJECT_ROOT/openapi.json\n# Generate typescript code\nyarn openapi-generator-cli generate -i $PROJECT_ROOT/openapi.json -g typescript-fetch -o $PROJECT_ROOT/src/api/\n```\n\n## Browser Clients Are Great, Until They're Not\n\nI am familiar with web front end tools -- React, javascript, parcel, css, canvas, etc. So, I liked the idea of building a front end for Memory Cache in the browser. No need to bundle things with [electron](https://www.electronjs.org/), and no need to train other developers I might be working with (who were also mostly familiar with web development).\n\nFor the most part, this worked out great. While the UI isn't \"beautiful\" or \"breathtaking\" -- it was painless and quick to build and it'd be easy for someone who really cared about it to come in and improve things.\n\nThat said, there were a couple of areas where working in the browser was pretty frustrating:\n\n1. You can't specify directories via a file picker. \n2. You can't directly send the user to a file URL.\n\n### No file picker for me\n\nThe way Memory Cache works is that the user specifies files in their filesystem that they want to add to their \"cache\"s. The server will make its own copies of the files in those directories for ingestion and such. The problem is that while browsers have built-in support for a file upload window, there's no way to tell the browser that we want the user to specify full paths to directories on their hard drive.\n\nIt's not surprising that browser's don't support this. This isn't really what they're made for. But it means that for this initial version of Memory Cache Hub, I settled for telling the user to type complete file paths into an input field rather than having a file picker UI. This feels really bad and particularly unpolished, even for a demo app.\n\n<figure>\n<img class=\"rounded-rect\" src=\"https://memorycache.ai/assets/images/2024-04-19-memory-cache-hub/screenshot_003.png\">\n<figcaption></figcaption>\n</figure>\n\n### No file previews\n\nThe browser acts as a file viewer if you specify the path of a file prefixed with `file://` in the address bar. This is convenient, because I wanted to let users easily view the files in their cache.\n\nUnfortunately, due to security concerns, the browser disallows redirects to `file://` links. This means that the best I could do for Memory Cache was provide a \"copy\" button that puts the `file://` URI onto the user's clipboard. Then, they can open a new tab, paste the URL and preview the file. This is a much worse experience.\n\n<figure>\n<img class=\"rounded-rect\" src=\"https://memorycache.ai/assets/images/2024-04-19-memory-cache-hub/screenshot_016.png\">\n<figcaption></figcaption>\n</figure>\n\nMy client could also have provided file previews (e.g. of PDF's) directly with the server sending the file contents to the client, but I didn't end up going down this route.\n\nAgain, this isn't surprising because I'm mostly using the browser as an application UI toolkit and that's not really what it's for. Electron (or something like it) would have been a better choice here.\n\n## Llamafiles are (relatively) painless\n\nUsing `llamafiles` for inference turned out to be an easy win. The dependencies of my python application stayed pretty simple because I didn't need to bring in hugging face / pytorch dependencies (and further separate platforms along `CUDA`/`ROCm`/`CPU` boundaries).\n\n<figure>\n<img class=\"rounded-rect\" src=\"https://memorycache.ai/assets/images/2024-04-19-memory-cache-hub/screenshot_009.png\">\n<figcaption></figcaption>\n</figure>\n\nThere are some \"gotchas\" with using `llamafiles`, most of which are documented in the [`llamafile README`](https://github.com/Mozilla-Ocho/llamafile?tab=readme-ov-file). For example, I don't end up enabling GPU support because I didn't spend time on handling errors that can occur if `llamafile` fails to move model weights to GPU for whatever reason. There are also still some platform-specific troubleshooting tips you need to follow if the `llamafile` server fails to start for whatever reason.\n\nStill, my overall feeling was that this was a pretty nice way to bundle an inference server with an application, and I hope to see more models bundled as `llamafile`s in the future.\n\n## Python and PyInstaller Pros and Cons \n\nI'm not very deeply embedded in the Python world, so figuring out how people built end-user programs with Python was new to me. For example, I know that [Blender](https://www.blender.org/) has a lot of python code, but as far as I can tell, the core is built with C and C++.\n\nI found `PyInstaller` and had success building standalone executables with it (as described in [this previous blog post](https://memorycache.ai/developer-blog/2024/03/07/devlog.html) and [this one too](https://memorycache.ai/developer-blog/2024/03/15/devlog.html)).\n\nIt worked, which is great. But there were some hurdles and downsides.\n\nThe first complaint is about the way `single-file` builds work. At startup, they need to unpack the supporting files. In our case, we had something like ~10,000 supporting files (which is probably our fault, not `PyInstaller`s) that get unpacked to a temporary directory. This takes ~30 seconds of basically just waiting around with no progress indicator or anything else of that nature. `PyInstaller` has an experimental feature for [adding a splash screen](https://pyinstaller.org/en/stable/usage.html#splash-screen-experimental), but I didn't end up trying it out because of the disclaimer at the top explaining that the feature doesn't work on MacOS. So, the single-file executable version of Memory Cache Hub appears as if it hangs for 30 seconds when you start it before eventually finishing the unpacking process.\n\nThe second complaint is not really about `PyInstaller` and more about using Python at all, which is that in the end we're still running a python interpreter at runtime. There's no real \"compile to bytecode/machine code\" (except for those dependencies written in something like [Cython](https://cython.org/)). It seems like python is the most well-supported ecosystem for developer tools for ML / AI, and part of me wishes I were spending my time in C or Rust. Not that I'm excellent with those languages, but considering that I get better at whatever I spend time doing, I'd rather be getting better at things that give me more control over what the computer is actually doing.\n\n<figure>\n<img class=\"rounded-rect\" src=\"https://memorycache.ai/assets/images/2024-04-19-memory-cache-hub/screenshot_018.png\">\n<figcaption></figcaption>\n</figure>\n\nNothing is stopping me from choosing different tools for my next project, and after all - `llama.cpp` is pretty darn popular and I'm looking forward to trying the (rust-based) [burn](https://github.com/tracel-ai/burn) project.\n\n## Github Actions and Large Files\n\nOk, so here's another problem with my gigantic bundled python executables with 10,000 files... My build pipeline takes 2+ hours to finish! \n\n<figure>\n<img class=\"rounded-rect\" src=\"https://memorycache.ai/assets/images/2024-04-19-memory-cache-hub/screenshot_021.png\">\n<figcaption></figcaption>\n</figure>\n\nUploading the build artifacts from the runner to github takes a long time -- especially for the zips that have over 10,000 files in them. This feels pretty terrible. Again, I think the problem is not with Github or PyInstaller or anything like that -- The problem is thinking that shipping 10,000 files was a good idea. It wasn't -- I regret it. haha.\n\n<figure>\n<img class=\"rounded-rect\" src=\"https://memorycache.ai/assets/images/2024-04-19-memory-cache-hub/screenshot_020.png\">\n<figcaption></figcaption>\n</figure>\n\n\n## There's a lot of non-AI work to be done\n\n90% of the effort I put into this project was completely unrelated to AI, machine learning, rag, etc. It was all, \"How do I build a python server\", \"How do we want this browser client to work?\", \"What's PyInstaller?\", \"How do we set up Github Actions?\" etc.\n\nThe idea was that once we had all of this ground work out of the way, we'd have a (user-facing) playground to experiment with whatever AI stuff we wanted. That's all fine and good, but I'm not sure how much time we're going to actually spend in that experiment phase, since in the meantime, there have been many other projects vying for our attention.\n\nMy thoughts about this at the moment are two fold.\n\nFirst, if your main goal is to experiment and learn something about AI or ML -- Don't bother trying to wrap it in an end-user application. Just write your python program or Jupyter notebook or whatever and do the learning. Don't worry if it doesn't work on other platforms or only supports whatever kind of GPU you happen to be running -- none of that changes the math / AI / deep learning / ML stuff that you were actually interested in. All of that other stuff is a distraction if all you wanted was the core thing.\n\nHowever -- if your goal is to experiment with an AI or ML thing that you want people to use -- Then get those people on board using your thing as fast as possible, even if that means getting on a call with that, having them share their desktop and following your instructions to set up a python environment. Whatever you need to do to get them actually running your code and using your thing and giving you feedback -- that's the hurdle you should cross. That doesn't mean you have to ship out to the whole world. Maybe you know your thing is not ready for that. But if you have a particular user in mind and you want them involved and giving you constant feedback, it's good to bring them in early.\n\n## What Now?\n\nLearning the build / deploy side of things was pretty helpful and useful. I'd never built a python application like this one before, and I enjoyed myself along the way.\n\nThere's been some interest in connecting Memory Cache more directly with the browser history, with Slack, with emails, and other document/information sources. That direction is probably pretty useful -- and a lot of other people are exploring that space too.\n\nHowever, I'll likely leave that to others. My next projects will be unrelated to Memory Cache. There are a lot of simple ideas I want to play around with in the space just to deepen my understanding of LLMs, and there are a lot of projects external to Mozilla that I'd like to learn more about and maybe contribute to.\n\n## Screenshots\n\n<figure>\n<img class=\"rounded-rect\" src=\"https://memorycache.ai/assets/images/2024-04-19-memory-cache-hub/screenshot_001.png\">\n<figcaption>Files</figcaption>\n</figure>\n\n<figure>\n<img class=\"rounded-rect\" src=\"https://memorycache.ai/assets/images/2024-04-19-memory-cache-hub/screenshot_022.png\">\n<figcaption>About Memory Cache</figcaption>\n</figure>\n\n<figure>\n<img class=\"rounded-rect\" src=\"https://memorycache.ai/assets/images/2024-04-19-memory-cache-hub/screenshot_004.png\">\n<figcaption>Vector Search</figcaption>\n</figure>\n\n<figure>\n<img class=\"rounded-rect\" src=\"https://memorycache.ai/assets/images/2024-04-19-memory-cache-hub/screenshot_005.png\">\n<figcaption>Chat depends on a model running</figcaption>\n</figure>\n\n<figure>\n<img class=\"rounded-rect\" src=\"https://memorycache.ai/assets/images/2024-04-19-memory-cache-hub/screenshot_007.png\">\n<figcaption>The model selection page</figcaption>\n</figure>\n\n<figure>\n<img class=\"rounded-rect\" src=\"https://memorycache.ai/assets/images/2024-04-19-memory-cache-hub/screenshot_009.png\">\n<figcaption>The model selection page</figcaption>\n</figure>\n\n<figure>\n<img class=\"rounded-rect\" src=\"https://memorycache.ai/assets/images/2024-04-19-memory-cache-hub/screenshot_010.png\">\n<figcaption>Retrieval augmented chat</figcaption>\n</figure>\n\n\n\n\n"
  },
  {
    "path": "docs/_sass/memorycache.scss",
    "content": "@import url('https://fonts.googleapis.com/css2?family=Work+Sans:wght@300;400;500;600&display=swap');\n\nbody {\n    font-family: 'Work Sans';\n}\n\n.site-title {\n    font-family: 'Work Sans';\n    color: coral;\n}\n\n.introduction {\n    background-image: url('../assets/images/header-background.png');\n    background-repeat:no-repeat;\n    background-size: 675px 210.75px;\n    background-position: right;\n    width: 100%;\n    min-height: 201px;\n\n    border-bottom-width: 1px;\n    border-bottom-style: solid;\n    border-bottom-color: #F0EFEF;\n}\n\n.page-content {\n  background-color: white;\n}\n\na {\n    color: #180AB8;\n}\n\n.site-logo {\n    width: 166px;\n    height: 36px;\n}\n\n.site-header {\n    border-top-width: 5px;\n    border-top-style: solid;\n    border-image: linear-gradient(90deg, #FFB3BB 0%, #FFDFBA 26.56%, #FFFFBA 50.52%, #87EBDA 76.04%, #BAE1FF 100%) 1 0 0 0;\n\n}\n\n.callout-left {\n  width: 75%;\n  padding-top: 5%;\n}\n\n.page-content {\n  border-top-width: 1px;\n  border-top-style: solid;\n  border-top-color: #F0EFEF;\n\n}\n\n.post-list-heading {\n  font-size: 16px;\n  padding-top: 5px;\n}\n\n.post-link {\n  font-size: 16px;\n}\n\n.moz-logo {\n  width: 128px;\n  float: right;\n}\n\n.detailed-overview {\n  padding-top: 5px;\n  border-top-width: 1px;\n  border-bottom-width: 1px;\n  border-top-style: solid;\n  border-bottom-style: solid;\n  border-top-color: #F0EFEF;\n  border-bottom-color: #F0EFEF;\n}\n\nfigcaption {\n    text-align: center;\n    font-style: italic;\n    margin: 1em 0 3em 0;\n}\n"
  },
  {
    "path": "docs/_sass/minima.scss",
    "content": "@charset \"utf-8\";\n\n// Define defaults for each variable.\n\n$base-font-family: -apple-system, BlinkMacSystemFont, \"Segoe UI\", Roboto, Helvetica, Arial, sans-serif, \"Apple Color Emoji\", \"Segoe UI Emoji\", \"Segoe UI Symbol\" !default;\n$base-font-size:   16px !default;\n$base-font-weight: 400 !default;\n$small-font-size:  $base-font-size * 0.875 !default;\n$base-line-height: 1.5 !default;\n\n$spacing-unit:     30px !default;\n\n$text-color:       #111 !default;\n$background-color: #fdfdfd !default;\n$brand-color:      #2a7ae2 !default;\n\n$grey-color:       #828282 !default;\n$grey-color-light: lighten($grey-color, 40%) !default;\n$grey-color-dark:  darken($grey-color, 25%) !default;\n\n$table-text-align: left !default;\n\n// Width of the content area\n$content-width:    800px !default;\n\n$on-palm:          600px !default;\n$on-laptop:        800px !default;\n\n// Use media queries like this:\n// @include media-query($on-palm) {\n//   .wrapper {\n//     padding-right: $spacing-unit / 2;\n//     padding-left: $spacing-unit / 2;\n//   }\n// }\n@mixin media-query($device) {\n  @media screen and (max-width: $device) {\n    @content;\n  }\n}\n\n@mixin relative-font-size($ratio) {\n  font-size: $base-font-size * $ratio;\n}\n\n// Import partials.\n@import\n  \"minima/base\",\n  \"minima/layout\",\n  \"minima/syntax-highlighting\"\n;\n"
  },
  {
    "path": "docs/about.markdown",
    "content": "---\nlayout: page\ntitle: About\npermalink: /about/\n---\n\nMemory Cache is an exploration into synthesis, discovery, and sharing of insights more effectively through the use of technology. \n\nUnlike most explorations into artificial intelligence, Memory Cache is designed to be completely personalized, on-device, and private. It is meant to explore the nuances of how individuals think, from the perspective of learning alongside us over time from the articles we read and save. \n"
  },
  {
    "path": "docs/assets/main.scss",
    "content": "---\n---\n\n@import \"minima\";\n@import \"memorycache\";\n\n"
  },
  {
    "path": "docs/faq.md",
    "content": "---\nlayout: default\ntitle: FAQ\n---\n# Frequently Asked Questions\n\n**Q: How do I try MemoryCache?**\n\nRight now (as of December 7, 2023), MemoryCache requires a few manual steps to set up the end to end workflow. There are three components: a) a Firefox extension, b) a local instance of privateGPT, and c) a symlinked folder between privateGPT and your local Downloads folder. There is also an optional configuration that can be done to a private build of Firefox to save files to your local machine as PDF files instead of HTML files. Check out the [GitHub repository](https://github.com/Mozilla-Ocho/Memory-Cache) for more detailed instructions. We are looking into ways to streamline the deployment of MemoryCache to require less manual configuration, but if you're here at this stage, you're at the very earliest stages of our explorations. \n\n**Q: Does MemoryCache send my data anywhere?**\n\nNo. One of the core principles of MemoryCache is that you have full control over the system, and that it all stays on your device. If you're a developer or someone who just likes to tinker with your computer applications, and you want to cloud-ify this, feel free! But we're looking to stay entirely local. \n\n**Q: Why is MemoryCache using an old language model and primordial privateGPT?**\n\nMemoryCache is using an old language model ([Nomic AI's gpt4all-j v1.3 groovy.ggml](https://huggingface.co/nomic-ai/gpt4all-j)) and primordial privateGPT because right now, this combo is the one that passes our criteria for the type of responses that it generates. This tech is almost a year old, and there have been many advancements in local AI that we'll be integrating in over time, but we're a small team exploring a lot of different subsets of this problem space and the quality of the insight generated is a sweet spot that we want to preserve. This is a temporary tradeoff, but we want to be careful to keep a consistent benchmark for insight generation.\n\nGPT-J was trained on the 'Pile' dataset, and the versions between the 1.0 release and 1.3 release also added the ShareGPT and Dolly datasettes. The Databricks Dolly dataset is licensed under the Creative Commons license with human contributions and wikipedia entries. The ShareGPT dataset is human prompts and ChatGPT output responses that were submitted by human users. \n\n**Q: What kind of tasks would I use MemoryCache for?**\n\nMemoryCache is ultimately leaning into the weird and creative parts of human insight. The goal with MemoryCache is to \"learn what you learn\", which is why you are in control of what you want files you want to augment the application with. This can be helpful for research, brainstorming, creative writing, and synthesis of new ideas to connect seemingly unrelated topics together to find new insights and learnings from the body of knowledge that matters most to you. \n\n**Q: Is this a Firefox project?**\n\nNo. MemoryCache is a hackathon-style project by the Mozilla Innovation Ecosystem team, not a Firefox project.While the project uses a Firefox extension as a way of collecting information, this is a set of scripts and tools to augment privateGPT, a native AI application. It's meant to streamline the process of saving information that you might read in the browser to store it in your own personal library of information.\n"
  },
  {
    "path": "docs/index.markdown",
    "content": "---\n# Feel free to add content and custom Front Matter to this file.\n# To modify the layout, see https://jekyllrb.com/docs/themes/#overriding-theme-defaults\n\nlayout: home\n---\n"
  },
  {
    "path": "docs/readme.md",
    "content": "Placeholder\n"
  },
  {
    "path": "extension/content-script.js",
    "content": "function getPageText() {\n  const head = document.head.innerHTML;\n  const body = document.body.innerText;\n  return `<!DOCTYPE html>\\n<head>\\n${head}\\n</head>\\n<body>\\n${body}\\n</body>\\n</html>`;\n}\n\nbrowser.runtime.onMessage.addListener((message, _sender) => {\n  console.log(\"[MemoryCache Extension] Received message:\", message);\n  if (message.action === \"getPageText\") {\n    return Promise.resolve(getPageText());\n  }\n});\n"
  },
  {
    "path": "extension/manifest.json",
    "content": "{\n    \"manifest_version\" : 2,\n    \"name\": \"MemoryCache\",\n    \"version\" : \"1.0\",\n    \"description\" : \"Saves a copy of reader view of a tab to a specific directory\",\n    \"icons\" : {\n        \"48\" : \"icons/memwrite-48.png\"\n    }, \n    \"permissions\" : [\n        \"downloads\", \n        \"<all_urls>\",\n        \"tabs\",\n        \"storage\"\n    ], \n    \"browser_action\" : {\n        \"browser_style\" : true,\n        \"default_icon\" : \"icons/memwrite-32.png\", \n        \"default_title\" : \"Memory Cache\", \n        \"default_popup\" : \"popup/memory_cache.html\"\n    },\n    \"content_scripts\": [\n        {\n            \"matches\": [\"<all_urls>\"],\n            \"js\": [\"content-script.js\"]\n        }\n    ]\n}\n"
  },
  {
    "path": "extension/popup/marked.esm.js",
    "content": "/**\n * marked v10.0.0 - a markdown parser\n * Copyright (c) 2011-2023, Christopher Jeffrey. (MIT Licensed)\n * https://github.com/markedjs/marked\n */\n\n/**\n * DO NOT EDIT THIS FILE\n * The code in this file is generated from files in ./src/\n */\n\n/**\n * Gets the original marked default options.\n */\nfunction _getDefaults() {\n    return {\n        async: false,\n        breaks: false,\n        extensions: null,\n        gfm: true,\n        hooks: null,\n        pedantic: false,\n        renderer: null,\n        silent: false,\n        tokenizer: null,\n        walkTokens: null\n    };\n}\nlet _defaults = _getDefaults();\nfunction changeDefaults(newDefaults) {\n    _defaults = newDefaults;\n}\n\n/**\n * Helpers\n */\nconst escapeTest = /[&<>\"']/;\nconst escapeReplace = new RegExp(escapeTest.source, 'g');\nconst escapeTestNoEncode = /[<>\"']|&(?!(#\\d{1,7}|#[Xx][a-fA-F0-9]{1,6}|\\w+);)/;\nconst escapeReplaceNoEncode = new RegExp(escapeTestNoEncode.source, 'g');\nconst escapeReplacements = {\n    '&': '&amp;',\n    '<': '&lt;',\n    '>': '&gt;',\n    '\"': '&quot;',\n    \"'\": '&#39;'\n};\nconst getEscapeReplacement = (ch) => escapeReplacements[ch];\nfunction escape(html, encode) {\n    if (encode) {\n        if (escapeTest.test(html)) {\n            return html.replace(escapeReplace, getEscapeReplacement);\n        }\n    }\n    else {\n        if (escapeTestNoEncode.test(html)) {\n            return html.replace(escapeReplaceNoEncode, getEscapeReplacement);\n        }\n    }\n    return html;\n}\nconst unescapeTest = /&(#(?:\\d+)|(?:#x[0-9A-Fa-f]+)|(?:\\w+));?/ig;\nfunction unescape(html) {\n    // explicitly match decimal, hex, and named HTML entities\n    return html.replace(unescapeTest, (_, n) => {\n        n = n.toLowerCase();\n        if (n === 'colon')\n            return ':';\n        if (n.charAt(0) === '#') {\n            return n.charAt(1) === 'x'\n                ? String.fromCharCode(parseInt(n.substring(2), 16))\n                : String.fromCharCode(+n.substring(1));\n        }\n        return '';\n    });\n}\nconst caret = /(^|[^\\[])\\^/g;\nfunction edit(regex, opt) {\n    regex = typeof regex === 'string' ? regex : regex.source;\n    opt = opt || '';\n    const obj = {\n        replace: (name, val) => {\n            val = typeof val === 'object' && 'source' in val ? val.source : val;\n            val = val.replace(caret, '$1');\n            regex = regex.replace(name, val);\n            return obj;\n        },\n        getRegex: () => {\n            return new RegExp(regex, opt);\n        }\n    };\n    return obj;\n}\nfunction cleanUrl(href) {\n    try {\n        href = encodeURI(href).replace(/%25/g, '%');\n    }\n    catch (e) {\n        return null;\n    }\n    return href;\n}\nconst noopTest = { exec: () => null };\nfunction splitCells(tableRow, count) {\n    // ensure that every cell-delimiting pipe has a space\n    // before it to distinguish it from an escaped pipe\n    const row = tableRow.replace(/\\|/g, (match, offset, str) => {\n        let escaped = false;\n        let curr = offset;\n        while (--curr >= 0 && str[curr] === '\\\\')\n            escaped = !escaped;\n        if (escaped) {\n            // odd number of slashes means | is escaped\n            // so we leave it alone\n            return '|';\n        }\n        else {\n            // add space before unescaped |\n            return ' |';\n        }\n    }), cells = row.split(/ \\|/);\n    let i = 0;\n    // First/last cell in a row cannot be empty if it has no leading/trailing pipe\n    if (!cells[0].trim()) {\n        cells.shift();\n    }\n    if (cells.length > 0 && !cells[cells.length - 1].trim()) {\n        cells.pop();\n    }\n    if (count) {\n        if (cells.length > count) {\n            cells.splice(count);\n        }\n        else {\n            while (cells.length < count)\n                cells.push('');\n        }\n    }\n    for (; i < cells.length; i++) {\n        // leading or trailing whitespace is ignored per the gfm spec\n        cells[i] = cells[i].trim().replace(/\\\\\\|/g, '|');\n    }\n    return cells;\n}\n/**\n * Remove trailing 'c's. Equivalent to str.replace(/c*$/, '').\n * /c*$/ is vulnerable to REDOS.\n *\n * @param str\n * @param c\n * @param invert Remove suffix of non-c chars instead. Default falsey.\n */\nfunction rtrim(str, c, invert) {\n    const l = str.length;\n    if (l === 0) {\n        return '';\n    }\n    // Length of suffix matching the invert condition.\n    let suffLen = 0;\n    // Step left until we fail to match the invert condition.\n    while (suffLen < l) {\n        const currChar = str.charAt(l - suffLen - 1);\n        if (currChar === c && !invert) {\n            suffLen++;\n        }\n        else if (currChar !== c && invert) {\n            suffLen++;\n        }\n        else {\n            break;\n        }\n    }\n    return str.slice(0, l - suffLen);\n}\nfunction findClosingBracket(str, b) {\n    if (str.indexOf(b[1]) === -1) {\n        return -1;\n    }\n    let level = 0;\n    for (let i = 0; i < str.length; i++) {\n        if (str[i] === '\\\\') {\n            i++;\n        }\n        else if (str[i] === b[0]) {\n            level++;\n        }\n        else if (str[i] === b[1]) {\n            level--;\n            if (level < 0) {\n                return i;\n            }\n        }\n    }\n    return -1;\n}\n\nfunction outputLink(cap, link, raw, lexer) {\n    const href = link.href;\n    const title = link.title ? escape(link.title) : null;\n    const text = cap[1].replace(/\\\\([\\[\\]])/g, '$1');\n    if (cap[0].charAt(0) !== '!') {\n        lexer.state.inLink = true;\n        const token = {\n            type: 'link',\n            raw,\n            href,\n            title,\n            text,\n            tokens: lexer.inlineTokens(text)\n        };\n        lexer.state.inLink = false;\n        return token;\n    }\n    return {\n        type: 'image',\n        raw,\n        href,\n        title,\n        text: escape(text)\n    };\n}\nfunction indentCodeCompensation(raw, text) {\n    const matchIndentToCode = raw.match(/^(\\s+)(?:```)/);\n    if (matchIndentToCode === null) {\n        return text;\n    }\n    const indentToCode = matchIndentToCode[1];\n    return text\n        .split('\\n')\n        .map(node => {\n        const matchIndentInNode = node.match(/^\\s+/);\n        if (matchIndentInNode === null) {\n            return node;\n        }\n        const [indentInNode] = matchIndentInNode;\n        if (indentInNode.length >= indentToCode.length) {\n            return node.slice(indentToCode.length);\n        }\n        return node;\n    })\n        .join('\\n');\n}\n/**\n * Tokenizer\n */\nclass _Tokenizer {\n    options;\n    // TODO: Fix this rules type\n    rules;\n    lexer;\n    constructor(options) {\n        this.options = options || _defaults;\n    }\n    space(src) {\n        const cap = this.rules.block.newline.exec(src);\n        if (cap && cap[0].length > 0) {\n            return {\n                type: 'space',\n                raw: cap[0]\n            };\n        }\n    }\n    code(src) {\n        const cap = this.rules.block.code.exec(src);\n        if (cap) {\n            const text = cap[0].replace(/^ {1,4}/gm, '');\n            return {\n                type: 'code',\n                raw: cap[0],\n                codeBlockStyle: 'indented',\n                text: !this.options.pedantic\n                    ? rtrim(text, '\\n')\n                    : text\n            };\n        }\n    }\n    fences(src) {\n        const cap = this.rules.block.fences.exec(src);\n        if (cap) {\n            const raw = cap[0];\n            const text = indentCodeCompensation(raw, cap[3] || '');\n            return {\n                type: 'code',\n                raw,\n                lang: cap[2] ? cap[2].trim().replace(this.rules.inline._escapes, '$1') : cap[2],\n                text\n            };\n        }\n    }\n    heading(src) {\n        const cap = this.rules.block.heading.exec(src);\n        if (cap) {\n            let text = cap[2].trim();\n            // remove trailing #s\n            if (/#$/.test(text)) {\n                const trimmed = rtrim(text, '#');\n                if (this.options.pedantic) {\n                    text = trimmed.trim();\n                }\n                else if (!trimmed || / $/.test(trimmed)) {\n                    // CommonMark requires space before trailing #s\n                    text = trimmed.trim();\n                }\n            }\n            return {\n                type: 'heading',\n                raw: cap[0],\n                depth: cap[1].length,\n                text,\n                tokens: this.lexer.inline(text)\n            };\n        }\n    }\n    hr(src) {\n        const cap = this.rules.block.hr.exec(src);\n        if (cap) {\n            return {\n                type: 'hr',\n                raw: cap[0]\n            };\n        }\n    }\n    blockquote(src) {\n        const cap = this.rules.block.blockquote.exec(src);\n        if (cap) {\n            const text = rtrim(cap[0].replace(/^ *>[ \\t]?/gm, ''), '\\n');\n            const top = this.lexer.state.top;\n            this.lexer.state.top = true;\n            const tokens = this.lexer.blockTokens(text);\n            this.lexer.state.top = top;\n            return {\n                type: 'blockquote',\n                raw: cap[0],\n                tokens,\n                text\n            };\n        }\n    }\n    list(src) {\n        let cap = this.rules.block.list.exec(src);\n        if (cap) {\n            let bull = cap[1].trim();\n            const isordered = bull.length > 1;\n            const list = {\n                type: 'list',\n                raw: '',\n                ordered: isordered,\n                start: isordered ? +bull.slice(0, -1) : '',\n                loose: false,\n                items: []\n            };\n            bull = isordered ? `\\\\d{1,9}\\\\${bull.slice(-1)}` : `\\\\${bull}`;\n            if (this.options.pedantic) {\n                bull = isordered ? bull : '[*+-]';\n            }\n            // Get next list item\n            const itemRegex = new RegExp(`^( {0,3}${bull})((?:[\\t ][^\\\\n]*)?(?:\\\\n|$))`);\n            let raw = '';\n            let itemContents = '';\n            let endsWithBlankLine = false;\n            // Check if current bullet point can start a new List Item\n            while (src) {\n                let endEarly = false;\n                if (!(cap = itemRegex.exec(src))) {\n                    break;\n                }\n                if (this.rules.block.hr.test(src)) { // End list if bullet was actually HR (possibly move into itemRegex?)\n                    break;\n                }\n                raw = cap[0];\n                src = src.substring(raw.length);\n                let line = cap[2].split('\\n', 1)[0].replace(/^\\t+/, (t) => ' '.repeat(3 * t.length));\n                let nextLine = src.split('\\n', 1)[0];\n                let indent = 0;\n                if (this.options.pedantic) {\n                    indent = 2;\n                    itemContents = line.trimStart();\n                }\n                else {\n                    indent = cap[2].search(/[^ ]/); // Find first non-space char\n                    indent = indent > 4 ? 1 : indent; // Treat indented code blocks (> 4 spaces) as having only 1 indent\n                    itemContents = line.slice(indent);\n                    indent += cap[1].length;\n                }\n                let blankLine = false;\n                if (!line && /^ *$/.test(nextLine)) { // Items begin with at most one blank line\n                    raw += nextLine + '\\n';\n                    src = src.substring(nextLine.length + 1);\n                    endEarly = true;\n                }\n                if (!endEarly) {\n                    const nextBulletRegex = new RegExp(`^ {0,${Math.min(3, indent - 1)}}(?:[*+-]|\\\\d{1,9}[.)])((?:[ \\t][^\\\\n]*)?(?:\\\\n|$))`);\n                    const hrRegex = new RegExp(`^ {0,${Math.min(3, indent - 1)}}((?:- *){3,}|(?:_ *){3,}|(?:\\\\* *){3,})(?:\\\\n+|$)`);\n                    const fencesBeginRegex = new RegExp(`^ {0,${Math.min(3, indent - 1)}}(?:\\`\\`\\`|~~~)`);\n                    const headingBeginRegex = new RegExp(`^ {0,${Math.min(3, indent - 1)}}#`);\n                    // Check if following lines should be included in List Item\n                    while (src) {\n                        const rawLine = src.split('\\n', 1)[0];\n                        nextLine = rawLine;\n                        // Re-align to follow commonmark nesting rules\n                        if (this.options.pedantic) {\n                            nextLine = nextLine.replace(/^ {1,4}(?=( {4})*[^ ])/g, '  ');\n                        }\n                        // End list item if found code fences\n                        if (fencesBeginRegex.test(nextLine)) {\n                            break;\n                        }\n                        // End list item if found start of new heading\n                        if (headingBeginRegex.test(nextLine)) {\n                            break;\n                        }\n                        // End list item if found start of new bullet\n                        if (nextBulletRegex.test(nextLine)) {\n                            break;\n                        }\n                        // Horizontal rule found\n                        if (hrRegex.test(src)) {\n                            break;\n                        }\n                        if (nextLine.search(/[^ ]/) >= indent || !nextLine.trim()) { // Dedent if possible\n                            itemContents += '\\n' + nextLine.slice(indent);\n                        }\n                        else {\n                            // not enough indentation\n                            if (blankLine) {\n                                break;\n                            }\n                            // paragraph continuation unless last line was a different block level element\n                            if (line.search(/[^ ]/) >= 4) { // indented code block\n                                break;\n                            }\n                            if (fencesBeginRegex.test(line)) {\n                                break;\n                            }\n                            if (headingBeginRegex.test(line)) {\n                                break;\n                            }\n                            if (hrRegex.test(line)) {\n                                break;\n                            }\n                            itemContents += '\\n' + nextLine;\n                        }\n                        if (!blankLine && !nextLine.trim()) { // Check if current line is blank\n                            blankLine = true;\n                        }\n                        raw += rawLine + '\\n';\n                        src = src.substring(rawLine.length + 1);\n                        line = nextLine.slice(indent);\n                    }\n                }\n                if (!list.loose) {\n                    // If the previous item ended with a blank line, the list is loose\n                    if (endsWithBlankLine) {\n                        list.loose = true;\n                    }\n                    else if (/\\n *\\n *$/.test(raw)) {\n                        endsWithBlankLine = true;\n                    }\n                }\n                let istask = null;\n                let ischecked;\n                // Check for task list items\n                if (this.options.gfm) {\n                    istask = /^\\[[ xX]\\] /.exec(itemContents);\n                    if (istask) {\n                        ischecked = istask[0] !== '[ ] ';\n                        itemContents = itemContents.replace(/^\\[[ xX]\\] +/, '');\n                    }\n                }\n                list.items.push({\n                    type: 'list_item',\n                    raw,\n                    task: !!istask,\n                    checked: ischecked,\n                    loose: false,\n                    text: itemContents,\n                    tokens: []\n                });\n                list.raw += raw;\n            }\n            // Do not consume newlines at end of final item. Alternatively, make itemRegex *start* with any newlines to simplify/speed up endsWithBlankLine logic\n            list.items[list.items.length - 1].raw = raw.trimEnd();\n            list.items[list.items.length - 1].text = itemContents.trimEnd();\n            list.raw = list.raw.trimEnd();\n            // Item child tokens handled here at end because we needed to have the final item to trim it first\n            for (let i = 0; i < list.items.length; i++) {\n                this.lexer.state.top = false;\n                list.items[i].tokens = this.lexer.blockTokens(list.items[i].text, []);\n                if (!list.loose) {\n                    // Check if list should be loose\n                    const spacers = list.items[i].tokens.filter(t => t.type === 'space');\n                    const hasMultipleLineBreaks = spacers.length > 0 && spacers.some(t => /\\n.*\\n/.test(t.raw));\n                    list.loose = hasMultipleLineBreaks;\n                }\n            }\n            // Set all items to loose if list is loose\n            if (list.loose) {\n                for (let i = 0; i < list.items.length; i++) {\n                    list.items[i].loose = true;\n                }\n            }\n            return list;\n        }\n    }\n    html(src) {\n        const cap = this.rules.block.html.exec(src);\n        if (cap) {\n            const token = {\n                type: 'html',\n                block: true,\n                raw: cap[0],\n                pre: cap[1] === 'pre' || cap[1] === 'script' || cap[1] === 'style',\n                text: cap[0]\n            };\n            return token;\n        }\n    }\n    def(src) {\n        const cap = this.rules.block.def.exec(src);\n        if (cap) {\n            const tag = cap[1].toLowerCase().replace(/\\s+/g, ' ');\n            const href = cap[2] ? cap[2].replace(/^<(.*)>$/, '$1').replace(this.rules.inline._escapes, '$1') : '';\n            const title = cap[3] ? cap[3].substring(1, cap[3].length - 1).replace(this.rules.inline._escapes, '$1') : cap[3];\n            return {\n                type: 'def',\n                tag,\n                raw: cap[0],\n                href,\n                title\n            };\n        }\n    }\n    table(src) {\n        const cap = this.rules.block.table.exec(src);\n        if (cap) {\n            if (!/[:|]/.test(cap[2])) {\n                // delimiter row must have a pipe (|) or colon (:) otherwise it is a setext heading\n                return;\n            }\n            const item = {\n                type: 'table',\n                raw: cap[0],\n                header: splitCells(cap[1]).map(c => {\n                    return { text: c, tokens: [] };\n                }),\n                align: cap[2].replace(/^\\||\\| *$/g, '').split('|'),\n                rows: cap[3] && cap[3].trim() ? cap[3].replace(/\\n[ \\t]*$/, '').split('\\n') : []\n            };\n            if (item.header.length === item.align.length) {\n                let l = item.align.length;\n                let i, j, k, row;\n                for (i = 0; i < l; i++) {\n                    const align = item.align[i];\n                    if (align) {\n                        if (/^ *-+: *$/.test(align)) {\n                            item.align[i] = 'right';\n                        }\n                        else if (/^ *:-+: *$/.test(align)) {\n                            item.align[i] = 'center';\n                        }\n                        else if (/^ *:-+ *$/.test(align)) {\n                            item.align[i] = 'left';\n                        }\n                        else {\n                            item.align[i] = null;\n                        }\n                    }\n                }\n                l = item.rows.length;\n                for (i = 0; i < l; i++) {\n                    item.rows[i] = splitCells(item.rows[i], item.header.length).map(c => {\n                        return { text: c, tokens: [] };\n                    });\n                }\n                // parse child tokens inside headers and cells\n                // header child tokens\n                l = item.header.length;\n                for (j = 0; j < l; j++) {\n                    item.header[j].tokens = this.lexer.inline(item.header[j].text);\n                }\n                // cell child tokens\n                l = item.rows.length;\n                for (j = 0; j < l; j++) {\n                    row = item.rows[j];\n                    for (k = 0; k < row.length; k++) {\n                        row[k].tokens = this.lexer.inline(row[k].text);\n                    }\n                }\n                return item;\n            }\n        }\n    }\n    lheading(src) {\n        const cap = this.rules.block.lheading.exec(src);\n        if (cap) {\n            return {\n                type: 'heading',\n                raw: cap[0],\n                depth: cap[2].charAt(0) === '=' ? 1 : 2,\n                text: cap[1],\n                tokens: this.lexer.inline(cap[1])\n            };\n        }\n    }\n    paragraph(src) {\n        const cap = this.rules.block.paragraph.exec(src);\n        if (cap) {\n            const text = cap[1].charAt(cap[1].length - 1) === '\\n'\n                ? cap[1].slice(0, -1)\n                : cap[1];\n            return {\n                type: 'paragraph',\n                raw: cap[0],\n                text,\n                tokens: this.lexer.inline(text)\n            };\n        }\n    }\n    text(src) {\n        const cap = this.rules.block.text.exec(src);\n        if (cap) {\n            return {\n                type: 'text',\n                raw: cap[0],\n                text: cap[0],\n                tokens: this.lexer.inline(cap[0])\n            };\n        }\n    }\n    escape(src) {\n        const cap = this.rules.inline.escape.exec(src);\n        if (cap) {\n            return {\n                type: 'escape',\n                raw: cap[0],\n                text: escape(cap[1])\n            };\n        }\n    }\n    tag(src) {\n        const cap = this.rules.inline.tag.exec(src);\n        if (cap) {\n            if (!this.lexer.state.inLink && /^<a /i.test(cap[0])) {\n                this.lexer.state.inLink = true;\n            }\n            else if (this.lexer.state.inLink && /^<\\/a>/i.test(cap[0])) {\n                this.lexer.state.inLink = false;\n            }\n            if (!this.lexer.state.inRawBlock && /^<(pre|code|kbd|script)(\\s|>)/i.test(cap[0])) {\n                this.lexer.state.inRawBlock = true;\n            }\n            else if (this.lexer.state.inRawBlock && /^<\\/(pre|code|kbd|script)(\\s|>)/i.test(cap[0])) {\n                this.lexer.state.inRawBlock = false;\n            }\n            return {\n                type: 'html',\n                raw: cap[0],\n                inLink: this.lexer.state.inLink,\n                inRawBlock: this.lexer.state.inRawBlock,\n                block: false,\n                text: cap[0]\n            };\n        }\n    }\n    link(src) {\n        const cap = this.rules.inline.link.exec(src);\n        if (cap) {\n            const trimmedUrl = cap[2].trim();\n            if (!this.options.pedantic && /^</.test(trimmedUrl)) {\n                // commonmark requires matching angle brackets\n                if (!(/>$/.test(trimmedUrl))) {\n                    return;\n                }\n                // ending angle bracket cannot be escaped\n                const rtrimSlash = rtrim(trimmedUrl.slice(0, -1), '\\\\');\n                if ((trimmedUrl.length - rtrimSlash.length) % 2 === 0) {\n                    return;\n                }\n            }\n            else {\n                // find closing parenthesis\n                const lastParenIndex = findClosingBracket(cap[2], '()');\n                if (lastParenIndex > -1) {\n                    const start = cap[0].indexOf('!') === 0 ? 5 : 4;\n                    const linkLen = start + cap[1].length + lastParenIndex;\n                    cap[2] = cap[2].substring(0, lastParenIndex);\n                    cap[0] = cap[0].substring(0, linkLen).trim();\n                    cap[3] = '';\n                }\n            }\n            let href = cap[2];\n            let title = '';\n            if (this.options.pedantic) {\n                // split pedantic href and title\n                const link = /^([^'\"]*[^\\s])\\s+(['\"])(.*)\\2/.exec(href);\n                if (link) {\n                    href = link[1];\n                    title = link[3];\n                }\n            }\n            else {\n                title = cap[3] ? cap[3].slice(1, -1) : '';\n            }\n            href = href.trim();\n            if (/^</.test(href)) {\n                if (this.options.pedantic && !(/>$/.test(trimmedUrl))) {\n                    // pedantic allows starting angle bracket without ending angle bracket\n                    href = href.slice(1);\n                }\n                else {\n                    href = href.slice(1, -1);\n                }\n            }\n            return outputLink(cap, {\n                href: href ? href.replace(this.rules.inline._escapes, '$1') : href,\n                title: title ? title.replace(this.rules.inline._escapes, '$1') : title\n            }, cap[0], this.lexer);\n        }\n    }\n    reflink(src, links) {\n        let cap;\n        if ((cap = this.rules.inline.reflink.exec(src))\n            || (cap = this.rules.inline.nolink.exec(src))) {\n            let link = (cap[2] || cap[1]).replace(/\\s+/g, ' ');\n            link = links[link.toLowerCase()];\n            if (!link) {\n                const text = cap[0].charAt(0);\n                return {\n                    type: 'text',\n                    raw: text,\n                    text\n                };\n            }\n            return outputLink(cap, link, cap[0], this.lexer);\n        }\n    }\n    emStrong(src, maskedSrc, prevChar = '') {\n        let match = this.rules.inline.emStrong.lDelim.exec(src);\n        if (!match)\n            return;\n        // _ can't be between two alphanumerics. \\p{L}\\p{N} includes non-english alphabet/numbers as well\n        if (match[3] && prevChar.match(/[\\p{L}\\p{N}]/u))\n            return;\n        const nextChar = match[1] || match[2] || '';\n        if (!nextChar || !prevChar || this.rules.inline.punctuation.exec(prevChar)) {\n            // unicode Regex counts emoji as 1 char; spread into array for proper count (used multiple times below)\n            const lLength = [...match[0]].length - 1;\n            let rDelim, rLength, delimTotal = lLength, midDelimTotal = 0;\n            const endReg = match[0][0] === '*' ? this.rules.inline.emStrong.rDelimAst : this.rules.inline.emStrong.rDelimUnd;\n            endReg.lastIndex = 0;\n            // Clip maskedSrc to same section of string as src (move to lexer?)\n            maskedSrc = maskedSrc.slice(-1 * src.length + lLength);\n            while ((match = endReg.exec(maskedSrc)) != null) {\n                rDelim = match[1] || match[2] || match[3] || match[4] || match[5] || match[6];\n                if (!rDelim)\n                    continue; // skip single * in __abc*abc__\n                rLength = [...rDelim].length;\n                if (match[3] || match[4]) { // found another Left Delim\n                    delimTotal += rLength;\n                    continue;\n                }\n                else if (match[5] || match[6]) { // either Left or Right Delim\n                    if (lLength % 3 && !((lLength + rLength) % 3)) {\n                        midDelimTotal += rLength;\n                        continue; // CommonMark Emphasis Rules 9-10\n                    }\n                }\n                delimTotal -= rLength;\n                if (delimTotal > 0)\n                    continue; // Haven't found enough closing delimiters\n                // Remove extra characters. *a*** -> *a*\n                rLength = Math.min(rLength, rLength + delimTotal + midDelimTotal);\n                // char length can be >1 for unicode characters;\n                const lastCharLength = [...match[0]][0].length;\n                const raw = src.slice(0, lLength + match.index + lastCharLength + rLength);\n                // Create `em` if smallest delimiter has odd char count. *a***\n                if (Math.min(lLength, rLength) % 2) {\n                    const text = raw.slice(1, -1);\n                    return {\n                        type: 'em',\n                        raw,\n                        text,\n                        tokens: this.lexer.inlineTokens(text)\n                    };\n                }\n                // Create 'strong' if smallest delimiter has even char count. **a***\n                const text = raw.slice(2, -2);\n                return {\n                    type: 'strong',\n                    raw,\n                    text,\n                    tokens: this.lexer.inlineTokens(text)\n                };\n            }\n        }\n    }\n    codespan(src) {\n        const cap = this.rules.inline.code.exec(src);\n        if (cap) {\n            let text = cap[2].replace(/\\n/g, ' ');\n            const hasNonSpaceChars = /[^ ]/.test(text);\n            const hasSpaceCharsOnBothEnds = /^ /.test(text) && / $/.test(text);\n            if (hasNonSpaceChars && hasSpaceCharsOnBothEnds) {\n                text = text.substring(1, text.length - 1);\n            }\n            text = escape(text, true);\n            return {\n                type: 'codespan',\n                raw: cap[0],\n                text\n            };\n        }\n    }\n    br(src) {\n        const cap = this.rules.inline.br.exec(src);\n        if (cap) {\n            return {\n                type: 'br',\n                raw: cap[0]\n            };\n        }\n    }\n    del(src) {\n        const cap = this.rules.inline.del.exec(src);\n        if (cap) {\n            return {\n                type: 'del',\n                raw: cap[0],\n                text: cap[2],\n                tokens: this.lexer.inlineTokens(cap[2])\n            };\n        }\n    }\n    autolink(src) {\n        const cap = this.rules.inline.autolink.exec(src);\n        if (cap) {\n            let text, href;\n            if (cap[2] === '@') {\n                text = escape(cap[1]);\n                href = 'mailto:' + text;\n            }\n            else {\n                text = escape(cap[1]);\n                href = text;\n            }\n            return {\n                type: 'link',\n                raw: cap[0],\n                text,\n                href,\n                tokens: [\n                    {\n                        type: 'text',\n                        raw: text,\n                        text\n                    }\n                ]\n            };\n        }\n    }\n    url(src) {\n        let cap;\n        if (cap = this.rules.inline.url.exec(src)) {\n            let text, href;\n            if (cap[2] === '@') {\n                text = escape(cap[0]);\n                href = 'mailto:' + text;\n            }\n            else {\n                // do extended autolink path validation\n                let prevCapZero;\n                do {\n                    prevCapZero = cap[0];\n                    cap[0] = this.rules.inline._backpedal.exec(cap[0])[0];\n                } while (prevCapZero !== cap[0]);\n                text = escape(cap[0]);\n                if (cap[1] === 'www.') {\n                    href = 'http://' + cap[0];\n                }\n                else {\n                    href = cap[0];\n                }\n            }\n            return {\n                type: 'link',\n                raw: cap[0],\n                text,\n                href,\n                tokens: [\n                    {\n                        type: 'text',\n                        raw: text,\n                        text\n                    }\n                ]\n            };\n        }\n    }\n    inlineText(src) {\n        const cap = this.rules.inline.text.exec(src);\n        if (cap) {\n            let text;\n            if (this.lexer.state.inRawBlock) {\n                text = cap[0];\n            }\n            else {\n                text = escape(cap[0]);\n            }\n            return {\n                type: 'text',\n                raw: cap[0],\n                text\n            };\n        }\n    }\n}\n\n/**\n * Block-Level Grammar\n */\n// Not all rules are defined in the object literal\n// @ts-expect-error\nconst block = {\n    newline: /^(?: *(?:\\n|$))+/,\n    code: /^( {4}[^\\n]+(?:\\n(?: *(?:\\n|$))*)?)+/,\n    fences: /^ {0,3}(`{3,}(?=[^`\\n]*(?:\\n|$))|~{3,})([^\\n]*)(?:\\n|$)(?:|([\\s\\S]*?)(?:\\n|$))(?: {0,3}\\1[~`]* *(?=\\n|$)|$)/,\n    hr: /^ {0,3}((?:-[\\t ]*){3,}|(?:_[ \\t]*){3,}|(?:\\*[ \\t]*){3,})(?:\\n+|$)/,\n    heading: /^ {0,3}(#{1,6})(?=\\s|$)(.*)(?:\\n+|$)/,\n    blockquote: /^( {0,3}> ?(paragraph|[^\\n]*)(?:\\n|$))+/,\n    list: /^( {0,3}bull)([ \\t][^\\n]+?)?(?:\\n|$)/,\n    html: '^ {0,3}(?:' // optional indentation\n        + '<(script|pre|style|textarea)[\\\\s>][\\\\s\\\\S]*?(?:</\\\\1>[^\\\\n]*\\\\n+|$)' // (1)\n        + '|comment[^\\\\n]*(\\\\n+|$)' // (2)\n        + '|<\\\\?[\\\\s\\\\S]*?(?:\\\\?>\\\\n*|$)' // (3)\n        + '|<![A-Z][\\\\s\\\\S]*?(?:>\\\\n*|$)' // (4)\n        + '|<!\\\\[CDATA\\\\[[\\\\s\\\\S]*?(?:\\\\]\\\\]>\\\\n*|$)' // (5)\n        + '|</?(tag)(?: +|\\\\n|/?>)[\\\\s\\\\S]*?(?:(?:\\\\n *)+\\\\n|$)' // (6)\n        + '|<(?!script|pre|style|textarea)([a-z][\\\\w-]*)(?:attribute)*? */?>(?=[ \\\\t]*(?:\\\\n|$))[\\\\s\\\\S]*?(?:(?:\\\\n *)+\\\\n|$)' // (7) open tag\n        + '|</(?!script|pre|style|textarea)[a-z][\\\\w-]*\\\\s*>(?=[ \\\\t]*(?:\\\\n|$))[\\\\s\\\\S]*?(?:(?:\\\\n *)+\\\\n|$)' // (7) closing tag\n        + ')',\n    def: /^ {0,3}\\[(label)\\]: *(?:\\n *)?([^<\\s][^\\s]*|<.*?>)(?:(?: +(?:\\n *)?| *\\n *)(title))? *(?:\\n+|$)/,\n    table: noopTest,\n    lheading: /^(?!bull )((?:.|\\n(?!\\s*?\\n|bull ))+?)\\n {0,3}(=+|-+) *(?:\\n+|$)/,\n    // regex template, placeholders will be replaced according to different paragraph\n    // interruption rules of commonmark and the original markdown spec:\n    _paragraph: /^([^\\n]+(?:\\n(?!hr|heading|lheading|blockquote|fences|list|html|table| +\\n)[^\\n]+)*)/,\n    text: /^[^\\n]+/\n};\nblock._label = /(?!\\s*\\])(?:\\\\.|[^\\[\\]\\\\])+/;\nblock._title = /(?:\"(?:\\\\\"?|[^\"\\\\])*\"|'[^'\\n]*(?:\\n[^'\\n]+)*\\n?'|\\([^()]*\\))/;\nblock.def = edit(block.def)\n    .replace('label', block._label)\n    .replace('title', block._title)\n    .getRegex();\nblock.bullet = /(?:[*+-]|\\d{1,9}[.)])/;\nblock.listItemStart = edit(/^( *)(bull) */)\n    .replace('bull', block.bullet)\n    .getRegex();\nblock.list = edit(block.list)\n    .replace(/bull/g, block.bullet)\n    .replace('hr', '\\\\n+(?=\\\\1?(?:(?:- *){3,}|(?:_ *){3,}|(?:\\\\* *){3,})(?:\\\\n+|$))')\n    .replace('def', '\\\\n+(?=' + block.def.source + ')')\n    .getRegex();\nblock._tag = 'address|article|aside|base|basefont|blockquote|body|caption'\n    + '|center|col|colgroup|dd|details|dialog|dir|div|dl|dt|fieldset|figcaption'\n    + '|figure|footer|form|frame|frameset|h[1-6]|head|header|hr|html|iframe'\n    + '|legend|li|link|main|menu|menuitem|meta|nav|noframes|ol|optgroup|option'\n    + '|p|param|section|source|summary|table|tbody|td|tfoot|th|thead|title|tr'\n    + '|track|ul';\nblock._comment = /<!--(?!-?>)[\\s\\S]*?(?:-->|$)/;\nblock.html = edit(block.html, 'i')\n    .replace('comment', block._comment)\n    .replace('tag', block._tag)\n    .replace('attribute', / +[a-zA-Z:_][\\w.:-]*(?: *= *\"[^\"\\n]*\"| *= *'[^'\\n]*'| *= *[^\\s\"'=<>`]+)?/)\n    .getRegex();\nblock.lheading = edit(block.lheading)\n    .replace(/bull/g, block.bullet) // lists can interrupt\n    .getRegex();\nblock.paragraph = edit(block._paragraph)\n    .replace('hr', block.hr)\n    .replace('heading', ' {0,3}#{1,6}(?:\\\\s|$)')\n    .replace('|lheading', '') // setex headings don't interrupt commonmark paragraphs\n    .replace('|table', '')\n    .replace('blockquote', ' {0,3}>')\n    .replace('fences', ' {0,3}(?:`{3,}(?=[^`\\\\n]*\\\\n)|~{3,})[^\\\\n]*\\\\n')\n    .replace('list', ' {0,3}(?:[*+-]|1[.)]) ') // only lists starting from 1 can interrupt\n    .replace('html', '</?(?:tag)(?: +|\\\\n|/?>)|<(?:script|pre|style|textarea|!--)')\n    .replace('tag', block._tag) // pars can be interrupted by type (6) html blocks\n    .getRegex();\nblock.blockquote = edit(block.blockquote)\n    .replace('paragraph', block.paragraph)\n    .getRegex();\n/**\n * Normal Block Grammar\n */\nblock.normal = { ...block };\n/**\n * GFM Block Grammar\n */\nblock.gfm = {\n    ...block.normal,\n    table: '^ *([^\\\\n ].*)\\\\n' // Header\n        + ' {0,3}((?:\\\\| *)?:?-+:? *(?:\\\\| *:?-+:? *)*(?:\\\\| *)?)' // Align\n        + '(?:\\\\n((?:(?! *\\\\n|hr|heading|blockquote|code|fences|list|html).*(?:\\\\n|$))*)\\\\n*|$)' // Cells\n};\nblock.gfm.table = edit(block.gfm.table)\n    .replace('hr', block.hr)\n    .replace('heading', ' {0,3}#{1,6}(?:\\\\s|$)')\n    .replace('blockquote', ' {0,3}>')\n    .replace('code', ' {4}[^\\\\n]')\n    .replace('fences', ' {0,3}(?:`{3,}(?=[^`\\\\n]*\\\\n)|~{3,})[^\\\\n]*\\\\n')\n    .replace('list', ' {0,3}(?:[*+-]|1[.)]) ') // only lists starting from 1 can interrupt\n    .replace('html', '</?(?:tag)(?: +|\\\\n|/?>)|<(?:script|pre|style|textarea|!--)')\n    .replace('tag', block._tag) // tables can be interrupted by type (6) html blocks\n    .getRegex();\nblock.gfm.paragraph = edit(block._paragraph)\n    .replace('hr', block.hr)\n    .replace('heading', ' {0,3}#{1,6}(?:\\\\s|$)')\n    .replace('|lheading', '') // setex headings don't interrupt commonmark paragraphs\n    .replace('table', block.gfm.table) // interrupt paragraphs with table\n    .replace('blockquote', ' {0,3}>')\n    .replace('fences', ' {0,3}(?:`{3,}(?=[^`\\\\n]*\\\\n)|~{3,})[^\\\\n]*\\\\n')\n    .replace('list', ' {0,3}(?:[*+-]|1[.)]) ') // only lists starting from 1 can interrupt\n    .replace('html', '</?(?:tag)(?: +|\\\\n|/?>)|<(?:script|pre|style|textarea|!--)')\n    .replace('tag', block._tag) // pars can be interrupted by type (6) html blocks\n    .getRegex();\n/**\n * Pedantic grammar (original John Gruber's loose markdown specification)\n */\nblock.pedantic = {\n    ...block.normal,\n    html: edit('^ *(?:comment *(?:\\\\n|\\\\s*$)'\n        + '|<(tag)[\\\\s\\\\S]+?</\\\\1> *(?:\\\\n{2,}|\\\\s*$)' // closed tag\n        + '|<tag(?:\"[^\"]*\"|\\'[^\\']*\\'|\\\\s[^\\'\"/>\\\\s]*)*?/?> *(?:\\\\n{2,}|\\\\s*$))')\n        .replace('comment', block._comment)\n        .replace(/tag/g, '(?!(?:'\n        + 'a|em|strong|small|s|cite|q|dfn|abbr|data|time|code|var|samp|kbd|sub'\n        + '|sup|i|b|u|mark|ruby|rt|rp|bdi|bdo|span|br|wbr|ins|del|img)'\n        + '\\\\b)\\\\w+(?!:|[^\\\\w\\\\s@]*@)\\\\b')\n        .getRegex(),\n    def: /^ *\\[([^\\]]+)\\]: *<?([^\\s>]+)>?(?: +([\"(][^\\n]+[\")]))? *(?:\\n+|$)/,\n    heading: /^(#{1,6})(.*)(?:\\n+|$)/,\n    fences: noopTest,\n    lheading: /^(.+?)\\n {0,3}(=+|-+) *(?:\\n+|$)/,\n    paragraph: edit(block.normal._paragraph)\n        .replace('hr', block.hr)\n        .replace('heading', ' *#{1,6} *[^\\n]')\n        .replace('lheading', block.lheading)\n        .replace('blockquote', ' {0,3}>')\n        .replace('|fences', '')\n        .replace('|list', '')\n        .replace('|html', '')\n        .getRegex()\n};\n/**\n * Inline-Level Grammar\n */\n// Not all rules are defined in the object literal\n// @ts-expect-error\nconst inline = {\n    escape: /^\\\\([!\"#$%&'()*+,\\-./:;<=>?@\\[\\]\\\\^_`{|}~])/,\n    autolink: /^<(scheme:[^\\s\\x00-\\x1f<>]*|email)>/,\n    url: noopTest,\n    tag: '^comment'\n        + '|^</[a-zA-Z][\\\\w:-]*\\\\s*>' // self-closing tag\n        + '|^<[a-zA-Z][\\\\w-]*(?:attribute)*?\\\\s*/?>' // open tag\n        + '|^<\\\\?[\\\\s\\\\S]*?\\\\?>' // processing instruction, e.g. <?php ?>\n        + '|^<![a-zA-Z]+\\\\s[\\\\s\\\\S]*?>' // declaration, e.g. <!DOCTYPE html>\n        + '|^<!\\\\[CDATA\\\\[[\\\\s\\\\S]*?\\\\]\\\\]>',\n    link: /^!?\\[(label)\\]\\(\\s*(href)(?:\\s+(title))?\\s*\\)/,\n    reflink: /^!?\\[(label)\\]\\[(ref)\\]/,\n    nolink: /^!?\\[(ref)\\](?:\\[\\])?/,\n    reflinkSearch: 'reflink|nolink(?!\\\\()',\n    emStrong: {\n        lDelim: /^(?:\\*+(?:((?!\\*)[punct])|[^\\s*]))|^_+(?:((?!_)[punct])|([^\\s_]))/,\n        //         (1) and (2) can only be a Right Delimiter. (3) and (4) can only be Left.  (5) and (6) can be either Left or Right.\n        //         | Skip orphan inside strong      | Consume to delim | (1) #***              | (2) a***#, a***                    | (3) #***a, ***a                  | (4) ***#                 | (5) #***#                         | (6) a***a\n        rDelimAst: /^[^_*]*?__[^_*]*?\\*[^_*]*?(?=__)|[^*]+(?=[^*])|(?!\\*)[punct](\\*+)(?=[\\s]|$)|[^punct\\s](\\*+)(?!\\*)(?=[punct\\s]|$)|(?!\\*)[punct\\s](\\*+)(?=[^punct\\s])|[\\s](\\*+)(?!\\*)(?=[punct])|(?!\\*)[punct](\\*+)(?!\\*)(?=[punct])|[^punct\\s](\\*+)(?=[^punct\\s])/,\n        rDelimUnd: /^[^_*]*?\\*\\*[^_*]*?_[^_*]*?(?=\\*\\*)|[^_]+(?=[^_])|(?!_)[punct](_+)(?=[\\s]|$)|[^punct\\s](_+)(?!_)(?=[punct\\s]|$)|(?!_)[punct\\s](_+)(?=[^punct\\s])|[\\s](_+)(?!_)(?=[punct])|(?!_)[punct](_+)(?!_)(?=[punct])/ // ^- Not allowed for _\n    },\n    code: /^(`+)([^`]|[^`][\\s\\S]*?[^`])\\1(?!`)/,\n    br: /^( {2,}|\\\\)\\n(?!\\s*$)/,\n    del: noopTest,\n    text: /^(`+|[^`])(?:(?= {2,}\\n)|[\\s\\S]*?(?:(?=[\\\\<!\\[`*_]|\\b_|$)|[^ ](?= {2,}\\n)))/,\n    punctuation: /^((?![*_])[\\spunctuation])/\n};\n// list of unicode punctuation marks, plus any missing characters from CommonMark spec\ninline._punctuation = '\\\\p{P}$+<=>`^|~';\ninline.punctuation = edit(inline.punctuation, 'u').replace(/punctuation/g, inline._punctuation).getRegex();\n// sequences em should skip over [title](link), `code`, <html>\ninline.blockSkip = /\\[[^[\\]]*?\\]\\([^\\(\\)]*?\\)|`[^`]*?`|<[^<>]*?>/g;\ninline.anyPunctuation = /\\\\[punct]/g;\ninline._escapes = /\\\\([punct])/g;\ninline._comment = edit(block._comment).replace('(?:-->|$)', '-->').getRegex();\ninline.emStrong.lDelim = edit(inline.emStrong.lDelim, 'u')\n    .replace(/punct/g, inline._punctuation)\n    .getRegex();\ninline.emStrong.rDelimAst = edit(inline.emStrong.rDelimAst, 'gu')\n    .replace(/punct/g, inline._punctuation)\n    .getRegex();\ninline.emStrong.rDelimUnd = edit(inline.emStrong.rDelimUnd, 'gu')\n    .replace(/punct/g, inline._punctuation)\n    .getRegex();\ninline.anyPunctuation = edit(inline.anyPunctuation, 'gu')\n    .replace(/punct/g, inline._punctuation)\n    .getRegex();\ninline._escapes = edit(inline._escapes, 'gu')\n    .replace(/punct/g, inline._punctuation)\n    .getRegex();\ninline._scheme = /[a-zA-Z][a-zA-Z0-9+.-]{1,31}/;\ninline._email = /[a-zA-Z0-9.!#$%&'*+/=?^_`{|}~-]+(@)[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)+(?![-_])/;\ninline.autolink = edit(inline.autolink)\n    .replace('scheme', inline._scheme)\n    .replace('email', inline._email)\n    .getRegex();\ninline._attribute = /\\s+[a-zA-Z:_][\\w.:-]*(?:\\s*=\\s*\"[^\"]*\"|\\s*=\\s*'[^']*'|\\s*=\\s*[^\\s\"'=<>`]+)?/;\ninline.tag = edit(inline.tag)\n    .replace('comment', inline._comment)\n    .replace('attribute', inline._attribute)\n    .getRegex();\ninline._label = /(?:\\[(?:\\\\.|[^\\[\\]\\\\])*\\]|\\\\.|`[^`]*`|[^\\[\\]\\\\`])*?/;\ninline._href = /<(?:\\\\.|[^\\n<>\\\\])+>|[^\\s\\x00-\\x1f]*/;\ninline._title = /\"(?:\\\\\"?|[^\"\\\\])*\"|'(?:\\\\'?|[^'\\\\])*'|\\((?:\\\\\\)?|[^)\\\\])*\\)/;\ninline.link = edit(inline.link)\n    .replace('label', inline._label)\n    .replace('href', inline._href)\n    .replace('title', inline._title)\n    .getRegex();\ninline.reflink = edit(inline.reflink)\n    .replace('label', inline._label)\n    .replace('ref', block._label)\n    .getRegex();\ninline.nolink = edit(inline.nolink)\n    .replace('ref', block._label)\n    .getRegex();\ninline.reflinkSearch = edit(inline.reflinkSearch, 'g')\n    .replace('reflink', inline.reflink)\n    .replace('nolink', inline.nolink)\n    .getRegex();\n/**\n * Normal Inline Grammar\n */\ninline.normal = { ...inline };\n/**\n * Pedantic Inline Grammar\n */\ninline.pedantic = {\n    ...inline.normal,\n    strong: {\n        start: /^__|\\*\\*/,\n        middle: /^__(?=\\S)([\\s\\S]*?\\S)__(?!_)|^\\*\\*(?=\\S)([\\s\\S]*?\\S)\\*\\*(?!\\*)/,\n        endAst: /\\*\\*(?!\\*)/g,\n        endUnd: /__(?!_)/g\n    },\n    em: {\n        start: /^_|\\*/,\n        middle: /^()\\*(?=\\S)([\\s\\S]*?\\S)\\*(?!\\*)|^_(?=\\S)([\\s\\S]*?\\S)_(?!_)/,\n        endAst: /\\*(?!\\*)/g,\n        endUnd: /_(?!_)/g\n    },\n    link: edit(/^!?\\[(label)\\]\\((.*?)\\)/)\n        .replace('label', inline._label)\n        .getRegex(),\n    reflink: edit(/^!?\\[(label)\\]\\s*\\[([^\\]]*)\\]/)\n        .replace('label', inline._label)\n        .getRegex()\n};\n/**\n * GFM Inline Grammar\n */\ninline.gfm = {\n    ...inline.normal,\n    escape: edit(inline.escape).replace('])', '~|])').getRegex(),\n    _extended_email: /[A-Za-z0-9._+-]+(@)[a-zA-Z0-9-_]+(?:\\.[a-zA-Z0-9-_]*[a-zA-Z0-9])+(?![-_])/,\n    url: /^((?:ftp|https?):\\/\\/|www\\.)(?:[a-zA-Z0-9\\-]+\\.?)+[^\\s<]*|^email/,\n    _backpedal: /(?:[^?!.,:;*_'\"~()&]+|\\([^)]*\\)|&(?![a-zA-Z0-9]+;$)|[?!.,:;*_'\"~)]+(?!$))+/,\n    del: /^(~~?)(?=[^\\s~])([\\s\\S]*?[^\\s~])\\1(?=[^~]|$)/,\n    text: /^([`~]+|[^`~])(?:(?= {2,}\\n)|(?=[a-zA-Z0-9.!#$%&'*+\\/=?_`{\\|}~-]+@)|[\\s\\S]*?(?:(?=[\\\\<!\\[`*~_]|\\b_|https?:\\/\\/|ftp:\\/\\/|www\\.|$)|[^ ](?= {2,}\\n)|[^a-zA-Z0-9.!#$%&'*+\\/=?_`{\\|}~-](?=[a-zA-Z0-9.!#$%&'*+\\/=?_`{\\|}~-]+@)))/\n};\ninline.gfm.url = edit(inline.gfm.url, 'i')\n    .replace('email', inline.gfm._extended_email)\n    .getRegex();\n/**\n * GFM + Line Breaks Inline Grammar\n */\ninline.breaks = {\n    ...inline.gfm,\n    br: edit(inline.br).replace('{2,}', '*').getRegex(),\n    text: edit(inline.gfm.text)\n        .replace('\\\\b_', '\\\\b_| {2,}\\\\n')\n        .replace(/\\{2,\\}/g, '*')\n        .getRegex()\n};\n\n/**\n * Block Lexer\n */\nclass _Lexer {\n    tokens;\n    options;\n    state;\n    tokenizer;\n    inlineQueue;\n    constructor(options) {\n        // TokenList cannot be created in one go\n        // @ts-expect-error\n        this.tokens = [];\n        this.tokens.links = Object.create(null);\n        this.options = options || _defaults;\n        this.options.tokenizer = this.options.tokenizer || new _Tokenizer();\n        this.tokenizer = this.options.tokenizer;\n        this.tokenizer.options = this.options;\n        this.tokenizer.lexer = this;\n        this.inlineQueue = [];\n        this.state = {\n            inLink: false,\n            inRawBlock: false,\n            top: true\n        };\n        const rules = {\n            block: block.normal,\n            inline: inline.normal\n        };\n        if (this.options.pedantic) {\n            rules.block = block.pedantic;\n            rules.inline = inline.pedantic;\n        }\n        else if (this.options.gfm) {\n            rules.block = block.gfm;\n            if (this.options.breaks) {\n                rules.inline = inline.breaks;\n            }\n            else {\n                rules.inline = inline.gfm;\n            }\n        }\n        this.tokenizer.rules = rules;\n    }\n    /**\n     * Expose Rules\n     */\n    static get rules() {\n        return {\n            block,\n            inline\n        };\n    }\n    /**\n     * Static Lex Method\n     */\n    static lex(src, options) {\n        const lexer = new _Lexer(options);\n        return lexer.lex(src);\n    }\n    /**\n     * Static Lex Inline Method\n     */\n    static lexInline(src, options) {\n        const lexer = new _Lexer(options);\n        return lexer.inlineTokens(src);\n    }\n    /**\n     * Preprocessing\n     */\n    lex(src) {\n        src = src\n            .replace(/\\r\\n|\\r/g, '\\n');\n        this.blockTokens(src, this.tokens);\n        let next;\n        while (next = this.inlineQueue.shift()) {\n            this.inlineTokens(next.src, next.tokens);\n        }\n        return this.tokens;\n    }\n    blockTokens(src, tokens = []) {\n        if (this.options.pedantic) {\n            src = src.replace(/\\t/g, '    ').replace(/^ +$/gm, '');\n        }\n        else {\n            src = src.replace(/^( *)(\\t+)/gm, (_, leading, tabs) => {\n                return leading + '    '.repeat(tabs.length);\n            });\n        }\n        let token;\n        let lastToken;\n        let cutSrc;\n        let lastParagraphClipped;\n        while (src) {\n            if (this.options.extensions\n                && this.options.extensions.block\n                && this.options.extensions.block.some((extTokenizer) => {\n                    if (token = extTokenizer.call({ lexer: this }, src, tokens)) {\n                        src = src.substring(token.raw.length);\n                        tokens.push(token);\n                        return true;\n                    }\n                    return false;\n                })) {\n                continue;\n            }\n            // newline\n            if (token = this.tokenizer.space(src)) {\n                src = src.substring(token.raw.length);\n                if (token.raw.length === 1 && tokens.length > 0) {\n                    // if there's a single \\n as a spacer, it's terminating the last line,\n                    // so move it there so that we don't get unnecessary paragraph tags\n                    tokens[tokens.length - 1].raw += '\\n';\n                }\n                else {\n                    tokens.push(token);\n                }\n                continue;\n            }\n            // code\n            if (token = this.tokenizer.code(src)) {\n                src = src.substring(token.raw.length);\n                lastToken = tokens[tokens.length - 1];\n                // An indented code block cannot interrupt a paragraph.\n                if (lastToken && (lastToken.type === 'paragraph' || lastToken.type === 'text')) {\n                    lastToken.raw += '\\n' + token.raw;\n                    lastToken.text += '\\n' + token.text;\n                    this.inlineQueue[this.inlineQueue.length - 1].src = lastToken.text;\n                }\n                else {\n                    tokens.push(token);\n                }\n                continue;\n            }\n            // fences\n            if (token = this.tokenizer.fences(src)) {\n                src = src.substring(token.raw.length);\n                tokens.push(token);\n                continue;\n            }\n            // heading\n            if (token = this.tokenizer.heading(src)) {\n                src = src.substring(token.raw.length);\n                tokens.push(token);\n                continue;\n            }\n            // hr\n            if (token = this.tokenizer.hr(src)) {\n                src = src.substring(token.raw.length);\n                tokens.push(token);\n                continue;\n            }\n            // blockquote\n            if (token = this.tokenizer.blockquote(src)) {\n                src = src.substring(token.raw.length);\n                tokens.push(token);\n                continue;\n            }\n            // list\n            if (token = this.tokenizer.list(src)) {\n                src = src.substring(token.raw.length);\n                tokens.push(token);\n                continue;\n            }\n            // html\n            if (token = this.tokenizer.html(src)) {\n                src = src.substring(token.raw.length);\n                tokens.push(token);\n                continue;\n            }\n            // def\n            if (token = this.tokenizer.def(src)) {\n                src = src.substring(token.raw.length);\n                lastToken = tokens[tokens.length - 1];\n                if (lastToken && (lastToken.type === 'paragraph' || lastToken.type === 'text')) {\n                    lastToken.raw += '\\n' + token.raw;\n                    lastToken.text += '\\n' + token.raw;\n                    this.inlineQueue[this.inlineQueue.length - 1].src = lastToken.text;\n                }\n                else if (!this.tokens.links[token.tag]) {\n                    this.tokens.links[token.tag] = {\n                        href: token.href,\n                        title: token.title\n                    };\n                }\n                continue;\n            }\n            // table (gfm)\n            if (token = this.tokenizer.table(src)) {\n                src = src.substring(token.raw.length);\n                tokens.push(token);\n                continue;\n            }\n            // lheading\n            if (token = this.tokenizer.lheading(src)) {\n                src = src.substring(token.raw.length);\n                tokens.push(token);\n                continue;\n            }\n            // top-level paragraph\n            // prevent paragraph consuming extensions by clipping 'src' to extension start\n            cutSrc = src;\n            if (this.options.extensions && this.options.extensions.startBlock) {\n                let startIndex = Infinity;\n                const tempSrc = src.slice(1);\n                let tempStart;\n                this.options.extensions.startBlock.forEach((getStartIndex) => {\n                    tempStart = getStartIndex.call({ lexer: this }, tempSrc);\n                    if (typeof tempStart === 'number' && tempStart >= 0) {\n                        startIndex = Math.min(startIndex, tempStart);\n                    }\n                });\n                if (startIndex < Infinity && startIndex >= 0) {\n                    cutSrc = src.substring(0, startIndex + 1);\n                }\n            }\n            if (this.state.top && (token = this.tokenizer.paragraph(cutSrc))) {\n                lastToken = tokens[tokens.length - 1];\n                if (lastParagraphClipped && lastToken.type === 'paragraph') {\n                    lastToken.raw += '\\n' + token.raw;\n                    lastToken.text += '\\n' + token.text;\n                    this.inlineQueue.pop();\n                    this.inlineQueue[this.inlineQueue.length - 1].src = lastToken.text;\n                }\n                else {\n                    tokens.push(token);\n                }\n                lastParagraphClipped = (cutSrc.length !== src.length);\n                src = src.substring(token.raw.length);\n                continue;\n            }\n            // text\n            if (token = this.tokenizer.text(src)) {\n                src = src.substring(token.raw.length);\n                lastToken = tokens[tokens.length - 1];\n                if (lastToken && lastToken.type === 'text') {\n                    lastToken.raw += '\\n' + token.raw;\n                    lastToken.text += '\\n' + token.text;\n                    this.inlineQueue.pop();\n                    this.inlineQueue[this.inlineQueue.length - 1].src = lastToken.text;\n                }\n                else {\n                    tokens.push(token);\n                }\n                continue;\n            }\n            if (src) {\n                const errMsg = 'Infinite loop on byte: ' + src.charCodeAt(0);\n                if (this.options.silent) {\n                    console.error(errMsg);\n                    break;\n                }\n                else {\n                    throw new Error(errMsg);\n                }\n            }\n        }\n        this.state.top = true;\n        return tokens;\n    }\n    inline(src, tokens = []) {\n        this.inlineQueue.push({ src, tokens });\n        return tokens;\n    }\n    /**\n     * Lexing/Compiling\n     */\n    inlineTokens(src, tokens = []) {\n        let token, lastToken, cutSrc;\n        // String with links masked to avoid interference with em and strong\n        let maskedSrc = src;\n        let match;\n        let keepPrevChar, prevChar;\n        // Mask out reflinks\n        if (this.tokens.links) {\n            const links = Object.keys(this.tokens.links);\n            if (links.length > 0) {\n                while ((match = this.tokenizer.rules.inline.reflinkSearch.exec(maskedSrc)) != null) {\n                    if (links.includes(match[0].slice(match[0].lastIndexOf('[') + 1, -1))) {\n                        maskedSrc = maskedSrc.slice(0, match.index) + '[' + 'a'.repeat(match[0].length - 2) + ']' + maskedSrc.slice(this.tokenizer.rules.inline.reflinkSearch.lastIndex);\n                    }\n                }\n            }\n        }\n        // Mask out other blocks\n        while ((match = this.tokenizer.rules.inline.blockSkip.exec(maskedSrc)) != null) {\n            maskedSrc = maskedSrc.slice(0, match.index) + '[' + 'a'.repeat(match[0].length - 2) + ']' + maskedSrc.slice(this.tokenizer.rules.inline.blockSkip.lastIndex);\n        }\n        // Mask out escaped characters\n        while ((match = this.tokenizer.rules.inline.anyPunctuation.exec(maskedSrc)) != null) {\n            maskedSrc = maskedSrc.slice(0, match.index) + '++' + maskedSrc.slice(this.tokenizer.rules.inline.anyPunctuation.lastIndex);\n        }\n        while (src) {\n            if (!keepPrevChar) {\n                prevChar = '';\n            }\n            keepPrevChar = false;\n            // extensions\n            if (this.options.extensions\n                && this.options.extensions.inline\n                && this.options.extensions.inline.some((extTokenizer) => {\n                    if (token = extTokenizer.call({ lexer: this }, src, tokens)) {\n                        src = src.substring(token.raw.length);\n                        tokens.push(token);\n                        return true;\n                    }\n                    return false;\n                })) {\n                continue;\n            }\n            // escape\n            if (token = this.tokenizer.escape(src)) {\n                src = src.substring(token.raw.length);\n                tokens.push(token);\n                continue;\n            }\n            // tag\n            if (token = this.tokenizer.tag(src)) {\n                src = src.substring(token.raw.length);\n                lastToken = tokens[tokens.length - 1];\n                if (lastToken && token.type === 'text' && lastToken.type === 'text') {\n                    lastToken.raw += token.raw;\n                    lastToken.text += token.text;\n                }\n                else {\n                    tokens.push(token);\n                }\n                continue;\n            }\n            // link\n            if (token = this.tokenizer.link(src)) {\n                src = src.substring(token.raw.length);\n                tokens.push(token);\n                continue;\n            }\n            // reflink, nolink\n            if (token = this.tokenizer.reflink(src, this.tokens.links)) {\n                src = src.substring(token.raw.length);\n                lastToken = tokens[tokens.length - 1];\n                if (lastToken && token.type === 'text' && lastToken.type === 'text') {\n                    lastToken.raw += token.raw;\n                    lastToken.text += token.text;\n                }\n                else {\n                    tokens.push(token);\n                }\n                continue;\n            }\n            // em & strong\n            if (token = this.tokenizer.emStrong(src, maskedSrc, prevChar)) {\n                src = src.substring(token.raw.length);\n                tokens.push(token);\n                continue;\n            }\n            // code\n            if (token = this.tokenizer.codespan(src)) {\n                src = src.substring(token.raw.length);\n                tokens.push(token);\n                continue;\n            }\n            // br\n            if (token = this.tokenizer.br(src)) {\n                src = src.substring(token.raw.length);\n                tokens.push(token);\n                continue;\n            }\n            // del (gfm)\n            if (token = this.tokenizer.del(src)) {\n                src = src.substring(token.raw.length);\n                tokens.push(token);\n                continue;\n            }\n            // autolink\n            if (token = this.tokenizer.autolink(src)) {\n                src = src.substring(token.raw.length);\n                tokens.push(token);\n                continue;\n            }\n            // url (gfm)\n            if (!this.state.inLink && (token = this.tokenizer.url(src))) {\n                src = src.substring(token.raw.length);\n                tokens.push(token);\n                continue;\n            }\n            // text\n            // prevent inlineText consuming extensions by clipping 'src' to extension start\n            cutSrc = src;\n            if (this.options.extensions && this.options.extensions.startInline) {\n                let startIndex = Infinity;\n                const tempSrc = src.slice(1);\n                let tempStart;\n                this.options.extensions.startInline.forEach((getStartIndex) => {\n                    tempStart = getStartIndex.call({ lexer: this }, tempSrc);\n                    if (typeof tempStart === 'number' && tempStart >= 0) {\n                        startIndex = Math.min(startIndex, tempStart);\n                    }\n                });\n                if (startIndex < Infinity && startIndex >= 0) {\n                    cutSrc = src.substring(0, startIndex + 1);\n                }\n            }\n            if (token = this.tokenizer.inlineText(cutSrc)) {\n                src = src.substring(token.raw.length);\n                if (token.raw.slice(-1) !== '_') { // Track prevChar before string of ____ started\n                    prevChar = token.raw.slice(-1);\n                }\n                keepPrevChar = true;\n                lastToken = tokens[tokens.length - 1];\n                if (lastToken && lastToken.type === 'text') {\n                    lastToken.raw += token.raw;\n                    lastToken.text += token.text;\n                }\n                else {\n                    tokens.push(token);\n                }\n                continue;\n            }\n            if (src) {\n                const errMsg = 'Infinite loop on byte: ' + src.charCodeAt(0);\n                if (this.options.silent) {\n                    console.error(errMsg);\n                    break;\n                }\n                else {\n                    throw new Error(errMsg);\n                }\n            }\n        }\n        return tokens;\n    }\n}\n\n/**\n * Renderer\n */\nclass _Renderer {\n    options;\n    constructor(options) {\n        this.options = options || _defaults;\n    }\n    code(code, infostring, escaped) {\n        const lang = (infostring || '').match(/^\\S*/)?.[0];\n        code = code.replace(/\\n$/, '') + '\\n';\n        if (!lang) {\n            return '<pre><code>'\n                + (escaped ? code : escape(code, true))\n                + '</code></pre>\\n';\n        }\n        return '<pre><code class=\"language-'\n            + escape(lang)\n            + '\">'\n            + (escaped ? code : escape(code, true))\n            + '</code></pre>\\n';\n    }\n    blockquote(quote) {\n        return `<blockquote>\\n${quote}</blockquote>\\n`;\n    }\n    html(html, block) {\n        return html;\n    }\n    heading(text, level, raw) {\n        // ignore IDs\n        return `<h${level}>${text}</h${level}>\\n`;\n    }\n    hr() {\n        return '<hr>\\n';\n    }\n    list(body, ordered, start) {\n        const type = ordered ? 'ol' : 'ul';\n        const startatt = (ordered && start !== 1) ? (' start=\"' + start + '\"') : '';\n        return '<' + type + startatt + '>\\n' + body + '</' + type + '>\\n';\n    }\n    listitem(text, task, checked) {\n        return `<li>${text}</li>\\n`;\n    }\n    checkbox(checked) {\n        return '<input '\n            + (checked ? 'checked=\"\" ' : '')\n            + 'disabled=\"\" type=\"checkbox\">';\n    }\n    paragraph(text) {\n        return `<p>${text}</p>\\n`;\n    }\n    table(header, body) {\n        if (body)\n            body = `<tbody>${body}</tbody>`;\n        return '<table>\\n'\n            + '<thead>\\n'\n            + header\n            + '</thead>\\n'\n            + body\n            + '</table>\\n';\n    }\n    tablerow(content) {\n        return `<tr>\\n${content}</tr>\\n`;\n    }\n    tablecell(content, flags) {\n        const type = flags.header ? 'th' : 'td';\n        const tag = flags.align\n            ? `<${type} align=\"${flags.align}\">`\n            : `<${type}>`;\n        return tag + content + `</${type}>\\n`;\n    }\n    /**\n     * span level renderer\n     */\n    strong(text) {\n        return `<strong>${text}</strong>`;\n    }\n    em(text) {\n        return `<em>${text}</em>`;\n    }\n    codespan(text) {\n        return `<code>${text}</code>`;\n    }\n    br() {\n        return '<br>';\n    }\n    del(text) {\n        return `<del>${text}</del>`;\n    }\n    link(href, title, text) {\n        const cleanHref = cleanUrl(href);\n        if (cleanHref === null) {\n            return text;\n        }\n        href = cleanHref;\n        let out = '<a href=\"' + href + '\"';\n        if (title) {\n            out += ' title=\"' + title + '\"';\n        }\n        out += '>' + text + '</a>';\n        return out;\n    }\n    image(href, title, text) {\n        const cleanHref = cleanUrl(href);\n        if (cleanHref === null) {\n            return text;\n        }\n        href = cleanHref;\n        let out = `<img src=\"${href}\" alt=\"${text}\"`;\n        if (title) {\n            out += ` title=\"${title}\"`;\n        }\n        out += '>';\n        return out;\n    }\n    text(text) {\n        return text;\n    }\n}\n\n/**\n * TextRenderer\n * returns only the textual part of the token\n */\nclass _TextRenderer {\n    // no need for block level renderers\n    strong(text) {\n        return text;\n    }\n    em(text) {\n        return text;\n    }\n    codespan(text) {\n        return text;\n    }\n    del(text) {\n        return text;\n    }\n    html(text) {\n        return text;\n    }\n    text(text) {\n        return text;\n    }\n    link(href, title, text) {\n        return '' + text;\n    }\n    image(href, title, text) {\n        return '' + text;\n    }\n    br() {\n        return '';\n    }\n}\n\n/**\n * Parsing & Compiling\n */\nclass _Parser {\n    options;\n    renderer;\n    textRenderer;\n    constructor(options) {\n        this.options = options || _defaults;\n        this.options.renderer = this.options.renderer || new _Renderer();\n        this.renderer = this.options.renderer;\n        this.renderer.options = this.options;\n        this.textRenderer = new _TextRenderer();\n    }\n    /**\n     * Static Parse Method\n     */\n    static parse(tokens, options) {\n        const parser = new _Parser(options);\n        return parser.parse(tokens);\n    }\n    /**\n     * Static Parse Inline Method\n     */\n    static parseInline(tokens, options) {\n        const parser = new _Parser(options);\n        return parser.parseInline(tokens);\n    }\n    /**\n     * Parse Loop\n     */\n    parse(tokens, top = true) {\n        let out = '';\n        for (let i = 0; i < tokens.length; i++) {\n            const token = tokens[i];\n            // Run any renderer extensions\n            if (this.options.extensions && this.options.extensions.renderers && this.options.extensions.renderers[token.type]) {\n                const genericToken = token;\n                const ret = this.options.extensions.renderers[genericToken.type].call({ parser: this }, genericToken);\n                if (ret !== false || !['space', 'hr', 'heading', 'code', 'table', 'blockquote', 'list', 'html', 'paragraph', 'text'].includes(genericToken.type)) {\n                    out += ret || '';\n                    continue;\n                }\n            }\n            switch (token.type) {\n                case 'space': {\n                    continue;\n                }\n                case 'hr': {\n                    out += this.renderer.hr();\n                    continue;\n                }\n                case 'heading': {\n                    const headingToken = token;\n                    out += this.renderer.heading(this.parseInline(headingToken.tokens), headingToken.depth, unescape(this.parseInline(headingToken.tokens, this.textRenderer)));\n                    continue;\n                }\n                case 'code': {\n                    const codeToken = token;\n                    out += this.renderer.code(codeToken.text, codeToken.lang, !!codeToken.escaped);\n                    continue;\n                }\n                case 'table': {\n                    const tableToken = token;\n                    let header = '';\n                    // header\n                    let cell = '';\n                    for (let j = 0; j < tableToken.header.length; j++) {\n                        cell += this.renderer.tablecell(this.parseInline(tableToken.header[j].tokens), { header: true, align: tableToken.align[j] });\n                    }\n                    header += this.renderer.tablerow(cell);\n                    let body = '';\n                    for (let j = 0; j < tableToken.rows.length; j++) {\n                        const row = tableToken.rows[j];\n                        cell = '';\n                        for (let k = 0; k < row.length; k++) {\n                            cell += this.renderer.tablecell(this.parseInline(row[k].tokens), { header: false, align: tableToken.align[k] });\n                        }\n                        body += this.renderer.tablerow(cell);\n                    }\n                    out += this.renderer.table(header, body);\n                    continue;\n                }\n                case 'blockquote': {\n                    const blockquoteToken = token;\n                    const body = this.parse(blockquoteToken.tokens);\n                    out += this.renderer.blockquote(body);\n                    continue;\n                }\n                case 'list': {\n                    const listToken = token;\n                    const ordered = listToken.ordered;\n                    const start = listToken.start;\n                    const loose = listToken.loose;\n                    let body = '';\n                    for (let j = 0; j < listToken.items.length; j++) {\n                        const item = listToken.items[j];\n                        const checked = item.checked;\n                        const task = item.task;\n                        let itemBody = '';\n                        if (item.task) {\n                            const checkbox = this.renderer.checkbox(!!checked);\n                            if (loose) {\n                                if (item.tokens.length > 0 && item.tokens[0].type === 'paragraph') {\n                                    item.tokens[0].text = checkbox + ' ' + item.tokens[0].text;\n                                    if (item.tokens[0].tokens && item.tokens[0].tokens.length > 0 && item.tokens[0].tokens[0].type === 'text') {\n                                        item.tokens[0].tokens[0].text = checkbox + ' ' + item.tokens[0].tokens[0].text;\n                                    }\n                                }\n                                else {\n                                    item.tokens.unshift({\n                                        type: 'text',\n                                        text: checkbox + ' '\n                                    });\n                                }\n                            }\n                            else {\n                                itemBody += checkbox + ' ';\n                            }\n                        }\n                        itemBody += this.parse(item.tokens, loose);\n                        body += this.renderer.listitem(itemBody, task, !!checked);\n                    }\n                    out += this.renderer.list(body, ordered, start);\n                    continue;\n                }\n                case 'html': {\n                    const htmlToken = token;\n                    out += this.renderer.html(htmlToken.text, htmlToken.block);\n                    continue;\n                }\n                case 'paragraph': {\n                    const paragraphToken = token;\n                    out += this.renderer.paragraph(this.parseInline(paragraphToken.tokens));\n                    continue;\n                }\n                case 'text': {\n                    let textToken = token;\n                    let body = textToken.tokens ? this.parseInline(textToken.tokens) : textToken.text;\n                    while (i + 1 < tokens.length && tokens[i + 1].type === 'text') {\n                        textToken = tokens[++i];\n                        body += '\\n' + (textToken.tokens ? this.parseInline(textToken.tokens) : textToken.text);\n                    }\n                    out += top ? this.renderer.paragraph(body) : body;\n                    continue;\n                }\n                default: {\n                    const errMsg = 'Token with \"' + token.type + '\" type was not found.';\n                    if (this.options.silent) {\n                        console.error(errMsg);\n                        return '';\n                    }\n                    else {\n                        throw new Error(errMsg);\n                    }\n                }\n            }\n        }\n        return out;\n    }\n    /**\n     * Parse Inline Tokens\n     */\n    parseInline(tokens, renderer) {\n        renderer = renderer || this.renderer;\n        let out = '';\n        for (let i = 0; i < tokens.length; i++) {\n            const token = tokens[i];\n            // Run any renderer extensions\n            if (this.options.extensions && this.options.extensions.renderers && this.options.extensions.renderers[token.type]) {\n                const ret = this.options.extensions.renderers[token.type].call({ parser: this }, token);\n                if (ret !== false || !['escape', 'html', 'link', 'image', 'strong', 'em', 'codespan', 'br', 'del', 'text'].includes(token.type)) {\n                    out += ret || '';\n                    continue;\n                }\n            }\n            switch (token.type) {\n                case 'escape': {\n                    const escapeToken = token;\n                    out += renderer.text(escapeToken.text);\n                    break;\n                }\n                case 'html': {\n                    const tagToken = token;\n                    out += renderer.html(tagToken.text);\n                    break;\n                }\n                case 'link': {\n                    const linkToken = token;\n                    out += renderer.link(linkToken.href, linkToken.title, this.parseInline(linkToken.tokens, renderer));\n                    break;\n                }\n                case 'image': {\n                    const imageToken = token;\n                    out += renderer.image(imageToken.href, imageToken.title, imageToken.text);\n                    break;\n                }\n                case 'strong': {\n                    const strongToken = token;\n                    out += renderer.strong(this.parseInline(strongToken.tokens, renderer));\n                    break;\n                }\n                case 'em': {\n                    const emToken = token;\n                    out += renderer.em(this.parseInline(emToken.tokens, renderer));\n                    break;\n                }\n                case 'codespan': {\n                    const codespanToken = token;\n                    out += renderer.codespan(codespanToken.text);\n                    break;\n                }\n                case 'br': {\n                    out += renderer.br();\n                    break;\n                }\n                case 'del': {\n                    const delToken = token;\n                    out += renderer.del(this.parseInline(delToken.tokens, renderer));\n                    break;\n                }\n                case 'text': {\n                    const textToken = token;\n                    out += renderer.text(textToken.text);\n                    break;\n                }\n                default: {\n                    const errMsg = 'Token with \"' + token.type + '\" type was not found.';\n                    if (this.options.silent) {\n                        console.error(errMsg);\n                        return '';\n                    }\n                    else {\n                        throw new Error(errMsg);\n                    }\n                }\n            }\n        }\n        return out;\n    }\n}\n\nclass _Hooks {\n    options;\n    constructor(options) {\n        this.options = options || _defaults;\n    }\n    static passThroughHooks = new Set([\n        'preprocess',\n        'postprocess'\n    ]);\n    /**\n     * Process markdown before marked\n     */\n    preprocess(markdown) {\n        return markdown;\n    }\n    /**\n     * Process HTML after marked is finished\n     */\n    postprocess(html) {\n        return html;\n    }\n}\n\nclass Marked {\n    defaults = _getDefaults();\n    options = this.setOptions;\n    parse = this.#parseMarkdown(_Lexer.lex, _Parser.parse);\n    parseInline = this.#parseMarkdown(_Lexer.lexInline, _Parser.parseInline);\n    Parser = _Parser;\n    Renderer = _Renderer;\n    TextRenderer = _TextRenderer;\n    Lexer = _Lexer;\n    Tokenizer = _Tokenizer;\n    Hooks = _Hooks;\n    constructor(...args) {\n        this.use(...args);\n    }\n    /**\n     * Run callback for every token\n     */\n    walkTokens(tokens, callback) {\n        let values = [];\n        for (const token of tokens) {\n            values = values.concat(callback.call(this, token));\n            switch (token.type) {\n                case 'table': {\n                    const tableToken = token;\n                    for (const cell of tableToken.header) {\n                        values = values.concat(this.walkTokens(cell.tokens, callback));\n                    }\n                    for (const row of tableToken.rows) {\n                        for (const cell of row) {\n                            values = values.concat(this.walkTokens(cell.tokens, callback));\n                        }\n                    }\n                    break;\n                }\n                case 'list': {\n                    const listToken = token;\n                    values = values.concat(this.walkTokens(listToken.items, callback));\n                    break;\n                }\n                default: {\n                    const genericToken = token;\n                    if (this.defaults.extensions?.childTokens?.[genericToken.type]) {\n                        this.defaults.extensions.childTokens[genericToken.type].forEach((childTokens) => {\n                            values = values.concat(this.walkTokens(genericToken[childTokens], callback));\n                        });\n                    }\n                    else if (genericToken.tokens) {\n                        values = values.concat(this.walkTokens(genericToken.tokens, callback));\n                    }\n                }\n            }\n        }\n        return values;\n    }\n    use(...args) {\n        const extensions = this.defaults.extensions || { renderers: {}, childTokens: {} };\n        args.forEach((pack) => {\n            // copy options to new object\n            const opts = { ...pack };\n            // set async to true if it was set to true before\n            opts.async = this.defaults.async || opts.async || false;\n            // ==-- Parse \"addon\" extensions --== //\n            if (pack.extensions) {\n                pack.extensions.forEach((ext) => {\n                    if (!ext.name) {\n                        throw new Error('extension name required');\n                    }\n                    if ('renderer' in ext) { // Renderer extensions\n                        const prevRenderer = extensions.renderers[ext.name];\n                        if (prevRenderer) {\n                            // Replace extension with func to run new extension but fall back if false\n                            extensions.renderers[ext.name] = function (...args) {\n                                let ret = ext.renderer.apply(this, args);\n                                if (ret === false) {\n                                    ret = prevRenderer.apply(this, args);\n                                }\n                                return ret;\n                            };\n                        }\n                        else {\n                            extensions.renderers[ext.name] = ext.renderer;\n                        }\n                    }\n                    if ('tokenizer' in ext) { // Tokenizer Extensions\n                        if (!ext.level || (ext.level !== 'block' && ext.level !== 'inline')) {\n                            throw new Error(\"extension level must be 'block' or 'inline'\");\n                        }\n                        const extLevel = extensions[ext.level];\n                        if (extLevel) {\n                            extLevel.unshift(ext.tokenizer);\n                        }\n                        else {\n                            extensions[ext.level] = [ext.tokenizer];\n                        }\n                        if (ext.start) { // Function to check for start of token\n                            if (ext.level === 'block') {\n                                if (extensions.startBlock) {\n                                    extensions.startBlock.push(ext.start);\n                                }\n                                else {\n                                    extensions.startBlock = [ext.start];\n                                }\n                            }\n                            else if (ext.level === 'inline') {\n                                if (extensions.startInline) {\n                                    extensions.startInline.push(ext.start);\n                                }\n                                else {\n                                    extensions.startInline = [ext.start];\n                                }\n                            }\n                        }\n                    }\n                    if ('childTokens' in ext && ext.childTokens) { // Child tokens to be visited by walkTokens\n                        extensions.childTokens[ext.name] = ext.childTokens;\n                    }\n                });\n                opts.extensions = extensions;\n            }\n            // ==-- Parse \"overwrite\" extensions --== //\n            if (pack.renderer) {\n                const renderer = this.defaults.renderer || new _Renderer(this.defaults);\n                for (const prop in pack.renderer) {\n                    const rendererFunc = pack.renderer[prop];\n                    const rendererKey = prop;\n                    const prevRenderer = renderer[rendererKey];\n                    // Replace renderer with func to run extension, but fall back if false\n                    renderer[rendererKey] = (...args) => {\n                        let ret = rendererFunc.apply(renderer, args);\n                        if (ret === false) {\n                            ret = prevRenderer.apply(renderer, args);\n                        }\n                        return ret || '';\n                    };\n                }\n                opts.renderer = renderer;\n            }\n            if (pack.tokenizer) {\n                const tokenizer = this.defaults.tokenizer || new _Tokenizer(this.defaults);\n                for (const prop in pack.tokenizer) {\n                    const tokenizerFunc = pack.tokenizer[prop];\n                    const tokenizerKey = prop;\n                    const prevTokenizer = tokenizer[tokenizerKey];\n                    // Replace tokenizer with func to run extension, but fall back if false\n                    tokenizer[tokenizerKey] = (...args) => {\n                        let ret = tokenizerFunc.apply(tokenizer, args);\n                        if (ret === false) {\n                            ret = prevTokenizer.apply(tokenizer, args);\n                        }\n                        return ret;\n                    };\n                }\n                opts.tokenizer = tokenizer;\n            }\n            // ==-- Parse Hooks extensions --== //\n            if (pack.hooks) {\n                const hooks = this.defaults.hooks || new _Hooks();\n                for (const prop in pack.hooks) {\n                    const hooksFunc = pack.hooks[prop];\n                    const hooksKey = prop;\n                    const prevHook = hooks[hooksKey];\n                    if (_Hooks.passThroughHooks.has(prop)) {\n                        hooks[hooksKey] = (arg) => {\n                            if (this.defaults.async) {\n                                return Promise.resolve(hooksFunc.call(hooks, arg)).then(ret => {\n                                    return prevHook.call(hooks, ret);\n                                });\n                            }\n                            const ret = hooksFunc.call(hooks, arg);\n                            return prevHook.call(hooks, ret);\n                        };\n                    }\n                    else {\n                        hooks[hooksKey] = (...args) => {\n                            let ret = hooksFunc.apply(hooks, args);\n                            if (ret === false) {\n                                ret = prevHook.apply(hooks, args);\n                            }\n                            return ret;\n                        };\n                    }\n                }\n                opts.hooks = hooks;\n            }\n            // ==-- Parse WalkTokens extensions --== //\n            if (pack.walkTokens) {\n                const walkTokens = this.defaults.walkTokens;\n                const packWalktokens = pack.walkTokens;\n                opts.walkTokens = function (token) {\n                    let values = [];\n                    values.push(packWalktokens.call(this, token));\n                    if (walkTokens) {\n                        values = values.concat(walkTokens.call(this, token));\n                    }\n                    return values;\n                };\n            }\n            this.defaults = { ...this.defaults, ...opts };\n        });\n        return this;\n    }\n    setOptions(opt) {\n        this.defaults = { ...this.defaults, ...opt };\n        return this;\n    }\n    lexer(src, options) {\n        return _Lexer.lex(src, options ?? this.defaults);\n    }\n    parser(tokens, options) {\n        return _Parser.parse(tokens, options ?? this.defaults);\n    }\n    #parseMarkdown(lexer, parser) {\n        return (src, options) => {\n            const origOpt = { ...options };\n            const opt = { ...this.defaults, ...origOpt };\n            // Show warning if an extension set async to true but the parse was called with async: false\n            if (this.defaults.async === true && origOpt.async === false) {\n                if (!opt.silent) {\n                    console.warn('marked(): The async option was set to true by an extension. The async: false option sent to parse will be ignored.');\n                }\n                opt.async = true;\n            }\n            const throwError = this.#onError(!!opt.silent, !!opt.async);\n            // throw error in case of non string input\n            if (typeof src === 'undefined' || src === null) {\n                return throwError(new Error('marked(): input parameter is undefined or null'));\n            }\n            if (typeof src !== 'string') {\n                return throwError(new Error('marked(): input parameter is of type '\n                    + Object.prototype.toString.call(src) + ', string expected'));\n            }\n            if (opt.hooks) {\n                opt.hooks.options = opt;\n            }\n            if (opt.async) {\n                return Promise.resolve(opt.hooks ? opt.hooks.preprocess(src) : src)\n                    .then(src => lexer(src, opt))\n                    .then(tokens => opt.walkTokens ? Promise.all(this.walkTokens(tokens, opt.walkTokens)).then(() => tokens) : tokens)\n                    .then(tokens => parser(tokens, opt))\n                    .then(html => opt.hooks ? opt.hooks.postprocess(html) : html)\n                    .catch(throwError);\n            }\n            try {\n                if (opt.hooks) {\n                    src = opt.hooks.preprocess(src);\n                }\n                const tokens = lexer(src, opt);\n                if (opt.walkTokens) {\n                    this.walkTokens(tokens, opt.walkTokens);\n                }\n                let html = parser(tokens, opt);\n                if (opt.hooks) {\n                    html = opt.hooks.postprocess(html);\n                }\n                return html;\n            }\n            catch (e) {\n                return throwError(e);\n            }\n        };\n    }\n    #onError(silent, async) {\n        return (e) => {\n            e.message += '\\nPlease report this to https://github.com/markedjs/marked.';\n            if (silent) {\n                const msg = '<p>An error occurred:</p><pre>'\n                    + escape(e.message + '', true)\n                    + '</pre>';\n                if (async) {\n                    return Promise.resolve(msg);\n                }\n                return msg;\n            }\n            if (async) {\n                return Promise.reject(e);\n            }\n            throw e;\n        };\n    }\n}\n\nconst markedInstance = new Marked();\nfunction marked(src, opt) {\n    return markedInstance.parse(src, opt);\n}\n/**\n * Sets the default options.\n *\n * @param options Hash of options\n */\nmarked.options =\n    marked.setOptions = function (options) {\n        markedInstance.setOptions(options);\n        marked.defaults = markedInstance.defaults;\n        changeDefaults(marked.defaults);\n        return marked;\n    };\n/**\n * Gets the original marked default options.\n */\nmarked.getDefaults = _getDefaults;\nmarked.defaults = _defaults;\n/**\n * Use Extension\n */\nmarked.use = function (...args) {\n    markedInstance.use(...args);\n    marked.defaults = markedInstance.defaults;\n    changeDefaults(marked.defaults);\n    return marked;\n};\n/**\n * Run callback for every token\n */\nmarked.walkTokens = function (tokens, callback) {\n    return markedInstance.walkTokens(tokens, callback);\n};\n/**\n * Compiles markdown to HTML without enclosing `p` tag.\n *\n * @param src String of markdown source to be compiled\n * @param options Hash of options\n * @return String of compiled HTML\n */\nmarked.parseInline = markedInstance.parseInline;\n/**\n * Expose\n */\nmarked.Parser = _Parser;\nmarked.parser = _Parser.parse;\nmarked.Renderer = _Renderer;\nmarked.TextRenderer = _TextRenderer;\nmarked.Lexer = _Lexer;\nmarked.lexer = _Lexer.lex;\nmarked.Tokenizer = _Tokenizer;\nmarked.Hooks = _Hooks;\nmarked.parse = marked;\nconst options = marked.options;\nconst setOptions = marked.setOptions;\nconst use = marked.use;\nconst walkTokens = marked.walkTokens;\nconst parseInline = marked.parseInline;\nconst parse = marked;\nconst parser = _Parser.parse;\nconst lexer = _Lexer.lex;\n\nexport { _Hooks as Hooks, _Lexer as Lexer, Marked, _Parser as Parser, _Renderer as Renderer, _TextRenderer as TextRenderer, _Tokenizer as Tokenizer, _defaults as defaults, _getDefaults as getDefaults, lexer, marked, options, parse, parseInline, parser, setOptions, use, walkTokens };\n//# sourceMappingURL=marked.esm.js.map\n"
  },
  {
    "path": "extension/popup/memory_cache.html",
    "content": "<!DOCTYPE html>\n\n<html>\n  <head>\n    <meta charset=\"utf-8\">\n    <link rel=\"preconnect\" href=\"https://fonts.googleapis.com\">\n    <link rel=\"preconnect\" href=\"https://fonts.gstatic.com\" crossorigin>\n    <link href=\"https://fonts.googleapis.com/css2?family=Work+Sans:wght@400;500&display=swap\" rel=\"stylesheet\">\n    <link rel=\"stylesheet\" href=\"styles.css\">\n  </head>\n\n<body>\n  <div id=\"header\">\n    <img id=\"headericon\" src=\"../icons/MC-LogoNov23.svg\"/>\n  </div>\n  <div class=\"body-container\">\n    <div id=\"save-pdf-button\" class=\"button primary-btn\">Save page as PDF</div>\n    <div id=\"save-html-button\" class=\"button primary-btn\">Save page as HTML</div>\n    <div class=\"border\"></div>\n    <div class=\"text-field\">\n      <label for=\"text-note\">Add quick note (markdown supported)</label>\n        <textarea id=\"text-note\" style=\"margin-bottom:4px;\"></textarea>\n        <!-- <div id=\"preview-note\" style=\"display:none;\"></div>\n        <button id=\"edit-button\" class=\"button secondary-btn\">Edit</button>\n        <button id=\"preview-button\" class=\"button secondary-btn\">Preview note</button> -->\n      </div>\n      <div id=\"save-note-button\" class=\"button primary-btn\"> Add text note</div>\n    </div>\n\n    <div class=\"footer\">\n      <a href=\"https://github.com/misslivirose/Memory-Cache\">View on GitHub</a>\n      <a href=\"https://memorycache.ai/\"> memorycache.ai/</a>\n    </div>\n\n    <script type=\"module\" src=\"marked.esm.js\"></script>\n    <script type=\"module\" src=\"memory_cache.js\"></script>\n</body>\n\n</html>\n"
  },
  {
    "path": "extension/popup/memory_cache.js",
    "content": "import { marked } from \"./marked.esm.js\";\n\nconst DOWNLOAD_SUBDIRECTORY = \"MemoryCache\";\n\n/*\nGenerate a file name based on date and time\n*/\nfunction generateFileName(ext) {\n  return (\n    new Date().toISOString().concat(0, 19).replaceAll(\":\", \".\") + \".\" + ext\n  );\n}\n\nasync function savePDF() {\n  try {\n    await browser.tabs.saveAsPDF({\n      toFileName: `${DOWNLOAD_SUBDIRECTORY}/PAGE${generateFileName(\"pdf\")}`,\n      silentMode: true, // silentMode requires a custom build of Firefox\n    });\n  } catch (_e) {\n    // Fallback to non-silent mode.\n    await browser.tabs.saveAsPDF({\n      // Omit the DOWNLOAD_SUBDIRECTORY prefix because saveAsPDF will not respect it.\n      toFileName: `PAGE${generateFileName(\"pdf\")}`,\n    });\n  }\n}\n\n// Send a message to the content script.\n//\n// We need code to run in the content script context for anything\n// that accesses the DOM or needs to outlive the popup window.\nfunction send(message) {\n  return new Promise((resolve, _reject) => {\n    browser.tabs.query({ active: true, currentWindow: true }, (tabs) => {\n      resolve(browser.tabs.sendMessage(tabs[0].id, message));\n    });\n  });\n}\n\nasync function saveHtml() {\n  const text = await send({ action: \"getPageText\" });\n  const filename = `${DOWNLOAD_SUBDIRECTORY}/PAGE${generateFileName(\"html\")}`;\n  const file = new File([text], filename, { type: \"text/plain\" });\n  const url = URL.createObjectURL(file);\n  browser.downloads.download({ url, filename, saveAs: false });\n}\n\nfunction saveNote() {\n  const text = document.querySelector(\"#text-note\").value;\n  const filename = `${DOWNLOAD_SUBDIRECTORY}/NOTE${generateFileName(\"md\")}`;\n  const file = new File([text], filename, { type: \"text/plain\" });\n  const url = URL.createObjectURL(file);\n  browser.downloads.download({ url, filename, saveAs: false });\n\n  document.querySelector(\"#text-note\").value = \"\";\n  browser.storage.local.set({ noteDraft: \"\" });\n}\n\nfunction debounce(func, delay) {\n  let debounceTimer;\n  return function () {\n    const context = this;\n    const args = arguments;\n    clearTimeout(debounceTimer);\n    debounceTimer = setTimeout(() => func.apply(context, args), delay);\n  };\n}\n\nfunction saveNoteDraft() {\n  const noteDraft = document.querySelector(\"#text-note\").value;\n  browser.storage.local.set({ noteDraft });\n}\n\ndocument.getElementById(\"save-pdf-button\").addEventListener(\"click\", savePDF);\ndocument.getElementById(\"save-html-button\").addEventListener(\"click\", saveHtml);\ndocument.getElementById(\"save-pdf-button\").addEventListener(\"click\", savePDF);\ndocument.getElementById(\"save-note-button\").addEventListener(\"click\", saveNote);\ndocument\n  .getElementById(\"text-note\")\n  .addEventListener(\"input\", debounce(saveNoteDraft, 250));\n\nbrowser.storage.local.get(\"noteDraft\").then((res) => {\n  if (res.noteDraft) {\n    document.querySelector(\"#text-note\").value = res.noteDraft;\n  }\n});\n\nfunction setTextView(showPreview) {\n  var textArea = document.getElementById(\"text-note\");\n  var previewDiv = document.getElementById(\"preview-note\");\n  if (showPreview) {\n    textArea.style.display = \"none\";\n    previewDiv.style.display = \"block\";\n\n    previewDiv.innerHTML = marked(textArea.value);\n  } else {\n    // Switch to editing mode\n    previewDiv.style.display = \"none\";\n    textArea.style.display = \"block\";\n  }\n}\n\ndocument.getElementById(\"edit-button\").addEventListener(\"click\", () => {\n  setTextView(false);\n});\n\ndocument.getElementById(\"preview-button\").addEventListener(\"click\", () => {\n  setTextView(true);\n});\n"
  },
  {
    "path": "extension/popup/styles.css",
    "content": ":root {\n    --border-radius: 8px;\n    --primary: linear-gradient(90deg, #FFB3BB 0%, #FFDFBA 26.56%, #FFFFBA 50.52%, #87EBDA 76.04%, #BAE1FF 100%);\n\t--primary-hover:linear-gradient(90deg, #FFA8B1 0%, #FFD9AD 26.56%, #FFFFAD 50.52%, #80EAD8 76.04%, #ADDCFF 100%);\n\t--primary-active: linear-gradient(90deg, #FF9EA8 0%, #FFD29E 26.56%, #FFFF9E 50.52%, #73E8D4 76.04%, #99D3FF 100%);\n\t--primary-content: #000000;\n\t--secondary: #180AB8;\n\t--secondary-hover: #1609A6;\n\t--secondary-active: #130893;\n\t--secondary-content: #fff;\n\t--inactive-content:#F9F9F9;\n\t--interaction-inactive:#B6B9BF;\n\t--base-100: #F6F6F6;\n\t--base-200: #fff;\n\t--base-content: #2d3d46;\n\t--base-content-subtle: #565D6D;\n\t--info: #3ac0f8;\n\t--info-content: #000;\n\t--warning: #fcbc23;\n\t--warning-content: #000;\n\t--success: #37d399;\n\t--success-content: #000;\n\t--error: #f87272;\n\t--error-content: #000;\n\t--border-1: #EDEDED;\n\n\t--xxs:4px;\n\t--xs:8px;\n\t--sm:12px;\n\t--md:16px;\n\t--lg:24px;\n\t--xl:40px;\n}\n\nbody {\n    background: var(--base-100);\n    font-weight: 500;\n    font-size: .9em;\n    font-family: 'Work Sans';\n    width: 290px;\n}\n\n#header {\n    line-height: 24px;\n    font-size: 16px;\n    color: var(--base-content);\n    background-color: var(--base-100);\n    border-bottom-color: var(--border-1);\n    border-bottom-width: 1px;\n    padding:8px;\n}\n\n.body-container {\n    padding:0 8px;\n    display:flex;\n    flex-direction: column;\n}\n\n.button {\n    cursor:pointer;\n    border-radius: var(--border-radius);\n    line-height: 21px;\n    letter-spacing: 0em;\n    text-align: center;\n    color: #565D6D;\n    padding:8px;\n}\n\n.primary-btn {\n    background: var(--primary);\n    margin-bottom:.8em;\n}\n.primary-btn:hover {\n    background:var(--primary-hover);\n}\n.primary-btn:active {\n    background:var(--primary-active);\n}\n\n.secondary-btn {\n    background:var(--base-100);\n    border: 2px solid var(--border-1);\n}\n.secondary-btn:hover {\n    background:var(--base-200);\n    border: 2px solid var(--border-1);\n}\n.secondary-btn:active {\n    background:var(--base-200);\n    border: 2px solid var(--border-1);\n}\n\n.text-field {\n    margin-bottom:8px;\n}\n.text-field textarea {\n    width: 96%;\n}\n.text-field label {\n    color: var(--base-content-subtle);\n}\n\n#text-note {\n    height: 100px;\n    border-radius: var(--border-radius);\n}\n\n.border {\n    margin:16px 0; \n    height:1px; \n    background-color: #C6C6C6;\n    border-radius: var(--border-radius);\n}\n\n.header {\n\n}\n.footer {\n    background-color:var(--base-200);\n    padding:8px;\n    display:flex;\n    justify-content: space-between;\n} \n\na {\n    text-decoration: none;\n    color:var(--secondary);\n    font-size: 14px;\n    font-weight: 400;\n}\n\na:hover, a:focus, a:visited {\n    color: var(--secondary-hover);\n}"
  },
  {
    "path": "scratch/backend/hub/.gitignore",
    "content": "*.log\n*.spec\ndist/\nbuild/\nvenv/\nsqlite.db\n__pycache__/\n*.pyc\n"
  },
  {
    "path": "scratch/backend/hub/PLAN.md",
    "content": "# Memory Cache Hub\n\nThe `hub` is a central component of Memory Cache:\n\n- It exposes APIs used by `browser-extension`, `browser-client`, and plugins.\n- It serves the static `browser-client` files over HTTP.\n- It downloads `llamafile`s and runs them as subprocesses.\n- It interacts with a vector database to ingest and retrieve document fragments.\n- It synthesizes queries and prompts for backend `llm`s on behalf of the user.\n\n## Control Flow\n\nWhen `memory-cache-hub` starts, it should:\n- Check whether another instance of `memory-cache-hub` is already running. If it is, log an error and exit.\n- Start the FastAPI server. If the port is already in use, log an error and exit.\n\nFrom then on, everything is driven by API requests.\n\n## API Endpoints\n\n| Route                                    | Method | Summary                                     |\n|:-----------------------------------------|:-------|:--------------------------------------------|\n| `/api/llamafile/list`                    | GET    | List available llamafiles                   |\n| `/api/llamafile/run`                     | POST   | Run a llamafile                             |\n| `/api/llamafile/stop`                    | POST   | Stop a running llamafile                    |\n| `/api/llamafile/get`                     | GET    | Get information about a llamafile           |\n| `/api/llamafile/download`                | POST   | Initiate download of a llamafile            |\n| `/api/llamafile/delete`                  | POST   | Delete a llamafile                          |\n| `/api/llamafile/check_download_progress` | POST   | Check download progress of a llamafile      |\n| `/api/threads/list`                      | GET    | List chat threads                           |\n| `/api/threads/get`                       | GET    | Get information about a chat thread         |\n| `/api/threads/create`                    | POST   | Create a new chat thread                    |\n| `/api/threads/delete`                    | POST   | Delete a chat thread                        |\n| `/api/threads/send_message`              | POST   | Send a message to a chat thread             |\n| `/api/threads/rag_send_message`          | POST   | Send a message to a chat thread using RAG   |\n| `/api/threads/get_messages`              | GET    | Get messages from a chat thread             |\n| `/api/threads/config`                    | POST   | Configure a chat thread                     |\n| `/api/datastore/ingest`                  | POST   | Ingest document fragments in the data store |\n| `/api/datastore/status`                  | GET    | Get status of the data store                |\n| `/api/datastore/config`                  | POST   | Configure the data store                    |\n| `/api/datastore/config`                  | GET    | Get configuration of the data store         |\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n"
  },
  {
    "path": "scratch/backend/hub/README.md",
    "content": "# Memory Cache Hub\n\nA backend for Memory Cache built on [langchain](https://python.langchain.com/), bundled as an executable with [PyInstaller](https://pyinstaller.org/). \n\n## Overview\n\nThe `hub` is a central component of Memory Cache:\n\n- It exposes APIs used by `browser-extension`, `browser-client`, and plugins.\n- It serves the static `browser-client` files over HTTP.\n- It downloads `llamafile`s and runs them as subprocesses.\n- It interacts with a vector database to ingest and retrieve document fragments.\n- It synthesizes queries and prompts for backend `llm`s on behalf of the user.\n\n## Usage\n```sh\nLLAMAFILES_DIR=~/media/llamafile ./dist/memory-cache-hub-gnu-linux\n```\n\n## Development\n\nYou can develop `hub` on your local machine or using the provided Docker development environment.\n\nIf you are developing on your local machine, you will need to install the dependencies listed in the `requirements/` files. We recommend using a virtual environment to manage these dependencies, as per the instructions in the various \"Building...\" sections below. \n\n### Development with virtual environment\n\nCreate a virtual environment:\n\n```bash\npython3.11 -m venv venv\n```\n\nActivate it:\n```\nsource venv/bin/activate\n```\n\nInstall the dependencies:\n\n```bash\npip install -r requirements/hub-base.txt \\\n    -r requirements/hub-cpu.txt \\\n    -r requirements/hub-builder.txt\n```\n\nRun the program:\n\n```bash\nLLAMAFILES_DIR=~/media/llamafile python3 src/hub.py\n```\n\nOr build with:\n\n``` sh\npython src/hub_build_gnu_linux.py\n```\n\n\n\n### Docker Development Environment\n\nA development environment for working on `hub` is provided by the Dockerfile `docker/Dockerfile.hub-dev`. \n\nThe basic workflow is to build this image and then bind mount the source code when you run the container. You will also want to bind mount a `LLAMAFILES_DIR` pointing to a directory where you'll store `llamafile`s. These can be quite large, so we avoid re-downloading them every time we start the container.\n\nWhen you are satisfied with development, you will want to package the `hub` as an executable with `PyInstaller`. Since `PyInstaller` does not support cross-compilation, you will need to run the build commands on the platform you are targeting. For example, to build a MacOS executable, you will need to run the build commands on a MacOS machine. \n\nExamples of how to build and use the development and builder images are provided in the sections below.\n\n#### Using the Docker Development Environment\n\nBuild the development image:\n\n```bash\ndocker build -f docker/Dockerfile.hub-dev -t memory-cache/hub-dev .\n```\n\nRun the development container:\n\n```bash\ndocker run -it --rm \\\n  -v $(pwd):/hub \\\n  -v ~/media/llamafile:/llamafiles \\\n  -e LLAMAFILES_DIR=/llamafiles \\\n  -p 8800:8800 \\\n  memory-cache/hub-dev \\\n  python3 src/hub.py\n```\n\nReplace `~/media/llamafile` with the path to the directory where you want to store `llamafile`s.\n\n#### Using the Docker Development Environment with NVIDIA GPUs\n\nIf you have an NVIDIA GPU, you'll need to make sure that you have the NVIDIA Container Toolkit installed and that you have the appropriate drivers and libraries installed on your host machine. A script for configuring an Ubuntu 22.04 machine can be found in the [OSAI-Ubuntu](https://github.com/johnshaughnessy/osai-ubuntu) repository.\n\nOnce you've set up your host machine, build the development image with CUDA support:\n\n```sh\ndocker build -f docker/Dockerfile.hub-dev-cuda -t memory-cache/hub-dev-cuda .\n```\n\nThen run the development container with CUDA support:\n\n```sh\ndocker run -it --rm \\\n  --gpus all \\\n  -v $(pwd):/hub \\\n  -v ~/media/llamafile:/llamafiles \\\n  -e LLAMAFILES_DIR=/llamafiles \\\n  -e CUDA_VISIBLE_DEVICES=1 \\\n  -p 8800:8800 \\\n  memory-cache/hub-dev-cuda \\\n  python3 src/hub.py\n```\n\n\n## Building for GNU/Linux\n\nBuild the builder image:\n\n```bash\ndocker build -f docker/Dockerfile.hub-builder-gnu-linux -t memory-cache/hub-builder-gnu-linux .\ndocker build -f docker/Dockerfile.hub-builder-old-gnu-linux -t memory-cache/hub-builder-old-gnu-linux .\n```\n\n\nRun the builder container:\n\n```bash\ndocker run -it --rm \\\n  -v $(pwd):/hub \\\n  memory-cache/hub-builder-gnu-linux\n\ndocker run -it --rm \\\n  -v $(pwd):/hub \\\n  memory-cache/hub-builder-old-gnu-linux\n```\n\nThe builder will generate `memory-cache-hub-gnu-linux` in the `dist` directory.\n\n## Building for MacOS\n\nOn MacOS, we use a python virtual environment to install the dependencies and run the build commands.\n\nCreate the virtual environment:\n\n```bash\npython3.11 -m venv venv\nsource venv/bin/activate\n```\n\nInstall the dependencies:\n\n```bash\npip install -r requirements/hub-base.txt \\\n    -r requirements/hub-cpu.txt \\\n    -r requirements/hub-builder.txt\n```\n\nBuild the executable:\n    \n```bash\npython3.11 src/hub_build_macos.py\n```\n\nThe builder will generate `memory-cache-hub-macos` in the `dist` directory.\n\nWhen you are done, deactivate the virtual environment:\n\n``` sh\ndeactivate\n```\n\nIf you want to remove the virtual environment, just delete the `venv` directory.\n\n## Building on Windows\n\nOn Windows, we use a python virtual environment to install the dependencies and run the build commands.\n\nInstall `python 3.11` from the [official website](https://www.python.org/downloads/).\n\nCreate the virtual environment:\n\n```bash\npy -3.11 -m venv venv\nvenv\\Scripts\\activate\n```\n\nInstall the dependencies:\n\n```bash\npip install -r requirements\\hub-base.txt -r requirements\\hub-cpu.txt -r requirements\\hub-builder.txt\n```\n\nBuild the executable:\n\n```bash\npython src\\hub_build_windows.py\n```\n\nThe builder will generate `memory-cache-hub-windows.exe` in the `dist` directory.\n\nWhen you are done, deactivate the virtual environment:\n\n``` sh\ndeactivate\n```\n\nIf you want to remove the virtual environment, just delete the `venv` directory.\n\n\n## Plan/TODO\n\n- [ ] Write Hello World server\n- [ ] Bundle with PyInstaller on Linux\n- [ ] Bundle with PyInstaller on MacOS\n- [ ] Bundle with PyInstaller on Windows\n- [ ] Test NVIDIA w/ CUDA\n- [ ] Test AMD w/ HIP/Rocm\n- [ ] Test x86-64\n- [ ] Test Apple silicon\n- [ ] Add llamafile management\n- [ ] Add ingestion\n- [ ] Add retrieval\n- [ ] Connect to browser client\n\n## Miscellaneous Notes\n\n### `python 3.11`\n\nWe use `python 3.11` (not `3.12` or later)  because `faiss-cpu` only supports up to `3.11` at the time of this writing: https://pypi.org/project/faiss-cpu/\n"
  },
  {
    "path": "scratch/backend/hub/docker/Dockerfile.hub-builder-gnu-linux",
    "content": "from ubuntu:22.04\n\nENV DEBIAN_FRONTEND=noninteractive\n\nRUN apt-get update && \\\n    apt-get install -y software-properties-common && \\\n    add-apt-repository ppa:deadsnakes/ppa && \\\n    apt-get update\n\nRUN apt-get install -y python3.11 python3.11-distutils python3.11-dev && \\\n    apt-get install -y python3-pip\n\nRUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.11 1 && \\\n    update-alternatives --set python3 /usr/bin/python3.11\n\nRUN python3.11 -m pip install --upgrade pip setuptools wheel\n\nWORKDIR /hub\n\nCOPY requirements/hub-base.txt ./\nRUN pip install --no-cache-dir -r hub-base.txt\n\nCOPY requirements/hub-cpu.txt ./\nRUN pip install --no-cache-dir -r hub-cpu.txt\n\n# GNU/Linux\n#\n# PyInstaller requires the ldd terminal application to discover the shared libraries required by each program or shared library. It is typically found in the distribution-package glibc or libc-bin.\n#\n# It also requires the objdump terminal application to extract information from object files and the objcopy terminal application to append data to the bootloader. These are typically found in the distribution-package binutils.\nRUN apt-get install -y binutils\nRUN apt-get install -y libc-bin\nCOPY requirements/hub-builder.txt ./\nRUN pip install --no-cache-dir -r hub-builder.txt\n\nCOPY . .\nCMD [ \"python3\", \"./src/hub_build_gnu_linux.py\" ]\n"
  },
  {
    "path": "scratch/backend/hub/docker/Dockerfile.hub-builder-old-gnu-linux",
    "content": "# Use an old version of ubuntu\nFROM ubuntu:20.04\n\nENV DEBIAN_FRONTEND=noninteractive\n\nRUN apt-get update && \\\n    apt-get install -y software-properties-common && \\\n    add-apt-repository ppa:deadsnakes/ppa && \\\n    apt-get update\n\nRUN apt-get install -y python3.11 python3.11-distutils python3.11-dev && \\\n    apt-get install -y python3-pip\n\nRUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.11 1 && \\\n    update-alternatives --set python3 /usr/bin/python3.11\n\n#RUN python3.11 -m pip install --upgrade pip setuptools wheel\n\nCOPY requirements/hub-base.txt ./\nRUN pip install --no-cache-dir -r hub-base.txt\n\nCOPY requirements/hub-cpu.txt ./\nRUN pip install --no-cache-dir -r hub-cpu.txt\n\nCOPY requirements/hub-builder.txt ./\nRUN pip install --no-cache-dir -r hub-builder.txt\n\nCOPY . .\nCMD [ \"python3\", \"./src/hub_build_gnu_linux.py\" ]\n"
  },
  {
    "path": "scratch/backend/hub/docker/Dockerfile.hub-dev",
    "content": "from ubuntu:22.04\n\nENV DEBIAN_FRONTEND=noninteractive\n\n# Install software-properties-common to add PPAs\nRUN apt-get update && \\\n    apt-get install -y software-properties-common && \\\n    add-apt-repository ppa:deadsnakes/ppa && \\\n    apt-get update\n\n# Install Python 3.11 and pip\nRUN apt-get install -y python3.11 python3.11-distutils && \\\n    apt-get install -y python3-pip\n\n# Update alternatives to use Python 3.11 as the default python3\nRUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.11 1 && \\\n    update-alternatives --set python3 /usr/bin/python3.11\n\n# Ensure pip is updated and set to use the correct Python version\nRUN python3.11 -m pip install --upgrade pip setuptools wheel\n\nRUN apt-get install -y wget\nRUN wget -O /usr/bin/ape https://cosmo.zip/pub/cosmos/bin/ape-$(uname -m).elf\nRUN chmod +x /usr/bin/ape\n# RUN sh -c \"echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register\"\n# RUN sh -c \"echo ':APE-jart:M::jartsr::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register\"\n\nWORKDIR /hub\n\nCOPY requirements/hub-base.txt ./\nRUN pip install --no-cache-dir -r hub-base.txt\n\nCOPY requirements/hub-cpu.txt ./\nRUN pip install --no-cache-dir -r hub-cpu.txt\n\nCOPY . .\n\n\nCMD [ \"python3\", \"./src/hub.py\" ]\n"
  },
  {
    "path": "scratch/backend/hub/docker/Dockerfile.hub-dev-cuda",
    "content": "FROM nvidia/cuda:12.3.1-devel-ubuntu22.04\n\nENV DEBIAN_FRONTEND=noninteractive\n\n# Install software-properties-common to add PPAs\nRUN apt-get update && \\\n    apt-get install -y software-properties-common && \\\n    add-apt-repository ppa:deadsnakes/ppa && \\\n    apt-get update\n\n# Install Python 3.11 and pip\nRUN apt-get install -y python3.11 python3.11-distutils && \\\n    apt-get install -y python3-pip\n\n# Update alternatives to use Python 3.11 as the default python3\nRUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.11 1 && \\\n    update-alternatives --set python3 /usr/bin/python3.11\n\n# Ensure pip is updated and set to use the correct Python version\nRUN python3.11 -m pip install --upgrade pip setuptools wheel\n\nRUN apt-get install -y wget\nRUN wget -O /usr/bin/ape https://cosmo.zip/pub/cosmos/bin/ape-$(uname -m).elf\nRUN chmod +x /usr/bin/ape\n# RUN sh -c \"echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register\"\n# RUN sh -c \"echo ':APE-jart:M::jartsr::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register\"\n\nWORKDIR /hub\n\nCOPY requirements/hub-base.txt ./\nRUN pip install --no-cache-dir -r hub-base.txt\n\nCOPY requirements/hub-cpu.txt ./\nRUN pip install --no-cache-dir -r hub-cpu.txt\n\nCOPY . .\n\n\nCMD [ \"python3\", \"./src/hub.py\" ]\n"
  },
  {
    "path": "scratch/backend/hub/requirements/hub-base.txt",
    "content": "aiofiles\nbs4\ncertifi\nfastapi\nlangchain\nlangchainhub\nlangchain-openai\nlangchain-cli\nlangserve[all]\nrequests ~= 2.31\ntqdm\nuvicorn\npsutil\n"
  },
  {
    "path": "scratch/backend/hub/requirements/hub-builder.txt",
    "content": "pyinstaller\n"
  },
  {
    "path": "scratch/backend/hub/requirements/hub-cpu.txt",
    "content": "faiss-cpu\n"
  },
  {
    "path": "scratch/backend/hub/src/api/llamafile_api.py",
    "content": "from fastapi import APIRouter\nfrom pydantic import BaseModel\nfrom llamafile_manager import get_llamafile_manager\nfrom typing import Optional\n\nrouter = APIRouter()\nmanager = get_llamafile_manager()\n\nclass LlamafileInfo(BaseModel):\n    name: str\n    url: str\n    downloaded: bool\n    running: bool\n    download_progress: Optional[int]\n\nclass ListLlamafilesResponse(BaseModel):\n    llamafiles: list[LlamafileInfo]\n\n@router.get(\"/list_llamafiles\")\nasync def list_llamafiles():\n    \"\"\"List all llamafiles, including those that have not been downloaded.\"\"\"\n    llamafiles = manager.list_all_llamafiles()\n    llamafile_infos = []\n    for info in llamafiles:\n        llamafile_infos.append(LlamafileInfo(name=info.name,\n                                             url=info.url,\n                                             downloaded=manager.has_llamafile(info.name),\n                                             running=manager.is_llamafile_running(info.name),\n                                             download_progress=manager.llamafile_download_progress(info.name)))\n\n    return ListLlamafilesResponse(llamafiles=llamafile_infos)\n\nclass GetLlamafileRequest(BaseModel):\n    name: str\n\nclass GetLlamafileResponse(BaseModel):\n    # Respond with the llamafile info or None if the llamafile is not found\n    llamafile: Optional[LlamafileInfo]\n\n@router.post(\"/get_llamafile\")\nasync def get_llamafile(request: GetLlamafileRequest):\n    \"\"\"Get the llamafile info for the llamafile of the given name.\"\"\"\n    all_llamafile_infos = manager.list_all_llamafiles()\n    llamafile = next((l for l in all_llamafile_infos if l.name == request.name), None)\n    if llamafile is None:\n        return GetLlamafileResponse(llamafile=None)\n\n    return GetLlamafileResponse(\n        llamafile=LlamafileInfo(name=llamafile.name,\n                                url=llamafile.url,\n                                downloaded=manager.has_llamafile(llamafile.name),\n                                running=manager.is_llamafile_running(llamafile.name),\n                                download_progress=manager.llamafile_download_progress(llamafile.name)))\n\nclass DownloadLlamafileRequest(BaseModel):\n    name: str\n\nclass DownloadLlamafileResponse(BaseModel):\n    success: bool\n\n@router.post(\"/download_llamafile\")\nasync def download_llamafile(request: DownloadLlamafileRequest):\n    \"\"\"Download the llamafile of the given name.\"\"\"\n    result = manager.download_llamafile_by_name(request.name)\n    return DownloadLlamafileResponse(success=result is not None)\n\nclass LlamafileDownloadProgressRequest(BaseModel):\n    name: str\nclass LlamafileDownloadProgressResponse(BaseModel):\n    progress: Optional[int]\n@router.post(\"/llamafile_download_progress\")\nasync def llamafile_download_progress(request: LlamafileDownloadProgressRequest):\n    \"\"\"Get the download progress of the llamafile of the given name.\"\"\"\n    progress = manager.llamafile_download_progress(request.name)\n    return LlamafileDownloadProgressResponse(progress=progress)\n\nclass RunLlamafileRequest(BaseModel):\n    name: str\n\nclass RunLlamafileResponse(BaseModel):\n    success: bool\n\n@router.post(\"/run_llamafile\")\nasync def run_llamafile(request: RunLlamafileRequest):\n    \"\"\"Download the llamafile of the given name.\"\"\"\n    # The given name might not be valid, in which case the manager will throw.\n    # If the manager throws, return success: false\n    try:\n        result = manager.run_llamafile(request.name, [\"--host\", \"0.0.0.0\",\n                                                      \"--port\", \"8800\",\n                                                      \"--nobrowser\"])\n                                                      #\"-ngl\", \"999\"])\n        return RunLlamafileResponse(success=True)\n    except ValueError:\n        return RunLlamafileResponse(success=False)\n\n\nclass StopLlamafileRequest(BaseModel):\n    name: str\n\nclass StopLlamafileResponse(BaseModel):\n    success: bool\n\n@router.post(\"/stop_llamafile\")\nasync def stop_llamafile(request: StopLlamafileRequest):\n    \"\"\"Stop the llamafile of the given name.\"\"\"\n    # The given name might not be valid, in which case the manager will throw.\n    # If the manager throws, return success: false\n    try:\n        result = manager.stop_llamafile_by_name(request.name)\n        return StopLlamafileResponse(success=result)\n    except ValueError:\n        return StopLlamafileResponse(success=False)\n"
  },
  {
    "path": "scratch/backend/hub/src/api/thread_api.py",
    "content": "from fastapi import APIRouter\nfrom pydantic import BaseModel\n\nrouter = APIRouter()\n\n\nclass ListThreadsResponse(BaseModel):\n    threads: list[str]\n\n@router.get(\"/list_threads\")\nasync def list_threads():\n    return ListThreadsResponse(threads=[\"thread1\", \"thread2\", \"thread3\"])\n\nclass GetThreadResponse(BaseModel):\n    messages: list[str]\n\n@router.get(\"/get_thread\")\nasync def get_thread():\n    return GetThreadResponse(messages=[\"message1\", \"message2\", \"message3\"])\n\nclass AppendToThreadRequest(BaseModel):\n    message: str\n\nclass AppendToThreadResponse(BaseModel):\n    success: bool\n\n@router.post(\"/append_to_thread\")\nasync def append_to_thread(request: AppendToThreadRequest):\n    return AppendToThreadResponse(success=True)\n"
  },
  {
    "path": "scratch/backend/hub/src/async_utils.py",
    "content": "import threading\nimport asyncio\n\ndef start_async_loop(loop):\n    asyncio.set_event_loop(loop)\n    loop.run_forever()\n\nasync def wait_for(coroutines, finish_event):\n    await asyncio.gather(*coroutines)\n    finish_event.set()\n\ndef run(coroutines, loop):\n    finish_event = threading.Event()\n    asyncio.run_coroutine_threadsafe(wait_for(coroutines, finish_event), loop)\n    return finish_event\n\ndef run_async(coroutines):\n    finish_event = threading.Event()\n    asyncio.run_coroutine_threadsafe(wait_for(coroutines, finish_event), get_my_loop())\n    return finish_event\n\nloop = None\ndef set_my_loop(l):\n    global loop\n    loop = l\n\ndef get_my_loop():\n    global loop\n    return loop\n"
  },
  {
    "path": "scratch/backend/hub/src/chat.py",
    "content": "from langchain_core.messages import HumanMessage, SystemMessage\nfrom langchain_openai import ChatOpenAI\nfrom langchain_core.prompts import ChatPromptTemplate\nfrom langchain_core.output_parsers import StrOutputParser\n\nchat = ChatOpenAI(temperature=0,\n                  openai_api_key=\"KEY\",\n                  base_url=\"http://localhost:8080/v1\")\n\nmessages = [\n    SystemMessage(\n        content=\"You are a helpful assistant. You keep answers short and to the point. You do not add any extra information. For example, if you are asked how to accomplish a command line task, you reply with the command to use with not additional comments.\"\n    ),\n    HumanMessage(\n        content=\"Who wrote Linux?\"\n    ),\n    SystemMessage(\n        content=\"Linus Torvalds\"\n    ),\n    HumanMessage(\n        content=\"When?\"\n    ),\n    SystemMessage(\n        content=\"1991\"\n    ),\n    HumanMessage(\n        content=\"What is the capital of France?\"\n    ),\n    SystemMessage(\n        content=\"Paris\"\n    ),\n    HumanMessage(\n        content=\"How do I add a git origin?\"\n    ),\n    SystemMessage(\n        content=\"git remote add <origin_name> <origin_url>\"\n    ),\n    HumanMessage(\n        content=\"How do I find the process on a port (in Linux)?\"\n    ),\n    SystemMessage(\n        content=\"lsof -i :<port_number>\"\n    ),\n]\n\nwhile True:\n    user_input = input(\"> \")\n    if user_input.lower() == \"exit\":\n        break\n    message = HumanMessage(content=f\"{user_input}\")\n    messages.append(message)\n    prompt = ChatPromptTemplate.from_messages(messages)\n    chain = prompt | chat | StrOutputParser()\n    response = chain.invoke({})\n    print(response)\n    messages.append(SystemMessage(content=response))\n"
  },
  {
    "path": "scratch/backend/hub/src/chat2.py",
    "content": "from langchain_community.chat_models import ChatOllama\nfrom langchain_core.output_parsers import StrOutputParser\nfrom langchain_core.prompts import ChatPromptTemplate\n\nllm = ChatOllama(model=\"mixtral:8x7b-instruct-v0.1-fp16\")\nllm.base_url = \"http://localhost:11434\"\nprompt = ChatPromptTemplate.from_template(\"Tell me a short joke about {topic}\")\nchain = prompt | llm | StrOutputParser()\nprint(chain.invoke({\"topic\": \"Space travel\"}))\n"
  },
  {
    "path": "scratch/backend/hub/src/chat3.py",
    "content": "from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder\nfrom langchain_core.runnables.history import RunnableWithMessageHistory\nfrom langchain_community.chat_message_histories import SQLChatMessageHistory\nfrom langchain_openai import ChatOpenAI\n\n# This is where we configure the session id\nconfig = {\"configurable\": {\"session_id\": \"test_session_id2\"}}\n\nprompt = ChatPromptTemplate.from_messages(\n    [\n        (\"system\", \"You are a helpful assistant.\"),\n        MessagesPlaceholder(variable_name=\"history\"),\n        (\"human\", \"{question}\"),\n    ]\n)\n\nchat = ChatOpenAI(temperature=0,\n                  openai_api_key=\"KEY\",\n                  base_url=\"http://localhost:8080/v1\")\n\nchain = prompt | chat\nchain_with_history = RunnableWithMessageHistory(\n    chain,\n    lambda session_id: SQLChatMessageHistory(\n        session_id=session_id, connection_string=\"sqlite:///sqlite.db\"\n    ),\n    input_messages_key=\"question\",\n    history_messages_key=\"history\",\n)\n\nprint(chain_with_history.invoke({\"question\": \"Whats my name\"}, config=config))\n"
  },
  {
    "path": "scratch/backend/hub/src/fastapi_app.py",
    "content": "from fastapi import FastAPI\nfrom fastapi.staticfiles import StaticFiles\n#from langchain_community.llms.llamafile import Llamafile\n#from langserve import add_routes\nimport os\nimport sys\nfrom api.thread_api import router as thread_router\nfrom api.llamafile_api import router as llamafile_router\nfrom fastapi.middleware.cors import CORSMiddleware\n\napp = FastAPI(\n    title=\"Memory Cache Hub\",\n    version=\"1.0\",\n    description=\"Manage llamafiles, document store, and vector database.\",\n)\n\norigins = [\n    \"http://localhost\",\n    \"http://localhost:8080\",\n    \"http://localhost:3000\",\n    \"http://192.168.0.141\",\n    \"http://192.168.0.141:8080\",\n    \"http://192.168.0.141:3000\",\n]\n\napp.add_middleware(\n    CORSMiddleware,\n    allow_origins=origins,\n    allow_credentials=True,\n    allow_methods=[\"*\"],\n    allow_headers=[\"*\"],\n)\n\napp.include_router(thread_router, prefix=\"/api/thread\")\napp.include_router(llamafile_router, prefix=\"/api/llamafile\")\n\n#llm = Llamafile(streaming=True)\n#llm.base_url = \"http://localhost:8800\"\n#add_routes(app, llm, path=\"/llm\")\n\nif getattr(sys, 'frozen', False) and hasattr(sys, '_MEIPASS'):\n    # The application is frozen by PyInstaller\n    bundle_dir = sys._MEIPASS\n    static_files_dir = os.path.join(bundle_dir, 'browser-client')\nelse:\n    # The application is running in a normal Python environment\n    bundle_dir = os.path.dirname(os.path.abspath(__file__))\n    static_files_dir = os.path.join(bundle_dir, '..', '..', '..', 'hub-browser-client', 'build')\n\napp.mount(\"/\", StaticFiles(directory=static_files_dir, html=True), name=\"static\")\n"
  },
  {
    "path": "scratch/backend/hub/src/gradio_app.py",
    "content": "import gradio as gr\nimport requests\nfrom time import sleep\n\n# Define functions that will interact with the FastAPI endpoints\ndef list_llamafiles():\n    # If the request fails, it throws an error. We do not want the app to crash, so we catch the error and return an empty string.\n    try:\n        response = requests.get(\"http://localhost:8001/api/llamafile_manager/list_llamafiles\")\n        if response.status_code == 200:\n            return \"\\n\".join(response.json())\n        return \"\"\n    except:\n        return \"\"\n\ndef has_llamafile(name):\n    response = requests.get(f\"http://localhost:8001/api/llamafile_manager/has_llamafile/{name}\")\n    if response.status_code == 200:\n        return response.json()\n    return \"Error checking llamafile.\"\n\ndef download_llamafile(url, name):\n    response = requests.post(\"http://localhost:8001/api/llamafile_manager/download_llamafile\", json={\"url\": url, \"name\": name})\n    if response.status_code == 200:\n        return \"Download initiated.\"\n    return \"Failed to initiate download.\"\n\ndef download_progress(url, name):\n    response = requests.post(\"http://localhost:8001/api/llamafile_manager/download_progress\", json={\"url\": url, \"name\": name})\n    if response.status_code == 200:\n        return response.json()\n    return \"Failed to get download progress.\"\n\ndef run_llamafile(name, args):\n    response = requests.post(\"http://localhost:8001/api/llamafile_manager/run_llamafile\", json={\"name\": name, \"args\": args.split()})\n    if response.status_code == 200:\n        return response.text\n    return \"Failed to run llamafile.\"\n\nnum = 0\ndef increment():\n    global num\n    while True:\n        num += 1\n        yield num\n        sleep(1)\n\ndef my_inc():\n    global num\n    def inner():\n        global num\n        num += 1\n        return num\n    return inner\n\n# Create the Gradio interface\nwith gr.Blocks() as app:\n\n    with gr.Tab(\"Llamafile Manager\"):\n        gr.Markdown(\"# Llamafile Manager\")\n        gr.Textbox(value = my_inc(), label = \"Seconds\", interactive=False, every=1)\n\n\n    with gr.Tab(\"List Llamafiles\"):\n        gr.Markdown(\"List all Llamafiles\")\n        gr.Button(\"List Llamafiles\").click(list_llamafiles, [], gr.Textbox(label=\"Llamafiles\"))\n\n    with gr.Tab(\"Check Llamafile\"):\n        gr.Markdown(\"Check if a Llamafile exists\")\n        name_input = gr.Textbox(label=\"Llamafile Name\")\n        gr.Button(\"Check\").click(has_llamafile, [name_input], gr.Textbox(label=\"Exists\"))\n\n    with gr.Tab(\"Download Llamafile\"):\n        gr.Markdown(\"Download a Llamafile\")\n        url_input = gr.Textbox(label=\"URL\")\n        name_input_download = gr.Textbox(label=\"Name\")\n        gr.Button(\"Download\").click(download_llamafile, [url_input, name_input_download], gr.Textbox(label=\"Status\"))\n\n    with gr.Tab(\"Download Progress\"):\n        gr.Markdown(\"Check Download Progress\")\n        url_input_progress = gr.Textbox(label=\"URL\")\n        name_input_progress = gr.Textbox(label=\"Name\")\n        gr.Button(\"Check Progress\").click(download_progress, [url_input_progress, name_input_progress], gr.Textbox(label=\"Progress\"))\n\n    with gr.Tab(\"Run Llamafile\"):\n        gr.Markdown(\"Run a Llamafile\")\n        name_input_run = gr.Textbox(label=\"Name\")\n        args_input_run = gr.Textbox(label=\"Args (space-separated)\")\n        gr.Button(\"Run\").click(run_llamafile, [name_input_run, args_input_run], gr.Textbox(label=\"Run Status\"))\n\niface = app\n"
  },
  {
    "path": "scratch/backend/hub/src/hub.py",
    "content": "from llamafile_manager import get_llamafile_manager\nfrom async_utils import start_async_loop, set_my_loop\nimport asyncio\nimport threading\nimport os\nimport uvicorn\nimport webbrowser\nfrom fastapi_app import app\n# from gradio_app import iface\n\ndef run_api_server():\n    uvicorn.run(app, host=\"0.0.0.0\", port=8001)\n\n# def run_gradio_interface():\n#     iface.launch()\n\nif __name__ == \"__main__\":\n\n    llamafiles_dir = os.environ.get('LLAMAFILES_DIR')\n    if not llamafiles_dir:\n        raise ValueError(\"LLAMAFILES_DIR environment variable is not set\")\n    manager = get_llamafile_manager(llamafiles_dir)\n\n    loop = asyncio.new_event_loop()\n    set_my_loop(loop)\n    t = threading.Thread(target=start_async_loop, args=(loop,), daemon=True)\n    t.start()\n\n    t2 = threading.Thread(target=run_api_server, daemon=True)\n    t2.start()\n\n    # t3 = threading.Thread(target=run_gradio_interface, daemon=True)\n    # t3.start()\n\n    webbrowser.open(\"http://localhost:8001/\", new=0)\n    #webbrowser.open(\"http://localhost:7860/\", new=0)\n\n    t.join()\n\n    manager.stop_all_llamafiles()\n"
  },
  {
    "path": "scratch/backend/hub/src/hub_build_gnu_linux.py",
    "content": "import PyInstaller.__main__\nimport os\n\ncurrent_directory = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))\nadditional_files = [\n    (os.path.join(current_directory, 'requirements', 'hub-base.txt'), '.'),\n    (os.path.join(current_directory, 'requirements', 'hub-cpu.txt'), '.'),\n    (os.path.join(current_directory, 'requirements', 'hub-builder.txt'), '.'),\n    (os.path.join(current_directory, '..', '..', 'hub-browser-client', 'build'), 'browser-client'),\n]\nentry_point = os.path.join(current_directory, \"src\", \"hub.py\")\n\nPyInstaller.__main__.run([\n    entry_point,\n    '--onefile',  # Bundle everything into a single executable\n    # '--hidden-import=module_name',  # Uncomment and replace with actual module names if there are hidden imports\n    '--clean',  # Clean PyInstaller build folder before building\n    '--name=memory-cache-hub-gnu-linux',\n] + [f'--add-data={src}{os.pathsep}{dst}' for src, dst in additional_files])\n"
  },
  {
    "path": "scratch/backend/hub/src/hub_build_macos.py",
    "content": "import PyInstaller.__main__\nimport os\n\ncurrent_directory = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))\nadditional_files = [\n    (os.path.join(current_directory, 'requirements', 'hub-base.txt'), '.'),\n    (os.path.join(current_directory, 'requirements', 'hub-cpu.txt'), '.'),\n    (os.path.join(current_directory, 'requirements', 'hub-builder.txt'), '.'),\n    (os.path.join(current_directory, '..', '..', 'hub-browser-client', 'build'), 'browser-client'),\n]\nentry_point = os.path.join(current_directory, \"src\", \"hub.py\")\n\nPyInstaller.__main__.run([\n    entry_point,\n    '--onefile',  # Bundle everything into a single executable\n    # '--hidden-import=module_name',  # Uncomment and replace with actual module names if there are hidden imports\n    '--clean',  # Clean PyInstaller build folder before building\n    '--name=memory-cache-hub-macos',\n] + [f'--add-data={src}{os.pathsep}{dst}' for src, dst in additional_files])\n"
  },
  {
    "path": "scratch/backend/hub/src/hub_build_windows.py",
    "content": "import PyInstaller.__main__\nimport os\n\ncurrent_directory = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))\nadditional_files = [\n    (os.path.join(current_directory, 'requirements', 'hub-base.txt'), '.'),\n    (os.path.join(current_directory, 'requirements', 'hub-cpu.txt'), '.'),\n    (os.path.join(current_directory, 'requirements', 'hub-builder.txt'), '.'),\n    (os.path.join(current_directory, '..', '..', 'hub-browser-client', 'build'), 'browser-client'),\n]\nentry_point = os.path.join(current_directory, \"src\", \"hub.py\")\n\nPyInstaller.__main__.run([\n    entry_point,\n    '--onefile',  # Bundle everything into a single executable\n    # '--hidden-import=module_name',  # Uncomment and replace with actual module names if there are hidden imports\n    '--clean',  # Clean PyInstaller build folder before building\n    '--name=memory-cache-hub-windows',\n] + [f'--add-data={src}{os.pathsep}{dst}' for src, dst in additional_files])\n"
  },
  {
    "path": "scratch/backend/hub/src/llamafile_infos.json",
    "content": "[\n  {\n    \"Model\": \"LLaVA 1.5\",\n    \"Size\": \"3.97 GB\",\n    \"License\": \"LLaMA 2\",\n    \"License URL\": \"https://ai.meta.com/resources/models-and-libraries/llama-downloads/\",\n    \"filename\": \"llava-v1.5-7b-q4.llamafile\",\n    \"url\": \"https://huggingface.co/jartine/llava-v1.5-7B-GGUF/resolve/main/llava-v1.5-7b-q4.llamafile?download=true\"\n  },\n  {\n    \"Model\": \"Mistral-7B-Instruct\",\n    \"Size\": \"5.15 GB\",\n    \"License\": \"Apache 2.0\",\n    \"License URL\": \"https://choosealicense.com/licenses/apache-2.0/\",\n    \"filename\": \"mistral-7b-instruct-v0.2.Q5_K_M.llamafile\",\n    \"url\": \"https://huggingface.co/jartine/Mistral-7B-Instruct-v0.2-llamafile/resolve/main/mistral-7b-instruct-v0.2.Q5_K_M.llamafile?download=true\"\n  },\n  {\n    \"Model\": \"Mixtral-8x7B-Instruct\",\n    \"Size\": \"30.03 GB\",\n    \"License\": \"Apache 2.0\",\n    \"License URL\": \"https://choosealicense.com/licenses/apache-2.0/\",\n    \"filename\": \"mixtral-8x7b-instruct-v0.1.Q5_K_M.llamafile\",\n    \"url\": \"https://huggingface.co/jartine/Mixtral-8x7B-Instruct-v0.1-llamafile/resolve/main/mixtral-8x7b-instruct-v0.1.Q5_K_M.llamafile?download=true\"\n  },\n  {\n    \"Model\": \"WizardCoder-Python-34B\",\n    \"Size\": \"22.23 GB\",\n    \"License\": \"LLaMA 2\",\n    \"License URL\": \"https://ai.meta.com/resources/models-and-libraries/llama-downloads/\",\n    \"filename\": \"wizardcoder-python-34b-v1.0.Q5_K_M.llamafile\",\n    \"url\": \"https://huggingface.co/jartine/WizardCoder-Python-34B-V1.0-llamafile/resolve/main/wizardcoder-python-34b-v1.0.Q5_K_M.llamafile?download=true\"\n  },\n  {\n    \"Model\": \"WizardCoder-Python-13B\",\n    \"Size\": \"7.33 GB\",\n    \"License\": \"LLaMA 2\",\n    \"License URL\": \"https://ai.meta.com/resources/models-and-libraries/llama-downloads/\",\n    \"filename\": \"wizardcoder-python-13b.llamafile\",\n    \"url\": \"https://huggingface.co/jartine/wizardcoder-13b-python/resolve/main/wizardcoder-python-13b.llamafile?download=true\"\n  },\n  {\n    \"Model\": \"TinyLlama-1.1B\",\n    \"Size\": \"0.76 GB\",\n    \"License\": \"Apache 2.0\",\n    \"License URL\": \"https://choosealicense.com/licenses/apache-2.0/\",\n    \"filename\": \"TinyLlama-1.1B-Chat-v1.0.Q5_K_M.llamafile\",\n    \"url\": \"https://huggingface.co/jartine/TinyLlama-1.1B-Chat-v1.0-GGUF/resolve/main/TinyLlama-1.1B-Chat-v1.0.Q5_K_M.llamafile?download=true\"\n  },\n  {\n    \"Model\": \"Rocket-3B\",\n    \"Size\": \"1.89 GB\",\n    \"License\": \"cc-by-sa-4.0\",\n    \"License URL\": \"https://creativecommons.org/licenses/by-sa/4.0/deed.en\",\n    \"filename\": \"rocket-3b.Q5_K_M.llamafile\",\n    \"url\": \"https://huggingface.co/jartine/rocket-3B-llamafile/resolve/main/rocket-3b.Q5_K_M.llamafile?download=true\"\n  },\n  {\n    \"Model\": \"Phi-2\",\n    \"Size\": \"1.96 GB\",\n    \"License\": \"MIT\",\n    \"License URL\": \"https://huggingface.co/microsoft/phi-2/resolve/main/LICENSE\",\n    \"filename\": \"phi-2.Q5_K_M.llamafile\",\n    \"url\": \"https://huggingface.co/jartine/phi-2-llamafile/resolve/main/phi-2.Q5_K_M.llamafile?download=true\"\n  }\n]\n"
  },
  {
    "path": "scratch/backend/hub/src/llamafile_infos.py",
    "content": "# llamafile_name_llava_v1_5_7b_q4 = \"llava-v1.5-7b-q4.llamafile\"\n# llamafile_url_llava_v1_5_7b_q4 = \"https://huggingface.co/jartine/llava-v1.5-7B-GGUF/resolve/main/llava-v1.5-7b-q4.llamafile?download=true\"\n\n# llamafile_name_mistral_7b_instruct_v0_2_q5_k_m = \"mistral-7b-instruct-v0.2.Q5_K_M.llamafile\"\n# llamafile_url_mistral_7b_instruct_v0_2_q5_k_m = \"https://huggingface.co/jartine/Mistral-7B-Instruct-v0.2-llamafile/resolve/main/mistral-7b-instruct-v0.2.Q5_K_M.llamafile?download=true\"\n\n# Parse the json in llamafile_infos.json\nimport json\nimport os\n\nclass LlamafileInfo:\n    def __init__(self, info_dict):\n        self.model = info_dict['Model']\n        self.size = info_dict['Size']\n        self.license = info_dict['License']\n        self.license_url = info_dict['License URL']\n        self.name = info_dict['filename']\n        self.url = info_dict['url']\n\ndef get_llamafile_infos():\n    llamafile_infos_path = os.path.join(os.path.dirname(__file__), \"llamafile_infos.json\")\n    with open(llamafile_infos_path, \"r\") as f:\n        llamafile_infos_dicts = json.load(f)\n\n    # Convert each dictionary to a LlamafileInfo object\n    llamafile_infos = [LlamafileInfo(info_dict) for info_dict in llamafile_infos_dicts]\n    return llamafile_infos\n"
  },
  {
    "path": "scratch/backend/hub/src/llamafile_manager.py",
    "content": "import os\nimport asyncio\nimport aiohttp\nimport aiofiles\n#import certifi\nimport asyncio\nimport subprocess\nimport psutil\nfrom llamafile_infos import get_llamafile_infos\nfrom async_utils import run_async\n\nclass DownloadHandle:\n    def __init__(self):\n        self.url = None\n        self.filename = None\n        self.llamafile_name = None\n        self.content_length = 0\n        self.written = 0\n        self.coroutine = None\n\n    def progress(self):\n        return int(100 * self.written / self.content_length if self.content_length > 0 else 0)\n\n    def __repr__(self):\n        return f\"DownloadHandle(url={self.url}, filename={self.filename}, content_length={self.content_length}, written={self.written})\"\n\nasync def download(handle: DownloadHandle):\n    # BUG On MacOS, https requests failed unless I disabled ssl checking.\n    # TODO Fix ssl issue on MacOS\n    #      This github issue may be related:\n    #      https://github.com/aio-libs/aiohttp/issues/955\n    #async with aiohttp.ClientSession() as session:\n    async with aiohttp.ClientSession(connector=aiohttp.TCPConnector(ssl=False)) as session:\n        async with session.get(handle.url) as response:\n            handle.content_length = int(response.headers.get('content-length', 0))\n            handle.written = 0\n            async with aiofiles.open(handle.filename, 'wb') as file:\n                async for data in response.content.iter_chunked(1024):\n                    await file.write(data)\n                    handle.written += len(data)\n\nasync def update_tqdm(pbar, handle: DownloadHandle):\n    while handle.progress() < 100:\n        # We don't know the total size until the download starts, so we update it here\n        pbar.total = handle.content_length / 1024\n        pbar.update(handle.written / 1024 - pbar.n)\n        await asyncio.sleep(0.1)\n\nclass RunHandle:\n    def __init__(self):\n        self.llamafile_name = None\n        self.filename = None\n        self.args = []\n        self.process = None\n\n    def __repr__(self):\n        return f\"RunHandle(filename={self.filename}, args={self.args}, process={self.process})\"\n\n_instance = None\n\ndef get_llamafile_manager(llamafiles_dir: str = None):\n    global _instance\n    if _instance is None:\n        _instance = LlamafileManager(llamafiles_dir)\n\n    if _instance.llamafiles_dir is None and llamafiles_dir is not None:\n        _instance.llamafiles_dir = llamafiles_dir\n\n    if _instance.llamafiles_dir != None and llamafiles_dir != _instance.llamafiles_dir:\n        raise ValueError(\"LlamafileManager already created with a different llamafiles_dir\")\n\n    return _instance\n\nclass LlamafileManager:\n    def __init__(self, llamafiles_dir: str):\n        self.llamafiles_dir = llamafiles_dir\n        self.download_handles = []\n        self.run_handles = []\n\n    def list_all_llamafiles(self):\n        return get_llamafile_infos()\n\n    def list_llamafiles(self):\n        return [f for f in os.listdir(self.llamafiles_dir) if f.endswith('.llamafile')]\n\n    def has_llamafile(self, name):\n        return name in self.list_llamafiles()\n\n    def download_llamafile_by_name(self, name):\n        for info in self.list_all_llamafiles():\n            if info.name == name:\n                return self.download_llamafile(info.url, info.name)\n        return None\n\n    def download_llamafile(self, url, name):\n        # If we already have a download handle for this file, delete that other handle\n        for handle in self.download_handles:\n            if handle.llamafile_name == name:\n                self.download_handles.remove(handle)\n                break\n\n        handle = DownloadHandle()\n        self.download_handles.append(handle)\n        handle.url = url\n        handle.llamafile_name = name\n        handle.filename = os.path.join(self.llamafiles_dir, name)\n        handle.coroutine = download(handle)\n        handle.finish_event = run_async([handle.coroutine])\n        return handle\n\n    def run_llamafile(self, name: str, args: list):\n        if not self.has_llamafile(name):\n            raise ValueError(f\"llamafile {name} is not available\")\n        handle = RunHandle()\n        self.run_handles.append(handle)\n        handle.llamafile_name = name\n        handle.filename = os.path.join(self.llamafiles_dir, name)\n        # Print the file path, and check if the file exists\n        print(handle.filename)\n        if not os.path.isfile(handle.filename):\n            raise FileNotFoundError(f\"{name} not found in {self.llamafiles_dir}\")\n        if os.name == 'posix' or os.name == 'darwin':\n            if not os.access(handle.filename, os.X_OK):\n                os.chmod(handle.filename, 0o755)\n        handle.args = args\n        cmd = f\"{handle.filename} {' '.join(args)}\"\n        handle.process = subprocess.Popen([\"sh\", \"-c\", cmd])\n        return handle\n\n    def is_llamafile_running(self, name: str):\n        return any(h for h in self.run_handles if h.llamafile_name == name)\n\n    def stop_llamafile_by_name(self, name: str):\n        for handle in self.run_handles:\n            if handle.llamafile_name == name:\n                return self.stop_llamafile(handle)\n        return False\n\n    def stop_llamafile(self, handle: RunHandle):\n        print(f\"Stopping process {handle.process.pid}\")\n        if handle.process.poll() is None:\n            try:\n                parent = psutil.Process(handle.process.pid)\n                children = parent.children(recursive=True)  # Get all child processes\n                for child in children:\n                    print(f\"Terminating child process {child.pid}, {child.name()}\")\n                    child.terminate()  # Terminate each child\n                gone, still_alive = psutil.wait_procs(children, timeout=3, callback=None)\n                for p in still_alive:\n                    p.kill()  # Force kill if still alive after timeout\n                print(f\"Terminating parent process {parent.pid}, {parent.name()}\")\n                handle.process.terminate()  # Terminate the parent process\n                handle.process.wait()  # Wait for the parent process to terminate\n            except psutil.NoSuchProcess:\n                print(f\"Process {handle.process.pid} does not exist anymore.\")\n        else:\n            print(f\"Process {handle.process.pid} is not running\")\n\n        self.run_handles.remove(handle)\n        return True\n\n    def stop_all_llamafiles(self):\n        for handle in self.run_handles:\n            self.stop_llamafile(handle)\n        self.run_handles.clear()\n\n    def llamafile_download_progress(self, name: str):\n        for handle in self.download_handles:\n            if handle.llamafile_name == name:\n                return handle.progress()\n        return None\n"
  },
  {
    "path": "scratch/backend/hub/src/static/index.html",
    "content": "<html><body>hello</body></html>\n"
  },
  {
    "path": "scratch/backend/langserve-demo/.gitignore",
    "content": "openai-api-key\n"
  },
  {
    "path": "scratch/backend/langserve-demo/Dockerfile.cpu",
    "content": "from python:3.11\n\nWORKDIR /usr/src/app\n\nCOPY requirements.txt ./\nRUN pip install --no-cache-dir -r requirements.txt\nCOPY requirements-cpu.txt ./\nRUN pip install --no-cache-dir -r requirements-cpu.txt\n\nCOPY . .\n\nCMD [ \"python3\", \"./serve.py\" ]\n"
  },
  {
    "path": "scratch/backend/langserve-demo/README.md",
    "content": "# Lang Serve Demo\n\nA demo app built with `langchain` and `langserve`.\n\n## Dockerfiles\n\n| Filename                 | Purpose                        |\n|:-------------------------|:-------------------------------|\n| `Dockerfile.cpu`         | Basic setup. CPU support only. |\n\n## Usage\n\nFrom within this directory, build with:\n\n``` sh\ndocker build -f Dockerfile.cpu -t memory-cache/lang-serve-demo-cpu .\n```\n\nSave an API key to a file called `openai-api-key` and run:\n\n``` sh\ndocker run \\\n  --rm \\\n  --name lang-serve-demo-cpu \\\n  -p 8800:8800 \\\n  -e OPENAI_API_KEY=\"$(cat openai-api-key)\" \\\n  -v ./:/usr/src/app/ \\\n  memory-cache/lang-serve-demo-cpu\n```\n\n(Note: I'll remove the OpenAI dependency shortly. It's only here because I'm starting with langchain's demo server.)\n\nThen run a client to interact with the server:\n\n``` sh\ndocker exec lang-serve-demo-cpu python client.py \n```\n\n\n## Miscellaneous Notes\n\n### Dockerfile.cpu: `python 3.11`\n\nWe use `python 3.11` (not `3.12` or later)  in `Dockerfile.cpu` because `faiss-cpu` only supports up to `3.11` at the time of this writing: https://pypi.org/project/faiss-cpu/\n\n\n\n\n"
  },
  {
    "path": "scratch/backend/langserve-demo/client.py",
    "content": "#!/usr/bin/env python3\n\nfrom langserve import RemoteRunnable\n\nremote_chain = RemoteRunnable(\"http://localhost:8800/agent/\")\nresponse = remote_chain.invoke({\n    \"input\": \"Hello, how are you?\",\n    \"chat_history\": []  # Providing an empty list as this is the first call\n})\n\n# Parse json response, then get the 'output' key\nprint(response[\"output\"])\n"
  },
  {
    "path": "scratch/backend/langserve-demo/requirements-cpu.txt",
    "content": "faiss-cpu\n"
  },
  {
    "path": "scratch/backend/langserve-demo/requirements.txt",
    "content": "bs4\nfastapi\nlangchain\nlangchainhub\nlangchain-openai\nlangchain-cli\nlangserve[all]\nuvicorn\n"
  },
  {
    "path": "scratch/backend/langserve-demo/serve.py",
    "content": "#!/usr/bin/env python\nfrom typing import List\n\nfrom fastapi import FastAPI\nfrom langchain_core.prompts import ChatPromptTemplate\nfrom langchain_openai import ChatOpenAI\nfrom langchain_community.document_loaders import WebBaseLoader\nfrom langchain_openai import OpenAIEmbeddings\nfrom langchain_community.vectorstores import FAISS\nfrom langchain.text_splitter import RecursiveCharacterTextSplitter\nfrom langchain.tools.retriever import create_retriever_tool\n# JFS: Remove Tavily Search\n#from langchain_community.tools.tavily_search import TavilySearchResults\nfrom langchain_openai import ChatOpenAI\nfrom langchain import hub\nfrom langchain.agents import create_openai_functions_agent\nfrom langchain.agents import AgentExecutor\nfrom langchain.pydantic_v1 import BaseModel, Field\nfrom langchain_core.messages import BaseMessage\nfrom langserve import add_routes\n\n# 1. Load Retriever\nloader = WebBaseLoader(\"https://docs.smith.langchain.com/overview\")\ndocs = loader.load()\ntext_splitter = RecursiveCharacterTextSplitter()\ndocuments = text_splitter.split_documents(docs)\nembeddings = OpenAIEmbeddings()\nvector = FAISS.from_documents(documents, embeddings)\nretriever = vector.as_retriever()\n\n# 2. Create Tools\nretriever_tool = create_retriever_tool(\n    retriever,\n    \"langsmith_search\",\n    \"Search for information about LangSmith. For any questions about LangSmith, you must use this tool!\",\n)\n# JFS : Remove Tavily Search\n#search = TavilySearchResults()\n#tools = [retriever_tool, search]\ntools = [retriever_tool]\n\n\n\n# 3. Create Agent\nprompt = hub.pull(\"hwchase17/openai-functions-agent\")\nllm = ChatOpenAI(model=\"gpt-3.5-turbo\", temperature=0)\nagent = create_openai_functions_agent(llm, tools, prompt)\nagent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)\n\n\n# 4. App definition\napp = FastAPI(\n  title=\"LangChain Server\",\n  version=\"1.0\",\n  description=\"A simple API server using LangChain's Runnable interfaces\",\n)\n\n# 5. Adding chain route\n\n# We need to add these input/output schemas because the current AgentExecutor\n# is lacking in schemas.\n\nclass Input(BaseModel):\n    input: str\n    chat_history: List[BaseMessage] = Field(\n        ...,\n        extra={\"widget\": {\"type\": \"chat\", \"input\": \"location\"}},\n    )\n\n\nclass Output(BaseModel):\n    output: str\n\nadd_routes(\n    app,\n    agent_executor.with_types(input_type=Input, output_type=Output),\n    path=\"/agent\",\n)\n\nif __name__ == \"__main__\":\n    import uvicorn\n\n    uvicorn.run(app, host=\"0.0.0.0\", port=8800)\n"
  },
  {
    "path": "scratch/backend/python-llamafile-manager/.gitignore",
    "content": "llama.log\nmain.log\nbuild/\ndist/\nbin/\npython-llamafile-manager-gnu-linux.spec\n"
  },
  {
    "path": "scratch/backend/python-llamafile-manager/Dockerfile.plm",
    "content": "from ubuntu:22.04\n\nRUN apt-get update\n\nRUN apt-get install -y python3 python3-pip\n\nWORKDIR /usr/src/app\n\nCOPY requirements.txt ./\nRUN pip install --no-cache-dir -r requirements.txt\n\nRUN apt-get install wget -y\nRUN wget -O /usr/bin/ape https://cosmo.zip/pub/cosmos/bin/ape-$(uname -m).elf\nRUN chmod +x /usr/bin/ape\n# RUN sh -c \"echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register\"\n# RUN sh -c \"echo ':APE-jart:M::jartsr::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register\"\n\nCOPY . .\n\nCMD [ \"python3\", \"./manager.py\" ]\n"
  },
  {
    "path": "scratch/backend/python-llamafile-manager/Dockerfile.plm-builder-gnu-linux",
    "content": "# GNU/Linux\n#\n# PyInstaller requires the ldd terminal application to discover the shared libraries required by each program or shared library. It is typically found in the distribution-package glibc or libc-bin.\n#\n# It also requires the objdump terminal application to extract information from object files and the objcopy terminal application to append data to the bootloader. These are typically found in the distribution-package binutils.\n\nFROM ubuntu:22.04\nRUN apt-get update\nRUN apt-get install -y binutils\nRUN apt-get install -y libc-bin\nRUN apt-get install -y python3 python3-pip\nRUN pip install pyinstaller\nWORKDIR /usr/src/app\nCOPY requirements.txt ./\nRUN pip install --no-cache-dir -r requirements.txt\nCOPY . .\nCMD [ \"python3\", \"./build_gnu_linux.py\" ]\n"
  },
  {
    "path": "scratch/backend/python-llamafile-manager/README.md",
    "content": "# Python Llamafile Manager\n\nA python program that downloads and executes [llamafiles](https://github.com/Mozilla-Ocho/llamafile), bundled with [PyInstaller](https://pyinstaller.org/en/stable/).\n\n## Why?\n\nI wrote a python program using `langchain` that assumes an LLM model is exposed somewhere via HTTP. `Llamafile`s are portable executable that expose an HTTP interface to LLMs. Rather than asking users to download and run `Llamafiles`, I want my python program to manage this on their behalf. I plan to bundle my python program with `PyInstaller`, so I will make sure that `python-llamafile-manager` can be bundled with `PyInstaller` too.\n\n## Usage\n\nFrom within this directory, build with:\n\n``` sh\ndocker build -f Dockerfile.plm -t memory-cache/python-llamafile-manager .\n```\n\nRun with:\n\n``` sh\ndocker run \\\n  --name python-llamafile-manager \\\n  -it \\\n  --rm \\\n  -e LLAMAFILE_BIN_DIR=/usr/src/app/bin \\\n  -v ~/media/llamafile/:/usr/src/app/bin/ \\\n  -v ./:/usr/src/app/ \\\n  -p 8800:8800 \\\n  memory-cache/python-llamafile-manager \\\n  python3 manager.py\n```\n\n## Packaging with PyInstaller\n\n### GNU/Linux\n\n> GNU/Linux\n> \n> PyInstaller requires the ldd terminal application to discover the shared libraries required by each program or shared library. It is typically found in the distribution-package glibc or libc-bin.\n> \n> It also requires the objdump terminal application to extract information from object files and the objcopy terminal application to append data to the bootloader. These are typically found in the distribution-package binutils.\n\n\n``` sh\ndocker build -f Dockerfile.plm-builder-gnu-linux -t memory-cache/plm-builder-gnu-linux .\n```\n\n``` sh\ndocker run \\\n  --name plm-builder-gnu-linux \\\n  -it \\\n  --rm \\\n  -v ./:/usr/src/app/ \\\n  memory-cache/plm-builder-gnu-linux\n```\n\n"
  },
  {
    "path": "scratch/backend/python-llamafile-manager/build_gnu_linux.py",
    "content": "import PyInstaller.__main__\nimport os\n\n# Define the path to the directory containing the llamafiles and other necessary files.\n# Assuming these files are in the same directory as the script for simplicity.\ncurrent_directory = os.path.dirname(os.path.abspath(__file__))\n\n# List of tuples specifying additional files/directories and their destination in the distribution.\n# Adjust paths as necessary.\nadditional_files = [\n    (os.path.join(current_directory, 'requirements.txt'), '.'),\n    # Add other necessary files or directories here.\n    # Example: (os.path.join(current_directory, 'data_folder'), 'data_folder')\n]\n\n# Entry point of the application\nentry_point = os.path.join(current_directory, 'manager.py')\n\n# Build the application with PyInstaller\nPyInstaller.__main__.run([\n    entry_point,\n    '--onefile',  # Bundle everything into a single executable\n    '--add-data=' + ';'.join([f'{src}{os.pathsep}{dst}' for src, dst in additional_files]),  # Add additional files/directories\n    # '--hidden-import=module_name',  # Uncomment and replace with actual module names if there are hidden imports\n    '--clean',  # Clean PyInstaller build folder before building\n    '--name=python-llamafile-manager-gnu-linux',  # Name of the generated executable\n    # Additional flags can be added as needed.\n])\n"
  },
  {
    "path": "scratch/backend/python-llamafile-manager/manager.py",
    "content": "import subprocess\nimport os\nimport stat\nimport requests\nimport sys\nfrom tqdm import tqdm\nfrom time import sleep\n\nurl_llava_v1_5_7b_q4 = \"https://huggingface.co/jartine/llava-v1.5-7B-GGUF/resolve/main/llava-v1.5-7b-q4.llamafile?download=true\"\n\ndef find_llamafiles(directory: str):\n    \"\"\"Check for files with .llamafile extension in the specified directory.\"\"\"\n    return [file for file in os.listdir(directory) if file.endswith('.llamafile')]\n\nprocess = None\n\ndef execute_llamafile(directory: str, filename: str, args: list):\n    \"\"\"Execute a .llamafile as a subprocess with optional arguments.\"\"\"\n    global process\n    filepath = os.path.join(directory, filename)\n\n    # print the file path\n    print(filepath)\n\n    if not os.path.isfile(filepath):\n        raise FileNotFoundError(f\"{filename} not found in {directory}\")\n\n    if not os.access(filepath, os.X_OK):\n        raise PermissionError(f\"{filename} is not executable\")\n\n    print(args)\n    print([filepath] + args)\n\n    process = subprocess.Popen([filepath] + args)\n\ndef is_process_alive():\n    \"\"\"Check if the subprocess is alive.\"\"\"\n    global process\n    return process is not None and process.poll() is None\n\ndef stop_process():\n    \"\"\"Stop the subprocess if it is running.\"\"\"\n    global process\n    if is_process_alive():\n        process.terminate()\n        process.wait()\n\ndef restart_process(directory: str, filename: str, args: list):\n    \"\"\"Restart the .llamafile subprocess with optional arguments.\"\"\"\n    stop_process()\n    execute_llamafile(directory, filename, args)\n\ndef download_file_with_tqdm(url: str, destination: str):\n    \"\"\"Download a file from a URL with a progress bar.\"\"\"\n    response = requests.get(url, stream=True)\n    total_size = int(response.headers.get('content-length', 0))\n    block_size = 1024\n    with open(destination, 'wb') as file:\n        for data in tqdm(response.iter_content(block_size), total=total_size/block_size, unit='KB', unit_scale=True):\n            file.write(data)\n\ndef download_file(url: str, destination: str):\n    \"\"\"Download a file from a URL.\"\"\"\n    response = requests.get(url)\n    with open(destination, 'wb') as file:\n        file.write(response.content)\n\ndef make_executable_unix(destination: str):\n    \"\"\"Mark a file as executable (Unix-like systems).\"\"\"\n    os.chmod(destination, os.stat(destination).st_mode | stat.S_IEXEC)\n\ndef make_executable_windows(destination: str):\n    \"\"\"Windows-specific handling to 'mark' a file as executable is not applicable.\"\"\"\n    pass  # Windows uses file associations to execute files, so no action needed here.\n\ndef download_and_make_executable(url: str, destination: str):\n    \"\"\"Download a file from a URL and mark it as executable.\"\"\"\n    #download_file(url, destination)\n    download_file_with_tqdm(url, destination)\n    if os.name != 'nt':\n        make_executable_unix(destination)\n    else:\n        make_executable_windows(destination)\n\n# Example usage:\nif __name__ == \"__main__\":\n    # Get directory from environment variable or use default\n    directory = os.environ.get('LLAMAFILE_BIN_DIR')\n    if directory is None:\n        # Print error and exit\n        print(\"Error: LLAMAFILE_BIN_DIR environment variable not set\")\n        sys.exit(1)\n\n    llamafiles = find_llamafiles(directory)\n    print(\"Found llamafiles:\", llamafiles)\n    if 'foo.llamafile' not in llamafiles:\n        response = input(\"foo.llamafile not found. Do you want to download foo.llamafile? (y/n): \")\n        if response.lower() == 'y':\n            download_and_make_executable(url_llava_v1_5_7b_q4, os.path.join(directory, 'foo.llamafile'))\n        else:\n            print(\"foo.llamafile not found and not downloaded\")\n            sys.exit(1)\n        llamafiles = find_llamafiles(directory)\n\n    if 'foo.llamafile' not in llamafiles:\n        print(\"Error: foo.llamafile not found\")\n        sys.exit(1)\n\n    execute_llamafile(directory, 'foo.llamafile', ['--host', '0.0.0.0', '--port', '8800'])\n\n    # Check if the process is running every 5 seconds\n    while is_process_alive():\n        print(\"foo.llamafile is running\")\n        sleep(5)\n"
  },
  {
    "path": "scratch/backend/python-llamafile-manager/requirements.txt",
    "content": "requests ~= 2.31\ntqdm\n"
  },
  {
    "path": "scratch/browser-client/.gitignore",
    "content": "build/\nnode_modules/\n"
  },
  {
    "path": "scratch/browser-client/README.md",
    "content": "# Memory Cache Browser Client\n\nA browser client for memory cache.\n\nTested with:\n\n- `node v18.18.2`\n- `npm 10.2.5`\n\nTo install dependencies, run `npm ci`.\n\nTo build the client, run `npm run build`.\n\nThis directory only contains the client-side code. The server is implemented in [a separate `privateGPT` repo](https://github.com/johnshaughnessy/privateGPT).\n\nDesign mockup for phase 1 of interface:\n![phase1-memorycacheui](https://github.com/Mozilla-Ocho/Memory-Cache/assets/100849201/9e0c37fb-9c96-4edd-8db1-1c0ab7f29acb)\n"
  },
  {
    "path": "scratch/browser-client/package.json",
    "content": "{\n  \"name\": \"memory-cache-browser-client\",\n  \"version\": \"1.0.0\",\n  \"description\": \"A browser client for memory cache.\",\n  \"main\": \"main.js\",\n  \"scripts\": {\n    \"build\": \"webpack --mode production\"\n  },\n  \"author\": \"\",\n  \"license\": \"MPL-2.0\",\n  \"devDependencies\": {\n    \"css-loader\": \"^6.8.1\",\n    \"html-webpack-plugin\": \"^5.5.4\",\n    \"style-loader\": \"^3.3.3\",\n    \"webpack\": \"^5.89.0\",\n    \"webpack-cli\": \"^5.1.4\"\n  },\n  \"dependencies\": {\n    \"socket.io-client\": \"^4.7.2\"\n  }\n}\n"
  },
  {
    "path": "scratch/browser-client/src/index.html",
    "content": "<!doctype html>\n<html>\n    <head>\n        <meta charset=\"utf-8\">\n        <meta name=\"description\" content=\"\">\n        <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">\n        <link rel=\"preconnect\" href=\"https://fonts.googleapis.com\">\n        <link rel=\"preconnect\" href=\"https://fonts.gstatic.com\" crossorigin>\n        <link href=\"https://fonts.googleapis.com/css2?family=Work+Sans:wght@300;400;500;600&display=swap\" rel=\"stylesheet\">\n        <title>Kate's Memory Cache</title>\n    </head>\n    <body>\n        <div class=\"mc-main-container\">\n            <div class=\"mc-header\">\n                <div class=\"mc-row\">\n                    <img class=\"mc-logo\" src=\"https://raw.githubusercontent.com/Mozilla-Ocho/Memory-Cache/f406a92a3bf8997697e666e1900916af77f134eb/browser-client/src/img/MC-Logo.png\">\n                    <p> &bull; Kate's Work Cache</p>\n                </div>\n            </div>\n            <div class=\"mc-body-container\">\n                <div class=\"mc-row\">\n                    <div class=\"mc-col mc-query\">\n                        <label for=\"chat-textarea\">What would you like to know?</label>\n                        <textarea class=\"mc-text-field\" id=\"chat-textarea\"></textarea>\n                    </div>\n                    <button id=\"send-button\" class=\"mc-button mc-query-btn\">\n                        <img src=\"https://raw.githubusercontent.com/Mozilla-Ocho/Memory-Cache/f406a92a3bf8997697e666e1900916af77f134eb/browser-client/src/img/MC-Brainprint1.svg\" width=\"16px\"/>\n                        Send query</button>\n                </div>\n                <div class=\"mc-row\">\n                    <div class=\"mc-chat-history\" id=\"chat-history\"></div>\n                </div>\n            </div>\n        </div>\n    </body>\n</html>\n"
  },
  {
    "path": "scratch/browser-client/src/main.js",
    "content": "import { io } from \"socket.io-client\";\nimport \"./styles.css\";\n\nconst socket = io(`http://${window.location.hostname}:5001`);\nsocket.on(\"connect\", () => {\n  console.log(\"connected\");\n});\n\nsocket.on(\"disconnect\", () => {\n  console.log(\"disconnected\");\n});\n\nsocket.on(\"message\", (message) => {\n  console.log(message);\n});\n\nsocket.on(\"error\", (error) => {\n  console.error(error);\n});\n\n// socket.emit(\"message\", JSON.stringify({ text: \"hello world!\" }));\n\nconst chatHistory = document.getElementById(\"chat-history\");\nconst chatTextArea = document.getElementById(\"chat-textarea\");\nconst sendButton = document.getElementById(\"send-button\");\n\nfunction ChatHistoryMessage(message) {\n  const messageElement = document.createElement(\"div\");\n  messageElement.classList.add(\"chat-history-message\");\n  messageElement.innerText = message;\n  return messageElement;\n}\nlet nextChatMessage = ChatHistoryMessage(\"Hello!\");\n// chatHistory.appendChild(nextChatMessage);\n\nconst replies = new Map();\n\nsocket.on(\"message\", (raw) => {\n  console.log(\"Received message from server:\", raw);\n  const message = JSON.parse(raw);\n  console.log(\"message\", message);\n\n  if (message.kind === \"first_reply\") {\n    nextChatMessage = ChatHistoryMessage(\"\");\n    chatHistory.appendChild(nextChatMessage);\n\n    replies.set(message.message_sid, {\n      first: message,\n      chatMessage: nextChatMessage,\n    });\n    nextChatMessage.innerText += message.text;\n  } else if (message.kind === \"second_reply\") {\n    const reply = replies.get(message.message_sid);\n    reply.second = message;\n    // reply.chatMessage.innerText += \"\\n\";\n    // reply.chatMessage.innerText += `\\n${message.text}\\n`;\n    // Send reply and text\n    reply.chatMessage.innerText += `\\n[Replied in ${message.time} seconds.]\\n${message.text}\\n`;\n  } else if (message.kind === \"source_document\") {\n    const { message_sid, kind, text, source } = message;\n    const reply = replies.get(message.message_sid);\n    reply.sourceDocuments = reply.sourceDocuments || [];\n    reply.sourceDocuments.push(message);\n    reply.chatMessage.innerText += `\\n\\n[${source}]\\n${text}\\n`;\n  }\n\n  // We don't scroll to the bottom while the server is sending us messages.\n});\n\nfunction sendChatMessage() {\n  const text = chatTextArea.value;\n  chatTextArea.value = \"\";\n\n  chatHistory.appendChild(ChatHistoryMessage(text));\n\n  // Scroll to the bottom of the chatHistory\n  chatHistory.scrollTop = chatHistory.scrollHeight;\n\n  // Send the message to the server\n  socket.send(\n    JSON.stringify({\n      text,\n    }),\n  ); // This will be sent as a JSON string\n  console.log(\"Message sent to server: \" + text);\n}\n\nlet isShiftPressed = false;\nchatTextArea.addEventListener(\"keydown\", (event) => {\n  if (event.key === \"Shift\") {\n    isShiftPressed = true;\n  }\n});\nchatTextArea.addEventListener(\"keyup\", (event) => {\n  if (event.key === \"Shift\") {\n    isShiftPressed = false;\n  }\n});\nchatTextArea.addEventListener(\"input\", (event) => {\n  if (event.inputType === \"insertLineBreak\" && !isShiftPressed) {\n    sendChatMessage();\n  }\n});\nsendButton.addEventListener(\"click\", sendChatMessage);\n"
  },
  {
    "path": "scratch/browser-client/src/styleguide.html",
    "content": "<!DOCTYPE html>\n\n<html>\n  <head>\n    <meta charset=\"utf-8\">\n    <link rel=\"preconnect\" href=\"https://fonts.googleapis.com\">\n    <link rel=\"preconnect\" href=\"https://fonts.gstatic.com\" crossorigin>\n    <link href=\"https://fonts.googleapis.com/css2?family=Work+Sans:wght@400;500&display=swap\" rel=\"stylesheet\">\n    <link rel=\"stylesheet\" href=\"./styling.css\">\n</head>\n\n<body>\n    <div class=\"body-container\">\n        <div class=\"section\">\n            <h2>Color variables</h2>\n            <div class=\"sub-section\">\n                <h3>Interactions</h3>\n                <div class=\"swatch-group\">\n                    <div class=\"swatch bg-primary\">Primary</div>\n                    <div class=\"swatch bg-primary-hover\">Primary Hover</div>\n                    <div class=\"swatch bg-primary-focus\">Primary Focus</div>\n                </div>\n                <div class=\"swatch-group\">\n                    <div class=\"swatch bg-secondary\">Secondary</div>\n                    <div class=\"swatch bg-secondary-hover\">Secondary Hover</div>\n                    <div class=\"swatch bg-secondary-focus\">Secondary Focus</div>\n                </div>\n                <div class=\"swatch-group\">\n                    <div class=\"swatch bg-interaction-inactive\">Interaction Inactive</div>    \n                </div>\n            </div>\n\n            <div class=\"sub-section\">\n                <h3>Base / Backgrounds</h3>\n                <div class=\"swatch-group\">\n                    <div class=\"swatch bg-base-100\">Base-100</div>\n                    <div class=\"swatch bg-base-200\">Base-200</div>\n                </div>\n            </div>\n            <div class=\"sub-section\">\n                <h3>Messaging</h3>\n                <div class=\"swatch-group\">\n                    <div class=\"swatch bg-info\">Info</div>\n                    <div class=\"swatch bg-success\">Success</div>\n                    <div class=\"swatch bg-warning\">Warning</div>\n                    <div class=\"swatch bg-error\">Error</div>    \n                </div>\n            </div>\n    </div>\n       \n</body>\n\n</html>"
  },
  {
    "path": "scratch/browser-client/src/styles.css",
    "content": ":root {\n  /* Font */\n  --font-family: \"Work Sans\", sans-serif;\n  --font-size-sm: .8em;\n  --font-size-md: 1em;\n  --font-size-lg: 1.6em;\n  --font-weight-normal: 400;\n  --font-weight-bold: 500;\n\n /* Avoiding using representational numbers for the styling system at this point to keep things simple */\n\t--primary: linear-gradient(90deg, #FFB3BB 0%, #FFDFBA 26.56%, #FFFFBA 50.52%, #87EBDA 76.04%, #BAE1FF 100%);\n\t--primary-hover:linear-gradient(90deg, #FFA8B1 0%, #FFD9AD 26.56%, #FFFFAD 50.52%, #80EAD8 76.04%, #ADDCFF 100%);\n\t--primary-focus: linear-gradient(90deg, #FF9EA8 0%, #FFD29E 26.56%, #FFFF9E 50.52%, #73E8D4 76.04%, #99D3FF 100%);\n\t--primary-content: #000000;\n\t--secondary: #FFBF76;\n\t--secondary-hover: #E5AC6A;\n\t--secondary-focus: #CC995E;\n\t--secondary-content: #000000;\n\t--inactive-content:#F9F9F9;\n\t--interaction-inactive:#B6B9BF;\n\t--base-100: #fff;\n\t--base-200: #F0EFEF;\n\t--base-content: #2d3d46;\n\t--base-content-subtle: #9296a0;\n\t--info: #3ac0f8;\n\t--info-content: #000;\n\t--warning: #fcbc23;\n\t--warning-content: #000;\n\t--success: #37d399;\n\t--success-content: #000;\n\t--error: #f87272;\n\t--error-content: #000;\n\t--border-1: #DDD;\n\n\t--border-radius: 4px;\n\n /* Layout spacer utilities */\n\t--xxs:4px;\n\t--xs:8px;\n\t--sm:12px;\n\t--md:16px;\n\t--lg:24px;\n\t--xl:40px;\n}\n\n /* Resets */\nhtml {\n\tpadding:0;\n\tfont-family:var(--font-family);\n}\n\nbody {\n\tmargin:0;\n\tfont-family:var(--font-family);\n\tfont-size: 1em;\n\tbackground: var(--base-200);\n}\n\n/* Layout */\n.mc-main-container {\n\tmargin:auto;\n\t/* background-color: var(--base-100); */\n\tmax-width: 1024px;\n\tdisplay: flex;\n\tflex-direction: column;\n}\n.mc-body-container {\n\tmargin:auto;\n\twidth: 800px;\n\tdisplay: flex;\n\tflex-direction: column;\n\tpadding-top: var(--md);\n}\n.mc-body-section {\n\tmargin-bottom: var(--md);\n\tdisplay: flex;\n\tmargin: var(--sm);\n\tjustify-content: space-between;\n}\n.mc-col {\n\tdisplay:flex;\n\tflex-direction: column;\n\tflex:1;\n}\n.mc-row {\n\tdisplay:flex;\n\tflex-direction: row;\n\tvertical-align: middle;\n\tflex:1;\n}\n.mc-header {\n\twidth:100%;\n\tborder-bottom: .1em solid var(--border-1);\n\tpadding: var(--md) var(--sm);\n\tfont-size:var(--font-size-md);\n}\n.mc-header p {\n\tpadding: 0;\n\tmargin:auto 0;\n}\n.mc-logo {\n\twidth: 150px;\n\theight: fit-content;\n\tmargin-right: var(--sm);\n}\n\n/* Buttons */\n.mc-button {\n    cursor:pointer;\n    border-radius: var(--border-radius);\n    line-height: 21px;\n    letter-spacing: 0em;\n    text-align: center;\n    color: #565D6D;\n    padding:8px;\n\tborder:none;\n}\n.mc-primary-btn {\n    background: var(--primary);\n    margin-bottom:.8em;\n\tdisplay: flex;\n\tpadding: var(--md) var(--lg);\n\tline-height: 24px;\n}\n\n.mc-primary-btn:hover {\n    background:var(--primary-hover);\n}\n.mc-primary-btn:active {\n    background:var(--primary-active);\n}\n\n/* Chat + Text */\n.mc-query {\n\tmargin-right:var(--lg);\n\tmargin-bottom:var(--lg);\n}\nlabel {\n\tmargin-bottom: var(--xs);\n}\ntextarea  {\n\tflex:1;\n\tmax-width: 100%;\n\tpadding: var(--sm);\n\tfont-family: var(--font-family);\n\tfont-size: var(--font-size-md);\n\tborder: 1px solid var(--border-1);\n\tresize: none;\n\tbox-sizing: border-box;\n\tmin-height: 90px;\n\tline-height: 150%;\n}\n\n#chat-textarea:focus-visible {\noutline:2px solid var(--secondary);\n}\n\n.mc-query-btn {\n\tbackground: var(--primary);\n\tdisplay: flex;\n\tpadding: var(--md) var(--lg);\n\tline-height: 24px;\n\tmax-height: 100%;\n\tmax-width: 100%;\n\tmargin: auto 0;\n\tfont-weight: var(--font-weight-bold);\n\tfont-size: .9em;\n\tfont-family: var(--font-family);\n}\n.mc-query-btn img {\n\tmargin-right:var(--sm);\n}\n.mc-chat-history {\n\twidth: 800px;\n\toverflow-y: scroll;\n\tdisplay: flex;\n\tflex-direction: column;\n\tmax-height: 65vh;\n\tpadding: var(--sm);\n}\n\n.mc-chat-history-message {\n\tpadding: 10px;\n\tmargin-bottom: 10px;\n\tborder-radius: 5px;\n\talign-self: flex-start;\n\tborder-radius: 5px;\n\tbackground: var(--base-100);\n}\n"
  },
  {
    "path": "scratch/browser-client/webpack.config.js",
    "content": "const path = require(\"path\");\nconst HtmlWebpackPlugin = require(\"html-webpack-plugin\");\n\nmodule.exports = {\n  entry: \"./src/main.js\",\n  output: {\n    path: path.resolve(__dirname, \"build\"),\n    filename: \"bundle.js\",\n  },\n  module: {\n    rules: [\n      {\n        test: /\\.css$/,\n        use: [\"style-loader\", \"css-loader\"],\n      },\n    ],\n  },\n  plugins: [\n    new HtmlWebpackPlugin({\n      template: \"./src/index.html\",\n    }),\n  ],\n};\n"
  },
  {
    "path": "scratch/hub-browser-client/.gitignore",
    "content": "# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.\n\n# dependencies\n/node_modules\n/.pnp\n.pnp.js\n\n# testing\n/coverage\n\n# production\n/build\n\n# misc\n.DS_Store\n.env.local\n.env.development.local\n.env.test.local\n.env.production.local\n\nnpm-debug.log*\nyarn-debug.log*\nyarn-error.log*\n"
  },
  {
    "path": "scratch/hub-browser-client/README.md",
    "content": "# Memory Cache Hub Browser Client\n\nA browser client for interacting with the memory cache hub.\n"
  },
  {
    "path": "scratch/hub-browser-client/config/env.js",
    "content": "'use strict';\n\nconst fs = require('fs');\nconst path = require('path');\nconst paths = require('./paths');\n\n// Make sure that including paths.js after env.js will read .env variables.\ndelete require.cache[require.resolve('./paths')];\n\nconst NODE_ENV = process.env.NODE_ENV;\nif (!NODE_ENV) {\n  throw new Error(\n    'The NODE_ENV environment variable is required but was not specified.'\n  );\n}\n\n// https://github.com/bkeepers/dotenv#what-other-env-files-can-i-use\nconst dotenvFiles = [\n  `${paths.dotenv}.${NODE_ENV}.local`,\n  // Don't include `.env.local` for `test` environment\n  // since normally you expect tests to produce the same\n  // results for everyone\n  NODE_ENV !== 'test' && `${paths.dotenv}.local`,\n  `${paths.dotenv}.${NODE_ENV}`,\n  paths.dotenv,\n].filter(Boolean);\n\n// Load environment variables from .env* files. Suppress warnings using silent\n// if this file is missing. dotenv will never modify any environment variables\n// that have already been set.  Variable expansion is supported in .env files.\n// https://github.com/motdotla/dotenv\n// https://github.com/motdotla/dotenv-expand\ndotenvFiles.forEach(dotenvFile => {\n  if (fs.existsSync(dotenvFile)) {\n    require('dotenv-expand')(\n      require('dotenv').config({\n        path: dotenvFile,\n      })\n    );\n  }\n});\n\n// We support resolving modules according to `NODE_PATH`.\n// This lets you use absolute paths in imports inside large monorepos:\n// https://github.com/facebook/create-react-app/issues/253.\n// It works similar to `NODE_PATH` in Node itself:\n// https://nodejs.org/api/modules.html#modules_loading_from_the_global_folders\n// Note that unlike in Node, only *relative* paths from `NODE_PATH` are honored.\n// Otherwise, we risk importing Node.js core modules into an app instead of webpack shims.\n// https://github.com/facebook/create-react-app/issues/1023#issuecomment-265344421\n// We also resolve them to make sure all tools using them work consistently.\nconst appDirectory = fs.realpathSync(process.cwd());\nprocess.env.NODE_PATH = (process.env.NODE_PATH || '')\n  .split(path.delimiter)\n  .filter(folder => folder && !path.isAbsolute(folder))\n  .map(folder => path.resolve(appDirectory, folder))\n  .join(path.delimiter);\n\n// Grab NODE_ENV and REACT_APP_* environment variables and prepare them to be\n// injected into the application via DefinePlugin in webpack configuration.\nconst REACT_APP = /^REACT_APP_/i;\n\nfunction getClientEnvironment(publicUrl) {\n  const raw = Object.keys(process.env)\n    .filter(key => REACT_APP.test(key))\n    .reduce(\n      (env, key) => {\n        env[key] = process.env[key];\n        return env;\n      },\n      {\n        // Useful for determining whether we’re running in production mode.\n        // Most importantly, it switches React into the correct mode.\n        NODE_ENV: process.env.NODE_ENV || 'development',\n        // Useful for resolving the correct path to static assets in `public`.\n        // For example, <img src={process.env.PUBLIC_URL + '/img/logo.png'} />.\n        // This should only be used as an escape hatch. Normally you would put\n        // images into the `src` and `import` them in code to get their paths.\n        PUBLIC_URL: publicUrl,\n        // We support configuring the sockjs pathname during development.\n        // These settings let a developer run multiple simultaneous projects.\n        // They are used as the connection `hostname`, `pathname` and `port`\n        // in webpackHotDevClient. They are used as the `sockHost`, `sockPath`\n        // and `sockPort` options in webpack-dev-server.\n        WDS_SOCKET_HOST: process.env.WDS_SOCKET_HOST,\n        WDS_SOCKET_PATH: process.env.WDS_SOCKET_PATH,\n        WDS_SOCKET_PORT: process.env.WDS_SOCKET_PORT,\n        // Whether or not react-refresh is enabled.\n        // It is defined here so it is available in the webpackHotDevClient.\n        FAST_REFRESH: process.env.FAST_REFRESH !== 'false',\n      }\n    );\n  // Stringify all values so we can feed into webpack DefinePlugin\n  const stringified = {\n    'process.env': Object.keys(raw).reduce((env, key) => {\n      env[key] = JSON.stringify(raw[key]);\n      return env;\n    }, {}),\n  };\n\n  return { raw, stringified };\n}\n\nmodule.exports = getClientEnvironment;\n"
  },
  {
    "path": "scratch/hub-browser-client/config/getHttpsConfig.js",
    "content": "'use strict';\n\nconst fs = require('fs');\nconst path = require('path');\nconst crypto = require('crypto');\nconst chalk = require('react-dev-utils/chalk');\nconst paths = require('./paths');\n\n// Ensure the certificate and key provided are valid and if not\n// throw an easy to debug error\nfunction validateKeyAndCerts({ cert, key, keyFile, crtFile }) {\n  let encrypted;\n  try {\n    // publicEncrypt will throw an error with an invalid cert\n    encrypted = crypto.publicEncrypt(cert, Buffer.from('test'));\n  } catch (err) {\n    throw new Error(\n      `The certificate \"${chalk.yellow(crtFile)}\" is invalid.\\n${err.message}`\n    );\n  }\n\n  try {\n    // privateDecrypt will throw an error with an invalid key\n    crypto.privateDecrypt(key, encrypted);\n  } catch (err) {\n    throw new Error(\n      `The certificate key \"${chalk.yellow(keyFile)}\" is invalid.\\n${\n        err.message\n      }`\n    );\n  }\n}\n\n// Read file and throw an error if it doesn't exist\nfunction readEnvFile(file, type) {\n  if (!fs.existsSync(file)) {\n    throw new Error(\n      `You specified ${chalk.cyan(\n        type\n      )} in your env, but the file \"${chalk.yellow(file)}\" can't be found.`\n    );\n  }\n  return fs.readFileSync(file);\n}\n\n// Get the https config\n// Return cert files if provided in env, otherwise just true or false\nfunction getHttpsConfig() {\n  const { SSL_CRT_FILE, SSL_KEY_FILE, HTTPS } = process.env;\n  const isHttps = HTTPS === 'true';\n\n  if (isHttps && SSL_CRT_FILE && SSL_KEY_FILE) {\n    const crtFile = path.resolve(paths.appPath, SSL_CRT_FILE);\n    const keyFile = path.resolve(paths.appPath, SSL_KEY_FILE);\n    const config = {\n      cert: readEnvFile(crtFile, 'SSL_CRT_FILE'),\n      key: readEnvFile(keyFile, 'SSL_KEY_FILE'),\n    };\n\n    validateKeyAndCerts({ ...config, keyFile, crtFile });\n    return config;\n  }\n  return isHttps;\n}\n\nmodule.exports = getHttpsConfig;\n"
  },
  {
    "path": "scratch/hub-browser-client/config/jest/babelTransform.js",
    "content": "'use strict';\n\nconst babelJest = require('babel-jest').default;\n\nconst hasJsxRuntime = (() => {\n  if (process.env.DISABLE_NEW_JSX_TRANSFORM === 'true') {\n    return false;\n  }\n\n  try {\n    require.resolve('react/jsx-runtime');\n    return true;\n  } catch (e) {\n    return false;\n  }\n})();\n\nmodule.exports = babelJest.createTransformer({\n  presets: [\n    [\n      require.resolve('babel-preset-react-app'),\n      {\n        runtime: hasJsxRuntime ? 'automatic' : 'classic',\n      },\n    ],\n  ],\n  babelrc: false,\n  configFile: false,\n});\n"
  },
  {
    "path": "scratch/hub-browser-client/config/jest/cssTransform.js",
    "content": "'use strict';\n\n// This is a custom Jest transformer turning style imports into empty objects.\n// http://facebook.github.io/jest/docs/en/webpack.html\n\nmodule.exports = {\n  process() {\n    return 'module.exports = {};';\n  },\n  getCacheKey() {\n    // The output is always the same.\n    return 'cssTransform';\n  },\n};\n"
  },
  {
    "path": "scratch/hub-browser-client/config/jest/fileTransform.js",
    "content": "'use strict';\n\nconst path = require('path');\nconst camelcase = require('camelcase');\n\n// This is a custom Jest transformer turning file imports into filenames.\n// http://facebook.github.io/jest/docs/en/webpack.html\n\nmodule.exports = {\n  process(src, filename) {\n    const assetFilename = JSON.stringify(path.basename(filename));\n\n    if (filename.match(/\\.svg$/)) {\n      // Based on how SVGR generates a component name:\n      // https://github.com/smooth-code/svgr/blob/01b194cf967347d43d4cbe6b434404731b87cf27/packages/core/src/state.js#L6\n      const pascalCaseFilename = camelcase(path.parse(filename).name, {\n        pascalCase: true,\n      });\n      const componentName = `Svg${pascalCaseFilename}`;\n      return `const React = require('react');\n      module.exports = {\n        __esModule: true,\n        default: ${assetFilename},\n        ReactComponent: React.forwardRef(function ${componentName}(props, ref) {\n          return {\n            $$typeof: Symbol.for('react.element'),\n            type: 'svg',\n            ref: ref,\n            key: null,\n            props: Object.assign({}, props, {\n              children: ${assetFilename}\n            })\n          };\n        }),\n      };`;\n    }\n\n    return `module.exports = ${assetFilename};`;\n  },\n};\n"
  },
  {
    "path": "scratch/hub-browser-client/config/modules.js",
    "content": "'use strict';\n\nconst fs = require('fs');\nconst path = require('path');\nconst paths = require('./paths');\nconst chalk = require('react-dev-utils/chalk');\nconst resolve = require('resolve');\n\n/**\n * Get additional module paths based on the baseUrl of a compilerOptions object.\n *\n * @param {Object} options\n */\nfunction getAdditionalModulePaths(options = {}) {\n  const baseUrl = options.baseUrl;\n\n  if (!baseUrl) {\n    return '';\n  }\n\n  const baseUrlResolved = path.resolve(paths.appPath, baseUrl);\n\n  // We don't need to do anything if `baseUrl` is set to `node_modules`. This is\n  // the default behavior.\n  if (path.relative(paths.appNodeModules, baseUrlResolved) === '') {\n    return null;\n  }\n\n  // Allow the user set the `baseUrl` to `appSrc`.\n  if (path.relative(paths.appSrc, baseUrlResolved) === '') {\n    return [paths.appSrc];\n  }\n\n  // If the path is equal to the root directory we ignore it here.\n  // We don't want to allow importing from the root directly as source files are\n  // not transpiled outside of `src`. We do allow importing them with the\n  // absolute path (e.g. `src/Components/Button.js`) but we set that up with\n  // an alias.\n  if (path.relative(paths.appPath, baseUrlResolved) === '') {\n    return null;\n  }\n\n  // Otherwise, throw an error.\n  throw new Error(\n    chalk.red.bold(\n      \"Your project's `baseUrl` can only be set to `src` or `node_modules`.\" +\n        ' Create React App does not support other values at this time.'\n    )\n  );\n}\n\n/**\n * Get webpack aliases based on the baseUrl of a compilerOptions object.\n *\n * @param {*} options\n */\nfunction getWebpackAliases(options = {}) {\n  const baseUrl = options.baseUrl;\n\n  if (!baseUrl) {\n    return {};\n  }\n\n  const baseUrlResolved = path.resolve(paths.appPath, baseUrl);\n\n  if (path.relative(paths.appPath, baseUrlResolved) === '') {\n    return {\n      src: paths.appSrc,\n    };\n  }\n}\n\n/**\n * Get jest aliases based on the baseUrl of a compilerOptions object.\n *\n * @param {*} options\n */\nfunction getJestAliases(options = {}) {\n  const baseUrl = options.baseUrl;\n\n  if (!baseUrl) {\n    return {};\n  }\n\n  const baseUrlResolved = path.resolve(paths.appPath, baseUrl);\n\n  if (path.relative(paths.appPath, baseUrlResolved) === '') {\n    return {\n      '^src/(.*)$': '<rootDir>/src/$1',\n    };\n  }\n}\n\nfunction getModules() {\n  // Check if TypeScript is setup\n  const hasTsConfig = fs.existsSync(paths.appTsConfig);\n  const hasJsConfig = fs.existsSync(paths.appJsConfig);\n\n  if (hasTsConfig && hasJsConfig) {\n    throw new Error(\n      'You have both a tsconfig.json and a jsconfig.json. If you are using TypeScript please remove your jsconfig.json file.'\n    );\n  }\n\n  let config;\n\n  // If there's a tsconfig.json we assume it's a\n  // TypeScript project and set up the config\n  // based on tsconfig.json\n  if (hasTsConfig) {\n    const ts = require(resolve.sync('typescript', {\n      basedir: paths.appNodeModules,\n    }));\n    config = ts.readConfigFile(paths.appTsConfig, ts.sys.readFile).config;\n    // Otherwise we'll check if there is jsconfig.json\n    // for non TS projects.\n  } else if (hasJsConfig) {\n    config = require(paths.appJsConfig);\n  }\n\n  config = config || {};\n  const options = config.compilerOptions || {};\n\n  const additionalModulePaths = getAdditionalModulePaths(options);\n\n  return {\n    additionalModulePaths: additionalModulePaths,\n    webpackAliases: getWebpackAliases(options),\n    jestAliases: getJestAliases(options),\n    hasTsConfig,\n  };\n}\n\nmodule.exports = getModules();\n"
  },
  {
    "path": "scratch/hub-browser-client/config/paths.js",
    "content": "'use strict';\n\nconst path = require('path');\nconst fs = require('fs');\nconst getPublicUrlOrPath = require('react-dev-utils/getPublicUrlOrPath');\n\n// Make sure any symlinks in the project folder are resolved:\n// https://github.com/facebook/create-react-app/issues/637\nconst appDirectory = fs.realpathSync(process.cwd());\nconst resolveApp = relativePath => path.resolve(appDirectory, relativePath);\n\n// We use `PUBLIC_URL` environment variable or \"homepage\" field to infer\n// \"public path\" at which the app is served.\n// webpack needs to know it to put the right <script> hrefs into HTML even in\n// single-page apps that may serve index.html for nested URLs like /todos/42.\n// We can't use a relative path in HTML because we don't want to load something\n// like /todos/42/static/js/bundle.7289d.js. We have to know the root.\nconst publicUrlOrPath = getPublicUrlOrPath(\n  process.env.NODE_ENV === 'development',\n  require(resolveApp('package.json')).homepage,\n  process.env.PUBLIC_URL\n);\n\nconst buildPath = process.env.BUILD_PATH || 'build';\n\nconst moduleFileExtensions = [\n  'web.mjs',\n  'mjs',\n  'web.js',\n  'js',\n  'web.ts',\n  'ts',\n  'web.tsx',\n  'tsx',\n  'json',\n  'web.jsx',\n  'jsx',\n];\n\n// Resolve file paths in the same order as webpack\nconst resolveModule = (resolveFn, filePath) => {\n  const extension = moduleFileExtensions.find(extension =>\n    fs.existsSync(resolveFn(`${filePath}.${extension}`))\n  );\n\n  if (extension) {\n    return resolveFn(`${filePath}.${extension}`);\n  }\n\n  return resolveFn(`${filePath}.js`);\n};\n\n// config after eject: we're in ./config/\nmodule.exports = {\n  dotenv: resolveApp('.env'),\n  appPath: resolveApp('.'),\n  appBuild: resolveApp(buildPath),\n  appPublic: resolveApp('public'),\n  appHtml: resolveApp('public/index.html'),\n  appIndexJs: resolveModule(resolveApp, 'src/index'),\n  appPackageJson: resolveApp('package.json'),\n  appSrc: resolveApp('src'),\n  appTsConfig: resolveApp('tsconfig.json'),\n  appJsConfig: resolveApp('jsconfig.json'),\n  yarnLockFile: resolveApp('yarn.lock'),\n  testsSetup: resolveModule(resolveApp, 'src/setupTests'),\n  proxySetup: resolveApp('src/setupProxy.js'),\n  appNodeModules: resolveApp('node_modules'),\n  appWebpackCache: resolveApp('node_modules/.cache'),\n  appTsBuildInfoFile: resolveApp('node_modules/.cache/tsconfig.tsbuildinfo'),\n  swSrc: resolveModule(resolveApp, 'src/service-worker'),\n  publicUrlOrPath,\n};\n\n\n\nmodule.exports.moduleFileExtensions = moduleFileExtensions;\n"
  },
  {
    "path": "scratch/hub-browser-client/config/webpack/persistentCache/createEnvironmentHash.js",
    "content": "'use strict';\nconst { createHash } = require('crypto');\n\nmodule.exports = env => {\n  const hash = createHash('md5');\n  hash.update(JSON.stringify(env));\n\n  return hash.digest('hex');\n};\n"
  },
  {
    "path": "scratch/hub-browser-client/config/webpack.config.js",
    "content": "'use strict';\n\nconst fs = require('fs');\nconst path = require('path');\nconst webpack = require('webpack');\nconst resolve = require('resolve');\nconst HtmlWebpackPlugin = require('html-webpack-plugin');\nconst CaseSensitivePathsPlugin = require('case-sensitive-paths-webpack-plugin');\nconst InlineChunkHtmlPlugin = require('react-dev-utils/InlineChunkHtmlPlugin');\nconst TerserPlugin = require('terser-webpack-plugin');\nconst MiniCssExtractPlugin = require('mini-css-extract-plugin');\nconst CssMinimizerPlugin = require('css-minimizer-webpack-plugin');\nconst { WebpackManifestPlugin } = require('webpack-manifest-plugin');\nconst InterpolateHtmlPlugin = require('react-dev-utils/InterpolateHtmlPlugin');\nconst WorkboxWebpackPlugin = require('workbox-webpack-plugin');\nconst ModuleScopePlugin = require('react-dev-utils/ModuleScopePlugin');\nconst getCSSModuleLocalIdent = require('react-dev-utils/getCSSModuleLocalIdent');\nconst ESLintPlugin = require('eslint-webpack-plugin');\nconst paths = require('./paths');\nconst modules = require('./modules');\nconst getClientEnvironment = require('./env');\nconst ModuleNotFoundPlugin = require('react-dev-utils/ModuleNotFoundPlugin');\nconst ForkTsCheckerWebpackPlugin =\n  process.env.TSC_COMPILE_ON_ERROR === 'true'\n    ? require('react-dev-utils/ForkTsCheckerWarningWebpackPlugin')\n    : require('react-dev-utils/ForkTsCheckerWebpackPlugin');\nconst ReactRefreshWebpackPlugin = require('@pmmmwh/react-refresh-webpack-plugin');\n\nconst createEnvironmentHash = require('./webpack/persistentCache/createEnvironmentHash');\n\n// Source maps are resource heavy and can cause out of memory issue for large source files.\nconst shouldUseSourceMap = process.env.GENERATE_SOURCEMAP !== 'false';\n\nconst reactRefreshRuntimeEntry = require.resolve('react-refresh/runtime');\nconst reactRefreshWebpackPluginRuntimeEntry = require.resolve(\n  '@pmmmwh/react-refresh-webpack-plugin'\n);\nconst babelRuntimeEntry = require.resolve('babel-preset-react-app');\nconst babelRuntimeEntryHelpers = require.resolve(\n  '@babel/runtime/helpers/esm/assertThisInitialized',\n  { paths: [babelRuntimeEntry] }\n);\nconst babelRuntimeRegenerator = require.resolve('@babel/runtime/regenerator', {\n  paths: [babelRuntimeEntry],\n});\n\n// Some apps do not need the benefits of saving a web request, so not inlining the chunk\n// makes for a smoother build process.\nconst shouldInlineRuntimeChunk = process.env.INLINE_RUNTIME_CHUNK !== 'false';\n\nconst emitErrorsAsWarnings = process.env.ESLINT_NO_DEV_ERRORS === 'true';\nconst disableESLintPlugin = process.env.DISABLE_ESLINT_PLUGIN === 'true';\n\nconst imageInlineSizeLimit = parseInt(\n  process.env.IMAGE_INLINE_SIZE_LIMIT || '10000'\n);\n\n// Check if TypeScript is setup\nconst useTypeScript = fs.existsSync(paths.appTsConfig);\n\n// Check if Tailwind config exists\nconst useTailwind = fs.existsSync(\n  path.join(paths.appPath, 'tailwind.config.js')\n);\n\n// Get the path to the uncompiled service worker (if it exists).\nconst swSrc = paths.swSrc;\n\n// style files regexes\nconst cssRegex = /\\.css$/;\nconst cssModuleRegex = /\\.module\\.css$/;\nconst sassRegex = /\\.(scss|sass)$/;\nconst sassModuleRegex = /\\.module\\.(scss|sass)$/;\n\nconst hasJsxRuntime = (() => {\n  if (process.env.DISABLE_NEW_JSX_TRANSFORM === 'true') {\n    return false;\n  }\n\n  try {\n    require.resolve('react/jsx-runtime');\n    return true;\n  } catch (e) {\n    return false;\n  }\n})();\n\n// This is the production and development configuration.\n// It is focused on developer experience, fast rebuilds, and a minimal bundle.\nmodule.exports = function (webpackEnv) {\n  const isEnvDevelopment = webpackEnv === 'development';\n  const isEnvProduction = webpackEnv === 'production';\n\n  // Variable used for enabling profiling in Production\n  // passed into alias object. Uses a flag if passed into the build command\n  const isEnvProductionProfile =\n    isEnvProduction && process.argv.includes('--profile');\n\n  // We will provide `paths.publicUrlOrPath` to our app\n  // as %PUBLIC_URL% in `index.html` and `process.env.PUBLIC_URL` in JavaScript.\n  // Omit trailing slash as %PUBLIC_URL%/xyz looks better than %PUBLIC_URL%xyz.\n  // Get environment variables to inject into our app.\n  const env = getClientEnvironment(paths.publicUrlOrPath.slice(0, -1));\n\n  const shouldUseReactRefresh = env.raw.FAST_REFRESH;\n\n  // common function to get style loaders\n  const getStyleLoaders = (cssOptions, preProcessor) => {\n    const loaders = [\n      isEnvDevelopment && require.resolve('style-loader'),\n      isEnvProduction && {\n        loader: MiniCssExtractPlugin.loader,\n        // css is located in `static/css`, use '../../' to locate index.html folder\n        // in production `paths.publicUrlOrPath` can be a relative path\n        options: paths.publicUrlOrPath.startsWith('.')\n          ? { publicPath: '../../' }\n          : {},\n      },\n      {\n        loader: require.resolve('css-loader'),\n        options: cssOptions,\n      },\n      {\n        // Options for PostCSS as we reference these options twice\n        // Adds vendor prefixing based on your specified browser support in\n        // package.json\n        loader: require.resolve('postcss-loader'),\n        options: {\n          postcssOptions: {\n            // Necessary for external CSS imports to work\n            // https://github.com/facebook/create-react-app/issues/2677\n            ident: 'postcss',\n            config: false,\n            plugins: !useTailwind\n              ? [\n                  'postcss-flexbugs-fixes',\n                  [\n                    'postcss-preset-env',\n                    {\n                      autoprefixer: {\n                        flexbox: 'no-2009',\n                      },\n                      stage: 3,\n                    },\n                  ],\n                  // Adds PostCSS Normalize as the reset css with default options,\n                  // so that it honors browserslist config in package.json\n                  // which in turn let's users customize the target behavior as per their needs.\n                  'postcss-normalize',\n                ]\n              : [\n                  'tailwindcss',\n                  'postcss-flexbugs-fixes',\n                  [\n                    'postcss-preset-env',\n                    {\n                      autoprefixer: {\n                        flexbox: 'no-2009',\n                      },\n                      stage: 3,\n                    },\n                  ],\n                ],\n          },\n          sourceMap: isEnvProduction ? shouldUseSourceMap : isEnvDevelopment,\n        },\n      },\n    ].filter(Boolean);\n    if (preProcessor) {\n      loaders.push(\n        {\n          loader: require.resolve('resolve-url-loader'),\n          options: {\n            sourceMap: isEnvProduction ? shouldUseSourceMap : isEnvDevelopment,\n            root: paths.appSrc,\n          },\n        },\n        {\n          loader: require.resolve(preProcessor),\n          options: {\n            sourceMap: true,\n          },\n        }\n      );\n    }\n    return loaders;\n  };\n\n  return {\n    target: ['browserslist'],\n    // Webpack noise constrained to errors and warnings\n    stats: 'errors-warnings',\n    mode: isEnvProduction ? 'production' : isEnvDevelopment && 'development',\n    // Stop compilation early in production\n    bail: isEnvProduction,\n    devtool: isEnvProduction\n      ? shouldUseSourceMap\n        ? 'source-map'\n        : false\n      : isEnvDevelopment && 'cheap-module-source-map',\n    // These are the \"entry points\" to our application.\n    // This means they will be the \"root\" imports that are included in JS bundle.\n    entry: paths.appIndexJs,\n    output: {\n      // The build folder.\n      path: paths.appBuild,\n      // Add /* filename */ comments to generated require()s in the output.\n      pathinfo: isEnvDevelopment,\n      // There will be one main bundle, and one file per asynchronous chunk.\n      // In development, it does not produce real files.\n      filename: isEnvProduction\n        ? 'static/js/[name].[contenthash:8].js'\n        : isEnvDevelopment && 'static/js/bundle.js',\n      // There are also additional JS chunk files if you use code splitting.\n      chunkFilename: isEnvProduction\n        ? 'static/js/[name].[contenthash:8].chunk.js'\n        : isEnvDevelopment && 'static/js/[name].chunk.js',\n      assetModuleFilename: 'static/media/[name].[hash][ext]',\n      // webpack uses `publicPath` to determine where the app is being served from.\n      // It requires a trailing slash, or the file assets will get an incorrect path.\n      // We inferred the \"public path\" (such as / or /my-project) from homepage.\n      publicPath: paths.publicUrlOrPath,\n      // Point sourcemap entries to original disk location (format as URL on Windows)\n      devtoolModuleFilenameTemplate: isEnvProduction\n        ? info =>\n            path\n              .relative(paths.appSrc, info.absoluteResourcePath)\n              .replace(/\\\\/g, '/')\n        : isEnvDevelopment &&\n          (info => path.resolve(info.absoluteResourcePath).replace(/\\\\/g, '/')),\n    },\n    cache: {\n      type: 'filesystem',\n      version: createEnvironmentHash(env.raw),\n      cacheDirectory: paths.appWebpackCache,\n      store: 'pack',\n      buildDependencies: {\n        defaultWebpack: ['webpack/lib/'],\n        config: [__filename],\n        tsconfig: [paths.appTsConfig, paths.appJsConfig].filter(f =>\n          fs.existsSync(f)\n        ),\n      },\n    },\n    infrastructureLogging: {\n      level: 'none',\n    },\n    optimization: {\n      minimize: isEnvProduction,\n      minimizer: [\n        // This is only used in production mode\n        new TerserPlugin({\n          terserOptions: {\n            parse: {\n              // We want terser to parse ecma 8 code. However, we don't want it\n              // to apply any minification steps that turns valid ecma 5 code\n              // into invalid ecma 5 code. This is why the 'compress' and 'output'\n              // sections only apply transformations that are ecma 5 safe\n              // https://github.com/facebook/create-react-app/pull/4234\n              ecma: 8,\n            },\n            compress: {\n              ecma: 5,\n              warnings: false,\n              // Disabled because of an issue with Uglify breaking seemingly valid code:\n              // https://github.com/facebook/create-react-app/issues/2376\n              // Pending further investigation:\n              // https://github.com/mishoo/UglifyJS2/issues/2011\n              comparisons: false,\n              // Disabled because of an issue with Terser breaking valid code:\n              // https://github.com/facebook/create-react-app/issues/5250\n              // Pending further investigation:\n              // https://github.com/terser-js/terser/issues/120\n              inline: 2,\n            },\n            mangle: {\n              safari10: true,\n            },\n            // Added for profiling in devtools\n            keep_classnames: isEnvProductionProfile,\n            keep_fnames: isEnvProductionProfile,\n            output: {\n              ecma: 5,\n              comments: false,\n              // Turned on because emoji and regex is not minified properly using default\n              // https://github.com/facebook/create-react-app/issues/2488\n              ascii_only: true,\n            },\n          },\n        }),\n        // This is only used in production mode\n        new CssMinimizerPlugin(),\n      ],\n    },\n    resolve: {\n      // This allows you to set a fallback for where webpack should look for modules.\n      // We placed these paths second because we want `node_modules` to \"win\"\n      // if there are any conflicts. This matches Node resolution mechanism.\n      // https://github.com/facebook/create-react-app/issues/253\n      modules: ['node_modules', paths.appNodeModules].concat(\n        modules.additionalModulePaths || []\n      ),\n      // These are the reasonable defaults supported by the Node ecosystem.\n      // We also include JSX as a common component filename extension to support\n      // some tools, although we do not recommend using it, see:\n      // https://github.com/facebook/create-react-app/issues/290\n      // `web` extension prefixes have been added for better support\n      // for React Native Web.\n      extensions: paths.moduleFileExtensions\n        .map(ext => `.${ext}`)\n        .filter(ext => useTypeScript || !ext.includes('ts')),\n      alias: {\n        // Support React Native Web\n        // https://www.smashingmagazine.com/2016/08/a-glimpse-into-the-future-with-react-native-for-web/\n        'react-native': 'react-native-web',\n        // Allows for better profiling with ReactDevTools\n        ...(isEnvProductionProfile && {\n          'react-dom$': 'react-dom/profiling',\n          'scheduler/tracing': 'scheduler/tracing-profiling',\n        }),\n        ...(modules.webpackAliases || {}),\n      },\n      plugins: [\n        // Prevents users from importing files from outside of src/ (or node_modules/).\n        // This often causes confusion because we only process files within src/ with babel.\n        // To fix this, we prevent you from importing files out of src/ -- if you'd like to,\n        // please link the files into your node_modules/ and let module-resolution kick in.\n        // Make sure your source files are compiled, as they will not be processed in any way.\n        new ModuleScopePlugin(paths.appSrc, [\n          paths.appPackageJson,\n          reactRefreshRuntimeEntry,\n          reactRefreshWebpackPluginRuntimeEntry,\n          babelRuntimeEntry,\n          babelRuntimeEntryHelpers,\n          babelRuntimeRegenerator,\n        ]),\n      ],\n    },\n    module: {\n      strictExportPresence: true,\n      rules: [\n        // Handle node_modules packages that contain sourcemaps\n        shouldUseSourceMap && {\n          enforce: 'pre',\n          exclude: /@babel(?:\\/|\\\\{1,2})runtime/,\n          test: /\\.(js|mjs|jsx|ts|tsx|css)$/,\n          loader: require.resolve('source-map-loader'),\n        },\n        {\n          // \"oneOf\" will traverse all following loaders until one will\n          // match the requirements. When no loader matches it will fall\n          // back to the \"file\" loader at the end of the loader list.\n          oneOf: [\n            // TODO: Merge this config once `image/avif` is in the mime-db\n            // https://github.com/jshttp/mime-db\n            {\n              test: [/\\.avif$/],\n              type: 'asset',\n              mimetype: 'image/avif',\n              parser: {\n                dataUrlCondition: {\n                  maxSize: imageInlineSizeLimit,\n                },\n              },\n            },\n            // \"url\" loader works like \"file\" loader except that it embeds assets\n            // smaller than specified limit in bytes as data URLs to avoid requests.\n            // A missing `test` is equivalent to a match.\n            {\n              test: [/\\.bmp$/, /\\.gif$/, /\\.jpe?g$/, /\\.png$/],\n              type: 'asset',\n              parser: {\n                dataUrlCondition: {\n                  maxSize: imageInlineSizeLimit,\n                },\n              },\n            },\n            {\n              test: /\\.svg$/,\n              use: [\n                {\n                  loader: require.resolve('@svgr/webpack'),\n                  options: {\n                    prettier: false,\n                    svgo: false,\n                    svgoConfig: {\n                      plugins: [{ removeViewBox: false }],\n                    },\n                    titleProp: true,\n                    ref: true,\n                  },\n                },\n                {\n                  loader: require.resolve('file-loader'),\n                  options: {\n                    name: 'static/media/[name].[hash].[ext]',\n                  },\n                },\n              ],\n              issuer: {\n                and: [/\\.(ts|tsx|js|jsx|md|mdx)$/],\n              },\n            },\n            // Process application JS with Babel.\n            // The preset includes JSX, Flow, TypeScript, and some ESnext features.\n            {\n              test: /\\.(js|mjs|jsx|ts|tsx)$/,\n              include: paths.appSrc,\n              loader: require.resolve('babel-loader'),\n              options: {\n                customize: require.resolve(\n                  'babel-preset-react-app/webpack-overrides'\n                ),\n                presets: [\n                  [\n                    require.resolve('babel-preset-react-app'),\n                    {\n                      runtime: hasJsxRuntime ? 'automatic' : 'classic',\n                    },\n                  ],\n                ],\n                \n                plugins: [\n                  isEnvDevelopment &&\n                    shouldUseReactRefresh &&\n                    require.resolve('react-refresh/babel'),\n                ].filter(Boolean),\n                // This is a feature of `babel-loader` for webpack (not Babel itself).\n                // It enables caching results in ./node_modules/.cache/babel-loader/\n                // directory for faster rebuilds.\n                cacheDirectory: true,\n                // See #6846 for context on why cacheCompression is disabled\n                cacheCompression: false,\n                compact: isEnvProduction,\n              },\n            },\n            // Process any JS outside of the app with Babel.\n            // Unlike the application JS, we only compile the standard ES features.\n            {\n              test: /\\.(js|mjs)$/,\n              exclude: /@babel(?:\\/|\\\\{1,2})runtime/,\n              loader: require.resolve('babel-loader'),\n              options: {\n                babelrc: false,\n                configFile: false,\n                compact: false,\n                presets: [\n                  [\n                    require.resolve('babel-preset-react-app/dependencies'),\n                    { helpers: true },\n                  ],\n                ],\n                cacheDirectory: true,\n                // See #6846 for context on why cacheCompression is disabled\n                cacheCompression: false,\n                \n                // Babel sourcemaps are needed for debugging into node_modules\n                // code.  Without the options below, debuggers like VSCode\n                // show incorrect code and set breakpoints on the wrong lines.\n                sourceMaps: shouldUseSourceMap,\n                inputSourceMap: shouldUseSourceMap,\n              },\n            },\n            // \"postcss\" loader applies autoprefixer to our CSS.\n            // \"css\" loader resolves paths in CSS and adds assets as dependencies.\n            // \"style\" loader turns CSS into JS modules that inject <style> tags.\n            // In production, we use MiniCSSExtractPlugin to extract that CSS\n            // to a file, but in development \"style\" loader enables hot editing\n            // of CSS.\n            // By default we support CSS Modules with the extension .module.css\n            {\n              test: cssRegex,\n              exclude: cssModuleRegex,\n              use: getStyleLoaders({\n                importLoaders: 1,\n                sourceMap: isEnvProduction\n                  ? shouldUseSourceMap\n                  : isEnvDevelopment,\n                modules: {\n                  mode: 'icss',\n                },\n              }),\n              // Don't consider CSS imports dead code even if the\n              // containing package claims to have no side effects.\n              // Remove this when webpack adds a warning or an error for this.\n              // See https://github.com/webpack/webpack/issues/6571\n              sideEffects: true,\n            },\n            // Adds support for CSS Modules (https://github.com/css-modules/css-modules)\n            // using the extension .module.css\n            {\n              test: cssModuleRegex,\n              use: getStyleLoaders({\n                importLoaders: 1,\n                sourceMap: isEnvProduction\n                  ? shouldUseSourceMap\n                  : isEnvDevelopment,\n                modules: {\n                  mode: 'local',\n                  getLocalIdent: getCSSModuleLocalIdent,\n                },\n              }),\n            },\n            // Opt-in support for SASS (using .scss or .sass extensions).\n            // By default we support SASS Modules with the\n            // extensions .module.scss or .module.sass\n            {\n              test: sassRegex,\n              exclude: sassModuleRegex,\n              use: getStyleLoaders(\n                {\n                  importLoaders: 3,\n                  sourceMap: isEnvProduction\n                    ? shouldUseSourceMap\n                    : isEnvDevelopment,\n                  modules: {\n                    mode: 'icss',\n                  },\n                },\n                'sass-loader'\n              ),\n              // Don't consider CSS imports dead code even if the\n              // containing package claims to have no side effects.\n              // Remove this when webpack adds a warning or an error for this.\n              // See https://github.com/webpack/webpack/issues/6571\n              sideEffects: true,\n            },\n            // Adds support for CSS Modules, but using SASS\n            // using the extension .module.scss or .module.sass\n            {\n              test: sassModuleRegex,\n              use: getStyleLoaders(\n                {\n                  importLoaders: 3,\n                  sourceMap: isEnvProduction\n                    ? shouldUseSourceMap\n                    : isEnvDevelopment,\n                  modules: {\n                    mode: 'local',\n                    getLocalIdent: getCSSModuleLocalIdent,\n                  },\n                },\n                'sass-loader'\n              ),\n            },\n            // \"file\" loader makes sure those assets get served by WebpackDevServer.\n            // When you `import` an asset, you get its (virtual) filename.\n            // In production, they would get copied to the `build` folder.\n            // This loader doesn't use a \"test\" so it will catch all modules\n            // that fall through the other loaders.\n            {\n              // Exclude `js` files to keep \"css\" loader working as it injects\n              // its runtime that would otherwise be processed through \"file\" loader.\n              // Also exclude `html` and `json` extensions so they get processed\n              // by webpacks internal loaders.\n              exclude: [/^$/, /\\.(js|mjs|jsx|ts|tsx)$/, /\\.html$/, /\\.json$/],\n              type: 'asset/resource',\n            },\n            // ** STOP ** Are you adding a new loader?\n            // Make sure to add the new loader(s) before the \"file\" loader.\n          ],\n        },\n      ].filter(Boolean),\n    },\n    plugins: [\n      // Generates an `index.html` file with the <script> injected.\n      new HtmlWebpackPlugin(\n        Object.assign(\n          {},\n          {\n            inject: true,\n            template: paths.appHtml,\n          },\n          isEnvProduction\n            ? {\n                minify: {\n                  removeComments: true,\n                  collapseWhitespace: true,\n                  removeRedundantAttributes: true,\n                  useShortDoctype: true,\n                  removeEmptyAttributes: true,\n                  removeStyleLinkTypeAttributes: true,\n                  keepClosingSlash: true,\n                  minifyJS: true,\n                  minifyCSS: true,\n                  minifyURLs: true,\n                },\n              }\n            : undefined\n        )\n      ),\n      // Inlines the webpack runtime script. This script is too small to warrant\n      // a network request.\n      // https://github.com/facebook/create-react-app/issues/5358\n      isEnvProduction &&\n        shouldInlineRuntimeChunk &&\n        new InlineChunkHtmlPlugin(HtmlWebpackPlugin, [/runtime-.+[.]js/]),\n      // Makes some environment variables available in index.html.\n      // The public URL is available as %PUBLIC_URL% in index.html, e.g.:\n      // <link rel=\"icon\" href=\"%PUBLIC_URL%/favicon.ico\">\n      // It will be an empty string unless you specify \"homepage\"\n      // in `package.json`, in which case it will be the pathname of that URL.\n      new InterpolateHtmlPlugin(HtmlWebpackPlugin, env.raw),\n      // This gives some necessary context to module not found errors, such as\n      // the requesting resource.\n      new ModuleNotFoundPlugin(paths.appPath),\n      // Makes some environment variables available to the JS code, for example:\n      // if (process.env.NODE_ENV === 'production') { ... }. See `./env.js`.\n      // It is absolutely essential that NODE_ENV is set to production\n      // during a production build.\n      // Otherwise React will be compiled in the very slow development mode.\n      new webpack.DefinePlugin(env.stringified),\n      // Experimental hot reloading for React .\n      // https://github.com/facebook/react/tree/main/packages/react-refresh\n      isEnvDevelopment &&\n        shouldUseReactRefresh &&\n        new ReactRefreshWebpackPlugin({\n          overlay: false,\n        }),\n      // Watcher doesn't work well if you mistype casing in a path so we use\n      // a plugin that prints an error when you attempt to do this.\n      // See https://github.com/facebook/create-react-app/issues/240\n      isEnvDevelopment && new CaseSensitivePathsPlugin(),\n      isEnvProduction &&\n        new MiniCssExtractPlugin({\n          // Options similar to the same options in webpackOptions.output\n          // both options are optional\n          filename: 'static/css/[name].[contenthash:8].css',\n          chunkFilename: 'static/css/[name].[contenthash:8].chunk.css',\n        }),\n      // Generate an asset manifest file with the following content:\n      // - \"files\" key: Mapping of all asset filenames to their corresponding\n      //   output file so that tools can pick it up without having to parse\n      //   `index.html`\n      // - \"entrypoints\" key: Array of files which are included in `index.html`,\n      //   can be used to reconstruct the HTML if necessary\n      new WebpackManifestPlugin({\n        fileName: 'asset-manifest.json',\n        publicPath: paths.publicUrlOrPath,\n        generate: (seed, files, entrypoints) => {\n          const manifestFiles = files.reduce((manifest, file) => {\n            manifest[file.name] = file.path;\n            return manifest;\n          }, seed);\n          const entrypointFiles = entrypoints.main.filter(\n            fileName => !fileName.endsWith('.map')\n          );\n\n          return {\n            files: manifestFiles,\n            entrypoints: entrypointFiles,\n          };\n        },\n      }),\n      // Moment.js is an extremely popular library that bundles large locale files\n      // by default due to how webpack interprets its code. This is a practical\n      // solution that requires the user to opt into importing specific locales.\n      // https://github.com/jmblog/how-to-optimize-momentjs-with-webpack\n      // You can remove this if you don't use Moment.js:\n      new webpack.IgnorePlugin({\n        resourceRegExp: /^\\.\\/locale$/,\n        contextRegExp: /moment$/,\n      }),\n      // Generate a service worker script that will precache, and keep up to date,\n      // the HTML & assets that are part of the webpack build.\n      isEnvProduction &&\n        fs.existsSync(swSrc) &&\n        new WorkboxWebpackPlugin.InjectManifest({\n          swSrc,\n          dontCacheBustURLsMatching: /\\.[0-9a-f]{8}\\./,\n          exclude: [/\\.map$/, /asset-manifest\\.json$/, /LICENSE/],\n          // Bump up the default maximum size (2mb) that's precached,\n          // to make lazy-loading failure scenarios less likely.\n          // See https://github.com/cra-template/pwa/issues/13#issuecomment-722667270\n          maximumFileSizeToCacheInBytes: 5 * 1024 * 1024,\n        }),\n      // TypeScript type checking\n      useTypeScript &&\n        new ForkTsCheckerWebpackPlugin({\n          async: isEnvDevelopment,\n          typescript: {\n            typescriptPath: resolve.sync('typescript', {\n              basedir: paths.appNodeModules,\n            }),\n            configOverwrite: {\n              compilerOptions: {\n                sourceMap: isEnvProduction\n                  ? shouldUseSourceMap\n                  : isEnvDevelopment,\n                skipLibCheck: true,\n                inlineSourceMap: false,\n                declarationMap: false,\n                noEmit: true,\n                incremental: true,\n                tsBuildInfoFile: paths.appTsBuildInfoFile,\n              },\n            },\n            context: paths.appPath,\n            diagnosticOptions: {\n              syntactic: true,\n            },\n            mode: 'write-references',\n            // profile: true,\n          },\n          issue: {\n            // This one is specifically to match during CI tests,\n            // as micromatch doesn't match\n            // '../cra-template-typescript/template/src/App.tsx'\n            // otherwise.\n            include: [\n              { file: '../**/src/**/*.{ts,tsx}' },\n              { file: '**/src/**/*.{ts,tsx}' },\n            ],\n            exclude: [\n              { file: '**/src/**/__tests__/**' },\n              { file: '**/src/**/?(*.){spec|test}.*' },\n              { file: '**/src/setupProxy.*' },\n              { file: '**/src/setupTests.*' },\n            ],\n          },\n          logger: {\n            infrastructure: 'silent',\n          },\n        }),\n      !disableESLintPlugin &&\n        new ESLintPlugin({\n          // Plugin options\n          extensions: ['js', 'mjs', 'jsx', 'ts', 'tsx'],\n          formatter: require.resolve('react-dev-utils/eslintFormatter'),\n          eslintPath: require.resolve('eslint'),\n          failOnError: !(isEnvDevelopment && emitErrorsAsWarnings),\n          context: paths.appSrc,\n          cache: true,\n          cacheLocation: path.resolve(\n            paths.appNodeModules,\n            '.cache/.eslintcache'\n          ),\n          // ESLint class options\n          cwd: paths.appPath,\n          resolvePluginsRelativeTo: __dirname,\n          baseConfig: {\n            extends: [require.resolve('eslint-config-react-app/base')],\n            rules: {\n              ...(!hasJsxRuntime && {\n                'react/react-in-jsx-scope': 'error',\n              }),\n            },\n          },\n        }),\n    ].filter(Boolean),\n    // Turn off performance processing because we utilize\n    // our own hints via the FileSizeReporter\n    performance: false,\n  };\n};\n"
  },
  {
    "path": "scratch/hub-browser-client/config/webpackDevServer.config.js",
    "content": "'use strict';\n\nconst fs = require('fs');\nconst evalSourceMapMiddleware = require('react-dev-utils/evalSourceMapMiddleware');\nconst noopServiceWorkerMiddleware = require('react-dev-utils/noopServiceWorkerMiddleware');\nconst ignoredFiles = require('react-dev-utils/ignoredFiles');\nconst redirectServedPath = require('react-dev-utils/redirectServedPathMiddleware');\nconst paths = require('./paths');\nconst getHttpsConfig = require('./getHttpsConfig');\n\nconst host = process.env.HOST || '0.0.0.0';\nconst sockHost = process.env.WDS_SOCKET_HOST;\nconst sockPath = process.env.WDS_SOCKET_PATH; // default: '/ws'\nconst sockPort = process.env.WDS_SOCKET_PORT;\n\nmodule.exports = function (proxy, allowedHost) {\n  const disableFirewall =\n    !proxy || process.env.DANGEROUSLY_DISABLE_HOST_CHECK === 'true';\n  return {\n    // WebpackDevServer 2.4.3 introduced a security fix that prevents remote\n    // websites from potentially accessing local content through DNS rebinding:\n    // https://github.com/webpack/webpack-dev-server/issues/887\n    // https://medium.com/webpack/webpack-dev-server-middleware-security-issues-1489d950874a\n    // However, it made several existing use cases such as development in cloud\n    // environment or subdomains in development significantly more complicated:\n    // https://github.com/facebook/create-react-app/issues/2271\n    // https://github.com/facebook/create-react-app/issues/2233\n    // While we're investigating better solutions, for now we will take a\n    // compromise. Since our WDS configuration only serves files in the `public`\n    // folder we won't consider accessing them a vulnerability. However, if you\n    // use the `proxy` feature, it gets more dangerous because it can expose\n    // remote code execution vulnerabilities in backends like Django and Rails.\n    // So we will disable the host check normally, but enable it if you have\n    // specified the `proxy` setting. Finally, we let you override it if you\n    // really know what you're doing with a special environment variable.\n    // Note: [\"localhost\", \".localhost\"] will support subdomains - but we might\n    // want to allow setting the allowedHosts manually for more complex setups\n    allowedHosts: disableFirewall ? 'all' : [allowedHost],\n    headers: {\n      'Access-Control-Allow-Origin': '*',\n      'Access-Control-Allow-Methods': '*',\n      'Access-Control-Allow-Headers': '*',\n    },\n    // Enable gzip compression of generated files.\n    compress: true,\n    static: {\n      // By default WebpackDevServer serves physical files from current directory\n      // in addition to all the virtual build products that it serves from memory.\n      // This is confusing because those files won’t automatically be available in\n      // production build folder unless we copy them. However, copying the whole\n      // project directory is dangerous because we may expose sensitive files.\n      // Instead, we establish a convention that only files in `public` directory\n      // get served. Our build script will copy `public` into the `build` folder.\n      // In `index.html`, you can get URL of `public` folder with %PUBLIC_URL%:\n      // <link rel=\"icon\" href=\"%PUBLIC_URL%/favicon.ico\">\n      // In JavaScript code, you can access it with `process.env.PUBLIC_URL`.\n      // Note that we only recommend to use `public` folder as an escape hatch\n      // for files like `favicon.ico`, `manifest.json`, and libraries that are\n      // for some reason broken when imported through webpack. If you just want to\n      // use an image, put it in `src` and `import` it from JavaScript instead.\n      directory: paths.appPublic,\n      publicPath: [paths.publicUrlOrPath],\n      // By default files from `contentBase` will not trigger a page reload.\n      watch: {\n        // Reportedly, this avoids CPU overload on some systems.\n        // https://github.com/facebook/create-react-app/issues/293\n        // src/node_modules is not ignored to support absolute imports\n        // https://github.com/facebook/create-react-app/issues/1065\n        ignored: ignoredFiles(paths.appSrc),\n      },\n    },\n    client: {\n      webSocketURL: {\n        // Enable custom sockjs pathname for websocket connection to hot reloading server.\n        // Enable custom sockjs hostname, pathname and port for websocket connection\n        // to hot reloading server.\n        hostname: sockHost,\n        pathname: sockPath,\n        port: sockPort,\n      },\n      overlay: {\n        errors: true,\n        warnings: false,\n      },\n    },\n    devMiddleware: {\n      // It is important to tell WebpackDevServer to use the same \"publicPath\" path as\n      // we specified in the webpack config. When homepage is '.', default to serving\n      // from the root.\n      // remove last slash so user can land on `/test` instead of `/test/`\n      publicPath: paths.publicUrlOrPath.slice(0, -1),\n    },\n\n    https: getHttpsConfig(),\n    host,\n    historyApiFallback: {\n      // Paths with dots should still use the history fallback.\n      // See https://github.com/facebook/create-react-app/issues/387.\n      disableDotRule: true,\n      index: paths.publicUrlOrPath,\n    },\n    // `proxy` is run between `before` and `after` `webpack-dev-server` hooks\n    proxy,\n    onBeforeSetupMiddleware(devServer) {\n      // Keep `evalSourceMapMiddleware`\n      // middlewares before `redirectServedPath` otherwise will not have any effect\n      // This lets us fetch source contents from webpack for the error overlay\n      devServer.app.use(evalSourceMapMiddleware(devServer));\n\n      if (fs.existsSync(paths.proxySetup)) {\n        // This registers user provided middleware for proxy reasons\n        require(paths.proxySetup)(devServer.app);\n      }\n    },\n    onAfterSetupMiddleware(devServer) {\n      // Redirect to `PUBLIC_URL` or `homepage` from `package.json` if url not match\n      devServer.app.use(redirectServedPath(paths.publicUrlOrPath));\n\n      // This service worker file is effectively a 'no-op' that will reset any\n      // previous service worker registered for the same host:port combination.\n      // We do this in development to avoid hitting the production cache if\n      // it used the same host and port.\n      // https://github.com/facebook/create-react-app/issues/2272#issuecomment-302832432\n      devServer.app.use(noopServiceWorkerMiddleware(paths.publicUrlOrPath));\n    },\n  };\n};\n"
  },
  {
    "path": "scratch/hub-browser-client/package.json",
    "content": "{\n  \"name\": \"hub-browser-client\",\n  \"version\": \"0.1.0\",\n  \"private\": true,\n  \"dependencies\": {\n    \"@babel/core\": \"^7.16.0\",\n    \"@pmmmwh/react-refresh-webpack-plugin\": \"^0.5.3\",\n    \"@svgr/webpack\": \"^5.5.0\",\n    \"@testing-library/jest-dom\": \"^5.17.0\",\n    \"@testing-library/react\": \"^13.4.0\",\n    \"@testing-library/user-event\": \"^13.5.0\",\n    \"@types/jest\": \"^27.5.2\",\n    \"@types/node\": \"^16.18.83\",\n    \"@types/react\": \"^18.2.58\",\n    \"@types/react-dom\": \"^18.2.19\",\n    \"babel-jest\": \"^27.4.2\",\n    \"babel-loader\": \"^8.2.3\",\n    \"babel-plugin-named-asset-import\": \"^0.3.8\",\n    \"babel-preset-react-app\": \"^10.0.1\",\n    \"bfj\": \"^7.0.2\",\n    \"browserslist\": \"^4.18.1\",\n    \"camelcase\": \"^6.2.1\",\n    \"case-sensitive-paths-webpack-plugin\": \"^2.4.0\",\n    \"css-loader\": \"^6.5.1\",\n    \"css-minimizer-webpack-plugin\": \"^3.2.0\",\n    \"dotenv\": \"^10.0.0\",\n    \"dotenv-expand\": \"^5.1.0\",\n    \"eslint\": \"^8.3.0\",\n    \"eslint-config-react-app\": \"^7.0.1\",\n    \"eslint-webpack-plugin\": \"^3.1.1\",\n    \"file-loader\": \"^6.2.0\",\n    \"fs-extra\": \"^10.0.0\",\n    \"html-webpack-plugin\": \"^5.5.0\",\n    \"identity-obj-proxy\": \"^3.0.0\",\n    \"jest\": \"^27.4.3\",\n    \"jest-resolve\": \"^27.4.2\",\n    \"jest-watch-typeahead\": \"^1.0.0\",\n    \"mini-css-extract-plugin\": \"^2.4.5\",\n    \"postcss\": \"^8.4.4\",\n    \"postcss-flexbugs-fixes\": \"^5.0.2\",\n    \"postcss-loader\": \"^6.2.1\",\n    \"postcss-normalize\": \"^10.0.1\",\n    \"postcss-preset-env\": \"^7.0.1\",\n    \"prompts\": \"^2.4.2\",\n    \"react\": \"^18.2.0\",\n    \"react-app-polyfill\": \"^3.0.0\",\n    \"react-dev-utils\": \"^12.0.1\",\n    \"react-dom\": \"^18.2.0\",\n    \"react-refresh\": \"^0.11.0\",\n    \"resolve\": \"^1.20.0\",\n    \"resolve-url-loader\": \"^4.0.0\",\n    \"sass-loader\": \"^12.3.0\",\n    \"semver\": \"^7.3.5\",\n    \"source-map-loader\": \"^3.0.0\",\n    \"style-loader\": \"^3.3.1\",\n    \"tailwindcss\": \"^3.0.2\",\n    \"terser-webpack-plugin\": \"^5.2.5\",\n    \"typescript\": \"^4.9.5\",\n    \"web-vitals\": \"^2.1.4\",\n    \"webpack\": \"^5.64.4\",\n    \"webpack-dev-server\": \"^4.6.0\",\n    \"webpack-manifest-plugin\": \"^4.0.2\",\n    \"workbox-webpack-plugin\": \"^6.4.1\"\n  },\n  \"scripts\": {\n    \"start\": \"node scripts/start.js\",\n    \"build\": \"node scripts/build.js\",\n    \"test\": \"node scripts/test.js\"\n  },\n  \"eslintConfig\": {\n    \"extends\": [\n      \"react-app\",\n      \"react-app/jest\"\n    ]\n  },\n  \"browserslist\": {\n    \"production\": [\n      \">0.2%\",\n      \"not dead\",\n      \"not op_mini all\"\n    ],\n    \"development\": [\n      \"last 1 chrome version\",\n      \"last 1 firefox version\",\n      \"last 1 safari version\"\n    ]\n  },\n  \"jest\": {\n    \"roots\": [\n      \"<rootDir>/src\"\n    ],\n    \"collectCoverageFrom\": [\n      \"src/**/*.{js,jsx,ts,tsx}\",\n      \"!src/**/*.d.ts\"\n    ],\n    \"setupFiles\": [\n      \"react-app-polyfill/jsdom\"\n    ],\n    \"setupFilesAfterEnv\": [\n      \"<rootDir>/src/setupTests.ts\"\n    ],\n    \"testMatch\": [\n      \"<rootDir>/src/**/__tests__/**/*.{js,jsx,ts,tsx}\",\n      \"<rootDir>/src/**/*.{spec,test}.{js,jsx,ts,tsx}\"\n    ],\n    \"testEnvironment\": \"jsdom\",\n    \"transform\": {\n      \"^.+\\\\.(js|jsx|mjs|cjs|ts|tsx)$\": \"<rootDir>/config/jest/babelTransform.js\",\n      \"^.+\\\\.css$\": \"<rootDir>/config/jest/cssTransform.js\",\n      \"^(?!.*\\\\.(js|jsx|mjs|cjs|ts|tsx|css|json)$)\": \"<rootDir>/config/jest/fileTransform.js\"\n    },\n    \"transformIgnorePatterns\": [\n      \"[/\\\\\\\\]node_modules[/\\\\\\\\].+\\\\.(js|jsx|mjs|cjs|ts|tsx)$\",\n      \"^.+\\\\.module\\\\.(css|sass|scss)$\"\n    ],\n    \"modulePaths\": [],\n    \"moduleNameMapper\": {\n      \"^react-native$\": \"react-native-web\",\n      \"^.+\\\\.module\\\\.(css|sass|scss)$\": \"identity-obj-proxy\"\n    },\n    \"moduleFileExtensions\": [\n      \"web.js\",\n      \"js\",\n      \"web.ts\",\n      \"ts\",\n      \"web.tsx\",\n      \"tsx\",\n      \"json\",\n      \"web.jsx\",\n      \"jsx\",\n      \"node\"\n    ],\n    \"watchPlugins\": [\n      \"jest-watch-typeahead/filename\",\n      \"jest-watch-typeahead/testname\"\n    ],\n    \"resetMocks\": true\n  },\n  \"babel\": {\n    \"presets\": [\n      \"react-app\"\n    ]\n  }\n}\n"
  },
  {
    "path": "scratch/hub-browser-client/public/index.html",
    "content": "<!DOCTYPE html>\n<html lang=\"en\">\n  <head>\n    <meta charset=\"utf-8\" />\n    <link rel=\"icon\" href=\"%PUBLIC_URL%/favicon.ico\" />\n    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\" />\n    <meta name=\"theme-color\" content=\"#000000\" />\n    <meta\n      name=\"description\"\n      content=\"Web site created using create-react-app\"\n    />\n    <link rel=\"apple-touch-icon\" href=\"%PUBLIC_URL%/logo192.png\" />\n    <!--\n      manifest.json provides metadata used when your web app is installed on a\n      user's mobile device or desktop. See https://developers.google.com/web/fundamentals/web-app-manifest/\n    -->\n    <link rel=\"manifest\" href=\"%PUBLIC_URL%/manifest.json\" />\n    <!--\n      Notice the use of %PUBLIC_URL% in the tags above.\n      It will be replaced with the URL of the `public` folder during the build.\n      Only files inside the `public` folder can be referenced from the HTML.\n\n      Unlike \"/favicon.ico\" or \"favicon.ico\", \"%PUBLIC_URL%/favicon.ico\" will\n      work correctly both with client-side routing and a non-root public URL.\n      Learn how to configure a non-root public URL by running `npm run build`.\n    -->\n    <title>React App</title>\n  </head>\n  <body>\n    <noscript>You need to enable JavaScript to run this app.</noscript>\n    <div id=\"root\"></div>\n    <!--\n      This HTML file is a template.\n      If you open it directly in the browser, you will see an empty page.\n\n      You can add webfonts, meta tags, or analytics to this file.\n      The build step will place the bundled scripts into the <body> tag.\n\n      To begin the development, run `npm start` or `yarn start`.\n      To create a production bundle, use `npm run build` or `yarn build`.\n    -->\n  </body>\n</html>\n"
  },
  {
    "path": "scratch/hub-browser-client/public/manifest.json",
    "content": "{\n  \"short_name\": \"React App\",\n  \"name\": \"Create React App Sample\",\n  \"icons\": [\n    {\n      \"src\": \"favicon.ico\",\n      \"sizes\": \"64x64 32x32 24x24 16x16\",\n      \"type\": \"image/x-icon\"\n    },\n    {\n      \"src\": \"logo192.png\",\n      \"type\": \"image/png\",\n      \"sizes\": \"192x192\"\n    },\n    {\n      \"src\": \"logo512.png\",\n      \"type\": \"image/png\",\n      \"sizes\": \"512x512\"\n    }\n  ],\n  \"start_url\": \".\",\n  \"display\": \"standalone\",\n  \"theme_color\": \"#000000\",\n  \"background_color\": \"#ffffff\"\n}\n"
  },
  {
    "path": "scratch/hub-browser-client/public/robots.txt",
    "content": "# https://www.robotstxt.org/robotstxt.html\nUser-agent: *\nDisallow:\n"
  },
  {
    "path": "scratch/hub-browser-client/scripts/build.js",
    "content": "'use strict';\n\n// Do this as the first thing so that any code reading it knows the right env.\nprocess.env.BABEL_ENV = 'production';\nprocess.env.NODE_ENV = 'production';\n\n// Makes the script crash on unhandled rejections instead of silently\n// ignoring them. In the future, promise rejections that are not handled will\n// terminate the Node.js process with a non-zero exit code.\nprocess.on('unhandledRejection', err => {\n  throw err;\n});\n\n// Ensure environment variables are read.\nrequire('../config/env');\n\nconst path = require('path');\nconst chalk = require('react-dev-utils/chalk');\nconst fs = require('fs-extra');\nconst bfj = require('bfj');\nconst webpack = require('webpack');\nconst configFactory = require('../config/webpack.config');\nconst paths = require('../config/paths');\nconst checkRequiredFiles = require('react-dev-utils/checkRequiredFiles');\nconst formatWebpackMessages = require('react-dev-utils/formatWebpackMessages');\nconst printHostingInstructions = require('react-dev-utils/printHostingInstructions');\nconst FileSizeReporter = require('react-dev-utils/FileSizeReporter');\nconst printBuildError = require('react-dev-utils/printBuildError');\n\nconst measureFileSizesBeforeBuild =\n  FileSizeReporter.measureFileSizesBeforeBuild;\nconst printFileSizesAfterBuild = FileSizeReporter.printFileSizesAfterBuild;\nconst useYarn = fs.existsSync(paths.yarnLockFile);\n\n// These sizes are pretty large. We'll warn for bundles exceeding them.\nconst WARN_AFTER_BUNDLE_GZIP_SIZE = 512 * 1024;\nconst WARN_AFTER_CHUNK_GZIP_SIZE = 1024 * 1024;\n\nconst isInteractive = process.stdout.isTTY;\n\n// Warn and crash if required files are missing\nif (!checkRequiredFiles([paths.appHtml, paths.appIndexJs])) {\n  process.exit(1);\n}\n\nconst argv = process.argv.slice(2);\nconst writeStatsJson = argv.indexOf('--stats') !== -1;\n\n// Generate configuration\nconst config = configFactory('production');\n\n// We require that you explicitly set browsers and do not fall back to\n// browserslist defaults.\nconst { checkBrowsers } = require('react-dev-utils/browsersHelper');\ncheckBrowsers(paths.appPath, isInteractive)\n  .then(() => {\n    // First, read the current file sizes in build directory.\n    // This lets us display how much they changed later.\n    return measureFileSizesBeforeBuild(paths.appBuild);\n  })\n  .then(previousFileSizes => {\n    // Remove all content but keep the directory so that\n    // if you're in it, you don't end up in Trash\n    fs.emptyDirSync(paths.appBuild);\n    // Merge with the public folder\n    copyPublicFolder();\n    // Start the webpack build\n    return build(previousFileSizes);\n  })\n  .then(\n    ({ stats, previousFileSizes, warnings }) => {\n      if (warnings.length) {\n        console.log(chalk.yellow('Compiled with warnings.\\n'));\n        console.log(warnings.join('\\n\\n'));\n        console.log(\n          '\\nSearch for the ' +\n            chalk.underline(chalk.yellow('keywords')) +\n            ' to learn more about each warning.'\n        );\n        console.log(\n          'To ignore, add ' +\n            chalk.cyan('// eslint-disable-next-line') +\n            ' to the line before.\\n'\n        );\n      } else {\n        console.log(chalk.green('Compiled successfully.\\n'));\n      }\n\n      console.log('File sizes after gzip:\\n');\n      printFileSizesAfterBuild(\n        stats,\n        previousFileSizes,\n        paths.appBuild,\n        WARN_AFTER_BUNDLE_GZIP_SIZE,\n        WARN_AFTER_CHUNK_GZIP_SIZE\n      );\n      console.log();\n\n      const appPackage = require(paths.appPackageJson);\n      const publicUrl = paths.publicUrlOrPath;\n      const publicPath = config.output.publicPath;\n      const buildFolder = path.relative(process.cwd(), paths.appBuild);\n      printHostingInstructions(\n        appPackage,\n        publicUrl,\n        publicPath,\n        buildFolder,\n        useYarn\n      );\n    },\n    err => {\n      const tscCompileOnError = process.env.TSC_COMPILE_ON_ERROR === 'true';\n      if (tscCompileOnError) {\n        console.log(\n          chalk.yellow(\n            'Compiled with the following type errors (you may want to check these before deploying your app):\\n'\n          )\n        );\n        printBuildError(err);\n      } else {\n        console.log(chalk.red('Failed to compile.\\n'));\n        printBuildError(err);\n        process.exit(1);\n      }\n    }\n  )\n  .catch(err => {\n    if (err && err.message) {\n      console.log(err.message);\n    }\n    process.exit(1);\n  });\n\n// Create the production build and print the deployment instructions.\nfunction build(previousFileSizes) {\n  console.log('Creating an optimized production build...');\n\n  const compiler = webpack(config);\n  return new Promise((resolve, reject) => {\n    compiler.run((err, stats) => {\n      let messages;\n      if (err) {\n        if (!err.message) {\n          return reject(err);\n        }\n\n        let errMessage = err.message;\n\n        // Add additional information for postcss errors\n        if (Object.prototype.hasOwnProperty.call(err, 'postcssNode')) {\n          errMessage +=\n            '\\nCompileError: Begins at CSS selector ' +\n            err['postcssNode'].selector;\n        }\n\n        messages = formatWebpackMessages({\n          errors: [errMessage],\n          warnings: [],\n        });\n      } else {\n        messages = formatWebpackMessages(\n          stats.toJson({ all: false, warnings: true, errors: true })\n        );\n      }\n      if (messages.errors.length) {\n        // Only keep the first error. Others are often indicative\n        // of the same problem, but confuse the reader with noise.\n        if (messages.errors.length > 1) {\n          messages.errors.length = 1;\n        }\n        return reject(new Error(messages.errors.join('\\n\\n')));\n      }\n      if (\n        process.env.CI &&\n        (typeof process.env.CI !== 'string' ||\n          process.env.CI.toLowerCase() !== 'false') &&\n        messages.warnings.length\n      ) {\n        // Ignore sourcemap warnings in CI builds. See #8227 for more info.\n        const filteredWarnings = messages.warnings.filter(\n          w => !/Failed to parse source map/.test(w)\n        );\n        if (filteredWarnings.length) {\n          console.log(\n            chalk.yellow(\n              '\\nTreating warnings as errors because process.env.CI = true.\\n' +\n                'Most CI servers set it automatically.\\n'\n            )\n          );\n          return reject(new Error(filteredWarnings.join('\\n\\n')));\n        }\n      }\n\n      const resolveArgs = {\n        stats,\n        previousFileSizes,\n        warnings: messages.warnings,\n      };\n\n      if (writeStatsJson) {\n        return bfj\n          .write(paths.appBuild + '/bundle-stats.json', stats.toJson())\n          .then(() => resolve(resolveArgs))\n          .catch(error => reject(new Error(error)));\n      }\n\n      return resolve(resolveArgs);\n    });\n  });\n}\n\nfunction copyPublicFolder() {\n  fs.copySync(paths.appPublic, paths.appBuild, {\n    dereference: true,\n    filter: file => file !== paths.appHtml,\n  });\n}\n"
  },
  {
    "path": "scratch/hub-browser-client/scripts/start.js",
    "content": "'use strict';\n\n// Do this as the first thing so that any code reading it knows the right env.\nprocess.env.BABEL_ENV = 'development';\nprocess.env.NODE_ENV = 'development';\n\n// Makes the script crash on unhandled rejections instead of silently\n// ignoring them. In the future, promise rejections that are not handled will\n// terminate the Node.js process with a non-zero exit code.\nprocess.on('unhandledRejection', err => {\n  throw err;\n});\n\n// Ensure environment variables are read.\nrequire('../config/env');\n\nconst fs = require('fs');\nconst chalk = require('react-dev-utils/chalk');\nconst webpack = require('webpack');\nconst WebpackDevServer = require('webpack-dev-server');\nconst clearConsole = require('react-dev-utils/clearConsole');\nconst checkRequiredFiles = require('react-dev-utils/checkRequiredFiles');\nconst {\n  choosePort,\n  createCompiler,\n  prepareProxy,\n  prepareUrls,\n} = require('react-dev-utils/WebpackDevServerUtils');\nconst openBrowser = require('react-dev-utils/openBrowser');\nconst semver = require('semver');\nconst paths = require('../config/paths');\nconst configFactory = require('../config/webpack.config');\nconst createDevServerConfig = require('../config/webpackDevServer.config');\nconst getClientEnvironment = require('../config/env');\nconst react = require(require.resolve('react', { paths: [paths.appPath] }));\n\nconst env = getClientEnvironment(paths.publicUrlOrPath.slice(0, -1));\nconst useYarn = fs.existsSync(paths.yarnLockFile);\nconst isInteractive = process.stdout.isTTY;\n\n// Warn and crash if required files are missing\nif (!checkRequiredFiles([paths.appHtml, paths.appIndexJs])) {\n  process.exit(1);\n}\n\n// Tools like Cloud9 rely on this.\nconst DEFAULT_PORT = parseInt(process.env.PORT, 10) || 3000;\nconst HOST = process.env.HOST || '0.0.0.0';\n\nif (process.env.HOST) {\n  console.log(\n    chalk.cyan(\n      `Attempting to bind to HOST environment variable: ${chalk.yellow(\n        chalk.bold(process.env.HOST)\n      )}`\n    )\n  );\n  console.log(\n    `If this was unintentional, check that you haven't mistakenly set it in your shell.`\n  );\n  console.log(\n    `Learn more here: ${chalk.yellow('https://cra.link/advanced-config')}`\n  );\n  console.log();\n}\n\n// We require that you explicitly set browsers and do not fall back to\n// browserslist defaults.\nconst { checkBrowsers } = require('react-dev-utils/browsersHelper');\ncheckBrowsers(paths.appPath, isInteractive)\n  .then(() => {\n    // We attempt to use the default port but if it is busy, we offer the user to\n    // run on a different port. `choosePort()` Promise resolves to the next free port.\n    return choosePort(HOST, DEFAULT_PORT);\n  })\n  .then(port => {\n    if (port == null) {\n      // We have not found a port.\n      return;\n    }\n\n    const config = configFactory('development');\n    const protocol = process.env.HTTPS === 'true' ? 'https' : 'http';\n    const appName = require(paths.appPackageJson).name;\n\n    const useTypeScript = fs.existsSync(paths.appTsConfig);\n    const urls = prepareUrls(\n      protocol,\n      HOST,\n      port,\n      paths.publicUrlOrPath.slice(0, -1)\n    );\n    // Create a webpack compiler that is configured with custom messages.\n    const compiler = createCompiler({\n      appName,\n      config,\n      urls,\n      useYarn,\n      useTypeScript,\n      webpack,\n    });\n    // Load proxy config\n    const proxySetting = require(paths.appPackageJson).proxy;\n    const proxyConfig = prepareProxy(\n      proxySetting,\n      paths.appPublic,\n      paths.publicUrlOrPath\n    );\n    // Serve webpack assets generated by the compiler over a web server.\n    const serverConfig = {\n      ...createDevServerConfig(proxyConfig, urls.lanUrlForConfig),\n      host: HOST,\n      port,\n    };\n    const devServer = new WebpackDevServer(serverConfig, compiler);\n    // Launch WebpackDevServer.\n    devServer.startCallback(() => {\n      if (isInteractive) {\n        clearConsole();\n      }\n\n      if (env.raw.FAST_REFRESH && semver.lt(react.version, '16.10.0')) {\n        console.log(\n          chalk.yellow(\n            `Fast Refresh requires React 16.10 or higher. You are using React ${react.version}.`\n          )\n        );\n      }\n\n      console.log(chalk.cyan('Starting the development server...\\n'));\n      openBrowser(urls.localUrlForBrowser);\n    });\n\n    ['SIGINT', 'SIGTERM'].forEach(function (sig) {\n      process.on(sig, function () {\n        devServer.close();\n        process.exit();\n      });\n    });\n\n    if (process.env.CI !== 'true') {\n      // Gracefully exit when stdin ends\n      process.stdin.on('end', function () {\n        devServer.close();\n        process.exit();\n      });\n    }\n  })\n  .catch(err => {\n    if (err && err.message) {\n      console.log(err.message);\n    }\n    process.exit(1);\n  });\n"
  },
  {
    "path": "scratch/hub-browser-client/scripts/test.js",
    "content": "'use strict';\n\n// Do this as the first thing so that any code reading it knows the right env.\nprocess.env.BABEL_ENV = 'test';\nprocess.env.NODE_ENV = 'test';\nprocess.env.PUBLIC_URL = '';\n\n// Makes the script crash on unhandled rejections instead of silently\n// ignoring them. In the future, promise rejections that are not handled will\n// terminate the Node.js process with a non-zero exit code.\nprocess.on('unhandledRejection', err => {\n  throw err;\n});\n\n// Ensure environment variables are read.\nrequire('../config/env');\n\nconst jest = require('jest');\nconst execSync = require('child_process').execSync;\nlet argv = process.argv.slice(2);\n\nfunction isInGitRepository() {\n  try {\n    execSync('git rev-parse --is-inside-work-tree', { stdio: 'ignore' });\n    return true;\n  } catch (e) {\n    return false;\n  }\n}\n\nfunction isInMercurialRepository() {\n  try {\n    execSync('hg --cwd . root', { stdio: 'ignore' });\n    return true;\n  } catch (e) {\n    return false;\n  }\n}\n\n// Watch unless on CI or explicitly running all tests\nif (\n  !process.env.CI &&\n  argv.indexOf('--watchAll') === -1 &&\n  argv.indexOf('--watchAll=false') === -1\n) {\n  // https://github.com/facebook/create-react-app/issues/5210\n  const hasSourceControl = isInGitRepository() || isInMercurialRepository();\n  argv.push(hasSourceControl ? '--watch' : '--watchAll');\n}\n\n\njest.run(argv);\n"
  },
  {
    "path": "scratch/hub-browser-client/src/App.css",
    "content": ".App {\n  text-align: center;\n}\n\n.App-logo {\n  height: 40vmin;\n  pointer-events: none;\n}\n\n@media (prefers-reduced-motion: no-preference) {\n  .App-logo {\n    animation: App-logo-spin infinite 20s linear;\n  }\n}\n\n.App-header {\n  background-color: #282c34;\n  min-height: 100vh;\n  display: flex;\n  flex-direction: column;\n  align-items: center;\n  justify-content: center;\n  font-size: calc(10px + 2vmin);\n  color: white;\n}\n\n.App-link {\n  color: #61dafb;\n}\n\n@keyframes App-logo-spin {\n  from {\n    transform: rotate(0deg);\n  }\n  to {\n    transform: rotate(360deg);\n  }\n}\n"
  },
  {
    "path": "scratch/hub-browser-client/src/App.test.tsx",
    "content": "import React from 'react';\nimport { render, screen } from '@testing-library/react';\nimport App from './App';\n\ntest('renders learn react link', () => {\n  render(<App />);\n  const linkElement = screen.getByText(/learn react/i);\n  expect(linkElement).toBeInTheDocument();\n});\n"
  },
  {
    "path": "scratch/hub-browser-client/src/App.tsx",
    "content": "import { useState, useEffect } from \"react\";\nimport \"./App.css\";\nimport { listLlamafiles } from \"./api/llamafile_api\";\nimport { ILlamafile } from \"./types\";\nimport { LlamafileDetails } from \"./components/llamafile_details\";\n\nfunction App() {\n  const [llamafiles, setLlamafiles] = useState<ILlamafile[]>([]);\n\n  useEffect(() => {\n    listLlamafiles().then(setLlamafiles);\n\n    const intervalId = setInterval(() => {\n      listLlamafiles().then(setLlamafiles);\n    }, 1000);\n\n    return () => clearInterval(intervalId);\n  }, []);\n\n  return (\n    <div className=\"App\">\n      {llamafiles.map((llamafile, index) => (\n        <div key={index}>\n          <LlamafileDetails key={llamafile.name} llamafile={llamafile} />\n        </div>\n      ))}\n    </div>\n  );\n}\n\nexport default App;\n"
  },
  {
    "path": "scratch/hub-browser-client/src/api/llamafile_api.ts",
    "content": "import {\n  ILlamafile,\n  IDownloadLlamafileRequest,\n  IDownloadLlamafileResponse,\n  IRunLlamafileRequest,\n  IRunLlamafileResponse,\n  IStopLlamafileRequest,\n  IStopLlamafileResponse,\n} from \"../types\";\n\nconst base = window.location;\nconst origin = `${base.protocol}//${base.hostname}`;\nconst port = 8001;\nconst llamafileApi = `${origin}:${port}/api/llamafile`;\n\n// List llamafiles\nexport async function listLlamafiles(): Promise<ILlamafile[]> {\n  const response = await fetch(`${llamafileApi}/list_llamafiles`);\n  const data = await response.json();\n  return data.llamafiles;\n}\n\n// Download llamafile\nexport async function downloadLlamafile(\n  name: string,\n): Promise<IDownloadLlamafileResponse> {\n  const request: IDownloadLlamafileRequest = { name };\n  const response = await fetch(`${llamafileApi}/download_llamafile`, {\n    method: \"POST\",\n    headers: {\n      \"Content-Type\": \"application/json\",\n    },\n    body: JSON.stringify(request),\n  });\n  const data = await response.json();\n  return data;\n}\n\n// Run llamafile\nexport async function runLlamafile(\n  name: string,\n): Promise<IRunLlamafileResponse> {\n  const request: IRunLlamafileRequest = { name };\n  const response = await fetch(`${llamafileApi}/run_llamafile`, {\n    method: \"POST\",\n    headers: {\n      \"Content-Type\": \"application/json\",\n    },\n    body: JSON.stringify(request),\n  });\n  const data = await response.json();\n  return data;\n}\n\n// Stop llamafile\nexport async function stopLlamafile(\n  name: string,\n): Promise<IStopLlamafileResponse> {\n  const request: IStopLlamafileRequest = { name };\n  const response = await fetch(`${llamafileApi}/stop_llamafile`, {\n    method: \"POST\",\n    headers: {\n      \"Content-Type\": \"application/json\",\n    },\n    body: JSON.stringify(request),\n  });\n  const data = await response.json();\n  return data;\n}\n"
  },
  {
    "path": "scratch/hub-browser-client/src/components/llamafile_details.tsx",
    "content": "import React, { useState } from \"react\";\nimport { ILlamafile } from \"../types\";\nimport {\n  downloadLlamafile,\n  runLlamafile,\n  stopLlamafile,\n} from \"../api/llamafile_api\";\n\nexport const LlamafileDetails: React.FC<{ llamafile: ILlamafile }> = ({\n  llamafile,\n}) => {\n  const [downloadButtonText, setDownloadButtonText] = useState(\"Download\");\n  return (\n    <div\n      style={{\n        margin: \"10px 0\",\n        padding: \"10px\",\n        border: \"1px solid #ccc\",\n        borderRadius: \"5px\",\n      }}\n    >\n      <h3>{llamafile.name}</h3>\n      <p>\n        URL:{\" \"}\n        <a href={llamafile.url} target=\"_blank\" rel=\"noopener noreferrer\">\n          {llamafile.url}\n        </a>\n      </p>\n      <p>Downloaded: {llamafile.downloaded ? \"Yes\" : \"No\"}</p>\n      <p>Running: {llamafile.running ? \"Yes\" : \"No\"}</p>\n      <p>Download progress: {llamafile.download_progress}</p>\n      <button\n        onClick={() => {\n          downloadLlamafile(llamafile.name).then((result) => {\n            setDownloadButtonText(\n              `Download ${result.success ? \"Started\" : \"Failed\"}`,\n            );\n          });\n        }}\n        disabled={downloadButtonText.includes(\"Started\")}\n      >\n        {downloadButtonText}\n      </button>\n      <button\n        onClick={() => {\n          runLlamafile(llamafile.name);\n        }}\n        disabled={llamafile.running}\n      >\n        {`${llamafile.running ? \"Running\" : \"Run\"}`}\n      </button>\n      <button\n        onClick={() => {\n          stopLlamafile(llamafile.name);\n        }}\n        disabled={!llamafile.running}\n      >\n        {\"Stop\"}\n      </button>\n    </div>\n  );\n};\n"
  },
  {
    "path": "scratch/hub-browser-client/src/index.css",
    "content": "body {\n  margin: 0;\n  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen',\n    'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue',\n    sans-serif;\n  -webkit-font-smoothing: antialiased;\n  -moz-osx-font-smoothing: grayscale;\n}\n\ncode {\n  font-family: source-code-pro, Menlo, Monaco, Consolas, 'Courier New',\n    monospace;\n}\n"
  },
  {
    "path": "scratch/hub-browser-client/src/index.tsx",
    "content": "import React from 'react';\nimport ReactDOM from 'react-dom/client';\nimport './index.css';\nimport App from './App';\nimport reportWebVitals from './reportWebVitals';\n\nconst root = ReactDOM.createRoot(\n  document.getElementById('root') as HTMLElement\n);\nroot.render(\n  <React.StrictMode>\n    <App />\n  </React.StrictMode>\n);\n\n// If you want to start measuring performance in your app, pass a function\n// to log results (for example: reportWebVitals(console.log))\n// or send to an analytics endpoint. Learn more: https://bit.ly/CRA-vitals\nreportWebVitals();\n"
  },
  {
    "path": "scratch/hub-browser-client/src/react-app-env.d.ts",
    "content": "/// <reference types=\"node\" />\n/// <reference types=\"react\" />\n/// <reference types=\"react-dom\" />\n\ndeclare namespace NodeJS {\n  interface ProcessEnv {\n    readonly NODE_ENV: 'development' | 'production' | 'test';\n    readonly PUBLIC_URL: string;\n  }\n}\n\ndeclare module '*.avif' {\n  const src: string;\n  export default src;\n}\n\ndeclare module '*.bmp' {\n  const src: string;\n  export default src;\n}\n\ndeclare module '*.gif' {\n  const src: string;\n  export default src;\n}\n\ndeclare module '*.jpg' {\n  const src: string;\n  export default src;\n}\n\ndeclare module '*.jpeg' {\n  const src: string;\n  export default src;\n}\n\ndeclare module '*.png' {\n  const src: string;\n  export default src;\n}\n\ndeclare module '*.webp' {\n    const src: string;\n    export default src;\n}\n\ndeclare module '*.svg' {\n  import * as React from 'react';\n\n  export const ReactComponent: React.FunctionComponent<React.SVGProps<\n    SVGSVGElement\n  > & { title?: string }>;\n\n  const src: string;\n  export default src;\n}\n\ndeclare module '*.module.css' {\n  const classes: { readonly [key: string]: string };\n  export default classes;\n}\n\ndeclare module '*.module.scss' {\n  const classes: { readonly [key: string]: string };\n  export default classes;\n}\n\ndeclare module '*.module.sass' {\n  const classes: { readonly [key: string]: string };\n  export default classes;\n}\n"
  },
  {
    "path": "scratch/hub-browser-client/src/reportWebVitals.ts",
    "content": "import { ReportHandler } from 'web-vitals';\n\nconst reportWebVitals = (onPerfEntry?: ReportHandler) => {\n  if (onPerfEntry && onPerfEntry instanceof Function) {\n    import('web-vitals').then(({ getCLS, getFID, getFCP, getLCP, getTTFB }) => {\n      getCLS(onPerfEntry);\n      getFID(onPerfEntry);\n      getFCP(onPerfEntry);\n      getLCP(onPerfEntry);\n      getTTFB(onPerfEntry);\n    });\n  }\n};\n\nexport default reportWebVitals;\n"
  },
  {
    "path": "scratch/hub-browser-client/src/setupTests.ts",
    "content": "// jest-dom adds custom jest matchers for asserting on DOM nodes.\n// allows you to do things like:\n// expect(element).toHaveTextContent(/react/i)\n// learn more: https://github.com/testing-library/jest-dom\nimport '@testing-library/jest-dom';\n"
  },
  {
    "path": "scratch/hub-browser-client/src/types.ts",
    "content": "export interface ILlamafile {\n  name: string;\n  url: string;\n  downloaded: boolean;\n  running: boolean;\n  download_progress: number | null;\n}\n\nexport interface IDownloadLlamafileRequest {\n  name: string;\n}\nexport interface IDownloadLlamafileResponse {\n  success: boolean;\n}\n\nexport interface IRunLlamafileRequest {\n  name: string;\n}\nexport interface IRunLlamafileResponse {\n  success: boolean;\n}\n\nexport interface IStopLlamafileRequest {\n  name: string;\n}\nexport interface IStopLlamafileResponse {\n  success: boolean;\n}\n"
  },
  {
    "path": "scratch/hub-browser-client/tsconfig.json",
    "content": "{\n  \"compilerOptions\": {\n    \"target\": \"es5\",\n    \"lib\": [\n      \"dom\",\n      \"dom.iterable\",\n      \"esnext\"\n    ],\n    \"allowJs\": true,\n    \"skipLibCheck\": true,\n    \"esModuleInterop\": true,\n    \"allowSyntheticDefaultImports\": true,\n    \"strict\": true,\n    \"forceConsistentCasingInFileNames\": true,\n    \"noFallthroughCasesInSwitch\": true,\n    \"module\": \"esnext\",\n    \"moduleResolution\": \"node\",\n    \"resolveJsonModule\": true,\n    \"isolatedModules\": true,\n    \"noEmit\": true,\n    \"jsx\": \"react-jsx\"\n  },\n  \"include\": [\n    \"src\"\n  ]\n}\n"
  },
  {
    "path": "scripts/run_ingest.sh",
    "content": "#! /bin/bash\n\nDIR=\"$HOME/Downloads/MemoryCache\"\ninotifywait -m -e close_write -e moved_to \"$DIR\" | while read path action file; do\n    fullPath=\"$path$file\"\n    fileSize=$(stat -c%s \"$fullPath\")\n    if [[ \"$file\" != *.part ]] && [[ $fileSize -gt 0 ]]; then\n        echo \"Ingest triggered by '$file' ($fileSize bytes).\"\n        python3 ingest.py\n    fi\ndone\n"
  }
]