Repository: hamzafarooq/advanced-llms-course Branch: main Commit: 341a76c30069 Files: 8 Total size: 228.8 KB Directory structure: gitextract_hvalzlf_/ ├── LICENSE ├── README.md ├── conversational search/ │ ├── .ipynb_checkpoints/ │ │ └── conversational_search_ares_api-checkpoint.ipynb │ ├── app.py │ ├── conversational_search_ares_api.ipynb │ └── travergo.md └── semantic cache/ ├── readme.md └── semantic_cache_from_scratch.ipynb ================================================ FILE CONTENTS ================================================ ================================================ FILE: LICENSE ================================================ Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright 2024 Hamza Farooq Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. ================================================ FILE: README.md ================================================ # advanced-llms Welcome to the comprehensive course on advancing your skills in building sophisticated Large Language Model (LLM) applications! Details about the course can be found here: https://maven.com/boring-bot/advanced-llm We have tried to build the most advanced LLM course currently being offered in the world. No pun intended. If you have already acquired knowledge about RAG, cosine similarity, vector databases, and Langchain, it's time to delve into the practical aspects of packaging and deploying these models in production environments. This course builds upon the fundamental building blocks of LLMs and covers the following key topics: 1. Fine-tuning: Learn advanced techniques for fine-tuning LLMs (ChatGPT and Open-source LLMs) to enhance their performance and adapt them to specific tasks or domains. 2. Model merging: Explore methods to merge multiple models, optimizing their collective capabilities for more robust and versatile language processing. 3. Inference speed exploration: Understand strategies to optimize and accelerate inference speeds, ensuring efficient real-time processing of language model outputs. 4. Quantization methods: Dive into techniques for model quantization, reducing model size while maintaining performance, crucial for deployment in resource-constrained environments. 5. Model hosting and deployments: Gain insights into best practices for hosting and deploying LLMs in production settings, ensuring seamless integration into diverse applications. 6. Semantic Caching: Learn how to build it all from scratch and implement it with GCP and REDIS 7. Guardrail and DSPy: Implement State of the Art Guardrail and learn how you can build applications with minimal prompting Throughout the course, we will analyze state-of-the-art AI products, reverse-engineering some through Python. Additionally, my collaboration with experienced Software Engineers on our team will provide valuable insights into integrating LLMs with Node.js for web application development. As a bonus, you'll have access to experimental products being developed at Traversaal.ai, my startup, allowing you to stay at the forefront of cutting-edge advancements in the field. Prerequisites for this course include proficiency in Python and a solid understanding of RAGs, as well as Encoder and Decoder models. If you feel the need for a more foundational course, consider checking out my other offering on LLMs: https://maven.com/boring-bot/ml-system-design Tools utilized in this course include VS Code, UNIX terminal, Jupyter Notebooks, and Conda package management, ensuring a hands-on and practical learning experience. ================================================ FILE: conversational search/.ipynb_checkpoints/conversational_search_ares_api-checkpoint.ipynb ================================================ { "cells": [ { "cell_type": "code", "execution_count": 4, "id": "4d1d583c-ff72-4f04-b15e-3a5e7a7a075d", "metadata": {}, "outputs": [], "source": [ "from openai import OpenAI\n", "import requests\n", "import streamlit as st\n", "from qdrant_client import QdrantClient\n", "\n", "from sentence_transformers import SentenceTransformer" ] }, { "cell_type": "code", "execution_count": 7, "id": "c1f28558-9957-4156-a69b-7b952351c5e7", "metadata": {}, "outputs": [], "source": [ "def getResponse(query):\n", " url = \"https://api-ares.traversaal.ai/live/predict\"\n", " payload = { \"query\": [query] }\n", " headers = {\n", " \"x-api-key\": st.secrets[\"TRAVERSAAL\"],\n", " \"content-type\": \"application/json\"\n", " }\n", "\n", " response = requests.post(url, json=payload, headers=headers)\n", " if response.status_code == 200:\n", " # Get the JSON content from the response\n", " json_content = response.json()\n", "\n", " # Specify the file path where you want to save the JSON content\n", " return json_content\n", " else:\n", " print(response.status_code)\n", " return \" \"" ] }, { "cell_type": "code", "execution_count": 63, "id": "02856991-8745-491f-9b9d-bd31e7ccc9bd", "metadata": {}, "outputs": [], "source": [ "def ares_api(query):\n", " response_json = getResponse(query);\n", " return (response_json['data']['response_text'])\n" ] }, { "cell_type": "code", "execution_count": 58, "id": "d36e7d39-1175-4099-9093-f4e1fea21804", "metadata": {}, "outputs": [], "source": [ "def canAnswer(description, q):\n", " client = initializeClient();\n", "\n", " prompt = f\"\"\"\n", " \\\"{description}\\\"\n", " \\n\n", " is there information about the following in the above text, make sure you will be able to enaswer the following question prcisely: {q}\n", " \\n\n", " answer in one word, \"yes\" or \"no\"\n", " \"\"\"\n", " stream = client.chat.completions.create(\n", " model=\"gpt-4\",\n", " messages=[{\"role\": \"user\", \"content\": prompt}],\n", " stream=True,\n", " )\n", " strr = \"\"\n", " for chunk in stream:\n", " if chunk.choices[0].delta.content is not None:\n", " strr += (chunk.choices[0].delta.content)\n", " return strr.lower() == \"yes\";\n", " \n" ] }, { "cell_type": "code", "execution_count": 60, "id": "933f1403-5336-4468-8890-4c0c409619c5", "metadata": {}, "outputs": [], "source": [ "sample_description = \"\"\"\n", " In the east of Paris, the hotel Campanile Bercy boasts an ideal location for exploring The City of\n", " LightsNature-lovers staying at Campanile Bercy will enjoy its privileged vicinity,taking walks in \n", " Bercy Park and its four gardens, strolling around the village \"\"Cour Saint Emilion\"\" as well as \n", " trying restaurants and cafes.\n", "\"\"\"" ] }, { "cell_type": "code", "execution_count": 68, "id": "8f7859db-f6fd-4c81-bc08-3f9851135b96", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "GOING TO TRAVERSE API\n", "The best restaurants near Hotel Campanile Bercy in Paris are Sushi Yuki, Fenetre sur Cour, L'Auberge Aveyronnaise, The Frog at Bercy Village, Pedra Alta, Fresh'Heure, Le Midnight Paris, and Zendo Sushi Restaurant. You can find more information [here](https://www.tripadvisor.com/RestaurantsNear-g187147-d233766-Hotel_Campanile_Paris_Bercy_Village-Paris_Ile_de_France.html).\n", "RECEIVED INFORMATION FROM ARES API\n" ] } ], "source": [ "prompt = \"whats are some good restaurants near by? \"\n", "if (not canAnswer(sample_description, prompt)):\n", " print(\"GOING TO TRAVERSE API\")\n", " x = ares_api(prompt + \"for hotel Campanile Bercy located in paris\")\n", " print(x)\n", " print(\"RECEIVED INFORMATION FROM ARES API\")\n", " sample_description += x\n", "else:\n", " print(\"Can Answer\")" ] }, { "cell_type": "code", "execution_count": 70, "id": "f31668c7-70e8-41b9-a2c8-e02a469d014e", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Some good restaurants near Hotel Campanile Bercy in Paris are:\n", "\n", "1. Sushi Yuki\n", "2. Fenetre sur Cour\n", "3. L'Auberge Aveyronnaise\n", "4. The Frog at Bercy Village\n", "5. Pedra Alta\n", "6. Fresh'Heure\n", "7. Le Midnight Paris\n", "8. Zendo Sushi Restaurant\n", "\n", "You can find more information and reviews on these restaurants [here](https://www.tripadvisor.com/RestaurantsNear-g187147-d233766-Hotel_Campanile_Paris_Bercy_Village-Paris_Ile_de_France.html). Enjoy your dining experience!" ] } ], "source": [ "client = OpenAI(api_key=st.secrets[\"OPENAI_API_KEY\"])\n", "stream = client.chat.completions.create(\n", " model='gpt-3.5-turbo',\n", " messages=[\n", " {\"role\": \"user\", \"content\": sample_description},\n", " {\"role\": \"user\", \"content\": prompt} \n", "\n", " ],\n", " stream=True,\n", ")\n", "for chunk in stream:\n", " if chunk.choices[0].delta.content is not None:\n", " print(chunk.choices[0].delta.content, end = '')\n" ] }, { "cell_type": "code", "execution_count": 54, "id": "834dd9e5-2b03-4c8a-9fd7-faba5fa27e92", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.5" } }, "nbformat": 4, "nbformat_minor": 5 } ================================================ FILE: conversational search/app.py ================================================ from openai import OpenAI import requests import streamlit as st from qdrant_client import QdrantClient from sentence_transformers import SentenceTransformer def getResponse(query): url = "https://api-ares.traversaal.ai/live/predict" payload = { "query": [query] } headers = { "x-api-key": st.secrets["TRAVERSAAL"], "content-type": "application/json" } response = requests.post(url, json=payload, headers=headers) if response.status_code == 200: # Get the JSON content from the response json_content = response.json() # Specify the file path where you want to save the JSON content return json_content else: print(response.status_code) return " " class NeuralSearcher: def __init__(self, collection_name): self.collection_name = collection_name # Initialize encoder model self.model = SentenceTransformer("all-MiniLM-L6-v2", device="cpu") # initialize Qdrant client # self.qdrant_client = QdrantClient("http://localhost:6333") self.qdrant_client = QdrantClient( url="https://ed55d75f-bb54-4c09-8907-8d112e6278a1.us-east4-0.gcp.cloud.qdrant.io", api_key=st.secrets["QDRANT_API_KEY"], ) def search(self, text: str): # Convert text query into vector vector = self.model.encode(text).tolist() # Use `vector` for search for closest vectors in the collection search_result = self.qdrant_client.search( collection_name=self.collection_name, query_vector=vector, query_filter=None, # If you don't want any filters for now limit=3, # 5 the most closest results is enough ) # `search_result` contains found vector ids with similarity scores along with the stored payload # In this function you are interested in payload only payloads = [hit.payload for hit in search_result] return payloads def initializeClient(): return OpenAI(api_key=st.secrets["OPENAI_API_KEY"]) def decode(hotel_description, query): client = initializeClient(); prompt = f""" this is the hotel description: \"{hotel_description}\" and these are my requirements \"{query}\" now tell me why the hotel might be a good fit for me given the requirements, make it consise. """ stream = client.chat.completions.create( model="gpt-4", messages=[{"role": "user", "content": prompt}], stream=True, ) str = "" for chunk in stream: if chunk.choices[0].delta.content is not None: str += (chunk.choices[0].delta.content) return str def canAnswer(description, q): client = initializeClient(); prompt = f""" \"{description}\" \n is there information about the following in the above text, make sure you will be able to enaswer the following question prcisely: {q} \n answer in one word, "yes" or "no" """ stream = client.chat.completions.create( model="gpt-4", messages=[{"role": "user", "content": prompt}], stream=True, ) strr = "" for chunk in stream: if chunk.choices[0].delta.content is not None: strr += (chunk.choices[0].delta.content) return strr.lower() == "yes"; def home_page(): # st.title("TraverGo") st.markdown("

TraverGo

", unsafe_allow_html=True) st.markdown("

Find any type of Hotel you want !

", unsafe_allow_html=True) if "chat" not in st.session_state: st.session_state["chat"] = False; def search_hotels(): query = st.text_input("Enter your hotel preferences:", placeholder ="clean and cheap hotel with good food and gym") if "load_state" not in st.session_state: st.session_state.load_state = False; # Perform semantic search when user submits query if query or st.session_state.load_state: # if query: # st.session_state['decoder'] = [0]; st.session_state.load_state=True; neural_searcher = NeuralSearcher(collection_name="hotel_descriptions") results = sorted(neural_searcher.search(query), key=lambda d: d['sentiment_rate_average']) st.subheader("Hotels") for hotel in results: explore_hotel(hotel, query) # Call a separate function for each hotel def explore_hotel(hotel, query): if "decoder" not in st.session_state: st.session_state['decoder'] = [0]; button = st.button(hotel['hotel_name']) if button or st.session_state.chat: if button and st.session_state.chat: st.session_state.chat = False; del st.session_state["messages"]; else: if button: st.session_state["value"] = hotel; st.session_state.chat = True; else: st.session_state["value"] = None; if st.session_state.decoder == [0]: x = (decode(hotel['hotel_description'][:1000], query)) st.session_state['value_1'] = x st.session_state.decoder = [st.session_state.decoder[0] + 1] st.write(x) elif (st.session_state.decoder == [1]): x = (decode(hotel['hotel_description'][:1000], query)) st.session_state['value_2'] = x st.session_state.decoder = [st.session_state.decoder[0] + 1]; st.write(x); elif st.session_state.decoder == [2]: x = (decode(hotel['hotel_description'][:1000], query)) st.session_state['value_3'] = x; st.session_state.decoder = [st.session_state.decoder[0] + 1]; st.write(x); if (st.session_state.decoder[0] >= 3): i = st.session_state.decoder[0] % 3 l = ['value_1', 'value_2', 'value_3'] st.session_state[l[i - 1]]; st.session_state.decoder = [st.session_state.decoder[0] + 1]; question = st.text_input(f"Enter a question about {hotel['hotel_name']}:"); if question: st.write(ares_api(question + " - " + hotel['hotel_name'] + "located in" + hotel['country'])) search_hotels() if (st.session_state.chat): chat_page() def ares_api(query): response_json = getResponse(query); # if response_json is not json: # return "Could not find information" return (response_json['data']['response_text']) def chat_page(): hotel = st.session_state["value"] # st.session_state.value = None if (hotel == None): return; st.write(hotel['hotel_name']); st.title("Conversation") # Set OpenAI API key from Streamlit secrets client = OpenAI(api_key=st.secrets["OPENAI_API_KEY"]) # st.session_state.pop("messages") # Set a default model if "openai_model" not in st.session_state: st.session_state["openai_model"] = "gpt-4" prompt = f"{hotel['hotel_description'][:1500]}\n\n everything before this point is the hotel description and reveiws. now you as a hotel advisor now, should give the best answerws based on the above text. Now wait for my questions." # Initialize chat history if "messages" not in st.session_state: st.session_state.messages = [{"role": "user", "content": prompt}] for message in st.session_state.messages[1:]: with st.chat_message(message["role"]): st.markdown(message["content"]) # Accept user input if prompt := st.chat_input("What is up?"): # Display user message in chat message container with st.chat_message("user"): st.markdown(prompt) if (not canAnswer(hotel['hotel_description'][:2000], prompt)): st.write("GOING TO ARES API") print("GOING TO TRAVERSE API") x = ares_api(prompt + "for" + hotel['hotel_name'] + "located in" + hotel['country']) print(x) st.write("RECEIVED INFORMATION FROM ARES API") st.session_state.messages[0]['content'] = x + "\n" + st.session_state.messages[0]['content']; st.session_state.messages.append({"role": "user", "content": prompt}) #Display assistant response in chat message container with st.chat_message("assistant"): stream = client.chat.completions.create( model=st.session_state["openai_model"], messages=[ {"role": m["role"], "content": m["content"]} for m in st.session_state.messages ], stream=True, ) response = st.write_stream(stream) st.session_state.messages.append({"role": "assistant", "content": response}) home_page() ================================================ FILE: conversational search/conversational_search_ares_api.ipynb ================================================ { "cells": [ { "cell_type": "code", "execution_count": 4, "id": "4d1d583c-ff72-4f04-b15e-3a5e7a7a075d", "metadata": {}, "outputs": [], "source": [ "from openai import OpenAI\n", "import requests\n", "import streamlit as st\n", "from qdrant_client import QdrantClient\n", "\n", "from sentence_transformers import SentenceTransformer" ] }, { "cell_type": "code", "execution_count": 7, "id": "c1f28558-9957-4156-a69b-7b952351c5e7", "metadata": {}, "outputs": [], "source": [ "def getResponse(query):\n", " url = \"https://api-ares.traversaal.ai/live/predict\"\n", " payload = { \"query\": [query] }\n", " headers = {\n", " \"x-api-key\": st.secrets[\"TRAVERSAAL\"],\n", " \"content-type\": \"application/json\"\n", " }\n", "\n", " response = requests.post(url, json=payload, headers=headers)\n", " if response.status_code == 200:\n", " # Get the JSON content from the response\n", " json_content = response.json()\n", "\n", " # Specify the file path where you want to save the JSON content\n", " return json_content\n", " else:\n", " print(response.status_code)\n", " return \" \"" ] }, { "cell_type": "code", "execution_count": 63, "id": "02856991-8745-491f-9b9d-bd31e7ccc9bd", "metadata": {}, "outputs": [], "source": [ "def ares_api(query):\n", " response_json = getResponse(query);\n", " return (response_json['data']['response_text'])\n" ] }, { "cell_type": "code", "execution_count": 58, "id": "d36e7d39-1175-4099-9093-f4e1fea21804", "metadata": {}, "outputs": [], "source": [ "def canAnswer(description, q):\n", " client = initializeClient();\n", "\n", " prompt = f\"\"\"\n", " \\\"{description}\\\"\n", " \\n\n", " is there information about the following in the above text, make sure you will be able to enaswer the following question prcisely: {q}\n", " \\n\n", " answer in one word, \"yes\" or \"no\"\n", " \"\"\"\n", " stream = client.chat.completions.create(\n", " model=\"gpt-4\",\n", " messages=[{\"role\": \"user\", \"content\": prompt}],\n", " stream=True,\n", " )\n", " strr = \"\"\n", " for chunk in stream:\n", " if chunk.choices[0].delta.content is not None:\n", " strr += (chunk.choices[0].delta.content)\n", " return strr.lower() == \"yes\";\n", " \n" ] }, { "cell_type": "code", "execution_count": 60, "id": "933f1403-5336-4468-8890-4c0c409619c5", "metadata": {}, "outputs": [], "source": [ "sample_description = \"\"\"\n", " In the east of Paris, the hotel Campanile Bercy boasts an ideal location for exploring The City of\n", " LightsNature-lovers staying at Campanile Bercy will enjoy its privileged vicinity,taking walks in \n", " Bercy Park and its four gardens, strolling around the village \"\"Cour Saint Emilion\"\" as well as \n", " trying restaurants and cafes.\n", "\"\"\"" ] }, { "cell_type": "code", "execution_count": 68, "id": "8f7859db-f6fd-4c81-bc08-3f9851135b96", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "GOING TO TRAVERSE API\n", "The best restaurants near Hotel Campanile Bercy in Paris are Sushi Yuki, Fenetre sur Cour, L'Auberge Aveyronnaise, The Frog at Bercy Village, Pedra Alta, Fresh'Heure, Le Midnight Paris, and Zendo Sushi Restaurant. You can find more information [here](https://www.tripadvisor.com/RestaurantsNear-g187147-d233766-Hotel_Campanile_Paris_Bercy_Village-Paris_Ile_de_France.html).\n", "RECEIVED INFORMATION FROM ARES API\n" ] } ], "source": [ "prompt = \"whats are some good restaurants near by? \"\n", "if (not canAnswer(sample_description, prompt)):\n", " print(\"GOING TO TRAVERSE API\")\n", " x = ares_api(prompt + \"for hotel Campanile Bercy located in paris\")\n", " print(x)\n", " print(\"RECEIVED INFORMATION FROM ARES API\")\n", " sample_description += x\n", "else:\n", " print(\"Can Answer\")" ] }, { "cell_type": "code", "execution_count": 70, "id": "f31668c7-70e8-41b9-a2c8-e02a469d014e", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Some good restaurants near Hotel Campanile Bercy in Paris are:\n", "\n", "1. Sushi Yuki\n", "2. Fenetre sur Cour\n", "3. L'Auberge Aveyronnaise\n", "4. The Frog at Bercy Village\n", "5. Pedra Alta\n", "6. Fresh'Heure\n", "7. Le Midnight Paris\n", "8. Zendo Sushi Restaurant\n", "\n", "You can find more information and reviews on these restaurants [here](https://www.tripadvisor.com/RestaurantsNear-g187147-d233766-Hotel_Campanile_Paris_Bercy_Village-Paris_Ile_de_France.html). Enjoy your dining experience!" ] } ], "source": [ "client = OpenAI(api_key=st.secrets[\"OPENAI_API_KEY\"])\n", "stream = client.chat.completions.create(\n", " model='gpt-3.5-turbo',\n", " messages=[\n", " {\"role\": \"user\", \"content\": sample_description},\n", " {\"role\": \"user\", \"content\": prompt} \n", "\n", " ],\n", " stream=True,\n", ")\n", "for chunk in stream:\n", " if chunk.choices[0].delta.content is not None:\n", " print(chunk.choices[0].delta.content, end = '')\n" ] }, { "cell_type": "code", "execution_count": 54, "id": "834dd9e5-2b03-4c8a-9fd7-faba5fa27e92", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.5" } }, "nbformat": 4, "nbformat_minor": 5 } ================================================ FILE: conversational search/travergo.md ================================================ --- title: Harnessing the Power of Ares API for Real-Time Insights in TraverGo’s Chatbot author_profile: true permalink: /projects/travergo_article --- {% include base_path %} TraverGo is the 1st place winning hackalytics project at Georgia Tech project for the Traversaal AI challenge. TraverGo's hotel search platform is an innovative way to help users find the perfect accommodations. Our platform offers powerful features like Neural Search and Sentiment Analysis to help users find their perfect hotel. One of its standout features is the seamless integration of a chatbot with the Ares API, ensuring users can get both hotel-specific answers and access real-time information about the surrounding area. Let's dive into how this integration works. # Why the Ares API? While your chatbot has a solid knowledge base of hotel descriptions and reviews, there will be times when a user's query goes beyond those confines. The Ares API acts as a powerful bridge between our chatbot and the vast knowledge base of Google. For instance some questions that a chatbot might find hard to answer are based on: 1. "Are there any good Italian restaurants near the hotel?" 2. "What's the nearest subway station?" 3. "Is there a museum within walking distance?" These types of questions fall outside the scope of our hotel-specific dataset. Ares API allows us to tap into Google's real-time data, providing accurate and up-to-date answers for our users. # Implementation Logic The following will be our flow of logic in dealing with a specific user question. 1. Dependency: We start by importing the ares library, granting us access to the Ares API. 2. Intent Detection: Our chatbot needs a way to distinguish when a user wants to utilize the Ares integration. We use a dedicated prompt to distinguish between instanecs where we know the answer. 3. Query Extraction: We isolate the actual question the user intends to ask Google. 4. Ares Search: The core of the integration. Relevent code is in the next section 5. Chatbot Response: Our chatbot delivers the formatted Ares API results to the user. # Integrating the Ares API into our chatbot ## Setting up the API To get started, you'll need: 1. Ares API Key: Sign up with the Ares API provider to get your API key. 2. Python Libraries: Install the requests and openai libraries (pip install requests openai). ```python import requests import openai import streamlit as st # If using Streamlit def getResponse(query): url = "https://api-ares.traversaal.ai/live/predict" payload = { "query": [query] } headers = { "x-api-key": st.secrets["TRAVERSAAL"], "content-type": "application/json" } response = requests.post(url, json=payload, headers=headers) if response.status_code == 200: # Get the JSON content from the response json_content = response.json() return json_content else: return "" ``` `getResponse`: This function handles sending a query to the Ares API endpoint, managing authentication, and processing the returned JSON response. You will need to get your own Ares API key in order to use this function. If the response from the GET request is 200 it means that it was successfull and hence will return the relevent data else it would return an empty string. We user another function `ares_api` to wrap the call to getResponse(), this makes it easier to get the relevent text from the API's response. ```python def ares_api(query): response_json = getResponse(query); return (response_json['data']['response_text']) ``` ## Delegation of tasks To ensure that we only delegate the relevent user question to the Ares API when we are unable to find the answer within our text corpus we use another decoder model which will tell us if the specific answer to the user question already exists within our code. To implement this we use the following function `canAnswer()` ```python def canAnswer(description, q): client = initializeClient(); prompt = f""" \"{description}\" \n is there information about the following in the above text, make sure you will be able to enaswer the following question prcisely: {q} \n answer in one word, "yes" or "no" """ stream = client.chat.completions.create( model="gpt-4", messages=[{"role": "user", "content": prompt}], stream=True, ) strr = "" for chunk in stream: if chunk.choices[0].delta.content is not None: strr += (chunk.choices[0].delta.content) return strr.lower() == "yes"; ``` `canAnswer`: This function leverages OpenAI's language models to intelligently determine if a question can be answered confidently based on a given description. ## Chatbot Integration This is the main integration of the Ares API with our chatbot. After initializing streamlit session states, we first append the hotel description to the message history of our chatbot as follows. We also include the relevent prompt to get the chatbot ready to answer our questions. ```python def chat_page(): hotel = st.session_state["value"] # st.session_state.value = None if (hotel == None): return; st.title("Conversation") # Set OpenAI API key from Streamlit secrets client = OpenAI(api_key=st.secrets["OPENAI_API_KEY"]) # st.session_state.pop("messages") # Set a default model if "openai_model" not in st.session_state: st.session_state["openai_model"] = "gpt-4" prompt = f"{hotel['hotel_description'][:1500]}\n\n everything before this point is the hotel description and reveiws. now you as a hotel advisor now, should give the best answerws based on the above text. Now wait for my questions." # Initialize chat history if "messages" not in st.session_state: st.session_state.messages = [{"role": "user", "content": prompt}] for message in st.session_state.messages[1:]: with st.chat_message(message["role"]): st.markdown(message["content"]) ``` In the above code we create our prompt with the hotel description and prepare our chatbot to answer our questions. We append this to the start of the message history every time the chatbot retrieves its message history from the streamlit session state. Now we imlement our logic of delegating the questions to the API if and when we cannot find them in our text. ```python def chat_page(): # ... (Code for initializing session states and appending description) ... # Accept user input if prompt := st.chat_input("What is up?"): # Display user message in chat message container with st.chat_message("user"): st.markdown(prompt) if (not canAnswer(hotel['hotel_description'][:2000], prompt)): x = ares_api(prompt + "for" + hotel['hotel_name'] + "located in" + hotel['country']) st.session_state.messages[0]['content'] = x + "\n" + st.session_state.messages[0]['content']; st.session_state.messages.append({"role": "user", "content": prompt}) ``` The relevent code bits are, ```python if (not canAnswer(hotel['hotel_description'][:2000], prompt)): x = ares_api(prompt + "for" + hotel['hotel_name'] + "located in" + hotel['country']) st.session_state.messages[0]['content'] = x + "\n" + st.session_state.messages[0]['content']; ``` In this code we first call our `canAnswer` function which uses our gpt-4 decoder model in order to determine if we can answer the question or not given our text. If `canAnswer` return true we continue as normal and let our chatbot answer the question. However if it returns false it implies that we cannot find the answer in our text. In this case we create a prompt to give to the API and receive the new information. Now that we have this information we append this to our hotel description and reviews that we already have. In this way our chatbot has access to both the new information from Google that we previously did not have access to and also have access to the hotel description and reviews that was in our dataset. ```python #Display assistant response in chat message container with st.chat_message("assistant"): stream = client.chat.completions.create( model=st.session_state["openai_model"], messages=[ {"role": m["role"], "content": m["content"]} for m in st.session_state.messages ], stream=True, ) response = st.write_stream(stream) st.session_state.messages.append({"role": "assistant", "content": response}) ``` The above code finally creates the response from gpt-4 using the client that we initialized. ================================================ FILE: semantic cache/readme.md ================================================ **Semantic Caching** Overview
Semantic Caching enhances the efficiency of semantic search operations by caching query results, thereby reducing redundant computations and trips to the LLM for similar queries. This approach speeds up response times, decreases computational load, and minimizes costs. The project utilizes a sentence transformer model for semantic understanding and integrates with the Qdrant vector database for storing and searching embeddings. Features
Semantic Understanding: Employs the all-mpnet-base-v2 model to encode questions and grasp their semantics effectively. A different encoder can be used during instantiating SemanticCaching class. Efficient Caching: Caches query results and employs a similarity threshold to determine cache hits, thereby improving response times for similar queries. ================================================ FILE: semantic cache/semantic_cache_from_scratch.ipynb ================================================ { "cells": [ { "cell_type": "markdown", "metadata": { "id": "view-in-github", "colab_type": "text" }, "source": [ "\"Open" ] }, { "cell_type": "markdown", "source": [ "If you use our code, please cite:\n", "\n", "@misc{2024
\n", " title = {Semantic Cache from Scratch},
\n", " author = {Hamza Farooq, Darshil Modi, Kanwal Mehreen, Nazila Shafiei},
\n", " keywords = {Semantic Cache},
\n", " year = {2024},
\n", " copyright = {MIT, non-exclusive license}
\n", "}" ], "metadata": { "id": "Pr3rEVniF9Vx" }, "id": "Pr3rEVniF9Vx" }, { "cell_type": "code", "source": [ "!pip install -U faiss-cpu sentence_transformers transformers" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "025_hZMnZUIE", "outputId": "1f2b722e-019d-475a-ef26-2d3f031e0b7b" }, "id": "025_hZMnZUIE", "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Collecting faiss-cpu\n", " Downloading faiss_cpu-1.8.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (27.0 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m27.0/27.0 MB\u001b[0m \u001b[31m58.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hCollecting sentence_transformers\n", " Downloading sentence_transformers-2.5.1-py3-none-any.whl (156 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m156.5/156.5 kB\u001b[0m \u001b[31m20.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: transformers in /usr/local/lib/python3.10/dist-packages (4.38.2)\n", "Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from faiss-cpu) (1.25.2)\n", "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from sentence_transformers) (4.66.2)\n", "Requirement already satisfied: torch>=1.11.0 in /usr/local/lib/python3.10/dist-packages (from sentence_transformers) (2.1.0+cu121)\n", "Requirement already satisfied: scikit-learn in /usr/local/lib/python3.10/dist-packages (from sentence_transformers) (1.2.2)\n", "Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from sentence_transformers) (1.11.4)\n", "Requirement already satisfied: huggingface-hub>=0.15.1 in /usr/local/lib/python3.10/dist-packages (from sentence_transformers) (0.20.3)\n", "Requirement already satisfied: Pillow in /usr/local/lib/python3.10/dist-packages (from sentence_transformers) (9.4.0)\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers) (3.13.1)\n", "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers) (23.2)\n", "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers) (6.0.1)\n", "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers) (2023.12.25)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers) (2.31.0)\n", "Requirement already satisfied: tokenizers<0.19,>=0.14 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.15.2)\n", "Requirement already satisfied: safetensors>=0.4.1 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.4.2)\n", "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.15.1->sentence_transformers) (2023.6.0)\n", "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.15.1->sentence_transformers) (4.10.0)\n", "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.11.0->sentence_transformers) (1.12)\n", "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.11.0->sentence_transformers) (3.2.1)\n", "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.11.0->sentence_transformers) (3.1.3)\n", "Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.11.0->sentence_transformers) (2.1.0)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (3.3.2)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (3.6)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (2.0.7)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (2024.2.2)\n", "Requirement already satisfied: joblib>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from scikit-learn->sentence_transformers) (1.3.2)\n", "Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn->sentence_transformers) (3.3.0)\n", "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=1.11.0->sentence_transformers) (2.1.5)\n", "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.11.0->sentence_transformers) (1.3.0)\n", "Installing collected packages: faiss-cpu, sentence_transformers\n", "Successfully installed faiss-cpu-1.8.0 sentence_transformers-2.5.1\n" ] } ] }, { "cell_type": "code", "execution_count": null, "id": "52273bc0-575b-4007-b63d-bfe53d4abde6", "metadata": { "id": "52273bc0-575b-4007-b63d-bfe53d4abde6" }, "outputs": [], "source": [ "import faiss\n", "import sqlite3\n", "from sentence_transformers import SentenceTransformer\n", "import torch\n", "from transformers import AutoModelForCausalLM, AutoTokenizer\n", "import numpy as np\n", "from pprint import pprint\n", "\n", "\n" ] }, { "cell_type": "markdown", "source": [ "\n", "# Traversaal Ares API Overview\n", "\n", "Traversaal Ares API is a cutting-edge solution designed to provide real-time search results generated from user queries. Leveraging advanced Large Language Models (LLMs), Ares connects to the internet to deliver accurate and factual information, including relevant URLs for reference. This API is tailored for speed and efficiency, providing lightning-fast search results within 3-4 seconds. Currently available for free during the beta phase, with priced solutions coming soon.\n", "\n", "## Key Features:\n", "- **Real-time Search Results:** Ares API offers unparalleled speed in generating search results.\n", "- **Internet Connectivity:** Connects to the internet to fetch the latest and most accurate information.\n", "- **Lightning-Fast Response:** Delivers search results with URLs in 3-4 seconds.\n", "- **Free Beta Access:** Available for free during the beta phase, with pricing plans to be introduced.\n", "- **Factual and Accurate:** Ensures the information provided is accurate and supported by relevant references.\n", "\n", "## Getting Started:\n", "To access the Ares API, sign up at [api.traversaal.ai](https://api.traversaal.ai) and refer to the usage documentation at [docs.traversaal.ai](https://docs.traversaal.ai/docs/intro).\n", "\n", "Experience the future of AI-driven search with Traversaal Ares API!\n" ], "metadata": { "id": "pvesG5KVOgtT" }, "id": "pvesG5KVOgtT" }, { "cell_type": "code", "execution_count": null, "id": "38331891-adb4-4d16-b26f-d74d7c9ce728", "metadata": { "id": "38331891-adb4-4d16-b26f-d74d7c9ce728" }, "outputs": [], "source": [ "import requests\n", "\n", "def make_prediction(data):\n", " url = \"https://api-ares.traversaal.ai/live/predict\"\n", " headers = {\n", " \"x-api-key\": \"ares_xxx\",\n", " \"content-type\": \"application/json\"\n", " }\n", "\n", " payload = {\"query\": data}\n", "\n", " try:\n", " response = requests.post(url, json=payload, headers=headers)\n", "\n", " if response.status_code == 200:\n", " # The request was successful\n", " print(\"Request was successful.\")\n", " # If the response contains JSON data, you can parse it using response.json()\n", " try:\n", " json_data = response.json()\n", " #print(\"Parsed JSON data:\", json_data)\n", " return json_data\n", " except ValueError:\n", " print(\"No JSON data in the response.\")\n", " return None\n", " else:\n", " # The request was not successful, handle the error\n", " print(f\"Request failed with status code {response.status_code}.\")\n", " return None\n", " except requests.exceptions.RequestException as e:\n", " print(f\"Error during request: {e}\")\n", " return None\n", "\n", "# Example usage\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "id": "19acb698-6e78-43af-83d2-4f29b68528a6", "metadata": { "id": "19acb698-6e78-43af-83d2-4f29b68528a6", "outputId": "dcd74af8-45bc-4842-e713-dc537e5f72a4", "colab": { "base_uri": "https://localhost:8080/" } }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Request was successful.\n" ] } ], "source": [ "response=make_prediction(['I am planning my 10th Anniversary, provide me a list of places in Boston which are quiet, private and climate controlled. '])" ] }, { "cell_type": "code", "source": [ "response" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "onZRAq55aMuj", "outputId": "e8be53f4-e578-463c-936c-515db033c3fa" }, "id": "onZRAq55aMuj", "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "{'data': {'response_text': \"Here are some places in Boston that are quiet, private, and climate controlled for your 10th Anniversary:\\n\\n1. The Liberty Hotel: This historic hotel offers elegant and private event spaces with climate control for a quiet and intimate celebration.\\n\\n2. The Lenox Hotel: Located in the heart of Boston, The Lenox Hotel offers luxurious and private venues for a quiet anniversary celebration. Their event spaces are climate controlled for your comfort.\\n\\n3. The Taj Boston: This iconic hotel features elegant and private event spaces that are perfect for a quiet and intimate anniversary celebration. The venues are climate controlled to ensure your comfort.\\n\\n4. The Boston Harbor Hotel: With stunning waterfront views, this hotel offers private event spaces that are quiet and climate controlled. It's a perfect choice for a romantic anniversary celebration.\\n\\n5. The Fairmont Copley Plaza: This historic hotel offers elegant and private event spaces that are climate controlled for a quiet and comfortable anniversary celebration.\\n\\nPlease note that availability and pricing may vary, so it's recommended to contact each venue directly for more information and to make reservations.\",\n", " 'web_url': ['https://www.thefoodlens.com/boston/sides/guide/restaurants-with-private-rooms/',\n", " 'https://swimply.com/explore/us-ma-boston/anniversary',\n", " 'https://www.ourescapeclause.com/2-days-in-boston-itinerary/',\n", " 'https://www.boston.gov/visiting-boston',\n", " 'https://en.wikipedia.org/wiki/Boston',\n", " 'https://helicoptertourboston.com/',\n", " 'https://www.afi.com/afis-100-years-100-movies-10th-anniversary-edition/',\n", " 'https://bostonathenaeum.org/visit/',\n", " 'https://www.nytimes.com/wirecutter/guides/heat-pump-buying-guide/',\n", " 'https://www.fitnyc.edu/']}}" ] }, "metadata": {}, "execution_count": 24 } ] }, { "cell_type": "code", "source": [ "pprint(response['data']['response_text'])" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "H9pAGBjTaDA8", "outputId": "f5f0bbe3-2ad2-4a69-e466-4c84f54b438c" }, "id": "H9pAGBjTaDA8", "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "('Here are some places in Boston that are quiet, private, and climate '\n", " 'controlled for your 10th Anniversary:\\n'\n", " '\\n'\n", " '1. The Liberty Hotel: This historic hotel offers elegant and private event '\n", " 'spaces with climate control for a quiet and intimate celebration.\\n'\n", " '\\n'\n", " '2. The Lenox Hotel: Located in the heart of Boston, The Lenox Hotel offers '\n", " 'luxurious and private venues for a quiet anniversary celebration. Their '\n", " 'event spaces are climate controlled for your comfort.\\n'\n", " '\\n'\n", " '3. The Taj Boston: This iconic hotel features elegant and private event '\n", " 'spaces that are perfect for a quiet and intimate anniversary celebration. '\n", " 'The venues are climate controlled to ensure your comfort.\\n'\n", " '\\n'\n", " '4. The Boston Harbor Hotel: With stunning waterfront views, this hotel '\n", " \"offers private event spaces that are quiet and climate controlled. It's a \"\n", " 'perfect choice for a romantic anniversary celebration.\\n'\n", " '\\n'\n", " '5. The Fairmont Copley Plaza: This historic hotel offers elegant and private '\n", " 'event spaces that are climate controlled for a quiet and comfortable '\n", " 'anniversary celebration.\\n'\n", " '\\n'\n", " \"Please note that availability and pricing may vary, so it's recommended to \"\n", " 'contact each venue directly for more information and to make reservations.')\n" ] } ] }, { "cell_type": "code", "source": [ "response['data']['web_url']" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "A3w7uAbOaLhE", "outputId": "61884828-d9f8-4640-a1cc-6bc12b16d12c" }, "id": "A3w7uAbOaLhE", "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "['https://sf.eater.com/maps/best-tacos-san-francisco',\n", " 'https://www.sftravel.com/article/where-to-find-best-tacos-san-francisco',\n", " 'https://lataco.com/san-francisco-best-tacos-guide',\n", " 'https://www.reddit.com/r/AskSF/comments/16bn1w1/best_tacos_in_sf/',\n", " 'https://www.femalefoodie.com/restaurant-reviews/best-tacos-in-san-francisco/',\n", " 'https://www.yelp.com/search?find_desc=Street+Tacos&find_loc=San+Francisco%2C+CA',\n", " 'https://traveloutlandish.com/blog/best-tacos-in-san-francisco-taquerias/',\n", " 'https://www.foodtalkcentral.com/t/sf-chronicle-bay-area-tacos/15225',\n", " 'https://www.yelp.com/search?find_desc=Tacos&find_loc=Outer+Sunset%2C+San+Francisco%2C+CA',\n", " 'https://www.toasttab.com/local/san-francisco-ca-restaurants/dish/tacos']" ] }, "metadata": {}, "execution_count": 7 } ] }, { "cell_type": "markdown", "source": [ "Instead of using an LLM endpoint, we will be using Ares API for retrieval and generation, however you can replace is with your own rag function in 'generate answer' function" ], "metadata": { "id": "WJ1f4VlZPGZQ" }, "id": "WJ1f4VlZPGZQ" }, { "cell_type": "code", "source": [ "import faiss\n", "import json\n", "import numpy as np\n", "from sentence_transformers import SentenceTransformer\n", "from transformers import AutoTokenizer, AutoModelForCausalLM\n", "import time\n", "\n", "class SemanticCaching:\n", " def __init__(self, json_file='cache.json'):\n", " # Initialize Faiss index with Euclidean distance\n", " self.index = faiss.IndexFlatL2(768) # Use IndexFlatL2 with Euclidean distance\n", " if self.index.is_trained:\n", " print('Index trained')\n", "\n", " # Initialize Sentence Transformer model\n", " self.encoder = SentenceTransformer('all-mpnet-base-v2')\n", "\n", "\n", " # Uncomment the following lines to use DialoGPT for question generation\n", " # self.tokenizer = AutoTokenizer.from_pretrained(\"microsoft/DialoGPT-large\")\n", " # self.model = AutoModelForCausalLM.from_pretrained(\"microsoft/DialoGPT-large\")\n", "\n", " # Set Euclidean distance threshold\n", " self.euclidean_threshold = 0.3\n", " self.json_file = json_file\n", " self.load_cache()\n", "\n", " def load_cache(self):\n", " # Load cache from JSON file, creating an empty cache if the file is not found\n", " try:\n", " with open(self.json_file, 'r') as file:\n", " self.cache = json.load(file)\n", " except FileNotFoundError:\n", " self.cache = {'questions': [], 'embeddings': [], 'answers': [], 'response_text': []}\n", "\n", " def save_cache(self):\n", " # Save the cache to the JSON file\n", " with open(self.json_file, 'w') as file:\n", " json.dump(self.cache, file)\n", "\n", " def ask(self, question: str) -> str:\n", " # Method to retrieve an answer from the cache or generate a new one\n", " start_time = time.time()\n", " try:\n", " l = [question]\n", " embedding = self.encoder.encode(l)\n", "\n", " # Search for the nearest neighbor in the index\n", " D, I = self.index.search(embedding, 1)\n", "\n", " if D[0] >= 0:\n", " if I[0][0] != -1 and D[0][0] <= self.euclidean_threshold:\n", " row_id = int(I[0][0])\n", " print(f'Found cache in row: {row_id} with score {1 - D[0][0]}')\n", " end_time = time.time()\n", " elapsed_time = end_time - start_time\n", " print(f\"Time taken: {elapsed_time} seconds\")\n", " return self.cache['response_text'][row_id]\n", "\n", " # Handle the case when there are not enough results or Euclidean distance is not met\n", " answer, response_text = self.generate_answer(question)\n", "\n", " self.cache['questions'].append(question)\n", " self.cache['embeddings'].append(embedding[0].tolist())\n", " self.cache['answers'].append(answer)\n", " self.cache['response_text'].append(response_text)\n", "\n", " self.index.add(embedding)\n", " self.save_cache()\n", " end_time = time.time()\n", " elapsed_time = end_time - start_time\n", " print(f\"Time taken: {elapsed_time} seconds\")\n", "\n", " return response_text\n", " except Exception as e:\n", " raise RuntimeError(f\"Error during 'ask' method: {e}\")\n", "\n", " def generate_answer(self, question: str) -> str:\n", " # Method to generate an answer using a separate function (make_prediction in this case)\n", " try:\n", " result = make_prediction([question])\n", " response_text = result['data']['response_text']\n", "\n", " return result, response_text\n", " except Exception as e:\n", " raise RuntimeError(f\"Error during 'generate_answer' method: {e}\")\n" ], "metadata": { "id": "yDHhY-OBSEIw" }, "id": "yDHhY-OBSEIw", "execution_count": null, "outputs": [] }, { "cell_type": "code", "execution_count": null, "id": "dc661dab-f7cc-4d74-9575-1c756b4cdef0", "metadata": { "id": "dc661dab-f7cc-4d74-9575-1c756b4cdef0", "outputId": "41f10ffd-2992-4a23-a780-c5045b7602f3", "colab": { "base_uri": "https://localhost:8080/", "height": 545, "referenced_widgets": [ "d26f2662cd874314b07622775c765a19", "08e25cb3485f48d795cb7299f82aee1b", "ae980ec05d164a0d8fcc959958162143", "eceb9664746a48419c9376a67f52aa6a", "55bf5300fce34317b8debf36fa4aa11b", "612616af9ab64387a4d62f9b655b9c32", "d7cb16dd36694f1091011b40d296fdae", "08cc047828db49d3832ca13121da1153", "b601a96b07ec4b6ab69d5a6d6f31791c", "74d5511253194de48e78524a7b241447", "4c4fcf13b09a4a82bd89b7ac4522893d", "4b4cf3df1e5e4faa90a5d8956709cd77", "73bae2447f854c0e9a49ae69f8af6ded", "c796de216ea84ad990f3c24509c205b4", "5d966fa926d34d3986ff99275106f17a", "fc92085d6bc547acb4f4b90badb64d72", "6d7d0ea8565c4b7ca88e1402d222edec", "484bad4dee464cd79b213e8658334b79", "e7f4668a76d44a9ba8719e243a5faadd", "4ff304b24c3849928c3bfba40cf1524d", "40b5c2eae94e4da5aa914393dbb5a582", "d52b914f8ba54756800bf64a7246efcf", "215977b71fd74e0b86f7306afd389259", "c0737c836b3041d3905aa16956481ca9", "9104e395402c4b42a90a0060c39f098c", "fc78be077b68479ba196dc3c0f46dee9", "f367c4ba19d2421abfa3e98dee12c265", "f5ddd5679d6441059641b3a9cd860ee1", "4632cc37bd004d21b1fdb8d3199985e4", "9b7c816e1dc14fd5ab4ab6217e67ee70", "78f36c8ae30e4386b9789a31a4a408a5", "86a617fe264246d58a47353780923f9a", "f7261262ffad44eb81be129d918a31a1", "8505d2e9d9624c3da07a7d70798e64e6", "1c943e9d17f242d6a35583037944076e", "5988e396fdb64850bbf47e1b6a150a22", "4b97620256cc473690beaf2222af34ab", "d56eb32d17da4ea988ff421ce766672f", "dfa8b0c2cfd74de09903f89491f963cd", "22d3e13378b744efb0e276b45b31b01a", "31ed254564e140b9aac71c12594d2ca0", "0194c265e4c348c8b585b895b0a44053", "1b027902273346c28d099a109a2d9d04", "6a161b655884467b84dd4dd93911925e", "2cbda9ef7e6049aa8e8bbe36ac976851", "42912111a19e4e5cad81ea8ec83f0748", "1c7e46cf7eb54e33b5943ac7fedd90ab", "919aaa942b40441290b343f0fb3a8616", "55bbd21943374efe839427eff0b5ed6e", "e03459054e2443e8bd0efc47d9ec1564", "9b42807bd53f4b61b7e822529a17ffdb", "513ec36acba24f79a96febdacbf92cd7", "fea3a4f554354e358a6c9ecbbc0e1dd0", "31f81746181d42bbbabddd703550cddd", "92d6e5c883bb4f749f1f046d4c9d893d", "345f23b203884b43a35425e07bc4ca90", "187096ed17434c52aa0d6e9f2cf59bd7", "7dbd0c2d911645f49e19ab7f3dc14f63", "41ed4e60f2c740f38baba818ad0a846b", "74c81dc0b7514838b8d4577303218075", "6d656a2a46754177b1ca54f1a45f4b0e", "b3bb64087de74e58ac5fce9a1ff8047b", "6f2fc4b994664b16aa765fd746bd0d73", "72dda700dceb466fb99d1d343f355505", "9febe219fe1c41b197fafe4469e86749", "3befba6473854e9f992184be75d987cb", "9b906ebf97fd49d7b7e54b22dc4c8626", "2080aa3d70274442a710a0a717a91e36", "cd659af72ebd47be89b40df4beeda71a", "2fb045a62f2e471ab31551a55750e69c", "3dee2f86671c49b491f244a74516c20f", "4a199a09db394432bd02e140e3671b44", "01f4f783727343e7a1f21036ce2e709b", "2da23dce504a4017a4a3f14475535033", "04b23ffface948f498aa6599ecc382f4", "6c82a77b96b94cffa855677d067205a1", "b7eff77c6ece4d49a97c3acd109f8a0d", "92b38a3dfe2f466b9f335cf915eaad1a", "56385af0c2674ad4b0a5741cd119bba3", "8da33bf41b004281ae741ef076fa9b82", "f216ef0424a3447384a93ab55a6e6421", "0ef1862cb5844ccb94ca2b6a823c3c93", "837d752dd7fc4fb1baefcee549352ae8", "d2bf16ab46c342ba9ad7fe5eac85b38d", "2f75289d5dd34bc9b609b57279474b7a", "8b52a1e4846d4b1eba4899f53955e0a3", "fc61f0026b3a4241bdf36b97b5c6feae", "9d7c354952304805a094d08f146d7e52", "a9445a03b945429a90eddba30eb96868", "dcd00d32bdfa4b6ba982c518ae6e6d06", "ec5f9cef553643b8b54940d9a436fcbb", "222b41a4c77d4019827d5fca172c6b64", "2e0da32ba666437784d7a0202981a337", "84e3d631060b4a1a846ef00f990f5dd8", "30545108c6d94b3bbddd0ceaed4c6f61", "f5d06abb52fe4208912c6f76216e3d4e", "3460b504675d40a48f77c1a0589041dd", "a569ff7c0c8148ec8d262e823491b29f", "4e03e09b341b4485a2f6dba2ad8ca97d", "2980d9dbd47f48ed94cb40d28efa3b9b", "d0a1ad9d7bca478aa0ded0b35ff60e11", "d0382c1b18af4070b55d161bed8ea86d", "202f764307624044b16d481ad790a9e3", "b86ee6e277a84d6eb73b81ec9b3fc3eb", "655aaf7fea974f919734adc349a15eb6", "777d43887b784e209927b90a7ec54f67", "e7525b91db2241b987e2ce6ef0cbcff8", "29d7c8c37fa640e9a8054dacc9172d77", "e18fa6c4389142c58cfb205ddf941cf3", "1d861a0b7a594c1489dbc400051935d3", "f8b1903fe3ed4026af29a73a127323f2", "f38c302156da44cfb4205b775a49569a", "722a86b2c70746999cbef379b28831e9", "ef22e19dfe6e48d2ad44ebc30b26e66c", "b616fcb1a7f84509981f7b3af54d42d8", "3aa6af900c864f5782ee59892606b5f1", "099ac8662c634ea3acf4a227c9f1b8ac", "e232943d2fc14edf9b5dd645fb3e0583", "18cc96977d9147ac990d52f5d9c939e7", "42d0616f952f4db493710f67cbd4b9c6", "c927227a107a43b59ede52b241fb5084" ] } }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Index trained\n" ] }, { "output_type": "stream", "name": "stderr", "text": [ "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:88: UserWarning: \n", "The secret `HF_TOKEN` does not exist in your Colab secrets.\n", "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n", "You will be able to reuse this secret in all of your notebooks.\n", "Please note that authentication is recommended but still optional to access public models or datasets.\n", " warnings.warn(\n" ] }, { "output_type": "display_data", "data": { "text/plain": [ "modules.json: 0%| | 0.00/349 [00:00