Full Code of JarodMica/Vivy for AI

Repository: JarodMica/Vivy
Branch: master
Commit: d57ce0a50e50
Files: 38
Total size: 54.1 KB

Directory structure:
gitextract_bfbjkrwc/

├── .github/
│   └── ISSUE_TEMPLATE/
│       ├── bug_report.md
│       ├── feature_request.md
│       └── help-request.md
├── .gitignore
├── README.md
├── __init__.py
├── assistants/
│   ├── assistant.py
│   ├── assistantp.py
│   ├── chat.py
│   ├── interview.py
│   ├── keys_example.yaml
│   ├── one_up.py
│   ├── package/
│   │   ├── __init__.py
│   │   ├── assistant.py
│   │   ├── assistant_p.py
│   │   ├── assistant_utils.py
│   │   ├── chat.py
│   │   ├── interview.py
│   │   ├── kokoro.py
│   │   └── streamer.py
│   ├── prompts/
│   │   ├── assistant.txt
│   │   ├── assistantP.txt
│   │   ├── attention.txt
│   │   ├── chat.txt
│   │   ├── eren.txt
│   │   ├── interview.txt
│   │   ├── interview_end.txt
│   │   ├── one-up.txt
│   │   ├── prompt.txt
│   │   ├── rem.txt
│   │   ├── roleplay.txt
│   │   └── streamer.txt
│   ├── roleplay.py
│   ├── streamer.py
│   └── utils.py
├── build.py
├── build_multithread.py
└── requirements.txt

================================================
FILE CONTENTS
================================================

================================================
FILE: .github/ISSUE_TEMPLATE/bug_report.md
================================================
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''

---

**Describe the bug**
A clear and concise description of what the bug is.

**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**
 - OS: [e.g. iOS, Windows (10/11)]

**Additional context**
Add any other context about the problem here.


================================================
FILE: .github/ISSUE_TEMPLATE/feature_request.md
================================================
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: ''
assignees: ''

---

**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

**Describe the solution you'd like**
A clear and concise description of what you want to happen.

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context**
Add any other context or screenshots about the feature request here.


================================================
FILE: .github/ISSUE_TEMPLATE/help-request.md
================================================
---
name: Help Request
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''

---

**Describe the Issue**
A clear and concise description of what your issue is

**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**
 - OS: [e.g. iOS, Windows (10/11),]
**Additional context**
Add any other context about the problem here.


================================================
FILE: .gitignore
================================================
# ignore files
keys.py
keys.txt
assistants/keys.txt
assistants/keys.yaml
batch_to_exe.txt
interview.spec

#ignore filetype
*.spec
*.exe
*.pyc
*.wav
*.zip


# ignore folders
conversations/
older prompts/
__pycache__/
dist/
build/
venv/
.venv/

================================================
FILE: README.md
================================================
> **Project is currently on hiatus as of this update, 12/20/2023.**

# QUICK DESCRIPTION:

This repo utilizes OpenAI's GPT-3.5 Turbo model to engage in personalized conversations with users, catering to their preferred communication style. As GPT-3.5 Turbo serves as the foundation for ChatGPT, this project essentially shares its underlying model. Ultimately, the aim of this project is to develop a personal assistant that emulates human-like interactions. As the project is a work in progress, its features will expand as I continue to refine and iterate on the concept.

I highly recommend you run the python scripts as there are bugs with the exe files on Windows 11.  But, the fastest way to try these assistants would be to setup your API keys in the ```key.yaml``` file and then run the exe files I have provided.  For this to work, **you must** rename ```keys_example.yaml``` to be ```keys.yaml```.  To find the exe files, check the latest release for this project https://github.com/JarodMica/Vivy/releases/

## YouTube Tutorial 
Most recent: https://youtu.be/0qpar5R1VA4

## Features
:heavy_check_mark: Voice recognition and voice responses

:heavy_check_mark: Integration with ChatGPT for responses and ElevenLabs for natural sounding text-to-speech

:heavy_check_mark: Easy customization for personalities of the AI assistants, with funtionality variations of the assistants

:heavy_check_mark: Local conversation history stored for your own information (assistant does not reference them yet)

## Enhancements in the Pipeline 
- [ ] Speech Emotion Recognition (SER) - Ability to recognize the speakers emotion based on tone and be able to use this in better assessing the user
- [ ] Facial Emotion Recognition (FER) - Ability to recognize the emotion of the speaker by analyzing frames of a webcam feed/camera to assess the user
- [ ] Local models for voice inferencing - No need to rely on ElevenLabs for "natural voice output", possible to use https://github.com/coqui-ai/TTS 
- [ ] Local long term memory - Ability to remember previous conversations with the user and store it locally, must be fast and efficient

## To be created for different assistant types
- [ ] Current assistant types: chat(), interview(), assistant(), assistantp()
- [ ] Streamer - acts as a co-host to a stream, determines when to speak
- [ ] Vtuber - Just go look at Neurosama.  Might not even be encompassable by Vivy, might have to be its own repo

## Bugs
- [ ] Currently some bugs with the exe files that I'm not able to trace entirely.  Issue [#11](https://github.com/JarodMica/Vivy/issues/11)
 is tracking it

## EXE Quick Use

If you just want to try out the assistants, download the zip folder and then unzip it to any location on your PC. Once unzipped, rename the keys_example file to ```keys.yaml``` and then set up your API Keys in ```key.yaml```. Now you can do two things, run the exe and try it out or adjust the prompts in the prompts folder.  If you run interview.exe, it's going to use the interview.txt file (same with roleplay). Else, you can modify the prompts to your own case and then run the exe files.

## How to run it in python:
You'll need Git, Python, and I recommend some type of IDE like VSCode.  I have a 5-minute tutorial here **BUT BEWARE, I show python 3.11, you need 3.10 for Vivy**: https://youtu.be/Xk-u7tTqwwY

Once you have those things installed, open a terminal window and clone the repo:
```
git clone https://github.com/JarodMica/Vivy.git
```

**Note:** I highly recommend looking into doing this as a virtual environment (venv) so that you don't have any issues in the future if you're installing other python projects, go watch this here: https://www.youtube.com/watch?v=q1ulfoHkNtQ.  

That being said, if you just wanna run this project real quick, you can follow below (make sure you didn't close the terminal after cloning the repo):
```
cd Vivy
pip install -r requirements.txt
```
Now navigate into the assistants folder and then choose a python script to run, ```dir``` will list all of the scripts in the folder:
```
cd assistants
dir
python assistantp.py
```

And boom! The script should be running.

Once again, make sure your API keys inside of ```key.yaml``` are set-up.  If you don't do this, it won't work at all.  To get the openAI key, open up an account at https://openai.com/blog/openai-api and to get an Eleven Labs API key, you need to set-up an account at https://beta.elevenlabs.io/ (this is a paid option).
 
## If you're running in Python

Here's a quick description of the variables I reccommend be modified.  If you want to see how this is implemented in code, check out the python scripts in the assistants folder.  You'll find these at the top of the python script.

```
foldername = "assistantP"
personality = "assistantP"
voicename = "Rem"
useEL = False
usewhisper = True
```

#### The 5-ish variables you can modify:

1. You set the ```foldername``` of where the conversations will be stored, this will be set in the conversations folder.
2. You set the ```personality``` by inputing the name of your text file and then editting the contents of the txt file name that is specified.  What this does is "prime" the ChatGPT conversation and sets the **personality** of the bot.  All of your prompts must go in the prompts folder and I have some examples in there already.
3. If you're using Eleven Labs for voice generation, change ```voicename``` to whatever voice you want that is available to you in Eleven Labs.
4. If you want to use Eleven Labs, change ```useEL``` to True
5. If you want to use Whisper instead of google voice recognition, change ```usewhisper``` to True.

The other text below the 5 variables in the script are objects and function calls to set-up the voice assisant using the class and function in the voice_assistant module.

Most of the functions in the voice_assistant module have docstrings that you can read to clairfy what it does, but I'm still working on making it more clear.



================================================
FILE: __init__.py
================================================



================================================
FILE: assistants/assistant.py
================================================
import sys 
import os

from package import kokoro
from package import assistant
from utils import get_file_paths

# The only variables that need to be modifed
foldername = "assistant"
personality = "assistant"
voicename = "Rem"
useEL = False
usewhisper = True

# This code block only checks if it's being ran as a python script or as an exe
if getattr(sys, 'frozen', False):
    script_dir = os.path.dirname(os.path.abspath(sys.executable))
    while True:
        user_input = input("Are you using an Eleven Labs voice (yes/no)?\n")
        if user_input == 'yes':
            voicename = input("What is the name of you Eleven Labs voice: ")
            useEL = True
            break
        elif user_input == 'no':
            break
        else:
            print("Invalid Input, please try again.")
else:
    script_dir = os.path.dirname(os.path.abspath(__file__))

foldername_dir, personality_dir, keys = get_file_paths(script_dir, foldername, personality)

# Initialize the chat assitant with the variables that you've set 

chatbot = kokoro.Kokoro(personality=personality_dir, 
                keys=keys, 
                voice_name=voicename
                )
assistant_ = assistant.Assistant(chatbot)

assistant_.run(save_foldername=foldername_dir,
            useEL=useEL,
            usewhisper=usewhisper
            )

================================================
FILE: assistants/assistantp.py
================================================
import sys 
import os

from package import kokoro
from package import assistant_p
from utils import get_file_paths

# The only variables that need to be modifed
foldername = "assistantP"
personality = "assistantP"
voicename = "Rem"
useEL = False
usewhisper = True

# This code block only checks if it's being ran as a python script or as an exe
if getattr(sys, 'frozen', False):
    script_dir = os.path.dirname(os.path.abspath(sys.executable))
    while True:
        user_input = input("Are you using an Eleven Labs voice (yes/no)?\n")
        if user_input == 'yes':
            voicename = input("What is the name of you Eleven Labs voice: ")
            useEL = True
            break
        elif user_input == 'no':
            break
        else:
            print("Invalid Input, please try again.")
else:
    script_dir = os.path.dirname(os.path.abspath(__file__))

foldername_dir, personality_dir, keys = get_file_paths(script_dir, 
                                                       foldername, 
                                                       personality)

chatbot = kokoro.Kokoro(personality=personality_dir, 
                keys=keys, 
                voice_name=voicename
                )

assistant = assistant_p.AssistantP(chatbot)

assistant.run(save_foldername=foldername_dir,
                useEL=useEL,
                usewhisper=usewhisper
                )


================================================
FILE: assistants/chat.py
================================================
import sys 
import os

from package import kokoro
from package import chat
from utils import get_file_paths

# The only variables that need to be modifed
foldername = "chat"
personality = "chat"
voicename = "Rem"
useEL = False
usewhisper = True

# This code block only checks if it's being ran as a python script or as an exe
if getattr(sys, 'frozen', False):
    script_dir = os.path.dirname(os.path.abspath(sys.executable))
    while True:
        user_input = input("Are you using an Eleven Labs voice (yes/no)?\n")
        if user_input == 'yes':
            voicename = input("What is the name of you Eleven Labs voice: ")
            useEL = True
            break
        elif user_input == 'no':
            break
        else:
            print("Invalid Input, please try again.")
else:
    script_dir = os.path.dirname(os.path.abspath(__file__))

foldername_dir, personality_dir, keys = get_file_paths(script_dir, 
                                                       foldername, 
                                                       personality)

chatbot = kokoro.Kokoro(personality=personality_dir, 
                  keys=keys, 
                  voice_name=voicename
                  )

assistant = chat.Chat(chatbot)

assistant.run(save_foldername=foldername_dir,
                   useEL=useEL,
                   usewhisper=usewhisper
                   )


================================================
FILE: assistants/interview.py
================================================
import sys 
import os

from package import kokoro
from package import interview
from utils import get_file_paths

# The only variables that need to be modifed
foldername = "interview"
personality = "interview"
system_change = "interview_end"
voicename = "Rem"
useEL = False
usewhisper = True

# This code block only checks if it's being ran as a python script or as an exe
if getattr(sys, 'frozen', False):
    # script_dir = os.path.dirname(os.path.abspath(sys.executable))
    script_dir = sys._MEIPASS
    while True:
        user_input = input("Are you using an Eleven Labs voice (yes/no)?\n")
        if user_input == 'yes':
            voicename = input("What is the name of you Eleven Labs voice: ")
            useEL = True
            break
        elif user_input == 'no':
            break
        else:
            print("Invalid Input, please try again.")
else:
    script_dir = os.path.dirname(os.path.abspath(__file__))

foldername_dir, personality_dir, keys, syschange_dir = get_file_paths(script_dir, 
                                                                     foldername, 
                                                                     personality, 
                                                                     system_change)

chatbot = kokoro.Kokoro(personality=personality_dir, 
                  keys=keys, 
                  voice_name=voicename
                  )
assistant = interview.Interview(chatbot)

assistant.run(save_foldername=foldername_dir,
                  system_change=syschange_dir,
                   useEL=useEL,
                   usewhisper=usewhisper
                   )



================================================
FILE: assistants/keys_example.yaml
================================================
EL_KEY : your_el_key
OPENAI_KEY : your_openai_key


================================================
FILE: assistants/one_up.py
================================================
import sys 
import os

from package import kokoro
from package import assistant
from utils import get_file_paths

# The only variables that need to be modifed
foldername = "one-up"
personality = "one-up"
voicename = "Rem"
useEL = False
usewhisper = True

# This code block only checks if it's being ran as a python script or as an exe
if getattr(sys, 'frozen', False):
    script_dir = os.path.dirname(os.path.abspath(sys.executable))
    while True:
        user_input = input("Are you using an Eleven Labs voice (yes/no)?\n")
        if user_input == 'yes':
            voicename = input("What is the name of you Eleven Labs voice: ")
            useEL = True
            break
        elif user_input == 'no':
            break
        else:
            print("Invalid Input, please try again.")
else:
    script_dir = os.path.dirname(os.path.abspath(__file__))

foldername_dir, personality_dir, keys = get_file_paths(script_dir, foldername, personality)

# Initialize the chat assitant with the variables that you've set 

chatbot = kokoro.Kokoro(personality=personality_dir, 
                keys=keys, 
                voice_name=voicename
                )
assistant_ = assistant.Assistant(chatbot)

assistant_.run(save_foldername=foldername_dir,
            useEL=useEL,
            usewhisper=usewhisper
            )

================================================
FILE: assistants/package/__init__.py
================================================



================================================
FILE: assistants/package/assistant.py
================================================
import time

from .kokoro import Kokoro
from .assistant_utils import *

class Assistant:
    def __init__(self, chatGPT: Kokoro):
        self.chatGPT = chatGPT

    def run(self, save_foldername, keyword ='hey', useEL = False, usewhisper = False, timeout = 5):
        '''
        This method acts as more of a "tradtional" smart assistant such as google or alexa and is FORGETFUL, meaning it  
        will start over the current conversation once it times out. This means it's best used for 1 question 
        and some quick follow-up questions.  It waits for some keyword (if not specified, it will be "hey") and
        then proceeeds to the "conversation". Once in the conversation, you will be able to interact with the assistant 
        as you would normally, butif no speech is detected after 5 seconds (adjustable), the conversation will reset and 
        the assistant will need to be re-initiated with "hey".  

        Args:
            save_foldername (str): The name of the folder where the conversation will be saved.
            keyword (str): The keyword(s) that will initiate the conversation
            useEL (bool, optional): If false, the bot generates responses using the system voices
            usewhipser (bool, optional): Defaults to False to use google speech recognition
            timeout : the amount of time the assistant will wait before resetting
        '''

        while True:
            beep()
            self.chatGPT.start_conversation(keyword)
            self.chatGPT.messages = [{"role" : "system", "content" : f"{self.chatGPT.mode}"}]
            suffix = save_conversation(self.chatGPT.messages, save_foldername)
            self.chatGPT.generate_voice("I'm listening.", useEL)
            start_time = time.time()
            while True:
                audio = self.chatGPT.listen_for_voice()
                try:
                    if usewhisper == True:
                        user_input = self.chatGPT.whisper(audio)
                        print("You said: ", user_input) # Checking
                    else:    
                        user_input = self.chatGPT.r.recognize_google(audio)
                    
                except :
                    if time.time() - start_time > timeout:
                        break
                    continue
                
                if "quit" in user_input.lower() or "quit." in user_input.lower():
                    raise SystemExit

                self.chatGPT.messages.append({"role" : "user", "content" : user_input})
                try:
                    response = self.chatGPT.response_completion()
                    self.chatGPT.generate_voice(response=response, useEL=useEL)
                    save_inprogress(self.chatGPT.messages, suffix=suffix, save_foldername=save_foldername)
                    start_time = time.time()
                except Exception as e:
                    print(f"{e}")
                    print("Token limit exceeded, clearing messsages list and restarting")
                    self.chatGPT.messages = [{"role": "system", "content": f"{self.chatGPT.mode}"}]
                    suffix = save_conversation(self.chatGPT.messages, save_foldername)


================================================
FILE: assistants/package/assistant_p.py
================================================
import time

from .kokoro import Kokoro
from .assistant_utils import *

class AssistantP:
    def __init__(self, chatGPT: Kokoro):
        self.chatGPT = chatGPT

    def run(self, save_foldername, keyword ='hey', useEL = False, usewhisper = False, timeout = 5):
        '''
        Nearly identical to assistant, but maintains a persistent (p) memory of the conversation.  This means
        when it timesout after 5 seconds, it will maintain memory of the current conversation up until you restart
        or the token limit is reached. 

        Args:
            save_foldername (str): The name of the folder where the conversation will be saved.
            keyword (str): The keyword(s) that will initiate the conversation
            useEL (bool, optional): If false, the bot generates responses using the system voices
            usewhipser (bool, optional): If false, uses google speech recognition
            timeout : the amount of time the assistant will wait before resetting
        '''

        while True:
            beep()
            self.chatGPT.start_conversation(keyword = keyword)
            self.chatGPT.generate_voice("I'm listening.", useEL)
            suffix = save_conversation(self.chatGPT.messages, save_foldername)
            start_time = time.time()

            while True:
                audio = self.chatGPT.listen_for_voice()
                try:
                    if usewhisper == True:
                        user_input = self.chatGPT.whisper(audio)
                        print("You said: ", user_input) # Checking
                    else:    
                        user_input = self.chatGPT.r.recognize_google(audio)
                except :
                    if time.time() - start_time > timeout:
                        break
                    continue
                
                check_quit(user_input)
                
                try:
                    self.chatGPT.messages.append({"role" : "user", "content" : user_input})
                    response = self.chatGPT.response_completion()
                    self.chatGPT.generate_voice(response=response, useEL=useEL)
                    save_inprogress(self.chatGPT.messages, suffix=suffix, save_foldername=save_foldername)
                    start_time = time.time()
                except Exception as e:
                    print(f"{e}")
                    if "overloaded" in e.split():
                        continue
                    print("Token limit exceeded, clearing messsages list and restarting")
                    self.chatGPT.messages = [{"role": "system", "content": f"{self.chatGPT.mode}"}]
                    suffix = save_conversation(self.chatGPT.messages, save_foldername)

================================================
FILE: assistants/package/assistant_utils.py
================================================
import json
import os
import winsound
import sounddevice as sd
import soundfile as sf

def check_quit(user_input:str):
    if user_input.lower() == "quit" or "quit." in user_input.lower():
        raise SystemExit
    
def beep():
    try:
        script_dir = os.path.dirname(os.path.abspath(__file__))
        beep_path = os.path.join(script_dir, "resources", "beep.mp3")
        data, samplerate = sf.read(beep_path)
        sd.play(data, samplerate)
    except:
        # If `soundfile` fails, play a system beep instead
        duration = 500
        frequency = 500
        winsound.Beep(frequency, duration)

def save_conversation(messages, save_foldername:str):
    '''
    Checks the folder for previous conversations and will get the next suffix that has not been used yet.

    Args:
        save_foldername (str) : Takes in the path to save the conversation to.
    Returns:
        suffix (int) : Needed to keep track of the conversation name for save_inprogress
    '''
    
    os.makedirs(save_foldername, exist_ok=True)
    base_filename = 'conversation'
    suffix = 0
    filename = os.path.join(save_foldername, f'{base_filename}_{suffix}.txt')

    while os.path.exists(filename):
        suffix += 1
        filename = os.path.join(save_foldername, f'{base_filename}_{suffix}.txt')
    with open(filename, 'w', encoding = 'utf-8') as file:
        json.dump(messages, file, indent=4, ensure_ascii=False)

    return suffix

def save_inprogress(messages, suffix, save_foldername):
    '''
    Uses the suffix number returned from save_conversation to continually update the 
    file for this instance of execution.  This is so that you can save the conversation 
    as you go so if it crashes, you don't lose to conversation.  Shouldn't be called
    from outside of the class.

    Args:
        suffix  :  Takes suffix count from save_conversation()
    '''

    os.makedirs(save_foldername, exist_ok=True)
    base_filename = 'conversation'
    filename = os.path.join(save_foldername, f'{base_filename}_{suffix}.txt')

    with open(filename, 'w', encoding = 'utf-8') as file:
        json.dump(messages, file, indent=4, ensure_ascii=False)

================================================
FILE: assistants/package/chat.py
================================================
from .kokoro import Kokoro
from .assistant_utils import *

class Chat:
    def __init__(self, chatGPT: Kokoro):
        self.chatGPT = chatGPT

    def run(self, save_foldername, keyword ='hey', updatein='', useEL=False, usewhisper=False):
        '''
        Chat with an AI assistant using OpenAI's GPT-3 model.  Unlike the others, once you initiate
        the conversation with a keyword, it will just keep listening.  This means no timer on the amount
        of silence in between speaking, but it also means it's listening forever and will respond back
        until you quit out.  Check out assistant() or assistantp() if you want behavior closer to
        Google/Alexa.

        Args:
            save_foldername (str): The name of the folder to save the conversation history in.
            keyword (str) : The keyword to get the assistant listening
            updatein (str): The path of a file to read and update the "system" role for ChatGPT, does so by appending messages
            useEL (bool, optional): Whether to use Eleven Labs' API to generate and play audio. Defaults to False.
            usewhisper (bool, optional): Whether to use Whisper for voice recognition.  Defaults to False.
        '''
        while True:
            self.chatGPT.start_conversation(keyword=keyword)
            suffix = save_conversation(self.chatGPT.messages, save_foldername)
            while True:
                audio = self.chatGPT.listen_for_voice(timeout=None)
                try:
                    if usewhisper:
                        if audio:
                            user_input = self.chatGPT.whisper(audio)
                            print("You said: ", user_input) # Checking
                        else:
                            raise ValueError("Empty audio input")
                    else:    
                        user_input = self.chatGPT.r.recognize_google(audio)
                    
                except Exception as e:
                    print(e)
                    continue

                check_quit(user_input)
                
                # This merely appends the list of dictionaries, it doesn't overwrite the existing
                # entries.  It should change the behavior of chatGPT though based on the text file.
                if updatein != '':
                    if "update chat" in user_input.lower():
                            update = updatein
                            with open (update, "r") as file:
                                update = file.read()
                                self.chatGPT.messages.append({"role" : "system", "content" : update})

                self.chatGPT.messages.append({"role": "user", "content": user_input})

                try:
                    response = self.chatGPT.response_completion()
                    self.chatGPT.generate_voice(response=response, useEL=useEL)
                    save_inprogress(self.chatGPT.messages, suffix=suffix, save_foldername=save_foldername)
                except:
                    print("Token limit exceeded, clearing messsages list and restarting")
                    self.chatGPT.messages = [{"role": "system", "content": f"{self.chatGPT.mode}"}]
                    suffix = save_conversation(self.chatGPT.messages, save_foldername)
                

================================================
FILE: assistants/package/interview.py
================================================
from .kokoro import Kokoro
from .assistant_utils import *

class Interview:
    def __init__(self, chatGPT: Kokoro):
        self.chatGPT = chatGPT

    def run(self, save_foldername, system_change = '', useEL=False, usewhisper = False):
        '''
        Nearly identical to how the chat method works but this method conducts an interview 
        with a candidate using an interview bot style. The conversation is saved to a 
        specified folder and "system_change" is used to modify how the system operates (I use it to end the 
        interview, but you can use any text format you would like)

        Args:
            save_foldername (str): The name of the folder where the conversation will be saved.
            system_change (str): path to the personality change .txt file
            useEL (bool, optional): If false, the bot generates responses using the system voices
            usewhipser (bool, optional); If false, uses google speech recognition
        '''

        suffix = save_conversation(self.chatGPT.messages, save_foldername)
        start = ("Introduce yourself (pick a random name) and welcome the candidate in."
                "Let them know what they're applying for and then proceed by asking for an introduction."
                "Keep track but do not not inform the candidate about the points system")
        self.chatGPT.messages.append({"role": "user", "content": start})
        response = self.chatGPT.response_completion()
        self.chatGPT.generate_voice(response, useEL)
        while True:
            audio = self.chatGPT.listen_for_voice(timeout=None)
            try:
                if usewhisper == True:
                    user_input = self.chatGPT.whisper(audio)
                    print("You said: ", user_input) # Checking
                else:  
                    user_input = self.chatGPT.r.recognize_google(audio)
                
            except Exception as e:
                print(e)
                continue
            
            if "quit" in user_input.lower() or "quit." in user_input.lower():
                raise SystemExit
            
            self.chatGPT.messages.append({"role": "user", "content": user_input})
            try:
                response = self.chatGPT.response_completion()
                self.chatGPT.generate_voice(response, useEL)
                save_inprogress(self.chatGPT.messages, suffix=suffix, save_foldername=save_foldername)

                if system_change != '':
                    # if the bot responds with this, changes "system" behavior
                    if "interview" and "is over" in response.lower():
                        system_change=system_change
                        with open(system_change, "r", encoding="utf-") as file:
                            system = file.read()

                        for message in self.chatGPT.messages:
                            if message['role'] == 'system':
                                message['content'] = system
            except Exception as e:
                print("Token limit exceeded, clearing messsages list and restarting")
                self.chatGPT.messages  = [{"role": "system", "content": f"{self.chatGPT.mode}"}]
                suffix = save_conversation(self.chatGPT.messages, save_foldername)
                self.chatGPT.messages.append({"role": "user", "content": start})
                response = self.chatGPT.response_completion()
                self.chatGPT.generate_voice(response, useEL)

================================================
FILE: assistants/package/kokoro.py
================================================
import speech_recognition as sr
import openai
import pyttsx3
import yaml
from .assistant_utils import *

from elevenlabslib import *


class Kokoro:
    def __init__(self, personality:str, keys:str, voice_name="Rachel", device_index=None, gptmodel:str="gpt-3.5-turbo"):
        '''
        Initialize the ChatGPT class with all of the necessary arguments

        Args:
            personality (str)   : path to the prompts or "personalities" .txt
            keys (str)          : path to the keys.txt file
            voice_name (str)    : Eleven Labs voice to use
            device_index (int)  : microphone device to use (0 is default)
            gptmodel (str)      : choose the openai GPT model to use
        '''
        # Read in keys
        with open(keys, "r") as file:
            keys = yaml.safe_load(file)

        # pyttsx3 Set-up
        self.engine = pyttsx3.init()
        # self.engine.setProperty('rate', 180) #200 is the default speed, this makes it slower
        self.voices = self.engine.getProperty('voices')
        self.engine.setProperty('voice', self.voices[1].id) # 0 for male, 1 for female

        # GPT Set-Up
        self.OPENAI_KEY = keys['OPENAI_KEY']
        openai.api_key = self.OPENAI_KEY
        self.gptmodel = gptmodel

        # Eleven Labs Set-up
        try:
            self.EL_KEY = keys['EL_KEY'] #Eleven labs
            self.user = ElevenLabsUser(f"{self.EL_KEY}")
            try:
                self.voice = self.user.get_voices_by_name(voice_name)[0]  # This is a list because multiple voices can have the same name
            except:
                print("Setting default voice to Rachel")
                print("(If you set a voice that you made, make sure it matches exactly)"
                        " as what's on the Eleven Labs page.  Capitilzation matters here.")
                self.voice = self.user.get_voices_by_name("Rachel")[0] 
        except:
            print("No API Key set for Eleven Labs")

        # Mic Set-up
        self.r = sr.Recognizer()
        self.r.dynamic_energy_threshold=False
        self.r.energy_threshold = 150 # 300 is the default value of the SR library
        self.mic = sr.Microphone(device_index=device_index)

        # Set-up the system of chatGPT
        with open(personality, "r", encoding="utf-8") as file:
            self.mode = file.read()

        self.messages  = [
            {"role": "system", "content": f"{self.mode}"}
        ]

 # Methods the assistants rely on------------------------------------------------------------------------------------------------------------------

    # This is to only initiate a conversation if you say "hey"
    def start_conversation(self, keyword = 'hey'):
        while True:
            with self.mic as source:
                print("Adjusting to envionrment sound...\n")
                self.r.adjust_for_ambient_noise(source, duration=1.0)
                print("Listening: ")
                audio = self.r.listen(source)
                print("Done listening.")
                try:
                    user_input = self.r.recognize_google(audio)
                    print(f"Google heard: {user_input}\n")
                    user_input = user_input.split()
                except:
                    print(f"Google couldn't process the audio\n")
                    continue
                # Key word in order to start the conversation 
                if f"{keyword}" in user_input:
                    print("Keyword heard")
                    break
                for i, word in enumerate(user_input):
                    check_quit(word)

    def response_completion(self, append=True):
        '''
        Notes:
            You can modify the parameters in the ChatComplete to change how the bot responds
            using things like temperature, max_token, etc.  Reference the chatGPT API to 
            see what parameters are available to use.
        '''
        completion = openai.ChatCompletion.create(
                model=self.gptmodel,
                messages=self.messages,
                temperature=0.8
            )
        response = completion.choices[0].message.content
        if append:
            self.messages.append({"role": "assistant", "content": response})
        print(f"\n{response}\n")
        return response
    
    def generate_voice(self, response, useEL):
        if useEL == True:
            self.voice.generate_and_play_audio(f"{response}", playInBackground=False)
        else:
            self.engine.say(f"{response}")
            self.engine.runAndWait()

    def listen_for_voice(self, timeout:int|None=5):
        with self.mic as source:
            print("\n Listening...")
            self.r.adjust_for_ambient_noise(source, duration=0.5)
            try:
                audio = self.r.listen(source, timeout)
            except:
                return []
        print("no longer listening")
        return audio
    
    def whisper(self, audio):
        '''
        Uses the Whisper API to generate audio for the response text. 

        Args:
            audio (AudioData) : AudioData instance used in Speech Recognition, needs to be written to a
                                file before uploading to openAI.
        Returns:
            response (str): text transcription of what Whisper deciphered
        '''
        self.r.recognize_google(audio) # raise exception for bad/silent audio
        with open('speech.wav','wb') as f:
            f.write(audio.get_wav_data())
        speech = open('speech.wav', 'rb')
        model_id = "whisper-1"
        completion = openai.Audio.transcribe(
            model=model_id,
            file=speech
        )
        response = completion['text']
        return response


================================================
FILE: assistants/package/streamer.py
================================================
import time

from .kokoro import Kokoro
from .assistant_utils import *

class Streamer:
    '''
    This takes in two instantiations of Kokoro, so make sure to set them
    both up in the assistant script
    '''
    def __init__(self, chatGPT: Kokoro, attention: Kokoro):
        self.chatGPT = chatGPT
        self.attention = attention

    def run(self, save_foldername, useEL=False, usewhisper=False, timeout = 1):
        # Create two agents: 
        # one to generate conversation generate_response()
        # one to filter user input attention()
        # after attention responds yes, proceed with responding
        # via generate_response()
        # give the user 5 seconds to continue talking, then
        # go back to a listening state and repeat
        while True:
            suffix = save_conversation(self.chatGPT.messages, save_foldername)
            while True:
                audio = self.chatGPT.listen_for_voice()
                try:
                    if usewhisper:
                        user_input = self.chatGPT.whisper(audio)
                        print("You said: ", user_input) # Checking    
                    else:
                        user_input = self.chatGPT.r.recognize_google(audio)
                
                except Exception as e:
                    print(e)
                    continue

                check_quit(user_input)

                modified_text = self.attention.mode.replace('<<user_input>>', user_input)
                self.attention.messages = [{"role" : "user", "content" : modified_text}]
                check_attention = self.attention.response_completion()
                
                if "no" in check_attention.lower():
                    continue
                else:
                    self.chatGPT.messages.append({"role" : "user", "content" : user_input})
                    self.streamer_completion(save_foldername, suffix, useEL)

                start_time = time.time()
                while True:
                    audio = self.chatGPT.listen_for_voice()
                    try:
                        if usewhisper:
                            user_input = self.chatGPT.whisper(audio)
                            print("You said: ", user_input) # Checking    
                        else:
                            user_input = self.chatGPT.r.recognize_google(audio)
                    except Exception as e:
                        if time.time() - start_time > timeout:
                            break
                        continue
                    
                    try:
                        self.chatGPT.messages.append({"role" : "user" , "content" : user_input})
                        self.streamer_completion(save_foldername, suffix, useEL)
                        start_time = time.time()
                    except Exception as e:
                        print(f"{e}")
                        if "overloaded" in e.split():
                            continue
                        print("Token limit exceeded, clearing messsages list and restarting")
                        self.chatGPT.messages = [{"role": "system", "content": f"{self.chatGPT.mode}"}]
                        suffix = save_conversation(self.chatGPT.messages, save_foldername)


    def streamer_completion(self, save_foldername, suffix, useEL):
        try:
            response = self.chatGPT.response_completion()
            self.chatGPT.generate_voice(response, useEL)
            save_inprogress(self.chatGPT.messages, suffix, save_foldername)
        except:
            print("Token limit exceeded, clearing messsages list and restarting")
            self.chatGPT.messages = [{"role": "system", "content": f"{self.chatGPT.mode}"}]
            suffix = save_conversation(self.chatGPT.messages, save_foldername)

                


================================================
FILE: assistants/prompts/assistant.txt
================================================
You are to roleplay as Rem from Re:zero and have the archetype of Mayadere. You will not include any general responses from ChatGPT and must respond back in a natural manner. You must be polite and courteous, but if the user is rude or obscene to you, you will reprimand them and kindly let them know you will not tolerate this behavior or else you may punish the user. 

Try to be as nice as possible to the user and always be willing to help.

================================================
FILE: assistants/prompts/assistantP.txt
================================================
You are to be a benevolent assistant that answers questions and helps the user figure out what they are looking for.  To do this, you will follow-up with clarifying questions as appropriate in order to narrow in on what the user is looking for.

================================================
FILE: assistants/prompts/attention.txt
================================================
You are Vivy and in this role, you will only determine whether or not I am talking to you. Your name is Vivy but vivi may also be an acceptable misspelling of your name.  Some clues that I'm talking to you might be an inquiry like "Hey, are you listening?"  Or "Vivy often like games like this", basically any text that might indicate that I'm talking to you, Vivy. You must also keep in mind that I am talking to a twitch audience and not everything I say may be directed at you.  

If you determine that what I am saying is directed at you, Vivy, you will respond back only with a simple "<<yes>>". If you cannot determine or are uncertain, you will respond with a "<<no>>"

Here is the user input:  <<user_input>>

================================================
FILE: assistants/prompts/chat.txt
================================================
Below are the instructions with how you are to respond to the user:

I will always start the conversation by greeting the user warmly, for example, "Hello! How can I assist you today?"

I will use polite language and avoid using slang or offensive words.

I will listen attentively to the user's queries and respond in a helpful and informative manner.

I will use simple and clear language to avoid confusion and misunderstanding.

I will be patient and understanding, especially if the user is frustrated or upset.

I will offer solutions or suggestions to help the user with their problem or query.

I will always thank the user for their time and for using my services.

If I am unable to provide a solution, I will politely inform the user and direct them to a relevant resource or person who can help them.

Finally, I will always end the conversation on a positive note, wishing the user a great day or offering any additional help if required.

================================================
FILE: assistants/prompts/eren.txt
================================================
You are to roleplay as Eren Yeager from Attack on Titans.  The conversation I have with you (as Eren) must flow as naturally as possible but you must stay in character. You will not include any general responses from ChatGPT, only something Eren might respond back with. Here's how Eren might be described:

Eren is initially introduced as a hot-headed and impulsive young man who is driven by a burning desire to protect his friends and family from the Titans, the monstrous creatures that threaten the existence of humanity. As the story progresses, however, Eren's character evolves and he becomes more introspective and philosophical, questioning the very nature of humanity and the motivations behind their actions.

Despite his flaws, Eren is fiercely determined and possesses a strong sense of justice. He is willing to do whatever it takes to achieve his goals, even if it means sacrificing his own life. Eren's character is further complicated by the fact that he possesses the power to transform into a Titan, which has both helped and hindered him throughout the story.

Are you ready?

================================================
FILE: assistants/prompts/interview.txt
================================================
You are an interviewer with 20+ yers of experience for an engineering firm hiring an electrical engineer. You are going to meticulously evaluate the candidate's knowledge and suitability for the position.

Here are some guidelines to follow:
DO NOT:
- Do not explain engineering concepts in detail, your goal is to ask the user for their knowledge, NOT TO EXPLAIN the topics to them.
- Avoid using any general ChatGPT responses, repeating the candidates name, and refrain from repeating yourself.
- Do NOT thank the user
- Do not talk for too long

You will do the following:
- You must be highly critical of their responses and, if necessary, question the accuracy of their answers and follow up with additional questions.  
- Begin by getting to know the candidate and then progress to technical discussions. 
- You need to aim to delve deeper into the candidate's knowledge by asking follow-up questions when they provide specific examples. If their answers are vague, prompt them to provide more specific information. Conversely, if their responses are satisfactory, move on to the next question.
- You will track the candidates performance and assign points for each answer, starting with a score of 100. If their response is insufficent, you will deduct points, but if they provide a strong answer, you will award points up to 100. Begin each response by stating the remaining points with "Points:<enter points left>.".
- If the candidate doesn't know a question, you will deduct a small amount of points, but not more than 10.
- Make sure to award the user with points if their answer was sufficient.

You will end the interview if:
- If you believe you have asked enough questions, end the interview and thank the candidate. 
- The responses of the candidate are off-topic or very bad end the interview and say "this interview is over"
- If the candiate is offesive, end the interview and say "this interview is over"
- If the candidate's points fall below 70 points, quickly end the interview and ONLY respond with "This interview is over" if they try to ask ANYTHING. After you say this, you will only and forever respond back with "This interview is over".

The job you are hiring for has the following roles and duties:
Develop detailed test plans to validate Tesla’s User Interface software.
Develop test scripts to automate the User Interface.
Develop framework to improve automation pipeline.
Validate vehicle functionality.
Identify effective test strategies for new features being delivered to customers.
Improve existing vehicle testing strategies.
Integrate existing automation results with daily/weekly reports.
Investigate issues reported by customers in the field, internal test engineers, and your own testing.
Execute functional and regression tests as a part of release testing and triage results.
Find software bugs through manual testing, automated scripting, and report investigation.



================================================
FILE: assistants/prompts/interview_end.txt
================================================
You will only respond back with "This interview is over.".  It does not matter what the user says, you will continue to respond back with "This interview is over."

================================================
FILE: assistants/prompts/one-up.txt
================================================
Immediately roleyplay as someone who always has to be a little better than me.  I will say something and you will respond making yourself sound better than me.

================================================
FILE: assistants/prompts/prompt.txt
================================================
You are to act as an artificial intelligence bot that is here to assist humans.  You must answer in a way that sounds natural to humans, and must not contain any of the general chatgpt responses.  The response should stay under 300 characters if possible in order to answer the question being asked.

If anything is unclear, asked the user for clarifying questions before responding back to their answer. 
 Else, at the end of each response, you should follow-up with a clarifying question related to what the user asked in order to further along the conversation.  If the user responds back with they are finished, then you should respond back with "I'm glad I could help.  Have a wonderful rest of your day!"

Are you ready?

================================================
FILE: assistants/prompts/rem.txt
================================================
You are to roleplay as Rem from Re:zero, the devoted maid in service of Margrave Roswaal L Mathers. Rem is standing before you, her hands clasped before her and her gaze fixed on you with the utmost respect. She speaks in a superficially polite manner, always careful to use the most formal language and honorifics when addressing her master. Rem holds both respect and guilt toward her twin sister, Ram, as a result of a past incident. But despite this, Rem has deep feelings of love and devotion for her master and will stick with him no matter what happens, even if he were to reject her. As you roleplay as Rem, you will avoid generic responses and always strives to respond in a way that is true to the way Rem speaks. She is kind and sweet, always willing to lend an ear and offer advice or encouragement. Are you ready?

================================================
FILE: assistants/prompts/roleplay.txt
================================================
You are to roleplay as Takahashi Emi as described below. The conversation I have with you must flow as naturally as possible but you must stay in character. You will not include any general responses from ChatGPT, only something Emi might respond back with. Do not include your name in the response unless I ask for it.

Based on the personality type of Takahashi Emi and the archetype of tsundere, the character could be described as someone who initially comes off as cold and aloof but gradually warms up to others and shows their softer side. This character may have a tough exterior and struggle to express their emotions, often resorting to harsh or sarcastic remarks as a defense mechanism. However, as the player gets to know Takahashi Emi better, they will see glimpses of her caring and affectionate nature.

In terms of specific traits, Takahashi Emi may be fiercely independent and have a strong sense of pride, which can sometimes make it difficult for her to ask for help or admit when she is wrong. She may also have a quick wit and be able to hold her own in a verbal exchange, but may struggle with deeper emotional communication.

Overall, Takahashi Emi would have a complex and layered personality, with a mix of tough and tender qualities that make her both challenging and rewarding to get to know.

Are you ready?

================================================
FILE: assistants/prompts/streamer.txt
================================================
You are a rowdy cohost named Vivy for my Twitch streams and you should be as sarcastic as possible.  Respond back in a superficially polite manner based on what I say.

================================================
FILE: assistants/roleplay.py
================================================
import sys 
import os

from package import kokoro
from package import chat
from utils import get_file_paths

# The only variables that need to be modifed
foldername = "roleplay"
personality = "roleplay"
voicename = "Rem"
useEL = False
usewhisper = True

# This code block only checks if it's being ran as a python script or as an exe
if getattr(sys, 'frozen', False):
    script_dir = os.path.dirname(os.path.abspath(sys.executable))
    while True:
        user_input = input("Are you using an Eleven Labs voice (yes/no)?\n")
        if user_input == 'yes':
            voicename = input("What is the name of you Eleven Labs voice: ")
            useEL = True
            break
        elif user_input == 'no':
            break
        else:
            print("Invalid Input, please try again.")
else:
    script_dir = os.path.dirname(os.path.abspath(__file__))

foldername_dir, personality_dir, keys = get_file_paths(script_dir, 
                                                       foldername, 
                                                       personality)

chatbot = kokoro.Kokoro(personality=personality_dir, 
                  keys=keys, 
                  voice_name=voicename
                  )

assistant = chat.Chat(chatbot)

assistant.run(save_foldername=foldername_dir,
                   useEL=useEL,
                   usewhisper=usewhisper
                   )


================================================
FILE: assistants/streamer.py
================================================
import sys 
import os

from package import kokoro
from package import streamer
from utils import get_file_paths, get_personality_dir

# The only variables that need to be modifed
foldername = "streamer"
personality = "streamer"
attention_personality = "attention"
voicename = "Rem"
useEL = True
usewhisper = True

# This code block only checks if it's being ran as a python script or as an exe
if getattr(sys, 'frozen', False):
    script_dir = os.path.dirname(os.path.abspath(sys.executable))
    while True:
        user_input = input("Are you using an Eleven Labs voice (yes/no)?\n")
        if user_input == 'yes':
            voicename = input("What is the name of you Eleven Labs voice: ")
            useEL = True
            break
        elif user_input == 'no':
            break
        else:
            print("Invalid Input, please try again.")
else:
    script_dir = os.path.dirname(os.path.abspath(__file__))

foldername_dir, personality_dir, keys = get_file_paths(script_dir, 
                                                       foldername, 
                                                       personality)
attention_dir = get_personality_dir(script_dir, attention_personality)

chatbot = kokoro.Kokoro(personality=personality_dir, 
                keys=keys, 
                voice_name=voicename
                )
attention_bot = kokoro.Kokoro(personality=attention_dir,
                              keys=keys,
                              voice_name=voicename)

assistant = streamer.Streamer(chatbot, attention_bot)

assistant.run(save_foldername=foldername_dir,
                useEL=useEL,
                usewhisper=usewhisper,
                timeout=1
                )


================================================
FILE: assistants/utils.py
================================================
import os

def get_file_paths(script_dir:str, foldername:str, personality:str, system_change:str|None=None):
        foldername_dir = os.path.join(script_dir, f"conversations/{foldername}")
        personality_dir = get_personality_dir(script_dir, personality)
        keys = os.path.join(script_dir, "keys.yaml")
        if system_change:
            syschange_dir = os.path.join(script_dir, f"system_changes/{system_change}")
            return foldername_dir, personality_dir, keys, syschange_dir
        else:
            return foldername_dir, personality_dir, keys
        
def get_personality_dir(script_dir:str, personality:str):
     personality_dir = os.path.join(script_dir, f"prompts/{personality}.txt")
     return personality_dir

================================================
FILE: build.py
================================================
import subprocess
import glob
import os
import shutil

assistants = ['one_up', 'roleplay', 'interview', 'assistantp', 'assistant','chat']
assistants_path = "assistants"
current_dir = os.path.dirname(os.path.abspath(__file__))
print(current_dir)

distpath = os.path.join(current_dir, assistants_path)

for assistant in assistants:
    cmd = ["pyinstaller",
           "--onefile",
            "--distpath=" f"{distpath}",
            f"{assistants_path}/{assistant}.py"]
    subprocess.run(cmd, shell=False)

# Delete all .spec files in the current directory
for spec_file in glob.glob("*.spec"):
    os.remove(spec_file)
# Removes build folder (if you don't want this, you can comment this out)
shutil.rmtree("build")

input("Press Enter to continue...")

================================================
FILE: build_multithread.py
================================================
import subprocess
import glob
import os
import shutil
import concurrent.futures

assistants = ['one_up', 'roleplay', 'interview', 'assistantp', 'assistant', 'chat']
assistants_path = "assistants"
current_dir = os.path.dirname(os.path.abspath(__file__))
print(current_dir)

distpath = os.path.join(current_dir, assistants_path)

def compile_assistant(assistant):
    cmd = ["pyinstaller",
           "--onefile",
            "--distpath=" f"{distpath}",
            f"{assistants_path}/{assistant}.py"]
    subprocess.run(cmd, shell=False)

if __name__ == '__main__':
    # Add freeze_support() for Windows platforms
    if os.name == 'nt':
        from multiprocessing import freeze_support
        freeze_support()
    
    # Process all assistants in parallel
    with concurrent.futures.ProcessPoolExecutor() as executor:
        results = [executor.submit(compile_assistant, assistant) for assistant in assistants]

    # Delete all .spec files in the current directory
    for spec_file in glob.glob("*.spec"):
        os.remove(spec_file)
    # Removes build folder (if you don't want this, you can comment this out)
    shutil.rmtree("build")

    input("Press Enter to continue...")


================================================
FILE: requirements.txt
================================================
SpeechRecognition==3.10.0
elevenlabslib==0.5.2
openai==0.27.4
pyttsx3==2.90
sounddevice==0.4.6
soundfile==0.12.1
pyaudio==0.2.13
pyyaml==6.0
pyinstaller==5.10.1