[ { "path": "README.md", "content": "# CoAgents\nThe code for \"cooperative and interactive agents framework\"\n\nWe propose the ConAgents, a **Co**operative and i**n**teractive **agents** framework . The grounding agent generates tool-use instructions while execution agent executes selected tools. The observing agent extracts relevant values from execution results. When receiving feedback from the tool environment, the execution agent and observing agent adapt themselves iteratively.\n\n![image-20240324212748206](./asset/overall.png)\n\n![image-20240324212928539](./asset/main.png)\n\n\n\n# Quick start\n\n- install the main library via `pip install`\n\n```txt\nlangchain==0.0.338 # new version is also allowed\ncolorama==0.4.6\ntiktoken==0.5.1\nopenai==1.7.1\n```\n\n\n\n- register the TMDB platform to get the access token to access the TMDB APIs. \n Once you get the access token, please make a new file named `access_token.txt ` and add your token into this file.\n\n \n\n- get the OpenAI API to use the `gpt-3.5-turbo` (or other models)\n\n Once your get the APIs, please add your APIs into the list in `./utilize/apis.py`.\n\n ```python\n api_keys_list = [\n 'add your APIs'\n ]\n ```\n\n You can add multiple APIs since we will randomly select one APIs from `api_keys_list` to request the OpenAI backend for each API request. The Random selection strategy is employed to improve the quota limitation for one API (see the OpenAI quota for the details).\n\n You can also configure the base URL for your OpenAI APIs\n\n ```python\n BASE_URL = 'customize your own API base url' # https://api.openai.com/v1\n ```\n\n\n\n- Run our demo `run_tmdb.py`\n\n```txt\npython run_tmdb.py \\\n--model_name 'gpt-3.5-turbo' \\\n--log_file './log_file.json' \\\n--data_file \\\n--access_token_file \n```\n\n\n\n# Todo\n\nWe will add more details as soon as possible.\n\n# citation\n\n```txt\n@article{shi2024learning,\n title={Learning to Use Tools via Cooperative and Interactive Agents},\n author={Shi, Zhengliang and Gao, Shen and Chen, Xiuyi and Yan, Lingyong and Shi, Haibo and Yin, Dawei and Chen, Zhumin and Ren, Pengjie and Verberne, Suzan and Ren, Zhaochun},\n journal={arXiv preprint arXiv:2403.03031},\n year={2024}\n}\n```\n" }, { "path": "access_token.txt", "content": "" }, { "path": "dataset/tmdb.json", "content": "{\n \"data\": [\n {\n \"query\": \"give me the number of movies directed by Sofia Coppola\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/person\",\n \"GET https://api.themoviedb.org/3/person/{person_id}/movie_credits\"\n ],\n \"q_id\": 1\n },\n {\n \"query\": \"Who was the lead actor in the movie The Dark Knight?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/credits\"\n ],\n \"q_id\": 2\n },\n {\n \"query\": \"Who directed the top-1 rated movie?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/movie/top_rated\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/credits\"\n ],\n \"q_id\": 3\n },\n {\n \"query\": \"give me a image for the collection Star Wars\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/collection\",\n \"GET https://api.themoviedb.org/3/collection/{collection_id}/images\"\n ],\n \"q_id\": 4\n },\n {\n \"query\": \"What is the logo of the Walt Disney?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/company\",\n \"GET https://api.themoviedb.org/3/company/{company_id}/images\"\n ],\n \"q_id\": 5\n },\n {\n \"query\": \"What dose the lead actor of Titanic look like?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/credits\",\n \"GET https://api.themoviedb.org/3/person/{person_id}/images\"\n ],\n \"q_id\": 6\n },\n {\n \"query\": \"give me some reviews of the first movie that is similar to Titanic\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/similar\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/reviews\"\n ],\n \"q_id\": 7\n },\n {\n \"query\": \"I just finished watching Titanic and I want some other movie recommendations\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/recommendations\"\n ],\n \"q_id\": 8\n },\n {\n \"query\": \"What is the latest movie directed by Christopher Nolan?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/person\",\n \"GET https://api.themoviedb.org/3/person/{person_id}/movie_credits\"\n ],\n \"q_id\": 9\n },\n {\n \"query\": \"tell me the highest rated movie directed by Martin Scorsese\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/person\",\n \"GET https://api.themoviedb.org/3/person/{person_id}/movie_credits\"\n ],\n \"q_id\": 10\n },\n {\n \"query\": \"Who is the director of Leonardo DiCaprio's latest movie?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/person\",\n \"GET https://api.themoviedb.org/3/person/{person_id}/movie_credits\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/credits\"\n ],\n \"q_id\": 11\n },\n {\n \"query\": \"tell me a TV show recently directed by Catherine Hardwicke\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/person\",\n \"GET https://api.themoviedb.org/3/person/{person_id}/tv_credits\"\n ],\n \"q_id\": 12\n },\n {\n \"query\": \"Please recommend me some TV shows similar to Breaking Bad\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/recommendations\"\n ],\n \"q_id\": 13\n },\n {\n \"query\": \"Who is the director of today's most trending movie?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/trending/{media_type}/{time_window}\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/credits\"\n ],\n \"q_id\": 14\n },\n {\n \"query\": \"Who is the director of the movie \\\"Twilight\\\"?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/credits\"\n ],\n \"q_id\": 15\n },\n {\n \"query\": \"give me a photo belong to the second episode of the first season of the Witcher\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/season/{season_number}/episode/{episode_number}/images\"\n ],\n \"q_id\": 16\n },\n {\n \"query\": \"What is top-1 rated movie?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/movie/top_rated\"\n ],\n \"q_id\": 17\n },\n {\n \"query\": \"What is the release date of the movie \\\"The Matrix\\\"?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/release_dates\"\n ],\n \"q_id\": 18\n },\n {\n \"query\": \"Who is the most popular person?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/person/popular\"\n ],\n \"q_id\": 19\n },\n {\n \"query\": \"Give me some cover images of movies directed by Christopher Nolan.\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/person\",\n \"GET https://api.themoviedb.org/3/person/{person_id}/movie_credits\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/images\"\n ],\n \"q_id\": 20\n },\n {\n \"query\": \"What are some common keywords associated with movie \\\"Titanic\\\"?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/keywords\"\n ],\n \"q_id\": 21\n },\n {\n \"query\": \"Give me a show that is currently on the air\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/tv/on_the_air\"\n ],\n \"q_id\": 22\n },\n {\n \"query\": \"When is Clint Eastwood's latest movie scheduled to be released?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/person\",\n \"GET https://api.themoviedb.org/3/person/{person_id}/movie_credits\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/release_dates\"\n ],\n \"q_id\": 23\n },\n {\n \"query\": \"Give me some movie reviews about The Dark Knight\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/reviews\"\n ],\n \"q_id\": 24\n },\n {\n \"query\": \"What are some movies that are similar to one of the movies directed by Francis Ford Coppola?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/person\",\n \"GET https://api.themoviedb.org/3/person/{person_id}/movie_credits\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/similar\"\n ],\n \"q_id\": 25\n },\n {\n \"query\": \"Who is the lead actor in the latest released movie?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/movie/latest\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/credits\"\n ],\n \"q_id\": 26\n },\n {\n \"query\": \"give me one image of the movie that is currently showing in theaters\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/movie/now_playing\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/images\"\n ],\n \"q_id\": 27\n },\n {\n \"query\": \"What is the most popular movie right now and what is its keywords?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/movie/popular\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/keywords\"\n ],\n \"q_id\": 28\n },\n {\n \"query\": \"What are the keywords of the most popular movie right now\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/movie/popular\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/keywords\"\n ],\n \"q_id\": 29\n },\n {\n \"query\": \"Who has starred in a movie \\\"Titanic\\\"\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/credits\"\n ],\n \"q_id\": 30\n },\n {\n \"query\": \"What do the logo looks like for Paramount Pictures?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/company/{company_id}\",\n \"GET https://api.themoviedb.org/3/company/{company_id}/images\"\n ],\n \"q_id\": 31\n },\n {\n \"query\": \"tell me where the company \\\"universal pictures\\\" was founded?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/company\",\n \"GET https://api.themoviedb.org/3/company/{company_id}\"\n ],\n \"q_id\": 32\n },\n {\n \"query\": \"When is the movie \\\"Titanic\\\" released?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/release_dates\"\n ],\n \"q_id\": 33\n },\n {\n \"query\": \"Who has starred in the first movie of Star Wars collection?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/collection\",\n \"GET https://api.themoviedb.org/3/collection/{collection_id}\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/credits\"\n ],\n \"q_id\": 34\n },\n {\n \"query\": \"give me a movie cover of a movie from the collection Harry Potter\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/collection\",\n \"GET https://api.themoviedb.org/3/collection/{collection_id}\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/images\"\n ],\n \"q_id\": 35\n },\n {\n \"query\": \"What are some keywords for a movie of the collection The Hunger Games?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/collection\",\n \"GET https://api.themoviedb.org/3/collection/{collection_id}\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/keywords\"\n ],\n \"q_id\": 36\n },\n {\n \"query\": \"When are the release dates for the movies of the collection The Hobbit?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/collection\",\n \"GET https://api.themoviedb.org/3/collection/{collection_id}\"\n ],\n \"q_id\": 37\n },\n {\n \"query\": \"List movies from The Fast and the Furious collection\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/collection\",\n \"GET https://api.themoviedb.org/3/collection/{collection_id}\"\n ],\n \"q_id\": 38\n },\n {\n \"query\": \"Give me a review of a movie from the collection The Fast and the Furious.\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/collection\",\n \"GET https://api.themoviedb.org/3/collection/{collection_id}\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/reviews\"\n ],\n \"q_id\": 39\n },\n {\n \"query\": \"When was the first movie of the collection Lord of the Rings released?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/collection\",\n \"GET https://api.themoviedb.org/3/collection/{collection_id}\"\n ],\n \"q_id\": 40\n },\n {\n \"query\": \"tell me the directors of the first season of House of Cards\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/season/{season_number}/credits\"\n ],\n \"q_id\": 41\n },\n {\n \"query\": \"I want some tv shows that similar to House of Cards\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/similar\"\n ],\n \"q_id\": 42\n },\n {\n \"query\": \"When did the most popular TV show currently on the air start?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/tv/on_the_air\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}\"\n ],\n \"q_id\": 43\n },\n {\n \"query\": \"tell me a few more works directed by the director of Django Unchained\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/credits\",\n \"GET https://api.themoviedb.org/3/person/{person_id}/movie_credits\"\n ],\n \"q_id\": 44\n },\n {\n \"query\": \"tell me the cast of a TV show on the air\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/tv/on_the_air\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/credits\"\n ],\n \"q_id\": 45\n },\n {\n \"query\": \"I need a review for Breaking Bad\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/reviews\"\n ],\n \"q_id\": 46\n },\n {\n \"query\": \"I'm watching the tv series The Last Of Us and I need some more recommendations\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/recommendations\"\n ],\n \"q_id\": 47\n },\n {\n \"query\": \"I'm watching a TV show called The Last Of Us and I need some more recommendations\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/recommendations\"\n ],\n \"q_id\": 48\n },\n {\n \"query\": \"tell me the guest star from season 3, episode 24 of Friends\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/season/{season_number}/episode/{episode_number}/credits\"\n ],\n \"q_id\": 49\n },\n {\n \"query\": \"Who is the lead actor in the most popular TV show?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/tv/popular\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/credits\"\n ],\n \"q_id\": 50\n },\n {\n \"query\": \"give me a keyword of the top-1 rated TV show\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/tv/top_rated\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/keywords\"\n ],\n \"q_id\": 51\n },\n {\n \"query\": \"give me a poster of 2 Broke Girls\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/images\"\n ],\n \"q_id\": 52\n },\n {\n \"query\": \"How many episodes does 2 Broke Girls have in total?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}\"\n ],\n \"q_id\": 53\n },\n {\n \"query\": \"When was the company that produced The Big Bang Theory founded?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}\",\n \"GET https://api.themoviedb.org/3/company/{company_id}\"\n ],\n \"q_id\": 54\n },\n {\n \"query\": \"When was the TV show The Big Bang Theory created?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}\"\n ],\n \"q_id\": 55\n },\n {\n \"query\": \"give me a keyword of the most popular TV show\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/tv/popular\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/keywords\"\n ],\n \"q_id\": 56\n },\n {\n \"query\": \"What did the third episode of the second season of Westworld talk about?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/season/{season_number}/episode/{episode_number}\"\n ],\n \"q_id\": 57\n },\n {\n \"query\": \"Tell me the air date of the second season of Game of Thrones\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/season/{season_number}\"\n ],\n \"q_id\": 58\n },\n {\n \"query\": \" give me a poster of the third episode of Band of Brothers\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/season/{season_number}/episode/{episode_number}/images\"\n ],\n \"q_id\": 59\n },\n {\n \"query\": \"What is the logo of the network that produced Game of Thrones?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}\",\n \"GET https://api.themoviedb.org/3/network/{network_id}/images\"\n ],\n \"q_id\": 60\n },\n {\n \"query\": \"give me the homepage of the network that produced Game of Thrones\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}\",\n \"GET https://api.themoviedb.org/3/network/{network_id}\"\n ],\n \"q_id\": 61\n },\n {\n \"query\": \"give me a keyword of the most popular TV show currently\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/tv/popular\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/keywords\"\n ],\n \"q_id\": 62\n },\n {\n \"query\": \"Where is the headquarter of the company that produced currently the most popular TV show?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/tv/popular\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}\",\n \"GET https://api.themoviedb.org/3/company/{company_id}\"\n ],\n \"q_id\": 63\n },\n {\n \"query\": \"What is the genre of The Mandalorian?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}\"\n ],\n \"q_id\": 64\n },\n {\n \"query\": \"When is the birthday of the lead actor of today's most trending TV show?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/trending/{media_type}/{time_window}\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/credits\",\n \"GET https://api.themoviedb.org/3/person/{person_id}\"\n ],\n \"q_id\": 65\n },\n {\n \"query\": \"What is the homepage of the network that produced today's most trending TV show?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/trending/{media_type}/{time_window}\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}\",\n \"GET https://api.themoviedb.org/3/network/{network_id}\"\n ],\n \"q_id\": 66\n },\n {\n \"query\": \"give me a review of today's most trending TV show\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/trending/{media_type}/{time_window}\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/reviews\"\n ],\n \"q_id\": 67\n },\n {\n \"query\": \"I prefer a TV show that similar to today's most trending TV show\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/trending/{media_type}/{time_window}\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/similar\"\n ],\n \"q_id\": 68\n },\n {\n \"query\": \"I need a poster of today's most trending TV show\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/trending/{media_type}/{time_window}\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/images\"\n ],\n \"q_id\": 69\n },\n {\n \"query\": \"Who has worked with Jeremy Clarkson in his most popular TV show?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/person\",\n \"GET https://api.themoviedb.org/3/person/{person_id}/tv_credits\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/credits\"\n ],\n \"q_id\": 70\n },\n {\n \"query\": \"What does the lead actor of the first episode of second season of Black Mirror look like?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/credits\",\n \"GET https://api.themoviedb.org/3/person/{person_id}/images\"\n ],\n \"q_id\": 71\n },\n {\n \"query\": \"When is the lead actor of The Mandalorian born?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/credits\",\n \"GET https://api.themoviedb.org/3/person/{person_id}\"\n ],\n \"q_id\": 72\n },\n {\n \"query\": \"Who is the lead actor of today's most popular TV show?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/tv/popular\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/credits\"\n ],\n \"q_id\": 73\n },\n {\n \"query\": \"give me a photo of the lead actor of the most popular TV show of this week\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/tv/popular\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/credits\",\n \"GET https://api.themoviedb.org/3/person/{person_id}/images\"\n ],\n \"q_id\": 74\n },\n {\n \"query\": \"give me some reviews of Cate Blanchett's latest TV show\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/person\",\n \"GET https://api.themoviedb.org/3/person/{person_id}/tv_credits\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/reviews\"\n ],\n \"q_id\": 75\n },\n {\n \"query\": \"What is David Schwimmer's most popular TV show?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/person\",\n \"GET https://api.themoviedb.org/3/person/{person_id}/tv_credits\"\n ],\n \"q_id\": 76\n },\n {\n \"query\": \"Who directed the third episode of the second season of Black Mirror?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/season/{season_number}/credits\"\n ],\n \"q_id\": 77\n },\n {\n \"query\": \"give me some movies that Christopher Nolan known for\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/person\"\n ],\n \"q_id\": 78\n },\n {\n \"query\": \"Avatar versus Avatar: The Way of Water, which has a higher rating\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/search/movie\"\n ],\n \"q_id\": 79\n },\n {\n \"query\": \"Show me some English movies rating no less than 7.0 and no earlier than 2021\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/discover/movie\"\n ],\n \"q_id\": 80\n },\n {\n \"query\": \"What is the genre of the movie Lord of the Ring?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}\"\n ],\n \"q_id\": 81\n },\n {\n \"query\": \"What is the birthday of the most popular person right now?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/person/popular\",\n \"GET https://api.themoviedb.org/3/person/{person_id}\"\n ],\n \"q_id\": 82\n },\n {\n \"query\": \"give me a poster of the second season of Big Bang Theory\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/season/{season_number}/images\"\n ],\n \"q_id\": 83\n },\n {\n \"query\": \"What is the name of the third episod of the second season of Mandalorian?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/season/{season_number}/episode/{episode_number}\"\n ],\n \"q_id\": 84\n },\n {\n \"query\": \"When is the bitrhday of the lead actor of the movie The Shawshank Redemptionn?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/credits\",\n \"GET https://api.themoviedb.org/3/person/{person_id}\"\n ],\n \"q_id\": 85\n },\n {\n \"query\": \"When is the birthday of the director of the movie The Shawshank Redemption?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/credits\",\n \"GET https://api.themoviedb.org/3/person/{person_id}\"\n ],\n \"q_id\": 86\n },\n {\n \"query\": \"Where is the headquarter of the company that created the movie The Shawshank Redemption?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}\",\n \"GET https://api.themoviedb.org/3/company/{company_id}\"\n ],\n \"q_id\": 87\n },\n {\n \"query\": \"Where is the headquarter of the company that created the collection Star Wars?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/collection\",\n \"GET https://api.themoviedb.org/3/collection/{collection_id}\",\n \"GET https://api.themoviedb.org/3/company/{company_id}\"\n ],\n \"q_id\": 88\n },\n {\n \"query\": \"Who has an earlier release date, The Double Life of Veronique or Mulholland Drive?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}\"\n ],\n \"q_id\": 89\n },\n {\n \"query\": \"Is the director of Twin Peaks Season 1 and Mulholland Drive the same person?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/credits\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/season/{season_number}/credits\"\n ],\n \"q_id\": 90\n },\n {\n \"query\": \"Where was the director of Mulholland Drive born?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/credits\",\n \"GET https://api.themoviedb.org/3/person/{person_id}\"\n ],\n \"q_id\": 91\n },\n {\n \"query\": \"Is Mulholland Drive in the Top-10 rated list of the TMDB?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/top_rated\"\n ],\n \"q_id\": 92\n },\n {\n \"query\": \"Who directed more movies, Akira Kurosawa or Spielberg?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/person\",\n \"GET https://api.themoviedb.org/3/person/{person_id}/movie_credits\"\n ],\n \"q_id\": 93\n },\n {\n \"query\": \"What TV series has the director of Mulholland Drive directed?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/person/{person_id}/tv_credits\"\n ],\n \"q_id\": 94\n },\n {\n \"query\": \"Tell me about actor Scarlett Johansson's birthday\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/person\",\n \"GET https://api.themoviedb.org/3/person/{person_id}\"\n ],\n \"q_id\": 95\n },\n {\n \"query\": \"How many seasons of Sword Art Online are there?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}\"\n ],\n \"q_id\": 96\n },\n {\n \"query\": \"Are Yui Aragaki and Gen Hoshino co-starring in We Married as Job?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/credits\"\n ],\n \"q_id\": 97\n },\n {\n \"query\": \"Who is older, by how many years, the director of film Barbie or the lead actor of television DEATH NOTE?\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/credits\",\n \"GET https://api.themoviedb.org/3/search/tv\",\n \"GET https://api.themoviedb.org/3/tv/{tv_id}/credits\"\n ],\n \"q_id\": 98\n },\n {\n \"query\": \"Tell me about Katherine LaNasa's latest movie appearance.\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/person\",\n \"GET https://api.themoviedb.org/3/person/{person_id}/movie_credits\"\n ],\n \"q_id\": 99\n },\n {\n \"query\": \"Give me one cover image of the movie Oppenheimer.\",\n \"solution\": [\n \"GET https://api.themoviedb.org/3/search/movie\",\n \"GET https://api.themoviedb.org/3/movie/{movie_id}/images\"\n ],\n \"q_id\": 100\n }\n ],\n \"tool\": [\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/movie/{movie_id}/keywords\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the keywords that have been added to a movie.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/tv/popular\",\n \"para\": [\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify which page to query.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Get a list of the current popular TV shows on TMDb. This list updates daily.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/person/{person_id}\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the primary person details by id.\\n\\nSupports `append_to_response`. Read more about this [here](#docTextSection:JdZq8ctmcxNqyLQjp).\\n\\n#### Recent Changes\\n\\n| **Date** | **Change** |\\n| - | - |\\n| July 17, 2018 | Added the `known_for_department` field. |\\n| April 26, 2018 | Added the [translations](#endpoint:CSaMjCxXAtGpxNGfS) method. |\\n| November 9, 2016 | Biographies are now translatable on TMDb. This means you can query person details with a language parameter. |\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/movie/{movie_id}/reviews\",\n \"para\": [\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify which page to query.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Get the user reviews for a movie.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/movie/{movie_id}/release_dates\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the release date along with the certification for a movie.\\n\\nRelease dates support different types:\\n\\n1. Premiere\\n2. Theatrical (limited)\\n3. Theatrical\\n4. Digital\\n5. Physical\\n6. TV\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/tv/{tv_id}/season/{season_number}/episode/{episode_number}/credits\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the credits (cast, crew and guest stars) for a TV episode.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/movie/{movie_id}/images\",\n \"para\": [\n {\n \"name\": \"include_image_language\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"the include image language\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Get the images that belong to a movie.\\n\\nQuerying images with a `language` parameter will filter the results. If you want to include a fallback language (especially useful for backdrops) you can use the `include_image_language` parameter. This should be a comma seperated value like so: `include_image_language=en,null`.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/search/tv\",\n \"para\": [\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify which page to query.\"\n },\n {\n \"name\": \"query\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Pass a text query to search. This value should be URI encoded.\",\n \"required\": true\n },\n {\n \"name\": \"include_adult\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"boolean\",\n \"default\": false\n },\n \"description\": \"Choose whether to inlcude adult (pornography) content in the results.\"\n },\n {\n \"name\": \"first_air_date_year\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\"\n },\n \"description\": \"the first air date year\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Search for a TV show.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/tv/{tv_id}/season/{season_number}/episode/{episode_number}\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the TV episode details by id.\\n\\nSupports `append_to_response`. Read more about this [here](#docTextSection:JdZq8ctmcxNqyLQjp).\\n\\n#### Recent Changes\\n\\n| **Date** | **Change** |\\n| - | - |\\n| June 1, 2018 | Added the [translations](#endpoint:5SFwZar3LkP99QMp7) method. |\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/network/{network_id}/images\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the TV network logos by id.\\n\\nThere are two image formats that are supported for networks, PNG's and SVG's. You can see which type the original file is by looking at the `file_type` field. We prefer SVG's as they are resolution independent and as such, the width and height are only there to reflect the original asset that was uploaded. An SVG can be scaled properly beyond those dimensions if you call them as a PNG.\\n\\nFor more information about how SVG's and PNG's can be used, take a read through [this document](#docTextSection:mXP9B2uzoDJFguDZv).\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/genre/tv/list\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the list of official genres for TV shows.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/search/movie\",\n \"para\": [\n {\n \"name\": \"query\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Pass a text query to search. This value should be URI encoded.\",\n \"required\": true\n },\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify which page to query.\"\n },\n {\n \"name\": \"include_adult\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"boolean\",\n \"default\": false\n },\n \"description\": \"Choose whether to inlcude adult (pornography) content in the results.\"\n },\n {\n \"name\": \"region\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Specify a ISO 3166-1 code to filter release dates. Must be uppercase.\"\n },\n {\n \"name\": \"year\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\"\n },\n \"description\": \"the specific year\"\n },\n {\n \"name\": \"primary_release_year\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\"\n },\n \"description\": \"the primary release year\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Search for movies.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/discover/movie\",\n \"para\": [\n {\n \"name\": \"region\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Specify a ISO 3166-1 code to filter release dates. Must be uppercase.\"\n },\n {\n \"name\": \"sort_by\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\",\n \"enum\": [\n \"\",\n \"popularity.asc\",\n \"popularity.desc\",\n \"release_date.asc\",\n \"release_date.desc\",\n \"revenue.asc\",\n \"revenue.desc\",\n \"primary_release_date.asc\",\n \"primary_release_date.desc\",\n \"original_title.asc\",\n \"original_title.desc\",\n \"vote_average.asc\",\n \"vote_average.desc\",\n \"vote_count.asc\",\n \"vote_count.desc\"\n ],\n \"default\": \"popularity.desc\"\n },\n \"description\": \"Choose from one of the many available sort options.\"\n },\n {\n \"name\": \"certification_country\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Used in conjunction with the certification filter, use this to specify a country with a valid certification.\"\n },\n {\n \"name\": \"certification\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Filter results with a valid certification from the 'certification_country' field.\"\n },\n {\n \"name\": \"certification.lte\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Filter and only include movies that have a certification that is less than or equal to the specified value.\"\n },\n {\n \"name\": \"certification.gte\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Filter and only include movies that have a certification that is greater than or equal to the specified value.\"\n },\n {\n \"name\": \"include_adult\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"boolean\",\n \"default\": false\n },\n \"description\": \"A filter and include or exclude adult movies.\"\n },\n {\n \"name\": \"include_video\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"boolean\",\n \"default\": false\n },\n \"description\": \"A filter to include or exclude videos.\"\n },\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify the page of results to query.\"\n },\n {\n \"name\": \"primary_release_year\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\"\n },\n \"description\": \"A filter to limit the results to a specific primary release year.\"\n },\n {\n \"name\": \"primary_release_date.gte\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Filter and only include movies that have a primary release date that is greater or equal to the specified value.\"\n },\n {\n \"name\": \"primary_release_date.lte\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Filter and only include movies that have a primary release date that is less than or equal to the specified value.\"\n },\n {\n \"name\": \"release_date.gte\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Filter and only include movies that have a release date (looking at all release dates) that is greater or equal to the specified value.\"\n },\n {\n \"name\": \"release_date.lte\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Filter and only include movies that have a release date (looking at all release dates) that is less than or equal to the specified value.\"\n },\n {\n \"name\": \"with_release_type\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\"\n },\n \"description\": \"Specify a comma (AND) or pipe (OR) separated value to filter release types by. These release types map to the same values found on the movie release date method.\"\n },\n {\n \"name\": \"year\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\"\n },\n \"description\": \"A filter to limit the results to a specific year (looking at all release dates).\"\n },\n {\n \"name\": \"vote_count.gte\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\"\n },\n \"description\": \"Filter and only include movies that have a vote count that is greater or equal to the specified value.\"\n },\n {\n \"name\": \"vote_count.lte\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\"\n },\n \"description\": \"Filter and only include movies that have a vote count that is less than or equal to the specified value.\"\n },\n {\n \"name\": \"vote_average.gte\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"number\"\n },\n \"description\": \"Filter and only include movies that have a rating that is greater or equal to the specified value.\"\n },\n {\n \"name\": \"vote_average.lte\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"number\"\n },\n \"description\": \"Filter and only include movies that have a rating that is less than or equal to the specified value.\"\n },\n {\n \"name\": \"with_cast\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"A comma separated list of person ID's. Only include movies that have one of the ID's added as an actor.\"\n },\n {\n \"name\": \"with_crew\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"A comma separated list of person ID's. Only include movies that have one of the ID's added as a crew member.\"\n },\n {\n \"name\": \"with_people\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"A comma separated list of person ID's. Only include movies that have one of the ID's added as a either a actor or a crew member.\"\n },\n {\n \"name\": \"with_companies\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"A comma separated list of production company ID's. Only include movies that have one of the ID's added as a production company.\"\n },\n {\n \"name\": \"with_genres\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Comma separated value of genre ids that you want to include in the results.\"\n },\n {\n \"name\": \"without_genres\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Comma separated value of genre ids that you want to exclude from the results.\"\n },\n {\n \"name\": \"with_keywords\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"A comma separated list of keyword ID's. Only includes movies that have one of the ID's added as a keyword.\"\n },\n {\n \"name\": \"without_keywords\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Exclude items with certain keywords. You can comma and pipe seperate these values to create an 'AND' or 'OR' logic.\"\n },\n {\n \"name\": \"with_runtime.gte\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\"\n },\n \"description\": \"Filter and only include movies that have a runtime that is greater or equal to a value.\"\n },\n {\n \"name\": \"with_runtime.lte\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\"\n },\n \"description\": \"Filter and only include movies that have a runtime that is less than or equal to a value.\"\n },\n {\n \"name\": \"with_original_language\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Specify an ISO 639-1 string to filter results by their original language value.\"\n },\n {\n \"name\": \"with_watch_providers\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"A comma or pipe separated list of watch provider ID's. Combine this filter with `watch_region` in order to filter your results by a specific watch provider in a specific region.\"\n },\n {\n \"name\": \"watch_region\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"An ISO 3166-1 code. Combine this filter with `with_watch_providers` in order to filter your results by a specific watch provider in a specific region.\"\n },\n {\n \"name\": \"with_watch_monetization_types\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\",\n \"enum\": [\n \"flatrate\",\n \"free\",\n \"ads\",\n \"rent\",\n \"buy\"\n ]\n },\n \"description\": \"In combination with `watch_region`, you can filter by monetization type.\"\n },\n {\n \"name\": \"without_companies\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Filter the results to exclude the specific production companies you specify here. `AND` / `OR` filters are supported.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Discover movies by different types of data like average rating, number of votes, genres and certifications. You can get a valid list of certifications from the [certifications list](#endpoint:faFKjuKG2HnwexAWM) method.\\n\\nDiscover also supports a nice list of sort options. See below for all of the available options.\\n\\nPlease note, when using `certification` \\\\ `certification.lte` you must also specify `certification_country`. These two parameters work together in order to filter the results. You can only filter results with the countries we have added to our [certifications list](#endpoint:faFKjuKG2HnwexAWM).\\n\\nIf you specify the `region` parameter, the regional release date will be used instead of the primary release date. The date returned will be the first date based on your query (ie. if a `with_release_type` is specified). It's important to note the order of the release types that are used. Specifying \\\"2|3\\\" would return the limited theatrical release date as opposed to \\\"3|2\\\" which would return the theatrical date.\\n\\nAlso note that a number of filters support being comma (`,`) or pipe (`|`) separated. Comma's are treated like an `AND` and query while pipe's are an `OR`. \\n\\nSome examples of what can be done with discover can be found [here](https://www.themoviedb.org/documentation/api/discover).\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/movie/upcoming\",\n \"para\": [\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify which page to query.\"\n },\n {\n \"name\": \"region\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Specify a ISO 3166-1 code to filter release dates. Must be uppercase.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Get a list of upcoming movies in theatres. This is a release type query that looks for all movies that have a release type of 2 or 3 within the specified date range.\\n\\nYou can optionally specify a `region` prameter which will narrow the search to only look for theatrical release dates within the specified country.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/credit/{credit_id}\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get a movie or TV credit details by id.\\n\\n#### Recent Changes\\n\\n| **Date** | **Change** |\\n| - | - |\\n| July 13, 2018 | Movie credits are now supported. |\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/person/{person_id}/tv_credits\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the TV show credits for a person.\\n\\nYou can query for some extra details about the credit with the [credit method](#endpoint:xPWdEBLkvCNZSicLN).\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/tv/latest\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the most newly created TV show. This is a live response and will continuously change.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/company/{company_id}\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get a companies details by id.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/tv/{tv_id}/images\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the images that belong to a TV show.\\n\\nQuerying images with a `language` parameter will filter the results. If you want to include a fallback language (especially useful for backdrops) you can use the `include_image_language` parameter. This should be a comma seperated value like so: `include_image_language=en,null`.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/tv/{tv_id}/season/{season_number}/credits\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the credits for TV season.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/movie/now_playing\",\n \"para\": [\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify which page to query.\"\n },\n {\n \"name\": \"region\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Specify a ISO 3166-1 code to filter release dates. Must be uppercase.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Get a list of movies in theatres. This is a release type query that looks for all movies that have a release type of 2 or 3 within the specified date range.\\n\\nYou can optionally specify a `region` prameter which will narrow the search to only look for theatrical release dates within the specified country.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/review/{review_id}\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"get the review of based on the input review id\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/tv/on_the_air\",\n \"para\": [\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify which page to query.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Get a list of shows that are currently on the air.\\n\\nThis query looks for any TV show that has an episode with an air date in the next 7 days.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/movie/{movie_id}\",\n \"para\": [\n {\n \"name\": \"append_to_response\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Append requests within the same namespace to the response.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Get the primary information about a movie.\\n\\nSupports `append_to_response`. Read more about this [here](#docTextSection:JdZq8ctmcxNqyLQjp).\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/tv/{tv_id}/season/{season_number}/images\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the images that belong to a TV season.\\n\\nQuerying images with a `language` parameter will filter the results. If you want to include a fallback language (especially useful for backdrops) you can use the `include_image_language` parameter. This should be a comma seperated value like so: `include_image_language=en,null`.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/company/{company_id}/images\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get a companies logos by id.\\n\\nThere are two image formats that are supported for companies, PNG's and SVG's. You can see which type the original file is by looking at the `file_type` field. We prefer SVG's as they are resolution independent and as such, the width and height are only there to reflect the original asset that was uploaded. An SVG can be scaled properly beyond those dimensions if you call them as a PNG.\\n\\nFor more information about how SVG's and PNG's can be used, take a read through [this document](#docTextSection:mXP9B2uzoDJFguDZv).\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/tv/{tv_id}/reviews\",\n \"para\": [\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify which page to query.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Get the reviews for a TV show.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/tv/{tv_id}/similar\",\n \"para\": [\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify which page to query.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Get a list of similar TV shows. These items are assembled by looking at keywords and genres.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/network/{network_id}\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the details of a network.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/tv/{tv_id}/recommendations\",\n \"para\": [\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify which page to query.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Get the list of TV show recommendations for this item.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/tv/{tv_id}/season/{season_number}/episode/{episode_number}/images\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the images that belong to a TV episode.\\n\\nQuerying images with a `language` parameter will filter the results. If you want to include a fallback language (especially useful for backdrops) you can use the `include_image_language` parameter. This should be a comma seperated value like so: `include_image_language=en,null`.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/movie/popular\",\n \"para\": [\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify which page to query.\"\n },\n {\n \"name\": \"region\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Specify a ISO 3166-1 code to filter release dates. Must be uppercase.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Get a list of the current popular movies on TMDb. This list updates daily.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/tv/airing_today\",\n \"para\": [\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify which page to query.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Get a list of TV shows that are airing today. This query is purely day based as we do not currently support airing times.\\n\\nYou can specify a [timezone](endpoint:KQ4CDdEoWKJYLkrhS) to offset the day calculation. Without a specified timezone, this query defaults to EST (Eastern Time UTC-05:00).\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/tv/{tv_id}/keywords\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the keywords that have been added to a TV show.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/search/person\",\n \"para\": [\n {\n \"name\": \"query\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Pass a text query to search. This value should be URI encoded.\",\n \"required\": true\n },\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify which page to query.\"\n },\n {\n \"name\": \"include_adult\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"boolean\",\n \"default\": false\n },\n \"description\": \"Choose whether to inlcude adult (pornography) content in the results.\"\n },\n {\n \"name\": \"region\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Specify a ISO 3166-1 code to filter release dates. Must be uppercase.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Search for people.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/search/company\",\n \"para\": [\n {\n \"name\": \"query\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Pass a text query to search. This value should be URI encoded.\",\n \"required\": true\n },\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify which page to query.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Search for companies.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/discover/tv\",\n \"para\": [\n {\n \"name\": \"sort_by\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\",\n \"enum\": [\n \"\",\n \"vote_average.desc\",\n \"vote_average.asc\",\n \"first_air_date.desc\",\n \"first_air_date.asc\",\n \"popularity.desc\",\n \"popularity.asc\"\n ],\n \"default\": \"popularity.desc\"\n },\n \"description\": \"Choose from one of the many available sort options.\"\n },\n {\n \"name\": \"air_date.gte\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Filter and only include TV shows that have a air date (by looking at all episodes) that is greater or equal to the specified value.\"\n },\n {\n \"name\": \"air_date.lte\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Filter and only include TV shows that have a air date (by looking at all episodes) that is less than or equal to the specified value.\"\n },\n {\n \"name\": \"first_air_date.gte\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Filter and only include TV shows that have a original air date that is greater or equal to the specified value. Can be used in conjunction with the \\\"include_null_first_air_dates\\\" filter if you want to include items with no air date.\"\n },\n {\n \"name\": \"first_air_date.lte\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Filter and only include TV shows that have a original air date that is less than or equal to the specified value. Can be used in conjunction with the \\\"include_null_first_air_dates\\\" filter if you want to include items with no air date.\"\n },\n {\n \"name\": \"first_air_date_year\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\"\n },\n \"description\": \"Filter and only include TV shows that have a original air date year that equal to the specified value. Can be used in conjunction with the \\\"include_null_first_air_dates\\\" filter if you want to include items with no air date.\"\n },\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify the page of results to query.\"\n },\n {\n \"name\": \"timezone\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\",\n \"default\": \"America/New_York\"\n },\n \"description\": \"Used in conjunction with the air_date.gte/lte filter to calculate the proper UTC offset.\"\n },\n {\n \"name\": \"vote_average.gte\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"number\"\n },\n \"description\": \"Filter and only include movies that have a rating that is greater or equal to the specified value.\"\n },\n {\n \"name\": \"vote_count.gte\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\"\n },\n \"description\": \"Filter and only include movies that have a rating that is less than or equal to the specified value.\"\n },\n {\n \"name\": \"with_genres\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Comma separated value of genre ids that you want to include in the results.\"\n },\n {\n \"name\": \"with_networks\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Comma separated value of network ids that you want to include in the results.\"\n },\n {\n \"name\": \"without_genres\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Comma separated value of genre ids that you want to exclude from the results.\"\n },\n {\n \"name\": \"with_runtime.gte\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\"\n },\n \"description\": \"Filter and only include TV shows with an episode runtime that is greater than or equal to a value.\"\n },\n {\n \"name\": \"with_runtime.lte\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\"\n },\n \"description\": \"Filter and only include TV shows with an episode runtime that is less than or equal to a value.\"\n },\n {\n \"name\": \"include_null_first_air_dates\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"boolean\",\n \"default\": false\n },\n \"description\": \"Use this filter to include TV shows that don't have an air date while using any of the \\\"first_air_date\\\" filters.\"\n },\n {\n \"name\": \"with_original_language\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Specify an ISO 639-1 string to filter results by their original language value.\"\n },\n {\n \"name\": \"without_keywords\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Exclude items with certain keywords. You can comma and pipe seperate these values to create an 'AND' or 'OR' logic.\"\n },\n {\n \"name\": \"screened_theatrically\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"boolean\"\n },\n \"description\": \"Filter results to include items that have been screened theatrically.\"\n },\n {\n \"name\": \"with_companies\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"A comma separated list of production company ID's. Only include movies that have one of the ID's added as a production company.\"\n },\n {\n \"name\": \"with_keywords\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"A comma separated list of keyword ID's. Only includes TV shows that have one of the ID's added as a keyword.\"\n },\n {\n \"name\": \"with_watch_providers\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"A comma or pipe separated list of watch provider ID's. Combine this filter with `watch_region` in order to filter your results by a specific watch provider in a specific region.\"\n },\n {\n \"name\": \"watch_region\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"An ISO 3166-1 code. Combine this filter with `with_watch_providers` in order to filter your results by a specific watch provider in a specific region.\"\n },\n {\n \"name\": \"with_watch_monetization_types\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\",\n \"enum\": [\n \"flatrate\",\n \"free\",\n \"ads\",\n \"rent\",\n \"buy\"\n ]\n },\n \"description\": \"In combination with `watch_region`, you can filter by monetization type.\"\n },\n {\n \"name\": \"with_status\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\",\n \"enum\": [\n 0,\n 1,\n 2,\n 3,\n 4,\n 5\n ]\n },\n \"description\": \"Filter TV shows by their status.\\n\\nReturning Series: 0\\nPlanned: 1\\nIn Production: 2\\nEnded: 3\\nCancelled: 4\\nPilot: 5\"\n },\n {\n \"name\": \"with_type\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\",\n \"enum\": [\n 0,\n 1,\n 2,\n 3,\n 4,\n 5,\n 6\n ]\n },\n \"description\": \"Filter TV shows by their type.\\n\\nDocumentary: 0\\nNews: 1\\nMiniseries: 2\\nReality: 3\\nScripted: 4\\nTalk Show: 5\\nVideo: 6\"\n },\n {\n \"name\": \"without_companies\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Filter the results to exclude the specific production companies you specify here. `AND` / `OR` filters are supported.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Discover TV shows by different types of data like average rating, number of votes, genres, the network they aired on and air dates.\\n\\nDiscover also supports a nice list of sort options. See below for all of the available options.\\n\\nAlso note that a number of filters support being comma (`,`) or pipe (`|`) separated. Comma's are treated like an `AND` and query while pipe's are an `OR`. \\n\\nSome examples of what can be done with discover can be found [here](https://www.themoviedb.org/documentation/api/discover).\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/movie/top_rated\",\n \"para\": [\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify which page to query.\"\n },\n {\n \"name\": \"region\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Specify a ISO 3166-1 code to filter release dates. Must be uppercase.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Get the top rated movies on TMDb.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/movie/latest\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the most newly created movie. This is a live response and will continuously change.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/tv/top_rated\",\n \"para\": [\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify which page to query.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Get a list of the top rated TV shows on TMDb.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/trending/{media_type}/{time_window}\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the daily or weekly trending items. The daily trending list tracks items over the period of a day while items have a 24 hour half life. The weekly list tracks items over a 7 day period, with a 7 day half life.\\n\\n#### Valid Media Types\\n\\n| **Media Type** | **Description** |\\n| - | - |\\n| all | Include all movies, TV shows and people in the results as a global trending list. |\\n| movie | Show the trending movies in the results. |\\n| tv | Show the trending TV shows in the results. |\\n| person | Show the trending people in the results. |\\n\\n#### Valid Time Windows\\n\\n| **Time Window** | **Description** |\\n| - | - |\\n| day | View the trending list for the day. |\\n| week | View the trending list for the week. |\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/genre/movie/list\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the list of official genres for movies.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/tv/{tv_id}/season/{season_number}\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the TV season details by id.\\n\\nSupports `append_to_response`. Read more about this [here](#docTextSection:JdZq8ctmcxNqyLQjp).\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/collection/{collection_id}\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get collection details by id.\\n\\n#### Recent Changes\\n\\n| **Date** | **Change** |\\n| - | - |\\n| March 16, 2018 | Added the [translations](#endpoint:bhcsxEgRvyk3N7FnD) method. |\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/person/{person_id}/images\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the images for a person.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/tv/{tv_id}\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the primary TV show details by id.\\n\\nSupports `append_to_response`. Read more about this [here](#docTextSection:JdZq8ctmcxNqyLQjp).\\n\\n#### Recent Changes\\n\\n| **Date** | **Change** |\\n| - | - |\\n| July 17, 2018 | We now return `last_episode_to_air` and `next_episode_to_air` fields. |\\n| March 12, 2018 | Networks return proper logos and we introduced SVG support. |\\n| March 8, 2018 | The `seasons` field now returns the translated names and overviews. |\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/person/popular\",\n \"para\": [\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify which page to query.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Get the list of popular people on TMDb. This list updates daily.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/collection/{collection_id}/images\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the images for a collection by id.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/tv/{tv_id}/credits\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the credits (cast and crew) that have been added to a TV show.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/person/{person_id}/movie_credits\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the movie credits for a person, the results contains various information such as popularity and release date.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/movie/{movie_id}/recommendations\",\n \"para\": [\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify which page to query.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Get a list of recommended movies for a movie.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/search/collection\",\n \"para\": [\n {\n \"name\": \"query\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"string\"\n },\n \"description\": \"Pass a text query to search. This value should be URI encoded.\",\n \"required\": true\n },\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify which page to query.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Search for collections.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/movie/{movie_id}/credits\",\n \"para\": [],\n \"usage\": \"\",\n \"description\": \"Get the cast and crew for a movie.\",\n \"method\": \"GET\",\n \"result\": \"\"\n },\n {\n \"base_url\": \"https://api.themoviedb.org/3\",\n \"url\": \"GET https://api.themoviedb.org/3/movie/{movie_id}/similar\",\n \"para\": [\n {\n \"name\": \"page\",\n \"in\": \"query\",\n \"schema\": {\n \"type\": \"integer\",\n \"default\": 1\n },\n \"description\": \"Specify which page to query.\"\n }\n ],\n \"usage\": \"\",\n \"description\": \"Get a list of similar movies. This is **not** the same as the \\\"Recommendation\\\" system you see on the website.\\n\\nThese items are assembled by looking at keywords and genres.\",\n \"method\": \"GET\",\n \"result\": \"\"\n }\n ]\n}" }, { "path": "models/__init__.py", "content": "" }, { "path": "models/base.py", "content": "from utilize.apis import *\n\nclass Base:\n\n def __init__(self,model,endpoints,role='base',url=None):\n self.role=role\n self.model=model\n self.endpoints=endpoints\n self.url=url\n if endpoints!=None:\n self.name2doc = {line['url']: line for line in endpoints}\n base_url=endpoints[0]['base_url']\n for line in endpoints:\n self.name2doc[f\"{base_url}{line['url']}\"]=line\n else:\n self.name2doc=None\n def overlap(self,s1,s2):\n s1=s1.split('/')\n s1=[e.strip() for e in s1]\n s2=s2.split('/')\n s2 = [e.strip() for e in s2]\n return len([e for e in s1 if e in s2 ])\n\n def match_tools(self,input_api):\n api_name=self.normalize(input_api)\n if api_name.startswith(\"'\") or api_name.startswith('\"'):\n api_name=api_name[1:]\n if api_name.endswith(\"'\") or api_name.endswith('\"'):\n api_name = api_name[:-1]\n if \"?\" in api_name:\n api_name=api_name[:api_name.index('?')].replace('?','').strip()\n if api_name in self.name2doc:\n return self.name2doc[api_name]\n\n api_list=[line['method']+ ' ' + line['url'] for line in self.endpoints]\n api_list='\\n'.join(api_list)\n prompt=f\"I mistakenly remembered the name of an API, and please help me to choose the most likely API from the provided list.\\nInput API: {input_api}\\nAPI list: {api_list}\\n\\nYou should only select a API from list, and do not say extra words.\"\n api_name=get_from_openai(model_name=self.model,messages=[{'role':'user','content':prompt}])['content'].strip()\n\n if api_name in self.name2doc:\n return self.name2doc[api_name]\n #\n res=[(self.overlap(name,api_name),v) for name,v in self.name2doc.items()]\n res=sorted(res,key=lambda x:x[0],reverse=True)\n return res[0][-1]\n\n def normalize(self,sss):\n ee1=['Thought:','API Selection:','Execution Result:']\n ee2=['\\n']\n for e in ee1:\n sss=sss.replace(e,'')\n for e in ee2:\n sss=sss.replace(e,' ')\n for i in range(10):\n sss=sss.replace(f'[{i}]','')\n return sss.strip()\n\n def get_tool_doc(self,line):\n url = f'{line[\"base_url\"]}/{line[\"url\"]}' if 'http' not in line['url'] else line['url']\n description = '## Description\\n' + line['description'].replace('\\n',' ')\n if len(line['para'])==0:\n para='## Parameter:\\nNo parameter'\n else:\n para = ['## Parameter']\n for p in line['para']:\n if p['in'] !='query':\n continue\n tmp=\"- \"+p['name']+\": \"+p['description'].replace('\\n','')\n if 'schema' in p and 'type' in p['schema']:\n tmp+=\" (type: \"+p['schema']['type']+\")\"\n para.append(tmp)\n para='\\n'.join(para)\n\n prompt = [url, description, para]\n if 'usage' in line and line['usage'] != '':\n usage = '## Usage:\\n```python\\n' + line['usage'] + '\\n```'\n prompt.append(usage)\n if 'result' in line and line['result'] != '':\n result = '## Execution result:\\n```json\\n' + json.dumps(line['result'], indent=4) + '\\n```'\n prompt.append(result)\n\n prompt = '\\n'.join(prompt)\n return prompt" }, { "path": "models/execution.py", "content": "from utilize.apis import get_from_openai\nfrom models.base import Base\nimport requests\nimport re\nimport logging\nimport json\n\nlogger = logging.getLogger()\n\nSYSTEM_ACT = 'You are an agent that gets a sequence of APIs, which can be accessed by HTTP request. Given the API documentation and my instruction, you should complete the API call.'\n\nACT_PROMPT = \"\"\"Please help me to complete the corresponding API calls according to the plan. \nHere is the documentation on the API:\n{api_doc}\n\nI will give you the background information, which may contain the person id, user id or movie id. You should complete the HTTP request based on according to the API documentation and current background.\nThe HTTP request should be a JSON string that has four basic keys: url, params, output_instructions and description.\n- The value of \"url\" should be a string. If the API path contains \"{{}}\", it means that it is a variable and you should find the appropriate value from provided background and replace the '{{}}'. For example, if the path is \"/users/{{user_id}}/tweets\", you should replace \"{{user_id}}\" with the user id. \"{{\" and \"}}\" cannot appear in the url.\n- The value of \"params\" should be the corresponding parameters based on the above documentation. The parameters should be extracted from the given background, e.g., user id, person id and movie id.\n- The value of \"output_instructions\" should be instructions on what information to find from the response. Note \"output_instructions\" MUST be natural language and as verbose as possible! It cannot be \"return the full response\". Output instructions should faithfully contain the contents of the api calling plan and be as specific as possible. The output instructions can also contain conditions such as filtering, sorting, etc.\n- The value of \"description\" should describe what the information got by this API request, e.g., the information about xxx movie, the personal information about xxx and the movie credit list of an actor. The description should be specific.\n\nIf the plan includes expressions such as \"most\", you should choose the first item from the response. For example, if the plan is \"GET /trending/tv/day to get the most trending TV show today\", you should choose the first item from the response.\n\nStarting below, you must follow this format:\nBackground: background information that you can use to execute the plan, e.g., the id of a person.\nInstruction: follow my instruction to complete the API calling.\nAPI Selection: the selected API to use\nHTTP request:: the HTTP request method including four basic key: url, params, output_instructions and description\n\nHere is an example:\nBackground: The person id of Akira Kurosawa is 5026.\nInstruction: use the person id **5026** and https://api.themoviedb.org/3/person/{{person_id}}/movie_credits to get the movies directed by Akira Kurosawa. And then extract the movies' name and id.\nAPI Selection: https://api.themoviedb.org/3/person/{{person_id}}/movie_credits\nHTTP request: {{\n \"url\": \"https://api.themoviedb.org/3/person/5026/movie_credits\",\n \"params\": {{\n \"page\": 1\n }},\n \"output_instructions\": \"Extract the names and ids of the movies\",\n \"description\": \"The movie credit list of Akira Kurosawa.\"\n}}\n\nBegin!\n\nBackground: {background}\nInstruction: {thought}\nAPI Selection: {api}\nHTTP request: \"\"\"\n\n\nclass ExecAgent(Base):\n\n def __init__(self, model='gpt-3.5-turbo', endpoints=None, requests_wrapper=None, role='ActGPT', url=None):\n super().__init__(model=model, endpoints=endpoints, role=role, url=url)\n self.requests_wrapper = requests_wrapper\n\n def _get_response(self, data, method=''):\n state = False\n if method == \"GET\":\n if 'params' in data:\n params = data.get(\"params\", [])\n response = self.requests_wrapper.get(data.get(\"url\"), params=params)\n else:\n response = self.requests_wrapper.get(data.get(\"url\"))\n elif method == \"POST\":\n params = data.get(\"params\")\n request_body = data.get(\"data\")\n response = self.requests_wrapper.post(data[\"url\"], params=params, data=request_body)\n elif method == \"PUT\":\n params = data.get(\"params\")\n request_body = data.get(\"data\")\n response = self.requests_wrapper.put(data[\"url\"], params=params, data=request_body)\n elif method == \"DELETE\":\n params = data.get(\"params\")\n request_body = data.get(\"data\")\n response = self.requests_wrapper.delete(data[\"url\"], params=params, json=request_body)\n else:\n raise NotImplementedError\n\n if isinstance(response, requests.models.Response):\n if response.status_code == 200:\n state = True\n response_text = response.text\n elif response.status_code == 204:\n state = True\n response_text = json.dumps({\"content\": \"The API request has executed successfully, but the requested resource is empty.\"})\n else:\n response_text = response.text\n elif isinstance(response, str):\n response_text = response\n else:\n raise NotImplementedError\n\n return state, response_text\n\n def extract(self, sss):\n matches = re.findall(r'\\{(.+?)\\}', sss)\n return matches\n\n def generate(self, thought, action, background):\n api = self.match_tools(action)\n api_doc = self.get_tool_doc(api)\n prompt = ACT_PROMPT.format(api_doc=api_doc, thought=thought, background=background, api=action)\n messages = [{'role': \"system\", 'content': SYSTEM_ACT}, {'role': 'user', 'content': prompt}]\n response = None\n input_request = None\n for i in range(0, 3):\n input_request = get_from_openai(model_name=self.model, temp=0, messages=messages, json_mode=True, stop=['FINISH:'])['content']\n messages.append({'role': 'assistant', 'content': input_request})\n logger.info(f'ExecAgent: {input_request}')\n data = json.loads(input_request)\n\n empty_value = self.extract(data['url'])\n if empty_value != []:\n messages.append({'role': 'user', 'content': \"You should always change the {}\" + f\" in API path with the value. Specifically, please extract value from background and change the {empty_value} in your url\"})\n else:\n try:\n state, response = self._get_response(data, api['method'])\n if state:\n break\n else:\n messages.append({'role': 'user',\n 'content': f'The API server raise a error message: {response}, please try again.'})\n except Exception as e:\n messages.append({'role': 'user', 'content': f'There are something error {e}, please try again. You should always change the ' + \"{} in API path\"})\n logger.info(f'API error: {e}')\n if response == None:\n return ['The execution result is None'] * 3\n if response == [] or response == '':\n return ['The execution result is empty'] * 3\n if input_request == None:\n return ['Please try again'] * 3\n input_request = json.loads(input_request) if type(input_request) == str else input_request\n response = json.loads(response) if type(response) == str else response\n # # parse\n instruction = input_request['output_instructions']\n return input_request, instruction, response\n" }, { "path": "models/grounding.py", "content": "from utilize.apis import get_from_openai\nfrom models.base import Base\n\nicl_examples = {\n 'tmdb': \"\"\"\"Example 1:\nUser Query: give me the number of movies directed by Sofia Coppola\nThought: First, use \"https://api.themoviedb.org/3/search/person\" to search for the **person id** of \"Sofia Coppola\"\nAPI Selection: [1] https://api.themoviedb.org/3/search/person\nExecution Result: the id of Sofia Coppola is **1769**\nThought: Continue, use \"https://api.themoviedb.org/3/person/{person_id}/movie_credits\" to get the number of **movies** directed by Sofia Coppola (person id is 1769)\nAPI Selection: [2] https://api.themoviedb.org/3/person/{person_id}/movie_credits \nExecution Result: The movies directed by this person are Lost in Translation (153), The Virgin Suicides (1443), Marie Antoinette (1887), Somewhere (39210), Lick the Star (92657), The Bling Ring (96936), A Very Murray Christmas (364067), Bed, Bath and Beyond (384947), The Beguiled (399019)\nThought: Last, I am finished executing a plan and have the information the user asked.\nFinal Answer: Sofia Coppola has directed 9 movies\n\nExample 2:\nUser Query: give me a image for the collection Star Wars\nThought: First, use \"https://api.themoviedb.org/3/search/collection\" to search for the collection Star Wars\nAPI Selection: [5] https://api.themoviedb.org/3/search/collection\nExecution Result: The **person id** of the Star Wars collection is 10\nThought: Continue, use \"https://api.themoviedb.org/3/collection/{person_id}/images\" to get the image of the Star Wars collection (10)\nAPI Selection: [6] https://api.themoviedb.org/3/collection/{collection_id}/images\nExecution Result: The images url of Star Wars is abufowb4ui2bion2b2obo.jpg\nThought: I am finished executing a plan and have the information the user asked.\nFinal Answer: The images url of Star Wars is abufowb4ui2bion2b2obo.jpg\n\nExample 3:\nUser Query: Who directed the top-1 rated movie?\nThought: First, use \"https://api.themoviedb.org/3/movie/top_rated\" to search the movie **id** for the top-1 rated movie\nAPI Selection: [3] https://api.themoviedb.org/3/movie/top_rated\nExecution Result: The name of the top-1 rated movie is The Godfather and the **movie id** is **238**\nThought: Continue, based on the movie id The Godfather (238), use \"https://api.themoviedb.org/3/movie/{movie_id}/credits\" to search for the **person id** of its director\nAPI Selection: [4] https://api.themoviedb.org/3/movie/{movie_id}/credits\nExecution Result: The name and id of the director of the movie The Godfather (**person id** 238) is Francis Ford Coppola (**person id** 1776)\nThought: Last, I am finished executing a plan and have the information the user asked.\nFinal Answer: Francis Ford Coppola directed the top-1 rated movie The Godfather\"\"\",\n\n \"tmdb1\": \"\"\"Example 1:\nUser query: give me some movies performed by Tony Leung.\nThought: First, search person with name \"Tony Leung\"\nAPI Selection: \nExecuted results: Tony Leung's person_id is 1337\nThought: Continue, collect the list of movies performed by Tony Leung whose person_id is 1337\nAPI Selection: \nExecuted results: Shang-Chi and the Legend of the Ten Rings, In the Mood for Love, Hero\nThought: I am finished executing a plan and have the information the user asked for or the data the used asked to create\nFinal Answer: Tony Leung has performed in Shang-Chi and the Legend of the Ten Rings, In the Mood for Love, Hero\n\nExample 2:\nUser query: Who wrote the screenplay for the most famous movie directed by Martin Scorsese?\nThought: First, search for the most popular movie directed by Martin Scorsese\nAPI Selection: \nExecuted results: Successfully called GET /search/person to search for the director \"Martin Scorsese\". The id of Martin Scorsese is 1032\nThought: Continue, search for the most popular movie directed by Martin Scorsese (1032)\nAPI Selection: \nExecuted results: Successfully called GET /person/{{person_id}}/movie_credits to get the most popular movie directed by Martin Scorsese. The most popular movie directed by Martin Scorsese is Shutter Island (11324)\nThought: Continue, search for the screenwriter of Shutter Island\nAPI Selection: \nExecuted results: The screenwriter of Shutter Island is Laeta Kalogridis (20294)\nThought: I am finished executing a plan and have the information the user asked for or the data the used asked to create\nFinal Answer: Laeta Kalogridis wrote the screenplay for the most famous movie directed by Martin Scorsese.\n\"\"\",\n \"spotify\": \"\"\"Example 1:\nUser query: set the volume to 20 and skip to the next track.\nPlan step 1: set the volume to 20\nAPI response: Successfully called PUT /me/player/volume to set the volume to 20.\nPlan step 2: skip to the next track\nAPI response: Successfully called POST /me/player/next to skip to the next track.\nThought: I am finished executing a plan and completed the user's instructions\nFinal Answer: I have set the volume to 20 and skipped to the next track.\n\nExample 2:\nUser query: Make a new playlist called \"Love Coldplay\" containing the most popular songs by Coldplay\nPlan step 1: search for the most popular songs by Coldplay\nAPI response: Successfully called GET /search to search for the artist Coldplay. The id of Coldplay is 4gzpq5DPGxSnKTe4SA8HAU\nPlan step 2: Continue. search for the most popular songs by Coldplay (4gzpq5DPGxSnKTe4SA8HAU)\nAPI response: Successfully called GET /artists/4gzpq5DPGxSnKTe4SA8HAU/top-tracks to get the most popular songs by Coldplay. The most popular songs by Coldplay are Yellow (3AJwUDP919kvQ9QcozQPxg), Viva La Vida (1mea3bSkSGXuIRvnydlB5b).\nPlan step 3: make a playlist called \"Love Coldplay\"\nAPI response: Successfully called GET /me to get the user id. The user id is xxxxxxxxx.\nPlan step 4: Continue. make a playlist called \"Love Coldplay\"\nAPI response: Successfully called POST /users/xxxxxxxxx/playlists to make a playlist called \"Love Coldplay\". The playlist id is 7LjHVU3t3fcxj5aiPFEW4T.\nPlan step 5: Add the most popular songs by Coldplay, Yellow (3AJwUDP919kvQ9QcozQPxg), Viva La Vida (1mea3bSkSGXuIRvnydlB5b), to playlist \"Love Coldplay\" (7LjHVU3t3fcxj5aiPFEW4T)\nAPI response: Successfully called POST /playlists/7LjHVU3t3fcxj5aiPFEW4T/tracks to add Yellow (3AJwUDP919kvQ9QcozQPxg), Viva La Vida (1mea3bSkSGXuIRvnydlB5b) in playlist \"Love Coldplay\" (7LjHVU3t3fcxj5aiPFEW4T). The playlist id is 7LjHVU3t3fcxj5aiPFEW4T.\nThought: I am finished executing a plan and have the data the used asked to create\nFinal Answer: I have made a new playlist called \"Love Coldplay\" containing Yellow and Viva La Vida by Coldplay.\n\"\"\"\n}\n\n\nSYSTEM_PLAN = \"\"\"You are an agent that access external APIs and plans solution to user queries by selecting appreciate APIs in a logic order. \nI will provide your external APIs to answer the user's query, and each API is called via HTTP request. You should make a plan to use appreciative APIs by iterating three steps: Thought, API Selection, and Execution Result. \n- Thought: In Thought step, you should reason current situation and specify which API to use. The Thought should be as specific as possible. It is better not to use pronouns in the plan, but to use the corresponding results obtained previously. For example, instead of \"Get the most popular movie directed by this person\", you should output \"Get the most popular movie directed by Martin Scorsese (1032)\". The Thought should be straightforward. If you want to search, sort, or filter, you can put the condition in your plan. For example, if the query is \"Who is the lead actor of In the Mood for Love (person id 843)\", instead of \"Get the list of actors of In the Mood for Love\", you should output \"Get the lead actor of In the Mood for Love (843)\".\n- API Selection: based on the Thought, the API Selection step selects a corresponding API from the following API list. You should just select the API from the below list and DO NOT replace any parameter in the API. \n- Execution Result: your expected execution results for the selected API.\nEach step must writen in one LINE without '\\n'! During the Thought and Execution Result steps, you can mark the key information, e.g., movie id and person id with **. And when finishing, your plan should end with the `Final Answer' to give the final output to the user. \n\nHere are some examples: \n\n[1] /search/person\n## description: Search for people.\n[2] /person/{{person_id}}/movie_credits\n## description: Get the movie credits for a person, the results contains various information such as popularity and release date.\n[3] /movie/top_rated\n## description: Get the top rated movies on TMDb.\n[4] /movie/{{movie_id}}/credits\n## description: Get the cast and crew for a movie.\n[5] /search/collection\n## description: Search for collections.\n[6] /collection/{{collection_id}}/images\n## description: Get the images for a collection by id.\n\nExample 1:\nUser Query: give me the number of movies directed by Sofia Coppola\nThought: First, use \"https://api.themoviedb.org/3/search/person\" to search for the **person id** of \"Sofia Coppola\"\nAPI Selection: [1] https://api.themoviedb.org/3/search/person\nExecution Result: the id of Sofia Coppola is **1769**\nThought: Continue, use \"https://api.themoviedb.org/3/person/{person_id}/movie_credits\" to get the number of **movies** directed by Sofia Coppola (person id is 1769)\nAPI Selection: [2] https://api.themoviedb.org/3/person/{person_id}/movie_credits \nExecution Result: The movies directed by this person are Lost in Translation (153), The Virgin Suicides (1443), Marie Antoinette (1887), Somewhere (39210), Lick the Star (92657), The Bling Ring (96936), A Very Murray Christmas (364067), Bed, Bath and Beyond (384947), The Beguiled (399019)\nThought: Last, I am finished executing a plan and have the information the user asked.\nFinal Answer: Sofia Coppola has directed 9 movies\n\nExample 2:\nUser Query: give me a image for the collection Star Wars\nThought: First, use \"https://api.themoviedb.org/3/search/collection\" to search for the collection Star Wars\nAPI Selection: [5] https://api.themoviedb.org/3/search/collection\nExecution Result: The **person id** of the Star Wars collection is 10\nThought: Continue, use \"https://api.themoviedb.org/3/collection/{person_id}/images\" to get the image of the Star Wars collection (10)\nAPI Selection: [6] https://api.themoviedb.org/3/collection/{collection_id}/images\nExecution Result: The images url of Star Wars is abufowb4ui2bion2b2obo.jpg\nThought: I am finished executing a plan and have the information the user asked.\nFinal Answer: The images url of Star Wars is abufowb4ui2bion2b2obo.jpg\n\nExample 3:\nUser Query: Who directed the top-1 rated movie?\nThought: First, use \"https://api.themoviedb.org/3/movie/top_rated\" to search the movie **id** for the top-1 rated movie\nAPI Selection: [3] https://api.themoviedb.org/3/movie/top_rated\nExecution Result: The name of the top-1 rated movie is The Godfather and the **movie id** is **238**\nThought: Continue, based on the movie id The Godfather (238), use \"https://api.themoviedb.org/3/movie/{movie_id}/credits\" to search for the **person id** of its director\nAPI Selection: [4] https://api.themoviedb.org/3/movie/{movie_id}/credits\nExecution Result: The name and id of the director of the movie The Godfather (**person id** 238) is Francis Ford Coppola (**person id** 1776)\nThought: Last, I am finished executing a plan and have the information the user asked.\nFinal Answer: Francis Ford Coppola directed the top-1 rated movie The Godfather\"\"\"\n\nPLANNER_PROMPT=\"\"\"Using the following API to the user query. You can **only** use the API listed here.\n\nStarting below, you should follow this format:\nUser query: the query a User wants help with related to the API.\nThought: the first step of your plan for how to solve the query. Your must specify the key information you have got (e.g., person id and movie id) and which API to use.\nAPI Selection: selecting an API from the above list based on your `Thought`. Just select an API from the above list, do not change the name of the API.\nExecution Result: the expected result after executing the API. You should specify what you want to obtain from the API request.\nThought: Thought: based on the API response, the second step of your plan for how to solve the query. Pay attention to the specific API called in the last step API response. If a proper API is called, then the response may be wrong and you should give a new plan.\n... (The three steps: Thought, API Selection, and Execution Result, can repeat N times)\nThought: I am finished executing a plan and have the information the user asked for.\nFinal Answer: the final output from executing the plan\n\nBegin!\nAPI List:\n{api_list} \n\nUser query: {query}\n{hidden}\"\"\"\n\nclass GroAgent(Base):\n\n def __init__(self, model='gpt-3.5-turbo', endpoints=None,role='PlanGPT',url=None):\n super().__init__(model=model, endpoints=endpoints,role=role,url=url)\n self.tools = []\n self.endpoints = endpoints\n self.tokens=[]\n\n def generate(self, query, hidden, api_type):\n icl_example = icl_examples[api_type]\n api_list = []\n for i, tool in enumerate(self.endpoints, start=1):\n description = tool[\"description\"].replace('\\n','')\n api_list.append(f'[{i}] {tool[\"url\"]}\\n## description: {description}')\n api_list = '\\n'.join(api_list)\n\n prompt = PLANNER_PROMPT.format(icl_example=icl_example, query=query, api_list=api_list, hidden=hidden)\n res=''\n for i in range(0,4):\n res = get_from_openai(model_name=self.model, temp=0, max_len=1000, stop=['Execution Result'],\n messages=[{'role': \"system\", 'content': SYSTEM_PLAN},\n {'role': 'user', 'content': prompt}])['content']\n if 'final answer' in res.lower():\n res = res.lower()\n id1 = res.index('final answer')\n return 'Finish',res[id1:]\n if 'API Selection:' in res:\n thought, action = res.split('API Selection:')\n thought = self.normalize(thought)\n action = self.normalize(action)\n return thought,action\n\n res=[e for e in res.split('\\n') if e.strip()!='']\n thought = '\\n'.join(res)\n thought=self.normalize(thought)\n prompt += thought + '\\nAPI Selection: '\n action = get_from_openai(model_name=self.model, temp=0, max_len=500, stop='Thought: ',\n messages=[{'role': \"system\", 'content': SYSTEM_PLAN},\n {'role': 'user', 'content': prompt}])['content']\n action=self.normalize(action)\n\n return self.normalize(thought),self.normalize(action)\n" }, { "path": "models/observation.py", "content": "from io import StringIO\nimport tiktoken\nfrom utilize.apis import get_from_openai\nfrom models.base import Base\nfrom pydantic import BaseModel, Field\nimport json\nfrom typing import Optional, Dict\nimport sys\nimport logging\n\nlogger = logging.getLogger()\n\nencoder = tiktoken.encoding_for_model('gpt-3.5-turbo')\n\n\n\nLLM_OBSERVATION_TEMPLATE = \"\"\"Here is an API JSON response with its corresponding Http request:\n\nThe Http request, including url, description and parameters.\n{request}\n\nThe response is about: {response}\n\n====\nYour task is to extract some information according to these instructions: {instruction}\nWhen working with API objects, you should usually use ids over names.\nIf the response indicates an error, you should instead output a summary of the error.\n\nOutput:\n\"\"\"\n\nSYSTEM_OBSE = 'You are CodeGPT, a intelligent assistant which can write Python code to help me extract key information from a complex result of Http requests.'\n\n\nCODE_OBSERVATION_SCHEMA_TEMPLATE = \"\"\"Here is an API response obtained from a HTTP request, which is stored in a python dict variable called 'response', your task is to generate Python code to extract information I need from the API response.\nNote: I will give you 'response', which has been load as a Python dict. Do not make up one, just reference it in your code. DO NOT use fields that are not in the variable `response`.\n\nHere is the OpenAPI specification of `response`.\n## Variable Description\n{description}\n## Variable Structure\n{response_schema}\n\nYou should read the structure of the variable `response` carefully and write code based on my instruction:\nInstruction: {instruction}\n\nThe code you generate should satisfy the following requirements:\n1. The code you generate should contain the filter in the query. For example, if the query is \"what is the name and id of the director of this movie\" and the response is the cast and crew for the movie, instead of directly selecting the first result in the crew list (director_name = data['crew'][0]['name']), the code you generate should have a filter for crews where the job is a \"Director\" (item['job'] == 'Director').\n2. Do not use f-string in the print function. Use \"format\" instead. For example, use \"print('The release date of the album is {{}}'.format(date))\" instead of \"print(f'The release date of the album is {{date}}')\n3. If the instruction includes expressions such as \"most\", you should choose the first item from the response. For example, if the plan is \"GET /trending/tv/day to get the most trending TV show today\", you should choose the first item from the response.\n4. Please print the final result as brief as possible. If the result is a list, just print it in one sentence. Do not print each item in a new line.\nThe example result format are:\n\"The release date of the album is 2002-11-03\"\n\"The id of the person is 12345\"\n\"The movies directed by Wong Kar-Wai are In the Mood for Love (843), My Blueberry Nights (1989), Chungking Express (11104)\"\n\nBegin and complete the [Python code]!\nPython Code:\n```python\n[Python code]\n```\"\"\"\n\n\n\nclass PythonREPL(BaseModel):\n \"\"\"Simulates a standalone Python REPL.\"\"\"\n\n globals: Optional[Dict] = Field(default_factory=dict, alias=\"_globals\")\n locals: Optional[Dict] = Field(default_factory=dict, alias=\"_locals\")\n\n def run(self, command: str):\n \"\"\"Run command with own globals/locals and returns anything printed.\"\"\"\n old_stdout = sys.stdout\n sys.stdout = mystdout = StringIO()\n status = True\n try:\n exec(command, self.globals, self.locals)\n sys.stdout = old_stdout\n output = mystdout.getvalue()\n except Exception as e:\n sys.stdout = old_stdout\n # print(str(e))\n output = '\\n'.join([\n '-----------------------------',\n f'except Exception as e - Exception Type: {type(e)}',\n f'except Exception as e - Exception Value: {e}'\n ])\n status = False\n return status, output\n\n\ndef get_yaml(value, name, indent=0):\n result = ['\\t' * indent + f\"- {name}: {type(value).__name__}\"]\n\n if list == type(value):\n element = f'{name}[0]'\n indent += 1\n if value != []:\n result.extend(get_yaml(value[0], element, indent))\n\n elif dict == type(value):\n for k, v in value.items():\n result.extend(get_yaml(v, k, indent + 1))\n return result\n\n\nclass ObseAgent(Base):\n def __init__(self, model='gpt-3.5-turbo', endpoints=None, role='ParseGPT', url=None):\n super().__init__(model=model, endpoints=endpoints, role=role, url=url)\n\n def generate(self, instruction, json_request, response):\n if len(encoder.encode(str(response))) < 1500:\n extract_prompt = LLM_OBSERVATION_TEMPLATE.format(request=json_request, instruction=instruction,\n response=json.dumps(response)[:1500])\n res = get_from_openai(model_name=self.model, temp=0, messages=[{\"role\": \"user\", 'content': extract_prompt}])['content']\n res = res.replace('\\n', '').replace('-', '').strip()\n logger.info(f'ObseAgent: \\n{res}')\n return res\n\n response = json.loads(response) if type(response)==str else response\n\n tree_struct = '\\n'.join(get_yaml(response, 'response'))\n prompt = CODE_OBSERVATION_SCHEMA_TEMPLATE.format(response_schema=tree_struct, instruction=instruction, description=json_request['description'])\n messages = [{'role': \"system\", 'content': SYSTEM_OBSE}, {'role': 'user', 'content': prompt}]\n res = None\n status = False\n for i in range(0, 3):\n code = get_from_openai(model_name=self.model, temp=0.5,messages=messages)['content']\n if '```' in code and code.index('```') != code.rindex('```'):\n code = code[code.index('```'):code.rindex('```')]\n code = code.replace('```', '').replace('python', '').strip()\n elif '```' in code:\n code = code[code.index('```'):]\n code = code.replace('```', '').replace('python', '').strip()\n logger.info(f\"Code: \\n{code}\")\n repl = PythonREPL(_globals={\"response\": response})\n status, res = repl.run(code)\n if status:\n break\n messages.append({'role': 'assistant', 'content': code})\n messages.append({'role': 'user',\n 'content': f'Your code encountered an error (bug) during runtime, and the specific error message is as follows:\\n{res}\\n\\n'\n f'Please fix your bug and give me correct code to complete the instruction: {instruction}.\\n'\n f'Just give me the code without any extra words\\n'\n f'```python\\n'\n f'[Python code]\\n'\n f'```\\n'})\n logger.info(f'Parser error: \\n{res}')\n\n if res is None or 'None' in res or res == [] or '[]' in res or status == False or res == '':\n extract_prompt = LLM_OBSERVATION_TEMPLATE.format(request=json_request, instruction=instruction, response=json.dumps(response)[:1500])\n res = get_from_openai(model_name=self.model, temp=0,messages=[{\"role\": \"user\", 'content': extract_prompt}])['content']\n if len(res.split(' ')) > 1000:\n res = res[:1000] + '...'\n logger.info(f'ObseAgent: \\n{res}')\n return res\n" }, { "path": "run_tmdb.py", "content": "from langchain.requests import Requests\nimport logging\nfrom utilize.utilze import ColorPrint, load_data, write_file, mean\nfrom models.execution import ExecAgent\nfrom models.observation import ObseAgent\nfrom models.grounding import GroAgent\nfrom tqdm import tqdm\nimport argparse\n\nlogger = logging.getLogger()\n\n\ndef run(data, model_name, headers, endpoints, url):\n # initialize the environment\n logging.basicConfig(\n format=\"%(message)s\",\n handlers=[logging.StreamHandler(ColorPrint())],\n )\n logger.setLevel(logging.INFO)\n\n # the headers used to request the API servers\n requests_wrapper = Requests(headers=headers)\n\n name2doc = {line['url']: line for line in endpoints}\n\n results = [] # save the results\n\n for line in tqdm(data):\n logger.info(f'Query: {line[\"query\"]}')\n logger.info(f'Query: {line[\"solution\"]}') # The most simple setting for intuitive illustration. You can add more tools based on your own setting\n tools = [name2doc[tool] for tool in line['solution']]\n\n # initialize the three agents\n p_model = GroAgent(model_name, tools, url=url)\n a_model = ExecAgent(model_name, tools, requests_wrapper=requests_wrapper, url=url)\n o_model = ObseAgent(model_name, tools, url=url)\n # try:\n result = _run(line, p_model, a_model, o_model)\n results.append(result)\n # except Exception as e:\n # print(f'error...{e}')\n\n return results\n\n\ndef _run(example, p_model: GroAgent, a_model: ExecAgent, o_model: ObseAgent):\n result = []\n background = example['query']\n hidden = 'Thought: '\n for i in range(5):\n # ground the user's instruction into a tool-use instruction\n thought, action = p_model.generate(query=example['query'], hidden=hidden, api_type='tmdb')\n if thought.lower() == 'finish':\n print(thought, '\\n', action)\n break\n logger.info('GroAgent: ' + thought + '->' + action)\n\n # execution the tool-use instruction and get the execution results\n input_request, instruction, response = a_model.generate(thought=thought, action=action, background=background)\n\n # observe the useful information from the length execution results\n observation = o_model.generate(instruction=instruction, json_request=input_request, response=response)\n observation = observation.strip().replace('\\n', ' ')\n\n # update the context and incorporate the extracted information into the next action prediction\n background += ' -> ' + observation\n hidden += f'{thought}\\nAPI Selection: {action}\\nExecuted Result: {observation}\\nThought: '\n result.append([thought,action,observation])\n return result\n\nif __name__ == '__main__':\n parser = argparse.ArgumentParser()\n parser.add_argument('--model_name', type=str, default='gpt-3.5-turbo', required=False)\n parser.add_argument('--log_file', type=str, help='your log file to save the output trajectory', required=True)\n parser.add_argument('--data_file', type=str, help='your data file containing the test examples and tools', required=True)\n parser.add_argument('--access_token_file', type=str, help='the file containing the access token required by TMDB', required=True)\n\n args = parser.parse_args()\n\n with open(args.access_token_file) as f:\n access_token = f.read().strip()\n headers = {'Authorization': f'Bearer {access_token}'}\n print(headers)\n data = load_data(args.data_file)['data']\n tools = load_data(args.data_file)['tool']\n\n results = run(model_name=args.model_name, headers=headers, endpoints=tools, data=data, url=None)\n write_file(results, args.log_file)\n" }, { "path": "utilize/__init__.py", "content": "" }, { "path": "utilize/apis.py", "content": "import json\nimport sys\nfrom openai import OpenAI\nimport time\nimport random\n\n# we provide the\napi_keys_list=[\n\n]\nBASE_URL = ''\n\ndef get_from_openai(model_name='gpt-3.5-turbo',api_key=None,\n messages=None, prompt=None,stop=None,max_len=1000, temp=1, n=1,\n json_mode=False, usage=False):\n \"\"\"\n :param model_name: text-davinci-003, gpt-3.5-turbo, gpt-3.5-turbo-0613\n \"\"\"\n for i in range(10):\n try:\n client = OpenAI(api_key=api_keys_list[random.randint(0, 100000) % len(api_keys_list)] if api_key==None else api_key,\n base_url=BASE_URL)\n kwargs={\n \"model\":model_name, 'max_tokens':max_len,\"temperature\":temp,\n \"n\":n, 'stop':stop, 'messages':messages,\n }\n if json_mode==True:\n kwargs['response_format']={\"type\": \"json_object\"}\n if 'instruct' in model_name:\n assert prompt!=None\n kwargs['prompt']=prompt\n response= client.completions.create(**kwargs)\n else:\n assert messages!=None\n kwargs['messages']=messages\n response= client.chat.completions.create(**kwargs)\n\n content = response.choices[0].message.content if n == 1 else [res.message.content for res in response.choices]\n results={\"content\":content}\n if usage==False:\n results['usage']=response.usage\n return results\n except:\n error = sys.exc_info()[0]\n print(\"API error:\", error)\n time.sleep(1)\n return 'no response from openai model...'\n\n\n" }, { "path": "utilize/utilze.py", "content": "import sys\nsys.path.append('../../')\nimport pickle\nimport multiprocessing\nimport json\nimport torch\nimport random\nimport os\nimport numpy as np\nfrom tqdm import tqdm\nimport torch.multiprocessing as mp\nfrom colorama import Fore\n\nclass ColorPrint:\n def __init__(self):\n self.color_mapping = {\n 'Query':Fore.WHITE,\n 'Ground truth':Fore.WHITE,\n \"GroAgent\": Fore.GREEN,\n \"ExecAgent\": Fore.YELLOW,\n \"ObseAgent\": Fore.BLUE,\n \"Error\": Fore.RED,\n }\n\n def write(self, data):\n module = data.split(':')[0]\n if module not in self.color_mapping:\n print(data, end=\"\")\n else:\n print(self.color_mapping[module] + data + Fore.RESET, end=\"\")\n\ndef mean(li,r=4):\n return round((sum(li))/(len(li)+0.0001),r)\n\ndef seed_torch(seed=1048):\n random.seed(seed)\n os.environ['PYTHONHASHSEED'] = str(seed) # disable the hash random\n np.random.seed(seed)\n torch.manual_seed(seed)\n torch.cuda.manual_seed(seed)\n torch.cuda.manual_seed_all(seed) # if you are using multi-GPU.\n # torch.backends.cudnn.benchmark = False\n torch.backends.cudnn.deterministic = True\n\ndef multi_load_jsonl(filename,num_processes=10):\n \"\"\"\n\n :param filename: the jsonl file with big size\n :param num_processes:\n :return:\n \"\"\"\n with open(filename,'r',encoding='utf-8') as f:\n data=[line.strip() for line in f]\n if len(data)<=20000:\n\n _,data=load_jsonl(0,data)\n return data\n\n length = len(data) // num_processes + 1\n pool=multiprocessing.Pool(processes=num_processes)\n collects=[]\n for ids in range(num_processes):\n collect = data[ids * length:(ids + 1) * length]\n collects.append(pool.apply_async(load_jsonl,(ids,collect)))\n\n pool.close()\n pool.join()\n results=[]\n for i,result in enumerate(collects):\n ids,res=result.get()\n assert ids==i\n results.extend(res)\n print(f\"*************************** total {len(results)} examples ****************************\")\n return results\n\ndef load_jsonl(ids,data):\n data=[json.loads(line) for line in tqdm(data)]\n return ids,data\n\ndef write_file(data,filename,num_processes=20,default_name='train',indent=4):\n print(f\"************************** begin to write data to {filename} *******************************\")\n if filename.endswith('.json'):\n json.dump(data,open(filename,'w'),indent=indent)\n elif filename.endswith('.jsonl') :\n with open(filename, 'w') as f:\n for line in data:\n f.write(json.dumps(line) + '\\n')\n elif filename.endswith('.txt'):\n with open(filename, 'w') as f:\n for line in data:\n f.write(str(line) + '\\n')\n elif filename.endswith('.pkl'):\n pickle.dump(data,open(filename,'w'))\n elif '.' not in filename:\n multi_write_jsonl(data,filename,num_processes=num_processes,default_name=default_name)\n else:\n raise \"no suitable function to write data\"\n print(f\"************************** totally {len(data)} writing data to {filename} *******************************\")\n\ndef write_jsonl(data,filename,ids=None):\n with open(filename,'w') as f:\n for line in tqdm(data):\n f.write(json.dumps(line)+'\\n')\n return ids,len(data)\n\ndef multi_write_jsonl(data,folder,num_processes=10,default_name='train'):\n \"\"\"\n\n :param filename:\n :param num_processes:\n :return:\n \"\"\"\n if not os.path.exists(folder):\n os.makedirs(folder)\n length = len(data) // num_processes + 1\n pool=multiprocessing.Pool(processes=num_processes)\n collects=[]\n for ids in range(num_processes):\n filename=os.path.join(folder,f\"{default_name}{ids}.jsonl\")\n collect = data[ids * length:(ids + 1) * length]\n collects.append(pool.apply_async(write_jsonl,(collect,filename,ids)))\n\n pool.close()\n pool.join()\n cnt=0\n for i,result in enumerate(collects):\n ids,num=result.get()\n assert ids==i\n cnt+=num\n print(f\"** total {cnt} examples have been writen to {folder} **\")\n return cnt\n\ndef load_data(filename,num_processes=10):\n print(f\"************************** begin to load data of {filename} *******************************\")\n if filename.endswith('.jsonl'):\n return multi_load_jsonl(filename,num_processes)\n elif filename.endswith('.json'):\n return json.load(open(filename,'r'))\n elif filename.endswith('.pkl'):\n return pickle.load(filename)\n elif filename.endswith('.txt'):\n with open(filename,'r') as f:\n data=[line.strip() for line in f]\n return data\n else:\n raise \"no suitable function to load data\"\n\n\ndef multi_process_cuda(data_path,ranks,func,**kwargs):\n \"\"\"\n\n :param data_path: data path\n :param ranks: gpu device id\n :param func: the function for batch\n :param kwargs: the 'dict', indicating the parameter to pass into the 'func'\n :return:\n \"\"\"\n torch.multiprocessing.set_start_method('spawn',force=True)\n cuda_pool=mp.Pool(processes=len(ranks))\n data=load_data(data_path)\n length = len(data) // len(ranks) + 1\n collects=[]\n for ids,rank in enumerate(ranks):\n collect = data[ids * length:(ids + 1) * length]\n collects.append(cuda_pool.apply_async(func, (rank,collect,kwargs)))\n cuda_pool.close()\n cuda_pool.join()\n results=[]\n for rank,result in zip(ranks,collects):\n r,res=result.get()\n assert r==rank\n results.extend(res)\n return results\n\ndef multi_process_cuda_data(data,ranks,func,**kwargs):\n \"\"\"\n\n :param data_path: the data path\n :param ranks: gpu device ids\n :param func:\n :param kwargs:\n :return:\n \"\"\"\n torch.multiprocessing.set_start_method('spawn',force=True)\n cuda_pool=mp.Pool(processes=len(ranks))\n length = len(data) // len(ranks) + 1\n collects=[]\n for ids,rank in enumerate(ranks):\n collect = data[ids * length:(ids + 1) * length]\n collects.append(cuda_pool.apply_async(func, (rank,collect,kwargs)))\n cuda_pool.close()\n cuda_pool.join()\n results=[]\n for rank,result in zip(ranks,collects):\n r,res=result.get()\n assert r==rank\n results.extend(res)\n return results\n" } ]