Repository: KalyanKS-NLP/llm-engineer-toolkit
Branch: main
Commit: a0b609f81582
Files: 3
Total size: 48.6 KB
Directory structure:
gitextract_5atf4lwj/
├── Images/
│ └── ReadMe.md
├── LICENSE
└── README.md
================================================
FILE CONTENTS
================================================
================================================
FILE: Images/ReadMe.md
================================================
================================================
FILE: LICENSE
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: README.md
================================================
# 👨🏻💻 LLM Engineer Toolkit
This repository contains a curated list of 120+ LLM libraries category wise.
<p align="center">
<a href="https://www.linkedin.com/in/kalyanksnlp/">
<img src="https://custom-icon-badges.demolab.com/badge/Kalyan%20KS-0A66C2?logo=linkedin-white&logoColor=fff" alt="LinkedIn">
</a>
<a href="https://x.com/kalyan_kpl">
<img src="https://img.shields.io/badge/Kalyan%20KS-%23000000.svg?logo=X&logoColor=white" alt="Twitter">
</a>
<a href="https://www.youtube.com/@kalyanksnlp">
<img src="https://img.shields.io/badge/Kalyan%20KS-%23FF0000.svg?logo=YouTube&logoColor=white" alt="Twitter">
</a>
</p>
## 🚀 LLM Interview Questions and Answers Book
Crack modern LLM and Generative AI interviews with this comprehensive, interview-focused guide designed specifically for ML Engineers, AI Engineers, Data Scientists and Software Engineers.
This book features **100+ carefully curated LLM interview questions**, each paired with **clear answers and in-depth explanations** so you truly understand the concepts interviewers care about. [Get the book here](https://kalyanksnlp.gumroad.com/l/llm-interview-questions-answers-book-kalyan-ks).
Use the **Coupon Code: LLMQA25** for an exclusive discount (50%) on the book. (Available only for a short period of time).
## Related Repositories
- 👨🏻💻[LLM Interview Questions and Answers Hub](https://github.com/KalyanKS-NLP/LLM-Interview-Questions-and-Answers-Hub) - 100+ LLM interview questions and answers (basic to advanced).
- 🚀[Prompt Engineering Techniques Hub](https://github.com/KalyanKS-NLP/Prompt-Engineering-Techniques-Hub) - 25+ prompt engineering techniques with LangChain implementations.
- 🩸[LLM, RAG and Agents Survey Papers Collection](https://github.com/KalyanKS-NLP/LLM-Survey-Papers-Collection) - Category wise collection of 200+ survey papers.
## Stay Updated with Generative AI, LLMs, Agents and RAG.
Join 🚀 [**AIxFunda** free newsletter](https://aixfunda.substack.com/) to get *latest updates* and *interesting tutorials* related to Generative AI, LLMs, Agents and RAG.
- ✨ Weekly GenAI updates
- 📄 Weekly LLM, Agents and RAG paper updates
- 📝 1 fresh blog post on an interesting topic every week
## Quick links
||||
|---|---|---|
| [🚀 LLM Training](#llm-training-and-fine-tuning) | [🧱 LLM Application Development](#llm-application-development) | [🩸LLM RAG](#llm-rag) |
| [🟩 LLM Inference](#llm-inference)| [🚧 LLM Serving](#llm-serving) | [📤 LLM Data Extraction](#llm-data-extraction) |
| [🌠 LLM Data Generation](#llm-data-generation) | [💎 LLM Agents](#llm-agents)|[⚖️ LLM Evaluation](#llm-evaluation) |
| [🔍 LLM Monitoring](#llm-monitoring) | [📅 LLM Prompts](#llm-prompts) | [📝 LLM Structured Outputs](#llm-structured-outputs) |
| [🛑 LLM Safety and Security](#llm-safety-and-security) | [💠 LLM Embedding Models](#llm-embedding-models) | [❇️ Others](#others) |
## LLM Training and Fine-Tuning
| Library | Description | Link |
|---------------------|-------------------------------------------------------------------------------------------------|------|
| unsloth | Fine-tune LLMs faster with less memory. | [Link](https://github.com/unslothai/unsloth) |
| PEFT | State-of-the-art Parameter-Efficient Fine-Tuning library. | [Link](https://github.com/huggingface/peft) |
| TRL | Train transformer language models with reinforcement learning. | [Link](https://github.com/huggingface/trl) |
| Transformers | Transformers provides thousands of pretrained models to perform tasks on different modalities such as text, vision, and audio. | [Link](https://github.com/huggingface/transformers) |
| Axolotl | Tool designed to streamline post-training for various AI models. | [Link](https://github.com/axolotl-ai-cloud/axolotl/) |
| LLMBox | A comprehensive library for implementing LLMs, including a unified training pipeline and comprehensive model evaluation. | [Link](https://github.com/RUCAIBox/LLMBox) |
| LitGPT | Train and fine-tune LLM lightning fast. | [Link](https://github.com/Lightning-AI/litgpt) |
| Mergoo | A library for easily merging multiple LLM experts, and efficiently train the merged LLM. | [Link](https://github.com/Leeroo-AI/mergoo) |
| Llama-Factory | Easy and efficient LLM fine-tuning. | [Link](https://github.com/hiyouga/LLaMA-Factory) |
| Ludwig | Low-code framework for building custom LLMs, neural networks, and other AI models. | [Link](https://github.com/ludwig-ai/ludwig) |
| Txtinstruct | A framework for training instruction-tuned models. | [Link](https://github.com/neuml/txtinstruct) |
| Lamini | An integrated LLM inference and tuning platform. | [Link](https://github.com/lamini-ai/lamini) |
| XTuring | xTuring provides fast, efficient and simple fine-tuning of open-source LLMs, such as Mistral, LLaMA, GPT-J, and more. | [Link](https://github.com/stochasticai/xTuring) |
| RL4LMs | A modular RL library to fine-tune language models to human preferences. | [Link](https://github.com/allenai/RL4LMs) |
| DeepSpeed | DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective. | [Link](https://github.com/deepspeedai/DeepSpeed) |
| torchtune | A PyTorch-native library specifically designed for fine-tuning LLMs. | [Link](https://github.com/pytorch/torchtune) |
| PyTorch Lightning | A library that offers a high-level interface for pretraining and fine-tuning LLMs. | [Link](https://github.com/Lightning-AI/pytorch-lightning) |
## LLM Application Development
<p align = "center"> <b> Frameworks </b> </p>
| Library | Description | Link |
|--------------|------------------------------------------------------------------------------------------------------|-------|
| LangChain | LangChain is a framework for developing applications powered by large language models (LLMs). | [Link](https://github.com/langchain-ai/langchain) |
| Llama Index | LlamaIndex is a data framework for your LLM applications. | [Link](https://github.com/run-llama/llama_index) |
| HayStack | Haystack is an end-to-end LLM framework that allows you to build applications powered by LLMs, Transformer models, vector search and more. | [Link](https://github.com/deepset-ai/haystack) |
| Prompt flow | A suite of development tools designed to streamline the end-to-end development cycle of LLM-based AI applications. | [Link](https://github.com/microsoft/promptflow) |
| Griptape | A modular Python framework for building AI-powered applications. | [Link](https://github.com/griptape-ai/griptape) |
| Weave | Weave is a toolkit for developing Generative AI applications. | [Link](https://github.com/wandb/weave) |
| Llama Stack | Build Llama Apps. | [Link](https://github.com/meta-llama/llama-stack) |
<p align = "center"> <b> Data Preparation </b> </p>
| Library | Description | Link |
|--------------|------------------------------------------------------------------------------------------------------|-------|
| Data Prep Kit | Data Prep Kit accelerates unstructured data preparation for LLM app developers. Developers can use Data Prep Kit to cleanse, transform, and enrich use case-specific unstructured data to pre-train LLMs, fine-tune LLMs, instruct-tune LLMs, or build RAG applications. | [Link](https://github.com/data-prep-kit/data-prep-kit) |
<p align = "center"> <b> Multi API Access </b> </p>
| Library | Description | Link |
|--------------|------------------------------------------------------------------------------------------------------|-------|
| LiteLLM | Library to call 100+ LLM APIs in OpenAI format. | [Link](https://github.com/BerriAI/litellm) |
| AI Gateway | A Blazing Fast AI Gateway with integrated Guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API. | [Link](https://github.com/Portkey-AI/gateway) |
<p align = "center"> <b> Routers </b> </p>
| Library | Description | Link |
|--------------|------------------------------------------------------------------------------------------------------|-------|
| RouteLLM | Framework for serving and evaluating LLM routers - save LLM costs without compromising quality. Drop-in replacement for OpenAI's client to route simpler queries to cheaper models. | [Link](https://github.com/lm-sys/RouteLLM) |
<p align = "center"> <b> Memory </b> </p>
| Library | Description | Link |
|--------------|------------------------------------------------------------------------------------------------------|-------|
| mem0 | The Memory layer for your AI apps. | [Link](https://github.com/mem0ai/mem0) |
| Memoripy | An AI memory layer with short- and long-term storage, semantic clustering, and optional memory decay for context-aware applications. | [Link](https://github.com/caspianmoon/memoripy) |
| Letta (MemGPT) | An open-source framework for building stateful LLM applications with advanced reasoning capabilities and transparent long-term memory | [Link](https://github.com/letta-ai/letta) |
| Memobase | A user profile-based memory system designed to bring long-term user memory to your Generative AI applications. | [Link](https://github.com/memodb-io/memobase) |
<p align = "center"> <b> Interface </b> </p>
| Library | Description | Link |
|--------------|------------------------------------------------------------------------------------------------------|-------|
| Streamlit | A faster way to build and share data apps. Streamlit lets you transform Python scripts into interactive web apps in minutes | [Link](https://github.com/streamlit/streamlit) |
| Gradio | Build and share delightful machine learning apps, all in Python. | [Link](https://github.com/gradio-app/gradio) |
| AI SDK UI | Build chat and generative user interfaces. | [Link](https://sdk.vercel.ai/docs/introduction) |
| AI-Gradio | Create AI apps powered by various AI providers. | [Link](https://github.com/AK391/ai-gradio) |
| Simpleaichat | Python package for easily interfacing with chat apps, with robust features and minimal code complexity. | [Link](https://github.com/minimaxir/simpleaichat) |
| Chainlit | Build production-ready Conversational AI applications in minutes. | [Link](https://github.com/Chainlit/chainlit) |
<p align = "center"> <b> Low Code </b> </p>
| Library | Description | Link |
|--------------|------------------------------------------------------------------------------------------------------|-------|
| LangFlow | LangFlow is a low-code app builder for RAG and multi-agent AI applications. It’s Python-based and agnostic to any model, API, or database. | [Link](https://github.com/langflow-ai/langflow) |
<p align = "center"> <b> Cache </b> </p>
| Library | Description | Link |
|--------------|------------------------------------------------------------------------------------------------------|-------|
| GPTCache | A Library for Creating Semantic Cache for LLM Queries. Slash Your LLM API Costs by 10x 💰, Boost Speed by 100x. Fully integrated with LangChain and LlamaIndex. | [Link](https://github.com/zilliztech/gptcache) |
## LLM RAG
| Library | Description | Link |
|---------------|----------------------------------------------------------------------------------------------------------------|-------|
| FastGraph RAG | Streamlined and promptable Fast GraphRAG framework designed for interpretable, high-precision, agent-driven retrieval workflows. | [Link](https://github.com/circlemind-ai/fast-graphrag) |
| Chonkie | RAG chunking library that is lightweight, lightning-fast, and easy to use. | [Link](https://github.com/chonkie-ai/chonkie) |
| RAGChecker | A Fine-grained Framework For Diagnosing RAG. | [Link](https://github.com/amazon-science/RAGChecker) |
| RAG to Riches | Build, scale, and deploy state-of-the-art Retrieval-Augmented Generation applications. | [Link](https://github.com/SciPhi-AI/R2R) |
| BeyondLLM | Beyond LLM offers an all-in-one toolkit for experimentation, evaluation, and deployment of Retrieval-Augmented Generation (RAG) systems. | [Link](https://github.com/aiplanethub/beyondllm) |
| SQLite-Vec | A vector search SQLite extension that runs anywhere! | [Link](https://github.com/asg017/sqlite-vec) |
| fastRAG | fastRAG is a research framework for efficient and optimized retrieval-augmented generative pipelines, incorporating state-of-the-art LLMs and Information Retrieval. | [Link](https://github.com/IntelLabs/fastRAG) |
| FlashRAG | A Python Toolkit for Efficient RAG Research. | [Link](https://github.com/RUC-NLPIR/FlashRAG) |
| Llmware | Unified framework for building enterprise RAG pipelines with small, specialized models. | [Link](https://github.com/llmware-ai/llmware) |
| Rerankers | A lightweight unified API for various reranking models. | [Link](https://github.com/AnswerDotAI/rerankers) |
| Vectara | Build Agentic RAG applications. | [Link](https://vectara.github.io/py-vectara-agentic/latest/) |
## LLM Inference
| Library | Description | Link |
|---------------|------------------------------------------------------------------------------------------------------|-------|
| llama.cpp | LLM inference in C/C++. | [Link](https://github.com/ggml-org/llama.cpp) |
| Ollama | Local LLM inference. | [Link](https://github.com/ollama/ollama) |
| vLLM | High-throughput and memory-efficient inference and serving engine for LLMs. | [Link](https://github.com/vllm-project/vllm) |
| TensorRT-LLM | TensorRT-LLM is a library for optimizing Large Language Model (LLM) inference. | [Link](https://github.com/NVIDIA/TensorRT-LLM) |
| WebLLM | High-performance In-browser LLM Inference Engine. | [Link](https://github.com/mlc-ai/web-llm) |
| LLM Compressor | Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment. | [Link](https://github.com/vllm-project/llm-compressor) |
| LightLLM | Python-based LLM inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance. | [Link](https://github.com/ModelTC/lightllm) |
| torchchat | Run PyTorch LLMs locally on servers, desktop, and mobile. | [Link](https://github.com/pytorch/torchchat) |
## LLM Serving
| Library | Description | Link |
|-----------|--------------------------------------------------------------------------|-------|
| Langcorn | Serving LangChain LLM apps and agents automagically with FastAPI. | [Link](https://github.com/msoedov/langcorn) |
| LitServe | Lightning-fast serving engine for any AI model of any size. It augments FastAPI with features like batching, streaming, and GPU autoscaling. | [Link](https://github.com/Lightning-AI/LitServe) |
## LLM Data Extraction
| Library | Description | Link |
|----------------|---------------------------------------------------------------------------------------------------------------------------------------|-------|
| Crawl4AI | Open-source LLM Friendly Web Crawler & Scraper. | [Link](https://github.com/unclecode/crawl4ai) |
| ScrapeGraphAI | A web scraping Python library that uses LLM and direct graph logic to create scraping pipelines for websites and local documents (XML, HTML, JSON, Markdown, etc.). | [Link](https://github.com/ScrapeGraphAI/Scrapegraph-ai) |
| Docling | Docling parses documents and exports them to the desired format with ease and speed. | [Link](https://github.com/DS4SD/docling) |
| Llama Parse | GenAI-native document parser that can parse complex document data for any downstream LLM use case (RAG, agents). | [Link](https://github.com/run-llama/llama_cloud_services) |
| PyMuPDF4LLM | PyMuPDF4LLM library makes it easier to extract PDF content in the format you need for LLM & RAG environments. | [Link](https://pymupdf.readthedocs.io/en/latest/pymupdf4llm/) |
| Crawlee | A web scraping and browser automation library. | [Link](https://github.com/apify/crawlee-python) |
| MegaParse | Parser for every type of document. | [Link](https://github.com/quivrhq/megaparse) |
| ExtractThinker | Document Intelligence library for LLMs. | [Link](https://github.com/enoch3712/ExtractThinker) |
## LLM Data Generation
| Library | Description | Link |
|--------------|--------------------------------------------------------------------------------------------------|-------|
| DataDreamer | DataDreamer is a powerful open-source Python library for prompting, synthetic data generation, and training workflows. | [Link](https://github.com/datadreamer-dev/DataDreamer) |
| fabricator | A flexible open-source framework to generate datasets with large language models. | [Link](https://github.com/flairNLP/fabricator) |
| Promptwright | Synthetic Dataset Generation Library. | [Link](https://github.com/stacklok/promptwright) |
| EasyInstruct | An Easy-to-use Instruction Processing Framework for Large Language Models. | [Link](https://github.com/zjunlp/EasyInstruct) |
## LLM Agents
| Library | Description | Link |
|----------------|---------------------------------------------------------------------------------------------------------|-------|
| CrewAI | Framework for orchestrating role-playing, autonomous AI agents. | [Link](https://github.com/crewAIInc/crewAI) |
| LangGraph | Build resilient language agents as graphs. | [Link](https://github.com/langchain-ai/langgraph) |
| Agno | Build AI Agents with memory, knowledge, tools, and reasoning. Chat with them using a beautiful Agent UI. | [Link](https://github.com/agno-agi/agno) |
| Agents SDK | Build agentic apps using LLMs with context, tools, hand off to other specialized agents. | [Link](https://platform.openai.com/docs/guides/agents-sdk) |
| AutoGen | An open-source framework for building AI agent systems. | [Link](https://github.com/microsoft/autogen) |
| Smolagents | Library to build powerful agents in a few lines of code. | [Link](https://github.com/huggingface/smolagents) |
| Pydantic AI | Python agent framework to build production grade applications with Generative AI. | [Link](https://ai.pydantic.dev/) |
| CAMEL | Open-source multi-agent framework with various toolkits and use-cases available. | [Link](https://github.com/camel-ai/camel) |
| BeeAI | Build production-ready multi-agent systems in Python. | [Link](https://github.com/i-am-bee/beeai-framework/tree/main/python) |
| gradio-tools | A Python library for converting Gradio apps into tools that can be leveraged by an LLM-based agent to complete its task. | [Link](https://github.com/freddyaboulton/gradio-tools) |
| Composio | Production Ready Toolset for AI Agents. | [Link](https://github.com/ComposioHQ/composio) |
| Atomic Agents | Building AI agents, atomically. | [Link](https://github.com/BrainBlend-AI/atomic-agents) |
| Memary | Open Source Memory Layer For Autonomous Agents. | [Link](https://github.com/kingjulio8238/Memary) |
| Browser Use | Make websites accessible for AI agents. | [Link](https://github.com/browser-use/browser-use) |
| OpenWebAgent | An Open Toolkit to Enable Web Agents on Large Language Models. | [Link](https://github.com/THUDM/OpenWebAgent/) |
| Lagent | A lightweight framework for building LLM-based agents. | [Link](https://github.com/InternLM/lagent) |
| LazyLLM | A Low-code Development Tool For Building Multi-agent LLMs Applications. | [Link](https://github.com/LazyAGI/LazyLLM) |
| Swarms | The Enterprise-Grade Production-Ready Multi-Agent Orchestration Framework. | [Link](https://github.com/kyegomez/swarms) |
| ChatArena | ChatArena is a library that provides multi-agent language game environments and facilitates research about autonomous LLM agents and their social interactions. | [Link](https://github.com/Farama-Foundation/chatarena) |
| Swarm | Educational framework exploring ergonomic, lightweight multi-agent orchestration. | [Link](https://github.com/openai/swarm) |
| AgentStack | The fastest way to build robust AI agents. | [Link](https://github.com/AgentOps-AI/AgentStack) |
| Archgw | Intelligent gateway for Agents. | [Link](https://github.com/katanemo/archgw) |
| Flow | A lightweight task engine for building AI agents. | [Link](https://github.com/lmnr-ai/flow) |
| AgentOps | Python SDK for AI agent monitoring. | [Link](https://github.com/AgentOps-AI/agentops) |
| Langroid | Multi-Agent framework. | [Link](https://github.com/langroid/langroid) |
| Agentarium | Framework for creating and managing simulations populated with AI-powered agents. | [Link](https://github.com/Thytu/Agentarium) |
| Upsonic | Reliable AI agent framework that supports MCP. | [Link](https://github.com/upsonic/upsonic) |
## LLM Evaluation
| Library | Description | Link |
|------------|-----------------------------------------------------------------------------------------------------------------|-------|
| Ragas | Ragas is your ultimate toolkit for evaluating and optimizing Large Language Model (LLM) applications. | [Link](https://github.com/explodinggradients/ragas) |
| Giskard | Open-Source Evaluation & Testing for ML & LLM systems. | [Link](https://github.com/Giskard-AI/giskard) |
| DeepEval | LLM Evaluation Framework | [Link](https://github.com/confident-ai/deepeval) |
| Lighteval | All-in-one toolkit for evaluating LLMs. | [Link](https://github.com/huggingface/lighteval) |
| Trulens | Evaluation and Tracking for LLM Experiments | [Link](https://github.com/truera/trulens) |
| PromptBench | A unified evaluation framework for large language models. | [Link](https://github.com/microsoft/promptbench) |
| LangTest | Deliver Safe & Effective Language Models. 60+ Test Types for Comparing LLM & NLP Models on Accuracy, Bias, Fairness, Robustness & More. | [Link](https://github.com/JohnSnowLabs/langtest) |
| EvalPlus | A rigorous evaluation framework for LLM4Code. | [Link](https://github.com/evalplus/evalplus) |
| FastChat | An open platform for training, serving, and evaluating large language model-based chatbots. | [Link](https://github.com/lm-sys/FastChat) |
| judges | A small library of LLM judges. | [Link](https://github.com/quotient-ai/judges) |
| Evals | Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks. | [Link](https://github.com/openai/evals) |
| AgentEvals | Evaluators and utilities for evaluating the performance of your agents. | [Link](https://github.com/langchain-ai/agentevals) |
| LLMBox | A comprehensive library for implementing LLMs, including a unified training pipeline and comprehensive model evaluation. | [Link](https://github.com/RUCAIBox/LLMBox) |
| Opik | An open-source end-to-end LLM Development Platform which also includes LLM evaluation. | [Link](https://github.com/comet-ml/opik) |
| PydanticAI Evals | A powerful evaluation framework designed to help you systematically evaluate the performance of LLM applications. | [Link](https://ai.pydantic.dev/evals/) |
| UQLM | A Python package for generation-time, zero-resource LLM hallucination using state-of-the-art uncertainty quantification techniques. | [Link](https://github.com/cvs-health/uqlm) |
## LLM Monitoring
| Library | Description | Link |
|----------------------|-------------------------------------------------------------------------------------------------|-------|
| MLflow | An open-source end-to-end MLOps/LLMOps Platform for tracking, evaluating, and monitoring LLM applications. | [Link](https://github.com/mlflow/mlflow) |
| Opik | An open-source end-to-end LLM Development Platform which also includes LLM monitoring. | [Link](https://github.com/comet-ml/opik) |
| LangSmith | Provides tools for logging, monitoring, and improving your LLM applications. | [Link](https://github.com/langchain-ai/langsmith-sdk) |
| Weights & Biases (W&B) | W&B provides features for tracking LLM performance. | [Link](https://github.com/wandb) |
| Helicone | Open source LLM-Observability Platform for Developers. One-line integration for monitoring, metrics, evals, agent tracing, prompt management, playground, etc. | [Link](https://github.com/Helicone/helicone) |
| Evidently | An open-source ML and LLM observability framework. | [Link](https://github.com/evidentlyai/evidently) |
| Phoenix | An open-source AI observability platform designed for experimentation, evaluation, and troubleshooting. | [Link](https://github.com/Arize-ai/phoenix) |
| Observers | A Lightweight Library for AI Observability. | [Link](https://github.com/cfahlgren1/observers) |
## LLM Prompts
| Library | Description | Link |
|---------------------|----------------------------------------------------------------------------------------------------------------|-------|
| PCToolkit | A Unified Plug-and-Play Prompt Compression Toolkit of Large Language Models. | [Link](https://github.com/3DAgentWorld/Toolkit-for-Prompt-Compression) |
| Selective Context | Selective Context compresses your prompt and context to allow LLMs (such as ChatGPT) to process 2x more content. | [Link](https://pypi.org/project/selective-context/) |
| LLMLingua | Library for compressing prompts to accelerate LLM inference. | [Link](https://github.com/microsoft/LLMLingua) |
| betterprompt | Test suite for LLM prompts before pushing them to production. | [Link](https://github.com/stjordanis/betterprompt) |
| Promptify | Solve NLP Problems with LLMs & easily generate different NLP Task prompts for popular generative models like GPT, PaLM, and more with Promptify. | [Link](https://github.com/promptslab/Promptify) |
| PromptSource | PromptSource is a toolkit for creating, sharing, and using natural language prompts. | [Link](https://pypi.org/project/promptsource/) |
| DSPy | DSPy is the open-source framework for programming—rather than prompting—language models. | [Link](https://github.com/stanfordnlp/dspy) |
| Py-priompt | Prompt design library. | [Link](https://github.com/zenbase-ai/py-priompt) |
| Promptimizer | Prompt optimization library. | [Link](https://github.com/hinthornw/promptimizer) |
## LLM Structured Outputs
| Library | Description | Link |
|------------|--------------------------------------------------------|------|
|Instructor | Python library for working with structured outputs from large language models (LLMs). Built on top of Pydantic, it provides a simple, transparent, and user-friendly API. | [Link](https://github.com/instructor-ai/instructor) |
| XGrammar | An open-source library for efficient, flexible, and portable structured generation. | [Link](https://github.com/mlc-ai/xgrammar) |
| Outlines | Robust (structured) text generation | [Link](https://github.com/dottxt-ai/outlines) |
| Guidance | Guidance is an efficient programming paradigm for steering language models. | [Link](https://github.com/guidance-ai/guidance) |
| LMQL | A language for constraint-guided and efficient LLM programming. | [Link](https://github.com/eth-sri/lmql) |
| Jsonformer | A Bulletproof Way to Generate Structured JSON from Language Models. | [Link](https://github.com/1rgs/jsonformer) |
## LLM Safety and Security
| Library | Description | Link |
|---------------|-----------------------------------------------------------|------|
| JailbreakEval | A collection of automated evaluators for assessing jailbreak attempts. | [Link](https://github.com/ThuCCSLab/JailbreakEval) |
| EasyJailbreak | An easy-to-use Python framework to generate adversarial jailbreak prompts. | [Link](https://github.com/EasyJailbreak/EasyJailbreak) |
| Guardrails | Adding guardrails to large language models. | [Link](https://github.com/guardrails-ai/guardrails) |
| LLM Guard | The Security Toolkit for LLM Interactions. | [Link](https://github.com/protectai/llm-guard) |
| AuditNLG | AuditNLG is an open-source library that can help reduce the risks associated with using generative AI systems for language. | [Link](https://github.com/salesforce/AuditNLG) |
| NeMo Guardrails | NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems. | [Link](https://github.com/NVIDIA/NeMo-Guardrails) |
| Garak | LLM vulnerability scanner | [Link](https://github.com/NVIDIA/garak) |
| DeepTeam | The LLM Red Teaming Framework | [Link](https://github.com/confident-ai/deepteam)|
## LLM Embedding Models
| Library | Description | Link |
|---------------------------|-----------------------------------------------------|------|
| Sentence-Transformers | State-of-the-Art Text Embeddings | [Link](https://github.com/UKPLab/sentence-transformers) |
| Model2Vec | Fast State-of-the-Art Static Embeddings | [Link](https://github.com/MinishLab/model2vec) |
| Text Embedding Inference | A blazing fast inference solution for text embeddings models. TEI enables high-performance extraction for the most popular models, including FlagEmbedding, Ember, GTE and E5. | [Link](https://github.com/huggingface/text-embeddings-inference) |
## Others
| Library | Description | Link |
|-------------------------|----------------------------------------------------------------------------------------------------------------------------------|------|
| Text Machina | A modular and extensible Python framework, designed to aid in the creation of high-quality, unbiased datasets to build robust models for MGT-related tasks such as detection, attribution, and boundary detection. | [Link](https://github.com/Genaios/TextMachina) |
| LLM Reasoners | A library for advanced large language model reasoning. | [Link](https://github.com/maitrix-org/llm-reasoners) |
| EasyEdit | An Easy-to-use Knowledge Editing Framework for Large Language Models. | [Link](https://github.com/zjunlp/EasyEdit) |
| CodeTF | CodeTF: One-stop Transformer Library for State-of-the-art Code LLM. | [Link](https://github.com/salesforce/CodeTF) |
| spacy-llm | This package integrates Large Language Models (LLMs) into spaCy, featuring a modular system for fast prototyping and prompting, and turning unstructured responses into robust outputs for various NLP tasks. | [Link](https://github.com/explosion/spacy-llm) |
| pandas-ai | Chat with your database (SQL, CSV, pandas, polars, MongoDB, NoSQL, etc.). | [Link](https://github.com/Sinaptik-AI/pandas-ai) |
| LLM Transparency Tool | An open-source interactive toolkit for analyzing internal workings of Transformer-based language models. | [Link](https://github.com/facebookresearch/llm-transparency-tool) |
| Vanna | Chat with your SQL database. Accurate Text-to-SQL Generation via LLMs using RAG. | [Link](https://github.com/vanna-ai/vanna) |
| mergekit | Tools for merging pretrained large language models. | [Link](https://github.com/arcee-ai/MergeKit) |
| MarkLLM | An Open-Source Toolkit for LLM Watermarking. | [Link](https://github.com/THU-BPM/MarkLLM) |
| LLMSanitize | An open-source library for contamination detection in NLP datasets and Large Language Models (LLMs). | [Link](https://github.com/ntunlp/LLMSanitize) |
| Annotateai | Automatically annotate papers using LLMs. | [Link](https://github.com/neuml/annotateai) |
| LLM Reasoner | Make any LLM think like OpenAI o1 and DeepSeek R1. | [Link](https://github.com/harishsg993010/LLM-Reasoner) |
## ⭐️ Star History
[](https://star-history.com/#)
Please consider giving a star, if you find this repository useful.
gitextract_5atf4lwj/ ├── Images/ │ └── ReadMe.md ├── LICENSE └── README.md
Condensed preview — 3 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (51K chars).
[
{
"path": "Images/ReadMe.md",
"chars": 1,
"preview": "\n"
},
{
"path": "LICENSE",
"chars": 11357,
"preview": " Apache License\n Version 2.0, January 2004\n "
},
{
"path": "README.md",
"chars": 38407,
"preview": "# 👨🏻💻 LLM Engineer Toolkit \nThis repository contains a curated list of 120+ LLM libraries category wise.\n<p align=\"cent"
}
]
About this extraction
This page contains the full source code of the KalyanKS-NLP/llm-engineer-toolkit GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 3 files (48.6 KB), approximately 10.3k tokens. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.