🎉🚀 Latest Release: v0.11.3 🚀🎉
Nodes Updated : Web search can be enabled on GPT node, Claude 4 available
UI : Node Search Bar, Shortcut for Popular Replicate Models
New Models available : Flux Kontext, Veo 3, Lyria 2, Imagen 4 available through the Replicate Node
---
## Overview
**AI-Flow** is an open-source, user-friendly UI that lets you visually design, manage, and monitor AI-driven workflows by seamlessly connecting multiple AI model APIs (e.g., OpenAI, StabilityAI, Replicate, Claude, Deepseek).
## Features
- **Visual Workflow Builder:** Drag-and-drop interface for crafting AI workflows.
- **Real-Time Monitoring:** Watch your workflow execute and track results.
- **Parallel Processing:** Nodes run in parallel whenever possible.
- **Model Management:** Easily organize and manage diverse AI models.
- **Import/Export:** Share or back up your workflows effortlessly.
## Supported Models
- **Replicate:** All models available through the Replicate API (FLUX.1, FLUX.1 Kontext, Imagen 4, Veo 3, Lyria 2, and many more)
- **OpenAI:** GPT-4o, GPT-4.1, TTS, o1, o3, o4.
- **StabilityAI:** Stable Diffusion 3.5, SDXL, Stable Video Diffusion, plus additional tools.
- **Others:** Claude, Deepseek, OpenRouter.
## Open Source vs. Cloud
**AI-Flow** is fully open source and available under the MIT License, empowering you to build and run your AI workflows on your personal machine.
For those seeking enhanced functionality and a polished experience, **AI-Flow Pro** on our cloud platform ([app.ai-flow.net](https://ai-flow.net/?ref=github)) offers advanced features, including:
- **Subflows & Loops:** Create complex, nested workflows and iterate tasks effortlessly.
- **API-Triggered Flows:** Initiate workflows via API calls for seamless automation.
- **Integrated Services:** Connect with external services such as Google Search, Airtable, Zapier, and Make.
- **Simplified Interface:** Transform workflows into streamlined tools with an intuitive UI.
The cloud version builds upon the foundation of the open-source project, giving you more power and flexibility while still letting you use your own API keys.
## Installation
> **Note:** To unlock full functionality, AI-Flow requires S3-compatible storage (with proper CORS settings) to host resources. Without it, features like File Upload or nodes that rely on external providers (e.g., StabilityAI) may not work as expected. Also, set `REPLICATE_API_KEY` in the App Parameters or in your environment to use the Replicate node.
### Method 1: Using the Executable (Windows Only)
> **Note:** This method is only available for Windows users.
1. Download the latest Windows version of AI-Flow from the official releases page: [AI-Flow Releases](https://ai-flow.net/release/)
2. Once downloaded, run the `.exe` file.
This will start a local server and open AI-Flow in a standalone window, giving you direct access to its user interface without needing to install anything else.
### Method 2 : Docker Installation
1. **Prepare Docker Compose:**
- Navigate to the `docker` directory:
```bash
cd docker
```
2. **Launch with Docker Compose:**
```bash
docker-compose up -d
```
3. **Access the Application:**
- Open [http://localhost:80](http://localhost:80) in your browser.
- To stop, run:
```bash
docker-compose stop
```
### Method 3 : Local Installation
1. **Clone the Repository:**
```bash
git clone https://github.com/DahnM20/ai-flow.git
cd ai-flow
```
2. **UI Setup:**
```bash
cd packages/ui
npm install
```
3. **Backend Setup:**
```bash
cd ../backend
poetry install
```
- **Windows Users:**
```bash
poetry shell
pip install -r requirements_windows.txt
```
4. **Run the Application:**
- Start the backend:
```bash
poetry run python server.py
```
- In a new terminal, start the UI:
```bash
cd packages/ui
npm start
```
- Open your browser and navigate to [http://localhost:3000](http://localhost:3000).
## Contributing
We welcome contributions! If you encounter issues or have feature ideas, please [open an issue](https://github.com/DahnM20/ai-flow/issues) or submit a pull request.
## License
This project is released under the [MIT License](LICENSE).
================================================
FILE: bin/generate_python_classes_from_ts.sh
================================================
npm i -g typescript-json-schema
typescript-json-schema "../packages/ui/src/nodes-configuration/types.ts" "*" --out "schema.json"
mv schema.json ../packages/backend/app/processors/components/
cd ../packages/backend/app/processors/components/
poetry run datamodel-codegen --input schema.json --input-file-type jsonschema --output model.py --output-model-type pydantic_v2.BaseModel --enum-field-as-literal all
rm schema.json
echo "model.py generated"
================================================
FILE: docker/README.md
================================================
## 🐳 Docker
### Docker Compose
1. Go to the docker directory: `cd ./docker`
2. Update the .yml if needed for the PORTS
3. Launch `docker-compose up` or `docker-compose up -d`
4. Open your browser and navigate to `http://localhost:3000`
5. Use `docker-compose stop` when you want to stop the app.
================================================
FILE: docker/docker-compose.it.yml
================================================
services:
backend:
container_name: ai-flow-backend
build:
context: ../packages/backend/
dockerfile: Dockerfile
ports:
- 5000:5000
environment:
- HOST=0.0.0.0
- PORT=5000
- DEPLOYMENT_ENV=LOCAL
- LOCAL_STORAGE_FOLDER_NAME=local_storage
- USE_MOCK=true
volumes:
- ./ai-flow-backend-storage:/app/local_storage
frontend:
container_name: ai-flow-frontend
build:
context: ../packages/ui/
dockerfile: Dockerfile
ports:
- 80:80
environment:
- VITE_APP_WS_HOST=localhost
- VITE_APP_WS_PORT=5000
================================================
FILE: docker/docker-compose.yml
================================================
services:
backend:
container_name: ai-flow-backend
build:
context: ../packages/backend/
dockerfile: Dockerfile
ports:
- 5001:5000
environment:
- HOST=0.0.0.0
- PORT=5000
- DEPLOYMENT_ENV=LOCAL
- REPLICATE_API_KEY=sample
- LOCAL_STORAGE_FOLDER_NAME=local_storage
volumes:
- ./ai-flow-backend-storage:/app/local_storage
frontend:
container_name: ai-flow-frontend
build:
context: ../packages/ui/
dockerfile: Dockerfile
args:
VITE_APP_WS_HOST: localhost
VITE_APP_WS_PORT: 5001
VITE_APP_API_REST_PORT: 5001
ports:
- 80:80
================================================
FILE: docker/healthcheck.sh
================================================
#!/bin/bash
if [ "$#" -ne 1 ]; then
echo "Usage: $0 "
exit 1
fi
URL="$1"
INTERVAL=5
MAX_ATTEMPTS=20
attempt=0
while [ $attempt -lt $MAX_ATTEMPTS ]; do
attempt=$(( $attempt + 1 ))
curl --fail --silent $URL && echo "Service is up!" && exit 0
echo "Service not ready yet. Waiting for $INTERVAL seconds. Attempt $attempt of $MAX_ATTEMPTS."
sleep $INTERVAL
done
echo "Service did not become ready after $MAX_ATTEMPTS attempts."
exit 1
================================================
FILE: integration_tests/.gitignore
================================================
# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
# dependencies
/node_modules
/.pnp
.pnp.js
# testing
/coverage
# production
/dist
# misc
.DS_Store
.env.local
.env.development.local
.env.test.local
.env.production.local
npm-debug.log*
yarn-debug.log*
yarn-error.log*
================================================
FILE: integration_tests/package.json
================================================
{
"name": "integration_tests",
"version": "1.0.0",
"description": "",
"main": "dist/index.js",
"scripts": {
"test": "mocha dist/tests/**/*Test.js",
"build": "tsc",
"pretest": "npm run build"
},
"author": "",
"license": "ISC",
"dependencies": {
"axios": "^1.5.0",
"chai": "^4.3.8",
"mocha": "^10.2.0",
"socket.io-client": "^4.7.2"
},
"devDependencies": {
"@types/chai": "^4.3.6",
"@types/minimist": "^1.2.2",
"@types/mocha": "^10.0.1",
"@types/node": "^20.5.9",
"@types/normalize-package-data": "^2.4.1",
"@types/socket.io-client": "^3.0.0",
"typescript": "^5.2.2"
}
}
================================================
FILE: integration_tests/tests/nodeProcessingOrder/nodeErrorTest.ts
================================================
import { expect } from "chai";
import {
disconnectSocket,
getSocket,
setupSocket,
} from "../../utils/testHooks";
import {
createRequestData,
flowWithFourParallelNodeStep,
flowWithoutLinks,
sequentialFlow,
} from "../../utils/requestDatas";
describe("Node errors test", function () {
this.timeout(30000);
beforeEach(function (done) {
setupSocket(done);
});
afterEach(function () {
disconnectSocket();
});
it("Error in sequential flow should stop the flow", function (done) {
const socket = getSocket();
const flowNodesWithOneError = structuredClone(sequentialFlow);
flowNodesWithOneError[1] = {
...flowNodesWithOneError[1],
raiseError: true,
};
socket.emit("process_file", createRequestData(flowNodesWithOneError));
let errorReceived = false;
let progressCount = 0;
const progressBeforeError = 1;
socket.on("error", (error) => {
errorReceived = true;
expect(progressCount).to.equal(progressBeforeError);
});
socket.on("progress", (progress) => {
if (errorReceived) {
done(new Error("Received progress after error"));
}
progressCount++;
if (progressCount > progressBeforeError) {
done(new Error(`Too many nodes sent progress`));
}
});
setTimeout(() => {
if (!errorReceived) {
socket.disconnect();
done(new Error("No error received within the expected time"));
} else {
done();
}
}, 10000);
});
it("Error in flow without link should run the others nodes", function (done) {
const socket = getSocket();
const flow = structuredClone(sequentialFlow);
flow[1] = {
...flow[1],
raiseError: true,
};
socket.emit("process_file", createRequestData(flow));
let errorReceived = false;
let progressCount = 0;
const maxProgressBeforeError = 2;
socket.on("error", (error) => {
errorReceived = true;
expect(progressCount).to.be.at.most(maxProgressBeforeError);
});
socket.on("progress", (progress) => {
progressCount++;
if (progressCount > maxProgressBeforeError) {
done(new Error(`Too many nodes sent progress`));
}
});
setTimeout(() => {
if (errorReceived && progressCount === 1) {
socket.disconnect();
done();
} else {
done(new Error("No error received within the expected time"));
}
}, 2000);
});
it("Error in flow with 4 parallel node should return result for all of them except the one in error", function (done) {
const socket = getSocket();
const flow = structuredClone(flowWithFourParallelNodeStep);
flow[4] = {
...flow[4],
raiseError: true,
};
flow[1] = {
...flow[1],
sleepDuration: 2,
}; //One node with high delay to be sure he is processed during the error raise
socket.emit("process_file", createRequestData(flow));
let errorReceived = false;
let progressCount = 0;
const maxProgressBeforeError = 4;
socket.on("error", (error) => {
errorReceived = true;
expect(progressCount).to.be.at.most(maxProgressBeforeError);
});
socket.on("progress", (progress) => {
progressCount++;
if (progressCount > maxProgressBeforeError) {
done(new Error(`Too many nodes sent progress`));
}
});
socket.on("run_end", (end) => {
if (progressCount !== maxProgressBeforeError) {
done(
new Error(`Not all nodes were processed before end of execution.`)
);
} else {
done();
}
});
});
});
================================================
FILE: integration_tests/tests/nodeProcessingOrder/nodeParallelExecutionDurationTest.ts
================================================
import { expect } from "chai";
import {
disconnectSocket,
getSocket,
setupSocket,
} from "../../utils/testHooks";
import {
createRequestData,
flowWithFourParallelNodeStep,
} from "../../utils/requestDatas";
describe("Node errors test", function () {
this.timeout(15000);
beforeEach(function (done) {
setupSocket(done);
});
afterEach(function () {
disconnectSocket();
});
it("4 parallel node with 2s sleep each should not compound time", function (done) {
const socket = getSocket();
const flow = structuredClone(flowWithFourParallelNodeStep);
flow[1] = {
...flow[1],
sleepDuration: 2,
};
flow[2] = {
...flow[2],
sleepDuration: 2,
};
flow[3] = {
...flow[3],
sleepDuration: 2,
};
flow[4] = {
...flow[4],
sleepDuration: 2,
};
const maxDurationMsExpected = 5000;
const timeStart = Date.now();
socket.emit("process_file", createRequestData(flow));
let progressCount = 0;
const maxProgress = 5;
socket.on("progress", (progress) => {
progressCount++;
if (progressCount > maxProgress) {
done(new Error(`Too many nodes sent progress`));
}
});
socket.on("run_end", (end) => {
if (progressCount !== maxProgress) {
done(
new Error(`Not all nodes were processed before end of execution.`)
);
} else {
const timeEnd = Date.now();
const duration = timeEnd - timeStart;
expect(duration).to.be.lessThan(maxDurationMsExpected);
done();
}
});
});
});
================================================
FILE: integration_tests/tests/nodeProcessingOrder/nodeWithChildrenTest.ts
================================================
import { expect } from "chai";
import { disconnectSocket, getSocket, setupSocket } from "../../utils/testHooks";
import { createRequestData } from "../../utils/requestDatas";
describe('node with children test', function () {
this.timeout(15000);
beforeEach(function (done) {
setupSocket(done);
});
afterEach(function () {
disconnectSocket();
});
const flowNodeWithChildren = [
{
inputs: [],
name: "1#llm-prompt",
processorType: "llm-prompt",
},
{
inputs: [
{
"inputNode": "1#llm-prompt",
"inputNodeOutputKey": 0
}
],
name: "2#llm-prompt",
processorType: "llm-prompt",
},
{
inputs: [
{
"inputNode": "1#llm-prompt",
"inputNodeOutputKey": 0
}
],
name: "3#stable-diffusion-stabilityai-prompt",
processorType: "stable-diffusion-stabilityai-prompt",
}
];
it('process_file should process the parent first, then its children', function (done) {
const socket = getSocket();
socket.emit('process_file', createRequestData(flowNodeWithChildren));
let processedNodes: string[] = [];
socket.on('progress', (data) => {
processedNodes.push(data.instanceName);
if (processedNodes.length === flowNodeWithChildren.length) {
try {
//First one needs to be the parent
expect(processedNodes[0]).to.equal(flowNodeWithChildren[0].name);
expect(processedNodes).to.includes(flowNodeWithChildren[1].name);
expect(processedNodes).to.includes(flowNodeWithChildren[2].name);
done();
} catch (error) {
done(error);
}
}
});
socket.on('error', (error) => {
done(new Error(`Error event received: ${JSON.stringify(error)}`));
});
});
});
================================================
FILE: integration_tests/tests/nodeProcessingOrder/nodeWithMultipleParentsTest.ts
================================================
import { expect } from "chai";
import { disconnectSocket, getSocket, setupSocket } from "../../utils/testHooks";
import { createRequestData } from "../../utils/requestDatas";
describe('node with multiple parent test', function () {
this.timeout(15000);
beforeEach(function (done) {
setupSocket(done);
});
afterEach(function () {
disconnectSocket();
});
const flowWithNodesWithMultipleParents = [
{
inputs: [],
name: "1#llm-prompt",
processorType: "llm-prompt",
},
{
inputs: [],
name: "2#llm-prompt",
processorType: "llm-prompt",
},
{
inputs: [
{
"inputNode": "1#llm-prompt",
"inputNodeOutputKey": 0
},
{
"inputNode": "2#llm-prompt",
"inputNodeOutputKey": 0
}
],
name: "3#stable-diffusion-stabilityai-prompt",
processorType: "stable-diffusion-stabilityai-prompt",
}
];
it('process_file should process both parents before the child', function (done) {
const socket = getSocket();
socket.emit('process_file', createRequestData(flowWithNodesWithMultipleParents));
let processedNodes: string[] = [];
socket.on('progress', (data) => {
processedNodes.push(data.instanceName);
if (processedNodes.length === flowWithNodesWithMultipleParents.length) {
try {
// Check if both parents are processed before the child
expect(processedNodes.includes("1#llm-prompt")).to.be.true;
expect(processedNodes.includes("2#llm-prompt")).to.be.true;
expect(processedNodes.indexOf("3#stable-diffusion-stabilityai-prompt")).to.be.greaterThan(processedNodes.indexOf("1#llm-prompt"));
expect(processedNodes.indexOf("3#stable-diffusion-stabilityai-prompt")).to.be.greaterThan(processedNodes.indexOf("2#llm-prompt"));
done();
} catch (error) {
done(error);
}
}
});
socket.on('error', (error) => {
done(new Error(`Error event received: ${JSON.stringify(error)}`));
});
});
});
================================================
FILE: integration_tests/tests/nodeProcessingOrder/nodesWithoutLinkTest.ts
================================================
import { expect } from "chai";
import { disconnectSocket, getSocket, setupSocket } from "../../utils/testHooks";
import { createRequestData } from "../../utils/requestDatas";
describe('node without link test', function () {
this.timeout(15000);
beforeEach(function (done) {
setupSocket(done);
});
afterEach(function () {
disconnectSocket();
});
const flowWithNodesWithoutLink = [
{
inputs: [],
name: "1#llm-prompt",
processorType: "llm-prompt",
},
{
inputs: [],
name: "2#llm-prompt",
processorType: "llm-prompt",
},
{
inputs: [],
name: "3#stable-diffusion-stabilityai-prompt",
processorType: "stable-diffusion-stabilityai-prompt",
}
];
it('process_file should process all the nodes', function (done) {
const socket = getSocket();
socket.emit('process_file', createRequestData(flowWithNodesWithoutLink));
let processedNodes: string[] = [];
socket.on('progress', (data) => {
processedNodes.push(data.instanceName);
if (processedNodes.length === flowWithNodesWithoutLink.length) {
try {
expect(processedNodes).to.includes(flowWithNodesWithoutLink[0].name);
expect(processedNodes).to.includes(flowWithNodesWithoutLink[1].name);
expect(processedNodes).to.includes(flowWithNodesWithoutLink[2].name);
done();
} catch (error) {
done(error);
}
}
});
socket.on('error', (error) => {
done(new Error(`Error event received: ${JSON.stringify(error)}`));
});
});
});
================================================
FILE: integration_tests/tests/nodeProcessingOrder/singleNodeTest.ts
================================================
import { expect } from "chai";
import { Socket, io } from "socket.io-client";
import { createRequestData } from "../../utils/requestDatas";
describe('single node test', function () {
this.timeout(5000);
let socket: Socket;
beforeEach(function (done) {
socket = io('http://localhost:5000');
socket.on('connect', function () {
done();
});
socket.on('connect_error', function (error) {
done(error);
});
});
afterEach(function () {
socket.disconnect();
});
const flowWithSingleNode = [
{
inputs: [],
name: "1#llm-prompt",
processorType: "llm-prompt",
}
];
it('process_file should trigger one progress event', function (done) {
socket.emit('process_file', createRequestData(flowWithSingleNode));
socket.once('progress', (data) => {
expect(data).to.have.property('instanceName').to.equal(flowWithSingleNode[0].name);
done();
});
socket.once('error', (error) => {
done(new Error(`Error event received: ${JSON.stringify(error)}`));
});
});
});
================================================
FILE: integration_tests/tests/socketEvents/processFileEventTest.ts
================================================
import { io, Socket } from "socket.io-client";
import { expect } from 'chai';
import { basicJsonFlow, getBasicProcessFileData, getJsonFlowWithMissingInputTextProcessFileData } from '../../utils/requestDatas';
describe('process_file event tests', function () {
this.timeout(5000);
let socket: Socket;
beforeEach(function (done) {
socket = io('http://localhost:5000');
socket.on('connect', function () {
done();
});
socket.on('connect_error', function (error) {
done(error);
});
});
afterEach(function () {
socket.disconnect();
});
it('process_file should trigger run_end event', function (done) {
const processFileData = getBasicProcessFileData();
socket.emit('process_file', processFileData);
socket.once('run_end', (data) => {
expect(data).to.have.property('output');
done();
});
socket.once('error', (error) => {
done(new Error(`Error event received: ${JSON.stringify(error)}`));
});
});
it('process_file should trigger progress event', function (done) {
const processFileData = getBasicProcessFileData();
socket.emit('process_file', processFileData);
socket.once('progress', (data) => {
expect(data).to.have.property('output').to.equal(basicJsonFlow[0].inputText);
expect(data).to.have.property('instanceName').to.equal(basicJsonFlow[0].name);
done();
});
socket.once('error', (error) => {
done(new Error(`Error event received: ${JSON.stringify(error)}`));
});
});
it('process_file should trigger current_node_running event', function (done) {
const processFileData = getBasicProcessFileData();
socket.emit('process_file', processFileData);
socket.once('current_node_running', (data) => {
expect(data).to.have.property('instanceName').to.equal(basicJsonFlow[0].name);
done();
});
socket.once('error', (error) => {
done(new Error(`Error event received: ${JSON.stringify(error)}`));
});
});
it('process_file with missing input should trigger error event', function (done) {
const processFileData = getJsonFlowWithMissingInputTextProcessFileData();
socket.emit('process_file', processFileData);
socket.once('error', (error) => {
done();
});
});
});
================================================
FILE: integration_tests/tests/socketEvents/runNodeEventTest.ts
================================================
import { io, Socket } from "socket.io-client";
import { expect } from 'chai';
import { basicJsonFlow, getBasicRunNodeData, getJsonFlowWithMissingInputTextProcessFileData } from '../../utils/requestDatas';
describe('run_node event tests', function () {
this.timeout(5000);
let socket: Socket;
beforeEach(function (done) {
socket = io('http://localhost:5000');
socket.on('connect', function () {
done();
});
socket.on('connect_error', function (error) {
done(error);
});
});
afterEach(function () {
socket.disconnect();
});
it('run_node should trigger progress event', function (done) {
const runNodeData = getBasicRunNodeData();
socket.emit('run_node', runNodeData);
socket.once('progress', (data) => {
expect(data).to.have.property('output').to.equal(basicJsonFlow[0].inputText);
expect(data).to.have.property('instanceName').to.equal(basicJsonFlow[0].name);
done();
});
socket.once('error', (error) => {
done(new Error(`Error event received: ${JSON.stringify(error)}`));
});
});
it('run_node should trigger current_node_running event', function (done) {
const runNodeData = getBasicRunNodeData();
socket.emit('run_node', runNodeData);
socket.once('current_node_running', (data) => {
expect(data).to.have.property('instanceName').to.equal(basicJsonFlow[0].name);
done();
});
socket.once('error', (error) => {
done(new Error(`Error event received: ${JSON.stringify(error)}`));
});
});
it('run_node with missing input should trigger error event', function (done) {
const processFileData = getJsonFlowWithMissingInputTextProcessFileData();
socket.emit('run_node', processFileData);
socket.once('error', (error) => {
done();
});
});
});
================================================
FILE: integration_tests/tests/socketEvents/socketConnectionTest.ts
================================================
import { io, Socket } from "socket.io-client";
import { expect } from 'chai';
describe('Socket.IO connection tests', function () {
let socket: Socket;
beforeEach(function (done: Mocha.Done): void {
socket = io('http://localhost:5000');
socket.on('connect', function (): void {
done();
});
socket.on('connect_error', function (error: any): void {
done(error);
});
});
afterEach(function (): void {
socket.disconnect();
});
it('should be connected to the server', function (done: Mocha.Done): void {
expect(socket.connected).to.be.true;
done();
});
it('should disconnect', function (done: Mocha.Done): void {
socket.disconnect();
expect(socket.connected).to.be.false;
done();
});
});
================================================
FILE: integration_tests/tsconfig.json
================================================
{
"compilerOptions": {
"target": "ES6",
"module": "commonjs",
"outDir": "./dist",
"rootDir": "./",
"strict": true
}
}
================================================
FILE: integration_tests/utils/requestDatas.ts
================================================
type ProcessFileData = {
jsonFile: string;
parameters: Record;
};
type RunNodeData = {
jsonFile: string;
parameters: Record;
nodeName: string;
};
export type Node = {
inputs: {
inputName?: string;
inputNode: string;
inputNodeOutputKey: number;
}[];
name: string;
processorType: string;
[key: string]: any;
};
const basicJsonFlow: Node[] = [
{
inputs: [],
name: "kbk1proh1#input-text",
processorType: "input-text",
inputText: "Hello World",
x: 1,
y: 1,
},
];
const jsonFlowWithMissingInputText: Node[] = [
{
inputs: [],
name: "kbk1proh1#input-text",
processorType: "input-text",
x: 1,
y: 1,
},
];
export const flowWithOneNonFreeNode: Node[] = [
{
inputs: [],
name: "1#stable-diffusion-stabilityai-prompt",
processorType: "stable-diffusion-stabilityai-prompt",
},
];
export const sequentialFlow: Node[] = [
{
inputs: [],
name: "1#llm-prompt",
processorType: "llm-prompt",
raiseError: false,
},
{
inputs: [
{
inputNode: "1#llm-prompt",
inputNodeOutputKey: 0,
},
],
name: "2#llm-prompt",
processorType: "llm-prompt",
raiseError: false,
},
{
inputs: [
{
inputNode: "2#llm-prompt",
inputNodeOutputKey: 0,
},
],
name: "3#stable-diffusion-stabilityai-prompt",
processorType: "stable-diffusion-stabilityai-prompt",
raiseError: false,
},
];
export const flowWithoutLinks: Node[] = [
{
inputs: [],
name: "1#llm-prompt",
processorType: "llm-prompt",
model: "gpt-4",
prompt: "hi",
raiseError: false,
},
{
inputs: [],
name: "2#llm-prompt",
processorType: "llm-prompt",
model: "gpt-4",
prompt: "hi",
raiseError: false,
},
{
inputs: [],
name: "3#stable-diffusion-stabilityai-prompt",
processorType: "stable-diffusion-stabilityai-prompt",
raiseError: false,
},
];
export const flowFreeNodesWithoutLink: Node[] = [
{
inputs: [],
name: "1#input-text",
processorType: "input-text",
inputText: "fake",
},
{
inputs: [],
name: "2#input-text",
processorType: "input-text",
inputText: "fake",
},
{
inputs: [],
name: "3#input-text",
processorType: "input-text",
inputText: "fake",
},
];
export const flowWithFourParallelNodeStep: Node[] = [
{
inputs: [],
name: "1#llm-prompt",
processorType: "llm-prompt",
raiseError: false,
sleepDuration: undefined,
},
{
inputs: [
{
inputNode: "1#llm-prompt",
inputNodeOutputKey: 0,
},
],
name: "2#llm-prompt",
processorType: "llm-prompt",
raiseError: false,
sleepDuration: undefined,
},
{
inputs: [
{
inputNode: "1#llm-prompt",
inputNodeOutputKey: 0,
},
],
name: "3#llm-prompt",
processorType: "llm-prompt",
raiseError: false,
sleepDuration: undefined,
},
{
inputs: [
{
inputNode: "1#llm-prompt",
inputNodeOutputKey: 0,
},
],
name: "4#llm-prompt",
processorType: "llm-prompt",
raiseError: false,
sleepDuration: undefined,
},
{
inputs: [
{
inputNode: "1#llm-prompt",
inputNodeOutputKey: 0,
},
],
name: "5#llm-prompt",
processorType: "llm-prompt",
raiseError: false,
sleepDuration: undefined,
},
];
function getBasicProcessFileData(): ProcessFileData {
return {
jsonFile: JSON.stringify(basicJsonFlow),
parameters: {
openaiApiKey: "apiKey",
},
};
}
function getBasicRunNodeData(): RunNodeData {
return {
jsonFile: JSON.stringify(basicJsonFlow),
nodeName: basicJsonFlow[0].name,
parameters: {
openaiApiKey: "apiKey",
},
};
}
function getJsonFlowWithMissingInputTextProcessFileData(): ProcessFileData {
return {
jsonFile: JSON.stringify(jsonFlowWithMissingInputText),
parameters: {
openaiApiKey: "apiKey",
},
};
}
function createRequestData(flow: any): ProcessFileData {
return {
jsonFile: JSON.stringify(flow),
parameters: {
openaiApiKey: "apiKey",
},
};
}
export {
basicJsonFlow,
jsonFlowWithMissingInputText,
getBasicProcessFileData,
getBasicRunNodeData,
getJsonFlowWithMissingInputTextProcessFileData,
createRequestData,
};
================================================
FILE: integration_tests/utils/testHooks.ts
================================================
import { Socket, io } from "socket.io-client";
let socket: Socket;
export const setupSocket = (done: any) => {
socket = io('http://localhost:5000');
socket.on('connect', function () {
done();
});
socket.on('connect_error', function (error) {
done(error);
});
};
export const disconnectSocket = () => {
socket.disconnect();
};
export const getSocket = () => socket;
================================================
FILE: packages/backend/.gitignore
================================================
# Fichiers générés par l'environnement de développement
__pycache__/
*.py[cod]
# Fichiers générés par l'IDE
.idea/
.vscode/
# Fichiers de logs
*.log
# Fichiers d'env
*.env
# Fichiers de build
build
dist
server.spec
# Local storage
local_storage/
================================================
FILE: packages/backend/Dockerfile
================================================
FROM python:3.9
# Default values
ENV HOST=0.0.0.0
ENV PORT=5000
WORKDIR /app
# System dependencies
RUN apt-get update && apt-get install -y \
build-essential \
libpq-dev \
python3-dev \
libssl-dev \
libffi-dev \
libmagic-dev \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# Playwright
ARG PLAYWRIGHT_VERSION=1.39
ENV PLAYWRIGHT_BROWSERS_PATH=/ms-playwright
RUN pip install playwright==$PLAYWRIGHT_VERSION && \
playwright install chromium && \
playwright install-deps chromium
# Poetry & Dependencies
RUN pip install --upgrade poetry \
&& poetry config virtualenvs.create false
COPY poetry.lock pyproject.toml /app/
RUN poetry install --no-interaction --no-root
# The rest of the app
COPY app /app/app/
COPY resources /app/resources
COPY tests/ /app/tests/
COPY server.py README.md /app/
COPY config.yaml /app/
EXPOSE 5000
CMD ["poetry", "run", "python", "server.py"]
================================================
FILE: packages/backend/README.md
================================================
================================================
FILE: packages/backend/app/env_config.py
================================================
import os
import sys
from typing import List, Optional
ENV_LOCAL = "LOCAL"
ENV_CLOUD = "CLOUD"
CURRENT_ENV = os.environ.get("DEPLOYMENT_ENV", ENV_LOCAL)
CURRENT_DIR = os.path.dirname(os.path.abspath(__file__))
BACKEND_DIR = os.path.dirname(CURRENT_DIR)
LOCAL_STORAGE_DIR = os.path.join(
BACKEND_DIR, os.getenv("LOCAL_STORAGE_FOLDER_NAME", "local_storage")
)
def get_static_folder() -> str:
if getattr(sys, "frozen", False):
base_path = sys._MEIPASS
build_dir = os.path.join(base_path, "build")
else:
base_path = os.path.dirname(os.path.abspath(__file__))
build_dir = os.path.join(base_path, "..", "..", "ui", "build")
return build_dir
def is_cloud_env() -> bool:
return CURRENT_ENV == ENV_CLOUD
def is_local_environment() -> bool:
return CURRENT_ENV == ENV_LOCAL
def is_mock_env() -> bool:
return os.getenv("USE_MOCK") == "true"
def is_server_static_files_enabled() -> bool:
return os.getenv("SERVE_STATIC_FILES") == "true"
def get_local_storage_folder_path() -> str:
return LOCAL_STORAGE_DIR
def get_flask_secret_key() -> Optional[str]:
return os.getenv("FLASK_SECRET_KEY")
def get_replicate_api_key() -> Optional[str]:
return os.getenv("REPLICATE_API_KEY")
def get_background_task_max_workers() -> int:
return int(os.getenv("BACKGROUND_TASK_MAX_WORKERS", "2"))
def use_async_browser() -> bool:
return os.getenv("USE_ASYNC_BROWSER") == "true"
def get_browser_tab_max_usage() -> int:
return int(os.getenv("BROWSER_TAB_MAX_USAGE", "100"))
def get_browser_tab_pool_size() -> int:
return int(os.getenv("BROWSER_TAB_POOL_SIZE", "3"))
def is_set_app_config_on_ui_enabled() -> bool:
return os.getenv("ENABLE_SET_APP_CONFIG_ON_UI", "true") == "true"
def is_s3_enabled() -> bool:
return os.getenv("S3_AWS_ACCESS_KEY_ID") is not None
================================================
FILE: packages/backend/app/flask/app_routes/__init__.py
================================================
================================================
FILE: packages/backend/app/flask/app_routes/image_routes.py
================================================
from app.env_config import (get_local_storage_folder_path)
from flask import Blueprint, send_from_directory
image_blueprint = Blueprint('image_blueprint', __name__)
@image_blueprint.route("/image/")
def serve_image(filename):
"""
Serve image from local storage.
"""
return send_from_directory(get_local_storage_folder_path(), filename)
================================================
FILE: packages/backend/app/flask/app_routes/node_routes.py
================================================
import json
from flask import Blueprint, request
from ...utils.node_extension_utils import get_dynamic_extension_config, get_extensions
# from ...utils.openapi_reader import OpenAPIReader
from ...utils.replicate_utils import (
get_highlighted_models_info,
get_model_openapi_schema,
get_replicate_collection_models,
get_replicate_collections,
get_replicate_models,
)
node_blueprint = Blueprint("node_blueprint", __name__)
@node_blueprint.route("/node/extensions")
def get_node_extensions():
extensions = get_extensions()
return {"extensions": extensions}
@node_blueprint.route("/node/extensions/dynamic", methods=["POST"])
def get_dynamic_extension():
request_body = request.json
if request_body is None:
raise Exception("Missing data")
processor_type = request_body.get("processorType")
data = request_body.get("data")
config = get_dynamic_extension_config(processor_type, data)
return config.dict()
@node_blueprint.route("/node/models")
def get_public_models():
cursor = request.args.get("cursor", None)
public = get_replicate_models(cursor=cursor)
highlighted = get_highlighted_models_info()
return {"public": public, "highlighted": highlighted}
@node_blueprint.route("/node/collections")
def get_collections():
return get_replicate_collections()
@node_blueprint.route("/node/collections/")
def get_collection_models(collection):
cursor = request.args.get("cursor", None)
return get_replicate_collection_models(collection, cursor=cursor)
@node_blueprint.route("/node/replicate/config/")
def get_config(model):
return get_model_openapi_schema(model)
# @node_blueprint.route("/node/openapi//models")
# def get_openapi_models(api_name):
# api_reader = OpenAPIReader(f"./resources/openapi/{api_name}.json")
# return api_reader.get_all_paths()
# @node_blueprint.route("/node/openapi//config/")
# def get_openapi_model_config(api_name, id):
# api_reader = OpenAPIReader(f"./resources/openapi/{api_name}.json")
# return api_reader.get_request_schema(id)
================================================
FILE: packages/backend/app/flask/app_routes/parameters_routes.py
================================================
import os
import yaml
from flask import Blueprint
parameters_blueprint = Blueprint("parameters_blueprint", __name__)
def load_config():
with open("config.yaml", "r") as file:
return yaml.safe_load(file)
@parameters_blueprint.route("/parameters", methods=["GET"])
def parameters():
config = load_config()
return config
================================================
FILE: packages/backend/app/flask/app_routes/static_routes.py
================================================
import os
from flask import Blueprint, send_from_directory
from ...env_config import get_static_folder
static_blueprint = Blueprint('static_blueprint', __name__)
@static_blueprint.route("/", defaults={"path": ""})
@static_blueprint.route("/")
def serve(path):
"""
Serve UI static files from the static folder.
"""
static_folder = get_static_folder()
if path != "" and os.path.exists(os.path.join(static_folder, path)):
return send_from_directory(static_folder, path)
else:
return send_from_directory(static_folder, "index.html")
================================================
FILE: packages/backend/app/flask/app_routes/upload_routes.py
================================================
import logging
from flask import Blueprint
from ...storage.storage_strategy import StorageStrategy
from ...root_injector import get_root_injector
from flask import request
upload_blueprint = Blueprint("upload_blueprint", __name__)
@upload_blueprint.route("/upload")
def upload_file():
"""
Serve image from local storage.
"""
logging.info("Uploading file")
storage_strategy = get_root_injector().get(StorageStrategy)
filename = request.args.get("filename")
try:
data = storage_strategy.get_upload_link(filename)
except Exception as e:
logging.error(e)
raise Exception(
"Error uploading file. "
"Please check your S3 configuration. "
"If you've not configured S3 please refer to docs.ai-flow.net/docs/file-upload"
)
json_link = {
"upload_data": data[0],
"download_link": data[1],
}
return json_link
================================================
FILE: packages/backend/app/flask/decorators.py
================================================
from functools import wraps
from flask import jsonify, request, g
from flask_socketio import emit
import json
def with_flow_data_validations(*validation_funcs):
def decorator(func):
@wraps(func)
def wrapper(data, *args, **kwargs):
try:
flow_data = json.loads(data.get("jsonFile", "{}"))
for validation_func in validation_funcs:
validation_func(flow_data)
return func(data, *args, **kwargs)
except Exception as e:
emit("error", {"error": str(e)})
return wrapper
return decorator
================================================
FILE: packages/backend/app/flask/flask_app.py
================================================
import logging
from flask import Flask, request, redirect
from flask_cors import CORS
import os
from ..env_config import get_flask_secret_key, get_static_folder
def create_app():
app = Flask(__name__, static_folder=get_static_folder())
if get_flask_secret_key() is not None :
logging.info("Flask secret key set")
app.config['SECRET_KEY'] = get_flask_secret_key()
else :
logging.warning("Flask secret key not set")
app.config['SECRET_KEY'] = "default_secret"
CORS(app)
if os.getenv("USE_HTTPS", "false").lower() == "true":
@app.before_request
def before_request():
if not request.is_secure:
url = request.url.replace("http://", "https://", 1)
return redirect(url, code=301)
logging.info("App created")
return app
================================================
FILE: packages/backend/app/flask/routes.py
================================================
import logging
from app.env_config import is_server_static_files_enabled, is_local_environment
from app.flask.socketio_init import flask_app
from .utils.constants import HTTP_OK
@flask_app.route("/healthcheck", methods=["GET"])
def healthcheck():
return "OK", HTTP_OK
from .app_routes.node_routes import node_blueprint
flask_app.register_blueprint(node_blueprint)
from .app_routes.upload_routes import upload_blueprint
flask_app.register_blueprint(upload_blueprint)
from .app_routes.parameters_routes import parameters_blueprint
flask_app.register_blueprint(parameters_blueprint)
if is_server_static_files_enabled():
from .app_routes.static_routes import static_blueprint
logging.info("Visual interface will be available at http://localhost:5000")
flask_app.register_blueprint(static_blueprint)
if is_local_environment():
from .app_routes.image_routes import image_blueprint
logging.info("Environment set to LOCAL")
flask_app.register_blueprint(image_blueprint)
================================================
FILE: packages/backend/app/flask/socketio_init.py
================================================
import eventlet
eventlet.monkey_patch(all=False, socket=True)
from flask_socketio import SocketIO
from .flask_app import create_app
flask_app = create_app()
socketio = SocketIO(flask_app, cors_allowed_origins="*", async_mode="eventlet")
================================================
FILE: packages/backend/app/flask/sockets.py
================================================
import eventlet
from ..env_config import is_set_app_config_on_ui_enabled
eventlet.monkey_patch(all=False, socket=True)
from app.flask.socketio_init import flask_app
from app.flask.socketio_init import socketio
import logging
import json
from flask import g, request, session
from flask_socketio import emit
from ..root_injector import (
get_root_injector,
refresh_root_injector,
)
from .utils.constants import PARAMETERS_FIELD_NAME, ENV_API_KEYS
from ..processors.launcher.processor_launcher import ProcessorLauncher
from ..processors.context.processor_context_flask_request import (
ProcessorContextFlaskRequest,
)
import traceback
import os
def populate_request_global_object(data):
"""
This function is responsible for initializing individual request objects either from the
environmental variables or from the data passed as arguments, ensuring that the necessary API
keys are available throughout the request for different processes.
Parameters:
data (dict): A dictionary containing potentially necessary keys: "openai_api_key" and "stabilityai_api_key".
"""
use_env = os.getenv("USE_ENV_API_KEYS", "false").lower()
logging.debug("use_env: %s", use_env)
if use_env == "true":
for key in ENV_API_KEYS:
env_key = key.upper()
value = os.getenv(env_key)
if not value:
raise Exception(f"Required {env_key} not provided in environment.")
setattr(g, f"session_{key}", value)
else:
if not PARAMETERS_FIELD_NAME in data:
raise Exception(f"No {PARAMETERS_FIELD_NAME} provided in data.")
for key, value in data[PARAMETERS_FIELD_NAME].items():
if value:
setattr(g, f"session_{key}", value)
else:
raise Exception(f"No {key} provided in data.")
@socketio.on("connect")
def handle_connect():
logging.info("Client connected")
@socketio.on("process_file")
def handle_process_file(data):
"""
This event handler is activated when a "process_file" event is received via Socket.IO. It allows to run every node in
the file, even if they have been executed before.
Parameters:
data (dict): A dictionary encompassing the event's payload, which comprises the JSON configuration file
("jsonFile").
"""
try:
populate_request_global_object(data)
flow_data = json.loads(data.get("jsonFile"))
launcher = get_root_injector().get(ProcessorLauncher)
launcher.set_context(ProcessorContextFlaskRequest(g, session, request.sid))
if flow_data:
processors = launcher.load_processors(flow_data)
output = launcher.launch_processors(processors)
logging.debug("Emitting processing_result event with output: %s", output)
emit("run_end", {"output": output})
else:
logging.warning("Invalid input or missing configuration file")
emit("error", {"error": "Invalid input or missing configuration file"})
except Exception as e:
emit("error", {"error": str(e)})
traceback.print_exc()
logging.error(f"An error occurred: {str(e)}")
@socketio.on("run_node")
def handle_run_node(data):
"""
This event handler is activated when a "run_node" event is received via Socket.IO. It facilitates the processing
of the specified node in the data payload, launching only the designated node and preceding nodes if they
haven't been executed earlier.
Parameters:
data (dict): A dictionary encompassing the event's payload, which comprises the JSON configuration file
("jsonFile") and the name of the node to run ("nodeName").
"""
try:
populate_request_global_object(data)
flow_data = json.loads(data.get("jsonFile"))
node_name = data.get("nodeName")
launcher = get_root_injector().get(ProcessorLauncher)
launcher.set_context(ProcessorContextFlaskRequest(g, session, request.sid))
if flow_data and node_name:
processors = launcher.load_processors_for_node(flow_data, node_name)
output = launcher.launch_processors_for_node(processors, node_name)
logging.debug("Emitting processing_result event with output: %s", output)
emit("run_end", {"output": output})
else:
logging.warning("Invalid input or missing parameters")
emit("error", {"error": "Invalid input or missing parameters"})
except Exception as e:
emit(
"error",
{"error": str(e), "nodeName": node_name},
)
traceback.print_exc()
logging.error(f"An error occurred: {node_name} - {str(e)}")
@socketio.on("disconnect")
def handle_disconnect():
logging.info("Client disconnected")
@socketio.on("update_app_config")
def handle_update_app_config(data):
if not is_set_app_config_on_ui_enabled():
return
logging.info("Updating app config")
config_keys = [
"S3_BUCKET_NAME",
"S3_AWS_ACCESS_KEY_ID",
"S3_AWS_SECRET_ACCESS_KEY",
"S3_AWS_REGION_NAME",
"S3_ENDPOINT_URL",
"REPLICATE_API_KEY",
]
for key in config_keys:
value = data.get(key)
if value is not None and str(value).strip():
logging.info(f"Setting {key}")
os.environ[key] = value
refresh_root_injector()
================================================
FILE: packages/backend/app/flask/utils/constants.py
================================================
HTTP_OK = 200
HTTP_BAD_REQUEST = 400
HTTP_NOT_FOUND = 404
HTTP_UNAUTHORIZED = 401
SESSION_USER_ID_KEY = "user_id"
PARAMETERS_FIELD_NAME = "parameters"
ENV_API_KEYS = [
"openai_api_key",
"stabilityai_api_key",
"replicate_api_key",
"anthropic_api_key",
"openrouter_api_key",
]
================================================
FILE: packages/backend/app/llms/utils/max_token_for_model.py
================================================
import tiktoken
DEFAULT_MAX_TOKEN = 4097
def max_token_for_model(model_name: str) -> int:
if "gpt-4o" in model_name:
return 128000
token_data = {
# GPT-4.1 models
"gpt-4.1": 1047576,
"gpt-4.1-mini": 1047576,
"gpt-4.1-nano": 1047576,
# GPT-4 models
"gpt-4o": 128000,
"gpt-4o-2024-11-20": 128000,
"gpt-4o-mini": 128000,
"gpt-4-turbo": 128000,
"gpt-4-turbo-preview": 128000,
"gpt-4-1106-preview": 128000,
"gpt-4-vision-preview": 128000,
"gpt-4": 8192,
"gpt-4-0613": 8192,
"gpt-4-32k": 32768,
"gpt-4-32k-0613": 32768,
"gpt-4-0314": 8192,
"gpt-4-32k-0314": 32768,
# GPT-3.5 models
"gpt-3.5-turbo": 16385,
"gpt-3.5-turbo-1106": 16385,
"gpt-3.5-turbo-16k": 16385,
"gpt-3.5-turbo-instruct": 4097,
"gpt-3.5-turbo-0613": 4097,
"gpt-3.5-turbo-16k-0613": 16385,
"gpt-3.5-turbo-0301": 4097,
# Other GPT-3.5 models
"text-davinci-003": 4097,
"text-davinci-002": 4097,
"code-davinci-002": 8001,
}
return token_data.get(model_name, DEFAULT_MAX_TOKEN)
def nb_token_for_input(input: str, model_name: str) -> int:
try:
return len(tiktoken.encoding_for_model(model_name).encode(input))
except Exception as e:
default_model_for_token = "gpt-4o"
return len(tiktoken.encoding_for_model(default_model_for_token).encode(input))
================================================
FILE: packages/backend/app/log_config.py
================================================
import logging
import colorlog
def setup_logger(name: str):
formatter = colorlog.ColoredFormatter(
"%(log_color)s%(levelname)-8s%(reset)s %(message)s",
datefmt=None,
reset=True,
log_colors={
"DEBUG": "cyan",
"INFO": "green",
"WARNING": "yellow",
"ERROR": "red",
"CRITICAL": "red",
},
)
logger = logging.getLogger(name)
handler = logging.StreamHandler()
handler.setFormatter(formatter)
logger.addHandler(handler)
return logger
root_logger = setup_logger("root")
root_logger.setLevel(logging.INFO)
================================================
FILE: packages/backend/app/processors/components/__init__.py
================================================
================================================
FILE: packages/backend/app/processors/components/core/__init__.py
================================================
================================================
FILE: packages/backend/app/processors/components/core/ai_data_splitter_processor.py
================================================
import logging
from ...context.processor_context import ProcessorContext
from ..processor import ContextAwareProcessor
from .processor_type_name_utils import ProcessorType
from openai import OpenAI
def interpret_escape_sequences(separator):
escape_dict = {
r"\n": "\n",
r"\r": "\r",
r"\t": "\t",
}
return escape_dict.get(separator, separator)
class AIDataSplitterProcessor(ContextAwareProcessor):
processor_type = ProcessorType.AI_DATA_SPLITTER
DEFAULT_SEPARATOR = ";"
AI_MODE = "ai"
MANUAL_MODE = "manual"
def __init__(self, config, context: ProcessorContext):
super().__init__(config, context)
self.nb_output = 0
self.model = "gpt-4o"
self.api_key = context.get_value("openai_api_key")
def get_llm_response(self, messages):
client = OpenAI(api_key=self.api_key)
kwargs = {"model": self.model, "input": messages}
response = client.responses.create(**kwargs)
return response.output_text
def process(self):
if self.get_input_processor() is None:
return ""
input_data = self.get_input_processor().get_output(
self.get_input_node_output_key()
)
mode = self.get_input_by_name("mode", self.AI_MODE)
if mode == self.AI_MODE:
self.init_context(input_data)
answer = self.get_llm_response(self.messages)
data_to_split = answer.encode("utf-8").decode("utf8")
self.set_output(
data_to_split.split(AIDataSplitterProcessor.DEFAULT_SEPARATOR)
)
self.nb_output = len(self._output)
if mode == self.MANUAL_MODE:
separator = self.get_input_by_name(
"separator", AIDataSplitterProcessor.DEFAULT_SEPARATOR
)
separator = interpret_escape_sequences(separator)
self.set_output(input_data.split(separator))
self.nb_output = len(self._output)
return self._output
def init_context(self, input_data: str) -> None:
"""
Initialize the context for the OpenAI Chat model with a set of standard messages.
Additional user input data can be provided, which will be added to the messages.
:param input_data: Additional information or text provided by the user that needs processing.
"""
# Define the system message with clear instructions and examples
system_msg = (
"You are an assistant whose task is to separate ideas or concepts from the input text using semicolons (;). "
"Do not include any meta-comments or self-references in your responses. "
"Here are some examples of how to perform the task: "
"\n\n"
"Example 1:\n"
"Input: 'The main idea is that dogs are very popular pets, and many people enjoy walking them in parks. Another important concept is that dogs need a lot of exercise to stay healthy.'\n"
"Output: 'Dogs are very popular pets; many people enjoy walking them in parks; dogs need a lot of exercise to stay healthy.'\n\n"
"Example 2:\n"
"Input: '1) A picture of a woman 2) A video with a bird 3) Air conditioner'\n"
"Output: 'A picture of a woman; A video with a bird; Air conditioner.'\n\n"
"Example 3:\n"
"Input: 'Here are two ideas: - Dogs are better than cats - Birds are beautiful'\n"
"Output: 'Dogs are better than cats; Birds are beautiful.'\n\n"
"Example 4:\n"
"Input: 'Crée une interprétation artistique numérique de la ville de New York la nuit sous la pluie, mettant l'accent sur les reflets lumineux sur les surfaces mouillées. Imagine et dessine un nouveau type de fleur qui n'existe pas encore dans la nature. Assure-toi qu'elle a une allure exotique et utilise des couleurs vives et uniques que l'on ne trouve pas couramment chez les fleurs. Conçois une image représentant une scène du futur, avec des villes futuristes, des technologies avancées et des formes de vie artificielles coexistant avec des formes de vie naturelles.'\n"
"Output: 'Crée une interprétation artistique numérique de la ville de New York la nuit sous la pluie, mettant l'accent sur les reflets lumineux sur les surfaces mouillées; Imagine et dessine un nouveau type de fleur qui n'existe pas encore dans la nature. Assure-toi qu'elle a une allure exotique et utilise des couleurs vives et uniques que l'on ne trouve pas couramment chez les fleurs; Conçois une image représentant une scène du futur, avec des villes futuristes, des technologies avancées et des formes de vie artificielles coexistant avec des formes de vie naturelles.'\n\n"
"After reading the input, output each distinct idea or concept separated by semicolons."
)
user_nb_output = self.get_input_by_name("nb_output", 0)
if user_nb_output > 1:
system_msg += f"\nThe estimated number of outputs for the next message is {user_nb_output}."
self.messages = [
{"role": "system", "content": system_msg},
{"role": "user", "content": input_data},
]
def cancel(self):
pass
================================================
FILE: packages/backend/app/processors/components/core/dall_e_prompt_processor.py
================================================
from ...context.processor_context import ProcessorContext
from ..processor import ContextAwareProcessor
from openai import OpenAI
from .processor_type_name_utils import ProcessorType
class DallEPromptProcessor(ContextAwareProcessor):
processor_type = ProcessorType.DALLE_PROMPT
DEFAULT_MODEL = "dall-e-3"
DEFAULT_SIZE = "1024x1024"
DEFAULT_QUALITY = "standard"
def __init__(self, config, context: ProcessorContext):
super().__init__(config, context)
self.prompt = config.get("prompt")
self.size = config.get("size", DallEPromptProcessor.DEFAULT_SIZE)
self.quality = config.get("quality", DallEPromptProcessor.DEFAULT_QUALITY)
def process(self):
if self.get_input_processor() is not None:
self.prompt = (
self.get_input_processor().get_output(self.get_input_node_output_key())
if self.prompt is None or len(self.prompt) == 0
else self.prompt
)
api_key = self._processor_context.get_value("openai_api_key")
client = OpenAI(
api_key=api_key,
)
response = client.images.generate(
model=DallEPromptProcessor.DEFAULT_MODEL,
prompt=self.prompt,
n=1,
size=self.size,
quality=self.quality,
)
return response.data[0].url
def cancel(self):
pass
================================================
FILE: packages/backend/app/processors/components/core/display_processor.py
================================================
from .processor_type_name_utils import ProcessorType
from ..processor import BasicProcessor
class DisplayProcessor(BasicProcessor):
processor_type = "display"
def __init__(self, config):
super().__init__(config)
def process(self):
input_data = None
if self.get_input_processor() is None:
return ""
input_data = self.get_input_processor().get_output(
self.get_input_node_output_key()
)
return input_data
================================================
FILE: packages/backend/app/processors/components/core/file_processor.py
================================================
from .processor_type_name_utils import ProcessorType
from ..processor import BasicProcessor
class FileProcessor(BasicProcessor):
processor_type = ProcessorType.FILE
def __init__(self, config):
super().__init__(config)
self.url = config["fileUrl"]
def process(self):
return self.url
================================================
FILE: packages/backend/app/processors/components/core/gpt_vision_processor.py
================================================
import re
from typing import Any, List
from ...launcher.event_type import EventType
from ...launcher.processor_event import ProcessorEvent
from ...context.processor_context import ProcessorContext
from ..processor import ContextAwareProcessor
from .processor_type_name_utils import ProcessorType
from openai import OpenAI
from urllib.parse import urlparse
class GPTVisionProcessor(ContextAwareProcessor):
processor_type = ProcessorType.GPT_VISION
DEFAULT_MODEL = "gpt-4o"
def __init__(self, config, context: ProcessorContext):
super().__init__(config, context)
def _gather_image_url_values(self) -> List[Any]:
"""
Pull the value of `element` plus every `element_` child field
in the order they appear in self.fields_names.
"""
# Match element_0, element_1, … whatever the UI generates
child_pattern = re.compile(r"^image_url_\d+$")
# Preserve original order: parent first, then the children
ordered_field_names = [
fname
for fname in self.fields_names
if fname == "image_url" or child_pattern.match(fname)
]
values = [self.get_input_by_name(fname, None) for fname in ordered_field_names]
return [v for v in values if v is not None]
def process(self):
self.vision_inputs = {
"prompt": self.get_input_by_name("prompt"),
}
images_urls = self._gather_image_url_values()
if (
self.vision_inputs["prompt"] is None
or len(self.vision_inputs["prompt"]) == 0
):
raise ValueError("No prompt provided.")
if len(images_urls) == 0:
raise ValueError("No image provided.")
for url in images_urls:
if not self.is_valid_url(url):
raise ValueError(f"Invalid URL provided. \n {url}")
api_key = self._processor_context.get_value("openai_api_key")
client = OpenAI(
api_key=api_key,
)
content = []
for image in images_urls:
content.append(
{
"type": "image_url",
"image_url": {"url": image},
}
)
content.append(
{
"type": "text",
"text": self.vision_inputs["prompt"],
}
)
response = client.chat.completions.create(
model=GPTVisionProcessor.DEFAULT_MODEL,
messages=[
{
"role": "user",
"content": content,
}
],
max_tokens=4096,
stream=True,
)
final_response = ""
for chunk in response:
if not chunk.choices[0].delta.content:
continue
final_response += chunk.choices[0].delta.content
event = ProcessorEvent(self, final_response)
self.notify(EventType.STREAMING, event)
return final_response
def is_valid_url(self, url):
try:
result = urlparse(url)
return all([result.scheme, result.netloc])
except Exception:
return False
def cancel(self):
pass
================================================
FILE: packages/backend/app/processors/components/core/input_image_processor.py
================================================
from .processor_type_name_utils import ProcessorType
from ..processor import BasicProcessor
class InputImageProcessor(BasicProcessor):
processor_type = ProcessorType.INPUT_IMAGE
def __init__(self, config):
super().__init__(config)
self.inputText = config["inputText"]
def process(self):
return self.inputText
================================================
FILE: packages/backend/app/processors/components/core/input_processor.py
================================================
from .processor_type_name_utils import ProcessorType
from ..processor import BasicProcessor
class InputProcessor(BasicProcessor):
processor_type = ProcessorType.INPUT_TEXT
def __init__(self, config):
super().__init__(config)
self.inputText = config["inputText"]
def process(self):
return self.inputText
================================================
FILE: packages/backend/app/processors/components/core/llm_prompt_processor.py
================================================
import logging
from app.processors.exceptions import LightException
from ...launcher.processor_event import ProcessorEvent
from ...launcher.event_type import EventType
from ....llms.utils.max_token_for_model import max_token_for_model, nb_token_for_input
from ...context.processor_context import ProcessorContext
from ..processor import ContextAwareProcessor
from openai import OpenAI
from .processor_type_name_utils import ProcessorType
class LLMPromptProcessor(ContextAwareProcessor):
processor_type = ProcessorType.LLM_PROMPT
DEFAULT_MODEL = "gpt-4o"
streaming = True
models_with_web_search = [
"gpt-4o",
"gpt-4o-mini",
"gpt-4.1",
"gpt-4.1-mini",
]
def __init__(self, config, context: ProcessorContext):
super().__init__(config, context)
self.model = config.get("model", LLMPromptProcessor.DEFAULT_MODEL)
self.prompt = config.get("prompt", None)
def handle_stream_answer(self, awnser):
event = ProcessorEvent(self, awnser)
self.notify(EventType.STREAMING, event)
def nb_tokens_from_messages(self, messages, model):
"""
Calculates the total number of tokens in a list of messages using nb_token_for_input.
"""
total_tokens = 0
token_overhead = 3
for message in messages:
content_tokens = nb_token_for_input(message["content"], model)
total_tokens += content_tokens + token_overhead
total_tokens += token_overhead
return total_tokens
def check_for_html_tags(self, text):
"""
Checks if the given text contains HTML tags or attributes.
"""
if " 1:
context = self.get_input_by_name("context", None)
self.prompt = self.get_input_by_name("prompt", None)
else:
if self.get_input_processor() is not None:
context = self.get_input_processor().get_output(
self.get_input_node_output_key()
)
if self.prompt is None:
raise Exception("No prompt provided")
self.init_context(context)
total_tokens = self.nb_tokens_from_messages(self.messages, self.model)
model_max_tokens = max_token_for_model(self.model)
if total_tokens > model_max_tokens:
logging.warning("Messages size: " + str(total_tokens))
logging.warning("Model capacity: " + str(model_max_tokens))
message = (
"The text size exceeds the model's capacity. "
"Consider using a model with greater context handling capabilities or utilize the 'Find Similar Text' node to create a cohesive, condensed version of the context."
)
if (
context and self.check_for_html_tags(context)
) or self.check_for_html_tags(self.prompt):
message += (
"\n\n"
"Note: HTML tags or attributes are detected within the data provided. If they are unnecessary for this task, removing them could significantly reduce the context size."
)
raise Exception(message)
client = OpenAI(api_key=api_key)
kwargs = {"model": self.model, "input": self.messages, "stream": self.streaming}
if search_enabled:
kwargs["tools"] = [
{
"type": "web_search_preview",
"search_context_size": search_context_size,
}
]
stream = client.responses.create(**kwargs)
final_response = ""
for event in stream:
type = event.type
if type == "response.output_text.delta":
final_response += event.delta
self.handle_stream_answer(final_response)
if type == "response.completed":
response_data = event.response
final_response = response_data.output_text
if type == "response.failed":
response_data = event.response
if not hasattr(response_data, "error"):
logging.warning(f"Error from OpenAI with no data: {response_data}")
continue
raise LightException(
f"Error from OpenAI : {response_data.error.message}"
)
if type == "error":
raise LightException(f"Error from OpenAI : {event.message}")
return final_response
def init_context(self, context: str) -> None:
"""
Initialise the context for the LLM model with a standard set of messages.
Additional user input data can be provided, which will be added to the messages.
:param context: additional information to be used by the assistant.
"""
if context is None:
system_msg = "You are a helpful assistant. "
user_msg_content = self.prompt
else:
system_msg = (
"You are a helpful assistant. "
"You will respond to requests indicated by the '#Request' tag, "
"using the context provided under the '#Context' tag."
"Your response should feel natural and seamless, as if you've internalized the context "
"and are answering the request without needing to directly point back to the information provided"
)
user_msg_content = f"#Context: {context} \n\n#Request: {self.prompt}"
self.messages = [
{"role": "system", "content": system_msg},
{"role": "user", "content": user_msg_content},
]
def cancel(self):
pass
================================================
FILE: packages/backend/app/processors/components/core/merge_processor.py
================================================
from ..processor import ContextAwareProcessor
from .processor_type_name_utils import ProcessorType, MergeModeEnum
class MergeProcessor(ContextAwareProcessor):
processor_type = ProcessorType.MERGER_PROMPT
def __init__(self, config, context):
super().__init__(config, context)
self.merge_mode = MergeModeEnum(int(config["mergeMode"]))
def update_prompt(self, inputs):
for idx, value in enumerate(inputs, start=1):
placeholder = f"${{input-{idx}}}"
self.prompt = self.prompt.replace(placeholder, str(value))
def process(self):
self.prompt = self.get_input_by_name("prompt", "")
input_names = self.get_input_names_from_config()
inputs = [self.get_input_by_name(name, "") for name in input_names]
self.update_prompt(inputs)
return self.prompt
def cancel(self):
pass
================================================
FILE: packages/backend/app/processors/components/core/processor_type_name_utils.py
================================================
from enum import Enum
class MergeModeEnum(Enum):
MERGE = 1
MERGE_AND_PROMPT = 2
class ProcessorType(Enum):
INPUT_TEXT = "input-text"
INPUT_IMAGE = "input-image"
URL_INPUT = "url_input"
LLM_PROMPT = "llm-prompt"
GPT_VISION = "gpt-vision"
YOUTUBE_TRANSCRIPT_INPUT = "youtube_transcript_input"
DALLE_PROMPT = "dalle-prompt"
STABLE_DIFFUSION_STABILITYAI_PROMPT = "stable-diffusion-stabilityai-prompt"
STABLE_VIDEO_DIFFUSION_REPLICATE = "stable-video-diffusion-replicate"
REPLICATE = "replicate"
MERGER_PROMPT = "merger-prompt"
AI_DATA_SPLITTER = "ai-data-splitter"
TRANSITION = "transition"
DISPLAY = "display"
FILE = "file"
STABLE_DIFFUSION_THREE = "stabilityai-stable-diffusion-3-processor"
TEXT_TO_SPEECH = "openai-text-to-speech-processor"
DOCUMENT_TO_TEXT = "document-to-text-processor"
STABILITYAI = "stabilityai-generic-processor"
CLAUDE = "claude-anthropic-processor"
REPLACE_TEXT = "replace-text"
================================================
FILE: packages/backend/app/processors/components/core/replicate_processor.py
================================================
from datetime import datetime
import logging
from queue import Queue
import time
from urllib.parse import urlparse
from app.env_config import is_s3_enabled
from ...launcher.event_type import EventType
from ...launcher.processor_event import ProcessorEvent
from ....utils.processor_utils import stream_download_file_as_binary
from ...exceptions import LightException
from ....utils.replicate_utils import (
get_input_schema_from_open_API_schema,
get_model_openapi_schema,
get_output_schema_from_open_API_schema,
)
from ...context.processor_context import ProcessorContext
from ..processor import ContextAwareProcessor
import replicate
from .processor_type_name_utils import ProcessorType
from ....tasks.task_exception import TaskAlreadyRegisteredError
from ....tasks.thread_pool_task_manager import add_task, register_task_processor
from ....tasks.task_utils import wait_for_result
class ReplicateProcessor(ContextAwareProcessor):
processor_type = ProcessorType.REPLICATE
def __init__(self, config, context: ProcessorContext):
super().__init__(config, context)
self.is_processing = False
self.config = config
self.model = config.get("model")
if self.model is None:
self.model = config.get("config").get("nodeName")
if ":" not in self.model:
logging.warning(f"Model {self.model} has no version")
raise Exception(f"Cannot find version for this model : {self.model}.")
self.model_name_withouth_version = self.model.split(":")[0]
def get_prediction_result(
self, prediction, processor, timeout=3600.0, initial_sleep=0.1, max_sleep=5.0
):
results_queue = Queue()
add_task("replicate_prediction_wait", (prediction, processor), results_queue)
try:
prediction = wait_for_result(
results_queue, timeout, initial_sleep, max_sleep
)
except TimeoutError as e:
raise TimeoutError("Prediction result timed out")
return prediction
@staticmethod
def wait_for_prediction_task(task_data):
prediction, processor = task_data
while prediction.status not in ["succeeded", "failed", "canceled"]:
time.sleep(prediction._client.poll_interval)
if prediction.status == "processing":
processor.is_processing = True
prediction.reload()
return prediction
def register_background_task(self):
try:
register_task_processor(
"replicate_prediction_wait",
self.wait_for_prediction_task,
max_concurrent_tasks=100,
)
except TaskAlreadyRegisteredError as e:
pass
def process(self):
api_key = self._processor_context.get_value("replicate_api_key")
self.schema = get_model_openapi_schema(self.model_name_withouth_version)
input_processors = self.get_input_processors()
input_output_keys = self.get_input_node_output_keys()
input_names = self.get_input_names()
if input_processors:
for processor, name, key in zip(
input_processors, input_names, input_output_keys
):
output = processor.get_output(key)
if output is None:
continue
input_type = self._get_nested_input_schema_property(name, "type")
if input_type == "integer":
output = int(output)
if input_type == "number":
output = float(output)
self.config[name] = output
api = replicate.Client(api_token=api_key)
output_schema = get_output_schema_from_open_API_schema(self.schema["schema"])
logging.debug(f"Output schema : {output_schema}")
output_type = output_schema.get("type")
output_array_display = output_schema.get("x-cog-array-display")
output_format = output_schema.get("format")
if not ":" in self.model:
logging.warning(f"Model {self.model} has no version")
raise Exception("Cannot find version for this model")
rest, version_id = self.model.split(":")
self.config["disable_safety_checker"] = True
try:
self.prediction = api.predictions.create(
version=version_id, input=self.config
)
except Exception as e:
logging.warning(f"Error while creating prediction : {e}")
raise LightException(
"Please review your input to ensure it aligns with the expected format. \n\n"
"For reference, you can review the examples here: \n"
f"https://replicate.com/{self.model_name_withouth_version}/examples\n\n"
f"Error message from Replicate: \n\n {e}"
)
self.register_background_task()
self.prediction = self.get_prediction_result(self.prediction, self)
if self.prediction.status != "succeeded":
replicate_error_message = self.prediction.error
message_str = f"Your Replicate prediction ended with status : {self.prediction.status} \n\n"
if replicate_error_message and self.prediction.status != "canceled":
message_str += (
f"There may be an issue with the parameters provided for the model '{self.model_name_withouth_version}'. \n\n"
"Please review your input to ensure it aligns with the expected format. \n\n"
"For reference, you can review the examples here: \n"
f"https://replicate.com/{self.model_name_withouth_version}/examples\n\n"
f"Error message from Replicate: {replicate_error_message}"
)
exception = Exception(message_str)
exception.rollback_not_needed = True
raise exception
output = self.prediction.output
self.metrics = self.prediction.metrics
isUriOutput = output_format == "uri"
if output_type == "array" and output_array_display == "concatenate":
output = "".join(output)
elif output_type == "array":
items_type = output_schema.get("items").get("type")
items_format = output_schema.get("items").get("format")
isUriOutput = items_format == "uri"
output = output
elif output_type == "string":
if isinstance(output, list):
output = "".join(output)
else:
output = [output]
event = ProcessorEvent(self, output)
self.notify(EventType.STREAMING, event)
if isUriOutput:
if isinstance(output, list):
new_output = []
for uri in output:
new_uri = self.upload_replicate_uri_to_storage(uri)
new_output.append(new_uri)
output = new_output
else:
output = self.upload_replicate_uri_to_storage(output)
return output
def upload_replicate_uri_to_storage(self, uri):
if not is_s3_enabled():
return uri
storage = self.get_storage()
timestamp_str = datetime.now().strftime("%Y%m%d%H%M%S%f")
extension = None
try:
parsed = urlparse(uri)
path = parsed.path
if "." in path:
extension = path.split(".")[-1]
else:
logging.warning("No extension found in URI: %s", uri)
except Exception as e:
logging.warning("Error extracting extension from URI (%s): %s", uri, str(e))
if not extension:
logging.warning("Aborting Upload - No extension found in URI: %s", uri)
return uri
filename = f"{self.name}-{timestamp_str}.{extension}"
file = stream_download_file_as_binary(uri)
url = storage.save(filename, file)
return url
def _get_nested_input_schema_property(self, property_name, nested_key):
return (
get_input_schema_from_open_API_schema(self.schema.get("schema", {}))
.get("properties", {})
.get(property_name, {})
.get(nested_key)
)
def cancel(self):
api_key = self._processor_context.get_value("replicate_api_key")
api = replicate.Client(api_token=api_key)
api.predictions.cancel(id=self.prediction.id)
================================================
FILE: packages/backend/app/processors/components/core/stable_diffusion_stabilityai_prompt_processor.py
================================================
import base64
from ...context.processor_context import ProcessorContext
from ..processor import ContextAwareProcessor
from datetime import datetime
import requests
import os
from .processor_type_name_utils import ProcessorType
class StableDiffusionStabilityAIPromptProcessor(ContextAwareProcessor):
processor_type = ProcessorType.STABLE_DIFFUSION_STABILITYAI_PROMPT
def __init__(self, config, context: ProcessorContext):
super().__init__(config, context)
self.prompt = config.get("prompt")
size = config.get("size", "1024x1024")
self.height = int(size.split("x")[0])
self.width = int(size.split("x")[1])
self.style_preset = config.get("style_preset", "")
self.samples = 1
self.engine_id = "stable-diffusion-xl-1024-v1-0"
self.api_host = os.getenv(
"STABLE_DIFFUSION_STABILITYAI_API_HOST", "https://api.stability.ai"
)
def prepare_and_process_response(self, response):
if response.status_code != 200:
raise Exception("Non-200 response: " + str(response.text))
data = response.json()
first_image = data["artifacts"][0]["base64"]
image_data = base64.b64decode(first_image)
storage = self.get_storage()
timestamp_str = datetime.now().strftime("%Y%m%d%H%M%S%f")
filename = f"{self.name}-{timestamp_str}.png"
url = storage.save(filename, image_data)
return url
def setup_data_to_send(self):
if self.get_input_processor() is not None:
self.prompt = (
self.get_input_processor().get_output(self.get_input_node_output_key())
if self.prompt is None or len(self.prompt) == 0
else self.prompt
)
data_to_send = {
"text_prompts": [{"text": f"{self.prompt}"}],
"cfg_scale": 7,
"height": self.height,
"width": self.width,
"samples": self.samples,
"steps": 30,
}
return data_to_send
def process(self):
data_to_send = self.setup_data_to_send()
api_key = self._processor_context.get_value("stabilityai_api_key")
response = requests.post(
f"{self.api_host}/v1/generation/{self.engine_id}/text-to-image",
headers={
"Content-Type": "application/json",
"Accept": "application/json",
"Authorization": f"Bearer {api_key}",
},
json=data_to_send,
)
return self.prepare_and_process_response(response)
def cancel(self):
pass
================================================
FILE: packages/backend/app/processors/components/core/stable_video_diffusion_replicate.py
================================================
import os
from urllib.parse import urlparse
from ...context.processor_context import ProcessorContext
from ..processor import ContextAwareProcessor
import replicate
from .processor_type_name_utils import ProcessorType
class StableVideoDiffusionReplicaterocessor(ContextAwareProcessor):
processor_type = ProcessorType.STABLE_VIDEO_DIFFUSION_REPLICATE
stable_video_diffusion_model = "stability-ai/stable-video-diffusion:3f0457e4619daac51203dedb472816fd4af51f3149fa7a9e0b5ffcf1b8172438"
def __init__(self, config, context: ProcessorContext):
super().__init__(config, context)
self.length = config.get("length", "14_frames_with_svd")
self.frames_per_second = config.get("frames_per_second", "6")
def process(self):
input_image_url = None
if self.get_input_processor() is not None:
input_image_url = self.get_input_processor().get_output(
self.get_input_node_output_key()
)
if input_image_url is None:
return "No image provided."
if not self.is_valid_url(input_image_url):
return "Invalid URL provided."
api_key = self._processor_context.get_value("replicate_api_key")
api = replicate.Client(api_token=api_key)
output = api.run(
StableVideoDiffusionReplicaterocessor.stable_video_diffusion_model,
input={
"cond_aug": 0.02,
"decoding_t": 7,
"input_image": input_image_url,
"video_length": self.length,
"sizing_strategy": "maintain_aspect_ratio",
"motion_bucket_id": 127,
"frames_per_second": int(self.frames_per_second),
},
)
return output
def is_valid_url(self, url):
try:
result = urlparse(url)
return all([result.scheme, result.netloc])
except Exception:
return False
def cancel(self):
pass
================================================
FILE: packages/backend/app/processors/components/core/transition_processor.py
================================================
from .processor_type_name_utils import ProcessorType
from ..processor import BasicProcessor
class TransitionProcessor(BasicProcessor):
processor_type = ProcessorType.TRANSITION
def __init__(self, config):
super().__init__(config)
def process(self):
input_data = None
if self.get_input_processor() is None:
return ""
input_data = self.get_input_processor().get_output(
self.get_input_node_output_key()
)
return input_data
================================================
FILE: packages/backend/app/processors/components/core/url_input_processor.py
================================================
import random
from bs4 import BeautifulSoup
import requests
from ....utils.processor_utils import is_valid_url
from ..processor import BasicProcessor
from .processor_type_name_utils import ProcessorType
import logging
from markdownify import markdownify
class URLInputProcessor(BasicProcessor):
WAIT_TIMEOUT = 60
GET_TIMEOUT = 20
processor_type = ProcessorType.URL_INPUT
USER_AGENTS = [
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.1 Safari/605.1.15",
"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:97.0) Gecko/20100101 Firefox/97.0",
]
def __init__(self, config):
super().__init__(config)
def get_random_user_agent():
return random.choice(URLInputProcessor.USER_AGENTS)
def fetch_content_simple(self):
"""
Fetches the website content using a simple GET request.
"""
try:
headers = {"User-Agent": URLInputProcessor.get_random_user_agent()}
response = requests.get(self.url, headers=headers, timeout=self.GET_TIMEOUT)
response.raise_for_status()
return response.text
except requests.RequestException as e:
logging.warning(f"Failed to fetch content using simple GET: {e}")
return None
def process(self):
self.url = self.get_input_by_name("url")
self.loading_mode = self.get_input_by_name("loading_mode", "browser")
self.effective_load_mode = self.loading_mode
# Validate URL input
if not self.url or not isinstance(self.url, str) or self.url.strip() == "":
raise Exception("No URL provided.", "noURLProvided")
self.url = self.url.strip()
self.original_url = self.url
if not (self.url.startswith("https://") or self.url.startswith("http://")):
logging.warning(
"URL does not start with 'https://' or 'http://' - compensating by prepending 'https://'."
)
self.url = "https://" + self.url
if not is_valid_url(self.url):
logging.warning(f"Invalid URL: {self.url}")
raise Exception(
f"The provided URL '{self.original_url}' is not valid.\n\n"
"Please ensure the URL follows the correct format, e.g., 'https://www.example.com' or 'https://example.com'."
)
# Get additional parameters
self.selectors = self.get_input_by_name("selectors", [])
self.selectors_to_remove = self.get_input_by_name("selectors_to_remove", [])
self.with_html_tags = self.get_input_by_name("with_html_tags", False)
self.with_html_attributes = self.get_input_by_name(
"with_html_attributes", False
)
response = None
task_data = {
"url": self.url,
"selectors": self.selectors,
"selectors_to_remove": self.selectors_to_remove,
"with_html_tags": self.with_html_tags,
"with_html_attributes": self.with_html_attributes,
}
content = self.fetch_content_simple()
response = self.process_content_with_beautiful_soup(content, task_data)
return response
def process_content_with_beautiful_soup(self, content, task_data):
"""
Process the HTML content using BeautifulSoup while considering the following parameters:
- selectors: a list of CSS selectors; if provided, only matching elements are kept.
- selectors_to_remove: a list of CSS selectors for elements that should be removed.
- with_html_tags: if True, the returned result will include HTML tags; otherwise, plain text.
- with_html_attributes: if True (and with_html_tags is True), HTML attributes will be kept;
otherwise, they will be stripped.
"""
if not content:
return ""
soup = BeautifulSoup(content, "html.parser")
selectors = task_data.get("selectors", [])
if isinstance(selectors, str):
selectors = [selectors]
selectors_to_remove = task_data.get("selectors_to_remove", [])
if isinstance(selectors_to_remove, str):
selectors_to_remove = [selectors_to_remove]
for selector in selectors_to_remove:
for element in soup.select(selector):
element.decompose()
if selectors:
selected_elements = soup.select(", ".join(selectors))
if not selected_elements:
selected_elements = [soup]
else:
selected_elements = [soup]
with_html_tags = task_data.get("with_html_tags", False)
with_html_attributes = task_data.get("with_html_attributes", False)
if with_html_tags:
if not with_html_attributes:
for element in selected_elements:
if hasattr(element, "attrs"):
element.attrs = {}
for tag in element.find_all(True):
tag.attrs = {}
html_output = "".join(str(element) for element in selected_elements)
return html_output
else:
html_output = "".join(str(element) for element in selected_elements)
text_output = markdownify(html_output)
return text_output
================================================
FILE: packages/backend/app/processors/components/core/youtube_transcript_input_processor.py
================================================
import logging
from ...utils.retry_mixin import RetryMixin
from ...exceptions import LightException
from ..processor import BasicProcessor
from youtube_transcript_api import (
YouTubeTranscriptApi,
TranscriptsDisabled,
NoTranscriptFound,
VideoUnavailable,
)
from .processor_type_name_utils import ProcessorType
class YoutubeTranscriptInputProcessor(BasicProcessor, RetryMixin):
processor_type = ProcessorType.YOUTUBE_TRANSCRIPT_INPUT
def __init__(self, config):
super().__init__(config)
self.max_retries = 2
self.retry_delay = 0
def get_video_id(self):
if "watch?v=" in self.url:
return self.url.split("watch?v=")[-1].split("&")[0]
elif "youtu.be/" in self.url:
return self.url.split("youtu.be/")[-1].split("?")[0]
else:
raise LightException(f"Invalid YouTube URL {self.url}")
def process_with_youtube_transcript_api(self):
video_id = self.get_video_id()
try:
transcript_data = self.get_transcript(video_id)
except (TranscriptsDisabled, NoTranscriptFound) as e:
logging.warning(
f"Transcript not available or disabled for video {self.url}"
)
logging.debug(e)
raise Exception(f"No transcription found for {self.url}")
except VideoUnavailable as e:
logging.warning(f"Video is unavailable")
logging.debug(e)
raise Exception(f"Video is unavailable for {self.url}")
except Exception as e:
logging.warning(f"Failed to retrieve transcript")
logging.debug(e)
raise Exception(self.create_no_transcript_error_message(e))
content = " ".join([entry["text"] for entry in transcript_data])
if not content:
raise Exception(f"No transcription found for {self.url}")
return content
def get_transcript(self, video_id):
"""Attempts to get the transcript in the requested language or translate if not available."""
try:
# Try to get the transcript in the requested language
return YouTubeTranscriptApi.get_transcript(
video_id, languages=[self.language]
)
except NoTranscriptFound:
# If transcript in the requested language is not found, try to find a translatable one
return self.get_translatable_transcript(video_id)
except Exception as e:
logging.debug(f"Failed to retrieve transcript with first proxy")
logging.debug(e)
# Retry with a new proxy
return YouTubeTranscriptApi.get_transcript(
video_id, languages=[self.language]
)
def get_translatable_transcript(self, video_id):
"""Finds a translatable transcript and translates it to the requested language."""
try:
# List all transcripts for the video
transcripts = YouTubeTranscriptApi.list_transcripts(video_id)
# Find an auto-generated, translatable transcript
for transcript in transcripts:
if transcript.is_translatable:
return transcript.translate(self.language).fetch()
# Raise an exception if no translatable transcript is found
raise NoTranscriptFound(
f"No translatable transcript available for video {video_id}"
)
except Exception as e:
logging.warning(f"Failed to find a translatable transcript")
logging.debug(e)
raise
def create_no_transcript_error_message(self, e):
requested_languages = getattr(e, "_requested_language_codes", None)
transcript_data = getattr(e, "_transcript_data", None)
error_message = f"Failed to retrieve transcript for {self.url} \n\nRequested Language: {requested_languages} \n\n{transcript_data}"
return error_message
def retrieve_transcript(self, url, language):
self.url = url
self.language = language
if not self.url:
raise Exception("No URL provided")
content = self.run_with_retry(self.process_with_youtube_transcript_api)
if not content:
raise Exception(f"No transcription found for {self.url}")
logging.info(f"Transcription for {self.url} retrieved successfully")
return content
def process(self):
url = self.get_input_by_name("url")
language = self.get_input_by_name("language")
logging.info(language)
return self.retrieve_transcript(url, language)
================================================
FILE: packages/backend/app/processors/components/extension/__init__.py
================================================
================================================
FILE: packages/backend/app/processors/components/extension/claude_anthropic_processor.py
================================================
import logging
from datetime import datetime
import anthropic
from ...context.processor_context import ProcessorContext
from ..model import Field, NodeConfig, Option, Condition, ConditionGroup
from .extension_processor import ContextAwareExtensionProcessor
from ...launcher.processor_event import ProcessorEvent
from ...launcher.event_type import EventType
class ClaudeAnthropicProcessor(ContextAwareExtensionProcessor):
processor_type = "claude-anthropic-processor"
model_config_map = {
"claude-3-7-sonnet-latest": {
"max_tokens": 8192,
"max_tokens_thinking": 64000,
},
"claude-3-5-haiku-latest": {
"max_tokens": 8192,
"max_tokens_thinking": 8192,
},
"claude-3-5-sonnet-latest": {
"max_tokens": 8192,
"max_tokens_thinking": 8192,
},
"claude-3-opus-latest": {
"max_tokens": 4096,
"max_tokens_thinking": 4096,
},
"claude-3-haiku-20240307": {
"max_tokens": 4096,
"max_tokens_thinking": 4096,
},
"claude-3-5-sonnet-20240620": {
"max_tokens": 8192,
"max_tokens_thinking": 8192,
},
"claude-opus-4-0": {
"max_tokens": 32000,
"max_tokens_thinking": 32000,
},
"claude-sonnet-4-0": {
"max_tokens": 64000,
"max_tokens_thinking": 64000,
},
}
def __init__(self, config, context: ProcessorContext):
super().__init__(config, context)
self.reasoning_content = ""
def get_node_config(self):
# Conditions
claude_thinking_condition = Condition(
field="model",
operator="in",
value=["claude-3-7-sonnet-latest", "claude-opus-4-0", "claude-sonnet-4-0"],
)
thinking_enabled_condition = Condition(
field="thinking", operator="equals", value=True
)
budget_token_condition = ConditionGroup(
conditions=[claude_thinking_condition, thinking_enabled_condition],
logic="AND",
)
# Fields
prompt = Field(
name="prompt",
label="prompt",
type="textarea",
required=True,
placeholder="InputTextPlaceholder",
hasHandle=True,
)
prompt_context = Field(
name="context",
label="context",
type="textfield",
placeholder="InputTextPlaceholder",
hasHandle=True,
description="Additional context that will be used to answer your prompt.",
)
temperature = Field(
name="temperature",
label="temperature",
type="slider",
min=0,
max=1,
defaultValue=1,
placeholder="InputTextPlaceholder",
description="Use temperature closer to 0.0 for analytical tasks, and closer to 1.0 for creative tasks.",
)
budget_token = Field(
name="budget_tokens",
label="budget_tokens",
type="slider",
defaultValue=1024,
max=63999,
min=1024,
condition=budget_token_condition,
description=(
"Determines how many tokens Claude can use for its internal reasoning process. "
"Larger budgets can enable more thorough analysis for complex problems, improving response quality."
),
)
model_options = [
Option(
default=False,
value="claude-3-7-sonnet-latest",
label="Claude 3.7 Sonnet",
),
Option(
default=False,
value="claude-3-5-haiku-latest",
label="Claude 3.5 Haiku",
),
Option(
default=False,
value="claude-3-5-sonnet-latest",
label="Claude 3.5 Sonnet",
),
Option(
default=False,
value="claude-3-opus-latest",
label="Claude 3 Opus",
),
Option(
default=False,
value="claude-3-haiku-20240307",
label="Claude 3 Haiku",
),
Option(
default=False,
value="claude-opus-4-0",
label="Claude 4 Opus",
),
Option(
default=True,
value="claude-sonnet-4-0",
label="Claude 4 Sonnet",
),
]
model = Field(
name="model",
label="model",
type="select",
options=model_options,
required=True,
)
thinking = Field(
name="thinking",
label="thinking",
type="boolean",
condition=claude_thinking_condition,
)
fields = [
prompt,
prompt_context,
model,
thinking,
budget_token,
temperature,
]
config = NodeConfig(
nodeName="ClaudeAnthropic",
processorType=self.processor_type,
icon="AnthropicLogo",
fields=fields,
outputType="markdown",
section="models",
helpMessage="claudeAnthropichHelp",
showHandlesNames=True,
)
return config
def handle_stream_awnser(self, awnser):
event = ProcessorEvent(self, awnser)
self.notify(EventType.STREAMING, event)
def process(self):
"""
Retrieve max_tokens from a map instead of the node config.
If 'thinking' is enabled and the model supports it, we choose a different max_tokens.
"""
prompt = self.get_input_by_name("prompt")
prompt_context = self.get_input_by_name("context", None)
model = self.get_input_by_name("model", "claude-3-5-sonnet-20240620")
temperature = self.get_input_by_name("temperature", 1)
thinking = self.get_input_by_name("thinking", False)
if "3-7" not in model and "4-0" not in model:
thinking = False
budget_tokens = None
if thinking:
budget_tokens = self.get_input_by_name("budget_tokens", 1024)
model_config = ClaudeAnthropicProcessor.model_config_map.get(
model,
ClaudeAnthropicProcessor.model_config_map["claude-3-5-sonnet-20240620"],
)
if thinking:
max_tokens = model_config["max_tokens_thinking"]
else:
max_tokens = model_config["max_tokens"]
if prompt is None:
return None
api_key = self._processor_context.get_value("anthropic_api_key")
if api_key is None:
raise Exception("No Anthropic API key found")
client = anthropic.Anthropic(api_key=api_key)
awnser = ""
if prompt_context is not None:
messages = [
{
"role": "user",
"content": f"Context: {prompt_context} \n Prompt: {prompt}",
}
]
else:
messages = [{"role": "user", "content": prompt}]
stream_kwargs = {
"model": model,
"temperature": temperature,
"max_tokens": max_tokens,
"messages": messages,
}
if thinking:
stream_kwargs["thinking"] = {
"budget_tokens": budget_tokens,
"type": "enabled",
}
with client.messages.stream(**stream_kwargs) as stream:
try:
current_block_type = None
for event in stream:
if event.type == "content_block_start":
current_block_type = event.content_block.type
elif event.type == "content_block_delta":
if event.delta.type == "thinking_delta":
self.reasoning_content += event.delta.thinking
elif event.delta.type == "text_delta":
awnser += event.delta.text
self.handle_stream_awnser(awnser)
elif event.type == "message_stop":
break
except Exception as e:
logging.error(f"An error occurred during streaming : {e}")
raise Exception("An error occurred during streaming")
finally:
stream.close()
return awnser
def cancel(self):
pass
================================================
FILE: packages/backend/app/processors/components/extension/deepseek_processor.py
================================================
from ...launcher.event_type import EventType
from ...launcher.processor_event import ProcessorEvent
from ...context.processor_context import ProcessorContext
from ..model import Field, NodeConfig, Option
from .extension_processor import ContextAwareExtensionProcessor
from openai import OpenAI
class DeepSeekProcessor(ContextAwareExtensionProcessor):
processor_type = "deepseek-processor"
streaming = True
def __init__(self, config, context: ProcessorContext):
super().__init__(config, context)
self.reasoning_content = ""
def get_node_config(self):
context = Field(
name="context",
label="context",
type="textfield",
required=False,
placeholder="ContextPlaceholder",
hasHandle=True,
)
text = Field(
name="prompt",
label="prompt",
type="textarea",
required=True,
placeholder="PromptPlaceholder",
hasHandle=True,
)
model_options = [
Option(
default=False,
value="deepseek-chat",
label="V3",
),
Option(
default=True,
value="deepseek-reasoner",
label="R1",
),
]
model = Field(
name="model",
type="option",
options=model_options,
required=True,
)
fields = [model, context, text]
config = NodeConfig(
nodeName="DeepSeek",
processorType=self.processor_type,
icon="DeepSeekLogo",
fields=fields,
outputType="text",
section="models",
helpMessage="deepSeekHelp",
showHandlesNames=True,
)
return config
def process(self):
prompt = self.get_input_by_name("prompt")
context = self.get_input_by_name("context", "")
model = self.get_input_by_name("model")
if prompt is None:
return None
api_key = self._processor_context.get_value("deepseek_api_key")
if api_key is None:
raise Exception("No DeepSeek API key found")
client = OpenAI(api_key=api_key, base_url="https://api.deepseek.com")
response = client.chat.completions.create(
model=model,
messages=[
{
"role": "user",
"content": f"{context} {prompt}",
}
],
stream=self.streaming,
)
if self.streaming:
final_response = ""
for chunk in response:
r_content = getattr(chunk.choices[0].delta, "reasoning_content", None)
if r_content is not None:
self.reasoning_content += r_content
if not chunk.choices[0].delta.content:
continue
final_response += chunk.choices[0].delta.content
event = ProcessorEvent(self, final_response)
self.notify(EventType.STREAMING, event)
return final_response
return response.choices[0].message.content
def cancel(self):
pass
================================================
FILE: packages/backend/app/processors/components/extension/document_to_text_processor.py
================================================
import logging
from queue import Queue
import requests
from ....tasks.task_exception import TaskAlreadyRegisteredError
from ..node_config_builder import FieldBuilder, NodeConfigBuilder
from ....tasks.thread_pool_task_manager import add_task, register_task_processor
from ....utils.processor_utils import (
create_temp_file_with_bytes_content,
get_max_file_size_in_mb,
is_accepted_url_file_size,
is_s3_file,
is_valid_url,
)
from ....tasks.task_utils import wait_for_result
from ..model import NodeConfig
from .extension_processor import BasicExtensionProcessor
from langchain.document_loaders import (
UnstructuredPDFLoader,
UnstructuredHTMLLoader,
CSVLoader,
JSONLoader,
TextLoader,
PyMuPDFLoader,
)
class DocumentToText(BasicExtensionProcessor):
processor_type = "document-to-text-processor"
WAIT_TIMEOUT = 60
def __init__(self, config):
super().__init__(config)
self.loaders = {
"application/pdf": PyMuPDFLoader,
"text/plain": TextLoader,
"text/csv": CSVLoader,
"text/html": UnstructuredHTMLLoader,
"application/json": JSONLoader,
}
self.accepted_mime_types = self.loaders.keys()
def get_node_config(self) -> NodeConfig:
urlField = (
FieldBuilder()
.set_name("document_url")
.set_label("document_url")
.set_type("textfield")
.set_required(True)
.set_placeholder("URLPlaceholder")
.set_has_handle(True)
.build()
)
return (
NodeConfigBuilder()
.set_node_name("DocumentToText")
.set_processor_type(self.processor_type)
.set_icon("FaFile")
.set_section("input")
.set_help_message("documentToTextHelp")
.set_show_handles(True)
.set_output_type("text")
.set_default_hide_output(True)
.add_field(urlField)
.build()
)
def get_loader_for_mime_type(self, mime_type, path):
"""Return an instance of the loader class associated with the given mime_type."""
loader_class = self.loaders.get(mime_type)
if loader_class:
return loader_class(file_path=path)
else:
return None
def load_document(self, loader):
results_queue = Queue()
add_task("document_loader", loader, results_queue)
document = None
try:
document = wait_for_result(results_queue)
except TimeoutError as e:
raise TimeoutError("Timeout - The document took too long to load")
return document
@staticmethod
def document_loader_task(loader):
return loader.load()
def register_background_task(self):
try:
register_task_processor("document_loader", self.document_loader_task)
except TaskAlreadyRegisteredError as e:
pass
def process(self):
url = self.get_input_by_name("document_url")
if not is_valid_url(url):
raise ValueError("Invalid URL")
if not is_s3_file(url) and not is_accepted_url_file_size(url):
raise ValueError(
f"File size is too large (Max : {get_max_file_size_in_mb()})"
)
r = requests.get(url)
if r.status_code != 200:
raise ValueError(
"Check the url of your file; returned status code %s" % r.status_code
)
mime_type = r.headers.get("Content-Type")
if not is_s3_file(url) and mime_type not in self.accepted_mime_types:
raise ValueError("The file type is not supported.")
temp_file, temp_dir = create_temp_file_with_bytes_content(r.content)
file_path = str(temp_file)
loader = self.get_loader_for_mime_type(mime_type, file_path)
self.register_background_task()
try:
document = self.load_document(loader)
if len(document) > 0:
output = ""
for doc in document:
output += doc.page_content
return output
else:
return None
except Exception as e:
logging.warning(f"Failed to load document from URL: {e}")
raise e
finally:
temp_dir.cleanup()
================================================
FILE: packages/backend/app/processors/components/extension/extension_processor.py
================================================
from ..model import NodeConfig
from ...context.processor_context import ProcessorContext
from ..processor import BasicProcessor, ContextAwareProcessor
class ExtensionProcessor:
"""Base interface for extension processors"""
def get_node_config(self) -> NodeConfig:
pass
class DynamicExtensionProcessor:
"""Base interface for dynamic extension processors - These nodes config are populated by an API call after a user choice"""
def get_dynamic_node_config(self, data) -> NodeConfig:
pass
class BasicExtensionProcessor(ExtensionProcessor, BasicProcessor):
"""A basic extension processor that does not depend on user-specific parameters.
Inherits basic processing capabilities from BasicProcessor and schema handling from ExtensionProcessor.
Args:
config (dict): Configuration dictionary for processor setup.
"""
def __init__(self, config):
super().__init__(config)
class ContextAwareExtensionProcessor(ExtensionProcessor, ContextAwareProcessor):
"""An extension processor that requires context about the user, such as user-specific settings or keys.
This class supports context-aware processing by incorporating user context into the processing flow.
Args:
config (dict): Configuration dictionary for processor setup.
context (ProcessorContext, optional): Context object containing user-specific parameters. Defaults to None.
"""
def __init__(self, config, context: ProcessorContext = None):
super().__init__(config)
self._processor_context = context
================================================
FILE: packages/backend/app/processors/components/extension/generate_number_processor.py
================================================
import random
from ..node_config_builder import FieldBuilder, NodeConfigBuilder
from ...context.processor_context import ProcessorContext
from .extension_processor import (
ContextAwareExtensionProcessor,
DynamicExtensionProcessor,
)
class GenerateNumberProcessor(
ContextAwareExtensionProcessor, DynamicExtensionProcessor
):
processor_type = "generate-number-processor"
def __init__(self, config, context: ProcessorContext):
super().__init__(config, context)
def get_node_config(self):
min_field = (
FieldBuilder()
.set_name("min")
.set_label("Min")
.set_type("numericfield")
.set_description("minimumValueForTheRandomNumber")
.set_default_value(0)
.build()
)
max_field = (
FieldBuilder()
.set_name("max")
.set_label("Max")
.set_type("numericfield")
.set_description("maximumValueForTheRandomNumber")
.set_default_value(1000)
.build()
)
return (
NodeConfigBuilder()
.set_node_name("Generate Number")
.set_processor_type(self.processor_type)
.set_section("tools")
.set_help_message("generateNumberHelp")
.set_show_handles(True)
.add_field(min_field)
.add_field(max_field)
.set_output_type("text")
.set_icon("GiPerspectiveDiceSix")
.build()
)
def process(self):
# Retrieve optional parameters; default values are used if they are not provided.
min_val = self.get_input_by_name("min")
max_val = self.get_input_by_name("max")
try:
min_val = int(min_val) if min_val is not None else 0
max_val = int(max_val) if max_val is not None else 500
except ValueError:
raise ValueError("Both 'min' and 'max' should be valid numbers")
if min_val > max_val:
raise ValueError("'min' should not be greater than 'max'")
# Generate and return a random number in the inclusive range [min_val, max_val]
random_number = random.randint(min_val, max_val)
return [random_number]
def cancel(self):
pass
================================================
FILE: packages/backend/app/processors/components/extension/gpt_image_processor.py
================================================
import base64
import mimetypes
import os
import re
from datetime import datetime
from io import BytesIO
from urllib.parse import unquote, urlparse
import requests
from openai import OpenAI
from ...context.processor_context import ProcessorContext
from ..model import Field, NodeConfig, Option
from ..node_config_builder import NodeConfigBuilder
from .extension_processor import (
ContextAwareExtensionProcessor,
DynamicExtensionProcessor,
)
class GPTImageProcessor(ContextAwareExtensionProcessor, DynamicExtensionProcessor):
processor_type = "gpt-image-processor"
# our two modes
methods = ["generate", "edit"]
def __init__(self, config, context: ProcessorContext):
super().__init__(config, context)
self.method = self.get_input_by_name("method")
def get_node_config(self):
# top-level mode selector
method_options = [
Option(default=(m == "generate"), value=m, label=m.title())
for m in self.methods
]
method_field = Field(
name="method",
label="mode",
type="select",
options=method_options,
required=True,
)
return (
NodeConfigBuilder()
.set_node_name("GPT Image")
.set_processor_type(self.processor_type)
.set_icon("OpenAILogo")
.set_help_message("gptImageHelp")
.set_section("models")
.add_field(method_field)
.set_is_dynamic(True)
.build()
)
# — builders for each mode's fields —
def build_generate_config(self, builder):
# same fields as your original generate case
builder.add_field(
Field(
name="model",
label="Model",
type="select",
options=[
Option(default=True, value="gpt-image-1", label="gpt-image-1")
],
required=True,
)
)
builder.add_field(
Field(
name="prompt",
label="Prompt",
type="textarea",
required=True,
placeholder="InputTextPlaceholder",
hasHandle=True,
)
)
builder.add_field(
Field(
name="size",
label="Size",
type="select",
options=[
Option(default=True, value="auto", label="auto"),
Option(default=False, value="1024x1024", label="1024x1024"),
Option(default=False, value="1536x1024", label="1536x1024"),
Option(default=False, value="1024x1536", label="1024x1536"),
],
required=True,
)
)
builder.add_field(
Field(
name="quality",
label="Quality",
type="select",
options=[
Option(default=True, value="auto", label="auto"),
Option(default=False, value="low", label="low"),
Option(default=False, value="medium", label="medium"),
Option(default=False, value="high", label="high"),
],
required=True,
)
)
builder.add_field(
Field(
name="background",
label="Background",
type="select",
options=[
Option(default=True, value="opaque", label="opaque"),
Option(default=False, value="transparent", label="transparent"),
],
required=True,
)
)
builder.add_field(
Field(
name="moderation",
label="Moderation",
type="select",
options=[
Option(default=False, value="auto", label="auto"),
Option(default=True, value="low", label="low"),
],
required=True,
)
)
builder.set_output_type("imageUrl")
def build_edit_config(self, builder):
# same fields as your original edit case
builder.add_field(
Field(
name="model",
label="Model",
type="select",
options=[
Option(default=True, value="gpt-image-1", label="gpt-image-1")
],
required=True,
)
)
builder.add_field(
Field(
name="prompt",
label="Prompt",
type="textarea",
required=True,
placeholder="InputTextPlaceholder",
hasHandle=True,
)
)
builder.add_field(
Field(
name="mask",
label="Mask",
type="fileUpload",
hasHandle=True,
description="gptImageMaskDescription",
)
)
builder.add_field(
Field(
name="image",
label="Image",
type="fileUpload",
hasHandle=True,
canAddChildrenFields=True,
)
)
builder.set_output_type("imageUrl")
method_config_builders = {
"generate": build_generate_config,
"edit": build_edit_config,
}
def get_dynamic_node_config(self, data) -> NodeConfig:
method = data["method"]
builder = (
NodeConfigBuilder()
.set_node_name(f"GPT Image – {method.title()}")
.set_processor_type(self.processor_type)
.set_icon("OpenAILogo")
.set_section("models")
.set_show_handles(True)
)
# inject the right fields
self.method_config_builders[method](self, builder)
return builder.build()
@staticmethod
def get_image_file_from_url(url):
response = requests.get(url)
response.raise_for_status()
parsed = urlparse(url)
filename = os.path.basename(parsed.path) or "image.png"
filename = unquote(filename)
if "." not in filename:
ext = mimetypes.guess_extension(response.headers.get("Content-Type", ""))
filename += ext or ".png"
buf = BytesIO(response.content)
buf.name = filename
return buf
def process(self):
prompt = self.get_input_by_name("prompt")
model = self.get_input_by_name("model")
api_key = self._processor_context.get_value("openai_api_key")
if api_key is None:
raise Exception("No OpenAI API key found")
client = OpenAI(api_key=api_key)
if self.method == "edit":
# gather all image_* fields just like before
images_fields = [
f for f in self.fields_names if re.match(r"^image_\d+$", f)
]
images_fields.insert(0, "image")
urls = [self.get_input_by_name(fld, None) for fld in images_fields]
urls = [u for u in urls if u]
files = [GPTImageProcessor.get_image_file_from_url(u) for u in urls]
mask = self.get_input_by_name("mask", None)
if mask:
mask = GPTImageProcessor.get_image_file_from_url(mask)
result = client.images.edit(
model=model,
prompt=prompt,
image=files,
mask=mask,
)
else:
result = client.images.edit(
model=model,
prompt=prompt,
image=files,
)
else:
# generate
size = self.get_input_by_name("size")
quality = self.get_input_by_name("quality")
background = self.get_input_by_name("background")
moderation = self.get_input_by_name("moderation")
result = client.images.generate(
model=model,
prompt=prompt,
size=size,
quality=quality,
background=background,
moderation=moderation,
)
img_b64 = result.data[0].b64_json
img_bytes = base64.b64decode(img_b64)
storage = self.get_storage()
fname = f"{self.name}-{datetime.now():%Y%m%d%H%M%S%f}.png"
return storage.save(fname, img_bytes)
def cancel(self):
pass
================================================
FILE: packages/backend/app/processors/components/extension/http_get_processor.py
================================================
import logging
import requests
import json
from urllib.parse import urlparse
from ..node_config_builder import FieldBuilder, NodeConfigBuilder
from ...context.processor_context import ProcessorContext
from .extension_processor import ContextAwareExtensionProcessor
class HttpGetProcessor(ContextAwareExtensionProcessor):
processor_type = "http-get-processor"
max_timeout = 5 # Maximum timeout in seconds
max_response_size_in_mb = 2
max_response_size = (
1024 * 1024 * max_response_size_in_mb
) # Maximum response size in bytes (2 MB)
def __init__(self, config, context: ProcessorContext):
super().__init__(config, context)
def get_node_config(self):
url_field = (
FieldBuilder()
.set_name("url")
.set_label("URL")
.set_type("textfield")
.set_required(True)
.set_placeholder("httpGetProcessorURLPlaceholder")
.set_description("httpGetProcessorURLDescription")
.set_has_handle(True)
.build()
)
headers_field = (
FieldBuilder()
.set_name("headers")
.set_label("Headers")
.set_type("dictionnary")
.set_description("httpGetProcessorHeadersDescription")
.build()
)
return (
NodeConfigBuilder()
.set_node_name("HTTP Get")
.set_processor_type(self.processor_type)
.set_icon("TbHttpGet")
.set_section("input")
.set_help_message("httpGetProcessorHelp")
.set_output_type("text")
.set_show_handles(True)
.add_field(url_field)
.add_field(headers_field)
.build()
)
def convert_headers_array_to_json(self, headers_array):
headers = {}
for header in headers_array:
headers[header["key"]] = header["value"]
return json.dumps(headers)
def process(self):
url = self.get_input_by_name("url")
headers = self.get_input_by_name("headers")
timeout = self.get_input_by_name("timeout")
if not url:
raise ValueError("URL is required.")
# Validate URL to prevent misuse
parsed_url = urlparse(url)
if not parsed_url.scheme.startswith("http"):
raise ValueError("Invalid URL scheme. Only HTTP and HTTPS are allowed.")
timeout = HttpGetProcessor.max_timeout
if headers:
headers = self.convert_headers_array_to_json(headers)
try:
headers = json.loads(headers)
except json.JSONDecodeError:
raise Exception("Headers must be a valid JSON.")
else:
headers = {}
try:
response = requests.get(
url=url,
headers=headers,
timeout=timeout,
allow_redirects=False,
stream=True,
)
response.raise_for_status()
except requests.exceptions.RequestException as e:
logging.warning(f"HTTP GET request failed: {str(e)}")
raise Exception(f"HTTP GET request failed: {str(e)}")
# Limit the response size
content = bytes()
total_size = 0
try:
for chunk in response.iter_content(chunk_size=8192):
content += chunk
total_size += len(chunk)
if total_size > HttpGetProcessor.max_response_size:
logging.warning("Response size exceeds maximum allowed limit.")
raise Exception(
f"Response size exceeds maximum allowed limit of {HttpGetProcessor.max_response_size_in_mb} MB. If need to load file, consider using the file node in URL mode."
)
finally:
response.close()
content_type = response.headers.get("Content-Type", "")
if "application/json" in content_type:
try:
return [json.loads(content.decode(response.encoding or "utf-8"))]
except ValueError:
raise Exception("Failed to parse JSON response.")
else:
return content.decode(response.encoding or "utf-8", errors="replace")
def cancel(self):
pass
================================================
FILE: packages/backend/app/processors/components/extension/open_router_processor.py
================================================
import logging
from ....env_config import is_local_environment
from ...launcher.event_type import EventType
from ...launcher.processor_event import ProcessorEvent
from ...context.processor_context import ProcessorContext
from ..model import Field, NodeConfig, Option, Condition
from .extension_processor import ContextAwareExtensionProcessor
from openai import OpenAI
import requests
from cachetools import TTLCache, cached
def load_models_from_file():
import json
import os
current_dir = os.path.dirname(os.path.abspath(__file__))
models_file_path = os.path.join(
current_dir,
"..",
"..",
"..",
"..",
"resources",
"data",
"openrouter_models.json",
)
with open(models_file_path, "r") as file:
models = json.load(file)
return models.get("data", [])
@cached(TTLCache(maxsize=1, ttl=120000))
def get_models():
"""
Fetches the list of available models from OpenRouter API.
Caches the result to avoid redundant API calls.
"""
url = "https://openrouter.ai/api/v1/models"
try:
response = requests.get(url, timeout=10)
response.raise_for_status()
models = response.json()
return models.get("data", [])
except Exception as e:
logging.warning(
f"Failed to fetch OpenRouter models - Loading from file instead: {e}"
)
return load_models_from_file()
@cached(TTLCache(maxsize=1, ttl=120000))
def get_text_to_image_model_ids():
"""
Returns a list of model IDs that support text to image generation.
"""
available_models = get_models()
text_image_model_ids = [
model["id"]
for model in available_models
if model.get("architecture").get("modality") == "text+image->text"
]
return text_image_model_ids
class OpenRouterProcessor(ContextAwareExtensionProcessor):
processor_type = "openrouter-processor"
streaming = True
def __init__(self, config, context: ProcessorContext):
super().__init__(config, context)
def get_node_config(self):
context = Field(
name="context",
label="context",
type="textfield",
required=False,
placeholder="ContextPlaceholder",
hasHandle=True,
)
text = Field(
name="prompt",
label="prompt",
type="textarea",
required=True,
placeholder="PromptPlaceholder",
hasHandle=True,
)
available_models = get_models()
target_default_model_id = "google/gemma-2-9b-it:free"
model_options = [
Option(
default=(model["id"] == target_default_model_id),
value=model["id"],
label=model.get("name", model["id"]),
)
for model in available_models
]
model_field = Field(
name="model",
label="model",
type="select",
options=model_options,
required=True,
)
text_image_model_ids = get_text_to_image_model_ids()
image_url_condition = Condition(
field="model", operator="in", value=text_image_model_ids
)
image_url = Field(
name="image_url",
label="Image URL",
type="textfield",
placeholder="InputImagePlaceholder",
hasHandle=True,
condition=image_url_condition,
)
fields = [model_field, image_url, context, text]
config = NodeConfig(
nodeName="OpenRouter",
processorType=self.processor_type,
icon="OpenRouterLogo",
fields=fields,
outputType="text",
section="models",
helpMessage="openRouterHelp",
showHandlesNames=True,
)
return config
def process(self):
prompt = self.get_input_by_name("prompt")
context = self.get_input_by_name("context", "")
model = self.get_input_by_name("model")
image_url = self.get_input_by_name("image_url", None)
if prompt is None:
return None
api_key = self._processor_context.get_value("openrouter_api_key")
if api_key is None:
raise Exception("No OpenRouter API key found")
client = OpenAI(base_url="https://openrouter.ai/api/v1", api_key=api_key)
text_image_model_ids = get_text_to_image_model_ids()
if image_url is not None and model in text_image_model_ids:
content = [
{
"type": "image_url",
"image_url": {"url": image_url},
},
{"type": "text", "text": prompt},
]
else:
content = f"{context} {prompt}"
response = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": content}],
stream=self.streaming,
)
if self.streaming:
final_response = ""
for chunk in response:
if not chunk.choices[0].delta.content:
continue
final_response += chunk.choices[0].delta.content
event = ProcessorEvent(self, final_response)
self.notify(EventType.STREAMING, event)
return final_response
return response.choices[0].message.content
def cancel(self):
pass
================================================
FILE: packages/backend/app/processors/components/extension/openai_reasoning_processor.py
================================================
import logging
from app.processors.exceptions import LightException
from ...launcher.event_type import EventType
from ...launcher.processor_event import ProcessorEvent
from ...context.processor_context import ProcessorContext
from ..model import Field, FieldCondition, NodeConfig, Option
from .extension_processor import ContextAwareExtensionProcessor
from openai import OpenAI
class OpenAIReasoningProcessor(ContextAwareExtensionProcessor):
processor_type = "openai-reasoning-processor"
streaming = True
models_with_reasoning_effort = ["o3-mini", "o4-mini", "o3"]
def __init__(self, config, context: ProcessorContext):
super().__init__(config, context)
def get_node_config(self):
context = Field(
name="context",
label="context",
type="textfield",
required=False,
placeholder="ContextPlaceholder",
hasHandle=True,
)
text = Field(
name="prompt",
label="prompt",
type="textarea",
required=True,
placeholder="PromptPlaceholder",
hasHandle=True,
)
model_options = [
Option(
default=True,
value="o4-mini",
label="o4-mini",
),
Option(
default=False,
value="o3-mini",
label="o3-mini",
),
Option(
default=False,
value="o3",
label="o3",
),
Option(
default=False,
value="o1-pro",
label="o1-pro",
),
Option(
default=False,
value="o1",
label="o1",
),
]
model = Field(
name="model",
type="option",
options=model_options,
required=True,
)
reasoning_effort_options = [
Option(
default=False,
value="low",
label="low",
),
Option(
default=True,
value="medium",
label="medium",
),
Option(
default=False,
value="high",
label="high",
),
]
reasoning_effort = Field(
name="reasoning_effort",
label="reasoning_effort",
type="select",
options=reasoning_effort_options,
condition=FieldCondition(
field="model",
operator="in",
value=OpenAIReasoningProcessor.models_with_reasoning_effort,
),
)
fields = [model, context, text, reasoning_effort]
config = NodeConfig(
nodeName="OpenAI o-series",
processorType=self.processor_type,
icon="OpenAILogo",
fields=fields,
outputType="text",
section="models",
helpMessage="openaio1Help",
showHandlesNames=True,
)
return config
def handle_stream_answer(self, awnser):
event = ProcessorEvent(self, awnser)
self.notify(EventType.STREAMING, event)
def process(self):
prompt = self.get_input_by_name("prompt")
context = self.get_input_by_name("context", "")
model = self.get_input_by_name("model")
reasoning_effort = self.get_input_by_name("reasoning_effort", "medium")
if prompt is None:
return None
api_key = self._processor_context.get_value("openai_api_key")
if api_key is None:
raise Exception("No OpenAI API key found")
client = OpenAI(api_key=api_key)
kwargs = {
"model": model,
"input": [{"role": "user", "content": f"{context} {prompt}"}],
"stream": self.streaming,
}
if model in OpenAIReasoningProcessor.models_with_reasoning_effort:
kwargs["reasoning"] = {"effort": reasoning_effort}
stream = client.responses.create(**kwargs)
final_response = ""
for event in stream:
type = event.type
if type == "response.output_text.delta":
final_response += event.delta
self.handle_stream_answer(final_response)
if type == "response.completed":
response_data = event.response
final_response = response_data.output_text
if type == "response.failed":
response_data = event.response
if not hasattr(response_data, "error"):
logging.warning(f"Error from OpenAI with no data: {response_data}")
continue
raise LightException(
f"Error from OpenAI : {response_data.error.message}"
)
if type == "error":
raise LightException(f"Error from OpenAI : {event.message}")
return final_response
def cancel(self):
pass
================================================
FILE: packages/backend/app/processors/components/extension/openai_text_to_speech_processor.py
================================================
import logging
import re
from ...context.processor_context import ProcessorContext
from ..model import Field, NodeConfig, Option, Condition
from .extension_processor import ContextAwareExtensionProcessor
from openai import OpenAI
from datetime import datetime
import io
from pydub import AudioSegment
import eventlet
class OpenAITextToSpeechProcessor(ContextAwareExtensionProcessor):
processor_type = "openai-text-to-speech-processor"
def __init__(self, config, context: ProcessorContext):
super().__init__(config, context)
def get_node_config(self):
text = Field(
name="text",
label="text",
type="textfield",
required=True,
placeholder="InputTextPlaceholder",
hasHandle=True,
)
voices_options = [
Option(
default=True,
value="alloy",
label="alloy",
),
Option(
default=False,
value="ash",
label="ash",
),
Option(
default=False,
value="ballad",
label="ballad",
),
Option(
default=False,
value="coral",
label="coral",
),
Option(
default=False,
value="echo",
label="echo",
),
Option(
default=False,
value="fable",
label="fable",
),
Option(
default=False,
value="onyx",
label="onyx",
),
Option(
default=False,
value="nova",
label="nova",
),
Option(
default=False,
value="sage",
label="sage",
),
Option(
default=False,
value="shimmer",
label="shimmer",
),
]
voice = Field(
name="voice",
label="voice",
type="select",
options=voices_options,
required=True,
)
model_options = [
Option(
default=True,
value="gpt-4o-mini-tts",
label="gpt-4o-mini-tts",
),
Option(
default=False,
value="tts-1",
label="tts-1",
),
Option(
default=False,
value="tts-1-hd",
label="tts-1-hd",
),
]
model = Field(
name="model",
label="model",
type="select",
options=model_options,
required=True,
)
instructions_enabled_condition = Condition(
field="model", operator="equals", value="gpt-4o-mini-tts"
)
instructions = Field(
name="instruction",
label="instruction",
type="textfield",
required=False,
placeholder="TTSInstructionPlaceholder",
description="TTSInstructionDescription",
hasHandle=True,
condition=instructions_enabled_condition,
)
fields = [text, model, voice, instructions]
config = NodeConfig(
nodeName="TextToSpeech",
processorType=self.processor_type,
icon="OpenAILogo",
fields=fields,
outputType="audioUrl",
section="models",
helpMessage="textToSpeechHelp",
showHandlesNames=True,
keywords=["Audio", "Speech", "OpenAI", "TTS"],
)
return config
def split_text_into_chunks(text, max_length=4096):
"""
Split text into chunks of up to max_length characters by packing as many whole sentences as possible.
If a single sentence exceeds max_length, split it into smaller parts.
"""
# Split text by sentence-ending punctuation followed by whitespace.
sentences = re.split(r"(?<=[.!?])\s+", text)
chunks = []
current_sentences = []
current_length = 0
for sentence in sentences:
sentence_length = len(sentence)
# Add a space if there is already a sentence in the current chunk.
additional_length = (
sentence_length if not current_sentences else sentence_length + 1
)
if current_length + additional_length <= max_length:
# Append sentence to the current chunk.
current_sentences.append(sentence)
current_length += additional_length
else:
# Flush the current chunk if it's not empty.
if current_sentences:
chunks.append(" ".join(current_sentences))
current_sentences = []
current_length = 0
# If the sentence itself is too long, split it into parts.
if sentence_length > max_length:
parts = [
sentence[i : i + max_length]
for i in range(0, sentence_length, max_length)
]
# All full parts are separate chunks.
chunks.extend(parts[:-1])
# The last part might be less than max_length; add it to current chunk.
current_sentences = [parts[-1]]
current_length = len(parts[-1])
else:
# Start a new chunk with the sentence.
current_sentences = [sentence]
current_length = sentence_length
if current_sentences:
chunks.append(" ".join(current_sentences))
return chunks
def process(self):
text = self.get_input_by_name("text")
voice = self.get_input_by_name("voice")
model = self.get_input_by_name("model")
instruction = self.get_input_by_name("instruction", None)
if text is None:
return None
api_key = self._processor_context.get_value("openai_api_key")
if api_key is None:
raise Exception("No OpenAI API key found")
client = OpenAI(api_key=api_key)
# Split text into chunks that are each less than or equal to 4096 characters.
chunks = OpenAITextToSpeechProcessor.split_text_into_chunks(text, 4096)
pool = eventlet.GreenPool(2)
def create_audio_segment(chunk):
kwargs = {
"model": model,
"voice": voice,
"input": chunk,
}
if instruction is not None:
kwargs["instructions"] = instruction
response = client.audio.speech.create(**kwargs)
if response is None:
return None
# Convert the response content (mp3 bytes) into an AudioSegment.
return AudioSegment.from_file(io.BytesIO(response.content), format="mp3")
# Process chunks concurrently; imap preserves the order of chunks.
audio_segments = list(pool.imap(create_audio_segment, chunks))
# Filter out any None segments.
audio_segments = [segment for segment in audio_segments if segment is not None]
if not audio_segments:
return None
# Merge the audio segments.
merged_audio = audio_segments[0]
for seg in audio_segments[1:]:
merged_audio += seg
# Export merged audio to a bytes buffer.
merged_audio_buffer = io.BytesIO()
merged_audio.export(merged_audio_buffer, format="mp3")
merged_audio_buffer.seek(0)
storage = self.get_storage()
timestamp_str = datetime.now().strftime("%Y%m%d%H%M%S%f")
filename = f"{self.name}-{timestamp_str}.mp3"
url = storage.save(filename, merged_audio_buffer.read())
# cleanup
merged_audio_buffer.close()
del merged_audio_buffer
del merged_audio
del audio_segments
return url
def cancel(self):
pass
================================================
FILE: packages/backend/app/processors/components/extension/replace_text_processor.py
================================================
import logging
import re
from ..node_config_builder import FieldBuilder, NodeConfigBuilder
from .extension_processor import BasicExtensionProcessor
from ..core.processor_type_name_utils import ProcessorType
class ReplaceTextProcessor(BasicExtensionProcessor):
processor_type = ProcessorType.REPLACE_TEXT
def __init__(self, config):
super().__init__(config)
def get_node_config(self):
input_text_field = (
FieldBuilder()
.set_name("input_text")
.set_label("Input Text")
.set_type("textarea")
.set_required(True)
.set_placeholder("ReplaceTextInputPlaceholder")
.set_has_handle(True)
.build()
)
search_text_field = (
FieldBuilder()
.set_name("search_text")
.set_label("Search Text")
.set_type("textfield")
.set_required(True)
.set_placeholder("ReplaceTextSearchPlaceholder")
.set_has_handle(True)
.build()
)
replacement_text_field = (
FieldBuilder()
.set_name("replacement_text")
.set_label("Replacement Text")
.set_type("textfield")
.set_required(True)
.set_placeholder("ReplaceTextReplacePlaceholder")
.set_has_handle(True)
.build()
)
replace_all_field = (
FieldBuilder()
.set_name("replace_all")
.set_label("Replace All Occurrences")
.set_type("boolean")
.set_default_value(True)
.build()
)
use_regex_field = (
FieldBuilder()
.set_name("use_regex")
.set_label("Use Regular Expression")
.set_type("boolean")
.set_default_value(False)
.build()
)
case_sensitivity_field = (
FieldBuilder()
.set_name("case_sensitivity")
.set_label("Case Sensitive")
.set_type("boolean")
.set_default_value(True)
.build()
)
return (
NodeConfigBuilder()
.set_node_name("ReplaceText")
.set_processor_type(self.processor_type.value)
.set_section("tools")
.set_help_message("replaceTextNodeHelp")
.set_show_handles(True)
.set_output_type("text")
.set_default_hide_output(False)
.add_field(input_text_field)
.add_field(search_text_field)
.add_field(replacement_text_field)
.add_field(replace_all_field)
.add_field(case_sensitivity_field)
.add_field(use_regex_field)
.set_icon("MdSwapHoriz")
.build()
)
def process(self):
input_text = self.get_input_by_name("input_text")
search_text = self.get_input_by_name("search_text")
replacement_text = self.get_input_by_name("replacement_text")
replace_all = self.get_input_by_name("replace_all")
use_regex = self.get_input_by_name("use_regex")
case_sensitivity = self.get_input_by_name("case_sensitivity")
flags = 0
if not case_sensitivity:
flags |= re.IGNORECASE
if use_regex:
try:
pattern = re.compile(search_text, flags)
count = 0 if replace_all else 1
result_text = pattern.sub(replacement_text, input_text, count=count)
except re.error as e:
logging.warning(f"Invalid regular expression: {e}")
result_text = input_text
else:
if not case_sensitivity:
escaped_search_text = re.escape(search_text)
pattern = re.compile(escaped_search_text, flags)
count = 0 if replace_all else 1
result_text = pattern.sub(replacement_text, input_text, count=count)
else:
if replace_all:
result_text = input_text.replace(search_text, replacement_text)
else:
result_text = input_text.replace(search_text, replacement_text, 1)
return [result_text]
================================================
FILE: packages/backend/app/processors/components/extension/stabilityai_generic_processor.py
================================================
import json
import logging
import os
from ..node_config_utils import get_sub_configuration
from ....utils.openapi_client import Client
from ....utils.processor_utils import (
stream_download_file_as_binary,
)
from ....utils.openapi_converter import OpenAPIConverter
from ....utils.openapi_reader import OpenAPIReader
from ..node_config_builder import FieldBuilder, NodeConfigBuilder
from ...context.processor_context import ProcessorContext
from ..model import NodeConfig, Option
from .extension_processor import (
ContextAwareExtensionProcessor,
DynamicExtensionProcessor,
)
from datetime import datetime
import re
class StabilityAIGenericProcessor(
ContextAwareExtensionProcessor, DynamicExtensionProcessor
):
processor_type = "stabilityai-generic-processor"
openapi_file_path = "./resources/openapi/stabilityai.json"
paths_denied = [
re.compile(r"/v1/"), # Contains'/v1/'
re.compile(r"/user/"), # Contains 'user'
re.compile(r"/engines/"), # Contains 'engines'
re.compile(r"/result/"), # Contains 'result'
re.compile(r"/v2alpha/"), # Contains 'v2alpha'
re.compile(r"/result"),
# Temporary
re.compile(r"/chat"), # api returns 404 for now
]
api_reader = None
all_paths_cache = None
pooling_paths_cache = None
allowed_paths_cache = None
def __init__(self, config, context: ProcessorContext):
super().__init__(config, context)
if StabilityAIGenericProcessor.allowed_paths_cache is None:
StabilityAIGenericProcessor.initialize_allowed_paths_cache()
self.api_host = os.getenv(
"STABLE_DIFFUSION_STABILITYAI_API_HOST", "https://api.stability.ai"
)
self.path = self.get_input_by_name("path")
self.initialize_api_config()
self.final_node_config = self.get_dynamic_node_config(dict(path=self.path))
@classmethod
def initialize_allowed_paths_cache(cls):
cls.api_reader = OpenAPIReader(StabilityAIGenericProcessor.openapi_file_path)
paths_names = cls.api_reader.get_all_paths_names()
cls.all_paths_cache = paths_names
cls.pooling_paths_cache = [path for path in paths_names if "/result/" in path]
cls.allowed_paths_cache = [
path
for path in paths_names
if not cls.is_path_banned(path, cls.paths_denied)
]
@staticmethod
def is_path_banned(path, denied_patterns):
return any(pattern.search(path) for pattern in denied_patterns)
@staticmethod
def get_pooling_path(path_selected):
for path in StabilityAIGenericProcessor.pooling_paths_cache:
if path.startswith(path_selected):
return path
return None
def transform_path_options_labels(options):
transformed_options = []
for option in options:
# Remove the first path element and split the rest
parts = re.sub(r"^/[^/]+/", "", option.label).split("/")
# Take the last two elements, or one if alone
if len(parts) > 1:
label = f"{parts[-2].capitalize()} - {parts[-1].replace('-', ' ').capitalize()}"
else:
label = parts[-1].replace("-", " ").capitalize()
transformed_option = Option(
default=option.default, value=option.value, label=label
)
transformed_options.append(transformed_option)
transformed_options.sort(key=lambda option: option.label)
return transformed_options
def get_node_config(self):
if StabilityAIGenericProcessor.allowed_paths_cache is None:
StabilityAIGenericProcessor.initialize_allowed_paths_cache()
path_options = [
Option(default=False, value=name, label=name)
for i, name in enumerate(StabilityAIGenericProcessor.allowed_paths_cache)
]
path_options = self.transform_path_options_labels(path_options)
path_options[0].default = True
path = (
FieldBuilder()
.set_name("path")
.set_label("Path")
.set_type("select")
.set_options(path_options)
.build()
)
return (
NodeConfigBuilder()
.set_node_name("StabilityAI")
.set_processor_type(self.processor_type)
.set_icon("StabilityAILogo")
.set_section("models")
.set_help_message("stableDiffusionPromptHelp")
.set_show_handles(True)
.add_field(path)
.set_is_dynamic(True) # Important
.build()
)
def initialize_api_config(self):
response_content_path = self.path
response_method = "post"
self.path_accept = StabilityAIGenericProcessor.api_reader.get_path_accept(
self.path, "post"
)
self.pooling_path = self.get_pooling_path(self.path)
if self.pooling_path is not None:
response_content_path = self.pooling_path
response_method = "get"
self.pooling_path_accept = (
StabilityAIGenericProcessor.api_reader.get_path_accept(
self.pooling_path, "get"
)
)
self.response_content_type = (
StabilityAIGenericProcessor.api_reader.get_response_content_type(
response_content_path, response_method
)[0]
)
print(f"Response content type {self.response_content_type}")
@staticmethod
def determine_output_type(path_accept):
if path_accept is None:
return None
elif path_accept == "video/*":
return "videoUrl"
elif "model" in path_accept:
return "3dUrl"
else:
return "imageUrl"
def get_dynamic_node_config(self, data) -> NodeConfig:
if StabilityAIGenericProcessor.allowed_paths_cache is None:
StabilityAIGenericProcessor.initialize_allowed_paths_cache()
selected_api_path = data["path"]
schema = StabilityAIGenericProcessor.api_reader.get_request_schema_for_path(
selected_api_path, "post"
)
path_accept = StabilityAIGenericProcessor.api_reader.get_path_accept(
selected_api_path, "post"
)
output_type = StabilityAIGenericProcessor.determine_output_type(path_accept)
pooling_path = self.get_pooling_path(selected_api_path)
if pooling_path is not None:
pooling_path_accept = (
StabilityAIGenericProcessor.api_reader.get_path_accept(
pooling_path, "get"
)
)
output_type = StabilityAIGenericProcessor.determine_output_type(
pooling_path_accept
)
builder = OpenAPIConverter().convert_schema_to_node_config(schema)
path_components = selected_api_path.split("/")
last_component = (
path_components[-1] if path_components[-1] else path_components[-2]
)
node_name = " ".join(word.capitalize() for word in last_component.split("-"))
(
builder.set_node_name(f"StabilityAI - {node_name}")
.set_processor_type(self.processor_type)
.set_icon("StabilityAILogo")
.set_section("models")
.set_help_message("stableDiffusionPromptHelp")
.set_show_handles(True)
)
if output_type is not None:
builder.set_output_type(output_type)
return builder.build()
def perform_pooling(self, client, path):
return client.pooling(path=path, accept=self.pooling_path_accept)
def prepare_and_process_response(self, response):
storage = self.get_storage()
timestamp_str = datetime.now().strftime("%Y%m%d%H%M%S%f")
extension = self.get_input_by_name("output_format")
if extension:
filename = f"{self.name}-{timestamp_str}.{extension}"
else:
if "gltf-binary" in self.response_content_type:
extension = "glb"
else:
extension = self.response_content_type.split("/")[-1]
filename = f"{self.name}-{timestamp_str}.{extension}"
url = storage.save(filename, response)
return url
def get_fields_from_config(self):
if self.final_node_config is None:
return []
if isinstance(self.final_node_config, NodeConfig):
return self.final_node_config.fields
discriminators_values = []
for discriminator_name in self.final_node_config.discriminatorFields:
value = self.get_input_by_name(discriminator_name)
discriminators_values.append(value)
corresponding_config = get_sub_configuration(
discriminators_values, self.final_node_config
)
if corresponding_config is None:
return []
return corresponding_config.config.fields
def quick_filter(self, data):
if "mode" in data:
if data["mode"] == "image-to-image":
if "aspect_ratio" in data:
del data["aspect_ratio"]
if data["mode"] == "text-to-image":
if "strength" in data:
del data["strength"]
if "image" in data:
del data["image"]
def process(self):
api_key = self._processor_context.get_value("stabilityai_api_key")
fields = self.get_fields_from_config()
data = {field.name: self.get_input_by_name(field.name) for field in fields}
self.quick_filter(data)
binaryFieldNames = [field.name for field in fields if field.isBinary]
files = {} if len(binaryFieldNames) > 0 else {"none": (None, "")}
for field_name in binaryFieldNames:
if field_name not in data:
files[field_name] = None
continue
url = data[field_name]
data[field_name] = None
del data[field_name]
if url:
files[field_name] = stream_download_file_as_binary(url)
else:
files[field_name] = None
client = Client(
api_token=api_key,
base_url=self.api_host,
)
response = client.post(
path=self.path, data=data, files=files, accept=self.path_accept
)
if self.pooling_path:
response_str = response.decode("utf-8")
response_json = json.loads(response_str)
key_name = "id"
key_value = response_json[key_name]
updated_pooling_path = self.pooling_path.replace(
"{" + key_name + "}", str(key_value)
)
response = self.perform_pooling(client, updated_pooling_path)
return self.prepare_and_process_response(response)
def cancel(self):
pass
================================================
FILE: packages/backend/app/processors/components/extension/stable_diffusion_three_processor.py
================================================
import logging
import os
import requests
from ..node_config_builder import FieldBuilder, NodeConfigBuilder
from ...context.processor_context import ProcessorContext
from ..model import Option
from .extension_processor import ContextAwareExtensionProcessor
from datetime import datetime
class StableDiffusionThreeProcessor(ContextAwareExtensionProcessor):
processor_type = "stabilityai-stable-diffusion-3-processor"
def __init__(self, config, context: ProcessorContext):
super().__init__(config, context)
self.api_host = os.getenv(
"STABLE_DIFFUSION_STABILITYAI_API_HOST", "https://api.stability.ai"
)
def get_node_config(self):
prompt = (
FieldBuilder()
.set_name("prompt")
.set_label("Prompt")
.set_type("textfield")
.set_required(True)
.set_placeholder("GenericPromptPlaceholder")
.set_has_handle(True)
.build()
)
negative_prompt = (
FieldBuilder()
.set_name("negative_prompt")
.set_label("Negative Prompt")
.set_type("textfield")
.set_placeholder("GenericNegativePromptPlaceholder")
.set_has_handle(True)
.build()
)
model_options = [
Option(
default=True, value="sd3.5-large", label="Stable Diffusion 3.5 Large"
),
Option(
default=False,
value="sd3.5-large-turbo",
label="Stable Diffusion 3.5 Large Turbo",
),
Option(default=False, value="sd3-large", label="Stable Diffusion 3 Large"),
Option(
default=False, value="sd3-medium", label="Stable Diffusion 3 Medium"
),
Option(
default=False,
value="sd3-large-turbo",
label="Stable Diffusion 3 Large Turbo",
),
]
model = (
FieldBuilder()
.set_name("model")
.set_label("Model")
.set_type("select")
.set_options(model_options)
.build()
)
aspect_ratio_options = [
Option(default=True, value="1:1", label="1:1"),
Option(default=False, value="16:9", label="16:9"),
Option(default=False, value="3:2", label="3:2"),
Option(default=False, value="2:3", label="2:3"),
Option(default=False, value="4:5", label="4:5"),
Option(default=False, value="5:4", label="5:4"),
Option(default=False, value="9:16", label="9:16"),
Option(default=False, value="9:21", label="9:21"),
Option(default=False, value="21:9", label="21:9"),
]
aspect_ratio = (
FieldBuilder()
.set_name("aspect_ratio")
.set_label("Aspect Ratio")
.set_type("select")
.set_options(aspect_ratio_options)
.build()
)
seed = (
FieldBuilder()
.set_name("seed")
.set_label("Seed")
.set_type("numericfield")
.set_placeholder("Enter a numeric seed")
.set_default_value(0)
.set_has_handle(True)
.build()
)
return (
NodeConfigBuilder()
.set_node_name("Stable Diffusion 3.5")
.set_processor_type(self.processor_type)
.set_icon("StabilityAILogo")
.set_section("models")
.set_help_message("stableDiffusionPromptHelp")
.set_output_type("imageUrl")
.set_show_handles(True)
.add_field(prompt)
.add_field(negative_prompt)
.add_field(model)
.add_field(aspect_ratio)
.add_field(seed)
.build()
)
def process(self):
prompt = self.get_input_by_name("prompt")
model = self.get_input_by_name("model")
seed = self.get_input_by_name("seed")
aspect_ratio = self.get_input_by_name("aspect_ratio")
negative_prompt = self.get_input_by_name("negative_prompt")
if prompt is None:
return None
api_key = self._processor_context.get_value("stabilityai_api_key")
data_to_send = {
"prompt": prompt,
"negative_prompt": negative_prompt if model != "sd3-turbo" else None,
"model": model,
"seed": seed,
"aspect_ratio": aspect_ratio,
}
response = requests.post(
f"{self.api_host}/v2beta/stable-image/generate/sd3",
headers={
"Accept": "image/*",
"Authorization": f"Bearer {api_key}",
},
files={"none": ""},
data=data_to_send,
)
return self.prepare_and_process_response(response)
def prepare_and_process_response(self, response):
if response.status_code != 200:
logging.warning(
f"API call to StabilityAI failed with status {response.status_code}: {response.text}"
)
logging.warning("User prompt : " + self.get_input_by_name("prompt") or "")
raise Exception(f"Error message from StabilityAI : \n {response.text}")
storage = self.get_storage()
timestamp_str = datetime.now().strftime("%Y%m%d%H%M%S%f")
filename = f"{self.name}-{timestamp_str}.png"
url = storage.save(filename, response.content)
return url
def cancel(self):
pass
================================================
FILE: packages/backend/app/processors/components/model.py
================================================
# generated by datamodel-codegen:
# filename: schema.json
# timestamp: 2025-05-26T04:44:24+00:00
from __future__ import annotations
from typing import Any, Dict, List, Optional, Union
from pydantic import BaseModel, RootModel
from typing_extensions import Literal
class Model(RootModel[Any]):
root: Any
class FieldType(
RootModel[
Literal[
'boolean',
'dictionnary',
'fileUpload',
'imageMaskCreator',
'input',
'inputInt',
'inputNameBar',
'json',
'list',
'nonRendered',
'numericfield',
'option',
'select',
'slider',
'switch',
'textToDisplay',
'textarea',
'textfield',
]
]
):
root: Literal[
'boolean',
'dictionnary',
'fileUpload',
'imageMaskCreator',
'input',
'inputInt',
'inputNameBar',
'json',
'list',
'nonRendered',
'numericfield',
'option',
'select',
'slider',
'switch',
'textToDisplay',
'textarea',
'textfield',
]
class Operator(
RootModel[
Literal[
'equals',
'exists',
'greater than',
'in',
'less than',
'not equals',
'not exists',
'not in',
]
]
):
root: Literal[
'equals',
'exists',
'greater than',
'in',
'less than',
'not equals',
'not exists',
'not in',
]
class Option(BaseModel):
default: Optional[bool] = None
label: Optional[str] = None
value: Optional[str] = None
class OutputType(
RootModel[
Literal[
'3dUrl',
'audioUrl',
'fileUrl',
'imageBase64',
'imageUrl',
'markdown',
'pdfUrl',
'text',
'videoUrl',
]
]
):
root: Literal[
'3dUrl',
'audioUrl',
'fileUrl',
'imageBase64',
'imageUrl',
'markdown',
'pdfUrl',
'text',
'videoUrl',
]
class SectionType(RootModel[Literal['image-generation', 'input', 'models', 'tools']]):
root: Literal['image-generation', 'input', 'models', 'tools']
class Condition(BaseModel):
field: Optional[str] = None
operator: Optional[Operator] = None
value: Optional[Any] = None
class ConditionGroup(BaseModel):
conditions: Optional[List[Condition]] = None
logic: Optional[Literal['AND', 'OR']] = None
class FieldCondition(RootModel[Union[Condition, ConditionGroup]]):
root: Union[Condition, ConditionGroup]
class OmitNodeConfigFieldsOutputType(BaseModel):
defaultHideOutput: Optional[bool] = None
hasInputHandle: Optional[bool] = None
helpMessage: Optional[str] = None
hideFieldsIfParent: Optional[bool] = None
icon: Optional[str] = None
inputNames: Optional[List[str]] = None
isBeta: Optional[bool] = None
isDynamicallyGenerated: Optional[bool] = None
nodeName: Optional[str] = None
processorType: Optional[str] = None
section: Optional[SectionType] = None
showHandlesNames: Optional[bool] = None
class Field(BaseModel):
allowDecimal: Optional[bool] = None
associatedField: Optional[str] = None
canAddChildrenFields: Optional[bool] = None
condition: Optional[FieldCondition] = None
defaultValue: Optional[Any] = None
description: Optional[str] = None
hasHandle: Optional[bool] = None
hidden: Optional[bool] = None
hideIfParent: Optional[bool] = None
isBinary: Optional[bool] = None
isChild: Optional[bool] = None
isLinked: Optional[bool] = None
label: Optional[str] = None
max: Optional[float] = None
min: Optional[float] = None
name: Optional[str] = None
options: Optional[List[Option]] = None
placeholder: Optional[str] = None
required: Optional[bool] = None
step: Optional[float] = None
type: Optional[FieldType] = None
withModalEdit: Optional[bool] = None
class NodeConfig(BaseModel):
defaultHideOutput: Optional[bool] = None
fields: Optional[List[Field]] = None
hasInputHandle: Optional[bool] = None
helpMessage: Optional[str] = None
hideFieldsIfParent: Optional[bool] = None
icon: Optional[str] = None
inputNames: Optional[List[str]] = None
isBeta: Optional[bool] = None
isDynamicallyGenerated: Optional[bool] = None
nodeName: Optional[str] = None
outputType: Optional[OutputType] = None
processorType: Optional[str] = None
section: Optional[SectionType] = None
showHandlesNames: Optional[bool] = None
class DiscriminatedNodeConfig(BaseModel):
config: Optional[NodeConfig] = None
discriminators: Optional[Dict[str, str]] = None
class NodeSubConfig(BaseModel):
discriminatorFields: Optional[List[str]] = None
subConfigurations: Optional[List[DiscriminatedNodeConfig]] = None
class NodeConfigVariant(NodeSubConfig, OmitNodeConfigFieldsOutputType):
pass
================================================
FILE: packages/backend/app/processors/components/node_config_builder.py
================================================
from typing import Dict, List, Optional, Union
from .model import (
DiscriminatedNodeConfig,
Field,
FieldType,
NodeConfig,
NodeConfigVariant,
Option,
OutputType,
SectionType,
)
class BaseNodeConfigBuilder:
def __init__(self):
self.nodeName: Optional[str] = None
self.processorType: Optional[str] = None
self.icon: Optional[str] = None
self.outputType: Optional[str] = None
self.section: Optional[str] = None
self.helpMessage: Optional[str] = None
self.showHandlesNames: Optional[bool] = False
self.isBeta: Optional[bool] = False
self.defaultHideOutput: Optional[bool] = False
def set_node_name(self, name: str) -> "BaseNodeConfigBuilder":
self.nodeName = name
return self
def set_processor_type(self, processor_type: str) -> "BaseNodeConfigBuilder":
self.processorType = processor_type
return self
def set_icon(self, icon: str) -> "BaseNodeConfigBuilder":
self.icon = icon
return self
def set_output_type(self, output_type: str) -> "BaseNodeConfigBuilder":
self.outputType = OutputType(root=output_type)
return self
def set_section(self, section: str) -> "BaseNodeConfigBuilder":
self.section = SectionType(root=section)
return self
def set_help_message(self, help_message: str) -> "BaseNodeConfigBuilder":
self.helpMessage = help_message
return self
def set_show_handles(self, show: bool) -> "BaseNodeConfigBuilder":
self.showHandlesNames = show
return self
def set_is_beta(self, beta: bool) -> "NodeConfigBuilder":
self.isBeta = beta
return self
def set_default_hide_output(self, hide: bool) -> "NodeConfigBuilder":
self.defaultHideOutput = hide
return self
class NodeConfigBuilder(BaseNodeConfigBuilder):
def __init__(self):
super().__init__()
self.fields: List[Field] = []
self.isDynamicallyGenerated: Optional[bool] = False
self.discriminators: Optional[Dict[str, str]] = None
def set_is_dynamic(self, dyna: bool) -> "NodeConfigBuilder":
self.isDynamicallyGenerated = dyna
return self
def set_fields(self, fields: List[Field]) -> "NodeConfigBuilder":
self.fields = fields
return self
def add_field(self, field: Field) -> "NodeConfigBuilder":
self.fields.append(field)
return self
def add_discriminator(self, key, value) -> "NodeConfigBuilder":
if self.discriminators is None:
self.discriminators = {}
self.discriminators[key] = value
return self
def build(self) -> NodeConfig:
baseConfig = NodeConfig(
nodeName=self.nodeName,
processorType=self.processorType,
icon=self.icon,
fields=self.fields,
outputType=self.outputType,
section=self.section,
helpMessage=self.helpMessage,
showHandlesNames=self.showHandlesNames,
isDynamicallyGenerated=self.isDynamicallyGenerated,
isBeta=self.isBeta,
defaultHideOutput=self.defaultHideOutput,
)
if self.discriminators is not None:
return DiscriminatedNodeConfig(
config=baseConfig, discriminators=self.discriminators
)
else:
return baseConfig
class NodeConfigVariantBuilder(BaseNodeConfigBuilder):
def __init__(self):
super().__init__()
self.subConfigurations: List[NodeConfig] = []
self.discriminatorFields: Optional[List[str]] = []
def add_discriminator_field(self, field: str) -> "NodeConfigVariantBuilder":
if self.discriminatorFields is None:
self.discriminatorFields = []
self.discriminatorFields.append(field)
return self
def add_sub_configuration(
self, sub_configuration: NodeConfig
) -> "NodeConfigVariantBuilder":
self.subConfigurations.append(sub_configuration)
return self
def build(self) -> NodeConfigVariant:
for subConfig in self.subConfigurations:
config = subConfig.config
config.showHandlesNames = self.showHandlesNames
config.icon = self.icon
config.nodeName = self.nodeName
config.outputType = self.outputType
config.section = self.section
config.processorType = self.processorType
config.helpMessage = self.helpMessage
return NodeConfigVariant(
subConfigurations=self.subConfigurations,
discriminatorFields=self.discriminatorFields,
)
class FieldBuilder:
def __init__(self):
self._field = Field()
def set_name(self, name: str) -> "FieldBuilder":
self._field.name = name
return self
def set_label(self, label: str) -> "FieldBuilder":
self._field.label = label
return self
def set_description(self, description: str) -> "FieldBuilder":
self._field.description = description
return self
def set_type(self, field_type: str) -> "FieldBuilder":
self._field.type = FieldType(root=field_type)
return self
def set_min(self, min: float) -> "FieldBuilder":
self._field.min = min
return self
def set_max(self, max: float) -> "FieldBuilder":
self._field.max = max
return self
def set_is_binary(self, binary: bool) -> "FieldBuilder":
self._field.isBinary = binary
return self
def set_placeholder(self, placeholder: str) -> "FieldBuilder":
self._field.placeholder = placeholder
return self
def set_required(self, required: bool) -> "FieldBuilder":
self._field.required = required
return self
def set_options(self, options: List[Option]) -> "FieldBuilder":
self._field.options = options
return self
def add_option(self, option: Option) -> "FieldBuilder":
if not self._field.options:
self._field.options = []
self._field.options.append(option)
return self
def set_default_value(self, default_value: Union[str, float]) -> "FieldBuilder":
self._field.defaultValue = default_value
return self
def set_has_handle(self, has_handle: bool) -> "FieldBuilder":
self._field.hasHandle = has_handle
return self
def build(self) -> Field:
return self._field
================================================
FILE: packages/backend/app/processors/components/node_config_utils.py
================================================
from .model import NodeConfigVariant
def get_sub_configuration(discriminators_values, node_config: NodeConfigVariant):
for subconfig in node_config.subConfigurations:
subconfig_discriminator_values = [
subconfig.discriminators[discriminator]
for discriminator in subconfig.discriminators
]
if subconfig_discriminator_values == discriminators_values:
return subconfig
================================================
FILE: packages/backend/app/processors/components/processor.py
================================================
from abc import ABC, abstractmethod
import json
import logging
from typing import Any, List, Optional, TypedDict, Union, Dict
from ..launcher.processor_event import ProcessorEvent
from ..launcher.event_type import EventType
from ..observer.observer import Observer
from .core.processor_type_name_utils import ProcessorType
from ...storage.storage_strategy import StorageStrategy
from ..context.processor_context import ProcessorContext
class BadKeyInputIndex(Exception):
"""Exception raised for index out of bounds in the output list."""
def __init__(self, message="This input key does not exists"):
self.message = message
super().__init__(self.message)
class InputItem(TypedDict, total=False):
inputName: Optional[str]
inputNode: str
inputNodeOutputKey: int
class Processor(ABC):
processor_type: Optional["ProcessorType"] = None
"""The type of the processor"""
observers: List[Observer] = []
"""The observers of the processor"""
storage_strategy: Optional["StorageStrategy"]
"""The storage strategy used by the processor"""
_processor_context: Optional["ProcessorContext"]
"""The context data of the processor"""
name: str
"""The name of the processor"""
_output: Optional[Any]
"""The output of the processor"""
inputs: Optional[List[InputItem]]
"""A list of inputs accepted by the processor."""
input_processors: List["Processor"]
"""The processors set as inputs"""
is_processing: bool
"""Flag indicating if the processor has started working, useful when using API with cold start"""
is_finished: bool
"""Flag indicating if the processor's has produced his output"""
_has_dynamic_behavior: bool
"""Flag indicating if the processor's behavior and execution time are unpredictable and subject to change at runtime."""
def __init__(self, config: Dict[str, Any]) -> None:
self.name = config["name"]
self.processor_type = config["processorType"]
self.observers = []
self._output = None
self.inputs = None
self._processor_context = None
self.input_processors = []
self.storage_strategy = None
self.is_finished = False
self._has_dynamic_behavior = False
self._config = config
if (
config.get("config") is not None
and config.get("config").get("fields") is not None
and config.get("config").get("fields") != []
):
self.fields = config.get("config").get("fields")
self.fields_names = [field["name"] for field in self.fields]
if config.get("inputs") is not None and config.get("inputs") != []:
self.inputs = config.get("inputs")
def cleanup(self) -> None:
self.input_processors = None
self._processor_context = None
self._output = None
self.storage_strategy = None
def process_and_update(self):
output = self.process()
if output is not None:
self.set_output(output)
return output
@abstractmethod
def process(self):
pass
@abstractmethod
def cancel(self) -> None:
pass
def add_observer(self, observer):
self.observers.append(observer)
def remove_observer(self, observer):
self.observers.remove(observer)
if len(self.observers) == 0:
self.observers = None
return self.observers
def notify(self, event: EventType, data: ProcessorEvent):
for observer in self.observers:
observer.notify(event, data)
def get_output(self, input_key=None) -> Optional[str]:
output = getattr(self, "_output", None)
if output is not None and isinstance(output, list) and len(output) > 0:
if input_key is not None:
if input_key < 0 or input_key >= len(output):
logging.warning(
f"Index {input_key} out of bounds for output of size {len(output)}."
)
return None
return output[input_key]
else:
return output
return None
def set_output(self, value: Union[List, str]) -> None:
if isinstance(value, list):
self._output = value
elif isinstance(value, str):
self._output = [value]
else:
raise TypeError("Value should be either a list or a string.")
self.is_finished = True
def get_inputs(self) -> Optional[List[InputItem]]:
return self.inputs
def get_input_processor(self) -> Optional["Processor"]:
if self.input_processors is None or len(self.input_processors) == 0:
return None
return self.input_processors[0]
def get_input_processors(self) -> List["Processor"]:
return self.input_processors
def get_input_node_output_key(self) -> Optional[int]:
if self.inputs is None or len(self.inputs) == 0:
return None
if self.inputs[0].get("inputNodeOutputKey") is None:
return 0
return self.inputs[0].get("inputNodeOutputKey")
def get_input_node_output_key_by_node_name(
self, input_node_name: str
) -> Optional[int]:
keys = []
for input in self.inputs:
if input.get("inputNode") == input_node_name:
keys.append(input.get("inputNodeOutputKey"))
return keys
def get_input_node_output_keys(self) -> Optional[List[int]]:
if self.inputs is None or len(self.inputs) == 0:
return None
return [input.get("inputNodeOutputKey") for input in self.inputs]
def get_input_names(self) -> Optional[List[str]]:
if self.inputs is None or len(self.inputs) == 0:
return None
return [input.get("inputName") for input in self.inputs]
def get_input_names_from_config(self) -> Optional[List[str]]:
return self._config.get("config").get("inputNames")
def get_input_by_name(
self, name: str, default=None, accept_object=False
) -> Optional[InputItem]:
input = self._config.get(name, default)
input_processors = self.get_input_processors()
input_output_keys = self.get_input_node_output_keys()
input_names = self.get_input_names()
if input_processors:
for processor, input_name, key in zip(
input_processors, input_names, input_output_keys
):
if input_name == name:
input_processor_output = processor.get_output(key)
if (
isinstance(input_processor_output, dict)
or isinstance(input_processor_output, list)
and not accept_object
):
input_processor_output = json.dumps(input_processor_output)
return input_processor_output
return input
def add_input_processor(self, input_processor: "Processor") -> None:
self.input_processors.append(input_processor)
def set_storage_strategy(self, storage_strategy: "StorageStrategy") -> None:
self.storage_strategy = storage_strategy
def __str__(self) -> str:
return f"Processor(name={self.name}, type={self.processor_type})"
def get_context(self) -> Optional["ProcessorContext"]:
return self._processor_context
def get_storage(self) -> Optional["StorageStrategy"]:
return self.storage_strategy
def has_dynamic_behavior(self) -> bool:
return self._has_dynamic_behavior
class BasicProcessor(Processor):
def __init__(self, config):
super().__init__(config)
def cancel(self):
pass
class ContextAwareProcessor(Processor):
def __init__(self, config, context: ProcessorContext = None):
super().__init__(config)
self._processor_context = context
================================================
FILE: packages/backend/app/processors/context/processor_context.py
================================================
from abc import ABC, abstractmethod
from typing import Optional
from typing import List
class ProcessorContext(ABC):
@abstractmethod
def get_context(self) -> "ProcessorContext":
pass
@abstractmethod
def get_current_user_id(self) -> Optional[str]:
pass
@abstractmethod
def get_session_id(self) -> Optional[str]:
pass
@abstractmethod
def get_parameter_names(self) -> List[str]:
"""
List all the parameter names currently stored in the context.
Returns:
A list of parameter names.
"""
pass
@abstractmethod
def get_value(self, name) -> Optional[str]:
"""
Retrieve the value associated with the specified parameter name.
Returns:
The value of the parameter if found, otherwise None.
"""
pass
================================================
FILE: packages/backend/app/processors/context/processor_context_flask_request.py
================================================
from typing import List, Optional
from ...flask.utils.constants import SESSION_USER_ID_KEY
from .processor_context import ProcessorContext
from copy import deepcopy
class ProcessorContextFlaskRequest(ProcessorContext):
parameter_prefix = "session_"
def __init__(self, g_context=None, session_data=None, session_id=None):
self.g_context = deepcopy(g_context) if g_context is not None else {}
self.session_data = deepcopy(session_data) if session_data is not None else {}
self.session_id = deepcopy(session_id) if session_id is not None else None
def get_context(self) -> "ProcessorContext":
"""Retrieve the stored Flask global context."""
return self.g_context
def get_current_user_id(self) -> str:
"""Retrieve the current user ID from the stored session data."""
return self.session_data.get(SESSION_USER_ID_KEY)
def get_session_id(self) -> str:
return self.session_id
def get_parameter_names(self) -> List[str]:
return [
key.replace(self.parameter_prefix, "")
for key in dir(self.g_context)
if not key.startswith("_") and key not in dir(type(self.g_context))
]
def get_value(self, name) -> Optional[str]:
return self.g_context.get(self.parameter_prefix + name)
================================================
FILE: packages/backend/app/processors/exceptions.py
================================================
class LightException(Exception):
def __init__(
self,
message: str,
langvar_message: str = "LightException",
langvar_values: dict = None,
):
self.message = message
self.langvar_message = langvar_message
self.langvar_values = langvar_values
super().__init__(f"{message}")
================================================
FILE: packages/backend/app/processors/factory/processor_factory.py
================================================
from abc import ABC, abstractmethod
from ...storage.storage_strategy import StorageStrategy
from ..context.processor_context import ProcessorContext
class ProcessorFactory(ABC):
@abstractmethod
def create_processor(
self,
config,
context: ProcessorContext = None,
storage_strategy: StorageStrategy = None,
):
pass
@abstractmethod
def load_processors(self):
pass
================================================
FILE: packages/backend/app/processors/factory/processor_factory_iter_modules.py
================================================
from enum import Enum
import importlib
import logging
import pkgutil
import inspect
from ..components.processor import Processor
from .processor_factory import ProcessorFactory
from injector import singleton
@singleton
class ProcessorFactoryIterModules(ProcessorFactory):
def __init__(self):
self._processors = {}
def register_processor(self, processor_type, processor_class):
self._processors[processor_type] = processor_class
def create_processor(self, config, context_data=None, storage_strategy=None):
processor_type = config["processorType"]
processor_class = self._processors.get(processor_type)
if not processor_class:
raise ValueError(f"Processor type '{processor_type}' not supported")
params = inspect.signature(processor_class.__init__).parameters
context_param = params.get("context")
processor = None
if context_param is not None:
processor = processor_class(config=config, context=context_data)
else:
processor = processor_class(config=config)
processor.set_storage_strategy(storage_strategy)
return processor
def load_processors(self):
self._load_recursive("app.processors.components")
def _load_recursive(self, package_name):
package = importlib.import_module(package_name)
prefix = package.__name__ + "."
for importer, module_name, is_pkg in pkgutil.iter_modules(
package.__path__, prefix
):
if is_pkg:
self._load_recursive(module_name)
else:
module = __import__(module_name, fromlist="dummy")
for attribute_name in dir(module):
attribute = getattr(module, attribute_name)
if isinstance(attribute, type) and issubclass(attribute, Processor):
if attribute.processor_type is not None:
processor_type_key = (
attribute.processor_type.value
if isinstance(attribute.processor_type, Enum)
else attribute.processor_type
)
self.register_processor(processor_type_key, attribute)
================================================
FILE: packages/backend/app/processors/launcher/abstract_topological_processor_launcher.py
================================================
from abc import abstractmethod
import json
import logging
from typing import List
from injector import inject
from .processor_launcher import ProcessorLauncher
from .event_type import EventType
from .processor_launcher_event import ProcessorLauncherEvent
from ..context.processor_context import ProcessorContext
from ..observer.observer import Observer
from ...storage.storage_strategy import StorageStrategy
from ..factory.processor_factory import ProcessorFactory
class AbstractTopologicalProcessorLauncher(ProcessorLauncher):
"""
Basic Processor Launcher emiting event through flask_socketio websockets
A class that launches processors based on configuration data.
"""
processor_factory: ProcessorFactory
storage_strategy: StorageStrategy
observers: List[Observer]
context: ProcessorContext
@inject
def __init__(
self,
processor_factory: ProcessorFactory,
storage_strategy: StorageStrategy,
observers: List[Observer] = None,
) -> None:
self.processor_factory = processor_factory
self.storage_strategy = storage_strategy
self.processor_factory.load_processors()
self.observers = observers or []
self.context = None
def set_context(self, context: ProcessorContext):
self.context = context
def add_observer(self, observer):
self.observers.append(observer)
def _load_config_data(self, fileName):
with open(fileName, "r") as file:
config_data = json.load(file)
return config_data
def _link_processors(self, processors):
for processor in processors.values():
if hasattr(processor, "inputs") and processor.inputs is not None:
for input in processor.inputs:
input_processor = processors.get(input.get("inputNode"))
if not input_processor:
logging.error(
f"Link_processors - processor name : '{processor.name}' - input_processor : '{input.get('inputNode')}'"
)
raise ValueError(
f"Input processor '{input.get('inputNode')}' not found"
)
processor.add_input_processor(input_processor)
def load_processors(self, config_data):
processors = {
config["name"]: self.processor_factory.create_processor(
config, self.context, self.storage_strategy
)
for config in config_data
}
self._link_processors(processors)
return processors
def get_node_by_name(self, config_data, node_name):
"""
Retrieves a node by its name from the available nodes.
Parameters:
config_data (list): A list of dictionaries containing the configuration data for each processor.
node_name (str): The name of the node to find.
Returns:
The node with the given name if found, otherwise None.
"""
for node in config_data:
if node.get("name") == node_name:
return node
return None
def notify_error(self, processor, e):
error_event_data = ProcessorLauncherEvent(
instance_name=processor.name,
user_id=self.context.get_current_user_id(),
processor=processor,
error=e,
session_id=self.context.get_session_id(),
processor_type=processor.processor_type,
)
self.notify_observers(EventType.ERROR.value, error_event_data)
def notify_streaming(self, processor, output, isDone=False, duration=0):
streaming_event_data = ProcessorLauncherEvent(
instance_name=processor.name,
user_id=self.context.get_current_user_id(),
output=output,
processor=processor,
isDone=isDone,
processor_type=processor.processor_type,
session_id=self.context.get_session_id(),
duration=duration,
)
self.notify_observers(EventType.STREAMING.value, streaming_event_data)
def notify_progress(self, processor, output, isDone=False, duration=0):
progress_event_data = ProcessorLauncherEvent(
instance_name=processor.name,
user_id=self.context.get_current_user_id(),
output=output,
processor=processor,
isDone=isDone,
processor_type=processor.processor_type,
session_id=self.context.get_session_id(),
duration=duration,
)
self.notify_observers(EventType.PROGRESS.value, progress_event_data)
def notify_current_node_running(self, processor):
current_node_running_event_data = ProcessorLauncherEvent(
instance_name=processor.name,
user_id=self.context.get_current_user_id(),
processor=processor,
session_id=self.context.get_session_id(),
processor_type=processor.processor_type,
)
self.notify_observers(
EventType.CURRENT_NODE_RUNNING.value, current_node_running_event_data
)
def load_required_processors(self, config_data, node_name):
"""
Loads the necessary processors based on the given configuration data and node name.
Parameters:
config_data (list): A list of dictionaries containing the configuration data for each processor.
node_name (str): The name of the node being processed.
Returns:
dict: A dictionary mapping processor names to their respective instances.
The function operates as follows:
- Iterates over each configuration in config_data.
- Creates a new processor instance based on the configuration.
- If outputData is not None and differs from node_name, the processor's output is set accordingly.
- Stores each processor instance in a dictionary with its name as the key.
"""
processors = {}
node = self.get_node_by_name(config_data, node_name)
if node and not node.get("inputs"):
processor = self.processor_factory.create_processor(
node, self.context, self.storage_strategy
)
processors[node["name"]] = processor
logging.debug(f"Created single processor for node - {node_name}")
else:
related_config_data = self.get_related_config_data(
config_data, node_name, []
)
related_config_data.reverse()
for config in related_config_data:
config_output = config.get("outputData", None)
if config_output is None or config["name"] == node_name:
logging.debug(f"Empty or current node - {config['name']}")
processor = self.processor_factory.create_processor(
config, self.context, self.storage_strategy
)
processors[config["name"]] = processor
else:
logging.debug(f"Non empty node - {config['name']}")
processor = self.processor_factory.create_processor(
config, self.context, self.storage_strategy
)
processor.set_output(config_output)
processors[config["name"]] = processor
return processors
def get_related_config_data(self, config_data, node_name, visited):
if node_name in visited:
return []
visited.append(node_name)
current_config = next(
(config for config in config_data if config["name"] == node_name), None
)
if not current_config:
return []
related_configs = [current_config]
for input in current_config.get("inputs", []):
related_configs.extend(
self.get_related_config_data(
config_data, input.get("inputNode"), visited
)
)
return related_configs
def load_processors_for_node(self, config_data, node_name):
processors = self.load_required_processors(config_data, node_name)
self._link_processors(processors)
return processors
@abstractmethod
def launch_processors(self, processors):
pass
@abstractmethod
def launch_processors_for_node(self, processors, node_name=None):
pass
def notify_observers(self, event, data):
for observer in self.observers:
observer.notify(event, data)
================================================
FILE: packages/backend/app/processors/launcher/async_processor_launcher.py
================================================
import gc
import threading
import time
import eventlet
from eventlet.semaphore import Semaphore
import logging
import traceback
from typing import Dict, List
from enum import Enum
from .processor_event import ProcessorEvent
from .event_type import EventType
from ..observer.observer import Observer
from ..components.processor import Processor
from .abstract_topological_processor_launcher import (
AbstractTopologicalProcessorLauncher,
)
class AsyncProcessorLauncher(AbstractTopologicalProcessorLauncher, Observer):
"""
AsyncProcessorLauncher extends the functionality of the Basic Processor Launcher.
The main enhancement in this class is the implementation of the 'launch_processors' method,
which leverages eventlet greenthreads. This allows for asynchronous execution of processors,
enabling efficient handling of I/O-bound tasks and improving the overall performance of processor execution.
"""
GREENTHREAD_POOL_SIZE = 7
class NodeState(Enum):
PENDING = 1
RUNNING = 2
COMPLETED = 3
ERROR = 4
class Node:
def __init__(self, id: str, parent_ids: List[str], processor: Processor):
self.id = id
self.parent_ids = parent_ids
self.state = AsyncProcessorLauncher.NodeState.PENDING
self.output = None
self.processor = processor
self.lock = Semaphore(1)
def run(self):
with self.lock:
if self.state != AsyncProcessorLauncher.NodeState.PENDING:
logging.warning(
f"Node {self.id} is already being processed or completed."
)
return self.output
self.state = AsyncProcessorLauncher.NodeState.RUNNING
try:
self.output = self.processor.process_and_update()
except Exception as e:
self.state = AsyncProcessorLauncher.NodeState.ERROR
raise e
self.state = AsyncProcessorLauncher.NodeState.COMPLETED
return self.output
def get_processor(self):
return self.processor
def get_input_processor_names(self, processor: Processor):
return [
input_processor.name for input_processor in processor.get_input_processors()
]
def convert_processors_to_node_dict(self, processors: List[Processor]):
nodes = {}
for processor in processors.values():
nodes[processor.name] = self.Node(
processor.name, self.get_input_processor_names(processor), processor
)
return nodes
def launch_processors(self, processors: List[Processor]):
for processor in processors.values():
processor.add_observer(self)
nodes = self.convert_processors_to_node_dict(processors)
pool = eventlet.GreenPool(AsyncProcessorLauncher.GREENTHREAD_POOL_SIZE)
logging.debug(nodes)
initialized_nodes = set()
while nodes:
error_detected = any(
node.state == AsyncProcessorLauncher.NodeState.ERROR
for node in nodes.values()
)
if error_detected:
logging.debug("A node is in ERROR state. Halting processing.")
break
for id, node in nodes.items():
if (
node.state == AsyncProcessorLauncher.NodeState.PENDING
and self.can_run(node, nodes)
and id not in initialized_nodes
):
logging.debug(f"Spawning green thread for node {id}.")
initialized_nodes.add(id)
pool.spawn(self.run_node, node)
eventlet.sleep(0.5)
nodes = self.remove_completed_nodes(nodes)
logging.debug(f"Remaining nodes: {[node.id for node in nodes.values()]}")
pool.waitall()
def remove_completed_nodes(self, nodes: List[Node]):
return {
id: n
for id, n in nodes.items()
if n.state not in [AsyncProcessorLauncher.NodeState.COMPLETED]
}
def can_run(self, node: Node, nodes: List[Node]):
# If parents aren't in the list, then the node can run
return all(parent_id not in nodes for parent_id in node.parent_ids)
def launch_processors_for_node(self, processors: List[Processor], node_name=None):
for processor in processors.values():
if processor.get_output() is None or processor.name == node_name:
processor.add_observer(self)
self.run_processor(processor)
if processor.name == node_name:
break
def run_processor(self, processor: "Processor"):
try:
self.notify_current_node_running(processor)
start_time = time.time()
output = processor.process_and_update()
end_time = time.time()
duration = end_time - start_time
self.notify_progress(processor, output, duration=duration, isDone=True)
except Exception as e:
self.notify_error(processor, e)
raise e
def run_node(self, node: Node):
try:
processor = node.get_processor()
self.notify_current_node_running(processor)
start_time = time.time()
output = node.run()
end_time = time.time()
duration = end_time - start_time
self.notify_progress(node.get_processor(), output, duration=duration)
except Exception as e:
node.state = AsyncProcessorLauncher.NodeState.ERROR
self.notify_error(node.get_processor(), e)
traceback.print_exc()
raise e
def notify(self, event: EventType, data: ProcessorEvent):
if event == EventType.STREAMING:
self.notify_streaming(data.source, data.output)
================================================
FILE: packages/backend/app/processors/launcher/basic_processor_launcher.py
================================================
from .abstract_topological_processor_launcher import AbstractTopologicalProcessorLauncher
class BasicProcessorLauncher(AbstractTopologicalProcessorLauncher):
"""
Basic Processor Launcher emiting event
A class that launches processors based on configuration data.
"""
def launch_processors(self, processors):
for processor in processors.values():
self.notify_current_node_running(processor)
try :
output = processor.process()
self.notify_progress(processor, output)
except Exception as e:
self.notify_error(processor, e)
raise e
def launch_processors_for_node(self, processors, node_name=None):
for processor in processors.values():
if processor.get_output() is None or processor.name == node_name:
self.notify_current_node_running(processor)
try :
output = processor.process()
self.notify_progress(processor, output)
except Exception as e:
self.notify_error(processor, e)
raise e
if processor.name == node_name:
break
================================================
FILE: packages/backend/app/processors/launcher/event_type.py
================================================
from enum import Enum
class EventType(Enum):
PROGRESS = "progress"
STREAMING = "streaming"
CURRENT_NODE_RUNNING = "current_node_running"
ERROR = "error"
================================================
FILE: packages/backend/app/processors/launcher/processor_event.py
================================================
from dataclasses import dataclass, field
from typing import Any
@dataclass
class ProcessorEvent:
source: Any = field(default=None)
output: Any = field(default=None)
error: str = field(default=None)
================================================
FILE: packages/backend/app/processors/launcher/processor_launcher.py
================================================
from abc import ABC, abstractmethod
from ..context.processor_context import ProcessorContext
class ProcessorLauncher(ABC):
@abstractmethod
def load_processors(self, config_data):
pass
@abstractmethod
def load_processors_for_node(self, config_data, node_name):
pass
@abstractmethod
def launch_processors(self, processor):
pass
@abstractmethod
def launch_processors_for_node(self, processors, node_name):
pass
@abstractmethod
def set_context(self, context: ProcessorContext):
pass
================================================
FILE: packages/backend/app/processors/launcher/processor_launcher_event.py
================================================
from dataclasses import dataclass, field
from typing import Any
from ..components.processor import Processor
@dataclass
class ProcessorLauncherEvent:
instance_name: str
user_id: int = field(default=None)
output: Any = field(default=None)
processor_type: str = field(default=None)
processor: Processor = field(default=None)
isDone: bool = field(default=False)
error: str = field(default=None)
session_id: str = field(default=None)
duration: float = field(default=0)
================================================
FILE: packages/backend/app/processors/observer/observer.py
================================================
from abc import ABC, abstractmethod
class Observer(ABC):
@abstractmethod
def notify(self, event, data):
pass
================================================
FILE: packages/backend/app/processors/observer/socketio_event_emitter.py
================================================
from ..launcher.event_type import EventType
from ..launcher.processor_launcher_event import ProcessorLauncherEvent
from .observer import Observer
import logging
from ...flask.socketio_init import socketio
class SocketIOEventEmitter(Observer):
"""
A SocketIO event emitter that emits events to clients connected via WebSocket.
This class implements the Observer pattern and is designed to emit events
to specific client sessions in a Flask-SocketIO application. It can be safely
executed within greenthreads, making it suitable for use in environments
where asynchronous operations and real-time communication are required.
Attributes:
None
Methods:
notify(event, data): Emits the specified event to the client associated
with the session ID in `data`. Handles exceptions
gracefully and logs emission details.
"""
def notify(self, event: EventType, data: ProcessorLauncherEvent):
if event == EventType.STREAMING.value:
event = EventType.PROGRESS.value
json_event = {}
json_event["instanceName"] = data.instance_name
if data.output is not None:
json_event["output"] = data.output
if data.isDone is not None:
json_event["isDone"] = data.isDone
if data.error is not None:
json_event["error"] = str(data.error)
try:
socketio.emit(event, json_event, to=data.session_id)
logging.debug(
f"Successfully emitted event {event} with data {json_event} to {data.session_id}"
)
except Exception as e:
logging.error(f"Error emitting event {event}: {e}")
================================================
FILE: packages/backend/app/processors/utils/retry_mixin.py
================================================
import time
import logging
class RetryMixin:
def run_with_retry(self, func, *args, **kwargs):
"""
Executes `func` with retries as defined in the processor configuration.
Expected configuration keys:
- max_retries: number of extra attempts (default 0 means no retry)
- retry_delay: delay (in seconds) between attempts (default 0)
"""
retries = getattr(self, "max_retries", 0)
delay = getattr(self, "retry_delay", 0)
for attempt in range(retries + 1):
try:
return func(*args, **kwargs)
except Exception as e:
logging.warning(
f"Attempt {attempt+1}/{retries+1} for {func.__name__} failed"
)
if attempt == retries:
raise
if delay:
time.sleep(delay)
================================================
FILE: packages/backend/app/root_injector.py
================================================
from typing import List
from injector import Injector, Binder, Module
from tests.utils.processor_factory_mock import ProcessorFactoryMock
from app.processors.launcher.async_processor_launcher import AsyncProcessorLauncher
from app.processors.observer.socketio_event_emitter import SocketIOEventEmitter
from app.processors.observer.observer import Observer
from app.storage.local_storage_strategy import LocalStorageStrategy
from app.storage.s3_storage_strategy import S3StorageStrategy
from app.storage.storage_strategy import StorageStrategy
from app.env_config import is_mock_env, is_s3_enabled
from app.processors.factory.processor_factory import ProcessorFactory
from app.processors.factory.processor_factory_iter_modules import (
ProcessorFactoryIterModules,
)
from app.processors.launcher.processor_launcher import ProcessorLauncher
import logging
class ProcessorFactoryModule(Module):
def configure(self, binder: Binder):
if is_mock_env():
fake_factory = ProcessorFactoryMock(with_delay=True)
binder.bind(ProcessorFactory, to=fake_factory)
else:
binder.bind(ProcessorFactory, to=ProcessorFactoryIterModules)
class StorageModule(Module):
def configure(self, binder: Binder):
if is_s3_enabled():
logging.info("Using S3 storage strategy")
binder.bind(StorageStrategy, to=S3StorageStrategy)
else:
logging.info("Using local storage strategy")
binder.bind(StorageStrategy, to=LocalStorageStrategy)
class ProcessorLauncherModule(Module):
def configure(self, binder: Binder):
binder.bind(ProcessorLauncher, to=AsyncProcessorLauncher)
observer_list = [SocketIOEventEmitter()]
binder.multibind(List[Observer], to=observer_list)
def create_application_injector() -> Injector:
injector = Injector(
[
ProcessorFactoryModule(),
StorageModule(),
ProcessorLauncherModule(),
],
auto_bind=True,
)
return injector
_current_injector: Injector = create_application_injector()
def get_root_injector() -> Injector:
return _current_injector
def refresh_root_injector() -> None:
global _current_injector
_current_injector = create_application_injector()
================================================
FILE: packages/backend/app/storage/local_storage_strategy.py
================================================
from typing import Any
from ..storage.storage_strategy import StorageStrategy
from werkzeug.utils import secure_filename
import os
from app.env_config import (
get_local_storage_folder_path,
)
from injector import singleton
@singleton
class LocalStorageStrategy(StorageStrategy):
"""Local storage strategy. To be used only when you're running the app on your own machine.
Every generated image is saved in a local directory."""
LOCAL_DIR = get_local_storage_folder_path()
def save(self, filename: str, data: Any) -> str:
if not os.path.exists(self.LOCAL_DIR):
os.makedirs(self.LOCAL_DIR)
secure_name = secure_filename(filename)
filepath = os.path.join(self.LOCAL_DIR, secure_name)
with open(filepath, "wb") as f:
f.write(data)
return self.get_url(secure_name)
def get_url(self, filename: str) -> str:
port = os.getenv("PORT")
return f"http://localhost:{port}/image/{filename}"
def get_file(self, filename: str) -> bytes:
pass
================================================
FILE: packages/backend/app/storage/s3_storage_strategy.py
================================================
import logging
from typing import Any
import uuid
from ..storage.storage_strategy import CloudStorageStrategy
import boto3
from botocore.config import Config
import os
from datetime import timedelta
from injector import singleton
import mimetypes
import requests
@singleton
class S3StorageStrategy(CloudStorageStrategy):
"""S3 storage strategy. For the cloud version, every generated image is saved in an S3 bucket for 12H."""
EXPIRATION = timedelta(hours=24)
UPLOAD_EXPIRATION = timedelta(minutes=10)
MAX_UPLOAD_SIZE_BYTES = int(os.getenv("MAX_UPLOAD_SIZE_MB", "300")) * 1024 * 1024
MAX_POOL_CONNECTIONS = int(os.getenv("MAX_POOL_CONNECTIONS", "100"))
def __init__(self):
self.BUCKET_NAME = os.getenv("S3_BUCKET_NAME")
endpoint_url = os.getenv("S3_ENDPOINT_URL")
if not endpoint_url:
endpoint_url = None
kwargs = {
"aws_access_key_id": os.getenv("S3_AWS_ACCESS_KEY_ID"),
"aws_secret_access_key": os.getenv("S3_AWS_SECRET_ACCESS_KEY"),
"region_name": os.getenv("S3_AWS_REGION_NAME"),
"config": Config(max_pool_connections=self.MAX_POOL_CONNECTIONS),
}
if endpoint_url is not None:
kwargs["endpoint_url"] = endpoint_url
self.s3_client = boto3.client(
"s3",
**kwargs,
)
def save(self, filename: str, data: Any, bucket_name: str = None) -> str:
if bucket_name is None:
bucket_name = self.BUCKET_NAME
self.s3_client.put_object(Bucket=bucket_name, Key=filename, Body=data)
url = self.s3_client.generate_presigned_url(
ClientMethod="get_object",
Params={"Bucket": bucket_name, "Key": filename},
ExpiresIn=int(self.EXPIRATION.total_seconds()),
)
return url
def get_upload_link(self, filename=None) -> str:
file_key = f"uploads/{uuid.uuid4()}"
content_type = None
if not mimetypes.guess_type("test.webp")[0]:
mimetypes.add_type("image/webp", ".webp")
if not mimetypes.guess_type("test.safetensors")[0]:
mimetypes.add_type("application/octet-stream", ".safetensors")
if filename:
extension = filename.split(".")[-1]
file_key += f".{extension}"
mime_type, _ = mimetypes.guess_type(filename)
content_type = mime_type
try:
upload_data = self.s3_client.generate_presigned_post(
Bucket=self.BUCKET_NAME,
Key=file_key,
Fields=None,
Conditions=[["content-length-range", 0, self.MAX_UPLOAD_SIZE_BYTES]],
ExpiresIn=int(self.UPLOAD_EXPIRATION.total_seconds()),
)
download_url = self.s3_client.generate_presigned_url(
ClientMethod="get_object",
Params={
"Bucket": self.BUCKET_NAME,
"Key": file_key,
"ResponseContentType": content_type,
},
ExpiresIn=int(self.EXPIRATION.total_seconds()),
)
except Exception as e:
logging.error(e)
raise Exception(
"Error uploading file. "
"Please check your S3 configuration. "
"If you've not configured S3 please refer to docs.ai-flow.net/docs/file-upload"
)
return upload_data, download_url
def get_url(self, filename: str, bucket_name: str = None) -> str:
"""Get presigned URL based on filename (URI)"""
if bucket_name is None:
bucket_name = self.BUCKET_NAME
try:
url = self.s3_client.generate_presigned_url(
ClientMethod="get_object",
Params={
"Bucket": bucket_name,
"Key": filename,
},
ExpiresIn=int(self.EXPIRATION.total_seconds()),
)
return url
except Exception as e:
logging.error(f"Error generating presigned URL for {filename}: {e}")
raise Exception("Error generating presigned URL. ")
def get_file(self, filename: str, bucket_name: str = None) -> bytes:
"""Get file based on filename (URI)"""
if filename.startswith("s3://"):
filename = filename[len("s3://") :]
filename = filename[filename.index("/") + 1 :]
if bucket_name is None:
bucket_name = self.BUCKET_NAME
try:
response = self.s3_client.get_object(Bucket=bucket_name, Key=filename)
return response["Body"].read()
except Exception as e:
logging.error(f"Error getting file {filename}: {e}")
raise Exception("Error getting file. ")
def upload_and_get_link(self, filename: str, bucket_name: str = None) -> str:
"""Upload file and get link based on filename (URI)"""
if bucket_name is None:
bucket_name = self.BUCKET_NAME
upload_data, download_url = self.get_upload_link(filename)
url = upload_data["url"]
fields = upload_data["fields"]
filepath = filename
if not os.path.isfile(filepath):
raise FileNotFoundError(f"File '{filename}' does not exist.")
with open(filepath, "rb") as file:
files = {"file": file}
response = requests.post(url, data=fields, files=files)
if response.status_code == 204:
logging.info("File uploaded successfully.")
return download_url
else:
logging.error(f"Failed to upload file: {response.text}")
response.raise_for_status()
================================================
FILE: packages/backend/app/storage/storage_strategy.py
================================================
from abc import ABC, abstractmethod
from typing import Any, Optional
class StorageStrategy(ABC):
"""Storage strategy interface. We use this storage strategy to save and get the url of documents.
This is especially useful for the image generated by the stable diffusion model."""
@abstractmethod
def save(self, filename: str, data: Any) -> Optional[str]:
pass
@abstractmethod
def get_url(self, filename: str) -> str:
pass
@abstractmethod
def get_file(self, filename: str, *args) -> bytes:
pass
class CloudStorageStrategy(StorageStrategy):
@abstractmethod
def get_upload_link(self, filename: str) -> str:
pass
================================================
FILE: packages/backend/app/tasks/green_pool_task_manager.py
================================================
import logging
from queue import Queue
import eventlet
from eventlet.green import threading
from .task_exception import TaskAlreadyRegisteredError
from ..env_config import get_background_task_max_workers
task_queues = {}
task_processors = {}
task_semaphores = {}
pool = eventlet.GreenPool(size=get_background_task_max_workers())
def register_task_processor(task_name, processor_func, max_concurrent_tasks=2):
if task_name in task_queues:
raise TaskAlreadyRegisteredError(task_name=task_name)
task_queue = Queue()
task_queues[task_name] = task_queue
task_processors[task_name] = processor_func
task_semaphores[task_name] = threading.Semaphore(max_concurrent_tasks)
logging.info(
f"Registered green pool task processor '{task_name}' with max_concurrent_tasks={max_concurrent_tasks}"
)
def process_task(task_name, task_data, task_result_queue):
semaphore = task_semaphores.get(task_name)
if semaphore is not None:
with semaphore:
if task_name in task_processors:
processor_func = task_processors[task_name]
result = processor_func(task_data)
task_result_queue.put(result)
else:
raise ValueError(f"Nao task processor registered for {task_name}")
else:
raise ValueError(f"No semaphore registered for {task_name}")
def add_task(task_name, task_data, result_queue):
if task_name in task_queues:
return pool.spawn(process_task, task_name, task_data, result_queue)
else:
raise ValueError(f"No task processor registered for {task_name}")
================================================
FILE: packages/backend/app/tasks/single_thread_tasks/browser/async_browser_task.py
================================================
import logging
import re
import asyncio
import threading
from ....utils.web_scrapping.async_browser_manager import (
AsyncBrowserManager,
)
browser_task_queue = None
event_loop = None
async def accept_cookies(page, cookies_consent_label, timeout=5000):
try:
await page.wait_for_selector(
f"button:has-text('{cookies_consent_label}')", timeout=timeout
)
accept_button = page.locator(
f"button:has-text('{cookies_consent_label}')"
).first
if not accept_button:
return
await accept_button.click()
await page.wait_for_timeout(2000)
except Exception as e:
logging.warning("Could not find or click the cookie accept button:", e)
def strip_attributes(html):
return re.sub(r"(<\w+)(\s+[^>]+)?(>)", r"\1\3", html)
async def fetch_url_content(
url,
browser_manager,
with_html_tags=False,
with_html_attributes=False,
selectors=None,
selectors_to_remove=None,
auto_consent_cookies=False,
enable_ad_blocker=False,
cookies_consent_label=None,
):
page, context = await browser_manager.get_tab()
try:
await page.goto(url, timeout=30000, wait_until="domcontentloaded")
except Exception as e:
logging.error(f"Failed to load page: {str(e)}")
return ""
try:
await page.wait_for_load_state("networkidle", timeout=10000)
content_attempts = 0
max_attempts = 3
while True:
content_attempts += 1
try:
content = await page.content()
break
except Exception as e:
if "Page.content" in str(e):
if content_attempts >= max_attempts:
logging.error(
f"Failed to retrieve page {url} content after {content_attempts} attempts: {str(e)}"
)
return ""
await page.wait_for_load_state("load", timeout=5000)
else:
raise
if selectors_to_remove:
for selector in selectors_to_remove:
elements = await page.query_selector_all(selector)
for element in elements:
await page.evaluate("(element) => element.remove()", element)
content = ""
if selectors and len(selectors) > 0:
for selector in selectors:
await page.wait_for_selector(selector, timeout=3000)
elements = await page.query_selector_all(selector)
for element in elements:
content_piece = (
await element.inner_html()
if with_html_tags
else await element.inner_text()
)
content += content_piece + "\n"
else:
content = (
await page.content()
if with_html_tags
else await page.inner_text("body")
)
if with_html_tags and not with_html_attributes:
content = strip_attributes(content)
except Exception as e:
logging.error(f"Error processing {url} page content: {str(e)}")
content = ""
finally:
await browser_manager.release_tab(page, context)
return content
async def scrapping_task(task_data, browser_manager):
selectors = task_data.get("selectors", [])
selectors_to_remove = task_data.get("selectors_to_remove", [])
with_html_tags = task_data.get("with_html_tags", False)
with_html_attributes = task_data.get("with_html_attributes", False)
url = task_data.get("url")
cookies_consent_label = task_data.get("cookies_consent_label", None)
auto_consent_cookies = task_data.get("auto_consent_cookies", False)
enable_ad_blocker = task_data.get("enable_ad_blocker", False)
content = await fetch_url_content(
url,
browser_manager,
with_html_tags=with_html_tags,
with_html_attributes=with_html_attributes,
selectors=selectors,
selectors_to_remove=selectors_to_remove,
cookies_consent_label=cookies_consent_label,
auto_consent_cookies=auto_consent_cookies,
enable_ad_blocker=enable_ad_blocker,
)
return content
async def add_task(task_data, result_queue):
await browser_task_queue.put((task_data, result_queue))
async def browser_task_worker():
global browser_task_queue
browser_task_queue = asyncio.Queue()
browser_manager = AsyncBrowserManager()
await browser_manager.initialize_browser()
logging.info("Starting browser task worker")
while True:
task_data, result_queue = await browser_task_queue.get()
if task_data is None: # Exit signal
logging.info("Exiting browser task worker")
break
try:
result = await scrapping_task(task_data, browser_manager)
result_queue.put(result)
except Exception as e:
logging.error(f"Error in browser task worker: {e}")
import traceback
traceback.print_exc()
event_loop = None
def start_event_loop():
global event_loop
event_loop = asyncio.new_event_loop()
asyncio.set_event_loop(event_loop)
event_loop.run_until_complete(browser_task_worker())
event_loop.run_forever()
def stop_event_loop():
event_loop.run_until_complete(browser_manager.close_browser())
event_loop.stop()
event_loop.close()
def add_task_sync(task_data, result_queue):
future = asyncio.run_coroutine_threadsafe(
add_task(task_data, result_queue), event_loop
)
return future
event_loop_thread = threading.Thread(target=start_event_loop)
event_loop_thread.start()
================================================
FILE: packages/backend/app/tasks/single_thread_tasks/browser/browser_task.py
================================================
import logging
import queue
import re
import threading
import time
from ....utils.web_scrapping.browser_manager import BrowserManager
browser_task_queue = queue.Queue()
def accept_cookies(page, cookies_consent_label, timeout=5000):
try:
page.wait_for_selector(
f"button:has-text('{cookies_consent_label}')", timeout=timeout
)
accept_button = page.locator(
f"button:has-text('{cookies_consent_label}')"
).first
accept_button.click()
page.wait_for_timeout(2000)
except Exception as e:
logging.warning("Could not find or click the cookie accept button:", e)
def strip_attributes(html):
return re.sub(r"(<\w+)(\s+[^>]+)?(>)", r"\1\3", html)
def fetch_url_content(
url,
browser_manager,
with_html_tags=False,
with_html_attributes=False,
selectors=None,
selectors_to_remove=None,
cookies_consent_label=None,
):
page, context = browser_manager.get_tab()
try:
page.goto(url, timeout=30000)
except Exception as e:
logging.error(f"Failed to load page: {str(e)}")
return ""
try:
if cookies_consent_label:
accept_cookies(page, cookies_consent_label)
if selectors_to_remove:
for selector in selectors_to_remove:
elements = page.query_selector_all(selector)
for element in elements:
page.evaluate("(element) => element.remove()", element)
content = ""
if selectors and len(selectors) > 0:
for selector in selectors:
elements = page.query_selector_all(selector)
for element in elements:
content_piece = (
element.inner_html() if with_html_tags else element.inner_text()
)
content += content_piece + "\n"
else:
content = page.content() if with_html_tags else page.inner_text("body")
if with_html_tags and not with_html_attributes:
content = strip_attributes(content)
except Exception as e:
logging.error(f"Error processing page content: {str(e)}")
content = ""
finally:
browser_manager.release_tab(page, context)
return content
def scrapping_task(task_data, browser_manager):
selectors = task_data.get("selectors", [])
selectors_to_remove = task_data.get("selectors_to_remove", [])
with_html_tags = task_data.get("with_html_tags", False)
with_html_attributes = task_data.get("with_html_attributes", False)
url = task_data.get("url")
cookies_consent_label = task_data.get("cookies_consent_label", None)
content = fetch_url_content(
url,
browser_manager,
with_html_tags=with_html_tags,
with_html_attributes=with_html_attributes,
selectors=selectors,
selectors_to_remove=selectors_to_remove,
cookies_consent_label=cookies_consent_label,
)
return content
def add_task_sync(task_data, result_queue):
browser_task_queue.put((task_data, result_queue))
def browser_thread_func(task_queue):
logging.info("Starting browser thread")
browser_manager = BrowserManager()
browser_manager.initialize_browser()
while True:
try:
task_data, result_queue = task_queue.get()
if task_data is None: # Exit signal
logging.info("Exiting browser thread")
break
result = scrapping_task(task_data, browser_manager)
result_queue.put(result)
except Exception as e:
logging.error(f"Error in browser thread: {e}")
finally:
time.sleep(0.1)
def stop_browser_thread():
browser_task_queue.put((None, None))
browser_thread.join()
browser_thread = threading.Thread(
target=browser_thread_func, args=(browser_task_queue,)
)
browser_thread.start()
================================================
FILE: packages/backend/app/tasks/task_exception.py
================================================
class TaskAlreadyRegisteredError(Exception):
"""Exception raised when attempting to register a task that is already registered."""
def __init__(self, task_name):
self.task_name = task_name
super().__init__(f"Task '{task_name}' is already registered.")
================================================
FILE: packages/backend/app/tasks/task_manager.py
================================================
import logging
from queue import Queue
from concurrent.futures import ThreadPoolExecutor
import threading
from .task_exception import TaskAlreadyRegisteredError
from ..env_config import get_background_task_max_workers
task_queues = {}
task_processors = {}
task_semaphores = {}
executor = ThreadPoolExecutor(max_workers=get_background_task_max_workers())
def register_task_processor(task_name, processor_func, max_concurrent_tasks=2):
if task_name in task_queues:
raise TaskAlreadyRegisteredError(task_name=task_name)
task_queue = Queue()
task_queues[task_name] = task_queue
task_processors[task_name] = processor_func
task_semaphores[task_name] = threading.Semaphore(max_concurrent_tasks)
logging.info(
f"Registered task processor '{task_name}' with max_concurrent_tasks={max_concurrent_tasks}"
)
def process_task(task_name, task_data, task_result_queue):
semaphore = task_semaphores.get(task_name)
if semaphore is not None:
with semaphore:
if task_name in task_processors:
processor_func = task_processors[task_name]
result = processor_func(task_data)
task_result_queue.put(result)
else:
raise ValueError(f"No task processor registered for {task_name}")
else:
raise ValueError(f"No semaphore registered for {task_name}")
def add_task(task_name, task_data, result_queue):
if task_name in task_queues:
executor.submit(process_task, task_name, task_data, result_queue)
else:
raise ValueError(f"No task processor registered for {task_name}")
================================================
FILE: packages/backend/app/tasks/task_utils.py
================================================
from queue import Empty
import time
import eventlet
def wait_for_result(queue, timeout=120, initial_sleep=0.1, max_sleep=5.0):
start_time = time.time()
sleep_duration = initial_sleep
while True:
try:
result = queue.get_nowait()
return result
except Empty:
if time.time() - start_time >= timeout:
raise TimeoutError("Operation timed out after the specified timeout")
eventlet.sleep(sleep_duration)
sleep_duration = min(sleep_duration * 1.5, max_sleep)
================================================
FILE: packages/backend/app/tasks/thread_pool_task_manager.py
================================================
from concurrent.futures import ThreadPoolExecutor
import logging
from queue import Queue
import threading
from .task_exception import TaskAlreadyRegisteredError
from ..env_config import get_background_task_max_workers
task_queues = {}
task_processors = {}
task_semaphores = {}
executor = ThreadPoolExecutor(max_workers=get_background_task_max_workers())
def register_task_processor(task_name, processor_func, max_concurrent_tasks=2):
if task_name in task_queues:
raise TaskAlreadyRegisteredError(task_name=task_name)
task_queue = Queue()
task_queues[task_name] = task_queue
task_processors[task_name] = processor_func
task_semaphores[task_name] = threading.Semaphore(max_concurrent_tasks)
logging.info(
f"Registered thread pool task processor '{task_name}' with max_concurrent_tasks={max_concurrent_tasks}"
)
def process_task(task_name, task_data, task_result_queue):
semaphore = task_semaphores.get(task_name)
if semaphore is not None:
with semaphore:
if task_name in task_processors:
processor_func = task_processors[task_name]
result = processor_func(task_data)
task_result_queue.put(result)
else:
raise ValueError(f"Nao task processor registered for {task_name}")
else:
raise ValueError(f"No semaphore registered for {task_name}")
def add_task(task_name, task_data, result_queue):
if task_name in task_queues:
return executor.submit(process_task, task_name, task_data, result_queue)
else:
raise ValueError(f"No task processor registered for {task_name}")
================================================
FILE: packages/backend/app/utils/node_extension_utils.py
================================================
import importlib
import logging
import os
import pkgutil
from cachetools import TTLCache, cached
from ..processors.components.extension.extension_processor import (
DynamicExtensionProcessor,
ExtensionProcessor,
)
very_long_ttl_cache = 120000
raw_blacklist = os.getenv("EXTENSIONS_BLACKLIST", "").strip()
EXTENSIONS_BLACKLIST = raw_blacklist.split(",") if raw_blacklist else []
raw_whitelist = os.getenv("EXTENSIONS_WHITELIST", "").strip()
EXTENSIONS_WHITELIST = raw_whitelist.split(",") if raw_whitelist else []
def _load_dynamic_extension(processor_type, data):
package = importlib.import_module("app.processors.components.extension")
prefix = package.__name__ + "."
for importer, module_name, is_pkg in pkgutil.iter_modules(package.__path__, prefix):
module = importlib.import_module(module_name)
for attribute_name in dir(module):
attribute = getattr(module, attribute_name)
if isinstance(attribute, type) and issubclass(
attribute, DynamicExtensionProcessor
):
if hasattr(attribute, "get_dynamic_node_config") and hasattr(
attribute, "processor_type"
):
if attribute.processor_type == processor_type:
schema = attribute.get_dynamic_node_config(attribute, data)
return schema
return None
def _load_all_extension_schemas():
schemas = []
package = importlib.import_module("app.processors.components.extension")
prefix = package.__name__ + "."
for importer, module_name, is_pkg in pkgutil.iter_modules(package.__path__, prefix):
module = importlib.import_module(module_name)
for attribute_name in dir(module):
try:
attribute = getattr(module, attribute_name)
if isinstance(attribute, type) and issubclass(
attribute, ExtensionProcessor
):
if hasattr(attribute, "get_node_config"):
schema = attribute.get_node_config(attribute)
if schema is not None:
schemas.append(schema)
except Exception as e:
logging.warning(
f"Error loading extension {module_name}.{attribute_name}"
)
continue
return schemas
def filter_extensions(extensions):
if len(EXTENSIONS_WHITELIST) > 0:
extensions = [e for e in extensions if e.processorType in EXTENSIONS_WHITELIST]
if len(EXTENSIONS_BLACKLIST) > 0:
extensions = [
e for e in extensions if e.processorType not in EXTENSIONS_BLACKLIST
]
return extensions
def get_extensions():
schemas = _load_all_extension_schemas()
schemas = filter_extensions(schemas)
schemas_dict = []
for schema in schemas:
if schema is not None:
schemas_dict.append(schema.dict())
return schemas_dict
def get_dynamic_extension_config(processor_type, data):
schema = _load_dynamic_extension(processor_type, data)
return schema
================================================
FILE: packages/backend/app/utils/openapi_client.py
================================================
import logging
from typing import Optional
import requests
import eventlet
class Client:
def __init__(
self,
api_token: Optional[str] = None,
base_url: Optional[str] = None,
timeout_ms: int = None,
**kwargs,
) -> None:
super().__init__()
self._api_token = api_token
self._base_url = base_url
self._timeout_ms = timeout_ms
self._client_kwargs = kwargs
def post(
self,
path: str,
data: Optional[dict] = None,
files: Optional[dict] = {"none": (None, "")},
content_type: str = None,
accept: str = "application/json",
**kwargs,
) -> dict:
headers = {
"Authorization": f"Bearer {self._api_token}",
"Accept": accept,
}
if files:
response = requests.post(
f"{self._base_url}{path}",
headers=headers,
files=files,
data=data,
timeout=self._timeout_ms,
**self._client_kwargs,
**kwargs,
)
else:
logging.info("JSON BOI")
response = requests.post(
f"{self._base_url}{path}",
headers=headers,
json=data,
timeout=self._timeout_ms,
**self._client_kwargs,
**kwargs,
)
if response.status_code == 200:
if accept == "application/json":
return response.json()
elif accept == "image/*":
return response.content
else:
return response.content
else:
raise Exception(response.text)
def get(
self,
path: str,
content_type: str = None,
accept: str = "application/json",
**kwargs,
) -> dict:
response = requests.get(
f"{self._base_url}{path}",
headers={
"Content-Type": content_type,
"Authorization": f"Bearer {self._api_token}",
"Accept": accept,
},
timeout=self._timeout_ms,
**self._client_kwargs,
**kwargs,
)
if response.status_code == 200:
if accept == "application/json":
return response.json()
elif accept == "image/*":
return response.content
else:
return response.content
else:
print(response.status_code)
if response.json()["status"] == "in-progress":
return response.json()
raise Exception(str(response.json()))
def pooling(
self,
path: str,
content_type: str = None,
accept: str = "application/json",
**kwargs,
) -> dict:
pooling_response = self.get(path=path, accept=accept)
while (
isinstance(pooling_response, dict)
and pooling_response.get("status") == "in-progress"
):
print("Pooling...")
eventlet.sleep(0.5)
pooling_response = self.get(path=path, accept=accept)
return pooling_response
# if __name__ == "__main__":
# api_reader = OpenAPIReader("../../resources/openapi/stabilityai.json")
# print("API Key Name:", api_reader.get_api_key_name())
# specific_path = "/v2beta/stable-image/generate/core" # Remplacez par un chemin valide de votre fichier OpenAPI
# params = api_reader.get_request_schema_for_path(specific_path, "POST")
# print(params)
# requestBody = {
# "prompt": "A cute baby sea otter",
# "aspect_ratio": "16:9",
# }
# serverUrl = api_reader.get_servers()[0]
# print(serverUrl)
# client = Client(
# api_token="sk-rQKpzMtDplCq8NFjnPxKOKkFzjmdEsticRFMUWUk11uPrflL",
# base_url=serverUrl,
# )
# response = client.post(specific_path, requestBody, accept="image/*")
# print(response)
================================================
FILE: packages/backend/app/utils/openapi_converter.py
================================================
import json
from ..processors.components.model import Option
from ..processors.components.node_config_builder import (
FieldBuilder,
NodeConfigBuilder,
NodeConfigVariantBuilder,
)
class OpenAPIConverter:
def __init__(self):
pass
def convert_enum_to_options(self, enum, defaultValue):
options = []
for value in enum:
options.append(
Option(default=(value == defaultValue), value=value, label=value)
)
return options
def convert_properties_to_fields(self, schema):
fields = []
properties = schema.get("properties", {})
for key, value in properties.items():
field_builder = FieldBuilder()
field_builder.set_name(key)
field_builder.set_label(key)
field_builder.set_description(value.get("description"))
field_type = value.get("type")
if field_type == "string":
if "enum" in value:
field_builder.set_type("select")
field_builder.set_options(
self.convert_enum_to_options(
value["enum"], value.get("default")
)
)
else:
field_builder.set_type("textfield")
field_builder.set_has_handle(True)
elif field_type == "number":
if "minimum" or "maximum" in value:
if value.get("maximum") > 1000:
field_builder.set_type("numericfield")
else:
field_builder.set_type("slider")
field_builder.set_min(value.get("minimum"))
field_builder.set_max(value.get("maximum"))
else:
field_builder.set_type("numericfield")
field_builder.set_has_handle(True)
else:
field_builder.set_type("textfield")
if "example" in value:
if "format" in value and value["format"] == "binary":
field_builder.set_placeholder("Resource URL")
field_builder.set_is_binary(True)
else:
field_builder.set_placeholder(value["example"])
if "default" in value:
field_builder.set_default_value(value["default"])
required_fields = schema.get("required", [])
if key in required_fields:
field_builder.set_required(True)
fields.append(field_builder.build())
return fields
def convert_schema_to_node_config(self, schema):
schema = schema.get("schema")
if schema.get("oneOf") and schema.get("discriminator"):
builder = NodeConfigVariantBuilder()
discriminatorName = schema.get("discriminator").get("propertyName")
mapping = schema.get("discriminator", {}).get("mapping")
builder.add_discriminator_field(discriminatorName)
for key, value in mapping.items():
fields = self.convert_properties_to_fields(value)
builder.add_sub_configuration(
NodeConfigBuilder()
.add_discriminator(discriminatorName, key)
.set_fields(fields)
.build()
)
else:
builder = NodeConfigBuilder()
# print(schema)
fields = self.convert_properties_to_fields(schema)
builder.set_fields(fields)
return builder
================================================
FILE: packages/backend/app/utils/openapi_reader.py
================================================
import json
import hashlib
from openapi_spec_validator.readers import read_from_filename
class OpenAPIReader:
HTTP_METHODS_LIST = ["get", "post", "put", "delete", "patch", "options", "head"]
def __init__(self, file_path):
spec_dict, base_uri = read_from_filename(file_path)
self._api_data = spec_dict
self._paths = self.get_all_paths()
def get_api_key_name(self):
security_schemes = self._api_data.get("components", {}).get(
"securitySchemes", {}
)
for key, value in security_schemes.items():
if value.get("type") == "apiKey":
return key
return None
def get_servers(self):
servers = self._api_data.get("servers", [])
return [server["url"] for server in servers if "url" in server]
def get_all_paths(self):
paths = self._api_data.get("paths", {})
path_details = {}
for path, operations in paths.items():
operations_info = {}
for operation_type, operation_details in operations.items():
if operation_type in OpenAPIReader.HTTP_METHODS_LIST:
operations_info[operation_type.upper()] = {
"summary": operation_details.get(
"summary", "No summary provided"
),
"description": operation_details.get(
"description", "No description provided"
),
}
path_details[path] = operations_info
return path_details
def get_all_paths_names(self):
return list(self._paths.keys())
def get_params_for_path(self, path, method):
path_details = self._api_data.get("paths", {}).get(path, {})
params = {}
for path_method, details in path_details.items():
if method.upper() == path_method.upper():
params[method.upper()] = details.get("parameters", [])
return json.dumps(params, indent=4)
def get_path_accept(self, path, method):
path_details = self._api_data.get("paths", {}).get(path, {})
parameters = path_details[method].get("parameters", {})
for param in parameters:
if param["name"] == "accept":
return param["schema"]["default"]
return None
def get_response_content_type(self, path, method):
path_details = self._api_data.get("paths", {}).get(path, {})
responses = path_details[method].get("responses", {})
response_200 = responses.get("200", {})
content_types = response_200.get("content", {})
return list(content_types.keys())
def get_request_schema_for_path(self, path, method, content_type=None):
path_details = self._api_data.get("paths", {}).get(path, {})
if method.lower() in path_details:
request_body = path_details[method.lower()].get("requestBody", {})
content_types = request_body.get("content", {})
resolved_request_body = {}
for request_content_type, content_details in content_types.items():
schema = content_details.get("schema", {})
resolved_schema = self.resolve_schema(
schema
) # Recursively resolve schema including nested $ref
resolved_request_body[request_content_type] = {
"schema": resolved_schema
}
if content_type and content_type in resolved_request_body:
return resolved_request_body[content_type]
elif resolved_request_body:
first_content_type = next(iter(resolved_request_body))
return resolved_request_body[first_content_type]
return "{}"
def merge_schemas(base_schema, additions):
if "required" in additions:
if "required" in base_schema:
base_schema["required"].extend(additions["required"])
else:
base_schema["required"] = additions["required"]
return base_schema
def resolve_schema(self, schema):
"""Recursively resolve schema references including nested references."""
if "$ref" in schema:
return self.resolve_ref(schema["$ref"])
elif "schema" in schema:
return self.resolve_schema(schema["schema"])
elif "allOf" in schema:
allOf = schema["allOf"]
resolved_parts = []
for item in allOf:
resolved_parts.append(self.resolve_schema(item))
resolved_path = resolved_parts[0]
for part in resolved_parts[1:]:
resolved_path = OpenAPIReader.merge_schemas(resolved_path, part)
return resolved_path
elif "oneOf" in schema:
oneOf = schema["oneOf"]
resolved_parts = []
for item in oneOf:
resolved_parts.append(self.resolve_schema(item))
schema["oneOf"] = resolved_parts
if "discriminator" in schema:
discriminator = schema["discriminator"]
discriminator_mapping = discriminator["mapping"]
discriminator_resolved = {}
for key, value in discriminator_mapping.items():
if type(value) != str:
discriminator_resolved[key] = value
else:
discriminator_resolved[key] = self.resolve_ref(value)
discriminator["mapping"] = discriminator_resolved
return schema
elif schema.get("type") == "object" and "properties" in schema:
# If the schema is of type object and has properties, recursively resolve each property
resolved_properties = {}
for prop, prop_schema in schema["properties"].items():
resolved_properties[prop] = self.resolve_schema(prop_schema)
# Return the schema with resolved properties, preserving other aspects of the schema such as 'required'
return {
"type": "object",
"properties": resolved_properties,
"required": schema.get("required", []),
}
elif schema.get("type") == "array" and "items" in schema:
resolved_items = self.resolve_schema(schema["items"])
return {"type": "array", "items": resolved_items}
return schema
def resolve_ref(self, ref):
ref_parts = ref.split("/")
if ref_parts[0] == "#": # Assumes the first part is '#'
component_part = self._api_data
for part in ref_parts[1:]:
component_part = component_part.get(part, {})
return component_part
return {}
def get_response_schema_for_path(self, path, method, content_type=None):
path_details = self._api_data.get("paths", {}).get(path, {})
if method.lower() in path_details:
responses = path_details[method.lower()].get("responses", {})
response_200 = responses.get("200", {})
if "content" in response_200:
content = response_200["content"]
if content_type:
# If a specific content type is requested, return that schema
if content_type in content:
return self.resolve_schema(content[content_type])
else:
# Otherwise, return the schema for the first available content type
first_content_type = list(content.keys())[0]
return self.resolve_schema(content[first_content_type])
return "{}"
# Usage de la classe
if __name__ == "__main__":
def resolve_references(schema, root):
if isinstance(schema, dict):
if "$ref" in schema:
ref_path = schema["$ref"]
ref_parts = ref_path.strip("#/").split("/")
ref_schema = root
for part in ref_parts:
ref_schema = ref_schema[part]
return resolve_references(ref_schema, root)
else:
return {k: resolve_references(v, root) for k, v in schema.items()}
elif isinstance(schema, list):
return [resolve_references(item, root) for item in schema]
else:
return schema
spec_dict, base_uri = read_from_filename("../../resources/openapi/stabilityai.json")
reader = OpenAPIReader("../../resources/openapi/stabilityai.json")
path_img_to_video = reader.get_request_schema_for_path(
"/v2beta/image-to-video", "post"
)
resolved_path = reader.resolve_schema(path_img_to_video)
print(json.dumps(resolved_path, indent=4))
================================================
FILE: packages/backend/app/utils/processor_utils.py
================================================
from datetime import datetime
import os
import tempfile
from urllib.parse import urlparse
import requests
def create_empty_tmp_file(prefix="tmp"):
temp_dir = tempfile.TemporaryDirectory()
timestamp_str = datetime.now().strftime("%Y%m%d%H%M%S%f")
temp_file = os.path.join(temp_dir.name, f"{prefix}-{timestamp_str}")
return temp_file, temp_dir
def create_temp_file_with_str_content(content):
temp_file, temp_dir = create_empty_tmp_file()
with open(temp_file, "w") as f:
f.write(content)
return temp_file, temp_dir
def create_temp_file_with_bytes_content(content):
temp_file, temp_dir = create_empty_tmp_file()
with open(temp_file, "wb") as f:
f.write(content)
return temp_file, temp_dir
def get_file_size_from_url(url):
response = requests.head(url)
if response.status_code != 200:
raise ValueError(
f"Failed to reach the URL: returned status code {response.status_code}"
)
content_length = response.headers.get("Content-Length")
return content_length
def get_file_size_from_url_in_mb(url):
content_length = get_file_size_from_url(url)
if content_length is None:
raise ValueError("Failed to get file size from URL")
return round(int(content_length) / (1024 * 1024), 2)
def get_max_file_size_in_mb():
return int(os.getenv("MAX_TMP_FILE_SIZE_MB", 300))
def is_s3_file(url):
bucket_name = os.getenv("S3_BUCKET_NAME")
if not bucket_name:
return False
return url.startswith(f"s3://{bucket_name}") or url.startswith(
f"https://{bucket_name}.s3.amazonaws.com"
)
def is_accepted_url_file_size(url):
size_mb = get_file_size_from_url_in_mb(url)
if size_mb > get_max_file_size_in_mb():
return False
return True
def is_valid_url(url):
result = urlparse(url)
if not all([result.scheme, result.netloc]):
return False
return True
def file_downloable_check(url):
if is_s3_file(url):
return
if not is_accepted_url_file_size(url):
raise ValueError(
f"File size is too large. Max file size is {get_max_file_size_in_mb()} MB"
)
if not is_valid_url(url):
raise ValueError(f"Invalid URL: {url}")
def download_file_as_binary(url):
try:
file_downloable_check(url)
except ValueError as e:
raise ValueError(f"Can't download file: {e}")
response = requests.get(url)
response.raise_for_status()
return response.content
def stream_download_file_as_binary(url):
try:
file_downloable_check(url)
except ValueError as e:
raise ValueError(f"Can't download file: {e}")
with requests.get(url, stream=True) as response:
response.raise_for_status()
chunks = []
for chunk in response.iter_content(chunk_size=8192):
chunks.append(chunk)
return b"".join(chunks)
================================================
FILE: packages/backend/app/utils/replicate_utils.py
================================================
import os
import requests
from ..env_config import get_replicate_api_key
from cachetools import TTLCache, cached
import logging
short_ttl_cache = 600
long_ttl_cache = 12000
very_long_ttl_cache = 120000
REPLICATE_API_URL = "https://api.replicate.com"
REPLICATE_MODEL_API_URL = f"{REPLICATE_API_URL}/v1/models"
REPLICATE_COLLECTION_API_URL = f"{REPLICATE_API_URL}/v1/collections"
@cached(TTLCache(maxsize=100, ttl=600))
def get_replicate_models(cursor: str = None):
api_token = get_replicate_api_key()
if not api_token:
raise Exception("Replicate API token not found in environment variables")
headers = {"Authorization": f"Token {api_token}"}
url = REPLICATE_MODEL_API_URL
if cursor:
url += f"?cursor={cursor}"
response = requests.get(url=url, headers=headers)
if response.status_code != 200:
raise Exception(f"Failed to fetch models: {response.status_code}")
data = response.json()
for model in data["results"]:
if "latest_version" in model:
del model["latest_version"]
if "default_example" in model:
del model["default_example"]
return data
@cached(TTLCache(maxsize=100, ttl=long_ttl_cache))
def get_replicate_collections():
api_token = get_replicate_api_key()
if not api_token:
raise Exception("Replicate API token not found in environment variables")
headers = {"Authorization": f"Token {api_token}"}
response = requests.get(REPLICATE_COLLECTION_API_URL, headers=headers)
if response.status_code != 200:
raise Exception(f"Failed to fetch collections: {response.status_code}")
collections = response.json()
return collections
@cached(TTLCache(maxsize=100, ttl=long_ttl_cache))
def get_replicate_collection_models(collection_slug: str, cursor=None):
api_token = get_replicate_api_key()
if not api_token:
raise Exception("Replicate API token not found in environment variables")
headers = {"Authorization": f"Token {api_token}"}
url = f"{REPLICATE_COLLECTION_API_URL}/{collection_slug}"
if cursor:
url += f"?cursor={cursor}"
response = requests.get(url=url, headers=headers)
if response.status_code != 200:
raise Exception(
f"Failed to fetch collections: {collection_slug} - {response.status_code}"
)
data = response.json()
for model in data["models"]:
if "latest_version" in model:
del model["latest_version"]
if "default_example" in model:
del model["default_example"]
return data
@cached(TTLCache(maxsize=100, ttl=very_long_ttl_cache))
def get_highlighted_models_info():
models_str = os.getenv("REPLICATE_MODELS_HIGHLIGHTED", None)
models = []
if models_str:
models = models_str.split(",")
models = [model.strip() for model in models]
else:
return None
models_info = []
for model in models:
try:
info = get_model_info(model)
models_info.append(info)
except Exception as e:
logging.error(f"Failed to fetch model info for {model}: {e}")
continue
return models_info
@cached(TTLCache(maxsize=100, ttl=long_ttl_cache))
def get_model_info(model_id: str):
api_token = get_replicate_api_key()
url = f"{REPLICATE_MODEL_API_URL}/{model_id}"
headers = {"Authorization": f"Token {api_token}"}
response = requests.get(url, headers=headers)
if response.status_code != 200:
raise Exception(
f"Failed to fetch model info: {model_id} - {response.status_code}"
)
model = response.json()
if "latest_version" in model:
del model["latest_version"]
if "default_example" in model:
del model["default_example"]
return model
@cached(TTLCache(maxsize=100, ttl=long_ttl_cache))
def get_model_openapi_schema(model_id: str):
api_token = get_replicate_api_key()
url = f"{REPLICATE_MODEL_API_URL}/{model_id}"
headers = {"Authorization": f"Token {api_token}"}
response = requests.get(url, headers=headers)
if response.status_code != 200:
raise Exception(
f"Failed to fetch model schema: {model_id} - {response.status_code}"
)
version = response.json().get("latest_version")
schema = version.get("openapi_schema", None)
if not schema:
raise Exception(
f"OpenAPI schema not found in the response - model_id : {model_id}"
)
return {
"schema": schema,
"modelId": version["id"],
}
def get_input_schema_from_open_API_schema(openapi_schema):
input_schema = openapi_schema["components"]["schemas"]["Input"]
return input_schema
def get_output_schema_from_open_API_schema(openapi_schema):
output_schema = openapi_schema["components"]["schemas"]["Output"]
return output_schema
================================================
FILE: packages/backend/app/utils/web_scrapping/async_browser_manager.py
================================================
import logging
import asyncio
from asyncio import Queue
from queue import Empty
import tempfile
import time
import zipfile
from ...env_config import get_browser_tab_max_usage, get_browser_tab_pool_size
from playwright.async_api import async_playwright
class AsyncBrowserManager:
def __init__(self):
self.playwright = None
self.browser = None
self.pool_size = get_browser_tab_pool_size()
self.max_usage = get_browser_tab_max_usage()
self.lock = asyncio.Lock()
self.tab_pool = Queue(maxsize=self.pool_size)
self.tab_usage_count = {}
async def initialize_browser(self):
self.playwright = await async_playwright().start()
await self.initialize_pool()
logging.info("Browser initialized")
def unzip_extension(zip_path, extract_to):
with zipfile.ZipFile(zip_path, "r") as zip_ref:
zip_ref.extractall(extract_to)
async def launch_context(self):
user_data_dir = tempfile.mkdtemp()
args = []
args.append("--headless=new")
context = await self.playwright.chromium.launch_persistent_context(
user_data_dir,
headless=False,
args=args,
viewport={"width": 1920, "height": 1080},
)
return context
async def initialize_pool(self):
for _ in range(self.pool_size):
context = await self.launch_context()
page = await context.new_page()
await self.tab_pool.put((page, context))
self.tab_usage_count[page] = 0
async def check_extensions_loaded(self, take_extensions_screenshot=False):
page, context = await self.get_tab()
await page.goto("chrome://extensions/")
# Wait for the extensions list to load
await page.wait_for_selector("extensions-manager")
# Extract the extensions displayed
extensions = await page.evaluate(
"""() => {
return new Promise((resolve, reject) => {
try {
chrome.management.getAll((extensions) => {
resolve(extensions);
});
} catch (error) {
reject(error);
}
});
}"""
)
extension_names = [ext["name"] for ext in extensions]
logging.info(f"Extensions loaded in Chromium : {extension_names}")
if take_extensions_screenshot:
screenshot_path = "extensions_screenshot.png"
await page.screenshot(path=screenshot_path)
logging.info(f"Screenshot saved to {screenshot_path}")
async def close_browser(self):
if self.browser:
await self.browser.close()
if self.playwright:
await self.playwright.stop()
async def _recycle_tab(self, page, context):
try:
await context.close() # Close existing context to free memory
except Exception as e:
logging.error(f"Error closing context: {e}")
context = await self.launch_context()
page = await context.new_page()
self.tab_usage_count[page] = 1
return page, context
async def get_tab(self, timeout=10):
start_time = time.time()
while time.time() - start_time < timeout:
try:
async with self.lock:
page, context = await self.tab_pool.get()
self.tab_usage_count[page] += 1
if self.tab_usage_count[page] > self.max_usage:
# Recycle tab if max usage is reached
page, context = await self._recycle_tab(page, context)
return page, context
except Empty:
await asyncio.sleep(0.1) # Yield control to other tasks
raise Exception("No available tabs in the pool after waiting")
async def release_tab(self, page, context):
async with self.lock:
try:
await context.clear_cookies()
await self.tab_pool.put((page, context))
except Exception as e:
logging.error(f"Error releasing tab: {e}")
# Recycle the tab if an error occurs during release
page, context = await self._recycle_tab(page, context)
await self.tab_pool.put((page, context))
================================================
FILE: packages/backend/app/utils/web_scrapping/browser_manager.py
================================================
import logging
import os
from queue import Empty, Queue
import time
import threading
from ...env_config import get_browser_tab_max_usage, get_browser_tab_pool_size
from playwright.sync_api import sync_playwright
class BrowserManager:
def __init__(self, pool_size=None, max_usage=None):
self.playwright = None
self.browser = None
self.pool_size = get_browser_tab_pool_size() if pool_size is None else pool_size
self.max_usage = get_browser_tab_max_usage() if max_usage is None else max_usage
self.lock = threading.Lock()
self.tab_pool = Queue(maxsize=self.pool_size)
self.tab_usage_count = {}
def initialize_browser(self):
self.playwright = sync_playwright().start()
self.browser = self.playwright.chromium.launch(headless=True)
self.initialize_pool()
logging.info("Browser initialized")
def initialize_pool(self):
for _ in range(self.pool_size):
context = self.browser.new_context()
page = context.new_page()
self.tab_pool.put((page, context))
self.tab_usage_count[page] = 0
def close_browser(self):
if self.browser:
self.browser.close()
if self.playwright:
self.playwright.stop()
def get_browser(self):
if not self.browser:
raise Exception("Browser not initialized")
return self.browser
def _recycle_tab(self, page, context):
context.close()
context = self.browser.new_context()
page = context.new_page()
self.tab_usage_count[page] = 1
return page, context
def get_tab(self, timeout=10):
start_time = time.time()
while time.time() - start_time < timeout:
try:
with self.lock:
page, context = self.tab_pool.get_nowait()
self.tab_usage_count[page] += 1
if self.tab_usage_count[page] > self.max_usage:
# Recycle tab if max usage is reached
page, context = self._recycle_tab(page, context)
return page, context
except Empty:
time.sleep(0.1) # Yield control to other threads
raise Exception("No available tabs in the pool after waiting")
def release_tab(self, page, context):
with self.lock:
page.goto("about:blank") # Clear the page by navigating to a blank page
context.clear_cookies()
self.tab_pool.put((page, context))
================================================
FILE: packages/backend/config.yaml
================================================
core:
openai_api_key:
tag: "core"
description: "API key for accessing OpenAI services."
stabilityai_api_key:
tag: "core"
description: "API key for accessing Stability AI services."
replicate_api_key:
tag: "core"
description: "API key for accessing Replicate services."
extension:
anthropic_api_key:
tag: "core"
description: "API key for accessing Anthropic services."
deepseek_api_key:
tag: "core"
description: "API key for accessing DeepSeek services."
openrouter_api_key:
tag: "core"
description: "API key for accessing OpenRouter services."
================================================
FILE: packages/backend/hooks/hook-app.processors.py
================================================
from PyInstaller.utils.hooks import collect_submodules
hiddenimports = collect_submodules('app.processors')
================================================
FILE: packages/backend/pyproject.toml
================================================
[tool.poetry]
name = "ai-flow-back"
version = "0.11.3"
description = ""
authors = ["DahnM20 "]
readme = "README.md"
packages = [{ include = "app" }]
[tool.poetry.dependencies]
python = ">=3.9,<3.12"
python-dotenv = "^1.0.0"
openai = "1.76.0"
flask = "^2.2.3"
flask-socketio = "^5.3.3"
flask-cors = "^3.0.10"
unstructured = "^0.6.3"
langchain = ">=0.0.303"
python-magic = "^0.4.14"
pytesseract = "^0.3.10"
requests = "^2.31.0"
tabulate = "^0.9.0"
pdf2image = "^1.16.3"
colorlog = "^6.7.0"
eventlet = "^0.33.3"
playwright = "1.39"
youtube-transcript-api = "^0.6.1"
pytube = "^15.0.0"
boto3 = "^1.28.52"
pyjwt = "^2.8.0"
jwcrypto = "^1.5.0"
python-jose = "^3.3.0"
cachetools = "^5.3.1"
flask-injector = "^0.15.0"
setuptools = "^68.2.2"
pypdf = "4.2.0"
replicate = "0.22.0"
tiktoken = "0.7.0"
openapi-spec-validator = "^0.7.1"
anthropic = "0.49.0"
pymupdf = "^1.24.7"
pydub = "^0.25.1"
markdownify = "^1.1.0"
beautifulsoup4 = "^4.13.3"
[tool.poetry.group.dev.dependencies]
datamodel-code-generator = "^0.25.5"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
================================================
FILE: packages/backend/requirements_windows.txt
================================================
python-magic-bin
================================================
FILE: packages/backend/resources/data/openrouter_models.json
================================================
{"data":[{"id":"deepseek/deepseek-chat","name":"DeepSeek V3","created":1735241320,"description":"DeepSeek-V3 is the latest model from the DeepSeek team, building upon the instruction following and coding abilities of the previous versions. Pre-trained on nearly 15 trillion tokens, the reported evaluations reveal that the model outperforms other open-source models and rivals leading closed-source models. For model details, please visit [the DeepSeek-V3 repo](https://github.com/deepseek-ai/DeepSeek-V3) for more information.","context_length":64000,"architecture":{"modality":"text->text","tokenizer":"Other","instruct_type":null},"pricing":{"prompt":"0.00000014","completion":"0.00000028","image":"0","request":"0"},"top_provider":{"context_length":64000,"max_completion_tokens":8192,"is_moderated":false},"per_request_limits":null},{"id":"qwen/qvq-72b-preview","name":"Qwen: QvQ 72B Preview","created":1735088567,"description":"QVQ-72B-Preview is an experimental research model developed by the [Qwen](/qwen) team, focusing on enhancing visual reasoning capabilities.\n\n## Performance\n\n| | **QVQ-72B-Preview** | o1-2024-12-17 | gpt-4o-2024-05-13 | Claude3.5 Sonnet-20241022 | Qwen2VL-72B |\n|----------------|-----------------|---------------|-------------------|----------------------------|-------------|\n| MMMU(val) | 70.3 | 77.3 | 69.1 | 70.4 | 64.5 |\n| MathVista(mini) | 71.4 | 71.0 | 63.8 | 65.3 | 70.5 |\n| MathVision(full) | 35.9 | – | 30.4 | 35.6 | 25.9 |\n| OlympiadBench | 20.4 | – | 25.9 | – | 11.2 |\n\n\n## Limitations\n\n1. **Language Mixing and Code-Switching:** The model might occasionally mix different languages or unexpectedly switch between them, potentially affecting the clarity of its responses.\n2. **Recursive Reasoning Loops:** There's a risk of the model getting caught in recursive reasoning loops, leading to lengthy responses that may not even arrive at a final answer.\n3. **Safety and Ethical Considerations:** Robust safety measures are needed to ensure reliable and safe performance. Users should exercise caution when deploying this model.\n4. **Performance and Benchmark Limitations:** Despite the improvements in visual reasoning, QVQ doesn’t entirely replace the capabilities of [Qwen2-VL-72B](/qwen/qwen-2-vl-72b-instruct). During multi-step visual reasoning, the model might gradually lose focus on the image content, leading to hallucinations. Moreover, QVQ doesn’t show significant improvement over [Qwen2-VL-72B](/qwen/qwen-2-vl-72b-instruct) in basic recognition tasks like identifying people, animals, or plants.\n\nNote: Currently, the model only supports single-round dialogues and image outputs. It does not support video inputs.","context_length":128000,"architecture":{"modality":"text+image->text","tokenizer":"Qwen","instruct_type":null},"pricing":{"prompt":"0.00000025","completion":"0.0000005","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"google/gemini-2.0-flash-thinking-exp:free","name":"Google: Gemini 2.0 Flash Thinking Experimental (free)","created":1734650026,"description":"Gemini 2.0 Flash Thinking Mode is an experimental model that's trained to generate the \"thinking process\" the model goes through as part of its response. As a result, Thinking Mode is capable of stronger reasoning capabilities in its responses than the [base Gemini 2.0 Flash model](/google/gemini-2.0-flash-exp).","context_length":40000,"architecture":{"modality":"text+image->text","tokenizer":"Gemini","instruct_type":null},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":40000,"max_completion_tokens":8000,"is_moderated":false},"per_request_limits":null},{"id":"sao10k/l3.3-euryale-70b","name":"Sao10K: Llama 3.3 Euryale 70B","created":1734535928,"description":"Euryale L3.3 70B is a model focused on creative roleplay from [Sao10k](https://ko-fi.com/sao10k). It is the successor of [Euryale L3 70B v2.2](/models/sao10k/l3-euryale-70b).","context_length":16384,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.0000015","completion":"0.0000015","image":"0","request":"0"},"top_provider":{"context_length":16384,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"inflatebot/mn-mag-mell-r1","name":"Inflatebot: Mag Mell R1 12B","created":1734535439,"description":"Mag Mell is a merge of pre-trained language models created using mergekit, based on [Mistral Nemo](/mistralai/mistral-nemo). It is a great roleplay and storytelling model which combines the best parts of many other models to be a general purpose solution for many usecases.\n\nIntended to be a general purpose \"Best of Nemo\" model for any fictional, creative use case. \n\nMag Mell is composed of 3 intermediate parts:\n- Hero (RP, trope coverage)\n- Monk (Intelligence, groundedness)\n- Deity (Prose, flair)","context_length":16000,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"chatml"},"pricing":{"prompt":"0.0000009","completion":"0.0000009","image":"0","request":"0"},"top_provider":{"context_length":16000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"openai/o1","name":"OpenAI: o1","created":1734459999,"description":"The latest and strongest model family from OpenAI, o1 is designed to spend more time thinking before responding. The o1 model series is trained with large-scale reinforcement learning to reason using chain of thought. \n\nThe o1 models are optimized for math, science, programming, and other STEM-related tasks. They consistently exhibit PhD-level accuracy on benchmarks in physics, chemistry, and biology. Learn more in the [launch announcement](https://openai.com/o1).\n","context_length":200000,"architecture":{"modality":"text+image->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.000015","completion":"0.00006","image":"0.021675","request":"0"},"top_provider":{"context_length":200000,"max_completion_tokens":100000,"is_moderated":true},"per_request_limits":null},{"id":"eva-unit-01/eva-llama-3.33-70b","name":"EVA Llama 3.33 70b","created":1734377303,"description":"EVA Llama 3.33 70b is a roleplay and storywriting specialist model. It is a full-parameter finetune of [Llama-3.3-70B-Instruct](https://openrouter.ai/meta-llama/llama-3.3-70b-instruct) on mixture of synthetic and natural data.\n\nIt uses Celeste 70B 0.1 data mixture, greatly expanding it to improve versatility, creativity and \"flavor\" of the resulting model\n\nThis model was built with Llama by Meta.\n","context_length":16384,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.000004","completion":"0.000006","image":"0","request":"0"},"top_provider":{"context_length":16384,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"x-ai/grok-2-vision-1212","name":"xAI: Grok 2 Vision 1212","created":1734237338,"description":"Grok 2 Vision 1212 advances image-based AI with stronger visual comprehension, refined instruction-following, and multilingual support. From object recognition to style analysis, it empowers developers to build more intuitive, visually aware applications. Its enhanced steerability and reasoning establish a robust foundation for next-generation image solutions.\n\nTo read more about this model, check out [xAI's announcement](https://x.ai/blog/grok-1212).","context_length":32768,"architecture":{"modality":"text+image->text","tokenizer":"Grok","instruct_type":null},"pricing":{"prompt":"0.000002","completion":"0.00001","image":"0.0036","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"x-ai/grok-2-1212","name":"xAI: Grok 2 1212","created":1734232814,"description":"Grok 2 1212 introduces significant enhancements to accuracy, instruction adherence, and multilingual support, making it a powerful and flexible choice for developers seeking a highly steerable, intelligent model.","context_length":131072,"architecture":{"modality":"text->text","tokenizer":"Grok","instruct_type":null},"pricing":{"prompt":"0.000002","completion":"0.00001","image":"0","request":"0"},"top_provider":{"context_length":131072,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"cohere/command-r7b-12-2024","name":"Cohere: Command R7B (12-2024)","created":1734158152,"description":"Command R7B (12-2024) is a small, fast update of the Command R+ model, delivered in December 2024. It excels at RAG, tool use, agents, and similar tasks requiring complex reasoning and multiple steps.","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"Cohere","instruct_type":null},"pricing":{"prompt":"0.0000000375","completion":"0.00000015","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":4000,"is_moderated":false},"per_request_limits":null},{"id":"google/gemini-2.0-flash-exp:free","name":"Google: Gemini Flash 2.0 Experimental (free)","created":1733937523,"description":"Gemini Flash 2.0 offers a significantly faster time to first token (TTFT) compared to [Gemini Flash 1.5](google/gemini-flash-1.5), while maintaining quality on par with larger models like [Gemini Pro 1.5](google/gemini-pro-1.5). It introduces notable enhancements in multimodal understanding, coding capabilities, complex instruction following, and function calling. These advancements come together to deliver more seamless and robust agentic experiences.","context_length":1048576,"architecture":{"modality":"text+image->text","tokenizer":"Gemini","instruct_type":null},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":1048576,"max_completion_tokens":8192,"is_moderated":false},"per_request_limits":null},{"id":"google/gemini-exp-1206:free","name":"Google: Gemini Experimental 1206 (free)","created":1733507713,"description":"Experimental release (December 6, 2024) of Gemini.","context_length":2097152,"architecture":{"modality":"text+image->text","tokenizer":"Gemini","instruct_type":null},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":2097152,"max_completion_tokens":8192,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3.3-70b-instruct","name":"Meta: Llama 3.3 70B Instruct","created":1733506137,"description":"The Meta Llama 3.3 multilingual large language model (LLM) is a pretrained and instruction tuned generative model in 70B (text in/text out). The Llama 3.3 instruction tuned text only model is optimized for multilingual dialogue use cases and outperforms many of the available open source and closed chat models on common industry benchmarks.\n\nSupported languages: English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.\n\n[Model Card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md)","context_length":131072,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.00000012","completion":"0.0000003","image":"0","request":"0"},"top_provider":{"context_length":131072,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"amazon/nova-lite-v1","name":"Amazon: Nova Lite 1.0","created":1733437363,"description":"Amazon Nova Lite 1.0 is a very low-cost multimodal model from Amazon that focused on fast processing of image, video, and text inputs to generate text output. Amazon Nova Lite can handle real-time customer interactions, document analysis, and visual question-answering tasks with high accuracy.\n\nWith an input context of 300K tokens, it can analyze multiple images or up to 30 minutes of video in a single input.","context_length":300000,"architecture":{"modality":"text+image->text","tokenizer":"Nova","instruct_type":null},"pricing":{"prompt":"0.00000006","completion":"0.00000024","image":"0.00009","request":"0"},"top_provider":{"context_length":300000,"max_completion_tokens":5120,"is_moderated":false},"per_request_limits":null},{"id":"amazon/nova-micro-v1","name":"Amazon: Nova Micro 1.0","created":1733437237,"description":"Amazon Nova Micro 1.0 is a text-only model that delivers the lowest latency responses in the Amazon Nova family of models at a very low cost. With a context length of 128K tokens and optimized for speed and cost, Amazon Nova Micro excels at tasks such as text summarization, translation, content classification, interactive chat, and brainstorming. It has simple mathematical reasoning and coding abilities.","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"Nova","instruct_type":null},"pricing":{"prompt":"0.000000035","completion":"0.00000014","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":5120,"is_moderated":false},"per_request_limits":null},{"id":"amazon/nova-pro-v1","name":"Amazon: Nova Pro 1.0","created":1733436303,"description":"Amazon Nova Pro 1.0 is a capable multimodal model from Amazon focused on providing a combination of accuracy, speed, and cost for a wide range of tasks. As of December 2024, it achieves state-of-the-art performance on key benchmarks including visual question answering (TextVQA) and video understanding (VATEX).\n\nAmazon Nova Pro demonstrates strong capabilities in processing both visual and textual information and at analyzing financial documents.\n\n**NOTE**: Video input is not supported at this time.","context_length":300000,"architecture":{"modality":"text+image->text","tokenizer":"Nova","instruct_type":null},"pricing":{"prompt":"0.0000008","completion":"0.0000032","image":"0.0012","request":"0"},"top_provider":{"context_length":300000,"max_completion_tokens":5120,"is_moderated":false},"per_request_limits":null},{"id":"qwen/qwq-32b-preview","name":"Qwen: QwQ 32B Preview","created":1732754541,"description":"QwQ-32B-Preview is an experimental research model focused on AI reasoning capabilities developed by the Qwen Team. As a preview release, it demonstrates promising analytical abilities while having several important limitations:\n\n1. **Language Mixing and Code-Switching**: The model may mix languages or switch between them unexpectedly, affecting response clarity.\n2. **Recursive Reasoning Loops**: The model may enter circular reasoning patterns, leading to lengthy responses without a conclusive answer.\n3. **Safety and Ethical Considerations**: The model requires enhanced safety measures to ensure reliable and secure performance, and users should exercise caution when deploying it.\n4. **Performance and Benchmark Limitations**: The model excels in math and coding but has room for improvement in other areas, such as common sense reasoning and nuanced language understanding.\n\n","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Qwen","instruct_type":null},"pricing":{"prompt":"0.00000012","completion":"0.00000018","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"google/gemini-exp-1121:free","name":"Google: Gemini Experimental 1121 (free)","created":1732216725,"description":"Experimental release (November 21st, 2024) of Gemini.","context_length":40960,"architecture":{"modality":"text+image->text","tokenizer":"Gemini","instruct_type":null},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":40960,"max_completion_tokens":8192,"is_moderated":false},"per_request_limits":null},{"id":"google/learnlm-1.5-pro-experimental:free","name":"Google: LearnLM 1.5 Pro Experimental (free)","created":1732216551,"description":"An experimental version of [Gemini 1.5 Pro](/google/gemini-pro-1.5) from Google.","context_length":40960,"architecture":{"modality":"text+image->text","tokenizer":"Gemini","instruct_type":null},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":40960,"max_completion_tokens":8192,"is_moderated":false},"per_request_limits":null},{"id":"eva-unit-01/eva-qwen-2.5-72b","name":"EVA Qwen2.5 72B","created":1732210606,"description":"EVA Qwen2.5 72B is a roleplay and storywriting specialist model. It's a full-parameter finetune of Qwen2.5-72B on mixture of synthetic and natural data.\n\nIt uses Celeste 70B 0.1 data mixture, greatly expanding it to improve versatility, creativity and \"flavor\" of the resulting model.","context_length":16384,"architecture":{"modality":"text->text","tokenizer":"Qwen","instruct_type":"chatml"},"pricing":{"prompt":"0.000004","completion":"0.000006","image":"0","request":"0"},"top_provider":{"context_length":16384,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"openai/gpt-4o-2024-11-20","name":"OpenAI: GPT-4o (2024-11-20)","created":1732127594,"description":"The 2024-11-20 version of GPT-4o offers a leveled-up creative writing ability with more natural, engaging, and tailored writing to improve relevance & readability. It’s also better at working with uploaded files, providing deeper insights & more thorough responses.\n\nGPT-4o (\"o\" for \"omni\") is OpenAI's latest AI model, supporting both text and image inputs with text outputs. It maintains the intelligence level of [GPT-4 Turbo](/models/openai/gpt-4-turbo) while being twice as fast and 50% more cost-effective. GPT-4o also offers improved performance in processing non-English languages and enhanced visual capabilities.","context_length":128000,"architecture":{"modality":"text+image->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.0000025","completion":"0.00001","image":"0.003613","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":16384,"is_moderated":true},"per_request_limits":null},{"id":"mistralai/mistral-large-2411","name":"Mistral Large 2411","created":1731978685,"description":"Mistral Large 2 2411 is an update of [Mistral Large 2](/mistralai/mistral-large) released together with [Pixtral Large 2411](/mistralai/pixtral-large-2411)\n\nIt provides a significant upgrade on the previous [Mistral Large 24.07](/mistralai/mistral-large-2407), with notable improvements in long context understanding, a new system prompt, and more accurate function calling.","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":null},"pricing":{"prompt":"0.000002","completion":"0.000006","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"mistralai/mistral-large-2407","name":"Mistral Large 2407","created":1731978415,"description":"This is Mistral AI's flagship model, Mistral Large 2 (version mistral-large-2407). It's a proprietary weights-available model and excels at reasoning, code, JSON, chat, and more. Read the launch announcement [here](https://mistral.ai/news/mistral-large-2407/).\n\nIt supports dozens of languages including French, German, Spanish, Italian, Portuguese, Arabic, Hindi, Russian, Chinese, Japanese, and Korean, along with 80+ coding languages including Python, Java, C, C++, JavaScript, and Bash. Its long context window allows precise information recall from large documents.\n","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":null},"pricing":{"prompt":"0.000002","completion":"0.000006","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"mistralai/pixtral-large-2411","name":"Mistral: Pixtral Large 2411","created":1731977388,"description":"Pixtral Large is a 124B parameter, open-weight, multimodal model built on top of [Mistral Large 2](/mistralai/mistral-large-2411). The model is able to understand documents, charts and natural images.\n\nThe model is available under the Mistral Research License (MRL) for research and educational use, and the Mistral Commercial License for experimentation, testing, and production for commercial purposes.\n\n","context_length":128000,"architecture":{"modality":"text+image->text","tokenizer":"Mistral","instruct_type":null},"pricing":{"prompt":"0.000002","completion":"0.000006","image":"0.002888","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"x-ai/grok-vision-beta","name":"xAI: Grok Vision Beta","created":1731976624,"description":"Grok Vision Beta is xAI's experimental language model with vision capability.\n\n","context_length":8192,"architecture":{"modality":"text+image->text","tokenizer":"Grok","instruct_type":null},"pricing":{"prompt":"0.000005","completion":"0.000015","image":"0.009","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"google/gemini-exp-1114:free","name":"Google: Gemini Experimental 1114 (free)","created":1731714740,"description":"Gemini 11-14 (2024) experimental model features \"quality\" improvements.","context_length":40960,"architecture":{"modality":"text+image->text","tokenizer":"Gemini","instruct_type":null},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":40960,"max_completion_tokens":8192,"is_moderated":false},"per_request_limits":null},{"id":"infermatic/mn-inferor-12b","name":"Infermatic: Mistral Nemo Inferor 12B","created":1731464428,"description":"Inferor 12B is a merge of top roleplay models, expert on immersive narratives and storytelling.\n\nThis model was merged using the [Model Stock](https://arxiv.org/abs/2403.19522) merge method using [anthracite-org/magnum-v4-12b](https://openrouter.ai/anthracite-org/magnum-v4-72b) as a base.\n","context_length":32000,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"mistral"},"pricing":{"prompt":"0.00000025","completion":"0.0000005","image":"0","request":"0"},"top_provider":{"context_length":32000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"qwen/qwen-2.5-coder-32b-instruct","name":"Qwen2.5 Coder 32B Instruct","created":1731368400,"description":"Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). Qwen2.5-Coder brings the following improvements upon CodeQwen1.5:\n\n- Significantly improvements in **code generation**, **code reasoning** and **code fixing**. \n- A more comprehensive foundation for real-world applications such as **Code Agents**. Not only enhancing coding capabilities but also maintaining its strengths in mathematics and general competencies.\n\nTo read more about its evaluation results, check out [Qwen 2.5 Coder's blog](https://qwenlm.github.io/blog/qwen2.5-coder-family/).","context_length":33000,"architecture":{"modality":"text->text","tokenizer":"Qwen","instruct_type":"chatml"},"pricing":{"prompt":"0.00000007","completion":"0.00000016","image":"0","request":"0"},"top_provider":{"context_length":33000,"max_completion_tokens":3000,"is_moderated":false},"per_request_limits":null},{"id":"raifle/sorcererlm-8x22b","name":"SorcererLM 8x22B","created":1731105083,"description":"SorcererLM is an advanced RP and storytelling model, built as a Low-rank 16-bit LoRA fine-tuned on [WizardLM-2 8x22B](/microsoft/wizardlm-2-8x22b).\n\n- Advanced reasoning and emotional intelligence for engaging and immersive interactions\n- Vivid writing capabilities enriched with spatial and contextual awareness\n- Enhanced narrative depth, promoting creative and dynamic storytelling","context_length":16000,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"vicuna"},"pricing":{"prompt":"0.0000045","completion":"0.0000045","image":"0","request":"0"},"top_provider":{"context_length":16000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"eva-unit-01/eva-qwen-2.5-32b","name":"EVA Qwen2.5 32B","created":1731104847,"description":"EVA Qwen2.5 32B is a roleplaying/storywriting specialist model. It's a full-parameter finetune of Qwen2.5-32B on mixture of synthetic and natural data.\n\nIt uses Celeste 70B 0.1 data mixture, greatly expanding it to improve versatility, creativity and \"flavor\" of the resulting model.","context_length":16384,"architecture":{"modality":"text->text","tokenizer":"Qwen","instruct_type":"chatml"},"pricing":{"prompt":"0.0000026","completion":"0.0000034","image":"0","request":"0"},"top_provider":{"context_length":16384,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"thedrummer/unslopnemo-12b","name":"Unslopnemo 12b","created":1731103448,"description":"UnslopNemo v4.1 is the latest addition from the creator of Rocinante, designed for adventure writing and role-play scenarios.","context_length":32000,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"mistral"},"pricing":{"prompt":"0.0000005","completion":"0.0000005","image":"0","request":"0"},"top_provider":{"context_length":32000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"anthropic/claude-3.5-haiku-20241022:beta","name":"Anthropic: Claude 3.5 Haiku (2024-10-22) (self-moderated)","created":1730678400,"description":"Claude 3.5 Haiku features enhancements across all skill sets including coding, tool use, and reasoning. As the fastest model in the Anthropic lineup, it offers rapid response times suitable for applications that require high interactivity and low latency, such as user-facing chatbots and on-the-fly code completions. It also excels in specialized tasks like data extraction and real-time content moderation, making it a versatile tool for a broad range of industries.\n\nIt does not support image inputs.\n\nSee the launch announcement and benchmark results [here](https://www.anthropic.com/news/3-5-models-and-computer-use)","context_length":200000,"architecture":{"modality":"text->text","tokenizer":"Claude","instruct_type":null},"pricing":{"prompt":"0.0000008","completion":"0.000004","image":"0","request":"0"},"top_provider":{"context_length":200000,"max_completion_tokens":8192,"is_moderated":false},"per_request_limits":null},{"id":"anthropic/claude-3.5-haiku-20241022","name":"Anthropic: Claude 3.5 Haiku (2024-10-22)","created":1730678400,"description":"Claude 3.5 Haiku features enhancements across all skill sets including coding, tool use, and reasoning. As the fastest model in the Anthropic lineup, it offers rapid response times suitable for applications that require high interactivity and low latency, such as user-facing chatbots and on-the-fly code completions. It also excels in specialized tasks like data extraction and real-time content moderation, making it a versatile tool for a broad range of industries.\n\nIt does not support image inputs.\n\nSee the launch announcement and benchmark results [here](https://www.anthropic.com/news/3-5-models-and-computer-use)","context_length":200000,"architecture":{"modality":"text->text","tokenizer":"Claude","instruct_type":null},"pricing":{"prompt":"0.0000008","completion":"0.000004","image":"0","request":"0"},"top_provider":{"context_length":200000,"max_completion_tokens":8192,"is_moderated":true},"per_request_limits":null},{"id":"anthropic/claude-3.5-haiku:beta","name":"Anthropic: Claude 3.5 Haiku (self-moderated)","created":1730678400,"description":"Claude 3.5 Haiku features offers enhanced capabilities in speed, coding accuracy, and tool use. Engineered to excel in real-time applications, it delivers quick response times that are essential for dynamic tasks such as chat interactions and immediate coding suggestions.\n\nThis makes it highly suitable for environments that demand both speed and precision, such as software development, customer service bots, and data management systems.\n\nThis model is currently pointing to [Claude 3.5 Haiku (2024-10-22)](/anthropic/claude-3-5-haiku-20241022).","context_length":200000,"architecture":{"modality":"text->text","tokenizer":"Claude","instruct_type":null},"pricing":{"prompt":"0.0000008","completion":"0.000004","image":"0","request":"0"},"top_provider":{"context_length":200000,"max_completion_tokens":8192,"is_moderated":false},"per_request_limits":null},{"id":"anthropic/claude-3.5-haiku","name":"Anthropic: Claude 3.5 Haiku","created":1730678400,"description":"Claude 3.5 Haiku features offers enhanced capabilities in speed, coding accuracy, and tool use. Engineered to excel in real-time applications, it delivers quick response times that are essential for dynamic tasks such as chat interactions and immediate coding suggestions.\n\nThis makes it highly suitable for environments that demand both speed and precision, such as software development, customer service bots, and data management systems.\n\nThis model is currently pointing to [Claude 3.5 Haiku (2024-10-22)](/anthropic/claude-3-5-haiku-20241022).","context_length":200000,"architecture":{"modality":"text->text","tokenizer":"Claude","instruct_type":null},"pricing":{"prompt":"0.0000008","completion":"0.000004","image":"0","request":"0"},"top_provider":{"context_length":200000,"max_completion_tokens":8192,"is_moderated":true},"per_request_limits":null},{"id":"neversleep/llama-3.1-lumimaid-70b","name":"NeverSleep: Lumimaid v0.2 70B","created":1729555200,"description":"Lumimaid v0.2 70B is a finetune of [Llama 3.1 70B](/meta-llama/llama-3.1-70b-instruct) with a \"HUGE step up dataset wise\" compared to Lumimaid v0.1. Sloppy chats output were purged.\n\nUsage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":16384,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.000003375","completion":"0.0000045","image":"0","request":"0"},"top_provider":{"context_length":16384,"max_completion_tokens":2048,"is_moderated":false},"per_request_limits":null},{"id":"anthracite-org/magnum-v4-72b","name":"Magnum v4 72B","created":1729555200,"description":"This is a series of models designed to replicate the prose quality of the Claude 3 models, specifically Sonnet(https://openrouter.ai/anthropic/claude-3.5-sonnet) and Opus(https://openrouter.ai/anthropic/claude-3-opus).\n\nThe model is fine-tuned on top of [Qwen2.5 72B](https://openrouter.ai/qwen/qwen-2.5-72b-instruct).","context_length":16384,"architecture":{"modality":"text->text","tokenizer":"Qwen","instruct_type":"chatml"},"pricing":{"prompt":"0.000001875","completion":"0.00000225","image":"0","request":"0"},"top_provider":{"context_length":16384,"max_completion_tokens":1024,"is_moderated":false},"per_request_limits":null},{"id":"anthropic/claude-3.5-sonnet:beta","name":"Anthropic: Claude 3.5 Sonnet (self-moderated)","created":1729555200,"description":"New Claude 3.5 Sonnet delivers better-than-Opus capabilities, faster-than-Sonnet speeds, at the same Sonnet prices. Sonnet is particularly good at:\n\n- Coding: Scores ~49% on SWE-Bench Verified, higher than the last best score, and without any fancy prompt scaffolding\n- Data science: Augments human data science expertise; navigates unstructured data while using multiple tools for insights\n- Visual processing: excelling at interpreting charts, graphs, and images, accurately transcribing text to derive insights beyond just the text alone\n- Agentic tasks: exceptional tool use, making it great at agentic tasks (i.e. complex, multi-step problem solving tasks that require engaging with other systems)\n\n#multimodal","context_length":200000,"architecture":{"modality":"text+image->text","tokenizer":"Claude","instruct_type":null},"pricing":{"prompt":"0.000003","completion":"0.000015","image":"0.0048","request":"0"},"top_provider":{"context_length":200000,"max_completion_tokens":8192,"is_moderated":false},"per_request_limits":null},{"id":"anthropic/claude-3.5-sonnet","name":"Anthropic: Claude 3.5 Sonnet","created":1729555200,"description":"New Claude 3.5 Sonnet delivers better-than-Opus capabilities, faster-than-Sonnet speeds, at the same Sonnet prices. Sonnet is particularly good at:\n\n- Coding: Scores ~49% on SWE-Bench Verified, higher than the last best score, and without any fancy prompt scaffolding\n- Data science: Augments human data science expertise; navigates unstructured data while using multiple tools for insights\n- Visual processing: excelling at interpreting charts, graphs, and images, accurately transcribing text to derive insights beyond just the text alone\n- Agentic tasks: exceptional tool use, making it great at agentic tasks (i.e. complex, multi-step problem solving tasks that require engaging with other systems)\n\n#multimodal","context_length":200000,"architecture":{"modality":"text+image->text","tokenizer":"Claude","instruct_type":null},"pricing":{"prompt":"0.000003","completion":"0.000015","image":"0.0048","request":"0"},"top_provider":{"context_length":200000,"max_completion_tokens":8192,"is_moderated":true},"per_request_limits":null},{"id":"x-ai/grok-beta","name":"xAI: Grok Beta","created":1729382400,"description":"Grok Beta is xAI's experimental language model with state-of-the-art reasoning capabilities, best for complex and multi-step use cases.\n\nIt is the successor of [Grok 2](https://x.ai/blog/grok-2) with enhanced context length.","context_length":131072,"architecture":{"modality":"text->text","tokenizer":"Grok","instruct_type":null},"pricing":{"prompt":"0.000005","completion":"0.000015","image":"0","request":"0"},"top_provider":{"context_length":131072,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"mistralai/ministral-8b","name":"Mistral: Ministral 8B","created":1729123200,"description":"Ministral 8B is an 8B parameter model featuring a unique interleaved sliding-window attention pattern for faster, memory-efficient inference. Designed for edge use cases, it supports up to 128k context length and excels in knowledge and reasoning tasks. It outperforms peers in the sub-10B category, making it perfect for low-latency, privacy-first applications.","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":null},"pricing":{"prompt":"0.0000001","completion":"0.0000001","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"mistralai/ministral-3b","name":"Mistral: Ministral 3B","created":1729123200,"description":"Ministral 3B is a 3B parameter model optimized for on-device and edge computing. It excels in knowledge, commonsense reasoning, and function-calling, outperforming larger models like Mistral 7B on most benchmarks. Supporting up to 128k context length, it’s ideal for orchestrating agentic workflows and specialist tasks with efficient inference.","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":null},"pricing":{"prompt":"0.00000004","completion":"0.00000004","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"qwen/qwen-2.5-7b-instruct","name":"Qwen2.5 7B Instruct","created":1729036800,"description":"Qwen2.5 7B is the latest series of Qwen large language models. Qwen2.5 brings the following improvements upon Qwen2:\n\n- Significantly more knowledge and has greatly improved capabilities in coding and mathematics, thanks to our specialized expert models in these domains.\n\n- Significant improvements in instruction following, generating long texts (over 8K tokens), understanding structured data (e.g, tables), and generating structured outputs especially JSON. More resilient to the diversity of system prompts, enhancing role-play implementation and condition-setting for chatbots.\n\n- Long-context Support up to 128K tokens and can generate up to 8K tokens.\n\n- Multilingual support for over 29 languages, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, Arabic, and more.\n\nUsage of this model is subject to [Tongyi Qianwen LICENSE AGREEMENT](https://huggingface.co/Qwen/Qwen1.5-110B-Chat/blob/main/LICENSE).","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Qwen","instruct_type":"chatml"},"pricing":{"prompt":"0.00000027","completion":"0.00000027","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"nvidia/llama-3.1-nemotron-70b-instruct","name":"NVIDIA: Llama 3.1 Nemotron 70B Instruct","created":1728950400,"description":"NVIDIA's Llama 3.1 Nemotron 70B is a language model designed for generating precise and useful responses. Leveraging [Llama 3.1 70B](/models/meta-llama/llama-3.1-70b-instruct) architecture and Reinforcement Learning from Human Feedback (RLHF), it excels in automatic alignment benchmarks. This model is tailored for applications requiring high accuracy in helpfulness and response generation, suitable for diverse user queries across multiple domains.\n\nUsage of this model is subject to [Meta's Acceptable Use Policy](https://www.llama.com/llama3/use-policy/).","context_length":131000,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.00000012","completion":"0.0000003","image":"0","request":"0"},"top_provider":{"context_length":131000,"max_completion_tokens":131000,"is_moderated":false},"per_request_limits":null},{"id":"inflection/inflection-3-pi","name":"Inflection: Inflection 3 Pi","created":1728604800,"description":"Inflection 3 Pi powers Inflection's [Pi](https://pi.ai) chatbot, including backstory, emotional intelligence, productivity, and safety. It has access to recent news, and excels in scenarios like customer support and roleplay.\n\nPi has been trained to mirror your tone and style, if you use more emojis, so will Pi! Try experimenting with various prompts and conversation styles.","context_length":8000,"architecture":{"modality":"text->text","tokenizer":"Other","instruct_type":null},"pricing":{"prompt":"0.0000025","completion":"0.00001","image":"0","request":"0"},"top_provider":{"context_length":8000,"max_completion_tokens":1024,"is_moderated":false},"per_request_limits":null},{"id":"inflection/inflection-3-productivity","name":"Inflection: Inflection 3 Productivity","created":1728604800,"description":"Inflection 3 Productivity is optimized for following instructions. It is better for tasks requiring JSON output or precise adherence to provided guidelines. It has access to recent news.\n\nFor emotional intelligence similar to Pi, see [Inflect 3 Pi](/inflection/inflection-3-pi)\n\nSee [Inflection's announcement](https://inflection.ai/blog/enterprise) for more details.","context_length":8000,"architecture":{"modality":"text->text","tokenizer":"Other","instruct_type":null},"pricing":{"prompt":"0.0000025","completion":"0.00001","image":"0","request":"0"},"top_provider":{"context_length":8000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"google/gemini-flash-1.5-8b","name":"Google: Gemini Flash 1.5 8B","created":1727913600,"description":"Gemini Flash 1.5 8B is optimized for speed and efficiency, offering enhanced performance in small prompt tasks like chat, transcription, and translation. With reduced latency, it is highly effective for real-time and large-scale operations. This model focuses on cost-effective solutions while maintaining high-quality results.\n\n[Click here to learn more about this model](https://developers.googleblog.com/en/gemini-15-flash-8b-is-now-generally-available-for-use/).\n\nUsage of Gemini is subject to Google's [Gemini Terms of Use](https://ai.google.dev/terms).","context_length":1000000,"architecture":{"modality":"text+image->text","tokenizer":"Gemini","instruct_type":null},"pricing":{"prompt":"0.0000000375","completion":"0.00000015","image":"0","request":"0"},"top_provider":{"context_length":1000000,"max_completion_tokens":8192,"is_moderated":false},"per_request_limits":null},{"id":"anthracite-org/magnum-v2-72b","name":"Magnum v2 72B","created":1727654400,"description":"From the maker of [Goliath](https://openrouter.ai/models/alpindale/goliath-120b), Magnum 72B is the seventh in a family of models designed to achieve the prose quality of the Claude 3 models, notably Opus & Sonnet.\n\nThe model is based on [Qwen2 72B](https://openrouter.ai/models/qwen/qwen-2-72b-instruct) and trained with 55 million tokens of highly curated roleplay (RP) data.","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Qwen","instruct_type":"chatml"},"pricing":{"prompt":"0.000003","completion":"0.000003","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"liquid/lfm-40b","name":"Liquid: LFM 40B MoE","created":1727654400,"description":"Liquid's 40.3B Mixture of Experts (MoE) model. Liquid Foundation Models (LFMs) are large neural networks built with computational units rooted in dynamic systems.\n\nLFMs are general-purpose AI models that can be used to model any kind of sequential data, including video, audio, text, time series, and signals.\n\nSee the [launch announcement](https://www.liquid.ai/liquid-foundation-models) for benchmarks and more info.","context_length":66000,"architecture":{"modality":"text->text","tokenizer":"Other","instruct_type":"vicuna"},"pricing":{"prompt":"0.00000015","completion":"0.00000015","image":"0","request":"0"},"top_provider":{"context_length":66000,"max_completion_tokens":66000,"is_moderated":false},"per_request_limits":null},{"id":"thedrummer/rocinante-12b","name":"Rocinante 12B","created":1727654400,"description":"Rocinante 12B is designed for engaging storytelling and rich prose.\n\nEarly testers have reported:\n- Expanded vocabulary with unique and expressive word choices\n- Enhanced creativity for vivid narratives\n- Adventure-filled and captivating stories","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Qwen","instruct_type":"chatml"},"pricing":{"prompt":"0.00000025","completion":"0.0000005","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3.2-3b-instruct:free","name":"Meta: Llama 3.2 3B Instruct (free)","created":1727222400,"description":"Llama 3.2 3B is a 3-billion-parameter multilingual large language model, optimized for advanced natural language processing tasks like dialogue generation, reasoning, and summarization. Designed with the latest transformer architecture, it supports eight languages, including English, Spanish, and Hindi, and is adaptable for additional languages.\n\nTrained on 9 trillion tokens, the Llama 3.2 3B model excels in instruction-following, complex reasoning, and tool use. Its balanced performance makes it ideal for applications needing accuracy and efficiency in text generation across multilingual settings.\n\nClick here for the [original model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/MODEL_CARD.md).\n\nUsage of this model is subject to [Meta's Acceptable Use Policy](https://www.llama.com/llama3/use-policy/).","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":2048,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3.2-3b-instruct","name":"Meta: Llama 3.2 3B Instruct","created":1727222400,"description":"Llama 3.2 3B is a 3-billion-parameter multilingual large language model, optimized for advanced natural language processing tasks like dialogue generation, reasoning, and summarization. Designed with the latest transformer architecture, it supports eight languages, including English, Spanish, and Hindi, and is adaptable for additional languages.\n\nTrained on 9 trillion tokens, the Llama 3.2 3B model excels in instruction-following, complex reasoning, and tool use. Its balanced performance makes it ideal for applications needing accuracy and efficiency in text generation across multilingual settings.\n\nClick here for the [original model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/MODEL_CARD.md).\n\nUsage of this model is subject to [Meta's Acceptable Use Policy](https://www.llama.com/llama3/use-policy/).","context_length":131000,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.000000015","completion":"0.000000025","image":"0","request":"0"},"top_provider":{"context_length":131000,"max_completion_tokens":131000,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3.2-1b-instruct:free","name":"Meta: Llama 3.2 1B Instruct (free)","created":1727222400,"description":"Llama 3.2 1B is a 1-billion-parameter language model focused on efficiently performing natural language tasks, such as summarization, dialogue, and multilingual text analysis. Its smaller size allows it to operate efficiently in low-resource environments while maintaining strong task performance.\n\nSupporting eight core languages and fine-tunable for more, Llama 1.3B is ideal for businesses or developers seeking lightweight yet powerful AI solutions that can operate in diverse multilingual settings without the high computational demand of larger models.\n\nClick here for the [original model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/MODEL_CARD.md).\n\nUsage of this model is subject to [Meta's Acceptable Use Policy](https://www.llama.com/llama3/use-policy/).","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":2048,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3.2-1b-instruct","name":"Meta: Llama 3.2 1B Instruct","created":1727222400,"description":"Llama 3.2 1B is a 1-billion-parameter language model focused on efficiently performing natural language tasks, such as summarization, dialogue, and multilingual text analysis. Its smaller size allows it to operate efficiently in low-resource environments while maintaining strong task performance.\n\nSupporting eight core languages and fine-tunable for more, Llama 1.3B is ideal for businesses or developers seeking lightweight yet powerful AI solutions that can operate in diverse multilingual settings without the high computational demand of larger models.\n\nClick here for the [original model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/MODEL_CARD.md).\n\nUsage of this model is subject to [Meta's Acceptable Use Policy](https://www.llama.com/llama3/use-policy/).","context_length":131072,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.00000001","completion":"0.00000001","image":"0","request":"0"},"top_provider":{"context_length":131072,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3.2-90b-vision-instruct:free","name":"Meta: Llama 3.2 90B Vision Instruct (free)","created":1727222400,"description":"The Llama 90B Vision model is a top-tier, 90-billion-parameter multimodal model designed for the most challenging visual reasoning and language tasks. It offers unparalleled accuracy in image captioning, visual question answering, and advanced image-text comprehension. Pre-trained on vast multimodal datasets and fine-tuned with human feedback, the Llama 90B Vision is engineered to handle the most demanding image-based AI tasks.\n\nThis model is perfect for industries requiring cutting-edge multimodal AI capabilities, particularly those dealing with complex, real-time visual and textual analysis.\n\nClick here for the [original model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/MODEL_CARD_VISION.md).\n\nUsage of this model is subject to [Meta's Acceptable Use Policy](https://www.llama.com/llama3/use-policy/).","context_length":4096,"architecture":{"modality":"text+image->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":2048,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3.2-90b-vision-instruct","name":"Meta: Llama 3.2 90B Vision Instruct","created":1727222400,"description":"The Llama 90B Vision model is a top-tier, 90-billion-parameter multimodal model designed for the most challenging visual reasoning and language tasks. It offers unparalleled accuracy in image captioning, visual question answering, and advanced image-text comprehension. Pre-trained on vast multimodal datasets and fine-tuned with human feedback, the Llama 90B Vision is engineered to handle the most demanding image-based AI tasks.\n\nThis model is perfect for industries requiring cutting-edge multimodal AI capabilities, particularly those dealing with complex, real-time visual and textual analysis.\n\nClick here for the [original model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/MODEL_CARD_VISION.md).\n\nUsage of this model is subject to [Meta's Acceptable Use Policy](https://www.llama.com/llama3/use-policy/).","context_length":131072,"architecture":{"modality":"text+image->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.0000009","completion":"0.0000009","image":"0.001301","request":"0"},"top_provider":{"context_length":131072,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3.2-11b-vision-instruct:free","name":"Meta: Llama 3.2 11B Vision Instruct (free)","created":1727222400,"description":"Llama 3.2 11B Vision is a multimodal model with 11 billion parameters, designed to handle tasks combining visual and textual data. It excels in tasks such as image captioning and visual question answering, bridging the gap between language generation and visual reasoning. Pre-trained on a massive dataset of image-text pairs, it performs well in complex, high-accuracy image analysis.\n\nIts ability to integrate visual understanding with language processing makes it an ideal solution for industries requiring comprehensive visual-linguistic AI applications, such as content creation, AI-driven customer service, and research.\n\nClick here for the [original model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/MODEL_CARD_VISION.md).\n\nUsage of this model is subject to [Meta's Acceptable Use Policy](https://www.llama.com/llama3/use-policy/).","context_length":8192,"architecture":{"modality":"text+image->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3.2-11b-vision-instruct","name":"Meta: Llama 3.2 11B Vision Instruct","created":1727222400,"description":"Llama 3.2 11B Vision is a multimodal model with 11 billion parameters, designed to handle tasks combining visual and textual data. It excels in tasks such as image captioning and visual question answering, bridging the gap between language generation and visual reasoning. Pre-trained on a massive dataset of image-text pairs, it performs well in complex, high-accuracy image analysis.\n\nIts ability to integrate visual understanding with language processing makes it an ideal solution for industries requiring comprehensive visual-linguistic AI applications, such as content creation, AI-driven customer service, and research.\n\nClick here for the [original model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/MODEL_CARD_VISION.md).\n\nUsage of this model is subject to [Meta's Acceptable Use Policy](https://www.llama.com/llama3/use-policy/).","context_length":131072,"architecture":{"modality":"text+image->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.000000055","completion":"0.000000055","image":"0.00007948","request":"0"},"top_provider":{"context_length":131072,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"qwen/qwen-2.5-72b-instruct","name":"Qwen2.5 72B Instruct","created":1726704000,"description":"Qwen2.5 72B is the latest series of Qwen large language models. Qwen2.5 brings the following improvements upon Qwen2:\n\n- Significantly more knowledge and has greatly improved capabilities in coding and mathematics, thanks to our specialized expert models in these domains.\n\n- Significant improvements in instruction following, generating long texts (over 8K tokens), understanding structured data (e.g, tables), and generating structured outputs especially JSON. More resilient to the diversity of system prompts, enhancing role-play implementation and condition-setting for chatbots.\n\n- Long-context Support up to 128K tokens and can generate up to 8K tokens.\n\n- Multilingual support for over 29 languages, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, Arabic, and more.\n\nUsage of this model is subject to [Tongyi Qianwen LICENSE AGREEMENT](https://huggingface.co/Qwen/Qwen1.5-110B-Chat/blob/main/LICENSE).","context_length":32000,"architecture":{"modality":"text->text","tokenizer":"Qwen","instruct_type":"chatml"},"pricing":{"prompt":"0.00000023","completion":"0.0000004","image":"0","request":"0"},"top_provider":{"context_length":32000,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"qwen/qwen-2-vl-72b-instruct","name":"Qwen2-VL 72B Instruct","created":1726617600,"description":"Qwen2 VL 72B is a multimodal LLM from the Qwen Team with the following key enhancements:\n\n- SoTA understanding of images of various resolution & ratio: Qwen2-VL achieves state-of-the-art performance on visual understanding benchmarks, including MathVista, DocVQA, RealWorldQA, MTVQA, etc.\n\n- Understanding videos of 20min+: Qwen2-VL can understand videos over 20 minutes for high-quality video-based question answering, dialog, content creation, etc.\n\n- Agent that can operate your mobiles, robots, etc.: with the abilities of complex reasoning and decision making, Qwen2-VL can be integrated with devices like mobile phones, robots, etc., for automatic operation based on visual environment and text instructions.\n\n- Multilingual Support: to serve global users, besides English and Chinese, Qwen2-VL now supports the understanding of texts in different languages inside images, including most European languages, Japanese, Korean, Arabic, Vietnamese, etc.\n\nFor more details, see this [blog post](https://qwenlm.github.io/blog/qwen2-vl/) and [GitHub repo](https://github.com/QwenLM/Qwen2-VL).\n\nUsage of this model is subject to [Tongyi Qianwen LICENSE AGREEMENT](https://huggingface.co/Qwen/Qwen1.5-110B-Chat/blob/main/LICENSE).","context_length":4096,"architecture":{"modality":"text+image->text","tokenizer":"Qwen","instruct_type":null},"pricing":{"prompt":"0.0000004","completion":"0.0000004","image":"0.000578","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"neversleep/llama-3.1-lumimaid-8b","name":"NeverSleep: Lumimaid v0.2 8B","created":1726358400,"description":"Lumimaid v0.2 8B is a finetune of [Llama 3.1 8B](/models/meta-llama/llama-3.1-8b-instruct) with a \"HUGE step up dataset wise\" compared to Lumimaid v0.1. Sloppy chats output were purged.\n\nUsage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.0000001875","completion":"0.000001125","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":2048,"is_moderated":false},"per_request_limits":null},{"id":"openai/o1-mini-2024-09-12","name":"OpenAI: o1-mini (2024-09-12)","created":1726099200,"description":"The latest and strongest model family from OpenAI, o1 is designed to spend more time thinking before responding.\n\nThe o1 models are optimized for math, science, programming, and other STEM-related tasks. They consistently exhibit PhD-level accuracy on benchmarks in physics, chemistry, and biology. Learn more in the [launch announcement](https://openai.com/o1).\n\nNote: This model is currently experimental and not suitable for production use-cases, and may be heavily rate-limited.","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.000003","completion":"0.000012","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":65536,"is_moderated":true},"per_request_limits":null},{"id":"openai/o1-preview","name":"OpenAI: o1-preview","created":1726099200,"description":"The latest and strongest model family from OpenAI, o1 is designed to spend more time thinking before responding.\n\nThe o1 models are optimized for math, science, programming, and other STEM-related tasks. They consistently exhibit PhD-level accuracy on benchmarks in physics, chemistry, and biology. Learn more in the [launch announcement](https://openai.com/o1).\n\nNote: This model is currently experimental and not suitable for production use-cases, and may be heavily rate-limited.","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.000015","completion":"0.00006","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":32768,"is_moderated":true},"per_request_limits":null},{"id":"openai/o1-preview-2024-09-12","name":"OpenAI: o1-preview (2024-09-12)","created":1726099200,"description":"The latest and strongest model family from OpenAI, o1 is designed to spend more time thinking before responding.\n\nThe o1 models are optimized for math, science, programming, and other STEM-related tasks. They consistently exhibit PhD-level accuracy on benchmarks in physics, chemistry, and biology. Learn more in the [launch announcement](https://openai.com/o1).\n\nNote: This model is currently experimental and not suitable for production use-cases, and may be heavily rate-limited.","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.000015","completion":"0.00006","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":32768,"is_moderated":true},"per_request_limits":null},{"id":"openai/o1-mini","name":"OpenAI: o1-mini","created":1726099200,"description":"The latest and strongest model family from OpenAI, o1 is designed to spend more time thinking before responding.\n\nThe o1 models are optimized for math, science, programming, and other STEM-related tasks. They consistently exhibit PhD-level accuracy on benchmarks in physics, chemistry, and biology. Learn more in the [launch announcement](https://openai.com/o1).\n\nNote: This model is currently experimental and not suitable for production use-cases, and may be heavily rate-limited.","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.000003","completion":"0.000012","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":65536,"is_moderated":true},"per_request_limits":null},{"id":"mistralai/pixtral-12b","name":"Mistral: Pixtral 12B","created":1725926400,"description":"The first multi-modal, text+image-to-text model from Mistral AI. Its weights were launched via torrent: https://x.com/mistralai/status/1833758285167722836.","context_length":4096,"architecture":{"modality":"text+image->text","tokenizer":"Mistral","instruct_type":null},"pricing":{"prompt":"0.0000001","completion":"0.0000001","image":"0.0001445","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"cohere/command-r-08-2024","name":"Cohere: Command R (08-2024)","created":1724976000,"description":"command-r-08-2024 is an update of the [Command R](/models/cohere/command-r) with improved performance for multilingual retrieval-augmented generation (RAG) and tool use. More broadly, it is better at math, code and reasoning and is competitive with the previous version of the larger Command R+ model.\n\nRead the launch post [here](https://docs.cohere.com/changelog/command-gets-refreshed).\n\nUse of this model is subject to Cohere's [Acceptable Use Policy](https://docs.cohere.com/docs/c4ai-acceptable-use-policy).","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"Cohere","instruct_type":null},"pricing":{"prompt":"0.0000001425","completion":"0.00000057","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":4000,"is_moderated":false},"per_request_limits":null},{"id":"cohere/command-r-plus-08-2024","name":"Cohere: Command R+ (08-2024)","created":1724976000,"description":"command-r-plus-08-2024 is an update of the [Command R+](/models/cohere/command-r-plus) with roughly 50% higher throughput and 25% lower latencies as compared to the previous Command R+ version, while keeping the hardware footprint the same.\n\nRead the launch post [here](https://docs.cohere.com/changelog/command-gets-refreshed).\n\nUse of this model is subject to Cohere's [Acceptable Use Policy](https://docs.cohere.com/docs/c4ai-acceptable-use-policy).","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"Cohere","instruct_type":null},"pricing":{"prompt":"0.000002375","completion":"0.0000095","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":4000,"is_moderated":false},"per_request_limits":null},{"id":"qwen/qwen-2-vl-7b-instruct","name":"Qwen2-VL 7B Instruct","created":1724803200,"description":"Qwen2 VL 7B is a multimodal LLM from the Qwen Team with the following key enhancements:\n\n- SoTA understanding of images of various resolution & ratio: Qwen2-VL achieves state-of-the-art performance on visual understanding benchmarks, including MathVista, DocVQA, RealWorldQA, MTVQA, etc.\n\n- Understanding videos of 20min+: Qwen2-VL can understand videos over 20 minutes for high-quality video-based question answering, dialog, content creation, etc.\n\n- Agent that can operate your mobiles, robots, etc.: with the abilities of complex reasoning and decision making, Qwen2-VL can be integrated with devices like mobile phones, robots, etc., for automatic operation based on visual environment and text instructions.\n\n- Multilingual Support: to serve global users, besides English and Chinese, Qwen2-VL now supports the understanding of texts in different languages inside images, including most European languages, Japanese, Korean, Arabic, Vietnamese, etc.\n\nFor more details, see this [blog post](https://qwenlm.github.io/blog/qwen2-vl/) and [GitHub repo](https://github.com/QwenLM/Qwen2-VL).\n\nUsage of this model is subject to [Tongyi Qianwen LICENSE AGREEMENT](https://huggingface.co/Qwen/Qwen1.5-110B-Chat/blob/main/LICENSE).","context_length":4096,"architecture":{"modality":"text+image->text","tokenizer":"Qwen","instruct_type":null},"pricing":{"prompt":"0.0000001","completion":"0.0000001","image":"0.0001445","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"google/gemini-flash-1.5-exp","name":"Google: Gemini Flash 1.5 Experimental","created":1724803200,"description":"Gemini 1.5 Flash Experimental is an experimental version of the [Gemini 1.5 Flash](/models/google/gemini-flash-1.5) model.\n\nUsage of Gemini is subject to Google's [Gemini Terms of Use](https://ai.google.dev/terms).\n\n#multimodal\n\nNote: This model is experimental and not suited for production use-cases. It may be removed or redirected to another model in the future.","context_length":1000000,"architecture":{"modality":"text+image->text","tokenizer":"Gemini","instruct_type":null},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":1000000,"max_completion_tokens":8192,"is_moderated":false},"per_request_limits":null},{"id":"sao10k/l3.1-euryale-70b","name":"Sao10K: Llama 3.1 Euryale 70B v2.2","created":1724803200,"description":"Euryale L3.1 70B v2.2 is a model focused on creative roleplay from [Sao10k](https://ko-fi.com/sao10k). It is the successor of [Euryale L3 70B v2.1](/models/sao10k/l3-euryale-70b).","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.00000035","completion":"0.0000004","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"google/gemini-flash-1.5-8b-exp","name":"Google: Gemini Flash 1.5 8B Experimental","created":1724803200,"description":"Gemini Flash 1.5 8B Experimental is an experimental, 8B parameter version of the [Gemini Flash 1.5](/models/google/gemini-flash-1.5) model.\n\nUsage of Gemini is subject to Google's [Gemini Terms of Use](https://ai.google.dev/terms).\n\n#multimodal\n\nNote: This model is currently experimental and not suitable for production use-cases, and may be heavily rate-limited.","context_length":1000000,"architecture":{"modality":"text+image->text","tokenizer":"Gemini","instruct_type":null},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":1000000,"max_completion_tokens":8192,"is_moderated":false},"per_request_limits":null},{"id":"ai21/jamba-1-5-large","name":"AI21: Jamba 1.5 Large","created":1724371200,"description":"Jamba 1.5 Large is part of AI21's new family of open models, offering superior speed, efficiency, and quality.\n\nIt features a 256K effective context window, the longest among open models, enabling improved performance on tasks like document summarization and analysis.\n\nBuilt on a novel SSM-Transformer architecture, it outperforms larger models like Llama 3.1 70B on benchmarks while maintaining resource efficiency.\n\nRead their [announcement](https://www.ai21.com/blog/announcing-jamba-model-family) to learn more.","context_length":256000,"architecture":{"modality":"text->text","tokenizer":"Other","instruct_type":null},"pricing":{"prompt":"0.000002","completion":"0.000008","image":"0","request":"0"},"top_provider":{"context_length":256000,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"ai21/jamba-1-5-mini","name":"AI21: Jamba 1.5 Mini","created":1724371200,"description":"Jamba 1.5 Mini is the world's first production-grade Mamba-based model, combining SSM and Transformer architectures for a 256K context window and high efficiency.\n\nIt works with 9 languages and can handle various writing and analysis tasks as well as or better than similar small models.\n\nThis model uses less computer memory and works faster with longer texts than previous designs.\n\nRead their [announcement](https://www.ai21.com/blog/announcing-jamba-model-family) to learn more.","context_length":256000,"architecture":{"modality":"text->text","tokenizer":"Other","instruct_type":null},"pricing":{"prompt":"0.0000002","completion":"0.0000004","image":"0","request":"0"},"top_provider":{"context_length":256000,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"microsoft/phi-3.5-mini-128k-instruct","name":"Microsoft: Phi-3.5 Mini 128K Instruct","created":1724198400,"description":"Phi-3.5 models are lightweight, state-of-the-art open models. These models were trained with Phi-3 datasets that include both synthetic data and the filtered, publicly available websites data, with a focus on high quality and reasoning-dense properties. Phi-3.5 Mini uses 3.8B parameters, and is a dense decoder-only transformer model using the same tokenizer as [Phi-3 Mini](/models/microsoft/phi-3-mini-128k-instruct).\n\nThe models underwent a rigorous enhancement process, incorporating both supervised fine-tuning, proximal policy optimization, and direct preference optimization to ensure precise instruction adherence and robust safety measures. When assessed against benchmarks that test common sense, language understanding, math, code, long context and logical reasoning, Phi-3.5 models showcased robust and state-of-the-art performance among models with less than 13 billion parameters.","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"Other","instruct_type":"phi3"},"pricing":{"prompt":"0.0000001","completion":"0.0000001","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"nousresearch/hermes-3-llama-3.1-70b","name":"Nous: Hermes 3 70B Instruct","created":1723939200,"description":"Hermes 3 is a generalist language model with many improvements over [Hermes 2](/models/nousresearch/nous-hermes-2-mistral-7b-dpo), including advanced agentic capabilities, much better roleplaying, reasoning, multi-turn conversation, long context coherence, and improvements across the board.\n\nHermes 3 70B is a competitive, if not superior finetune of the [Llama-3.1 70B foundation model](/models/meta-llama/llama-3.1-70b-instruct), focused on aligning LLMs to the user, with powerful steering capabilities and control given to the end user.\n\nThe Hermes 3 series builds and expands on the Hermes 2 set of capabilities, including more powerful and reliable function calling and structured output capabilities, generalist assistant capabilities, and improved code generation skills.","context_length":131000,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"chatml"},"pricing":{"prompt":"0.00000012","completion":"0.0000003","image":"0","request":"0"},"top_provider":{"context_length":131000,"max_completion_tokens":131000,"is_moderated":false},"per_request_limits":null},{"id":"nousresearch/hermes-3-llama-3.1-405b","name":"Nous: Hermes 3 405B Instruct","created":1723766400,"description":"Hermes 3 is a generalist language model with many improvements over Hermes 2, including advanced agentic capabilities, much better roleplaying, reasoning, multi-turn conversation, long context coherence, and improvements across the board.\n\nHermes 3 405B is a frontier-level, full-parameter finetune of the Llama-3.1 405B foundation model, focused on aligning LLMs to the user, with powerful steering capabilities and control given to the end user.\n\nThe Hermes 3 series builds and expands on the Hermes 2 set of capabilities, including more powerful and reliable function calling and structured output capabilities, generalist assistant capabilities, and improved code generation skills.\n\nHermes 3 is competitive, if not superior, to Llama-3.1 Instruct models at general capabilities, with varying strengths and weaknesses attributable between the two.","context_length":131072,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"chatml"},"pricing":{"prompt":"0.0000008","completion":"0.0000008","image":"0","request":"0"},"top_provider":{"context_length":131072,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"perplexity/llama-3.1-sonar-huge-128k-online","name":"Perplexity: Llama 3.1 Sonar 405B Online","created":1723593600,"description":"Llama 3.1 Sonar is Perplexity's latest model family. It surpasses their earlier Sonar models in cost-efficiency, speed, and performance. The model is built upon the Llama 3.1 405B and has internet access.","context_length":127072,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":null},"pricing":{"prompt":"0.000005","completion":"0.000005","image":"0","request":"0.005"},"top_provider":{"context_length":127072,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"openai/chatgpt-4o-latest","name":"OpenAI: ChatGPT-4o","created":1723593600,"description":"OpenAI ChatGPT 4o is continually updated by OpenAI to point to the current version of GPT-4o used by ChatGPT. It therefore differs slightly from the API version of [GPT-4o](/models/openai/gpt-4o) in that it has additional RLHF. It is intended for research and evaluation.\n\nOpenAI notes that this model is not suited for production use-cases as it may be removed or redirected to another model in the future.","context_length":128000,"architecture":{"modality":"text+image->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.000005","completion":"0.000015","image":"0.007225","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":16384,"is_moderated":true},"per_request_limits":null},{"id":"sao10k/l3-lunaris-8b","name":"Sao10K: Llama 3 8B Lunaris","created":1723507200,"description":"Lunaris 8B is a versatile generalist and roleplaying model based on Llama 3. It's a strategic merge of multiple models, designed to balance creativity with improved logic and general knowledge.\n\nCreated by [Sao10k](https://huggingface.co/Sao10k), this model aims to offer an improved experience over Stheno v3.2, with enhanced creativity and logical reasoning.\n\nFor best results, use with Llama 3 Instruct context template, temperature 1.4, and min_p 0.1.","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.00000003","completion":"0.00000006","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"aetherwiing/mn-starcannon-12b","name":"Aetherwiing: Starcannon 12B","created":1723507200,"description":"Starcannon 12B v2 is a creative roleplay and story writing model, based on Mistral Nemo, using [nothingiisreal/mn-celeste-12b](/nothingiisreal/mn-celeste-12b) as a base, with [intervitens/mini-magnum-12b-v1.1](https://huggingface.co/intervitens/mini-magnum-12b-v1.1) merged in using the [TIES](https://arxiv.org/abs/2306.01708) method.\n\nAlthough more similar to Magnum overall, the model remains very creative, with a pleasant writing style. It is recommended for people wanting more variety than Magnum, and yet more verbose prose than Celeste.","context_length":16384,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"chatml"},"pricing":{"prompt":"0.0000008","completion":"0.0000012","image":"0","request":"0"},"top_provider":{"context_length":16384,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"openai/gpt-4o-2024-08-06","name":"OpenAI: GPT-4o (2024-08-06)","created":1722902400,"description":"The 2024-08-06 version of GPT-4o offers improved performance in structured outputs, with the ability to supply a JSON schema in the respone_format. Read more [here](https://openai.com/index/introducing-structured-outputs-in-the-api/).\n\nGPT-4o (\"o\" for \"omni\") is OpenAI's latest AI model, supporting both text and image inputs with text outputs. It maintains the intelligence level of [GPT-4 Turbo](/models/openai/gpt-4-turbo) while being twice as fast and 50% more cost-effective. GPT-4o also offers improved performance in processing non-English languages and enhanced visual capabilities.\n\nFor benchmarking against other models, it was briefly called [\"im-also-a-good-gpt2-chatbot\"](https://twitter.com/LiamFedus/status/1790064963966370209)","context_length":128000,"architecture":{"modality":"text+image->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.0000025","completion":"0.00001","image":"0.003613","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":16384,"is_moderated":true},"per_request_limits":null},{"id":"meta-llama/llama-3.1-405b","name":"Meta: Llama 3.1 405B (base)","created":1722556800,"description":"Meta's latest class of model (Llama 3.1) launched with a variety of sizes & flavors. This is the base 405B pre-trained version.\n\nIt has demonstrated strong performance compared to leading closed-source models in human evaluations.\n\nTo read more about the model release, [click here](https://ai.meta.com/blog/meta-llama-3/). Usage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"none"},"pricing":{"prompt":"0.000002","completion":"0.000002","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"nothingiisreal/mn-celeste-12b","name":"Mistral Nemo 12B Celeste","created":1722556800,"description":"A specialized story writing and roleplaying model based on Mistral's NeMo 12B Instruct. Fine-tuned on curated datasets including Reddit Writing Prompts and Opus Instruct 25K.\n\nThis model excels at creative writing, offering improved NSFW capabilities, with smarter and more active narration. It demonstrates remarkable versatility in both SFW and NSFW scenarios, with strong Out of Character (OOC) steering capabilities, allowing fine-tuned control over narrative direction and character behavior.\n\nCheck out the model's [HuggingFace page](https://huggingface.co/nothingiisreal/MN-12B-Celeste-V1.9) for details on what parameters and prompts work best!","context_length":16384,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"chatml"},"pricing":{"prompt":"0.0000008","completion":"0.0000012","image":"0","request":"0"},"top_provider":{"context_length":16384,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"perplexity/llama-3.1-sonar-small-128k-chat","name":"Perplexity: Llama 3.1 Sonar 8B","created":1722470400,"description":"Llama 3.1 Sonar is Perplexity's latest model family. It surpasses their earlier Sonar models in cost-efficiency, speed, and performance.\n\nThis is a normal offline LLM, but the [online version](/models/perplexity/llama-3.1-sonar-small-128k-online) of this model has Internet access.","context_length":131072,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":null},"pricing":{"prompt":"0.0000002","completion":"0.0000002","image":"0","request":"0"},"top_provider":{"context_length":131072,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"google/gemini-pro-1.5-exp","name":"Google: Gemini Pro 1.5 Experimental","created":1722470400,"description":"Gemini 1.5 Pro Experimental is a bleeding-edge version of the [Gemini 1.5 Pro](/models/google/gemini-pro-1.5) model. Because it's currently experimental, it will be **heavily rate-limited** by Google.\n\nUsage of Gemini is subject to Google's [Gemini Terms of Use](https://ai.google.dev/terms).\n\n#multimodal","context_length":1000000,"architecture":{"modality":"text+image->text","tokenizer":"Gemini","instruct_type":null},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":1000000,"max_completion_tokens":8192,"is_moderated":false},"per_request_limits":null},{"id":"perplexity/llama-3.1-sonar-large-128k-chat","name":"Perplexity: Llama 3.1 Sonar 70B","created":1722470400,"description":"Llama 3.1 Sonar is Perplexity's latest model family. It surpasses their earlier Sonar models in cost-efficiency, speed, and performance.\n\nThis is a normal offline LLM, but the [online version](/models/perplexity/llama-3.1-sonar-large-128k-online) of this model has Internet access.","context_length":131072,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":null},"pricing":{"prompt":"0.000001","completion":"0.000001","image":"0","request":"0"},"top_provider":{"context_length":131072,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"perplexity/llama-3.1-sonar-large-128k-online","name":"Perplexity: Llama 3.1 Sonar 70B Online","created":1722470400,"description":"Llama 3.1 Sonar is Perplexity's latest model family. It surpasses their earlier Sonar models in cost-efficiency, speed, and performance.\n\nThis is the online version of the [offline chat model](/models/perplexity/llama-3.1-sonar-large-128k-chat). It is focused on delivering helpful, up-to-date, and factual responses. #online","context_length":127072,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":null},"pricing":{"prompt":"0.000001","completion":"0.000001","image":"0","request":"0.005"},"top_provider":{"context_length":127072,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"perplexity/llama-3.1-sonar-small-128k-online","name":"Perplexity: Llama 3.1 Sonar 8B Online","created":1722470400,"description":"Llama 3.1 Sonar is Perplexity's latest model family. It surpasses their earlier Sonar models in cost-efficiency, speed, and performance.\n\nThis is the online version of the [offline chat model](/models/perplexity/llama-3.1-sonar-small-128k-chat). It is focused on delivering helpful, up-to-date, and factual responses. #online","context_length":127072,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":null},"pricing":{"prompt":"0.0000002","completion":"0.0000002","image":"0","request":"0.005"},"top_provider":{"context_length":127072,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3.1-405b-instruct:free","name":"Meta: Llama 3.1 405B Instruct (free)","created":1721692800,"description":"The highly anticipated 400B class of Llama3 is here! Clocking in at 128k context with impressive eval scores, the Meta AI team continues to push the frontier of open-source LLMs.\n\nMeta's latest class of model (Llama 3.1) launched with a variety of sizes & flavors. This 405B instruct-tuned version is optimized for high quality dialogue usecases.\n\nIt has demonstrated strong performance compared to leading closed-source models including GPT-4o and Claude 3.5 Sonnet in evaluations.\n\nTo read more about the model release, [click here](https://ai.meta.com/blog/meta-llama-3-1/). Usage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":8000,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":8000,"max_completion_tokens":4000,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3.1-405b-instruct","name":"Meta: Llama 3.1 405B Instruct","created":1721692800,"description":"The highly anticipated 400B class of Llama3 is here! Clocking in at 128k context with impressive eval scores, the Meta AI team continues to push the frontier of open-source LLMs.\n\nMeta's latest class of model (Llama 3.1) launched with a variety of sizes & flavors. This 405B instruct-tuned version is optimized for high quality dialogue usecases.\n\nIt has demonstrated strong performance compared to leading closed-source models including GPT-4o and Claude 3.5 Sonnet in evaluations.\n\nTo read more about the model release, [click here](https://ai.meta.com/blog/meta-llama-3-1/). Usage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":32000,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.0000008","completion":"0.0000008","image":"0","request":"0"},"top_provider":{"context_length":32000,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3.1-405b-instruct:nitro","name":"Meta: Llama 3.1 405B Instruct (nitro)","created":1721692800,"description":"The highly anticipated 400B class of Llama3 is here! Clocking in at 128k context with impressive eval scores, the Meta AI team continues to push the frontier of open-source LLMs.\n\nMeta's latest class of model (Llama 3.1) launched with a variety of sizes & flavors. This 405B instruct-tuned version is optimized for high quality dialogue usecases.\n\nIt has demonstrated strong performance compared to leading closed-source models including GPT-4o and Claude 3.5 Sonnet in evaluations.\n\nTo read more about the model release, [click here](https://ai.meta.com/blog/meta-llama-3-1/). Usage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":8000,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.00001462","completion":"0.00001462","image":"0","request":"0"},"top_provider":{"context_length":8000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3.1-8b-instruct:free","name":"Meta: Llama 3.1 8B Instruct (free)","created":1721692800,"description":"Meta's latest class of model (Llama 3.1) launched with a variety of sizes & flavors. This 8B instruct-tuned version is fast and efficient.\n\nIt has demonstrated strong performance compared to leading closed-source models in human evaluations.\n\nTo read more about the model release, [click here](https://ai.meta.com/blog/meta-llama-3-1/). Usage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3.1-8b-instruct","name":"Meta: Llama 3.1 8B Instruct","created":1721692800,"description":"Meta's latest class of model (Llama 3.1) launched with a variety of sizes & flavors. This 8B instruct-tuned version is fast and efficient.\n\nIt has demonstrated strong performance compared to leading closed-source models in human evaluations.\n\nTo read more about the model release, [click here](https://ai.meta.com/blog/meta-llama-3-1/). Usage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":131072,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.00000002","completion":"0.00000005","image":"0","request":"0"},"top_provider":{"context_length":131072,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3.1-70b-instruct:free","name":"Meta: Llama 3.1 70B Instruct (free)","created":1721692800,"description":"Meta's latest class of model (Llama 3.1) launched with a variety of sizes & flavors. This 70B instruct-tuned version is optimized for high quality dialogue usecases.\n\nIt has demonstrated strong performance compared to leading closed-source models in human evaluations.\n\nTo read more about the model release, [click here](https://ai.meta.com/blog/meta-llama-3-1/). Usage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3.1-70b-instruct","name":"Meta: Llama 3.1 70B Instruct","created":1721692800,"description":"Meta's latest class of model (Llama 3.1) launched with a variety of sizes & flavors. This 70B instruct-tuned version is optimized for high quality dialogue usecases.\n\nIt has demonstrated strong performance compared to leading closed-source models in human evaluations.\n\nTo read more about the model release, [click here](https://ai.meta.com/blog/meta-llama-3-1/). Usage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":131072,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.00000012","completion":"0.0000003","image":"0","request":"0"},"top_provider":{"context_length":131072,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3.1-70b-instruct:nitro","name":"Meta: Llama 3.1 70B Instruct (nitro)","created":1721692800,"description":"Meta's latest class of model (Llama 3.1) launched with a variety of sizes & flavors. This 70B instruct-tuned version is optimized for high quality dialogue usecases.\n\nIt has demonstrated strong performance compared to leading closed-source models in human evaluations.\n\nTo read more about the model release, [click here](https://ai.meta.com/blog/meta-llama-3-1/). Usage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":64000,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.00000325","completion":"0.00000325","image":"0","request":"0"},"top_provider":{"context_length":64000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"mistralai/mistral-nemo","name":"Mistral: Mistral Nemo","created":1721347200,"description":"A 12B parameter model with a 128k token context length built by Mistral in collaboration with NVIDIA.\n\nThe model is multilingual, supporting English, French, German, Spanish, Italian, Portuguese, Chinese, Japanese, Korean, Arabic, and Hindi.\n\nIt supports function calling and is released under the Apache 2.0 license.","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"mistral"},"pricing":{"prompt":"0.000000035","completion":"0.00000008","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"mistralai/codestral-mamba","name":"Mistral: Codestral Mamba","created":1721347200,"description":"A 7.3B parameter Mamba-based model designed for code and reasoning tasks.\n\n- Linear time inference, allowing for theoretically infinite sequence lengths\n- 256k token context window\n- Optimized for quick responses, especially beneficial for code productivity\n- Performs comparably to state-of-the-art transformer models in code and reasoning tasks\n- Available under the Apache 2.0 license for free use, modification, and distribution","context_length":256000,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":null},"pricing":{"prompt":"0.00000025","completion":"0.00000025","image":"0","request":"0"},"top_provider":{"context_length":256000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"openai/gpt-4o-mini","name":"OpenAI: GPT-4o-mini","created":1721260800,"description":"GPT-4o mini is OpenAI's newest model after [GPT-4 Omni](/models/openai/gpt-4o), supporting both text and image inputs with text outputs.\n\nAs their most advanced small model, it is many multiples more affordable than other recent frontier models, and more than 60% cheaper than [GPT-3.5 Turbo](/models/openai/gpt-3.5-turbo). It maintains SOTA intelligence, while being significantly more cost-effective.\n\nGPT-4o mini achieves an 82% score on MMLU and presently ranks higher than GPT-4 on chat preferences [common leaderboards](https://arena.lmsys.org/).\n\nCheck out the [launch announcement](https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/) to learn more.\n\n#multimodal","context_length":128000,"architecture":{"modality":"text+image->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.00000015","completion":"0.0000006","image":"0.007225","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":16384,"is_moderated":true},"per_request_limits":null},{"id":"openai/gpt-4o-mini-2024-07-18","name":"OpenAI: GPT-4o-mini (2024-07-18)","created":1721260800,"description":"GPT-4o mini is OpenAI's newest model after [GPT-4 Omni](/models/openai/gpt-4o), supporting both text and image inputs with text outputs.\n\nAs their most advanced small model, it is many multiples more affordable than other recent frontier models, and more than 60% cheaper than [GPT-3.5 Turbo](/models/openai/gpt-3.5-turbo). It maintains SOTA intelligence, while being significantly more cost-effective.\n\nGPT-4o mini achieves an 82% score on MMLU and presently ranks higher than GPT-4 on chat preferences [common leaderboards](https://arena.lmsys.org/).\n\nCheck out the [launch announcement](https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/) to learn more.\n\n#multimodal","context_length":128000,"architecture":{"modality":"text+image->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.00000015","completion":"0.0000006","image":"0.007225","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":16384,"is_moderated":true},"per_request_limits":null},{"id":"qwen/qwen-2-7b-instruct:free","name":"Qwen 2 7B Instruct (free)","created":1721088000,"description":"Qwen2 7B is a transformer-based model that excels in language understanding, multilingual capabilities, coding, mathematics, and reasoning.\n\nIt features SwiGLU activation, attention QKV bias, and group query attention. It is pretrained on extensive data with supervised finetuning and direct preference optimization.\n\nFor more details, see this [blog post](https://qwenlm.github.io/blog/qwen2/) and [GitHub repo](https://github.com/QwenLM/Qwen2).\n\nUsage of this model is subject to [Tongyi Qianwen LICENSE AGREEMENT](https://huggingface.co/Qwen/Qwen1.5-110B-Chat/blob/main/LICENSE).","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Qwen","instruct_type":"chatml"},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"qwen/qwen-2-7b-instruct","name":"Qwen 2 7B Instruct","created":1721088000,"description":"Qwen2 7B is a transformer-based model that excels in language understanding, multilingual capabilities, coding, mathematics, and reasoning.\n\nIt features SwiGLU activation, attention QKV bias, and group query attention. It is pretrained on extensive data with supervised finetuning and direct preference optimization.\n\nFor more details, see this [blog post](https://qwenlm.github.io/blog/qwen2/) and [GitHub repo](https://github.com/QwenLM/Qwen2).\n\nUsage of this model is subject to [Tongyi Qianwen LICENSE AGREEMENT](https://huggingface.co/Qwen/Qwen1.5-110B-Chat/blob/main/LICENSE).","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Qwen","instruct_type":"chatml"},"pricing":{"prompt":"0.000000054","completion":"0.000000054","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"google/gemma-2-27b-it","name":"Google: Gemma 2 27B","created":1720828800,"description":"Gemma 2 27B by Google is an open model built from the same research and technology used to create the [Gemini models](/models?q=gemini).\n\nGemma models are well-suited for a variety of text generation tasks, including question answering, summarization, and reasoning.\n\nSee the [launch announcement](https://blog.google/technology/developers/google-gemma-2/) for more details. Usage of Gemma is subject to Google's [Gemma Terms of Use](https://ai.google.dev/gemma/terms).","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Gemini","instruct_type":"gemma"},"pricing":{"prompt":"0.00000027","completion":"0.00000027","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"alpindale/magnum-72b","name":"Magnum 72B","created":1720656000,"description":"From the maker of [Goliath](https://openrouter.ai/models/alpindale/goliath-120b), Magnum 72B is the first in a new family of models designed to achieve the prose quality of the Claude 3 models, notably Opus & Sonnet.\n\nThe model is based on [Qwen2 72B](https://openrouter.ai/models/qwen/qwen-2-72b-instruct) and trained with 55 million tokens of highly curated roleplay (RP) data.","context_length":16384,"architecture":{"modality":"text->text","tokenizer":"Qwen","instruct_type":"chatml"},"pricing":{"prompt":"0.000001875","completion":"0.00000225","image":"0","request":"0"},"top_provider":{"context_length":16384,"max_completion_tokens":1024,"is_moderated":false},"per_request_limits":null},{"id":"google/gemma-2-9b-it:free","name":"Google: Gemma 2 9B (free)","created":1719532800,"description":"Gemma 2 9B by Google is an advanced, open-source language model that sets a new standard for efficiency and performance in its size class.\n\nDesigned for a wide variety of tasks, it empowers developers and researchers to build innovative applications, while maintaining accessibility, safety, and cost-effectiveness.\n\nSee the [launch announcement](https://blog.google/technology/developers/google-gemma-2/) for more details. Usage of Gemma is subject to Google's [Gemma Terms of Use](https://ai.google.dev/gemma/terms).","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Gemini","instruct_type":"gemma"},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":2048,"is_moderated":false},"per_request_limits":null},{"id":"google/gemma-2-9b-it","name":"Google: Gemma 2 9B","created":1719532800,"description":"Gemma 2 9B by Google is an advanced, open-source language model that sets a new standard for efficiency and performance in its size class.\n\nDesigned for a wide variety of tasks, it empowers developers and researchers to build innovative applications, while maintaining accessibility, safety, and cost-effectiveness.\n\nSee the [launch announcement](https://blog.google/technology/developers/google-gemma-2/) for more details. Usage of Gemma is subject to Google's [Gemma Terms of Use](https://ai.google.dev/gemma/terms).","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Gemini","instruct_type":"gemma"},"pricing":{"prompt":"0.00000003","completion":"0.00000006","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"01-ai/yi-large","name":"01.AI: Yi Large","created":1719273600,"description":"The Yi Large model was designed by 01.AI with the following usecases in mind: knowledge search, data classification, human-like chat bots, and customer service.\n\nIt stands out for its multilingual proficiency, particularly in Spanish, Chinese, Japanese, German, and French.\n\nCheck out the [launch announcement](https://01-ai.github.io/blog/01.ai-yi-large-llm-launch) to learn more.","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Yi","instruct_type":null},"pricing":{"prompt":"0.000003","completion":"0.000003","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"ai21/jamba-instruct","name":"AI21: Jamba Instruct","created":1719273600,"description":"The Jamba-Instruct model, introduced by AI21 Labs, is an instruction-tuned variant of their hybrid SSM-Transformer Jamba model, specifically optimized for enterprise applications.\n\n- 256K Context Window: It can process extensive information, equivalent to a 400-page novel, which is beneficial for tasks involving large documents such as financial reports or legal documents\n- Safety and Accuracy: Jamba-Instruct is designed with enhanced safety features to ensure secure deployment in enterprise environments, reducing the risk and cost of implementation\n\nRead their [announcement](https://www.ai21.com/blog/announcing-jamba) to learn more.\n\nJamba has a knowledge cutoff of February 2024.","context_length":256000,"architecture":{"modality":"text->text","tokenizer":"Other","instruct_type":null},"pricing":{"prompt":"0.0000005","completion":"0.0000007","image":"0","request":"0"},"top_provider":{"context_length":256000,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"anthropic/claude-3.5-sonnet-20240620:beta","name":"Anthropic: Claude 3.5 Sonnet (2024-06-20) (self-moderated)","created":1718841600,"description":"Claude 3.5 Sonnet delivers better-than-Opus capabilities, faster-than-Sonnet speeds, at the same Sonnet prices. Sonnet is particularly good at:\n\n- Coding: Autonomously writes, edits, and runs code with reasoning and troubleshooting\n- Data science: Augments human data science expertise; navigates unstructured data while using multiple tools for insights\n- Visual processing: excelling at interpreting charts, graphs, and images, accurately transcribing text to derive insights beyond just the text alone\n- Agentic tasks: exceptional tool use, making it great at agentic tasks (i.e. complex, multi-step problem solving tasks that require engaging with other systems)\n\nFor the latest version (2024-10-23), check out [Claude 3.5 Sonnet](/anthropic/claude-3.5-sonnet).\n\n#multimodal","context_length":200000,"architecture":{"modality":"text+image->text","tokenizer":"Claude","instruct_type":null},"pricing":{"prompt":"0.000003","completion":"0.000015","image":"0.0048","request":"0"},"top_provider":{"context_length":200000,"max_completion_tokens":8192,"is_moderated":false},"per_request_limits":null},{"id":"anthropic/claude-3.5-sonnet-20240620","name":"Anthropic: Claude 3.5 Sonnet (2024-06-20)","created":1718841600,"description":"Claude 3.5 Sonnet delivers better-than-Opus capabilities, faster-than-Sonnet speeds, at the same Sonnet prices. Sonnet is particularly good at:\n\n- Coding: Autonomously writes, edits, and runs code with reasoning and troubleshooting\n- Data science: Augments human data science expertise; navigates unstructured data while using multiple tools for insights\n- Visual processing: excelling at interpreting charts, graphs, and images, accurately transcribing text to derive insights beyond just the text alone\n- Agentic tasks: exceptional tool use, making it great at agentic tasks (i.e. complex, multi-step problem solving tasks that require engaging with other systems)\n\nFor the latest version (2024-10-23), check out [Claude 3.5 Sonnet](/anthropic/claude-3.5-sonnet).\n\n#multimodal","context_length":200000,"architecture":{"modality":"text+image->text","tokenizer":"Claude","instruct_type":null},"pricing":{"prompt":"0.000003","completion":"0.000015","image":"0.0048","request":"0"},"top_provider":{"context_length":200000,"max_completion_tokens":8192,"is_moderated":true},"per_request_limits":null},{"id":"sao10k/l3-euryale-70b","name":"Sao10k: Llama 3 Euryale 70B v2.1","created":1718668800,"description":"Euryale 70B v2.1 is a model focused on creative roleplay from [Sao10k](https://ko-fi.com/sao10k).\n\n- Better prompt adherence.\n- Better anatomy / spatial awareness.\n- Adapts much better to unique and custom formatting / reply formats.\n- Very creative, lots of unique swipes.\n- Is not restrictive during roleplays.","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.00000035","completion":"0.0000004","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"cognitivecomputations/dolphin-mixtral-8x22b","name":"Dolphin 2.9.2 Mixtral 8x22B 🐬","created":1717804800,"description":"Dolphin 2.9 is designed for instruction following, conversational, and coding. This model is a finetune of [Mixtral 8x22B Instruct](/models/mistralai/mixtral-8x22b-instruct). It features a 64k context length and was fine-tuned with a 16k sequence length using ChatML templates.\n\nThis model is a successor to [Dolphin Mixtral 8x7B](/models/cognitivecomputations/dolphin-mixtral-8x7b).\n\nThe model is uncensored and is stripped of alignment and bias. It requires an external alignment layer for ethical use. Users are cautioned to use this highly compliant model responsibly, as detailed in a blog post about uncensored models at [erichartford.com/uncensored-models](https://erichartford.com/uncensored-models).\n\n#moe #uncensored","context_length":16000,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"chatml"},"pricing":{"prompt":"0.0000009","completion":"0.0000009","image":"0","request":"0"},"top_provider":{"context_length":16000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"qwen/qwen-2-72b-instruct","name":"Qwen 2 72B Instruct","created":1717718400,"description":"Qwen2 72B is a transformer-based model that excels in language understanding, multilingual capabilities, coding, mathematics, and reasoning.\n\nIt features SwiGLU activation, attention QKV bias, and group query attention. It is pretrained on extensive data with supervised finetuning and direct preference optimization.\n\nFor more details, see this [blog post](https://qwenlm.github.io/blog/qwen2/) and [GitHub repo](https://github.com/QwenLM/Qwen2).\n\nUsage of this model is subject to [Tongyi Qianwen LICENSE AGREEMENT](https://huggingface.co/Qwen/Qwen1.5-110B-Chat/blob/main/LICENSE).","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Qwen","instruct_type":"chatml"},"pricing":{"prompt":"0.00000034","completion":"0.00000039","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"mistralai/mistral-7b-instruct:free","name":"Mistral: Mistral 7B Instruct (free)","created":1716768000,"description":"A high-performing, industry-standard 7.3B parameter model, with optimizations for speed and context length.\n\n*Mistral 7B Instruct has multiple version variants, and this is intended to be the latest version.*","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"mistral"},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"mistralai/mistral-7b-instruct","name":"Mistral: Mistral 7B Instruct","created":1716768000,"description":"A high-performing, industry-standard 7.3B parameter model, with optimizations for speed and context length.\n\n*Mistral 7B Instruct has multiple version variants, and this is intended to be the latest version.*","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"mistral"},"pricing":{"prompt":"0.00000003","completion":"0.000000055","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"mistralai/mistral-7b-instruct:nitro","name":"Mistral: Mistral 7B Instruct (nitro)","created":1716768000,"description":"A high-performing, industry-standard 7.3B parameter model, with optimizations for speed and context length.\n\n*Mistral 7B Instruct has multiple version variants, and this is intended to be the latest version.*","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"mistral"},"pricing":{"prompt":"0.00000007","completion":"0.00000007","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"mistralai/mistral-7b-instruct-v0.3","name":"Mistral: Mistral 7B Instruct v0.3","created":1716768000,"description":"A high-performing, industry-standard 7.3B parameter model, with optimizations for speed and context length.\n\nAn improved version of [Mistral 7B Instruct v0.2](/models/mistralai/mistral-7b-instruct-v0.2), with the following changes:\n\n- Extended vocabulary to 32768\n- Supports v3 Tokenizer\n- Supports function calling\n\nNOTE: Support for function calling depends on the provider.","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"mistral"},"pricing":{"prompt":"0.00000003","completion":"0.000000055","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"nousresearch/hermes-2-pro-llama-3-8b","name":"NousResearch: Hermes 2 Pro - Llama-3 8B","created":1716768000,"description":"Hermes 2 Pro is an upgraded, retrained version of Nous Hermes 2, consisting of an updated and cleaned version of the OpenHermes 2.5 Dataset, as well as a newly introduced Function Calling and JSON Mode dataset developed in-house.","context_length":131000,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"chatml"},"pricing":{"prompt":"0.000000025","completion":"0.00000004","image":"0","request":"0"},"top_provider":{"context_length":131000,"max_completion_tokens":131000,"is_moderated":false},"per_request_limits":null},{"id":"microsoft/phi-3-mini-128k-instruct:free","name":"Microsoft: Phi-3 Mini 128K Instruct (free)","created":1716681600,"description":"Phi-3 Mini is a powerful 3.8B parameter model designed for advanced language understanding, reasoning, and instruction following. Optimized through supervised fine-tuning and preference adjustments, it excels in tasks involving common sense, mathematics, logical reasoning, and code processing.\n\nAt time of release, Phi-3 Medium demonstrated state-of-the-art performance among lightweight models. This model is static, trained on an offline dataset with an October 2023 cutoff date.","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Other","instruct_type":"phi3"},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"microsoft/phi-3-mini-128k-instruct","name":"Microsoft: Phi-3 Mini 128K Instruct","created":1716681600,"description":"Phi-3 Mini is a powerful 3.8B parameter model designed for advanced language understanding, reasoning, and instruction following. Optimized through supervised fine-tuning and preference adjustments, it excels in tasks involving common sense, mathematics, logical reasoning, and code processing.\n\nAt time of release, Phi-3 Medium demonstrated state-of-the-art performance among lightweight models. This model is static, trained on an offline dataset with an October 2023 cutoff date.","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"Other","instruct_type":"phi3"},"pricing":{"prompt":"0.0000001","completion":"0.0000001","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"microsoft/phi-3-medium-128k-instruct:free","name":"Microsoft: Phi-3 Medium 128K Instruct (free)","created":1716508800,"description":"Phi-3 128K Medium is a powerful 14-billion parameter model designed for advanced language understanding, reasoning, and instruction following. Optimized through supervised fine-tuning and preference adjustments, it excels in tasks involving common sense, mathematics, logical reasoning, and code processing.\n\nAt time of release, Phi-3 Medium demonstrated state-of-the-art performance among lightweight models. In the MMLU-Pro eval, the model even comes close to a Llama3 70B level of performance.\n\nFor 4k context length, try [Phi-3 Medium 4K](/models/microsoft/phi-3-medium-4k-instruct).","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Other","instruct_type":"phi3"},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"microsoft/phi-3-medium-128k-instruct","name":"Microsoft: Phi-3 Medium 128K Instruct","created":1716508800,"description":"Phi-3 128K Medium is a powerful 14-billion parameter model designed for advanced language understanding, reasoning, and instruction following. Optimized through supervised fine-tuning and preference adjustments, it excels in tasks involving common sense, mathematics, logical reasoning, and code processing.\n\nAt time of release, Phi-3 Medium demonstrated state-of-the-art performance among lightweight models. In the MMLU-Pro eval, the model even comes close to a Llama3 70B level of performance.\n\nFor 4k context length, try [Phi-3 Medium 4K](/models/microsoft/phi-3-medium-4k-instruct).","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"Other","instruct_type":"phi3"},"pricing":{"prompt":"0.000001","completion":"0.000001","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"neversleep/llama-3-lumimaid-70b","name":"NeverSleep: Llama 3 Lumimaid 70B","created":1715817600,"description":"The NeverSleep team is back, with a Llama 3 70B finetune trained on their curated roleplay data. Striking a balance between eRP and RP, Lumimaid was designed to be serious, yet uncensored when necessary.\n\nTo enhance it's overall intelligence and chat capability, roughly 40% of the training data was not roleplay. This provides a breadth of knowledge to access, while still keeping roleplay as the primary strength.\n\nUsage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.000003375","completion":"0.0000045","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":2048,"is_moderated":false},"per_request_limits":null},{"id":"google/gemini-flash-1.5","name":"Google: Gemini Flash 1.5","created":1715644800,"description":"Gemini 1.5 Flash is a foundation model that performs well at a variety of multimodal tasks such as visual understanding, classification, summarization, and creating content from image, audio and video. It's adept at processing visual and text inputs such as photographs, documents, infographics, and screenshots.\n\nGemini 1.5 Flash is designed for high-volume, high-frequency tasks where cost and latency matter. On most common tasks, Flash achieves comparable quality to other Gemini Pro models at a significantly reduced cost. Flash is well-suited for applications like chat assistants and on-demand content generation where speed and scale matter.\n\nUsage of Gemini is subject to Google's [Gemini Terms of Use](https://ai.google.dev/terms).\n\n#multimodal","context_length":1000000,"architecture":{"modality":"text+image->text","tokenizer":"Gemini","instruct_type":null},"pricing":{"prompt":"0.000000075","completion":"0.0000003","image":"0.00004","request":"0"},"top_provider":{"context_length":1000000,"max_completion_tokens":8192,"is_moderated":false},"per_request_limits":null},{"id":"perplexity/llama-3-sonar-large-32k-chat","name":"Perplexity: Llama3 Sonar 70B","created":1715644800,"description":"Llama3 Sonar is Perplexity's latest model family. It surpasses their earlier Sonar models in cost-efficiency, speed, and performance.\n\nThis is a normal offline LLM, but the [online version](/models/perplexity/llama-3-sonar-large-32k-online) of this model has Internet access.","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":null},"pricing":{"prompt":"0.000001","completion":"0.000001","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"perplexity/llama-3-sonar-large-32k-online","name":"Perplexity: Llama3 Sonar 70B Online","created":1715644800,"description":"Llama3 Sonar is Perplexity's latest model family. It surpasses their earlier Sonar models in cost-efficiency, speed, and performance.\n\nThis is the online version of the [offline chat model](/models/perplexity/llama-3-sonar-large-32k-chat). It is focused on delivering helpful, up-to-date, and factual responses. #online","context_length":28000,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":null},"pricing":{"prompt":"0.000001","completion":"0.000001","image":"0","request":"0.005"},"top_provider":{"context_length":28000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"deepseek/deepseek-chat-v2.5","name":"DeepSeek V2.5","created":1715644800,"description":"DeepSeek-V2.5 is an upgraded version that combines DeepSeek-V2-Chat and DeepSeek-Coder-V2-Instruct. The new model integrates the general and coding abilities of the two previous versions. For model details, please visit [DeepSeek-V2 page](https://github.com/deepseek-ai/DeepSeek-V2) for more information.","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Other","instruct_type":null},"pricing":{"prompt":"0.000002","completion":"0.000002","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"perplexity/llama-3-sonar-small-32k-chat","name":"Perplexity: Llama3 Sonar 8B","created":1715644800,"description":"Llama3 Sonar is Perplexity's latest model family. It surpasses their earlier Sonar models in cost-efficiency, speed, and performance.\n\nThis is a normal offline LLM, but the [online version](/models/perplexity/llama-3-sonar-small-32k-online) of this model has Internet access.","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":null},"pricing":{"prompt":"0.0000002","completion":"0.0000002","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"openai/gpt-4o-2024-05-13","name":"OpenAI: GPT-4o (2024-05-13)","created":1715558400,"description":"GPT-4o (\"o\" for \"omni\") is OpenAI's latest AI model, supporting both text and image inputs with text outputs. It maintains the intelligence level of [GPT-4 Turbo](/models/openai/gpt-4-turbo) while being twice as fast and 50% more cost-effective. GPT-4o also offers improved performance in processing non-English languages and enhanced visual capabilities.\n\nFor benchmarking against other models, it was briefly called [\"im-also-a-good-gpt2-chatbot\"](https://twitter.com/LiamFedus/status/1790064963966370209)\n\n#multimodal","context_length":128000,"architecture":{"modality":"text+image->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.000005","completion":"0.000015","image":"0.007225","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":4096,"is_moderated":true},"per_request_limits":null},{"id":"meta-llama/llama-guard-2-8b","name":"Meta: LlamaGuard 2 8B","created":1715558400,"description":"This safeguard model has 8B parameters and is based on the Llama 3 family. Just like is predecessor, [LlamaGuard 1](https://huggingface.co/meta-llama/LlamaGuard-7b), it can do both prompt and response classification.\n\nLlamaGuard 2 acts as a normal LLM would, generating text that indicates whether the given input/output is safe/unsafe. If deemed unsafe, it will also share the content categories violated.\n\nFor best results, please use raw prompt input or the `/completions` endpoint, instead of the chat API.\n\nIt has demonstrated strong performance compared to leading closed-source models in human evaluations.\n\nTo read more about the model release, [click here](https://ai.meta.com/blog/meta-llama-3/). Usage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"none"},"pricing":{"prompt":"0.00000018","completion":"0.00000018","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"openai/gpt-4o","name":"OpenAI: GPT-4o","created":1715558400,"description":"GPT-4o (\"o\" for \"omni\") is OpenAI's latest AI model, supporting both text and image inputs with text outputs. It maintains the intelligence level of [GPT-4 Turbo](/models/openai/gpt-4-turbo) while being twice as fast and 50% more cost-effective. GPT-4o also offers improved performance in processing non-English languages and enhanced visual capabilities.\n\nFor benchmarking against other models, it was briefly called [\"im-also-a-good-gpt2-chatbot\"](https://twitter.com/LiamFedus/status/1790064963966370209)\n\n#multimodal","context_length":128000,"architecture":{"modality":"text+image->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.0000025","completion":"0.00001","image":"0.003613","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":16384,"is_moderated":true},"per_request_limits":null},{"id":"openai/gpt-4o:extended","name":"OpenAI: GPT-4o (extended)","created":1715558400,"description":"GPT-4o (\"o\" for \"omni\") is OpenAI's latest AI model, supporting both text and image inputs with text outputs. It maintains the intelligence level of [GPT-4 Turbo](/models/openai/gpt-4-turbo) while being twice as fast and 50% more cost-effective. GPT-4o also offers improved performance in processing non-English languages and enhanced visual capabilities.\n\nFor benchmarking against other models, it was briefly called [\"im-also-a-good-gpt2-chatbot\"](https://twitter.com/LiamFedus/status/1790064963966370209)\n\n#multimodal","context_length":128000,"architecture":{"modality":"text+image->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.000006","completion":"0.000018","image":"0.007225","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":64000,"is_moderated":true},"per_request_limits":null},{"id":"neversleep/llama-3-lumimaid-8b:extended","name":"NeverSleep: Llama 3 Lumimaid 8B (extended)","created":1714780800,"description":"The NeverSleep team is back, with a Llama 3 8B finetune trained on their curated roleplay data. Striking a balance between eRP and RP, Lumimaid was designed to be serious, yet uncensored when necessary.\n\nTo enhance it's overall intelligence and chat capability, roughly 40% of the training data was not roleplay. This provides a breadth of knowledge to access, while still keeping roleplay as the primary strength.\n\nUsage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":24576,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.0000001875","completion":"0.000001125","image":"0","request":"0"},"top_provider":{"context_length":24576,"max_completion_tokens":2048,"is_moderated":false},"per_request_limits":null},{"id":"neversleep/llama-3-lumimaid-8b","name":"NeverSleep: Llama 3 Lumimaid 8B","created":1714780800,"description":"The NeverSleep team is back, with a Llama 3 8B finetune trained on their curated roleplay data. Striking a balance between eRP and RP, Lumimaid was designed to be serious, yet uncensored when necessary.\n\nTo enhance it's overall intelligence and chat capability, roughly 40% of the training data was not roleplay. This provides a breadth of knowledge to access, while still keeping roleplay as the primary strength.\n\nUsage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":24576,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.0000001875","completion":"0.000001125","image":"0","request":"0"},"top_provider":{"context_length":24576,"max_completion_tokens":2048,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3-8b-instruct:free","name":"Meta: Llama 3 8B Instruct (free)","created":1713398400,"description":"Meta's latest class of model (Llama 3) launched with a variety of sizes & flavors. This 8B instruct-tuned version was optimized for high quality dialogue usecases.\n\nIt has demonstrated strong performance compared to leading closed-source models in human evaluations.\n\nTo read more about the model release, [click here](https://ai.meta.com/blog/meta-llama-3/). Usage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3-8b-instruct","name":"Meta: Llama 3 8B Instruct","created":1713398400,"description":"Meta's latest class of model (Llama 3) launched with a variety of sizes & flavors. This 8B instruct-tuned version was optimized for high quality dialogue usecases.\n\nIt has demonstrated strong performance compared to leading closed-source models in human evaluations.\n\nTo read more about the model release, [click here](https://ai.meta.com/blog/meta-llama-3/). Usage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.00000003","completion":"0.00000006","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3-8b-instruct:extended","name":"Meta: Llama 3 8B Instruct (extended)","created":1713398400,"description":"Meta's latest class of model (Llama 3) launched with a variety of sizes & flavors. This 8B instruct-tuned version was optimized for high quality dialogue usecases.\n\nIt has demonstrated strong performance compared to leading closed-source models in human evaluations.\n\nTo read more about the model release, [click here](https://ai.meta.com/blog/meta-llama-3/). Usage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":16384,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.0000001875","completion":"0.000001125","image":"0","request":"0"},"top_provider":{"context_length":16384,"max_completion_tokens":2048,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3-8b-instruct:nitro","name":"Meta: Llama 3 8B Instruct (nitro)","created":1713398400,"description":"Meta's latest class of model (Llama 3) launched with a variety of sizes & flavors. This 8B instruct-tuned version was optimized for high quality dialogue usecases.\n\nIt has demonstrated strong performance compared to leading closed-source models in human evaluations.\n\nTo read more about the model release, [click here](https://ai.meta.com/blog/meta-llama-3/). Usage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.0000002","completion":"0.0000002","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3-70b-instruct","name":"Meta: Llama 3 70B Instruct","created":1713398400,"description":"Meta's latest class of model (Llama 3) launched with a variety of sizes & flavors. This 70B instruct-tuned version was optimized for high quality dialogue usecases.\n\nIt has demonstrated strong performance compared to leading closed-source models in human evaluations.\n\nTo read more about the model release, [click here](https://ai.meta.com/blog/meta-llama-3/). Usage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.00000023","completion":"0.0000004","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-3-70b-instruct:nitro","name":"Meta: Llama 3 70B Instruct (nitro)","created":1713398400,"description":"Meta's latest class of model (Llama 3) launched with a variety of sizes & flavors. This 70B instruct-tuned version was optimized for high quality dialogue usecases.\n\nIt has demonstrated strong performance compared to leading closed-source models in human evaluations.\n\nTo read more about the model release, [click here](https://ai.meta.com/blog/meta-llama-3/). Usage of this model is subject to [Meta's Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Llama3","instruct_type":"llama3"},"pricing":{"prompt":"0.000000792","completion":"0.000000792","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"mistralai/mixtral-8x22b-instruct","name":"Mistral: Mixtral 8x22B Instruct","created":1713312000,"description":"Mistral's official instruct fine-tuned version of [Mixtral 8x22B](/models/mistralai/mixtral-8x22b). It uses 39B active parameters out of 141B, offering unparalleled cost efficiency for its size. Its strengths include:\n- strong math, coding, and reasoning\n- large context length (64k)\n- fluency in English, French, Italian, German, and Spanish\n\nSee benchmarks on the launch announcement [here](https://mistral.ai/news/mixtral-8x22b/).\n#moe","context_length":65536,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"mistral"},"pricing":{"prompt":"0.0000009","completion":"0.0000009","image":"0","request":"0"},"top_provider":{"context_length":65536,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"microsoft/wizardlm-2-8x22b","name":"WizardLM-2 8x22B","created":1713225600,"description":"WizardLM-2 8x22B is Microsoft AI's most advanced Wizard model. It demonstrates highly competitive performance compared to leading proprietary models, and it consistently outperforms all existing state-of-the-art opensource models.\n\nIt is an instruct finetune of [Mixtral 8x22B](/models/mistralai/mixtral-8x22b).\n\nTo read more about the model release, [click here](https://wizardlm.github.io/WizardLM2/).\n\n#moe","context_length":65536,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"vicuna"},"pricing":{"prompt":"0.0000005","completion":"0.0000005","image":"0","request":"0"},"top_provider":{"context_length":65536,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"microsoft/wizardlm-2-7b","name":"WizardLM-2 7B","created":1713225600,"description":"WizardLM-2 7B is the smaller variant of Microsoft AI's latest Wizard model. It is the fastest and achieves comparable performance with existing 10x larger opensource leading models\n\nIt is a finetune of [Mistral 7B Instruct](/models/mistralai/mistral-7b-instruct), using the same technique as [WizardLM-2 8x22B](/models/microsoft/wizardlm-2-8x22b).\n\nTo read more about the model release, [click here](https://wizardlm.github.io/WizardLM2/).\n\n#moe","context_length":32000,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"vicuna"},"pricing":{"prompt":"0.000000055","completion":"0.000000055","image":"0","request":"0"},"top_provider":{"context_length":32000,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"google/gemini-pro-1.5","name":"Google: Gemini Pro 1.5","created":1712620800,"description":"Google's latest multimodal model, supports image and video[0] in text or chat prompts.\n\nOptimized for language tasks including:\n\n- Code generation\n- Text generation\n- Text editing\n- Problem solving\n- Recommendations\n- Information extraction\n- Data extraction or generation\n- AI agents\n\nUsage of Gemini is subject to Google's [Gemini Terms of Use](https://ai.google.dev/terms).\n\n* [0]: Video input is not available through OpenRouter at this time.","context_length":2000000,"architecture":{"modality":"text+image->text","tokenizer":"Gemini","instruct_type":null},"pricing":{"prompt":"0.00000125","completion":"0.000005","image":"0.0006575","request":"0"},"top_provider":{"context_length":2000000,"max_completion_tokens":8192,"is_moderated":false},"per_request_limits":null},{"id":"openai/gpt-4-turbo","name":"OpenAI: GPT-4 Turbo","created":1712620800,"description":"The latest GPT-4 Turbo model with vision capabilities. Vision requests can now use JSON mode and function calling.\n\nTraining data: up to December 2023.","context_length":128000,"architecture":{"modality":"text+image->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.00001","completion":"0.00003","image":"0.01445","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":4096,"is_moderated":true},"per_request_limits":null},{"id":"cohere/command-r-plus","name":"Cohere: Command R+","created":1712188800,"description":"Command R+ is a new, 104B-parameter LLM from Cohere. It's useful for roleplay, general consumer usecases, and Retrieval Augmented Generation (RAG).\n\nIt offers multilingual support for ten key languages to facilitate global business operations. See benchmarks and the launch post [here](https://txt.cohere.com/command-r-plus-microsoft-azure/).\n\nUse of this model is subject to Cohere's [Acceptable Use Policy](https://docs.cohere.com/docs/c4ai-acceptable-use-policy).","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"Cohere","instruct_type":null},"pricing":{"prompt":"0.00000285","completion":"0.00001425","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":4000,"is_moderated":false},"per_request_limits":null},{"id":"cohere/command-r-plus-04-2024","name":"Cohere: Command R+ (04-2024)","created":1712016000,"description":"Command R+ is a new, 104B-parameter LLM from Cohere. It's useful for roleplay, general consumer usecases, and Retrieval Augmented Generation (RAG).\n\nIt offers multilingual support for ten key languages to facilitate global business operations. See benchmarks and the launch post [here](https://txt.cohere.com/command-r-plus-microsoft-azure/).\n\nUse of this model is subject to Cohere's [Acceptable Use Policy](https://docs.cohere.com/docs/c4ai-acceptable-use-policy).","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"Cohere","instruct_type":null},"pricing":{"prompt":"0.00000285","completion":"0.00001425","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":4000,"is_moderated":false},"per_request_limits":null},{"id":"databricks/dbrx-instruct","name":"Databricks: DBRX 132B Instruct","created":1711670400,"description":"DBRX is a new open source large language model developed by Databricks. At 132B, it outperforms existing open source LLMs like Llama 2 70B and [Mixtral-8x7b](/models/mistralai/mixtral-8x7b) on standard industry benchmarks for language understanding, programming, math, and logic.\n\nIt uses a fine-grained mixture-of-experts (MoE) architecture. 36B parameters are active on any input. It was pre-trained on 12T tokens of text and code data. Compared to other open MoE models like Mixtral-8x7B and Grok-1, DBRX is fine-grained, meaning it uses a larger number of smaller experts.\n\nSee the launch announcement and benchmark results [here](https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm).\n\n#moe","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Other","instruct_type":"chatml"},"pricing":{"prompt":"0.00000108","completion":"0.00000108","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"sophosympatheia/midnight-rose-70b","name":"Midnight Rose 70B","created":1711065600,"description":"A merge with a complex family tree, this model was crafted for roleplaying and storytelling. Midnight Rose is a successor to Rogue Rose and Aurora Nights and improves upon them both. It wants to produce lengthy output by default and is the best creative writing merge produced so far by sophosympatheia.\n\nDescending from earlier versions of Midnight Rose and [Wizard Tulu Dolphin 70B](https://huggingface.co/sophosympatheia/Wizard-Tulu-Dolphin-70B-v1.0), it inherits the best qualities of each.","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Llama2","instruct_type":"airoboros"},"pricing":{"prompt":"0.0000008","completion":"0.0000008","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"cohere/command","name":"Cohere: Command","created":1710374400,"description":"Command is an instruction-following conversational model that performs language tasks with high quality, more reliably and with a longer context than our base generative models.\n\nUse of this model is subject to Cohere's [Acceptable Use Policy](https://docs.cohere.com/docs/c4ai-acceptable-use-policy).","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Cohere","instruct_type":null},"pricing":{"prompt":"0.00000095","completion":"0.0000019","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":4000,"is_moderated":false},"per_request_limits":null},{"id":"cohere/command-r","name":"Cohere: Command R","created":1710374400,"description":"Command-R is a 35B parameter model that performs conversational language tasks at a higher quality, more reliably, and with a longer context than previous models. It can be used for complex workflows like code generation, retrieval augmented generation (RAG), tool use, and agents.\n\nRead the launch post [here](https://txt.cohere.com/command-r/).\n\nUse of this model is subject to Cohere's [Acceptable Use Policy](https://docs.cohere.com/docs/c4ai-acceptable-use-policy).","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"Cohere","instruct_type":null},"pricing":{"prompt":"0.000000475","completion":"0.000001425","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":4000,"is_moderated":false},"per_request_limits":null},{"id":"anthropic/claude-3-haiku:beta","name":"Anthropic: Claude 3 Haiku (self-moderated)","created":1710288000,"description":"Claude 3 Haiku is Anthropic's fastest and most compact model for\nnear-instant responsiveness. Quick and accurate targeted performance.\n\nSee the launch announcement and benchmark results [here](https://www.anthropic.com/news/claude-3-haiku)\n\n#multimodal","context_length":200000,"architecture":{"modality":"text+image->text","tokenizer":"Claude","instruct_type":null},"pricing":{"prompt":"0.00000025","completion":"0.00000125","image":"0.0004","request":"0"},"top_provider":{"context_length":200000,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"anthropic/claude-3-haiku","name":"Anthropic: Claude 3 Haiku","created":1710288000,"description":"Claude 3 Haiku is Anthropic's fastest and most compact model for\nnear-instant responsiveness. Quick and accurate targeted performance.\n\nSee the launch announcement and benchmark results [here](https://www.anthropic.com/news/claude-3-haiku)\n\n#multimodal","context_length":200000,"architecture":{"modality":"text+image->text","tokenizer":"Claude","instruct_type":null},"pricing":{"prompt":"0.00000025","completion":"0.00000125","image":"0.0004","request":"0"},"top_provider":{"context_length":200000,"max_completion_tokens":4096,"is_moderated":true},"per_request_limits":null},{"id":"anthropic/claude-3-opus:beta","name":"Anthropic: Claude 3 Opus (self-moderated)","created":1709596800,"description":"Claude 3 Opus is Anthropic's most powerful model for highly complex tasks. It boasts top-level performance, intelligence, fluency, and understanding.\n\nSee the launch announcement and benchmark results [here](https://www.anthropic.com/news/claude-3-family)\n\n#multimodal","context_length":200000,"architecture":{"modality":"text+image->text","tokenizer":"Claude","instruct_type":null},"pricing":{"prompt":"0.000015","completion":"0.000075","image":"0.024","request":"0"},"top_provider":{"context_length":200000,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"anthropic/claude-3-opus","name":"Anthropic: Claude 3 Opus","created":1709596800,"description":"Claude 3 Opus is Anthropic's most powerful model for highly complex tasks. It boasts top-level performance, intelligence, fluency, and understanding.\n\nSee the launch announcement and benchmark results [here](https://www.anthropic.com/news/claude-3-family)\n\n#multimodal","context_length":200000,"architecture":{"modality":"text+image->text","tokenizer":"Claude","instruct_type":null},"pricing":{"prompt":"0.000015","completion":"0.000075","image":"0.024","request":"0"},"top_provider":{"context_length":200000,"max_completion_tokens":4096,"is_moderated":true},"per_request_limits":null},{"id":"anthropic/claude-3-sonnet:beta","name":"Anthropic: Claude 3 Sonnet (self-moderated)","created":1709596800,"description":"Claude 3 Sonnet is an ideal balance of intelligence and speed for enterprise workloads. Maximum utility at a lower price, dependable, balanced for scaled deployments.\n\nSee the launch announcement and benchmark results [here](https://www.anthropic.com/news/claude-3-family)\n\n#multimodal","context_length":200000,"architecture":{"modality":"text+image->text","tokenizer":"Claude","instruct_type":null},"pricing":{"prompt":"0.000003","completion":"0.000015","image":"0.0048","request":"0"},"top_provider":{"context_length":200000,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"anthropic/claude-3-sonnet","name":"Anthropic: Claude 3 Sonnet","created":1709596800,"description":"Claude 3 Sonnet is an ideal balance of intelligence and speed for enterprise workloads. Maximum utility at a lower price, dependable, balanced for scaled deployments.\n\nSee the launch announcement and benchmark results [here](https://www.anthropic.com/news/claude-3-family)\n\n#multimodal","context_length":200000,"architecture":{"modality":"text+image->text","tokenizer":"Claude","instruct_type":null},"pricing":{"prompt":"0.000003","completion":"0.000015","image":"0.0048","request":"0"},"top_provider":{"context_length":200000,"max_completion_tokens":4096,"is_moderated":true},"per_request_limits":null},{"id":"cohere/command-r-03-2024","name":"Cohere: Command R (03-2024)","created":1709341200,"description":"Command-R is a 35B parameter model that performs conversational language tasks at a higher quality, more reliably, and with a longer context than previous models. It can be used for complex workflows like code generation, retrieval augmented generation (RAG), tool use, and agents.\n\nRead the launch post [here](https://txt.cohere.com/command-r/).\n\nUse of this model is subject to Cohere's [Acceptable Use Policy](https://docs.cohere.com/docs/c4ai-acceptable-use-policy).","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"Cohere","instruct_type":null},"pricing":{"prompt":"0.000000475","completion":"0.000001425","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":4000,"is_moderated":false},"per_request_limits":null},{"id":"mistralai/mistral-large","name":"Mistral Large","created":1708905600,"description":"This is Mistral AI's flagship model, Mistral Large 2 (version `mistral-large-2407`). It's a proprietary weights-available model and excels at reasoning, code, JSON, chat, and more. Read the launch announcement [here](https://mistral.ai/news/mistral-large-2407/).\n\nIt supports dozens of languages including French, German, Spanish, Italian, Portuguese, Arabic, Hindi, Russian, Chinese, Japanese, and Korean, along with 80+ coding languages including Python, Java, C, C++, JavaScript, and Bash. Its long context window allows precise information recall from large documents.","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":null},"pricing":{"prompt":"0.000002","completion":"0.000006","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"openai/gpt-3.5-turbo-0613","name":"OpenAI: GPT-3.5 Turbo (older v0613)","created":1706140800,"description":"GPT-3.5 Turbo is OpenAI's fastest model. It can understand and generate natural language or code, and is optimized for chat and traditional completion tasks.\n\nTraining data up to Sep 2021.","context_length":4095,"architecture":{"modality":"text->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.000001","completion":"0.000002","image":"0","request":"0"},"top_provider":{"context_length":4095,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"openai/gpt-4-turbo-preview","name":"OpenAI: GPT-4 Turbo Preview","created":1706140800,"description":"The preview GPT-4 model with improved instruction following, JSON mode, reproducible outputs, parallel function calling, and more. Training data: up to Dec 2023.\n\n**Note:** heavily rate limited by OpenAI while in preview.","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.00001","completion":"0.00003","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":4096,"is_moderated":true},"per_request_limits":null},{"id":"nousresearch/nous-hermes-2-mixtral-8x7b-dpo","name":"Nous: Hermes 2 Mixtral 8x7B DPO","created":1705363200,"description":"Nous Hermes 2 Mixtral 8x7B DPO is the new flagship Nous Research model trained over the [Mixtral 8x7B MoE LLM](/models/mistralai/mixtral-8x7b).\n\nThe model was trained on over 1,000,000 entries of primarily [GPT-4](/models/openai/gpt-4) generated data, as well as other high quality data from open datasets across the AI landscape, achieving state of the art performance on a variety of tasks.\n\n#moe","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"chatml"},"pricing":{"prompt":"0.00000054","completion":"0.00000054","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"mistralai/mistral-small","name":"Mistral Small","created":1704844800,"description":"With 22 billion parameters, Mistral Small v24.09 offers a convenient mid-point between (Mistral NeMo 12B)[/mistralai/mistral-nemo] and (Mistral Large 2)[/mistralai/mistral-large], providing a cost-effective solution that can be deployed across various platforms and environments. It has better reasoning, exhibits more capabilities, can produce and reason about code, and is multiligual, supporting English, French, German, Italian, and Spanish.","context_length":32000,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":null},"pricing":{"prompt":"0.0000002","completion":"0.0000006","image":"0","request":"0"},"top_provider":{"context_length":32000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"mistralai/mistral-tiny","name":"Mistral Tiny","created":1704844800,"description":"This model is currently powered by Mistral-7B-v0.2, and incorporates a \"better\" fine-tuning than [Mistral 7B](/models/mistralai/mistral-7b-instruct-v0.1), inspired by community work. It's best used for large batch processing tasks where cost is a significant factor but reasoning capabilities are not crucial.","context_length":32000,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":null},"pricing":{"prompt":"0.00000025","completion":"0.00000025","image":"0","request":"0"},"top_provider":{"context_length":32000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"mistralai/mistral-medium","name":"Mistral Medium","created":1704844800,"description":"This is Mistral AI's closed-source, medium-sided model. It's powered by a closed-source prototype and excels at reasoning, code, JSON, chat, and more. In benchmarks, it compares with many of the flagship models of other companies.","context_length":32000,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":null},"pricing":{"prompt":"0.00000275","completion":"0.0000081","image":"0","request":"0"},"top_provider":{"context_length":32000,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"mistralai/mistral-7b-instruct-v0.2","name":"Mistral: Mistral 7B Instruct v0.2","created":1703721600,"description":"A high-performing, industry-standard 7.3B parameter model, with optimizations for speed and context length.\n\nAn improved version of [Mistral 7B Instruct](/modelsmistralai/mistral-7b-instruct-v0.1), with the following changes:\n\n- 32k context window (vs 8k context in v0.1)\n- Rope-theta = 1e6\n- No Sliding-Window Attention","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"mistral"},"pricing":{"prompt":"0.00000018","completion":"0.00000018","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"cognitivecomputations/dolphin-mixtral-8x7b","name":"Dolphin 2.6 Mixtral 8x7B 🐬","created":1703116800,"description":"This is a 16k context fine-tune of [Mixtral-8x7b](/models/mistralai/mixtral-8x7b). It excels in coding tasks due to extensive training with coding data and is known for its obedience, although it lacks DPO tuning.\n\nThe model is uncensored and is stripped of alignment and bias. It requires an external alignment layer for ethical use. Users are cautioned to use this highly compliant model responsibly, as detailed in a blog post about uncensored models at [erichartford.com/uncensored-models](https://erichartford.com/uncensored-models).\n\n#moe #uncensored","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"chatml"},"pricing":{"prompt":"0.0000005","completion":"0.0000005","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"google/gemini-pro-vision","name":"Google: Gemini Pro Vision 1.0","created":1702425600,"description":"Google's flagship multimodal model, supporting image and video in text or chat prompts for a text or code response.\n\nSee the benchmarks and prompting guidelines from [Deepmind](https://deepmind.google/technologies/gemini/).\n\nUsage of Gemini is subject to Google's [Gemini Terms of Use](https://ai.google.dev/terms).\n\n#multimodal","context_length":16384,"architecture":{"modality":"text+image->text","tokenizer":"Gemini","instruct_type":null},"pricing":{"prompt":"0.0000005","completion":"0.0000015","image":"0.0025","request":"0"},"top_provider":{"context_length":16384,"max_completion_tokens":2048,"is_moderated":false},"per_request_limits":null},{"id":"google/gemini-pro","name":"Google: Gemini Pro 1.0","created":1702425600,"description":"Google's flagship text generation model. Designed to handle natural language tasks, multiturn text and code chat, and code generation.\n\nSee the benchmarks and prompting guidelines from [Deepmind](https://deepmind.google/technologies/gemini/).\n\nUsage of Gemini is subject to Google's [Gemini Terms of Use](https://ai.google.dev/terms).","context_length":32760,"architecture":{"modality":"text->text","tokenizer":"Gemini","instruct_type":null},"pricing":{"prompt":"0.0000005","completion":"0.0000015","image":"0.0025","request":"0"},"top_provider":{"context_length":32760,"max_completion_tokens":8192,"is_moderated":false},"per_request_limits":null},{"id":"mistralai/mixtral-8x7b","name":"Mistral: Mixtral 8x7B (base)","created":1702166400,"description":"Mixtral 8x7B is a pretrained generative Sparse Mixture of Experts, by Mistral AI. Incorporates 8 experts (feed-forward networks) for a total of 47B parameters. Base model (not fine-tuned for instructions) - see [Mixtral 8x7B Instruct](/models/mistralai/mixtral-8x7b-instruct) for an instruct-tuned model.\n\n#moe","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"none"},"pricing":{"prompt":"0.00000054","completion":"0.00000054","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"mistralai/mixtral-8x7b-instruct","name":"Mistral: Mixtral 8x7B Instruct","created":1702166400,"description":"Mixtral 8x7B Instruct is a pretrained generative Sparse Mixture of Experts, by Mistral AI, for chat and instruction use. Incorporates 8 experts (feed-forward networks) for a total of 47 billion parameters.\n\nInstruct model fine-tuned by Mistral. #moe","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"mistral"},"pricing":{"prompt":"0.00000024","completion":"0.00000024","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"mistralai/mixtral-8x7b-instruct:nitro","name":"Mistral: Mixtral 8x7B Instruct (nitro)","created":1702166400,"description":"Mixtral 8x7B Instruct is a pretrained generative Sparse Mixture of Experts, by Mistral AI, for chat and instruction use. Incorporates 8 experts (feed-forward networks) for a total of 47 billion parameters.\n\nInstruct model fine-tuned by Mistral. #moe","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"mistral"},"pricing":{"prompt":"0.00000054","completion":"0.00000054","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"openchat/openchat-7b:free","name":"OpenChat 3.5 7B (free)","created":1701129600,"description":"OpenChat 7B is a library of open-source language models, fine-tuned with \"C-RLFT (Conditioned Reinforcement Learning Fine-Tuning)\" - a strategy inspired by offline reinforcement learning. It has been trained on mixed-quality data without preference labels.\n\n- For OpenChat fine-tuned on Mistral 7B, check out [OpenChat 7B](/models/openchat/openchat-7b).\n- For OpenChat fine-tuned on Llama 8B, check out [OpenChat 8B](/models/openchat/openchat-8b).\n\n#open-source","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"openchat"},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"openchat/openchat-7b","name":"OpenChat 3.5 7B","created":1701129600,"description":"OpenChat 7B is a library of open-source language models, fine-tuned with \"C-RLFT (Conditioned Reinforcement Learning Fine-Tuning)\" - a strategy inspired by offline reinforcement learning. It has been trained on mixed-quality data without preference labels.\n\n- For OpenChat fine-tuned on Mistral 7B, check out [OpenChat 7B](/models/openchat/openchat-7b).\n- For OpenChat fine-tuned on Llama 8B, check out [OpenChat 8B](/models/openchat/openchat-8b).\n\n#open-source","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"openchat"},"pricing":{"prompt":"0.000000055","completion":"0.000000055","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"neversleep/noromaid-20b","name":"Noromaid 20B","created":1700956800,"description":"A collab between IkariDev and Undi. This merge is suitable for RP, ERP, and general knowledge.\n\n#merge #uncensored","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Llama2","instruct_type":"alpaca"},"pricing":{"prompt":"0.0000015","completion":"0.00000225","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":2048,"is_moderated":false},"per_request_limits":null},{"id":"anthropic/claude-2:beta","name":"Anthropic: Claude v2 (self-moderated)","created":1700611200,"description":"Claude 2 delivers advancements in key capabilities for enterprises—including an industry-leading 200K token context window, significant reductions in rates of model hallucination, system prompts and a new beta feature: tool use.","context_length":200000,"architecture":{"modality":"text->text","tokenizer":"Claude","instruct_type":null},"pricing":{"prompt":"0.000008","completion":"0.000024","image":"0","request":"0"},"top_provider":{"context_length":200000,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"anthropic/claude-2","name":"Anthropic: Claude v2","created":1700611200,"description":"Claude 2 delivers advancements in key capabilities for enterprises—including an industry-leading 200K token context window, significant reductions in rates of model hallucination, system prompts and a new beta feature: tool use.","context_length":200000,"architecture":{"modality":"text->text","tokenizer":"Claude","instruct_type":null},"pricing":{"prompt":"0.000008","completion":"0.000024","image":"0","request":"0"},"top_provider":{"context_length":200000,"max_completion_tokens":4096,"is_moderated":true},"per_request_limits":null},{"id":"anthropic/claude-2.1:beta","name":"Anthropic: Claude v2.1 (self-moderated)","created":1700611200,"description":"Claude 2 delivers advancements in key capabilities for enterprises—including an industry-leading 200K token context window, significant reductions in rates of model hallucination, system prompts and a new beta feature: tool use.","context_length":200000,"architecture":{"modality":"text->text","tokenizer":"Claude","instruct_type":null},"pricing":{"prompt":"0.000008","completion":"0.000024","image":"0","request":"0"},"top_provider":{"context_length":200000,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"anthropic/claude-2.1","name":"Anthropic: Claude v2.1","created":1700611200,"description":"Claude 2 delivers advancements in key capabilities for enterprises—including an industry-leading 200K token context window, significant reductions in rates of model hallucination, system prompts and a new beta feature: tool use.","context_length":200000,"architecture":{"modality":"text->text","tokenizer":"Claude","instruct_type":null},"pricing":{"prompt":"0.000008","completion":"0.000024","image":"0","request":"0"},"top_provider":{"context_length":200000,"max_completion_tokens":4096,"is_moderated":true},"per_request_limits":null},{"id":"teknium/openhermes-2.5-mistral-7b","name":"OpenHermes 2.5 Mistral 7B","created":1700438400,"description":"A continuation of [OpenHermes 2 model](/models/teknium/openhermes-2-mistral-7b), trained on additional code datasets.\nPotentially the most interesting finding from training on a good ratio (est. of around 7-14% of the total dataset) of code instruction was that it has boosted several non-code benchmarks, including TruthfulQA, AGIEval, and GPT4All suite. It did however reduce BigBench benchmark score, but the net gain overall is significant.","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"chatml"},"pricing":{"prompt":"0.00000017","completion":"0.00000017","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"lizpreciatior/lzlv-70b-fp16-hf","name":"lzlv 70B","created":1699747200,"description":"A Mythomax/MLewd_13B-style merge of selected 70B models.\nA multi-model merge of several LLaMA2 70B finetunes for roleplaying and creative work. The goal was to create a model that combines creativity with intelligence for an enhanced experience.\n\n#merge #uncensored","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Llama2","instruct_type":"airoboros"},"pricing":{"prompt":"0.00000035","completion":"0.0000004","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"undi95/toppy-m-7b:free","name":"Toppy M 7B (free)","created":1699574400,"description":"A wild 7B parameter model that merges several models using the new task_arithmetic merge method from mergekit.\nList of merged models:\n- NousResearch/Nous-Capybara-7B-V1.9\n- [HuggingFaceH4/zephyr-7b-beta](/models/huggingfaceh4/zephyr-7b-beta)\n- lemonilia/AshhLimaRP-Mistral-7B\n- Vulkane/120-Days-of-Sodom-LoRA-Mistral-7b\n- Undi95/Mistral-pippa-sharegpt-7b-qlora\n\n#merge #uncensored","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"alpaca"},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":2048,"is_moderated":false},"per_request_limits":null},{"id":"undi95/toppy-m-7b:nitro","name":"Toppy M 7B (nitro)","created":1699574400,"description":"A wild 7B parameter model that merges several models using the new task_arithmetic merge method from mergekit.\nList of merged models:\n- NousResearch/Nous-Capybara-7B-V1.9\n- [HuggingFaceH4/zephyr-7b-beta](/models/huggingfaceh4/zephyr-7b-beta)\n- lemonilia/AshhLimaRP-Mistral-7B\n- Vulkane/120-Days-of-Sodom-LoRA-Mistral-7b\n- Undi95/Mistral-pippa-sharegpt-7b-qlora\n\n#merge #uncensored","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"alpaca"},"pricing":{"prompt":"0.00000007","completion":"0.00000007","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"undi95/toppy-m-7b","name":"Toppy M 7B","created":1699574400,"description":"A wild 7B parameter model that merges several models using the new task_arithmetic merge method from mergekit.\nList of merged models:\n- NousResearch/Nous-Capybara-7B-V1.9\n- [HuggingFaceH4/zephyr-7b-beta](/models/huggingfaceh4/zephyr-7b-beta)\n- lemonilia/AshhLimaRP-Mistral-7B\n- Vulkane/120-Days-of-Sodom-LoRA-Mistral-7b\n- Undi95/Mistral-pippa-sharegpt-7b-qlora\n\n#merge #uncensored","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"alpaca"},"pricing":{"prompt":"0.00000007","completion":"0.00000007","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"alpindale/goliath-120b","name":"Goliath 120B","created":1699574400,"description":"A large LLM created by combining two fine-tuned Llama 70B models into one 120B model. Combines Xwin and Euryale.\n\nCredits to\n- [@chargoddard](https://huggingface.co/chargoddard) for developing the framework used to merge the model - [mergekit](https://github.com/cg123/mergekit).\n- [@Undi95](https://huggingface.co/Undi95) for helping with the merge ratios.\n\n#merge","context_length":6144,"architecture":{"modality":"text->text","tokenizer":"Llama2","instruct_type":"airoboros"},"pricing":{"prompt":"0.000009375","completion":"0.000009375","image":"0","request":"0"},"top_provider":{"context_length":6144,"max_completion_tokens":512,"is_moderated":false},"per_request_limits":null},{"id":"openrouter/auto","name":"Auto Router (best for prompt)","created":1699401600,"description":"Your prompt will be processed by a meta-model and routed to one of dozens of models (see below), optimizing for the best possible output.\n\nTo see which model was used, visit [Activity](/activity), or read the `model` attribute of the response. Your response will be priced at the same rate as the routed model.\n\nLearn more about how Not Diamond's meta-model works [here](https://docs.notdiamond.ai/docs/how-not-diamond-works).\n\nRequests will be routed to the following models:\n- [openai/gpt-4o-2024-08-06](/openai/gpt-4o-2024-08-06)\n- [openai/gpt-4o-2024-05-13](/openai/gpt-4o-2024-05-13)\n- [openai/gpt-4o-mini-2024-07-18](/openai/gpt-4o-mini-2024-07-18)\n- [openai/chatgpt-4o-latest](/openai/chatgpt-4o-latest)\n- [openai/o1-preview-2024-09-12](/openai/o1-preview-2024-09-12)\n- [openai/o1-mini-2024-09-12](/openai/o1-mini-2024-09-12)\n- [anthropic/claude-3.5-sonnet](/anthropic/claude-3.5-sonnet)\n- [anthropic/claude-3.5-haiku](/anthropic/claude-3.5-haiku)\n- [anthropic/claude-3-opus](/anthropic/claude-3-opus)\n- [anthropic/claude-2.1](/anthropic/claude-2.1)\n- [google/gemini-pro-1.5](/google/gemini-pro-1.5)\n- [google/gemini-flash-1.5](/google/gemini-flash-1.5)\n- [mistralai/mistral-large-2407](/mistralai/mistral-large-2407)\n- [mistralai/mistral-nemo](/mistralai/mistral-nemo)\n- [meta-llama/llama-3.1-70b-instruct](/meta-llama/llama-3.1-70b-instruct)\n- [meta-llama/llama-3.1-405b-instruct](/meta-llama/llama-3.1-405b-instruct)\n- [mistralai/mixtral-8x22b-instruct](/mistralai/mixtral-8x22b-instruct)\n- [cohere/command-r-plus](/cohere/command-r-plus)\n- [cohere/command-r](/cohere/command-r)","context_length":2000000,"architecture":{"modality":"text->text","tokenizer":"Router","instruct_type":null},"pricing":{"prompt":"-1","completion":"-1","request":"-1","image":"-1"},"top_provider":{"context_length":null,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"openai/gpt-3.5-turbo-1106","name":"OpenAI: GPT-3.5 Turbo 16k (older v1106)","created":1699228800,"description":"An older GPT-3.5 Turbo model with improved instruction following, JSON mode, reproducible outputs, parallel function calling, and more. Training data: up to Sep 2021.","context_length":16385,"architecture":{"modality":"text->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.000001","completion":"0.000002","image":"0","request":"0"},"top_provider":{"context_length":16385,"max_completion_tokens":4096,"is_moderated":true},"per_request_limits":null},{"id":"openai/gpt-4-1106-preview","name":"OpenAI: GPT-4 Turbo (older v1106)","created":1699228800,"description":"The latest GPT-4 Turbo model with vision capabilities. Vision requests can now use JSON mode and function calling.\n\nTraining data: up to April 2023.","context_length":128000,"architecture":{"modality":"text->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.00001","completion":"0.00003","image":"0","request":"0"},"top_provider":{"context_length":128000,"max_completion_tokens":4096,"is_moderated":true},"per_request_limits":null},{"id":"google/palm-2-chat-bison-32k","name":"Google: PaLM 2 Chat 32k","created":1698969600,"description":"PaLM 2 is a language model by Google with improved multilingual, reasoning and coding capabilities.","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"PaLM","instruct_type":null},"pricing":{"prompt":"0.000001","completion":"0.000002","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":8192,"is_moderated":false},"per_request_limits":null},{"id":"google/palm-2-codechat-bison-32k","name":"Google: PaLM 2 Code Chat 32k","created":1698969600,"description":"PaLM 2 fine-tuned for chatbot conversations that help with code-related questions.","context_length":32768,"architecture":{"modality":"text->text","tokenizer":"PaLM","instruct_type":null},"pricing":{"prompt":"0.000001","completion":"0.000002","image":"0","request":"0"},"top_provider":{"context_length":32768,"max_completion_tokens":8192,"is_moderated":false},"per_request_limits":null},{"id":"jondurbin/airoboros-l2-70b","name":"Airoboros 70B","created":1698537600,"description":"A Llama 2 70B fine-tune using synthetic data (the Airoboros dataset).\n\nCurrently based on [jondurbin/airoboros-l2-70b](https://huggingface.co/jondurbin/airoboros-l2-70b-2.2.1), but might get updated in the future.","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Llama2","instruct_type":"airoboros"},"pricing":{"prompt":"0.0000005","completion":"0.0000005","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"xwin-lm/xwin-lm-70b","name":"Xwin 70B","created":1697328000,"description":"Xwin-LM aims to develop and open-source alignment tech for LLMs. Our first release, built-upon on the [Llama2](/models/${Model.Llama_2_13B_Chat}) base models, ranked TOP-1 on AlpacaEval. Notably, it's the first to surpass [GPT-4](/models/${Model.GPT_4}) on this benchmark. The project will be continuously updated.","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Llama2","instruct_type":"airoboros"},"pricing":{"prompt":"0.00000375","completion":"0.00000375","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":512,"is_moderated":false},"per_request_limits":null},{"id":"openai/gpt-3.5-turbo-instruct","name":"OpenAI: GPT-3.5 Turbo Instruct","created":1695859200,"description":"This model is a variant of GPT-3.5 Turbo tuned for instructional prompts and omitting chat-related optimizations. Training data: up to Sep 2021.","context_length":4095,"architecture":{"modality":"text->text","tokenizer":"GPT","instruct_type":"chatml"},"pricing":{"prompt":"0.0000015","completion":"0.000002","image":"0","request":"0"},"top_provider":{"context_length":4095,"max_completion_tokens":4096,"is_moderated":true},"per_request_limits":null},{"id":"mistralai/mistral-7b-instruct-v0.1","name":"Mistral: Mistral 7B Instruct v0.1","created":1695859200,"description":"A 7.3B parameter model that outperforms Llama 2 13B on all benchmarks, with optimizations for speed and context length.","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"mistral"},"pricing":{"prompt":"0.00000018","completion":"0.00000018","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"pygmalionai/mythalion-13b","name":"Pygmalion: Mythalion 13B","created":1693612800,"description":"A blend of the new Pygmalion-13b and MythoMax. #merge","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Llama2","instruct_type":"alpaca"},"pricing":{"prompt":"0.0000008","completion":"0.0000012","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"openai/gpt-3.5-turbo-16k","name":"OpenAI: GPT-3.5 Turbo 16k","created":1693180800,"description":"This model offers four times the context length of gpt-3.5-turbo, allowing it to support approximately 20 pages of text in a single request at a higher cost. Training data: up to Sep 2021.","context_length":16385,"architecture":{"modality":"text->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.000003","completion":"0.000004","image":"0","request":"0"},"top_provider":{"context_length":16385,"max_completion_tokens":4096,"is_moderated":true},"per_request_limits":null},{"id":"openai/gpt-4-32k","name":"OpenAI: GPT-4 32k","created":1693180800,"description":"GPT-4-32k is an extended version of GPT-4, with the same capabilities but quadrupled context length, allowing for processing up to 40 pages of text in a single pass. This is particularly beneficial for handling longer content like interacting with PDFs without an external vector database. Training data: up to Sep 2021.","context_length":32767,"architecture":{"modality":"text->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.00006","completion":"0.00012","image":"0","request":"0"},"top_provider":{"context_length":32767,"max_completion_tokens":4096,"is_moderated":true},"per_request_limits":null},{"id":"openai/gpt-4-32k-0314","name":"OpenAI: GPT-4 32k (older v0314)","created":1693180800,"description":"GPT-4-32k is an extended version of GPT-4, with the same capabilities but quadrupled context length, allowing for processing up to 40 pages of text in a single pass. This is particularly beneficial for handling longer content like interacting with PDFs without an external vector database. Training data: up to Sep 2021.","context_length":32767,"architecture":{"modality":"text->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.00006","completion":"0.00012","image":"0","request":"0"},"top_provider":{"context_length":32767,"max_completion_tokens":4096,"is_moderated":true},"per_request_limits":null},{"id":"nousresearch/nous-hermes-llama2-13b","name":"Nous: Hermes 13B","created":1692489600,"description":"A state-of-the-art language model fine-tuned on over 300k instructions by Nous Research, with Teknium and Emozilla leading the fine tuning process.","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Llama2","instruct_type":"alpaca"},"pricing":{"prompt":"0.00000017","completion":"0.00000017","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"mancer/weaver","name":"Mancer: Weaver (alpha)","created":1690934400,"description":"An attempt to recreate Claude-style verbosity, but don't expect the same level of coherence or memory. Meant for use in roleplay/narrative situations.","context_length":8000,"architecture":{"modality":"text->text","tokenizer":"Llama2","instruct_type":"alpaca"},"pricing":{"prompt":"0.0000015","completion":"0.00000225","image":"0","request":"0"},"top_provider":{"context_length":8000,"max_completion_tokens":1000,"is_moderated":false},"per_request_limits":null},{"id":"huggingfaceh4/zephyr-7b-beta:free","name":"Hugging Face: Zephyr 7B (free)","created":1690934400,"description":"Zephyr is a series of language models that are trained to act as helpful assistants. Zephyr-7B-β is the second model in the series, and is a fine-tuned version of [mistralai/Mistral-7B-v0.1](/models/mistralai/mistral-7b-instruct-v0.1) that was trained on a mix of publicly available, synthetic datasets using Direct Preference Optimization (DPO).","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Mistral","instruct_type":"zephyr"},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":2048,"is_moderated":false},"per_request_limits":null},{"id":"anthropic/claude-2.0:beta","name":"Anthropic: Claude v2.0 (self-moderated)","created":1690502400,"description":"Anthropic's flagship model. Superior performance on tasks that require complex reasoning. Supports hundreds of pages of text.","context_length":100000,"architecture":{"modality":"text->text","tokenizer":"Claude","instruct_type":null},"pricing":{"prompt":"0.000008","completion":"0.000024","image":"0","request":"0"},"top_provider":{"context_length":100000,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"anthropic/claude-2.0","name":"Anthropic: Claude v2.0","created":1690502400,"description":"Anthropic's flagship model. Superior performance on tasks that require complex reasoning. Supports hundreds of pages of text.","context_length":100000,"architecture":{"modality":"text->text","tokenizer":"Claude","instruct_type":null},"pricing":{"prompt":"0.000008","completion":"0.000024","image":"0","request":"0"},"top_provider":{"context_length":100000,"max_completion_tokens":4096,"is_moderated":true},"per_request_limits":null},{"id":"undi95/remm-slerp-l2-13b","name":"ReMM SLERP 13B","created":1689984000,"description":"A recreation trial of the original MythoMax-L2-B13 but with updated models. #merge","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Llama2","instruct_type":"alpaca"},"pricing":{"prompt":"0.0000008","completion":"0.0000012","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"undi95/remm-slerp-l2-13b:extended","name":"ReMM SLERP 13B (extended)","created":1689984000,"description":"A recreation trial of the original MythoMax-L2-B13 but with updated models. #merge","context_length":6144,"architecture":{"modality":"text->text","tokenizer":"Llama2","instruct_type":"alpaca"},"pricing":{"prompt":"0.000001125","completion":"0.000001125","image":"0","request":"0"},"top_provider":{"context_length":6144,"max_completion_tokens":512,"is_moderated":false},"per_request_limits":null},{"id":"google/palm-2-chat-bison","name":"Google: PaLM 2 Chat","created":1689811200,"description":"PaLM 2 is a language model by Google with improved multilingual, reasoning and coding capabilities.","context_length":9216,"architecture":{"modality":"text->text","tokenizer":"PaLM","instruct_type":null},"pricing":{"prompt":"0.000001","completion":"0.000002","image":"0","request":"0"},"top_provider":{"context_length":9216,"max_completion_tokens":1024,"is_moderated":false},"per_request_limits":null},{"id":"google/palm-2-codechat-bison","name":"Google: PaLM 2 Code Chat","created":1689811200,"description":"PaLM 2 fine-tuned for chatbot conversations that help with code-related questions.","context_length":7168,"architecture":{"modality":"text->text","tokenizer":"PaLM","instruct_type":null},"pricing":{"prompt":"0.000001","completion":"0.000002","image":"0","request":"0"},"top_provider":{"context_length":7168,"max_completion_tokens":1024,"is_moderated":false},"per_request_limits":null},{"id":"gryphe/mythomax-l2-13b:free","name":"MythoMax 13B (free)","created":1688256000,"description":"One of the highest performing and most popular fine-tunes of Llama 2 13B, with rich descriptions and roleplay. #merge","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Llama2","instruct_type":"alpaca"},"pricing":{"prompt":"0","completion":"0","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":2048,"is_moderated":false},"per_request_limits":null},{"id":"gryphe/mythomax-l2-13b","name":"MythoMax 13B","created":1688256000,"description":"One of the highest performing and most popular fine-tunes of Llama 2 13B, with rich descriptions and roleplay. #merge","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Llama2","instruct_type":"alpaca"},"pricing":{"prompt":"0.000000065","completion":"0.000000065","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":4096,"is_moderated":false},"per_request_limits":null},{"id":"gryphe/mythomax-l2-13b:nitro","name":"MythoMax 13B (nitro)","created":1688256000,"description":"One of the highest performing and most popular fine-tunes of Llama 2 13B, with rich descriptions and roleplay. #merge","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Llama2","instruct_type":"alpaca"},"pricing":{"prompt":"0.0000002","completion":"0.0000002","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"gryphe/mythomax-l2-13b:extended","name":"MythoMax 13B (extended)","created":1688256000,"description":"One of the highest performing and most popular fine-tunes of Llama 2 13B, with rich descriptions and roleplay. #merge","context_length":8192,"architecture":{"modality":"text->text","tokenizer":"Llama2","instruct_type":"alpaca"},"pricing":{"prompt":"0.000001125","completion":"0.000001125","image":"0","request":"0"},"top_provider":{"context_length":8192,"max_completion_tokens":512,"is_moderated":false},"per_request_limits":null},{"id":"meta-llama/llama-2-13b-chat","name":"Meta: Llama 2 13B Chat","created":1687219200,"description":"A 13 billion parameter language model from Meta, fine tuned for chat completions","context_length":4096,"architecture":{"modality":"text->text","tokenizer":"Llama2","instruct_type":"llama2"},"pricing":{"prompt":"0.000000198","completion":"0.000000198","image":"0","request":"0"},"top_provider":{"context_length":4096,"max_completion_tokens":null,"is_moderated":false},"per_request_limits":null},{"id":"openai/gpt-3.5-turbo","name":"OpenAI: GPT-3.5 Turbo","created":1685232000,"description":"GPT-3.5 Turbo is OpenAI's fastest model. It can understand and generate natural language or code, and is optimized for chat and traditional completion tasks.\n\nTraining data up to Sep 2021.","context_length":16385,"architecture":{"modality":"text->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.0000005","completion":"0.0000015","image":"0","request":"0"},"top_provider":{"context_length":16385,"max_completion_tokens":4096,"is_moderated":true},"per_request_limits":null},{"id":"openai/gpt-3.5-turbo-0125","name":"OpenAI: GPT-3.5 Turbo 16k","created":1685232000,"description":"The latest GPT-3.5 Turbo model with improved instruction following, JSON mode, reproducible outputs, parallel function calling, and more. Training data: up to Sep 2021.\n\nThis version has a higher accuracy at responding in requested formats and a fix for a bug which caused a text encoding issue for non-English language function calls.","context_length":16385,"architecture":{"modality":"text->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.0000005","completion":"0.0000015","image":"0","request":"0"},"top_provider":{"context_length":16385,"max_completion_tokens":4096,"is_moderated":true},"per_request_limits":null},{"id":"openai/gpt-4","name":"OpenAI: GPT-4","created":1685232000,"description":"OpenAI's flagship model, GPT-4 is a large-scale multimodal language model capable of solving difficult problems with greater accuracy than previous models due to its broader general knowledge and advanced reasoning capabilities. Training data: up to Sep 2021.","context_length":8191,"architecture":{"modality":"text->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.00003","completion":"0.00006","image":"0","request":"0"},"top_provider":{"context_length":8191,"max_completion_tokens":4096,"is_moderated":true},"per_request_limits":null},{"id":"openai/gpt-4-0314","name":"OpenAI: GPT-4 (older v0314)","created":1685232000,"description":"GPT-4-0314 is the first version of GPT-4 released, with a context length of 8,192 tokens, and was supported until June 14. Training data: up to Sep 2021.","context_length":8191,"architecture":{"modality":"text->text","tokenizer":"GPT","instruct_type":null},"pricing":{"prompt":"0.00003","completion":"0.00006","image":"0","request":"0"},"top_provider":{"context_length":8191,"max_completion_tokens":4096,"is_moderated":true},"per_request_limits":null}]}
================================================
FILE: packages/backend/resources/openapi/stabilityai.json
================================================
{
"openapi": "3.0.3",
"info": {
"version": "v2beta",
"title": "StabilityAI REST API",
"description": "Welcome to the Stability Platform API. As of March 2024, we are building the REST v2beta API service to be the primary API service for the Stability Platform. \nAll AI services on other APIs (gRPC, REST v1, RESTv2alpha) will continue to be maintained, however they will not receive\nnew features or parameters.\n\nIf you are a REST v2alpha user, we strongly recommend that you adjust the URL calls for the specific services that you are using over to the equivalent REST v2beta URL. Normally, this means simply replacing \"v2alpha\" with \"v2beta\". We are not deprecating v2alpha URLs at this time for users that are currently using them.\n\n#### Authentication\n\nYou will need your [Stability API key](https://platform.stability.ai/account/keys) in order to make requests to this API.\nMake sure you never share your API key with anyone, and you never commit it to a public repository. Include this key in \nthe `Authorization` header of your requests.\n\n#### Rate limiting\n\nThis API is rate-limited to 150 requests every 10 seconds. If you exceed this limit, you will receive a `429` response\nand be timed out for 60 seconds. If you find this limit too restrictive, please reach out to us via [this form](https://stabilityplatform.freshdesk.com/support/home).\n\n#### Support\n\nPlease see our [FAQ](https://platform.stability.ai/faq) for answers to common questions. If you have any other questions or concerns,\nplease reach out to us via [this form](https://stabilityplatform.freshdesk.com/support/tickets/new).\n\nTo see the health of our APIs, please check our [Status Page](https://stabilityai.instatus.com/)."
},
"servers": [
{
"url": "https://api.stability.ai"
}
],
"security": [
{
"STABILITY_API_KEY": []
}
],
"tags": [
{
"name": "Edit",
"description": "Tools for editing your own and generated images.\n\n**[Erase](/docs/api-reference#tag/Edit/paths/~1v2beta~1stable-image~1edit~1erase/post)**\n\nThe Erase service removes unwanted objects, such as blemishes on portraits or items on desks, using image masks.\n\n**[Outpaint](/docs/api-reference#tag/Edit/paths/~1v2beta~1stable-image~1edit~1outpaint/post)**\n\nThe outpaint service inserts additional content in an image to fill in the space in any direction, allowing you to \"zoom-out\" of an image.\n\n**[Inpaint](/docs/api-reference#tag/Edit/paths/~1v2beta~1stable-image~1edit~1inpaint/post)**\n\nThe Inpaint service modifies images by filling in or replacing specified areas with new content based on the content of a \"mask\" image.\n\n**[Search and Replace](/docs/api-reference#tag/Edit/paths/~1v2beta~1stable-image~1edit~1search-and-replace/post)**\n\nThe Search and Replace service, similar to inpaint, allows to replace specified areas with new content, but this time with the help of a prompt instead of a mask. The service will automatically segment the object and replace it with the object requested in the prompt.\n\n**[Search and Recolor](/docs/api-reference#tag/Edit/paths/~1v2beta~1stable-image~1edit~1search-and-recolor/post)**\n\nThe Search and Recolor service is another derivative of the inpaint service and provides the ability to change the color of a specific object in an image using a prompt. The Search and Recolor service will automatically segment the object and recolor it using the colors requested in the prompt.\n\n**[Remove Background](/docs/api-reference#tag/Edit/paths/~1v2beta~1stable-image~1edit~1remove-background/post)**\n\nThe Remove Background service accurately segments the foreground from an image to removes the background."
},
{
"name": "Upscale",
"description": "Tools for increasing the size and resolution of your existing images.\n\n**[Fast Upscaler](/docs/api-reference#tag/Upscale/paths/~1v2beta~1stable-image~1upscale~1fast/post)**\n\nThis service enhances image resolution by 4x using predictive and generative AI. This lightweight and fast service (processing in ~1 second) is ideal for enhancing the quality of compressed images, making it suitable for social media posts and other applications.\n\n**[Conservative Upscaler](/docs/api-reference#tag/Upscale/paths/~1v2beta~1stable-image~1upscale~1conservative/post)**\n\nThis service can upscale images by 20 to 40 times up to a 4 megapixel output image with minimal alteration to the original image. The Conservative Upscaler can upscale images as small as 64x64 pixels directly to a 4 megapixel output. Use this option if you directly need a 4 megapixel output.\n\n**[Creative Upscaler](/docs/api-reference#tag/Upscale/paths/~1v2beta~1stable-image~1upscale~1creative/post)**\n\nThe service can upscale highly degraded images (lower than 1 megapixel) with a creative twist to provide high resolution results."
},
{
"name": "Generate",
"description": "Tools to generate new images from text, or create variations of existing images. Our different services include:\n\n**[Stable Image Ultra](/docs/api-reference#tag/Generate/paths/~1v2beta~1stable-image~1generate~1ultra/post)**: Photorealistic, Large-Scale Output\n\nOur state of the art text to image model based on Stable Diffusion 3.5. Stable Image Ultra Produces the highest quality, photorealistic outputs perfect for professional print media and large format applications. Stable Image Ultra excels at rendering exceptional detail and realism.\n\n**[Stable Image Core](/docs/api-reference#tag/Generate/paths/~1v2beta~1stable-image~1generate~1core/post)**: Fast and Affordable\n\nOptimized for fast and affordable image generation, great for rapidly iterating on concepts during ideation. Stable Image Core is the next generation model following Stable Diffusion XL.\n\n**[Stable Diffusion 3 & 3.5 Model Suite](/docs/api-reference#tag/Generate/paths/~1v2beta~1stable-image~1generate~1sd3/post)**: Stability AI's latest base models\n\nThe different versions of our open models are available via API, letting you test and adjust speed and quality based on your use case. All model versions strike a balance between generation speed and output quality and are ideal for creating high-volume, high-quality digital assets like websites, newsletters, and marketing materials."
},
{
"name": "Control",
"description": "Tools for generating precise, controlled variations of existing images or sketches.\n\n**[Sketch](/docs/api-reference#tag/Control/paths/~1v2beta~1stable-image~1control~1sketch/post)**\n\nThis service upgrades sketches to refined outputs with precise control. For non-sketch images, it allows detailed manipulation of the final appearance by leveraging the contour lines and edges within the image. \n\n**[Structure](/docs/api-reference#tag/Control/paths/~1v2beta~1stable-image~1control~1structure/post)**\n\nThis service excels in generating images by maintaining the structure of an input image, making it especially valuable for advanced content creation scenarios such as recreating scenes or rendering characters from models.\n\n**[Style](/docs/api-reference#tag/Control/paths/~1v2beta~1stable-image~1control~1style/post)**\n\nThis service extracts stylistic elements from an input image (control image) and uses it to guide the creation of an output image based on the prompt. The result is a new image in the same style as the control image."
},
{
"name": "Results",
"description": "Tools for fetching the results of your async generations."
},
{
"name": "User",
"description": "Manage your Stability account, and view account/organization balances."
},
{
"name": "Engines",
"description": "Enumerate engines that work with 'Version 1' REST API endpoints."
},
{
"name": "SDXL 1.0 & SD1.6",
"description": "Generate images using SDXL 1.0 or SD1.6."
}
],
"paths": {
"/v2alpha/generation/image-to-video": {
"post": {
"tags": ["v2alpha/generation"],
"summary": "image-to-video",
"description": "Generate a short video based on an initial image with [Stable Video Diffusion](https://static1.squarespace.com/static/6213c340453c3f502425776e/t/655ce779b9d47d342a93c890/1700587395994/stable_video_diffusion.pdf),\na latent video diffusion model. \n\n\n\n### How to generate a video\nVideo generations are asynchronous, so after starting a generation use the `id` returned in the response to poll [/v2alpha/generation/image-to-video/result/{id}](#tag/v2alphageneration/paths/~1v2alpha~1generation~1image-to-video~1result~1%7Bid%7D/get) for results.\n\n### Price\nFlat rate of 20 cents per generation.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2alpha/generation/image-to-video\",\n headers={\"authorization\": f\"Bearer sk-MYAPIKEY\"},\n files={\"image\": open(\"./kittens-in-space.png\", \"rb\")},\n data={\n \"seed\": 0,\n \"cfg_scale\": 1.8,\n \"motion_bucket_id\": 127\n },\n)\n\nprint(\"Generation ID:\", response.json().get('id'))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import fs from \"node:fs\";\nimport axios from \"axios\";\nimport FormData from \"form-data\";\n\nconst data = new FormData();\ndata.append(\"image\", fs.readFileSync(\"./image.png\"), \"image.png\");\ndata.append(\"seed\", 0);\ndata.append(\"cfg_scale\", 1.8);\ndata.append(\"motion_bucket_id\", 127);\n\nconst response = await axios.request({\n url: `https://api.stability.ai/v2alpha/generation/image-to-video`,\n method: \"post\",\n validateStatus: undefined,\n headers: {\n authorization: `Bearer sk-MYAPIKEY`,\n ...data.getHeaders(),\n },\n data: data,\n});\n\nconsole.log(\"Generation ID:\", response.data.id);"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2alpha/generation/image-to-video\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -F image=@\"./image.png\" \\\n -F seed=0 \\\n -F cfg_scale=1.8 \\\n -F motion_bucket_id=127 \\\n -o \"./output.json\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"$ref": "#/components/schemas/ImageToVideoRequest"
}
}
}
},
"responses": {
"200": {
"description": "Video generation started. Poll for results using the `id` in the response [here](#tag/v2alphageneration/paths/~1v2alpha~1generation~1image-to-video~1result~1%7Bid%7D/get).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"$ref": "#/components/schemas/GenerationID"
}
},
"required": ["id"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2alpha/generation/image-to-video/result/{id}": {
"get": {
"tags": ["v2alpha/generation"],
"summary": "image-to-video/result",
"description": "Fetch the result of an image-to-video generation by ID. Make sure you use the same API key that you used to\ngenerate the video, otherwise you will receive a `404` response.\n\n### How is progress reported?\nYour generation is either `in-progress` (i.e. status code `202`) or it is complete (i.e. status code `200`). \nWe may add more fine-grained progress reporting in the future (e.g. a numerical progress).\n\n### How long are results stored?\nResults are stored for 24 hours after generation. After that, the results are deleted and you will need to \nre-generate your video.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\ngeneration_id = \"e52772ac75b...\"\n\nresponse = requests.request(\n \"GET\",\n f\"https://api.stability.ai/v2alpha/generation/image-to-video/result/{generation_id}\",\n headers={\n 'Accept': \"video/*\", # Use 'application/json' to receive base64 encoded JSON\n 'authorization': f\"Bearer sk-MYAPIKEY\"\n },\n)\n\nif response.status_code == 202:\n print(\"Generation in-progress, try again in 10 seconds.\")\nelif response.status_code == 200:\n print(\"Generation complete!\")\n with open(\"video.mp4\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import axios from \"axios\";\nimport fs from \"node:fs\";\n\nconst generationID = \"e52772ac75b...\";\n\nconst response = await axios.request({\n url: `https://api.stability.ai/v2alpha/generation/image-to-video/result/${generationID}`,\n method: \"GET\",\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: {\n accept: \"video/*\", // Use 'application/json' to receive base64 encoded JSON\n authorization: `Bearer sk-MYAPIKEY`,\n },\n});\n\nif (response.status === 202) {\n console.log(\"Generation is still running, try again in 10 seconds.\");\n} else if (response.status === 200) {\n console.log(\"Generation is complete!\");\n fs.writeFileSync(\"video.mp4\", Buffer.from(response.data));\n} else {\n throw new Error(`Response ${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "generation_id=\"e52772ac75b...\"\nurl=\"https://api.stability.ai/v2alpha/generation/image-to-video/result/$generation_id\"\nhttp_status=$(curl -sS -f -o \"./output.mp4\" -w '%{http_code}' -H \"authorization: sk-MYAPIKEY\" -H 'accept: video/*' \"$url\")\n\ncase $http_status in\n 202)\n echo \"Still processing. Retrying in 10 seconds...\"\n ;;\n 200)\n echo \"Download complete!\"\n ;;\n 4*|5*)\n mv \"./output.mp4\" \"./error.json\"\n echo \"Error: Check ./error.json for details.\"\n exit 1\n ;;\nesac"
}
],
"parameters": [
{
"schema": {
"$ref": "#/components/schemas/GenerationID"
},
"required": true,
"name": "id",
"in": "path"
},
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"default": "video/*",
"description": "Specify `video/*` to get the video bytes directly. Otherwise specify `application/json` to receive the video as base64 encoded JSON.",
"enum": ["video/*", "application/json"]
},
"required": false,
"name": "accept",
"in": "header"
}
],
"responses": {
"200": {
"description": "The result of your video generation.",
"headers": {
"x-request-id": {
"description": "A unique identifier for this request.",
"schema": {
"type": "string"
}
},
"content-type": {
"description": "The format of the generated video.\n\n To receive the bytes of the video directly, specify `video/*` in the accept header. To receive the bytes base64 encoded inside of a JSON payload, specify `application/json`.",
"examples": {
"mp4": {
"description": "raw bytes",
"value": "video/mp4"
},
"mp4JSON": {
"description": "base64 encoded",
"value": "application/json; type=video/mp4"
}
},
"schema": {
"type": "string"
}
},
"finish-reason": {
"schema": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"]
},
"description": "Indicates the reason the generation finished. \n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and one or more frames have been blurred as a result.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `finish_reason`."
},
"seed": {
"description": "The seed used as random noise for this generation.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `seed`.",
"example": "343940597",
"schema": {
"type": "string"
}
}
},
"content": {
"video/mp4": {
"schema": {
"type": "string",
"description": "The bytes of the generated video.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated mp4.\n(Caution: may contain cats)"
},
"application/json; type=video/mp4": {
"schema": {
"type": "object",
"properties": {
"video": {
"type": "string",
"description": "The generated video, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and one or more frames have been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["video", "finish_reason"]
}
}
}
},
"202": {
"description": "Your image-to-video generation is still in-progress.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"$ref": "#/components/schemas/GenerationID"
},
"status": {
"type": "string",
"enum": ["in-progress"],
"description": "The status of your generation."
}
},
"required": ["id", "status"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"404": {
"description": "id: the generation either does not exist or has expired.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2bca35116bc5431d6dc4b4ea2ef3da2f",
"name": "generation_not_found",
"errors": [
"id: the generation either does not exist or has expired."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2alpha/generation/stable-image/upscale": {
"post": {
"tags": ["v2alpha/generation"],
"summary": "stable-image/upscale",
"description": "Takes images between 64x64 and 1 megapixel and upscales them all the way to **4K** resolution. Put more \ngenerally, it can upscale images ~20-40x times while preserving, and often enhancing, quality.\n\n### How to use\n - Invoke this endpoint with the required parameters to start a generation\n - Use that `id` in the response to poll for results at the [upscale/result/{id}](#tag/v2alphageneration/paths/~1v2alpha~1generation~1stable-image~1upscale~1result~1%7Bid%7D/get) endpoint\n - Rate-limiting or other errors may occur if you poll more than once every 10 seconds\n \n### Price\nFlat rate of 25 cents per generation.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2alpha/generation/stable-image/upscale\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\"\n },\n files={\n \"image\": open(\"./kitten-in-space.png\", \"rb\")\n },\n data={\n \"prompt\": \"cute fluffy white kitten floating in space, pastel colors\",\n \"output_format\": \"webp\",\n },\n)\n\nprint(\"Generation ID:\", response.json().get('id'))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import fs from \"node:fs\";\nimport axios from \"axios\";\nimport FormData from \"form-data\";\n\nconst formData = {\n image: fs.createReadStream(\"./kitten-in-space.png\"),\n prompt: \"cute fluffy white kitten floating in space, pastel colors\",\n output_format: \"webp\"\n};\n\nconst response = await axios.postForm(\n `https://api.stability.ai/v2alpha/generation/stable-image/upscale`,\n axios.toFormData(formData, new FormData()),\n {\n validateStatus: undefined,\n headers: { Authorization: `Bearer sk-MYAPIKEY` },\n },\n);\n\nconsole.log(\"Generation ID:\", response.data.id);"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2alpha/generation/stable-image/upscale\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -F image=@\"./kitten-in-rainforest.png\" \\\n -F prompt=\"cute fluffy white kitten sitting in a rainforest, pastel colors\" \\\n -F output_format=webp \\\n -o \"./output.json\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The image you wish to upscale.\n\nSupported Formats:\n- jpeg\n- png\n- webp\n\nValidation Rules:\n- Every side must be at least 64 pixels\n- Total pixel count must be between 4,096 and 1,048,576 pixels",
"format": "binary",
"example": "./some/image.png"
},
"prompt": {
"type": "string",
"minLength": 1,
"maxLength": 10000,
"description": "What you wish to see in the output image. A strong, descriptive prompt that clearly defines \nelements, colors, and subjects will lead to better results. \n\nTo control the weight of a given word use the format `(word:weight)`, \nwhere `word` is the word you'd like to control the weight of and `weight` \nis a value between 0 and 1. For example: `The sky was a crisp (blue:0.3) and (green:0.8)`\nwould convey a sky that was blue and green, but more green than blue."
},
"negative_prompt": {
"type": "string",
"maxLength": 10000,
"description": "A blurb of text describing what you **do not** wish to see in the output image. \nThis is an advanced feature."
},
"output_format": {
"type": "string",
"enum": ["jpeg", "png", "webp"],
"default": "png",
"description": "Dictates the `content-type` of the generated image."
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "A specific value that is used to guide the 'randomness' of the generation. (Omit this parameter or pass `0` to use a random seed.)"
},
"creativity": {
"type": "number",
"minimum": 0,
"maximum": 0.35,
"default": 0.3,
"description": "Indicates how creative the model should be when upscaling an image.\nHigher values will result in more details being added to the image during upscaling."
}
},
"required": ["image", "prompt"]
}
}
}
},
"responses": {
"200": {
"description": "Upscaling was successful!",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"$ref": "#/components/schemas/GenerationID"
}
},
"required": ["id"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2alpha/generation/stable-image/upscale/result/{id}": {
"get": {
"tags": ["v2alpha/generation"],
"summary": "stable-image/upscale/result",
"description": "Fetch the result of an upscale generation by ID. Make sure to use the same API key to fetch the generation result\nthat you used to create the generation, otherwise you will receive a `404` response.\n\n### How is progress reported?\nYour generation is either `in-progress` (i.e. status code `202`) or it is complete (i.e. status code `200`). \nWe may add more fine-grained progress reporting in the future (e.g. a numerical progress).\n\n### How long are results stored?\nResults are stored for 24 hours after generation. After that, the results are deleted.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\ngeneration_id = \"e52772ac75b...\"\n\nresponse = requests.request(\n \"GET\",\n f\"https://api.stability.ai/v2alpha/generation/stable-image/upscale/result/{generation_id}\",\n headers={\n 'Accept': \"image/*\", # Use 'application/json' to receive base64 encoded JSON\n 'authorization': f\"Bearer sk-MYAPIKEY\"\n },\n)\n\nif response.status_code == 202:\n print(\"Generation in-progress, try again in 10 seconds.\")\nelif response.status_code == 200:\n print(\"Generation complete!\")\n with open(\"upscaled.webp\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import axios from \"axios\";\nimport fs from \"node:fs\";\n\nconst generationID = \"e52772ac75b...\";\n\nconst response = await axios.request({\n url: `https://api.stability.ai/v2alpha/generation/stable-image/upscale/result/${generationID}`,\n method: \"GET\",\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: {\n accept: \"image/*\", // Use 'application/json' to receive base64 encoded JSON\n authorization: `Bearer sk-MYAPIKEY`,\n },\n});\n\nif (response.status === 202) {\n console.log(\"Generation is still running, try again in 10 seconds.\");\n} else if (response.status === 200) {\n console.log(\"Generation is complete!\");\n fs.writeFileSync(\"upscaled.webp\", Buffer.from(response.data));\n} else {\n throw new Error(`Response ${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "generation_id=\"e52772ac75b...\"\nurl=\"https://api.stability.ai/v2alpha/generation/stable-image/upscale/result/$generation_id\"\nhttp_status=$(curl -sS -f -o \"./upscaled.webp\" -w '%{http_code}' -H \"authorization: sk-MYAPIKEY\" -H 'accept: image/*' \"$url\")\n\ncase $http_status in\n 202)\n echo \"Still processing. Retrying in 10 seconds...\"\n ;;\n 200)\n echo \"Download complete!\"\n ;;\n 4*|5*)\n mv \"./upscaled.webp\" \"./error.json\"\n echo \"Error: Check ./error.json for details.\"\n exit 1\n ;;\nesac"
}
],
"parameters": [
{
"schema": {
"$ref": "#/components/schemas/GenerationID"
},
"required": true,
"name": "id",
"in": "path"
},
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"default": "image/*",
"description": "Specify `image/*` to get the image bytes directly. Otherwise specify `application/json` to receive the image as base64 encoded JSON.",
"enum": ["image/*", "application/json"]
},
"required": false,
"name": "accept",
"in": "header"
}
],
"responses": {
"200": {
"description": "The upscale was successful.",
"headers": {
"x-request-id": {
"description": "A unique identifier for this request.",
"schema": {
"type": "string"
}
},
"content-type": {
"description": "The format of the generated image.\n\n To receive the bytes of the image directly, specify `image/*` in the accept header. To receive the bytes base64 encoded inside of a JSON payload, specify `application/json`.",
"examples": {
"jpeg": {
"description": "raw bytes",
"value": "image/jpeg"
},
"jpegJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/jpeg"
},
"png": {
"description": "raw bytes",
"value": "image/png"
},
"pngJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/png"
},
"webp": {
"description": "raw bytes",
"value": "image/webp"
},
"webpJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/webp"
}
},
"schema": {
"type": "string"
}
},
"finish-reason": {
"schema": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"]
},
"description": "Indicates the reason the generation finished. \n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `finish_reason`."
},
"seed": {
"description": "The seed used as random noise for this generation.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `seed`.",
"example": "343940597",
"schema": {
"type": "string"
}
}
},
"content": {
"image/jpeg": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated jpeg.\n(Caution: may contain cats)"
},
"application/json; type=image/jpeg": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/png": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated png.\n(Caution: may contain cats)"
},
"application/json; type=image/png": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/webp": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated webp.\n(Caution: may contain cats)"
},
"application/json; type=image/webp": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
}
}
},
"202": {
"description": "Your upscale generation is still in-progress.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"$ref": "#/components/schemas/GenerationID"
},
"status": {
"type": "string",
"enum": ["in-progress"],
"description": "The status of your generation."
}
},
"required": ["id", "status"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"404": {
"description": "id: the generation either does not exist or has expired.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2bca35116bc5431d6dc4b4ea2ef3da2f",
"name": "generation_not_found",
"errors": [
"id: the generation either does not exist or has expired."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2alpha/generation/stable-image/inpaint": {
"post": {
"tags": ["v2alpha/generation"],
"summary": "stable-image/inpaint",
"description": "Inpaint an existing image, with or without a mask, using our latest-and-greatest inpainting model.\n\n### Search-and-Replace Mode\nThis mode is ideal for individuals of all levels of skill in design. It can be used for straightforward \nadjustments to images. The service will automatically mask the most appropriate object based on the contents\nof the `search_prompt`, and replace it with a generated result based on the `prompt`.\n\n**How to use:** set the `mode` parameter to `search` and provide a short description of what to \nsearch-and-replace in the `search_prompt` parameter.\n\n### Mask Mode\nThis mode allows for precise control of generative fill tasks on an image, down to the level of \nindividual pixels. Design professionals can provide a `mask` for the section of the image to be replaced, \nand use standard image prompting to describe the full image as it should appear after the editing. \nThe resulting image will incorporate all of the elements described in the `prompt`.\n\n**How to use:** set the `mode` parameter to `mask` and either pass in an `image` with an alpha channel \nor provide an explicit mask image to the `mask` parameter. If both are present the `mask` parameter will\ntake precedence.\n\n### Price\n- Requests with `mode` set to `search` cost 4 cents.\n- Requests with `mode` set to `mask` cost 3 cents.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2alpha/generation/stable-image/inpaint\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\"\n },\n files={\n \"image\": open(\"./husky-in-a-field.png\", \"rb\")\n },\n data={\n \"prompt\": \"golden retriever in a field\",\n \"mode\": \"search\",\n \"search_prompt\": \"dog\",\n \"output_format\": \"webp\",\n },\n)\n\nif response.status_code == 200:\n with open(\"./golden-retriever-in-a-field.webp\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import fs from \"node:fs\";\nimport axios from \"axios\";\nimport FormData from \"form-data\";\n\nconst formData = {\n image: fs.createReadStream(\"./husky-in-a-field.png\"),\n prompt: \"golden retriever standing in a field\",\n mode: \"search\",\n search_prompt: \"dog\",\n output_format: \"webp\"\n};\n\nconst response = await axios.postForm(\n `https://api.stability.ai/v2alpha/generation/stable-image/inpaint`,\n axios.toFormData(formData, new FormData()),\n {\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: { Authorization: `Bearer sk-MYAPIKEY`, accept: \"image/*\" },\n },\n);\n\nif(response.status === 200) {\n fs.writeFileSync(\"./golden-retriever-in-a-field.webp\", Buffer.from(response.data));\n} else {\n throw new Error(`${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2alpha/generation/stable-image/inpaint\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -H \"accept: image/*\" \\\n -F image=@\"./husky-in-a-field.png\" \\\n -F prompt=\"golden retriever in a field\" \\\n -F mode=\"search\" \\\n -F search_prompt=\"dog\" \\\n -F output_format=\"webp\" \\\n -o \"./golden-retriever-in-a-field.webp\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
},
{
"schema": {
"type": "string",
"default": "image/*",
"description": "Specify `image/*` to get the image bytes directly. Otherwise specify `application/json` to receive the image as base64 encoded JSON.",
"enum": ["image/*", "application/json"]
},
"required": false,
"name": "accept",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"oneOf": [
{
"$ref": "#/components/schemas/InpaintingSearchModeRequestBody"
},
{
"$ref": "#/components/schemas/InpaintingMaskingModeRequestBody"
}
],
"discriminator": {
"propertyName": "mode",
"mapping": {
"search": "#/components/schemas/InpaintingSearchModeRequestBody",
"mask": "#/components/schemas/InpaintingMaskingModeRequestBody"
}
}
}
}
}
},
"responses": {
"200": {
"description": "Inpainting was successful.",
"headers": {
"x-request-id": {
"description": "A unique identifier for this request.",
"schema": {
"type": "string"
}
},
"content-type": {
"description": "The format of the generated image.\n\n To receive the bytes of the image directly, specify `image/*` in the accept header. To receive the bytes base64 encoded inside of a JSON payload, specify `application/json`.",
"examples": {
"jpeg": {
"description": "raw bytes",
"value": "image/jpeg"
},
"jpegJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/jpeg"
},
"png": {
"description": "raw bytes",
"value": "image/png"
},
"pngJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/png"
},
"webp": {
"description": "raw bytes",
"value": "image/webp"
},
"webpJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/webp"
}
},
"schema": {
"type": "string"
}
},
"finish-reason": {
"schema": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"]
},
"description": "Indicates the reason the generation finished. \n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `finish_reason`."
},
"seed": {
"description": "The seed used as random noise for this generation.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `seed`.",
"example": "343940597",
"schema": {
"type": "string"
}
}
},
"content": {
"image/jpeg": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated jpeg.\n(Caution: may contain cats)"
},
"application/json; type=image/jpeg": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/png": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated png.\n(Caution: may contain cats)"
},
"application/json; type=image/png": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/webp": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated webp.\n(Caution: may contain cats)"
},
"application/json; type=image/webp": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/image-to-video": {
"post": {
"tags": ["Image-to-Video"],
"summary": "Start generation",
"description": "Generate a short video based on an initial image with [Stable Video Diffusion](https://static1.squarespace.com/static/6213c340453c3f502425776e/t/655ce779b9d47d342a93c890/1700587395994/stable_video_diffusion.pdf),\na latent video diffusion model. \n\n\n\n### How to use\nPlease invoke this endpoint with a `POST` request.\n\nThe headers of the request must include an API key in the `authorization` field. The body of the request must be\n`multipart/form-data`.\n\nThe body of the request should include:\n- `image`\n\nThe body may optionally include:\n- `seed`\n- `cfg_scale`\n- `motion_bucket_id`\n\n> **Note:** for more details about these parameters please see the request schema below.\n\nAfter invoking this endpoint with the required parameters, use the `id` in the response to poll for results at the\n[image-to-video/result/{id}](#tag/Image-to-Video/paths/~1v2beta~1image-to-video~1result~1%7Bid%7D/get) endpoint. Rate-limiting or other errors may occur if you poll more than once every 10 seconds.\n\n### Credits\nFlat rate of 20 credits per successful generation. You will not be charged for failed generations.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2beta/image-to-video\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\"\n },\n files={\n \"image\": open(\"./kittens-in-space.png\", \"rb\")\n },\n data={\n \"seed\": 0,\n \"cfg_scale\": 1.8,\n \"motion_bucket_id\": 127\n },\n)\n\nprint(\"Generation ID:\", response.json().get('id'))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import fs from \"node:fs\";\nimport axios from \"axios\";\nimport FormData from \"form-data\";\n\nconst data = new FormData();\ndata.append(\"image\", fs.readFileSync(\"./image.png\"), \"image.png\");\ndata.append(\"seed\", 0);\ndata.append(\"cfg_scale\", 1.8);\ndata.append(\"motion_bucket_id\", 127);\n\nconst response = await axios.request({\n url: `https://api.stability.ai/v2beta/image-to-video`,\n method: \"post\",\n validateStatus: undefined,\n headers: {\n authorization: `Bearer sk-MYAPIKEY`,\n ...data.getHeaders(),\n },\n data: data,\n});\n\nconsole.log(\"Generation ID:\", response.data.id);"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2beta/image-to-video\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -F image=@\"./image.png\" \\\n -F seed=0 \\\n -F cfg_scale=1.8 \\\n -F motion_bucket_id=127 \\\n -o \"./output.json\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"$ref": "#/components/schemas/ImageToVideoRequest"
}
}
}
},
"responses": {
"200": {
"description": "Video generation started. Poll for results using the `id` in the response [here](#tag/Image-to-Video/paths/~1v2beta~1image-to-video~1result~1%7Bid%7D/get).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"$ref": "#/components/schemas/GenerationID"
}
},
"required": ["id"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"413": {
"description": "Your request was larger than 10MiB.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "4212a4b66fbe1cedca4bf2133d35dca5",
"name": "payload_too_large",
"errors": [
"body: payloads cannot be larger than 10MiB in size"
]
}
}
}
}
},
"422": {
"description": "Your request was well-formed, but rejected. See the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
},
"examples": {
"Invalid Language": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "invalid_language",
"errors": [
"English is the only supported language for this service."
]
}
},
"Public Figure Detected": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "public_figure",
"errors": [
"Our system detected the likeness of a public figure in your image. To comply with our guidelines, this request cannot be processed. Please upload a different image."
]
}
}
}
}
}
},
"429": {
"description": "You have made more than 150 requests in 10 seconds.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "rate_limit_exceeded",
"name": "rate_limit_exceeded",
"errors": [
"You have exceeded the rate limit of 150 requests within a 10 second period, and have been timed out for 60 seconds."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/image-to-video/result/{id}": {
"get": {
"tags": ["Image-to-Video"],
"summary": "Fetch generation result",
"description": "Fetch the result of an image-to-video generation by ID.\n\nMake sure to use the same API key to fetch the generation result that you used to create the generation, \notherwise you will receive a `404` response.\n\n### How to use\nPlease invoke this endpoint with a `GET` request.\n\nThe headers of the request must include an API key in the `authorization` field and the ID\nof your generation must be in the path.\n\n### How is progress reported?\nYour generation is either `in-progress` (i.e. status code `202`) or it is complete (i.e. status code `200`). \nWe may add more fine-grained progress reporting in the future (e.g. a numerical progress).\n\n### How long are results stored?\nResults are stored for 24 hours after generation. After that, the results are deleted and you will need to \nre-generate your video.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\ngeneration_id = \"e52772ac75b...\"\n\nresponse = requests.request(\n \"GET\",\n f\"https://api.stability.ai/v2beta/image-to-video/result/{generation_id}\",\n headers={\n 'accept': \"video/*\", # Use 'application/json' to receive base64 encoded JSON\n 'authorization': f\"Bearer sk-MYAPIKEY\"\n },\n)\n\nif response.status_code == 202:\n print(\"Generation in-progress, try again in 10 seconds.\")\nelif response.status_code == 200:\n print(\"Generation complete!\")\n with open(\"video.mp4\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import axios from \"axios\";\nimport fs from \"node:fs\";\n\nconst generationID = \"e52772ac75b...\";\n\nconst response = await axios.request({\n url: `https://api.stability.ai/v2beta/image-to-video/result/${generationID}`,\n method: \"GET\",\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: {\n Authorization: `Bearer sk-MYAPIKEY`,\n Accept: \"video/*\", // Use 'application/json' to receive base64 encoded JSON\n },\n});\n\nif (response.status === 202) {\n console.log(\"Generation is still running, try again in 10 seconds.\");\n} else if (response.status === 200) {\n console.log(\"Generation is complete!\");\n fs.writeFileSync(\"video.mp4\", Buffer.from(response.data));\n} else {\n throw new Error(`Response ${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "generation_id=\"e52772ac75b...\"\nurl=\"https://api.stability.ai/v2beta/image-to-video/result/$generation_id\"\nhttp_status=$(curl -sS -f -o \"./output.mp4\" -w '%{http_code}' -H \"authorization: sk-MYAPIKEY\" -H 'accept: video/*' \"$url\")\n\ncase $http_status in\n 202)\n echo \"Still processing. Retrying in 10 seconds...\"\n ;;\n 200)\n echo \"Download complete!\"\n ;;\n 4*|5*)\n mv \"./output.mp4\" \"./error.json\"\n echo \"Error: Check ./error.json for details.\"\n exit 1\n ;;\nesac"
}
],
"parameters": [
{
"schema": {
"$ref": "#/components/schemas/GenerationID"
},
"required": true,
"name": "id",
"in": "path"
},
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"default": "video/*",
"description": "Specify `video/*` to receive the bytes of the video directly. Otherwise specify `application/json` to receive the video as base64 encoded JSON.",
"enum": ["video/*", "application/json"]
},
"required": false,
"name": "accept",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"responses": {
"200": {
"description": "The result of your video generation.",
"headers": {
"x-request-id": {
"description": "A unique identifier for this request.",
"schema": {
"type": "string"
}
},
"content-type": {
"description": "The format of the generated video.\n\n To receive the bytes of the video directly, specify `video/*` in the accept header. To receive the bytes base64 encoded inside of a JSON payload, specify `application/json`.",
"examples": {
"mp4": {
"description": "raw bytes",
"value": "video/mp4"
},
"mp4JSON": {
"description": "base64 encoded",
"value": "application/json; type=video/mp4"
}
},
"schema": {
"type": "string"
}
},
"finish-reason": {
"schema": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"]
},
"description": "Indicates the reason the generation finished. \n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and one or more frames have been blurred as a result.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `finish_reason`."
},
"seed": {
"description": "The seed used as random noise for this generation.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `seed`.",
"example": "343940597",
"schema": {
"type": "string"
}
}
},
"content": {
"video/mp4": {
"schema": {
"type": "string",
"description": "The bytes of the generated video.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated mp4.\n(Caution: may contain cats)"
},
"application/json; type=video/mp4": {
"schema": {
"type": "object",
"properties": {
"video": {
"type": "string",
"description": "The generated video, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and one or more frames have been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["video", "finish_reason"]
}
}
}
},
"202": {
"description": "Your image-to-video generation is still in-progress.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"$ref": "#/components/schemas/GenerationID"
},
"status": {
"type": "string",
"enum": ["in-progress"],
"description": "The status of your generation."
}
},
"required": ["id", "status"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"404": {
"description": "id: the generation either does not exist or has expired.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2bca35116bc5431d6dc4b4ea2ef3da2f",
"name": "generation_not_found",
"errors": [
"id: the generation either does not exist or has expired."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/3d/stable-fast-3d": {
"post": {
"tags": ["3D"],
"summary": "Stable Fast 3D",
"description": "Stable Fast 3D generates high-quality 3D assets from a single 2D input image.\n\n### Try it out\nGrab your [API key](https://platform.stability.ai/account/keys) and head over to [](https://colab.research.google.com/github/stability-ai/stability-sdk/blob/main/nbs/Stable_3D_API.ipynb)\n\n### How to use\nPlease invoke this endpoint with a `POST` request.\n\nThe headers of the request must include an API key in the `authorization` field. The body of the request must be\n`multipart/form-data`.\n\nThe body of the request should include:\n- `image`\n\nThe body may optionally include:\n- `texture_resolution`\n- `foreground_ratio`\n\n> **Note:** for more details about these parameters please see the request schema below.\n\n### Output\nThe output is a binary blob that includes a glTF asset, including JSON, buffers, and images. \nSee the [GLB File Format Specification](https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#glb-file-format-specification) for more details.\n\n### Credits\nFlat rate of 2 credits per successful generation. You will not be charged for failed generations.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2beta/3d/stable-fast-3d\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\",\n },\n files={\n \"image\": open(\"./cat-statue.png\", \"rb\")\n },\n data={},\n)\n\nif response.status_code == 200:\n with open(\"./3d-cat-statue.glb\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import axios from \"axios\";\nimport FormData from \"form-data\";\nimport fs from \"node:fs\";\n\nconst payload = {\n image: fs.createReadStream(\"./cat-statue.png\"),\n};\n\nconst response = await axios.postForm(\n `https://api.stability.ai/v2beta/3d/stable-fast-3d`,\n axios.toFormData(payload, new FormData()),\n {\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: {\n Authorization: `Bearer sk-MYAPIKEY`,\n },\n },\n);\n\nif (response.status === 200) {\n fs.writeFileSync(\"./3d-cat-statue.glb\", Buffer.from(response.data));\n} else {\n throw new Error(`${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2beta/3d/stable-fast-3d\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -F image=@\"./cat-statue.png\" \\\n -o \"./3d-cat-statue.glb\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The image to generate a 3D model from.\n\nSupported Formats:\n- jpeg\n- png\n- webp\n\nValidation Rules:\n- Every side must be at least 64 pixels\n- Total pixel count must be between 4,096 and 4,194,304 pixels",
"format": "binary",
"example": "./some/image.png"
},
"texture_resolution": {
"type": "string",
"enum": ["512", "1024", "2048"],
"default": "1024",
"description": "Determines the resolution of the textures used for both the albedo (color) map\nand the normal map. The resolution is specified in pixels, and a higher value\ncorresponds to a higher level of detail in the textures, allowing for more\nintricate and precise rendering of surfaces. However, increasing the resolution\nalso results in larger asset sizes, which may impact loading times and\nperformance. 1024 is a good default value and rarely requires changing."
},
"foreground_ratio": {
"type": "number",
"minimum": 0.1,
"maximum": 1,
"default": 0.85,
"description": "Controls the amount of padding around the object to be processed within the frame.\nThis ratio determines the relative size of the object compared to the total frame\nsize. A higher ratio means less padding and a larger object, while a lower ratio\nincreases the padding, effectively reducing the object’s size within the frame. This\ncan be useful when a long and narrow object, such as a car or bus, is viewed from the\nfront (the narrow side). Here, lowering the foreground ratio might help prevent the\ngenerated 3D assets from appearing squished or distorted. The default value of 0.85 \nis good for most objects."
},
"remesh": {
"type": "string",
"enum": ["none", "triangle", "quad"],
"default": "none",
"description": "Controls the remeshing algorithm used to generate the 3D model. The remeshing\nalgorithm determines how the 3D model is constructed from the input image. The\ndefault value of \"none\" means that the model is generated without remeshing,\nwhich is suitable for most use cases. The \"triangle\" option generates a model\nwith triangular faces, while the \"quad\" option generates a model with quadrilateral\nfaces. The \"quad\" option is useful when the 3D model will be used in DCC tools such\nas Maya or Blender."
},
"vertex_count": {
"type": "number",
"minimum": -1,
"maximum": 20000,
"default": -1,
"description": "If specified, the result will have approximately this many vertices (and consequently fewer faces) in the simplified mesh. \n\nSetting this value to -1 (the default value) means that a limit is not set."
}
},
"required": ["image"]
}
}
}
},
"responses": {
"200": {
"description": "Generation was successful.",
"headers": {
"content-type": {
"description": "The format of the 3D model.",
"schema": {
"type": "string",
"example": "model/gltf-binary"
}
}
},
"content": {
"model/gltf-binary": {
"schema": {
"type": "string",
"description": "The bytes of the generated 3D model.",
"format": "binary"
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"413": {
"description": "Your request was larger than 10MiB.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "4212a4b66fbe1cedca4bf2133d35dca5",
"name": "payload_too_large",
"errors": [
"body: payloads cannot be larger than 10MiB in size"
]
}
}
}
}
},
"429": {
"description": "You have made more than 150 requests in 10 seconds.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "rate_limit_exceeded",
"name": "rate_limit_exceeded",
"errors": [
"You have exceeded the rate limit of 150 requests within a 10 second period, and have been timed out for 60 seconds."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/3d/stable-point-aware-3d": {
"post": {
"tags": ["3D"],
"summary": "Stable Point Aware 3D",
"description": "Stable Point Aware 3D (SPAR3D) can make real-time edits and create the complete structure \nof a 3D object from a single image in a few seconds. SPAR3D combines the strengths of \npoint-cloud diffusion (probabilistic) and mesh regression (deterministic) to have improved \ndetails on the unseen back regions in the input image. \n\nCompared to our previous model [Stable Fast 3D](#tag/3D/paths/~1v2beta~13d~1stable-fast-3d/post), this new \none allows editing of backside information using the point cloud representation and also \nleverages a larger Diffusion model to generally improve the depth and backside \npredictions.\n\nRead more about the model capabilities [here](https://bit.ly/4h7cpgF). \n\nThis API is currently in \npreview. Please don’t hesitate to [contact us](https://stability.ai/contact) with any questions. \n\n### Try it out\nGrab your [API key](https://platform.stability.ai/account/keys) and head over to [](https://colab.research.google.com/github/stability-ai/stability-sdk/blob/main/nbs/Stable_3D_API.ipynb)\n\n### How to use\nPlease invoke this endpoint with a `POST` request.\n\nThe headers of the request must include an API key in the `authorization` field. The body of the request must be\n`multipart/form-data`.\n\nThe body of the request should include:\n- `image`\n\nThe body may optionally include:\n- `texture_resolution`\n- `foreground_ratio`\n- `remesh`\n- `target_type`\n- `target_count`\n- `guidance_scale`\n- `seed`\n\n> **Note:** for more details about these parameters please see the request schema below.\n\n### Output\nThe output is a binary blob that includes a glTF asset, including JSON, buffers, and images. \nSee the [GLB File Format Specification](https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#glb-file-format-specification) for more details.\n\n### Credits\nFlat rate of 4 credits per successful generation. You will not be charged for failed generations.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2beta/3d/stable-point-aware-3d\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\",\n },\n files={\n \"image\": open(\"./cat-statue.png\", \"rb\")\n },\n data={},\n)\n\nif response.status_code == 200:\n with open(\"./3d-cat-statue.glb\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import axios from \"axios\";\nimport FormData from \"form-data\";\nimport fs from \"node:fs\";\n\nconst payload = {\n image: fs.createReadStream(\"./cat-statue.png\"),\n};\n\nconst response = await axios.postForm(\n `https://api.stability.ai/v2beta/3d/stable-point-aware-3d`,\n axios.toFormData(payload, new FormData()),\n {\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: {\n Authorization: `Bearer sk-MYAPIKEY`,\n },\n },\n);\n\nif (response.status === 200) {\n fs.writeFileSync(\"./3d-cat-statue.glb\", Buffer.from(response.data));\n} else {\n throw new Error(`${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2beta/3d/stable-point-aware-3d\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -F image=@\"./cat-statue.png\" \\\n -o \"./3d-cat-statue.glb\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The image to generate a 3D model from.\n\nSupported Formats:\n- jpeg\n- png\n- webp\n\nValidation Rules:\n- Every side must be at least 64 pixels\n- Total pixel count must be between 4,096 and 4,194,304 pixels",
"format": "binary",
"example": "./some/image.png"
},
"texture_resolution": {
"type": "string",
"enum": ["512", "1024", "2048"],
"default": "1024",
"description": "Determines the resolution of the textures used for both the albedo (color) map and the \nnormal map. The resolution is specified in pixels, and a higher value corresponds to a \nhigher level of detail in the textures, allowing for more intricate and precise rendering \nof surfaces. However, increasing the resolution also results in larger asset sizes, which \nmay impact loading times and performance. `1024` is a good default value and rarely requires \nchanging."
},
"foreground_ratio": {
"type": "number",
"minimum": 1,
"maximum": 2,
"default": 1.3,
"description": "Controls the amount of padding around the object to be processed within the frame. This \nratio determines the relative size of the object compared to the total frame size. A \nhigher ratio means less padding and a larger object, while a lower ratio increases the \npadding, effectively reducing the object’s size within the frame. This can be useful when \na long and narrow object, such as a car or bus, is viewed from the front (the narrow \nside). Here, lowering the foreground ratio might help prevent the generated 3D assets from \nappearing squished or distorted. The default value of `1.3` is good for most objects."
},
"remesh": {
"type": "string",
"enum": ["none", "triangle", "quad"],
"default": "none",
"description": "Controls the remeshing algorithm used to generate the 3D model. The remeshing algorithm \ndetermines how the 3D model is constructed from the input image. The default value of \n\"none\" means that the model is generated without remeshing, which is suitable for most use \ncases. The \"triangle\" option generates a model with triangular faces, while the \"quad\" \noption generates a model with quadrilateral faces. The \"quad\" option is useful when the 3D \nmodel will be used in DCC tools such as Maya or Blender."
},
"target_type": {
"type": "string",
"enum": ["none", "vertex", "face"],
"default": "none",
"description": "If set to `vertex` or `face`, the result will have approximately `target_count` many vertices or \nfaces in the simplified mesh, respectively."
},
"target_count": {
"type": "number",
"minimum": 100,
"maximum": 20000,
"default": 1000,
"description": "This sets the target vertex or face count defined by `target_type`. Selecting extremely low \ncounts reduces the quality of the mesh severely and values of 1,000 - 10,000 are recommended."
},
"guidance_scale": {
"type": "number",
"minimum": 1,
"maximum": 10,
"default": 3,
"description": "This sets the guidance scaling of the point diffusion module. Lower values produce less \ndetail and higher can introduce artifacts. The default of `3` produces best results."
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "A specific value that is used to guide the 'randomness' of the generation. (Omit this parameter or pass `0` to use a random seed.)"
}
},
"required": ["image"]
}
}
}
},
"responses": {
"200": {
"description": "Generation was successful.",
"headers": {
"content-type": {
"description": "The format of the 3D model.",
"schema": {
"type": "string",
"example": "model/gltf-binary"
}
}
},
"content": {
"model/gltf-binary": {
"schema": {
"type": "string",
"description": "The bytes of the generated 3D model.",
"format": "binary"
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"413": {
"description": "Your request was larger than 10MiB.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "4212a4b66fbe1cedca4bf2133d35dca5",
"name": "payload_too_large",
"errors": [
"body: payloads cannot be larger than 10MiB in size"
]
}
}
}
}
},
"429": {
"description": "You have made more than 150 requests in 10 seconds.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "rate_limit_exceeded",
"name": "rate_limit_exceeded",
"errors": [
"You have exceeded the rate limit of 150 requests within a 10 second period, and have been timed out for 60 seconds."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/results/{id}": {
"get": {
"tags": ["Results"],
"summary": "Fetch async generation result",
"description": "Fetch the result of a generation by ID. \n\nMake sure to use the same API key to fetch the generation result that you used to create the generation, \notherwise you will receive a `404` response.\n\n### How to use\nPlease invoke this endpoint with a `GET` request.\n\nThe headers of the request must include an API key in the `authorization` field and the ID\nof your generation must be in the path.\n\n### How is progress reported?\nYour generation is either `in-progress` (i.e. status code `202`) or it is complete (i.e. status code `200`). \nWe may add more fine-grained progress reporting in the future (e.g. a numerical progress).\n\n### How long are results stored?\nResults are stored for 24 hours after generation. After that, the results are deleted.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\ngeneration_id = \"e52772ac75b...\"\n\nresponse = requests.request(\n \"GET\",\n f\"https://api.stability.ai/v2beta/results/{generation_id}\",\n headers={\n 'accept': \"image/*\", # Use 'application/json' to receive base64 encoded JSON\n 'authorization': f\"Bearer sk-MYAPIKEY\"\n },\n)\n\nif response.status_code == 202:\n print(\"Generation in-progress, try again in 10 seconds.\")\nelif response.status_code == 200:\n print(\"Generation complete!\")\n with open(\"result.webp\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import axios from \"axios\";\nimport fs from \"node:fs\";\n\nconst generationID = \"e52772ac75b...\";\n\nconst response = await axios.request({\n url: `https://api.stability.ai/v2beta/results/${generationID}`,\n method: \"GET\",\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: {\n Authorization: `Bearer sk-MYAPIKEY`,\n Accept: \"image/*\", // Use 'application/json' to receive base64 encoded JSON\n },\n});\n\nif (response.status === 202) {\n console.log(\"Generation is still running, try again in 10 seconds.\");\n} else if (response.status === 200) {\n console.log(\"Generation is complete!\");\n fs.writeFileSync(\"result.webp\", Buffer.from(response.data));\n} else {\n throw new Error(`Response ${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "generation_id=\"e52772ac75b...\"\nurl=\"https://api.stability.ai/v2beta/results/$generation_id\"\nhttp_status=$(curl -sS -f -o \"./result.webp\" -w '%{http_code}' -H \"authorization: sk-MYAPIKEY\" -H 'accept: image/*' \"$url\")\n\ncase $http_status in\n 202)\n echo \"Still processing. Retrying in 10 seconds...\"\n ;;\n 200)\n echo \"Download complete!\"\n ;;\n 4*|5*)\n mv \"./result.webp\" \"./error.json\"\n echo \"Error: Check ./error.json for details.\"\n exit 1\n ;;\nesac"
}
],
"parameters": [
{
"schema": {
"$ref": "#/components/schemas/GenerationID"
},
"required": true,
"name": "id",
"in": "path"
},
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"default": "*/*",
"description": "Specify `*/*` to receive the bytes of the result directly. Otherwise specify `application/json` to receive the result as base64 encoded JSON.",
"enum": ["*/*", "application/json"]
},
"required": false,
"name": "accept",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"responses": {
"200": {
"description": "Generation finished.",
"headers": {
"x-request-id": {
"description": "A unique identifier for this request.",
"schema": {
"type": "string"
}
},
"content-type": {
"description": "The format of the generated image.\n\n To receive the bytes of the image directly, specify `image/*` in the accept header. To receive the bytes base64 encoded inside of a JSON payload, specify `application/json`.",
"examples": {
"jpeg": {
"description": "raw bytes",
"value": "image/jpeg"
},
"jpegJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/jpeg"
},
"png": {
"description": "raw bytes",
"value": "image/png"
},
"pngJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/png"
},
"webp": {
"description": "raw bytes",
"value": "image/webp"
},
"webpJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/webp"
}
},
"schema": {
"type": "string"
}
},
"finish-reason": {
"schema": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"]
},
"description": "Indicates the reason the generation finished. \n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `finish_reason`."
},
"seed": {
"description": "The seed used as random noise for this generation.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `seed`.",
"example": "343940597",
"schema": {
"type": "string"
}
}
},
"content": {
"image/jpeg": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated jpeg.\n(Caution: may contain cats)"
},
"application/json; type=image/jpeg": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/png": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated png.\n(Caution: may contain cats)"
},
"application/json; type=image/png": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/webp": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated webp.\n(Caution: may contain cats)"
},
"application/json; type=image/webp": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
}
}
},
"202": {
"description": "Your generation is still in-progress.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"$ref": "#/components/schemas/GenerationID"
},
"status": {
"type": "string",
"enum": ["in-progress"],
"description": "The status of your generation."
}
},
"required": ["id", "status"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"404": {
"description": "id: the generation either does not exist or has expired.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2bca35116bc5431d6dc4b4ea2ef3da2f",
"name": "generation_not_found",
"errors": [
"id: the generation either does not exist or has expired."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/stable-image/upscale/conservative": {
"post": {
"tags": ["Upscale"],
"summary": "Conservative",
"description": "Takes images between 64x64 and 1 megapixel and upscales them all the way to 4K resolution. Put more generally, it can upscale images ~20-40x times while preserving all aspects. Conservative Upscale minimizes alterations to the image and should not be used to reimagine an image.\n\n### Try it out\nGrab your [API key](https://platform.stability.ai/account/keys) and head over to [](https://colab.research.google.com/github/stability-ai/stability-sdk/blob/main/nbs/Stable_Image_API_Public.ipynb#scrollTo=t1Q4w2uvvza0)\n\n### How to use\n\nPlease invoke this endpoint with a `POST` request.\n\nThe headers of the request must include an API key in the `authorization` field. The body of the request must be\n`multipart/form-data`, and the `accept` header should be set to one of the following:\n - `image/*` to receive the image in the format specified by the `output_format` parameter.\n - `application/json` to receive the image encoded as base64 in a JSON response.\n \nThe body of the request must include:\n- `image`\n- `prompt`\n\nOptionally, the body of the request may also include:\n- `negative_prompt`\n- `seed`\n- `output_format`\n- `creativity`\n\n> **Note:** for more details about these parameters please see the request schema below.\n\n### Output\nThe resolution of the generated image will be 4 megapixels.\n\n### Credits\nFlat rate of 25 credits per successful generation. You will not be charged for failed generations.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2beta/stable-image/upscale/conservative\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\",\n \"accept\": \"image/*\"\n },\n files={\n \"image\": open(\"./low-res-flower.jpg\", \"rb\"),\n },\n data={\n \"prompt\": \"a flower\",\n \"output_format\": \"webp\",\n },\n)\n\nif response.status_code == 200:\n with open(\"./flower.webp\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import fs from \"node:fs\";\nimport axios from \"axios\";\nimport FormData from \"form-data\";\n\nconst payload = {\n image: fs.createReadStream(\"./low-res-flower.jpg\"),\n prompt: \"a flower\",\n output_format: \"webp\"\n};\n\nconst response = await axios.postForm(\n `https://api.stability.ai/v2beta/stable-image/upscale/conservative`,\n axios.toFormData(payload, new FormData()),\n {\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: { \n Authorization: `Bearer sk-MYAPIKEY`, \n Accept: \"image/*\" \n },\n },\n);\n\nif(response.status === 200) {\n fs.writeFileSync(\"./flower.webp\", Buffer.from(response.data));\n} else {\n throw new Error(`${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2beta/stable-image/upscale/conservative\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -H \"accept: image/*\" \\\n -F image=@\"./low-res-flower.jpg\" \\\n -F prompt=\"a flower\" \\\n -F output_format=\"webp\" \\\n -o \"./flower.webp\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
},
{
"schema": {
"type": "string",
"default": "image/*",
"description": "Specify `image/*` to receive the bytes of the image directly. Otherwise specify `application/json` to receive the image as base64 encoded JSON.",
"enum": ["image/*", "application/json"]
},
"required": false,
"name": "accept",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The image you wish to upscale.\n\nSupported Formats:\n- jpeg\n- png\n- webp\n\nValidation Rules:\n- Every side must be at least 64 pixels\n- Total pixel count must be between 4,096 and 9,437,184 pixels\n- The aspect ratio must be between 1:2.5 and 2.5:1",
"format": "binary",
"example": "./some/image.png"
},
"prompt": {
"type": "string",
"minLength": 1,
"maxLength": 10000,
"description": "What you wish to see in the output image. A strong, descriptive prompt that clearly defines \nelements, colors, and subjects will lead to better results. \n\nTo control the weight of a given word use the format `(word:weight)`, \nwhere `word` is the word you'd like to control the weight of and `weight` \nis a value between 0 and 1. For example: `The sky was a crisp (blue:0.3) and (green:0.8)`\nwould convey a sky that was blue and green, but more green than blue."
},
"negative_prompt": {
"type": "string",
"maxLength": 10000,
"description": "A blurb of text describing what you **do not** wish to see in the output image. \nThis is an advanced feature."
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "A specific value that is used to guide the 'randomness' of the generation. (Omit this parameter or pass `0` to use a random seed.)"
},
"output_format": {
"type": "string",
"enum": ["jpeg", "png", "webp"],
"default": "png",
"description": "Dictates the `content-type` of the generated image."
},
"creativity": {
"$ref": "#/components/schemas/Creativity"
}
},
"required": ["image", "prompt"]
}
}
}
},
"responses": {
"200": {
"description": "Upscale was successful.",
"headers": {
"x-request-id": {
"description": "A unique identifier for this request.",
"schema": {
"type": "string"
}
},
"content-type": {
"description": "The format of the generated image.\n\n To receive the bytes of the image directly, specify `image/*` in the accept header. To receive the bytes base64 encoded inside of a JSON payload, specify `application/json`.",
"examples": {
"jpeg": {
"description": "raw bytes",
"value": "image/jpeg"
},
"jpegJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/jpeg"
},
"png": {
"description": "raw bytes",
"value": "image/png"
},
"pngJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/png"
},
"webp": {
"description": "raw bytes",
"value": "image/webp"
},
"webpJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/webp"
}
},
"schema": {
"type": "string"
}
},
"finish-reason": {
"schema": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"]
},
"description": "Indicates the reason the generation finished. \n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `finish_reason`."
},
"seed": {
"description": "The seed used as random noise for this generation.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `seed`.",
"example": "343940597",
"schema": {
"type": "string"
}
}
},
"content": {
"image/jpeg": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated jpeg.\n(Caution: may contain cats)"
},
"application/json; type=image/jpeg": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/png": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated png.\n(Caution: may contain cats)"
},
"application/json; type=image/png": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/webp": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated webp.\n(Caution: may contain cats)"
},
"application/json; type=image/webp": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"413": {
"description": "Your request was larger than 10MiB.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "4212a4b66fbe1cedca4bf2133d35dca5",
"name": "payload_too_large",
"errors": [
"body: payloads cannot be larger than 10MiB in size"
]
}
}
}
}
},
"422": {
"description": "Your request was well-formed, but rejected. See the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
},
"examples": {
"Invalid Language": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "invalid_language",
"errors": [
"English is the only supported language for this service."
]
}
},
"Public Figure Detected": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "public_figure",
"errors": [
"Our system detected the likeness of a public figure in your image. To comply with our guidelines, this request cannot be processed. Please upload a different image."
]
}
}
}
}
}
},
"429": {
"description": "You have made more than 150 requests in 10 seconds.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "rate_limit_exceeded",
"name": "rate_limit_exceeded",
"errors": [
"You have exceeded the rate limit of 150 requests within a 10 second period, and have been timed out for 60 seconds."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/stable-image/upscale/creative": {
"post": {
"tags": ["Upscale"],
"summary": "Creative Upscale (async)",
"description": "Takes images between 64x64 and 1 megapixel and upscales them all the way to **4K** resolution. Put more \ngenerally, it can upscale images ~20-40x times while preserving, and often enhancing, quality. \nCreative Upscale **works best on highly degraded images and is not for photos of 1mp or above** as it performs \nheavy reimagining (controlled by creativity scale).\n\n### Try it out\nGrab your [API key](https://platform.stability.ai/account/keys) and head over to [](https://colab.research.google.com/github/stability-ai/stability-sdk/blob/main/nbs/Stable_Image_API_Public.ipynb#scrollTo=QXxi9tfI425t)\n\n\n### How to use\nPlease invoke this endpoint with a `POST` request.\n\nThe headers of the request must include an API key in the `authorization` field. The body of the request must be\n`multipart/form-data`.\n\nThe body of the request should include:\n- `image`\n- `prompt`\n\nThe body may optionally include:\n- `seed`\n- `negative_prompt`\n- `output_format`\n- `creativity`\n\n> **Note:** for more details about these parameters please see the request schema below.\n\n### Results\nAfter invoking this endpoint with the required parameters, use the `id` in the response to poll for results at the\n[results/{id} endpoint](#tag/Results/paths/~1v2beta~1results~1%7Bid%7D/get). Rate-limiting or other errors may occur if you poll more than once every 10 seconds.\n\n### Credits\nFlat rate of 25 credits per successful generation. You will not be charged for failed generations.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2beta/stable-image/upscale/creative\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\",\n \"accept\": \"image/*\"\n },\n files={\n \"image\": open(\"./kitten-in-space.png\", \"rb\")\n },\n data={\n \"prompt\": \"cute fluffy white kitten floating in space, pastel colors\",\n \"output_format\": \"webp\",\n },\n)\n\nprint(\"Generation ID:\", response.json().get('id'))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import fs from \"node:fs\";\nimport axios from \"axios\";\nimport FormData from \"form-data\";\n\nconst payload = {\n image: fs.createReadStream(\"./kitten-in-space.png\"),\n prompt: \"cute fluffy white kitten floating in space, pastel colors\",\n output_format: \"webp\"\n};\n\nconst response = await axios.postForm(\n `https://api.stability.ai/v2beta/stable-image/upscale/creative`,\n axios.toFormData(payload, new FormData()),\n {\n validateStatus: undefined,\n headers: { \n Authorization: `Bearer sk-MYAPIKEY`\n },\n },\n);\n\nconsole.log(\"Generation ID:\", response.data.id);"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2beta/stable-image/upscale/creative\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -F image=@\"./kitten-in-rainforest.png\" \\\n -F prompt=\"cute fluffy white kitten sitting in a rainforest, pastel colors\" \\\n -F output_format=webp \\\n -o \"./output.json\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The image you wish to upscale.\n\nSupported Formats:\n- jpeg\n- png\n- webp\n\nValidation Rules:\n- Every side must be at least 64 pixels\n- Total pixel count must be between 4,096 and 1,048,576 pixels",
"format": "binary",
"example": "./some/image.png"
},
"prompt": {
"type": "string",
"minLength": 1,
"maxLength": 10000,
"description": "What you wish to see in the output image. A strong, descriptive prompt that clearly defines \nelements, colors, and subjects will lead to better results. \n\nTo control the weight of a given word use the format `(word:weight)`, \nwhere `word` is the word you'd like to control the weight of and `weight` \nis a value between 0 and 1. For example: `The sky was a crisp (blue:0.3) and (green:0.8)`\nwould convey a sky that was blue and green, but more green than blue."
},
"negative_prompt": {
"type": "string",
"maxLength": 10000,
"description": "A blurb of text describing what you **do not** wish to see in the output image. \nThis is an advanced feature."
},
"output_format": {
"type": "string",
"enum": ["jpeg", "png", "webp"],
"default": "png",
"description": "Dictates the `content-type` of the generated image."
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "A specific value that is used to guide the 'randomness' of the generation. (Omit this parameter or pass `0` to use a random seed.)"
},
"creativity": {
"type": "number",
"minimum": 0,
"maximum": 0.35,
"default": 0.3,
"description": "Indicates how creative the model should be when upscaling an image.\nHigher values will result in more details being added to the image during upscaling."
}
},
"required": ["image", "prompt"]
}
}
}
},
"responses": {
"200": {
"description": "Upscale was started.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"$ref": "#/components/schemas/GenerationID"
}
},
"required": ["id"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"413": {
"description": "Your request was larger than 10MiB.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "4212a4b66fbe1cedca4bf2133d35dca5",
"name": "payload_too_large",
"errors": [
"body: payloads cannot be larger than 10MiB in size"
]
}
}
}
}
},
"422": {
"description": "Your request was well-formed, but rejected. See the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
},
"examples": {
"Invalid Language": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "invalid_language",
"errors": [
"English is the only supported language for this service."
]
}
},
"Public Figure Detected": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "public_figure",
"errors": [
"Our system detected the likeness of a public figure in your image. To comply with our guidelines, this request cannot be processed. Please upload a different image."
]
}
}
}
}
}
},
"429": {
"description": "You have made more than 150 requests in 10 seconds.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "rate_limit_exceeded",
"name": "rate_limit_exceeded",
"errors": [
"You have exceeded the rate limit of 150 requests within a 10 second period, and have been timed out for 60 seconds."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/stable-image/upscale/creative/result/{id}": {
"get": {
"tags": [],
"summary": "Fetch Creative Upscale result",
"description": "Fetch the result of an upscale generation by ID. \n\nMake sure to use the same API key to fetch the generation result that you used to create the generation, \notherwise you will receive a `404` response.\n\n### How to use\nPlease invoke this endpoint with a `GET` request.\n\nThe headers of the request must include an API key in the `authorization` field and the ID\nof your generation must be in the path.\n\n### How is progress reported?\nYour generation is either `in-progress` (i.e. status code `202`) or it is complete (i.e. status code `200`). \nWe may add more fine-grained progress reporting in the future (e.g. a numerical progress).\n\n### How long are results stored?\nResults are stored for 24 hours after generation. After that, the results are deleted.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\ngeneration_id = \"e52772ac75b...\"\n\nresponse = requests.request(\n \"GET\",\n f\"https://api.stability.ai/v2beta/stable-image/upscale/creative/result/{generation_id}\",\n headers={\n 'accept': \"image/*\", # Use 'application/json' to receive base64 encoded JSON\n 'authorization': f\"Bearer sk-MYAPIKEY\"\n },\n)\n\nif response.status_code == 202:\n print(\"Generation in-progress, try again in 10 seconds.\")\nelif response.status_code == 200:\n print(\"Generation complete!\")\n with open(\"upscaled.webp\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import axios from \"axios\";\nimport fs from \"node:fs\";\n\nconst generationID = \"e52772ac75b...\";\n\nconst response = await axios.request({\n url: `https://api.stability.ai/v2beta/stable-image/upscale/creative/result/${generationID}`,\n method: \"GET\",\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: {\n Authorization: `Bearer sk-MYAPIKEY`,\n Accept: \"image/*\", // Use 'application/json' to receive base64 encoded JSON\n },\n});\n\nif (response.status === 202) {\n console.log(\"Generation is still running, try again in 10 seconds.\");\n} else if (response.status === 200) {\n console.log(\"Generation is complete!\");\n fs.writeFileSync(\"upscaled.webp\", Buffer.from(response.data));\n} else {\n throw new Error(`Response ${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "generation_id=\"e52772ac75b...\"\nurl=\"https://api.stability.ai/v2beta/stable-image/upscale/creative/result/$generation_id\"\nhttp_status=$(curl -sS -f -o \"./upscaled.webp\" -w '%{http_code}' -H \"authorization: sk-MYAPIKEY\" -H 'accept: image/*' \"$url\")\n\ncase $http_status in\n 202)\n echo \"Still processing. Retrying in 10 seconds...\"\n ;;\n 200)\n echo \"Download complete!\"\n ;;\n 4*|5*)\n mv \"./upscaled.webp\" \"./error.json\"\n echo \"Error: Check ./error.json for details.\"\n exit 1\n ;;\nesac"
}
],
"parameters": [
{
"schema": {
"$ref": "#/components/schemas/GenerationID"
},
"required": true,
"name": "id",
"in": "path"
},
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"default": "image/*",
"description": "Specify `image/*` to receive the bytes of the image directly. Otherwise specify `application/json` to receive the image as base64 encoded JSON.",
"enum": ["image/*", "application/json"]
},
"required": false,
"name": "accept",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"responses": {
"200": {
"description": "Upscale finished.",
"headers": {
"x-request-id": {
"description": "A unique identifier for this request.",
"schema": {
"type": "string"
}
},
"content-type": {
"description": "The format of the generated image.\n\n To receive the bytes of the image directly, specify `image/*` in the accept header. To receive the bytes base64 encoded inside of a JSON payload, specify `application/json`.",
"examples": {
"jpeg": {
"description": "raw bytes",
"value": "image/jpeg"
},
"jpegJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/jpeg"
},
"png": {
"description": "raw bytes",
"value": "image/png"
},
"pngJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/png"
},
"webp": {
"description": "raw bytes",
"value": "image/webp"
},
"webpJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/webp"
}
},
"schema": {
"type": "string"
}
},
"finish-reason": {
"schema": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"]
},
"description": "Indicates the reason the generation finished. \n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `finish_reason`."
},
"seed": {
"description": "The seed used as random noise for this generation.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `seed`.",
"example": "343940597",
"schema": {
"type": "string"
}
}
},
"content": {
"image/jpeg": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated jpeg.\n(Caution: may contain cats)"
},
"application/json; type=image/jpeg": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/png": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated png.\n(Caution: may contain cats)"
},
"application/json; type=image/png": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/webp": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated webp.\n(Caution: may contain cats)"
},
"application/json; type=image/webp": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
}
}
},
"202": {
"description": "Your upscale generation is still in-progress.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"$ref": "#/components/schemas/GenerationID"
},
"status": {
"type": "string",
"enum": ["in-progress"],
"description": "The status of your generation."
}
},
"required": ["id", "status"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"404": {
"description": "id: the generation either does not exist or has expired.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2bca35116bc5431d6dc4b4ea2ef3da2f",
"name": "generation_not_found",
"errors": [
"id: the generation either does not exist or has expired."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/stable-image/upscale/fast": {
"post": {
"tags": ["Upscale"],
"summary": "Fast",
"description": "Our Fast Upscaler service enhances image resolution by 4x using predictive and generative AI. This lightweight and fast service (processing in ~1 second) is ideal for enhancing the quality of compressed images, making it suitable for social media posts and other applications.\n\n### Try it out\nGrab your [API key](https://platform.stability.ai/account/keys) and head over to [](https://colab.research.google.com/github/stability-ai/stability-sdk/blob/main/nbs/Stable_Image_API_Public.ipynb#scrollTo=t1Q4w2uvvza0)\n\n### How to use\n\nPlease invoke this endpoint with a `POST` request.\n\nThe headers of the request must include an API key in the `authorization` field. The body of the request must be\n`multipart/form-data`, and the `accept` header should be set to one of the following:\n - `image/*` to receive the image in the format specified by the `output_format` parameter.\n - `application/json` to receive the image encoded as base64 in a JSON response.\n \nThe body of the request must include:\n- `image`\n\nOptionally, the body of the request may also include:\n- `output_format`\n\n> **Note:** for more details about these parameters please see the request schema below.\n\n### Output\nThe resolution of the generated image is 4 times that of the input image with a maximum size of 16 megapixels.\n\n### Credits\nFlat rate of 1 credit per successful generation. You will not be charged for failed generations.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2beta/stable-image/upscale/fast\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\",\n \"accept\": \"image/*\"\n },\n files={\n \"image\": open(\"./low-res-flower.jpg\", \"rb\"),\n },\n data={\n \"output_format\": \"webp\",\n },\n)\n\nif response.status_code == 200:\n with open(\"./flower.webp\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import fs from \"node:fs\";\nimport axios from \"axios\";\nimport FormData from \"form-data\";\n\nconst payload = {\n image: fs.createReadStream(\"./low-res-flower.jpg\"),\n output_format: \"webp\"\n};\n\nconst response = await axios.postForm(\n `https://api.stability.ai/v2beta/stable-image/upscale/fast`,\n axios.toFormData(payload, new FormData()),\n {\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: { \n Authorization: `Bearer sk-MYAPIKEY`, \n Accept: \"image/*\" \n },\n },\n);\n\nif(response.status === 200) {\n fs.writeFileSync(\"./flower.webp\", Buffer.from(response.data));\n} else {\n throw new Error(`${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2beta/stable-image/upscale/fast\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -H \"accept: image/*\" \\\n -F image=@\"./low-res-flower.jpg\" \\\n -F output_format=\"webp\" \\\n -o \"./flower.webp\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
},
{
"schema": {
"type": "string",
"default": "image/*",
"description": "Specify `image/*` to receive the bytes of the image directly. Otherwise specify `application/json` to receive the image as base64 encoded JSON.",
"enum": ["image/*", "application/json"]
},
"required": false,
"name": "accept",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The image you wish to upscale.\n\nSupported Formats:\n- jpeg\n- png\n- webp\n\nValidation Rules:\n- Width must be between 32 and 1,536 pixels\n- Height must be between 32 and 1,536 pixels\n- Total pixel count must be between 1,024 and 1,048,576 pixels",
"format": "binary",
"example": "./some/image.png"
},
"output_format": {
"type": "string",
"enum": ["jpeg", "png", "webp"],
"default": "png",
"description": "Dictates the `content-type` of the generated image."
}
},
"required": ["image"]
}
}
}
},
"responses": {
"200": {
"description": "Upscale was successful.",
"headers": {
"x-request-id": {
"description": "A unique identifier for this request.",
"schema": {
"type": "string"
}
},
"content-type": {
"description": "The format of the generated image.\n\n To receive the bytes of the image directly, specify `image/*` in the accept header. To receive the bytes base64 encoded inside of a JSON payload, specify `application/json`.",
"examples": {
"jpeg": {
"description": "raw bytes",
"value": "image/jpeg"
},
"jpegJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/jpeg"
},
"png": {
"description": "raw bytes",
"value": "image/png"
},
"pngJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/png"
},
"webp": {
"description": "raw bytes",
"value": "image/webp"
},
"webpJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/webp"
}
},
"schema": {
"type": "string"
}
},
"finish-reason": {
"schema": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"]
},
"description": "Indicates the reason the generation finished. \n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `finish_reason`."
},
"seed": {
"description": "The seed used as random noise for this generation.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `seed`.",
"example": "343940597",
"schema": {
"type": "string"
}
}
},
"content": {
"image/jpeg": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated jpeg.\n(Caution: may contain cats)"
},
"application/json; type=image/jpeg": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/png": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated png.\n(Caution: may contain cats)"
},
"application/json; type=image/png": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/webp": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated webp.\n(Caution: may contain cats)"
},
"application/json; type=image/webp": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"413": {
"description": "Your request was larger than 10MiB.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "4212a4b66fbe1cedca4bf2133d35dca5",
"name": "payload_too_large",
"errors": [
"body: payloads cannot be larger than 10MiB in size"
]
}
}
}
}
},
"422": {
"description": "Your request was well-formed, but rejected. See the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
},
"examples": {
"Invalid Language": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "invalid_language",
"errors": [
"English is the only supported language for this service."
]
}
},
"Public Figure Detected": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "public_figure",
"errors": [
"Our system detected the likeness of a public figure in your image. To comply with our guidelines, this request cannot be processed. Please upload a different image."
]
}
}
}
}
}
},
"429": {
"description": "You have made more than 150 requests in 10 seconds.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "rate_limit_exceeded",
"name": "rate_limit_exceeded",
"errors": [
"You have exceeded the rate limit of 150 requests within a 10 second period, and have been timed out for 60 seconds."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/stable-image/edit/erase": {
"post": {
"tags": ["Edit"],
"summary": "Erase",
"description": "The Erase service removes unwanted objects, such as blemishes on portraits or items on desks, using image masks.\n\nThe mask is provided in one of two ways:\n 1. Explicitly passing in a separate image via the `mask` parameter \n 2. Derived from the alpha channel of the `image` parameter.\n\n### Try it out\nGrab your [API key](https://platform.stability.ai/account/keys) and head over to [](https://colab.research.google.com/github/stability-ai/stability-sdk/blob/main/nbs/Stable_Image_API_Public.ipynb#scrollTo=t1Q4w2uvvza0)\n\n### How to use\n\nPlease invoke this endpoint with a `POST` request.\n\nThe headers of the request must include an API key in the `authorization` field. The body of the request must be\n`multipart/form-data`, and the `accept` header should be set to one of the following:\n - `image/*` to receive the image in the format specified by the `output_format` parameter.\n - `application/json` to receive the image encoded as base64 in a JSON response.\n \nThe body of the request must include:\n- `image`\n\nOptionally, the body of the request may also include:\n- `mask`\n- `seed`\n- `output_format`\n\n> **Note:** for more details about these parameters please see the request schema below.\n\n### Output\nThe resolution of the generated image will be 4 megapixels.\n\n### Credits\nFlat rate of 3 credits per successful generation. You will not be charged for failed generations.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2beta/stable-image/edit/erase\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\",\n \"accept\": \"image/*\"\n },\n files={\n \"image\": open(\"./kangaroo-on-the-beach.png\", \"rb\"),\n \"mask\": open(\"./mask-of-kangaroo.png\", \"rb\"),\n },\n data={\n \"output_format\": \"webp\",\n },\n)\n\nif response.status_code == 200:\n with open(\"./just-the-beach.webp\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import fs from \"node:fs\";\nimport axios from \"axios\";\nimport FormData from \"form-data\";\n\nconst payload = {\n image: fs.createReadStream(\"./kangaroo-on-the-beach.png\"),\n mask: fs.createReadStream(\"./mask-of-kangaroo.png\"),\n output_format: \"webp\"\n};\n\nconst response = await axios.postForm(\n `https://api.stability.ai/v2beta/stable-image/edit/erase`,\n axios.toFormData(payload, new FormData()),\n {\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: { \n Authorization: `Bearer sk-MYAPIKEY`, \n Accept: \"image/*\" \n },\n },\n);\n\nif(response.status === 200) {\n fs.writeFileSync(\"./just-the-beach.webp\", Buffer.from(response.data));\n} else {\n throw new Error(`${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2beta/stable-image/edit/erase\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -H \"accept: image/*\" \\\n -F image=@\"./kangaroo-on-the-beach.png\" \\\n -F mask=@\"./mask-of-kangaroo.png\" \\\n -F output_format=\"webp\" \\\n -o \"./just-the-beach.webp\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
},
{
"schema": {
"type": "string",
"default": "image/*",
"description": "Specify `image/*` to receive the bytes of the image directly. Otherwise specify `application/json` to receive the image as base64 encoded JSON.",
"enum": ["image/*", "application/json"]
},
"required": false,
"name": "accept",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The image you wish to erase from.\n\nSupported Formats:\n- jpeg\n- png\n- webp\n\nValidation Rules:\n- Every side must be at least 64 pixels\n- Total pixel count must be between 4,096 and 9,437,184 pixels",
"format": "binary",
"example": "./some/image.png"
},
"mask": {
"type": "string",
"description": "Controls the strength of the inpainting process on a per-pixel basis, either via a \nsecond image (passed into this parameter) or via the alpha channel of the `image` parameter.\n\n**Passing in a Mask** \n\nThe image passed to this parameter should be a black and white image that represents, \nat any pixel, the strength of inpainting based on how dark or light the given pixel is. \nCompletely black pixels represent no inpainting strength while completely white pixels \nrepresent maximum strength.\n\nIn the event the mask is a different size than the `image` parameter, it will be automatically resized.\n\n**Alpha Channel Support**\n\nIf you don't provide an explicit mask, one will be derived from the alpha channel of the `image` parameter.\nTransparent pixels will be inpainted while opaque pixels will be preserved.\n\nIn the event an `image` with an alpha channel is provided along with a `mask`, the `mask` will take precedence.",
"format": "binary",
"example": "./some/image.png"
},
"grow_mask": {
"type": "number",
"minimum": 0,
"maximum": 20,
"default": 5,
"description": "Grows the edges of the mask outward in all directions by the specified number of pixels. The expanded area around the mask will be blurred, which can help smooth the transition between inpainted content and the original image.\n\nTry this parameter if you notice seams or rough edges around the inpainted content.\n\n> Note: Excessive growth may obscure fine details in the mask and/or merge nearby masked regions."
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "A specific value that is used to guide the 'randomness' of the generation. (Omit this parameter or pass `0` to use a random seed.)"
},
"output_format": {
"type": "string",
"enum": ["jpeg", "png", "webp"],
"default": "png",
"description": "Dictates the `content-type` of the generated image."
}
},
"required": ["image", "prompt"]
}
}
}
},
"responses": {
"200": {
"description": "Erase was successful.",
"headers": {
"x-request-id": {
"description": "A unique identifier for this request.",
"schema": {
"type": "string"
}
},
"content-type": {
"description": "The format of the generated image.\n\n To receive the bytes of the image directly, specify `image/*` in the accept header. To receive the bytes base64 encoded inside of a JSON payload, specify `application/json`.",
"examples": {
"jpeg": {
"description": "raw bytes",
"value": "image/jpeg"
},
"jpegJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/jpeg"
},
"png": {
"description": "raw bytes",
"value": "image/png"
},
"pngJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/png"
},
"webp": {
"description": "raw bytes",
"value": "image/webp"
},
"webpJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/webp"
}
},
"schema": {
"type": "string"
}
},
"finish-reason": {
"schema": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"]
},
"description": "Indicates the reason the generation finished. \n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `finish_reason`."
},
"seed": {
"description": "The seed used as random noise for this generation.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `seed`.",
"example": "343940597",
"schema": {
"type": "string"
}
}
},
"content": {
"image/jpeg": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated jpeg.\n(Caution: may contain cats)"
},
"application/json; type=image/jpeg": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/png": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated png.\n(Caution: may contain cats)"
},
"application/json; type=image/png": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/webp": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated webp.\n(Caution: may contain cats)"
},
"application/json; type=image/webp": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"413": {
"description": "Your request was larger than 10MiB.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "4212a4b66fbe1cedca4bf2133d35dca5",
"name": "payload_too_large",
"errors": [
"body: payloads cannot be larger than 10MiB in size"
]
}
}
}
}
},
"429": {
"description": "You have made more than 150 requests in 10 seconds.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "rate_limit_exceeded",
"name": "rate_limit_exceeded",
"errors": [
"You have exceeded the rate limit of 150 requests within a 10 second period, and have been timed out for 60 seconds."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/stable-image/edit/inpaint": {
"post": {
"tags": ["Edit"],
"summary": "Inpaint",
"description": "Intelligently modify images by filling in or replacing specified areas with new content based\non the content of a \"mask\" image. \n\nThe \"mask\" is provided in one of two ways:\n 1. Explicitly passing in a separate image via the `mask` parameter \n 2. Derived from the alpha channel of the `image` parameter.\n\n### Try it out\nGrab your [API key](https://platform.stability.ai/account/keys) and head over to [](https://colab.research.google.com/github/stability-ai/stability-sdk/blob/main/nbs/Stable_Image_API_Public.ipynb#scrollTo=t1Q4w2uvvza0)\n\n### How to use\n\nPlease invoke this endpoint with a `POST` request.\n\nThe headers of the request must include an API key in the `authorization` field. The body of the request must be\n`multipart/form-data`, and the `accept` header should be set to one of the following:\n - `image/*` to receive the image in the format specified by the `output_format` parameter.\n - `application/json` to receive the image encoded as base64 in a JSON response.\n \nThe body of the request must include:\n- `image`\n- `prompt`\n\nOptionally, the body of the request may also include:\n- `mask`\n- `negative_prompt`\n- `seed`\n- `output_format`\n\n> **Note:** for more details about these parameters please see the request schema below.\n\n### Output\nThe resolution of the generated image will be 4 megapixels.\n\n### Credits\nFlat rate of 3 credits per successful generation. You will not be charged for failed generations.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2beta/stable-image/edit/inpaint\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\",\n \"accept\": \"image/*\"\n },\n files={\n \"image\": open(\"./dog-wearing-vr-goggles.png\", \"rb\"),\n \"mask\": open(\"./mask.png\", \"rb\"),\n },\n data={\n \"prompt\": \"dog wearing black glasses\",\n \"output_format\": \"webp\",\n },\n)\n\nif response.status_code == 200:\n with open(\"./dog-wearing-black-glasses.webp\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import fs from \"node:fs\";\nimport axios from \"axios\";\nimport FormData from \"form-data\";\n\nconst payload = {\n image: fs.createReadStream(\"./dog-wearing-vr-goggles.png\"),\n mask: fs.createReadStream(\"./mask.png\"),\n prompt: \"dog wearing black glasses\",\n output_format: \"webp\"\n};\n\nconst response = await axios.postForm(\n `https://api.stability.ai/v2beta/stable-image/edit/inpaint`,\n axios.toFormData(payload, new FormData()),\n {\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: { \n Authorization: `Bearer sk-MYAPIKEY`, \n Accept: \"image/*\" \n },\n },\n);\n\nif(response.status === 200) {\n fs.writeFileSync(\"./dog-wearing-black-glasses.webp\", Buffer.from(response.data));\n} else {\n throw new Error(`${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2beta/stable-image/edit/inpaint\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -H \"accept: image/*\" \\\n -F image=@\"./dog-wearing-vr-goggles.png\" \\\n -F mask=@\"./mask.png\" \\\n -F prompt=\"golden retriever in a field\" \\\n -F output_format=\"webp\" \\\n -o \"./dog-wearing-black-glasses.webp\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
},
{
"schema": {
"type": "string",
"default": "image/*",
"description": "Specify `image/*` to receive the bytes of the image directly. Otherwise specify `application/json` to receive the image as base64 encoded JSON.",
"enum": ["image/*", "application/json"]
},
"required": false,
"name": "accept",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The image you wish to inpaint.\n\nSupported Formats:\n- jpeg\n- png\n- webp\n\nValidation Rules:\n- Every side must be at least 64 pixels\n- Total pixel count must be between 4,096 and 9,437,184 pixels",
"format": "binary",
"example": "./some/image.png"
},
"prompt": {
"type": "string",
"minLength": 1,
"maxLength": 10000,
"description": "What you wish to see in the output image. A strong, descriptive prompt that clearly defines \nelements, colors, and subjects will lead to better results. \n\nTo control the weight of a given word use the format `(word:weight)`, \nwhere `word` is the word you'd like to control the weight of and `weight` \nis a value between 0 and 1. For example: `The sky was a crisp (blue:0.3) and (green:0.8)`\nwould convey a sky that was blue and green, but more green than blue."
},
"negative_prompt": {
"type": "string",
"maxLength": 10000,
"description": "A blurb of text describing what you **do not** wish to see in the output image. \nThis is an advanced feature."
},
"mask": {
"type": "string",
"description": "Controls the strength of the inpainting process on a per-pixel basis, either via a \nsecond image (passed into this parameter) or via the alpha channel of the `image` parameter.\n\n**Passing in a Mask** \n\nThe image passed to this parameter should be a black and white image that represents, \nat any pixel, the strength of inpainting based on how dark or light the given pixel is. \nCompletely black pixels represent no inpainting strength while completely white pixels \nrepresent maximum strength.\n\nIn the event the mask is a different size than the `image` parameter, it will be automatically resized.\n\n**Alpha Channel Support**\n\nIf you don't provide an explicit mask, one will be derived from the alpha channel of the `image` parameter.\nTransparent pixels will be inpainted while opaque pixels will be preserved.\n\nIn the event an `image` with an alpha channel is provided along with a `mask`, the `mask` will take precedence.",
"format": "binary",
"example": "./some/image.png"
},
"grow_mask": {
"type": "number",
"minimum": 0,
"maximum": 100,
"default": 5,
"description": "Grows the edges of the mask outward in all directions by the specified number of pixels. The expanded area around the mask will be blurred, which can help smooth the transition between inpainted content and the original image.\n\nTry this parameter if you notice seams or rough edges around the inpainted content.\n\n> Note: Excessive growth may obscure fine details in the mask and/or merge nearby masked regions."
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "A specific value that is used to guide the 'randomness' of the generation. (Omit this parameter or pass `0` to use a random seed.)"
},
"output_format": {
"type": "string",
"enum": ["jpeg", "png", "webp"],
"default": "png",
"description": "Dictates the `content-type` of the generated image."
}
},
"required": ["image", "prompt"]
}
}
}
},
"responses": {
"200": {
"description": "Inpainting was successful.",
"headers": {
"x-request-id": {
"description": "A unique identifier for this request.",
"schema": {
"type": "string"
}
},
"content-type": {
"description": "The format of the generated image.\n\n To receive the bytes of the image directly, specify `image/*` in the accept header. To receive the bytes base64 encoded inside of a JSON payload, specify `application/json`.",
"examples": {
"jpeg": {
"description": "raw bytes",
"value": "image/jpeg"
},
"jpegJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/jpeg"
},
"png": {
"description": "raw bytes",
"value": "image/png"
},
"pngJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/png"
},
"webp": {
"description": "raw bytes",
"value": "image/webp"
},
"webpJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/webp"
}
},
"schema": {
"type": "string"
}
},
"finish-reason": {
"schema": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"]
},
"description": "Indicates the reason the generation finished. \n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `finish_reason`."
},
"seed": {
"description": "The seed used as random noise for this generation.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `seed`.",
"example": "343940597",
"schema": {
"type": "string"
}
}
},
"content": {
"image/jpeg": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated jpeg.\n(Caution: may contain cats)"
},
"application/json; type=image/jpeg": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/png": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated png.\n(Caution: may contain cats)"
},
"application/json; type=image/png": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/webp": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated webp.\n(Caution: may contain cats)"
},
"application/json; type=image/webp": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"413": {
"description": "Your request was larger than 10MiB.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "4212a4b66fbe1cedca4bf2133d35dca5",
"name": "payload_too_large",
"errors": [
"body: payloads cannot be larger than 10MiB in size"
]
}
}
}
}
},
"422": {
"description": "Your request was well-formed, but rejected. See the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
},
"examples": {
"Invalid Language": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "invalid_language",
"errors": [
"English is the only supported language for this service."
]
}
},
"Public Figure Detected": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "public_figure",
"errors": [
"Our system detected the likeness of a public figure in your image. To comply with our guidelines, this request cannot be processed. Please upload a different image."
]
}
}
}
}
}
},
"429": {
"description": "You have made more than 150 requests in 10 seconds.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "rate_limit_exceeded",
"name": "rate_limit_exceeded",
"errors": [
"You have exceeded the rate limit of 150 requests within a 10 second period, and have been timed out for 60 seconds."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/stable-image/edit/outpaint": {
"post": {
"tags": ["Edit"],
"summary": "Outpaint",
"description": "The Outpaint service inserts additional content in an image to fill in the space in any direction. \nCompared to other automated or manual attempts to expand the content in an image, the Outpaint service \nshould minimize artifacts and signs that the original image has been edited.\n\n### Try it out\nGrab your [API key](https://platform.stability.ai/account/keys) and head over to [](https://colab.research.google.com/github/stability-ai/stability-sdk/blob/main/nbs/Stable_Image_API_Public.ipynb#scrollTo=bZ2yK7VQSgLw)\n\n### How to use\n\nPlease invoke this endpoint with a POST request.\n\nThe headers of the request must include an API key in the `authorization` field. The body of the request must be\n`multipart/form-data`, and the `accept` header should be set to one of the following:\n - `image/*` to receive the image in the format specified by the `output_format` parameter.\n - `application/json` to receive the image encoded as base64 in a JSON response.\n \nThe body of the request must include:\n- `image`\n\nAlong with _at least one_ outpaint direction:\n- `left`\n- `right`\n- `up`\n- `down`\n\n> **Note:** for best quality use outpaint direction values smaller or equal to your source image dimensions.\n \nEach of these parameters should be set to a number between 0 and 2000, representing the number of pixels to outpaint in that direction.\n\nOptionally, the body of the request may also include:\n- `prompt`\n- `seed`\n- `output_format`\n- `creativity`\n\n> **Note:** for more details about these parameters please see the request schema below.\n\n### Credits\nFlat rate of 4 credits per successful generation. You will not be charged for failed generations.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2beta/stable-image/edit/outpaint\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\",\n \"accept\": \"image/*\"\n },\n files={\n \"image\": open(\"./husky-in-a-field.png\", \"rb\")\n },\n data={\n \"left\": 200,\n \"down\": 200,\n \"output_format\": \"webp\"\n },\n)\n\nif response.status_code == 200:\n with open(\"./husky-in-a-huge-field.webp\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import fs from \"node:fs\";\nimport axios from \"axios\";\nimport FormData from \"form-data\";\n\nconst payload = {\n image: fs.createReadStream(\"./husky-in-a-field.png\"),\n left: 200,\n down: 200,\n output_format: \"webp\",\n};\n\nconst response = await axios.postForm(\n `https://api.stability.ai/v2beta/stable-image/edit/outpaint`,\n axios.toFormData(payload, new FormData()),\n {\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: { \n Authorization: `Bearer sk-MYAPIKEY`, \n Accept: \"image/*\" \n },\n },\n);\n\nif(response.status === 200) {\n fs.writeFileSync(\"./husky-in-a-huge-field.webp\", Buffer.from(response.data));\n} else {\n throw new Error(`${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2beta/stable-image/edit/outpaint\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -H \"accept: image/*\" \\\n -F image=@\"./husky-in-a-field.png\" \\\n -F left=200 \\\n -F bottom=200 \\\n -F output_format=\"webp\" \\\n -o \"./husky-in-a-huge-field.webp\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
},
{
"schema": {
"type": "string",
"default": "image/*",
"description": "Specify `image/*` to receive the bytes of the image directly. Otherwise specify `application/json` to receive the image as base64 encoded JSON.",
"enum": ["image/*", "application/json"]
},
"required": false,
"name": "accept",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The image you wish to outpaint.\n\nSupported Formats:\n- jpeg\n- png\n- webp\n\nValidation Rules:\n- Every side must be at least 64 pixels\n- Total pixel count must be between 4,096 and 9,437,184 pixels\n- The aspect ratio must be between 1:2.5 and 2.5:1",
"format": "binary",
"example": "./some/image.png"
},
"left": {
"type": "integer",
"minimum": 0,
"maximum": 2000,
"default": 0,
"description": "The number of pixels to outpaint on the left side of the image. At least one outpainting direction must be supplied with a non-zero value."
},
"right": {
"type": "integer",
"minimum": 0,
"maximum": 2000,
"default": 0,
"description": "The number of pixels to outpaint on the right side of the image. At least one outpainting direction must be supplied with a non-zero value."
},
"up": {
"type": "integer",
"minimum": 0,
"maximum": 2000,
"default": 0,
"description": "The number of pixels to outpaint on the top of the image. At least one outpainting direction must be supplied with a non-zero value."
},
"down": {
"type": "integer",
"minimum": 0,
"maximum": 2000,
"default": 0,
"description": "The number of pixels to outpaint on the bottom of the image. At least one outpainting direction must be supplied with a non-zero value."
},
"creativity": {
"allOf": [
{
"$ref": "#/components/schemas/Creativity"
},
{
"minimum": 0,
"maximum": 1,
"default": 0.5
}
]
},
"prompt": {
"type": "string",
"minLength": 0,
"maxLength": 10000,
"description": "What you wish to see in the output image. A strong, descriptive prompt that clearly defines \nelements, colors, and subjects will lead to better results. \n\nTo control the weight of a given word use the format `(word:weight)`, \nwhere `word` is the word you'd like to control the weight of and `weight` \nis a value between 0 and 1. For example: `The sky was a crisp (blue:0.3) and (green:0.8)`\nwould convey a sky that was blue and green, but more green than blue."
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "A specific value that is used to guide the 'randomness' of the generation. (Omit this parameter or pass `0` to use a random seed.)"
},
"output_format": {
"type": "string",
"enum": ["png", "jpeg", "webp"],
"default": "png",
"description": "Dictates the `content-type` of the generated image."
}
},
"required": ["image"]
}
}
}
},
"responses": {
"200": {
"description": "Outpainting was successful.",
"headers": {
"x-request-id": {
"description": "A unique identifier for this request.",
"schema": {
"type": "string"
}
},
"content-type": {
"description": "The format of the generated image.\n\n To receive the bytes of the image directly, specify `image/*` in the accept header. To receive the bytes base64 encoded inside of a JSON payload, specify `application/json`.",
"examples": {
"png": {
"description": "raw bytes",
"value": "image/png"
},
"pngJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/png"
},
"jpeg": {
"description": "raw bytes",
"value": "image/jpeg"
},
"jpegJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/jpeg"
},
"webp": {
"description": "raw bytes",
"value": "image/webp"
},
"webpJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/webp"
}
},
"schema": {
"type": "string"
}
},
"finish-reason": {
"schema": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"]
},
"description": "Indicates the reason the generation finished. \n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `finish_reason`."
},
"seed": {
"description": "The seed used as random noise for this generation.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `seed`.",
"example": "343940597",
"schema": {
"type": "string"
}
}
},
"content": {
"image/png": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated png.\n(Caution: may contain cats)"
},
"application/json; type=image/png": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/jpeg": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated jpeg.\n(Caution: may contain cats)"
},
"application/json; type=image/jpeg": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/webp": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated webp.\n(Caution: may contain cats)"
},
"application/json; type=image/webp": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"413": {
"description": "Your request was larger than 10MiB.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "4212a4b66fbe1cedca4bf2133d35dca5",
"name": "payload_too_large",
"errors": [
"body: payloads cannot be larger than 10MiB in size"
]
}
}
}
}
},
"422": {
"description": "Your request was well-formed, but rejected. See the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
},
"examples": {
"Invalid Language": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "invalid_language",
"errors": [
"English is the only supported language for this service."
]
}
},
"Public Figure Detected": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "public_figure",
"errors": [
"Our system detected the likeness of a public figure in your image. To comply with our guidelines, this request cannot be processed. Please upload a different image."
]
}
}
}
}
}
},
"429": {
"description": "You have made more than 150 requests in 10 seconds.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "rate_limit_exceeded",
"name": "rate_limit_exceeded",
"errors": [
"You have exceeded the rate limit of 150 requests within a 10 second period, and have been timed out for 60 seconds."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/stable-image/edit/search-and-replace": {
"post": {
"tags": ["Edit"],
"summary": "Search and Replace",
"description": "The Search and Replace service is a specific version of inpainting that does not require a mask. \nInstead, users can leverage a `search_prompt` to identify an object in simple language to be replaced. \nThe service will automatically segment the object and replace it with the object requested in the prompt.\n\n### Try it out\nGrab your [API key](https://platform.stability.ai/account/keys) and head over to [](https://colab.research.google.com/github/stability-ai/stability-sdk/blob/main/nbs/Stable_Image_API_Public.ipynb#scrollTo=0lDpGa2jAmAs)\n\n### How to use\nPlease invoke this endpoint with a `POST` request.\n\nThe headers of the request must include an API key in the `authorization` field. The body of the request must be\n`multipart/form-data`, and the `accept` header should be set to one of the following:\n - `image/*` to receive the image in the format specified by the `output_format` parameter.\n - `application/json` to receive the image encoded as base64 in a JSON response.\n\nThe body of the request should include:\n- `image`\n- `prompt`\n- `search_prompt`\n\nThe body may optionally include:\n- `seed`\n- `negative_prompt`\n- `output_format`\n\n> **Note:** for more details about these parameters please see the request schema below.\n\n### Output\nThe resolution of the generated image will be 4 megapixels.\n\n### Credits\nFlat rate of 4 credits per successful generation. You will not be charged for failed generations.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2beta/stable-image/edit/search-and-replace\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\",\n \"accept\": \"image/*\"\n },\n files={\n \"image\": open(\"./husky-in-a-field.png\", \"rb\")\n },\n data={\n \"prompt\": \"golden retriever in a field\",\n \"search_prompt\": \"dog\",\n \"output_format\": \"webp\",\n },\n)\n\nif response.status_code == 200:\n with open(\"./golden-retriever-in-a-field.webp\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import fs from \"node:fs\";\nimport axios from \"axios\";\nimport FormData from \"form-data\";\n\nconst payload = {\n image: fs.createReadStream(\"./husky-in-a-field.png\"),\n prompt: \"golden retriever standing in a field\",\n search_prompt: \"dog\",\n output_format: \"webp\"\n};\n\nconst response = await axios.postForm(\n `https://api.stability.ai/v2beta/stable-image/edit/search-and-replace`,\n axios.toFormData(payload, new FormData()),\n {\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: { \n Authorization: `Bearer sk-MYAPIKEY`, \n Accept: \"image/*\"\n },\n },\n);\n\nif(response.status === 200) {\n fs.writeFileSync(\"./golden-retriever-in-a-field.webp\", Buffer.from(response.data));\n} else {\n throw new Error(`${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2beta/stable-image/edit/search-and-replace\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -H \"accept: image/*\" \\\n -F image=@\"./husky-in-a-field.png\" \\\n -F prompt=\"golden retriever in a field\" \\\n -F search_prompt=\"dog\" \\\n -F output_format=\"webp\" \\\n -o \"./golden-retriever-in-a-field.webp\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
},
{
"schema": {
"type": "string",
"default": "image/*",
"description": "Specify `image/*` to receive the bytes of the image directly. Otherwise specify `application/json` to receive the image as base64 encoded JSON.",
"enum": ["image/*", "application/json"]
},
"required": false,
"name": "accept",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "An image containing content you wish to replace.\n\nSupported Formats:\n- jpeg\n- png\n- webp\n\nValidation Rules:\n- Every side must be at least 64 pixels\n- Total pixel count must be between 4,096 and 9,437,184 pixels\n- The aspect ratio must be between 1:2.5 and 2.5:1",
"format": "binary",
"example": "./some/image.png"
},
"prompt": {
"type": "string",
"minLength": 1,
"maxLength": 10000,
"description": "What you wish to see in the output image. A strong, descriptive prompt that clearly defines \nelements, colors, and subjects will lead to better results. \n\nTo control the weight of a given word use the format `(word:weight)`, \nwhere `word` is the word you'd like to control the weight of and `weight` \nis a value between 0 and 1. For example: `The sky was a crisp (blue:0.3) and (green:0.8)`\nwould convey a sky that was blue and green, but more green than blue."
},
"search_prompt": {
"type": "string",
"maxLength": 10000,
"description": "Short description of what to inpaint in the `image`.",
"example": "glasses"
},
"negative_prompt": {
"type": "string",
"maxLength": 10000,
"description": "A blurb of text describing what you **do not** wish to see in the output image. \nThis is an advanced feature."
},
"grow_mask": {
"type": "number",
"minimum": 0,
"maximum": 20,
"default": 3,
"description": "Grows the edges of the mask outward in all directions by the specified number of pixels. The expanded area around the mask will be blurred, which can help smooth the transition between inpainted content and the original image.\n\nTry this parameter if you notice seams or rough edges around the inpainted content.\n\n> Note: Excessive growth may obscure fine details in the mask and/or merge nearby masked regions."
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "A specific value that is used to guide the 'randomness' of the generation. (Omit this parameter or pass `0` to use a random seed.)"
},
"output_format": {
"type": "string",
"enum": ["jpeg", "png", "webp"],
"default": "png",
"description": "Dictates the `content-type` of the generated image."
}
},
"required": ["image", "prompt", "search_prompt"]
}
}
}
},
"responses": {
"200": {
"description": "Search-and-Replace was successful.",
"headers": {
"x-request-id": {
"description": "A unique identifier for this request.",
"schema": {
"type": "string"
}
},
"content-type": {
"description": "The format of the generated image.\n\n To receive the bytes of the image directly, specify `image/*` in the accept header. To receive the bytes base64 encoded inside of a JSON payload, specify `application/json`.",
"examples": {
"jpeg": {
"description": "raw bytes",
"value": "image/jpeg"
},
"jpegJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/jpeg"
},
"png": {
"description": "raw bytes",
"value": "image/png"
},
"pngJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/png"
},
"webp": {
"description": "raw bytes",
"value": "image/webp"
},
"webpJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/webp"
}
},
"schema": {
"type": "string"
}
},
"finish-reason": {
"schema": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"]
},
"description": "Indicates the reason the generation finished. \n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `finish_reason`."
},
"seed": {
"description": "The seed used as random noise for this generation.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `seed`.",
"example": "343940597",
"schema": {
"type": "string"
}
}
},
"content": {
"image/jpeg": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated jpeg.\n(Caution: may contain cats)"
},
"application/json; type=image/jpeg": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/png": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated png.\n(Caution: may contain cats)"
},
"application/json; type=image/png": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/webp": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated webp.\n(Caution: may contain cats)"
},
"application/json; type=image/webp": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"413": {
"description": "Your request was larger than 10MiB.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "4212a4b66fbe1cedca4bf2133d35dca5",
"name": "payload_too_large",
"errors": [
"body: payloads cannot be larger than 10MiB in size"
]
}
}
}
}
},
"422": {
"description": "Your request was well-formed, but rejected. See the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
},
"examples": {
"Invalid Language": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "invalid_language",
"errors": [
"English is the only supported language for this service."
]
}
},
"Public Figure Detected": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "public_figure",
"errors": [
"Our system detected the likeness of a public figure in your image. To comply with our guidelines, this request cannot be processed. Please upload a different image."
]
}
}
}
}
}
},
"429": {
"description": "You have made more than 150 requests in 10 seconds.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "rate_limit_exceeded",
"name": "rate_limit_exceeded",
"errors": [
"You have exceeded the rate limit of 150 requests within a 10 second period, and have been timed out for 60 seconds."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/stable-image/edit/search-and-recolor": {
"post": {
"tags": ["Edit"],
"summary": "Search and Recolor",
"description": "The Search and Recolor service provides the ability to change the color of a specific object in an image using a prompt.\nThis service is a specific version of inpainting that does not require a mask. The Search and Recolor \nservice will automatically segment the object and recolor it using the colors requested in the prompt.\n\n### Try it out\nGrab your [API key](https://platform.stability.ai/account/keys) and head over to [](https://colab.research.google.com/github/stability-ai/stability-sdk/blob/main/nbs/Stable_Image_API_Public.ipynb#scrollTo=mtgSh4Stj3l)\n\n### How to use\nPlease invoke this endpoint with a `POST` request.\n\nThe headers of the request must include an API key in the `authorization` field. The body of the request must be\n`multipart/form-data`, and the `accept` header should be set to one of the following:\n - `image/*` to receive the image in the format specified by the `output_format` parameter.\n - `application/json` to receive the image encoded as base64 in a JSON response.\n\nThe body of the request should include:\n- `image`\n- `prompt`\n- `select_prompt`\n\nThe body may optionally include:\n- `grow_mask`\n- `seed`\n- `negative_prompt`\n- `output_format`\n\n> **Note:** for more details about these parameters please see the request schema below.\n\n### Output\nThe resolution of the generated image will match the resolution of the input image.\n\n### Credits\nFlat rate of 5 credits per successful generation. You will not be charged for failed generations.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2beta/stable-image/edit/search-and-recolor\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\",\n \"accept\": \"image/*\"\n },\n files={\n \"image\": open(\"./red-car.png\", \"rb\")\n },\n data={\n \"prompt\": \"a yellow car\",\n \"select_prompt\": \"car\",\n \"output_format\": \"webp\",\n },\n)\n\nif response.status_code == 200:\n with open(\"./yellow-car.webp\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import fs from \"node:fs\";\nimport axios from \"axios\";\nimport FormData from \"form-data\";\n\nconst payload = {\n image: fs.createReadStream(\"./red-car.png\"),\n prompt: \"a yellow car\",\n select_prompt: \"car\",\n output_format: \"webp\"\n};\n\nconst response = await axios.postForm(\n `https://api.stability.ai/v2beta/stable-image/edit/search-and-recolor`,\n axios.toFormData(payload, new FormData()),\n {\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: { \n Authorization: `Bearer sk-MYAPIKEY`, \n Accept: \"image/*\"\n },\n },\n);\n\nif(response.status === 200) {\n fs.writeFileSync(\"./yellow-car.webp\", Buffer.from(response.data));\n} else {\n throw new Error(`${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2beta/stable-image/edit/search-and-recolor\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -H \"accept: image/*\" \\\n -F image=@\"./red-car.png\" \\\n -F prompt=\"a yellow car\" \\\n -F select_prompt=\"car\" \\\n -F output_format=\"webp\" \\\n -o \"./yellow-car.webp\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
},
{
"schema": {
"type": "string",
"default": "image/*",
"description": "Specify `image/*` to receive the bytes of the image directly. Otherwise specify `application/json` to receive the image as base64 encoded JSON.",
"enum": ["image/*", "application/json"]
},
"required": false,
"name": "accept",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "An image containing content you wish to recolor.\n\nSupported Formats:\n- jpeg\n- png\n- webp\n\nValidation Rules:\n- Every side must be at least 64 pixels\n- Total pixel count must be between 4,096 and 9,437,184 pixels\n- The aspect ratio must be between 1:2.5 and 2.5:1",
"format": "binary",
"example": "./some/image.png"
},
"prompt": {
"type": "string",
"minLength": 1,
"maxLength": 10000,
"description": "What you wish to see in the output image. A strong, descriptive prompt that clearly defines \nelements, colors, and subjects will lead to better results. \n\nTo control the weight of a given word use the format `(word:weight)`, \nwhere `word` is the word you'd like to control the weight of and `weight` \nis a value between 0 and 1. For example: `The sky was a crisp (blue:0.3) and (green:0.8)`\nwould convey a sky that was blue and green, but more green than blue."
},
"select_prompt": {
"type": "string",
"maxLength": 10000,
"description": "Short description of what to search for in the `image`.",
"example": "glasses"
},
"negative_prompt": {
"type": "string",
"maxLength": 10000,
"description": "A blurb of text describing what you **do not** wish to see in the output image. \nThis is an advanced feature."
},
"grow_mask": {
"type": "number",
"minimum": 0,
"maximum": 20,
"default": 3,
"description": "Grows the edges of the mask outward in all directions by the specified number of pixels. The expanded area around the mask will be blurred, which can help smooth the transition between inpainted content and the original image.\n\nTry this parameter if you notice seams or rough edges around the inpainted content.\n\n> Note: Excessive growth may obscure fine details in the mask and/or merge nearby masked regions."
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "A specific value that is used to guide the 'randomness' of the generation. (Omit this parameter or pass `0` to use a random seed.)"
},
"output_format": {
"type": "string",
"enum": ["jpeg", "png", "webp"],
"default": "png",
"description": "Dictates the `content-type` of the generated image."
}
},
"required": ["image", "prompt", "select_prompt"]
}
}
}
},
"responses": {
"200": {
"description": "Search-and-Recolor was successful.",
"headers": {
"x-request-id": {
"description": "A unique identifier for this request.",
"schema": {
"type": "string"
}
},
"content-type": {
"description": "The format of the generated image.\n\n To receive the bytes of the image directly, specify `image/*` in the accept header. To receive the bytes base64 encoded inside of a JSON payload, specify `application/json`.",
"examples": {
"jpeg": {
"description": "raw bytes",
"value": "image/jpeg"
},
"jpegJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/jpeg"
},
"png": {
"description": "raw bytes",
"value": "image/png"
},
"pngJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/png"
},
"webp": {
"description": "raw bytes",
"value": "image/webp"
},
"webpJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/webp"
}
},
"schema": {
"type": "string"
}
},
"finish-reason": {
"schema": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"]
},
"description": "Indicates the reason the generation finished. \n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `finish_reason`."
},
"seed": {
"description": "The seed used as random noise for this generation.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `seed`.",
"example": "343940597",
"schema": {
"type": "string"
}
}
},
"content": {
"image/jpeg": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated jpeg.\n(Caution: may contain cats)"
},
"application/json; type=image/jpeg": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/png": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated png.\n(Caution: may contain cats)"
},
"application/json; type=image/png": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/webp": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated webp.\n(Caution: may contain cats)"
},
"application/json; type=image/webp": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"413": {
"description": "Your request was larger than 10MiB.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "4212a4b66fbe1cedca4bf2133d35dca5",
"name": "payload_too_large",
"errors": [
"body: payloads cannot be larger than 10MiB in size"
]
}
}
}
}
},
"422": {
"description": "Your request was well-formed, but rejected. See the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
},
"examples": {
"Invalid Language": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "invalid_language",
"errors": [
"English is the only supported language for this service."
]
}
},
"Public Figure Detected": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "public_figure",
"errors": [
"Our system detected the likeness of a public figure in your image. To comply with our guidelines, this request cannot be processed. Please upload a different image."
]
}
}
}
}
}
},
"429": {
"description": "You have made more than 150 requests in 10 seconds.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "rate_limit_exceeded",
"name": "rate_limit_exceeded",
"errors": [
"You have exceeded the rate limit of 150 requests within a 10 second period, and have been timed out for 60 seconds."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/stable-image/edit/remove-background": {
"post": {
"tags": ["Edit"],
"summary": "Remove Background",
"description": "The Remove Background service accurately segments the foreground from an image and implements \nand removes the background.\n\n### Try it out\nGrab your [API key](https://platform.stability.ai/account/keys) and head over to [](https://colab.research.google.com/github/stability-ai/stability-sdk/blob/main/nbs/Stable_Image_API_Public.ipynb#scrollTo=VHofb3LAVmqi)\n\n\n### How to use\n\nPlease invoke this endpoint with a `POST` request.\n\nThe headers of the request must include an API key in the `authorization` field. The body of the request must be\n`multipart/form-data`, and the `accept` header should be set to one of the following:\n - `image/*` to receive the image in the format specified by the `output_format` parameter.\n - `application/json` to receive the image encoded as base64 in a JSON response.\n \nThe body of the request must include:\n- `image`\n\nOptionally, the body of the request may also include:\n- `output_format`\n\n> **Note:** for more details about these parameters please see the request schema below.\n\n### Credits\nFlat rate of 2 credits per successful generation. You will not be charged for failed generations.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2beta/stable-image/edit/remove-background\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\",\n \"accept\": \"image/*\"\n },\n files={\n \"image\": open(\"./husky-in-a-field.png\", \"rb\")\n },\n data={\n \"output_format\": \"webp\"\n },\n)\n\nif response.status_code == 200:\n with open(\"./husky.webp\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import fs from \"node:fs\";\nimport axios from \"axios\";\nimport FormData from \"form-data\";\n\nconst payload = {\n image: fs.createReadStream(\"./husky-in-a-field.png\"),\n output_format: \"webp\"\n};\n\nconst response = await axios.postForm(\n `https://api.stability.ai/v2beta/stable-image/edit/remove-background`,\n axios.toFormData(payload, new FormData()),\n {\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: { \n Authorization: `Bearer sk-MYAPIKEY`, \n Accept: \"image/*\" \n },\n },\n);\n\nif(response.status === 200) {\n fs.writeFileSync(\"./husky.webp\", Buffer.from(response.data));\n} else {\n throw new Error(`${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2beta/stable-image/edit/remove-background\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -H \"accept: image/*\" \\\n -F image=@\"./husky-in-a-field.png\" \\\n -F output_format=\"webp\" \\\n -o \"./husky.webp\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
},
{
"schema": {
"type": "string",
"default": "image/*",
"description": "Specify `image/*` to receive the bytes of the image directly. Otherwise specify `application/json` to receive the image as base64 encoded JSON.",
"enum": ["image/*", "application/json"]
},
"required": false,
"name": "accept",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The image whose background you wish to remove.\n\nSupported Formats:\n- jpeg\n- png\n- webp\n\nValidation Rules:\n- Every side must be at least 64 pixels\n- Total pixel count must be between 4,096 and 4,194,304 pixels",
"format": "binary",
"example": "./some/image.png"
},
"output_format": {
"type": "string",
"enum": ["png", "webp"],
"default": "png",
"description": "Dictates the `content-type` of the generated image."
}
},
"required": ["image"]
}
}
}
},
"responses": {
"200": {
"description": "Background successfully removed.",
"headers": {
"x-request-id": {
"description": "A unique identifier for this request.",
"schema": {
"type": "string"
}
},
"content-type": {
"description": "The format of the generated image.\n\n To receive the bytes of the image directly, specify `image/*` in the accept header. To receive the bytes base64 encoded inside of a JSON payload, specify `application/json`.",
"examples": {
"png": {
"description": "raw bytes",
"value": "image/png"
},
"pngJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/png"
},
"webp": {
"description": "raw bytes",
"value": "image/webp"
},
"webpJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/webp"
}
},
"schema": {
"type": "string"
}
},
"finish-reason": {
"schema": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"]
},
"description": "Indicates the reason the generation finished. \n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `finish_reason`."
},
"seed": {
"description": "The seed used as random noise for this generation.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `seed`.",
"example": "343940597",
"schema": {
"type": "string"
}
}
},
"content": {
"image/png": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated png.\n(Caution: may contain cats)"
},
"application/json; type=image/png": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/webp": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated webp.\n(Caution: may contain cats)"
},
"application/json; type=image/webp": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"413": {
"description": "Your request was larger than 10MiB.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "4212a4b66fbe1cedca4bf2133d35dca5",
"name": "payload_too_large",
"errors": [
"body: payloads cannot be larger than 10MiB in size"
]
}
}
}
}
},
"429": {
"description": "You have made more than 150 requests in 10 seconds.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "rate_limit_exceeded",
"name": "rate_limit_exceeded",
"errors": [
"You have exceeded the rate limit of 150 requests within a 10 second period, and have been timed out for 60 seconds."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/stable-image/edit/replace-background-and-relight": {
"post": {
"tags": ["Edit"],
"summary": "Replace Background and Relight (async)",
"description": "The Replace Background and Relight edit service lets users swap backgrounds with\nAI-generated or uploaded images while adjusting lighting to match the subject. This\nnew API provides a streamlined image editing solution and can serve e-commerce, real\nestate, photography, and creative projects.\n\nSome of the things you can do include:\n - Background Replacement: Remove existing background and add new ones.\n - AI Background Generation: Create new backgrounds using AI generated images based on prompts.\n - Relighting: Adjust lighting in images that are under or overexposed.\n - Flexible Inputs: Use your own background image or generate one.\n - Lighting Adjustments: Modify light reference, direction, and strength.\n\n### Try it out\nGrab your [API key](https://platform.stability.ai/account/keys) and head over to [](https://colab.research.google.com/github/stability-ai/stability-sdk/blob/main/nbs/Stable_Image_API_Public.ipynb#scrollTo=mtgSh4Stj3l)\n\n### How to use\nPlease invoke this endpoint with a `POST` request.\n\nThe headers of the request must include an API key in the `authorization` field. The body of the request must be\n`multipart/form-data`.\n\nThe body of the request should include:\n- `subject_image`\n- `background_prompt` and/or `background_reference`\n\nThe body may optionally include:\n- `light_reference` or `light_source_direction`\n- `light_source_strength` (requires `light_reference` or `light_source_direction`)\n- `foreground_prompt`\n- `negative_prompt`\n- `preserve_original_subject`\n- `original_background_depth`\n- `keep_original_background`\n- `light_source_strength`\n- `seed`\n- `output_format`\n\n> **Note:** for more details about these parameters please see the request schema below.\n\n### Results\nAfter invoking this endpoint with the required parameters, use the `id` in the response to poll for results at the\n[results/{id} endpoint](#tag/Results/paths/~1v2beta~1results~1%7Bid%7D/get). Rate-limiting or other errors may occur if you poll more than once every 10 seconds.\n\n### Credits\nFlat rate of 8 credits per successful generation. You will not be charged for failed generations.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2beta/stable-image/edit/replace-background-and-relight\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\",\n \"accept\": \"image/*\"\n },\n files={\n \"subject_image\": open(\"./husky-in-a-field.png\", \"rb\")\n },\n data={\n \"background_prompt\": \"cinematic lighting\",\n \"output_format\": \"webp\",\n },\n)\n\nprint(\"Generation ID:\", response.json().get('id'))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import fs from \"node:fs\";\nimport axios from \"axios\";\nimport FormData from \"form-data\";\n\nconst payload = {\n subject_image: fs.createReadStream(\"./husky-in-a-field.png\"),\n background_prompt: \"cinematic lighting\",\n output_format: \"webp\"\n};\n\nconst response = await axios.postForm(\n `https://api.stability.ai/v2beta/stable-image/edit/replace-background-and-relight`,\n axios.toFormData(payload, new FormData()),\n {\n validateStatus: undefined,\n headers: { \n Authorization: `Bearer sk-MYAPIKEY`, \n },\n },\n);\n\nconsole.log(\"Generation ID:\", response.data.id);"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2beta/stable-image/edit/replace-background-and-relight\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -H \"accept: image/*\" \\\n -F subject_image=@\"./husky-in-a-field.png\" \\\n -F background_prompt=\"cinematic lighting\" \\\n -F output_format=\"webp\" \\\n -o \"./output.json\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"type": "object",
"properties": {
"subject_image": {
"type": "string",
"description": "An image containing the subject that you wish to change background and relight.\n\nSupported Formats:\n- jpeg\n- png\n- webp\n\nValidation Rules:\n- Every side must be at least 64 pixels\n- Total pixel count must be between 4,096 and 9,437,184 pixels\n- The aspect ratio must be between 1:2.5 and 2.5:1",
"format": "binary",
"example": "./some/image.png"
},
"background_reference": {
"type": "string",
"description": "An image whose style you wish to use in the background. Similar to the Control: Style API,\nstylistic elements from this image are added to the background.\n\n> **Important:** either `background_reference` or `background_prompt` must be provided.\n\nSupported Formats:\n- jpeg\n- png\n- webp\n\nValidation Rules:\n- Every side must be at least 64 pixels\n- Total pixel count must be between 4,096 and 9,437,184 pixels",
"format": "binary",
"example": "./some/image.png"
},
"background_prompt": {
"type": "string",
"maxLength": 10000,
"description": "What you wish to see in the background of the output image. This could be a description\nof the desired background scene, or just a description of the lighting if modifying the\nlight source through `light_source_direction` or `light_reference`.\n\n> **Important:** either `background_reference` or `background_prompt` must be provided."
},
"foreground_prompt": {
"type": "string",
"maxLength": 10000,
"description": "Description of the subject. Use this to prevent elements of the background from\nbleeding into the subject. For example, if you find your subject is turning \ngreen with a forest in the background, try putting a short description of the \nsubject in this field."
},
"negative_prompt": {
"type": "string",
"maxLength": 10000,
"description": "A blurb of text describing what you **do not** wish to see in the output image. \nThis is an advanced feature."
},
"preserve_original_subject": {
"type": "number",
"minimum": 0,
"maximum": 1,
"default": 0.6,
"description": "How much to overlay the original subject to exactly match the original image. A \n1.0 is an exact pixel match for the subject, and 0.0 is a close match but will \nhave new lighting qualities. This is an advanced feature."
},
"original_background_depth": {
"type": "number",
"minimum": 0,
"maximum": 1,
"default": 0.5,
"description": "Controls the generated background to have the same depth as the original subject image. This is an advanced feature."
},
"keep_original_background": {
"type": "string",
"enum": ["true", "false"],
"default": "false",
"description": "Whether to keep the background of the original image. When this is on, the background\nwill have different lighting than the original image that changes based on the other\nparameters in this API."
},
"light_source_direction": {
"type": "string",
"enum": ["left", "right", "above", "below"],
"description": "Direction of the light source."
},
"light_reference": {
"type": "string",
"description": "An image with the desired lighting. Lighter sections of the light_reference image will correspond to sections with brighter lighting in the output image.\n\nSupported Formats:\n- jpeg\n- png\n- webp\n\nValidation Rules:\n- Every side must be at least 64 pixels\n- Total pixel count must be between 4,096 and 9,437,184 pixels",
"format": "binary",
"example": "./some/image.png"
},
"light_source_strength": {
"type": "number",
"minimum": 0,
"maximum": 1,
"default": 0.3,
"description": "If using `light_reference_image` or `light_source_direction`, controls the strength \nof the light source. 1.0 is brighter and 0.0 is dimmer. This is an advanced feature.\n\n> **Important:** Use of this parameter requires `light_reference` or `light_source_direction` to be provided."
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "A specific value that is used to guide the 'randomness' of the generation. (Omit this parameter or pass `0` to use a random seed.)"
},
"output_format": {
"type": "string",
"enum": ["jpeg", "png", "webp"],
"default": "png",
"description": "Dictates the `content-type` of the generated image."
}
},
"required": ["subject_image"]
}
}
}
},
"responses": {
"200": {
"description": "Replace Background and Relight was started.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"$ref": "#/components/schemas/GenerationID"
}
},
"required": ["id"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"413": {
"description": "Your request was larger than 10MiB.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "4212a4b66fbe1cedca4bf2133d35dca5",
"name": "payload_too_large",
"errors": [
"body: payloads cannot be larger than 10MiB in size"
]
}
}
}
}
},
"422": {
"description": "Your request was well-formed, but rejected. See the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
},
"examples": {
"Invalid Language": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "invalid_language",
"errors": [
"English is the only supported language for this service."
]
}
},
"Public Figure Detected": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "public_figure",
"errors": [
"Our system detected the likeness of a public figure in your image. To comply with our guidelines, this request cannot be processed. Please upload a different image."
]
}
}
}
}
}
},
"429": {
"description": "You have made more than 150 requests in 10 seconds.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "rate_limit_exceeded",
"name": "rate_limit_exceeded",
"errors": [
"You have exceeded the rate limit of 150 requests within a 10 second period, and have been timed out for 60 seconds."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/stable-image/generate/ultra": {
"post": {
"tags": ["Generate"],
"summary": "Stable Image Ultra",
"description": "Our most advanced text to image generation service, Stable Image Ultra creates the highest quality images\nwith unprecedented prompt understanding. Ultra excels in typography, complex compositions, dynamic lighting, \nvibrant hues, and overall cohesion and structure of an art piece. Made from the most advanced models,\nincluding Stable Diffusion 3.5, Ultra offers the best of the Stable Diffusion ecosystem.\n\n### Try it out\nGrab your [API key](https://platform.stability.ai/account/keys) and head over to [](https://colab.research.google.com/github/stability-ai/stability-sdk/blob/main/nbs/Stable_Image_API_Public.ipynb#scrollTo=yXhs626oZdr1)\n\n### How to use\nPlease invoke this endpoint with a `POST` request.\n\nThe headers of the request must include an API key in the `authorization` field. The body of the request must be\n`multipart/form-data`. The accept header should be set to one of the following:\n- `image/*` to receive the image in the format specified by the `output_format` parameter.\n- `application/json` to receive the image in the format specified by the `output_format` parameter, but encoded to base64 in a JSON response.\n\nThe only required parameter is the `prompt` field, which should contain the text prompt for the image generation.\n\nThe body of the request should include:\n- `prompt` - text to generate the image from\n\nThe body may optionally include:\n- `image` - the image to use as the starting point for the generation\n- `strength` - controls how much influence the `image` parameter has on the output image\n- `aspect_ratio` - the aspect ratio of the output image\n- `negative_prompt` - keywords of what you **do not** wish to see in the output image\n- `seed` - the randomness seed to use for the generation\n- `output_format` - the the format of the output image\n\n> **Note:** for the full list of optional parameters, please see the request schema below.\n\n### Output\nThe resolution of the generated image will be 1 megapixel. The default resolution is 1024x1024.\n\n### Credits\nThe Ultra service uses 8 credits per successful result. You will not be charged for failed results.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2beta/stable-image/generate/ultra\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\",\n \"accept\": \"image/*\"\n },\n files={\"none\": ''},\n data={\n \"prompt\": \"Lighthouse on a cliff overlooking the ocean\",\n \"output_format\": \"webp\",\n },\n)\n\nif response.status_code == 200:\n with open(\"./lighthouse.webp\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import fs from \"node:fs\";\nimport axios from \"axios\";\nimport FormData from \"form-data\";\n\nconst payload = {\n prompt: \"Lighthouse on a cliff overlooking the ocean\",\n output_format: \"webp\"\n};\n\nconst response = await axios.postForm(\n `https://api.stability.ai/v2beta/stable-image/generate/ultra`,\n axios.toFormData(payload, new FormData()),\n {\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: { \n Authorization: `Bearer sk-MYAPIKEY`, \n Accept: \"image/*\" \n },\n },\n);\n\nif(response.status === 200) {\n fs.writeFileSync(\"./lighthouse.webp\", Buffer.from(response.data));\n} else {\n throw new Error(`${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2beta/stable-image/generate/ultra\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -H \"accept: image/*\" \\\n -F prompt=\"Lighthouse on a cliff overlooking the ocean\" \\\n -F output_format=\"webp\" \\\n -o \"./lighthouse.webp\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
},
{
"schema": {
"type": "string",
"default": "image/*",
"description": "Specify `image/*` to receive the bytes of the image directly. Otherwise specify `application/json` to receive the image as base64 encoded JSON.",
"enum": ["image/*", "application/json"]
},
"required": false,
"name": "accept",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"type": "object",
"properties": {
"prompt": {
"type": "string",
"minLength": 1,
"maxLength": 10000,
"description": "What you wish to see in the output image. A strong, descriptive prompt that clearly defines \nelements, colors, and subjects will lead to better results. \n\nTo control the weight of a given word use the format `(word:weight)`, \nwhere `word` is the word you'd like to control the weight of and `weight` \nis a value between 0 and 1. For example: `The sky was a crisp (blue:0.3) and (green:0.8)`\nwould convey a sky that was blue and green, but more green than blue."
},
"negative_prompt": {
"type": "string",
"maxLength": 10000,
"description": "A blurb of text describing what you **do not** wish to see in the output image. \nThis is an advanced feature."
},
"aspect_ratio": {
"type": "string",
"enum": [
"21:9",
"16:9",
"3:2",
"5:4",
"1:1",
"4:5",
"2:3",
"9:16",
"9:21"
],
"default": "1:1",
"description": "Controls the aspect ratio of the generated image."
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "A specific value that is used to guide the 'randomness' of the generation. (Omit this parameter or pass `0` to use a random seed.)"
},
"output_format": {
"type": "string",
"enum": ["jpeg", "png", "webp"],
"default": "png",
"description": "Dictates the `content-type` of the generated image."
},
"image": {
"type": "string",
"description": "The image to use as the starting point for the generation.\n\n> **Important:** The `strength` parameter is required when `image` is provided.\n\nSupported Formats:\n- jpeg\n- png\n- webp\n\nValidation Rules:\n- Width must be between 64 and 16,384 pixels\n- Height must be between 64 and 16,384 pixels\n- Total pixel count must be at least 4,096 pixels",
"format": "binary",
"example": "./some/image.png"
},
"strength": {
"type": "number",
"minimum": 0,
"maximum": 1,
"description": "Sometimes referred to as _denoising_, this parameter controls how much influence the \n`image` parameter has on the generated image. A value of 0 would yield an image that \nis identical to the input. A value of 1 would be as if you passed in no image at all.\n\n> **Important:** This parameter is required when `image` is provided."
}
},
"required": ["prompt"]
}
}
}
},
"responses": {
"200": {
"description": "Generation was successful.",
"headers": {
"x-request-id": {
"description": "A unique identifier for this request.",
"schema": {
"type": "string"
}
},
"content-type": {
"description": "The format of the generated image.\n\n To receive the bytes of the image directly, specify `image/*` in the accept header. To receive the bytes base64 encoded inside of a JSON payload, specify `application/json`.",
"examples": {
"jpeg": {
"description": "raw bytes",
"value": "image/jpeg"
},
"jpegJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/jpeg"
},
"png": {
"description": "raw bytes",
"value": "image/png"
},
"pngJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/png"
},
"webp": {
"description": "raw bytes",
"value": "image/webp"
},
"webpJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/webp"
}
},
"schema": {
"type": "string"
}
},
"finish-reason": {
"schema": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"]
},
"description": "Indicates the reason the generation finished. \n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `finish_reason`."
},
"seed": {
"description": "The seed used as random noise for this generation.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `seed`.",
"example": "343940597",
"schema": {
"type": "string"
}
}
},
"content": {
"image/jpeg": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated jpeg.\n(Caution: may contain cats)"
},
"application/json; type=image/jpeg": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/png": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated png.\n(Caution: may contain cats)"
},
"application/json; type=image/png": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/webp": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated webp.\n(Caution: may contain cats)"
},
"application/json; type=image/webp": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"413": {
"description": "Your request was larger than 10MiB.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "4212a4b66fbe1cedca4bf2133d35dca5",
"name": "payload_too_large",
"errors": [
"body: payloads cannot be larger than 10MiB in size"
]
}
}
}
}
},
"422": {
"description": "Your request was well-formed, but rejected. See the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
},
"examples": {
"Invalid Language": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "invalid_language",
"errors": [
"English is the only supported language for this service."
]
}
},
"Public Figure Detected": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "public_figure",
"errors": [
"Our system detected the likeness of a public figure in your image. To comply with our guidelines, this request cannot be processed. Please upload a different image."
]
}
}
}
}
}
},
"429": {
"description": "You have made more than 150 requests in 10 seconds.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "rate_limit_exceeded",
"name": "rate_limit_exceeded",
"errors": [
"You have exceeded the rate limit of 150 requests within a 10 second period, and have been timed out for 60 seconds."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/stable-image/generate/core": {
"post": {
"tags": ["Generate"],
"summary": "Stable Image Core",
"description": "Our primary service for text-to-image generation, Stable Image Core represents the best quality achievable at high \nspeed. No prompt engineering is required! Try asking for a style, a scene, or a character, and see what you get.\n\n### Try it out\nGrab your [API key](https://platform.stability.ai/account/keys) and head over to [](https://colab.research.google.com/github/stability-ai/stability-sdk/blob/main/nbs/Stable_Image_API_Public.ipynb#scrollTo=yXhs626oZdr1)\n\n### How to use\nPlease invoke this endpoint with a `POST` request.\n\nThe headers of the request must include an API key in the `authorization` field. The body of the request must be\n`multipart/form-data`, and the `accept` header should be set to one of the following:\n - `image/*` to receive the image in the format specified by the `output_format` parameter.\n - `application/json` to receive the image encoded as base64 in a JSON response.\n\nThe body of the request should include:\n- `prompt`\n\nThe body may optionally include:\n- `aspect_ratio`\n- `negative_prompt`\n- `seed`\n- `style_preset`\n- `output_format`\n\n> **Note:** for more details about these parameters please see the request schema below.\n\n### Output\nThe resolution of the generated image will be 1.5 megapixels.\n\n### Credits\nFlat rate of 3 credits per successful generation. You will not be charged for failed generations.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2beta/stable-image/generate/core\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\",\n \"accept\": \"image/*\"\n },\n files={\"none\": ''},\n data={\n \"prompt\": \"Lighthouse on a cliff overlooking the ocean\",\n \"output_format\": \"webp\",\n },\n)\n\nif response.status_code == 200:\n with open(\"./lighthouse.webp\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import fs from \"node:fs\";\nimport axios from \"axios\";\nimport FormData from \"form-data\";\n\nconst payload = {\n prompt: \"Lighthouse on a cliff overlooking the ocean\",\n output_format: \"webp\"\n};\n\nconst response = await axios.postForm(\n `https://api.stability.ai/v2beta/stable-image/generate/core`,\n axios.toFormData(payload, new FormData()),\n {\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: { \n Authorization: `Bearer sk-MYAPIKEY`, \n Accept: \"image/*\" \n },\n },\n);\n\nif(response.status === 200) {\n fs.writeFileSync(\"./lighthouse.webp\", Buffer.from(response.data));\n} else {\n throw new Error(`${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2beta/stable-image/generate/core\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -H \"accept: image/*\" \\\n -F prompt=\"Lighthouse on a cliff overlooking the ocean\" \\\n -F output_format=\"webp\" \\\n -o \"./lighthouse.webp\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
},
{
"schema": {
"type": "string",
"default": "image/*",
"description": "Specify `image/*` to receive the bytes of the image directly. Otherwise specify `application/json` to receive the image as base64 encoded JSON.",
"enum": ["image/*", "application/json"]
},
"required": false,
"name": "accept",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"type": "object",
"properties": {
"prompt": {
"type": "string",
"minLength": 1,
"maxLength": 10000,
"description": "What you wish to see in the output image. A strong, descriptive prompt that clearly defines \nelements, colors, and subjects will lead to better results. \n\nTo control the weight of a given word use the format `(word:weight)`, \nwhere `word` is the word you'd like to control the weight of and `weight` \nis a value between 0 and 1. For example: `The sky was a crisp (blue:0.3) and (green:0.8)`\nwould convey a sky that was blue and green, but more green than blue."
},
"aspect_ratio": {
"type": "string",
"enum": [
"21:9",
"16:9",
"3:2",
"5:4",
"1:1",
"4:5",
"2:3",
"9:16",
"9:21"
],
"default": "1:1",
"description": "Controls the aspect ratio of the generated image."
},
"negative_prompt": {
"type": "string",
"maxLength": 10000,
"description": "A blurb of text describing what you **do not** wish to see in the output image. \nThis is an advanced feature."
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "A specific value that is used to guide the 'randomness' of the generation. (Omit this parameter or pass `0` to use a random seed.)"
},
"style_preset": {
"type": "string",
"enum": [
"enhance",
"anime",
"photographic",
"digital-art",
"comic-book",
"fantasy-art",
"line-art",
"analog-film",
"neon-punk",
"isometric",
"low-poly",
"origami",
"modeling-compound",
"cinematic",
"3d-model",
"pixel-art",
"tile-texture"
],
"description": "Guides the image model towards a particular style."
},
"output_format": {
"type": "string",
"enum": ["png", "jpeg", "webp"],
"default": "png",
"description": "Dictates the `content-type` of the generated image."
}
},
"required": ["prompt"]
}
}
}
},
"responses": {
"200": {
"description": "Generation was successful.",
"headers": {
"x-request-id": {
"description": "A unique identifier for this request.",
"schema": {
"type": "string"
}
},
"content-type": {
"description": "The format of the generated image.\n\n To receive the bytes of the image directly, specify `image/*` in the accept header. To receive the bytes base64 encoded inside of a JSON payload, specify `application/json`.",
"examples": {
"png": {
"description": "raw bytes",
"value": "image/png"
},
"pngJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/png"
},
"jpeg": {
"description": "raw bytes",
"value": "image/jpeg"
},
"jpegJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/jpeg"
},
"webp": {
"description": "raw bytes",
"value": "image/webp"
},
"webpJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/webp"
}
},
"schema": {
"type": "string"
}
},
"finish-reason": {
"schema": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"]
},
"description": "Indicates the reason the generation finished. \n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `finish_reason`."
},
"seed": {
"description": "The seed used as random noise for this generation.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `seed`.",
"example": "343940597",
"schema": {
"type": "string"
}
}
},
"content": {
"image/png": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated png.\n(Caution: may contain cats)"
},
"application/json; type=image/png": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/jpeg": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated jpeg.\n(Caution: may contain cats)"
},
"application/json; type=image/jpeg": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/webp": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated webp.\n(Caution: may contain cats)"
},
"application/json; type=image/webp": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"422": {
"description": "Your request was well-formed, but rejected. See the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
},
"examples": {
"Invalid Language": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "invalid_language",
"errors": [
"English is the only supported language for this service."
]
}
},
"Public Figure Detected": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "public_figure",
"errors": [
"Our system detected the likeness of a public figure in your image. To comply with our guidelines, this request cannot be processed. Please upload a different image."
]
}
}
}
}
}
},
"429": {
"description": "You have made more than 150 requests in 10 seconds.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "rate_limit_exceeded",
"name": "rate_limit_exceeded",
"errors": [
"You have exceeded the rate limit of 150 requests within a 10 second period, and have been timed out for 60 seconds."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/stable-image/generate/sd3": {
"post": {
"tags": ["Generate"],
"summary": "Stable Diffusion 3.0 & 3.5",
"description": "Generate using Stable Diffusion 3.5 models, Stability AI latest base model:\n\n- **Stable Diffusion 3.5 Large**: At 8 billion parameters, with superior quality and\n prompt adherence, this base model is the most powerful in the Stable Diffusion\n family. This model is ideal for professional use cases at 1 megapixel resolution.\n\n- **Stable Diffusion 3.5 Large Turbo**: A distilled version of Stable Diffusion 3.5 Large.\n SD3.5 Large Turbo generates high-quality images with exceptional prompt adherence\n in just 4 steps, making it considerably faster than Stable Diffusion 3.5 Large.\n\n- **Stable Diffusion 3.5 Medium**: With 2.5 billion parameters, the model delivers an\n optimal balance between prompt accuracy and image quality, making it an efficient\n choice for fast high-performance image generation.\n\nRead more about the model capabilities [here](https://stability.ai/news/introducing-stable-diffusion-3-5).\n\nStable Diffusion 3.0 models are also supported, powered by [Fireworks AI](https://fireworks.ai/). API status can be reviewed [here](https://readme.fireworks.ai/page/application-status).\n\n- **SD3 Large**: the 8 billion parameter model\n- **SD3 Large Turbo**: the 8 billion parameter model with a faster inference time\n- **SD3 Medium**: the 2 billion parameter model\n\n### Try it out\nGrab your [API key](https://platform.stability.ai/account/keys) and head over to [](https://colab.research.google.com/github/stability-ai/stability-sdk/blob/main/nbs/SD3_API.ipynb)\n\n### How to use\nPlease invoke this endpoint with a `POST` request.\n\nThe headers of the request must include an API key in the `authorization` field. The body of the request must be\n`multipart/form-data`. The accept header should be set to one of the following:\n- `image/*` to receive the image in the format specified by the `output_format` parameter.\n- `application/json` to receive the image encoded as base64 in a JSON response.\n\n#### **Generating with a prompt**\nCommonly referred to as **text-to-image**, this mode generates an image from text alone. While the only required\nparameter is the `prompt`, it also supports an `aspect_ratio` parameter which can be used to control the\naspect ratio of the generated image.\n\n#### **Generating with a prompt *and* an image**\nCommonly referred to as **image-to-image**, this mode also generates an image from text but uses an existing image as the\nstarting point. The required parameters are:\n- `prompt` - text to generate the image from\n- `image` - the image to use as the starting point for the generation\n- `strength` - controls how much influence the `image` parameter has on the output image\n- `mode` - must be set to `image-to-image`\n\n> **Note:** maximum request size is 10MiB.\n\n#### **Optional Parameters:**\nBoth modes support the following optional parameters:\n- `model` - the model to use (SD3 Large, SD3 Large Turbo, or SD3 Medium)\n- `output_format` - the the format of the output image\n- `seed` - the randomness seed to use for the generation\n- `negative_prompt` - keywords of what you **do not** wish to see in the output image\n- `cfg_scale` - controls how strictly the diffusion process adheres to the prompt text\n\n> **Note:** for more details about these parameters please see the request schema below.\n\n### Output\nThe resolution of the generated image will be 1MP. The default resolution is 1024x1024.\n\n### Credits\n- **SD 3.5 & 3.0 Large**: Flat rate of 6.5 credits per successful generation.\n- **SD 3.5 & 3.0 Large Turbo**: Flat rate of 4 credits per successful generation.\n- **SD 3.5 & 3.0 Medium**: Flat rate of 3.5 credits per successful generation.\n\nAs always, you will not be charged for failed generations.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2beta/stable-image/generate/sd3\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\",\n \"accept\": \"image/*\"\n },\n files={\"none\": ''},\n data={\n \"prompt\": \"Lighthouse on a cliff overlooking the ocean\",\n \"output_format\": \"jpeg\",\n },\n)\n\nif response.status_code == 200:\n with open(\"./lighthouse.jpeg\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import fs from \"node:fs\";\nimport axios from \"axios\";\nimport FormData from \"form-data\";\n\nconst payload = {\n prompt: \"Lighthouse on a cliff overlooking the ocean\",\n output_format: \"jpeg\"\n};\n\nconst response = await axios.postForm(\n `https://api.stability.ai/v2beta/stable-image/generate/sd3`,\n axios.toFormData(payload, new FormData()),\n {\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: { \n Authorization: `Bearer sk-MYAPIKEY`, \n Accept: \"image/*\" \n },\n },\n);\n\nif(response.status === 200) {\n fs.writeFileSync(\"./lighthouse.jpeg\", Buffer.from(response.data));\n} else {\n throw new Error(`${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2beta/stable-image/generate/sd3\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -H \"accept: image/*\" \\\n -F prompt=\"Lighthouse on a cliff overlooking the ocean\" \\\n -F output_format=\"jpeg\" \\\n -o \"./lighthouse.jpeg\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
},
{
"schema": {
"type": "string",
"default": "image/*",
"description": "Specify `image/*` to receive the bytes of the image directly. Otherwise specify `application/json` to receive the image as base64 encoded JSON.",
"enum": ["image/*", "application/json"]
},
"required": false,
"name": "accept",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"type": "object",
"properties": {
"prompt": {
"type": "string",
"minLength": 1,
"maxLength": 10000,
"description": "What you wish to see in the output image. A strong, descriptive prompt that clearly defines\nelements, colors, and subjects will lead to better results."
},
"mode": {
"type": "string",
"enum": ["text-to-image", "image-to-image"],
"default": "text-to-image",
"description": "Controls whether this is a text-to-image or image-to-image generation, which affects which parameters are required:\n- **text-to-image** requires only the `prompt` parameter\n- **image-to-image** requires the `prompt`, `image`, and `strength` parameters",
"title": "GenerationMode"
},
"image": {
"type": "string",
"description": "The image to use as the starting point for the generation.\n\nSupported formats:\n - jpeg\n - png\n - webp\n\nSupported dimensions:\n - Every side must be at least 64 pixels\n \n> **Important:** This parameter is only valid for **image-to-image** requests.",
"format": "binary"
},
"strength": {
"type": "number",
"minimum": 0,
"maximum": 1,
"description": "Sometimes referred to as _denoising_, this parameter controls how much influence the \n`image` parameter has on the generated image. A value of 0 would yield an image that \nis identical to the input. A value of 1 would be as if you passed in no image at all.\n\n> **Important:** This parameter is only valid for **image-to-image** requests."
},
"aspect_ratio": {
"type": "string",
"enum": [
"21:9",
"16:9",
"3:2",
"5:4",
"1:1",
"4:5",
"2:3",
"9:16",
"9:21"
],
"default": "1:1",
"description": "Controls the aspect ratio of the generated image. Defaults to 1:1.\n\n> **Important:** This parameter is only valid for **text-to-image** requests."
},
"model": {
"type": "string",
"enum": [
"sd3.5-large",
"sd3.5-large-turbo",
"sd3.5-medium",
"sd3-medium",
"sd3-large",
"sd3-large-turbo"
],
"default": "sd3.5-large",
"description": "The model to use for generation.\n\n- `sd3.5-large` requires 6.5 credits per generation\n- `sd3.5-large-turbo` requires 4 credits per generation\n- `sd3.5-medium` requires 3.5 credits per generation\n- `sd3-large` requires 6.5 credits per generation\n- `sd3-large-turbo` requires 4 credits per generation\n- `sd3-medium` requires 3.5 credits per generation"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "A specific value that is used to guide the 'randomness' of the generation. (Omit this parameter or pass `0` to use a random seed.)"
},
"output_format": {
"type": "string",
"enum": ["png", "jpeg"],
"default": "png",
"description": "Dictates the `content-type` of the generated image."
},
"negative_prompt": {
"type": "string",
"maxLength": 10000,
"description": "Keywords of what you **do not** wish to see in the output image.\nThis is an advanced feature.\n\n> **Important:** This parameter does **not** work with `sd3-large-turbo`."
},
"cfg_scale": {
"type": "number",
"minimum": 1,
"maximum": 10,
"description": "How strictly the diffusion process adheres to the prompt text (higher values keep your image closer to your prompt)."
}
},
"required": ["prompt"]
}
}
}
},
"responses": {
"200": {
"description": "Generation was successful.",
"headers": {
"x-request-id": {
"description": "A unique identifier for this request.",
"schema": {
"type": "string"
}
},
"content-type": {
"description": "The format of the generated image.\n\n To receive the bytes of the image directly, specify `image/*` in the accept header. To receive the bytes base64 encoded inside of a JSON payload, specify `application/json`.",
"examples": {
"png": {
"description": "raw bytes",
"value": "image/png"
},
"pngJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/png"
},
"jpeg": {
"description": "raw bytes",
"value": "image/jpeg"
},
"jpegJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/jpeg"
}
},
"schema": {
"type": "string"
}
},
"finish-reason": {
"schema": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"]
},
"description": "Indicates the reason the generation finished. \n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `finish_reason`."
},
"seed": {
"description": "The seed used as random noise for this generation.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `seed`.",
"example": "343940597",
"schema": {
"type": "string"
}
}
},
"content": {
"image/png": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated png.\n(Caution: may contain cats)"
},
"application/json; type=image/png": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/jpeg": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated jpeg.\n(Caution: may contain cats)"
},
"application/json; type=image/jpeg": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"413": {
"description": "Your request was larger than 10MiB.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "4212a4b66fbe1cedca4bf2133d35dca5",
"name": "payload_too_large",
"errors": [
"body: payloads cannot be larger than 10MiB in size"
]
}
}
}
}
},
"422": {
"description": "Your request was well-formed, but rejected. See the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
},
"examples": {
"Invalid Language": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "invalid_language",
"errors": [
"English is the only supported language for this service."
]
}
},
"Public Figure Detected": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "public_figure",
"errors": [
"Our system detected the likeness of a public figure in your image. To comply with our guidelines, this request cannot be processed. Please upload a different image."
]
}
}
}
}
}
},
"429": {
"description": "You have made more than 150 requests in 10 seconds.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "rate_limit_exceeded",
"name": "rate_limit_exceeded",
"errors": [
"You have exceeded the rate limit of 150 requests within a 10 second period, and have been timed out for 60 seconds."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/stable-image/control/sketch": {
"post": {
"tags": ["Control"],
"summary": "Sketch",
"description": "This service offers an ideal solution for design projects that require brainstorming and\nfrequent iterations. It upgrades rough hand-drawn sketches to refined outputs with precise \ncontrol. For non-sketch images, it allows detailed manipulation of the final appearance by \nleveraging the contour lines and edges within the image.\n\n### Try it out\nGrab your [API key](https://platform.stability.ai/account/keys) and head over to [](https://colab.research.google.com/github/stability-ai/stability-sdk/blob/main/nbs/Stable_Image_API_Public.ipynb#scrollTo=ZKIAqHzJzzUo)\n\n### How to use\nPlease invoke this endpoint with a `POST` request.\n\nThe headers of the request must include an API key in the `authorization` field. The body of the request must be\n`multipart/form-data`, and the `accept` header should be set to one of the following:\n - `image/*` to receive the image in the format specified by the `output_format` parameter.\n - `application/json` to receive the image encoded as base64 in a JSON response.\n\nThe body of the request should include:\n- `image`\n- `prompt`\n\nThe body may optionally include:\n- `control_strength`\n- `negative_prompt`\n- `seed`\n- `output_format`\n\n> **Note:** for more details about these parameters please see the request schema below.\n\n### Output\nThe resolution of the generated image will match that of the input image.\n\n### Credits\nFlat rate of 3 credits per successful generation. You will not be charged for failed generations.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2beta/stable-image/control/sketch\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\",\n \"accept\": \"image/*\"\n },\n files={\n \"image\": open(\"./sketch.png\", \"rb\")\n },\n data={\n \"prompt\": \"a medieval castle on a hill\",\n \"control_strength\": 0.7,\n \"output_format\": \"webp\"\n },\n)\n\nif response.status_code == 200:\n with open(\"./castle.webp\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import axios from \"axios\";\nimport FormData from \"form-data\";\nimport fs from \"node:fs\";\n\nconst payload = {\n image: fs.createReadStream(\"./sketch.png\"),\n prompt: \"a medieval castle on a hill\",\n control_strength: 0.6,\n output_format: \"webp\",\n};\n\nconst response = await axios.postForm(\n `https://api.stability.ai/v2beta/stable-image/control/sketch`,\n axios.toFormData(payload, new FormData()),\n {\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: {\n Authorization: `Bearer sk-MYAPIKEY`,\n Accept: \"image/*\"\n },\n },\n);\n\nif (response.status === 200) {\n fs.writeFileSync(\"./castle.webp\", Buffer.from(response.data));\n} else {\n throw new Error(`${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2beta/stable-image/control/sketch\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -H \"accept: image/*\" \\\n -F image=@\"./sketch.png\" \\\n -F prompt=\"a medieval castle on a hill\" \\\n -F control_strength=0.7 \\\n -F output_format=\"webp\" \\\n -o \"./castle.webp\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
},
{
"schema": {
"type": "string",
"default": "image/*",
"description": "Specify `image/*` to receive the bytes of the image directly. Otherwise specify `application/json` to receive the image as base64 encoded JSON.",
"enum": ["image/*", "application/json"]
},
"required": false,
"name": "accept",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"type": "object",
"properties": {
"prompt": {
"type": "string",
"minLength": 1,
"maxLength": 10000,
"description": "What you wish to see in the output image. A strong, descriptive prompt that clearly defines \nelements, colors, and subjects will lead to better results. \n\nTo control the weight of a given word use the format `(word:weight)`, \nwhere `word` is the word you'd like to control the weight of and `weight` \nis a value between 0 and 1. For example: `The sky was a crisp (blue:0.3) and (green:0.8)`\nwould convey a sky that was blue and green, but more green than blue."
},
"image": {
"type": "string",
"description": "Supported Formats:\n- jpeg\n- png\n- webp\n\nImage Dimensions:\n- Every side must be at least 64 pixels\n- The total pixel count cannot exceed 9,437,184 pixels (e.g. 3072x3072, 4096x2304, etc.)\n\nImage Aspect Ratio:\n- Must be between 1:2.5 and 2.5:1 (i.e. cannot be too tall or too wide)",
"format": "binary",
"example": "./some/image.png"
},
"control_strength": {
"type": "number",
"minimum": 0,
"maximum": 1,
"default": 0.7,
"description": "How much influence, or control, the `image` has on the generation. Represented as a float between 0 and 1, where 0 is the least influence and 1 is the maximum."
},
"negative_prompt": {
"type": "string",
"maxLength": 10000,
"description": "A blurb of text describing what you **do not** wish to see in the output image. \nThis is an advanced feature."
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "A specific value that is used to guide the 'randomness' of the generation. (Omit this parameter or pass `0` to use a random seed.)"
},
"output_format": {
"type": "string",
"enum": ["png", "jpeg", "webp"],
"default": "png",
"description": "Dictates the `content-type` of the generated image."
}
},
"required": ["prompt", "image"]
}
}
}
},
"responses": {
"200": {
"description": "Generation was successful.",
"headers": {
"x-request-id": {
"description": "A unique identifier for this request.",
"schema": {
"type": "string"
}
},
"content-type": {
"description": "The format of the generated image.\n\n To receive the bytes of the image directly, specify `image/*` in the accept header. To receive the bytes base64 encoded inside of a JSON payload, specify `application/json`.",
"examples": {
"png": {
"description": "raw bytes",
"value": "image/png"
},
"pngJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/png"
},
"jpeg": {
"description": "raw bytes",
"value": "image/jpeg"
},
"jpegJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/jpeg"
},
"webp": {
"description": "raw bytes",
"value": "image/webp"
},
"webpJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/webp"
}
},
"schema": {
"type": "string"
}
},
"finish-reason": {
"schema": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"]
},
"description": "Indicates the reason the generation finished. \n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `finish_reason`."
},
"seed": {
"description": "The seed used as random noise for this generation.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `seed`.",
"example": "343940597",
"schema": {
"type": "string"
}
}
},
"content": {
"image/png": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated png.\n(Caution: may contain cats)"
},
"application/json; type=image/png": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/jpeg": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated jpeg.\n(Caution: may contain cats)"
},
"application/json; type=image/jpeg": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/webp": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated webp.\n(Caution: may contain cats)"
},
"application/json; type=image/webp": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"413": {
"description": "Your request was larger than 10MiB.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "4212a4b66fbe1cedca4bf2133d35dca5",
"name": "payload_too_large",
"errors": [
"body: payloads cannot be larger than 10MiB in size"
]
}
}
}
}
},
"422": {
"description": "Your request was well-formed, but rejected. See the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
},
"examples": {
"Invalid Language": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "invalid_language",
"errors": [
"English is the only supported language for this service."
]
}
},
"Public Figure Detected": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "public_figure",
"errors": [
"Our system detected the likeness of a public figure in your image. To comply with our guidelines, this request cannot be processed. Please upload a different image."
]
}
}
}
}
}
},
"429": {
"description": "You have made more than 150 requests in 10 seconds.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "rate_limit_exceeded",
"name": "rate_limit_exceeded",
"errors": [
"You have exceeded the rate limit of 150 requests within a 10 second period, and have been timed out for 60 seconds."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/stable-image/control/structure": {
"post": {
"tags": ["Control"],
"summary": "Structure",
"description": "This service excels in generating images by maintaining the structure of an input image, \nmaking it especially valuable for advanced content creation scenarios such as recreating \nscenes or rendering characters from models.\n\n### Try it out\nGrab your [API key](https://platform.stability.ai/account/keys) and head over to [](https://colab.research.google.com/github/stability-ai/stability-sdk/blob/main/nbs/Stable_Image_API_Public.ipynb#scrollTo=59RaZazXz0AU)\n\n### How to use\nPlease invoke this endpoint with a `POST` request.\n\nThe headers of the request must include an API key in the `authorization` field. The body of the request must be\n`multipart/form-data`, and the `accept` header should be set to one of the following:\n - `image/*` to receive the image in the format specified by the `output_format` parameter.\n - `application/json` to receive the image encoded as base64 in a JSON response.\n\nThe body of the request should include:\n- `image`\n- `prompt`\n\nThe body may optionally include:\n- `control_strength`\n- `negative_prompt`\n- `seed`\n- `output_format`\n\n> **Note:** for more details about these parameters please see the request schema below.\n\n### Output\nThe resolution of the generated image will match that of the input image.\n\n### Credits\nFlat rate of 3 credits per successful generation. You will not be charged for failed generations.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2beta/stable-image/control/structure\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\",\n \"accept\": \"image/*\"\n },\n files={\n \"image\": open(\"./cat-statue.png\", \"rb\")\n },\n data={\n \"prompt\": \"a well manicured shrub in an english garden\",\n \"control_strength\": 0.7,\n \"output_format\": \"webp\"\n },\n)\n\nif response.status_code == 200:\n with open(\"./shrub-in-a-garden.webp\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import axios from \"axios\";\nimport FormData from \"form-data\";\nimport fs from \"node:fs\";\n\nconst payload = {\n image: fs.createReadStream(\"./cat-statue.png\"),\n prompt: \"a well manicured shrub in an english garden\",\n control_strength: 0.6,\n output_format: \"webp\",\n};\n\nconst response = await axios.postForm(\n `https://api.stability.ai/v2beta/stable-image/control/structure`,\n axios.toFormData(payload, new FormData()),\n {\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: {\n Authorization: `Bearer sk-MYAPIKEY`,\n Accept: \"image/*\"\n },\n },\n);\n\nif (response.status === 200) {\n fs.writeFileSync(\"./shrub-in-a-garden.webp\", Buffer.from(response.data));\n} else {\n throw new Error(`${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2beta/stable-image/control/structure\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -H \"accept: image/*\" \\\n -F image=@\"./cat-statue.png\" \\\n -F prompt=\"a well manicured shrub in an english garden\" \\\n -F control_strength=0.7 \\\n -F output_format=\"webp\" \\\n -o \"./shrub-in-a-garden.webp\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
},
{
"schema": {
"type": "string",
"default": "image/*",
"description": "Specify `image/*` to receive the bytes of the image directly. Otherwise specify `application/json` to receive the image as base64 encoded JSON.",
"enum": ["image/*", "application/json"]
},
"required": false,
"name": "accept",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"type": "object",
"properties": {
"prompt": {
"type": "string",
"minLength": 1,
"maxLength": 10000,
"description": "What you wish to see in the output image. A strong, descriptive prompt that clearly defines \nelements, colors, and subjects will lead to better results. \n\nTo control the weight of a given word use the format `(word:weight)`, \nwhere `word` is the word you'd like to control the weight of and `weight` \nis a value between 0 and 1. For example: `The sky was a crisp (blue:0.3) and (green:0.8)`\nwould convey a sky that was blue and green, but more green than blue."
},
"image": {
"type": "string",
"description": "An image whose structure you wish to use as the foundation for a generation.\n\nSupported Formats:\n- jpeg\n- png\n- webp\n\nValidation Rules:\n- Every side must be at least 64 pixels\n- Total pixel count must be between 4,096 and 9,437,184 pixels\n- The aspect ratio must be between 1:2.5 and 2.5:1",
"format": "binary",
"example": "./some/image.png"
},
"control_strength": {
"type": "number",
"minimum": 0,
"maximum": 1,
"default": 0.7,
"description": "How much influence, or control, the `image` has on the generation. Represented as a float between 0 and 1, where 0 is the least influence and 1 is the maximum."
},
"negative_prompt": {
"type": "string",
"maxLength": 10000,
"description": "A blurb of text describing what you **do not** wish to see in the output image. \nThis is an advanced feature."
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "A specific value that is used to guide the 'randomness' of the generation. (Omit this parameter or pass `0` to use a random seed.)"
},
"output_format": {
"type": "string",
"enum": ["png", "jpeg", "webp"],
"default": "png",
"description": "Dictates the `content-type` of the generated image."
}
},
"required": ["prompt", "image"]
}
}
}
},
"responses": {
"200": {
"description": "Generation was successful.",
"headers": {
"x-request-id": {
"description": "A unique identifier for this request.",
"schema": {
"type": "string"
}
},
"content-type": {
"description": "The format of the generated image.\n\n To receive the bytes of the image directly, specify `image/*` in the accept header. To receive the bytes base64 encoded inside of a JSON payload, specify `application/json`.",
"examples": {
"png": {
"description": "raw bytes",
"value": "image/png"
},
"pngJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/png"
},
"jpeg": {
"description": "raw bytes",
"value": "image/jpeg"
},
"jpegJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/jpeg"
},
"webp": {
"description": "raw bytes",
"value": "image/webp"
},
"webpJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/webp"
}
},
"schema": {
"type": "string"
}
},
"finish-reason": {
"schema": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"]
},
"description": "Indicates the reason the generation finished. \n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `finish_reason`."
},
"seed": {
"description": "The seed used as random noise for this generation.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `seed`.",
"example": "343940597",
"schema": {
"type": "string"
}
}
},
"content": {
"image/png": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated png.\n(Caution: may contain cats)"
},
"application/json; type=image/png": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/jpeg": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated jpeg.\n(Caution: may contain cats)"
},
"application/json; type=image/jpeg": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/webp": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated webp.\n(Caution: may contain cats)"
},
"application/json; type=image/webp": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"413": {
"description": "Your request was larger than 10MiB.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "4212a4b66fbe1cedca4bf2133d35dca5",
"name": "payload_too_large",
"errors": [
"body: payloads cannot be larger than 10MiB in size"
]
}
}
}
}
},
"422": {
"description": "Your request was well-formed, but rejected. See the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
},
"examples": {
"Invalid Language": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "invalid_language",
"errors": [
"English is the only supported language for this service."
]
}
},
"Public Figure Detected": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "public_figure",
"errors": [
"Our system detected the likeness of a public figure in your image. To comply with our guidelines, this request cannot be processed. Please upload a different image."
]
}
}
}
}
}
},
"429": {
"description": "You have made more than 150 requests in 10 seconds.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "rate_limit_exceeded",
"name": "rate_limit_exceeded",
"errors": [
"You have exceeded the rate limit of 150 requests within a 10 second period, and have been timed out for 60 seconds."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v2beta/stable-image/control/style": {
"post": {
"tags": ["Control"],
"summary": "Style",
"description": "This service extracts stylistic elements from an input image (control image) and uses it to guide the creation of an output image based on the prompt. The result is a new image in the same style as the control image.\n\n### Try it out\nGrab your [API key](https://platform.stability.ai/account/keys) and head over to [](https://colab.research.google.com/github/stability-ai/stability-sdk/blob/main/nbs/Stable_Image_API_Public.ipynb#scrollTo=y0WKjG72RvTE)\n\n### How to use\nPlease invoke this endpoint with a `POST` request.\n\nThe headers of the request must include an API key in the `authorization` field. The body of the request must be\n`multipart/form-data`, and the `accept` header should be set to one of the following:\n - `image/*` to receive the image in the format specified by the `output_format` parameter.\n - `application/json` to receive the image encoded as base64 in a JSON response.\n\nThe body of the request should include:\n- `image`\n- `prompt`\n\nThe body may optionally include:\n- `negative_prompt`\n- `aspect_ratio`\n- `fidelity`\n- `seed`\n- `output_format`\n\n> **Note:** for more details about these parameters please see the request schema below.\n\n### Output\nThe resolution of the generated image will be 1MP. The default resolution is 1024x1024.\n\n### Credits\nFlat rate of 4 credits per successful generation. You will not be charged for failed generations.",
"x-codeSamples": [
{
"lang": "python",
"label": "Python",
"source": "import requests\n\nresponse = requests.post(\n f\"https://api.stability.ai/v2beta/stable-image/control/style\",\n headers={\n \"authorization\": f\"Bearer sk-MYAPIKEY\",\n \"accept\": \"image/*\"\n },\n files={\n \"image\": open(\"./cinematic-portrait.png\", \"rb\")\n },\n data={\n \"prompt\": \"a majestic portrait of a chicken\",\n \"output_format\": \"webp\"\n },\n)\n\nif response.status_code == 200:\n with open(\"./chicken-portrait.webp\", 'wb') as file:\n file.write(response.content)\nelse:\n raise Exception(str(response.json()))"
},
{
"lang": "javascript",
"label": "JavaScript",
"source": "import axios from \"axios\";\nimport FormData from \"form-data\";\nimport fs from \"node:fs\";\n\nconst payload = {\n image: fs.createReadStream(\"./cinematic-portrait.png\"),\n prompt: \"a majestic portrait of a chicken\",\n output_format: \"webp\",\n};\n\nconst response = await axios.postForm(\n `https://api.stability.ai/v2beta/stable-image/control/style`,\n axios.toFormData(payload, new FormData()),\n {\n validateStatus: undefined,\n responseType: \"arraybuffer\",\n headers: {\n Authorization: `Bearer sk-MYAPIKEY`,\n Accept: \"image/*\"\n },\n },\n);\n\nif (response.status === 200) {\n fs.writeFileSync(\"./chicken-portrait.webp\", Buffer.from(response.data));\n} else {\n throw new Error(`${response.status}: ${response.data.toString()}`);\n}"
},
{
"lang": "terminal",
"label": "cURL",
"source": "curl -f -sS \"https://api.stability.ai/v2beta/stable-image/control/style\" \\\n -H \"authorization: Bearer sk-MYAPIKEY\" \\\n -H \"accept: image/*\" \\\n -F image=@\"./cinematic-portrait.png\" \\\n -F prompt=\"a majestic portrait of a chicken\" \\\n -F output_format=\"webp\" \\\n -o \"./chicken-portrait.webp\""
}
],
"parameters": [
{
"schema": {
"type": "string",
"description": "Your [Stability API key](https://platform.stability.ai/account/keys), used to authenticate your requests. Although you may have multiple keys in your account, you should use the same key for all requests to this API.",
"minLength": 1
},
"required": true,
"name": "authorization",
"in": "header"
},
{
"schema": {
"type": "string",
"minLength": 1,
"description": "The content type of the request body. Do not manually specify this header; your HTTP client library will automatically include the appropriate boundary parameter.",
"example": "multipart/form-data"
},
"required": true,
"name": "content-type",
"in": "header"
},
{
"schema": {
"type": "string",
"default": "image/*",
"description": "Specify `image/*` to receive the bytes of the image directly. Otherwise specify `application/json` to receive the image as base64 encoded JSON.",
"enum": ["image/*", "application/json"]
},
"required": false,
"name": "accept",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientID"
},
"required": false,
"name": "stability-client-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientUserID"
},
"required": false,
"name": "stability-client-user-id",
"in": "header"
},
{
"schema": {
"$ref": "#/components/schemas/StabilityClientVersion"
},
"required": false,
"name": "stability-client-version",
"in": "header"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"type": "object",
"properties": {
"prompt": {
"type": "string",
"minLength": 1,
"maxLength": 10000,
"description": "What you wish to see in the output image. A strong, descriptive prompt that clearly defines \nelements, colors, and subjects will lead to better results. \n\nTo control the weight of a given word use the format `(word:weight)`, \nwhere `word` is the word you'd like to control the weight of and `weight` \nis a value between 0 and 1. For example: `The sky was a crisp (blue:0.3) and (green:0.8)`\nwould convey a sky that was blue and green, but more green than blue."
},
"image": {
"type": "string",
"description": "An image whose style you wish to use as the foundation for a generation.\n\nSupported Formats:\n- jpeg\n- png\n- webp\n\nValidation Rules:\n- Every side must be at least 64 pixels\n- Total pixel count must be between 4,096 and 9,437,184 pixels\n- The aspect ratio must be between 1:2.5 and 2.5:1",
"format": "binary",
"example": "./some/image.png"
},
"negative_prompt": {
"type": "string",
"maxLength": 10000,
"description": "A blurb of text describing what you **do not** wish to see in the output image. \nThis is an advanced feature."
},
"aspect_ratio": {
"type": "string",
"enum": [
"21:9",
"16:9",
"3:2",
"5:4",
"1:1",
"4:5",
"2:3",
"9:16",
"9:21"
],
"default": "1:1",
"description": "Controls the aspect ratio of the generated image."
},
"fidelity": {
"type": "number",
"minimum": 0,
"maximum": 1,
"default": 0.5,
"description": "How closely the output image's style resembles the input image's style."
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "A specific value that is used to guide the 'randomness' of the generation. (Omit this parameter or pass `0` to use a random seed.)"
},
"output_format": {
"type": "string",
"enum": ["png", "jpeg", "webp"],
"default": "png",
"description": "Dictates the `content-type` of the generated image."
}
},
"required": ["prompt", "image"]
}
}
}
},
"responses": {
"200": {
"description": "Generation was successful.",
"headers": {
"x-request-id": {
"description": "A unique identifier for this request.",
"schema": {
"type": "string"
}
},
"content-type": {
"description": "The format of the generated image.\n\n To receive the bytes of the image directly, specify `image/*` in the accept header. To receive the bytes base64 encoded inside of a JSON payload, specify `application/json`.",
"examples": {
"png": {
"description": "raw bytes",
"value": "image/png"
},
"pngJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/png"
},
"jpeg": {
"description": "raw bytes",
"value": "image/jpeg"
},
"jpegJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/jpeg"
},
"webp": {
"description": "raw bytes",
"value": "image/webp"
},
"webpJSON": {
"description": "base64 encoded",
"value": "application/json; type=image/webp"
}
},
"schema": {
"type": "string"
}
},
"finish-reason": {
"schema": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"]
},
"description": "Indicates the reason the generation finished. \n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `finish_reason`."
},
"seed": {
"description": "The seed used as random noise for this generation.\n\n> **NOTE:** This header is absent on JSON encoded responses because it is present in the body as `seed`.",
"example": "343940597",
"schema": {
"type": "string"
}
}
},
"content": {
"image/png": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated png.\n(Caution: may contain cats)"
},
"application/json; type=image/png": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/jpeg": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated jpeg.\n(Caution: may contain cats)"
},
"application/json; type=image/jpeg": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
},
"image/webp": {
"schema": {
"type": "string",
"description": "The bytes of the generated image.\n\nThe `finish-reason` and `seed` will be present as headers.",
"format": "binary"
},
"example": "The bytes of the generated webp.\n(Caution: may contain cats)"
},
"application/json; type=image/webp": {
"schema": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The generated image, encoded to base64.",
"example": "AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1..."
},
"finish_reason": {
"type": "string",
"enum": ["SUCCESS", "CONTENT_FILTERED"],
"description": "The reason the generation finished.\n\n- `SUCCESS` = successful generation.\n- `CONTENT_FILTERED` = successful generation, however the output violated our content moderation \npolicy and has been blurred as a result.",
"example": "SUCCESS"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "The seed used as random noise for this generation.",
"example": 343940597
}
},
"required": ["image", "finish_reason"]
}
}
}
},
"400": {
"description": "Invalid parameter(s), see the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
}
}
}
},
"403": {
"description": "Your request was flagged by our content moderation system.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ContentModerationResponse"
}
}
}
},
"413": {
"description": "Your request was larger than 10MiB.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "4212a4b66fbe1cedca4bf2133d35dca5",
"name": "payload_too_large",
"errors": [
"body: payloads cannot be larger than 10MiB in size"
]
}
}
}
}
},
"422": {
"description": "Your request was well-formed, but rejected. See the `errors` field for details.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"]
},
"examples": {
"Invalid Language": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "invalid_language",
"errors": [
"English is the only supported language for this service."
]
}
},
"Public Figure Detected": {
"value": {
"id": "ff54b236a3acdde1522cb1ba641c43ed",
"name": "public_figure",
"errors": [
"Our system detected the likeness of a public figure in your image. To comply with our guidelines, this request cannot be processed. Please upload a different image."
]
}
}
}
}
}
},
"429": {
"description": "You have made more than 150 requests in 10 seconds.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "rate_limit_exceeded",
"name": "rate_limit_exceeded",
"errors": [
"You have exceeded the rate limit of 150 requests within a 10 second period, and have been timed out for 60 seconds."
]
}
}
}
}
},
"500": {
"description": "An internal error occurred. If the problem persists [contact support](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Short-hand name for an error, useful for discriminating between errors with the same status code.",
"example": "bad_request"
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"example": {
"id": "2a1b2d4eafe2bc6ab4cd4d5c6133f513",
"name": "internal_error",
"errors": [
"An unexpected server error has occurred, please try again later."
]
}
}
}
}
}
}
}
},
"/v1/generation/{engine_id}/text-to-image": {
"post": {
"description": "Generate an image from a text prompt. \n### Using SDXL 1.0\nUse `stable-diffusion-xl-1024-v1-0` as the `engine_id` of your request and pass in `height` & `width` as one of the following combinations:\n- 1024x1024 (default)\n- 1152x896\n- 896x1152\n- 1216x832\n- 1344x768\n- 768x1344\n- 1536x640\n- 640x1536 \n\n### SDXL 1.0 Pricing\nWhen specifying 30 steps or fewer, generation costs 0.9 credits.\n\nWhen specifying above 30 steps, generation cost is determiend using the following formula:\n\n `cost = 0.9 * (steps / 30)`\n\n### Using SD 1.6\nSD1.6 is a flexible-resolution base model allowing you to generate non-standard aspect ratios. The model is optimized for a resolution of 512 x 512 pixels. To generate 1 megapixel outputs, we recommend using SDXL 1.0, which is available at the same price.\n\nPass in `stable-diffusion-v1-6` as the `engine_id` of your request and ensure the `height` & `width` you pass in adhere to the following restrictions:\n- No dimension can be less than 320 pixels\n- No dimension can be greater than 1536 pixels\n- Height and width must be specified in increments of 64\n- The default resolution is 512 x 512\n",
"operationId": "textToImage",
"summary": "Text-to-image",
"tags": ["SDXL 1.0 & SD1.6"],
"parameters": [
{
"$ref": "#/components/parameters/engineID"
},
{
"$ref": "#/components/parameters/accept"
},
{
"$ref": "#/components/parameters/organization"
},
{
"$ref": "#/components/parameters/stabilityClientID"
},
{
"$ref": "#/components/parameters/stabilityClientVersion"
}
],
"requestBody": {
"content": {
"application/json": {
"example": {
"cfg_scale": 7,
"height": 512,
"width": 512,
"sampler": "K_DPM_2_ANCESTRAL",
"samples": 1,
"steps": 30,
"text_prompts": [
{
"text": "A lighthouse on a cliff",
"weight": 1
}
]
},
"schema": {
"$ref": "#/components/schemas/TextToImageRequestBody"
}
}
},
"required": true
},
"responses": {
"200": {
"$ref": "#/components/responses/GenerationResponse"
},
"400": {
"$ref": "#/components/responses/400FromGeneration"
},
"401": {
"$ref": "#/components/responses/401"
},
"403": {
"$ref": "#/components/responses/403"
},
"404": {
"$ref": "#/components/responses/404"
},
"500": {
"$ref": "#/components/responses/500"
}
},
"security": [
{
"STABILITY_API_KEY": []
}
],
"x-codeSamples": [
{
"lang": "Python",
"source": "import base64\nimport os\nimport requests\n\nengine_id = \"stable-diffusion-v1-6\"\napi_host = os.getenv('API_HOST', 'https://api.stability.ai')\napi_key = os.getenv(\"STABILITY_API_KEY\")\n\nif api_key is None:\n raise Exception(\"Missing Stability API key.\")\n\nresponse = requests.post(\n f\"{api_host}/v1/generation/{engine_id}/text-to-image\",\n headers={\n \"Content-Type\": \"application/json\",\n \"Accept\": \"application/json\",\n \"Authorization\": f\"Bearer {api_key}\"\n },\n json={\n \"text_prompts\": [\n {\n \"text\": \"A lighthouse on a cliff\"\n }\n ],\n \"cfg_scale\": 7,\n \"height\": 1024,\n \"width\": 1024,\n \"samples\": 1,\n \"steps\": 30,\n },\n)\n\nif response.status_code != 200:\n raise Exception(\"Non-200 response: \" + str(response.text))\n\ndata = response.json()\n\nfor i, image in enumerate(data[\"artifacts\"]):\n with open(f\"./out/v1_txt2img_{i}.png\", \"wb\") as f:\n f.write(base64.b64decode(image[\"base64\"]))\n"
},
{
"label": "TypeScript",
"lang": "Javascript",
"source": "import fetch from 'node-fetch'\nimport fs from 'node:fs'\n\nconst engineId = 'stable-diffusion-v1-6'\nconst apiHost = process.env.API_HOST ?? 'https://api.stability.ai'\nconst apiKey = process.env.STABILITY_API_KEY\n\nif (!apiKey) throw new Error('Missing Stability API key.')\n\nconst response = await fetch(\n `${apiHost}/v1/generation/${engineId}/text-to-image`,\n {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json',\n Accept: 'application/json',\n Authorization: `Bearer ${apiKey}`,\n },\n body: JSON.stringify({\n text_prompts: [\n {\n text: 'A lighthouse on a cliff',\n },\n ],\n cfg_scale: 7,\n height: 1024,\n width: 1024,\n steps: 30,\n samples: 1,\n }),\n }\n)\n\nif (!response.ok) {\n throw new Error(`Non-200 response: ${await response.text()}`)\n}\n\ninterface GenerationResponse {\n artifacts: Array<{\n base64: string\n seed: number\n finishReason: string\n }>\n}\n\nconst responseJSON = (await response.json()) as GenerationResponse\n\nresponseJSON.artifacts.forEach((image, index) => {\n fs.writeFileSync(\n `./out/v1_txt2img_${index}.png`,\n Buffer.from(image.base64, 'base64')\n )\n})\n"
},
{
"lang": "Go",
"source": "package main\n\nimport (\n\t\"bytes\"\n\t\"encoding/base64\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"os\"\n)\n\ntype TextToImageImage struct {\n\tBase64 string `json:\"base64\"`\n\tSeed uint32 `json:\"seed\"`\n\tFinishReason string `json:\"finishReason\"`\n}\n\ntype TextToImageResponse struct {\n\tImages []TextToImageImage `json:\"artifacts\"`\n}\n\nfunc main() {\n\t// Build REST endpoint URL w/ specified engine\n\tengineId := \"stable-diffusion-v1-6\"\n\tapiHost, hasApiHost := os.LookupEnv(\"API_HOST\")\n\tif !hasApiHost {\n\t\tapiHost = \"https://api.stability.ai\"\n\t}\n\treqUrl := apiHost + \"/v1/generation/\" + engineId + \"/text-to-image\"\n\n\t// Acquire an API key from the environment\n\tapiKey, hasAPIKey := os.LookupEnv(\"STABILITY_API_KEY\")\n\tif !hasAPIKey {\n\t\tpanic(\"Missing STABILITY_API_KEY environment variable\")\n\t}\n\n\tvar data = []byte(`{\n\t\t\"text_prompts\": [\n\t\t {\n\t\t\t\"text\": \"A lighthouse on a cliff\"\n\t\t }\n\t\t],\n\t\t\"cfg_scale\": 7,\n\t\t\"height\": 1024,\n\t\t\"width\": 1024,\n\t\t\"samples\": 1,\n\t\t\"steps\": 30\n \t}`)\n\n\treq, _ := http.NewRequest(\"POST\", reqUrl, bytes.NewBuffer(data))\n\treq.Header.Add(\"Content-Type\", \"application/json\")\n\treq.Header.Add(\"Accept\", \"application/json\")\n\treq.Header.Add(\"Authorization\", \"Bearer \"+apiKey)\n\n\t// Execute the request & read all the bytes of the body\n\tres, _ := http.DefaultClient.Do(req)\n\tdefer res.Body.Close()\n\n\tif res.StatusCode != 200 {\n\t\tvar body map[string]interface{}\n\t\tif err := json.NewDecoder(res.Body).Decode(&body); err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\tpanic(fmt.Sprintf(\"Non-200 response: %s\", body))\n\t}\n\n\t// Decode the JSON body\n\tvar body TextToImageResponse\n\tif err := json.NewDecoder(res.Body).Decode(&body); err != nil {\n\t\tpanic(err)\n\t}\n\n\t// Write the images to disk\n\tfor i, image := range body.Images {\n\t\toutFile := fmt.Sprintf(\"./out/v1_txt2img_%d.png\", i)\n\t\tfile, err := os.Create(outFile)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\n\t\timageBytes, err := base64.StdEncoding.DecodeString(image.Base64)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\n\t\tif _, err := file.Write(imageBytes); err != nil {\n\t\t\tpanic(err)\n\t\t}\n\n\t\tif err := file.Close(); err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t}\n}\n"
},
{
"lang": "cURL",
"source": "if [ -z \"$STABILITY_API_KEY\" ]; then\n echo \"STABILITY_API_KEY environment variable is not set\"\n exit 1\nfi\n\nOUTPUT_FILE=./out/v1_txt2img.png\nBASE_URL=${API_HOST:-https://api.stability.ai}\nURL=\"$BASE_URL/v1/generation/stable-diffusion-v1-6/text-to-image\"\n\ncurl -f -sS -X POST \"$URL\" \\\n -H 'Content-Type: application/json' \\\n -H 'Accept: image/png' \\\n -H \"Authorization: Bearer $STABILITY_API_KEY\" \\\n --data-raw '{\n \"text_prompts\": [\n {\n \"text\": \"A lighthouse on a cliff\"\n }\n ],\n \"cfg_scale\": 7,\n \"height\": 1024,\n \"width\": 1024,\n \"samples\": 1,\n \"steps\": 30\n }' \\\n -o \"$OUTPUT_FILE\"\n"
}
]
}
},
"/v1/generation/{engine_id}/image-to-image": {
"post": {
"description": "Produce an image from an existing image using a text prompt. \n### How to control strength of generation\nTo preserve only roughly 35% of the initial image, pass in either `init_image_mode=IMAGE_STRENGTH` and `image_strength=0.35` or `init_image_mode=STEP_SCHEDULE` and `step_schedule_start=0.65`. Both of these are equivalent, however `init_image_mode=STEP_SCHEDULE` also lets you pass in `step_schedule_end`, which can provide an extra level of control for those who need it. For more details, see the specific fields below. \n\n> NOTE: Only **Version 1** engines will work with this endpoint.",
"operationId": "imageToImage",
"summary": "Image-to-image with prompt",
"tags": ["SDXL 1.0 & SD1.6"],
"parameters": [
{
"$ref": "#/components/parameters/engineID"
},
{
"$ref": "#/components/parameters/accept"
},
{
"$ref": "#/components/parameters/organization"
},
{
"$ref": "#/components/parameters/stabilityClientID"
},
{
"$ref": "#/components/parameters/stabilityClientVersion"
}
],
"requestBody": {
"content": {
"multipart/form-data": {
"schema": {
"$ref": "#/components/schemas/ImageToImageRequestBody"
},
"examples": {
"IMAGE_STRENGTH": {
"summary": "Using IMAGE_STRENGTH",
"description": "Request using 35% image_strength",
"value": {
"image_strength": 0.35,
"init_image_mode": "IMAGE_STRENGTH",
"init_image": "",
"text_prompts[0][text]": "A dog space commander",
"text_prompts[0][weight]": 1,
"cfg_scale": 7,
"sampler": "K_DPM_2_ANCESTRAL",
"samples": 3,
"steps": 30
}
},
"STEP_SCHEDULE": {
"summary": "Using STEP_SCHEDULE",
"description": "Equivalent request using step_schedule_start",
"value": {
"step_schedule_start": 0.65,
"init_image_mode": "STEP_SCHEDULE",
"init_image": "",
"text_prompts[0][text]": "A dog space commander",
"text_prompts[0][weight]": 1,
"cfg_scale": 7,
"sampler": "K_DPM_2_ANCESTRAL",
"samples": 3,
"steps": 30
}
}
}
}
},
"required": true
},
"responses": {
"200": {
"$ref": "#/components/responses/GenerationResponse"
},
"400": {
"$ref": "#/components/responses/400FromGeneration"
},
"401": {
"$ref": "#/components/responses/401"
},
"403": {
"$ref": "#/components/responses/403"
},
"404": {
"$ref": "#/components/responses/404"
},
"500": {
"$ref": "#/components/responses/500"
}
},
"security": [
{
"STABILITY_API_KEY": []
}
],
"x-codeSamples": [
{
"lang": "Python",
"source": "import base64\nimport os\nimport requests\n\nengine_id = \"stable-diffusion-v1-6\"\napi_host = os.getenv(\"API_HOST\", \"https://api.stability.ai\")\napi_key = os.getenv(\"STABILITY_API_KEY\")\n\nif api_key is None:\n raise Exception(\"Missing Stability API key.\")\n\nresponse = requests.post(\n f\"{api_host}/v1/generation/{engine_id}/image-to-image\",\n headers={\n \"Accept\": \"application/json\",\n \"Authorization\": f\"Bearer {api_key}\"\n },\n files={\n \"init_image\": open(\"../init_image.png\", \"rb\")\n },\n data={\n \"image_strength\": 0.35,\n \"init_image_mode\": \"IMAGE_STRENGTH\",\n \"text_prompts[0][text]\": \"Galactic dog with a cape\",\n \"cfg_scale\": 7,\n \"samples\": 1,\n \"steps\": 30,\n }\n)\n\nif response.status_code != 200:\n raise Exception(\"Non-200 response: \" + str(response.text))\n\ndata = response.json()\n\nfor i, image in enumerate(data[\"artifacts\"]):\n with open(f\"./out/v1_img2img_{i}.png\", \"wb\") as f:\n f.write(base64.b64decode(image[\"base64\"]))\n"
},
{
"label": "TypeScript",
"lang": "Javascript",
"source": "import fetch from 'node-fetch'\nimport FormData from 'form-data'\nimport fs from 'node:fs'\n\nconst engineId = 'stable-diffusion-v1-6'\nconst apiHost = process.env.API_HOST ?? 'https://api.stability.ai'\nconst apiKey = process.env.STABILITY_API_KEY\n\nif (!apiKey) throw new Error('Missing Stability API key.')\n\n// NOTE: This example is using a NodeJS FormData library.\n// Browsers should use their native FormData class.\n// React Native apps should also use their native FormData class.\nconst formData = new FormData()\nformData.append('init_image', fs.readFileSync('../init_image.png'))\nformData.append('init_image_mode', 'IMAGE_STRENGTH')\nformData.append('image_strength', 0.35)\nformData.append('text_prompts[0][text]', 'Galactic dog wearing a cape')\nformData.append('cfg_scale', 7)\nformData.append('samples', 1)\nformData.append('steps', 30)\n\nconst response = await fetch(\n `${apiHost}/v1/generation/${engineId}/image-to-image`,\n {\n method: 'POST',\n headers: {\n ...formData.getHeaders(),\n Accept: 'application/json',\n Authorization: `Bearer ${apiKey}`,\n },\n body: formData,\n }\n)\n\nif (!response.ok) {\n throw new Error(`Non-200 response: ${await response.text()}`)\n}\n\ninterface GenerationResponse {\n artifacts: Array<{\n base64: string\n seed: number\n finishReason: string\n }>\n}\n\nconst responseJSON = (await response.json()) as GenerationResponse\n\nresponseJSON.artifacts.forEach((image, index) => {\n fs.writeFileSync(\n `out/v1_img2img_${index}.png`,\n Buffer.from(image.base64, 'base64')\n )\n})\n"
},
{
"lang": "Go",
"source": "package main\n\nimport (\n\t\"bytes\"\n\t\"encoding/base64\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"mime/multipart\"\n\t\"net/http\"\n\t\"os\"\n)\n\ntype ImageToImageImage struct {\n\tBase64 string `json:\"base64\"`\n\tSeed uint32 `json:\"seed\"`\n\tFinishReason string `json:\"finishReason\"`\n}\n\ntype ImageToImageResponse struct {\n\tImages []ImageToImageImage `json:\"artifacts\"`\n}\n\nfunc main() {\n\tengineId := \"stable-diffusion-v1-6\"\n\n\t// Build REST endpoint URL\n\tapiHost, hasApiHost := os.LookupEnv(\"API_HOST\")\n\tif !hasApiHost {\n\t\tapiHost = \"https://api.stability.ai\"\n\t}\n\treqUrl := apiHost + \"/v1/generation/\" + engineId + \"/image-to-image\"\n\n\t// Acquire an API key from the environment\n\tapiKey, hasAPIKey := os.LookupEnv(\"STABILITY_API_KEY\")\n\tif !hasAPIKey {\n\t\tpanic(\"Missing STABILITY_API_KEY environment variable\")\n\t}\n\n\tdata := &bytes.Buffer{}\n\twriter := multipart.NewWriter(data)\n\n\t// Write the init image to the request\n\tinitImageWriter, _ := writer.CreateFormField(\"init_image\")\n\tinitImageFile, initImageErr := os.Open(\"../init_image.png\")\n\tif initImageErr != nil {\n\t\tpanic(\"Could not open init_image.png\")\n\t}\n\t_, _ = io.Copy(initImageWriter, initImageFile)\n\n\t// Write the options to the request\n\t_ = writer.WriteField(\"init_image_mode\", \"IMAGE_STRENGTH\")\n\t_ = writer.WriteField(\"image_strength\", \"0.35\")\n\t_ = writer.WriteField(\"text_prompts[0][text]\", \"Galactic dog with a cape\")\n\t_ = writer.WriteField(\"cfg_scale\", \"7\")\n\t_ = writer.WriteField(\"samples\", \"1\")\n\t_ = writer.WriteField(\"steps\", \"30\")\n\twriter.Close()\n\n\t// Execute the request\n\tpayload := bytes.NewReader(data.Bytes())\n\treq, _ := http.NewRequest(\"POST\", reqUrl, payload)\n\treq.Header.Add(\"Content-Type\", writer.FormDataContentType())\n\treq.Header.Add(\"Accept\", \"application/json\")\n\treq.Header.Add(\"Authorization\", \"Bearer \"+apiKey)\n\tres, _ := http.DefaultClient.Do(req)\n\tdefer res.Body.Close()\n\n\tif res.StatusCode != 200 {\n\t\tvar body map[string]interface{}\n\t\tif err := json.NewDecoder(res.Body).Decode(&body); err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\tpanic(fmt.Sprintf(\"Non-200 response: %s\", body))\n\t}\n\n\t// Decode the JSON body\n\tvar body ImageToImageResponse\n\tif err := json.NewDecoder(res.Body).Decode(&body); err != nil {\n\t\tpanic(err)\n\t}\n\n\t// Write the images to disk\n\tfor i, image := range body.Images {\n\t\toutFile := fmt.Sprintf(\"./out/v1_img2img_%d.png\", i)\n\t\tfile, err := os.Create(outFile)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\n\t\timageBytes, err := base64.StdEncoding.DecodeString(image.Base64)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\n\t\tif _, err := file.Write(imageBytes); err != nil {\n\t\t\tpanic(err)\n\t\t}\n\n\t\tif err := file.Close(); err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t}\n}\n"
},
{
"lang": "cURL",
"source": "if [ -z \"$STABILITY_API_KEY\" ]; then\n echo \"STABILITY_API_KEY environment variable is not set\"\n exit 1\nfi\n\nOUTPUT_FILE=./out/v1_img2img.png\nBASE_URL=${API_HOST:-https://api.stability.ai}\nURL=\"$BASE_URL/v1/generation/stable-diffusion-v1-6/image-to-image\"\n\ncurl -f -sS -X POST \"$URL\" \\\n -H 'Content-Type: multipart/form-data' \\\n -H 'Accept: image/png' \\\n -H \"Authorization: Bearer $STABILITY_API_KEY\" \\\n -F 'init_image=@\"../init_image.png\"' \\\n -F 'init_image_mode=IMAGE_STRENGTH' \\\n -F 'image_strength=0.35' \\\n -F 'text_prompts[0][text]=A galactic dog in space' \\\n -F 'cfg_scale=7' \\\n -F 'samples=1' \\\n -F 'steps=30' \\\n -o \"$OUTPUT_FILE\"\n"
}
]
}
},
"/v1/generation/{engine_id}/image-to-image/masking": {
"post": {
"description": "Selectively modify portions of an image using a mask. The `mask` must be the same shape and size as the init image. This endpoint also supports `image` parameters with alpha channels. See below for more details. \n\n> NOTE: Only **Version 1** engines will work with this endpoint.",
"operationId": "masking",
"summary": "Image-to-image with a mask",
"tags": ["SDXL 1.0 & SD1.6"],
"parameters": [
{
"example": "stable-diffusion-xl-1024-v1-0",
"in": "path",
"name": "engine_id",
"required": true,
"schema": {
"type": "string"
}
},
{
"$ref": "#/components/parameters/accept"
},
{
"$ref": "#/components/parameters/organization"
},
{
"$ref": "#/components/parameters/stabilityClientID"
},
{
"$ref": "#/components/parameters/stabilityClientVersion"
}
],
"requestBody": {
"required": true,
"content": {
"multipart/form-data": {
"schema": {
"$ref": "#/components/schemas/MaskingRequestBody"
},
"examples": {
"MASK_IMAGE_BLACK": {
"value": {
"mask_source": "MASK_IMAGE_BLACK",
"init_image": "",
"mask_image": "",
"text_prompts[0][text]": "A dog space commander",
"text_prompts[0][weight]": 1,
"cfg_scale": 7,
"sampler": "K_DPM_2_ANCESTRAL",
"samples": 3,
"steps": 30
}
},
"MASK_IMAGE_WHITE": {
"value": {
"mask_source": "MASK_IMAGE_WHITE",
"init_image": "",
"mask_image": "",
"text_prompts[0][text]": "A dog space commander",
"text_prompts[0][weight]": 1,
"cfg_scale": 7,
"sampler": "K_DPM_2_ANCESTRAL",
"samples": 3,
"steps": 30
}
},
"INIT_IMAGE_ALPHA": {
"value": {
"mask_source": "INIT_IMAGE_ALPHA",
"init_image": "",
"text_prompts[0][text]": "A dog space commander",
"text_prompts[0][weight]": 1,
"cfg_scale": 7,
"sampler": "K_DPM_2_ANCESTRAL",
"samples": 3,
"steps": 30
}
}
}
}
}
},
"responses": {
"200": {
"$ref": "#/components/responses/GenerationResponse"
},
"400": {
"$ref": "#/components/responses/400FromGeneration"
},
"401": {
"$ref": "#/components/responses/401"
},
"403": {
"$ref": "#/components/responses/403"
},
"404": {
"$ref": "#/components/responses/404"
},
"500": {
"$ref": "#/components/responses/500"
}
},
"security": [
{
"STABILITY_API_KEY": []
}
],
"x-codeSamples": [
{
"lang": "Python",
"source": "import base64\nimport os\nimport requests\n\nengine_id = \"stable-diffusion-v1-6\"\napi_host = os.getenv('API_HOST', 'https://api.stability.ai')\napi_key = os.getenv(\"STABILITY_API_KEY\")\n\nif api_key is None:\n raise Exception(\"Missing Stability API key.\")\n\nresponse = requests.post(\n f\"{api_host}/v1/generation/{engine_id}/image-to-image/masking\",\n headers={\n \"Accept\": 'application/json',\n \"Authorization\": f\"Bearer {api_key}\"\n },\n files={\n 'init_image': open(\"../init_image.png\", 'rb'),\n 'mask_image': open(\"../mask_image_black.png\", 'rb'),\n },\n data={\n \"mask_source\": \"MASK_IMAGE_BLACK\",\n \"text_prompts[0][text]\": \"A large spiral galaxy with a bright central bulge and a ring of stars around it\",\n \"cfg_scale\": 7,\n \"clip_guidance_preset\": \"FAST_BLUE\",\n \"samples\": 1,\n \"steps\": 30,\n }\n)\n\nif response.status_code != 200:\n raise Exception(\"Non-200 response: \" + str(response.text))\n\ndata = response.json()\n\nfor i, image in enumerate(data[\"artifacts\"]):\n with open(f\"./out/v1_img2img_masking_{i}.png\", \"wb\") as f:\n f.write(base64.b64decode(image[\"base64\"]))\n"
},
{
"label": "TypeScript",
"lang": "Javascript",
"source": "import fetch from 'node-fetch'\nimport FormData from 'form-data'\nimport fs from 'node:fs'\n\nconst engineId = 'stable-diffusion-v1-6'\nconst apiHost = process.env.API_HOST ?? 'https://api.stability.ai'\nconst apiKey = process.env.STABILITY_API_KEY\n\nif (!apiKey) throw new Error('Missing Stability API key.')\n\n// NOTE: This example is using a NodeJS FormData library. Browser\n// implementations should use their native FormData class. React Native\n// implementations should also use their native FormData class.\nconst formData = new FormData()\nformData.append('init_image', fs.readFileSync('../init_image.png'))\nformData.append('mask_image', fs.readFileSync('../mask_image_black.png'))\nformData.append('mask_source', 'MASK_IMAGE_BLACK')\nformData.append(\n 'text_prompts[0][text]',\n 'A large spiral galaxy with a bright central bulge and a ring of stars around it'\n)\nformData.append('cfg_scale', '7')\nformData.append('clip_guidance_preset', 'FAST_BLUE')\nformData.append('samples', 1)\nformData.append('steps', 30)\n\nconst response = await fetch(\n `${apiHost}/v1/generation/${engineId}/image-to-image/masking`,\n {\n method: 'POST',\n headers: {\n ...formData.getHeaders(),\n Accept: 'application/json',\n Authorization: `Bearer ${apiKey}`,\n },\n body: formData,\n }\n)\n\nif (!response.ok) {\n throw new Error(`Non-200 response: ${await response.text()}`)\n}\n\ninterface GenerationResponse {\n artifacts: Array<{\n base64: string\n seed: number\n finishReason: string\n }>\n}\n\nconst responseJSON = (await response.json()) as GenerationResponse\n\nresponseJSON.artifacts.forEach((image, index) => {\n fs.writeFileSync(\n `out/v1_img2img_masking_${index}.png`,\n Buffer.from(image.base64, 'base64')\n )\n})\n"
},
{
"lang": "Go",
"source": "package main\n\nimport (\n\t\"bytes\"\n\t\"encoding/base64\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"mime/multipart\"\n\t\"net/http\"\n\t\"os\"\n)\n\ntype MaskingImage struct {\n\tBase64 string `json:\"base64\"`\n\tSeed uint32 `json:\"seed\"`\n\tFinishReason string `json:\"finishReason\"`\n}\n\ntype MaskingResponse struct {\n\tImages []MaskingImage `json:\"artifacts\"`\n}\n\nfunc main() {\n\tengineId := \"stable-diffusion-v1-6\"\n\n\t// Build REST endpoint URL\n\tapiHost, hasApiHost := os.LookupEnv(\"API_HOST\")\n\tif !hasApiHost {\n\t\tapiHost = \"https://api.stability.ai\"\n\t}\n\treqUrl := apiHost + \"/v1/generation/\" + engineId + \"/image-to-image/masking\"\n\n\t// Acquire an API key from the environment\n\tapiKey, hasAPIKey := os.LookupEnv(\"STABILITY_API_KEY\")\n\tif !hasAPIKey {\n\t\tpanic(\"Missing STABILITY_API_KEY environment variable\")\n\t}\n\n\tdata := &bytes.Buffer{}\n\twriter := multipart.NewWriter(data)\n\n\t// Write the init image to the request\n\tinitImageWriter, _ := writer.CreateFormField(\"init_image\")\n\tinitImageFile, initImageErr := os.Open(\"../init_image.png\")\n\tif initImageErr != nil {\n\t\tpanic(\"Could not open init_image.png\")\n\t}\n\t_, _ = io.Copy(initImageWriter, initImageFile)\n\n\t// Write the mask image to the request\n\tmaskImageWriter, _ := writer.CreateFormField(\"mask_image\")\n\tmaskImageFile, maskImageErr := os.Open(\"../mask_image_black.png\")\n\tif maskImageErr != nil {\n\t\tpanic(\"Could not open mask_image_white.png\")\n\t}\n\t_, _ = io.Copy(maskImageWriter, maskImageFile)\n\n\t// Write the options to the request\n\t_ = writer.WriteField(\"mask_source\", \"MASK_IMAGE_BLACK\")\n\t_ = writer.WriteField(\"text_prompts[0][text]\", \"A large spiral galaxy with a bright central bulge and a ring of stars around it\")\n\t_ = writer.WriteField(\"cfg_scale\", \"7\")\n\t_ = writer.WriteField(\"clip_guidance_preset\", \"FAST_BLUE\")\n\t_ = writer.WriteField(\"samples\", \"1\")\n\t_ = writer.WriteField(\"steps\", \"30\")\n\twriter.Close()\n\n\t// Execute the request & read all the bytes of the response\n\tpayload := bytes.NewReader(data.Bytes())\n\treq, _ := http.NewRequest(\"POST\", reqUrl, payload)\n\treq.Header.Add(\"Content-Type\", writer.FormDataContentType())\n\treq.Header.Add(\"Accept\", \"application/json\")\n\treq.Header.Add(\"Authorization\", \"Bearer \"+apiKey)\n\tres, _ := http.DefaultClient.Do(req)\n\tdefer res.Body.Close()\n\n\tif res.StatusCode != 200 {\n\t\tvar body map[string]interface{}\n\t\tif err := json.NewDecoder(res.Body).Decode(&body); err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\tpanic(fmt.Sprintf(\"Non-200 response: %s\", body))\n\t}\n\n\t// Decode the JSON body\n\tvar body MaskingResponse\n\tif err := json.NewDecoder(res.Body).Decode(&body); err != nil {\n\t\tpanic(err)\n\t}\n\n\t// Write the images to disk\n\tfor i, image := range body.Images {\n\t\toutFile := fmt.Sprintf(\"./out/v1_img2img_masking_%d.png\", i)\n\t\tfile, err := os.Create(outFile)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\n\t\timageBytes, err := base64.StdEncoding.DecodeString(image.Base64)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\n\t\tif _, err := file.Write(imageBytes); err != nil {\n\t\t\tpanic(err)\n\t\t}\n\n\t\tif err := file.Close(); err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t}\n}\n"
},
{
"lang": "cURL",
"source": "#!/bin/sh\n\nset -e\n\nif [ -z \"$STABILITY_API_KEY\" ]; then\n echo \"STABILITY_API_KEY environment variable is not set\"\n exit 1\nfi\n\nOUTPUT_FILE=./out/v1_img2img_masking.png\nBASE_URL=${API_HOST:-https://api.stability.ai}\nURL=\"$BASE_URL/v1/generation/stable-diffusion-v1-6/image-to-image/masking\"\n\ncurl -f -sS -X POST \"$URL\" \\\n -H 'Content-Type: multipart/form-data' \\\n -H 'Accept: image/png' \\\n -H \"Authorization: Bearer $STABILITY_API_KEY\" \\\n -F 'init_image=@\"../init_image.png\"' \\\n -F 'mask_image=@\"../mask_image_black.png\"' \\\n -F 'mask_source=MASK_IMAGE_BLACK' \\\n -F 'text_prompts[0][text]=A large spiral galaxy with a bright central bulge and a ring of stars around it' \\\n -F 'cfg_scale=7' \\\n -F 'clip_guidance_preset=FAST_BLUE' \\\n -F 'samples=1' \\\n -F 'steps=30' \\\n -o \"$OUTPUT_FILE\"\n"
}
]
}
},
"/v1/engines/list": {
"get": {
"description": "List all engines available to your organization/user",
"operationId": "listEngines",
"summary": "List engines",
"tags": ["Engines"],
"parameters": [
{
"$ref": "#/components/parameters/organization"
},
{
"$ref": "#/components/parameters/stabilityClientID"
},
{
"$ref": "#/components/parameters/stabilityClientVersion"
}
],
"responses": {
"200": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ListEnginesResponseBody"
}
}
},
"description": "OK response."
},
"401": {
"$ref": "#/components/responses/401"
},
"500": {
"$ref": "#/components/responses/500"
}
},
"security": [
{
"STABILITY_API_KEY": []
}
],
"x-codeSamples": [
{
"lang": "Python",
"source": "import os\nimport requests\n\napi_host = os.getenv('API_HOST', 'https://api.stability.ai')\nurl = f\"{api_host}/v1/engines/list\"\n\napi_key = os.getenv(\"STABILITY_API_KEY\")\nif api_key is None:\n raise Exception(\"Missing Stability API key.\")\n\nresponse = requests.get(url, headers={\n \"Authorization\": f\"Bearer {api_key}\"\n})\n\nif response.status_code != 200:\n raise Exception(\"Non-200 response: \" + str(response.text))\n\n# Do something with the payload...\npayload = response.json()\n\n"
},
{
"label": "TypeScript",
"lang": "Javascript",
"source": "import fetch from 'node-fetch'\n\nconst apiHost = process.env.API_HOST ?? 'https://api.stability.ai'\nconst url = `${apiHost}/v1/engines/list`\n\nconst apiKey = process.env.STABILITY_API_KEY\nif (!apiKey) throw new Error('Missing Stability API key.')\n\nconst response = await fetch(url, {\n method: 'GET',\n headers: {\n Authorization: `Bearer ${apiKey}`,\n },\n})\n\nif (!response.ok) {\n throw new Error(`Non-200 response: ${await response.text()}`)\n}\n\ninterface Payload {\n engines: Array<{\n id: string\n name: string\n description: string\n type: string\n }>\n}\n\n// Do something with the payload...\nconst payload = (await response.json()) as Payload\n"
},
{
"lang": "Go",
"source": "package main\n\nimport (\n\t\"io\"\n\t\"net/http\"\n\t\"os\"\n)\n\nfunc main() {\n\t// Build REST endpoint URL\n\tapiHost, hasApiHost := os.LookupEnv(\"API_HOST\")\n\tif !hasApiHost {\n\t\tapiHost = \"https://api.stability.ai\"\n\t}\n\treqUrl := apiHost + \"/v1/engines/list\"\n\n\t// Acquire an API key from the environment\n\tapiKey, hasAPIKey := os.LookupEnv(\"STABILITY_API_KEY\")\n\tif !hasAPIKey {\n\t\tpanic(\"Missing STABILITY_API_KEY environment variable\")\n\t}\n\n\t// Execute the request & read all the bytes of the response\n\treq, _ := http.NewRequest(\"GET\", reqUrl, nil)\n\treq.Header.Add(\"Authorization\", \"Bearer \"+apiKey)\n\tres, _ := http.DefaultClient.Do(req)\n\tdefer res.Body.Close()\n\tbody, _ := io.ReadAll(res.Body)\n\n\tif res.StatusCode != 200 {\n\t\tpanic(\"Non-200 response: \" + string(body))\n\t}\n\n\t// Do something with the payload...\n\t// payload := string(body)\n}\n"
},
{
"lang": "cURL",
"source": "if [ -z \"$STABILITY_API_KEY\" ]; then\n echo \"STABILITY_API_KEY environment variable is not set\"\n exit 1\nfi\n\nBASE_URL=${API_HOST:-https://api.stability.ai}\nURL=\"$BASE_URL/v1/engines/list\"\n\ncurl -f -sS \"$URL\" \\\n -H 'Accept: application/json' \\\n -H \"Authorization: Bearer $STABILITY_API_KEY\"\n"
}
]
}
},
"/v1/user/account": {
"get": {
"description": "Get information about the account associated with the provided API key",
"operationId": "userAccount",
"summary": "Account details",
"tags": ["User"],
"responses": {
"200": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/AccountResponseBody"
}
}
},
"description": "OK response."
},
"401": {
"$ref": "#/components/responses/401"
},
"500": {
"$ref": "#/components/responses/500"
}
},
"security": [
{
"STABILITY_API_KEY": []
}
],
"x-codeSamples": [
{
"lang": "Python",
"source": "import os\nimport requests\n\napi_host = os.getenv('API_HOST', 'https://api.stability.ai')\nurl = f\"{api_host}/v1/user/account\"\n\napi_key = os.getenv(\"STABILITY_API_KEY\")\nif api_key is None:\n raise Exception(\"Missing Stability API key.\")\n\nresponse = requests.get(url, headers={\n \"Authorization\": f\"Bearer {api_key}\"\n})\n\nif response.status_code != 200:\n raise Exception(\"Non-200 response: \" + str(response.text))\n\n# Do something with the payload...\npayload = response.json()\n\n"
},
{
"label": "TypeScript",
"lang": "Javascript",
"source": "import fetch from 'node-fetch'\n\nconst apiHost = process.env.API_HOST ?? 'https://api.stability.ai'\nconst url = `${apiHost}/v1/user/account`\n\nconst apiKey = process.env.STABILITY_API_KEY\nif (!apiKey) throw new Error('Missing Stability API key.')\n\nconst response = await fetch(url, {\n method: 'GET',\n headers: {\n Authorization: `Bearer ${apiKey}`,\n },\n})\n\nif (!response.ok) {\n throw new Error(`Non-200 response: ${await response.text()}`)\n}\n\ninterface User {\n id: string\n profile_picture: string\n email: string\n organizations?: Array<{\n id: string\n name: string\n role: string\n is_default: boolean\n }>\n}\n\n// Do something with the user...\nconst user = (await response.json()) as User\n"
},
{
"lang": "Go",
"source": "package main\n\nimport (\n\t\"io\"\n\t\"net/http\"\n\t\"os\"\n)\n\nfunc main() {\n\t// Build REST endpoint URL\n\tapiHost, hasApiHost := os.LookupEnv(\"API_HOST\")\n\tif !hasApiHost {\n\t\tapiHost = \"https://api.stability.ai\"\n\t}\n\treqUrl := apiHost + \"/v1/user/account\"\n\n\t// Acquire an API key from the environment\n\tapiKey, hasAPIKey := os.LookupEnv(\"STABILITY_API_KEY\")\n\tif !hasAPIKey {\n\t\tpanic(\"Missing STABILITY_API_KEY environment variable\")\n\t}\n\n\t// Build the request\n\treq, _ := http.NewRequest(\"GET\", reqUrl, nil)\n\treq.Header.Add(\"Authorization\", \"Bearer \"+apiKey)\n\n\t// Execute the request\n\tres, _ := http.DefaultClient.Do(req)\n\tdefer res.Body.Close()\n\tbody, _ := io.ReadAll(res.Body)\n\n\tif res.StatusCode != 200 {\n\t\tpanic(\"Non-200 response: \" + string(body))\n\t}\n\n\t// Do something with the payload...\n\t// payload := string(body)\n}\n"
},
{
"lang": "cURL",
"source": "if [ -z \"$STABILITY_API_KEY\" ]; then\n echo \"STABILITY_API_KEY environment variable is not set\"\n exit 1\nfi\n\n# Determine the URL to use for the request\nBASE_URL=${API_HOST:-https://api.stability.ai}\nURL=\"$BASE_URL/v1/user/account\"\n\ncurl -f -sS \"$URL\" \\\n -H 'Accept: application/json' \\\n -H \"Authorization: Bearer $STABILITY_API_KEY\"\n"
}
]
}
},
"/v1/user/balance": {
"get": {
"description": "Get the credit balance of the account/organization associated with the API key",
"operationId": "userBalance",
"summary": "Account balance",
"tags": ["User"],
"parameters": [
{
"$ref": "#/components/parameters/organization"
},
{
"$ref": "#/components/parameters/stabilityClientID"
},
{
"$ref": "#/components/parameters/stabilityClientVersion"
}
],
"responses": {
"200": {
"content": {
"application/json": {
"example": {
"credits": 0.6336833840314097
},
"schema": {
"$ref": "#/components/schemas/BalanceResponseBody"
}
}
},
"description": "OK response."
},
"401": {
"$ref": "#/components/responses/401"
},
"500": {
"$ref": "#/components/responses/500"
}
},
"security": [
{
"STABILITY_API_KEY": []
}
],
"x-codeSamples": [
{
"lang": "Python",
"source": "import os\nimport requests\n\napi_host = os.getenv('API_HOST', 'https://api.stability.ai')\nurl = f\"{api_host}/v1/user/balance\"\n\napi_key = os.getenv(\"STABILITY_API_KEY\")\nif api_key is None:\n raise Exception(\"Missing Stability API key.\")\n\nresponse = requests.get(url, headers={\n \"Authorization\": f\"Bearer {api_key}\"\n})\n\nif response.status_code != 200:\n raise Exception(\"Non-200 response: \" + str(response.text))\n\n# Do something with the payload...\npayload = response.json()\n\n"
},
{
"label": "TypeScript",
"lang": "Javascript",
"source": "import fetch from 'node-fetch'\n\nconst apiHost = process.env.API_HOST ?? 'https://api.stability.ai'\nconst url = `${apiHost}/v1/user/balance`\n\nconst apiKey = process.env.STABILITY_API_KEY\nif (!apiKey) throw new Error('Missing Stability API key.')\n\nconst response = await fetch(url, {\n method: 'GET',\n headers: {\n Authorization: `Bearer ${apiKey}`,\n },\n})\n\nif (!response.ok) {\n throw new Error(`Non-200 response: ${await response.text()}`)\n}\n\ninterface Balance {\n credits: number\n}\n\n// Do something with the balance...\nconst balance = (await response.json()) as Balance\n"
},
{
"lang": "Go",
"source": "package main\n\nimport (\n\t\"io\"\n\t\"net/http\"\n\t\"os\"\n)\n\nfunc main() {\n\t// Build REST endpoint URL\n\tapiHost, hasApiHost := os.LookupEnv(\"API_HOST\")\n\tif !hasApiHost {\n\t\tapiHost = \"https://api.stability.ai\"\n\t}\n\treqUrl := apiHost + \"/v1/user/balance\"\n\n\t// Acquire an API key from the environment\n\tapiKey, hasAPIKey := os.LookupEnv(\"STABILITY_API_KEY\")\n\tif !hasAPIKey {\n\t\tpanic(\"Missing STABILITY_API_KEY environment variable\")\n\t}\n\n\t// Build the request\n\treq, _ := http.NewRequest(\"GET\", reqUrl, nil)\n\treq.Header.Add(\"Authorization\", \"Bearer \"+apiKey)\n\n\t// Execute the request\n\tres, _ := http.DefaultClient.Do(req)\n\tdefer res.Body.Close()\n\tbody, _ := io.ReadAll(res.Body)\n\n\tif res.StatusCode != 200 {\n\t\tpanic(\"Non-200 response: \" + string(body))\n\t}\n\n\t// Do something with the payload...\n\t// payload := string(body)\n}\n"
},
{
"lang": "cURL",
"source": "if [ -z \"$STABILITY_API_KEY\" ]; then\n echo \"STABILITY_API_KEY environment variable is not set\"\n exit 1\nfi\n\n# Determine the URL to use for the request\nBASE_URL=${API_HOST:-https://api.stability.ai}\nURL=\"$BASE_URL/v1/user/balance\"\n\ncurl -f -sS \"$URL\" \\\n -H 'Content-Type: application/json' \\\n -H \"Authorization: Bearer $STABILITY_API_KEY\"\n"
}
]
}
}
},
"components": {
"schemas": {
"GenerationID": {
"type": "string",
"minLength": 64,
"maxLength": 64,
"description": "The `id` of a generation, typically used for async generations, that can be used to check the status of the generation or retrieve the result.",
"example": "a6dc6c6e20acda010fe14d71f180658f2896ed9b4ec25aa99a6ff06c796987c4"
},
"ImageToVideoRequest": {
"type": "object",
"properties": {
"image": {
"type": "string",
"description": "The source image used in the video generation process.\n\nSupported Formats:\n- jpeg\n- png\n\nSupported Dimensions:\n- 1024x576\n- 576x1024\n- 768x768",
"format": "binary",
"example": "./some/image.png"
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "A specific value that is used to guide the 'randomness' of the generation. (Omit this parameter or pass `0` to use a random seed.)"
},
"cfg_scale": {
"type": "number",
"minimum": 0,
"maximum": 10,
"default": 1.8,
"description": "How strongly the video sticks to the original image. Use lower values to allow the model more freedom to make changes and higher values to correct motion distortions."
},
"motion_bucket_id": {
"type": "number",
"minimum": 1,
"maximum": 255,
"default": 127,
"description": "Lower values generally result in less motion in the output video, while higher values generally result in more motion. This parameter corresponds to the motion_bucket_id parameter from the [paper](https://static1.squarespace.com/static/6213c340453c3f502425776e/t/655ce779b9d47d342a93c890/1700587395994/stable_video_diffusion.pdf)."
}
},
"required": ["image"]
},
"ContentModerationResponse": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"description": "A unique identifier associated with this error. Please include this in any [support tickets](https://stabilityplatform.freshdesk.com/support/tickets/new) \nyou file, as it will greatly assist us in diagnosing the root cause of the problem.",
"example": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"
},
"name": {
"type": "string",
"minLength": 1,
"description": "Our content moderation system has flagged some part of your request and subsequently denied it. You were not charged for this request. While this may at times be frustrating, it is necessary to maintain the integrity of our platform and ensure a safe experience for all users.\n\nIf you would like to provide feedback, please use the [Support Form](https://stabilityplatform.freshdesk.com/support/tickets/new).",
"enum": ["content_moderation"]
},
"errors": {
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"description": "One or more error messages indicating what went wrong.",
"example": ["some-field: is required"]
}
},
"required": ["id", "name", "errors"],
"description": "Your request was flagged by our content moderation system.",
"example": {
"id": "ed14db44362126aab3cbd25cca51ffe3",
"name": "content_moderation",
"errors": [
"Your request was flagged by our content moderation system, as a result your request was denied and you were not charged."
]
}
},
"InpaintingSearchModeRequestBody": {
"type": "object",
"properties": {
"mode": {
"type": "string",
"enum": ["search"],
"description": "Controls how the model decides which areas to inpaint and which areas to leave alone. \n\nSpecifying `mask` requires:\n - Provide an explicit mask image in the `mask` parameter\n - Use the alpha channel of the `image` parameter as the mask\n \nSpecifying `search` requires:\n - Provide a small description of what to inpaint in the `search_prompt` parameter"
},
"search_prompt": {
"type": "string",
"description": "Short description of what to inpaint in the `image`.",
"example": "glasses"
},
"image": {
"type": "string",
"description": "The image you wish to inpaint.\n\nSupported Formats:\n- jpeg\n- png\n- webp\n\nValidation Rules:\n- Every side must be at least 64 pixels\n- Total pixel count must be between 4,096 and 9,437,184 pixels",
"format": "binary",
"example": "./some/image.png"
},
"prompt": {
"type": "string",
"minLength": 1,
"maxLength": 10000,
"description": "What you wish to see in the output image. A strong, descriptive prompt that clearly defines \nelements, colors, and subjects will lead to better results. \n\nTo control the weight of a given word use the format `(word:weight)`, \nwhere `word` is the word you'd like to control the weight of and `weight` \nis a value between 0 and 1. For example: `The sky was a crisp (blue:0.3) and (green:0.8)`\nwould convey a sky that was blue and green, but more green than blue."
},
"negative_prompt": {
"type": "string",
"maxLength": 10000,
"description": "A blurb of text describing what you **do not** wish to see in the output image. \nThis is an advanced feature."
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "A specific value that is used to guide the 'randomness' of the generation. (Omit this parameter or pass `0` to use a random seed.)"
},
"output_format": {
"type": "string",
"enum": ["jpeg", "png", "webp"],
"default": "png",
"description": "Dictates the `content-type` of the generated image."
}
},
"required": ["image", "prompt", "mode", "search_prompt"]
},
"InpaintingMaskingModeRequestBody": {
"type": "object",
"properties": {
"mode": {
"type": "string",
"enum": ["mask"],
"description": "Controls how the model decides which areas to inpaint and which areas to leave alone. \n\nSpecifying `mask` requires:\n - Provide an explicit mask image in the `mask` parameter\n - Use the alpha channel of the `image` parameter as the mask\n \nSpecifying `search` requires:\n - Provide a small description of what to inpaint in the `search_prompt` parameter"
},
"mask": {
"type": "string",
"description": "Controls the strength of the inpainting process on a per-pixel basis, either via a \nsecond image (passed into this parameter) or via the alpha channel of the `image` parameter.\n\n**Passing in a Mask** \n\nThe image passed to this parameter should be a black and white image that represents, \nat any pixel, the strength of inpainting based on how dark or light the given pixel is. \nCompletely black pixels represent no inpainting strength while completely white pixels \nrepresent maximum strength.\n\nIn the event the mask is a different size than the `image` parameter, it will be automatically resized.\n\n**Alpha Channel Support**\n\nIf you don't provide an explicit mask, one will be derived from the alpha channel of the `image` parameter.\nTransparent pixels will be inpainted while opaque pixels will be preserved.\n\nIn the event an `image` with an alpha channel is provided along with a `mask`, the `mask` will take precedence.",
"format": "binary",
"example": "./some/image.png"
},
"image": {
"type": "string",
"description": "The image you wish to inpaint.\n\nSupported Formats:\n- jpeg\n- png\n- webp\n\nValidation Rules:\n- Every side must be at least 64 pixels\n- Total pixel count must be between 4,096 and 9,437,184 pixels",
"format": "binary",
"example": "./some/image.png"
},
"prompt": {
"type": "string",
"minLength": 1,
"maxLength": 10000,
"description": "What you wish to see in the output image. A strong, descriptive prompt that clearly defines \nelements, colors, and subjects will lead to better results. \n\nTo control the weight of a given word use the format `(word:weight)`, \nwhere `word` is the word you'd like to control the weight of and `weight` \nis a value between 0 and 1. For example: `The sky was a crisp (blue:0.3) and (green:0.8)`\nwould convey a sky that was blue and green, but more green than blue."
},
"negative_prompt": {
"type": "string",
"maxLength": 10000,
"description": "A blurb of text describing what you **do not** wish to see in the output image. \nThis is an advanced feature."
},
"seed": {
"type": "number",
"minimum": 0,
"maximum": 4294967294,
"default": 0,
"description": "A specific value that is used to guide the 'randomness' of the generation. (Omit this parameter or pass `0` to use a random seed.)"
},
"output_format": {
"type": "string",
"enum": ["jpeg", "png", "webp"],
"default": "png",
"description": "Dictates the `content-type` of the generated image."
}
},
"required": ["image", "prompt", "mode"]
},
"StabilityClientID": {
"type": "string",
"maxLength": 256,
"description": "The name of your application, used to help us communicate app-specific debugging or moderation issues to you.",
"example": "my-awesome-app"
},
"StabilityClientUserID": {
"type": "string",
"maxLength": 256,
"description": "A unique identifier for your end user. Used to help us communicate user-specific debugging or moderation issues to you. Feel free to obfuscate this value to protect user privacy.",
"example": "DiscordUser#9999"
},
"StabilityClientVersion": {
"type": "string",
"maxLength": 256,
"description": "The version of your application, used to help us communicate version-specific debugging or moderation issues to you.",
"example": "1.2.1"
},
"Creativity": {
"type": "number",
"minimum": 0.2,
"maximum": 0.5,
"default": 0.35,
"description": "Controls the likelihood of creating additional details not heavily conditioned by the init image."
},
"Engine": {
"type": "object",
"properties": {
"description": {
"type": "string"
},
"id": {
"type": "string",
"x-go-name": "ID",
"description": "Unique identifier for the engine",
"example": "stable-diffusion-v1-6"
},
"name": {
"type": "string",
"description": "Name of the engine",
"example": "Stable Diffusion XL v1.0"
},
"type": {
"type": "string",
"description": "The type of content this engine produces",
"example": "PICTURE",
"enum": [
"AUDIO",
"CLASSIFICATION",
"PICTURE",
"STORAGE",
"TEXT",
"VIDEO"
]
}
},
"required": ["id", "name", "description", "type"]
},
"Error": {
"type": "object",
"x-go-name": "RESTError",
"properties": {
"id": {
"x-go-name": "ID",
"type": "string",
"description": "A unique identifier for this particular occurrence of the problem.",
"example": "296a972f-666a-44a1-a3df-c9c28a1f56c0"
},
"name": {
"type": "string",
"description": "The short-name of this class of errors e.g. `bad_request`.",
"example": "bad_request"
},
"message": {
"type": "string",
"description": "A human-readable explanation specific to this occurrence of the problem.",
"example": "Header parameter Authorization is required, but not found"
}
},
"required": ["name", "id", "message", "status"]
},
"CfgScale": {
"type": "number",
"description": "How strictly the diffusion process adheres to the prompt text (higher values keep your image closer to your prompt)",
"default": 7,
"example": 7,
"minimum": 0,
"maximum": 35
},
"ClipGuidancePreset": {
"type": "string",
"default": "NONE",
"example": "FAST_BLUE",
"enum": [
"FAST_BLUE",
"FAST_GREEN",
"NONE",
"SIMPLE",
"SLOW",
"SLOWER",
"SLOWEST"
]
},
"UpscaleImageHeight": {
"x-go-type": "uint64",
"type": "integer",
"description": "Desired height of the output image. Only one of `width` or `height` may be specified.",
"minimum": 512
},
"UpscaleImageWidth": {
"x-go-type": "uint64",
"type": "integer",
"description": "Desired width of the output image. Only one of `width` or `height` may be specified.",
"minimum": 512
},
"DiffuseImageHeight": {
"x-go-type": "uint64",
"type": "integer",
"description": "Height of the image to generate, in pixels, in an increment divisible by 64.",
"multipleOf": 64,
"default": 512,
"example": 512,
"minimum": 128
},
"DiffuseImageWidth": {
"x-go-type": "uint64",
"type": "integer",
"description": "Width of the image to generate, in pixels, in an increment divisible by 64.",
"multipleOf": 64,
"default": 512,
"example": 512,
"minimum": 128
},
"Sampler": {
"type": "string",
"description": "Which sampler to use for the diffusion process. If this value is omitted we'll automatically select an appropriate sampler for you.",
"example": "K_DPM_2_ANCESTRAL",
"enum": [
"DDIM",
"DDPM",
"K_DPMPP_2M",
"K_DPMPP_2S_ANCESTRAL",
"K_DPM_2",
"K_DPM_2_ANCESTRAL",
"K_EULER",
"K_EULER_ANCESTRAL",
"K_HEUN",
"K_LMS"
]
},
"Samples": {
"x-go-type": "uint64",
"type": "integer",
"description": "Number of images to generate",
"default": 1,
"example": 1,
"minimum": 1,
"maximum": 10
},
"Seed": {
"type": "integer",
"x-go-type": "uint32",
"description": "Random noise seed (omit this option or use `0` for a random seed)",
"default": 0,
"example": 0,
"minimum": 0,
"maximum": 4294967295
},
"Steps": {
"x-go-type": "uint64",
"type": "integer",
"description": "Number of diffusion steps to run.",
"default": 30,
"example": 50,
"minimum": 10,
"maximum": 50
},
"Extras": {
"type": "object",
"description": "Extra parameters passed to the engine.\nThese parameters are used for in-development or experimental features and may change\nwithout warning, so please use with caution."
},
"StylePreset": {
"type": "string",
"enum": [
"enhance",
"anime",
"photographic",
"digital-art",
"comic-book",
"fantasy-art",
"line-art",
"analog-film",
"neon-punk",
"isometric",
"low-poly",
"origami",
"modeling-compound",
"cinematic",
"3d-model",
"pixel-art",
"tile-texture"
],
"description": "Pass in a style preset to guide the image model towards a particular style.\nThis list of style presets is subject to change."
},
"TextPrompt": {
"type": "object",
"properties": {
"text": {
"type": "string",
"description": "The prompt itself",
"example": "A lighthouse on a cliff",
"maxLength": 2000
},
"weight": {
"type": "number",
"description": "Weight of the prompt (use negative numbers for negative prompts)",
"example": 0.8167237,
"format": "float"
}
},
"description": "Text prompt for image generation",
"required": ["text"]
},
"TextPromptsForTextToImage": {
"title": "TextPrompts",
"type": "array",
"items": {
"$ref": "#/components/schemas/TextPrompt"
},
"minItems": 1,
"description": "An array of text prompts to use for generation.\n\nGiven a text prompt with the text `A lighthouse on a cliff` and a weight of `0.5`, it would be represented as:\n\n```\n\"text_prompts\": [\n {\n \"text\": \"A lighthouse on a cliff\",\n \"weight\": 0.5\n }\n]\n```"
},
"TextPrompts": {
"description": "An array of text prompts to use for generation.\n\nDue to how arrays are represented in `multipart/form-data` requests, prompts must adhere to the format `text_prompts[index][text|weight]`,\nwhere `index` is some integer used to tie the text and weight together. While `index` does not have to be sequential, duplicate entries \nwill override previous entries, so it is recommended to use sequential indices.\n\nGiven a text prompt with the text `A lighthouse on a cliff` and a weight of `0.5`, it would be represented as:\n```\ntext_prompts[0][text]: \"A lighthouse on a cliff\"\ntext_prompts[0][weight]: 0.5\n```\n\nTo add another prompt to that request simply provide the values under a new `index`:\n\n```\ntext_prompts[0][text]: \"A lighthouse on a cliff\"\ntext_prompts[0][weight]: 0.5\ntext_prompts[1][text]: \"land, ground, dirt, grass\"\ntext_prompts[1][weight]: -0.9\n```",
"type": "array",
"items": {
"$ref": "#/components/schemas/TextPrompt"
},
"minItems": 1
},
"InputImage": {
"x-go-type": "[]byte",
"type": "string",
"description": "The image to upscale using ESRGAN.",
"example": "",
"format": "binary"
},
"InitImage": {
"x-go-type": "[]byte",
"type": "string",
"description": "Image used to initialize the diffusion process, in lieu of random noise.",
"example": "",
"format": "binary"
},
"InitImageStrength": {
"type": "number",
"description": "How much influence the `init_image` has on the diffusion process. Values close to `1` will yield images very similar to the `init_image` while values close to `0` will yield images wildly different than the `init_image`. The behavior of this is meant to mirror DreamStudio's \"Image Strength\" slider.
This parameter is just an alternate way to set `step_schedule_start`, which is done via the calculation `1 - image_strength`. For example, passing in an Image Strength of 35% (`0.35`) would result in a `step_schedule_start` of `0.65`.\n",
"example": 0.4,
"minimum": 0,
"maximum": 1,
"format": "float",
"default": 0.35
},
"InitImageMode": {
"type": "string",
"description": "Whether to use `image_strength` or `step_schedule_*` to control how much influence the `init_image` has on the result.",
"enum": ["IMAGE_STRENGTH", "STEP_SCHEDULE"],
"default": "IMAGE_STRENGTH"
},
"StepScheduleStart": {
"type": "number",
"description": "Skips a proportion of the start of the diffusion steps, allowing the init_image to influence the final generated image. Lower values will result in more influence from the init_image, while higher values will result in more influence from the diffusion steps. (e.g. a value of `0` would simply return you the init_image, where a value of `1` would return you a completely different image.)",
"default": 0.65,
"example": 0.4,
"minimum": 0,
"maximum": 1
},
"StepScheduleEnd": {
"type": "number",
"description": "Skips a proportion of the end of the diffusion steps, allowing the init_image to influence the final generated image. Lower values will result in more influence from the init_image, while higher values will result in more influence from the diffusion steps.",
"example": 0.01,
"minimum": 0,
"maximum": 1
},
"MaskImage": {
"x-go-type": "[]byte",
"type": "string",
"description": "Optional grayscale mask that allows for influence over which pixels are eligible for diffusion and at what strength. Must be the same dimensions as the `init_image`. Use the `mask_source` option to specify whether the white or black pixels should be inpainted.",
"example": "",
"format": "binary"
},
"MaskSource": {
"type": "string",
"description": "For any given pixel, the mask determines the strength of generation on a linear scale. This parameter determines where to source the mask from:\n- `MASK_IMAGE_WHITE` will use the white pixels of the mask_image as the mask, where white pixels are completely replaced and black pixels are unchanged\n- `MASK_IMAGE_BLACK` will use the black pixels of the mask_image as the mask, where black pixels are completely replaced and white pixels are unchanged\n- `INIT_IMAGE_ALPHA` will use the alpha channel of the init_image as the mask, where fully transparent pixels are completely replaced and fully opaque pixels are unchanged"
},
"GenerationRequestOptionalParams": {
"type": "object",
"description": "Represents the optional parameters that can be passed to any generation request.",
"properties": {
"cfg_scale": {
"$ref": "#/components/schemas/CfgScale"
},
"clip_guidance_preset": {
"$ref": "#/components/schemas/ClipGuidancePreset"
},
"sampler": {
"$ref": "#/components/schemas/Sampler"
},
"samples": {
"$ref": "#/components/schemas/Samples"
},
"seed": {
"$ref": "#/components/schemas/Seed"
},
"steps": {
"$ref": "#/components/schemas/Steps"
},
"style_preset": {
"$ref": "#/components/schemas/StylePreset"
},
"extras": {
"$ref": "#/components/schemas/Extras"
}
}
},
"RealESRGANUpscaleRequestBody": {
"type": "object",
"properties": {
"image": {
"$ref": "#/components/schemas/InputImage"
},
"width": {
"$ref": "#/components/schemas/UpscaleImageWidth"
},
"height": {
"$ref": "#/components/schemas/UpscaleImageHeight"
}
},
"required": ["image"]
},
"ImageToImageRequestBody": {
"type": "object",
"properties": {
"text_prompts": {
"$ref": "#/components/schemas/TextPrompts"
},
"init_image": {
"$ref": "#/components/schemas/InitImage"
},
"init_image_mode": {
"$ref": "#/components/schemas/InitImageMode"
},
"image_strength": {
"$ref": "#/components/schemas/InitImageStrength"
},
"step_schedule_start": {
"$ref": "#/components/schemas/StepScheduleStart"
},
"step_schedule_end": {
"$ref": "#/components/schemas/StepScheduleEnd"
},
"cfg_scale": {
"$ref": "#/components/schemas/CfgScale"
},
"clip_guidance_preset": {
"$ref": "#/components/schemas/ClipGuidancePreset"
},
"sampler": {
"$ref": "#/components/schemas/Sampler"
},
"samples": {
"$ref": "#/components/schemas/Samples"
},
"seed": {
"$ref": "#/components/schemas/Seed"
},
"steps": {
"$ref": "#/components/schemas/Steps"
},
"style_preset": {
"$ref": "#/components/schemas/StylePreset"
},
"extras": {
"$ref": "#/components/schemas/Extras"
}
},
"required": ["text_prompts", "init_image"],
"discriminator": {
"propertyName": "init_image_mode",
"mapping": {
"IMAGE_STRENGTH": "#/components/schemas/ImageToImageUsingImageStrengthRequestBody",
"STEP_SCHEDULE": "#/components/schemas/ImageToImageUsingStepScheduleRequestBody"
}
}
},
"ImageToImageUsingImageStrengthRequestBody": {
"allOf": [
{
"type": "object",
"properties": {
"text_prompts": {
"$ref": "#/components/schemas/TextPrompts"
},
"init_image": {
"$ref": "#/components/schemas/InitImage"
},
"init_image_mode": {
"$ref": "#/components/schemas/InitImageMode"
},
"image_strength": {
"$ref": "#/components/schemas/InitImageStrength"
}
},
"required": ["text_prompts", "init_image"]
},
{
"$ref": "#/components/schemas/GenerationRequestOptionalParams"
}
]
},
"ImageToImageUsingStepScheduleRequestBody": {
"allOf": [
{
"type": "object",
"properties": {
"text_prompts": {
"$ref": "#/components/schemas/TextPrompts"
},
"init_image": {
"$ref": "#/components/schemas/InitImage"
},
"init_image_mode": {
"$ref": "#/components/schemas/InitImageMode"
},
"step_schedule_start": {
"$ref": "#/components/schemas/StepScheduleStart"
},
"step_schedule_end": {
"$ref": "#/components/schemas/StepScheduleEnd"
}
},
"required": ["text_prompts", "init_image"]
},
{
"$ref": "#/components/schemas/GenerationRequestOptionalParams"
}
]
},
"MaskingRequestBody": {
"type": "object",
"properties": {
"init_image": {
"$ref": "#/components/schemas/InitImage"
},
"mask_source": {
"$ref": "#/components/schemas/MaskSource"
},
"mask_image": {
"$ref": "#/components/schemas/MaskImage"
},
"text_prompts": {
"$ref": "#/components/schemas/TextPrompts"
},
"cfg_scale": {
"$ref": "#/components/schemas/CfgScale"
},
"clip_guidance_preset": {
"$ref": "#/components/schemas/ClipGuidancePreset"
},
"sampler": {
"$ref": "#/components/schemas/Sampler"
},
"samples": {
"$ref": "#/components/schemas/Samples"
},
"seed": {
"$ref": "#/components/schemas/Seed"
},
"steps": {
"$ref": "#/components/schemas/Steps"
},
"style_preset": {
"$ref": "#/components/schemas/StylePreset"
},
"extras": {
"$ref": "#/components/schemas/Extras"
}
},
"required": ["text_prompts", "init_image", "mask_source"],
"discriminator": {
"propertyName": "mask_source",
"mapping": {
"MASK_IMAGE_BLACK": "#/components/schemas/MaskingUsingMaskImageRequestBody",
"MASK_IMAGE_WHITE": "#/components/schemas/MaskingUsingMaskImageRequestBody",
"INIT_IMAGE_ALPHA": "#/components/schemas/MaskingUsingInitImageAlphaRequestBody"
}
}
},
"MaskingUsingMaskImageRequestBody": {
"allOf": [
{
"type": "object",
"properties": {
"text_prompts": {
"$ref": "#/components/schemas/TextPrompts"
},
"init_image": {
"$ref": "#/components/schemas/InitImage"
},
"mask_source": {
"$ref": "#/components/schemas/MaskSource"
},
"mask_image": {
"$ref": "#/components/schemas/MaskImage"
}
},
"required": [
"init_image",
"mask_image",
"text_prompts",
"mask_source"
]
},
{
"$ref": "#/components/schemas/GenerationRequestOptionalParams"
}
]
},
"MaskingUsingInitImageAlphaRequestBody": {
"allOf": [
{
"type": "object",
"properties": {
"text_prompts": {
"$ref": "#/components/schemas/TextPrompts"
},
"init_image": {
"$ref": "#/components/schemas/InitImage"
},
"mask_source": {
"$ref": "#/components/schemas/MaskSource"
}
},
"required": ["init_image", "text_prompts", "mask_source"]
},
{
"$ref": "#/components/schemas/GenerationRequestOptionalParams"
}
]
},
"TextToImageRequestBody": {
"type": "object",
"allOf": [
{
"type": "object",
"properties": {
"height": {
"$ref": "#/components/schemas/DiffuseImageHeight"
},
"width": {
"$ref": "#/components/schemas/DiffuseImageWidth"
},
"text_prompts": {
"$ref": "#/components/schemas/TextPromptsForTextToImage"
}
},
"required": ["text_prompts"]
},
{
"$ref": "#/components/schemas/GenerationRequestOptionalParams"
}
],
"example": {
"cfg_scale": 7,
"height": 512,
"width": 512,
"sampler": "K_DPM_2_ANCESTRAL",
"samples": 1,
"seed": 0,
"steps": 30,
"text_prompts": [
{
"text": "A lighthouse on a cliff",
"weight": 1
}
]
},
"required": ["text_prompts"]
},
"AccountResponseBody": {
"type": "object",
"properties": {
"email": {
"type": "string",
"description": "The user's email",
"example": "example@stability.ai",
"format": "email"
},
"id": {
"type": "string",
"description": "The user's ID",
"example": "user-1234",
"x-go-name": "ID"
},
"organizations": {
"type": "array",
"example": [
{
"id": "org-5678",
"name": "Another Organization",
"role": "MEMBER",
"is_default": true
},
{
"id": "org-1234",
"name": "My Organization",
"role": "MEMBER",
"is_default": false
}
],
"items": {
"$ref": "#/components/schemas/OrganizationMembership"
},
"description": "The user's organizations"
},
"profile_picture": {
"type": "string",
"description": "The user's profile picture",
"example": "https://api.stability.ai/example.png",
"format": "uri"
}
},
"required": ["id", "email", "organizations"]
},
"BalanceResponseBody": {
"type": "object",
"properties": {
"credits": {
"type": "number",
"description": "The balance of the account/organization associated with the API key",
"example": 0.41122252265928866,
"format": "double"
}
},
"example": {
"credits": 0.07903292496944721
},
"required": ["credits"]
},
"ListEnginesResponseBody": {
"type": "array",
"description": "The engines available to your user/organization",
"items": {
"$ref": "#/components/schemas/Engine"
},
"example": [
{
"description": "Stability-AI Stable Diffusion v1.6",
"id": "stable-diffusion-v1-6",
"name": "Stable Diffusion v1.6",
"type": "PICTURE"
},
{
"description": "Stability-AI Stable Diffusion XL v1.0",
"id": "stable-diffusion-xl-1024-v1-0",
"name": "Stable Diffusion XL v1.0",
"type": "PICTURE"
}
]
},
"FinishReason": {
"type": "string",
"description": "The result of the generation process.\n- `SUCCESS` indicates success\n- `ERROR` indicates an error\n- `CONTENT_FILTERED` indicates the result affected by the content filter and may be blurred.\n\nThis header is only present when the `Accept` is set to `image/png`. Otherwise it is returned in the response body.",
"enum": ["SUCCESS", "ERROR", "CONTENT_FILTERED"]
},
"Image": {
"type": "object",
"properties": {
"base64": {
"type": "string",
"x-go-type-skip-optional-pointer": true,
"description": "Image encoded in base64"
},
"finishReason": {
"type": "string",
"x-go-type-skip-optional-pointer": true,
"example": "CONTENT_FILTERED",
"enum": ["SUCCESS", "ERROR", "CONTENT_FILTERED"]
},
"seed": {
"type": "number",
"x-go-type-skip-optional-pointer": true,
"description": "The seed associated with this image",
"example": 1229191277
}
},
"example": [
{
"base64": "...very long string...",
"finishReason": "SUCCESS",
"seed": 1050625087
},
{
"base64": "...very long string...",
"finishReason": "CONTENT_FILTERED",
"seed": 1229191277
}
]
},
"OrganizationMembership": {
"type": "object",
"properties": {
"id": {
"type": "string",
"example": "org-123456",
"x-go-name": "ID"
},
"is_default": {
"type": "boolean",
"example": false
},
"name": {
"type": "string",
"example": "My Organization"
},
"role": {
"type": "string",
"example": "MEMBER"
}
},
"required": ["id", "name", "role", "is_default"]
}
},
"parameters": {
"upscaleEngineID": {
"in": "path",
"name": "engine_id",
"required": true,
"schema": {
"type": "string"
},
"examples": {
"ESRGAN_X2_PLUS": {
"description": "ESRGAN x2 Upscaler",
"value": "esrgan-v1-x2plus"
}
}
},
"engineID": {
"examples": {
"default": {
"value": "stable-diffusion-v1-6",
"description": "Stable Diffusion v1.6"
},
"stable-diffusion-xl-1024-v1-0": {
"value": "stable-diffusion-xl-1024-v1-0",
"description": "Stable Diffusion XL v1.0"
}
},
"in": "path",
"name": "engine_id",
"required": true,
"schema": {
"type": "string"
}
},
"organization": {
"allowEmptyValue": false,
"description": "Allows for requests to be scoped to an organization other than the user's default. If not provided, the user's default organization will be used.",
"example": "org-123456",
"in": "header",
"name": "Organization",
"x-go-name": "OrganizationID",
"schema": {
"type": "string"
}
},
"stabilityClientID": {
"allowEmptyValue": false,
"description": "Used to identify the source of requests, such as the client application or sub-organization. Optional, but recommended for organizational clarity.",
"example": "my-great-plugin",
"in": "header",
"name": "Stability-Client-ID",
"schema": {
"type": "string"
}
},
"stabilityClientVersion": {
"allowEmptyValue": false,
"description": "Used to identify the version of the application or service making the requests. Optional, but recommended for organizational clarity.",
"example": "1.2.1",
"in": "header",
"name": "Stability-Client-Version",
"schema": {
"type": "string"
}
},
"accept": {
"allowEmptyValue": false,
"in": "header",
"name": "Accept",
"description": "The format of the response. Leave blank for JSON, or set to 'image/png' for a PNG image.",
"schema": {
"default": "application/json",
"enum": ["application/json", "image/png"],
"type": "string"
}
}
},
"securitySchemes": {
"STABILITY_API_KEY": {
"type": "apiKey",
"scheme": "bearer",
"name": "authorization",
"in": "header",
"description": "Use your [Stability API key](https://platform.stability.ai/account/keys) to authentication requests to this App."
}
},
"responses": {
"401": {
"description": "unauthorized: API key missing or invalid",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Error"
},
"example": {
"id": "9160aa70-222f-4a36-9eb7-475e2668362a",
"name": "unauthorized",
"message": "missing authorization header"
}
}
}
},
"403": {
"description": "permission_denied: You lack the necessary permissions to perform this action",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Error"
},
"example": {
"id": "5cf19777-d17f-49fe-9bd9-39ff0ec6bb50",
"name": "permission_denied",
"message": "You do not have permission to access this resource"
}
}
}
},
"404": {
"description": "not_found: The requested resource/engine was not found",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Error"
},
"example": {
"id": "92b19e7f-22a2-4e71-a821-90edda229293",
"name": "not_found",
"message": "The specified engine (ID some-fake-engine) was not found."
}
}
}
},
"500": {
"description": "server_error: Some unexpected server error occurred",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Error"
},
"example": {
"id": "f81964d6-619b-453e-97bc-9fd7ac3f04e7",
"name": "server_error",
"message": "An unexpected server error occurred, please try again."
}
}
}
},
"GenerationResponse": {
"description": "Generation successful.",
"content": {
"application/json": {
"schema": {
"description": "An array of results from the generation request, where each image is a base64 encoded PNG.",
"type": "object",
"properties": {
"artifacts": {
"type": "array",
"x-go-type-skip-optional-pointer": true,
"items": {
"$ref": "#/components/schemas/Image"
}
}
}
}
},
"image/png": {
"example": "The bytes of the generated image, what did you expect?",
"schema": {
"description": "The bytes of the generated PNG image",
"format": "binary",
"type": "string"
}
}
},
"headers": {
"Content-Length": {
"$ref": "#/components/headers/Content-Length"
},
"Content-Type": {
"$ref": "#/components/headers/Content-Type"
},
"Finish-Reason": {
"$ref": "#/components/headers/Finish-Reason"
},
"Seed": {
"$ref": "#/components/headers/Seed"
}
}
},
"400FromGeneration": {
"description": "bad_request: one or more parameters were invalid.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Error"
},
"example": {
"id": "296a972f-666a-44a1-a3df-c9c28a1f56c0",
"name": "bad_request",
"message": "init_image: is required"
}
}
}
},
"400FromUpscale": {
"description": "One or more parameters were invalid.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Error"
},
"example": {
"id": "296a972f-666a-44a1-a3df-c9c28a1f56c0",
"name": "bad_request",
"message": "image: is required"
}
}
}
}
},
"headers": {
"Content-Length": {
"required": true,
"schema": {
"type": "integer"
}
},
"Content-Type": {
"required": true,
"schema": {
"enum": ["application/json", "image/png"],
"type": "string"
}
},
"Finish-Reason": {
"schema": {
"$ref": "#/components/schemas/FinishReason"
}
},
"Seed": {
"example": 3817857576,
"schema": {
"example": 787078103,
"type": "integer"
},
"description": "The seed used to generate the image. This header is only present when the `Accept` is set to `image/png`. Otherwise it is returned in the response body."
}
}
},
"x-tagGroups": [
{
"name": "Stable Image",
"tags": ["Generate", "Upscale", "Edit", "Control", "Results"]
},
{
"name": "3D & Video",
"tags": ["3D", "Image-to-Video"]
},
{
"name": "Version 1",
"tags": ["SDXL 1.0 & SD1.6", "Engines", "User"]
}
]
}
================================================
FILE: packages/backend/server.py
================================================
from app.log_config import root_logger
import sys
import os
from dotenv import load_dotenv
load_dotenv()
from app.flask.socketio_init import flask_app, socketio
import app.flask.sockets
import app.flask.routes
import app.tasks.single_thread_tasks.browser.async_browser_task
if __name__ == "__main__":
host = os.getenv("HOST", "127.0.0.1")
port = int(os.getenv("PORT", 8000))
root_logger.info(f"Starting application on {host}:{port}...")
root_logger.info("You can stop the application by pressing Ctrl+C at any time.")
# If we're running in a PyInstaller bundle
if getattr(sys, "frozen", False) and hasattr(sys, "_MEIPASS"):
os.environ["PLAYWRIGHT_BROWSERS_PATH"] = os.path.join(
sys._MEIPASS, "ms-playwright"
)
root_logger.warning("Protocol set to HTTP")
socketio.run(flask_app, port=port, host=host)
================================================
FILE: packages/backend/tests/unit/test_processor_factory.py
================================================
import unittest
from app.processors.factory.processor_factory_iter_modules import (
ProcessorFactoryIterModules,
)
from app.processors.components.processor import BasicProcessor, ContextAwareProcessor
class DummyProcessor(BasicProcessor):
processor_type = "dummy_processor"
def process(self):
pass
def cancel(self):
pass
class APIDummyProcessor(ContextAwareProcessor):
processor_type = "api_dummy_processor"
def __init__(self, config, context=None):
super().__init__(config)
self._processor_context = context
def process(self):
pass
def cancel(self):
pass
class TestProcessorFactory(unittest.TestCase):
def setUp(self):
self.factory = ProcessorFactoryIterModules()
def test_register_and_create_simple_processor(self):
self.factory.register_processor(DummyProcessor.processor_type, DummyProcessor)
processor = self.factory.create_processor(
{"processorType": "dummy_processor", "name": "dummy_processor"}
)
self.assertIsInstance(processor, DummyProcessor)
self.assertIsInstance(processor, BasicProcessor)
def test_create_unknown_processor_raises_exception(self):
with self.assertRaises(ValueError):
self.factory.create_processor(
{"processorType": "unknown_processor", "name": "unknown_processor"}
)
def test_create_processor_with_api_context_data(self):
self.factory.register_processor(
APIDummyProcessor.processor_type, APIDummyProcessor
)
processor = self.factory.create_processor(
{"processorType": "api_dummy_processor", "name": "api_dummy_processor"},
context_data="api_data",
)
self.assertIsInstance(processor, APIDummyProcessor)
self.assertIsInstance(processor, ContextAwareProcessor)
self.assertEqual(processor._processor_context, "api_data")
================================================
FILE: packages/backend/tests/unit/test_processor_launcher.py
================================================
import unittest
from unittest.mock import MagicMock, Mock, patch, mock_open
from app.processors.launcher.basic_processor_launcher import BasicProcessorLauncher
from app.processors.factory.processor_factory_iter_modules import (
ProcessorFactoryIterModules,
)
from dotenv import load_dotenv
from app.tasks.single_thread_tasks.browser.browser_task import (
stop_browser_thread,
)
load_dotenv()
from app.flask.socketio_init import socketio
class NoInputMock(MagicMock):
def __getattr__(self, name):
if name == "inputs":
raise AttributeError(
f"'{type(self).__name__}' object has no attribute 'inputs'"
)
return super().__getattr__(name)
class TestProcessorLauncher(unittest.TestCase):
def test_load_config_data_valid_file(self):
factory = ProcessorFactoryIterModules()
launcher = BasicProcessorLauncher(factory, None)
m = mock_open(read_data='{"key": "value"}')
with patch("builtins.open", m):
data = launcher._load_config_data("fake_file_path")
self.assertEqual(data, {"key": "value"})
def test_link_processors_valid(self):
factory = ProcessorFactoryIterModules()
launcher = BasicProcessorLauncher(factory, None)
processor1 = Mock()
processor1.name = "processor1"
processor1.inputs = [{"inputNode": "processor2"}]
processor2 = NoInputMock()
processor2.name = "processor2"
processors = {
"processor1": processor1,
"processor2": processor2,
}
launcher._link_processors(processors)
processor1.add_input_processor.assert_called_once_with(processor2)
stop_browser_thread()
================================================
FILE: packages/backend/tests/unit/test_stable_diffusion_stabilityai_prompt_processor.py
================================================
import unittest
from unittest.mock import ANY, patch, Mock
import re
from app.processors.components.core.stable_diffusion_stabilityai_prompt_processor import (
StableDiffusionStabilityAIPromptProcessor,
)
from app.storage.local_storage_strategy import LocalStorageStrategy
from app.processors.components.core.input_processor import InputProcessor
from tests.utils.processor_context_mock import ProcessorContextMock
class TestStableDiffusionStabilityAIPromptProcessor(unittest.TestCase):
@staticmethod
def get_default_valid_config():
return {
"inputs": [{"inputNode": "0rhbnaol3#gpt-no-context-prompt"}],
"name": "vd6up8r0m#stable-diffusion-stabilityai-prompt",
"processorType": "stable-diffusion-stabilityai-prompt",
"size": "1024x1024",
"x": -1961.3132869508825,
"y": -73.26855714327525,
}
@patch("requests.post")
def test_process_returns_valid_image_url_on_successful_api_response(
self, mock_post
):
# Arrange
API_KEY = "FAKE"
mock_response = Mock()
mock_response.status_code = 200
mock_response.json.return_value = {
"artifacts": [{"base64": "R0lGODlhAQABAAAAACw="}]
}
mock_post.return_value = mock_response
url_pattern = re.compile(r"http?://[^\s]+")
config = self.get_default_valid_config()
clean_name = config["name"].replace("#", "")
api_context_data = ProcessorContextMock(API_KEY)
processor = StableDiffusionStabilityAIPromptProcessor(config, api_context_data)
processor.set_storage_strategy(LocalStorageStrategy())
# Act
url = processor.process()
# Assert
self.assertTrue(url_pattern.match(url))
self.assertIn(clean_name, url)
self.assertTrue(url.endswith(".png"))
mock_post.assert_called_once_with(
ANY,
headers=ANY,
json=ANY,
)
@patch("requests.post")
def test_process_transmit_prompt_to_api(self, mock_post):
# Arrange
API_KEY = "FAKE"
mock_response = Mock()
mock_response.status_code = 200
mock_response.json.return_value = {
"artifacts": [{"base64": "R0lGODlhAQABAAAAACw="}]
}
mock_post.return_value = mock_response
config = self.get_default_valid_config()
expected_prompt = "Expected prompt"
config["prompt"] = expected_prompt
api_context_data = ProcessorContextMock(API_KEY)
processor = StableDiffusionStabilityAIPromptProcessor(config, api_context_data)
processor.set_storage_strategy(LocalStorageStrategy())
# Act
processor.process()
# Assert
called_args = mock_post.call_args[1]
sent_json = called_args.get("json")
transmitted_prompts = sent_json["text_prompts"]
self.assertTrue(
any(prompt["text"] == expected_prompt for prompt in transmitted_prompts)
)
@patch("requests.post")
def test_when_linked_to_input_node_transmits_input_node_output_to_the_api(
self, mock_post
):
# Arrange
API_KEY = "FAKE"
expected_prompt = "Expected prompt"
input_processor = InputProcessor(
{
"inputs": [],
"name": "p00tklq5w#input-text",
"processorType": "input-text",
"inputText": expected_prompt,
}
)
input_processor.set_output(expected_prompt)
mock_response = Mock()
mock_response.status_code = 200
mock_response.json.return_value = {
"artifacts": [{"base64": "R0lGODlhAQABAAAAACw="}]
}
mock_post.return_value = mock_response
config = self.get_default_valid_config()
api_context_data = ProcessorContextMock(API_KEY)
processor = StableDiffusionStabilityAIPromptProcessor(config, api_context_data)
processor.set_storage_strategy(LocalStorageStrategy())
processor.add_input_processor(input_processor)
# Act
processor.process()
# Assert
called_args = mock_post.call_args[1]
sent_json = called_args.get("json")
transmitted_prompts = sent_json["text_prompts"]
self.assertTrue(
any(prompt["text"] == expected_prompt for prompt in transmitted_prompts)
)
================================================
FILE: packages/backend/tests/utils/openai_mock_utils.py
================================================
from unittest.mock import Mock
def create_mocked_openai_response(
model="gpt-4", api_key="000000000", response_content="Mocked Response"
):
"""
Create a mocked response for OpenAI.
:param model: The model to be used.
:param api_key: The API key to be used.
:param response_content: The content for the mocked response.
:return: A mocked response for OpenAI.
"""
mock_message = Mock()
mock_message.content = response_content
mock_choice = Mock()
mock_choice.message = mock_message
mock_response = Mock()
mock_response.choices = [mock_choice]
return mock_response
================================================
FILE: packages/backend/tests/utils/processor_context_mock.py
================================================
from typing import List
from app.processors.context.processor_context import ProcessorContext
from typing import List
class ProcessorContextMock(ProcessorContext):
def __init__(self, api_key, user_id=0, session_id=0) -> None:
super().__init__()
self.api_key = api_key
self.user_id = user_id
self.session_id = session_id
def get_context(self):
return self.api_key
def get_current_user_id(self):
return self.user_id
def get_session_id(self):
return self.user_id
def get_parameter_names(self) -> List[str]:
return super().get_parameter_names()
def get_value(self, name):
if "api_key" in name:
return self.api_key
return super().get_value(name)
def is_using_personal_keys(self, source_name):
return False
================================================
FILE: packages/backend/tests/utils/processor_factory_mock.py
================================================
import logging
import random
import eventlet
import time
from injector import singleton
from unittest.mock import MagicMock
from app.processors.factory.processor_factory_iter_modules import (
ProcessorFactoryIterModules,
)
from app.processors.components.core.processor_type_name_utils import (
ProcessorType,
)
from .processor_context_mock import ProcessorContextMock
@singleton
class ProcessorFactoryMock(ProcessorFactoryIterModules):
MIN_DELAY = 0.1
MAX_DELAY = 1
NON_MOCKED_PROCESSORS = [
ProcessorType.INPUT_TEXT.value,
ProcessorType.INPUT_IMAGE.value,
ProcessorType.URL_INPUT.value,
ProcessorType.DISPLAY.value,
ProcessorType.TRANSITION.value,
]
def __init__(
self,
fake_text_output=None,
fake_img_output=None,
fake_multiple_output=None,
with_delay=False,
sleep_duration=None,
):
super().__init__()
self._mock_processors = {}
self.fake_text_output = fake_text_output
self.fake_img_output = fake_img_output
self.fake_multiple_output = fake_multiple_output
self.with_delay = with_delay
def create_mock_processor(
self, config, processor_type: ProcessorType, processor_class: str
):
mock_processor = MagicMock(spec=processor_class)
mock_processor.name = config.get("name", "default_processor_name")
mock_processor.processor_type = processor_type
mock_processor.input_processors = []
mock_processor._processor_context = ProcessorContextMock("")
if config.get("inputs") is not None and config.get("inputs") != []:
mock_processor.inputs = config.get("inputs")
def fake_process(*args, **kwargs):
if self.with_delay:
sleep_duration = random.uniform(
ProcessorFactoryMock.MIN_DELAY, ProcessorFactoryMock.MAX_DELAY
)
eventlet.sleep(sleep_duration)
if config.get("sleepDuration") is not None:
sleep_duration = config.get("sleepDuration")
logging.info(f"Sleeping for {sleep_duration} seconds")
eventlet.sleep(sleep_duration)
logging.info("Awake")
if mock_processor.processor_type in [
ProcessorType.DALLE_PROMPT.value,
ProcessorType.STABLE_DIFFUSION_STABILITYAI_PROMPT.value,
]:
output = (
[self.fake_img_output]
if self.fake_img_output is not None
else [
"https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/v0.4.0-sample-1.png"
]
)
elif mock_processor.processor_type in [
ProcessorType.AI_DATA_SPLITTER.value
]:
output = (
self.fake_multiple_output
if self.fake_multiple_output is not None
else ["Lorem Ipsum", "Lorem Ipsum"]
)
else:
output = (
self.fake_text_output
if self.fake_text_output is not None
else "Lorem Ipsum"
)
mock_processor.set_output(output)
mock_processor.is_finished = True
return output
def fake_process_raise_error(*args, **kwargs):
logging.error("MockProcessor - Fake Error")
raise Exception("Mock Processor error")
def fake_add_input_processor(input_processor):
mock_processor.input_processors.append(input_processor)
def get_input_processors():
return mock_processor.input_processors
def fake_has_dynamic_behavior():
return False
def fake_get_input_by_name(input_name, default_value=""):
return default_value
mock_processor.process_and_update = (
fake_process
if config.get("raiseError", False) == False
else fake_process_raise_error
)
mock_processor.add_input_processor = fake_add_input_processor
mock_processor.get_input_processors = get_input_processors
mock_processor.has_dynamic_behavior = fake_has_dynamic_behavior
mock_processor.get_input_by_name = fake_get_input_by_name
self._mock_processors[processor_type] = mock_processor
return mock_processor
def create_processor(self, config, context=None, storage_strategy=None):
processor_type = config["processorType"]
processor_class = self._processors.get(processor_type)
if not processor_class:
raise ValueError(f"Processor type '{processor_type}' not supported")
if (
processor_type in ProcessorFactoryMock.NON_MOCKED_PROCESSORS
and config.get("raiseError", False) == False
):
processor = processor_class(config=config)
else:
processor = self.create_mock_processor(
config, processor_type, processor_class
)
return processor
================================================
FILE: packages/ui/.gitignore
================================================
# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
# dependencies
/node_modules
/.pnp
.pnp.js
# testing
/coverage
# production
/build
# misc
.DS_Store
.env.local
.env.development.local
.env.test.local
.env.production.local
npm-debug.log*
yarn-debug.log*
yarn-error.log*
#amplify-do-not-edit-begin
amplify/\#current-cloud-backend
amplify/.config/local-*
amplify/logs
amplify/mock-data
amplify/mock-api-resources
amplify/backend/amplify-meta.json
amplify/backend/.temp
build/
dist/
node_modules/
aws-exports.js
awsconfiguration.json
amplifyconfiguration.json
amplifyconfiguration.dart
amplify-build-config.json
amplify-gradle-config.json
amplifytools.xcconfig
.secret-*
**.sample
#amplify-do-not-edit-end
================================================
FILE: packages/ui/.prettierignore
================================================
node_modules
# Ignore artifacts:
build
coverage
================================================
FILE: packages/ui/Dockerfile
================================================
FROM node:21 as build
WORKDIR /app
ARG VITE_APP_WS_HOST
ARG VITE_APP_WS_PORT
ARG VITE_APP_API_REST_PORT
ARG VITE_APP_USE_HTTPS
ARG VITE_APP_VERSION
ENV VITE_APP_WS_HOST=$VITE_APP_WS_HOST
ENV VITE_APP_WS_PORT=$VITE_APP_WS_PORT
ENV VITE_APP_API_REST_PORT=$VITE_APP_API_REST_PORT
ENV VITE_APP_USE_HTTPS=$VITE_APP_USE_HTTPS
ENV VITE_APP_VERSION=$VITE_APP_VERSION
COPY package.json package-lock.json /app/
RUN npm ci
COPY . /app/
RUN npm run build
RUN ls -al /app
FROM nginx:1.21
COPY --from=build ./app/build /usr/share/nginx/html
COPY nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
================================================
FILE: packages/ui/README.md
================================================
# Getting Started with Create React App
This project was bootstrapped with [Create React App](https://github.com/facebook/create-react-app).
## Available Scripts
In the project directory, you can run:
### `npm start`
Runs the app in the development mode.\
Open [http://localhost:3000](http://localhost:3000) to view it in the browser.
The page will reload if you make edits.\
You will also see any lint errors in the console.
### `npm test`
Launches the test runner in the interactive watch mode.\
See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information.
### `npm run build`
Builds the app for production to the `build` folder.\
It correctly bundles React in production mode and optimizes the build for the best performance.
The build is minified and the filenames include the hashes.\
Your app is ready to be deployed!
See the section about [deployment](https://facebook.github.io/create-react-app/docs/deployment) for more information.
### `npm run eject`
**Note: this is a one-way operation. Once you `eject`, you can’t go back!**
If you aren’t satisfied with the build tool and configuration choices, you can `eject` at any time. This command will remove the single build dependency from your project.
Instead, it will copy all the configuration files and the transitive dependencies (webpack, Babel, ESLint, etc) right into your project so you have full control over them. All of the commands except `eject` will still work, but they will point to the copied scripts so you can tweak them. At this point you’re on your own.
You don’t have to ever use `eject`. The curated feature set is suitable for small and middle deployments, and you shouldn’t feel obligated to use this feature. However we understand that this tool wouldn’t be useful if you couldn’t customize it when you are ready for it.
## Learn More
You can learn more in the [Create React App documentation](https://facebook.github.io/create-react-app/docs/getting-started).
To learn React, check out the [React documentation](https://reactjs.org/).
================================================
FILE: packages/ui/index.html
================================================
AI Flow
================================================
FILE: packages/ui/jest.config.ts
================================================
import type { Config } from "@jest/types";
const config: Config.InitialOptions = {
verbose: true,
preset: "ts-jest",
testEnvironment: "node",
testMatch: ["**/test/**/*.ts", "**/?(*.)+(spec|test).ts"],
};
export default config;
================================================
FILE: packages/ui/nginx.conf
================================================
server {
listen 80;
server_name localhost;
root /usr/share/nginx/html;
index index.html;
location / {
try_files $uri $uri/ /index.html;
add_header Content-Security-Policy "script-src 'self' 'unsafe-inline';";
}
}
================================================
FILE: packages/ui/package.json
================================================
{
"name": "ai-flow-front",
"version": "0.11.3",
"private": true,
"dependencies": {
"@headlessui/react": "^1.7.17",
"@mantine/core": "^7.7.1",
"@mantine/hooks": "^7.7.1",
"@testing-library/jest-dom": "^5.16.5",
"@testing-library/react": "^13.4.0",
"@testing-library/user-event": "^13.5.0",
"@types/chai": "^4.3.12",
"@types/lodash.debounce": "^4.0.8",
"@types/node": "^20.12.12",
"@types/three": "^0.164.0",
"@vitejs/plugin-react-swc": "^3.6.0",
"autoprefixer": "^10.4.14",
"axios": "^1.6.2",
"dotenv": "^16.0.3",
"framer-motion": "^10.16.4",
"github-markdown-css": "^5.4.0",
"i18next": "^22.5.0",
"i18next-browser-languagedetector": "^7.0.1",
"i18next-http-backend": "^2.2.1",
"lodash.debounce": "^4.0.8",
"polished": "^4.2.2",
"rdndmb-html5-to-touch": "^8.0.3",
"react": "^18.2.0",
"react-copy-to-clipboard": "^5.1.0",
"react-dnd": "^16.0.1",
"react-dnd-html5-backend": "^16.0.1",
"react-dnd-multi-backend": "^8.0.3",
"react-dnd-touch-backend": "^16.0.1",
"react-dom": "^18.2.0",
"react-dropzone": "^14.2.3",
"react-grid-layout": "^1.4.4",
"react-i18next": "^12.3.1",
"react-icons": "^4.11.0",
"react-joyride": "^2.7.2",
"react-markdown": "^9.0.0",
"react-resizable": "^3.0.5",
"react-sketch-canvas": "^6.2.0",
"react-switch": "^7.0.0",
"react-syntax-highlighter": "^15.6.1",
"react-toastify": "^9.1.3",
"react-tooltip": "^5.13.1",
"react18-json-view": "^0.2.9",
"reactflow": "^11.7.2",
"remark-gfm": "^4.0.0",
"socket.io-client": "^4.6.1",
"styled-components": "^5.3.10",
"tailwindcss": "^3.3.2",
"three": "^0.164.1",
"typescript": "^4.9.5",
"typescript-json-schema": "^0.63.0",
"video.js": "^8.12.0",
"videojs-wavesurfer": "^3.10.0",
"vite": "^5.2.11",
"vite-plugin-svgr": "^4.2.0",
"web-vitals": "^2.1.4"
},
"scripts": {
"start": "vite --host",
"build": "tsc && vite build",
"serve": "vite preview",
"test": "vitest",
"test:e2e": "playwright test"
},
"eslintConfig": {
"extends": [
"react-app",
"react-app/jest"
]
},
"browserslist": {
"production": [
">0.2%",
"not dead",
"not op_mini all"
],
"development": [
"last 1 chrome version",
"last 1 firefox version",
"last 1 safari version"
]
},
"devDependencies": {
"@playwright/test": "^1.45.1",
"@types/dompurify": "^3.0.2",
"@types/jest": "^29.5.12",
"@types/react": "^18.3.3",
"@types/react-copy-to-clipboard": "^5.0.7",
"@types/react-dom": "^18.3.0",
"@types/react-grid-layout": "^1.3.5",
"@types/react-syntax-highlighter": "^15.5.13",
"@types/styled-components": "^5.1.26",
"@types/video.js": "^7.3.58",
"@vitest/coverage-v8": "^1.6.0",
"jsdom": "^24.0.0",
"postcss": "^8.4.38",
"postcss-preset-mantine": "^1.13.0",
"postcss-simple-vars": "^7.0.1",
"prettier": "^3.2.4",
"prettier-plugin-tailwindcss": "^0.5.11",
"ts-jest": "^29.1.2",
"vite-bundle-visualizer": "^1.2.1",
"vitest": "^1.6.0"
}
}
================================================
FILE: packages/ui/postcss.config.cjs
================================================
module.exports = {
plugins: {
"postcss-preset-mantine": {},
"postcss-simple-vars": {
variables: {
"mantine-breakpoint-xs": "36em",
"mantine-breakpoint-sm": "48em",
"mantine-breakpoint-md": "62em",
"mantine-breakpoint-lg": "75em",
"mantine-breakpoint-xl": "88em",
},
},
},
};
================================================
FILE: packages/ui/postcss.config.js
================================================
module.exports = {
plugins: {
tailwindcss: {},
autoprefixer: {},
},
}
================================================
FILE: packages/ui/prettier.config.js
================================================
module.exports = {
plugins: ['prettier-plugin-tailwindcss'],
}
================================================
FILE: packages/ui/public/health
================================================
OK
================================================
FILE: packages/ui/public/locales/en/aiActions.json
================================================
{
"Summary": "Summary",
"SpellCheck": "Spell Check",
"VisualPrompt": "Visual Prompt",
"ConstructiveCritique": "Constructive Critique",
"SimpleExplanation": "Simple Explanation",
"Paraphrase": "Paraphrase",
"SentimentAnalysis": "Sentiment Analysis",
"TextExtension": "Text Extension",
"ClickToShowOutput": "Click to show output"
}
================================================
FILE: packages/ui/public/locales/en/config.json
================================================
{
"configurationTitle": "Configuration",
"apiKeyDisclaimer": "We do not use or store your API keys.",
"openSourceDisclaimer": "The application code is open source.",
"apiKeyRevokeReminder": "Remember, you can revoke your keys at any time and generate new ones.",
"closeButtonLabel": "Close",
"validateButtonLabel": "Validate",
"likeProjectPrompt": "If you like this project, you can add a star on:",
"supportProjectPrompt": "You can support the future of the project and contact us through",
"Logout": "Logout",
"sections.core": "Base parameters",
"parameters.core.openai_api_key": "OpenAI API Key",
"parameters.core.stabilityai_api_key": "StabilityAI API Key",
"parameters.core.replicate_api_key": "Replicate API Key",
"parameters.extension.anthropic_api_key": "Anthropic API Key",
"sections.extension": "Extensions",
"userTabLabel": "User parameters",
"appParametersLabel": "App parameters",
"displayTabLabel": "Display parameters",
"nodesDisplayed": "Nodes enabled",
"configUpdated": "Configuration updated successfully",
"ShowMinimap": "Show minimap",
"UI": "User Interface",
"input": "Inputs",
"models": "Models",
"tools": "Tools",
"parameters.extension.deepseek_api_key": "DeepSeek API Key",
"parameters.extension.openrouter_api_key": "OpenRouter API Key",
"incompleteLoadingPleaseRestart": "The application failed to load all data. Please restart the app."
}
================================================
FILE: packages/ui/public/locales/en/dialogs.json
================================================
{
"attachNodeTitle": "Attach Node",
"attachNodeAction": "Attach"
}
================================================
FILE: packages/ui/public/locales/en/flow.json
================================================
{
"Flow": "Flow",
"AddTab": "Add Tab",
"ShowOnlyOutputs": "Show only outputs",
"ShowOnlyParams": "Show only params",
"ApiKeyRequiredMessage": "Please provide your API Key in the configuration settings to run your flow successfully.",
"Prompt": "Prompt",
"ClickToShowOutput": "Click to show output",
"RoleInitPrompt": "Indicate the role you wish the AI to adopt in the upcoming interactions. For example, 'Behave as a Literary Critic'.",
"EnterURL": "Web Extractor",
"YoutubeTranscriptNodeName": "Youtube Transcript",
"URLPlaceholder": "Input a URL",
"Input": "Inputs",
"InputImage": "Image",
"InputPlaceholder": "Enter your text here to serve as input for other nodes.",
"InputImagePlaceholder": "Enter the URL of the image you want to use.",
"DALLE": "DALL-E",
"ClickToShowImageOutput": "Click to show image output",
"JsonView": "JSON View",
"TopologicalView": "Outputs",
"currentNodeView": "Current Node",
"Upload": "Upload",
"Download": "Download",
"Text": "Text",
"URL": "URL",
"YoutubeVideo": "Youtube Transcript",
"Models": "Models",
"GPT": "GPT Model",
"GPTPrompt": "GPT Prompt",
"DataSplitter": "Data Splitter",
"ReplicateModel": "Replicate",
"NoContextPrompt": "GPT Prompt",
"PromptPlaceholder": "Enter your prompt here, for example 'Create a Twitter thread based on the data I sent you.'",
"MergePromptPlaceholder": "Enter your text for the merge, using ${input-1} and ${input-2} as placeholders. You can add additional text, for example: \n Answer to ${input-1} by considering ${input-2}.",
"VisionPromptPlaceholder": "Please enter your prompt here to start the image analysis. For example, you could write 'Describe this image'",
"VisionImageURLPlaceholder": "Enter the URL of the image you want to use.",
"DallEPromptPlaceholder": "Enter your prompt here, for example 'A dog and a cat playing in the desert'",
"ImageGeneration": "Image Generation",
"AdvancedSection": "Advanced",
"AiAction": "AI Action",
"LLMPrompt": "GPT",
"AiDataSplitter": "Data Splitter",
"MergerNode": "Merge Text",
"inputHelp": "This node is used to input text.",
"inputImageHelp": "This node is used to input an image URL.",
"urlInputHelp": "Enter a valid URL and the node will retrieve the data from this URL.",
"youtubeTranscriptHelp": "This node retrieves the subtitles of a YouTube video from its URL.",
"gptHelp": "This node allows you to configure a GPT model, specify its role, and the data it will use to respond.",
"gptPromptHelp": "This node allows you to query a GPT model. It shares its context with other nodes connected to the model.",
"noContextPromptHelp": "This node allows you to query GPT without context, just from input data and a prompt. You don't have to connect it to a GPT Model.",
"dallePromptHelp": "This node uses the DALL-E model to generate images from a textual description.",
"stableDiffusionPromptHelp": "This node uses the Stable Diffusion model to generate images from a textual description.",
"stableVideoDiffusionPromptHelp": "This node uses Replicate to launch Stable Video Diffusion.",
"aiActionPromptHelp": "This node uses GPT-4 to perform simple action, without having to write a prompt.",
"llmPromtHelp": "This nodes allows you to send prompt to GPT-3.5 or GPT-4",
"replicateHelp": "This nodes uses Replicate to give access to a large amount of models.",
"mergerPromptHelp": "This nodes allows you to merge 2 inputs.",
"gptVisionPromptHelp": "This node uses GPT-4 Vision and takes an image URL as input.",
"dataSplitterHelp": "This node is used to split data into several parts. You can specify upstream how many parts you want to create. You can also run it individually so that it finds the exact number of outputs to generate.",
"socketConnectionLost": "The connection has been lost.",
"ClickToSelectModel": "Click to select model",
"Or": "OR",
"EnterModelNameDirectly": "Enter model name directly",
"Load": "Load",
"LoadMore": "Load more",
"SpotlightModels": "Spotlight Models",
"AllModels": "All Models",
"EdgeType": "Edge type",
"CannotChangeTabWhileRunning": "Cannot change tab while running",
"EnterUrlToDesiredFile": "Enter file URL",
"Transition": "Transition",
"transitionHelp": "This node can be used to organize the flow.",
"MissingFieldsMessage": "Required fields are missing",
"Node": "Node",
"MissingFields": "Missing fields",
"CannotDeleteLastFlow": "Cannot delete the last flow",
"HideSidebar": "Hide sidebar",
"ShowSidebar": "Show sidebar",
"fileUploadHelp": "This node can be used to load a file.",
"llmPromptHelp": "This node allows you to send prompt to GPT-3.5 or GPT-4",
"Output": "Output",
"Inputs": "Inputs",
"Parameters": "Parameters",
"Duplicate": "Duplicate",
"OpeninSidepane": "Open in sidepane",
"ClearOutput": "Clear Output",
"RemoveNode": "Remove Node",
"ExpiredURL": "Expired URL",
"NoNodeSelected": "No node selected yet.",
"ClickOnNodeToSelectIt": "Click on any node to select it.",
"Field": "Field",
"DragAndDropNodes": "Drag and drop nodes onto the canvas to add them.",
"CopiedToClipboard": "Copied to clipboard.",
"DocumentToText": "Document-to-Text",
"documentToTextHelp": "Convert .pdf .txt .csv .json .html file to simple text",
"TextToSpeech": "Text-to-Speech",
"textToSpeechHelp": "Convert a text to an audio file using OpenAI tts model",
"error.upload_failed": "Upload failed. Please check your configuration to enable file upload.",
"InputTextPlaceholder": "Enter your text here",
"DownloadFile": "Download File",
"FileUploaded": "File uploaded",
"GenericPromptPlaceholder": "Enter your prompt here",
"GenericNegativePromptPlaceholder": "Enter your negative prompt here",
"EnterCustomName": "Enter custom name",
"NodeColor": "Change node color",
"ChangeName": "Change name",
"RemoveFlow": "Remove flow",
"HideHint": "Hide",
"TextDocumentHint": "Please note that this node only provides files as URLs. To use a document (.pdf, .txt) in text format, consider using the 'Document-to-Text' node.",
"Display": "Display",
"displayHelp": "This resizable node allows you to display content.",
"Validate": "Validate",
"AI": "AI",
"Separator": "Separator",
"ClaudeAnthropic": "Claude",
"claudeAnthropichHelp": "This node uses Claude from Anthropic to generate text.",
"noDataAvailableForThisNode": "No data available for this node",
"learnMore": "Learn more:",
"Help": "Help",
"cookiesConsentLabelPlaceholder": "Agree to all",
"cookiesConsentLabelHelp": "For some pages, we need to click on the cookie consent button to access the data. This instruction helps locate the button.",
"EditTextContent": "Edit text content",
"ShowCoordinates": "Show Coordinates",
"ShowNodesConfig": "Show Nodes Config",
"DeleteAll": "Delete All",
"DeleteOutputs": "Delete Outputs",
"ReplaceText": "Replace Text",
"ReplaceTextInputPlaceholder": "Enter the full text where the term will be replaced.",
"ReplaceTextSearchPlaceholder": "Enter the term or regex pattern to be replaced.",
"ReplaceTextReplacePlaceholder": "Enter the replacement term.",
"replaceTextNodeHelp": "Use this node to find and replace specific text or patterns within the input.",
"openaio1Help": "Advanced language models trained for complex reasoning, excelling in scientific, mathematical, and programming challenges.",
"ContextPlaceholder": "Additionnal context that will be used to answer your prompt.",
"deepSeekHelp": "Access DeepSeek LLMs through this node.",
"openRouterHelp": "OpenRouter provides completion API to multiple models & providers. This nodes requires you to provide an API Key.",
"Generate Number": "Generate Number",
"generateNumberHelp": "Generate a random number",
"httpGetProcessorURLPlaceholder": "Enter the URL to request",
"httpGetProcessorURLDescription": "The URL that the HTTP GET request will be sent to.",
"httpGetProcessorHeadersPlaceholder": "Enter headers in JSON format",
"httpGetProcessorHeadersDescription": "The headers to include in the HTTP GET request.",
"httpGetProcessorHelp": "Send an HTTP GET request with the specified headers.",
"gptImageHelp": "Generate or Edit an image using GPT Image",
"gptImageMaskDescription": "You can provide a mask to indicate where the image should be edited. You can use the prompt to describe the full new image, not just the erased area. If you provide multiple input images, the mask will be applied to the first image.",
"dallEDeprecated": "Most recent OpenAI models are now available via the new GPT Image node and are superior to DALL-E. DALL-E remains available if needed.",
"TTSInstructionPlaceholder": "Ex : Speak in a cheerful and positive tone.",
"TTSInstructionDescription": " Prompt the model to control aspects of speech (accent, emotional range, intonation, speed, tone, ...)",
"PopularModels": "Popular Models",
"removeBackgroundDescription": "Remove the background from an image using the StabilityAI API.",
"upscaleFastDescription": "Upscale an image using StabilityAI API",
"fluxDescription": "Generate an image using the FLUX model.",
"fluxKontextDescription": "A state-of-the-art text-based image editing model that delivers high-quality outputs with excellent prompt following and consistent results for transforming images through natural language",
"faceswapDescription": "Seamlessly swap faces between images, allowing for realistic and precise facial replacements.",
"removeBgDescription": "Remove the background from an image using lucataco/remove-bg from Replicate API.",
"upscaleDescription": "Real-ESRGAN with optional face correction and adjustable upscale",
"GoogleUpscaleDescription": "Upscale an image using Google's model.",
"moondreamDescription": "Moondream is a vision model that responds to prompts about a given image.",
"llamaDescription": "Meta's flagship 405 billion parameter language model, fine-tuned for chat completions.",
"imagenDescription": "Imagen produce stunning, detailed images with precision.",
"recraftSVGDescription": "Recraft SVG offers advanced vector image generation, enabling scalable and creative SVG designs.",
"recraftDescription": "Recraft V3 is a text-to-image model with the ability to generate long texts, and images in a wide list of styles. ",
"video01Description": "Video-01 provides dynamic video generation.",
"video01LiveDescription": "Video-01-Live provides dynamic video generation, ideal for 2D animation.",
"klingDescription": "Kling delivers robust video generation and animation solutions, available in both professional and standard versions.",
"veo3Description": "Google’s flagship Veo 3 text to video model, with audio"
}
================================================
FILE: packages/ui/public/locales/en/nodeHelp.json
================================================
{
"input-text": {
"description": "Text Node can be used to transfer text input to other nodes.",
"docUrls": [
{
"url": "https://docs.ai-flow.net/docs/nodes-presentation/Input-nodes",
"label": "Add an input in AI-FLOW"
}
]
},
"url_input": {
"description": "Retrieve textual content from an URL.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/web-extractor-demo.gif",
"docUrls": [
{
"url": "https://docs.ai-flow.net/docs/nodes-presentation/Input-nodes#url",
"label": "Add an input in AI-FLOW"
}
]
},
"llm-prompt": {
"description": "Processes inputs using GPT models by OpenAI, which can understand and generate responses based on the context provided by the user.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/gpt-demo.gif",
"docUrls": [
{
"url": "https://docs.ai-flow.net/blog/summarize-doc-post",
"label": "How to Summarize Documents or Ask Questions Using AI-FLOW"
},
{
"url": "https://docs.ai-flow.net/blog/summarize-ytb-post",
"label": "How to Summarize a YouTube Video Using AI-FLOW"
}
]
},
"gpt-vision": {
"description": "Use GPT-4-o to analyze a picture.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/vision-demo.gif",
"docUrls": []
},
"youtube_transcript_input": {
"description": "Captures transcripts directly from YouTube API, allowing for further processing like translation, summarization, or keyword extraction.",
"docUrls": [
{
"url": "https://docs.ai-flow.net/blog/summarize-ytb-post",
"label": "How to Summarize a YouTube Video Using AI-FLOW"
}
]
},
"dalle-prompt": {
"description": "Allows users to create detailed prompts to generate images using the DALL-E 3 model, combining creativity and AI for custom visual content. Please note that OpenAI only allow 5-7 images per minute. Don't forget to save your file; OpenAI host files for 1H.",
"docUrls": [
{
"url": "https://docs.ai-flow.net/docs/nodes-presentation/text-to-image-processing",
"label": "Image Generation in AI-FLOW"
}
]
},
"stable-diffusion-stabilityai-prompt": {
"description": "Stable Diffusion SDXL model by Stability AI, offering quick and low cost image generation. Don't forget to save your file; files are available for 12 hours.",
"docUrls": []
},
"merger-prompt": {
"description": "Used for combining two outputs. Each output need to be used with his specific identifier that will be replaced dynamically e.g ${input-1} and ${input-2}. Use the buttons at the top of the node to insert the identifier automatically !",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/merge-demo.gif",
"docUrls": []
},
"claude-anthropic-processor": {
"description": "Processes inputs using Claude 3 by Anthropic, which can understand and generate responses based on the context provided by the user.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/claude-demo.gif",
"docUrls": [
{
"url": "https://docs.ai-flow.net/blog/anthropic-claude-api",
"label": "Access Claude 3 from Anthropic API through AI-FLOW"
}
]
},
"document-to-text-processor": {
"description": "Converts various document formats into plain text, enabling text extraction for processing and analysis. This nodes supports .pdf, .txt, .json, .html, .csv.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/document-to-text-demo.gif",
"docUrls": [
{
"url": "https://docs.ai-flow.net/blog/summarize-doc-post",
"label": "How to Summarize Documents or Ask Questions Using AI-FLOW"
}
]
},
"openai-text-to-speech-processor": {
"description": "Converts text to natural-sounding speech using OpenAI's advanced text-to-speech models, facilitating accessibility and multimedia applications. Don't forget to save your file, OpenAI host the file for 1H.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/tts-demo.gif",
"docUrls": []
},
"stabilityai-generic-processor": {
"description": "A versatile node capable of interfacing with StabilityAI API to perform various task such as remove background, search and replace and more. Don't forget to save your file; files are available for 12 hours. ",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/stabilityai-demo.gif",
"docUrls": []
},
"stabilityai-stable-diffusion-3-processor": {
"description": "Integrates the latest Stable Diffusion 3 capabilities for high-quality image generation. Don't forget to save your file; files are available for 12 hours.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/sd3-demo.gif",
"docUrls": [
{
"url": "https://docs.ai-flow.net/blog/stable-diffusion-3-api",
"label": "Access Stable Diffusion 3 API through AI-FLOW"
}
]
},
"file": {
"description": "Handles the uploading, storage, and retrieval of files, supporting various file types for use within the system. This node does not extract file content. Files are available 12H.",
"docUrls": [
{
"url": "https://docs.ai-flow.net/blog/summarize-doc-post",
"label": "How to Summarize Documents or Ask Questions Using AI-FLOW"
}
]
},
"ai-data-splitter": {
"description": "Split an input into multiple outputs using two available modes: AI mode and Manual mode. In Manual mode, you must specify a separator. This can be useful for generating content based on a list of ideas or concepts. You can specify an estimated number of outputs to prepare your flow accordingly.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/splitter-demo.gif",
"docUrls": [
{
"url": "https://docs.ai-flow.net/docs/nodes-presentation/split-input",
"label": "Split input with AI"
}
]
},
"replicate": {
"description": "A versatile node capable of interfacing with the Replicate API. Explore various model for text, image, audio, 3d model generation ! Replicate cost vary per model, and per time usage. Please not that less used models needs a warmup time, for this reason, first launch can be longer. Don't forget to save your file; files are available for 12 hours.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/replicate-demo.gif",
"docUrls": [
{
"url": "https://docs.ai-flow.net/docs/nodes-presentation/replicate-node",
"label": "Access Diverse AI Models through Replicate"
}
]
},
"transition": {
"description": "Use this node to organize your flow. The transition node only transfer output to another node.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/transition-demo.gif",
"docUrls": []
},
"display": {
"description": "This resizeable node can be use to display every output at the size you wish. You can also use it as an intermidiary node.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/display-demo.gif",
"docUrls": []
},
"deepseek-processor": {
"description": "The DeepSeek node is designed to interact with the DeepSeek API, enabling you to access various V3 and R1 models."
},
"generate-number-processor": {
"description": "The Generate Number node is designed to generate a random number within a specified range."
},
"openrouter-processor": {
"description": "OpenRouter provides an OpenAI-compatible completion API to multiple models & providers. \n\nThis node is provided as a User Integration, to use it, please provide your OpenRouter API Key in the Secure Store."
},
"http-get-processor": {
"description": "Send a HTTP GET request with the specified headers."
},
"gpt-image-processor": {
"description": "The GPT Image node is designed to interact with the GPT Image API, enabling you to generate and edit images based on a prompt and reference images."
}
}
================================================
FILE: packages/ui/public/locales/en/tips.json
================================================
{
"tips": [
{
"title": "Getting started with AI-Flow",
"description": "This guide will help you get started with AI-Flow, adding nodes, connecting them, customizing workspace.",
"url": "https://docs.ai-flow.net/blog/getting-started-with-ai-flow/",
"imgUrl": "https://docs.ai-flow.net/img/blog-card-images/app-overview-r.png",
"newFeature": true,
"timeEstimated": "4min read"
},
{
"title": "Replicate Node Usage",
"description": "Seamlessly Integrate Replicate API with AI-FLOW for AI workflow automation.",
"url": "https://docs.ai-flow.net/blog/replicate-node/",
"imgUrl": "https://docs.ai-flow.net/img/page-images/replicate-node/model-popup.png",
"newFeature": true,
"timeEstimated": "2min read"
},
{
"title": "How to Use Subflows",
"description": "This feature allows you to create custom nodes based on your flows. ",
"url": "https://docs.ai-flow.net/docs/pro-features/api-builder/subflow/",
"imgUrl": "https://docs.ai-flow.net/img/page-images/api-builder/subflow-preview-3.png",
"newFeature": true,
"timeEstimated": "2min read"
},
{
"title": "How to Create Loops",
"description": "This feature allows you loop on a Subflow. ",
"url": "https://docs.ai-flow.net/docs/pro-features/api-builder/subflow-loop/",
"imgUrl": "https://docs.ai-flow.net/img/page-images/api-builder/subflow-loop-4.png",
"newFeature": true,
"timeEstimated": "4min read"
},
{
"title": "StabilityAI API with AI-FLOW",
"description": "This integration offer a versatile range of image processing capabilities.",
"url": "https://docs.ai-flow.net/blog/stabilityai-api/",
"imgUrl": "https://docs.ai-flow.net/img/blog-images/stabilityai.png",
"timeEstimated": "2min read"
},
{
"title": "Accessing the API Builder View",
"description": "This view allows you to monitor the current state of the API, learn how to use your API, and more.",
"url": "https://docs.ai-flow.net/docs/pro-features/api-builder/builder-view/",
"imgUrl": "https://docs.ai-flow.net/img/blog-card-images/blog-api-builder-1.png",
"newFeature": true,
"timeEstimated": "3min read"
},
{
"title": "Add Webhooks to your Flow",
"description": "The Webhook Node is a powerful tool that allows you to send outputs as webhooks. ",
"url": "https://docs.ai-flow.net/docs/pro-features/api-builder/webhooks/",
"imgUrl": "https://docs.ai-flow.net/img/blog-card-images/blog-api-builder-1.png",
"newFeature": true,
"timeEstimated": "3min read"
},
{
"title": "Run Flow through API",
"description": "Discover how to create and manage an API around a given Flow to integrate it seamlessly to other tools.",
"url": "https://docs.ai-flow.net/docs/category/api-builder/",
"imgUrl": "https://docs.ai-flow.net/img/blog-card-images/blog-api-builder-1.png",
"newFeature": true,
"timeEstimated": "2min read"
},
{
"title": "Full Documentation",
"description": "The main page of AI-FLOW Documentation, accessible through https://docs.ai-flow.net/",
"url": "https://docs.ai-flow.net/",
"imgUrl": "https://docs.ai-flow.net/img/ai-flow-social-card.png"
}
],
"docAvailable": "Full documentation is available here : ",
"tipsSection": "Tips"
}
================================================
FILE: packages/ui/public/locales/en/tour.json
================================================
{
"firstTimeHere": "First time here?",
"discoverApp": "Unlock tips to make the most of our app in just 15 seconds!",
"iKnowTheApp": "I know the app",
"letsStart": "Let's start!",
"welcomeToAIFLOW": "Welcome to AI-FLOW",
"addNodesWithDragAndDrop": "Easily add nodes to your canvas with a simple drag & drop.",
"dragAndDrop": "Drag and Drop",
"addingNodes": "Adding Nodes",
"runningANode": "Running a Node",
"connectingNodes": "Connecting Nodes",
"runEverything": "Run Everything",
"exploringMoreModels": "Exploring More Models",
"youveGotTheBasics": "You've got the basics!",
"executeSingleNode": "You can execute a single node by clicking the run button.",
"runNode": "Run Node",
"handlesExplanation": "Blue handles are for inputs, and orange handles are for outputs. For GPT Nodes, inputs add context to your prompts.",
"connectNodes": "Connect Nodes",
"executeAllNodesDescription": "This button executes all nodes in your flow, overwriting previous outputs.",
"replicateNodeDescription": "Expand your capabilities with the Replicate Node, providing access to a wide range of models for advanced use-cases.",
"replicateNode": "Replicate Node",
"checkHelpForAdvanced": "For advanced use-cases, check the Help section at the bottom left.",
"configDescription": "Here you can add your API Keys to be able to use the app.",
"config": "Config"
}
================================================
FILE: packages/ui/public/locales/en/version.json
================================================
{
"versionInfo": {
"versionNumber": "v0.7.3",
"description": "Discover the latest features added in version 0.7.3"
},
"features": [
{
"title": "Improved Web Extractor",
"description": "You can now customized how data is extracted."
},
{
"title": "New Action: Help",
"description": "Each node now includes a 'Help' action that enables you to learn how to use it."
}
],
"articles": [
{
"title": "Generate Consistent Characters Using AI - Part 1",
"url": "https://docs.ai-flow.net/blog/generate-consistent-characters-ai/"
},
{
"title": "How to automate story and image creation using AI - Part 2",
"url": "https://docs.ai-flow.net/blog/automate-story-creation-2/"
},
{
"title": "How to use Documents in AI-FLOW",
"url": "https://docs.ai-flow.net/blog/summarize-doc-post/"
}
],
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/gif-v0.7.3.gif",
"newVersionAvailable": "A new version is now available !",
"newVersionDefaultMessage": "New features and bug fixes are available. To access them, please refresh your page.",
"refresh": "Refresh"
}
================================================
FILE: packages/ui/public/locales/fr/aiActions.json
================================================
{
"Summary": "Résumé",
"SpellCheck": "Vérification Orthographique",
"VisualPrompt": "Prompt Visuelle",
"ConstructiveCritique": "Critique Constructive",
"SimpleExplanation": "Explication Simple",
"Paraphrase": "Paraphrase",
"SentimentAnalysis": "Analyse de Sentiment",
"TextExtension": "Extension du Texte",
"ClickToShowOutput": "Cliquez pour afficher le résultat"
}
================================================
FILE: packages/ui/public/locales/fr/config.json
================================================
{
"configurationTitle": "Configuration",
"apiKeyDisclaimer": "Nous n'utilisons ni ne stockons vos clés API.",
"openSourceDisclaimer": "Ce projet est open source.",
"apiKeyRevokeReminder": "N'oubliez pas, vous pouvez révoquer vos clés à tout moment et en générer de nouvelles.",
"closeButtonLabel": "Fermer",
"validateButtonLabel": "Valider",
"likeProjectPrompt": "Si vous aimez ce projet, vous pouvez ajouter une étoile sur:",
"supportProjectPrompt": "Vous pouvez soutenir l'avenir du projet et nous contacter via",
"Logout": "Se déconnecter",
"sections.core": "Paramètres de base",
"parameters.core.openai_api_key": "Clé API OpenAI",
"parameters.core.stabilityai_api_key": "Clé API StabilityAI",
"parameters.core.replicate_api_key": "Clé API Replicate",
"parameters.extension.anthropic_api_key": "Clé API Anthropic",
"sections.extension": "Extensions",
"userTabLabel": "Paramètres utilisateur",
"appParametersLabel": "Paramètres de l'application",
"displayTabLabel": "Paramètres d'affichage",
"nodesDisplayed": "Nœuds activés",
"configUpdated": "Configuration mise à jour avec succès",
"ShowMinimap": "Afficher la minimap",
"UI": "Interface Utilisateur",
"input": "Entrées",
"models": "Modèles",
"tools": "Outils",
"parameters.extension.deepseek_api_key": "Clé API DeepSeek",
"parameters.extension.openrouter_api_key": "Clé API OpenRouter",
"incompleteLoadingPleaseRestart": "L’application n’a pas pu charger toutes les données. Veuillez redémarrer l’application."
}
================================================
FILE: packages/ui/public/locales/fr/dialogs.json
================================================
{
"attachNodeTitle": "Attacher un noeud",
"attachNodeAction": "Attacher"
}
================================================
FILE: packages/ui/public/locales/fr/flow.json
================================================
{
"Flow": "Flow",
"AddTab": "Ajouter un Flow",
"ShowOnlyOutputs": "Afficher uniquement les résultats",
"ShowOnlyParams": "Afficher uniquement les paramètres",
"Prompt": "Prompt",
"ApiKeyRequiredMessage": "Veuillez fournir votre clé API dans les paramètres de configuration pour exécuter correctement votre flow.",
"ClickToShowOutput": "Cliquez pour afficher le résultat",
"RoleInitPrompt": "Indiquez le rôle que vous souhaitez que l'IA adopte lors de vos prochaines interactions. Par exemple, 'Comporte toi comme un Critique Littéraire'.",
"EnterURL": "Extracteur Web",
"YoutubeTranscriptNodeName": "Transcription Youtube",
"URLPlaceholder": "Saisissez une URL",
"Input": "Entrées",
"InputImage": "Image",
"InputPlaceholder": "Saisissez votre texte ici pour qu'il serve d'entrée à d'autres nœuds",
"InputImagePlaceholder": "Saisissez l'URL vers l'image que vous souhaitez utiliser",
"DALLE": "DALL-E",
"ClickToShowImageOutput": "Cliquez pour afficher la sortie d'image",
"JsonView": "Vue JSON",
"TopologicalView": "Résultats",
"currentNodeView": "Noeud selectionné",
"Upload": "Importer",
"Download": "Télécharger",
"Text": "Texte",
"URL": "URL",
"YoutubeVideo": "Transcription Youtube",
"Models": "Modèles",
"GPT": "Modèle GPT",
"GPTPrompt": "GPT Prompt",
"NoContextPrompt": "GPT Prompt",
"PromptPlaceholder": "Entrez votre prompt ici, par exemple 'Crée un thread Twitter basé sur les données que je t'ai envoyées.'",
"MergePromptPlaceholder": "Entrez votre texte pour la fusion en utilisant les constantes ${input-1} et ${input-2}. Vous pouvez ajouter du texte supplémentaire, par exemple : \n Répond à ${input-1} en tenant compte de ${input-2}.",
"VisionPromptPlaceholder": "Entrez votre prompt ici pour analyser l'image en entrée. Par exemple 'Crée une description de cette image'",
"VisionImageURLPlaceholder": "Saisissez l'URL vers l'image que vous souhaitez utiliser",
"DallEPromptPlaceholder": "Entrez votre prompt ici, par exemple 'Un chien et un chat jouant dans le désert'",
"DataSplitter": "Data Splitter",
"ImageGeneration": "Génération d'image",
"AdvancedSection": "Avancé",
"AiAction": "Action IA",
"LLMPrompt": "GPT",
"AiDataSplitter": "Data Splitter",
"MergerNode": "Fusion de Texte",
"ReplicateModel": "Replicate",
"inputHelp": "Ce noeud sert à saisir du texte.",
"inputImageHelp": "Ce noeud permet de visualiser une image via une URL.",
"urlInputHelp": "Saisissez une URL valide et le noeud récupérera les données de cette URL.",
"youtubeTranscriptHelp": "Ce noeud récupère les sous titres d'une vidéo Youtube à partir de son URL",
"gptHelp": "Ce noeud permet de configurer un modèle GPT, de lui préciser son rôle, et les données sur lesquelles il se basera pour répondre.",
"gptPromptHelp": "Ce noeud permet d'interroger un modèle GPT. Il partage son contexte avec les autres noeuds connectés au modèle.",
"noContextPromptHelp": "Ce noeud permet d'interroger GPT sans contexte, juste à partir de données d'entrées et d'un prompt. Pas besoin de le connecter à un modèle.",
"stableDiffusionPromptHelp": "Ce noeud utilise le modèle Stable Diffusion pour générer des images à partir d'une description textuelle.",
"stableVideoDiffusionPromptHelp": "Ce noeud utilise Replicate pour lancer le modèle Stable Video Diffusion.",
"aiActionPromptHelp": "Ce noeud permet de réaliser des actions simples avec GPT-4 sans préciser de prompt.",
"llmPromtHelp": "Ce noeud permet de traiter une prompt avec GPT-3.5 ou GPT-4",
"dataSplitterHelp": "Ce noeud sert à diviser les données en plusieurs parties. Vous pouvez spécifier en amont combien de parties vous voulez créer. Vous pouvez également le lancer unitairement pour qu'il trouve le nombre exact de sorties à générer.",
"replicateHelp": "Ce nœud utilise Replicate pour donner accès à un grand nombre de modèles.",
"mergerPromptHelp": "Ce nœud vous permet de fusionner 2 entrées.",
"gptVisionPromptHelp": "Ce nœud utilise GPT-4 Vision et prend une URL d'image comme entrée.",
"socketConnectionLost": "La connexion a été perdue. \n\n Vous pouvez réessayer plus tard ou installer l'application localement pour éviter ce type de problème à l'avenir.",
"ClickToSelectModel": "Cliquez pour sélectionner un modèle",
"Or": "OU",
"EnterModelNameDirectly": "Entrez directement le nom du modèle",
"Load": "Charger",
"LoadMore": "Charger plus",
"SpotlightModels": "Modèles Vedettes",
"AllModels": "Tous les Modèles",
"EdgeType": "Type d'arête",
"CannotChangeTabWhileRunning": "Vous ne pouvez pas changer d'onglet quand un lancement est en cours.",
"Transition": "Transition",
"transitionHelp": "Ce noeud sert uniquement à organiser le flow.",
"MissingFieldsMessage": "Des champs nécessaires sont manquants",
"Node": "Noeud",
"MissingFields": "Champ manquant",
"CannotDeleteLastFlow": "Impossible de supprimer le dernier Flow",
"HideSidebar": "Cacher la barre",
"ShowSidebar": "Afficher la barre",
"File": "Fichier",
"EnterUrlToDesiredFile": "Entrez l'URL",
"fileUploadHelp": "Utilisez ce noeud pour charger un fichier",
"llmPromptHelp": "Ce noeud permet d'envoyer des prompts à GPT-3.5 ou GPT-4",
"Output": "Résultat",
"Inputs": "Entrées",
"Parameters": "Paramètres",
"Duplicate": "Dupliquer",
"OpeninSidepane": "Ouvrir dans le bandeau",
"ClearOutput": "Supprimer le résultat",
"RemoveNode": "Supprimer le noeud",
"ExpiredURL": "URL expirée",
"NoNodeSelected": "Aucun nœud sélectionné pour le moment.",
"ClickOnNodeToSelectIt": "Veuillez cliquer sur un nœud pour le sélectionner.",
"Field": "Champ",
"DragAndDropNodes": "Glissez et déposez les nœuds pour les ajouter.",
"CopiedToClipboard": "Copié dans le presse-papiers.",
"DocumentToText": "Document vers Texte",
"documentToTextHelp": "Convertir un fichier .pdf, .txt, .csv, .json, .html en texte simple",
"TextToSpeech": "Texte vers Audio",
"textToSpeechHelp": "Convertir un texte en fichier audio en utilisant le modèle tts d'OpenAI",
"error.upload_failed": "Echec de l'upload. Vérifiez votre configuration pour pouvoir activer l'upload.",
"InputTextPlaceholder": "Entrez votre texte ici",
"DownloadFile": "Télécharger le fichier",
"FileUploaded": "Fichier hébergé",
"GenericPromptPlaceholder": "Entrez vos instructions ici",
"GenericNegativePromptPlaceholder": "Entrez vos instructions négatives ici",
"EnterCustomName": "Nom personalisé :",
"NodeColor": "Couleur du noeud",
"ChangeName": "Changer le nom",
"RemoveFlow": "Supprimer le flow",
"HideHint": "Cacher",
"TextDocumentHint": "Veuillez noter que ce nœud fournit les fichiers uniquement sous forme d'URLs. Pour utiliser un document (.pdf, .txt) en format texte, envisagez d'utiliser le nœud 'Document-en-Texte'.",
"Display": "Affichage",
"displayHelp": "Ce noeud redimensionable permet d'afficher du contenu.",
"Validate": "Valider",
"AI": "IA",
"Separator": "Séparateur",
"ClaudeAnthropic": "Claude",
"claudeAnthropichHelp": "Ce noeud utilise le modèle Claude d'Anthropic pour traiter des instructions textuelles.",
"noDataAvailableForThisNode": "Pas de données disponibles pour ce noeud.",
"learnMore": "En apprendre plus :",
"Help": "Aide",
"cookiesConsentLabelPlaceholder": "Accepter tout",
"cookiesConsentLabelHelp": "Pour certaines pages, nous devons cliquer sur le bouton de consentement aux cookies pour accéder aux données. Cette instruction aide à localiser le bouton.",
"EditTextContent": "Editer le texte",
"ShowCoordinates": "Afficher les coordonnées",
"ShowNodesConfig": "Afficher les configurations des noeuds",
"DeleteAll": "Tout supprimer",
"DeleteOutputs": "Supprimer les résultats",
"ReplaceText": "Remplacer Texte",
"ReplaceTextInputPlaceholder": "Entrez le texte complet où le terme sera remplacé.",
"ReplaceTextSearchPlaceholder": "Entrez le terme ou le motif regex à remplacer.",
"ReplaceTextReplacePlaceholder": "Entrez le terme de remplacement.",
"replaceTextNodeHelp": "Utilisez ce nœud pour rechercher et remplacer un texte spécifique ou des motifs dans l'entrée.",
"openaio1Help": "Des modèles de langage avancés formés pour le raisonnement complexe, excellant dans les défis scientifiques, mathématiques et de programmation.",
"ContextPlaceholder": "Contexte additionnel qui sera utilisé pour répondre au prompt",
"deepSeekHelp": "Accédez aux LLMs DeepSeek via ce noeud.",
"openRouterHelp": "OpenRouter donne accès à plusieurs LLMs et providers. Ce noeud nécessite de fournir une clé API.",
"Generate Number": "Générer un nombre",
"generateNumberHelp": "Génère un nombre aléatoire",
"httpGetProcessorURLPlaceholder": "Enter the URL to request",
"httpGetProcessorURLDescription": "The URL that the HTTP GET request will be sent to.",
"httpGetProcessorHeadersPlaceholder": "Enter headers in JSON format",
"httpGetProcessorHeadersDescription": "The headers to include in the HTTP GET request.",
"httpGetProcessorHelp": "Send an HTTP GET request with the specified headers.",
"gptImageHelp": "Generate or Edit an image using GPT Image",
"gptImageMaskDescription": "You can provide a mask to indicate where the image should be edited. You can use the prompt to describe the full new image, not just the erased area. If you provide multiple input images, the mask will be applied to the first image.",
"dallEDeprecated": "Most recent OpenAI models are now available via the new GPT Image node and are superior to DALL-E. DALL-E remains available if needed.",
"TTSInstructionPlaceholder": "Ex : Parlez avec un ton joyeux et positif.",
"TTSInstructionDescription": "Invitez le modèle à contrôler les aspects de la parole (accent, émotion, intonation, vitesse, ton, ...)",
"PopularModels": "Modèles Populaires",
"removeBackgroundDescription": "Supprimez l'arrière-plan d'une image en utilisant l'API StabilityAI.",
"upscaleFastDescription": "Augmenter la résolution d'une image avec l'API StabilityAI",
"fluxDescription": "Générez une image en utilisant le modèle FLUX.",
"fluxKontextDescription": "Un modèle d’édition d’image basé sur le texte à la pointe de la technologie, offrant des résultats de haute qualité, respectant fidèlement les instructions des prompts et garantissant une transformation cohérente des images via le langage naturel.",
"faceswapDescription": "Échangez des visages entre des images de manière transparente, permettant des remplacements faciaux réalistes et précis.",
"removeBgDescription": "Supprimez le fond d'une image avec le modèle lucataco/remove-bg sur Replicate",
"upscaleDescription": "Modèle Real-ESRGAN pour augmenter la résolution d'une image",
"moondreamDescription": "Moondream est un modèle de vision qui répond aux commandes concernant une image donnée.",
"llamaDescription": "Le modèle de langage phare de Meta, doté de 405 milliards de paramètres, affiné pour la complétion des discussions.",
"imagenDescription": "Google Imagen produit des images époustouflantes, détaillées et précises.",
"recraftSVGDescription": "Recraft SVG propose une génération d'images vectorielles avancée.",
"recraftDescription": "Recraft V3 produit des images époustouflantes, détaillées et précises.",
"video01Description": "Video-01 offre une génération dynamique de vidéos.",
"video01LiveDescription": "Video-01-Live offre une génération dynamique de vidéos, idéale pour l'animation 2D.",
"klingDescription": "Kling propose des solutions robustes de génération vidéo et d'animation, disponibles en versions professionnelle et standard.",
"veo3Description": "Modèle phare de Google, Veo 3, de texte à vidéo, avec audio"
}
================================================
FILE: packages/ui/public/locales/fr/nodeHelp.json
================================================
{
"input-text": {
"description": "Le nœud de texte peut être utilisé pour transférer une entrée de texte à d'autres nœuds.",
"docUrls": [
{
"url": "https://docs.ai-flow.net/docs/nodes-presentation/Input-nodes",
"label": "Ajouter une entrée dans AI-FLOW"
}
]
},
"url_input": {
"description": "Récupère le contenu textuel d'une page web à partir d'une URL.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/web-extractor-demo.gif",
"docUrls": [
{
"url": "https://docs.ai-flow.net/docs/nodes-presentation/Input-nodes#url",
"label": "Ajouter une entrée dans AI-FLOW"
}
]
},
"llm-prompt": {
"description": "Traite les entrées en utilisant les modèles GPT d'OpenAI, qui peuvent comprendre et générer des réponses en fonction du contexte fourni par l'utilisateur.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/gpt-demo.gif",
"docUrls": [
{
"url": "https://docs.ai-flow.net/fr/blog/summarize-doc-post",
"label": "Comment Résumer des Documents ou Poser des Questions en Utilisant AI-FLOW"
},
{
"url": "https://docs.ai-flow.net/fr/blog/summarize-ytb-post",
"label": "Comment Résumer une Vidéo YouTube en Utilisant AI-FLOW"
}
]
},
"gpt-vision": {
"description": "Utilise GPT-4 Vision pour analyser une image.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/vision-demo.gif",
"docUrls": []
},
"youtube_transcript_input": {
"description": "Capture les transcriptions directement depuis l'API de YouTube, permettant un traitement ultérieur tel que la traduction, la résumé, ou l'extraction de mots clés.",
"docUrls": [
{
"url": "https://docs.ai-flow.net/fr/blog/summarize-ytb-post",
"label": "Comment Résumer une Vidéo YouTube en Utilisant AI-FLOW"
}
]
},
"dalle-prompt": {
"description": "Permet aux utilisateurs de créer des descriptions détaillées pour générer des images à l'aide du modèle DALL-E 3, combinant créativité et IA pour un contenu visuel personnalisé. Veuillez noter qu'OpenAI ne permet que 5-7 images par minute. N'oubliez pas de sauvegarder votre fichier; OpenAI héberge les fichiers pendant 1H.",
"docUrls": [
{
"url": "https://docs.ai-flow.net/docs/nodes-presentation/text-to-image-processing",
"label": "Génération d'image dans AI-FLOW"
}
]
},
"stable-diffusion-stabilityai-prompt": {
"description": "Modèle Stable Diffusion SDXL par Stability AI, offrant une génération d'images rapide et à faible coût. N'oubliez pas de sauvegarder votre fichier ; les fichiers sont disponibles pendant 12 heures.",
"docUrls": []
},
"merger-prompt": {
"description": "Utilisé pour combiner deux sorties. Chaque sortie doit être utilisée avec son identifiant spécifique qui sera remplacé dynamiquement, par exemple ${input-1} et ${input-2}. Utilisez les boutons en haut du nœud pour insérer automatiquement les identifiants.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/merge-demo.gif",
"docUrls": []
},
"claude-anthropic-processor": {
"description": "Traite les entrées en utilisant Claude 3 par Anthropic, qui peut comprendre et générer des réponses en fonction du contexte fourni par l'utilisateur.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/claude-demo.gif",
"docUrls": [
{
"url": "https://docs.ai-flow.net/fr/blog/anthropic-claude-api",
"label": "Accédez à Claude 3 via l'API d'Anthropic grâce à AI-FLOW"
}
]
},
"document-to-text-processor": {
"description": "Convertit divers formats de documents en texte brut, permettant l'extraction de texte pour le traitement et l'analyse. Ce nœud prend en charge les formats .pdf, .txt, .json, .html, .csv.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/document-to-text-demo.gif",
"docUrls": [
{
"url": "https://docs.ai-flow.net/fr/blog/summarize-doc-post",
"label": "Comment Résumer des Documents ou Poser des Questions en Utilisant AI-FLOW"
}
]
},
"openai-text-to-speech-processor": {
"description": "Convertit le texte en parole naturelle à l'aide des modèles avancés de synthèse vocale d'OpenAI, facilitant l'accessibilité et les applications multimédia. N'oubliez pas de sauvegarder votre fichier, OpenAI héberge le fichier pendant 1H.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/tts-demo.gif",
"docUrls": []
},
"stabilityai-generic-processor": {
"description": "Un nœud polyvalent capable d'interfacer avec l'API de StabilityAI pour effectuer diverses tâches telles que supprimer l'arrière-plan, rechercher et remplacer, et plus encore. N'oubliez pas de sauvegarder votre fichier ; les fichiers sont disponibles pendant 12 heures.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/stabilityai-demo.gif",
"docUrls": []
},
"stabilityai-stable-diffusion-3-processor": {
"description": "Intègre les dernières capacités de Stable Diffusion 3 pour une génération d'images de haute qualité. N'oubliez pas de sauvegarder votre fichier ; les fichiers sont disponibles pendant 12 heures.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/sd3-demo.gif",
"docUrls": [
{
"url": "https://docs.ai-flow.net/fr/blog/stable-diffusion-3-api",
"label": "Access Stable Diffusion 3 API through AI-FLOW"
}
]
},
"file": {
"description": "Permet d'héberger un fichier et de retourner une URL permettant d'y accéder. Supporte divers types de fichiers pour une utilisation au sein du système. Ce nœud n'extrait pas le contenu des fichiers. Les fichiers sont disponibles pendant 12H.",
"docUrls": [
{
"url": "https://docs.ai-flow.net/fr/blog/summarize-doc-post",
"label": "Comment Résumer des Documents ou Poser des Questions en Utilisant AI-FLOW"
}
]
},
"ai-data-splitter": {
"description": "Divise une entrée en plusieurs sorties en utilisant deux modes disponibles : mode AI et mode Manuel. En mode Manuel, vous devez spécifier un séparateur. Cela peut être utile pour générer du contenu basé sur une liste d'idées ou de concepts. Vous pouvez spécifier un nombre estimé de sorties pour préparer votre flux en conséquence.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/splitter-demo.gif",
"docUrls": [
{
"url": "https://docs.ai-flow.net/docs/nodes-presentation/split-input",
"label": "Séparer une entrée avec l'IA"
}
]
},
"replicate": {
"description": "Un nœud polyvalent capable d'interfacer avec l'API Replicate. Explorez divers modèles pour la génération de texte, d'images, d'audio, de modèles 3D. Les fichiers en sortie sont accessibles 12H.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/replicate-demo.gif",
"docUrls": [
{
"url": "https://docs.ai-flow.net/docs/nodes-presentation/replicate-node",
"label": "Accéder a divers modèles IA via Replicate"
}
]
},
"transition": {
"description": "Utilisez ce nœud pour organiser votre flux. Le nœud de transition ne transfère la sortie qu'à un autre nœud.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/transition-demo.gif",
"docUrls": []
},
"display": {
"description": "Ce nœud redimensionnable peut être utilisé pour afficher chaque sortie à la taille que vous souhaitez. Vous pouvez également l'utiliser comme un nœud intermédiaire.",
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/node-help-img/display-demo.gif",
"docUrls": []
},
"deepseek-processor": {
"description": "Le nœud DeepSeek est conçu pour interagir avec l'API DeepSeek, vous permettant d'accéder à différents modèles tels que V3 et R1."
},
"openrouter-processor": {
"description": "OpenRouter donne accès à plusieurs LLMs et providers. Ce noeud nécessite de fournir une clé API."
},
"generate-number-processor": {
"description": "Le nœud « Générer un nombre » est conçu pour générer un nombre aléatoire dans une plage spécifiée."
},
"http-get-processor": {
"description": "Ce noeud permet d'envoyer une requête HTTP GET sur l'URL spécifiée, avec les headers souhaités."
},
"gpt-image-processor": {
"description": "Le noeud GPT Image vous permet de générer et de modifier des images à partir d'une instruction et d'images de référence. "
}
}
================================================
FILE: packages/ui/public/locales/fr/tips.json
================================================
{
"tips": [
{
"title": "Bien débuter sur AI-Flow",
"description": "Ce guide vous montrera l'essentiel pour bien débuter.",
"url": "https://docs.ai-flow.net/fr/blog/getting-started-with-ai-flow/",
"imgUrl": "https://docs.ai-flow.net/img/blog-card-images/app-overview-r.png",
"newFeature": true,
"timeEstimated": "4min de lecture"
},
{
"title": "Utilisation du noeud Replicate",
"description": "Intégrez l'API Replicate avec AI-FLOW.",
"url": "https://docs.ai-flow.net/fr/blog/replicate-node/",
"imgUrl": "https://docs.ai-flow.net/img/page-images/replicate-node/model-popup.png",
"newFeature": true,
"timeEstimated": "2min de lecture"
},
{
"title": "Comment utiliser les sous-flow",
"description": "Cette fonctionnalité vous permet de créer des nœuds personnalisés basés sur vos flows.",
"url": "https://docs.ai-flow.net/docs/pro-features/api-builder/subflow/",
"imgUrl": "https://docs.ai-flow.net/img/page-images/api-builder/subflow-preview-3.png",
"newFeature": true,
"timeEstimated": "2min de lecture"
},
{
"title": "Comment créer des boucles",
"description": "Cette fonctionnalité vous permet d'itérer sur un sous-flow.",
"url": "https://docs.ai-flow.net/docs/pro-features/api-builder/subflow-loop/",
"imgUrl": "https://docs.ai-flow.net/img/page-images/api-builder/subflow-loop-4.png",
"newFeature": true,
"timeEstimated": "4min read"
},
{
"title": "StabilityAI avec AI-FLOW",
"description": "Cette intégration offre une gamme polyvalente de capacités de traitement d'image.",
"url": "https://docs.ai-flow.net/fr/blog/stabilityai-api/",
"imgUrl": "https://docs.ai-flow.net/img/blog-card-images/blog-sd3.png",
"timeEstimated": "3min de lecture"
},
{
"title": "Vue API Builder",
"description": "Cette vue vous permet de surveiller l'état actuel de l'API, d'apprendre à utiliser votre API, et plus encore.",
"url": "https://docs.ai-flow.net/docs/pro-features/api-builder/builder-view/",
"imgUrl": "https://docs.ai-flow.net/img/blog-card-images/blog-api-builder-1.png",
"newFeature": true,
"timeEstimated": "3min de lecture"
},
{
"title": "Ajouter des Webhooks à vos Flows",
"description": "Le nœud Webhook est un outil puissant qui vous permet d'envoyer des sorties sous forme de webhooks.",
"url": "https://docs.ai-flow.net/docs/pro-features/api-builder/webhooks/",
"imgUrl": "https://docs.ai-flow.net/img/blog-card-images/blog-api-builder-1.png",
"newFeature": true,
"timeEstimated": "2min de lecture"
},
{
"title": "Lancer un Flow via l'API",
"description": "Découvrez comment créer et gérer une API autour d'un Flow donné pour l'intégrer parfaitement à d'autres outils.",
"url": "https://docs.ai-flow.net/docs/category/api-builder/",
"imgUrl": "https://docs.ai-flow.net/img/blog-card-images/blog-api-builder-1.png",
"newFeature": true,
"timeEstimated": "2min de lecture"
},
{
"title": "Documentation Complète",
"description": "La page principale de la documentation AI-FLOW, accessible via https://docs.ai-flow.net/",
"url": "https://docs.ai-flow.net/",
"imgUrl": "https://docs.ai-flow.net/img/ai-flow-social-card.png"
}
],
"docAvailable": "La documentation complète est disponible ici : ",
"tipsSection": "Astuces"
}
================================================
FILE: packages/ui/public/locales/fr/tour.json
================================================
{
"firstTimeHere": "Première visite ?",
"discoverApp": "Découvrez des astuces pour profiter pleinement de notre application en moins de 15 secondes !",
"iKnowTheApp": "Je connais l'application",
"letsStart": "Commençons !",
"welcomeToAIFLOW": "Bienvenue sur AI-FLOW",
"addNodesWithDragAndDrop": "Ajoutez facilement des nœuds à votre canevas avec un simple glisser-déposer.",
"dragAndDrop": "Glisser-Déposer",
"addingNodes": "Ajouter un Nœud",
"runningANode": "Exécuter un Nœud",
"connectingNodes": "Connecter des Nœuds",
"runEverything": "Tout Exécuter",
"exploringMoreModels": "Explorer Plus de Modèles",
"youveGotTheBasics": "Vous avez les bases !",
"executeSingleNode": "Vous pouvez exécuter un seul nœud en cliquant sur le bouton d'exécution.",
"runNode": "Exécuter un Nœud",
"handlesExplanation": "Les poignées bleues sont pour les entrées, et les poignées oranges pour les sorties. Pour les Nœuds GPT, les entrées ajoutent du contexte à vos invites.",
"connectNodes": "Connecter les Nœuds",
"executeAllNodesDescription": "Ce bouton exécute tous les nœuds dans votre flux, en écrasant les sorties précédentes.",
"replicateNodeDescription": "Étendez vos capacités avec le Nœud Replicate, offrant un accès à une large gamme de modèles pour des cas d'usage avancés.",
"replicateNode": "Replicate",
"checkHelpForAdvanced": "Pour des cas d'usage avancés, consultez la section Aide en bas à gauche.",
"configDescription": "Vous pouvez ajouter vos clés APIs via ce menu pour utiliser l'application.",
"config": "Configuration"
}
================================================
FILE: packages/ui/public/locales/fr/version.json
================================================
{
"versionInfo": {
"versionNumber": "v0.7.3",
"description": "Voici les nouveautés de la v0.7.3"
},
"features": [
{
"title": "Amélioration de l'extracteur Web",
"description": "Vous pouvez désormais mieux cibler l'extraction de données."
},
{
"title": "Nouvelle action : Aide",
"description": "Chaque noeud possède désormais une action 'Aide' qui vous permet de découvrir comment l'utiliser."
}
],
"articles": [
{
"title": "Générer des Personnages Cohérents avec l'IA - Partie 1",
"url": "https://docs.ai-flow.net/fr/blog/generate-consistent-characters-ai/"
},
{
"title": "Comment automatiser la création d'histoires et d'images à l'aide de l'IA - Partie 2",
"url": "https://docs.ai-flow.net/fr/blog/automate-story-creation-2/"
},
{
"title": "Comment Utiliser des Documents dans AI-FLOW",
"url": "https://docs.ai-flow.net/fr/blog/summarize-doc-post/"
}
],
"imageUrl": "https://ai-flow-public-assets.s3.eu-west-3.amazonaws.com/gif-v0.7.3.gif",
"newVersionAvailable": "Une nouvelle version est maintenant disponible !",
"newVersionDefaultMessage": "De nouvelles fonctionnalités et des corrections de bugs sont disponibles, pour y accéder, veuillez rafraîchir votre page.",
"refresh": "Rafraîchir"
}
================================================
FILE: packages/ui/public/robots.txt
================================================
# https://www.robotstxt.org/robotstxt.html
User-agent: *
Disallow:
================================================
FILE: packages/ui/public/samples/intro.json
================================================
[
{
"inputs": [],
"name": "3jexlwros#llm-prompt",
"processorType": "llm-prompt",
"model": "gpt-4o",
"x": -1130.048690482733,
"y": -885.266525660136
}
]
================================================
FILE: packages/ui/public/site.webmanifest
================================================
{"name":"","short_name":"","icons":[{"src":"/android-chrome-192x192.png","sizes":"192x192","type":"image/png"},{"src":"/android-chrome-512x512.png","sizes":"512x512","type":"image/png"}],"theme_color":"#ffffff","background_color":"#ffffff","display":"standalone"}
================================================
FILE: packages/ui/src/App.tsx
================================================
import { useContext, useEffect, useMemo, useState } from "react";
import FlowTabs, { FlowTab } from "./layout/main-layout/AppLayout";
import { ThemeContext } from "./providers/ThemeProvider";
import { DndProvider } from "react-dnd";
import { MultiBackend } from "react-dnd-multi-backend";
import { HTML5toTouch } from "rdndmb-html5-to-touch";
import { AppTour } from "./components/tour/AppTour";
import { VisibilityProvider } from "./providers/VisibilityProvider";
import { Tooltip } from "react-tooltip";
import { loadExtensions } from "./nodes-configuration/nodeConfig";
import { loadAllNodesTypes } from "./utils/mappings";
import { loadParameters } from "./components/popups/config-popup/parameters";
import { SocketProvider } from "./providers/SocketProvider";
import { getAllTabs } from "./services/tabStorage";
import { convertJsonToFlow } from "./utils/flowUtils";
import { UserMessage } from "./components/popups/UserMessagePopup";
import { useTranslation } from "react-i18next";
interface AppProps {
onLoadingComplete: () => void;
}
const App = ({ onLoadingComplete }: AppProps) => {
const { dark } = useContext(ThemeContext);
const [runTour, setRunTour] = useState(false);
const [configLoaded, setConfigLoaded] = useState(false);
const [showApp, setShowApp] = useState(false);
const [allTabs, setAllTabs] = useState([]);
const [fixedAlert, setFixedAlert] = useState(null);
const { t } = useTranslation("config");
const [appMounted, setComponentsMounted] = useState(false);
useEffect(() => {
if (dark) {
document.body.classList.add("dark-theme");
} else {
document.body.classList.remove("dark-theme");
}
}, [dark]);
useEffect(() => {
loadAppData();
}, []);
useEffect(() => {
if (showApp) {
setComponentsMounted(true);
}
}, [showApp]);
const loadIntroFile = async () => {
const firstVisit = localStorage.getItem("firstVisit") !== "false";
const savedFlowTabs = localStorage.getItem("flowTabs");
if (firstVisit && !savedFlowTabs) {
try {
const response = await fetch("/samples/intro.json");
if (!response.ok) {
throw new Error("Failed to fetch intro file");
}
const jsonData = await response.json();
const defaultTab: FlowTab = convertJsonToFlow(jsonData);
localStorage.setItem("firstVisit", "false");
return [defaultTab];
} catch (error) {
console.error("Cannot load sample file :", error);
}
}
return [];
};
async function loadAppData() {
try {
await loadParameters();
await loadExtensions();
const defaultTabs = await loadIntroFile();
const allTabs = await getAllTabs();
if (allTabs.length === 0) {
allTabs.push(...defaultTabs);
}
setAllTabs(allTabs);
} catch (error) {
console.error("Failed to load app data:", error);
console.error("Default parameters will be loaded");
setFixedAlert({
content: t("incompleteLoadingPleaseRestart"),
});
} finally {
loadAllNodesTypes();
setConfigLoaded(true);
setShowApp(true);
onLoadingComplete();
}
}
return (
<>
{configLoaded && (
{/* Only show the "Show More" button if the component hasn't been displayed yet */}
{!!selectSubnodeComponent && !showMore && (
)}
{/* Render the component returned by selectSubnodeComponent when showMore is true */}
{!!selectSubnodeComponent && showMore && (
),
ReplicateLogo: () => (
),
YoutubeLogo: () => (
),
AnthropicLogo: () => (
),
StabilityAILogo: () => (
),
AirTableLogo: () => (
),
OpenRouterLogo: () => (
),
FaGoogle,
ZapierIcon: () => 
),
DeepSeekLogo: () => (
),
GeminiIcon: () => (
),
FaRandom,
GiPerspectiveDiceSix,
};
export const getIconComponent = (type: string) => ICON_MAP[type];
================================================
FILE: packages/ui/src/components/nodes/utils/TextareaModal.tsx
================================================
import { Button, Textarea } from "@mantine/core";
import { ChangeEvent, useState } from "react";
import ReactDOM from "react-dom";
import { useTranslation } from "react-i18next";
import { FaTimes } from "react-icons/fa";
interface TextareaModalProps {
initValue: string;
onChange: (value: string) => void;
onClose: () => void;
fieldName?: string;
}
export function TextareaModal({
initValue,
fieldName,
onChange,
onClose,
}: TextareaModalProps) {
const { t } = useTranslation("flow");
const [value, setValue] = useState
{article.newFeature && (
