|
================================================
FILE: README_ja.md
================================================
# SenseVoice
「[简体中文](./README_zh.md)」|「[English](./README.md)」|「日本語」
SenseVoiceは、音声認識（ASR）、言語識別（LID）、音声感情認識（SER）、および音響イベント分類（AEC）または音響イベント検出（AED）を含む音声理解能力を備えた音声基盤モデルです。本プロジェクトでは、SenseVoiceモデルの紹介と、複数のタスクテストセットでのベンチマーク、およびモデルの体験に必要な環境のインストールと推論方法を提供します。
|
================================================
FILE: README_zh.md
================================================
# SenseVoice
「简体中文」|「[English](./README.md)」|「[日本語](./README_ja.md)」
SenseVoice 是具有音频理解能力的音频基础模型，包括语音识别（ASR）、语种识别（LID）、语音情感识别（SER）和声学事件分类（AEC）或声学事件检测（AED）。本项目提供 SenseVoice 模型的介绍以及在多个任务测试集上的 benchmark，以及体验模型所需的环境安装的与推理方式。
|
================================================
FILE: api.py
================================================
# Set the device with environment, default is cuda:0
# export SENSEVOICE_DEVICE=cuda:1
import os, re
from fastapi import FastAPI, File, Form, UploadFile
from fastapi.responses import HTMLResponse
from typing_extensions import Annotated
from typing import List
from enum import Enum
import torchaudio
from model import SenseVoiceSmall
from funasr.utils.postprocess_utils import rich_transcription_postprocess
from io import BytesIO
TARGET_FS = 16000
class Language(str, Enum):
auto = "auto"
zh = "zh"
en = "en"
yue = "yue"
ja = "ja"
ko = "ko"
nospeech = "nospeech"
model_dir = "iic/SenseVoiceSmall"
m, kwargs = SenseVoiceSmall.from_pretrained(model=model_dir, device=os.getenv("SENSEVOICE_DEVICE", "cuda:0"))
m.eval()
regex = r"<\|.*\|>"
app = FastAPI()
@app.get("/", response_class=HTMLResponse)
async def root():
return """
SenseVoice-Small is an encoder-only speech foundation model designed for rapid voice understanding. It encompasses a variety of features including automatic speech recognition (ASR), spoken language identification (LID), speech emotion recognition (SER), and acoustic event detection (AED). SenseVoice-Small supports multilingual recognition for Chinese, English, Cantonese, Japanese, and Korean. Additionally, it offers exceptionally low inference latency, performing 7 times faster than Whisper-small and 17 times faster than Whisper-large.
Upload an audio file or input through a microphone, then select the task and language. the audio is transcribed into corresponding text along with associated emotions (😊 happy, 😡 angry/exicting, 😔 sad) and types of sound events (😀 laughter, 🎼 music, 👏 applause, 🤧 cough&sneeze, 😭 cry). The event labels are placed in the front of the text and the emotion are in the back of the text.
Recommended audio input duration is below 30 seconds. For audio longer than 30 seconds, local deployment is recommended.
SenseVoice: multilingual speech understanding model
FunASR: fundamental speech recognition toolkit
CosyVoice: high-quality multilingual TTS model