Repository: chenxinfeng4/ffmpegcv
Branch: main
Commit: 1847362ade5f
Files: 22
Total size: 117.1 KB
Directory structure:
gitextract_5kjt031g/
├── .github/
│ └── workflows/
│ └── python-publish.yml
├── .gitignore
├── README.md
├── README_CN.md
├── ffmpegcv/
│ ├── __init__.py
│ ├── ffmpeg_noblock.py
│ ├── ffmpeg_reader.py
│ ├── ffmpeg_reader_camera.py
│ ├── ffmpeg_reader_cuda.py
│ ├── ffmpeg_reader_noblock.py
│ ├── ffmpeg_reader_pannels.py
│ ├── ffmpeg_reader_qsv.py
│ ├── ffmpeg_reader_stream.py
│ ├── ffmpeg_reader_stream_realtime.py
│ ├── ffmpeg_writer.py
│ ├── ffmpeg_writer_noblock.py
│ ├── ffmpeg_writer_qsv.py
│ ├── ffmpeg_writer_stream_realtime.py
│ ├── stream_info.py
│ ├── version.py
│ └── video_info.py
└── setup.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .github/workflows/python-publish.yml
================================================
# This workflow will upload a Python Package using Twine when a release is created
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python#publishing-to-package-registries
# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.
name: Upload Python Package
on:
release:
types: [published]
permissions:
contents: read
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v3
with:
python-version: '3.x'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build
- name: Build package
run: python -m build
- name: Publish package
uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
with:
skip-existing: true
user: __token__
password: ${{ secrets.PYPI_API_TOKEN }}
================================================
FILE: .gitignore
================================================
/build
*.pyc
/dist
/ffmpegcv.egg-info
================================================
FILE: README.md
================================================
# FFMPEGCV is an alternative to OPENCV for video reading&writing.
[](https://pypi.org/project/ffmpegcv/)
[](https://pepy.tech/project/ffmpegcv)
English Version | [中文版本](./README_CN.md)
Here is the Python version of ffmpegcv. For the C++ version, please visit [FFMPEGCV-CPP](https://github.com/chenxinfeng4/ffmpegcv-cpp)
The ffmpegcv provide Video Reader and Video Witer with ffmpeg backbone, which are faster and powerful than cv2. Integrating ffmpegcv into your deeplearning pipeline is very smooth.
- The ffmpegcv is api **compatible** to open-cv.
- The ffmpegcv can use **GPU accelerate** encoding and decoding*.
- The ffmpegcv supports much more video **codecs** v.s. open-cv.
- The ffmpegcv supports **RGB** & BGR & GRAY format as you like.
- The ffmpegcv supports fp32 CHW & HWC format shortcut to CUDA memory.
- The ffmpegcv supports **Stream reading** (IP Camera) in low latency.
- The ffmpegcv supports ROI operations.You can **crop**, **resize** and **pad** the ROI.
In all, ffmpegcv is just similar to opencv api. But it has more codecs and does't require opencv installed at all. It's great for deeplearning pipeline.
<p align="center">
<img src="https://i.imghippo.com/files/cg9641723107581.jpg" width="95%">
</p>
## Functions:
- `VideoWriter`: Write a video file.
- `VideoCapture`: Read a video file.
- `VideoCaptureNV`: Read a video file by NVIDIA GPU.
- `VideoCaptureQSV`: Read a video file by Intel QuickSync Video.
- `VideoCaptureCAM`: Read a camera.
- `VideoCaptureStream`: Read a RTP/RTSP/RTMP/HTTP stream.
- `VideoCaptureStreamRT`: Read a RTSP stream (IP Camera) in real time low latency as possible.
- `noblock`: Read/Write a video file in background using mulitprocssing.
- `toCUDA`: Translate a video/stream as CHW/HWC-float32 format into CUDA device, >2x faster.
## Install
You need to download ffmpeg before you can use ffmpegcv.
```
#1A. LINUX: sudo apt install ffmpeg
#1B. MAC (No NVIDIA GPU): brew install ffmpeg
#1C. WINDOWS: download ffmpeg and add to the path
#1D. CONDA: conda install ffmpeg=6.0.0 #don't use the default 4.x.x version
#2A. python
pip install ffmpegcv #stable verison
pip install git+https://github.com/chenxinfeng4/ffmpegcv #latest verison
#2B. recommand only when you want advanced functions. See the toCUDA section
pip install ffmpegcv[cuda]
```
## When should choose `ffmpegcv` other than `opencv`:
- The `opencv` is hard to install. The ffmpegcv only requires `numpy` and `FFmpeg`, works across Mac/Windows/Linux platforms.
- The `opencv` packages too much image processing toolbox. You just want a simple video/camero IO with GPU accessible.
- The `opencv` didn't support profiling `h264`/`h265` and other video writers.
- You want to **crop**, **resize** and **pad** the video/camero ROI.
- You are interested in deeplearning pipeline.
## Basic example
Read a video by CPU, and rewrite it by GPU.
```python
vidin = ffmpegcv.VideoCapture(vfile_in)
vidout = ffmpegcv.VideoWriterNV(vfile_out, 'h264', vidin.fps) #NVIDIA-GPU
with vidin, vidout:
for frame in vidin:
cv2.imshow('image', frame)
vidout.write(frame)
```
Read the camera.
```python
# by device ID
cap = ffmpegcv.VideoCaptureCAM(0)
# by device name
cap = ffmpegcv.VideoCaptureCAM("Integrated Camera")
```
Deeplearning pipeline.
```python
"""
—————————— NVIDIA GPU accelerating ⤴⤴ ———————
| |
V V
video -> decode -> crop -> resize -> RGB -> CUDA:CHW float32 -> model
"""
cap = ffmpegcv.toCUDA(
ffmpegcv.VideoCaptureNV(file, pix_fmt='nv12', resize=(W,H)),
tensor_format='chw')
for frame_CHW_cuda in cap:
frame_CHW_cuda = (frame_CHW_cuda - mean) / std
result = model(frame_CHW_cuda)
```
## Cross platform
The ffmpegcv is based on Python+FFmpeg, it can cross platform among `Windows, Linux, Mac, X86, Arm`systems.
## GPU Acceleration
- Support **NVIDIA** card only, test in x86_64 only.
- Works in **Windows**, **Linux** and **Anaconda**.
- Works in the **Google Colab** notebook.
- Infeasible in the **MacOS**. That ffmpeg didn't supports NVIDIA at all.
> \* The ffmegcv GPU reader is a bit slower than CPU reader, but much faster when use ROI operations (crop, resize, pad).
## Codecs
| Codecs | OpenCV-reader | ffmpegcv-cpu-r | gpu-r | OpenCV-writer | ffmpegcv-cpu-w | gpu-w |
| ----------- | ------------- | ---------------- | ---- | ------------- | ---------------- | ---- |
| h264 | √ | √ | √ | × | √ | √ |
| h265 (hevc) | not sure | √ | √ | × | √ | √ |
| mjpeg | √ | √ | × | √ | √ | × |
| mpeg | √ | √ | × | √ | √ | × |
| others | not sure | ffmpeg -decoders | × | not sure | ffmpeg -encoders | × |
## Benchmark
*On the way...（maybe never）*
## Video Reader
---
The ffmpegcv is just similar to opencv in api.
```python
# open cv
import cv2
cap = cv2.VideoCapture(file)
while True:
ret, frame = cap.read()
if not ret:
break
pass
# ffmpegcv
import ffmpegcv
cap = ffmpegcv.VideoCapture(file)
while True:
ret, frame = cap.read()
if not ret:
break
pass
cap.release()
# alternative
cap = ffmpegcv.VideoCapture(file)
nframe = len(cap)
for frame in cap:
pass
cap.release()
# more pythonic, recommand
with ffmpegcv.VideoCapture(file) as cap:
nframe = len(cap)
for iframe, frame in enumerate(cap):
if iframe>100: break
pass
```
Use GPU to accelerate decoding. It depends on the video codes.
h264_nvcuvid, hevc_nvcuvid ....
```python
cap_cpu = ffmpegcv.VideoCapture(file)
cap_gpu0 = ffmpegcv.VideoCaptureNV(file) #NVIDIA GPU0
cap_gpu1 = ffmpegcv.VideoCaptureNV(file, gpu=1) #NVIDIA GPU1
cap_qsv = ffmpegcv.VideoCaptureQSV(file) #Intel QSV, experimental
```
Use `rgb24` instead of `bgr24`. The `gray` version would be more efficient.
```python
cap = ffmpegcv.VideoCapture(file, pix_fmt='rgb24') #rgb24, bgr24, gray
ret, frame = cap.read()
plt.imshow(frame)
```
### ROI Operations
You can crop, resize and pad the video. These ROI operation is `ffmpegcv-GPU` > `ffmpegcv-CPU` >> `opencv`.
**Crop** video, which will be much faster than read the whole canvas. The top-left corner is (0, 0).
```python
cap = ffmpegcv.VideoCapture(file, crop_xywh=(0, 0, 640, 480))
```
**Resize** the video to the given size.
```python
cap = ffmpegcv.VideoCapture(file, resize=(640, 480))
```
**Resize** and keep the aspect ratio with black border **padding**.
```python
cap = ffmpegcv.VideoCapture(file, resize=(640, 480), resize_keepratio=True)
```
**Crop** and then **resize** the video.
```python
cap = ffmpegcv.VideoCapture(file, crop_xywh=(0, 0, 640, 480), resize=(512, 512))
```
### Extend Options
**INFILE_OPTIONS**: Add extra options to ffmpeg input.
```python
cap = ffmpegcv.VideoCapture(file, infile_options='-re -stream_loop -1')
# equivalent ffmpeg command
ffmpeg INFILE_OPTIONS -i FILE -f rawvideo pipe:
```
## toCUDA device
---
The ffmpegcv can translate the video/stream from HWC-uint8 cpu to CHW-float32 in CUDA device. It significantly reduce your cpu load, and get >2x faster than your manually convertion.
Prepare your environment. The cuda environment is required. The `pycuda` package is required. The `pytorch` package is non-essential.
> nvcc --version # check you've installed NVIDIA CUDA Compiler. Already installed if
> you've installed Tensorflow-gpu or Pytorch-gpu
>
> pip install ffmpegcv[cuda] #auto install pycuda
```python
# Read a video file to CUDA device, original
cap = ffmpegcv.VideoCaptureNV(file, pix_fmt='rgb24')
ret, frame_HWC_CPU = cap.read()
frame_CHW_CUDA = torch.from_numpy(frame_HWC_CPU).permute(2, 0, 1).cuda().contiguous().float() # 120fps, 1200% CPU load
# speed up
cap = toCUDA(ffmpegcv.VideoCapture(file, pix_fmt='yuv420p')) #pix_fmt: 'yuv420p' or 'nv12' only
cap = toCUDA(ffmpegcv.VideoCaptureNV(file, pix_fmt='nv12')) #'nv12' is better for gpu
cap = toCUDA(vid, tensor_format='chw') #tensor format:'chw'(default) or 'hwc', fp32 precision
cap = toCUDA(vid, gpu=1) #choose gpu
# read to the cuda device
ret, frame_CHW_pycuda = cap.read() #380fps, 200% CPU load, dtype is [pycuda array]
ret, frame_CHW_pycudamem = cap.read_cudamem() #dtype is [pycuda mem_alloc]
ret, frame_CHW_CUDA = cap.read_torch() #dtype is [pytorch tensor]
ret, _ = cap.read_torch(frame_CHW_CUDA) #no copy, but need to specify the output memory
frame_CHW_pycuda[:] = (frame_CHW_pycuda - mean) / std #normalize
```
How can `toCUDA` make it faster in your deeplearning pipeline than `opencv` or `ffmpeg`?
> 1. The opencv/ffmpeg uses the cpu to convert video pix_fmt from original YUV to RGB24, which is slow. The ffmpegcv use the cuda to accelerate pix_fmt convertion.
> 2. Use `yuv420p` or `nv12` can save the cpu load and reduce the memory copy from CPU to GPU.
> 3. The ffmpeg stores the image as HWC format. The ffmpegcv can use HWC & CHW format to accelerate the video reading.
## Video Writer
---
```python
# cv2
out = cv2.VideoWriter('outpy.avi',
cv2.VideoWriter_fourcc('M','J','P','G'),
10,
(w, h))
out.write(frame1)
out.write(frame2)
out.release()
# ffmpegcv, default codec is 'h264' in cpu 'h265' in gpu.
# frameSize is decided by the size of the first frame.
# use the 'mp4/mkv' instead of 'avi' to avoid the codec outdated.
out = ffmpegcv.VideoWriter('outpy.mp4', None, 10)
out.write(frame1)
out.write(frame2)
out.release()
# more pythonic
with ffmpegcv.VideoWriter('outpy.mp4', None, 10) as out:
out.write(frame1)
out.write(frame2)
```
Use GPU to accelerate encoding. Such as h264_nvenc, hevc_nvenc.
```python
out_cpu = ffmpegcv.VideoWriter('outpy.mp4', None, 10)
out_gpu0 = ffmpegcv.VideoWriterNV('outpy.mp4', 'h264', 10) #NVIDIA GPU0
out_gpu1 = ffmpegcv.VideoWriterNV('outpy.mp4', 'hevc', 10, gpu=1) #NVIDIA GPU1
out_qsv = ffmpegcv.VideoWriterQSV('outpy.mp4', 'h264', 10) #Intel QSV, experimental
```
Input image is rgb24 instead of bgr24
```python
out = ffmpegcv.VideoWriter('outpy.mp4', None, 10, pix_fmt='rgb24')
```
Resize the video
```python
out_resz = ffmpegcv.VideoWriter('outpy.mp4', None, 10, resize=(640, 480)) #Resize
```
## Video Reader and Writer
---
```python
import ffmpegcv
vfile_in = 'A.mp4'
vfile_out = 'A_h264.mp4'
vidin = ffmpegcv.VideoCapture(vfile_in)
vidout = ffmpegcv.VideoWriter(vfile_out, None, vidin.fps)
with vidin, vidout:
for frame in vidin:
vidout.write(frame)
```
## Camera Reader
---
**Experimental feature**. The ffmpegcv offers Camera reader. Which is consistent with VideoFiler reader.
- The `VideoCaptureCAM` aims to support ROI operations. The Opencv will be general fascinating than ffmpegcv in camera read. **I recommand the opencv in most camera reading case**.
- The ffmpegcv can use name to retrieve the camera device. Use `ffmpegcv.VideoCaptureCAM("Integrated Camera")` is readable than `cv2.VideoCaptureCAM(0)`.
- The `VideoCaptureCAM` will be laggy and dropping frames if your post-process takes long time. The VideoCaptureCAM will buffer the recent frames.
- The `VideoCaptureCAM` is continously working on background even if you didn't read it. **Please release it in time**.
- Works perfect in Windows, not-perfect in Linux and macOS.
```python
import cv2
cap = cv2.VideoCapture(0)
while True:
ret, frame = cap.read()
cv2.imshow('frame', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
# ffmpegcv, in Windows&Linux
import ffmpegcv
cap = ffmpegcv.VideoCaptureCAM(0)
while True:
ret, frame = cap.read()
cv2.imshow('frame', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
# ffmpegcv use by camera name, in Windows&Linux
cap = ffmpegcv.VideoCaptureCAM("Integrated Camera")
# ffmpegcv use camera path if multiple cameras conflict
cap = ffmpegcv.VideoCaptureCAM('@device_pnp_\\\\?\\usb#vid_2304&'
'pid_oot#media#0001#{65e8773d-8f56-11d0-a3b9-00a0c9223196}'
'\\global')
# ffmpegcv use camera with ROI operations
cap = ffmpegcv.VideoCaptureCAM("Integrated Camera", crop_xywh=(0, 0, 640, 480), resize=(512, 512), resize_keepratio=True)
```
**List all camera devices**
```python
from ffmpegcv.ffmpeg_reader_camera import query_camera_devices
devices = query_camera_devices()
print(devices)
```
>{0: ('Integrated Camera', '@device_pnp_\\\\?\\usb#vid_2304&pid_oot#media#0001#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\\global'),
1: ('OBS Virtual Camera', '@device_sw_{860BB310-5D01-11D0-BD3B-00A0C911CE86}\\{A3FCE0F5-3493-419F-958A-ABA1250EC20B}')}
**Set the camera resolution, fps, vcodec/pixel-format**
```python
from ffmpegcv.ffmpeg_reader_camera import query_camera_options
options = query_camera_options(0) # or query_camera_options("Integrated Camera")
print(options)
cap = ffmpegcv.VideoCaptureCAM(0, **options[-1])
```
>[{'camcodec': 'mjpeg', 'campix_fmt': None, 'camsize_wh': (1280, 720), 'camfps': 60.0002}, {'camcodec': 'mjpeg', 'campix_fmt': None, 'camsize_wh': (640, 480), 'camfps': 60.0002}, {'camcodec': 'mjpeg', 'campix_fmt': None, 'camsize_wh': (1920, 1080), 'camfps': 60.0002}, {'camcodec': None, 'campix_fmt': 'yuyv422', 'camsize_wh': (1280, 720), 'camfps': 10}, {'camcodec': None, 'campix_fmt': 'yuyv422', 'camsize_wh': (640, 480), 'camfps': 30}, {'camcodec': None, 'campix_fmt': 'yuyv422', 'camsize_wh': (1920, 1080), 'camfps': 5}]
**Known issues**
1. The VideoCaptureCAM didn't give a smooth experience in macOS. You must specify all the camera parameters. And the query_camera_options woun't give any suggestion. That's because the `ffmpeg` cannot list device options using mac native `avfoundation`.
```python
# The macOS requires full argument.
cap = ffmpegcv.VideoCaptureCAM('FaceTime HD Camera', camsize_wh=(1280,720), camfps=30, campix_fmt='nv12')
```
2. The VideoCaptureCAM cann't list the FPS in linux. Because the `ffmpeg` cound't query the device's FPS using linux native `v4l2` module. However, it's just OK to let the FPS empty.
## Stream Reader (Live streaming, RTSP IP cameras)
**Experimental feature**. The ffmpegcv offers Stream reader. Which is consistent with VideoFiler reader, and more similiar to the camera.
Becareful when using it.
- Support `RTSP`, `RTP`, `RTMP`, `HTTP`, `HTTPS` streams.
- The `VideoCaptureStream` will be laggy and dropping frames if your post-process takes long time. The VideoCaptureCAM will buffer the recent frames.
- The `VideoCaptureStreamRT` is continously working on background even if you didn't read it. **Please release it in time**.
- **Low latency** RTSP IP camera reader. Batter than opencv.
- *It's still experimental*. Recommand you to use opencv.
```python
# opencv
import cv2
stream_url = 'http://devimages.apple.com.edgekey.net/streaming/examples/bipbop_4x3/gear2/prog_index.m3u8'
cap = cv2.VideoCapture(stream_url, cv2.CAP_FFMPEG)
if not cap.isOpened():
print('Cannot open the stream')
exit(-1)
while True:
ret, frame = cap.read()
if not ret:
break
pass
# ffmpegcv
import ffmpegcv
cap = ffmpegcv.VideoCaptureStream(stream_url)
while True:
ret, frame = cap.read()
if not ret:
break
pass
# ffmpegcv, IP Camera Low-latency
# e.g. HIK Vision IP Camera, `101` Main camera stream, `102` the second
stream_url = 'rtsp://admin:PASSWD@192.168.1.xxx:8554/Streaming/Channels/102'
cap = ffmpegcv.VideoCaptureStreamRT(stream_url) # Low latency & recent buffered
cap = ffmpegcv.ReadLiveLast(ffmpegcv.VideoCaptureStreamRT, stream_url) #no buffer
while True:
ret, frame = cap.read()
if not ret:
break
pass
```
## Noblock
A proxy to automatic prepare frames in backgroud, which does not block when reading&writing current frame (multiprocessing). This make your python program more efficient in CPU usage. Up to 2x boost.
> ffmpegcv.VideoCapture(*args) -> ffmpegcv.noblock(ffmpegcv.VideoCapture, *args)
>
> ffmpegcv.VideoWriter(*args) -> ffmpegcv.noblock(ffmpegcv.VideoWriter, *args)
```python
#Proxy any VideoCapture&VideoWriter args and kargs
vid_noblock = ffmpegcv.noblock(ffmpegcv.VideoCapture, vfile, pix_fmt='rbg24')
# this is fast
def cpu_tense(): time.sleep(0.01)
for _ in tqdm.trange(1000):
ret, img = vid_noblock.read() #current img is already buffered, take no time
cpu_tense() #meanwhile, the next img is buffering in background
# this is slow
vid = ffmpegcv.VideoCapture(vfile, pix_fmt='rbg24')
for _ in tqdm.trange(1000):
ret, img = vid.read() #this read will block cpu, take time
cpu_tense()
```
================================================
FILE: README_CN.md
================================================
# FFMPEGCV 读写视频，替代 OPENCV.
[](https://pypi.org/project/ffmpegcv/)
[](https://pypistats.org/packages/ffmpegcv)
[English Version](./README.md) | 中文版本
ffmpegcv提供了基于ffmpeg的视频读取器和视频编写器，比cv2更快和更强大。适合深度学习的视频处理。
- ffmpegcv与open-cv具有**兼容**的API。
- ffmpegcv可以使用**GPU加速**编码和解码。
- ffmpegcv支持比open-cv更多的**视频编码器**。
- ffmpegcv原生支持**RGB**/BGR/灰度像素格式。
- ffmpegcv支持网络**流视频读取** (网线监控相机)。
- ffmpegcv支持ROI（感兴趣区域）操作，可以对ROI进行**裁剪**、**调整大小**和**填充**。
总之，ffmpegcv与opencv的API非常相似。但它具有更多的编码器，并且不需要安装opencv。
- ffmpegcv支持导出图像帧到CUDA设备。
<p align="center">
<img src="https://i.imghippo.com/files/cg9641723107581.jpg" width="95%">
</p>
## 功能：
- `VideoWriter`：写入视频文件。
- `VideoCapture`：读取视频文件。
- `VideoCaptureNV`：使用NVIDIA GPU读取视频文件。
- `VideoCaptureQSV`: 使用Intel集成显卡读取视频文件.
- `VideoCaptureCAM`：读取摄像头。
- `VideoCaptureStream`：读取RTP/RTSP/RTMP/HTTP流。
- `VideoCaptureStreamRT`: 读取RTSP流 (网线监控相机)，实时、低延迟。
- `noblock`：在后台读取视频文件（更快）,使用多进程。
- `toCUDA`：将图像帧导出到CUDA设备，以 CHW/HWC-float32 格式存储，超过2倍性能提升。
## 安装
在使用ffmpegcv之前，您需要下载`ffmpeg`。
```
#1A. LINUX: sudo apt install ffmpeg
#1B. MAC: brew install ffmpeg
#1C. WINDOWS: 下载ffmpeg并添加至环境变量的路径中
#1D. CONDA: conda install ffmpeg=6.0.0
#2. python
pip install ffmpegcv #stable verison
pip install git+https://github.com/chenxinfeng4/ffmpegcv #latest verison
```
## 何时选择 `ffmpegcv` 而不是 `opencv`：
- 安装`opencv`比较困难。ffmpegcv仅需要`numpy`和`FFmpeg`，可以在Mac/Windows/Linux平台上工作。
- `opencv`包含太多的图像处理工具箱，而您只是想使用带GPU支持的简单视频/摄像头输入输出操作。
- `opencv`不支持`h264`/`h265`和其他视频编码器。
- 您想对视频/摄像头的感兴趣区域（ROI）进行**裁剪**、**调整大小**和**填充**操作。
## 基本示例
通过CPU读取视频，并通过GPU重写视频。
```python
vidin = ffmpegcv.VideoCapture(vfile_in)
vidout = ffmpegcv.VideoWriterNV(vfile_out, 'h264', vidin.fps) #NVIDIA 显卡
with vidin, vidout:
for frame in vidin:
cv2.imshow('image', frame)
vidout.write(frame)
```
读取摄像头。
```python
# 通过设备ID
cap = ffmpegcv.VideoCaptureCAM(0)
# 通过设备名称
cap = ffmpegcv.VideoCaptureCAM("Integrated Camera")
```
深度学习流水线
```python
"""
—————————— NVIDIA GPU 加速 ⤴⤴ ———————
| |
V V
视频 -> 解码器 -> 裁剪 -> 缩放 -> RGB -> CUDA:CHW float32 -> 模型
"""
cap = ffmpegcv.toCUDA(
ffmpegcv.VideoCaptureNV(file, pix_fmt='nv12', resize=(W,H)),
tensor_format='chw')
for frame_CHW_cuda in cap:
frame_CHW_cuda = (frame_CHW_cuda - mean) / std
result = model(frame_CHW_cuda)
```
## GPU加速
- 仅支持NVIDIA显卡，在 x86_64 上测试。
- 原生支持**Windows**, **Linux**, **Anaconda**。
- 在**Google Colab**上顺利运行。
- 在**MacOS**仅能使用CPU功能，上无法进行GPU加速，因为Mac根本就不支持NVIDIA。
> 在CPU数量充足的条件下，GPU读取速度可能比CPU读取速度稍慢。在使用感兴趣区域（ROI）操作（裁剪、调整大小、填充）时，GPU优势更凸显。
## 编解码器
| 编解码器 | OpenCV读取器 | ffmpegcv-CPU读取器 | GPU读取器 | OpenCV写入器 | ffmpegcv-CPU写入器 | GPU写入器 |
| ----------- | ------------- | ---------------- | ---- | ------------- | ---------------- | ---- |
| h264 | √ | √ | √ | × | √ | √ |
| h265 (hevc) | 不确定 | √ | √ | × | √ | √ |
| mjpeg | √ | √ | × | √ | √ | × |
| mpeg | √ | √ | × | √ | √ | × |
| 其他 | 不确定 | ffmpeg -decoders | × | 不确定 | ffmpeg -encoders | × |
## 基准测试
*正在进行中...(遥遥无期)*
## 视频读取器
---
ffmpegcv与opencv在API上非常类似。
```python
# OpenCV
import cv2
cap = cv2.VideoCapture(file)
while True:
ret, frame = cap.read()
if not ret:
break
pass
# ffmpegcv
import ffmpegcv
cap = ffmpegcv.VideoCapture(file)
while True:
ret, frame = cap.read()
if not ret:
break
pass
cap.release()
# 另一种写法
cap = ffmpegcv.VideoCapture(file)
nframe = len(cap)
for frame in cap:
pass
cap.release()
# 更加Pythonic的写法，推荐使用
with ffmpegcv.VideoCapture(file) as cap:
nframe = len(cap)
for iframe, frame in enumerate(cap):
if iframe>100: break
pass
```
使用GPU加速解码。具体取决于视频编码格式。
h264_nvcuvid, hevc_nvcuvid ....
```python
cap_cpu = ffmpegcv.VideoCapture(file)
cap_gpu = ffmpegcv.VideoCapture(file, codec='h264_cuvid') # NVIDIA GPU0
cap_gpu0 = ffmpegcv.VideoCaptureNV(file) # NVIDIA GPU0
cap_gpu1 = ffmpegcv.VideoCaptureNV(file, gpu=1) # NVIDIA GPU1
cap_qsv = ffmpegcv.VideoCaptureQSV(file) #Intel QSV, 测试中
```
使用`rgb24`代替`bgr24`。`gray`版本会更高效。
```python
cap = ffmpegcv.VideoCapture(file, pix_fmt='rgb24') # rgb24, bgr24, gray
ret, frame = cap.read()
plt.imshow(frame)
```
### 感兴趣区域（ROI）操作
您可以对视频进行裁剪、调整大小和填充。这些ROI操作中，`ffmpegcv-GPU` > `ffmpegcv-CPU` >> `opencv` 在性能上。
**裁剪**视频，比读取整个画布要快得多。
```python
cap = ffmpegcv.VideoCapture(file, crop_xywh=(0, 0, 640, 480))
```
将视频调整为给定大小的**大小**。
```python
cap = ffmpegcv.VideoCapture(file, resize=(640, 480))
```
**调整大小**并保持宽高比，使用黑色边框进行**填充**。
```python
cap = ffmpegcv.VideoCapture(file, resize=(640, 480), resize_keepratio=True)
```
对视频进行**裁剪**，然后进行**调整大小**。左上角为坐标 (0,0)。
```python
cap = ffmpegcv.VideoCapture(file, crop_xywh=(0, 0, 640, 480), resize=(512, 512))
```
### 扩展选项
**INFILE_OPTIONS**: 扩展 ffmpeg 输入选项。
```python
cap = ffmpegcv.VideoCapture(file, infile_options='-re -stream_loop -1')
# 等效于下面的 ffmpeg 命令
ffmpeg INFILE_OPTIONS -i FILE -f rawvideo pipe:
```
## toCUDA 将图像帧快速导出到CUDA设备
---
ffmpegcv 可以将 HWC-uint8 cpu 中的视频/流转换为 CUDA 设备中的 CHW-float32。它可以显著减少你的 CPU 负载，并比你的手动转换快 2 倍以上。
准备环境。你需要具备 cuda 环境，并且安装 pycuda 包。注意，pytorch 包是非必须的。
> nvcc --version # 检查你是否已经安装了 NVIDIA CUDA 编译器
> pip install pycuda # 安装 pycuda
```python
# 读取视频到CUDA设备，加速前
cap = ffmpegcv.VideoCaptureNV(file, pix_fmt='rgb24')
ret, frame_HWC_CPU = cap.read()
frame_CHW_CUDA = torch.from_numpy(frame_HWC_CPU).permute(2, 0, 1).cuda().contiguous().float() # 120fps, 1200% CPU 使用率
# 加速后
cap = toCUDA(ffmpegcv.VideoCapture(file, pix_fmt='yuv420p')) #必须设置, yuv420p 针对 cpu
cap = toCUDA(ffmpegcv.VideoCaptureNV(file, pix_fmt='nv12')) #必须设置, nv12 针对 gpu
cap = toCUDA(vid, tensor_format='chw') #tensor 格式:'chw'(默认) or 'hwc'
cap = toCUDA(vid, gpu=1) #选择 gpu
ret, frame_CHW_pycuda = cap.read() #380fps, 200% CPU load, [pycuda array]
ret, frame_CHW_pycudamem = cap.read_cudamem() #same as [pycuda mem_alloc]
ret, frame_CHW_CUDA = cap.read_torch() #same as [pytorch tensor]
ret, _ = cap.read_torch(frame_CHW_CUDA) #不拷贝, 但需要提前分配内存
frame_CHW_pycuda[:] = (frame_CHW_pycuda - mean) / std #归一化
```
为什么在深度学习流水线中使用 toCUDA 会更快？
> 1. ffmpeg 使用 CPU 将视频像素格式从原始 YUV 转换为 RGB24，这个过程很慢。`toCUDA` 使用 cuda 加速像素格式转换。
> 2. 使用 yuv420p 或 nv12 可以节省 CPU 负载并减少从 CPU 到 GPU 的内存复制。
> 3. ffmpeg 将图像存储为 HWC 格式。ffmpegcv 可以使用 HWC 和 CHW 格式来加速视频存储。
## 视频写入器
---
```python
# cv2
out = cv2.VideoWriter('outpy.avi',
cv2.VideoWriter_fourcc('M','J','P','G'),
10,
(w, h))
out.write(frame1)
out.write(frame2)
out.release()
# ffmpegcv，默认的编码器为'h264'在CPU上，'h265'在GPU上。
# 帧大小由第一帧决定
# 使用 'mp4/mkv' 来替代古老的 'avi' 格式
out = ffmpegcv.VideoWriter('outpy.mp4', None, 10)
out.write(frame1)
out.write(frame2)
out.release()
# 更加Pythonic的写法
with ffmpegcv.VideoWriter('outpy.mp4', None, 10) as out:
out.write(frame1)
out.write(frame2)
```
使用GPU加速编码。例如h264_nvenc，hevc_nvenc。
```python
out_cpu = ffmpegcv.VideoWriter('outpy.mp4', None, 10)
out_gpu0 = ffmpegcv.VideoWriterNV('outpy.mp4', 'h264', 10) # NVIDIA GPU0
out_gpu1 = ffmpegcv.VideoWriterNV('outpy.mp4', 'hevc', 10, gpu=1) # NVIDIA GPU1
out_qsv = ffmpegcv.VideoWriterQSV('outpy.mp4', 'h264', 10) #Intel QSV, 测试中
```
输入图像使用rgb24而不是bgr24。
```python
out = ffmpegcv.VideoWriter('outpy.mp4', None, 10, pix_fmt='rgb24')
```
缩放图像尺寸
```python
out_resz = ffmpegcv.VideoWriter('outpy.mp4', None, 10, resize=(640, 480))
```
## 视频读取器和写入器
---
```python
import ffmpegcv
vfile_in = 'A.mp4'
vfile_out = 'A_h264.mp4'
vidin = ffmpegcv.VideoCapture(vfile_in)
vidout = ffmpegcv.VideoWriter(vfile_out, None, vidin.fps)
with vidin, vidout:
for frame in vidin:
vidout.write(frame)
```
## 相机读取器
---
**实验性功能**。ffmpegcv提供了相机读取器。与VideoCapture读取器一致。
- VideoCaptureCAM旨在支持感兴趣区域（ROI）操作。在相机读取方面，Opencv比ffmpegcv更具吸引力。**对于大多数相机读取情况，我推荐使用Opencv**。
- ffmpegcv可以使用名称检索相机设备，使用`ffmpegcv.VideoCaptureCAM("Integrated Camera")`比使用`cv2.VideoCaptureCAM(0)`更易读。
- 如果后处理时间过长，VideoCaptureCAM将会出现卡顿和丢帧。VideoCaptureCAM会缓冲最近的帧。
- 即使没有读取视频帧，VideoCaptureCAM也会在后台不断工作。**请及时释放资源**。
- 在Windows上表现良好，在Linux和macOS上表现不完美。
```python
import cv2
cap = cv2.VideoCapture(0)
while True:
ret, frame = cap.read()
cv2.imshow('frame', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
# ffmpegcv，在Windows和Linux上
import ffmpegcv
cap = ffmpegcv.VideoCaptureCAM(0)
while True:
ret, frame = cap.read()
cv2.imshow('frame', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
# ffmpegcv 使用相机名称，在Windows和Linux上
cap = ffmpegcv.VideoCaptureCAM("Integrated Camera")
# ffmpegcv 使用相机路径（避免多个相机冲突）
cap = ffmpegcv.VideoCaptureCAM('@device_pnp_\\\\?\\usb#vid_2304&'
'pid_oot#media#0001#{65e8773d-8f56-11d0-a3b9-00a0c9223196}'
'\\global')
# ffmpegcv 使用具有ROI操作的相机
cap = ffmpegcv.VideoCaptureCAM("Integrated Camera", crop_xywh=(0, 0, 640, 480), resize=(512, 512), resize_keepratio=True)
```
**列出所有相机设备**
```python
from ffmpegcv.ffmpeg_reader_camera import query_camera_devices
devices = query_camera_devices()
print(devices)
```
>{0: ('Integrated Camera', '@device_pnp_\\\\?\\usb#vid_2304&pid_oot#media#0001#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\\global'),
1: ('OBS Virtual Camera', '@device_sw_{860BB310-5D01-11D0-BD3B-00A0C911CE86}\\{A3FCE0F5-3493-419F-958A-ABA1250EC20B}')}
**设置相机的分辨率、帧率、视频编码/像素格式**
```python
from ffmpegcv.ffmpeg_reader_camera import query_camera_options
options = query_camera_options(0) # 或者 query_camera_options("Integrated Camera")
print(options)
cap = ffmpegcv.VideoCaptureCAM(0, **options[-1])
```
>[{'camcodec': 'mjpeg', 'campix_fmt': None, 'camsize_wh': (1280, 720), 'camfps': 60.0002}, {'camcodec': 'mjpeg', 'campix_fmt': None, 'camsize_wh': (640, 480), 'camfps': 60.0002}, {'camcodec': 'mjpeg', 'campix_fmt': None, 'camsize_wh': (1920, 1080), 'camfps': 60.0002}, {'camcodec': None, 'campix_fmt': 'yuyv422', 'camsize_wh': (1280, 720), 'camfps': 10}, {'camcodec': None, 'campix_fmt': 'yuyv422', 'camsize_wh': (640, 480), 'camfps': 30}, {'camcodec': None, 'campix_fmt': 'yuyv422', 'camsize_wh': (1920, 1080), 'camfps': 5}]
**已知问题**
1. VideoCaptureCAM在macOS上的体验不太流畅。你必须指定所有相机参数。而且query_camera_options不会给出任何建议。这是因为`ffmpeg`无法使用mac本机的`avfoundation`列出设备选项。
```python
# macOS需要提供完整参数。
cap = ffmpegcv.VideoCaptureCAM('FaceTime HD Camera', camsize_wh=(1280,720), camfps=30, campix_fmt='nv12')
```
2. 在Linux上VideoCaptureCAM无法列出FPS，因为`ffmpeg`无法使用Linux本机的`v4l2`模块查询设备的FPS。不过，让FPS为空也没问题。
## 流读取器 （直播流，网络监控摄像头）
**实验性功能**。ffmpegcv提供了流读取器，与VideoFile读取器一致，更类似于相机。
- 支持`RTSP`、`RTP`、`RTMP`、`HTTP`、`HTTPS`流。
- 如果后处理时间过长，VideoCaptureStream会出现卡顿和丢帧。VideoCaptureCAM会缓冲最近的帧。
- 即使没有读取视频帧，VideoCaptureStream也会在后台不断工作。**请及时释放资源**。
- 这仍然是实验性功能。建议您使用opencv。
```python
# opencv
import cv2
stream_url = 'http://devimages.apple.com.edgekey.net/streaming/examples/bipbop_4x3/gear2/prog_index.m3u8'
cap = cv2.VideoCapture(stream_url, cv2.CAP_FFMPEG)
if not cap.isOpened():
print('无法打开流')
exit(-1)
while True:
ret, frame = cap.read()
if not ret:
break
pass
# ffmpegcv
import ffmpegcv
cap = ffmpegcv.VideoCaptureStream(stream_url)
while True:
ret, frame = cap.read()
if not ret:
break
pass
# ffmpegcv, 网络监控摄像头
# 例如 海康威视, `101` 主视频流, `102` 子视频流
stream_url = 'rtsp://admin:PASSWD@192.168.1.xxx:8554/Streaming/Channels/102'
cap = ffmpegcv.VideoCaptureStreamRT(stream_url) # 低延迟 & 缓存
cap = ffmpegcv.ReadLiveLast(ffmpegcv.VideoCaptureStreamRT, stream_url) #不缓存
while True:
ret, frame = cap.read()
if not ret:
break
pass
```
## FFmpegReaderNoblock
更快的读写取视频。利用多进程在后台自动准备帧，这样在读写当前帧时不会阻塞。这使得您的Python程序在CPU使用方面更高效。带来最大翻倍效率提升。
> ffmpegcv.VideoCapture(*args) -> ffmpegcv.noblock(ffmpegcv.VideoCapture, *args)
>
> ffmpegcv.VideoWriter(*args) -> ffmpegcv.noblock(ffmpegcv.VideoWriter, *args)
```python
# 代理任何 VideoCapture&VideoWriter 的参数和kargs
vid_noblock = ffmpegcv.noblock(ffmpegcv.VideoCapture, vfile, pix_fmt='rbg24')
# 这很快
def cpu_tense(): time.sleep(0.01)
for _ in tqdm.trange(1000):
ret, img = vid_noblock.read() #当前图像已经被缓冲，不会占用时间
cpu_tense() #同时，下一帧在后台缓冲
# 这很慢
vid = ffmpegcv.VideoCapture(vfile, pix_fmt='rbg24')
for _ in tqdm.trange(2000):
ret, img = vid.read() #此读取将阻塞CPU，占用时间
cpu_tense()
```
================================================
FILE: ffmpegcv/__init__.py
================================================
from .ffmpeg_reader import FFmpegReader, FFmpegReaderNV
from .ffmpeg_writer import FFmpegWriter, FFmpegWriterNV
from .ffmpeg_reader_camera import FFmpegReaderCAM
from .ffmpeg_reader_stream import FFmpegReaderStream
from .ffmpeg_reader_stream_realtime import FFmpegReaderStreamRT, FFmpegReaderStreamRTNV
from .ffmpeg_writer_stream_realtime import FFmpegWriterStreamRT
from .ffmpeg_reader_qsv import FFmpegReaderQSV
from .ffmpeg_writer_qsv import FFmpegWriterQSV
from .ffmpeg_reader_pannels import FFmpegReaderPannels
from .ffmpeg_noblock import noblock, ReadLiveLast
from .video_info import get_num_NVIDIA_GPUs
import shutil
from subprocess import DEVNULL, check_output
from .version import __version__
def _check():
if not shutil.which("ffmpeg") or not shutil.which("ffprobe"):
raise RuntimeError(
"The ffmpeg is not installed. \n\n"
"Please install ffmpeg via:\n "
"conda install ffmpeg"
)
_check()
_check_nvidia_init = None
def _check_nvidia():
global _check_nvidia_init
run = lambda x: check_output(x, shell=True, stderr=DEVNULL)
if _check_nvidia_init is None:
calling_output = run("ffmpeg -h encoder=hevc_nvenc")
if "AVOptions" not in calling_output.decode("utf-8"):
raise RuntimeError(
"The ffmpeg is not compiled with NVENC support.\n\n"
"Please re-compile ffmpeg following the instructions at:\n "
"https://docs.nvidia.com/video-technologies/video-codec-sdk/ffmpeg-with-nvidia-gpu/"
)
calling_output = run("ffmpeg -h decoder=hevc_cuvid")
if "AVOptions" not in calling_output.decode("utf-8"):
raise RuntimeError(
"The ffmpeg is not compiled with NVENC support.\n\n"
"Please re-compile ffmpeg following the instructions at:\n "
"https://docs.nvidia.com/video-technologies/video-codec-sdk/ffmpeg-with-nvidia-gpu/"
)
if get_num_NVIDIA_GPUs() == 0:
raise RuntimeError(
"No NVIDIA GPU found.\n\n"
"Please use a NVIDIA GPU card listed at:\n "
"https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new"
)
_check_nvidia_init = True
return True
def VideoCapture(
file,
codec=None,
pix_fmt="bgr24",
crop_xywh=None,
resize=None,
resize_keepratio=True,
resize_keepratioalign="center",
infile_options=None
) -> FFmpegReader:
"""
Alternative to cv2.VideoCapture
Parameters
----------
file : str
Path to video file.
codec : str
Codec to use. Optional. Default is `None`.
pix_fmt : str
Pixel format. ['bgr24' | 'rgb24']. Optional. Default is 'bgr24'.
crop_xywh : tuple
Crop the frame. (x, y, width, height). Optional. Default is `None`.
resize : tuple
Resize the video to the given size. Optional. Default is `None`.
resize_keepratio : bool
Keep the aspect ratio and the border is black. Optional. Default is `True`.
resize_keepratioalign : str
Align the image to the `center`, `topleft`, `topright`, `bottomleft` or
`bottomright`. Optional. Default is 'center'.
infile_options : str
Additional options for ffmpeg. Optional. Default is `None`.
Examples
--------
opencv
```
cap = cv2.VideoCapture(file)
while True:
ret, frame = cap.read()
if not ret:
break
pass
```
ffmpegcv
```
cap = ffmpegcv.VideoCapture(file)
while True:
ret, frame = cap.read()
if not ret:
break
pass
```
Or use iterator
```
cap = ffmpegcv.VideoCapture(file)
for frame in cap:
pass
counts = len(cap)
```
Use GPU to accelerate decoding
```
cap_cpu = ffmpegcv.VideoCapture(file)
cap_gpu = ffmpegcv.VideoCaptureNV(file)
```
Use rgb24 instead of bgr24
```
cap = ffmpegcv.VideoCapture(file, pix_fmt='rgb24')
```
Crop video.
```python
cap = ffmpegcv.VideoCapture(file, crop_xywh=(0, 0, 640, 480))
```
Resize the video to the given size
```
cap = ffmpegcv.VideoCapture(file, resize=(640, 480))
```
Resize and keep the aspect ratio with black border
```
cap = ffmpegcv.VideoCapture(file, resize=(640, 480), resize_keepratio=True)
```
Crop and then resize the video.
```python
cap = ffmpegcv.VideoCapture(file, crop_xywh=(0, 0, 640, 480), resize=(512, 512))
```
Author: Chenxinfeng 2022-04-16, cxf529125853@163.com
"""
return FFmpegReader.VideoReader(
file, codec, pix_fmt, crop_xywh, resize, resize_keepratio, resize_keepratioalign, infile_options
)
VideoReader = VideoCapture
def VideoWriter(
file, codec=None, fps=30, pix_fmt="bgr24", bitrate=None, resize=None, preset=None
) -> FFmpegWriter:
"""
Alternative to cv2.VideoWriter
Parameters
----------
file : str
Path to video file.
codec : str
Codec to use. Optional. Default is `None` (x264).
fps : number
Frames per second. Optional. Default is 30.
pix_fmt : str
Pixel format of input. ['bgr24' | 'rgb24']. Optional. Default is 'bgr24'.
bitrate : str
Bitrate of output video. Optional. Default is `None`.
resize : tuple
Frame size of output. (width, height). Optional. Default is `None`.
preset : str
Preset of ffmpeg. Optional. Default is `None`.
Examples
--------
opencv
```
out = cv2.VideoWriter('outpy.avi',
cv2.VideoWriter_fourcc('M','J','P','G'),
10,
(w, h))
out.write(frame1)
out.write(frame2)
out.release()
```
ffmpegcv
```
out = ffmpegcv.VideoWriter('outpy.avi', None, 10)
out.write(frame1)
out.write(frame2)
out.release()
```
frameSize is decided by the size of the first frame
```
out = ffmpegcv.VideoWriter('outpy.avi', None, 10)
```
Use GPU to accelerate encoding
```
out_cpu = ffmpegcv.VideoWriter('outpy.avi', None, 10)
out_gpu = ffmpegcv.VideoWriter('outpy.avi', 'h264_nvenc', 10)
```
Use rgb24 instead of bgr24
```
out = ffmpegcv.VideoWriter('outpy.avi', None, 10, pix_fmt='rgb24')
out.write(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))
```
Author: Chenxinfeng 2022-04-16, cxf529125853@163.com
"""
return FFmpegWriter.VideoWriter(
file, codec, fps, pix_fmt, bitrate, resize, preset=preset
)
def VideoCaptureNV(
file,
pix_fmt="bgr24",
crop_xywh=None,
resize=None,
resize_keepratio=True,
resize_keepratioalign="center",
infile_options=None,
gpu=0
) -> FFmpegReaderNV:
"""
`ffmpegcv.VideoCaptureNV` is a gpu version for `ffmpegcv.VideoCapture`.
"""
_check_nvidia()
return FFmpegReaderNV.VideoReader(
file, pix_fmt, crop_xywh, resize, resize_keepratio,
resize_keepratioalign, infile_options, gpu
)
VideoReaderNV = VideoCaptureNV
def VideoCaptureQSV(
file,
pix_fmt="bgr24",
crop_xywh=None,
resize=None,
resize_keepratio=True,
resize_keepratioalign="center",
infile_options=None,
gpu=0
) -> FFmpegReaderQSV:
"""
`ffmpegcv.VideoCaptureQSV` is a gpu version for `ffmpegcv.VideoCapture`.
"""
return FFmpegReaderQSV.VideoReader(
file, pix_fmt, crop_xywh, resize, resize_keepratio,
resize_keepratioalign, infile_options, gpu
)
VideoReaderQSV = VideoCaptureQSV
def VideoWriterNV(
file,
codec=None,
fps=30,
pix_fmt="bgr24",
gpu=0,
bitrate=None,
resize=None,
preset=None,
) -> FFmpegWriterNV:
"""
`ffmpegcv.VideoWriterNV` is a gpu version for `ffmpegcv.VideoWriter`.
"""
_check_nvidia()
return FFmpegWriterNV.VideoWriter(
file, codec, fps, pix_fmt, gpu, bitrate, resize, preset=preset
)
def VideoWriterQSV(
file,
codec=None,
fps=30,
pix_fmt="bgr24",
gpu=0,
bitrate=None,
resize=None,
preset=None,
) -> FFmpegWriterQSV:
"""
`ffmpegcv.VideoWriterQSV` is a gpu version for `ffmpegcv.VideoWriter`.
"""
return FFmpegWriterQSV.VideoWriter(
file, codec, fps, pix_fmt, gpu, bitrate, resize, preset=preset
)
def VideoWriterStreamRT(
url, pix_fmt="bgr24", bitrate=None, resize=None, preset=None
) -> FFmpegWriterStreamRT:
return FFmpegWriterStreamRT.VideoWriter(
url, "libx264", pix_fmt, bitrate, resize, preset
)
def VideoCaptureCAM(
camname,
pix_fmt="bgr24",
crop_xywh=None,
resize=None,
resize_keepratio=True,
resize_keepratioalign="center",
camsize_wh=None,
camfps=None,
camcodec=None,
campix_fmt=None
) -> FFmpegReaderCAM:
"""
Alternative to cv2.VideoCapture
Parameters
----------
file : see ffmpegcv.VideoReader
codec : see ffmpegcv.VideoReader
pix_fmt : see ffmpegcv.VideoReader
crop_xywh : see ffmpegcv.VideoReader
resize : see ffmpegcv.VideoReader
resize_keepratio : see ffmpegcv.VideoReader
resize_keepratioalign : see ffmpegcv.VideoReader
camsize_wh: tuple or None
Camera resolution (width, height). e.g (800, 600)
camfps: float or None
Camera framerate. e.g. 30.
camcodec: str or None
Camera codec. e.g. 'mjpeg' or 'h264'.
campix_fmt: str or None
Camera pixel format. e.g. 'rgb24' or 'yuv420p'.
Just set one of `camcodec` or `campix_fmt`.
Examples
--------
opencv
```
cap = cv2.VideoCapture(0)
while True:
ret, frame = cap.read()
if not ret:
break
pass
```
ffmpegcv
```
cap = ffmpegcv.VideoCaptureCAM(0)
while True:
ret, frame = cap.read()
if not ret:
break
pass
```
Or use camera name
```
cap = ffmpegcv.VideoCaptureCAM("Integrated Camera")
```
Use full camera parameter
```
cap = ffmpegcv.VideoCaptureCAM('FaceTime HD Camera',
camsize_wh = (1280,720),
camfps = 30,
campix_fmt = 'nv12')
```
Use camera with ROI operations
```
cap = ffmpegcv.VideoCaptureCAM("Integrated Camera",
crop_xywh = (0, 0, 640, 480),
resize = (512, 512),
resize_keepratio = True)
```
Author: Chenxinfeng 2023-05-11, cxf529125853@163.com
"""
return FFmpegReaderCAM.VideoReader(
camname,
pix_fmt,
crop_xywh,
resize,
resize_keepratio,
resize_keepratioalign,
camsize_wh=camsize_wh,
camfps=camfps,
camcodec=camcodec,
campix_fmt=campix_fmt
)
VideoReaderCAM = VideoCaptureCAM
def VideoCaptureStream(
stream_url,
codec=None,
pix_fmt="bgr24",
crop_xywh=None,
resize=None,
resize_keepratio=True,
resize_keepratioalign="center",
infile_options=None,
timeout=None
) -> FFmpegReaderStream:
"""
Alternative to cv2.VideoCapture
Parameters
----------
stream_url : RTSP, RTP, RTMP, HTTP, HTTPS url
codec : see ffmpegcv.VideoReader
pix_fmt : see ffmpegcv.VideoReader
crop_xywh : see ffmpegcv.VideoReader
resize : see ffmpegcv.VideoReader
resize_keepratio : see ffmpegcv.VideoReader
resize_keepratioalign : see ffmpegcv.VideoReader
infile_options : see ffmpegcv.VideoReader
timeout : waits in seconds for stream video to connect
Examples
--------
opencv
```
stream_url = 'http://devimages.apple.com.edgekey.net/streaming/examples/bipbop_4x3/gear2/prog_index.m3u8'
cap = cv2.VideoCapture(stream_url, cv2.CAP_FFMPEG)
if not cap.isOpened():
print('Cannot open the stream')
exit(-1)
while True:
ret, frame = cap.read()
if not ret:
break
pass
```
ffmpegcv
```
cap = ffmpegcv.VideoCaptureStream(stream_url)
while True:
ret, frame = cap.read()
if not ret:
break
pass
```
Author: Chenxinfeng 2023-05-31, cxf529125853@163.com
"""
return FFmpegReaderStream.VideoReader(
stream_url,
codec,
pix_fmt,
crop_xywh,
resize,
resize_keepratio,
resize_keepratioalign,
infile_options,
timeout
)
VideoReaderStream = VideoCaptureStream
def VideoCaptureStreamRT(
stream_url,
codec=None,
pix_fmt="bgr24",
crop_xywh=None,
resize=None,
resize_keepratio=True,
resize_keepratioalign="center",
infile_options=None,
gpu=None,
timeout=None,
) -> FFmpegReaderStreamRT:
if gpu is None:
return FFmpegReaderStreamRT.VideoReader(
stream_url,
codec,
pix_fmt,
crop_xywh,
resize,
resize_keepratio,
resize_keepratioalign,
infile_options,
timeout=timeout
)
else:
return FFmpegReaderStreamRTNV.VideoReader(
stream_url,
codec,
pix_fmt,
crop_xywh,
resize,
resize_keepratio,
resize_keepratioalign,
infile_options,
gpu=gpu,
timeout=timeout
)
VideoReaderStreamRT = VideoCaptureStreamRT
def VideoCapturePannels(
file: str, crop_xywh_l: list, codec=None, pix_fmt="bgr24", resize=None
):
"""
Alternative to cv2.VideoCapture
Parameters
----------
file : str
Path to video file.
crop_xywh_l : list of crop_xywh
Crop the frame. [(x0, y0, w0, h0), (x1, y1, w1, h1), ...].
codec : str
Codec to use. Optional. Default is `None`.
pix_fmt : str
Pixel format. ['bgr24' | 'rgb24' | 'gray']. Optional. Default is 'bgr24'.
resize : tuple as (w,h)
Resize the pannels to identical size. Optional. Default is `None`.
Does not keep the ratio.
Examples
--------
ffmpegcv
```
w,h = 1280,800
cap = ffmpegcv.VideoCapturePannels(file,
[[0,0,w,h], [w,0,w,h],[0,h,w,h], [w,h,w,h]])
while True:
ret, frame_pannels = cap.read()
if not ret:
break
print(frame_pannels.shape) # shape=(4,h,w,3)
```
Author: Chenxinfeng 2024-02-15, cxf529125853@163.com
"""
return FFmpegReaderPannels.VideoReader(
file,
crop_xywh_l,
codec,
pix_fmt,
resize,
)
VideoReaderPannels = VideoCapturePannels
def toCUDA(vid: FFmpegReader, gpu: int = 0, tensor_format: str = "chw") -> FFmpegReader:
"""
Convert frames to CUDA tensor float32 in 'chw' or 'hwc' format.
"""
from ffmpegcv.ffmpeg_reader_cuda import FFmpegReaderCUDA
return FFmpegReaderCUDA(vid, gpu, tensor_format)
================================================
FILE: ffmpegcv/ffmpeg_noblock.py
================================================
from .ffmpeg_reader_noblock import FFmpegReaderNoblock
from .ffmpeg_writer_noblock import FFmpegWriterNoblock
from typing import Callable
import ffmpegcv
import threading
import numpy as np
import queue
def noblock(fun:Callable, *v_args, **v_kargs):
readerfuns = (ffmpegcv.VideoCapture, ffmpegcv.VideoCaptureNV)
writerfuns = (ffmpegcv.VideoWriter, ffmpegcv.VideoWriterNV)
if fun in readerfuns:
proxyfun = FFmpegReaderNoblock(fun, *v_args, **v_kargs)
elif fun in writerfuns:
proxyfun = FFmpegWriterNoblock(fun, *v_args, **v_kargs)
else:
raise ValueError('The function is not supported as a Reader or Writer')
return proxyfun
class ReadLiveLast(threading.Thread, ffmpegcv.FFmpegReader):
def __init__(self, fun, *args, **kvargs):
threading.Thread.__init__(self)
ffmpegcv.FFmpegReader.__init__(self)
self.vid = vid = fun(*args, **kvargs)
props_name = ['width', 'height', 'fps', 'count', 'codec', 'ffmpeg_cmd',
'size', 'pix_fmt', 'out_numpy_shape', 'iframe',
'duration', 'origin_width', 'origin_height']
for name in props_name:
setattr(self, name, getattr(vid, name, None))
self.img = np.zeros(self.out_numpy_shape, dtype=np.uint8)
self.ret = True
self._isopen = True
self._q = queue.Queue(maxsize=1) # synchronize new frame
self._lock = threading.Lock()
self.start()
def read(self):
if self.ret:
self._q.get() # if reading too freq, then wait until new frame
self.iframe += 1
return self.ret, self.img
def release(self):
with self._lock:
self._isopen = False
self.vid.release()
def run(self):
while True:
with self._lock:
if self._isopen:
self.ret, self.img = self.vid.read()
else:
break
if not self._q.full():
self._q.put(None)
if not self.ret:
break
================================================
FILE: ffmpegcv/ffmpeg_reader.py
================================================
import numpy as np
import pprint
import warnings
import os
import sys
import select
from .video_info import (
run_async_reader as run_async,
get_info,
get_num_NVIDIA_GPUs,
decoder_to_nvidia,
release_process,
)
def get_videofilter_cpu(
originsize: list,
pix_fmt: str,
crop_xywh: list,
resize: list,
resize_keepratio: bool,
resize_keepratioalign: str,
):
"""
ONGONING: common filter for video/cam/stream capture.
"""
assert pix_fmt in ["rgb24", "bgr24", "yuv420p", "yuvj420p", "nv12", "gray"]
origin_width, origin_height = originsize
if crop_xywh:
crop_w, crop_h = crop_xywh[2:]
if not all([n % 2 == 0] for n in crop_xywh):
print("Warning 'crop_xywh' would be replaced into even numbers")
crop_xywh = [int(n//2*2) for n in crop_xywh]
assert crop_w <= origin_width and crop_h <= origin_height
x, y, w, h = crop_xywh
cropopt = f"crop={w}:{h}:{x}:{y}"
else:
crop_w, crop_h = origin_width, origin_height
cropopt = ""
crop_wh = (crop_w, crop_h)
if resize is None or tuple(resize) == crop_wh:
scaleopt = ""
padopt = ""
final_size_wh = crop_wh
else:
final_size_wh = (dst_width, dst_height) = resize
assert all([n % 2 == 0] for n in resize), "'resize' must be even number"
if not resize_keepratio:
scaleopt = f"scale={dst_width}x{dst_height}"
padopt = ""
else:
re_width, re_height = crop_w / (crop_h / dst_height), dst_height
if re_width > dst_width:
re_width, re_height = dst_width, crop_h / (crop_w / dst_width)
re_width, re_height = int(re_width), int(re_height)
scaleopt = f"scale={re_width}x{re_height}"
if resize_keepratioalign is None:
resize_keepratioalign = "center"
paddings = {
"center": ((dst_width - re_width) // 2, (dst_height - re_height) // 2,),
"topleft": (0, 0),
"topright": (dst_width - re_width, 0),
"bottomleft": (0, dst_height - re_height),
"bottomright": (dst_width - re_width, dst_height - re_height),
}
assert (
resize_keepratioalign in paddings
), 'resize_keepratioalign must be one of "center"(mmpose), "topleft"(mmdetection), "topright", "bottomleft", "bottomright"'
xpading, ypading = paddings[resize_keepratioalign]
padopt = f"pad={dst_width}:{dst_height}:{xpading}:{ypading}:black"
pix_fmtopt = "extractplanes=y" if pix_fmt == "gray" else ""
if any([cropopt, scaleopt, padopt, pix_fmtopt]):
filterstr = ",".join(x for x in [cropopt, scaleopt, padopt, pix_fmtopt] if x)
filteropt = f"-vf {filterstr}"
else:
filteropt = ""
return crop_wh, final_size_wh, filteropt
def get_videofilter_gpu(
originsize: list,
pix_fmt: str,
crop_xywh: list,
resize: list,
resize_keepratio: bool,
resize_keepratioalign: str,
):
assert pix_fmt in ["rgb24", "bgr24", "yuv420p", "yuvj420p", "nv12", "gray"]
origin_width, origin_height = originsize
if crop_xywh:
crop_w, crop_h = crop_xywh[2:]
assert all([n % 2 == 0] for n in crop_xywh), "'crop_xywh' must be even number"
assert crop_w <= origin_width and crop_h <= origin_height
x, y, w, h = crop_xywh
top, bottom, left, right = (
y,
origin_height - (y + h),
x,
origin_width - (x + w),
) # crop length
cropopt = f"-crop {top}x{bottom}x{left}x{right}"
else:
crop_w, crop_h = origin_width, origin_height
cropopt = ""
crop_wh = (crop_w, crop_h)
filteropt = ""
scaleopt = ""
if resize is None or tuple(resize) == crop_wh:
final_size_wh = crop_wh
else:
final_size_wh = (dst_width, dst_height) = resize
assert all([n % 2 == 0] for n in resize), "'resize' must be even number"
if not resize_keepratio:
scaleopt = f"-resize {dst_width}x{dst_height}"
else:
re_width, re_height = crop_w / (crop_h / dst_height), dst_height
if re_width > dst_width:
re_width, re_height = dst_width, crop_h / (crop_w / dst_width)
re_width, re_height = int(re_width), int(re_height)
scaleopt = f"-resize {re_width}x{re_height}"
if resize_keepratioalign is None:
resize_keepratioalign = "center"
paddings = {
"center": ((dst_width - re_width) // 2, (dst_height - re_height) // 2,),
"topleft": (0, 0),
"topright": (dst_width - re_width, 0),
"bottomleft": (0, dst_height - re_height),
"bottomright": (dst_width - re_width, dst_height - re_height),
}
assert (
resize_keepratioalign in paddings
), 'resize_keepratioalign must be one of "center"(mmpose), "topleft"(mmdetection), "topright", "bottomleft", "bottomright"'
xpading, ypading = paddings[resize_keepratioalign]
padopt = f"pad={dst_width}:{dst_height}:{xpading}:{ypading}:black"
filteropt = f"-vf {padopt}"
if pix_fmt == "gray":
if filteropt:
filteropt = f"{filteropt},extractplanes=y"
else:
filteropt = f"-vf extractplanes=y"
return crop_wh, final_size_wh, [cropopt, scaleopt, filteropt]
def get_outnumpyshape(size_wh: list, pix_fmt: str) -> tuple:
width, height = size_wh
assert (not pix_fmt == "yuv420p") or (
height % 2 == 0 and width % 2 == 0
), "yuv420p must be even"
out_numpy_shape = {
"rgb24": (height, width, 3),
"bgr24": (height, width, 3),
"yuv420p": (int(height * 1.5), width),
"yuvj420p": (int(height * 1.5), width),
"nv12": (int(height * 1.5), width),
"gray": (height, width, 1),
}[pix_fmt]
return out_numpy_shape
class FFmpegReader:
def __init__(self):
self.filename:str = ''
self.iframe:int = -1
self.width:int = None
self.height:int = None
self.size = (None, None)
self.waitInit:bool = True
self.process = None
self._isopen:bool = True
self.debug:bool = False
self.fps:float = None
self.out_numpy_shape = (None, None, None)
def __repr__(self):
props = pprint.pformat(self.__dict__).replace("{", " ").replace("}", " ")
return f"{self.__class__}\n" + props
def __enter__(self):
return self
def __exit__(self, type, value, traceback):
self.release()
def __len__(self):
return self.count
def __iter__(self):
return self
def __next__(self):
ret, img = self.read()
if ret:
return img
else:
raise StopIteration
@staticmethod
def VideoReader(
filename,
codec,
pix_fmt,
crop_xywh,
resize,
resize_keepratio,
resize_keepratioalign,
infile_options=None
):
assert os.path.exists(filename) and os.path.isfile(
filename
), f"{filename} not exists"
vid = FFmpegReader()
vid.filename = filename
videoinfo = get_info(filename)
vid.origin_width = videoinfo.width
vid.origin_height = videoinfo.height
vid.fps = videoinfo.fps
vid.count = videoinfo.count
vid.duration = videoinfo.duration
vid.pix_fmt = pix_fmt
vid.codec = videoinfo.codec
if infile_options is None:
infile_options = ''
if codec is not None:
warnings.warn(
"The 'codec' parameter is auto detected and will be removed "
"in future versions. Please refrain from using this parameter.",
DeprecationWarning
)
(
(vid.crop_width, vid.crop_height),
(vid.width, vid.height),
filteropt,
) = get_videofilter_cpu(
(vid.origin_width, vid.origin_height),
pix_fmt,
crop_xywh,
resize,
resize_keepratio,
resize_keepratioalign,
)
vid.size = (vid.width, vid.height)
vid.ffmpeg_cmd = (
f"ffmpeg -loglevel error {infile_options}"
f' -vcodec {vid.codec} -r {vid.fps} -i "{filename}" '
f" {filteropt} -pix_fmt {pix_fmt} -r {vid.fps} -f rawvideo pipe:"
)
vid.out_numpy_shape = get_outnumpyshape(vid.size, pix_fmt)
return vid
def read(self):
if self.waitInit:
self.process = run_async(self.ffmpeg_cmd)
self.waitInit = False
in_bytes = self.process.stdout.read(np.prod(self.out_numpy_shape))
# check if ffmpeg process error
# if self.process.stderr.readable():
# print('---a')
# data = self.process.stderr.read()
# sys.stderr.buffer.write(data)
# print('---f')
if not in_bytes:
self.release()
return False, None
self.iframe += 1
img = np.frombuffer(in_bytes, np.uint8).reshape(self.out_numpy_shape)
return True, img
def isOpened(self):
return self._isopen
def release(self):
self._isopen = False
release_process(self.process, forcekill=True)
def close(self):
return self.release()
class FFmpegReaderNV(FFmpegReader):
def _get_opts(
vid,
videoinfo,
crop_xywh,
resize,
resize_keepratio,
resize_keepratioalign,
isgray,
):
vid.origin_width = videoinfo.width
vid.origin_height = videoinfo.height
vid.fps = videoinfo.fps
vid.count = videoinfo.count
vid.duration = videoinfo.duration
vid.width, vid.height = vid.origin_width, vid.origin_height
vid.codec = videoinfo.codec
assert vid.origin_height % 2 == 0, "height must be even"
assert vid.origin_width % 2 == 0, "width must be even"
if crop_xywh:
crop_w, crop_h = crop_xywh[2:]
vid.width, vid.height = crop_w, crop_h
x, y, w, h = crop_xywh
top, bottom, left, right = (
y,
vid.origin_height - (y + h),
x,
vid.origin_width - (x + w),
) # crop length
cropopt = f"-crop {top}x{bottom}x{left}x{right}"
else:
crop_w, crop_h = vid.origin_width, vid.origin_height
cropopt = ""
vid.crop_width, vid.crop_height = crop_w, crop_h
if resize is None or tuple(resize) == (vid.crop_width, vid.crop_height):
scaleopt = ""
filteropt = ""
else:
vid.width, vid.height = dst_width, dst_height = resize
if not resize_keepratio:
scaleopt = f"-resize {dst_width}x{dst_height}"
filteropt = ""
else:
re_width, re_height = crop_w / (crop_h / dst_height), dst_height
if re_width > dst_width:
re_width, re_height = dst_width, crop_h / (crop_w / dst_width)
re_width, re_height = int(re_width), int(re_height)
scaleopt = f"-resize {re_width}x{re_height}"
if resize_keepratioalign is None:
resize_keepratioalign = "center"
paddings = {
"center": (
(dst_width - re_width) // 2,
(dst_height - re_height) // 2,
),
"topleft": (0, 0),
"topright": (dst_width - re_width, 0),
"bottomleft": (0, dst_height - re_height),
"bottomright": (dst_width - re_width, dst_height - re_height),
}
assert (
resize_keepratioalign in paddings
), 'resize_keepratioalign must be one of "center"(mmpose), "topleft"(mmdetection), "topright", "bottomleft", "bottomright"'
xpading, ypading = paddings[resize_keepratioalign]
padopt = f"pad={dst_width}:{dst_height}:{xpading}:{ypading}:black"
filteropt = f"-vf {padopt}"
if isgray:
if filteropt:
filteropt = f"{filteropt},extractplanes=y"
else:
filteropt = f"-vf extractplanes=y"
vid.size = (vid.width, vid.height)
return cropopt, scaleopt, filteropt
@staticmethod
def VideoReader(
filename,
pix_fmt,
crop_xywh,
resize,
resize_keepratio,
resize_keepratioalign,
infile_options,
gpu,
):
assert os.path.exists(filename) and os.path.isfile(
filename
), f"{filename} not exists"
assert pix_fmt in ["rgb24", "bgr24", "yuv420p", "yuvj420p", "nv12", "gray"]
numGPU = get_num_NVIDIA_GPUs()
assert numGPU > 0, "No GPU found"
gpu = int(gpu) % numGPU if gpu is not None else 0
assert (
resize is None or len(resize) == 2
), "resize must be a tuple of (width, height)"
if infile_options is None:
infile_options = ''
videoinfo = get_info(filename)
vid = FFmpegReaderNV()
vid.filename = filename
isgray = pix_fmt == "gray"
cropopt, scaleopt, filteropt = vid._get_opts(
videoinfo,
crop_xywh,
resize,
resize_keepratio,
resize_keepratioalign,
isgray,
)
vid.codecNV = decoder_to_nvidia(vid.codec)
vid.ffmpeg_cmd = (
f"ffmpeg -loglevel error -hwaccel cuda -hwaccel_device {gpu} {infile_options} "
f' -vcodec {vid.codecNV} {cropopt} {scaleopt} -r {vid.fps} -i "{filename}" '
f" {filteropt} -pix_fmt {pix_fmt} -r {vid.fps} -f rawvideo pipe:"
)
vid.pix_fmt = pix_fmt
vid.out_numpy_shape = get_outnumpyshape(vid.size, pix_fmt)
return vid
================================================
FILE: ffmpegcv/ffmpeg_reader_camera.py
================================================
import numpy as np
import pprint
from .video_info import run_async, release_process
import re
import subprocess
from threading import Thread
from queue import Queue
import sys
import os
from ffmpegcv.ffmpeg_reader import get_videofilter_cpu, get_outnumpyshape
class platform:
win = 0
linux = 1
mac = 2
other = 3
if sys.platform.startswith("linux"):
this_os = platform.linux
elif sys.platform.startswith("win32"):
this_os = platform.win
elif sys.platform.startswith("darwin"):
this_os = platform.mac
else:
this_os = platform.other
def _query_camera_divices_mac() -> dict:
# run the command 'ffmpeg -f avfoundation -list_devices true -i "" '
command = 'ffmpeg -hide_banner -f avfoundation -list_devices true -i ""'
process = subprocess.Popen(
command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE
)
stdout, stderr = process.communicate()
# parse the output into a dictionary
lines = stderr.decode("utf-8").split("AVFoundation audio devices:")[0].split("\n")
id_device_map = dict()
device_id_pattern = re.compile(r"\[[^\]]*?\] \[(\d*)\]")
device_name_pattern = re.compile(r".*\] (.*)")
for line in lines[1:-1]:
device_id = int(re.search(device_id_pattern, line).group(1))
device_name = re.search(device_name_pattern, line).group(1)
id_device_map[device_id] = (device_name, device_id)
return id_device_map
def _query_camera_divices_win() -> dict:
command = "ffmpeg -hide_banner -list_devices true -f dshow -i dummy"
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = process.communicate()
dshowliststr = stderr.decode("utf-8")
dshowliststr = dshowliststr.split("DirectShow audio devices")[0]
pattern = re.compile(r'\[*?\] *"([^"]*)"')
matches = pattern.findall(dshowliststr)
alternative_pattern = re.compile(r'Alternative name "(.*)"')
alternative_names = alternative_pattern.findall(dshowliststr)
assert len(matches) == len(alternative_names)
id_device_map = {
i: device for i, device in enumerate(zip(matches, alternative_names))
}
if len(id_device_map) == 0:
print("No camera divice found")
return id_device_map
def _query_camera_divices_linux() -> dict:
"edit from https://github.com/p513817/python-get-cam-name/blob/master/get_cam_name.py"
root = "/sys/class/video4linux"
cam_info = []
for index in sorted([file for file in os.listdir(root)]):
# Get Camera Name From /sys/class/video4linux/<video*>/name
real_index_file = os.path.realpath("/sys/class/video4linux/" + index + "/index")
with open(real_index_file, "r") as name_file:
_index = name_file.read().rstrip()
if _index != "0":
continue
real_file = os.path.realpath("/sys/class/video4linux/" + index + "/name")
with open(real_file, "r") as name_file:
name = name_file.read().rstrip()
name = name.split(":")[0]
# Setup Each Camera and Index ( video* )
cam_info.append((name, "/dev/" + index))
id_device_map = {i: vname for i, vname in enumerate(cam_info)}
return id_device_map
def query_camera_devices(verbose_dict: bool = False) -> dict:
result = {
platform.linux: _query_camera_divices_linux,
platform.mac: _query_camera_divices_mac,
platform.win: _query_camera_divices_win,
}[this_os]()
if verbose_dict:
dict_by_v0 = {v[0]: v for v in result.values()}
dict_by_v1 = {v[1]: v for v in result.values()}
result.update(dict_by_v0)
result.update(dict_by_v1)
return result
def _query_camera_options_mac(cam_id_name) -> str:
print(
"\033[33m"
+ "FFmpeg& FFmpegcv CAN NOT query the camera options in MAC platform."
+ "\033[0m"
)
print("Please find the proper parameter other way.")
return [{"camsize_wh": None, "camfps": None}]
def _query_camera_options_linux(cam_id_name) -> str:
print(
"\033[33m"
+ "FFmpeg& FFmpegcv CAN NOT query the camera FPS in Linux platform."
+ "\033[0m"
)
print("Please find the proper parameter other way.")
camname = query_camera_devices(verbose_dict=True)[cam_id_name][1]
command = f'ffmpeg -hide_banner -f v4l2 -list_formats all -i "{camname}"'
process = subprocess.Popen(
command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE
)
stdout, stderr = process.communicate()
lines = stderr.decode("utf-8").split("\n")
lines = [l for l in lines if "v4l2" in l]
outlist = []
for line in lines:
_, vcodec, *_, resolutions = line.split(":")
vcodec = vcodec.strip()
israw = "Raw" in line
camcodec = None if israw else vcodec
campix_fmt = vcodec if israw else None
resolutions = resolutions.strip()
camsize_wh_l = [tuple(map(int, r.split("x"))) for r in resolutions.split()]
outlist.extend(
[
{
"camcodec": camcodec,
"campix_fmt": campix_fmt,
"camsize_wh": wh,
"camfps": None,
}
for wh in camsize_wh_l
]
)
return outlist
def _query_camera_options_win(cam_id_name) -> str:
if isinstance(cam_id_name, int):
id_device_map = query_camera_devices()
camname = id_device_map[cam_id_name][1]
elif isinstance(cam_id_name, str):
camname = cam_id_name
else:
raise ValueError("Not valid camname")
command = f'ffmpeg -hide_banner -f dshow -list_options true -i video="{camname}"'
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = process.communicate()
dshowliststr = stderr.decode("utf-8").replace("\r\n", "\n").replace("\r", "\n")
dshowlist = [s for s in dshowliststr.split("\n") if "fps=" in s]
from collections import OrderedDict
unique_dshowlist = list(OrderedDict.fromkeys(dshowlist))
outlist = []
for text in unique_dshowlist:
cam_options = dict()
cam_options["camcodec"] = (
re.search(r"vcodec=(\w+)", text).group(1) if "vcodec" in text else None
)
cam_options["campix_fmt"] = (
re.search(r"pixel_format=(\w+)", text).group(1)
if "pixel_format" in text
else None
)
camsize_wh = re.search(r"min s=(\w+)", text).group(1)
cam_options["camsize_wh"] = tuple(int(v) for v in camsize_wh.split("x"))
camfps = float(re.findall(r"fps=([\d.]+)", text)[-1])
cam_options["camfps"] = int(camfps) if int(camfps) == camfps else camfps
outlist.append(cam_options)
return outlist
def query_camera_options(cam_id_name) -> str:
return {
platform.linux: _query_camera_options_linux,
platform.mac: _query_camera_options_mac,
platform.win: _query_camera_options_win,
}[this_os](cam_id_name)
class ProducerThread(Thread):
def __init__(self, vid, q):
super(ProducerThread, self).__init__()
self.vid = vid
self.q = q
def run(self):
q = self.q
while True:
if not self.vid.isOpened():
break
ret, img = self.vid.read_()
if q.full():
q.get() # drop frames
q.put((ret, img))
class FFmpegReaderCAM:
def __init__(self):
self.iframe = -1
self._isopen = True
def __repr__(self):
props = pprint.pformat(self.__dict__).replace("{", " ").replace("}", " ")
return f"{self.__class__}\n" + props
def __enter__(self):
return self
def __exit__(self, type, value, traceback):
self.release()
def __iter__(self):
return self
def __next__(self):
ret, img = self.read()
if ret:
return img
else:
raise StopIteration
@staticmethod
def VideoReader(
cam_id_name,
pix_fmt,
crop_xywh,
resize,
resize_keepratio,
resize_keepratioalign,
camsize_wh=None,
camfps=None,
camcodec=None,
campix_fmt=None,
step=1,
):
vid = FFmpegReaderCAM()
if this_os == platform.mac:
# use cam_id as the device marker
if isinstance(cam_id_name, str):
id_device_map = query_camera_devices()
camname = cam_id_name
id_device_map.update({v[0]: v for v in id_device_map.values()})
camid = id_device_map[cam_id_name][1]
else:
camname = None
camid = cam_id_name
elif this_os == platform.linux:
id_device_map = query_camera_devices(verbose_dict=True)
camname = id_device_map[cam_id_name][-1]
camid = None
else:
if isinstance(cam_id_name, int):
id_device_map = query_camera_devices()
camname = id_device_map[cam_id_name][1]
camid = cam_id_name
else:
camname = cam_id_name
camid = None
vid.camname = camname
vid.camid = camid
if camsize_wh is None:
cam_options = query_camera_options(camname)
resolutions = [c["camsize_wh"] for c in cam_options]
camsize_wh = max(resolutions, key=lambda x: sum(x))
assert len(camsize_wh) == 2
vid.origin_width, vid.origin_height = camsize_wh
opt_camfps = f" -framerate {camfps} " if camfps else ""
vid.camfps = camfps if camfps else None
opt_camcodec_ = {
platform.linux: "input_format",
platform.mac: "",
platform.win: "vcodec",
}[this_os]
opt_camcodec = f" -{opt_camcodec_} {camcodec} " if camcodec else ""
vid.camcodec = camcodec if camcodec else None
vid.pix_fmt = pix_fmt
opt_campix_fmt_ = {
platform.linux: "input_format",
platform.mac: "pixel_format",
platform.win: "pixel_format",
}[this_os]
opt_campix_fmt = f" -{opt_campix_fmt_} {campix_fmt} " if campix_fmt else ""
vid.campix_fmt = campix_fmt if campix_fmt else None
opt_camname = {
platform.linux: f'"{camname}"',
platform.win: f'video="{camname}"',
platform.mac: f"{camid}:none",
}[this_os]
(vid.crop_width, vid.crop_height), (vid.width, vid.height), filteropt = get_videofilter_cpu(
(vid.origin_width, vid.origin_height), pix_fmt, crop_xywh, resize, resize_keepratio, resize_keepratioalign)
vid.size = (vid.width, vid.height)
opt_driver_ = {
platform.linux: "v4l2",
platform.mac: "avfoundation",
platform.win: "dshow",
}[this_os]
vid.ffmpeg_cmd = (
f"ffmpeg -loglevel warning "
f" -f {opt_driver_} "
f" -video_size {vid.origin_width}x{vid.origin_height} "
f" {opt_camfps} {opt_camcodec} {opt_campix_fmt} "
f" -i {opt_camname} "
f" {filteropt} -pix_fmt {pix_fmt} -f rawvideo pipe:"
)
vid.out_numpy_shape = get_outnumpyshape(vid.size, pix_fmt)
vid.process = run_async(vid.ffmpeg_cmd)
# producer
assert step >= 1 and isinstance(step, int)
vid.step = step
vid.q = Queue(maxsize=30)
producer = ProducerThread(vid, vid.q)
producer.start()
return vid
def read_(self):
for i in range(self.step):
in_bytes = self.process.stdout.read(np.prod(self.out_numpy_shape))
if not in_bytes:
self.release()
return False, None
self.iframe += 1
img = None
img = np.frombuffer(in_bytes, np.uint8).reshape(self.out_numpy_shape)
return True, img
def read(self):
ret, img = self.q.get()
return ret, img
def isOpened(self):
return self._isopen
def release(self):
self._isopen = False
release_process(self.process)
def close(self):
return self.release()
================================================
FILE: ffmpegcv/ffmpeg_reader_cuda.py
================================================
import pycuda.driver as cuda
from pycuda.driver import PointerHolderBase
from pycuda.compiler import SourceModule
from pycuda import gpuarray
from ffmpegcv.ffmpeg_reader import FFmpegReader, FFmpegReaderNV
import numpy as np
from typing import Tuple
cuda.init()
mod_code = ("""
__device__ void yuv_to_rgb(unsigned char &y, unsigned char &u, unsigned char &v,
float &r, float &g, float &b)
{
// https://fourcc.org/fccyvrgb.php
float Y_val = (float)1.164 * ((float)y - 16.0);
float U_val = (float)u - 128.0;
float V_val = (float)v - 128.0;
r = Y_val + 1.596 * V_val;
r = max(0.0, min(255.0, r)); // clamp(r, 0.0, 255.0);
g = Y_val - 0.813 * V_val - 0.391 * U_val;
g = max(0.0, min(255.0, g));
b = Y_val + 2.018 * U_val;
b = max(0.0, min(255.0, b));
}
__global__ void yuv420p_CHW_fp32(unsigned char *YUV420p, float *RGB24, int *width_, int *height_)
{
int width = *width_; int height = *height_;
// Get the thread index
int x = blockIdx.x * blockDim.x + threadIdx.x;
int y = blockIdx.y * blockDim.y + threadIdx.y;
// Check if we are within the bounds of the image
if (x >= width || y >= height)
return;
// Get the Y, U, and V values for this pixel
auto w_h = width * height;
auto yW = y * width;
auto out_ind = yW + x;
unsigned char *Y = YUV420p;
unsigned char *U = YUV420p + w_h;
unsigned char *V = U + w_h/4;
int delta = (y/2)*(width/2)+x/2;
yuv_to_rgb(Y[out_ind], U[delta], V[delta],
RGB24[out_ind], RGB24[out_ind + w_h], RGB24[out_ind + w_h*2]);
}
__global__ void yuv420p_HWC_fp32(unsigned char *YUV420p, float *RGB24, int *width_, int *height_)
{
int width = *width_; int height = *height_;
// Get the thread index
int x = blockIdx.x * blockDim.x + threadIdx.x;
int y = blockIdx.y * blockDim.y + threadIdx.y;
// Check if we are within the bounds of the image
if (x >= width || y >= height)
return;
// Get the Y, U, and V values for this pixel
auto w_h = width * height;
auto yW = y * width;
auto out_ind = yW + x;
unsigned char *Y = YUV420p;
unsigned char *U = YUV420p + w_h;
unsigned char *V = U + w_h/4;
int delta = (y/2)*(width/2)+x/2;
auto ind = (yW + x)*3;
yuv_to_rgb(Y[out_ind], U[delta], V[delta],
RGB24[ind], RGB24[ind+1], RGB24[ind+2]);
}
__global__ void NV12_CHW_fp32(unsigned char *NV12, float *RGB24, int *width_, int *height_)
{
int width = *width_; int height = *height_;
// Get the thread index
int x = blockIdx.x * blockDim.x + threadIdx.x;
int y = blockIdx.y * blockDim.y + threadIdx.y;
// Check if we are within the bounds of the image
if (x >= width || y >= height)
return;
// Get the Y, U, and V values for this pixel
auto w_h = width * height;
auto yW = y * width;
auto out_ind = yW + x;
unsigned char *Y = NV12 + out_ind;
unsigned char *UV = NV12 + w_h + (y / 2) * width + (x / 2) * 2;
yuv_to_rgb(Y[0], UV[0], UV[1],
RGB24[out_ind], RGB24[out_ind + w_h], RGB24[out_ind + w_h*2]);
}
__global__ void NV12_HWC_fp32(unsigned char *NV12, float *RGB24, int *width_, int *height_)
{
int width = *width_; int height = *height_;
// Get the thread index
int x = blockIdx.x * blockDim.x + threadIdx.x;
int y = blockIdx.y * blockDim.y + threadIdx.y;
// Check if we are within the bounds of the image
if (x >= width || y >= height)
return;
// Get the Y, U, and V values for this pixel
auto w_h = width * height;
auto yW = y * width;
auto out_ind = yW + x;
unsigned char *Y = NV12 + out_ind;
unsigned char *UV = NV12 + w_h + (y / 2) * width + (x / 2) * 2;
auto ind = (yW + x)*3;
yuv_to_rgb(Y[0], UV[0], UV[1],
RGB24[ind], RGB24[ind+1], RGB24[ind+2]);
}
"""
)
def load_cuda_module():
mod = SourceModule(mod_code)
converter = {('yuv420p', 'chw'): mod.get_function('yuv420p_CHW_fp32'),
('yuv420p', 'hwc'): mod.get_function('yuv420p_HWC_fp32'),
('nv12', 'chw'): mod.get_function('NV12_CHW_fp32'),
('nv12', 'hwc'): mod.get_function('NV12_HWC_fp32')}
return converter
class Holder(PointerHolderBase):
def __init__(self, tensor):
super().__init__()
self.tensor = tensor
self.gpudata = tensor.data_ptr()
def get_pointer(self):
return self.tensor.data_ptr()
def __index__(self):
return self.gpudata
def tensor_to_gpuarray(tensor) -> gpuarray.GPUArray:
'''Convert a :class:`torch.Tensor` to a :class:`pycuda.gpuarray.GPUArray`. The underlying
storage will be shared, so that modifications to the array will reflect in the tensor object.
Parameters
----------
tensor : torch.Tensor
Returns
-------
pycuda.gpuarray.GPUArray
Raises
------
ValueError
If the ``tensor`` does not live on the gpu
'''
return gpuarray.GPUArray(tensor.shape, dtype=np.float32, gpudata=Holder(tensor))
class PycudaContext:
def __init__(self, gpu=0):
self.ctx = cuda.Device(gpu).make_context()
def __enter__(self):
if self.ctx is not None:
self.ctx.push()
return self
def __exit__(self, *args, **kwargs):
if self.ctx is not None:
self.ctx.pop()
def __del__(self):
if self.ctx is not None:
self.ctx.pop()
class FFmpegReaderCUDA(FFmpegReader):
def __init__(self, vid:FFmpegReader, gpu=0, tensor_format='hwc'):
assert vid.pix_fmt in ['yuv420p', 'nv12'], 'Set pix_fmt to yuv420p or nv12. Auto convert to rgb in cuda.'
assert tensor_format in ['hwc', 'chw'], 'tensor_format must be hwc or chw'
if isinstance(vid, FFmpegReaderNV) and vid.pix_fmt != 'nv12':
print('--Tips: please use VideoCaptureNV(..., pix_fmt="NV12") for better performance.')
elif not isinstance(vid, FFmpegReaderNV) and vid.pix_fmt == 'nv12':
print('--Tips: please use VideoCapture(..., pix_fmt="yuv420p") for better performance.')
# work like normal FFmpegReaderObj
props_name = ['width', 'height', 'fps', 'count', 'codec', 'ffmpeg_cmd',
'size', 'out_numpy_shape', 'iframe',
'duration', 'origin_width', 'origin_height']
for name in props_name:
setattr(self, name, getattr(vid, name, None))
self.ctx = PycudaContext(gpu)
self.pix_fmt = 'rgb24'
self.vid = vid
self.out_numpy_shape = (vid.height, vid.width, 3) if tensor_format == 'hwc' else (3, vid.height, vid.width)
self.torch_device = f'cuda:{gpu}'
self.block_size = (16, 16, 1)
self.grid_size = ((self.width + self.block_size[0] - 1) // self.block_size[0],
(self.height + self.block_size[1] - 1) // self.block_size[1])
self.process = None
with self.ctx:
self.converter = load_cuda_module()[(vid.pix_fmt, tensor_format)]
def read(self, out_MAT:gpuarray.GPUArray=None) -> Tuple[bool, gpuarray.GPUArray]:
self.waitInit = False
ret, frame_yuv420p = self.vid.read()
if not ret:
return False, None
with self.ctx:
if out_MAT is None:
out_MAT = gpuarray.empty(self.out_numpy_shape, dtype=np.float32)
self.converter(cuda.In(frame_yuv420p), out_MAT,
cuda.In(np.int32(self.width)), cuda.In(np.int32(self.height)),
block=self.block_size, grid=self.grid_size)
return True, out_MAT
def read_cudamem(self, out_MAT:cuda.DeviceAllocation=None) -> Tuple[bool, cuda.DeviceAllocation]:
self.waitInit = False
ret, frame_yuv420p = self.vid.read()
if not ret:
return False, None
with self.ctx:
if out_MAT is None:
out_MAT = cuda.mem_alloc(int(np.prod(self.out_numpy_shape) *
np.dtype(np.float32).itemsize))
self.converter(cuda.In(frame_yuv420p), out_MAT,
cuda.In(np.int32(self.width)), cuda.In(np.int32(self.height)),
block=self.block_size, grid=self.grid_size)
return True, out_MAT
def read_torch(self, out_MAT=None):
import torch
self.waitInit = False
ret, frame_yuv420p = self.vid.read()
if not ret:
return False, None
with self.ctx:
if out_MAT is None:
out_MAT = torch.empty(self.out_numpy_shape, dtype=torch.float32, device=self.torch_device)
tensor_proxy = tensor_to_gpuarray(out_MAT)
self.converter(cuda.In(frame_yuv420p), tensor_proxy.gpudata,
cuda.In(np.int32(self.width)), cuda.In(np.int32(self.height)),
block=self.block_size, grid=self.grid_size)
return True, out_MAT
def release(self):
self.vid.release()
super().release()
================================================
FILE: ffmpegcv/ffmpeg_reader_noblock.py
================================================
import multiprocessing
from multiprocessing import Queue
import numpy as np
from .ffmpeg_reader import FFmpegReader
NFRAME = 10
class FFmpegReaderNoblock(FFmpegReader):
def __init__(self,
vcap_fun,
*vcap_args, **vcap_kwargs):
vid:FFmpegReader = vcap_fun(*vcap_args, **vcap_kwargs)
vid.release()
# work like normal FFmpegReaderObj
props_name = ['width', 'height', 'fps', 'count', 'codec', 'ffmpeg_cmd',
'size', 'pix_fmt', 'out_numpy_shape', 'iframe',
'duration', 'origin_width', 'origin_height']
for name in props_name:
setattr(self, name, getattr(vid, name, None))
# 创建共享内存的NumPy数组
shared_array = multiprocessing.Array('b', int(NFRAME*np.prod(self.out_numpy_shape)))
# 将共享内存的NumPy数组转换为NumPy数组
self.np_array = np.frombuffer(shared_array.get_obj(), dtype=np.uint8).reshape((NFRAME,*self.out_numpy_shape))
self.shared_array = shared_array
self.vcap_args = vcap_args
self.vcap_kwargs = vcap_kwargs
self.q = Queue(maxsize=(NFRAME-2)) #buffer index, gluttonous snake NO biting its own tail
self.vcap_fun = vcap_fun
self.has_init = False
self.process = None
def read(self):
if not self.has_init:
self.has_init = True
process = multiprocessing.Process(target=child_process,
args=(self.shared_array, self.q, self.vcap_fun,
self.vcap_args, self.vcap_kwargs))
process.start()
self.process = process
data_id = self.q.get() # 读取子进程写入的数据
if data_id is None:
return False, None
else:
self.iframe += 1
return True, self.np_array[data_id]
def child_process(shared_array, q:Queue, vcap_fun, vcap_args, vcap_kwargs):
vid = vcap_fun(*vcap_args, **vcap_kwargs)
np_array = np.frombuffer(shared_array.get_obj(), dtype=np.uint8).reshape((NFRAME,*vid.out_numpy_shape))
with vid:
for i, img in enumerate(vid):
iloop = i % NFRAME
# 在子进程中修改共享内存的NumPy数组
np_array[iloop] = img
q.put(iloop) # 通知主进程已经写入了
q.put(None)
================================================
FILE: ffmpegcv/ffmpeg_reader_pannels.py
================================================
import os
import numpy as np
from ffmpegcv.ffmpeg_reader import FFmpegReader, get_outnumpyshape
from ffmpegcv.video_info import (
get_info,
run_async
)
class FFmpegReaderPannels(FFmpegReader):
@staticmethod
def VideoReader(
filename:str,
crop_xywh_l:list,
codec,
pix_fmt='bgr24',
resize=None
):
assert os.path.exists(filename) and os.path.isfile(
filename
), f"{filename} not exists"
assert pix_fmt in ["rgb24", "bgr24", "yuv420p", "nv12", "gray"]
vid = FFmpegReaderPannels()
crop_xywh_l = np.array(crop_xywh_l)
vid.crop_xywh_l = crop_xywh_l
videoinfo = get_info(filename)
vid.origin_width = videoinfo.width
vid.origin_height = videoinfo.height
vid.fps = videoinfo.fps
vid.count = videoinfo.count
vid.duration = videoinfo.duration
vid.pix_fmt = pix_fmt
vid.codec = codec if codec else videoinfo.codec
vid.crop_width_l = crop_xywh_l[:,2]
vid.crop_height_l = crop_xywh_l[:,3]
vid.size_l = crop_xywh_l[:,2:][:,::-1]
vid.npannel = len(crop_xywh_l)
vid.out_numpy_shape_l = [get_outnumpyshape(s[::-1], pix_fmt) for s in vid.size_l]
if len(set(vid.crop_width_l)) == len(set(vid.crop_height_l)) == 1:
vid.is_pannel_similar = True
vid.crop_width = vid.crop_width_l[0]
vid.crop_height = vid.crop_height_l[0]
vid.size = vid.size_l[0]
vid.out_numpy_shape = (vid.npannel, *vid.out_numpy_shape_l[0])
else:
vid.is_pannel_similar = False
vid.crop_width = vid.crop_height = vid.size = None
vid.out_numpy_shape = (np.sum(np.prod(s) for s in vid.out_numpy_shape_l),)
VINSRCs =''.join(f'[VSRC{i}]' for i in range(vid.npannel))
pix_fmtopt = ',extractplanes=y' if pix_fmt=='gray' else ''
CROPs = ';'.join(f'[VSRC{i}]crop={w}:{h}:{x}:{y}{pix_fmtopt}[VPANEL{i}]'
for i, (x,y,w,h) in enumerate(vid.crop_xywh_l))
filteropt = f' -filter_complex "split={vid.npannel}{VINSRCs};{CROPs}"'
outmaps = ''.join(f' -map [VPANEL{i}] -pix_fmt {pix_fmt} -r {vid.fps} -f rawvideo pipe:'
for i in range(vid.npannel))
vid.ffmpeg_cmd = (
f"ffmpeg -loglevel warning "
f' -r {vid.fps} -i "{filename}" '
f" {filteropt} {outmaps}"
)
return vid
def read(self):
if self.waitInit:
self.process = run_async(self.ffmpeg_cmd)
self.waitInit = False
in_bytes = self.process.stdout.read(np.prod(self.out_numpy_shape))
if not in_bytes:
self.release()
return False, None
self.iframe += 1
img0 = np.frombuffer(in_bytes, np.uint8)
if self.is_pannel_similar:
img = img0.reshape(self.out_numpy_shape)
else:
img = []
for out_numpy_shape in self.out_numpy_shape_l:
nbuff = np.prod(out_numpy_shape)
img.append(img0[:nbuff].reshape(out_numpy_shape))
img0 = img0[nbuff:]
return True, img
================================================
FILE: ffmpegcv/ffmpeg_reader_qsv.py
================================================
import os
from ffmpegcv.ffmpeg_reader import FFmpegReader, get_videofilter_cpu, get_outnumpyshape
from .video_info import (
get_info,
get_num_QSV_GPUs,
decoder_to_qsv,
)
class FFmpegReaderQSV(FFmpegReader):
@staticmethod
def VideoReader(
filename,
pix_fmt,
crop_xywh,
resize,
resize_keepratio,
resize_keepratioalign,
infile_options,
gpu,
):
"""
TODO: 1. only 1 gpu is recognized
TODO: 2. 'crop_xywh', 'resize*' are not supported yet
"""
assert os.path.exists(filename) and os.path.isfile(
filename
), f"{filename} not exists"
assert gpu is None or gpu == 0, "Cannot use multiple QSV gpu yet."
numGPU = get_num_QSV_GPUs()
assert numGPU > 0, "No GPU found"
gpu = int(gpu) % numGPU if gpu is not None else 0
assert (
resize is None or len(resize) == 2
), "resize must be a tuple of (width, height)"
vid = FFmpegReaderQSV()
videoinfo = get_info(filename)
vid.origin_width = videoinfo.width
vid.origin_height = videoinfo.height
vid.fps = videoinfo.fps
vid.count = videoinfo.count
vid.duration = videoinfo.duration
vid.codecQSV = decoder_to_qsv(videoinfo.codec)
vid.pix_fmt = pix_fmt
if infile_options is None:
infile_options = ''
(
(vid.crop_width, vid.crop_height),
(vid.width, vid.height),
filteropt,
) = get_videofilter_cpu(
(vid.origin_width, vid.origin_height),
pix_fmt,
crop_xywh,
resize,
resize_keepratio,
resize_keepratioalign,
)
vid.size = (vid.width, vid.height)
vid.ffmpeg_cmd = (
f"ffmpeg -loglevel warning {infile_options}"
f' -vcodec {vid.codecQSV} -r {vid.fps} -i "{filename}" '
f" {filteropt} -pix_fmt {pix_fmt} -r {vid.fps} -f rawvideo pipe:"
)
vid.out_numpy_shape = get_outnumpyshape(vid.size, pix_fmt)
return vid
================================================
FILE: ffmpegcv/ffmpeg_reader_stream.py
================================================
from .video_info import run_async
from queue import Queue
from ffmpegcv.stream_info import get_info
from ffmpegcv.ffmpeg_reader_camera import FFmpegReaderCAM, ProducerThread
from ffmpegcv.ffmpeg_reader import (
get_videofilter_cpu,
get_outnumpyshape,
get_videofilter_gpu,
get_num_NVIDIA_GPUs,
decoder_to_nvidia,
)
class FFmpegReaderStream(FFmpegReaderCAM):
def __init__(self):
super().__init__()
@staticmethod
def VideoReader(
stream_url,
codec,
pix_fmt,
crop_xywh,
resize,
resize_keepratio,
resize_keepratioalign,
infile_options,
timeout,
):
vid = FFmpegReaderStream()
videoinfo = get_info(stream_url, timeout)
vid.origin_width = videoinfo.width
vid.origin_height = videoinfo.height
vid.fps = videoinfo.fps
vid.codec = codec if codec else videoinfo.codec
vid.count = videoinfo.count
vid.duration = videoinfo.duration
vid.pix_fmt = pix_fmt
if infile_options is None:
infile_options = ''
(
(vid.crop_width, vid.crop_height),
(vid.width, vid.height),
filteropt,
) = get_videofilter_cpu(
(vid.origin_width, vid.origin_height),
pix_fmt,
crop_xywh,
resize,
resize_keepratio,
resize_keepratioalign,
)
vid.size = (vid.width, vid.height)
rtsp_opt = '' if not stream_url.startswith('rtsp://') else '-rtsp_flags prefer_tcp -pkt_size 736 '
vid.ffmpeg_cmd = (
f"ffmpeg -loglevel warning {infile_options}"
f" {rtsp_opt} "
f" -vcodec {vid.codec} -i {stream_url} "
f" {filteropt} -pix_fmt {pix_fmt} -f rawvideo pipe:"
)
vid.out_numpy_shape = get_outnumpyshape(vid.size, pix_fmt)
vid.process = run_async(vid.ffmpeg_cmd)
vid.isopened = True
# producer
vid.step = 1
vid.q = Queue(maxsize=30)
producer = ProducerThread(vid, vid.q)
producer.start()
return vid
class FFmpegReaderStreamNV(FFmpegReaderCAM):
def __init__(self):
super().__init__()
@staticmethod
def VideoReader(
stream_url,
codec,
pix_fmt,
crop_xywh,
resize,
resize_keepratio,
resize_keepratioalign,
infile_options,
gpu,
timeout,
):
numGPU = get_num_NVIDIA_GPUs()
vid = FFmpegReaderStreamNV()
videoinfo = get_info(stream_url, timeout)
vid.origin_width = videoinfo.width
vid.origin_height = videoinfo.height
vid.fps = videoinfo.fps
vid.codec = codec if codec else videoinfo.codec
vid.codecNV = decoder_to_nvidia(vid.codec)
vid.count = videoinfo.count
vid.duration = videoinfo.duration
vid.pix_fmt = pix_fmt
if infile_options is None:
infile_options = ''
(
(vid.crop_width, vid.crop_height),
(vid.width, vid.height),
(cropopt, scaleopt, filteropt),
) = get_videofilter_gpu(
(vid.origin_width, vid.origin_height),
pix_fmt,
crop_xywh,
resize,
resize_keepratio,
resize_keepratioalign,
)
vid.size = (vid.width, vid.height)
rtsp_opt = '' if not stream_url.startswith('rtsp://') else '-rtsp_flags prefer_tcp -pkt_size 736 '
vid.ffmpeg_cmd = (
f"ffmpeg -loglevel warning -hwaccel cuda -hwaccel_device {gpu}"
f" {infile_options} {rtsp_opt} "
f' -vcodec {vid.codecNV} {cropopt} {scaleopt} -i "{stream_url}" '
f" {filteropt} -pix_fmt {pix_fmt} -f rawvideo pipe:"
)
vid.out_numpy_shape = get_outnumpyshape(vid.size, pix_fmt)
vid.process = run_async(vid.ffmpeg_cmd)
vid.isopened = True
# producer
vid.step = 1
vid.q = Queue(maxsize=30)
producer = ProducerThread(vid, vid.q)
producer.start()
return vid
================================================
FILE: ffmpegcv/ffmpeg_reader_stream_realtime.py
================================================
from ffmpegcv.ffmpeg_reader import (
FFmpegReader,
get_videofilter_cpu,
get_outnumpyshape,
get_videofilter_gpu,
get_num_NVIDIA_GPUs,
decoder_to_nvidia,
)
from ffmpegcv.stream_info import get_info
class FFmpegReaderStreamRT(FFmpegReader):
def __init__(self):
super().__init__()
@staticmethod
def VideoReader(
stream_url,
codec,
pix_fmt,
crop_xywh,
resize,
resize_keepratio,
resize_keepratioalign,
infile_options,
timeout,
):
vid = FFmpegReaderStreamRT()
videoinfo = get_info(stream_url, timeout)
vid.origin_width = videoinfo.width
vid.origin_height = videoinfo.height
vid.fps = videoinfo.fps
vid.codec = codec if codec else videoinfo.codec
vid.count = videoinfo.count
vid.duration = videoinfo.duration
vid.pix_fmt = pix_fmt
if infile_options is None:
infile_options = ''
(
(vid.crop_width, vid.crop_height),
(vid.width, vid.height),
filteropt,
) = get_videofilter_cpu(
(vid.origin_width, vid.origin_height),
pix_fmt,
crop_xywh,
resize,
resize_keepratio,
resize_keepratioalign,
)
vid.size = (vid.width, vid.height)
rtsp_opt = '' if not stream_url.startswith('rtsp://') else '-rtsp_flags prefer_tcp -pkt_size 736 '
vid.ffmpeg_cmd = (
f"ffmpeg -loglevel error "
f" {infile_options} {rtsp_opt} "
"-fflags nobuffer -flags low_delay -strict experimental "
f" -vcodec {vid.codec} -i {stream_url}"
f" {filteropt} -pix_fmt {pix_fmt} -f rawvideo pipe:"
)
vid.out_numpy_shape = get_outnumpyshape(vid.size, pix_fmt)
return vid
class FFmpegReaderStreamRTNV(FFmpegReader):
def __init__(self):
super().__init__()
@staticmethod
def VideoReader(
stream_url,
codec,
pix_fmt,
crop_xywh,
resize,
resize_keepratio,
resize_keepratioalign,
infile_options,
gpu,
timeout,
):
vid = FFmpegReaderStreamRTNV()
videoinfo = get_info(stream_url, timeout)
vid.origin_width = videoinfo.width
vid.origin_height = videoinfo.height
vid.fps = videoinfo.fps
vid.codec = codec if codec else videoinfo.codec
vid.codecNV = decoder_to_nvidia(vid.codec)
vid.count = videoinfo.count
vid.duration = videoinfo.duration
vid.pix_fmt = pix_fmt
if infile_options is None:
infile_options = ''
numGPU = get_num_NVIDIA_GPUs()
assert numGPU > 0, "No GPU found"
gpu = int(gpu) % numGPU if gpu is not None else 0
(
(vid.crop_width, vid.crop_height),
(vid.width, vid.height),
(cropopt, scaleopt, filteropt),
) = get_videofilter_gpu(
(vid.origin_width, vid.origin_height),
pix_fmt,
crop_xywh,
resize,
resize_keepratio,
resize_keepratioalign,
)
vid.size = (vid.width, vid.height)
rtsp_opt = "-rtsp_transport tcp " if stream_url.startswith("rtsp://") else ""
vid.ffmpeg_cmd = (
f"ffmpeg -loglevel error -hwaccel cuda -hwaccel_device {gpu} "
f" {infile_options} {rtsp_opt} "
f"{cropopt} {scaleopt} "
"-fflags nobuffer -flags low_delay -strict experimental "
f' -vcodec {vid.codecNV} -i "{stream_url}" '
f" {filteropt} -pix_fmt {pix_fmt} -f rawvideo pipe:"
)
vid.out_numpy_shape = get_outnumpyshape(vid.size, pix_fmt)
return vid
================================================
FILE: ffmpegcv/ffmpeg_writer.py
================================================
import numpy as np
import warnings
import pprint
import select
import sys
from .video_info import run_async, release_process_writer, get_num_NVIDIA_GPUs
IN_COLAB = "google.colab" in sys.modules
class FFmpegWriter:
def __init__(self):
self.iframe = -1
self.size = None
self.width, self.height = None, None
self.waitInit = True
self._isopen = True
def __enter__(self):
return self
def __exit__(self, type, value, traceback):
self.release()
def __del__(self):
self.release()
def __repr__(self):
props = pprint.pformat(self.__dict__).replace("{", " ").replace("}", " ")
return f"{self.__class__}\n" + props
@staticmethod
def VideoWriter(
filename, codec, fps, pix_fmt, bitrate=None, resize=None, preset=None
):
if codec is None:
codec = "h264"
elif not isinstance(codec, str):
codec = "h264"
warnings.simplefilter(
"""
Codec should be a string. Eg `h264`, `h264_nvenc`.
You may used CV2.VideoWriter_fourcc, which will be ignored.
"""
)
assert resize is None or len(resize) == 2
vid = FFmpegWriter()
vid.fps = fps
vid.codec, vid.pix_fmt, vid.filename = codec, pix_fmt, filename
vid.bitrate = bitrate
vid.resize = resize
vid.preset = preset
return vid
def _init_video_stream(self):
bitrate_str = f"-b:v {self.bitrate} " if self.bitrate else ""
rtsp_str = f"-f rtsp" if self.filename.startswith("rtsp://") else ""
filter_str = (
""
if self.resize == self.size
else f"-vf scale={self.resize[0]}:{self.resize[1]}"
)
target_pix_fmt = getattr(self, "target_pix_fmt", "yuv420p")
preset_str = f"-preset {self.preset} " if self.preset else ""
self.ffmpeg_cmd = (
f"ffmpeg -y -loglevel error "
f"-f rawvideo -pix_fmt {self.pix_fmt} -s {self.width}x{self.height} -r {self.fps} -i pipe: "
f"{bitrate_str} "
f"-r {self.fps} -c:v {self.codec} "
f"{preset_str}"
f"{filter_str} {rtsp_str} "
f'-pix_fmt {target_pix_fmt} "{self.filename}"'
)
self.process = run_async(self.ffmpeg_cmd)
def write(self, img: np.ndarray):
if self.waitInit:
if self.pix_fmt in ("nv12", "yuv420p", "yuvj420p"):
height_15, width = img.shape[:2]
assert width % 2 == 0 and height_15 * 2 % 3 == 0
height = int(height_15 / 1.5)
else:
height, width = img.shape[:2]
self.width, self.height = width, height
self.in_numpy_shape = img.shape
self.size = (width, height)
self.resize = self.size if self.resize is None else tuple(self.resize)
self._init_video_stream()
self.waitInit = False
self.iframe += 1
assert self.in_numpy_shape == img.shape
img = img.astype(np.uint8).tobytes()
self.process.stdin.write(img)
stderrreadable, _, _ = select.select([self.process.stderr], [], [], 0)
if stderrreadable:
data = self.process.stderr.read(1024)
sys.stderr.buffer.write(data)
def isOpened(self):
return self._isopen
def release(self):
self._isopen = False
if hasattr(self, "process"):
release_process_writer(self.process)
def close(self):
return self.release()
class FFmpegWriterNV(FFmpegWriter):
@staticmethod
def VideoWriter(
filename, codec, fps, pix_fmt, gpu, bitrate=None, resize=None, preset=None
):
numGPU = get_num_NVIDIA_GPUs()
assert numGPU
gpu = int(gpu) % numGPU if gpu is not None else 0
if codec is None:
codec = "hevc_nvenc"
elif not isinstance(codec, str):
codec = "hevc_nvenc"
warnings.simplefilter(
"""
Codec should be a string. Eg `h264`, `h264_nvenc`.
You may used CV2.VideoWriter_fourcc, which will be ignored.
"""
)
elif codec.endswith("_nvenc"):
codec = codec
else:
codec = codec + "_nvenc"
assert codec in [
"hevc_nvenc",
"h264_nvenc",
], "codec should be `hevc_nvenc` or `h264_nvenc`"
assert resize is None or len(resize) == 2
vid = FFmpegWriterNV()
vid.fps = fps
vid.codec, vid.pix_fmt, vid.filename = codec, pix_fmt, filename
vid.gpu = gpu
vid.bitrate = bitrate
vid.resize = resize
vid.preset = preset if preset is not None else ("default" if IN_COLAB else "fast")
return vid
def _init_video_stream(self):
bitrate_str = f"-b:v {self.bitrate} " if self.bitrate else ""
rtsp_str = f"-f rtsp" if self.filename.startswith("rtsp://") else ""
filter_str = (
""
if self.resize == self.size
else f"-vf scale={self.resize[0]}:{self.resize[1]}"
)
self.ffmpeg_cmd = (
f"ffmpeg -y -loglevel error "
f"-f rawvideo -pix_fmt {self.pix_fmt} -s {self.width}x{self.height} -r {self.fps} -i pipe: "
f"-preset {self.preset} {bitrate_str} "
f"-r {self.fps} -gpu {self.gpu} -c:v {self.codec} "
f"{filter_str} {rtsp_str} "
f'-pix_fmt yuv420p "{self.filename}"'
)
self.process = run_async(self.ffmpeg_cmd)
================================================
FILE: ffmpegcv/ffmpeg_writer_noblock.py
================================================
from multiprocessing import Queue, Process, Array
import numpy as np
from .ffmpeg_writer import FFmpegWriter
NFRAME = 10
class FFmpegWriterNoblock(FFmpegWriter):
def __init__(self,
vwriter_fun,
*vwriter_args, **vwriter_kwargs):
super().__init__()
vid:FFmpegWriter = vwriter_fun(*vwriter_args, **vwriter_kwargs)
vid.release()
props_name = ['width', 'height', 'fps', 'codec', 'pix_fmt',
'filename', 'size', 'bitrate']
for name in props_name:
setattr(self, name, getattr(vid, name, None))
self.vwriter_fun = vwriter_fun
self.vwriter_args = vwriter_args
self.vwriter_kwargs = vwriter_kwargs
self.q = Queue(maxsize=(NFRAME-2)) #buffer index, gluttonous snake NO biting its own tail
self.waitInit = True
self.process = None
def write(self, img:np.ndarray):
if self.waitInit:
if self.size is None:
self.size = (img.shape[1], img.shape[0])
else:
assert tuple(self.size) == (img.shape[1], img.shape[0])
self.in_numpy_shape = img.shape
self._init_share_array()
process = Process(target=child_process,
args=(self.shared_array, self.q, self.in_numpy_shape,
self.vwriter_fun, self.vwriter_args, self.vwriter_kwargs))
process.start()
self.process = process
self.width, self.height = self.size
self.waitInit = False
self.iframe += 1
data_id = self.iframe % NFRAME
self.np_array[data_id] = img
self.q.put(data_id)
def _init_share_array(self):
self.shared_array = Array('b', int(NFRAME*np.prod(self.in_numpy_shape)))
self.np_array = np.frombuffer(self.shared_array.get_obj(), dtype=np.uint8).reshape((NFRAME,*self.in_numpy_shape))
def release(self):
if self.process is not None and self.process.is_alive():
self.q.put(None)
self.process.join()
def child_process(shared_array, q:Queue, in_numpy_shape, vwriter_fun, vwriter_args, vwriter_kwargs):
vid = vwriter_fun(*vwriter_args, **vwriter_kwargs)
np_array = np.frombuffer(shared_array.get_obj(), dtype=np.uint8).reshape((NFRAME,*in_numpy_shape))
with vid:
while True:
data_id = q.get()
if data_id is None:
break
else:
img = np_array[data_id]
vid.write(img)
================================================
FILE: ffmpegcv/ffmpeg_writer_qsv.py
================================================
import warnings
from ffmpegcv.ffmpeg_writer import FFmpegWriter
from .video_info import (
run_async,
get_num_QSV_GPUs,
decoder_to_qsv,
)
class FFmpegWriterQSV(FFmpegWriter):
@staticmethod
def VideoWriter(filename, codec, fps, pix_fmt, gpu, bitrate=None, resize=None, preset=None):
assert gpu is None or gpu == 0, 'Cannot use multiple QSV gpu yet.'
numGPU = get_num_QSV_GPUs()
assert numGPU
gpu = int(gpu) % numGPU if gpu is not None else 0
if codec is None:
codec = "hevc_qsv"
elif not isinstance(codec, str):
codec = "hevc_qsv"
warnings.simplefilter(
"""
Codec should be a string. Eg `h264`, `hevc`.
You may used CV2.VideoWriter_fourcc, which will be ignored.
"""
)
else:
codec = decoder_to_qsv(codec)
assert resize is None or len(resize) == 2
vid = FFmpegWriterQSV()
vid.fps = fps
vid.codec, vid.pix_fmt, vid.filename = codec, pix_fmt, filename
vid.gpu = gpu
vid.bitrate = bitrate
vid.resize = resize
vid.preset = preset
return vid
================================================
FILE: ffmpegcv/ffmpeg_writer_stream_realtime.py
================================================
from .video_info import run_async
from ffmpegcv.ffmpeg_writer import FFmpegWriter
class FFmpegWriterStreamRT(FFmpegWriter):
@staticmethod
def VideoWriter(
filename: str, codec, pix_fmt, bitrate=None, resize=None, preset=None
) -> FFmpegWriter:
assert codec in ["h264", "libx264", "x264", "mpeg4"]
assert pix_fmt in ["bgr24", "rgb24", "gray"]
# assert filename.startswith('rtmp://'), 'currently only support rtmp'
assert resize is None or len(resize) == 2
vid = FFmpegWriterStreamRT()
vid.filename = filename
vid.codec = codec
vid.pix_fmt = pix_fmt
vid.bitrate = bitrate
vid.resize = resize
if preset is not None:
print("Preset is auto configured in FFmpegWriterStreamRT")
vid.preset = "ultrafast"
return vid
def _init_video_stream(self):
bitrate_str = f"-b:v {self.bitrate} " if self.bitrate else ""
rtsp_str = f"-f rtsp" if self.filename.startswith("rtsp://") else ""
filter_str = (
""
if self.resize == self.size
else f"-vf scale={self.resize[0]}:{self.resize[1]}"
)
self.ffmpeg_cmd = (
f"ffmpeg -loglevel warning "
f"-f rawvideo -pix_fmt {self.pix_fmt} -s {self.width}x{self.height} -i pipe: "
f"{bitrate_str} -f flv -rtsp_transport tcp "
f" -tune zerolatency -preset {self.preset} "
f"{filter_str} {rtsp_str} "
f' -c:v {self.codec} -g 50 -pix_fmt yuv420p "{self.filename}"'
)
self.process = run_async(self.ffmpeg_cmd)
================================================
FILE: ffmpegcv/stream_info.py
================================================
import subprocess
from collections import namedtuple
import json
import shlex
def get_info(stream_url, timeout=None, duration_ms: int = 100):
rtsp_opt = '' if not stream_url.startswith('rtsp://') else '-rtsp_flags prefer_tcp -pkt_size 736 '
analyze_duration = f'-analyzeduration {duration_ms * 1000}'
cmd = (f'ffprobe -v quiet -print_format json=compact=1 {rtsp_opt} {analyze_duration} '
f'-select_streams v:0 -show_format -show_streams "{stream_url}"')
output = subprocess.check_output(shlex.split(cmd), shell=False, timeout=timeout)
data: dict = json.loads(output)
vinfo: dict = data['streams'][0]
StreamInfo = namedtuple(
"StreamInfo", ["width", "height", "fps", "count", "codec", "duration"]
)
outinfo = dict()
outinfo["width"] = int(vinfo["width"])
outinfo["height"] = int(vinfo["height"])
if 'avg_frame_rate' in vinfo and vinfo['avg_frame_rate']!= '0/0':
outinfo["fps"] = eval(vinfo["avg_frame_rate"])
else:
outinfo["fps"] = eval(vinfo["r_frame_rate"])
outinfo["count"] = None
outinfo["codec"] = vinfo["codec_name"]
outinfo["duration"] = None
streaminfo = StreamInfo(**outinfo)
return streaminfo
if __name__ == "__main__":
stream_url = "http://devimages.apple.com.edgekey.net/streaming/examples/bipbop_4x3/gear2/prog_index.m3u8"
streaminfo = get_info(stream_url)
print(streaminfo)
================================================
FILE: ffmpegcv/version.py
================================================
__version__='0.3.19'
================================================
FILE: ffmpegcv/video_info.py
================================================
import subprocess
from subprocess import Popen, PIPE
import re
from collections import namedtuple
import json
import shlex
import platform
scan_the_whole = {"mkv", "flv", "ts"} # scan the whole file to the count, slow
_is_windows = platform.system() == "Windows"
_inited_get_num_NVIDIA_GPUs = False
_inited_get_num_QSV_GPUs = False
_num_NVIDIA_GPUs = -1
_num_QSV_GPUs = -1
def get_info(video: str):
do_scan_the_whole = video.split(".")[-1] in scan_the_whole
def ffprobe_info_(do_scan_the_whole):
use_count_packets = '-count_packets' if do_scan_the_whole else ''
cmd = 'ffprobe -v quiet -print_format json=compact=1 -select_streams v:0 {} -show_streams "{}"'.format(
use_count_packets, video)
output = subprocess.check_output(shlex.split(cmd), shell=False)
data: dict = json.loads(output)
vinfo: dict = data['streams'][0]
return vinfo
vinfo = ffprobe_info_(do_scan_the_whole)
if "nb_frames" not in vinfo:
do_scan_the_whole = True
vinfo = ffprobe_info_(do_scan_the_whole)
# VideoInfo = namedtuple(
# "VideoInfo", ["width", "height", "fps", "count", "codec", "duration", "pix_fmt"]
# )
VideoInfo = namedtuple(
"VideoInfo", ["width", "height", "fps", "count", "codec", "duration"]
)
outinfo = dict()
outinfo["width"] = int(vinfo["width"])
outinfo["height"] = int(vinfo["height"])
outinfo["fps"] = eval(vinfo["r_frame_rate"])
outinfo["count"] = int(
vinfo["nb_read_packets" if do_scan_the_whole else "nb_frames"]
) # nb_read_packets | nb_frames
outinfo["codec"] = vinfo["codec_name"]
# outinfo['pix_fmt'] = vinfo['pix_fmt']
outinfo["duration"] = (
float(vinfo["duration"])
if "duration" in vinfo
else outinfo["count"] / outinfo["fps"]
)
videoinfo = VideoInfo(**outinfo)
return videoinfo
def get_info_precise(video: str):
videoinfo = get_info(video)
cmd = (
"ffprobe -v error -select_streams v:0 -show_entries frame=pts_time "
f' -of default=noprint_wrappers=1:nokey=1 -read_intervals 0%+#1,99999% "{video}"'
)
output = subprocess.check_output(
shlex.split(cmd), shell=False, stderr=subprocess.DEVNULL
)
pts_start, *_, pts_end = output.decode().split()
pts_start, pts_end = float(pts_start), float(pts_end)
videoinfod = videoinfo._asdict()
duration_ = pts_end - pts_start
videoinfod["fps"] = round((videoinfo.count - 1) / duration_, 3)
videoinfod["duration"] = round(duration_ + 1 / videoinfod["fps"], 3)
videoinfo_precise = videoinfo.__class__(*videoinfod.values())
return videoinfo_precise
def get_num_NVIDIA_GPUs():
global _num_NVIDIA_GPUs, _inited_get_num_NVIDIA_GPUs
if not _inited_get_num_NVIDIA_GPUs:
cmd = "ffmpeg -f lavfi -i nullsrc -c:v h264_nvenc -gpu list -f null -"
p = Popen(cmd.split(), shell=False, stdin=PIPE, stdout=PIPE, stderr=PIPE)
stdout, stderr = p.communicate(b"")
p.stdin.close()
p.stdout.close()
p.terminate()
pattern = re.compile(r"GPU #\d+ - < ")
nv_info = pattern.findall(stderr.decode())
_num_NVIDIA_GPUs = len(nv_info)
_inited_get_num_NVIDIA_GPUs = True
return _num_NVIDIA_GPUs
def get_num_QSV_GPUs():
global _num_QSV_GPUs, _inited_get_num_QSV_GPUs
if not _inited_get_num_QSV_GPUs:
cmd = "ffmpeg -hide_banner -f qsv -h encoder=h264_qsv"
p = Popen(cmd.split(), shell=False, stdin=PIPE, stdout=PIPE, stderr=PIPE)
stdout, stderr = p.communicate(b"")
_num_QSV_GPUs = 1 if len(stdout) > 50 else 0
_inited_get_num_QSV_GPUs = True
return _num_QSV_GPUs
def encoder_to_nvidia(codec):
codec_map = {"h264": "h264_nvenc", "hevc": "hevc_nvenc"}
if codec in codec_map:
return codec_map[codec]
elif codec in codec_map.values():
return codec
else:
raise Exception("No NV codec found for %s" % codec)
def encoder_to_qsv(codec):
codec_map = {
"h264": "h264_qsv",
"hevc": "hevc_qsv",
"mjpeg": "mjpeg_qsv",
"mpeg2video": "mpeg2_qsv",
"vp9": "vp9_qsv",
}
if codec in codec_map:
return codec_map[codec]
elif codec in codec_map.values():
return codec
else:
raise Exception("No QSV codec found for %s" % codec)
def decoder_to_nvidia(codec):
codec_map = {
"av1": "av1_cuvid",
"h264": "h264_cuvid",
"x264": "h264_cuvid",
"hevc": "hevc_cuvid",
"x265": "hevc_cuvid",
"h265": "hevc_cuvid",
"mjpeg": "mjpeg_cuvid",
"mpeg1video": "mpeg1_cuvid",
"mpeg2video": "mpeg2_cuvid",
"mpeg4": "mpeg4_cuvid",
"vp1": "vp1_cuvid",
"vp8": "vp8_cuvid",
"vp9": "vp9_cuvid",
}
if codec in codec_map:
return codec_map[codec]
elif codec in codec_map.values():
return codec
else:
raise Exception("No NV codec found for %s" % codec)
def decoder_to_qsv(codec):
codec_map = {
"av1": "av1_qsv",
"h264": "h264_qsv",
"hevc": "hevc_qsv",
"mjpeg": "mjpeg_qsv",
"mpeg2video": "mpeg2_qsv",
"vc1": "vc1_qsv",
"vp8": "vp8_qsv",
"vp9": "vp9_qsv",
}
if codec in codec_map:
return codec_map[codec]
elif codec in codec_map.values():
return codec
else:
raise Exception("No QSV codec found for %s" % codec)
def run_async(args):
bufsize = -1
if isinstance(args, str):
args = shlex.split(args)
return Popen(
args,
stdin=PIPE,
stdout=PIPE,
stderr=PIPE,
shell=False,
bufsize=bufsize,
)
def run_async_reader(args):
bufsize = -1
if isinstance(args, str):
args = shlex.split(args)
return Popen(
args,
stdin=None,
stdout=PIPE,
stderr=subprocess.DEVNULL,
shell=False,
bufsize=bufsize,
)
def release_process(process: Popen, forcekill=False):
if hasattr(process, "stdin") and process.stdin is not None:
process.stdin.close()
if hasattr(process, "stdout") and process.stdout is not None:
process.stdout.close()
if hasattr(process, "stderr") and process.stderr is not None:
process.stderr.close()
if forcekill and hasattr(process, "terminate") and not _is_windows:
process.terminate()
if forcekill and hasattr(process, "wait"):
process.wait()
def release_process_writer(process: Popen):
if hasattr(process, "stdin"):
process.stdin.close()
if hasattr(process, "stdout"):
process.stdout.close()
if hasattr(process, "wait"):
process.wait()
================================================
FILE: setup.py
================================================
# setup.py
from setuptools import setup, find_packages
from pathlib import Path
this_directory = Path(__file__).parent
long_description = (this_directory / "README.md").read_text("utf-8")
setup(
name="ffmpegcv", # 应用名
version="0.3.18", # 版本号
packages=find_packages(include=["ffmpegcv*"]), # 包括在安装包内的 Python 包
author="chenxf",
author_email="cxf529125853@163.com",
url="https://github.com/chenxinfeng4/ffmpegcv",
long_description=long_description,
long_description_content_type="text/markdown",
# 添加依赖项
python_requires=">=3.6",
install_requires=[
"numpy",
],
extras_require={"cuda": ["pycuda"]}, # 定义一个名为cuda的可选依赖项，并指定pycuda
)
gitextract_5kjt031g/ ├── .github/ │ └── workflows/ │ └── python-publish.yml ├── .gitignore ├── README.md ├── README_CN.md ├── ffmpegcv/ │ ├── __init__.py │ ├── ffmpeg_noblock.py │ ├── ffmpeg_reader.py │ ├── ffmpeg_reader_camera.py │ ├── ffmpeg_reader_cuda.py │ ├── ffmpeg_reader_noblock.py │ ├── ffmpeg_reader_pannels.py │ ├── ffmpeg_reader_qsv.py │ ├── ffmpeg_reader_stream.py │ ├── ffmpeg_reader_stream_realtime.py │ ├── ffmpeg_writer.py │ ├── ffmpeg_writer_noblock.py │ ├── ffmpeg_writer_qsv.py │ ├── ffmpeg_writer_stream_realtime.py │ ├── stream_info.py │ ├── version.py │ └── video_info.py └── setup.py
SYMBOL INDEX (141 symbols across 16 files)
FILE: ffmpegcv/__init__.py
function _check (line 18) | def _check():
function _check_nvidia (line 32) | def _check_nvidia():
function VideoCapture (line 64) | def VideoCapture(
function VideoWriter (line 167) | def VideoWriter(
function VideoCaptureNV (line 234) | def VideoCaptureNV(
function VideoCaptureQSV (line 257) | def VideoCaptureQSV(
function VideoWriterNV (line 279) | def VideoWriterNV(
function VideoWriterQSV (line 298) | def VideoWriterQSV(
function VideoWriterStreamRT (line 316) | def VideoWriterStreamRT(
function VideoCaptureCAM (line 324) | def VideoCaptureCAM(
function VideoCaptureStream (line 418) | def VideoCaptureStream(
function VideoCaptureStreamRT (line 490) | def VideoCaptureStreamRT(
function VideoCapturePannels (line 532) | def VideoCapturePannels(
function toCUDA (line 584) | def toCUDA(vid: FFmpegReader, gpu: int = 0, tensor_format: str = "chw") ...
FILE: ffmpegcv/ffmpeg_noblock.py
function noblock (line 10) | def noblock(fun:Callable, *v_args, **v_kargs):
class ReadLiveLast (line 24) | class ReadLiveLast(threading.Thread, ffmpegcv.FFmpegReader):
method __init__ (line 25) | def __init__(self, fun, *args, **kvargs):
method read (line 42) | def read(self):
method release (line 48) | def release(self):
method run (line 53) | def run(self):
FILE: ffmpegcv/ffmpeg_reader.py
function get_videofilter_cpu (line 16) | def get_videofilter_cpu(
function get_videofilter_gpu (line 82) | def get_videofilter_gpu(
function get_outnumpyshape (line 149) | def get_outnumpyshape(size_wh: list, pix_fmt: str) -> tuple:
class FFmpegReader (line 165) | class FFmpegReader:
method __init__ (line 166) | def __init__(self):
method __repr__ (line 179) | def __repr__(self):
method __enter__ (line 183) | def __enter__(self):
method __exit__ (line 186) | def __exit__(self, type, value, traceback):
method __len__ (line 189) | def __len__(self):
method __iter__ (line 192) | def __iter__(self):
method __next__ (line 195) | def __next__(self):
method VideoReader (line 203) | def VideoReader(
method read (line 260) | def read(self):
method isOpened (line 281) | def isOpened(self):
method release (line 284) | def release(self):
method close (line 288) | def close(self):
class FFmpegReaderNV (line 292) | class FFmpegReaderNV(FFmpegReader):
method _get_opts (line 293) | def _get_opts(
method VideoReader (line 371) | def VideoReader(
FILE: ffmpegcv/ffmpeg_reader_camera.py
class platform (line 13) | class platform:
function _query_camera_divices_mac (line 30) | def _query_camera_divices_mac() -> dict:
function _query_camera_divices_win (line 50) | def _query_camera_divices_win() -> dict:
function _query_camera_divices_linux (line 69) | def _query_camera_divices_linux() -> dict:
function query_camera_devices (line 94) | def query_camera_devices(verbose_dict: bool = False) -> dict:
function _query_camera_options_mac (line 109) | def _query_camera_options_mac(cam_id_name) -> str:
function _query_camera_options_linux (line 119) | def _query_camera_options_linux(cam_id_name) -> str:
function _query_camera_options_win (line 158) | def _query_camera_options_win(cam_id_name) -> str:
function query_camera_options (line 193) | def query_camera_options(cam_id_name) -> str:
class ProducerThread (line 201) | class ProducerThread(Thread):
method __init__ (line 202) | def __init__(self, vid, q):
method run (line 207) | def run(self):
class FFmpegReaderCAM (line 219) | class FFmpegReaderCAM:
method __init__ (line 220) | def __init__(self):
method __repr__ (line 224) | def __repr__(self):
method __enter__ (line 228) | def __enter__(self):
method __exit__ (line 231) | def __exit__(self, type, value, traceback):
method __iter__ (line 234) | def __iter__(self):
method __next__ (line 237) | def __next__(self):
method VideoReader (line 245) | def VideoReader(
method read_ (line 349) | def read_(self):
method read (line 361) | def read(self):
method isOpened (line 365) | def isOpened(self):
method release (line 368) | def release(self):
method close (line 372) | def close(self):
FILE: ffmpegcv/ffmpeg_reader_cuda.py
function load_cuda_module (line 123) | def load_cuda_module():
class Holder (line 132) | class Holder(PointerHolderBase):
method __init__ (line 133) | def __init__(self, tensor):
method get_pointer (line 138) | def get_pointer(self):
method __index__ (line 141) | def __index__(self):
function tensor_to_gpuarray (line 145) | def tensor_to_gpuarray(tensor) -> gpuarray.GPUArray:
class PycudaContext (line 162) | class PycudaContext:
method __init__ (line 163) | def __init__(self, gpu=0):
method __enter__ (line 166) | def __enter__(self):
method __exit__ (line 171) | def __exit__(self, *args, **kwargs):
method __del__ (line 175) | def __del__(self):
class FFmpegReaderCUDA (line 180) | class FFmpegReaderCUDA(FFmpegReader):
method __init__ (line 181) | def __init__(self, vid:FFmpegReader, gpu=0, tensor_format='hwc'):
method read (line 207) | def read(self, out_MAT:gpuarray.GPUArray=None) -> Tuple[bool, gpuarray...
method read_cudamem (line 221) | def read_cudamem(self, out_MAT:cuda.DeviceAllocation=None) -> Tuple[bo...
method read_torch (line 236) | def read_torch(self, out_MAT=None):
method release (line 252) | def release(self):
FILE: ffmpegcv/ffmpeg_reader_noblock.py
class FFmpegReaderNoblock (line 8) | class FFmpegReaderNoblock(FFmpegReader):
method __init__ (line 9) | def __init__(self,
method read (line 36) | def read(self):
function child_process (line 53) | def child_process(shared_array, q:Queue, vcap_fun, vcap_args, vcap_kwargs):
FILE: ffmpegcv/ffmpeg_reader_pannels.py
class FFmpegReaderPannels (line 11) | class FFmpegReaderPannels(FFmpegReader):
method VideoReader (line 13) | def VideoReader(
method read (line 67) | def read(self):
FILE: ffmpegcv/ffmpeg_reader_qsv.py
class FFmpegReaderQSV (line 10) | class FFmpegReaderQSV(FFmpegReader):
method VideoReader (line 12) | def VideoReader(
FILE: ffmpegcv/ffmpeg_reader_stream.py
class FFmpegReaderStream (line 14) | class FFmpegReaderStream(FFmpegReaderCAM):
method __init__ (line 15) | def __init__(self):
method VideoReader (line 19) | def VideoReader(
class FFmpegReaderStreamNV (line 77) | class FFmpegReaderStreamNV(FFmpegReaderCAM):
method __init__ (line 78) | def __init__(self):
method VideoReader (line 82) | def VideoReader(
FILE: ffmpegcv/ffmpeg_reader_stream_realtime.py
class FFmpegReaderStreamRT (line 12) | class FFmpegReaderStreamRT(FFmpegReader):
method __init__ (line 13) | def __init__(self):
method VideoReader (line 17) | def VideoReader(
class FFmpegReaderStreamRTNV (line 68) | class FFmpegReaderStreamRTNV(FFmpegReader):
method __init__ (line 69) | def __init__(self):
method VideoReader (line 73) | def VideoReader(
FILE: ffmpegcv/ffmpeg_writer.py
class FFmpegWriter (line 12) | class FFmpegWriter:
method __init__ (line 13) | def __init__(self):
method __enter__ (line 20) | def __enter__(self):
method __exit__ (line 23) | def __exit__(self, type, value, traceback):
method __del__ (line 26) | def __del__(self):
method __repr__ (line 29) | def __repr__(self):
method VideoWriter (line 34) | def VideoWriter(
method _init_video_stream (line 57) | def _init_video_stream(self):
method write (line 79) | def write(self, img: np.ndarray):
method isOpened (line 104) | def isOpened(self):
method release (line 107) | def release(self):
method close (line 112) | def close(self):
class FFmpegWriterNV (line 116) | class FFmpegWriterNV(FFmpegWriter):
method VideoWriter (line 118) | def VideoWriter(
method _init_video_stream (line 153) | def _init_video_stream(self):
FILE: ffmpegcv/ffmpeg_writer_noblock.py
class FFmpegWriterNoblock (line 7) | class FFmpegWriterNoblock(FFmpegWriter):
method __init__ (line 8) | def __init__(self,
method write (line 27) | def write(self, img:np.ndarray):
method _init_share_array (line 48) | def _init_share_array(self):
method release (line 52) | def release(self):
function child_process (line 58) | def child_process(shared_array, q:Queue, in_numpy_shape, vwriter_fun, vw...
FILE: ffmpegcv/ffmpeg_writer_qsv.py
class FFmpegWriterQSV (line 10) | class FFmpegWriterQSV(FFmpegWriter):
method VideoWriter (line 12) | def VideoWriter(filename, codec, fps, pix_fmt, gpu, bitrate=None, resi...
FILE: ffmpegcv/ffmpeg_writer_stream_realtime.py
class FFmpegWriterStreamRT (line 5) | class FFmpegWriterStreamRT(FFmpegWriter):
method VideoWriter (line 7) | def VideoWriter(
method _init_video_stream (line 25) | def _init_video_stream(self):
FILE: ffmpegcv/stream_info.py
function get_info (line 7) | def get_info(stream_url, timeout=None, duration_ms: int = 100):
FILE: ffmpegcv/video_info.py
function get_info (line 18) | def get_info(video: str):
function get_info_precise (line 63) | def get_info_precise(video: str):
function get_num_NVIDIA_GPUs (line 82) | def get_num_NVIDIA_GPUs():
function get_num_QSV_GPUs (line 98) | def get_num_QSV_GPUs():
function encoder_to_nvidia (line 109) | def encoder_to_nvidia(codec):
function encoder_to_qsv (line 120) | def encoder_to_qsv(codec):
function decoder_to_nvidia (line 137) | def decoder_to_nvidia(codec):
function decoder_to_qsv (line 162) | def decoder_to_qsv(codec):
function run_async (line 182) | def run_async(args):
function run_async_reader (line 195) | def run_async_reader(args):
function release_process (line 210) | def release_process(process: Popen, forcekill=False):
function release_process_writer (line 223) | def release_process_writer(process: Popen):
Condensed preview — 22 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (130K chars).
[
{
"path": ".github/workflows/python-publish.yml",
"chars": 1112,
"preview": "# This workflow will upload a Python Package using Twine when a release is created\n# For more information see: https://d"
},
{
"path": ".gitignore",
"chars": 37,
"preview": "/build\n*.pyc\n/dist\n/ffmpegcv.egg-info"
},
{
"path": "README.md",
"chars": 17227,
"preview": "# FFMPEGCV is an alternative to OPENCV for video reading&writing.\n![Python versions](https://img.shields.io/badge/Python"
},
{
"path": "README_CN.md",
"chars": 12913,
"preview": "# FFMPEGCV 读写视频，替代 OPENCV.\n\n[![PyPI version](http"
},
{
"path": "ffmpegcv/__init__.py",
"chars": 15115,
"preview": "from .ffmpeg_reader import FFmpegReader, FFmpegReaderNV\nfrom .ffmpeg_writer import FFmpegWriter, FFmpegWriterNV\nfrom .ff"
},
{
"path": "ffmpegcv/ffmpeg_noblock.py",
"chars": 2095,
"preview": "from .ffmpeg_reader_noblock import FFmpegReaderNoblock\nfrom .ffmpeg_writer_noblock import FFmpegWriterNoblock\nfrom typin"
},
{
"path": "ffmpegcv/ffmpeg_reader.py",
"chars": 14315,
"preview": "import numpy as np\nimport pprint\nimport warnings\nimport os\nimport sys\nimport select\nfrom .video_info import (\n run_as"
},
{
"path": "ffmpegcv/ffmpeg_reader_camera.py",
"chars": 12324,
"preview": "import numpy as np\nimport pprint\nfrom .video_info import run_async, release_process\nimport re\nimport subprocess\nfrom thr"
},
{
"path": "ffmpegcv/ffmpeg_reader_cuda.py",
"chars": 9109,
"preview": "import pycuda.driver as cuda\nfrom pycuda.driver import PointerHolderBase\nfrom pycuda.compiler import SourceModule\nfrom p"
},
{
"path": "ffmpegcv/ffmpeg_reader_noblock.py",
"chars": 2340,
"preview": "import multiprocessing\nfrom multiprocessing import Queue\nimport numpy as np\nfrom .ffmpeg_reader import FFmpegReader\n\nNFR"
},
{
"path": "ffmpegcv/ffmpeg_reader_pannels.py",
"chars": 3234,
"preview": "import os\nimport numpy as np\nfrom ffmpegcv.ffmpeg_reader import FFmpegReader, get_outnumpyshape\n\nfrom ffmpegcv.video_inf"
},
{
"path": "ffmpegcv/ffmpeg_reader_qsv.py",
"chars": 2150,
"preview": "import os\nfrom ffmpegcv.ffmpeg_reader import FFmpegReader, get_videofilter_cpu, get_outnumpyshape\nfrom .video_info impor"
},
{
"path": "ffmpegcv/ffmpeg_reader_stream.py",
"chars": 4145,
"preview": "from .video_info import run_async\nfrom queue import Queue\nfrom ffmpegcv.stream_info import get_info\nfrom ffmpegcv.ffmpeg"
},
{
"path": "ffmpegcv/ffmpeg_reader_stream_realtime.py",
"chars": 3813,
"preview": "from ffmpegcv.ffmpeg_reader import (\n FFmpegReader,\n get_videofilter_cpu,\n get_outnumpyshape,\n get_videofilt"
},
{
"path": "ffmpegcv/ffmpeg_writer.py",
"chars": 5671,
"preview": "import numpy as np\nimport warnings\nimport pprint\nimport select\nimport sys\nfrom .video_info import run_async, release_pro"
},
{
"path": "ffmpegcv/ffmpeg_writer_noblock.py",
"chars": 2564,
"preview": "from multiprocessing import Queue, Process, Array\nimport numpy as np\nfrom .ffmpeg_writer import FFmpegWriter\n\nNFRAME = 1"
},
{
"path": "ffmpegcv/ffmpeg_writer_qsv.py",
"chars": 1210,
"preview": "import warnings\nfrom ffmpegcv.ffmpeg_writer import FFmpegWriter\nfrom .video_info import (\n run_async,\n get_num_QSV"
},
{
"path": "ffmpegcv/ffmpeg_writer_stream_realtime.py",
"chars": 1629,
"preview": "from .video_info import run_async\nfrom ffmpegcv.ffmpeg_writer import FFmpegWriter\n\n\nclass FFmpegWriterStreamRT(FFmpegWri"
},
{
"path": "ffmpegcv/stream_info.py",
"chars": 1412,
"preview": "import subprocess\nfrom collections import namedtuple\nimport json\nimport shlex\n\n\ndef get_info(stream_url, timeout=None, d"
},
{
"path": "ffmpegcv/version.py",
"chars": 20,
"preview": "__version__='0.3.19'"
},
{
"path": "ffmpegcv/video_info.py",
"chars": 6764,
"preview": "import subprocess\nfrom subprocess import Popen, PIPE\nimport re\nfrom collections import namedtuple\nimport json\nimport shl"
},
{
"path": "setup.py",
"chars": 691,
"preview": "# setup.py\nfrom setuptools import setup, find_packages\nfrom pathlib import Path\n\nthis_directory = Path(__file__).parent\n"
}
]
About this extraction
This page contains the full source code of the chenxinfeng4/ffmpegcv GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 22 files (117.1 KB), approximately 34.1k tokens, and a symbol index with 141 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.