[
{
"path": ".github/workflows/python-publish.yml",
"content": "# This workflow will upload a Python Package using Twine when a release is created\n# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python#publishing-to-package-registries\n\n# This workflow uses actions that are not certified by GitHub.\n# They are provided by a third-party and are governed by\n# separate terms of service, privacy policy, and support\n# documentation.\n\nname: Upload Python Package\n\non:\n release:\n types: [published]\n\npermissions:\n contents: read\n\njobs:\n deploy:\n\n runs-on: ubuntu-latest\n\n steps:\n - uses: actions/checkout@v3\n - name: Set up Python\n uses: actions/setup-python@v3\n with:\n python-version: '3.x'\n - name: Install dependencies\n run: |\n python -m pip install --upgrade pip\n pip install build\n - name: Build package\n run: python -m build\n - name: Publish package\n uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29\n with:\n skip-existing: true\n user: __token__\n password: ${{ secrets.PYPI_API_TOKEN }}\n"
},
{
"path": ".gitignore",
"content": "/build\n*.pyc\n/dist\n/ffmpegcv.egg-info"
},
{
"path": "README.md",
"content": "# FFMPEGCV is an alternative to OPENCV for video reading&writing.\n\n[](https://pypi.org/project/ffmpegcv/)\n[](https://pepy.tech/project/ffmpegcv)\n\n\n\nEnglish Version | [中文版本](./README_CN.md)\n\nHere is the Python version of ffmpegcv. For the C++ version, please visit [FFMPEGCV-CPP](https://github.com/chenxinfeng4/ffmpegcv-cpp)\n\nThe ffmpegcv provide Video Reader and Video Witer with ffmpeg backbone, which are faster and powerful than cv2. Integrating ffmpegcv into your deeplearning pipeline is very smooth.\n\n- The ffmpegcv is api **compatible** to open-cv. \n- The ffmpegcv can use **GPU accelerate** encoding and decoding*.\n- The ffmpegcv supports much more video **codecs** v.s. open-cv.\n- The ffmpegcv supports **RGB** & BGR & GRAY format as you like.\n- The ffmpegcv supports fp32 CHW & HWC format shortcut to CUDA memory.\n- The ffmpegcv supports **Stream reading** (IP Camera) in low latency.\n- The ffmpegcv supports ROI operations.You can **crop**, **resize** and **pad** the ROI.\n\nIn all, ffmpegcv is just similar to opencv api. But it has more codecs and does't require opencv installed at all. It's great for deeplearning pipeline.\n\n\n\n![](\"https://i.imghippo.com/files/cg9641723107581.jpg\")
\n
\n\n## Functions:\n- `VideoWriter`: Write a video file.\n- `VideoCapture`: Read a video file.\n- `VideoCaptureNV`: Read a video file by NVIDIA GPU.\n- `VideoCaptureQSV`: Read a video file by Intel QuickSync Video.\n- `VideoCaptureCAM`: Read a camera.\n- `VideoCaptureStream`: Read a RTP/RTSP/RTMP/HTTP stream.\n- `VideoCaptureStreamRT`: Read a RTSP stream (IP Camera) in real time low latency as possible.\n- `noblock`: Read/Write a video file in background using mulitprocssing.\n- `toCUDA`: Translate a video/stream as CHW/HWC-float32 format into CUDA device, >2x faster.\n\n## Install\nYou need to download ffmpeg before you can use ffmpegcv.\n```\n #1A. LINUX: sudo apt install ffmpeg\n #1B. MAC (No NVIDIA GPU): brew install ffmpeg\n #1C. WINDOWS: download ffmpeg and add to the path\n #1D. CONDA: conda install ffmpeg=6.0.0 #don't use the default 4.x.x version\n \n #2A. python\n pip install ffmpegcv #stable verison\n pip install git+https://github.com/chenxinfeng4/ffmpegcv #latest verison\n\n #2B. recommand only when you want advanced functions. See the toCUDA section\n pip install ffmpegcv[cuda]\n```\n\n## When should choose `ffmpegcv` other than `opencv`:\n- The `opencv` is hard to install. The ffmpegcv only requires `numpy` and `FFmpeg`, works across Mac/Windows/Linux platforms.\n- The `opencv` packages too much image processing toolbox. You just want a simple video/camero IO with GPU accessible.\n- The `opencv` didn't support profiling `h264`/`h265` and other video writers.\n- You want to **crop**, **resize** and **pad** the video/camero ROI.\n- You are interested in deeplearning pipeline.\n## Basic example\nRead a video by CPU, and rewrite it by GPU.\n```python\nvidin = ffmpegcv.VideoCapture(vfile_in)\nvidout = ffmpegcv.VideoWriterNV(vfile_out, 'h264', vidin.fps) #NVIDIA-GPU\n\nwith vidin, vidout:\n for frame in vidin:\n cv2.imshow('image', frame)\n vidout.write(frame)\n```\n\nRead the camera.\n```python\n# by device ID\ncap = ffmpegcv.VideoCaptureCAM(0)\n# by device name\ncap = ffmpegcv.VideoCaptureCAM(\"Integrated Camera\")\n```\n\nDeeplearning pipeline.\n```python\n\"\"\"\n —————————— NVIDIA GPU accelerating ⤴⤴ ———————\n | |\n V V\nvideo -> decode -> crop -> resize -> RGB -> CUDA:CHW float32 -> model\n\"\"\"\ncap = ffmpegcv.toCUDA(\n ffmpegcv.VideoCaptureNV(file, pix_fmt='nv12', resize=(W,H)),\n tensor_format='chw')\n\nfor frame_CHW_cuda in cap:\n frame_CHW_cuda = (frame_CHW_cuda - mean) / std\n result = model(frame_CHW_cuda)\n```\n## Cross platform\n\nThe ffmpegcv is based on Python+FFmpeg, it can cross platform among `Windows, Linux, Mac, X86, Arm`systems.\n\n## GPU Acceleration\n\n- Support **NVIDIA** card only, test in x86_64 only.\n- Works in **Windows**, **Linux** and **Anaconda**.\n- Works in the **Google Colab** notebook. \n- Infeasible in the **MacOS**. That ffmpeg didn't supports NVIDIA at all.\n\n> \\* The ffmegcv GPU reader is a bit slower than CPU reader, but much faster when use ROI operations (crop, resize, pad).\n\n## Codecs\n\n| Codecs | OpenCV-reader | ffmpegcv-cpu-r | gpu-r | OpenCV-writer | ffmpegcv-cpu-w | gpu-w |\n| ----------- | ------------- | ---------------- | ---- | ------------- | ---------------- | ---- |\n| h264 | √ | √ | √ | × | √ | √ |\n| h265 (hevc) | not sure | √ | √ | × | √ | √ |\n| mjpeg | √ | √ | × | √ | √ | × |\n| mpeg | √ | √ | × | √ | √ | × |\n| others | not sure | ffmpeg -decoders | × | not sure | ffmpeg -encoders | × |\n\n## Benchmark\n*On the way...（maybe never）*\n\n\n## Video Reader\n---\nThe ffmpegcv is just similar to opencv in api.\n```python\n# open cv\nimport cv2\ncap = cv2.VideoCapture(file)\nwhile True:\n ret, frame = cap.read()\n if not ret:\n break\n pass\n\n# ffmpegcv\nimport ffmpegcv\ncap = ffmpegcv.VideoCapture(file)\nwhile True:\n ret, frame = cap.read()\n if not ret:\n break\n pass\ncap.release()\n\n# alternative\ncap = ffmpegcv.VideoCapture(file)\nnframe = len(cap)\nfor frame in cap:\n pass\ncap.release()\n\n# more pythonic, recommand\nwith ffmpegcv.VideoCapture(file) as cap:\n nframe = len(cap)\n for iframe, frame in enumerate(cap):\n if iframe>100: break\n pass\n```\n\nUse GPU to accelerate decoding. It depends on the video codes.\nh264_nvcuvid, hevc_nvcuvid ....\n```python\ncap_cpu = ffmpegcv.VideoCapture(file)\ncap_gpu0 = ffmpegcv.VideoCaptureNV(file) #NVIDIA GPU0\ncap_gpu1 = ffmpegcv.VideoCaptureNV(file, gpu=1) #NVIDIA GPU1\ncap_qsv = ffmpegcv.VideoCaptureQSV(file) #Intel QSV, experimental\n```\n\nUse `rgb24` instead of `bgr24`. The `gray` version would be more efficient.\n```python\ncap = ffmpegcv.VideoCapture(file, pix_fmt='rgb24') #rgb24, bgr24, gray\nret, frame = cap.read()\nplt.imshow(frame)\n```\n\n### ROI Operations\nYou can crop, resize and pad the video. These ROI operation is `ffmpegcv-GPU` > `ffmpegcv-CPU` >> `opencv`.\n\n**Crop** video, which will be much faster than read the whole canvas. The top-left corner is (0, 0).\n```python\ncap = ffmpegcv.VideoCapture(file, crop_xywh=(0, 0, 640, 480))\n```\n\n**Resize** the video to the given size.\n```python\ncap = ffmpegcv.VideoCapture(file, resize=(640, 480))\n```\n\n**Resize** and keep the aspect ratio with black border **padding**.\n```python\ncap = ffmpegcv.VideoCapture(file, resize=(640, 480), resize_keepratio=True)\n```\n\n**Crop** and then **resize** the video.\n```python\ncap = ffmpegcv.VideoCapture(file, crop_xywh=(0, 0, 640, 480), resize=(512, 512))\n```\n\n### Extend Options\n**INFILE_OPTIONS**: Add extra options to ffmpeg input. \n\n```python\ncap = ffmpegcv.VideoCapture(file, infile_options='-re -stream_loop -1')\n\n# equivalent ffmpeg command\nffmpeg INFILE_OPTIONS -i FILE -f rawvideo pipe:\n```\n\n## toCUDA device\n---\nThe ffmpegcv can translate the video/stream from HWC-uint8 cpu to CHW-float32 in CUDA device. It significantly reduce your cpu load, and get >2x faster than your manually convertion.\n\nPrepare your environment. The cuda environment is required. The `pycuda` package is required. The `pytorch` package is non-essential.\n> nvcc --version # check you've installed NVIDIA CUDA Compiler. Already installed if\n> you've installed Tensorflow-gpu or Pytorch-gpu\n>\n> pip install ffmpegcv[cuda] #auto install pycuda\n\n```python\n# Read a video file to CUDA device, original\ncap = ffmpegcv.VideoCaptureNV(file, pix_fmt='rgb24')\nret, frame_HWC_CPU = cap.read()\nframe_CHW_CUDA = torch.from_numpy(frame_HWC_CPU).permute(2, 0, 1).cuda().contiguous().float() # 120fps, 1200% CPU load\n\n# speed up\ncap = toCUDA(ffmpegcv.VideoCapture(file, pix_fmt='yuv420p')) #pix_fmt: 'yuv420p' or 'nv12' only\ncap = toCUDA(ffmpegcv.VideoCaptureNV(file, pix_fmt='nv12')) #'nv12' is better for gpu\ncap = toCUDA(vid, tensor_format='chw') #tensor format:'chw'(default) or 'hwc', fp32 precision\ncap = toCUDA(vid, gpu=1) #choose gpu\n\n# read to the cuda device\nret, frame_CHW_pycuda = cap.read() #380fps, 200% CPU load, dtype is [pycuda array]\nret, frame_CHW_pycudamem = cap.read_cudamem() #dtype is [pycuda mem_alloc]\nret, frame_CHW_CUDA = cap.read_torch() #dtype is [pytorch tensor]\nret, _ = cap.read_torch(frame_CHW_CUDA) #no copy, but need to specify the output memory\n\nframe_CHW_pycuda[:] = (frame_CHW_pycuda - mean) / std #normalize\n```\n\nHow can `toCUDA` make it faster in your deeplearning pipeline than `opencv` or `ffmpeg`?\n> 1. The opencv/ffmpeg uses the cpu to convert video pix_fmt from original YUV to RGB24, which is slow. The ffmpegcv use the cuda to accelerate pix_fmt convertion.\n> 2. Use `yuv420p` or `nv12` can save the cpu load and reduce the memory copy from CPU to GPU.\n> 3. The ffmpeg stores the image as HWC format. The ffmpegcv can use HWC & CHW format to accelerate the video reading.\n\n## Video Writer\n---\n\n```python\n# cv2\nout = cv2.VideoWriter('outpy.avi',\n cv2.VideoWriter_fourcc('M','J','P','G'), \n 10, \n (w, h))\nout.write(frame1)\nout.write(frame2)\nout.release()\n\n# ffmpegcv, default codec is 'h264' in cpu 'h265' in gpu.\n# frameSize is decided by the size of the first frame.\n# use the 'mp4/mkv' instead of 'avi' to avoid the codec outdated.\nout = ffmpegcv.VideoWriter('outpy.mp4', None, 10)\nout.write(frame1)\nout.write(frame2)\nout.release()\n\n# more pythonic\nwith ffmpegcv.VideoWriter('outpy.mp4', None, 10) as out:\n out.write(frame1)\n out.write(frame2)\n```\n\n\nUse GPU to accelerate encoding. Such as h264_nvenc, hevc_nvenc.\n```python\nout_cpu = ffmpegcv.VideoWriter('outpy.mp4', None, 10)\nout_gpu0 = ffmpegcv.VideoWriterNV('outpy.mp4', 'h264', 10) #NVIDIA GPU0\nout_gpu1 = ffmpegcv.VideoWriterNV('outpy.mp4', 'hevc', 10, gpu=1) #NVIDIA GPU1\nout_qsv = ffmpegcv.VideoWriterQSV('outpy.mp4', 'h264', 10) #Intel QSV, experimental\n\n```\n\nInput image is rgb24 instead of bgr24\n```python\nout = ffmpegcv.VideoWriter('outpy.mp4', None, 10, pix_fmt='rgb24')\n```\n\nResize the video\n```python\nout_resz = ffmpegcv.VideoWriter('outpy.mp4', None, 10, resize=(640, 480)) #Resize\n```\n## Video Reader and Writer\n---\n```python\nimport ffmpegcv\nvfile_in = 'A.mp4'\nvfile_out = 'A_h264.mp4'\nvidin = ffmpegcv.VideoCapture(vfile_in)\nvidout = ffmpegcv.VideoWriter(vfile_out, None, vidin.fps)\n\nwith vidin, vidout:\n for frame in vidin:\n vidout.write(frame)\n```\n\n## Camera Reader\n---\n**Experimental feature**. The ffmpegcv offers Camera reader. Which is consistent with VideoFiler reader. \n\n- The `VideoCaptureCAM` aims to support ROI operations. The Opencv will be general fascinating than ffmpegcv in camera read. **I recommand the opencv in most camera reading case**.\n- The ffmpegcv can use name to retrieve the camera device. Use `ffmpegcv.VideoCaptureCAM(\"Integrated Camera\")` is readable than `cv2.VideoCaptureCAM(0)`.\n- The `VideoCaptureCAM` will be laggy and dropping frames if your post-process takes long time. The VideoCaptureCAM will buffer the recent frames.\n- The `VideoCaptureCAM` is continously working on background even if you didn't read it. **Please release it in time**.\n- Works perfect in Windows, not-perfect in Linux and macOS.\n\n```python\nimport cv2\ncap = cv2.VideoCapture(0)\nwhile True:\n ret, frame = cap.read()\n cv2.imshow('frame', frame)\n if cv2.waitKey(1) & 0xFF == ord('q'):\n break\ncap.release()\n\n# ffmpegcv, in Windows&Linux\nimport ffmpegcv\ncap = ffmpegcv.VideoCaptureCAM(0)\nwhile True:\n ret, frame = cap.read()\n cv2.imshow('frame', frame)\n if cv2.waitKey(1) & 0xFF == ord('q'):\n break\ncap.release()\n\n# ffmpegcv use by camera name, in Windows&Linux\ncap = ffmpegcv.VideoCaptureCAM(\"Integrated Camera\")\n\n# ffmpegcv use camera path if multiple cameras conflict\ncap = ffmpegcv.VideoCaptureCAM('@device_pnp_\\\\\\\\?\\\\usb#vid_2304&'\n 'pid_oot#media#0001#{65e8773d-8f56-11d0-a3b9-00a0c9223196}'\n '\\\\global')\n\n# ffmpegcv use camera with ROI operations\ncap = ffmpegcv.VideoCaptureCAM(\"Integrated Camera\", crop_xywh=(0, 0, 640, 480), resize=(512, 512), resize_keepratio=True)\n\n\n```\n\n**List all camera devices**\n```python\nfrom ffmpegcv.ffmpeg_reader_camera import query_camera_devices\n\ndevices = query_camera_devices()\nprint(devices)\n```\n>{0: ('Integrated Camera', '@device_pnp_\\\\\\\\?\\\\usb#vid_2304&pid_oot#media#0001#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\\\\global'), \n1: ('OBS Virtual Camera', '@device_sw_{860BB310-5D01-11D0-BD3B-00A0C911CE86}\\\\{A3FCE0F5-3493-419F-958A-ABA1250EC20B}')}\n\n\n**Set the camera resolution, fps, vcodec/pixel-format**\n\n```python\nfrom ffmpegcv.ffmpeg_reader_camera import query_camera_options\n\noptions = query_camera_options(0) # or query_camera_options(\"Integrated Camera\") \nprint(options)\ncap = ffmpegcv.VideoCaptureCAM(0, **options[-1])\n```\n>[{'camcodec': 'mjpeg', 'campix_fmt': None, 'camsize_wh': (1280, 720), 'camfps': 60.0002}, {'camcodec': 'mjpeg', 'campix_fmt': None, 'camsize_wh': (640, 480), 'camfps': 60.0002}, {'camcodec': 'mjpeg', 'campix_fmt': None, 'camsize_wh': (1920, 1080), 'camfps': 60.0002}, {'camcodec': None, 'campix_fmt': 'yuyv422', 'camsize_wh': (1280, 720), 'camfps': 10}, {'camcodec': None, 'campix_fmt': 'yuyv422', 'camsize_wh': (640, 480), 'camfps': 30}, {'camcodec': None, 'campix_fmt': 'yuyv422', 'camsize_wh': (1920, 1080), 'camfps': 5}]\n\n**Known issues**\n1. The VideoCaptureCAM didn't give a smooth experience in macOS. You must specify all the camera parameters. And the query_camera_options woun't give any suggestion. That's because the `ffmpeg` cannot list device options using mac native `avfoundation`. \n```python\n# The macOS requires full argument.\ncap = ffmpegcv.VideoCaptureCAM('FaceTime HD Camera', camsize_wh=(1280,720), camfps=30, campix_fmt='nv12')\n```\n\n2. The VideoCaptureCAM cann't list the FPS in linux. Because the `ffmpeg` cound't query the device's FPS using linux native `v4l2` module. However, it's just OK to let the FPS empty.\n\n\n## Stream Reader (Live streaming, RTSP IP cameras)\n**Experimental feature**. The ffmpegcv offers Stream reader. Which is consistent with VideoFiler reader, and more similiar to the camera.\nBecareful when using it.\n\n- Support `RTSP`, `RTP`, `RTMP`, `HTTP`, `HTTPS` streams.\n- The `VideoCaptureStream` will be laggy and dropping frames if your post-process takes long time. The VideoCaptureCAM will buffer the recent frames.\n- The `VideoCaptureStreamRT` is continously working on background even if you didn't read it. **Please release it in time**.\n- **Low latency** RTSP IP camera reader. Batter than opencv.\n- *It's still experimental*. Recommand you to use opencv.\n\n\n```python\n# opencv\nimport cv2\nstream_url = 'http://devimages.apple.com.edgekey.net/streaming/examples/bipbop_4x3/gear2/prog_index.m3u8'\ncap = cv2.VideoCapture(stream_url, cv2.CAP_FFMPEG)\n\nif not cap.isOpened():\n print('Cannot open the stream')\n exit(-1)\n\nwhile True:\n ret, frame = cap.read()\n if not ret:\n break\n pass\n\n# ffmpegcv\nimport ffmpegcv\ncap = ffmpegcv.VideoCaptureStream(stream_url)\nwhile True:\n ret, frame = cap.read()\n if not ret:\n break\n pass\n\n# ffmpegcv, IP Camera Low-latency\n# e.g. HIK Vision IP Camera, `101` Main camera stream, `102` the second\nstream_url = 'rtsp://admin:PASSWD@192.168.1.xxx:8554/Streaming/Channels/102'\ncap = ffmpegcv.VideoCaptureStreamRT(stream_url) # Low latency & recent buffered\ncap = ffmpegcv.ReadLiveLast(ffmpegcv.VideoCaptureStreamRT, stream_url) #no buffer\nwhile True:\n ret, frame = cap.read()\n if not ret:\n break\n pass\n```\n\n## Noblock\nA proxy to automatic prepare frames in backgroud, which does not block when reading&writing current frame (multiprocessing). This make your python program more efficient in CPU usage. Up to 2x boost.\n\n> ffmpegcv.VideoCapture(*args) -> ffmpegcv.noblock(ffmpegcv.VideoCapture, *args)\n> \n> ffmpegcv.VideoWriter(*args) -> ffmpegcv.noblock(ffmpegcv.VideoWriter, *args)\n```python\n#Proxy any VideoCapture&VideoWriter args and kargs\nvid_noblock = ffmpegcv.noblock(ffmpegcv.VideoCapture, vfile, pix_fmt='rbg24')\n\n# this is fast\ndef cpu_tense(): time.sleep(0.01)\nfor _ in tqdm.trange(1000):\n ret, img = vid_noblock.read() #current img is already buffered, take no time\n cpu_tense() #meanwhile, the next img is buffering in background\n\n# this is slow\nvid = ffmpegcv.VideoCapture(vfile, pix_fmt='rbg24')\nfor _ in tqdm.trange(1000):\n ret, img = vid.read() #this read will block cpu, take time\n cpu_tense()\n```"
},
{
"path": "README_CN.md",
"content": "# FFMPEGCV 读写视频，替代 OPENCV.\n\n[](https://pypi.org/project/ffmpegcv/)\n[](https://pypistats.org/packages/ffmpegcv)\n\n\n\n[English Version](./README.md) | 中文版本\n\nffmpegcv提供了基于ffmpeg的视频读取器和视频编写器，比cv2更快和更强大。适合深度学习的视频处理。\n\n- ffmpegcv与open-cv具有**兼容**的API。\n- ffmpegcv可以使用**GPU加速**编码和解码。\n- ffmpegcv支持比open-cv更多的**视频编码器**。\n- ffmpegcv原生支持**RGB**/BGR/灰度像素格式。\n- ffmpegcv支持网络**流视频读取** (网线监控相机)。\n- ffmpegcv支持ROI（感兴趣区域）操作，可以对ROI进行**裁剪**、**调整大小**和**填充**。\n总之，ffmpegcv与opencv的API非常相似。但它具有更多的编码器，并且不需要安装opencv。\n- ffmpegcv支持导出图像帧到CUDA设备。\n\n\n\n
\n\n## 功能：\n- `VideoWriter`：写入视频文件。\n- `VideoCapture`：读取视频文件。\n- `VideoCaptureNV`：使用NVIDIA GPU读取视频文件。\n- `VideoCaptureQSV`: 使用Intel集成显卡读取视频文件.\n- `VideoCaptureCAM`：读取摄像头。\n- `VideoCaptureStream`：读取RTP/RTSP/RTMP/HTTP流。\n- `VideoCaptureStreamRT`: 读取RTSP流 (网线监控相机)，实时、低延迟。\n- `noblock`：在后台读取视频文件（更快）,使用多进程。\n- `toCUDA`：将图像帧导出到CUDA设备，以 CHW/HWC-float32 格式存储，超过2倍性能提升。\n\n## 安装\n在使用ffmpegcv之前，您需要下载`ffmpeg`。\n```\n #1A. LINUX: sudo apt install ffmpeg\n #1B. MAC: brew install ffmpeg\n #1C. WINDOWS: 下载ffmpeg并添加至环境变量的路径中\n #1D. CONDA: conda install ffmpeg=6.0.0\n \n #2. python\n pip install ffmpegcv #stable verison\n pip install git+https://github.com/chenxinfeng4/ffmpegcv #latest verison\n ```\n\n## 何时选择 `ffmpegcv` 而不是 `opencv`：\n- 安装`opencv`比较困难。ffmpegcv仅需要`numpy`和`FFmpeg`，可以在Mac/Windows/Linux平台上工作。\n- `opencv`包含太多的图像处理工具箱，而您只是想使用带GPU支持的简单视频/摄像头输入输出操作。\n- `opencv`不支持`h264`/`h265`和其他视频编码器。\n- 您想对视频/摄像头的感兴趣区域（ROI）进行**裁剪**、**调整大小**和**填充**操作。\n\n\n## 基本示例\n通过CPU读取视频，并通过GPU重写视频。\n```python\nvidin = ffmpegcv.VideoCapture(vfile_in)\nvidout = ffmpegcv.VideoWriterNV(vfile_out, 'h264', vidin.fps) #NVIDIA 显卡\n\nwith vidin, vidout:\n for frame in vidin:\n cv2.imshow('image', frame)\n vidout.write(frame)\n```\n\n读取摄像头。\n```python\n# 通过设备ID\ncap = ffmpegcv.VideoCaptureCAM(0)\n# 通过设备名称\ncap = ffmpegcv.VideoCaptureCAM(\"Integrated Camera\")\n```\n\n深度学习流水线\n```python\n\"\"\"\n —————————— NVIDIA GPU 加速 ⤴⤴ ———————\n | |\n V V\n视频 -> 解码器 -> 裁剪 -> 缩放 -> RGB -> CUDA:CHW float32 -> 模型\n\"\"\"\ncap = ffmpegcv.toCUDA(\n ffmpegcv.VideoCaptureNV(file, pix_fmt='nv12', resize=(W,H)),\n tensor_format='chw')\n\nfor frame_CHW_cuda in cap:\n frame_CHW_cuda = (frame_CHW_cuda - mean) / std\n result = model(frame_CHW_cuda)\n```\n\n## GPU加速\n- 仅支持NVIDIA显卡，在 x86_64 上测试。\n- 原生支持**Windows**, **Linux**, **Anaconda**。\n- 在**Google Colab**上顺利运行。\n- 在**MacOS**仅能使用CPU功能，上无法进行GPU加速，因为Mac根本就不支持NVIDIA。\n\n> 在CPU数量充足的条件下，GPU读取速度可能比CPU读取速度稍慢。在使用感兴趣区域（ROI）操作（裁剪、调整大小、填充）时，GPU优势更凸显。\n\n## 编解码器\n\n| 编解码器 | OpenCV读取器 | ffmpegcv-CPU读取器 | GPU读取器 | OpenCV写入器 | ffmpegcv-CPU写入器 | GPU写入器 |\n| ----------- | ------------- | ---------------- | ---- | ------------- | ---------------- | ---- |\n| h264 | √ | √ | √ | × | √ | √ |\n| h265 (hevc) | 不确定 | √ | √ | × | √ | √ |\n| mjpeg | √ | √ | × | √ | √ | × |\n| mpeg | √ | √ | × | √ | √ | × |\n| 其他 | 不确定 | ffmpeg -decoders | × | 不确定 | ffmpeg -encoders | × |\n\n## 基准测试\n*正在进行中...(遥遥无期)*\n\n\n## 视频读取器\n---\nffmpegcv与opencv在API上非常类似。\n```python\n# OpenCV\nimport cv2\ncap = cv2.VideoCapture(file)\nwhile True:\n ret, frame = cap.read()\n if not ret:\n break\n pass\n\n# ffmpegcv\nimport ffmpegcv\ncap = ffmpegcv.VideoCapture(file)\nwhile True:\n ret, frame = cap.read()\n if not ret:\n break\n pass\ncap.release()\n\n# 另一种写法\ncap = ffmpegcv.VideoCapture(file)\nnframe = len(cap)\nfor frame in cap:\n pass\ncap.release()\n\n# 更加Pythonic的写法，推荐使用\nwith ffmpegcv.VideoCapture(file) as cap:\n nframe = len(cap)\n for iframe, frame in enumerate(cap):\n if iframe>100: break\n pass\n```\n\n使用GPU加速解码。具体取决于视频编码格式。\nh264_nvcuvid, hevc_nvcuvid ....\n```python\ncap_cpu = ffmpegcv.VideoCapture(file)\ncap_gpu = ffmpegcv.VideoCapture(file, codec='h264_cuvid') # NVIDIA GPU0\ncap_gpu0 = ffmpegcv.VideoCaptureNV(file) # NVIDIA GPU0\ncap_gpu1 = ffmpegcv.VideoCaptureNV(file, gpu=1) # NVIDIA GPU1\ncap_qsv = ffmpegcv.VideoCaptureQSV(file) #Intel QSV, 测试中\n```\n\n使用`rgb24`代替`bgr24`。`gray`版本会更高效。\n```python\ncap = ffmpegcv.VideoCapture(file, pix_fmt='rgb24') # rgb24, bgr24, gray\nret, frame = cap.read()\nplt.imshow(frame)\n```\n\n### 感兴趣区域（ROI）操作\n您可以对视频进行裁剪、调整大小和填充。这些ROI操作中，`ffmpegcv-GPU` > `ffmpegcv-CPU` >> `opencv` 在性能上。\n\n**裁剪**视频，比读取整个画布要快得多。\n```python\ncap = ffmpegcv.VideoCapture(file, crop_xywh=(0, 0, 640, 480))\n```\n\n将视频调整为给定大小的**大小**。\n```python\ncap = ffmpegcv.VideoCapture(file, resize=(640, 480))\n```\n\n**调整大小**并保持宽高比，使用黑色边框进行**填充**。\n```python\ncap = ffmpegcv.VideoCapture(file, resize=(640, 480), resize_keepratio=True)\n```\n\n对视频进行**裁剪**，然后进行**调整大小**。左上角为坐标 (0,0)。\n```python\ncap = ffmpegcv.VideoCapture(file, crop_xywh=(0, 0, 640, 480), resize=(512, 512))\n```\n### 扩展选项\n**INFILE_OPTIONS**: 扩展 ffmpeg 输入选项。 \n\n```python\ncap = ffmpegcv.VideoCapture(file, infile_options='-re -stream_loop -1')\n\n# 等效于下面的 ffmpeg 命令\nffmpeg INFILE_OPTIONS -i FILE -f rawvideo pipe:\n```\n\n## toCUDA 将图像帧快速导出到CUDA设备\n---\nffmpegcv 可以将 HWC-uint8 cpu 中的视频/流转换为 CUDA 设备中的 CHW-float32。它可以显著减少你的 CPU 负载，并比你的手动转换快 2 倍以上。\n\n准备环境。你需要具备 cuda 环境，并且安装 pycuda 包。注意，pytorch 包是非必须的。 \n> nvcc --version # 检查你是否已经安装了 NVIDIA CUDA 编译器\n> pip install pycuda # 安装 pycuda\n\n```python\n# 读取视频到CUDA设备，加速前\ncap = ffmpegcv.VideoCaptureNV(file, pix_fmt='rgb24')\nret, frame_HWC_CPU = cap.read()\nframe_CHW_CUDA = torch.from_numpy(frame_HWC_CPU).permute(2, 0, 1).cuda().contiguous().float() # 120fps, 1200% CPU 使用率\n\n# 加速后\ncap = toCUDA(ffmpegcv.VideoCapture(file, pix_fmt='yuv420p')) #必须设置, yuv420p 针对 cpu\ncap = toCUDA(ffmpegcv.VideoCaptureNV(file, pix_fmt='nv12')) #必须设置, nv12 针对 gpu\ncap = toCUDA(vid, tensor_format='chw') #tensor 格式:'chw'(默认) or 'hwc'\ncap = toCUDA(vid, gpu=1) #选择 gpu\n\nret, frame_CHW_pycuda = cap.read() #380fps, 200% CPU load, [pycuda array]\nret, frame_CHW_pycudamem = cap.read_cudamem() #same as [pycuda mem_alloc]\nret, frame_CHW_CUDA = cap.read_torch() #same as [pytorch tensor]\nret, _ = cap.read_torch(frame_CHW_CUDA) #不拷贝, 但需要提前分配内存\n\nframe_CHW_pycuda[:] = (frame_CHW_pycuda - mean) / std #归一化\n```\n\n为什么在深度学习流水线中使用 toCUDA 会更快？\n\n> 1. ffmpeg 使用 CPU 将视频像素格式从原始 YUV 转换为 RGB24，这个过程很慢。`toCUDA` 使用 cuda 加速像素格式转换。\n> 2. 使用 yuv420p 或 nv12 可以节省 CPU 负载并减少从 CPU 到 GPU 的内存复制。\n> 3. ffmpeg 将图像存储为 HWC 格式。ffmpegcv 可以使用 HWC 和 CHW 格式来加速视频存储。\n\n## 视频写入器\n---\n```python\n# cv2\nout = cv2.VideoWriter('outpy.avi',\n cv2.VideoWriter_fourcc('M','J','P','G'), \n 10, \n (w, h))\nout.write(frame1)\nout.write(frame2)\nout.release()\n\n# ffmpegcv，默认的编码器为'h264'在CPU上，'h265'在GPU上。\n# 帧大小由第一帧决定\n# 使用 'mp4/mkv' 来替代古老的 'avi' 格式\nout = ffmpegcv.VideoWriter('outpy.mp4', None, 10)\nout.write(frame1)\nout.write(frame2)\nout.release()\n\n# 更加Pythonic的写法\nwith ffmpegcv.VideoWriter('outpy.mp4', None, 10) as out:\n out.write(frame1)\n out.write(frame2)\n```\n\n使用GPU加速编码。例如h264_nvenc，hevc_nvenc。\n```python\nout_cpu = ffmpegcv.VideoWriter('outpy.mp4', None, 10)\nout_gpu0 = ffmpegcv.VideoWriterNV('outpy.mp4', 'h264', 10) # NVIDIA GPU0\nout_gpu1 = ffmpegcv.VideoWriterNV('outpy.mp4', 'hevc', 10, gpu=1) # NVIDIA GPU1\nout_qsv = ffmpegcv.VideoWriterQSV('outpy.mp4', 'h264', 10) #Intel QSV, 测试中\n```\n\n输入图像使用rgb24而不是bgr24。\n```python\nout = ffmpegcv.VideoWriter('outpy.mp4', None, 10, pix_fmt='rgb24')\n```\n\n缩放图像尺寸\n```python\nout_resz = ffmpegcv.VideoWriter('outpy.mp4', None, 10, resize=(640, 480)) \n```\n\n## 视频读取器和写入器\n---\n```python\nimport ffmpegcv\nvfile_in = 'A.mp4'\nvfile_out = 'A_h264.mp4'\nvidin = ffmpegcv.VideoCapture(vfile_in)\nvidout = ffmpegcv.VideoWriter(vfile_out, None, vidin.fps)\n\nwith vidin, vidout:\n for frame in vidin:\n vidout.write(frame)\n```\n\n## 相机读取器\n---\n**实验性功能**。ffmpegcv提供了相机读取器。与VideoCapture读取器一致。\n\n- VideoCaptureCAM旨在支持感兴趣区域（ROI）操作。在相机读取方面，Opencv比ffmpegcv更具吸引力。**对于大多数相机读取情况，我推荐使用Opencv**。\n- ffmpegcv可以使用名称检索相机设备，使用`ffmpegcv.VideoCaptureCAM(\"Integrated Camera\")`比使用`cv2.VideoCaptureCAM(0)`更易读。\n- 如果后处理时间过长，VideoCaptureCAM将会出现卡顿和丢帧。VideoCaptureCAM会缓冲最近的帧。\n- 即使没有读取视频帧，VideoCaptureCAM也会在后台不断工作。**请及时释放资源**。\n- 在Windows上表现良好，在Linux和macOS上表现不完美。\n\n```python\nimport cv2\ncap = cv2.VideoCapture(0)\nwhile True:\n ret, frame = cap.read()\n cv2.imshow('frame', frame)\n if cv2.waitKey(1) & 0xFF == ord('q'):\n break\ncap.release()\n\n# ffmpegcv，在Windows和Linux上\nimport ffmpegcv\ncap = ffmpegcv.VideoCaptureCAM(0)\nwhile True:\n ret, frame = cap.read()\n cv2.imshow('frame', frame)\n if cv2.waitKey(1) & 0xFF == ord('q'):\n break\ncap.release()\n\n# ffmpegcv 使用相机名称，在Windows和Linux上\ncap = ffmpegcv.VideoCaptureCAM(\"Integrated Camera\")\n\n# ffmpegcv 使用相机路径（避免多个相机冲突）\ncap = ffmpegcv.VideoCaptureCAM('@device_pnp_\\\\\\\\?\\\\usb#vid_2304&'\n 'pid_oot#media#0001#{65e8773d-8f56-11d0-a3b9-00a0c9223196}'\n '\\\\global')\n\n# ffmpegcv 使用具有ROI操作的相机\ncap = ffmpegcv.VideoCaptureCAM(\"Integrated Camera\", crop_xywh=(0, 0, 640, 480), resize=(512, 512), resize_keepratio=True)\n\n\n```\n\n**列出所有相机设备**\n```python\nfrom ffmpegcv.ffmpeg_reader_camera import query_camera_devices\n\ndevices = query_camera_devices()\nprint(devices)\n```\n>{0: ('Integrated Camera', '@device_pnp_\\\\\\\\?\\\\usb#vid_2304&pid_oot#media#0001#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\\\\global'), \n1: ('OBS Virtual Camera', '@device_sw_{860BB310-5D01-11D0-BD3B-00A0C911CE86}\\\\{A3FCE0F5-3493-419F-958A-ABA1250EC20B}')}\n\n\n**设置相机的分辨率、帧率、视频编码/像素格式**\n\n```python\nfrom ffmpegcv.ffmpeg_reader_camera import query_camera_options\n\noptions = query_camera_options(0) # 或者 query_camera_options(\"Integrated Camera\") \nprint(options)\ncap = ffmpegcv.VideoCaptureCAM(0, **options[-1])\n```\n>[{'camcodec': 'mjpeg', 'campix_fmt': None, 'camsize_wh': (1280, 720), 'camfps': 60.0002}, {'camcodec': 'mjpeg', 'campix_fmt': None, 'camsize_wh': (640, 480), 'camfps': 60.0002}, {'camcodec': 'mjpeg', 'campix_fmt': None, 'camsize_wh': (1920, 1080), 'camfps': 60.0002}, {'camcodec': None, 'campix_fmt': 'yuyv422', 'camsize_wh': (1280, 720), 'camfps': 10}, {'camcodec': None, 'campix_fmt': 'yuyv422', 'camsize_wh': (640, 480), 'camfps': 30}, {'camcodec': None, 'campix_fmt': 'yuyv422', 'camsize_wh': (1920, 1080), 'camfps': 5}]\n\n**已知问题**\n1. VideoCaptureCAM在macOS上的体验不太流畅。你必须指定所有相机参数。而且query_camera_options不会给出任何建议。这是因为`ffmpeg`无法使用mac本机的`avfoundation`列出设备选项。\n```python\n# macOS需要提供完整参数。\ncap = ffmpegcv.VideoCaptureCAM('FaceTime HD Camera', camsize_wh=(1280,720), camfps=30, campix_fmt='nv12')\n```\n\n2. 在Linux上VideoCaptureCAM无法列出FPS，因为`ffmpeg`无法使用Linux本机的`v4l2`模块查询设备的FPS。不过，让FPS为空也没问题。\n\n## 流读取器 （直播流，网络监控摄像头）\n**实验性功能**。ffmpegcv提供了流读取器，与VideoFile读取器一致，更类似于相机。\n\n- 支持`RTSP`、`RTP`、`RTMP`、`HTTP`、`HTTPS`流。\n- 如果后处理时间过长，VideoCaptureStream会出现卡顿和丢帧。VideoCaptureCAM会缓冲最近的帧。\n- 即使没有读取视频帧，VideoCaptureStream也会在后台不断工作。**请及时释放资源**。\n- 这仍然是实验性功能。建议您使用opencv。\n\n```python\n# opencv\nimport cv2\nstream_url = 'http://devimages.apple.com.edgekey.net/streaming/examples/bipbop_4x3/gear2/prog_index.m3u8'\ncap = cv2.VideoCapture(stream_url, cv2.CAP_FFMPEG)\n\nif not cap.isOpened():\n print('无法打开流')\n exit(-1)\n\nwhile True:\n ret, frame = cap.read()\n if not ret:\n break\n pass\n\n# ffmpegcv\nimport ffmpegcv\ncap = ffmpegcv.VideoCaptureStream(stream_url)\nwhile True:\n ret, frame = cap.read()\n if not ret:\n break\n pass\n\n# ffmpegcv, 网络监控摄像头\n# 例如 海康威视, `101` 主视频流, `102` 子视频流\nstream_url = 'rtsp://admin:PASSWD@192.168.1.xxx:8554/Streaming/Channels/102'\ncap = ffmpegcv.VideoCaptureStreamRT(stream_url) # 低延迟 & 缓存\ncap = ffmpegcv.ReadLiveLast(ffmpegcv.VideoCaptureStreamRT, stream_url) #不缓存\nwhile True:\n ret, frame = cap.read()\n if not ret:\n break\n pass\n```\n\n## FFmpegReaderNoblock\n更快的读写取视频。利用多进程在后台自动准备帧，这样在读写当前帧时不会阻塞。这使得您的Python程序在CPU使用方面更高效。带来最大翻倍效率提升。\n\n> ffmpegcv.VideoCapture(*args) -> ffmpegcv.noblock(ffmpegcv.VideoCapture, *args)\n>\n> ffmpegcv.VideoWriter(*args) -> ffmpegcv.noblock(ffmpegcv.VideoWriter, *args)\n\n```python\n# 代理任何 VideoCapture&VideoWriter 的参数和kargs\nvid_noblock = ffmpegcv.noblock(ffmpegcv.VideoCapture, vfile, pix_fmt='rbg24')\n\n# 这很快\ndef cpu_tense(): time.sleep(0.01)\nfor _ in tqdm.trange(1000):\n ret, img = vid_noblock.read() #当前图像已经被缓冲，不会占用时间\n cpu_tense() #同时，下一帧在后台缓冲\n\n# 这很慢\nvid = ffmpegcv.VideoCapture(vfile, pix_fmt='rbg24')\nfor _ in tqdm.trange(2000):\n ret, img = vid.read() #此读取将阻塞CPU，占用时间\n cpu_tense()\n```\n"
},
{
"path": "ffmpegcv/__init__.py",
"content": "from .ffmpeg_reader import FFmpegReader, FFmpegReaderNV\nfrom .ffmpeg_writer import FFmpegWriter, FFmpegWriterNV\nfrom .ffmpeg_reader_camera import FFmpegReaderCAM\nfrom .ffmpeg_reader_stream import FFmpegReaderStream\nfrom .ffmpeg_reader_stream_realtime import FFmpegReaderStreamRT, FFmpegReaderStreamRTNV\nfrom .ffmpeg_writer_stream_realtime import FFmpegWriterStreamRT\nfrom .ffmpeg_reader_qsv import FFmpegReaderQSV\nfrom .ffmpeg_writer_qsv import FFmpegWriterQSV\nfrom .ffmpeg_reader_pannels import FFmpegReaderPannels\nfrom .ffmpeg_noblock import noblock, ReadLiveLast\nfrom .video_info import get_num_NVIDIA_GPUs\nimport shutil\nfrom subprocess import DEVNULL, check_output\n\nfrom .version import __version__\n\n\ndef _check():\n if not shutil.which(\"ffmpeg\") or not shutil.which(\"ffprobe\"):\n raise RuntimeError(\n \"The ffmpeg is not installed. \\n\\n\"\n \"Please install ffmpeg via:\\n \"\n \"conda install ffmpeg\"\n )\n\n\n_check()\n\n_check_nvidia_init = None\n\n\ndef _check_nvidia():\n global _check_nvidia_init\n run = lambda x: check_output(x, shell=True, stderr=DEVNULL)\n if _check_nvidia_init is None:\n calling_output = run(\"ffmpeg -h encoder=hevc_nvenc\")\n if \"AVOptions\" not in calling_output.decode(\"utf-8\"):\n raise RuntimeError(\n \"The ffmpeg is not compiled with NVENC support.\\n\\n\"\n \"Please re-compile ffmpeg following the instructions at:\\n \"\n \"https://docs.nvidia.com/video-technologies/video-codec-sdk/ffmpeg-with-nvidia-gpu/\"\n )\n\n calling_output = run(\"ffmpeg -h decoder=hevc_cuvid\")\n if \"AVOptions\" not in calling_output.decode(\"utf-8\"):\n raise RuntimeError(\n \"The ffmpeg is not compiled with NVENC support.\\n\\n\"\n \"Please re-compile ffmpeg following the instructions at:\\n \"\n \"https://docs.nvidia.com/video-technologies/video-codec-sdk/ffmpeg-with-nvidia-gpu/\"\n )\n\n if get_num_NVIDIA_GPUs() == 0:\n raise RuntimeError(\n \"No NVIDIA GPU found.\\n\\n\"\n \"Please use a NVIDIA GPU card listed at:\\n \"\n \"https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new\"\n )\n\n _check_nvidia_init = True\n\n return True\n\n\ndef VideoCapture(\n file,\n codec=None,\n pix_fmt=\"bgr24\",\n crop_xywh=None,\n resize=None,\n resize_keepratio=True,\n resize_keepratioalign=\"center\",\n infile_options=None\n) -> FFmpegReader:\n \"\"\"\n Alternative to cv2.VideoCapture\n\n Parameters\n ----------\n file : str\n Path to video file.\n codec : str\n Codec to use. Optional. Default is `None`.\n pix_fmt : str\n Pixel format. ['bgr24' | 'rgb24']. Optional. Default is 'bgr24'.\n crop_xywh : tuple\n Crop the frame. (x, y, width, height). Optional. Default is `None`.\n resize : tuple\n Resize the video to the given size. Optional. Default is `None`.\n resize_keepratio : bool\n Keep the aspect ratio and the border is black. Optional. Default is `True`.\n resize_keepratioalign : str\n Align the image to the `center`, `topleft`, `topright`, `bottomleft` or\n `bottomright`. Optional. Default is 'center'.\n infile_options : str\n Additional options for ffmpeg. Optional. Default is `None`.\n\n Examples\n --------\n opencv\n ```\n cap = cv2.VideoCapture(file)\n while True:\n ret, frame = cap.read()\n if not ret:\n break\n pass\n ```\n\n ffmpegcv\n ```\n cap = ffmpegcv.VideoCapture(file)\n while True:\n ret, frame = cap.read()\n if not ret:\n break\n pass\n ```\n\n Or use iterator\n ```\n cap = ffmpegcv.VideoCapture(file)\n for frame in cap:\n pass\n counts = len(cap)\n ```\n\n Use GPU to accelerate decoding\n ```\n cap_cpu = ffmpegcv.VideoCapture(file)\n cap_gpu = ffmpegcv.VideoCaptureNV(file)\n ```\n\n Use rgb24 instead of bgr24\n ```\n cap = ffmpegcv.VideoCapture(file, pix_fmt='rgb24')\n ```\n\n Crop video.\n ```python\n cap = ffmpegcv.VideoCapture(file, crop_xywh=(0, 0, 640, 480))\n ```\n\n Resize the video to the given size\n ```\n cap = ffmpegcv.VideoCapture(file, resize=(640, 480))\n ```\n\n Resize and keep the aspect ratio with black border\n ```\n cap = ffmpegcv.VideoCapture(file, resize=(640, 480), resize_keepratio=True)\n ```\n\n Crop and then resize the video.\n ```python\n cap = ffmpegcv.VideoCapture(file, crop_xywh=(0, 0, 640, 480), resize=(512, 512))\n ```\n Author: Chenxinfeng 2022-04-16, cxf529125853@163.com\n \"\"\"\n return FFmpegReader.VideoReader(\n file, codec, pix_fmt, crop_xywh, resize, resize_keepratio, resize_keepratioalign, infile_options\n )\n\n\nVideoReader = VideoCapture\n\n\ndef VideoWriter(\n file, codec=None, fps=30, pix_fmt=\"bgr24\", bitrate=None, resize=None, preset=None\n) -> FFmpegWriter:\n \"\"\"\n Alternative to cv2.VideoWriter\n\n Parameters\n ----------\n file : str\n Path to video file.\n codec : str\n Codec to use. Optional. Default is `None` (x264).\n fps : number\n Frames per second. Optional. Default is 30.\n pix_fmt : str\n Pixel format of input. ['bgr24' | 'rgb24']. Optional. Default is 'bgr24'.\n bitrate : str\n Bitrate of output video. Optional. Default is `None`.\n resize : tuple\n Frame size of output. (width, height). Optional. Default is `None`.\n preset : str\n Preset of ffmpeg. Optional. Default is `None`.\n Examples\n --------\n opencv\n ```\n out = cv2.VideoWriter('outpy.avi',\n cv2.VideoWriter_fourcc('M','J','P','G'),\n 10,\n (w, h))\n out.write(frame1)\n out.write(frame2)\n out.release()\n ```\n\n ffmpegcv\n ```\n out = ffmpegcv.VideoWriter('outpy.avi', None, 10)\n out.write(frame1)\n out.write(frame2)\n out.release()\n ```\n\n frameSize is decided by the size of the first frame\n ```\n out = ffmpegcv.VideoWriter('outpy.avi', None, 10)\n ```\n\n Use GPU to accelerate encoding\n ```\n out_cpu = ffmpegcv.VideoWriter('outpy.avi', None, 10)\n out_gpu = ffmpegcv.VideoWriter('outpy.avi', 'h264_nvenc', 10)\n ```\n\n Use rgb24 instead of bgr24\n ```\n out = ffmpegcv.VideoWriter('outpy.avi', None, 10, pix_fmt='rgb24')\n out.write(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))\n ```\n\n Author: Chenxinfeng 2022-04-16, cxf529125853@163.com\n \"\"\"\n return FFmpegWriter.VideoWriter(\n file, codec, fps, pix_fmt, bitrate, resize, preset=preset\n )\n\n\ndef VideoCaptureNV(\n file,\n pix_fmt=\"bgr24\",\n crop_xywh=None,\n resize=None,\n resize_keepratio=True,\n resize_keepratioalign=\"center\",\n infile_options=None,\n gpu=0\n) -> FFmpegReaderNV:\n \"\"\"\n `ffmpegcv.VideoCaptureNV` is a gpu version for `ffmpegcv.VideoCapture`.\n \"\"\"\n _check_nvidia()\n return FFmpegReaderNV.VideoReader(\n file, pix_fmt, crop_xywh, resize, resize_keepratio, \n resize_keepratioalign, infile_options, gpu\n )\n\n\nVideoReaderNV = VideoCaptureNV\n\n\ndef VideoCaptureQSV(\n file,\n pix_fmt=\"bgr24\",\n crop_xywh=None,\n resize=None,\n resize_keepratio=True,\n resize_keepratioalign=\"center\",\n infile_options=None,\n gpu=0\n) -> FFmpegReaderQSV:\n \"\"\"\n `ffmpegcv.VideoCaptureQSV` is a gpu version for `ffmpegcv.VideoCapture`.\n \"\"\"\n return FFmpegReaderQSV.VideoReader(\n file, pix_fmt, crop_xywh, resize, resize_keepratio, \n resize_keepratioalign, infile_options, gpu\n )\n\n\nVideoReaderQSV = VideoCaptureQSV\n\n\ndef VideoWriterNV(\n file,\n codec=None,\n fps=30,\n pix_fmt=\"bgr24\",\n gpu=0,\n bitrate=None,\n resize=None,\n preset=None,\n) -> FFmpegWriterNV:\n \"\"\"\n `ffmpegcv.VideoWriterNV` is a gpu version for `ffmpegcv.VideoWriter`.\n \"\"\"\n _check_nvidia()\n return FFmpegWriterNV.VideoWriter(\n file, codec, fps, pix_fmt, gpu, bitrate, resize, preset=preset\n )\n\n\ndef VideoWriterQSV(\n file,\n codec=None,\n fps=30,\n pix_fmt=\"bgr24\",\n gpu=0,\n bitrate=None,\n resize=None,\n preset=None,\n) -> FFmpegWriterQSV:\n \"\"\"\n `ffmpegcv.VideoWriterQSV` is a gpu version for `ffmpegcv.VideoWriter`.\n \"\"\"\n return FFmpegWriterQSV.VideoWriter(\n file, codec, fps, pix_fmt, gpu, bitrate, resize, preset=preset\n )\n\n\ndef VideoWriterStreamRT(\n url, pix_fmt=\"bgr24\", bitrate=None, resize=None, preset=None\n) -> FFmpegWriterStreamRT:\n return FFmpegWriterStreamRT.VideoWriter(\n url, \"libx264\", pix_fmt, bitrate, resize, preset\n )\n\n\ndef VideoCaptureCAM(\n camname,\n pix_fmt=\"bgr24\",\n crop_xywh=None,\n resize=None,\n resize_keepratio=True,\n resize_keepratioalign=\"center\",\n camsize_wh=None,\n camfps=None,\n camcodec=None,\n campix_fmt=None\n) -> FFmpegReaderCAM:\n \"\"\"\n Alternative to cv2.VideoCapture\n\n Parameters\n ----------\n file : see ffmpegcv.VideoReader\n codec : see ffmpegcv.VideoReader\n pix_fmt : see ffmpegcv.VideoReader\n crop_xywh : see ffmpegcv.VideoReader\n resize : see ffmpegcv.VideoReader\n resize_keepratio : see ffmpegcv.VideoReader\n resize_keepratioalign : see ffmpegcv.VideoReader\n camsize_wh: tuple or None\n Camera resolution (width, height). e.g (800, 600)\n camfps: float or None\n Camera framerate. e.g. 30.\n camcodec: str or None\n Camera codec. e.g. 'mjpeg' or 'h264'.\n campix_fmt: str or None\n Camera pixel format. e.g. 'rgb24' or 'yuv420p'.\n Just set one of `camcodec` or `campix_fmt`.\n Examples\n --------\n opencv\n ```\n cap = cv2.VideoCapture(0)\n while True:\n ret, frame = cap.read()\n if not ret:\n break\n pass\n ```\n\n ffmpegcv\n ```\n cap = ffmpegcv.VideoCaptureCAM(0)\n while True:\n ret, frame = cap.read()\n if not ret:\n break\n pass\n ```\n\n Or use camera name\n ```\n cap = ffmpegcv.VideoCaptureCAM(\"Integrated Camera\")\n ```\n\n Use full camera parameter\n ```\n cap = ffmpegcv.VideoCaptureCAM('FaceTime HD Camera',\n camsize_wh = (1280,720),\n camfps = 30,\n campix_fmt = 'nv12')\n ```\n\n Use camera with ROI operations\n ```\n cap = ffmpegcv.VideoCaptureCAM(\"Integrated Camera\",\n crop_xywh = (0, 0, 640, 480),\n resize = (512, 512),\n resize_keepratio = True)\n ```\n Author: Chenxinfeng 2023-05-11, cxf529125853@163.com\n \"\"\"\n return FFmpegReaderCAM.VideoReader(\n camname,\n pix_fmt,\n crop_xywh,\n resize,\n resize_keepratio,\n resize_keepratioalign,\n camsize_wh=camsize_wh,\n camfps=camfps,\n camcodec=camcodec,\n campix_fmt=campix_fmt\n )\n\n\nVideoReaderCAM = VideoCaptureCAM\n\n\ndef VideoCaptureStream(\n stream_url,\n codec=None,\n pix_fmt=\"bgr24\",\n crop_xywh=None,\n resize=None,\n resize_keepratio=True,\n resize_keepratioalign=\"center\",\n infile_options=None,\n timeout=None\n) -> FFmpegReaderStream:\n \"\"\"\n Alternative to cv2.VideoCapture\n\n Parameters\n ----------\n stream_url : RTSP, RTP, RTMP, HTTP, HTTPS url\n codec : see ffmpegcv.VideoReader\n pix_fmt : see ffmpegcv.VideoReader\n crop_xywh : see ffmpegcv.VideoReader\n resize : see ffmpegcv.VideoReader\n resize_keepratio : see ffmpegcv.VideoReader\n resize_keepratioalign : see ffmpegcv.VideoReader\n infile_options : see ffmpegcv.VideoReader\n timeout : waits in seconds for stream video to connect\n\n Examples\n --------\n opencv\n ```\n stream_url = 'http://devimages.apple.com.edgekey.net/streaming/examples/bipbop_4x3/gear2/prog_index.m3u8'\n cap = cv2.VideoCapture(stream_url, cv2.CAP_FFMPEG)\n\n if not cap.isOpened():\n print('Cannot open the stream')\n exit(-1)\n\n while True:\n ret, frame = cap.read()\n if not ret:\n break\n pass\n ```\n\n ffmpegcv\n ```\n cap = ffmpegcv.VideoCaptureStream(stream_url)\n while True:\n ret, frame = cap.read()\n if not ret:\n break\n pass\n ```\n\n Author: Chenxinfeng 2023-05-31, cxf529125853@163.com\n \"\"\"\n return FFmpegReaderStream.VideoReader(\n stream_url,\n codec,\n pix_fmt,\n crop_xywh,\n resize,\n resize_keepratio,\n resize_keepratioalign,\n infile_options,\n timeout\n )\n\n\nVideoReaderStream = VideoCaptureStream\n\n\ndef VideoCaptureStreamRT(\n stream_url,\n codec=None,\n pix_fmt=\"bgr24\",\n crop_xywh=None,\n resize=None,\n resize_keepratio=True,\n resize_keepratioalign=\"center\",\n infile_options=None,\n gpu=None,\n timeout=None,\n) -> FFmpegReaderStreamRT:\n if gpu is None:\n return FFmpegReaderStreamRT.VideoReader(\n stream_url,\n codec,\n pix_fmt,\n crop_xywh,\n resize,\n resize_keepratio,\n resize_keepratioalign,\n infile_options,\n timeout=timeout\n )\n else:\n return FFmpegReaderStreamRTNV.VideoReader(\n stream_url,\n codec,\n pix_fmt,\n crop_xywh,\n resize,\n resize_keepratio,\n resize_keepratioalign,\n infile_options,\n gpu=gpu,\n timeout=timeout\n )\n\n\nVideoReaderStreamRT = VideoCaptureStreamRT\n\n\ndef VideoCapturePannels(\n file: str, crop_xywh_l: list, codec=None, pix_fmt=\"bgr24\", resize=None\n):\n \"\"\"\n Alternative to cv2.VideoCapture\n\n Parameters\n ----------\n file : str\n Path to video file.\n crop_xywh_l : list of crop_xywh\n Crop the frame. [(x0, y0, w0, h0), (x1, y1, w1, h1), ...].\n codec : str\n Codec to use. Optional. Default is `None`.\n pix_fmt : str\n Pixel format. ['bgr24' | 'rgb24' | 'gray']. Optional. Default is 'bgr24'.\n resize : tuple as (w,h)\n Resize the pannels to identical size. Optional. Default is `None`.\n Does not keep the ratio.\n\n\n Examples\n --------\n\n\n ffmpegcv\n ```\n w,h = 1280,800\n cap = ffmpegcv.VideoCapturePannels(file,\n [[0,0,w,h], [w,0,w,h],[0,h,w,h], [w,h,w,h]])\n while True:\n ret, frame_pannels = cap.read()\n if not ret:\n break\n print(frame_pannels.shape) # shape=(4,h,w,3)\n ```\n\n Author: Chenxinfeng 2024-02-15, cxf529125853@163.com\n \"\"\"\n\n return FFmpegReaderPannels.VideoReader(\n file,\n crop_xywh_l,\n codec,\n pix_fmt,\n resize,\n )\n\n\nVideoReaderPannels = VideoCapturePannels\n\n\ndef toCUDA(vid: FFmpegReader, gpu: int = 0, tensor_format: str = \"chw\") -> FFmpegReader:\n \"\"\"\n Convert frames to CUDA tensor float32 in 'chw' or 'hwc' format.\n \"\"\"\n from ffmpegcv.ffmpeg_reader_cuda import FFmpegReaderCUDA\n\n return FFmpegReaderCUDA(vid, gpu, tensor_format)\n"
},
{
"path": "ffmpegcv/ffmpeg_noblock.py",
"content": "from .ffmpeg_reader_noblock import FFmpegReaderNoblock\nfrom .ffmpeg_writer_noblock import FFmpegWriterNoblock\nfrom typing import Callable\nimport ffmpegcv\nimport threading\nimport numpy as np\nimport queue\n\n\ndef noblock(fun:Callable, *v_args, **v_kargs):\n readerfuns = (ffmpegcv.VideoCapture, ffmpegcv.VideoCaptureNV)\n writerfuns = (ffmpegcv.VideoWriter, ffmpegcv.VideoWriterNV)\n\n if fun in readerfuns:\n proxyfun = FFmpegReaderNoblock(fun, *v_args, **v_kargs)\n elif fun in writerfuns:\n proxyfun = FFmpegWriterNoblock(fun, *v_args, **v_kargs)\n else:\n raise ValueError('The function is not supported as a Reader or Writer')\n \n return proxyfun\n\n\nclass ReadLiveLast(threading.Thread, ffmpegcv.FFmpegReader):\n def __init__(self, fun, *args, **kvargs):\n threading.Thread.__init__(self)\n ffmpegcv.FFmpegReader.__init__(self)\n self.vid = vid = fun(*args, **kvargs)\n props_name = ['width', 'height', 'fps', 'count', 'codec', 'ffmpeg_cmd',\n 'size', 'pix_fmt', 'out_numpy_shape', 'iframe', \n 'duration', 'origin_width', 'origin_height']\n for name in props_name:\n setattr(self, name, getattr(vid, name, None))\n\n self.img = np.zeros(self.out_numpy_shape, dtype=np.uint8)\n self.ret = True\n self._isopen = True\n self._q = queue.Queue(maxsize=1) # synchronize new frame\n self._lock = threading.Lock()\n self.start()\n\n def read(self):\n if self.ret:\n self._q.get() # if reading too freq, then wait until new frame\n self.iframe += 1\n return self.ret, self.img\n \n def release(self):\n with self._lock:\n self._isopen = False\n self.vid.release()\n\n def run(self):\n while True:\n with self._lock:\n if self._isopen:\n self.ret, self.img = self.vid.read()\n else:\n break\n if not self._q.full():\n self._q.put(None)\n if not self.ret:\n break"
},
{
"path": "ffmpegcv/ffmpeg_reader.py",
"content": "import numpy as np\nimport pprint\nimport warnings\nimport os\nimport sys\nimport select\nfrom .video_info import (\n run_async_reader as run_async,\n get_info,\n get_num_NVIDIA_GPUs,\n decoder_to_nvidia,\n release_process,\n)\n\n\ndef get_videofilter_cpu(\n originsize: list,\n pix_fmt: str,\n crop_xywh: list,\n resize: list,\n resize_keepratio: bool,\n resize_keepratioalign: str,\n):\n \"\"\"\n ONGONING: common filter for video/cam/stream capture.\n \"\"\"\n assert pix_fmt in [\"rgb24\", \"bgr24\", \"yuv420p\", \"yuvj420p\", \"nv12\", \"gray\"]\n origin_width, origin_height = originsize\n if crop_xywh:\n crop_w, crop_h = crop_xywh[2:]\n if not all([n % 2 == 0] for n in crop_xywh):\n print(\"Warning 'crop_xywh' would be replaced into even numbers\")\n crop_xywh = [int(n//2*2) for n in crop_xywh]\n assert crop_w <= origin_width and crop_h <= origin_height\n x, y, w, h = crop_xywh\n cropopt = f\"crop={w}:{h}:{x}:{y}\"\n else:\n crop_w, crop_h = origin_width, origin_height\n cropopt = \"\"\n\n crop_wh = (crop_w, crop_h)\n if resize is None or tuple(resize) == crop_wh:\n scaleopt = \"\"\n padopt = \"\"\n final_size_wh = crop_wh\n else:\n final_size_wh = (dst_width, dst_height) = resize\n assert all([n % 2 == 0] for n in resize), \"'resize' must be even number\"\n if not resize_keepratio:\n scaleopt = f\"scale={dst_width}x{dst_height}\"\n padopt = \"\"\n else:\n re_width, re_height = crop_w / (crop_h / dst_height), dst_height\n if re_width > dst_width:\n re_width, re_height = dst_width, crop_h / (crop_w / dst_width)\n re_width, re_height = int(re_width), int(re_height)\n scaleopt = f\"scale={re_width}x{re_height}\"\n if resize_keepratioalign is None:\n resize_keepratioalign = \"center\"\n paddings = {\n \"center\": ((dst_width - re_width) // 2, (dst_height - re_height) // 2,),\n \"topleft\": (0, 0),\n \"topright\": (dst_width - re_width, 0),\n \"bottomleft\": (0, dst_height - re_height),\n \"bottomright\": (dst_width - re_width, dst_height - re_height),\n }\n assert (\n resize_keepratioalign in paddings\n ), 'resize_keepratioalign must be one of \"center\"(mmpose), \"topleft\"(mmdetection), \"topright\", \"bottomleft\", \"bottomright\"'\n xpading, ypading = paddings[resize_keepratioalign]\n padopt = f\"pad={dst_width}:{dst_height}:{xpading}:{ypading}:black\"\n\n pix_fmtopt = \"extractplanes=y\" if pix_fmt == \"gray\" else \"\"\n if any([cropopt, scaleopt, padopt, pix_fmtopt]):\n filterstr = \",\".join(x for x in [cropopt, scaleopt, padopt, pix_fmtopt] if x)\n filteropt = f\"-vf {filterstr}\"\n else:\n filteropt = \"\"\n return crop_wh, final_size_wh, filteropt\n\n\ndef get_videofilter_gpu(\n originsize: list,\n pix_fmt: str,\n crop_xywh: list,\n resize: list,\n resize_keepratio: bool,\n resize_keepratioalign: str,\n):\n assert pix_fmt in [\"rgb24\", \"bgr24\", \"yuv420p\", \"yuvj420p\", \"nv12\", \"gray\"]\n origin_width, origin_height = originsize\n if crop_xywh:\n crop_w, crop_h = crop_xywh[2:]\n assert all([n % 2 == 0] for n in crop_xywh), \"'crop_xywh' must be even number\"\n assert crop_w <= origin_width and crop_h <= origin_height\n x, y, w, h = crop_xywh\n top, bottom, left, right = (\n y,\n origin_height - (y + h),\n x,\n origin_width - (x + w),\n ) # crop length\n cropopt = f\"-crop {top}x{bottom}x{left}x{right}\"\n else:\n crop_w, crop_h = origin_width, origin_height\n cropopt = \"\"\n\n crop_wh = (crop_w, crop_h)\n filteropt = \"\"\n scaleopt = \"\"\n if resize is None or tuple(resize) == crop_wh:\n final_size_wh = crop_wh\n else:\n final_size_wh = (dst_width, dst_height) = resize\n assert all([n % 2 == 0] for n in resize), \"'resize' must be even number\"\n if not resize_keepratio:\n scaleopt = f\"-resize {dst_width}x{dst_height}\"\n else:\n re_width, re_height = crop_w / (crop_h / dst_height), dst_height\n if re_width > dst_width:\n re_width, re_height = dst_width, crop_h / (crop_w / dst_width)\n re_width, re_height = int(re_width), int(re_height)\n scaleopt = f\"-resize {re_width}x{re_height}\"\n if resize_keepratioalign is None:\n resize_keepratioalign = \"center\"\n paddings = {\n \"center\": ((dst_width - re_width) // 2, (dst_height - re_height) // 2,),\n \"topleft\": (0, 0),\n \"topright\": (dst_width - re_width, 0),\n \"bottomleft\": (0, dst_height - re_height),\n \"bottomright\": (dst_width - re_width, dst_height - re_height),\n }\n assert (\n resize_keepratioalign in paddings\n ), 'resize_keepratioalign must be one of \"center\"(mmpose), \"topleft\"(mmdetection), \"topright\", \"bottomleft\", \"bottomright\"'\n xpading, ypading = paddings[resize_keepratioalign]\n padopt = f\"pad={dst_width}:{dst_height}:{xpading}:{ypading}:black\"\n filteropt = f\"-vf {padopt}\"\n\n if pix_fmt == \"gray\":\n if filteropt:\n filteropt = f\"{filteropt},extractplanes=y\"\n else:\n filteropt = f\"-vf extractplanes=y\"\n\n return crop_wh, final_size_wh, [cropopt, scaleopt, filteropt]\n\n\ndef get_outnumpyshape(size_wh: list, pix_fmt: str) -> tuple:\n width, height = size_wh\n assert (not pix_fmt == \"yuv420p\") or (\n height % 2 == 0 and width % 2 == 0\n ), \"yuv420p must be even\"\n out_numpy_shape = {\n \"rgb24\": (height, width, 3),\n \"bgr24\": (height, width, 3),\n \"yuv420p\": (int(height * 1.5), width),\n \"yuvj420p\": (int(height * 1.5), width),\n \"nv12\": (int(height * 1.5), width),\n \"gray\": (height, width, 1),\n }[pix_fmt]\n return out_numpy_shape\n\n\nclass FFmpegReader:\n def __init__(self):\n self.filename:str = ''\n self.iframe:int = -1\n self.width:int = None\n self.height:int = None\n self.size = (None, None)\n self.waitInit:bool = True\n self.process = None\n self._isopen:bool = True\n self.debug:bool = False\n self.fps:float = None\n self.out_numpy_shape = (None, None, None)\n\n def __repr__(self):\n props = pprint.pformat(self.__dict__).replace(\"{\", \" \").replace(\"}\", \" \")\n return f\"{self.__class__}\\n\" + props\n\n def __enter__(self):\n return self\n\n def __exit__(self, type, value, traceback):\n self.release()\n\n def __len__(self):\n return self.count\n\n def __iter__(self):\n return self\n\n def __next__(self):\n ret, img = self.read()\n if ret:\n return img\n else:\n raise StopIteration\n\n @staticmethod\n def VideoReader(\n filename,\n codec,\n pix_fmt,\n crop_xywh,\n resize,\n resize_keepratio,\n resize_keepratioalign,\n infile_options=None\n ):\n assert os.path.exists(filename) and os.path.isfile(\n filename\n ), f\"{filename} not exists\"\n\n vid = FFmpegReader()\n vid.filename = filename\n videoinfo = get_info(filename)\n vid.origin_width = videoinfo.width\n vid.origin_height = videoinfo.height\n vid.fps = videoinfo.fps\n vid.count = videoinfo.count\n vid.duration = videoinfo.duration\n vid.pix_fmt = pix_fmt\n vid.codec = videoinfo.codec\n\n if infile_options is None:\n infile_options = ''\n \n if codec is not None:\n warnings.warn(\n \"The 'codec' parameter is auto detected and will be removed \" \n \"in future versions. Please refrain from using this parameter.\",\n DeprecationWarning\n )\n\n (\n (vid.crop_width, vid.crop_height),\n (vid.width, vid.height),\n filteropt,\n ) = get_videofilter_cpu(\n (vid.origin_width, vid.origin_height),\n pix_fmt,\n crop_xywh,\n resize,\n resize_keepratio,\n resize_keepratioalign,\n )\n vid.size = (vid.width, vid.height)\n\n vid.ffmpeg_cmd = (\n f\"ffmpeg -loglevel error {infile_options}\"\n f' -vcodec {vid.codec} -r {vid.fps} -i \"{filename}\" '\n f\" {filteropt} -pix_fmt {pix_fmt} -r {vid.fps} -f rawvideo pipe:\"\n )\n vid.out_numpy_shape = get_outnumpyshape(vid.size, pix_fmt)\n return vid\n\n def read(self):\n if self.waitInit:\n self.process = run_async(self.ffmpeg_cmd)\n self.waitInit = False\n \n in_bytes = self.process.stdout.read(np.prod(self.out_numpy_shape))\n # check if ffmpeg process error\n # if self.process.stderr.readable():\n # print('---a')\n # data = self.process.stderr.read()\n # sys.stderr.buffer.write(data)\n # print('---f')\n\n if not in_bytes:\n self.release()\n return False, None\n self.iframe += 1\n img = np.frombuffer(in_bytes, np.uint8).reshape(self.out_numpy_shape)\n\n return True, img\n\n def isOpened(self):\n return self._isopen\n\n def release(self):\n self._isopen = False\n release_process(self.process, forcekill=True)\n\n def close(self):\n return self.release()\n\n\nclass FFmpegReaderNV(FFmpegReader):\n def _get_opts(\n vid,\n videoinfo,\n crop_xywh,\n resize,\n resize_keepratio,\n resize_keepratioalign,\n isgray,\n ):\n vid.origin_width = videoinfo.width\n vid.origin_height = videoinfo.height\n vid.fps = videoinfo.fps\n vid.count = videoinfo.count\n vid.duration = videoinfo.duration\n vid.width, vid.height = vid.origin_width, vid.origin_height\n vid.codec = videoinfo.codec\n assert vid.origin_height % 2 == 0, \"height must be even\"\n assert vid.origin_width % 2 == 0, \"width must be even\"\n if crop_xywh:\n crop_w, crop_h = crop_xywh[2:]\n vid.width, vid.height = crop_w, crop_h\n x, y, w, h = crop_xywh\n top, bottom, left, right = (\n y,\n vid.origin_height - (y + h),\n x,\n vid.origin_width - (x + w),\n ) # crop length\n cropopt = f\"-crop {top}x{bottom}x{left}x{right}\"\n else:\n crop_w, crop_h = vid.origin_width, vid.origin_height\n cropopt = \"\"\n\n vid.crop_width, vid.crop_height = crop_w, crop_h\n\n if resize is None or tuple(resize) == (vid.crop_width, vid.crop_height):\n scaleopt = \"\"\n filteropt = \"\"\n else:\n vid.width, vid.height = dst_width, dst_height = resize\n if not resize_keepratio:\n scaleopt = f\"-resize {dst_width}x{dst_height}\"\n filteropt = \"\"\n else:\n re_width, re_height = crop_w / (crop_h / dst_height), dst_height\n if re_width > dst_width:\n re_width, re_height = dst_width, crop_h / (crop_w / dst_width)\n re_width, re_height = int(re_width), int(re_height)\n scaleopt = f\"-resize {re_width}x{re_height}\"\n if resize_keepratioalign is None:\n resize_keepratioalign = \"center\"\n paddings = {\n \"center\": (\n (dst_width - re_width) // 2,\n (dst_height - re_height) // 2,\n ),\n \"topleft\": (0, 0),\n \"topright\": (dst_width - re_width, 0),\n \"bottomleft\": (0, dst_height - re_height),\n \"bottomright\": (dst_width - re_width, dst_height - re_height),\n }\n assert (\n resize_keepratioalign in paddings\n ), 'resize_keepratioalign must be one of \"center\"(mmpose), \"topleft\"(mmdetection), \"topright\", \"bottomleft\", \"bottomright\"'\n xpading, ypading = paddings[resize_keepratioalign]\n padopt = f\"pad={dst_width}:{dst_height}:{xpading}:{ypading}:black\"\n filteropt = f\"-vf {padopt}\"\n\n if isgray:\n if filteropt:\n filteropt = f\"{filteropt},extractplanes=y\"\n else:\n filteropt = f\"-vf extractplanes=y\"\n\n vid.size = (vid.width, vid.height)\n return cropopt, scaleopt, filteropt\n\n @staticmethod\n def VideoReader(\n filename,\n pix_fmt,\n crop_xywh,\n resize,\n resize_keepratio,\n resize_keepratioalign,\n infile_options,\n gpu,\n ):\n assert os.path.exists(filename) and os.path.isfile(\n filename\n ), f\"{filename} not exists\"\n assert pix_fmt in [\"rgb24\", \"bgr24\", \"yuv420p\", \"yuvj420p\", \"nv12\", \"gray\"]\n numGPU = get_num_NVIDIA_GPUs()\n assert numGPU > 0, \"No GPU found\"\n gpu = int(gpu) % numGPU if gpu is not None else 0\n assert (\n resize is None or len(resize) == 2\n ), \"resize must be a tuple of (width, height)\"\n if infile_options is None:\n infile_options = ''\n videoinfo = get_info(filename)\n vid = FFmpegReaderNV()\n vid.filename = filename\n isgray = pix_fmt == \"gray\"\n cropopt, scaleopt, filteropt = vid._get_opts(\n videoinfo,\n crop_xywh,\n resize,\n resize_keepratio,\n resize_keepratioalign,\n isgray,\n )\n vid.codecNV = decoder_to_nvidia(vid.codec)\n\n vid.ffmpeg_cmd = (\n f\"ffmpeg -loglevel error -hwaccel cuda -hwaccel_device {gpu} {infile_options} \"\n f' -vcodec {vid.codecNV} {cropopt} {scaleopt} -r {vid.fps} -i \"{filename}\" '\n f\" {filteropt} -pix_fmt {pix_fmt} -r {vid.fps} -f rawvideo pipe:\"\n )\n\n vid.pix_fmt = pix_fmt\n vid.out_numpy_shape = get_outnumpyshape(vid.size, pix_fmt)\n return vid\n"
},
{
"path": "ffmpegcv/ffmpeg_reader_camera.py",
"content": "import numpy as np\nimport pprint\nfrom .video_info import run_async, release_process\nimport re\nimport subprocess\nfrom threading import Thread\nfrom queue import Queue\nimport sys\nimport os\nfrom ffmpegcv.ffmpeg_reader import get_videofilter_cpu, get_outnumpyshape\n\n\nclass platform:\n win = 0\n linux = 1\n mac = 2\n other = 3\n\n\nif sys.platform.startswith(\"linux\"):\n this_os = platform.linux\nelif sys.platform.startswith(\"win32\"):\n this_os = platform.win\nelif sys.platform.startswith(\"darwin\"):\n this_os = platform.mac\nelse:\n this_os = platform.other\n\n\ndef _query_camera_divices_mac() -> dict:\n # run the command 'ffmpeg -f avfoundation -list_devices true -i \"\" '\n command = 'ffmpeg -hide_banner -f avfoundation -list_devices true -i \"\"'\n process = subprocess.Popen(\n command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE\n )\n stdout, stderr = process.communicate()\n\n # parse the output into a dictionary\n lines = stderr.decode(\"utf-8\").split(\"AVFoundation audio devices:\")[0].split(\"\\n\")\n id_device_map = dict()\n device_id_pattern = re.compile(r\"\\[[^\\]]*?\\] \\[(\\d*)\\]\")\n device_name_pattern = re.compile(r\".*\\] (.*)\")\n for line in lines[1:-1]:\n device_id = int(re.search(device_id_pattern, line).group(1))\n device_name = re.search(device_name_pattern, line).group(1)\n id_device_map[device_id] = (device_name, device_id)\n return id_device_map\n\n\ndef _query_camera_divices_win() -> dict:\n command = \"ffmpeg -hide_banner -list_devices true -f dshow -i dummy\"\n process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)\n stdout, stderr = process.communicate()\n dshowliststr = stderr.decode(\"utf-8\")\n dshowliststr = dshowliststr.split(\"DirectShow audio devices\")[0]\n pattern = re.compile(r'\\[*?\\] *\"([^\"]*)\"')\n matches = pattern.findall(dshowliststr)\n alternative_pattern = re.compile(r'Alternative name \"(.*)\"')\n alternative_names = alternative_pattern.findall(dshowliststr)\n assert len(matches) == len(alternative_names)\n id_device_map = {\n i: device for i, device in enumerate(zip(matches, alternative_names))\n }\n if len(id_device_map) == 0:\n print(\"No camera divice found\")\n return id_device_map\n\n\ndef _query_camera_divices_linux() -> dict:\n \"edit from https://github.com/p513817/python-get-cam-name/blob/master/get_cam_name.py\"\n root = \"/sys/class/video4linux\"\n cam_info = []\n\n for index in sorted([file for file in os.listdir(root)]):\n # Get Camera Name From /sys/class/video4linux//name\n real_index_file = os.path.realpath(\"/sys/class/video4linux/\" + index + \"/index\")\n with open(real_index_file, \"r\") as name_file:\n _index = name_file.read().rstrip()\n if _index != \"0\":\n continue\n\n real_file = os.path.realpath(\"/sys/class/video4linux/\" + index + \"/name\")\n with open(real_file, \"r\") as name_file:\n name = name_file.read().rstrip()\n name = name.split(\":\")[0]\n\n # Setup Each Camera and Index ( video* )\n cam_info.append((name, \"/dev/\" + index))\n\n id_device_map = {i: vname for i, vname in enumerate(cam_info)}\n return id_device_map\n\n\ndef query_camera_devices(verbose_dict: bool = False) -> dict:\n result = {\n platform.linux: _query_camera_divices_linux,\n platform.mac: _query_camera_divices_mac,\n platform.win: _query_camera_divices_win,\n }[this_os]()\n if verbose_dict:\n dict_by_v0 = {v[0]: v for v in result.values()}\n dict_by_v1 = {v[1]: v for v in result.values()}\n result.update(dict_by_v0)\n result.update(dict_by_v1)\n\n return result\n\n\ndef _query_camera_options_mac(cam_id_name) -> str:\n print(\n \"\\033[33m\"\n + \"FFmpeg& FFmpegcv CAN NOT query the camera options in MAC platform.\"\n + \"\\033[0m\"\n )\n print(\"Please find the proper parameter other way.\")\n return [{\"camsize_wh\": None, \"camfps\": None}]\n\n\ndef _query_camera_options_linux(cam_id_name) -> str:\n print(\n \"\\033[33m\"\n + \"FFmpeg& FFmpegcv CAN NOT query the camera FPS in Linux platform.\"\n + \"\\033[0m\"\n )\n print(\"Please find the proper parameter other way.\")\n camname = query_camera_devices(verbose_dict=True)[cam_id_name][1]\n command = f'ffmpeg -hide_banner -f v4l2 -list_formats all -i \"{camname}\"'\n process = subprocess.Popen(\n command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE\n )\n stdout, stderr = process.communicate()\n lines = stderr.decode(\"utf-8\").split(\"\\n\")\n lines = [l for l in lines if \"v4l2\" in l]\n outlist = []\n for line in lines:\n _, vcodec, *_, resolutions = line.split(\":\")\n vcodec = vcodec.strip()\n israw = \"Raw\" in line\n camcodec = None if israw else vcodec\n campix_fmt = vcodec if israw else None\n resolutions = resolutions.strip()\n\n camsize_wh_l = [tuple(map(int, r.split(\"x\"))) for r in resolutions.split()]\n outlist.extend(\n [\n {\n \"camcodec\": camcodec,\n \"campix_fmt\": campix_fmt,\n \"camsize_wh\": wh,\n \"camfps\": None,\n }\n for wh in camsize_wh_l\n ]\n )\n return outlist\n\n\ndef _query_camera_options_win(cam_id_name) -> str:\n if isinstance(cam_id_name, int):\n id_device_map = query_camera_devices()\n camname = id_device_map[cam_id_name][1]\n elif isinstance(cam_id_name, str):\n camname = cam_id_name\n else:\n raise ValueError(\"Not valid camname\")\n command = f'ffmpeg -hide_banner -f dshow -list_options true -i video=\"{camname}\"'\n process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)\n stdout, stderr = process.communicate()\n dshowliststr = stderr.decode(\"utf-8\").replace(\"\\r\\n\", \"\\n\").replace(\"\\r\", \"\\n\")\n dshowlist = [s for s in dshowliststr.split(\"\\n\") if \"fps=\" in s]\n from collections import OrderedDict\n\n unique_dshowlist = list(OrderedDict.fromkeys(dshowlist))\n outlist = []\n for text in unique_dshowlist:\n cam_options = dict()\n cam_options[\"camcodec\"] = (\n re.search(r\"vcodec=(\\w+)\", text).group(1) if \"vcodec\" in text else None\n )\n cam_options[\"campix_fmt\"] = (\n re.search(r\"pixel_format=(\\w+)\", text).group(1)\n if \"pixel_format\" in text\n else None\n )\n camsize_wh = re.search(r\"min s=(\\w+)\", text).group(1)\n cam_options[\"camsize_wh\"] = tuple(int(v) for v in camsize_wh.split(\"x\"))\n camfps = float(re.findall(r\"fps=([\\d.]+)\", text)[-1])\n cam_options[\"camfps\"] = int(camfps) if int(camfps) == camfps else camfps\n outlist.append(cam_options)\n return outlist\n\n\ndef query_camera_options(cam_id_name) -> str:\n return {\n platform.linux: _query_camera_options_linux,\n platform.mac: _query_camera_options_mac,\n platform.win: _query_camera_options_win,\n }[this_os](cam_id_name)\n\n\nclass ProducerThread(Thread):\n def __init__(self, vid, q):\n super(ProducerThread, self).__init__()\n self.vid = vid\n self.q = q\n\n def run(self):\n q = self.q\n while True:\n if not self.vid.isOpened():\n break\n ret, img = self.vid.read_()\n\n if q.full():\n q.get() # drop frames\n q.put((ret, img)) \n\n\nclass FFmpegReaderCAM:\n def __init__(self):\n self.iframe = -1\n self._isopen = True\n\n def __repr__(self):\n props = pprint.pformat(self.__dict__).replace(\"{\", \" \").replace(\"}\", \" \")\n return f\"{self.__class__}\\n\" + props\n\n def __enter__(self):\n return self\n\n def __exit__(self, type, value, traceback):\n self.release()\n\n def __iter__(self):\n return self\n\n def __next__(self):\n ret, img = self.read()\n if ret:\n return img\n else:\n raise StopIteration\n\n @staticmethod\n def VideoReader(\n cam_id_name,\n pix_fmt,\n crop_xywh,\n resize,\n resize_keepratio,\n resize_keepratioalign,\n camsize_wh=None,\n camfps=None,\n camcodec=None,\n campix_fmt=None,\n step=1,\n ):\n vid = FFmpegReaderCAM()\n if this_os == platform.mac:\n # use cam_id as the device marker\n if isinstance(cam_id_name, str):\n id_device_map = query_camera_devices()\n camname = cam_id_name\n id_device_map.update({v[0]: v for v in id_device_map.values()})\n camid = id_device_map[cam_id_name][1]\n else:\n camname = None\n camid = cam_id_name\n elif this_os == platform.linux:\n id_device_map = query_camera_devices(verbose_dict=True)\n camname = id_device_map[cam_id_name][-1]\n camid = None\n else:\n if isinstance(cam_id_name, int):\n id_device_map = query_camera_devices()\n camname = id_device_map[cam_id_name][1]\n camid = cam_id_name\n else:\n camname = cam_id_name\n camid = None\n\n vid.camname = camname\n vid.camid = camid\n\n if camsize_wh is None:\n cam_options = query_camera_options(camname)\n resolutions = [c[\"camsize_wh\"] for c in cam_options]\n camsize_wh = max(resolutions, key=lambda x: sum(x))\n\n assert len(camsize_wh) == 2\n vid.origin_width, vid.origin_height = camsize_wh\n\n opt_camfps = f\" -framerate {camfps} \" if camfps else \"\"\n vid.camfps = camfps if camfps else None\n\n opt_camcodec_ = {\n platform.linux: \"input_format\",\n platform.mac: \"\",\n platform.win: \"vcodec\",\n }[this_os]\n opt_camcodec = f\" -{opt_camcodec_} {camcodec} \" if camcodec else \"\"\n vid.camcodec = camcodec if camcodec else None\n vid.pix_fmt = pix_fmt\n\n opt_campix_fmt_ = {\n platform.linux: \"input_format\",\n platform.mac: \"pixel_format\",\n platform.win: \"pixel_format\",\n }[this_os]\n opt_campix_fmt = f\" -{opt_campix_fmt_} {campix_fmt} \" if campix_fmt else \"\"\n vid.campix_fmt = campix_fmt if campix_fmt else None\n\n opt_camname = {\n platform.linux: f'\"{camname}\"',\n platform.win: f'video=\"{camname}\"',\n platform.mac: f\"{camid}:none\",\n }[this_os]\n\n (vid.crop_width, vid.crop_height), (vid.width, vid.height), filteropt = get_videofilter_cpu(\n (vid.origin_width, vid.origin_height), pix_fmt, crop_xywh, resize, resize_keepratio, resize_keepratioalign)\n vid.size = (vid.width, vid.height)\n\n opt_driver_ = {\n platform.linux: \"v4l2\",\n platform.mac: \"avfoundation\",\n platform.win: \"dshow\",\n }[this_os]\n\n vid.ffmpeg_cmd = (\n f\"ffmpeg -loglevel warning \"\n f\" -f {opt_driver_} \"\n f\" -video_size {vid.origin_width}x{vid.origin_height} \"\n f\" {opt_camfps} {opt_camcodec} {opt_campix_fmt} \"\n f\" -i {opt_camname} \"\n f\" {filteropt} -pix_fmt {pix_fmt} -f rawvideo pipe:\"\n )\n\n vid.out_numpy_shape = get_outnumpyshape(vid.size, pix_fmt)\n vid.process = run_async(vid.ffmpeg_cmd)\n\n # producer\n assert step >= 1 and isinstance(step, int)\n vid.step = step\n vid.q = Queue(maxsize=30)\n producer = ProducerThread(vid, vid.q)\n producer.start()\n return vid\n\n def read_(self):\n for i in range(self.step):\n in_bytes = self.process.stdout.read(np.prod(self.out_numpy_shape))\n if not in_bytes:\n self.release()\n return False, None\n\n self.iframe += 1\n img = None\n img = np.frombuffer(in_bytes, np.uint8).reshape(self.out_numpy_shape)\n return True, img\n\n def read(self):\n ret, img = self.q.get()\n return ret, img\n\n def isOpened(self):\n return self._isopen\n \n def release(self):\n self._isopen = False\n release_process(self.process)\n\n def close(self):\n return self.release()\n"
},
{
"path": "ffmpegcv/ffmpeg_reader_cuda.py",
"content": "import pycuda.driver as cuda\nfrom pycuda.driver import PointerHolderBase\nfrom pycuda.compiler import SourceModule\nfrom pycuda import gpuarray\nfrom ffmpegcv.ffmpeg_reader import FFmpegReader, FFmpegReaderNV\nimport numpy as np\nfrom typing import Tuple\n\n\ncuda.init()\n\nmod_code = (\"\"\"\n__device__ void yuv_to_rgb(unsigned char &y, unsigned char &u, unsigned char &v, \n float &r, float &g, float &b)\n{\n // https://fourcc.org/fccyvrgb.php\n float Y_val = (float)1.164 * ((float)y - 16.0);\n float U_val = (float)u - 128.0;\n float V_val = (float)v - 128.0;\n r = Y_val + 1.596 * V_val;\n r = max(0.0, min(255.0, r)); // clamp(r, 0.0, 255.0);\n g = Y_val - 0.813 * V_val - 0.391 * U_val;\n g = max(0.0, min(255.0, g));\n b = Y_val + 2.018 * U_val;\n b = max(0.0, min(255.0, b));\n}\n\n__global__ void yuv420p_CHW_fp32(unsigned char *YUV420p, float *RGB24, int *width_, int *height_)\n{\n int width = *width_; int height = *height_;\n // Get the thread index\n int x = blockIdx.x * blockDim.x + threadIdx.x;\n int y = blockIdx.y * blockDim.y + threadIdx.y;\n\n // Check if we are within the bounds of the image\n if (x >= width || y >= height)\n return;\n\n // Get the Y, U, and V values for this pixel\n auto w_h = width * height;\n auto yW = y * width;\n auto out_ind = yW + x;\n unsigned char *Y = YUV420p;\n unsigned char *U = YUV420p + w_h;\n unsigned char *V = U + w_h/4;\n int delta = (y/2)*(width/2)+x/2;\n\n yuv_to_rgb(Y[out_ind], U[delta], V[delta],\n RGB24[out_ind], RGB24[out_ind + w_h], RGB24[out_ind + w_h*2]);\n}\n\n__global__ void yuv420p_HWC_fp32(unsigned char *YUV420p, float *RGB24, int *width_, int *height_)\n{\n int width = *width_; int height = *height_;\n // Get the thread index\n int x = blockIdx.x * blockDim.x + threadIdx.x;\n int y = blockIdx.y * blockDim.y + threadIdx.y;\n\n // Check if we are within the bounds of the image\n if (x >= width || y >= height)\n return;\n\n // Get the Y, U, and V values for this pixel\n auto w_h = width * height;\n auto yW = y * width;\n auto out_ind = yW + x;\n unsigned char *Y = YUV420p;\n unsigned char *U = YUV420p + w_h;\n unsigned char *V = U + w_h/4;\n int delta = (y/2)*(width/2)+x/2;\n auto ind = (yW + x)*3;\n\n yuv_to_rgb(Y[out_ind], U[delta], V[delta],\n RGB24[ind], RGB24[ind+1], RGB24[ind+2]);\n}\n\n__global__ void NV12_CHW_fp32(unsigned char *NV12, float *RGB24, int *width_, int *height_)\n{\n int width = *width_; int height = *height_;\n // Get the thread index\n int x = blockIdx.x * blockDim.x + threadIdx.x;\n int y = blockIdx.y * blockDim.y + threadIdx.y;\n\n // Check if we are within the bounds of the image\n if (x >= width || y >= height)\n return;\n\n // Get the Y, U, and V values for this pixel\n auto w_h = width * height;\n auto yW = y * width;\n auto out_ind = yW + x;\n unsigned char *Y = NV12 + out_ind;\n unsigned char *UV = NV12 + w_h + (y / 2) * width + (x / 2) * 2;\n yuv_to_rgb(Y[0], UV[0], UV[1],\n RGB24[out_ind], RGB24[out_ind + w_h], RGB24[out_ind + w_h*2]);\n}\n\n__global__ void NV12_HWC_fp32(unsigned char *NV12, float *RGB24, int *width_, int *height_)\n{\n int width = *width_; int height = *height_;\n // Get the thread index\n int x = blockIdx.x * blockDim.x + threadIdx.x;\n int y = blockIdx.y * blockDim.y + threadIdx.y;\n\n // Check if we are within the bounds of the image\n if (x >= width || y >= height)\n return;\n\n // Get the Y, U, and V values for this pixel\n auto w_h = width * height;\n auto yW = y * width;\n auto out_ind = yW + x;\n unsigned char *Y = NV12 + out_ind;\n unsigned char *UV = NV12 + w_h + (y / 2) * width + (x / 2) * 2;\n auto ind = (yW + x)*3;\n yuv_to_rgb(Y[0], UV[0], UV[1],\n RGB24[ind], RGB24[ind+1], RGB24[ind+2]);\n}\n\"\"\"\n)\n\n\ndef load_cuda_module():\n mod = SourceModule(mod_code)\n converter = {('yuv420p', 'chw'): mod.get_function('yuv420p_CHW_fp32'),\n ('yuv420p', 'hwc'): mod.get_function('yuv420p_HWC_fp32'),\n ('nv12', 'chw'): mod.get_function('NV12_CHW_fp32'),\n ('nv12', 'hwc'): mod.get_function('NV12_HWC_fp32')}\n return converter\n\n\nclass Holder(PointerHolderBase):\n def __init__(self, tensor):\n super().__init__()\n self.tensor = tensor\n self.gpudata = tensor.data_ptr()\n\n def get_pointer(self):\n return self.tensor.data_ptr()\n\n def __index__(self):\n return self.gpudata\n\n\ndef tensor_to_gpuarray(tensor) -> gpuarray.GPUArray:\n '''Convert a :class:`torch.Tensor` to a :class:`pycuda.gpuarray.GPUArray`. The underlying\n storage will be shared, so that modifications to the array will reflect in the tensor object.\n Parameters\n ----------\n tensor : torch.Tensor\n Returns\n -------\n pycuda.gpuarray.GPUArray\n Raises\n ------\n ValueError\n If the ``tensor`` does not live on the gpu\n '''\n return gpuarray.GPUArray(tensor.shape, dtype=np.float32, gpudata=Holder(tensor))\n\n\nclass PycudaContext:\n def __init__(self, gpu=0):\n self.ctx = cuda.Device(gpu).make_context()\n\n def __enter__(self):\n if self.ctx is not None:\n self.ctx.push()\n return self\n\n def __exit__(self, *args, **kwargs):\n if self.ctx is not None:\n self.ctx.pop()\n\n def __del__(self):\n if self.ctx is not None:\n self.ctx.pop()\n\n\nclass FFmpegReaderCUDA(FFmpegReader):\n def __init__(self, vid:FFmpegReader, gpu=0, tensor_format='hwc'):\n assert vid.pix_fmt in ['yuv420p', 'nv12'], 'Set pix_fmt to yuv420p or nv12. Auto convert to rgb in cuda.'\n assert tensor_format in ['hwc', 'chw'], 'tensor_format must be hwc or chw'\n if isinstance(vid, FFmpegReaderNV) and vid.pix_fmt != 'nv12':\n print('--Tips: please use VideoCaptureNV(..., pix_fmt=\"NV12\") for better performance.')\n elif not isinstance(vid, FFmpegReaderNV) and vid.pix_fmt == 'nv12':\n print('--Tips: please use VideoCapture(..., pix_fmt=\"yuv420p\") for better performance.')\n\n # work like normal FFmpegReaderObj\n props_name = ['width', 'height', 'fps', 'count', 'codec', 'ffmpeg_cmd',\n 'size', 'out_numpy_shape', 'iframe', \n 'duration', 'origin_width', 'origin_height']\n for name in props_name:\n setattr(self, name, getattr(vid, name, None))\n self.ctx = PycudaContext(gpu)\n self.pix_fmt = 'rgb24'\n self.vid = vid\n self.out_numpy_shape = (vid.height, vid.width, 3) if tensor_format == 'hwc' else (3, vid.height, vid.width)\n self.torch_device = f'cuda:{gpu}'\n self.block_size = (16, 16, 1)\n self.grid_size = ((self.width + self.block_size[0] - 1) // self.block_size[0],\n (self.height + self.block_size[1] - 1) // self.block_size[1])\n self.process = None\n with self.ctx:\n self.converter = load_cuda_module()[(vid.pix_fmt, tensor_format)]\n\n def read(self, out_MAT:gpuarray.GPUArray=None) -> Tuple[bool, gpuarray.GPUArray]:\n self.waitInit = False\n ret, frame_yuv420p = self.vid.read()\n if not ret:\n return False, None\n \n with self.ctx:\n if out_MAT is None:\n out_MAT = gpuarray.empty(self.out_numpy_shape, dtype=np.float32)\n self.converter(cuda.In(frame_yuv420p), out_MAT, \n cuda.In(np.int32(self.width)), cuda.In(np.int32(self.height)),\n block=self.block_size, grid=self.grid_size)\n return True, out_MAT\n \n def read_cudamem(self, out_MAT:cuda.DeviceAllocation=None) -> Tuple[bool, cuda.DeviceAllocation]:\n self.waitInit = False\n ret, frame_yuv420p = self.vid.read()\n if not ret:\n return False, None\n \n with self.ctx:\n if out_MAT is None:\n out_MAT = cuda.mem_alloc(int(np.prod(self.out_numpy_shape) * \n np.dtype(np.float32).itemsize))\n self.converter(cuda.In(frame_yuv420p), out_MAT, \n cuda.In(np.int32(self.width)), cuda.In(np.int32(self.height)),\n block=self.block_size, grid=self.grid_size)\n return True, out_MAT\n \n def read_torch(self, out_MAT=None):\n import torch\n self.waitInit = False\n ret, frame_yuv420p = self.vid.read()\n if not ret:\n return False, None\n \n with self.ctx:\n if out_MAT is None:\n out_MAT = torch.empty(self.out_numpy_shape, dtype=torch.float32, device=self.torch_device)\n tensor_proxy = tensor_to_gpuarray(out_MAT)\n self.converter(cuda.In(frame_yuv420p), tensor_proxy.gpudata, \n cuda.In(np.int32(self.width)), cuda.In(np.int32(self.height)),\n block=self.block_size, grid=self.grid_size)\n return True, out_MAT\n\n def release(self):\n self.vid.release()\n super().release()\n"
},
{
"path": "ffmpegcv/ffmpeg_reader_noblock.py",
"content": "import multiprocessing\nfrom multiprocessing import Queue\nimport numpy as np\nfrom .ffmpeg_reader import FFmpegReader\n\nNFRAME = 10\n\nclass FFmpegReaderNoblock(FFmpegReader):\n def __init__(self, \n vcap_fun,\n *vcap_args, **vcap_kwargs):\n vid:FFmpegReader = vcap_fun(*vcap_args, **vcap_kwargs)\n vid.release()\n\n # work like normal FFmpegReaderObj\n props_name = ['width', 'height', 'fps', 'count', 'codec', 'ffmpeg_cmd',\n 'size', 'pix_fmt', 'out_numpy_shape', 'iframe', \n 'duration', 'origin_width', 'origin_height']\n for name in props_name:\n setattr(self, name, getattr(vid, name, None))\n \n # 创建共享内存的NumPy数组\n shared_array = multiprocessing.Array('b', int(NFRAME*np.prod(self.out_numpy_shape)))\n\n # 将共享内存的NumPy数组转换为NumPy数组\n self.np_array = np.frombuffer(shared_array.get_obj(), dtype=np.uint8).reshape((NFRAME,*self.out_numpy_shape))\n \n self.shared_array = shared_array\n self.vcap_args = vcap_args\n self.vcap_kwargs = vcap_kwargs\n self.q = Queue(maxsize=(NFRAME-2)) #buffer index, gluttonous snake NO biting its own tail\n self.vcap_fun = vcap_fun\n self.has_init = False\n self.process = None\n\n def read(self):\n if not self.has_init:\n self.has_init = True\n process = multiprocessing.Process(target=child_process, \n args=(self.shared_array, self.q, self.vcap_fun,\n self.vcap_args, self.vcap_kwargs))\n process.start()\n self.process = process\n \n data_id = self.q.get() # 读取子进程写入的数据\n if data_id is None:\n return False, None\n else:\n self.iframe += 1\n return True, self.np_array[data_id]\n\n\ndef child_process(shared_array, q:Queue, vcap_fun, vcap_args, vcap_kwargs):\n vid = vcap_fun(*vcap_args, **vcap_kwargs)\n np_array = np.frombuffer(shared_array.get_obj(), dtype=np.uint8).reshape((NFRAME,*vid.out_numpy_shape))\n with vid:\n for i, img in enumerate(vid):\n iloop = i % NFRAME\n # 在子进程中修改共享内存的NumPy数组\n np_array[iloop] = img\n q.put(iloop) # 通知主进程已经写入了\n q.put(None)\n"
},
{
"path": "ffmpegcv/ffmpeg_reader_pannels.py",
"content": "import os\nimport numpy as np\nfrom ffmpegcv.ffmpeg_reader import FFmpegReader, get_outnumpyshape\n\nfrom ffmpegcv.video_info import (\n get_info,\n run_async\n)\n\n\nclass FFmpegReaderPannels(FFmpegReader):\n @staticmethod\n def VideoReader(\n filename:str,\n crop_xywh_l:list,\n codec,\n pix_fmt='bgr24',\n resize=None\n ):\n assert os.path.exists(filename) and os.path.isfile(\n filename\n ), f\"{filename} not exists\"\n assert pix_fmt in [\"rgb24\", \"bgr24\", \"yuv420p\", \"nv12\", \"gray\"]\n vid = FFmpegReaderPannels()\n crop_xywh_l = np.array(crop_xywh_l)\n vid.crop_xywh_l = crop_xywh_l\n videoinfo = get_info(filename)\n vid.origin_width = videoinfo.width\n vid.origin_height = videoinfo.height\n vid.fps = videoinfo.fps\n vid.count = videoinfo.count\n vid.duration = videoinfo.duration\n vid.pix_fmt = pix_fmt\n vid.codec = codec if codec else videoinfo.codec\n\n vid.crop_width_l = crop_xywh_l[:,2]\n vid.crop_height_l = crop_xywh_l[:,3]\n vid.size_l = crop_xywh_l[:,2:][:,::-1]\n vid.npannel = len(crop_xywh_l)\n vid.out_numpy_shape_l = [get_outnumpyshape(s[::-1], pix_fmt) for s in vid.size_l]\n if len(set(vid.crop_width_l)) == len(set(vid.crop_height_l)) == 1:\n vid.is_pannel_similar = True\n vid.crop_width = vid.crop_width_l[0]\n vid.crop_height = vid.crop_height_l[0]\n vid.size = vid.size_l[0]\n vid.out_numpy_shape = (vid.npannel, *vid.out_numpy_shape_l[0])\n else:\n vid.is_pannel_similar = False\n vid.crop_width = vid.crop_height = vid.size = None\n vid.out_numpy_shape = (np.sum(np.prod(s) for s in vid.out_numpy_shape_l),)\n\n VINSRCs =''.join(f'[VSRC{i}]' for i in range(vid.npannel))\n pix_fmtopt = ',extractplanes=y' if pix_fmt=='gray' else ''\n CROPs = ';'.join(f'[VSRC{i}]crop={w}:{h}:{x}:{y}{pix_fmtopt}[VPANEL{i}]'\n for i, (x,y,w,h) in enumerate(vid.crop_xywh_l))\n filteropt = f' -filter_complex \"split={vid.npannel}{VINSRCs};{CROPs}\"'\n outmaps = ''.join(f' -map [VPANEL{i}] -pix_fmt {pix_fmt} -r {vid.fps} -f rawvideo pipe:'\n for i in range(vid.npannel))\n\n vid.ffmpeg_cmd = (\n f\"ffmpeg -loglevel warning \"\n f' -r {vid.fps} -i \"{filename}\" '\n f\" {filteropt} {outmaps}\"\n )\n return vid\n \n def read(self):\n if self.waitInit:\n self.process = run_async(self.ffmpeg_cmd)\n self.waitInit = False\n \n in_bytes = self.process.stdout.read(np.prod(self.out_numpy_shape))\n if not in_bytes:\n self.release()\n return False, None\n self.iframe += 1\n img0 = np.frombuffer(in_bytes, np.uint8)\n if self.is_pannel_similar:\n img = img0.reshape(self.out_numpy_shape)\n else:\n img = []\n for out_numpy_shape in self.out_numpy_shape_l:\n nbuff = np.prod(out_numpy_shape)\n img.append(img0[:nbuff].reshape(out_numpy_shape))\n img0 = img0[nbuff:]\n\n return True, img\n"
},
{
"path": "ffmpegcv/ffmpeg_reader_qsv.py",
"content": "import os\nfrom ffmpegcv.ffmpeg_reader import FFmpegReader, get_videofilter_cpu, get_outnumpyshape\nfrom .video_info import (\n get_info,\n get_num_QSV_GPUs,\n decoder_to_qsv,\n)\n\n\nclass FFmpegReaderQSV(FFmpegReader):\n @staticmethod\n def VideoReader(\n filename,\n pix_fmt,\n crop_xywh,\n resize,\n resize_keepratio,\n resize_keepratioalign,\n infile_options,\n gpu,\n ):\n \"\"\"\n TODO: 1. only 1 gpu is recognized\n TODO: 2. 'crop_xywh', 'resize*' are not supported yet\n \"\"\"\n assert os.path.exists(filename) and os.path.isfile(\n filename\n ), f\"{filename} not exists\"\n assert gpu is None or gpu == 0, \"Cannot use multiple QSV gpu yet.\"\n numGPU = get_num_QSV_GPUs()\n assert numGPU > 0, \"No GPU found\"\n gpu = int(gpu) % numGPU if gpu is not None else 0\n assert (\n resize is None or len(resize) == 2\n ), \"resize must be a tuple of (width, height)\"\n\n vid = FFmpegReaderQSV()\n videoinfo = get_info(filename)\n vid.origin_width = videoinfo.width\n vid.origin_height = videoinfo.height\n vid.fps = videoinfo.fps\n vid.count = videoinfo.count\n vid.duration = videoinfo.duration\n vid.codecQSV = decoder_to_qsv(videoinfo.codec)\n vid.pix_fmt = pix_fmt\n\n if infile_options is None:\n infile_options = ''\n \n (\n (vid.crop_width, vid.crop_height),\n (vid.width, vid.height),\n filteropt,\n ) = get_videofilter_cpu(\n (vid.origin_width, vid.origin_height),\n pix_fmt,\n crop_xywh,\n resize,\n resize_keepratio,\n resize_keepratioalign,\n )\n vid.size = (vid.width, vid.height)\n\n vid.ffmpeg_cmd = (\n f\"ffmpeg -loglevel warning {infile_options}\"\n f' -vcodec {vid.codecQSV} -r {vid.fps} -i \"{filename}\" '\n f\" {filteropt} -pix_fmt {pix_fmt} -r {vid.fps} -f rawvideo pipe:\"\n )\n\n vid.out_numpy_shape = get_outnumpyshape(vid.size, pix_fmt)\n return vid\n"
},
{
"path": "ffmpegcv/ffmpeg_reader_stream.py",
"content": "from .video_info import run_async\nfrom queue import Queue\nfrom ffmpegcv.stream_info import get_info\nfrom ffmpegcv.ffmpeg_reader_camera import FFmpegReaderCAM, ProducerThread\nfrom ffmpegcv.ffmpeg_reader import (\n get_videofilter_cpu,\n get_outnumpyshape,\n get_videofilter_gpu,\n get_num_NVIDIA_GPUs,\n decoder_to_nvidia,\n)\n\n\nclass FFmpegReaderStream(FFmpegReaderCAM):\n def __init__(self):\n super().__init__()\n\n @staticmethod\n def VideoReader(\n stream_url,\n codec,\n pix_fmt,\n crop_xywh,\n resize,\n resize_keepratio,\n resize_keepratioalign,\n infile_options,\n timeout,\n ):\n\n vid = FFmpegReaderStream()\n videoinfo = get_info(stream_url, timeout)\n vid.origin_width = videoinfo.width\n vid.origin_height = videoinfo.height\n vid.fps = videoinfo.fps\n vid.codec = codec if codec else videoinfo.codec\n vid.count = videoinfo.count\n vid.duration = videoinfo.duration\n vid.pix_fmt = pix_fmt\n\n if infile_options is None:\n infile_options = ''\n (\n (vid.crop_width, vid.crop_height),\n (vid.width, vid.height),\n filteropt,\n ) = get_videofilter_cpu(\n (vid.origin_width, vid.origin_height),\n pix_fmt,\n crop_xywh,\n resize,\n resize_keepratio,\n resize_keepratioalign,\n )\n vid.size = (vid.width, vid.height)\n\n rtsp_opt = '' if not stream_url.startswith('rtsp://') else '-rtsp_flags prefer_tcp -pkt_size 736 '\n vid.ffmpeg_cmd = (\n f\"ffmpeg -loglevel warning {infile_options}\"\n f\" {rtsp_opt} \"\n f\" -vcodec {vid.codec} -i {stream_url} \"\n f\" {filteropt} -pix_fmt {pix_fmt} -f rawvideo pipe:\"\n )\n vid.out_numpy_shape = get_outnumpyshape(vid.size, pix_fmt)\n vid.process = run_async(vid.ffmpeg_cmd)\n\n vid.isopened = True\n\n # producer\n vid.step = 1\n vid.q = Queue(maxsize=30)\n producer = ProducerThread(vid, vid.q)\n producer.start()\n return vid\n\n\nclass FFmpegReaderStreamNV(FFmpegReaderCAM):\n def __init__(self):\n super().__init__()\n\n @staticmethod\n def VideoReader(\n stream_url,\n codec,\n pix_fmt,\n crop_xywh,\n resize,\n resize_keepratio,\n resize_keepratioalign,\n infile_options,\n gpu,\n timeout,\n ):\n numGPU = get_num_NVIDIA_GPUs()\n vid = FFmpegReaderStreamNV()\n videoinfo = get_info(stream_url, timeout)\n vid.origin_width = videoinfo.width\n vid.origin_height = videoinfo.height\n vid.fps = videoinfo.fps\n vid.codec = codec if codec else videoinfo.codec\n vid.codecNV = decoder_to_nvidia(vid.codec)\n vid.count = videoinfo.count\n vid.duration = videoinfo.duration\n vid.pix_fmt = pix_fmt\n\n if infile_options is None:\n infile_options = ''\n \n (\n (vid.crop_width, vid.crop_height),\n (vid.width, vid.height),\n (cropopt, scaleopt, filteropt),\n ) = get_videofilter_gpu(\n (vid.origin_width, vid.origin_height),\n pix_fmt,\n crop_xywh,\n resize,\n resize_keepratio,\n resize_keepratioalign,\n )\n vid.size = (vid.width, vid.height)\n\n rtsp_opt = '' if not stream_url.startswith('rtsp://') else '-rtsp_flags prefer_tcp -pkt_size 736 '\n vid.ffmpeg_cmd = (\n f\"ffmpeg -loglevel warning -hwaccel cuda -hwaccel_device {gpu}\"\n f\" {infile_options} {rtsp_opt} \"\n f' -vcodec {vid.codecNV} {cropopt} {scaleopt} -i \"{stream_url}\" '\n f\" {filteropt} -pix_fmt {pix_fmt} -f rawvideo pipe:\"\n )\n vid.out_numpy_shape = get_outnumpyshape(vid.size, pix_fmt)\n vid.process = run_async(vid.ffmpeg_cmd)\n\n vid.isopened = True\n\n # producer\n vid.step = 1\n vid.q = Queue(maxsize=30)\n producer = ProducerThread(vid, vid.q)\n producer.start()\n return vid\n"
},
{
"path": "ffmpegcv/ffmpeg_reader_stream_realtime.py",
"content": "from ffmpegcv.ffmpeg_reader import (\n FFmpegReader,\n get_videofilter_cpu,\n get_outnumpyshape,\n get_videofilter_gpu,\n get_num_NVIDIA_GPUs,\n decoder_to_nvidia,\n)\nfrom ffmpegcv.stream_info import get_info\n\n\nclass FFmpegReaderStreamRT(FFmpegReader):\n def __init__(self):\n super().__init__()\n\n @staticmethod\n def VideoReader(\n stream_url,\n codec,\n pix_fmt,\n crop_xywh,\n resize,\n resize_keepratio,\n resize_keepratioalign,\n infile_options,\n timeout,\n ):\n vid = FFmpegReaderStreamRT()\n videoinfo = get_info(stream_url, timeout)\n vid.origin_width = videoinfo.width\n vid.origin_height = videoinfo.height\n vid.fps = videoinfo.fps\n vid.codec = codec if codec else videoinfo.codec\n vid.count = videoinfo.count\n vid.duration = videoinfo.duration\n vid.pix_fmt = pix_fmt\n\n if infile_options is None:\n infile_options = ''\n\n (\n (vid.crop_width, vid.crop_height),\n (vid.width, vid.height),\n filteropt,\n ) = get_videofilter_cpu(\n (vid.origin_width, vid.origin_height),\n pix_fmt,\n crop_xywh,\n resize,\n resize_keepratio,\n resize_keepratioalign,\n )\n vid.size = (vid.width, vid.height)\n\n rtsp_opt = '' if not stream_url.startswith('rtsp://') else '-rtsp_flags prefer_tcp -pkt_size 736 '\n vid.ffmpeg_cmd = (\n f\"ffmpeg -loglevel error \"\n f\" {infile_options} {rtsp_opt} \"\n \"-fflags nobuffer -flags low_delay -strict experimental \"\n f\" -vcodec {vid.codec} -i {stream_url}\"\n f\" {filteropt} -pix_fmt {pix_fmt} -f rawvideo pipe:\"\n )\n\n vid.out_numpy_shape = get_outnumpyshape(vid.size, pix_fmt)\n return vid\n\n\nclass FFmpegReaderStreamRTNV(FFmpegReader):\n def __init__(self):\n super().__init__()\n\n @staticmethod\n def VideoReader(\n stream_url,\n codec,\n pix_fmt,\n crop_xywh,\n resize,\n resize_keepratio,\n resize_keepratioalign,\n infile_options,\n gpu,\n timeout,\n ):\n vid = FFmpegReaderStreamRTNV()\n videoinfo = get_info(stream_url, timeout)\n vid.origin_width = videoinfo.width\n vid.origin_height = videoinfo.height\n vid.fps = videoinfo.fps\n vid.codec = codec if codec else videoinfo.codec\n vid.codecNV = decoder_to_nvidia(vid.codec)\n vid.count = videoinfo.count\n vid.duration = videoinfo.duration\n vid.pix_fmt = pix_fmt\n\n if infile_options is None:\n infile_options = ''\n\n numGPU = get_num_NVIDIA_GPUs()\n assert numGPU > 0, \"No GPU found\"\n gpu = int(gpu) % numGPU if gpu is not None else 0\n\n (\n (vid.crop_width, vid.crop_height),\n (vid.width, vid.height),\n (cropopt, scaleopt, filteropt),\n ) = get_videofilter_gpu(\n (vid.origin_width, vid.origin_height),\n pix_fmt,\n crop_xywh,\n resize,\n resize_keepratio,\n resize_keepratioalign,\n )\n vid.size = (vid.width, vid.height)\n\n rtsp_opt = \"-rtsp_transport tcp \" if stream_url.startswith(\"rtsp://\") else \"\"\n vid.ffmpeg_cmd = (\n f\"ffmpeg -loglevel error -hwaccel cuda -hwaccel_device {gpu} \"\n f\" {infile_options} {rtsp_opt} \"\n f\"{cropopt} {scaleopt} \"\n \"-fflags nobuffer -flags low_delay -strict experimental \"\n f' -vcodec {vid.codecNV} -i \"{stream_url}\" '\n f\" {filteropt} -pix_fmt {pix_fmt} -f rawvideo pipe:\"\n )\n\n vid.out_numpy_shape = get_outnumpyshape(vid.size, pix_fmt)\n return vid\n"
},
{
"path": "ffmpegcv/ffmpeg_writer.py",
"content": "import numpy as np\nimport warnings\nimport pprint\nimport select\nimport sys\nfrom .video_info import run_async, release_process_writer, get_num_NVIDIA_GPUs\n\n\nIN_COLAB = \"google.colab\" in sys.modules\n\n\nclass FFmpegWriter:\n def __init__(self):\n self.iframe = -1\n self.size = None\n self.width, self.height = None, None\n self.waitInit = True\n self._isopen = True\n\n def __enter__(self):\n return self\n\n def __exit__(self, type, value, traceback):\n self.release()\n\n def __del__(self):\n self.release()\n\n def __repr__(self):\n props = pprint.pformat(self.__dict__).replace(\"{\", \" \").replace(\"}\", \" \")\n return f\"{self.__class__}\\n\" + props\n\n @staticmethod\n def VideoWriter(\n filename, codec, fps, pix_fmt, bitrate=None, resize=None, preset=None\n ):\n if codec is None:\n codec = \"h264\"\n elif not isinstance(codec, str):\n codec = \"h264\"\n warnings.simplefilter(\n \"\"\"\n Codec should be a string. Eg `h264`, `h264_nvenc`. \n You may used CV2.VideoWriter_fourcc, which will be ignored.\n \"\"\"\n )\n assert resize is None or len(resize) == 2\n\n vid = FFmpegWriter()\n vid.fps = fps\n vid.codec, vid.pix_fmt, vid.filename = codec, pix_fmt, filename\n vid.bitrate = bitrate\n vid.resize = resize\n vid.preset = preset\n return vid\n\n def _init_video_stream(self):\n bitrate_str = f\"-b:v {self.bitrate} \" if self.bitrate else \"\"\n rtsp_str = f\"-f rtsp\" if self.filename.startswith(\"rtsp://\") else \"\"\n filter_str = (\n \"\"\n if self.resize == self.size\n else f\"-vf scale={self.resize[0]}:{self.resize[1]}\"\n )\n target_pix_fmt = getattr(self, \"target_pix_fmt\", \"yuv420p\")\n preset_str = f\"-preset {self.preset} \" if self.preset else \"\"\n\n self.ffmpeg_cmd = (\n f\"ffmpeg -y -loglevel error \"\n f\"-f rawvideo -pix_fmt {self.pix_fmt} -s {self.width}x{self.height} -r {self.fps} -i pipe: \"\n f\"{bitrate_str} \"\n f\"-r {self.fps} -c:v {self.codec} \"\n f\"{preset_str}\"\n f\"{filter_str} {rtsp_str} \"\n f'-pix_fmt {target_pix_fmt} \"{self.filename}\"'\n )\n self.process = run_async(self.ffmpeg_cmd)\n\n def write(self, img: np.ndarray):\n if self.waitInit:\n if self.pix_fmt in (\"nv12\", \"yuv420p\", \"yuvj420p\"):\n height_15, width = img.shape[:2]\n assert width % 2 == 0 and height_15 * 2 % 3 == 0\n height = int(height_15 / 1.5)\n else:\n height, width = img.shape[:2]\n self.width, self.height = width, height\n self.in_numpy_shape = img.shape\n self.size = (width, height)\n self.resize = self.size if self.resize is None else tuple(self.resize)\n self._init_video_stream()\n self.waitInit = False\n\n self.iframe += 1\n assert self.in_numpy_shape == img.shape\n img = img.astype(np.uint8).tobytes()\n self.process.stdin.write(img)\n\n stderrreadable, _, _ = select.select([self.process.stderr], [], [], 0)\n if stderrreadable:\n data = self.process.stderr.read(1024)\n sys.stderr.buffer.write(data)\n\n def isOpened(self):\n return self._isopen\n\n def release(self):\n self._isopen = False\n if hasattr(self, \"process\"):\n release_process_writer(self.process)\n\n def close(self):\n return self.release()\n\n\nclass FFmpegWriterNV(FFmpegWriter):\n @staticmethod\n def VideoWriter(\n filename, codec, fps, pix_fmt, gpu, bitrate=None, resize=None, preset=None\n ):\n numGPU = get_num_NVIDIA_GPUs()\n assert numGPU\n gpu = int(gpu) % numGPU if gpu is not None else 0\n if codec is None:\n codec = \"hevc_nvenc\"\n elif not isinstance(codec, str):\n codec = \"hevc_nvenc\"\n warnings.simplefilter(\n \"\"\"\n Codec should be a string. Eg `h264`, `h264_nvenc`. \n You may used CV2.VideoWriter_fourcc, which will be ignored.\n \"\"\"\n )\n elif codec.endswith(\"_nvenc\"):\n codec = codec\n else:\n codec = codec + \"_nvenc\"\n assert codec in [\n \"hevc_nvenc\",\n \"h264_nvenc\",\n ], \"codec should be `hevc_nvenc` or `h264_nvenc`\"\n assert resize is None or len(resize) == 2\n\n vid = FFmpegWriterNV()\n vid.fps = fps\n vid.codec, vid.pix_fmt, vid.filename = codec, pix_fmt, filename\n vid.gpu = gpu\n vid.bitrate = bitrate\n vid.resize = resize\n vid.preset = preset if preset is not None else (\"default\" if IN_COLAB else \"fast\")\n return vid\n\n def _init_video_stream(self):\n bitrate_str = f\"-b:v {self.bitrate} \" if self.bitrate else \"\"\n rtsp_str = f\"-f rtsp\" if self.filename.startswith(\"rtsp://\") else \"\"\n filter_str = (\n \"\"\n if self.resize == self.size\n else f\"-vf scale={self.resize[0]}:{self.resize[1]}\"\n )\n self.ffmpeg_cmd = (\n f\"ffmpeg -y -loglevel error \"\n f\"-f rawvideo -pix_fmt {self.pix_fmt} -s {self.width}x{self.height} -r {self.fps} -i pipe: \"\n f\"-preset {self.preset} {bitrate_str} \"\n f\"-r {self.fps} -gpu {self.gpu} -c:v {self.codec} \"\n f\"{filter_str} {rtsp_str} \"\n f'-pix_fmt yuv420p \"{self.filename}\"'\n )\n self.process = run_async(self.ffmpeg_cmd)\n"
},
{
"path": "ffmpegcv/ffmpeg_writer_noblock.py",
"content": "from multiprocessing import Queue, Process, Array\nimport numpy as np\nfrom .ffmpeg_writer import FFmpegWriter\n\nNFRAME = 10\n\nclass FFmpegWriterNoblock(FFmpegWriter):\n def __init__(self,\n vwriter_fun,\n *vwriter_args, **vwriter_kwargs):\n super().__init__()\n vid:FFmpegWriter = vwriter_fun(*vwriter_args, **vwriter_kwargs)\n vid.release()\n\n props_name = ['width', 'height', 'fps', 'codec', 'pix_fmt',\n 'filename', 'size', 'bitrate']\n for name in props_name:\n setattr(self, name, getattr(vid, name, None))\n\n self.vwriter_fun = vwriter_fun\n self.vwriter_args = vwriter_args\n self.vwriter_kwargs = vwriter_kwargs\n self.q = Queue(maxsize=(NFRAME-2)) #buffer index, gluttonous snake NO biting its own tail\n self.waitInit = True\n self.process = None\n\n def write(self, img:np.ndarray):\n if self.waitInit:\n if self.size is None:\n self.size = (img.shape[1], img.shape[0])\n else:\n assert tuple(self.size) == (img.shape[1], img.shape[0])\n self.in_numpy_shape = img.shape\n self._init_share_array()\n process = Process(target=child_process, \n args=(self.shared_array, self.q, self.in_numpy_shape,\n self.vwriter_fun, self.vwriter_args, self.vwriter_kwargs))\n process.start()\n self.process = process\n self.width, self.height = self.size\n self.waitInit = False\n\n self.iframe += 1\n data_id = self.iframe % NFRAME\n self.np_array[data_id] = img\n self.q.put(data_id)\n\n def _init_share_array(self):\n self.shared_array = Array('b', int(NFRAME*np.prod(self.in_numpy_shape)))\n self.np_array = np.frombuffer(self.shared_array.get_obj(), dtype=np.uint8).reshape((NFRAME,*self.in_numpy_shape))\n\n def release(self):\n if self.process is not None and self.process.is_alive():\n self.q.put(None)\n self.process.join()\n\n\ndef child_process(shared_array, q:Queue, in_numpy_shape, vwriter_fun, vwriter_args, vwriter_kwargs):\n vid = vwriter_fun(*vwriter_args, **vwriter_kwargs)\n np_array = np.frombuffer(shared_array.get_obj(), dtype=np.uint8).reshape((NFRAME,*in_numpy_shape))\n with vid:\n while True:\n data_id = q.get()\n if data_id is None:\n break\n else:\n img = np_array[data_id]\n vid.write(img)\n"
},
{
"path": "ffmpegcv/ffmpeg_writer_qsv.py",
"content": "import warnings\nfrom ffmpegcv.ffmpeg_writer import FFmpegWriter\nfrom .video_info import (\n run_async,\n get_num_QSV_GPUs,\n decoder_to_qsv,\n)\n\n\nclass FFmpegWriterQSV(FFmpegWriter):\n @staticmethod\n def VideoWriter(filename, codec, fps, pix_fmt, gpu, bitrate=None, resize=None, preset=None):\n assert gpu is None or gpu == 0, 'Cannot use multiple QSV gpu yet.'\n numGPU = get_num_QSV_GPUs()\n assert numGPU\n gpu = int(gpu) % numGPU if gpu is not None else 0\n if codec is None:\n codec = \"hevc_qsv\"\n elif not isinstance(codec, str):\n codec = \"hevc_qsv\"\n warnings.simplefilter(\n \"\"\"\n Codec should be a string. Eg `h264`, `hevc`. \n You may used CV2.VideoWriter_fourcc, which will be ignored.\n \"\"\"\n )\n else:\n codec = decoder_to_qsv(codec)\n assert resize is None or len(resize) == 2\n\n vid = FFmpegWriterQSV()\n vid.fps = fps\n vid.codec, vid.pix_fmt, vid.filename = codec, pix_fmt, filename\n vid.gpu = gpu\n vid.bitrate = bitrate\n vid.resize = resize\n vid.preset = preset\n return vid\n"
},
{
"path": "ffmpegcv/ffmpeg_writer_stream_realtime.py",
"content": "from .video_info import run_async\nfrom ffmpegcv.ffmpeg_writer import FFmpegWriter\n\n\nclass FFmpegWriterStreamRT(FFmpegWriter):\n @staticmethod\n def VideoWriter(\n filename: str, codec, pix_fmt, bitrate=None, resize=None, preset=None\n ) -> FFmpegWriter:\n assert codec in [\"h264\", \"libx264\", \"x264\", \"mpeg4\"]\n assert pix_fmt in [\"bgr24\", \"rgb24\", \"gray\"]\n # assert filename.startswith('rtmp://'), 'currently only support rtmp'\n assert resize is None or len(resize) == 2\n vid = FFmpegWriterStreamRT()\n vid.filename = filename\n vid.codec = codec\n vid.pix_fmt = pix_fmt\n vid.bitrate = bitrate\n vid.resize = resize\n if preset is not None:\n print(\"Preset is auto configured in FFmpegWriterStreamRT\")\n vid.preset = \"ultrafast\"\n return vid\n\n def _init_video_stream(self):\n bitrate_str = f\"-b:v {self.bitrate} \" if self.bitrate else \"\"\n rtsp_str = f\"-f rtsp\" if self.filename.startswith(\"rtsp://\") else \"\"\n filter_str = (\n \"\"\n if self.resize == self.size\n else f\"-vf scale={self.resize[0]}:{self.resize[1]}\"\n )\n self.ffmpeg_cmd = (\n f\"ffmpeg -loglevel warning \"\n f\"-f rawvideo -pix_fmt {self.pix_fmt} -s {self.width}x{self.height} -i pipe: \"\n f\"{bitrate_str} -f flv -rtsp_transport tcp \"\n f\" -tune zerolatency -preset {self.preset} \"\n f\"{filter_str} {rtsp_str} \"\n f' -c:v {self.codec} -g 50 -pix_fmt yuv420p \"{self.filename}\"'\n )\n self.process = run_async(self.ffmpeg_cmd)\n"
},
{
"path": "ffmpegcv/stream_info.py",
"content": "import subprocess\nfrom collections import namedtuple\nimport json\nimport shlex\n\n\ndef get_info(stream_url, timeout=None, duration_ms: int = 100):\n rtsp_opt = '' if not stream_url.startswith('rtsp://') else '-rtsp_flags prefer_tcp -pkt_size 736 '\n analyze_duration = f'-analyzeduration {duration_ms * 1000}'\n cmd = (f'ffprobe -v quiet -print_format json=compact=1 {rtsp_opt} {analyze_duration} '\n f'-select_streams v:0 -show_format -show_streams \"{stream_url}\"')\n output = subprocess.check_output(shlex.split(cmd), shell=False, timeout=timeout)\n data: dict = json.loads(output)\n vinfo: dict = data['streams'][0]\n\n StreamInfo = namedtuple(\n \"StreamInfo\", [\"width\", \"height\", \"fps\", \"count\", \"codec\", \"duration\"]\n )\n outinfo = dict()\n outinfo[\"width\"] = int(vinfo[\"width\"])\n outinfo[\"height\"] = int(vinfo[\"height\"])\n if 'avg_frame_rate' in vinfo and vinfo['avg_frame_rate']!= '0/0':\n outinfo[\"fps\"] = eval(vinfo[\"avg_frame_rate\"])\n else:\n outinfo[\"fps\"] = eval(vinfo[\"r_frame_rate\"])\n outinfo[\"count\"] = None\n outinfo[\"codec\"] = vinfo[\"codec_name\"]\n outinfo[\"duration\"] = None\n streaminfo = StreamInfo(**outinfo)\n\n return streaminfo\n\n\nif __name__ == \"__main__\":\n stream_url = \"http://devimages.apple.com.edgekey.net/streaming/examples/bipbop_4x3/gear2/prog_index.m3u8\"\n streaminfo = get_info(stream_url)\n print(streaminfo)\n"
},
{
"path": "ffmpegcv/version.py",
"content": "__version__='0.3.19'"
},
{
"path": "ffmpegcv/video_info.py",
"content": "import subprocess\nfrom subprocess import Popen, PIPE\nimport re\nfrom collections import namedtuple\nimport json\nimport shlex\nimport platform\n\nscan_the_whole = {\"mkv\", \"flv\", \"ts\"} # scan the whole file to the count, slow\n\n_is_windows = platform.system() == \"Windows\"\n_inited_get_num_NVIDIA_GPUs = False\n_inited_get_num_QSV_GPUs = False\n_num_NVIDIA_GPUs = -1\n_num_QSV_GPUs = -1\n\n\ndef get_info(video: str):\n do_scan_the_whole = video.split(\".\")[-1] in scan_the_whole\n\n def ffprobe_info_(do_scan_the_whole):\n use_count_packets = '-count_packets' if do_scan_the_whole else ''\n cmd = 'ffprobe -v quiet -print_format json=compact=1 -select_streams v:0 {} -show_streams \"{}\"'.format(\n use_count_packets, video)\n\n output = subprocess.check_output(shlex.split(cmd), shell=False)\n data: dict = json.loads(output)\n vinfo: dict = data['streams'][0]\n return vinfo\n\n vinfo = ffprobe_info_(do_scan_the_whole)\n\n if \"nb_frames\" not in vinfo:\n do_scan_the_whole = True\n vinfo = ffprobe_info_(do_scan_the_whole)\n\n # VideoInfo = namedtuple(\n # \"VideoInfo\", [\"width\", \"height\", \"fps\", \"count\", \"codec\", \"duration\", \"pix_fmt\"]\n # )\n VideoInfo = namedtuple(\n \"VideoInfo\", [\"width\", \"height\", \"fps\", \"count\", \"codec\", \"duration\"]\n )\n outinfo = dict()\n outinfo[\"width\"] = int(vinfo[\"width\"])\n outinfo[\"height\"] = int(vinfo[\"height\"])\n outinfo[\"fps\"] = eval(vinfo[\"r_frame_rate\"])\n outinfo[\"count\"] = int(\n vinfo[\"nb_read_packets\" if do_scan_the_whole else \"nb_frames\"]\n ) # nb_read_packets | nb_frames\n outinfo[\"codec\"] = vinfo[\"codec_name\"]\n # outinfo['pix_fmt'] = vinfo['pix_fmt']\n\n outinfo[\"duration\"] = (\n float(vinfo[\"duration\"])\n if \"duration\" in vinfo\n else outinfo[\"count\"] / outinfo[\"fps\"]\n )\n videoinfo = VideoInfo(**outinfo)\n\n return videoinfo\n\n\ndef get_info_precise(video: str):\n videoinfo = get_info(video)\n cmd = (\n \"ffprobe -v error -select_streams v:0 -show_entries frame=pts_time \"\n f' -of default=noprint_wrappers=1:nokey=1 -read_intervals 0%+#1,99999% \"{video}\"'\n )\n output = subprocess.check_output(\n shlex.split(cmd), shell=False, stderr=subprocess.DEVNULL\n )\n pts_start, *_, pts_end = output.decode().split()\n pts_start, pts_end = float(pts_start), float(pts_end)\n videoinfod = videoinfo._asdict()\n duration_ = pts_end - pts_start\n videoinfod[\"fps\"] = round((videoinfo.count - 1) / duration_, 3)\n videoinfod[\"duration\"] = round(duration_ + 1 / videoinfod[\"fps\"], 3)\n videoinfo_precise = videoinfo.__class__(*videoinfod.values())\n return videoinfo_precise\n\n\ndef get_num_NVIDIA_GPUs():\n global _num_NVIDIA_GPUs, _inited_get_num_NVIDIA_GPUs\n if not _inited_get_num_NVIDIA_GPUs:\n cmd = \"ffmpeg -f lavfi -i nullsrc -c:v h264_nvenc -gpu list -f null -\"\n p = Popen(cmd.split(), shell=False, stdin=PIPE, stdout=PIPE, stderr=PIPE)\n stdout, stderr = p.communicate(b\"\")\n p.stdin.close()\n p.stdout.close()\n p.terminate()\n pattern = re.compile(r\"GPU #\\d+ - < \")\n nv_info = pattern.findall(stderr.decode())\n _num_NVIDIA_GPUs = len(nv_info)\n _inited_get_num_NVIDIA_GPUs = True\n return _num_NVIDIA_GPUs\n\n\ndef get_num_QSV_GPUs():\n global _num_QSV_GPUs, _inited_get_num_QSV_GPUs\n if not _inited_get_num_QSV_GPUs:\n cmd = \"ffmpeg -hide_banner -f qsv -h encoder=h264_qsv\"\n p = Popen(cmd.split(), shell=False, stdin=PIPE, stdout=PIPE, stderr=PIPE)\n stdout, stderr = p.communicate(b\"\")\n _num_QSV_GPUs = 1 if len(stdout) > 50 else 0\n _inited_get_num_QSV_GPUs = True\n return _num_QSV_GPUs\n\n\ndef encoder_to_nvidia(codec):\n codec_map = {\"h264\": \"h264_nvenc\", \"hevc\": \"hevc_nvenc\"}\n\n if codec in codec_map:\n return codec_map[codec]\n elif codec in codec_map.values():\n return codec\n else:\n raise Exception(\"No NV codec found for %s\" % codec)\n\n\ndef encoder_to_qsv(codec):\n codec_map = {\n \"h264\": \"h264_qsv\",\n \"hevc\": \"hevc_qsv\",\n \"mjpeg\": \"mjpeg_qsv\",\n \"mpeg2video\": \"mpeg2_qsv\",\n \"vp9\": \"vp9_qsv\",\n }\n\n if codec in codec_map:\n return codec_map[codec]\n elif codec in codec_map.values():\n return codec\n else:\n raise Exception(\"No QSV codec found for %s\" % codec)\n\n\ndef decoder_to_nvidia(codec):\n codec_map = {\n \"av1\": \"av1_cuvid\",\n \"h264\": \"h264_cuvid\",\n \"x264\": \"h264_cuvid\",\n \"hevc\": \"hevc_cuvid\",\n \"x265\": \"hevc_cuvid\",\n \"h265\": \"hevc_cuvid\",\n \"mjpeg\": \"mjpeg_cuvid\",\n \"mpeg1video\": \"mpeg1_cuvid\",\n \"mpeg2video\": \"mpeg2_cuvid\",\n \"mpeg4\": \"mpeg4_cuvid\",\n \"vp1\": \"vp1_cuvid\",\n \"vp8\": \"vp8_cuvid\",\n \"vp9\": \"vp9_cuvid\",\n }\n\n if codec in codec_map:\n return codec_map[codec]\n elif codec in codec_map.values():\n return codec\n else:\n raise Exception(\"No NV codec found for %s\" % codec)\n\n\ndef decoder_to_qsv(codec):\n codec_map = {\n \"av1\": \"av1_qsv\",\n \"h264\": \"h264_qsv\",\n \"hevc\": \"hevc_qsv\",\n \"mjpeg\": \"mjpeg_qsv\",\n \"mpeg2video\": \"mpeg2_qsv\",\n \"vc1\": \"vc1_qsv\",\n \"vp8\": \"vp8_qsv\",\n \"vp9\": \"vp9_qsv\",\n }\n\n if codec in codec_map:\n return codec_map[codec]\n elif codec in codec_map.values():\n return codec\n else:\n raise Exception(\"No QSV codec found for %s\" % codec)\n\n\ndef run_async(args):\n bufsize = -1\n if isinstance(args, str):\n args = shlex.split(args)\n return Popen(\n args,\n stdin=PIPE,\n stdout=PIPE,\n stderr=PIPE,\n shell=False,\n bufsize=bufsize,\n )\n\ndef run_async_reader(args):\n bufsize = -1\n if isinstance(args, str):\n args = shlex.split(args)\n\n return Popen(\n args,\n stdin=None,\n stdout=PIPE,\n stderr=subprocess.DEVNULL,\n shell=False,\n bufsize=bufsize,\n )\n\n\ndef release_process(process: Popen, forcekill=False):\n if hasattr(process, \"stdin\") and process.stdin is not None:\n process.stdin.close()\n if hasattr(process, \"stdout\") and process.stdout is not None:\n process.stdout.close()\n if hasattr(process, \"stderr\") and process.stderr is not None:\n process.stderr.close()\n if forcekill and hasattr(process, \"terminate\") and not _is_windows:\n process.terminate()\n if forcekill and hasattr(process, \"wait\"):\n process.wait()\n\n\ndef release_process_writer(process: Popen):\n if hasattr(process, \"stdin\"):\n process.stdin.close()\n if hasattr(process, \"stdout\"):\n process.stdout.close()\n if hasattr(process, \"wait\"):\n process.wait()\n"
},
{
"path": "setup.py",
"content": "# setup.py\nfrom setuptools import setup, find_packages\nfrom pathlib import Path\n\nthis_directory = Path(__file__).parent\nlong_description = (this_directory / \"README.md\").read_text(\"utf-8\")\n\nsetup(\n name=\"ffmpegcv\", # 应用名\n version=\"0.3.18\", # 版本号\n packages=find_packages(include=[\"ffmpegcv*\"]), # 包括在安装包内的 Python 包\n author=\"chenxf\",\n author_email=\"cxf529125853@163.com\",\n url=\"https://github.com/chenxinfeng4/ffmpegcv\",\n long_description=long_description,\n long_description_content_type=\"text/markdown\",\n # 添加依赖项\n python_requires=\">=3.6\",\n install_requires=[\n \"numpy\",\n ],\n extras_require={\"cuda\": [\"pycuda\"]}, # 定义一个名为cuda的可选依赖项，并指定pycuda\n)\n"
}
]