, 1 April 1989
Ty Coon, President of Vice
This General Public License does not permit incorporating your program into
proprietary programs. If your program is a subroutine library, you may
consider it more useful to permit linking proprietary applications with the
library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License.
"CLASSPATH" EXCEPTION TO THE GPL
Linking this library statically or dynamically with other modules is making
a combined work based on this library. Thus, the terms and conditions of
the GNU General Public License cover the whole combination.
As a special exception, the copyright holders of this library give you
permission to link this library with independent modules to produce an
executable, regardless of the license terms of these independent modules,
and to copy and distribute the resulting executable under terms of your
choice, provided that you also meet, for each linked independent module,
the terms and conditions of the license of that module. An independent
module is a module which is not derived from or based on this library. If
you modify this library, you may extend this exception to your version of
the library, but you are not obligated to do so. If you do not wish to do
so, delete this exception statement from your version.
================================================
FILE: README.md
================================================
JavaCV
======
[](https://gitter.im/bytedeco/javacv) [](https://maven-badges.herokuapp.com/maven-central/org.bytedeco/javacv-platform) [](http://bytedeco.org/builds/) [](https://travis-ci.org/bytedeco/javacv) Commercial support: [](https://xscode.com/bytedeco/javacv)
Introduction
------------
JavaCV uses wrappers from the [JavaCPP Presets](https://github.com/bytedeco/javacpp-presets) of commonly used libraries by researchers in the field of computer vision ([OpenCV](http://opencv.org/), [FFmpeg](http://ffmpeg.org/), [libdc1394](http://damien.douxchamps.net/ieee1394/libdc1394/), [FlyCapture](https://www.flir.com/products/flycapture-sdk/), [Spinnaker](https://www.flir.com/products/spinnaker-sdk/), [OpenKinect](http://openkinect.org/), [librealsense](https://github.com/IntelRealSense/librealsense), [CL PS3 Eye Driver](https://codelaboratories.com/downloads/), [videoInput](http://muonics.net/school/spring05/videoInput/), [ARToolKitPlus](https://launchpad.net/artoolkitplus), [flandmark](https://github.com/uricamic/flandmark), [Leptonica](http://www.leptonica.org/), and [Tesseract](https://github.com/tesseract-ocr/tesseract)) and provides utility classes to make their functionality easier to use on the Java platform, including Android.
JavaCV also comes with hardware accelerated full-screen image display (`CanvasFrame` and `GLCanvasFrame`), easy-to-use methods to execute code in parallel on multiple cores (`Parallel`), user-friendly geometric and color calibration of cameras and projectors (`GeometricCalibrator`, `ProCamGeometricCalibrator`, `ProCamColorCalibrator`), detection and matching of feature points (`ObjectFinder`), a set of classes that implement direct image alignment of projector-camera systems (mainly `GNImageAligner`, `ProjectiveTransformer`, `ProjectiveColorTransformer`, `ProCamTransformer`, and `ReflectanceInitializer`), a blob analysis package (`Blobs`), as well as miscellaneous functionality in the `JavaCV` class. Some of these classes also have an OpenCL and OpenGL counterpart, their names ending with `CL` or starting with `GL`, i.e.: `JavaCVCL`, `GLCanvasFrame`, etc.
To learn how to use the API, since documentation currently lacks, please refer to the [Sample Usage](#sample-usage) section below as well as the [sample programs](https://github.com/bytedeco/javacv/tree/master/samples/), including two for Android (`FacePreview.java` and `RecordActivity.java`), also found in the `samples` directory. You may also find it useful to refer to the source code of [ProCamCalib](https://github.com/bytedeco/procamcalib) and [ProCamTracker](https://github.com/bytedeco/procamtracker) as well as [examples ported from OpenCV2 Cookbook](https://github.com/bytedeco/javacv-examples/) and the associated [wiki pages](https://github.com/bytedeco/javacv-examples/tree/master/OpenCV_Cookbook).
Please keep me informed of any updates or fixes you make to the code so that I may integrate them into the next release. Thank you! And feel free to ask questions on [the mailing list](http://groups.google.com/group/javacv) or [the discussion forum](https://github.com/bytedeco/javacv/discussions) if you encounter any problems with the software! I am sure it is far from perfect...
Downloads
---------
Archives containing JAR files are available as [releases](https://github.com/bytedeco/javacv/releases). The binary archive contains builds for Android, iOS, Linux, Mac OS X, and Windows. The JAR files for specific child modules or platforms can also be obtained individually from the [Maven Central Repository](http://search.maven.org/#search|ga|1|bytedeco).
To install manually the JAR files, follow the instructions in the [Manual Installation](#manual-installation) section below.
We can also have everything downloaded and installed automatically with:
* Maven (inside the `pom.xml` file)
```xml
org.bytedeco
javacv-platform
1.5.13
```
* Gradle (inside the `build.gradle.kts` or `build.gradle` file)
```groovy
dependencies {
implementation("org.bytedeco:javacv-platform:1.5.13")
}
```
* Leiningen (inside the `project.clj` file)
```clojure
:dependencies [
[org.bytedeco/javacv-platform "1.5.13"]
]
```
* sbt (inside the `build.sbt` file)
```scala
libraryDependencies += "org.bytedeco" % "javacv-platform" % "1.5.13"
```
This downloads binaries for all platforms, but to get binaries for only one platform we can set the `javacpp.platform` system property (via the `-D` command line option) to something like `android-arm`, `linux-x86_64`, `macosx-x86_64`, `windows-x86_64`, etc. Please refer to the [README.md file of the JavaCPP Presets](https://github.com/bytedeco/javacpp-presets#downloads) for details. Another option available to Gradle users is [Gradle JavaCPP](https://github.com/bytedeco/gradle-javacpp), and similarly for Scala users there is [SBT-JavaCV](https://github.com/bytedeco/sbt-javacv).
Required Software
-----------------
To use JavaCV, you will first need to download and install the following software:
* An implementation of Java SE 8 or newer:
* OpenJDK http://openjdk.java.net/install/ or
* Oracle JDK http://www.oracle.com/technetwork/java/javase/downloads/ or
* IBM JDK http://www.ibm.com/developerworks/java/jdk/ or
* Microsoft JDK https://www.microsoft.com/openjdk etc
Further, although not always required, some functionality of JavaCV also relies on:
* CL Eye Platform SDK (Windows only) http://codelaboratories.com/downloads/
* Android SDK API 24 or newer http://developer.android.com/sdk/
* JOCL and JOGL from JogAmp http://jogamp.org/
Finally, please make sure everything has the same bitness: **32-bit and 64-bit modules do not mix under any circumstances**.
Manual Installation
-------------------
Simply put all the desired JAR files (`opencv*.jar`, `ffmpeg*.jar`, etc.), in addition to `javacpp.jar` and `javacv.jar`, somewhere in your class path. Here are some more specific instructions for common cases:
NetBeans (Java SE 8 or newer):
1. In the Projects window, right-click the Libraries node of your project, and select "Add JAR/Folder...".
2. Locate the JAR files, select them, and click OK.
Eclipse (Java SE 8 or newer):
1. Navigate to Project > Properties > Java Build Path > Libraries and click "Add External JARs...".
2. Locate the JAR files, select them, and click OK.
Visual Studio Code (Java SE 8 or newer):
1. Navigate to Java Projects > Referenced Libraries, and click `+`.
2. Locate the JAR files, select them, and click OK.
IntelliJ IDEA (Android 7.0 or newer):
1. Follow the instructions on this page: http://developer.android.com/training/basics/firstapp/
2. Copy all the JAR files into the `app/libs` subdirectory.
3. Navigate to File > Project Structure > app > Dependencies, click `+`, and select "2 File dependency".
4. Select all the JAR files from the `libs` subdirectory.
5. In the AndroidManifest.xml add `android:extractNativeLibs="true"`
After that, the wrapper classes for OpenCV and FFmpeg, for example, can automatically access all of their C/C++ APIs:
* [OpenCV documentation](http://docs.opencv.org/master/)
* [FFmpeg documentation](http://ffmpeg.org/doxygen/trunk/)
Sample Usage
------------
The class definitions are basically ports to Java of the original header files in C/C++, and I deliberately decided to keep as much of the original syntax as possible. For example, here is a method that tries to load an image file, smooth it, and save it back to disk:
```java
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_imgcodecs.*;
public class Smoother {
public static void smooth(String filename) {
Mat image = imread(filename);
if (image != null) {
GaussianBlur(image, image, new Size(3, 3), 0);
imwrite(filename, image);
}
}
}
```
JavaCV also comes with helper classes and methods on top of OpenCV and FFmpeg to facilitate their integration to the Java platform. Here is a small demo program demonstrating the most frequently useful parts:
```java
import java.io.File;
import java.net.URL;
import org.bytedeco.javacv.*;
import org.bytedeco.javacpp.*;
import org.bytedeco.javacpp.indexer.*;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import org.bytedeco.opencv.opencv_calib3d.*;
import org.bytedeco.opencv.opencv_objdetect.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_calib3d.*;
import static org.bytedeco.opencv.global.opencv_objdetect.*;
public class Demo {
public static void main(String[] args) throws Exception {
String classifierName = null;
if (args.length > 0) {
classifierName = args[0];
} else {
URL url = new URL("https://raw.github.com/opencv/opencv/master/data/haarcascades/haarcascade_frontalface_alt.xml");
File file = Loader.cacheResource(url);
classifierName = file.getAbsolutePath();
}
// We can "cast" Pointer objects by instantiating a new object of the desired class.
CascadeClassifier classifier = new CascadeClassifier(classifierName);
if (classifier == null) {
System.err.println("Error loading classifier file \"" + classifierName + "\".");
System.exit(1);
}
// The available FrameGrabber classes include OpenCVFrameGrabber (opencv_videoio),
// DC1394FrameGrabber, FlyCapture2FrameGrabber, OpenKinectFrameGrabber, OpenKinect2FrameGrabber,
// RealSenseFrameGrabber, RealSense2FrameGrabber, PS3EyeFrameGrabber, VideoInputFrameGrabber, and FFmpegFrameGrabber.
FrameGrabber grabber = FrameGrabber.createDefault(0);
grabber.start();
// CanvasFrame, FrameGrabber, and FrameRecorder use Frame objects to communicate image data.
// We need a FrameConverter to interface with other APIs (Android, Java 2D, JavaFX, Tesseract, OpenCV, etc).
OpenCVFrameConverter.ToMat converter = new OpenCVFrameConverter.ToMat();
// FAQ about IplImage and Mat objects from OpenCV:
// - For custom raw processing of data, createBuffer() returns an NIO direct
// buffer wrapped around the memory pointed by imageData, and under Android we can
// also use that Buffer with Bitmap.copyPixelsFromBuffer() and copyPixelsToBuffer().
// - To get a BufferedImage from an IplImage, or vice versa, we can chain calls to
// Java2DFrameConverter and OpenCVFrameConverter, one after the other.
// - Java2DFrameConverter also has static copy() methods that we can use to transfer
// data more directly between BufferedImage and IplImage or Mat via Frame objects.
Mat grabbedImage = converter.convert(grabber.grab());
int height = grabbedImage.rows();
int width = grabbedImage.cols();
// Objects allocated with `new`, clone(), or a create*() factory method are automatically released
// by the garbage collector, but may still be explicitly released by calling deallocate().
// You shall NOT call cvReleaseImage(), cvReleaseMemStorage(), etc. on objects allocated this way.
Mat grayImage = new Mat(height, width, CV_8UC1);
Mat rotatedImage = grabbedImage.clone();
// The OpenCVFrameRecorder class simply uses the VideoWriter of opencv_videoio,
// but FFmpegFrameRecorder also exists as a more versatile alternative.
FrameRecorder recorder = FrameRecorder.createDefault("output.avi", width, height);
recorder.start();
// CanvasFrame is a JFrame containing a Canvas component, which is hardware accelerated.
// It can also switch into full-screen mode when called with a screenNumber.
// We should also specify the relative monitor/camera response for proper gamma correction.
CanvasFrame frame = new CanvasFrame("Some Title", CanvasFrame.getDefaultGamma()/grabber.getGamma());
// Let's create some random 3D rotation...
Mat randomR = new Mat(3, 3, CV_64FC1),
randomAxis = new Mat(3, 1, CV_64FC1);
// We can easily and efficiently access the elements of matrices and images
// through an Indexer object with the set of get() and put() methods.
DoubleIndexer Ridx = randomR.createIndexer(),
axisIdx = randomAxis.createIndexer();
axisIdx.put(0, (Math.random() - 0.5) / 4,
(Math.random() - 0.5) / 4,
(Math.random() - 0.5) / 4);
Rodrigues(randomAxis, randomR);
double f = (width + height) / 2.0; Ridx.put(0, 2, Ridx.get(0, 2) * f);
Ridx.put(1, 2, Ridx.get(1, 2) * f);
Ridx.put(2, 0, Ridx.get(2, 0) / f); Ridx.put(2, 1, Ridx.get(2, 1) / f);
System.out.println(Ridx);
// We can allocate native arrays using constructors taking an integer as argument.
Point hatPoints = new Point(3);
while (frame.isVisible() && (grabbedImage = converter.convert(grabber.grab())) != null) {
// Let's try to detect some faces! but we need a grayscale image...
cvtColor(grabbedImage, grayImage, CV_BGR2GRAY);
RectVector faces = new RectVector();
classifier.detectMultiScale(grayImage, faces);
long total = faces.size();
for (long i = 0; i < total; i++) {
Rect r = faces.get(i);
int x = r.x(), y = r.y(), w = r.width(), h = r.height();
rectangle(grabbedImage, new Point(x, y), new Point(x + w, y + h), Scalar.RED, 1, CV_AA, 0);
// To access or pass as argument the elements of a native array, call position() before.
hatPoints.position(0).x(x - w / 10 ).y(y - h / 10);
hatPoints.position(1).x(x + w * 11 / 10).y(y - h / 10);
hatPoints.position(2).x(x + w / 2 ).y(y - h / 2 );
fillConvexPoly(grabbedImage, hatPoints.position(0), 3, Scalar.GREEN, CV_AA, 0);
}
// Let's find some contours! but first some thresholding...
threshold(grayImage, grayImage, 64, 255, CV_THRESH_BINARY);
// To check if an output argument is null we may call either isNull() or equals(null).
MatVector contours = new MatVector();
findContours(grayImage, contours, CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE);
long n = contours.size();
for (long i = 0; i < n; i++) {
Mat contour = contours.get(i);
Mat points = new Mat();
approxPolyDP(contour, points, arcLength(contour, true) * 0.02, true);
drawContours(grabbedImage, new MatVector(points), -1, Scalar.BLUE);
}
warpPerspective(grabbedImage, rotatedImage, randomR, rotatedImage.size());
Frame rotatedFrame = converter.convert(rotatedImage);
frame.showImage(rotatedFrame);
recorder.record(rotatedFrame);
}
frame.dispose();
recorder.stop();
grabber.stop();
}
}
```
Furthermore, after creating a `pom.xml` file with the following content:
```xml
4.0.0
org.bytedeco.javacv
demo
1.5.13
1.8
1.8
org.bytedeco
javacv-platform
1.5.13
org.bytedeco
opencv-platform-gpu
4.13.0-1.5.13
org.bytedeco
ffmpeg-platform-gpl
8.0.1-1.5.13
.
```
And by placing the source code above in `Demo.java`, or similarly for other classes found in the [`samples`](samples), we can use the following command to have everything first installed automatically and then executed by Maven:
```bash
$ mvn compile exec:java -Dexec.mainClass=Demo
```
**Note**: In case of errors, please make sure that the `artifactId` in the `pom.xml` file reads `javacv-platform`, not `javacv` only, for example. The artifact `javacv-platform` adds all the necessary binary dependencies.
Build Instructions
------------------
If the binary files available above are not enough for your needs, you might need to rebuild them from the source code. To this end, the project files were created for:
* Maven 3.x http://maven.apache.org/download.html
* JavaCPP 1.5.13 https://github.com/bytedeco/javacpp
* JavaCPP Presets 1.5.13 https://github.com/bytedeco/javacpp-presets
Once installed, simply call the usual `mvn install` command for JavaCPP, its Presets, and JavaCV. By default, no other dependencies than a C++ compiler for JavaCPP are required. Please refer to the comments inside the `pom.xml` files for further details.
Instead of building the native libraries manually, we can run `mvn install` for JavaCV only and rely on the snapshot artifacts from the CI builds:
* http://bytedeco.org/builds/
----
Project lead: Samuel Audet [samuel.audet `at` gmail.com](mailto:samuel.audet at gmail.com)
Developer site: https://github.com/bytedeco/javacv
Discussion group: http://groups.google.com/group/javacv
================================================
FILE: platform/pom.xml
================================================
4.0.0
org.bytedeco
javacpp-presets
1.5.14-SNAPSHOT
org.bytedeco
javacv-platform
JavaCV Platform
${project.version}
org.bytedeco
javacv
${project.version}
org.bytedeco
javacpp-platform
${javacpp.version}
org.bytedeco
openblas-platform
0.3.31-${javacpp.version}
org.bytedeco
opencv-platform
4.13.0-${javacpp.version}
org.bytedeco
ffmpeg-platform
8.0.1-${javacpp.version}
org.bytedeco
flycapture-platform
2.13.3.31-1.5.9
org.bytedeco
libdc1394-platform
2.2.6-1.5.9
org.bytedeco
libfreenect-platform
0.5.7-1.5.9
org.bytedeco
libfreenect2-platform
0.2.0-1.5.9
org.bytedeco
librealsense-platform
1.12.4-1.5.9
org.bytedeco
librealsense2-platform
2.53.1-1.5.9
org.bytedeco
videoinput-platform
0.200-1.5.9
org.bytedeco
artoolkitplus-platform
2.3.1-1.5.9
org.bytedeco
leptonica-platform
1.87.0-${javacpp.version}
org.bytedeco
tesseract-platform
5.5.2-${javacpp.version}
junit
junit
4.13.2
test
org.bytedeco
opencv-platform-gpu
4.13.0-${javacpp.version}
test
org.bytedeco
ffmpeg-platform-gpl
8.0.1-${javacpp.version}
test
central-portal-snapshots
Central Portal Snapshots
https://central.sonatype.com/repository/maven-snapshots/
false
true
maven-jar-plugin
3.5.0
default-jar
javacv.jar javacpp-platform.jar openblas-platform.jar opencv-platform.jar ffmpeg-platform.jar flycapture-platform.jar libdc1394-platform.jar libfreenect-platform.jar libfreenect2-platform.jar librealsense-platform.jar librealsense2-platform.jar videoinput-platform.jar artoolkitplus-platform.jar flandmark-platform.jar leptonica-platform.jar tesseract-platform.jar
org/bytedeco/javacv/
empty-javadoc-jar
jar
javadoc
empty-sources-jar
jar
sources
org.moditect
moditect-maven-plugin
1.3.0
9
true
${project.build.directory}
add-module-infos
package
add-module-info
${project.build.directory}/${project.artifactId}.jar
${project.basedir}/src/main/java9/module-info.java
maven-dependency-plugin
3.10.0
properties
properties
copy-dependencies
copy-dependencies
${project.build.directory}
true
maven-surefire-plugin
3.5.5
-Xmx2g
maven-assembly-plugin
3.8.0
false
src/main/assembly/bin.xml
src/main/assembly/src.xml
package
single
sign-artifacts
performRelease
true
central-portal-staging
Central Portal Staging
https://central.sonatype.com/api/v1/publisher/deployments/download/
true
false
maven-gpg-plugin
3.2.8
sign-artifacts
verify
sign
${env.GPG_PASSPHRASE}
false
================================================
FILE: platform/src/main/assembly/bin.xml
================================================
${project.version}-bin
zip
${project.artifactId}-${project.version}-bin
${project.basedir}/..
/
samples/*
CHANGELOG*
README*
LICENSE*
NOTICE*
${project.build.directory}
/
*.jar
android*.jar
gluegen*.jar
hamcrest*.jar
junit*.jar
jogl*.jar
jocl*.jar
*-javadoc.jar
*-sources.jar
0644
${project.build.directory}/site
docs
================================================
FILE: platform/src/main/assembly/src.xml
================================================
${project.version}-src
zip
${project.artifactId}-${project.version}
${project.basedir}/..
/
true
**/target/**
**/cppbuild/**
================================================
FILE: platform/src/main/java9/module-info.java
================================================
module org.bytedeco.javacv.platform {
requires transitive org.bytedeco.javacv;
requires org.bytedeco.opencv.platform;
requires org.bytedeco.ffmpeg.platform;
requires org.bytedeco.flycapture.platform;
requires org.bytedeco.libdc1394.platform;
requires org.bytedeco.libfreenect.platform;
requires org.bytedeco.libfreenect2.platform;
requires org.bytedeco.librealsense.platform;
requires org.bytedeco.librealsense2.platform;
requires org.bytedeco.videoinput.platform;
requires org.bytedeco.artoolkitplus.platform;
requires org.bytedeco.flandmark.platform;
requires org.bytedeco.leptonica.platform;
requires org.bytedeco.tesseract.platform;
}
================================================
FILE: platform/src/test/java/org/bytedeco/javacv/FrameConverterTest.java
================================================
/*
* Copyright (C) 2015-2016 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.awt.image.BufferedImage;
import java.awt.image.DataBufferByte;
import java.awt.image.DataBufferInt;
import java.awt.image.WritableRaster;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.IntBuffer;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.javacpp.indexer.Indexer;
import org.bytedeco.javacpp.indexer.UByteIndexer;
import org.junit.Test;
import org.bytedeco.leptonica.PIX;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
import static org.junit.Assert.*;
/**
* Test cases for FrameConverter classes. Also uses other classes from JavaCV.
*
* @author Samuel Audet
*/
public class FrameConverterTest {
@Test public void testAndroidFrameConverter() {
System.out.println("AndroidFrameConverter");
AndroidFrameConverter converter = new AndroidFrameConverter();
int width = 512;
int height = 1024;
byte[] yuvData = new byte[3 * width * height / 2];
for (int i = 0; i < yuvData.length; i++) {
yuvData[i] = (byte)i;
}
Mat yuvImage = new Mat(3 * height / 2, width, CV_8UC1, new BytePointer(yuvData));
Mat bgrImage = new Mat(height, width, CV_8UC3);
cvtColor(yuvImage, bgrImage, CV_YUV2BGR_NV21);
Frame bgrFrame = converter.convert(yuvData, width, height);
UByteIndexer bgrImageIdx = bgrImage.createIndexer();
UByteIndexer bgrFrameIdx = bgrFrame.createIndexer();
assertEquals(bgrImageIdx.rows(), bgrFrameIdx.rows());
assertEquals(bgrImageIdx.cols(), bgrFrameIdx.cols());
assertEquals(bgrImageIdx.channels(), bgrFrameIdx.channels());
for (int i = 0; i < bgrImageIdx.rows(); i++) {
for (int j = 0; j < bgrImageIdx.cols(); j++) {
for (int k = 0; k < bgrImageIdx.channels(); k++) {
assertEquals((float)bgrImageIdx.get(i, j, k), (float)bgrFrameIdx.get(i, j, k), 1.0f);
}
}
}
bgrImageIdx.release();
bgrFrameIdx.release();
Frame grayFrame = new Frame(1024 + 1, 768, Frame.DEPTH_UBYTE, 1);
Frame colorFrame = new Frame(640 + 1, 480, Frame.DEPTH_UBYTE, 3);
UByteIndexer grayFrameIdx = grayFrame.createIndexer();
for (int i = 0; i < grayFrameIdx.rows(); i++) {
for (int j = 0; j < grayFrameIdx.cols(); j++) {
grayFrameIdx.put(i, j, i + j);
}
}
UByteIndexer colorFrameIdx = colorFrame.createIndexer();
for (int i = 0; i < colorFrameIdx.rows(); i++) {
for (int j = 0; j < colorFrameIdx.cols(); j++) {
for (int k = 0; k < colorFrameIdx.channels(); k++) {
colorFrameIdx.put(i, j, k, i + j + k);
}
}
}
width = grayFrame.imageWidth;
height = grayFrame.imageHeight;
int stride = grayFrame.imageStride;
int rowBytes = width * 4;
ByteBuffer in = (ByteBuffer)grayFrame.image[0];
ByteBuffer buffer = converter.gray2rgba(in, width, height, stride, rowBytes);
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
// GRAY -> RGBA
byte B = in.get(y * stride + x);
assertEquals(buffer.get(y * rowBytes + 4 * x ), B);
assertEquals(buffer.get(y * rowBytes + 4 * x + 1), B);
assertEquals(buffer.get(y * rowBytes + 4 * x + 2), B);
assertEquals(buffer.get(y * rowBytes + 4 * x + 3), (byte)0xFF);
}
}
width = colorFrame.imageWidth;
height = colorFrame.imageHeight;
stride = colorFrame.imageStride;
rowBytes = width * 4;
in = (ByteBuffer)colorFrame.image[0];
buffer = converter.bgr2rgba(in, width, height, stride, rowBytes);
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
// BGR -> RGBA
byte B = in.get(y * stride + 3 * x );
byte G = in.get(y * stride + 3 * x + 1);
byte R = in.get(y * stride + 3 * x + 2);
assertEquals(buffer.get(y * rowBytes + 4 * x ), R);
assertEquals(buffer.get(y * rowBytes + 4 * x + 1), G);
assertEquals(buffer.get(y * rowBytes + 4 * x + 2), B);
assertEquals(buffer.get(y * rowBytes + 4 * x + 3), (byte)0xFF);
}
}
colorFrameIdx.release();
grayFrameIdx.release();
converter.close();
colorFrame.close();
grayFrame.close();
}
@Test public void testJava2DFrameConverter() {
System.out.println("Java2DFrameConverter");
int[] depths = {Frame.DEPTH_UBYTE, Frame.DEPTH_SHORT, Frame.DEPTH_FLOAT};
int[] channels = {1, 3, 4};
for (int i = 0; i < depths.length; i++) {
for (int j = 0; j < channels.length; j++) {
Frame frame = new Frame(640 + 1, 480, depths[i], channels[j]);
Java2DFrameConverter converter = new Java2DFrameConverter();
Indexer frameIdx = frame.createIndexer();
for (int y = 0; y < frameIdx.rows(); y++) {
for (int x = 0; x < frameIdx.cols(); x++) {
for (int z = 0; z < frameIdx.channels(); z++) {
frameIdx.putDouble(new long[] {y, x, z}, y + x + z);
}
}
}
BufferedImage image = converter.convert(frame);
converter.frame = null;
Frame frame2 = converter.convert(image);
Indexer frame2Idx = frame2.createIndexer();
for (int y = 0; y < frameIdx.rows(); y++) {
for (int x = 0; x < frameIdx.cols(); x++) {
for (int z = 0; z < frameIdx.channels(); z++) {
double value = frameIdx.getDouble(y, x, z);
assertEquals(value, frame2Idx.getDouble(y, x, z), 0);
}
}
}
try {
frame2Idx.getDouble(frameIdx.rows() + 1, frameIdx.cols() + 1);
fail("IndexOutOfBoundsException should have been thrown.");
} catch (IndexOutOfBoundsException e) { }
frameIdx.release();
frame2Idx.release();
converter.close();
frame.close();
}
}
int[] types = {BufferedImage.TYPE_INT_RGB, BufferedImage.TYPE_INT_ARGB,
BufferedImage.TYPE_INT_ARGB_PRE, BufferedImage.TYPE_INT_BGR};
for (int i = 0; i < types.length; i++) {
BufferedImage image = new BufferedImage(640 + 1, 480, types[i]);
Java2DFrameConverter converter = new Java2DFrameConverter();
WritableRaster raster = image.getRaster();
int[] array = ((DataBufferInt)raster.getDataBuffer()).getData();
for (int j = 0; j < array.length; j++) {
array[j] = j;
}
Frame frame = converter.convert(image);
converter.bufferedImage = null;
BufferedImage image2 = converter.convert(frame);
WritableRaster raster2 = image2.getRaster();
byte[] array2 = ((DataBufferByte)raster2.getDataBuffer()).getData();
for (int j = 0; j < array.length; j++) {
int n = ((array2[4 * j ] & 0xFF) << 24) | ((array2[4 * j + 1] & 0xFF) << 16)
| ((array2[4 * j + 2] & 0xFF) << 8) | (array2[4 * j + 3] & 0xFF);
assertEquals(array[j], n);
}
converter.close();
}
}
@Test public void testOpenCVFrameConverter() {
System.out.println("OpenCVFrameConverter");
Loader.load(org.bytedeco.opencv.opencv_java.class);
for (int depth = 8; depth <= 64; depth *= 2) {
assertEquals(depth, OpenCVFrameConverter.getFrameDepth(OpenCVFrameConverter.getIplImageDepth(depth)));
assertEquals(depth, OpenCVFrameConverter.getFrameDepth(OpenCVFrameConverter.getMatDepth(depth)));
if (depth < 64) {
assertEquals(-depth, OpenCVFrameConverter.getFrameDepth(OpenCVFrameConverter.getIplImageDepth(-depth)));
assertEquals(-depth, OpenCVFrameConverter.getFrameDepth(OpenCVFrameConverter.getMatDepth(-depth)));
}
}
Frame frame = new Frame(640 + 1, 480, Frame.DEPTH_UBYTE, 3);
OpenCVFrameConverter.ToIplImage converter1 = new OpenCVFrameConverter.ToIplImage();
OpenCVFrameConverter.ToMat converter2 = new OpenCVFrameConverter.ToMat();
OpenCVFrameConverter.ToOrgOpenCvCoreMat converter3 = new OpenCVFrameConverter.ToOrgOpenCvCoreMat();
UByteIndexer frameIdx = frame.createIndexer();
for (int i = 0; i < frameIdx.rows(); i++) {
for (int j = 0; j < frameIdx.cols(); j++) {
for (int k = 0; k < frameIdx.channels(); k++) {
frameIdx.put(i, j, k, i + j + k);
}
}
}
IplImage image = converter1.convert(frame);
Mat mat = converter2.convert(frame);
final org.opencv.core.Mat cvmat = converter3.convert(frame);
converter1.frame = null;
converter2.frame = null;
converter3.frame = null;
Frame frame1 = converter1.convert(image);
Frame frame2 = converter2.convert(mat);
Frame frame3 = converter3.convert(cvmat);
assertEquals(frame2.opaque, mat);
assertEquals(frame3.opaque, cvmat);
Mat mat2 = new Mat(mat.rows(), mat.cols(), mat.type(), mat.data(), mat.step());
org.opencv.core.Mat cvmat2 = new org.opencv.core.Mat(cvmat.rows(), cvmat.cols(), cvmat.type(),
new BytePointer() { { address = cvmat.dataAddr(); } }.capacity(cvmat.rows() * cvmat.cols() * cvmat.elemSize()).asByteBuffer(),
cvmat.step1() * cvmat.elemSize1());
assertNotEquals(mat, mat2);
assertNotEquals(cvmat, cvmat2);
frame2 = converter2.convert(mat2);
frame3 = converter3.convert(cvmat2);
assertEquals(frame2.opaque, mat2);
assertEquals(frame3.opaque, cvmat2);
assertEquals(frame3.imageStride, cvmat2.step1() * cvmat2.elemSize1());
UByteIndexer frame1Idx = frame1.createIndexer();
UByteIndexer frame2Idx = frame2.createIndexer();
UByteIndexer frame3Idx = frame3.createIndexer();
for (int i = 0; i < frameIdx.rows(); i++) {
for (int j = 0; j < frameIdx.cols(); j++) {
for (int k = 0; k < frameIdx.channels(); k++) {
int b = frameIdx.get(i, j, k);
assertEquals(b, frame1Idx.get(i, j, k));
assertEquals(b, frame2Idx.get(i, j, k));
assertEquals(b, frame3Idx.get(i, j, k));
}
}
}
try {
frame1Idx.get(frameIdx.rows() + 1, frameIdx.cols() + 1);
fail("IndexOutOfBoundsException should have been thrown.");
} catch (IndexOutOfBoundsException e) { }
try {
frame2Idx.get(frameIdx.rows() + 1, frameIdx.cols() + 1);
fail("IndexOutOfBoundsException should have been thrown.");
} catch (IndexOutOfBoundsException e) { }
try {
frame3Idx.get(frameIdx.rows() + 1, frameIdx.cols() + 1);
fail("IndexOutOfBoundsException should have been thrown.");
} catch (IndexOutOfBoundsException e) { }
frameIdx.release();
frame1Idx.release();
frame2Idx.release();
frame3Idx.release();
converter1.close();
converter2.close();
converter3.close();
frame.close();
}
@Test public void testLeptonicaFrameConverter() {
System.out.println("LeptonicaFrameConverter");
Frame frame = new Frame(640 + 1, 480, Frame.DEPTH_UBYTE, 3);
LeptonicaFrameConverter converter = new LeptonicaFrameConverter();
UByteIndexer frameIdx = frame.createIndexer();
for (int i = 0; i < frameIdx.rows(); i++) {
for (int j = 0; j < frameIdx.cols(); j++) {
for (int k = 0; k < frameIdx.channels(); k++) {
frameIdx.put(i, j, k, i + j + k);
}
}
}
PIX pix = converter.convert(frame);
converter.frame = null;
Frame frame1 = converter.convert(pix);
// assertEquals(frame1.opaque, pix);
PIX pix2 = PIX.createHeader(pix.w(), pix.h(), pix.d()).data(pix.data()).wpl(pix.wpl());
assertNotEquals(pix, pix2);
Frame frame2 = converter.convert(pix2);
// assertEquals(frame2.opaque, pix2);
IntBuffer frameBuf = ((ByteBuffer)frame.image[0].position(0)).asIntBuffer();
IntBuffer frame1Buf = ((ByteBuffer)frame1.image[0].position(0)).asIntBuffer();
IntBuffer frame2Buf = ((ByteBuffer)frame2.image[0].position(0)).asIntBuffer();
IntBuffer pixBuf = pix.createBuffer().order(ByteOrder.BIG_ENDIAN).asIntBuffer();
IntBuffer pix2Buf = pix2.createBuffer().order(ByteOrder.BIG_ENDIAN).asIntBuffer();
for (int i = 0; i < frameBuf.capacity(); i++) {
int j = frameBuf.get(i);
assertEquals(j, frame1Buf.get(i));
assertEquals(j, frame2Buf.get(i));
assertEquals(j, pixBuf.get(i));
assertEquals(j, pix2Buf.get(i));
}
try {
frame1Buf.get(frameBuf.capacity() + 1);
fail("IndexOutOfBoundsException should have been thrown.");
} catch (IndexOutOfBoundsException e) { }
try {
frame2Buf.get(frameBuf.capacity() + 1);
fail("IndexOutOfBoundsException should have been thrown.");
} catch (IndexOutOfBoundsException e) { }
try {
pixBuf.get(frameBuf.capacity() + 1);
fail("IndexOutOfBoundsException should have been thrown.");
} catch (IndexOutOfBoundsException e) { }
try {
pix2Buf.get(frameBuf.capacity() + 1);
fail("IndexOutOfBoundsException should have been thrown.");
} catch (IndexOutOfBoundsException e) { }
pix2.deallocate();
pix.deallocate();
converter.close();
frame.close();
}
}
================================================
FILE: platform/src/test/java/org/bytedeco/javacv/FrameFilterTest.java
================================================
/*
* Copyright (C) 2018 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.io.File;
import java.nio.ByteBuffer;
import java.nio.ShortBuffer;
import org.bytedeco.javacpp.Loader;
import org.junit.Test;
import static org.bytedeco.ffmpeg.global.avcodec.*;
import static org.bytedeco.ffmpeg.global.avutil.*;
import static org.junit.Assert.*;
/**
* Test cases for FrameFilter classes. Also uses other classes from JavaCV.
*
* @author Samuel Audet
*/
public class FrameFilterTest {
@Test
public void testFFmpegFrameFilter() {
System.out.println("FFmpegFrameFilter");
File tempFile = new File(Loader.getTempDir(), "test.mov");
try {
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(tempFile, 800, 600, 2);
recorder.setFormat("mov");
recorder.setPixelFormat(AV_PIX_FMT_YUV420P);
recorder.setFrameRate(30);
recorder.setVideoCodec(AV_CODEC_ID_H264);
recorder.setVideoQuality(10);
recorder.setSampleFormat(AV_SAMPLE_FMT_FLTP);
recorder.setSampleRate(48000);
recorder.setAudioCodec(AV_CODEC_ID_AAC);
recorder.setAudioQuality(10);
recorder.start();
int n = 1000;
Frame frame = new Frame(800, 600, Frame.DEPTH_UBYTE, 3);
for (int i = 0; i < n; i++) {
recorder.record(frame);
}
Frame audioFrame = new Frame();
ShortBuffer audioBuffer = ShortBuffer.allocate(48000 * 2 * n / 30);
audioFrame.sampleRate = 48000;
audioFrame.audioChannels = 2;
audioFrame.samples = new ShortBuffer[] {audioBuffer};
recorder.record(audioFrame);
recorder.stop();
recorder.release();
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(tempFile);
grabber.setSampleMode(FrameGrabber.SampleMode.FLOAT);
grabber.start();
FFmpegFrameFilter filter = new FFmpegFrameFilter(
"scale=400x300,transpose=cclock_flip,format=gray",
"volume=0.5,aformat=sample_fmts=u8:channel_layouts=mono",
grabber.getImageWidth(), grabber.getImageHeight(), grabber.getAudioChannels());
filter.setPixelFormat(grabber.getPixelFormat());
filter.setSampleFormat(grabber.getSampleFormat());
filter.setFrameRate(grabber.getFrameRate());
filter.setSampleRate(grabber.getSampleRate());
filter.start();
FFmpegFrameFilter nullFilter = new FFmpegFrameFilter(null, null, 0, 0, 0);
nullFilter.start();
int a = 0, b = 0, c = 0, d = 0;
Frame frame2;
while ((frame2 = grabber.grab()) != null) {
if (frame2.image != null) {
a++;
}
if (frame2.samples != null) {
b++;
}
filter.push(frame2);
Frame frame3;
while ((frame3 = filter.pull()) != null) {
if (frame3.image != null) {
c++;
assertEquals(300, frame3.imageWidth);
assertEquals(400, frame3.imageHeight);
assertEquals(1, frame3.imageChannels);
}
if (frame3.samples != null) {
d++;
assertEquals(1, frame3.audioChannels);
assertEquals(1, frame3.samples.length);
assertTrue(frame3.samples[0] instanceof ByteBuffer);
assertEquals(frame2.samples.length, frame3.samples.length);
assertEquals(frame2.samples[0].limit() / 2, frame3.samples[0].limit());
}
assertEquals(frame2.timestamp, frame3.timestamp);
}
nullFilter.push(frame2);
assertEquals(frame2, nullFilter.pull());
}
filter.push(null);
assertEquals(null, filter.pull());
assertEquals(a, c);
assertEquals(b, d);
assertEquals(null, grabber.grab());
filter.stop();
filter.release();
grabber.restart();
grabber.stop();
grabber.release();
frame.close();
} catch (Exception e) {
e.printStackTrace();
fail("Exception should not have been thrown: " + e);
} finally {
tempFile.delete();
}
}
@Test
public void testFFmpegFrameFilterMultipleInputs() {
System.out.println("FFmpegFrameFilterMultipleInputs");
File tempFile = new File(Loader.getTempDir(), "test.avi");
try {
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(tempFile, 320, 200, 2);
recorder.setVideoCodec(AV_CODEC_ID_VP8);
recorder.setAudioCodec(AV_CODEC_ID_VORBIS);
recorder.start();
int n = 1000;
Frame frame = new Frame(320, 200, Frame.DEPTH_UBYTE, 3);
for (int i = 0; i < n; i++) {
recorder.record(frame);
}
Frame audioFrame = new Frame();
ShortBuffer audioBuffer = ShortBuffer.allocate(8000 * 2 * n / 30);
audioFrame.sampleRate = 8000;
audioFrame.audioChannels = 2;
audioFrame.samples = new ShortBuffer[] {audioBuffer};
recorder.record(audioFrame);
recorder.stop();
recorder.release();
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(tempFile);
grabber.start();
FFmpegFrameFilter filter = new FFmpegFrameFilter(
"[0:v][1:v]hstack=inputs=2[v]",
"[0:a][1:a]amerge[a]",
grabber.getImageWidth(), grabber.getImageHeight(), grabber.getAudioChannels());
filter.setPixelFormat(grabber.getPixelFormat());
filter.setSampleFormat(grabber.getSampleFormat());
filter.setVideoInputs(2);
filter.setAudioInputs(2);
filter.start();
int a = 0, b = 0, c = 0, d = 0;
Frame frame2;
while ((frame2 = grabber.grab()) != null) {
if (frame2.image != null) {
a++;
}
if (frame2.samples != null) {
b++;
}
filter.push(0, frame2);
filter.push(1, frame2);
Frame frame3;
while ((frame3 = filter.pull()) != null) {
if (frame3.image != null) {
c++;
assertEquals(640, frame3.imageWidth);
assertEquals(200, frame3.imageHeight);
assertEquals(3, frame3.imageChannels);
}
if (frame3.samples != null) {
d++;
assertEquals(4, frame3.audioChannels);
assertEquals(1, frame3.samples.length);
assertTrue(frame3.samples[0] instanceof ShortBuffer);
assertEquals(frame2.samples.length, frame3.samples.length);
assertEquals(2 * frame2.samples[0].limit(), frame3.samples[0].limit());
}
}
}
filter.push(0, null);
filter.push(1, null);
assertEquals(null, filter.pull());
assertEquals(a, c);
assertEquals(b, d);
assertEquals(null, grabber.grab());
filter.stop();
filter.release();
grabber.restart();
grabber.stop();
grabber.release();
frame.close();
} catch (Exception e) {
e.printStackTrace();
fail("Exception should not have been thrown: " + e);
} finally {
tempFile.delete();
}
}
}
================================================
FILE: platform/src/test/java/org/bytedeco/javacv/FrameGrabberChangingResolutionTest.java
================================================
/*
* Copyright (C) 2016-2017 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.FloatBuffer;
import java.nio.ShortBuffer;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.javacpp.indexer.UByteIndexer;
import org.junit.Test;
import static org.bytedeco.ffmpeg.global.avcodec.*;
import static org.bytedeco.ffmpeg.global.avutil.*;
import static org.junit.Assert.*;
/**
* Complex Test case for FrameGrabber classes - change the resolution during runtime.
* Also uses other classes from JavaCV.
*
* @author Samuel Audet, Michael Fritscher
*/
public class FrameGrabberChangingResolutionTest {
private File tempFile = new File(Loader.getTempDir(), "test.mkv");
private File tempTargetFile = new File(Loader.getTempDir(), "target.mkv");
private boolean endRequested;
private void makeTestfile() throws Exception {
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(new FileOutputStream(tempFile), 640, 480, 2);
recorder.setFormat("matroska"); // mp4 doesn't support streaming
recorder.setPixelFormat(AV_PIX_FMT_YUV420P);
recorder.setVideoCodec(AV_CODEC_ID_H264);
recorder.setVideoQuality(0); // lossless
recorder.setFrameRate(30);
recorder.startUnsafe();
Frame[] frames = new Frame[60];
for (int n = 0; n < frames.length; n++) {
Frame frame = new Frame(640, 480, Frame.DEPTH_UBYTE, 3);
UByteIndexer frameIdx = frame.createIndexer();
for (int i = 0; i < frameIdx.rows(); i++) {
for (int j = 0; j < frameIdx.cols(); j++) {
for (int k = 0; k < frameIdx.channels(); k++) {
frameIdx.put(i, j, k, n + i + j + k);
}
}
}
recorder.record(frame);
frames[n] = frame;
}
recorder.stop();
recorder.release();
for (int n = 0; n < frames.length; n++) {
frames[n].close();
}
}
final public void setupUDPSender(final int x, final int y, final int bandwidth, final int count) throws IOException {
final FFmpegFrameGrabber fg = new FFmpegFrameGrabber(tempFile);
fg.setFrameRate(30);
final FFmpegFrameRecorder fr = new FFmpegFrameRecorder("udp://127.0.0.1:2345", 0);
fr.setVideoCodecName("mpeg2video");
fr.setFormat("mpegts");
fr.setImageWidth(x);
fr.setImageHeight(y);
fr.setVideoBitrate(bandwidth);
fr.setFrameRate(30);
fg.startUnsafe();
fr.startUnsafe();
final boolean[] b = new boolean[1];
Thread t = new Thread() {
public void run() {
try {
for (int i = 0; i < count; i++) {
/*- System.out.println("S: " + fg.getFrameNumber() + " " + fg.getTimestamp() + " "
+ fg.getFrameRate() + " " + fg.getImageWidth() + "x" + fg.getImageHeight() + " "
+ fg.getVideoCodec() + " " + fg.getVideoBitrate() + " " + i); */
Frame source = fg.grabFrame();
fr.record(source);
}
fg.close();
fr.close();
b[0] = true;
} catch (Exception e) {
e.printStackTrace();
fail("Exception should not have been thrown: " + e);
try {
fg.close();
fr.close();
} catch (Exception e1) {
e1.printStackTrace();
}
b[0] = true;
}
}
};
t.setName("Sender");
t.start();
while (!b[0]) {
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
final public void setupUDPReceiver() throws IOException {
Thread t = new Thread() {
public void run() {
FFmpegFrameGrabber fg = new FFmpegFrameGrabber("udp://127.0.0.1:2345");
fg.setFrameRate(30);
FFmpegFrameRecorder fr = new FFmpegFrameRecorder(tempTargetFile, 0);
fr.setVideoCodecName("mpeg2video");
fr.setFormat("mpegts");
fr.setImageWidth(640);
fr.setImageHeight(480);
fr.setVideoBitrate(8000000);
fr.setFrameRate(30);
try {
fg.startUnsafe();
fr.startUnsafe();
} catch (Exception e) {
e.printStackTrace();
}
// Tests whether the width of the picture changes trough all
// qualities and every step has a few pictures.
try {
int n = 0;
int m = 0; // Pictures in this quality
int q = 0; // which quality state?
int[] qualities = { 160, 320, 640, 160, 320, 640, 320, 160 };
while (!endRequested) {
/*- System.out.println("R: " + fg.getFrameNumber() + " " + fg.getTimestamp() + " "
+ fg.getFrameRate() + " " + fg.getImageWidth() + "x" + fg.getImageHeight() + " "
+ fg.getVideoCodec() + " " + fg.getVideoBitrate()); */
Frame source = fg.grabFrame();
n++;
m++;
// System.out.println("WRITTEN: " + n + " " + m + " " +
// q + " " + source.imageWidth);
if (source.imageWidth != qualities[q]) {
q++;
assertEquals(source.imageWidth, qualities[q]);
assertTrue(m > 5);
assertTrue(m <= 60);
m = 0;
}
fr.record(source);
}
assertEquals(q, qualities.length - 1);
assertTrue(n > 300);
assertTrue(n <= 480);
fr.close();
} catch (Exception e) {
e.printStackTrace();
fail("Exception should not have been thrown: " + e);
try {
fg.close();
fr.close();
} catch (Exception e1) {
e1.printStackTrace();
}
}
}
};
t.setName("Receiver");
t.start();
}
@Test
public void testFFmpegFrameGrabber() {
System.out.println("FFmpegFrameGrabber");
try {
makeTestfile();
setupUDPReceiver();
System.out.println("Changing to 160x120");
setupUDPSender(160, 120, 50000, 60);
System.out.println("Changing to 320x240");
setupUDPSender(320, 240, 100000, 60);
System.out.println("Changing to 640x480");
setupUDPSender(640, 480, 200000, 60);
System.out.println("Changing to 160x120");
setupUDPSender(160, 120, 50000, 60);
System.out.println("Changing to 320x240");
setupUDPSender(320, 240, 100000, 60);
System.out.println("Changing to 640x480");
setupUDPSender(640, 480, 200000, 60);
System.out.println("Changing to 320x240");
setupUDPSender(320, 240, 100000, 60);
System.out.println("Changing to 160x120");
setupUDPSender(160, 120, 50000, 60);
Thread.sleep(3000);
endRequested = true;
} catch (Exception e) {
tempFile.delete();
tempTargetFile.delete();
e.printStackTrace();
fail("Exception should not have been thrown: " + e);
}
try {
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(new FileInputStream(tempTargetFile));
grabber.setSampleMode(FrameGrabber.SampleMode.FLOAT);
grabber.startUnsafe();
int n = 0;
Frame frame2;
while ((frame2 = grabber.grab()) != null) {
if (frame2.image != null) {
n++;
assertEquals(640, frame2.imageWidth);
}
}
// It seems that ffmpeg lose some frames while switching (ideal
// value would be 240)
// System.out.println("END NUMBER: " + n);
assertTrue(n > 300);
assertTrue(n <= 480);
assertEquals(null, grabber.grab());
grabber.stop();
grabber.release();
} catch (Exception e) {
e.printStackTrace();
fail("Exception should not have been thrown: " + e);
} finally {
tempFile.delete();
tempTargetFile.delete();
}
}
}
================================================
FILE: platform/src/test/java/org/bytedeco/javacv/FrameGrabberTest.java
================================================
/*
* Copyright (C) 2016-2023 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.FloatBuffer;
import java.nio.ShortBuffer;
import java.util.Random;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.javacpp.PointerScope;
import org.bytedeco.javacpp.indexer.UByteIndexer;
import org.junit.Test;
import static org.bytedeco.ffmpeg.global.avcodec.*;
import static org.bytedeco.ffmpeg.global.avutil.*;
import static org.junit.Assert.*;
/**
* Test cases for FrameGrabber classes. Also uses other classes from JavaCV.
*
* @author Samuel Audet
*/
public class FrameGrabberTest {
@Test
public void testFFmpegFrameGrabber() {
System.out.println("FFmpegFrameGrabber");
File tempFile = new File(Loader.getTempDir(), "test.mkv");
try {
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(new FileOutputStream(tempFile), 640, 480, 2);
recorder.setFormat("matroska"); // mp4 doesn't support streaming
recorder.setPixelFormat(AV_PIX_FMT_BGR24);
recorder.setVideoCodecName("jpegls");
recorder.setVideoQuality(0); // lossless
recorder.setSampleFormat(AV_SAMPLE_FMT_S16);
recorder.setSampleRate(44100);
recorder.setAudioCodecName("pcm_s16le");
recorder.start();
Frame[] frames = new Frame[1000];
for (int n = 0; n < frames.length; n++) {
Frame frame = new Frame(640, 480, Frame.DEPTH_UBYTE, 3);
UByteIndexer frameIdx = frame.createIndexer();
for (int i = 0; i < frameIdx.rows(); i++) {
for (int j = 0; j < frameIdx.cols(); j++) {
for (int k = 0; k < frameIdx.channels(); k++) {
frameIdx.put(i, j, k, n + i + j + k);
}
}
}
recorder.record(frame);
frames[n] = frame;
}
Frame audioFrame = new Frame();
ShortBuffer audioBuffer = ShortBuffer.allocate(64 * 1024);
audioFrame.sampleRate = 44100;
audioFrame.audioChannels = 2;
audioFrame.samples = new ShortBuffer[] {audioBuffer};
for (int i = 0; i < audioBuffer.capacity(); i++) {
audioBuffer.put(i, (short)i);
}
recorder.record(audioFrame);
recorder.stop();
recorder.release();
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(new FileInputStream(tempFile));
grabber.setSampleMode(FrameGrabber.SampleMode.FLOAT);
grabber.start();
int n = 0, m = 0;
Frame frame2;
long startTime = System.nanoTime();
while ((frame2 = grabber.grabAtFrameRate()) != null) {
long delay = frame2.timestamp * 1000 - (System.nanoTime() - startTime);
if (delay < -1_000_000_000 / grabber.getFrameRate()) {
// skip to catch up with frame rate
if (frame2.image != null) {
n++;
} else {
m++;
}
continue;
}
Frame clone2 = frame2.clone();
if (frame2.image != null) {
Frame frame = frames[n++];
assertEquals(frame.imageWidth, frame2.imageWidth);
assertEquals(frame.imageHeight, frame2.imageHeight);
assertEquals(frame.imageChannels, frame2.imageChannels);
assertEquals(frame.imageWidth, clone2.imageWidth);
assertEquals(frame.imageHeight, clone2.imageHeight);
assertEquals(frame.imageChannels, clone2.imageChannels);
UByteIndexer frameIdx = frame.createIndexer();
UByteIndexer frame2Idx = frame2.createIndexer();
UByteIndexer clone2Idx = clone2.createIndexer();
for (int i = 0; i < frameIdx.rows(); i++) {
for (int j = 0; j < frameIdx.cols(); j++) {
for (int k = 0; k < frameIdx.channels(); k++) {
int b = frameIdx.get(i, j, k);
assertEquals(b, frame2Idx.get(i, j, k));
assertEquals(b, clone2Idx.get(i, j, k));
}
}
}
} else {
FloatBuffer audioBuffer2 = (FloatBuffer)frame2.samples[0];
FloatBuffer cloneBuffer2 = (FloatBuffer)clone2.samples[0];
while (audioBuffer2.hasRemaining()) {
assertEquals((float)audioBuffer.get(m) / (Short.MAX_VALUE + 1), audioBuffer2.get(), 0);
assertEquals((float)audioBuffer.get(m) / (Short.MAX_VALUE + 1), cloneBuffer2.get(), 0);
m++;
}
}
clone2.close();
}
long stopTime = System.nanoTime();
assertEquals(n, (stopTime - startTime) * grabber.getFrameRate() / 1_000_000_000, 3.0);
assertEquals(frames.length, n);
assertEquals(null, grabber.grab());
grabber.restart();
grabber.stop();
grabber.release();
for (n = 0; n < frames.length; n++) {
frames[n].close();
}
} catch (Exception e) {
e.printStackTrace();
fail("Exception should not have been thrown: " + e);
} finally {
tempFile.delete();
}
}
@Test
public void testFFmpegFrameGrabberLockingTest() {
final boolean[] failed = {false};
final int numberOfInstances = 20;
System.out.println("FFmpegFrameGrabberLocking");
Runnable[] runables = new Runnable[numberOfInstances];
Thread[] threads = new Thread[numberOfInstances];
final boolean[] finish = new boolean[numberOfInstances];
for (int instance = 0; instance < numberOfInstances; instance++) {
final int instance_final = instance;
Runnable r = new Runnable() {
public void run() {
File tempFile = new File(Loader.getTempDir(), "test" + instance_final + ".mkv");
try (PointerScope scope = new PointerScope()) {
FFmpegLogCallback.set();
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(new FileOutputStream(tempFile), 640, 480, 2);
recorder.setFormat("matroska"); // mp4 doesn't support streaming
recorder.setPixelFormat(AV_PIX_FMT_BGR24);
recorder.setVideoCodecName("jpegls");
recorder.setVideoQuality(0); // lossless
recorder.setSampleFormat(AV_SAMPLE_FMT_S16);
recorder.setSampleRate(44100);
recorder.setAudioCodecName("pcm_s16le");
recorder.startUnsafe();
Frame[] frames = new Frame[10];
for (int n = 0; n < frames.length; n++) {
Frame frame = new Frame(640, 480, Frame.DEPTH_UBYTE, 3);
UByteIndexer frameIdx = frame.createIndexer();
for (int i = 0; i < frameIdx.rows(); i++) {
for (int j = 0; j < frameIdx.cols(); j++) {
for (int k = 0; k < frameIdx.channels(); k++) {
frameIdx.put(i, j, k, n + i + j + k);
}
}
}
recorder.record(frame);
frames[n] = frame;
}
Frame audioFrame = new Frame();
ShortBuffer audioBuffer = ShortBuffer.allocate(64 * 1024);
audioFrame.sampleRate = 44100;
audioFrame.audioChannels = 2;
audioFrame.samples = new ShortBuffer[] { audioBuffer };
for (int i = 0; i < audioBuffer.capacity(); i++) {
audioBuffer.put(i, (short) i);
}
recorder.record(audioFrame);
recorder.stop();
recorder.release();
Thread.sleep(1000);
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(new FileInputStream(tempFile));
grabber.setSampleMode(FrameGrabber.SampleMode.FLOAT);
grabber.startUnsafe();
int n = 0, m = 0;
Frame frame2;
while ((frame2 = grabber.grab()) != null) {
if (frame2.image != null) {
Frame frame = frames[n++];
assertEquals(frame.imageWidth, frame2.imageWidth);
assertEquals(frame.imageHeight, frame2.imageHeight);
assertEquals(frame.imageChannels, frame2.imageChannels);
UByteIndexer frameIdx = frame.createIndexer();
UByteIndexer frame2Idx = frame2.createIndexer();
for (int i = 0; i < frameIdx.rows(); i++) {
for (int j = 0; j < frameIdx.cols(); j++) {
for (int k = 0; k < frameIdx.channels(); k++) {
int b = frameIdx.get(i, j, k);
assertEquals(b, frame2Idx.get(i, j, k));
}
}
}
} else {
FloatBuffer audioBuffer2 = (FloatBuffer) frame2.samples[0];
while (audioBuffer2.hasRemaining()) {
assertEquals((float) audioBuffer.get(m++) / (Short.MAX_VALUE + 1),
audioBuffer2.get(), 0);
}
}
}
assertEquals(frames.length, n);
assertEquals(null, grabber.grab());
grabber.restart();
grabber.stop();
grabber.release();
for (n = 0; n < frames.length; n++) {
frames[n].close();
}
} catch (Error | Exception e) {
failed[0] = true;
e.printStackTrace();
fail("Exception should not have been thrown: " + e);
} finally {
tempFile.delete();
finish[instance_final] = true;
}
}
};
runables[instance_final] = r;
}
for (int instance = 0; instance < numberOfInstances; instance++) {
threads[instance] = new Thread(runables[instance]);
threads[instance].setName("Testthread-" + instance);
}
for (int instance = 0; instance < numberOfInstances; instance++) {
threads[instance].start();
}
while (true) {
boolean finished = true;
for (int instance = 0; instance < numberOfInstances; instance++) {
if (!finish[instance]) {
finished = false;
break;
}
}
if (!finished) {
System.out.println("Still waiting...");
try {
Thread.sleep(500);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
} else {
break;
}
}
assertFalse(failed[0]);
}
@Test
public void testFFmpegFrameGrabberSeeking() throws IOException {
System.out.println("FFmpegFrameGrabberSeeking");
for(int seektestnum = 0; seektestnum < 3; seektestnum++) try (PointerScope scope = new PointerScope()) {
FFmpegLogCallback.set();
String fileName = seektestnum==0?"testAV.mp4":seektestnum==1?"testV.mp4":"testA.mp4";
File tempFile = new File(Loader.getTempDir(), fileName);
tempFile.deleteOnExit();
FFmpegFrameRecorder recorder = seektestnum == 0? new FFmpegFrameRecorder(tempFile, 640, 480, 2)
: seektestnum == 1? new FFmpegFrameRecorder(tempFile, 640, 480, 0)
: new FFmpegFrameRecorder(tempFile, 0, 0, 2);
recorder.setFormat("mp4");
recorder.setFrameRate(30);
recorder.setPixelFormat(AV_PIX_FMT_YUV420P);
recorder.setVideoCodec(AV_CODEC_ID_MPEG4);
recorder.setVideoQuality(10);
recorder.setSampleRate(48000);
recorder.setSampleFormat(AV_SAMPLE_FMT_FLTP);
recorder.setAudioCodec(AV_CODEC_ID_AAC);
recorder.setAudioQuality(0);
recorder.setDisplayRotation((seektestnum - 2) * 90.0);
recorder.start();
if (seektestnum!=2) {
Frame frame = new Frame(640, 480, Frame.DEPTH_UBYTE, 3);
UByteIndexer frameIdx = frame.createIndexer();
for (int n = 0; n < 10000; n++) {
for (int i = 0; i < frameIdx.rows(); i++) {
for (int j = 0; j < frameIdx.cols(); j++) {
for (int k = 0; k < frameIdx.channels(); k++) {
frameIdx.put(i, j, k, n + i + j + k);
}
}
}
recorder.record(frame);
if (n == 5000 && seektestnum!=1){
Frame audioFrame = new Frame();
ShortBuffer audioBuffer = ShortBuffer.allocate(48000 * 2 * 10000 / 30);
audioFrame.sampleRate = 48000;
audioFrame.audioChannels = 2;
audioFrame.samples = new ShortBuffer[] {audioBuffer};
for (int i = 0; i < audioBuffer.capacity(); i++) {
audioBuffer.put(i, (short)i);
}
recorder.record(audioFrame);
}
}
frame.close();
} else {
Frame audioFrame = new Frame();
ShortBuffer audioBuffer = ShortBuffer.allocate(48000 * 2 * 10000 / 30);
audioFrame.sampleRate = 48000;
audioFrame.audioChannels = 2;
audioFrame.samples = new ShortBuffer[] {audioBuffer};
for (int i = 0; i < audioBuffer.capacity(); i++) {
audioBuffer.put(i, (short)i);
}
recorder.record(audioFrame);
}
recorder.stop();
recorder.release();
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(tempFile);
grabber.setVideoOption("threads", "1"); // more precise without threads
grabber.start();
assertEquals((seektestnum - 2) * 90.0, grabber.getDisplayRotation(), 0);
int length = (int) ( grabber.getLengthInTime() - 1000000L);
System.out.println();
System.out.println("Seek in file containing "+(seektestnum==0?"video and audio":seektestnum==1?"video only":"audio only"));
System.out.println("============================================");
System.out.println("Testing file "+tempFile.getName());
System.out.println("Length = "+grabber.getLengthInTime());
System.out.println("Framerate = "+grabber.getFrameRate());
System.out.println();
System.out.println("has video stream = "+(grabber.hasVideo()?"YES":"NO")+", has audio stream = "+(grabber.hasAudio()?"YES":"NO"));
long tolerance = 1000000L + (grabber.getFrameRate() > 0.0? (long) (5000000/grabber.getFrameRate()):500000L);
Random random = new Random(29);
for (int frametypenum = 0; frametypenum < 4; frametypenum++) {
long mindelta = Long.MAX_VALUE;
long maxdelta = Long.MIN_VALUE;
System.out.println();
System.out.println("Seek by "
+ (frametypenum == 0 ? "any" : frametypenum == 1 ? "video" : frametypenum == 2 ? "audio" : "old method")
+ (frametypenum == 0 ? " frames" : ""));
System.out.println("--------------------");
for (int i = 0; i < 200; i++) {
long timestamp = random.nextInt(length);
switch (frametypenum) {
case 0:
grabber.setTimestamp(timestamp, true);
break;
case 1:
grabber.setVideoTimestamp(timestamp);
break;
case 2:
grabber.setAudioTimestamp(timestamp);
break;
case 3:
grabber.setTimestamp(timestamp);
break;
}
Frame frame = grabber.grab();
long timestamp2 = grabber.getTimestamp();
long delta = timestamp2 - timestamp;
if (delta > maxdelta) maxdelta = delta;
if (delta < mindelta) mindelta = delta;
assertTrue(frame.image != null ^ frame.samples != null);
System.out.println(timestamp2 + " - " + timestamp + " = " + delta + " type: " + frame.getTypes());
assertTrue(Math.abs(delta) < tolerance);
/*
if (seektestnum==0) {
boolean wasVideo = frame.image != null;
boolean wasAudio = frame.samples != null;
Frame frame2 = grabber.grab();
while ((wasVideo && frame2.image != null)
|| (wasAudio && frame2.samples != null)) {
frame2 = grabber.grab();
}
assertTrue(wasVideo ^ frame2.image != null);
assertTrue(wasAudio ^ frame2.samples != null);
long timestamp3 = grabber.getTimestamp();
System.out.println(timestamp3 + " - " + timestamp + " = " + (timestamp3 - timestamp));
assertTrue(timestamp3 >= timestamp - tolerance && timestamp3 < timestamp + tolerance);
}
*/
}
System.out.println();
System.out.println("------------------------------------");
System.out.println("delta from " + mindelta + " to " + maxdelta);
System.out.println();
}
if (seektestnum==0) {
System.out.println();
System.out.println("======== Check sequential setVideoFrameNumber (issue #1697) ========");
for (int i = 0; i < 10; i++) {
grabber.setVideoFrameNumber(i);
long timestamp = grabber.grabImage().timestamp;
System.out.println("frame number:" + i + " timestamp:" + timestamp);
assertTrue(i == Math.round(timestamp * grabber.getFrameRate() / 1000000L));
}
}
if (seektestnum==2) {
long count1 = 0;
long duration = grabber.getLengthInTime();
System.out.println();
System.out.println("======== Check seeking in audio ========");
System.out.println("FrameRate = "+grabber.getFrameRate()+" AudioFrameRate = "+grabber.getAudioFrameRate()+", duration = "+duration+" audio frames = "+grabber.getLengthInAudioFrames());
double deltaTimeStamp=0.0;
if (grabber.hasAudio() && grabber.getAudioFrameRate() > 0) {
deltaTimeStamp = 1000000.0/grabber.getAudioFrameRate();
}
System.out.println("AudioFrameDuration = "+deltaTimeStamp);
System.out.println();
System.out.println("======== Check setAudioFrameNumber ========");
count1=0;
while (count1++<1000) {
int audioFrameToSeek = random.nextInt(grabber.getLengthInAudioFrames()-100);
grabber.setAudioFrameNumber(audioFrameToSeek);
Frame setFrame = grabber.grabSamples();
if (setFrame == null) {
System.out.println("null frame after seek to audio frame");
} else {
long audioTs = grabber.getTimestamp();
System.out.println("audioFrame # "+audioFrameToSeek+", timeStamp = "+audioTs+", difference = "+Math.round(audioTs*grabber.getAudioFrameRate()/1000000 - audioFrameToSeek));
assertTrue(Math.abs(audioTs*grabber.getAudioFrameRate()/1000000 - audioFrameToSeek)<10);
}
}
}
grabber.stop();
System.out.println();
System.out.println("======= seek in " +fileName+" is finished ===========" );
}
}
}
================================================
FILE: platform/src/test/java/org/bytedeco/javacv/SeekableByteArrayOutputStreamTest.java
================================================
/*
* Copyright (C) 2019 Sven Vorlauf
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import static org.junit.Assert.*;
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.util.Arrays;
import java.util.Random;
import org.bytedeco.ffmpeg.global.avcodec;
import org.bytedeco.ffmpeg.global.avutil;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.javacpp.indexer.UByteIndexer;
import org.hamcrest.core.IsEqual;
import org.hamcrest.core.IsNot;
import org.junit.Test;
public class SeekableByteArrayOutputStreamTest {
private static final int WIDTH = 640;
private static final int HEIGHT = 360;
private static final int FRAME_COUNT = 100;
private int writeByte(byte[] originalBytes, int offset, SeekableByteArrayOutputStream byteArrayOutputStream) {
byteArrayOutputStream.write(originalBytes[offset]);
return 1;
}
private int writePartialBytes(byte[] originalBytes, int offset, Random random,
SeekableByteArrayOutputStream byteArrayOutputStream) throws IOException {
int chunkSize = Math.min(random.nextInt(50), originalBytes.length - offset);
byteArrayOutputStream.write(originalBytes, offset, chunkSize);
return chunkSize;
}
private int writeBytes(byte[] originalBytes, int offset, Random random,
SeekableByteArrayOutputStream byteArrayOutputStream) throws IOException {
int chunkSize = Math.min(random.nextInt(50), originalBytes.length - offset);
byteArrayOutputStream.write(Arrays.copyOfRange(originalBytes, offset, offset + chunkSize));
return chunkSize;
}
private void createVideo(FFmpegFrameRecorder recorder) throws Exception {
recorder.setVideoCodec(avcodec.AV_CODEC_ID_MPEG4);
recorder.setFormat("mp4");
recorder.setFrameRate(30);
recorder.setPixelFormat(avutil.AV_PIX_FMT_YUV420P);
recorder.start();
for (int n = 0; n < FRAME_COUNT; n++) {
Frame frame = new Frame(WIDTH, HEIGHT, Frame.DEPTH_UBYTE, 3);
UByteIndexer frameIdx = frame.createIndexer();
for (int i = 0; i < frameIdx.rows(); i++) {
for (int j = 0; j < frameIdx.cols(); j++) {
for (int k = 0; k < frameIdx.channels(); k++) {
frameIdx.put(i, j, k, n + i + j + k);
}
}
}
recorder.record(frame);
frame.close();
}
recorder.close();
}
@Test
public void serialWriteByteTest() {
System.out.println("SeekableByteArrayOutputStreamSerialWriteByte");
Random random = new Random(-1);
byte[] originalBytes = new byte[1000];
random.nextBytes(originalBytes);
try (SeekableByteArrayOutputStream byteArrayOutputStream = new SeekableByteArrayOutputStream()) {
int offset = 0;
while (offset < originalBytes.length) {
offset += writeByte(originalBytes, offset, byteArrayOutputStream);
}
assertArrayEquals(originalBytes, byteArrayOutputStream.toByteArray());
} catch (Exception e) {
fail("Exception should not have been thrown: " + e);
}
}
@Test
public void serialWriteBytesTest() {
System.out.println("SeekableByteArrayOutputStreamSerialWriteBytes");
Random random = new Random(-1);
byte[] originalBytes = new byte[1000];
random.nextBytes(originalBytes);
try (SeekableByteArrayOutputStream byteArrayOutputStream = new SeekableByteArrayOutputStream()) {
int offset = 0;
while (offset < originalBytes.length) {
offset += writeBytes(originalBytes, offset, random, byteArrayOutputStream);
}
assertArrayEquals(originalBytes, byteArrayOutputStream.toByteArray());
} catch (Exception e) {
fail("Exception should not have been thrown: " + e);
}
}
@Test
public void serialWritePartialBytesTest() {
System.out.println("SeekableByteArrayOutputStreamSerialWritePartialBytes");
Random random = new Random(-1);
byte[] originalBytes = new byte[1000];
random.nextBytes(originalBytes);
try (SeekableByteArrayOutputStream byteArrayOutputStream = new SeekableByteArrayOutputStream()) {
int offset = 0;
while (offset < originalBytes.length) {
offset += writePartialBytes(originalBytes, offset, random, byteArrayOutputStream);
}
assertArrayEquals(originalBytes, byteArrayOutputStream.toByteArray());
} catch (Exception e) {
fail("Exception should not have been thrown: " + e);
}
}
@Test
public void serialWriteTest() {
System.out.println("SeekableByteArrayOutputStreamSerialWrite");
Random random = new Random(-1);
byte[] originalBytes = new byte[1000];
random.nextBytes(originalBytes);
try (SeekableByteArrayOutputStream byteArrayOutputStream = new SeekableByteArrayOutputStream()) {
int offset = 0;
while (offset < originalBytes.length) {
switch (random.nextInt(3)) {
case 0:
offset += writeByte(originalBytes, offset, byteArrayOutputStream);
break;
case 1:
offset += writeBytes(originalBytes, offset, random, byteArrayOutputStream);
break;
case 2:
offset += writePartialBytes(originalBytes, offset, random, byteArrayOutputStream);
break;
}
}
assertArrayEquals(originalBytes, byteArrayOutputStream.toByteArray());
} catch (Exception e) {
fail("Exception should not have been thrown: " + e);
}
}
public void seekWriteTest() {
System.out.println("SeekableByteArrayOutputStreamSeekWrite");
Random random = new Random(-1);
byte[] originalBytes = new byte[1000];
random.nextBytes(originalBytes);
try (SeekableByteArrayOutputStream byteArrayOutputStream = new SeekableByteArrayOutputStream()) {
int offset = 0;
for (int i = 0; i < 10; i++) {
// write 100 bytes
byteArrayOutputStream.write(originalBytes, offset, 100);
int position = random.nextInt(offset + 20);
int newBytesPosition = position + 500 % 1000;
int length = 10 + random.nextInt(20);
// get current bytes
byte[] writtenOriginalBytes = Arrays.copyOfRange(byteArrayOutputStream.toByteArray(), position,
position + length);
// bytes to write at the new position
byte[] newBytes = Arrays.copyOfRange(originalBytes, newBytesPosition, newBytesPosition + length);
// just assert that the new bytes are different to the written ones
assertThat(writtenOriginalBytes, IsNot.not(IsEqual.equalTo(newBytes)));
// replace bytes
byteArrayOutputStream.seek(position, 0);
byteArrayOutputStream.write(newBytes);
byte[] writtenNewBytes = Arrays.copyOfRange(byteArrayOutputStream.toByteArray(), position,
position + length);
assertThat(newBytes, IsEqual.equalTo(writtenNewBytes));
// write back original bytes
byteArrayOutputStream.seek(position, 0);
byteArrayOutputStream.write(originalBytes, position, length);
// get back to the end of the stream
byteArrayOutputStream.seek(offset, 0);
offset += 100;
}
while (offset < originalBytes.length) {
switch (random.nextInt(3)) {
case 0:
offset += writeByte(originalBytes, offset, byteArrayOutputStream);
break;
case 1:
offset += writeBytes(originalBytes, offset, random, byteArrayOutputStream);
break;
case 2:
offset += writePartialBytes(originalBytes, offset, random, byteArrayOutputStream);
break;
}
}
assertArrayEquals(originalBytes, byteArrayOutputStream.toByteArray());
} catch (Exception e) {
fail("Exception should not have been thrown: " + e);
}
}
@Test
public void testVideoBytesEqual() {
// if this test fails it might be due to indeterministic multithreaded encoding
System.out.println("SeekableByteArrayOutputStreamVideo");
File tempFile = new File(Loader.getTempDir(), "test.mp4");
try {
createVideo(new FFmpegFrameRecorder(tempFile, WIDTH, HEIGHT, 0));
byte[] fileBytes = Files.readAllBytes(tempFile.toPath());
SeekableByteArrayOutputStream byteArrayOutputStream = new SeekableByteArrayOutputStream();
createVideo(new FFmpegFrameRecorder(byteArrayOutputStream, WIDTH, HEIGHT, 0));
assertArrayEquals(fileBytes, byteArrayOutputStream.toByteArray());
} catch (Exception e) {
fail("Exception should not have been thrown: " + e);
} finally {
tempFile.delete();
}
}
}
================================================
FILE: pom.xml
================================================
4.0.0
org.bytedeco
javacv
1.5.14-SNAPSHOT
JavaCV
Java interface to OpenCV, FFmpeg, and more
http://bytedeco.org/javacv/
Apache License, Version 2.0
http://www.apache.org/licenses/LICENSE-2.0
repo
GNU General Public License (GPL) version 2, or any later version
http://www.gnu.org/licenses/
repo
GPLv2 with Classpath exception
http://www.gnu.org/software/classpath/license.html
repo
Samuel Audet
samuel.audet@gmail.com
https://github.com/bytedeco/javacv
scm:git:git://github.com/bytedeco/javacv.git
scm:git:ssh://git@github.com/bytedeco/javacv.git
UTF-8
yyyyMMddhhmm
${project.version}
org.bytedeco
javacpp
${javacpp.version}
org.bytedeco
openblas
0.3.31-${javacpp.version}
org.bytedeco
opencv
4.13.0-${javacpp.version}
org.bytedeco
ffmpeg
8.0.1-${javacpp.version}
org.bytedeco
flycapture
2.13.3.31-1.5.9
org.bytedeco
libdc1394
2.2.6-1.5.9
org.bytedeco
libfreenect
0.5.7-1.5.9
org.bytedeco
libfreenect2
0.2.0-1.5.9
org.bytedeco
librealsense
1.12.4-1.5.9
org.bytedeco
librealsense2
2.53.1-1.5.9
org.bytedeco
videoinput
0.200-1.5.9
org.bytedeco
artoolkitplus
2.3.1-1.5.9
org.bytedeco
leptonica
1.87.0-${javacpp.version}
org.bytedeco
tesseract
5.5.2-${javacpp.version}
com.google.android
android
4.1.1.4
*
*
true
org.jogamp.gluegen
gluegen-rt-main
2.6.0
true
org.jogamp.jogl
jogl-all-main
2.6.0
true
org.jogamp.jocl
jocl-main
2.6.0
true
com.badlogicgames.gdx
gdx
1.14.0
true
central-portal-snapshots
Central Portal Snapshots
https://central.sonatype.com/repository/maven-snapshots/
false
true
${project.artifactId}
maven-compiler-plugin
3.15.0
1.8
1.8
true
org/bytedeco/javacv/FFmpegLockCallback.java
org/bytedeco/javacv/FlyCaptureFrameGrabber.java
maven-jar-plugin
3.5.0
org.bytedeco.javacv.JavaCV
true
javacpp.jar openblas.jar opencv.jar ffmpeg.jar flycapture.jar libdc1394.jar libfreenect.jar libfreenect2.jar librealsense.jar librealsense2.jar videoinput.jar artoolkitplus.jar flandmark.jar leptonica.jar tesseract.jar
${project.name}
Bytedeco
${project.version}
${project.name}
Bytedeco
${project.version}
true
org.moditect
moditect-maven-plugin
1.3.0
9
true
${project.build.directory}
add-module-infos
package
add-module-info
${project.build.directory}/${project.artifactId}.jar
${project.basedir}/src/main/java9/module-info.java
maven-install-plugin
3.1.4
true
maven-source-plugin
3.4.0
attach-sources
leave-disabled-to-not-generate-sources-twice-on-release
attach-source
jar-no-fork
maven-javadoc-plugin
3.12.0
attach-javadocs
jar
org/bytedeco/javacv/FlyCaptureFrameGrabber.java
org/bytedeco/javacv/FFmpegLockCallback.java
OpenJFX-8
(,11)
net.java.openjfx.backport
openjfx-78-backport
1.8.0-ea-b96.1
provided
OpenJFX-11
11
org.openjfx
javafx-graphics
11
OpenJFX-17
17
org.openjfx
javafx-graphics
17
OpenJFX-21
21
org.openjfx
javafx-graphics
21
OpenJFX-25
25
org.openjfx
javafx-graphics
25
doclint-java8-disable
[1.8,)
maven-javadoc-plugin
none
false
8
sign-artifacts
performRelease
true
central-portal-staging
Central Portal Staging
https://central.sonatype.com/api/v1/publisher/deployments/download/
true
false
maven-gpg-plugin
3.2.8
sign-artifacts
verify
sign
${env.GPG_PASSPHRASE}
false
central-publishing
!altDeploymentRepository
org.sonatype.central
central-publishing-maven-plugin
0.8.0
true
central
false
required
================================================
FILE: samples/AudioSplitMergeHelper.java
================================================
import org.bytedeco.javacv.*;
import java.nio.Buffer;
import java.nio.ShortBuffer;
/**
* This code is a sample which split a 2-channel stereo audio into 2 single-channel mono audios
* or merge 2 single-channel mono audios into a 2-channel stereo.
*
* The code has been tested on s16le audio.
*
* s16le means short 16bit little end. For other format, you may need change the ShortBuffer to other Buffer subclass
*
* For s16lep, s32lep,xxxxxp format, the sample point arrangement format is no longer in ‘LRLRLR'.
* Instead, it is arragement in format 'LLLLLL','RRRRRR'. So you have to change the short copy code.
*
*
* ///////////////////////////////////////////////////////////////////////////
* JavaCV is an excellent open-source streaming processing framework in the Java field
*
* But I see many people, especially in China, making profits for themselves by introducing its usage,
* which is not in line with the concept of open source projects.
* I hope that If this code helped you, you can share your experience and knowledge with others in the world, rather
* than for personal gain. Spread the spirit of open source.
* ///////////////////////////////////////////////////////////////////////////
*
* Acknowledge: Thanks for my hot girlfriend.
*
* @author steeveen
* @date 2023/7/1 14:32
*/
public class AudioSplitMergeHelper {
/**
* split a 2-channel stereo audio into 2 single-channel mono audios
*
* If you want to split this 2-channel stereo to 2 single-channel stereo, you should create 2 2-channel stereos
* and fill one channel with 0 data. It is similar in principle, so the code won't go into too much here.
*
* @param input the file path which is to be splited
* @param outputLeft the file path which store the left channel audio file
* @param outputRight the file path which store the right channel audio file
* @throws FrameGrabber.Exception
* @throws FrameRecorder.Exception
*/
public static void split(String input, String outputLeft, String outputRight) throws FrameGrabber.Exception, FrameRecorder.Exception {
//grabber from input
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(input);
grabber.start();
//two recorders for two channels
FFmpegFrameRecorder leftRecorder = new FFmpegFrameRecorder(outputLeft, 1);
leftRecorder.setSampleRate(grabber.getSampleRate());
leftRecorder.start();
FFmpegFrameRecorder rightRecorder = new FFmpegFrameRecorder(outputRight, 1);
rightRecorder.setSampleRate(grabber.getSampleRate());
rightRecorder.start();
Frame frame = null;
while ((frame = grabber.grabSamples()) != null) {
// use s16le for example. so select ShortBuffer to receive the sample
ShortBuffer sb = (ShortBuffer) frame.samples[0];
short[] shorts = new short[sb.limit()];
sb.get(shorts);
//Split the LRLRLR to LLL in left channel and RRR int right channel
Frame leftFrame = frame.clone();
ShortBuffer leftSb = ShortBuffer.allocate(sb.capacity() / 2);
leftFrame.samples = new Buffer[]{leftSb};
leftFrame.audioChannels = 1;
Frame rightFrame = frame.clone();
ShortBuffer rightSb = ShortBuffer.allocate(sb.capacity() / 2);
rightFrame.samples = new Buffer[]{rightSb};
rightFrame.audioChannels = 1;
for (int i = 0; i < shorts.length; i++) {
if (i % 2 == 0) {
leftSb.put(shorts[i]);
} else {
rightSb.put(shorts[i]);
}
}
// reset the buffer to read mode
leftSb.rewind();
rightSb.rewind();
leftRecorder.record(leftFrame);
rightRecorder.record(rightFrame);
}
//release source
grabber.close();
leftRecorder.close();
rightRecorder.close();
}
/**
* Merge 2 single-channel mono audios into a 2-channel stereo.
* As usual the two input audios should have the same parameter and length;
*
* @param inputLeft the left channel to be merged in
* @param inputRight the right channel to be merged in
* @param output the merged stereo audio
* @throws FFmpegFrameGrabber.Exception
* @throws FFmpegFrameRecorder.Exception
*/
public static void merge(String inputLeft, String inputRight, String output) throws FrameGrabber.Exception, FrameRecorder.Exception {
FFmpegFrameGrabber leftGrabber = new FFmpegFrameGrabber(inputLeft);
leftGrabber.start();
FFmpegFrameGrabber rightGrabber = new FFmpegFrameGrabber(inputRight);
rightGrabber.start();
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(output, 2);
//you'd better confirm the two input have the same samplerate. otherwise, you should control it manually by yourself
recorder.setSampleRate(leftGrabber.getSampleRate());
recorder.start();
Frame leftFrame = null;
Frame rightFrame = null;
int index = 0;
int maxLength = leftGrabber.getLengthInAudioFrames();
while (index < maxLength) {
// carry the bit data from two input into result frame by frame
leftFrame = leftGrabber.grabSamples();
rightFrame = rightGrabber.grabSamples();
ShortBuffer leftSb = (ShortBuffer) leftFrame.samples[0];
ShortBuffer rightSb = (ShortBuffer) rightFrame.samples[0];
short[] leftShorts = new short[leftSb.limit()];
short[] rightShorts = new short[rightSb.limit()];
leftSb.get(leftShorts);
rightSb.get(rightShorts);
ShortBuffer mergeSb = ShortBuffer.allocate(leftSb.capacity() + rightSb.capacity());
// create a template from the existing frame
Frame mergeFrame = leftFrame.clone();
// replace the frame tempalte by our merged buffer
mergeFrame.samples = new Buffer[]{mergeSb};
mergeFrame.audioChannels = 2;
for (int i = 0; i < leftShorts.length; i++) {
mergeSb.put(leftShorts[i]);
mergeSb.put(rightShorts[i]);
}
//reset buffer to read mode
mergeSb.flip();
recorder.record(mergeFrame);
index++;
}
//release source
leftGrabber.close();
rightGrabber.close();
recorder.close();
}
}
================================================
FILE: samples/BioInspiredRetina.java
================================================
import java.awt.*;
import java.awt.image.BufferedImage;
import java.io.File;
import javax.imageio.ImageIO;
import org.bytedeco.javacpp.tools.Slf4jLogger;
import org.bytedeco.javacv.CanvasFrame;
import org.bytedeco.javacv.Java2DFrameConverter;
import org.bytedeco.javacv.OpenCVFrameConverter;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import org.bytedeco.opencv.opencv_bioinspired.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_bioinspired.*;
/**
* Bioinspired Retina demonstration
* This retina model allows spatio-temporal image processing
* As a summary, these are the retina model properties:
* It applies a spectral whithening (mid-frequency details enhancement)
* high frequency spatio-temporal noise reduction
* low frequency luminance to be reduced (luminance range compression)
* local logarithmic luminance compression allows details to be enhanced in low light conditions
*
* Created by mbetzel on 04.09.2016.
*/
public class BioInspiredRetina {
static {
System.setProperty("org.bytedeco.javacpp.logger", "slf4jlogger");
System.setProperty("org.slf4j.simpleLogger.defaultLogLevel", "debug");
}
private static final Slf4jLogger logger = (Slf4jLogger) org.bytedeco.javacpp.tools.Logger.create(BioInspiredRetina.class);
public static void main(String[] args) {
try {
logger.info(String.valueOf(logger.isDebugEnabled()));
logger.info("Start");
new BioInspiredRetina().execute(args);
logger.info("Stop");
} catch (Exception e) {
e.printStackTrace();
}
}
private void execute(String[] args) throws Exception {
BufferedImage bufferedImage = args.length >= 1 ? ImageIO.read(new File(args[0])) : ImageIO.read(this.getClass().getResourceAsStream("BlackBalls.jpg"));
System.out.println("Image type: " + bufferedImage.getType());
Mat matrix = new OpenCVFrameConverter.ToMat().convert(new Java2DFrameConverter().convert(bufferedImage));
normalize(matrix, matrix, 0, 255, NORM_MINMAX, -1, noArray());
showImage(matrix);
matrix.convertTo(matrix, CV_32F);
Mat gammaTransformedImage = new Mat(matrix.size(), CV_32F);
pow(matrix, 1. / 5, gammaTransformedImage);
Retina retina = Retina.create(gammaTransformedImage.size());
Mat retinaOutput_parvo = new Mat();
Mat retinaOutput_magno = new Mat();
retina.clearBuffers();
retina.run(gammaTransformedImage);
retina.getParvo(retinaOutput_parvo);
retina.getMagno(retinaOutput_magno);
showImage(retinaOutput_parvo);
showImage(retinaOutput_magno);
}
private void showImage(Mat matrix) {
CanvasFrame canvasFrame = new CanvasFrame("Retina demonstration", 1);
canvasFrame.setDefaultCloseOperation(javax.swing.JFrame.EXIT_ON_CLOSE);
canvasFrame.setCanvasSize(640, 480);
Canvas canvas = canvasFrame.getCanvas();
canvasFrame.getContentPane().removeAll();
ScrollPane scrollPane = new ScrollPane();
scrollPane.add(canvas);
canvasFrame.add(scrollPane);
canvasFrame.showImage(new OpenCVFrameConverter.ToMat().convert(matrix));
}
}
================================================
FILE: samples/BlobDemo.java
================================================
import org.bytedeco.javacv.Blobs;
import org.bytedeco.javacv.CanvasFrame;
import org.bytedeco.javacv.OpenCVFrameConverter;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgcodecs.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
///////////////////////////////////////////////////////////////////
//* *//
//* As the author of this code, I place all of this code into *//
//* the public domain. Users can use it for any legal purpose. *//
//* *//
//* - Dave Grossman *//
//* *//
///////////////////////////////////////////////////////////////////
public class BlobDemo
{
public static void main(String[] args)
{
System.out.println("STARTING...\n");
demo();
System.out.println("ALL DONE");
}
public static void demo()
{
int MinArea = 6;
int ErodeCount =0;
int DilateCount = 0;
IplImage RawImage = null;
// Read an image.
for(int k = 0; k < 7; k++)
{
if(k == 0) { RawImage = cvLoadImage("BlackBalls.jpg"); MinArea = 250; ErodeCount = 0; DilateCount = 1; }
else if(k == 1) { RawImage = cvLoadImage("Shapes1.jpg"); MinArea = 6; ErodeCount = 0; DilateCount = 1; }
else if(k == 2) { RawImage = cvLoadImage("Shapes2.jpg"); MinArea = 250; ErodeCount = 0; DilateCount = 1; }
else if(k == 3) { RawImage = cvLoadImage("Blob1.jpg"); MinArea = 2800; ErodeCount = 1; DilateCount = 1; }
else if(k == 4) { RawImage = cvLoadImage("Blob2.jpg"); MinArea = 2800; ErodeCount = 1; DilateCount = 1; }
else if(k == 5) { RawImage = cvLoadImage("Blob3.jpg"); MinArea = 2800; ErodeCount = 1; DilateCount = 1; }
else if(k == 6) { RawImage = cvLoadImage("Rice.jpg"); MinArea = 30; ErodeCount = 2; DilateCount = 1; }
//ShowImage(RawImage, "RawImage", 512);
IplImage GrayImage = cvCreateImage(cvGetSize(RawImage), IPL_DEPTH_8U, 1);
cvCvtColor(RawImage, GrayImage, CV_BGR2GRAY);
//ShowImage(GrayImage, "GrayImage", 512);
IplImage BWImage = cvCreateImage(cvGetSize(GrayImage), IPL_DEPTH_8U, 1);
cvThreshold(GrayImage, BWImage, 127, 255, CV_THRESH_BINARY);
//ShowImage(BWImage, "BWImage");
IplImage WorkingImage = cvCreateImage(cvGetSize(BWImage), IPL_DEPTH_8U, 1);
cvErode(BWImage, WorkingImage, null, ErodeCount);
cvDilate(WorkingImage, WorkingImage, null, DilateCount);
//ShowImage(WorkingImage, "WorkingImage", 512);
//cvSaveImage("Working.jpg", WorkingImage);
//PrintGrayImage(WorkingImage, "WorkingImage");
//BinaryHistogram(WorkingImage);
Blobs Regions = new Blobs();
Regions.BlobAnalysis(
WorkingImage, // image
-1, -1, // ROI start col, row
-1, -1, // ROI cols, rows
1, // border (0 = black; 1 = white)
MinArea); // minarea
Regions.PrintRegionData();
for(int i = 1; i <= Blobs.MaxLabel; i++)
{
double [] Region = Blobs.RegionData[i];
int Parent = (int) Region[Blobs.BLOBPARENT];
int Color = (int) Region[Blobs.BLOBCOLOR];
int MinX = (int) Region[Blobs.BLOBMINX];
int MaxX = (int) Region[Blobs.BLOBMAXX];
int MinY = (int) Region[Blobs.BLOBMINY];
int MaxY = (int) Region[Blobs.BLOBMAXY];
Highlight(RawImage, MinX, MinY, MaxX, MaxY, 1);
}
ShowImage(RawImage, "RawImage", 512);
cvReleaseImage(GrayImage); GrayImage = null;
cvReleaseImage(BWImage); BWImage = null;
cvReleaseImage(WorkingImage); WorkingImage = null;
}
cvReleaseImage(RawImage); RawImage = null;
}
// Versions with 2, 3, and 4 parms respectively
public static void ShowImage(IplImage image, String caption)
{
CvMat mat = image.asCvMat();
int width = mat.cols(); if(width < 1) width = 1;
int height = mat.rows(); if(height < 1) height = 1;
double aspect = 1.0 * width / height;
if(height < 128) { height = 128; width = (int) ( height * aspect ); }
if(width < 128) width = 128;
height = (int) ( width / aspect );
ShowImage(image, caption, width, height);
}
public static void ShowImage(IplImage image, String caption, int size)
{
if(size < 128) size = 128;
CvMat mat = image.asCvMat();
int width = mat.cols(); if(width < 1) width = 1;
int height = mat.rows(); if(height < 1) height = 1;
double aspect = 1.0 * width / height;
if(height != size) { height = size; width = (int) ( height * aspect ); }
if(width != size) width = size;
height = (int) ( width / aspect );
ShowImage(image, caption, width, height);
}
public static void ShowImage(IplImage image, String caption, int width, int height)
{
CanvasFrame canvas = new CanvasFrame(caption, 1); // gamma=1
canvas.setDefaultCloseOperation(javax.swing.JFrame.EXIT_ON_CLOSE);
canvas.setCanvasSize(width, height);
OpenCVFrameConverter converter = new OpenCVFrameConverter.ToIplImage();
canvas.showImage(converter.convert(image));
}
public static void Highlight(IplImage image, int [] inVec)
{
Highlight(image, inVec[0], inVec[1], inVec[2], inVec[3], 1);
}
public static void Highlight(IplImage image, int [] inVec, int Thick)
{
Highlight(image, inVec[0], inVec[1], inVec[2], inVec[3], Thick);
}
public static void Highlight(IplImage image, int xMin, int yMin, int xMax, int yMax)
{
Highlight(image, xMin, yMin, xMax, yMax, 1);
}
public static void Highlight(IplImage image, int xMin, int yMin, int xMax, int yMax, int Thick)
{
CvPoint pt1 = cvPoint(xMin,yMin);
CvPoint pt2 = cvPoint(xMax,yMax);
CvScalar color = cvScalar(255,0,0,0); // blue [green] [red]
cvRectangle(image, pt1, pt2, color, Thick, 4, 0);
}
public static void PrintGrayImage(IplImage image, String caption)
{
int size = 512; // impractical to print anything larger
CvMat mat = image.asCvMat();
int cols = mat.cols(); if(cols < 1) cols = 1;
int rows = mat.rows(); if(rows < 1) rows = 1;
double aspect = 1.0 * cols / rows;
if(rows > size) { rows = size; cols = (int) ( rows * aspect ); }
if(cols > size) cols = size;
rows = (int) ( cols / aspect );
PrintGrayImage(image, caption, 0, cols, 0, rows);
}
public static void PrintGrayImage(IplImage image, String caption, int MinX, int MaxX, int MinY, int MaxY)
{
int size = 512; // impractical to print anything larger
CvMat mat = image.asCvMat();
int cols = mat.cols(); if(cols < 1) cols = 1;
int rows = mat.rows(); if(rows < 1) rows = 1;
if(MinX < 0) MinX = 0; if(MinX > cols) MinX = cols;
if(MaxX < 0) MaxX = 0; if(MaxX > cols) MaxX = cols;
if(MinY < 0) MinY = 0; if(MinY > rows) MinY = rows;
if(MaxY < 0) MaxY = 0; if(MaxY > rows) MaxY = rows;
System.out.println("\n" + caption);
System.out.print(" +");
for(int icol = MinX; icol < MaxX; icol++) System.out.print("-");
System.out.println("+");
for(int irow = MinY; irow < MaxY; irow++)
{
if(irow<10) System.out.print(" ");
if(irow<100) System.out.print(" ");
System.out.print(irow);
System.out.print("|");
for(int icol = MinX; icol < MaxX; icol++)
{
int val = (int) mat.get(irow,icol);
String C = " ";
if(val == 0) C = "*";
System.out.print(C);
}
System.out.println("|");
}
System.out.print(" +");
for(int icol = MinX; icol < MaxX; icol++) System.out.print("-");
System.out.println("+");
}
public static void PrintImageProperties(IplImage image)
{
CvMat mat = image.asCvMat();
int cols = mat.cols();
int rows = mat.rows();
int depth = mat.depth();
System.out.println("ImageProperties for " + image + " : cols=" + cols + " rows=" + rows + " depth=" + depth);
}
public static float BinaryHistogram(IplImage image)
{
CvScalar Sum = cvSum(image);
float WhitePixels = (float) ( Sum.getVal(0) / 255 );
CvMat mat = image.asCvMat();
float TotalPixels = mat.cols() * mat.rows();
//float BlackPixels = TotalPixels - WhitePixels;
return WhitePixels / TotalPixels;
}
// Counterclockwise small angle rotation by skewing - Does not stretch border pixels
public static IplImage SkewGrayImage(IplImage Src, double angle) // angle is in radians
{
//double radians = - Math.PI * angle / 360.0; // Half because skew is horizontal and vertical
double sin = - Math.sin(angle);
double AbsSin = Math.abs(sin);
int nChannels = Src.nChannels();
if(nChannels != 1)
{
System.out.println("ERROR: SkewGrayImage: Require 1 channel: nChannels=" + nChannels);
System.exit(1);
}
CvMat SrcMat = Src.asCvMat();
int SrcCols = SrcMat.cols();
int SrcRows = SrcMat.rows();
double WidthSkew = AbsSin * SrcRows;
double HeightSkew = AbsSin * SrcCols;
int DstCols = (int) ( SrcCols + WidthSkew );
int DstRows = (int) ( SrcRows + HeightSkew );
CvMat DstMat = cvCreateMat(DstRows, DstCols, CV_8UC1); // Type matches IPL_DEPTH_8U
cvSetZero(DstMat);
cvNot(DstMat, DstMat);
for(int irow = 0; irow < DstRows; irow++)
{
int dcol = (int) ( WidthSkew * irow / SrcRows );
for(int icol = 0; icol < DstCols; icol++)
{
int drow = (int) ( HeightSkew - HeightSkew * icol / SrcCols );
int jrow = irow - drow;
int jcol = icol - dcol;
if(jrow < 0 || jcol < 0 || jrow >= SrcRows || jcol >= SrcCols) DstMat.put(irow, icol, 255);
else DstMat.put(irow, icol, (int) SrcMat.get(jrow,jcol));
}
}
IplImage Dst = cvCreateImage(cvSize(DstCols, DstRows), IPL_DEPTH_8U, 1);
Dst = DstMat.asIplImage();
return Dst;
}
public static IplImage TransposeImage(IplImage SrcImage)
{
CvMat mat = SrcImage.asCvMat();
int cols = mat.cols();
int rows = mat.rows();
IplImage DstImage = cvCreateImage(cvSize(rows, cols), IPL_DEPTH_8U, 1);
cvTranspose(SrcImage, DstImage);
cvFlip(DstImage,DstImage,1);
return DstImage;
}
}
================================================
FILE: samples/CaffeGooglenet.java
================================================
/*
* JavaCV version of OpenCV caffe_googlenet.cpp
* https://github.com/ludv1x/opencv_contrib/blob/master/modules/dnn/samples/caffe_googlenet.cpp
*
* Paolo Bolettieri
*/
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_dnn.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_dnn.*;
import static org.bytedeco.opencv.global.opencv_imgcodecs.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
public class CaffeGooglenet {
/* Find best class for the blob (i. e. class with maximal probability) */
public static void getMaxClass(Mat probBlob, Point classId, double[] classProb) {
Mat probMat = probBlob.reshape(1, 1); //reshape the blob to 1x1000 matrix
minMaxLoc(probMat, null, classProb, null, classId, null);
}
public static List readClassNames() {
String filename = "synset_words.txt";
List classNames = null;
try (BufferedReader br = new BufferedReader(new FileReader(new File(filename)))) {
classNames = new ArrayList();
String name = null;
while ((name = br.readLine()) != null) {
classNames.add(name.substring(name.indexOf(' ')+1));
}
} catch (IOException ex) {
System.err.println("File with classes labels not found " + filename);
System.exit(-1);
}
return classNames;
}
public static void main(String[] args) throws Exception {
String modelTxt = "bvlc_googlenet.prototxt";
String modelBin = "bvlc_googlenet.caffemodel";
String imageFile = (args.length > 0) ? args[0] : "space_shuttle.jpg";
//! [Initialize network]
Net net = null;
try { //Try to import Caffe GoogleNet model
net = readNetFromCaffe(modelTxt, modelBin);
} catch (Exception e) { //Importer can throw errors, we will catch them
e.printStackTrace();
}
if (net == null || net.empty()) {
System.err.println("Can't load network by using the following files: ");
System.err.println("prototxt: " + modelTxt);
System.err.println("caffemodel: " + modelBin);
System.err.println("bvlc_googlenet.caffemodel can be downloaded here:");
System.err.println("http://dl.caffe.berkeleyvision.org/bvlc_googlenet.caffemodel");
System.exit(-1);
}
//! [Initialize network]
//! [Prepare blob]
Mat img = imread(imageFile);
if (img.empty()) {
System.err.println("Can't read image from the file: " + imageFile);
System.exit(-1);
}
resize(img, img, new Size(224, 224)); //GoogLeNet accepts only 224x224 RGB-images
Mat inputBlob = blobFromImage(img); //Convert Mat to 4-dimensional dnn::Blob from image
//! [Prepare blob]
//! [Set input blob]
net.setInput(inputBlob, "data", 1.0, null); //set the network input
//! [Set input blob]
//! [Make forward pass]
Mat prob = net.forward("prob"); //compute output
//! [Make forward pass]
//! [Gather output]
Point classId = new Point();
double[] classProb = new double[1];
getMaxClass(prob, classId, classProb);//find the best class
//! [Gather output]
//! [Print results]
List classNames = readClassNames();
System.out.println("Best class: #" + classId.x() + " '" + classNames.get(classId.x()) + "'");
System.out.println("Best class: #" + classId.x());
System.out.println("Probability: " + classProb[0] * 100 + "%");
//! [Print results]
} //main
}
================================================
FILE: samples/ColoredObjectTrack.java
================================================
/*
* Just an example using the opencv to make a colored object tracking,
* i adpted this code to bytedeco/javacv, i think this will help some people.
*
* Waldemar
*/
import java.awt.Color;
import java.awt.Graphics;
import java.awt.image.BufferedImage;
import javax.swing.JPanel;
import org.bytedeco.javacv.CanvasFrame;
import org.bytedeco.javacv.FrameGrabber;
import org.bytedeco.javacv.Java2DFrameConverter;
import org.bytedeco.javacv.OpenCVFrameConverter;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgcodecs.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
public class ColoredObjectTrack implements Runnable {
public static void main(String[] args) {
ColoredObjectTrack cot = new ColoredObjectTrack();
Thread th = new Thread(cot);
th.start();
}
final int INTERVAL = 10;// 1sec
final int CAMERA_NUM = 0; // Default camera for this time
/**
* Correct the color range- it depends upon the object, camera quality,
* environment.
*/
static CvScalar rgba_min = cvScalar(0, 0, 130, 0);// RED wide dabur birko
static CvScalar rgba_max = cvScalar(80, 80, 255, 0);
IplImage image;
CanvasFrame canvas = new CanvasFrame("Web Cam Live");
CanvasFrame path = new CanvasFrame("Detection");
int ii = 0;
JPanel jp = new JPanel();
public ColoredObjectTrack() {
canvas.setDefaultCloseOperation(javax.swing.JFrame.EXIT_ON_CLOSE);
path.setDefaultCloseOperation(javax.swing.JFrame.EXIT_ON_CLOSE);
path.setContentPane(jp);
}
@Override
public void run() {
try {
FrameGrabber grabber = FrameGrabber.createDefault(CAMERA_NUM);
OpenCVFrameConverter.ToIplImage converter = new OpenCVFrameConverter.ToIplImage();
grabber.start();
IplImage img;
int posX = 0;
int posY = 0;
while (true) {
img = converter.convert(grabber.grab());
if (img != null) {
// show image on window
cvFlip(img, img, 1);// l-r = 90_degrees_steps_anti_clockwise
canvas.showImage(converter.convert(img));
IplImage detectThrs = getThresholdImage(img);
CvMoments moments = new CvMoments();
cvMoments(detectThrs, moments, 1);
double mom10 = cvGetSpatialMoment(moments, 1, 0);
double mom01 = cvGetSpatialMoment(moments, 0, 1);
double area = cvGetCentralMoment(moments, 0, 0);
posX = (int) (mom10 / area);
posY = (int) (mom01 / area);
// only if its a valid position
if (posX > 0 && posY > 0) {
paint(img, posX, posY);
}
}
// Thread.sleep(INTERVAL);
}
} catch (Exception e) {
}
}
private void paint(IplImage img, int posX, int posY) {
Graphics g = jp.getGraphics();
path.setSize(img.width(), img.height());
// g.clearRect(0, 0, img.width(), img.height());
g.setColor(Color.RED);
// g.fillOval(posX, posY, 20, 20);
g.drawOval(posX, posY, 20, 20);
System.out.println(posX + " , " + posY);
}
private IplImage getThresholdImage(IplImage orgImg) {
IplImage imgThreshold = cvCreateImage(cvGetSize(orgImg), 8, 1);
//
cvInRangeS(orgImg, rgba_min, rgba_max, imgThreshold);// red
cvSmooth(imgThreshold, imgThreshold, CV_MEDIAN, 15,0,0,0);
cvSaveImage(++ii + "dsmthreshold.jpg", imgThreshold);
return imgThreshold;
}
public IplImage Equalize(BufferedImage bufferedimg) {
Java2DFrameConverter converter1 = new Java2DFrameConverter();
OpenCVFrameConverter.ToIplImage converter2 = new OpenCVFrameConverter.ToIplImage();
IplImage iploriginal = converter2.convert(converter1.convert(bufferedimg));
IplImage srcimg = IplImage.create(iploriginal.width(), iploriginal.height(), IPL_DEPTH_8U, 1);
IplImage destimg = IplImage.create(iploriginal.width(), iploriginal.height(), IPL_DEPTH_8U, 1);
cvCvtColor(iploriginal, srcimg, CV_BGR2GRAY);
cvEqualizeHist(srcimg, destimg);
return destimg;
}
}
================================================
FILE: samples/DeepLearningFaceDetection.java
================================================
import org.bytedeco.javacpp.indexer.FloatIndexer;
import org.bytedeco.javacv.CanvasFrame;
import org.bytedeco.javacv.OpenCVFrameConverter;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_dnn.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import org.bytedeco.opencv.opencv_videoio.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_dnn.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_videoio.*;
/**
* Created on Jul 28, 2018
*
* @author Taha Emara
* Email : taha@emaraic.com
*
* This example does face detection using deep learning model which provides a
* great accuracy compared to OpenCV face detection using Haar cascades.
*
* This example is based on this code
* https://github.com/opencv/opencv/blob/master/modules/dnn/misc/face_detector_accuracy.py
*
* To run this example you need two files: deploy.prototxt can be downloaded
* from
* https://github.com/opencv/opencv/blob/master/samples/dnn/face_detector/deploy.prototxt
*
* and res10_300x300_ssd_iter_140000.caffemodel
* https://github.com/opencv/opencv_3rdparty/blob/dnn_samples_face_detector_20170830/res10_300x300_ssd_iter_140000.caffemodel
*
*/
public class DeepLearningFaceDetection {
private static final String PROTO_FILE = "deploy.prototxt";
private static final String CAFFE_MODEL_FILE = "res10_300x300_ssd_iter_140000.caffemodel";
private static final OpenCVFrameConverter.ToIplImage converter = new OpenCVFrameConverter.ToIplImage();
private static Net net = null;
static {
net = readNetFromCaffe(PROTO_FILE, CAFFE_MODEL_FILE);
}
private static void detectAndDraw(Mat image) {//detect faces and draw a blue rectangle arroung each face
resize(image, image, new Size(300, 300));//resize the image to match the input size of the model
//create a 4-dimensional blob from image with NCHW (Number of images in the batch -for training only-, Channel, Height, Width) dimensions order,
//for more detailes read the official docs at https://docs.opencv.org/trunk/d6/d0f/group__dnn.html#gabd0e76da3c6ad15c08b01ef21ad55dd8
Mat blob = blobFromImage(image, 1.0, new Size(300, 300), new Scalar(104.0, 177.0, 123.0, 0), false, false, CV_32F);
net.setInput(blob);//set the input to network model
Mat output = net.forward();//feed forward the input to the netwrok to get the output matrix
Mat ne = new Mat(new Size(output.size(3), output.size(2)), CV_32F, output.ptr(0, 0));//extract a 2d matrix for 4d output matrix with form of (number of detections x 7)
FloatIndexer srcIndexer = ne.createIndexer(); // create indexer to access elements of the matric
for (int i = 0; i < output.size(3); i++) {//iterate to extract elements
float confidence = srcIndexer.get(i, 2);
float f1 = srcIndexer.get(i, 3);
float f2 = srcIndexer.get(i, 4);
float f3 = srcIndexer.get(i, 5);
float f4 = srcIndexer.get(i, 6);
if (confidence > .6) {
float tx = f1 * 300;//top left point's x
float ty = f2 * 300;//top left point's y
float bx = f3 * 300;//bottom right point's x
float by = f4 * 300;//bottom right point's y
rectangle(image, new Rect(new Point((int) tx, (int) ty), new Point((int) bx, (int) by)), new Scalar(255, 0, 0, 0));//print blue rectangle
}
}
}
public static void main(String[] args) {
VideoCapture capture = new VideoCapture();
capture.set(CAP_PROP_FRAME_WIDTH, 1280);
capture.set(CAP_PROP_FRAME_HEIGHT, 720);
if (!capture.open(0)) {
System.out.println("Can not open the cam !!!");
}
Mat colorimg = new Mat();
CanvasFrame mainframe = new CanvasFrame("Face Detection", CanvasFrame.getDefaultGamma() / 2.2);
mainframe.setDefaultCloseOperation(javax.swing.JFrame.EXIT_ON_CLOSE);
mainframe.setCanvasSize(600, 600);
mainframe.setLocationRelativeTo(null);
mainframe.setVisible(true);
while (true) {
while (capture.read(colorimg) && mainframe.isVisible()) {
detectAndDraw(colorimg);
mainframe.showImage(converter.convert(colorimg));
try {
Thread.sleep(50);
} catch (InterruptedException ex) {
System.out.println(ex.getMessage());
}
}
}
}
}
================================================
FILE: samples/DeinterlacedVideoPlayer.java
================================================
import org.bytedeco.javacpp.Loader;
import org.bytedeco.javacv.FFmpegFrameFilter;
import org.bytedeco.javacv.Frame;
import org.bytedeco.javacv.FrameFilter;
import org.bytedeco.javacv.FrameGrabber;
import org.bytedeco.javacv.FrameGrabber.Exception;
import org.bytedeco.javacv.OpenCVFrameGrabber;
import org.bytedeco.ffmpeg.global.avutil;
public class DeinterlacedVideoPlayer {
private static final int DEVICE_ID = 0;
private static final int WIDTH = 640;
private static final int HEIGHT = 480;
private static final int FRAMERATE = 25;
private static final int PIXEL_FORMAT = avutil.AV_PIX_FMT_BGR24;;
private String ffmpegString = "yadif=mode=0:parity=-1:deint=0,format=bgr24";
private FrameGrabber grabber;
public DeinterlacedVideoPlayer() {}
public void start() {
FrameFilter filter = null;
try {
startFrameGrabber();
Frame frame = null;
while ((frame = grabber.grab()) != null) {
if (filter == null) {
filter = new FFmpegFrameFilter(ffmpegString, frame.imageWidth, frame.imageHeight);
filter.setPixelFormat(PIXEL_FORMAT);
filter.start();
}
filter.push(frame);
frame = filter.pull();
// do something with the filtered frame
}
} catch (Exception | org.bytedeco.javacv.FrameFilter.Exception e) {
throw new RuntimeException(e.getMessage(), e);
} finally {
releaseGrabberAndFilter(this.grabber, filter);
}
}
private void startFrameGrabber() throws Exception {
grabber = new OpenCVFrameGrabber(DEVICE_ID);
grabber.setImageWidth(WIDTH);
grabber.setImageHeight(HEIGHT);
grabber.setFrameRate(FRAMERATE);
grabber.setPixelFormat(PIXEL_FORMAT);
grabber.start();
}
private void releaseGrabberAndFilter(FrameGrabber grabber, FrameFilter filter) {
try {
if (grabber != null) {
grabber.release();
}
} catch (Exception e) {
throw new RuntimeException("Cannot release frame grabber!", e);
} finally {
releaseFilter(filter);
}
}
private void releaseFilter(FrameFilter filter) {
if (filter == null) {
return;
}
try {
filter.close();
} catch (org.bytedeco.javacv.FrameFilter.Exception e) {
throw new RuntimeException("Cannot close frame filter!", e);
}
}
}
================================================
FILE: samples/Demo.java
================================================
/*
* Copyright (C) 2009-2018 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import java.io.File;
import java.net.URL;
import org.bytedeco.javacv.*;
import org.bytedeco.javacpp.*;
import org.bytedeco.javacpp.indexer.*;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import org.bytedeco.opencv.opencv_calib3d.*;
import org.bytedeco.opencv.opencv_objdetect.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_calib3d.*;
import static org.bytedeco.opencv.global.opencv_objdetect.*;
public class Demo {
public static void main(String[] args) throws Exception {
String classifierName = null;
if (args.length > 0) {
classifierName = args[0];
} else {
URL url = new URL("https://raw.github.com/opencv/opencv/master/data/haarcascades/haarcascade_frontalface_alt.xml");
File file = Loader.cacheResource(url);
classifierName = file.getAbsolutePath();
}
// We can "cast" Pointer objects by instantiating a new object of the desired class.
CascadeClassifier classifier = new CascadeClassifier(classifierName);
if (classifier == null) {
System.err.println("Error loading classifier file \"" + classifierName + "\".");
System.exit(1);
}
// The available FrameGrabber classes include OpenCVFrameGrabber (opencv_videoio),
// DC1394FrameGrabber, FlyCapture2FrameGrabber, OpenKinectFrameGrabber, OpenKinect2FrameGrabber,
// RealSenseFrameGrabber, RealSense2FrameGrabber, PS3EyeFrameGrabber, VideoInputFrameGrabber, and FFmpegFrameGrabber.
FrameGrabber grabber = FrameGrabber.createDefault(0);
grabber.start();
// CanvasFrame, FrameGrabber, and FrameRecorder use Frame objects to communicate image data.
// We need a FrameConverter to interface with other APIs (Android, Java 2D, JavaFX, Tesseract, OpenCV, etc).
OpenCVFrameConverter.ToMat converter = new OpenCVFrameConverter.ToMat();
// FAQ about IplImage and Mat objects from OpenCV:
// - For custom raw processing of data, createBuffer() returns an NIO direct
// buffer wrapped around the memory pointed by imageData, and under Android we can
// also use that Buffer with Bitmap.copyPixelsFromBuffer() and copyPixelsToBuffer().
// - To get a BufferedImage from an IplImage, or vice versa, we can chain calls to
// Java2DFrameConverter and OpenCVFrameConverter, one after the other.
// - Java2DFrameConverter also has static copy() methods that we can use to transfer
// data more directly between BufferedImage and IplImage or Mat via Frame objects.
Mat grabbedImage = converter.convert(grabber.grab());
int height = grabbedImage.rows();
int width = grabbedImage.cols();
// Objects allocated with `new`, clone(), or a create*() factory method are automatically released
// by the garbage collector, but may still be explicitly released by calling deallocate().
// You shall NOT call cvReleaseImage(), cvReleaseMemStorage(), etc. on objects allocated this way.
Mat grayImage = new Mat(height, width, CV_8UC1);
Mat rotatedImage = grabbedImage.clone();
// The OpenCVFrameRecorder class simply uses the VideoWriter of opencv_videoio,
// but FFmpegFrameRecorder also exists as a more versatile alternative.
FrameRecorder recorder = FrameRecorder.createDefault("output.avi", width, height);
recorder.start();
// CanvasFrame is a JFrame containing a Canvas component, which is hardware accelerated.
// It can also switch into full-screen mode when called with a screenNumber.
// We should also specify the relative monitor/camera response for proper gamma correction.
CanvasFrame frame = new CanvasFrame("Some Title", CanvasFrame.getDefaultGamma()/grabber.getGamma());
// Let's create some random 3D rotation...
Mat randomR = new Mat(3, 3, CV_64FC1),
randomAxis = new Mat(3, 1, CV_64FC1);
// We can easily and efficiently access the elements of matrices and images
// through an Indexer object with the set of get() and put() methods.
DoubleIndexer Ridx = randomR.createIndexer(),
axisIdx = randomAxis.createIndexer();
axisIdx.put(0, (Math.random() - 0.5) / 4,
(Math.random() - 0.5) / 4,
(Math.random() - 0.5) / 4);
Rodrigues(randomAxis, randomR);
double f = (width + height) / 2.0; Ridx.put(0, 2, Ridx.get(0, 2) * f);
Ridx.put(1, 2, Ridx.get(1, 2) * f);
Ridx.put(2, 0, Ridx.get(2, 0) / f); Ridx.put(2, 1, Ridx.get(2, 1) / f);
System.out.println(Ridx);
// We can allocate native arrays using constructors taking an integer as argument.
Point hatPoints = new Point(3);
while (frame.isVisible() && (grabbedImage = converter.convert(grabber.grab())) != null) {
// Let's try to detect some faces! but we need a grayscale image...
cvtColor(grabbedImage, grayImage, CV_BGR2GRAY);
RectVector faces = new RectVector();
classifier.detectMultiScale(grayImage, faces);
long total = faces.size();
for (long i = 0; i < total; i++) {
Rect r = faces.get(i);
int x = r.x(), y = r.y(), w = r.width(), h = r.height();
rectangle(grabbedImage, new Point(x, y), new Point(x + w, y + h), Scalar.RED, 1, CV_AA, 0);
// To access or pass as argument the elements of a native array, call position() before.
hatPoints.position(0).x(x - w / 10 ).y(y - h / 10);
hatPoints.position(1).x(x + w * 11 / 10).y(y - h / 10);
hatPoints.position(2).x(x + w / 2 ).y(y - h / 2 );
fillConvexPoly(grabbedImage, hatPoints.position(0), 3, Scalar.GREEN, CV_AA, 0);
}
// Let's find some contours! but first some thresholding...
threshold(grayImage, grayImage, 64, 255, CV_THRESH_BINARY);
// To check if an output argument is null we may call either isNull() or equals(null).
MatVector contours = new MatVector();
findContours(grayImage, contours, CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE);
long n = contours.size();
for (long i = 0; i < n; i++) {
Mat contour = contours.get(i);
Mat points = new Mat();
approxPolyDP(contour, points, arcLength(contour, true) * 0.02, true);
drawContours(grabbedImage, new MatVector(points), -1, Scalar.BLUE);
}
warpPerspective(grabbedImage, rotatedImage, randomR, rotatedImage.size());
Frame rotatedFrame = converter.convert(rotatedImage);
frame.showImage(rotatedFrame);
recorder.record(rotatedFrame);
}
frame.dispose();
recorder.stop();
grabber.stop();
}
}
================================================
FILE: samples/FFmpegStreamingTimeout.java
================================================
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import org.bytedeco.javacpp.Pointer;
import org.bytedeco.javacv.FFmpegFrameGrabber;
import org.bytedeco.javacv.Frame;
import org.bytedeco.javacv.FrameGrabber;
import org.bytedeco.ffmpeg.avformat.*;
import static org.bytedeco.ffmpeg.global.avformat.*;
/**
*
* @author Dmitriy Gerashenko
*/
public class FFmpegStreamingTimeout {
/**
* There is no universal option for streaming timeout. Each of protocols has
* its own list of options.
*/
private static enum TimeoutOption {
/**
* Depends on protocol (FTP, HTTP, RTMP, RTSP, SMB, SSH, TCP, UDP, or UNIX).
* http://ffmpeg.org/ffmpeg-all.html
*
* Specific for RTSP:
* Set socket TCP I/O timeout in microseconds.
* http://ffmpeg.org/ffmpeg-all.html#rtsp
*/
TIMEOUT,
/**
* Protocols
*
* Maximum time to wait for (network) read/write operations to complete,
* in microseconds.
*
* http://ffmpeg.org/ffmpeg-all.html#Protocols
*/
RW_TIMEOUT;
public String getKey() {
return toString().toLowerCase();
}
}
private static final String SOURCE_RTSP = "rtsp://184.72.239.149/vod/mp4:BigBuckBunny_115k.mov";
private static final int TIMEOUT = 10; // In seconds.
public static void main(String[] args) {
rtspStreamingTest();
// testWithCallback(); // This is not working properly. It's just for test.
}
private static void rtspStreamingTest() {
try {
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(SOURCE_RTSP);
/**
* "rw_timeout" - IS IGNORED when a network cable have been
* unplugged before a connection but the option takes effect after a
* connection was established.
*
* "timeout" - works fine.
*/
grabber.setOption(
TimeoutOption.TIMEOUT.getKey(),
String.valueOf(TIMEOUT * 1000000)
); // In microseconds.
grabber.start();
Frame frame = null;
/**
* When network is disabled (before grabber was started) grabber
* throws exception: "org.bytedeco.javacv.FrameGrabber$Exception:
* avformat_open_input() error -138: Could not open input...".
*
* When connections is lost (after a few grabbed frames)
* grabber.grab() returns null without exception.
*/
while ((frame = grabber.grab()) != null) {
System.out.println("frame grabbed at " + grabber.getTimestamp());
}
System.out.println("loop end with frame: " + frame);
} catch (FrameGrabber.Exception ex) {
System.out.println("exception: " + ex);
}
System.out.println("end");
}
private static void testWithCallback() {
try {
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(SOURCE_RTSP);
/**
* grabber.getFormatContext() is null before grabber.start().
*
* But if network is disabled grabber.start() will never return.
*
* That's why interrupt_callback not suitable for "network disabled
* case".
*/
grabber.start();
final AtomicBoolean interruptFlag = new AtomicBoolean(false);
AVIOInterruptCB.Callback_Pointer cp = new AVIOInterruptCB.Callback_Pointer() {
@Override
public int call(Pointer pointer) {
// 0 - continue, 1 - exit
int interruptFlagInt = interruptFlag.get() ? 1 : 0;
System.out.println("callback, interrupt flag == " + interruptFlagInt);
return interruptFlagInt;
}
};
AVFormatContext oc = grabber.getFormatContext();
avformat_alloc_context();
AVIOInterruptCB cb = new AVIOInterruptCB();
cb.callback(cp);
oc.interrupt_callback(cb);
new Thread(new Runnable() { public void run() {
try {
TimeUnit.SECONDS.sleep(TIMEOUT);
interruptFlag.set(true);
System.out.println("interrupt flag was changed");
} catch (InterruptedException ex) {
System.out.println("exception in interruption thread: " + ex);
}
}}).start();
Frame frame = null;
/**
* On one of my RTSP cams grabber stops calling callback on
* connection lost. I think it's has something to do with message:
* "[swscaler @ 0000000029af49e0] deprecated pixel format used, make
* sure you did set range correctly".
*
* So there is at least one case when grabber stops calling
* callback.
*/
while ((frame = grabber.grab()) != null) {
System.out.println("frame grabbed at " + grabber.getTimestamp());
}
System.out.println("loop end with frame: " + frame);
} catch (FrameGrabber.Exception ex) {
System.out.println("exception: " + ex);
}
System.out.println("end");
}
}
================================================
FILE: samples/FaceApplet.html
================================================
FaceApplet
A browser with JavaScript enabled is required for this page to operate properly.
FaceApplet
================================================
FILE: samples/FaceApplet.java
================================================
import java.applet.Applet;
import java.awt.BasicStroke;
import java.awt.Color;
import java.awt.Graphics;
import java.awt.Graphics2D;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.javacv.Frame;
import org.bytedeco.javacv.FrameGrabber;
import org.bytedeco.javacv.Java2DFrameConverter;
import org.bytedeco.javacv.OpenCVFrameConverter;
import org.bytedeco.javacv.OpenCVFrameGrabber;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import org.bytedeco.opencv.opencv_objdetect.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_objdetect.*;
/**
*
* @author Samuel Audet
*/
public class FaceApplet extends Applet implements Runnable {
private CvHaarClassifierCascade classifier = null;
private CvMemStorage storage = null;
private FrameGrabber grabber = null;
private IplImage grabbedImage = null, grayImage = null, smallImage = null;
private CvSeq faces = null;
private boolean stop = false;
private Exception exception = null;
OpenCVFrameConverter.ToIplImage grabberConverter = new OpenCVFrameConverter.ToIplImage();
Java2DFrameConverter paintConverter = new Java2DFrameConverter();
@Override public void init() {
try {
// Load the classifier file from Java resources.
String classiferName = "haarcascade_frontalface_alt.xml";
File classifierFile = Loader.extractResource(classiferName, null, "classifier", ".xml");
if (classifierFile == null || classifierFile.length() <= 0) {
throw new IOException("Could not extract \"" + classiferName + "\" from Java resources.");
}
// Preload the opencv_objdetect module to work around a known bug.
Loader.load(opencv_objdetect.class);
classifier = new CvHaarClassifierCascade(cvLoad(classifierFile.getAbsolutePath()));
classifierFile.delete();
if (classifier.isNull()) {
throw new IOException("Could not load the classifier file.");
}
storage = CvMemStorage.create();
} catch (Exception e) {
if (exception == null) {
exception = e;
repaint();
}
}
}
@Override public void start() {
try {
new Thread(this).start();
} catch (Exception e) {
if (exception == null) {
exception = e;
repaint();
}
}
}
public void run() {
try {
try {
grabber = FrameGrabber.createDefault(0);
grabber.setImageWidth(getWidth());
grabber.setImageHeight(getHeight());
grabber.start();
grabbedImage = grabberConverter.convert(grabber.grab());
} catch (Exception e) {
if (grabber != null) grabber.release();
grabber = new OpenCVFrameGrabber(0);
grabber.setImageWidth(getWidth());
grabber.setImageHeight(getHeight());
grabber.start();
grabbedImage = grabberConverter.convert(grabber.grab());
}
grayImage = IplImage.create(grabbedImage.width(), grabbedImage.height(), IPL_DEPTH_8U, 1);
smallImage = IplImage.create(grabbedImage.width()/4, grabbedImage.height()/4, IPL_DEPTH_8U, 1);
stop = false;
while (!stop && (grabbedImage = grabberConverter.convert(grabber.grab())) != null) {
if (faces == null) {
cvClearMemStorage(storage);
cvCvtColor(grabbedImage, grayImage, CV_BGR2GRAY);
cvResize(grayImage, smallImage, CV_INTER_AREA);
faces = cvHaarDetectObjects(smallImage, classifier, storage, 1.1, 3,
CV_HAAR_FIND_BIGGEST_OBJECT | CV_HAAR_DO_ROUGH_SEARCH);
repaint();
}
}
grabbedImage = grayImage = smallImage = null;
grabber.stop();
grabber.release();
grabber = null;
} catch (Exception e) {
if (exception == null) {
exception = e;
repaint();
}
}
}
@Override public void update(Graphics g) {
paint(g);
}
@Override public void paint(Graphics g) {
if (grabbedImage != null) {
Frame frame = grabberConverter.convert(grabbedImage);
BufferedImage image = paintConverter.getBufferedImage(frame, 2.2/grabber.getGamma());
Graphics2D g2 = image.createGraphics();
if (faces != null) {
g2.setColor(Color.RED);
g2.setStroke(new BasicStroke(2));
int total = faces.total();
for (int i = 0; i < total; i++) {
CvRect r = new CvRect(cvGetSeqElem(faces, i));
g2.drawRect(r.x()*4, r.y()*4, r.width()*4, r.height()*4);
}
faces = null;
}
g.drawImage(image, 0, 0, null);
}
if (exception != null) {
int y = 0, h = g.getFontMetrics().getHeight();
g.drawString(exception.toString(), 5, y += h);
for (StackTraceElement e : exception.getStackTrace()) {
g.drawString(" at " + e.toString(), 5, y += h);
}
}
}
@Override public void stop() {
stop = true;
}
@Override public void destroy() { }
}
================================================
FILE: samples/FaceApplet.jnlp
================================================
FaceApplet
Samuel Audet
================================================
FILE: samples/FacePreview.java
================================================
/*
* Copyright (C) 2010-2019 Samuel Audet
*
* FacePreview - A fusion of OpenCV's facedetect and Android's CameraPreview samples,
* with JavaCV + JavaCPP as the glue in between.
*
* This file was based on CameraPreview.java that came with the Samples for
* Android SDK API 8, revision 1 and contained the following copyright notice:
*
* Copyright (C) 2007 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*
* IMPORTANT - Make sure the AndroidManifest.xml file looks like this:
*
*
*
*
*
*
*
*
*
*
*/
package org.bytedeco.javacv.facepreview;
import android.app.Activity;
import android.app.AlertDialog;
import android.content.Context;
import android.graphics.Canvas;
import android.graphics.Color;
import android.graphics.ImageFormat;
import android.graphics.Paint;
import android.hardware.Camera;
import android.hardware.Camera.Size;
import android.os.Bundle;
import android.view.SurfaceHolder;
import android.view.SurfaceView;
import android.view.View;
import android.view.Window;
import android.view.WindowManager;
import android.widget.FrameLayout;
import java.io.File;
import java.io.IOException;
import java.net.URL;
import java.nio.ByteBuffer;
import java.util.List;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_objdetect.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_objdetect.*;
// ----------------------------------------------------------------------
public class FacePreview extends Activity {
private FrameLayout layout;
private FaceView faceView;
private Preview mPreview;
@Override
protected void onCreate(Bundle savedInstanceState) {
// Hide the window title.
requestWindowFeature(Window.FEATURE_NO_TITLE);
super.onCreate(savedInstanceState);
getWindow().addFlags(WindowManager.LayoutParams.FLAG_FULLSCREEN);
// Create our Preview view and set it as the content of our activity.
try {
layout = new FrameLayout(this);
faceView = new FaceView(this);
mPreview = new Preview(this, faceView);
layout.addView(mPreview);
layout.addView(faceView);
setContentView(layout);
} catch (IOException e) {
e.printStackTrace();
new AlertDialog.Builder(this).setMessage(e.getMessage()).create().show();
}
}
}
// ----------------------------------------------------------------------
class FaceView extends View implements Camera.PreviewCallback {
public static final int SUBSAMPLING_FACTOR = 4;
private Mat grayImage;
private CascadeClassifier classifier;
private RectVector faces;
public FaceView(FacePreview context) throws IOException {
super(context);
// Load the classifier file from Java resources.
File classifierFile = Loader.extractResource(getClass(),
"/org/bytedeco/javacv/facepreview/haarcascade_frontalface_alt.xml",
context.getCacheDir(), "classifier", ".xml");
if (classifierFile == null || classifierFile.length() <= 0) {
throw new IOException("Could not extract the classifier file from Java resource.");
}
classifier = new CascadeClassifier(classifierFile.getAbsolutePath());
classifierFile.delete();
if (classifier.isNull()) {
throw new IOException("Could not load the classifier file.");
}
}
public void onPreviewFrame(final byte[] data, final Camera camera) {
try {
Camera.Size size = camera.getParameters().getPreviewSize();
processImage(data, size.width, size.height);
camera.addCallbackBuffer(data);
} catch (RuntimeException e) {
// The camera has probably just been released, ignore.
}
}
protected void processImage(byte[] data, int width, int height) {
// First, downsample our image and convert it into a grayscale Mat
int f = SUBSAMPLING_FACTOR;
if (grayImage == null || grayImage.cols() != width/f || grayImage.rows() != height/f) {
grayImage = new Mat(height/f, width/f, CV_8UC1);
}
int imageWidth = grayImage.cols();
int imageHeight = grayImage.rows();
int dataStride = f*width;
int imageStride = (int)grayImage.step(0);
ByteBuffer imageBuffer = grayImage.createBuffer();
for (int y = 0; y < imageHeight; y++) {
int dataLine = y*dataStride;
int imageLine = y*imageStride;
for (int x = 0; x < imageWidth; x++) {
imageBuffer.put(imageLine + x, data[dataLine + f*x]);
}
}
faces = new RectVector();
classifier.detectMultiScale(grayImage, faces);
postInvalidate();
}
@Override
protected void onDraw(Canvas canvas) {
Paint paint = new Paint();
paint.setColor(Color.RED);
paint.setTextSize(20);
String s = "FacePreview - This side up.";
float textWidth = paint.measureText(s);
canvas.drawText(s, (getWidth()-textWidth)/2, 20, paint);
if (faces != null) {
paint.setStrokeWidth(2);
paint.setStyle(Paint.Style.STROKE);
float scaleX = (float)getWidth()/grayImage.cols();
float scaleY = (float)getHeight()/grayImage.rows();
long total = faces.size();
for (long i = 0; i < total; i++) {
Rect r = faces.get(i);
int x = r.x(), y = r.y(), w = r.width(), h = r.height();
canvas.drawRect(x*scaleX, y*scaleY, (x+w)*scaleX, (y+h)*scaleY, paint);
}
}
}
}
// ----------------------------------------------------------------------
class Preview extends SurfaceView implements SurfaceHolder.Callback {
SurfaceHolder mHolder;
Camera mCamera;
Camera.PreviewCallback previewCallback;
Preview(Context context, Camera.PreviewCallback previewCallback) {
super(context);
this.previewCallback = previewCallback;
// Install a SurfaceHolder.Callback so we get notified when the
// underlying surface is created and destroyed.
mHolder = getHolder();
mHolder.addCallback(this);
mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
}
public void surfaceCreated(SurfaceHolder holder) {
// The Surface has been created, acquire the camera and tell it where
// to draw.
mCamera = Camera.open();
try {
mCamera.setPreviewDisplay(holder);
} catch (IOException exception) {
mCamera.release();
mCamera = null;
// TODO: add more exception handling logic here
}
}
public void surfaceDestroyed(SurfaceHolder holder) {
// Surface will be destroyed when we return, so stop the preview.
// Because the CameraDevice object is not a shared resource, it's very
// important to release it when the activity is paused.
mCamera.stopPreview();
mCamera.release();
mCamera = null;
}
private Size getOptimalPreviewSize(List sizes, int w, int h) {
final double ASPECT_TOLERANCE = 0.05;
double targetRatio = (double) w / h;
if (sizes == null) return null;
Size optimalSize = null;
double minDiff = Double.MAX_VALUE;
int targetHeight = h;
// Try to find an size match aspect ratio and size
for (Size size : sizes) {
double ratio = (double) size.width / size.height;
if (Math.abs(ratio - targetRatio) > ASPECT_TOLERANCE) continue;
if (Math.abs(size.height - targetHeight) < minDiff) {
optimalSize = size;
minDiff = Math.abs(size.height - targetHeight);
}
}
// Cannot find the one match the aspect ratio, ignore the requirement
if (optimalSize == null) {
minDiff = Double.MAX_VALUE;
for (Size size : sizes) {
if (Math.abs(size.height - targetHeight) < minDiff) {
optimalSize = size;
minDiff = Math.abs(size.height - targetHeight);
}
}
}
return optimalSize;
}
public void surfaceChanged(SurfaceHolder holder, int format, int w, int h) {
// Now that the size is known, set up the camera parameters and begin
// the preview.
Camera.Parameters parameters = mCamera.getParameters();
List sizes = parameters.getSupportedPreviewSizes();
Size optimalSize = getOptimalPreviewSize(sizes, w, h);
parameters.setPreviewSize(optimalSize.width, optimalSize.height);
mCamera.setParameters(parameters);
if (previewCallback != null) {
mCamera.setPreviewCallbackWithBuffer(previewCallback);
Camera.Size size = parameters.getPreviewSize();
byte[] data = new byte[size.width*size.height*
ImageFormat.getBitsPerPixel(parameters.getPreviewFormat())/8];
mCamera.addCallbackBuffer(data);
}
mCamera.startPreview();
}
}
================================================
FILE: samples/FaceRecognizerInVideo.java
================================================
import java.io.File;
import org.bytedeco.javacpp.IntPointer;
import org.bytedeco.javacpp.DoublePointer;
import org.bytedeco.javacv.Frame;
import org.bytedeco.javacv.FrameGrabber.Exception;
import org.bytedeco.javacv.OpenCVFrameConverter;
import org.bytedeco.javacv.OpenCVFrameGrabber;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_face.*;
import org.bytedeco.opencv.opencv_highgui.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import org.bytedeco.opencv.opencv_objdetect.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_face.*;
import static org.bytedeco.opencv.global.opencv_highgui.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_objdetect.*;
/**
* This is an example how to detect face in a video file with javacv
* @author Vincent He (chinadragon0515@gmail.com)
*
*/
public class FaceRecognizerInVideo {
public static void main(String[] args) throws Exception {
OpenCVFrameConverter.ToMat converterToMat = new OpenCVFrameConverter.ToMat();
if (args.length < 2) {
System.out.println("Two parameters are required to run this program, first parameter is the analized video and second parameter is the trained result for fisher faces.");
}
String videoFileName = args[0];
String trainedResult = args[1];
CascadeClassifier face_cascade = new CascadeClassifier(
"data\\haarcascade_frontalface_default.xml");
FaceRecognizer lbphFaceRecognizer = LBPHFaceRecognizer.create();
lbphFaceRecognizer.read(trainedResult);
File f = new File(videoFileName);
OpenCVFrameGrabber grabber = null;
try {
grabber = OpenCVFrameGrabber.createDefault(f);
grabber.start();
} catch (Exception e) {
System.err.println("Failed start the grabber.");
}
Frame videoFrame = null;
Mat videoMat = new Mat();
while (true) {
videoFrame = grabber.grab();
videoMat = converterToMat.convert(videoFrame);
Mat videoMatGray = new Mat();
// Convert the current frame to grayscale:
cvtColor(videoMat, videoMatGray, COLOR_BGRA2GRAY);
equalizeHist(videoMatGray, videoMatGray);
Point p = new Point();
RectVector faces = new RectVector();
// Find the faces in the frame:
face_cascade.detectMultiScale(videoMatGray, faces);
// At this point you have the position of the faces in
// faces. Now we'll get the faces, make a prediction and
// annotate it in the video. Cool or what?
for (int i = 0; i < faces.size(); i++) {
Rect face_i = faces.get(i);
Mat face = new Mat(videoMatGray, face_i);
// If fisher face recognizer is used, the face need to be
// resized.
// resize(face, face_resized, new Size(im_width, im_height),
// 1.0, 1.0, INTER_CUBIC);
// Now perform the prediction, see how easy that is:
IntPointer label = new IntPointer(1);
DoublePointer confidence = new DoublePointer(1);
lbphFaceRecognizer.predict(face, label, confidence);
int prediction = label.get(0);
// And finally write all we've found out to the original image!
// First of all draw a green rectangle around the detected face:
rectangle(videoMat, face_i, new Scalar(0, 255, 0, 1));
// Create the text we will annotate the box with:
String box_text = "Prediction = " + prediction;
// Calculate the position for annotated text (make sure we don't
// put illegal values in there):
int pos_x = Math.max(face_i.tl().x() - 10, 0);
int pos_y = Math.max(face_i.tl().y() - 10, 0);
// And now put it into the image:
putText(videoMat, box_text, new Point(pos_x, pos_y),
FONT_HERSHEY_PLAIN, 1.0, new Scalar(0, 255, 0, 2.0));
}
// Show the result:
imshow("face_recognizer", videoMat);
char key = (char) waitKey(20);
// Exit this loop on escape:
if (key == 27) {
destroyAllWindows();
break;
}
}
}
}
================================================
FILE: samples/HoughLines.java
================================================
import javax.swing.JFrame;
import org.bytedeco.javacpp.*;
import org.bytedeco.javacv.*;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_imgcodecs.*;
/**
* C to Java translation of the houghlines.c sample provided in the c sample directory of OpenCV 2.1,
* using the JavaCV Java wrapper of OpenCV 2.2 developped by Samuel Audet.
*
* @author Jeremy Nicola
* jeremy.nicola@gmail.com
*/
public class HoughLines {
/**
* usage: java HoughLines imageDir\imageName TransformType
*/
public static void main(String[] args) {
String fileName = args.length >= 1 ? args[0] : "pic1.png"; // if no params provided, compute the defaut image
IplImage src = cvLoadImage(fileName, 0);
IplImage dst;
IplImage colorDst;
CvMemStorage storage = cvCreateMemStorage(0);
CvSeq lines = new CvSeq();
CanvasFrame source = new CanvasFrame("Source");
CanvasFrame hough = new CanvasFrame("Hough");
OpenCVFrameConverter.ToIplImage sourceConverter = new OpenCVFrameConverter.ToIplImage();
OpenCVFrameConverter.ToIplImage houghConverter = new OpenCVFrameConverter.ToIplImage();
if (src == null) {
System.out.println("Couldn't load source image.");
return;
}
dst = cvCreateImage(cvGetSize(src), src.depth(), 1);
colorDst = cvCreateImage(cvGetSize(src), src.depth(), 3);
cvCanny(src, dst, 50, 200, 3);
cvCvtColor(dst, colorDst, CV_GRAY2BGR);
/*
* apply the probabilistic hough transform
* which returns for each line deteced two points ((x1, y1); (x2,y2))
* defining the detected segment
*/
if (args.length == 2 && args[1].contentEquals("probabilistic")) {
System.out.println("Using the Probabilistic Hough Transform");
lines = cvHoughLines2(dst, storage, CV_HOUGH_PROBABILISTIC, 1, Math.PI / 180, 40, 50, 10, 0, CV_PI);
for (int i = 0; i < lines.total(); i++) {
// Based on JavaCPP, the equivalent of the C code:
// CvPoint* line = (CvPoint*)cvGetSeqElem(lines,i);
// CvPoint first=line[0], second=line[1]
// is:
Pointer line = cvGetSeqElem(lines, i);
CvPoint pt1 = new CvPoint(line).position(0);
CvPoint pt2 = new CvPoint(line).position(1);
System.out.println("Line spotted: ");
System.out.println("\t pt1: " + pt1);
System.out.println("\t pt2: " + pt2);
cvLine(colorDst, pt1, pt2, CV_RGB(255, 0, 0), 3, CV_AA, 0); // draw the segment on the image
}
}
/*
* Apply the multiscale hough transform which returns for each line two float parameters (rho, theta)
* rho: distance from the origin of the image to the line
* theta: angle between the x-axis and the normal line of the detected line
*/
else if(args.length==2 && args[1].contentEquals("multiscale")){
System.out.println("Using the multiscale Hough Transform"); //
lines = cvHoughLines2(dst, storage, CV_HOUGH_MULTI_SCALE, 1, Math.PI / 180, 40, 1, 1, 0, CV_PI);
for (int i = 0; i < lines.total(); i++) {
CvPoint2D32f point = new CvPoint2D32f(cvGetSeqElem(lines, i));
float rho=point.x();
float theta=point.y();
double a = Math.cos((double) theta), b = Math.sin((double) theta);
double x0 = a * rho, y0 = b * rho;
CvPoint pt1 = cvPoint((int) Math.round(x0 + 1000 * (-b)), (int) Math.round(y0 + 1000 * (a))), pt2 = cvPoint((int) Math.round(x0 - 1000 * (-b)), (int) Math.round(y0 - 1000 * (a)));
System.out.println("Line spoted: ");
System.out.println("\t rho= " + rho);
System.out.println("\t theta= " + theta);
cvLine(colorDst, pt1, pt2, CV_RGB(255, 0, 0), 3, CV_AA, 0);
}
}
/*
* Default: apply the standard hough transform. Outputs: same as the multiscale output.
*/
else {
System.out.println("Using the Standard Hough Transform");
lines = cvHoughLines2(dst, storage, CV_HOUGH_STANDARD, 1, Math.PI / 180, 90, 0, 0, 0, CV_PI);
for (int i = 0; i < lines.total(); i++) {
CvPoint2D32f point = new CvPoint2D32f(cvGetSeqElem(lines, i));
float rho=point.x();
float theta=point.y();
double a = Math.cos((double) theta), b = Math.sin((double) theta);
double x0 = a * rho, y0 = b * rho;
CvPoint pt1 = cvPoint((int) Math.round(x0 + 1000 * (-b)), (int) Math.round(y0 + 1000 * (a))), pt2 = cvPoint((int) Math.round(x0 - 1000 * (-b)), (int) Math.round(y0 - 1000 * (a)));
System.out.println("Line spotted: ");
System.out.println("\t rho= " + rho);
System.out.println("\t theta= " + theta);
cvLine(colorDst, pt1, pt2, CV_RGB(255, 0, 0), 3, CV_AA, 0);
}
}
source.showImage(sourceConverter.convert(src));
hough.showImage(houghConverter.convert(colorDst));
source.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
hough.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
}
}
================================================
FILE: samples/ImageSegmentation.java
================================================
/*
* JavaCV version of OpenCV imageSegmentation.cpp
* https://github.com/opencv/opencv/blob/master/samples/cpp/tutorial_code/ImgTrans/imageSegmentation.cpp
*
* The OpenCV example image is available at the following address
* https://github.com/opencv/opencv/blob/master/samples/data/cards.png
*
* Paolo Bolettieri
*/
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import org.bytedeco.javacpp.indexer.FloatIndexer;
import org.bytedeco.javacpp.indexer.IntIndexer;
import org.bytedeco.javacpp.indexer.UByteIndexer;
import org.bytedeco.javacv.CanvasFrame;
import org.bytedeco.javacv.OpenCVFrameConverter;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgcodecs.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
public class ImageSegmentation {
private static final int[] WHITE = {255, 255, 255};
private static final int[] BLACK = {0, 0, 0};
public static void main(String[] args) {
// Load the image
Mat src = imread(args[0]);
// Check if everything was fine
if (src.data().isNull())
return;
// Show source image
imshow("Source Image", src);
// Change the background from white to black, since that will help later to extract
// better results during the use of Distance Transform
UByteIndexer srcIndexer = src.createIndexer();
for (int x = 0; x < srcIndexer.rows(); x++) {
for (int y = 0; y < srcIndexer.cols(); y++) {
int[] values = new int[3];
srcIndexer.get(x, y, values);
if (Arrays.equals(values, WHITE)) {
srcIndexer.put(x, y, BLACK);
}
}
}
// Show output image
imshow("Black Background Image", src);
// Create a kernel that we will use for accuting/sharpening our image
Mat kernel = Mat.ones(3, 3, CV_32F).asMat();
FloatIndexer kernelIndexer = kernel.createIndexer();
kernelIndexer.put(1, 1, -8); // an approximation of second derivative, a quite strong kernel
// do the laplacian filtering as it is
// well, we need to convert everything in something more deeper then CV_8U
// because the kernel has some negative values,
// and we can expect in general to have a Laplacian image with negative values
// BUT a 8bits unsigned int (the one we are working with) can contain values from 0 to 255
// so the possible negative number will be truncated
Mat imgLaplacian = new Mat();
Mat sharp = src; // copy source image to another temporary one
filter2D(sharp, imgLaplacian, CV_32F, kernel);
src.convertTo(sharp, CV_32F);
Mat imgResult = subtract(sharp, imgLaplacian).asMat();
// convert back to 8bits gray scale
imgResult.convertTo(imgResult, CV_8UC3);
imgLaplacian.convertTo(imgLaplacian, CV_8UC3);
// imshow( "Laplace Filtered Image", imgLaplacian );
imshow("New Sharped Image", imgResult);
src = imgResult; // copy back
// Create binary image from source image
Mat bw = new Mat();
cvtColor(src, bw, CV_BGR2GRAY);
threshold(bw, bw, 40, 255, CV_THRESH_BINARY | CV_THRESH_OTSU);
imshow("Binary Image", bw);
// Perform the distance transform algorithm
Mat dist = new Mat();
distanceTransform(bw, dist, CV_DIST_L2, 3);
// Normalize the distance image for range = {0.0, 1.0}
// so we can visualize and threshold it
normalize(dist, dist, 0, 1., NORM_MINMAX, -1, null);
imshow("Distance Transform Image", dist);
// Threshold to obtain the peaks
// This will be the markers for the foreground objects
threshold(dist, dist, .4, 1., CV_THRESH_BINARY);
// Dilate a bit the dist image
Mat kernel1 = Mat.ones(3, 3, CV_8UC1).asMat();
dilate(dist, dist, kernel1);
imshow("Peaks", dist);
// Create the CV_8U version of the distance image
// It is needed for findContours()
Mat dist_8u = new Mat();
dist.convertTo(dist_8u, CV_8U);
// Find total markers
MatVector contours = new MatVector();
findContours(dist_8u, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
// Create the marker image for the watershed algorithm
Mat markers = Mat.zeros(dist.size(), CV_32SC1).asMat();
// Draw the foreground markers
for (int i = 0; i < contours.size(); i++)
drawContours(markers, contours, i, Scalar.all((i) + 1));
// Draw the background marker
circle(markers, new Point(5, 5), 3, RGB(255, 255, 255));
imshow("Markers", multiply(markers, 10000).asMat());
// Perform the watershed algorithm
watershed(src, markers);
Mat mark = Mat.zeros(markers.size(), CV_8UC1).asMat();
markers.convertTo(mark, CV_8UC1);
bitwise_not(mark, mark);
// imshow("Markers_v2", mark); // uncomment this if you want to see how the mark
// image looks like at that point
// Generate random colors
List colors = new ArrayList();
for (int i = 0; i < contours.size(); i++) {
int b = theRNG().uniform(0, 255);
int g = theRNG().uniform(0, 255);
int r = theRNG().uniform(0, 255);
int[] color = { b, g, r };
colors.add(color);
}
// Create the result image
Mat dst = Mat.zeros(markers.size(), CV_8UC3).asMat();
// Fill labeled objects with random colors
IntIndexer markersIndexer = markers.createIndexer();
UByteIndexer dstIndexer = dst.createIndexer();
for (int i = 0; i < markersIndexer.rows(); i++) {
for (int j = 0; j < markersIndexer.cols(); j++) {
int index = markersIndexer.get(i, j);
if (index > 0 && index <= contours.size())
dstIndexer.put(i, j, colors.get(index - 1));
else
dstIndexer.put(i, j, BLACK);
}
}
// Visualize the final image
imshow("Final Result", dst);
}
//I wrote a custom imshow method for problems using the OpenCV original one
private static void imshow(String txt, Mat img) {
CanvasFrame canvasFrame = new CanvasFrame(txt);
canvasFrame.setDefaultCloseOperation(javax.swing.JFrame.EXIT_ON_CLOSE);
canvasFrame.setCanvasSize(img.cols(), img.rows());
canvasFrame.showImage(new OpenCVFrameConverter.ToMat().convert(img));
}
}
================================================
FILE: samples/JavaFxPlayVideoAndAudio.java
================================================
import java.nio.ByteBuffer;
import java.nio.ShortBuffer;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.logging.Level;
import java.util.logging.Logger;
import javafx.application.Application;
import javafx.application.Platform;
import javafx.scene.Scene;
import javafx.scene.image.Image;
import javafx.scene.image.ImageView;
import javafx.scene.layout.StackPane;
import javafx.stage.Stage;
import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.DataLine;
import javax.sound.sampled.SourceDataLine;
import org.bytedeco.javacv.FFmpegFrameGrabber;
import org.bytedeco.javacv.Frame;
import org.bytedeco.javacv.JavaFXFrameConverter;
/**
* @author Dmitriy Gerashenko
* @author Jarek Sacha
*/
public class JavaFxPlayVideoAndAudio extends Application {
private static class PlaybackTimer {
private Long startTime = -1L;
private final DataLine soundLine;
public PlaybackTimer(DataLine soundLine) {
this.soundLine = soundLine;
}
public PlaybackTimer() {
this.soundLine = null;
}
public void start() {
if (soundLine == null) {
startTime = System.nanoTime();
}
}
public long elapsedMicros() {
if (soundLine == null) {
if (startTime < 0) {
throw new IllegalStateException("PlaybackTimer not initialized.");
}
return (System.nanoTime() - startTime) / 1000;
} else {
return soundLine.getMicrosecondPosition();
}
}
}
private static final Logger LOG = Logger.getLogger(JavaFxPlayVideoAndAudio.class.getName());
private static volatile Thread playThread;
public static void main(String[] args) {
launch(args);
}
@Override
public void start(final Stage primaryStage) throws Exception {
final StackPane root = new StackPane();
final ImageView imageView = new ImageView();
root.getChildren().add(imageView);
imageView.fitWidthProperty().bind(primaryStage.widthProperty());
imageView.fitHeightProperty().bind(primaryStage.heightProperty());
final Scene scene = new Scene(root, 640, 480);
primaryStage.setTitle("Video + audio");
primaryStage.setScene(scene);
primaryStage.show();
playThread = new Thread(new Runnable() { public void run() {
try {
final String videoFilename = getParameters().getRaw().get(0);
final FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(videoFilename);
grabber.start();
primaryStage.setWidth(grabber.getImageWidth());
primaryStage.setHeight(grabber.getImageHeight());
final PlaybackTimer playbackTimer;
final SourceDataLine soundLine;
if (grabber.getAudioChannels() > 0) {
final AudioFormat audioFormat = new AudioFormat(grabber.getSampleRate(), 16, grabber.getAudioChannels(), true, true);
final DataLine.Info info = new DataLine.Info(SourceDataLine.class, audioFormat);
soundLine = (SourceDataLine) AudioSystem.getLine(info);
soundLine.open(audioFormat);
soundLine.start();
playbackTimer = new PlaybackTimer(soundLine);
} else {
soundLine = null;
playbackTimer = new PlaybackTimer();
}
final JavaFXFrameConverter converter = new JavaFXFrameConverter();
final ExecutorService audioExecutor = Executors.newSingleThreadExecutor();
final ExecutorService imageExecutor = Executors.newSingleThreadExecutor();
final long maxReadAheadBufferMicros = 1000 * 1000L;
long lastTimeStamp = -1L;
while (!Thread.interrupted()) {
final Frame frame = grabber.grab();
if (frame == null) {
break;
}
if (lastTimeStamp < 0) {
playbackTimer.start();
}
lastTimeStamp = frame.timestamp;
if (frame.image != null) {
final Frame imageFrame = frame.clone();
imageExecutor.submit(new Runnable() {
public void run() {
final Image image = converter.convert(imageFrame);
final long timeStampDeltaMicros = imageFrame.timestamp - playbackTimer.elapsedMicros();
imageFrame.close();
if (timeStampDeltaMicros > 0) {
final long delayMillis = timeStampDeltaMicros / 1000L;
try {
Thread.sleep(delayMillis);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
Platform.runLater(new Runnable() {
public void run() {
imageView.setImage(image);
}
});
}
});
} else if (frame.samples != null) {
if (soundLine == null) {
throw new IllegalStateException("Internal error: sound playback not initialized");
}
final ShortBuffer channelSamplesShortBuffer = (ShortBuffer) frame.samples[0];
channelSamplesShortBuffer.rewind();
final ByteBuffer outBuffer = ByteBuffer.allocate(channelSamplesShortBuffer.capacity() * 2);
for (int i = 0; i < channelSamplesShortBuffer.capacity(); i++) {
short val = channelSamplesShortBuffer.get(i);
outBuffer.putShort(val);
}
audioExecutor.submit(new Runnable() {
public void run() {
soundLine.write(outBuffer.array(), 0, outBuffer.capacity());
outBuffer.clear();
}
});
}
final long timeStampDeltaMicros = frame.timestamp - playbackTimer.elapsedMicros();
if (timeStampDeltaMicros > maxReadAheadBufferMicros) {
Thread.sleep((timeStampDeltaMicros - maxReadAheadBufferMicros) / 1000);
}
}
if (!Thread.interrupted()) {
long delay = (lastTimeStamp - playbackTimer.elapsedMicros()) / 1000 +
Math.round(1 / grabber.getFrameRate() * 1000);
Thread.sleep(Math.max(0, delay));
}
grabber.stop();
grabber.release();
if (soundLine != null) {
soundLine.stop();
}
audioExecutor.shutdownNow();
audioExecutor.awaitTermination(10, TimeUnit.SECONDS);
imageExecutor.shutdownNow();
imageExecutor.awaitTermination(10, TimeUnit.SECONDS);
Platform.exit();
} catch (Exception exception) {
LOG.log(Level.SEVERE, null, exception);
System.exit(1);
}
}});
playThread.start();
}
@Override
public void stop() {
playThread.interrupt();
}
}
================================================
FILE: samples/KazemiFacemarkExample.java
================================================
/**
* Kazemi Facemark example for JavaCV
*
* @author Théophile Gonos
*
* Link to Kazemi model :
* https://raw.githubusercontent.com/opencv/opencv_3rdparty/contrib_face_alignment_20170818/face_landmark_model.dat
*/
import java.io.IOException;
import java.net.URISyntaxException;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_face.*;
import org.bytedeco.opencv.opencv_highgui.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import org.bytedeco.opencv.opencv_objdetect.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_face.*;
import static org.bytedeco.opencv.global.opencv_highgui.*;
import static org.bytedeco.opencv.global.opencv_imgcodecs.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_objdetect.*;
public class KazemiFacemarkExample {
public static void main(String[] args) throws IOException, URISyntaxException, InterruptedException {
// Load Face Detector
CascadeClassifier faceDetector = new CascadeClassifier ("haarcascade_frontalface_alt2.xml");
// Create an instance of Facemark
FacemarkKazemi facemark = FacemarkKazemi.create();
// Load landmark detector
facemark.loadModel("face_landmark_model.dat");
// Load image
Mat img = imread("face.jpg");
// convert to grayscale and equalize histograe for better detection
Mat gray = new Mat ();
cvtColor(img, gray, COLOR_BGR2GRAY);
equalizeHist( gray, gray );
// Find faces on the image
RectVector faces = new RectVector ();
faceDetector.detectMultiScale(gray, faces);
System.out.println ("Faces detected: "+faces.size());
// Variable for landmarks.
// Landmarks for one face is a vector of points
// There can be more than one face in the image.
Point2fVectorVector landmarks = new Point2fVectorVector();
// Run landmark detector
boolean success = facemark.fit(img, faces, landmarks);
if(success) {
// If successful, render the landmarks on each face
for (long i = 0; i < landmarks.size(); i++) {
Point2fVector v = landmarks.get(i);
drawFacemarks(img, v, Scalar.YELLOW);
}
}
// Display results
imshow("Kazemi Facial Landmark", img);
cvWaitKey(0);
// Save results
imwrite ("kazemi_landmarks.jpg", img);
}
}
================================================
FILE: samples/LBFFacemarkExampleWithVideo.java
================================================
/**
* LBF Facemark example for JavaCV with Video camera and Transparent API
*
* @author Théophile Gonos
*
* you can find the lbfmodel here:
* https://raw.githubusercontent.com/kurnianggoro/GSOC2017/master/data/lbfmodel.yaml
*/
import java.io.IOException;
import java.net.URISyntaxException;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_face.*;
import org.bytedeco.opencv.opencv_highgui.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import org.bytedeco.opencv.opencv_objdetect.*;
import org.bytedeco.opencv.opencv_videoio.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_face.*;
import static org.bytedeco.opencv.global.opencv_highgui.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_objdetect.*;
import static org.bytedeco.opencv.global.opencv_videoio.*;
public class LBFFacemarkExampleWithVideo {
/**
* @param args the command line arguments
* @throws java.io.IOException
* @throws java.net.URISyntaxException
* @throws java.lang.InterruptedException
*/
public static void main(String[] args) throws IOException, URISyntaxException, InterruptedException {
// Load Face Detector
CascadeClassifier faceDetector = new CascadeClassifier ("haarcascade_frontalface_alt2.xml");
// Create an instance of Facemark
Facemark facemark = FacemarkLBF.create();
// Load landmark detector
facemark.loadModel("lbfmodel.yaml");
// Set up webcam for video capture
VideoCapture cam = new VideoCapture (0);
// Variable to store a video frame and its grayscale
Mat frame = new Mat ();
// Read a frame
while(cam.read(frame)) {
// convert to grayscale and equalize histograe for better detection
// + use of transparent API
UMat gray = new UMat ();
frame.copyTo(gray);
cvtColor(gray, gray, COLOR_BGR2GRAY);
equalizeHist( gray, gray );
// Find faces on the image
RectVector faces = new RectVector ();
faceDetector.detectMultiScale(gray, faces);
System.out.println ("Faces detected: "+faces.size());
// Verify is at least one face is detected
// With some Facemark algorithms it crashes if there is no faces
if (!faces.empty()) {
// Variable for landmarks.
// Landmarks for one face is a vector of points
// There can be more than one face in the image.
Point2fVectorVector landmarks = new Point2fVectorVector();
// Run landmark detector
boolean success = facemark.fit(frame, faces, landmarks);
if(success) {
// If successful, render the landmarks on the face
for (long i = 0; i < landmarks.size(); i++) {
Point2fVector v = landmarks.get(i);
drawFacemarks(frame, v, Scalar.YELLOW);
}
}
}
// Display results
imshow("LBF Facial Landmark", frame);
// Exit loop if ESC is pressed
if (waitKey(1) == 27) break;
}
}
}
================================================
FILE: samples/MotionDetector.java
================================================
/*
* I developed some code for recognize motion detections with JavaCV.
* Actually, it works with an array of Rect, performing, every cicle, an
* intersection test with area of difference with the rect of interests
* (this array is callad "sa", stands for SizedArea). I hope could this
* helps someone.
*
* Feel free to ask about any question regarding the code above, cheers!
*
* Angelo Marchesin
*/
import org.bytedeco.javacpp.*;
import org.bytedeco.javacv.*;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
public class MotionDetector {
public static void main(String[] args) throws Exception {
OpenCVFrameGrabber grabber = new OpenCVFrameGrabber(0);
OpenCVFrameConverter.ToIplImage converter = new OpenCVFrameConverter.ToIplImage();
grabber.start();
IplImage frame = converter.convert(grabber.grab());
IplImage image = null;
IplImage prevImage = null;
IplImage diff = null;
CanvasFrame canvasFrame = new CanvasFrame("Some Title");
canvasFrame.setCanvasSize(frame.width(), frame.height());
CvMemStorage storage = CvMemStorage.create();
while (canvasFrame.isVisible() && (frame = converter.convert(grabber.grab())) != null) {
cvClearMemStorage(storage);
cvSmooth(frame, frame, CV_GAUSSIAN, 9, 9, 2, 2);
if (image == null) {
image = IplImage.create(frame.width(), frame.height(), IPL_DEPTH_8U, 1);
cvCvtColor(frame, image, CV_RGB2GRAY);
} else {
prevImage = IplImage.create(frame.width(), frame.height(), IPL_DEPTH_8U, 1);
prevImage = image;
image = IplImage.create(frame.width(), frame.height(), IPL_DEPTH_8U, 1);
cvCvtColor(frame, image, CV_RGB2GRAY);
}
if (diff == null) {
diff = IplImage.create(frame.width(), frame.height(), IPL_DEPTH_8U, 1);
}
if (prevImage != null) {
// perform ABS difference
cvAbsDiff(image, prevImage, diff);
// do some threshold for wipe away useless details
cvThreshold(diff, diff, 64, 255, CV_THRESH_BINARY);
canvasFrame.showImage(converter.convert(diff));
// recognize contours
CvSeq contour = new CvSeq(null);
cvFindContours(diff, storage, contour, Loader.sizeof(CvContour.class), CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE);
while (contour != null && !contour.isNull()) {
if (contour.elem_size() > 0) {
CvBox2D box = cvMinAreaRect2(contour, storage);
// test intersection
if (box != null) {
CvPoint2D32f center = box.center();
CvSize2D32f size = box.size();
/* for (int i = 0; i < sa.length; i++) {
if ((Math.abs(center.x - (sa[i].offsetX + sa[i].width / 2))) < ((size.width / 2) + (sa[i].width / 2)) &&
(Math.abs(center.y - (sa[i].offsetY + sa[i].height / 2))) < ((size.height / 2) + (sa[i].height / 2))) {
if (!alarmedZones.containsKey(i)) {
alarmedZones.put(i, true);
activeAlarms.put(i, 1);
} else {
activeAlarms.remove(i);
activeAlarms.put(i, 1);
}
System.out.println("Motion Detected in the area no: " + i +
" Located at points: (" + sa[i].x + ", " + sa[i].y+ ") -"
+ " (" + (sa[i].x +sa[i].width) + ", "
+ (sa[i].y+sa[i].height) + ")");
}
}
*/
}
}
contour = contour.h_next();
}
}
}
grabber.stop();
canvasFrame.dispose();
}
}
================================================
FILE: samples/OpenCVFaceRecognizer.java
================================================
import java.io.File;
import java.io.FilenameFilter;
import java.nio.IntBuffer;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.IntPointer;
import org.bytedeco.javacpp.DoublePointer;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_face.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_face.*;
import static org.bytedeco.opencv.global.opencv_imgcodecs.*;
/**
* I couldn't find any tutorial on how to perform face recognition using OpenCV and Java,
* so I decided to share a viable solution here. The solution is very inefficient in its
* current form as the training model is built at each run, however it shows what's needed
* to make it work.
*
* The class below takes two arguments: The path to the directory containing the training
* faces and the path to the image you want to classify. Not that all images has to be of
* the same size and that the faces already has to be cropped out of their original images
* (Take a look here http://fivedots.coe.psu.ac.th/~ad/jg/nui07/index.html if you haven't
* done the face detection yet).
*
* For the simplicity of this post, the class also requires that the training images have
* filename format: -rest_of_filename.png. For example:
*
* 1-jon_doe_1.png
* 1-jon_doe_2.png
* 2-jane_doe_1.png
* 2-jane_doe_2.png
* ...and so on.
*
* Source: http://pcbje.com/2012/12/doing-face-recognition-with-javacv/
*
* @author Petter Christian Bjelland
*/
public class OpenCVFaceRecognizer {
public static void main(String[] args) {
String trainingDir = args[0];
Mat testImage = imread(args[1], IMREAD_GRAYSCALE);
File root = new File(trainingDir);
FilenameFilter imgFilter = new FilenameFilter() {
public boolean accept(File dir, String name) {
name = name.toLowerCase();
return name.endsWith(".jpg") || name.endsWith(".pgm") || name.endsWith(".png");
}
};
File[] imageFiles = root.listFiles(imgFilter);
MatVector images = new MatVector(imageFiles.length);
Mat labels = new Mat(imageFiles.length, 1, CV_32SC1);
IntBuffer labelsBuf = labels.createBuffer();
int counter = 0;
for (File image : imageFiles) {
Mat img = imread(image.getAbsolutePath(), IMREAD_GRAYSCALE);
int label = Integer.parseInt(image.getName().split("\\-")[0]);
images.put(counter, img);
labelsBuf.put(counter, label);
counter++;
}
FaceRecognizer faceRecognizer = FisherFaceRecognizer.create();
// FaceRecognizer faceRecognizer = EigenFaceRecognizer.create();
// FaceRecognizer faceRecognizer = LBPHFaceRecognizer.create();
faceRecognizer.train(images, labels);
IntPointer label = new IntPointer(1);
DoublePointer confidence = new DoublePointer(1);
faceRecognizer.predict(testImage, label, confidence);
int predictedLabel = label.get(0);
System.out.println("Predicted label: " + predictedLabel);
}
}
================================================
FILE: samples/OpenCVFeatures2dSerialization.java
================================================
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.nio.charset.StandardCharsets;
import javax.imageio.ImageIO;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacv.Java2DFrameConverter;
import org.bytedeco.javacv.OpenCVFrameConverter;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_features2d.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_features2d.*;
/**
* (De)serialize OpenCV structures using XML to files and memory
*
* Created by Maurice Betzel on 24.11.2017.
*/
public class OpenCVFeatures2dSerialization {
public static void main(String[] args) throws IOException {
String imageFile = (args.length > 0) ? args[0] : "Blob3.jpg";
BufferedImage bufferedImage = ImageIO.read(new File(imageFile));
try (Mat matrix = new OpenCVFrameConverter.ToMat().convert(new Java2DFrameConverter().convert(bufferedImage))
) {
String fileName = "serialized.xml";
serializeFile(matrix, fileName);
deserializeFile(fileName);
String serialized = serializeMemory(matrix);
System.out.println(serialized);
deserializeMemory(serialized);
}
}
private static void serializeFile(Mat matrix, String fileName) throws UnsupportedEncodingException {
try (KeyPointVector keyPointVectorSerialize = new KeyPointVector(); Mat objectDescriptorsSerialize = new Mat(); AKAZE akaze = AKAZE.create();
FileStorage fileStorage = new FileStorage(fileName, FileStorage.WRITE, StandardCharsets.UTF_8.name())
) {
akaze.detectAndCompute(matrix, new Mat(), keyPointVectorSerialize, objectDescriptorsSerialize, false);
System.out.println("Vector size: " + keyPointVectorSerialize.size());
System.out.println("Descriptor size: " + objectDescriptorsSerialize.cols());
write(fileStorage, "keyPoints", keyPointVectorSerialize);
write(fileStorage, "descriptors", objectDescriptorsSerialize);
fileStorage.release();
}
}
private static void deserializeFile(String file) {
try (KeyPointVector keyPointVectorDeserialize = new KeyPointVector(); Mat objectDescriptorsDeserialize = new Mat();
FileStorage fileStorage = new FileStorage(file, FileStorage.READ, StandardCharsets.UTF_8.name());
FileNode keyPointsFileNode = fileStorage.get("keyPoints"); FileNode descriptorsFileNode = fileStorage.get("descriptors")
) {
read(keyPointsFileNode, keyPointVectorDeserialize);
read(descriptorsFileNode, objectDescriptorsDeserialize);
System.out.println("Vector size: " + keyPointVectorDeserialize.size());
System.out.println("Descriptor size: " + objectDescriptorsDeserialize.cols());
fileStorage.release();
}
}
private static String serializeMemory(Mat matrix) throws UnsupportedEncodingException {
try (KeyPointVector keyPointVectorSerialize = new KeyPointVector(); Mat objectDescriptorsSerialize = new Mat(); AKAZE akaze = AKAZE.create();
FileStorage fileStorage = new FileStorage(".xml", FileStorage.WRITE | FileStorage.MEMORY, StandardCharsets.UTF_8.name())
) {
akaze.detectAndCompute(matrix, new Mat(), keyPointVectorSerialize, objectDescriptorsSerialize, false);
System.out.println("Vector size: " + keyPointVectorSerialize.size());
System.out.println("Descriptor size: " + objectDescriptorsSerialize.cols());
write(fileStorage, "keyPoints", keyPointVectorSerialize);
write(fileStorage, "descriptors", objectDescriptorsSerialize);
BytePointer bytePointer = fileStorage.releaseAndGetString();
return bytePointer.getString(StandardCharsets.UTF_8.name());
}
}
private static void deserializeMemory(String serialized) {
try (KeyPointVector keyPointVectorDeserialize = new KeyPointVector(); Mat objectDescriptorsDeserialize = new Mat();
FileStorage fileStorage = new FileStorage(serialized, FileStorage.READ | FileStorage.MEMORY, StandardCharsets.UTF_8.name());
FileNode keyPointsFileNode = fileStorage.get("keyPoints"); FileNode descriptorsFileNode = fileStorage.get("descriptors")
) {
read(keyPointsFileNode, keyPointVectorDeserialize);
read(descriptorsFileNode, objectDescriptorsDeserialize);
System.out.println("Vector size: " + keyPointVectorDeserialize.size());
System.out.println("Descriptor size: " + objectDescriptorsDeserialize.cols());
fileStorage.release();
}
}
}
================================================
FILE: samples/OpticalFlowDense.java
================================================
import java.nio.FloatBuffer;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_optflow.*;
import org.bytedeco.opencv.opencv_video.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgcodecs.*;
import static org.bytedeco.opencv.global.opencv_optflow.*;
import static org.bytedeco.opencv.global.opencv_video.*;
/**
* This code will calculate the optical flow for every pixel using DenseOpticalFlow between two images (Frame-1 &
* Frame-2) and put the velocity of every pixel to another image (OF) in their coordinate.
*
* @author Dawit Gebreyohannes
*/
public class OpticalFlowDense {
public static void main(final String[] args) {
final Mat pFrame = imread("samples/image0.png", IMREAD_GRAYSCALE),
cFrame = imread("samples/image1.png", IMREAD_GRAYSCALE),
pGray = new Mat(), cGray = new Mat(), Optical_Flow = new Mat();
pFrame.convertTo(pGray, CV_32FC1);
cFrame.convertTo(cGray, CV_32FC1);
final DenseOpticalFlow tvl1 = DualTVL1OpticalFlow.create();
tvl1.calc(pGray, cGray, Optical_Flow);
final Mat OF = new Mat(pGray.rows(), pGray.cols(), CV_32FC1);
final FloatBuffer in = Optical_Flow.createBuffer(),
out = OF.createBuffer();
final int height = pGray.rows(), width = pGray.cols();
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
final float xVelocity = in.get();
final float yVelocity = in.get();
final float pixelVelocity = (float) Math
.sqrt(xVelocity * xVelocity + yVelocity * yVelocity);
out.put(pixelVelocity);
}
}
imwrite("OF.png", OF);
}
}
================================================
FILE: samples/OpticalFlowTracker.java
================================================
/*
* Because I believe that examples are the easiest way how to use JavaCV, I am
* sending a sample based on http://dasl.mem.drexel.edu/~noahKuntz/openCVTut9.html
*
* burgetrm@gmail.com
*/
import org.bytedeco.javacv.*;
import org.bytedeco.javacpp.*;
import org.bytedeco.javacpp.indexer.*;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_highgui.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import org.bytedeco.opencv.opencv_video.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_highgui.*;
import static org.bytedeco.opencv.global.opencv_imgcodecs.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_video.*;
public class OpticalFlowTracker {
private static final int MAX_CORNERS = 500;
private static final int win_size = 15;
public static void main(String[] args) {
// Load two images and allocate other structures
Mat imgA = imread(
"image0.png",
IMREAD_GRAYSCALE);
Mat imgB = imread(
"image1.png",
IMREAD_GRAYSCALE);
// Mat imgC = imread("OpticalFlow1.png",
// IMREAD_UNCHANGED);
Mat imgC = imread(
"image0.png",
IMREAD_UNCHANGED);
// Get the features for tracking
Mat cornersA = new Mat();
goodFeaturesToTrack(imgA, cornersA, MAX_CORNERS,
0.05, 5.0, null, 3, false, 0.04);
cornerSubPix(imgA, cornersA,
new Size(win_size, win_size), new Size(-1, -1),
new TermCriteria(CV_TERMCRIT_ITER | CV_TERMCRIT_EPS, 20, 0.03));
// Call Lucas Kanade algorithm
Mat features_found = new Mat();
Mat feature_errors = new Mat();
Mat cornersB = new Mat();
calcOpticalFlowPyrLK(imgA, imgB, cornersA, cornersB,
features_found, feature_errors, new Size(win_size, win_size), 5,
new TermCriteria(CV_TERMCRIT_ITER | CV_TERMCRIT_EPS, 20, 0.3), 0, 1e-4);
// Make an image of the results
FloatIndexer cornersAidx = cornersA.createIndexer();
FloatIndexer cornersBidx = cornersB.createIndexer();
UByteIndexer features_found_idx = features_found.createIndexer();
FloatIndexer feature_errors_idx = feature_errors.createIndexer();
for (int i = 0; i < cornersAidx.size(0); i++) {
if (features_found_idx.get(i) == 0 || feature_errors_idx.get(i) > 550) {
System.out.println("Error is " + feature_errors_idx.get(i) + "/n");
continue;
}
System.out.println("Got it/n");
Point p0 = new Point(Math.round(cornersAidx.get(i, 0)),
Math.round(cornersAidx.get(i, 1)));
Point p1 = new Point(Math.round(cornersBidx.get(i, 0)),
Math.round(cornersBidx.get(i, 1)));
line(imgC, p0, p1, RGB(255, 0, 0),
2, 8, 0);
}
imwrite(
"image0-1.png",
imgC);
namedWindow("LKpyr_OpticalFlow", 0);
imshow("LKpyr_OpticalFlow", imgC);
waitKey(0);
}
}
================================================
FILE: samples/PacketRecorderTest.java
================================================
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Date;
import org.bytedeco.javacv.*;
import org.bytedeco.ffmpeg.avcodec.AVPacket;
public class PacketRecorderTest {
private static final DateFormat DATE_FORMAT = new SimpleDateFormat("yyyyMMdd__hhmmSSS");
private static final int RECORD_LENGTH = 5000;
private static final boolean AUDIO_ENABLED = false;
public static void main(String[] args) throws FrameRecorder.Exception, FrameGrabber.Exception {
String inputFile = "/home/usr/videos/VIDEO_FILE_NAME.mp4";
// Decodes-encodes
String outputFile = "/tmp/" + DATE_FORMAT.format(new Date()) + "_frameRecord.mp4";
PacketRecorderTest.frameRecord(inputFile, outputFile);
// copies codec (no need to re-encode)
outputFile = "/tmp/" + DATE_FORMAT.format(new Date()) + "_packetRecord.mp4";
PacketRecorderTest.packetRecord(inputFile, outputFile);
}
public static void frameRecord(String inputFile, String outputFile) throws FrameGrabber.Exception, FrameRecorder.Exception {
int audioChannel = AUDIO_ENABLED ? 1 : 0;
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(inputFile);
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(outputFile, 1280, 720, audioChannel);
grabber.start();
recorder.start();
Frame frame;
long t1 = System.currentTimeMillis();
while ((frame = grabber.grabFrame(AUDIO_ENABLED, true, true, false)) != null) {
recorder.record(frame);
if ((System.currentTimeMillis() - t1) > RECORD_LENGTH) {
break;
}
}
recorder.stop();
grabber.stop();
}
public static void packetRecord(String inputFile, String outputFile) throws FrameGrabber.Exception, FrameRecorder.Exception {
int audioChannel = AUDIO_ENABLED ? 1 : 0;
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(inputFile);
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(outputFile, 1280, 720, audioChannel);
grabber.start();
recorder.start(grabber.getFormatContext());
AVPacket packet;
long t1 = System.currentTimeMillis();
while ((packet = grabber.grabPacket()) != null) {
recorder.recordPacket(packet);
if ((System.currentTimeMillis() - t1) > RECORD_LENGTH) {
break;
}
}
recorder.stop();
grabber.stop();
}
}
================================================
FILE: samples/PerspectiveWarpDemo.java
================================================
import org.bytedeco.javacpp.FloatPointer;
import org.bytedeco.javacv.CanvasFrame;
import org.bytedeco.javacv.OpenCVFrameConverter;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgcodecs.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
* Created by Johan Swanberg on 2018-09-21
*
* An example of how to use the perspective warp method in JavaCV.
*/
public class PerspectiveWarpDemo extends Thread {
static CanvasFrame frame = new CanvasFrame("Perspective Warp Demo - warped image");
static CanvasFrame frameUnedited = new CanvasFrame("Perspective Warp Demo - Unedited image");
public static void main(String[] args) {
Mat image = imread("shapes1.jpg");
Mat perspectiveWarpedImg = performPerspectiveWarp(image, 30, 0, 200, 0, 400, 250, 40, 260);
OpenCVFrameConverter converter = new OpenCVFrameConverter.ToIplImage();
frame.showImage(converter.convert(perspectiveWarpedImg));
frameUnedited.showImage(converter.convert(image));
image.release();
perspectiveWarpedImg.release();
frameUnedited.setDefaultCloseOperation(javax.swing.JFrame.EXIT_ON_CLOSE);
frameUnedited.setLocationRelativeTo(null);
frame.setDefaultCloseOperation(javax.swing.JFrame.EXIT_ON_CLOSE);
frame.setLocation(frameUnedited.getX()+frameUnedited.getWidth(), frameUnedited.getY());
}
/**
* Performs a perspective warp that takes four corners and stretches them to the corners of the image.
* x1,y1 represents the top left corner, 2 top right, going clockwise.
* This method does not release/deallocate the input image Mat, call inputMat.release() after this method
* if you don't plan on using the input more after this method.
*
* @param imageMat The image to perform the stretch on
* @return A stretched image mat.
*/
private static Mat performPerspectiveWarp(Mat imageMat, int x1, int y1, int x2, int y2, int x3, int y3, int x4, int y4) {
double originalImgWidth = imageMat.size().width();
double originalImgHeight = imageMat.size().height();
FloatPointer srcCorners = new FloatPointer(
x1, y1,
x2, y2,
x3, y3,
x4, y4);
FloatPointer dstCorners = new FloatPointer(
0, 0,
(int) originalImgWidth, 0,
(int) originalImgWidth, (int) originalImgHeight,
0, (int) originalImgHeight);
//create matrices with width 2 to hold the x,y values, and 4 rows, to hold the 4 different corners.
Mat src = new Mat(new Size(2, 4), CV_32F, srcCorners);
Mat dst = new Mat(new Size(2, 4), CV_32F, dstCorners);
Mat perspective = getPerspectiveTransform(src, dst);
Mat result = new Mat();
warpPerspective(imageMat, result, perspective, new Size((int) originalImgWidth, (int) originalImgHeight));
src.release();
dst.release();
srcCorners.deallocate();
dstCorners.deallocate();
return result;
}
}
================================================
FILE: samples/PrincipalComponentAnalysis.java
================================================
import java.awt.image.BufferedImage;
import java.io.File;
import java.util.ArrayList;
import javax.imageio.ImageIO;
import org.bytedeco.javacpp.indexer.DoubleIndexer;
import org.bytedeco.javacpp.indexer.IntIndexer;
import org.bytedeco.javacpp.tools.Slf4jLogger;
import org.bytedeco.javacv.CanvasFrame;
import org.bytedeco.javacv.Java2DFrameConverter;
import org.bytedeco.javacv.OpenCVFrameConverter;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
* PrincipalComponentAnalysis with JavaCV
* https://github.com/bytedeco/javacv
* Based on "Introduction to Principal Component Analysis (PrincipalComponentAnalysis) ":
* http://docs.opencv.org/3.0.0/d1/dee/tutorial_introduction_to_pca.html
*
* @author Maurice Betzel
*/
public class PrincipalComponentAnalysis {
static {
System.setProperty("org.bytedeco.javacpp.logger", "slf4jlogger");
System.setProperty("org.slf4j.simpleLogger.defaultLogLevel", "debug");
}
private static final Slf4jLogger logger = (Slf4jLogger) org.bytedeco.javacpp.tools.Logger.create(PrincipalComponentAnalysis.class);
public static void main(String[] args) {
try {
logger.info(String.valueOf(logger.isDebugEnabled()));
logger.info("Start");
new PrincipalComponentAnalysis().execute(args);
logger.info("Stop");
} catch (Exception e) {
e.printStackTrace();
}
}
private void execute(String[] args) throws Exception {
// If no params provided, compute the default image
BufferedImage bufferedImage = args.length >= 1 ? ImageIO.read(new File(args[0])) : ImageIO.read(this.getClass().getResourceAsStream("shapes2.jpg"));
System.out.println("Image type: " + bufferedImage.getType());
// Convert BufferedImage to Mat and create AutoCloseable objects
try (Mat matrix = new OpenCVFrameConverter.ToMat().convert(new Java2DFrameConverter().convert(bufferedImage));
Mat mask = new Mat();
Mat gray = new Mat();
Mat denoised = new Mat();
Mat bin = new Mat();
Mat hierarchy = new Mat();
MatVector contours = new MatVector()) {
printMat(matrix);
cvtColor(matrix, gray, COLOR_BGR2GRAY);
//Normalize
GaussianBlur(gray, denoised, new Size(5, 5), 0);
threshold(denoised, mask, 0, 255, THRESH_BINARY_INV | THRESH_OTSU);
normalize(gray, gray, 0, 255, NORM_MINMAX, -1, mask);
// Convert image to binary
threshold(gray, bin, 150, 255, THRESH_BINARY);
// Find contours
findContours(bin, contours, hierarchy, RETR_TREE, CHAIN_APPROX_NONE);
long contourCount = contours.size();
System.out.println("Countour count " + contourCount);
for (int i = 0; i < contourCount; ++i) {
// Calculate the area of each contour
Mat contour = contours.get(i);
double area = contourArea(contour);
// Ignore contours that are too small or too large
if (area > 128 && area < 8192) {
principalComponentAnalysis(contour, i, matrix);
}
}
CanvasFrame canvas = new CanvasFrame("PrincipalComponentAnalysis", 1);
canvas.setDefaultCloseOperation(javax.swing.JFrame.EXIT_ON_CLOSE);
canvas.setCanvasSize(320, 240);
OpenCVFrameConverter converter = new OpenCVFrameConverter.ToIplImage();
canvas.showImage(converter.convert(matrix));
}
}
// contour is a one dimensional array
private void principalComponentAnalysis(Mat contour, int entry, Mat matrix) throws Exception {
PCA pca_analysis = null;
Mat mean = null;
Mat eigenVector = null;
Mat eigenValues = null;
//Construct a buffer used by the pca analysis
try (Mat data_pts = new Mat(contour.rows(), 2, CV_64FC1);
Mat placeholder = new Mat();
Point cntr = new Point()) {
IntIndexer contourIndexer = contour.createIndexer();
DoubleIndexer data_idx = data_pts.createIndexer();
for (int i = 0; i < contour.rows(); i++) {
data_idx.put(i, 0, contourIndexer.get(i, 0));
data_idx.put(i, 1, contourIndexer.get(i, 1));
}
contourIndexer.release();
data_idx.release();
//Perform PrincipalComponentAnalysis analysis
ArrayList eigen_vecs = new ArrayList(2);
ArrayList eigen_val = new ArrayList(2);
pca_analysis = new PCA(data_pts, placeholder, CV_PCA_DATA_AS_ROW);
mean = pca_analysis.mean();
eigenVector = pca_analysis.eigenvectors();
eigenValues = pca_analysis.eigenvalues();
DoubleIndexer mean_idx = mean.createIndexer();
DoubleIndexer eigenVectorIndexer = eigenVector.createIndexer();
DoubleIndexer eigenValuesIndexer = eigenValues.createIndexer();
for (int i = 0; i < 2; ++i) {
eigen_vecs.add(new Point2d(eigenVectorIndexer.get(i, 0), eigenVectorIndexer.get(i, 1)));
eigen_val.add(eigenValuesIndexer.get(0, i));
}
double cntrX = mean_idx.get(0, 0);
double cntrY = mean_idx.get(0, 1);
mean_idx.release();
eigenVectorIndexer.release();
eigenValuesIndexer.release();
double x1 = cntrX + 0.02 * (eigen_vecs.get(0).x() * eigen_val.get(0));
double y1 = cntrY + 0.02 * (eigen_vecs.get(0).y() * eigen_val.get(0));
double x2 = cntrX - 0.02 * (eigen_vecs.get(1).x() * eigen_val.get(1));
double y2 = cntrY - 0.02 * (eigen_vecs.get(1).y() * eigen_val.get(1));
// Draw the principal components, keep accuracy during calculations
cntr.x((int) Math.rint(cntrX));
cntr.y((int) Math.rint(cntrY));
circle(matrix, cntr, 5, new Scalar(255, 0, 255, 0));
double radian1 = Math.atan2(cntrY - y1, cntrX - x1);
double radian2 = Math.atan2(cntrY - y2, cntrX - x2);
double hypotenuse1 = Math.sqrt((cntrY - y1) * (cntrY - y1) + (cntrX - x1) * (cntrX - x1));
double hypotenuse2 = Math.sqrt((cntrY - y2) * (cntrY - y2) + (cntrX - x2) * (cntrX - x2));
//Enhance the vector signal by a factor of 2
double point1x = cntrX - 2 * hypotenuse1 * Math.cos(radian1);
double point1y = cntrY - 2 * hypotenuse1 * Math.sin(radian1);
double point2x = cntrX - 2 * hypotenuse2 * Math.cos(radian2);
double point2y = cntrY - 2 * hypotenuse2 * Math.sin(radian2);
drawAxis(matrix, radian1, cntr, point1x, point1y, Scalar.BLUE);
drawAxis(matrix, radian2, cntr, point2x, point2y, Scalar.CYAN);
} finally {
if(pca_analysis != null) {
pca_analysis.deallocate();
}
if(mean != null) {
mean.deallocate();
}
if(eigenVector != null) {
eigenVector.deallocate();
}
if(eigenValues != null) {
eigenValues.deallocate();
}
}
}
private void drawAxis(Mat matrix, double radian, Point cntr, double x, double y, Scalar colour) throws Exception {
try(Point q = new Point((int) x, (int) y);
Point arrowHook1 = new Point((int) (q.x() + 9 * Math.cos(radian + CV_PI / 4)), (int) (q.y() + 9 * Math.sin(radian + CV_PI / 4)));
Point arrowHook2 = new Point((int) (q.x() + 9 * Math.cos(radian - CV_PI / 4)), (int) (q.y() + 9 * Math.sin(radian - CV_PI / 4)))) {
// draw
line(matrix, cntr, q, colour);
line(matrix, arrowHook1, q, colour);
line(matrix, arrowHook2, q, colour);
}
}
public static void printMat(Mat mat) {
System.out.println("Channels: " + mat.channels());
System.out.println("Rows: " + mat.rows());
System.out.println("Cols: " + mat.cols());
System.out.println("Type: " + mat.type());
System.out.println("Dims: " + mat.dims());
System.out.println("Depth: " + mat.depth());
}
}
================================================
FILE: samples/RLSA.java
================================================
import java.awt.image.BufferedImage;
import java.io.File;
import java.nio.ByteBuffer;
import javax.imageio.ImageIO;
import org.bytedeco.javacv.Java2DFrameConverter;
import org.bytedeco.javacv.OpenCVFrameConverter;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
* Based on "Implementation Run Length Smoothing Algorithm in C++":
* http://stackoverflow.com/questions/21554431/implementation-run-length-smoothing-algorithm-in-c
*
* @author Nicholas Woodward
*/
public class RLSA {
public static void main(String[] args) {
String imagePath = args[0].trim();
IplImage image = null;
try {
Java2DFrameConverter converter1 = new Java2DFrameConverter();
OpenCVFrameConverter.ToIplImage converter2 = new OpenCVFrameConverter.ToIplImage();
BufferedImage img = ImageIO.read(new File(imagePath));
image = converter2.convert(converter1.convert(img));
} catch (Exception ex) {
ex.printStackTrace();
}
if (image != null) {
IplImage rlsaImage = runLengthSmoothingAlgorithm(image);
// do something with the result image
image.release();
rlsaImage.release();
}
}
public static IplImage runLengthSmoothingAlgorithm(IplImage image) {
IplImage gry = image.clone();
cvThreshold(gry, gry, 128, 255, CV_THRESH_BINARY_INV);
CvMat tmpImg = gry.asCvMat();
ByteBuffer buffer = gry.getByteBuffer();
CvScalar temp = new CvScalar();
temp.val(255);
int hor_thres = 77; // adjust for your text size
int zero_count = 0;
int one_flag = 0;
for (int i = 0; i < tmpImg.rows(); i++) {
for (int j = 0; j < tmpImg.cols(); j++) {
int ind = i * gry.widthStep() + j * gry.nChannels() + 1;
double val = -1;
if (ind < buffer.capacity()) {
val = (buffer.get(ind) & 0xFF);
}
if (val == 255) {
if (one_flag == 255) {
if (zero_count <= hor_thres) {
tmpImg.put(i, j, 255);
for (int n = (j-zero_count); n < j; n++) {
tmpImg.put(i, n, 255);
}
} else {
one_flag = 0;
}
zero_count = 0;
}
one_flag = 255;
} else if (one_flag == 255) {
zero_count = zero_count + 1;
}
}
}
int ver_thres = 44; // adjustable
zero_count = 0;
one_flag = 0;
for (int i = 0; i < tmpImg.cols(); i++) {
for (int j = 0; j < tmpImg.rows(); j++) {
int ind = j * gry.widthStep() + i * gry.nChannels() + 1;
double val = -1;
if (ind < buffer.capacity()) {
val = (buffer.get(ind) & 0xFF);
}
if (val == 255) {
if (one_flag == 255) {
if (zero_count <= ver_thres) {
tmpImg.put(j, i, 255);
for (int n = ((j-zero_count) >= 0) ? (j-zero_count) : 0; n < j; n++) {
tmpImg.put(n, i, 255);
}
} else {
one_flag = 0;
}
zero_count = 0;
}
one_flag = 255;
} else if (one_flag == 255) {
zero_count = zero_count + 1;
}
}
}
return gry;
}
}
================================================
FILE: samples/RealSense2DepthMeasuring.java
================================================
/*
* Copyright (C) 2019 Florian Bruggisser
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import org.bytedeco.javacv.CanvasFrame;
import org.bytedeco.javacv.Frame;
import org.bytedeco.javacv.FrameGrabber;
import org.bytedeco.javacv.RealSense2FrameGrabber;
import java.awt.event.MouseAdapter;
import java.awt.event.MouseEvent;
public class RealSense2DepthMeasuring {
public static void main(String[] args) throws FrameGrabber.Exception {
final RealSense2FrameGrabber rs2 = new RealSense2FrameGrabber();
// list all cameras
for (RealSense2FrameGrabber.RealSense2DeviceInfo info : rs2.getDeviceInfos()) {
System.out.printf("Device: %s %s %s Locked: %b\n",
info.getName(),
info.getFirmware(),
info.getSerialNumber(),
info.isLocked());
}
// enable the color & depth stream of the realsense camera
rs2.enableColorStream(640, 480, 30);
rs2.enableDepthStream(640, 480, 30);
// here are more examples of streams:
/*
rs2.enableColorStream(640, 480, 30); // color stream
rs2.enableIRStream(640, 480, 90); // ir stream
rs2.enableStream(new RealSense2FrameGrabber.RealSenseStream(
RS2_STREAM_INFRARED,
2,
new Size(640, 480),
30,
RS2_FORMAT_Y8
)); // second ir stream
*/
// start realsense camera
rs2.start();
// start frame to view the stream
CanvasFrame canvasFrame = new CanvasFrame("RealSense");
canvasFrame.setCanvasSize(rs2.getImageWidth(), rs2.getImageHeight());
// add mouse listener to see the depth at the clicked point
canvasFrame.getCanvas().addMouseListener(new MouseAdapter() {
@Override
public void mousePressed(MouseEvent e) {
try {
System.out.println("Depth: " + rs2.getDistance(e.getX(), e.getY()));
} catch (FrameGrabber.Exception ex) {
ex.printStackTrace();
}
}
});
// run canvas
while (canvasFrame.isVisible()) {
// trigger camera to capture images
rs2.trigger();
// display images -> grab will return the first stream added
// use rs2.grabDepth(), rs2.grabColor() and rs2.grabIR() for the other streams
Frame frame = rs2.grab();
if (frame == null) {
System.err.println("Frame is null!");
break;
}
// display frame
canvasFrame.showImage(frame);
}
// close realsense camera
rs2.stop();
rs2.release();
canvasFrame.dispose();
}
}
================================================
FILE: samples/RecordActivity.java
================================================
/*
* Copyright (C) 2012,2013 Qianliang Zhang, Shawn Van Every, Samuel Audet
*
* IMPORTANT - Make sure the AndroidManifest.xml file looks like this:
*
*
*
*
*
*
*
*
*
*
*
* And the res/layout/main.xml file like this:
*
*
*
*
*/
package org.bytedeco.javacv.recordactivity;
import android.app.Activity;
import android.content.Context;
import android.content.pm.ActivityInfo;
import android.hardware.Camera;
import android.hardware.Camera.PreviewCallback;
import android.media.AudioFormat;
import android.media.AudioRecord;
import android.media.MediaRecorder;
import android.os.Bundle;
import android.os.PowerManager;
import android.util.Log;
import android.view.Display;
import android.view.KeyEvent;
import android.view.LayoutInflater;
import android.view.SurfaceHolder;
import android.view.SurfaceView;
import android.view.View;
import android.view.View.OnClickListener;
import android.view.WindowManager;
import android.widget.Button;
import android.widget.LinearLayout;
import android.widget.RelativeLayout;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.ShortBuffer;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
import org.bytedeco.ffmpeg.global.avutil;
import org.bytedeco.javacv.FFmpegFrameFilter;
import org.bytedeco.javacv.FFmpegFrameRecorder;
import org.bytedeco.javacv.Frame;
import org.bytedeco.javacv.FrameFilter;
public class RecordActivity extends Activity implements OnClickListener {
private final static String CLASS_LABEL = "RecordActivity";
private final static String LOG_TAG = CLASS_LABEL;
private String ffmpeg_link = "/mnt/sdcard/stream.flv";
long startTime = 0;
boolean recording = false;
private FFmpegFrameRecorder recorder;
private boolean isPreviewOn = false;
/*Filter information, change boolean to true if adding a fitler*/
private boolean addFilter = true;
private String filterString = "";
FFmpegFrameFilter filter;
private int sampleAudioRateInHz = 44100;
private int imageWidth = 320;
private int imageHeight = 240;
private int frameRate = 30;
/* audio data getting thread */
private AudioRecord audioRecord;
private AudioRecordRunnable audioRecordRunnable;
private Thread audioThread;
volatile boolean runAudioThread = true;
/* video data getting thread */
private Camera cameraDevice;
private CameraView cameraView;
private Frame yuvImage = null;
/* layout setting */
private final int bg_screen_bx = 232;
private final int bg_screen_by = 128;
private final int bg_screen_width = 700;
private final int bg_screen_height = 500;
private final int bg_width = 1123;
private final int bg_height = 715;
private final int live_width = 640;
private final int live_height = 480;
private int screenWidth, screenHeight;
private Button btnRecorderControl;
/* The number of seconds in the continuous record loop (or 0 to disable loop). */
final int RECORD_LENGTH = 0;
Frame[] images;
long[] timestamps;
ShortBuffer[] samples;
int imagesIndex, samplesIndex;
@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_LANDSCAPE);
setContentView(R.layout.main);
initLayout();
}
@Override
protected void onDestroy() {
super.onDestroy();
recording = false;
if (cameraView != null) {
cameraView.stopPreview();
}
if(cameraDevice != null) {
cameraDevice.stopPreview();
cameraDevice.release();
cameraDevice = null;
}
}
private void initLayout() {
/* get size of screen */
Display display = ((WindowManager) getSystemService(Context.WINDOW_SERVICE)).getDefaultDisplay();
screenWidth = display.getWidth();
screenHeight = display.getHeight();
RelativeLayout.LayoutParams layoutParam = null;
LayoutInflater myInflate = null;
myInflate = (LayoutInflater) getSystemService(Context.LAYOUT_INFLATER_SERVICE);
RelativeLayout topLayout = new RelativeLayout(this);
setContentView(topLayout);
LinearLayout preViewLayout = (LinearLayout) myInflate.inflate(R.layout.main, null);
layoutParam = new RelativeLayout.LayoutParams(screenWidth, screenHeight);
topLayout.addView(preViewLayout, layoutParam);
/* add control button: start and stop */
btnRecorderControl = (Button) findViewById(R.id.recorder_control);
btnRecorderControl.setText("Start");
btnRecorderControl.setOnClickListener(this);
/* add camera view */
int display_width_d = (int) (1.0 * bg_screen_width * screenWidth / bg_width);
int display_height_d = (int) (1.0 * bg_screen_height * screenHeight / bg_height);
int prev_rw, prev_rh;
if (1.0 * display_width_d / display_height_d > 1.0 * live_width / live_height) {
prev_rh = display_height_d;
prev_rw = (int) (1.0 * display_height_d * live_width / live_height);
} else {
prev_rw = display_width_d;
prev_rh = (int) (1.0 * display_width_d * live_height / live_width);
}
layoutParam = new RelativeLayout.LayoutParams(prev_rw, prev_rh);
layoutParam.topMargin = (int) (1.0 * bg_screen_by * screenHeight / bg_height);
layoutParam.leftMargin = (int) (1.0 * bg_screen_bx * screenWidth / bg_width);
cameraDevice = Camera.open();
Log.i(LOG_TAG, "cameara open");
cameraView = new CameraView(this, cameraDevice);
topLayout.addView(cameraView, layoutParam);
Log.i(LOG_TAG, "cameara preview start: OK");
}
//---------------------------------------
// initialize ffmpeg_recorder
//---------------------------------------
private void initRecorder() {
Log.w(LOG_TAG,"init recorder");
Log.i(LOG_TAG, "ffmpeg_url: " + ffmpeg_link);
recorder = new FFmpegFrameRecorder(ffmpeg_link, imageWidth, imageHeight, 1);
recorder.setFormat("flv");
recorder.setSampleRate(sampleAudioRateInHz);
// Set in the surface changed method
recorder.setFrameRate(frameRate);
// The filterString is any ffmpeg filter.
// Here is the link for a list: https://ffmpeg.org/ffmpeg-filters.html
filterString = "transpose=2,crop=w=200:h=200:x=0:y=0";
filter = new FFmpegFrameFilter(filterString, imageWidth, imageHeight);
//default format on android
filter.setPixelFormat(avutil.AV_PIX_FMT_NV21);
if (RECORD_LENGTH > 0) {
imagesIndex = 0;
images = new Frame[RECORD_LENGTH * frameRate];
timestamps = new long[images.length];
for (int i = 0; i < images.length; i++) {
images[i] = new Frame(imageWidth, imageHeight, Frame.DEPTH_UBYTE, 2);
timestamps[i] = -1;
}
} else if (yuvImage == null) {
yuvImage = new Frame(imageWidth, imageHeight, Frame.DEPTH_UBYTE, 2);
Log.i(LOG_TAG, "create yuvImage");
}
Log.i(LOG_TAG, "recorder initialize success");
audioRecordRunnable = new AudioRecordRunnable();
audioThread = new Thread(audioRecordRunnable);
runAudioThread = true;
}
public void startRecording() {
initRecorder();
try {
recorder.start();
startTime = System.currentTimeMillis();
recording = true;
audioThread.start();
if(addFilter) {
filter.start();
}
} catch (FFmpegFrameRecorder.Exception | FrameFilter.Exception e) {
e.printStackTrace();
}
}
public void stopRecording() {
runAudioThread = false;
try {
audioThread.join();
} catch (InterruptedException e) {
// reset interrupt to be nice
Thread.currentThread().interrupt();
return;
}
audioRecordRunnable = null;
audioThread = null;
if (recorder != null && recording) {
if (RECORD_LENGTH > 0) {
Log.v(LOG_TAG,"Writing frames");
try {
int firstIndex = imagesIndex % samples.length;
int lastIndex = (imagesIndex - 1) % images.length;
if (imagesIndex <= images.length) {
firstIndex = 0;
lastIndex = imagesIndex - 1;
}
if ((startTime = timestamps[lastIndex] - RECORD_LENGTH * 1000000L) < 0) {
startTime = 0;
}
if (lastIndex < firstIndex) {
lastIndex += images.length;
}
for (int i = firstIndex; i <= lastIndex; i++) {
long t = timestamps[i % timestamps.length] - startTime;
if (t >= 0) {
if (t > recorder.getTimestamp()) {
recorder.setTimestamp(t);
}
recorder.record(images[i % images.length]);
}
}
firstIndex = samplesIndex % samples.length;
lastIndex = (samplesIndex - 1) % samples.length;
if (samplesIndex <= samples.length) {
firstIndex = 0;
lastIndex = samplesIndex - 1;
}
if (lastIndex < firstIndex) {
lastIndex += samples.length;
}
for (int i = firstIndex; i <= lastIndex; i++) {
recorder.recordSamples(samples[i % samples.length]);
}
} catch (FFmpegFrameRecorder.Exception e) {
Log.v(LOG_TAG,e.getMessage());
e.printStackTrace();
}
}
recording = false;
Log.v(LOG_TAG,"Finishing recording, calling stop and release on recorder");
try {
recorder.stop();
recorder.release();
filter.stop();
filter.release();
} catch (FFmpegFrameRecorder.Exception | FrameFilter.Exception e) {
e.printStackTrace();
}
recorder = null;
}
}
@Override
public boolean onKeyDown(int keyCode, KeyEvent event) {
if (keyCode == KeyEvent.KEYCODE_BACK) {
if (recording) {
stopRecording();
}
finish();
return true;
}
return super.onKeyDown(keyCode, event);
}
//---------------------------------------------
// audio thread, gets and encodes audio data
//---------------------------------------------
class AudioRecordRunnable implements Runnable {
@Override
public void run() {
android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_URGENT_AUDIO);
// Audio
int bufferSize;
ShortBuffer audioData;
int bufferReadResult;
bufferSize = AudioRecord.getMinBufferSize(sampleAudioRateInHz,
AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT);
audioRecord = new AudioRecord(MediaRecorder.AudioSource.MIC, sampleAudioRateInHz,
AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT, bufferSize);
if (RECORD_LENGTH > 0) {
samplesIndex = 0;
samples = new ShortBuffer[RECORD_LENGTH * sampleAudioRateInHz * 2 / bufferSize + 1];
for (int i = 0; i < samples.length; i++) {
samples[i] = ShortBuffer.allocate(bufferSize);
}
} else {
audioData = ShortBuffer.allocate(bufferSize);
}
Log.d(LOG_TAG, "audioRecord.startRecording()");
audioRecord.startRecording();
/* ffmpeg_audio encoding loop */
while (runAudioThread) {
if (RECORD_LENGTH > 0) {
audioData = samples[samplesIndex++ % samples.length];
audioData.position(0).limit(0);
}
//Log.v(LOG_TAG,"recording? " + recording);
bufferReadResult = audioRecord.read(audioData.array(), 0, audioData.capacity());
audioData.limit(bufferReadResult);
if (bufferReadResult > 0) {
Log.v(LOG_TAG,"bufferReadResult: " + bufferReadResult);
// If "recording" isn't true when start this thread, it never get's set according to this if statement...!!!
// Why? Good question...
if (recording) {
if (RECORD_LENGTH <= 0) try {
recorder.recordSamples(audioData);
//Log.v(LOG_TAG,"recording " + 1024*i + " to " + 1024*i+1024);
} catch (FFmpegFrameRecorder.Exception e) {
Log.v(LOG_TAG,e.getMessage());
e.printStackTrace();
}
}
}
}
Log.v(LOG_TAG,"AudioThread Finished, release audioRecord");
/* encoding finish, release recorder */
if (audioRecord != null) {
audioRecord.stop();
audioRecord.release();
audioRecord = null;
Log.v(LOG_TAG,"audioRecord released");
}
}
}
//---------------------------------------------
// camera thread, gets and encodes video data
//---------------------------------------------
class CameraView extends SurfaceView implements SurfaceHolder.Callback, PreviewCallback {
private SurfaceHolder mHolder;
private Camera mCamera;
public CameraView(Context context, Camera camera) {
super(context);
Log.w("camera","camera view");
mCamera = camera;
mHolder = getHolder();
mHolder.addCallback(CameraView.this);
mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
mCamera.setPreviewCallback(CameraView.this);
}
@Override
public void surfaceCreated(SurfaceHolder holder) {
try {
stopPreview();
mCamera.setPreviewDisplay(holder);
} catch (IOException exception) {
mCamera.release();
mCamera = null;
}
}
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
stopPreview();
Camera.Parameters camParams = mCamera.getParameters();
List sizes = camParams.getSupportedPreviewSizes();
// Sort the list in ascending order
Collections.sort(sizes, new Comparator() {
public int compare(final Camera.Size a, final Camera.Size b) {
return a.width * a.height - b.width * b.height;
}
});
// Pick the first preview size that is equal or bigger, or pick the last (biggest) option if we cannot
// reach the initial settings of imageWidth/imageHeight.
for (int i = 0; i < sizes.size(); i++) {
if ((sizes.get(i).width >= imageWidth && sizes.get(i).height >= imageHeight) || i == sizes.size() - 1) {
imageWidth = sizes.get(i).width;
imageHeight = sizes.get(i).height;
Log.v(LOG_TAG, "Changed to supported resolution: " + imageWidth + "x" + imageHeight);
break;
}
}
camParams.setPreviewSize(imageWidth, imageHeight);
Log.v(LOG_TAG,"Setting imageWidth: " + imageWidth + " imageHeight: " + imageHeight + " frameRate: " + frameRate);
camParams.setPreviewFrameRate(frameRate);
Log.v(LOG_TAG,"Preview Framerate: " + camParams.getPreviewFrameRate());
mCamera.setParameters(camParams);
// Set the holder (which might have changed) again
try {
mCamera.setPreviewDisplay(holder);
mCamera.setPreviewCallback(CameraView.this);
startPreview();
} catch (Exception e) {
Log.e(LOG_TAG, "Could not set preview display in surfaceChanged");
}
}
@Override
public void surfaceDestroyed(SurfaceHolder holder) {
try {
mHolder.addCallback(null);
mCamera.setPreviewCallback(null);
} catch (RuntimeException e) {
// The camera has probably just been released, ignore.
}
}
public void startPreview() {
if (!isPreviewOn && mCamera != null) {
isPreviewOn = true;
mCamera.startPreview();
}
}
public void stopPreview() {
if (isPreviewOn && mCamera != null) {
isPreviewOn = false;
mCamera.stopPreview();
}
}
@Override
public void onPreviewFrame(byte[] data, Camera camera) {
if (audioRecord == null || audioRecord.getRecordingState() != AudioRecord.RECORDSTATE_RECORDING) {
startTime = System.currentTimeMillis();
return;
}
if (RECORD_LENGTH > 0) {
int i = imagesIndex++ % images.length;
yuvImage = images[i];
timestamps[i] = 1000 * (System.currentTimeMillis() - startTime);
}
/* get video data */
if (yuvImage != null && recording) {
((ByteBuffer)yuvImage.image[0].position(0)).put(data);
if (RECORD_LENGTH <= 0) try {
Log.v(LOG_TAG,"Writing Frame");
long t = 1000 * (System.currentTimeMillis() - startTime);
if (t > recorder.getTimestamp()) {
recorder.setTimestamp(t);
}
if(addFilter) {
filter.push(yuvImage);
Frame frame2;
while ((frame2 = filter.pull()) != null) {
recorder.record(frame2, filter.getPixelFormat());
}
} else {
recorder.record(yuvImage);
}
} catch (FFmpegFrameRecorder.Exception | FrameFilter.Exception e) {
Log.v(LOG_TAG,e.getMessage());
e.printStackTrace();
}
}
}
}
@Override
public void onClick(View v) {
if (!recording) {
startRecording();
Log.w(LOG_TAG, "Start Button Pushed");
btnRecorderControl.setText("Stop");
} else {
// This will trigger the audio recording loop to stop and then set isRecorderStart = false;
stopRecording();
Log.w(LOG_TAG, "Stop Button Pushed");
btnRecorderControl.setText("Start");
}
}
}
================================================
FILE: samples/Similarity.java
================================================
/**
* Copyright 2021 JavaCV
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import org.bytedeco.opencv.global.opencv_core;
import org.bytedeco.opencv.global.opencv_imgcodecs;
import org.bytedeco.opencv.global.opencv_imgproc;
import org.bytedeco.opencv.opencv_core.Mat;
import org.bytedeco.opencv.opencv_core.Scalar;
import org.bytedeco.opencv.opencv_core.Size;
/**
* OpenCV similarity measurement examples:
* https://docs.opencv.org/master/d5/dc4/tutorial_video_input_psnr_ssim.html
*
* @author n-kai-cj
*/
public class Similarity {
private static double getPSNR(Mat I1, Mat I2) {
Mat s1 = new Mat();
opencv_core.absdiff(I1, I2, s1); // |I1 - I2|
s1.convertTo(s1, opencv_core.CV_32F); // cannot make a square on 8 bits
s1 = s1.mul(s1).asMat(); // |I1 - I2|^2
Scalar s = opencv_core.sumElems(s1); // sum elements per channel
double sse = s.get(0) + s.get(1) + s.get(2); // sum channels
if (sse <= 1e-10) { // for small values return zero
return 0;
} else {
double mse = sse / (double) (I1.channels() * I1.total());
double psnr = 10.0 * Math.log10((255 * 255) / mse);
return psnr;
}
}
private static Scalar getMSSIM(Mat i1, Mat i2) {
double C1 = 6.5025, C2 = 58.5225;
/***************************** INITS **********************************/
int d = opencv_core.CV_32F;
Mat I1 = new Mat();
Mat I2 = new Mat();
i1.convertTo(I1, d); // cannot calculate on one byte large values
i2.convertTo(I2, d);
Mat I2_2 = I2.mul(I2).asMat(); // I2^2
Mat I1_2 = I1.mul(I1).asMat(); // I1^2
Mat I1_I2 = I1.mul(I2).asMat(); // I1 * I2
/*************************** END INITS **********************************/
// PRELIMINARY COMPUTING
Mat mu1 = new Mat();
Mat mu2 = new Mat();
opencv_imgproc.GaussianBlur(I1, mu1, new Size(11, 11), 1.5);
opencv_imgproc.GaussianBlur(I2, mu2, new Size(11, 11), 1.5);
Mat mu1_2 = mu1.mul(mu1).asMat();
Mat mu2_2 = mu2.mul(mu2).asMat();
Mat mu1_mu2 = mu1.mul(mu2).asMat();
Mat sigma1_2 = new Mat();
Mat sigma2_2 = new Mat();
Mat sigma12 = new Mat();
opencv_imgproc.GaussianBlur(I1_2, sigma1_2, new Size(11, 11), 1.5);
sigma1_2 = opencv_core.subtract(sigma1_2, mu1_2).asMat();
opencv_imgproc.GaussianBlur(I2_2, sigma2_2, new Size(11, 11), 1.5);
sigma2_2 = opencv_core.subtract(sigma2_2, mu2_2).asMat();
opencv_imgproc.GaussianBlur(I1_I2, sigma12, new Size(11, 11), 1.5);
sigma12 = opencv_core.subtract(sigma12, mu1_mu2).asMat();
Mat t1, t2, t3;
t1 = opencv_core.add(opencv_core.multiply(2, mu1_mu2), Scalar.all(C1)).asMat();
t2 = opencv_core.add(opencv_core.multiply(2, sigma12), Scalar.all(C2)).asMat();
t3 = t1.mul(t2).asMat(); // t3 = ((2*mu1_mu2 + C1).*(2*sigma12 + C2))
t1 = opencv_core.add(opencv_core.add(mu1_2, mu2_2), Scalar.all(C1)).asMat();
t2 = opencv_core.add(opencv_core.add(sigma1_2, sigma2_2), Scalar.all(C2)).asMat();
t1 = t1.mul(t2).asMat(); // t1 =((mu1_2 + mu2_2 + C1).*(sigma1_2 + sigma2_2 + C2))
Mat ssim_map = new Mat();
opencv_core.divide(t3, t1, ssim_map); // ssim_map = t3./t1;
Scalar mssim = opencv_core.mean(ssim_map); // mssim = average of ssim map
return mssim;
}
public static void main(String[] args) {
Mat img1 = opencv_imgcodecs.imread("face.jpg");
Mat img2 = img1.clone();
opencv_imgproc.GaussianBlur(img2, img2, new Size(15, 15), 10);
double psnr = getPSNR(img1, img2);
Scalar mssim = getMSSIM(img1, img2);
System.out.println("PSNR: " + psnr);
System.out.printf("SSIM: %f, %f, %f\n", mssim.get(0), mssim.get(1), mssim.get(2));
}
}
================================================
FILE: samples/Smoother.java
================================================
/*
* Copyright (C) 2009-2018 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_imgcodecs.*;
public class Smoother {
public static void smooth(String filename) {
Mat image = imread(filename);
if (image != null) {
GaussianBlur(image, image, new Size(3, 3), 0);
imwrite(filename, image);
}
}
}
================================================
FILE: samples/Square.java
================================================
import java.awt.event.KeyEvent;
import org.bytedeco.javacpp.*;
import org.bytedeco.javacv.*;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_imgcodecs.*;
/**
* I was unable to find the OpenCV squares.c translated into JavaCV, so this
* is a line-by-line translation of the C source.
* The squares.c source used, found here:
* https://code.ros.org/trac/opencv/browser/trunk/opencv/samples/c/squares.c?rev=1429
*
* This is a demo class for finding squares/rectangles in an image, using JavaCV.
*
* All individual imports are kept as is; if you are like me,
* you probably dislike the catch all .* import when trying to understand stuff.
*
* The major headache of the C code was figuring out the
* "drawLines" method, and what parameters "cvPolyLine" is supposed to use.
*
* @author geir.ruud@digitalinferno.com
*/
public class Square {
int thresh = 50;
IplImage img = null;
IplImage img0 = null;
CvMemStorage storage = null;
String wndname = "Square Detection Demo";
// Java spesific
CanvasFrame canvas = null;
OpenCVFrameConverter.ToIplImage converter = new OpenCVFrameConverter.ToIplImage();
// helper function:
// finds a cosine of angle between vectors
// from pt0->pt1 and from pt0->pt2
double angle(CvPoint pt1, CvPoint pt2, CvPoint pt0) {
double dx1 = pt1.x() - pt0.x();
double dy1 = pt1.y() - pt0.y();
double dx2 = pt2.x() - pt0.x();
double dy2 = pt2.y() - pt0.y();
return (dx1*dx2 + dy1*dy2) / Math.sqrt((dx1*dx1 + dy1*dy1) * (dx2*dx2 + dy2*dy2) + 1e-10);
}
// returns sequence of squares detected on the image.
// the sequence is stored in the specified memory storage
CvSeq findSquares4(IplImage img, CvMemStorage storage) {
// Java translation: moved into loop
// CvSeq contours = new CvSeq();
int i, c, l, N = 11;
CvSize sz = cvSize(img.width() & -2, img.height() & -2);
IplImage timg = cvCloneImage(img); // make a copy of input image
IplImage gray = cvCreateImage(sz, 8, 1);
IplImage pyr = cvCreateImage(cvSize(sz.width()/2, sz.height()/2), 8, 3);
IplImage tgray = null;
// Java translation: moved into loop
// CvSeq result = null;
// double s = 0.0, t = 0.0;
// create empty sequence that will contain points -
// 4 points per square (the square's vertices)
CvSeq squares = cvCreateSeq(0, Loader.sizeof(CvSeq.class), Loader.sizeof(CvPoint.class), storage);
// select the maximum ROI in the image
// with the width and height divisible by 2
cvSetImageROI(timg, cvRect(0, 0, sz.width(), sz.height()));
// down-scale and upscale the image to filter out the noise
cvPyrDown(timg, pyr, 7);
cvPyrUp(pyr, timg, 7);
tgray = cvCreateImage(sz, 8, 1);
// find squares in every color plane of the image
for (c = 0; c < 3; c++) {
// extract the c-th color plane
cvSetImageCOI(timg, c+1);
cvCopy(timg, tgray);
// try several threshold levels
for (l = 0; l < N; l++) {
// hack: use Canny instead of zero threshold level.
// Canny helps to catch squares with gradient shading
if (l == 0) {
// apply Canny. Take the upper threshold from slider
// and set the lower to 0 (which forces edges merging)
cvCanny(tgray, gray, 0, thresh, 5);
// dilate canny output to remove potential
// holes between edge segments
cvDilate(gray, gray, null, 1);
} else {
// apply threshold if l!=0:
// tgray(x,y) = gray(x,y) < (l+1)*255/N ? 255 : 0
cvThreshold(tgray, gray, (l+1)*255/N, 255, CV_THRESH_BINARY);
}
// find contours and store them all as a list
// Java translation: moved into the loop
CvSeq contours = new CvSeq();
cvFindContours(gray, storage, contours, Loader.sizeof(CvContour.class), CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE, cvPoint(0,0));
// test each contour
while (contours != null && !contours.isNull()) {
// approximate contour with accuracy proportional
// to the contour perimeter
// Java translation: moved into the loop
CvSeq result = cvApproxPoly(contours, Loader.sizeof(CvContour.class), storage, CV_POLY_APPROX_DP, cvContourPerimeter(contours)*0.02, 0);
// square contours should have 4 vertices after approximation
// relatively large area (to filter out noisy contours)
// and be convex.
// Note: absolute value of an area is used because
// area may be positive or negative - in accordance with the
// contour orientation
if(result.total() == 4 && Math.abs(cvContourArea(result, CV_WHOLE_SEQ, 0)) > 1000 && cvCheckContourConvexity(result) != 0) {
// Java translation: moved into loop
double s = 0.0, t = 0.0;
for( i = 0; i < 5; i++ ) {
// find minimum angle between joint
// edges (maximum of cosine)
if( i >= 2 ) {
// Java translation:
// Comment from the HoughLines.java sample code:
// " Based on JavaCPP, the equivalent of the C code:
// CvPoint* line = (CvPoint*)cvGetSeqElem(lines,i);
// CvPoint first=line[0];
// CvPoint second=line[1];
// is:
// Pointer line = cvGetSeqElem(lines, i);
// CvPoint first = new CvPoint(line).position(0);
// CvPoint second = new CvPoint(line).position(1);
// "
// ... so after some trial and error this seem to work
// t = fabs(angle(
// (CvPoint*)cvGetSeqElem( result, i ),
// (CvPoint*)cvGetSeqElem( result, i-2 ),
// (CvPoint*)cvGetSeqElem( result, i-1 )));
t = Math.abs(angle(new CvPoint(cvGetSeqElem(result, i)),
new CvPoint(cvGetSeqElem(result, i-2)),
new CvPoint(cvGetSeqElem(result, i-1))));
s = s > t ? s : t;
}
}
// if cosines of all angles are small
// (all angles are ~90 degree) then write quandrange
// vertices to resultant sequence
if (s < 0.3)
for( i = 0; i < 4; i++ ) {
cvSeqPush(squares, cvGetSeqElem(result, i));
}
}
// take the next contour
contours = contours.h_next();
}
}
}
// release all the temporary images
cvReleaseImage(gray);
cvReleaseImage(pyr);
cvReleaseImage(tgray);
cvReleaseImage(timg);
return squares;
}
// the function draws all the squares in the image
void drawSquares(IplImage img, CvSeq squares) {
// Java translation: Here the code is somewhat different from the C version.
// I was unable to get straight forward CvPoint[] arrays
// working with "reader" and the "CV_READ_SEQ_ELEM".
// CvSeqReader reader = new CvSeqReader();
IplImage cpy = cvCloneImage(img);
int i = 0;
// Used by attempt 3
// Create a "super"-slice, consisting of the entire sequence of squares
CvSlice slice = new CvSlice(squares);
// initialize reader of the sequence
// cvStartReadSeq(squares, reader, 0);
// read 4 sequence elements at a time (all vertices of a square)
for(i = 0; i < squares.total(); i += 4) {
// // Attempt 1:
// // This does not work, uses the "reader"
// CvPoint pt[] = new CvPoint[]{new CvPoint(1), new CvPoint(1), new CvPoint(1), new CvPoint(1)};
// PointerPointer rect = new PointerPointer(pt);
// int count[] = new int[]{4};
//
// CV_READ_SEQ_ELEM(pt[0], reader);
// CV_READ_SEQ_ELEM(pt[1], reader);
// CV_READ_SEQ_ELEM(pt[2], reader);
// CV_READ_SEQ_ELEM(pt[3], reader);
// // Attempt 2:
// // This works, somewhat similar to the C code, somewhat messy, does not use the "reader"
// CvPoint pt[] = new CvPoint[]{
// new CvPoint(cvGetSeqElem(squares, i)),
// new CvPoint(cvGetSeqElem(squares, i + 1)),
// new CvPoint(cvGetSeqElem(squares, i + 2)),
// new CvPoint(cvGetSeqElem(squares, i + 3))};
// PointerPointer rect = new PointerPointer(pt);
// int count[] = new int[]{4};
// Attempt 3:
// This works, may be the "cleanest" solution, does not use the "reader"
CvPoint rect = new CvPoint(4);
IntPointer count = new IntPointer(1).put(4);
// get the 4 corner slice from the "super"-slice
cvCvtSeqToArray(squares, rect, slice.start_index(i).end_index(i + 4));
// // Attempt 4:
// // This works, and look the most like the original C code, uses the "reader"
// CvPoint rect = new CvPoint(4);
// int count[] = new int[]{4};
//
// // read 4 vertices
// CV_READ_SEQ_ELEM(rect.position(0), reader);
// CV_READ_SEQ_ELEM(rect.position(1), reader);
// CV_READ_SEQ_ELEM(rect.position(2), reader);
// CV_READ_SEQ_ELEM(rect.position(3), reader);
// draw the square as a closed polyline
// Java translation: gotcha (re-)setting the opening "position" of the CvPoint sequence thing
cvPolyLine(cpy, rect.position(0), count, 1, 1, CV_RGB(0,255,0), 3, CV_AA, 0);
}
// show the resultant image
// cvShowImage(wndname, cpy);
canvas.showImage(converter.convert(cpy));
cvReleaseImage(cpy);
}
String names[] = new String[]{ "pic1.png", "pic2.png", "pic3.png",
"pic4.png", "pic5.png", "pic6.png" };
public static void main(String args[]) throws Exception {
new Square().main();
}
public void main() throws InterruptedException {
// Java translation: c not used
int i; // , c;
// create memory storage that will contain all the dynamic data
storage = cvCreateMemStorage(0);
for(i = 0; i < names.length; i++) {
// load i-th image
// Java translation
String filePathAndName = Square.class.getClassLoader().getResource(names[i]).getPath();
filePathAndName = filePathAndName == null || filePathAndName.isEmpty() ? names[i] : filePathAndName;
// img0 = cvLoadImage(names[i], 1);
img0 = cvLoadImage(filePathAndName, 1);
if (img0 == null) {
System.err.println("Couldn't load " + names[i]);
continue;
}
img = cvCloneImage(img0);
// create window and a trackbar (slider) with parent "image" and set callback
// (the slider regulates upper threshold, passed to Canny edge detector)
// Java translation
canvas = new CanvasFrame(wndname, 1);
canvas.setDefaultCloseOperation(javax.swing.JFrame.EXIT_ON_CLOSE);
// cvNamedWindow( wndname, 1 );
// find and draw the squares
drawSquares(img, findSquares4(img, storage));
// wait for key.
// Also the function cvWaitKey takes care of event processing
// Java translation
KeyEvent key = canvas.waitKey(0);
// c = cvWaitKey(0);
// release both images
cvReleaseImage(img);
cvReleaseImage(img0);
// clear memory storage - reset free space position
cvClearMemStorage(storage);
if (key.getKeyCode() == 27) {
break;
}
}
// cvDestroyWindow( wndname );
if (canvas != null) {
canvas.dispose();
}
}
}
================================================
FILE: samples/TemplateMatching.java
================================================
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ThreadLocalRandom;
import org.bytedeco.javacv.*;
import org.bytedeco.javacpp.*;
import org.bytedeco.javacpp.indexer.FloatIndexer;
import org.bytedeco.opencv.opencv_calib3d.*;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_highgui.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import org.bytedeco.opencv.opencv_objdetect.*;
import static org.bytedeco.opencv.global.opencv_calib3d.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_highgui.*;
import static org.bytedeco.opencv.global.opencv_imgcodecs.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_objdetect.*;
/**
* Example of template javacv (opencv) template matching using the last java build
*
* We need 2 default parameters like this (source image, image to find )
* "C:\Users\Waldema\Desktop\bg.jpg" "C:\Users\Waldema\Desktop\imageToFind.jpg"
*
* @author Waldemar Neto
*/
public class TemplateMatching {
public static void main(String[] args) throws Exception {
newStyle(args);
//oldStyle(args);
}
public static void newStyle(String[] args){
//read in image default colors
Mat sourceColor = imread(args[0]);
Mat sourceGrey = new Mat(sourceColor.size(), CV_8UC1);
cvtColor(sourceColor, sourceGrey, COLOR_BGR2GRAY);
//load in template in grey
Mat template = imread(args[1],IMREAD_GRAYSCALE);//int = 0
//Size for the result image
Size size = new Size(sourceGrey.cols()-template.cols()+1, sourceGrey.rows()-template.rows()+1);
Mat result = new Mat(size, CV_32FC1);
matchTemplate(sourceGrey, template, result, TM_CCORR_NORMED);
DoublePointer minVal= new DoublePointer();
DoublePointer maxVal= new DoublePointer();
Point min = new Point();
Point max = new Point();
minMaxLoc(result, minVal, maxVal, min, max, null);
rectangle(sourceColor,new Rect(max.x(),max.y(),template.cols(),template.rows()), randColor(), 2, 0, 0);
imshow("Original marked", sourceColor);
imshow("Ttemplate", template);
imshow("Results matrix", result);
waitKey(0);
destroyAllWindows();
}
// some usefull things.
public static Scalar randColor(){
int b,g,r;
b= ThreadLocalRandom.current().nextInt(0, 255 + 1);
g= ThreadLocalRandom.current().nextInt(0, 255 + 1);
r= ThreadLocalRandom.current().nextInt(0, 255 + 1);
return new Scalar (b,g,r,0);
}
public static List getPointsFromMatAboveThreshold(Mat m, float t){
List matches = new ArrayList();
FloatIndexer indexer = m.createIndexer();
for (int y = 0; y < m.rows(); y++) {
for (int x = 0; x < m.cols(); x++) {
if (indexer.get(y,x)>t) {
System.out.println("(" + x + "," + y +") = "+ indexer.get(y,x));
matches.add(new Point(x, y));
}
}
}
return matches;
}
public static void oldStyle(String[] args){
//get color source image to draw red rect on later
IplImage srcColor = cvLoadImage(args[0]);
//create blank 1 channel image same size as the source
IplImage src = cvCreateImage(cvGetSize(srcColor), IPL_DEPTH_8U, 1);
//convert source to grey and copy to src
cvCvtColor(srcColor, src, CV_BGR2GRAY);
//get the image to match loaded in greyscale.
IplImage tmp = cvLoadImage(args[1], 0);
//this image will hold the strength of the match
//as the template is translated across the image
IplImage result = cvCreateImage(
cvSize(src.width() - tmp.width() + 1,
src.height() - tmp.height() + 1), IPL_DEPTH_32F, src.nChannels());
cvZero(result);
// Match Template Function from OpenCV
cvMatchTemplate(src, tmp, result, CV_TM_CCORR_NORMED);
// double[] min_val = new double[2];
// double[] max_val = new double[2];
DoublePointer min_val = new DoublePointer();
DoublePointer max_val = new DoublePointer();
CvPoint minLoc = new CvPoint();
CvPoint maxLoc = new CvPoint();
cvMinMaxLoc(result, min_val, max_val, minLoc, maxLoc, null);
// Get the Max or Min Correlation Value
// System.out.println(Arrays.toString(min_val));
// System.out.println(Arrays.toString(max_val));
CvPoint point = new CvPoint();
point.x(maxLoc.x() + tmp.width());
point.y(maxLoc.y() + tmp.height());
// cvMinMaxLoc(src, min_val, max_val,0,0,result);
cvRectangle(srcColor, maxLoc, point, CvScalar.RED, 2, 8, 0); // Draw a
// Rectangle for
// Matched
// Region
cvShowImage("Lena Image", srcColor);
cvWaitKey(0);
cvReleaseImage(srcColor);
cvReleaseImage(src);
cvReleaseImage(tmp);
cvReleaseImage(result);
}
}
================================================
FILE: samples/WebcamAndMicrophoneCapture.java
================================================
/**
* @author Ben Davenport
*
* This class is a simple example for broadcasting a video capture device (ie, webcam) and an audio capture device (ie, microphone)
* using an FFmpegFrameRecorder.
*
* FFmpegFrameRecorder allows the output destination to be either a FILE or an RTMP endpoint (Wowza, FMS, et al)
*
* IMPORTANT: There are potential timing issues with audio/video synchronicity across threads, I am working on finding a solution, but
* chime in if you can fig it out :o)
*/
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.ShortBuffer;
import java.util.concurrent.ScheduledThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.DataLine;
import javax.sound.sampled.LineUnavailableException;
import javax.sound.sampled.Mixer;
import javax.sound.sampled.TargetDataLine;
import org.bytedeco.ffmpeg.global.avcodec;
import org.bytedeco.javacv.CanvasFrame;
import org.bytedeco.javacv.FFmpegFrameRecorder;
import org.bytedeco.javacv.Frame;
import org.bytedeco.javacv.FrameRecorder.Exception;
import org.bytedeco.javacv.OpenCVFrameGrabber;
public class WebcamAndMicrophoneCapture
{
final private static int WEBCAM_DEVICE_INDEX = 1;
final private static int AUDIO_DEVICE_INDEX = 4;
final private static int FRAME_RATE = 30;
final private static int GOP_LENGTH_IN_FRAMES = 60;
private static long startTime = 0;
private static long videoTS = 0;
public static void main(String[] args) throws Exception, org.bytedeco.javacv.FrameGrabber.Exception
{
final int captureWidth = 1280;
final int captureHeight = 720;
// The available FrameGrabber classes include OpenCVFrameGrabber (opencv_videoio),
// DC1394FrameGrabber, FlyCapture2FrameGrabber, OpenKinectFrameGrabber,
// PS3EyeFrameGrabber, VideoInputFrameGrabber, and FFmpegFrameGrabber.
final OpenCVFrameGrabber grabber = new OpenCVFrameGrabber(WEBCAM_DEVICE_INDEX);
grabber.setImageWidth(captureWidth);
grabber.setImageHeight(captureHeight);
grabber.start();
// org.bytedeco.javacv.FFmpegFrameRecorder.FFmpegFrameRecorder(String
// filename, int imageWidth, int imageHeight, int audioChannels)
// For each param, we're passing in...
// filename = either a path to a local file we wish to create, or an
// RTMP url to an FMS / Wowza server
// imageWidth = width we specified for the grabber
// imageHeight = height we specified for the grabber
// audioChannels = 2, because we like stereo
final FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(
"rtmp://my-streaming-server/app_name_here/instance_name/stream_name",
captureWidth, captureHeight, 2);
recorder.setInterleaved(true);
// decrease "startup" latency in FFMPEG (see:
// https://trac.ffmpeg.org/wiki/StreamingGuide)
recorder.setVideoOption("tune", "zerolatency");
// tradeoff between quality and encode speed
// possible values are ultrafast,superfast, veryfast, faster, fast,
// medium, slow, slower, veryslow
// ultrafast offers us the least amount of compression (lower encoder
// CPU) at the cost of a larger stream size
// at the other end, veryslow provides the best compression (high
// encoder CPU) while lowering the stream size
// (see: https://trac.ffmpeg.org/wiki/Encode/H.264)
recorder.setVideoOption("preset", "ultrafast");
// Constant Rate Factor (see: https://trac.ffmpeg.org/wiki/Encode/H.264)
recorder.setVideoOption("crf", "28");
// 2000 kb/s, reasonable "sane" area for 720
recorder.setVideoBitrate(2000000);
recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
recorder.setFormat("flv");
// FPS (frames per second)
recorder.setFrameRate(FRAME_RATE);
// Key frame interval, in our case every 2 seconds -> 30 (fps) * 2 = 60
// (gop length)
recorder.setGopSize(GOP_LENGTH_IN_FRAMES);
// We don't want variable bitrate audio
recorder.setAudioOption("crf", "0");
// Highest quality
recorder.setAudioQuality(0);
// 192 Kbps
recorder.setAudioBitrate(192000);
recorder.setSampleRate(44100);
recorder.setAudioChannels(2);
recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
// Jack 'n coke... do it...
recorder.start();
// Thread for audio capture, this could be in a nested private class if you prefer...
new Thread(new Runnable() {
@Override
public void run()
{
// Pick a format...
// NOTE: It is better to enumerate the formats that the system supports,
// because getLine() can error out with any particular format...
// For us: 44.1 sample rate, 16 bits, stereo, signed, little endian
AudioFormat audioFormat = new AudioFormat(44100.0F, 16, 2, true, false);
// Get TargetDataLine with that format
Mixer.Info[] minfoSet = AudioSystem.getMixerInfo();
Mixer mixer = AudioSystem.getMixer(minfoSet[AUDIO_DEVICE_INDEX]);
DataLine.Info dataLineInfo = new DataLine.Info(TargetDataLine.class, audioFormat);
try
{
// Open and start capturing audio
// It's possible to have more control over the chosen audio device with this line:
// TargetDataLine line = (TargetDataLine)mixer.getLine(dataLineInfo);
final TargetDataLine line = (TargetDataLine)AudioSystem.getLine(dataLineInfo);
line.open(audioFormat);
line.start();
final int sampleRate = (int) audioFormat.getSampleRate();
final int numChannels = audioFormat.getChannels();
// Let's initialize our audio buffer...
final int audioBufferSize = sampleRate * numChannels;
final byte[] audioBytes = new byte[audioBufferSize];
// Using a ScheduledThreadPoolExecutor vs a while loop with
// a Thread.sleep will allow
// us to get around some OS specific timing issues, and keep
// to a more precise
// clock as the fixed rate accounts for garbage collection
// time, etc
// a similar approach could be used for the webcam capture
// as well, if you wish
ScheduledThreadPoolExecutor exec = new ScheduledThreadPoolExecutor(1);
exec.scheduleAtFixedRate(new Runnable() {
@Override
public void run()
{
try
{
// Read from the line... non-blocking
int nBytesRead = 0;
while (nBytesRead == 0) {
nBytesRead = line.read(audioBytes, 0, line.available());
}
// Since we specified 16 bits in the AudioFormat,
// we need to convert our read byte[] to short[]
// (see source from FFmpegFrameRecorder.recordSamples for AV_SAMPLE_FMT_S16)
// Let's initialize our short[] array
int nSamplesRead = nBytesRead / 2;
short[] samples = new short[nSamplesRead];
// Let's wrap our short[] into a ShortBuffer and
// pass it to recordSamples
ByteBuffer.wrap(audioBytes).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(samples);
ShortBuffer sBuff = ShortBuffer.wrap(samples, 0, nSamplesRead);
// recorder is instance of
// org.bytedeco.javacv.FFmpegFrameRecorder
recorder.recordSamples(sampleRate, numChannels, sBuff);
}
catch (org.bytedeco.javacv.FrameRecorder.Exception e)
{
e.printStackTrace();
}
}
}, 0, (long) 1000 / FRAME_RATE, TimeUnit.MILLISECONDS);
}
catch (LineUnavailableException e1)
{
e1.printStackTrace();
}
}
}).start();
// A really nice hardware accelerated component for our preview...
final CanvasFrame cFrame = new CanvasFrame("Capture Preview", CanvasFrame.getDefaultGamma() / grabber.getGamma());
Frame capturedFrame = null;
// While we are capturing...
while ((capturedFrame = grabber.grab()) != null)
{
if (cFrame.isVisible())
{
// Show our frame in the preview
cFrame.showImage(capturedFrame);
}
// Let's define our start time...
// This needs to be initialized as close to when we'll use it as
// possible,
// as the delta from assignment to computed time could be too high
if (startTime == 0)
startTime = System.currentTimeMillis();
// Create timestamp for this frame
videoTS = 1000 * (System.currentTimeMillis() - startTime);
// Check for AV drift
if (videoTS > recorder.getTimestamp())
{
System.out.println(
"Lip-flap correction: "
+ videoTS + " : "
+ recorder.getTimestamp() + " -> "
+ (videoTS - recorder.getTimestamp()));
// We tell the recorder to write this frame at this timestamp
recorder.setTimestamp(videoTS);
}
// Send the frame to the org.bytedeco.javacv.FFmpegFrameRecorder
recorder.record(capturedFrame);
}
cFrame.dispose();
recorder.stop();
grabber.stop();
}
}
================================================
FILE: samples/YOLONet.java
================================================
/*
MIT License
Copyright (c) 2021 Florian Bruggisser
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
/*
YOLONet Example Information
---------------------------
This is a basic implementation of a YOLO object detection network inference example.
It works with most variants of YOLO (YOLOv2, YOLOv2-tiny, YOLOv3, YOLOv3-tiny, YOLOv3-tiny-prn, YOLOv4, YOLOv4-tiny).
YOLO9000 is not support by OpenCV DNN.
To run the example download the following files and place them in the root folder of your project:
YOLOv4 Configuration: https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov4.cfg
YOLOv4 Weights: https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v3_optimal/yolov4.weights
COCO Names: https://raw.githubusercontent.com/AlexeyAB/darknet/master/data/coco.names
Dog Demo Image: https://raw.githubusercontent.com/AlexeyAB/darknet/master/data/dog.jpg
For faster inferencing CUDA is highly recommended.
On CPU it is recommended to decrease the width & height of the network or use the tiny variants.
*/
import org.bytedeco.javacpp.FloatPointer;
import org.bytedeco.javacpp.IntPointer;
import org.bytedeco.javacpp.indexer.FloatIndexer;
import org.bytedeco.opencv.global.opencv_dnn;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_dnn.Net;
import org.bytedeco.opencv.opencv_text.FloatVector;
import org.bytedeco.opencv.opencv_text.IntVector;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
import static org.bytedeco.opencv.global.opencv_core.CV_32F;
import static org.bytedeco.opencv.global.opencv_core.getCudaEnabledDeviceCount;
import static org.bytedeco.opencv.global.opencv_dnn.*;
import static org.bytedeco.opencv.global.opencv_highgui.imshow;
import static org.bytedeco.opencv.global.opencv_highgui.waitKey;
import static org.bytedeco.opencv.global.opencv_imgcodecs.imread;
import static org.bytedeco.opencv.global.opencv_imgproc.LINE_8;
import static org.bytedeco.opencv.global.opencv_imgproc.rectangle;
public class YOLONet {
public static void main(String[] args) {
Mat image = imread("dog.jpg");
YOLONet yolo = new YOLONet(
"yolov4.cfg",
"yolov4.weights",
"coco.names",
608, 608);
yolo.setup();
List results = yolo.predict(image);
System.out.printf("Detected %d objects:\n", results.size());
for(ObjectDetectionResult result : results) {
System.out.printf("\t%s - %.2f%%\n", result.className, result.confidence * 100f);
// annotate on image
rectangle(image,
new Point(result.x, result.y),
new Point(result.x + result.width, result.y + result.height),
Scalar.MAGENTA, 2, LINE_8, 0);
}
imshow("YOLO", image);
waitKey();
}
private Path configPath;
private Path weightsPath;
private Path namesPath;
private int width;
private int height;
private float confidenceThreshold = 0.5f;
private float nmsThreshold = 0.4f;
private Net net;
private StringVector outNames;
private List names;
/**
* Creates a new YOLO network.
* @param configPath Path to the configuration file.
* @param weightsPath Path to the weights file.
* @param namesPath Path to the names file.
* @param width Width of the network as defined in the configuration.
* @param height Height of the network as defined in the configuration.
*/
public YOLONet(String configPath, String weightsPath, String namesPath, int width, int height) {
this.configPath = Paths.get(configPath);
this.weightsPath = Paths.get(weightsPath);
this.namesPath = Paths.get(namesPath);
this.width = width;
this.height = height;
}
/**
* Initialises the network.
*
* @return True if the network initialisation was successful.
*/
public boolean setup() {
net = readNetFromDarknet(
configPath.toAbsolutePath().toString(),
weightsPath.toAbsolutePath().toString());
// setup output layers
outNames = net.getUnconnectedOutLayersNames();
// enable cuda backend if available
if (getCudaEnabledDeviceCount() > 0) {
net.setPreferableBackend(opencv_dnn.DNN_BACKEND_CUDA);
net.setPreferableTarget(opencv_dnn.DNN_TARGET_CUDA);
}
// read names file
try {
names = Files.readAllLines(namesPath);
} catch (IOException e) {
System.err.println("Could not read names file!");
e.printStackTrace();
}
return !net.empty();
}
/**
* Runs the object detection on the frame.
*
* @param frame Input frame.
* @return List of objects that have been detected.
*/
public List predict(Mat frame) {
Mat inputBlob = blobFromImage(frame,
1 / 255.0,
new Size(width, height),
new Scalar(0.0),
true, false, CV_32F);
net.setInput(inputBlob);
// run detection
MatVector outs = new MatVector(outNames.size());
net.forward(outs, outNames);
// evaluate result
List result = postprocess(frame, outs);
// cleanup
outs.releaseReference();
inputBlob.release();
return result;
}
/**
* Remove the bounding boxes with low confidence using non-maxima suppression
*
* @param frame Input frame
* @param outs Network outputs
* @return List of objects
*/
private List postprocess(Mat frame, MatVector outs) {
final IntVector classIds = new IntVector();
final FloatVector confidences = new FloatVector();
final RectVector boxes = new RectVector();
for (int i = 0; i < outs.size(); ++i) {
// extract the bounding boxes that have a high enough score
// and assign their highest confidence class prediction.
Mat result = outs.get(i);
FloatIndexer data = result.createIndexer();
for (int j = 0; j < result.rows(); j++) {
// minMaxLoc implemented in java because it is 1D
int maxIndex = -1;
float maxScore = Float.MIN_VALUE;
for (int k = 5; k < result.cols(); k++) {
float score = data.get(j, k);
if (score > maxScore) {
maxScore = score;
maxIndex = k - 5;
}
}
if (maxScore > confidenceThreshold) {
int centerX = (int) (data.get(j, 0) * frame.cols());
int centerY = (int) (data.get(j, 1) * frame.rows());
int width = (int) (data.get(j, 2) * frame.cols());
int height = (int) (data.get(j, 3) * frame.rows());
int left = centerX - width / 2;
int top = centerY - height / 2;
classIds.push_back(maxIndex);
confidences.push_back(maxScore);
boxes.push_back(new Rect(left, top, width, height));
}
}
data.release();
result.release();
}
// remove overlapping bounding boxes with NMS
IntPointer indices = new IntPointer(confidences.size());
FloatPointer confidencesPointer = new FloatPointer(confidences.size());
confidencesPointer.put(confidences.get());
NMSBoxes(boxes, confidencesPointer, confidenceThreshold, nmsThreshold, indices, 1.f, 0);
// create result list
List detections = new ArrayList<>();
for (int i = 0; i < indices.limit(); ++i) {
final int idx = indices.get(i);
final Rect box = boxes.get(idx);
final int clsId = classIds.get(idx);
detections.add(new ObjectDetectionResult() {{
classId = clsId;
className = names.get(clsId);
confidence = confidences.get(idx);
x = box.x();
y = box.y();
width = box.width();
height = box.height();
}});
box.releaseReference();
}
// cleanup
indices.releaseReference();
confidencesPointer.releaseReference();
classIds.releaseReference();
confidences.releaseReference();
boxes.releaseReference();
return detections;
}
/**
* Dataclass for object detection result.
*/
public static class ObjectDetectionResult {
public int classId;
public String className;
public float confidence;
public int x;
public int y;
public int width;
public int height;
}
}
================================================
FILE: samples/haarcascade_frontalface_alt2.xml
================================================
BOOST
HAAR
20
20
109
0
20
<_>
3
3.5069230198860168e-01
<_>
0 1 0 4.3272329494357109e-03 -1 -2 1 1.3076160103082657e-02
3.8381900638341904e-02 8.9652568101882935e-01
2.6293140649795532e-01
<_>
0 1 2 5.2434601821005344e-04 -1 -2 3 4.4573000632226467e-03
1.0216630250215530e-01 1.2384019792079926e-01
6.9103831052780151e-01
<_>
1 0 4 -9.2708261217921972e-04 -1 -2 5 3.3989109215326607e-04
1.9536970555782318e-01 2.1014410257339478e-01
8.2586747407913208e-01
<_>
9
3.4721779823303223e+00
<_>
0 1 6 2.3025739938020706e-03 -1 -2 7 4.4174338690936565e-03
1.0183759778738022e-01 8.2190579175949097e-01
1.9565549492835999e-01
<_>
0 1 8 2.2203210741281509e-02 -1 -2 9 -1.7283110355492681e-04
2.2054070234298706e-01 7.3263257741928101e-02
5.9314841032028198e-01
<_>
0 1 10 4.3567270040512085e-03 -1 -2 11
-2.6032889727503061e-03
1.8441149592399597e-01 4.0322139859199524e-01
8.0665212869644165e-01
<_>
0 1 12 1.7309630056843162e-03 -1 -2 13
-7.8146401792764664e-03
2.5483280420303345e-01 6.0570698976516724e-01
2.7790638804435730e-01
<_>
0 1 14 -8.7343417108058929e-03 -1 -2 15
9.4522320432588458e-04
2.8899800777435303e-01 7.6165872812271118e-01
3.4956431388854980e-01
<_>
1 0 16 4.9414858222007751e-02 -1 -2 17
4.4891750440001488e-03
8.1516528129577637e-01 2.8087830543518066e-01
6.0277748107910156e-01
<_>
1 0 18 6.0313619673252106e-02 -1 -2 19
-1.0762850288301706e-03
7.6075017452239990e-01 4.4440358877182007e-01
1.4373120665550232e-01
<_>
1 0 20 -9.5083238556981087e-03 -1 -2 21
7.6601309701800346e-03
5.3181701898574829e-01 5.4110521078109741e-01
2.1806870400905609e-01
<_>
1 0 22 7.6467678882181644e-03 -1 -2 23
-8.4662932204082608e-04
1.1589600145816803e-01 2.3406790196895599e-01
5.9903818368911743e-01
<_>
14
5.9844889640808105e+00
<_>
1 0 24 -4.8506218008697033e-03 -1 -2 25
-4.6141650527715683e-03
1.8054960668087006e-01 2.1778939664363861e-01
8.0182367563247681e-01
<_>
0 1 26 -2.4301309604197741e-03 -1 -2 27
4.1787960799410939e-04
1.1413549631834030e-01 1.2030939757823944e-01
6.1085307598114014e-01
<_>
0 1 28 1.0010929545387626e-03 -1 -2 29
1.0577100329101086e-03
2.0799599587917328e-01 3.3020541071891785e-01
7.5110942125320435e-01
<_>
1 0 30 1.2376549420878291e-03 -1 -2 31
3.5315038985572755e-04
2.7682220935821533e-01 1.6682930290699005e-01
5.8294767141342163e-01
<_>
0 1 32 -1.1953660286962986e-02 -1 -2 33
1.4182999730110168e-03
1.5087880194187164e-01 4.3912279605865479e-01
7.6465952396392822e-01
<_>
1 0 34 3.4642980899661779e-03 -1 -2 35
-1.4948950149118900e-02
2.6515561342239380e-01 2.2980530560016632e-01
5.4421657323837280e-01
<_>
1 0 36 -1.0506849503144622e-03 -1 -2 37
-4.0782918222248554e-03
3.6228439211845398e-01 2.6012599468231201e-01
7.2336578369140625e-01
<_>
0 1 38 5.4242828628048301e-04 -1 -2 39
-7.3204059153795242e-03
3.8496789336204529e-01 2.9655128717422485e-01
5.4803091287612915e-01
<_>
0 1 40 1.1421289527788758e-03 -1 -2 41
1.1783400550484657e-03
4.1047701239585876e-01 7.2390240430831909e-01
2.7872839570045471e-01
<_>
0 1 42 4.4077109545469284e-02 -1 -2 43
3.7900090683251619e-03
5.6405162811279297e-01 5.9475481510162354e-01
3.3120200037956238e-01
<_>
0 1 44 -2.4291418958455324e-03 -1 -2 45
9.4262324273586273e-03
6.6032320261001587e-01 4.6806651353836060e-01
2.0643380284309387e-01
<_>
0 1 46 8.0630257725715637e-03 -1 -2 47
5.2240812219679356e-03
5.2988511323928833e-01 5.2816027402877808e-01
1.9095499813556671e-01
<_>
0 1 48 -7.0630568079650402e-03 -1 -2 49
5.6897541508078575e-03
1.3806459307670593e-01 5.4906368255615234e-01
1.2602810561656952e-01
<_>
0 1 50 1.2472929665818810e-03 -1 -2 51
4.9543488770723343e-02
2.3726630210876465e-01 5.2401661872863770e-01
1.7692160606384277e-01
<_>
19
8.5117864608764648e+00
<_>
1 0 52 -4.9326149746775627e-03 -1 -2 53
2.7918140403926373e-05
1.9980649650096893e-01 2.2993800044059753e-01
7.3932111263275146e-01
<_>
1 0 54 3.0876200180500746e-03 -1 -2 55
7.4669660534709692e-06
1.5338400006294250e-01 2.0368589460849762e-01
5.8549159765243530e-01
<_>
0 1 56 1.8739729421213269e-03 -1 -2 57
9.3380251200869679e-04
2.0498959720134735e-01 3.2341998815536499e-01
7.3230141401290894e-01
<_>
0 1 58 1.9151850137859583e-03 -1 -2 59
-5.9683797881007195e-03
3.0451491475105286e-01 2.9321339726448059e-01
5.6212961673736572e-01
<_>
0 1 60 -7.2115601506084204e-04 -1 -2 61
-5.9663117863237858e-03
3.6580368876457214e-01 2.7121558785438538e-01
7.2263348102569580e-01
<_>
0 1 62 3.0874179676175117e-02 -1 -2 63
-1.1099710129201412e-02
4.4198378920555115e-01 3.6129769682884216e-01
5.2514511346817017e-01
<_>
0 1 64 2.1164179779589176e-03 -1 -2 65
-9.4317439943552017e-03
3.6286169290542603e-01 1.6010950505733490e-01
7.0522767305374146e-01
<_>
0 1 66 -3.5266019403934479e-03 -1 -2 67
-1.6907559474930167e-03
1.3012880086898804e-01 1.7863239347934723e-01
5.5215299129486084e-01
<_>
0 1 68 4.6470930101349950e-04 -1 -2 69
-1.0215570218861103e-02
3.4873831272125244e-01 2.6739910244941711e-01
6.6679191589355469e-01
<_>
1 0 70 1.2634709710255265e-03 -1 -2 71
-1.1875299736857414e-02
3.4378638863563538e-01 5.9953361749649048e-01
3.4977179765701294e-01
<_>
0 1 72 -1.0732339695096016e-02 -1 -2 73
7.1836481802165508e-03
2.1504899859428406e-01 6.2714362144470215e-01
2.5195419788360596e-01
<_>
0 1 74 -2.8340889140963554e-02 -1 -2 75
-4.5813230099156499e-04
8.2411892712116241e-02 5.9100568294525146e-01
3.7052011489868164e-01
<_>
1 0 76 4.2940340936183929e-03 -1 -2 77
1.0751079767942429e-02
1.5947279334068298e-01 5.9804809093475342e-01
2.8325080871582031e-01
<_>
1 0 78 2.2465119138360023e-02 -1 -2 79
-5.7988539338111877e-02
7.8770911693572998e-01 1.5557409822940826e-01
5.2396571636199951e-01
<_>
1 0 80 7.2110891342163086e-03 -1 -2 81
-4.8367571085691452e-02
6.6203659772872925e-01 1.4247199892997742e-01
4.4298338890075684e-01
<_>
0 1 82 -1.4418059960007668e-02 -1 -2 83
-2.3156389594078064e-02
1.5885409712791443e-01 2.3757989704608917e-01
5.2171349525451660e-01
<_>
1 0 84 7.6985340565443039e-03 -1 -2 85
-5.6248619221150875e-03
1.9417250156402588e-01 6.2784057855606079e-01
3.7460449337959290e-01
<_>
1 0 86 -7.2936748620122671e-04 -1 -2 87
6.1783898854628205e-04
3.8409221172332764e-01 3.1064930558204651e-01
5.5378472805023193e-01
<_>
1 0 88 -4.5803939428878948e-05 -1 -2 89
-1.4719359569426160e-05
3.4444490075111389e-01 2.7295520901679993e-01
6.4289510250091553e-01
<_>
19
8.4680156707763672e+00
<_>
0 1 90 -1.3469370314851403e-03 -1 -2 91
-2.4774789344519377e-03
1.6570860147476196e-01 2.2738510370254517e-01
6.9893497228622437e-01
<_>
0 1 92 5.2632777951657772e-03 -1 -2 93
4.9075339920818806e-03
1.5120740234851837e-01 5.5644702911376953e-01
1.6054420173168182e-01
<_>
0 1 94 -2.3254349362105131e-03 -1 -2 95
-1.4665479538962245e-03
1.8802590668201447e-01 3.1224989891052246e-01
7.1653962135314941e-01
<_>
1 0 96 -1.2311690300703049e-01 -1 -2 97
2.2108340635895729e-03
3.8595831394195557e-01 2.4552939832210541e-01
5.6957101821899414e-01
<_>
0 1 98 2.0661531016230583e-03 -1 -2 99
3.6130280932411551e-04
2.7165201306343079e-01 2.2933620214462280e-01
7.2086298465728760e-01
<_>
1 0 100 7.9957872629165649e-02 -1 -2 101
2.6064720004796982e-03
7.8336209058761597e-01 5.5452322959899902e-01
2.5506898760795593e-01
<_>
1 0 102 6.5699010156095028e-03 -1 -2 103
1.6259610420092940e-03
1.8193900585174561e-01 3.5298758745193481e-01
6.5528190135955811e-01
<_>
0 1 104 3.6204981151968241e-03 -1 -2 105
-4.4391951523721218e-03
5.4623097181320190e-01 1.3598430156707764e-01
5.4158151149749756e-01
<_>
0 1 106 -9.0540945529937744e-03 -1 -2 107
-4.6067481162026525e-04
1.1151199787855148e-01 5.8467197418212891e-01
2.5983488559722900e-01
<_>
0 1 108 -5.6621041148900986e-03 -1 -2 109
5.1165837794542313e-03
1.6105690598487854e-01 5.3766787052154541e-01
1.7394550144672394e-01
<_>
0 1 110 -2.1362339612096548e-03 -1 -2 111
-5.4809921421110630e-03
1.9020730257034302e-01 3.2720080018043518e-01
6.3648408651351929e-01
<_>
0 1 112 -8.1061907112598419e-03 -1 -2 113
6.0048708692193031e-03
6.9148528575897217e-01 4.3273261189460754e-01
6.9638431072235107e-01
<_>
0 1 114 -8.7028548121452332e-02 -1 -2 115
-4.7809639945626259e-03
8.5941338539123535e-01 9.7394466400146484e-02
4.5870301127433777e-01
<_>
0 1 116 -2.2166660055518150e-03 -1 -2 117
1.3642730191349983e-03
2.5546258687973022e-01 3.3190909028053284e-01
5.9641027450561523e-01
<_>
0 1 118 -9.0077864006161690e-03 -1 -2 119
-1.5494120307266712e-02
2.6665949821472168e-01 1.8481859564781189e-01
6.2459707260131836e-01
<_>
1 0 120 -4.2165028862655163e-03 -1 -2 121
4.3249759823083878e-02
5.3799271583557129e-01 5.1830291748046875e-01
2.1704199910163879e-01
<_>
1 0 122 2.8786511393263936e-04 -1 -2 123
1.2373150093480945e-03
2.6133841276168823e-01 2.7865320444107056e-01
5.9089881181716919e-01
<_>
1 0 124 1.9528300035744905e-03 -1 -2 125
-1.4947060262784362e-03
2.6128691434860229e-01 5.9154129028320312e-01
3.4557819366455078e-01
<_>
1 0 126 3.5878680646419525e-03 -1 -2 127
-2.5938691105693579e-03
1.5870520472526550e-01 1.2704110145568848e-01
5.9794288873672485e-01
<_>
27
1.2578499794006348e+01
<_>
0 1 128 3.5810680128633976e-03 -1 -2 129
-2.8552350122481585e-03
1.9951049983501434e-01 7.3730701208114624e-01
2.9217371344566345e-01
<_>
0 1 130 1.9758539274334908e-03 -1 -2 131
3.2583118882030249e-03
1.9564199447631836e-01 5.6920468807220459e-01
1.8390649557113647e-01
<_>
0 1 132 2.3711679386906326e-04 -1 -2 133
2.5942500215023756e-03
2.1716670691967010e-01 2.7199891209602356e-01
7.1502441167831421e-01
<_>
0 1 134 -2.5032449513673782e-02 -1 -2 135
6.3087949529290199e-03
1.8251839280128479e-01 5.6998378038406372e-01
3.5098528861999512e-01
<_>
1 0 136 -3.2494920305907726e-03 -1 -2 137
-1.4885730110108852e-02
4.0239268541336060e-01 3.6040958762168884e-01
7.2919952869415283e-01
<_>
1 0 138 8.0623216927051544e-03 -1 -2 139
2.7405679225921631e-02
6.4914900064468384e-01 5.5189931392669678e-01
2.6596811413764954e-01
<_>
1 0 140 3.4368600696325302e-02 -1 -2 141
-2.7292970567941666e-02
6.7125129699707031e-01 1.6913780570030212e-01
4.3262779712677002e-01
<_>
0 1 142 7.4452121043577790e-04 -1 -2 143
7.0336280623450875e-04
3.4051001071929932e-01 5.5167931318283081e-01
3.3113878965377808e-01
<_>
0 1 144 -1.2275460362434387e-01 -1 -2 145
3.2559928949922323e-03
1.6753150522708893e-01 3.6157518625259399e-01
6.4207828044891357e-01
<_>
0 1 146 -3.2090399414300919e-02 -1 -2 147
3.2957999501377344e-03
2.9210790991783142e-01 5.6130319833755493e-01
3.3578601479530334e-01
<_>
0 1 148 -3.2273170072585344e-03 -1 -2 149
1.1171669466421008e-03
6.9706428050994873e-01 3.5411500930786133e-01
6.1440062522888184e-01
<_>
1 0 150 -1.7279950901865959e-02 -1 -2 151
1.1741200461983681e-02
5.5371809005737305e-01 5.3419572114944458e-01
2.7571049332618713e-01
<_>
1 0 152 4.6405228786170483e-03 -1 -2 153
-1.6913030296564102e-02
2.4895210564136505e-01 1.7119289934635162e-01
5.5239528417587280e-01
<_>
1 0 154 1.0060169734060764e-02 -1 -2 155
-6.0715491417795420e-04
8.2734507322311401e-01 3.7793910503387451e-01
5.4762518405914307e-01
<_>
1 0 156 -1.0865400545299053e-03 -1 -2 157
8.9362077414989471e-03
3.2965409755706787e-01 6.0628837347030640e-01
2.4342200160026550e-01
<_>
1 0 158 -2.6372660067863762e-04 -1 -2 159
1.3110050000250340e-02
3.8140949606895447e-01 5.5176162719726562e-01
3.7268930673599243e-01
<_>
0 1 160 -2.9806280508637428e-03 -1 -2 161
-4.1619571857154369e-03
1.2296640127897263e-01 7.2522747516632080e-01
4.9734550714492798e-01
<_>
0 1 162 3.3842328935861588e-02 -1 -2 163
-1.2564560165628791e-03
5.3483128547668457e-01 5.8519148826599121e-01
4.3841668963432312e-01
<_>
0 1 164 -1.9635230302810669e-02 -1 -2 165
-9.9625496659427881e-04
2.2978340089321136e-01 6.2959378957748413e-01
4.1315990686416626e-01
<_>
0 1 166 -2.3127110674977303e-02 -1 -2 167
2.3525709286332130e-02
1.6954590380191803e-01 5.1741302013397217e-01
5.9519391506910324e-02
<_>
0 1 168 -1.9356520846486092e-02 -1 -2 169
-4.1787112131714821e-03
1.3572479784488678e-01 2.9966288805007935e-01
5.7916951179504395e-01
<_>
1 0 170 3.1488779932260513e-03 -1 -2 171
7.3972279205918312e-03
6.5925890207290649e-01 5.3071719408035278e-01
3.7951210141181946e-01
<_>
0 1 172 7.1955118983169086e-06 -1 -2 173
4.7114409506320953e-02
3.1283149123191833e-01 5.5378931760787964e-01
1.0273090004920959e-01
<_>
0 1 174 7.2878710925579071e-03 -1 -2 175
-6.1887511983513832e-03
4.6608591079711914e-01 7.1588581800460815e-01
4.7244489192962646e-01
<_>
1 0 176 2.9757320880889893e-03 -1 -2 177
-1.8449809867888689e-03
5.9345688670873642e-02 7.0273017883300781e-01
4.7187310457229614e-01
<_>
0 1 178 1.0239540279144421e-04 -1 -2 179
2.4277009069919586e-03
5.8947342634201050e-01 4.8623558878898621e-01
5.2475881576538086e-01
<_>
0 1 180 -6.4751312136650085e-02 -1 -2 181
3.9380151429213583e-04
6.9174712896347046e-01 4.6696171164512634e-01
2.3824059963226318e-01
<_>
31
1.4546750068664551e+01
<_>
0 1 182 1.4397440245375037e-03 -1 -2 183
-5.4068560712039471e-04
2.7734708786010742e-01 7.4271547794342041e-01
2.4797350168228149e-01
<_>
1 0 184 -7.1237959673453588e-06 -1 -2 185
-2.3661039303988218e-03
2.1995030343532562e-01 5.8899897336959839e-01
2.5957161188125610e-01
<_>
0 1 186 1.7343269428238273e-03 -1 -2 187
1.5874590026214719e-03
1.8601259589195251e-01 4.1518709063529968e-01
7.1034741401672363e-01
<_>
1 0 188 3.7285638973116875e-03 -1 -2 189
-1.2883819639682770e-01
2.5279670953750610e-01 1.3930009305477142e-01
5.2545148134231567e-01
<_>
1 0 190 7.9412180930376053e-03 -1 -2 191
-1.2661729939281940e-02
2.4877290427684784e-01 2.7107000350952148e-01
6.6188377141952515e-01
<_>
0 1 192 3.0146789868013002e-05 -1 -2 193
-1.6330160200595856e-02
3.8128259778022766e-01 2.3264320194721222e-01
5.2630108594894409e-01
<_>
0 1 194 1.4622770322603174e-05 -1 -2 195
-2.0858660340309143e-02
4.2933320999145508e-01 1.6004039347171783e-01
6.7823147773742676e-01
<_>
1 0 196 2.8194559272378683e-03 -1 -2 197
3.7899368908256292e-03
6.6792941093444824e-01 4.5877051353454590e-01
7.1762388944625854e-01
<_>
1 0 198 3.5344641655683517e-02 -1 -2 199
-1.1571600334718823e-03
1.8640750646591187e-01 5.5382597446441650e-01
3.1504508852958679e-01
<_>
0 1 200 -5.8742752298712730e-03 -1 -2 201
-1.5201780115603469e-05
2.8287911415100098e-01 5.8702242374420166e-01
3.7048238515853882e-01
<_>
1 0 202 -2.2681879636365920e-04 -1 -2 203
3.7845689803361893e-03
4.2189309000968933e-01 6.6670012474060059e-01
2.4611820280551910e-01
<_>
1 0 204 -8.5295992903411388e-05 -1 -2 205
-4.4394891709089279e-02
3.5575878620147705e-01 1.6655470430850983e-01
5.2348488569259644e-01
<_>
0 1 206 1.0126030538231134e-03 -1 -2 207
-7.6327780261635780e-03
2.8846129775047302e-01 2.9693400859832764e-01
6.0801112651824951e-01
<_>
0 1 208 4.0330411866307259e-03 -1 -2 209
1.3676689565181732e-01
4.5363900065422058e-01 5.1772642135620117e-01
1.4491820335388184e-01
<_>
0 1 210 -5.0060478970408440e-03 -1 -2 211
-1.2475839816033840e-02
7.6169097423553467e-01 2.1597060561180115e-01
5.4601877927780151e-01
<_>
1 0 212 -9.4012258341535926e-04 -1 -2 213
-1.2191980145871639e-02
3.9262959361076355e-01 3.4788811206817627e-01
5.5426627397537231e-01
<_>
0 1 214 -5.4959481349214911e-04 -1 -2 215
-2.1802430273965001e-04
6.0642760992050171e-01 5.6974071264266968e-01
1.7797139286994934e-01
<_>
0 1 216 6.9115799851715565e-03 -1 -2 217
-9.7631698008626699e-04
5.3793722391128540e-01 3.3278390765190125e-01
5.4615312814712524e-01
<_>
0 1 218 -8.7870173156261444e-03 -1 -2 219
-1.6761029837653041e-03
2.1161609888076782e-01 6.6358232498168945e-01
4.3658590316772461e-01
<_>
1 0 220 -5.5694948881864548e-02 -1 -2 221
-1.9844379276037216e-02
5.3874248266220093e-01 1.6028049588203430e-01
5.3304588794708252e-01
<_>
0 1 222 -7.4751611100509763e-04 -1 -2 223
2.3032890632748604e-02
2.9174768924713135e-01 5.6081241369247437e-01
1.9979810714721680e-01
<_>
1 0 224 -3.0700280331075191e-03 -1 -2 225
-1.1636839481070638e-03
3.9383140206336975e-01 5.7574361562728882e-01
4.2394569516181946e-01
<_>
1 0 226 2.2464339435100555e-01 -1 -2 227
1.4412109740078449e-03
7.6765531301498413e-01 5.3538662195205688e-01
2.5147768855094910e-01
<_>
0 1 228 -3.0011249706149101e-02 -1 -2 229
-5.3078960627317429e-02
2.3649039864540100e-01 2.3858639597892761e-01
5.4146647453308105e-01
<_>
1 0 230 2.0800929050892591e-03 -1 -2 231
-4.0738182142376900e-03
6.5116149187088013e-01 6.0304141044616699e-01
3.5877010226249695e-01
<_>
1 0 232 -1.9529370591044426e-02 -1 -2 233
-5.3309470415115356e-02
5.4235929250717163e-01 2.3609539866447449e-01
5.4017579555511475e-01
<_>
0 1 234 -3.4849561750888824e-02 -1 -2 235
-1.2658450007438660e-01
2.8369858860969543e-01 1.8135160207748413e-01
5.4210460186004639e-01
<_>
0 1 236 7.3325118137290701e-06 -1 -2 237
-1.1843870393931866e-02
3.9803659915924072e-01 2.6163849234580994e-01
5.2377301454544067e-01
<_>
0 1 238 -4.8470678739249706e-03 -1 -2 239
8.1693977117538452e-03
2.4381080269813538e-01 5.3271460533142090e-01
8.1903767585754395e-01
<_>
1 0 240 -6.4716790802776814e-03 -1 -2 241
-1.5188479665084742e-05
4.6796938776969910e-01 5.5639117956161499e-01
4.3675860762596130e-01
<_>
1 0 242 3.0696711037307978e-03 -1 -2 243
-1.6296720423270017e-04
6.6643488407135010e-01 5.5946111679077148e-01
3.0427119135856628e-01
<_>
39
1.8572250366210938e+01
<_>
1 0 244 -9.8275858908891678e-03 -1 -2 245
-4.1693858802318573e-03
2.1160189807415009e-01 6.9246852397918701e-01
3.0437770485877991e-01
<_>
0 1 246 3.5341319744475186e-04 -1 -2 247
4.8054549843072891e-03
3.1832858920097351e-01 5.4565590620040894e-01
2.5222688913345337e-01
<_>
0 1 248 2.1071180526632816e-04 -1 -2 249
-2.8318869881331921e-03
2.9026180505752563e-01 3.1304559111595154e-01
6.8849372863769531e-01
<_>
1 0 250 -7.5633679443853907e-06 -1 -2 251
-8.2888139877468348e-04
2.9624658823013306e-01 3.0996260046958923e-01
5.7525151968002319e-01
<_>
0 1 252 1.6209259629249573e-03 -1 -2 253
9.1338958591222763e-03
3.9931958913803101e-01 4.8273721337318420e-01
7.5378328561782837e-01
<_>
0 1 254 -4.1212290525436401e-03 -1 -2 255
-2.5447290390729904e-03
2.6169270277023315e-01 3.1087028980255127e-01
5.4912358522415161e-01
<_>
0 1 256 -6.2652782071381807e-04 -1 -2 257
-3.6596331483451650e-05
3.2396918535232544e-01 6.5174108743667603e-01
4.1789120435714722e-01
<_>
1 0 258 1.3882719911634922e-02 -1 -2 259
1.0493700392544270e-03
6.7712038755416870e-01 4.1595110297203064e-01
5.6528919935226440e-01
<_>
1 0 260 1.8215360119938850e-02 -1 -2 261
-1.1334580369293690e-02
7.6896011829376221e-01 2.8733238577842712e-01
4.9889329075813293e-01
<_>
1 0 262 -4.1097560897469521e-03 -1 -2 263
4.2612891411408782e-04
5.4630082845687866e-01 3.6312350630760193e-01
5.5125522613525391e-01
<_>
1 0 264 6.0301548801362514e-03 -1 -2 265
3.3587709185667336e-04
1.1437670141458511e-01 2.8910788893699646e-01
5.4473417997360229e-01
<_>
1 0 266 6.2279507983475924e-04 -1 -2 267
-2.5837119668722153e-02
3.0234318971633911e-01 2.1670059859752655e-01
5.2781528234481812e-01
<_>
1 0 268 2.1774910390377045e-02 -1 -2 269
1.7682299949228764e-03
3.2548341155052185e-01 5.2630507946014404e-01
7.5263291597366333e-01
<_>
0 1 270 -1.3793810270726681e-02 -1 -2 271
-5.0852829590439796e-03
7.4103301763534546e-01 6.8366098403930664e-01
4.5790711045265198e-01
<_>
1 0 272 6.1795017682015896e-03 -1 -2 273
1.0030319914221764e-02
7.4499362707138062e-01 4.8607799410820007e-01
2.3614570498466492e-01
<_>
0 1 274 -6.4201927743852139e-03 -1 -2 275
-5.6961281225085258e-03
1.4673270285129547e-01 2.3478199541568756e-01
5.3233772516250610e-01
<_>
0 1 276 -7.1498160250484943e-03 -1 -2 277
2.4450740311294794e-03
1.4770570397377014e-01 3.4985339641571045e-01
5.8035618066787720e-01
<_>
1 0 278 -3.7503410130739212e-02 -1 -2 279
4.7799441381357610e-04
5.2595508098602295e-01 4.3628829717636108e-01
6.2089228630065918e-01
<_>
0 1 280 -7.0806080475449562e-03 -1 -2 281
3.2818000763654709e-02
2.0394609868526459e-01 5.1983588933944702e-01
1.3711960613727570e-01
<_>
1 0 282 6.5188988810405135e-04 -1 -2 283
4.6485587954521179e-03
6.3234299421310425e-01 4.7201630473136902e-01
6.5670871734619141e-01
<_>
0 1 284 -1.9827929791063070e-03 -1 -2 285
-1.6011310508474708e-03
6.0530602931976318e-01 5.0905191898345947e-01
3.1169331073760986e-01
<_>
0 1 286 -3.0539939180016518e-03 -1 -2 287
4.3212040327489376e-04
3.4298041462898254e-01 3.8384029269218445e-01
5.7755982875823975e-01
<_>
0 1 288 -2.7452120557427406e-02 -1 -2 289
9.3099439982324839e-04
2.1434690058231354e-01 5.9529662132263184e-01
3.7601581215858459e-01
<_>
0 1 290 6.7144189961254597e-03 -1 -2 291
-3.3701690845191479e-03
5.6926268339157104e-01 5.7843041419982910e-01
3.9742821455001831e-01
<_>
0 1 292 -1.8903959542512894e-02 -1 -2 293
-6.5850871615111828e-03
1.8188929557800293e-01 6.8491101264953613e-01
4.3515840172767639e-01
<_>
1 0 294 5.8810501359403133e-03 -1 -2 295
8.0092082498595119e-04
2.7266609668731689e-01 4.2364311218261719e-01
5.8446758985519409e-01
<_>
1 0 296 1.8510579830035567e-03 -1 -2 297
6.3273650594055653e-03
3.3713209629058838e-01 5.2702218294143677e-01
8.0536508560180664e-01
<_>
0 1 298 -3.3820930402725935e-03 -1 -2 299
-1.9292969955131412e-03
2.8660181164741516e-01 5.8889460563659668e-01
3.8957870006561279e-01
<_>
1 0 300 1.4995220117270947e-02 -1 -2 301
-2.6330750435590744e-02
2.1778169274330139e-01 1.7753170430660248e-01
5.6714701652526855e-01
<_>
1 0 302 -4.1734222322702408e-03 -1 -2 303
2.7268350124359131e-02
4.6529620885848999e-01 4.7683110833168030e-01
5.6952387094497681e-01
<_>
1 0 304 9.8880263976752758e-04 -1 -2 305
-1.0528849670663476e-03
3.3974018692970276e-01 6.2500411272048950e-01
4.2884120345115662e-01
<_>
0 1 306 5.2288072183728218e-03 -1 -2 307
3.0395459383726120e-02
5.3477621078491211e-01 4.1155189275741577e-01
5.6607538461685181e-01
<_>
0 1 308 -7.9113930463790894e-02 -1 -2 309
1.8231669440865517e-02
7.8813230991363525e-01 3.6043399572372437e-01
5.5695050954818726e-01
<_>
0 1 310 5.2288072183728218e-03 -1 -2 311
4.3922828626818955e-04
5.4166442155838013e-01 5.5071568489074707e-01
3.8822770118713379e-01
<_>
0 1 312 -8.6501962505280972e-04 -1 -2 313
1.0326979681849480e-03
3.1858509778976440e-01 5.5783641338348389e-01
3.2192459702491760e-01
<_>
0 1 314 -7.2997747920453548e-03 -1 -2 315
-9.3629042385146022e-04
7.0732331275939941e-01 5.5580157041549683e-01
4.6138420701026917e-01
<_>
0 1 316 -6.0483231209218502e-03 -1 -2 317
6.7529221996665001e-03
6.8692898750305176e-01 4.8703178763389587e-01
2.6503708958625793e-01
<_>
0 1 318 5.3078029304742813e-02 -1 -2 319
-1.0225810110569000e-03
5.2815151214599609e-01 6.0858821868896484e-01
4.3048679828643799e-01
<_>
1 0 320 3.1270649284124374e-02 -1 -2 321
-6.3522169366478920e-03
5.4458320140838623e-01 5.3283357620239258e-01
2.3643240332603455e-01
<_>
45
2.1578119277954102e+01
<_>
1 0 322 -6.2215630896389484e-03 -1 -2 323
2.1097389981150627e-03
2.6255810260772705e-01 1.5649929642677307e-01
6.7928832769393921e-01
<_>
0 1 324 1.0845859535038471e-02 -1 -2 325
6.4230401767417789e-04
3.4858089685440063e-01 3.6982551217079163e-01
5.9216582775115967e-01
<_>
1 0 326 7.3311722371727228e-04 -1 -2 327
1.0134200565516949e-03
3.0070841312408447e-01 3.6249229311943054e-01
7.0724260807037354e-01
<_>
0 1 328 1.1093559674918652e-02 -1 -2 329
-7.9127531498670578e-03
4.4167020916938782e-01 3.0287081003189087e-01
5.4173761606216431e-01
<_>
0 1 330 1.2905309908092022e-02 -1 -2 331
-4.2430912144482136e-03
4.3745040893554688e-01 4.4015899300575256e-01
7.5651907920837402e-01
<_>
0 1 332 -2.1304309484548867e-04 -1 -2 333
-2.2308640182018280e-03
2.3107869923114777e-01 3.5681959986686707e-01
5.7499992847442627e-01
<_>
0 1 334 2.6400520000606775e-03 -1 -2 335
7.5101032853126526e-02
3.5936889052391052e-01 6.3635677099227905e-01
2.3270289599895477e-01
<_>
0 1 336 -7.7012968249619007e-03 -1 -2 337
1.5588370151817799e-03
7.0746237039566040e-01 5.7002371549606323e-01
3.5904508829116821e-01
<_>
0 1 338 -4.7687938786111772e-04 -1 -2 339
8.4234727546572685e-04
2.8054410219192505e-01 4.1254189610481262e-01
6.1779958009719849e-01
<_>
1 0 340 -1.2825109995901585e-02 -1 -2 341
-6.5156567143276334e-04
5.4030781984329224e-01 5.6336438655853271e-01
3.3565390110015869e-01
<_>
0 1 342 -1.2006159871816635e-02 -1 -2 343
1.3213419588282704e-03
7.1095108985900879e-01 4.9038508534431458e-01
2.8245830535888672e-01
<_>
0 1 344 -2.0307440310716629e-02 -1 -2 345
4.0180929936468601e-03
1.8913699686527252e-01 5.3779661655426025e-01
3.1194949150085449e-01
<_>
1 0 346 4.5315311290323734e-03 -1 -2 347
-4.4381739571690559e-03
7.2067582607269287e-01 1.8546679615974426e-01
4.9817329645156860e-01
<_>
1 0 348 1.5692010056227446e-03 -1 -2 349
-4.9516442231833935e-03
2.6382741332054138e-01 6.8710672855377197e-01
4.7146868705749512e-01
<_>
0 1 350 -2.7429679408669472e-02 -1 -2 351
1.4181969454512000e-03
1.5482850372791290e-01 4.3768429756164551e-01
6.3273680210113525e-01
<_>
0 1 352 -1.3078940100967884e-02 -1 -2 353
-3.5092779435217381e-03
3.1668141484260559e-01 6.1997437477111816e-01
4.3796870112419128e-01
<_>
1 0 354 1.8920730799436569e-02 -1 -2 355
2.1683350205421448e-03
1.4707140624523163e-01 5.8094590902328491e-01
3.4319490194320679e-01
<_>
0 1 356 1.6401590546593070e-03 -1 -2 357
1.4005920093040913e-04
3.9594578742980957e-01 3.2400250434875488e-01
5.6466472148895264e-01
<_>
1 0 358 -3.3137591090053320e-03 -1 -2 359
-2.9459029901772738e-03
4.2745280265808105e-01 3.3416679501533508e-01
6.6279602050781250e-01
<_>
0 1 360 1.3612229668069631e-04 -1 -2 361
6.0512032359838486e-04
4.0469279885292053e-01 5.4840582609176636e-01
3.5699409246444702e-01
<_>
0 1 362 -1.7513990402221680e-02 -1 -2 363
-1.8735030665993690e-02
1.8241509795188904e-01 7.9718202352523804e-01
5.0685691833496094e-01
<_>
1 0 364 1.2065649963915348e-02 -1 -2 365
-2.6544178836047649e-03
2.1670070290565491e-01 6.5841788053512573e-01
4.6282431483268738e-01
<_>
1 0 366 1.4501289697363973e-03 -1 -2 367
1.0954019613564014e-02
2.0902520418167114e-01 5.1123052835464478e-01
7.7845758199691772e-01
<_>
0 1 368 1.5771709382534027e-02 -1 -2 369
-1.4252689667046070e-02
5.1323592662811279e-01 1.7424149811267853e-01
5.2671480178833008e-01
<_>
0 1 370 3.0411860279855318e-05 -1 -2 371
2.3486299440264702e-02
3.4184479713439941e-01 5.6312650442123413e-01
2.0063939690589905e-01
<_>
1 0 372 5.2205449901521206e-03 -1 -2 373
-2.5812430307269096e-02
6.2496489286422729e-01 3.2032281160354614e-01
5.1993298530578613e-01
<_>
0 1 374 -1.9526650430634618e-03 -1 -2 375
-8.1470049917697906e-03
6.1407059431076050e-01 6.5928959846496582e-01
3.7111249566078186e-01
<_>
1 0 376 3.2962448894977570e-03 -1 -2 377
-1.3961310032755136e-03
2.9521119594573975e-01 3.3208039402961731e-01
5.5284148454666138e-01
<_>
0 1 378 -4.1055441834032536e-03 -1 -2 379
-1.0888779535889626e-02
1.7105500400066376e-01 3.3594349026679993e-01
5.6749051809310913e-01
<_>
1 0 380 -7.6768421567976475e-03 -1 -2 381
-9.7729787230491638e-03
4.7732418775558472e-01 8.0810451507568359e-01
4.8458281159400940e-01
<_>
1 0 382 6.0439710505306721e-03 -1 -2 383
-4.6134641161188483e-04
6.7840021848678589e-01 5.5146390199661255e-01
3.6423599720001221e-01
<_>
1 0 384 5.7992361485958099e-02 -1 -2 385
5.9384980704635382e-04
1.2544350326061249e-01 4.4248789548873901e-01
5.7284617424011230e-01
<_>
0 1 386 -6.2353480607271194e-03 -1 -2 387
-1.2784929946064949e-02
2.8050419688224792e-01 1.9509120285511017e-01
5.6529247760772705e-01
<_>
1 0 388 4.1973669431172311e-04 -1 -2 389
8.0646801507100463e-04
6.1664837598800659e-01 4.5265799760818481e-01
5.9444868564605713e-01
<_>
1 0 390 -1.6339010326191783e-03 -1 -2 391
-4.8299999907612801e-03
4.0869420766830444e-01 2.7935269474983215e-01
6.4449352025985718e-01
<_>
1 0 392 -6.3992068171501160e-03 -1 -2 393
1.0819199681282043e-01
5.6716561317443848e-01 5.3118121623992920e-01
2.6143568754196167e-01
<_>
1 0 394 6.5056560561060905e-04 -1 -2 395
2.0611250773072243e-02
2.9967740178108215e-01 4.4899430871009827e-01
6.8882799148559570e-01
<_>
1 0 396 -2.5129050016403198e-02 -1 -2 397
1.7922939732670784e-03
5.1968640089035034e-01 3.4669959545135498e-01
5.5335879325866699e-01
<_>
1 0 398 1.5626220265403390e-03 -1 -2 399
-6.1898730928078294e-04
3.0814400315284729e-01 2.6938709616661072e-01
5.5444890260696411e-01
<_>
0 1 400 4.8111421056091785e-03 -1 -2 401
2.2484229411929846e-03
5.5878478288650513e-01 4.6721130609512329e-01
6.0908252000808716e-01
<_>
0 1 402 -3.0147239565849304e-02 -1 -2 403
2.7548679709434509e-01
9.0275919437408447e-01 4.7198349237442017e-01
2.1969200670719147e-01
<_>
1 0 404 3.6894630175083876e-03 -1 -2 405
7.2957701049745083e-03
6.2730091810226440e-01 4.8392179608345032e-01
6.9090622663497925e-01
<_>
0 1 406 -5.6211069226264954e-02 -1 -2 407
-2.6478560175746679e-03
1.7384879291057587e-01 6.3041448593139648e-01
4.4743019342422485e-01
<_>
1 0 408 -1.4534000074490905e-03 -1 -2 409
2.8540920466184616e-03
5.3025382757186890e-01 5.3383970260620117e-01
3.7968829274177551e-01
<_>
1 0 410 5.8243022067472339e-04 -1 -2 411
9.2509482055902481e-04
3.2698369026184082e-01 4.5548120141029358e-01
6.3583481311798096e-01
<_>
47
2.2585290908813477e+01
<_>
0 1 412 1.9806440919637680e-02 -1 -2 413
7.0395611692219973e-04
2.8097251057624817e-01 3.1198260188102722e-01
7.0903062820434570e-01
<_>
0 1 414 2.5563780218362808e-03 -1 -2 415
1.0824160417541862e-03
2.9819479584693909e-01 3.0205601453781128e-01
5.8088111877441406e-01
<_>
1 0 416 -9.2893769033253193e-04 -1 -2 417
-1.8009729683399200e-02
3.7381029129028320e-01 2.1631260216236115e-01
6.6192537546157837e-01
<_>
1 0 418 2.3500190582126379e-03 -1 -2 419
8.1822491483762860e-04
2.9104039072990417e-01 5.5786228179931641e-01
3.3666279911994934e-01
<_>
0 1 420 6.2095321482047439e-04 -1 -2 421
9.6780969761312008e-04
4.0724259614944458e-01 6.8595957756042480e-01
3.1054618954658508e-01
<_>
1 0 422 4.8000211245380342e-04 -1 -2 423
9.0538640506565571e-05
3.3373329043388367e-01 3.3709588646888733e-01
5.4512107372283936e-01
<_>
0 1 424 -4.3914798647165298e-02 -1 -2 425
-5.6501338258385658e-03
2.6256701350212097e-01 6.0504627227783203e-01
3.2324150204658508e-01
<_>
1 0 426 3.8661491125822067e-03 -1 -2 427
-6.3069426687434316e-05
3.2626131176948547e-01 5.8173078298568726e-01
4.1643899679183960e-01
<_>
1 0 428 5.2533738315105438e-02 -1 -2 429
1.3818660518154502e-03
7.0953989028930664e-01 5.2928757667541504e-01
2.5413888692855835e-01
<_>
1 0 430 -8.9264067355543375e-04 -1 -2 431
8.5579507052898407e-02
4.0853410959243774e-01 5.2632361650466919e-01
3.0032029747962952e-01
<_>
1 0 432 -1.8343339615967125e-04 -1 -2 433
-9.7924815490841866e-03
4.0292051434516907e-01 3.5213199257850647e-01
6.6640049219131470e-01
<_>
0 1 434 1.4428620226681232e-02 -1 -2 435
-4.5687001198530197e-02
4.5935660600662231e-01 1.4747560024261475e-01
5.1786321401596069e-01
<_>
0 1 436 -2.5763090234249830e-03 -1 -2 437
-3.8301859050989151e-02
1.8372780084609985e-01 8.0826580524444580e-01
5.1666879653930664e-01
<_>
0 1 438 2.8978290501981974e-03 -1 -2 439
-2.5165060069411993e-03
4.7980138659477234e-01 3.3462959527969360e-01
5.4444491863250732e-01
<_>
0 1 440 5.6281982688233256e-04 -1 -2 441
3.6684391088783741e-03
3.5890269279479980e-01 5.9831297397613525e-01
2.9839640855789185e-01
<_>
1 0 442 2.1319789811968803e-03 -1 -2 443
7.6037310063838959e-03
6.1632239818572998e-01 5.2171301841735840e-01
2.0541590452194214e-01
<_>
1 0 444 -1.1668079969240353e-04 -1 -2 445
3.1659509986639023e-03
3.4466689825057983e-01 5.5974847078323364e-01
2.6737868785858154e-01
<_>
0 1 446 -2.2569499909877777e-02 -1 -2 447
2.7129601221531630e-04
6.9002681970596313e-01 4.4866389036178589e-01
5.5087852478027344e-01
<_>
0 1 448 -1.5434459783136845e-02 -1 -2 449
-8.4861656650900841e-03
2.0483230054378510e-01 1.2549529969692230e-01
5.0603562593460083e-01
<_>
0 1 450 -1.1807470023632050e-01 -1 -2 451
-1.2300079688429832e-03
6.7633062601089478e-02 5.6607007980346680e-01
4.2922011017799377e-01
<_>
0 1 452 -7.0290351286530495e-03 -1 -2 453
8.9325206354260445e-03
7.1364039182662964e-01 4.3388760089874268e-01
7.0608752965927124e-01
<_>
1 0 454 -4.7735981643199921e-02 -1 -2 455
-4.4155579060316086e-02
5.2686852216720581e-01 2.5805801153182983e-01
5.4069608449935913e-01
<_>
0 1 456 -2.5983480736613274e-02 -1 -2 457
-4.7885831445455551e-03
1.9050540030002594e-01 2.5518929958343506e-01
5.3390771150588989e-01
<_>
0 1 458 6.7423451691865921e-03 -1 -2 459
1.1654750443994999e-02
4.6933099627494812e-01 5.2619642019271851e-01
3.1454348564147949e-01
<_>
0 1 460 -5.6982729583978653e-03 -1 -2 461
-7.2983349673449993e-03
1.7568530142307281e-01 7.7747297286987305e-01
5.1242929697036743e-01
<_>
0 1 462 7.9091778025031090e-03 -1 -2 463
-1.5874979726504534e-04
5.2845597267150879e-01 3.8878020644187927e-01
5.5011737346649170e-01
<_>
0 1 464 -6.2235877849161625e-03 -1 -2 465
1.3308860361576080e-03
2.4898290634155273e-01 4.2621460556983948e-01
5.9350621700286865e-01
<_>
1 0 466 5.2055278792977333e-03 -1 -2 467
1.4065169729292393e-02
2.5452229380607605e-01 4.8519900441169739e-01
7.0214188098907471e-01
<_>
0 1 468 -6.7384149879217148e-03 -1 -2 469
3.3406780567020178e-03
7.1432709693908691e-01 5.1757252216339111e-01
2.8086438775062561e-01
<_>
1 0 470 -1.1880699545145035e-02 -1 -2 471
1.4226379571482539e-03
5.1732218265533447e-01 4.5028659701347351e-01
5.7956951856613159e-01
<_>
1 0 472 2.9858129564672709e-03 -1 -2 473
-2.0481580868363380e-03
1.9151160120964050e-01 6.5024322271347046e-01
4.5593151450157166e-01
<_>
0 1 474 1.7122729914262891e-03 -1 -2 475
-1.6980869695544243e-02
5.3762471675872803e-01 7.0562332868576050e-01
4.9146059155464172e-01
<_>
0 1 476 -1.1290470138192177e-03 -1 -2 477
2.8620059601962566e-03
2.6787060499191284e-01 4.4108539819717407e-01
6.3683199882507324e-01
<_>
0 1 478 -3.8065758999437094e-03 -1 -2 479
5.9090270660817623e-03
2.7635639905929565e-01 4.8673018813133240e-01
6.7287760972976685e-01
<_>
0 1 480 1.1004370171576738e-03 -1 -2 481
-2.3396299220621586e-03
4.0705141425132751e-01 2.6049488782882690e-01
6.1548602581024170e-01
<_>
0 1 482 -3.6068160552531481e-03 -1 -2 483
4.0831189602613449e-02
5.7319998741149902e-01 4.9733769893646240e-01
7.3870068788528442e-01
<_>
0 1 484 -7.1082250215113163e-03 -1 -2 485
-9.3759730225428939e-04
6.9847512245178223e-01 2.6911678910255432e-01
4.7417798638343811e-01
<_>
0 1 486 -1.6740820137783885e-03 -1 -2 487
8.8287703692913055e-02
3.5510140657424927e-01 5.2446138858795166e-01
2.0966500043869019e-01
<_>
0 1 488 8.2009629113599658e-04 -1 -2 489
-7.6624617213383317e-04
4.1310968995094299e-01 4.6202930808067322e-01
6.7754101753234863e-01
<_>
1 0 490 6.5769668435677886e-04 -1 -2 491
-2.1304790861904621e-03
5.6282752752304077e-01 5.5768597126007080e-01
4.5776501297950745e-01
<_>
1 0 492 -3.7317050737328827e-04 -1 -2 493
-1.1172230355441570e-02
4.9592560529708862e-01 5.6256359815597534e-01
2.0471079647541046e-01
<_>
1 0 494 4.3435219675302505e-02 -1 -2 495
9.6736161503940821e-04
2.2421480715274811e-01 4.5333439111709595e-01
6.1999320983886719e-01
<_>
0 1 496 -3.1452889088541269e-03 -1 -2 497
1.5233129961416125e-03
6.6627562046051025e-01 5.0079882144927979e-01
2.3849929869174957e-01
<_>
1 0 498 2.0854279864579439e-03 -1 -2 499
3.6098200827836990e-02
3.7535008788108826e-01 5.1771712303161621e-01
1.6344930231571198e-01
<_>
1 0 500 1.6179570229724050e-03 -1 -2 501
-6.2132300809025764e-04
2.5873818993568420e-01 6.2995338439941406e-01
4.6587899327278137e-01
<_>
1 0 502 7.1878539165481925e-04 -1 -2 503
-3.9339520037174225e-02
3.3540761470794678e-01 2.1541289985179901e-01
5.2357137203216553e-01
<_>
0 1 504 -1.0988829890266061e-03 -1 -2 505
2.1191420964896679e-03
6.4688968658447266e-01 2.8930890560150146e-01
5.2548158168792725e-01
<_>
53
2.5609300613403320e+01
<_>
0 1 506 5.2359891124069691e-03 -1 -2 507
-2.2169889416545630e-03
3.2997110486030579e-01 7.0415931940078735e-01
3.2354658842086792e-01
<_>
1 0 508 -8.2303592935204506e-03 -1 -2 509
-8.2303592935204506e-03
4.9611708521842957e-01 7.1280431747436523e-01
4.9611708521842957e-01
<_>
0 1 510 4.5343261444941163e-04 -1 -2 511
-4.1777061414904892e-04
3.2084721326828003e-01 6.6139167547225952e-01
3.5513329505920410e-01
<_>
0 1 512 2.7823769487440586e-03 -1 -2 513
-6.0361868236213923e-05
3.7101349234580994e-01 5.7463937997817993e-01
3.8948801159858704e-01
<_>
1 0 514 3.5061789676547050e-03 -1 -2 515
1.7013119941111654e-04
3.0541029572486877e-01 2.8855779767036438e-01
6.4877450466156006e-01
<_>
1 0 516 -2.3378930054605007e-03 -1 -2 517
-2.1369170863181353e-03
3.1744310259819031e-01 3.8209199905395508e-01
5.2328932285308838e-01
<_>
0 1 518 1.0250400518998504e-03 -1 -2 519
-4.4726220949087292e-05
3.6227950453758240e-01 6.5389591455459595e-01
4.0036809444427490e-01
<_>
1 0 520 5.7102291611954570e-04 -1 -2 521
5.7743012439459562e-04
3.8931730389595032e-01 5.6145328283309937e-01
3.6876440048217773e-01
<_>
1 0 522 7.9692091094329953e-04 -1 -2 523
3.5945948911830783e-04
6.4430278539657593e-01 3.3808529376983643e-01
5.8246481418609619e-01
<_>
1 0 524 4.3973900028504431e-04 -1 -2 525
-8.9061429025605321e-04
3.9387670159339905e-01 3.4279710054397583e-01
5.5156987905502319e-01
<_>
1 0 526 5.4110242053866386e-03 -1 -2 527
-8.5764907998964190e-04
3.8035380840301514e-01 6.4395052194595337e-01
4.1683459281921387e-01
<_>
0 1 528 -2.2000649943947792e-02 -1 -2 529
-7.8731682151556015e-03
6.6546010971069336e-01 4.1827228665351868e-01
5.6047242879867554e-01
<_>
0 1 530 -2.7444459497928619e-02 -1 -2 531
1.9792269449681044e-03
6.5868628025054932e-01 3.2449120283126831e-01
4.8828700184822083e-01
<_>
0 1 532 -5.6783691979944706e-03 -1 -2 533
1.5057219570735469e-05
2.2290790081024170e-01 4.1072851419448853e-01
5.7475912570953369e-01
<_>
0 1 534 -5.4136710241436958e-03 -1 -2 535
5.3679239936172962e-03
2.0657970011234283e-01 4.9264231324195862e-01
7.1394848823547363e-01
<_>
0 1 536 -3.1426660716533661e-03 -1 -2 537
1.0907390154898167e-02
6.7800867557525635e-01 5.2149301767349243e-01
1.1439959704875946e-01
<_>
1 0 538 5.8436761610209942e-03 -1 -2 539
9.0507230197545141e-05
1.9375260174274445e-01 3.8125771284103394e-01
5.5141878128051758e-01
<_>
0 1 540 -1.6345789656043053e-02 -1 -2 541
1.5987500082701445e-03
2.4740239977836609e-01 4.8177829384803772e-01
5.9230798482894897e-01
<_>
0 1 542 -4.0257978253066540e-03 -1 -2 543
-6.7750471644103527e-03
7.5082087516784668e-01 2.8798109292984009e-01
5.1996952295303345e-01
<_>
0 1 544 -3.2470689620822668e-03 -1 -2 545
1.5409620245918632e-03
3.0449101328849792e-01 4.0634828805923462e-01
5.6765627861022949e-01
<_>
0 1 546 -1.2858119793236256e-02 -1 -2 547
-1.4824670506641269e-04
9.6717558801174164e-02 4.5378330349922180e-01
6.1153751611709595e-01
<_>
1 0 548 -9.0210810303688049e-03 -1 -2 549
-2.8795029968023300e-02
4.8077508807182312e-01 3.4037950634956360e-01
5.2555292844772339e-01
<_>
1 0 550 9.0210810303688049e-03 -1 -2 551
7.4121179059147835e-03
7.5058358907699585e-01 5.4554468393325806e-01
3.2260689139366150e-01
<_>
0 1 552 -3.7217529024928808e-03 -1 -2 553
1.9865889847278595e-01
2.3118489980697632e-01 5.2710479497909546e-01
1.4699299633502960e-01
<_>
0 1 554 1.5208719560177997e-05 -1 -2 555
-3.9089918136596680e-03
3.6781388521194458e-01 7.1319299936294556e-01
4.9938669800758362e-01
<_>
0 1 556 2.5106288958340883e-03 -1 -2 557
2.3921660613268614e-04
5.3120541572570801e-01 4.6893781423568726e-01
5.7140219211578369e-01
<_>
1 0 558 6.9443131797015667e-03 -1 -2 559
1.2065629707649350e-03
6.9487977027893066e-01 4.0045049786567688e-01
5.8748817443847656e-01
<_>
0 1 560 2.5106288958340883e-03 -1 -2 561
1.7514040227979422e-03
5.3295719623565674e-01 5.5458492040634155e-01
3.4495818614959717e-01
<_>
0 1 562 -4.1978210210800171e-03 -1 -2 563
1.3092850567772985e-03
1.2171830236911774e-01 5.3750497102737427e-01
3.4156250953674316e-01
<_>
0 1 564 6.7396182566881180e-04 -1 -2 565
-1.0530710220336914e-02
4.1951790452003479e-01 3.4607538580894470e-01
5.1558601856231689e-01
<_>
0 1 566 -4.0672299265861511e-01 -1 -2 567
-2.6314549148082733e-02
5.8065678924322128e-02 1.4734490215778351e-01
5.5593782663345337e-01
<_>
1 0 568 2.2557149641215801e-03 -1 -2 569
1.2154860422015190e-02
5.4777151346206665e-01 4.2077910900115967e-01
5.6218808889389038e-01
<_>
0 1 570 -1.8436539918184280e-02 -1 -2 571
5.3676147945225239e-04
6.4471471309661865e-01 2.7651271224021912e-01
4.8885959386825562e-01
<_>
1 0 572 -2.6265541091561317e-03 -1 -2 573
-5.1119807176291943e-04
5.2646911144256592e-01 5.7853102684020996e-01
4.2911028861999512e-01
<_>
1 0 574 4.1454841266386211e-04 -1 -2 575
-5.5028748465701938e-04
3.4554108977317810e-01 6.0269188880920410e-01
4.1438931226730347e-01
<_>
0 1 576 -1.0347720235586166e-03 -1 -2 577
-3.3966631162911654e-03
6.0952937602996826e-01 6.1082822084426880e-01
4.7077208757400513e-01
<_>
1 0 578 3.1795909162610769e-03 -1 -2 579
-1.6528950072824955e-04
3.2443669438362122e-01 3.8307571411132812e-01
5.7343262434005737e-01
<_>
1 0 580 8.3725210279226303e-03 -1 -2 581
-2.5799809955060482e-03
6.6109192371368408e-01 6.1393070220947266e-01
4.6861499547958374e-01
<_>
1 0 582 9.0194388758391142e-04 -1 -2 583
3.6952210939489305e-04
3.5200220346450806e-01 2.5787541270256042e-01
5.4672420024871826e-01
<_>
0 1 584 9.9746137857437134e-04 -1 -2 585
-3.6688039544969797e-03
4.8201468586921692e-01 5.7101500034332275e-01
4.8319110274314880e-01
<_>
0 1 586 -8.9501030743122101e-04 -1 -2 587
5.1904921419918537e-03
6.1336791515350342e-01 4.9285829067230225e-01
2.5813090801239014e-01
<_>
0 1 588 4.2274440056644380e-04 -1 -2 589
8.5176713764667511e-03
4.4711241126060486e-01 5.1610249280929565e-01
3.3165338635444641e-01
<_>
0 1 590 -3.6623608320951462e-02 -1 -2 591
-4.1103712283074856e-03
9.2606216669082642e-02 8.5221147537231445e-01
5.1379078626632690e-01
<_>
1 0 592 -6.6017331555485725e-03 -1 -2 593
2.5578640401363373e-02
5.4590600728988647e-01 5.2193528413772583e-01
1.9271859526634216e-01
<_>
1 0 594 1.1447439901530743e-02 -1 -2 595
7.2427501436322927e-04
1.9160020351409912e-01 5.2315711975097656e-01
3.5353401303291321e-01
<_>
1 0 596 9.7127500921487808e-03 -1 -2 597
-1.1337569914758205e-02
6.4641010761260986e-01 7.3830378055572510e-01
4.9647438526153564e-01
<_>
0 1 598 -8.1453882157802582e-03 -1 -2 599
-8.5570756345987320e-03
3.6117058992385864e-01 3.4219071269035339e-01
5.9435117244720459e-01
<_>
0 1 600 2.2993308957666159e-03 -1 -2 601
3.8430930580943823e-03
4.5501041412353516e-01 4.7168621420860291e-01
6.6561907529830933e-01
<_>
1 0 602 -9.9116540513932705e-04 -1 -2 603
2.5496469810605049e-02
4.5927169919013977e-01 6.5634012222290039e-01
1.2588350474834442e-01
<_>
1 0 604 -1.5748359262943268e-02 -1 -2 605
-1.8046120181679726e-02
5.2395021915435791e-01 8.0158519744873047e-01
5.0079578161239624e-01
<_>
1 0 606 1.0323390364646912e-02 -1 -2 607
1.6452240524813533e-03
2.2748200595378876e-01 4.3519461154937744e-01
5.8676278591156006e-01
<_>
0 1 608 1.5881149098277092e-02 -1 -2 609
1.0586519725620747e-02
4.4650518894195557e-01 4.5444580912590027e-01
5.7071107625961304e-01
<_>
0 1 610 -2.1531689912080765e-02 -1 -2 611
5.2480469457805157e-03
6.5276437997817993e-01 3.4447279572486877e-01
5.3246361017227173e-01
<_>
67
3.2647129058837891e+01
<_>
0 1 612 1.8219340126961470e-03 -1 -2 613
8.1313941627740860e-03
3.1087881326675415e-01 3.1332370638847351e-01
6.6458672285079956e-01
<_>
0 1 614 1.7055979697033763e-03 -1 -2 615
-7.4483548814896494e-05
2.6401311159133911e-01 5.6472051143646240e-01
3.4853729605674744e-01
<_>
1 0 616 3.8342390325851738e-04 -1 -2 617
3.1868910882622004e-03
3.1406548619270325e-01 6.4891988039016724e-01
3.8877290487289429e-01
<_>
1 0 618 1.6044320166110992e-01 -1 -2 619
-6.7285560071468353e-03
7.2165298461914062e-01 1.6531379520893097e-01
5.1398259401321411e-01
<_>
0 1 620 7.2638481469766703e-06 -1 -2 621
5.5551197146996856e-04
3.1406199932098389e-01 5.9936988353729248e-01
3.3173981308937073e-01
<_>
0 1 622 -1.0822320356965065e-02 -1 -2 623
-4.5834020711481571e-03
2.6529380679130554e-01 1.8495689332485199e-01
5.3139579296112061e-01
<_>
1 0 624 -3.0205070506781340e-03 -1 -2 625
7.7864617109298706e-02
4.0400999784469604e-01 6.1581897735595703e-01
1.7864869534969330e-01
<_>
0 1 626 2.6494380086660385e-02 -1 -2 627
3.6912109702825546e-02
4.5110899209976196e-01 4.5282199978828430e-01
5.9722828865051270e-01
<_>
1 0 628 5.7857790961861610e-03 -1 -2 629
9.3849771656095982e-04
2.5338920950889587e-01 3.4104120731353760e-01
5.9236437082290649e-01
<_>
0 1 630 -1.1003199964761734e-02 -1 -2 631
-1.1737640015780926e-03
6.9580441713333130e-01 3.8510841131210327e-01
5.4081892967224121e-01
<_>
0 1 632 -3.6596669815480709e-03 -1 -2 633
-2.4822750128805637e-03
2.0093089342117310e-01 6.2953931093215942e-01
4.3950408697128296e-01
<_>
0 1 634 -4.4606071896851063e-03 -1 -2 635
-3.5969649907201529e-03
2.4052999913692474e-01 5.4501742124557495e-01
3.7823578715324402e-01
<_>
0 1 636 -3.6222559865564108e-03 -1 -2 637
1.2059339787811041e-03
3.0338969826698303e-01 4.6337789297103882e-01
6.3359522819519043e-01
<_>
1 0 638 4.3124938383698463e-03 -1 -2 639
-4.4961250387132168e-03
6.5988260507583618e-01 6.6216969490051270e-01
4.7552469372749329e-01
<_>
0 1 640 -1.3860689941793680e-03 -1 -2 641
-5.1588460337370634e-04
2.8012010455131531e-01 3.8294890522956848e-01
5.6236267089843750e-01
<_>
0 1 642 7.0330002927221358e-05 -1 -2 643
-2.0976549421902746e-04
4.5363429188728333e-01 5.6081390380859375e-01
4.2657798528671265e-01
<_>
1 0 644 1.3642259873449802e-03 -1 -2 645
1.5483660390600562e-03
2.6370918750762939e-01 4.1707509756088257e-01
5.9329879283905029e-01
<_>
0 1 646 1.9179609417915344e-01 -1 -2 647
-4.4776909053325653e-03
5.2567642927169800e-01 6.6326218843460083e-01
4.8925888538360596e-01
<_>
0 1 648 -1.2649179995059967e-01 -1 -2 649
6.5253327193204314e-05
1.4997789263725281e-01 4.2333200573921204e-01
5.7560402154922485e-01
<_>
0 1 650 4.1856421157717705e-03 -1 -2 651
2.7478230185806751e-04
5.2888268232345581e-01 4.5240178704261780e-01
5.6041252613067627e-01
<_>
0 1 652 -2.2906810045242310e-03 -1 -2 653
1.6744500026106834e-03
5.5782741308212280e-01 3.3230578899383545e-01
5.5587881803512573e-01
<_>
1 0 654 1.2349759927019477e-03 -1 -2 655
-8.7158754467964172e-03
3.6539471149444580e-01 1.9245339930057526e-01
5.3136497735977173e-01
<_>
1 0 656 4.6613621525466442e-03 -1 -2 657
-8.5815992206335068e-03
2.0277309417724609e-01 7.6360601186752319e-01
5.1408261060714722e-01
<_>
0 1 658 1.4352120459079742e-02 -1 -2 659
-7.7948719263076782e-03
5.2529758214950562e-01 2.6329371333122253e-01
5.3286892175674438e-01
<_>
0 1 660 -3.4155680332332850e-03 -1 -2 661
-4.2639090679585934e-03
2.4160879850387573e-01 3.9365449547767639e-01
5.4787421226501465e-01
<_>
0 1 662 8.7177697569131851e-03 -1 -2 663
-3.2232629600912333e-03
4.7881990671157837e-01 3.6316120624542236e-01
5.2883160114288330e-01
<_>
0 1 664 -4.2188368737697601e-02 -1 -2 665
1.9875749945640564e-02
6.9311392307281494e-01 4.5201000571250916e-01
6.8550550937652588e-01
<_>
1 0 666 -3.1134510412812233e-02 -1 -2 667
5.7032387703657150e-03
5.3004240989685059e-01 5.6068921089172363e-01
4.2306229472160339e-01
<_>
1 0 668 5.2733682096004486e-03 -1 -2 669
-3.1231069006025791e-03
3.2472288608551025e-01 1.9856959581375122e-01
5.3498727083206177e-01
<_>
0 1 670 4.6453849063254893e-04 -1 -2 671
3.0355889350175858e-02
4.2075088620185852e-01 5.1534587144851685e-01
3.1181010603904724e-01
<_>
0 1 672 -4.2992769740521908e-03 -1 -2 673
1.9509199773892760e-04
3.2745069265365601e-01 5.9530782699584961e-01
4.2255210876464844e-01
<_>
0 1 674 -7.7784480527043343e-03 -1 -2 675
1.6917599365115166e-02
7.2111797332763672e-01 4.9365919828414917e-01
7.0302772521972656e-01
<_>
0 1 676 -5.1948569715023041e-02 -1 -2 677
-5.4751220159232616e-03
1.4255349338054657e-01 6.0593318939208984e-01
4.3939951062202454e-01
<_>
0 1 678 1.5210839592327829e-05 -1 -2 679
1.0235579684376717e-03
4.4888499379158020e-01 4.2565500736236572e-01
5.7954382896423340e-01
<_>
0 1 680 -1.0427719826111570e-04 -1 -2 681
8.7853781878948212e-03
4.2460399866104126e-01 4.9580091238021851e-01
6.7594307661056519e-01
<_>
0 1 682 3.4012699034065008e-03 -1 -2 683
5.8582378551363945e-04
5.4234808683395386e-01 3.6365428566932678e-01
5.4643487930297852e-01
<_>
0 1 684 -2.2973360028117895e-03 -1 -2 685
-1.4330189675092697e-02
2.5488188862800598e-01 6.5876567363739014e-01
4.5328021049499512e-01
<_>
0 1 686 9.8565965890884399e-04 -1 -2 687
-4.6640761196613312e-02
3.8227710127830505e-01 3.0773219466209412e-01
5.2441328763961792e-01
<_>
0 1 688 -1.1907300353050232e-01 -1 -2 689
1.9333280622959137e-02
1.0338629782199860e-01 5.5547451972961426e-01
3.2213169336318970e-01
<_>
0 1 690 3.1427849084138870e-02 -1 -2 691
2.0082130504306406e-04
4.6823790669441223e-01 5.3730702400207520e-01
3.8006669282913208e-01
<_>
0 1 692 -6.2584900297224522e-03 -1 -2 693
8.2861045375466347e-03
1.7992070317268372e-01 5.0950688123703003e-01
7.5446051359176636e-01
<_>
0 1 694 2.0529709290713072e-03 -1 -2 695
3.2524869311600924e-03
5.6286448240280151e-01 4.8016890883445740e-01
5.8021020889282227e-01
<_>
0 1 696 -3.1884901225566864e-02 -1 -2 697
1.8379340181127191e-03
1.7427450418472290e-01 3.4665969014167786e-01
5.1071548461914062e-01
<_>
1 0 698 -4.8512680223211646e-04 -1 -2 699
-2.5407879147678614e-03
5.3260862827301025e-01 6.3427752256393433e-01
4.9926930665969849e-01
<_>
0 1 700 -5.1559060811996460e-03 -1 -2 701
-4.4968750327825546e-02
3.4334290027618408e-01 1.8681369721889496e-01
5.2154648303985596e-01
<_>
1 0 702 5.8984281495213509e-03 -1 -2 703
3.2763120252639055e-03
6.2293052673339844e-01 4.9357721209526062e-01
7.2179448604583740e-01
<_>
1 0 704 -1.0161520185647532e-04 -1 -2 705
-1.6290300118271261e-04
5.0079762935638428e-01 6.0241490602493286e-01
2.3295080661773682e-01
<_>
0 1 706 9.0541364625096321e-03 -1 -2 707
3.5398490726947784e-02
4.5104169845581055e-01 5.1419967412948608e-01
2.8602918982505798e-01
<_>
0 1 708 5.6469351984560490e-03 -1 -2 709
-2.4807190056890249e-03
4.7049251198768616e-01 4.1798511147499084e-01
6.7266470193862915e-01
<_>
0 1 710 -4.1088787838816643e-03 -1 -2 711
-2.0714469719678164e-03
5.8098018169403076e-01 6.0747838020324707e-01
4.5240598917007446e-01
<_>
0 1 712 -2.8939060866832733e-03 -1 -2 713
1.3467279495671391e-03
3.3835199475288391e-01 5.6969100236892700e-01
3.9708450436592102e-01
<_>
0 1 714 -9.0779133141040802e-02 -1 -2 715
-8.3171762526035309e-02
1.5027019381523132e-01 7.5736707448959351e-01
4.9364370107650757e-01
<_>
0 1 716 -1.4107000315561891e-03 -1 -2 717
5.5668760091066360e-02
3.3909329771995544e-01 5.0250971317291260e-01
7.4220830202102661e-01
<_>
0 1 718 5.7701539248228073e-02 -1 -2 719
-4.2503291368484497e-01
5.1973718404769897e-01 9.7346916794776917e-02
5.1857399940490723e-01
<_>
0 1 720 -4.4380719191394746e-04 -1 -2 721
1.7924769781529903e-04
3.6493501067161560e-01 5.6192791461944580e-01
3.7602970004081726e-01
<_>
1 0 722 5.0382469780743122e-03 -1 -2 723
1.5191170386970043e-02
6.3284450769424438e-01 4.9360820651054382e-01
7.4265247583389282e-01
<_>
0 1 724 -1.2300389818847179e-02 -1 -2 725
1.5168030513450503e-03
1.3893499970436096e-01 5.0919622182846069e-01
3.4826481342315674e-01
<_>
1 0 726 9.5754547510296106e-04 -1 -2 727
-1.8962200731039047e-02
6.0363167524337769e-01 2.3191730678081512e-01
5.1166528463363647e-01
<_>
0 1 728 -2.2272260859608650e-02 -1 -2 729
-2.5145230814814568e-02
6.5550220012664795e-01 1.3260710239410400e-01
4.6740341186523438e-01
<_>
0 1 730 1.9533900544047356e-02 -1 -2 731
-1.1231349781155586e-03
5.1820272207260132e-01 6.3182431459426880e-01
4.8255190253257751e-01
<_>
0 1 732 -1.4861139934509993e-03 -1 -2 733
3.5002888762392104e-04
2.9186710715293884e-01 5.6213712692260742e-01
4.2492130398750305e-01
<_>
1 0 734 -1.1231349781155586e-03 -1 -2 735
1.0409739799797535e-02
4.8137450218200684e-01 5.1840060949325562e-01
2.0512230694293976e-01
<_>
0 1 736 -8.7832562625408173e-02 -1 -2 737
1.6584879485890269e-03
1.1799219995737076e-01 4.9878111481666565e-01
6.9737559556961060e-01
<_>
1 0 738 -2.3008750285953283e-03 -1 -2 739
3.3026169985532761e-02
5.3398311138153076e-01 5.0332891941070557e-01
6.8519067764282227e-01
<_>
0 1 740 -1.3585069682449102e-03 -1 -2 741
7.8067491995170712e-04
3.0028221011161804e-01 4.5930838584899902e-01
6.4400452375411987e-01
<_>
1 0 742 -1.8025759607553482e-02 -1 -2 743
1.2354910140857100e-03
5.3112912178039551e-01 4.7291061282157898e-01
5.7214611768722534e-01
<_>
0 1 744 -9.2583027435466647e-04 -1 -2 745
8.0123997759073973e-04
3.6623328924179077e-01 5.3619897365570068e-01
3.0086329579353333e-01
<_>
63
3.0672130584716797e+01
<_>
0 1 746 2.4914839304983616e-03 -1 -2 747
-5.0488598644733429e-02
3.4223890304565430e-01 7.7034580707550049e-01
4.5163908600807190e-01
<_>
1 0 748 -7.7838351717218757e-04 -1 -2 749
2.3572890495415777e-04
3.2563421130180359e-01 3.4065559506416321e-01
5.8970272541046143e-01
<_>
0 1 750 4.5575071126222610e-03 -1 -2 751
8.1241987645626068e-03
4.3065789341926575e-01 7.1495872735977173e-01
4.3456849455833435e-01
<_>
0 1 752 -4.4612158671952784e-04 -1 -2 753
-2.8972938889637589e-04
3.2959741353988647e-01 5.8456200361251831e-01
3.5266879200935364e-01
<_>
0 1 754 7.1604831646254752e-06 -1 -2 755
-3.8497708737850189e-04
4.0819549560546875e-01 4.2031130194664001e-01
6.6341269016265869e-01
<_>
0 1 756 1.9489860278554261e-04 -1 -2 757
-1.7083849757909775e-02
3.9424669742584229e-01 2.2940720617771149e-01
5.2389609813690186e-01
<_>
0 1 758 8.3513697609305382e-04 -1 -2 759
7.5499608647078276e-04
3.0260318517684937e-01 6.0321962833404541e-01
3.4124588966369629e-01
<_>
1 0 760 8.0216713249683380e-03 -1 -2 761
-3.8930509239435196e-02
7.3062407970428467e-01 3.5993251204490662e-01
5.2343809604644775e-01
<_>
1 0 762 -7.0348767621908337e-05 -1 -2 763
-8.5350573062896729e-03
3.4937581419944763e-01 2.7461090683937073e-01
5.6265860795974731e-01
<_>
0 1 764 1.0854450054466724e-02 -1 -2 765
4.5329501153901219e-04
5.2822262048721313e-01 4.5220491290092468e-01
6.0543018579483032e-01
<_>
0 1 766 1.8117150466423482e-04 -1 -2 767
4.6641560038551688e-04
3.3068621158599854e-01 1.4550000429153442e-01
5.3849279880523682e-01
<_>
1 0 768 -8.4854792803525925e-03 -1 -2 769
-1.8934309482574463e-02
4.8141559958457947e-01 3.5637411475181580e-01
5.4051452875137329e-01
<_>
1 0 770 4.9814549274742603e-03 -1 -2 771
3.4286780282855034e-03
6.9577431678771973e-01 5.0508928298950195e-01
2.3169949650764465e-01
<_>
1 0 772 4.4203791185282171e-04 -1 -2 773
2.3822550429031253e-04
6.0185819864273071e-01 4.7550821304321289e-01
5.5852377414703369e-01
<_>
0 1 774 -6.4261639490723610e-03 -1 -2 775
9.9637769162654877e-03
2.2824659943580627e-01 4.0405881404876709e-01
5.6501698493957520e-01
<_>
0 1 776 1.3654050417244434e-02 -1 -2 777
-9.9892877042293549e-03
5.2677392959594727e-01 6.7940497398376465e-01
4.7970339655876160e-01
<_>
1 0 778 3.6558631807565689e-02 -1 -2 779
4.8999379941960797e-05
8.8425733149051666e-02 4.0207880735397339e-01
5.4573321342468262e-01
<_>
0 1 780 1.3654050417244434e-02 -1 -2 781
1.8802779959514737e-03
5.2676129341125488e-01 4.8060521483421326e-01
6.3943648338317871e-01
<_>
0 1 782 -1.3654050417244434e-02 -1 -2 783
1.2778700329363346e-03
1.7248100042343140e-01 4.4798240065574646e-01
6.3100087642669678e-01
<_>
1 0 784 9.8843395244330168e-04 -1 -2 785
1.4511500012304168e-05
5.9481692314147949e-01 4.8541748523712158e-01
5.3093612194061279e-01
<_>
0 1 786 -2.2775429533794522e-04 -1 -2 787
-1.4753740280866623e-02
3.1836318969726562e-01 3.0849760770797729e-01
5.3520262241363525e-01
<_>
0 1 788 -3.4148250706493855e-03 -1 -2 789
7.5806681998074055e-03
6.1153268814086914e-01 4.9516460299491882e-01
7.0613312721252441e-01
<_>
1 0 790 -5.7734688743948936e-03 -1 -2 791
7.4033669079653919e-05
3.7542209029197693e-01 4.1155171394348145e-01
5.8894449472427368e-01
<_>
0 1 792 -8.2278084009885788e-03 -1 -2 793
5.3380909375846386e-03
9.5610566437244415e-02 5.3005087375640869e-01
3.9618980884552002e-01
<_>
0 1 794 -2.7049109339714050e-03 -1 -2 795
7.7341338619589806e-03
6.4818692207336426e-01 5.1104402542114258e-01
3.1215190887451172e-01
<_>
0 1 796 1.0886609554290771e-02 -1 -2 797
1.1038660071790218e-02
4.8014289140701294e-01 5.4297101497650146e-01
4.1623631119728088e-01
<_>
0 1 798 -1.0054199956357479e-02 -1 -2 799
7.7072880230844021e-03
7.3293352127075195e-01 5.3568720817565918e-01
3.4555470943450928e-01
<_>
0 1 800 -5.8278098003938794e-04 -1 -2 801
-2.5739220436662436e-03
3.6550220847129822e-01 3.7767601013183594e-01
5.3917747735977173e-01
<_>
0 1 802 -7.0167761296033859e-03 -1 -2 803
-1.7727289814502001e-03
4.0393048524856567e-01 6.9504439830780029e-01
4.9811169505119324e-01
<_>
1 0 804 -1.6318289563059807e-02 -1 -2 805
-1.1663000099360943e-02
5.2967327833175659e-01 5.8426398038864136e-01
4.7895029187202454e-01
<_>
1 0 806 2.5881489273160696e-03 -1 -2 807
-3.7328999023884535e-03
6.0921788215637207e-01 6.7217427492141724e-01
4.0668940544128418e-01
<_>
0 1 808 -1.4355930034071207e-03 -1 -2 809
1.8340899841859937e-03
3.5850879549980164e-01 5.3711581230163574e-01
4.0335071086883545e-01
<_>
1 0 810 1.2280289828777313e-01 -1 -2 811
5.0228700041770935e-02
1.5475720167160034e-01 5.4338437318801880e-01
8.4292672574520111e-02
<_>
1 0 812 -2.1437000483274460e-02 -1 -2 813
-3.1009620055556297e-02
4.8600539565086365e-01 1.8330100178718567e-01
5.2075541019439697e-01
<_>
0 1 814 -1.2973720207810402e-02 -1 -2 815
1.5818020328879356e-03
7.0482409000396729e-01 4.1705870628356934e-01
5.8651638031005859e-01
<_>
1 0 816 -9.7806248813867569e-03 -1 -2 817
1.1735740117728710e-03
5.3079181909561157e-01 5.5224531888961792e-01
3.5071650147438049e-01
<_>
1 0 818 1.4651629608124495e-03 -1 -2 819
2.3532148916274309e-03
3.0426511168479919e-01 5.3393232822418213e-01
2.8062361478805542e-01
<_>
0 1 820 -6.1809681355953217e-03 -1 -2 821
6.5688649192452431e-04
6.4101332426071167e-01 5.6208711862564087e-01
4.3903189897537231e-01
<_>
1 0 822 2.6228010654449463e-02 -1 -2 823
-1.7958110198378563e-02
6.4455568790435791e-01 2.0027139782905579e-01
4.6246650815010071e-01
<_>
1 0 824 -7.6468721963465214e-03 -1 -2 825
-2.7482809964567423e-03
5.2632009983062744e-01 5.8739811182022095e-01
4.8366001248359680e-01
<_>
1 0 826 1.3851850293576717e-02 -1 -2 827
2.6369190309196711e-03
1.5661309659481049e-01 4.2701789736747742e-01
5.8066600561141968e-01
<_>
0 1 828 -3.1513599678874016e-03 -1 -2 829
-1.4788460248382762e-05
6.2158662080764771e-01 5.5766427516937256e-01
4.1220021247863770e-01
<_>
0 1 830 -7.3676988482475281e-02 -1 -2 831
-3.0912780202925205e-03
1.5367099642753601e-01 6.3442689180374146e-01
4.5074120163917542e-01
<_>
0 1 832 7.9240966588258743e-03 -1 -2 833
8.5778040811419487e-03
5.4579752683639526e-01 5.4016572237014771e-01
3.8907998800277710e-01
<_>
1 0 834 5.5403169244527817e-03 -1 -2 835
-1.1886510037584230e-04
3.5556110739707947e-01 5.8367502689361572e-01
4.2743161320686340e-01
<_>
0 1 836 -1.8408369272947311e-02 -1 -2 837
-2.3490579333156347e-03
5.8604401350021362e-01 4.4989579916000366e-01
5.4981988668441772e-01
<_>
1 0 838 -7.6157399453222752e-03 -1 -2 839
-3.3190969843417406e-03
4.1009929776191711e-01 6.7013788223266602e-01
4.3530011177062988e-01
<_>
1 0 840 -9.4642979092895985e-04 -1 -2 841
8.7858550250530243e-03
5.3911769390106201e-01 5.5040502548217773e-01
3.9909350872039795e-01
<_>
1 0 842 1.6395459533669055e-04 -1 -2 843
-2.3508940357714891e-03
3.5929331183433533e-01 4.0341728925704956e-01
5.8060771226882935e-01
<_>
1 0 844 7.5449963333085179e-05 -1 -2 845
2.7018489316105843e-02
5.4123848676681519e-01 4.9449229240417480e-01
5.5894362926483154e-01
<_>
1 0 846 8.4561208495870233e-04 -1 -2 847
-1.1687109945341945e-03
5.8092182874679565e-01 4.7469571232795715e-01
2.8458958864212036e-01
<_>
1 0 848 2.2897500544786453e-02 -1 -2 849
7.0879262685775757e-01
2.4144110083580017e-01 5.1957648992538452e-01
1.0300920158624649e-01
<_>
1 0 850 3.7483830004930496e-02 -1 -2 851
1.2827500468119979e-03
1.8146389722824097e-01 4.2460718750953674e-01
5.7079732418060303e-01
<_>
0 1 852 -5.1718312315642834e-03 -1 -2 853
2.7545939665287733e-03
6.1433231830596924e-01 5.2056711912155151e-01
4.2204418778419495e-01
<_>
0 1 854 -3.6072919610887766e-03 -1 -2 855
-2.5258748792111874e-04
3.1825920939445496e-01 5.7104682922363281e-01
4.2260938882827759e-01
<_>
1 0 856 -7.0514748804271221e-03 -1 -2 857
-5.4323761723935604e-03
5.1628297567367554e-01 2.6662889122962952e-01
5.2146798372268677e-01
<_>
1 0 858 -1.4652940080850385e-05 -1 -2 859
-1.8556920113041997e-03
3.9817610383033752e-01 3.3227631449699402e-01
5.7058340311050415e-01
<_>
1 0 860 4.7609540633857250e-03 -1 -2 861
1.5676260227337480e-03
6.6365581750869751e-01 5.5055677890777588e-01
4.4206619262695312e-01
<_>
1 0 862 5.4239919409155846e-03 -1 -2 863
-6.4692399464547634e-03
5.9599381685256958e-01 5.3695940971374512e-01
3.7443399429321289e-01
<_>
0 1 864 -7.8038539504632354e-04 -1 -2 865
4.5086450874805450e-02
4.1035950183868408e-01 5.1775068044662476e-01
1.8781000375747681e-01
<_>
0 1 866 -5.1405387930572033e-03 -1 -2 867
-2.1236129105091095e-02
2.3528920114040375e-01 1.7087510228157043e-01
5.4249739646911621e-01
<_>
0 1 868 -2.3763340432196856e-03 -1 -2 869
5.4122589528560638e-02
5.8365309238433838e-01 5.1174330711364746e-01
1.8659310042858124e-01
<_>
0 1 870 -5.3492980077862740e-04 -1 -2 871
-5.8454048121348023e-04
5.1086932420730591e-01 4.7754910588264465e-01
2.4398539960384369e-01
<_>
71
3.4677078247070312e+01
<_>
0 1 872 3.0031939968466759e-03 -1 -2 873
6.9161207647994161e-04
3.3496499061584473e-01 4.5183679461479187e-01
7.2893542051315308e-01
<_>
0 1 874 1.1212790384888649e-02 -1 -2 875
-7.6108198845759034e-04
2.9508009552955627e-01 5.6690549850463867e-01
2.8308510780334473e-01
<_>
0 1 876 1.1984579759882763e-04 -1 -2 877
-1.9725349557120353e-04
4.0905779600143433e-01 6.9514942169189453e-01
4.6378681063652039e-01
<_>
1 0 878 -5.5180420167744160e-03 -1 -2 879
1.2148249661549926e-03
3.1676751375198364e-01 3.3167061209678650e-01
5.3963977098464966e-01
<_>
0 1 880 -4.2497441172599792e-03 -1 -2 881
-9.4915721565485001e-03
2.6005738973617554e-01 7.4842947721481323e-01
5.0731921195983887e-01
<_>
1 0 882 6.5378600265830755e-04 -1 -2 883
-4.9741100519895554e-04
3.9520108699798584e-01 5.8802747726440430e-01
3.5521200299263000e-01
<_>
0 1 884 -4.3079249560832977e-02 -1 -2 885
-5.1999092102050781e-04
2.4348780512809753e-01 3.1955629587173462e-01
5.5854547023773193e-01
<_>
1 0 886 -4.5451628975570202e-03 -1 -2 887
-7.9610403627157211e-03
4.8452898859977722e-01 3.8011810183525085e-01
5.3585118055343628e-01
<_>
1 0 888 -3.1919340835884213e-04 -1 -2 889
-1.9223889335989952e-02
4.3563291430473328e-01 2.6130661368370056e-01
6.1554962396621704e-01
<_>
0 1 890 -1.3076990144327283e-03 -1 -2 891
1.9825039431452751e-02
5.9420621395111084e-01 4.9454280734062195e-01
7.3848551511764526e-01
<_>
0 1 892 -2.2013280540704727e-03 -1 -2 893
-7.8596705570816994e-03
2.2144819796085358e-01 3.6009770631790161e-01
5.2985501289367676e-01
<_>
1 0 894 1.4142199652269483e-03 -1 -2 895
-1.1232759803533554e-02
5.7765662670135498e-01 6.9344568252563477e-01
4.8272070288658142e-01
<_>
1 0 896 2.9746301006525755e-03 -1 -2 897
5.3283828310668468e-04
3.2166770100593567e-01 3.9625000953674316e-01
5.6803637742996216e-01
<_>
1 0 898 1.0105259716510773e-02 -1 -2 899
-1.1653699912130833e-02
7.5674182176589966e-01 6.5235567092895508e-01
5.0270539522171021e-01
<_>
0 1 900 -7.0609981194138527e-03 -1 -2 901
2.2343141026794910e-03
2.5387701392173767e-01 4.3872770667076111e-01
6.1776322126388550e-01
<_>
1 0 902 -2.9802279546856880e-02 -1 -2 903
1.1611840454861522e-03
5.2011400461196899e-01 4.6479099988937378e-01
6.1842548847198486e-01
<_>
1 0 904 9.4824447296559811e-04 -1 -2 905
4.1284630424343050e-04
3.0409941077232361e-01 4.5188081264495850e-01
6.2457829713821411e-01
<_>
0 1 906 -3.1203540042042732e-02 -1 -2 907
2.7652881108224392e-03
2.7889358997344971e-01 4.6985000371932983e-01
6.5024542808532715e-01
<_>
1 0 908 2.5644779205322266e-02 -1 -2 909
-7.5331530533730984e-03
1.8051710724830627e-01 3.2080689072608948e-01
5.5220228433609009e-01
<_>
1 0 910 3.2047149725258350e-03 -1 -2 911
-2.4282479716930538e-04
6.4369338750839233e-01 5.6767052412033081e-01
4.5091038942337036e-01
<_>
0 1 912 -6.1979342717677355e-04 -1 -2 913
-8.0101029016077518e-04
3.1221461296081543e-01 2.9651939868927002e-01
5.2304947376251221e-01
<_>
1 0 914 -9.1816839994862676e-04 -1 -2 915
1.2239529751241207e-03
5.4647117853164673e-01 4.6185028553009033e-01
5.6795489788055420e-01
<_>
0 1 916 -6.8743730662390590e-04 -1 -2 917
-1.8252469599246979e-03
5.4308801889419556e-01 5.4336231946945190e-01
3.3852210640907288e-01
<_>
1 0 918 -7.4570789001882076e-03 -1 -2 919
5.3775748237967491e-03
5.2655947208404541e-01 4.8572158813476562e-01
6.8151241540908813e-01
<_>
1 0 920 3.7602309603244066e-03 -1 -2 921
8.7752222316339612e-04
2.8321608901023865e-01 3.9668309688568115e-01
5.5124807357788086e-01
<_>
1 0 922 5.5084479972720146e-03 -1 -2 923
-7.5949047459289432e-04
6.7846202850341797e-01 3.9065030217170715e-01
5.4572027921676636e-01
<_>
1 0 924 1.6352660022675991e-03 -1 -2 925
-1.2750849418807775e-04
3.6402040719985962e-01 5.8297240734100342e-01
4.1949799656867981e-01
<_>
0 1 926 2.2067610174417496e-02 -1 -2 927
-1.9203789532184601e-02
4.6067029237747192e-01 3.2614830136299133e-01
5.2360808849334717e-01
<_>
0 1 928 -1.2998109683394432e-02 -1 -2 929
-3.1332690268754959e-03
7.0221120119094849e-01 2.8704708814620972e-01
5.0764769315719604e-01
<_>
1 0 930 -5.2937557920813560e-03 -1 -2 931
2.1857069805264473e-03
4.7095209360122681e-01 4.7082918882369995e-01
6.1698418855667114e-01
<_>
0 1 932 -4.5750709250569344e-03 -1 -2 933
-4.5152138918638229e-02
3.1142529845237732e-01 1.8514350056648254e-01
5.5048149824142456e-01
<_>
1 0 934 -2.7783559635281563e-03 -1 -2 935
-2.5752480141818523e-03
4.9373480677604675e-01 6.1529481410980225e-01
4.7354999184608459e-01
<_>
1 0 936 1.1614130344241858e-03 -1 -2 937
2.3350189439952374e-03
6.5105718374252319e-01 4.0883418917655945e-01
5.6841522455215454e-01
<_>
1 0 938 3.8499289657920599e-03 -1 -2 939
2.4529630318284035e-03
3.0258288979530334e-01 5.2325028181076050e-01
2.0176209509372711e-01
<_>
1 0 940 3.6731390282511711e-03 -1 -2 941
2.1937100682407618e-03
6.4284259080886841e-01 4.3288651108741760e-01
6.4205098152160645e-01
<_>
1 0 942 -6.4666871912777424e-03 -1 -2 943
-5.7186251506209373e-03
5.2540659904479980e-01 2.4909840524196625e-01
5.2876192331314087e-01
<_>
1 0 944 9.9941878579556942e-04 -1 -2 945
-7.8276498243212700e-04
3.3297958970069885e-01 3.5983449220657349e-01
5.4983407258987427e-01
<_>
0 1 946 4.3231188319623470e-03 -1 -2 947
4.0838290005922318e-03
4.8187050223350525e-01 5.2663302421569824e-01
3.1057891249656677e-01
<_>
1 0 948 3.0515898833982646e-04 -1 -2 949
1.2640280183404684e-03
3.9952918887138367e-01 3.2284379005432129e-01
5.8192151784896851e-01
<_>
0 1 950 -1.0152660310268402e-02 -1 -2 951
-2.6863690000027418e-03
8.0260711908340454e-01 3.8756170868873596e-01
5.4665708541870117e-01
<_>
1 0 952 -9.0515613555908203e-03 -1 -2 953
-6.3204211182892323e-03
4.3720579147338867e-01 1.1265510320663452e-01
6.3954162597656250e-01
<_>
0 1 954 2.6117300149053335e-03 -1 -2 955
1.4339019544422626e-02
5.4239892959594727e-01 4.9792730808258057e-01
6.0422360897064209e-01
<_>
1 0 956 2.8452780097723007e-03 -1 -2 957
1.4783289771003183e-05
3.4910920262336731e-01 4.1950678825378418e-01
5.7759660482406616e-01
<_>
0 1 958 8.1814555451273918e-03 -1 -2 959
6.6321990452706814e-03
4.8859870433807373e-01 5.4444682598114014e-01
4.4209951162338257e-01
<_>
0 1 960 -2.2483461070805788e-03 -1 -2 961
1.2374560348689556e-02
6.6997921466827393e-01 4.4786059856414795e-01
6.5648937225341797e-01
<_>
1 0 962 -6.6516688093543053e-03 -1 -2 963
-8.5750613361597061e-03
5.5118787288665771e-01 4.0174451470375061e-01
5.4055362939834595e-01
<_>
1 0 964 6.5078441984951496e-03 -1 -2 965
2.8675209730863571e-02
2.2943930327892303e-01 5.1779001951217651e-01
3.5677561163902283e-01
<_>
0 1 966 7.0673860609531403e-03 -1 -2 967
1.2367829913273454e-03
5.5646997690200806e-01 3.6276981234550476e-01
5.5724138021469116e-01
<_>
1 0 968 7.4818679131567478e-03 -1 -2 969
4.7109839506447315e-03
6.7849111557006836e-01 4.1212528944015503e-01
6.0722357034683228e-01
<_>
1 0 970 -6.9405790418386459e-03 -1 -2 971
3.3302098512649536e-02
5.4597669839859009e-01 5.2767068147659302e-01
2.3749159276485443e-01
<_>
1 0 972 3.6104630678892136e-02 -1 -2 973
1.9674649462103844e-02
7.2492793202400208e-02 4.6263459324836731e-01
8.2089632749557495e-01
<_>
0 1 974 3.4766150638461113e-03 -1 -2 975
1.3987369602546096e-03
5.2087318897247314e-01 5.4844141006469727e-01
4.2300349473953247e-01
<_>
1 0 976 4.0974249131977558e-03 -1 -2 977
2.6973790954798460e-03
2.7805531024932861e-01 5.4038310050964355e-01
3.7909889221191406e-01
<_>
1 0 978 -5.6591699831187725e-03 -1 -2 979
3.9460969856008887e-04
4.7983360290527344e-01 3.7669500708580017e-01
5.4292291402816772e-01
<_>
1 0 980 2.1750570740550756e-03 -1 -2 981
1.4614439569413662e-03
6.2071627378463745e-01 3.3579450845718384e-01
5.1426321268081665e-01
<_>
1 0 982 -5.3006567759439349e-04 -1 -2 983
1.4869309961795807e-01
5.3446400165557861e-01 5.1596081256866455e-01
2.5618231296539307e-01
<_>
1 0 984 -5.8816498494707048e-05 -1 -2 985
-1.6275369562208652e-03
5.1230919361114502e-01 6.0176461935043335e-01
3.1093719601631165e-01
<_>
0 1 986 -1.2881809845566750e-02 -1 -2 987
9.4982917653396726e-04
2.7122870087623596e-01 5.4424422979354858e-01
4.0288880467414856e-01
<_>
1 0 988 -1.2315999716520309e-02 -1 -2 989
9.0286601334810257e-03
4.7360658645629883e-01 7.4514347314834595e-01
3.4879919886589050e-01
<_>
0 1 990 -8.6876116693019867e-02 -1 -2 991
-1.5107560102478601e-05
2.2903330624103546e-01 5.5178898572921753e-01
4.3931490182876587e-01
<_>
0 1 992 -1.7457660287618637e-02 -1 -2 993
-2.5219470262527466e-03
9.0167902410030365e-02 6.2335401773452759e-01
4.7894591093063354e-01
<_>
0 1 994 1.0656520025804639e-03 -1 -2 995
-4.2540300637483597e-03
5.4896962642669678e-01 5.5798089504241943e-01
4.3758779764175415e-01
<_>
0 1 996 -9.0349102392792702e-03 -1 -2 997
-1.5230999561026692e-03
3.5791561007499695e-01 5.6136602163314819e-01
3.9390438795089722e-01
<_>
1 0 998 2.8441150207072496e-03 -1 -2 999
-3.2824429217725992e-03
3.9015549421310425e-01 4.5286190509796143e-01
5.4413431882858276e-01
<_>
1 0 1000 3.2161718991119415e-05 -1 -2 1001
3.0118400900391862e-05
5.8031117916107178e-01 3.3368501067161560e-01
5.5048561096191406e-01
<_>
0 1 1002 -5.6150099262595177e-03 -1 -2 1003
-1.7389209941029549e-02
6.1247891187667847e-01 8.7271630764007568e-02
5.2045881748199463e-01
<_>
0 1 1004 -4.4361080654198304e-05 -1 -2 1005
1.0354899859521538e-04
3.9353290200233459e-01 5.9188538789749146e-01
4.1196140646934509e-01
<_>
0 1 1006 1.5939630102366209e-03 -1 -2 1007
2.5440789759159088e-03
4.8396238684654236e-01 4.7873649001121521e-01
6.3606631755828857e-01
<_>
0 1 1008 1.5083180187502876e-05 -1 -2 1009
-9.9282202427275479e-05
4.2311170697212219e-01 4.2745891213417053e-01
6.0940480232238770e-01
<_>
1 0 1010 5.5371708003804088e-04 -1 -2 1011
1.9186759600415826e-03
4.2719879746437073e-01 4.4971078634262085e-01
5.5491220951080322e-01
<_>
1 0 1012 -5.0764222396537662e-04 -1 -2 1013
1.7236480489373207e-03
5.4771959781646729e-01 2.8829228878021240e-01
5.6151270866394043e-01
<_>
75
3.6726501464843750e+01
<_>
0 1 1014 1.3092169538140297e-02 -1 -2 1015
4.1446479735895991e-04
3.3388701081275940e-01 3.0993521213531494e-01
6.6774922609329224e-01
<_>
0 1 1016 2.1835729479789734e-02 -1 -2 1017
4.8323940485715866e-02
4.3690490722656250e-01 4.3017241358757019e-01
6.1538851261138916e-01
<_>
0 1 1018 1.6091950237751007e-03 -1 -2 1019
1.3469760306179523e-03
3.3873260021209717e-01 6.2487137317657471e-01
3.5941308736801147e-01
<_>
0 1 1020 1.7729059618432075e-04 -1 -2 1021
3.6743620876222849e-04
3.8684248924255371e-01 4.4093450903892517e-01
5.4764741659164429e-01
<_>
0 1 1022 -1.2352119665592909e-03 -1 -2 1023
1.1705530341714621e-03
3.2601711153984070e-01 4.1113489866256714e-01
6.0881638526916504e-01
<_>
1 0 1024 -2.9695429475395940e-05 -1 -2 1025
2.7050738572143018e-04
4.2694228887557983e-01 4.3064668774604797e-01
5.8105140924453735e-01
<_>
1 0 1026 -7.9626210208516568e-05 -1 -2 1027
3.3152441028505564e-04
3.6691430211067200e-01 4.6106639504432678e-01
6.2905901670455933e-01
<_>
1 0 1028 -5.2305828779935837e-02 -1 -2 1029
2.6880469173192978e-02
5.3286898136138916e-01 5.2132612466812134e-01
3.2312199473381042e-01
<_>
1 0 1030 -2.4203000066336244e-04 -1 -2 1031
-1.6424639616161585e-03
3.5685700178146362e-01 3.4406611323356628e-01
5.6256049871444702e-01
<_>
1 0 1032 -2.6830288697965443e-04 -1 -2 1033
-2.2649629972875118e-03
4.5611730217933655e-01 5.3213518857955933e-01
3.6741548776626587e-01
<_>
1 0 1034 1.5627209097146988e-02 -1 -2 1035
1.6211320459842682e-01
2.0293539762496948e-01 5.5630332231521606e-01
2.6188498735427856e-01
<_>
0 1 1036 -3.7391691002994776e-03 -1 -2 1037
-2.0878419745713472e-03
6.0621947050094604e-01 5.9507638216018677e-01
4.5451170206069946e-01
<_>
1 0 1038 2.3334210272878408e-03 -1 -2 1039
6.5116386394947767e-05
6.4355242252349854e-01 3.5207340121269226e-01
5.1797789335250854e-01
<_>
0 1 1040 7.4625718407332897e-03 -1 -2 1041
-2.2032689303159714e-02
5.3266882896423340e-01 3.4919810295104980e-01
5.4292368888854980e-01
<_>
0 1 1042 -8.3081610500812531e-03 -1 -2 1043
-4.3259368976578116e-04
2.0840230584144592e-01 3.9652720093727112e-01
5.4254537820816040e-01
<_>
1 0 1044 -3.2209228724241257e-02 -1 -2 1045
-9.0424838708713651e-04
5.3064119815826416e-01 5.4503858089447021e-01
4.2566969990730286e-01
<_>
1 0 1046 2.2727500181645155e-03 -1 -2 1047
5.9820008464157581e-03
5.9686112403869629e-01 4.7581401467323303e-01
3.1509441137313843e-01
<_>
1 0 1048 -5.8856618124991655e-04 -1 -2 1049
-8.8227191008627415e-04
4.8477488756179810e-01 5.4263162612915039e-01
4.3383410573005676e-01
<_>
1 0 1050 -7.4473457061685622e-05 -1 -2 1051
3.9148979703895748e-04
4.2875099182128906e-01 6.3451850414276123e-01
4.1018518805503845e-01
<_>
1 0 1052 -3.6939629353582859e-03 -1 -2 1053
-1.1207849718630314e-02
4.8491048812866211e-01 4.1463369131088257e-01
5.4712641239166260e-01
<_>
0 1 1054 -1.0337409563362598e-02 -1 -2 1055
3.6883640568703413e-03
2.8771838545799255e-01 5.1019018888473511e-01
7.2169512510299683e-01
<_>
1 0 1056 -3.8984280545264482e-03 -1 -2 1057
-5.9986729174852371e-03
5.2761822938919067e-01 6.6184598207473755e-01
4.8416310548782349e-01
<_>
1 0 1058 4.5043681748211384e-03 -1 -2 1059
1.7799530178308487e-02
1.8741579353809357e-01 4.6169349551200867e-01
7.0889657735824585e-01
<_>
0 1 1060 -1.8462570384144783e-02 -1 -2 1061
1.4931300029275008e-05
3.0019798874855042e-01 4.5618081092834473e-01
5.6107878684997559e-01
<_>
0 1 1062 -8.6021229624748230e-02 -1 -2 1063
-6.0818758356617764e-05
2.3417009413242340e-01 5.6722861528396606e-01
4.1999641060829163e-01
<_>
1 0 1064 1.2670679716393352e-03 -1 -2 1065
1.3699879636988044e-03
6.2074822187423706e-01 5.3949588537216187e-01
3.8238629698753357e-01
<_>
1 0 1066 3.3162781037390232e-03 -1 -2 1067
-1.4532039640471339e-03
7.0616811513900757e-01 3.0655130743980408e-01
4.8273730278015137e-01
<_>
1 0 1068 -7.1492061018943787e-02 -1 -2 1069
1.9857978913933039e-03
5.1931220293045044e-01 4.6424350142478943e-01
5.8076947927474976e-01
<_>
1 0 1070 6.2516499310731888e-03 -1 -2 1071
2.7005500160157681e-03
2.9498139023780823e-01 4.5858868956565857e-01
6.0223537683486938e-01
<_>
0 1 1072 1.1130389757454395e-02 -1 -2 1073
1.5092849731445312e-02
4.3578410148620605e-01 4.5615398883819580e-01
6.1190617084503174e-01
<_>
0 1 1074 -2.7943300083279610e-02 -1 -2 1075
4.4036991312168539e-05
6.5371441841125488e-01 3.4747231006622314e-01
5.3369677066802979e-01
<_>
0 1 1076 -1.2232770211994648e-02 -1 -2 1077
-6.8591412855312228e-04
3.7316760420799255e-01 5.7172292470932007e-01
4.7933790087699890e-01
<_>
0 1 1078 -3.8992990739643574e-03 -1 -2 1079
4.9113907152786851e-04
4.0564361214637756e-01 6.1740481853485107e-01
4.4717541337013245e-01
<_>
1 0 1080 8.2117747515439987e-03 -1 -2 1081
-4.5564480125904083e-02
6.1796981096267700e-01 2.2854949533939362e-01
5.2495658397674561e-01
<_>
0 1 1082 -5.3631910122931004e-03 -1 -2 1083
-1.2274970300495625e-02
1.7849500477313995e-01 7.2619527578353882e-01
4.5503988862037659e-01
<_>
0 1 1084 5.4185991175472736e-03 -1 -2 1085
8.1846961984410882e-04
5.2529907226562500e-01 5.4452222585678101e-01
3.2722181081771851e-01
<_>
1 0 1086 4.1358140297234058e-03 -1 -2 1087
3.9578010910190642e-04
7.0138317346572876e-01 4.9659439921379089e-01
3.2955980300903320e-01
<_>
0 1 1088 4.6887691132724285e-03 -1 -2 1089
-1.8255440518260002e-02
5.3626418113708496e-01 6.4961087703704834e-01
4.7571370005607605e-01
<_>
0 1 1090 -6.2736468389630318e-03 -1 -2 1091
2.4320168886333704e-03
2.3437410593032837e-01 4.6201181411743164e-01
6.8984192609786987e-01
<_>
0 1 1092 -4.9617629498243332e-02 -1 -2 1093
1.1701210169121623e-03
2.1007199585437775e-01 4.6215289831161499e-01
5.7971358299255371e-01
<_>
0 1 1094 -4.5237291604280472e-02 -1 -2 1095
4.7563421539962292e-03
2.1182620525360107e-01 4.8846149444580078e-01
6.8724989891052246e-01
<_>
1 0 1096 -1.4835969544947147e-02 -1 -2 1097
7.7436608262360096e-04
5.2751058340072632e-01 4.1723209619522095e-01
5.4911398887634277e-01
<_>
1 0 1098 1.4835969544947147e-02 -1 -2 1099
-8.0892542609944940e-04
2.1248769760131836e-01 5.4952150583267212e-01
4.2077958583831787e-01
<_>
0 1 1100 7.7517668250948191e-04 -1 -2 1101
-6.7618978209793568e-03
3.3219420909881592e-01 2.2129580378532410e-01
5.2326530218124390e-01
<_>
0 1 1102 -4.0135860443115234e-02 -1 -2 1103
-3.3651469275355339e-03
1.1017960309982300e-01 3.8101008534431458e-01
5.6172919273376465e-01
<_>
1 0 1104 7.4713007779791951e-04 -1 -2 1105
-4.2727389372885227e-03
5.7950568199157715e-01 6.3922691345214844e-01
4.7114381194114685e-01
<_>
1 0 1106 3.6202510818839073e-03 -1 -2 1107
4.7307618660852313e-04
3.4098839759826660e-01 3.6593028903007507e-01
5.3881710767745972e-01
<_>
1 0 1108 3.3094909042119980e-02 -1 -2 1109
-1.1544119566679001e-02
7.1703857183456421e-01 6.3868182897567749e-01
4.6813040971755981e-01
<_>
0 1 1110 -7.4234469793736935e-03 -1 -2 1111
-4.2252950370311737e-03
3.2637009024620056e-01 5.7678192853927612e-01
4.3464180827140808e-01
<_>
0 1 1112 1.8133109435439110e-02 -1 -2 1113
7.0903049781918526e-03
4.6978279948234558e-01 4.4373890757560730e-01
6.0616689920425415e-01
<_>
0 1 1114 -1.3272940181195736e-02 -1 -2 1115
1.4632199599873275e-04
6.5585112571716309e-01 3.3763539791107178e-01
5.0916552543640137e-01
<_>
0 1 1116 -3.5790191031992435e-03 -1 -2 1117
-4.6997101162560284e-04
2.9478839039802551e-01 5.5569821596145630e-01
4.6654561161994934e-01
<_>
0 1 1118 -4.8179440200328827e-02 -1 -2 1119
-9.2581362696364522e-04
7.3383557796478271e-01 3.5438719391822815e-01
5.2851498126983643e-01
<_>
0 1 1120 -1.4780730009078979e-02 -1 -2 1121
-1.0027450323104858e-01
1.9444419443607330e-01 9.9049292504787445e-02
5.1398539543151855e-01
<_>
0 1 1122 -9.3848101096227765e-04 -1 -2 1123
-2.8861360624432564e-03
5.8271098136901855e-01 3.4414279460906982e-01
5.1488387584686279e-01
<_>
1 0 1124 -4.3682761490345001e-02 -1 -2 1125
2.6115700602531433e-03
5.2079981565475464e-01 4.8355031013488770e-01
6.3222199678421021e-01
<_>
1 0 1126 4.3682761490345001e-02 -1 -2 1127
1.7179530113935471e-03
1.3645380735397339e-01 4.5373201370239258e-01
6.0667508840560913e-01
<_>
1 0 1128 -3.3964909613132477e-02 -1 -2 1129
-1.0993590112775564e-03
4.9683749675750732e-01 5.8316808938980103e-01
4.6882399916648865e-01
<_>
1 0 1130 5.4301079362630844e-02 -1 -2 1131
1.0993590112775564e-03
7.5682890415191650e-01 4.3301481008529663e-01
5.7684689760208130e-01
<_>
1 0 1132 -1.4954120160837192e-05 -1 -2 1133
3.1415868550539017e-02
4.4432818889617920e-01 5.2744728326797485e-01
3.0378559231758118e-01
<_>
1 0 1134 1.0831849649548531e-02 -1 -2 1135
8.6545711383223534e-04
3.5817208886146545e-01 5.9375840425491333e-01
4.2946299910545349e-01
<_>
1 0 1136 2.2743160370737314e-03 -1 -2 1137
3.9340821094810963e-03
5.9545767307281494e-01 4.7922229766845703e-01
5.8561331033706665e-01
<_>
1 0 1138 8.1451907753944397e-03 -1 -2 1139
-5.2763288840651512e-03
3.5734778642654419e-01 4.0260228514671326e-01
5.7647430896759033e-01
<_>
1 0 1140 -8.3787851035594940e-03 -1 -2 1141
1.5621910570189357e-03
4.9813330173492432e-01 4.7365880012512207e-01
5.5836081504821777e-01
<_>
1 0 1142 3.2318739686161280e-03 -1 -2 1143
6.6804019734263420e-03
6.1674368381500244e-01 4.1314241290092468e-01
6.2806951999664307e-01
<_>
0 1 1144 -3.3396480139344931e-03 -1 -2 1145
-2.0933480560779572e-01
3.4463581442832947e-01 1.0386580228805542e-01
5.2044892311096191e-01
<_>
1 0 1146 6.3805822283029556e-03 -1 -2 1147
-6.0137799009680748e-03
2.1674020588397980e-01 6.7383992671966553e-01
4.8966509103775024e-01
<_>
1 0 1148 -8.1756077706813812e-03 -1 -2 1149
6.3951779156923294e-04
5.1779150962829590e-01 4.8196458816528320e-01
5.4644381999969482e-01
<_>
1 0 1150 1.0127760469913483e-03 -1 -2 1151
4.9784599104896188e-04
3.4235960245132446e-01 4.4884610176086426e-01
5.9126710891723633e-01
<_>
1 0 1152 1.3596490316558629e-04 -1 -2 1153
1.3571660034358501e-02
5.5688631534576416e-01 5.1610678434371948e-01
1.7130009829998016e-01
<_>
1 0 1154 3.0259079721872695e-05 -1 -2 1155
-3.2625840976834297e-03
4.9162039160728455e-01 6.4046627283096313e-01
2.8590849041938782e-01
<_>
1 0 1156 -1.9217010412830859e-04 -1 -2 1157
2.1993879228830338e-02
5.4592829942703247e-01 4.7157138586044312e-01
5.6900751590728760e-01
<_>
1 0 1158 7.8907777788117528e-04 -1 -2 1159
5.0893891602754593e-04
3.2798269391059875e-01 4.3020078539848328e-01
5.6960451602935791e-01
<_>
1 0 1160 1.1662710312521085e-04 -1 -2 1161
8.0604078248143196e-03
5.3872352838516235e-01 5.0214231014251709e-01
5.9653222560882568e-01
<_>
1 0 1162 9.5925969071686268e-04 -1 -2 1163
-1.9526129588484764e-02
3.4734940528869629e-01 6.4755451679229736e-01
4.6437820792198181e-01
<_>
78
3.8236038208007812e+01
<_>
0 1 1164 4.1242439299821854e-02 -1 -2 1165
1.5626709908246994e-02
3.3933150768280029e-01 5.1041001081466675e-01
7.7728152275085449e-01
<_>
0 1 1166 2.9947189614176750e-04 -1 -2 1167
-1.0037609608843923e-03
3.6646738648414612e-01 5.4056507349014282e-01
3.9262050390243530e-01
<_>
0 1 1168 6.8128242855891585e-04 -1 -2 1169
1.3098999625071883e-04
4.2515191435813904e-01 4.1351449489593506e-01
6.9257462024688721e-01
<_>
1 0 1170 3.1696720980107784e-03 -1 -2 1171
-2.0587369799613953e-03
3.4558731317520142e-01 2.2341939806938171e-01
5.2861189842224121e-01
<_>
1 0 1172 -4.6395038953050971e-04 -1 -2 1173
3.5089480224996805e-03
4.2065200209617615e-01 6.5029817819595337e-01
4.1175979375839233e-01
<_>
1 0 1174 -2.3975980002433062e-03 -1 -2 1175
1.0901279747486115e-03
3.6733010411262512e-01 2.9062381386756897e-01
5.4451119899749756e-01
<_>
0 1 1176 -1.6524370585102588e-04 -1 -2 1177
-4.1602319106459618e-04
4.2335158586502075e-01 3.8863611221313477e-01
6.2691658735275269e-01
<_>
0 1 1178 -2.3739910102449358e-04 -1 -2 1179
2.4739760905504227e-02
5.5244511365890503e-01 4.9600958824157715e-01
5.3734910488128662e-01
<_>
0 1 1180 -1.5342839993536472e-02 -1 -2 1181
1.1540469713509083e-02
6.8494051694869995e-01 4.0372350811958313e-01
6.7869400978088379e-01
<_>
1 0 1182 6.4230621792376041e-03 -1 -2 1183
1.2977809645235538e-02
3.8146761059761047e-01 5.5270588397979736e-01
3.7449559569358826e-01
<_>
0 1 1184 1.1063399724662304e-03 -1 -2 1185
1.3743690215051174e-03
3.5209289193153381e-01 5.6419032812118530e-01
3.0750259757041931e-01
<_>
0 1 1186 1.6233779489994049e-02 -1 -2 1187
-8.1519351806491613e-04
4.8888280987739563e-01 5.4563212394714355e-01
4.7435501217842102e-01
<_>
0 1 1188 -9.0782493352890015e-02 -1 -2 1189
1.1665210127830505e-02
2.9252481460571289e-01 4.6884548664093018e-01
6.2303477525711060e-01
<_>
0 1 1190 -2.3286409676074982e-02 -1 -2 1191
2.1559339947998524e-03
6.8958431482315063e-01 5.3558021783828735e-01
3.4234660863876343e-01
<_>
0 1 1192 -4.3167220428586006e-03 -1 -2 1193
1.5610599657520652e-03
5.9370762109756470e-01 4.7086599469184875e-01
2.7369970083236694e-01
<_>
0 1 1194 1.4076639898121357e-02 -1 -2 1195
7.1018589660525322e-03
5.2871561050415039e-01 5.3361928462982178e-01
3.2248139381408691e-01
<_>
0 1 1196 -4.8221647739410400e-03 -1 -2 1197
-5.3852899000048637e-03
2.9839101433753967e-01 5.6239992380142212e-01
4.2959120869636536e-01
<_>
1 0 1198 7.3483278974890709e-03 -1 -2 1199
-3.5707519855350256e-03
6.8139612674713135e-01 5.8579689264297485e-01
4.6034291386604309e-01
<_>
1 0 1200 2.3340100888162851e-03 -1 -2 1201
4.7432780265808105e-03
2.7448511123657227e-01 5.0475269556045532e-01
2.3627419769763947e-01
<_>
0 1 1202 6.5055489540100098e-03 -1 -2 1203
1.2589249759912491e-02
5.2422481775283813e-01 4.8236909508705139e-01
6.7525368928909302e-01
<_>
0 1 1204 -6.3358368352055550e-03 -1 -2 1205
-5.7639651931822300e-03
1.7346349358558655e-01 6.3543808460235596e-01
4.5874750614166260e-01
<_>
0 1 1206 1.3599749654531479e-03 -1 -2 1207
2.8404260054230690e-02
4.5803809165954590e-01 5.1763808727264404e-01
1.2043850123882294e-01
<_>
0 1 1208 -9.2958156019449234e-03 -1 -2 1209
-1.1800320353358984e-03
2.3379570245742798e-01 3.9028140902519226e-01
5.6529301404953003e-01
<_>
0 1 1210 -2.0948140881955624e-03 -1 -2 1211
4.1679958812892437e-03
5.5120289325714111e-01 5.4559761285781860e-01
4.7989490628242493e-01
<_>
1 0 1212 5.4458891972899437e-03 -1 -2 1213
-1.2766510481014848e-03
6.1270868778228760e-01 5.3171318769454956e-01
3.8509321212768555e-01
<_>
0 1 1214 5.9404270723462105e-04 -1 -2 1215
4.2309608310461044e-02
5.4464370012283325e-01 5.2346438169479370e-01
2.2130440175533295e-01
<_>
0 1 1216 5.6189671158790588e-03 -1 -2 1217
7.2401198558509350e-03
4.9161979556083679e-01 1.4714759588241577e-01
4.8528939485549927e-01
<_>
0 1 1218 -4.5610670931637287e-03 -1 -2 1219
4.5506159949582070e-05
2.7737739682197571e-01 4.6264618635177612e-01
5.7680791616439819e-01
<_>
0 1 1220 -6.1903791502118111e-03 -1 -2 1221
8.1186462193727493e-04
1.6442899405956268e-01 4.7785910964012146e-01
6.2618649005889893e-01
<_>
0 1 1222 1.3779809698462486e-02 -1 -2 1223
1.1290319962427020e-03
5.2573078870773315e-01 5.4980480670928955e-01
3.9831069111824036e-01
<_>
0 1 1224 -1.0610350000206381e-04 -1 -2 1225
1.6695790691301227e-04
4.0335190296173096e-01 4.1493400931358337e-01
5.7953411340713501e-01
<_>
1 0 1226 1.1290319962427020e-03 -1 -2 1227
-1.2019349634647369e-01
3.9341148734092712e-01 7.3400482535362244e-02
5.2025860548019409e-01
<_>
0 1 1228 -1.5230740420520306e-02 -1 -2 1229
3.5759829916059971e-03
3.7495058774948120e-01 5.0781500339508057e-01
6.6060662269592285e-01
<_>
0 1 1230 1.3479460030794144e-02 -1 -2 1231
-2.1162950433790684e-03
4.5477110147476196e-01 3.3110061287879944e-01
5.3842592239379883e-01
<_>
0 1 1232 -1.7877709120512009e-02 -1 -2 1233
1.0931970318779349e-03
6.5132528543472290e-01 5.2647650241851807e-01
3.4569910168647766e-01
<_>
0 1 1234 -3.0553159303963184e-03 -1 -2 1235
3.6365049891173840e-03
6.2686139345169067e-01 5.3992128372192383e-01
4.3453970551490784e-01
<_>
0 1 1236 9.7896481747739017e-05 -1 -2 1237
-3.2714448752813041e-04
3.8356059789657593e-01 3.3376678824424744e-01
5.5391657352447510e-01
<_>
1 0 1238 4.3425030889920890e-04 -1 -2 1239
1.4005579985678196e-02
5.7882702350616455e-01 5.2750778198242188e-01
2.7011251449584961e-01
<_>
0 1 1240 -9.2654931358993053e-04 -1 -2 1241
3.9504268206655979e-03
5.8522802591323853e-01 4.7283369302749634e-01
3.3139181137084961e-01
<_>
1 0 1242 -5.8086868375539780e-04 -1 -2 1243
-1.2018020264804363e-02
4.2588108777999878e-01 5.6097871065139771e-01
4.8951920866966248e-01
<_>
0 1 1244 -1.4521540701389313e-01 -1 -2 1245
-6.6049019806087017e-03
4.3894480913877487e-02 4.2291709780693054e-01
5.6162929534912109e-01
<_>
1 0 1246 -3.4909751266241074e-02 -1 -2 1247
3.7478420417755842e-03
4.7881281375885010e-01 4.8002821207046509e-01
5.8013892173767090e-01
<_>
1 0 1248 3.3038031309843063e-02 -1 -2 1249
3.6872599739581347e-03
7.0781761407852173e-01 4.4496241211891174e-01
5.9577310085296631e-01
<_>
0 1 1250 -4.5311939902603626e-03 -1 -2 1251
4.1058510541915894e-03
4.1770470142364502e-01 5.3729480504989624e-01
3.7369269132614136e-01
<_>
0 1 1252 -8.7599847465753555e-03 -1 -2 1253
-2.3003309965133667e-02
6.6588079929351807e-01 2.6479220390319824e-01
5.1018178462982178e-01
<_>
0 1 1254 5.3664818406105042e-03 -1 -2 1255
3.8971770554780960e-02
4.5486348867416382e-01 5.1570618152618408e-01
3.4364390373229980e-01
<_>
0 1 1256 -2.7767190709710121e-02 -1 -2 1257
-9.8894089460372925e-03
2.3543910682201385e-01 6.8877410888671875e-01
5.1110517978668213e-01
<_>
0 1 1258 -3.2073140610009432e-03 -1 -2 1259
-6.7484978353604674e-04
5.4388678073883057e-01 5.4511487483978271e-01
4.8313531279563904e-01
<_>
0 1 1260 -5.1947520114481449e-03 -1 -2 1261
-2.6169899501837790e-04
2.1134190261363983e-01 5.2736818790435791e-01
3.9925870299339294e-01
<_>
0 1 1262 2.2421479225158691e-03 -1 -2 1263
-1.2139769969508052e-03
4.6882608532905579e-01 5.5042350292205811e-01
4.3848711252212524e-01
<_>
0 1 1264 -2.9469770379364491e-03 -1 -2 1265
-3.9291830034926534e-04
3.8928470015525818e-01 6.0017228126525879e-01
4.5616629719734192e-01
<_>
1 0 1266 6.2550729513168335e-01 -1 -2 1267
9.7744520753622055e-03
6.8125613033771515e-02 4.8130258917808533e-01
5.6206572055816650e-01
<_>
1 0 1268 9.4378247857093811e-02 -1 -2 1269
-1.9560910295695066e-03
6.6632293164730072e-02 3.5882329940795898e-01
5.2954071760177612e-01
<_>
0 1 1270 9.0652769431471825e-03 -1 -2 1271
4.2138071148656309e-04
4.8226881027221680e-01 4.6703329682350159e-01
5.6831127405166626e-01
<_>
1 0 1272 -4.4220191193744540e-04 -1 -2 1273
-4.7313501127064228e-03
5.3607952594757080e-01 6.1372458934783936e-01
3.1880891323089600e-01
<_>
0 1 1274 1.5395509544759989e-03 -1 -2 1275
2.4315000046044588e-03
4.4877201318740845e-01 4.8941668868064880e-01
6.7166537046432495e-01
<_>
0 1 1276 -1.5581619925796986e-02 -1 -2 1277
1.0816920548677444e-03
3.3367419242858887e-01 4.7182199358940125e-01
5.9606271982192993e-01
<_>
0 1 1278 -2.2197659127414227e-03 -1 -2 1279
-9.3048671260476112e-04
3.5885548591613770e-01 6.2187129259109497e-01
4.8173001408576965e-01
<_>
0 1 1280 -4.7418707981705666e-03 -1 -2 1281
-6.2950369901955128e-03
2.5500270724296570e-01 6.7280787229537964e-01
5.0510638952255249e-01
<_>
0 1 1282 3.5216049291193485e-03 -1 -2 1283
-2.4289379362016916e-03
5.4019099473953247e-01 5.4194617271423340e-01
4.3471428751945496e-01
<_>
0 1 1284 -2.5261470582336187e-03 -1 -2 1285
-1.4817339833825827e-03
6.9706249237060547e-01 3.2634168863296509e-01
4.9178731441497803e-01
<_>
0 1 1286 -2.2474530339241028e-01 -1 -2 1287
2.8342509176582098e-03
7.2937291115522385e-03 4.5792299509048462e-01
5.3798812627792358e-01
<_>
0 1 1288 -2.0821610465645790e-02 -1 -2 1289
1.4896340144332498e-04
6.0240888595581055e-01 3.3361440896987915e-01
4.9628159403800964e-01
<_>
0 1 1290 -3.3524499740451574e-03 -1 -2 1291
-3.7279881536960602e-02
3.5587510466575623e-01 1.6985629498958588e-01
5.2089858055114746e-01
<_>
1 0 1292 1.3896770542487502e-04 -1 -2 1293
-3.1912620761431754e-04
5.5906862020492554e-01 5.8487337827682495e-01
3.7958368659019470e-01
<_>
1 0 1294 5.4003461264073849e-04 -1 -2 1295
3.8956850767135620e-03
5.6702882051467896e-01 5.1826947927474976e-01
3.3277091383934021e-01
<_>
1 0 1296 1.6084529925137758e-03 -1 -2 1297
-5.7474587811157107e-04
5.4104858636856079e-01 6.0226422548294067e-01
3.6446440219879150e-01
<_>
1 0 1298 1.3435039669275284e-02 -1 -2 1299
2.1368139423429966e-03
3.4412819147109985e-01 5.2924340963363647e-01
2.7470758557319641e-01
<_>
1 0 1300 1.4157629571855068e-02 -1 -2 1301
5.3884391672909260e-03
8.0278682708740234e-01 5.2223151922225952e-01
3.5867279767990112e-01
<_>
0 1 1302 8.8013410568237305e-03 -1 -2 1303
3.8858849438838661e-04
4.9003869295120239e-01 4.6810561418533325e-01
5.7219529151916504e-01
<_>
0 1 1304 -2.2143588867038488e-03 -1 -2 1305
-8.4642972797155380e-03
5.3888058662414551e-01 6.6755378246307373e-01
3.4484419226646423e-01
<_>
1 0 1306 1.5044390223920345e-02 -1 -2 1307
7.6346402056515217e-03
9.2396140098571777e-01 4.8848968744277954e-01
6.3060528039932251e-01
<_>
1 0 1308 3.3895121305249631e-04 -1 -2 1309
2.1157610171940178e-04
3.9974310994148254e-01 5.6639820337295532e-01
3.9729809761047363e-01
<_>
1 0 1310 -2.7514949440956116e-02 -1 -2 1311
5.1603060215711594e-02
5.2010637521743774e-01 5.1407301425933838e-01
1.2451309710741043e-01
<_>
1 0 1312 3.7510651163756847e-03 -1 -2 1313
-2.1457639522850513e-03
3.8020950555801392e-01 3.3094480633735657e-01
5.4745388031005859e-01
<_>
1 0 1314 -5.8178009930998087e-04 -1 -2 1315
-9.3638541875407100e-04
4.8926019668579102e-01 5.9373992681503296e-01
4.6646690368652344e-01
<_>
1 0 1316 4.1667491197586060e-02 -1 -2 1317
-6.7763780243694782e-03
7.0213532447814941e-01 3.2227510213851929e-01
5.0683951377868652e-01
<_>
1 0 1318 -2.9170580673962831e-03 -1 -2 1319
3.2789530814625323e-04
4.7177010774612427e-01 4.5093831419944763e-01
5.6511628627777100e-01
<_>
91
4.4682968139648438e+01
<_>
0 1 1320 1.1729800142347813e-02 -1 -2 1321
1.1712179984897375e-03
3.8052248954772949e-01 3.1400179862976074e-01
6.8581461906433105e-01
<_>
1 0 1322 9.3555096536874771e-03 -1 -2 1323
1.6570610459893942e-03
6.8346732854843140e-01 2.9924729466438293e-01
5.4756778478622437e-01
<_>
1 0 1324 -1.3387809740379453e-03 -1 -2 1325
1.7580550047568977e-04
2.9414069652557373e-01 3.8969779014587402e-01
5.8729708194732666e-01
<_>
0 1 1326 -2.9473248869180679e-03 -1 -2 1327
8.3220899105072021e-03
3.5765719413757324e-01 5.2324008941650391e-01
3.2310879230499268e-01
<_>
1 0 1328 7.4366689659655094e-03 -1 -2 1329
-2.1322889369912446e-04
6.7156732082366943e-01 5.4705417156219482e-01
3.8633960485458374e-01
<_>
0 1 1330 -7.8024631366133690e-03 -1 -2 1331
5.6611228501424193e-04
2.7714601159095764e-01 4.6891361474990845e-01
5.8519637584686279e-01
<_>
0 1 1332 -9.2346500605344772e-03 -1 -2 1333
-1.4676499631605111e-05
2.7043971419334412e-01 5.6225502490997314e-01
3.5793170332908630e-01
<_>
0 1 1334 9.7007937729358673e-03 -1 -2 1335
-3.5320650786161423e-03
4.1738718748092651e-01 4.1950130462646484e-01
5.5494689941406250e-01
<_>
1 0 1336 2.1616410464048386e-02 -1 -2 1337
3.4567608963698149e-03
2.8573909401893616e-01 6.0245329141616821e-01
4.3775078654289246e-01
<_>
0 1 1338 2.2914320230484009e-02 -1 -2 1339
3.4328910987824202e-03
4.6893501281738281e-01 4.6646049618721008e-01
5.7625621557235718e-01
<_>
0 1 1340 -8.6510833352804184e-03 -1 -2 1341
1.4510039472952485e-03
6.3817399740219116e-01 3.7114879488945007e-01
5.5307507514953613e-01
<_>
0 1 1342 7.8191719949245453e-03 -1 -2 1343
2.0798550394829363e-04
5.2643620967864990e-01 3.7305128574371338e-01
5.4457312822341919e-01
<_>
0 1 1344 -3.9962218143045902e-03 -1 -2 1345
-1.5010139577498194e-05
2.4381700158119202e-01 5.3246712684631348e-01
3.6829888820648193e-01
<_>
0 1 1346 -4.2428788729012012e-03 -1 -2 1347
9.1374982148408890e-03
6.4814740419387817e-01 4.8961588740348816e-01
6.5588432550430298e-01
<_>
1 0 1348 8.8254585862159729e-03 -1 -2 1349
9.4092212384566665e-04
3.6138701438903809e-01 5.5028957128524780e-01
3.6325180530548096e-01
<_>
0 1 1350 -1.2503350153565407e-02 -1 -2 1351
8.6759645491838455e-03
2.2611320018768311e-01 4.9878901243209839e-01
6.8471962213516235e-01
<_>
0 1 1352 -1.0416760109364986e-02 -1 -2 1353
2.7432460337877274e-03
2.4462990462779999e-01 3.5115250945091248e-01
5.3998267650604248e-01
<_>
0 1 1354 -4.2385691776871681e-03 -1 -2 1355
1.8325870856642723e-02
6.8236732482910156e-01 4.8915800452232361e-01
7.1356189250946045e-01
<_>
0 1 1356 -2.4334540590643883e-02 -1 -2 1357
4.6469361404888332e-04
3.5225218534469604e-01 4.0498688817024231e-01
5.5158257484436035e-01
<_>
1 0 1358 3.4260009415447712e-03 -1 -2 1359
-2.5827318895608187e-03
4.1267699003219604e-01 2.8994289040565491e-01
5.3864318132400513e-01
<_>
1 0 1360 1.0545699624344707e-03 -1 -2 1361
-9.1257691383361816e-04
3.7713441252708435e-01 5.8273869752883911e-01
4.2675569653511047e-01
<_>
0 1 1362 2.6589010376483202e-03 -1 -2 1363
4.8598358407616615e-03
4.6881249547004700e-01 4.8539221286773682e-01
6.1636447906494141e-01
<_>
1 0 1364 8.0638676881790161e-03 -1 -2 1365
-7.5898370705544949e-03
1.7491950094699860e-01 6.8261897563934326e-01
4.8940700292587280e-01
<_>
0 1 1366 3.6368070868775249e-04 -1 -2 1367
6.2594950199127197e-02
4.6145960688591003e-01 5.1830172538757324e-01
2.6866960525512695e-01
<_>
0 1 1368 -4.9753207713365555e-03 -1 -2 1369
-2.0880119409412146e-03
1.7584669589996338e-01 6.3693821430206299e-01
4.9300441145896912e-01
<_>
1 0 1370 9.5644511748105288e-04 -1 -2 1371
-3.1721461564302444e-02
4.1393989324569702e-01 6.0455572605133057e-01
4.8163640499114990e-01
<_>
0 1 1372 1.2898689601570368e-03 -1 -2 1373
9.8405163735151291e-03
5.4508107900619507e-01 2.9240009188652039e-01
6.6996061801910400e-01
<_>
1 0 1374 1.2237089686095715e-03 -1 -2 1375
-8.4232585504651070e-03
6.2828367948532104e-01 5.9865701198577881e-01
4.8525801301002502e-01
<_>
0 1 1376 -7.2726322105154395e-04 -1 -2 1377
4.6842931769788265e-03
3.3400490880012512e-01 5.1689237356185913e-01
2.6794800162315369e-01
<_>
0 1 1378 -1.0379579616710544e-03 -1 -2 1379
9.1342730447649956e-03
5.9257918596267700e-01 5.4377281665802002e-01
4.3468001484870911e-01
<_>
0 1 1380 1.4971119817346334e-03 -1 -2 1381
1.5762320253998041e-03
4.1295009851455688e-01 4.5228740572929382e-01
6.5562921762466431e-01
<_>
0 1 1382 8.7496247142553329e-03 -1 -2 1383
-8.5103599121794105e-04
4.5320340991020203e-01 3.7859839200973511e-01
5.4169750213623047e-01
<_>
0 1 1384 -1.7325570806860924e-02 -1 -2 1385
-8.3266440778970718e-03
6.8842482566833496e-01 3.0913260579109192e-01
5.2436548471450806e-01
<_>
0 1 1386 1.5157909729168750e-05 -1 -2 1387
1.8041470320895314e-03
4.7657939791679382e-01 4.7253859043121338e-01
5.7165551185607910e-01
<_>
1 0 1388 3.0691560823470354e-03 -1 -2 1389
-5.2225510444259271e-05
2.1433599293231964e-01 5.6532102823257446e-01
4.3851110339164734e-01
<_>
1 0 1390 1.0072169970953837e-04 -1 -2 1391
1.3573700562119484e-04
5.9247761964797974e-01 4.5734488964080811e-01
5.7693827152252197e-01
<_>
1 0 1392 9.2137878527864814e-04 -1 -2 1393
3.0316581251099706e-04
5.9926092624664307e-01 3.6100810766220093e-01
5.0493258237838745e-01
<_>
1 0 1394 3.9582479745149612e-02 -1 -2 1395
4.7519680112600327e-02
1.5384890139102936e-01 5.2161407470703125e-01
1.4283910393714905e-01
<_>
1 0 1396 1.8871759995818138e-02 -1 -2 1397
-3.9876459049992263e-04
2.8255069255828857e-01 4.0350168943405151e-01
5.4377931356430054e-01
<_>
0 1 1398 4.6556600136682391e-04 -1 -2 1399
6.7090610973536968e-03
4.6689969301223755e-01 5.3313547372817993e-01
4.1365718841552734e-01
<_>
0 1 1400 -1.8931160448119044e-03 -1 -2 1401
-1.3056949712336063e-02
7.1551632881164551e-01 3.1178998947143555e-01
5.2084398269653320e-01
<_>
1 0 1402 -1.9484119547996670e-04 -1 -2 1403
1.5093220099515747e-05
4.6376588940620422e-01 4.5616531372070312e-01
5.4452341794967651e-01
<_>
1 0 1404 -7.1617960202274844e-06 -1 -2 1405
3.0164679628796875e-04
4.1931080818176270e-01 5.9662377834320068e-01
4.1005000472068787e-01
<_>
0 1 1406 4.4195181690156460e-03 -1 -2 1407
-7.3984181508421898e-03
4.8450559377670288e-01 6.2068462371826172e-01
4.9312090873718262e-01
<_>
1 0 1408 -7.8031201846897602e-03 -1 -2 1409
-1.0731429792940617e-02
5.2824628353118896e-01 9.1048341989517212e-01
3.4559220075607300e-01
<_>
0 1 1410 1.4246780192479491e-03 -1 -2 1411
-8.2717568147927523e-05
4.7085541486740112e-01 5.6516230106353760e-01
4.7310239076614380e-01
<_>
1 0 1412 4.4803409837186337e-03 -1 -2 1413
3.0789140146225691e-03
6.1758869886398315e-01 5.1395332813262939e-01
3.4230878949165344e-01
<_>
1 0 1414 -1.1310289846733212e-03 -1 -2 1415
-1.0410690447315574e-03
4.9182820320129395e-01 5.9420871734619141e-01
4.9230429530143738e-01
<_>
1 0 1416 1.1648540385067463e-03 -1 -2 1417
9.0057362103834748e-04
6.4052718877792358e-01 4.5043969154357910e-01
6.1920768022537231e-01
<_>
0 1 1418 6.8781538866460323e-03 -1 -2 1419
-3.5283900797367096e-02
5.3748130798339844e-01 2.2471010684967041e-01
5.2171707153320312e-01
<_>
0 1 1420 -1.3320200378075242e-03 -1 -2 1421
-2.3177571129053831e-03
2.5547030568122864e-01 3.7925159931182861e-01
5.2432268857955933e-01
<_>
0 1 1422 2.1332940377760679e-04 -1 -2 1423
1.3467900454998016e-02
3.8603371381759644e-01 5.3806877136230469e-01
4.1783639788627625e-01
<_>
0 1 1424 -1.2829169863834977e-03 -1 -2 1425
5.1571638323366642e-04
6.1336231231689453e-01 4.0285378694534302e-01
5.5368518829345703e-01
<_>
0 1 1426 3.9254198782145977e-03 -1 -2 1427
-3.3780589699745178e-02
5.2799212932586670e-01 2.3346750438213348e-01
5.1759117841720581e-01
<_>
0 1 1428 -3.7853721529245377e-02 -1 -2 1429
-4.0752900531515479e-04
1.0748530179262161e-01 5.3459298610687256e-01
4.1989380121231079e-01
<_>
0 1 1430 -3.1193809118121862e-03 -1 -2 1431
-1.5714969485998154e-02
3.8558250665664673e-01 3.3351901173591614e-01
5.2632021903991699e-01
<_>
0 1 1432 -7.8525702701881528e-04 -1 -2 1433
-2.8750501223839819e-04
5.8603972196578979e-01 5.4377847909927368e-01
3.7161049246788025e-01
<_>
1 0 1434 2.8016859665513039e-02 -1 -2 1435
-1.9018839811906219e-03
3.3307549357414246e-01 5.3665977716445923e-01
4.6937939524650574e-01
<_>
1 0 1436 2.0647559314966202e-02 -1 -2 1437
4.3002571910619736e-03
1.0069560259580612e-01 4.8160359263420105e-01
6.2156772613525391e-01
<_>
0 1 1438 1.3459140434861183e-02 -1 -2 1439
-1.0320040397346020e-02
5.4619538784027100e-01 4.5784530043601990e-01
5.4193097352981567e-01
<_>
1 0 1440 3.1990748643875122e-01 -1 -2 1441
9.2198798665776849e-04
2.0080469548702240e-01 5.1932811737060547e-01
3.9121940732002258e-01
<_>
0 1 1442 4.1852539288811386e-04 -1 -2 1443
3.5891108564101160e-04
4.2997440695762634e-01 4.3445029854774475e-01
5.5319738388061523e-01
<_>
0 1 1444 -2.0992439985275269e-01 -1 -2 1445
-4.9328152090311050e-03
1.0757210105657578e-01 5.7627969980239868e-01
4.5746439695358276e-01
<_>
1 0 1446 2.3409130517393351e-03 -1 -2 1447
4.7120270319283009e-03
7.4768078327178955e-01 5.2617651224136353e-01
4.5055508613586426e-01
<_>
0 1 1448 2.8713190928101540e-02 -1 -2 1449
-2.6156550738960505e-03
4.4071030616760254e-01 4.2442709207534790e-01
6.8929767608642578e-01
<_>
0 1 1450 -1.3558969832956791e-02 -1 -2 1451
-3.0331799644045532e-04
1.2522679567337036e-01 4.0777918696403503e-01
5.4428178071975708e-01
<_>
0 1 1452 -5.5601762142032385e-04 -1 -2 1453
2.4025330785661936e-03
5.3780037164688110e-01 3.1665799021720886e-01
5.2857381105422974e-01
<_>
1 0 1454 -3.4089901018887758e-03 -1 -2 1455
8.0019602319225669e-04
4.9052149057388306e-01 4.5227360725402832e-01
5.5806142091751099e-01
<_>
1 0 1456 2.1901070140302181e-03 -1 -2 1457
3.3745369873940945e-03
6.6126817464828491e-01 5.1077651977539062e-01
3.3869299292564392e-01
<_>
1 0 1458 8.0019602319225669e-04 -1 -2 1459
1.7346069216728210e-02
5.7075601816177368e-01 5.0160211324691772e-01
6.3064599037170410e-01
<_>
0 1 1460 -1.9568449351936579e-03 -1 -2 1461
-1.1229019612073898e-02
3.0178061127662659e-01 6.2938511371612549e-01
4.5204889774322510e-01
<_>
0 1 1462 -2.6608388870954514e-03 -1 -2 1463
-1.1615100316703320e-02
3.3440071344375610e-01 2.8253790736198425e-01
5.1509708166122437e-01
<_>
0 1 1464 -9.5248602330684662e-02 -1 -2 1465
7.3701781220734119e-03
1.3982650637626648e-01 5.2939987182617188e-01
2.3317280411720276e-01
<_>
1 0 1466 -1.4953900128602982e-02 -1 -2 1467
5.7038792874664068e-04
4.9404659867286682e-01 5.4665708541870117e-01
4.6267679333686829e-01
<_>
1 0 1468 5.8516198769211769e-03 -1 -2 1469
2.1150549582671374e-04
6.2700408697128296e-01 5.5081409215927124e-01
4.0618729591369629e-01
<_>
1 0 1470 -6.9679190346505493e-06 -1 -2 1471
-7.9677387839183211e-04
4.0965679287910461e-01 5.6155568361282349e-01
4.6668860316276550e-01
<_>
1 0 1472 1.9459480419754982e-02 -1 -2 1473
-1.1160830035805702e-02
2.3114809393882751e-01 3.0870118737220764e-01
5.5146622657775879e-01
<_>
1 0 1474 1.4056149870157242e-02 -1 -2 1475
-3.2958350493572652e-04
7.0050561428070068e-01 5.7974857091903687e-01
4.6916508674621582e-01
<_>
0 1 1476 -5.4636420682072639e-03 -1 -2 1477
5.8881669247057289e-05
5.9285950660705566e-01 3.7413978576660156e-01
5.1701688766479492e-01
<_>
0 1 1478 6.6343429498374462e-03 -1 -2 1479
4.5263409614562988e-02
5.4149878025054932e-01 5.1803272962570190e-01
1.5296840667724609e-01
<_>
0 1 1480 -8.0646127462387085e-03 -1 -2 1481
4.7389548853971064e-04
2.5154680013656616e-01 5.1219987869262695e-01
3.7259489297866821e-01
<_>
1 0 1482 1.4877359717502259e-05 -1 -2 1483
2.4321159347891808e-02
5.5324357748031616e-01 4.9607661366462708e-01
5.9833151102066040e-01
<_>
0 1 1484 6.9931396865285933e-05 -1 -2 1485
2.6287760119885206e-03
4.1639530658721924e-01 5.8801448345184326e-01
3.3996629714965820e-01
<_>
1 0 1486 3.8190539926290512e-03 -1 -2 1487
-2.5989150628447533e-02
7.8466212749481201e-01 3.2881140708923340e-01
5.1550877094268799e-01
<_>
0 1 1488 1.2062400346621871e-03 -1 -2 1489
-1.5557400183752179e-03
4.5960599184036255e-01 3.1269869208335876e-01
7.1833992004394531e-01
<_>
1 0 1490 -2.2691930644214153e-03 -1 -2 1491
2.3287249496206641e-04
5.2740061283111572e-01 4.8786661028862000e-01
5.6151527166366577e-01
<_>
1 0 1492 -5.5999699980020523e-03 -1 -2 1493
-1.0496189817786217e-02
5.1608121395111084e-01 5.7016140222549438e-01
3.2048508524894714e-01
<_>
0 1 1494 -1.4814930182183161e-05 -1 -2 1495
-6.4287078566849232e-04
5.5388379096984863e-01 5.3494292497634888e-01
4.4721511006355286e-01
<_>
0 1 1496 -1.8891949730459601e-04 -1 -2 1497
-9.0413521975278854e-03
5.0128370523452759e-01 2.5629359483718872e-01
4.5033830404281616e-01
<_>
1 0 1498 7.9534705728292465e-03 -1 -2 1499
-2.7908999472856522e-03
2.6304998993873596e-01 5.7565087080001831e-01
4.8548638820648193e-01
<_>
1 0 1500 3.2857100013643503e-03 -1 -2 1501
7.7063008211553097e-04
4.0847519040107727e-01 4.0733560919761658e-01
5.9202408790588379e-01
<_>
97
4.7763450622558594e+01
<_>
0 1 1502 6.3021942973136902e-02 -1 -2 1503
-2.8374609537422657e-03
3.4193828701972961e-01 6.8295639753341675e-01
4.4045230746269226e-01
<_>
0 1 1504 4.6461950987577438e-02 -1 -2 1505
2.9152540490031242e-02
4.3917450308799744e-01 4.6010631322860718e-01
6.3579368591308594e-01
<_>
1 0 1506 -1.4000290320836939e-05 -1 -2 1507
-1.2757079675793648e-03
3.7300100922584534e-01 3.0938240885734558e-01
5.9013700485229492e-01
<_>
0 1 1508 1.3596529606729746e-03 -1 -2 1509
1.7991929780691862e-04
4.3375650048255920e-01 4.2175039649009705e-01
5.8468478918075562e-01
<_>
1 0 1510 -1.4166639630275313e-05 -1 -2 1511
6.0252390539972112e-05
4.0846911072731018e-01 5.0872868299484253e-01
7.2771841287612915e-01
<_>
1 0 1512 6.4320368692278862e-03 -1 -2 1513
4.6682319953106344e-04
2.9679030179977417e-01 4.1104629635810852e-01
5.5812197923660278e-01
<_>
0 1 1514 5.7436279021203518e-03 -1 -2 1515
3.2019240316003561e-03
4.2873099446296692e-01 4.2661958932876587e-01
6.4440459012985229e-01
<_>
1 0 1516 -5.7637941790744662e-04 -1 -2 1517
-3.7901920732110739e-03
4.0848249197006226e-01 3.1819209456443787e-01
5.2306932210922241e-01
<_>
1 0 1518 4.8914109356701374e-03 -1 -2 1519
4.6459292061626911e-03
3.5483568906784058e-01 5.6105977296829224e-01
2.6938489079475403e-01
<_>
0 1 1520 -6.8799369037151337e-03 -1 -2 1521
-1.8147470429539680e-02
6.2354081869125366e-01 2.8619819879531860e-01
5.2268481254577637e-01
<_>
1 0 1522 1.1409220314817503e-04 -1 -2 1523
-5.4334272863343358e-04
3.2578331232070923e-01 3.8829690217971802e-01
5.3411662578582764e-01
<_>
0 1 1524 -2.7602489572018385e-03 -1 -2 1525
-1.9730569329112768e-03
6.3539659976959229e-01 5.8807611465454102e-01
4.5930901169776917e-01
<_>
1 0 1526 2.4565239436924458e-03 -1 -2 1527
1.9392010290175676e-04
3.1340101361274719e-01 5.2771317958831787e-01
3.6041069030761719e-01
<_>
0 1 1528 7.8643016517162323e-02 -1 -2 1529
6.5276869572699070e-03
5.2903419733047485e-01 4.6544799208641052e-01
6.0449051856994629e-01
<_>
0 1 1530 -7.8716799616813660e-02 -1 -2 1531
5.7298499159514904e-03
2.5411269068717957e-01 4.3669191002845764e-01
5.8228862285614014e-01
<_>
1 0 1532 6.2386557692661881e-04 -1 -2 1533
-8.5267230868339539e-02
5.4726922512054443e-01 1.4616079628467560e-01
5.1818108558654785e-01
<_>
1 0 1534 4.0981110185384750e-02 -1 -2 1535
7.7135749161243439e-03
1.2701350450515747e-01 4.8326849937438965e-01
2.2235789895057678e-01
<_>
0 1 1536 -6.8663940764963627e-03 -1 -2 1537
1.4559639617800713e-02
5.9189289808273315e-01 4.7615069150924683e-01
5.7272237539291382e-01
<_>
0 1 1538 -1.0064310394227505e-02 -1 -2 1539
3.6274080630391836e-03
3.6367309093475342e-01 5.2717310190200806e-01
2.7405250072479248e-01
<_>
0 1 1540 -2.3421540390700102e-03 -1 -2 1541
-2.4686409160494804e-02
5.4977840185165405e-01 6.0598951578140259e-01
4.9603140354156494e-01
<_>
1 0 1542 1.9456120207905769e-04 -1 -2 1543
3.1714211218059063e-04
3.7694650888442993e-01 4.0623620152473450e-01
5.6682151556015015e-01
<_>
0 1 1544 2.0793990697711706e-03 -1 -2 1545
1.7982709687203169e-03
4.6186569333076477e-01 4.8675051331520081e-01
6.5184497833251953e-01
<_>
0 1 1546 -2.2287059982772917e-04 -1 -2 1547
3.2623921288177371e-04
5.6775957345962524e-01 3.7107339501380920e-01
5.6766051054000854e-01
<_>
0 1 1548 -6.6792681813240051e-02 -1 -2 1549
-1.4869889710098505e-03
2.5115218758583069e-01 3.8867509365081787e-01
5.2622538805007935e-01
<_>
0 1 1550 -5.0454870797693729e-03 -1 -2 1551
-4.8297587782144547e-03
6.5574729442596436e-01 5.9341061115264893e-01
4.2859220504760742e-01
<_>
1 0 1552 -1.0722599690780044e-03 -1 -2 1553
8.7901195511221886e-03
5.4260587692260742e-01 5.3513032197952271e-01
4.8342779278755188e-01
<_>
0 1 1554 -7.1750381030142307e-03 -1 -2 1555
1.1251230025663972e-03
2.0671689510345459e-01 5.1122522354125977e-01
3.4687140583992004e-01
<_>
0 1 1556 1.0634710080921650e-02 -1 -2 1557
-1.1763219721615314e-02
4.4790080189704895e-01 6.2539017200469971e-01
4.9689871072769165e-01
<_>
1 0 1558 9.2324063181877136e-02 -1 -2 1559
1.8991080578416586e-03
2.0313039422035217e-01 5.6187218427658081e-01
4.0465721487998962e-01
<_>
1 0 1560 -1.0510340332984924e-02 -1 -2 1561
-7.4531312566250563e-04
4.9432641267776489e-01 5.6134277582168579e-01
3.8453319668769836e-01
<_>
1 0 1562 8.0041000619530678e-03 -1 -2 1563
5.8110528625547886e-03
7.7598422765731812e-01 4.6247330307960510e-01
6.2862771749496460e-01
<_>
0 1 1564 -2.7918580919504166e-02 -1 -2 1565
2.1739399526268244e-03
2.4093140661716461e-01 5.3455048799514771e-01
3.5079580545425415e-01
<_>
0 1 1566 -4.0639587678015232e-03 -1 -2 1567
6.0017139185220003e-04
6.6471010446548462e-01 4.9985098838806152e-01
3.0221650004386902e-01
<_>
1 0 1568 1.9214770291000605e-03 -1 -2 1569
-1.3860830105841160e-02
5.9191507101058960e-01 6.3517677783966064e-01
4.9933108687400818e-01
<_>
1 0 1570 2.3006850853562355e-02 -1 -2 1571
-1.3857929734513164e-03
1.9023360311985016e-01 5.2533692121505737e-01
3.9858600497245789e-01
<_>
0 1 1572 1.2637410545721650e-03 -1 -2 1573
-1.4675210230052471e-02
4.6661040186882019e-01 3.8231649994850159e-01
5.3266328573226929e-01
<_>
0 1 1574 -2.9535070061683655e-03 -1 -2 1575
-1.7189770005643368e-03
7.0636558532714844e-01 3.8134628534317017e-01
5.2467352151870728e-01
<_>
1 0 1576 -4.2484089499339461e-04 -1 -2 1577
-8.5248658433556557e-04
4.7916388511657715e-01 4.4912180304527283e-01
5.3709012269973755e-01
<_>
1 0 1578 8.9034568518400192e-03 -1 -2 1579
1.4895649655954912e-05
2.0764739811420441e-01 4.4476351141929626e-01
5.6671631336212158e-01
<_>
0 1 1580 -4.7091601300053298e-04 -1 -2 1581
4.3084810022264719e-04
5.4650712013244629e-01 5.4932618141174316e-01
4.5807081460952759e-01
<_>
0 1 1582 -6.3893961487337947e-04 -1 -2 1583
-7.3733746830839664e-05
5.5015718936920166e-01 5.0857907533645630e-01
3.3056980371475220e-01
<_>
0 1 1584 -8.8991485536098480e-03 -1 -2 1585
-1.0253350250422955e-02
4.2764690518379211e-01 1.1232180148363113e-01
5.1527231931686401e-01
<_>
0 1 1586 -5.9637490659952164e-02 -1 -2 1587
2.1707199513912201e-02
7.3867720365524292e-01 4.9962919950485229e-01
1.3394139707088470e-01
<_>
0 1 1588 9.9107045680284500e-03 -1 -2 1589
-1.0998300276696682e-02
4.6790120005607605e-01 6.9286561012268066e-01
5.0120681524276733e-01
<_>
1 0 1590 7.4608891736716032e-04 -1 -2 1591
2.9539171373471618e-04
5.8335822820663452e-01 3.8263911008834839e-01
5.5663508176803589e-01
<_>
1 0 1592 5.0054129213094711e-02 -1 -2 1593
-7.2330660186707973e-03
3.0027210712432861e-01 5.9080427885055542e-01
5.0008708238601685e-01
<_>
0 1 1594 -2.6863380335271358e-03 -1 -2 1595
-1.0195849463343620e-03
3.9750349521636963e-01 3.6976858973503113e-01
5.7561928033828735e-01
<_>
0 1 1596 -2.0204920321702957e-02 -1 -2 1597
2.1340379025787115e-03
6.3752681016921997e-01 5.3632658720016479e-01
4.4331708550453186e-01
<_>
0 1 1598 -1.8348889425396919e-03 -1 -2 1599
-5.9489468112587929e-03
5.8289992809295654e-01 2.6806709170341492e-01
4.6428859233856201e-01
<_>
0 1 1600 -2.3030120064504445e-04 -1 -2 1601
5.0581009127199650e-03
5.4753202199935913e-01 5.3208339214324951e-01
4.6464928984642029e-01
<_>
0 1 1602 -5.1950011402368546e-04 -1 -2 1603
-6.8620947422459722e-04
5.2327448129653931e-01 4.9350860714912415e-01
3.1031179428100586e-01
<_>
0 1 1604 -7.4936267919838428e-03 -1 -2 1605
-1.5682930126786232e-02
2.8830468654632568e-01 3.6403131484985352e-01
5.3687548637390137e-01
<_>
0 1 1606 -3.2649750355631113e-03 -1 -2 1607
3.8463930832222104e-04
6.4686310291290283e-01 5.2596598863601685e-01
3.8314279913902283e-01
<_>
1 0 1608 4.4492390006780624e-03 -1 -2 1609
2.3118320852518082e-02
2.0868189632892609e-01 4.9785330891609192e-01
5.9612572193145752e-01
<_>
1 0 1610 2.0835159812122583e-03 -1 -2 1611
1.1513150529935956e-03
5.7464218139648438e-01 3.5868450999259949e-01
5.3634738922119141e-01
<_>
1 0 1612 3.6104708909988403e-02 -1 -2 1613
3.6256198654882610e-04
2.8331369161605835e-01 5.4777222871780396e-01
4.1105321049690247e-01
<_>
0 1 1614 -3.4635469783097506e-03 -1 -2 1615
-2.8796829283237457e-03
5.9903860092163086e-01 5.7252532243728638e-01
4.1495120525360107e-01
<_>
1 0 1616 -8.1119500100612640e-03 -1 -2 1617
4.5932079665362835e-03
5.3963518142700195e-01 5.3797042369842529e-01
3.8913029432296753e-01
<_>
1 0 1618 7.0014740340411663e-03 -1 -2 1619
8.0169539432972670e-04
3.7146711349487305e-01 5.5295670032501221e-01
3.7558048963546753e-01
<_>
1 0 1620 -8.6652329191565514e-03 -1 -2 1621
-2.7315050829201937e-03
5.0257730484008789e-01 5.8503222465515137e-01
4.6175739169120789e-01
<_>
1 0 1622 1.3301590224727988e-03 -1 -2 1623
-4.2648240923881531e-03
5.9377008676528931e-01 5.6453680992126465e-01
3.9376249909400940e-01
<_>
0 1 1624 6.3251499086618423e-03 -1 -2 1625
-3.0753740575164557e-03
5.1821058988571167e-01 3.0074161291122437e-01
5.1964038610458374e-01
<_>
0 1 1626 -7.3622138006612659e-04 -1 -2 1627
3.0082479497650638e-05
3.6975800991058350e-01 4.3275931477546692e-01
5.7158088684082031e-01
<_>
0 1 1628 -3.8722730241715908e-03 -1 -2 1629
6.2879058532416821e-04
3.4737130999565125e-01 5.4382592439651489e-01
4.4539061188697815e-01
<_>
1 0 1630 1.3411579420790076e-03 -1 -2 1631
-8.3681922405958176e-03
6.5117138624191284e-01 1.4432950317859650e-01
4.8881998658180237e-01
<_>
1 0 1632 9.3305751215666533e-04 -1 -2 1633
-1.0746510233730078e-03
3.9511090517044067e-01 3.9102658629417419e-01
5.3495037555694580e-01
<_>
0 1 1634 -1.8610050901770592e-02 -1 -2 1635
1.3651419430971146e-03
1.2757439911365509e-01 5.0382888317108154e-01
6.9513040781021118e-01
<_>
0 1 1636 7.3744421824812889e-03 -1 -2 1637
8.4163323044776917e-03
5.2534431219100952e-01 5.0112438201904297e-01
7.3113328218460083e-01
<_>
0 1 1638 5.1413988694548607e-03 -1 -2 1639
4.5847031287848949e-03
4.9535360932350159e-01 2.5355559587478638e-01
6.4624428749084473e-01
<_>
1 0 1640 2.8565239161252975e-02 -1 -2 1641
4.3958800961263478e-04
2.3307220637798309e-01 4.7022441029548645e-01
5.5445492267608643e-01
<_>
1 0 1642 3.1459458172321320e-02 -1 -2 1643
5.6011630222201347e-03
3.3689688891172409e-02 4.7871211171150208e-01
6.3383519649505615e-01
<_>
0 1 1644 7.1835669223219156e-04 -1 -2 1645
-5.5303089320659637e-03
5.4314869642257690e-01 4.1058328747749329e-01
5.4039907455444336e-01
<_>
1 0 1646 1.4129279879853129e-03 -1 -2 1647
2.5530709535814822e-04
3.1055399775505066e-01 4.2544719576835632e-01
5.4471540451049805e-01
<_>
1 0 1648 3.1966410460881889e-04 -1 -2 1649
5.0411392003297806e-03
6.1183619499206543e-01 5.2900421619415283e-01
4.2247870564460754e-01
<_>
0 1 1650 7.7617880888283253e-03 -1 -2 1651
2.9374631121754646e-03
4.3153458833694458e-01 6.6292631626129150e-01
3.0289649963378906e-01
<_>
1 0 1652 -1.6497720498591661e-03 -1 -2 1653
-5.8834417723119259e-03
5.4918527603149414e-01 3.1885540485382080e-01
5.1842892169952393e-01
<_>
1 0 1654 8.7459187489002943e-04 -1 -2 1655
-1.5308779664337635e-02
3.3288308978080750e-01 3.9236080646514893e-01
5.2351391315460205e-01
<_>
1 0 1656 3.2292451709508896e-02 -1 -2 1657
-4.3842519517056644e-04
5.9776467084884644e-01 4.5416879653930664e-01
5.3694289922714233e-01
<_>
1 0 1658 1.5429529594257474e-03 -1 -2 1659
-2.4733028840273619e-03
6.3181412220001221e-01 3.4906330704689026e-01
4.7590249776840210e-01
<_>
1 0 1660 2.0994939841330051e-03 -1 -2 1661
-5.7541108690202236e-03
5.8871978521347046e-01 5.9613317251205444e-01
4.8419830203056335e-01
<_>
0 1 1662 -1.0233130306005478e-02 -1 -2 1663
2.2554509341716766e-01
1.7054040729999542e-01 4.7793799638748169e-01
9.7879663109779358e-02
<_>
1 0 1664 2.9666559770703316e-02 -1 -2 1665
-2.8518449980765581e-03
5.8222240209579468e-01 5.4596269130706787e-01
4.6100661158561707e-01
<_>
1 0 1666 9.7465328872203827e-04 -1 -2 1667
1.4044740055396687e-05
3.6703228950500488e-01 4.3023860454559326e-01
5.6917107105255127e-01
<_>
0 1 1668 -1.7579430714249611e-02 -1 -2 1669
-5.2381679415702820e-02
6.9173210859298706e-01 7.1100401878356934e-01
5.0601547956466675e-01
<_>
0 1 1670 -1.1242110282182693e-02 -1 -2 1671
-3.6728400737047195e-03
8.7691891193389893e-01 6.5191918611526489e-01
4.5460689067840576e-01
<_>
0 1 1672 3.5082760732620955e-03 -1 -2 1673
6.1679710634052753e-03
5.3298658132553101e-01 5.2204591035842896e-01
2.9535189270973206e-01
<_>
1 0 1674 -9.7009900491684675e-04 -1 -2 1675
-1.0957010090351105e-02
5.0486332178115845e-01 5.8373582363128662e-01
3.0200859904289246e-01
<_>
0 1 1676 -8.3272513002157211e-03 -1 -2 1677
2.9798380637657829e-05
3.1580638885498047e-01 4.3863898515701294e-01
5.4432111978530884e-01
<_>
1 0 1678 2.8244039276614785e-04 -1 -2 1679
-8.1364117795601487e-04
5.6253957748413086e-01 5.2811980247497559e-01
3.4014078974723816e-01
<_>
1 0 1680 1.8008040497079492e-03 -1 -2 1681
-6.9944779388606548e-03
3.4716591238975525e-01 4.4816970825195312e-01
5.3857702016830444e-01
<_>
0 1 1682 4.5625398342963308e-05 -1 -2 1683
-7.3189922841265798e-04
4.4925129413604736e-01 4.1673120856285095e-01
6.0211020708084106e-01
<_>
0 1 1684 -2.9980219551362097e-04 -1 -2 1685
-2.9060940505587496e-05
4.1484281420707703e-01 5.5920898914337158e-01
4.0732109546661377e-01
<_>
0 1 1686 -5.9742690064013004e-04 -1 -2 1687
1.4831830048933625e-04
6.0889142751693726e-01 5.2983051538467407e-01
3.7619501352310181e-01
<_>
1 0 1688 -2.9441029764711857e-03 -1 -2 1689
1.3741210103034973e-01
4.7160848975181580e-01 5.1013368368148804e-01
4.6746801584959030e-02
<_>
0 1 1690 -8.8414177298545837e-02 -1 -2 1691
7.0610277354717255e-02
1.1818689852952957e-01 5.1190632581710815e-01
7.7784419059753418e-01
<_>
0 1 1692 -7.7188978902995586e-03 -1 -2 1693
1.5115399844944477e-02
1.8741349875926971e-01 4.9800279736518860e-01
7.0058178901672363e-01
<_>
0 1 1694 1.0671879863366485e-03 -1 -2 1695
7.0487911580130458e-04
4.4822388887405396e-01 6.2657529115676880e-01
4.4026550650596619e-01
<_>
90
4.4251281738281250e+01
<_>
1 0 1696 -9.8690733313560486e-02 -1 -2 1697
6.2373418360948563e-02
3.9994749426841736e-01 5.2477848529815674e-01
8.1935757398605347e-01
<_>
0 1 1698 1.9496519817039371e-03 -1 -2 1699
-8.9139147894456983e-04
3.5298168659210205e-01 5.8527278900146484e-01
3.2459780573844910e-01
<_>
0 1 1700 -5.5150408297777176e-04 -1 -2 1701
-1.1721949558705091e-03
3.8928169012069702e-01 4.3350520730018616e-01
6.5206241607666016e-01
<_>
1 0 1702 -7.4480642797425389e-04 -1 -2 1703
-2.6264840271323919e-03
4.0411350131034851e-01 5.6249821186065674e-01
3.9675250649452209e-01
<_>
0 1 1704 -3.9712688885629177e-04 -1 -2 1705
3.5984949208796024e-03
3.8561120629310608e-01 5.9978890419006348e-01
4.2416140437126160e-01
<_>
1 0 1706 5.3080618381500244e-03 -1 -2 1707
9.6319877775385976e-04
6.6601687669754028e-01 4.4813790917396545e-01
5.5834877490997314e-01
<_>
0 1 1708 5.0776469288393855e-04 -1 -2 1709
3.6223160568624735e-03
3.5354590415954590e-01 3.4098070859909058e-01
5.4206877946853638e-01
<_>
0 1 1710 -6.2061410397291183e-02 -1 -2 1711
6.4387189922854304e-04
1.9340839982032776e-01 4.0836268663406372e-01
5.4902219772338867e-01
<_>
1 0 1712 2.6239909231662750e-02 -1 -2 1713
8.1940297968685627e-04
2.2857080399990082e-01 4.6486678719520569e-01
6.0173559188842773e-01
<_>
1 0 1714 2.3833119485061616e-04 -1 -2 1715
-1.5869759954512119e-03
3.5980388522148132e-01 4.2596510052680969e-01
5.4764348268508911e-01
<_>
0 1 1716 -6.7263417877256870e-03 -1 -2 1717
1.1006110347807407e-02
6.5072381496429443e-01 5.1494097709655762e-01
3.3629849553108215e-01
<_>
1 0 1718 7.1445819921791553e-03 -1 -2 1719
-4.7233798541128635e-03
2.6729300618171692e-01 5.6521821022033691e-01
4.2981448769569397e-01
<_>
1 0 1720 9.8437406122684479e-03 -1 -2 1721
1.5124640412977897e-05
1.1518859863281250e-01 4.3735980987548828e-01
5.6121289730072021e-01
<_>
0 1 1722 3.9908871054649353e-02 -1 -2 1723
5.3903679363429546e-03
5.2046489715576172e-01 4.8134678602218628e-01
6.3612091541290283e-01
<_>
0 1 1724 -3.9908871054649353e-02 -1 -2 1725
5.3903679363429546e-03
1.5068709850311279e-01 4.5816949009895325e-01
6.2002408504486084e-01
<_>
1 0 1726 6.7005190066993237e-03 -1 -2 1727
-1.2623789720237255e-02
3.4322351217269897e-01 3.0882269144058228e-01
5.2267378568649292e-01
<_>
1 0 1728 1.1806610040366650e-02 -1 -2 1729
-3.4257229417562485e-03
7.1879392862319946e-01 3.1208148598670959e-01
5.0658440589904785e-01
<_>
0 1 1730 3.9385299896821380e-04 -1 -2 1731
3.4388188272714615e-02
4.7545841336250305e-01 5.2616578340530396e-01
3.3501741290092468e-01
<_>
0 1 1732 -7.5009986758232117e-02 -1 -2 1733
4.9022492021322250e-04
1.7134809494018555e-01 4.7258019447326660e-01
5.9564691781997681e-01
<_>
0 1 1734 -8.5525289177894592e-03 -1 -2 1735
1.3135520566720515e-04
6.5582227706909180e-01 4.8354008793830872e-01
5.5869138240814209e-01
<_>
1 0 1736 4.7948658466339111e-03 -1 -2 1737
2.0124691072851419e-03
2.6457059383392334e-01 3.6579450964927673e-01
5.1247721910476685e-01
<_>
0 1 1738 -1.1785479635000229e-01 -1 -2 1739
1.5575019642710686e-03
2.3856540024280548e-01 5.4904741048812866e-01
4.2747479677200317e-01
<_>
0 1 1740 -1.5573759563267231e-02 -1 -2 1741
-2.1854790393263102e-03
6.9389009475708008e-01 3.6459881067276001e-01
5.0925260782241821e-01
<_>
0 1 1742 2.9272339306771755e-03 -1 -2 1743
6.4663668163120747e-03
4.6858081221580505e-01 4.9734100699424744e-01
7.7260971069335938e-01
<_>
0 1 1744 -7.6140360906720161e-03 -1 -2 1745
4.1512572206556797e-03
6.8774658441543579e-01 4.7885251045227051e-01
6.9216579198837280e-01
<_>
0 1 1746 2.7711640577763319e-03 -1 -2 1747
-1.2836109846830368e-02
5.4818397760391235e-01 3.8001629710197449e-01
5.2044928073883057e-01
<_>
0 1 1748 -2.4380050599575043e-03 -1 -2 1749
2.1713329479098320e-03
2.5824350118637085e-01 4.9611631035804749e-01
3.2152029871940613e-01
<_>
1 0 1750 6.2800728483125567e-04 -1 -2 1751
-9.7982389852404594e-03
5.4604238271713257e-01 6.0465437173843384e-01
4.9399220943450928e-01
<_>
1 0 1752 7.3543828912079334e-03 -1 -2 1753
-1.4665040187537670e-02
5.2910941839218140e-01 5.4461228847503662e-01
3.5673621296882629e-01
<_>
0 1 1754 3.0244510620832443e-02 -1 -2 1755
-5.6660208851099014e-02
5.5183291435241699e-01 6.9309788942337036e-01
5.0933879613876343e-01
<_>
0 1 1756 -5.6967479176819324e-03 -1 -2 1757
3.0806770548224449e-02
3.2015261054039001e-01 4.9892461299896240e-01
2.2770540416240692e-01
<_>
0 1 1758 2.2748769260942936e-03 -1 -2 1759
2.0436900667846203e-03
4.8109310865402222e-01 5.2838671207427979e-01
3.2559248805046082e-01
<_>
0 1 1760 -8.6277956143021584e-03 -1 -2 1761
6.5113382879644632e-04
6.2665361166000366e-01 5.0971370935440063e-01
3.1919100880622864e-01
<_>
0 1 1762 8.8188261725008488e-04 -1 -2 1763
-1.4594909735023975e-02
4.5495858788490295e-01 2.6450389623641968e-01
5.1538681983947754e-01
<_>
0 1 1764 -1.2304580304771662e-03 -1 -2 1765
-2.1867299801670015e-04
6.1975848674774170e-01 5.4691988229751587e-01
4.2068558931350708e-01
<_>
0 1 1766 -1.0909959673881531e-03 -1 -2 1767
3.5210378700867295e-04
4.1407600045204163e-01 5.4766088724136353e-01
4.1550210118293762e-01
<_>
0 1 1768 -7.2563779540359974e-03 -1 -2 1769
1.4701850013807416e-03
7.1604692935943604e-01 5.2408081293106079e-01
3.7296628952026367e-01
<_>
0 1 1770 1.1472719779703766e-04 -1 -2 1771
3.0506469774991274e-03
4.0337988734245300e-01 5.2639859914779663e-01
3.5600930452346802e-01
<_>
0 1 1772 2.6269949739798903e-04 -1 -2 1773
-3.6365550477057695e-03
4.5697999000549316e-01 3.0425709486007690e-01
5.8682537078857422e-01
<_>
1 0 1774 -8.4893293678760529e-03 -1 -2 1775
5.8107408694922924e-03
4.9141570925712585e-01 4.9185299873352051e-01
6.2669628858566284e-01
<_>
1 0 1776 7.5583951547741890e-04 -1 -2 1777
-2.2017690353095531e-03
5.6332361698150635e-01 5.5539160966873169e-01
3.8276460766792297e-01
<_>
0 1 1778 2.7908938936889172e-03 -1 -2 1779
-1.8228569533675909e-03
5.4986977577209473e-01 4.3822830915451050e-01
5.4240328073501587e-01
<_>
0 1 1780 -7.2495508939027786e-03 -1 -2 1781
-6.8744522286579013e-04
2.8881219029426575e-01 3.4726551175117493e-01
5.0763708353042603e-01
<_>
0 1 1782 2.5174440816044807e-03 -1 -2 1783
-1.0151379741728306e-02
4.6612051129341125e-01 3.7447750568389893e-01
5.2940011024475098e-01
<_>
1 0 1784 -4.1399952024221420e-03 -1 -2 1785
-4.7078551724553108e-03
4.6604850888252258e-01 4.1750618815422058e-01
6.9163060188293457e-01
<_>
1 0 1786 4.1981041431427002e-02 -1 -2 1787
-1.4272999949753284e-02
2.0182150602340698e-01 7.5111979246139526e-01
5.0320839881896973e-01
<_>
1 0 1788 4.0869521908462048e-03 -1 -2 1789
1.7606799956411123e-03
2.5045138597488403e-01 3.3014011383056641e-01
5.2183371782302856e-01
<_>
0 1 1790 1.2550549581646919e-04 -1 -2 1791
-2.9503209516406059e-03
4.6144428849220276e-01 4.6199500560760498e-01
5.2470302581787109e-01
<_>
0 1 1792 -1.1312420247122645e-03 -1 -2 1793
-1.6983180539682508e-03
6.3143682479858398e-01 3.4013068675994873e-01
5.0555270910263062e-01
<_>
1 0 1794 -1.1457820422947407e-02 -1 -2 1795
-8.4962565451860428e-03
4.9399960041046143e-01 2.9654508829116821e-01
5.1943677663803101e-01
<_>
1 0 1796 1.1919089592993259e-02 -1 -2 1797
6.4416420646011829e-03
7.8869980573654175e-01 5.1069867610931396e-01
2.9671460390090942e-01
<_>
0 1 1798 -8.7857811013236642e-04 -1 -2 1799
-2.0312711130827665e-03
5.7143712043762207e-01 4.4812008738517761e-01
5.3849118947982788e-01
<_>
0 1 1800 -1.5262430533766747e-03 -1 -2 1801
4.2860880494117737e-03
6.1935687065124512e-01 4.3398851156234741e-01
7.6972991228103638e-01
<_>
1 0 1802 3.5010920837521553e-03 -1 -2 1803
1.2587670236825943e-02
3.1713891029357910e-01 5.2466988563537598e-01
4.2412081360816956e-01
<_>
0 1 1804 2.6207490009255707e-04 -1 -2 1805
4.4701730075757951e-05
4.2318999767303467e-01 4.1741389036178589e-01
5.9196037054061890e-01
<_>
0 1 1806 7.8084698179736733e-04 -1 -2 1807
8.8851212058216333e-04
4.2773890495300293e-01 3.7201610207557678e-01
5.2268189191818237e-01
<_>
0 1 1808 2.3369069676846266e-03 -1 -2 1809
1.6688359901309013e-03
5.4780668020248413e-01 3.6286789178848267e-01
6.1500048637390137e-01
<_>
0 1 1810 3.0844469438306987e-04 -1 -2 1811
3.4617560449987650e-03
4.7470751404762268e-01 4.5801380276679993e-01
5.5856817960739136e-01
<_>
0 1 1812 1.8961310386657715e-02 -1 -2 1813
1.7347310483455658e-01
5.2988010644912720e-01 3.6983850598335266e-01
8.4986197948455811e-01
<_>
1 0 1814 2.0020549709443003e-04 -1 -2 1815
1.0967060225084424e-03
5.5656617879867554e-01 4.7957131266593933e-01
6.2862598896026611e-01
<_>
0 1 1816 1.5107099898159504e-04 -1 -2 1817
-3.4463501069694757e-03
4.0524059534072876e-01 6.1730152368545532e-01
4.4142639636993408e-01
<_>
1 0 1818 8.5176620632410049e-03 -1 -2 1819
-3.5812109708786011e-02
3.5705709457397461e-01 3.1513288617134094e-01
5.2527028322219849e-01
<_>
0 1 1820 -2.1155400201678276e-02 -1 -2 1821
8.9890940580517054e-04
6.1247211694717407e-01 5.1699757575988770e-01
3.5962718725204468e-01
<_>
1 0 1822 -1.5613760333508253e-03 -1 -2 1823
6.7120860330760479e-04
4.9149879813194275e-01 4.5462110638618469e-01
5.3958117961883545e-01
<_>
0 1 1824 -2.1597029641270638e-02 -1 -2 1825
-2.4947229772806168e-02
1.9031339883804321e-01 6.9740772247314453e-01
4.9677160382270813e-01
<_>
0 1 1826 1.8725979607552290e-03 -1 -2 1827
6.3912719488143921e-03
4.7489479184150696e-01 5.1801782846450806e-01
2.9243218898773193e-01
<_>
0 1 1828 -9.1552399098873138e-03 -1 -2 1829
2.1715660113841295e-03
7.6658701896667480e-01 5.2155512571334839e-01
3.3657190203666687e-01
<_>
1 0 1830 1.2330369791015983e-03 -1 -2 1831
-4.0785901364870369e-04
6.2609577178955078e-01 4.5335099101066589e-01
5.3864890336990356e-01
<_>
0 1 1832 4.6437609125860035e-04 -1 -2 1833
-1.1600199650274590e-04
4.1034960746765137e-01 5.8303910493850708e-01
4.3041059374809265e-01
<_>
0 1 1834 -1.2718720361590385e-02 -1 -2 1835
8.9431880041956902e-05
2.1325829625129700e-01 4.8728910088539124e-01
5.4589152336120605e-01
<_>
0 1 1836 -3.3913689549081028e-04 -1 -2 1837
-1.8026340752840042e-02
3.9743649959564209e-01 7.5685507059097290e-01
5.0456118583679199e-01
<_>
1 0 1838 6.9179181009531021e-03 -1 -2 1839
-1.1839679791592062e-04
3.9662998914718628e-01 4.1980829834938049e-01
5.4358041286468506e-01
<_>
0 1 1840 -3.9474181830883026e-03 -1 -2 1841
6.0050919273635373e-05
6.3694578409194946e-01 5.2695667743682861e-01
3.8122430443763733e-01
<_>
1 0 1842 9.1423643752932549e-03 -1 -2 1843
2.1305440168362111e-04
4.1567629575729370e-01 3.5235330462455750e-01
5.3494542837142944e-01
<_>
1 0 1844 -2.0855850016232580e-04 -1 -2 1845
1.3130389852449298e-03
4.4033220410346985e-01 6.0581612586975098e-01
4.4682189822196960e-01
<_>
1 0 1846 -2.9134768992662430e-03 -1 -2 1847
2.9645769391208887e-03
4.8257058858871460e-01 4.8359981179237366e-01
6.0392779111862183e-01
<_>
1 0 1848 1.7772549763321877e-03 -1 -2 1849
-7.7136349864304066e-03
6.8718272447586060e-01 2.8422209620475769e-01
5.1454281806945801e-01
<_>
1 0 1850 5.1027478184551001e-04 -1 -2 1851
1.7460630042478442e-03
6.0244262218475342e-01 4.7566100955009460e-01
5.7211542129516602e-01
<_>
1 0 1852 3.8068278809078038e-04 -1 -2 1853
2.8228890150785446e-03
4.9310690164566040e-01 3.3116981387138367e-01
6.2275981903076172e-01
<_>
1 0 1854 -5.3000478073954582e-03 -1 -2 1855
4.4951299059903249e-05
5.2320927381515503e-01 3.9952319860458374e-01
5.3147977590560913e-01
<_>
0 1 1856 3.2752458937466145e-03 -1 -2 1857
-2.8162579983472824e-03
4.4816198945045471e-01 3.9079719781875610e-01
6.6716408729553223e-01
<_>
0 1 1858 1.4112279750406742e-03 -1 -2 1859
8.3062034100294113e-03
5.3570109605789185e-01 4.7709658741950989e-01
5.5700999498367310e-01
<_>
0 1 1860 2.2164839319884777e-03 -1 -2 1861
-4.9868631176650524e-03
4.9471241235733032e-01 5.2413070201873779e-01
2.5126549601554871e-01
<_>
1 0 1862 -3.6664260551333427e-03 -1 -2 1863
-1.0581229813396931e-02
4.6195539832115173e-01 6.3017189502716064e-01
4.9730318784713745e-01
<_>
1 0 1864 7.3366491124033928e-03 -1 -2 1865
-3.9318940252996981e-04
2.8709700703620911e-01 4.2528051137924194e-01
5.5792468786239624e-01
<_>
0 1 1866 -8.1375334411859512e-03 -1 -2 1867
2.4809150490909815e-03
5.7473158836364746e-01 5.2033740282058716e-01
3.9035668969154358e-01
<_>
1 0 1868 8.8749779388308525e-04 -1 -2 1869
-4.2194919660687447e-04
5.5343210697174072e-01 5.3380441665649414e-01
3.9258408546447754e-01
<_>
0 1 1870 -7.9790111631155014e-03 -1 -2 1871
1.1439629597589374e-03
4.1443160176277161e-01 4.7013729810714722e-01
5.2817362546920776e-01
<_>
1 0 1872 7.5542130507528782e-03 -1 -2 1873
1.0288399644196033e-03
2.5272560119628906e-01 5.6051462888717651e-01
4.2978560924530029e-01
<_>
1 0 1874 -1.7234670231118798e-03 -1 -2 1875
5.7586699724197388e-01
4.8396828770637512e-01 5.1105028390884399e-01
8.0489329993724823e-02
<_>
109
5.3755569458007812e+01
<_>
0 1 1876 6.6640521399676800e-03 -1 -2 1877
8.9905522763729095e-03
3.8289201259613037e-01 4.8584291338920593e-01
7.3549592494964600e-01
<_>
1 0 1878 5.7154200039803982e-03 -1 -2 1879
1.1257929727435112e-03
6.7232239246368408e-01 4.4295778870582581e-01
6.0707777738571167e-01
<_>
1 0 1880 -9.1789010912179947e-04 -1 -2 1881
-1.0492859873920679e-03
3.0763450264930725e-01 5.5936437845230103e-01
3.6510229110717773e-01
<_>
0 1 1882 3.5453929740469903e-05 -1 -2 1883
2.9015709878876805e-04
4.2779681086540222e-01 4.5835450291633606e-01
5.2846831083297729e-01
<_>
1 0 1884 1.6071660502348095e-04 -1 -2 1885
-5.2961107576265931e-04
3.7981921434402466e-01 3.8504371047019958e-01
5.9396880865097046e-01
<_>
0 1 1886 2.6682569296099246e-04 -1 -2 1887
-1.3492540165316314e-04
4.1230249404907227e-01 5.7605999708175659e-01
4.2376458644866943e-01
<_>
0 1 1888 -1.0841679759323597e-02 -1 -2 1889
1.2077829800546169e-02
3.9299210906028748e-01 5.7619231939315796e-01
2.7804449200630188e-01
<_>
0 1 1890 2.2128869313746691e-03 -1 -2 1891
-1.5266190283000469e-02
4.7945070266723633e-01 7.4055880308151245e-02
5.1535779237747192e-01
<_>
1 0 1892 6.7929533543065190e-05 -1 -2 1893
1.7633590323384851e-04
5.8587378263473511e-01 3.5676109790802002e-01
5.5989629030227661e-01
<_>
1 0 1894 8.1311381654813886e-04 -1 -2 1895
3.2630451023578644e-03
5.3468507528305054e-01 4.7825369238853455e-01
5.4567539691925049e-01
<_>
0 1 1896 -3.9503918960690498e-03 -1 -2 1897
-3.9864578866399825e-04
2.8318119049072266e-01 5.4852157831192017e-01
4.1596978902816772e-01
<_>
0 1 1898 -1.1432520113885403e-02 -1 -2 1899
5.3339172154664993e-03
5.6391012668609619e-01 4.5969840884208679e-01
5.9312427043914795e-01
<_>
1 0 1900 8.3193257451057434e-03 -1 -2 1901
-4.2479918920435011e-04
3.2306200265884399e-01 3.7952938675880432e-01
5.4086112976074219e-01
<_>
0 1 1902 -1.1189430207014084e-01 -1 -2 1903
-7.5553781352937222e-03
1.1322979629039764e-01 6.3393700122833252e-01
4.8387709259986877e-01
<_>
0 1 1904 -7.0337029173970222e-03 -1 -2 1905
-1.4833680354058743e-02
5.6652551889419556e-01 6.7514181137084961e-01
4.1409450769424438e-01
<_>
1 0 1906 8.7506724521517754e-03 -1 -2 1907
1.6645010327920318e-03
3.5612589120864868e-01 5.3472799062728882e-01
3.6497798562049866e-01
<_>
1 0 1908 9.4900820404291153e-03 -1 -2 1909
1.1133110383525491e-03
2.7546560764312744e-01 4.2259928584098816e-01
5.6291788816452026e-01
<_>
0 1 1910 9.4940755516290665e-03 -1 -2 1911
-1.5396620146930218e-03
4.9060368537902832e-01 4.0070518851280212e-01
5.3807091712951660e-01
<_>
1 0 1912 1.3434959948062897e-01 -1 -2 1913
-9.4940755516290665e-03
2.2146719694137573e-01 7.3531562089920044e-01
5.0050330162048340e-01
<_>
1 0 1914 2.0011790096759796e-02 -1 -2 1915
-1.8875009845942259e-03
3.3279061317443848e-01 3.9152890443801880e-01
5.4018497467041016e-01
<_>
1 0 1916 7.1842782199382782e-03 -1 -2 1917
1.6976969782263041e-03
7.1766048669815063e-01 4.5269781351089478e-01
6.0769128799438477e-01
<_>
1 0 1918 4.9219978973269463e-03 -1 -2 1919
1.1803199537098408e-02
2.5698339939117432e-01 4.9996379017829895e-01
5.9582281112670898e-01
<_>
0 1 1920 -9.7703449428081512e-03 -1 -2 1921
2.1174899302423000e-03
3.4590938687324524e-01 4.5151269435882568e-01
5.8297157287597656e-01
<_>
0 1 1922 9.4801411032676697e-03 -1 -2 1923
-2.6078789960592985e-03
4.8073920607566833e-01 3.4622168540954590e-01
5.2015948295593262e-01
<_>
0 1 1924 -5.7252747938036919e-03 -1 -2 1925
-8.2325618714094162e-03
6.5998530387878418e-01 2.8218281269073486e-01
5.1252847909927368e-01
<_>
0 1 1926 8.9571950957179070e-04 -1 -2 1927
-1.5021569561213255e-04
4.8838189244270325e-01 4.8299181461334229e-01
5.4287171363830566e-01
<_>
0 1 1928 4.8489659093320370e-04 -1 -2 1929
-9.6192650496959686e-02
4.4345989823341370e-01 2.2566360235214233e-01
5.9562277793884277e-01
<_>
0 1 1930 -1.1053519556298852e-03 -1 -2 1931
-1.0215040296316147e-01
4.5272240042686462e-01 2.8443491458892822e-01
5.1864528656005859e-01
<_>
1 0 1932 3.0147889629006386e-03 -1 -2 1933
7.6131648384034634e-03
3.8089990615844727e-01 5.7186990976333618e-01
4.2625638842582703e-01
<_>
1 0 1934 1.5197630273178220e-03 -1 -2 1935
-1.4197279699146748e-02
5.9427189826965332e-01 7.7311038970947266e-01
4.9976539611816406e-01
<_>
0 1 1936 -1.3818879611790180e-02 -1 -2 1937
-5.0701329018920660e-04
6.6811382770538330e-01 3.3056080341339111e-01
4.7499749064445496e-01
<_>
0 1 1938 -9.3537531793117523e-03 -1 -2 1939
-9.4771059229969978e-03
2.8609329462051392e-01 6.1888831853866577e-01
4.8421001434326172e-01
<_>
1 0 1940 1.6923650400713086e-03 -1 -2 1941
5.8652542065829039e-04
6.0702490806579590e-01 3.7826898694038391e-01
5.3681969642639160e-01
<_>
0 1 1942 -2.5826620403677225e-03 -1 -2 1943
-2.7307639829814434e-03
3.6902099847793579e-01 3.8571149110794067e-01
5.3181087970733643e-01
<_>
1 0 1944 2.1871570497751236e-02 -1 -2 1945
-1.5010299648565706e-05
2.3270089924335480e-01 5.5607229471206665e-01
4.3014100193977356e-01
<_>
1 0 1946 5.3583700209856033e-03 -1 -2 1947
5.0057549960911274e-03
6.7676377296447754e-01 5.1949042081832886e-01
3.6128538846969604e-01
<_>
0 1 1948 -1.9030070398002863e-03 -1 -2 1949
-7.8506693243980408e-03
3.2378450036048889e-01 1.1948519945144653e-01
4.9917238950729370e-01
<_>
1 0 1950 -2.7093670796602964e-03 -1 -2 1951
1.4138079714030027e-03
4.8549601435661316e-01 4.8723229765892029e-01
5.9035778045654297e-01
<_>
1 0 1952 9.0300198644399643e-03 -1 -2 1953
-9.7925681620836258e-04
6.5473157167434692e-01 5.8492732048034668e-01
4.5542308688163757e-01
<_>
1 0 1954 1.3984439428895712e-03 -1 -2 1955
8.3372107474133372e-04
4.0646260976791382e-01 5.3995430469512939e-01
4.1528099775314331e-01
<_>
1 0 1956 1.0551059618592262e-02 -1 -2 1957
8.8344102550763637e-05
1.7966809868812561e-01 4.2518630623817444e-01
5.4135227203369141e-01
<_>
1 0 1958 -4.1022308170795441e-02 -1 -2 1959
7.5065628625452518e-03
5.2281248569488525e-01 4.8537430167198181e-01
6.0934442281723022e-01
<_>
1 0 1960 4.1022308170795441e-02 -1 -2 1961
-5.3961377125233412e-04
2.2050240635871887e-01 5.6927317380905151e-01
4.4687569141387939e-01
<_>
0 1 1962 -6.8696036934852600e-02 -1 -2 1963
-1.8447940237820148e-03
1.4833140373229980e-01 6.2112838029861450e-01
4.9666011333465576e-01
<_>
0 1 1964 -6.0959919355809689e-03 -1 -2 1965
-4.2068301700055599e-03
2.2946719825267792e-01 6.4070910215377808e-01
4.7485628724098206e-01
<_>
1 0 1966 -7.1332789957523346e-04 -1 -2 1967
1.1756779998540878e-01
5.3549361228942871e-01 5.1369780302047729e-01
1.0595739819109440e-02
<_>
0 1 1968 5.9354289987822995e-05 -1 -2 1969
-6.3173691742122173e-03
3.7118038535118103e-01 1.7120739817619324e-01
5.0617581605911255e-01
<_>
1 0 1970 1.4941499568521976e-02 -1 -2 1971
-2.0789399277418852e-03
6.7291188240051270e-01 4.4106459617614746e-01
5.4440277814865112e-01
<_>
0 1 1972 -7.0736219640821218e-04 -1 -2 1973
-3.1247111037373543e-03
5.5689108371734619e-01 5.0238692760467529e-01
3.5624051094055176e-01
<_>
1 0 1974 -7.8919378574937582e-04 -1 -2 1975
1.0179580189287663e-02
5.4567861557006836e-01 5.5451387166976929e-01
4.6223109960556030e-01
<_>
1 0 1976 -2.7506109327077866e-03 -1 -2 1977
1.0601329617202282e-02
4.9425360560417175e-01 2.9612338542938232e-01
5.9643387794494629e-01
<_>
0 1 1978 5.1466780714690685e-03 -1 -2 1979
7.6321147382259369e-02
5.4952287673950195e-01 5.1739591360092163e-01
2.9402169585227966e-01
<_>
0 1 1980 -1.5027689514681697e-03 -1 -2 1981
1.2266670353710651e-02
3.1062999367713928e-01 4.6511501073837280e-01
6.8466138839721680e-01
<_>
1 0 1982 -3.1118579208850861e-02 -1 -2 1983
2.8905589133501053e-02
5.2260571718215942e-01 5.1822441816329956e-01
2.7054280042648315e-01
<_>
1 0 1984 4.7598380595445633e-02 -1 -2 1985
3.0808549374341965e-02
1.1095120012760162e-01 4.9386250972747803e-01
1.4041109383106232e-01
<_>
1 0 1986 -2.1277810446918011e-04 -1 -2 1987
7.8969962894916534e-02
4.3923568725585938e-01 5.2165520191192627e-01
2.2941139340400696e-01
<_>
0 1 1988 -1.0257950052618980e-02 -1 -2 1989
1.2604889925569296e-03
6.1766529083251953e-01 5.2362227439880371e-01
3.3289659023284912e-01
<_>
1 0 1990 -3.3490460366010666e-02 -1 -2 1991
-5.9202767442911863e-04
4.8661869764328003e-01 4.1164070367813110e-01
5.3956401348114014e-01
<_>
1 0 1992 3.0320750738610514e-05 -1 -2 1993
-5.4369680583477020e-04
5.6107360124588013e-01 5.6213891506195068e-01
3.4612038731575012e-01
<_>
1 0 1994 -3.3490460366010666e-02 -1 -2 1995
-5.9202767442911863e-04
4.8967620730400085e-01 4.3054041266441345e-01
5.3407138586044312e-01
<_>
0 1 1996 2.0550889894366264e-03 -1 -2 1997
-4.4353571720421314e-03
5.5449998378753662e-01 6.0385400056838989e-01
3.7465929985046387e-01
<_>
1 0 1998 -8.4170423448085785e-02 -1 -2 1999
6.7419027909636497e-03
5.0073480606079102e-01 5.2980971336364746e-01
4.7161450982093811e-01
<_>
1 0 2000 1.0278150439262390e-02 -1 -2 2001
5.8800862170755863e-03
6.2693750858306885e-01 5.1548278331756592e-01
3.8130408525466919e-01
<_>
1 0 2002 -6.9679190346505493e-06 -1 -2 2003
8.2419527461752295e-04
4.4402399659156799e-01 4.6975341439247131e-01
5.4855042695999146e-01
<_>
0 1 2004 -5.5268318392336369e-03 -1 -2 2005
9.6128671430051327e-04
5.5136048793792725e-01 3.6186391115188599e-01
5.8384567499160767e-01
<_>
1 0 2006 2.4810510221868753e-03 -1 -2 2007
-1.0480589699000120e-03
2.5232228636741638e-01 4.1172578930854797e-01
5.3929960727691650e-01
<_>
0 1 2008 -6.1287907883524895e-03 -1 -2 2009
1.1682329932227731e-04
6.7263299226760864e-01 5.0411927700042725e-01
3.6077290773391724e-01
<_>
0 1 2010 -3.9909478276968002e-02 -1 -2 2011
1.5859459526836872e-03
1.5637390315532684e-01 4.8919808864593506e-01
5.7798451185226440e-01
<_>
0 1 2012 -2.2690229117870331e-02 -1 -2 2013
2.0916070789098740e-03
2.1868790686130524e-01 4.7715771198272705e-01
6.0992312431335449e-01
<_>
0 1 2014 -2.4715419858694077e-02 -1 -2 2015
-1.3419450260698795e-02
3.4639969468116760e-01 3.6306929588317871e-01
5.2521961927413940e-01
<_>
0 1 2016 -6.0629472136497498e-03 -1 -2 2017
-2.0921030081808567e-03
6.6663217544555664e-01 3.3995470404624939e-01
5.0356978178024292e-01
<_>
0 1 2018 2.5961859151721001e-02 -1 -2 2019
1.7908669542521238e-04
5.0368028879165649e-01 5.4185307025909424e-01
4.3189769983291626e-01
<_>
0 1 2020 -3.1546850223094225e-03 -1 -2 2021
-1.1397759662941098e-03
7.2210252285003662e-01 3.3209729194641113e-01
5.0244337320327759e-01
<_>
0 1 2022 -4.7840211540460587e-02 -1 -2 2023
4.1577088995836675e-04
1.9387650489807129e-01 4.8021888732910156e-01
5.7307147979736328e-01
<_>
0 1 2024 -4.4247039477340877e-04 -1 -2 2025
1.4479350065812469e-03
4.2625150084495544e-01 5.7191711664199829e-01
4.0641531348228455e-01
<_>
0 1 2026 1.5701510012149811e-02 -1 -2 2027
2.7805729769170284e-04
4.9957260489463806e-01 5.2892869710922241e-01
4.5817288756370544e-01
<_>
0 1 2028 -2.9010509606450796e-03 -1 -2 2029
2.0830519497394562e-04
6.0121482610702515e-01 5.0579768419265747e-01
3.5994321107864380e-01
<_>
1 0 2030 -5.1530029624700546e-02 -1 -2 2031
1.7163449956569821e-04
4.9917969107627869e-01 4.6754699945449829e-01
5.3747731447219849e-01
<_>
1 0 2032 2.3614279925823212e-02 -1 -2 2033
-5.6427798699587584e-04
6.5864789485931396e-01 3.8532960414886475e-01
5.1960402727127075e-01
<_>
1 0 2034 6.6903959959745407e-03 -1 -2 2035
-4.8789530992507935e-03
6.0042357444763184e-01 3.2932278513908386e-01
5.2452367544174194e-01
<_>
0 1 2036 -6.8537332117557526e-03 -1 -2 2037
9.9893810693174601e-04
2.5659140944480896e-01 4.6154940128326416e-01
5.9424322843551636e-01
<_>
0 1 2038 -1.3354700058698654e-04 -1 -2 2039
1.0165109997615218e-03
5.4873758554458618e-01 4.5783591270446777e-01
5.4269278049468994e-01
<_>
1 0 2040 9.1216771397739649e-04 -1 -2 2041
1.0080259526148438e-03
3.9394611120223999e-01 4.0497899055480957e-01
5.5207037925720215e-01
<_>
1 0 2042 -1.3102490629535168e-04 -1 -2 2043
5.5228749988600612e-04
4.8790889978408813e-01 4.8449438810348511e-01
5.5128258466720581e-01
<_>
1 0 2044 -1.2130969844292849e-04 -1 -2 2045
-1.5112989785848185e-05
4.3679711222648621e-01 6.4259552955627441e-01
4.8818269371986389e-01
<_>
1 0 2046 -4.0125829400494695e-04 -1 -2 2047
-6.5766851184889674e-04
5.3720992803573608e-01 5.8345532417297363e-01
4.8690780997276306e-01
<_>
1 0 2048 6.2220421386882663e-04 -1 -2 2049
1.4663359615951777e-03
3.8246369361877441e-01 4.8134881258010864e-01
6.9667392969131470e-01
<_>
0 1 2050 -4.9547709524631500e-02 -1 -2 2051
1.3017569435760379e-03
5.3927659988403320e-02 5.3374558687210083e-01
4.1607481241226196e-01
<_>
0 1 2052 -4.4914530590176582e-03 -1 -2 2053
1.6592369647696614e-03
5.9974372386932373e-01 3.7271851301193237e-01
5.1156342029571533e-01
<_>
0 1 2054 6.4695458859205246e-03 -1 -2 2055
4.9810269847512245e-03
5.2520352602005005e-01 5.2567178010940552e-01
3.9344060420989990e-01
<_>
0 1 2056 -3.8536980748176575e-02 -1 -2 2057
-2.8275650739669800e-01
2.0619249343872070e-01 6.1883211135864258e-02
4.9250578880310059e-01
<_>
0 1 2058 -9.0301828458905220e-03 -1 -2 2059
-4.3866269290447235e-02
3.1575900316238403e-01 2.0336820185184479e-01
5.1647698879241943e-01
<_>
0 1 2060 -4.5701069757342339e-03 -1 -2 2061
-2.3362410720437765e-03
6.6111832857131958e-01 2.8077891469001770e-01
4.9628761410713196e-01
<_>
0 1 2062 5.3960331715643406e-03 -1 -2 2063
-2.6297608856111765e-03
5.1463878154754639e-01 6.2844878435134888e-01
4.9555888772010803e-01
<_>
0 1 2064 -3.8577478844672441e-03 -1 -2 2065
1.3963800156489015e-03
1.4867480099201202e-01 4.7013381123542786e-01
6.3209718465805054e-01
<_>
1 0 2066 -8.8699469342827797e-03 -1 -2 2067
-7.0626288652420044e-04
5.2868181467056274e-01 4.6483701467514038e-01
5.3332102298736572e-01
<_>
0 1 2068 4.2645810171961784e-03 -1 -2 2069
6.1572100967168808e-02
5.0848782062530518e-01 3.6296251416206360e-01
8.7571567296981812e-01
<_>
1 0 2070 -4.5381980016827583e-03 -1 -2 2071
-4.0877899155020714e-03
4.8566961288452148e-01 4.5841160416603088e-01
5.4202407598495483e-01
<_>
1 0 2072 6.4308601431548595e-03 -1 -2 2073
7.0455260574817657e-03
2.7073028683662415e-01 5.0574868917465210e-01
7.0265239477157593e-01
<_>
1 0 2074 -2.3246440105140209e-03 -1 -2 2075
6.0276601288933307e-05
4.8272788524627686e-01 4.2472490668296814e-01
5.5087631940841675e-01
<_>
1 0 2076 1.8084559589624405e-02 -1 -2 2077
8.4693520329892635e-04
8.1048011779785156e-01 5.1546192169189453e-01
3.5143798589706421e-01
<_>
1 0 2078 -2.6931039988994598e-02 -1 -2 2079
-4.2346641421318054e-03
4.8868888616561890e-01 4.6223780512809753e-01
5.3824782371520996e-01
<_>
1 0 2080 2.6947110891342163e-02 -1 -2 2081
4.6446882188320160e-03
6.3665962219238281e-01 5.3685069084167480e-01
3.7654298543930054e-01
<_>
0 1 2082 -6.9577661342918873e-03 -1 -2 2083
8.7609712500125170e-04
4.2346870899200439e-01 4.6724060177803040e-01
5.3506839275360107e-01
<_>
1 0 2084 1.6103329835459590e-03 -1 -2 2085
-1.2848590267822146e-03
5.7327628135681152e-01 5.4817992448806763e-01
3.7845930457115173e-01
<_>
0 1 2086 1.0243539698421955e-02 -1 -2 2087
2.6889349101111293e-04
5.1559072732925415e-01 5.3531897068023682e-01
4.3871539831161499e-01
<_>
0 1 2088 3.7903659977018833e-03 -1 -2 2089
-2.9369680210947990e-02
5.0320029258728027e-01 5.8735388517379761e-01
2.2154450416564941e-01
<_>
1 0 2090 6.0743088833987713e-03 -1 -2 2091
-1.2710720300674438e-02
5.4170298576354980e-01 6.0565119981765747e-01
4.9851819872856140e-01
<_>
0 1 2092 -5.9445449151098728e-03 -1 -2 2093
-2.8927479870617390e-03
3.3520698547363281e-01 6.9292408227920532e-01
4.7782200574874878e-01
<_>
<_>
2 7 16 4 -1.
<_>
2 9 16 2 2.
<_>
<_>
8 4 3 14 -1.
<_>
8 11 3 7 2.
<_>
<_>
13 6 1 6 -1.
<_>
13 9 1 3 2.
<_>
<_>
4 2 12 8 -1.
<_>
8 2 4 8 3.
<_>
<_>
6 3 1 9 -1.
<_>
6 6 1 3 3.
<_>
<_>
3 7 14 9 -1.
<_>
3 10 14 3 3.
<_>
<_>
4 7 4 4 -1.
<_>
4 9 4 2 2.
<_>
<_>
9 4 2 16 -1.
<_>
9 12 2 8 2.
<_>
<_>
1 1 18 5 -1.
<_>
7 1 6 5 3.
<_>
<_>
4 5 13 8 -1.
<_>
4 9 13 4 2.
<_>
<_>
1 7 16 9 -1.
<_>
1 10 16 3 3.
<_>
<_>
2 0 15 4 -1.
<_>
2 2 15 2 2.
<_>
<_>
7 5 6 4 -1.
<_>
9 5 2 4 3.
<_>
<_>
6 3 8 9 -1.
<_>
6 6 8 3 3.
<_>
<_>
8 12 3 8 -1.
<_>
8 16 3 4 2.
<_>
<_>
3 16 2 2 -1.
<_>
3 17 2 1 2.
<_>
<_>
14 1 6 12 -1.
<_>
14 1 3 12 2.
<_>
<_>
4 4 12 6 -1.
<_>
8 4 4 6 3.
<_>
<_>
0 2 6 15 -1.
<_>
3 2 3 15 2.
<_>
<_>
5 4 9 6 -1.
<_>
5 6 9 2 3.
<_>
<_>
13 11 6 3 -1.
<_>
13 12 6 1 3.
<_>
<_>
12 12 6 4 -1.
<_>
12 14 6 2 2.
<_>
<_>
1 11 6 3 -1.
<_>
1 12 6 1 3.
<_>
<_>
2 5 5 8 -1.
<_>
2 9 5 4 2.
<_>
<_>
5 4 10 4 -1.
<_>
5 6 10 2 2.
<_>
<_>
2 4 16 12 -1.
<_>
2 8 16 4 3.
<_>
<_>
4 5 12 6 -1.
<_>
8 5 4 6 3.
<_>
<_>
13 7 2 9 -1.
<_>
13 10 2 3 3.
<_>
<_>
5 7 2 9 -1.
<_>
5 10 2 3 3.
<_>
<_>
7 1 6 8 -1.
<_>
9 1 2 8 3.
<_>
<_>
12 0 4 12 -1.
<_>
14 0 2 6 2.
<_>
12 6 2 6 2.
<_>
<_>
5 8 10 2 -1.
<_>
5 9 10 1 2.
<_>
<_>
5 1 6 4 -1.
<_>
7 1 2 4 3.
<_>
<_>
0 3 9 12 -1.
<_>
3 3 3 12 3.
<_>
<_>
9 8 3 12 -1.
<_>
9 12 3 4 3.
<_>
<_>
0 5 20 15 -1.
<_>
0 10 20 5 3.
<_>
<_>
2 2 6 8 -1.
<_>
2 2 3 4 2.
<_>
5 6 3 4 2.
<_>
<_>
2 1 6 2 -1.
<_>
2 2 6 1 2.
<_>
<_>
10 15 6 4 -1.
<_>
13 15 3 2 2.
<_>
10 17 3 2 2.
<_>
<_>
12 14 2 6 -1.
<_>
12 16 2 2 3.
<_>
<_>
5 15 4 4 -1.
<_>
5 15 2 2 2.
<_>
7 17 2 2 2.
<_>
<_>
7 18 1 2 -1.
<_>
7 19 1 1 2.
<_>
<_>
4 5 12 10 -1.
<_>
10 5 6 5 2.
<_>
4 10 6 5 2.
<_>
<_>
7 4 8 12 -1.
<_>
11 4 4 6 2.
<_>
7 10 4 6 2.
<_>
<_>
9 11 2 3 -1.
<_>
9 12 2 1 3.
<_>
<_>
3 3 12 12 -1.
<_>
3 3 6 6 2.
<_>
9 9 6 6 2.
<_>
<_>
15 11 5 3 -1.
<_>
15 12 5 1 3.
<_>
<_>
10 18 3 2 -1.
<_>
11 18 1 2 3.
<_>
<_>
0 11 5 3 -1.
<_>
0 12 5 1 3.
<_>
<_>
7 18 3 2 -1.
<_>
8 18 1 2 3.
<_>
<_>
2 8 16 2 -1.
<_>
2 9 16 1 2.
<_>
<_>
9 6 5 12 -1.
<_>
9 12 5 6 2.
<_>
<_>
6 3 8 6 -1.
<_>
6 6 8 3 2.
<_>
<_>
4 7 12 2 -1.
<_>
8 7 4 2 3.
<_>
<_>
10 9 6 8 -1.
<_>
10 13 6 4 2.
<_>
<_>
12 5 3 10 -1.
<_>
12 10 3 5 2.
<_>
<_>
4 6 3 9 -1.
<_>
4 9 3 3 3.
<_>
<_>
7 4 6 4 -1.
<_>
9 4 2 4 3.
<_>
<_>
12 3 8 3 -1.
<_>
12 3 4 3 2.
<_>
<_>
15 0 3 6 -1.
<_>
15 3 3 3 2.
<_>
<_>
2 12 10 8 -1.
<_>
2 12 5 4 2.
<_>
7 16 5 4 2.
<_>
<_>
5 5 6 8 -1.
<_>
5 9 6 4 2.
<_>
<_>
12 3 8 3 -1.
<_>
12 3 4 3 2.
<_>
<_>
15 0 3 6 -1.
<_>
15 3 3 3 2.
<_>
<_>
0 3 8 3 -1.
<_>
4 3 4 3 2.
<_>
<_>
2 1 4 4 -1.
<_>
2 3 4 2 2.
<_>
<_>
10 2 3 2 -1.
<_>
11 2 1 2 3.
<_>
<_>
10 3 3 1 -1.
<_>
11 3 1 1 3.
<_>
<_>
7 15 3 4 -1.
<_>
7 17 3 2 2.
<_>
<_>
4 13 3 6 -1.
<_>
4 15 3 2 3.
<_>
<_>
10 5 1 14 -1.
<_>
10 12 1 7 2.
<_>
<_>
5 4 10 6 -1.
<_>
5 6 10 2 3.
<_>
<_>
5 0 6 3 -1.
<_>
7 0 2 3 3.
<_>
<_>
6 0 3 5 -1.
<_>
7 0 1 5 3.
<_>
<_>
7 15 6 5 -1.
<_>
9 15 2 5 3.
<_>
<_>
9 10 2 6 -1.
<_>
9 12 2 2 3.
<_>
<_>
8 17 3 2 -1.
<_>
9 17 1 2 3.
<_>
<_>
1 12 7 6 -1.
<_>
1 14 7 2 3.
<_>
<_>
9 6 3 7 -1.
<_>
10 6 1 7 3.
<_>
<_>
16 3 4 9 -1.
<_>
16 6 4 3 3.
<_>
<_>
8 6 3 7 -1.
<_>
9 6 1 7 3.
<_>
<_>
0 5 18 8 -1.
<_>
0 5 9 4 2.
<_>
9 9 9 4 2.
<_>
<_>
13 5 2 10 -1.
<_>
13 10 2 5 2.
<_>
<_>
12 10 2 6 -1.
<_>
12 13 2 3 2.
<_>
<_>
7 0 3 5 -1.
<_>
8 0 1 5 3.
<_>
<_>
6 5 8 6 -1.
<_>
6 7 8 2 3.
<_>
<_>
10 3 6 14 -1.
<_>
13 3 3 7 2.
<_>
10 10 3 7 2.
<_>
<_>
13 5 1 8 -1.
<_>
13 9 1 4 2.
<_>
<_>
4 3 6 14 -1.
<_>
4 3 3 7 2.
<_>
7 10 3 7 2.
<_>
<_>
6 5 1 8 -1.
<_>
6 9 1 4 2.
<_>
<_>
8 1 1 6 -1.
<_>
8 3 1 2 3.
<_>
<_>
2 0 15 2 -1.
<_>
2 1 15 1 2.
<_>
<_>
0 7 20 6 -1.
<_>
0 9 20 2 3.
<_>
<_>
10 10 6 8 -1.
<_>
10 14 6 4 2.
<_>
<_>
7 1 3 2 -1.
<_>
8 1 1 2 3.
<_>
<_>
8 1 2 2 -1.
<_>
9 1 1 2 2.
<_>
<_>
4 3 12 9 -1.
<_>
4 6 12 3 3.
<_>
<_>
6 5 9 5 -1.
<_>
9 5 3 5 3.
<_>
<_>
5 5 9 5 -1.
<_>
8 5 3 5 3.
<_>
<_>
4 6 6 12 -1.
<_>
4 10 6 4 3.
<_>
<_>
13 0 6 18 -1.
<_>
13 0 3 18 2.
<_>
<_>
10 8 1 12 -1.
<_>
10 12 1 4 3.
<_>
<_>
3 2 6 10 -1.
<_>
3 2 3 5 2.
<_>
6 7 3 5 2.
<_>
<_>
1 2 4 6 -1.
<_>
3 2 2 6 2.
<_>
<_>
9 18 3 2 -1.
<_>
10 18 1 2 3.
<_>
<_>
10 18 3 2 -1.
<_>
11 18 1 2 3.
<_>
<_>
2 8 2 6 -1.
<_>
2 10 2 2 3.
<_>
<_>
7 5 6 6 -1.
<_>
7 7 6 2 3.
<_>
<_>
7 19 6 1 -1.
<_>
9 19 2 1 3.
<_>
<_>
10 18 3 2 -1.
<_>
11 18 1 2 3.
<_>
<_>
8 3 3 1 -1.
<_>
9 3 1 1 3.
<_>
<_>
2 2 16 2 -1.
<_>
2 2 8 1 2.
<_>
10 3 8 1 2.
<_>
<_>
8 11 5 3 -1.
<_>
8 12 5 1 3.
<_>
<_>
7 13 6 3 -1.
<_>
7 14 6 1 3.
<_>
<_>
0 1 6 15 -1.
<_>
2 1 2 15 3.
<_>
<_>
2 12 2 3 -1.
<_>
2 13 2 1 3.
<_>
<_>
16 13 1 3 -1.
<_>
16 14 1 1 3.
<_>
<_>
13 7 6 4 -1.
<_>
16 7 3 2 2.
<_>
13 9 3 2 2.
<_>
<_>
7 13 3 6 -1.
<_>
7 16 3 3 2.
<_>
<_>
7 5 1 14 -1.
<_>
7 12 1 7 2.
<_>
<_>
15 12 2 3 -1.
<_>
15 13 2 1 3.
<_>
<_>
10 5 3 14 -1.
<_>
10 12 3 7 2.
<_>
<_>
6 10 2 6 -1.
<_>
6 13 2 3 2.
<_>
<_>
6 5 1 8 -1.
<_>
6 9 1 4 2.
<_>
<_>
13 11 2 1 -1.
<_>
13 11 1 1 2.
<_>
<_>
12 1 6 10 -1.
<_>
15 1 3 5 2.
<_>
12 6 3 5 2.
<_>
<_>
3 12 2 3 -1.
<_>
3 13 2 1 3.
<_>
<_>
9 18 2 1 -1.
<_>
10 18 1 1 2.
<_>
<_>
1 0 17 9 -1.
<_>
1 3 17 3 3.
<_>
<_>
1 2 8 8 -1.
<_>
1 2 4 4 2.
<_>
5 6 4 4 2.
<_>
<_>
9 5 6 4 -1.
<_>
9 5 3 4 2.
<_>
<_>
10 9 7 10 -1.
<_>
10 14 7 5 2.
<_>
<_>
5 5 6 4 -1.
<_>
8 5 3 4 2.
<_>
<_>
0 7 20 6 -1.
<_>
0 9 20 2 3.
<_>
<_>
6 5 9 10 -1.
<_>
6 10 9 5 2.
<_>
<_>
8 4 4 12 -1.
<_>
8 10 4 6 2.
<_>
<_>
6 6 8 3 -1.
<_>
6 7 8 1 3.
<_>
<_>
3 13 10 6 -1.
<_>
3 13 5 3 2.
<_>
8 16 5 3 2.
<_>
<_>
15 1 4 11 -1.
<_>
15 1 2 11 2.
<_>
<_>
5 7 10 10 -1.
<_>
10 7 5 5 2.
<_>
5 12 5 5 2.
<_>
<_>
1 1 4 11 -1.
<_>
3 1 2 11 2.
<_>
<_>
1 5 8 12 -1.
<_>
1 11 8 6 2.
<_>
<_>
13 7 6 4 -1.
<_>
16 7 3 2 2.
<_>
13 9 3 2 2.
<_>
<_>
11 10 7 4 -1.
<_>
11 12 7 2 2.
<_>
<_>
0 4 20 12 -1.
<_>
0 4 10 6 2.
<_>
10 10 10 6 2.
<_>
<_>
1 5 6 15 -1.
<_>
1 10 6 5 3.
<_>
<_>
11 10 3 8 -1.
<_>
11 14 3 4 2.
<_>
<_>
11 12 7 6 -1.
<_>
11 14 7 2 3.
<_>
<_>
9 11 2 3 -1.
<_>
9 12 2 1 3.
<_>
<_>
8 13 4 3 -1.
<_>
8 14 4 1 3.
<_>
<_>
3 14 14 4 -1.
<_>
10 14 7 2 2.
<_>
3 16 7 2 2.
<_>
<_>
18 7 2 4 -1.
<_>
18 9 2 2 2.
<_>
<_>
3 12 6 6 -1.
<_>
3 14 6 2 3.
<_>
<_>
0 4 3 6 -1.
<_>
0 6 3 2 3.
<_>
<_>
9 14 3 3 -1.
<_>
9 15 3 1 3.
<_>
<_>
10 7 10 4 -1.
<_>
15 7 5 2 2.
<_>
10 9 5 2 2.
<_>
<_>
7 2 6 8 -1.
<_>
7 6 6 4 2.
<_>
<_>
6 3 6 2 -1.
<_>
8 3 2 2 3.
<_>
<_>
10 6 3 5 -1.
<_>
11 6 1 5 3.
<_>
<_>
9 0 6 19 -1.
<_>
11 0 2 19 3.
<_>
<_>
3 12 1 2 -1.
<_>
3 13 1 1 2.
<_>
<_>
7 14 5 3 -1.
<_>
7 15 5 1 3.
<_>
<_>
2 1 18 4 -1.
<_>
11 1 9 2 2.
<_>
2 3 9 2 2.
<_>
<_>
10 5 3 8 -1.
<_>
11 5 1 8 3.
<_>
<_>
0 1 18 4 -1.
<_>
0 1 9 2 2.
<_>
9 3 9 2 2.
<_>
<_>
7 5 3 8 -1.
<_>
8 5 1 8 3.
<_>
<_>
9 5 2 6 -1.
<_>
9 7 2 2 3.
<_>
<_>
10 8 5 2 -1.
<_>
10 9 5 1 2.
<_>
<_>
2 10 15 1 -1.
<_>
7 10 5 1 3.
<_>
<_>
2 7 2 6 -1.
<_>
2 9 2 2 3.
<_>
<_>
9 14 3 3 -1.
<_>
9 15 3 1 3.
<_>
<_>
9 7 4 10 -1.
<_>
9 12 4 5 2.
<_>
<_>
0 8 8 2 -1.
<_>
0 8 4 1 2.
<_>
4 9 4 1 2.
<_>
<_>
5 9 10 8 -1.
<_>
5 9 5 4 2.
<_>
10 13 5 4 2.
<_>
<_>
9 7 2 4 -1.
<_>
9 7 1 4 2.
<_>
<_>
9 6 3 4 -1.
<_>
10 6 1 4 3.
<_>
<_>
8 3 2 1 -1.
<_>
9 3 1 1 2.
<_>
<_>
8 6 3 4 -1.
<_>
9 6 1 4 3.
<_>
<_>
12 0 4 14 -1.
<_>
14 0 2 7 2.
<_>
12 7 2 7 2.
<_>
<_>
12 5 6 9 -1.
<_>
12 5 3 9 2.
<_>
<_>
0 2 6 16 -1.
<_>
3 2 3 16 2.
<_>
<_>
1 12 4 2 -1.
<_>
1 13 4 1 2.
<_>
<_>
7 7 6 1 -1.
<_>
9 7 2 1 3.
<_>
<_>
8 3 4 9 -1.
<_>
8 6 4 3 3.
<_>
<_>
12 10 4 6 -1.
<_>
12 13 4 3 2.
<_>
<_>
8 1 8 16 -1.
<_>
12 1 4 8 2.
<_>
8 9 4 8 2.
<_>
<_>
4 6 3 6 -1.
<_>
4 9 3 3 2.
<_>
<_>
1 3 6 2 -1.
<_>
4 3 3 2 2.
<_>
<_>
9 8 3 12 -1.
<_>
9 12 3 4 3.
<_>
<_>
10 9 7 10 -1.
<_>
10 14 7 5 2.
<_>
<_>
3 9 7 10 -1.
<_>
3 14 7 5 2.
<_>
<_>
7 5 1 14 -1.
<_>
7 12 1 7 2.
<_>
<_>
13 14 1 6 -1.
<_>
13 16 1 2 3.
<_>
<_>
14 12 3 6 -1.
<_>
14 14 3 2 3.
<_>
<_>
6 14 1 6 -1.
<_>
6 16 1 2 3.
<_>
<_>
3 12 3 6 -1.
<_>
3 14 3 2 3.
<_>
<_>
8 13 5 3 -1.
<_>
8 14 5 1 3.
<_>
<_>
9 14 2 3 -1.
<_>
9 15 2 1 3.
<_>
<_>
5 1 10 8 -1.
<_>
5 1 5 4 2.
<_>
10 5 5 4 2.
<_>
<_>
6 4 5 4 -1.
<_>
6 6 5 2 2.
<_>
<_>
1 10 18 1 -1.
<_>
7 10 6 1 3.
<_>
<_>
11 10 4 3 -1.
<_>
11 10 2 3 2.
<_>
<_>
5 11 6 1 -1.
<_>
7 11 2 1 3.
<_>
<_>
3 13 2 3 -1.
<_>
3 14 2 1 3.
<_>
<_>
12 12 3 4 -1.
<_>
12 14 3 2 2.
<_>
<_>
11 10 5 6 -1.
<_>
11 12 5 2 3.
<_>
<_>
0 8 16 2 -1.
<_>
0 9 16 1 2.
<_>
<_>
2 1 3 4 -1.
<_>
2 3 3 2 2.
<_>
<_>
9 7 3 3 -1.
<_>
10 7 1 3 3.
<_>
<_>
5 6 12 6 -1.
<_>
9 6 4 6 3.
<_>
<_>
8 7 3 3 -1.
<_>
9 7 1 3 3.
<_>
<_>
3 6 12 6 -1.
<_>
7 6 4 6 3.
<_>
<_>
10 5 6 5 -1.
<_>
12 5 2 5 3.
<_>
<_>
5 7 10 2 -1.
<_>
5 7 5 2 2.
<_>
<_>
4 5 6 5 -1.
<_>
6 5 2 5 3.
<_>
<_>
9 3 2 10 -1.
<_>
9 8 2 5 2.
<_>
<_>
3 1 16 2 -1.
<_>
11 1 8 1 2.
<_>
3 2 8 1 2.
<_>
<_>
9 9 3 2 -1.
<_>
9 10 3 1 2.
<_>
<_>
1 1 16 2 -1.
<_>
1 1 8 1 2.
<_>
9 2 8 1 2.
<_>
<_>
8 14 1 3 -1.
<_>
8 15 1 1 3.
<_>
<_>
4 5 12 10 -1.
<_>
10 5 6 5 2.
<_>
4 10 6 5 2.
<_>
<_>
7 13 6 6 -1.
<_>
10 13 3 3 2.
<_>
7 16 3 3 2.
<_>
<_>
8 9 3 2 -1.
<_>
8 10 3 1 2.
<_>
<_>
7 2 6 4 -1.
<_>
9 2 2 4 3.
<_>
<_>
6 6 9 3 -1.
<_>
6 7 9 1 3.
<_>
<_>
10 7 6 1 -1.
<_>
12 7 2 1 3.
<_>
<_>
0 0 18 6 -1.
<_>
6 0 6 6 3.
<_>
<_>
6 10 2 6 -1.
<_>
6 13 2 3 2.
<_>
<_>
11 12 3 6 -1.
<_>
11 15 3 3 2.
<_>
<_>
4 4 12 12 -1.
<_>
10 4 6 6 2.
<_>
4 10 6 6 2.
<_>
<_>
1 2 3 6 -1.
<_>
2 2 1 6 3.
<_>
<_>
1 5 3 7 -1.
<_>
2 5 1 7 3.
<_>
<_>
4 13 12 4 -1.
<_>
10 13 6 2 2.
<_>
4 15 6 2 2.
<_>
<_>
3 3 17 12 -1.
<_>
3 9 17 6 2.
<_>
<_>
3 3 14 12 -1.
<_>
3 3 7 6 2.
<_>
10 9 7 6 2.
<_>
<_>
2 11 16 9 -1.
<_>
2 14 16 3 3.
<_>
<_>
9 14 3 6 -1.
<_>
9 17 3 3 2.
<_>
<_>
8 14 4 6 -1.
<_>
10 14 2 3 2.
<_>
8 17 2 3 2.
<_>
<_>
6 2 6 1 -1.
<_>
8 2 2 1 3.
<_>
<_>
9 5 2 5 -1.
<_>
10 5 1 5 2.
<_>
<_>
9 8 3 5 -1.
<_>
10 8 1 5 3.
<_>
<_>
9 12 6 1 -1.
<_>
9 12 3 1 2.
<_>
<_>
8 8 3 5 -1.
<_>
9 8 1 5 3.
<_>
<_>
6 10 4 3 -1.
<_>
8 10 2 3 2.
<_>
<_>
0 4 20 6 -1.
<_>
0 6 20 2 3.
<_>
<_>
1 3 8 6 -1.
<_>
1 3 4 3 2.
<_>
5 6 4 3 2.
<_>
<_>
7 15 6 4 -1.
<_>
7 17 6 2 2.
<_>
<_>
3 10 14 10 -1.
<_>
3 15 14 5 2.
<_>
<_>
6 4 4 4 -1.
<_>
8 4 2 4 2.
<_>
<_>
0 4 20 10 -1.
<_>
0 9 20 5 2.
<_>
<_>
9 4 2 14 -1.
<_>
9 11 2 7 2.
<_>
<_>
2 0 16 4 -1.
<_>
2 2 16 2 2.
<_>
<_>
4 12 6 8 -1.
<_>
4 12 3 4 2.
<_>
7 16 3 4 2.
<_>
<_>
0 5 6 7 -1.
<_>
3 5 3 7 2.
<_>
<_>
10 7 10 4 -1.
<_>
15 7 5 2 2.
<_>
10 9 5 2 2.
<_>
<_>
5 8 12 1 -1.
<_>
9 8 4 1 3.
<_>
<_>
9 9 2 2 -1.
<_>
9 10 2 1 2.
<_>
<_>
9 4 2 4 -1.
<_>
9 6 2 2 2.
<_>
<_>
9 6 3 6 -1.
<_>
10 6 1 6 3.
<_>
<_>
12 7 6 4 -1.
<_>
15 7 3 2 2.
<_>
12 9 3 2 2.
<_>
<_>
8 6 3 6 -1.
<_>
9 6 1 6 3.
<_>
<_>
1 6 18 6 -1.
<_>
1 6 9 3 2.
<_>
10 9 9 3 2.
<_>
<_>
9 1 3 3 -1.
<_>
10 1 1 3 3.
<_>
<_>
10 8 5 2 -1.
<_>
10 9 5 1 2.
<_>
<_>
8 1 3 3 -1.
<_>
9 1 1 3 3.
<_>
<_>
5 8 5 2 -1.
<_>
5 9 5 1 2.
<_>
<_>
8 6 8 8 -1.
<_>
12 6 4 4 2.
<_>
8 10 4 4 2.
<_>
<_>
5 7 10 2 -1.
<_>
5 7 5 2 2.
<_>
<_>
4 5 12 10 -1.
<_>
4 5 6 5 2.
<_>
10 10 6 5 2.
<_>
<_>
5 5 2 3 -1.
<_>
5 6 2 1 3.
<_>
<_>
7 14 6 3 -1.
<_>
7 15 6 1 3.
<_>
<_>
9 14 3 3 -1.
<_>
9 15 3 1 3.
<_>
<_>
8 14 3 3 -1.
<_>
8 15 3 1 3.
<_>
<_>
1 10 8 9 -1.
<_>
1 13 8 3 3.
<_>
<_>
9 7 2 3 -1.
<_>
9 8 2 1 3.
<_>
<_>
12 3 3 3 -1.
<_>
13 3 1 3 3.
<_>
<_>
5 3 3 3 -1.
<_>
6 3 1 3 3.
<_>
<_>
5 6 2 12 -1.
<_>
5 10 2 4 3.
<_>
<_>
1 11 18 4 -1.
<_>
10 11 9 2 2.
<_>
1 13 9 2 2.
<_>
<_>
7 12 6 2 -1.
<_>
7 13 6 1 2.
<_>
<_>
6 0 3 6 -1.
<_>
7 0 1 6 3.
<_>
<_>
0 11 18 4 -1.
<_>
0 11 9 2 2.
<_>
9 13 9 2 2.
<_>
<_>
7 12 6 2 -1.
<_>
7 13 6 1 2.
<_>
<_>
9 12 3 3 -1.
<_>
9 13 3 1 3.
<_>
<_>
9 12 2 3 -1.
<_>
9 13 2 1 3.
<_>
<_>
8 11 4 3 -1.
<_>
8 12 4 1 3.
<_>
<_>
13 3 4 2 -1.
<_>
13 4 4 1 2.
<_>
<_>
4 0 12 2 -1.
<_>
4 1 12 1 2.
<_>
<_>
6 9 8 8 -1.
<_>
6 9 4 4 2.
<_>
10 13 4 4 2.
<_>
<_>
1 11 6 2 -1.
<_>
1 12 6 1 2.
<_>
<_>
2 5 18 8 -1.
<_>
11 5 9 4 2.
<_>
2 9 9 4 2.
<_>
<_>
7 1 6 10 -1.
<_>
7 6 6 5 2.
<_>
<_>
0 3 3 6 -1.
<_>
0 5 3 2 3.
<_>
<_>
4 5 4 3 -1.
<_>
4 6 4 1 3.
<_>
<_>
19 3 1 6 -1.
<_>
19 5 1 2 3.
<_>
<_>
6 15 8 2 -1.
<_>
6 16 8 1 2.
<_>
<_>
0 3 1 6 -1.
<_>
0 5 1 2 3.
<_>
<_>
5 5 3 3 -1.
<_>
5 6 3 1 3.
<_>
<_>
8 8 4 3 -1.
<_>
8 9 4 1 3.
<_>
<_>
10 6 6 3 -1.
<_>
12 6 2 3 3.
<_>
<_>
8 13 2 6 -1.
<_>
8 16 2 3 2.
<_>
<_>
9 11 2 8 -1.
<_>
9 15 2 4 2.
<_>
<_>
10 6 6 3 -1.
<_>
12 6 2 3 3.
<_>
<_>
5 15 15 5 -1.
<_>
10 15 5 5 3.
<_>
<_>
2 14 2 2 -1.
<_>
2 15 2 1 2.
<_>
<_>
4 7 6 2 -1.
<_>
6 7 2 2 3.
<_>
<_>
8 3 6 1 -1.
<_>
10 3 2 1 3.
<_>
<_>
1 0 18 12 -1.
<_>
7 0 6 12 3.
<_>
<_>
0 14 8 6 -1.
<_>
4 14 4 6 2.
<_>
<_>
0 15 15 5 -1.
<_>
5 15 5 5 3.
<_>
<_>
8 3 6 1 -1.
<_>
10 3 2 1 3.
<_>
<_>
11 11 3 6 -1.
<_>
11 14 3 3 2.
<_>
<_>
6 3 6 1 -1.
<_>
8 3 2 1 3.
<_>
<_>
6 11 3 6 -1.
<_>
6 14 3 3 2.
<_>
<_>
9 6 3 4 -1.
<_>
10 6 1 4 3.
<_>
<_>
12 10 4 7 -1.
<_>
12 10 2 7 2.
<_>
<_>
8 6 3 4 -1.
<_>
9 6 1 4 3.
<_>
<_>
4 6 4 7 -1.
<_>
6 6 2 7 2.
<_>
<_>
10 3 4 12 -1.
<_>
10 3 2 12 2.
<_>
<_>
10 8 3 4 -1.
<_>
11 8 1 4 3.
<_>
<_>
1 0 18 14 -1.
<_>
7 0 6 14 3.
<_>
<_>
2 8 6 11 -1.
<_>
5 8 3 11 2.
<_>
<_>
1 4 15 4 -1.
<_>
1 6 15 2 2.
<_>
<_>
5 5 10 8 -1.
<_>
5 9 10 4 2.
<_>
<_>
14 2 6 8 -1.
<_>
14 2 3 8 2.
<_>
<_>
11 6 6 14 -1.
<_>
14 6 3 7 2.
<_>
11 13 3 7 2.
<_>
<_>
9 5 2 12 -1.
<_>
9 11 2 6 2.
<_>
<_>
3 7 4 6 -1.
<_>
3 9 4 2 3.
<_>
<_>
14 3 6 6 -1.
<_>
14 3 3 6 2.
<_>
<_>
15 2 4 4 -1.
<_>
15 4 4 2 2.
<_>
<_>
0 2 6 7 -1.
<_>
3 2 3 7 2.
<_>
<_>
3 6 6 14 -1.
<_>
3 6 3 7 2.
<_>
6 13 3 7 2.
<_>
<_>
4 6 16 8 -1.
<_>
4 10 16 4 2.
<_>
<_>
10 12 2 8 -1.
<_>
10 16 2 4 2.
<_>
<_>
7 0 6 20 -1.
<_>
9 0 2 20 3.
<_>
<_>
1 7 16 12 -1.
<_>
1 7 8 6 2.
<_>
9 13 8 6 2.
<_>
<_>
9 11 3 3 -1.
<_>
9 12 3 1 3.
<_>
<_>
11 9 4 5 -1.
<_>
11 9 2 5 2.
<_>
<_>
3 3 1 2 -1.
<_>
3 4 1 1 2.
<_>
<_>
7 17 5 3 -1.
<_>
7 18 5 1 3.
<_>
<_>
8 12 4 8 -1.
<_>
10 12 2 4 2.
<_>
8 16 2 4 2.
<_>
<_>
7 4 10 12 -1.
<_>
12 4 5 6 2.
<_>
7 10 5 6 2.
<_>
<_>
8 14 4 3 -1.
<_>
8 15 4 1 3.
<_>
<_>
5 9 4 5 -1.
<_>
7 9 2 5 2.
<_>
<_>
9 9 8 2 -1.
<_>
9 9 4 2 2.
<_>
<_>
14 15 5 2 -1.
<_>
14 16 5 1 2.
<_>
<_>
9 14 2 3 -1.
<_>
9 15 2 1 3.
<_>
<_>
1 7 8 4 -1.
<_>
1 7 4 2 2.
<_>
5 9 4 2 2.
<_>
<_>
19 3 1 2 -1.
<_>
19 4 1 1 2.
<_>
<_>
9 12 2 3 -1.
<_>
9 13 2 1 3.
<_>
<_>
3 14 14 4 -1.
<_>
3 14 7 2 2.
<_>
10 16 7 2 2.
<_>
<_>
5 0 10 2 -1.
<_>
5 1 10 1 2.
<_>
<_>
11 14 4 6 -1.
<_>
11 16 4 2 3.
<_>
<_>
7 14 6 3 -1.
<_>
7 15 6 1 3.
<_>
<_>
7 13 6 6 -1.
<_>
7 13 3 3 2.
<_>
10 16 3 3 2.
<_>
<_>
0 2 1 6 -1.
<_>
0 4 1 2 3.
<_>
<_>
6 7 8 2 -1.
<_>
6 8 8 1 2.
<_>
<_>
9 7 6 1 -1.
<_>
9 7 3 1 2.
<_>
<_>
7 1 6 10 -1.
<_>
7 6 6 5 2.
<_>
<_>
0 2 6 2 -1.
<_>
0 3 6 1 2.
<_>
<_>
11 4 2 4 -1.
<_>
11 4 1 4 2.
<_>
<_>
11 10 3 6 -1.
<_>
11 13 3 3 2.
<_>
<_>
3 9 8 2 -1.
<_>
7 9 4 2 2.
<_>
<_>
0 0 4 6 -1.
<_>
2 0 2 6 2.
<_>
<_>
7 0 6 2 -1.
<_>
9 0 2 2 3.
<_>
<_>
9 15 2 3 -1.
<_>
9 16 2 1 3.
<_>
<_>
3 12 1 2 -1.
<_>
3 13 1 1 2.
<_>
<_>
4 5 11 3 -1.
<_>
4 6 11 1 3.
<_>
<_>
11 4 2 4 -1.
<_>
11 4 1 4 2.
<_>
<_>
8 3 6 3 -1.
<_>
10 3 2 3 3.
<_>
<_>
7 4 2 4 -1.
<_>
8 4 1 4 2.
<_>
<_>
6 3 6 3 -1.
<_>
8 3 2 3 3.
<_>
<_>
11 4 4 3 -1.
<_>
11 5 4 1 3.
<_>
<_>
11 8 2 8 -1.
<_>
11 12 2 4 2.
<_>
<_>
8 7 3 5 -1.
<_>
9 7 1 5 3.
<_>
<_>
9 7 2 5 -1.
<_>
10 7 1 5 2.
<_>
<_>
14 11 1 6 -1.
<_>
14 13 1 2 3.
<_>
<_>
8 8 4 3 -1.
<_>
8 9 4 1 3.
<_>
<_>
0 3 2 2 -1.
<_>
0 4 2 1 2.
<_>
<_>
4 14 5 6 -1.
<_>
4 16 5 2 3.
<_>
<_>
11 4 4 3 -1.
<_>
11 5 4 1 3.
<_>
<_>
12 4 3 3 -1.
<_>
12 5 3 1 3.
<_>
<_>
5 4 4 3 -1.
<_>
5 5 4 1 3.
<_>
<_>
5 15 4 2 -1.
<_>
7 15 2 2 2.
<_>
<_>
15 1 5 9 -1.
<_>
15 4 5 3 3.
<_>
<_>
9 10 3 3 -1.
<_>
9 11 3 1 3.
<_>
<_>
1 6 2 6 -1.
<_>
1 8 2 2 3.
<_>
<_>
2 4 8 15 -1.
<_>
2 9 8 5 3.
<_>
<_>
9 12 3 2 -1.
<_>
9 13 3 1 2.
<_>
<_>
9 12 3 3 -1.
<_>
9 13 3 1 3.
<_>
<_>
7 6 3 5 -1.
<_>
8 6 1 5 3.
<_>
<_>
5 3 6 2 -1.
<_>
7 3 2 2 3.
<_>
<_>
6 1 8 10 -1.
<_>
10 1 4 5 2.
<_>
6 6 4 5 2.
<_>
<_>
0 0 20 10 -1.
<_>
10 0 10 5 2.
<_>
0 5 10 5 2.
<_>
<_>
6 3 3 1 -1.
<_>
7 3 1 1 3.
<_>
<_>
0 2 6 8 -1.
<_>
2 2 2 8 3.
<_>
<_>
11 10 3 4 -1.
<_>
11 12 3 2 2.
<_>
<_>
12 6 3 8 -1.
<_>
12 10 3 4 2.
<_>
<_>
6 10 3 4 -1.
<_>
6 12 3 2 2.
<_>
<_>
5 6 3 8 -1.
<_>
5 10 3 4 2.
<_>
<_>
2 6 18 6 -1.
<_>
11 6 9 3 2.
<_>
2 9 9 3 2.
<_>
<_>
7 14 7 3 -1.
<_>
7 15 7 1 3.
<_>
<_>
0 0 2 12 -1.
<_>
1 0 1 12 2.
<_>
<_>
1 2 18 16 -1.
<_>
1 10 18 8 2.
<_>
<_>
9 13 5 3 -1.
<_>
9 14 5 1 3.
<_>
<_>
8 13 4 3 -1.
<_>
8 14 4 1 3.
<_>
<_>
0 6 18 6 -1.
<_>
0 6 9 3 2.
<_>
9 9 9 3 2.
<_>
<_>
7 13 6 3 -1.
<_>
7 14 6 1 3.
<_>
<_>
17 4 1 3 -1.
<_>
17 5 1 1 3.
<_>
<_>
12 11 1 9 -1.
<_>
12 14 1 3 3.
<_>
<_>
2 4 1 3 -1.
<_>
2 5 1 1 3.
<_>
<_>
5 4 2 3 -1.
<_>
5 5 2 1 3.
<_>
<_>
1 2 18 3 -1.
<_>
7 2 6 3 3.
<_>
<_>
0 1 20 6 -1.
<_>
0 3 20 2 3.
<_>
<_>
7 5 6 3 -1.
<_>
9 5 2 3 3.
<_>
<_>
13 7 6 4 -1.
<_>
16 7 3 2 2.
<_>
13 9 3 2 2.
<_>
<_>
3 1 4 10 -1.
<_>
3 1 2 5 2.
<_>
5 6 2 5 2.
<_>
<_>
0 4 19 10 -1.
<_>
0 9 19 5 2.
<_>
<_>
9 8 3 12 -1.
<_>
9 12 3 4 3.
<_>
<_>
11 18 5 2 -1.
<_>
11 19 5 1 2.
<_>
<_>
5 16 6 4 -1.
<_>
5 16 3 2 2.
<_>
8 18 3 2 2.
<_>
<_>
5 18 3 2 -1.
<_>
5 19 3 1 2.
<_>
<_>
13 11 3 2 -1.
<_>
13 12 3 1 2.
<_>
<_>
8 5 8 4 -1.
<_>
8 5 4 4 2.
<_>
<_>
1 2 18 6 -1.
<_>
1 2 9 3 2.
<_>
10 5 9 3 2.
<_>
<_>
3 5 14 6 -1.
<_>
3 7 14 2 3.
<_>
<_>
18 1 2 6 -1.
<_>
18 3 2 2 3.
<_>
<_>
9 11 6 1 -1.
<_>
11 11 2 1 3.
<_>
<_>
0 2 6 11 -1.
<_>
3 2 3 11 2.
<_>
<_>
4 12 2 3 -1.
<_>
4 13 2 1 3.
<_>
<_>
6 12 9 2 -1.
<_>
9 12 3 2 3.
<_>
<_>
9 4 6 15 -1.
<_>
9 4 3 15 2.
<_>
<_>
5 11 6 1 -1.
<_>
7 11 2 1 3.
<_>
<_>
5 4 6 15 -1.
<_>
8 4 3 15 2.
<_>
<_>
14 12 6 7 -1.
<_>
14 12 3 7 2.
<_>
<_>
18 3 2 9 -1.
<_>
18 6 2 3 3.
<_>
<_>
8 1 3 1 -1.
<_>
9 1 1 1 3.
<_>
<_>
0 12 6 7 -1.
<_>
3 12 3 7 2.
<_>
<_>
13 7 6 4 -1.
<_>
16 7 3 2 2.
<_>
13 9 3 2 2.
<_>
<_>
8 0 10 2 -1.
<_>
8 1 10 1 2.
<_>
<_>
1 7 6 4 -1.
<_>
1 7 3 2 2.
<_>
4 9 3 2 2.
<_>
<_>
1 2 3 3 -1.
<_>
1 3 3 1 3.
<_>
<_>
9 13 4 3 -1.
<_>
9 14 4 1 3.
<_>
<_>
12 13 7 2 -1.
<_>
12 14 7 1 2.
<_>
<_>
5 12 9 2 -1.
<_>
8 12 3 2 3.
<_>
<_>
6 10 4 8 -1.
<_>
6 14 4 4 2.
<_>
<_>
1 0 18 4 -1.
<_>
7 0 6 4 3.
<_>
<_>
12 0 5 2 -1.
<_>
12 1 5 1 2.
<_>
<_>
7 7 1 12 -1.
<_>
7 13 1 6 2.
<_>
<_>
6 2 3 4 -1.
<_>
7 2 1 4 3.
<_>
<_>
0 13 20 6 -1.
<_>
0 15 20 2 3.
<_>
<_>
8 5 12 2 -1.
<_>
14 5 6 1 2.
<_>
8 6 6 1 2.
<_>
<_>
8 14 2 3 -1.
<_>
8 15 2 1 3.
<_>
<_>
8 14 4 3 -1.
<_>
8 15 4 1 3.
<_>
<_>
12 13 7 6 -1.
<_>
12 15 7 2 3.
<_>
<_>
6 0 8 12 -1.
<_>
10 0 4 6 2.
<_>
6 6 4 6 2.
<_>
<_>
0 15 9 4 -1.
<_>
0 17 9 2 2.
<_>
<_>
9 0 2 5 -1.
<_>
10 0 1 5 2.
<_>
<_>
9 5 2 6 -1.
<_>
9 5 1 6 2.
<_>
<_>
17 2 3 6 -1.
<_>
17 4 3 2 3.
<_>
<_>
3 11 2 3 -1.
<_>
3 12 2 1 3.
<_>
<_>
7 13 3 3 -1.
<_>
7 14 3 1 3.
<_>
<_>
14 12 5 3 -1.
<_>
14 13 5 1 3.
<_>
<_>
4 8 14 3 -1.
<_>
4 9 14 1 3.
<_>
<_>
1 12 5 3 -1.
<_>
1 13 5 1 3.
<_>
<_>
1 15 12 2 -1.
<_>
1 15 6 1 2.
<_>
7 16 6 1 2.
<_>
<_>
12 11 4 2 -1.
<_>
12 12 4 1 2.
<_>
<_>
9 8 3 5 -1.
<_>
10 8 1 5 3.
<_>
<_>
9 5 2 6 -1.
<_>
10 5 1 6 2.
<_>
<_>
0 2 3 6 -1.
<_>
0 4 3 2 3.
<_>
<_>
12 11 4 2 -1.
<_>
12 12 4 1 2.
<_>
<_>
9 7 3 5 -1.
<_>
10 7 1 5 3.
<_>
<_>
4 11 4 2 -1.
<_>
4 12 4 1 2.
<_>
<_>
8 8 3 5 -1.
<_>
9 8 1 5 3.
<_>
<_>
9 3 3 1 -1.
<_>
10 3 1 1 3.
<_>
<_>
16 5 3 8 -1.
<_>
17 5 1 8 3.
<_>
<_>
8 3 3 1 -1.
<_>
9 3 1 1 3.
<_>
<_>
1 5 3 8 -1.
<_>
2 5 1 8 3.
<_>
<_>
10 1 3 3 -1.
<_>
11 1 1 3 3.
<_>
<_>
17 5 2 4 -1.
<_>
17 5 1 4 2.
<_>
<_>
2 8 14 3 -1.
<_>
2 9 14 1 3.
<_>
<_>
9 7 1 3 -1.
<_>
9 8 1 1 3.
<_>
<_>
6 1 8 10 -1.
<_>
6 6 8 5 2.
<_>
<_>
13 0 6 8 -1.
<_>
16 0 3 4 2.
<_>
13 4 3 4 2.
<_>
<_>
1 5 2 4 -1.
<_>
2 5 1 4 2.
<_>
<_>
4 2 12 2 -1.
<_>
4 3 12 1 2.
<_>
<_>
8 8 4 4 -1.
<_>
8 10 4 2 2.
<_>
<_>
5 6 12 4 -1.
<_>
9 6 4 4 3.
<_>
<_>
1 2 8 1 -1.
<_>
5 2 4 1 2.
<_>
<_>
1 1 6 10 -1.
<_>
3 1 2 10 3.
<_>
<_>
8 6 8 2 -1.
<_>
8 6 4 2 2.
<_>
<_>
10 7 6 6 -1.
<_>
12 7 2 6 3.
<_>
<_>
4 6 8 2 -1.
<_>
8 6 4 2 2.
<_>
<_>
4 7 6 6 -1.
<_>
6 7 2 6 3.
<_>
<_>
3 14 16 4 -1.
<_>
3 16 16 2 2.
<_>
<_>
8 12 4 2 -1.
<_>
8 13 4 1 2.
<_>
<_>
8 12 3 3 -1.
<_>
8 13 3 1 3.
<_>
<_>
5 12 6 1 -1.
<_>
8 12 3 1 2.
<_>
<_>
18 10 2 3 -1.
<_>
18 11 2 1 3.
<_>
<_>
16 8 4 6 -1.
<_>
16 10 4 2 3.
<_>
<_>
8 3 2 1 -1.
<_>
9 3 1 1 2.
<_>
<_>
7 1 3 9 -1.
<_>
8 1 1 9 3.
<_>
<_>
5 11 11 6 -1.
<_>
5 14 11 3 2.
<_>
<_>
12 2 3 14 -1.
<_>
12 9 3 7 2.
<_>
<_>
8 7 3 3 -1.
<_>
9 7 1 3 3.
<_>
<_>
3 5 12 5 -1.
<_>
7 5 4 5 3.
<_>
<_>
1 2 6 3 -1.
<_>
4 2 3 3 2.
<_>
<_>
5 5 6 10 -1.
<_>
5 5 3 5 2.
<_>
8 10 3 5 2.
<_>
<_>
16 18 2 2 -1.
<_>
16 18 1 2 2.
<_>
<_>
16 18 2 2 -1.
<_>
16 18 1 2 2.
<_>
<_>
8 4 2 5 -1.
<_>
9 4 1 5 2.
<_>
<_>
8 4 1 4 -1.
<_>
8 6 1 2 2.
<_>
<_>
7 15 12 4 -1.
<_>
13 15 6 2 2.
<_>
7 17 6 2 2.
<_>
<_>
11 18 6 2 -1.
<_>
11 19 6 1 2.
<_>
<_>
7 7 4 10 -1.
<_>
7 12 4 5 2.
<_>
<_>
5 6 10 8 -1.
<_>
5 10 10 4 2.
<_>
<_>
11 1 6 12 -1.
<_>
14 1 3 6 2.
<_>
11 7 3 6 2.
<_>
<_>
5 8 12 1 -1.
<_>
9 8 4 1 3.
<_>
<_>
4 7 3 6 -1.
<_>
4 9 3 2 3.
<_>
<_>
4 11 3 4 -1.
<_>
4 13 3 2 2.
<_>
<_>
14 16 2 2 -1.
<_>
14 17 2 1 2.
<_>
<_>
15 15 2 2 -1.
<_>
15 16 2 1 2.
<_>
<_>
7 12 6 2 -1.
<_>
7 13 6 1 2.
<_>
<_>
8 13 4 2 -1.
<_>
8 14 4 1 2.
<_>
<_>
11 1 6 12 -1.
<_>
14 1 3 6 2.
<_>
11 7 3 6 2.
<_>
<_>
12 2 4 2 -1.
<_>
12 3 4 1 2.
<_>
<_>
3 10 12 6 -1.
<_>
3 10 6 3 2.
<_>
9 13 6 3 2.
<_>
<_>
3 1 6 12 -1.
<_>
3 1 3 6 2.
<_>
6 7 3 6 2.
<_>
<_>
16 6 4 14 -1.
<_>
18 6 2 7 2.
<_>
16 13 2 7 2.
<_>
<_>
5 1 10 8 -1.
<_>
10 1 5 4 2.
<_>
5 5 5 4 2.
<_>
<_>
0 6 4 14 -1.
<_>
0 6 2 7 2.
<_>
2 13 2 7 2.
<_>
<_>
1 15 12 4 -1.
<_>
1 15 6 2 2.
<_>
7 17 6 2 2.
<_>
<_>
10 17 3 3 -1.
<_>
11 17 1 3 3.
<_>
<_>
11 2 2 6 -1.
<_>
12 2 1 3 2.
<_>
11 5 1 3 2.
<_>
<_>
7 17 3 3 -1.
<_>
8 17 1 3 3.
<_>
<_>
8 15 4 3 -1.
<_>
8 16 4 1 3.
<_>
<_>
10 15 4 2 -1.
<_>
12 15 2 1 2.
<_>
10 16 2 1 2.
<_>
<_>
13 13 4 3 -1.
<_>
13 14 4 1 3.
<_>
<_>
3 13 4 3 -1.
<_>
3 14 4 1 3.
<_>
<_>
7 2 2 6 -1.
<_>
7 2 1 3 2.
<_>
8 5 1 3 2.
<_>
<_>
2 1 16 3 -1.
<_>
2 2 16 1 3.
<_>
<_>
10 15 4 2 -1.
<_>
12 15 2 1 2.
<_>
10 16 2 1 2.
<_>
<_>
6 15 4 2 -1.
<_>
6 15 2 1 2.
<_>
8 16 2 1 2.
<_>
<_>
3 0 13 3 -1.
<_>
3 1 13 1 3.
<_>
<_>
0 9 20 3 -1.
<_>
0 10 20 1 3.
<_>
<_>
6 7 9 2 -1.
<_>
6 8 9 1 2.
<_>
<_>
8 14 3 6 -1.
<_>
9 14 1 6 3.
<_>
<_>
9 10 2 2 -1.
<_>
9 11 2 1 2.
<_>
<_>
9 7 2 5 -1.
<_>
9 7 1 5 2.
<_>
<_>
5 6 10 3 -1.
<_>
5 6 5 3 2.
<_>
<_>
9 7 2 5 -1.
<_>
10 7 1 5 2.
<_>
<_>
5 6 10 3 -1.
<_>
10 6 5 3 2.
<_>
<_>
13 9 2 2 -1.
<_>
13 9 1 2 2.
<_>
<_>
4 3 12 11 -1.
<_>
8 3 4 11 3.
<_>
<_>
7 1 2 7 -1.
<_>
8 1 1 7 2.
<_>
<_>
7 4 3 8 -1.
<_>
8 4 1 8 3.
<_>
<_>
13 9 2 2 -1.
<_>
13 9 1 2 2.
<_>
<_>
11 6 2 2 -1.
<_>
12 6 1 1 2.
<_>
11 7 1 1 2.
<_>
<_>
5 4 2 3 -1.
<_>
5 5 2 1 3.
<_>
<_>
6 5 1 3 -1.
<_>
6 6 1 1 3.
<_>
<_>
13 9 2 2 -1.
<_>
13 9 1 2 2.
<_>
<_>
16 14 3 3 -1.
<_>
16 15 3 1 3.
<_>
<_>
5 9 2 2 -1.
<_>
6 9 1 2 2.
<_>
<_>
1 14 3 3 -1.
<_>
1 15 3 1 3.
<_>
<_>
13 1 1 6 -1.
<_>
13 3 1 2 3.
<_>
<_>
13 3 7 2 -1.
<_>
13 4 7 1 2.
<_>
<_>
0 6 20 14 -1.
<_>
0 13 20 7 2.
<_>
<_>
0 4 3 6 -1.
<_>
0 6 3 2 3.
<_>
<_>
10 1 9 6 -1.
<_>
10 3 9 2 3.
<_>
<_>
8 0 12 5 -1.
<_>
8 0 6 5 2.
<_>
<_>
0 0 18 5 -1.
<_>
6 0 6 5 3.
<_>
<_>
1 1 9 6 -1.
<_>
1 3 9 2 3.
<_>
<_>
15 15 2 2 -1.
<_>
15 16 2 1 2.
<_>
<_>
13 16 3 4 -1.
<_>
13 18 3 2 2.
<_>
<_>
3 15 2 2 -1.
<_>
3 16 2 1 2.
<_>
<_>
4 16 3 4 -1.
<_>
4 18 3 2 2.
<_>
<_>
11 14 1 3 -1.
<_>
11 15 1 1 3.
<_>
<_>
9 13 5 3 -1.
<_>
9 14 5 1 3.
<_>
<_>
0 0 3 6 -1.
<_>
0 2 3 2 3.
<_>
<_>
4 1 6 3 -1.
<_>
6 1 2 3 3.
<_>
<_>
9 13 4 3 -1.
<_>
9 14 4 1 3.
<_>
<_>
8 15 5 3 -1.
<_>
8 16 5 1 3.
<_>
<_>
8 3 3 2 -1.
<_>
9 3 1 2 3.
<_>
<_>
1 8 18 2 -1.
<_>
1 9 18 1 2.
<_>
<_>
11 14 1 3 -1.
<_>
11 15 1 1 3.
<_>
<_>
8 13 6 3 -1.
<_>
8 14 6 1 3.
<_>
<_>
8 14 1 3 -1.
<_>
8 15 1 1 3.
<_>
<_>
4 13 12 4 -1.
<_>
4 13 6 2 2.
<_>
10 15 6 2 2.
<_>
<_>
10 7 2 2 -1.
<_>
10 7 1 2 2.
<_>
<_>
13 4 2 8 -1.
<_>
14 4 1 4 2.
<_>
13 8 1 4 2.
<_>
<_>
0 5 4 6 -1.
<_>
0 7 4 2 3.
<_>
<_>
8 7 2 2 -1.
<_>
9 7 1 2 2.
<_>
<_>
13 0 3 7 -1.
<_>
14 0 1 7 3.
<_>
<_>
11 2 2 14 -1.
<_>
11 2 1 14 2.
<_>
<_>
4 0 3 7 -1.
<_>
5 0 1 7 3.
<_>
<_>
5 5 8 12 -1.
<_>
5 5 4 6 2.
<_>
9 11 4 6 2.
<_>
<_>
11 4 6 3 -1.
<_>
11 5 6 1 3.
<_>
<_>
12 3 4 3 -1.
<_>
12 4 4 1 3.
<_>
<_>
5 5 10 12 -1.
<_>
5 5 5 6 2.
<_>
10 11 5 6 2.
<_>
<_>
3 6 12 3 -1.
<_>
9 6 6 3 2.
<_>
<_>
9 6 2 7 -1.
<_>
9 6 1 7 2.
<_>
<_>
9 5 2 4 -1.
<_>
9 5 1 4 2.
<_>
<_>
8 7 3 3 -1.
<_>
9 7 1 3 3.
<_>
<_>
5 1 6 4 -1.
<_>
7 1 2 4 3.
<_>
<_>
13 16 7 3 -1.
<_>
13 17 7 1 3.
<_>
<_>
12 4 3 3 -1.
<_>
12 5 3 1 3.
<_>
<_>
0 16 7 3 -1.
<_>
0 17 7 1 3.
<_>
<_>
5 4 3 3 -1.
<_>
5 5 3 1 3.
<_>
<_>
12 9 8 10 -1.
<_>
12 9 4 10 2.
<_>
<_>
8 10 12 5 -1.
<_>
12 10 4 5 3.
<_>
<_>
0 9 8 10 -1.
<_>
4 9 4 10 2.
<_>
<_>
0 10 12 5 -1.
<_>
4 10 4 5 3.
<_>
<_>
2 3 6 2 -1.
<_>
5 3 3 2 2.
<_>
<_>
0 0 17 9 -1.
<_>
0 3 17 3 3.
<_>
<_>
4 7 12 2 -1.
<_>
8 7 4 2 3.
<_>
<_>
10 4 6 4 -1.
<_>
12 4 2 4 3.
<_>
<_>
0 10 20 4 -1.
<_>
0 12 20 2 2.
<_>
<_>
4 3 6 5 -1.
<_>
6 3 2 5 3.
<_>
<_>
1 1 18 4 -1.
<_>
7 1 6 4 3.
<_>
<_>
13 9 2 3 -1.
<_>
13 9 1 3 2.
<_>
<_>
6 15 7 4 -1.
<_>
6 17 7 2 2.
<_>
<_>
3 17 4 2 -1.
<_>
3 18 4 1 2.
<_>
<_>
9 4 8 10 -1.
<_>
9 9 8 5 2.
<_>
<_>
9 17 3 2 -1.
<_>
10 17 1 2 3.
<_>
<_>
8 2 4 8 -1.
<_>
8 6 4 4 2.
<_>
<_>
3 4 14 12 -1.
<_>
3 4 7 6 2.
<_>
10 10 7 6 2.
<_>
<_>
7 7 6 4 -1.
<_>
9 7 2 4 3.
<_>
<_>
6 7 9 4 -1.
<_>
6 9 9 2 2.
<_>
<_>
2 10 3 3 -1.
<_>
2 11 3 1 3.
<_>
<_>
4 6 2 9 -1.
<_>
4 9 2 3 3.
<_>
<_>
9 11 3 3 -1.
<_>
9 12 3 1 3.
<_>
<_>
3 1 15 2 -1.
<_>
3 2 15 1 2.
<_>
<_>
9 8 2 3 -1.
<_>
9 9 2 1 3.
<_>
<_>
9 6 2 5 -1.
<_>
10 6 1 5 2.
<_>
<_>
9 7 2 3 -1.
<_>
9 8 2 1 3.
<_>
<_>
4 10 12 10 -1.
<_>
4 15 12 5 2.
<_>
<_>
0 10 4 2 -1.
<_>
0 11 4 1 2.
<_>
<_>
5 15 9 2 -1.
<_>
5 16 9 1 2.
<_>
<_>
8 14 6 3 -1.
<_>
8 15 6 1 3.
<_>
<_>
8 16 4 3 -1.
<_>
8 17 4 1 3.
<_>
<_>
8 9 4 2 -1.
<_>
8 10 4 1 2.
<_>
<_>
3 3 14 2 -1.
<_>
3 4 14 1 2.
<_>
<_>
11 12 1 2 -1.
<_>
11 13 1 1 2.
<_>
<_>
4 12 12 1 -1.
<_>
8 12 4 1 3.
<_>
<_>
0 2 1 2 -1.
<_>
0 3 1 1 2.
<_>
<_>
7 4 4 6 -1.
<_>
9 4 2 6 2.
<_>
<_>
0 2 20 14 -1.
<_>
10 2 10 7 2.
<_>
0 9 10 7 2.
<_>
<_>
14 6 1 3 -1.
<_>
14 7 1 1 3.
<_>
<_>
0 4 20 12 -1.
<_>
0 4 10 6 2.
<_>
10 10 10 6 2.
<_>
<_>
8 12 1 2 -1.
<_>
8 13 1 1 2.
<_>
<_>
9 18 3 2 -1.
<_>
10 18 1 2 3.
<_>
<_>
9 17 6 2 -1.
<_>
11 17 2 2 3.
<_>
<_>
5 6 2 3 -1.
<_>
5 7 2 1 3.
<_>
<_>
5 4 3 3 -1.
<_>
5 5 3 1 3.
<_>
<_>
14 15 3 2 -1.
<_>
14 16 3 1 2.
<_>
<_>
11 3 3 4 -1.
<_>
12 3 1 4 3.
<_>
<_>
3 15 3 2 -1.
<_>
3 16 3 1 2.
<_>
<_>
9 12 2 3 -1.
<_>
9 13 2 1 3.
<_>
<_>
9 13 3 7 -1.
<_>
10 13 1 7 3.
<_>
<_>
12 12 5 3 -1.
<_>
12 13 5 1 3.
<_>
<_>
8 18 3 2 -1.
<_>
9 18 1 2 3.
<_>
<_>
4 7 12 4 -1.
<_>
4 7 6 2 2.
<_>
10 9 6 2 2.
<_>
<_>
6 19 14 1 -1.
<_>
6 19 7 1 2.
<_>
<_>
16 14 3 2 -1.
<_>
16 15 3 1 2.
<_>
<_>
1 0 6 10 -1.
<_>
1 0 3 5 2.
<_>
4 5 3 5 2.
<_>
<_>
1 0 4 10 -1.
<_>
1 0 2 5 2.
<_>
3 5 2 5 2.
<_>
<_>
15 3 5 6 -1.
<_>
15 5 5 2 3.
<_>
<_>
9 5 2 15 -1.
<_>
9 10 2 5 3.
<_>
<_>
0 3 5 6 -1.
<_>
0 5 5 2 3.
<_>
<_>
6 0 3 2 -1.
<_>
7 0 1 2 3.
<_>
<_>
12 8 8 2 -1.
<_>
16 8 4 1 2.
<_>
12 9 4 1 2.
<_>
<_>
5 8 12 1 -1.
<_>
9 8 4 1 3.
<_>
<_>
3 13 3 3 -1.
<_>
3 14 3 1 3.
<_>
<_>
5 13 3 2 -1.
<_>
5 14 3 1 2.
<_>
<_>
9 15 3 3 -1.
<_>
9 16 3 1 3.
<_>
<_>
7 15 7 3 -1.
<_>
7 16 7 1 3.
<_>
<_>
3 14 11 6 -1.
<_>
3 16 11 2 3.
<_>
<_>
0 19 14 1 -1.
<_>
7 19 7 1 2.
<_>
<_>
9 17 6 2 -1.
<_>
11 17 2 2 3.
<_>
<_>
12 11 6 2 -1.
<_>
14 11 2 2 3.
<_>
<_>
5 17 6 2 -1.
<_>
7 17 2 2 3.
<_>
<_>
0 1 9 10 -1.
<_>
3 1 3 10 3.
<_>
<_>
10 1 3 3 -1.
<_>
11 1 1 3 3.
<_>
<_>
9 5 6 4 -1.
<_>
9 5 3 4 2.
<_>
<_>
7 1 3 3 -1.
<_>
8 1 1 3 3.
<_>
<_>
0 4 4 11 -1.
<_>
2 4 2 11 2.
<_>
<_>
9 5 6 4 -1.
<_>
9 5 3 4 2.
<_>
<_>
6 0 8 10 -1.
<_>
10 0 4 5 2.
<_>
6 5 4 5 2.
<_>
<_>
6 6 5 14 -1.
<_>
6 13 5 7 2.
<_>
<_>
8 5 4 14 -1.
<_>
8 12 4 7 2.
<_>
<_>
7 7 6 5 -1.
<_>
9 7 2 5 3.
<_>
<_>
9 3 3 9 -1.
<_>
9 6 3 3 3.
<_>
<_>
8 1 3 3 -1.
<_>
9 1 1 3 3.
<_>
<_>
9 6 2 4 -1.
<_>
10 6 1 4 2.
<_>
<_>
10 8 6 9 -1.
<_>
10 8 3 9 2.
<_>
<_>
16 4 3 8 -1.
<_>
17 4 1 8 3.
<_>
<_>
5 9 10 6 -1.
<_>
5 9 5 3 2.
<_>
10 12 5 3 2.
<_>
<_>
5 5 6 4 -1.
<_>
8 5 3 4 2.
<_>
<_>
9 8 4 2 -1.
<_>
9 9 4 1 2.
<_>
<_>
11 7 2 2 -1.
<_>
11 7 1 2 2.
<_>
<_>
8 12 4 8 -1.
<_>
8 12 2 4 2.
<_>
10 16 2 4 2.
<_>
<_>
0 1 4 9 -1.
<_>
0 4 4 3 3.
<_>
<_>
9 10 3 3 -1.
<_>
9 11 3 1 3.
<_>
<_>
8 11 4 2 -1.
<_>
8 12 4 1 2.
<_>
<_>
7 8 4 2 -1.
<_>
7 9 4 1 2.
<_>
<_>
7 8 6 1 -1.
<_>
9 8 2 1 3.
<_>
<_>
16 0 4 9 -1.
<_>
16 0 2 9 2.
<_>
<_>
16 0 3 6 -1.
<_>
16 3 3 3 2.
<_>
<_>
0 0 4 9 -1.
<_>
2 0 2 9 2.
<_>
<_>
1 0 3 6 -1.
<_>
1 3 3 3 2.
<_>
<_>
9 7 6 9 -1.
<_>
11 7 2 9 3.
<_>
<_>
10 6 3 6 -1.
<_>
11 6 1 6 3.
<_>
<_>
1 2 18 2 -1.
<_>
1 2 9 1 2.
<_>
10 3 9 1 2.
<_>
<_>
5 8 6 8 -1.
<_>
7 8 2 8 3.
<_>
<_>
9 0 6 16 -1.
<_>
11 0 2 16 3.
<_>
<_>
14 1 6 18 -1.
<_>
17 1 3 9 2.
<_>
14 10 3 9 2.
<_>
<_>
2 9 2 3 -1.
<_>
2 10 2 1 3.
<_>
<_>
0 1 6 18 -1.
<_>
0 1 3 9 2.
<_>
3 10 3 9 2.
<_>
<_>
11 8 4 12 -1.
<_>
11 8 2 12 2.
<_>
<_>
2 1 18 18 -1.
<_>
2 10 18 9 2.
<_>
<_>
6 3 3 1 -1.
<_>
7 3 1 1 3.
<_>
<_>
4 12 2 2 -1.
<_>
4 13 2 1 2.
<_>
<_>
8 13 5 3 -1.
<_>
8 14 5 1 3.
<_>
<_>
8 14 4 3 -1.
<_>
8 15 4 1 3.
<_>
<_>
3 12 5 3 -1.
<_>
3 13 5 1 3.
<_>
<_>
6 3 3 4 -1.
<_>
7 3 1 4 3.
<_>
<_>
11 10 2 2 -1.
<_>
12 10 1 1 2.
<_>
11 11 1 1 2.
<_>
<_>
5 8 12 1 -1.
<_>
9 8 4 1 3.
<_>
<_>
8 4 4 8 -1.
<_>
10 4 2 8 2.
<_>
<_>
6 6 8 5 -1.
<_>
10 6 4 5 2.
<_>
<_>
10 4 6 4 -1.
<_>
12 4 2 4 3.
<_>
<_>
12 7 2 2 -1.
<_>
13 7 1 1 2.
<_>
12 8 1 1 2.
<_>
<_>
3 5 10 8 -1.
<_>
3 9 10 4 2.
<_>
<_>
7 1 2 12 -1.
<_>
7 7 2 6 2.
<_>
<_>
12 7 2 2 -1.
<_>
13 7 1 1 2.
<_>
12 8 1 1 2.
<_>
<_>
11 13 1 6 -1.
<_>
11 16 1 3 2.
<_>
<_>
5 1 6 15 -1.
<_>
7 1 2 15 3.
<_>
<_>
6 7 2 2 -1.
<_>
6 7 1 1 2.
<_>
7 8 1 1 2.
<_>
<_>
17 5 2 2 -1.
<_>
17 6 2 1 2.
<_>
<_>
10 3 4 10 -1.
<_>
12 3 2 5 2.
<_>
10 8 2 5 2.
<_>
<_>
1 5 2 2 -1.
<_>
1 6 2 1 2.
<_>
<_>
7 10 2 2 -1.
<_>
7 10 1 1 2.
<_>
8 11 1 1 2.
<_>
<_>
3 12 14 4 -1.
<_>
10 12 7 2 2.
<_>
3 14 7 2 2.
<_>
<_>
9 15 3 2 -1.
<_>
9 16 3 1 2.
<_>
<_>
1 13 3 3 -1.
<_>
1 14 3 1 3.
<_>
<_>
0 3 1 2 -1.
<_>
0 4 1 1 2.
<_>
<_>
7 7 6 1 -1.
<_>
9 7 2 1 3.
<_>
<_>
0 4 16 6 -1.
<_>
0 6 16 2 3.
<_>
<_>
9 3 2 14 -1.
<_>
9 10 2 7 2.
<_>
<_>
12 0 4 3 -1.
<_>
12 0 2 3 2.
<_>
<_>
4 18 12 2 -1.
<_>
8 18 4 2 3.
<_>
<_>
4 10 12 4 -1.
<_>
8 10 4 4 3.
<_>
<_>
9 9 2 2 -1.
<_>
9 10 2 1 2.
<_>
<_>
14 1 2 8 -1.
<_>
15 1 1 4 2.
<_>
14 5 1 4 2.
<_>
<_>
3 4 9 1 -1.
<_>
6 4 3 1 3.
<_>
<_>
3 3 4 2 -1.
<_>
3 4 4 1 2.
<_>
<_>
11 15 2 4 -1.
<_>
11 17 2 2 2.
<_>
<_>
14 13 2 6 -1.
<_>
14 15 2 2 3.
<_>
<_>
6 6 1 6 -1.
<_>
6 9 1 3 2.
<_>
<_>
6 10 8 8 -1.
<_>
6 14 8 4 2.
<_>
<_>
8 13 4 3 -1.
<_>
8 14 4 1 3.
<_>
<_>
10 11 4 8 -1.
<_>
10 15 4 4 2.
<_>
<_>
5 11 6 1 -1.
<_>
7 11 2 1 3.
<_>
<_>
5 4 6 10 -1.
<_>
8 4 3 10 2.
<_>
<_>
14 2 6 3 -1.
<_>
14 3 6 1 3.
<_>
<_>
9 12 3 2 -1.
<_>
9 13 3 1 2.
<_>
<_>
8 1 4 6 -1.
<_>
8 3 4 2 3.
<_>
<_>
3 5 13 8 -1.
<_>
3 9 13 4 2.
<_>
<_>
12 5 5 3 -1.
<_>
12 6 5 1 3.
<_>
<_>
5 14 15 6 -1.
<_>
5 16 15 2 3.
<_>
<_>
3 5 5 3 -1.
<_>
3 6 5 1 3.
<_>
<_>
9 14 2 6 -1.
<_>
9 14 1 3 2.
<_>
10 17 1 3 2.
<_>
<_>
9 12 3 2 -1.
<_>
9 13 3 1 2.
<_>
<_>
9 13 3 2 -1.
<_>
9 14 3 1 2.
<_>
<_>
0 2 6 3 -1.
<_>
0 3 6 1 3.
<_>
<_>
0 1 9 11 -1.
<_>
3 1 3 11 3.
<_>
<_>
8 13 4 6 -1.
<_>
10 13 2 3 2.
<_>
8 16 2 3 2.
<_>
<_>
7 13 6 3 -1.
<_>
7 14 6 1 3.
<_>
<_>
3 12 14 4 -1.
<_>
3 12 7 2 2.
<_>
10 14 7 2 2.
<_>
<_>
7 14 1 4 -1.
<_>
7 16 1 2 2.
<_>
<_>
8 13 4 6 -1.
<_>
10 13 2 3 2.
<_>
8 16 2 3 2.
<_>
<_>
10 14 1 3 -1.
<_>
10 15 1 1 3.
<_>
<_>
8 13 4 6 -1.
<_>
8 13 2 3 2.
<_>
10 16 2 3 2.
<_>
<_>
9 14 1 3 -1.
<_>
9 15 1 1 3.
<_>
<_>
10 15 2 3 -1.
<_>
10 16 2 1 3.
<_>
<_>
11 16 1 2 -1.
<_>
11 17 1 1 2.
<_>
<_>
9 0 2 2 -1.
<_>
9 1 2 1 2.
<_>
<_>
0 1 5 8 -1.
<_>
0 5 5 4 2.
<_>
<_>
10 14 2 3 -1.
<_>
10 15 2 1 3.
<_>
<_>
10 13 2 3 -1.
<_>
10 14 2 1 3.
<_>
<_>
0 3 16 6 -1.
<_>
0 6 16 3 2.
<_>
<_>
4 1 2 2 -1.
<_>
5 1 1 2 2.
<_>
<_>
9 7 2 3 -1.
<_>
9 8 2 1 3.
<_>
<_>
10 8 2 12 -1.
<_>
10 12 2 4 3.
<_>
<_>
9 7 2 2 -1.
<_>
10 7 1 2 2.
<_>
<_>
5 0 6 8 -1.
<_>
7 0 2 8 3.
<_>
<_>
9 7 3 6 -1.
<_>
10 7 1 6 3.
<_>
<_>
8 12 10 8 -1.
<_>
8 16 10 4 2.
<_>
<_>
8 7 3 6 -1.
<_>
9 7 1 6 3.
<_>
<_>
4 7 12 2 -1.
<_>
10 7 6 2 2.
<_>
<_>
8 6 8 3 -1.
<_>
8 6 4 3 2.
<_>
<_>
16 15 3 3 -1.
<_>
16 16 3 1 3.
<_>
<_>
4 6 12 3 -1.
<_>
10 6 6 3 2.
<_>
<_>
7 8 3 5 -1.
<_>
8 8 1 5 3.
<_>
<_>
0 10 20 2 -1.
<_>
10 10 10 1 2.
<_>
0 11 10 1 2.
<_>
<_>
11 16 9 4 -1.
<_>
14 16 3 4 3.
<_>
<_>
0 5 3 4 -1.
<_>
1 5 1 4 3.
<_>
<_>
8 15 4 2 -1.
<_>
8 15 2 1 2.
<_>
10 16 2 1 2.
<_>
<_>
1 8 19 3 -1.
<_>
1 9 19 1 3.
<_>
<_>
15 16 3 3 -1.
<_>
15 17 3 1 3.
<_>
<_>
0 4 20 10 -1.
<_>
0 4 10 5 2.
<_>
10 9 10 5 2.
<_>
<_>
2 14 7 6 -1.
<_>
2 16 7 2 3.
<_>
<_>
8 6 6 6 -1.
<_>
10 6 2 6 3.
<_>
<_>
16 4 4 6 -1.
<_>
16 6 4 2 3.
<_>
<_>
7 13 6 3 -1.
<_>
7 14 6 1 3.
<_>
<_>
7 13 4 3 -1.
<_>
7 14 4 1 3.
<_>
<_>
13 13 6 2 -1.
<_>
13 14 6 1 2.
<_>
<_>
14 12 2 3 -1.
<_>
14 13 2 1 3.
<_>
<_>
1 13 6 2 -1.
<_>
1 14 6 1 2.
<_>
<_>
4 12 2 3 -1.
<_>
4 13 2 1 3.
<_>
<_>
17 4 3 5 -1.
<_>
18 4 1 5 3.
<_>
<_>
5 5 14 8 -1.
<_>
12 5 7 4 2.
<_>
5 9 7 4 2.
<_>
<_>
6 8 6 5 -1.
<_>
8 8 2 5 3.
<_>
<_>
0 4 4 6 -1.
<_>
0 6 4 2 3.
<_>
<_>
9 1 3 6 -1.
<_>
10 1 1 6 3.
<_>
<_>
10 4 6 3 -1.
<_>
10 5 6 1 3.
<_>
<_>
8 1 3 6 -1.
<_>
9 1 1 6 3.
<_>
<_>
4 4 6 3 -1.
<_>
4 5 6 1 3.
<_>
<_>
12 4 3 3 -1.
<_>
12 5 3 1 3.
<_>
<_>
12 11 4 2 -1.
<_>
12 12 4 1 2.
<_>
<_>
0 2 20 6 -1.
<_>
0 2 10 3 2.
<_>
10 5 10 3 2.
<_>
<_>
5 4 3 3 -1.
<_>
5 5 3 1 3.
<_>
<_>
2 10 16 4 -1.
<_>
10 10 8 2 2.
<_>
2 12 8 2 2.
<_>
<_>
3 10 16 6 -1.
<_>
11 10 8 3 2.
<_>
3 13 8 3 2.
<_>
<_>
1 10 16 6 -1.
<_>
1 10 8 3 2.
<_>
9 13 8 3 2.
<_>
<_>
4 7 2 4 -1.
<_>
5 7 1 4 2.
<_>
<_>
11 16 9 4 -1.
<_>
14 16 3 4 3.
<_>
<_>
3 16 14 4 -1.
<_>
10 16 7 2 2.
<_>
3 18 7 2 2.
<_>
<_>
0 16 9 4 -1.
<_>
3 16 3 4 3.
<_>
<_>
1 14 6 6 -1.
<_>
1 14 3 3 2.
<_>
4 17 3 3 2.
<_>
<_>
9 0 2 1 -1.
<_>
9 0 1 1 2.
<_>
<_>
6 7 8 10 -1.
<_>
10 7 4 5 2.
<_>
6 12 4 5 2.
<_>
<_>
2 15 1 2 -1.
<_>
2 16 1 1 2.
<_>
<_>
0 14 7 6 -1.
<_>
0 16 7 2 3.
<_>
<_>
7 8 6 2 -1.
<_>
7 9 6 1 2.
<_>
<_>
9 2 2 15 -1.
<_>
9 7 2 5 3.
<_>
<_>
5 6 2 2 -1.
<_>
5 7 2 1 2.
<_>
<_>
6 6 8 3 -1.
<_>
6 7 8 1 3.
<_>
<_>
12 13 5 6 -1.
<_>
12 15 5 2 3.
<_>
<_>
0 0 20 18 -1.
<_>
0 9 20 9 2.
<_>
<_>
5 1 6 6 -1.
<_>
7 1 2 6 3.
<_>
<_>
5 1 4 9 -1.
<_>
7 1 2 9 2.
<_>
<_>
1 19 18 1 -1.
<_>
7 19 6 1 3.
<_>
<_>
14 16 5 2 -1.
<_>
14 17 5 1 2.
<_>
<_>
0 5 15 10 -1.
<_>
0 10 15 5 2.
<_>
<_>
7 15 4 2 -1.
<_>
7 15 2 1 2.
<_>
9 16 2 1 2.
<_>
<_>
14 11 2 2 -1.
<_>
14 12 2 1 2.
<_>
<_>
9 8 3 3 -1.
<_>
9 9 3 1 3.
<_>
<_>
4 11 2 2 -1.
<_>
4 12 2 1 2.
<_>
<_>
8 8 3 3 -1.
<_>
8 9 3 1 3.
<_>
<_>
9 10 2 3 -1.
<_>
9 11 2 1 3.
<_>
<_>
8 8 4 3 -1.
<_>
8 9 4 1 3.
<_>
<_>
1 9 4 10 -1.
<_>
1 9 2 5 2.
<_>
3 14 2 5 2.
<_>
<_>
0 12 6 8 -1.
<_>
2 12 2 8 3.
<_>
<_>
9 1 4 2 -1.
<_>
11 1 2 1 2.
<_>
9 2 2 1 2.
<_>
<_>
12 13 7 6 -1.
<_>
12 15 7 2 3.
<_>
<_>
7 0 2 3 -1.
<_>
7 1 2 1 3.
<_>
<_>
7 14 6 3 -1.
<_>
9 14 2 3 3.
<_>
<_>
9 6 6 4 -1.
<_>
11 6 2 4 3.
<_>
<_>
8 10 8 3 -1.
<_>
8 10 4 3 2.
<_>
<_>
6 10 4 3 -1.
<_>
8 10 2 3 2.
<_>
<_>
6 8 3 5 -1.
<_>
7 8 1 5 3.
<_>
<_>
0 4 8 1 -1.
<_>
4 4 4 1 2.
<_>
<_>
8 2 2 6 -1.
<_>
8 2 1 3 2.
<_>
9 5 1 3 2.
<_>
<_>
0 7 20 6 -1.
<_>
0 9 20 2 3.
<_>
<_>
12 10 3 6 -1.
<_>
12 13 3 3 2.
<_>
<_>
8 15 1 4 -1.
<_>
8 17 1 2 2.
<_>
<_>
5 16 2 4 -1.
<_>
5 18 2 2 2.
<_>
<_>
6 2 8 12 -1.
<_>
6 6 8 4 3.
<_>
<_>
4 7 12 2 -1.
<_>
8 7 4 2 3.
<_>
<_>
7 0 6 1 -1.
<_>
9 0 2 1 3.
<_>
<_>
8 11 3 3 -1.
<_>
8 12 3 1 3.
<_>
<_>
12 11 3 6 -1.
<_>
12 14 3 3 2.
<_>
<_>
11 2 6 10 -1.
<_>
14 2 3 5 2.
<_>
11 7 3 5 2.
<_>
<_>
5 7 10 12 -1.
<_>
5 7 5 6 2.
<_>
10 13 5 6 2.
<_>
<_>
4 4 2 10 -1.
<_>
4 9 2 5 2.
<_>
<_>
9 7 2 3 -1.
<_>
9 7 1 3 2.
<_>
<_>
11 9 6 2 -1.
<_>
11 9 3 2 2.
<_>
<_>
4 7 2 2 -1.
<_>
5 7 1 2 2.
<_>
<_>
0 2 4 6 -1.
<_>
0 4 4 2 3.
<_>
<_>
10 7 3 4 -1.
<_>
11 7 1 4 3.
<_>
<_>
9 7 3 5 -1.
<_>
10 7 1 5 3.
<_>
<_>
9 1 1 3 -1.
<_>
9 2 1 1 3.
<_>
<_>
0 6 16 6 -1.
<_>
0 6 8 3 2.
<_>
8 9 8 3 2.
<_>
<_>
10 15 3 3 -1.
<_>
10 16 3 1 3.
<_>
<_>
9 14 4 3 -1.
<_>
9 15 4 1 3.
<_>
<_>
3 2 6 10 -1.
<_>
3 2 3 5 2.
<_>
6 7 3 5 2.
<_>
<_>
3 0 14 2 -1.
<_>
3 1 14 1 2.
<_>
<_>
9 14 3 3 -1.
<_>
9 15 3 1 3.
<_>
<_>
10 15 3 3 -1.
<_>
10 16 3 1 3.
<_>
<_>
9 13 2 6 -1.
<_>
9 16 2 3 2.
<_>
<_>
7 13 6 3 -1.
<_>
7 14 6 1 3.
<_>
<_>
12 11 3 6 -1.
<_>
12 14 3 3 2.
<_>
<_>
8 12 5 2 -1.
<_>
8 13 5 1 2.
<_>
<_>
5 11 3 6 -1.
<_>
5 14 3 3 2.
<_>
<_>
8 12 3 2 -1.
<_>
8 13 3 1 2.
<_>
<_>
11 13 7 6 -1.
<_>
11 15 7 2 3.
<_>
<_>
7 14 6 3 -1.
<_>
7 15 6 1 3.
<_>
<_>
3 13 14 4 -1.
<_>
3 13 7 2 2.
<_>
10 15 7 2 2.
<_>
<_>
8 14 4 6 -1.
<_>
8 14 2 3 2.
<_>
10 17 2 3 2.
<_>
<_>
8 15 4 3 -1.
<_>
8 16 4 1 3.
<_>
<_>
7 16 6 2 -1.
<_>
9 16 2 2 3.
<_>
<_>
7 7 6 2 -1.
<_>
7 8 6 1 2.
<_>
<_>
3 9 13 3 -1.
<_>
3 10 13 1 3.
<_>
<_>
9 8 3 4 -1.
<_>
9 10 3 2 2.
<_>
<_>
8 10 4 3 -1.
<_>
8 11 4 1 3.
<_>
<_>
7 7 3 4 -1.
<_>
8 7 1 4 3.
<_>
<_>
8 7 3 5 -1.
<_>
9 7 1 5 3.
<_>
<_>
12 3 3 4 -1.
<_>
13 3 1 4 3.
<_>
<_>
9 7 2 3 -1.
<_>
9 7 1 3 2.
<_>
<_>
5 3 3 4 -1.
<_>
6 3 1 4 3.
<_>
<_>
3 7 12 1 -1.
<_>
7 7 4 1 3.
<_>
<_>
12 5 3 3 -1.
<_>
12 6 3 1 3.
<_>
<_>
11 2 6 2 -1.
<_>
11 3 6 1 2.
<_>
<_>
3 2 14 2 -1.
<_>
3 2 7 1 2.
<_>
10 3 7 1 2.
<_>
<_>
6 1 7 14 -1.
<_>
6 8 7 7 2.
<_>
<_>
8 0 12 5 -1.
<_>
8 0 6 5 2.
<_>
<_>
1 9 18 1 -1.
<_>
7 9 6 1 3.
<_>
<_>
0 0 10 5 -1.
<_>
5 0 5 5 2.
<_>
<_>
2 5 8 15 -1.
<_>
2 10 8 5 3.
<_>
<_>
12 5 3 3 -1.
<_>
12 6 3 1 3.
<_>
<_>
13 4 2 3 -1.
<_>
13 5 2 1 3.
<_>
<_>
2 15 4 3 -1.
<_>
2 16 4 1 3.
<_>
<_>
5 6 10 3 -1.
<_>
10 6 5 3 2.
<_>
<_>
11 6 2 2 -1.
<_>
12 6 1 1 2.
<_>
11 7 1 1 2.
<_>
<_>
12 4 4 3 -1.
<_>
12 5 4 1 3.
<_>
<_>
7 6 2 2 -1.
<_>
7 6 1 1 2.
<_>
8 7 1 1 2.
<_>
<_>
4 4 4 3 -1.
<_>
4 5 4 1 3.
<_>
<_>
11 4 3 3 -1.
<_>
12 4 1 3 3.
<_>
<_>
9 3 2 1 -1.
<_>
9 3 1 1 2.
<_>
<_>
4 5 5 3 -1.
<_>
4 6 5 1 3.
<_>
<_>
4 6 4 3 -1.
<_>
4 7 4 1 3.
<_>
<_>
11 4 3 3 -1.
<_>
12 4 1 3 3.
<_>
<_>
8 8 4 3 -1.
<_>
8 9 4 1 3.
<_>
<_>
6 4 3 3 -1.
<_>
7 4 1 3 3.
<_>
<_>
4 14 1 3 -1.
<_>
4 15 1 1 3.
<_>
<_>
9 7 2 3 -1.
<_>
9 7 1 3 2.
<_>
<_>
17 0 3 2 -1.
<_>
17 1 3 1 2.
<_>
<_>
8 10 2 9 -1.
<_>
8 13 2 3 3.
<_>
<_>
0 8 18 2 -1.
<_>
0 9 18 1 2.
<_>
<_>
9 15 2 3 -1.
<_>
9 16 2 1 3.
<_>
<_>
8 7 4 3 -1.
<_>
8 8 4 1 3.
<_>
<_>
1 14 6 6 -1.
<_>
1 14 3 3 2.
<_>
4 17 3 3 2.
<_>
<_>
0 18 6 2 -1.
<_>
0 19 6 1 2.
<_>
<_>
12 9 4 3 -1.
<_>
12 9 2 3 2.
<_>
<_>
9 8 3 8 -1.
<_>
10 8 1 8 3.
<_>
<_>
4 9 4 3 -1.
<_>
6 9 2 3 2.
<_>
<_>
4 18 6 1 -1.
<_>
6 18 2 1 3.
<_>
<_>
9 7 3 2 -1.
<_>
10 7 1 2 3.
<_>
<_>
6 7 8 12 -1.
<_>
10 7 4 6 2.
<_>
6 13 4 6 2.
<_>
<_>
8 7 3 2 -1.
<_>
9 7 1 2 3.
<_>
<_>
8 7 3 6 -1.
<_>
9 7 1 6 3.
<_>
<_>
3 16 14 4 -1.
<_>
10 16 7 2 2.
<_>
3 18 7 2 2.
<_>
<_>
1 14 18 4 -1.
<_>
10 14 9 2 2.
<_>
1 16 9 2 2.
<_>
<_>
8 7 3 3 -1.
<_>
8 8 3 1 3.
<_>
<_>
0 4 20 12 -1.
<_>
0 4 10 6 2.
<_>
10 10 10 6 2.
<_>
<_>
5 5 10 12 -1.
<_>
10 5 5 6 2.
<_>
5 11 5 6 2.
<_>
<_>
10 2 4 7 -1.
<_>
10 2 2 7 2.
<_>
<_>
8 11 4 3 -1.
<_>
8 12 4 1 3.
<_>
<_>
8 12 3 3 -1.
<_>
8 13 3 1 3.
<_>
<_>
13 13 5 6 -1.
<_>
13 15 5 2 3.
<_>
<_>
7 0 6 6 -1.
<_>
9 0 2 6 3.
<_>
<_>
2 13 5 6 -1.
<_>
2 15 5 2 3.
<_>
<_>
0 4 2 12 -1.
<_>
0 4 1 6 2.
<_>
1 10 1 6 2.
<_>
<_>
9 19 3 1 -1.
<_>
10 19 1 1 3.
<_>
<_>
18 0 2 6 -1.
<_>
18 2 2 2 3.
<_>
<_>
0 3 1 6 -1.
<_>
0 5 1 2 3.
<_>
<_>
0 0 3 6 -1.
<_>
0 2 3 2 3.
<_>
<_>
17 2 3 7 -1.
<_>
18 2 1 7 3.
<_>
<_>
10 3 4 7 -1.
<_>
10 3 2 7 2.
<_>
<_>
0 2 3 7 -1.
<_>
1 2 1 7 3.
<_>
<_>
6 2 4 8 -1.
<_>
8 2 2 8 2.
<_>
<_>
13 0 1 4 -1.
<_>
13 2 1 2 2.
<_>
<_>
5 1 12 5 -1.
<_>
9 1 4 5 3.
<_>
<_>
6 0 1 4 -1.
<_>
6 2 1 2 2.
<_>
<_>
3 1 12 5 -1.
<_>
7 1 4 5 3.
<_>
<_>
9 12 3 8 -1.
<_>
10 12 1 8 3.
<_>
<_>
7 13 6 1 -1.
<_>
9 13 2 1 3.
<_>
<_>
7 14 6 3 -1.
<_>
7 15 6 1 3.
<_>
<_>
5 16 7 3 -1.
<_>
5 17 7 1 3.
<_>
<_>
0 12 20 6 -1.
<_>
0 14 20 2 3.
<_>
<_>
4 18 14 2 -1.
<_>
4 19 14 1 2.
<_>
<_>
8 12 3 8 -1.
<_>
9 12 1 8 3.
<_>
<_>
7 13 3 3 -1.
<_>
7 14 3 1 3.
<_>
<_>
5 5 12 10 -1.
<_>
11 5 6 5 2.
<_>
5 10 6 5 2.
<_>
<_>
8 1 5 10 -1.
<_>
8 6 5 5 2.
<_>
<_>
5 4 9 12 -1.
<_>
5 10 9 6 2.
<_>
<_>
7 13 6 6 -1.
<_>
7 15 6 2 3.
<_>
<_>
8 4 5 16 -1.
<_>
8 12 5 8 2.
<_>
<_>
8 12 4 6 -1.
<_>
8 15 4 3 2.
<_>
<_>
7 13 2 2 -1.
<_>
7 13 1 1 2.
<_>
8 14 1 1 2.
<_>
<_>
7 12 2 2 -1.
<_>
7 12 1 1 2.
<_>
8 13 1 1 2.
<_>
<_>
18 0 2 14 -1.
<_>
18 0 1 14 2.
<_>
<_>
12 11 7 2 -1.
<_>
12 12 7 1 2.
<_>
<_>
1 18 1 2 -1.
<_>
1 19 1 1 2.
<_>
<_>
2 18 1 2 -1.
<_>
2 19 1 1 2.
<_>
<_>
9 7 2 1 -1.
<_>
9 7 1 1 2.
<_>
<_>
9 6 2 3 -1.
<_>
9 6 1 3 2.
<_>
<_>
3 1 2 2 -1.
<_>
4 1 1 2 2.
<_>
<_>
3 0 3 2 -1.
<_>
3 1 3 1 2.
<_>
<_>
12 10 3 4 -1.
<_>
12 12 3 2 2.
<_>
<_>
7 7 8 2 -1.
<_>
7 8 8 1 2.
<_>
<_>
8 8 3 4 -1.
<_>
8 10 3 2 2.
<_>
<_>
7 12 6 3 -1.
<_>
7 13 6 1 3.
<_>
<_>
0 2 10 3 -1.
<_>
5 2 5 3 2.
<_>
<_>
0 1 20 6 -1.
<_>
0 3 20 2 3.
<_>
<_>
7 6 6 3 -1.
<_>
9 6 2 3 3.
<_>
<_>
3 7 14 4 -1.
<_>
3 9 14 2 2.
<_>
<_>
5 7 3 6 -1.
<_>
5 9 3 2 3.
<_>
<_>
8 8 3 12 -1.
<_>
8 12 3 4 3.
<_>
<_>
9 17 6 2 -1.
<_>
12 17 3 1 2.
<_>
9 18 3 1 2.
<_>
<_>
10 17 4 3 -1.
<_>
10 18 4 1 3.
<_>
<_>
4 2 4 2 -1.
<_>
4 3 4 1 2.
<_>
<_>
7 3 6 14 -1.
<_>
9 3 2 14 3.
<_>
<_>
15 13 1 6 -1.
<_>
15 16 1 3 2.
<_>
<_>
13 14 2 6 -1.
<_>
13 16 2 2 3.
<_>
<_>
4 11 5 6 -1.
<_>
4 14 5 3 2.
<_>
<_>
4 17 4 2 -1.
<_>
6 17 2 2 2.
<_>
<_>
0 6 20 2 -1.
<_>
0 6 10 2 2.
<_>
<_>
6 5 10 12 -1.
<_>
11 5 5 6 2.
<_>
6 11 5 6 2.
<_>
<_>
4 0 2 12 -1.
<_>
4 0 1 6 2.
<_>
5 6 1 6 2.
<_>
<_>
4 1 6 2 -1.
<_>
6 1 2 2 3.
<_>
<_>
13 7 2 1 -1.
<_>
13 7 1 1 2.
<_>
<_>
5 5 15 6 -1.
<_>
5 7 15 2 3.
<_>
<_>
1 10 18 2 -1.
<_>
1 10 9 1 2.
<_>
10 11 9 1 2.
<_>
<_>
1 6 15 7 -1.
<_>
6 6 5 7 3.
<_>
<_>
8 14 4 3 -1.
<_>
8 15 4 1 3.
<_>
<_>
9 14 3 3 -1.
<_>
9 15 3 1 3.
<_>
<_>
8 14 4 3 -1.
<_>
8 15 4 1 3.
<_>
<_>
8 13 3 2 -1.
<_>
8 14 3 1 2.
<_>
<_>
15 14 5 3 -1.
<_>
15 15 5 1 3.
<_>
<_>
0 14 20 1 -1.
<_>
0 14 10 1 2.
<_>
<_>
0 14 6 3 -1.
<_>
0 15 6 1 3.
<_>
<_>
5 3 4 2 -1.
<_>
5 4 4 1 2.
<_>
<_>
0 6 20 1 -1.
<_>
0 6 10 1 2.
<_>
<_>
6 3 10 14 -1.
<_>
11 3 5 7 2.
<_>
6 10 5 7 2.
<_>
<_>
8 12 4 2 -1.
<_>
8 13 4 1 2.
<_>
<_>
6 3 8 6 -1.
<_>
6 3 4 3 2.
<_>
10 6 4 3 2.
<_>
<_>
13 7 2 1 -1.
<_>
13 7 1 1 2.
<_>
<_>
6 3 10 14 -1.
<_>
11 3 5 7 2.
<_>
6 10 5 7 2.
<_>
<_>
5 7 2 1 -1.
<_>
6 7 1 1 2.
<_>
<_>
4 3 10 14 -1.
<_>
4 3 5 7 2.
<_>
9 10 5 7 2.
<_>
<_>
9 7 2 2 -1.
<_>
9 7 1 2 2.
<_>
<_>
0 3 20 1 -1.
<_>
0 3 10 1 2.
<_>
<_>
2 1 10 3 -1.
<_>
2 2 10 1 3.
<_>
<_>
9 7 2 2 -1.
<_>
10 7 1 2 2.
<_>
<_>
9 17 3 2 -1.
<_>
10 17 1 2 3.
<_>
<_>
9 7 3 6 -1.
<_>
10 7 1 6 3.
<_>
<_>
8 17 3 2 -1.
<_>
9 17 1 2 3.
<_>
<_>
8 7 3 6 -1.
<_>
9 7 1 6 3.
<_>
<_>
16 3 4 6 -1.
<_>
16 5 4 2 3.
<_>
<_>
15 6 2 12 -1.
<_>
16 6 1 6 2.
<_>
15 12 1 6 2.
<_>
<_>
1 4 18 10 -1.
<_>
1 4 9 5 2.
<_>
10 9 9 5 2.
<_>
<_>
9 4 2 4 -1.
<_>
9 6 2 2 2.
<_>
<_>
12 5 3 2 -1.
<_>
12 6 3 1 2.
<_>
<_>
5 12 10 4 -1.
<_>
5 14 10 2 2.
<_>
<_>
5 5 3 2 -1.
<_>
5 6 3 1 2.
<_>
<_>
4 6 12 6 -1.
<_>
8 6 4 6 3.
<_>
<_>
14 4 6 6 -1.
<_>
14 6 6 2 3.
<_>
<_>
16 0 4 6 -1.
<_>
18 0 2 3 2.
<_>
16 3 2 3 2.
<_>
<_>
0 4 6 6 -1.
<_>
0 6 6 2 3.
<_>
<_>
0 0 4 6 -1.
<_>
0 0 2 3 2.
<_>
2 3 2 3 2.
<_>
<_>
12 0 8 5 -1.
<_>
12 0 4 5 2.
<_>
<_>
16 0 4 17 -1.
<_>
16 0 2 17 2.
<_>
<_>
1 0 18 20 -1.
<_>
7 0 6 20 3.
<_>
<_>
6 0 2 5 -1.
<_>
7 0 1 5 2.
<_>
<_>
0 6 20 1 -1.
<_>
0 6 10 1 2.
<_>
<_>
8 7 6 4 -1.
<_>
10 7 2 4 3.
<_>
<_>
1 1 16 4 -1.
<_>
1 1 8 2 2.
<_>
9 3 8 2 2.
<_>
<_>
7 2 4 2 -1.
<_>
7 2 2 1 2.
<_>
9 3 2 1 2.
<_>
<_>
7 4 9 3 -1.
<_>
7 5 9 1 3.
<_>
<_>
10 4 5 12 -1.
<_>
10 10 5 6 2.
<_>
<_>
3 12 2 3 -1.
<_>
3 13 2 1 3.
<_>
<_>
8 8 3 5 -1.
<_>
9 8 1 5 3.
<_>
<_>
13 9 2 3 -1.
<_>
13 9 1 3 2.
<_>
<_>
15 11 2 2 -1.
<_>
15 12 2 1 2.
<_>
<_>
5 6 2 3 -1.
<_>
5 7 2 1 3.
<_>
<_>
2 11 6 2 -1.
<_>
2 12 6 1 2.
<_>
<_>
15 11 4 3 -1.
<_>
15 12 4 1 3.
<_>
<_>
16 0 4 17 -1.
<_>
16 0 2 17 2.
<_>
<_>
1 11 4 3 -1.
<_>
1 12 4 1 3.
<_>
<_>
9 11 1 3 -1.
<_>
9 12 1 1 3.
<_>
<_>
10 9 6 7 -1.
<_>
10 9 3 7 2.
<_>
<_>
8 15 4 2 -1.
<_>
8 16 4 1 2.
<_>
<_>
4 9 6 7 -1.
<_>
7 9 3 7 2.
<_>
<_>
9 14 2 3 -1.
<_>
9 15 2 1 3.
<_>
<_>
0 2 20 2 -1.
<_>
10 2 10 1 2.
<_>
0 3 10 1 2.
<_>
<_>
6 7 8 2 -1.
<_>
6 8 8 1 2.
<_>
<_>
0 2 20 2 -1.
<_>
0 2 10 1 2.
<_>
10 3 10 1 2.
<_>
<_>
3 1 2 10 -1.
<_>
3 1 1 5 2.
<_>
4 6 1 5 2.
<_>
<_>
13 4 1 10 -1.
<_>
13 9 1 5 2.
<_>
<_>
9 8 4 3 -1.
<_>
9 9 4 1 3.
<_>
<_>
2 11 16 4 -1.
<_>
2 11 8 2 2.
<_>
10 13 8 2 2.
<_>
<_>
5 1 3 5 -1.
<_>
6 1 1 5 3.
<_>
<_>
9 10 2 3 -1.
<_>
9 11 2 1 3.
<_>
<_>
9 11 2 2 -1.
<_>
9 12 2 1 2.
<_>
<_>
0 10 20 2 -1.
<_>
0 11 20 1 2.
<_>
<_>
1 7 6 4 -1.
<_>
1 7 3 2 2.
<_>
4 9 3 2 2.
<_>
<_>
12 0 8 8 -1.
<_>
16 0 4 4 2.
<_>
12 4 4 4 2.
<_>
<_>
14 1 6 4 -1.
<_>
16 1 2 4 3.
<_>
<_>
6 3 2 14 -1.
<_>
6 10 2 7 2.
<_>
<_>
6 1 7 12 -1.
<_>
6 7 7 6 2.
<_>
<_>
5 0 15 5 -1.
<_>
10 0 5 5 3.
<_>
<_>
15 0 4 10 -1.
<_>
15 0 2 10 2.
<_>
<_>
1 0 18 3 -1.
<_>
7 0 6 3 3.
<_>
<_>
0 0 17 2 -1.
<_>
0 1 17 1 2.
<_>
<_>
10 0 3 3 -1.
<_>
11 0 1 3 3.
<_>
<_>
10 0 3 12 -1.
<_>
11 0 1 12 3.
<_>
<_>
1 3 4 16 -1.
<_>
1 3 2 8 2.
<_>
3 11 2 8 2.
<_>
<_>
7 0 3 3 -1.
<_>
8 0 1 3 3.
<_>
<_>
9 13 2 6 -1.
<_>
9 16 2 3 2.
<_>
<_>
9 0 6 13 -1.
<_>
11 0 2 13 3.
<_>
<_>
7 7 3 2 -1.
<_>
8 7 1 2 3.
<_>
<_>
8 2 1 12 -1.
<_>
8 6 1 4 3.
<_>
<_>
4 10 12 6 -1.
<_>
10 10 6 3 2.
<_>
4 13 6 3 2.
<_>
<_>
13 5 2 3 -1.
<_>
13 6 2 1 3.
<_>
<_>
4 10 12 6 -1.
<_>
4 10 6 3 2.
<_>
10 13 6 3 2.
<_>
<_>
5 5 2 3 -1.
<_>
5 6 2 1 3.
<_>
<_>
8 6 6 7 -1.
<_>
10 6 2 7 3.
<_>
<_>
9 6 2 4 -1.
<_>
9 6 1 4 2.
<_>
<_>
6 6 6 7 -1.
<_>
8 6 2 7 3.
<_>
<_>
9 6 2 4 -1.
<_>
10 6 1 4 2.
<_>
<_>
12 9 2 3 -1.
<_>
12 9 1 3 2.
<_>
<_>
0 6 20 1 -1.
<_>
0 6 10 1 2.
<_>
<_>
5 7 10 2 -1.
<_>
10 7 5 2 2.
<_>
<_>
1 16 4 3 -1.
<_>
1 17 4 1 3.
<_>
<_>
12 4 3 3 -1.
<_>
12 5 3 1 3.
<_>
<_>
10 3 5 3 -1.
<_>
10 4 5 1 3.
<_>
<_>
3 9 14 8 -1.
<_>
3 9 7 4 2.
<_>
10 13 7 4 2.
<_>
<_>
6 8 8 10 -1.
<_>
6 8 4 5 2.
<_>
10 13 4 5 2.
<_>
<_>
12 4 3 3 -1.
<_>
12 5 3 1 3.
<_>
<_>
10 3 5 3 -1.
<_>
10 4 5 1 3.
<_>
<_>
5 4 3 3 -1.
<_>
5 5 3 1 3.
<_>
<_>
5 3 5 3 -1.
<_>
5 4 5 1 3.
<_>
<_>
13 16 2 3 -1.
<_>
13 17 2 1 3.
<_>
<_>
0 5 20 6 -1.
<_>
0 7 20 2 3.
<_>
<_>
3 14 3 3 -1.
<_>
3 15 3 1 3.
<_>
<_>
7 15 5 3 -1.
<_>
7 16 5 1 3.
<_>
<_>
12 9 2 3 -1.
<_>
12 9 1 3 2.
<_>
<_>
15 13 2 6 -1.
<_>
15 13 1 6 2.
<_>
<_>
6 9 2 3 -1.
<_>
7 9 1 3 2.
<_>
<_>
3 13 2 6 -1.
<_>
4 13 1 6 2.
<_>
<_>
11 4 2 4 -1.
<_>
11 4 1 4 2.
<_>
<_>
13 4 2 5 -1.
<_>
13 4 1 5 2.
<_>
<_>
7 4 2 4 -1.
<_>
8 4 1 4 2.
<_>
<_>
5 4 2 5 -1.
<_>
6 4 1 5 2.
<_>
<_>
19 6 1 2 -1.
<_>
19 7 1 1 2.
<_>
<_>
12 7 8 13 -1.
<_>
12 7 4 13 2.
<_>
<_>
0 6 1 2 -1.
<_>
0 7 1 1 2.
<_>
<_>
6 15 4 3 -1.
<_>
6 16 4 1 3.
<_>
<_>
11 8 2 2 -1.
<_>
11 9 2 1 2.
<_>
<_>
11 7 2 4 -1.
<_>
11 7 1 4 2.
<_>
<_>
4 13 2 3 -1.
<_>
4 14 2 1 3.
<_>
<_>
0 17 18 3 -1.
<_>
6 17 6 3 3.
<_>
<_>
1 0 18 5 -1.
<_>
7 0 6 5 3.
<_>
<_>
5 7 3 4 -1.
<_>
5 9 3 2 2.
<_>
<_>
10 6 2 2 -1.
<_>
10 6 1 2 2.
<_>
<_>
6 4 14 4 -1.
<_>
13 4 7 2 2.
<_>
6 6 7 2 2.
<_>
<_>
5 16 6 4 -1.
<_>
5 16 3 2 2.
<_>
8 18 3 2 2.
<_>
<_>
7 15 2 4 -1.
<_>
7 17 2 2 2.
<_>
<_>
8 5 5 14 -1.
<_>
8 12 5 7 2.
<_>
<_>
9 9 2 2 -1.
<_>
9 10 2 1 2.
<_>
<_>
7 5 3 7 -1.
<_>
8 5 1 7 3.
<_>
<_>
0 0 3 9 -1.
<_>
0 3 3 3 3.
<_>
<_>
8 6 8 8 -1.
<_>
12 6 4 4 2.
<_>
8 10 4 4 2.
<_>
<_>
4 8 13 2 -1.
<_>
4 9 13 1 2.
<_>
<_>
4 3 6 1 -1.
<_>
6 3 2 1 3.
<_>
<_>
9 1 2 6 -1.
<_>
9 3 2 2 3.
<_>
<_>
10 5 6 4 -1.
<_>
12 5 2 4 3.
<_>
<_>
9 5 2 12 -1.
<_>
9 9 2 4 3.
<_>
<_>
8 14 4 3 -1.
<_>
8 15 4 1 3.
<_>
<_>
8 12 4 3 -1.
<_>
8 13 4 1 3.
<_>
<_>
10 3 6 7 -1.
<_>
12 3 2 7 3.
<_>
<_>
3 10 16 6 -1.
<_>
3 12 16 2 3.
<_>
<_>
5 5 3 10 -1.
<_>
5 10 3 5 2.
<_>
<_>
6 10 3 6 -1.
<_>
6 13 3 3 2.
<_>
<_>
17 2 2 12 -1.
<_>
17 2 1 12 2.
<_>
<_>
16 6 2 14 -1.
<_>
16 13 2 7 2.
<_>
<_>
3 11 12 9 -1.
<_>
3 14 12 3 3.
<_>
<_>
0 2 4 12 -1.
<_>
2 2 2 12 2.
<_>
<_>
18 0 2 18 -1.
<_>
18 0 1 18 2.
<_>
<_>
16 12 3 2 -1.
<_>
16 13 3 1 2.
<_>
<_>
0 2 2 15 -1.
<_>
1 2 1 15 2.
<_>
<_>
1 10 2 4 -1.
<_>
1 12 2 2 2.
<_>
<_>
11 1 2 18 -1.
<_>
11 1 1 18 2.
<_>
<_>
3 2 14 2 -1.
<_>
10 2 7 1 2.
<_>
3 3 7 1 2.
<_>
<_>
7 1 2 18 -1.
<_>
8 1 1 18 2.
<_>
<_>
6 1 8 12 -1.
<_>
6 7 8 6 2.
<_>
<_>
8 14 4 3 -1.
<_>
8 15 4 1 3.
<_>
<_>
7 14 6 3 -1.
<_>
7 15 6 1 3.
<_>
<_>
0 13 5 2 -1.
<_>
0 14 5 1 2.
<_>
<_>
9 0 2 6 -1.
<_>
9 0 1 3 2.
<_>
10 3 1 3 2.
<_>
<_>
9 0 2 6 -1.
<_>
10 0 1 3 2.
<_>
9 3 1 3 2.
<_>
<_>
9 7 3 6 -1.
<_>
10 7 1 6 3.
<_>
<_>
9 0 2 6 -1.
<_>
9 0 1 3 2.
<_>
10 3 1 3 2.
<_>
<_>
8 7 3 6 -1.
<_>
9 7 1 6 3.
<_>
<_>
9 6 2 6 -1.
<_>
9 6 1 6 2.
<_>
<_>
9 4 4 3 -1.
<_>
9 4 2 3 2.
<_>
<_>
0 4 4 3 -1.
<_>
0 5 4 1 3.
<_>
<_>
8 7 4 2 -1.
<_>
8 8 4 1 2.
<_>
<_>
10 6 6 3 -1.
<_>
12 6 2 3 3.
<_>
<_>
9 6 3 12 -1.
<_>
9 10 3 4 3.
<_>
<_>
5 4 2 3 -1.
<_>
5 5 2 1 3.
<_>
<_>
5 6 1 3 -1.
<_>
5 7 1 1 3.
<_>
<_>
9 17 3 2 -1.
<_>
10 17 1 2 3.
<_>
<_>
0 7 20 2 -1.
<_>
0 8 20 1 2.
<_>
<_>
4 3 6 7 -1.
<_>
6 3 2 7 3.
<_>
<_>
5 10 6 10 -1.
<_>
5 10 3 5 2.
<_>
8 15 3 5 2.
<_>
<_>
9 17 3 2 -1.
<_>
10 17 1 2 3.
<_>
<_>
9 10 2 2 -1.
<_>
9 11 2 1 2.
<_>
<_>
8 17 3 2 -1.
<_>
9 17 1 2 3.
<_>
<_>
5 6 1 3 -1.
<_>
5 7 1 1 3.
<_>
<_>
0 1 20 2 -1.
<_>
10 1 10 1 2.
<_>
0 2 10 1 2.
<_>
<_>
14 2 6 9 -1.
<_>
14 5 6 3 3.
<_>
<_>
5 3 3 2 -1.
<_>
5 4 3 1 2.
<_>
<_>
5 4 4 2 -1.
<_>
7 4 2 2 2.
<_>
<_>
14 2 6 9 -1.
<_>
14 5 6 3 3.
<_>
<_>
0 12 20 6 -1.
<_>
0 14 20 2 3.
<_>
<_>
2 2 16 4 -1.
<_>
2 2 8 2 2.
<_>
10 4 8 2 2.
<_>
<_>
7 12 5 3 -1.
<_>
7 13 5 1 3.
<_>
<_>
14 9 6 10 -1.
<_>
14 9 3 10 2.
<_>
<_>
16 6 3 2 -1.
<_>
16 7 3 1 2.
<_>
<_>
0 9 6 10 -1.
<_>
3 9 3 10 2.
<_>
<_>
0 16 5 2 -1.
<_>
0 17 5 1 2.
<_>
<_>
9 12 2 3 -1.
<_>
9 13 2 1 3.
<_>
<_>
9 7 2 12 -1.
<_>
9 11 2 4 3.
<_>
<_>
3 2 6 2 -1.
<_>
5 2 2 2 3.
<_>
<_>
4 1 1 2 -1.
<_>
4 2 1 1 2.
<_>
<_>
11 15 1 2 -1.
<_>
11 16 1 1 2.
<_>
<_>
3 1 16 2 -1.
<_>
11 1 8 1 2.
<_>
3 2 8 1 2.
<_>
<_>
3 6 2 2 -1.
<_>
3 6 1 1 2.
<_>
4 7 1 1 2.
<_>
<_>
5 11 10 6 -1.
<_>
5 11 5 3 2.
<_>
10 14 5 3 2.
<_>
<_>
10 11 4 6 -1.
<_>
10 14 4 3 2.
<_>
<_>
14 9 6 11 -1.
<_>
16 9 2 11 3.
<_>
<_>
0 9 6 11 -1.
<_>
2 9 2 11 3.
<_>
<_>
2 11 16 6 -1.
<_>
2 11 8 3 2.
<_>
10 14 8 3 2.
<_>
<_>
12 0 8 10 -1.
<_>
16 0 4 5 2.
<_>
12 5 4 5 2.
<_>
<_>
14 2 6 4 -1.
<_>
16 2 2 4 3.
<_>
<_>
0 0 8 10 -1.
<_>
0 0 4 5 2.
<_>
4 5 4 5 2.
<_>
<_>
0 2 6 4 -1.
<_>
2 2 2 4 3.
<_>
<_>
4 9 15 2 -1.
<_>
9 9 5 2 3.
<_>
<_>
12 3 4 8 -1.
<_>
14 3 2 4 2.
<_>
12 7 2 4 2.
<_>
<_>
9 2 2 9 -1.
<_>
10 2 1 9 2.
<_>
<_>
0 2 20 1 -1.
<_>
10 2 10 1 2.
<_>
<_>
16 1 4 5 -1.
<_>
16 1 2 5 2.
<_>
<_>
16 0 4 6 -1.
<_>
16 3 4 3 2.
<_>
<_>
4 3 6 4 -1.
<_>
6 3 2 4 3.
<_>
<_>
0 0 18 5 -1.
<_>
6 0 6 5 3.
<_>
<_>
6 2 12 14 -1.
<_>
12 2 6 7 2.
<_>
6 9 6 7 2.
<_>
<_>
11 8 3 5 -1.
<_>
12 8 1 5 3.
<_>
<_>
5 12 2 2 -1.
<_>
5 13 2 1 2.
<_>
<_>
5 10 4 3 -1.
<_>
7 10 2 3 2.
<_>
<_>
4 9 15 2 -1.
<_>
9 9 5 2 3.
<_>
<_>
10 7 6 2 -1.
<_>
12 7 2 2 3.
<_>
<_>
1 9 15 2 -1.
<_>
6 9 5 2 3.
<_>
<_>
5 0 2 10 -1.
<_>
5 0 1 5 2.
<_>
6 5 1 5 2.
<_>
<_>
0 0 20 14 -1.
<_>
0 7 20 7 2.
<_>
<_>
12 7 8 4 -1.
<_>
12 7 4 4 2.
<_>
<_>
0 7 8 4 -1.
<_>
4 7 4 4 2.
<_>
<_>
8 1 3 3 -1.
<_>
9 1 1 3 3.
<_>
<_>
9 7 3 4 -1.
<_>
10 7 1 4 3.
<_>
<_>
9 9 3 1 -1.
<_>
10 9 1 1 3.
<_>
<_>
8 9 3 2 -1.
<_>
8 10 3 1 2.
<_>
<_>
8 4 2 8 -1.
<_>
8 4 1 4 2.
<_>
9 8 1 4 2.
<_>
<_>
5 8 12 3 -1.
<_>
5 9 12 1 3.
<_>
<_>
11 14 1 3 -1.
<_>
11 15 1 1 3.
<_>
<_>
6 10 3 6 -1.
<_>
6 12 3 2 3.
<_>
<_>
4 17 8 3 -1.
<_>
4 18 8 1 3.
<_>
<_>
17 6 2 3 -1.
<_>
17 7 2 1 3.
<_>
<_>
9 12 2 2 -1.
<_>
10 12 1 1 2.
<_>
9 13 1 1 2.
<_>
<_>
9 13 2 4 -1.
<_>
9 13 1 2 2.
<_>
10 15 1 2 2.
<_>
<_>
9 11 2 3 -1.
<_>
9 12 2 1 3.
<_>
<_>
5 5 12 10 -1.
<_>
11 5 6 5 2.
<_>
5 10 6 5 2.
<_>
<_>
6 3 12 12 -1.
<_>
12 3 6 6 2.
<_>
6 9 6 6 2.
<_>
<_>
5 7 2 2 -1.
<_>
5 7 1 1 2.
<_>
6 8 1 1 2.
<_>
<_>
4 3 3 2 -1.
<_>
5 3 1 2 3.
<_>
<_>
6 2 12 14 -1.
<_>
12 2 6 7 2.
<_>
6 9 6 7 2.
<_>
<_>
5 2 12 3 -1.
<_>
9 2 4 3 3.
<_>
<_>
1 1 18 17 -1.
<_>
7 1 6 17 3.
<_>
<_>
0 9 10 1 -1.
<_>
5 9 5 1 2.
<_>
<_>
16 8 4 3 -1.
<_>
16 9 4 1 3.
<_>
<_>
7 13 6 6 -1.
<_>
7 16 6 3 2.
<_>
<_>
6 14 1 6 -1.
<_>
6 16 1 2 3.
<_>
<_>
6 17 4 2 -1.
<_>
6 18 4 1 2.
<_>
<_>
10 18 6 2 -1.
<_>
13 18 3 1 2.
<_>
10 19 3 1 2.
<_>
<_>
16 8 1 3 -1.
<_>
16 9 1 1 3.
<_>
<_>
8 13 4 3 -1.
<_>
8 14 4 1 3.
<_>
<_>
9 15 1 2 -1.
<_>
9 16 1 1 2.
<_>
<_>
13 0 3 12 -1.
<_>
14 0 1 12 3.
<_>
<_>
15 11 1 3 -1.
<_>
15 12 1 1 3.
<_>
<_>
8 15 3 3 -1.
<_>
8 16 3 1 3.
<_>
<_>
4 0 3 12 -1.
<_>
5 0 1 12 3.
<_>
<_>
9 7 3 3 -1.
<_>
10 7 1 3 3.
<_>
<_>
9 9 3 1 -1.
<_>
10 9 1 1 3.
<_>
<_>
2 2 12 14 -1.
<_>
2 2 6 7 2.
<_>
8 9 6 7 2.
<_>
<_>
4 2 12 3 -1.
<_>
8 2 4 3 3.
<_>
<_>
18 18 2 2 -1.
<_>
18 18 1 2 2.
<_>
<_>
17 2 3 8 -1.
<_>
18 2 1 8 3.
<_>
<_>
0 18 2 2 -1.
<_>
1 18 1 2 2.
<_>
<_>
6 11 2 6 -1.
<_>
6 14 2 3 2.
<_>
<_>
13 10 5 6 -1.
<_>
13 12 5 2 3.
<_>
<_>
5 8 15 3 -1.
<_>
5 9 15 1 3.
<_>
<_>
2 10 5 6 -1.
<_>
2 12 5 2 3.
<_>
<_>
0 8 15 3 -1.
<_>
0 9 15 1 3.
<_>
<_>
16 2 3 1 -1.
<_>
17 2 1 1 3.
<_>
<_>
17 4 3 2 -1.
<_>
18 4 1 2 3.
<_>
<_>
0 8 8 12 -1.
<_>
0 8 4 6 2.
<_>
4 14 4 6 2.
<_>
<_>
1 7 8 6 -1.
<_>
1 7 4 3 2.
<_>
5 10 4 3 2.
<_>
<_>
14 1 6 2 -1.
<_>
16 1 2 2 3.
<_>
<_>
15 0 4 4 -1.
<_>
17 0 2 2 2.
<_>
15 2 2 2 2.
<_>
<_>
1 1 4 11 -1.
<_>
3 1 2 11 2.
<_>
<_>
5 5 1 8 -1.
<_>
5 9 1 4 2.
<_>
<_>
7 7 6 1 -1.
<_>
9 7 2 1 3.
<_>
<_>
4 7 12 2 -1.
<_>
8 7 4 2 3.
<_>
<_>
8 4 4 4 -1.
<_>
8 6 4 2 2.
<_>
<_>
2 4 9 1 -1.
<_>
5 4 3 1 3.
<_>
<_>
9 12 2 8 -1.
<_>
9 16 2 4 2.
<_>
<_>
3 8 14 12 -1.
<_>
3 14 14 6 2.
<_>
<_>
6 13 7 3 -1.
<_>
6 14 7 1 3.
<_>
<_>
5 9 6 3 -1.
<_>
7 9 2 3 3.
<_>
<_>
12 1 6 3 -1.
<_>
12 2 6 1 3.
<_>
<_>
8 12 6 2 -1.
<_>
8 13 6 1 2.
<_>
<_>
0 2 18 2 -1.
<_>
0 2 9 1 2.
<_>
9 3 9 1 2.
<_>
<_>
6 10 3 6 -1.
<_>
6 13 3 3 2.
<_>
<_>
14 0 6 6 -1.
<_>
14 0 3 6 2.
<_>
<_>
15 0 5 8 -1.
<_>
15 4 5 4 2.
<_>
<_>
7 16 6 4 -1.
<_>
9 16 2 4 3.
<_>
<_>
2 11 14 4 -1.
<_>
2 11 7 2 2.
<_>
9 13 7 2 2.
<_>
<_>
14 10 6 10 -1.
<_>
14 10 3 10 2.
<_>
<_>
9 8 10 12 -1.
<_>
14 8 5 6 2.
<_>
9 14 5 6 2.
<_>
<_>
0 10 6 10 -1.
<_>
3 10 3 10 2.
<_>
<_>
1 8 10 12 -1.
<_>
1 8 5 6 2.
<_>
6 14 5 6 2.
<_>
<_>
9 3 6 1 -1.
<_>
11 3 2 1 3.
<_>
<_>
7 4 6 3 -1.
<_>
9 4 2 3 3.
<_>
<_>
5 3 6 1 -1.
<_>
7 3 2 1 3.
<_>
<_>
4 5 6 3 -1.
<_>
6 5 2 3 3.
<_>
<_>
9 16 3 3 -1.
<_>
9 17 3 1 3.
<_>
<_>
8 14 6 3 -1.
<_>
8 15 6 1 3.
<_>
<_>
6 0 8 12 -1.
<_>
6 0 4 6 2.
<_>
10 6 4 6 2.
<_>
<_>
4 12 2 3 -1.
<_>
4 13 2 1 3.
<_>
<_>
12 16 6 3 -1.
<_>
12 17 6 1 3.
<_>
<_>
7 12 7 2 -1.
<_>
7 13 7 1 2.
<_>
<_>
2 16 6 3 -1.
<_>
2 17 6 1 3.
<_>
<_>
0 7 16 6 -1.
<_>
0 10 16 3 2.
<_>
<_>
9 7 3 3 -1.
<_>
10 7 1 3 3.
<_>
<_>
9 7 3 5 -1.
<_>
10 7 1 5 3.
<_>
<_>
0 5 20 10 -1.
<_>
0 5 10 5 2.
<_>
10 10 10 5 2.
<_>
<_>
3 1 4 2 -1.
<_>
5 1 2 2 2.
<_>
<_>
7 6 8 10 -1.
<_>
11 6 4 5 2.
<_>
7 11 4 5 2.
<_>
<_>
17 6 3 2 -1.
<_>
17 7 3 1 2.
<_>
<_>
5 6 8 10 -1.
<_>
5 6 4 5 2.
<_>
9 11 4 5 2.
<_>
<_>
5 12 10 6 -1.
<_>
5 14 10 2 3.
<_>
<_>
9 7 3 3 -1.
<_>
10 7 1 3 3.
<_>
<_>
10 3 2 6 -1.
<_>
11 3 1 3 2.
<_>
10 6 1 3 2.
<_>
<_>
0 4 3 3 -1.
<_>
0 5 3 1 3.
<_>
<_>
3 16 8 4 -1.
<_>
3 16 4 2 2.
<_>
7 18 4 2 2.
<_>
<_>
8 13 5 2 -1.
<_>
8 14 5 1 2.
<_>
<_>
8 7 4 12 -1.
<_>
8 11 4 4 3.
<_>
<_>
5 9 2 2 -1.
<_>
6 9 1 2 2.
<_>
<_>
9 15 2 3 -1.
<_>
9 16 2 1 3.
<_>
<_>
13 9 2 3 -1.
<_>
13 9 1 3 2.
<_>
<_>
14 0 6 17 -1.
<_>
16 0 2 17 3.
<_>
<_>
5 10 2 2 -1.
<_>
6 10 1 2 2.
<_>
<_>
2 9 9 1 -1.
<_>
5 9 3 1 3.
<_>
<_>
9 11 2 3 -1.
<_>
9 12 2 1 3.
<_>
<_>
7 11 6 3 -1.
<_>
7 12 6 1 3.
<_>
<_>
0 6 3 2 -1.
<_>
0 7 3 1 2.
<_>
<_>
7 0 6 1 -1.
<_>
9 0 2 1 3.
<_>
<_>
9 16 3 3 -1.
<_>
9 17 3 1 3.
<_>
<_>
2 13 17 6 -1.
<_>
2 16 17 3 2.
<_>
<_>
1 3 3 7 -1.
<_>
2 3 1 7 3.
<_>
<_>
1 1 6 4 -1.
<_>
3 1 2 4 3.
<_>
<_>
14 1 6 5 -1.
<_>
14 1 3 5 2.
<_>
<_>
13 2 3 2 -1.
<_>
13 3 3 1 2.
<_>
<_>
0 1 6 5 -1.
<_>
3 1 3 5 2.
<_>
<_>
2 3 2 6 -1.
<_>
2 5 2 2 3.
<_>
<_>
9 10 3 2 -1.
<_>
9 11 3 1 2.
<_>
<_>
8 13 4 3 -1.
<_>
8 14 4 1 3.
<_>
<_>
6 3 3 1 -1.
<_>
7 3 1 1 3.
<_>
<_>
8 2 3 12 -1.
<_>
8 6 3 4 3.
<_>
<_>
11 12 1 2 -1.
<_>
11 13 1 1 2.
<_>
<_>
11 12 2 2 -1.
<_>
12 12 1 1 2.
<_>
11 13 1 1 2.
<_>
<_>
5 5 2 2 -1.
<_>
5 6 2 1 2.
<_>
<_>
5 4 1 3 -1.
<_>
5 5 1 1 3.
<_>
<_>
3 11 16 4 -1.
<_>
11 11 8 2 2.
<_>
3 13 8 2 2.
<_>
<_>
0 10 20 3 -1.
<_>
0 11 20 1 3.
<_>
<_>
1 11 16 4 -1.
<_>
1 11 8 2 2.
<_>
9 13 8 2 2.
<_>
<_>
4 2 4 2 -1.
<_>
4 3 4 1 2.
<_>
<_>
12 6 2 2 -1.
<_>
13 6 1 1 2.
<_>
12 7 1 1 2.
<_>
<_>
12 11 6 6 -1.
<_>
12 13 6 2 3.
<_>
<_>
6 6 2 2 -1.
<_>
6 6 1 1 2.
<_>
7 7 1 1 2.
<_>
<_>
6 4 4 16 -1.
<_>
8 4 2 16 2.
<_>
<_>
11 18 3 2 -1.
<_>
11 19 3 1 2.
<_>
<_>
9 17 6 2 -1.
<_>
12 17 3 1 2.
<_>
9 18 3 1 2.
<_>
<_>
2 13 5 2 -1.
<_>
2 14 5 1 2.
<_>
<_>
3 15 2 2 -1.
<_>
3 16 2 1 2.
<_>
<_>
9 7 3 3 -1.
<_>
10 7 1 3 3.
<_>
<_>
9 6 2 6 -1.
<_>
9 6 1 6 2.
<_>
<_>
1 14 7 6 -1.
<_>
1 16 7 2 3.
<_>
<_>
8 1 2 11 -1.
<_>
9 1 1 11 2.
<_>
<_>
9 7 2 4 -1.
<_>
9 7 1 4 2.
<_>
<_>
11 10 2 1 -1.
<_>
11 10 1 1 2.
<_>
<_>
0 3 3 9 -1.
<_>
1 3 1 9 3.
<_>
<_>
0 3 3 6 -1.
<_>
0 5 3 2 3.
<_>
<_>
11 15 2 2 -1.
<_>
12 15 1 1 2.
<_>
11 16 1 1 2.
<_>
<_>
11 14 2 2 -1.
<_>
12 14 1 1 2.
<_>
11 15 1 1 2.
<_>
<_>
7 15 2 2 -1.
<_>
7 15 1 1 2.
<_>
8 16 1 1 2.
<_>
<_>
7 14 2 2 -1.
<_>
7 14 1 1 2.
<_>
8 15 1 1 2.
<_>
<_>
8 13 4 6 -1.
<_>
10 13 2 3 2.
<_>
8 16 2 3 2.
<_>
<_>
2 14 16 4 -1.
<_>
10 14 8 2 2.
<_>
2 16 8 2 2.
<_>
<_>
9 8 2 2 -1.
<_>
9 9 2 1 2.
<_>
<_>
7 7 5 3 -1.
<_>
7 8 5 1 3.
<_>
<_>
7 5 6 2 -1.
<_>
9 5 2 2 3.
<_>
<_>
9 1 6 18 -1.
<_>
11 1 2 18 3.
<_>
<_>
8 6 3 4 -1.
<_>
9 6 1 4 3.
<_>
<_>
8 5 2 4 -1.
<_>
8 5 1 2 2.
<_>
9 7 1 2 2.
<_>
<_>
9 13 2 6 -1.
<_>
10 13 1 3 2.
<_>
9 16 1 3 2.
<_>
<_>
11 0 3 18 -1.
<_>
12 0 1 18 3.
<_>
<_>
6 0 3 18 -1.
<_>
7 0 1 18 3.
<_>
<_>
5 15 4 2 -1.
<_>
7 15 2 2 2.
<_>
<_>
1 9 18 1 -1.
<_>
7 9 6 1 3.
<_>
<_>
0 0 20 3 -1.
<_>
0 1 20 1 3.
<_>
<_>
9 6 2 4 -1.
<_>
10 6 1 4 2.
<_>
<_>
6 10 6 2 -1.
<_>
8 10 2 2 3.
<_>
<_>
0 7 20 1 -1.
<_>
0 7 10 1 2.
<_>
<_>
11 3 5 4 -1.
<_>
11 5 5 2 2.
<_>
<_>
5 7 10 1 -1.
<_>
10 7 5 1 2.
<_>
<_>
8 10 3 3 -1.
<_>
8 11 3 1 3.
<_>
<_>
2 0 16 8 -1.
<_>
10 0 8 4 2.
<_>
2 4 8 4 2.
<_>
<_>
11 0 9 10 -1.
<_>
11 5 9 5 2.
<_>
<_>
0 2 8 18 -1.
<_>
4 2 4 18 2.
<_>
<_>
0 0 2 6 -1.
<_>
0 2 2 2 3.
<_>
<_>
6 0 9 2 -1.
<_>
6 1 9 1 2.
<_>
<_>
4 1 12 2 -1.
<_>
4 2 12 1 2.
<_>
<_>
2 1 16 14 -1.
<_>
2 8 16 7 2.
<_>
<_>
5 1 8 12 -1.
<_>
5 7 8 6 2.
<_>
<_>
9 11 2 2 -1.
<_>
9 12 2 1 2.
<_>
<_>
9 10 5 6 -1.
<_>
9 12 5 2 3.
<_>
<_>
3 0 13 8 -1.
<_>
3 4 13 4 2.
<_>
<_>
6 7 5 8 -1.
<_>
6 11 5 4 2.
<_>
<_>
9 5 2 3 -1.
<_>
9 6 2 1 3.
<_>
<_>
6 8 8 3 -1.
<_>
6 9 8 1 3.
<_>
<_>
2 2 7 6 -1.
<_>
2 5 7 3 2.
<_>
<_>
2 1 14 4 -1.
<_>
2 1 7 2 2.
<_>
9 3 7 2 2.
<_>
<_>
11 14 1 3 -1.
<_>
11 15 1 1 3.
<_>
<_>
6 15 8 2 -1.
<_>
6 16 8 1 2.
<_>
<_>
8 14 1 3 -1.
<_>
8 15 1 1 3.
<_>
<_>
8 11 2 8 -1.
<_>
8 15 2 4 2.
<_>
<_>
6 15 8 2 -1.
<_>
6 16 8 1 2.
<_>
<_>
7 16 8 3 -1.
<_>
7 17 8 1 3.
<_>
<_>
0 16 2 2 -1.
<_>
0 17 2 1 2.
<_>
<_>
1 16 8 4 -1.
<_>
1 16 4 2 2.
<_>
5 18 4 2 2.
<_>
<_>
2 9 16 3 -1.
<_>
2 10 16 1 3.
<_>
<_>
13 11 2 4 -1.
<_>
13 11 1 4 2.
<_>
<_>
0 13 16 6 -1.
<_>
0 15 16 2 3.
<_>
<_>
5 11 2 4 -1.
<_>
6 11 1 4 2.
<_>
<_>
18 2 2 18 -1.
<_>
19 2 1 9 2.
<_>
18 11 1 9 2.
<_>
<_>
19 7 1 9 -1.
<_>
19 10 1 3 3.
<_>
<_>
0 2 2 18 -1.
<_>
0 2 1 9 2.
<_>
1 11 1 9 2.
<_>
<_>
0 7 1 9 -1.
<_>
0 10 1 3 3.
<_>
<_>
14 12 2 2 -1.
<_>
14 13 2 1 2.
<_>
<_>
11 14 2 3 -1.
<_>
11 15 2 1 3.
<_>
<_>
7 8 6 2 -1.
<_>
7 9 6 1 2.
<_>
<_>
7 12 4 6 -1.
<_>
7 12 2 3 2.
<_>
9 15 2 3 2.
<_>
<_>
8 13 5 3 -1.
<_>
8 14 5 1 3.
<_>
<_>
12 14 2 2 -1.
<_>
13 14 1 1 2.
<_>
12 15 1 1 2.
<_>
<_>
7 13 6 3 -1.
<_>
7 14 6 1 3.
<_>
<_>
7 13 5 2 -1.
<_>
7 14 5 1 2.
<_>
<_>
2 10 16 4 -1.
<_>
10 10 8 2 2.
<_>
2 12 8 2 2.
<_>
<_>
7 0 6 6 -1.
<_>
9 0 2 6 3.
<_>
<_>
7 1 6 3 -1.
<_>
7 2 6 1 3.
<_>
<_>
0 12 6 2 -1.
<_>
0 13 6 1 2.
<_>
<_>
6 3 11 2 -1.
<_>
6 4 11 1 2.
<_>
<_>
12 0 8 6 -1.
<_>
16 0 4 3 2.
<_>
12 3 4 3 2.
<_>
<_>
8 12 1 2 -1.
<_>
8 13 1 1 2.
<_>
<_>
8 8 1 12 -1.
<_>
8 12 1 4 3.
<_>
<_>
11 11 2 2 -1.
<_>
12 11 1 1 2.
<_>
11 12 1 1 2.
<_>
<_>
12 7 3 13 -1.
<_>
13 7 1 13 3.
<_>
<_>
7 11 2 2 -1.
<_>
7 11 1 1 2.
<_>
8 12 1 1 2.
<_>
<_>
3 13 1 3 -1.
<_>
3 14 1 1 3.
<_>
<_>
10 18 3 2 -1.
<_>
11 18 1 2 3.
<_>
<_>
11 11 2 1 -1.
<_>
11 11 1 1 2.
<_>
<_>
1 10 5 9 -1.
<_>
1 13 5 3 3.
<_>
<_>
4 8 6 4 -1.
<_>
6 8 2 4 3.
<_>
<_>
13 12 1 4 -1.
<_>
13 14 1 2 2.
<_>
<_>
11 3 4 14 -1.
<_>
13 3 2 7 2.
<_>
11 10 2 7 2.
<_>
<_>
6 12 1 4 -1.
<_>
6 14 1 2 2.
<_>
<_>
5 3 4 14 -1.
<_>
5 3 2 7 2.
<_>
7 10 2 7 2.
<_>
<_>
10 18 3 2 -1.
<_>
11 18 1 2 3.
<_>
<_>
9 12 3 3 -1.
<_>
9 13 3 1 3.
<_>
<_>
2 2 12 6 -1.
<_>
2 2 6 3 2.
<_>
8 5 6 3 2.
<_>
<_>
6 6 6 2 -1.
<_>
9 6 3 2 2.
<_>
<_>
1 0 18 12 -1.
<_>
7 0 6 12 3.
<_>
<_>
5 7 6 4 -1.
<_>
5 7 3 2 2.
<_>
8 9 3 2 2.
<_>
<_>
5 7 10 4 -1.
<_>
5 9 10 2 2.
<_>
<_>
7 7 6 4 -1.
<_>
9 7 2 4 3.
<_>
<_>
9 5 2 2 -1.
<_>
9 6 2 1 2.
<_>
<_>
9 9 2 2 -1.
<_>
9 10 2 1 2.
<_>
<_>
6 17 8 3 -1.
<_>
6 18 8 1 3.
<_>
<_>
9 17 6 2 -1.
<_>
12 17 3 1 2.
<_>
9 18 3 1 2.
<_>
<_>
4 12 2 2 -1.
<_>
4 13 2 1 2.
<_>
<_>
3 12 9 2 -1.
<_>
3 13 9 1 2.
<_>
<_>
8 3 6 1 -1.
<_>
10 3 2 1 3.
<_>
<_>
9 3 4 6 -1.
<_>
11 3 2 3 2.
<_>
9 6 2 3 2.
<_>
<_>
0 3 6 5 -1.
<_>
3 3 3 5 2.
<_>
<_>
2 0 2 18 -1.
<_>
2 6 2 6 3.
<_>
<_>
14 2 4 9 -1.
<_>
14 5 4 3 3.
<_>
<_>
10 18 3 2 -1.
<_>
11 18 1 2 3.
<_>
<_>
2 2 4 9 -1.
<_>
2 5 4 3 3.
<_>
<_>
7 18 3 2 -1.
<_>
8 18 1 2 3.
<_>
<_>
10 14 3 3 -1.
<_>
10 15 3 1 3.
<_>
<_>
10 12 2 6 -1.
<_>
10 15 2 3 2.
<_>
<_>
7 5 3 6 -1.
<_>
7 7 3 2 3.
<_>
<_>
3 3 6 2 -1.
<_>
3 4 6 1 2.
<_>
<_>
8 4 7 3 -1.
<_>
8 5 7 1 3.
<_>
<_>
13 6 2 3 -1.
<_>
13 7 2 1 3.
<_>
<_>
8 8 2 12 -1.
<_>
8 12 2 4 3.
<_>
<_>
5 4 8 14 -1.
<_>
5 4 4 7 2.
<_>
9 11 4 7 2.
<_>
<_>
0 1 20 8 -1.
<_>
10 1 10 4 2.
<_>
0 5 10 4 2.
<_>
<_>
4 0 12 2 -1.
<_>
4 1 12 1 2.
<_>
<_>
0 1 20 8 -1.
<_>
0 1 10 4 2.
<_>
10 5 10 4 2.
<_>
<_>
4 0 12 2 -1.
<_>
4 1 12 1 2.
<_>
<_>
9 5 6 3 -1.
<_>
9 5 3 3 2.
<_>
<_>
8 13 10 6 -1.
<_>
8 15 10 2 3.
<_>
<_>
5 5 6 3 -1.
<_>
8 5 3 3 2.
<_>
<_>
6 3 6 1 -1.
<_>
8 3 2 1 3.
<_>
<_>
11 18 9 2 -1.
<_>
14 18 3 2 3.
<_>
<_>
13 11 6 7 -1.
<_>
13 11 3 7 2.
<_>
<_>
4 6 12 10 -1.
<_>
4 6 6 5 2.
<_>
10 11 6 5 2.
<_>
<_>
8 17 3 3 -1.
<_>
9 17 1 3 3.
<_>
<_>
11 18 9 2 -1.
<_>
14 18 3 2 3.
<_>
<_>
13 11 6 8 -1.
<_>
13 11 3 8 2.
<_>
<_>
4 16 2 2 -1.
<_>
4 17 2 1 2.
<_>
<_>
7 15 4 4 -1.
<_>
7 17 4 2 2.
<_>
<_>
12 4 3 3 -1.
<_>
12 5 3 1 3.
<_>
<_>
13 6 2 3 -1.
<_>
13 7 2 1 3.
<_>
<_>
5 11 6 1 -1.
<_>
7 11 2 1 3.
<_>
<_>
7 10 3 1 -1.
<_>
8 10 1 1 3.
<_>
<_>
0 12 20 4 -1.
<_>
0 14 20 2 2.
<_>
<_>
10 2 3 2 -1.
<_>
10 3 3 1 2.
<_>
<_>
5 4 3 3 -1.
<_>
5 5 3 1 3.
<_>
<_>
5 5 4 3 -1.
<_>
5 6 4 1 3.
<_>
<_>
8 8 4 3 -1.
<_>
8 9 4 1 3.
<_>
<_>
10 4 2 12 -1.
<_>
10 8 2 4 3.
<_>
<_>
0 3 4 3 -1.
<_>
0 4 4 1 3.
<_>
<_>
1 3 2 3 -1.
<_>
1 4 2 1 3.
<_>
<_>
16 1 4 11 -1.
<_>
16 1 2 11 2.
<_>
<_>
18 2 2 16 -1.
<_>
19 2 1 8 2.
<_>
18 10 1 8 2.
<_>
<_>
1 8 6 12 -1.
<_>
3 8 2 12 3.
<_>
<_>
7 2 6 2 -1.
<_>
7 2 3 1 2.
<_>
10 3 3 1 2.
<_>
<_>
12 4 8 2 -1.
<_>
16 4 4 1 2.
<_>
12 5 4 1 2.
<_>
<_>
10 6 6 2 -1.
<_>
12 6 2 2 3.
<_>
<_>
0 4 8 2 -1.
<_>
0 4 4 1 2.
<_>
4 5 4 1 2.
<_>
<_>
1 3 3 5 -1.
<_>
2 3 1 5 3.
<_>
<_>
16 3 4 6 -1.
<_>
16 5 4 2 3.
<_>
<_>
8 6 4 3 -1.
<_>
8 7 4 1 3.
<_>
<_>
8 14 1 3 -1.
<_>
8 15 1 1 3.
<_>
<_>
4 11 1 2 -1.
<_>
4 12 1 1 2.
<_>
<_>
8 14 6 3 -1.
<_>
8 15 6 1 3.
<_>
<_>
7 15 7 3 -1.
<_>
7 16 7 1 3.
<_>
<_>
9 12 2 8 -1.
<_>
9 16 2 4 2.
<_>
<_>
4 6 6 2 -1.
<_>
6 6 2 2 3.
<_>
<_>
12 7 4 2 -1.
<_>
12 8 4 1 2.
<_>
<_>
5 3 13 10 -1.
<_>
5 8 13 5 2.
<_>
<_>
4 7 4 2 -1.
<_>
4 8 4 1 2.
<_>
<_>
0 8 16 2 -1.
<_>
0 8 8 1 2.
<_>
8 9 8 1 2.
<_>
<_>
11 8 2 5 -1.
<_>
11 8 1 5 2.
<_>
<_>
10 0 6 13 -1.
<_>
10 0 3 13 2.
<_>
<_>
1 6 4 2 -1.
<_>
1 7 4 1 2.
<_>
<_>
4 3 2 1 -1.
<_>
5 3 1 1 2.
<_>
<_>
11 8 2 5 -1.
<_>
11 8 1 5 2.
<_>
<_>
12 10 4 8 -1.
<_>
12 10 2 8 2.
<_>
<_>
7 8 2 5 -1.
<_>
8 8 1 5 2.
<_>
<_>
4 10 4 8 -1.
<_>
6 10 2 8 2.
<_>
<_>
6 7 9 12 -1.
<_>
9 7 3 12 3.
<_>
<_>
11 13 2 3 -1.
<_>
11 13 1 3 2.
<_>
<_>
7 10 6 10 -1.
<_>
10 10 3 10 2.
<_>
<_>
8 11 4 8 -1.
<_>
8 11 2 4 2.
<_>
10 15 2 4 2.
<_>
<_>
16 1 4 11 -1.
<_>
16 1 2 11 2.
<_>
<_>
18 2 2 4 -1.
<_>
18 2 1 4 2.
<_>
<_>
5 6 6 2 -1.
<_>
5 6 3 1 2.
<_>
8 7 3 1 2.
<_>
<_>
5 4 1 3 -1.
<_>
5 5 1 1 3.
<_>
<_>
11 1 4 14 -1.
<_>
11 1 2 14 2.
<_>
<_>
4 2 12 3 -1.
<_>
8 2 4 3 3.
<_>
<_>
5 1 4 14 -1.
<_>
7 1 2 14 2.
<_>
<_>
7 3 6 2 -1.
<_>
9 3 2 2 3.
<_>
<_>
2 0 18 4 -1.
<_>
8 0 6 4 3.
<_>
<_>
9 5 2 10 -1.
<_>
9 10 2 5 2.
<_>
<_>
8 6 3 4 -1.
<_>
9 6 1 4 3.
<_>
<_>
5 5 9 11 -1.
<_>
8 5 3 11 3.
<_>
<_>
10 6 3 5 -1.
<_>
11 6 1 5 3.
<_>
<_>
8 9 6 5 -1.
<_>
8 9 3 5 2.
<_>
<_>
7 6 3 5 -1.
<_>
8 6 1 5 3.
<_>
<_>
6 10 6 3 -1.
<_>
9 10 3 3 2.
<_>
<_>
10 0 3 7 -1.
<_>
11 0 1 7 3.
<_>
<_>
0 3 20 12 -1.
<_>
0 9 20 6 2.
<_>
<_>
9 7 2 2 -1.
<_>
10 7 1 2 2.
<_>
<_>
5 9 4 1 -1.
<_>
7 9 2 1 2.
<_>
<_>
13 13 3 2 -1.
<_>
13 14 3 1 2.
<_>
<_>
16 9 4 6 -1.
<_>
16 9 2 6 2.
<_>
<_>
7 15 6 3 -1.
<_>
7 16 6 1 3.
<_>
<_>
6 16 7 3 -1.
<_>
6 17 7 1 3.
<_>
<_>
11 14 9 6 -1.
<_>
11 16 9 2 3.
<_>
<_>
19 14 1 3 -1.
<_>
19 15 1 1 3.
<_>
<_>
0 9 6 6 -1.
<_>
3 9 3 6 2.
<_>
<_>
0 19 9 1 -1.
<_>
3 19 3 1 3.
<_>
<_>
11 14 9 6 -1.
<_>
11 16 9 2 3.
<_>
<_>
12 12 6 6 -1.
<_>
12 14 6 2 3.
<_>
<_>
1 14 8 6 -1.
<_>
1 16 8 2 3.
<_>
<_>
8 1 3 2 -1.
<_>
9 1 1 2 3.
<_>
<_>
18 2 2 4 -1.
<_>
18 2 1 4 2.
<_>
<_>
14 0 6 3 -1.
<_>
16 0 2 3 3.
<_>
<_>
0 2 2 4 -1.
<_>
1 2 1 4 2.
<_>
<_>
0 0 6 3 -1.
<_>
2 0 2 3 3.
<_>
<_>
9 0 3 2 -1.
<_>
10 0 1 2 3.
<_>
<_>
12 1 2 2 -1.
<_>
12 1 1 2 2.
<_>
<_>
8 0 3 2 -1.
<_>
9 0 1 2 3.
<_>
<_>
6 1 2 2 -1.
<_>
7 1 1 2 2.
<_>
<_>
10 8 2 3 -1.
<_>
10 9 2 1 3.
<_>
<_>
13 15 6 2 -1.
<_>
13 16 6 1 2.
<_>
<_>
8 12 2 2 -1.
<_>
8 12 1 1 2.
<_>
9 13 1 1 2.
<_>
<_>
8 15 3 5 -1.
<_>
9 15 1 5 3.
<_>
<_>
8 6 4 12 -1.
<_>
8 12 4 6 2.
<_>
<_>
7 6 7 8 -1.
<_>
7 10 7 4 2.
<_>
<_>
0 11 8 2 -1.
<_>
0 12 8 1 2.
<_>
<_>
8 11 2 2 -1.
<_>
8 11 1 1 2.
<_>
9 12 1 1 2.
<_>
<_>
7 7 12 1 -1.
<_>
11 7 4 1 3.
<_>
<_>
10 8 3 2 -1.
<_>
11 8 1 2 3.
<_>
<_>
1 7 12 1 -1.
<_>
5 7 4 1 3.
<_>
<_>
6 5 8 2 -1.
<_>
6 5 4 1 2.
<_>
10 6 4 1 2.
<_>
<_>
9 10 3 10 -1.
<_>
10 10 1 10 3.
<_>
<_>
16 0 2 4 -1.
<_>
16 0 1 4 2.
<_>
<_>
8 10 3 10 -1.
<_>
9 10 1 10 3.
<_>
<_>
9 10 2 3 -1.
<_>
9 11 2 1 3.
<_>
<_>
8 9 4 2 -1.
<_>
10 9 2 1 2.
<_>
8 10 2 1 2.
<_>
<_>
12 14 7 6 -1.
<_>
12 16 7 2 3.
<_>
<_>
6 1 3 1 -1.
<_>
7 1 1 1 3.
<_>
<_>
2 0 2 4 -1.
<_>
3 0 1 4 2.
<_>
<_>
11 11 2 2 -1.
<_>
12 11 1 1 2.
<_>
11 12 1 1 2.
<_>
<_>
12 12 6 6 -1.
<_>
12 14 6 2 3.
<_>
<_>
1 0 6 10 -1.
<_>
1 0 3 5 2.
<_>
4 5 3 5 2.
<_>
<_>
3 0 2 9 -1.
<_>
3 3 2 3 3.
<_>
<_>
14 13 3 2 -1.
<_>
14 14 3 1 2.
<_>
<_>
15 2 3 2 -1.
<_>
15 3 3 1 2.
<_>
<_>
2 13 5 2 -1.
<_>
2 14 5 1 2.
<_>
<_>
3 4 12 10 -1.
<_>
3 4 6 5 2.
<_>
9 9 6 5 2.
<_>
<_>
5 1 14 6 -1.
<_>
5 3 14 2 3.
<_>
<_>
15 3 3 2 -1.
<_>
15 4 3 1 2.
<_>
<_>
7 11 2 2 -1.
<_>
7 11 1 1 2.
<_>
8 12 1 1 2.
<_>
<_>
2 14 6 6 -1.
<_>
2 16 6 2 3.
<_>
<_>
6 13 8 3 -1.
<_>
6 14 8 1 3.
<_>
<_>
1 19 18 1 -1.
<_>
7 19 6 1 3.
<_>
<_>
8 12 1 6 -1.
<_>
8 15 1 3 2.
<_>
<_>
0 0 14 15 -1.
<_>
0 5 14 5 3.
<_>
<_>
3 0 16 8 -1.
<_>
3 4 16 4 2.
<_>
<_>
6 1 8 12 -1.
<_>
6 7 8 6 2.
<_>
<_>
5 3 3 3 -1.
<_>
6 3 1 3 3.
<_>
<_>
5 1 3 4 -1.
<_>
6 1 1 4 3.
<_>
<_>
15 14 4 6 -1.
<_>
17 14 2 3 2.
<_>
15 17 2 3 2.
<_>
<_>
12 11 6 8 -1.
<_>
15 11 3 4 2.
<_>
12 15 3 4 2.
<_>
<_>
8 7 2 4 -1.
<_>
9 7 1 4 2.
<_>
<_>
6 11 3 1 -1.
<_>
7 11 1 1 3.
<_>
<_>
12 3 2 14 -1.
<_>
12 3 1 14 2.
<_>
<_>
12 11 6 2 -1.
<_>
15 11 3 1 2.
<_>
12 12 3 1 2.
<_>
<_>
0 2 5 2 -1.
<_>
0 3 5 1 2.
<_>
<_>
0 0 15 1 -1.
<_>
5 0 5 1 3.
<_>
<_>
12 11 6 2 -1.
<_>
15 11 3 1 2.
<_>
12 12 3 1 2.
<_>
<_>
10 5 2 2 -1.
<_>
10 5 1 2 2.
<_>
<_>
9 7 2 2 -1.
<_>
10 7 1 2 2.
<_>
<_>
9 0 2 10 -1.
<_>
9 0 1 5 2.
<_>
10 5 1 5 2.
<_>
<_>
18 14 2 2 -1.
<_>
18 15 2 1 2.
<_>
<_>
13 11 4 9 -1.
<_>
13 14 4 3 3.
<_>
<_>
8 13 2 2 -1.
<_>
8 13 1 1 2.
<_>
9 14 1 1 2.
<_>
<_>
7 8 4 3 -1.
<_>
7 9 4 1 3.
<_>
<_>
8 9 4 2 -1.
<_>
8 10 4 1 2.
<_>
<_>
13 12 4 2 -1.
<_>
13 13 4 1 2.
<_>
<_>
6 14 2 2 -1.
<_>
6 14 1 1 2.
<_>
7 15 1 1 2.
<_>
<_>
0 14 2 2 -1.
<_>
0 15 2 1 2.
<_>
<_>
7 13 6 3 -1.
<_>
7 14 6 1 3.
<_>
<_>
7 9 10 6 -1.
<_>
7 11 10 2 3.
<_>
<_>
2 9 12 4 -1.
<_>
6 9 4 4 3.
<_>
<_>
7 9 6 11 -1.
<_>
10 9 3 11 2.
<_>
<_>
9 7 2 3 -1.
<_>
9 8 2 1 3.
<_>
<_>
9 14 4 3 -1.
<_>
9 15 4 1 3.
<_>
<_>
2 3 3 17 -1.
<_>
3 3 1 17 3.
<_>
<_>
0 11 6 3 -1.
<_>
0 12 6 1 3.
<_>
<_>
4 3 11 9 -1.
<_>
4 6 11 3 3.
<_>
<_>
0 2 6 11 -1.
<_>
3 2 3 11 2.
<_>
<_>
13 0 4 5 -1.
<_>
13 0 2 5 2.
<_>
<_>
9 7 6 4 -1.
<_>
12 7 3 2 2.
<_>
9 9 3 2 2.
<_>
<_>
5 7 8 2 -1.
<_>
9 7 4 2 2.
<_>
<_>
1 8 15 1 -1.
<_>
6 8 5 1 3.
<_>
<_>
4 12 12 2 -1.
<_>
8 12 4 2 3.
<_>
<_>
13 0 4 10 -1.
<_>
15 0 2 5 2.
<_>
13 5 2 5 2.
<_>
<_>
9 9 2 2 -1.
<_>
9 10 2 1 2.
<_>
<_>
3 9 6 2 -1.
<_>
6 9 3 2 2.
<_>
<_>
8 17 4 3 -1.
<_>
8 18 4 1 3.
<_>
<_>
8 3 9 2 -1.
<_>
11 3 3 2 3.
<_>
<_>
3 3 9 2 -1.
<_>
6 3 3 2 3.
<_>
<_>
5 0 9 14 -1.
<_>
8 0 3 14 3.
<_>
<_>
7 3 7 10 -1.
<_>
7 8 7 5 2.
<_>
<_>
4 8 13 3 -1.
<_>
4 9 13 1 3.
<_>
<_>
3 12 14 4 -1.
<_>
3 12 7 2 2.
<_>
10 14 7 2 2.
<_>
<_>
8 12 4 2 -1.
<_>
8 13 4 1 2.
<_>
<_>
6 10 9 8 -1.
<_>
6 14 9 4 2.
<_>
<_>
9 12 2 8 -1.
<_>
9 16 2 4 2.
<_>
<_>
8 12 3 3 -1.
<_>
8 13 3 1 3.
<_>
<_>
5 5 4 10 -1.
<_>
7 5 2 10 2.
<_>
<_>
14 15 3 3 -1.
<_>
14 16 3 1 3.
<_>
<_>
4 6 13 3 -1.
<_>
4 7 13 1 3.
<_>
<_>
3 15 3 3 -1.
<_>
3 16 3 1 3.
<_>
<_>
3 9 4 2 -1.
<_>
3 9 2 1 2.
<_>
5 10 2 1 2.
<_>
<_>
0 11 20 4 -1.
<_>
10 11 10 2 2.
<_>
0 13 10 2 2.
<_>
<_>
8 15 4 3 -1.
<_>
8 16 4 1 3.
<_>
<_>
0 11 20 4 -1.
<_>
0 11 10 2 2.
<_>
10 13 10 2 2.
<_>
<_>
8 15 4 3 -1.
<_>
8 16 4 1 3.
<_>
<_>
10 13 1 6 -1.
<_>
10 16 1 3 2.
<_>
<_>
2 1 18 2 -1.
<_>
11 1 9 1 2.
<_>
2 2 9 1 2.
<_>
<_>
8 14 3 3 -1.
<_>
8 15 3 1 3.
<_>
<_>
4 1 6 1 -1.
<_>
6 1 2 1 3.
<_>
<_>
11 13 1 3 -1.
<_>
11 14 1 1 3.
<_>
<_>
13 5 2 12 -1.
<_>
13 11 2 6 2.
<_>
<_>
1 14 18 6 -1.
<_>
1 16 18 2 3.
<_>
<_>
8 13 1 3 -1.
<_>
8 14 1 1 3.
<_>
<_>
7 13 6 3 -1.
<_>
7 14 6 1 3.
<_>
<_>
9 10 3 2 -1.
<_>
9 11 3 1 2.
<_>
<_>
5 1 3 3 -1.
<_>
6 1 1 3 3.
<_>
<_>
5 5 6 5 -1.
<_>
8 5 3 5 2.
<_>
<_>
7 5 6 14 -1.
<_>
7 12 6 7 2.
<_>
<_>
7 16 6 2 -1.
<_>
9 16 2 2 3.
<_>
<_>
0 2 2 12 -1.
<_>
1 2 1 12 2.
<_>
<_>
1 0 5 3 -1.
<_>
1 1 5 1 3.
<_>
<_>
12 4 3 3 -1.
<_>
12 5 3 1 3.
<_>
<_>
12 6 3 3 -1.
<_>
12 7 3 1 3.
<_>
<_>
5 4 3 3 -1.
<_>
5 5 3 1 3.
<_>
<_>
5 6 3 3 -1.
<_>
5 7 3 1 3.
<_>
<_>
8 12 4 8 -1.
<_>
10 12 2 4 2.
<_>
8 16 2 4 2.
<_>
<_>
2 17 18 2 -1.
<_>
11 17 9 1 2.
<_>
2 18 9 1 2.
<_>
<_>
9 3 2 2 -1.
<_>
9 4 2 1 2.
<_>
<_>
8 5 4 6 -1.
<_>
8 7 4 2 3.
<_>
<_>
9 0 8 6 -1.
<_>
9 2 8 2 3.
<_>
<_>
1 0 18 4 -1.
<_>
7 0 6 4 3.
<_>
<_>
0 0 4 8 -1.
<_>
2 0 2 8 2.
<_>
<_>
0 4 6 9 -1.
<_>
2 4 2 9 3.
<_>
<_>
1 4 18 2 -1.
<_>
7 4 6 2 3.
<_>
<_>
8 16 12 4 -1.
<_>
14 16 6 2 2.
<_>
8 18 6 2 2.
<_>
<_>
0 0 18 2 -1.
<_>
0 0 9 1 2.
<_>
9 1 9 1 2.
<_>
<_>
3 0 3 18 -1.
<_>
4 0 1 18 3.
<_>
<_>
14 9 4 7 -1.
<_>
14 9 2 7 2.
<_>
<_>
15 14 2 2 -1.
<_>
15 15 2 1 2.
<_>
<_>
2 9 4 7 -1.
<_>
4 9 2 7 2.
<_>
<_>
3 14 2 2 -1.
<_>
3 15 2 1 2.
<_>
<_>
11 0 6 6 -1.
<_>
11 2 6 2 3.
<_>
<_>
14 0 2 6 -1.
<_>
15 0 1 3 2.
<_>
14 3 1 3 2.
<_>
<_>
7 11 2 2 -1.
<_>
7 11 1 1 2.
<_>
8 12 1 1 2.
<_>
<_>
7 10 2 2 -1.
<_>
8 10 1 2 2.
<_>
<_>
9 14 2 6 -1.
<_>
9 17 2 3 2.
<_>
<_>
12 18 4 2 -1.
<_>
12 19 4 1 2.
<_>
<_>
8 17 4 3 -1.
<_>
8 18 4 1 3.
<_>
<_>
2 18 8 2 -1.
<_>
2 19 8 1 2.
<_>
<_>
2 9 16 3 -1.
<_>
2 10 16 1 3.
<_>
<_>
9 9 2 2 -1.
<_>
9 10 2 1 2.
<_>
<_>
5 14 2 4 -1.
<_>
5 14 1 2 2.
<_>
6 16 1 2 2.
<_>
<_>
8 9 4 2 -1.
<_>
8 9 2 1 2.
<_>
10 10 2 1 2.
<_>
<_>
9 5 2 5 -1.
<_>
9 5 1 5 2.
<_>
<_>
9 9 3 2 -1.
<_>
10 9 1 2 3.
<_>
<_>
8 9 3 2 -1.
<_>
9 9 1 2 3.
<_>
<_>
8 8 3 6 -1.
<_>
9 8 1 6 3.
<_>
<_>
8 12 4 8 -1.
<_>
10 12 2 4 2.
<_>
8 16 2 4 2.
<_>
<_>
2 17 16 2 -1.
<_>
10 17 8 1 2.
<_>
2 18 8 1 2.
<_>
<_>
8 12 3 8 -1.
<_>
9 12 1 8 3.
<_>
<_>
3 10 1 3 -1.
<_>
3 11 1 1 3.
<_>
<_>
9 14 10 6 -1.
<_>
14 14 5 3 2.
<_>
9 17 5 3 2.
<_>
<_>
14 13 3 6 -1.
<_>
14 15 3 2 3.
<_>
<_>
1 19 18 1 -1.
<_>
7 19 6 1 3.
<_>
<_>
2 10 15 2 -1.
<_>
7 10 5 2 3.
<_>
<_>
4 17 16 3 -1.
<_>
4 18 16 1 3.
<_>
<_>
8 6 4 9 -1.
<_>
8 9 4 3 3.
<_>
<_>
9 16 2 4 -1.
<_>
9 16 1 2 2.
<_>
10 18 1 2 2.
<_>
<_>
5 5 10 8 -1.
<_>
5 9 10 4 2.
<_>
<_>
13 1 4 2 -1.
<_>
13 1 2 2 2.
<_>
<_>
14 0 3 6 -1.
<_>
14 2 3 2 3.
<_>
<_>
6 7 2 2 -1.
<_>
6 7 1 1 2.
<_>
7 8 1 1 2.
<_>
<_>
7 1 6 1 -1.
<_>
9 1 2 1 3.
<_>
<_>
9 11 3 3 -1.
<_>
9 12 3 1 3.
<_>
<_>
12 9 3 3 -1.
<_>
13 9 1 3 3.
<_>
<_>
8 11 3 3 -1.
<_>
8 12 3 1 3.
<_>
<_>
5 9 3 3 -1.
<_>
6 9 1 3 3.
<_>
<_>
10 11 1 3 -1.
<_>
10 12 1 1 3.
<_>
<_>
7 9 6 4 -1.
<_>
10 9 3 2 2.
<_>
7 11 3 2 2.
<_>
<_>
4 7 2 2 -1.
<_>
4 7 1 1 2.
<_>
5 8 1 1 2.
<_>
<_>
5 7 3 1 -1.
<_>
6 7 1 1 3.
<_>
<_>
18 3 2 3 -1.
<_>
18 4 2 1 3.
<_>
<_>
13 1 4 2 -1.
<_>
13 1 2 2 2.
<_>
<_>
3 1 4 2 -1.
<_>
5 1 2 2 2.
<_>
<_>
3 0 5 2 -1.
<_>
3 1 5 1 2.
<_>
<_>
14 7 6 4 -1.
<_>
17 7 3 2 2.
<_>
14 9 3 2 2.
<_>
<_>
4 8 16 2 -1.
<_>
4 9 16 1 2.
<_>
<_>
2 11 5 6 -1.
<_>
2 13 5 2 3.
<_>
<_>
5 16 2 4 -1.
<_>
5 16 1 2 2.
<_>
6 18 1 2 2.
<_>
<_>
15 6 2 12 -1.
<_>
16 6 1 6 2.
<_>
15 12 1 6 2.
<_>
<_>
13 3 6 16 -1.
<_>
15 3 2 16 3.
<_>
<_>
4 5 12 12 -1.
<_>
4 5 6 6 2.
<_>
10 11 6 6 2.
<_>
<_>
5 1 10 13 -1.
<_>
10 1 5 13 2.
<_>
<_>
11 5 2 2 -1.
<_>
12 5 1 1 2.
<_>
11 6 1 1 2.
<_>
<_>
13 5 1 3 -1.
<_>
13 6 1 1 3.
<_>
<_>
7 4 2 4 -1.
<_>
7 4 1 2 2.
<_>
8 6 1 2 2.
<_>
<_>
7 5 6 4 -1.
<_>
10 5 3 4 2.
<_>
<_>
12 4 4 6 -1.
<_>
14 4 2 3 2.
<_>
12 7 2 3 2.
<_>
<_>
12 11 7 6 -1.
<_>
12 13 7 2 3.
<_>
<_>
5 6 6 6 -1.
<_>
7 6 2 6 3.
<_>
<_>
9 8 2 2 -1.
<_>
9 9 2 1 2.
<_>
<_>
15 6 2 2 -1.
<_>
16 6 1 1 2.
<_>
15 7 1 1 2.
<_>
<_>
14 7 4 4 -1.
<_>
16 7 2 2 2.
<_>
14 9 2 2 2.
<_>
<_>
5 5 6 2 -1.
<_>
7 5 2 2 3.
<_>
<_>
1 19 18 1 -1.
<_>
7 19 6 1 3.
<_>
<_>
12 3 3 3 -1.
<_>
12 4 3 1 3.
<_>
<_>
16 0 2 3 -1.
<_>
16 1 2 1 3.
<_>
<_>
5 3 3 3 -1.
<_>
5 4 3 1 3.
<_>
<_>
2 0 2 3 -1.
<_>
2 1 2 1 3.
<_>
<_>
15 6 2 2 -1.
<_>
16 6 1 1 2.
<_>
15 7 1 1 2.
<_>
<_>
10 13 1 6 -1.
<_>
10 16 1 3 2.
<_>
<_>
0 7 10 2 -1.
<_>
0 7 5 1 2.
<_>
5 8 5 1 2.
<_>
<_>
3 10 6 2 -1.
<_>
3 11 6 1 2.
<_>
<_>
12 18 4 2 -1.
<_>
12 19 4 1 2.
<_>
<_>
12 18 2 2 -1.
<_>
13 18 1 1 2.
<_>
12 19 1 1 2.
<_>
<_>
6 19 2 1 -1.
<_>
7 19 1 1 2.
<_>
<_>
0 4 2 16 -1.
<_>
0 4 1 8 2.
<_>
1 12 1 8 2.
<_>
<_>
16 1 4 9 -1.
<_>
16 4 4 3 3.
<_>
<_>
10 2 1 2 -1.
<_>
10 3 1 1 2.
<_>
<_>
4 14 4 6 -1.
<_>
4 14 2 3 2.
<_>
6 17 2 3 2.
<_>
<_>
4 15 1 4 -1.
<_>
4 17 1 2 2.
<_>
<_>
0 2 20 4 -1.
<_>
10 2 10 2 2.
<_>
0 4 10 2 2.
<_>
<_>
14 5 2 8 -1.
<_>
14 9 2 4 2.
<_>
<_>
5 12 4 5 -1.
<_>
7 12 2 5 2.
<_>
<_>
0 13 9 6 -1.
<_>
0 15 9 2 3.
<_>
<_>
9 14 11 3 -1.
<_>
9 15 11 1 3.
<_>
<_>
7 14 7 3 -1.
<_>
7 15 7 1 3.
<_>
<_>
3 6 2 2 -1.
<_>
3 6 1 1 2.
<_>
4 7 1 1 2.
<_>
<_>
6 7 2 7 -1.
<_>
7 7 1 7 2.
<_>
<_>
14 5 1 3 -1.
<_>
14 6 1 1 3.
<_>
<_>
13 4 4 3 -1.
<_>
13 5 4 1 3.
<_>
<_>
2 7 4 4 -1.
<_>
2 7 2 2 2.
<_>
4 9 2 2 2.
<_>
<_>
2 9 13 6 -1.
<_>
2 12 13 3 2.
<_>
<_>
10 1 3 4 -1.
<_>
11 1 1 4 3.
<_>
<_>
9 8 5 2 -1.
<_>
9 9 5 1 2.
<_>
<_>
0 14 11 3 -1.
<_>
0 15 11 1 3.
<_>
<_>
8 11 2 8 -1.
<_>
8 15 2 4 2.
<_>
<_>
5 11 10 6 -1.
<_>
5 14 10 3 2.
<_>
<_>
5 13 15 5 -1.
<_>
10 13 5 5 3.
<_>
<_>
8 10 1 10 -1.
<_>
8 15 1 5 2.
<_>
<_>
4 14 6 2 -1.
<_>
6 14 2 2 3.
<_>
<_>
7 14 7 3 -1.
<_>
7 15 7 1 3.
<_>
<_>
7 16 9 3 -1.
<_>
7 17 9 1 3.
<_>
<_>
8 7 3 3 -1.
<_>
8 8 3 1 3.
<_>
<_>
3 5 1 6 -1.
<_>
3 8 1 3 2.
<_>
<_>
6 5 11 2 -1.
<_>
6 6 11 1 2.
<_>
<_>
9 0 3 2 -1.
<_>
10 0 1 2 3.
<_>
<_>
5 5 1 3 -1.
<_>
5 6 1 1 3.
<_>
<_>
8 7 3 2 -1.
<_>
9 7 1 2 3.
<_>
<_>
5 2 10 6 -1.
<_>
10 2 5 3 2.
<_>
5 5 5 3 2.
<_>
<_>
8 4 6 4 -1.
<_>
8 4 3 4 2.
<_>
<_>
8 16 3 4 -1.
<_>
9 16 1 4 3.
<_>
<_>
9 13 2 6 -1.
<_>
9 13 1 3 2.
<_>
10 16 1 3 2.
<_>
<_>
9 8 3 1 -1.
<_>
10 8 1 1 3.
<_>
<_>
2 5 18 15 -1.
<_>
2 10 18 5 3.
<_>
<_>
1 3 6 2 -1.
<_>
4 3 3 2 2.
<_>
<_>
7 6 6 2 -1.
<_>
9 6 2 2 3.
<_>
<_>
8 17 4 3 -1.
<_>
8 18 4 1 3.
<_>
<_>
10 13 2 3 -1.
<_>
10 14 2 1 3.
<_>
<_>
0 10 20 4 -1.
<_>
0 12 20 2 2.
<_>
<_>
5 7 6 4 -1.
<_>
5 7 3 2 2.
<_>
8 9 3 2 2.
<_>
<_>
11 12 1 2 -1.
<_>
11 13 1 1 2.
<_>
<_>
10 10 2 3 -1.
<_>
10 11 2 1 3.
<_>
<_>
9 5 2 2 -1.
<_>
9 6 2 1 2.
<_>
<_>
4 4 1 10 -1.
<_>
4 9 1 5 2.
<_>
<_>
11 18 4 2 -1.
<_>
11 18 2 2 2.
<_>
<_>
12 18 3 2 -1.
<_>
12 19 3 1 2.
<_>
<_>
0 6 16 6 -1.
<_>
0 6 8 3 2.
<_>
8 9 8 3 2.
<_>
<_>
7 6 4 12 -1.
<_>
7 12 4 6 2.
<_>
<_>
11 18 4 2 -1.
<_>
11 18 2 2 2.
<_>
<_>
12 18 3 2 -1.
<_>
12 19 3 1 2.
<_>
<_>
8 12 1 2 -1.
<_>
8 13 1 1 2.
<_>
<_>
8 13 1 3 -1.
<_>
8 14 1 1 3.
<_>
<_>
11 18 4 2 -1.
<_>
11 18 2 2 2.
<_>
<_>
14 12 4 6 -1.
<_>
14 12 2 6 2.
<_>
<_>
6 0 3 4 -1.
<_>
7 0 1 4 3.
<_>
<_>
4 0 2 8 -1.
<_>
4 0 1 4 2.
<_>
5 4 1 4 2.
<_>
<_>
11 17 9 3 -1.
<_>
14 17 3 3 3.
<_>
<_>
16 2 4 5 -1.
<_>
16 2 2 5 2.
<_>
<_>
0 2 5 9 -1.
<_>
0 5 5 3 3.
<_>
<_>
7 2 3 2 -1.
<_>
8 2 1 2 3.
<_>
<_>
11 17 9 3 -1.
<_>
14 17 3 3 3.
<_>
<_>
16 2 4 5 -1.
<_>
16 2 2 5 2.
<_>
<_>
0 17 9 3 -1.
<_>
3 17 3 3 3.
<_>
<_>
0 2 4 5 -1.
<_>
2 2 2 5 2.
<_>
<_>
5 11 10 9 -1.
<_>
5 14 10 3 3.
<_>
<_>
9 6 3 3 -1.
<_>
9 7 3 1 3.
<_>
<_>
3 17 5 3 -1.
<_>
3 18 5 1 3.
<_>
<_>
7 5 4 7 -1.
<_>
9 5 2 7 2.
<_>
<_>
9 8 2 5 -1.
<_>
9 8 1 5 2.
<_>
<_>
2 2 18 2 -1.
<_>
2 3 18 1 2.
<_>
<_>
2 8 15 6 -1.
<_>
7 8 5 6 3.
<_>
<_>
9 8 2 5 -1.
<_>
10 8 1 5 2.
<_>
<_>
12 10 4 6 -1.
<_>
12 12 4 2 3.
<_>
<_>
14 3 6 2 -1.
<_>
14 4 6 1 2.
<_>
<_>
5 5 2 3 -1.
<_>
5 6 2 1 3.
<_>
<_>
4 6 3 3 -1.
<_>
4 7 3 1 3.
<_>
<_>
14 12 3 3 -1.
<_>
14 13 3 1 3.
<_>
<_>
6 12 11 3 -1.
<_>
6 13 11 1 3.
<_>
<_>
1 2 3 6 -1.
<_>
1 4 3 2 3.
<_>
<_>
1 0 4 7 -1.
<_>
3 0 2 7 2.
<_>
<_>
9 8 3 4 -1.
<_>
10 8 1 4 3.
<_>
<_>
10 9 2 2 -1.
<_>
10 10 2 1 2.
<_>
<_>
8 8 3 4 -1.
<_>
9 8 1 4 3.
<_>
<_>
4 4 10 10 -1.
<_>
4 9 10 5 2.
<_>
<_>
9 10 3 2 -1.
<_>
10 10 1 2 3.
<_>
<_>
9 10 3 2 -1.
<_>
9 11 3 1 2.
<_>
<_>
8 10 3 2 -1.
<_>
9 10 1 2 3.
<_>
<_>
2 4 14 12 -1.
<_>
2 4 7 6 2.
<_>
9 10 7 6 2.
<_>
<_>
10 12 1 6 -1.
<_>
10 15 1 3 2.
<_>
<_>
7 3 8 16 -1.
<_>
11 3 4 8 2.
<_>
7 11 4 8 2.
<_>
<_>
5 6 8 10 -1.
<_>
5 6 4 5 2.
<_>
9 11 4 5 2.
<_>
<_>
6 2 8 8 -1.
<_>
6 2 4 4 2.
<_>
10 6 4 4 2.
<_>
<_>
10 5 4 2 -1.
<_>
12 5 2 1 2.
<_>
10 6 2 1 2.
<_>
<_>
12 4 3 3 -1.
<_>
12 5 3 1 3.
<_>
<_>
4 19 12 1 -1.
<_>
8 19 4 1 3.
<_>
<_>
8 2 3 1 -1.
<_>
9 2 1 1 3.
<_>
<_>
13 17 4 3 -1.
<_>
13 18 4 1 3.
<_>
<_>
7 14 6 3 -1.
<_>
7 15 6 1 3.
<_>
<_>
9 14 2 3 -1.
<_>
9 15 2 1 3.
<_>
<_>
7 15 6 3 -1.
<_>
7 16 6 1 3.
<_>
<_>
10 18 3 2 -1.
<_>
11 18 1 2 3.
<_>
<_>
14 12 2 3 -1.
<_>
14 13 2 1 3.
<_>
<_>
4 10 4 6 -1.
<_>
4 12 4 2 3.
<_>
<_>
4 13 3 2 -1.
<_>
4 14 3 1 2.
<_>
<_>
9 16 2 3 -1.
<_>
9 17 2 1 3.
<_>
<_>
10 18 3 2 -1.
<_>
11 18 1 2 3.
<_>
<_>
7 18 3 2 -1.
<_>
8 18 1 2 3.
<_>
<_>
1 10 4 2 -1.
<_>
1 11 4 1 2.
<_>
<_>
12 4 6 3 -1.
<_>
12 5 6 1 3.
<_>
<_>
14 4 1 3 -1.
<_>
14 5 1 1 3.
<_>
<_>
2 4 6 3 -1.
<_>
2 5 6 1 3.
<_>
<_>
5 4 1 3 -1.
<_>
5 5 1 1 3.
<_>
<_>
14 12 3 3 -1.
<_>
14 13 3 1 3.
<_>
<_>
15 12 2 3 -1.
<_>
15 13 2 1 3.
<_>
<_>
3 16 4 3 -1.
<_>
3 17 4 1 3.
<_>
<_>
8 0 4 2 -1.
<_>
8 1 4 1 2.
<_>
<_>
0 0 20 1 -1.
<_>
0 0 10 1 2.
<_>
<_>
9 7 3 4 -1.
<_>
10 7 1 4 3.
<_>
<_>
0 0 20 1 -1.
<_>
10 0 10 1 2.
<_>
<_>
8 7 3 4 -1.
<_>
9 7 1 4 3.
<_>
<_>
1 6 19 3 -1.
<_>
1 7 19 1 3.
<_>
<_>
12 7 4 2 -1.
<_>
12 8 4 1 2.
<_>
<_>
7 8 3 3 -1.
<_>
7 9 3 1 3.
<_>
<_>
7 7 3 3 -1.
<_>
8 7 1 3 3.
<_>
<_>
2 9 16 3 -1.
<_>
2 10 16 1 3.
<_>
<_>
9 4 2 12 -1.
<_>
9 8 2 4 3.
<_>
<_>
7 3 2 5 -1.
<_>
8 3 1 5 2.
<_>
<_>
9 7 2 3 -1.
<_>
9 8 2 1 3.
<_>
<_>
9 14 4 3 -1.
<_>
9 15 4 1 3.
<_>
<_>
7 8 6 4 -1.
<_>
10 8 3 2 2.
<_>
7 10 3 2 2.
<_>
<_>
9 7 2 2 -1.
<_>
10 7 1 2 2.
<_>
<_>
5 5 6 6 -1.
<_>
7 5 2 6 3.
<_>
<_>
9 1 3 6 -1.
<_>
10 1 1 6 3.
<_>
<_>
4 5 12 2 -1.
<_>
8 5 4 2 3.
<_>
<_>
4 2 6 4 -1.
<_>
6 2 2 4 3.
<_>
<_>
4 7 8 2 -1.
<_>
4 8 8 1 2.
<_>
<_>
3 6 14 6 -1.
<_>
10 6 7 3 2.
<_>
3 9 7 3 2.
<_>
<_>
3 6 14 3 -1.
<_>
3 6 7 3 2.
<_>
<_>
0 5 2 2 -1.
<_>
0 6 2 1 2.
<_>
<_>
8 13 4 3 -1.
<_>
8 14 4 1 3.
<_>
<_>
13 0 3 20 -1.
<_>
14 0 1 20 3.
<_>
<_>
10 8 10 3 -1.
<_>
10 9 10 1 3.
<_>
<_>
4 0 3 20 -1.
<_>
5 0 1 20 3.
<_>
<_>
0 8 10 3 -1.
<_>
0 9 10 1 3.
<_>
<_>
12 5 3 4 -1.
<_>
13 5 1 4 3.
<_>
<_>
6 7 12 4 -1.
<_>
10 7 4 4 3.
<_>
<_>
1 14 6 6 -1.
<_>
1 14 3 3 2.
<_>
4 17 3 3 2.
<_>
<_>
1 17 6 2 -1.
<_>
1 18 6 1 2.
<_>
<_>
14 8 6 12 -1.
<_>
17 8 3 6 2.
<_>
14 14 3 6 2.
<_>
<_>
18 5 2 2 -1.
<_>
18 6 2 1 2.
<_>
<_>
3 16 4 2 -1.
<_>
3 16 2 1 2.
<_>
5 17 2 1 2.
<_>
<_>
2 16 6 2 -1.
<_>
4 16 2 2 3.
<_>
<_>
14 8 6 12 -1.
<_>
17 8 3 6 2.
<_>
14 14 3 6 2.
<_>
<_>
18 5 2 2 -1.
<_>
18 6 2 1 2.
<_>
<_>
5 16 9 2 -1.
<_>
8 16 3 2 3.
<_>
<_>
3 14 6 6 -1.
<_>
3 14 3 3 2.
<_>
6 17 3 3 2.
<_>
<_>
14 8 6 12 -1.
<_>
17 8 3 6 2.
<_>
14 14 3 6 2.
<_>
<_>
11 7 2 12 -1.
<_>
11 11 2 4 3.
<_>
<_>
0 8 6 12 -1.
<_>
0 8 3 6 2.
<_>
3 14 3 6 2.
<_>
<_>
7 7 2 12 -1.
<_>
7 11 2 4 3.
<_>
<_>
14 12 1 2 -1.
<_>
14 13 1 1 2.
<_>
<_>
12 13 8 1 -1.
<_>
12 13 4 1 2.
<_>
<_>
0 3 16 6 -1.
<_>
0 6 16 3 2.
<_>
<_>
1 4 8 2 -1.
<_>
1 4 4 1 2.
<_>
5 5 4 1 2.
<_>
<_>
14 12 1 2 -1.
<_>
14 13 1 1 2.
<_>
<_>
15 12 2 3 -1.
<_>
15 13 2 1 3.
<_>
<_>
8 16 3 3 -1.
<_>
8 17 3 1 3.
<_>
<_>
5 12 1 2 -1.
<_>
5 13 1 1 2.
<_>
<_>
13 4 3 15 -1.
<_>
14 4 1 15 3.
<_>
<_>
17 3 2 6 -1.
<_>
18 3 1 3 2.
<_>
17 6 1 3 2.
<_>
<_>
4 4 3 15 -1.
<_>
5 4 1 15 3.
<_>
<_>
1 3 2 6 -1.
<_>
1 3 1 3 2.
<_>
2 6 1 3 2.
<_>
<_>
7 15 12 4 -1.
<_>
7 17 12 2 2.
<_>
<_>
1 0 19 3 -1.
<_>
1 1 19 1 3.
<_>
<_>
3 17 10 2 -1.
<_>
3 17 5 1 2.
<_>
8 18 5 1 2.
<_>
<_>
2 5 10 15 -1.
<_>
2 10 10 5 3.
<_>
<_>
13 8 3 4 -1.
<_>
13 10 3 2 2.
<_>
<_>
19 13 1 2 -1.
<_>
19 14 1 1 2.
<_>
<_>
4 8 3 4 -1.
<_>
4 10 3 2 2.
<_>
<_>
0 13 1 2 -1.
<_>
0 14 1 1 2.
<_>
<_>
12 7 2 12 -1.
<_>
12 13 2 6 2.
<_>
<_>
14 7 2 2 -1.
<_>
15 7 1 1 2.
<_>
14 8 1 1 2.
<_>
<_>
5 3 8 2 -1.
<_>
5 4 8 1 2.
<_>
<_>
0 2 2 6 -1.
<_>
0 4 2 2 3.
<_>
<_>
18 2 2 12 -1.
<_>
19 2 1 6 2.
<_>
18 8 1 6 2.
<_>
<_>
18 1 1 2 -1.
<_>
18 2 1 1 2.
<_>
<_>
0 2 2 12 -1.
<_>
0 2 1 6 2.
<_>
1 8 1 6 2.
<_>
<_>
1 1 1 2 -1.
<_>
1 2 1 1 2.
<_>
<_>
16 4 4 14 -1.
<_>
18 4 2 7 2.
<_>
16 11 2 7 2.
<_>
<_>
10 14 1 6 -1.
<_>
10 17 1 3 2.
<_>
<_>
0 4 4 14 -1.
<_>
0 4 2 7 2.
<_>
2 11 2 7 2.
<_>
<_>
9 14 1 6 -1.
<_>
9 17 1 3 2.
<_>
<_>
9 14 4 3 -1.
<_>
9 15 4 1 3.
<_>
<_>
4 7 12 2 -1.
<_>
8 7 4 2 3.
<_>
<_>
0 8 4 3 -1.
<_>
0 9 4 1 3.
<_>
<_>
4 7 2 2 -1.
<_>
4 7 1 1 2.
<_>
5 8 1 1 2.
<_>
<_>
13 7 2 1 -1.
<_>
13 7 1 1 2.
<_>
<_>
11 4 4 5 -1.
<_>
11 4 2 5 2.
<_>
<_>
4 8 3 3 -1.
<_>
5 8 1 3 3.
<_>
<_>
0 3 8 1 -1.
<_>
4 3 4 1 2.
<_>
<_>
13 7 2 1 -1.
<_>
13 7 1 1 2.
<_>
<_>
14 7 3 2 -1.
<_>
15 7 1 2 3.
<_>
<_>
5 7 2 1 -1.
<_>
6 7 1 1 2.
<_>
<_>
3 7 3 2 -1.
<_>
4 7 1 2 3.
<_>
<_>
18 5 2 2 -1.
<_>
18 6 2 1 2.
<_>
<_>
12 14 2 2 -1.
<_>
13 14 1 1 2.
<_>
12 15 1 1 2.
<_>
<_>
0 5 2 2 -1.
<_>
0 6 2 1 2.
<_>
<_>
6 14 2 2 -1.
<_>
6 14 1 1 2.
<_>
7 15 1 1 2.
<_>
<_>
7 12 6 5 -1.
<_>
9 12 2 5 3.
<_>
<_>
12 17 5 2 -1.
<_>
12 18 5 1 2.
<_>
<_>
1 11 6 3 -1.
<_>
4 11 3 3 2.
<_>
<_>
1 9 6 3 -1.
<_>
4 9 3 3 2.
<_>
<_>
12 7 2 12 -1.
<_>
12 13 2 6 2.
<_>
<_>
8 7 5 3 -1.
<_>
8 8 5 1 3.
<_>
<_>
6 7 2 12 -1.
<_>
6 13 2 6 2.
<_>
<_>
1 2 9 18 -1.
<_>
4 2 3 18 3.
<_>
<_>
12 17 5 2 -1.
<_>
12 18 5 1 2.
<_>
<_>
4 7 12 2 -1.
<_>
4 7 6 2 2.
<_>
<_>
6 7 6 1 -1.
<_>
8 7 2 1 3.
<_>
<_>
7 3 3 2 -1.
<_>
8 3 1 2 3.
<_>
<_>
9 4 3 1 -1.
<_>
10 4 1 1 3.
<_>
<_>
11 11 3 1 -1.
<_>
12 11 1 1 3.
<_>
<_>
8 4 3 1 -1.
<_>
9 4 1 1 3.
<_>
<_>
6 11 3 1 -1.
<_>
7 11 1 1 3.
<_>
<_>
12 13 6 6 -1.
<_>
12 15 6 2 3.
<_>
<_>
14 13 1 6 -1.
<_>
14 15 1 2 3.
<_>
<_>
2 13 6 6 -1.
<_>
2 15 6 2 3.
<_>
<_>
1 5 18 1 -1.
<_>
7 5 6 1 3.
<_>
<_>
4 7 12 2 -1.
<_>
10 7 6 1 2.
<_>
4 8 6 1 2.
<_>
<_>
6 1 8 10 -1.
<_>
10 1 4 5 2.
<_>
6 6 4 5 2.
<_>
<_>
3 13 4 3 -1.
<_>
3 14 4 1 3.
<_>
<_>
6 13 4 3 -1.
<_>
6 14 4 1 3.
<_>
<_>
9 14 4 3 -1.
<_>
9 15 4 1 3.
<_>
<_>
12 9 2 3 -1.
<_>
12 10 2 1 3.
<_>
<_>
7 14 4 3 -1.
<_>
7 15 4 1 3.
<_>
<_>
9 0 2 1 -1.
<_>
10 0 1 1 2.
<_>
<_>
5 0 10 5 -1.
<_>
5 0 5 5 2.
<_>
<_>
6 6 8 7 -1.
<_>
6 6 4 7 2.
<_>
<_>
5 0 10 5 -1.
<_>
10 0 5 5 2.
<_>
<_>
6 6 8 7 -1.
<_>
10 6 4 7 2.
<_>
<_>
5 9 10 8 -1.
<_>
10 9 5 4 2.
<_>
5 13 5 4 2.
<_>
<_>
10 0 4 10 -1.
<_>
12 0 2 5 2.
<_>
10 5 2 5 2.
<_>
<_>
1 4 8 3 -1.
<_>
1 5 8 1 3.
<_>
<_>
4 4 8 3 -1.
<_>
4 5 8 1 3.
<_>
<_>
9 7 4 3 -1.
<_>
9 8 4 1 3.
<_>
<_>
12 8 3 12 -1.
<_>
12 14 3 6 2.
<_>
<_>
7 7 4 3 -1.
<_>
7 8 4 1 3.
<_>
<_>
5 8 3 12 -1.
<_>
5 14 3 6 2.
<_>
<_>
10 0 7 6 -1.
<_>
10 2 7 2 3.
<_>
<_>
2 1 18 1 -1.
<_>
8 1 6 1 3.
<_>
<_>
5 0 3 8 -1.
<_>
6 0 1 8 3.
<_>
<_>
4 7 4 2 -1.
<_>
4 8 4 1 2.
================================================
FILE: samples/pom.xml
================================================
4.0.0
org.bytedeco.javacv
demo
1.5.13
1.8
1.8
org.bytedeco
javacv-platform
1.5.13
org.bytedeco
opencv-platform-gpu
4.13.0-1.5.13
org.bytedeco
ffmpeg-platform-gpl
8.0.1-1.5.13
.
maven-compiler-plugin
FaceApplet.java
FacePreview.java
RecordActivity.java
================================================
FILE: src/main/java/cl/eye/CLCamera.java
================================================
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//
// This file is part of CL-EyeMulticam SDK
//
// Java JNI CLEyeMulticam wrapper
//
// It allows the use of multiple CL-Eye cameras in your own Java applications
//
// For updates and file downloads go to: http://codelaboratories.com/research/view/cl-eye-muticamera-sdk
//
// Copyright 2008-2012 (c) Code Laboratories, Inc. All rights reserved.
//
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
package cl.eye;
//import processing.core.*;
public class CLCamera
{
// camera color mode
public static int CLEYE_MONO_PROCESSED = 0;
public static int CLEYE_COLOR_PROCESSED = 1;
public static int CLEYE_MONO_RAW = 2;
public static int CLEYE_COLOR_RAW = 3;
public static int CLEYE_BAYER_RAW = 4;
// camera resolution
public static int CLEYE_QVGA = 0;
public static int CLEYE_VGA = 1;
// camera sensor parameters
public static int CLEYE_AUTO_GAIN = 0; // [0, 1]
public static int CLEYE_GAIN = 1; // [0, 79]
public static int CLEYE_AUTO_EXPOSURE = 2; // [0, 1]
public static int CLEYE_EXPOSURE = 3; // [0, 511]
public static int CLEYE_AUTO_WHITEBALANCE = 4; // [0, 1]
public static int CLEYE_WHITEBALANCE_RED = 5; // [0, 255]
public static int CLEYE_WHITEBALANCE_GREEN = 6; // [0, 255]
public static int CLEYE_WHITEBALANCE_BLUE = 7; // [0, 255]
// camera linear transform parameters
public static int CLEYE_HFLIP = 8; // [0, 1]
public static int CLEYE_VFLIP = 9; // [0, 1]
public static int CLEYE_HKEYSTONE = 10; // [-500, 500]
public static int CLEYE_VKEYSTONE = 11; // [-500, 500]
public static int CLEYE_XOFFSET = 12; // [-500, 500]
public static int CLEYE_YOFFSET = 13; // [-500, 500]
public static int CLEYE_ROTATION = 14; // [-500, 500]
public static int CLEYE_ZOOM = 15; // [-500, 500]
// camera non-linear transform parameters
public static int CLEYE_LENSCORRECTION1 = 16; // [-500, 500]
public static int CLEYE_LENSCORRECTION2 = 17; // [-500, 500]
public static int CLEYE_LENSCORRECTION3 = 18; // [-500, 500]
public static int CLEYE_LENSBRIGHTNESS = 19; // [-500, 500]
native static int CLEyeGetCameraCount();
native static String CLEyeGetCameraUUID(int index);
native static int CLEyeCreateCamera(int cameraIndex, int mode, int resolution, int framerate);
native static boolean CLEyeDestroyCamera(int cameraIndex);
native static boolean CLEyeCameraStart(int cameraInstance);
native static boolean CLEyeCameraStop(int cameraInstance);
native static boolean CLEyeSetCameraParameter(int cameraInstance, int param, int val);
native static int CLEyeGetCameraParameter(int cameraInstance, int param);
native static boolean CLEyeCameraGetFrame(int cameraInstance, int[] imgData, int waitTimeout);
private int cameraInstance = 0;
// private PApplet parent;
private static boolean libraryLoaded = false;
private static String dllpathx32 = "C://Program Files//Code Laboratories//CL-Eye Platform SDK//Bin//CLEyeMulticam.dll";
private static String dllpathx64 = "C://Program Files (x86)//Code Laboratories//CL-Eye Platform SDK//Bin//CLEyeMulticam.dll";
// static methods
static
{
try
{
System.load(dllpathx32);
libraryLoaded = true;
System.out.println("CLEyeMulticam.dll loaded");
}
catch(UnsatisfiedLinkError e1)
{
System.out.println("(1) Could not find the CLEyeMulticam.dll");
try
{
System.load(dllpathx64);
libraryLoaded = true;
System.out.println("CLEyeMulticam.dll loaded");
}
catch(UnsatisfiedLinkError e2)
{
System.out.println("(2) Could not find the CLEyeMulticam.dll");
}
}
}
public static boolean IsLibraryLoaded()
{
return libraryLoaded;
}
public static void loadLibrary(String libraryPath)
{
if(libraryLoaded) return;
try
{
System.load(libraryPath);
System.out.println("CLEyeMulticam.dll loaded");
}
catch(UnsatisfiedLinkError e1)
{
System.out.println("(3) Could not find the CLEyeMulticam.dll (Custom Path)");
}
}
public static int cameraCount()
{
return CLEyeGetCameraCount();
}
public static String cameraUUID(int index)
{
return CLEyeGetCameraUUID(index);
}
// public methods
// public CLCamera(PApplet parent)
// {
// this.parent = parent;
// parent.registerDispose(this);
// }
public void dispose()
{
stopCamera();
destroyCamera();
}
public boolean createCamera(int cameraIndex, int mode, int resolution, int framerate)
{
cameraInstance = CLEyeCreateCamera(cameraIndex, mode, resolution, framerate);
return cameraInstance != 0;
}
public boolean destroyCamera()
{
return CLEyeDestroyCamera(cameraInstance);
}
public boolean startCamera()
{
return CLEyeCameraStart(cameraInstance);
}
public boolean stopCamera()
{
return CLEyeCameraStop(cameraInstance);
}
public boolean getCameraFrame(int[] imgData, int waitTimeout)
{
return CLEyeCameraGetFrame(cameraInstance, imgData, waitTimeout);
}
public boolean setCameraParam(int param, int val)
{
return CLEyeSetCameraParameter(cameraInstance, param, val);
}
public int getCameraParam(int param)
{
return CLEyeGetCameraParameter(cameraInstance, param);
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/AndroidFrameConverter.java
================================================
/*
* Copyright (C) 2015-2016 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import android.graphics.Bitmap;
import android.hardware.Camera;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
/**
* A utility class to copy data between {@link Frame} and {@link Bitmap}.
* Since {@link Bitmap} does not expose its internal buffer, we cannot share
* allocated memory with {@link Frame}.
*
* This class is not optimized for speed. For best performance, convert first
* your data to and from RGBA with optimized functions from FFmpeg or OpenCV.
* Further, pixel formats other than grayscale, BGR, and RGBA are not well
* supported. Their conversions might fail in undefined ways.
*
* @author Samuel Audet
*/
public class AndroidFrameConverter extends FrameConverter {
Bitmap bitmap;
ByteBuffer buffer;
byte[] row;
/**
* Convert YUV 4:2:0 SP (NV21) data to BGR, as received, for example,
* via {@link Camera.PreviewCallback#onPreviewFrame(byte[],Camera)}.
*/
public Frame convert(byte[] data, int width, int height) {
if (frame == null || frame.imageWidth != width
|| frame.imageHeight != height || frame.imageChannels != 3) {
if (frame != null) {
frame.close();
}
frame = new Frame(width, height, Frame.DEPTH_UBYTE, 3);
}
ByteBuffer out = (ByteBuffer)frame.image[0];
int stride = frame.imageStride;
// ported from https://android.googlesource.com/platform/development/+/master/tools/yuv420sp2rgb/yuv420sp2rgb.c
int offset = height * width;
for (int i = 0; i < height; i++) {
for (int j = 0; j < width; j++) {
int Y = data[i * width + j] & 0xFF;
int V = data[offset + (i/2) * width + 2 * (j/2) ] & 0xFF;
int U = data[offset + (i/2) * width + 2 * (j/2) + 1] & 0xFF;
// Yuv Convert
Y -= 16;
U -= 128;
V -= 128;
if (Y < 0)
Y = 0;
// R = (int)(1.164 * Y + 2.018 * U);
// G = (int)(1.164 * Y - 0.813 * V - 0.391 * U);
// B = (int)(1.164 * Y + 1.596 * V);
int B = (int)(1192 * Y + 2066 * U);
int G = (int)(1192 * Y - 833 * V - 400 * U);
int R = (int)(1192 * Y + 1634 * V);
R = Math.min(262143, Math.max(0, R));
G = Math.min(262143, Math.max(0, G));
B = Math.min(262143, Math.max(0, B));
R >>= 10; R &= 0xff;
G >>= 10; G &= 0xff;
B >>= 10; B &= 0xff;
out.put(i * stride + 3 * j, (byte)B);
out.put(i * stride + 3 * j + 1, (byte)G);
out.put(i * stride + 3 * j + 2, (byte)R);
}
}
return frame;
}
@Override public Frame convert(Bitmap bitmap) {
if (bitmap == null) {
return null;
}
int channels = 0;
switch (bitmap.getConfig()) {
case ALPHA_8: channels = 1; break;
case RGB_565:
case ARGB_4444: channels = 2; break;
case ARGB_8888: channels = 4; break;
default: assert false;
}
if (frame == null || frame.imageWidth != bitmap.getWidth() || frame.imageStride != bitmap.getRowBytes()
|| frame.imageHeight != bitmap.getHeight() || frame.imageChannels != channels) {
if (frame != null) {
frame.close();
}
frame = new Frame(bitmap.getWidth(), bitmap.getHeight(), Frame.DEPTH_UBYTE, channels, bitmap.getRowBytes());
}
bitmap.copyPixelsToBuffer(frame.image[0].position(0));
return frame;
}
ByteBuffer gray2rgba(ByteBuffer in, int width, int height, int stride, int rowBytes) {
if (buffer == null || buffer.capacity() < height * rowBytes) {
buffer = ByteBuffer.allocate(height * rowBytes);
}
if (row == null || row.length != stride)
row = new byte[stride];
for (int y = 0; y < height; y++) {
in.position(y * stride);
in.get(row);
for (int x = 0; x < width; x++) {
// GRAY -> RGBA
byte B = row[x];
int rgba = (B & 0xff) << 24 |
(B & 0xff) << 16 |
(B & 0xff) << 8 | 0xff;
buffer.putInt(y * rowBytes + 4 * x, rgba);
}
}
return buffer;
}
ByteBuffer bgr2rgba(ByteBuffer in, int width, int height, int stride, int rowBytes) {
if (!in.order().equals(ByteOrder.LITTLE_ENDIAN)) {
in = in.order(ByteOrder.LITTLE_ENDIAN);
}
if (buffer == null || buffer.capacity() < height * rowBytes) {
buffer = ByteBuffer.allocate(height * rowBytes);
}
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
// BGR -> RGBA
int rgb;
if (x < width - 1 || y < height - 1) {
rgb = in.getInt(y * stride + 3 * x);
} else {
int b = in.get(y * stride + 3 * x ) & 0xff;
int g = in.get(y * stride + 3 * x + 1) & 0xff;
int r = in.get(y * stride + 3 * x + 2) & 0xff;
rgb = (r << 16) | (g << 8) | b;
}
buffer.putInt(y * rowBytes + 4 * x, (rgb << 8) | 0xff);
}
}
return buffer;
}
@Override public Bitmap convert(Frame frame) {
if (frame == null || frame.image == null) {
return null;
}
Bitmap.Config config = null;
switch (frame.imageChannels) {
case 2: config = Bitmap.Config.RGB_565; break;
case 1:
case 3:
case 4: config = Bitmap.Config.ARGB_8888; break;
default: assert false;
}
if (bitmap == null || bitmap.getWidth() != frame.imageWidth
|| bitmap.getHeight() != frame.imageHeight || bitmap.getConfig() != config) {
bitmap = Bitmap.createBitmap(frame.imageWidth, frame.imageHeight, config);
}
// assume frame.imageDepth == Frame.DEPTH_UBYTE
ByteBuffer in = (ByteBuffer)frame.image[0];
int width = frame.imageWidth;
int height = frame.imageHeight;
int stride = frame.imageStride;
int rowBytes = bitmap.getRowBytes();
if (frame.imageChannels == 1) {
gray2rgba(in, width, height, stride, rowBytes);
bitmap.copyPixelsFromBuffer(buffer.position(0));
} else if (frame.imageChannels == 3) {
bgr2rgba(in, width, height, stride, rowBytes);
bitmap.copyPixelsFromBuffer(buffer.position(0));
} else {
// assume matching strides
bitmap.copyPixelsFromBuffer(in.position(0));
}
return bitmap;
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/BaseChildSettings.java
================================================
/*
* Copyright (C) 2009-2011 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.beans.PropertyChangeEvent;
import java.beans.PropertyChangeListener;
import java.beans.PropertyVetoException;
import java.beans.beancontext.BeanContextChildSupport;
import java.util.ListResourceBundle;
import java.util.concurrent.Callable;
import java.util.logging.Level;
import java.util.logging.LogRecord;
/**
*
* @author Samuel Audet
*/
public class BaseChildSettings extends BeanContextChildSupport implements Comparable {
public void addPropertyChangeListener(PropertyChangeListener listener) {
pcSupport.addPropertyChangeListener(listener);
}
public void removePropertyChangeListener(PropertyChangeListener listener) {
pcSupport.removePropertyChangeListener(listener);
}
public int compareTo(BaseChildSettings o) {
return getName().compareTo(o.getName());
}
protected String getName() {
return "";
}
public static class PropertyVetoExceptionThatNetBeansLikes extends PropertyVetoException implements Callable {
public PropertyVetoExceptionThatNetBeansLikes(String mess, PropertyChangeEvent evt) {
super(mess, evt);
}
public Object call() throws Exception {
LogRecord lg = new LogRecord(Level.ALL, getMessage());
lg.setResourceBundle(new ListResourceBundle() {
protected Object[][] getContents() {
return new Object[][] { {getMessage(), getMessage()} };
}
});
return new LogRecord[] { lg };
}
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/BaseSettings.java
================================================
/*
* Copyright (C) 2009-2011 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.beans.PropertyChangeListener;
import java.beans.beancontext.BeanContextSupport;
import java.util.Arrays;
/**
*
* @author Samuel Audet
*/
public class BaseSettings extends BeanContextSupport implements Comparable {
public void addPropertyChangeListener(PropertyChangeListener listener) {
pcSupport.addPropertyChangeListener(listener);
for (Object s : toArray()) {
if (s instanceof BaseChildSettings) {
((BaseChildSettings)s).addPropertyChangeListener(listener);
} else if (s instanceof BaseSettings) {
((BaseSettings)s).addPropertyChangeListener(listener);
}
}
}
public void removePropertyChangeListener(PropertyChangeListener listener) {
pcSupport.removePropertyChangeListener(listener);
for (Object s : toArray()) {
if (s instanceof BaseChildSettings) {
((BaseChildSettings)s).removePropertyChangeListener(listener);
} else if (s instanceof BaseSettings) {
((BaseSettings)s).addPropertyChangeListener(listener);
}
}
}
public int compareTo(BaseSettings o) {
return getName().compareTo(o.getName());
}
protected String getName() {
return "";
}
@Override public Object[] toArray() {
Object[] a = super.toArray();
Arrays.sort(a);
return a;
}
@Override public Object[] toArray(Object[] a) {
a = super.toArray(a);
Arrays.sort(a);
return a;
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/Blobs.java
================================================
package org.bytedeco.javacv;
import org.bytedeco.opencv.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_core.*;
//***************************************************************//
//* Blob analysis package Version3.0 3 Oct 2012 *//
//* - Version 1.0: 8 Aug 2003 *//
//* - Version 1.2: 3 Jan 2008 *//
//* - Version 1.3: 5 Jan 2008 Add BLOBCOLOR *//
//* - Version 1.4: 13 January 2008 Add ROI function *//
//* - Version 1.5: 13 April 2008 Fix perimeter on Region 0 *//
//* - Version 1.6: 1 May 2008 Reduce size of working storage *//
//* - Version 1.7: 2 May 2008 Speed up run code initialization *//
//* - Version 1.8: 4 May 2008 Fix bugs in perimeter & Reg 0 *//
//* - Version 2.0: 3 Jan 2009 Add labeling functionality *//
//* - Version 3.0: 3 Oct 2012 Convert to Java *//
//* - Eliminate labeling functionality (but it's still there) *//
//* - Simplify (at slight expense of performance) *//
//* - Reduce to 4 connectivity *//
//* *//
//* Input: IplImage binary image *//
//* Output: attributes of each connected region *//
//* Internal data: labeled array (could easily be externalized) *//
//* Author: Dave Grossman *//
//* Email: dgrossman2@gmail.com *//
//* Acknowledgement: my code is based on an algorithm that was *//
//* to the best of my knowledge originally developed by Gerry *//
//* Agin of SRI around the year 1973. I have not been able to *//
//* find any published references to his earlier work. I posted *//
//* early versions of my program to OpenCV, where they morphed *//
//* eventually into cvBlobsLib. *//
//* *//
//* As the author of this code, I place all of this code into *//
//* the public domain. Users can use it for any legal purpose. *//
//* *//
//* - Dave Grossman *//
//* *//
//* Typical calling sequence: *//
//* Blobs Blob = new Blobs(); *//
//* Blob.BlobAnalysis( *//
//* image3, // image *//
//* -1, -1, // ROI start col, row (-1 means full image) *//
//* -1, -1, // ROI cols, rows *//
//* 0, // border (0 = black; 1 = white) *//
//* 20); // minarea *//
//* Blob.PrintRegionData(); *//
//* int BlobLabel = Blob.NextRegion( *//
//* -1, // parentcolor (-1 = ignore) *//
//* 0, // color (0 = black; 1 = white; -1 = ignore *//
//* 100, // minarea *//
//* 500, // maxarea *//
//* 15); // starting label (default 0) *//
//* *//
//* Ellipse properties can be derived from moments: *//
//* h = (XX + YY) / 2 *//
//* Major axis = h + sqrt ( h^2 - XX * YY + XY^2) *//
//* Minor axis = h - sqrt ( h^2 - XX * YY2 + XY^2) *//
//* Eccentricity = (sqrt(abs(XX - YY)) + 4 * XY)/AREA *//
//***************************************************************//
public class Blobs
{
// The following parameters should be configured by the user:
// On ScanSnap Manager, "Best" setting = 300dpi gray level
// jpg compression is set to minimum so that quality is highest
// Each page jpg image is then a little under 1 MB
static int BLOBROWCOUNT = 3500; // 11 inches * 8.5 inches standard page
static int BLOBCOLCOUNT = 2700; // with some added cushion to be safe
// Allow for vast number of blobs so there is no memory overrun
static int BLOBTOTALCOUNT = (BLOBROWCOUNT + BLOBCOLCOUNT) * 5;
//--------------------------------------------------------------
// Do not change anything below this line
public static int BLOBLABEL = 0;
public static int BLOBPARENT = 1;
public static int BLOBCOLOR = 2;
public static int BLOBAREA = 3;
public static int BLOBPERIMETER = 4;
public static int BLOBSUMX = 5;
public static int BLOBSUMY = 6;
public static int BLOBSUMXX = 7;
public static int BLOBSUMYY = 8;
public static int BLOBSUMXY = 9;
public static int BLOBMINX = 10;
public static int BLOBMAXX = 11;
public static int BLOBMINY = 12;
public static int BLOBMAXY = 13;
public static int BLOBDATACOUNT = 14;
public static int [][] LabelMat = new int [BLOBROWCOUNT][BLOBCOLCOUNT];
public static double [][] RegionData = new double [BLOBTOTALCOUNT][BLOBDATACOUNT];
public static int MaxLabel;
public int LabelA, LabelB, LabelC, LabelD;
public int ColorA, ColorB, ColorC, ColorD;
public int jrow, jcol; // index within ROI
public static int [] SubsumedLabel = new int [BLOBTOTALCOUNT];
public static int [] CondensationMap = new int [BLOBTOTALCOUNT];
// Print out all the data for all the regions (blobs)
public void PrintRegionData() { PrintRegionData(0, MaxLabel); }
public void PrintRegionData(int Label0, int Label1)
{
if(Label0 < 0) Label0 = 0;
if(Label1 > MaxLabel) Label1 = MaxLabel;
if(Label1 < Label0) return;
for(int Label = Label0; Label <= Label1; Label++)
{
double [] Property = RegionData[Label];
int ThisLabel = (int)Property[BLOBLABEL];
int ThisParent = (int)Property[BLOBPARENT];
int ThisColor = (int)Property[BLOBCOLOR];
double ThisArea = Property[BLOBAREA];
double ThisPerimeter = Property[BLOBPERIMETER];
double ThisSumX = Property[BLOBSUMX];
double ThisSumY = Property[BLOBSUMY];
double ThisSumXX = Property[BLOBSUMXX];
double ThisSumYY = Property[BLOBSUMYY];
double ThisSumXY = Property[BLOBSUMXY];
int ThisMinX = (int)Property[BLOBMINX];
int ThisMaxX = (int)Property[BLOBMAXX];
int ThisMinY = (int)Property[BLOBMINY];
int ThisMaxY = (int)Property[BLOBMAXY];
String Str1 = " " + Label + ": L[" + ThisLabel + "] P[" + ThisParent + "] C[" + ThisColor + "]";
String Str2 = " AP[" + ThisArea + ", " + ThisPerimeter + "]";
String Str3 = " M1[" + ThisSumX + ", " + ThisSumY + "] M2[" + ThisSumXX + ", " + ThisSumYY + ", " + ThisSumXY + "]";
String Str4 = " MINMAX[" + ThisMinX + ", " + ThisMaxX + ", " + ThisMinY + ", " + ThisMaxY + "]";
String Str = Str1 + Str2 + Str3 + Str4;
System.out.println(Str);
}
System.out.println();
}
// Determine the next (higher number) region that meets the desired conditions
public static int NextRegion(int Parent, int Color, double MinArea, double MaxArea, int Label)
{
double DParent = (double) Parent;
double DColor = (double) Color; if(DColor > 0) DColor = 1;
int i;
for(i = Label; i <= MaxLabel; i++)
{
double [] Region = RegionData[i];
double ThisParent = Region[BLOBPARENT];
double ThisColor = Region[BLOBCOLOR];
if(DParent >= 0 && DParent != ThisParent) continue;
if(DColor >= 0 && DColor != ThisColor) continue;
if(Region[BLOBAREA] < MinArea || Region[BLOBAREA] > MaxArea) continue;
break; // We have a match!
}
if(i > MaxLabel) i = -1; // Use -1 to flag that there was no match
return i;
}
// Determine the prior (lower number) region that meets the desired conditions
public static int PriorRegion(int Parent, int Color, double MinArea, double MaxArea, int Label)
{
double DParent = (double) Parent;
double DColor = (double) Color; if(DColor > 0) DColor = 1;
int i;
for(i = Label; i >= 0; i--)
{
double [] Region = RegionData[i];
double ThisParent = Region[BLOBPARENT];
double ThisColor = Region[BLOBCOLOR];
if(DParent >= 0 && DParent != ThisParent) continue;
if(DColor >= 0 && DColor != ThisColor) continue;
if(Region[BLOBAREA] < MinArea || Region[BLOBAREA] > MaxArea) continue;
break; // We have a match!
}
if(i < 0) i = -1; // Use -1 to flag that there was no match
return i;
}
public void ResetRegion(int Label)
{
double [] RegionD = RegionData[Label];
RegionD[BLOBLABEL] =
RegionD[BLOBPARENT] =
RegionD[BLOBCOLOR] =
RegionD[BLOBAREA] =
RegionD[BLOBPERIMETER] =
RegionD[BLOBSUMX] =
RegionD[BLOBSUMY] =
RegionD[BLOBSUMXX] =
RegionD[BLOBSUMYY] =
RegionD[BLOBSUMXY] =
RegionD[BLOBMINX] =
RegionD[BLOBMAXX] =
RegionD[BLOBMINY] =
RegionD[BLOBMAXY] = 0.0;
System.arraycopy(RegionD,0,RegionData[Label],0,BLOBDATACOUNT); // RegionData[Label] <- RegionD;
}
public void OldRegion(
int NewLabelD, // 3rd update this (may be the same as Label1 or Label2)
int Label1, // 1st increment this by 1
int Label2) // 2nd increment this by 1
{
int DeltaPerimeter = 0;
if(Label1 >= 0 && Label1 != NewLabelD)
{
DeltaPerimeter++;
double [] Region1 = RegionData[Label1];
Region1[BLOBPERIMETER]++;
System.arraycopy(Region1,0,RegionData[Label1],0,BLOBDATACOUNT); // RegionData[Label1] <- Region1;
}
if(Label2 >= 0 && Label2 != NewLabelD)
{
DeltaPerimeter++;
double [] Region2 = RegionData[Label2];
Region2[BLOBPERIMETER]++;
System.arraycopy(Region2,0,RegionData[Label2],0,BLOBDATACOUNT); // RegionData[Label2] <- Region2;
}
LabelD = NewLabelD;
double [] RegionD = RegionData[LabelD];
RegionD[BLOBLABEL] = LabelD;
RegionD[BLOBPARENT] += 0.0; // no change
RegionD[BLOBCOLOR] += 0.0; // no change
RegionD[BLOBAREA] += 1.0;
RegionD[BLOBPERIMETER] += DeltaPerimeter;
RegionD[BLOBSUMX] += jcol;
RegionD[BLOBSUMY] += jrow;
RegionD[BLOBSUMXX] += jcol*jcol;
RegionD[BLOBSUMYY] += jrow*jrow;
RegionD[BLOBSUMXY] += jcol*jrow;
RegionD[BLOBMINX] = Math.min(RegionD[BLOBMINX], jcol);
RegionD[BLOBMAXX] = Math.max(RegionD[BLOBMAXX], jcol);
RegionD[BLOBMINY] = Math.min(RegionD[BLOBMINY], jrow);
RegionD[BLOBMAXY] = Math.max(RegionD[BLOBMAXY], jrow);
System.arraycopy(RegionD,0,RegionData[LabelD],0,BLOBDATACOUNT); // RegionData[LabelD] <- RegionD;
}
public void NewRegion(int ParentLabel)
{
LabelD = ++MaxLabel;
double [] RegionD = RegionData[LabelD];
RegionD[BLOBLABEL] = LabelD;
RegionD[BLOBPARENT] = (double) ParentLabel;
RegionD[BLOBCOLOR] = ColorD;
RegionD[BLOBAREA] = 1.0;
RegionD[BLOBPERIMETER] = 2.0;
RegionD[BLOBSUMX] = jcol;
RegionD[BLOBSUMY] = jrow;
RegionD[BLOBSUMXX] = jcol*jcol;
RegionD[BLOBSUMYY] = jrow*jrow;
RegionD[BLOBSUMXY] = jcol*jrow;
RegionD[BLOBMINX] = jcol;
RegionD[BLOBMAXX] = jcol;
RegionD[BLOBMINY] = jrow;
RegionD[BLOBMAXY] = jrow;
System.arraycopy(RegionD,0,RegionData[LabelD],0,BLOBDATACOUNT); // RegionData[LabelD] <- RegionD;
SubsumedLabel[LabelD] = -1; // Flag label as not subsumed
double [] RegionB = RegionData[LabelB];
RegionB[BLOBPERIMETER]++;
System.arraycopy(RegionB,0,RegionData[LabelB],0,BLOBDATACOUNT); // RegionData[LabelB] <- RegionB;
double [] RegionC = RegionData[LabelC];
RegionC[BLOBPERIMETER]++;
System.arraycopy(RegionC,0,RegionData[LabelC],0,BLOBDATACOUNT); // RegionData[LabelC] <- RegionC;
}
public void Subsume(int GoodLabel, int BadLabel, int PSign) // Combine data with parent
{
LabelD = GoodLabel;
double [] GoodRegion = RegionData[GoodLabel];
double [] BadRegion = RegionData[BadLabel];
GoodRegion[BLOBLABEL] = GoodRegion[BLOBLABEL]; // no change
GoodRegion[BLOBPARENT] = GoodRegion[BLOBPARENT]; // no change
GoodRegion[BLOBCOLOR] = GoodRegion[BLOBCOLOR]; // no change
GoodRegion[BLOBAREA] += BadRegion[BLOBAREA];
GoodRegion[BLOBPERIMETER] += BadRegion[BLOBPERIMETER] * PSign; // + external or - internal perimeter
GoodRegion[BLOBSUMX] += BadRegion[BLOBSUMX];
GoodRegion[BLOBSUMY] += BadRegion[BLOBSUMY];
GoodRegion[BLOBSUMXX] += BadRegion[BLOBSUMXX];
GoodRegion[BLOBSUMYY] += BadRegion[BLOBSUMYY];
GoodRegion[BLOBSUMXY] += BadRegion[BLOBSUMXY];
GoodRegion[BLOBMINX] = Math.min(GoodRegion[BLOBMINX], BadRegion[BLOBMINX]);
GoodRegion[BLOBMAXX] = Math.max(GoodRegion[BLOBMAXX], BadRegion[BLOBMAXX]);
GoodRegion[BLOBMINY] = Math.min(GoodRegion[BLOBMINY], BadRegion[BLOBMINY]);
GoodRegion[BLOBMAXY] = Math.max(GoodRegion[BLOBMAXY], BadRegion[BLOBMAXY]);
System.arraycopy(GoodRegion,0,RegionData[GoodLabel],0,BLOBDATACOUNT); // RegionData[GoodLabel] <- GoodRegion;
}
public static int SubsumptionChain(int x) { return SubsumptionChain(x, 0); }
public static int SubsumptionChain(int x, int Print)
{
String Str = "";
if(Print > 0) Str = "Subsumption chain for " + x + ": ";
int Lastx = x;
while(x > -1)
{
Lastx = x;
if(Print > 0) Str += " " + x;
if(x == 0) break;
x = SubsumedLabel[x];
}
if(Print > 0) System.out.println(Str);
return Lastx;
}
//---------------------------------------------------------------------------------------
// Main blob analysis routine
//---------------------------------------------------------------------------------------
// RegionData[0] is the border. It has Property[BLOBPARENT] = 0.
public int BlobAnalysis(IplImage Src, // input image
int Col0, int Row0, // start of ROI
int Cols, int Rows, // size of ROI
int Border, // border color (0 = black; 1 = white)
int MinArea) // minimum region area
{
CvMat SrcMat = Src.asCvMat();
int SrcCols = SrcMat.cols();
int SrcRows = SrcMat.rows();
if(Col0 < 0) Col0 = 0;
if(Row0 < 0) Row0 = 0;
if(Cols < 0) Cols = SrcCols;
if(Rows < 0) Rows = SrcRows;
if(Col0 + Cols > SrcCols) Cols = SrcCols - Col0;
if(Row0 + Rows > SrcRows) Rows = SrcRows - Row0;
if(Cols > BLOBCOLCOUNT || Rows > BLOBROWCOUNT )
{
System.out.println("Error in Class Blobs: Image too large: Edit Blobs.java");
System.exit(666);
return 0;
}
// Initialization
int FillLabel = 0;
int FillColor = 0; if(Border > 0) { FillColor = 1; }
LabelA = LabelB = LabelC = LabelD = 0;
ColorA = ColorB = ColorC = ColorD = FillColor;
for(int k = 0; k < BLOBTOTALCOUNT; k++) SubsumedLabel[k] = -1;
// Initialize border region
MaxLabel = 0;
double [] BorderRegion = RegionData[0];
BorderRegion[BLOBLABEL] = 0.0;
BorderRegion[BLOBPARENT] = -1.0;
BorderRegion[BLOBAREA] = Rows + Cols + 4; // Top, left, and 4 corners
BorderRegion[BLOBCOLOR] = FillColor;
BorderRegion[BLOBSUMX] = 0.5 * ( (2.0 + Cols) * (Cols - 1.0) ) - Rows - 1 ;
BorderRegion[BLOBSUMY] = 0.5 * ( (2.0 + Rows) * (Rows - 1.0) ) - Cols - 1 ;
BorderRegion[BLOBMINX] = -1;
BorderRegion[BLOBMINY] = -1;
BorderRegion[BLOBMAXX] = Cols + 1.0;
BorderRegion[BLOBMAXY] = Rows + 1.0;
System.arraycopy(BorderRegion,0,RegionData[0],0,BLOBDATACOUNT); // RegionData[0] <- BorderRegion;
// The cells are identified this way
// Last |AB|
// This |CD|
//
// With 4 connectivity, there are 8 possibilities for the cells:
// No color transition Color transition
// Case 1 2 3 4 5 6 7 8
// Last Row |pp|pp|pq|pq| |pp|pp|pq|pq|
// This Row |pP|qQ|pP|qQ| |pQ|qP|pQ|qP|
//
// Region numbers are p, q, r, x; where p<>q
// Upper case letter is the current element at column=x row=y
// Color is 0 or 1 (1 stands for 255 in the actual image)
// Note that Case 4 is complicated because it joins two regions
//--------------------------
// Case 1: Colors A=B; C=D; A=C
// Case 2: Colors A=B; C=D; A<>C
// Case 3: Colors A<>B;C=D; A=C
// Case 4: Colors A<>B;C=D; A<>C
// Case 5: Colors A=B; C<>D; A=C
// Case 6: Colors A=B; C<>D; A<>C
// Case 7: Colors A<>B;C<>D; A=C
// Case 8: Colors A<>B;C<>D; A<>C
//--------------------------
// Loop over rows of ROI. irow = Row0 is 1st row of image; irow = Row0+Row is last row of image.
for(int irow = Row0; irow < Row0+Rows; irow++) // index within Src
{
jrow = irow - Row0; // index within ROI. 0 is first row. Rows is last row.
// Loop over columns of ROI.
for(int icol = Col0; icol < Col0+Cols; icol++) // index within Src
{
jcol = icol - Col0; // index within ROI
// initialize
ColorA = ColorB = ColorC = FillColor;
LabelA = LabelB = LabelC = LabelD = 0;
ColorD = (int) SrcMat.get(jrow,jcol); // fetch color of cell
if(jrow == 0 || jcol == 0) // first column or row
{
if(jcol > 0)
{
ColorC = (int) SrcMat.get(jrow,jcol-1);
LabelC = LabelMat[jrow][jcol-1];
}
if(jrow > 0)
{
ColorB = (int) SrcMat.get(jrow-1,jcol);
LabelB = LabelMat[jrow-1][jcol];
}
}
else
{
ColorA = (int) SrcMat.get(jrow-1,jcol-1); if(ColorA > 0) ColorA = 1;
ColorB = (int) SrcMat.get(jrow-1,jcol); if(ColorB > 0) ColorB = 1;
ColorC = (int) SrcMat.get(jrow,jcol-1); if(ColorC > 0) ColorC = 1;
LabelA = LabelMat[jrow-1][jcol-1];
LabelB = LabelMat[jrow-1][jcol];
LabelC = LabelMat[jrow][jcol-1];
}
if(ColorA > 0) ColorA = 1;
if(ColorB > 0) ColorB = 1;
if(ColorC > 0) ColorC = 1;
if(ColorD > 0) ColorD = 1;
// Determine Case
int Case = 0;
if(ColorA == ColorB)
{
if(ColorC == ColorD) { if(ColorA == ColorC) Case = 1; else Case = 2; }
else { if(ColorA == ColorC) Case = 5; else Case = 6; }
}
else
{
if(ColorC == ColorD) { if(ColorA == ColorC) Case = 3; else Case = 4; }
else { if(ColorA == ColorC) Case = 7; else Case = 8; }
}
// Take appropriate action
if(Case == 1) { OldRegion(LabelC, -1, -1); }
else if(Case == 2 || Case == 3) { OldRegion(LabelC, LabelB, LabelC); }
else if(Case == 5 || Case == 8) // Isolated
{
if((jrow == Rows || jcol == Cols) && ColorD == FillColor) { OldRegion(0, -1, -1); } // attached to border region 0
else NewRegion(LabelB);
}
else if(Case == 6 || Case == 7) { OldRegion(LabelB, LabelB, LabelC); }
else // Case 4 - The complicated situation
{
int LabelBRoot = SubsumptionChain(LabelB);
int LabelCRoot = SubsumptionChain(LabelC);
int LabelRoot = Math.min(LabelBRoot, LabelCRoot);
int LabelX;
if(LabelBRoot < LabelCRoot) { OldRegion(LabelB, -1, -1); LabelX = LabelC; }
else { OldRegion(LabelC, -1, -1); LabelX = LabelB; }
int NextLabelX = LabelX;
while(LabelRoot < LabelX)
{
NextLabelX = SubsumedLabel[LabelX];
SubsumedLabel[LabelX] = LabelRoot;
LabelX = NextLabelX;
}
}
// Last column or row. Final corner was handled earlier in Cases 5 and 8.
if((jrow == Rows || jcol == Cols) && ColorD == FillColor)
{
if(jcol < Cols) // bottom row
{
if(ColorC != FillColor) // Subsume B chain to border region 0
{
int LabelRoot = SubsumptionChain(LabelB);
SubsumedLabel[LabelRoot] = 0;
}
}
else if(jrow < Rows) // right column
{
if(ColorB != FillColor) // Subsume C chain to border region 0
{
int LabelRoot = SubsumptionChain(LabelC);
SubsumedLabel[LabelRoot] = 0;
}
}
OldRegion(0, -1, -1); // attached to border region 0
}
LabelMat[jrow][jcol] = LabelD;
}
}
// Compute Condensation map
int Offset = 0;
for(int Label = 1; Label <= MaxLabel; Label++)
{
if(SubsumedLabel[Label] > -1) Offset++;
CondensationMap[Label] = Label - Offset;
}
// Subsume regions that were flagged as connected; Perimeters add
for(int Label = 1; Label <= MaxLabel; Label++)
{
int BetterLabel = SubsumptionChain(Label);
if(BetterLabel != Label) Subsume(BetterLabel, Label, 1);
}
// Condense subsumed regions
int NewMaxLabel = 0;
for(int OldLabel = 1; OldLabel <= MaxLabel; OldLabel++)
{
if(SubsumedLabel[OldLabel] < 0) // Renumber valid regions only
{
double [] OldRegion = RegionData[OldLabel];
int OldParent = (int) OldRegion[BLOBPARENT];
int NewLabel = CondensationMap[OldLabel];
int NewParent = SubsumptionChain(OldParent);
NewParent = CondensationMap[NewParent];
OldRegion[BLOBLABEL] = (double) NewLabel;
OldRegion[BLOBPARENT] = (double) NewParent;
System.arraycopy(OldRegion,0,RegionData[NewLabel],0,BLOBDATACOUNT); //RegionData[NewLabel] <- ThisRegion;
NewMaxLabel = NewLabel;
}
}
// Zero out unneeded high labels
for(int Label = NewMaxLabel+1; Label <= MaxLabel; Label++) ResetRegion(Label);
MaxLabel = NewMaxLabel;
// Flag for subsumption regions that have too small area
for(int Label = MaxLabel; Label > 0; Label--)
{
double [] ThisRegion = RegionData[Label];
int ThisArea = (int) ThisRegion[BLOBAREA];
if(ThisArea < MinArea)
{
int ThisParent = (int) ThisRegion[BLOBPARENT];
SubsumedLabel[Label] = ThisParent; // Flag this label as having been subsumed
}
else SubsumedLabel[Label] = -1;
}
// Compute Condensation map
Offset = 0;
for(int Label = 1; Label <= MaxLabel; Label++)
{
if(SubsumedLabel[Label] > -1) Offset++;
CondensationMap[Label] = Label - Offset;
}
// Subsume regions that were flagged as enclosed; Perimeters subtract
for(int Label = 1; Label <= MaxLabel; Label++)
{
int BetterLabel = SubsumptionChain(Label);
if(BetterLabel != Label) Subsume(BetterLabel, Label, -1);
}
// Condense subsumed regions
for(int OldLabel = 1; OldLabel <= MaxLabel; OldLabel++)
{
if(SubsumedLabel[OldLabel] < 0) // Renumber valid regions only
{
double [] OldRegion = RegionData[OldLabel];
int OldParent = (int) OldRegion[BLOBPARENT];
int NewLabel = CondensationMap[OldLabel];
int NewParent = SubsumptionChain(OldParent);
NewParent = CondensationMap[NewParent];
OldRegion[BLOBLABEL] = (double) NewLabel;
OldRegion[BLOBPARENT] = (double) NewParent;
System.arraycopy(OldRegion,0,RegionData[NewLabel],0,BLOBDATACOUNT); //RegionData[NewLabel] <- ThisRegion;
NewMaxLabel = NewLabel;
}
}
// Zero out unneeded high labels
for(int Label = NewMaxLabel+1; Label <= MaxLabel; Label++) ResetRegion(Label);
MaxLabel = NewMaxLabel;
// Normalize summation fields into moments
for(int Label = 0; Label <= MaxLabel; Label++)
{
double [] ThisRegion = RegionData[Label];
// Extract fields
double Area = ThisRegion[BLOBAREA];
double SumX = ThisRegion[BLOBSUMX];
double SumY = ThisRegion[BLOBSUMY];
double SumXX = ThisRegion[BLOBSUMXX];
double SumYY = ThisRegion[BLOBSUMYY];
double SumXY = ThisRegion[BLOBSUMXY];
// Get averages
SumX /= Area;
SumY /= Area;
SumXX /= Area;
SumYY /= Area;
SumXY /= Area;
// Create moments
SumXX -= SumX * SumX;
SumYY -= SumY * SumY;
SumXY -= SumX * SumY;
if(SumXY > -1.0E-14 && SumXY < 1.0E-14) SumXY = (float) 0.0; // Eliminate roundoff error
ThisRegion[BLOBSUMX] = SumX;
ThisRegion[BLOBSUMY] = SumY;
ThisRegion[BLOBSUMXX] = SumXX;
ThisRegion[BLOBSUMYY] = SumYY;
ThisRegion[BLOBSUMXY] = SumXY;
System.arraycopy(ThisRegion,0,RegionData[Label],0,BLOBDATACOUNT); // RegionData[Label] <- ThisRegion;
}
// Adjust border region
BorderRegion = RegionData[0];
BorderRegion[BLOBSUMXX] = BorderRegion[BLOBSUMYY] = BorderRegion[BLOBSUMXY] = 0; // Mark invalid fields
System.arraycopy(BorderRegion,0,RegionData[0],0,BLOBDATACOUNT); // RegionData[0] <- BorderRegion;
return MaxLabel;
}
// Sort RegionData array on any column. (I couldn't figure out how to use the built-in java sort.)
static double iField, jField;
static double [] iProperty, jProperty;
public static void SortRegions(int Col)
{
for(int i = 0; i < MaxLabel; i++)
{
for(int j = i+1; j <= Blobs.MaxLabel; j++)
{
iProperty = RegionData[i];
jProperty = RegionData[j];
iField = iProperty[Col];
jField = jProperty[Col];
if(iField > jField)
{
RegionData[i] = jProperty;
RegionData[j] = iProperty;
}
}
}
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/BufferRing.java
================================================
/*
* Copyright (C) 2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
/**
*
* @author Samuel Audet
*/
public class BufferRing {
public BufferRing(BufferFactory factory, int size) {
buffers = new Object[size];
for (int i = 0; i < size; i++) {
buffers[i] = factory.create();
}
position = 0;
}
public interface BufferFactory {
B create();
}
public interface ReleasableBuffer {
void release();
}
private Object[] buffers;
private int position;
public int capacity() {
return buffers.length;
}
public int position() {
return position;
}
public BufferRing position(int position) {
this.position = ((position % buffers.length) + buffers.length) % buffers.length;
return this;
}
@SuppressWarnings("unchecked")
public B get() {
return (B)buffers[position];
}
@SuppressWarnings("unchecked")
public B get(int offset) {
return (B)buffers[((position + offset) % buffers.length + buffers.length) % buffers.length];
}
@SuppressWarnings("unchecked")
public void release() {
for (int i = 0; i < buffers.length; i++) {
((B)buffers[i]).release();
}
buffers = null;
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/CameraDevice.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.beans.PropertyChangeEvent;
import java.beans.PropertyChangeListener;
import java.beans.PropertyVetoException;
import java.io.File;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import org.bytedeco.javacpp.Pointer;
import org.bytedeco.opencv.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_core.*;
/**
*
* @author Samuel Audet
*/
public class CameraDevice extends ProjectiveDevice {
public CameraDevice(String name) {
super(name);
}
public CameraDevice(String name, String filename) throws Exception {
super(name, filename);
settings.setImageWidth(imageWidth);
settings.setImageHeight(imageHeight);
}
public CameraDevice(String name, FileStorage fs) throws Exception {
super(name, fs);
settings.setImageWidth(imageWidth);
settings.setImageHeight(imageHeight);
}
public CameraDevice(Settings settings) throws Exception {
super((ProjectiveDevice.Settings)settings);
}
public interface Settings {
String getName();
void setName(String name);
double getResponseGamma();
void setResponseGamma(double gamma);
Integer getDeviceNumber();
void setDeviceNumber(Integer deviceNumber) throws PropertyVetoException;
File getDeviceFile();
void setDeviceFile(File deviceFile) throws PropertyVetoException;
String getDeviceFilename();
void setDeviceFilename(String deviceFilename) throws PropertyVetoException;
String getDevicePath();
void setDevicePath(String devicePath) throws PropertyVetoException;
Class getFrameGrabber();
void setFrameGrabber(Class frameGrabber);
String getDescription();
String getFormat();
void setFormat(String format);
int getImageWidth();
void setImageWidth(int imageWidth);
int getImageHeight();
void setImageHeight(int imageHeight);
double getFrameRate();
void setFrameRate(double frameRate);
boolean isTriggerMode();
void setTriggerMode(boolean triggerMode);
int getBitsPerPixel();
void setBitsPerPixel(int bitsPerPixel);
FrameGrabber.ImageMode getImageMode();
void setImageMode(FrameGrabber.ImageMode imageMode);
int getTimeout();
void setTimeout(int timeout);
int getNumBuffers();
void setNumBuffers(int numBuffers);
boolean isDeinterlace();
void setDeinterlace(boolean deinterlace);
void addPropertyChangeListener(PropertyChangeListener listener);
void removePropertyChangeListener(PropertyChangeListener listener);
}
public static class SettingsImplementation extends ProjectiveDevice.Settings implements Settings {
public SettingsImplementation() { name = "Camera 0"; }
public SettingsImplementation(ProjectiveDevice.Settings settings) {
super(settings);
if (settings instanceof SettingsImplementation) {
SettingsImplementation s = (SettingsImplementation)settings;
this.deviceNumber = s.deviceNumber;
this.deviceFile = s.deviceFile;
this.devicePath = s.devicePath;
this.frameGrabber = s.frameGrabber;
this.format = s.format;
this.imageWidth = s.imageWidth;
this.imageHeight = s.imageHeight;
this.frameRate = s.frameRate;
this.triggerMode = s.triggerMode;
this.bpp = s.bpp;
this.imageMode = s.imageMode;
this.timeout = s.timeout;
this.numBuffers = s.numBuffers;
this.deinterlace = s.deinterlace;
}
}
Integer deviceNumber = null;
File deviceFile = null;
String devicePath = null;
Class frameGrabber = null;
public Integer getDeviceNumber() {
return deviceNumber;
}
public void setDeviceNumber(Integer deviceNumber) throws PropertyVetoException {
if (deviceNumber != null) {
try {
if (frameGrabber != null) {
try {
frameGrabber.getConstructor(int.class);
} catch (NoSuchMethodException e) {
frameGrabber.getConstructor(Integer.class);
}
}
setDevicePath(null);
setDeviceFile(null);
} catch (NoSuchMethodException e) {
throw new PropertyVetoExceptionThatNetBeansLikes(frameGrabber.getSimpleName() + " does not accept a deviceNumber.",
new PropertyChangeEvent(this, "deviceNumber", this.deviceNumber, this.deviceNumber = null));
}
}
String oldDescription = getDescription();
firePropertyChange("deviceNumber", this.deviceNumber, this.deviceNumber = deviceNumber);
firePropertyChange("description", oldDescription, getDescription());
}
public File getDeviceFile() {
return deviceFile;
}
public void setDeviceFile(File deviceFile) throws PropertyVetoException {
if (deviceFile != null) {
try {
if (frameGrabber != null) {
frameGrabber.getConstructor(File.class);
}
setDeviceNumber(null);
setDevicePath(null);
} catch (NoSuchMethodException e) {
deviceFile = null;
throw new PropertyVetoExceptionThatNetBeansLikes(frameGrabber.getSimpleName() + " does not accept a deviceFile.",
new PropertyChangeEvent(this, "deviceFile", this.deviceFile, this.deviceFile = null));
}
}
String oldDescription = getDescription();
firePropertyChange("deviceFile", this.deviceFile, this.deviceFile = deviceFile);
firePropertyChange("description", oldDescription, getDescription());
}
public String getDeviceFilename() {
return getDeviceFile() == null ? "" : getDeviceFile().getPath();
}
public void setDeviceFilename(String deviceFilename) throws PropertyVetoException {
setDeviceFile(deviceFilename == null || deviceFilename.length() == 0 ?
null : new File(deviceFilename));
}
public String getDevicePath() {
return devicePath;
}
public void setDevicePath(String devicePath) throws PropertyVetoException {
if (devicePath != null) {
try {
if (frameGrabber != null) {
frameGrabber.getConstructor(String.class);
}
setDeviceNumber(null);
setDeviceFile(null);
} catch (NoSuchMethodException e) {
devicePath = "";
throw new PropertyVetoExceptionThatNetBeansLikes(frameGrabber.getSimpleName() + " does not accept a devicePath.",
new PropertyChangeEvent(this, "devicePath", this.devicePath, this.devicePath = null));
}
}
String oldDescription = getDescription();
firePropertyChange("devicePath", this.devicePath, this.devicePath = devicePath);
firePropertyChange("description", oldDescription, getDescription());
}
public Class getFrameGrabber() {
return frameGrabber;
}
public void setFrameGrabber(Class frameGrabber) {
String oldDescription = getDescription();
firePropertyChange("frameGrabber", this.frameGrabber, this.frameGrabber = frameGrabber);
firePropertyChange("description", oldDescription, getDescription());
if (frameGrabber == null) {
firePropertyChange("deviceNumber", this.deviceNumber, this.deviceNumber = null);
firePropertyChange("deviceFile", this.deviceFile, this.deviceFile = null);
firePropertyChange("devicePath", this.devicePath, this.devicePath = null);
return;
}
boolean hasDeviceNumber = false;
try {
frameGrabber.getConstructor(int.class);
hasDeviceNumber = true;
} catch (NoSuchMethodException e) {
try {
frameGrabber.getConstructor(Integer.class);
hasDeviceNumber = true;
} catch (NoSuchMethodException e2) {
firePropertyChange("deviceNumber", this.deviceNumber, this.deviceNumber = null);
}
}
try {
frameGrabber.getConstructor(File.class);
} catch (NoSuchMethodException e) {
firePropertyChange("deviceFile", this.deviceFile, this.deviceFile = null);
}
try {
frameGrabber.getConstructor(String.class);
} catch (NoSuchMethodException e) {
firePropertyChange("devicePath", this.devicePath, this.devicePath = null);
}
if (hasDeviceNumber && deviceNumber == null && deviceFile == null && devicePath == null) {
try {
setDeviceNumber(0);
} catch (PropertyVetoException e) { }
}
}
public String getDescription() {
String[] descriptions = null;
try {
Method m = frameGrabber.getMethod("getDeviceDescriptions");
descriptions = (String[])m.invoke(null);
} catch (java.lang.Exception ex) { }
if (descriptions != null && deviceNumber != null && deviceNumber < descriptions.length) {
return descriptions[deviceNumber];
} else {
return "";
}
}
String format = "";
int imageWidth = 0, imageHeight = 0;
double frameRate = 0;
boolean triggerMode = false;
int bpp = 0;
FrameGrabber.ImageMode imageMode = FrameGrabber.ImageMode.COLOR;
int timeout = 10000;
int numBuffers = 4;
boolean deinterlace = false;
public String getFormat() {
return format;
}
public void setFormat(String format) {
this.format = format;
}
public int getImageWidth() {
return imageWidth;
}
public void setImageWidth(int imageWidth) {
this.imageWidth = imageWidth;
}
public int getImageHeight() {
return imageHeight;
}
public void setImageHeight(int imageHeight) {
this.imageHeight = imageHeight;
}
public double getFrameRate() {
return frameRate;
}
public void setFrameRate(double frameRate) {
this.frameRate = frameRate;
}
public boolean isTriggerMode() {
return triggerMode;
}
public void setTriggerMode(boolean triggerMode) {
this.triggerMode = triggerMode;
}
public int getBitsPerPixel() {
return bpp;
}
public void setBitsPerPixel(int bitsPerPixel) {
this.bpp = bitsPerPixel;
}
public FrameGrabber.ImageMode getImageMode() {
return imageMode;
}
public void setImageMode(FrameGrabber.ImageMode imageMode) {
this.imageMode = imageMode;
}
public int getTimeout() {
return timeout;
}
public void setTimeout(int timeout) {
this.timeout = timeout;
}
public int getNumBuffers() {
return numBuffers;
}
public void setNumBuffers(int numBuffers) {
this.numBuffers = numBuffers;
}
public boolean isDeinterlace() {
return deinterlace;
}
public void setDeinterlace(boolean deinterlace) {
this.deinterlace = deinterlace;
}
}
// pouah.. hurray for Scala!
public static class CalibrationSettings extends ProjectiveDevice.CalibrationSettings implements Settings {
public CalibrationSettings() { }
public CalibrationSettings(ProjectiveDevice.CalibrationSettings settings) {
super(settings);
if (settings instanceof CalibrationSettings) {
si = new SettingsImplementation(((CalibrationSettings)settings).si);
}
}
SettingsImplementation si = new SettingsImplementation() {
@Override public void firePropertyChange(String propertyName, Object oldValue, Object newValue) {
CalibrationSettings.this.firePropertyChange(propertyName, oldValue, newValue);
}
};
@Override public String getName() { return si.getName(); }
@Override public void setName(String name) { si.setName(name); }
@Override public double getResponseGamma() { return si.getResponseGamma(); }
@Override public void setResponseGamma(double responseGamma) { si.setResponseGamma(responseGamma); }
// @Override public double getNominalDistance() { return si.getNominalDistance(); }
// @Override public void setNominalDistance(double nominalDistance) { si.setNominalDistance(nominalDistance); }
public Integer getDeviceNumber() { return si.getDeviceNumber(); }
public void setDeviceNumber(Integer deviceNumber) throws PropertyVetoException { si.setDeviceNumber(deviceNumber); }
public File getDeviceFile() { return si.getDeviceFile(); }
public void setDeviceFile(File deviceFile) throws PropertyVetoException { si.setDeviceFile(deviceFile); }
public String getDeviceFilename() { return si.getDeviceFilename(); }
public void setDeviceFilename(String deviceFilename) throws PropertyVetoException { si.setDeviceFilename(deviceFilename); }
public String getDevicePath() { return si.getDevicePath(); }
public void setDevicePath(String devicePath) throws PropertyVetoException { si.setDevicePath(devicePath); }
public Class getFrameGrabber() { return si.getFrameGrabber(); }
public void setFrameGrabber(Class frameGrabber) { si.setFrameGrabber(frameGrabber); }
public String getDescription() { return si.getDescription(); }
public String getFormat() { return si.getFormat(); }
public void setFormat(String format) { si.setFormat(format); }
public int getImageWidth() { return si.getImageWidth(); }
public void setImageWidth(int imageWidth) { si.setImageWidth(imageWidth); }
public int getImageHeight() { return si.getImageHeight(); }
public void setImageHeight(int imageHeight) { si.setImageHeight(imageHeight); }
public double getFrameRate() { return si.getFrameRate(); }
public void setFrameRate(double frameRate) { si.setFrameRate(frameRate); }
public boolean isTriggerMode() { return si.isTriggerMode(); }
public void setTriggerMode(boolean triggerMode) { si.setTriggerMode(triggerMode); }
public int getBitsPerPixel() { return si.getBitsPerPixel(); }
public void setBitsPerPixel(int bitsPerPixel) { si.setBitsPerPixel(bitsPerPixel); }
public FrameGrabber.ImageMode getImageMode() { return si.getImageMode(); }
public void setImageMode(FrameGrabber.ImageMode imageMode) { si.setImageMode(imageMode); }
public int getTimeout() { return si.getTimeout(); }
public void setTimeout(int timeout) { si.setTimeout(timeout); }
public int getNumBuffers() { return si.getNumBuffers(); }
public void setNumBuffers(int numBuffers) { si.setNumBuffers(numBuffers); }
public boolean isDeinterlace() { return si.isDeinterlace(); }
public void setDeinterlace(boolean deinterlace) { si.setDeinterlace(deinterlace); }
}
public static class CalibratedSettings extends ProjectiveDevice.CalibratedSettings implements Settings {
public CalibratedSettings() { }
public CalibratedSettings(ProjectiveDevice.CalibratedSettings settings) {
super(settings);
if (settings instanceof CalibratedSettings) {
si = new SettingsImplementation(((CalibratedSettings)settings).si);
}
}
SettingsImplementation si = new SettingsImplementation() {
@Override public void firePropertyChange(String propertyName, Object oldValue, Object newValue) {
CalibratedSettings.this.firePropertyChange(propertyName, oldValue, newValue);
}
};
@Override public String getName() { return si.getName(); }
@Override public void setName(String name) { si.setName(name); }
@Override public double getResponseGamma() { return si.getResponseGamma(); }
@Override public void setResponseGamma(double responseGamma) { si.setResponseGamma(responseGamma); }
// @Override public double getNominalDistance() { return si.getNominalDistance(); }
// @Override public void setNominalDistance(double nominalDistance) { si.setNominalDistance(nominalDistance); }
public Integer getDeviceNumber() { return si.getDeviceNumber(); }
public void setDeviceNumber(Integer deviceNumber) throws PropertyVetoException { si.setDeviceNumber(deviceNumber); }
public File getDeviceFile() { return si.getDeviceFile(); }
public void setDeviceFile(File deviceFile) throws PropertyVetoException { si.setDeviceFile(deviceFile); }
public String getDeviceFilename() { return si.getDeviceFilename(); }
public void setDeviceFilename(String deviceFilename) throws PropertyVetoException { si.setDeviceFilename(deviceFilename); }
public String getDevicePath() { return si.getDevicePath(); }
public void setDevicePath(String devicePath) throws PropertyVetoException { si.setDevicePath(devicePath); }
public Class getFrameGrabber() { return si.getFrameGrabber(); }
public void setFrameGrabber(Class frameGrabber) { si.setFrameGrabber(frameGrabber); }
public String getDescription() { return si.getDescription(); }
public String getFormat() { return si.getFormat(); }
public void setFormat(String format) { si.setFormat(format); }
public int getImageWidth() { return si.getImageWidth(); }
public void setImageWidth(int imageWidth) { si.setImageWidth(imageWidth); }
public int getImageHeight() { return si.getImageHeight(); }
public void setImageHeight(int imageHeight) { si.setImageHeight(imageHeight); }
public double getFrameRate() { return si.getFrameRate(); }
public void setFrameRate(double frameRate) { si.setFrameRate(frameRate); }
public boolean isTriggerMode() { return si.isTriggerMode(); }
public void setTriggerMode(boolean triggerMode) { si.setTriggerMode(triggerMode); }
public int getBitsPerPixel() { return si.getBitsPerPixel(); }
public void setBitsPerPixel(int bitsPerPixel) { si.setBitsPerPixel(bitsPerPixel); }
public FrameGrabber.ImageMode getImageMode() { return si.getImageMode(); }
public void setImageMode(FrameGrabber.ImageMode imageMode) { si.setImageMode(imageMode); }
public int getTimeout() { return si.getTimeout(); }
public void setTimeout(int timeout) { si.setTimeout(timeout); }
public int getNumBuffers() { return si.getNumBuffers(); }
public void setNumBuffers(int numBuffers) { si.setNumBuffers(numBuffers); }
public boolean isDeinterlace() { return si.isDeinterlace(); }
public void setDeinterlace(boolean deinterlace) { si.setDeinterlace(deinterlace); }
}
private Settings settings;
@Override public ProjectiveDevice.Settings getSettings() {
return (ProjectiveDevice.Settings)settings;
}
public void setSettings(Settings settings) {
setSettings((ProjectiveDevice.Settings)settings);
}
@Override public void setSettings(ProjectiveDevice.Settings settings) {
super.setSettings(settings);
if (settings instanceof ProjectiveDevice.CalibrationSettings) {
this.settings = new CalibrationSettings((ProjectiveDevice.CalibrationSettings)settings);
} else if (settings instanceof ProjectiveDevice.CalibratedSettings) {
this.settings = new CalibratedSettings((ProjectiveDevice.CalibratedSettings)settings);
} else {
this.settings = new SettingsImplementation((ProjectiveDevice.Settings)settings);
}
if (this.settings.getName() == null || this.settings.getName().length() == 0) {
this.settings.setName("Camera " + String.format("%2d", this.settings.getDeviceNumber()));
}
}
public FrameGrabber createFrameGrabber() throws FrameGrabber.Exception {
try {
settings.getFrameGrabber().getMethod("tryLoad").invoke(null);
FrameGrabber f;
if (settings.getDeviceFile() != null) {
f = settings.getFrameGrabber().getConstructor(File.class).newInstance(settings.getDeviceFile());
} else if (settings.getDevicePath() != null && settings.getDevicePath().length() > 0) {
f = settings.getFrameGrabber().getConstructor(String.class).newInstance(settings.getDevicePath());
} else {
int number = settings.getDeviceNumber() == null ? 0 : settings.getDeviceNumber();
try {
f = settings.getFrameGrabber().getConstructor(int.class).newInstance(number);
} catch (NoSuchMethodException e) {
f = settings.getFrameGrabber().getConstructor(Integer.class).newInstance(number);
}
}
f.setFormat(settings.getFormat());
f.setImageWidth(settings.getImageWidth());
f.setImageHeight(settings.getImageHeight());
f.setFrameRate(settings.getFrameRate());
f.setTriggerMode(settings.isTriggerMode());
f.setBitsPerPixel(settings.getBitsPerPixel());
f.setImageMode(settings.getImageMode());
f.setTimeout(settings.getTimeout());
f.setNumBuffers(settings.getNumBuffers());
f.setGamma(settings.getResponseGamma());
f.setDeinterlace(settings.isDeinterlace());
return f;
} catch (Throwable t) {
if (t instanceof InvocationTargetException) {
t = ((InvocationTargetException)t).getCause();
}
if (t instanceof FrameGrabber.Exception) {
throw (FrameGrabber.Exception)t;
} else {
throw new FrameGrabber.Exception("Failed to create " + settings.getFrameGrabber(), t);
}
}
}
public static CameraDevice[] read(String filename) throws Exception {
FileStorage fs = new FileStorage(filename, FileStorage.READ);
CameraDevice[] devices = read(fs);
fs.release();
return devices;
}
public static CameraDevice[] read(FileStorage fs) throws Exception {
FileNode node = fs.get("Cameras");
FileNodeIterator seq = node.begin();
int count = (int)seq.remaining();
CameraDevice[] devices = new CameraDevice[count];
for (int i = 0; i < count; i++, seq.increment()) {
FileNode n = seq.multiply();
if (n.empty()) continue;
String name = n.asBytePointer().getString();
devices[i] = new CameraDevice(name, fs);
}
return devices;
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/CameraSettings.java
================================================
/*
* Copyright (C) 2009-2011 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.beans.PropertyVetoException;
/**
*
* @author Samuel Audet
*/
public class CameraSettings extends BaseSettings {
public CameraSettings() {
this(false);
}
public CameraSettings(boolean calibrated) {
this.calibrated = calibrated;
}
boolean calibrated = false;
double monitorWindowsScale = 1.0;
Class frameGrabber = null;
public int getQuantity() {
return size();
}
public void setQuantity(int quantity) throws PropertyVetoException {
quantity = Math.max(1, quantity);
Object[] a = toArray();
int i = a.length;
while (i > quantity) {
remove(a[i-1]);
i--;
}
while (i < quantity) {
CameraDevice.Settings c = calibrated ? new CameraDevice.CalibratedSettings() :
new CameraDevice.CalibrationSettings();
c.setName("Camera " + String.format("%2d", i));
c.setDeviceNumber(i);
c.setFrameGrabber(frameGrabber);
add(c);
i++;
}
pcSupport.firePropertyChange("quantity", a.length, quantity);
}
public double getMonitorWindowsScale() {
return monitorWindowsScale;
}
public void setMonitorWindowsScale(double monitorWindowsScale) {
this.monitorWindowsScale = monitorWindowsScale;
}
public Class getFrameGrabber() {
return frameGrabber;
}
public void setFrameGrabber(Class frameGrabber) {
pcSupport.firePropertyChange("frameGrabber", this.frameGrabber, this.frameGrabber = frameGrabber);
}
@Override public CameraDevice.Settings[] toArray() {
return (CameraDevice.Settings[])toArray(new CameraDevice.Settings[size()]);
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/CanvasFrame.java
================================================
/*
* Copyright (C) 2009-2015 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.awt.Canvas;
import java.awt.Color;
import java.awt.Component;
import java.awt.Dimension;
import java.awt.DisplayMode;
import java.awt.EventQueue;
import java.awt.Graphics;
import java.awt.Graphics2D;
import java.awt.GraphicsConfiguration;
import java.awt.GraphicsDevice;
import java.awt.GraphicsEnvironment;
import java.awt.Image;
import java.awt.KeyEventDispatcher;
import java.awt.KeyboardFocusManager;
import java.awt.Transparency;
import java.awt.color.ColorSpace;
import java.awt.color.ICC_ColorSpace;
import java.awt.color.ICC_ProfileRGB;
import java.awt.event.ComponentAdapter;
import java.awt.event.ComponentEvent;
import java.awt.event.KeyEvent;
import java.awt.image.BufferStrategy;
import java.awt.image.BufferedImage;
import javax.swing.JFrame;
import javax.swing.JRootPane;
/**
*
* @author Samuel Audet
*
* Make sure OpenGL or XRender is enabled to get low latency, something like
* export _JAVA_OPTIONS=-Dsun.java2d.opengl=True
* export _JAVA_OPTIONS=-Dsun.java2d.xrender=True
*/
public class CanvasFrame extends JFrame {
public static class Exception extends java.lang.Exception {
public Exception(String message) { super(message); }
public Exception(String message, Throwable cause) { super(message, cause); }
}
public static String[] getScreenDescriptions() {
GraphicsDevice[] screens = getScreenDevices();
String[] descriptions = new String[screens.length];
for (int i = 0; i < screens.length; i++) {
descriptions[i] = screens[i].getIDstring();
}
return descriptions;
}
public static DisplayMode getDisplayMode(int screenNumber) {
GraphicsDevice[] screens = getScreenDevices();
if (screenNumber >= 0 && screenNumber < screens.length) {
return screens[screenNumber].getDisplayMode();
} else {
return null;
}
}
public static double getGamma(int screenNumber) {
GraphicsDevice[] screens = getScreenDevices();
if (screenNumber >= 0 && screenNumber < screens.length) {
return getGamma(screens[screenNumber]);
} else {
return 0.0;
}
}
public static double getDefaultGamma() {
return getGamma(getDefaultScreenDevice());
}
public static double getGamma(GraphicsDevice screen) {
ColorSpace cs = screen.getDefaultConfiguration().getColorModel().getColorSpace();
if (cs.isCS_sRGB()) {
return 2.2;
} else {
try {
return ((ICC_ProfileRGB)((ICC_ColorSpace)cs).getProfile()).getGamma(0);
} catch (RuntimeException e) { }
}
return 0.0;
}
public static GraphicsDevice getScreenDevice(int screenNumber) throws Exception {
GraphicsDevice[] screens = getScreenDevices();
if (screenNumber >= screens.length) {
throw new Exception("CanvasFrame Error: Screen number " + screenNumber + " not found. " +
"There are only " + screens.length + " screens.");
}
return screens[screenNumber];//.getDefaultConfiguration();
}
public static GraphicsDevice[] getScreenDevices() {
return GraphicsEnvironment.getLocalGraphicsEnvironment().getScreenDevices();
}
public static GraphicsDevice getDefaultScreenDevice() {
return GraphicsEnvironment.getLocalGraphicsEnvironment().getDefaultScreenDevice();
}
public CanvasFrame(String title) {
this(title, 0.0);
}
public CanvasFrame(String title, double gamma) {
super(title);
init(false, null, gamma);
}
public CanvasFrame(String title, GraphicsConfiguration gc) {
this(title, gc, 0.0);
}
public CanvasFrame(String title, GraphicsConfiguration gc, double gamma) {
super(title, gc);
init(false, null, gamma);
}
public CanvasFrame(String title, int screenNumber, DisplayMode displayMode) throws Exception {
this(title, screenNumber, displayMode, 0.0);
}
public CanvasFrame(String title, int screenNumber, DisplayMode displayMode, double gamma) throws Exception {
super(title, getScreenDevice(screenNumber).getDefaultConfiguration());
init(true, displayMode, gamma);
}
private void init(final boolean fullScreen, final DisplayMode displayMode, final double gamma) {
Runnable r = new Runnable() { public void run() {
KeyboardFocusManager.getCurrentKeyboardFocusManager().
addKeyEventDispatcher(keyEventDispatch);
GraphicsDevice gd = getGraphicsConfiguration().getDevice();
DisplayMode d = gd.getDisplayMode(), d2 = null;
if (displayMode != null && d != null) {
int w = displayMode.getWidth();
int h = displayMode.getHeight();
int b = displayMode.getBitDepth();
int r = displayMode.getRefreshRate();
d2 = new DisplayMode(w > 0 ? w : d.getWidth(), h > 0 ? h : d.getHeight(),
b > 0 ? b : d.getBitDepth(), r > 0 ? r : d.getRefreshRate());
}
if (fullScreen) {
setUndecorated(true);
getRootPane().setWindowDecorationStyle(JRootPane.NONE);
setResizable(false);
gd.setFullScreenWindow(CanvasFrame.this);
} else {
setLocationByPlatform(true);
}
if (d2 != null && !d2.equals(d)) {
gd.setDisplayMode(d2);
}
double g = gamma == 0.0 ? getGamma(gd) : gamma;
inverseGamma = g == 0.0 ? 1.0 : 1.0/g;
// Must be called after the fullscreen stuff, but before
// getting our BufferStrategy or even creating our Canvas
setVisible(true);
initCanvas(fullScreen, displayMode, gamma);
}};
if (EventQueue.isDispatchThread()) {
r.run();
} else {
try {
EventQueue.invokeAndWait(r);
} catch (java.lang.Exception ex) { }
}
}
protected void initCanvas(boolean fullScreen, DisplayMode displayMode, double gamma) {
canvas = new Canvas() {
@Override public void update(Graphics g) {
paint(g);
}
@Override public void paint(Graphics g) {
// Calling BufferStrategy.show() here sometimes throws
// NullPointerException or IllegalStateException,
// but otherwise seems to work fine.
try {
if (canvas.getWidth() <= 0 || canvas.getHeight() <= 0) {
return;
}
BufferStrategy strategy = canvas.getBufferStrategy();
do {
do {
g = strategy.getDrawGraphics();
if (color != null) {
g.setColor(color);
g.fillRect(0, 0, getWidth(), getHeight());
}
if (image != null) {
g.drawImage(image, 0, 0, getWidth(), getHeight(), null);
}
if (buffer != null) {
g.drawImage(buffer, 0, 0, getWidth(), getHeight(), null);
}
g.dispose();
} while (strategy.contentsRestored());
strategy.show();
} while (strategy.contentsLost());
} catch (NullPointerException e) {
} catch (IllegalStateException e) { }
}
};
if (fullScreen) {
canvas.setSize(getSize());
needInitialResize = false;
} else {
canvas.setSize(10,10); // mac bug
needInitialResize = true;
}
getContentPane().add(canvas);
canvas.setVisible(true);
canvas.createBufferStrategy(2);
//canvas.setIgnoreRepaint(true);
}
// used for example as debugging console...
public static CanvasFrame global = null;
// Latency is about 60 ms on Metacity and Windows XP, and 90 ms on Compiz Fusion,
// but we set the default to twice as much to take into account the roundtrip
// camera latency as well, just to be sure
public static final long DEFAULT_LATENCY = 200;
private long latency = DEFAULT_LATENCY;
private KeyEvent keyEvent = null;
private KeyEventDispatcher keyEventDispatch = new KeyEventDispatcher() {
public boolean dispatchKeyEvent(KeyEvent e) {
if (e.getID() == KeyEvent.KEY_PRESSED) {
synchronized (CanvasFrame.this) {
keyEvent = e;
CanvasFrame.this.notify();
}
}
return false;
}
};
protected Canvas canvas = null;
protected boolean needInitialResize = false;
protected double initialScale = 1.0;
protected double inverseGamma = 1.0;
private Color color = null;
private Image image = null;
private BufferedImage buffer = null;
private Java2DFrameConverter converter = new Java2DFrameConverter();
public long getLatency() {
// if there exists some way to estimate the latency in real time,
// add it here
return latency;
}
public void setLatency(long latency) {
this.latency = latency;
}
public void waitLatency() throws InterruptedException {
Thread.sleep(getLatency());
}
public KeyEvent waitKey() throws InterruptedException {
return waitKey(0);
}
public synchronized KeyEvent waitKey(int delay) throws InterruptedException {
if (delay >= 0) {
keyEvent = null;
wait(delay);
}
KeyEvent e = keyEvent;
keyEvent = null;
return e;
}
public Canvas getCanvas() {
return canvas;
}
public Dimension getCanvasSize() {
return canvas.getSize();
}
public void setCanvasSize(final int width, final int height) {
Dimension d = getCanvasSize();
if (d.width == width && d.height == height) {
return;
}
Runnable r = new Runnable() { public void run() {
// There is apparently a bug in Java code for Linux, and what happens goes like this:
// 1. Canvas gets resized, checks the visible area (has not changed) and updates
// BufferStrategy with the same size. 2. pack() resizes the frame and changes
// the visible area 3. We call Canvas.setSize() with different dimensions, to make
// it check the visible area and reallocate the BufferStrategy almost correctly
// 4. Finally, we resize the Canvas to the desired size... phew!
setExtendedState(NORMAL); // force unmaximization
canvas.setSize(width, height);
pack();
canvas.setSize(width+1, height+1);
canvas.setSize(width, height);
needInitialResize = false;
}};
if (EventQueue.isDispatchThread()) {
r.run();
} else {
try {
EventQueue.invokeAndWait(r);
} catch (java.lang.Exception ex) { }
}
}
public double getCanvasScale() {
return initialScale;
}
public void setCanvasScale(double initialScale) {
this.initialScale = initialScale;
this.needInitialResize = true;
}
public Graphics2D createGraphics() {
if (buffer == null || buffer.getWidth() != canvas.getWidth() || buffer.getHeight() != canvas.getHeight()) {
BufferedImage newbuffer = canvas.getGraphicsConfiguration().createCompatibleImage(
canvas.getWidth(), canvas.getHeight(), Transparency.TRANSLUCENT);
if (buffer != null) {
Graphics g = newbuffer.getGraphics();
g.drawImage(buffer, 0, 0, null);
g.dispose();
}
buffer = newbuffer;
}
return buffer.createGraphics();
}
public void releaseGraphics(Graphics2D g) {
g.dispose();
canvas.paint(null);
}
public void showColor(Color color) {
this.color = color;
this.image = null;
canvas.paint(null);
}
// Java2D will do gamma correction for TYPE_CUSTOM BufferedImage, but
// not for the standard types, so we need to do it manually.
public void showImage(Frame image) {
showImage(image, false);
}
public void showImage(Frame image, boolean flipChannels) {
showImage(converter.getBufferedImage(image, converter.getBufferedImageType(image) ==
BufferedImage.TYPE_CUSTOM ? 1.0 : inverseGamma, flipChannels, null));
}
public void showImage(Image image) {
if (image == null) {
return;
} else if (isResizable() && needInitialResize) {
int w = (int)Math.round(image.getWidth (null)*initialScale);
int h = (int)Math.round(image.getHeight(null)*initialScale);
setCanvasSize(w, h);
}
this.color = null;
this.image = image;
canvas.paint(null);
}
// This should not be called from the event dispatch thread (EDT),
// but if it is, it should not totally crash... In the worst case,
// it will simply timeout waiting for the moved events.
public static void tile(final CanvasFrame[] frames) {
class MovedListener extends ComponentAdapter {
boolean moved = false;
@Override public void componentMoved(ComponentEvent e) {
moved = true;
Component c = e.getComponent();
synchronized (c) {
c.notify();
}
}
}
final MovedListener movedListener = new MovedListener();
// layout the canvas frames for the cameras in tiles
int canvasCols = (int)Math.round(Math.sqrt(frames.length));
if (canvasCols*canvasCols < frames.length) {
// if we don't get a square, favor horizontal layouts
// since screens are usually wider than cameras...
// and we also have title bars, tasks bar, menus, etc that
// takes up vertical space
canvasCols++;
}
int canvasX = 0, canvasY = 0;
int canvasMaxY = 0;
for (int i = 0; i < frames.length; i++) {
final int n = i;
final int x = canvasX;
final int y = canvasY;
try {
movedListener.moved = false;
EventQueue.invokeLater(new Runnable() {
public void run() {
frames[n].addComponentListener(movedListener);
frames[n].setLocation(x, y);
}
});
int count = 0;
while (!movedListener.moved && count < 5) {
// wait until the window manager actually places our window...
// wait a maximum of 500 ms since this does not work if
// we are on the event dispatch thread. also some window
// managers like Windows do not always send us the event...
synchronized (frames[n]) {
frames[n].wait(100);
}
count++;
}
EventQueue.invokeLater(new Runnable() {
public void run() {
frames[n].removeComponentListener(movedListener);
}
});
} catch (java.lang.Exception ex) { }
canvasX = frames[i].getX()+frames[i].getWidth();
canvasMaxY = Math.max(canvasMaxY, frames[i].getY()+frames[i].getHeight());
if ((i+1)%canvasCols == 0) {
canvasX = 0;
canvasY = canvasMaxY;
}
}
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/ColorCalibrator.java
================================================
/*
* Copyright (C) 2009-2011 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.awt.Color;
import org.bytedeco.opencv.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_core.*;
/**
*
* @author Samuel Audet
*/
public class ColorCalibrator {
public ColorCalibrator(ProjectiveDevice device) {
this.device = device;
}
private ProjectiveDevice device;
public double calibrate(Color[] referenceColors, Color[] deviceColors) {
assert(referenceColors.length == deviceColors.length);
int[] order = device.getRGBColorOrder();
// solve for X and a in c = X p + a
CvMat A = CvMat.create(referenceColors.length*3, 12);
CvMat b = CvMat.create(referenceColors.length*3, 1);
CvMat x = CvMat.create(12, 1);
double gamma = device.getSettings().getResponseGamma();
for (int i = 0; i < referenceColors.length; i++) {
float[] dc = deviceColors [i].getRGBColorComponents(null);
float[] rc = referenceColors[i].getRGBColorComponents(null);
double dc1 = Math.pow(dc[order[0]], gamma);
double dc2 = Math.pow(dc[order[1]], gamma);
double dc3 = Math.pow(dc[order[2]], gamma);
for (int j = 0; j < 3; j++) {
int k = i*36 + j*16;
A.put(k , dc1);
A.put(k+1, dc2);
A.put(k+2, dc3);
A.put(k+3, 1.0);
if (j < 2) {
for (int m = 0; m < 12; m++) {
A.put(k+4+m, 0.0);
}
}
}
b.put(i*3 , rc[order[0]]);
b.put(i*3+1, rc[order[1]]);
b.put(i*3+2, rc[order[2]]);
}
//System.out.println("A =\n" + A);
//System.out.println("b =\n" + b);
// A.height = b.height = 18;
if (cvSolve(A, b, x, CV_SVD) != 1.0) {
System.out.println("Error solving.");
}
// compute RMSE and R^2 coefficient ...
CvMat b2 = CvMat.create(b.rows(), 1);
cvMatMul(A, x, b2);
double MSE = cvNorm(b, b2)*cvNorm(b, b2)/b.rows();
double RMSE = Math.sqrt(MSE);
CvScalar mean = new CvScalar(), stddev = new CvScalar();
cvAvgSdv(b, mean, stddev, null);
double R2 = 1 - MSE/(stddev.val(0)*stddev.val(0));
//System.out.println("RMSE: " + RMSE + " R2: " + R2);
//System.out.println("b2 =\n" + b2);
device.colorMixingMatrix = CvMat.create(3, 3);
device.additiveLight = CvMat.create(3, 1);
for (int i = 0; i < 3; i++) {
double x0 = x.get(i*4 );
double x1 = x.get(i*4+1);
double x2 = x.get(i*4+2);
double x3 = x.get(i*4+3);
device.colorMixingMatrix.put(i*3 , x0);
device.colorMixingMatrix.put(i*3+1, x1);
device.colorMixingMatrix.put(i*3+2, x2);
device.additiveLight .put(i, x3);
}
//System.out.println(device.colorMixingMatrix);
//System.out.println(device.additiveLight);
device.colorR2 = R2;
return device.avgColorErr = RMSE;
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/DC1394FrameGrabber.java
================================================
/*
* Copyright (C) 2009-2016 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.io.File;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.ShortBuffer;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.libdc1394.*;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.libdc1394.global.dc1394.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
*
* @author Samuel Audet
*/
public class DC1394FrameGrabber extends FrameGrabber {
public static String[] getDeviceDescriptions() throws Exception {
tryLoad();
dc1394_t d = dc1394_new();
if (d == null) {
throw new Exception("dc1394_new() Error: Failed to initialize libdc1394.");
}
dc1394camera_list_t list = new dc1394camera_list_t(null);
int err = dc1394_camera_enumerate(d, list);
if (err != DC1394_SUCCESS) {
throw new Exception("dc1394_camera_enumerate() Error " + err + ": Failed to enumerate cameras.");
}
int num = list.num();
String[] descriptions = new String[num];
if (num > 0) {
dc1394camera_id_t ids = list.ids();
for (int i = 0; i < num; i ++) {
ids.position(i);
dc1394camera_t camera = dc1394_camera_new_unit(d, ids.guid(), ids.unit());
if (camera == null) {
throw new Exception("dc1394_camera_new_unit() Error: Failed to initialize camera with GUID 0x" +
Long.toHexString(ids.guid())+ " / " + camera.unit() + ".");
}
descriptions[i] = camera.vendor().getString() + " " + camera.model().getString() + " 0x" +
Long.toHexString(camera.guid()) + " / " + camera.unit();
dc1394_camera_free(camera);
}
}
dc1394_camera_free_list(list);
dc1394_free(d);
return descriptions;
}
public static DC1394FrameGrabber createDefault(File deviceFile) throws Exception { throw new Exception(DC1394FrameGrabber.class + " does not support device files."); }
public static DC1394FrameGrabber createDefault(String devicePath) throws Exception { throw new Exception(DC1394FrameGrabber.class + " does not support device paths."); }
public static DC1394FrameGrabber createDefault(int deviceNumber) throws Exception { return new DC1394FrameGrabber(deviceNumber); }
private static Exception loadingException = null;
public static void tryLoad() throws Exception {
if (loadingException != null) {
throw loadingException;
} else {
try {
Loader.load(org.bytedeco.libdc1394.global.dc1394.class);
} catch (Throwable t) {
throw loadingException = new Exception("Failed to load " + DC1394FrameGrabber.class, t);
}
}
}
public DC1394FrameGrabber(int deviceNumber) throws Exception {
d = dc1394_new();
dc1394camera_list_t list = new dc1394camera_list_t(null);
int err = dc1394_camera_enumerate (d, list);
if (err != DC1394_SUCCESS) {
throw new Exception("dc1394_camera_enumerate() Error " + err + ": Failed to enumerate cameras.");
}
int num = list.num();
if (num <= deviceNumber) {
throw new Exception("DC1394Grabber() Error: Camera number " + deviceNumber +
" not found. There are only " + num + " devices.");
}
dc1394camera_id_t ids = list.ids().position(deviceNumber);
camera = dc1394_camera_new_unit(d, ids.guid(), ids.unit());
if (camera == null) {
throw new Exception("dc1394_camera_new_unit() Error: Failed to initialize camera with GUID 0x" +
Long.toHexString(ids.guid())+ " / " + camera.unit() + ".");
}
dc1394_camera_free_list(list);
//System.out.println("Using camera with GUID 0x" + Long.toHexString(camera.guid) + " / " + camera.unit);
}
public void release() throws Exception {
if (camera != null) {
stop();
dc1394_camera_free(camera);
camera = null;
}
if (d != null) {
dc1394_free(d);
d = null;
}
}
@Override protected void finalize() throws Throwable {
super.finalize();
release();
}
private static final boolean linux = Loader.getPlatform().startsWith("linux");
private dc1394_t d = null;
private dc1394camera_t camera = null;
private pollfd fds = linux ? new pollfd() : null;
private boolean oneShotMode = false;
private boolean resetDone = false;
private dc1394video_frame_t[] raw_image =
{ new dc1394video_frame_t(null), new dc1394video_frame_t(null) };
private dc1394video_frame_t conv_image = new dc1394video_frame_t();
private dc1394video_frame_t frame = null;
private dc1394video_frame_t enqueue_image = null;
private IplImage temp_image, return_image = null;
private FrameConverter converter = new OpenCVFrameConverter.ToIplImage();
private final int[] out = new int[1];
private final float[] outFloat = new float[1];
private final float[] gammaOut = new float[1];
@Override public double getGamma() {
return Float.isNaN(gammaOut[0]) || Float.isInfinite(gammaOut[0]) || gammaOut[0] == 0.0f ? 2.2 : gammaOut[0];
}
@Override public int getImageWidth() {
return return_image == null ? super.getImageWidth() : return_image.width();
}
@Override public int getImageHeight() {
return return_image == null ? super.getImageHeight() : return_image.height();
}
@Override public double getFrameRate() {
if (camera == null) {
return super.getFrameRate();
} else {
if (dc1394_feature_get_absolute_value(camera,
DC1394_FEATURE_FRAME_RATE, outFloat) != DC1394_SUCCESS) {
dc1394_video_get_framerate(camera, out);
dc1394_framerate_as_float(out[0], outFloat);
}
return outFloat[0];
}
}
@Override public void setImageMode(ImageMode imageMode) {
if (imageMode != this.imageMode) {
temp_image = null;
return_image = null;
}
super.setImageMode(imageMode);
}
public void start() throws Exception {
start(true, true);
}
public void start(boolean tryReset, boolean try1394b) throws Exception {
int c = -1;
if (imageMode == ImageMode.COLOR || imageMode == ImageMode.RAW) {
if (imageWidth <= 0 || imageHeight <= 0) {
c = -1;
} else if (imageWidth <= 640 && imageHeight <= 480) {
c = DC1394_VIDEO_MODE_640x480_RGB8;
} else if (imageWidth <= 800 && imageHeight <= 600) {
c = DC1394_VIDEO_MODE_800x600_RGB8;
} else if (imageWidth <= 1024 && imageHeight <= 768) {
c = DC1394_VIDEO_MODE_1024x768_RGB8;
} else if (imageWidth <= 1280 && imageHeight <= 960) {
c = DC1394_VIDEO_MODE_1280x960_RGB8;
} else if (imageWidth <= 1600 && imageHeight <= 1200) {
c = DC1394_VIDEO_MODE_1600x1200_RGB8;
}
} else if (imageMode == ImageMode.GRAY) {
if (imageWidth <= 0 || imageHeight <= 0) {
c = -1;
} else if (imageWidth <= 640 && imageHeight <= 480) {
c = bpp > 8 ? DC1394_VIDEO_MODE_640x480_MONO16 : DC1394_VIDEO_MODE_640x480_MONO8;
} else if (imageWidth <= 800 && imageHeight <= 600) {
c = bpp > 8 ? DC1394_VIDEO_MODE_800x600_MONO16 : DC1394_VIDEO_MODE_800x600_MONO8;
} else if (imageWidth <= 1024 && imageHeight <= 768) {
c = bpp > 8 ? DC1394_VIDEO_MODE_1024x768_MONO16 : DC1394_VIDEO_MODE_1024x768_MONO8;
} else if (imageWidth <= 1280 && imageHeight <= 960) {
c = bpp > 8 ? DC1394_VIDEO_MODE_1280x960_MONO16 : DC1394_VIDEO_MODE_1280x960_MONO8;
} else if (imageWidth <= 1600 && imageHeight <= 1200) {
c = bpp > 8 ? DC1394_VIDEO_MODE_1600x1200_MONO16 : DC1394_VIDEO_MODE_1600x1200_MONO8;
}
}
if (c == -1) {
// otherwise, still need to set current video mode to kick start the ISO channel...
dc1394_video_get_mode(camera, out);
c = out[0];
}
int f = -1;
if (frameRate <= 0) {
f = -1;
} else if (frameRate <= 1.876) {
f = DC1394_FRAMERATE_1_875;
} else if (frameRate <= 3.76) {
f = DC1394_FRAMERATE_3_75;
} else if (frameRate <= 7.51) {
f = DC1394_FRAMERATE_7_5;
} else if (frameRate <= 15.01) {
f = DC1394_FRAMERATE_15;
} else if (frameRate <= 30.01) {
f = DC1394_FRAMERATE_30;
} else if (frameRate <= 60.01) {
f = DC1394_FRAMERATE_60;
} else if (frameRate <= 120.01) {
f = DC1394_FRAMERATE_120;
} else if (frameRate <= 240.01) {
f = DC1394_FRAMERATE_240;
}
if (f == -1) {
// otherwise, still need to set current framerate to kick start the ISO channel...
dc1394_video_get_framerate(camera, out);
f = out[0];
}
try {
oneShotMode = false;
if (triggerMode) {
int err = dc1394_external_trigger_set_power(camera, DC1394_ON);
if (err != DC1394_SUCCESS) {
// no trigger support, use one-shot mode instead
oneShotMode = true;
} else {
err = dc1394_external_trigger_set_mode(camera, DC1394_TRIGGER_MODE_14);
if (err != DC1394_SUCCESS) {
// try with trigger mode 0 instead
err = dc1394_external_trigger_set_mode(camera, DC1394_TRIGGER_MODE_0);
}
err = dc1394_external_trigger_set_source(camera, DC1394_TRIGGER_SOURCE_SOFTWARE);
if (err != DC1394_SUCCESS) {
// no support for software trigger, use one-shot mode instead
oneShotMode = true;
dc1394_external_trigger_set_power(camera, DC1394_OFF);
}
}
}
int err = dc1394_video_set_operation_mode(camera, DC1394_OPERATION_MODE_LEGACY);
if (try1394b) {
err = dc1394_video_set_operation_mode(camera, DC1394_OPERATION_MODE_1394B);
if (err == DC1394_SUCCESS) {
err = dc1394_video_set_iso_speed(camera, DC1394_ISO_SPEED_800);
}
}
if (err != DC1394_SUCCESS || !try1394b) {
err = dc1394_video_set_iso_speed(camera, DC1394_ISO_SPEED_400);
if (err != DC1394_SUCCESS) {
throw new Exception("dc1394_video_set_iso_speed() Error " + err + ": Could not set maximum iso speed.");
}
}
err = dc1394_video_set_mode(camera, c);
if (err != DC1394_SUCCESS) {
throw new Exception("dc1394_video_set_mode() Error " + err + ": Could not set video mode.");
}
if (dc1394_is_video_mode_scalable(c) == DC1394_TRUE) {
err = dc1394_format7_set_roi(camera, c,
DC1394_QUERY_FROM_CAMERA, DC1394_QUERY_FROM_CAMERA,
DC1394_QUERY_FROM_CAMERA, DC1394_QUERY_FROM_CAMERA,
DC1394_QUERY_FROM_CAMERA, DC1394_QUERY_FROM_CAMERA);
if (err != DC1394_SUCCESS) {
throw new Exception("dc1394_format7_set_roi() Error " + err + ": Could not set format7 mode.");
}
} else {
err = dc1394_video_set_framerate(camera, f);
if (err != DC1394_SUCCESS) {
throw new Exception("dc1394_video_set_framerate() Error " + err + ": Could not set framerate.");
}
}
err = dc1394_capture_setup(camera, numBuffers, DC1394_CAPTURE_FLAGS_DEFAULT);
if (err != DC1394_SUCCESS) {
throw new Exception("dc1394_capture_setup() Error " + err + ": Could not setup camera-\n" +
"make sure that the video mode and framerate are\nsupported by your camera.");
}
if (gamma != 0.0) {
err = dc1394_feature_set_absolute_value(camera, DC1394_FEATURE_GAMMA, (float)gamma);
if (err != DC1394_SUCCESS) {
throw new Exception("dc1394_feature_set_absolute_value() Error " + err + ": Could not set gamma.");
}
}
err = dc1394_feature_get_absolute_value(camera, DC1394_FEATURE_GAMMA, gammaOut);
if (err != DC1394_SUCCESS) {
gammaOut[0] = 2.2f;
}
if (linux) {
fds.fd(dc1394_capture_get_fileno(camera));
}
if (!oneShotMode) {
err = dc1394_video_set_transmission(camera, DC1394_ON);
if (err != DC1394_SUCCESS) {
throw new Exception("dc1394_video_set_transmission() Error " + err + ": Could not start camera iso transmission.");
}
}
} catch (Exception e) {
// if we couldn't start, try again with a bus reset
if (tryReset && !resetDone) {
try {
dc1394_reset_bus(camera);
Thread.sleep(100);
resetDone = true;
start(false, try1394b);
} catch (InterruptedException ex) {
// reset interrupt to be nice
Thread.currentThread().interrupt();
throw new Exception("dc1394_reset_bus() Error: Could not reset bus and try to start again.", ex);
}
} else {
throw e;
}
} finally {
resetDone = false;
}
if (linux && try1394b) {
if (triggerMode) {
trigger();
}
fds.events(POLLIN);
if (poll(fds, 1, timeout) == 0) {
// we are obviously not getting anything..
// try again without 1394b
stop();
start(tryReset, false);
} else if (triggerMode) {
grab();
enqueue();
}
}
}
public void stop() throws Exception {
enqueue_image = null;
temp_image = null;
return_image = null;
timestamp = 0;
frameNumber = 0;
int err = dc1394_video_set_transmission(camera, DC1394_OFF);
if (err != DC1394_SUCCESS) {
throw new Exception("dc1394_video_set_transmission() Error " + err + ": Could not stop the camera?");
}
err = dc1394_capture_stop(camera);
if (err != DC1394_SUCCESS && err != DC1394_CAPTURE_IS_NOT_SET) {
throw new Exception("dc1394_capture_stop() Error " + err + ": Could not stop the camera?");
}
err = dc1394_external_trigger_get_mode(camera, out);
if (err == DC1394_SUCCESS && out[0] >= DC1394_TRIGGER_MODE_0) {
err = dc1394_external_trigger_set_power(camera, DC1394_OFF);
if (err != DC1394_SUCCESS) {
throw new Exception("dc1394_external_trigger_set_power() Error " + err + ": Could not switch off external trigger.");
}
}
}
private void enqueue() throws Exception {
enqueue(enqueue_image);
enqueue_image = null;
}
private void enqueue(dc1394video_frame_t image) throws Exception {
if (image != null) {
int err = dc1394_capture_enqueue(camera, image);
if (err != DC1394_SUCCESS) {
throw new Exception("dc1394_capture_enqueue() Error " + err + ": Could not release a frame.");
}
}
}
public void trigger() throws Exception {
enqueue();
if (oneShotMode) {
int err = dc1394_video_set_one_shot(camera, DC1394_ON);
if (err != DC1394_SUCCESS) {
throw new Exception("dc1394_video_set_one_shot() Error " + err + ": Could not set camera into one-shot mode.");
}
} else {
long time = System.currentTimeMillis();
do {
dc1394_software_trigger_get_power(camera, out);
if (System.currentTimeMillis() - time > timeout) {
break;
//throw new Exception("trigger() Error: Timeout occured.");
}
} while (out[0] == DC1394_ON);
int err = dc1394_software_trigger_set_power(camera, DC1394_ON);
if (err != DC1394_SUCCESS) {
throw new Exception("dc1394_software_trigger_set_power() Error " + err + ": Could not trigger camera.");
}
}
}
public Frame grab() throws Exception {
enqueue();
if (linux) {
fds.events(POLLIN);
if (poll(fds, 1, timeout) == 0) {
throw new Exception("poll() Error: Timeout occured. (Has start() been called?)");
}
}
int i = 0;
int err = dc1394_capture_dequeue(camera, DC1394_CAPTURE_POLICY_WAIT, raw_image[i]);
if (err != DC1394_SUCCESS) {
throw new Exception("dc1394_capture_dequeue(WAIT) Error " + err + ": Could not capture a frame. (Has start() been called?)");
}
// try to poll for more images, to get the most recent one...
int numDequeued = 0;
while (!raw_image[i].isNull()) {
enqueue();
enqueue_image = raw_image[i];
i = (i+1)%2;
numDequeued++;
err = dc1394_capture_dequeue(camera, DC1394_CAPTURE_POLICY_POLL, raw_image[i]);
if (err != DC1394_SUCCESS) {
throw new Exception("dc1394_capture_dequeue(POLL) Error " + err + ": Could not capture a frame.");
}
}
frame = raw_image[(i+1)%2];
int w = frame.size(0);
int h = frame.size(1);
int depth = frame.data_depth();
int iplDepth = 0;
switch (depth) {
case 8: iplDepth = IPL_DEPTH_8U; break;
case 16: iplDepth = IPL_DEPTH_16U; break;
default: assert false;
}
int stride = frame.stride();
int size = frame.image_bytes();
int numChannels = stride/w*8/depth;
ByteOrder frameEndian = frame.little_endian() != 0 ?
ByteOrder.LITTLE_ENDIAN : ByteOrder.BIG_ENDIAN;
boolean alreadySwapped = false;
int color_coding = frame.color_coding();
boolean colorbayer = color_coding == DC1394_COLOR_CODING_RAW8 ||
color_coding == DC1394_COLOR_CODING_RAW16;
boolean colorrgb = color_coding == DC1394_COLOR_CODING_RGB8 ||
color_coding == DC1394_COLOR_CODING_RGB16;
boolean coloryuv = color_coding == DC1394_COLOR_CODING_YUV411 ||
color_coding == DC1394_COLOR_CODING_YUV422 ||
color_coding == DC1394_COLOR_CODING_YUV444;
BytePointer imageData = frame.image();
if ((depth <= 8 || frameEndian.equals(ByteOrder.nativeOrder())) && !coloryuv &&
(imageMode == ImageMode.RAW || (imageMode == ImageMode.COLOR && numChannels == 3) ||
(imageMode == ImageMode.GRAY && numChannels == 1 && !colorbayer))) {
if (return_image == null) {
return_image = IplImage.createHeader(w, h, iplDepth, numChannels);
}
return_image.widthStep(stride);
return_image.imageSize(size);
return_image.imageData(imageData);
} else {
// in the padding, there's sometimes timeframe information and stuff
// that libdc1394 will copy for us, so we need to allocate it
int padding_bytes = frame.padding_bytes();
int padding1 = (int)Math.ceil((double)padding_bytes/(w * depth/8));
int padding3 = (int)Math.ceil((double)padding_bytes/(w*3*depth/8));
if (return_image == null) {
int c = imageMode == ImageMode.COLOR ? 3 : 1;
int padding = imageMode == ImageMode.COLOR ? padding3 : padding1;
return_image = IplImage.create(w, h+padding, iplDepth, c);
return_image.height(return_image.height() - padding);
}
if (temp_image == null) {
if (imageMode == ImageMode.COLOR &&
(numChannels > 1 || depth > 8) && !coloryuv && !colorbayer) {
temp_image = IplImage.create(w, h+padding1, iplDepth, numChannels);
temp_image.height(temp_image.height() - padding1);
} else if (imageMode == ImageMode.GRAY &&
(coloryuv || colorbayer || (colorrgb && depth > 8))) {
temp_image = IplImage.create(w, h+padding3, iplDepth, 3);
temp_image.height(temp_image.height() - padding3);
} else if (imageMode == ImageMode.GRAY && colorrgb) {
temp_image = IplImage.createHeader(w, h, iplDepth, 3);
} else if (imageMode == ImageMode.COLOR && numChannels == 1 && !coloryuv && !colorbayer) {
temp_image = IplImage.createHeader(w, h, iplDepth, 1);
} else {
temp_image = return_image;
}
}
conv_image.size(0, temp_image.width());
conv_image.size(1, temp_image.height());
if (depth > 8) {
conv_image.color_coding(imageMode == ImageMode.RAW ? DC1394_COLOR_CODING_RAW16 :
temp_image.nChannels() == 1 ? DC1394_COLOR_CODING_MONO16 :
DC1394_COLOR_CODING_RGB16);
conv_image.data_depth(16);
} else {
conv_image.color_coding(imageMode == ImageMode.RAW ? DC1394_COLOR_CODING_RAW8 :
temp_image.nChannels() == 1 ? DC1394_COLOR_CODING_MONO8 :
DC1394_COLOR_CODING_RGB8);
conv_image.data_depth(8);
}
conv_image.stride(temp_image.widthStep());
int temp_size = temp_image.imageSize();
conv_image.allocated_image_bytes(temp_size).
total_bytes(temp_size).image_bytes(temp_size);
conv_image.image(temp_image.imageData());
if (colorbayer) {
// from raw Bayer... invert R and B to get BGR images
// (like OpenCV wants them) instead of RGB
int c = frame.color_filter();
if (c == DC1394_COLOR_FILTER_RGGB) {
frame.color_filter(DC1394_COLOR_FILTER_BGGR);
} else if (c == DC1394_COLOR_FILTER_GBRG) {
frame.color_filter(DC1394_COLOR_FILTER_GRBG);
} else if (c == DC1394_COLOR_FILTER_GRBG) {
frame.color_filter(DC1394_COLOR_FILTER_GBRG);
} else if (c == DC1394_COLOR_FILTER_BGGR) {
frame.color_filter(DC1394_COLOR_FILTER_RGGB);
} else {
assert false;
}
// other better methods than "simple" give garbage at 16 bits..
err = dc1394_debayer_frames(frame, conv_image, DC1394_BAYER_METHOD_SIMPLE);
frame.color_filter(c);
if (err != DC1394_SUCCESS) {
throw new Exception("dc1394_debayer_frames() Error " + err + ": Could not debayer frame.");
}
} else if (depth > 8 &&
frame.data_depth() == conv_image.data_depth() &&
frame.color_coding() == conv_image.color_coding() &&
frame.stride() == conv_image.stride()) {
// we just need a copy to swap bytes..
ShortBuffer in = frame.getByteBuffer().order(frameEndian).asShortBuffer();
ShortBuffer out = temp_image.getByteBuffer().order(ByteOrder.nativeOrder()).asShortBuffer();
out.put(in);
alreadySwapped = true;
} else if ((imageMode == ImageMode.GRAY && colorrgb) ||
(imageMode == ImageMode.COLOR && numChannels == 1 && !coloryuv && !colorbayer)) {
temp_image.widthStep(stride);
temp_image.imageSize(size);
temp_image.imageData(imageData);
} else if (!colorrgb && (colorbayer || coloryuv || numChannels > 1)) {
// from YUV, etc.
err = dc1394_convert_frames(frame, conv_image);
if (err != DC1394_SUCCESS) {
throw new Exception("dc1394_convert_frames() Error " + err + ": Could not convert frame.");
}
}
if (!alreadySwapped && depth > 8 && !frameEndian.equals(ByteOrder.nativeOrder())) {
// ack, the camera's endianness doesn't correspond to our machine ...
// swap bytes of 16-bit images
ByteBuffer bb = temp_image.getByteBuffer();
ShortBuffer in = bb.order(frameEndian).asShortBuffer();
ShortBuffer out = bb.order(ByteOrder.nativeOrder()).asShortBuffer();
out.put(in);
}
// should we copy the padding as well?
if (imageMode == ImageMode.COLOR && numChannels == 1 && !coloryuv && !colorbayer) {
cvCvtColor(temp_image, return_image, CV_GRAY2BGR);
} else if (imageMode == ImageMode.GRAY && (colorbayer || colorrgb || coloryuv)) {
cvCvtColor(temp_image, return_image, CV_BGR2GRAY);
}
}
switch (frame.color_filter()) {
case DC1394_COLOR_FILTER_RGGB: sensorPattern = SENSOR_PATTERN_RGGB; break;
case DC1394_COLOR_FILTER_GBRG: sensorPattern = SENSOR_PATTERN_GBRG; break;
case DC1394_COLOR_FILTER_GRBG: sensorPattern = SENSOR_PATTERN_GRBG; break;
case DC1394_COLOR_FILTER_BGGR: sensorPattern = SENSOR_PATTERN_BGGR; break;
default: sensorPattern = -1L;
}
enqueue_image = frame;
timestamp = frame.timestamp();
frameNumber += numDequeued;
// int[] cycle_timer = { 0 };
// long[] local_time = { 0 };
// dc1394_read_cycle_timer(camera, cycle_timer, local_time);
//System.out.println("frame age = " + (local_time[0] - timestamp));
return converter.convert(return_image);
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/FFmpegFrameFilter.java
================================================
/*
* Copyright (C) 2015-2024 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*
* Based on the filtering_video.c file included in FFmpeg 2.7.1
* which is covered by the following copyright notice:
*
* Copyright (c) 2010 Nicolas George
* Copyright (c) 2011 Stefano Sabatini
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/
package org.bytedeco.javacv;
import java.nio.Buffer;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.DoubleBuffer;
import java.nio.FloatBuffer;
import java.nio.IntBuffer;
import java.nio.ShortBuffer;
import java.util.Locale;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.DoublePointer;
import org.bytedeco.javacpp.FloatPointer;
import org.bytedeco.javacpp.IntPointer;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.javacpp.Pointer;
import org.bytedeco.javacpp.PointerScope;
import org.bytedeco.javacpp.PointerPointer;
import org.bytedeco.javacpp.ShortPointer;
import org.bytedeco.ffmpeg.avcodec.*;
import org.bytedeco.ffmpeg.avfilter.*;
import org.bytedeco.ffmpeg.avformat.*;
import org.bytedeco.ffmpeg.avutil.*;
import static org.bytedeco.ffmpeg.global.avcodec.*;
import static org.bytedeco.ffmpeg.global.avfilter.*;
import static org.bytedeco.ffmpeg.global.avformat.*;
import static org.bytedeco.ffmpeg.global.avutil.*;
/**
* A {@link FrameFilter} that uses FFmpeg to filter frames. We can refer to
* FFmpeg Filters Documentation
* to get a list of filters and the options we can use. The input image width and
* height must be specified on the constructor, while other optional values may be
* set via corresponding properties.
*
* @author Samuel Audet
*/
public class FFmpegFrameFilter extends FrameFilter {
public static class Exception extends FrameFilter.Exception {
public Exception(String message) { super(message + " (For more details, make sure FFmpegLogCallback.set() has been called.)"); }
public Exception(String message, Throwable cause) { super(message, cause); }
}
private static Exception loadingException = null;
public static void tryLoad() throws Exception {
if (loadingException != null) {
throw loadingException;
} else {
try {
Loader.load(org.bytedeco.ffmpeg.global.avutil.class);
Loader.load(org.bytedeco.ffmpeg.global.avcodec.class);
Loader.load(org.bytedeco.ffmpeg.global.avformat.class);
// Loader.load(org.bytedeco.ffmpeg.global.postproc.class);
Loader.load(org.bytedeco.ffmpeg.global.swresample.class);
Loader.load(org.bytedeco.ffmpeg.global.swscale.class);
Loader.load(org.bytedeco.ffmpeg.global.avfilter.class);
// av_register_all();
// avfilter_register_all();
} catch (Throwable t) {
if (t instanceof Exception) {
throw loadingException = (Exception)t;
} else {
throw loadingException = new Exception("Failed to load " + FFmpegFrameRecorder.class, t);
}
}
}
}
static {
try {
tryLoad();
} catch (Exception ex) { }
}
public FFmpegFrameFilter(String videoFilters, String audioFilters, int imageWidth, int imageHeight, int audioChannels) {
this.filters = videoFilters;
this.imageWidth = imageWidth;
this.imageHeight = imageHeight;
this.pixelFormat = AV_PIX_FMT_BGR24;
this.frameRate = 30;
this.aspectRatio = 0;
this.videoInputs = 1;
this.afilters = audioFilters;
this.audioChannels = audioChannels;
this.sampleFormat = AV_SAMPLE_FMT_S16;
this.sampleRate = 44100;
this.audioInputs = 1;
}
public FFmpegFrameFilter(String filters, int imageWidth, int imageHeight) {
this(filters, null, imageWidth, imageHeight, 0);
}
public FFmpegFrameFilter(String afilters, int audioChannels) {
this(null, afilters, 0, 0, audioChannels);
}
@Override public void release() throws Exception {
synchronized (org.bytedeco.ffmpeg.global.avfilter.class) {
releaseUnsafe();
}
}
public synchronized void releaseUnsafe() throws Exception {
started = false;
if (default_layout != null) {
default_layout.releaseReference();
default_layout = null;
}
if (image_ptr2 != null) {
for (int i = 0; i < image_ptr2.length; i++) {
av_free(image_ptr2[i]);
}
image_ptr2 = null;
}
if (filter_graph != null) {
avfilter_graph_free(filter_graph);
buffersink_ctx.releaseReference();
for (int i = 0; i < buffersrc_ctx.length; i++) {
buffersrc_ctx[i].releaseReference();
setpts_ctx[i].releaseReference();
}
time_base.releaseReference();
buffersink_ctx = null;
buffersrc_ctx = null;
setpts_ctx = null;
filter_graph = null;
time_base = null;
}
if (afilter_graph != null) {
avfilter_graph_free(afilter_graph);
abuffersink_ctx.releaseReference();
for (int i = 0; i < abuffersrc_ctx.length; i++) {
abuffersrc_ctx[i].releaseReference();
asetpts_ctx[i].releaseReference();
}
atime_base.releaseReference();
abuffersink_ctx = null;
abuffersrc_ctx = null;
asetpts_ctx = null;
afilter_graph = null;
atime_base = null;
}
if (image_frame != null) {
av_frame_free(image_frame);
image_frame = null;
}
if (samples_frame != null) {
av_frame_free(samples_frame);
samples_frame = null;
}
if (filt_frame != null) {
av_frame_free(filt_frame);
filt_frame = null;
}
frame = null;
Pointer.trimMemory();
}
@Override protected void finalize() throws Throwable {
super.finalize();
release();
}
AVFilterContext buffersink_ctx;
AVFilterContext[] buffersrc_ctx;
AVFilterContext[] setpts_ctx;
AVFilterGraph filter_graph;
AVRational time_base;
AVFilterContext abuffersink_ctx;
AVFilterContext[] abuffersrc_ctx;
AVFilterContext[] asetpts_ctx;
AVFilterGraph afilter_graph;
AVRational atime_base;
AVFrame image_frame;
AVFrame samples_frame;
AVFrame filt_frame;
BytePointer[] image_ptr, image_ptr2;
BytePointer[] samples_ptr;
Buffer[] image_buf, image_buf2;
Buffer[] samples_buf;
Frame frame, inframe;
AVChannelLayout default_layout;
private volatile boolean started = false;
@Override public int getImageWidth() {
return buffersink_ctx != null ? av_buffersink_get_w(buffersink_ctx) : super.getImageWidth();
}
@Override public int getImageHeight() {
return buffersink_ctx != null ? av_buffersink_get_h(buffersink_ctx) : super.getImageHeight();
}
@Override public int getPixelFormat() {
return buffersink_ctx != null ? av_buffersink_get_format(buffersink_ctx) : super.getPixelFormat();
}
@Override public double getFrameRate() {
if (buffersink_ctx != null) {
AVRational r = av_buffersink_get_frame_rate(buffersink_ctx);
if (r.num() == 0 && r.den() == 0) {
r = av_buffersink_get_time_base(buffersink_ctx);
return (double)r.den() / r.num();
}
return (double)r.num() / r.den();
} else {
return super.getFrameRate();
}
}
@Override public double getAspectRatio() {
if (buffersink_ctx != null) {
AVRational r = av_buffersink_get_sample_aspect_ratio(buffersink_ctx);
double a = (double)r.num() / r.den();
return a == 0.0 ? 1.0 : a;
} else {
return super.getAspectRatio();
}
}
@Override public int getAudioChannels() {
return abuffersink_ctx != null ? av_buffersink_get_channels(abuffersink_ctx) : super.getAudioChannels();
}
@Override public int getSampleFormat() {
return abuffersink_ctx != null ? av_buffersink_get_format(abuffersink_ctx) : super.getSampleFormat();
}
@Override public int getSampleRate() {
return abuffersink_ctx != null ? av_buffersink_get_sample_rate(abuffersink_ctx) : super.getSampleRate();
}
@Override public void start() throws Exception {
synchronized (org.bytedeco.ffmpeg.global.avfilter.class) {
startUnsafe();
}
}
public synchronized void startUnsafe() throws Exception {
try (PointerScope scope = new PointerScope()) {
if (frame != null) {
throw new Exception("start() has already been called: Call stop() before calling start() again.");
}
image_frame = av_frame_alloc();
samples_frame = av_frame_alloc();
filt_frame = av_frame_alloc();
image_ptr = new BytePointer[] { null };
image_ptr2 = new BytePointer[] { null };
image_buf = new Buffer[] { null };
image_buf2 = new Buffer[] { null };
samples_ptr = new BytePointer[] { null };
samples_buf = new Buffer[] { null };
frame = new Frame();
default_layout = new AVChannelLayout().retainReference();
if (videoFilterArgs != null && videoInputs != videoFilterArgs.length) {
throw new Exception("The length of videoFilterArgs is different from videoInputs");
}
if (audioFilterArgs != null && audioInputs != audioFilterArgs.length) {
throw new Exception("The length of audioFilterArgs is different from audioInputs");
}
if (image_frame == null || samples_frame == null || filt_frame == null) {
throw new Exception("Could not allocate frames");
}
if (filters != null && imageWidth > 0 && imageHeight > 0 && videoInputs > 0) {
startVideoUnsafe();
}
if (afilters != null && audioChannels > 0 && audioInputs > 0) {
startAudioUnsafe();
}
started = true;
}
}
private void startVideoUnsafe() throws Exception {
int ret;
AVFilter buffersrc = avfilter_get_by_name("buffer");
AVFilter buffersink = avfilter_get_by_name("buffersink");
AVFilter setpts = avfilter_get_by_name("setpts");
AVFilterInOut[] outputs = new AVFilterInOut[videoInputs];
AVFilterInOut inputs = avfilter_inout_alloc();
AVRational frame_rate = av_d2q(frameRate, 1001000);
AVRational time_base = av_inv_q(frame_rate);
int pix_fmts[] = { pixelFormat, AV_PIX_FMT_NONE };
try {
filter_graph = avfilter_graph_alloc();
if (outputs == null || inputs == null || filter_graph == null) {
throw new Exception("Could not allocate video filter graph: Out of memory?");
}
/* buffer video source: the decoded frames from the decoder will be inserted here. */
AVRational r = av_d2q(aspectRatio > 0 ? aspectRatio : 1, 255);
buffersrc_ctx = new AVFilterContext[videoInputs];
setpts_ctx = new AVFilterContext[videoInputs];
for (int i = 0; i < videoInputs; i++) {
String name = videoInputs > 1 ? i + ":v" : "in";
outputs[i] = avfilter_inout_alloc();
String args = videoFilterArgs != null && videoFilterArgs[i] != null ? videoFilterArgs[i]
: String.format(Locale.ROOT, "video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:pixel_aspect=%d/%d:frame_rate=%d/%d",
imageWidth, imageHeight, pixelFormat, time_base.num(), time_base.den(), r.num(), r.den(), frame_rate.num(), frame_rate.den());
ret = avfilter_graph_create_filter(buffersrc_ctx[i] = new AVFilterContext().retainReference(), buffersrc, name,
args, null, filter_graph);
if (ret < 0) {
throw new Exception("avfilter_graph_create_filter() error " + ret + ": Cannot create video buffer source.");
}
ret = avfilter_graph_create_filter(setpts_ctx[i] = new AVFilterContext().retainReference(), setpts, videoInputs > 1 ? "setpts" + i : "setpts",
"N", null, filter_graph);
if (ret < 0) {
throw new Exception("avfilter_graph_create_filter() error " + ret + ": Cannot create setpts filter.");
}
ret = avfilter_link(buffersrc_ctx[i], 0, setpts_ctx[i], 0);
if (ret < 0) {
throw new Exception("avfilter_graph_create_filter() error " + ret + ": Cannot link setpts filter.");
}
/*
* Set the endpoints for the filter graph. The filter_graph will
* be linked to the graph described by filters_descr.
*/
/*
* The buffer source output must be connected to the input pad of
* the first filter described by filters_descr; since the first
* filter input label is not specified, it is set to "in" by
* default.
*/
outputs[i].name(av_strdup(new BytePointer(name)));
outputs[i].filter_ctx(setpts_ctx[i]);
outputs[i].pad_idx(0);
outputs[i].next(null);
if (i > 0) {
outputs[i - 1].next(outputs[i]);
}
}
String name = videoInputs > 1 ? "v" : "out";
/* buffer video sink: to terminate the filter chain. */
ret = avfilter_graph_create_filter(buffersink_ctx = new AVFilterContext().retainReference(), buffersink, name,
null, null, filter_graph);
if (ret < 0) {
throw new Exception("avfilter_graph_create_filter() error " + ret + ": Cannot create video buffer sink.");
}
// ret = av_opt_set_bin(buffersink_ctx, "pix_fmts", new BytePointer(new IntPointer(pix_fmts)), 4, AV_OPT_SEARCH_CHILDREN);
// if (ret < 0) {
// throw new Exception("av_opt_set_bin() error " + ret + ": Cannot set output pixel format.");
// }
/*
* The buffer sink input must be connected to the output pad of
* the last filter described by filters_descr; since the last
* filter output label is not specified, it is set to "out" by
* default.
*/
inputs.name(av_strdup(new BytePointer(name)));
inputs.filter_ctx(buffersink_ctx);
inputs.pad_idx(0);
inputs.next(null);
if ((ret = avfilter_graph_parse_ptr(filter_graph, filters,
inputs, outputs[0], null)) < 0) {
throw new Exception("avfilter_graph_parse_ptr() error " + ret);
}
if ((ret = avfilter_graph_config(filter_graph, null)) < 0) {
throw new Exception("avfilter_graph_config() error " + ret);
}
this.time_base = av_buffersink_get_time_base(buffersink_ctx).retainReference();
} finally {
avfilter_inout_free(inputs);
avfilter_inout_free(outputs[0]);
}
}
private void startAudioUnsafe() throws Exception {
int ret;
AVFilter abuffersrc = avfilter_get_by_name("abuffer");
AVFilter abuffersink = avfilter_get_by_name("abuffersink");
AVFilter asetpts = avfilter_get_by_name("asetpts");
AVFilterInOut[] aoutputs = new AVFilterInOut[audioInputs];
AVFilterInOut ainputs = avfilter_inout_alloc();
int sample_fmts[] = { sampleFormat, AV_PIX_FMT_NONE };
try {
afilter_graph = avfilter_graph_alloc();
if (aoutputs == null || ainputs == null || afilter_graph == null) {
throw new Exception("Could not allocate audio filter graph: Out of memory?");
}
abuffersrc_ctx = new AVFilterContext[audioInputs];
asetpts_ctx = new AVFilterContext[audioInputs];
for (int i = 0; i < audioInputs; i++) {
String name = audioInputs > 1 ? i + ":a" : "in";
aoutputs[i] = avfilter_inout_alloc();
/* buffer audio source: the decoded frames from the decoder will be inserted here. */
av_channel_layout_default(default_layout, audioChannels);
String aargs = audioFilterArgs != null && audioFilterArgs[i] != null ? audioFilterArgs[i]
: String.format(Locale.ROOT, "channels=%d:sample_fmt=%d:sample_rate=%d:channel_layout=%d",
audioChannels, sampleFormat, sampleRate, default_layout.u_mask());
ret = avfilter_graph_create_filter(abuffersrc_ctx[i] = new AVFilterContext().retainReference(), abuffersrc, name,
aargs, null, afilter_graph);
if (ret < 0) {
throw new Exception("avfilter_graph_create_filter() error " + ret + ": Cannot create audio buffer source.");
}
ret = avfilter_graph_create_filter(asetpts_ctx[i] = new AVFilterContext().retainReference(), asetpts, audioInputs > 1 ? "asetpts" + i : "asetpts",
"N", null, afilter_graph);
if (ret < 0) {
throw new Exception("avfilter_graph_create_filter() error " + ret + ": Cannot create asetpts filter.");
}
ret = avfilter_link(abuffersrc_ctx[i], 0, asetpts_ctx[i], 0);
if (ret < 0) {
throw new Exception("avfilter_graph_create_filter() error " + ret + ": Cannot link asetpts filter.");
}
/*
* Set the endpoints for the filter graph. The filter_graph will
* be linked to the graph described by filters_descr.
*/
/*
* The buffer source output must be connected to the input pad of
* the first filter described by filters_descr; since the first
* filter input label is not specified, it is set to "in" by
* default.
*/
aoutputs[i].name(av_strdup(new BytePointer(name)));
aoutputs[i].filter_ctx(asetpts_ctx[i]);
aoutputs[i].pad_idx(0);
aoutputs[i].next(null);
if (i > 0) {
aoutputs[i - 1].next(aoutputs[i]);
}
}
String name = audioInputs > 1 ? "a" : "out";
/* buffer audio sink: to terminate the filter chain. */
ret = avfilter_graph_create_filter(abuffersink_ctx = new AVFilterContext().retainReference(), abuffersink, name,
null, null, afilter_graph);
if (ret < 0) {
throw new Exception("avfilter_graph_create_filter() error " + ret + ": Cannot create audio buffer sink.");
}
// ret = av_opt_set_bin(abuffersink_ctx, "sample_fmts", new BytePointer(new IntPointer(sample_fmts)), 4, AV_OPT_SEARCH_CHILDREN);
// if (ret < 0) {
// throw new Exception("av_opt_set_bin() error " + ret + ": Cannot set output sample format.");
// }
/*
* The buffer sink input must be connected to the output pad of
* the last filter described by filters_descr; since the last
* filter output label is not specified, it is set to "out" by
* default.
*/
ainputs.name(av_strdup(new BytePointer(name)));
ainputs.filter_ctx(abuffersink_ctx);
ainputs.pad_idx(0);
ainputs.next(null);
if ((ret = avfilter_graph_parse_ptr(afilter_graph, afilters,
ainputs, aoutputs[0], null)) < 0) {
throw new Exception("avfilter_graph_parse_ptr() error " + ret);
}
if ((ret = avfilter_graph_config(afilter_graph, null)) < 0) {
throw new Exception("avfilter_graph_config() error " + ret);
}
this.atime_base = av_buffersink_get_time_base(abuffersink_ctx).retainReference();
} finally {
avfilter_inout_free(ainputs);
avfilter_inout_free(aoutputs[0]);
}
}
@Override public void stop() throws Exception {
release();
}
@Override public void push(Frame frame) throws Exception {
push(frame, frame != null && frame.opaque instanceof AVFrame ? ((AVFrame)frame.opaque).format() : AV_PIX_FMT_NONE);
}
public void push(Frame frame, int pixelFormat) throws Exception {
push(0, frame, pixelFormat);
}
public void push(int n, Frame frame) throws Exception {
push(n, frame, frame != null && frame.opaque instanceof AVFrame ? ((AVFrame)frame.opaque).format() : AV_PIX_FMT_NONE);
}
public synchronized void push(int n, Frame frame, int pixelFormat) throws Exception {
if (!started) {
throw new Exception("start() was not called successfully!");
}
inframe = frame;
if (frame != null && frame.image != null && buffersrc_ctx != null) {
image_frame.pts(frame.timestamp * time_base.den() / (1000000L * time_base.num()));
pushImage(n, frame.imageWidth, frame.imageHeight, frame.imageDepth,
frame.imageChannels, frame.imageStride, pixelFormat, frame.image);
}
if (frame != null && frame.samples != null && abuffersrc_ctx != null) {
samples_frame.pts(frame.timestamp * atime_base.den() / (1000000L * atime_base.num()));
pushSamples(n, frame.audioChannels, sampleRate, sampleFormat, frame.samples);
}
if (frame == null || (frame.image == null && frame.samples == null)) {
// indicate EOF as required, for example, by the "palettegen" filter
if (buffersrc_ctx != null && n < buffersrc_ctx.length) {
av_buffersrc_add_frame_flags(buffersrc_ctx[n], null, AV_BUFFERSRC_FLAG_PUSH);
}
if (abuffersrc_ctx != null && n < abuffersrc_ctx.length) {
av_buffersrc_add_frame_flags(abuffersrc_ctx[n], null, AV_BUFFERSRC_FLAG_PUSH);
}
}
}
public synchronized void pushImage(int n, int width, int height, int depth, int channels, int stride, int pixelFormat, Buffer ... image) throws Exception {
try (PointerScope scope = new PointerScope()) {
if (!started) {
throw new Exception("start() was not called successfully!");
}
int ret;
int step = stride * Math.abs(depth) / 8;
BytePointer data = image[0] instanceof ByteBuffer
? new BytePointer((ByteBuffer)image[0]).position(0)
: new BytePointer(new Pointer(image[0]).position(0));
if (pixelFormat == AV_PIX_FMT_NONE) {
if ((depth == Frame.DEPTH_UBYTE || depth == Frame.DEPTH_BYTE) && channels == 3) {
pixelFormat = AV_PIX_FMT_BGR24;
} else if ((depth == Frame.DEPTH_UBYTE || depth == Frame.DEPTH_BYTE) && channels == 1) {
pixelFormat = AV_PIX_FMT_GRAY8;
} else if ((depth == Frame.DEPTH_USHORT || depth == Frame.DEPTH_SHORT) && channels == 1) {
pixelFormat = ByteOrder.nativeOrder().equals(ByteOrder.BIG_ENDIAN) ?
AV_PIX_FMT_GRAY16BE : AV_PIX_FMT_GRAY16LE;
} else if ((depth == Frame.DEPTH_UBYTE || depth == Frame.DEPTH_BYTE) && channels == 4) {
pixelFormat = AV_PIX_FMT_RGBA;
} else if ((depth == Frame.DEPTH_UBYTE || depth == Frame.DEPTH_BYTE) && channels == 2) {
pixelFormat = AV_PIX_FMT_NV21; // Android's camera capture format
} else {
throw new Exception("Could not guess pixel format of image: depth=" + depth + ", channels=" + channels);
}
}
if (pixelFormat == AV_PIX_FMT_NV21) {
step = width;
}
av_image_fill_arrays(new PointerPointer(image_frame), image_frame.linesize(), data, pixelFormat, width, height, 1);
image_frame.linesize(0, step);
image_frame.format(pixelFormat);
image_frame.width(width);
image_frame.height(height);
/* push the decoded frame into the filtergraph */
if ((ret = av_buffersrc_add_frame_flags(buffersrc_ctx[n], image_frame, AV_BUFFERSRC_FLAG_KEEP_REF | AV_BUFFERSRC_FLAG_PUSH)) < 0) {
throw new Exception("av_buffersrc_add_frame_flags() error " + ret + ": Error while feeding the filtergraph.");
}
}
}
public synchronized void pushSamples(int n, int audioChannels, int sampleRate, int sampleFormat, Buffer ... samples) throws Exception {
try (PointerScope scope = new PointerScope()) {
if (!started) {
throw new Exception("start() was not called successfully!");
}
int ret;
Pointer[] data = new Pointer[samples.length];
int sampleSize = samples != null ? ((samples[0].limit() - samples[0].position()) / (samples.length > 1 ? 1 : audioChannels)) : 0;
if (samples != null && samples[0] instanceof ByteBuffer) {
sampleFormat = data.length > 1 ? AV_SAMPLE_FMT_U8P : AV_SAMPLE_FMT_U8;
for (int i = 0; i < data.length; i++) {
data[i] = new BytePointer((ByteBuffer)samples[i]);
}
} else if (samples != null && samples[0] instanceof ShortBuffer) {
sampleFormat = data.length > 1 ? AV_SAMPLE_FMT_S16P : AV_SAMPLE_FMT_S16;
for (int i = 0; i < data.length; i++) {
data[i] = new ShortPointer((ShortBuffer)samples[i]);
}
} else if (samples != null && samples[0] instanceof IntBuffer) {
sampleFormat = data.length > 1 ? AV_SAMPLE_FMT_S32P : AV_SAMPLE_FMT_S32;
for (int i = 0; i < data.length; i++) {
data[i] = new IntPointer((IntBuffer)samples[i]);
}
} else if (samples != null && samples[0] instanceof FloatBuffer) {
sampleFormat = data.length > 1 ? AV_SAMPLE_FMT_FLTP : AV_SAMPLE_FMT_FLT;
for (int i = 0; i < data.length; i++) {
data[i] = new FloatPointer((FloatBuffer)samples[i]);
}
} else if (samples != null && samples[0] instanceof DoubleBuffer) {
sampleFormat = data.length > 1 ? AV_SAMPLE_FMT_DBLP : AV_SAMPLE_FMT_DBL;
for (int i = 0; i < data.length; i++) {
data[i] = new DoublePointer((DoubleBuffer)samples[i]);
}
} else if (samples != null) {
for (int i = 0; i < data.length; i++) {
data[i] = new Pointer(samples[i]);
}
}
av_samples_fill_arrays(new PointerPointer(samples_frame), samples_frame.linesize(), new BytePointer(data[0]), audioChannels, sampleSize, sampleFormat, 1);
for (int i = 0; i < samples.length; i++) {
samples_frame.data(i, new BytePointer(data[i]));
}
av_channel_layout_default(default_layout, audioChannels);
samples_frame.ch_layout(default_layout);
samples_frame.nb_samples(sampleSize);
samples_frame.format(sampleFormat);
samples_frame.sample_rate(sampleRate);
/* push the decoded frame into the filtergraph */
if ((ret = av_buffersrc_add_frame_flags(abuffersrc_ctx[n], samples_frame, AV_BUFFERSRC_FLAG_KEEP_REF | AV_BUFFERSRC_FLAG_PUSH)) < 0) {
throw new Exception("av_buffersrc_add_frame_flags() error " + ret + ": Error while feeding the filtergraph.");
}
}
}
@Override public synchronized Frame pull() throws Exception {
if (!started) {
throw new Exception("start() was not called successfully!");
}
frame.keyFrame = false;
frame.imageWidth = 0;
frame.imageHeight = 0;
frame.imageDepth = 0;
frame.imageChannels = 0;
frame.imageStride = 0;
frame.image = null;
frame.sampleRate = 0;
frame.audioChannels = 0;
frame.samples = null;
frame.opaque = null;
Frame f = null;
if (f == null && buffersrc_ctx != null) {
f = pullImage();
}
if (f == null && abuffersrc_ctx != null) {
f = pullSamples();
}
if (f == null && inframe != null
&& ((inframe.image != null && buffersrc_ctx == null)
|| (inframe.samples != null && abuffersrc_ctx == null))) {
f = inframe;
}
inframe = null;
return f;
}
public synchronized Frame pullImage() throws Exception {
try (PointerScope scope = new PointerScope()) {
if (!started) {
throw new Exception("start() was not called successfully!");
}
av_frame_unref(filt_frame);
/* pull a filtered frame from the filtergraph */
int ret = av_buffersink_get_frame(buffersink_ctx, filt_frame);
if (ret == AVERROR_EAGAIN() || ret == AVERROR_EOF()) {
return null;
} else if (ret < 0) {
throw new Exception("av_buffersink_get_frame(): Error occurred: "
+ av_make_error_string(new BytePointer(256), 256, ret).getString());
}
frame.imageWidth = filt_frame.width();
frame.imageHeight = filt_frame.height();
frame.imageDepth = Frame.DEPTH_UBYTE;
if (filt_frame.data(1) == null) {
frame.imageStride = filt_frame.linesize(0);
BytePointer ptr = filt_frame.data(0);
// Fix bug on vflip filter, frame.imageStride can be negative
// see https://github.com/bytedeco/javacv/issues/975
if (ptr != null && !ptr.equals(image_ptr[0])) {
image_ptr[0] = ptr.capacity(frame.imageHeight * Math.abs(frame.imageStride));
image_buf[0] = ptr.asBuffer();
}
frame.image = image_buf;
frame.image[0].position(0).limit(frame.imageHeight * Math.abs(frame.imageStride));
frame.imageChannels = Math.abs(frame.imageStride) / frame.imageWidth;
frame.opaque = filt_frame;
} else {
frame.imageStride = frame.imageWidth;
int size = av_image_get_buffer_size(filt_frame.format(), frame.imageWidth, frame.imageHeight, 1);
if (image_ptr2[0] == null || image_ptr2[0].capacity() < size) {
av_free(image_ptr2[0]);
image_ptr2[0] = new BytePointer(av_malloc(size)).capacity(size);
image_buf2[0] = image_ptr2[0].asBuffer();
}
frame.image = image_buf2;
frame.image[0].position(0).limit(size);
frame.imageChannels = (size + frame.imageWidth * frame.imageHeight - 1) / (frame.imageWidth * frame.imageHeight);
ret = av_image_copy_to_buffer(image_ptr2[0].position(0), (int)image_ptr2[0].capacity(),
new PointerPointer(filt_frame), filt_frame.linesize(), filt_frame.format(), frame.imageWidth, frame.imageHeight, 1);
if (ret < 0) {
throw new Exception("av_image_copy_to_buffer() error " + ret + ": Cannot pull image.");
}
frame.opaque = image_ptr2[0];
}
frame.timestamp = 1000000L * filt_frame.pts() * time_base.num() / time_base.den();
return frame;
}
}
public synchronized Frame pullSamples() throws Exception {
try (PointerScope scope = new PointerScope()) {
if (!started) {
throw new Exception("start() was not called successfully!");
}
av_frame_unref(filt_frame);
/* pull a filtered frame from the filtergraph */
int ret = av_buffersink_get_frame(abuffersink_ctx, filt_frame);
if (ret == AVERROR_EAGAIN() || ret == AVERROR_EOF()) {
return null;
} else if (ret < 0) {
throw new Exception("av_buffersink_get_frame(): Error occurred: "
+ av_make_error_string(new BytePointer(256), 256, ret).getString());
}
int sample_format = filt_frame.format();
int planes = av_sample_fmt_is_planar(sample_format) != 0 ? (int)filt_frame.ch_layout().nb_channels() : 1;
int data_size = av_samples_get_buffer_size((IntPointer)null, filt_frame.ch_layout().nb_channels(),
filt_frame.nb_samples(), filt_frame.format(), 1) / planes;
if (samples_buf == null || samples_buf.length != planes) {
samples_ptr = new BytePointer[planes];
samples_buf = new Buffer[planes];
}
frame.audioChannels = filt_frame.ch_layout().nb_channels();
frame.sampleRate = filt_frame.sample_rate();
frame.samples = samples_buf;
frame.opaque = filt_frame;
int sample_size = data_size / av_get_bytes_per_sample(sample_format);
for (int i = 0; i < planes; i++) {
BytePointer p = filt_frame.data(i);
if (!p.equals(samples_ptr[i]) || samples_ptr[i].capacity() < data_size) {
samples_ptr[i] = p.capacity(data_size);
ByteBuffer b = p.asBuffer();
switch (sample_format) {
case AV_SAMPLE_FMT_U8:
case AV_SAMPLE_FMT_U8P: samples_buf[i] = b; break;
case AV_SAMPLE_FMT_S16:
case AV_SAMPLE_FMT_S16P: samples_buf[i] = b.asShortBuffer(); break;
case AV_SAMPLE_FMT_S32:
case AV_SAMPLE_FMT_S32P: samples_buf[i] = b.asIntBuffer(); break;
case AV_SAMPLE_FMT_FLT:
case AV_SAMPLE_FMT_FLTP: samples_buf[i] = b.asFloatBuffer(); break;
case AV_SAMPLE_FMT_DBL:
case AV_SAMPLE_FMT_DBLP: samples_buf[i] = b.asDoubleBuffer(); break;
default: assert false;
}
}
samples_buf[i].position(0).limit(sample_size);
}
frame.timestamp = 1000000L * filt_frame.pts() * atime_base.num() / atime_base.den();
return frame;
}
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/FFmpegFrameGrabber.java
================================================
/*
* Copyright (C) 2009-2025 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*
* Based on the avcodec_sample.0.5.0.c file available at
* http://web.me.com/dhoerl/Home/Tech_Blog/Entries/2009/1/22_Revised_avcodec_sample.c_files/avcodec_sample.0.5.0.c
* by Martin Böhme, Stephen Dranger, and David Hoerl
* as well as on the decoding_encoding.c file included in FFmpeg 0.11.1,
* and on the decode_video.c file included in FFmpeg 4.4,
* which is covered by the following copyright notice:
*
* Copyright (c) 2001 Fabrice Bellard
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/
package org.bytedeco.javacv;
import java.io.BufferedInputStream;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.nio.Buffer;
import java.nio.ByteBuffer;
import java.util.Collections;
import java.util.EnumSet;
import java.util.HashMap;
import java.util.Map;
import java.util.Map.Entry;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.DoublePointer;
import org.bytedeco.javacpp.IntPointer;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.javacpp.Pointer;
import org.bytedeco.javacpp.PointerScope;
import org.bytedeco.javacpp.PointerPointer;
import org.bytedeco.ffmpeg.avcodec.*;
import org.bytedeco.ffmpeg.avformat.*;
import org.bytedeco.ffmpeg.avutil.*;
import org.bytedeco.ffmpeg.swresample.*;
import org.bytedeco.ffmpeg.swscale.*;
import static org.bytedeco.ffmpeg.global.avcodec.*;
import static org.bytedeco.ffmpeg.global.avdevice.*;
import static org.bytedeco.ffmpeg.global.avformat.*;
import static org.bytedeco.ffmpeg.global.avutil.*;
import static org.bytedeco.ffmpeg.global.swresample.*;
import static org.bytedeco.ffmpeg.global.swscale.*;
/**
*
* @author Samuel Audet
*/
public class FFmpegFrameGrabber extends FrameGrabber {
public static class Exception extends FrameGrabber.Exception {
public Exception(String message) { super(message + " (For more details, make sure FFmpegLogCallback.set() has been called.)"); }
public Exception(String message, Throwable cause) { super(message, cause); }
}
public static String[] getDeviceDescriptions() throws Exception {
tryLoad();
throw new UnsupportedOperationException("Device enumeration not support by FFmpeg.");
}
public static FFmpegFrameGrabber createDefault(File deviceFile) throws Exception { return new FFmpegFrameGrabber(deviceFile); }
public static FFmpegFrameGrabber createDefault(String devicePath) throws Exception { return new FFmpegFrameGrabber(devicePath); }
public static FFmpegFrameGrabber createDefault(int deviceNumber) throws Exception { throw new Exception(FFmpegFrameGrabber.class + " does not support device numbers."); }
private static Exception loadingException = null;
public static void tryLoad() throws Exception {
if (loadingException != null) {
throw loadingException;
} else {
try {
Loader.load(org.bytedeco.ffmpeg.global.avutil.class);
Loader.load(org.bytedeco.ffmpeg.global.swresample.class);
Loader.load(org.bytedeco.ffmpeg.global.avcodec.class);
Loader.load(org.bytedeco.ffmpeg.global.avformat.class);
Loader.load(org.bytedeco.ffmpeg.global.swscale.class);
// Register all formats and codecs
av_jni_set_java_vm(Loader.getJavaVM(), null);
// avcodec_register_all();
// av_register_all();
avformat_network_init();
Loader.load(org.bytedeco.ffmpeg.global.avdevice.class);
avdevice_register_all();
} catch (Throwable t) {
if (t instanceof Exception) {
throw loadingException = (Exception)t;
} else {
throw loadingException = new Exception("Failed to load " + FFmpegFrameGrabber.class, t);
}
}
}
}
static {
try {
tryLoad();
// FFmpegLockCallback.init();
} catch (Exception ex) { }
}
public FFmpegFrameGrabber(URL url) {
this(url.toString());
}
public FFmpegFrameGrabber(File file) {
this(file.getAbsolutePath());
}
public FFmpegFrameGrabber(String filename) {
this.filename = filename;
this.pixelFormat = AV_PIX_FMT_NONE;
this.sampleFormat = AV_SAMPLE_FMT_NONE;
}
/** Calls {@code FFmpegFrameGrabber(inputStream, Integer.MAX_VALUE - 8)}
* so that the whole input stream is seekable. */
public FFmpegFrameGrabber(InputStream inputStream) {
this(inputStream, Integer.MAX_VALUE - 8);
}
/** Set maximumSize to 0 to disable seek and minimize startup time. */
public FFmpegFrameGrabber(InputStream inputStream, int maximumSize) {
this.inputStream = inputStream;
this.closeInputStream = true;
this.pixelFormat = AV_PIX_FMT_NONE;
this.sampleFormat = AV_SAMPLE_FMT_NONE;
this.maximumSize = maximumSize;
}
public void release() throws Exception {
synchronized (org.bytedeco.ffmpeg.global.avcodec.class) {
releaseUnsafe();
}
}
public synchronized void releaseUnsafe() throws Exception {
started = false;
if (plane_ptr != null && plane_ptr2 != null) {
plane_ptr.releaseReference();
plane_ptr2.releaseReference();
plane_ptr = plane_ptr2 = null;
}
if (pkt != null) {
if (pkt.stream_index() != -1) {
av_packet_unref(pkt);
}
pkt.releaseReference();
pkt = null;
}
if (default_layout != null) {
default_layout.releaseReference();
default_layout = null;
}
// Free the RGB image
if (image_ptr != null) {
for (int i = 0; i < image_ptr.length; i++) {
if (imageMode != ImageMode.RAW) {
av_free(image_ptr[i]);
}
}
image_ptr = null;
}
if (picture_rgb != null) {
av_frame_free(picture_rgb);
picture_rgb = null;
}
// Free the native format picture frame
if (picture != null) {
av_frame_free(picture);
picture = null;
}
// Close the video codec
if (video_c != null) {
avcodec_free_context(video_c);
video_c = null;
}
// Free the audio samples frame
if (samples_frame != null) {
av_frame_free(samples_frame);
samples_frame = null;
}
// Close the audio codec
if (audio_c != null) {
avcodec_free_context(audio_c);
audio_c = null;
}
// Close the video file
if (inputStream == null && oc != null && !oc.isNull()) {
avformat_close_input(oc);
oc = null;
}
if (img_convert_ctx != null) {
sws_freeContext(img_convert_ctx);
img_convert_ctx = null;
}
if (samples_ptr_out != null) {
for (int i = 0; i < samples_ptr_out.length; i++) {
av_free(samples_ptr_out[i].position(0));
}
samples_ptr_out = null;
samples_buf_out = null;
}
if (samples_convert_ctx != null) {
swr_free(samples_convert_ctx);
samples_convert_ctx.releaseReference();
samples_convert_ctx = null;
}
frameGrabbed = false;
frame = null;
timestamp = 0;
frameNumber = 0;
if (inputStream != null) {
try {
if (oc == null) {
// when called a second time
if (closeInputStream) {
inputStream.close();
}
} else if (maximumSize > 0) {
try {
inputStream.reset();
} catch (IOException ex) {
// "Resetting to invalid mark", give up?
System.err.println("Error on InputStream.reset(): " + ex);
}
}
} catch (IOException ex) {
throw new Exception("Error on InputStream.close(): ", ex);
} finally {
inputStreams.remove(oc);
if (avio != null) {
if (avio.buffer() != null) {
av_free(avio.buffer());
avio.buffer(null);
}
av_free(avio);
avio = null;
}
if (oc != null) {
avformat_close_input(oc);
oc = null;
}
}
}
Pointer.trimMemory();
}
@Override protected void finalize() throws Throwable {
super.finalize();
release();
}
static Map inputStreams = Collections.synchronizedMap(new HashMap());
static class ReadCallback extends Read_packet_Pointer_BytePointer_int {
@Override public int call(Pointer opaque, BytePointer buf, int buf_size) {
try {
byte[] b = new byte[buf_size];
InputStream is = inputStreams.get(opaque);
int size = is.read(b, 0, buf_size);
if (size < 0) {
return AVERROR_EOF();
} else {
buf.put(b, 0, size);
return size;
}
}
catch (Throwable t) {
System.err.println("Error on InputStream.read(): " + t);
return -1;
}
}
}
static class SeekCallback extends Seek_Pointer_long_int {
@Override public long call(Pointer opaque, long offset, int whence) {
try {
InputStream is = inputStreams.get(opaque);
long size = 0;
switch (whence) {
case 0: is.reset(); break; // SEEK_SET
case 1: break; // SEEK_CUR
case 2: // SEEK_END
is.reset();
while (true) {
long n = is.skip(Long.MAX_VALUE);
if (n == 0) break;
size += n;
}
offset += size;
is.reset();
break;
case AVSEEK_SIZE:
long remaining = 0;
while (true) {
long n = is.skip(Long.MAX_VALUE);
if (n == 0) break;
remaining += n;
}
is.reset();
while (true) {
long n = is.skip(Long.MAX_VALUE);
if (n == 0) break;
size += n;
}
offset = size - remaining;
is.reset();
break;
default: return -1;
}
long remaining = offset;
while (remaining > 0) {
long skipped = is.skip(remaining);
if (skipped == 0) break; // end of the stream
remaining -= skipped;
}
return whence == AVSEEK_SIZE ? size : 0;
} catch (Throwable t) {
System.err.println("Error on InputStream.reset() or skip(): " + t);
return -1;
}
}
}
static ReadCallback readCallback = new ReadCallback().retainReference();
static SeekCallback seekCallback = new SeekCallback().retainReference();
private InputStream inputStream;
private boolean closeInputStream;
private int maximumSize;
private AVIOContext avio;
private String filename;
private AVFormatContext oc;
private AVStream video_st, audio_st;
private AVCodecContext video_c, audio_c;
private AVFrame picture, picture_rgb;
private BytePointer[] image_ptr;
private Buffer[] image_buf;
private AVFrame samples_frame;
private BytePointer[] samples_ptr;
private Buffer[] samples_buf;
private BytePointer[] samples_ptr_out;
private Buffer[] samples_buf_out;
private PointerPointer plane_ptr, plane_ptr2;
private AVPacket pkt;
private SwsContext img_convert_ctx;
private SwrContext samples_convert_ctx;
private int samples_channels, samples_format, samples_rate;
private boolean frameGrabbed;
private Frame frame;
private int[] streams;
private AVChannelLayout default_layout;
private volatile boolean started = false;
public boolean isCloseInputStream() {
return closeInputStream;
}
public void setCloseInputStream(boolean closeInputStream) {
this.closeInputStream = closeInputStream;
}
/**
* Is there a video stream?
* @return {@code video_st!=null;}
*/
public boolean hasVideo() {
return video_st!=null;
}
/**
* Is there an audio stream?
* @return {@code audio_st!=null;}
*/
public boolean hasAudio() {
return audio_st!=null;
}
@Override public double getGamma() {
// default to a gamma of 2.2 for cheap Webcams, DV cameras, etc.
if (gamma == 0.0) {
return 2.2;
} else {
return gamma;
}
}
@Override public String getFormat() {
if (oc == null) {
return super.getFormat();
} else {
return oc.iformat().name().getString();
}
}
@Override public int getImageWidth() {
return imageWidth > 0 || video_c == null ? super.getImageWidth() : video_c.width();
}
@Override public int getImageHeight() {
return imageHeight > 0 || video_c == null ? super.getImageHeight() : video_c.height();
}
@Override public int getAudioChannels() {
return audioChannels > 0 || audio_c == null ? super.getAudioChannels() : audio_c.ch_layout().nb_channels();
}
@Override public int getPixelFormat() {
if (imageMode == ImageMode.COLOR || imageMode == ImageMode.GRAY) {
if (pixelFormat == AV_PIX_FMT_NONE) {
return imageMode == ImageMode.COLOR ? AV_PIX_FMT_BGR24 : AV_PIX_FMT_GRAY8;
} else {
return pixelFormat;
}
} else if (video_c != null) { // RAW
return video_c.pix_fmt();
} else {
return super.getPixelFormat();
}
}
@Override public int getVideoCodec() {
return video_c == null ? super.getVideoCodec() : video_c.codec_id();
}
@Override
public String getVideoCodecName(){
return video_c == null ? super.getVideoCodecName() : video_c.codec().name().getString();
}
@Override public int getVideoBitrate() {
return video_c == null ? super.getVideoBitrate() : (int)video_c.bit_rate();
}
@Override public double getAspectRatio() {
if (video_st == null) {
return super.getAspectRatio();
} else {
AVRational r = av_guess_sample_aspect_ratio(oc, video_st, picture);
double a = (double)r.num() / r.den();
return a == 0.0 ? 1.0 : a;
}
}
/** Returns {@link #getVideoFrameRate()} */
@Override public double getFrameRate() {
return getVideoFrameRate();
}
/**Estimation of audio frames per second.
*
* Care must be taken as this method may require unnecessary call of
* grabFrame(true, false, false, false, false) with frameGrabbed set to true.
*
* @return (double) getSampleRate()) / samples_frame.nb_samples()
* if samples_frame.nb_samples() is not zero, otherwise return 0
*/
public double getAudioFrameRate() {
if (audio_st == null) {
return 0.0;
} else {
if (samples_frame == null || samples_frame.nb_samples() == 0) {
try {
grabFrame(true, false, false, false, false);
frameGrabbed = true;
} catch (Exception e) {
return 0.0;
}
}
if (samples_frame != null && samples_frame.nb_samples() != 0)
return ((double) getSampleRate()) / samples_frame.nb_samples();
else return 0.0;
}
}
public double getVideoFrameRate() {
if (video_st == null) {
return super.getFrameRate();
} else {
AVRational r = video_st.avg_frame_rate();
if (r.num() == 0 && r.den() == 0) {
r = video_st.r_frame_rate();
}
return (double)r.num() / r.den();
}
}
@Override public int getAudioCodec() {
return audio_c == null ? super.getAudioCodec() : audio_c.codec_id();
}
@Override public String getAudioCodecName() {
return audio_c == null ? super.getAudioCodecName() : audio_c.codec().name().getString();
}
@Override public int getAudioBitrate() {
return audio_c == null ? super.getAudioBitrate() : (int)audio_c.bit_rate();
}
@Override public int getSampleFormat() {
if (sampleMode == SampleMode.SHORT || sampleMode == SampleMode.FLOAT) {
if (sampleFormat == AV_SAMPLE_FMT_NONE) {
return sampleMode == SampleMode.SHORT ? AV_SAMPLE_FMT_S16 : AV_SAMPLE_FMT_FLT;
} else {
return sampleFormat;
}
} else if (audio_c != null) { // RAW
return audio_c.sample_fmt();
} else {
return super.getSampleFormat();
}
}
@Override public int getSampleRate() {
return sampleRate > 0 || audio_c == null ? super.getSampleRate() : audio_c.sample_rate();
}
@Override public Map getMetadata() {
if (oc == null) {
return super.getMetadata();
}
AVDictionaryEntry entry = null;
Map metadata = new HashMap();
while ((entry = av_dict_get(oc.metadata(), "", entry, AV_DICT_IGNORE_SUFFIX)) != null) {
metadata.put(entry.key().getString(charset), entry.value().getString(charset));
}
return metadata;
}
@Override public Map getVideoMetadata() {
if (video_st == null) {
return super.getVideoMetadata();
}
AVDictionaryEntry entry = null;
Map metadata = new HashMap();
while ((entry = av_dict_get(video_st.metadata(), "", entry, AV_DICT_IGNORE_SUFFIX)) != null) {
metadata.put(entry.key().getString(charset), entry.value().getString(charset));
}
return metadata;
}
@Override public Map getAudioMetadata() {
if (audio_st == null) {
return super.getAudioMetadata();
}
AVDictionaryEntry entry = null;
Map metadata = new HashMap();
while ((entry = av_dict_get(audio_st.metadata(), "", entry, AV_DICT_IGNORE_SUFFIX)) != null) {
metadata.put(entry.key().getString(charset), entry.value().getString(charset));
}
return metadata;
}
@Override public String getMetadata(String key) {
if (oc == null) {
return super.getMetadata(key);
}
AVDictionaryEntry entry = av_dict_get(oc.metadata(), key, null, 0);
return entry == null || entry.value() == null ? null : entry.value().getString(charset);
}
@Override public String getVideoMetadata(String key) {
if (video_st == null) {
return super.getVideoMetadata(key);
}
AVDictionaryEntry entry = av_dict_get(video_st.metadata(), key, null, 0);
return entry == null || entry.value() == null ? null : entry.value().getString(charset);
}
@Override public String getAudioMetadata(String key) {
if (audio_st == null) {
return super.getAudioMetadata(key);
}
AVDictionaryEntry entry = av_dict_get(audio_st.metadata(), key, null, 0);
return entry == null || entry.value() == null ? null : entry.value().getString(charset);
}
@Override public Map getVideoSideData() {
if (video_st == null) {
return super.getVideoSideData();
}
videoSideData = new HashMap();
for (int i = 0; i < video_st.codecpar().nb_coded_side_data(); i++) {
AVPacketSideData sd = video_st.codecpar().coded_side_data().position(i);
String key = av_packet_side_data_name(sd.type()).getString();
Buffer value = sd.data().capacity(sd.size()).asBuffer();
videoSideData.put(key, value);
}
return videoSideData;
}
@Override public Buffer getVideoSideData(String key) {
return getVideoSideData().get(key);
}
/** Returns the rotation in degrees from the side data of the video stream, or 0 if unknown. */
public double getDisplayRotation() {
ByteBuffer b = (ByteBuffer)getVideoSideData("Display Matrix");
return b != null ? av_display_rotation_get(new IntPointer(new BytePointer(b))) : 0;
}
@Override public Map getAudioSideData() {
if (audio_st == null) {
return super.getAudioSideData();
}
audioSideData = new HashMap();
for (int i = 0; i < audio_st.codecpar().nb_coded_side_data(); i++) {
AVPacketSideData sd = audio_st.codecpar().coded_side_data().position(i);
String key = av_packet_side_data_name(sd.type()).getString();
Buffer value = sd.data().capacity(sd.size()).asBuffer();
audioSideData.put(key, value);
}
return audioSideData;
}
@Override public Buffer getAudioSideData(String key) {
return getAudioSideData().get(key);
}
/** default override of super.setFrameNumber implies setting
* of a frame close to a video frame having that number */
@Override public void setFrameNumber(int frameNumber) throws Exception {
if (hasVideo()) setTimestamp(Math.round((1000000L * frameNumber + 500000L)/ getFrameRate()));
else super.frameNumber = frameNumber;
}
/** if there is video stream tries to seek to video frame with corresponding timestamp
* otherwise sets super.frameNumber only because frameRate==0 if there is no video stream */
public void setVideoFrameNumber(int frameNumber) throws Exception {
// best guess, AVSEEK_FLAG_FRAME has not been implemented in FFmpeg...
if (hasVideo()) setVideoTimestamp(Math.round((1000000L * frameNumber + 500000L)/ getFrameRate()));
else super.frameNumber = frameNumber;
}
/** if there is audio stream tries to seek to audio frame with corresponding timestamp
* ignoring otherwise */
public void setAudioFrameNumber(int frameNumber) throws Exception {
// best guess, AVSEEK_FLAG_FRAME has not been implemented in FFmpeg...
if (hasAudio()) setAudioTimestamp(Math.round((1000000L * frameNumber + 500000L)/ getAudioFrameRate()));
}
/** setTimestamp without checking frame content (using old code used in JavaCV versions prior to 1.4.1) */
@Override public void setTimestamp(long timestamp) throws Exception {
setTimestamp(timestamp, false);
}
/** setTimestamp with possibility to select between old quick seek code or new code
* doing check of frame content. The frame check can be useful with corrupted files, when seeking may
* end up with an empty frame not containing video nor audio */
public void setTimestamp(long timestamp, boolean checkFrame) throws Exception {
setTimestamp(timestamp, checkFrame ? EnumSet.of(Frame.Type.VIDEO, Frame.Type.AUDIO) : null);
}
/** setTimestamp with resulting video frame type if there is a video stream.
* This should provide precise seek to a video frame containing the requested timestamp
* in most cases.
* */
public void setVideoTimestamp(long timestamp) throws Exception {
setTimestamp(timestamp, EnumSet.of(Frame.Type.VIDEO));
}
/** setTimestamp with resulting audio frame type if there is an audio stream.
* This should provide precise seek to an audio frame containing the requested timestamp
* in most cases.
* */
public void setAudioTimestamp(long timestamp) throws Exception {
setTimestamp(timestamp, EnumSet.of(Frame.Type.AUDIO));
}
/** setTimestamp with a priority the resulting frame should be:
* video (frameTypesToSeek contains only Frame.Type.VIDEO),
* audio (frameTypesToSeek contains only Frame.Type.AUDIO),
* or any (frameTypesToSeek contains both)
*/
private synchronized void setTimestamp(long timestamp, EnumSet frameTypesToSeek) throws Exception {
int ret;
if (oc == null) {
super.timestamp = timestamp;
} else {
timestamp = timestamp * AV_TIME_BASE / 1000000L;
/* the stream start time */
long ts0 = oc.start_time() != AV_NOPTS_VALUE ? oc.start_time() : 0;
if (frameTypesToSeek != null //new code providing check of frame content while seeking to the timestamp
&& (frameTypesToSeek.contains(Frame.Type.VIDEO) || frameTypesToSeek.contains(Frame.Type.AUDIO))
&& (hasVideo() || hasAudio())) {
/* After the call of ffmpeg's avformat_seek_file(...) with the flag set to AVSEEK_FLAG_BACKWARD
* the decoding position should be located before the requested timestamp in a closest position
* from which all the active streams can be decoded successfully.
* The following seeking consists of two stages:
* 1. Grab frames till the frame corresponding to that "closest" position
* (the first frame containing decoded data).
*
* 2. Grab frames till the desired timestamp is reached. The number of steps is restricted
* by doubled estimation of frames between that "closest" position and the desired position.
*
* frameTypesToSeek parameter sets the preferred type of frames to seek.
* It can be chosen from three possible types: VIDEO, AUDIO or any of them.
* The setting means only a preference in the type. That is, if VIDEO or AUDIO is
* specified but the file does not have video or audio stream - any type will be used instead.
*/
/* Check if file contains requested streams */
if ((frameTypesToSeek.contains(Frame.Type.VIDEO) && !hasVideo() ) ||
(frameTypesToSeek.contains(Frame.Type.AUDIO) && !hasAudio() ))
frameTypesToSeek = EnumSet.of(Frame.Type.VIDEO, Frame.Type.AUDIO);
/* If frameTypesToSeek is set explicitly to VIDEO or AUDIO
* we need to use start time of the corresponding stream
* instead of the common start time
*/
if (frameTypesToSeek.size()==1) {
if (frameTypesToSeek.contains(Frame.Type.VIDEO)) {
if (video_st!=null && video_st.start_time() != AV_NOPTS_VALUE) {
AVRational time_base = video_st.time_base();
ts0 = 1000000L * video_st.start_time() * time_base.num() / time_base.den();
}
}
else if (frameTypesToSeek.contains(Frame.Type.AUDIO)) {
if (audio_st!=null && audio_st.start_time() != AV_NOPTS_VALUE) {
AVRational time_base = audio_st.time_base();
ts0 = 1000000L * audio_st.start_time() * time_base.num() / time_base.den();
}
}
}
/* Sometimes the ffmpeg's avformat_seek_file(...) function brings us not to a position before
* the desired but few frames after. In case we need a frame-precision seek we may
* try to request an earlier timestamp.
*/
long early_ts = timestamp;
/* add the stream start time */
timestamp += ts0;
early_ts += ts0;
long initialSeekPosition = Long.MIN_VALUE;
long maxSeekSteps = 0;
long count = 0;
Frame seekFrame = null;
do {
if ((ret = avformat_seek_file(oc, -1, 0L, early_ts, early_ts, AVSEEK_FLAG_BACKWARD)) < 0)
throw new Exception("avformat_seek_file() error " + ret + ": Could not seek file to timestamp " + timestamp + ".");
if (video_c != null) {
avcodec_flush_buffers(video_c);
}
if (audio_c != null) {
avcodec_flush_buffers(audio_c);
}
if (pkt.stream_index() != -1) {
av_packet_unref(pkt);
pkt.stream_index(-1);
}
seekFrame = grabFrame(frameTypesToSeek.contains(Frame.Type.AUDIO), frameTypesToSeek.contains(Frame.Type.VIDEO), false, false, false);
if (seekFrame == null) return;
initialSeekPosition = seekFrame.timestamp;
if(early_ts==0L) break;
early_ts-=500000L;
if(early_ts<0) early_ts=0L;
} while (initialSeekPosition>timestamp);
double frameDuration = 0.0;
if (seekFrame.image != null && this.getFrameRate() > 0)
frameDuration = AV_TIME_BASE / (double)getFrameRate();
else if (seekFrame.samples != null && samples_frame != null && getSampleRate() > 0) {
frameDuration = AV_TIME_BASE * samples_frame.nb_samples() / (double)getSampleRate();
}
if(frameDuration>0.0) {
maxSeekSteps = 0; //no more grab if the distance to the requested timestamp is smaller than frameDuration
if (timestamp - initialSeekPosition + 1 > frameDuration) //allow for a rounding error
maxSeekSteps = (long)(10*(timestamp - initialSeekPosition)/frameDuration);
}
else if (initialSeekPosition < timestamp) maxSeekSteps = 1000;
double delta = 0.0; //for the timestamp correction
count = 0;
while(count < maxSeekSteps) {
seekFrame = grabFrame(frameTypesToSeek.contains(Frame.Type.AUDIO), frameTypesToSeek.contains(Frame.Type.VIDEO), false, false, false);
if (seekFrame == null) return; //is it better to throw NullPointerException?
count++;
double ts=seekFrame.timestamp;
frameDuration = 0.0;
if (seekFrame.image != null && this.getFrameRate() > 0)
frameDuration = AV_TIME_BASE / (double)getFrameRate();
else if (seekFrame.samples != null && samples_frame != null && getSampleRate() > 0)
frameDuration = AV_TIME_BASE * samples_frame.nb_samples() / (double)getSampleRate();
delta = 0.0;
if (frameDuration>0.0) {
delta = (ts-ts0)/frameDuration - Math.round((ts-ts0)/frameDuration);
if (Math.abs(delta)>0.2) delta=0.0;
}
ts-=delta*frameDuration; // corrected timestamp
if (ts + frameDuration > timestamp) break;
}
} else { //old quick seeking code used in JavaCV versions prior to 1.4.1
/* add the stream start time */
timestamp += ts0;
if ((ret = avformat_seek_file(oc, -1, Long.MIN_VALUE, timestamp, Long.MAX_VALUE, AVSEEK_FLAG_BACKWARD)) < 0) {
throw new Exception("avformat_seek_file() error " + ret + ": Could not seek file to timestamp " + timestamp + ".");
}
if (video_c != null) {
avcodec_flush_buffers(video_c);
}
if (audio_c != null) {
avcodec_flush_buffers(audio_c);
}
if (pkt.stream_index() != -1) {
av_packet_unref(pkt);
pkt.stream_index(-1);
}
/* comparing to timestamp +/- 1 avoids rouding issues for framerates
which are no proper divisors of 1000000, e.g. where
av_frame_get_best_effort_timestamp in grabFrame sets this.timestamp
to ...666 and the given timestamp has been rounded to ...667
(or vice versa)
*/
int count = 0; // prevent infinite loops with corrupted files
while (this.timestamp > timestamp + 1 && grabFrame(true, true, false, false) != null && count++ < 1000) {
// flush frames if seeking backwards
}
count = 0;
while (this.timestamp < timestamp - 1 && grabFrame(true, true, false, false) != null && count++ < 1000) {
// decode up to the desired frame
}
}
frameGrabbed = true;
}
}
/** Returns {@link #getLengthInVideoFrames()} */
@Override public int getLengthInFrames() {
// best guess...
return getLengthInVideoFrames();
}
@Override public long getLengthInTime() {
return oc.duration() * 1000000L / AV_TIME_BASE;
}
/** Returns {@code (int) Math.round(getLengthInTime() * getFrameRate() / 1000000L)}, which is an approximation in general. */
public int getLengthInVideoFrames() {
// best guess...
return (int) Math.round(getLengthInTime() * getFrameRate() / 1000000L);
}
public int getLengthInAudioFrames() {
// best guess...
double afr = getAudioFrameRate();
if (afr > 0) return (int) (getLengthInTime() * afr / 1000000L);
else return 0;
}
public AVFormatContext getFormatContext() {
return oc;
}
/** Calls {@code start(true)}. */
@Override public void start() throws Exception {
start(true);
}
/** Set findStreamInfo to false to minimize startup time, at the expense of robustness. */
public void start(boolean findStreamInfo) throws Exception {
synchronized (org.bytedeco.ffmpeg.global.avcodec.class) {
startUnsafe(findStreamInfo);
}
}
public void startUnsafe() throws Exception {
startUnsafe(true);
}
public synchronized void startUnsafe(boolean findStreamInfo) throws Exception {
try (PointerScope scope = new PointerScope()) {
if (oc != null && !oc.isNull()) {
throw new Exception("start() has already been called: Call stop() before calling start() again.");
}
int ret;
img_convert_ctx = null;
oc = new AVFormatContext(null);
video_c = null;
audio_c = null;
plane_ptr = new PointerPointer(AVFrame.AV_NUM_DATA_POINTERS).retainReference();
plane_ptr2 = new PointerPointer(AVFrame.AV_NUM_DATA_POINTERS).retainReference();
pkt = new AVPacket().retainReference();
frameGrabbed = false;
frame = new Frame();
timestamp = 0;
frameNumber = 0;
default_layout = new AVChannelLayout().retainReference();
pkt.stream_index(-1);
// Open video file
AVInputFormat f = null;
if (format != null && format.length() > 0) {
if ((f = av_find_input_format(format)) == null) {
throw new Exception("av_find_input_format() error: Could not find input format \"" + format + "\".");
}
}
AVDictionary options = new AVDictionary(null);
if (frameRate > 0) {
AVRational r = av_d2q(frameRate, 1001000);
av_dict_set(options, "framerate", r.num() + "/" + r.den(), 0);
}
if (pixelFormat >= 0) {
av_dict_set(options, "pixel_format", av_get_pix_fmt_name(pixelFormat).getString(), 0);
} else if (imageMode != ImageMode.RAW) {
av_dict_set(options, "pixel_format", imageMode == ImageMode.COLOR ? "bgr24" : "gray8", 0);
}
if (imageWidth > 0 && imageHeight > 0) {
av_dict_set(options, "video_size", imageWidth + "x" + imageHeight, 0);
}
if (sampleRate > 0) {
av_dict_set(options, "sample_rate", "" + sampleRate, 0);
}
if (audioChannels > 0) {
av_dict_set(options, "channels", "" + audioChannels, 0);
}
for (Entry e : this.options.entrySet()) {
av_dict_set(options, e.getKey(), e.getValue(), 0);
}
if (inputStream != null) {
if (!inputStream.markSupported()) {
inputStream = new BufferedInputStream(inputStream);
}
inputStream.mark(maximumSize);
oc = avformat_alloc_context();
avio = avio_alloc_context(new BytePointer(av_malloc(4096)), 4096, 0, oc, readCallback, null, maximumSize > 0 ? seekCallback : null);
oc.pb(avio);
filename = inputStream.toString();
inputStreams.put(oc, inputStream);
}
if ((ret = avformat_open_input(oc, filename, f, options)) < 0) {
av_dict_set(options, "pixel_format", null, 0);
if ((ret = avformat_open_input(oc, filename, f, options)) < 0) {
throw new Exception("avformat_open_input() error " + ret + ": Could not open input \"" + filename + "\". (Has setFormat() been called?)");
}
}
FFmpegLogCallback.logRejectedOptions(options, "avformat_open_input");
av_dict_free(options);
oc.max_delay(maxDelay);
// Retrieve stream information, if desired
if (findStreamInfo && (ret = avformat_find_stream_info(oc, (PointerPointer)null)) < 0) {
throw new Exception("avformat_find_stream_info() error " + ret + ": Could not find stream information.");
}
if (av_log_get_level() >= AV_LOG_INFO) {
// Dump information about file onto standard error
av_dump_format(oc, 0, filename, 0);
}
// Find the first stream with the user-specified disposition property
int nb_streams = oc.nb_streams();
for (int i = 0; i < nb_streams; i++) {
AVStream st = oc.streams(i);
AVCodecParameters par = st.codecpar();
if (videoStream < 0 && par.codec_type() == AVMEDIA_TYPE_VIDEO && st.disposition() == videoDisposition) {
videoStream = i;
} else if (audioStream < 0 && par.codec_type() == AVMEDIA_TYPE_AUDIO && st.disposition() == audioDisposition) {
audioStream = i;
}
}
// Find the first video and audio stream, unless the user specified otherwise
video_st = audio_st = null;
AVCodecParameters video_par = null, audio_par = null;
streams = new int[nb_streams];
for (int i = 0; i < nb_streams; i++) {
AVStream st = oc.streams(i);
// Get a pointer to the codec context for the video or audio stream
AVCodecParameters par = st.codecpar();
streams[i] = par.codec_type();
if (video_st == null && par.codec_type() == AVMEDIA_TYPE_VIDEO
&& par.codec_id() != AV_CODEC_ID_NONE && (videoStream < 0 || videoStream == i)) {
video_st = st;
video_par = par;
videoStream = i;
} else if (audio_st == null && par.codec_type() == AVMEDIA_TYPE_AUDIO
&& par.codec_id() != AV_CODEC_ID_NONE && (audioStream < 0 || audioStream == i)) {
audio_st = st;
audio_par = par;
audioStream = i;
}
}
if (video_st == null && audio_st == null) {
throw new Exception("Did not find a video or audio stream inside \"" + filename
+ "\" for videoStream == " + videoStream + " and audioStream == " + audioStream + ".");
}
if (video_st != null) {
// Find the decoder for the video stream
AVCodec codec = avcodec_find_decoder_by_name(videoCodecName);
if (codec == null) {
codec = avcodec_find_decoder(video_par.codec_id());
}
if (codec == null) {
throw new Exception("avcodec_find_decoder() error: Unsupported video format or codec not found: " + video_par.codec_id() + ".");
}
/* Allocate a codec context for the decoder */
if ((video_c = avcodec_alloc_context3(codec)) == null) {
throw new Exception("avcodec_alloc_context3() error: Could not allocate video decoding context.");
}
/* copy the stream parameters from the muxer */
if ((ret = avcodec_parameters_to_context(video_c, video_st.codecpar())) < 0) {
releaseUnsafe();
throw new Exception("avcodec_parameters_to_context() error " + ret + ": Could not copy the video stream parameters.");
}
options = new AVDictionary(null);
for (Entry e : videoOptions.entrySet()) {
av_dict_set(options, e.getKey(), e.getValue(), 0);
}
// Enable multithreading when available
video_c.thread_count(0);
// Open video codec
if ((ret = avcodec_open2(video_c, codec, options)) < 0) {
throw new Exception("avcodec_open2() error " + ret + ": Could not open video codec.");
}
FFmpegLogCallback.logRejectedOptions(options, "avcodec_open2");
av_dict_free(options);
// Hack to correct wrong frame rates that seem to be generated by some codecs
if (video_c.time_base().num() > 1000 && video_c.time_base().den() == 1) {
video_c.time_base().den(1000);
}
// Allocate video frame and an AVFrame structure for the RGB image
if ((picture = av_frame_alloc()) == null) {
throw new Exception("av_frame_alloc() error: Could not allocate raw picture frame.");
}
if ((picture_rgb = av_frame_alloc()) == null) {
throw new Exception("av_frame_alloc() error: Could not allocate RGB picture frame.");
}
initPictureRGB();
}
if (audio_st != null) {
// Find the decoder for the audio stream
AVCodec codec = avcodec_find_decoder_by_name(audioCodecName);
if (codec == null) {
codec = avcodec_find_decoder(audio_par.codec_id());
}
if (codec == null) {
throw new Exception("avcodec_find_decoder() error: Unsupported audio format or codec not found: " + audio_par.codec_id() + ".");
}
/* Allocate a codec context for the decoder */
if ((audio_c = avcodec_alloc_context3(codec)) == null) {
throw new Exception("avcodec_alloc_context3() error: Could not allocate audio decoding context.");
}
/* copy the stream parameters from the muxer */
if ((ret = avcodec_parameters_to_context(audio_c, audio_st.codecpar())) < 0) {
releaseUnsafe();
throw new Exception("avcodec_parameters_to_context() error " + ret + ": Could not copy the audio stream parameters.");
}
options = new AVDictionary(null);
for (Entry e : audioOptions.entrySet()) {
av_dict_set(options, e.getKey(), e.getValue(), 0);
}
// Enable multithreading when available
audio_c.thread_count(0);
// Open audio codec
if ((ret = avcodec_open2(audio_c, codec, options)) < 0) {
throw new Exception("avcodec_open2() error " + ret + ": Could not open audio codec.");
}
FFmpegLogCallback.logRejectedOptions(options, "avcodec_open2");
av_dict_free(options);
// Allocate audio samples frame
if ((samples_frame = av_frame_alloc()) == null) {
throw new Exception("av_frame_alloc() error: Could not allocate audio frame.");
}
samples_ptr = new BytePointer[] { null };
samples_buf = new Buffer[] { null };
}
started = true;
}
}
private void initPictureRGB() {
int width = imageWidth > 0 ? imageWidth : video_c.width();
int height = imageHeight > 0 ? imageHeight : video_c.height();
switch (imageMode) {
case COLOR:
case GRAY:
// If size changes I new allocation is needed -> free the old one.
if (image_ptr != null) {
// First kill all references, then free it.
image_buf = null;
BytePointer[] temp = image_ptr;
image_ptr = null;
av_free(temp[0]);
}
int fmt = getPixelFormat();
// work around bug in swscale: https://trac.ffmpeg.org/ticket/1031
int align = 64;
int stride = width;
for (int i = 1; i <= align; i += i) {
stride = (width + (i - 1)) & ~(i - 1);
av_image_fill_linesizes(picture_rgb.linesize(), fmt, stride);
if ((picture_rgb.linesize(0) & (align - 1)) == 0) {
break;
}
}
// Determine required buffer size and allocate buffer
int size = av_image_get_buffer_size(fmt, stride, height, 1);
image_ptr = new BytePointer[] { new BytePointer(av_malloc(size)).capacity(size) };
image_buf = new Buffer[] { image_ptr[0].asBuffer() };
// Assign appropriate parts of buffer to image planes in picture_rgb
// Note that picture_rgb is an AVFrame, but AVFrame is a superset of AVPicture
av_image_fill_arrays(new PointerPointer(picture_rgb), picture_rgb.linesize(), image_ptr[0], fmt, stride, height, 1);
picture_rgb.format(fmt);
picture_rgb.width(width);
picture_rgb.height(height);
break;
case RAW:
image_ptr = new BytePointer[] { null };
image_buf = new Buffer[] { null };
break;
default:
assert false;
}
}
@Override public void stop() throws Exception {
release();
}
@Override public synchronized void trigger() throws Exception {
if (oc == null || oc.isNull()) {
throw new Exception("Could not trigger: No AVFormatContext. (Has start() been called?)");
}
if (pkt.stream_index() != -1) {
av_packet_unref(pkt);
pkt.stream_index(-1);
}
for (int i = 0; i < numBuffers+1; i++) {
if (av_read_frame(oc, pkt) < 0) {
return;
}
av_packet_unref(pkt);
}
}
private void processImage() throws Exception {
frame.imageWidth = imageWidth > 0 ? imageWidth : video_c.width();
frame.imageHeight = imageHeight > 0 ? imageHeight : video_c.height();
frame.imageDepth = Frame.DEPTH_UBYTE;
switch (imageMode) {
case COLOR:
case GRAY:
// Deinterlace Picture
if (deinterlace) {
throw new Exception("Cannot deinterlace: Functionality moved to FFmpegFrameFilter.");
}
// Has the size changed?
if (frame.imageWidth != picture_rgb.width() || frame.imageHeight != picture_rgb.height()) {
initPictureRGB();
}
// Copy "metadata" fields
av_frame_copy_props(picture_rgb, picture);
// Convert the image into BGR or GRAY format that OpenCV uses
img_convert_ctx = sws_getCachedContext(img_convert_ctx,
video_c.width(), video_c.height(), video_c.pix_fmt(),
frame.imageWidth, frame.imageHeight, getPixelFormat(),
imageScalingFlags != 0 ? imageScalingFlags : SWS_BILINEAR,
null, null, (DoublePointer)null);
if (img_convert_ctx == null) {
throw new Exception("sws_getCachedContext() error: Cannot initialize the conversion context.");
}
// Convert the image from its native format to RGB or GRAY
sws_scale(img_convert_ctx, new PointerPointer(picture), picture.linesize(), 0,
video_c.height(), new PointerPointer(picture_rgb), picture_rgb.linesize());
frame.imageStride = picture_rgb.linesize(0);
frame.image = image_buf;
frame.opaque = picture_rgb;
break;
case RAW:
frame.imageStride = picture.linesize(0);
BytePointer ptr = picture.data(0);
if (ptr != null && !ptr.equals(image_ptr[0])) {
image_ptr[0] = ptr.capacity(frame.imageHeight * frame.imageStride);
image_buf[0] = ptr.asBuffer();
}
frame.image = image_buf;
frame.opaque = picture;
break;
default:
assert false;
}
frame.image[0].limit(frame.imageHeight * frame.imageStride);
frame.imageChannels = frame.imageStride / frame.imageWidth;
}
private void processSamples() throws Exception {
int ret;
int sample_format = samples_frame.format();
int planes = av_sample_fmt_is_planar(sample_format) != 0 ? (int)samples_frame.ch_layout().nb_channels() : 1;
int data_size = av_samples_get_buffer_size((IntPointer)null, audio_c.ch_layout().nb_channels(),
samples_frame.nb_samples(), audio_c.sample_fmt(), 1) / planes;
if (samples_buf == null || samples_buf.length != planes) {
samples_ptr = new BytePointer[planes];
samples_buf = new Buffer[planes];
}
frame.sampleRate = audio_c.sample_rate();
frame.audioChannels = audio_c.ch_layout().nb_channels();
frame.samples = samples_buf;
frame.opaque = samples_frame;
int sample_size = data_size / av_get_bytes_per_sample(sample_format);
for (int i = 0; i < planes; i++) {
BytePointer p = samples_frame.data(i);
if (!p.equals(samples_ptr[i]) || samples_ptr[i].capacity() < data_size) {
samples_ptr[i] = p.capacity(data_size);
ByteBuffer b = p.asBuffer();
switch (sample_format) {
case AV_SAMPLE_FMT_U8:
case AV_SAMPLE_FMT_U8P: samples_buf[i] = b; break;
case AV_SAMPLE_FMT_S16:
case AV_SAMPLE_FMT_S16P: samples_buf[i] = b.asShortBuffer(); break;
case AV_SAMPLE_FMT_S32:
case AV_SAMPLE_FMT_S32P: samples_buf[i] = b.asIntBuffer(); break;
case AV_SAMPLE_FMT_FLT:
case AV_SAMPLE_FMT_FLTP: samples_buf[i] = b.asFloatBuffer(); break;
case AV_SAMPLE_FMT_DBL:
case AV_SAMPLE_FMT_DBLP: samples_buf[i] = b.asDoubleBuffer(); break;
default: assert false;
}
}
samples_buf[i].position(0).limit(sample_size);
}
if (audio_c.ch_layout().nb_channels() != getAudioChannels() || audio_c.sample_fmt() != getSampleFormat() || audio_c.sample_rate() != getSampleRate()) {
if (samples_convert_ctx == null || samples_channels != getAudioChannels() || samples_format != getSampleFormat() || samples_rate != getSampleRate()) {
if (samples_convert_ctx == null) {
samples_convert_ctx = new SwrContext().retainReference();
}
av_channel_layout_default(default_layout, getAudioChannels());
if ((ret = swr_alloc_set_opts2(samples_convert_ctx, default_layout, getSampleFormat(), getSampleRate(),
audio_c.ch_layout(), audio_c.sample_fmt(), audio_c.sample_rate(), 0, null)) < 0) {
throw new Exception("swr_alloc_set_opts2() error " + ret + ": Cannot allocate the conversion context.");
} else if ((ret = swr_init(samples_convert_ctx)) < 0) {
throw new Exception("swr_init() error " + ret + ": Cannot initialize the conversion context.");
}
samples_channels = getAudioChannels();
samples_format = getSampleFormat();
samples_rate = getSampleRate();
}
int sample_size_in = samples_frame.nb_samples();
int planes_out = av_sample_fmt_is_planar(samples_format) != 0 ? (int)samples_frame.ch_layout().nb_channels() : 1;
int sample_size_out = swr_get_out_samples(samples_convert_ctx, sample_size_in);
int sample_bytes_out = av_get_bytes_per_sample(samples_format);
int buffer_size_out = sample_size_out * sample_bytes_out * (planes_out > 1 ? 1 : samples_channels);
if (samples_buf_out == null || samples_buf.length != planes_out || samples_ptr_out[0].capacity() < buffer_size_out) {
for (int i = 0; samples_ptr_out != null && i < samples_ptr_out.length; i++) {
av_free(samples_ptr_out[i].position(0));
}
samples_ptr_out = new BytePointer[planes_out];
samples_buf_out = new Buffer[planes_out];
for (int i = 0; i < planes_out; i++) {
samples_ptr_out[i] = new BytePointer(av_malloc(buffer_size_out)).capacity(buffer_size_out);
ByteBuffer b = samples_ptr_out[i].asBuffer();
switch (samples_format) {
case AV_SAMPLE_FMT_U8:
case AV_SAMPLE_FMT_U8P: samples_buf_out[i] = b; break;
case AV_SAMPLE_FMT_S16:
case AV_SAMPLE_FMT_S16P: samples_buf_out[i] = b.asShortBuffer(); break;
case AV_SAMPLE_FMT_S32:
case AV_SAMPLE_FMT_S32P: samples_buf_out[i] = b.asIntBuffer(); break;
case AV_SAMPLE_FMT_FLT:
case AV_SAMPLE_FMT_FLTP: samples_buf_out[i] = b.asFloatBuffer(); break;
case AV_SAMPLE_FMT_DBL:
case AV_SAMPLE_FMT_DBLP: samples_buf_out[i] = b.asDoubleBuffer(); break;
default: assert false;
}
}
}
frame.sampleRate = samples_rate;
frame.audioChannels = samples_channels;
frame.samples = samples_buf_out;
if ((ret = swr_convert(samples_convert_ctx, plane_ptr.put(samples_ptr_out), sample_size_out, plane_ptr2.put(samples_ptr), sample_size_in)) < 0) {
throw new Exception("swr_convert() error " + ret + ": Cannot convert audio samples.");
}
for (int i = 0; i < planes_out; i++) {
samples_ptr_out[i].position(0).limit(ret * (planes_out > 1 ? 1 : samples_channels));
samples_buf_out[i].position(0).limit(ret * (planes_out > 1 ? 1 : samples_channels));
}
}
}
public Frame grab() throws Exception {
return grabFrame(true, true, true, false, true);
}
public Frame grabImage() throws Exception {
return grabFrame(false, true, true, false, false);
}
public Frame grabSamples() throws Exception {
return grabFrame(true, false, true, false, false);
}
public Frame grabKeyFrame() throws Exception {
return grabFrame(false, true, true, true, false);
}
public Frame grabFrame(boolean doAudio, boolean doVideo, boolean doProcessing, boolean keyFrames) throws Exception {
return grabFrame(doAudio, doVideo, doProcessing, keyFrames, true);
}
public synchronized Frame grabFrame(boolean doAudio, boolean doVideo, boolean doProcessing, boolean keyFrames, boolean doData) throws Exception {
try (PointerScope scope = new PointerScope()) {
if (oc == null || oc.isNull()) {
throw new Exception("Could not grab: No AVFormatContext. (Has start() been called?)");
} else if ((!doVideo || video_st == null) && (!doAudio || audio_st == null) && !doData) {
return null;
}
if (!started) {
throw new Exception("start() was not called successfully!");
}
boolean videoFrameGrabbed = frameGrabbed && frame.image != null;
boolean audioFrameGrabbed = frameGrabbed && frame.samples != null;
boolean dataFrameGrabbed = frameGrabbed && frame.data != null;
frameGrabbed = false;
if (doVideo && videoFrameGrabbed) {
if (doProcessing) {
processImage();
}
frame.keyFrame = (picture.flags() & AVFrame.AV_FRAME_FLAG_KEY) != 0;
return frame;
} else if (doAudio && audioFrameGrabbed) {
if (doProcessing) {
processSamples();
}
frame.keyFrame = (samples_frame.flags() & AVFrame.AV_FRAME_FLAG_KEY) != 0;
return frame;
} else if (doData && dataFrameGrabbed) {
return frame;
}
frame.keyFrame = false;
frame.imageWidth = 0;
frame.imageHeight = 0;
frame.imageDepth = 0;
frame.imageChannels = 0;
frame.imageStride = 0;
frame.image = null;
frame.sampleRate = 0;
frame.audioChannels = 0;
frame.samples = null;
frame.data = null;
frame.opaque = null;
frame.type = null;
boolean done = false;
boolean readPacket = pkt.stream_index() == -1;
while (!done) {
int ret = 0;
if (readPacket) {
if (pkt.stream_index() != -1) {
// Free the packet that was allocated by av_read_frame
av_packet_unref(pkt);
pkt.stream_index(-1);
}
if ((ret = av_read_frame(oc, pkt)) < 0) {
if (ret == AVERROR_EAGAIN()) {
try {
Thread.sleep(10);
continue;
} catch (InterruptedException ex) {
// reset interrupt to be nice
Thread.currentThread().interrupt();
return null;
}
}
if ((doVideo && video_st != null) || (doAudio && audio_st != null)) {
// The video or audio codec may have buffered some frames
pkt.stream_index(doVideo && video_st != null ? video_st.index() : audio_st.index());
pkt.flags(AV_PKT_FLAG_KEY);
pkt.data(null);
pkt.size(0);
} else {
pkt.stream_index(-1);
return null;
}
}
}
frame.streamIndex = pkt.stream_index();
// Is this a packet from the video stream?
if (doVideo && video_st != null && frame.streamIndex == video_st.index()
&& (!keyFrames || pkt.flags() == AV_PKT_FLAG_KEY)) {
// Decode video frame
if (readPacket) {
ret = avcodec_send_packet(video_c, pkt);
if (pkt.data() == null && pkt.size() == 0) {
pkt.stream_index(-1);
}
if (ret == AVERROR_EAGAIN() || ret == AVERROR_EOF()) {
// The video codec may have buffered some frames
} else if (ret < 0) {
// Ignore errors to emulate the behavior of the old API
// throw new Exception("avcodec_send_packet() error " + ret + ": Error sending a video packet for decoding.");
}
}
// Did we get a video frame?
while (!done) {
ret = avcodec_receive_frame(video_c, picture);
if (ret == AVERROR_EAGAIN() || ret == AVERROR_EOF()) {
if (pkt.data() == null && pkt.size() == 0) {
pkt.stream_index(-1);
doVideo = false;
if (doAudio) {
readPacket = false;
break;
}
return null;
} else {
readPacket = true;
break;
}
} else if (ret < 0) {
// Ignore errors to emulate the behavior of the old API
// throw new Exception("avcodec_receive_frame() error " + ret + ": Error during video decoding.");
readPacket = true;
break;
}
if (!keyFrames || picture.pict_type() == AV_PICTURE_TYPE_I) {
long pts = picture.best_effort_timestamp();
AVRational time_base = video_st.time_base();
timestamp = 1000000L * pts * time_base.num() / time_base.den();
long ts0 = oc.start_time() != AV_NOPTS_VALUE ? oc.start_time() : 0;
// best guess, AVCodecContext.frame_number = number of decoded frames...
frameNumber = (int)Math.round((timestamp - ts0) * getFrameRate() / 1000000L);
frame.image = image_buf;
if (doProcessing) {
processImage();
}
/* the picture is allocated by the decoder. no need to
free it */
done = true;
frame.timestamp = timestamp;
frame.keyFrame = (picture.flags() & AVFrame.AV_FRAME_FLAG_KEY) != 0;
frame.pictType = (char)av_get_picture_type_char(picture.pict_type());
frame.type = Frame.Type.VIDEO;
}
}
} else if (doAudio && audio_st != null && frame.streamIndex == audio_st.index()) {
// Decode audio frame
if (readPacket) {
ret = avcodec_send_packet(audio_c, pkt);
if (ret < 0) {
// Ignore errors to emulate the behavior of the old API
// throw new Exception("avcodec_send_packet() error " + ret + ": Error sending an audio packet for decoding.");
}
}
// Did we get an audio frame?
while (!done) {
ret = avcodec_receive_frame(audio_c, samples_frame);
if (ret == AVERROR_EAGAIN() || ret == AVERROR_EOF()) {
if (pkt.data() == null && pkt.size() == 0) {
pkt.stream_index(-1);
doAudio = false;
return null;
} else {
readPacket = true;
break;
}
} else if (ret < 0) {
// Ignore errors to emulate the behavior of the old API
// throw new Exception("avcodec_receive_frame() error " + ret + ": Error during audio decoding.");
readPacket = true;
break;
}
long pts = samples_frame.best_effort_timestamp();
AVRational time_base = audio_st.time_base();
timestamp = 1000000L * pts * time_base.num() / time_base.den();
frame.samples = samples_buf;
/* if a frame has been decoded, output it */
if (doProcessing) {
processSamples();
}
done = true;
frame.timestamp = timestamp;
frame.keyFrame = (samples_frame.flags() & AVFrame.AV_FRAME_FLAG_KEY) != 0;
frame.type = Frame.Type.AUDIO;
}
} else if (readPacket && doData
&& frame.streamIndex > -1 && frame.streamIndex < streams.length
&& streams[frame.streamIndex] != AVMEDIA_TYPE_VIDEO && streams[frame.streamIndex] != AVMEDIA_TYPE_AUDIO) {
// Export the stream byte data for non audio / video frames
frame.data = pkt.data().position(0).capacity(pkt.size()).asByteBuffer();
frame.opaque = pkt;
done = true;
switch (streams[frame.streamIndex]) {
case AVMEDIA_TYPE_DATA: frame.type = Frame.Type.DATA; break;
case AVMEDIA_TYPE_SUBTITLE: frame.type = Frame.Type.SUBTITLE; break;
case AVMEDIA_TYPE_ATTACHMENT: frame.type = Frame.Type.ATTACHMENT; break;
default: frame.type = null;
}
} else {
// Current packet is not needed (different stream index required)
readPacket = true;
}
}
return frame;
}
}
public synchronized AVPacket grabPacket() throws Exception {
if (oc == null || oc.isNull()) {
throw new Exception("Could not grab: No AVFormatContext. (Has start() been called?)");
}
if (!started) {
throw new Exception("start() was not called successfully!");
}
// Return the next frame of a stream.
if (av_read_frame(oc, pkt) < 0) {
return null;
}
return pkt;
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/FFmpegFrameRecorder.java
================================================
/*
* Copyright (C) 2009-2025 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*
* Based on the output-example.c file included in FFmpeg 0.6.5
* as well as on the decoding_encoding.c file included in FFmpeg 0.11.1,
* and on the encode_video.c file included in FFmpeg 4.4,
* which are covered by the following copyright notice:
*
* Libavformat API example: Output a media file in any supported
* libavformat format. The default codecs are used.
*
* Copyright (c) 2001,2003 Fabrice Bellard
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/
package org.bytedeco.javacv;
import java.io.File;
import java.io.IOException;
import java.io.OutputStream;
import java.net.URL;
import java.nio.Buffer;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.DoubleBuffer;
import java.nio.FloatBuffer;
import java.nio.IntBuffer;
import java.nio.ShortBuffer;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
import java.util.Map.Entry;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.DoublePointer;
import org.bytedeco.javacpp.FloatPointer;
import org.bytedeco.javacpp.IntPointer;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.javacpp.Pointer;
import org.bytedeco.javacpp.PointerScope;
import org.bytedeco.javacpp.PointerPointer;
import org.bytedeco.javacpp.ShortPointer;
import org.bytedeco.ffmpeg.avcodec.*;
import org.bytedeco.ffmpeg.avformat.*;
import org.bytedeco.ffmpeg.avutil.*;
import org.bytedeco.ffmpeg.swresample.*;
import org.bytedeco.ffmpeg.swscale.*;
import static org.bytedeco.ffmpeg.global.avcodec.*;
import static org.bytedeco.ffmpeg.global.avdevice.*;
import static org.bytedeco.ffmpeg.global.avformat.*;
import static org.bytedeco.ffmpeg.global.avutil.*;
import static org.bytedeco.ffmpeg.global.swresample.*;
import static org.bytedeco.ffmpeg.global.swscale.*;
/**
*
* @author Samuel Audet
*/
public class FFmpegFrameRecorder extends FrameRecorder {
public static class Exception extends FrameRecorder.Exception {
public Exception(String message) { super(message + " (For more details, make sure FFmpegLogCallback.set() has been called.)"); }
public Exception(String message, Throwable cause) { super(message, cause); }
}
public static FFmpegFrameRecorder createDefault(File f, int w, int h) throws Exception { return new FFmpegFrameRecorder(f, w, h); }
public static FFmpegFrameRecorder createDefault(String f, int w, int h) throws Exception { return new FFmpegFrameRecorder(f, w, h); }
private static Exception loadingException = null;
public static void tryLoad() throws Exception {
if (loadingException != null) {
throw loadingException;
} else {
try {
Loader.load(org.bytedeco.ffmpeg.global.avutil.class);
Loader.load(org.bytedeco.ffmpeg.global.swresample.class);
Loader.load(org.bytedeco.ffmpeg.global.avcodec.class);
Loader.load(org.bytedeco.ffmpeg.global.avformat.class);
Loader.load(org.bytedeco.ffmpeg.global.swscale.class);
/* initialize libavcodec, and register all codecs and formats */
av_jni_set_java_vm(Loader.getJavaVM(), null);
// avcodec_register_all();
// av_register_all();
avformat_network_init();
Loader.load(org.bytedeco.ffmpeg.global.avdevice.class);
avdevice_register_all();
} catch (Throwable t) {
if (t instanceof Exception) {
throw loadingException = (Exception)t;
} else {
throw loadingException = new Exception("Failed to load " + FFmpegFrameRecorder.class, t);
}
}
}
}
static {
try {
tryLoad();
// FFmpegLockCallback.init();
} catch (Exception ex) { }
}
public FFmpegFrameRecorder(URL url, int audioChannels) {
this(url.toString(), 0, 0, audioChannels);
}
public FFmpegFrameRecorder(File file, int audioChannels) {
this(file, 0, 0, audioChannels);
}
public FFmpegFrameRecorder(String filename, int audioChannels) {
this(filename, 0, 0, audioChannels);
}
public FFmpegFrameRecorder(URL url, int imageWidth, int imageHeight) {
this(url.toString(), imageWidth, imageHeight, 0);
}
public FFmpegFrameRecorder(File file, int imageWidth, int imageHeight) {
this(file, imageWidth, imageHeight, 0);
}
public FFmpegFrameRecorder(String filename, int imageWidth, int imageHeight) {
this(filename, imageWidth, imageHeight, 0);
}
public FFmpegFrameRecorder(URL url, int imageWidth, int imageHeight, int audioChannels) {
this(url.toString(), imageWidth, imageHeight, audioChannels);
}
public FFmpegFrameRecorder(File file, int imageWidth, int imageHeight, int audioChannels) {
this(file.getAbsolutePath(), imageWidth, imageHeight, audioChannels);
}
public FFmpegFrameRecorder(String filename, int imageWidth, int imageHeight, int audioChannels) {
this.filename = filename;
this.imageWidth = imageWidth;
this.imageHeight = imageHeight;
this.audioChannels = audioChannels;
this.pixelFormat = AV_PIX_FMT_NONE;
this.videoCodec = AV_CODEC_ID_NONE;
this.videoBitrate = 400000;
this.frameRate = 30;
this.sampleFormat = AV_SAMPLE_FMT_NONE;
this.audioCodec = AV_CODEC_ID_NONE;
this.audioBitrate = 64000;
this.sampleRate = 44100;
this.interleaved = true;
}
public FFmpegFrameRecorder(OutputStream outputStream, int audioChannels) {
this(outputStream.toString(), audioChannels);
this.outputStream = outputStream;
this.closeOutputStream = true;
}
public FFmpegFrameRecorder(OutputStream outputStream, int imageWidth, int imageHeight) {
this(outputStream.toString(), imageWidth, imageHeight);
this.outputStream = outputStream;
this.closeOutputStream = true;
}
public FFmpegFrameRecorder(OutputStream outputStream, int imageWidth, int imageHeight, int audioChannels) {
this(outputStream.toString(), imageWidth, imageHeight, audioChannels);
this.outputStream = outputStream;
this.closeOutputStream = true;
}
public void release() throws Exception {
synchronized (org.bytedeco.ffmpeg.global.avcodec.class) {
releaseUnsafe();
}
}
public synchronized void releaseUnsafe() throws Exception {
started = false;
if (display_matrix != null) {
display_matrix.releaseReference();
}
if (plane_ptr != null && plane_ptr2 != null) {
plane_ptr.releaseReference();
plane_ptr2.releaseReference();
plane_ptr = plane_ptr2 = null;
}
if (video_pkt != null && audio_pkt != null) {
video_pkt.releaseReference();
audio_pkt.releaseReference();
video_pkt = audio_pkt = null;
}
if (default_layout != null) {
default_layout.releaseReference();
default_layout = null;
}
/* close each codec */
if (video_c != null) {
avcodec_free_context(video_c);
video_c = null;
}
if (audio_c != null) {
avcodec_free_context(audio_c);
audio_c = null;
}
if (picture_buf != null) {
av_free(picture_buf);
picture_buf = null;
}
if (picture != null) {
av_frame_free(picture);
picture = null;
}
if (tmp_picture != null) {
av_frame_free(tmp_picture);
tmp_picture = null;
}
if (video_outbuf != null) {
av_free(video_outbuf);
video_outbuf = null;
}
if (frame != null) {
av_frame_free(frame);
frame = null;
}
if (samples_in != null) {
for (int i = 0; i < samples_in.length; i++) {
if (samples_in[i] != null) {
samples_in[i].releaseReference();
}
}
samples_in = null;
}
if (samples_out != null) {
for (int i = 0; i < samples_out.length; i++) {
av_free(samples_out[i].position(0));
}
samples_out = null;
}
if (audio_outbuf != null) {
av_free(audio_outbuf);
audio_outbuf = null;
}
if (video_st != null && video_st.metadata() != null) {
av_dict_free(video_st.metadata());
video_st.metadata(null);
}
if (audio_st != null && audio_st.metadata() != null) {
av_dict_free(audio_st.metadata());
audio_st.metadata(null);
}
video_st = null;
audio_st = null;
filename = null;
AVFormatContext outputStreamKey = oc;
if (oc != null && !oc.isNull()) {
if (outputStream == null && (oformat.flags() & AVFMT_NOFILE) == 0) {
/* close the output file */
avio_close(oc.pb());
}
/* free the streams */
avformat_free_context(oc);
oc = null;
}
if (img_convert_ctx != null) {
sws_freeContext(img_convert_ctx);
img_convert_ctx = null;
}
if (samples_convert_ctx != null) {
swr_free(samples_convert_ctx);
samples_convert_ctx.releaseReference();
samples_convert_ctx = null;
}
if (outputStream != null) {
try {
if (closeOutputStream) {
outputStream.close();
}
} catch (IOException ex) {
throw new Exception("Error on OutputStream.close(): ", ex);
} finally {
outputStream = null;
outputStreams.remove(outputStreamKey);
if (avio != null) {
if (avio.buffer() != null) {
av_free(avio.buffer());
avio.buffer(null);
}
av_free(avio);
avio = null;
}
}
}
Pointer.trimMemory();
}
@Override protected void finalize() throws Throwable {
super.finalize();
release();
}
static Map outputStreams = Collections.synchronizedMap(new HashMap());
static class WriteCallback extends Write_packet_Pointer_BytePointer_int {
@Override public int call(Pointer opaque, BytePointer buf, int buf_size) {
try {
byte[] b = new byte[buf_size];
OutputStream os = outputStreams.get(opaque);
buf.get(b, 0, buf_size);
os.write(b, 0, buf_size);
return buf_size;
}
catch (Throwable t) {
System.err.println("Error on OutputStream.write(): " + t);
return -1;
}
}
}
static WriteCallback writeCallback = new WriteCallback().retainReference();
static class SeekCallback extends Seek_Pointer_long_int {
@Override public long call(Pointer opaque, long offset, int whence) {
try {
OutputStream os = outputStreams.get(opaque);
((Seekable)os).seek(offset, whence);
return 0;
}
catch (Throwable t) {
System.err.println("Error on OutputStream.seek(): " + t);
return -1;
}
}
}
static SeekCallback seekCallback = new SeekCallback().retainReference();
private OutputStream outputStream;
private boolean closeOutputStream;
private AVIOContext avio;
private String filename;
private AVFrame picture, tmp_picture;
private BytePointer picture_buf;
private BytePointer video_outbuf;
private int video_outbuf_size;
private AVFrame frame;
private Pointer[] samples_in;
private BytePointer[] samples_out;
private BytePointer audio_outbuf;
private int audio_outbuf_size;
private int audio_input_frame_size;
private AVOutputFormat oformat;
private AVFormatContext oc;
private AVCodec video_codec, audio_codec;
private AVCodecContext video_c, audio_c;
private AVStream video_st, audio_st;
private SwsContext img_convert_ctx;
private SwrContext samples_convert_ctx;
private int samples_channels, samples_format, samples_rate;
private PointerPointer plane_ptr, plane_ptr2;
private AVPacket video_pkt, audio_pkt;
private int[] got_video_packet, got_audio_packet;
private boolean wrote_samples = false;
private AVFormatContext ifmt_ctx;
private IntPointer display_matrix;
private AVChannelLayout default_layout;
private volatile boolean started = false;
public boolean isCloseOutputStream() {
return closeOutputStream;
}
public void setCloseOutputStream(boolean closeOutputStream) {
this.closeOutputStream = closeOutputStream;
}
/** Sets the rotation in degrees to the side data of the video stream. */
public void setDisplayRotation(double angle) {
if (display_matrix == null) {
display_matrix = new IntPointer(9).retainReference();
}
av_display_rotation_set(display_matrix, -angle);
setVideoSideData("Display Matrix", display_matrix.asByteBuffer());
}
@Override public int getFrameNumber() {
return picture == null ? super.getFrameNumber() : (int)picture.pts();
}
@Override public void setFrameNumber(int frameNumber) {
if (picture == null) { super.setFrameNumber(frameNumber); } else { picture.pts(frameNumber); }
}
/** Returns best guess for timestamp in microseconds... */
@Override public long getTimestamp() {
return Math.round(getFrameNumber() * 1000000L / getFrameRate());
}
@Override public void setTimestamp(long timestamp) {
setFrameNumber((int)Math.round(timestamp * getFrameRate() / 1000000L));
}
public void start(AVFormatContext inputFormatContext) throws Exception {
this.ifmt_ctx = inputFormatContext;
start();
}
@Override public void start() throws Exception {
synchronized (org.bytedeco.ffmpeg.global.avcodec.class) {
startUnsafe();
}
}
public synchronized void startUnsafe() throws Exception {
try (PointerScope scope = new PointerScope()) {
if (oc != null && !oc.isNull()) {
throw new Exception("start() has already been called: Call stop() before calling start() again.");
}
int ret;
picture = null;
tmp_picture = null;
picture_buf = null;
frame = null;
video_outbuf = null;
audio_outbuf = null;
oc = new AVFormatContext(null);
video_c = null;
audio_c = null;
video_st = null;
audio_st = null;
plane_ptr = new PointerPointer(AVFrame.AV_NUM_DATA_POINTERS).retainReference();
plane_ptr2 = new PointerPointer(AVFrame.AV_NUM_DATA_POINTERS).retainReference();
video_pkt = new AVPacket().retainReference();
audio_pkt = new AVPacket().retainReference();
wrote_samples = false;
got_video_packet = new int[1];
got_audio_packet = new int[1];
default_layout = new AVChannelLayout().retainReference();
/* auto detect the output format from the name. */
String format_name = format == null || format.length() == 0 ? null : format;
if ((oformat = av_guess_format(format_name, filename, null)) == null) {
int proto = filename.indexOf("://");
if (proto > 0) {
format_name = filename.substring(0, proto);
}
if ((oformat = av_guess_format(format_name, filename, null)) == null) {
throw new Exception("av_guess_format() error: Could not guess output format for \"" + filename + "\" and " + format + " format.");
}
}
format_name = oformat.name().getString();
/* allocate the output media context */
if (avformat_alloc_output_context2(oc, null, format_name, filename) < 0) {
throw new Exception("avformat_alloc_context2() error:\tCould not allocate format context");
}
if (outputStream != null) {
avio = avio_alloc_context(new BytePointer(av_malloc(4096)), 4096, 1, oc, null, writeCallback, outputStream instanceof Seekable ? seekCallback : null);
oc.pb(avio);
filename = outputStream.toString();
outputStreams.put(oc, outputStream);
}
oc.oformat(oformat);
oc.url(new BytePointer(av_malloc(filename.getBytes().length + 1)).putString(filename));
oc.max_delay(maxDelay);
/* add the audio and video streams using the format codecs
and initialize the codecs */
AVStream inpVideoStream = null, inpAudioStream = null;
if (ifmt_ctx != null) {
// get input video and audio stream indices from ifmt_ctx
for (int idx = 0; idx < ifmt_ctx.nb_streams(); idx++) {
AVStream inputStream = ifmt_ctx.streams(idx);
if (inputStream.codecpar().codec_type() == AVMEDIA_TYPE_VIDEO) {
inpVideoStream = inputStream;
videoCodec = inpVideoStream.codecpar().codec_id();
if (inpVideoStream.r_frame_rate().num() != AV_NOPTS_VALUE && inpVideoStream.r_frame_rate().den() != 0) {
frameRate = (inpVideoStream.r_frame_rate().num())*1.0d / (inpVideoStream.r_frame_rate().den());
}
} else if (inputStream.codecpar().codec_type() == AVMEDIA_TYPE_AUDIO) {
inpAudioStream = inputStream;
audioCodec = inpAudioStream.codecpar().codec_id();
}
}
}
if (imageWidth > 0 && imageHeight > 0) {
if (videoCodec == AV_CODEC_ID_NONE) {
videoCodec = oformat.video_codec();
}
// if (videoCodec != AV_CODEC_ID_NONE) {
// oformat.video_codec(videoCodec);
// } else if ("flv".equals(format_name)) {
// oformat.video_codec(AV_CODEC_ID_FLV1);
// } else if ("mp4".equals(format_name)) {
// oformat.video_codec(AV_CODEC_ID_MPEG4);
// } else if ("3gp".equals(format_name)) {
// oformat.video_codec(AV_CODEC_ID_H263);
// } else if ("avi".equals(format_name)) {
// oformat.video_codec(AV_CODEC_ID_HUFFYUV);
// }
/* find the video encoder */
if ((video_codec = avcodec_find_encoder_by_name(videoCodecName)) == null &&
(video_codec = avcodec_find_encoder(videoCodec)) == null) {
releaseUnsafe();
throw new Exception("avcodec_find_encoder() error: Video codec not found.");
}
// oformat.video_codec(video_codec.id());
AVRational frame_rate = av_d2q(frameRate, 1001000);
AVRational supported_framerates = video_codec.supported_framerates();
if (supported_framerates != null) {
int idx = av_find_nearest_q_idx(frame_rate, supported_framerates);
frame_rate = supported_framerates.position(idx);
}
/* add a video output stream */
if ((video_st = avformat_new_stream(oc, null)) == null) {
releaseUnsafe();
throw new Exception("avformat_new_stream() error: Could not allocate video stream.");
}
if ((video_c = avcodec_alloc_context3(video_codec)) == null) {
releaseUnsafe();
throw new Exception("avcodec_alloc_context3() error: Could not allocate video encoding context.");
}
if (inpVideoStream != null) {
if ((ret = avcodec_parameters_copy(video_st.codecpar(), inpVideoStream.codecpar())) < 0) {
releaseUnsafe();
throw new Exception("avcodec_parameters_copy() error " + ret + ": Failed to copy video stream codec parameters from input to output");
}
videoBitrate = (int)inpVideoStream.codecpar().bit_rate();
pixelFormat = inpVideoStream.codecpar().format();
aspectRatio = inpVideoStream.codecpar().sample_aspect_ratio().num()*1.0d/ inpVideoStream.codecpar().sample_aspect_ratio().den();
// videoQuality = inpVideoStream.codecpar().global_quality();
video_c.codec_tag(0);
}
video_c.codec_id(video_codec.id());
video_c.codec_type(AVMEDIA_TYPE_VIDEO);
/* put sample parameters */
video_c.bit_rate(videoBitrate);
/* resolution must be a multiple of two. Scale height to maintain the aspect ratio. */
if (imageWidth % 2 == 1) {
int roundedWidth = imageWidth + 1;
imageHeight = (roundedWidth * imageHeight + imageWidth / 2) / imageWidth;
imageWidth = roundedWidth;
}
video_c.width(imageWidth);
video_c.height(imageHeight);
if (aspectRatio > 0) {
AVRational r = av_d2q(aspectRatio, 255);
video_c.sample_aspect_ratio(r);
video_st.sample_aspect_ratio(r);
}
/* time base: this is the fundamental unit of time (in seconds) in terms
of which frame timestamps are represented. for fixed-fps content,
timebase should be 1/framerate and timestamp increments should be
identically 1. */
AVRational time_base = av_inv_q(frame_rate);
video_c.time_base(time_base);
video_st.time_base(time_base);
video_st.avg_frame_rate(frame_rate);
// video_st.codec().time_base(time_base); // "deprecated", but this is actually required
if (gopSize >= 0) {
video_c.gop_size(gopSize); /* emit one intra frame every gopSize frames at most */
}
if (videoProfile >= 0) {
video_c.profile(videoProfile);
}
if (videoQuality >= 0) {
video_c.flags(video_c.flags() | AV_CODEC_FLAG_QSCALE);
video_c.global_quality((int)Math.round(FF_QP2LAMBDA * videoQuality));
}
if (pixelFormat != AV_PIX_FMT_NONE) {
video_c.pix_fmt(pixelFormat);
} else if (video_c.codec_id() == AV_CODEC_ID_RAWVIDEO || video_c.codec_id() == AV_CODEC_ID_PNG ||
video_c.codec_id() == AV_CODEC_ID_HUFFYUV || video_c.codec_id() == AV_CODEC_ID_FFV1) {
video_c.pix_fmt(AV_PIX_FMT_RGB32); // appropriate for common lossless formats
} else if (video_c.codec_id() == AV_CODEC_ID_JPEGLS) {
video_c.pix_fmt(AV_PIX_FMT_BGR24);
} else if (video_c.codec_id() == AV_CODEC_ID_MJPEG || video_c.codec_id() == AV_CODEC_ID_MJPEGB) {
video_c.pix_fmt(AV_PIX_FMT_YUVJ420P);
} else {
video_c.pix_fmt(AV_PIX_FMT_YUV420P); // lossy, but works with about everything
}
if (video_c.codec_id() == AV_CODEC_ID_MPEG2VIDEO) {
/* just for testing, we also add B frames */
video_c.max_b_frames(2);
} else if (video_c.codec_id() == AV_CODEC_ID_MPEG1VIDEO) {
/* Needed to avoid using macroblocks in which some coeffs overflow.
This does not happen with normal video, it just happens here as
the motion of the chroma plane does not match the luma plane. */
video_c.mb_decision(2);
} else if (video_c.codec_id() == AV_CODEC_ID_H263) {
// H.263 does not support any other resolution than the following
if (imageWidth <= 128 && imageHeight <= 96) {
video_c.width(128).height(96);
} else if (imageWidth <= 176 && imageHeight <= 144) {
video_c.width(176).height(144);
} else if (imageWidth <= 352 && imageHeight <= 288) {
video_c.width(352).height(288);
} else if (imageWidth <= 704 && imageHeight <= 576) {
video_c.width(704).height(576);
} else {
video_c.width(1408).height(1152);
}
} else if (video_c.codec_id() == AV_CODEC_ID_H264) {
// default to constrained baseline to produce content that plays back on anything,
// without any significant tradeoffs for most use cases
if (videoProfile < 0) {
video_c.profile(AV_PROFILE_H264_CONSTRAINED_BASELINE);
}
}
// some formats want stream headers to be separate
if ((oformat.flags() & AVFMT_GLOBALHEADER) != 0) {
video_c.flags(video_c.flags() | AV_CODEC_FLAG_GLOBAL_HEADER);
}
if ((video_codec.capabilities() & AV_CODEC_CAP_EXPERIMENTAL) != 0) {
video_c.strict_std_compliance(FF_COMPLIANCE_EXPERIMENTAL);
}
if (maxBFrames >= 0) {
video_c.max_b_frames(maxBFrames);
video_c.has_b_frames(maxBFrames == 0 ? 0 : 1);
}
if (trellis >= 0) {
video_c.trellis(trellis);
}
}
/*
* add an audio output stream
*/
if (audioChannels > 0 && audioBitrate > 0 && sampleRate > 0) {
if (audioCodec == AV_CODEC_ID_NONE) {
audioCodec = oformat.audio_codec();
}
// if (audioCodec != AV_CODEC_ID_NONE) {
// oformat.audio_codec(audioCodec);
// } else if ("flv".equals(format_name) || "mp4".equals(format_name) || "3gp".equals(format_name)) {
// oformat.audio_codec(AV_CODEC_ID_AAC);
// } else if ("avi".equals(format_name)) {
// oformat.audio_codec(AV_CODEC_ID_PCM_S16LE);
// }
/* find the audio encoder */
if ((audio_codec = avcodec_find_encoder_by_name(audioCodecName)) == null &&
(audio_codec = avcodec_find_encoder(audioCodec)) == null) {
releaseUnsafe();
throw new Exception("avcodec_find_encoder() error: Audio codec not found.");
}
// oformat.audio_codec(audio_codec.id());
AVRational sample_rate = av_d2q(sampleRate, 1001000);
if ((audio_st = avformat_new_stream(oc, null)) == null) {
releaseUnsafe();
throw new Exception("avformat_new_stream() error: Could not allocate audio stream.");
}
if ((audio_c = avcodec_alloc_context3(audio_codec)) == null) {
releaseUnsafe();
throw new Exception("avcodec_alloc_context3() error: Could not allocate audio encoding context.");
}
if (inpAudioStream != null && audioChannels > 0) {
if ((ret = avcodec_parameters_copy(audio_st.codecpar(), inpAudioStream.codecpar())) < 0) {
throw new Exception("avcodec_parameters_copy() error " + ret + ": Failed to copy audio stream codec parameters from input to output");
}
audioBitrate = (int) inpAudioStream.codecpar().bit_rate();
sampleRate = inpAudioStream.codecpar().sample_rate();
audioChannels = inpAudioStream.codecpar().ch_layout().nb_channels();
sampleFormat = inpAudioStream.codecpar().format();
// audioQuality = inpAudioStream.codecpar().global_quality();
audio_c.codec_tag(0);
// audio_st.pts(inpAudioStream.pts());
audio_st.duration(inpAudioStream.duration());
audio_st.time_base().num(inpAudioStream.time_base().num());
audio_st.time_base().den(inpAudioStream.time_base().den());
}
audio_c.codec_id(audio_codec.id());
audio_c.codec_type(AVMEDIA_TYPE_AUDIO);
/* put sample parameters */
audio_c.bit_rate(audioBitrate);
audio_c.sample_rate(sampleRate);
av_channel_layout_default(default_layout, audioChannels);
audio_c.ch_layout(default_layout);
if (sampleFormat != AV_SAMPLE_FMT_NONE) {
audio_c.sample_fmt(sampleFormat);
} else {
// use AV_SAMPLE_FMT_S16 by default, if available
audio_c.sample_fmt(AV_SAMPLE_FMT_FLTP);
IntPointer formats = audio_c.codec().sample_fmts();
for (int i = 0; formats.get(i) != -1; i++) {
if (formats.get(i) == AV_SAMPLE_FMT_S16) {
audio_c.sample_fmt(AV_SAMPLE_FMT_S16);
break;
}
}
}
AVRational time_base = av_inv_q(sample_rate);
audio_c.time_base(time_base);
audio_st.time_base(time_base);
// audio_st.codec().time_base(time_base); // "deprecated", but this is actually required
switch (audio_c.sample_fmt()) {
case AV_SAMPLE_FMT_U8:
case AV_SAMPLE_FMT_U8P: audio_c.bits_per_raw_sample(8); break;
case AV_SAMPLE_FMT_S16:
case AV_SAMPLE_FMT_S16P: audio_c.bits_per_raw_sample(16); break;
case AV_SAMPLE_FMT_S32:
case AV_SAMPLE_FMT_S32P: audio_c.bits_per_raw_sample(32); break;
case AV_SAMPLE_FMT_FLT:
case AV_SAMPLE_FMT_FLTP: audio_c.bits_per_raw_sample(32); break;
case AV_SAMPLE_FMT_DBL:
case AV_SAMPLE_FMT_DBLP: audio_c.bits_per_raw_sample(64); break;
default: assert false;
}
if (audioQuality >= 0) {
audio_c.flags(audio_c.flags() | AV_CODEC_FLAG_QSCALE);
audio_c.global_quality((int)Math.round(FF_QP2LAMBDA * audioQuality));
}
// some formats want stream headers to be separate
if ((oformat.flags() & AVFMT_GLOBALHEADER) != 0) {
audio_c.flags(audio_c.flags() | AV_CODEC_FLAG_GLOBAL_HEADER);
}
if ((audio_codec.capabilities() & AV_CODEC_CAP_EXPERIMENTAL) != 0) {
audio_c.strict_std_compliance(FF_COMPLIANCE_EXPERIMENTAL);
}
}
/* now that all the parameters are set, we can open the audio and
video codecs and allocate the necessary encode buffers */
if (video_st != null && inpVideoStream == null) {
AVDictionary options = new AVDictionary(null);
if (videoQuality >= 0) {
av_dict_set(options, "crf", "" + videoQuality, 0);
}
for (Entry e : videoOptions.entrySet()) {
av_dict_set(options, e.getKey(), e.getValue(), 0);
}
// Enable multithreading when available
video_c.thread_count(0);
/* open the codec */
if ((ret = avcodec_open2(video_c, video_codec, options)) < 0) {
releaseUnsafe();
av_dict_free(options);
throw new Exception("avcodec_open2() error " + ret + ": Could not open video codec.");
}
FFmpegLogCallback.logRejectedOptions(options, "avcodec_open2");
av_dict_free(options);
video_outbuf = null;
// if ((oformat.flags() & AVFMT_RAWPICTURE) == 0) {
// /* allocate output buffer */
// /* XXX: API change will be done */
// /* buffers passed into lav* can be allocated any way you prefer,
// as long as they're aligned enough for the architecture, and
// they're freed appropriately (such as using av_free for buffers
// allocated with av_malloc) */
// video_outbuf_size = Math.max(256 * 1024, 8 * video_c.width() * video_c.height()); // a la ffmpeg.c
// video_outbuf = new BytePointer(av_malloc(video_outbuf_size));
// }
/* allocate the encoded raw picture */
if ((picture = av_frame_alloc()) == null) {
releaseUnsafe();
throw new Exception("av_frame_alloc() error: Could not allocate picture.");
}
picture.pts(0); // magic required by libx264
int size = av_image_get_buffer_size(video_c.pix_fmt(), video_c.width(), video_c.height(), 1);
if ((picture_buf = new BytePointer(av_malloc(size))).isNull()) {
releaseUnsafe();
throw new Exception("av_malloc() error: Could not allocate picture buffer.");
}
/* if the output format is not equal to the image format, then a temporary
picture is needed too. It is then converted to the required output format */
if ((tmp_picture = av_frame_alloc()) == null) {
releaseUnsafe();
throw new Exception("av_frame_alloc() error: Could not allocate temporary picture.");
}
/* copy the stream parameters to the muxer */
if ((ret = avcodec_parameters_from_context(video_st.codecpar(), video_c)) < 0) {
releaseUnsafe();
throw new Exception("avcodec_parameters_from_context() error " + ret + ": Could not copy the video stream parameters.");
}
AVDictionary metadata = new AVDictionary(null);
for (Entry e : videoMetadata.entrySet()) {
av_dict_set(metadata, new BytePointer(e.getKey(), charset), new BytePointer(e.getValue(), charset), 0);
}
video_st.metadata(metadata);
for (Entry e : videoSideData.entrySet()) {
int type = -1;
for (int i = 0; i < AV_PKT_DATA_NB; i++) {
BytePointer s = av_packet_side_data_name(i);
if (s != null && !s.isNull() && e.getKey().equals(s.getString())) {
type = i;
break;
}
}
Pointer p = new Pointer(e.getValue());
AVPacketSideData coded_side_data = video_st.codecpar().coded_side_data();
int[] nb_coded_side_data = {video_st.codecpar().nb_coded_side_data()};
AVPacketSideData b = av_packet_side_data_new(coded_side_data, nb_coded_side_data, type, p.capacity(), 0);
video_st.codecpar().coded_side_data(coded_side_data);
video_st.codecpar().nb_coded_side_data(nb_coded_side_data[0]);
if (b != null && !b.isNull() && b.data() != null && !b.data().isNull()) {
b.data().capacity(p.capacity()).put(p);
}
}
}
if (audio_st != null && inpAudioStream == null) {
AVDictionary options = new AVDictionary(null);
if (audioQuality >= 0) {
av_dict_set(options, "crf", "" + audioQuality, 0);
}
for (Entry e : audioOptions.entrySet()) {
av_dict_set(options, e.getKey(), e.getValue(), 0);
}
// Enable multithreading when available
audio_c.thread_count(0);
/* open the codec */
if ((ret = avcodec_open2(audio_c, audio_codec, options)) < 0) {
releaseUnsafe();
av_dict_free(options);
throw new Exception("avcodec_open2() error " + ret + ": Could not open audio codec.");
}
FFmpegLogCallback.logRejectedOptions(options, "avcodec_open2");
av_dict_free(options);
audio_outbuf_size = 256 * 1024;
audio_outbuf = new BytePointer(av_malloc(audio_outbuf_size));
/* ugly hack for PCM codecs (will be removed ASAP with new PCM
support to compute the input frame size in samples */
if (audio_c.frame_size() <= 1) {
audio_outbuf_size = 16 * 1024; // AV_INPUT_BUFFER_MIN_SIZE;
audio_input_frame_size = audio_outbuf_size / audio_c.ch_layout().nb_channels();
switch (audio_c.codec_id()) {
case AV_CODEC_ID_PCM_S16LE:
case AV_CODEC_ID_PCM_S16BE:
case AV_CODEC_ID_PCM_U16LE:
case AV_CODEC_ID_PCM_U16BE:
audio_input_frame_size >>= 1;
break;
default:
break;
}
} else {
audio_input_frame_size = audio_c.frame_size();
}
//int bufferSize = audio_input_frame_size * audio_c.bits_per_raw_sample()/8 * audio_c.ch_layout().nb_channels();
int planes = av_sample_fmt_is_planar(audio_c.sample_fmt()) != 0 ? (int)audio_c.ch_layout().nb_channels() : 1;
int data_size = av_samples_get_buffer_size((IntPointer)null, audio_c.ch_layout().nb_channels(),
audio_input_frame_size, audio_c.sample_fmt(), 1) / planes;
samples_out = new BytePointer[planes];
for (int i = 0; i < samples_out.length; i++) {
samples_out[i] = new BytePointer(av_malloc(data_size)).capacity(data_size);
}
samples_in = new Pointer[AVFrame.AV_NUM_DATA_POINTERS];
/* allocate the audio frame */
if ((frame = av_frame_alloc()) == null) {
releaseUnsafe();
throw new Exception("av_frame_alloc() error: Could not allocate audio frame.");
}
frame.pts(0); // magic required by libvorbis and webm
/* copy the stream parameters to the muxer */
if ((ret = avcodec_parameters_from_context(audio_st.codecpar(), audio_c)) < 0) {
releaseUnsafe();
throw new Exception("avcodec_parameters_from_context() error " + ret + ": Could not copy the audio stream parameters.");
}
AVDictionary metadata = new AVDictionary(null);
for (Entry e : audioMetadata.entrySet()) {
av_dict_set(metadata, new BytePointer(e.getKey(), charset), new BytePointer(e.getValue(), charset), 0);
}
audio_st.metadata(metadata);
for (Entry e : audioSideData.entrySet()) {
int type = -1;
for (int i = 0; i < AV_PKT_DATA_NB; i++) {
BytePointer s = av_packet_side_data_name(i);
if (s != null && !s.isNull() && e.getKey().equals(s.getString())) {
type = i;
break;
}
}
Pointer p = new Pointer(e.getValue());
AVPacketSideData coded_side_data = audio_st.codecpar().coded_side_data();
int[] nb_coded_side_data = {audio_st.codecpar().nb_coded_side_data()};
AVPacketSideData b = av_packet_side_data_new(coded_side_data, nb_coded_side_data, type, p.capacity(), 0);
audio_st.codecpar().coded_side_data(coded_side_data);
audio_st.codecpar().nb_coded_side_data(nb_coded_side_data[0]);
if (b != null && !b.isNull() && b.data() != null && !b.data().isNull()) {
b.data().capacity(p.capacity()).put(p);
}
}
}
AVDictionary options = new AVDictionary(null);
for (Entry e : this.options.entrySet()) {
av_dict_set(options, e.getKey(), e.getValue(), 0);
}
/* open the output file, if needed */
if (outputStream == null && (oformat.flags() & AVFMT_NOFILE) == 0) {
AVIOContext pb = new AVIOContext(null);
if ((ret = avio_open2(pb, filename, AVIO_FLAG_WRITE, null, options)) < 0) {
String errorMsg = "avio_open2 error() error " + ret + ": Could not open '" + filename + "'";
releaseUnsafe();
av_dict_free(options);
throw new Exception(errorMsg);
}
FFmpegLogCallback.logRejectedOptions(options, "avio_open2");
oc.pb(pb);
}
AVDictionary metadata = new AVDictionary(null);
for (Entry e : this.metadata.entrySet()) {
av_dict_set(metadata, new BytePointer(e.getKey(), charset), new BytePointer(e.getValue(), charset), 0);
}
/* write the stream header, if any */
if ((ret = avformat_write_header(oc.metadata(metadata), options)) < 0) {
String errorMsg = "avformat_write_header error() error " + ret + ": Could not write header to '" + filename + "'";
releaseUnsafe();
av_dict_free(options);
throw new Exception(errorMsg);
}
FFmpegLogCallback.logRejectedOptions(options, "avformat_write_header");
av_dict_free(options);
if (av_log_get_level() >= AV_LOG_INFO) {
av_dump_format(oc, 0, filename, 1);
}
started = true;
}
}
public synchronized void flush() throws Exception {
synchronized (oc) {
/* flush all the buffers */
while (video_st != null && ifmt_ctx == null && recordImage(0, 0, 0, 0, 0, AV_PIX_FMT_NONE, (Buffer[])null));
while (audio_st != null && ifmt_ctx == null && recordSamples(0, 0, (Buffer[])null));
if (interleaved && (video_st != null || audio_st != null)) {
av_interleaved_write_frame(oc, null);
} else {
av_write_frame(oc, null);
}
}
}
public void stop() throws Exception {
if (oc != null) {
try {
flush();
/* write the trailer, if any */
av_write_trailer(oc);
} finally {
release();
}
}
}
@Override public void record(Frame frame) throws Exception {
record(frame, frame != null && frame.opaque instanceof AVFrame ? ((AVFrame)frame.opaque).format() : AV_PIX_FMT_NONE);
}
public synchronized void record(Frame frame, int pixelFormat) throws Exception {
if (frame == null || (frame.image == null && frame.samples == null && frame.data == null)) {
recordImage(0, 0, 0, 0, 0, pixelFormat, (Buffer[])null);
} else {
if (frame.image != null) {
frame.keyFrame = recordImage(frame.imageWidth, frame.imageHeight, frame.imageDepth,
frame.imageChannels, frame.imageStride, pixelFormat, frame.image);
}
if (frame.samples != null) {
frame.keyFrame = recordSamples(frame.sampleRate, frame.audioChannels, frame.samples);
}
}
}
public synchronized boolean recordImage(int width, int height, int depth, int channels, int stride, int pixelFormat, Buffer ... image) throws Exception {
try (PointerScope scope = new PointerScope()) {
if (video_st == null) {
throw new Exception("No video output stream (Is imageWidth > 0 && imageHeight > 0 and has start() been called?)");
}
if (!started) {
throw new Exception("start() was not called successfully!");
}
int ret;
if (image == null || image.length == 0) {
/* no more frame to compress. The codec has a latency of a few
frames if using B frames, so we get the last frames by
passing the same picture again */
} else {
int step = stride * Math.abs(depth) / 8;
BytePointer data = image[0] instanceof ByteBuffer
? new BytePointer((ByteBuffer)image[0]).position(0)
: new BytePointer(new Pointer(image[0]).position(0));
if (pixelFormat == AV_PIX_FMT_NONE) {
if ((depth == Frame.DEPTH_UBYTE || depth == Frame.DEPTH_BYTE) && channels == 3) {
pixelFormat = AV_PIX_FMT_BGR24;
} else if ((depth == Frame.DEPTH_UBYTE || depth == Frame.DEPTH_BYTE) && channels == 1) {
pixelFormat = AV_PIX_FMT_GRAY8;
} else if ((depth == Frame.DEPTH_USHORT || depth == Frame.DEPTH_SHORT) && channels == 1) {
pixelFormat = ByteOrder.nativeOrder().equals(ByteOrder.BIG_ENDIAN) ?
AV_PIX_FMT_GRAY16BE : AV_PIX_FMT_GRAY16LE;
} else if ((depth == Frame.DEPTH_UBYTE || depth == Frame.DEPTH_BYTE) && channels == 4) {
pixelFormat = AV_PIX_FMT_RGBA;
} else if ((depth == Frame.DEPTH_UBYTE || depth == Frame.DEPTH_BYTE) && channels == 2) {
pixelFormat = AV_PIX_FMT_NV21; // Android's camera capture format
} else {
throw new Exception("Could not guess pixel format of image: depth=" + depth + ", channels=" + channels);
}
}
if (pixelFormat == AV_PIX_FMT_NV21) {
step = width;
}
if (video_c.pix_fmt() != pixelFormat || video_c.width() != width || video_c.height() != height) {
/* convert to the codec pixel format if needed */
img_convert_ctx = sws_getCachedContext(img_convert_ctx, width, height, pixelFormat,
video_c.width(), video_c.height(), video_c.pix_fmt(),
imageScalingFlags != 0 ? imageScalingFlags : SWS_BILINEAR,
null, null, (DoublePointer)null);
if (img_convert_ctx == null) {
throw new Exception("sws_getCachedContext() error: Cannot initialize the conversion context.");
}
av_image_fill_arrays(new PointerPointer(tmp_picture), tmp_picture.linesize(), data, pixelFormat, width, height, 1);
av_image_fill_arrays(new PointerPointer(picture), picture.linesize(), picture_buf, video_c.pix_fmt(), video_c.width(), video_c.height(), 1);
tmp_picture.linesize(0, step);
tmp_picture.format(pixelFormat);
tmp_picture.width(width);
tmp_picture.height(height);
picture.format(video_c.pix_fmt());
picture.width(video_c.width());
picture.height(video_c.height());
sws_scale(img_convert_ctx, new PointerPointer(tmp_picture), tmp_picture.linesize(),
0, height, new PointerPointer(picture), picture.linesize());
} else {
av_image_fill_arrays(new PointerPointer(picture), picture.linesize(), data, pixelFormat, width, height, 1);
picture.linesize(0, step);
picture.format(pixelFormat);
picture.width(width);
picture.height(height);
}
}
// if ((oformat.flags() & AVFMT_RAWPICTURE) != 0) {
// if (image == null || image.length == 0) {
// return false;
// }
// /* raw video case. The API may change slightly in the future for that? */
// av_init_packet(video_pkt);
// video_pkt.flags(video_pkt.flags() | AV_PKT_FLAG_KEY);
// video_pkt.stream_index(video_st.index());
// video_pkt.data(new BytePointer(picture));
// video_pkt.size(Loader.sizeof(AVFrame.class));
// } else {
/* encode the image */
picture.quality(video_c.global_quality());
if ((ret = avcodec_send_frame(video_c, image == null || image.length == 0 ? null : picture)) < 0
&& image != null && image.length != 0) {
throw new Exception("avcodec_send_frame() error " + ret + ": Error sending a video frame for encoding.");
}
picture.pts(picture.pts() + 1); // magic required by libx264
/* if zero size, it means the image was buffered */
got_video_packet[0] = 0;
while (ret >= 0) {
av_new_packet(video_pkt, video_outbuf_size);
ret = avcodec_receive_packet(video_c, video_pkt);
if (ret == AVERROR_EAGAIN() || ret == AVERROR_EOF()) {
av_packet_unref(video_pkt);
break;
} else if (ret < 0) {
av_packet_unref(video_pkt);
throw new Exception("avcodec_receive_packet() error " + ret + ": Error during video encoding.");
}
got_video_packet[0] = 1;
if (video_pkt.pts() != AV_NOPTS_VALUE) {
video_pkt.pts(av_rescale_q(video_pkt.pts(), video_c.time_base(), video_st.time_base()));
}
if (video_pkt.dts() != AV_NOPTS_VALUE) {
video_pkt.dts(av_rescale_q(video_pkt.dts(), video_c.time_base(), video_st.time_base()));
}
video_pkt.stream_index(video_st.index());
/* write the compressed frame in the media file */
writePacket(AVMEDIA_TYPE_VIDEO, video_pkt);
}
// }
return image != null ? (video_pkt.flags() & AV_PKT_FLAG_KEY) != 0 : got_video_packet[0] != 0;
}
}
public boolean recordSamples(Buffer ... samples) throws Exception {
return recordSamples(0, 0, samples);
}
public synchronized boolean recordSamples(int sampleRate, int audioChannels, Buffer ... samples) throws Exception {
try (PointerScope scope = new PointerScope()) {
if (audio_st == null) {
throw new Exception("No audio output stream (Is audioChannels > 0 and has start() been called?)");
}
if (!started) {
throw new Exception("start() was not called successfully!");
}
if (samples == null && samples_convert_ctx == null) {
// We haven't tried to record any samples yet so we don't need to flush.
return false;
}
int ret;
if (sampleRate <= 0) {
sampleRate = audio_c.sample_rate();
}
if (audioChannels <= 0) {
audioChannels = audio_c.ch_layout().nb_channels();
}
int inputSize = samples != null ? samples[0].limit() - samples[0].position() : 0;
int inputFormat = samples_format;
int inputChannels = samples != null && samples.length > 1 ? 1 : audioChannels;
int inputDepth = 0;
int outputFormat = audio_c.sample_fmt();
int outputChannels = samples_out.length > 1 ? 1 : audio_c.ch_layout().nb_channels();
int outputDepth = av_get_bytes_per_sample(outputFormat);
if (samples != null && samples[0] instanceof ByteBuffer) {
inputFormat = samples.length > 1 ? AV_SAMPLE_FMT_U8P : AV_SAMPLE_FMT_U8;
inputDepth = 1;
for (int i = 0; i < samples.length; i++) {
ByteBuffer b = (ByteBuffer)samples[i];
if (samples_in[i] instanceof BytePointer && samples_in[i].capacity() >= inputSize && b.hasArray()) {
((BytePointer)samples_in[i]).position(0).put(b.array(), b.position(), inputSize);
} else {
if (samples_in[i] != null) {
samples_in[i].releaseReference();
}
samples_in[i] = new BytePointer(b).retainReference();
}
}
} else if (samples != null && samples[0] instanceof ShortBuffer) {
inputFormat = samples.length > 1 ? AV_SAMPLE_FMT_S16P : AV_SAMPLE_FMT_S16;
inputDepth = 2;
for (int i = 0; i < samples.length; i++) {
ShortBuffer b = (ShortBuffer)samples[i];
if (samples_in[i] instanceof ShortPointer && samples_in[i].capacity() >= inputSize && b.hasArray()) {
((ShortPointer)samples_in[i]).position(0).put(b.array(), samples[i].position(), inputSize);
} else {
if (samples_in[i] != null) {
samples_in[i].releaseReference();
}
samples_in[i] = new ShortPointer(b).retainReference();
}
}
} else if (samples != null && samples[0] instanceof IntBuffer) {
inputFormat = samples.length > 1 ? AV_SAMPLE_FMT_S32P : AV_SAMPLE_FMT_S32;
inputDepth = 4;
for (int i = 0; i < samples.length; i++) {
IntBuffer b = (IntBuffer)samples[i];
if (samples_in[i] instanceof IntPointer && samples_in[i].capacity() >= inputSize && b.hasArray()) {
((IntPointer)samples_in[i]).position(0).put(b.array(), samples[i].position(), inputSize);
} else {
if (samples_in[i] != null) {
samples_in[i].releaseReference();
}
samples_in[i] = new IntPointer(b).retainReference();
}
}
} else if (samples != null && samples[0] instanceof FloatBuffer) {
inputFormat = samples.length > 1 ? AV_SAMPLE_FMT_FLTP : AV_SAMPLE_FMT_FLT;
inputDepth = 4;
for (int i = 0; i < samples.length; i++) {
FloatBuffer b = (FloatBuffer)samples[i];
if (samples_in[i] instanceof FloatPointer && samples_in[i].capacity() >= inputSize && b.hasArray()) {
((FloatPointer)samples_in[i]).position(0).put(b.array(), b.position(), inputSize);
} else {
if (samples_in[i] != null) {
samples_in[i].releaseReference();
}
samples_in[i] = new FloatPointer(b).retainReference();
}
}
} else if (samples != null && samples[0] instanceof DoubleBuffer) {
inputFormat = samples.length > 1 ? AV_SAMPLE_FMT_DBLP : AV_SAMPLE_FMT_DBL;
inputDepth = 8;
for (int i = 0; i < samples.length; i++) {
DoubleBuffer b = (DoubleBuffer)samples[i];
if (samples_in[i] instanceof DoublePointer && samples_in[i].capacity() >= inputSize && b.hasArray()) {
((DoublePointer)samples_in[i]).position(0).put(b.array(), b.position(), inputSize);
} else {
if (samples_in[i] != null) {
samples_in[i].releaseReference();
}
samples_in[i] = new DoublePointer(b).retainReference();
}
}
} else if (samples != null) {
throw new Exception("Audio samples Buffer has unsupported type: " + samples);
}
boolean formatChanged = samples_channels != audioChannels || samples_format != inputFormat || samples_rate != sampleRate;
if (samples != null && (samples_convert_ctx == null || formatChanged)) {
if (samples_convert_ctx == null) {
samples_convert_ctx = new SwrContext().retainReference();
}
if ((ret = swr_alloc_set_opts2(samples_convert_ctx, audio_c.ch_layout(), outputFormat, audio_c.sample_rate(),
default_layout, inputFormat, sampleRate, 0, null)) < 0) {
throw new Exception("swr_alloc_set_opts2() error " + ret + ": Cannot allocate the conversion context.");
} else if ((ret = swr_init(samples_convert_ctx)) < 0) {
throw new Exception("swr_init() error " + ret + ": Cannot initialize the conversion context.");
}
samples_channels = audioChannels;
samples_format = inputFormat;
samples_rate = sampleRate;
}
for (int i = 0; samples != null && i < samples.length; i++) {
samples_in[i].position(samples_in[i].position() * inputDepth).
limit((samples_in[i].position() + inputSize) * inputDepth);
}
while (true) {
int inputCount = (int)Math.min(samples != null ? (samples_in[0].limit() - samples_in[0].position()) / (inputChannels * inputDepth) : 0, Integer.MAX_VALUE);
int outputCount = (int)Math.min((samples_out[0].limit() - samples_out[0].position()) / (outputChannels * outputDepth), Integer.MAX_VALUE);
inputCount = Math.min(inputCount, (outputCount * sampleRate + audio_c.sample_rate() - 1) / audio_c.sample_rate());
for (int i = 0; samples != null && i < samples.length; i++) {
plane_ptr.put(i, samples_in[i]);
}
for (int i = 0; i < samples_out.length; i++) {
plane_ptr2.put(i, samples_out[i]);
}
if (samples == null && inputCount == 0 && plane_ptr != null) {
plane_ptr.releaseReference();
// needs to be null to flush swr context.
plane_ptr = null;
}
if ((ret = swr_convert(samples_convert_ctx, plane_ptr2, outputCount, plane_ptr, inputCount)) < 0) {
throw new Exception("swr_convert() error " + ret + ": Cannot convert audio samples.");
} else if (ret == 0 && inputCount == 0) {
break;
}
for (int i = 0; samples != null && i < samples.length; i++) {
samples_in[i].position(samples_in[i].position() + inputCount * inputChannels * inputDepth);
}
for (int i = 0; i < samples_out.length; i++) {
samples_out[i].position(samples_out[i].position() + ret * outputChannels * outputDepth);
}
if (samples == null || samples_out[0].position() >= samples_out[0].limit()) {
writeSamples(audio_input_frame_size);
}
}
if (samples == null && samples_out[0].position() > 0) {
// Typically samples_out[0].limit() is double the audio_input_frame_size --> sampleDivisor = 2
double sampleDivisor = Math.floor((int)Math.min(samples_out[0].limit(), Integer.MAX_VALUE) / audio_input_frame_size);
writeSamples((int)Math.floor((int)samples_out[0].position() / sampleDivisor));
return writeFrame((AVFrame)null);
}
return samples != null ? (frame.flags() & AVFrame.AV_FRAME_FLAG_KEY) != 0 : writeFrame((AVFrame)null);
}
}
private void writeSamples(int nb_samples) throws Exception {
if (samples_out == null || samples_out.length == 0) {
return;
}
frame.nb_samples(nb_samples);
avcodec_fill_audio_frame(frame, audio_c.ch_layout().nb_channels(), audio_c.sample_fmt(), samples_out[0], (int)samples_out[0].position(), 0);
for (int i = 0; i < samples_out.length; i++) {
int linesize = 0;
if (samples_out[0].position() > 0 && samples_out[0].position() < samples_out[0].limit()) {
// align the end of the buffer to a 32-byte boundary as sometimes required by FFmpeg
linesize = ((int)samples_out[i].position() + 31) & ~31;
} else {
linesize = (int)Math.min(samples_out[i].limit(), Integer.MAX_VALUE);
}
frame.data(i, samples_out[i].position(0));
frame.linesize(i, linesize);
}
frame.ch_layout(audio_c.ch_layout());
frame.format(audio_c.sample_fmt());
frame.quality(audio_c.global_quality());
writeFrame(frame);
wrote_samples = true;
}
private boolean writeFrame(AVFrame frame) throws Exception {
int ret;
if ((ret = avcodec_send_frame(audio_c, frame)) < 0 && frame != null) {
throw new Exception("avcodec_send_frame() error " + ret + ": Error sending an audio frame for encoding.");
}
if (frame != null) {
frame.pts(frame.pts() + frame.nb_samples()); // magic required by libvorbis and webm
}
/* if zero size, it means the image was buffered */
got_audio_packet[0] = 0;
while (ret >= 0) {
av_new_packet(audio_pkt, audio_outbuf_size);
ret = avcodec_receive_packet(audio_c, audio_pkt);
if (ret == AVERROR_EAGAIN() || ret == AVERROR_EOF()) {
av_packet_unref(audio_pkt);
break;
} else if (ret < 0) {
av_packet_unref(audio_pkt);
throw new Exception("avcodec_receive_packet() error " + ret + ": Error during audio encoding.");
}
got_audio_packet[0] = 1;
if (audio_pkt.pts() != AV_NOPTS_VALUE) {
audio_pkt.pts(av_rescale_q(audio_pkt.pts(), audio_c.time_base(), audio_st.time_base()));
}
if (audio_pkt.dts() != AV_NOPTS_VALUE) {
audio_pkt.dts(av_rescale_q(audio_pkt.dts(), audio_c.time_base(), audio_st.time_base()));
}
audio_pkt.flags(audio_pkt.flags() | AV_PKT_FLAG_KEY);
audio_pkt.stream_index(audio_st.index());
/* write the compressed frame in the media file */
writePacket(AVMEDIA_TYPE_AUDIO, audio_pkt);
if (frame == null && !wrote_samples) {
break;
}
}
return got_audio_packet[0] != 0;
}
private void writePacket(int mediaType, AVPacket avPacket) throws Exception {
AVStream avStream = (mediaType == AVMEDIA_TYPE_VIDEO) ? video_st : (mediaType == AVMEDIA_TYPE_AUDIO) ? audio_st : null;
String mediaTypeStr = (mediaType == AVMEDIA_TYPE_VIDEO) ? "video" : (mediaType == AVMEDIA_TYPE_AUDIO) ? "audio" : "unsupported media stream type";
synchronized (oc) {
int ret;
if (interleaved && avStream != null) {
if ((ret = av_interleaved_write_frame(oc, avPacket)) < 0) {
av_packet_unref(avPacket);
throw new Exception("av_interleaved_write_frame() error " + ret + " while writing interleaved " + mediaTypeStr + " packet.");
}
} else {
if ((ret = av_write_frame(oc, avPacket)) < 0) {
av_packet_unref(avPacket);
throw new Exception("av_write_frame() error " + ret + " while writing " + mediaTypeStr + " packet.");
}
}
}
av_packet_unref(avPacket);
}
public synchronized boolean recordPacket(AVPacket pkt) throws Exception {
if (ifmt_ctx == null) {
throw new Exception("No input format context (Has start(AVFormatContext) been called?)");
}
if (!started) {
throw new Exception("start() was not called successfully!");
}
if (pkt == null) {
return false;
}
AVStream in_stream = ifmt_ctx.streams(pkt.stream_index());
/**
* Repair the problem of error decoding and playback caused by the absence of dts/pts
* in the output audio/video file or audio/video stream,
* Comment out this line of code so that PTS / DTS can specify the timestamp manually.
*/
// pkt.dts(AV_NOPTS_VALUE);
// pkt.pts(AV_NOPTS_VALUE);
pkt.pos(-1);
if (in_stream.codecpar().codec_type() == AVMEDIA_TYPE_VIDEO && video_st != null) {
pkt.stream_index(video_st.index());
pkt.duration((int) av_rescale_q(pkt.duration(), in_stream.time_base(), video_st.time_base()));
pkt.pts(av_rescale_q_rnd(pkt.pts(), in_stream.time_base(), video_st.time_base(),(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX)));//Increase pts calculation
pkt.dts(av_rescale_q_rnd(pkt.dts(), in_stream.time_base(), video_st.time_base(),(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX)));//Increase dts calculation
writePacket(AVMEDIA_TYPE_VIDEO, pkt);
} else if (in_stream.codecpar().codec_type() == AVMEDIA_TYPE_AUDIO && audio_st != null && (audioChannels > 0)) {
pkt.stream_index(audio_st.index());
pkt.duration((int) av_rescale_q(pkt.duration(), in_stream.time_base(), audio_st.time_base()));
pkt.pts(av_rescale_q_rnd(pkt.pts(), in_stream.time_base(), audio_st.time_base(),(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX)));//Increase pts calculation
pkt.dts(av_rescale_q_rnd(pkt.dts(), in_stream.time_base(), audio_st.time_base(),(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX)));//Increase dts calculation
writePacket(AVMEDIA_TYPE_AUDIO, pkt);
}
return true;
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/FFmpegLockCallback.java
================================================
package org.bytedeco.javacv;
import java.util.HashMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
import org.bytedeco.javacpp.IntPointer;
import org.bytedeco.javacpp.PointerPointer;
import org.bytedeco.javacpp.annotation.Cast;
import org.bytedeco.ffmpeg.avcodec.*;
import static org.bytedeco.ffmpeg.global.avcodec.*;
public class FFmpegLockCallback {
private static boolean initialized = false;
private static AtomicInteger lockCounter = new AtomicInteger(0);
private static HashMap lockArray = new HashMap<>();
private static Cb_PointerPointer_int lockCallback = new Cb_PointerPointer_int() {
@Override
public int call(@SuppressWarnings("rawtypes") @Cast("void**") PointerPointer mutex, @Cast("AVLockOp") int op) {
int number;
Lock l;
// System.out.println "Locking: " + op);
switch (op) {
case AV_LOCK_CREATE:
number = lockCounter.incrementAndGet();
// System.out.println("Command: " + op + " number: " + number);
new IntPointer(mutex).put(0, number);
lockArray.put(number, new ReentrantLock());
return 0;
case AV_LOCK_OBTAIN:
number = new IntPointer(mutex).get(0);
// System.out.println("Command: " + op + " number: " + number);
l = lockArray.get(number);
if (l == null) {
System.err.println("Lock not found!");
return -1;
}
l.lock();
return 0;
case AV_LOCK_RELEASE:
number = new IntPointer(mutex).get(0);
// System.out.println("Command: " + op + " number: " + number);
l = lockArray.get(number);
if (l == null) {
System.err.println("Lock not found!");
return -1;
}
l.unlock();
return 0;
case AV_LOCK_DESTROY:
number = new IntPointer(mutex).get(0);
// System.out.println("Command: " + op + " number: " + number);
lockArray.remove(number);
mutex.put(0, null);
return 0;
default:
return -1;
}
}
}.retainReference();
public static synchronized void init() {
if (!initialized) {
initialized = true;
av_lockmgr_register(lockCallback);
}
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/FFmpegLogCallback.java
================================================
/*
* Copyright (C) 2015-2021 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.tools.Logger;
import org.bytedeco.ffmpeg.avutil.*;
import static org.bytedeco.ffmpeg.global.avutil.*;
/**
* A utility class to redirect to Java log messages from FFmpeg.
*
* @see Logger
*
* @author Samuel Audet
*/
public class FFmpegLogCallback extends LogCallback {
private static final Logger logger = Logger.create(FFmpegLogCallback.class);
static final FFmpegLogCallback instance = new FFmpegLogCallback().retainReference();
/** Returns an instance that can be used with {@link org.bytedeco.ffmpeg.global.avutil#setLogCallback(LogCallback)}. */
public static FFmpegLogCallback getInstance() {
return instance;
}
/** Calls {@code avutil.setLogCallback(getInstance())}. */
public static void set() {
setLogCallback(getInstance());
}
/** Returns {@code av_log_get_level()}. **/
public static int getLevel() {
return av_log_get_level();
}
/** Calls {@code av_log_set_level(level)}. **/
public static void setLevel(int level) {
av_log_set_level(level);
}
/** Logs the given rejected options regarding the given command */
public static void logRejectedOptions(final AVDictionary options, final String command) {
if (getLevel() >= AV_LOG_INFO && av_dict_count(options) > 0) {
final StringBuilder sb = new StringBuilder(command + " rejected some options:");
AVDictionaryEntry e = null;
while ((e = av_dict_iterate(options, e)) != null) {
sb.append("\tOption: ").append(e.key().getString()).append(", value: ").append(e.value().getString());
}
logger.info(sb.toString());
}
}
@Override public void call(int level, BytePointer msg) {
switch (level) {
case AV_LOG_PANIC:
case AV_LOG_FATAL:
case AV_LOG_ERROR:
logger.error(msg.getString());
break;
case AV_LOG_WARNING:
logger.warn(msg.getString());
break;
case AV_LOG_INFO:
logger.info(msg.getString());
break;
case AV_LOG_VERBOSE:
case AV_LOG_DEBUG:
case AV_LOG_TRACE:
logger.debug(msg.getString());
break;
default:
assert false;
}
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/FlyCapture2FrameGrabber.java
================================================
/*
* Copyright (C) 2014,2017 Jeremy Laviole, Samuel Audet, Jarek Sacha
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.io.File;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.ShortBuffer;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.IntPointer;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.flycapture.FlyCapture2.*;
import org.bytedeco.flycapture.FlyCapture2.Error;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.flycapture.global.FlyCapture2.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
*
* @author Jeremy Laviole
* @author Jarek Sacha
*/
public class FlyCapture2FrameGrabber extends FrameGrabber {
public static String[] getDeviceDescriptions() throws FrameGrabber.Exception {
tryLoad();
BusManager busMgr = new BusManager();
int[] numCameras = new int[1];
busMgr.GetNumOfCameras(numCameras);
String[] descriptions = new String[numCameras[0]];
for (int i = 0; i < numCameras[0]; i++) {
PGRGuid guid = new PGRGuid();
Error error = busMgr.GetCameraFromIndex(i, guid);
if (error.notEquals(PGRERROR_OK)) {
PrintError(error);
System.exit(-1);
}
Camera cam = new Camera();
// Connect to a camera
error = cam.Connect(guid);
if (error.notEquals(PGRERROR_OK)) {
PrintError(error);
}
// Get the camera information
CameraInfo camInfo = new CameraInfo();
error = cam.GetCameraInfo(camInfo);
if (error.notEquals(PGRERROR_OK)) {
PrintError(error);
}
descriptions[i] = CameraInfo(camInfo);
}
return descriptions;
}
static void PrintError(Error error) {
error.PrintErrorTrace();
}
static String CameraInfo(CameraInfo pCamInfo) {
return "\n*** CAMERA INFORMATION ***\n"
+ "Serial number - " + pCamInfo.serialNumber() + "\n"
+ "Camera model - " + pCamInfo.modelName().getString() + "\n"
+ "Camera vendor - " + pCamInfo.vendorName().getString() + "\n"
+ "Sensor - " + pCamInfo.sensorInfo().getString() + "\n"
+ "Resolution - " + pCamInfo.sensorResolution().getString() + "\n"
+ "Firmware version - " + pCamInfo.firmwareVersion().getString() + "\n"
+ "Firmware build time - " + pCamInfo.firmwareBuildTime().getString() + "\n";
}
public static FlyCapture2FrameGrabber createDefault(File deviceFile) throws FrameGrabber.Exception {
return null;
}
public static FlyCapture2FrameGrabber createDefault(String devicePath) throws FrameGrabber.Exception {
return null;
}
public static FlyCapture2FrameGrabber createDefault(int deviceNumber) throws FrameGrabber.Exception {
return new FlyCapture2FrameGrabber(deviceNumber);
}
private static FrameGrabber.Exception loadingException = null;
public static void tryLoad() throws FrameGrabber.Exception {
if (loadingException != null) {
loadingException.printStackTrace();
throw loadingException;
} else {
try {
Loader.load(org.bytedeco.flycapture.global.FlyCapture2.class);
} catch (Throwable t) {
throw loadingException = new FrameGrabber.Exception("Failed to load " + FlyCapture2FrameGrabber.class, t);
}
}
}
public FlyCapture2FrameGrabber(int deviceNumber) throws FrameGrabber.Exception {
int[] numCameras = new int[1];
busMgr.GetNumOfCameras(numCameras);
// Get the camera
PGRGuid guid = new PGRGuid();
Error error = busMgr.GetCameraFromIndex(deviceNumber, guid);
if (error.notEquals(PGRERROR_OK)) {
PrintError(error);
System.exit(-1);
}
camera = new Camera();
// Connect to a camera
error = camera.Connect(guid);
if (error.notEquals(PGRERROR_OK)) {
PrintError(error);
}
// Get the camera information
cameraInfo = new CameraInfo();
error = camera.GetCameraInfo(cameraInfo);
if (error.notEquals(PGRERROR_OK)) {
PrintError(error);
}
}
public void release() throws FrameGrabber.Exception {
if (camera != null) {
stop();
camera.Disconnect();
camera = null;
}
}
@Override
protected void finalize() throws Throwable {
super.finalize();
release();
}
public static final int INITIALIZE = 0x000,
TRIGGER_INQ = 0x530,
IS_CAMERA_POWER = 0x400,
CAMERA_POWER = 0x610,
SOFTWARE_TRIGGER = 0x62C,
SOFT_ASYNC_TRIGGER = 0x102C,
IMAGE_DATA_FORMAT = 0x1048;
private BusManager busMgr = new BusManager();
private Camera camera;
private CameraInfo cameraInfo;
private Image raw_image = new Image();
private Image conv_image = new Image();
private IplImage temp_image, return_image = null;
private FrameConverter converter = new OpenCVFrameConverter.ToIplImage();
private final int[] regOut = new int[1];
private final float[] outFloat = new float[1];
private final float[] gammaOut = new float[1];
@Override
public double getGamma() {
return Float.isNaN(gammaOut[0]) || Float.isInfinite(gammaOut[0]) || gammaOut[0] == 0.0f ? 2.2 : gammaOut[0];
}
@Override
public int getImageWidth() {
return return_image == null ? super.getImageWidth() : return_image.width();
}
@Override
public int getImageHeight() {
return return_image == null ? super.getImageHeight() : return_image.height();
}
@Override
public double getFrameRate() {
if (camera == null || camera.isNull()) {
return super.getFrameRate();
} else {
IntPointer videoMode = new IntPointer(1L);
IntPointer frameRate = new IntPointer(1L);
camera.GetVideoModeAndFrameRate(videoMode, frameRate);
return frameRate.get(0);
}
}
@Override
public void setImageMode(FrameGrabber.ImageMode imageMode) {
if (imageMode != this.imageMode) {
temp_image = null;
return_image = null;
}
super.setImageMode(imageMode);
}
static final int VIDEOMODE_ANY = -1;
public void start() throws FrameGrabber.Exception {
int f = FRAMERATE_30; // TODO: Default 30 ?
if (frameRate <= 0) {
f = FRAMERATE_30;
} else if (frameRate <= 1.876) {
f = FRAMERATE_1_875;
} else if (frameRate <= 3.76) {
f = FRAMERATE_3_75;
} else if (frameRate <= 7.51) {
f = FRAMERATE_7_5;
} else if (frameRate <= 15.01) {
f = FRAMERATE_15;
} else if (frameRate <= 30.01) {
f = FRAMERATE_30;
} else if (frameRate <= 60.01) {
f = FRAMERATE_60;
} else if (frameRate <= 120.01) {
f = FRAMERATE_120;
} else if (frameRate <= 240.01) {
f = FRAMERATE_240;
}
int c = VIDEOMODE_ANY;
if (imageMode == FrameGrabber.ImageMode.COLOR || imageMode == FrameGrabber.ImageMode.RAW) {
if (imageWidth <= 0 || imageHeight <= 0) {
c = VIDEOMODE_ANY;
} else if (imageWidth <= 640 && imageHeight <= 480) {
c = VIDEOMODE_640x480RGB;
} else if (imageWidth <= 800 && imageHeight <= 600) {
c = VIDEOMODE_800x600RGB;
} else if (imageWidth <= 1024 && imageHeight <= 768) {
c = VIDEOMODE_1024x768RGB;
} else if (imageWidth <= 1280 && imageHeight <= 960) {
c = VIDEOMODE_1280x960RGB;
} else if (imageWidth <= 1600 && imageHeight <= 1200) {
c = VIDEOMODE_1600x1200RGB;
}
} else if (imageMode == FrameGrabber.ImageMode.GRAY) {
if (imageWidth <= 0 || imageHeight <= 0) {
c = VIDEOMODE_ANY;
} else if (imageWidth <= 640 && imageHeight <= 480) {
c = bpp > 8 ? VIDEOMODE_640x480Y16 : VIDEOMODE_640x480Y8;
} else if (imageWidth <= 800 && imageHeight <= 600) {
c = bpp > 8 ? VIDEOMODE_800x600Y16 : VIDEOMODE_800x600Y8;
} else if (imageWidth <= 1024 && imageHeight <= 768) {
c = bpp > 8 ? VIDEOMODE_1024x768Y16 : VIDEOMODE_1024x768Y8;
} else if (imageWidth <= 1280 && imageHeight <= 960) {
c = bpp > 8 ? VIDEOMODE_1280x960Y16 : VIDEOMODE_1280x960Y8;
} else if (imageWidth <= 1600 && imageHeight <= 1200) {
c = bpp > 8 ? VIDEOMODE_1600x1200Y16 : VIDEOMODE_1600x1200Y8;
}
}
// set or reset trigger mode
TriggerMode tm = new TriggerMode();
Error error = camera.GetTriggerMode(tm);
if (error.notEquals(PGRERROR_OK)) {
PrintError(error);
throw new FrameGrabber.Exception("GetTriggerMode() Error " + error.GetDescription());
}
tm.onOff(triggerMode);
tm.source(7);
tm.mode(14);
tm.parameter(0);
error = camera.SetTriggerMode(tm);
if (error.notEquals(PGRERROR_OK)) {
// try with trigger mode 0 instead
tm.onOff(true);
tm.source(7);
tm.mode(0);
tm.parameter(0);
error = camera.SetTriggerMode(tm);
if (error.notEquals(PGRERROR_OK)) {
PrintError(error);
throw new FrameGrabber.Exception("SetTriggerMode() Error " + error.GetDescription());
}
}
if (triggerMode) {
waitForTriggerReady();
}
// try to match the endianness to our platform
error = camera.ReadRegister(IMAGE_DATA_FORMAT, regOut);
if (error.notEquals(PGRERROR_OK)) {
PrintError(error);
throw new FrameGrabber.Exception("ReadRegister(IMAGE_DATA_FORMAT, regOut) Error " + error.GetDescription());
}
int reg;
if (ByteOrder.nativeOrder().equals(ByteOrder.BIG_ENDIAN)) {
reg = regOut[0] | 0x1;
} else {
reg = regOut[0] & ~0x1;
}
error = camera.WriteRegister(IMAGE_DATA_FORMAT, reg);
if (error.notEquals(PGRERROR_OK)) {
PrintError(error);
throw new FrameGrabber.Exception("WriteRegister(IMAGE_DATA_FORMAT, reg) Error " + error.GetDescription());
}
// TODO: set fastest bus speed ? This may lead to system instability. Use default.
// set `gamma`
Property gammaProp = new Property(GAMMA);
if (gamma != 0.0) {
error = camera.GetProperty(gammaProp);
if (error.notEquals(PGRERROR_OK)) {
throw new FrameGrabber.Exception("GetProperty(gammaProp) Error " + error.GetDescription());
}
gammaProp.onOff(true);
gammaProp.absControl(true);
gammaProp.absValue((float)gamma);
camera.SetProperty(gammaProp);
error = camera.SetProperty(gammaProp);
if (error.notEquals(PGRERROR_OK)) {
PrintError(error);
throw new FrameGrabber.Exception("SetProperty(gammaProp) Error " + error.GetDescription());
}
}
error = camera.GetProperty(gammaProp);
if (error.notEquals(PGRERROR_OK)) {
gammaOut[0] = 2.2f;
} else {
gammaOut[0] = gammaProp.absValue();
}
// set `timeout`
error = camera.StartCapture();
if (error.notEquals(PGRERROR_OK)) {
PrintError(error);
throw new FrameGrabber.Exception("StartCapture() Error " + error.GetDescription());
}
// Get the camera configuration
FC2Config config = new FC2Config();
error = camera.GetConfiguration(config);
if (error.notEquals(PGRERROR_OK)) {
PrintError(error);
throw new FrameGrabber.Exception("GetConfiguration() Error " + error.GetDescription());
}
// Set the grab timeout to 5 seconds
config.grabTimeout(timeout);
// Set the camera configuration
error = camera.SetConfiguration(config);
if (error.notEquals(PGRERROR_OK)) {
PrintError(error);
throw new FrameGrabber.Exception("SetConfiguration() Error " + error.GetDescription());
}
}
private void waitForTriggerReady() throws Exception {
// wait for trigger to be ready...
long time = System.currentTimeMillis();
do {
Error error = camera.ReadRegister(SOFTWARE_TRIGGER, regOut);
if (error.notEquals(PGRERROR_OK)) {
PrintError(error);
throw new FrameGrabber.Exception("GetTriggerMode() Error " + error.GetDescription());
}
if (System.currentTimeMillis() - time > timeout) {
break;
//throw new Exception("waitForTriggerReady() Error: Timeout occured.");
}
} while((regOut[0] >>> 31) != 0);
}
public void stop() throws FrameGrabber.Exception {
Error error = camera.StopCapture();
if (error.notEquals(PGRERROR_OK)) {
PrintError(error);
throw new FrameGrabber.Exception("flycapture camera StopCapture() Error " + error);
}
temp_image = null;
return_image = null;
timestamp = 0;
frameNumber = 0;
}
/**
* @throws org.bytedeco.javacv.FrameGrabber.Exception
*/
public void trigger() throws FrameGrabber.Exception {
waitForTriggerReady();
Error error = camera.FireSoftwareTrigger();
if (error.notEquals(PGRERROR_OK)) {
throw new FrameGrabber.Exception("flycaptureSetCameraRegister() Error " + error);
}
}
private int getNumChannels(int pixelFormat) {
switch (pixelFormat) {
case PIXEL_FORMAT_BGR:
case PIXEL_FORMAT_RGB8:
case PIXEL_FORMAT_RGB16:
case PIXEL_FORMAT_S_RGB16:
return 3;
case PIXEL_FORMAT_MONO8:
case PIXEL_FORMAT_MONO16:
case PIXEL_FORMAT_RAW8:
case PIXEL_FORMAT_RAW16:
case PIXEL_FORMAT_S_MONO16:
return 1;
case PIXEL_FORMAT_BGRU:
return 4;
case PIXEL_FORMAT_411YUV8:
case PIXEL_FORMAT_422YUV8:
case PIXEL_FORMAT_444YUV8:
default:
return -1;
}
}
private int getDepth(int pixelFormat) {
switch (pixelFormat) {
case PIXEL_FORMAT_BGR:
case PIXEL_FORMAT_RGB8:
case PIXEL_FORMAT_MONO8:
case PIXEL_FORMAT_RAW8:
case PIXEL_FORMAT_BGRU:
return IPL_DEPTH_8U;
case PIXEL_FORMAT_MONO16:
case PIXEL_FORMAT_RAW16:
case PIXEL_FORMAT_RGB16:
return IPL_DEPTH_16U;
case PIXEL_FORMAT_S_MONO16:
case PIXEL_FORMAT_S_RGB16:
return IPL_DEPTH_16S;
case PIXEL_FORMAT_411YUV8:
case PIXEL_FORMAT_422YUV8:
case PIXEL_FORMAT_444YUV8:
default:
return IPL_DEPTH_8U;
}
}
private void setPixelFormat(Image image, int pixelFormat) {
image.SetDimensions(image.GetRows(),
image.GetCols(),
image.GetStride(),
pixelFormat,
image.GetBayerTileFormat());
}
private void setStride(Image image, int stride) {
image.SetDimensions(image.GetRows(),
image.GetCols(),
stride,
image.GetPixelFormat(),
image.GetBayerTileFormat());
}
public Frame grab() throws FrameGrabber.Exception {
Error error = camera.RetrieveBuffer(raw_image);
if (error.notEquals(PGRERROR_OK)) {
throw new FrameGrabber.Exception("flycaptureGrabImage2() Error " + error + " (Has start() been called?)");
}
int w = raw_image.GetCols();
int h = raw_image.GetRows();
int format = raw_image.GetPixelFormat();
int depth = getDepth(format);
int stride = raw_image.GetStride();
int size = h * stride;
int numChannels = getNumChannels(format);
error = camera.ReadRegister(IMAGE_DATA_FORMAT, regOut);
if (error.notEquals(PGRERROR_OK)) {
throw new FrameGrabber.Exception("flycaptureGetCameraRegister() Error " + error);
}
ByteOrder frameEndian = (regOut[0] & 0x1) != 0
? ByteOrder.BIG_ENDIAN : ByteOrder.LITTLE_ENDIAN;
boolean alreadySwapped = false;
boolean colorbayer = raw_image.GetBayerTileFormat() != NONE;
boolean colorrgb = format == PIXEL_FORMAT_RGB8 || format == PIXEL_FORMAT_RGB16
|| format == PIXEL_FORMAT_BGR || format == PIXEL_FORMAT_BGRU;
boolean coloryuv = format == PIXEL_FORMAT_411YUV8 || format == PIXEL_FORMAT_422YUV8
|| format == PIXEL_FORMAT_444YUV8;
BytePointer imageData = raw_image.GetData().capacity(raw_image.GetDataSize());
if ((depth == IPL_DEPTH_8U || frameEndian.equals(ByteOrder.nativeOrder()))
&& (imageMode == FrameGrabber.ImageMode.RAW || (imageMode == FrameGrabber.ImageMode.COLOR && numChannels == 3)
|| (imageMode == FrameGrabber.ImageMode.GRAY && numChannels == 1 && !colorbayer))) {
if (return_image == null) {
return_image = IplImage.createHeader(w, h, depth, numChannels);
}
return_image.widthStep(stride);
return_image.imageSize(size);
return_image.imageData(imageData);
} else {
if (return_image == null) {
return_image = IplImage.create(w, h, depth, imageMode == FrameGrabber.ImageMode.COLOR ? 3 : 1);
}
if (temp_image == null) {
if (imageMode == FrameGrabber.ImageMode.COLOR
&& (numChannels > 1 || depth > 8) && !coloryuv && !colorbayer) {
temp_image = IplImage.create(w, h, depth, numChannels);
} else if (imageMode == FrameGrabber.ImageMode.GRAY && colorbayer) {
temp_image = IplImage.create(w, h, depth, 3);
} else if (imageMode == FrameGrabber.ImageMode.GRAY && colorrgb) {
temp_image = IplImage.createHeader(w, h, depth, 3);
} else if (imageMode == FrameGrabber.ImageMode.COLOR && numChannels == 1 && !coloryuv && !colorbayer) {
temp_image = IplImage.createHeader(w, h, depth, 1);
} else {
temp_image = return_image;
}
}
setStride(conv_image, temp_image.widthStep());
conv_image.SetData(temp_image.imageData(), temp_image.width() * temp_image.height() * temp_image.depth());
if (depth == IPL_DEPTH_8U) {
setPixelFormat(conv_image, imageMode == FrameGrabber.ImageMode.RAW ? PIXEL_FORMAT_RAW8
: temp_image.nChannels() == 1 ? PIXEL_FORMAT_MONO8 : PIXEL_FORMAT_BGR);
} else {
setPixelFormat(conv_image, imageMode == FrameGrabber.ImageMode.RAW ? PIXEL_FORMAT_RAW16
: temp_image.nChannels() == 1 ? PIXEL_FORMAT_MONO16 : PIXEL_FORMAT_RGB16);
}
if (depth != IPL_DEPTH_8U && conv_image.GetPixelFormat() == format && conv_image.GetStride() == stride) {
// we just need a copy to swap bytes..
ShortBuffer in = imageData.asByteBuffer().order(frameEndian).asShortBuffer();
ShortBuffer out = temp_image.getByteBuffer().order(ByteOrder.nativeOrder()).asShortBuffer();
out.put(in);
alreadySwapped = true;
} else if ((imageMode == FrameGrabber.ImageMode.GRAY && colorrgb)
|| (imageMode == FrameGrabber.ImageMode.COLOR && numChannels == 1 && !coloryuv && !colorbayer)) {
temp_image.widthStep(stride);
temp_image.imageSize(size);
temp_image.imageData(imageData);
} else if (!colorrgb && (colorbayer || coloryuv || numChannels > 1)) {
error = raw_image.Convert(conv_image);
// error = flycaptureConvertImage(context, raw_image, conv_image);
if (error.notEquals(PGRERROR_OK)) {
PrintError(error);
throw new FrameGrabber.Exception("raw_image.Convert Error " + error);
}
}
if (!alreadySwapped && depth != IPL_DEPTH_8U
&& !frameEndian.equals(ByteOrder.nativeOrder())) {
// ack, the camera's endianness doesn't correspond to our machine ...
// swap bytes of 16-bit images
ByteBuffer bb = temp_image.getByteBuffer();
ShortBuffer in = bb.order(frameEndian).asShortBuffer();
ShortBuffer out = bb.order(ByteOrder.nativeOrder()).asShortBuffer();
out.put(in);
}
if (imageMode == FrameGrabber.ImageMode.COLOR && numChannels == 1 && !coloryuv && !colorbayer) {
cvCvtColor(temp_image, return_image, CV_GRAY2BGR);
} else if (imageMode == FrameGrabber.ImageMode.GRAY && (colorbayer || colorrgb)) {
cvCvtColor(temp_image, return_image, CV_BGR2GRAY);
}
}
int bayerFormat = cameraInfo.bayerTileFormat();
switch (bayerFormat) {
case BGGR:
sensorPattern = SENSOR_PATTERN_BGGR;
break;
case GBRG:
sensorPattern = SENSOR_PATTERN_GBRG;
break;
case GRBG:
sensorPattern = SENSOR_PATTERN_GRBG;
break;
case RGGB:
sensorPattern = SENSOR_PATTERN_RGGB;
break;
default:
sensorPattern = -1L;
}
TimeStamp timeStamp = raw_image.GetTimeStamp();
timestamp = timeStamp.seconds() * 1000000L + timeStamp.microSeconds();
return converter.convert(return_image);
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/FlyCaptureFrameGrabber.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.io.File;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.ShortBuffer;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.flycapture.PGRFlyCapture.*;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.flycapture.global.PGRFlyCapture.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
*
* @author Samuel Audet
*/
public class FlyCaptureFrameGrabber extends FrameGrabber {
public static String[] getDeviceDescriptions() throws Exception {
tryLoad();
int[] count = new int[1];
int error = flycaptureBusCameraCount(count);
if (error != FLYCAPTURE_OK) {
throw new Exception("flycaptureBusCameraCount() Error " + error);
}
int c = count[0];
String[] descriptions = new String[c];
if (c > 0) {
FlyCaptureInfoEx info = new FlyCaptureInfoEx(c);
error = flycaptureBusEnumerateCamerasEx(info, count);
if (error != FLYCAPTURE_OK) {
throw new Exception("flycaptureBusEnumerateCamerasEx() Error " + error);
}
for (int i = 0; i < descriptions.length; i++) {
info.position(i);
descriptions[i] = info.pszVendorName() + " " +
info.pszModelName() + " " + info.SerialNumber();
}
}
return descriptions;
}
public static FlyCaptureFrameGrabber createDefault(File deviceFile) throws Exception { throw new Exception(FlyCaptureFrameGrabber.class + " does not support device files."); }
public static FlyCaptureFrameGrabber createDefault(String devicePath) throws Exception { throw new Exception(FlyCaptureFrameGrabber.class + " does not support device paths."); }
public static FlyCaptureFrameGrabber createDefault(int deviceNumber) throws Exception { return new FlyCaptureFrameGrabber(deviceNumber); }
private static Exception loadingException = null;
public static void tryLoad() throws Exception {
if (loadingException != null) {
throw loadingException;
} else {
try {
Loader.load(org.bytedeco.javacpp.PGRFlyCapture.class);
} catch (Throwable t) {
throw loadingException = new Exception("Failed to load " + FlyCaptureFrameGrabber.class, t);
}
}
}
public FlyCaptureFrameGrabber(int deviceNumber) throws Exception {
int error = flycaptureCreateContext(context);
if (error != FLYCAPTURE_OK) {
throw new Exception("flycaptureCreateContext() Error " + error);
}
error = flycaptureInitializePlus(context, deviceNumber, numBuffers, (BytePointer)null);
if (error != FLYCAPTURE_OK) {
throw new Exception("flycaptureInitialize() Error " + error);
}
}
public void release() throws Exception {
if (context != null) {
stop();
int error = flycaptureDestroyContext(context);
context = null;
if (error != FLYCAPTURE_OK) {
throw new Exception("flycaptureDestroyContext() Error " + error);
}
}
}
@Override protected void finalize() throws Throwable {
super.finalize();
release();
}
public static final int
INITIALIZE = 0x000,
TRIGGER_INQ = 0x530,
IS_CAMERA_POWER = 0x400,
CAMERA_POWER = 0x610,
SOFTWARE_TRIGGER = 0x62C,
SOFT_ASYNC_TRIGGER = 0x102C,
IMAGE_DATA_FORMAT = 0x1048;
private FlyCaptureContext context = new FlyCaptureContext(null);
private FlyCaptureImage raw_image = new FlyCaptureImage();
private FlyCaptureImage conv_image = new FlyCaptureImage();
private IplImage temp_image, return_image = null;
private FrameConverter converter = new OpenCVFrameConverter.ToIplImage();
private final int[] regOut = new int[1];
private final float[] outFloat = new float[1];
private final float[] gammaOut = new float[1];
@Override public double getGamma() {
return Float.isNaN(gammaOut[0]) || Float.isInfinite(gammaOut[0]) || gammaOut[0] == 0.0f ? 2.2 : gammaOut[0];
}
@Override public int getImageWidth() {
return return_image == null ? super.getImageWidth() : return_image.width();
}
@Override public int getImageHeight() {
return return_image == null ? super.getImageHeight() : return_image.height();
}
@Override public double getFrameRate() {
if (context == null || context.isNull()) {
return super.getFrameRate();
} else {
flycaptureGetCameraAbsProperty(context, FLYCAPTURE_FRAME_RATE, outFloat);
return outFloat[0];
}
}
@Override public void setImageMode(ImageMode imageMode) {
if (imageMode != this.imageMode) {
temp_image = null;
return_image = null;
}
super.setImageMode(imageMode);
}
public void start() throws Exception {
int f = FLYCAPTURE_FRAMERATE_ANY;
if (frameRate <= 0) {
f = FLYCAPTURE_FRAMERATE_ANY;
} else if (frameRate <= 1.876) {
f = FLYCAPTURE_FRAMERATE_1_875;
} else if (frameRate <= 3.76) {
f = FLYCAPTURE_FRAMERATE_3_75;
} else if (frameRate <= 7.51) {
f = FLYCAPTURE_FRAMERATE_7_5;
} else if (frameRate <= 15.01) {
f = FLYCAPTURE_FRAMERATE_15;
} else if (frameRate <= 30.01) {
f = FLYCAPTURE_FRAMERATE_30;
} else if (frameRate <= 60.01) {
f = FLYCAPTURE_FRAMERATE_60;
} else if (frameRate <= 120.01) {
f = FLYCAPTURE_FRAMERATE_120;
} else if (frameRate <= 240.01) {
f = FLYCAPTURE_FRAMERATE_240;
}
int c = FLYCAPTURE_VIDEOMODE_ANY;
if (imageMode == ImageMode.COLOR || imageMode == ImageMode.RAW) {
if (imageWidth <= 0 || imageHeight <= 0) {
c = FLYCAPTURE_VIDEOMODE_ANY;
} else if (imageWidth <= 640 && imageHeight <= 480) {
c = FLYCAPTURE_VIDEOMODE_640x480RGB;
} else if (imageWidth <= 800 && imageHeight <= 600) {
c = FLYCAPTURE_VIDEOMODE_800x600RGB;
} else if (imageWidth <= 1024 && imageHeight <= 768) {
c = FLYCAPTURE_VIDEOMODE_1024x768RGB;
} else if (imageWidth <= 1280 && imageHeight <= 960) {
c = FLYCAPTURE_VIDEOMODE_1280x960RGB;
} else if (imageWidth <= 1600 && imageHeight <= 1200) {
c = FLYCAPTURE_VIDEOMODE_1600x1200RGB;
}
} else if (imageMode == ImageMode.GRAY) {
if (imageWidth <= 0 || imageHeight <= 0) {
c = FLYCAPTURE_VIDEOMODE_ANY;
} else if (imageWidth <= 640 && imageHeight <= 480) {
c = bpp > 8 ? FLYCAPTURE_VIDEOMODE_640x480Y16 : FLYCAPTURE_VIDEOMODE_640x480Y8;
} else if (imageWidth <= 800 && imageHeight <= 600) {
c = bpp > 8 ? FLYCAPTURE_VIDEOMODE_800x600Y16 : FLYCAPTURE_VIDEOMODE_800x600Y8;
} else if (imageWidth <= 1024 && imageHeight <= 768) {
c = bpp > 8 ? FLYCAPTURE_VIDEOMODE_1024x768Y16 : FLYCAPTURE_VIDEOMODE_1024x768Y8;
} else if (imageWidth <= 1280 && imageHeight <= 960) {
c = bpp > 8 ? FLYCAPTURE_VIDEOMODE_1280x960Y16 : FLYCAPTURE_VIDEOMODE_1280x960Y8;
} else if (imageWidth <= 1600 && imageHeight <= 1200) {
c = bpp > 8 ? FLYCAPTURE_VIDEOMODE_1600x1200Y16 : FLYCAPTURE_VIDEOMODE_1600x1200Y8;
}
}
// set or reset trigger mode
int[] iPolarity = new int[1];
int[] iSource = new int[1];
int[] iRawValue = new int[1];
int[] iMode = new int[1];
int error = flycaptureGetTrigger(context, (boolean[])null, iPolarity, iSource, iRawValue, iMode, null);
if (error != FLYCAPTURE_OK) {
throw new Exception("flycaptureGetTrigger() Error " + error);
}
error = flycaptureSetTrigger(context, triggerMode, iPolarity[0], 7, 14, 0);
if (error != FLYCAPTURE_OK) {
// try with trigger mode 0 instead
error = flycaptureSetTrigger(context, true, iPolarity[0], 7, 0, 0);
}
if (error != FLYCAPTURE_OK) {
throw new Exception("flycaptureSetTrigger() Error " + error);
}
if (triggerMode) {
waitForTriggerReady();
}
// try to match the endianness to our platform
error = flycaptureGetCameraRegister(context, IMAGE_DATA_FORMAT, regOut);
if (error != FLYCAPTURE_OK) {
throw new Exception("flycaptureGetCameraRegister() Error " + error);
}
int reg;
if (ByteOrder.nativeOrder().equals(ByteOrder.BIG_ENDIAN)) {
reg = regOut[0] | 0x1;
} else {
reg = regOut[0] & ~0x1;
}
error = flycaptureSetCameraRegister(context, IMAGE_DATA_FORMAT, reg);
if (error != FLYCAPTURE_OK) {
throw new Exception("flycaptureSetCameraRegister() Error " + error);
}
error = flycaptureSetBusSpeed(context, FLYCAPTURE_S_FASTEST, FLYCAPTURE_S_FASTEST);
if (error != FLYCAPTURE_OK) {
error = flycaptureSetBusSpeed(context,
FLYCAPTURE_ANY, FLYCAPTURE_ANY);
if (error != FLYCAPTURE_OK) {
throw new Exception("flycaptureSetBusSpeed() Error " + error);
}
}
if (gamma != 0.0) {
error = flycaptureSetCameraAbsProperty(context, FLYCAPTURE_GAMMA, (float)gamma);
if (error != FLYCAPTURE_OK) {
throw new Exception("flycaptureSetCameraAbsProperty() Error " + error + ": Could not set gamma.");
}
}
error = flycaptureGetCameraAbsProperty(context, FLYCAPTURE_GAMMA, gammaOut);
if (error != FLYCAPTURE_OK) {
gammaOut[0] = 2.2f;
}
error = flycaptureStart(context, c, f);
if (error != FLYCAPTURE_OK) {
throw new Exception("flycaptureStart() Error " + error);
}
error = flycaptureSetGrabTimeoutEx(context, timeout);
if (error != FLYCAPTURE_OK) {
throw new Exception("flycaptureSetGrabTimeoutEx() Error " + error);
}
}
private void waitForTriggerReady() throws Exception {
// wait for trigger to be ready...
long time = System.currentTimeMillis();
do {
int error = flycaptureGetCameraRegister(context, SOFTWARE_TRIGGER, regOut);
if (error != FLYCAPTURE_OK) {
throw new Exception("flycaptureGetCameraRegister() Error " + error);
}
if (System.currentTimeMillis() - time > timeout) {
break;
//throw new Exception("waitForTriggerReady() Error: Timeout occured.");
}
} while((regOut[0] >>> 31) != 0);
}
public void stop() throws Exception {
int error = flycaptureStop(context);
if (error != FLYCAPTURE_OK && error != FLYCAPTURE_FAILED) {
throw new Exception("flycaptureStop() Error " + error);
}
temp_image = null;
return_image = null;
timestamp = 0;
frameNumber = 0;
}
public void trigger() throws Exception {
waitForTriggerReady();
int error = flycaptureSetCameraRegister(context, SOFT_ASYNC_TRIGGER, 0x80000000);
if (error != FLYCAPTURE_OK) {
throw new Exception("flycaptureSetCameraRegister() Error " + error);
}
}
private int getNumChannels(int pixelFormat) {
switch (pixelFormat) {
case FLYCAPTURE_BGR:
case FLYCAPTURE_RGB8:
case FLYCAPTURE_RGB16:
case FLYCAPTURE_S_RGB16:
return 3;
case FLYCAPTURE_MONO8:
case FLYCAPTURE_MONO16:
case FLYCAPTURE_RAW8:
case FLYCAPTURE_RAW16:
case FLYCAPTURE_S_MONO16:
return 1;
case FLYCAPTURE_BGRU:
return 4;
case FLYCAPTURE_411YUV8:
case FLYCAPTURE_422YUV8:
case FLYCAPTURE_444YUV8:
default:
return -1;
}
}
private int getDepth(int pixelFormat) {
switch (pixelFormat) {
case FLYCAPTURE_BGR:
case FLYCAPTURE_RGB8:
case FLYCAPTURE_MONO8:
case FLYCAPTURE_RAW8:
case FLYCAPTURE_BGRU:
return IPL_DEPTH_8U;
case FLYCAPTURE_MONO16:
case FLYCAPTURE_RAW16:
case FLYCAPTURE_RGB16:
return IPL_DEPTH_16U;
case FLYCAPTURE_S_MONO16:
case FLYCAPTURE_S_RGB16:
return IPL_DEPTH_16S;
case FLYCAPTURE_411YUV8:
case FLYCAPTURE_422YUV8:
case FLYCAPTURE_444YUV8:
default:
return IPL_DEPTH_8U;
}
}
public Frame grab() throws Exception {
int error = flycaptureGrabImage2(context, raw_image);
if (error != FLYCAPTURE_OK) {
throw new Exception("flycaptureGrabImage2() Error " + error + " (Has start() been called?)");
}
int w = raw_image.iCols();
int h = raw_image.iRows();
int format = raw_image.pixelFormat();
int depth = getDepth(format);
int stride = raw_image.iRowInc();
int size = h*stride;
int numChannels = getNumChannels(format);
error = flycaptureGetCameraRegister(context, IMAGE_DATA_FORMAT, regOut);
if (error != FLYCAPTURE_OK) {
throw new Exception("flycaptureGetCameraRegister() Error " + error);
}
ByteOrder frameEndian = (regOut[0] & 0x1) != 0 ?
ByteOrder.BIG_ENDIAN : ByteOrder.LITTLE_ENDIAN;
boolean alreadySwapped = false;
boolean colorbayer = raw_image.bStippled();
boolean colorrgb = format == FLYCAPTURE_RGB8 || format == FLYCAPTURE_RGB16 ||
format == FLYCAPTURE_BGR || format == FLYCAPTURE_BGRU;
boolean coloryuv = format == FLYCAPTURE_411YUV8 || format == FLYCAPTURE_422YUV8 ||
format == FLYCAPTURE_444YUV8;
BytePointer imageData = raw_image.pData();
if ((depth == IPL_DEPTH_8U || frameEndian.equals(ByteOrder.nativeOrder())) &&
(imageMode == ImageMode.RAW || (imageMode == ImageMode.COLOR && numChannels == 3) ||
(imageMode == ImageMode.GRAY && numChannels == 1 && !colorbayer))) {
if (return_image == null) {
return_image = IplImage.createHeader(w, h, depth, numChannels);
}
return_image.widthStep(stride);
return_image.imageSize(size);
return_image.imageData(imageData);
} else {
if (return_image == null) {
return_image = IplImage.create(w, h, depth, imageMode == ImageMode.COLOR ? 3 : 1);
}
if (temp_image == null) {
if (imageMode == ImageMode.COLOR &&
(numChannels > 1 || depth > 8) && !coloryuv && !colorbayer) {
temp_image = IplImage.create(w, h, depth, numChannels);
} else if (imageMode == ImageMode.GRAY && colorbayer) {
temp_image = IplImage.create(w, h, depth, 3);
} else if (imageMode == ImageMode.GRAY && colorrgb) {
temp_image = IplImage.createHeader(w, h, depth, 3);
} else if (imageMode == ImageMode.COLOR && numChannels == 1 && !coloryuv && !colorbayer) {
temp_image = IplImage.createHeader(w, h, depth, 1);
} else {
temp_image = return_image;
}
}
conv_image.iRowInc(temp_image.widthStep());
conv_image.pData(temp_image.imageData());
if (depth == IPL_DEPTH_8U) {
conv_image.pixelFormat(imageMode == ImageMode.RAW ? FLYCAPTURE_RAW8 :
temp_image.nChannels() == 1 ? FLYCAPTURE_MONO8 : FLYCAPTURE_BGR);
} else {
conv_image.pixelFormat(imageMode == ImageMode.RAW ? FLYCAPTURE_RAW16 :
temp_image.nChannels() == 1 ? FLYCAPTURE_MONO16 : FLYCAPTURE_RGB16);
}
if (depth != IPL_DEPTH_8U && conv_image.pixelFormat() == format && conv_image.iRowInc() == stride) {
// we just need a copy to swap bytes..
ShortBuffer in = raw_image.getByteBuffer().order(frameEndian).asShortBuffer();
ShortBuffer out = temp_image.getByteBuffer().order(ByteOrder.nativeOrder()).asShortBuffer();
out.put(in);
alreadySwapped = true;
} else if ((imageMode == ImageMode.GRAY && colorrgb) ||
(imageMode == ImageMode.COLOR && numChannels == 1 && !coloryuv && !colorbayer)) {
temp_image.widthStep(stride);
temp_image.imageSize(size);
temp_image.imageData(imageData);
} else if (!colorrgb && (colorbayer || coloryuv || numChannels > 1)) {
error = flycaptureConvertImage(context, raw_image, conv_image);
if (error != FLYCAPTURE_OK) {
throw new Exception("flycaptureConvertImage() Error " + error);
}
}
if (!alreadySwapped && depth != IPL_DEPTH_8U &&
!frameEndian.equals(ByteOrder.nativeOrder())) {
// ack, the camera's endianness doesn't correspond to our machine ...
// swap bytes of 16-bit images
ByteBuffer bb = temp_image.getByteBuffer();
ShortBuffer in = bb.order(frameEndian).asShortBuffer();
ShortBuffer out = bb.order(ByteOrder.nativeOrder()).asShortBuffer();
out.put(in);
}
if (imageMode == ImageMode.COLOR && numChannels == 1 && !coloryuv && !colorbayer) {
cvCvtColor(temp_image, return_image, CV_GRAY2BGR);
} else if (imageMode == ImageMode.GRAY && (colorbayer || colorrgb)) {
cvCvtColor(temp_image, return_image, CV_BGR2GRAY);
}
}
error = flycaptureGetColorTileFormat(context, regOut);
if (error != FLYCAPTURE_OK) {
sensorPattern = -1L;
} else switch (regOut[0]) {
case FLYCAPTURE_STIPPLEDFORMAT_BGGR: sensorPattern = SENSOR_PATTERN_BGGR; break;
case FLYCAPTURE_STIPPLEDFORMAT_GBRG: sensorPattern = SENSOR_PATTERN_GBRG; break;
case FLYCAPTURE_STIPPLEDFORMAT_GRBG: sensorPattern = SENSOR_PATTERN_GRBG; break;
case FLYCAPTURE_STIPPLEDFORMAT_RGGB: sensorPattern = SENSOR_PATTERN_RGGB; break;
default: sensorPattern = -1L;
}
FlyCaptureTimestamp timeStamp = raw_image.timeStamp();
timestamp = timeStamp.ulSeconds() * 1000000L + timeStamp.ulMicroSeconds();
return converter.convert(return_image);
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/Frame.java
================================================
/*
* Copyright (C) 2015-2021 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.nio.Buffer;
import java.nio.ByteBuffer;
import java.nio.DoubleBuffer;
import java.nio.FloatBuffer;
import java.nio.IntBuffer;
import java.nio.LongBuffer;
import java.nio.ShortBuffer;
import java.util.EnumSet;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.DoublePointer;
import org.bytedeco.javacpp.FloatPointer;
import org.bytedeco.javacpp.IntPointer;
import org.bytedeco.javacpp.LongPointer;
import org.bytedeco.javacpp.Pointer;
import org.bytedeco.javacpp.ShortPointer;
import org.bytedeco.javacpp.indexer.ByteIndexer;
import org.bytedeco.javacpp.indexer.DoubleIndexer;
import org.bytedeco.javacpp.indexer.FloatIndexer;
import org.bytedeco.javacpp.indexer.Indexable;
import org.bytedeco.javacpp.indexer.Indexer;
import org.bytedeco.javacpp.indexer.IntIndexer;
import org.bytedeco.javacpp.indexer.LongIndexer;
import org.bytedeco.javacpp.indexer.ShortIndexer;
import org.bytedeco.javacpp.indexer.UByteIndexer;
import org.bytedeco.javacpp.indexer.UShortIndexer;
/**
* A class to manage the data of audio and video frames. It it used by
* {@link CanvasFrame}, {@link FrameGrabber}, {@link FrameRecorder}, and their
* subclasses. We can also make the link with other APIs, such as Android,
* Java 2D, FFmpeg, and OpenCV, via a {@link FrameConverter}.
*
* @author Samuel Audet
*/
public class Frame implements AutoCloseable, Indexable {
/** A flag set by a FrameGrabber or a FrameRecorder to indicate a key frame. */
public boolean keyFrame;
/** The type of the image frame ('I', 'P', 'B', etc). */
public char pictType;
/** Constants to be used for {@link #imageDepth}. */
public static final int
DEPTH_BYTE = -8,
DEPTH_UBYTE = 8,
DEPTH_SHORT = -16,
DEPTH_USHORT = 16,
DEPTH_INT = -32,
DEPTH_LONG = -64,
DEPTH_FLOAT = 32,
DEPTH_DOUBLE = 64;
/** Constants defining data type in the frame. */
public static enum Type {
VIDEO,
AUDIO,
DATA,
SUBTITLE,
ATTACHMENT
}
/** Information associated with the {@link #image} field. */
public int imageWidth, imageHeight, imageDepth, imageChannels, imageStride;
/**
* Buffers to hold image pixels from multiple channels for a video frame.
* Most of the software supports packed data only, but an array is provided
* to allow users to store images in a planar format as well.
*/
public Buffer[] image;
/** Information associated with the {@link #samples} field. */
public int sampleRate, audioChannels;
/** Buffers to hold audio samples from multiple channels for an audio frame. */
public Buffer[] samples;
/** Buffer to hold a data stream associated with a frame. */
public ByteBuffer data;
/** Stream number the audio|video|other data is associated with. */
public int streamIndex;
/** The type of the stream. */
public Type type;
/** The underlying data object, for example, Pointer, AVFrame, IplImage, or Mat. */
public Object opaque;
/** Timestamp of the frame creation in microseconds. */
public long timestamp;
/** Returns {@code Math.abs(depth) / 8}. */
public static int pixelSize(int depth) {
return Math.abs(depth) / 8;
}
/** Empty constructor. */
public Frame() { }
/** Allocates a new packed image frame in native memory where rows are 8-byte aligned. */
public Frame(int width, int height, int depth, int channels) {
this(width, height, depth, channels, ((width * channels * pixelSize(depth) + 7) & ~7) / pixelSize(depth));
}
public Frame(int width, int height, int depth, int channels, int imageStride) {
this.imageWidth = width;
this.imageHeight = height;
this.imageDepth = depth;
this.imageChannels = channels;
this.imageStride = imageStride;
this.pictType = '\0';
this.image = new Buffer[1];
this.data = null;
this.streamIndex = -1;
this.type = null;
Pointer pointer = new BytePointer(imageHeight * imageStride * pixelSize(depth));
ByteBuffer buffer = pointer.asByteBuffer();
switch (imageDepth) {
case DEPTH_BYTE:
case DEPTH_UBYTE: image[0] = buffer; break;
case DEPTH_SHORT:
case DEPTH_USHORT: image[0] = buffer.asShortBuffer(); break;
case DEPTH_INT: image[0] = buffer.asIntBuffer(); break;
case DEPTH_LONG: image[0] = buffer.asLongBuffer(); break;
case DEPTH_FLOAT: image[0] = buffer.asFloatBuffer(); break;
case DEPTH_DOUBLE: image[0] = buffer.asDoubleBuffer(); break;
default: throw new UnsupportedOperationException("Unsupported depth value: " + imageDepth);
}
opaque = new Pointer[] {pointer.retainReference()};
}
/** Returns {@code createIndexer(true, 0)}. */
public I createIndexer() {
return (I)createIndexer(true, 0);
}
@Override public I createIndexer(boolean direct) {
return (I)createIndexer(direct, 0);
}
/** Returns an {@link Indexer} for the i th image plane. */
public I createIndexer(boolean direct, int i) {
long[] sizes = {imageHeight, imageWidth, imageChannels};
long[] strides = {imageStride, imageChannels, 1};
Buffer buffer = image[i];
Object array = buffer.hasArray() ? buffer.array() : null;
switch (imageDepth) {
case DEPTH_UBYTE:
return array != null ? (I)UByteIndexer.create((byte[])array, sizes, strides).indexable(this)
: direct ? (I)UByteIndexer.create((ByteBuffer)buffer, sizes, strides).indexable(this)
: (I)UByteIndexer.create(new BytePointer((ByteBuffer)buffer), sizes, strides, false).indexable(this);
case DEPTH_BYTE:
return array != null ? (I)ByteIndexer.create((byte[])array, sizes, strides).indexable(this)
: direct ? (I)ByteIndexer.create((ByteBuffer)buffer, sizes, strides).indexable(this)
: (I)ByteIndexer.create(new BytePointer((ByteBuffer)buffer), sizes, strides, false).indexable(this);
case DEPTH_USHORT:
return array != null ? (I)UShortIndexer.create((short[])array, sizes, strides).indexable(this)
: direct ? (I)UShortIndexer.create((ShortBuffer)buffer, sizes, strides).indexable(this)
: (I)UShortIndexer.create(new ShortPointer((ShortBuffer)buffer), sizes, strides, false).indexable(this);
case DEPTH_SHORT:
return array != null ? (I)ShortIndexer.create((short[])array, sizes, strides).indexable(this)
: direct ? (I)ShortIndexer.create((ShortBuffer)buffer, sizes, strides).indexable(this)
: (I)ShortIndexer.create(new ShortPointer((ShortBuffer)buffer), sizes, strides, false).indexable(this);
case DEPTH_INT:
return array != null ? (I)IntIndexer.create((int[])array, sizes, strides).indexable(this)
: direct ? (I)IntIndexer.create((IntBuffer)buffer, sizes, strides).indexable(this)
: (I)IntIndexer.create(new IntPointer((IntBuffer)buffer), sizes, strides, false).indexable(this);
case DEPTH_LONG:
return array != null ? (I)LongIndexer.create((long[])array, sizes, strides).indexable(this)
: direct ? (I)LongIndexer.create((LongBuffer)buffer, sizes, strides).indexable(this)
: (I)LongIndexer.create(new LongPointer((LongBuffer)buffer), sizes, strides, false).indexable(this);
case DEPTH_FLOAT:
return array != null ? (I)FloatIndexer.create((float[])array, sizes, strides).indexable(this)
: direct ? (I)FloatIndexer.create((FloatBuffer)buffer, sizes, strides).indexable(this)
: (I)FloatIndexer.create(new FloatPointer((FloatBuffer)buffer), sizes, strides, false).indexable(this);
case DEPTH_DOUBLE:
return array != null ? (I)DoubleIndexer.create((double[])array, sizes, strides).indexable(this)
: direct ? (I)DoubleIndexer.create((DoubleBuffer)buffer, sizes, strides).indexable(this)
: (I)DoubleIndexer.create(new DoublePointer((DoubleBuffer)buffer), sizes, strides, false).indexable(this);
default: assert false;
}
return null;
}
/**Care must be taken if this method is to be used in conjunction with movie recordings.
* Cloning a frame containing a full HD picture (alpha channel included) would take 1920 x 1080 * 4 = 8.294.400 Bytes.
* Expect a heap overflow exception when using this method without cleaning up.
*
* @return A deep copy of this frame.
* @see {@link #cloneBufferArray}
*
* Extension proposed by Dragos Dutu
* */
@Override
public Frame clone() {
Frame newFrame = new Frame();
// Video part
newFrame.imageWidth = imageWidth;
newFrame.imageHeight = imageHeight;
newFrame.imageDepth = imageDepth;
newFrame.imageChannels = imageChannels;
newFrame.imageStride = imageStride;
newFrame.keyFrame = keyFrame;
newFrame.pictType = pictType;
newFrame.streamIndex = streamIndex;
newFrame.type = type;
newFrame.opaque = new Pointer[3];
if (image != null) {
newFrame.image = new Buffer[image.length];
((Pointer[])newFrame.opaque)[0] = cloneBufferArray(image, newFrame.image);
}
// Audio part
newFrame.audioChannels = audioChannels;
newFrame.sampleRate = sampleRate;
if (samples != null) {
newFrame.samples = new Buffer[samples.length];
((Pointer[])newFrame.opaque)[1] = cloneBufferArray(samples, newFrame.samples);
}
// Other data streams
if (data != null) {
ByteBuffer[] dst = new ByteBuffer[1];
((Pointer[])newFrame.opaque)[2] = cloneBufferArray(new ByteBuffer[]{data}, dst);
newFrame.data = dst[0];
}
// Add timestamp
newFrame.timestamp = timestamp;
return newFrame;
}
/**
* This private method takes a buffer array as input and returns a deep copy.
* It is assumed that all buffers in the input array are of the same subclass.
*
* @param srcBuffers - Buffer array to be cloned
* @param clonedBuffers - Buffer array to fill with clones
* @return Opaque object to store
*
* @author Extension proposed by Dragos Dutu
*/
private static Pointer cloneBufferArray(Buffer[] srcBuffers, Buffer[] clonedBuffers) {
Pointer opaque = null;
if (srcBuffers != null && srcBuffers.length > 0) {
int totalCapacity = 0;
for (int i = 0; i < srcBuffers.length; i++) {
srcBuffers[i].rewind();
totalCapacity += srcBuffers[i].capacity();
}
/*
* In order to optimize the transfer we need a type check.
*
* Most CPUs support hardware memory transfer for different data
* types, so it's faster to copy more bytes at once rather
* than one byte per iteration as in case of ByteBuffer.
*
* For example, Intel CPUs support MOVSB (byte transfer), MOVSW
* (word transfer), MOVSD (double word transfer), MOVSS (32 bit
* scalar single precision floating point), MOVSQ (quad word
* transfer) and so on...
*
* Type checking may be improved by changing the order in
* which a buffer is checked against. If it's likely that the
* expected buffer is of type "ShortBuffer", then it should be
* checked at first place.
*
*/
if (srcBuffers[0] instanceof ByteBuffer) {
BytePointer pointer = new BytePointer(totalCapacity);
for (int i = 0; i < srcBuffers.length; i++) {
clonedBuffers[i] = pointer.limit(pointer.position() + srcBuffers[i].limit())
.asBuffer().put((ByteBuffer)srcBuffers[i]);
pointer.position(pointer.limit());
}
opaque = pointer;
} else if (srcBuffers[0] instanceof ShortBuffer) {
ShortPointer pointer = new ShortPointer(totalCapacity);
for (int i = 0; i < srcBuffers.length; i++) {
clonedBuffers[i] = pointer.limit(pointer.position() + srcBuffers[i].limit())
.asBuffer().put((ShortBuffer)srcBuffers[i]);
pointer.position(pointer.limit());
}
opaque = pointer;
} else if (srcBuffers[0] instanceof IntBuffer) {
IntPointer pointer = new IntPointer(totalCapacity);
for (int i = 0; i < srcBuffers.length; i++) {
clonedBuffers[i] = pointer.limit(pointer.position() + srcBuffers[i].limit())
.asBuffer().put((IntBuffer)srcBuffers[i]);
pointer.position(pointer.limit());
}
opaque = pointer;
} else if (srcBuffers[0] instanceof LongBuffer) {
LongPointer pointer = new LongPointer(totalCapacity);
for (int i = 0; i < srcBuffers.length; i++) {
clonedBuffers[i] = pointer.limit(pointer.position() + srcBuffers[i].limit())
.asBuffer().put((LongBuffer)srcBuffers[i]);
pointer.position(pointer.limit());
}
opaque = pointer;
} else if (srcBuffers[0] instanceof FloatBuffer) {
FloatPointer pointer = new FloatPointer(totalCapacity);
for (int i = 0; i < srcBuffers.length; i++) {
clonedBuffers[i] = pointer.limit(pointer.position() + srcBuffers[i].limit())
.asBuffer().put((FloatBuffer)srcBuffers[i]);
pointer.position(pointer.limit());
}
opaque = pointer;
} else if (srcBuffers[0] instanceof DoubleBuffer) {
DoublePointer pointer = new DoublePointer(totalCapacity);
for (int i = 0; i < srcBuffers.length; i++) {
clonedBuffers[i] = pointer.limit(pointer.position() + srcBuffers[i].limit())
.asBuffer().put((DoubleBuffer)srcBuffers[i]);
pointer.position(pointer.limit());
}
opaque = pointer;
}
for (int i = 0; i < srcBuffers.length; i++) {
srcBuffers[i].rewind();
clonedBuffers[i].rewind();
}
}
if (opaque != null) {
opaque.retainReference();
}
return opaque;
}
/** Returns types of data containing in the frame */
public EnumSet getTypes() {
EnumSet type = EnumSet.noneOf(Type.class);
if (image != null) type.add(Type.VIDEO);
if (samples != null) type.add(Type.AUDIO);
if (data != null) type.add(Type.DATA);
return type;
}
@Override public void close() {
if (opaque instanceof Pointer[]) {
for (Pointer p : (Pointer[])opaque) {
if (p != null) {
p.releaseReference();
p = null;
}
}
opaque = null;
}
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/FrameConverter.java
================================================
/*
* Copyright (C) 2015-2021 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
/**
* Defines two methods to convert between a {@link Frame} and another generic
* data object that can contain the same data. The idea with this design is
* to allow users to convert easily between multiple potentially mutually
* exclusive types of image data objects over which we have no control. Because
* of this, and for performance reasons, any object returned by this class is
* guaranteed to remain valid only until the next call to {@code convert()},
* anywhere in a chain of {@code FrameConverter} objects, and only as long as
* the latter themselves are not closed or garbage collected.
*
* @author Samuel Audet
*/
public abstract class FrameConverter implements AutoCloseable {
protected Frame frame;
public abstract Frame convert(F f);
public abstract F convert(Frame frame);
@Override public void close() {
if (frame != null) {
frame.close();
frame = null;
}
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/FrameFilter.java
================================================
/*
* Copyright (C) 2015-2018 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.io.Closeable;
import java.io.IOException;
/**
* A frame processor that may filter video and audio frames, or both.
* After calling {@link #start()}, we can add frames to the graph with
* {@link #push(Frame)} and get the filtered ones with {@link #pull()}.
*
* @author Samuel Audet
*/
public abstract class FrameFilter implements Closeable {
public static FrameFilter createDefault(String filtersDescr, int imageWidth, int imageHeight) throws Exception {
return new FFmpegFrameFilter(filtersDescr, imageWidth, imageHeight);
}
protected String filters;
protected int imageWidth;
protected int imageHeight;
protected int pixelFormat;
protected double frameRate;
protected double aspectRatio;
protected int videoInputs;
protected String[] videoFilterArgs;
protected String afilters;
protected int audioChannels;
protected int sampleFormat;
protected int sampleRate;
protected int audioInputs;
protected String[] audioFilterArgs;
public String getFilters() {
return filters;
}
public void setFilters(String filters) {
this.filters = filters;
}
public int getImageWidth() {
return imageWidth;
}
public void setImageWidth(int imageWidth) {
this.imageWidth = imageWidth;
}
public int getImageHeight() {
return imageHeight;
}
public void setImageHeight(int imageHeight) {
this.imageHeight = imageHeight;
}
public int getPixelFormat() {
return pixelFormat;
}
public void setPixelFormat(int pixelFormat) {
this.pixelFormat = pixelFormat;
}
public double getFrameRate() {
return frameRate;
}
public void setFrameRate(double frameRate) {
this.frameRate = frameRate;
}
public double getAspectRatio() {
return aspectRatio;
}
public void setAspectRatio(double aspectRatio) {
this.aspectRatio = aspectRatio;
}
public int getVideoInputs() {
return videoInputs;
}
public void setVideoInputs(int videoInputs) {
this.videoInputs = videoInputs;
}
public String[] getVideoFilterArgs() {
return videoFilterArgs;
}
public void setVideoFilterArgs(String[] videoFilterArgs) {
this.videoFilterArgs = videoFilterArgs;
}
public int getAudioChannels() {
return audioChannels;
}
public void setAudioChannels(int audioChannels) {
this.audioChannels = audioChannels;
}
public int getSampleFormat() {
return sampleFormat;
}
public void setSampleFormat(int sampleFormat) {
this.sampleFormat = sampleFormat;
}
public int getSampleRate() {
return sampleRate;
}
public void setSampleRate(int sampleRate) {
this.sampleRate = sampleRate;
}
public int getAudioInputs() {
return audioInputs;
}
public void setAudioInputs(int audioInputs) {
this.audioInputs = audioInputs;
}
public String[] getAudioFilterArgs() {
return audioFilterArgs;
}
public void setAudioFilterArgs(String[] audioFilterArgs) {
this.audioFilterArgs = audioFilterArgs;
}
public static class Exception extends IOException {
public Exception(String message) { super(message); }
public Exception(String message, Throwable cause) { super(message, cause); }
}
public abstract void start() throws Exception;
public abstract void stop() throws Exception;
public abstract void push(Frame frame) throws Exception;
public abstract Frame pull() throws Exception;
public abstract void release() throws Exception;
@Override public void close() throws Exception {
stop();
release();
}
public void restart() throws Exception {
stop();
start();
}
public void flush() throws Exception {
while (pull() != null);
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/FrameGrabber.java
================================================
/*
* Copyright (C) 2009-2022 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.beans.PropertyEditorSupport;
import java.io.Closeable;
import java.io.File;
import java.io.IOException;
import java.lang.reflect.InvocationTargetException;
import java.nio.Buffer;
import java.nio.charset.Charset;
import java.util.Arrays;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
/**
*
* @author Samuel Audet
*/
public abstract class FrameGrabber implements Closeable {
public static final List list = new LinkedList(Arrays.asList(new String[] {
"DC1394", "FlyCapture", "FlyCapture2", "OpenKinect", "OpenKinect2", "RealSense", "RealSense2", "PS3Eye", "VideoInput", "OpenCV", "FFmpeg", "IPCamera" }));
public static void init() {
for (String name : list) {
try {
Class c = get(name);
c.getMethod("tryLoad").invoke(null);
} catch (Throwable t) {
continue;
}
}
}
public static Class getDefault() {
// select first frame grabber that can load and that may have some cameras..
for (String name : list) {
try {
Class c = get(name);
c.getMethod("tryLoad").invoke(null);
boolean mayContainCameras = false;
try {
String[] s = (String[])c.getMethod("getDeviceDescriptions").invoke(null);
if (s.length > 0) {
mayContainCameras = true;
}
} catch (Throwable t) {
if (t.getCause() instanceof UnsupportedOperationException) {
mayContainCameras = true;
}
}
if (mayContainCameras) {
return c;
}
} catch (Throwable t) {
continue;
}
}
return null;
}
public static Class get(String className) throws Exception {
className = FrameGrabber.class.getPackage().getName() + "." + className;
try {
return Class.forName(className).asSubclass(FrameGrabber.class);
} catch (ClassNotFoundException e) {
String className2 = className + "FrameGrabber";
try {
return Class.forName(className2).asSubclass(FrameGrabber.class);
} catch (ClassNotFoundException ex) {
throw new Exception("Could not get FrameGrabber class for " + className + " or " + className2, e);
}
}
}
public static FrameGrabber create(Class c, Class p, Object o) throws Exception {
Throwable cause = null;
try {
return c.getConstructor(p).newInstance(o);
} catch (InstantiationException ex) {
cause = ex;
} catch (IllegalAccessException ex) {
cause = ex;
} catch (IllegalArgumentException ex) {
cause = ex;
} catch (NoSuchMethodException ex) {
cause = ex;
} catch (InvocationTargetException ex) {
cause = ex.getCause();
}
throw new Exception("Could not create new " + c.getSimpleName() + "(" + o + ")", cause);
}
public static FrameGrabber createDefault(File deviceFile) throws Exception {
return create(getDefault(), File.class, deviceFile);
}
public static FrameGrabber createDefault(String devicePath) throws Exception {
return create(getDefault(), String.class, devicePath);
}
public static FrameGrabber createDefault(int deviceNumber) throws Exception {
try {
return create(getDefault(), int.class, deviceNumber);
} catch (Exception ex) {
return create(getDefault(), Integer.class, deviceNumber);
}
}
public static FrameGrabber create(String className, File deviceFile) throws Exception {
return create(get(className), File.class, deviceFile);
}
public static FrameGrabber create(String className, String devicePath) throws Exception {
return create(get(className), String.class, devicePath);
}
public static FrameGrabber create(String className, int deviceNumber) throws Exception {
try {
return create(get(className), int.class, deviceNumber);
} catch (Exception ex) {
return create(get(className), Integer.class, deviceNumber);
}
}
public static class PropertyEditor extends PropertyEditorSupport {
@Override public String getAsText() {
Class c = (Class)getValue();
return c == null ? "null" : c.getSimpleName().split("FrameGrabber")[0];
}
@Override public void setAsText(String s) {
if (s == null) {
setValue(null);
}
try {
setValue(get(s));
} catch (Exception ex) {
throw new IllegalArgumentException(ex);
}
}
@Override public String[] getTags() {
return list.toArray(new String[list.size()]);
}
}
public static enum ImageMode {
COLOR, GRAY, RAW
}
public static enum SampleMode {
SHORT, FLOAT, RAW
}
public static final long
SENSOR_PATTERN_RGGB = 0,
SENSOR_PATTERN_GBRG = (1L << 32),
SENSOR_PATTERN_GRBG = 1,
SENSOR_PATTERN_BGGR = (1L << 32) | 1;
protected int videoStream = -1, audioStream = -1;
protected int videoDisposition = 0, audioDisposition = 0;
protected String format = null, videoCodecName = null, audioCodecName = null;
protected int imageWidth = 0, imageHeight = 0, audioChannels = 0;
protected ImageMode imageMode = ImageMode.COLOR;
protected long sensorPattern = -1L;
protected int pixelFormat = -1, videoCodec, videoBitrate = 0, imageScalingFlags = 0;
protected double aspectRatio = 0, frameRate = 0;
protected SampleMode sampleMode = SampleMode.SHORT;
protected int sampleFormat = -1, audioCodec, audioBitrate = 0, sampleRate = 0;
protected boolean triggerMode = false;
protected int bpp = 0;
protected int timeout = 10000;
protected int numBuffers = 4;
protected double gamma = 0.0;
protected boolean deinterlace = false;
protected Charset charset = Charset.defaultCharset();
protected Map options = new HashMap();
protected Map videoOptions = new HashMap();
protected Map audioOptions = new HashMap();
protected Map metadata = new HashMap();
protected Map videoMetadata = new HashMap();
protected Map audioMetadata = new HashMap();
protected Map videoSideData = new HashMap();
protected Map audioSideData = new HashMap();
protected int frameNumber = 0;
protected long timestamp = 0;
protected int maxDelay = -1;
protected long startTime = 0;
public int getVideoStream() {
return videoStream;
}
public void setVideoStream(int videoStream) {
this.videoStream = videoStream;
}
public int getAudioStream() {
return audioStream;
}
public void setAudioStream(int audioStream) {
this.audioStream = audioStream;
}
public void setVideoDisposition(int videoDisposition) {
this.videoDisposition = videoDisposition;
}
public int getVideoDisposition() {
return videoDisposition;
}
public void setAudioDisposition(int audioDisposition) {
this.audioDisposition = audioDisposition;
}
public int getAudioDisposition() {
return audioDisposition;
}
public String getFormat() {
return format;
}
public void setFormat(String format) {
this.format = format;
}
public String getVideoCodecName() {
return videoCodecName;
}
public void setVideoCodecName(String videoCodecName) {
this.videoCodecName = videoCodecName;
}
public String getAudioCodecName() {
return audioCodecName;
}
public void setAudioCodecName(String audioCodecName) {
this.audioCodecName = audioCodecName;
}
public int getImageWidth() {
return imageWidth;
}
public void setImageWidth(int imageWidth) {
this.imageWidth = imageWidth;
}
public int getImageHeight() {
return imageHeight;
}
public void setImageHeight(int imageHeight) {
this.imageHeight = imageHeight;
}
public int getAudioChannels() {
return audioChannels;
}
public void setAudioChannels(int audioChannels) {
this.audioChannels = audioChannels;
}
public ImageMode getImageMode() {
return imageMode;
}
public void setImageMode(ImageMode imageMode) {
this.imageMode = imageMode;
}
public long getSensorPattern() {
return sensorPattern;
}
public void setSensorPattern(long sensorPattern) {
this.sensorPattern = sensorPattern;
}
public int getPixelFormat() {
return pixelFormat;
}
public void setPixelFormat(int pixelFormat) {
this.pixelFormat = pixelFormat;
}
public int getVideoCodec() {
return videoCodec;
}
public void setVideoCodec(int videoCodec) {
this.videoCodec = videoCodec;
}
public int getVideoBitrate() {
return videoBitrate;
}
public void setVideoBitrate(int videoBitrate) {
this.videoBitrate = videoBitrate;
}
public int getImageScalingFlags() {
return imageScalingFlags;
}
public void setImageScalingFlags(int imageScalingFlags) {
this.imageScalingFlags = imageScalingFlags;
}
public double getAspectRatio() {
return aspectRatio;
}
public void setAspectRatio(double aspectRatio) {
this.aspectRatio = aspectRatio;
}
public double getFrameRate() {
return frameRate;
}
public void setFrameRate(double frameRate) {
this.frameRate = frameRate;
}
public int getAudioCodec() {
return audioCodec;
}
public void setAudioCodec(int audioCodec) {
this.audioCodec = audioCodec;
}
public int getAudioBitrate() {
return audioBitrate;
}
public void setAudioBitrate(int audioBitrate) {
this.audioBitrate = audioBitrate;
}
public SampleMode getSampleMode() {
return sampleMode;
}
public void setSampleMode(SampleMode samplesMode) {
this.sampleMode = samplesMode;
}
public int getSampleFormat() {
return sampleFormat;
}
public void setSampleFormat(int sampleFormat) {
this.sampleFormat = sampleFormat;
}
public int getSampleRate() {
return sampleRate;
}
public void setSampleRate(int sampleRate) {
this.sampleRate = sampleRate;
}
public boolean isTriggerMode() {
return triggerMode;
}
public void setTriggerMode(boolean triggerMode) {
this.triggerMode = triggerMode;
}
public int getBitsPerPixel() {
return bpp;
}
public void setBitsPerPixel(int bitsPerPixel) {
this.bpp = bitsPerPixel;
}
public int getTimeout() {
return timeout;
}
public void setTimeout(int timeout) {
this.timeout = timeout;
}
public int getNumBuffers() {
return numBuffers;
}
public void setNumBuffers(int numBuffers) {
this.numBuffers = numBuffers;
}
public double getGamma() {
return gamma;
}
public void setGamma(double gamma) {
this.gamma = gamma;
}
public boolean isDeinterlace() {
return deinterlace;
}
public void setDeinterlace(boolean deinterlace) {
this.deinterlace = deinterlace;
}
public Charset getCharset() {
return charset;
}
public void setCharset(Charset charset) {
this.charset = charset;
}
public Map getOptions() {
return options;
}
public void setOptions(Map options) {
this.options = options;
}
public Map getVideoOptions() {
return videoOptions;
}
public void setVideoOptions(Map options) {
this.videoOptions = options;
}
public Map getAudioOptions() {
return audioOptions;
}
public void setAudioOptions(Map options) {
this.audioOptions = options;
}
public Map getMetadata() {
return metadata;
}
public void setMetadata(Map metadata) {
this.metadata = metadata;
}
public Map getVideoMetadata() {
return videoMetadata;
}
public void setVideoMetadata(Map metadata) {
this.videoMetadata = metadata;
}
public Map getAudioMetadata() {
return audioMetadata;
}
public void setAudioMetadata(Map metadata) {
this.audioMetadata = metadata;
}
public String getOption(String key) {
return options.get(key);
}
public void setOption(String key, String value) {
options.put(key, value);
}
public String getVideoOption(String key) {
return videoOptions.get(key);
}
public void setVideoOption(String key, String value) {
videoOptions.put(key, value);
}
public String getAudioOption(String key) {
return audioOptions.get(key);
}
public void setAudioOption(String key, String value) {
audioOptions.put(key, value);
}
public String getMetadata(String key) {
return metadata.get(key);
}
public void setMetadata(String key, String value) {
metadata.put(key, value);
}
public String getVideoMetadata(String key) {
return videoMetadata.get(key);
}
public void setVideoMetadata(String key, String value) {
videoMetadata.put(key, value);
}
public String getAudioMetadata(String key) {
return audioMetadata.get(key);
}
public void setAudioMetadata(String key, String value) {
audioMetadata.put(key, value);
}
public Map getVideoSideData() {
return videoSideData;
}
public void setVideoSideData(Map videoSideData) {
this.videoSideData = videoSideData;
}
public Buffer getVideoSideData(String key) {
return videoSideData.get(key);
}
public void setVideoSideData(String key, Buffer value) {
videoSideData.put(key, value);
}
public Map getAudioSideData() {
return audioSideData;
}
public void setAudioSideData(Map audioSideData) {
this.audioSideData = audioSideData;
}
public Buffer getAudioSideData(String key) {
return audioSideData.get(key);
}
public void setAudioSideData(String key, Buffer value) {
audioSideData.put(key, value);
}
public int getFrameNumber() {
return frameNumber;
}
public void setFrameNumber(int frameNumber) throws Exception {
this.frameNumber = frameNumber;
}
public long getTimestamp() {
return timestamp;
}
public void setTimestamp(long timestamp) throws Exception {
this.timestamp = timestamp;
}
public int getMaxDelay() {
return maxDelay;
}
public void setMaxDelay(int maxDelay) {
this.maxDelay = maxDelay;
}
public int getLengthInFrames() {
return 0;
}
public long getLengthInTime() {
return 0;
}
public static class Exception extends IOException {
public Exception(String message) { super(message); }
public Exception(String message, Throwable cause) { super(message, cause); }
}
public abstract void start() throws Exception;
public abstract void stop() throws Exception;
public abstract void trigger() throws Exception;
@Override public void close() throws Exception {
stop();
release();
}
/**
* Each call to grab stores the new image in the memory address for the previously returned frame.
* grabber.grab() == grabber.grab()
*
* Why? future = null;
private Frame delayedFrame = null;
private long delayedTime = 0;
public void delayedGrab(final long delayTime) {
delayedFrame = null;
delayedTime = 0;
final long start = System.nanoTime()/1000;
if (future != null && !future.isDone()) {
return;
}
future = executor.submit(new Callable() { public Void call() throws Exception {
do {
delayedFrame = grab();
delayedTime = System.nanoTime()/1000 - start;
} while (delayedTime < delayTime);
return null;
}});
}
public long getDelayedTime() throws InterruptedException, ExecutionException {
if (future == null) {
return 0;
}
future.get();
return delayedTime;
}
public Frame getDelayedFrame() throws InterruptedException, ExecutionException {
if (future == null) {
return null;
}
future.get();
return delayedFrame;
}
public static class Array {
// declared protected to force users to use createArray(), which
// can be overridden without changing the calling code...
protected Array(FrameGrabber[] frameGrabbers) {
setFrameGrabbers(frameGrabbers);
}
private Frame[] grabbedFrames = null;
private long[] latencies = null;
private long[] bestLatencies = null;
private long lastNewestTimestamp = 0;
private long bestInterval = Long.MAX_VALUE;
protected FrameGrabber[] frameGrabbers = null;
public FrameGrabber[] getFrameGrabbers() {
return frameGrabbers;
}
public void setFrameGrabbers(FrameGrabber[] frameGrabbers) {
this.frameGrabbers = frameGrabbers;
grabbedFrames = new Frame[frameGrabbers.length];
latencies = new long[frameGrabbers.length];
bestLatencies = null;
lastNewestTimestamp = 0;
}
public int size() {
return frameGrabbers.length;
}
public void start() throws Exception {
for (FrameGrabber f : frameGrabbers) {
f.start();
}
}
public void stop() throws Exception {
for (FrameGrabber f : frameGrabbers) {
f.stop();
}
}
// should be overriden to implement a broadcast trigger...
public void trigger() throws Exception {
for (FrameGrabber f : frameGrabbers) {
if (f.isTriggerMode()) {
f.trigger();
}
}
}
// should be overriden to implement a broadcast grab...
public Frame[] grab() throws Exception {
if (frameGrabbers.length == 1) {
grabbedFrames[0] = frameGrabbers[0].grab();
return grabbedFrames;
}
// assume we sometimes get perfectly synchronized images,
// so save the best latencies we find as the perfectly
// synchronized case, so we know what to aim for in
// cases of missing/dropped frames ...
long newestTimestamp = 0;
boolean unsynchronized = false;
for (int i = 0; i < frameGrabbers.length; i++) {
grabbedFrames[i] = frameGrabbers[i].grab();
if (grabbedFrames[i] != null) {
newestTimestamp = Math.max(newestTimestamp, frameGrabbers[i].getTimestamp());
}
if (frameGrabbers[i].getClass() != frameGrabbers[(i + 1) % frameGrabbers.length].getClass()) {
// assume we can't synchronize different types of cameras with each other
unsynchronized = true;
}
}
if (unsynchronized) {
return grabbedFrames;
}
for (int i = 0; i < frameGrabbers.length; i++) {
if (grabbedFrames[i] != null) {
latencies[i] = newestTimestamp - Math.max(0, frameGrabbers[i].getTimestamp());
}
}
if (bestLatencies == null) {
bestLatencies = Arrays.copyOf(latencies, latencies.length);
} else {
int sum1 = 0, sum2 = 0;
for (int i = 0; i < frameGrabbers.length; i++) {
sum1 += latencies[i];
sum2 += bestLatencies[i];
}
if (sum1 < sum2) {
bestLatencies = Arrays.copyOf(latencies, latencies.length);
}
}
// we cannot have latencies higher than the time between frames..
// or something too close to it anyway... 90% is good?
bestInterval = Math.min(bestInterval, newestTimestamp-lastNewestTimestamp);
for (int i = 0; i < bestLatencies.length; i++) {
bestLatencies[i] = Math.min(bestLatencies[i], bestInterval*9/10);
}
// try to synchronize by attempting to land within 10% of
// the bestLatencies looking up to 2 frames ahead ...
for (int j = 0; j < 2; j++) {
for (int i = 0; i < frameGrabbers.length; i++) {
if (frameGrabbers[i].isTriggerMode() || grabbedFrames[i] == null) {
continue;
}
int latency = (int)(newestTimestamp - Math.max(0, frameGrabbers[i].getTimestamp()));
while (latency-bestLatencies[i] > 0.1*bestLatencies[i]) {
grabbedFrames[i] = frameGrabbers[i].grab();
if (grabbedFrames[i] == null) {
break;
}
latency = (int)(newestTimestamp - Math.max(0, frameGrabbers[i].getTimestamp()));
if (latency < 0) {
// woops, a camera seems to have dropped a frame somewhere...
// bump up the newestTimestamp
newestTimestamp = Math.max(0, frameGrabbers[i].getTimestamp());
break;
}
}
}
}
//for (int i = 0; i < frameGrabbers.length; i++) {
// long latency = newestTimestamp - Math.max(0, frameGrabbers[i].getTimestamp());
// System.out.print(bestLatencies[i] + " " + latency + " ");
//}
//System.out.println(" " + bestInterval);
lastNewestTimestamp = newestTimestamp;
return grabbedFrames;
}
public void release() throws Exception {
for (FrameGrabber f : frameGrabbers) {
f.release();
}
}
}
public Array createArray(FrameGrabber[] frameGrabbers) {
return new Array(frameGrabbers);
}
/** Returns {@code frame = grab()} after {@code waitForTimestamp(frame)}. */
public Frame grabAtFrameRate() throws Exception, InterruptedException {
Frame frame = grab();
if (frame != null) {
waitForTimestamp(frame);
}
return frame;
}
/** Returns true if {@code Thread.sleep()} had to be called. */
public boolean waitForTimestamp(Frame frame) throws InterruptedException {
if (startTime == 0) {
startTime = System.nanoTime() / 1000 - frame.timestamp;
} else {
long delay = frame.timestamp - (System.nanoTime() / 1000 - startTime);
if (delay > 0) {
Thread.sleep(delay / 1000, (int)(delay % 1000) * 1000);
return true;
}
}
return false;
}
public void resetStartTime() {
startTime = 0;
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/FrameRecorder.java
================================================
/*
* Copyright (C) 2009-2023 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.io.Closeable;
import java.io.File;
import java.io.IOException;
import java.lang.reflect.InvocationTargetException;
import java.nio.Buffer;
import java.nio.charset.Charset;
import java.util.Arrays;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
/**
*
* @author Samuel Audet
*/
public abstract class FrameRecorder implements Closeable {
public static final List list = new LinkedList(Arrays.asList(new String[] { "FFmpeg", "OpenCV" }));
public static void init() {
for (String name : list) {
try {
Class c = get(name);
c.getMethod("tryLoad").invoke(null);
} catch (Throwable t) { }
}
}
public static Class getDefault() {
// select first frame recorder that can load..
for (String name : list) {
try {
Class c = get(name);
c.getMethod("tryLoad").invoke(null);
return c;
} catch (Throwable t) { }
}
return null;
}
public static Class get(String className) throws Exception {
className = FrameRecorder.class.getPackage().getName() + "." + className;
try {
return Class.forName(className).asSubclass(FrameRecorder.class);
} catch (ClassNotFoundException e) {
String className2 = className + "FrameRecorder";
try {
return Class.forName(className2).asSubclass(FrameRecorder.class);
} catch (ClassNotFoundException ex) {
throw new Exception("Could not get FrameRecorder class for " + className + " or " + className2, e);
}
}
}
public static FrameRecorder create(Class c, Class p, Object o, int w, int h) throws Exception {
Throwable cause = null;
try {
return (FrameRecorder)c.getConstructor(p, int.class, int.class).newInstance(o, w, h);
} catch (InstantiationException ex) {
cause = ex;
} catch (IllegalAccessException ex) {
cause = ex;
} catch (IllegalArgumentException ex) {
cause = ex;
} catch (NoSuchMethodException ex) {
cause = ex;
} catch (InvocationTargetException ex) {
cause = ex.getCause();
}
throw new Exception("Could not create new " + c.getSimpleName() + "(" + o + ", " + w + ", " + h + ")", cause);
}
public static FrameRecorder createDefault(File file, int width, int height) throws Exception {
return create(getDefault(), File.class, file, width, height);
}
public static FrameRecorder createDefault(String filename, int width, int height) throws Exception {
return create(getDefault(), String.class, filename, width, height);
}
public static FrameRecorder create(String className, File file, int width, int height) throws Exception {
return create(get(className), File.class, file, width, height);
}
public static FrameRecorder create(String className, String filename, int width, int height) throws Exception {
return create(get(className), String.class, filename, width, height);
}
protected String format, videoCodecName, audioCodecName;
protected int imageWidth, imageHeight, audioChannels;
protected int pixelFormat, videoCodec, videoBitrate, imageScalingFlags, gopSize = -1, videoProfile = -1;
protected double aspectRatio, frameRate, videoQuality = -1;
protected int sampleFormat, audioCodec, audioBitrate, sampleRate;
protected double audioQuality = -1;
protected boolean interleaved;
protected Charset charset = Charset.defaultCharset();
protected Map options = new HashMap();
protected Map videoOptions = new HashMap();
protected Map audioOptions = new HashMap();
protected Map metadata = new HashMap();
protected Map videoMetadata = new HashMap();
protected Map audioMetadata = new HashMap();
protected Map videoSideData = new HashMap();
protected Map audioSideData = new HashMap();
protected int frameNumber = 0;
protected long timestamp = 0;
protected int maxBFrames = -1;
protected int trellis = -1;
protected int maxDelay = -1;
public String getFormat() {
return format;
}
public void setFormat(String format) {
this.format = format;
}
public String getVideoCodecName() {
return videoCodecName;
}
public void setVideoCodecName(String videoCodecName) {
this.videoCodecName = videoCodecName;
}
public String getAudioCodecName() {
return audioCodecName;
}
public void setAudioCodecName(String audioCodecName) {
this.audioCodecName = audioCodecName;
}
public int getImageWidth() {
return imageWidth;
}
public void setImageWidth(int imageWidth) {
this.imageWidth = imageWidth;
}
public int getImageHeight() {
return imageHeight;
}
public void setImageHeight(int imageHeight) {
this.imageHeight = imageHeight;
}
public int getAudioChannels() {
return audioChannels;
}
public void setAudioChannels(int audioChannels) {
this.audioChannels = audioChannels;
}
public int getPixelFormat() {
return pixelFormat;
}
public void setPixelFormat(int pixelFormat) {
this.pixelFormat = pixelFormat;
}
public int getVideoCodec() {
return videoCodec;
}
public void setVideoCodec(int videoCodec) {
this.videoCodec = videoCodec;
}
public int getVideoBitrate() {
return videoBitrate;
}
public void setVideoBitrate(int videoBitrate) {
this.videoBitrate = videoBitrate;
}
public int getImageScalingFlags() {
return imageScalingFlags;
}
public void setImageScalingFlags(int imageScalingFlags) {
this.imageScalingFlags = imageScalingFlags;
}
public int getGopSize() {
return gopSize;
}
public void setGopSize(int gopSize) {
this.gopSize = gopSize;
}
public int getVideoProfile() {
return videoProfile;
}
public void setVideoProfile(int videoProfile) {
this.videoProfile = videoProfile;
}
public double getAspectRatio() {
return aspectRatio;
}
public void setAspectRatio(double aspectRatio) {
this.aspectRatio = aspectRatio;
}
public double getFrameRate() {
return frameRate;
}
public void setFrameRate(double frameRate) {
this.frameRate = frameRate;
}
public double getVideoQuality() {
return videoQuality;
}
public void setVideoQuality(double videoQuality) {
this.videoQuality = videoQuality;
}
public int getSampleFormat() {
return sampleFormat;
}
public void setSampleFormat(int sampleFormat) {
this.sampleFormat = sampleFormat;
}
public int getAudioCodec() {
return audioCodec;
}
public void setAudioCodec(int audioCodec) {
this.audioCodec = audioCodec;
}
public int getAudioBitrate() {
return audioBitrate;
}
public void setAudioBitrate(int audioBitrate) {
this.audioBitrate = audioBitrate;
}
public int getSampleRate() {
return sampleRate;
}
public void setSampleRate(int sampleRate) {
this.sampleRate = sampleRate;
}
public double getAudioQuality() {
return audioQuality;
}
public void setAudioQuality(double audioQuality) {
this.audioQuality = audioQuality;
}
public boolean isInterleaved() {
return interleaved;
}
public void setInterleaved(boolean interleaved) {
this.interleaved = interleaved;
}
public Charset getCharset() {
return charset;
}
public void setCharset(Charset charset) {
this.charset = charset;
}
public Map getOptions() {
return options;
}
public void setOptions(Map options) {
this.options = options;
}
public Map getVideoOptions() {
return videoOptions;
}
public void setVideoOptions(Map options) {
this.videoOptions = options;
}
public Map getAudioOptions() {
return audioOptions;
}
public void setAudioOptions(Map options) {
this.audioOptions = options;
}
public Map getMetadata() {
return metadata;
}
public void setMetadata(Map metadata) {
this.metadata = metadata;
}
public Map getVideoMetadata() {
return videoMetadata;
}
public void setVideoMetadata(Map metadata) {
this.videoMetadata = metadata;
}
public Map getAudioMetadata() {
return audioMetadata;
}
public void setAudioMetadata(Map metadata) {
this.audioMetadata = metadata;
}
public String getOption(String key) {
return options.get(key);
}
public void setOption(String key, String value) {
options.put(key, value);
}
public String getVideoOption(String key) {
return videoOptions.get(key);
}
public void setVideoOption(String key, String value) {
videoOptions.put(key, value);
}
public String getAudioOption(String key) {
return audioOptions.get(key);
}
public void setAudioOption(String key, String value) {
audioOptions.put(key, value);
}
public String getMetadata(String key) {
return metadata.get(key);
}
public void setMetadata(String key, String value) {
metadata.put(key, value);
}
public String getVideoMetadata(String key) {
return videoMetadata.get(key);
}
public void setVideoMetadata(String key, String value) {
videoMetadata.put(key, value);
}
public String getAudioMetadata(String key) {
return audioMetadata.get(key);
}
public void setAudioMetadata(String key, String value) {
audioMetadata.put(key, value);
}
public Map getVideoSideData() {
return videoSideData;
}
public void setVideoSideData(Map videoSideData) {
this.videoSideData = videoSideData;
}
public Buffer getVideoSideData(String key) {
return videoSideData.get(key);
}
public void setVideoSideData(String key, Buffer value) {
videoSideData.put(key, value);
}
public Map getAudioSideData() {
return audioSideData;
}
public void setAudioSideData(Map audioSideData) {
this.audioSideData = audioSideData;
}
public Buffer getAudioSideData(String key) {
return audioSideData.get(key);
}
public void setAudioSideData(String key, Buffer value) {
audioSideData.put(key, value);
}
public int getFrameNumber() {
return frameNumber;
}
public void setFrameNumber(int frameNumber) {
this.frameNumber = frameNumber;
}
public long getTimestamp() {
return timestamp;
}
public void setTimestamp(long timestamp) {
this.timestamp = timestamp;
}
public int getMaxBFrames() {
return maxBFrames;
}
public void setMaxBFrames(int maxBFrames) {
this.maxBFrames = maxBFrames;
}
public int getTrellis() {
return trellis;
}
public void setTrellis(int trellis) {
this.trellis = trellis;
}
public int getMaxDelay() {
return maxDelay;
}
public void setMaxDelay(int maxDelay) {
this.maxDelay = maxDelay;
}
public static class Exception extends IOException {
public Exception(String message) { super(message); }
public Exception(String message, Throwable cause) { super(message, cause); }
}
public abstract void start() throws Exception;
public abstract void flush() throws Exception;
public abstract void stop() throws Exception;
public abstract void record(Frame frame) throws Exception;
public abstract void release() throws Exception;
@Override public void close() throws Exception {
stop();
release();
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/GLCanvasFrame.java
================================================
/*
* Copyright (C) 2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import com.jogamp.opencl.CLImage2d;
import com.jogamp.opencl.gl.CLGLImage2d;
import com.jogamp.opengl.GL2;
import com.jogamp.opengl.GLAutoDrawable;
import com.jogamp.opengl.GLCapabilitiesImmutable;
import com.jogamp.opengl.GLContext;
import com.jogamp.opengl.GLEventListener;
import com.jogamp.opengl.awt.GLCanvas;
import com.jogamp.opengl.util.Gamma;
import java.awt.Color;
import java.awt.DisplayMode;
import java.awt.EventQueue;
import java.awt.GraphicsConfiguration;
import java.awt.Image;
import java.awt.image.BufferedImage;
import java.awt.image.DataBuffer;
import java.awt.image.DataBufferByte;
import java.awt.image.DataBufferDouble;
import java.awt.image.DataBufferFloat;
import java.awt.image.DataBufferInt;
import java.awt.image.DataBufferShort;
import java.awt.image.DataBufferUShort;
import java.nio.Buffer;
import java.nio.ByteBuffer;
import java.nio.DoubleBuffer;
import java.nio.FloatBuffer;
import java.nio.IntBuffer;
import java.nio.ShortBuffer;
import javax.swing.JFrame;
import org.bytedeco.opencv.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgcodecs.*;
/**
*
* @author Samuel Audet
*/
public class GLCanvasFrame extends CanvasFrame {
public GLCanvasFrame(String title) {
this(title, 0.0);
}
public GLCanvasFrame(String title, double gamma) {
super(title, gamma);
init(false, null, null);
}
public GLCanvasFrame(String title, GraphicsConfiguration gc,
GLCapabilitiesImmutable caps, GLContext shareWith) {
this(title, gc, caps, shareWith, 0.0);
}
public GLCanvasFrame(String title, GraphicsConfiguration gc,
GLCapabilitiesImmutable caps, GLContext shareWith, double gamma) {
super(title, gc, gamma);
init(false, caps, shareWith);
}
public GLCanvasFrame(String title, int screenNumber, DisplayMode displayMode) throws Exception {
this(title, screenNumber, displayMode, 0.0);
}
public GLCanvasFrame(String title, int screenNumber, DisplayMode displayMode, double gamma) throws Exception {
super(title, screenNumber, displayMode, gamma);
init(true, null, null);
}
public GLCanvasFrame(String title, int screenNumber, DisplayMode displayMode,
GLCapabilitiesImmutable caps, GLContext shareWith) throws Exception {
this(title, screenNumber, displayMode, caps, shareWith, 0.0);
}
public GLCanvasFrame(String title, int screenNumber, DisplayMode displayMode,
GLCapabilitiesImmutable caps, GLContext shareWith, double gamma) throws Exception {
super(title, screenNumber, displayMode, gamma);
init(true, caps, shareWith);
}
private void init(final boolean fullScreen,
final GLCapabilitiesImmutable caps, final GLContext shareWith) {
Runnable r = new Runnable() { public void run() {
String wasErase = System.setProperty("sun.awt.noerasebackground", "true");
canvas = new GLCanvas(caps);
if (shareWith != null) {
((GLCanvas)canvas).setSharedContext(shareWith);
}
((GLCanvas)canvas).addGLEventListener(eventListener);
if (fullScreen) {
canvas.setSize(getSize());
needInitialResize = false;
} else {
canvas.setSize(1, 1); // or we do not get a GLContext
needInitialResize = true;
}
getContentPane().add(canvas);
canvas.setVisible(true);
if (wasErase != null) {
System.setProperty("sun.awt.noerasebackground", wasErase);
} else {
System.clearProperty("sun.awt.noerasebackground");
}
}};
if (EventQueue.isDispatchThread()) {
r.run();
} else {
try {
EventQueue.invokeAndWait(r);
} catch (java.lang.Exception ex) { }
}
}
@Override protected void initCanvas(boolean fullScreen, DisplayMode displayMode, double gamma) { }
private int[] params = new int[2];
private Color color = null;
private int width, height, format, type;
private Buffer buffer = null;
private int frameBuffer = 0, renderBuffer = 0;
private GLEventListener eventListener = new GLEventListener() {
public void init(GLAutoDrawable drawable) {
GL2 gl = drawable.getGL().getGL2();
gl.setSwapInterval(1); // Sync to VBlank
if (inverseGamma != 1.0) {
// Yeah baby, gamma correction in hardware!
Gamma.setDisplayGamma(drawable, (float)inverseGamma, 0, 1);
}
gl.glGenFramebuffers(1, params, 0);
frameBuffer = params[0];
}
public void dispose(GLAutoDrawable drawable) {
GL2 gl = drawable.getGL().getGL2();
params[0] = frameBuffer;
gl.glDeleteFramebuffers(1, params, 0);
if (inverseGamma != 1.0) {
Gamma.resetDisplayGamma(drawable);
}
}
public void display(GLAutoDrawable drawable) {
GL2 gl = drawable.getGL().getGL2();
if (color != null) {
gl.glClearColor(color.getRed()/255f, color.getGreen()/255f, color.getBlue()/255f, 1f);
gl.glClear(GL2.GL_COLOR_BUFFER_BIT);
} else if (buffer != null) {
if (isResizable() && needInitialResize) {
int w = (int)Math.round(width *initialScale);
int h = (int)Math.round(height*initialScale);
setCanvasSize(w, h);
}
gl.glWindowPos2i(0, canvas.getHeight());
gl.glPixelZoom((float)canvas.getWidth()/width, -(float)canvas.getHeight()/height);
// XXX: Tell OpenGL about the alignment of buffer via glPixelStore(), somehow...
gl.glDrawPixels(width, height, format, type, buffer);
} else if (renderBuffer > 0) {
gl.glBindRenderbuffer(GL2.GL_RENDERBUFFER, renderBuffer);
gl.glGetRenderbufferParameteriv(GL2.GL_RENDERBUFFER,
GL2.GL_RENDERBUFFER_WIDTH, params, 0);
gl.glGetRenderbufferParameteriv(GL2.GL_RENDERBUFFER,
GL2.GL_RENDERBUFFER_HEIGHT, params, 1);
if (isResizable() && needInitialResize) {
int w = (int)Math.round(params[0]*initialScale);
int h = (int)Math.round(params[1]*initialScale);
setCanvasSize(w, h);
}
gl.glBindFramebuffer(GL2.GL_READ_FRAMEBUFFER, frameBuffer);
gl.glFramebufferRenderbuffer(GL2.GL_READ_FRAMEBUFFER,
GL2.GL_COLOR_ATTACHMENT0, GL2.GL_RENDERBUFFER, renderBuffer);
// Often GL_RENDERBUFFER_WIDTH == 0 and GL_RENDERBUFFER_HEIGHT == 1,
// while glCheckFramebufferStatus() returns
// GL_FRAMEBUFFER_INCOMPLETE_MISSING_ATTACHMENT here,
// but, for a given object, it either never happens or
// always happens ... NVIDIA driver bug?
//System.out.println(params[0] + " " + params[1] +
// " glCheckFramebufferStatus = " + gl.glCheckFramebufferStatus(GL2.GL_READ_FRAMEBUFFER));
assert gl.glCheckFramebufferStatus(GL2.GL_READ_FRAMEBUFFER) == GL2.GL_FRAMEBUFFER_COMPLETE;
gl.glBlitFramebuffer(0, 0, params[0], params[1],
0, canvas.getHeight(), canvas.getWidth(), 0,
GL2.GL_COLOR_BUFFER_BIT, GL2.GL_LINEAR);
}
}
public void reshape(GLAutoDrawable drawable, int x, int y, int width, int height) { }
};
public GLCanvas getGLCanvas() {
return (GLCanvas)canvas;
}
@Override public void showColor(Color color) {
this.color = color;
this.buffer = null;
getGLCanvas().display();
}
@Override public void showImage(Frame frame) {
showImage(frame, false);
}
@Override public void showImage(Frame frame, boolean flipChannels) {
if (flipChannels) {
throw new RuntimeException("GLCanvasFrame does not support channel flipping.");
}
if (frame == null) {
return;
}
this.color = null;
this.width = frame.imageWidth;
this.height = frame.imageHeight;
this.buffer = frame.image[0];
switch (frame.imageDepth) {
case Frame.DEPTH_BYTE: this.type = GL2.GL_BYTE; break;
case Frame.DEPTH_UBYTE: this.type = GL2.GL_UNSIGNED_BYTE; break;
case Frame.DEPTH_SHORT: this.type = GL2.GL_SHORT; break;
case Frame.DEPTH_USHORT: this.type = GL2.GL_UNSIGNED_SHORT; break;
case Frame.DEPTH_INT: this.type = GL2.GL_INT; break;
case Frame.DEPTH_FLOAT: this.type = GL2.GL_FLOAT; break;
case Frame.DEPTH_DOUBLE: this.type = GL2.GL_DOUBLE; break;
default: assert false;
}
switch (frame.imageChannels) {
case 1: this.format = GL2.GL_LUMINANCE; break;
case 2: this.format = GL2.GL_RG; break;
case 3: this.format = GL2.GL_RGB; break;
case 4: this.format = GL2.GL_RGBA; break;
default: assert false;
}
getGLCanvas().display();
}
@Override public void showImage(Image image) {
if (!(image instanceof BufferedImage)) {
throw new RuntimeException("GLCanvasFrame does not support " + image + ", BufferedImage required.");
}
showImage((BufferedImage)image);
}
public void showImage(BufferedImage image) {
if (image == null) {
return;
}
this.color = null;
this.width = image.getWidth();
this.height = image.getHeight();
DataBuffer buffer = image.getRaster().getDataBuffer();
if (buffer instanceof DataBufferByte) {
this.buffer = ByteBuffer.wrap(((DataBufferByte)buffer).getData());
this.type = GL2.GL_UNSIGNED_BYTE;
} else if (buffer instanceof DataBufferDouble) {
this.buffer = DoubleBuffer.wrap(((DataBufferDouble)buffer).getData());
this.type = GL2.GL_DOUBLE;
} else if (buffer instanceof DataBufferFloat) {
this.buffer = FloatBuffer.wrap(((DataBufferFloat)buffer).getData());
this.type = GL2.GL_FLOAT;
} else if (buffer instanceof DataBufferInt) {
this.buffer = IntBuffer.wrap(((DataBufferInt)buffer).getData());
this.type = GL2.GL_INT;
} else if (buffer instanceof DataBufferShort) {
this.buffer = ShortBuffer.wrap(((DataBufferShort)buffer).getData());
this.type = GL2.GL_SHORT;
} else if (buffer instanceof DataBufferUShort) {
this.buffer = ShortBuffer.wrap(((DataBufferUShort)buffer).getData());
this.type = GL2.GL_UNSIGNED_SHORT;
} else {
assert false;
}
switch (image.getSampleModel().getNumBands()) {
case 1: this.format = GL2.GL_LUMINANCE; break;
case 2: this.format = GL2.GL_RG; break;
case 3: this.format = GL2.GL_RGB; break;
case 4: this.format = GL2.GL_RGBA; break;
default: assert false;
}
getGLCanvas().display();
}
public void showImage(int renderBuffer) {
if (renderBuffer <= 0) {
return;
}
this.color = null;
this.buffer = null;
this.renderBuffer = renderBuffer;
getGLCanvas().display();
}
private static GLCanvasFrame canvasFrame;
public static void main(String[] args) throws java.lang.Exception {
EventQueue.invokeAndWait(new Runnable() {
public void run() {
try {
canvasFrame = new GLCanvasFrame("Some Title"/*, context.getGLContext()*/);
canvasFrame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
//canvasFrame.setCanvasSize(640, 480);
canvasFrame.showColor(Color.BLUE);
} catch (java.lang.Exception ex) {
ex.printStackTrace();
}
}
});
final JavaCVCL context = new JavaCVCL(canvasFrame.getGLCanvas().getContext());
final IplImage image = cvLoadImageBGRA("/usr/share/opencv/samples/c/lena.jpg"/*args[0]*/);
//final IplImage image = cvLoadImage("/usr/share/opencv/samples/c/lena.jpg"/*args[0]*/, 0);
//final IplImage image = IplImage.create(640, 480, IPL_DEPTH_32F, 4);
final CLGLImage2d imageCLGL = context.createCLGLImageFrom(image);
//final CLImage2d imageCL = context.createCLImageFrom(image);
context.acquireGLObject(imageCLGL);
context.writeImage(imageCLGL, image, true);
context.releaseGLObject(imageCLGL);
//System.out.println(imageCLGL.getFormat());
//System.exit(0);
canvasFrame.setCanvasScale(0.5);
for (int i = 0; i < 1000; i++) {
canvasFrame.showImage(imageCLGL.getGLObjectID());
Thread.sleep(10);
canvasFrame.showColor(Color.RED);
Thread.sleep(10);
}
canvasFrame.waitKey();
context.release();
System.exit(0);
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/GNImageAligner.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.util.Arrays;
import org.bytedeco.javacv.ImageTransformer.Data;
import org.bytedeco.javacv.ImageTransformer.Parameters;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
*
* @author Samuel Audet
*/
public class GNImageAligner implements ImageAligner {
public GNImageAligner(ImageTransformer transformer, Parameters initialParameters,
IplImage template0, double[] roiPts, IplImage target0) {
this(transformer, initialParameters, template0, roiPts, target0, new Settings());
}
public GNImageAligner(ImageTransformer transformer, Parameters initialParameters,
IplImage template0, double[] roiPts, IplImage target0, Settings settings) {
this(transformer, initialParameters);
setSettings(settings);
final int minLevel = settings.pyramidLevelMin;
final int maxLevel = settings.pyramidLevelMax;
this.template = new IplImage[maxLevel+1];
this.target = new IplImage[maxLevel+1];
this.transformed = new IplImage[maxLevel+1];
this.residual = new IplImage[maxLevel+1];
this.mask = new IplImage[maxLevel+1];
int w = template0 != null ? template0.width() : target0.width();
int h = template0 != null ? template0.height() : target0.height();
int c = template0 != null ? template0.nChannels() : target0.nChannels();
int o = template0 != null ? template0.origin() : target0.origin();
for (int i = minLevel; i <= maxLevel; i++) {
if (i == minLevel && (template0 != null && template0.depth() == IPL_DEPTH_32F)) {
template[i] = template0;
} else {
template[i] = IplImage.create(w, h, IPL_DEPTH_32F, c, o);
}
if (i == minLevel && (target0 != null && target0.depth() == IPL_DEPTH_32F)) {
target[i] = target0;
} else {
target[i] = IplImage.create(w, h, IPL_DEPTH_32F, c, o);
}
transformed[i] = IplImage.create(w, h, IPL_DEPTH_32F, c, o);
residual [i] = IplImage.create(w, h, IPL_DEPTH_32F, c, o);
mask [i] = IplImage.create(w, h, IPL_DEPTH_8U, 1, o);
w /= 2;
h /= 2;
}
this.hessianGradientTransformerData = new Data[n];
for (int i = 0; i < n; i++) {
hessianGradientTransformerData[i] = new Data(template[pyramidLevel],
transformed[pyramidLevel], residual[pyramidLevel], mask[pyramidLevel],
0, 0, pyramidLevel, null, null, n);
}
this.residualTransformerData = new Data[] { new Data(template[pyramidLevel],
target[pyramidLevel], null, mask[pyramidLevel],
0, 0, pyramidLevel, transformed[pyramidLevel], residual[pyramidLevel], 1) };
setConstrained(settings.constrained);
setTemplateImage(template0, roiPts);
setTargetImage(target0);
}
protected GNImageAligner(ImageTransformer transformer, Parameters initialParameters) {
this.n = initialParameters.size();
this.srcRoiPts = CvMat.create(4, 1, CV_64F, 2);
this.dstRoiPts = CvMat.create(4, 1, CV_64F, 2);
this.dstRoiPtsArray = new CvPoint(4);
this.roi = new CvRect();
this.temproi = new CvRect();
this.transformer = transformer;
this.parameters = initialParameters.clone();
this.parametersArray = new Parameters[] { parameters };
this.tempParameters = new Parameters[n];
for (int i = 0; i < tempParameters.length; i++) {
this.tempParameters[i] = initialParameters.clone();
}
subspaceParameters = parameters.getSubspace();
if (subspaceParameters != null) {
tempSubspaceParameters = new double[Parallel.getNumThreads()][];
for (int i = 0; i < tempSubspaceParameters.length; i++) {
tempSubspaceParameters[i] = subspaceParameters.clone();
}
// for (double d : subspaceParameters) {
// System.out.print(d + " ");
// }
// System.out.println();
}
}
public static class Settings extends ImageAligner.Settings implements Cloneable {
public Settings() { }
public Settings(Settings s) {
super(s);
stepSize = s.stepSize;
lineSearch = s.lineSearch;
deltaMin = s.deltaMin;
deltaMax = s.deltaMax;
displacementMax = s.displacementMax;
alphaSubspace = s.alphaSubspace;
alphaTikhonov = s.alphaTikhonov;
gammaTgamma = s.gammaTgamma;
constrained = s.constrained;
}
double stepSize = 0.1;
double[] lineSearch = {1.0, 0.25};
double deltaMin = 10;
double deltaMax = 300;
double displacementMax = 0.2;
double alphaSubspace = 0.1;
double alphaTikhonov = 0;
CvMat gammaTgamma = null;
boolean constrained = false;
public double getStepSize() {
return stepSize;
}
public void setStepSize(double stepSize) {
this.stepSize = stepSize;
}
public double[] getLineSearch() {
return lineSearch;
}
public void setLineSearch(double[] lineSearch) {
this.lineSearch = lineSearch;
}
public double getDeltaMin() {
return deltaMin;
}
public void setDeltaMin(double deltaMin) {
this.deltaMin = deltaMin;
}
public double getDeltaMax() {
return deltaMax;
}
public void setDeltaMax(double deltaMax) {
this.deltaMax = deltaMax;
}
public double getDisplacementMax() {
return displacementMax;
}
public void setDisplacementMax(double displacementMax) {
this.displacementMax = displacementMax;
}
public double getAlphaSubspace() {
return alphaSubspace;
}
public void setAlphaSubspace(double alphaSubspace) {
this.alphaSubspace = alphaSubspace;
}
public double getAlphaTikhonov() {
return alphaTikhonov;
}
public void setAlphaTikhonov(double alphaTikhonov) {
this.alphaTikhonov = alphaTikhonov;
}
public CvMat getGammaTgamma() {
return gammaTgamma;
}
public void setGammaTgamma(CvMat gammaTgamma) {
this.gammaTgamma = gammaTgamma;
}
// public boolean isConstrained() {
// return constrained;
// }
// public void setConstrained(boolean constrained) {
// this.constrained = constrained;
// }
@Override public Settings clone() {
return new Settings(this);
}
}
protected Settings settings;
public Settings getSettings() {
return settings;
}
public void setSettings(ImageAligner.Settings settings) {
this.settings = (Settings)settings;
}
protected final int n;
protected IplImage[] template, target, transformed, residual, mask;
protected IplImage[] images = new IplImage[5];
protected CvMat srcRoiPts, dstRoiPts;
protected CvPoint dstRoiPtsArray;
protected CvRect roi, temproi;
protected ImageTransformer transformer;
protected Data[] hessianGradientTransformerData, residualTransformerData;
protected Parameters parameters, parametersArray[], tempParameters[], priorParameters;
protected CvMat hessian, gradient, update, prior;
protected double[] constraintGrad, subspaceResidual, subspaceJacobian[], updateScale;
protected boolean[] subspaceCorrelated;
protected int pyramidLevel;
protected double RMSE;
protected boolean residualUpdateNeeded = true;
protected int lastLinePosition = 0;
protected int trials = 0;
// protected double prevOutlierRatio = 0;
protected double[] subspaceParameters, tempSubspaceParameters[];
public IplImage getTemplateImage() {
return template[pyramidLevel];
}
public void setTemplateImage(IplImage template0, double[] roiPts) {
final int minLevel = settings.pyramidLevelMin;
final int maxLevel = settings.pyramidLevelMax;
if (roiPts == null && template0 != null) {
int w = template0.width() << minLevel;
int h = template0.height() << minLevel;
this.srcRoiPts.put(0.0, 0.0, w, 0.0, w, h, 0, h);
} else if (roiPts != null) {
this.srcRoiPts.put(roiPts);
}
if (template0 == null) {
return;
}
if (template0.depth() == IPL_DEPTH_32F) {
template[minLevel] = template0;
} else {
cvConvertScale(template0, template[minLevel], 1.0/template0.highValue(), 0);
}
for (int i = minLevel+1; i <= maxLevel; i++) {
cvPyrDown(template[i-1], template[i], CV_GAUSSIAN_5x5);
}
setPyramidLevel(maxLevel);
}
public IplImage getTargetImage() {
return target[pyramidLevel];
}
public void setTargetImage(IplImage target0) {
final int minLevel = settings.pyramidLevelMin;
final int maxLevel = settings.pyramidLevelMax;
if (target0 == null) {
return;
}
if (target0.depth() == IPL_DEPTH_32F) {
target[minLevel] = target0;
}
if (settings.displacementMax > 0) {
transformer.transform(srcRoiPts, dstRoiPts, parameters, false);
double[] pts = dstRoiPts.get();
for (int i = 0; i < pts.length; i++) {
pts[i] /= (1< prevRMSE; j++) {
RMSE = prevRMSE;
parameters.set(resetParameters);
if (subspaceParameters != null) {
System.arraycopy(resetSubspaceParameters, 0, subspaceParameters, 0, subspaceParameters.length);
}
lastLinePosition = j;
for (int i = 0; i < n; i++) {
parameters.set(i, parameters.get(i) + settings.lineSearch[j]*update.get(i)*updateScale[i]);
}
for (int i = n; i < update.length(); i++) {
subspaceParameters[i-n] += settings.lineSearch[j]*update.get(i)*updateScale[i];
}
residualUpdateNeeded = true;
}
double deltaNorm = 0;
if (delta != null) {
for (int i = 0; i < delta.length && i < updateScale.length; i++) {
delta[i] = settings.lineSearch[lastLinePosition]*update.get(i)*updateScale[i];
}
deltaNorm = JavaCV.norm(Arrays.copyOf(delta, n));
}
boolean invalid = getRMSE() > prevRMSE || deltaNorm > settings.deltaMax ||
Double.isNaN(RMSE) || Double.isInfinite(RMSE);
if (invalid) {
RMSE = prevRMSE;
parameters.set(prevParameters);
if (subspaceParameters != null) {
System.arraycopy(prevSubspaceParameters, 0, subspaceParameters, 0, subspaceParameters.length);
}
residualUpdateNeeded = true;
}
if (invalid && deltaNorm > settings.deltaMin && ++trials < 2) {
return false;
} else if (invalid || deltaNorm < settings.deltaMin) {
trials = 0;
if (pyramidLevel > settings.pyramidLevelMin) {
setPyramidLevel(pyramidLevel-1);
} else {
converged = true;
}
} else {
trials = 0;
}
return converged;
}
protected void doHessianGradient(final double[] scale) {
final double constraintError = parameters.getConstraintError();
final double stepSize = settings.stepSize;
cvSetZero(gradient);
cvSetZero(hessian);
Parallel.loop(0, n, new Parallel.Looper() {
public void loop(int from, int to, int looperID) {
// for (int i = 0; i < n; i++) {
for (int i = from; i < to; i++) {
tempParameters[i].set(parameters);
tempParameters[i].set(i, tempParameters[i].get(i) + /*(1< 0 &&
settings.alphaSubspace != 0.0) {
final int m = subspaceParameters.length;
// double[][] subspaceHessian = new double[n+m][n+m];
// double[] subspaceGradient = new double[n+m];
Arrays.fill(subspaceCorrelated, false);
tempParameters[0].set(parameters);
tempParameters[0].setSubspace(subspaceParameters);
Parallel.loop(0, n+m, tempSubspaceParameters.length, new Parallel.Looper() {
public void loop(int from, int to, int looperID) {
// int looperID = 0;
// for (int i = 0; i < n+m; i++) {
for (int i = from; i < to; i++) {
if (i < n) {
Arrays.fill(subspaceJacobian[i], 0);
subspaceJacobian[i][i] = scale[i];
} else {
System.arraycopy(subspaceParameters, 0, tempSubspaceParameters[looperID], 0, m);
tempSubspaceParameters[looperID][i-n] += stepSize;
tempParameters[i-n+1].set(parameters);
tempParameters[i-n+1].setSubspace(tempSubspaceParameters[looperID]);
scale[i] = tempSubspaceParameters[looperID][i-n] - subspaceParameters[i-n];
for (int j = 0; j < n; j++) {
subspaceJacobian[i][j] = tempParameters[0].get(j) - tempParameters[i-n+1].get(j);
subspaceCorrelated[j] |= subspaceJacobian[i][j] != 0; // this may not work in parallel...
}
}
}}});
int subspaceCorrelatedCount = 0;
// double subspaceRMSE = 0;
for (int i = 0; i < n; i++) {
subspaceResidual[i] = parameters.get(i) - tempParameters[0].get(i);
// subspaceRMSE += subspaceResidual[i]*subspaceResidual[i];
if (subspaceCorrelated[i]) {
subspaceCorrelatedCount++;
}
}
// subspaceRMSE = Math.sqrt(subspaceRMSE/n);
//System.out.println((float)RMSE + " " + (float)subspaceRMSE);
final double K = settings.alphaSubspace*settings.alphaSubspace * RMSE*RMSE/
subspaceCorrelatedCount;//(subspaceRMSE*subspaceRMSE);
Parallel.loop(0, n+m, new Parallel.Looper() {
public void loop(int from, int to, int looperID) {
// int looperID = 0;
// for (int i = 0; i < n+m; i++) {
for (int i = from; i < to; i++) {
if (i < n && !subspaceCorrelated[i]) {
continue;
}
for (int j = i; j < n+m; j++) {
if (j < n && !subspaceCorrelated[j]) {
continue;
}
double h = 0;
for (int k = 0; k < n; k++) {
h += subspaceJacobian[i][k]*subspaceJacobian[j][k];
}
// subspaceHessian[i][j] = h;
h = hessian.get(i, j) + K*h;
hessian.put(i, j, h);
hessian.put(j, i, h);
}
double g = 0;
for (int k = 0; k < n; k++) {
g -= subspaceJacobian[i][k]*subspaceResidual[k];
}
// subspaceGradient[i] = g;
g = gradient.get(i) + K*g;
gradient.put(i, g);
}}});
}
// add Tikhonov regularization
int rows = hessian.rows(), cols = hessian.cols();
for (int i = 0; i < rows; i++) {
for (int j = 0; j < cols; j++) {
double h = hessian.get(i, j);
double g = 0;
if (settings.gammaTgamma != null && i < settings.gammaTgamma.rows() && j < settings.gammaTgamma.cols()) {
g = settings.gammaTgamma.get(i, j);
}
double a = 0;
if (i == j && i < n) {
a = settings.alphaTikhonov * settings.alphaTikhonov;
}
hessian.put(i, j, h + g + a);
}
}
}
protected void doRoi() {
transformer.transform(srcRoiPts, dstRoiPts, parameters, false);
double[] pts = dstRoiPts.get();
for (int i = 0; i < pts.length; i++) {
pts[i] /= (1< minLevel) templateCL[i].release();
if (i > minLevel) targetCL [i].release();
transformedCL[i].release();
residualCL [i].release();
maskCL [i].release();
}
templateCL = targetCL = transformedCL = residualCL = maskCL = null;
}
// NVIDIA drivers crash if we don't delete those before terminating
context.getGLContext().makeCurrent();
GL2 gl = context.getGL2();
if (maskfb != null) {
gl.glDeleteFramebuffers(maxLevel+1, maskfb, 0);
maskfb = null;
}
if (maskrb != null) {
gl.glDeleteRenderbuffers(maxLevel+1, maskrb, 0);
maskrb = null;
}
}
@Override protected void finalize() throws Throwable {
super.finalize();
release();
}
private final JavaCVCL context;
private CLImage2d[] templateCL, targetCL, transformedCL, residualCL;
private CLGLImage2d[] maskCL;
private int[] maskrb, maskfb;
private CLImage2d[] imagesCL = new CLImage2d[5];
private InputData inputData;
private OutputData outputData;
private boolean[] templateChanged;
@Override public IplImage getTemplateImage() {
return getTemplateImage(true);
}
public IplImage getTemplateImage(boolean blocking) {
if (templateChanged[pyramidLevel]) {
templateChanged[pyramidLevel] = false;
return template[pyramidLevel] = context.readImage(getTemplateImageCL(), template[pyramidLevel], blocking);
} else {
return template[pyramidLevel];
}
}
@Override public void setTemplateImage(IplImage template0, double[] roiPts) {
context.writeImage(templateCL[settings.pyramidLevelMin], template0, false);
setTemplateImageCL(templateCL[settings.pyramidLevelMin], roiPts);
}
@Override public IplImage getTargetImage() {
return getTargetImage(true);
}
public IplImage getTargetImage(boolean blocking) {
return target[pyramidLevel] = context.readImage(getTargetImageCL(), target[pyramidLevel], blocking);
}
@Override public void setTargetImage(IplImage target0) {
context.writeImage(targetCL[settings.pyramidLevelMin], target0, false);
setTargetImageCL(targetCL[settings.pyramidLevelMin]);
}
@Override public IplImage getTransformedImage() {
return getTransformedImage(true);
}
public IplImage getTransformedImage(boolean blocking) {
return transformed[pyramidLevel] = context.readImage(getTransformedImageCL(), transformed[pyramidLevel], blocking);
}
@Override public IplImage getResidualImage() {
return getResidualImage(true);
}
public IplImage getResidualImage(boolean blocking) {
return residual[pyramidLevel] = context.readImage(getResidualImageCL(), residual[pyramidLevel], blocking);
}
@Override public IplImage getMaskImage() {
return getMaskImage(true);
}
public IplImage getMaskImage(boolean blocking) {
context.acquireGLObject(maskCL[pyramidLevel]);
mask[pyramidLevel] = context.readImage(getMaskImageCL(), mask[pyramidLevel], blocking);
context.releaseGLObject(maskCL[pyramidLevel]);
return mask[pyramidLevel];
}
@Override public double getRMSE() {
if (residualUpdateNeeded) {
doRoi();
doResidual();
}
return RMSE;
}
@Override public int getPixelCount() {
if (residualUpdateNeeded) {
doRoi();
doResidual();
}
return outputData.dstCount;
}
@Override public int getOutlierCount() {
return outputData.dstCountOutlier;
}
@Override public CvRect getRoi() {
if (residualUpdateNeeded) {
doRoi();
}
return roi.x(inputData.roiX).y(inputData.roiY).
width(inputData.roiWidth).height(inputData.roiHeight);
}
@Override public IplImage[] getImages() {
return getImages(true);
}
public IplImage[] getImages(boolean blocking) {
images[0] = getTemplateImage(false);
images[1] = getTargetImage(false);
images[2] = getTransformedImage(false);
images[3] = getResidualImage(false);
images[4] = getMaskImage(blocking);
return images;
}
public CLImage2d getTemplateImageCL() {
return templateCL[pyramidLevel];
}
public void setTemplateImageCL(CLImage2d template0, double[] roiPts) {
final int minLevel = settings.pyramidLevelMin;
final int maxLevel = settings.pyramidLevelMax;
if (roiPts == null && template0 != null) {
int w = template0.width << minLevel;
int h = template0.height << minLevel;
this.srcRoiPts.put(0.0, 0.0, w, 0.0, w, h, 0, h);
} else {
this.srcRoiPts.put(roiPts);
}
if (template0 == null) {
return;
}
// if (templateCL == null || templateCL.length != settings.pyramidLevels) {
// templateCL = new CLImage2d[settings.pyramidLevels];
// }
templateCL[minLevel] = template0;
for (int i = minLevel+1; i <= maxLevel; i++) {
// if (templateCL[i] == null) {
// int w = templateCL[i-1].width/2;
// int h = templateCL[i-1].height/2;
// CLImageFormat format = new CLImageFormat(CLImageFormat.ChannelOrder.RGBA, CLImageFormat.ChannelType.FLOAT);
// templateCL[i] = JavaCVCL.getCLContext().createImage2d(w, h, format);
// }
context.pyrDown(templateCL[i-1], templateCL[i]);
}
setPyramidLevel(maxLevel);
Arrays.fill(templateChanged, true);
}
public CLImage2d getTargetImageCL() {
return targetCL[pyramidLevel];
}
public void setTargetImageCL(CLImage2d target0) {
final int minLevel = settings.pyramidLevelMin;
final int maxLevel = settings.pyramidLevelMax;
// if (targetCL == null || targetCL.length != settings.pyramidLevels) {
// targetCL = new CLImage2d[settings.pyramidLevels];
// }
targetCL[minLevel] = target0;
for (int i = minLevel+1; i <= maxLevel; i++) {
// if (targetCL[i] == null) {
// int w = targetCL[i-1].width/2;
// int h = targetCL[i-1].height/2;
// CLImageFormat format = new CLImageFormat(CLImageFormat.ChannelOrder.RGBA, CLImageFormat.ChannelType.FLOAT);
// targetCL[i] = JavaCVCL.getCLContext().createImage2d(w, h, format);
// }
context.pyrDown(targetCL[i-1], targetCL[i]);
}
setPyramidLevel(maxLevel);
}
public CLImage2d getTransformedImageCL() {
if (residualUpdateNeeded) {
doRoi();
doResidual();
}
return transformedCL[pyramidLevel];
}
public CLImage2d getResidualImageCL() {
if (residualUpdateNeeded) {
doRoi();
doResidual();
}
return residualCL[pyramidLevel];
}
public CLImage2d getMaskImageCL() {
return maskCL[pyramidLevel];
}
public CLImage2d[] getImagesCL() {
imagesCL[0] = templateCL [pyramidLevel];
imagesCL[1] = targetCL [pyramidLevel];
imagesCL[2] = transformedCL[pyramidLevel];
imagesCL[3] = residualCL [pyramidLevel];
imagesCL[4] = maskCL [pyramidLevel];
return imagesCL;
}
@Override protected void doHessianGradient(final double[] scale) {
final double constraintError = parameters.getConstraintError();
final double stepSize = settings.stepSize;
cvSetZero(gradient);
cvSetZero(hessian);
Parallel.loop(0, n, new Parallel.Looper() {
public void loop(int from, int to, int looperID) {
// for (int i = 0; i < n; i++) {
for (int i = from; i < to; i++) {
tempParameters[i].set(parameters);
tempParameters[i].set(i, tempParameters[i].get(i) + /*(1< allObjectMarkers = new LinkedList();
private LinkedList allImageMarkers = new LinkedList();
private IplImage tempImage = null;
private Marker[] lastDetectedMarkers = null;
private CvMat warp = CvMat.create(3, 3);
private CvMat prevWarp = CvMat.create(3, 3);
private CvMat lastWarp = CvMat.create(3, 3);
private CvMat warpSrcPts = CvMat.create(1, 4, CV_64F, 2);
private CvMat warpDstPts = CvMat.create(1, 4, CV_64F, 2);
private CvMat tempPts = CvMat.create(1, 4, CV_64F, 2);
public MarkerDetector getMarkerDetector() {
return markerDetector;
}
public MarkedPlane getMarkedPlane() {
return markedPlane;
}
public ProjectiveDevice getProjectiveDevice() {
return projectiveDevice;
}
public LinkedList getAllObjectMarkers() {
return allObjectMarkers;
}
public void setAllObjectMarkers(LinkedList allObjectMarkers) {
this.allObjectMarkers = allObjectMarkers;
}
public LinkedList getAllImageMarkers() {
return allImageMarkers;
}
public void setAllImageMarkers(LinkedList allImageMarkers) {
this.allImageMarkers = allImageMarkers;
}
public Marker[] processImage(IplImage image) {
projectiveDevice.imageWidth = image.width();
projectiveDevice.imageHeight = image.height();
final boolean whiteMarkers = markedPlane.getForegroundColor().magnitude() >
markedPlane.getBackgroundColor().magnitude();
if (image.depth() > 8) {
if (tempImage == null ||
tempImage.width() != image.width() ||
tempImage.height() != image.height()) {
tempImage = (IplImage)IplImage.create(image.width(), image.height(),
IPL_DEPTH_8U, 1, image.origin());
}
cvConvertScale(image, tempImage, 1.0/(1<<8), 0);
lastDetectedMarkers = markerDetector.detect(tempImage, whiteMarkers);
} else {
lastDetectedMarkers = markerDetector.detect(image, whiteMarkers);
}
// First, check if we detected enough markers
if (lastDetectedMarkers.length < markedPlane.getMarkers().length*settings.detectedBoardMin) {
return null;
}
// then check by how much the corners of the calibration board moved
markedPlane.getTotalWarp(lastDetectedMarkers, warp);
cvPerspectiveTransform (warpSrcPts, warpDstPts, warp);
cvPerspectiveTransform (warpSrcPts, tempPts, prevWarp);
double rmsePrev = cvNorm(warpDstPts, tempPts);
cvPerspectiveTransform (warpSrcPts, tempPts, lastWarp);
double rmseLast = cvNorm(warpDstPts, tempPts);
//System.out.println("rmsePrev = " + rmsePrev + " rmseLast = " + rmseLast);
// save warp for next iteration...
cvCopy(warp, prevWarp);
// send upstream our recommendation for addition or not of these markers...
int imageSize = (image.width()+image.height())/2;
if (rmsePrev < settings.patternSteadySize*imageSize &&
rmseLast > settings.patternMovedSize*imageSize) {
return lastDetectedMarkers;
} else {
return null;
}
}
public void drawMarkers(IplImage image) {
markerDetector.draw(image, lastDetectedMarkers);
}
public void addMarkers() {
addMarkers(markedPlane.getMarkers(), lastDetectedMarkers);
}
public void addMarkers(Marker[] imageMarkers) {
addMarkers(markedPlane.getMarkers(), imageMarkers);
}
public void addMarkers(Marker[] objectMarkers, Marker[] imageMarkers) {
// add only matching markers...
int maxLength = Math.min(objectMarkers.length, imageMarkers.length);
Marker[] om = new Marker[maxLength];
Marker[] im = new Marker[maxLength];
int i = 0;
for (Marker m1 : objectMarkers) {
for (Marker m2 : imageMarkers) {
if (m1.id == m2.id) {
om[i] = m1;
im[i] = m2;
i++;
break;
}
}
}
if (i < maxLength) {
om = Arrays.copyOf(om, i);
im = Arrays.copyOf(im, i);
}
allObjectMarkers.add(om);
allImageMarkers.add(im);
// we added the detected markers, so save last computed warp too...
cvCopy(prevWarp, lastWarp);
}
public int getImageCount() {
assert(allObjectMarkers.size() == allImageMarkers.size());
return allObjectMarkers.size();
}
private Point3fVectorVector getObjectPoints(CvMat points, CvMat counts) {
FloatBuffer pointsBuf = points.getFloatBuffer();
IntBuffer countsBuf = counts.getIntBuffer();
int n = counts.length();
Point3fVectorVector vectors = new Point3fVectorVector(n);
for (int i = 0; i < n; i++) {
int m = countsBuf.get();
Point3fVector vector = new Point3fVector(m);
for (int j = 0; j < m; j++) {
vector.put(j, new Point3f(pointsBuf.get(), pointsBuf.get(), pointsBuf.get()));
}
vectors.put(i, vector);
}
return vectors;
}
private Point2fVectorVector getImagePoints(CvMat points, CvMat counts) {
FloatBuffer pointsBuf = points.getFloatBuffer();
IntBuffer countsBuf = counts.getIntBuffer();
int n = counts.length();
Point2fVectorVector vectors = new Point2fVectorVector(n);
for (int i = 0; i < n; i++) {
int m = countsBuf.get();
Point2fVector vector = new Point2fVector(m);
for (int j = 0; j < m; j++) {
vector.put(j, new Point2f(pointsBuf.get(), pointsBuf.get()));
}
vectors.put(i, vector);
}
return vectors;
}
private CvMat[] getPoints(boolean useCenters) {
// fill up pointCounts, objectPoints and imagePoints, with data from
// srcMarkers and dstMarkers
assert(allObjectMarkers.size() == allImageMarkers.size());
Iterator i1 = allObjectMarkers.iterator(),
i2 = allImageMarkers.iterator();
CvMat pointCounts = CvMat.create(1, allImageMarkers.size(), CV_32S, 1);
IntBuffer pointCountsBuf = pointCounts.getIntBuffer();
int totalPointCount = 0;
while (i1.hasNext() && i2.hasNext()) {
Marker[] m1 = i1.next(),
m2 = i2.next();
assert(m1.length == m2.length);
int n = m1.length*(useCenters ? 1 : 4);
pointCountsBuf.put(n);
totalPointCount += n;
}
i1 = allObjectMarkers.iterator();
i2 = allImageMarkers.iterator();
CvMat objectPoints = CvMat.create(1, totalPointCount, CV_32F, 3);
CvMat imagePoints = CvMat.create(1, totalPointCount, CV_32F, 2);
FloatBuffer objectPointsBuf = objectPoints.getFloatBuffer();
FloatBuffer imagePointsBuf = imagePoints.getFloatBuffer();
while (i1.hasNext() && i2.hasNext()) {
Marker[] m1 = i1.next(),
m2 = i2.next();
for (int j = 0; j < m1.length; j++) {
if (useCenters) {
double[] c1 = m1[j].getCenter();
objectPointsBuf.put((float)c1[0]);
objectPointsBuf.put((float)c1[1]);
objectPointsBuf.put(0);
double[] c2 = m2[j].getCenter();
imagePointsBuf.put((float)c2[0]);
imagePointsBuf.put((float)c2[1]);
} else { // use corners...
for (int k = 0; k < 4; k++) {
objectPointsBuf.put((float)m1[j].corners[2*k ]);
objectPointsBuf.put((float)m1[j].corners[2*k + 1]);
objectPointsBuf.put(0);
imagePointsBuf.put((float)m2[j].corners[2*k ]);
imagePointsBuf.put((float)m2[j].corners[2*k + 1]);
}
}
}
}
return new CvMat[] { objectPoints, imagePoints, pointCounts };
}
public static double[] computeReprojectionError(CvMat object_points,
CvMat image_points, CvMat point_counts, CvMat camera_matrix,
CvMat dist_coeffs, CvMat rot_vects, CvMat trans_vects,
CvMat per_view_errors ) {
CvMat image_points2 = CvMat.create(image_points.rows(),
image_points.cols(), image_points.type());
int i, image_count = rot_vects.rows(), points_so_far = 0;
double total_err = 0, max_err = 0, err;
CvMat object_points_i = new CvMat(),
image_points_i = new CvMat(),
image_points2_i = new CvMat();
IntBuffer point_counts_buf = point_counts.getIntBuffer();
CvMat rot_vect = new CvMat(), trans_vect = new CvMat();
for (i = 0; i < image_count; i++) {
object_points_i.reset();
image_points_i .reset();
image_points2_i.reset();
int point_count = point_counts_buf.get(i);
cvGetCols(object_points, object_points_i,
points_so_far, points_so_far + point_count);
cvGetCols(image_points, image_points_i,
points_so_far, points_so_far + point_count);
cvGetCols(image_points2, image_points2_i,
points_so_far, points_so_far + point_count);
points_so_far += point_count;
cvGetRows(rot_vects, rot_vect, i, i+1, 1);
cvGetRows(trans_vects, trans_vect, i, i+1, 1);
projectPoints(cvarrToMat(object_points_i), cvarrToMat(rot_vect), cvarrToMat(trans_vect),
cvarrToMat(camera_matrix), cvarrToMat(dist_coeffs), cvarrToMat(image_points2_i));
err = cvNorm(image_points_i, image_points2_i);
err *= err;
if (per_view_errors != null)
per_view_errors.put(i, Math.sqrt(err/point_count));
total_err += err;
for (int j = 0; j < point_count; j++) {
double x1 = image_points_i .get(0, j, 0);
double y1 = image_points_i .get(0, j, 1);
double x2 = image_points2_i.get(0, j, 0);
double y2 = image_points2_i.get(0, j, 1);
double dx = x1-x2;
double dy = y1-y2;
err = Math.sqrt(dx*dx + dy*dy);
if (err > max_err) {
max_err = err;
}
}
}
return new double[] { Math.sqrt(total_err/points_so_far), max_err };
}
public double[] calibrate(boolean useCenters) {
ProjectiveDevice d = projectiveDevice;
CalibrationSettings dsettings = (CalibrationSettings)d.getSettings();
if (d.cameraMatrix == null) {
d.cameraMatrix = CvMat.create(3, 3);
cvSetZero(d.cameraMatrix);
if ((dsettings.flags & CV_CALIB_FIX_ASPECT_RATIO) != 0) {
d.cameraMatrix.put(0, dsettings.initAspectRatio);
d.cameraMatrix.put(4, 1.);
}
}
int kn = dsettings.isFixK3() ? 4 : 5;
if (dsettings.isRationalModel() && !dsettings.isFixK4() &&
!dsettings.isFixK4() && !dsettings.isFixK5()) {
kn = 8;
}
if (d.distortionCoeffs == null || d.distortionCoeffs.cols() != kn) {
d.distortionCoeffs = CvMat.create(1, kn);
cvSetZero(d.distortionCoeffs);
}
CvMat rotVects = new CvMat(), transVects = new CvMat();
d.extrParams = CvMat.create(allImageMarkers.size(), 6);
cvGetCols(d.extrParams, rotVects, 0, 3);
cvGetCols(d.extrParams, transVects, 3, 6);
CvMat[] points = getPoints(useCenters);
MatVector rvecs = new MatVector();
MatVector tvecs = new MatVector();
Mat distortionCoeffs = new Mat();
calibrateCamera(getObjectPoints(points[0], points[2]), getImagePoints(points[1], points[2]),
new Size(d.imageWidth, d.imageHeight),
cvarrToMat(d.cameraMatrix), distortionCoeffs,
rvecs, tvecs, dsettings.flags,
new TermCriteria(CV_TERMCRIT_ITER | CV_TERMCRIT_EPS, 30, JavaCV.DBL_EPSILON));
int n = (int)rvecs.size();
CvMat row = new CvMat();
for (int i = 0; i < n; i++) {
cvTranspose(cvMat(rvecs.get(i)), cvGetRow(rotVects, row, i));
cvTranspose(cvMat(tvecs.get(i)), cvGetRow(transVects, row, i));
}
d.distortionCoeffs = cvMat(distortionCoeffs).clone();
if (cvCheckArr(d.cameraMatrix, CV_CHECK_QUIET, 0, 0) != 0 &&
cvCheckArr(d.distortionCoeffs, CV_CHECK_QUIET, 0, 0) != 0 &&
cvCheckArr(d.extrParams, CV_CHECK_QUIET, 0, 0) != 0) {
d.reprojErrs = CvMat.create(1, allImageMarkers.size());
double[] err = computeReprojectionError(points[0], points[1], points[2],
d.cameraMatrix, d.distortionCoeffs, rotVects, transVects, d.reprojErrs);
d.avgReprojErr = err[0];
d.maxReprojErr = err[1];
// d.nominalDistance = d.getNominalDistance(markedPlane);
return err;
} else {
d.cameraMatrix = null;
d.avgReprojErr = -1;
d.maxReprojErr = -1;
return null;
}
}
public static double[] computeStereoError(CvMat imagePoints1, CvMat imagePoints2,
CvMat M1, CvMat D1, CvMat M2, CvMat D2, CvMat F) {
// CALIBRATION QUALITY CHECK
// because the output fundamental matrix implicitly
// includes all the output information,
// we can check the quality of calibration using the
// epipolar geometry constraint: m2^t*F*m1=0
int N = imagePoints1.cols();
CvMat L1 = CvMat.create(1, N, CV_32F, 3);
CvMat L2 = CvMat.create(1, N, CV_32F, 3);
//Always work in undistorted space
undistortPoints(cvarrToMat(imagePoints1), cvarrToMat(imagePoints1), cvarrToMat(M1), cvarrToMat(D1), null, cvarrToMat(M1));
undistortPoints(cvarrToMat(imagePoints2), cvarrToMat(imagePoints2), cvarrToMat(M2), cvarrToMat(D2), null, cvarrToMat(M2));
//imagePoints1.put(d1.undistort(imagePoints1.get()));
//imagePoints2.put(d2.undistort(imagePoints2.get()));
computeCorrespondEpilines(cvarrToMat(imagePoints1), 1, cvarrToMat(F), cvarrToMat(L1));
computeCorrespondEpilines(cvarrToMat(imagePoints2), 2, cvarrToMat(F), cvarrToMat(L2));
double avgErr = 0, maxErr = 0;
CvMat p1 = imagePoints1, p2 = imagePoints2;
for(int i = 0; i < N; i++ ) {
double e1 = p1.get(0, i, 0)*L2.get(0, i, 0) +
p1.get(0, i, 1)*L2.get(0, i, 1) + L2.get(0, i, 2);
double e2 = p2.get(0, i, 0)*L1.get(0, i, 0) +
p2.get(0, i, 1)*L1.get(0, i, 1) + L1.get(0, i, 2);
double err = e1*e1 + e2*e2;
avgErr += err;
err = Math.sqrt(err);
if (err > maxErr) {
maxErr = err;
}
}
return new double[] { Math.sqrt(avgErr/N), maxErr };
}
public double[] calibrateStereo(boolean useCenters, GeometricCalibrator peer) {
ProjectiveDevice d = projectiveDevice;
ProjectiveDevice dp = peer.projectiveDevice;
CalibrationSettings dsettings = (CalibrationSettings)d.getSettings();
CalibrationSettings dpsettings = (CalibrationSettings)dp.getSettings();
CvMat[] points1 = getPoints(useCenters);
CvMat[] points2 = peer.getPoints(useCenters);
// find points in common from points1 and points2
// since points1[0] and points2[0] might not be equal...
FloatBuffer objPts1 = points1[0].getFloatBuffer();
FloatBuffer imgPts1 = points1[1].getFloatBuffer();
IntBuffer imgCount1 = points1[2].getIntBuffer();
FloatBuffer objPts2 = points2[0].getFloatBuffer();
FloatBuffer imgPts2 = points2[1].getFloatBuffer();
IntBuffer imgCount2 = points2[2].getIntBuffer();
assert(imgCount1.capacity() == imgCount2.capacity());
CvMat objectPointsMat = CvMat.create(1, Math.min(objPts1.capacity(), objPts2.capacity()), CV_32F, 3);
CvMat imagePoints1Mat = CvMat.create(1, Math.min(imgPts1.capacity(), imgPts2.capacity()), CV_32F, 2);
CvMat imagePoints2Mat = CvMat.create(1, Math.min(imgPts1.capacity(), imgPts2.capacity()), CV_32F, 2);
CvMat pointCountsMat = CvMat.create(1, imgCount1.capacity(), CV_32S, 1);
FloatBuffer objectPoints = objectPointsMat.getFloatBuffer();
FloatBuffer imagePoints1 = imagePoints1Mat.getFloatBuffer();
FloatBuffer imagePoints2 = imagePoints2Mat.getFloatBuffer();
IntBuffer pointCounts = pointCountsMat .getIntBuffer();
int end1 = 0, end2 = 0;
for (int i = 0; i < imgCount1.capacity(); i++) {
int start1 = end1;
int start2 = end2;
end1 = start1 + imgCount1.get(i);
end2 = start2 + imgCount2.get(i);
int count = 0;
for (int j = start1; j < end1; j++) {
float x1 = objPts1.get(j*3 );
float y1 = objPts1.get(j*3+1);
float z1 = objPts1.get(j*3+2);
for (int k = start2; k < end2; k++) {
float x2 = objPts2.get(k*3 );
float y2 = objPts2.get(k*3+1);
float z2 = objPts2.get(k*3+2);
if (x1 == x2 && y1 == y2 && z1 == z2) {
objectPoints.put(x1);
objectPoints.put(y1);
objectPoints.put(z1);
imagePoints1.put(imgPts1.get(j*2));
imagePoints1.put(imgPts1.get(j*2+1));
imagePoints2.put(imgPts2.get(k*2));
imagePoints2.put(imgPts2.get(k*2+1));
count++;
break;
}
}
}
if (count > 0) {
pointCounts.put(count);
}
}
objectPointsMat.cols(objectPoints.position()/3);
imagePoints1Mat.cols(imagePoints1.position()/2);
imagePoints2Mat.cols(imagePoints2.position()/2);
pointCountsMat .cols(pointCounts .position());
// place our ProjectiveDevice at the origin...
d.R = CvMat.create(3, 3); d.R.put(1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0);
d.T = CvMat.create(3, 1); d.T.put(0.0, 0.0, 0.0);
d.E = CvMat.create(3, 3); cvSetZero(d.E);
d.F = CvMat.create(3, 3); cvSetZero(d.F);
dp.R = CvMat.create(3, 3);
dp.T = CvMat.create(3, 1);
dp.E = CvMat.create(3, 3);
dp.F = CvMat.create(3, 3);
stereoCalibrate(getObjectPoints(objectPointsMat, pointCountsMat), getImagePoints(imagePoints1Mat, pointCountsMat), getImagePoints(imagePoints2Mat, pointCountsMat),
cvarrToMat(d.cameraMatrix), cvarrToMat(d.distortionCoeffs), cvarrToMat(dp.cameraMatrix), cvarrToMat(dp.distortionCoeffs),
new Size(d.imageWidth, d.imageHeight), cvarrToMat(dp.R), cvarrToMat(dp.T), cvarrToMat(dp.E), cvarrToMat(dp.F), dpsettings.flags,
new TermCriteria(CV_TERMCRIT_ITER+CV_TERMCRIT_EPS,100,1e-6));
// compute and return epipolar error...
d.avgEpipolarErr = 0.0;
d.maxEpipolarErr = 0.0;
double err[] = computeStereoError(imagePoints1Mat, imagePoints2Mat,
d.cameraMatrix, d.distortionCoeffs,
dp.cameraMatrix, dp.distortionCoeffs, dp.F);
dp.avgEpipolarErr = err[0];
dp.maxEpipolarErr = err[1];
return err;
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/HandMouse.java
================================================
/*
* Copyright (C) 2011-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.nio.ByteBuffer;
import java.nio.FloatBuffer;
import java.nio.IntBuffer;
import org.bytedeco.javacpp.IntPointer;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
*
* @author Samuel Audet
*/
public class HandMouse {
public HandMouse() {
this(new Settings());
}
public HandMouse(Settings settings) {
setSettings(settings);
}
public static class Settings extends BaseChildSettings {
public Settings() { }
public Settings(Settings s) {
s.mopIterations = mopIterations;
s.clickSteadySize = clickSteadySize;
s.clickSteadyTime = clickSteadyTime;
s.edgeAreaMin = edgeAreaMin;
s.edgeAreaMax = edgeAreaMax;
s.thresholdHigh = thresholdHigh;
s.thresholdLow = thresholdLow;
s.brightnessMin = brightnessMin;
s.updateAlpha = updateAlpha;
}
int mopIterations = 1;
double clickSteadySize = 0.05;
long clickSteadyTime = 250;
double edgeAreaMin = 0.001;
double edgeAreaMax = 0.1;
double thresholdHigh = 0.5;
double thresholdLow = 0.25;
double brightnessMin = 0.1;
double updateAlpha = 0.5;
public int getMopIterations() {
return mopIterations;
}
public void setMopIterations(int mopIterations) {
this.mopIterations = mopIterations;
}
public double getClickSteadySize() {
return clickSteadySize;
}
public void setClickSteadySize(double clickSteadySize) {
this.clickSteadySize = clickSteadySize;
}
public long getClickSteadyTime() {
return clickSteadyTime;
}
public void setClickSteadyTime(long clickSteadyTime) {
this.clickSteadyTime = clickSteadyTime;
}
public double getEdgeAreaMin() {
return edgeAreaMin;
}
public void setEdgeAreaMin(double edgeAreaMin) {
this.edgeAreaMin = edgeAreaMin;
}
public double getEdgeAreaMax() {
return edgeAreaMax;
}
public void setEdgeAreaMax(double edgeAreaMax) {
this.edgeAreaMax = edgeAreaMax;
}
public double getThresholdHigh() {
return thresholdHigh;
}
public void setThresholdHigh(double thresholdHigh) {
this.thresholdHigh = thresholdHigh;
}
public double getThresholdLow() {
return thresholdLow;
}
public void setThresholdLow(double thresholdLow) {
this.thresholdLow = thresholdLow;
}
public double getBrightnessMin() {
return brightnessMin;
}
public void setBrightnessMin(double brightnessMin) {
this.brightnessMin = brightnessMin;
}
public double getUpdateAlpha() {
return updateAlpha;
}
public void setUpdateAlpha(double updateAlpha) {
this.updateAlpha = updateAlpha;
}
}
private Settings settings;
public Settings getSettings() {
return settings;
}
public void setSettings(Settings settings) {
this.settings = settings;
}
private IplImage relativeResidual = null, binaryImage = null;
private CvRect roi = null;
private CvMemStorage storage = CvMemStorage.create();
private int contourPointsSize = 0;
private IntPointer intPointer = new IntPointer(1);
private CvPoint contourPoints = null;
private IntBuffer contourPointsBuffer = null;
private CvMoments moments = new CvMoments();
private double edgeX = 0, edgeY = 0, centerX = 0, centerY = 0;
private double imageTipX = -1, tipX = -1, prevTipX = -1;
private double imageTipY = -1, tipY = -1, prevTipY = -1;
private long tipTime = 0, prevTipTime = 0;
private CvPoint pt1 = new CvPoint(), pt2 = new CvPoint();
private boolean imageUpdateNeeded = false;
public void reset() {
tipX = tipY = prevTipX = prevTipY = -1;
}
public void update(IplImage[] images, int pyramidLevel, CvRect roi, double[] roiPts) {
this.roi = roi;
// double RMSE = aligner.getRMSE()*((GNImageAligner)aligner).prevOutlierRatio;
// double threshold = RMSE*settings.threshold;
// double threshold2 = RMSE*settings.threshold2;//threshold*threshold;
IplImage target = images[1];
IplImage transformed = images[2];
IplImage residual = images[3];
IplImage mask = images[4];
int width = roi.width();
int height = roi.height();
int channels = residual.nChannels();
relativeResidual = IplImage.createIfNotCompatible(relativeResidual, mask);
binaryImage = IplImage.createIfNotCompatible(binaryImage, mask);
cvResetImageROI(relativeResidual);
cvResetImageROI(binaryImage);
double brightnessMin = (channels > 3 ? 3 : channels)*settings.brightnessMin;
double contourEdgeAreaMax = (width+height)/2*width*height*settings.edgeAreaMax;
double contourEdgeAreaMin = (width+height)/2*width*height*settings.edgeAreaMin;
ByteBuffer maskBuf = mask.getByteBuffer();
FloatBuffer residualBuf = residual.getFloatBuffer();
FloatBuffer targetBuf = target.getFloatBuffer();
FloatBuffer transformedBuf = transformed.getFloatBuffer();
ByteBuffer relResBuf = relativeResidual.getByteBuffer();
while (maskBuf.hasRemaining() && residualBuf.hasRemaining() &&
targetBuf.hasRemaining() && transformedBuf.hasRemaining() &&
relResBuf.hasRemaining()) {
byte m = maskBuf.get();
if (m == 0) {
residualBuf.position(residualBuf.position() + channels);
targetBuf.position(targetBuf.position() + channels);
transformedBuf.position(transformedBuf.position() + channels);
relResBuf.put((byte)0);
} else {
double relativeNorm = 0;
double brightness = 0;
for (int z = 0; z < channels; z++) {
float r = Math.abs(residualBuf.get());
float c = targetBuf.get();
float t = transformedBuf.get();
if (z < 3) {
float maxct = Math.max(c,t);
brightness += maxct;
relativeNorm = Math.max(r/maxct, relativeNorm);
} // ignore alpha channel
}
if (brightness < brightnessMin) {
relResBuf.put((byte)0);
} else {
relResBuf.put((byte)Math.round(255 / settings.thresholdHigh *
Math.min(relativeNorm, settings.thresholdHigh)));
}
}
}
JavaCV.hysteresisThreshold(relativeResidual, binaryImage,
255, 255*settings.thresholdLow/settings.thresholdHigh, 255);
int roiX = roi.x(), roiY = roi.y();
cvSetImageROI(binaryImage, roi);
if (settings.mopIterations > 0) {
cvMorphologyEx(binaryImage, binaryImage, null, null, CV_MOP_OPEN, settings.mopIterations);
cvMorphologyEx(binaryImage, binaryImage, null, null, CV_MOP_CLOSE, settings.mopIterations);
}
CvSeq contour = new CvContour(null);
cvFindContours(binaryImage, storage, contour, Loader.sizeof(CvContour.class),
CV_RETR_EXTERNAL, CV_CHAIN_APPROX_NONE);
double largestContourEdgeArea = 0;
CvSeq largestContour = null;
while (contour != null && !contour.isNull()) {
contourPointsSize = contour.total();
if (contourPoints == null || contourPoints.capacity() < contourPointsSize) {
contourPoints = new CvPoint(contourPointsSize);
contourPointsBuffer = contourPoints.asByteBuffer().asIntBuffer();
}
cvCvtSeqToArray(contour, contourPoints.position(0));
double[] edgePts = new double[roiPts.length];
for (int i = 0; i < roiPts.length/2; i++) {
edgePts[2*i ] = roiPts[2*i ]/(1< contourEdgeAreaMin && contourEdgeArea < contourEdgeAreaMax &&
contourEdgeArea > largestContourEdgeArea) {
largestContourEdgeArea = contourEdgeArea;
largestContour = contour;
double inv_m00 = 1 / m00;
edgeX = m10 * inv_m00;
edgeY = m01 * inv_m00;
}
contour = contour.h_next();
}
if (isClick()) {
prevTipX = -1;
prevTipY = -1;
prevTipTime = 0;
} else if (!isSteady()) {
prevTipX = tipX;
prevTipY = tipY;
prevTipTime = System.currentTimeMillis();
}
if (largestContour == null) {
tipX = -1;
tipY = -1;
tipTime = 0;
imageUpdateNeeded = false;
} else {
cvMoments(largestContour, moments, 0);
double inv_m00 = 1 / moments.m00();
centerX = moments.m10() * inv_m00;
centerY = moments.m01() * inv_m00;
contourPointsSize = largestContour.total();
cvCvtSeqToArray(largestContour, contourPoints.position(0));
double tipDist2 = 0;
int tipIndex = 0;
for (int i = 0; i < contourPointsSize; i++) {
int x = contourPointsBuffer.get(2*i ),
y = contourPointsBuffer.get(2*i + 1);
double dx = centerX - edgeX;
double dy = centerY - edgeY;
double d2 = dx*dx + dy*dy;
double u = ((x - edgeX)*dx + (y - edgeY)*dy) / d2;
double px = edgeX + u*dx;
double py = edgeY + u*dy;
dx = px - edgeX;
dy = py - edgeY;
d2 = dx*dx + dy*dy;
if (d2 > tipDist2) {
tipIndex = i;
tipDist2 = d2;
}
}
double a = imageTipX < 0 || imageTipY < 0 ? 1.0 : settings.updateAlpha;
imageTipX = a*contourPointsBuffer.get(2*tipIndex ) + (1-a)*imageTipX;
imageTipY = a*contourPointsBuffer.get(2*tipIndex + 1) + (1-a)*imageTipY;
tipX = (imageTipX+roiX)*(1<= 0 && tipY >= 0 && prevTipX >= 0 && prevTipY >= 0) {
double dx = tipX - prevTipX;
double dy = tipY - prevTipY;
int imageSize = (roi.width() + roi.height())/2;
double steadySize = settings.clickSteadySize*imageSize;
return dx*dx + dy*dy < steadySize*steadySize;
}
return false;
}
public boolean isClick() {
return isSteady() && tipTime - prevTipTime > settings.clickSteadyTime;
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/IPCameraFrameGrabber.java
================================================
/*
* Copyright (C) 2013 Greg Perry
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.Loader;
import javax.imageio.ImageIO;
import java.awt.image.BufferedImage;
import java.io.ByteArrayInputStream;
import java.io.DataInputStream;
import java.io.EOFException;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
import java.util.concurrent.TimeUnit;
import org.bytedeco.opencv.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgcodecs.*;
public class IPCameraFrameGrabber extends FrameGrabber {
/*
* excellent reference - http://www.jpegcameras.com/ foscam url
* http://host/videostream.cgi?user=username&pwd=password
* http://192.168.0.59:60/videostream.cgi?user=admin&pwd=password android ip
* cam http://192.168.0.57:8080/videofeed
*/
private static Exception loadingException = null;
public static void tryLoad() throws Exception {
if (loadingException != null) {
throw loadingException;
} else {
try {
Loader.load(org.bytedeco.opencv.global.opencv_highgui.class);
} catch (Throwable t) {
throw loadingException = new Exception("Failed to load " + IPCameraFrameGrabber.class, t);
}
}
}
private final FrameConverter converter = new OpenCVFrameConverter.ToMat();
private final URL url;
private final int connectionTimeout;
private final int readTimeout;
private DataInputStream input;
private byte[] pixelBuffer = new byte[1024];
private Mat decoded = null;
/**
* @param url The URL to create the camera connection with.
* @param startTimeout How long should this wait on the connection while trying to {@link #start()} before
* timing out.
* If this value is less than zero it will be ignored.
* {@link URLConnection#setConnectTimeout(int)}
* @param grabTimeout How long should grab wait while reading the connection before timing out.
* If this value is less than zero it will be ignored.
* {@link URLConnection#setReadTimeout(int)}
* @param timeUnit The time unit to use for the connection and read timeout.
* If this value is null then the start timeout and grab timeout will be ignored.
*/
public IPCameraFrameGrabber(URL url, int startTimeout, int grabTimeout, TimeUnit timeUnit) {
super(); // Always good practice to do this
if (url == null) {
throw new IllegalArgumentException("URL can not be null");
}
this.url = url;
if (timeUnit != null) {
this.connectionTimeout = toIntExact(TimeUnit.MILLISECONDS.convert(startTimeout, timeUnit));
this.readTimeout = toIntExact(TimeUnit.MILLISECONDS.convert(grabTimeout, timeUnit));
} else {
this.connectionTimeout = -1;
this.readTimeout = -1;
}
}
public IPCameraFrameGrabber(String urlstr, int connectionTimeout, int readTimeout, TimeUnit timeUnit) throws MalformedURLException {
this(new URL(urlstr), connectionTimeout, readTimeout, timeUnit);
}
/**
* @param urlstr A string to be used to create the URL.
* @throws MalformedURLException if the urlstr is a malformed URL
* @deprecated By not setting the connection timeout and the read timeout if your network ever crashes
* then {@link #start()} or {@link #grab()} can hang for upwards of 45 to 60 seconds before failing.
* You should always explicitly set the connectionTimeout and readTimeout so that your application can
* respond appropriately to a loss or failure to connect.
*/
@Deprecated
public IPCameraFrameGrabber(String urlstr) throws MalformedURLException {
this(new URL(urlstr), -1, -1, null);
}
@Override
public void start() throws Exception {
try {
/*
* We don't need to keep a reference to the connection
* after it is opened in the parent class.
* It never uses it outside of start.
*/
final URLConnection connection = url.openConnection();
// If the class was initialized with timeout values then configure those
if (connectionTimeout >= 0) {
connection.setConnectTimeout(connectionTimeout);
}
if (readTimeout >= 0) {
connection.setReadTimeout(readTimeout);
}
input = new DataInputStream(connection.getInputStream());
} catch (IOException e) {
throw new Exception(e.getMessage(), e);
}
}
@Override
public void stop() throws Exception {
if (input != null) {
try {
input.close();
} catch (IOException e) {
throw new Exception(e.getMessage(), e);
} finally {
// Close may have failed but there's really nothing we can do about it at this point
input = null;
// Don't set the url to null, it may be needed to restart this object
releaseDecoded();
}
}
}
@Override
public void trigger() throws Exception {
}
@Override
public Frame grab() throws Exception {
try {
final byte[] b = readImage();
final Mat mat = new Mat(1, b.length, CV_8UC1, new BytePointer(b));
releaseDecoded();
return converter.convert(decoded = imdecode(mat, IMREAD_COLOR));
} catch (IOException e) {
throw new Exception(e.getMessage(), e);
}
}
public BufferedImage grabBufferedImage() throws IOException {
BufferedImage bi = ImageIO.read(new ByteArrayInputStream(readImage()));
return bi;
}
/**
* Ensures that if the decoded image is not null that it gets released and set to null.
* If the image was not set to null then trying to release a null pointer will cause a
* segfault.
*/
private void releaseDecoded() {
if (decoded != null) {
decoded.release();
decoded = null;
}
}
private byte[] readImage() throws IOException {
final StringBuffer sb = new StringBuffer();
int c;
// read http subheader
while ((c = input.read()) != -1) {
if (c > 0) {
sb.append((char) c);
if (c == 13) {
sb.append((char) input.read());// '10'+
c = input.read();
sb.append((char) c);
if (c == 13) {
sb.append((char) input.read());// '10'
break; // done with subheader
}
}
}
}
// find embedded jpeg in stream
/*
* Some cameras return headers 'content-length' using different casing
* Eg. Axis cameras return 'Content-Length:' while TrendNet cameras return 'content-length:'
*/
final String subheader = sb.toString().toLowerCase();
//log.debug(subheader);
// Yay! - server was nice and sent content length
int c0 = subheader.indexOf("content-length: ");
final int c1 = subheader.indexOf('\r', c0);
if (c0 < 0) {
//log.info("no content length returning null");
throw new EOFException("The camera stream ended unexpectedly");
}
c0 += 16;
final int contentLength = Integer.parseInt(subheader.substring(c0, c1).trim());
//log.debug("Content-Length: " + contentLength);
// adaptive size - careful - don't want a 2G jpeg
ensureBufferCapacity(contentLength);
input.readFully(pixelBuffer, 0, contentLength);
input.read();// \r
input.read();// \n
input.read();// \r
input.read();// \n
return pixelBuffer;
}
@Override
public void release() throws Exception {
}
/**
* Grow the pixel buffer if necessary. Using this method instead of allocating a new buffer every time a frame
* is grabbed improves performance by reducing the frequency of garbage collections. In a simple test, the
* original version of IPCameraFrameGrabber that allocated a 4096 element byte array for every read
* caused about 200MB of allocations within 13 seconds. In this version, almost no additional heap space
* is typically allocated per frame.
*/
private void ensureBufferCapacity(int desiredCapacity) {
int capacity = pixelBuffer.length;
while (capacity < desiredCapacity) {
capacity *= 2;
}
if (capacity > pixelBuffer.length) {
pixelBuffer = new byte[capacity];
}
}
/**
* Returns the value of the {@code long} argument;
* throwing an exception if the value overflows an {@code int}.
*
* @param value the long value
* @return the argument as an int
* @throws ArithmeticException if the {@code argument} overflows an int
* @see Java 8 Implementation
*/
private static int toIntExact(long value) {
if ((int) value != value) {
throw new ArithmeticException("integer overflow");
}
return (int) value;
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/ImageAligner.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import org.bytedeco.javacv.ImageTransformer.Parameters;
import org.bytedeco.opencv.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_core.*;
/**
*
* @author Samuel Audet
*/
public interface ImageAligner {
public class Settings extends BaseChildSettings implements Cloneable {
public Settings() { }
public Settings(Settings s) {
pyramidLevelMin = s.pyramidLevelMin;
pyramidLevelMax = s.pyramidLevelMax;
thresholdsZero = s.thresholdsZero;
thresholdsOutlier = s.thresholdsOutlier;
thresholdsMulRMSE = s.thresholdsMulRMSE;
}
int pyramidLevelMin = 0;
int pyramidLevelMax = 4;
double[] thresholdsZero = { 0.04, 0.03, 0.02, 0.01, 0 };
double[] thresholdsOutlier = { 0.2 };
boolean thresholdsMulRMSE = false;
public int getPyramidLevelMin() {
return pyramidLevelMin;
}
public void setPyramidLevelMin(int pyramidLevelMin) {
this.pyramidLevelMin = pyramidLevelMin;
}
public int getPyramidLevelMax() {
return pyramidLevelMax;
}
public void setPyramidLevelMax(int pyramidLevelMax) {
this.pyramidLevelMax = pyramidLevelMax;
}
public double[] getThresholdsZero() {
return thresholdsZero;
}
public void setThresholdsZero(double[] thresholdsZero) {
this.thresholdsZero = thresholdsZero;
}
public double[] getThresholdsOutlier() {
return thresholdsOutlier;
}
public void setThresholdsOutlier(double[] thresholdsOutlier) {
this.thresholdsOutlier = thresholdsOutlier;
}
public boolean isThresholdsMulRMSE() {
return thresholdsMulRMSE;
}
public void setThresholdsMulRMSE(boolean thresholdsMulRMSE) {
this.thresholdsMulRMSE = thresholdsMulRMSE;
}
@Override public Settings clone() {
return new Settings(this);
}
}
Settings getSettings();
void setSettings(Settings settings);
IplImage getTemplateImage();
void setTemplateImage(IplImage template0, double[] roiPts);
IplImage getTargetImage();
void setTargetImage(IplImage target0);
int getPyramidLevel();
void setPyramidLevel(int pyramidLevel);
Parameters getParameters();
void setParameters(Parameters parameters);
double[] getTransformedRoiPts();
IplImage getTransformedImage();
IplImage getResidualImage();
IplImage getMaskImage();
double getRMSE();
CvRect getRoi();
IplImage[] getImages();
boolean iterate(double[] delta);
}
================================================
FILE: src/main/java/org/bytedeco/javacv/ImageAlignerCL.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import com.jogamp.opencl.CLImage2d;
/**
*
* @author Samuel Audet
*/
public interface ImageAlignerCL extends ImageAligner {
CLImage2d getTemplateImageCL();
void setTemplateImageCL(CLImage2d template0, double[] roiPts);
CLImage2d getTargetImageCL();
void setTargetImageCL(CLImage2d target0);
CLImage2d getTransformedImageCL();
CLImage2d getResidualImageCL();
CLImage2d getMaskImageCL();
CLImage2d[] getImagesCL();
}
================================================
FILE: src/main/java/org/bytedeco/javacv/ImageTransformer.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.DoubleBuffer;
import org.bytedeco.opencv.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_core.*;
/**
*
* @author Samuel Audet
*/
public interface ImageTransformer {
public class Data {
public Data() { this(null, null, null, null, 0, 0, 0, null, null, 0); }
public Data(IplImage srcImg, IplImage subImg, IplImage srcDotImg, IplImage mask,
double zeroThreshold, double outlierThreshold, int pyramidLevel,
IplImage transImg, IplImage dstImg, int dstDstDotLength) {
this.srcImg = srcImg;
this.subImg = subImg;
this.srcDotImg = srcDotImg;
this.mask = mask;
this.zeroThreshold = zeroThreshold;
this.outlierThreshold = outlierThreshold;
this.pyramidLevel = pyramidLevel;
this.transImg = transImg;
this.dstImg = dstImg;
this.dstDstDot = dstDstDotLength == 0 ? null :
ByteBuffer.allocateDirect(dstDstDotLength*8).
order(ByteOrder.nativeOrder()).asDoubleBuffer();
}
// input
public IplImage srcImg, subImg, srcDotImg, mask;
public double zeroThreshold, outlierThreshold;
public int pyramidLevel;
// output
public IplImage transImg, dstImg;
public int dstCount, dstCountZero, dstCountOutlier;
public double srcDstDot;
public DoubleBuffer dstDstDot;
}
public interface Parameters extends Cloneable {
int size();
double[] get();
double get(int i);
void set(double ... p);
void set(int i, double p);
void set(Parameters p);
void reset(boolean asIdentity);
double getConstraintError();
void compose(Parameters p1, boolean inverse1, Parameters p2, boolean inverse2);
boolean preoptimize();
double[] getSubspace();
void setSubspace(double ... p);
Parameters clone();
}
Parameters createParameters();
void transform(Data[] data, CvRect roi, Parameters[] parameters, boolean[] inverses);
void transform(CvMat srcPts, CvMat dstPts, Parameters parameters, boolean inverse);
}
================================================
FILE: src/main/java/org/bytedeco/javacv/ImageTransformerCL.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import com.jogamp.opencl.CLBuffer;
import com.jogamp.opencl.CLImage2d;
import java.nio.ByteBuffer;
import java.nio.FloatBuffer;
/**
*
* @author Samuel Audet
*/
public interface ImageTransformerCL extends ImageTransformer {
public class InputData {
public InputData() { this(true); }
public InputData(boolean autoWrite) { this.autoWrite = autoWrite; }
public int pyramidLevel = 0;
public int roiX = 0, roiY = 0, roiWidth = 0, roiHeight = 0;
public double zeroThreshold = 0, outlierThreshold = 0;
CLBuffer buffer = null;
boolean autoWrite = true;
CLBuffer getBuffer(JavaCVCL context) {
int structSize = 4*4;
if (buffer == null || buffer.getCLSize() < structSize) {
if (buffer != null) buffer.release();
buffer = context.getCLContext().createByteBuffer(structSize, CLBuffer.Mem.READ_ONLY);
}
return buffer;
}
public CLBuffer writeBuffer(JavaCVCL context) {
getBuffer(context);
ByteBuffer byteBuffer = (ByteBuffer)buffer.getBuffer().rewind();
byteBuffer.putInt(roiY).putInt(roiHeight).putFloat((float)zeroThreshold)
.putFloat((float)outlierThreshold).rewind();
context.writeBuffer(buffer, false); // upload input data
return buffer;
}
}
public class OutputData {
public OutputData() { this(true); }
public OutputData(boolean autoRead) { this.autoRead = autoRead; }
public int dstCount = 0, dstCountZero = 0, dstCountOutlier = 0;
public FloatBuffer srcDstDot = null, dstDstDot = null;
CLBuffer buffer = null;
boolean autoRead = true;
CLBuffer getBuffer(JavaCVCL context, int dotSize, int reduceSize) {
int structSize = 4*(4 + dotSize + dotSize*dotSize);
if (buffer == null || buffer.getCLSize() < structSize*reduceSize) {
if (buffer != null) buffer.release();
buffer = context.getCLContext().createByteBuffer(structSize*reduceSize);
ByteBuffer byteBuffer = buffer.getBuffer();
byteBuffer.position(4*4); srcDstDot = byteBuffer.asFloatBuffer();
byteBuffer.position(4*(4 + dotSize)); dstDstDot = byteBuffer.asFloatBuffer();
byteBuffer.rewind();
}
return buffer;
}
public CLBuffer readBuffer(JavaCVCL context) {
//getBuffer(context, dotSize, reduceSize);
context.readBuffer(buffer, true); // read results back (blocking read)
ByteBuffer byteBuffer = buffer.getBuffer();
dstCount = byteBuffer.getInt(4);
dstCountZero = byteBuffer.getInt(8);
dstCountOutlier = byteBuffer.getInt(12);
return buffer;
}
}
JavaCVCL getContext();
void transform(CLImage2d srcImg, CLImage2d subImg, CLImage2d srcDotImg, CLImage2d transImg, CLImage2d dstImg,
CLImage2d mask, ImageTransformer.Parameters[] parameters, boolean[] inverses, InputData inputData, OutputData outputData);
}
================================================
FILE: src/main/java/org/bytedeco/javacv/Java2DFrameConverter.java
================================================
/*
* Copyright (C) 2015-2019 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.awt.Graphics;
import java.awt.Rectangle;
import java.awt.Transparency;
import java.awt.color.ColorSpace;
import java.awt.image.BufferedImage;
import java.awt.image.ColorModel;
import java.awt.image.ComponentColorModel;
import java.awt.image.ComponentSampleModel;
import java.awt.image.DataBuffer;
import java.awt.image.DataBufferByte;
import java.awt.image.DataBufferDouble;
import java.awt.image.DataBufferFloat;
import java.awt.image.DataBufferInt;
import java.awt.image.DataBufferShort;
import java.awt.image.DataBufferUShort;
import java.awt.image.MultiPixelPackedSampleModel;
import java.awt.image.Raster;
import java.awt.image.SampleModel;
import java.awt.image.SinglePixelPackedSampleModel;
import java.awt.image.WritableRaster;
import java.nio.Buffer;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.DoubleBuffer;
import java.nio.FloatBuffer;
import java.nio.IntBuffer;
import java.nio.ShortBuffer;
/**
* A utility class to copy data between {@link Frame} and {@link BufferedImage}.
* Since {@link BufferedImage} does not support NIO buffers, we cannot share
* allocated memory with {@link Frame}.
*
* @author Samuel Audet
*/
public class Java2DFrameConverter extends FrameConverter {
@Override public Frame convert(BufferedImage img) {
return getFrame(img);
}
@Override public BufferedImage convert(Frame frame) {
return getBufferedImage(frame);
}
/**
* @param source
* @return null if source is null
*/
public static BufferedImage cloneBufferedImage(BufferedImage source) {
if (source == null) {
return null;
}
int type = source.getType();
if (type == BufferedImage.TYPE_CUSTOM) {
return new BufferedImage(
source.getColorModel(),
source.copyData(null),
source.isAlphaPremultiplied(),
null
);
} else {
BufferedImage copy = new BufferedImage(source.getWidth(), source.getHeight(), type);
Graphics g = copy.getGraphics();
g.drawImage(source, 0, 0, null);
g.dispose();
return copy;
}
}
public static final byte[]
gamma22 = new byte[256],
gamma22inv = new byte[256];
static {
for (int i = 0; i < 256; i++) {
gamma22[i] = (byte)Math.round(Math.pow(i/255.0, 2.2)*255.0);
gamma22inv[i] = (byte)Math.round(Math.pow(i/255.0, 1/2.2)*255.0);
}
}
public static int decodeGamma22(int value) {
return gamma22[value & 0xFF] & 0xFF;
}
public static int encodeGamma22(int value) {
return gamma22inv[value & 0xFF] & 0xFF;
}
public static void flipCopyWithGamma(ByteBuffer srcBuf, int srcBufferIndex, int srcStep,
ByteBuffer dstBuf, int dstBufferIndex, int dstStep,
boolean signed, double gamma, boolean flip, int channels) {
assert srcBuf != dstBuf;
int w = Math.min(srcStep, dstStep);
int srcLine = srcBufferIndex, dstLine = dstBufferIndex;
byte[] buffer = new byte[channels];
while (srcLine < srcBuf.capacity() && dstLine < dstBuf.capacity()) {
if (flip) {
srcBufferIndex = srcBuf.capacity() - srcLine - srcStep;
} else {
srcBufferIndex = srcLine;
}
dstBufferIndex = dstLine;
w = Math.min(Math.min(w, srcBuf.capacity() - srcBufferIndex), dstBuf.capacity() - dstBufferIndex);
if (signed) {
if (channels > 1) {
for (int x = 0; x < w; x+=channels) {
for (int z = 0; z < channels; z++) {
int in = srcBuf.get(srcBufferIndex++);
byte out;
if (gamma == 1.0) {
out = (byte)in;
} else {
out = (byte)Math.round(Math.pow((double)in/Byte.MAX_VALUE, gamma)*Byte.MAX_VALUE);
}
buffer[z] = out;
}
for (int z = channels-1; z >= 0; z--) {
dstBuf.put(dstBufferIndex++, buffer[z]);
}
}
} else {
for (int x = 0; x < w; x++) {
int in = srcBuf.get(srcBufferIndex++);
byte out;
if (gamma == 1.0) {
out = (byte)in;
} else {
out = (byte)Math.round(Math.pow((double)in/Byte.MAX_VALUE, gamma)*Byte.MAX_VALUE);
}
dstBuf.put(dstBufferIndex++, out);
}
}
} else {
if (channels > 1) {
for (int x = 0; x < w; x+=channels) {
for (int z = 0; z < channels; z++) {
byte out;
int in = srcBuf.get(srcBufferIndex++) & 0xFF;
if (gamma == 1.0) {
out = (byte)in;
} else if (gamma == 2.2) {
out = gamma22[in];
} else if (gamma == 1/2.2) {
out = gamma22inv[in];
} else {
out = (byte)Math.round(Math.pow((double)in/0xFF, gamma)*0xFF);
}
buffer[z] = out;
}
for (int z = channels-1; z >= 0; z--) {
dstBuf.put(dstBufferIndex++, buffer[z]);
}
}
} else {
for (int x = 0; x < w; x++) {
byte out;
int in = srcBuf.get(srcBufferIndex++) & 0xFF;
if (gamma == 1.0) {
out = (byte)in;
} else if (gamma == 2.2) {
out = gamma22[in];
} else if (gamma == 1/2.2) {
out = gamma22inv[in];
} else {
out = (byte)Math.round(Math.pow((double)in/0xFF, gamma)*0xFF);
}
dstBuf.put(dstBufferIndex++, out);
}
}
}
srcLine += srcStep;
dstLine += dstStep;
}
}
public static void flipCopyWithGamma(ShortBuffer srcBuf, int srcBufferIndex, int srcStep,
ShortBuffer dstBuf, int dstBufferIndex, int dstStep,
boolean signed, double gamma, boolean flip, int channels) {
assert srcBuf != dstBuf;
int w = Math.min(srcStep, dstStep);
int srcLine = srcBufferIndex, dstLine = dstBufferIndex;
short[] buffer = new short[channels];
while (srcLine < srcBuf.capacity() && dstLine < dstBuf.capacity()) {
if (flip) {
srcBufferIndex = srcBuf.capacity() - srcLine - srcStep;
} else {
srcBufferIndex = srcLine;
}
dstBufferIndex = dstLine;
w = Math.min(Math.min(w, srcBuf.capacity() - srcBufferIndex), dstBuf.capacity() - dstBufferIndex);
if (signed) {
if (channels > 1) {
for (int x = 0; x < w; x+=channels) {
for (int z = 0; z < channels; z++) {
int in = srcBuf.get(srcBufferIndex++);
short out;
if (gamma == 1.0) {
out = (short)in;
} else {
out = (short)Math.round(Math.pow((double)in/Short.MAX_VALUE, gamma)*Short.MAX_VALUE);
}
buffer[z] = out;
}
for (int z = channels-1; z >= 0; z--) {
dstBuf.put(dstBufferIndex++, buffer[z]);
}
}
} else {
for (int x = 0; x < w; x++) {
int in = srcBuf.get(srcBufferIndex++);
short out;
if (gamma == 1.0) {
out = (short)in;
} else {
out = (short)Math.round(Math.pow((double)in/Short.MAX_VALUE, gamma)*Short.MAX_VALUE);
}
dstBuf.put(dstBufferIndex++, out);
}
}
} else {
if (channels > 1) {
for (int x = 0; x < w; x+=channels) {
for (int z = 0; z < channels; z++) {
int in = srcBuf.get(srcBufferIndex++);
short out;
if (gamma == 1.0) {
out = (short)in;
} else {
out = (short)Math.round(Math.pow((double)in/0xFFFF, gamma)*0xFFFF);
}
buffer[z] = out;
}
for (int z = channels-1; z >= 0; z--) {
dstBuf.put(dstBufferIndex++, buffer[z]);
}
}
} else {
for (int x = 0; x < w; x++) {
int in = srcBuf.get(srcBufferIndex++) & 0xFFFF;
short out;
if (gamma == 1.0) {
out = (short)in;
} else {
out = (short)Math.round(Math.pow((double)in/0xFFFF, gamma)*0xFFFF);
}
dstBuf.put(dstBufferIndex++, out);
}
}
}
srcLine += srcStep;
dstLine += dstStep;
}
}
public static void flipCopyWithGamma(IntBuffer srcBuf, int srcBufferIndex, int srcStep,
IntBuffer dstBuf, int dstBufferIndex, int dstStep,
double gamma, boolean flip, int channels) {
assert srcBuf != dstBuf;
int w = Math.min(srcStep, dstStep);
int srcLine = srcBufferIndex, dstLine = dstBufferIndex;
int[] buffer = new int[channels];
while (srcLine < srcBuf.capacity() && dstLine < dstBuf.capacity()) {
if (flip) {
srcBufferIndex = srcBuf.capacity() - srcLine - srcStep;
} else {
srcBufferIndex = srcLine;
}
dstBufferIndex = dstLine;
w = Math.min(Math.min(w, srcBuf.capacity() - srcBufferIndex), dstBuf.capacity() - dstBufferIndex);
if (channels > 1) {
for (int x = 0; x < w; x+=channels) {
for (int z = 0; z < channels; z++) {
int in = srcBuf.get(srcBufferIndex++);
int out;
if (gamma == 1.0) {
out = (int)in;
} else {
out = (int)Math.round(Math.pow((double)in/Integer.MAX_VALUE, gamma)*Integer.MAX_VALUE);
}
buffer[z] = out;
}
for (int z = channels-1; z >= 0; z--) {
dstBuf.put(dstBufferIndex++, buffer[z]);
}
}
} else {
for (int x = 0; x < w; x++) {
int in = srcBuf.get(srcBufferIndex++);
int out;
if (gamma == 1.0) {
out = in;
} else {
out = (int)Math.round(Math.pow((double)in/Integer.MAX_VALUE, gamma)*Integer.MAX_VALUE);
}
dstBuf.put(dstBufferIndex++, out);
}
}
srcLine += srcStep;
dstLine += dstStep;
}
}
public static void flipCopyWithGamma(FloatBuffer srcBuf, int srcBufferIndex, int srcStep,
FloatBuffer dstBuf, int dstBufferIndex, int dstStep,
double gamma, boolean flip, int channels) {
assert srcBuf != dstBuf;
int w = Math.min(srcStep, dstStep);
int srcLine = srcBufferIndex, dstLine = dstBufferIndex;
float[] buffer = new float[channels];
while (srcLine < srcBuf.capacity() && dstLine < dstBuf.capacity()) {
if (flip) {
srcBufferIndex = srcBuf.capacity() - srcLine - srcStep;
} else {
srcBufferIndex = srcLine;
}
dstBufferIndex = dstLine;
w = Math.min(Math.min(w, srcBuf.capacity() - srcBufferIndex), dstBuf.capacity() - dstBufferIndex);
if (channels > 1) {
for (int x = 0; x < w; x+=channels) {
for (int z = 0; z < channels; z++) {
float in = srcBuf.get(srcBufferIndex++);
float out;
if (gamma == 1.0) {
out = in;
} else {
out = (float)Math.pow(in, gamma);
}
buffer[z] = out;
}
for (int z = channels-1; z >= 0; z--) {
dstBuf.put(dstBufferIndex++, buffer[z]);
}
}
} else {
for (int x = 0; x < w; x++) {
float in = srcBuf.get(srcBufferIndex++);
float out;
if (gamma == 1.0) {
out = in;
} else {
out = (float)Math.pow(in, gamma);
}
dstBuf.put(dstBufferIndex++, out);
}
}
srcLine += srcStep;
dstLine += dstStep;
}
}
public static void flipCopyWithGamma(DoubleBuffer srcBuf, int srcBufferIndex, int srcStep,
DoubleBuffer dstBuf, int dstBufferIndex, int dstStep,
double gamma, boolean flip, int channels) {
assert srcBuf != dstBuf;
int w = Math.min(srcStep, dstStep);
int srcLine = srcBufferIndex, dstLine = dstBufferIndex;
double[] buffer = new double[channels];
while (srcLine < srcBuf.capacity() && dstLine < dstBuf.capacity()) {
if (flip) {
srcBufferIndex = srcBuf.capacity() - srcLine - srcStep;
} else {
srcBufferIndex = srcLine;
}
dstBufferIndex = dstLine;
w = Math.min(Math.min(w, srcBuf.capacity() - srcBufferIndex), dstBuf.capacity() - dstBufferIndex);
if (channels > 1) {
for (int x = 0; x < w; x+=channels) {
for (int z = 0; z < channels; z++) {
double in = srcBuf.get(srcBufferIndex++);
double out;
if (gamma == 1.0) {
out = in;
} else {
out = Math.pow(in, gamma);
}
buffer[z] = out;
}
for (int z = channels-1; z >= 0; z--) {
dstBuf.put(dstBufferIndex++, buffer[z]);
}
}
} else {
for (int x = 0; x < w; x++) {
double in = srcBuf.get(srcBufferIndex++);
double out;
if (gamma == 1.0) {
out = in;
} else {
out = Math.pow(in, gamma);
}
dstBuf.put(dstBufferIndex++, out);
}
}
srcLine += srcStep;
dstLine += dstStep;
}
}
public static void applyGamma(Frame frame, double gamma) {
applyGamma(frame.image[0], frame.imageDepth, frame.imageStride, gamma);
}
public static void applyGamma(Buffer buffer, int depth, int stride, double gamma) {
if (gamma == 1.0) {
return;
}
switch (depth) {
case Frame.DEPTH_UBYTE:
flipCopyWithGamma(((ByteBuffer)buffer).asReadOnlyBuffer(), 0, stride, (ByteBuffer)buffer, 0, stride, false, gamma, false, 0);
break;
case Frame.DEPTH_BYTE:
flipCopyWithGamma(((ByteBuffer)buffer).asReadOnlyBuffer(), 0, stride, (ByteBuffer)buffer, 0, stride, true, gamma, false, 0);
break;
case Frame.DEPTH_USHORT:
flipCopyWithGamma(((ShortBuffer)buffer).asReadOnlyBuffer(), 0, stride, (ShortBuffer)buffer, 0, stride, false, gamma, false, 0);
break;
case Frame.DEPTH_SHORT:
flipCopyWithGamma(((ShortBuffer)buffer).asReadOnlyBuffer(), 0, stride, (ShortBuffer)buffer, 0, stride, true, gamma, false, 0);
break;
case Frame.DEPTH_INT:
flipCopyWithGamma(((IntBuffer)buffer).asReadOnlyBuffer(), 0, stride, (IntBuffer)buffer, 0, stride, gamma, false, 0);
break;
case Frame.DEPTH_FLOAT:
flipCopyWithGamma(((FloatBuffer)buffer).asReadOnlyBuffer(), 0, stride, (FloatBuffer)buffer, 0, stride, gamma, false, 0);
break;
case Frame.DEPTH_DOUBLE:
flipCopyWithGamma(((DoubleBuffer)buffer).asReadOnlyBuffer(), 0, stride, (DoubleBuffer)buffer, 0, stride, gamma, false, 0);
break;
default:
assert false;
}
}
public static void copy(Frame frame, BufferedImage bufferedImage) {
copy(frame, bufferedImage, 1.0);
}
public static void copy(Frame frame, BufferedImage bufferedImage, double gamma) {
copy(frame, bufferedImage, gamma, false, null);
}
public static void copy(Frame frame, BufferedImage bufferedImage, double gamma, boolean flipChannels, Rectangle roi) {
Buffer in = frame.image[0];
int bufferIndex = roi == null ? 0 : roi.y*frame.imageStride + roi.x*frame.imageChannels;
SampleModel sm = bufferedImage.getSampleModel();
Raster r = bufferedImage.getRaster();
DataBuffer out = r.getDataBuffer();
int x = -r.getSampleModelTranslateX();
int y = -r.getSampleModelTranslateY();
int step = sm.getWidth()*sm.getNumBands();
int channels = sm.getNumBands();
if (sm instanceof ComponentSampleModel) {
step = ((ComponentSampleModel)sm).getScanlineStride();
channels = ((ComponentSampleModel)sm).getPixelStride();
} else if (sm instanceof SinglePixelPackedSampleModel) {
step = ((SinglePixelPackedSampleModel)sm).getScanlineStride();
channels = 1;
} else if (sm instanceof MultiPixelPackedSampleModel) {
step = ((MultiPixelPackedSampleModel)sm).getScanlineStride();
channels = ((MultiPixelPackedSampleModel)sm).getPixelBitStride()/8; // ??
}
int start = y*step + x*channels;
if (out instanceof DataBufferByte) {
byte[] a = ((DataBufferByte)out).getData();
flipCopyWithGamma((ByteBuffer)in, bufferIndex, frame.imageStride, ByteBuffer.wrap(a), start, step, false, gamma, false, flipChannels ? channels : 0);
} else if (out instanceof DataBufferDouble) {
double[] a = ((DataBufferDouble)out).getData();
flipCopyWithGamma((DoubleBuffer)in, bufferIndex, frame.imageStride, DoubleBuffer.wrap(a), start, step, gamma, false, flipChannels ? channels : 0);
} else if (out instanceof DataBufferFloat) {
float[] a = ((DataBufferFloat)out).getData();
flipCopyWithGamma((FloatBuffer)in, bufferIndex, frame.imageStride, FloatBuffer.wrap(a), start, step, gamma, false, flipChannels ? channels : 0);
} else if (out instanceof DataBufferInt) {
int[] a = ((DataBufferInt)out).getData();
int stride = frame.imageStride;
if (in instanceof ByteBuffer) {
in = ((ByteBuffer)in).order(flipChannels ? ByteOrder.LITTLE_ENDIAN : ByteOrder.BIG_ENDIAN).asIntBuffer();
stride /= 4;
}
flipCopyWithGamma((IntBuffer)in, bufferIndex, stride, IntBuffer.wrap(a), start, step, gamma, false, flipChannels ? channels : 0);
} else if (out instanceof DataBufferShort) {
short[] a = ((DataBufferShort)out).getData();
flipCopyWithGamma((ShortBuffer)in, bufferIndex, frame.imageStride, ShortBuffer.wrap(a), start, step, true, gamma, false, flipChannels ? channels : 0);
} else if (out instanceof DataBufferUShort) {
short[] a = ((DataBufferUShort)out).getData();
flipCopyWithGamma((ShortBuffer)in, bufferIndex, frame.imageStride, ShortBuffer.wrap(a), start, step, false, gamma, false, flipChannels ? channels : 0);
} else {
assert false;
}
}
public static void copy(BufferedImage image, Frame frame) {
copy(image, frame, 1.0);
}
public static void copy(BufferedImage image, Frame frame, double gamma) {
copy(image, frame, gamma, false, null);
}
public static void copy(BufferedImage image, Frame frame, double gamma, boolean flipChannels, Rectangle roi) {
Buffer out = frame.image[0];
int bufferIndex = roi == null ? 0 : roi.y*frame.imageStride + roi.x*frame.imageChannels;
SampleModel sm = image.getSampleModel();
Raster r = image.getRaster();
DataBuffer in = r.getDataBuffer();
int x = -r.getSampleModelTranslateX();
int y = -r.getSampleModelTranslateY();
int step = sm.getWidth()*sm.getNumBands();
int channels = sm.getNumBands();
if (sm instanceof ComponentSampleModel) {
step = ((ComponentSampleModel)sm).getScanlineStride();
channels = ((ComponentSampleModel)sm).getPixelStride();
} else if (sm instanceof SinglePixelPackedSampleModel) {
step = ((SinglePixelPackedSampleModel)sm).getScanlineStride();
channels = 1;
} else if (sm instanceof MultiPixelPackedSampleModel) {
step = ((MultiPixelPackedSampleModel)sm).getScanlineStride();
channels = ((MultiPixelPackedSampleModel)sm).getPixelBitStride()/8; // ??
}
int start = y*step + x*channels;
if (in instanceof DataBufferByte) {
byte[] a = ((DataBufferByte)in).getData();
flipCopyWithGamma(ByteBuffer.wrap(a), start, step, (ByteBuffer)out, bufferIndex, frame.imageStride, false, gamma, false, flipChannels ? channels : 0);
} else if (in instanceof DataBufferDouble) {
double[] a = ((DataBufferDouble)in).getData();
flipCopyWithGamma(DoubleBuffer.wrap(a), start, step, (DoubleBuffer)out, bufferIndex, frame.imageStride, gamma, false, flipChannels ? channels : 0);
} else if (in instanceof DataBufferFloat) {
float[] a = ((DataBufferFloat)in).getData();
flipCopyWithGamma(FloatBuffer.wrap(a), start, step, (FloatBuffer)out, bufferIndex, frame.imageStride, gamma, false, flipChannels ? channels : 0);
} else if (in instanceof DataBufferInt) {
int[] a = ((DataBufferInt)in).getData();
int stride = frame.imageStride;
if (out instanceof ByteBuffer) {
out = ((ByteBuffer)out).order(flipChannels ? ByteOrder.LITTLE_ENDIAN : ByteOrder.BIG_ENDIAN).asIntBuffer();
stride /= 4;
}
flipCopyWithGamma(IntBuffer.wrap(a), start, step, (IntBuffer)out, bufferIndex, stride, gamma, false, flipChannels ? channels : 0);
} else if (in instanceof DataBufferShort) {
short[] a = ((DataBufferShort)in).getData();
flipCopyWithGamma(ShortBuffer.wrap(a), start, step, (ShortBuffer)out, bufferIndex, frame.imageStride, true, gamma, false, flipChannels ? channels : 0);
} else if (in instanceof DataBufferUShort) {
short[] a = ((DataBufferUShort)in).getData();
flipCopyWithGamma(ShortBuffer.wrap(a), start, step, (ShortBuffer)out, bufferIndex, frame.imageStride, false, gamma, false, flipChannels ? channels : 0);
} else {
assert false;
}
}
protected BufferedImage bufferedImage = null;
public static int getBufferedImageType(Frame frame) {
// precanned BufferedImage types are confusing... in practice though,
// they all use the sRGB color model when blitting:
// http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5051418
// and we should use them because they are *A LOT* faster with Java 2D.
// workaround: do gamma correction ourselves ("gamma" parameter)
// since we'll never use getRGB() and setRGB(), right?
int type = BufferedImage.TYPE_CUSTOM;
if (frame.imageChannels == 1) {
if (frame.imageDepth == Frame.DEPTH_UBYTE || frame.imageDepth == Frame.DEPTH_BYTE) {
type = BufferedImage.TYPE_BYTE_GRAY;
} else if (frame.imageDepth == Frame.DEPTH_USHORT) {
type = BufferedImage.TYPE_USHORT_GRAY;
}
} else if (frame.imageChannels == 3) {
if (frame.imageDepth == Frame.DEPTH_UBYTE || frame.imageDepth == Frame.DEPTH_BYTE) {
type = BufferedImage.TYPE_3BYTE_BGR;
}
} else if (frame.imageChannels == 4) {
// The channels end up reversed of what we need for OpenCL.
// We work around this in copyTo() and copyFrom() by
// inversing the channels to let us use RGBA in our IplImage.
if (frame.imageDepth == Frame.DEPTH_UBYTE || frame.imageDepth == Frame.DEPTH_BYTE) {
type = BufferedImage.TYPE_4BYTE_ABGR;
}
}
return type;
}
public BufferedImage getBufferedImage(Frame frame) {
return getBufferedImage(frame, 1.0);
}
public BufferedImage getBufferedImage(Frame frame, double gamma) {
return getBufferedImage(frame, gamma, false, null);
}
public BufferedImage getBufferedImage(Frame frame, double gamma, boolean flipChannels, ColorSpace cs) {
if (frame == null || frame.image == null) {
return null;
}
int type = getBufferedImageType(frame);
if (bufferedImage == null || bufferedImage.getWidth() != frame.imageWidth
|| bufferedImage.getHeight() != frame.imageHeight || bufferedImage.getType() != type) {
bufferedImage = type == BufferedImage.TYPE_CUSTOM || cs != null ? null
: new BufferedImage(frame.imageWidth, frame.imageHeight, type);
}
if (bufferedImage == null) {
boolean alpha = false;
int[] offsets = null;
if (frame.imageChannels == 1) {
alpha = false;
if (cs == null) {
cs = ColorSpace.getInstance(ColorSpace.CS_GRAY);
}
offsets = new int[] {0};
} else if (frame.imageChannels == 3) {
alpha = false;
if (cs == null) {
cs = ColorSpace.getInstance(ColorSpace.CS_LINEAR_RGB);
}
// raster in "BGR" order like OpenCV..
offsets = new int[] {2, 1, 0};
} else if (frame.imageChannels == 4) {
alpha = true;
if (cs == null) {
cs = ColorSpace.getInstance(ColorSpace.CS_LINEAR_RGB);
}
// raster in "RGBA" order for OpenCL.. alpha needs to be last
offsets = new int[] {0, 1, 2, 3};
} else {
assert false;
}
ColorModel cm = null;
WritableRaster wr = null;
if (frame.imageDepth == Frame.DEPTH_UBYTE || frame.imageDepth == Frame.DEPTH_BYTE) {
cm = new ComponentColorModel(cs, alpha,
false, Transparency.OPAQUE, DataBuffer.TYPE_BYTE);
wr = Raster.createWritableRaster(new ComponentSampleModel(
DataBuffer.TYPE_BYTE, frame.imageWidth, frame.imageHeight, frame.imageChannels, frame.imageStride,
offsets), null);
} else if (frame.imageDepth == Frame.DEPTH_USHORT) {
cm = new ComponentColorModel(cs, alpha,
false, Transparency.OPAQUE, DataBuffer.TYPE_USHORT);
wr = Raster.createWritableRaster(new ComponentSampleModel(
DataBuffer.TYPE_USHORT, frame.imageWidth, frame.imageHeight, frame.imageChannels, frame.imageStride,
offsets), null);
} else if (frame.imageDepth == Frame.DEPTH_SHORT) {
cm = new ComponentColorModel(cs, alpha,
false, Transparency.OPAQUE, DataBuffer.TYPE_SHORT);
wr = Raster.createWritableRaster(new ComponentSampleModel(
DataBuffer.TYPE_SHORT, frame.imageWidth, frame.imageHeight, frame.imageChannels, frame.imageStride,
offsets), null);
} else if (frame.imageDepth == Frame.DEPTH_INT) {
cm = new ComponentColorModel(cs, alpha,
false, Transparency.OPAQUE, DataBuffer.TYPE_INT);
wr = Raster.createWritableRaster(new ComponentSampleModel(
DataBuffer.TYPE_INT, frame.imageWidth, frame.imageHeight, frame.imageChannels, frame.imageStride,
offsets), null);
} else if (frame.imageDepth == Frame.DEPTH_FLOAT) {
cm = new ComponentColorModel(cs, alpha,
false, Transparency.OPAQUE, DataBuffer.TYPE_FLOAT);
wr = Raster.createWritableRaster(new ComponentSampleModel(
DataBuffer.TYPE_FLOAT, frame.imageWidth, frame.imageHeight, frame.imageChannels, frame.imageStride,
offsets), null);
} else if (frame.imageDepth == Frame.DEPTH_DOUBLE) {
cm = new ComponentColorModel(cs, alpha,
false, Transparency.OPAQUE, DataBuffer.TYPE_DOUBLE);
wr = Raster.createWritableRaster(new ComponentSampleModel(
DataBuffer.TYPE_DOUBLE, frame.imageWidth, frame.imageHeight, frame.imageChannels, frame.imageStride,
offsets), null);
} else {
assert false;
}
bufferedImage = new BufferedImage(cm, wr, false, null);
}
if (bufferedImage != null) {
copy(frame, bufferedImage, gamma, flipChannels, null);
}
return bufferedImage;
}
/**
* Returns a Frame based on a BufferedImage.
*/
public Frame getFrame(BufferedImage image) {
return getFrame(image, 1.0);
}
/**
* Returns a Frame based on a BufferedImage, and given gamma.
*/
public Frame getFrame(BufferedImage image, double gamma) {
return getFrame(image, gamma, false);
}
/**
* Returns a Frame based on a BufferedImage, given gamma, and inverted channels flag.
*/
public Frame getFrame(BufferedImage image, double gamma, boolean flipChannels) {
if (image == null) {
return null;
}
SampleModel sm = image.getSampleModel();
int depth = 0, numChannels = sm.getNumBands();
switch (image.getType()) {
case BufferedImage.TYPE_INT_RGB:
case BufferedImage.TYPE_INT_ARGB:
case BufferedImage.TYPE_INT_ARGB_PRE:
case BufferedImage.TYPE_INT_BGR:
depth = Frame.DEPTH_UBYTE;
numChannels = 4;
break;
}
if (depth == 0 || numChannels == 0) {
switch (sm.getDataType()) {
case DataBuffer.TYPE_BYTE: depth = Frame.DEPTH_UBYTE; break;
case DataBuffer.TYPE_USHORT: depth = Frame.DEPTH_USHORT; break;
case DataBuffer.TYPE_SHORT: depth = Frame.DEPTH_SHORT; break;
case DataBuffer.TYPE_INT: depth = Frame.DEPTH_INT; break;
case DataBuffer.TYPE_FLOAT: depth = Frame.DEPTH_FLOAT; break;
case DataBuffer.TYPE_DOUBLE: depth = Frame.DEPTH_DOUBLE; break;
default: assert false;
}
}
if (frame == null || frame.imageWidth != image.getWidth() || frame.imageHeight != image.getHeight()
|| frame.imageDepth != depth || frame.imageChannels != numChannels) {
if (frame != null) {
frame.close();
}
frame = new Frame(image.getWidth(), image.getHeight(), depth, numChannels);
}
copy(image, frame, gamma, flipChannels, null);
return frame;
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/Java2DFrameUtils.java
================================================
package org.bytedeco.javacv;
import java.awt.image.BufferedImage;
import java.awt.image.ColorModel;
import java.awt.image.WritableRaster;
import org.bytedeco.javacv.Frame;
import org.bytedeco.javacv.Java2DFrameConverter;
import org.bytedeco.javacv.OpenCVFrameConverter;
import org.bytedeco.opencv.opencv_core.IplImage;
import org.bytedeco.opencv.opencv_core.Mat;
/**
* Convenience class for performing various conversions between Mat, IplImage,
* BufferedImage and Frame objects. Methods are synchronized because the
* underlying JavaCV converters aren't safe for concurrent access.
*
* All created Frame, Mat, IplImages and BufferedImages are cloned internally
* after creation so that their memory locations remain valid after the
* converters which created them are closed or garbage collected. This is safer
* for the caller, but may be slower.
*
* If performance is critical, use the *FrameConverter classes directly, after
* reading about the image validity constraints (eg, images data is only valid
* until next call to the converter).
*
* @see Java2DFrameConverter crashes JVM
* @see FrameConverter
*
* @author Sam West, Joel Wong, Sep 2016.
*
*/
public class Java2DFrameUtils {
private static OpenCVFrameConverter.ToIplImage iplConv = new OpenCVFrameConverter.ToIplImage();
private static OpenCVFrameConverter.ToMat matConv = new OpenCVFrameConverter.ToMat();
private static Java2DFrameConverter biConv = new Java2DFrameConverter();
/**
* Clones (deep copies the data) of a {@link BufferedImage}. Necessary when
* converting to BufferedImages from JavaCV types to avoid re-using the same
* memory locations.
*
* @param source
* @return
*/
public static BufferedImage deepCopy(BufferedImage source) {
return Java2DFrameConverter.cloneBufferedImage(source);
}
public synchronized static BufferedImage toBufferedImage(IplImage src) {
try (Frame f = iplConv.convert(src).clone()) {
return deepCopy(biConv.getBufferedImage(f));
}
}
public synchronized static BufferedImage toBufferedImage(Mat src) {
try (Frame f = matConv.convert(src).clone()) {
return deepCopy(biConv.getBufferedImage(f));
}
}
public synchronized static BufferedImage toBufferedImage(Frame src) {
try (Frame f = src.clone()) {
return deepCopy(biConv.getBufferedImage(f));
}
}
public synchronized static IplImage toIplImage(Mat src){
return iplConv.convertToIplImage(matConv.convert(src)).clone();
}
public synchronized static IplImage toIplImage(Frame src){
return iplConv.convertToIplImage(src).clone();
}
public synchronized static IplImage toIplImage(BufferedImage src){
return iplConv.convertToIplImage(biConv.convert(src)).clone();
}
public synchronized static Mat toMat(IplImage src){
return matConv.convertToMat(iplConv.convert(src).clone());
}
public synchronized static Mat toMat(Frame src){
return matConv.convertToMat(src).clone();
}
public synchronized static Mat toMat(BufferedImage src){
return matConv.convertToMat(biConv.convert(src)).clone();
}
public synchronized static Frame toFrame(IplImage src){
return iplConv.convert(src).clone();
}
public synchronized static Frame toFrame(Mat src){
return matConv.convert(src).clone();
}
public synchronized static Frame toFrame(BufferedImage src){
return biConv.convert(src).clone();
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/JavaCV.java
================================================
/*
* Copyright (C) 2009-2016 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.nio.ByteBuffer;
import java.nio.DoubleBuffer;
import java.nio.FloatBuffer;
import java.nio.IntBuffer;
import java.nio.ShortBuffer;
import java.util.Arrays;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import org.bytedeco.opencv.global.opencv_core;
import org.bytedeco.opencv.global.opencv_imgproc;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
*
* @author Samuel Audet
*/
public class JavaCV {
public static final double
SQRT2 = 1.41421356237309504880,
FLT_EPSILON = 1.19209290e-7F,
DBL_EPSILON = 2.2204460492503131e-16;
/** returns the distance^2 between the line (x1, y1) (x2, y2) and the point (x3, y3) */
public static double distanceToLine(double x1, double y1, double x2, double y2, double x3, double y3) {
double dx = x2 - x1;
double dy = y2 - y1;
double d2 = dx*dx + dy*dy;
double u = ((x3 - x1)*dx + (y3 - y1)*dy) / d2;
double x = x1 + u*dx;
double y = y1 + u*dy;
dx = x - x3;
dy = y - y3;
return dx*dx + dy*dy;
}
/**
* returns the largest rectangle of given aspect ratio and angle,
* bounded by the contour and sharing the same centroid
*/
private static ThreadLocal moments = CvMoments.createThreadLocal();
public static CvBox2D boundedRect(CvMat contour, CvBox2D box) {
int contourLength = contour.length();
CvMoments m = moments.get();
cvMoments(contour, m, 0);
double inv_m00 = 1 / m.m00();
double centerX = m.m10() * inv_m00;
double centerY = m.m01() * inv_m00;
float[] pts = new float[8];
CvPoint2D32f center = box.center();
CvSize2D32f size = box.size();
center.put(centerX, centerY);
cvBoxPoints(box, pts);
float scale = Float.POSITIVE_INFINITY;
for (int i = 0; i < 4; i++) {
double x1 = centerX, y1 = centerY,
x2 = pts[2*i], y2 = pts[2*i + 1];
for (int j = 0; j < contourLength; j++) {
int k = (j + 1) % contourLength;
double x3 = contour.get(2*j), y3 = contour.get(2*j + 1),
x4 = contour.get(2*k), y4 = contour.get(2*k + 1);
double d = (y4 - y3) * (x2 - x1) - (x4 - x3) * (y2 - y1);
double ua = ((x4 - x3) * (y1 - y3) - (y4 - y3) * (x1 - x3))/d,
ub = ((x2 - x1) * (y1 - y3) - (y2 - y1) * (x1 - x3))/d;
if (ub >= 0 && ub <= 1 && ua >= 0 && ua < scale) {
scale = (float)ua;
}
}
}
size.width(scale*size.width()).height(scale*size.height());
return box;
}
/**
* Similar to cvBoundingRect(), but can also pad the output with some extra
* pixels, useful to use as ROI for operations with interpolation. Further
* aligns the region to specified boundaries, for easier vectorization and
* subsampling, and also uses on input the rect argument as a maximum boundary.
*/
public static CvRect boundingRect(double[] contour, CvRect rect,
int padX, int padY, int alignX, int alignY) {
double minX = contour[0];
double minY = contour[1];
double maxX = contour[0];
double maxY = contour[1];
for (int i = 1; i < contour.length/2; i++) {
double x = contour[2*i ];
double y = contour[2*i+1];
minX = Math.min(minX, x);
minY = Math.min(minY, y);
maxX = Math.max(maxX, x);
maxY = Math.max(maxY, y);
}
int x = (int)Math.floor(Math.max(rect.x(), minX-padX)/alignX)*alignX;
int y = (int)Math.floor(Math.max(rect.y(), minY-padY)/alignY)*alignY;
int width = (int)Math.ceil(Math.min(rect.width(), maxX+padX)/alignX)*alignX - x;
int height = (int)Math.ceil(Math.min(rect.height(), maxY+padY)/alignY)*alignY - y;
return rect.x(x).y(y).width(Math.max(0, width)).height(Math.max(0, height));
}
private static ThreadLocal
A8x8 = CvMat.createThreadLocal(8, 8),
b8x1 = CvMat.createThreadLocal(8, 1),
x8x1 = CvMat.createThreadLocal(8, 1);
/**
* this is basically cvGetPerspectiveTransform() using CV_LU instead of
* CV_SVD, because the latter gives inaccurate results...
* Consider using {@link opencv_imgproc#getPerspectiveTransform} instead.
*/
public static CvMat getPerspectiveTransform(double[] src, double[] dst, CvMat map_matrix) {
// creating and releasing matrices via NIO here in this function
// can easily become a bottleneck, so we use ThreadLocal references
CvMat A = A8x8.get();
CvMat b = b8x1.get();
CvMat x = x8x1.get();
for(int i = 0; i < 4; ++i ) {
A.put(i*8+0, src[i*2]); A.put((i+4)*8+3, src[i*2]);
A.put(i*8+1, src[i*2+1]); A.put((i+4)*8+4, src[i*2+1]);
A.put(i*8+2, 1); A.put((i+4)*8+5, 1);
A.put(i*8+3, 0); A.put(i*8+4, 0); A.put(i*8+5, 0);
A.put((i+4)*8+0, 0); A.put((i+4)*8+1, 0); A.put((i+4)*8+2, 0);
A.put(i*8+6, -src[i*2] *dst[i*2]);
A.put(i*8+7, -src[i*2+1]*dst[i*2]);
A.put((i+4)*8+6, -src[i*2] *dst[i*2+1]);
A.put((i+4)*8+7, -src[i*2+1]*dst[i*2+1]);
b.put(i, dst[i*2]);
b.put(i+4, dst[i*2+1]);
}
cvSolve(A, b, x, CV_LU);
map_matrix.put(x.get());
map_matrix.put(8, 1);
return map_matrix;
}
/** Consider using {@link opencv_core#perspectiveTransform} instead. */
public static void perspectiveTransform(double[] src, double[] dst, CvMat map_matrix) {
double[] mat = map_matrix.get();
for (int j = 0; j < src.length; j += 2) {
double x = src[j], y = src[j + 1];
double w = x*mat[6] + y*mat[7] + mat[8];
if (Math.abs(w) > FLT_EPSILON) {
w = 1.0/w;
dst[j] = (x*mat[0] + y*mat[1] + mat[2])*w;
dst[j+1] = (x*mat[3] + y*mat[4] + mat[5])*w;
} else {
dst[j] = dst[j+1] = 0;
}
}
}
private static ThreadLocal
A3x3 = CvMat.createThreadLocal(3, 3), b3x1 = CvMat.createThreadLocal(3, 1);
public static CvMat getPlaneParameters(double[] src, double[] dst,
CvMat invSrcK, CvMat dstK, CvMat R, CvMat t, CvMat n) {
CvMat A = A3x3.get(), b = b3x1.get();
double[] x = new double[6], y = new double[6];
perspectiveTransform(src, x, invSrcK);
cvInvert(dstK, A);
perspectiveTransform(dst, y, A);
for (int i = 0; i < 3; i++) {
A.put(i, 0, (t.get(2)*y[i*2] - t.get(0))*x[i*2 ]);
A.put(i, 1, (t.get(2)*y[i*2] - t.get(0))*x[i*2+1]);
A.put(i, 2, t.get(2)*y[i*2] - t.get(0));
b.put(i, (R.get(2, 0)*x[i*2] + R.get(2, 1)*x[i*2+1] + R.get(2, 2))*y[i*2] -
(R.get(0, 0)*x[i*2] + R.get(0, 1)*x[i*2+1] + R.get(0, 2)));
}
cvSolve(A, b, n, CV_LU);
return n;
}
private static ThreadLocal
n3x1 = CvMat.createThreadLocal(3, 1);
public static CvMat getPerspectiveTransform(double[] src, double[] dst,
CvMat invSrcK, CvMat dstK, CvMat R, CvMat t, CvMat H) {
CvMat n = n3x1.get();
getPlaneParameters(src, dst, invSrcK, dstK, R, t, n);
// H = R - t*n^T
cvGEMM(t, n, -1, R, 1, H, CV_GEMM_B_T);
// H = dstK * H * srcK^-1
cvMatMul(dstK, H, H);
cvMatMul(H, invSrcK, H);
return H;
}
private static ThreadLocal
H3x3 = CvMat.createThreadLocal(3, 3);
public static void perspectiveTransform(double[] src, double[] dst,
CvMat invSrcK, CvMat dstK, CvMat R, CvMat t, CvMat n, boolean invert) {
CvMat H = H3x3.get();
// H = R - t*n^T
cvGEMM(t, n, -1, R, 1, H, CV_GEMM_B_T);
// H = dstK * H * srcK^-1
cvMatMul(dstK, H, H);
cvMatMul(H, invSrcK, H);
if (invert) {
cvInvert(H, H);
}
perspectiveTransform(src, dst, H);
}
private static ThreadLocal
M3x2 = CvMat.createThreadLocal(3, 2), S2x2 = CvMat.createThreadLocal(2, 2),
U3x2 = CvMat.createThreadLocal(3, 2), V2x2 = CvMat.createThreadLocal(2, 2);
/**
* Algorithms for Plane-Based Pose Estimation, Peter Sturm
* This assumes plane parameters n == z axis.
*/
public static void HtoRt(CvMat H, CvMat R, CvMat t) {
CvMat M = M3x2.get(), S = S2x2.get(),
U = U3x2.get(), V = V2x2.get();
M.put(H.get(0), H.get(1),
H.get(3), H.get(4),
H.get(6), H.get(7));
cvSVD(M, S, U, V, CV_SVD_V_T);
double lambda = S.get(3);
t.put(H.get(2)/lambda, H.get(5)/lambda, H.get(8)/lambda);
cvMatMul(U, V, M);
R.put(M.get(0), M.get(1), M.get(2)*M.get(5) - M.get(3)*M.get(4),
M.get(2), M.get(3), M.get(1)*M.get(4) - M.get(0)*M.get(5),
M.get(4), M.get(5), M.get(0)*M.get(3) - M.get(1)*M.get(2));
}
private static ThreadLocal
R13x3 = CvMat.createThreadLocal(3, 3), R23x3 = CvMat.createThreadLocal(3, 3),
t13x1 = CvMat.createThreadLocal(3, 1), t23x1 = CvMat.createThreadLocal(3, 1),
n13x1 = CvMat.createThreadLocal(3, 1), n23x1 = CvMat.createThreadLocal(3, 1),
H13x3 = CvMat.createThreadLocal(3, 3), H23x3 = CvMat.createThreadLocal(3, 3);
public static double HnToRt(CvMat H, CvMat n, CvMat R, CvMat t) {
CvMat S = S3x3.get(), U = U3x3.get(), V = V3x3.get();
cvSVD(H, S, U, V, 0);
CvMat R1 = R13x3.get(), R2 = R23x3.get(),
t1 = t13x1.get(), t2 = t23x1.get(),
n1 = n13x1.get(), n2 = n23x1.get(),
H1 = H13x3.get(), H2 = H23x3.get();
double zeta = homogToRt(S, U, V, R1, t1, n1, R2, t2, n2);
// H = (R^-1 * H)/s2
cvGEMM(R1, H, 1/S.get(4), null, 0, H1, CV_GEMM_A_T);
cvGEMM(R2, H, 1/S.get(4), null, 0, H2, CV_GEMM_A_T);
// H = H - I
H1.put(0, H1.get(0)-1); H1.put(4, H1.get(4)-1); H1.put(8, H1.get(8)-1);
H2.put(0, H2.get(0)-1); H2.put(4, H2.get(4)-1); H2.put(8, H2.get(8)-1);
// Now H should ~= -tn^T, so extract "average" t
double d = Math.abs (n.get(0)) + Math.abs (n.get(1)) + Math.abs (n.get(2));
double s[] = { -Math.signum(n.get(0)), -Math.signum(n.get(1)), -Math.signum(n.get(2)) };
t1.put(0.0, 0.0, 0.0);
t2.put(0.0, 0.0, 0.0);
for (int i = 0; i < 3; i++) {
t1.put(0, t1.get(0) + s[i]*H1.get(i) /d);
t1.put(1, t1.get(1) + s[i]*H1.get(i+3)/d);
t1.put(2, t1.get(2) + s[i]*H1.get(i+6)/d);
t2.put(0, t2.get(0) + s[i]*H2.get(i) /d);
t2.put(1, t2.get(1) + s[i]*H2.get(i+3)/d);
t2.put(2, t2.get(2) + s[i]*H2.get(i+6)/d);
}
// H = H + tn^T
cvGEMM(t1, n, 1, H1, 1, H1, CV_GEMM_B_T);
cvGEMM(t2, n, 1, H2, 1, H2, CV_GEMM_B_T);
// take what's left as the error of the model,
// this either indicates inaccurate camera matrix K or normal vector n
double err1 = cvNorm(H1);
double err2 = cvNorm(H2);
double err;
if (err1 < err2) {
if (R != null) {
R.put(R1);
}
if (t != null) {
t.put(t1);
}
err = err1;
} else {
if (R != null) {
R.put(R2);
}
if (t != null) {
t.put(t2);
}
err = err2;
}
return err;
}
private static ThreadLocal
S3x3 = CvMat.createThreadLocal(3, 3),
U3x3 = CvMat.createThreadLocal(3, 3),
V3x3 = CvMat.createThreadLocal(3, 3);
/**
* Ported to Java/OpenCV from
* Bill Triggs. Autocalibration from Planar Scenes. In 5th European Conference
* on Computer Vision (ECCV ’98), volume I, pages 89–105. Springer-Verlag, 1998.
*/
public static double homogToRt(CvMat H,
CvMat R1, CvMat t1, CvMat n1,
CvMat R2, CvMat t2, CvMat n2) {
CvMat S = S3x3.get(), U = U3x3.get(), V = V3x3.get();
cvSVD(H, S, U, V, 0);
double zeta = homogToRt(S, U, V, R1, t1, n1, R2, t2, n2);
return zeta;
}
public static double homogToRt(CvMat S, CvMat U, CvMat V,
CvMat R1, CvMat t1, CvMat n1,
CvMat R2, CvMat t2, CvMat n2) {
double s1 = S.get(0)/S.get(4);
double s3 = S.get(8)/S.get(4);
double zeta = s1-s3;
double a1 = Math.sqrt(1 - s3*s3);
double b1 = Math.sqrt(s1*s1 - 1);
double[] ab = unitize(a1, b1);
double[] cd = unitize(1+s1*s3, a1*b1);
double[] ef = unitize(-ab[1]/s1, -ab[0]/s3);
R1.put(cd[0],0,cd[1], 0,1,0, -cd[1],0,cd[0]);
cvGEMM(U , R1, 1, null, 0, R1, 0);
cvGEMM(R1, V, 1, null, 0, R1, CV_GEMM_B_T);
R2.put(cd[0],0,-cd[1], 0,1,0, cd[1],0,cd[0]);
cvGEMM(U , R2, 1, null, 0, R2, 0);
cvGEMM(R2, V, 1, null, 0, R2, CV_GEMM_B_T);
double[] v1 = { V.get(0), V.get(3), V.get(6) };
double[] v3 = { V.get(2), V.get(5), V.get(8) };
double sign1 = 1, sign2 = 1;
for (int i = 2; i >= 0; i--) {
n1.put(i, sign1*(ab[1]*v1[i] - ab[0]*v3[i]));
n2.put(i, sign2*(ab[1]*v1[i] + ab[0]*v3[i]));
t1.put(i, sign1*(ef[0]*v1[i] + ef[1]*v3[i]));
t2.put(i, sign2*(ef[0]*v1[i] - ef[1]*v3[i]));
if (i == 2) {
if (n1.get(2) < 0) {
n1.put(2, -n1.get(2));
t1.put(2, -t1.get(2));
sign1 = -1;
}
if (n2.get(2) < 0) {
n2.put(2, -n2.get(2));
t2.put(2, -t2.get(2));
sign2 = -1;
}
}
}
return zeta;
}
public static double[] unitize(double a, double b) {
double norm = Math.sqrt(a*a + b*b);
if (norm > FLT_EPSILON) {
a = a / norm;
b = b / norm;
}
return new double[] { a, b };
}
/** more sophisticated than cvAdaptiveThreshold() */
public static void adaptiveThreshold(IplImage srcImage, final IplImage sumImage,
final IplImage sqSumImage, final IplImage dstImage, final boolean invert,
final int windowMax, final int windowMin, final double varMultiplier, final double k) {
final int w = srcImage.width();
final int h = srcImage.height();
final int srcChannels = srcImage.nChannels();
final int srcDepth = srcImage.depth();
final int dstDepth = dstImage.depth();
if (srcChannels > 1 && dstDepth == IPL_DEPTH_8U) {
cvCvtColor(srcImage, dstImage, srcChannels == 4 ? CV_RGBA2GRAY : CV_BGR2GRAY);
srcImage = dstImage;
}
final ByteBuffer srcBuf = srcImage.getByteBuffer();
final ByteBuffer dstBuf = dstImage.getByteBuffer();
final DoubleBuffer sumBuf = sumImage.getDoubleBuffer();
final DoubleBuffer sqSumBuf = sqSumImage.getDoubleBuffer();
final int srcStep = srcImage.widthStep();
final int dstStep = dstImage.widthStep();
final int sumStep = sumImage.widthStep();
final int sqSumStep = sqSumImage.widthStep();
// compute integral images
cvIntegral(srcImage, sumImage, sqSumImage, null);
// try to detect a reasonable maximum and minimum intensity
// for thresholds instead of simply 0 and 255...
double totalMean = sumBuf.get((h-1)*sumStep/8 + (w-1)) -
sumBuf.get((h-1)*sumStep/8) -
sumBuf.get(w-1) + sumBuf.get(0);
totalMean /= w*h;
double totalSqMean = sqSumBuf.get((h-1)*sqSumStep/8 + (w-1)) -
sqSumBuf.get((h-1)*sqSumStep/8) -
sqSumBuf.get(w-1) + sqSumBuf.get(0);
totalSqMean /= w*h;
double totalVar = totalSqMean - totalMean*totalMean;
//double totalDev = Math.sqrt(totalVar);
//System.out.println(totalDev);
final double targetVar = totalVar*varMultiplier;
//for (int y = 0; y < h; y++) {
Parallel.loop(0, h, new Parallel.Looper() {
public void loop(int from, int to, int looperID) {
for (int y = from; y < to; y++) {
for (int x = 0; x < w; x++) {
double var = 0, mean = 0, sqMean = 0;
int upperLimit = windowMax;
int lowerLimit = windowMin;
int window = upperLimit; // start with windowMax
while (upperLimit - lowerLimit > 2) {
int x1 = Math.max(x-window/2, 0);
int x2 = Math.min(x+window/2+1, w);
int y1 = Math.max(y-window/2, 0);
int y2 = Math.min(y+window/2+1, h);
mean = sumBuf.get(y2*sumStep/8 + x2) -
sumBuf.get(y2*sumStep/8 + x1) -
sumBuf.get(y1*sumStep/8 + x2) +
sumBuf.get(y1*sumStep/8 + x1);
mean /= window*window;
sqMean = sqSumBuf.get(y2*sqSumStep/8 + x2) -
sqSumBuf.get(y2*sqSumStep/8 + x1) -
sqSumBuf.get(y1*sqSumStep/8 + x2) +
sqSumBuf.get(y1*sqSumStep/8 + x1);
sqMean /= window*window;
var = sqMean - mean*mean;
// if we're at maximum window size, but variance is
// too low anyway, let's break out immediately
if (window == upperLimit && var < targetVar) {
break;
}
// otherwise, start binary search
if (var > targetVar) {
upperLimit = window;
} else {
lowerLimit = window;
}
window = lowerLimit + (upperLimit-lowerLimit)/2;
window = (window/2)*2 + 1;
}
double value = 0;
if (srcDepth == IPL_DEPTH_8U) {
value = srcBuf.get(y*srcStep + x) & 0xFF;
} else if (srcDepth == IPL_DEPTH_32F) {
value = srcBuf.getFloat(y*srcStep + 4*x);
} else if (srcDepth == IPL_DEPTH_64F) {
value = srcBuf.getDouble(y*srcStep + 8*x);
} else {
// cvIntegral() does not support other image types,
// so we should not be able to get here.
assert false;
}
if (invert) {
//double threshold = 255 - (255 - mean) * (1 + 0.1*(Math.sqrt(var)/128 - 1));
double threshold = 255 - (255 - mean) * k;
dstBuf.put(y*dstStep + x, (value < threshold ? (byte)0xFF : (byte)0x00));
} else {
//double threshold = mean * (1 + k*(Math.sqrt(var)/128 - 1));
double threshold = mean * k;
dstBuf.put(y*dstStep + x, (value > threshold ? (byte)0xFF : (byte)0x00));
}
}
}
}});
}
/** similar to hysteresis thresholding as used by the Canny edge detector */
public static void hysteresisThreshold(IplImage srcImage, IplImage dstImage,
double highThresh, double lowThresh, double maxValue) {
int highThreshold = (int)Math.round(highThresh);
int lowThreshold = (int)Math.round(lowThresh);
byte lowValue = 0;
byte medValue = (byte)Math.round(maxValue/2);
byte highValue = (byte)Math.round(maxValue);
int height = srcImage.height();
int width = srcImage.width();
ByteBuffer srcData = srcImage.getByteBuffer();
ByteBuffer dstData = dstImage.getByteBuffer();
int srcStep = srcImage.widthStep();
int dstStep = dstImage.widthStep();
int srcIndex = 0;
int dstIndex = 0;
//
// first pass forward
//
// first line
int i = 0;
int in = srcData.get(srcIndex+i)&0xFF;
if (in >= highThreshold) {
dstData.put(dstIndex+i, highValue);
} else if (in < lowThreshold) {
dstData.put(dstIndex+i, lowValue);
} else {
dstData.put(dstIndex+i, medValue);
}
for (i = 1; i < width-1; i++) {
in = srcData.get(srcIndex+i)&0xFF;
if (in >= highThreshold) {
dstData.put(dstIndex+i, highValue);
} else if (in < lowThreshold) {
dstData.put(dstIndex+i, lowValue);
} else {
byte prev = dstData.get(dstIndex+i-1);
if (prev == highValue) {
dstData.put(dstIndex+i, highValue);
} else {
dstData.put(dstIndex+i, medValue);
}
}
}
i = width-1;
in = srcData.get(srcIndex+i)&0xFF;
if (in >= highThreshold) {
dstData.put(dstIndex+i, highValue);
} else if (in < lowThreshold) {
dstData.put(dstIndex+i, lowValue);
} else {
byte prev = dstData.get(dstIndex+i-1);
if (prev == highValue) {
dstData.put(dstIndex+i, highValue);
} else {
dstData.put(dstIndex+i, medValue);
}
}
height--;
// other lines
while (height-- > 0) {
srcIndex += srcStep;
dstIndex += dstStep;
// first column
i = 0;
in = srcData.get(srcIndex+i)&0xFF;
if (in >= highThreshold) {
dstData.put(dstIndex+i, highValue);
} else if (in < lowThreshold) {
dstData.put(dstIndex+i, lowValue);
} else {
byte prev1 = dstData.get(dstIndex+i-dstStep);
byte prev2 = dstData.get(dstIndex+i-dstStep+1);
if (prev1 == highValue || prev2 == highValue) {
dstData.put(dstIndex+i, highValue);
} else {
dstData.put(dstIndex+i, medValue);
}
}
// other columns
for (i = 1; i < width-1; i++) {
in = srcData.get(srcIndex+i)&0xFF;
if (in >= highThreshold) {
dstData.put(dstIndex+i, highValue);
} else if (in < lowThreshold) {
dstData.put(dstIndex+i, lowValue);
} else {
byte prev1 = dstData.get(dstIndex+i-1);
byte prev2 = dstData.get(dstIndex+i-dstStep-1);
byte prev3 = dstData.get(dstIndex+i-dstStep);
byte prev4 = dstData.get(dstIndex+i-dstStep+1);
if (prev1 == highValue || prev2 == highValue ||
prev3 == highValue || prev4 == highValue) {
dstData.put(dstIndex+i, highValue);
} else {
dstData.put(dstIndex+i, medValue);
}
}
}
// last column
i = width-1;
in = srcData.get(srcIndex+i)&0xFF;
if (in >= highThreshold) {
dstData.put(dstIndex+i, highValue);
} else if (in < lowThreshold) {
dstData.put(dstIndex+i, lowValue);
} else {
byte prev1 = dstData.get(dstIndex+i-1);
byte prev2 = dstData.get(dstIndex+i-dstStep-1);
byte prev3 = dstData.get(dstIndex+i-dstStep);
if (prev1 == highValue || prev2 == highValue ||
prev3 == highValue) {
dstData.put(dstIndex+i, highValue);
} else {
dstData.put(dstIndex+i, medValue);
}
}
}
height = srcImage.height();
width = srcImage.width();
dstIndex = (height-1)*dstStep;
//
// second pass backward
//
// first (actually last) line
i = width-1;
if (dstData.get(dstIndex+i) == medValue) {
dstData.put(dstIndex+i, lowValue);
}
for (i = width-2; i > 0 ; i--) {
if (dstData.get(dstIndex+i) == medValue) {
if (dstData.get(dstIndex+i+1) == highValue) {
dstData.put(dstIndex+i, highValue);
} else {
dstData.put(dstIndex+i, lowValue);
}
}
}
i = 0;
if (dstData.get(dstIndex+i) == medValue) {
if (dstData.get(dstIndex+i+1) == highValue) {
dstData.put(dstIndex+i, highValue);
} else {
dstData.put(dstIndex+i, lowValue);
}
}
height--;
// other lines
while (height-- > 0) {
dstIndex -= dstStep;
// first column
i = width-1;
if (dstData.get(dstIndex+i) == medValue) {
if (dstData.get(dstIndex+i+dstStep) == highValue ||
dstData.get(dstIndex+i+dstStep-1) == highValue) {
dstData.put(dstIndex+i, highValue);
} else {
dstData.put(dstIndex+i, lowValue);
}
}
// other columns
for (i = width-2; i > 0 ; i--) {
if (dstData.get(dstIndex+i) == medValue) {
if (dstData.get(dstIndex+i+1) == highValue ||
dstData.get(dstIndex+i+dstStep+1) == highValue ||
dstData.get(dstIndex+i+dstStep) == highValue ||
dstData.get(dstIndex+i+dstStep-1) == highValue) {
dstData.put(dstIndex+i, highValue);
} else {
dstData.put(dstIndex+i, lowValue);
}
}
}
// last column
i = 0;
if (dstData.get(dstIndex+i) == medValue) {
if (dstData.get(dstIndex+i+1) == highValue ||
dstData.get(dstIndex+i+dstStep+1) == highValue ||
dstData.get(dstIndex+i+dstStep) == highValue) {
dstData.put(dstIndex+i, highValue);
} else {
dstData.put(dstIndex+i, lowValue);
}
}
}
}
/** Clamps image intensities between min and max. */
public static void clamp(IplImage src, IplImage dst, double min, double max) {
switch (src.depth()) {
case IPL_DEPTH_8U: {
ByteBuffer sb = src.getByteBuffer();
ByteBuffer db = dst.getByteBuffer();
for (int i = 0; i < sb.capacity(); i++) {
db.put(i, (byte)Math.max(Math.min(sb.get(i) & 0xFF,max),min));
}
break;
}
case IPL_DEPTH_16U: {
ShortBuffer sb = src.getShortBuffer();
ShortBuffer db = dst.getShortBuffer();
for (int i = 0; i < sb.capacity(); i++) {
db.put(i, (short)Math.max(Math.min(sb.get(i) & 0xFFFF,max),min));
}
break;
}
case IPL_DEPTH_32F: {
FloatBuffer sb = src.getFloatBuffer();
FloatBuffer db = dst.getFloatBuffer();
for (int i = 0; i < sb.capacity(); i++) {
db.put(i, (float)Math.max(Math.min(sb.get(i),max),min));
}
break;
}
case IPL_DEPTH_8S: {
ByteBuffer sb = src.getByteBuffer();
ByteBuffer db = dst.getByteBuffer();
for (int i = 0; i < sb.capacity(); i++) {
db.put(i, (byte)Math.max(Math.min(sb.get(i),max),min));
}
break;
}
case IPL_DEPTH_16S: {
ShortBuffer sb = src.getShortBuffer();
ShortBuffer db = dst.getShortBuffer();
for (int i = 0; i < sb.capacity(); i++) {
db.put(i, (short)Math.max(Math.min(sb.get(i),max),min));
}
break;
}
case IPL_DEPTH_32S: {
IntBuffer sb = src.getIntBuffer();
IntBuffer db = dst.getIntBuffer();
for (int i = 0; i < sb.capacity(); i++) {
db.put(i, (int)Math.max(Math.min(sb.get(i),max),min));
}
break;
}
case IPL_DEPTH_64F: {
DoubleBuffer sb = src.getDoubleBuffer();
DoubleBuffer db = dst.getDoubleBuffer();
for (int i = 0; i < sb.capacity(); i++) {
db.put(i, Math.max(Math.min(sb.get(i),max),min));
}
break;
}
default: assert(false);
}
}
/** vector norm 2 */
public static double norm(double[] v) {
return norm(v, 2.0);
}
/** vector norm p */
public static double norm(double[] v, double p) {
double norm = 0;
if (p == 1.0) {
for (double e : v) {
norm += Math.abs(e);
}
} else if (p == 2.0) {
for (double e : v) {
norm += e*e;
}
norm = Math.sqrt(norm);
} else if (p == Double.POSITIVE_INFINITY) {
for (double e : v) {
e = Math.abs(e);
if (e > norm) {
norm = e;
}
}
} else if (p == Double.NEGATIVE_INFINITY) {
norm = Double.MAX_VALUE;
for (double e : v) {
e = Math.abs(e);
if (e < norm) {
norm = e;
}
}
} else {
for (double e : v) {
norm += Math.pow(Math.abs(e), p);
}
norm = Math.pow(norm, 1/p);
}
return norm;
}
/** induced norm 2 */
public static double norm(CvMat A) {
return norm(A, 2.0);
}
/** induced norm p */
public static double norm(CvMat A, double p) {
return norm(A, p, null);
}
/** induced norm p */
public static double norm(CvMat A, double p, CvMat W) {
double norm = -1;
if (p == 1.0) {
int cols = A.cols(), rows = A.rows();
for (int j = 0; j < cols; j++) {
double n = 0;
for (int i = 0; i < rows; i++) {
n += Math.abs(A.get(i, j));
}
norm = Math.max(n, norm);
}
} else if (p == 2.0) {
int size = Math.min(A.rows(), A.cols());
if (W == null || W.rows() != size || W.cols() != 1) {
W = CvMat.create(size, 1);
}
cvSVD(A, W, null, null, 0);
norm = W.get(0); // largest singular value
} else if (p == Double.POSITIVE_INFINITY) {
int rows = A.rows(), cols = A.cols();
for (int i = 0; i < rows; i++) {
double n = 0;
for (int j = 0; j < cols; j++) {
n += Math.abs(A.get(i, j));
}
norm = Math.max(n, norm);
}
} else {
assert(false);
}
return norm;
}
public static double cond(CvMat A) {
return cond(A, 2.0);
}
public static double cond(CvMat A, double p) {
return cond(A, p, null);
}
public static double cond(CvMat A, double p, CvMat W) {
double cond = -1;
if (p == 2.0) {
int size = Math.min(A.rows(), A.cols());
if (W == null || W.rows() != size || W.cols() != 1) {
W = CvMat.create(size, 1);
}
cvSVD(A, W, null, null, 0);
cond = W.get(0)/W.get(W.length()-1); // largest/smallest singular value
} else {
// should put something faster here if we're really serious
// about using something other than the 2-norm
int rows = A.rows(), cols = A.cols();
if (W == null || W.rows() != rows || W.cols() != cols) {
W = CvMat.create(rows, cols);
}
CvMat Ainv = W;
cvInvert(A, Ainv);
cond = norm(A, p)*norm(Ainv, p);
}
return cond;
}
public static double median(double[] doubles) {
double[] sorted = doubles.clone();
Arrays.sort(sorted);
if (doubles.length%2 == 0) {
return (sorted[doubles.length/2 - 1] + sorted[doubles.length/2])/2;
} else {
return sorted[doubles.length/2];
}
}
public static T median(T[] objects) {
T[] sorted = objects.clone();
Arrays.sort(sorted);
return sorted[sorted.length/2];
}
public static void fractalTriangleWave(double[] line, int i, int j, double a) {
fractalTriangleWave(line, i, j, a, -1);
}
public static void fractalTriangleWave(double[] line, int i, int j, double a, int roughness) {
int m = (j-i)/2+i;
if (i == j || i == m) {
return;
}
line[m] = (line[i]+line[j])/2 + a;
if (roughness > 0 && line.length > roughness*(j-i)) {
fractalTriangleWave(line, i, m, 0, roughness);
fractalTriangleWave(line, m, j, 0, roughness);
} else {
fractalTriangleWave(line, i, m, a/SQRT2, roughness);
fractalTriangleWave(line, m, j, -a/SQRT2, roughness);
}
}
public static void fractalTriangleWave(IplImage image, CvMat H) {
fractalTriangleWave(image, H, -1);
}
public static void fractalTriangleWave(IplImage image, CvMat H, int roughness) {
assert (image.depth() == IPL_DEPTH_32F);
double[] line = new double[image.width()];
fractalTriangleWave(line, 0, line.length/2, 1, roughness);
fractalTriangleWave(line, line.length/2, line.length-1, -1, roughness);
double[] minMax = { Double.MAX_VALUE, Double.MIN_VALUE };
int height = image.height();
int width = image.width();
int channels = image.nChannels();
int step = image.widthStep();
int start = 0;
if (image.roi() != null) {
height = image.roi().height();
width = image.roi().width();
start = image.roi().yOffset()*step/4 + image.roi().xOffset()*channels;
}
FloatBuffer fb = image.getFloatBuffer(start);
double[] h = H == null ? null : H.get();
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
for (int z = 0; z < channels; z++) {
double sum = 0.0;
if (h == null) {
sum += line[x];
} else {
double x2 = (h[0]*x + h[1]*y + h[2])/(h[6]*x + h[7]*y + h[8]);
while (x2 < 0) {
x2 += line.length;
}
int xi2 = (int)x2;
double xn = x2 - xi2;
sum += line[ xi2 %line.length]*(1-xn) +
line[(xi2+1)%line.length]*xn;
}
minMax[0] = Math.min(minMax[0], sum);
minMax[1] = Math.max(minMax[1], sum);
fb.put(y*step/4 + x*channels + z, (float)sum);
}
}
}
cvConvertScale(image, image, 1/(minMax[1]-minMax[0]),
-minMax[0]/(minMax[1]-minMax[0]));
}
public static void main(String[] args) {
String version = JavaCV.class.getPackage().getImplementationVersion();
if (version == null) {
version = "unknown";
}
System.out.println(
"JavaCV version " + version + "\n" +
"Copyright (C) 2009-2018 Samuel Audet \n" +
"Project site: https://github.com/bytedeco/javacv");
System.exit(0);
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/JavaCVCL.java
================================================
/*
* Copyright (C) 2011-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import com.jogamp.opencl.CLCommandQueue;
import com.jogamp.opencl.CLBuffer;
import com.jogamp.opencl.CLContext;
import com.jogamp.opencl.CLDevice;
import com.jogamp.opencl.CLEventList;
import com.jogamp.opencl.CLImage2d;
import com.jogamp.opencl.CLImageFormat;
import com.jogamp.opencl.CLImageFormat.ChannelOrder;
import com.jogamp.opencl.CLImageFormat.ChannelType;
import com.jogamp.opencl.CLKernel;
import com.jogamp.opencl.CLMemory;
import com.jogamp.opencl.CLObject;
import com.jogamp.opencl.CLPlatform;
import com.jogamp.opencl.CLProgram;
import com.jogamp.opencl.CLProgram.CompilerOptions;
import com.jogamp.opencl.gl.CLGLContext;
import com.jogamp.opencl.gl.CLGLImage2d;
import com.jogamp.opencl.gl.CLGLObject;
import com.jogamp.opengl.GL;
import com.jogamp.opengl.GL2;
import com.jogamp.opengl.GLCapabilities;
import com.jogamp.opengl.GLCapabilitiesImmutable;
import com.jogamp.opengl.GLContext;
import com.jogamp.opengl.GLDrawableFactory;
import com.jogamp.opengl.GLException;
import com.jogamp.opengl.GLProfile;
import com.jogamp.opengl.glu.GLU;
import java.io.InputStream;
import java.io.IOException;
import java.io.SequenceInputStream;
import java.nio.ByteBuffer;
import java.util.Vector;
import java.util.logging.Logger;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgcodecs.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
*
* @author Samuel Audet
*
* To make NVIDIA drivers happy, we may need to limit the maximum
* heap memory size of Java to 1 GB, by defining something like
* export _JAVA_OPTIONS=-Xmx1G
*
*/
public class JavaCVCL {
public JavaCVCL(CLContext context) {
this(context, context.getDevices()[0]);
}
public JavaCVCL(CLContext context, CLDevice device) {
// this.pbuffer = null;
this.context = context;
this.glu = context instanceof CLGLContext ? new GLU() : null;
this.commandQueue = device.createCommandQueue(/*Mode.PROFILING_MODE*/);
CLKernel[] kernels = buildKernels(fastCompilerOptions, "JavaCV.cl", "pyrDown", "remap", "remapBayer");
this.pyrDownKernel = kernels[0];
this.remapKernel = kernels[1];
this.remapBayerKernel = kernels[2];
}
public static GLCapabilities getDefaultGLCapabilities(GLProfile profile) {
GLCapabilities caps = new GLCapabilities(profile != null ? profile : GLProfile.getDefault());
// Without line below, there is an error on Windows.
caps.setDoubleBuffered(false);
return caps;
}
public JavaCVCL() {
this(false);
}
public JavaCVCL(boolean createPbuffer) {
this(createPbuffer ? getDefaultGLCapabilities(null) : null, null, null);
}
public JavaCVCL(GLContext shareWith) {
this(getDefaultGLCapabilities(shareWith == null ? null :
shareWith.getGLDrawable().getGLProfile()), shareWith, null);
}
public JavaCVCL(GLCapabilitiesImmutable caps, GLContext shareWith, CLDevice device) {
// GLPbuffer pbuffer = null;
// if (caps != null) {
// GLDrawableFactory factory = GLDrawableFactory.getFactory(caps.getGLProfile());
// if (factory.canCreateGLPbuffer(null, caps.getGLProfile())) {
// try {
// // makes a new buffer
// pbuffer = factory.createGLPbuffer(null, caps, null, 32, 32, shareWith);
// // required for drawing to the buffer
// pbuffer.createContext(shareWith).makeCurrent();
// } catch (GLException e) {
// logger.warning("Could not create PBuffer: " + e);
// }
// } else {
// logger.warning("OpenGL implementation does not support PBuffers.");
// }
// }
// this.pbuffer = pbuffer;
GLContext glContext = GLContext.getCurrent();
if (device == null && glContext != null) {
// woohoo! we have a GLContext
// find gl compatible device
CLDevice[] devices = CLPlatform.getDefault().listCLDevices();
for (CLDevice d : devices) {
if(d.isGLMemorySharingSupported()) {
device = d;
break;
}
}
// if(null==device) {
// throw new RuntimeException("couldn't find any CL/GL memory sharing devices ..");
// }
}
if (glContext != null && device != null) {
// create OpenCL context before creating any OpenGL objects
// you want to share with OpenCL (AMD driver requirement)
context = CLGLContext.create(glContext, device);
glu = GLU.createGLU();
} else if (device != null) {
context = CLContext.create(device);
glu = null;
} else {
// find a CL implementation
//CLPlatform platform = CLPlatform.getDefault(/*type(CPU)*/);
context = CLContext.create(/*platform.getMaxFlopsDevice()*/);
device = context.getDevices()[0];
glu = null;
}
// creade a command queue with benchmarking flag set
commandQueue = device.createCommandQueue(/*Mode.PROFILING_MODE*/);
CLKernel[] kernels = buildKernels(fastCompilerOptions, "JavaCV.cl", "pyrDown", "remap", "remapBayer");
this.pyrDownKernel = kernels[0];
this.remapKernel = kernels[1];
this.remapBayerKernel = kernels[2];
}
public void release() {
if (!context.isReleased()) {
context.release();
// if (pbuffer != null) {
// pbuffer.getContext().makeCurrent();
// pbuffer.getContext().release();
// pbuffer.getContext().destroy();
// pbuffer.destroy();
// }
}
}
@Override protected void finalize() throws Throwable {
super.finalize();
release();
}
public static final String fastCompilerOptions = // "-cl-nv-verbose " +
CompilerOptions.FAST_RELAXED_MATH + " " + CompilerOptions.ENABLE_MAD;
private static final Logger logger = Logger.getLogger(JavaCVCL.class.getName());
// private final GLPbuffer pbuffer;
private final CLContext context;
private final CLCommandQueue commandQueue;
private final GLU glu;
private final CLKernel pyrDownKernel, remapKernel, remapBayerKernel;
public CLContext getCLContext() {
return context;
}
public CLCommandQueue getCLCommandQueue() {
return commandQueue;
}
public CLGLContext getCLGLContext() {
return context instanceof CLGLContext ? (CLGLContext)context : null;
}
public GLContext getGLContext() {
return context instanceof CLGLContext ? ((CLGLContext)context).getGLContext() : null;
}
public GL getGL() {
GLContext glContext = getGLContext();
return glContext != null ? glContext.getGL() : null;
}
public GL2 getGL2() {
GL gl = getGL();
return gl != null ? gl.getGL2() : null;
}
public GLU getGLU() {
return glu;
}
public CLKernel buildKernel(String resourceNames, String kernelName) {
return buildKernels(fastCompilerOptions, Loader.getCallerClass(2), resourceNames, kernelName)[0];
}
public CLKernel buildKernel(String compilerOptions, String resourceNames, String kernelName) {
return buildKernels(compilerOptions, Loader.getCallerClass(2), resourceNames, kernelName)[0];
}
public CLKernel[] buildKernels(String compilerOptions, String resourceNames, String ... kernelNames) {
return buildKernels(compilerOptions, Loader.getCallerClass(2), resourceNames, kernelNames);
}
public CLKernel[] buildKernels(String compilerOptions, Class resourceClass, String resourceNames, String ... kernelNames) {
try {
//load and compile program for the chosen device
InputStream s;
String[] a = resourceNames.split(":");
if (a.length == 1) {
s = resourceClass.getResourceAsStream(a[0]);
} else {
Vector vs = new Vector(a.length);
for (String name : a) {
vs.addElement(resourceClass.getResourceAsStream(name));
}
s = new SequenceInputStream(vs.elements());
}
CLProgram program = context.createProgram(s);
//System.out.println("Building " + resourceNames + "...");
program.build(compilerOptions);
//System.out.println(program.getBuildLog());
assert program.isExecutable();
// create kernel and set function parameters
CLKernel[] kernels = new CLKernel[kernelNames.length];
for (int i = 0; i < kernelNames.length; i++) {
kernels[i] = program.createCLKernel(kernelNames[i]);
}
return kernels;
} catch(IOException ex) {
throw (Error)new LinkageError(ex.toString()).initCause(ex);
}
}
public CLImage2d createCLImageFrom(IplImage image, CLImage2d.Mem ... flags) {
int width = image.width();
int height = image.height();
int pitch = image.widthStep();
ByteBuffer buffer = image.getByteBuffer();
ChannelOrder order = null;
ChannelType type = null;
int size = 0;
switch (image.depth()) {
case IPL_DEPTH_8S: type = ChannelType.SNORM_INT8; size = 1; break;
case IPL_DEPTH_8U: type = ChannelType.UNORM_INT8; size = 1; break;
case IPL_DEPTH_16S: type = ChannelType.SNORM_INT16; size = 2; break;
case IPL_DEPTH_16U: type = ChannelType.UNORM_INT16; size = 2; break;
case IPL_DEPTH_32S: type = ChannelType.SIGNED_INT32; size = 4; break;
case IPL_DEPTH_32F: type = ChannelType.FLOAT; size = 4; break;
default: assert false;
}
switch (image.nChannels()) {
case 1: order = ChannelOrder.LUMINANCE; break;
case 2: order = ChannelOrder.RG; size *= 2; break;
case 3: order = ChannelOrder.RGB; size *= 3; break;
case 4: order = ChannelOrder.RGBA; size *= 4; break;
default: assert false;
}
// NVIDIA drivers do not like it when width != pitch/size
if (width != pitch/size) {
width = pitch/size;
}
CLImageFormat format = new CLImageFormat(order, type);
return context.createImage2d(buffer, width, height, /*pitch,*/ format, flags);
}
public CLGLImage2d createCLGLImageFrom(IplImage image, CLImage2d.Mem ... flags) {
GL2 gl = getGL2();
if (gl == null) {
return null;
}
int width = image.width();
int height = image.height();
int pitch = image.widthStep();
//ByteBuffer buffer = image.getByteBuffer();
int format = 0;
int size = 0;
switch (image.nChannels()) {
case 1:
switch (image.depth()) {
case IPL_DEPTH_8S: format = GL2.GL_LUMINANCE8_SNORM; size = 1; break;
case IPL_DEPTH_8U: format = GL2.GL_LUMINANCE8; size = 1; break;
case IPL_DEPTH_16S: format = GL2.GL_LUMINANCE16_SNORM; size = 2; break;
case IPL_DEPTH_16U: format = GL2.GL_LUMINANCE16; size = 2; break;
case IPL_DEPTH_32S: format = GL2.GL_LUMINANCE32I; size = 4; break;
case IPL_DEPTH_32F: format = GL2.GL_LUMINANCE32F; size = 4; break;
default: assert false;
}
break;
case 2:
switch (image.depth()) {
case IPL_DEPTH_8S: format = GL2.GL_RG8_SNORM; size = 2; break;
case IPL_DEPTH_8U: format = GL2.GL_RG8; size = 2; break;
case IPL_DEPTH_16S: format = GL2.GL_RG16_SNORM; size = 4; break;
case IPL_DEPTH_16U: format = GL2.GL_RG16; size = 4; break;
case IPL_DEPTH_32S: format = GL2.GL_RG32I; size = 8; break;
case IPL_DEPTH_32F: format = GL2.GL_RG32F; size = 8; break;
default: assert false;
}
break;
case 3:
switch (image.depth()) {
case IPL_DEPTH_8S: format = GL2.GL_RGB8_SNORM; size = 3; break;
case IPL_DEPTH_8U: format = GL2.GL_RGB8; size = 3; break;
case IPL_DEPTH_16S: format = GL2.GL_RGB16_SNORM; size = 6; break;
case IPL_DEPTH_16U: format = GL2.GL_RGB16; size = 6; break;
case IPL_DEPTH_32S: format = GL2.GL_RGB32I; size = 12; break;
case IPL_DEPTH_32F: format = GL2.GL_RGB32F; size = 12; break;
default: assert false;
}
break;
case 4:
switch (image.depth()) {
case IPL_DEPTH_8S: format = GL2.GL_RGBA8_SNORM; size = 4; break;
case IPL_DEPTH_8U: format = GL2.GL_RGBA8; size = 4; break;
case IPL_DEPTH_16S: format = GL2.GL_RGBA16_SNORM; size = 8; break;
case IPL_DEPTH_16U: format = GL2.GL_RGBA16; size = 8; break;
case IPL_DEPTH_32S: format = GL2.GL_RGBA32I; size = 16; break;
case IPL_DEPTH_32F: format = GL2.GL_RGBA32F; size = 16; break;
default: assert false;
}
break;
default: assert false;
}
// NVIDIA drivers do not like it when width != pitch/size
if (width != pitch/size) {
width = pitch/size;
}
int[] renderBuffer = new int[1];
gl.glGenRenderbuffers(1, renderBuffer, 0);
gl.glBindRenderbuffer(GL2.GL_RENDERBUFFER, renderBuffer[0]);
gl.glRenderbufferStorage(GL2.GL_RENDERBUFFER, format, width, height);
return getCLGLContext().createFromGLRenderbuffer(renderBuffer[0], flags);
}
public void releaseCLGLImage(CLGLImage2d image) {
image.release();
getGL2().glDeleteRenderbuffers(1, new int[] { image.getGLObjectID() }, 0);
}
@SuppressWarnings("unchecked")
public CLBuffer createPinnedBuffer(int size) {
// as per NVIDIA's OpenCL Best Practices Guide
CLBuffer pinnedBuffer = context.createBuffer(size, CLMemory.Mem.ALLOCATE_BUFFER);
ByteBuffer byteBuffer = commandQueue.putMapBuffer(pinnedBuffer, CLMemory.Map.READ_WRITE, true);
pinnedBuffer.use(byteBuffer);
return pinnedBuffer;
}
class PinnedIplImage extends IplImage {
PinnedIplImage(int width, int height, int depth, int channels) {
super(cvCreateImageHeader(new CvSize().width(width).height(height), depth, channels));
pinnedBuffer = createPinnedBuffer(imageSize());
imageData(new BytePointer(getByteBuffer()));
}
final CLBuffer pinnedBuffer;
public CLBuffer getCLBuffer() {
return pinnedBuffer;
}
@Override public ByteBuffer getByteBuffer() {
return (ByteBuffer)pinnedBuffer.getBuffer();
}
@Override public void release() {
commandQueue.putUnmapMemory(pinnedBuffer, getByteBuffer());
pinnedBuffer.release();
cvReleaseImageHeader(this);
}
}
public IplImage createPinnedIplImage(int width, int height, int depth, int channels) {
return new PinnedIplImage(width, height, depth, channels);
}
public IplImage createIplImageFrom(CLImage2d image) {
int width = image.width;
int height = image.height;
CLImageFormat format = image.getFormat();
ChannelOrder order = format.getImageChannelOrder();
ChannelType type = format.getImageChannelDataType();
int depth = 0, channels = 0;
switch (order) {
case R:
case A:
case INTENSITY:
case LUMINANCE:
channels = 1;
break;
case Rx:
case RG:
case RA:
channels = 2;
break;
case RGx:
case RGB:
channels = 3;
break;
case RGBx:
case RGBA:
case ARGB:
case BGRA:
channels = 4;
break;
default: assert false;
}
switch (type) {
case SIGNED_INT8:
case SNORM_INT8: depth = IPL_DEPTH_8S; break;
case UNSIGNED_INT8:
case UNORM_INT8: depth = IPL_DEPTH_8U; break;
case SIGNED_INT16:
case SNORM_INT16: depth = IPL_DEPTH_16S; break;
case UNSIGNED_INT16:
case UNORM_INT16: depth = IPL_DEPTH_16U; break;
case UNSIGNED_INT32:
case SIGNED_INT32: depth = IPL_DEPTH_32S; break;
case FLOAT: depth = IPL_DEPTH_32F; break;
case HALF_FLOAT:
case UNORM_SHORT_565:
case UNORM_SHORT_555:
case UNORM_INT_101010:
default: assert false;
}
return IplImage.create(width, height, depth, channels);
//return createPinnedIplImage(width, height, depth, channels);
}
@SuppressWarnings("unchecked")
public IplImage readImage(CLImage2d clImg, IplImage iplImage, boolean blocking) {
if (iplImage == null) {
iplImage = createIplImageFrom(clImg);
}
int x = 0, y = 0;
int width = clImg.width;
int height = clImg.height;
int pitch = iplImage.widthStep();
ByteBuffer buffer = iplImage.getByteBuffer();
IplROI roi = iplImage.roi();
if (roi != null) {
x = roi.xOffset();
y = roi.yOffset();
width = roi.width();
height = roi.height();
int pixelSize = iplImage.nChannels()*((iplImage.depth()&~IPL_DEPTH_SIGN)/8);
buffer = iplImage.getByteBuffer(y*pitch + x*pixelSize);
}
clImg.use(buffer);
commandQueue.putReadImage(clImg, pitch, x, y, width, height, blocking);
return iplImage;
}
@SuppressWarnings("unchecked")
public CLImage2d writeImage(CLImage2d clImg, IplImage iplImage, boolean blocking) {
if (clImg == null) {
clImg = createCLImageFrom(iplImage);
}
int x = 0, y = 0;
int width = iplImage.width();
int height = iplImage.height();
int pitch = iplImage.widthStep();
ByteBuffer buffer = iplImage.getByteBuffer();
IplROI roi = iplImage.roi();
if (roi != null) {
x = roi.xOffset();
y = roi.yOffset();
width = roi.width();
height = roi.height();
int pixelSize = iplImage.nChannels()*((iplImage.depth()&~IPL_DEPTH_SIGN)/8);
buffer = iplImage.getByteBuffer(y*pitch + x*pixelSize);
}
clImg.use(buffer);
commandQueue.putWriteImage(clImg, pitch, x, y, width, height, blocking);
return clImg;
}
public void acquireGLObject(CLObject object) {
if (object instanceof CLGLObject) {
commandQueue.putAcquireGLObject((CLGLObject)object);
}
}
public void releaseGLObject(CLObject object) {
if (object instanceof CLGLObject) {
commandQueue.putReleaseGLObject((CLGLObject)object);
}
}
public void readBuffer(CLBuffer buffer, boolean blocking) {
commandQueue.putReadBuffer(buffer, blocking);
}
public void writeBuffer(CLBuffer buffer, boolean blocking) {
commandQueue.putWriteBuffer(buffer, blocking);
}
public void executeKernel(CLKernel kernel,
long globalWorkOffsetX, long globalWorkSizeX, long localWorkSizeX) {
commandQueue.put1DRangeKernel(kernel,
globalWorkOffsetX, globalWorkSizeX, localWorkSizeX);
}
public void executeKernel(CLKernel kernel,
long globalWorkOffsetX, long globalWorkSizeX, long localWorkSizeX, CLEventList events) {
commandQueue.put1DRangeKernel(kernel,
globalWorkOffsetX, globalWorkSizeX, localWorkSizeX, events);
}
public void executeKernel(CLKernel kernel,
long globalWorkOffsetX, long globalWorkSizeX, long localWorkSizeX,
CLEventList condition, CLEventList events) {
commandQueue.put1DRangeKernel(kernel,
globalWorkOffsetX, globalWorkSizeX, localWorkSizeX, condition, events);
}
public void executeKernel(CLKernel kernel,
long globalWorkOffsetX, long globalWorkOffsetY,
long globalWorkSizeX, long globalWorkSizeY,
long localWorkSizeX, long localWorkSizeY) {
commandQueue.put2DRangeKernel(kernel,
globalWorkOffsetX, globalWorkOffsetY,
globalWorkSizeX, globalWorkSizeY,
localWorkSizeX, localWorkSizeY);
}
public void executeKernel(CLKernel kernel,
long globalWorkOffsetX, long globalWorkOffsetY,
long globalWorkSizeX, long globalWorkSizeY,
long localWorkSizeX, long localWorkSizeY, CLEventList events) {
commandQueue.put2DRangeKernel(kernel,
globalWorkOffsetX, globalWorkOffsetY,
globalWorkSizeX, globalWorkSizeY,
localWorkSizeX, localWorkSizeY, events);
}
public void executeKernel(CLKernel kernel,
long globalWorkOffsetX, long globalWorkOffsetY,
long globalWorkSizeX, long globalWorkSizeY,
long localWorkSizeX, long localWorkSizeY,
CLEventList condition, CLEventList events) {
commandQueue.put2DRangeKernel(kernel,
globalWorkOffsetX, globalWorkOffsetY,
globalWorkSizeX, globalWorkSizeY,
localWorkSizeX, localWorkSizeY, condition, events);
}
public void executeKernel(CLKernel kernel,
long globalWorkOffsetX, long globalWorkOffsetY, long globalWorkOffsetZ,
long globalWorkSizeX, long globalWorkSizeY, long globalWorkSizeZ,
long localWorkSizeX, long localWorkSizeY, long localWorkSizeZ) {
commandQueue.put3DRangeKernel(kernel,
globalWorkOffsetX, globalWorkOffsetY, globalWorkOffsetZ,
globalWorkSizeX, globalWorkSizeY, globalWorkSizeZ,
localWorkSizeX, localWorkSizeY, localWorkSizeZ);
}
public void executeKernel(CLKernel kernel,
long globalWorkOffsetX, long globalWorkOffsetY, long globalWorkOffsetZ,
long globalWorkSizeX, long globalWorkSizeY, long globalWorkSizeZ,
long localWorkSizeX, long localWorkSizeY, long localWorkSizeZ, CLEventList events) {
commandQueue.put3DRangeKernel(kernel,
globalWorkOffsetX, globalWorkOffsetY, globalWorkOffsetZ,
globalWorkSizeX, globalWorkSizeY, globalWorkSizeZ,
localWorkSizeX, localWorkSizeY, localWorkSizeZ, events);
}
public void executeKernel(CLKernel kernel,
long globalWorkOffsetX, long globalWorkOffsetY, long globalWorkOffsetZ,
long globalWorkSizeX, long globalWorkSizeY, long globalWorkSizeZ,
long localWorkSizeX, long localWorkSizeY, long localWorkSizeZ,
CLEventList condition, CLEventList events) {
commandQueue.put3DRangeKernel(kernel,
globalWorkOffsetX, globalWorkOffsetY, globalWorkOffsetZ,
globalWorkSizeX, globalWorkSizeY, globalWorkSizeZ,
localWorkSizeX, localWorkSizeY, localWorkSizeZ, condition, events);
}
public void finish() {
commandQueue.finish();
}
public void flush() {
commandQueue.flush();
}
public static int alignCeil(int x, int n) {
return (x + n-1)/n*n;
}
public static int alignFloor(int x, int n) {
return x/n*n;
}
public void pyrDown(CLImage2d srcImg, CLImage2d dstImg) {
CLEventList list = null;//new CLEventList(1);
pyrDownKernel.putArg(srcImg).putArg(dstImg).rewind();
executeKernel(pyrDownKernel, 0, 0, alignCeil(dstImg.width, 2), alignCeil(dstImg.height, 64), 2, 64, list); // execute program
// finish();
// CLEvent event = list.getEvent(0);
// System.out.println("pyrDown: " + (event.getProfilingInfo(CLEvent.ProfilingCommand.END) -
// event.getProfilingInfo(CLEvent.ProfilingCommand.START))/1000000.0);
}
public void remap(CLImage2d srcImg, CLImage2d dstImg, CLImage2d mapxImg, CLImage2d mapyImg) {
remap(srcImg, dstImg, mapxImg, mapyImg, -1L);
}
public void remap(CLImage2d srcImg, CLImage2d dstImg, CLImage2d mapxImg, CLImage2d mapyImg, long sensorPattern) {
CLEventList list = null;//new CLEventList(1);
CLKernel kernel;
if (sensorPattern != -1L) {
kernel = remapBayerKernel.putArg(srcImg).putArg(dstImg).putArg(mapxImg).putArg(mapyImg).putArg(sensorPattern).rewind();
} else {
kernel = remapKernel.putArg(srcImg).putArg(dstImg).putArg(mapxImg).putArg(mapyImg).rewind();
}
executeKernel(kernel, 0, 0, alignCeil(dstImg.width, 2), alignCeil(dstImg.height, 64), 2, 64, list); // execute program
// finish();
// CLEvent event = list.getEvent(0);
// System.out.println("remap: " + (event.getProfilingInfo(CLEvent.ProfilingCommand.END) -
// event.getProfilingInfo(CLEvent.ProfilingCommand.START))/1000000.0);
}
public static void main(String[] args) {
JavaCVCL context = new JavaCVCL();
CLImageFormat[] formats = context.getCLContext().getSupportedImage2dFormats();
for (CLImageFormat f : formats) {
System.out.println(f);
}
CameraDevice camera = new CameraDevice("Camera");
camera.imageWidth = 1280;
camera.imageHeight = 960;
camera.cameraMatrix = CvMat.create(3, 3);
double f = camera.imageWidth*2.5;
camera.cameraMatrix.put(f, 0.0, camera.imageWidth /2,
0.0, f, camera.imageHeight/2,
0.0, 0.0, 1);
camera.R = CvMat.create(3, 3);
cvSetIdentity(camera.R);
camera.T = CvMat.create(3, 1);
cvSetZero(camera.T);
camera.distortionCoeffs = CvMat.create(1, 4);
cvSetZero(camera.distortionCoeffs);
camera.distortionCoeffs.put(0.2);
camera.colorMixingMatrix = CvMat.create(3, 3);
cvSetIdentity(camera.colorMixingMatrix);
IplImage srcImg = cvLoadImageRGBA(args[0]);
//IplImage dstImg = srcImg.clone();
IplImage downDst = IplImage.create(srcImg.width()/2, srcImg.height()/2, IPL_DEPTH_8U /*IPL_DEPTH_32F*/, 4);
camera.setFixedPointMaps(false);
camera.setMapsPyramidLevel(1);
IplImage mapxImg = camera.getUndistortMap1();
IplImage mapyImg = camera.getUndistortMap2();
long start = System.nanoTime();
cvRemap(srcImg, downDst, mapxImg, mapyImg, CV_INTER_LINEAR | CV_WARP_FILL_OUTLIERS, CvScalar.ZERO);
System.out.println("cvRemap: " + (System.nanoTime()-start)/1000000.0);
cvSaveImage("/tmp/opencv.png", downDst);
CLImage2d src = context.createCLImageFrom(srcImg);
// CLImage2d dst = context.createCLImageFrom(dstImg);
CLImage2d dst = context.createCLImageFrom(downDst);
CLImage2d mapx = context.createCLImageFrom(mapxImg);
CLImage2d mapy = context.createCLImageFrom(mapyImg);
context.writeImage(src, srcImg, false);
context.writeImage(mapx, mapxImg, false);
context.writeImage(mapy, mapyImg, false);
//context.pyrDown(src, dst);
context.remap(src, dst, mapx, mapy);
context.readImage(dst, downDst, true);
//cvConvertScale(downDst, downDst, 255, 0);
cvSaveImage("/tmp/javacvcl.png", downDst);
context.release();
System.exit(0);
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/JavaCvErrorCallback.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.awt.Component;
import java.awt.EventQueue;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.swing.JOptionPane;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.Pointer;
import org.bytedeco.opencv.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_core.*;
/**
*
* @author Samuel Audet
*/
public class JavaCvErrorCallback extends CvErrorCallback {
static JavaCvErrorCallback instance;
public JavaCvErrorCallback() {
this(false);
}
public JavaCvErrorCallback(boolean showDialog) {
this(showDialog, null);
}
public JavaCvErrorCallback(boolean showDialog, Component parent) {
this(showDialog, parent, 0);
}
public JavaCvErrorCallback(boolean showDialog, Component parent, int rc) {
instance = this;
this.parent = parent;
this.showDialog = showDialog;
this.rc = rc;
}
private long lastErrorTime = 0;
private Component parent;
private boolean showDialog;
private int rc;
@Override public int call(int status, BytePointer func_name, BytePointer err_msg,
BytePointer file_name, int line, Pointer userdata) {
final String title = "OpenCV Error";
final String message = cvErrorStr(status) +
" (" + err_msg.getString() + ")\nin function " +
func_name.getString() + ", " + file_name.getString() + "(" + line + ")";
Logger.getLogger(JavaCvErrorCallback.class.getName()).log(Level.SEVERE,
title + ": " + message, new java.lang.Exception("Strack trace"));
if (showDialog) {
// Show no more than 1 dialog per second since we cannot stop OpenCV
// from processing and throwing more errors. Maybe in the future
// when JavaCPP allows us to throw Exceptions across...
if (System.currentTimeMillis() - lastErrorTime > 1000) {
EventQueue.invokeLater(new Runnable() {
public void run() {
JOptionPane.showMessageDialog(parent, message,
title, JOptionPane.ERROR_MESSAGE);
}
});
}
lastErrorTime = System.currentTimeMillis();
}
return rc; // 0 = please don't terminate
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/JavaFXFrameConverter.java
================================================
/*
* Copyright (C) 2018 Samuel Audet, Johan Vos
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.nio.Buffer;
import java.nio.ByteBuffer;
import java.nio.IntBuffer;
import javafx.scene.image.Image;
import javafx.scene.image.PixelFormat;
import javafx.scene.image.PixelReader;
import javafx.scene.image.WritableImage;
import javafx.scene.image.WritablePixelFormat;
import javafx.scene.paint.Color;
/**
*
* Convert Frames into JavaFX images and vice versa
* @author johan
*/
public class JavaFXFrameConverter extends FrameConverter {
@Override
public Frame convert(Image f) {
throw new UnsupportedOperationException("conversion from Image to Frame not supported yet.");
}
@Override
public Image convert(Frame frame) {
int iw = frame.imageWidth;
int ih = frame.imageHeight;
PixelReader pr = new FramePixelReader(frame);
WritableImage answer = new WritableImage(pr, iw, ih);
return answer;
}
class FramePixelReader implements PixelReader {
Frame frame;
FramePixelReader(Frame f) {
this.frame = f;
}
@Override
public PixelFormat getPixelFormat() {
throw new UnsupportedOperationException("getPixelFormat not supported yet.");
}
@Override
public int getArgb(int x, int y) {
throw new UnsupportedOperationException("getArgb not supported yet.");
}
@Override
public Color getColor(int x, int y) {
throw new UnsupportedOperationException("getColor not supported yet.");
}
@Override
public void getPixels(int x, int y, int w, int h, WritablePixelFormat pixelformat, T buffer, int scanlineStride) {
int fss = frame.imageStride;
if (frame.imageChannels != 3) {
throw new UnsupportedOperationException("We only support frames with imageChannels = 3 (BGR)");
}
if (buffer instanceof ByteBuffer) {
ByteBuffer bb = (ByteBuffer) buffer;
ByteBuffer b = (ByteBuffer) frame.image[0];
for (int i = y; i < y + h; i++) {
for (int j = x; j < x + w; j++) {
int base = 3 * j;
bb.put(b.get(fss * i + base));
bb.put(b.get(fss * i + base + 1));
bb.put(b.get(fss * i + base + 2));
bb.put((byte) 255);
}
}
} else throw new UnsupportedOperationException ("We only support bytebuffers at the moment");
}
@Override
public void getPixels(int x, int y, int w, int h, WritablePixelFormat pixelformat, byte[] buffer, int offset, int scanlineStride) {
throw new UnsupportedOperationException("getPixels Not supported yet.");
}
@Override
public void getPixels(int x, int y, int w, int h, WritablePixelFormat pixelformat, int[] buffer, int offset, int scanlineStride) {
throw new UnsupportedOperationException("getPixelsNot supported yet.");
}
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/LeptonicaFrameConverter.java
================================================
/*
* Copyright (C) 2018-2021 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.nio.Buffer;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.javacpp.Pointer;
import org.bytedeco.leptonica.*;
import static org.bytedeco.leptonica.global.leptonica.*;
/**
* A utility class to map data between {@link Frame} and {@link PIX},
* which is the image data structure used by Tesseract.
* Currently supports only plain PIX images, not FPIX or DPIX ones.
*
* @author Samuel Audet
*/
public class LeptonicaFrameConverter extends FrameConverter {
static { Loader.load(org.bytedeco.leptonica.global.leptonica.class); }
PIX pix;
BytePointer frameData, pixData;
ByteBuffer frameBuffer, pixBuffer;
static boolean isEqual(Frame frame, PIX pix) {
return pix != null && frame != null && frame.image != null && frame.image.length > 0
&& frame.imageWidth == pix.w() && frame.imageHeight == pix.h()
&& frame.imageChannels == pix.d() / 8 && frame.imageDepth == Frame.DEPTH_UBYTE
&& (ByteOrder.nativeOrder().equals(ByteOrder.LITTLE_ENDIAN)
|| new Pointer(frame.image[0]).address() == pix.data().address())
&& frame.imageStride * Math.abs(frame.imageDepth) / 8 == pix.wpl() * 4;
}
public PIX convert(Frame frame) {
if (frame == null || frame.image == null) {
return null;
} else if (frame.opaque instanceof PIX) {
return (PIX)frame.opaque;
} else if (!isEqual(frame, pix)) {
Pointer data;
if (ByteOrder.nativeOrder().equals(ByteOrder.LITTLE_ENDIAN)) {
if (pixData == null || pixData.capacity() < frame.imageHeight * frame.imageStride) {
if (pixData != null) {
pixData.releaseReference();
}
pixData = new BytePointer(frame.imageHeight * frame.imageStride).retainReference();
}
data = pixData;
pixBuffer = data.asByteBuffer().order(ByteOrder.BIG_ENDIAN);
} else {
data = new Pointer(frame.image[0].position(0));
}
if (pix != null) {
pix.releaseReference();
}
pix = PIX.create(frame.imageWidth, frame.imageHeight, frame.imageChannels * 8, data)
.wpl(frame.imageStride / 4 * Math.abs(frame.imageDepth) / 8).retainReference();
}
if (ByteOrder.nativeOrder().equals(ByteOrder.LITTLE_ENDIAN)) {
((ByteBuffer)pixBuffer.position(0)).asIntBuffer()
.put(((ByteBuffer)frame.image[0].position(0)).order(ByteOrder.LITTLE_ENDIAN).asIntBuffer());
}
return pix;
}
public Frame convert(PIX pix) {
if (pix == null) {
return null;
}
PIX tempPix = null;
if (pix.colormap() != null) {
tempPix = pix = pixRemoveColormap(pix, REMOVE_CMAP_TO_FULL_COLOR);
} else if (pix.d() < 8) {
switch (pix.d()) {
case 1:
tempPix = pix = pixConvert1To8(null, pix, (byte)0, (byte)255);
break;
case 2:
tempPix = pix = pixConvert2To8(pix, (byte)0, (byte)85, (byte)170, (byte)255, 0);
break;
case 4:
tempPix = pix = pixConvert4To8(pix, 0);
break;
default:
assert false;
}
}
if (!isEqual(frame, pix)) {
frame = new Frame();
frame.imageWidth = pix.w();
frame.imageHeight = pix.h();
frame.imageDepth = Frame.DEPTH_UBYTE;
frame.imageChannels = pix.d() / 8;
frame.imageStride = pix.wpl() * 4;
if (ByteOrder.nativeOrder().equals(ByteOrder.LITTLE_ENDIAN)) {
if (frameData == null || frameData.capacity() < frame.imageHeight * frame.imageStride) {
if (frameData != null) {
frameData.releaseReference();
}
frameData = new BytePointer(frame.imageHeight * frame.imageStride).retainReference();
}
frameBuffer = frameData.asByteBuffer().order(ByteOrder.LITTLE_ENDIAN);
frame.opaque = frameData;
frame.image = new Buffer[] { frameBuffer };
} else {
if (tempPix != null) {
if (this.pix != null) {
this.pix.releaseReference();
}
this.pix = pix = pix.clone();
}
frame.opaque = pix;
frame.image = new Buffer[] { pix.createBuffer() };
}
}
if (ByteOrder.nativeOrder().equals(ByteOrder.LITTLE_ENDIAN)) {
((ByteBuffer)frameBuffer.position(0)).asIntBuffer()
.put(pix.createBuffer().order(ByteOrder.BIG_ENDIAN).asIntBuffer());
}
if (tempPix != null) {
pixDestroy(tempPix);
}
return frame;
}
@Override public void close() {
super.close();
if (pix != null) {
pix.releaseReference();
pix = null;
}
if (pixData != null) {
pixData.releaseReference();
pixData = null;
}
if (frameData != null) {
frameData.releaseReference();
frameData = null;
}
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/LibgdxFrameConverter.java
================================================
/*
* Copyright (C) 2025 shan-luan
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import com.badlogic.gdx.graphics.Pixmap;
import java.nio.ByteBuffer;
/**
* A utility class for converting between {@link Pixmap} and {@link Frame}.
* The alpha channel is not converted and memory cannot be shared.
*
* @author shan-luan
*/
public class LibgdxFrameConverter extends FrameConverter {
/**
* Converts a {@link Pixmap} to a {@link Frame}.
*
* @param pixmap the Pixmap to convert,RGBA8888 format
* @return the converted Frame
*/
@Override
public Frame convert(Pixmap pixmap) {
if (pixmap == null) return null;
Frame frame = new Frame(pixmap.getWidth(), pixmap.getHeight(), Frame.DEPTH_UBYTE, 3, pixmap.getWidth() * 3);
ByteBuffer pixmapBuffer = pixmap.getPixels().duplicate();
ByteBuffer frameBuffer = (ByteBuffer) frame.image[0];
int numPixels = pixmap.getWidth() * pixmap.getHeight();
for (int i = 0; i < numPixels; i++) {
byte r = pixmapBuffer.get();
byte g = pixmapBuffer.get();
byte b = pixmapBuffer.get();
pixmapBuffer.position(pixmapBuffer.position() + 1);
frameBuffer.put(b);
frameBuffer.put(g);
frameBuffer.put(r);
}
frameBuffer.flip();
return frame;
}
/**
* Converts a {@link Frame} to a {@link Pixmap}.
*
* @param frame the Frame to convert
* @return the converted Pixmap, RGBA8888 format
*/
@Override
public Pixmap convert(Frame frame) {
if (frame == null || frame.image[0] == null) return null;
Pixmap pixmap = new Pixmap(frame.imageWidth, frame.imageHeight, Pixmap.Format.RGBA8888);
ByteBuffer frameBuffer = ((ByteBuffer) frame.image[0]).duplicate();
ByteBuffer pixmapBuffer = pixmap.getPixels();
pixmapBuffer.position(0);
frameBuffer.rewind();
int numPixels = frame.imageWidth * frame.imageHeight;
for (int i = 0; i < numPixels; i++) {
byte b = frameBuffer.get();
byte g = frameBuffer.get();
byte r = frameBuffer.get();
pixmapBuffer.put(r);
pixmapBuffer.put(g);
pixmapBuffer.put(b);
pixmapBuffer.put((byte) -1);// alpha always set to 255
}
pixmapBuffer.flip();
return pixmap;
}
/**
* Converts a {@link Frame} to a {@link Pixmap}.
* Available only when the format of the Frame is {@link org.bytedeco.ffmpeg.global.avutil
* #AV_PIX_FMT_RGBA}.
* Faster than {@link #convert(Frame)}
*
* @param frame the Frame to convert
* @return the converted Pixmap, RGBA8888 format
*/
public Pixmap fastConvert(Frame frame) {
if (frame == null || frame.image[0] == null) return null;
Pixmap pixmap = new Pixmap(frame.imageWidth, frame.imageHeight, Pixmap.Format.RGBA8888);
ByteBuffer frameBuffer = ((ByteBuffer) frame.image[0]).duplicate();
ByteBuffer pixmapBuffer = pixmap.getPixels();
pixmapBuffer.position(0);
frameBuffer.rewind();
pixmapBuffer.put(frameBuffer);
pixmapBuffer.flip();
return pixmap;
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/MarkedPlane.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import org.bytedeco.opencv.opencv_calib3d.*;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_calib3d.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
*
* @author Samuel Audet
*/
public class MarkedPlane {
public MarkedPlane(int width, int height, Marker[] planeMarkers, double superScale) {
this(width, height, planeMarkers, false, CvScalar.BLACK, CvScalar.WHITE, superScale);
}
public MarkedPlane(int width, int height, Marker[] markers,
boolean initPrewarp, CvScalar foregroundColor, CvScalar backgroundColor, double superScale) {
this.markers = markers;
this.foregroundColor = foregroundColor;
this.backgroundColor = backgroundColor;
// this.srcPts = CvMat.create(planeMarkers.length*4, 2);
// this.dstPts = CvMat.create(planeMarkers.length*4, 2);
this.prewarp = null;
// this.totalWarp = CvMat.create(3, 3);
// this.tempWarp = CvMat.create(3, 3);
if (initPrewarp) {
prewarp = CvMat.create(3, 3);
double minx = Double.MAX_VALUE, miny = Double.MAX_VALUE,
maxx = Double.MIN_VALUE, maxy = Double.MIN_VALUE;
for (Marker m : markers) {
double[] c = m.corners;
minx = Math.min(Math.min(Math.min(Math.min(minx, c[0]), c[2]), c[4]), c[6]);
miny = Math.min(Math.min(Math.min(Math.min(miny, c[1]), c[3]), c[5]), c[7]);
maxx = Math.max(Math.max(Math.max(Math.max(maxx, c[0]), c[2]), c[4]), c[6]);
maxy = Math.max(Math.max(Math.max(Math.max(maxy, c[1]), c[3]), c[5]), c[7]);
}
double aspect = (maxx-minx)/(maxy-miny);
if (aspect > (double)width/height) {
double h = (double)width/aspect;
// srcPtsBuf.position(0); srcPtsBuf.put(new double[] { minx, miny, maxx, miny, maxx, maxy, minx, maxy });
// dstPtsBuf.position(0); dstPtsBuf.put(new double[] { 0, height-h, width, height-h, width, height, 0, height });
// srcPts.height = dstPts.height = 4;
// cvFindHomography(srcPts, dstPts, preWarp);
JavaCV.getPerspectiveTransform(
new double[] { minx, miny, maxx, miny, maxx, maxy, minx, maxy },
new double[] { 0, height-h, width, height-h, width, height, 0, height }, prewarp);
} else {
double w = height*aspect;
// srcPtsBuf.position(0); srcPtsBuf.put(new double[] { minx, miny, maxx, miny, maxx, maxy, minx, maxy });
// dstPtsBuf.position(0); dstPtsBuf.put(new double[] { 0, 0, w, 0, w, height, 0, height });
// srcPts.height = dstPts.height = 4;
// cvFindHomography(srcPts, dstPts, preWarp);
JavaCV.getPerspectiveTransform(
new double[] { minx, miny, maxx, miny, maxx, maxy, minx, maxy },
new double[] { 0, 0, w, 0, w, height, 0, height }, prewarp);
}
}
if (width > 0 && height > 0) {
planeImage = IplImage.create(width, height, IPL_DEPTH_8U, 1);
if (superScale == 1.0) {
superPlaneImage = null;
} else {
superPlaneImage = IplImage.create((int)Math.ceil(width*superScale),
(int)Math.ceil(height*superScale), IPL_DEPTH_8U, 1);
}
setPrewarp(prewarp);
}
localSrcPts = CvMat.createThreadLocal(markers.length*4, 2);
localDstPts = CvMat.createThreadLocal(markers.length*4, 2);
}
private Marker[] markers = null;
// private CvPoint tempPts = new CvPoint(4);
// private CvMat srcPts, dstPts;
private CvMat prewarp;//, totalWarp, tempWarp;
private IplImage planeImage = null, superPlaneImage = null;
private CvScalar foregroundColor, backgroundColor;
private ThreadLocal localSrcPts, localDstPts;
public CvScalar getForegroundColor() {
return foregroundColor;
}
public void setForegroundColor(CvScalar foregroundColor) {
this.foregroundColor = foregroundColor;
setPrewarp(prewarp);
}
public CvScalar getBackgroundColor() {
return backgroundColor;
}
public void setBackgroundColor(CvScalar backgroundColor) {
this.backgroundColor = backgroundColor;
setPrewarp(prewarp);
}
public Marker[] getMarkers() {
return markers;
}
public void setColors(CvScalar foregroundColor, CvScalar backgroundColor) {
this.foregroundColor = foregroundColor;
this.backgroundColor = backgroundColor;
setPrewarp(prewarp);
}
public CvMat getPrewarp() {
return prewarp;
}
public void setPrewarp(CvMat prewarp) {
this.prewarp = prewarp;
if (superPlaneImage == null) {
cvSet(planeImage, backgroundColor);
} else {
cvSet(superPlaneImage, backgroundColor);
}
for (int i = 0; i < markers.length; i++) {
if (superPlaneImage == null) {
markers[i].draw(planeImage, foregroundColor, 1.0, prewarp);
} else {
markers[i].draw(superPlaneImage, foregroundColor, (double)
superPlaneImage.width()/planeImage.width(), prewarp);
}
}
if (superPlaneImage != null) {
cvResize(superPlaneImage, planeImage, CV_INTER_AREA);
}
//cvSaveImage("planeImage.png", planeImage);
}
public IplImage getImage() {
return planeImage;
}
public int getWidth() {
return planeImage.width();
}
public int getHeight() {
return planeImage.height();
}
public double getTotalWarp(Marker[] imagedMarkers, CvMat totalWarp) {
return getTotalWarp(imagedMarkers, totalWarp, false);
}
private static ThreadLocal
tempWarp3x3 = CvMat.createThreadLocal(3, 3);
public double getTotalWarp(Marker[] imagedMarkers, CvMat totalWarp, boolean useCenters) {
double rmse = Double.POSITIVE_INFINITY;
int pointsPerMarker = useCenters ? 1 : 4;
CvMat srcPts = localSrcPts.get(); srcPts.rows(markers.length*pointsPerMarker);
CvMat dstPts = localDstPts.get(); dstPts.rows(markers.length*pointsPerMarker);
int numPoints = 0;
for (Marker m1 : markers) {
for (Marker m2 : imagedMarkers) {
if (m1.id == m2.id) {
if (useCenters) {
srcPts.put(numPoints*2, m1.getCenter());
dstPts.put(numPoints*2, m2.getCenter());
} else {
srcPts.put(numPoints*2, m1.corners);
dstPts.put(numPoints*2, m2.corners);
}
numPoints += pointsPerMarker;
break;
}
}
}
if (numPoints > 4 || (srcPts.rows() == 4 && numPoints == 4)) {
// compute homography ... should we use a robust method?
srcPts.rows(numPoints); dstPts.rows(numPoints);
if (numPoints == 4) {
JavaCV.getPerspectiveTransform(srcPts.get(), dstPts.get(), totalWarp);
} else {
cvCopy(cvMat(findHomography(cvarrToMat(srcPts), cvarrToMat(dstPts))), totalWarp);
}
// compute transformed source<->dest RMSE
srcPts.cols(1); srcPts.type(CV_64F, 2);
dstPts.cols(1); dstPts.type(CV_64F, 2);
cvPerspectiveTransform(srcPts, srcPts, totalWarp);
srcPts.cols(2); srcPts.type(CV_64F, 1);
dstPts.cols(2); dstPts.type(CV_64F, 1);
rmse = 0;
for (int i = 0; i < numPoints; i++) {
double dx = dstPts.get(i*2 )-srcPts.get(i*2 );
double dy = dstPts.get(i*2+1)-srcPts.get(i*2+1);
rmse += dx*dx+dy*dy;
}
rmse = Math.sqrt(rmse/numPoints);
// System.out.println(rmse);
if (prewarp != null) {
// remove pre-warp from total warp
CvMat tempWarp = tempWarp3x3.get();
cvInvert(prewarp, tempWarp);
cvMatMul(totalWarp, tempWarp, totalWarp);
}
// System.out.println("totalWarp:\n" + totalWarp);
}
return rmse;
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/Marker.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.nio.ByteBuffer;
import java.util.Arrays;
import org.bytedeco.artoolkitplus.*;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.artoolkitplus.global.ARToolKitPlus.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
*
* @author Samuel Audet
*/
public class Marker implements Cloneable {
public Marker(int id, double[] corners, double confidence) {
this.id = id;
this.corners = corners;
this.confidence = confidence;
}
public Marker(int id, double ... corners) {
this(id, corners, 1.0);
}
@Override public Marker clone() {
return new Marker(id, corners.clone(), confidence);
}
public int id;
public double[] corners;
public double confidence;
@Override public int hashCode() {
int hash = 7;
hash = 37 * hash + this.id;
hash = 37 * hash + (this.corners != null ? this.corners.hashCode() : 0);
return hash;
}
@Override public boolean equals(Object o) {
if (o instanceof Marker) {
Marker m = (Marker)o;
return m.id == id && Arrays.equals(m.corners, corners);
}
return false;
}
public double[] getCenter() {
double x = 0, y = 0;
if (true) {
// the centroid is not what we want as it does not remain at
// the same physical point under projective transformations..
// But it has the advantage of averaging noise better, and does
// give better results
for (int i = 0; i < 4; i++) {
x += corners[2*i ];
y += corners[2*i+1];
}
x /= 4;
y /= 4;
} else {
double x1 = corners[0]; double y1 = corners[1];
double x2 = corners[4]; double y2 = corners[5];
double x3 = corners[2]; double y3 = corners[3];
double x4 = corners[6]; double y4 = corners[7];
double u = ((x4-x3)*(y1-y3) - (y4-y3)*(x1-x3))/
((y4-y3)*(x2-x1) - (x4-x3)*(y2-y1));
x = x1 + u*(x2-x1);
y = y1 + u*(y2-y1);
}
return new double[] { x, y };
}
public IplImage getImage() {
return getImage(id);
}
private static IplImage imageCache[] = new IplImage[4096];
public static IplImage getImage(int id) {
if (imageCache[id] == null) {
imageCache[id] = IplImage.create(8, 8, IPL_DEPTH_8U, 1);
createImagePatternBCH(id, imageCache[id].getByteBuffer());
}
return imageCache[id];
}
private static final double[] src = { 0, 0, 8, 0, 8, 8, 0, 8 };
public void draw(IplImage image) {
draw(image, CvScalar.BLACK, 1, null);
}
public void draw(IplImage image, CvScalar color, double scale, CvMat prewarp) {
draw(image, color, scale, scale, prewarp);
}
private static ThreadLocal
H3x3 = CvMat.createThreadLocal(3, 3),
srcPts4x1 = CvMat.createThreadLocal(4, 1, CV_64F, 2),
dstPts4x1 = CvMat.createThreadLocal(4, 1, CV_64F, 2);
public void draw(IplImage image, CvScalar color, double scaleX, double scaleY, CvMat prewarp) {
CvMat H = H3x3.get();
JavaCV.getPerspectiveTransform(src, corners, H);
if (prewarp != null) {
cvGEMM(prewarp, H, 1, null, 0, H, 0);
}
IplImage marker = getImage();
ByteBuffer mbuf = marker.getByteBuffer();
CvMat srcPts = srcPts4x1.get();
CvMat dstPts = dstPts4x1.get();
CvPoint tempPts = new CvPoint(4);
int h = marker.height();
int w = marker.width();
for (int y = 0; y < h; y++) {
for (int x = 0; x < w; x++) {
if (mbuf.get(y*w + x) == 0) {
srcPts.put((double)x, y, x+1, y, x+1, y+1, x, y+1);
//System.out.println("srcPts" + srcPts);
cvPerspectiveTransform(srcPts, dstPts, H);
//System.out.println("dstPts" + dstPts);
double centerx = 0, centery = 0;
for (int i = 0; i < 4; i++) {
centerx += dstPts.get(i*2 );
centery += dstPts.get(i*2+1);
}
centerx /= 4;
centery /= 4;
for (int i = 0; i < 4; i++) {
double a = dstPts.get(i*2 );
double b = dstPts.get(i*2+1);
double dx = centerx - a;
double dy = centery - b;
dx = dx < 0 ? -1 : 0;
dy = dy < 0 ? -1 : 0;
tempPts.position(i).x((int)Math.round((a*scaleX + dx) * (1<<16)));
tempPts.position(i).y((int)Math.round((b*scaleY + dy) * (1<<16)));
}
cvFillConvexPoly(image, tempPts.position(0), 4, color, 8/*CV_AA*/, 16);
}
}
}
}
public static class ArraySettings extends BaseChildSettings {
int rows = 8, columns = 12;
double sizeX = 200, sizeY = 200, spacingX = 300, spacingY = 300;
boolean checkered = true;
public int getRows() {
return rows;
}
public void setRows(int rows) {
firePropertyChange("rows", this.rows, this.rows = rows);
}
public int getColumns() {
return columns;
}
public void setColumns(int columns) {
firePropertyChange("columns", this.columns, this.columns = columns);
}
public double getSizeX() {
return sizeX;
}
public void setSizeX(double sizeX) {
firePropertyChange("sizeX", this.sizeX, this.sizeX = sizeX);
}
public double getSizeY() {
return sizeY;
}
public void setSizeY(double sizeY) {
firePropertyChange("sizeY", this.sizeY, this.sizeY = sizeY);
}
public double getSpacingX() {
return spacingX;
}
public void setSpacingX(double spacingX) {
firePropertyChange("spacingX", this.spacingX, this.spacingX = spacingX);
}
public double getSpacingY() {
return spacingY;
}
public void setSpacingY(double spacingY) {
firePropertyChange("spacingY", this.spacingY, this.spacingY = spacingY);
}
public boolean isCheckered() {
return checkered;
}
public void setCheckered(boolean checkered) {
firePropertyChange("checkered", this.checkered, this.checkered = checkered);
}
}
public static Marker[][] createArray(ArraySettings settings) {
return createArray(settings, 0, 0);
}
public static Marker[][] createArray(ArraySettings settings, double marginx, double marginy) {
Marker[] markers = new Marker[settings.rows*settings.columns];
int id = 0;
for (int y = 0; y < settings.rows; y++) {
for (int x = 0; x < settings.columns; x++) {
double sx = settings.sizeX/2;
double sy = settings.sizeY/2;
double cx = x*settings.spacingX + sx + marginx;
double cy = y*settings.spacingY + sy + marginy;
markers[id] = new Marker(id, new double[] {
cx-sx, cy-sy, cx+sx, cy-sy, cx+sx, cy+sy, cx-sx, cy+sy }, 1);
id++;
}
}
if (!settings.checkered) {
return new Marker[][] { markers };
} else {
Marker[] markers1 = new Marker[markers.length/2];
Marker[] markers2 = new Marker[markers.length/2];
for (int i = 0; i < markers.length; i++) {
int x = i%settings.columns;
int y = i/settings.columns;
if (x%2==0 ^ y%2==0) {
markers2[i/2] = markers[i];
} else {
markers1[i/2] = markers[i];
}
}
return new Marker[][] { markers2, markers1 };
}
}
public static Marker[][] createArray(int rows, int columns, double sizeX, double sizeY,
double spacingX, double spacingY, boolean checkered, double marginx, double marginy) {
ArraySettings s = new ArraySettings();
s.rows = rows; s.columns = columns;
s.sizeX = sizeX; s.sizeY = sizeY;
s.spacingX = spacingX; s.spacingY = spacingY;
s.checkered = checkered;
return createArray(s, marginx, marginy);
}
public static void applyWarp(Marker[] markers, CvMat warp) {
CvMat pts = srcPts4x1.get();
for (Marker m : markers) {
cvPerspectiveTransform(pts.put(m.corners), pts, warp);
pts.get(m.corners);
}
}
@Override public String toString() {
String s = "[" + id + ": " +
"(" + corners[0] + ", " + corners[1] + ") " +
"(" + corners[2] + ", " + corners[3] + ") " +
"(" + corners[4] + ", " + corners[5] + ") " +
"(" + corners[6] + ", " + corners[7] + ")]";
return s;
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/MarkerDetector.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.util.Arrays;
import org.bytedeco.javacpp.IntPointer;
import org.bytedeco.artoolkitplus.*;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.artoolkitplus.global.ARToolKitPlus.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
*
* @author Samuel Audet
*/
public class MarkerDetector {
public MarkerDetector(Settings settings) {
setSettings(settings);
}
public MarkerDetector() {
this(new Settings());
}
// the k's will depend strongly on the ratio between ambient light
// (including minimum projector intensity) and the intensity level
// used for the projector markers... this is because we use binary
// thresholding while we actually have three levels..
public static class Settings extends BaseChildSettings {
int thresholdWindowMin = 5;
int thresholdWindowMax = 63;
double thresholdVarMultiplier = 1.0;
double thresholdKBlackMarkers = 0.6;
double thresholdKWhiteMarkers = 1.0;
int subPixelWindow = 11;
public int getThresholdWindowMin() {
return thresholdWindowMin;
}
public void setThresholdWindowMin(int thresholdWindowMin) {
this.thresholdWindowMin = thresholdWindowMin;
}
public int getThresholdWindowMax() {
return thresholdWindowMax;
}
public void setThresholdWindowMax(int thresholdWindowMax) {
this.thresholdWindowMax = thresholdWindowMax;
}
public double getThresholdVarMultiplier() {
return thresholdVarMultiplier;
}
public void setThresholdVarMultiplier(double thresholdVarMultiplier) {
this.thresholdVarMultiplier = thresholdVarMultiplier;
}
public double getThresholdKBlackMarkers() {
return thresholdKBlackMarkers;
}
public void setThresholdKBlackMarkers(double thresholdKBlackMarkers) {
this.thresholdKBlackMarkers = thresholdKBlackMarkers;
}
public double getThresholdKWhiteMarkers() {
return thresholdKWhiteMarkers;
}
public void setThresholdKWhiteMarkers(double thresholdKWhiteMarkers) {
this.thresholdKWhiteMarkers = thresholdKWhiteMarkers;
}
public int getSubPixelWindow() {
return subPixelWindow;
}
public void setSubPixelWindow(int subPixelWindow) {
this.subPixelWindow = subPixelWindow;
}
}
private Settings settings;
public Settings getSettings() {
return settings;
}
public void setSettings(Settings settings) {
this.settings = settings;
this.subPixelSize = cvSize(settings.subPixelWindow/2, settings.subPixelWindow/2);
this.subPixelZeroZone = cvSize(-1,-1);
this.subPixelTermCriteria = cvTermCriteria(CV_TERMCRIT_EPS, 100, 0.001);
}
private MultiTracker tracker = null;
private IntPointer markerNum = new IntPointer(1);
private int width = 0, height = 0, depth = 0, channels = 0;
private IplImage tempImage, tempImage2, sumImage, sqSumImage, thresholdedImage;
private CvMat points = CvMat.create(1, 4, CV_32F, 2);
private CvPoint2D32f corners = new CvPoint2D32f(4);
private CvMemStorage memory = CvMemStorage.create();
private CvSize subPixelSize = null, subPixelZeroZone = null;
private CvTermCriteria subPixelTermCriteria = null;
private CvFont font = cvFont(1, 1);
private CvSize textSize = new CvSize();
public IplImage getThresholdedImage() {
return thresholdedImage;
}
private void init(IplImage image) {
if (tracker != null && image.width() == width && image.height() == height &&
image.depth() == depth && image.nChannels() == channels) {
return;
}
width = image.width();
height = image.height();
depth = image.depth();
channels = image.nChannels();
if (depth != IPL_DEPTH_8U || channels > 1) {
tempImage = IplImage.create(width, height, IPL_DEPTH_8U, 1);
}
if (depth != IPL_DEPTH_8U && channels > 1) {
tempImage2 = IplImage.create(width, height, IPL_DEPTH_8U, 3);
}
sumImage = IplImage.create(width+1, height+1, IPL_DEPTH_64F, 1);
sqSumImage = IplImage.create(width+1, height+1, IPL_DEPTH_64F, 1);
thresholdedImage = IplImage.create(width, height, IPL_DEPTH_8U, 1);
tracker = new MultiTracker(thresholdedImage.widthStep(), thresholdedImage.height());
// if (depth != IPL_DEPTH_8U) {
// throw new Exception("Unsupported format: IplImage must have depth == IPL_DEPTH_8U.");
// }
int pixfmt = PIXEL_FORMAT_LUM;
// switch (nChannels) {
// case 4: pixfmt = PIXEL_FORMAT_BGRA; break;
// case 3: pixfmt = PIXEL_FORMAT_BGR; break;
// case 1: pixfmt = PIXEL_FORMAT_LUM; break;
// default:
// throw new Exception("Unsupported format: No support for IplImage with " + channels + " channels.");
// }
tracker.setPixelFormat(pixfmt);
// if(!tracker.init("data/LogitechPro4000.dat",
// "data/markerboard_480-499.cfg", 1.0f, 1000.0f, null)) {
// throw new Exception("ERROR: init() failed.");
// }
tracker.setBorderWidth(0.125f);
// tracker.setThreshold(128);
// tracker.activateAutoThreshold(true);
// tracker.setNumAutoThresholdRetries(10);
tracker.setUndistortionMode(UNDIST_NONE);
// tracker.setPoseEstimator(POSE_ESTIMATOR_RPP);
tracker.setMarkerMode(MARKER_ID_BCH);
tracker.setImageProcessingMode(IMAGE_FULL_RES);
}
public Marker[] detect(IplImage image, boolean whiteMarkers) {
init(image);
if (depth != IPL_DEPTH_8U && channels > 1) {
cvConvertScale(image, tempImage2, 255/image.highValue(), 0);
cvCvtColor(tempImage2, tempImage, channels > 3 ? CV_RGBA2GRAY : CV_BGR2GRAY);
image = tempImage;
} else if (depth != IPL_DEPTH_8U) {
cvConvertScale(image, tempImage, 255/image.highValue(), 0);
image = tempImage;
} else if (channels > 1) {
cvCvtColor(image, tempImage, channels > 3 ? CV_RGBA2GRAY : CV_BGR2GRAY);
image = tempImage;
}
//long time1 = System.currentTimeMillis();
JavaCV.adaptiveThreshold(image, sumImage, sqSumImage, thresholdedImage, whiteMarkers,
settings.thresholdWindowMax, settings.thresholdWindowMin, settings.thresholdVarMultiplier,
whiteMarkers ? settings.thresholdKWhiteMarkers : settings.thresholdKBlackMarkers);
//CanvasFrame.global.showImage(thresholded, 0.5);
//CanvasFrame.global.waitKey();
//long time2 = System.currentTimeMillis();
int n = 0;
ARMarkerInfo markers = new ARMarkerInfo(null);
tracker.arDetectMarkerLite(thresholdedImage.imageData(), 128 /*tracker.getThreshold()*/, markers, markerNum);
//long time3 = System.currentTimeMillis();
Marker[] markers2 = new Marker[markerNum.get(0)];
for (int i = 0; i < markers2.length && !markers.isNull(); i++) {
markers.position(i);
int id = markers.id();
if (id < 0) {
// no detected ID...
continue;
}
int dir = markers.dir();
float confidence = markers.cf();
float[] vertex = new float[8];
markers.vertex().get(vertex);
int w = settings.subPixelWindow/2+1;
if (vertex[0]-w < 0 || vertex[0]+w >= width || vertex[1]-w < 0 || vertex[1]+w >= height ||
vertex[2]-w < 0 || vertex[2]+w >= width || vertex[3]-w < 0 || vertex[3]+w >= height ||
vertex[4]-w < 0 || vertex[4]+w >= width || vertex[5]-w < 0 || vertex[5]+w >= height ||
vertex[6]-w < 0 || vertex[6]+w >= width || vertex[7]-w < 0 || vertex[7]+w >= height) {
// too tight for cvFindCornerSubPix...
continue;
}
points.getFloatBuffer().put(vertex);
CvBox2D box = cvMinAreaRect2(points, memory);
float bw = box.size().width();
float bh = box.size().height();
cvClearMemStorage(memory);
if (bw <= 0 || bh <= 0 || bw/bh < 0.1 || bw/bh > 10) {
// marker is too "flat" to have been IDed correctly...
continue;
}
for (int j = 0; j < 4; j++) {
corners.position(j).put(vertex[2*j], vertex[2*j+1]);
}
if (false) {
// move the search window a bit (max 1/4 of the window) toward the center...
// this allows us to cram more markers closer to one another
double cx = 0, cy = 0;
for (int j = 0; j < 4; j++) {
corners.position(j);
cx += corners.x();
cy += corners.y();
}
cx /= 4;
cy /= 4;
for (int j = 0; j < 4; j++) {
corners.position(j);
float x = corners.x();
float y = corners.y();
double dx = cx - x;
double dy = cy - y;
corners.x(x + (float)Math.signum(dx)*(settings.subPixelWindow/4));
corners.y(y + (float)Math.signum(dy)*(settings.subPixelWindow/4));
}
}
cvFindCornerSubPix(image, corners.position(0), 4, subPixelSize, subPixelZeroZone, subPixelTermCriteria);
double[] d = { corners.position((4-dir)%4).x(), corners.position((4-dir)%4).y(),
corners.position((5-dir)%4).x(), corners.position((5-dir)%4).y(),
corners.position((6-dir)%4).x(), corners.position((6-dir)%4).y(),
corners.position((7-dir)%4).x(), corners.position((7-dir)%4).y() };
markers2[n++] = new Marker(id, d, confidence);
}
//long time4 = System.currentTimeMillis();
//System.out.println("thresholdTime = " + (time2-time1) + " detectTime = " + (time3-time2) + " subPixTime = " + (time4-time3));
//cvCvtColor(thresholdedImage, image, CV_GRAY2BGR);
//cvCopy(thresholdedImage, image, null);
return Arrays.copyOf(markers2, n);
}
public void draw(IplImage image, Marker[] markers) {
for (Marker m : markers) {
int cx = 0, cy = 0;
int[] pts = new int[8];
for (int i = 0; i < 4; i++) {
int x = (int)Math.round(m.corners[i*2 ] * (1<<16));
int y = (int)Math.round(m.corners[i*2+1] * (1<<16));
pts[2*i ] = x;
pts[2*i + 1] = y;
cx += x;
cy += y;
// draw little colored squares in corners to confirm that the corners
// are returned in the right order...
// CvPoint pt2a = cvPoint(pts[i].x+200000, pts[i].y+200000);
// cvRectangle(image, pts, pt2a,
// i == 0? CV_RGB(maxIntensity, 0, 0) :
// i == 1? CV_RGB(0, maxIntensity, 0) :
// i == 2? CV_RGB(0, 0, maxIntensity) :
// CV_RGB(maxIntensity, maxIntensity, maxIntensity),
// CV_FILLED, CV_AA, 16);
}
cx /= 4;
cy /= 4;
cvPolyLine(image, pts, new int[] { pts.length/2 }, 1, 1, CV_RGB(0, 0, image.highValue()), 1, CV_AA, 16);
String text = Integer.toString(m.id);
int[] baseline = new int[1];
cvGetTextSize(text, font, textSize, baseline);
int[] pt1 = { cx - (textSize.width() *3/2 << 16)/2,
cy + (textSize.height()*3/2 << 16)/2 };
int[] pt2 = { cx + (textSize.width() *3/2 << 16)/2,
cy - (textSize.height()*3/2 << 16)/2 };
cvRectangle(image, pt1, pt2, CV_RGB(0, image.highValue(), 0), CV_FILLED, CV_AA, 16);
int[] pt = { (int)Math.round((double)cx/(1<<16) - textSize.width()/2),
(int)Math.round((double)cy/(1<<16) + textSize.height()/2) + 1 };
cvPutText(image, text, pt, font, CvScalar.BLACK);
}
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/ObjectFinder.java
================================================
/*
* Copyright (C) 2009-2015 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*
* Adapted from the find_obj.cpp sample in the source package of OpenCV 2.3.1:
*
* A Demo to OpenCV Implementation of SURF
* Further Information Refer to "SURF: Speed-Up Robust Feature"
* Author: Liu Liu
* liuliu.1987+opencv@gmail.com
*/
package org.bytedeco.javacv;
import java.nio.ByteBuffer;
import java.nio.FloatBuffer;
import java.nio.IntBuffer;
import java.util.ArrayList;
import java.util.logging.Logger;
import org.bytedeco.opencv.opencv_calib3d.*;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_features2d.*;
import org.bytedeco.opencv.opencv_flann.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_calib3d.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_features2d.*;
import static org.bytedeco.opencv.global.opencv_flann.*;
import static org.bytedeco.opencv.global.opencv_imgcodecs.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
*
* @author Samuel Audet
*
* ObjectFinder does not work out-of-the-box under Android, because it lacks the standard
* java.beans.beancontext package. We can work around it by doing the following *BEFORE*
* following the instructions in the README.md file:
*
* 1. Remove BaseChildSettings.class and BaseSettings.class from javacv.jar
* 2. Follow the instructions in the README.md file
* 3. In your project, define empty classes BaseChildSettings and BaseSettings under the org.bytedeco.javacv package name
*/
public class ObjectFinder {
public ObjectFinder(IplImage objectImage) {
settings = new Settings();
settings.objectImage = objectImage;
setSettings(settings);
}
public ObjectFinder(Settings settings) {
setSettings(settings);
}
public static class Settings extends BaseChildSettings {
IplImage objectImage = null;
AKAZE detector = AKAZE.create();
double distanceThreshold = 0.75;
int matchesMin = 4;
double ransacReprojThreshold = 1.0;
boolean useFLANN = false;
public IplImage getObjectImage() {
return objectImage;
}
public void setObjectImage(IplImage objectImage) {
this.objectImage = objectImage;
}
public int getDescriptorType() {
return detector.getDescriptorType();
}
public void setDescriptorType(int dtype) {
detector.setDescriptorType(dtype);
}
public int getDescriptorSize() {
return detector.getDescriptorSize();
}
public void setDescriptorSize(int dsize) {
detector.setDescriptorSize(dsize);
}
public int getDescriptorChannels() {
return detector.getDescriptorChannels();
}
public void setDescriptorChannels(int dch) {
detector.setDescriptorChannels(dch);
}
public double getThreshold() {
return detector.getThreshold();
}
public void setThreshold(double threshold) {
detector.setThreshold(threshold);
}
public int getNOctaves() {
return detector.getNOctaves();
}
public void setNOctaves(int nOctaves) {
detector.setNOctaves(nOctaves);
}
public int getNOctaveLayers() {
return detector.getNOctaveLayers();
}
public void setNOctaveLayers(int nOctaveLayers) {
detector.setNOctaveLayers(nOctaveLayers);
}
public double getDistanceThreshold() {
return distanceThreshold;
}
public void setDistanceThreshold(double distanceThreshold) {
this.distanceThreshold = distanceThreshold;
}
public int getMatchesMin() {
return matchesMin;
}
public void setMatchesMin(int matchesMin) {
this.matchesMin = matchesMin;
}
public double getRansacReprojThreshold() {
return ransacReprojThreshold;
}
public void setRansacReprojThreshold(double ransacReprojThreshold) {
this.ransacReprojThreshold = ransacReprojThreshold;
}
public boolean isUseFLANN() {
return useFLANN;
}
public void setUseFLANN(boolean useFLANN) {
this.useFLANN = useFLANN;
}
}
Settings settings;
public Settings getSettings() {
return settings;
}
public void setSettings(Settings settings) {
this.settings = settings;
objectKeypoints = new KeyPointVector();
objectDescriptors = new Mat();
settings.detector.detectAndCompute(cvarrToMat(settings.objectImage),
new Mat(), objectKeypoints, objectDescriptors, false);
int total = (int)objectKeypoints.size();
if (settings.useFLANN) {
indicesMat = new Mat(total, 2, CV_32SC1);
distsMat = new Mat(total, 2, CV_32FC1);
flannIndex = new Index();
indexParams = new LshIndexParams(12, 20, 2); // using LSH Hamming distance
searchParams = new SearchParams(64, 0, true); // maximum number of leafs checked
searchParams.deallocate(false); // for some reason FLANN seems to do it for us
}
pt1 = new Mat(total, 1, CV_32FC2);
pt2 = new Mat(total, 1, CV_32FC2);
mask = new Mat(total, 1, CV_8UC1);
H = new Mat(3, 3, CV_64FC1);
ptpairs = new ArrayList(2*objectDescriptors.rows());
logger.info(total + " object descriptors");
}
static final Logger logger = Logger.getLogger(ObjectFinder.class.getName());
KeyPointVector objectKeypoints = null, imageKeypoints = null;
Mat objectDescriptors = null, imageDescriptors = null;
Mat indicesMat, distsMat;
Index flannIndex = null;
IndexParams indexParams = null;
SearchParams searchParams = null;
Mat pt1 = null, pt2 = null, mask = null, H = null;
ArrayList ptpairs = null;
public double[] find(IplImage image) {
if (objectDescriptors.rows() < settings.getMatchesMin()) {
return null;
}
imageKeypoints = new KeyPointVector();
imageDescriptors = new Mat();
settings.detector.detectAndCompute(cvarrToMat(image),
new Mat(), imageKeypoints, imageDescriptors, false);
if (imageDescriptors.rows() < settings.getMatchesMin()) {
return null;
}
int total = (int)imageKeypoints.size();
logger.info(total + " image descriptors");
int w = settings.objectImage.width();
int h = settings.objectImage.height();
double[] srcCorners = {0, 0, w, 0, w, h, 0, h};
double[] dstCorners = locatePlanarObject(objectKeypoints, objectDescriptors,
imageKeypoints, imageDescriptors, srcCorners);
return dstCorners;
}
static final int[] bits = new int[256];
static {
for (int i = 0; i < bits.length; i++) {
for (int j = i; j != 0; j >>= 1) {
bits[i] += j & 0x1;
}
}
}
int compareDescriptors(ByteBuffer d1, ByteBuffer d2, int best) {
int totalCost = 0;
assert d1.limit() - d1.position() == d2.limit() - d2.position();
while (d1.position() < d1.limit()) {
totalCost += bits[(d1.get() ^ d2.get()) & 0xFF];
if (totalCost > best)
break;
}
return totalCost;
}
int naiveNearestNeighbor(ByteBuffer vec, ByteBuffer modelDescriptors) {
int neighbor = -1;
int d, dist1 = Integer.MAX_VALUE, dist2 = Integer.MAX_VALUE;
int size = vec.limit() - vec.position();
for (int i = 0; i * size < modelDescriptors.capacity(); i++) {
ByteBuffer mvec = (ByteBuffer)modelDescriptors.position(i * size).limit((i + 1) * size);
d = compareDescriptors((ByteBuffer)vec.reset(), mvec, dist2);
if (d < dist1) {
dist2 = dist1;
dist1 = d;
neighbor = i;
} else if (d < dist2) {
dist2 = d;
}
}
if (dist1 < settings.distanceThreshold*dist2)
return neighbor;
return -1;
}
void findPairs(Mat objectDescriptors, Mat imageDescriptors) {
int size = imageDescriptors.cols();
ByteBuffer objectBuf = objectDescriptors.createBuffer();
ByteBuffer imageBuf = imageDescriptors.createBuffer();
for (int i = 0; i * size < objectBuf.capacity(); i++) {
ByteBuffer descriptor = (ByteBuffer)objectBuf.position(i * size).limit((i + 1) * size).mark();
int nearestNeighbor = naiveNearestNeighbor(descriptor, imageBuf);
if (nearestNeighbor >= 0) {
ptpairs.add(i);
ptpairs.add(nearestNeighbor);
}
}
}
void flannFindPairs(Mat objectDescriptors, Mat imageDescriptors) {
int length = objectDescriptors.rows();
// find nearest neighbors using FLANN
flannIndex.build(imageDescriptors, indexParams, FLANN_DIST_HAMMING);
flannIndex.knnSearch(objectDescriptors, indicesMat, distsMat, 2, searchParams);
IntBuffer indicesBuf = indicesMat.createBuffer();
IntBuffer distsBuf = distsMat.createBuffer();
for (int i = 0; i < length; i++) {
if (distsBuf.get(2*i) < settings.distanceThreshold*distsBuf.get(2*i+1)) {
ptpairs.add(i);
ptpairs.add(indicesBuf.get(2*i));
}
}
}
/** a rough implementation for object location */
double[] locatePlanarObject(KeyPointVector objectKeypoints, Mat objectDescriptors,
KeyPointVector imageKeypoints, Mat imageDescriptors, double[] srcCorners) {
ptpairs.clear();
if (settings.useFLANN) {
flannFindPairs(objectDescriptors, imageDescriptors);
} else {
findPairs(objectDescriptors, imageDescriptors);
}
int n = ptpairs.size()/2;
logger.info(n + " matching pairs found");
if (n < settings.matchesMin) {
return null;
}
pt1 .resize(n);
pt2 .resize(n);
mask.resize(n);
FloatBuffer pt1Idx = pt1.createBuffer();
FloatBuffer pt2Idx = pt2.createBuffer();
for (int i = 0; i < n; i++) {
Point2f p1 = objectKeypoints.get(ptpairs.get(2*i)).pt();
pt1Idx.put(2*i, p1.x()); pt1Idx.put(2*i+1, p1.y());
Point2f p2 = imageKeypoints.get(ptpairs.get(2*i+1)).pt();
pt2Idx.put(2*i, p2.x()); pt2Idx.put(2*i+1, p2.y());
}
H = findHomography(pt1, pt2, CV_RANSAC, settings.ransacReprojThreshold, mask, 2000, 0.995);
if (H.empty() || countNonZero(mask) < settings.matchesMin) {
return null;
}
double[] h = (double[])H.createIndexer(false).array();
double[] dstCorners = new double[srcCorners.length];
for(int i = 0; i < srcCorners.length/2; i++) {
double x = srcCorners[2*i], y = srcCorners[2*i + 1];
double Z = 1/(h[6]*x + h[7]*y + h[8]);
double X = (h[0]*x + h[1]*y + h[2])*Z;
double Y = (h[3]*x + h[4]*y + h[5])*Z;
dstCorners[2*i ] = X;
dstCorners[2*i + 1] = Y;
}
return dstCorners;
}
public static void main(String[] args) throws Exception {
// Logger.getLogger("org.bytedeco.javacv").setLevel(Level.OFF);
String objectFilename = args.length == 2 ? args[0] : "/usr/local/share/OpenCV/samples/c/box.png";
String sceneFilename = args.length == 2 ? args[1] : "/usr/local/share/OpenCV/samples/c/box_in_scene.png";
IplImage object = cvLoadImage(objectFilename, IMREAD_GRAYSCALE);
IplImage image = cvLoadImage(sceneFilename, IMREAD_GRAYSCALE);
if (object == null || image == null) {
System.err.println("Can not load " + objectFilename + " and/or " + sceneFilename);
System.exit(-1);
}
IplImage objectColor = IplImage.create(object.width(), object.height(), 8, 3);
cvCvtColor(object, objectColor, CV_GRAY2BGR);
IplImage correspond = IplImage.create(image.width(), object.height()+ image.height(), 8, 1);
cvSetImageROI(correspond, cvRect(0, 0, object.width(), object.height()));
cvCopy(object, correspond);
cvSetImageROI(correspond, cvRect(0, object.height(), correspond.width(), correspond.height()));
cvCopy(image, correspond);
cvResetImageROI(correspond);
ObjectFinder.Settings settings = new ObjectFinder.Settings();
settings.objectImage = object;
settings.useFLANN = true;
settings.ransacReprojThreshold = 5;
ObjectFinder finder = new ObjectFinder(settings);
long start = System.currentTimeMillis();
double[] dst_corners = finder.find(image);
System.out.println("Finding time = " + (System.currentTimeMillis() - start) + " ms");
if (dst_corners != null) {
for (int i = 0; i < 4; i++) {
int j = (i+1)%4;
int x1 = (int)Math.round(dst_corners[2*i ]);
int y1 = (int)Math.round(dst_corners[2*i + 1]);
int x2 = (int)Math.round(dst_corners[2*j ]);
int y2 = (int)Math.round(dst_corners[2*j + 1]);
line(cvarrToMat(correspond), new Point(x1, y1 + object.height()),
new Point(x2, y2 + object.height()),
Scalar.WHITE, 1, 8, 0);
}
}
for (int i = 0; i < finder.ptpairs.size(); i += 2) {
Point2f pt1 = finder.objectKeypoints.get(finder.ptpairs.get(i)).pt();
Point2f pt2 = finder.imageKeypoints.get(finder.ptpairs.get(i + 1)).pt();
line(cvarrToMat(correspond), new Point(Math.round(pt1.x()), Math.round(pt1.y())),
new Point(Math.round(pt2.x()), Math.round(pt2.y() + object.height())),
Scalar.WHITE, 1, 8, 0);
}
CanvasFrame objectFrame = new CanvasFrame("Object");
CanvasFrame correspondFrame = new CanvasFrame("Object Correspond");
OpenCVFrameConverter converter = new OpenCVFrameConverter.ToIplImage();
correspondFrame.showImage(converter.convert(correspond));
for (int i = 0; i < finder.objectKeypoints.size(); i++) {
KeyPoint r = finder.objectKeypoints.get(i);
Point center = new Point(Math.round(r.pt().x()), Math.round(r.pt().y()));
int radius = Math.round(r.size() / 2);
circle(cvarrToMat(objectColor), center, radius, Scalar.RED, 1, 8, 0);
}
objectFrame.showImage(converter.convert(objectColor));
objectFrame.waitKey();
objectFrame.dispose();
correspondFrame.dispose();
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/OpenCVFrameConverter.java
================================================
/*
* Copyright (C) 2015-2021 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.nio.Buffer;
import java.nio.ByteBuffer;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.javacpp.Pointer;
import org.bytedeco.opencv.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_core.*;
/**
* A utility class to map data between {@link Frame} and {@link IplImage} or {@link Mat}.
* Since this is an abstract class, one must choose between two concrete classes:
* {@link ToIplImage} or {@link ToMat}. {@link ToOrgOpenCvCoreMat} is also available to
* do the same with {@link org.opencv.core.Mat} from the official Java API of OpenCV.
*
* @author Samuel Audet
*/
public abstract class OpenCVFrameConverter extends FrameConverter {
static { Loader.load(org.bytedeco.opencv.global.opencv_core.class); }
IplImage img;
Mat mat;
org.opencv.core.Mat orgOpenCvCoreMat;
public static class ToIplImage extends OpenCVFrameConverter {
@Override public Frame convert(IplImage img) { return super.convert(img); }
@Override public IplImage convert(Frame frame) { return convertToIplImage(frame); }
}
public static class ToMat extends OpenCVFrameConverter {
@Override public Frame convert(Mat mat) { return super.convert(mat); }
@Override public Mat convert(Frame frame) { return convertToMat(frame); }
}
public static class ToOrgOpenCvCoreMat extends OpenCVFrameConverter {
@Override public Frame convert(org.opencv.core.Mat mat) { return super.convert(mat); }
@Override public org.opencv.core.Mat convert(Frame frame) { return convertToOrgOpenCvCoreMat(frame); }
}
public static int getFrameDepth(int depth) {
switch (depth) {
case IPL_DEPTH_8U: case CV_8U: return Frame.DEPTH_UBYTE;
case IPL_DEPTH_8S: case CV_8S: return Frame.DEPTH_BYTE;
case IPL_DEPTH_16U: case CV_16U: return Frame.DEPTH_USHORT;
case IPL_DEPTH_16S: case CV_16S: return Frame.DEPTH_SHORT;
case IPL_DEPTH_32F: case CV_32F: return Frame.DEPTH_FLOAT;
case IPL_DEPTH_32S: case CV_32S: return Frame.DEPTH_INT;
case IPL_DEPTH_64F: case CV_64F: return Frame.DEPTH_DOUBLE;
default: return -1;
}
}
public static int getIplImageDepth(int depth) {
switch (depth) {
case Frame.DEPTH_UBYTE: return IPL_DEPTH_8U;
case Frame.DEPTH_BYTE: return IPL_DEPTH_8S;
case Frame.DEPTH_USHORT: return IPL_DEPTH_16U;
case Frame.DEPTH_SHORT: return IPL_DEPTH_16S;
case Frame.DEPTH_FLOAT: return IPL_DEPTH_32F;
case Frame.DEPTH_INT: return IPL_DEPTH_32S;
case Frame.DEPTH_DOUBLE: return IPL_DEPTH_64F;
default: return -1;
}
}
static boolean isEqual(Frame frame, IplImage img) {
return img != null && frame != null && frame.image != null && frame.image.length > 0
&& frame.imageWidth == img.width() && frame.imageHeight == img.height()
&& frame.imageChannels == img.nChannels() && getIplImageDepth(frame.imageDepth) == img.depth()
&& new Pointer(frame.image[0].position(0)).address() == img.imageData().address()
&& frame.imageStride * Math.abs(frame.imageDepth) / 8 == img.widthStep();
}
public IplImage convertToIplImage(Frame frame) {
if (frame == null || frame.image == null) {
return null;
} else if (frame.opaque instanceof IplImage) {
return (IplImage)frame.opaque;
} else if (!isEqual(frame, img)) {
int depth = getIplImageDepth(frame.imageDepth);
if (img != null) {
img.releaseReference();
}
img = depth < 0 ? null : (IplImage)IplImage.create(frame.imageWidth, frame.imageHeight, depth, frame.imageChannels, new Pointer(frame.image[0].position(0)))
.widthStep(frame.imageStride * Math.abs(frame.imageDepth) / 8)
.imageSize(frame.image[0].capacity() * Math.abs(frame.imageDepth) / 8).retainReference();
}
return img;
}
public Frame convert(IplImage img) {
if (img == null) {
return null;
} else if (!isEqual(frame, img)) {
frame = new Frame();
frame.imageWidth = img.width();
frame.imageHeight = img.height();
frame.imageDepth = getFrameDepth(img.depth());
frame.imageChannels = img.nChannels();
frame.imageStride = img.widthStep() * 8 / Math.abs(frame.imageDepth);
frame.image = new Buffer[] { img.createBuffer() };
}
frame.opaque = img;
return frame;
}
public static int getMatDepth(int depth) {
switch (depth) {
case Frame.DEPTH_UBYTE: return CV_8U;
case Frame.DEPTH_BYTE: return CV_8S;
case Frame.DEPTH_USHORT: return CV_16U;
case Frame.DEPTH_SHORT: return CV_16S;
case Frame.DEPTH_FLOAT: return CV_32F;
case Frame.DEPTH_INT: return CV_32S;
case Frame.DEPTH_DOUBLE: return CV_64F;
default: return -1;
}
}
static boolean isEqual(Frame frame, Mat mat) {
return mat != null && frame != null && frame.image != null && frame.image.length > 0
&& frame.imageWidth == mat.cols() && frame.imageHeight == mat.rows()
&& frame.imageChannels == mat.channels() && getMatDepth(frame.imageDepth) == mat.depth()
&& new Pointer(frame.image[0].position(0)).address() == mat.data().address()
&& frame.imageStride * Math.abs(frame.imageDepth) / 8 == (int)mat.step();
}
public Mat convertToMat(Frame frame) {
if (frame == null || frame.image == null) {
return null;
} else if (frame.opaque instanceof Mat) {
return (Mat)frame.opaque;
} else if (!isEqual(frame, mat)) {
int depth = getMatDepth(frame.imageDepth);
if (mat != null) {
mat.releaseReference();
}
mat = depth < 0 ? null : (Mat)new Mat(frame.imageHeight, frame.imageWidth, CV_MAKETYPE(depth, frame.imageChannels),
new Pointer(frame.image[0].position(0)), frame.imageStride * Math.abs(frame.imageDepth) / 8).retainReference();
}
return mat;
}
public Frame convert(Mat mat) {
if (mat == null) {
return null;
} else if (!isEqual(frame, mat)) {
frame = new Frame();
frame.imageWidth = mat.cols();
frame.imageHeight = mat.rows();
frame.imageDepth = getFrameDepth(mat.depth());
frame.imageChannels = mat.channels();
frame.imageStride = (int)mat.step() * 8 / Math.abs(frame.imageDepth);
frame.image = new Buffer[] { mat.createBuffer() };
}
frame.opaque = mat;
return frame;
}
static boolean isEqual(Frame frame, org.opencv.core.Mat mat) {
return mat != null && frame != null && frame.image != null && frame.image.length > 0
&& frame.imageWidth == mat.cols() && frame.imageHeight == mat.rows()
&& frame.imageChannels == mat.channels() && getMatDepth(frame.imageDepth) == mat.depth()
&& new Pointer(frame.image[0].position(0)).address() == mat.dataAddr();
}
public org.opencv.core.Mat convertToOrgOpenCvCoreMat(Frame frame) {
if (frame == null || frame.image == null) {
return null;
} else if (frame.opaque instanceof org.opencv.core.Mat) {
return (org.opencv.core.Mat)frame.opaque;
} else if (!isEqual(frame, mat)) {
int depth = getMatDepth(frame.imageDepth);
orgOpenCvCoreMat = depth < 0 ? null : new org.opencv.core.Mat(frame.imageHeight, frame.imageWidth,
CV_MAKETYPE(depth, frame.imageChannels), new BytePointer(new Pointer(frame.image[0].position(0)))
.capacity(frame.image[0].capacity() * Math.abs(frame.imageDepth) / 8).asByteBuffer(),
frame.imageStride * Math.abs(frame.imageDepth) / 8);
}
return orgOpenCvCoreMat;
}
public Frame convert(final org.opencv.core.Mat mat) {
if (mat == null) {
return null;
} else if (!isEqual(frame, mat)) {
frame = new Frame();
frame.imageWidth = mat.cols();
frame.imageHeight = mat.rows();
frame.imageDepth = getFrameDepth(mat.depth());
frame.imageChannels = mat.channels();
frame.imageStride = (int)mat.step1();
ByteBuffer byteBuffer = new BytePointer() { { address = mat.dataAddr(); } }.capacity(mat.rows() * mat.step1() * mat.elemSize1()).asByteBuffer();
switch (mat.depth()) {
case CV_8U:
case CV_8S:
frame.image = new Buffer[] { byteBuffer };
break;
case CV_16U:
case CV_16S:
frame.image = new Buffer[] { byteBuffer.asShortBuffer() };
break;
case CV_32F:
frame.image = new Buffer[] { byteBuffer.asFloatBuffer() };
break;
case CV_32S:
frame.image = new Buffer[] { byteBuffer.asIntBuffer() };
break;
case CV_64F:
frame.image = new Buffer[] { byteBuffer.asDoubleBuffer() };
break;
default:
frame.image = null;
break;
}
}
frame.opaque = mat;
return frame;
}
@Override public void close() {
super.close();
if (img != null) {
img.releaseReference();
img = null;
}
if (mat != null) {
mat.releaseReference();
mat = null;
}
if (orgOpenCvCoreMat != null) {
orgOpenCvCoreMat.release();
orgOpenCvCoreMat = null;
}
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/OpenCVFrameGrabber.java
================================================
/*
* Copyright (C) 2009-2019 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.io.File;
import java.util.Map;
import java.util.Map.Entry;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import org.bytedeco.opencv.opencv_videoio.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_videoio.*;
/**
*
* @author Samuel Audet
* @author Lloyd (github.com/lloydmeta)
*/
public class OpenCVFrameGrabber extends FrameGrabber {
public static String[] getDeviceDescriptions() throws Exception {
tryLoad();
throw new UnsupportedOperationException("Device enumeration not support by OpenCV.");
}
public static OpenCVFrameGrabber createDefault(File deviceFile) throws Exception { return new OpenCVFrameGrabber(deviceFile); }
public static OpenCVFrameGrabber createDefault(String devicePath) throws Exception { return new OpenCVFrameGrabber(devicePath); }
public static OpenCVFrameGrabber createDefault(int deviceNumber) throws Exception { return new OpenCVFrameGrabber(deviceNumber); }
private static Exception loadingException = null;
public static void tryLoad() throws Exception {
if (loadingException != null) {
throw loadingException;
} else {
try {
Loader.load(org.bytedeco.opencv.global.opencv_highgui.class);
} catch (Throwable t) {
throw loadingException = new Exception("Failed to load " + OpenCVFrameGrabber.class, t);
}
}
}
public OpenCVFrameGrabber(int deviceNumber) {
this.deviceNumber = deviceNumber;
}
public OpenCVFrameGrabber(File file) {
this(file.getAbsolutePath());
}
public OpenCVFrameGrabber(File file, int apiPreference) {
this(file.getAbsolutePath(), apiPreference);
}
public OpenCVFrameGrabber(String filename) {
this.filename = filename;
}
public OpenCVFrameGrabber(String filename, int apiPreference) {
this.filename = filename;
this.apiPreference = apiPreference;
}
public void release() throws Exception {
stop();
}
@Override protected void finalize() throws Throwable {
super.finalize();
release();
}
private int deviceNumber = 0;
private String filename = null;
private int apiPreference = 0;
private VideoCapture capture = null;
private Mat returnMatrix = null;
private final OpenCVFrameConverter converter = new OpenCVFrameConverter.ToMat();
private final Mat mat = new Mat();
@Override public double getGamma() {
// default to a gamma of 2.2 for cheap Webcams, DV cameras, etc.
if (gamma == 0.0) {
return 2.2;
} else {
return gamma;
}
}
@Override public String getFormat() {
if (capture == null) {
return super.getFormat();
} else {
int fourcc = (int)capture.get(CAP_PROP_FOURCC);
return "" + (char)( fourcc & 0xFF) +
(char)((fourcc >> 8) & 0xFF) +
(char)((fourcc >> 16) & 0xFF) +
(char)((fourcc >> 24) & 0xFF);
}
}
@Override public int getImageWidth() {
if (returnMatrix != null) {
return returnMatrix.cols();
} else {
return capture == null ? super.getImageWidth() : (int)capture.get(CAP_PROP_FRAME_WIDTH);
}
}
@Override public int getImageHeight() {
if (returnMatrix != null) {
return returnMatrix.rows();
} else {
return capture == null ? super.getImageHeight() : (int)capture.get(CAP_PROP_FRAME_HEIGHT);
}
}
@Override public int getPixelFormat() {
return capture == null ? super.getPixelFormat() : (int)capture.get(CAP_PROP_CONVERT_RGB);
}
@Override public double getFrameRate() {
return capture == null ? super.getFrameRate() : (int)capture.get(CAP_PROP_FPS);
}
@Override public void setImageMode(ImageMode imageMode) {
if (imageMode != this.imageMode) {
returnMatrix = null;
}
super.setImageMode(imageMode);
}
@Override public int getFrameNumber() {
return capture == null ? super.getFrameNumber() :
(int)capture.get(CAP_PROP_POS_FRAMES);
}
@Override public void setFrameNumber(int frameNumber) throws Exception {
if (capture == null) {
super.setFrameNumber(frameNumber);
} else {
if (!capture.set(CAP_PROP_POS_FRAMES, frameNumber)) {
throw new Exception("set() Error: Could not set CAP_PROP_POS_FRAMES to " + frameNumber + ".");
}
}
}
@Override public long getTimestamp() {
return capture == null ? super.getTimestamp() :
Math.round(capture.get(CAP_PROP_POS_MSEC)*1000);
}
@Override public void setTimestamp(long timestamp) throws Exception {
if (capture == null) {
super.setTimestamp(timestamp);
} else {
if (!capture.set(CAP_PROP_POS_MSEC, timestamp/1000.0)) {
throw new Exception("set() Error: Could not set CAP_PROP_POS_MSEC to " + timestamp/1000.0 + ".");
}
}
}
@Override public int getLengthInFrames() {
return capture == null ? super.getLengthInFrames() :
(int)capture.get(CAP_PROP_FRAME_COUNT);
}
@Override public long getLengthInTime() {
return Math.round(getLengthInFrames() * 1000000L / getFrameRate());
}
public double getOption(int propId) {
if (capture != null) {
return capture.get(propId);
}
return Double.parseDouble(options.get(Integer.toString(propId)));
}
/**
*
* @param propId Property ID, look at opencv_videoio for possible values
* @param value
*/
public void setOption(int propId, double value) {
options.put(Integer.toString(propId), Double.toString(value));
if (capture != null) {
capture.set(propId, value);
}
}
public void start() throws Exception {
if (filename != null && filename.length() > 0) {
if (apiPreference > 0) {
capture = new VideoCapture(filename, apiPreference);
} else {
capture = new VideoCapture(filename);
}
} else {
capture = new VideoCapture(deviceNumber);
}
if (format != null && format.length() >= 4) {
format = format.toUpperCase();
byte cc0 = (byte)format.charAt(0);
byte cc1 = (byte)format.charAt(1);
byte cc2 = (byte)format.charAt(2);
byte cc3 = (byte)format.charAt(3);
capture.set(CAP_PROP_FOURCC, VideoWriter.fourcc(cc0, cc1, cc2, cc3));
}
if (imageWidth > 0) {
if (!capture.set(CAP_PROP_FRAME_WIDTH, imageWidth)) {
capture.set(CAP_PROP_FRAME_WIDTH, imageWidth);
}
}
if (imageHeight > 0) {
if (!capture.set(CAP_PROP_FRAME_HEIGHT, imageHeight)) {
capture.set(CAP_PROP_FRAME_HEIGHT, imageHeight);
}
}
if (frameRate > 0) {
capture.set(CAP_PROP_FPS, frameRate);
}
if (bpp > 0) {
capture.set(CAP_PROP_FORMAT, bpp); // ??
}
if (imageMode == ImageMode.RAW) {
capture.set(CAP_PROP_CONVERT_RGB, 0);
}
for (Entry e : options.entrySet()) {
capture.set(Integer.parseInt(e.getKey()), Double.parseDouble(e.getValue()));
}
Mat mat = new Mat();
try {
// Before retrieve() starts returning something else then null
// QTKit sometimes requires some "warm-up" time for some reason...
// The first frame on Linux is sometimes null as well,
// so it's probably a good idea to run this for all platforms... ?
int count = 0;
while (count++ < 100 && !capture.read(mat)) {
Thread.sleep(100);
}
} catch (InterruptedException ex) {
// reset interrupt to be nice
Thread.currentThread().interrupt();
}
if (!capture.read(mat)) {
throw new Exception("read() Error: Could not read frame in start().");
}
if (!triggerMode) {
if (!capture.grab()) {
throw new Exception("grab() Error: Could not grab frame. (Has start() been called?)");
}
}
}
public void stop() throws Exception {
if (capture != null) {
capture.release();
capture = null;
}
}
public void trigger() throws Exception {
Mat mat = new Mat();
for (int i = 0; i < numBuffers+1; i++) {
capture.read(mat);
}
if (!capture.grab()) {
throw new Exception("grab() Error: Could not grab frame. (Has start() been called?)");
}
}
public Frame grab() throws Exception {
if (!capture.retrieve(mat)) {
throw new Exception("retrieve() Error: Could not retrieve frame. (Has start() been called?)");
}
if (!triggerMode) {
if (!capture.grab()) {
throw new Exception("grab() Error: Could not grab frame. (Has start() been called?)");
}
}
if (imageMode == ImageMode.GRAY && mat.channels() > 1) {
if (returnMatrix == null) {
returnMatrix = new Mat(mat.rows(), mat.cols(), mat.depth(), 1);
}
cvtColor(mat, returnMatrix, CV_BGR2GRAY);
} else if (imageMode == ImageMode.COLOR && mat.channels() == 1) {
if (returnMatrix == null) {
returnMatrix = new Mat(mat.rows(), mat.cols(), mat.depth(), 3);
}
cvtColor(mat, returnMatrix, CV_GRAY2BGR);
} else {
returnMatrix = mat;
}
return converter.convert(returnMatrix);
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/OpenCVFrameRecorder.java
================================================
/*
* Copyright (C) 2009-2019 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.io.File;
import java.util.Map;
import java.util.Map.Entry;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_videoio.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_videoio.*;
/**
*
* @author Samuel Audet
*/
public class OpenCVFrameRecorder extends FrameRecorder {
public static OpenCVFrameRecorder createDefault(File f, int w, int h) throws Exception { return new OpenCVFrameRecorder(f, w, h); }
public static OpenCVFrameRecorder createDefault(String f, int w, int h) throws Exception { return new OpenCVFrameRecorder(f, w, h); }
private static Exception loadingException = null;
public static void tryLoad() throws Exception {
if (loadingException != null) {
throw loadingException;
} else {
try {
Loader.load(org.bytedeco.opencv.global.opencv_highgui.class);
} catch (Throwable t) {
throw loadingException = new Exception("Failed to load " + OpenCVFrameRecorder.class, t);
}
}
}
public OpenCVFrameRecorder(File file, int imageWidth, int imageHeight) {
this(file.getAbsolutePath(), imageWidth, imageHeight);
}
public OpenCVFrameRecorder(String filename, int imageWidth, int imageHeight) {
this.filename = filename;
this.imageWidth = imageWidth;
this.imageHeight = imageHeight;
this.pixelFormat = 1;
// this.videoCodec = windows ? CV_FOURCC_PROMPT : CV_FOURCC_DEFAULT;
this.videoCodec = windows ? -1 : VideoWriter.fourcc((byte)'I', (byte)'Y', (byte)'U', (byte)'V');
this.frameRate = 30;
}
public void release() throws Exception {
if (writer != null) {
writer.release();
writer = null;
}
}
@Override protected void finalize() throws Throwable {
super.finalize();
release();
}
private static final boolean windows = Loader.getPlatform().startsWith("windows");
private String filename;
private VideoWriter writer = null;
private OpenCVFrameConverter.ToMat converter = new OpenCVFrameConverter.ToMat();
public double getOption(int propId) {
if (writer != null) {
return writer.get(propId);
}
return Double.parseDouble(options.get(Integer.toString(propId)));
}
/**
*
* @param propId Property ID, look at opencv_videoio for possible values
* @param value
*/
public void setOption(int propId, double value) {
options.put(Integer.toString(propId), Double.toString(value));
if (writer != null) {
writer.set(propId, value);
}
}
public void start() throws Exception {
writer = new VideoWriter(filename, fourCCCodec(), frameRate, new Size(imageWidth, imageHeight), isColour());
for (Entry e : options.entrySet()) {
writer.set(Integer.parseInt(e.getKey()), Double.parseDouble(e.getValue()));
}
}
/**
* Pixel format is an int and maps to colour if != 0, greyscale otherwise.
*/
private boolean isColour() {
return pixelFormat != 0;
}
/**
* VideoCodec in JavaCV jargon is the same as FourCC code in OpenCV speak
*/
private int fourCCCodec() {
return videoCodec;
}
public void flush() throws Exception {
}
public void stop() throws Exception {
release();
}
public void record(Frame frame) throws Exception {
Mat mat = converter.convert(frame);
if (writer != null) {
writer.write(mat);
} else {
throw new Exception("Cannot record: There is no writer (Has start() been called?)");
}
frame.keyFrame = true;
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/OpenKinect2FrameGrabber.java
================================================
/*
* Copyright (C) 2014 Jeremy Laviole, Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.io.File;
import java.nio.ByteOrder;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.javacpp.Pointer;
import org.bytedeco.libfreenect2.*;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.libfreenect2.global.freenect2.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
*
* @author Jeremy Laviole
*/
public class OpenKinect2FrameGrabber extends FrameGrabber {
public static String[] getDeviceDescriptions() throws FrameGrabber.Exception {
tryLoad();
String[] desc = new String[freenect2Context.enumerateDevices()];
for (int i = 0; i < desc.length; i++) {
desc[i] = freenect2Context.getDeviceSerialNumber(i).getString();
}
return desc;
}
public static int DEFAULT_DEPTH_WIDTH = 640;
public static int DEFAULT_DEPTH_HEIGHT = 480;
public static int DEFAULT_COLOR_WIDTH = 640;
public static int DEFAULT_COLOR_HEIGHT = 480;
private ByteOrder byteOrder = ByteOrder.BIG_ENDIAN;
private int depthImageWidth = DEFAULT_DEPTH_WIDTH;
private int depthImageHeight = DEFAULT_DEPTH_HEIGHT;
private int depthFrameRate = 60;
private int IRImageWidth = DEFAULT_DEPTH_WIDTH;
private int IRImageHeight = DEFAULT_DEPTH_HEIGHT;
private int IRFrameRate = 60;
private SyncMultiFrameListener frameListener;
public ByteOrder getByteOrder() {
return byteOrder;
}
public void setByteOrder(ByteOrder byteOrder) {
this.byteOrder = byteOrder;
}
public static OpenKinect2FrameGrabber createDefault(int deviceNumber) throws FrameGrabber.Exception {
return new OpenKinect2FrameGrabber(deviceNumber);
}
public static OpenKinect2FrameGrabber createDefault(File deviceFile) throws Exception {
throw new Exception(OpenKinect2FrameGrabber.class + " does not support File devices.");
}
public static OpenKinect2FrameGrabber createDefault(String devicePath) throws Exception {
throw new Exception(OpenKinect2FrameGrabber.class + " does not support path.");
}
private static FrameGrabber.Exception loadingException = null;
private static Freenect2 freenect2Context = null;
public static void tryLoad() throws FrameGrabber.Exception {
if (loadingException != null) {
loadingException.printStackTrace();
throw loadingException;
} else {
try {
if (freenect2Context != null) {
return;
}
Loader.load(org.bytedeco.libfreenect2.global.freenect2.class);
// Context is shared accross cameras.
freenect2Context = new Freenect2();
} catch (Throwable t) {
throw loadingException = new FrameGrabber.Exception("Failed to load " + OpenKinect2FrameGrabber.class, t);
}
}
}
private boolean colorEnabled = false;
private boolean depthEnabled = false;
private boolean IREnabled = false;
private int deviceNumber = 0;
private String serial = null;
private Freenect2Device device = null;
private int frameTypes = 0;
public OpenKinect2FrameGrabber(int deviceNumber) {
this.deviceNumber = deviceNumber;
}
public void enableColorStream() {
if (!colorEnabled) {
frameTypes |= org.bytedeco.libfreenect2.Frame.Color;
colorEnabled = true;
}
}
public void enableDepthStream() {
if (!depthEnabled) {
frameTypes |= org.bytedeco.libfreenect2.Frame.Depth;
depthEnabled = true;
}
}
public void enableIRStream() {
if (!IREnabled) {
frameTypes |= org.bytedeco.libfreenect2.Frame.Ir;
IREnabled = true;
}
}
public void release() throws FrameGrabber.Exception {
}
@Override
protected void finalize() throws Throwable {
super.finalize();
release();
}
@Override
public void start() throws FrameGrabber.Exception {
startDevice(null);
}
public void startDevice(PacketPipeline pipeline) throws FrameGrabber.Exception {
if (freenect2Context == null) {
try {
OpenKinect2FrameGrabber.tryLoad();
} catch (Exception e) {
System.out.println("Exception in the TryLoad !" + e);
e.printStackTrace();
}
}
if (freenect2Context == null) {
throw new Exception("FATAL error: OpenKinect2 camera: driver could not load.");
}
if (freenect2Context.enumerateDevices() == 0) {
throw new Exception("FATAL error: OpenKinect2: no device connected!");
}
device = null;
// pipeline = new CpuPacketPipeline();
// pipeline = new libfreenect2::OpenGLPacketPipeline();
// pipeline = new libfreenect2::OpenCLPacketPipeline(deviceId);
// pipeline = new libfreenect2::CudaPacketPipeline(deviceId);
serial = freenect2Context.getDeviceSerialNumber(deviceNumber).getString();
device = (pipeline != null) ? freenect2Context.openDevice(serial, pipeline) : freenect2Context.openDevice(serial);
frameListener = new SyncMultiFrameListener(frameTypes);
if (colorEnabled) {
device.setColorFrameListener(frameListener);
}
if (depthEnabled || IREnabled) {
device.setIrAndDepthFrameListener(frameListener);
}
rawVideoImage = IplImage.createHeader(1920, 1080, IPL_DEPTH_8U, 4);
device.start();
System.out.println("OpenKinect2 device started.");
System.out.println("Serial: " + device.getSerialNumber().getString());
System.out.println("Firmware: " + device.getFirmwareVersion().getString());
}
/**
*
* @throws Exception
*/
@Override
public void stop() throws FrameGrabber.Exception {
device.stop();
frameNumber = 0;
}
// private Pointer rawVideoImageData;
private IplImage rawVideoImage = null;
private IplImage videoImageRGBA = null;
private boolean hasFirstGoodColorImage = false;
private BytePointer videoBuffer = null;
protected void grabVideo() {
int iplDepth = IPL_DEPTH_8U;
org.bytedeco.libfreenect2.Frame rgb = frames.get(org.bytedeco.libfreenect2.Frame.Color);
int channels = (int) rgb.bytes_per_pixel();
int deviceWidth = (int) rgb.width();
int deviceHeight = (int) rgb.height();
BytePointer rawVideoImageData = rgb.data();
if (rawVideoImage == null) {
rawVideoImage = IplImage.createHeader(deviceWidth, deviceHeight, iplDepth, channels);
}
cvSetData(rawVideoImage, rawVideoImageData, deviceWidth * channels * iplDepth / 8);
if (videoImageRGBA == null) {
videoImageRGBA = rawVideoImage.clone();
}
cvCvtColor(rawVideoImage, videoImageRGBA, COLOR_BGRA2RGBA);
}
private IplImage rawIRImage = null;
protected void grabIR() {
/**
* 512x424 float. Range is [0.0, 65535.0].
*/
org.bytedeco.libfreenect2.Frame IRImage = frames.get(org.bytedeco.libfreenect2.Frame.Ir);
int channels = 1;
int iplDepth = IPL_DEPTH_32F;
int bpp = (int) IRImage.bytes_per_pixel();
int deviceWidth = (int) IRImage.width();
int deviceHeight = (int) IRImage.height();
Pointer rawIRData = IRImage.data();
if (rawIRImage == null) {
rawIRImage = IplImage.createHeader(deviceWidth, deviceHeight, iplDepth, channels);
}
cvSetData(rawIRImage, rawIRData, deviceWidth * channels * iplDepth / 8);
}
private IplImage rawDepthImage = null;
protected void grabDepth() {
/**
* 512x424 float also ?.
*/
org.bytedeco.libfreenect2.Frame depthImage = frames.get(org.bytedeco.libfreenect2.Frame.Depth);
int channels = 1;
int iplDepth = IPL_DEPTH_32F;
int bpp = (int) depthImage.bytes_per_pixel();
int deviceWidth = (int) depthImage.width();
int deviceHeight = (int) depthImage.height();
Pointer rawDepthData = depthImage.data();
if (rawDepthImage == null) {
rawDepthImage = IplImage.createHeader(deviceWidth, deviceHeight, iplDepth, channels);
}
cvSetData(rawDepthImage, rawDepthData, deviceWidth * channels * iplDepth / 8);
}
private FrameMap frames = new FrameMap();
/**
*
* @return null grabs all images, get them with grabColor, grabDepth, and
* grabIR instead.
* @throws org.bytedeco.javacv.FrameGrabber.Exception
*/
public Frame grab() throws Exception {
if (!frameListener.waitForNewFrame(frames, 10 * 1000)) // 10 seconds
{
System.out.println("Openkinect2: timeout!");
// TODO: throw exception
}
frameNumber++;
if (colorEnabled) {
grabVideo();
}
if (IREnabled) {
grabIR();
}
if (depthEnabled) {
grabDepth();
}
frameListener.release(frames);
return null;
}
public IplImage getVideoImage() {
return videoImageRGBA;
// return rawVideoImage;
}
public IplImage getIRImage() {
return rawIRImage;
}
public IplImage getDepthImage() {
return rawDepthImage;
}
@Override
public void trigger() throws Exception {
// device.wait_for_frames();
}
public int getDepthImageWidth() {
return depthImageWidth;
}
public void setDepthImageWidth(int depthImageWidth) {
this.depthImageWidth = depthImageWidth;
}
public int getDepthImageHeight() {
return depthImageHeight;
}
public void setDepthImageHeight(int depthImageHeight) {
this.depthImageHeight = depthImageHeight;
}
public int getIRImageWidth() {
return IRImageWidth;
}
public void setIRImageWidth(int IRImageWidth) {
this.IRImageWidth = IRImageWidth;
}
public int getIRImageHeight() {
return IRImageHeight;
}
public void setIRImageHeight(int IRImageHeight) {
this.IRImageHeight = IRImageHeight;
}
public int getDepthFrameRate() {
return depthFrameRate;
}
public void setDepthFrameRate(int frameRate) {
this.depthFrameRate = frameRate;
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/OpenKinectFrameGrabber.java
================================================
/*
* Copyright (C) 2011-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.io.File;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.ShortBuffer;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.Pointer;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.libfreenect.*;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.libfreenect.global.freenect.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
*
* @author Samuel Audet
*/
public class OpenKinectFrameGrabber extends FrameGrabber {
public static String[] getDeviceDescriptions() throws Exception {
tryLoad();
freenect_context ctx = new freenect_context(null);
int err = freenect_init(ctx, null);
if (err < 0) {
throw new Exception("freenect_init() Error " + err + ": Failed to init context.");
}
int count = freenect_num_devices(ctx);
if (count < 0) {
throw new Exception("freenect_num_devices() Error " + err + ": Failed to get number of devices.");
}
String[] descriptions = new String[count];
for (int i = 0; i < descriptions.length; i++) {
descriptions[i] = "Kinect #" + i;
}
err = freenect_shutdown(ctx);
if (err < 0) {
throw new Exception("freenect_shutdown() Error " + err + ": Failed to shutdown context.");
}
return descriptions;
}
public static OpenKinectFrameGrabber createDefault(File deviceFile) throws Exception { throw new Exception(OpenKinectFrameGrabber.class + " does not support device files."); }
public static OpenKinectFrameGrabber createDefault(String devicePath) throws Exception { throw new Exception(OpenKinectFrameGrabber.class + " does not support device paths."); }
public static OpenKinectFrameGrabber createDefault(int deviceNumber) throws Exception { return new OpenKinectFrameGrabber(deviceNumber); }
private static Exception loadingException = null;
public static void tryLoad() throws Exception {
if (loadingException != null) {
throw loadingException;
} else {
try {
Loader.load(org.bytedeco.libfreenect.global.freenect.class);
} catch (Throwable t) {
throw loadingException = new Exception("Failed to load " + OpenKinectFrameGrabber.class, t);
}
}
}
public OpenKinectFrameGrabber(int deviceNumber) {
this.deviceNumber = deviceNumber;
}
public void release() throws Exception {
stop();
}
@Override protected void finalize() throws Throwable {
super.finalize();
release();
}
private int deviceNumber = 0;
private boolean depth = false; // default to "video"
private BytePointer rawDepthImageData = new BytePointer((Pointer)null),
rawVideoImageData = new BytePointer((Pointer)null),
rawIRImageData = new BytePointer((Pointer)null);
private IplImage rawDepthImage = null, rawVideoImage = null, rawIRImage = null, returnImage = null;
private FrameConverter converter = new OpenCVFrameConverter.ToIplImage();
private int[] timestamp = { 0 };
private ByteOrder byteOrder = ByteOrder.BIG_ENDIAN;
private int depthFormat = -1;
private int videoFormat = -1;
public ByteOrder getByteOrder() {
return byteOrder;
}
public void setByteOrder(ByteOrder byteOrder) {
this.byteOrder = byteOrder;
}
public int getDepthFormat() {
return depthFormat;
}
public void setDepthFormat(int depthFormat) {
this.depthFormat = depthFormat;
}
public int getVideoFormat() {
return videoFormat;
}
public void setVideoFormat(int videoFormat) {
this.videoFormat = videoFormat;
}
@Override public double getGamma() {
// I guess a default gamma of 2.2 is reasonable...
if (gamma == 0.0) {
return 2.2;
} else {
return gamma;
}
}
@Override public void setImageMode(ImageMode imageMode) {
if (imageMode != this.imageMode) {
returnImage = null;
}
super.setImageMode(imageMode);
}
public void start() throws Exception {
depth = "depth".equalsIgnoreCase(format);
}
public void stop() throws Exception {
freenect_sync_stop();
}
public void trigger() throws Exception {
for (int i = 0; i < numBuffers+1; i++) {
if (depth) {
int fmt = depthFormat < 0 ? bpp : depthFormat; // default bpp == 0 == FREENECT_DEPTH_11BIT
int err = freenect_sync_get_depth(rawDepthImageData, timestamp, deviceNumber, fmt);
if (err != 0) {
throw new Exception("freenect_sync_get_depth() Error " + err + ": Failed to get depth synchronously.");
}
} else {
int fmt = videoFormat < 0 ? bpp : videoFormat; // default bpp == 0 == FREENECT_VIDEO_RGB
int err = freenect_sync_get_video(rawVideoImageData, timestamp, deviceNumber, fmt);
if (err != 0) {
throw new Exception("freenect_sync_get_video() Error " + err + ": Failed to get video synchronously.");
}
}
}
}
public IplImage grabDepth() throws Exception {
int fmt = depthFormat < 0 ? bpp : depthFormat; // default bpp == 0 == FREENECT_DEPTH_11BIT
int iplDepth = IPL_DEPTH_16U, channels = 1;
switch (fmt) {
case FREENECT_DEPTH_11BIT:
case FREENECT_DEPTH_REGISTERED:
case FREENECT_DEPTH_MM:
case FREENECT_DEPTH_10BIT: iplDepth = IPL_DEPTH_16U; channels = 1; break;
case FREENECT_DEPTH_11BIT_PACKED:
case FREENECT_DEPTH_10BIT_PACKED:
default: assert false;
}
int err = freenect_sync_get_depth(rawDepthImageData, timestamp, deviceNumber, fmt);
if (err != 0) {
throw new Exception("freenect_sync_get_depth() Error " + err + ": Failed to get depth synchronously.");
}
int w = 640, h = 480; // how to get the resolution ??
if (rawDepthImage == null || rawDepthImage.width() != w || rawDepthImage.height() != h) {
rawDepthImage = IplImage.createHeader(w, h, iplDepth, channels);
}
cvSetData(rawDepthImage, rawDepthImageData, w*channels*iplDepth/8);
if (iplDepth > 8 && !ByteOrder.nativeOrder().equals(byteOrder)) {
// ack, the camera's endianness doesn't correspond to our machine ...
// swap bytes of 16-bit images
ByteBuffer bb = rawDepthImage.getByteBuffer();
ShortBuffer in = bb.order(ByteOrder.BIG_ENDIAN ).asShortBuffer();
ShortBuffer out = bb.order(ByteOrder.LITTLE_ENDIAN).asShortBuffer();
out.put(in);
}
super.timestamp = timestamp[0];
return rawDepthImage;
}
public IplImage grabVideo() throws Exception {
int fmt = videoFormat < 0 ? bpp : videoFormat; // default bpp == 0 == FREENECT_VIDEO_RGB
int iplDepth = IPL_DEPTH_8U, channels = 3;
switch (fmt) {
case FREENECT_VIDEO_RGB: iplDepth = IPL_DEPTH_8U; channels = 3; break;
case FREENECT_VIDEO_BAYER:
case FREENECT_VIDEO_IR_8BIT: iplDepth = IPL_DEPTH_8U; channels = 1; break;
case FREENECT_VIDEO_IR_10BIT: iplDepth = IPL_DEPTH_16U; channels = 1; break;
case FREENECT_VIDEO_YUV_RGB: iplDepth = IPL_DEPTH_8U; channels = 3; break;
case FREENECT_VIDEO_YUV_RAW: iplDepth = IPL_DEPTH_8U; channels = 2; break;
case FREENECT_VIDEO_IR_10BIT_PACKED:
default: assert false;
}
int err = freenect_sync_get_video(rawVideoImageData, timestamp, deviceNumber, fmt);
if (err != 0) {
throw new Exception("freenect_sync_get_video() Error " + err + ": Failed to get video synchronously.");
}
int w = 640, h = 480; // how to get the resolution ??
if (rawVideoImage == null || rawVideoImage.width() != w || rawVideoImage.height() != h) {
rawVideoImage = IplImage.createHeader(w, h, iplDepth, channels);
}
cvSetData(rawVideoImage, rawVideoImageData, w*channels*iplDepth/8);
if (iplDepth > 8 && !ByteOrder.nativeOrder().equals(byteOrder)) {
// ack, the camera's endianness doesn't correspond to our machine ...
// swap bytes of 16-bit images
ByteBuffer bb = rawVideoImage.getByteBuffer();
ShortBuffer in = bb.order(ByteOrder.BIG_ENDIAN ).asShortBuffer();
ShortBuffer out = bb.order(ByteOrder.LITTLE_ENDIAN).asShortBuffer();
out.put(in);
}
if (channels == 3) {
cvCvtColor(rawVideoImage, rawVideoImage, CV_RGB2BGR);
}
super.timestamp = timestamp[0];
return rawVideoImage;
}
public IplImage grabIR() throws Exception {
int iplDepth = IPL_DEPTH_8U, channels = 1;
int err = freenect_sync_get_video(rawIRImageData, timestamp, deviceNumber, FREENECT_VIDEO_IR_8BIT);
if (err != 0) {
throw new Exception("freenect_sync_get_video() Error " + err + ": Failed to get video synchronously.");
}
int w = 640, h = 480; // how to get the resolution ??
if (rawIRImage == null || rawIRImage.width() != w || rawIRImage.height() != h) {
rawIRImage = IplImage.createHeader(w, h, iplDepth, channels);
}
cvSetData(rawIRImage, rawIRImageData, w*channels*iplDepth/8);
super.timestamp = timestamp[0];
return rawIRImage;
}
public Frame grab() throws Exception {
IplImage image = depth ? grabDepth() : grabVideo();
int w = image.width();
int h = image.height();
int iplDepth = image.depth();
int channels = image.nChannels();
if (imageMode == ImageMode.COLOR && channels == 1) {
if (returnImage == null) {
returnImage = IplImage.create(w, h, iplDepth, 3);
}
cvCvtColor(image, returnImage, CV_GRAY2BGR);
return converter.convert(returnImage);
} else if (imageMode == ImageMode.GRAY && channels == 3) {
if (returnImage == null) {
returnImage = IplImage.create(w, h, iplDepth, 1);
}
cvCvtColor(image, returnImage, CV_BGR2GRAY);
return converter.convert(returnImage);
} else {
return converter.convert(image);
}
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/PS3EyeFrameGrabber.java
================================================
/*
* Copyright (C) 2011-2012 Jiri Masa, Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import cl.eye.CLCamera;
import java.io.File;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/** Minimal Sony PS3 Eye camera grabber implementation.
*
* It allows grabbing of frames at higher speed than OpenCVFrameGrabber or VideoInputFrameGrabber.
* Underlying implementation of last two grabbers is limited to 30 FPS. PS3 allows grabbing
* at maximum speed of 75 FPS in VGA and 187 FPS in QVGA modes.
*
* This code was developed and tested with CLEyeMulticam.dll, version 1.2.0.1008. The dll library
* is part of Code Laboratories CL-Eye Platform SDK and is distributed as part of CLEyeMulticam
* Redistributable Dynamic Link Library. For license, download and installation see http://www.codelaboratories.com.
*
* The grab() method returns an internal instance of IplImage image with fresh camera frame. This returned image
* have to be considered "read only" and the caller needs to create it's own copy or clone that image.
* Calling of release() method for this image shall be avoided.
* Based on used resolution the image is in format either 640x480 or 320x240, IPL_DEPTH_8U, 4 channel (color) or 1 channel (gray).
* timestamp is set to actual value of System.nanoTime()/1000 obtained after return from the CL driver.
*
* Typical use case scenario:
* create new instance of PS3MiniGrabber
* set camera parameters
* start() grabber
* wait at least 2 frames
* grab() in loop
* stop() grabber
* release() internal resources
*
* Note:
* This code depends on the cl.eye.CLCamera class from Code Laboratories CL-Eye
* Platform SDK. It is suggested to download SDK and edit the sample file
* ....\cl\eye\CLCamera.java. A few references to processing.core.PApplet class
* shall be removed and the file recompiled. The tailored file is not included
* here namely because of unclear licence.
*
* @author jmasa, jmasa@cmail.cz
*
*/
public class PS3EyeFrameGrabber extends FrameGrabber {
public static String[] getDeviceDescriptions() throws Exception {
tryLoad();
String[] descriptions = new String[CLCamera.cameraCount()];
for (int i = 0; i < descriptions.length; i++) {
descriptions[i] = CLCamera.cameraUUID(i);
}
return descriptions;
}
public static PS3EyeFrameGrabber createDefault(File deviceFile) throws Exception { throw new Exception(PS3EyeFrameGrabber.class + " does not support device files."); }
public static PS3EyeFrameGrabber createDefault(String devicePath) throws Exception { throw new Exception(PS3EyeFrameGrabber.class + " does not support device paths."); }
public static PS3EyeFrameGrabber createDefault(int deviceNumber) throws Exception { return new PS3EyeFrameGrabber(deviceNumber); }
private static Exception loadingException = null;
public static void tryLoad() throws Exception {
if (loadingException != null) {
throw loadingException;
} else {
try {
CLCamera.IsLibraryLoaded();
} catch (Throwable t) {
throw loadingException = new Exception("Failed to load " + PS3EyeFrameGrabber.class, t);
}
}
}
CLCamera camera;
int cameraIndex = 0;
int[] ps3_frame = null; // buffer for PS3 camera frame data
byte[] ipl_frame = null; // buffer for RGB-3ch, not allocated unless grab_RGB3() is called
IplImage image_4ch = null;
IplImage image_1ch = null;
FrameConverter converter = new OpenCVFrameConverter.ToIplImage();
String stat; // status of PS3 camera handling - mostly for debugging
String uuid; // assigned camera unique key
// variables for trigger() implementation
//
protected enum Triggered {NO_TRIGGER, HAS_FRAME, NO_FRAME};
protected Triggered triggered = Triggered.NO_TRIGGER;
/** Default grabber, camera idx = 0, color mode, VGA resolution, 60 FPS frame rate.
*
*/
public PS3EyeFrameGrabber() throws Exception {
this(0);
}
/** Color mode, VGA resolution, 60 FPS frame rate.
* @param cameraIndex system wide camera index
*/
public PS3EyeFrameGrabber(int cameraIndex) throws Exception {
this(cameraIndex, 640, 480, 60);
}
public PS3EyeFrameGrabber(int cameraIndex, int imageWidth, int imageHeight, int framerate) throws Exception {
this(cameraIndex, 640, 480, 60, null);
}
/** Creates grabber, the caller can control basic image and grabbing parameters.
*
* @param cameraIndex - zero based index of used camera (OS system wide)
* @param imageWidth - width of image
* @param imageHeight - height of image
* @param framerate - frame rate - see CLCamera for allowed frame rates based on resolution
* @param applet - PApplet object required by CLCamera
* @throws Exception - if parameters don't follow CLCamera definition or camera is not created
*/
public PS3EyeFrameGrabber(int cameraIndex, int imageWidth, int imageHeight, int framerate, Object applet) throws Exception {
camera = null;
if (! CLCamera.IsLibraryLoaded()) {
throw new Exception("CLEye multicam dll not loaded");
}
this.camera = new CLCamera();
this.cameraIndex = cameraIndex;
stat = "created";
uuid = CLCamera.cameraUUID(cameraIndex);
if (((imageWidth == 640) && (imageHeight == 480)) ||
((imageWidth == 320) && (imageHeight == 240))) {
setImageWidth(imageWidth);
setImageHeight(imageHeight);
}
else throw new Exception("Only 640x480 or 320x240 images supported");
setImageMode(ImageMode.COLOR);
setFrameRate((double) framerate);
setTimeout(1 + 1000/framerate);
setBitsPerPixel(8);
setTriggerMode(false);
setNumBuffers(4);
}
/**
* @return system wide number of installed/detected Sony PS3 Eye cameras
*/
public static int getCameraCount() {
return CLCamera.cameraCount();
}
/** Ask the driver for all installed PS3 cameras. Resulting array is sorted in order of camera index.
* Its size is defined by CLCamera.cameraCount().
*
* @return array of camera unique uuids or null if there is no PS3 camera
*/
public static String[] listPS3Cameras() {
int no = getCameraCount();
String[] uuids;
if (no > 0) {
uuids = new String[no];
for (--no; no >=0; no--) { uuids[no] = CLCamera.cameraUUID(no); }
return uuids;
}
return null;
}
/** Make IplImage form raw int[] frame data
* Note: NO array size checks!!
*
* @param frame int[] image frame data
* @return internal IplImage set to frame
*/
public IplImage makeImage(int[] frame) {
image_4ch.getIntBuffer().put(ps3_frame);
return image_4ch;
}
/** Grab one frame and return it as int[] (in the internal camera format RGBA).
* Note: use makeImage() to create RGBA, 4-ch image
* @return frame as int[] without any processing or null if frame is not available
*/
public int[] grab_raw() {
if (camera.getCameraFrame(ps3_frame, timeout)) {
return ps3_frame;
}
else return null;
}
public void trigger() throws Exception {
for (int i = 0; i < numBuffers+1; i++) {
grab_raw();
}
if ((ps3_frame = grab_raw()) != null) {
triggered = Triggered.HAS_FRAME;
timestamp = System.nanoTime()/1000;
}
else
triggered = Triggered.NO_FRAME;
}
/** Grab and convert one frame, default timeout is (1 + 1000/framerate) [milliseconds].
* Every successful call returns an internal (preallocated) 640x480 or 320x240, IPL_DEPTH_8U, 4-channel image.
* The caller shall consider it "read only" and make a copy/clone of it before further processing.
*
* The call might block for timeout [milliseconds].
* @return the image or null if there is no new image
*/
public IplImage grab_RGB4() {
if (camera.getCameraFrame(ps3_frame, timeout)) {
timestamp = System.nanoTime()/1000;
image_4ch.getIntBuffer().put(ps3_frame);
return image_4ch;
}
else return null;
}
/** Grab one frame;
* the caller have to make a copy of returned image before processing.
*
* It will throw null pointer exception if not started before grabbing.
* @return "read-only" RGB, 4-channel or GRAY/1-channel image, it throws exception if no image is available
*/
@Override
public Frame grab() throws Exception {
IplImage img = null;
switch (triggered) {
case NO_TRIGGER:
img = grab_RGB4();
break;
case HAS_FRAME:
triggered = Triggered.NO_TRIGGER;
img = makeImage(ps3_frame);
break;
case NO_FRAME:
triggered = Triggered.NO_TRIGGER;
return null;
default: // just schizophrenia - for future enhancement
throw new Exception("Int. error - unknown triggering state");
}
if ((img != null) && (imageMode == ImageMode.GRAY)) {
cvCvtColor(img, image_1ch, CV_RGB2GRAY);
img = image_1ch;
}
return converter.convert(img);
}
/**
* Start camera first (before grabbing).
*/
public void start() throws Exception {
boolean b;
if (ps3_frame == null) {
ps3_frame = new int[ imageWidth * imageHeight ];
image_4ch = IplImage.create(imageWidth, imageHeight, IPL_DEPTH_8U, 4);
image_1ch = IplImage.create(imageWidth, imageHeight, IPL_DEPTH_8U, 1);
}
b = camera.createCamera(
cameraIndex,
(imageMode == ImageMode.GRAY) ? CLCamera.CLEYE_MONO_PROCESSED : CLCamera.CLEYE_COLOR_PROCESSED,
(imageWidth == 320 && imageHeight == 240) ? CLCamera.CLEYE_QVGA : CLCamera.CLEYE_VGA,
(int)frameRate);
if (!b) throw new Exception("Low level createCamera() failed");
b = camera.startCamera();
if (!b) throw new Exception("Camera start() failed");
stat = "started";
}
/**
* Stop camera. It can be re-started if needed.
*/
public void stop() throws Exception {
boolean b = camera.stopCamera();
if (b) stat = "stopped";
else throw new Exception("Camera stop() failed");
}
/** Release resources:
* - CL driver internal resources binded with camera HW
* - internal IplImage
* After calling this function, this mini-grabber object instance can not be used anymore.
*/
public void release() {
if (camera != null) {
camera.dispose();
camera = null;
}
if (image_4ch != null) {
image_4ch.release();
image_4ch = null;
}
if (image_1ch != null) {
image_1ch.release();
image_1ch = null;
}
if (ipl_frame != null) ipl_frame = null;
if (ps3_frame != null) ps3_frame = null;
stat = "released";
}
/** Release internal resources, the same as calling release()
*/
public void dispose() {
release();
}
@Override protected void finalize() throws Throwable {
super.finalize();
release();
}
/** Return internal CLCamera object, mainly to set camera parameters,
* changing camera parameters must be done on stopped camera and before start() is called.
* See CL SDK - setCameraParameter(int param, int val) function.
*
* @return internal CLCamera instance
*/
public CLCamera getCamera() { return camera; }
public String getUUID() { return uuid; }
/**
* @return status and camera parameters of the grabber
*/
@Override public String toString() {
return "UUID="+uuid + "; status=" + stat + "; timeout=" + timeout
+ "; "
+ ((camera != null) ? camera.toString() : "")
;
}
/** Just for testing - loads the CL CLEyeMulticam.dll file, invokes driver
* and lists available cameras.
*
* @param argv - argv is not used
*/
public static void main(String[] argv) {
String[] uuids = listPS3Cameras();
for (int i = 0; i < uuids.length; i++)
System.out.println(i+": "+uuids[i]);
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/Parallel.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
/**
*
* @author Samuel Audet
*/
public class Parallel {
private static final ExecutorService threadPool = Executors.newCachedThreadPool();
public static final String NUM_THREADS = "org.bytedeco.javacv.numthreads";
public static int getNumThreads() {
try {
String s = System.getProperty(NUM_THREADS);
if (s != null) {
return Integer.valueOf(s);
}
} catch (NumberFormatException e) {
throw new RuntimeException(e);
}
return getNumCores();
}
public static void setNumThreads(int numThreads) {
System.setProperty(NUM_THREADS, Integer.toString(numThreads));
}
public static int getNumCores() {
return Runtime.getRuntime().availableProcessors();
}
public static void run(Runnable ... runnables) {
if (runnables.length == 1) {
runnables[0].run();
return;
}
Future[] futures = new Future[runnables.length];
for (int i = 0; i < runnables.length; i++) {
futures[i] = threadPool.submit(runnables[i]);
}
Throwable error = null;
try {
for (Future f : futures) {
if (!f.isDone()) {
f.get();
}
}
} catch (Throwable t) {
error = t;
}
if (error != null) {
for (Future f : futures) {
f.cancel(true);
}
throw new RuntimeException(error);
}
}
public interface Looper {
void loop(int from, int to, int looperID);
}
public static void loop(int from, int to, final Looper looper) {
loop(from, to, getNumThreads(), looper);
}
public static void loop(int from, int to, int numThreads, final Looper looper) {
int numLoopers = Math.min(to-from, numThreads > 0 ? numThreads : getNumCores());
Runnable[] runnables = new Runnable[numLoopers];
for (int i = 0; i < numLoopers; i++) {
final int subFrom = (to-from)*i/numLoopers + from;
final int subTo = (to-from)*(i+1)/numLoopers + from;
final int looperID = i;
runnables[i] = new Runnable() {
public void run() {
looper.loop(subFrom, subTo, looperID);
}
};
}
run(runnables);
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/ProCamColorCalibrator.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.awt.Color;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
*
* @author Samuel Audet
*/
public class ProCamColorCalibrator {
public ProCamColorCalibrator(Settings settings, MarkerDetector.Settings detectorSettings,
MarkedPlane boardPlane, CameraDevice camera, ProjectorDevice projector) {
this.settings = settings;
this.markerDetector = new MarkerDetector(detectorSettings);
this.boardPlane = boardPlane;
this.camera = camera;
this.projector = projector;
Marker[] boardMarkers = boardPlane.getMarkers();
boardSrcPts = CvMat.create(4 + boardMarkers.length*4, 1, CV_64F, 2);
boardDstPts = CvMat.create(4 + boardMarkers.length*4, 1, CV_64F, 2);
boardSrcPts.put(0.0, 0.0,
boardPlane.getWidth(), 0.0,
boardPlane.getWidth(), boardPlane.getHeight(),
0.0, boardPlane.getHeight());
for (int i = 0; i < boardMarkers.length; i++) {
boardSrcPts.put(8 + i*8, boardMarkers[i].corners);
}
projSrcPts = CvMat.create(4, 1, CV_64F, 2);
projDstPts = CvMat.create(4, 1, CV_64F, 2);
projSrcPts.put(0.0, 0.0,
projector.imageWidth-1, 0.0,
projector.imageWidth-1, projector.imageHeight-1,
0.0, projector.imageHeight-1);
camKinv = CvMat.create(3, 3);
// CvMat projKinv = CvMat.create(3, 3);
cvInvert(camera.cameraMatrix, camKinv);
// cvInvert(projector.cameraMatrix, projKinv);
}
public static class Settings extends BaseChildSettings {
int samplesPerChannel = 4;
double trimmingFraction = 0.01;
double detectedBoardMin = 0.5;
public int getSamplesPerChannel() {
return samplesPerChannel;
}
public void setSamplesPerChannel(int samplesPerChannel) {
this.samplesPerChannel = samplesPerChannel;
}
// public double getTrimmingFraction() {
// return trimmingFraction;
// }
// public void setTrimmingFraction(double trimmingFraction) {
// this.trimmingFraction = trimmingFraction;
// }
public double getDetectedBoardMin() {
return detectedBoardMin;
}
public void setDetectedBoardMin(double detectedBoardMin) {
this.detectedBoardMin = detectedBoardMin;
}
}
private Settings settings;
private MarkerDetector markerDetector = null;
private MarkedPlane boardPlane = null;
private CameraDevice camera = null;
private ProjectorDevice projector = null;
private Color[] projectorColors = null, cameraColors = null;
private int counter = 0;
private CvMat boardSrcPts, boardDstPts;
private CvMat projSrcPts, projDstPts;
private CvMat camKinv;
private IplImage mask, mask2, undistImage;
public int getColorCount() {
return counter;
}
public Color[] getProjectorColors() {
double invgamma = 1/projector.getSettings().getResponseGamma();
int s = settings.samplesPerChannel;
if (projectorColors == null) {
projectorColors = new Color[s*s*s];
cameraColors = new Color[s*s*s];
for (int i = 0; i < projectorColors.length; i++) {
int j = i/s;
int k = j/s;
double r = Math.pow((double)(i%s)/(s-1), invgamma);
double g = Math.pow((double)(j%s)/(s-1), invgamma);
double b = Math.pow((double)(k%s)/(s-1), invgamma);
projectorColors[i] = new Color((float)r, (float)g, (float)b);
}
}
return projectorColors;
}
public Color getProjectorColor() {
return getProjectorColors()[counter];
}
public Color[] getCameraColors() {
return cameraColors;
}
public Color getCameraColor() {
return getCameraColors()[counter];
}
public void addCameraColor() {
counter++;
}
public void addCameraColor(Color color) {
cameraColors[counter++] = color;
}
public IplImage getMaskImage() {
return mask;
}
public IplImage getUndistortedCameraImage() {
return undistImage;
}
// public CvScalar getProjectorColor() {
// if (counter == 0) {
// return CvScalar.CV_RGB(0,0,0);
// } else if (counter == 1) {
// return CvScalar.CV_RGB(255,255,255);
// }
// // smallest (power of 2) number of channel that can accomodate "counter" number of samples
// int level = (int)Math.ceil(Math.log(Math.cbrt(counter+1))/Math.log(2));
// int samplesPerChannel = (int)Math.pow(2, level)+1;
// int prevSamplesPerChannel = (int)Math.pow(2, level-1)+1;
// int ignoreAmount = prevSamplesPerChannel*prevSamplesPerChannel*prevSamplesPerChannel;
// int samplesToJump = counter - Math.max(ignoreAmount, 2);
//
// int n = 0;
// for (int i = 0; ; i++) {
// int j = i/samplesPerChannel;
// int k = j/samplesPerChannel;
//
// int ri = (i%samplesPerChannel);
// int gi = (j%samplesPerChannel);
// int bi = (k%samplesPerChannel);
//
// // only count odd samples, that are "in between"
// // samples we have already returned previously
// if (ri%2 != 0 || gi%2 != 0 || bi%2 != 0) {
// n++;
// }
//
// if (n > samplesToJump) {
// int r = ri*255/(samplesPerChannel-1);
// int g = gi*255/(samplesPerChannel-1);
// int b = bi*255/(samplesPerChannel-1);
// return CvScalar.CV_RGB(r,g,b);
// }
// }
//
// }
private static ThreadLocal
H3x3 = CvMat.createThreadLocal(3, 3),
R3x3 = CvMat.createThreadLocal(3, 3),
t3x1 = CvMat.createThreadLocal(3, 1),
n3x1 = CvMat.createThreadLocal(3, 1),
z3x1 = CvMat.createThreadLocal(3, 1);
public boolean processCameraImage(IplImage cameraImage) {
if (undistImage == null ||
undistImage.width() != cameraImage.width() ||
undistImage.height() != cameraImage.height() ||
undistImage.depth() != cameraImage.depth()) {
undistImage = cameraImage.clone();
}
if (mask == null || mask2 == null ||
mask.width() != cameraImage.width() || mask2.width() != cameraImage.width() ||
mask.height() != cameraImage.height() || mask2.height() != cameraImage.width()) {
mask = IplImage.create(cameraImage.width(), cameraImage.height(),
IPL_DEPTH_8U, 1, cameraImage.origin());
mask2 = IplImage.create(cameraImage.width(), cameraImage.height(),
IPL_DEPTH_8U, 1, cameraImage.origin());
}
CvMat H = H3x3.get();
CvMat R = R3x3.get();
CvMat t = t3x1.get();
CvMat n = n3x1.get();
CvMat z = z3x1.get();
z.put(0.0, 0.0, 1.0);
// detect the markers in the camera image, to
// 1. find the expected attenuation due to geometry
// 2. use only regions we know are "white" on the board
camera.undistort(cameraImage, undistImage);
Marker[] detectedBoardMarkers = markerDetector.detect(undistImage, false);
if (detectedBoardMarkers.length >= boardPlane.getMarkers().length*settings.detectedBoardMin) {
// use detected markers in the camera image, to
// 1. find the expected attenuation due to geometry
// 2. use only regions we know are "white" on the board
boardPlane.getTotalWarp(detectedBoardMarkers, H);
cvPerspectiveTransform(boardSrcPts, boardDstPts, H);
double[] boardPts = boardDstPts.get();
// Extract R and t from the board homography, using it as our
// "first camera", so we need to use z as the normal of the plane...
cvMatMul(camKinv, H, R);
double error = JavaCV.HnToRt(R, z, R, t);
// System.out.println(error);
// find the plane equation of the board in the camera's frame
// (normal vector n and distance d) and get the back-projection
// matrix of the camera
cvMatMul(R, z, n);
double d = cvDotProduct(t, z);
// find the homography from the camera to the projector
// H = K_p (R - T*n^T/d)) K_c^-1
cvGEMM (projector.T, n, -1/d, projector.R, 1, H, CV_GEMM_B_T);
cvMatMul(projector.cameraMatrix, H, H);
cvMatMul(H, camKinv, H);
cvConvertScale(H, H, 1/H.get(8), 0);
// System.out.println(H);
// find the homography from the projector to the camera
cvInvert(H, H);
cvConvertScale(H, H, 1/H.get(8), 0);
// reproject projector edges into the camera image
cvPerspectiveTransform(projSrcPts, projDstPts, H);
double[] projPts = projDstPts.get();
// create mask containing only blank regions of the board
// intersected with regions coverable by the projector
cvSetZero(mask);
double cx = 0, cy = 0;
for (int j = 0; j < 4; j++) {
cx += boardPts[j*2 ];
cy += boardPts[j*2 + 1];
}
cx/=4; cy/=4;
for (int j = 0; j < 4; j++) {
boardPts[j*2 ] -= (boardPts[j*2 ] - cx)*settings.trimmingFraction;
boardPts[j*2 + 1] -= (boardPts[j*2 + 1] - cy)*settings.trimmingFraction;
}
cvFillConvexPoly(mask, new CvPoint(4).put((byte)16, boardPts, 0, 8),
4, CvScalar.WHITE, 8, 16);
for (int j = 0; j < (boardPts.length-8)/8; j++) {
cvFillConvexPoly(mask, new CvPoint(4).put((byte)16, boardPts, 8 + j*8, 8),
4, CvScalar.BLACK, 8, 16);
}
cvSetZero(mask2);
cx = 0; cy = 0;
for (int j = 0; j < 4; j++) {
cx += projPts[j*2 ];
cy += projPts[j*2 + 1];
}
cx/=4; cy/=4;
for (int j = 0; j < 4; j++) {
projPts[j*2 ] -= (projPts[j*2 ] - cx)*settings.trimmingFraction;
projPts[j*2 + 1] -= (projPts[j*2 + 1] - cy)*settings.trimmingFraction;
}
cvFillConvexPoly(mask2, new CvPoint(4).put((byte)16, projPts, 0, 8),
4, CvScalar.WHITE, 8, 16);
cvAnd(mask, mask2, mask, null);
cvErode(mask, mask, null, 1);
//cvSaveImage("masked" + i + ".png", cameraImages[i]);
//try {
// Thread.sleep(1000);
//} catch (InterruptedException ex) { }
// take the average as the camera response, and also
// compensate for attenuation caused by the geometry
CvScalar c = cvAvg(undistImage, mask);
int[] o = camera.getRGBColorOrder();
double s = cameraImage.highValue();
cameraColors[counter] = new Color((float)(c.val(o[0])/s),
(float)(c.val(o[1])/s), (float)(c.val(o[2])/s));
return true;
}
return false;
}
public double calibrate() {
Color[] cc = getCameraColors();
Color[] pc = getProjectorColors();
assert (counter == pc.length);
ColorCalibrator calibrator = new ColorCalibrator(projector);
projector.avgColorErr = calibrator.calibrate(cc, pc);
camera.colorMixingMatrix = CvMat.create(3, 3);
camera.additiveLight = CvMat.create(3, 1);
cvSetIdentity(camera.colorMixingMatrix);
cvSetZero (camera.additiveLight);
counter = 0;
return projector.avgColorErr;
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/ProCamGeometricCalibrator.java
================================================
/*
* Copyright (C) 2009-2011 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.util.Arrays;
import java.util.Iterator;
import java.util.LinkedList;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
*
* @author Samuel Audet
*/
public class ProCamGeometricCalibrator {
public ProCamGeometricCalibrator(Settings settings, MarkerDetector.Settings detectorSettings,
MarkedPlane boardPlane, MarkedPlane projectorPlane,
ProjectiveDevice camera, ProjectiveDevice projector) {
this(settings, detectorSettings, boardPlane, projectorPlane,
new GeometricCalibrator[] {
new GeometricCalibrator(settings, detectorSettings, boardPlane, camera)},
new GeometricCalibrator(settings, detectorSettings, projectorPlane, projector));
}
@SuppressWarnings("unchecked")
public ProCamGeometricCalibrator(Settings settings, MarkerDetector.Settings detectorSettings,
MarkedPlane boardPlane, MarkedPlane projectorPlane,
GeometricCalibrator[] cameraCalibrators, GeometricCalibrator projectorCalibrator) {
this.settings = settings;
this.detectorSettings = detectorSettings;
this.boardPlane = boardPlane;
this.projectorPlane = projectorPlane;
this.cameraCalibrators = cameraCalibrators;
int n = cameraCalibrators.length;
markerDetectors = new MarkerDetector[n];
// "unchecked" warning here
allImagedBoardMarkers = new LinkedList[n];
grayscaleImage = new IplImage[n];
tempImage1 = new IplImage[n];
tempImage2 = new IplImage[n];
lastDetectedMarkers1 = new Marker[n][];
lastDetectedMarkers2 = new Marker[n][];
rmseBoardWarp = new double[n];
rmseProjWarp = new double[n];
boardWarp = new CvMat[n];
projWarp = new CvMat[n];
prevBoardWarp = new CvMat[n];
lastBoardWarp = new CvMat[n];
tempPts1 = new CvMat[n];
tempPts2 = new CvMat[n];
for (int i = 0; i < n; i++) {
markerDetectors[i] = new MarkerDetector(detectorSettings);
allImagedBoardMarkers[i] = new LinkedList();
grayscaleImage[i] = null;
tempImage1[i] = null;
tempImage2[i] = null;
lastDetectedMarkers1[i] = null;
lastDetectedMarkers2[i] = null;
rmseBoardWarp[i] = Double.POSITIVE_INFINITY;
rmseProjWarp[i] = Double.POSITIVE_INFINITY;
boardWarp[i] = CvMat.create(3, 3);
projWarp[i] = CvMat.create(3, 3);
prevBoardWarp[i] = CvMat.create(3, 3);
lastBoardWarp[i] = CvMat.create(3, 3);
cvSetIdentity(prevBoardWarp[i]);
cvSetIdentity(lastBoardWarp[i]);
tempPts1[i] = CvMat.create(1, 4, CV_64F, 2);
tempPts2[i] = CvMat.create(1, 4, CV_64F, 2);
}
this.projectorCalibrator = projectorCalibrator;
this.boardWarpSrcPts = CvMat.create(1, 4, CV_64F, 2);
if (boardPlane != null) {
int w = boardPlane.getImage().width();
int h = boardPlane.getImage().height();
boardWarpSrcPts.put(0.0, 0.0, w, 0.0, w, h, 0.0, h);
}
if (projectorPlane != null) {
int w = projectorPlane.getImage().width();
int h = projectorPlane.getImage().height();
projectorCalibrator.getProjectiveDevice().imageWidth = w;
projectorCalibrator.getProjectiveDevice().imageHeight = h;
}
}
private final int
MSB_IMAGE_SHIFT = 8,
LSB_IMAGE_SHIFT = 7;
public static class Settings extends GeometricCalibrator.Settings {
double detectedProjectorMin = 0.5;
boolean useOnlyIntersection = true;
double prewarpUpdateErrorMax = 0.01;
public double getDetectedProjectorMin() {
return detectedProjectorMin;
}
public void setDetectedProjectorMin(double detectedProjectorMin) {
this.detectedProjectorMin = detectedProjectorMin;
}
public boolean isUseOnlyIntersection() {
return useOnlyIntersection;
}
public void setUseOnlyIntersection(boolean useOnlyIntersection) {
this.useOnlyIntersection = useOnlyIntersection;
}
public double getPrewarpUpdateErrorMax() {
return prewarpUpdateErrorMax;
}
public void setPrewarpUpdateErrorMax(double prewarpUpdateErrorMax) {
this.prewarpUpdateErrorMax = prewarpUpdateErrorMax;
}
}
private Settings settings;
private MarkerDetector.Settings detectorSettings;
// (possibly) multiple camera stuff in arrays
private GeometricCalibrator[] cameraCalibrators;
private MarkerDetector[] markerDetectors;
// keep our own list of markers for the camera, since cameraCalibrators
// might be used outside ProCamGeometricCalibrator as well...
LinkedList[] allImagedBoardMarkers;
private IplImage[] grayscaleImage, tempImage1, tempImage2;
private Marker[][] lastDetectedMarkers1, lastDetectedMarkers2;
private double[] rmseBoardWarp, rmseProjWarp;
private CvMat[] boardWarp, projWarp;
private CvMat[] prevBoardWarp, lastBoardWarp;
private CvMat[] tempPts1, tempPts2;
// single board and projector stuff
private boolean updatePrewarp = false;
private final MarkedPlane boardPlane, projectorPlane;
private final GeometricCalibrator projectorCalibrator;
private final CvMat boardWarpSrcPts;
public MarkedPlane getBoardPlane() {
return boardPlane;
}
public MarkedPlane getProjectorPlane() {
return projectorPlane;
}
public GeometricCalibrator[] getCameraCalibrators() {
return cameraCalibrators;
}
public GeometricCalibrator getProjectorCalibrator() {
return projectorCalibrator;
}
public int getImageCount() {
int n = projectorCalibrator.getImageCount()/cameraCalibrators.length;
for (GeometricCalibrator c : cameraCalibrators) {
assert(c.getImageCount() == n);
}
return n;
}
public Marker[][] processCameraImage(IplImage cameraImage) {
return processCameraImage(cameraImage, 0);
}
public Marker[][] processCameraImage(IplImage cameraImage, final int cameraNumber) {
cameraCalibrators[cameraNumber].getProjectiveDevice().imageWidth = cameraImage.width();
cameraCalibrators[cameraNumber].getProjectiveDevice().imageHeight = cameraImage.height();
if (cameraImage.nChannels() > 1) {
if (grayscaleImage[cameraNumber] == null ||
grayscaleImage[cameraNumber].width() != cameraImage.width() ||
grayscaleImage[cameraNumber].height() != cameraImage.height() ||
grayscaleImage[cameraNumber].depth() != cameraImage.depth()) {
grayscaleImage[cameraNumber] = IplImage.create(cameraImage.width(),
cameraImage.height(), cameraImage.depth(), 1, cameraImage.origin());
}
cvCvtColor(cameraImage, grayscaleImage[cameraNumber], CV_BGR2GRAY);
} else {
grayscaleImage[cameraNumber] = cameraImage;
}
final boolean boardWhiteMarkers = boardPlane.getForegroundColor().magnitude() >
boardPlane.getBackgroundColor().magnitude();
final boolean projWhiteMarkers = projectorPlane.getForegroundColor().magnitude() >
projectorPlane.getBackgroundColor().magnitude();
if (grayscaleImage[cameraNumber].depth() > 8) {
if (tempImage1[cameraNumber] == null ||
tempImage1[cameraNumber].width() != grayscaleImage[cameraNumber].width() ||
tempImage1[cameraNumber].height() != grayscaleImage[cameraNumber].height()) {
tempImage1[cameraNumber] = IplImage.create(grayscaleImage[cameraNumber].width(),
grayscaleImage[cameraNumber].height(), IPL_DEPTH_8U, 1,
grayscaleImage[cameraNumber].origin());
tempImage2[cameraNumber] = IplImage.create(grayscaleImage[cameraNumber].width(),
grayscaleImage[cameraNumber].height(), IPL_DEPTH_8U, 1,
grayscaleImage[cameraNumber].origin());
}
Parallel.run(new Runnable() { public void run() {
cvConvertScale(grayscaleImage[cameraNumber],
tempImage1[cameraNumber], 1.0/(1< allDistortedProjectorMarkers = projectorCalibrator.getAllObjectMarkers(),
distortedProjectorMarkersAtOrigin = new LinkedList(),
allUndistortedProjectorMarkers = new LinkedList(),
undistortedProjectorMarkersAtOrigin = new LinkedList();
Iterator ip = allDistortedProjectorMarkers.iterator();
// "unchecked" warning here
Iterator[] ib = new Iterator[cameraCalibrators.length];
for (int cameraNumber = 0; cameraNumber < cameraCalibrators.length; cameraNumber++) {
ib[cameraNumber] = allImagedBoardMarkers[cameraNumber].iterator();
}
// iterate over all the saved markers in the right order...
// eew, this is getting ugly...
while (ip.hasNext()) {
for (int cameraNumber = 0; cameraNumber < cameraCalibrators.length; cameraNumber++) {
double maxError = settings.prewarpUpdateErrorMax *
(cameraCalibrators[cameraNumber].getProjectiveDevice().imageWidth+
cameraCalibrators[cameraNumber].getProjectiveDevice().imageHeight)/2;
Marker[] distortedBoardMarkers = ib[cameraNumber].next(),
distortedProjectorMarkers = ip.next(),
undistortedBoardMarkers = new Marker[distortedBoardMarkers.length],
undistortedProjectorMarkers = new Marker[distortedProjectorMarkers.length];
// remove radial distortion from all points imaged by the camera
for (int i = 0; i < distortedBoardMarkers.length; i++) {
Marker m = undistortedBoardMarkers[i] = distortedBoardMarkers[i].clone();
m.corners = cameraCalibrators[cameraNumber].getProjectiveDevice().undistort(m.corners);
}
for (int i = 0; i < distortedProjectorMarkers.length; i++) {
Marker m = undistortedProjectorMarkers[i] = distortedProjectorMarkers[i].clone();
m.corners = cameraCalibrators[cameraNumber].getProjectiveDevice().undistort(m.corners);
}
// remove linear distortion/warping of camera imaged markers from
// the projector, to get their physical location on the board
if (boardPlane.getTotalWarp(undistortedBoardMarkers, boardWarp[cameraNumber]) > maxError) {
assert(false);
}
cvInvert(boardWarp[cameraNumber], boardWarp[cameraNumber]);
Marker.applyWarp(undistortedProjectorMarkers, boardWarp[cameraNumber]);
// tadam, we not have undistorted "object" corners for the projector..
allUndistortedProjectorMarkers.add(undistortedProjectorMarkers);
if (cameraCalibrators[cameraNumber] == calibratorAtOrigin) {
undistortedProjectorMarkersAtOrigin.add(undistortedProjectorMarkers);
distortedProjectorMarkersAtOrigin.add(distortedProjectorMarkers);
} else {
undistortedProjectorMarkersAtOrigin.add(new Marker[0]);
distortedProjectorMarkersAtOrigin.add(new Marker[0]);
}
}
}
// calibrate projector
projectorCalibrator.setAllObjectMarkers(allUndistortedProjectorMarkers);
double[] reprojErr = projectorCalibrator.calibrate(useCenters);
// projectorCalibrator.getProjectiveDevice().nominalDistance =
// projectorCalibrator.getProjectiveDevice().getNominalDistance(boardPlane);
// calibrate as a stereo pair (find rotation and translation)
// let's use the projector markers only...
LinkedList om = calibratorAtOrigin.getAllObjectMarkers(),
im = calibratorAtOrigin.getAllImageMarkers();
calibratorAtOrigin.setAllObjectMarkers(undistortedProjectorMarkersAtOrigin);
calibratorAtOrigin.setAllImageMarkers(distortedProjectorMarkersAtOrigin);
double[] epipolarErr = calibratorAtOrigin.calibrateStereo(useCenters, projectorCalibrator);
// reset everything as it was before we started, so we get the same
// result if called a second time..
projectorCalibrator.setAllObjectMarkers(allDistortedProjectorMarkers);
calibratorAtOrigin.setAllObjectMarkers(om);
calibratorAtOrigin.setAllImageMarkers(im);
return new double[] { reprojErr[0], reprojErr[1], epipolarErr[0], epipolarErr[1] };
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/ProCamTransformer.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import org.bytedeco.opencv.opencv_calib3d.*;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.javacv.cvkernels.*;
import static org.bytedeco.opencv.global.opencv_calib3d.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
*
* @author Samuel Audet
*/
public class ProCamTransformer implements ImageTransformer {
public ProCamTransformer(double[] referencePoints,
CameraDevice camera, ProjectorDevice projector) {
this(referencePoints, camera, projector, null);
}
public ProCamTransformer(double[] referencePoints,
CameraDevice camera, ProjectorDevice projector, CvMat n) {
this.camera = camera;
this.projector = projector;
if (referencePoints != null) {
this.surfaceTransformer = new ProjectiveColorTransformer(
camera.cameraMatrix, camera.cameraMatrix, null, null, n,
referencePoints, null, null, 3, 0);
}
double[] referencePoints1 = { 0, 0, camera.imageWidth/2, camera.imageHeight, camera.imageWidth, 0 };
double[] referencePoints2 = { 0, 0, projector.imageWidth/2, projector.imageHeight, projector.imageWidth, 0 };
if (n != null) {
invCameraMatrix = CvMat.create(3, 3);
cvInvert(camera.cameraMatrix, invCameraMatrix);
JavaCV.perspectiveTransform(referencePoints2, referencePoints1,
invCameraMatrix, projector.cameraMatrix, projector.R, projector.T, n, true);
}
this.projectorTransformer = new ProjectiveColorTransformer(
camera.cameraMatrix, projector.cameraMatrix, projector.R, projector.T, null,
referencePoints1, referencePoints2, projector.colorMixingMatrix,
/*surfaceTransformer == null ? 3 : */1, 3);
// CvMat n2 = createParameters().getN();
if (referencePoints != null && n != null) {
frontoParallelH = camera.getFrontoParallelH(referencePoints, n, CvMat.create(3, 3));
invFrontoParallelH = frontoParallelH.clone();
cvInvert(frontoParallelH, invFrontoParallelH);
}
}
protected CameraDevice camera = null;
protected ProjectorDevice projector = null;
protected ProjectiveColorTransformer surfaceTransformer = null;
protected ProjectiveColorTransformer projectorTransformer = null;
protected IplImage[] projectorImage = null, surfaceImage = null;
protected CvScalar fillColor = cvScalar(0.0, 0.0, 0.0, 1.0);
protected CvRect roi = new CvRect();
protected CvMat frontoParallelH = null, invFrontoParallelH = null;
protected CvMat invCameraMatrix = null;
protected KernelData kernelData = null;
protected CvMat[] H1 = null;
protected CvMat[] H2 = null;
protected CvMat[] X = null;
public int getNumGains() {
return projectorTransformer.getNumGains();
}
public int getNumBiases() {
return projectorTransformer.getNumBiases();
}
public CvScalar getFillColor() {
return fillColor;
}
public void setFillColor(CvScalar fillColor) {
this.fillColor = fillColor;
}
public ProjectiveColorTransformer getSurfaceTransformer() {
return surfaceTransformer;
}
public ProjectiveColorTransformer getProjectorTransformer() {
return projectorTransformer;
}
public IplImage getProjectorImage(int pyramidLevel) {
return projectorImage[pyramidLevel];
}
public void setProjectorImage(IplImage projectorImage0, int minLevel, int maxLevel) {
setProjectorImage(projectorImage0, minLevel, maxLevel, true);
}
public void setProjectorImage(IplImage projectorImage0, int minLevel, int maxLevel, boolean convertToFloat) {
if (projectorImage == null || projectorImage.length != maxLevel+1) {
projectorImage = new IplImage[maxLevel+1];
}
if (projectorImage0.depth() == IPL_DEPTH_32F || !convertToFloat) {
projectorImage[minLevel] = projectorImage0;
} else {
if (projectorImage[minLevel] == null) {
projectorImage[minLevel] = IplImage.create(projectorImage0.width(), projectorImage0.height(),
IPL_DEPTH_32F, projectorImage0.nChannels(), projectorImage0.origin());
}
IplROI ir = projectorImage0.roi();
if (ir != null) {
int align = 1<<(maxLevel+1);
roi.x(Math.max(0, (int)Math.floor((double)ir.xOffset()/align)*align));
roi.y(Math.max(0, (int)Math.floor((double)ir.yOffset()/align)*align));
roi.width (Math.min(projectorImage0.width(), (int)Math.ceil((double)ir.width() /align)*align));
roi.height(Math.min(projectorImage0.height(), (int)Math.ceil((double)ir.height()/align)*align));
cvSetImageROI(projectorImage0, roi);
cvSetImageROI(projectorImage[minLevel], roi);
} else {
cvResetImageROI(projectorImage0);
cvResetImageROI(projectorImage[minLevel]);
}
cvConvertScale(projectorImage0, projectorImage[minLevel], 1.0/255.0, 0);
}
// CvScalar.ByValue average = cvAvg(projectorImage[0], null);
// cvSubS(projectorImage[0], average, projectorImage[0], null);
for (int i = minLevel+1; i <= maxLevel; i++) {
int w = projectorImage[i-1].width()/2;
int h = projectorImage[i-1].height()/2;
int d = projectorImage[i-1].depth();
int c = projectorImage[i-1].nChannels();
int o = projectorImage[i-1].origin();
if (projectorImage[i] == null) {
projectorImage[i] = IplImage.create(w, h, d, c, o);
}
IplROI ir = projectorImage[i-1].roi();
if (ir != null) {
roi.x(ir.xOffset()/2); roi.width (ir.width() /2);
roi.y(ir.yOffset()/2); roi.height(ir.height()/2);
cvSetImageROI(projectorImage[i], roi);
} else {
cvResetImageROI(projectorImage[i]);
}
cvPyrDown(projectorImage[i-1], projectorImage[i], CV_GAUSSIAN_5x5);
cvResetImageROI(projectorImage[i-1]);
}
}
public IplImage getSurfaceImage(int pyramidLevel) {
return surfaceImage[pyramidLevel];
}
public void setSurfaceImage(IplImage surfaceImage0, int pyramidLevels) {
if (surfaceImage == null || surfaceImage.length != pyramidLevels) {
surfaceImage = new IplImage[pyramidLevels];
}
surfaceImage[0] = surfaceImage0;
cvResetImageROI(surfaceImage0);
for (int i = 1; i < pyramidLevels; i++) {
int w = surfaceImage[i-1].width()/2;
int h = surfaceImage[i-1].height()/2;
int d = surfaceImage[i-1].depth();
int c = surfaceImage[i-1].nChannels();
int o = surfaceImage[i-1].origin();
if (surfaceImage[i] == null) {
surfaceImage[i] = IplImage.create(w, h, d, c, o);
} else {
cvResetImageROI(surfaceImage[i]);
}
cvPyrDown(surfaceImage[i-1], surfaceImage[i], CV_GAUSSIAN_5x5);
}
}
protected void prepareTransforms(CvMat H1, CvMat H2, CvMat X, int pyramidLevel, Parameters p) {
ProjectiveColorTransformer.Parameters cameraParameters = p.getSurfaceParameters();
ProjectiveColorTransformer.Parameters projectorParameters = p.getProjectorParameters();
if (surfaceTransformer != null) {
cvInvert(cameraParameters.getH(), H1);
}
cvInvert(projectorParameters.getH(), H2);
// adjust the scale of the transformation based on the pyramid level
if (pyramidLevel > 0) {
int scale = 1< 0) {
// System.err.println(data[0].dstCountZero + " out of " + data[0].dstCount
// + " are zero = " + 100*data[0].dstCountZero/data[0].dstCount + "%");
// }
}
public Parameters createParameters() {
return new Parameters();
}
public class Parameters implements ImageTransformer.Parameters {
protected Parameters() {
reset(false);
}
protected Parameters(ProjectiveColorTransformer.Parameters surfaceParameters,
ProjectiveColorTransformer.Parameters projectorParameters) {
reset(surfaceParameters, projectorParameters);
}
private ProjectiveColorTransformer.Parameters surfaceParameters = null;
private ProjectiveColorTransformer.Parameters projectorParameters = null;
private IplImage[] tempImage = null;
private CvMat H = CvMat.create(3, 3), R = CvMat.create(3, 3),
n = CvMat.create(3, 1), t = CvMat.create(3, 1);
public ProjectiveColorTransformer.Parameters getSurfaceParameters() {
return surfaceParameters;
}
public ProjectiveColorTransformer.Parameters getProjectorParameters() {
return projectorParameters;
}
private int getSizeForSurface() {
return surfaceTransformer == null ? 0 : surfaceParameters.size() -
surfaceTransformer.getNumGains() - surfaceTransformer.getNumBiases();
}
private int getSizeForProjector() {
return projectorParameters.size();
}
public int size() {
return getSizeForSurface() + getSizeForProjector();
}
public double[] get() {
double[] p = new double[size()];
for (int i = 0; i < p.length; i++) {
p[i] = get(i);
}
return p;
}
public double get(int i) {
if (i < getSizeForSurface()) {
return surfaceParameters.get(i);
} else {
return projectorParameters.get(i-getSizeForSurface());
}
}
public void set(double ... p) {
for (int i = 0; i < p.length; i++) {
set(i, p[i]);
}
}
public void set(int i, double p) {
if (i < getSizeForSurface()) {
surfaceParameters.set(i, p);
} else {
projectorParameters.set(i-getSizeForSurface(), p);
}
}
public void set(ImageTransformer.Parameters p) {
Parameters pcp = (Parameters)p;
if (surfaceTransformer != null) {
surfaceParameters.set(pcp.getSurfaceParameters());
surfaceParameters.resetColor(false);
}
projectorParameters.set(pcp.getProjectorParameters());
}
public void reset(boolean asIdentity) {
reset(null, null);
}
public void reset(ProjectiveColorTransformer.Parameters surfaceParameters,
ProjectiveColorTransformer.Parameters projectorParameters) {
if (surfaceParameters == null && surfaceTransformer != null) {
surfaceParameters = surfaceTransformer.createParameters();
}
if (projectorParameters == null) {
projectorParameters = projectorTransformer.createParameters();
}
this.surfaceParameters = surfaceParameters;
this.projectorParameters = projectorParameters;
setSubspace(getSubspace());
}
// public boolean addDelta(int i) {
// return addDelta(i, 1);
// }
// public boolean addDelta(int i, double scale) {
// // gradient varies linearly with intensity, so
// // the increment value is not very important, but
// // referenceCameraImage is good only for the value 1,
// // so let's use that
// if (i < getSizeForSurface()) {
// surfaceParameters.addDelta(i, scale);
// projectorParameters.setUpdateNeeded(true);
// } else {
// projectorParameters.addDelta(i-getSizeForSurface(), scale);
// }
//
// return false;
// }
public double getConstraintError() {
double error = surfaceTransformer == null ? 0 : surfaceParameters.getConstraintError();
projectorParameters.update();
return error;
}
public void compose(ImageTransformer.Parameters p1, boolean inverse1,
ImageTransformer.Parameters p2, boolean inverse2) {
throw new UnsupportedOperationException("Compose operation not supported.");
}
public boolean preoptimize() {
double[] p = setSubspaceInternal(getSubspaceInternal());
if (p != null) {
set(8, p[8]);
set(9, p[9]);
set(10, p[10]);
return true;
}
return false;
}
public void setSubspace(double ... p) {
double[] dst = setSubspaceInternal(p);
if (dst != null) {
set(dst);
}
}
public double[] getSubspace() {
return getSubspaceInternal();
}
private double[] setSubspaceInternal(double ... p) {
if (invFrontoParallelH == null) {
return null;
}
double[] dst = new double[8+3];
t.put(p[0], p[1], p[2]);
Rodrigues(cvarrToMat(t), cvarrToMat(R), null);
t.put(p[3], p[4], p[5]);
// compute new H
H.put(R.get(0), R.get(1), t.get(0),
R.get(3), R.get(4), t.get(1),
R.get(6), R.get(7), t.get(2));
cvMatMul(H, invFrontoParallelH, H);
cvMatMul(surfaceTransformer.getK2(), H, H);
cvMatMul(H, surfaceTransformer.getInvK1(), H);
// compute new n, rotation from the z-axis
cvGEMM(R, t, 1, null, 0, t, CV_GEMM_A_T);
double scale = 1/t.get(2);
n.put(0.0, 0.0, 1.0);
cvGEMM(R, n, scale, null, 0, n, 0);
// compute and set new three points
double[] src = projectorTransformer.getReferencePoints2();
JavaCV.perspectiveTransform(src, dst,
projectorTransformer.getInvK1(),projectorTransformer.getK2(),
projectorTransformer.getR(), projectorTransformer.getT(), n, true);
dst[8] = dst[0];
dst[9] = dst[2];
dst[10] = dst[4];
// compute and set new four points
JavaCV.perspectiveTransform(surfaceTransformer.getReferencePoints1(), dst, H);
return dst;
}
private double[] getSubspaceInternal() {
if (frontoParallelH == null) {
return null;
}
cvMatMul(surfaceTransformer.getK1(), frontoParallelH, H);
cvMatMul(surfaceParameters .getH(), H, H);
cvMatMul(surfaceTransformer.getInvK2(), H, H);
JavaCV.HtoRt(H, R, t);
Rodrigues(cvarrToMat(R), cvarrToMat(n), null);
double[] p = { n.get(0), n.get(1), n.get(2),
t.get(0), t.get(1), t.get(2) };
return p;
}
public CvMat getN() {
double[] src = projectorTransformer.getReferencePoints2();
double[] dst = projectorTransformer.getReferencePoints1().clone();
dst[0] = projectorParameters.get(0);
dst[2] = projectorParameters.get(1);
dst[4] = projectorParameters.get(2);
// get plane parameters n, but since we model the target to be
// the camera, we have to inverse everything before calling
// getPlaneParameters() and reframe the n it returns
cvTranspose(projectorTransformer.getR(), R);
cvGEMM(R, projectorTransformer.getT(), -1, null, 0, t, 0);
JavaCV.getPlaneParameters(src, dst, projectorTransformer.getInvK2(),
projectorTransformer.getK1(), R, t, n);
double d = 1 + cvDotProduct(n, projectorTransformer.getT());
cvGEMM(R, n, 1/d, null, 0, n, 0);
return n;
}
public CvMat getN0() {
n = getN();
if (surfaceTransformer == null) {
return n;
}
// remove projective effect of the current n,
// leaving only the effect of n0
camera.getFrontoParallelH(surfaceParameters.get(), n, R);
cvInvert(surfaceParameters.getH(), H);
cvMatMul(H, surfaceTransformer.getK2(), H);
cvMatMul(H, R, H);
cvMatMul(surfaceTransformer.getInvK1(), H, H);
JavaCV.HtoRt(H, R, t);
// compute n0, as a rotation from the z-axis
cvGEMM(R, t, 1, null, 0, t, CV_GEMM_A_T);
double scale = 1/t.get(2);
n.put(0.0, 0.0, 1.0);
cvGEMM(R, n, scale, null, 0, n, 0);
return n;
}
@Override public Parameters clone() {
Parameters p = new Parameters();
p.surfaceParameters = surfaceParameters == null ? null : surfaceParameters.clone();
p.projectorParameters = projectorParameters.clone();
return p;
}
@Override public String toString() {
if (surfaceParameters != null) {
return surfaceParameters.toString() + projectorParameters.toString();
} else {
return projectorParameters.toString();
}
}
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/ProCamTransformerCL.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import com.jogamp.common.os.Platform;
import com.jogamp.opencl.CLBuffer;
import com.jogamp.opencl.CLEventList;
import com.jogamp.opencl.CLImage2d;
import com.jogamp.opencl.CLImageFormat;
import com.jogamp.opencl.CLKernel;
import com.jogamp.opencl.CLMemory;
import java.nio.FloatBuffer;
import org.bytedeco.opencv.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_core.*;
/**
*
* @author Samuel Audet
*/
public class ProCamTransformerCL extends ProCamTransformer implements ImageTransformerCL {
public ProCamTransformerCL(JavaCVCL context, double[] referencePoints,
CameraDevice camera, ProjectorDevice projector) {
this(context, referencePoints, camera, projector, null);
}
public ProCamTransformerCL(JavaCVCL context, double[] referencePoints,
CameraDevice camera, ProjectorDevice projector, CvMat n) {
super(referencePoints, camera, projector, n);
final int dotSize = createParameters().size();
this.context = context;
this.nullSize = Platform.is32Bit() ? 4 : 8;
this.H1Buffer = surfaceTransformer == null ? null :
context.getCLContext().createFloatBuffer(dotSize*9, CLBuffer.Mem.READ_ONLY);
this.H2Buffer = context.getCLContext().createFloatBuffer(dotSize*9, CLBuffer.Mem.READ_ONLY);
this.XBuffer = context.getCLContext().createFloatBuffer(dotSize*16, CLBuffer.Mem.READ_ONLY);
if (getClass() == ProCamTransformerCL.class) {
CLKernel[] kernels = context.buildKernels(
JavaCVCL.fastCompilerOptions + " -cl-nv-maxrregcount=32 -DDOT_SIZE=" + dotSize,
//JavaCVCL.fastCompilerOptions + " -DDOT_SIZE=" + dotSize,
"ImageTransformer.cl:ProCamTransformer.cl",
"transformOne", "transformSub", "transformDot", "reduceOutputData");
oneKernel = kernels[0];
subKernel = kernels[1];
dotKernel = kernels[2];
reduceKernel = kernels[3];
}
}
private static final ThreadLocal
H13x3 = CvMat.createThreadLocal(3, 3),
H23x3 = CvMat.createThreadLocal(3, 3),
X4x4 = CvMat.createThreadLocal(4, 4);
protected final JavaCVCL context;
protected final int nullSize;
protected final CLBuffer H1Buffer, H2Buffer, XBuffer;
protected CLImage2d[] projectorImageCL = null, surfaceImageCL = null;
private CLKernel oneKernel, subKernel, dotKernel, reduceKernel;
public JavaCVCL getContext() {
return context;
}
public ProjectiveColorTransformerCL getSurfaceTransformerCL() {
return (ProjectiveColorTransformerCL)surfaceTransformer;
}
public ProjectiveColorTransformerCL getProjectorTransformerCL() {
return (ProjectiveColorTransformerCL)projectorTransformer;
}
public CLImage2d getProjectorImageCL(int pyramidLevel) {
return projectorImageCL[pyramidLevel];
}
public void setProjectorImageCL(CLImage2d projectorImage0, int minPyramidLevel, int maxPyramidLevel) {
if (projectorImageCL == null || projectorImageCL.length != maxPyramidLevel+1) {
projectorImageCL = new CLImage2d[maxPyramidLevel+1];
}
projectorImageCL[minPyramidLevel] = projectorImage0;
for (int i = minPyramidLevel+1; i <= maxPyramidLevel; i++) {
if (projectorImageCL[i] == null) {
int w = projectorImageCL[i-1].width/2;
int h = projectorImageCL[i-1].height/2;
CLImageFormat format = new CLImageFormat(CLImageFormat.ChannelOrder.RGBA, CLImageFormat.ChannelType.FLOAT);
projectorImageCL[i] = context.getCLContext().createImage2d(w, h, format);
}
context.pyrDown(projectorImageCL[i-1], projectorImageCL[i]);
}
}
public CLImage2d getSurfaceImageCL(int pyramidLevel) {
return surfaceImageCL[pyramidLevel];
}
public void setSurfaceImageCL(CLImage2d surfaceImage0, int pyramidLevels) {
if (surfaceImageCL == null || surfaceImageCL.length != pyramidLevels) {
surfaceImageCL = new CLImage2d[pyramidLevels];
}
surfaceImageCL[0] = surfaceImage0;
for (int i = 1; i < pyramidLevels; i++) {
if (surfaceImageCL[i] == null) {
int w = surfaceImageCL[i-1].width/2;
int h = surfaceImageCL[i-1].height/2;
CLImageFormat format = new CLImageFormat(CLImageFormat.ChannelOrder.RGBA, CLImageFormat.ChannelType.FLOAT);
surfaceImageCL[i] = context.getCLContext().createImage2d(w, h, format);
}
context.pyrDown(surfaceImageCL[i-1], surfaceImageCL[i]);
}
}
protected void prepareTransforms(CLBuffer H1Buffer, CLBuffer H2Buffer, CLBuffer XBuffer,
int pyramidLevel, ImageTransformer.Parameters[] parameters) {
FloatBuffer floatH1 = surfaceTransformer == null ? null : (FloatBuffer)H1Buffer.getBuffer().rewind();
FloatBuffer floatH2 = (FloatBuffer)H2Buffer.getBuffer().rewind();
FloatBuffer floatX = (FloatBuffer) XBuffer.getBuffer().rewind();
CvMat H1 = H13x3.get();
CvMat H2 = H23x3.get();
CvMat X = X4x4.get();
for (int i = 0; i < parameters.length; i++) {
prepareTransforms(surfaceTransformer == null ? null : H1,
H2, X, pyramidLevel, (ProCamTransformer.Parameters)parameters[i]);
for (int j = 0; j < 9; j++) {
if (surfaceTransformer != null) {
floatH1.put((float)H1.get(j));
}
floatH2.put((float)H2.get(j));
}
for (int j = 0; j < 16; j++) {
floatX.put((float)X.get(j));
}
}
if (surfaceTransformer != null) {
floatH1.rewind();
}
floatH2.rewind();
floatX.rewind();
}
@Override public void transform(CLImage2d srcImg, CLImage2d subImg, CLImage2d srcDotImg,
CLImage2d transImg, CLImage2d dstImg, CLImage2d maskImg,
ImageTransformer.Parameters[] parameters, boolean[] inverses,
InputData inputData, OutputData outputData) {
if (inverses != null) {
for (int i = 0; i < inverses.length; i++) {
if (inverses[i]) {
throw new UnsupportedOperationException("Inverse transform not supported.");
}
}
}
prepareTransforms(H1Buffer, H2Buffer, XBuffer, inputData.pyramidLevel, parameters);
final int dotSize = parameters[0].size();
final int localSize = parameters.length > 1 ? parameters.length : (inputData.roiWidth > 32 ? 64 : 32);
final int globalSize = JavaCVCL.alignCeil(inputData.roiWidth, localSize);
final int reduceSize = globalSize/localSize;
// allocate buffers if necessary
CLBuffer inputBuffer = inputData.getBuffer(context);
CLBuffer outputBuffer = outputData.getBuffer(context, dotSize, reduceSize);
CLEventList list = new CLEventList(1);
// setup kernel
if (surfaceTransformer != null) {
context.writeBuffer(H1Buffer, false); // upload H1
}
context.writeBuffer(H2Buffer, false); // upload H2
context.writeBuffer(XBuffer, false); // upload X
if (inputData.autoWrite) {
inputData.writeBuffer(context);
}
CLImage2d srcImg2 = projectorImageCL[inputData.pyramidLevel];
CLKernel kernel = null;
if (subImg == null) {
assert parameters.length == 1;
kernel = oneKernel.putArg(srcImg2).putArg(srcImg).putArg(dstImg == null ? transImg : dstImg).putArg(maskImg).putArg(H2Buffer);
} else if (srcDotImg == null) {
assert parameters.length == 1;
kernel = subKernel.putArg(srcImg2).putArg(srcImg).putArg(subImg).putArg(transImg).putArg(dstImg).putArg(maskImg).putArg(H2Buffer);
} else {
assert parameters.length == dotSize;
kernel = dotKernel.putArg(srcImg2).putArg(srcImg).putArg(subImg).putArg(srcDotImg).putArg(maskImg).putArg(H2Buffer);
//System.out.println(kernel.getWorkGroupSize(context.getCLCommandQueue().getDevice()));
}
if (H1Buffer != null) { kernel.putArg(H1Buffer); } else { kernel.putNullArg(nullSize); }
kernel.putArg(XBuffer).putArg(inputBuffer).putArg(outputBuffer).rewind();
context.executeKernel(kernel, inputData.roiX, 0, 0,
globalSize, 1, parameters.length,
localSize, 1, parameters.length, list); // execute program
if (reduceSize > 1) {
reduceKernel.putArg(outputBuffer).rewind();
context.executeKernel(reduceKernel, 0, reduceSize, reduceSize);
}
if (outputData.autoRead) {
outputData.readBuffer(context);
}
// CLEvent event = list.getEvent(0);
// System.out.println((event.getProfilingInfo(CLEvent.ProfilingCommand.END) -
// event.getProfilingInfo(CLEvent.ProfilingCommand.START))/1000000.0);
// long res = q.getDevice().getProfilingTimerResolution();
// System.out.println(res);
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/ProjectiveColorTransformer.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.util.Arrays;
import org.bytedeco.opencv.opencv_core.*;
import static org.bytedeco.javacv.cvkernels.*;
import static org.bytedeco.opencv.global.opencv_core.*;
/**
*
* @author Samuel Audet
*/
public class ProjectiveColorTransformer extends ProjectiveTransformer {
public ProjectiveColorTransformer(CvMat K1, CvMat K2, CvMat R, CvMat t,
CvMat n, double[] referencePoints1, double[] referencePoints2,
CvMat X, int numGains, int numBiases) {
super(K1, K2, R, t, n, referencePoints1, referencePoints2);
this.X = X == null ? null : X.clone();
this.numGains = numGains;
this.numBiases = numBiases;
}
protected static ThreadLocal
X24x4 = CvMat.createThreadLocal(4, 4),
temp3x1 = CvMat.createThreadLocal(3, 1);
protected CvMat X = null;
protected int numGains = 0, numBiases = 0;
protected CvMat[] X2 = null;
public CvMat getX() {
return X;
}
public int getNumGains() {
return numGains;
}
public int getNumBiases() {
return numBiases;
}
public void transformColor(IplImage srcImage, IplImage dstImage, CvRect roi,
int pyramidLevel, ImageTransformer.Parameters parameters, boolean inverse) {
Parameters p = ((Parameters)parameters);
if ((Arrays.equals(p.getColorParameters(), p.getIdentityColorParameters()) &&
(X == null || p.fakeIdentity)) || (X == null && numGains == 0 && numBiases == 0)) {
if (srcImage != dstImage) {
cvCopy(srcImage, dstImage);
}
return;
}
CvMat X2 = X24x4.get();
prepareColorTransform(X2, pyramidLevel, p, inverse);
X2.rows(3);
// do color transformation
if (roi == null) {
cvResetImageROI(dstImage);
} else {
cvSetImageROI(dstImage, roi);
}
X2.put(0, 3, X2.get(0, 3)*dstImage.highValue());
X2.put(1, 3, X2.get(1, 3)*dstImage.highValue());
X2.put(2, 3, X2.get(2, 3)*dstImage.highValue());
cvTransform(srcImage, dstImage, X2, null);
X2.rows(4);
}
protected void prepareColorTransform(CvMat X2, int pyramidLevel, Parameters p, boolean inverse) {
CvMat A = p.getA(), b = p.getB();
cvSetIdentity(X2);
X2.rows(3); X2.cols(3);
if (p.fakeIdentity && !inverse) {
X2.put(A);
} else if (A != null && X != null) {
cvMatMul(X, A, X2);
} else if (X == null) {
X2.put(A);
} else if (A == null) {
X2.put(X);
}
X2.rows(4); X2.cols(4);
if (b != null) {
X2.put(0, 3, b.get(0));
X2.put(1, 3, b.get(1));
X2.put(2, 3, b.get(2));
}
if (inverse) {
// CV_LU doesn't work on OpenCV 2.0 with rows > 3 ...
cvInvert(X2, X2, CV_SVD);
}
}
@Override public void transform(Data[] data, CvRect roi, ImageTransformer.Parameters[] parameters, boolean[] inverses) {
assert data.length == parameters.length;
if (kernelData == null || kernelData.capacity() < data.length) {
kernelData = new KernelData(data.length);
}
if (H == null || H.length < data.length) {
H = new CvMat[data.length];
for (int i = 0; i < H.length; i++) {
H[i] = CvMat.create(3, 3);
}
}
if (X2 == null || X2.length < data.length) {
X2 = new CvMat[data.length];
for (int i = 0; i < X2.length; i++) {
X2[i] = CvMat.create(4, 4);
}
}
for (int i = 0; i < data.length; i++) {
kernelData.position(i);
kernelData.srcImg(data[i].srcImg);
kernelData.srcImg2(null);
kernelData.subImg(data[i].subImg);
kernelData.srcDotImg(data[i].srcDotImg);
kernelData.mask(data[i].mask);
kernelData.zeroThreshold(data[i].zeroThreshold);
kernelData.outlierThreshold(data[i].outlierThreshold);
boolean inverse = inverses == null ? false : inverses[i];
prepareHomography (H[i], data[i].pyramidLevel, (Parameters)parameters[i], inverse);
prepareColorTransform(X2[i], data[i].pyramidLevel, (Parameters)parameters[i], inverse);
kernelData.H1(H[i]);
kernelData.H2(null);
kernelData.X(X2[i]);
kernelData.transImg(data[i].transImg);
kernelData.dstImg(data[i].dstImg);
kernelData.dstDstDot(data[i].dstDstDot);
}
long fullCapacity = kernelData.capacity();
kernelData.capacity(data.length);
multiWarpColorTransform(kernelData, roi, getFillColor());
kernelData.capacity(fullCapacity);
for (int i = 0; i < data.length; i++) {
kernelData.position(i);
data[i].dstCount = kernelData.dstCount();
data[i].dstCountZero = kernelData.dstCountZero();
data[i].dstCountOutlier = kernelData.dstCountOutlier();
data[i].srcDstDot = kernelData.srcDstDot();
}
}
@Override public Parameters createParameters() {
return new Parameters();
}
public class Parameters extends ProjectiveTransformer.Parameters {
protected Parameters() {
identityColorParameters = new double[numGains + numBiases];
if (numGains > 0) {
A = CvMat.create(3, 3);
cvSetIdentity(A);
}
if (numBiases > 0) {
b = CvMat.create(3, 1);
cvSetZero(b);
}
switch (numGains) {
case 0: assert (A == null); break;
case 1: identityColorParameters[0] =
(A.get(0) + A.get(4) + A.get(8))/3; break;
case 3: identityColorParameters[0] = A.get(0);
identityColorParameters[1] = A.get(4);
identityColorParameters[2] = A.get(8); break;
case 9: A.get(0, identityColorParameters, 0, 9); break;
default: assert (false);
}
switch (numBiases) {
case 0: assert (b == null); break;
case 1: identityColorParameters[numGains] =
(b.get(0) + b.get(1) + b.get(2))/3; break;
case 3: b.get(0, identityColorParameters, numGains, 3); break;
default: assert (false);
}
reset(false);
}
protected double[] colorParameters = null, identityColorParameters = null;
private CvMat A = null, b = null;
public double[] getColorParameters() {
return colorParameters;
}
public double[] getIdentityColorParameters() {
return identityColorParameters;
}
@Override public int size() {
return super.size() + numGains + numBiases;
}
@Override public double get(int i) {
int s = super.size();
if (i < s) {
return super.get(i);
} else {
return colorParameters[i-s];
}
}
@Override public void set(int i, double p) {
int s = super.size();
if (i < s) {
super.set(i, p);
} else {
if (colorParameters[i-s] != p) {
colorParameters[i-s] = p;
setUpdateNeeded(true);
}
}
}
@Override public void reset(boolean asIdentity) {
super.reset(asIdentity);
resetColor(asIdentity);
}
public void resetColor(boolean asIdentity) {
if (identityColorParameters != null) {
if (!Arrays.equals(colorParameters, identityColorParameters) ||
fakeIdentity != asIdentity) {
fakeIdentity = asIdentity;
colorParameters = identityColorParameters.clone();
setUpdateNeeded(true);
}
}
}
// @Override public boolean addDelta(int i, double scale) {
// int s = super.size();
// if (i < s) {
// return super.addDelta(i, scale);
// } else {
// // gradient varies linearly with intensity, so
// // the increment value is not very important, but
// // referenceCameraImage is good only for the value 1,
// // so let's use that
// int channel = i-s;
// colorParameters[channel] += scale;//1;
// setUpdateNeeded(true);
// return false;
// }
// }
@Override public void compose(ImageTransformer.Parameters p1, boolean inverse1,
ImageTransformer.Parameters p2, boolean inverse2) {
super.compose(p1, inverse1, p2, inverse2);
composeColor(p1, inverse1, p2, inverse2);
}
public void composeColor(ImageTransformer.Parameters p1, boolean inverse1,
ImageTransformer.Parameters p2, boolean inverse2) {
assert (!inverse1 && !inverse2);
Parameters pp1 = (Parameters)p1, pp2 = (Parameters)p2;
CvMat A1 = pp1.getA(), b1 = pp1.getB();
CvMat A2 = pp2.getA(), b2 = pp2.getB();
if (b != null) {
if (pp1.fakeIdentity && X != null) {
CvMat temp = temp3x1.get();
cvMatMul(X, b1, temp);
b1 = temp;
}
if (A2 == null && b2 == null) {
cvCopy(b1, b);
} else if (b1 == null) {
cvCopy(b2, b);
} else if (b2 == null) {
cvMatMul(A2, b1, b);
} else {
cvGEMM(A2, b1, 1.0, b2, 1.0, b, 0);
}
}
if (A != null) {
if (A1 == null) {
cvCopy(A2, A);
} else if (A2 == null) {
cvCopy(A1, A);
} else {
cvMatMul(A2, A1, A);
}
}
switch (numGains) {
case 0: assert (A == null); break;
case 1: colorParameters[0] =
(A.get(0) + A.get(4) + A.get(8))/3; break;
case 3: colorParameters[0] = A.get(0);
colorParameters[1] = A.get(4);
colorParameters[2] = A.get(8); break;
case 9: A.get(0, colorParameters, 0, 9); break;
default: assert (false);
}
switch (numBiases) {
case 0: assert (b == null); break;
case 1: colorParameters[numGains] =
(b.get(0) + b.get(1) + b.get(2))/3; break;
case 3: b.get(0, colorParameters, numGains, 3); break;
default: assert (false);
}
}
public CvMat getA() {
update();
return A;
}
public CvMat getB() {
update();
return b;
}
@Override protected void update() {
if (!isUpdateNeeded()) {
return;
}
switch (numGains) {
case 0: assert (A == null); break;
case 1: A.put(0, colorParameters[0]);
A.put(4, colorParameters[0]);
A.put(8, colorParameters[0]); break;
case 3: A.put(0, colorParameters[0]);
A.put(4, colorParameters[1]);
A.put(8, colorParameters[2]); break;
case 9: A.put(0, colorParameters, 0, 9); break;
default: assert (false);
}
switch (numBiases) {
case 0: assert (b == null); break;
case 1: b.put(0, colorParameters[numGains]);
b.put(1, colorParameters[numGains]);
b.put(2, colorParameters[numGains]); break;
case 3: b.put(0, colorParameters, numGains, 3); break;
default: assert (false);
}
super.update();
setUpdateNeeded(false);
}
@Override public Parameters clone() {
Parameters p = new Parameters();
p.set(this);
return p;
}
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/ProjectiveColorTransformerCL.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import com.jogamp.opencl.CLBuffer;
import com.jogamp.opencl.CLEventList;
import com.jogamp.opencl.CLImage2d;
import com.jogamp.opencl.CLKernel;
import java.nio.FloatBuffer;
import org.bytedeco.opencv.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_core.*;
/**
*
* @author Samuel Audet
*/
public class ProjectiveColorTransformerCL extends ProjectiveColorTransformer implements ImageTransformerCL {
public ProjectiveColorTransformerCL(JavaCVCL context, CvMat K1, CvMat K2, CvMat R, CvMat t,
CvMat n, double[] referencePoints1, double[] referencePoints2,
CvMat X, int numGains, int numBiases) {
super(K1, K2, R, t, n, referencePoints1, referencePoints2, X, numGains, numBiases);
final int dotSize = createParameters().size();
this.context = context;
this.HBuffer = context.getCLContext().createFloatBuffer(dotSize*9, CLBuffer.Mem.READ_ONLY);
this.XBuffer = context.getCLContext().createFloatBuffer(dotSize*16, CLBuffer.Mem.READ_ONLY);
if (getClass() == ProjectiveColorTransformerCL.class) {
CLKernel[] kernels = context.buildKernels(
JavaCVCL.fastCompilerOptions + " -DDOT_SIZE=" + dotSize,
"ImageTransformer.cl:ProjectiveColorTransformer.cl",
"transformOne", "transformSub", "transformDot", "reduceOutputData");
oneKernel = kernels[0];
subKernel = kernels[1];
dotKernel = kernels[2];
reduceKernel = kernels[3];
}
}
protected final JavaCVCL context;
protected final CLBuffer HBuffer, XBuffer;
private CLKernel oneKernel, subKernel, dotKernel, reduceKernel;
public JavaCVCL getContext() {
return context;
}
protected void prepareHomographies(CLBuffer HBuffer, int pyramidLevel,
ImageTransformer.Parameters[] parameters, boolean[] inverses) {
FloatBuffer floatH = (FloatBuffer)HBuffer.getBuffer().rewind();
CvMat H = H3x3.get();
for (int i = 0; i < parameters.length; i++) {
prepareHomography(H, pyramidLevel, (ProjectiveColorTransformer.Parameters)parameters[i],
inverses == null ? false : inverses[i]);
for (int j = 0; j < 9; j++) {
floatH.put((float)H.get(j));
}
}
floatH.rewind();
}
protected void prepareColorTransforms(CLBuffer XBuffer, int pyramidLevel,
ImageTransformer.Parameters[] parameters, boolean[] inverses) {
FloatBuffer floatX = (FloatBuffer)XBuffer.getBuffer().rewind();
CvMat X2 = X24x4.get();
for (int i = 0; i < parameters.length; i++) {
prepareColorTransform(X2, pyramidLevel, (ProjectiveColorTransformer.Parameters)parameters[i],
inverses == null ? false : inverses[i]);
for (int j = 0; j < 16; j++) {
floatX.put((float)X2.get(j));
}
}
floatX.rewind();
}
@Override public void transform(CLImage2d srcImg, CLImage2d subImg, CLImage2d srcDotImg,
CLImage2d transImg, CLImage2d dstImg, CLImage2d maskImg,
ImageTransformer.Parameters[] parameters, boolean[] inverses,
InputData inputData, OutputData outputData) {
prepareHomographies(HBuffer, inputData.pyramidLevel, parameters, inverses);
prepareColorTransforms(XBuffer, inputData.pyramidLevel, parameters, inverses);
final int dotSize = parameters[0].size();
final int localSize = parameters.length > 1 ? parameters.length : (inputData.roiWidth > 32 ? 64 : 32);
final int globalSize = JavaCVCL.alignCeil(inputData.roiWidth, localSize);
final int reduceSize = globalSize/localSize;
// allocate buffers if necessary
CLBuffer inputBuffer = inputData.getBuffer(context);
CLBuffer outputBuffer = outputData.getBuffer(context, dotSize, reduceSize);
CLEventList list = new CLEventList(1);
// setup kernel
context.writeBuffer(HBuffer, false); // upload H
context.writeBuffer(XBuffer, false); // upload X
if (inputData.autoWrite) {
inputData.writeBuffer(context);
}
CLKernel kernel = null;
if (subImg == null) {
assert parameters.length == 1;
kernel = oneKernel.putArg(srcImg).putArg(dstImg == null ? transImg : dstImg).putArg(maskImg)
.putArg(HBuffer).putArg(XBuffer).putArg(inputBuffer).putArg(outputBuffer).rewind();
} else if (srcDotImg == null) {
assert parameters.length == 1;
kernel = subKernel.putArg(srcImg).putArg(subImg).putArg(transImg).putArg(dstImg).putArg(maskImg)
.putArg(HBuffer).putArg(XBuffer).putArg(inputBuffer).putArg(outputBuffer).rewind();
} else {
assert parameters.length == dotSize;
kernel = dotKernel.putArg(srcImg).putArg(subImg).putArg(srcDotImg).putArg(maskImg)
.putArg(HBuffer).putArg(XBuffer).putArg(inputBuffer).putArg(outputBuffer).rewind();
}
context.executeKernel(kernel, inputData.roiX, 0, 0,
globalSize, 1, parameters.length,
localSize, 1, parameters.length, list); // execute program
if (reduceSize > 1) {
reduceKernel.putArg(outputBuffer).rewind();
context.executeKernel(reduceKernel, 0, reduceSize, reduceSize);
}
if (outputData.autoRead) {
outputData.readBuffer(context);
}
// CLEvent event = list.getEvent(0);
// System.out.println((event.getProfilingInfo(CLEvent.ProfilingCommand.END) -
// event.getProfilingInfo(CLEvent.ProfilingCommand.START))/1000000.0);
// long res = q.getDevice().getProfilingTimerResolution();
// System.out.println(res);
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/ProjectiveDevice.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.io.File;
import java.nio.FloatBuffer;
import java.util.Arrays;
import org.bytedeco.opencv.global.opencv_core;
import org.bytedeco.opencv.opencv_calib3d.*;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_calib3d.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
*
* @author Samuel Audet
*/
public class ProjectiveDevice {
public ProjectiveDevice(String name) {
Settings s = new Settings();
s.name = name;
setSettings(s);
}
public ProjectiveDevice(String name, File file) throws Exception {
this(name);
readParameters(file);
}
public ProjectiveDevice(String name, String filename) throws Exception {
this(name);
readParameters(filename);
}
public ProjectiveDevice(String name, FileStorage fs) throws Exception {
this(name);
readParameters(fs);
}
public ProjectiveDevice(Settings settings) throws Exception {
setSettings(settings);
if (settings instanceof CalibratedSettings) {
readParameters(((CalibratedSettings)settings).parametersFile);
}
}
public static class Settings extends BaseChildSettings {
public Settings() { }
public Settings(ProjectiveDevice.Settings settings) {
this.name = settings.name;
this.responseGamma = settings.responseGamma;
// this.nominalDistance = settings.nominalDistance;
}
String name = "";
double responseGamma = 0.0;
// double nominalDistance = 20000;
@Override public String getName() {
return name;
}
public void setName(String name) {
firePropertyChange("name", this.name, this.name = name);
}
public double getResponseGamma() {
return responseGamma;
}
public void setResponseGamma(double responseGamma) {
this.responseGamma = responseGamma;
}
// public double getNominalDistance() {
// return nominalDistance;
// }
// public void setNominalDistance(double nominalDistance) {
// this.nominalDistance = nominalDistance;
// }
}
public static class CalibrationSettings extends Settings {
public CalibrationSettings() { }
public CalibrationSettings(ProjectiveDevice.CalibrationSettings settings) {
super(settings);
this.initAspectRatio = settings.initAspectRatio;
this.flags = settings.flags;
}
double initAspectRatio = 1.0;
int flags = CV_CALIB_FIX_K3 | CV_CALIB_FIX_K4 |
CV_CALIB_FIX_K5 | CV_CALIB_FIX_K6 | CV_CALIB_FIX_INTRINSIC;
public double getInitAspectRatio() {
return initAspectRatio;
}
public void setInitAspectRatio(double initAspectRatio) {
this.initAspectRatio = initAspectRatio;
}
public boolean isUseIntrinsicGuess() {
return (flags & CV_CALIB_USE_INTRINSIC_GUESS) != 0;
}
public void setUseIntrinsicGuess(boolean useIntrinsicGuess) {
if (useIntrinsicGuess) {
flags |= CV_CALIB_USE_INTRINSIC_GUESS;
} else {
flags &= ~CV_CALIB_USE_INTRINSIC_GUESS;
}
}
public boolean isFixAspectRatio() {
return (flags & CV_CALIB_FIX_ASPECT_RATIO) != 0;
}
public void setFixAspectRatio(boolean fixAspectRatio) {
if (fixAspectRatio) {
flags |= CV_CALIB_FIX_ASPECT_RATIO;
} else {
flags &= ~CV_CALIB_FIX_ASPECT_RATIO;
}
}
public boolean isFixPrincipalPoint() {
return (flags & CV_CALIB_FIX_PRINCIPAL_POINT) != 0;
}
public void setFixPrincipalPoint(boolean fixPrincipalPoint) {
if (fixPrincipalPoint) {
flags |= CV_CALIB_FIX_PRINCIPAL_POINT;
} else {
flags &= ~CV_CALIB_FIX_PRINCIPAL_POINT;
}
}
public boolean isZeroTangentDist() {
return (flags & CV_CALIB_ZERO_TANGENT_DIST) != 0;
}
public void setZeroTangentDist(boolean zeroTangentDist) {
if (zeroTangentDist) {
flags |= CV_CALIB_ZERO_TANGENT_DIST;
} else {
flags &= ~CV_CALIB_ZERO_TANGENT_DIST;
}
}
public boolean isFixFocalLength() {
return (flags & CV_CALIB_FIX_FOCAL_LENGTH) != 0;
}
public void setFixFocalLength(boolean fixFocalLength) {
if (fixFocalLength) {
flags |= CV_CALIB_FIX_FOCAL_LENGTH;
} else {
flags &= ~CV_CALIB_FIX_FOCAL_LENGTH;
}
}
public boolean isFixK1() {
return (flags & CV_CALIB_FIX_K1) != 0;
}
public void setFixK1(boolean fixK1) {
if (fixK1) {
flags |= CV_CALIB_FIX_K1;
} else {
flags &= ~CV_CALIB_FIX_K1;
}
}
public boolean isFixK2() {
return (flags & CV_CALIB_FIX_K2) != 0;
}
public void setFixK2(boolean fixK2) {
if (fixK2) {
flags |= CV_CALIB_FIX_K2;
} else {
flags &= ~CV_CALIB_FIX_K2;
}
}
public boolean isFixK3() {
return (flags & CV_CALIB_FIX_K3) != 0;
}
public void setFixK3(boolean fixK3) {
if (fixK3) {
flags |= CV_CALIB_FIX_K3;
} else {
flags &= ~CV_CALIB_FIX_K3;
}
}
public boolean isFixK4() {
return (flags & CV_CALIB_FIX_K4) != 0;
}
public void setFixK4(boolean fixK4) {
if (fixK4) {
flags |= CV_CALIB_FIX_K4;
} else {
flags &= ~CV_CALIB_FIX_K4;
}
}
public boolean isFixK5() {
return (flags & CV_CALIB_FIX_K5) != 0;
}
public void setFixK5(boolean fixK5) {
if (fixK5) {
flags |= CV_CALIB_FIX_K5;
} else {
flags &= ~CV_CALIB_FIX_K5;
}
}
public boolean isFixK6() {
return (flags & CV_CALIB_FIX_K6) != 0;
}
public void setFixK6(boolean fixK6) {
if (fixK6) {
flags |= CV_CALIB_FIX_K6;
} else {
flags &= ~CV_CALIB_FIX_K6;
}
}
public boolean isRationalModel() {
return (flags & CV_CALIB_RATIONAL_MODEL) != 0;
}
public void setRationalModel(boolean rationalModel) {
if (rationalModel) {
flags |= CV_CALIB_RATIONAL_MODEL;
} else {
flags &= ~CV_CALIB_RATIONAL_MODEL;
}
}
public boolean isStereoFixIntrinsic() {
return (flags & CV_CALIB_FIX_INTRINSIC) != 0;
}
public void setStereoFixIntrinsic(boolean stereoFixIntrinsic) {
if (stereoFixIntrinsic) {
flags |= CV_CALIB_FIX_INTRINSIC;
} else {
flags &= ~CV_CALIB_FIX_INTRINSIC;
}
}
public boolean isStereoSameFocalLength() {
return (flags & CV_CALIB_SAME_FOCAL_LENGTH) != 0;
}
public void setStereoSameFocalLength(boolean stereoSameFocalLength) {
if (stereoSameFocalLength) {
flags |= CV_CALIB_SAME_FOCAL_LENGTH;
} else {
flags &= ~CV_CALIB_SAME_FOCAL_LENGTH;
}
}
}
public static class CalibratedSettings extends Settings {
public CalibratedSettings() { }
public CalibratedSettings(ProjectiveDevice.CalibratedSettings settings) {
super(settings);
this.parametersFile = settings.parametersFile;
}
File parametersFile = new File("calibration.yaml");
public File getParametersFile() {
return parametersFile;
}
public void setParametersFile(File parametersFile) {
this.parametersFile = parametersFile;
}
public String getParametersFilename() {
return parametersFile == null ? "" : parametersFile.getPath();
}
public void setParametersFilename(String parametersFilename) {
this.parametersFile = parametersFilename == null ||
parametersFilename.length() == 0 ? null : new File(parametersFilename);
}
}
private Settings settings;
public Settings getSettings() {
return settings;
}
public void setSettings(Settings settings) {
this.settings = settings;
}
public int imageWidth = 0, imageHeight = 0;
public CvMat cameraMatrix = null, distortionCoeffs = null,
extrParams = null, reprojErrs = null;
public double avgReprojErr, maxReprojErr;
// public double nominalDistance = 0;
public CvMat R = null, T = null, E = null, F = null;
public double avgEpipolarErr, maxEpipolarErr;
public String colorOrder = "BGR";
public CvMat colorMixingMatrix = null, additiveLight = null;
public double avgColorErr, colorR2 = 1.0;
public void rescale(int imageWidth, int imageHeight) {
if ((imageWidth != this.imageWidth || imageHeight != this.imageHeight) &&
cameraMatrix != null) {
double sx = (double)imageWidth /this.imageWidth;
double sy = (double)imageHeight/this.imageHeight;
cameraMatrix.put(0, sx*cameraMatrix.get(0));
cameraMatrix.put(1, sx*cameraMatrix.get(1));
cameraMatrix.put(2, sx*cameraMatrix.get(2));
cameraMatrix.put(3, sy*cameraMatrix.get(3));
cameraMatrix.put(4, sy*cameraMatrix.get(4));
cameraMatrix.put(5, sy*cameraMatrix.get(5));
this.imageWidth = imageWidth;
this.imageHeight = imageHeight;
int p = mapsPyramidLevel;
undistortMaps1[p] = undistortMaps2[p] = distortMaps1[p] = distortMaps2[p] = null;
}
}
public int[] getRGBColorOrder() {
int[] order = new int[3];
for (int i = 0; i < 3; i++) {
switch (Character.toUpperCase(colorOrder.charAt(i))) {
case 'B': order[i] = 2; break;
case 'G': order[i] = 1; break;
case 'R': order[i] = 0; break;
default: assert (false);
}
}
return order;
}
// public double getNominalDistance(int objectWidth, int objectHeight) {
// double f = (cameraMatrix.get(0)+cameraMatrix.get(4))/(2*cameraMatrix.get(8));
// double imageSize = Math.sqrt(imageWidth *imageWidth +
// imageHeight*imageHeight);
// double objectSize = Math.sqrt(objectWidth *objectWidth +
// objectHeight*objectHeight);
// return f*objectSize/imageSize;
// }
// public double getNominalDistance(MarkedPlane board) {
// return getNominalDistance(board.getWidth(), board.getHeight());
// }
//Compensates for radial and tangential distortion. Model From Oulu university.
//Code ported from Camera Calibration Toolbox for Matlab by Jean-Yves Bouguet
//http://www.vision.caltech.edu/bouguetj/calib_doc/
//function name: comp_distortion_oulu()
//
//INPUT: xd: distorted (normalized) point coordinates in the image plane (2xN matrix)
// k: Distortion coefficients (radial and tangential) (4x1 vector)
//
//OUTPUT: x: undistorted (normalized) point coordinates in the image plane (2xN matrix)
//
//Method: Iterative method for compensation.
//
//NOTE: This compensation has to be done after the subtraction
// of the principal point, and division by the focal length.
public static double[] undistort(double[] xd, double[] k) {
double k1 = k[0];
double k2 = k[1];
double k3 = k.length > 4 ? k[4] : 0;
// XXX: need to use those new distortion coefficients
double k4 = k.length > 5 ? k[5] : 0;
double k5 = k.length > 6 ? k[6] : 0;
double k6 = k.length > 7 ? k[7] : 0;
double p1 = k[2];
double p2 = k[3];
double[] xu = xd.clone(); // initial guess
for (int i = 0; i < xd.length/2; i++) {
double x = xu[i*2], y = xu[i*2 + 1];
double xo = xd[i*2], yo = xd[i*2 + 1];
for (int j = 0; j < 20; j++) {
double r_2 = x*x + y*y;
double k_radial = 1 + k1*r_2 + k2*r_2*r_2 + k3*r_2*r_2*r_2;
double delta_x = 2*p1*x*y + p2*(r_2 + 2*x*x);
double delta_y = p1*(r_2 + 2*y*y) + 2*p2*x*y;
x = (xo - delta_x)/k_radial;
y = (yo - delta_y)/k_radial;
}
xu[i*2] = x; xu[i*2 + 1] = y;
}
return xu;
}
public double[] undistort(double ... x) {
double[] xn = normalize(x, cameraMatrix);
double[] xu = undistort(xn, distortionCoeffs.get());
return unnormalize(xu, cameraMatrix);
}
public static double[] distort(double[] xu, double[] k) {
double k1 = k[0];
double k2 = k[1];
double k3 = k.length > 4 ? k[4] : 0;
// XXX: need to use those new distortion coefficients
double k4 = k.length > 5 ? k[5] : 0;
double k5 = k.length > 6 ? k[6] : 0;
double k6 = k.length > 7 ? k[7] : 0;
double p1 = k[2];
double p2 = k[3];
double[] xd = xu.clone();
for (int i = 0; i < xu.length/2; i++) {
double x = xu[i*2 ],
y = xu[i*2 + 1];
double r_2 = x*x + y*y;
double k_radial = 1 + k1*r_2 + k2*r_2*r_2 + k3*r_2*r_2*r_2;
double delta_x = 2*p1*x*y + p2*(r_2 + 2*x*x);
double delta_y = p1*(r_2 + 2*y*y) + 2*p2*x*y;
xd[i*2 ] = x*k_radial + delta_x;
xd[i*2 + 1] = y*k_radial + delta_y;
}
return xd;
}
public double[] distort(double ... x) {
double[] xn = normalize(x, cameraMatrix);
double[] xd = distort(xn, distortionCoeffs.get());
return unnormalize(xd, cameraMatrix);
}
public static double[] normalize(double[] xu, CvMat K) {
double[] xn = xu.clone();
double fx = K.get(0)/K.get(8);
double fy = K.get(4)/K.get(8);
double dx = K.get(2)/K.get(8);
double dy = K.get(5)/K.get(8);
double s = K.get(1)/K.get(8);
for (int i = 0; i < xu.length/2; i++) {
xn[i*2 ] = (xu[i*2 ] - dx)/fx - s*(xu[i*2 + 1] + dy)/(fx*fy);
xn[i*2 + 1] = (xu[i*2 + 1] - dy)/fy;
}
return xn;
}
public static double[] unnormalize(double[] xn, CvMat K) {
double[] xu = xn.clone();
double fx = K.get(0)/K.get(8);
double fy = K.get(4)/K.get(8);
double dx = K.get(2)/K.get(8);
double dy = K.get(5)/K.get(8);
double s = K.get(1)/K.get(8);
for (int i = 0; i < xn.length/2; i++) {
xu[i*2 ] = fx*xn[i*2 ] + dx + s*xn[i*2 + 1];
xu[i*2 + 1] = fy*xn[i*2 + 1] + dy;
}
return xu;
}
private boolean fixedPointMaps = false;
private int mapsPyramidLevel = 0;
private IplImage[] undistortMaps1 = { null }, undistortMaps2 = { null };
private IplImage[] distortMaps1 = { null }, distortMaps2 = { null };
private IplImage tempImage = null;
public boolean isFixedPointMaps() {
return fixedPointMaps;
}
public void setFixedPointMaps(boolean fixedPointMaps) {
if (this.fixedPointMaps != fixedPointMaps) {
this.fixedPointMaps = fixedPointMaps;
int p = mapsPyramidLevel;
undistortMaps1[p] = undistortMaps2[p] = distortMaps1[p] = distortMaps2[p] = null;
}
}
public int getMapsPyramidLevel() {
return mapsPyramidLevel;
}
public void setMapsPyramidLevel(int mapsPyramidLevel) {
if (this.mapsPyramidLevel != mapsPyramidLevel) {
this.mapsPyramidLevel = mapsPyramidLevel;
int p = mapsPyramidLevel;
if (p >= undistortMaps1.length || p >= undistortMaps2.length ||
p >= distortMaps1.length || p >= distortMaps2.length) {
undistortMaps1 = Arrays.copyOf(undistortMaps1, p+1);
undistortMaps2 = Arrays.copyOf(undistortMaps2, p+1);
distortMaps1 = Arrays.copyOf(distortMaps1, p+1);
distortMaps2 = Arrays.copyOf(distortMaps2, p+1);
}
}
}
private void initUndistortMaps() {
//cvUndistort2(src, dst, cameraMatrix, distortionCoeffs);
int p = mapsPyramidLevel;
if (undistortMaps1[p] == null || undistortMaps2[p] == null) {
if (fixedPointMaps) {
undistortMaps1[p] = IplImage.create(imageWidth, imageHeight, IPL_DEPTH_16S, 2);
undistortMaps2[p] = IplImage.create(imageWidth, imageHeight, IPL_DEPTH_16U, 1);
} else {
undistortMaps1[p] = IplImage.create(imageWidth, imageHeight, IPL_DEPTH_32F, 1);
undistortMaps2[p] = IplImage.create(imageWidth, imageHeight, IPL_DEPTH_32F, 1);
}
Mat A = cvarrToMat(cameraMatrix);
Mat m1 = cvarrToMat(undistortMaps1[p]);
Mat m2 = cvarrToMat(undistortMaps2[p]);
initUndistortRectifyMap(A, cvarrToMat(distortionCoeffs), new Mat(), A, m1.size(), m1.type(), m1, m2);
if (mapsPyramidLevel > 0) {
IplImage map1 = undistortMaps1[p];
IplImage map2 = undistortMaps2[p];
int w = imageWidth >> p;
int h = imageHeight >> p;
undistortMaps1[p] = IplImage.create(w, h, map1.depth(), map1.nChannels());
undistortMaps2[p] = IplImage.create(w, h, map2.depth(), map2.nChannels());
cvResize(map1, undistortMaps1[p], CV_INTER_NN);
cvResize(map2, undistortMaps2[p], CV_INTER_NN);
// FloatBuffer m1 = map1.getFloatBuffer();
// FloatBuffer n1 = undistortMaps1[p].getFloatBuffer();
// for (int i = 0; i < 8; i++) {
// System.out.println(m1.get(1280*2*i) - n1.get(640*i));
// }
}
}
// if (undistortMap1 == null || undistortMap2 == null) {
// IplImage mapx = IplImage.create(imageWidth, imageHeight, IPL_DEPTH_32F, 1);
// IplImage mapy = IplImage.create(imageWidth, imageHeight, IPL_DEPTH_32F, 1);
// FloatBuffer bufx = mapx.getFloatBuffer();
// FloatBuffer bufy = mapy.getFloatBuffer();
// int width = mapx.width();
// int height = mapx.height();
// for (int y = 0; y < height; y++) {
// for (int x = 0; x < width; x++) {
// double[] undistxy = undistort(x, y);
// bufx.put((float)undistxy[0]);
// bufy.put((float)undistxy[1]);
// }
// }
// if (useFixedPointMaps) {
// undistortMap1 = IplImage.create(imageWidth, imageHeight, IPL_DEPTH_16S, 2);
// undistortMap2 = IplImage.create(imageWidth, imageHeight, is20or21 ? IPL_DEPTH_16U : IPL_DEPTH_16S, 1);
// cvConvertMaps(mapx, mapy, undistortMap1, undistortMap2);
// mapx.release();
// mapy.release();
// } else {
// undistortMap1 = mapx;
// undistortMap2 = mapy;
// }
// }
}
public IplImage getUndistortMap1() {
initUndistortMaps();
return undistortMaps1[mapsPyramidLevel];
}
public IplImage getUndistortMap2() {
initUndistortMaps();
return undistortMaps2[mapsPyramidLevel];
}
public void undistort(IplImage src, IplImage dst) {
if (src != null && dst != null) {
initUndistortMaps();
cvRemap(src, dst, undistortMaps1[mapsPyramidLevel], undistortMaps2[mapsPyramidLevel],
CV_INTER_LINEAR | CV_WARP_FILL_OUTLIERS, CvScalar.ZERO);
}
}
public IplImage undistort(IplImage image) {
if (image != null) {
initUndistortMaps();
tempImage = IplImage.createIfNotCompatible(tempImage, image);
cvResetImageROI(tempImage);
cvRemap(image, tempImage, undistortMaps1[mapsPyramidLevel], undistortMaps2[mapsPyramidLevel],
CV_INTER_LINEAR | CV_WARP_FILL_OUTLIERS, CvScalar.ZERO);
return tempImage;
}
return null;
}
private void initDistortMaps() {
int p = mapsPyramidLevel;
if (distortMaps1[p] == null || distortMaps2[p] == null) {
IplImage mapx = IplImage.create(imageWidth, imageHeight, IPL_DEPTH_32F, 1);
IplImage mapy = IplImage.create(imageWidth, imageHeight, IPL_DEPTH_32F, 1);
FloatBuffer bufx = mapx.getFloatBuffer();
FloatBuffer bufy = mapy.getFloatBuffer();
int width = mapx.width();
int height = mapx.height();
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
double[] distxy = undistort(x, y);
bufx.put((float)distxy[0]);
bufy.put((float)distxy[1]);
}
}
if (fixedPointMaps) {
distortMaps1[p] = IplImage.create(imageWidth, imageHeight, IPL_DEPTH_16S, 2);
distortMaps2[p] = IplImage.create(imageWidth, imageHeight, IPL_DEPTH_16U /* IPL_DEPTH_16S */, 1);
cvConvertMaps(mapx, mapy, distortMaps1[p], distortMaps2[p]);
mapx.release();
mapy.release();
} else {
distortMaps1[p] = mapx;
distortMaps2[p] = mapy;
}
if (mapsPyramidLevel > 0) {
IplImage map1 = distortMaps1[p];
IplImage map2 = distortMaps2[p];
int w = imageWidth >> p;
int h = imageHeight >> p;
distortMaps1[p] = IplImage.create(w, h, map1.depth(), map1.nChannels());
distortMaps2[p] = IplImage.create(w, h, map2.depth(), map2.nChannels());
cvResize(map1, distortMaps1[p], CV_INTER_NN);
cvResize(map2, distortMaps2[p], CV_INTER_NN);
}
}
}
public IplImage getDistortMap1() {
initDistortMaps();
return distortMaps1[mapsPyramidLevel];
}
public IplImage getDistortMap2() {
initDistortMaps();
return distortMaps2[mapsPyramidLevel];
}
public void distort(IplImage src, IplImage dst) {
if (src != null && dst != null) {
initDistortMaps();
cvRemap(src, dst, distortMaps1[mapsPyramidLevel], distortMaps2[mapsPyramidLevel],
CV_INTER_LINEAR | CV_WARP_FILL_OUTLIERS, CvScalar.ZERO);
}
}
public IplImage distort(IplImage image) {
if (image != null) {
initDistortMaps();
tempImage = IplImage.createIfNotCompatible(tempImage, image);
cvRemap(image, tempImage, distortMaps1[mapsPyramidLevel], distortMaps2[mapsPyramidLevel],
CV_INTER_LINEAR | CV_WARP_FILL_OUTLIERS, CvScalar.ZERO);
return tempImage;
}
return null;
}
// B = [ (-R^T t)*plane^T - plane^T*(-R^T t) I ] [ (K R)^-1 ]
// [ 0 0 0 ]
// where plane = [ n | d ]
// B is a 4x3 matrix
private static ThreadLocal
temp3x3 = CvMat.createThreadLocal(3, 3);
public CvMat getBackProjectionMatrix(CvMat n, double d, CvMat B) {
CvMat temp = temp3x3.get();
temp.cols(1); temp.step(temp.step()/3);
B.rows(3);
cvGEMM(R, T, -1, null, 0, temp, CV_GEMM_A_T);
cvGEMM(temp, n, 1, null, 0, B, CV_GEMM_B_T);
double a = cvDotProduct(n, temp) + d;
B.put(0, B.get(0) - a);
B.put(4, B.get(4) - a);
B.put(8, B.get(8) - a);
B.rows(4);
temp.cols(3); temp.step(temp.step()*3);
B.put(9, n.get());
cvMatMul(cameraMatrix, R, temp);
cvInvert(temp, temp, CV_LU);
cvMatMul(B, temp, B);
cvConvertScale(B, B, 1/B.get(11), 0);
return B;
}
private static ThreadLocal
B4x3 = CvMat.createThreadLocal(4, 3),
a4x1 = CvMat.createThreadLocal(4, 1),
t3x1 = CvMat.createThreadLocal(3, 1);
public CvMat getFrontoParallelH(double[] roipts, CvMat n, CvMat H) {
CvMat B = B4x3.get(), a = a4x1.get(), t = t3x1.get();
// compute rotation from n to z-axis
double s = Math.signum(n.get(2));
double[] dir = JavaCV.unitize(-s*n.get(1), s*n.get(0));
double theta = Math.acos(s*n.get(2)/JavaCV.norm(n.get()));
t.put(theta*dir[0], theta*dir[1], 0.0);
Rodrigues(cvarrToMat(t), cvarrToMat(H), null);
// and from z-axis to device axis
cvMatMul(R, H, H);
double x = 0, y = 0;
if (roipts != null) {
// find the middle of the ROI
double x1 = roipts[0], y1 = roipts[1],
x2 = roipts[4], y2 = roipts[5],
x3 = roipts[2], y3 = roipts[3],
x4 = roipts[6], y4 = roipts[7];
double u = ((x4-x3)*(y1-y3) - (y4-y3)*(x1-x3))/
((y4-y3)*(x2-x1) - (x4-x3)*(y2-y1));
x = x1 + u*(x2-x1);
y = y1 + u*(y2-y1);
}
// compute 3D point from the middle of the ROI, and
// form optimal homography for n
getBackProjectionMatrix(n, -1, B);
t.put(x, y, 1);
cvMatMul(B, t, a);
H.put(2, a.get(0)/a.get(3));
H.put(5, a.get(1)/a.get(3));
H.put(8, a.get(2)/a.get(3));
return H;
}
// returns a homography that can be applied to pixel coordinates
private static ThreadLocal
relativeR3x3 = CvMat.createThreadLocal(3, 3),
relativeT3x1 = CvMat.createThreadLocal(3, 1),
R13x3 = CvMat.createThreadLocal(3, 3), P13x4 = CvMat.createThreadLocal(3, 4),
R23x3 = CvMat.createThreadLocal(3, 3), P23x4 = CvMat.createThreadLocal(3, 4);
public CvMat getRectifyingHomography(ProjectiveDevice peer, CvMat H) {
CvMat relativeR = relativeR3x3.get(), relativeT = relativeT3x1.get();
cvGEMM(R, peer.R, 1, null, 0, relativeR, CV_GEMM_B_T);
cvGEMM(relativeR, peer.T, -1, T, 1, relativeT, 0);
CvMat R1 = R13x3.get(); CvMat P1 = P13x4.get();
CvMat R2 = R23x3.get(); CvMat P2 = P23x4.get();
Size imageSize = new Size((peer.imageWidth + imageWidth )/2,
(peer.imageHeight + imageHeight)/2); // ?
stereoRectify(cvarrToMat(peer.cameraMatrix), cvarrToMat(peer.distortionCoeffs),
cvarrToMat( cameraMatrix), cvarrToMat( distortionCoeffs),
imageSize, cvarrToMat(relativeR), cvarrToMat(relativeT),
cvarrToMat(R1), cvarrToMat(R2), cvarrToMat(P1), cvarrToMat(P2),
new Mat(), 0, -1, new Size(), null, null);
cvMatMul(cameraMatrix, R2, R2);
cvInvert(cameraMatrix, R1);
cvMatMul(R2, R1, H);
return H;
}
public static class Exception extends java.lang.Exception {
public Exception(String message) { super(message); }
public Exception(String message, Throwable cause) { super(message, cause); }
}
public static ProjectiveDevice[] read(String filename) throws Exception {
FileStorage fs = new FileStorage(filename, FileStorage.READ);
CameraDevice [] cameraDevices = CameraDevice .read(fs);
ProjectorDevice [] projectorDevices = ProjectorDevice.read(fs);
ProjectiveDevice[] devices = new ProjectiveDevice[cameraDevices.length+projectorDevices.length];
int i = 0;
for (ProjectiveDevice d : cameraDevices) {
devices[i++] = d;
}
for (ProjectiveDevice d : projectorDevices) {
devices[i++] = d;
}
fs.release();
return devices;
}
public static void write(String filename, ProjectiveDevice[] ... devices) {
int totalLength = 0;
for (ProjectiveDevice[] ds : devices) {
totalLength += ds.length;
}
ProjectiveDevice[] allDevices = new ProjectiveDevice[totalLength];
int i = 0;
for (ProjectiveDevice[] ds : devices) {
for (ProjectiveDevice d : ds) {
allDevices[i++] = d;
}
}
write(filename, allDevices);
}
public static void write(String filename, ProjectiveDevice ... devices) {
FileStorage fs = new FileStorage(filename, FileStorage.WRITE);
shiftLeft(shiftLeft(fs, "Cameras"), "[");
for (ProjectiveDevice d : devices) {
if (d instanceof CameraDevice) {
opencv_core.write(fs, d.getSettings().getName());
}
}
shiftLeft(fs, "]");
shiftLeft(shiftLeft(fs, "Projectors"), "[");
for (ProjectiveDevice d : devices) {
if (d instanceof ProjectorDevice) {
opencv_core.write(fs, d.getSettings().getName());
}
}
shiftLeft(fs, "]");
for (ProjectiveDevice d : devices) {
d.writeParameters(fs);
}
fs.release();
}
public void writeParameters(File file) {
writeParameters(file.getAbsolutePath());
}
public void writeParameters(String filename) {
FileStorage fs = new FileStorage(filename, FileStorage.WRITE);
writeParameters(fs);
fs.release();
}
public void writeParameters(FileStorage fs) {
shiftLeft(shiftLeft(fs, getSettings().getName()), "{");
opencv_core.write(fs, "imageWidth", imageWidth);
opencv_core.write(fs, "imageHeight", imageHeight);
opencv_core.write(fs, "responseGamma", getSettings().getResponseGamma());
// opencv_core.write(fs, "initAspectRatio", settings.initAspectRatio);
// opencv_core.write(fs, "flags", getSettings().flags);
if (cameraMatrix != null)
opencv_core.write(fs, "cameraMatrix", cvarrToMat(cameraMatrix));
if (distortionCoeffs != null)
opencv_core.write(fs, "distortionCoeffs", cvarrToMat(distortionCoeffs));
if (extrParams != null)
opencv_core.write(fs, "extrParams", cvarrToMat(extrParams));
if (reprojErrs != null)
opencv_core.write(fs, "reprojErrs", cvarrToMat(reprojErrs));
opencv_core.write(fs, "avgReprojErr", avgReprojErr);
opencv_core.write(fs, "maxReprojErr", maxReprojErr);
// opencv_core.write(fs, "nominalDistance", nominalDistance);
if (R != null)
opencv_core.write(fs, "R", cvarrToMat(R));
if (T != null)
opencv_core.write(fs, "T", cvarrToMat(T));
if (E != null)
opencv_core.write(fs, "E", cvarrToMat(E));
if (F != null)
opencv_core.write(fs, "F", cvarrToMat(F));
opencv_core.write(fs, "avgEpipolarErr", avgEpipolarErr);
opencv_core.write(fs, "maxEpipolarErr", maxEpipolarErr);
opencv_core.write(fs, "colorOrder", colorOrder);
if (colorMixingMatrix != null)
opencv_core.write(fs, "colorMixingMatrix", cvarrToMat(colorMixingMatrix));
if (additiveLight != null)
opencv_core.write(fs, "additiveLight", cvarrToMat(additiveLight));
opencv_core.write(fs, "avgColorErr", avgColorErr);
opencv_core.write(fs, "colorR2", colorR2);
shiftLeft(fs, "}");
}
public void readParameters(File file) throws Exception {
readParameters(file.getAbsolutePath());
}
public void readParameters(String filename) throws Exception {
FileStorage fs = new FileStorage(filename, FileStorage.READ);
readParameters(fs);
fs.release();
}
public void readParameters(FileStorage fs) throws Exception {
if (fs == null) {
throw new Exception("Error: FileStorage is null, cannot read parameters for device " +
getSettings().getName() + ". Is the parametersFile correct?");
}
FileNode fn = fs.get(getSettings().getName());
if (fn == null) {
throw new Exception("Error: FileNode is null, cannot read parameters for device " +
getSettings().getName() + ". Is the name correct?");
}
FileNode n;
if ((n = fn.get("imageWidth")).isInt()) imageWidth = n.asInt();
if ((n = fn.get("imageHeight")).isInt()) imageHeight = n.asInt();
if ((n = fn.get("gamma")).isReal()) getSettings().setResponseGamma(n.asDouble());
// if ((n = fn.get("initAspectRatio")).isReal()) getSettings().setInitAspectRatio(n.asDouble());
// if ((n = fn.get("flags")).isInt()) getSettings().setFlags(n.asInt());
Mat m = new Mat();
opencv_core.read(fn.get("cameraMatrix"), m);
cameraMatrix = m.empty() ? null : cvMat(m).clone();
opencv_core.read(fn.get("distortionCoeffs"), m);
distortionCoeffs = m.empty() ? null : cvMat(m).clone();
opencv_core.read(fn.get("extrParams"), m);
extrParams = m.empty() ? null : cvMat(m).clone();
opencv_core.read(fn.get("reprojErrs"), m);
reprojErrs = m.empty() ? null : cvMat(m).clone();
if ((n = fn.get("avgReprojErr")).isReal()) avgReprojErr = n.asDouble();
if ((n = fn.get("maxReprojErr")).isReal()) maxReprojErr = n.asDouble();
// if ((n = fn.get("nominalDistance")).isReal()) nominalDistance = n.asDouble();
opencv_core.read(fn.get("R"), m);
R = m.empty() ? null : cvMat(m).clone();
opencv_core.read(fn.get("T"), m);
T = m.empty() ? null : cvMat(m).clone();
opencv_core.read(fn.get("E"), m);
E = m.empty() ? null : cvMat(m).clone();
opencv_core.read(fn.get("F"), m);
F = m.empty() ? null : cvMat(m).clone();
if ((n = fn.get("avgEpipolarErr")).isReal()) avgEpipolarErr = n.asDouble();
if ((n = fn.get("maxEpipolarErr")).isReal()) maxEpipolarErr = n.asDouble();
if ((n = fn.get("colorOrder")).isString()) colorOrder = n.asBytePointer().getString();
opencv_core.read(fn.get("colorMixingMatrix"), m);
colorMixingMatrix = m.empty() ? null : cvMat(m).clone();
opencv_core.read(fn.get("additiveLight"), m);
additiveLight = m.empty() ? null : cvMat(m).clone();
if ((n = fn.get("avgColorErr")).isReal()) avgColorErr = n.asDouble();
if ((n = fn.get("colorR2")).isReal()) colorR2 = n.asDouble();
}
@Override public String toString() {
String s =
getSettings().getName() + " (" + imageWidth + " x " + imageHeight + ")\n";
for (int i = 0; i < getSettings().getName().length(); i++) {
s += "=";
}
s += "\n" +
"Intrinsics\n" +
"----------\n" +
"camera matrix = " + (cameraMatrix == null ? "null" : cameraMatrix.toString(16)) + "\n" +
"distortion coefficients = " + (distortionCoeffs == null ? "null" : distortionCoeffs) + "\n" +
"reprojection RMS/max error (pixels) = " + (float)avgReprojErr + " / " + (float)maxReprojErr + "\n\n" +
"Extrinsics\n" +
"----------\n" +
"rotation = " + (R == null ? "null" : R.toString(11)) + "\n" +
"translation = " + (T == null ? "null" : T.toString(14)) + "\n" +
"epipolar RMS/max error (pixels) = " + (float)avgEpipolarErr + " / " + (float)maxEpipolarErr + "\n\n" +
"Color\n" +
"-----\n" +
"order = " + colorOrder + "\n" +
"mixing matrix = " + (colorMixingMatrix == null ? "null" : colorMixingMatrix.toString(16)) + "\n" +
"additive light = " + (additiveLight == null ? "null" : additiveLight.toString(17)) + "\n" +
"normalized RMSE (intensity) = " + (float)avgColorErr + "\n" +
"R^2 (intensity) = " + (float)colorR2;
return s;
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/ProjectiveTransformer.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import org.bytedeco.opencv.opencv_calib3d.*;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.javacv.cvkernels.*;
import static org.bytedeco.opencv.global.opencv_calib3d.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
*
* @author Samuel Audet
*/
public class ProjectiveTransformer implements ImageTransformer {
public ProjectiveTransformer() {
this(null, null, null, null, null, new double[0], null);
}
public ProjectiveTransformer(double[] referencePoints) {
this(null, null, null, null, null, referencePoints, null);
}
public ProjectiveTransformer(ProjectiveDevice d1, ProjectiveDevice d2, CvMat n,
double[] referencePoints1, double[] referencePoints2) {
// assuming d1 has identity values, use d2's stuff directly
this(d1.cameraMatrix, d2.cameraMatrix, d2.R, d2.T, n, referencePoints1, referencePoints2);
}
public ProjectiveTransformer(CvMat K1, CvMat K2, CvMat R, CvMat t, CvMat n,
double[] referencePoints1, double[] referencePoints2) {
this.K1 = K1 == null ? null : K1.clone();
this.K2 = K2 == null ? null : K2.clone();
this.invK1 = K1 == null ? null : K1.clone();
this.invK2 = K2 == null ? null : K2.clone();
if (K1 != null) {
cvInvert(K1, invK1);
}
if (K2 != null) {
cvInvert(K2, invK2);
}
this.R = R == null ? null : R.clone();
this.t = t == null ? null : t.clone();
this.n = n == null ? null : n.clone();
this.referencePoints1 = referencePoints1 == null ? null : referencePoints1.clone();
this.referencePoints2 = referencePoints2 == null ? null : referencePoints2.clone();
}
protected static ThreadLocal
H3x3 = CvMat.createThreadLocal(3, 3),
pts4x1 = CvMat.createThreadLocal(4, 1, CV_64F, 2);
protected CvMat K1 = null, K2 = null, invK1 = null, invK2 = null, R = null, t = null, n = null;
protected double[] referencePoints1 = null, referencePoints2 = null;
protected CvScalar fillColor = cvScalar(0.0, 0.0, 0.0, 1.0);
protected KernelData kernelData = null;
protected CvMat[] H = null;
public CvScalar getFillColor() {
return fillColor;
}
public void setFillColor(CvScalar fillColor) {
this.fillColor = fillColor;
}
public double[] getReferencePoints1() {
return referencePoints1;
}
public double[] getReferencePoints2() {
return referencePoints2;
}
public CvMat getK1() {
return K1;
}
public CvMat getK2() {
return K2;
}
public CvMat getInvK1() {
return invK1;
}
public CvMat getInvK2() {
return invK2;
}
public CvMat getR() {
return R;
}
public CvMat getT() {
return t;
}
public CvMat getN() {
return n;
}
protected void prepareHomography(CvMat H, int pyramidLevel, Parameters p, boolean inverse) {
if (K2 != null && invK1 != null && R != null && t != null && p.fakeIdentity) {
// no identity available for plane parameter...
// fakeIdentity needs to be implemented..
cvSetIdentity(H);
return;
}
if (inverse) {
H.put(p.getH());
} else {
cvInvert(p.getH(), H);
}
// adjust the scale of the transformation based on the pyramid level
if (pyramidLevel > 0) {
int scale = 1<= 8) {
// projectiveParameters[i] += scale;
// } else {
// // translation vector
//
// // assuming a reference plane at [0, 0, 1],
// // this is about 1% of image resolution?
// projectiveParameters[i] += 0.01 * scale;
// }
// }
// setUpdateNeeded(true);
// return false;
// }
public double getConstraintError() {
update();
return constraintError;
}
public void set(CvMat setH, boolean inverse) {
if (projectiveParameters.length == 8 && referencePoints1 != null) {
if (inverse) {
cvInvert(setH, H);
} else if (setH != H) {
cvCopy(setH, H);
}
if (referencePoints1.length == 0) {
// direct homography parameterization
for (int i = 0; i < 8; i++) {
projectiveParameters[i] = H.get(i)/H.get(8);
}
} else {
// 4 point parametrization
CvMat pts = pts4x1.get().put(referencePoints1);
cvPerspectiveTransform(pts, pts, H);
pts.get(projectiveParameters);
}
setUpdateNeeded(true);
} else {
throw new UnsupportedOperationException("Set homography operation not supported.");
}
}
public void compose(ImageTransformer.Parameters p1, boolean inverse1,
ImageTransformer.Parameters p2, boolean inverse2) {
Parameters pp1 = (Parameters)p1, pp2 = (Parameters)p2;
if (K2 != null && invK1 != null && R != null && t != null && pp1.fakeIdentity) {
// no identity available for plane parameter...
// fakeIdentity needs to be implemented..
return;
}
compose(pp1.getH(), inverse1, pp2.getH(), inverse2);
}
public void compose(CvMat H1, boolean inverse1, CvMat H2, boolean inverse2) {
if (inverse1 && inverse2) {
cvMatMul(H2, H1, H);
cvInvert(H, H);
} else if (inverse1) {
cvInvert(H1, H);
cvMatMul(H, H2, H);
} else if (inverse2) {
cvInvert(H2, H);
cvMatMul(H1, H, H);
} else {
cvMatMul(H1, H2, H);
}
set(H, false);
}
public CvMat getH() {
update();
return H;
}
public CvMat getN() {
update();
return n2;
}
public CvMat getR() {
update();
return R2;
}
public CvMat getT() {
update();
return t2;
}
protected void update() {
if (!isUpdateNeeded()) {
return;
}
if (referencePoints1 != null && (referencePoints1.length == 0 || referencePoints1.length == 8)) {
if (referencePoints1.length == 0) {
// direct homography parameterization
H.put(0, projectiveParameters, 0, 8);
H.put(8, 1);
} else {
// 4 point parameterization
JavaCV.getPerspectiveTransform(referencePoints1, projectiveParameters, H);
}
// if (K1 != null && invK2 != null && n != null) {
// CvMat Hprime = CvMat.take(3, 3);
// cvCopy(H, Hprime);
// if (R2 == null) {
// R2 = CvMat.create(3, 3);
// }
// if (t2 == null) {
// t2 = CvMat.create(3, 1);
// }
//
// cvMatMul(invK2, Hprime, Hprime);
// cvMatMul(Hprime, K1, Hprime);
//
// // get 3D rotation and translation, with given n
// constraintError = JavaCV.HnToRt(Hprime, n, R2, t2);
// //System.out.println(constraintError);
// Hprime.pool();
// }
} else if (K2 != null && invK1 != null) {
if (R != null && t != null) {
// 3D plane motion, with given R and t
// if (n2 == null) {
// n2 = CvMat.create(3, 1);
// }
double[] src = referencePoints2;
double[] dst = { projectiveParameters[0], referencePoints1[1],
projectiveParameters[1], referencePoints1[3],
projectiveParameters[2], referencePoints1[5] };
if (R2 == null) {
R2 = CvMat.create(3, 3);
}
if (t2 == null) {
t2 = CvMat.create(3, 1);
}
cvTranspose(R, R2);
cvGEMM(R2, t, -1, null, 0, t2, 0);
JavaCV.getPerspectiveTransform(src, dst, invK2, K1, R2, t2, H);
//cvConvertScale(H, H, 1/H.get(8), 0);
//System.out.println(H);
// n2.put(projectiveParameters);
// // H = R-t*n^T
// cvGEMM(t, n2, -1, R, 1, H, CV_GEMM_B_T);
} else {
// 3D rotation and translation, with given n
if (n != null) {
n2 = n; // take n from transformer
} else {
if (n2 == null) {
n2 = CvMat.create(3, 1);
}
n2.put(0, projectiveParameters, 8, 3); // take n from parameters
}
// put rotation angle and translation in matrices
if (R2 == null) {
R2 = CvMat.create(3, 3);
}
if (t2 == null) {
t2 = CvMat.create(3, 1);
}
t2.put(0, projectiveParameters, 0, 3);
Rodrigues(cvarrToMat(t2), cvarrToMat(R2), null);
t2.put(0, projectiveParameters, 3, 3);
// H = R-tn^T
cvGEMM(t2, n2, -1, R2, 1, H, CV_GEMM_B_T);
}
// // H = K2 * H * K1^-1
// cvMatMul(K2, H, H);
// cvMatMul(H, invK1, H);
}
setUpdateNeeded(false);
}
// public void project() {
// CvMat t2 = getT(), n2 = getN(), R2 = getR();
// cvSetIdentity(H);
// cvGEMM(t2, n2, -1, H, 1, H, CV_GEMM_B_T);
// cvMatMul(R2, H, H);
// cvMatMul(K2, H, H);
// cvMatMul(H, invK1, H);
//
// CvMat pts = CvMat.take(4, 1, CV_64F, 2);
// pts.put(referencePoints);
// cvPerspectiveTransform(pts, pts, H);
// pts.get(projectiveParameters);
// pts.pool();
// }
public boolean preoptimize() {
return false;
}
public double[] getSubspace() {
return null;
}
public void setSubspace(double ... p) {
}
@Override public Parameters clone() {
Parameters p = new Parameters();
p.set(this);
return p;
}
@Override public String toString() {
String s = "[";
double[] p = get();
for (int i = 0; i < p.length; i++) {
s += (float)p[i];
if (i < p.length-1) {
s+= ", ";
}
}
s += "]";
return s;
}
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/ProjectiveTransformerCL.java
================================================
/*
* Copyright (C) 2011-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import com.jogamp.opencl.CLBuffer;
import com.jogamp.opencl.CLEventList;
import com.jogamp.opencl.CLImage2d;
import com.jogamp.opencl.CLKernel;
import java.nio.FloatBuffer;
import org.bytedeco.opencv.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_core.*;
/**
*
* @author Samuel Audet
*/
public class ProjectiveTransformerCL extends ProjectiveTransformer implements ImageTransformerCL {
public ProjectiveTransformerCL(JavaCVCL context) {
this(context, null, null, null, null, null, new double[0], null);
}
public ProjectiveTransformerCL(JavaCVCL context, double[] referencePoints) {
this(context, null, null, null, null, null, referencePoints, null);
}
public ProjectiveTransformerCL(JavaCVCL context, ProjectiveDevice d1, ProjectiveDevice d2, CvMat n,
double[] referencePoints1, double[] referencePoints2) {
// assuming d1 has identity values, use d2's stuff directly
this(context, d1.cameraMatrix, d2.cameraMatrix, d2.R, d2.T, n, referencePoints1, referencePoints2);
}
public ProjectiveTransformerCL(JavaCVCL context, CvMat K1, CvMat K2, CvMat R, CvMat t, CvMat n,
double[] referencePoints1, double[] referencePoints2) {
super(K1, K2, R, t, n, referencePoints1, referencePoints2);
final int dotSize = createParameters().size();
this.context = context;
this.HBuffer = context.getCLContext().createFloatBuffer(dotSize*9, CLBuffer.Mem.READ_ONLY);
if (getClass() == ProjectiveTransformerCL.class) {
CLKernel[] kernels = context.buildKernels(
JavaCVCL.fastCompilerOptions + " -DDOT_SIZE=" + dotSize,
"ImageTransformer.cl:ProjectiveTransformer.cl",
"transformOne", "transformSub", "transformDot", "reduceOutputData");
this.oneKernel = kernels[0];
this.subKernel = kernels[1];
this.dotKernel = kernels[2];
this.reduceKernel = kernels[3];
}
}
protected final JavaCVCL context;
protected final CLBuffer HBuffer;
private CLKernel oneKernel, subKernel, dotKernel, reduceKernel;
public JavaCVCL getContext() {
return context;
}
protected void prepareHomographies(CLBuffer HBuffer, int pyramidLevel,
ImageTransformer.Parameters[] parameters, boolean[] inverses) {
FloatBuffer floatH = (FloatBuffer)HBuffer.getBuffer().rewind();
CvMat H = H3x3.get();
for (int i = 0; i < parameters.length; i++) {
prepareHomography(H, pyramidLevel, (ProjectiveTransformer.Parameters)parameters[i],
inverses == null ? false : inverses[i]);
for (int j = 0; j < 9; j++) {
floatH.put((float)H.get(j));
}
}
floatH.rewind();
}
public void transform(CLImage2d srcImg, CLImage2d subImg, CLImage2d srcDotImg,
CLImage2d transImg, CLImage2d dstImg, CLImage2d maskImg,
ImageTransformer.Parameters[] parameters, boolean[] inverses,
InputData inputData, OutputData outputData) {
prepareHomographies(HBuffer, inputData.pyramidLevel, parameters, inverses);
final int dotSize = parameters[0].size();
final int localSize = parameters.length > 1 ? parameters.length : (inputData.roiWidth > 32 ? 64 : 32);
final int globalSize = JavaCVCL.alignCeil(inputData.roiWidth, localSize);
final int reduceSize = globalSize/localSize;
// allocate buffers if necessary
CLBuffer inputBuffer = inputData.getBuffer(context);
CLBuffer outputBuffer = outputData.getBuffer(context, dotSize, reduceSize);
CLEventList list = new CLEventList(1);
// setup kernel
context.writeBuffer(HBuffer, false); // upload H
if (inputData.autoWrite) {
inputData.writeBuffer(context);
}
CLKernel kernel = null;
if (subImg == null) {
assert parameters.length == 1;
kernel = oneKernel.putArg(srcImg).putArg(dstImg == null ? transImg : dstImg).putArg(maskImg)
.putArg(HBuffer).putArg(inputBuffer).putArg(outputBuffer).rewind();
} else if (srcDotImg == null) {
assert parameters.length == 1;
kernel = subKernel.putArg(srcImg).putArg(subImg).putArg(transImg).putArg(dstImg).putArg(maskImg)
.putArg(HBuffer).putArg(inputBuffer).putArg(outputBuffer).rewind();
} else {
assert parameters.length == dotSize;
kernel = dotKernel.putArg(srcImg).putArg(subImg).putArg(srcDotImg).putArg(maskImg)
.putArg(HBuffer).putArg(inputBuffer).putArg(outputBuffer).rewind();
}
context.executeKernel(kernel, inputData.roiX, 0, 0,
globalSize, 1, parameters.length,
localSize, 1, parameters.length, list); // execute program
if (reduceSize > 1) {
reduceKernel.putArg(outputBuffer).rewind();
context.executeKernel(reduceKernel, 0, reduceSize, reduceSize);
}
if (outputData.autoRead) {
outputData.readBuffer(context);
}
// CLEvent event = list.getEvent(0);
// System.out.println(kernel + " " + (event.getProfilingInfo(CLEvent.ProfilingCommand.END) -
// event.getProfilingInfo(CLEvent.ProfilingCommand.START))/1000000.0);
// long res = q.getDevice().getProfilingTimerResolution();
// System.out.println(res);
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/ProjectorDevice.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.awt.Dimension;
import java.awt.DisplayMode;
import java.beans.PropertyChangeListener;
import java.lang.reflect.InvocationTargetException;
import org.bytedeco.javacpp.Pointer;
import org.bytedeco.opencv.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_core.*;
/**
*
* @author Samuel Audet
*/
public class ProjectorDevice extends ProjectiveDevice {
public ProjectorDevice(String name) {
super(name);
}
public ProjectorDevice(String name, String filename) throws Exception {
super(name, filename);
settings.setImageWidth(imageWidth);
settings.setImageHeight(imageHeight);
}
public ProjectorDevice(String name, FileStorage fs) throws Exception {
super(name, fs);
settings.setImageWidth(imageWidth);
settings.setImageHeight(imageHeight);
}
public ProjectorDevice(Settings settings) throws Exception {
super((ProjectiveDevice.Settings)settings);
}
public interface Settings {
String getName();
void setName(String name);
double getResponseGamma();
void setResponseGamma(double gamma);
int getScreenNumber();
void setScreenNumber(int screenNumber);
long getLatency();
void setLatency(long latency);
String getDescription();
int getImageWidth();
void setImageWidth(int imageWidth);
int getImageHeight();
void setImageHeight(int imageHeight);
int getBitDepth();
void setBitDepth(int bitDepth);
int getRefreshRate();
void setRefreshRate(int refreshRate);
boolean isUseOpenGL();
void setUseOpenGL(boolean useOpenGL);
void addPropertyChangeListener(PropertyChangeListener listener);
void removePropertyChangeListener(PropertyChangeListener listener);
}
public static class SettingsImplementation extends ProjectiveDevice.Settings implements Settings {
public SettingsImplementation() { name = "Projector 0"; setScreenNumber(screenNumber); }
public SettingsImplementation(ProjectiveDevice.Settings settings) {
super(settings);
if (settings instanceof SettingsImplementation) {
SettingsImplementation s = (SettingsImplementation)settings;
this.screenNumber = s.screenNumber;
this.latency = s.latency;
this.imageWidth = s.imageWidth;
this.imageHeight = s.imageHeight;
this.bitDepth = s.bitDepth;
this.refreshRate = s.refreshRate;
this.useOpenGL = s.useOpenGL;
}
}
int screenNumber = CanvasFrame.getScreenDevices().length > 1 ? 1 : 0;
long latency = CanvasFrame.DEFAULT_LATENCY;
public int getScreenNumber() {
return screenNumber;
}
public void setScreenNumber(int screenNumber) {
DisplayMode d = CanvasFrame.getDisplayMode(screenNumber);
String oldDescription = getDescription();
firePropertyChange("screenNumber", this.screenNumber, this.screenNumber = screenNumber);
firePropertyChange("description", oldDescription, getDescription());
firePropertyChange("imageWidth", this.imageWidth, this.imageWidth = d == null ? 0 : d.getWidth());
firePropertyChange("imageHeight", this.imageHeight, this.imageHeight = d == null ? 0 : d.getHeight());
firePropertyChange("bitDepth", this.bitDepth, this.bitDepth = d == null ? 0 : d.getBitDepth());
firePropertyChange("refreshRate", this.refreshRate, this.refreshRate = d == null ? 0 : d.getRefreshRate());
firePropertyChange("responseGamma", this.responseGamma, this.responseGamma = CanvasFrame.getGamma(screenNumber));
}
public long getLatency() {
return latency;
}
public void setLatency(long latency) {
this.latency = latency;
}
public String getDescription() {
String[] descriptions = null;
descriptions = CanvasFrame.getScreenDescriptions();
if (descriptions != null && screenNumber >= 0 && screenNumber < descriptions.length) {
return descriptions[screenNumber];
} else {
return "";
}
}
int imageWidth = 0, imageHeight = 0, bitDepth = 0, refreshRate = 0;
public int getImageWidth() {
return imageWidth;
}
public void setImageWidth(int imageWidth) {
firePropertyChange("imageWidth", this.imageWidth, this.imageWidth = imageWidth);
}
public int getImageHeight() {
return imageHeight;
}
public void setImageHeight(int imageHeight) {
firePropertyChange("imageHeight", this.imageHeight, this.imageHeight = imageHeight);
}
public int getBitDepth() {
return bitDepth;
}
public void setBitDepth(int bitDepth) {
this.bitDepth = bitDepth;
}
public int getRefreshRate() {
return refreshRate;
}
public void setRefreshRate(int refreshRate) {
this.refreshRate = refreshRate;
}
private boolean useOpenGL = false;
public boolean isUseOpenGL() {
return useOpenGL;
}
public void setUseOpenGL(boolean useOpenGL) {
this.useOpenGL = useOpenGL;
}
}
// pouah.. hurray for Scala!
public static class CalibrationSettings extends ProjectiveDevice.CalibrationSettings implements Settings {
public CalibrationSettings() { }
public CalibrationSettings(ProjectiveDevice.CalibrationSettings settings) {
super(settings);
if (settings instanceof CalibrationSettings) {
CalibrationSettings s = (CalibrationSettings)settings;
si = new SettingsImplementation(s.si);
this.brightnessBackground = s.brightnessBackground;
this.brightnessForeground = s.brightnessForeground;
}
}
SettingsImplementation si = new SettingsImplementation() {
@Override public void firePropertyChange(String propertyName, Object oldValue, Object newValue) {
CalibrationSettings.this.firePropertyChange(propertyName, oldValue, newValue);
}
};
@Override public String getName() { return si.getName(); }
@Override public void setName(String name) { si.setName(name); }
@Override public double getResponseGamma() { return si.getResponseGamma(); }
@Override public void setResponseGamma(double responseGamma) { si.setResponseGamma(responseGamma); }
// @Override public double getNominalDistance() { return si.getNominalDistance(); }
// @Override public void setNominalDistance(double nominalDistance) { si.setNominalDistance(nominalDistance); }
public int getScreenNumber() { return si.getScreenNumber(); }
public void setScreenNumber(int screenNumber) { si.setScreenNumber(screenNumber); }
public long getLatency() { return si.getLatency(); }
public void setLatency(long latency) { si.setLatency(latency); }
public String getDescription() { return si.getDescription(); }
public int getImageWidth() { return si.getImageWidth(); }
public void setImageWidth(int imageWidth) { si.setImageWidth(imageWidth); }
public int getImageHeight() { return si.getImageHeight(); }
public void setImageHeight(int imageHeight) { si.setImageHeight(imageHeight); }
public int getBitDepth() { return si.getBitDepth(); }
public void setBitDepth(int bitDepth) { si.setBitDepth(bitDepth); }
public int getRefreshRate() { return si.getRefreshRate(); }
public void setRefreshRate(int refreshRate) { si.setRefreshRate(refreshRate); }
public boolean isUseOpenGL() { return si.isUseOpenGL(); }
public void setUseOpenGL(boolean useOpenGL) { si.setUseOpenGL(useOpenGL); }
double brightnessBackground = 0.0, brightnessForeground = 1.0;
public double getBrightnessBackground() {
return brightnessBackground;
}
public void setBrightnessBackground(double brightnessBackground) {
firePropertyChange("brightnessBackground", this.brightnessBackground,
this.brightnessBackground = brightnessBackground);
}
public double getBrightnessForeground() {
return brightnessForeground;
}
public void setBrightnessForeground(double brightnessForeground) {
firePropertyChange("brightnessForeground", this.brightnessForeground,
this.brightnessForeground = brightnessForeground);
}
}
public static class CalibratedSettings extends ProjectiveDevice.CalibratedSettings implements Settings {
public CalibratedSettings() { }
public CalibratedSettings(ProjectiveDevice.CalibratedSettings settings) {
super(settings);
if (settings instanceof CalibratedSettings) {
si = new SettingsImplementation(((CalibratedSettings)settings).si);
}
}
SettingsImplementation si = new SettingsImplementation() {
@Override public void firePropertyChange(String propertyName, Object oldValue, Object newValue) {
CalibratedSettings.this.firePropertyChange(propertyName, oldValue, newValue);
}
};
@Override public String getName() { return si.getName(); }
@Override public void setName(String name) { si.setName(name); }
@Override public double getResponseGamma() { return si.getResponseGamma(); }
@Override public void setResponseGamma(double responseGamma) { si.setResponseGamma(responseGamma); }
// @Override public double getNominalDistance() { return si.getNominalDistance(); }
// @Override public void setNominalDistance(double nominalDistance) { si.setNominalDistance(nominalDistance); }
public int getScreenNumber() { return si.getScreenNumber(); }
public void setScreenNumber(int screenNumber) { si.setScreenNumber(screenNumber); }
public long getLatency() { return si.getLatency(); }
public void setLatency(long latency) { si.setLatency(latency); }
public String getDescription() { return si.getDescription(); }
public int getImageWidth() { return si.getImageWidth(); }
public void setImageWidth(int imageWidth) { si.setImageWidth(imageWidth); }
public int getImageHeight() { return si.getImageHeight(); }
public void setImageHeight(int imageHeight) { si.setImageHeight(imageHeight); }
public int getBitDepth() { return si.getBitDepth(); }
public void setBitDepth(int bitDepth) { si.setBitDepth(bitDepth); }
public int getRefreshRate() { return si.getRefreshRate(); }
public void setRefreshRate(int refreshRate) { si.setRefreshRate(refreshRate); }
public boolean isUseOpenGL() { return si.isUseOpenGL(); }
public void setUseOpenGL(boolean useOpenGL) { si.setUseOpenGL(useOpenGL); }
}
private Settings settings;
@Override public ProjectiveDevice.Settings getSettings() {
return (ProjectiveDevice.Settings)settings;
}
public void setSettings(Settings settings) {
setSettings((ProjectiveDevice.Settings)settings);
}
@Override public void setSettings(ProjectiveDevice.Settings settings) {
super.setSettings(settings);
if (settings instanceof ProjectiveDevice.CalibrationSettings) {
this.settings = new CalibrationSettings((ProjectiveDevice.CalibrationSettings)settings);
} else if (settings instanceof ProjectiveDevice.CalibratedSettings) {
this.settings = new CalibratedSettings((ProjectiveDevice.CalibratedSettings)settings);
} else {
this.settings = new SettingsImplementation((ProjectiveDevice.Settings)settings);
}
if (this.settings.getName() == null || this.settings.getName().length() == 0) {
this.settings.setName("Projector " + String.format("%2d", this.settings.getScreenNumber()));
}
}
public CanvasFrame createCanvasFrame() throws CanvasFrame.Exception {
if (settings.getScreenNumber() < 0) {
return null;
}
DisplayMode d = new DisplayMode(settings.getImageWidth(), settings.getImageHeight(),
settings.getBitDepth(), settings.getRefreshRate());
CanvasFrame c = null;
Throwable cause = null;
try {
c = Class.forName(CanvasFrame.class.getPackage().getName() + (settings.isUseOpenGL() ? ".GLCanvasFrame" : ".CanvasFrame")).
asSubclass(CanvasFrame.class).getConstructor(String.class, int.class, DisplayMode.class, double.class).
newInstance(settings.getName(), settings.getScreenNumber(), d, settings.getResponseGamma());
} catch (ClassNotFoundException ex) {
cause = ex;
} catch (InstantiationException ex) {
cause = ex;
} catch (IllegalAccessException ex) {
cause = ex;
} catch (IllegalArgumentException ex) {
cause = ex;
} catch (NoSuchMethodException ex) {
cause = ex;
} catch (InvocationTargetException ex) {
cause = ex.getCause();
}
if (cause != null) {
if (cause instanceof CanvasFrame.Exception) {
throw (CanvasFrame.Exception)cause;
} else {
throw new CanvasFrame.Exception("Failed to create CanvasFrame", cause);
}
}
c.setLatency(settings.getLatency());
Dimension size = c.getCanvasSize();
if (size.width != imageWidth || size.height != imageHeight) {
rescale(size.width, size.height);
}
return c;
}
private static ThreadLocal
B4x3 = CvMat.createThreadLocal(4, 3),
x23x1 = CvMat.createThreadLocal(3, 1),
x34x1 = CvMat.createThreadLocal(4, 1);
public double getAttenuation(double x, double y, CvMat n, double d) {
CvMat B = B4x3.get();
CvMat x2 = x23x1.get();
CvMat x3 = x34x1.get();
getBackProjectionMatrix(n, d, B);
x2.put(x, y, 1);
cvMatMul(B, x2, x3);
// find the direction and the distance of that middle point to the
// projector and use it to compute the expected overall attenuation
// cos(theta) * nominal_distance^2 / distance^2
// we assume a perfectly Lambertian surface ... ugh ...
// at a sufficient distance from the projector... ugh...
cvGEMM(R, T, -1, null, 0, x2, CV_GEMM_A_T);
x3.rows(3);
cvAddWeighted(x3, 1/x3.get(3), x2, -1, 0, x2);
double distance2 = cvDotProduct(x2, x2);
double distance = Math.sqrt(distance2);
double cosangle = -Math.signum(d)*cvDotProduct(x2, n)/
(distance * Math.sqrt(cvDotProduct(n, n)));
double attenuation = cosangle/distance2;
// System.out.println(distance + " " + cosangle + " " + attenuation);
x3.rows(4);
return attenuation;
}
public static ProjectorDevice[] read(String filename) throws Exception {
FileStorage fs = new FileStorage(filename, FileStorage.READ);
ProjectorDevice[] devices = read(fs);
fs.release();
return devices;
}
public static ProjectorDevice[] read(FileStorage fs) throws Exception {
FileNode node = fs.get("Projectors");
FileNodeIterator seq = node.begin();
int count = (int)seq.remaining();
ProjectorDevice[] devices = new ProjectorDevice[count];
for (int i = 0; i < count; i++, seq.increment()) {
FileNode n = seq.multiply();
if (n.empty()) continue;
String name = n.asBytePointer().getString();
devices[i] = new ProjectorDevice(name, fs);
}
return devices;
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/ProjectorSettings.java
================================================
/*
* Copyright (C) 2009-2010 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.beans.PropertyChangeListener;
/**
*
* @author Samuel Audet
*/
public class ProjectorSettings extends BaseSettings {
public ProjectorSettings() {
this(false);
}
public ProjectorSettings(boolean calibrated) {
this.calibrated = calibrated;
}
boolean calibrated = false;
public int getQuantity() {
return size();
}
public void setQuantity(int quantity) {
Object[] a = toArray();
int i = a.length;
while (i > quantity) {
remove(a[i-1]);
i--;
}
while (i < quantity) {
ProjectorDevice.Settings c = calibrated ? new ProjectorDevice.CalibratedSettings() :
new ProjectorDevice.CalibrationSettings();
c.setName("Projector " + String.format("%2d", i));
c.setScreenNumber(c.getScreenNumber()+i);
add(c);
for (PropertyChangeListener l : pcSupport.getPropertyChangeListeners()) {
((BaseChildSettings)c).addPropertyChangeListener(l);
}
i++;
}
pcSupport.firePropertyChange("quantity", a.length, quantity);
}
@Override public ProjectorDevice.Settings[] toArray() {
return (ProjectorDevice.Settings[])toArray(new ProjectorDevice.Settings[size()]);
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/RealSense2FrameGrabber.java
================================================
/*
* Copyright (C) 2019-2022 Florian Bruggisser, Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import org.bytedeco.javacpp.IntPointer;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.javacpp.Pointer;
import org.bytedeco.librealsense2.*;
import org.bytedeco.opencv.opencv_core.IplImage;
import org.bytedeco.opencv.opencv_core.Size;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import static org.bytedeco.librealsense2.global.realsense2.*;
import static org.bytedeco.opencv.global.opencv_core.*;
public class RealSense2FrameGrabber extends FrameGrabber {
private static FrameGrabber.Exception loadingException = null;
private static final int defaultFrameRate = 30;
private static final int defaultWidth = 640;
private static final int defaultHeight = 480;
public static void tryLoad() throws FrameGrabber.Exception {
if (loadingException != null) {
loadingException.printStackTrace();
throw loadingException;
} else {
try {
Loader.load(org.bytedeco.librealsense2.presets.realsense2.class);
System.out.println("RealSense2 devices found: " + getDeviceDescriptions().length);
} catch (Throwable t) {
throw loadingException = new FrameGrabber.Exception("Failed to load " + RealSense2FrameGrabber.class, t);
}
}
}
private rs2_error error = new rs2_error();
private rs2_context context;
private rs2_device device;
private rs2_pipeline pipeline;
private rs2_config config;
private rs2_pipeline_profile pipelineProfile;
private rs2_frame frameset;
private int deviceNumber;
private List streams = new ArrayList<>();
private FrameConverter converter = new OpenCVFrameConverter.ToIplImage();
public RealSense2FrameGrabber() throws Exception {
this(0);
}
public RealSense2FrameGrabber(int deviceNumber) throws Exception {
this.deviceNumber = deviceNumber;
// create context
this.context = createContext();
}
public List getDeviceInfos() throws Exception {
List devices = new ArrayList<>();
rs2_device_list deviceList = createDeviceList();
int count = rs2_get_device_count(deviceList, error);
checkError(error);
for (int i = 0; i < count; i++) {
rs2_device device = createDevice(deviceList, i);
devices.add(new RealSense2DeviceInfo(
getDeviceInfo(device, RS2_CAMERA_INFO_NAME),
getDeviceInfo(device, RS2_CAMERA_INFO_SERIAL_NUMBER),
getDeviceInfo(device, RS2_CAMERA_INFO_FIRMWARE_VERSION),
toBoolean(getDeviceInfo(device, RS2_CAMERA_INFO_ADVANCED_MODE)),
toBoolean(getDeviceInfo(device, RS2_CAMERA_INFO_CAMERA_LOCKED))
));
rs2_delete_device(device);
}
rs2_delete_device_list(deviceList);
return devices;
}
public static String[] getDeviceDescriptions() throws Exception {
RealSense2FrameGrabber rs2 = new RealSense2FrameGrabber();
List infos = rs2.getDeviceInfos();
rs2.release();
String[] deviceDescriptions = new String[infos.size()];
for (int i = 0; i < deviceDescriptions.length; i++) {
RealSense2DeviceInfo info = infos.get(i);
deviceDescriptions[i] = info.toString();
}
return deviceDescriptions;
}
public void disableAllStreams() {
streams.clear();
}
public List getEnabledStreams() {
return this.streams;
}
public void enableStream(RealSenseStream stream) {
streams.add(stream);
}
public void enableColorStream(int width, int height, int frameRate) {
enableStream(new RealSenseStream(
RS2_STREAM_COLOR,
0,
new Size(width, height),
frameRate,
RS2_FORMAT_BGR8
));
}
public void enableDepthStream(int width, int height, int frameRate) {
enableStream(new RealSenseStream(
RS2_STREAM_DEPTH,
0,
new Size(width, height),
frameRate,
RS2_FORMAT_Z16
));
}
public void enableIRStream(int width, int height, int frameRate, int index) {
enableStream(new RealSenseStream(
RS2_STREAM_INFRARED,
index,
new Size(width, height),
frameRate,
RS2_FORMAT_Y8
));
}
public void enableIRStream(int width, int height, int frameRate) {
enableIRStream(width, height, frameRate, 1);
}
public void open() throws Exception {
// check if device is available
if (getDeviceCount() <= 0) {
throw new Exception("No realsense2 device is connected.");
}
// create device
rs2_device_list devices = createDeviceList();
this.device = createDevice(devices, this.deviceNumber);
rs2_delete_device_list(devices);
}
@Override
public void start() throws Exception {
if (this.device == null) {
open();
}
// create pipeline
this.pipeline = createPipeline();
this.config = createConfig();
// check if streams is not empty
if (streams.isEmpty()) {
enableAllVideoStreams();
Collections.sort(streams);
}
// enable streams
for (RealSenseStream stream : streams) {
rs2_config_enable_stream(config,
stream.type,
stream.index,
stream.size.width(),
stream.size.height(),
stream.format,
stream.frameRate,
error);
checkError(error);
}
// set image width & height to the largest stream
RealSenseStream largestStream = getLargestStreamByArea();
this.imageWidth = largestStream.size.width();
this.imageHeight = largestStream.size.height();
// start pipeline
pipelineProfile = rs2_pipeline_start_with_config(pipeline, config, error);
checkError(error);
}
@Override
public void stop() throws Exception {
rs2_pipeline_stop(this.pipeline, error);
checkError(error);
rs2_release_frame(this.frameset);
rs2_delete_pipeline_profile(this.pipelineProfile);
rs2_delete_config(this.config);
rs2_delete_pipeline(this.pipeline);
rs2_delete_device(this.device);
this.device = null;
}
private void readNextFrameSet() throws Exception {
// release previous frame
rs2_release_frame(this.frameset);
// read frames
this.frameset = rs2_pipeline_wait_for_frames(pipeline, RS2_DEFAULT_TIMEOUT, error);
checkError(error);
}
@Override
public void trigger() throws Exception {
// set trigger load flag
if (!triggerMode)
triggerMode = true;
// read frames
readNextFrameSet();
}
@Override
public Frame grab() throws Exception {
int videoStreamId = Math.max(0, videoStream);
RealSenseStream stream = streams.get(videoStreamId);
switch (stream.type) {
case RS2_STREAM_DEPTH:
return grabDepth();
case RS2_STREAM_INFRARED:
return grabIR();
default:
return grabColor();
}
}
public Frame grab(int streamType, int streamIndex, int iplDepth, int channels) throws Exception {
if (!triggerMode)
readNextFrameSet();
return grabCVFrame(streamType, streamIndex, iplDepth, channels);
}
public float getDistance(int x, int y) throws Exception {
rs2_frame frame = findFrameByStreamType(this.frameset, RS2_STREAM_DEPTH, 0);
if (frame == null)
return -1f;
float distance = rs2_depth_frame_get_distance(frame, x, y, error);
checkError(error);
rs2_release_frame(frame);
return distance;
}
public Frame grabColor() throws Exception {
if (!triggerMode)
readNextFrameSet();
return grabCVFrame(RS2_STREAM_COLOR, 0, IPL_DEPTH_8U, 3);
}
public Frame grabDepth() throws Exception {
if (!triggerMode)
readNextFrameSet();
return grabCVFrame(RS2_STREAM_DEPTH, 0, IPL_DEPTH_16U, 1);
}
public Frame grabIR() throws Exception {
return grabIR(0);
}
public Frame grabIR(int streamIndex) throws Exception {
if (!triggerMode)
readNextFrameSet();
return grabCVFrame(RS2_STREAM_INFRARED, streamIndex, IPL_DEPTH_8U, 1);
}
private RealSenseStream getLargestStreamByArea() {
RealSenseStream largest = streams.get(0);
for (RealSenseStream rs : streams) {
if (rs.size.area() > largest.size.area()) {
largest = rs;
}
}
return largest;
}
private Frame grabCVFrame(int streamType, int streamIndex, int iplDepth, int iplChannels) throws Exception {
Frame outputFrame;
// get frame of type if available
rs2_frame frame = findFrameByStreamType(this.frameset, streamType, streamIndex);
if (frame == null)
return null;
// get frame data
Pointer frameData = getFrameData(frame);
Size size = getFrameSize(frame);
// create cv frame
IplImage image = IplImage.createHeader(size.width(), size.height(), iplDepth, iplChannels);
cvSetData(image, frameData, size.width() * iplChannels * iplDepth / 8);
outputFrame = converter.convert(image);
// add timestamp
double timestamp = getFrameTimeStamp(frame);
outputFrame.timestamp = Math.round(timestamp);
// cleanup
rs2_release_frame(frame);
return outputFrame;
}
private rs2_frame findFrameByStreamType(rs2_frame frameset, int streamType, int index) throws Exception {
rs2_frame result = null;
// read frames
int frameCount = rs2_embedded_frames_count(frameset, error);
checkError(error);
int i = 0;
int searchIndex = 0;
while (i < frameCount) {
rs2_frame frame = rs2_extract_frame(frameset, i, error);
checkError(error);
// get stream profile data
rs2_stream_profile streamProfile = getStreamProfile(frame);
StreamProfileData streamProfileData = getStreamProfileData(streamProfile);
// compare stream type
if (streamType == streamProfileData.nativeStreamIndex.get()) {
if (searchIndex == index) {
result = frame;
break;
}
searchIndex++;
}
rs2_release_frame(frame);
i++;
}
return result;
}
@Override
public void release() {
rs2_delete_device(this.device);
rs2_delete_context(this.context);
}
public void setSensorOption(Rs2SensorType sensorType, int optionIndex, boolean value) throws Exception {
setSensorOption(sensorType, optionIndex, value ? 1f : 0f);
}
public void setSensorOption(Rs2SensorType sensorType, int optionIndex, float value) throws Exception {
rs2_sensor[] sensors = getSensors(device);
for (rs2_sensor sensor : sensors) {
checkError(error);
// check if name matches
String name = getSensorInfo(sensor, RS2_CAMERA_INFO_NAME);
if (sensorType.getName().equals(name)) {
rs2_options options = new rs2_options(sensor);
setRs2Option(options, optionIndex, value);
}
// cleanup
rs2_delete_sensor(sensor);
}
}
private rs2_context createContext() throws Exception {
rs2_context context = rs2_create_context(RS2_API_VERSION, error);
checkError(error);
return context;
}
private rs2_device_list createDeviceList() throws Exception {
rs2_device_list deviceList = rs2_query_devices(context, error);
checkError(error);
return deviceList;
}
private rs2_device createDevice(rs2_device_list deviceList, int index) throws Exception {
rs2_device device = rs2_create_device(deviceList, index, error);
checkError(error);
return device;
}
private rs2_pipeline createPipeline() throws Exception {
rs2_pipeline pipeline = rs2_create_pipeline(context, error);
checkError(error);
return pipeline;
}
private rs2_config createConfig() throws Exception {
rs2_config config = rs2_create_config(error);
checkError(error);
return config;
}
private double getFrameTimeStamp(rs2_frame frame) throws Exception {
double timestamp = rs2_get_frame_timestamp(frame, error);
checkError(error);
return timestamp;
}
private int getDeviceCount() throws Exception {
rs2_device_list deviceList = createDeviceList();
int count = rs2_get_device_count(deviceList, error);
checkError(error);
rs2_delete_device_list(deviceList);
return count;
}
private String getDeviceInfo(rs2_device device, int info) throws Exception {
// check if info is supported
rs2_error error = new rs2_error();
boolean isSupported = toBoolean(rs2_supports_device_info(device, info, error));
checkError(error);
if (!isSupported)
return null;
// read device info
String infoText = rs2_get_device_info(device, info, error).getString();
checkError(error);
return infoText;
}
private String getSensorInfo(rs2_sensor sensor, int info) throws Exception {
// check if info is supported
rs2_error error = new rs2_error();
boolean isSupported = toBoolean(rs2_supports_sensor_info(sensor, info, error));
checkError(error);
if (!isSupported)
return null;
// read sensor info
String infoText = rs2_get_sensor_info(sensor, info, error).getString();
checkError(error);
return infoText;
}
private Pointer getFrameData(rs2_frame frame) throws Exception {
Pointer frameData = rs2_get_frame_data(frame, error);
checkError(error);
return frameData;
}
private Size getFrameSize(rs2_frame frame) throws Exception {
int width = rs2_get_frame_width(frame, error);
checkError(error);
int height = rs2_get_frame_height(frame, error);
checkError(error);
return new Size(width, height);
}
private rs2_stream_profile getStreamProfile(rs2_frame frame) throws Exception {
rs2_stream_profile streamProfile = rs2_get_frame_stream_profile(frame, error);
checkError(error);
return streamProfile;
}
private StreamProfileData getStreamProfileData(rs2_stream_profile streamProfile) throws Exception {
StreamProfileData profileData = new StreamProfileData();
if (isStreamProfile(streamProfile, RS2_EXTENSION_VIDEO_PROFILE)) {
VideoStreamProfileData videoStreamProfileData = new VideoStreamProfileData();
rs2_get_video_stream_resolution(streamProfile,
videoStreamProfileData.width,
videoStreamProfileData.height,
error);
checkError(error);
profileData = videoStreamProfileData;
}
rs2_get_stream_profile_data(streamProfile,
profileData.nativeStreamIndex,
profileData.nativeFormatIndex,
profileData.index,
profileData.uniqueId,
profileData.frameRate,
error);
checkError(error);
return profileData;
}
private boolean isSensorExtendableTo(rs2_sensor sensor, int extension) throws Exception {
boolean isExtandable = toBoolean(rs2_is_sensor_extendable_to(sensor, extension, error));
checkError(error);
return isExtandable;
}
private boolean isStreamProfile(rs2_stream_profile profile, int type) throws Exception {
boolean isOfType = toBoolean(rs2_stream_profile_is(profile, type, error));
checkError(error);
return isOfType;
}
private boolean matchesVideoStreamProfile(rs2_stream_profile profile,
int streamType,
int streamFormat,
int frameRate,
int width,
int height) throws Exception {
VideoStreamProfileData data = (VideoStreamProfileData) getStreamProfileData(profile);
return streamType == data.nativeStreamIndex.get()
&& streamFormat == data.nativeFormatIndex.get()
&& frameRate == data.frameRate.get()
&& width == data.width.get()
&& height == data.height.get();
}
private void setRs2Option(rs2_options options, int optionIndex, float value) throws Exception {
boolean isSupported = toBoolean(rs2_supports_option(options, optionIndex, error));
checkError(error);
if (!isSupported) {
throw new Exception("Option " + optionIndex + " is not supported!");
}
rs2_set_option(options, optionIndex, value, error);
checkError(error);
}
private rs2_sensor[] getSensors(rs2_device device) throws Exception {
rs2_sensor_list sensorList = rs2_query_sensors(device, error);
checkError(error);
int sensorCount = rs2_get_sensors_count(sensorList, error);
checkError(error);
rs2_sensor[] sensors = new rs2_sensor[sensorCount];
for (int i = 0; i < sensorCount; i++) {
rs2_sensor sensor = rs2_create_sensor(sensorList, i, error);
checkError(error);
sensors[i] = sensor;
}
rs2_delete_sensor_list(sensorList);
return sensors;
}
private rs2_stream_profile[] getStreamProfiles(rs2_sensor sensor) throws Exception {
rs2_stream_profile_list streamList = rs2_get_stream_profiles(sensor, error);
checkError(error);
int streamProfileCount = rs2_get_stream_profiles_count(streamList, error);
checkError(error);
rs2_stream_profile[] profiles = new rs2_stream_profile[streamProfileCount];
for (int i = 0; i < streamProfileCount; i++) {
rs2_stream_profile profile = rs2_get_stream_profile(streamList, i, error);
checkError(error);
profiles[i] = profile;
}
rs2_delete_stream_profiles_list(streamList);
return profiles;
}
private void enableAllVideoStreams() throws Exception {
for (rs2_sensor sensor : getSensors(device)) {
if (!isSensorExtendableTo(sensor, RS2_EXTENSION_VIDEO)) {
rs2_delete_sensor(sensor);
continue;
}
for (rs2_stream_profile profile : getStreamProfiles(sensor)) {
int rsFrameRate = frameRate > 0 ? (int) frameRate : defaultFrameRate;
int rsWidth = imageWidth > 0 ? imageWidth : defaultWidth;
int rsHeight = imageWidth > 0 ? imageWidth : defaultHeight;
if (matchesVideoStreamProfile(profile, RS2_STREAM_DEPTH, RS2_FORMAT_Z16, rsFrameRate, rsWidth, rsHeight)) {
enableDepthStream(imageWidth, imageHeight, rsFrameRate);
} else if (matchesVideoStreamProfile(profile, RS2_STREAM_COLOR, RS2_FORMAT_RGB8, rsFrameRate, rsWidth, rsHeight)) {
enableColorStream(imageWidth, imageHeight, rsFrameRate);
} else if (matchesVideoStreamProfile(profile, RS2_STREAM_INFRARED, RS2_FORMAT_Y8, rsFrameRate, rsWidth, rsHeight)) {
enableIRStream(imageWidth, imageHeight, rsFrameRate);
}
}
rs2_delete_sensor(sensor);
}
}
private static void checkError(rs2_error e) throws Exception {
if (!e.isNull()) {
throw new Exception(String.format("rs_error was raised when calling %s(%s):\n%s\n",
rs2_get_failed_function(e).getString(),
rs2_get_failed_args(e).getString(),
rs2_get_error_message(e).getString()));
}
}
private static boolean toBoolean(int value) {
return value >= 1;
}
private static boolean toBoolean(String value) {
if (value == null)
return false;
return value.equals("YES");
}
static class StreamProfileData {
IntPointer nativeStreamIndex = new IntPointer(1);
IntPointer nativeFormatIndex = new IntPointer(1);
IntPointer index = new IntPointer(1);
IntPointer uniqueId = new IntPointer(1);
IntPointer frameRate = new IntPointer(1);
}
static class VideoStreamProfileData extends StreamProfileData {
IntPointer width = new IntPointer(1);
IntPointer height = new IntPointer(1);
}
public static class RealSenseStream implements Comparable {
private int type;
private int index;
private Size size;
private int frameRate;
private int format;
public RealSenseStream(int type, int index, Size size, int frameRate, int format) {
this.type = type;
this.index = index;
this.size = size;
this.frameRate = frameRate;
this.format = format;
}
public int getType() {
return type;
}
public int getIndex() {
return index;
}
public Size getSize() {
return size;
}
public int getFrameRate() {
return frameRate;
}
public int getFormat() {
return format;
}
@Override
public int compareTo(RealSenseStream o) {
return Integer.compare(getType(), o.getType());
}
}
public static class RealSense2DeviceInfo {
private String name;
private String serialNumber;
private String firmware;
private boolean inAdvancedMode;
private boolean locked;
RealSense2DeviceInfo(String name, String serialNumber, String firmware, boolean inAdvancedMode, boolean locked) {
this.name = name;
this.serialNumber = serialNumber;
this.firmware = firmware;
this.inAdvancedMode = inAdvancedMode;
this.locked = locked;
}
public String getName() {
return name;
}
public String getSerialNumber() {
return serialNumber;
}
public String getFirmware() {
return firmware;
}
public boolean isInAdvancedMode() {
return inAdvancedMode;
}
public boolean isLocked() {
return locked;
}
@Override
public String toString() {
return String.format("%s", name);
}
}
public enum Rs2SensorType {
StereoModule("Stereo Module"),
RGBCamera("RGB Camera");
private String name;
Rs2SensorType(String name) {
this.name = name;
}
public String getName() {
return name;
}
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/RealSenseFrameGrabber.java
================================================
/*
* Copyright (C) 2014 Jeremy Laviole, Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.io.File;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.ShortBuffer;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.javacpp.Pointer;
import org.bytedeco.librealsense.*;
import org.bytedeco.librealsense.global.RealSense;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
*
* @author Jeremy Laviole
*/
public class RealSenseFrameGrabber extends FrameGrabber {
public static String[] getDeviceDescriptions() throws FrameGrabber.Exception {
tryLoad();
String[] desc = new String[context.get_device_count()];
for (int i = 0; i < desc.length; i++) {
desc[i] = context.get_device(i).get_name().getString();
}
return desc;
}
public static int DEFAULT_DEPTH_WIDTH = 640;
public static int DEFAULT_DEPTH_HEIGHT = 480;
public static int DEFAULT_COLOR_WIDTH = 1280;
public static int DEFAULT_COLOR_HEIGHT = 720;
public static int DEFAULT_COLOR_FRAMERATE = 30;
private ByteOrder byteOrder = ByteOrder.BIG_ENDIAN;
private int depthImageWidth = DEFAULT_DEPTH_WIDTH;
private int depthImageHeight = DEFAULT_DEPTH_HEIGHT;
private int depthFrameRate = 30;
private int IRImageWidth = DEFAULT_DEPTH_WIDTH;
private int IRImageHeight = DEFAULT_DEPTH_HEIGHT;
private int IRFrameRate = 30;
public ByteOrder getByteOrder() {
return byteOrder;
}
public void setByteOrder(ByteOrder byteOrder) {
this.byteOrder = byteOrder;
}
public static RealSenseFrameGrabber createDefault(int deviceNumber) throws FrameGrabber.Exception {
return new RealSenseFrameGrabber(deviceNumber);
}
public static RealSenseFrameGrabber createDefault(File deviceFile) throws Exception {
throw new Exception(RealSenseFrameGrabber.class + " does not support File devices.");
}
public static RealSenseFrameGrabber createDefault(String devicePath) throws Exception {
throw new Exception(RealSenseFrameGrabber.class + " does not support path.");
}
private static FrameGrabber.Exception loadingException = null;
public static void tryLoad() throws FrameGrabber.Exception {
if (loadingException != null) {
loadingException.printStackTrace();
throw loadingException;
} else {
try {
if (context != null) {
return;
}
Loader.load(RealSense.class);
// Context is shared accross cameras.
context = new context();
System.out.println("RealSense devices found: " + context.get_device_count());
} catch (Throwable t) {
throw loadingException = new FrameGrabber.Exception("Failed to load " + RealSenseFrameGrabber.class, t);
}
}
}
private static context context = null;
private int deviceNumber = 0;
private device device = null;
private static device globalDevice = null;
private boolean depth = false; // default to "video"
private boolean colorEnabled = false;
private boolean depthEnabled = false;
private boolean IREnabled = false;
private FrameConverter converter = new OpenCVFrameConverter.ToIplImage();
public RealSenseFrameGrabber(int deviceNumber) {
this.deviceNumber = deviceNumber;
}
public static void main(String[] args) {
context context = new context();
System.out.println("Devices found: " + context.get_device_count());
device device = context.get_device(0);
System.out.println("Using device 0, an " + device.get_name());
System.out.println(" Serial number: " + device.get_serial());
}
public void enableColorStream() {
if (!colorEnabled) {
if (imageWidth == 0) {
imageWidth = DEFAULT_COLOR_WIDTH;
}
if (imageHeight == 0) {
imageHeight = DEFAULT_COLOR_HEIGHT;
}
if (frameRate == 0) {
frameRate = DEFAULT_COLOR_FRAMERATE;
}
colorEnabled = true;
}
}
public void disableColorStream() {
if (colorEnabled) {
device.disable_stream(RealSense.color);
colorEnabled = false;
}
}
public void enableDepthStream() {
if (!depthEnabled) {
depthEnabled = true;
}
}
public void disableDepthStream() {
if (depthEnabled) {
device.disable_stream(RealSense.depth);
depthEnabled = false;
}
}
public void enableIRStream() {
if (!IREnabled) {
IREnabled = true;
}
}
public void disableIRStream() {
if (IREnabled) {
device.disable_stream(RealSense.infrared);
IREnabled = false;
}
}
public void release() throws FrameGrabber.Exception {
}
@Override
protected void finalize() throws Throwable {
super.finalize();
release();
}
/**
* Warning can lead to unsafe situations.
*
* @return
*/
public device getRealSenseDevice() {
return this.device;
}
public float getDepthScale() {
return device.get_depth_scale();
}
@Override
public double getFrameRate() { // TODO: check this.
return super.getFrameRate();
}
private boolean startedOnce = false;
private boolean behaveAsColorFrameGrabber = false;
public device loadDevice() throws FrameGrabber.Exception {
if (context == null) {
context = new context();
}
if (context == null || context.get_device_count() <= deviceNumber) {
throw new Exception("FATAL error: Realsense camera: " + deviceNumber + " not connected/found");
}
device = context.get_device(deviceNumber);
return device;
}
@Override
public void start() throws FrameGrabber.Exception {
// There is already a device, we reboot everything (for Procamcalib).
if (globalDevice != null) {
globalDevice.close();
context.close();
globalDevice = null;
context = null;
}
if (context == null) {
context = new context();
}
if (context == null || context.get_device_count() <= deviceNumber) {
throw new Exception("FATAL error: Realsense camera: " + deviceNumber + " not connected/found");
}
if (device == null) {
device = context.get_device(deviceNumber);
}
globalDevice = device;
// Choose the camera to enable by format.
if (format != null) {
switch (format) {
case "rgb":
this.enableColorStream();
break;
case "ir":
this.enableIRStream();
break;
case "depth":
this.enableDepthStream();
break;
}
}
if (colorEnabled) {
device.enable_stream(RealSense.color, imageWidth, imageHeight, RealSense.rgb8, (int) frameRate);
}
if (IREnabled) {
device.enable_stream(RealSense.infrared, IRImageWidth, IRImageHeight, RealSense.y8, IRFrameRate);
}
if (depthEnabled) {
device.enable_stream(RealSense.depth, depthImageWidth, depthImageHeight, RealSense.z16, depthFrameRate);
}
// if no stream is select, just get the color.
if (!colorEnabled && !IREnabled && !depthEnabled) {
enableColorStream();
device.enable_stream(RealSense.color, imageWidth, imageHeight, RealSense.rgb8, (int) frameRate);
behaveAsColorFrameGrabber = true;
}
device.start();
}
/**
*
* @throws Exception
*/
@Override
public void stop() throws FrameGrabber.Exception {
device.stop();
// colorEnabled = false;
// IREnabled = false;
// depthEnabled = false;
frameNumber = 0;
}
private Pointer rawDepthImageData = new Pointer((Pointer) null),
rawVideoImageData = new Pointer((Pointer) null),
rawIRImageData = new Pointer((Pointer) null);
private IplImage rawDepthImage = null, rawVideoImage = null, rawIRImage = null, returnImage = null;
public IplImage grabDepth() {
if (!depthEnabled) {
System.out.println("Depth stream not enabled, impossible to get the image.");
return null;
}
rawDepthImageData = device.get_frame_data(RealSense.depth);
// ShortBuffer bb = data.position(0).limit(640 * 480 * 2).asByteBuffer().asShortBuffer();
int iplDepth = IPL_DEPTH_16U, channels = 1;
int deviceWidth = device.get_stream_width(RealSense.depth);
int deviceHeight = device.get_stream_height(RealSense.depth);
// AUTOMATIC
// int deviceWidth = 0;
// int deviceHeight = 0;
if (rawDepthImage == null || rawDepthImage.width() != deviceWidth || rawDepthImage.height() != deviceHeight) {
rawDepthImage = IplImage.createHeader(deviceWidth, deviceHeight, iplDepth, channels);
}
cvSetData(rawDepthImage, rawDepthImageData, deviceWidth * channels * iplDepth / 8);
// if (iplDepth > 8 && !ByteOrder.nativeOrder().equals(byteOrder)) {
// // ack, the camera's endianness doesn't correspond to our machine ...
// // swap bytes of 16-bit images
// ByteBuffer bb = rawDepthImage.getByteBuffer();
// ShortBuffer in = bb.order(ByteOrder.BIG_ENDIAN).asShortBuffer();
// ShortBuffer out = bb.order(ByteOrder.LITTLE_ENDIAN).asShortBuffer();
// out.put(in);
// }
return rawDepthImage;
}
public IplImage grabVideo() {
if (!colorEnabled) {
System.out.println("Color stream not enabled, impossible to get the image.");
return null;
}
int iplDepth = IPL_DEPTH_8U, channels = 3;
rawVideoImageData = device.get_frame_data(RealSense.color);
int deviceWidth = device.get_stream_width(RealSense.color);
int deviceHeight = device.get_stream_height(RealSense.color);
if (rawVideoImage == null || rawVideoImage.width() != deviceWidth || rawVideoImage.height() != deviceHeight) {
rawVideoImage = IplImage.createHeader(deviceWidth, deviceHeight, iplDepth, channels);
}
cvSetData(rawVideoImage, rawVideoImageData, deviceWidth * channels * iplDepth / 8);
// if (iplDepth > 8 && !ByteOrder.nativeOrder().equals(byteOrder)) {
// // ack, the camera's endianness doesn't correspond to our machine ...
// // swap bytes of 16-bit images
// ByteBuffer bb = rawVideoImage.getByteBuffer();
// ShortBuffer in = bb.order(ByteOrder.BIG_ENDIAN).asShortBuffer();
// ShortBuffer out = bb.order(ByteOrder.LITTLE_ENDIAN).asShortBuffer();
// out.put(in);
// }
// if (channels == 3) {
// cvCvtColor(rawVideoImage, rawVideoImage, CV_BGR2RGB);
// }
return rawVideoImage;
}
public IplImage grabIR() {
if (!IREnabled) {
System.out.println("IR stream not enabled, impossible to get the image.");
return null;
}
int iplDepth = IPL_DEPTH_8U, channels = 1;
rawIRImageData = device.get_frame_data(RealSense.infrared);
int deviceWidth = device.get_stream_width(RealSense.infrared);
int deviceHeight = device.get_stream_height(RealSense.infrared);
if (rawIRImage == null || rawIRImage.width() != deviceWidth || rawIRImage.height() != deviceHeight) {
rawIRImage = IplImage.createHeader(deviceWidth, deviceHeight, iplDepth, channels);
}
cvSetData(rawIRImage, rawIRImageData, deviceWidth * channels * iplDepth / 8);
// if (iplDepth > 8 && !ByteOrder.nativeOrder().equals(byteOrder)) {
// // ack, the camera's endianness doesn't correspond to our machine ...
// // swap bytes of 16-bit images
// ByteBuffer bb = rawIRImage.getByteBuffer();
// ShortBuffer in = bb.order(ByteOrder.BIG_ENDIAN).asShortBuffer();
// ShortBuffer out = bb.order(ByteOrder.LITTLE_ENDIAN).asShortBuffer();
// out.put(in);
// }
return rawIRImage;
}
/**
*
* @return null grabs all images, get them with grabColor, grabDepth, and
* grabIR instead.
* @throws org.bytedeco.javacv.FrameGrabber.Exception
*/
public Frame grab() throws Exception {
device.wait_for_frames();
// frameNumber++;
// For Framegrabber
if (colorEnabled && behaveAsColorFrameGrabber) {
IplImage image = grabVideo();
if (returnImage == null) {
int deviceWidth = device.get_stream_width(RealSense.color);
int deviceHeight = device.get_stream_height(RealSense.color);
// returnImage = IplImage.create(deviceWidth, deviceHeight, IPL_DEPTH_8U, 3);
returnImage = IplImage.create(deviceWidth, deviceHeight, IPL_DEPTH_8U, 1);
}
cvCvtColor(image, returnImage, CV_BGR2GRAY);
return converter.convert(returnImage);
} else {
if (IREnabled) {
return converter.convert(grabIR());
} else {
if (depthEnabled) {
// Fake colors
IplImage image = grabDepth();
if (returnImage == null) {
int deviceWidth = device.get_stream_width(RealSense.depth);
int deviceHeight = device.get_stream_height(RealSense.depth);
// returnImage = IplImage.create(deviceWidth, deviceHeight, IPL_DEPTH_8U, 3);
returnImage = IplImage.create(deviceWidth, deviceHeight, IPL_DEPTH_8U, 1);
}
return converter.convert(returnImage);
}
}
}
return null;
}
@Override
public void trigger() throws Exception {
device.wait_for_frames();
}
public int getDepthImageWidth() {
return depthImageWidth;
}
public void setDepthImageWidth(int depthImageWidth) {
this.depthImageWidth = depthImageWidth;
}
public int getDepthImageHeight() {
return depthImageHeight;
}
public void setDepthImageHeight(int depthImageHeight) {
this.depthImageHeight = depthImageHeight;
}
public int getIRImageWidth() {
return IRImageWidth;
}
public void setIRImageWidth(int IRImageWidth) {
this.IRImageWidth = IRImageWidth;
}
public int getIRImageHeight() {
return IRImageHeight;
}
public void setIRImageHeight(int IRImageHeight) {
this.IRImageHeight = IRImageHeight;
}
public int getDepthFrameRate() {
return depthFrameRate;
}
public void setDepthFrameRate(int frameRate) {
this.depthFrameRate = frameRate;
}
public int getIRFrameRate() {
return IRFrameRate;
}
public void setIRFrameRate(int IRFrameRate) {
this.IRFrameRate = IRFrameRate;
}
@Override
public double getGamma() {
// I guess a default gamma of 2.2 is reasonable...
if (gamma == 0.0) {
return 2.2;
} else {
return gamma;
}
}
// Gamma from the device is not good.
// @Override
// public double getGamma(){
// double gamma = device.get_option(RealSense.RS_OPTION_COLOR_GAMMA);
// System.out.println("Getting cameraGamma " + gamma);
// return gamma;
// }
// --- Presets ---
public void setPreset(int preset) {
/* Provide access to several recommend sets of option presets for ivcam */
RealSense.apply_ivcam_preset(device, preset);
}
public void setShortRange() {
setPreset(RealSense.RS_IVCAM_PRESET_SHORT_RANGE);
}
public void setLongRange() {
setPreset(RealSense.RS_IVCAM_PRESET_LONG_RANGE);
}
public void setMidRange() {
setPreset(RealSense.RS_IVCAM_PRESET_MID_RANGE);
}
public void setDefaultPreset() {
setPreset(RealSense.RS_IVCAM_PRESET_DEFAULT);
}
public void setObjectScanningPreset() {
setPreset(RealSense.RS_IVCAM_PRESET_OBJECT_SCANNING);
}
public void setCursorPreset() {
setPreset(RealSense.RS_IVCAM_PRESET_GR_CURSOR);
}
public void setGestureRecognitionPreset() {
setPreset(RealSense.RS_IVCAM_PRESET_GESTURE_RECOGNITION);
}
public void setBackgroundSegmentationPreset() {
setPreset(RealSense.RS_IVCAM_PRESET_BACKGROUND_SEGMENTATION);
}
public void setIROnlyPreset() {
setPreset(RealSense.RS_IVCAM_PRESET_IR_ONLY);
}
// ---- Options ----
public void setOption(int option, int value) {
device.set_option(option, value);
}
/**
* Enable / disable color backlight compensation
*/
public void set(int value) {
setOption(RealSense.RS_OPTION_COLOR_BACKLIGHT_COMPENSATION, value);
}
/**
* Color image brightness
*/
public void setColorBrightness(int value) {
setOption(RealSense.RS_OPTION_COLOR_BRIGHTNESS, value);
}
/**
* COLOR image contrast
*/
public void setColorContrast(int value) {
setOption(RealSense.RS_OPTION_COLOR_CONTRAST, value);
}
/**
* Controls exposure time of color camera. Setting any value will disable
* auto exposure
*/
public void setColorExposure(int value) {
setOption(RealSense.RS_OPTION_COLOR_EXPOSURE, value);
}
/**
* Color image gain
*/
public void setColorGain(int value) {
setOption(RealSense.RS_OPTION_COLOR_GAIN, value);
}
/**
* Color image gamma setting
*/
public void setColorGamma(int value) {
setOption(RealSense.RS_OPTION_COLOR_GAMMA, value);
}
/**
* Color image hue
*/
public void setColorHue(int value) {
setOption(RealSense.RS_OPTION_COLOR_HUE, value);
}
/**
* Color image saturation setting
*/
public void setColorSaturation(int value) {
setOption(RealSense.RS_OPTION_COLOR_SATURATION, value);
}
/**
* Color image sharpness setting
*/
public void setColorSharpness(int value) {
setOption(RealSense.RS_OPTION_COLOR_SHARPNESS, value);
}
/**
* Controls white balance of color image. Setting any value will disable
* auto white balance
*/
public void setColorWhiteBalance(int value) {
setOption(RealSense.RS_OPTION_COLOR_WHITE_BALANCE, value);
}
/**
* Enable / disable color image auto-exposure
*/
public void setColorEnableAutoExposure(int value) {
setOption(RealSense.RS_OPTION_COLOR_ENABLE_AUTO_EXPOSURE, value);
}
/**
* Enable / disable color image auto-white-balance
*/
public void setColorEnableAutoWhiteBalance(int value) {
setOption(RealSense.RS_OPTION_COLOR_ENABLE_AUTO_WHITE_BALANCE, value);
}
/**
* Power of the F200 / SR300 projector, with 0 meaning projector off
*/
public void setLaserPower(int value) {
setOption(RealSense.RS_OPTION_F200_LASER_POWER, value);
}
/**
* Set the number of patterns projected per frame. The higher the accuracy
* value the more patterns projected. Increasing the number of patterns help
* to achieve better accuracy. Note that this control is affecting the Depth
* FPS
*/
public void setAccuracy(int value) {
setOption(RealSense.RS_OPTION_F200_ACCURACY, value);
}
/**
* Motion vs. Range trade-off, with lower values allowing for better motion
* sensitivity and higher values allowing for better depth range
*/
public void setMotionRange(int value) {
setOption(RealSense.RS_OPTION_F200_MOTION_RANGE, value);
}
/**
* Set the filter to apply to each depth frame. Each one of the filter is
* optimized per the application requirements
*/
public void setFilterOption(int value) {
setOption(RealSense.RS_OPTION_F200_FILTER_OPTION, value);
}
/**
* The confidence level threshold used by the Depth algorithm pipe to set
* whether a pixel will get a valid range or will be marked with invalid
* range
*/
public void setConfidenceThreshold(int value) {
setOption(RealSense.RS_OPTION_F200_CONFIDENCE_THRESHOLD, value);
}
/**
* (F200-only) Allows to reduce FPS without restarting streaming. Valid
* values are {2, 5, 15, 30, 60}
*/
public void setDynamicFPS(int value) {
setOption(RealSense.RS_OPTION_F200_DYNAMIC_FPS, value);
}
/**
* Enables / disables R200 auto-exposure. This will affect both IR and depth
* image.
*/
public void setLR_AutoExposureEnabled(int value) {
setOption(RealSense.RS_OPTION_R200_LR_AUTO_EXPOSURE_ENABLED, value);
}
/**
* IR image gain
*/
public void setLR_Gain(int value) {
setOption(RealSense.RS_OPTION_R200_LR_GAIN, value);
}
/**
* This control allows manual adjustment of the exposure time value for the
* L/R imagers
*/
public void setLR_Exposure(int value) {
setOption(RealSense.RS_OPTION_R200_LR_EXPOSURE, value);
}
/**
* Enables / disables R200 emitter
*/
public void setEmitterEnabled(int value) {
setOption(RealSense.RS_OPTION_R200_EMITTER_ENABLED, value);
}
/**
* Micrometers per increment in integer depth values, 1000 is default (mm
* scale). Set before streaming
*/
public void setDepthUnits(int value) {
setOption(RealSense.RS_OPTION_R200_DEPTH_UNITS, value);
}
/**
* Minimum depth in current depth units that will be output. Any values less
* than �Min Depth� will be mapped to 0 during the conversion between
* disparity and depth. Set before streaming
*/
public void setDepthClampMin(int value) {
setOption(RealSense.RS_OPTION_R200_DEPTH_CLAMP_MIN, value);
}
/**
* Maximum depth in current depth units that will be output. Any values
* greater than �Max Depth� will be mapped to 0 during the conversion
* between disparity and depth. Set before streaming
*/
public void setDepthClampMax(int value) {
setOption(RealSense.RS_OPTION_R200_DEPTH_CLAMP_MAX, value);
}
/**
* The disparity scale factor used when in disparity output mode. Can only
* be set before streaming
*/
public void setDisparityMultiplier(int value) {
setOption(RealSense.RS_OPTION_R200_DISPARITY_MULTIPLIER, value);
}
/**
* {0 - 512}. Can only be set before streaming starts
*/
public void setDisparityShift(int value) {
setOption(RealSense.RS_OPTION_R200_DISPARITY_SHIFT, value);
}
/**
* (Requires LR-Auto-Exposure ON) Mean intensity set point
*/
public void setAutoExposureMeanIntensitySetPoint(int value) {
setOption(RealSense.RS_OPTION_R200_AUTO_EXPOSURE_MEAN_INTENSITY_SET_POINT, value);
}
/**
* (Requires LR-Auto-Exposure ON) Bright ratio set point
*/
public void setAutoExposureBrightRatioSetPoint(int value) {
setOption(RealSense.RS_OPTION_R200_AUTO_EXPOSURE_BRIGHT_RATIO_SET_POINT, value);
}
/**
* (Requires LR-Auto-Exposure ON) Kp Gain
*/
public void setAutoExposureKpGain(int value) {
setOption(RealSense.RS_OPTION_R200_AUTO_EXPOSURE_KP_GAIN, value);
}
/**
* (Requires LR-Auto-Exposure ON) Kp Exposure
*/
public void setAutoExposureKpExposure(int value) {
setOption(RealSense.RS_OPTION_R200_AUTO_EXPOSURE_KP_EXPOSURE, value);
}
/**
* (Requires LR-Auto-Exposure ON) Kp Dark Threshold
*/
public void setAutoExposureKpDarkThreshold(int value) {
setOption(RealSense.RS_OPTION_R200_AUTO_EXPOSURE_KP_DARK_THRESHOLD, value);
}
/**
* (Requires LR-Auto-Exposure ON) Auto-Exposure region-of-interest top edge
* (in pixels)
*/
public void setAutoExposureTopEdge(int value) {
setOption(RealSense.RS_OPTION_R200_AUTO_EXPOSURE_TOP_EDGE, value);
}
/**
* (Requires LR-Auto-Exposure ON) Auto-Exposure region-of-interest bottom
* edge (in pixels)
*/
public void setAutoExposureBottomEdge(int value) {
setOption(RealSense.RS_OPTION_R200_AUTO_EXPOSURE_BOTTOM_EDGE, value);
}
/**
* (Requires LR-Auto-Exposure ON) Auto-Exposure region-of-interest left edge
* (in pixels)
*/
public void setAutoExposureLeftEdge(int value) {
setOption(RealSense.RS_OPTION_R200_AUTO_EXPOSURE_LEFT_EDGE, value);
}
/**
* (Requires LR-Auto-Exposure ON) Auto-Exposure region-of-interest right
* edge (in pixels)
*/
public void setAutoExposureRightEdge(int value) {
setOption(RealSense.RS_OPTION_R200_AUTO_EXPOSURE_RIGHT_EDGE, value);
}
/**
* Value to subtract when estimating the median of the correlation surface
*/
public void setDepthControlEstimateMedianDecrement(int value) {
setOption(RealSense.RS_OPTION_R200_DEPTH_CONTROL_ESTIMATE_MEDIAN_DECREMENT, value);
}
/**
* Value to add when estimating the median of the correlation surface
*/
public void setDepthControlEstimateMedianIncrement(int value) {
setOption(RealSense.RS_OPTION_R200_DEPTH_CONTROL_ESTIMATE_MEDIAN_INCREMENT, value);
}
/**
* A threshold by how much the winning score must beat the median
*/
public void setDepthControlMedianThreshold(int value) {
setOption(RealSense.RS_OPTION_R200_DEPTH_CONTROL_MEDIAN_THRESHOLD, value);
}
/**
* The minimum correlation score that is considered acceptable
*/
public void setDepthControlMinimumThreshold(int value) {
setOption(RealSense.RS_OPTION_R200_DEPTH_CONTROL_SCORE_MINIMUM_THRESHOLD, value);
}
/**
* The maximum correlation score that is considered acceptable
*/
public void setDepthControlScoreMaximumThreshold(int value) {
setOption(RealSense.RS_OPTION_R200_DEPTH_CONTROL_SCORE_MAXIMUM_THRESHOLD, value);
}
/**
* A parameter for determining whether the texture in the region is
* sufficient to justify a depth result
*/
public void setDepthControlTextureCountThreshold(int value) {
setOption(RealSense.RS_OPTION_R200_DEPTH_CONTROL_TEXTURE_COUNT_THRESHOLD, value);
}
/**
* A parameter for determining whether the texture in the region is
* sufficient to justify a depth result
*/
public void setDepthControlTextureDifference(int value) {
setOption(RealSense.RS_OPTION_R200_DEPTH_CONTROL_TEXTURE_DIFFERENCE_THRESHOLD, value);
}
/**
* A threshold on how much the minimum correlation score must differ from
* the next best score
*/
public void setDepthControlSecondPeakThreshold(int value) {
setOption(RealSense.RS_OPTION_R200_DEPTH_CONTROL_SECOND_PEAK_THRESHOLD, value);
}
/**
* Neighbor threshold value for depth calculation
*/
public void setDepthControlNeighborThreshold(int value) {
setOption(RealSense.RS_OPTION_R200_DEPTH_CONTROL_NEIGHBOR_THRESHOLD, value);
}
/**
* Left-Right threshold value for depth calculation
*/
public void setDepthControlLRThreshold(int value) {
setOption(RealSense.RS_OPTION_R200_DEPTH_CONTROL_LR_THRESHOLD, value);
}
/**
* Fisheye image exposure time in msec
*/
public void setFisheyeExposure(int value) {
setOption(RealSense.RS_OPTION_FISHEYE_EXPOSURE, value);
}
/**
* Fisheye image gain
*/
public void setFisheyeGain(int value) {
setOption(RealSense.RS_OPTION_FISHEYE_GAIN, value);
}
/**
* Enables / disables fisheye strobe. When enabled this will align
* timestamps to common clock-domain with the motion events
*/
public void setFisheyeStobe(int value) {
setOption(RealSense.RS_OPTION_FISHEYE_STROBE, value);
}
/**
* Enables / disables fisheye external trigger mode. When enabled fisheye
* image will be aquired in-sync with the depth image
*/
public void setFisheyeExternalTrigger(int value) {
setOption(RealSense.RS_OPTION_FISHEYE_EXTERNAL_TRIGGER, value);
}
/**
* Enable / disable fisheye auto-exposure
*/
public void setFisheyeEnableAutoExposure(int value) {
setOption(RealSense.RS_OPTION_FISHEYE_ENABLE_AUTO_EXPOSURE, value);
}
/**
* 0 - static auto-exposure, 1 - anti-flicker auto-exposure, 2 - hybrid
*/
public void setFisheyeAutoExposureMode(int value) {
setOption(RealSense.RS_OPTION_FISHEYE_AUTO_EXPOSURE_MODE, value);
}
/**
* Fisheye auto-exposure anti-flicker rate, can be 50 or 60 Hz
*/
public void setFisheyeAutoExposureAntiflickerRate(int value) {
setOption(RealSense.RS_OPTION_FISHEYE_AUTO_EXPOSURE_ANTIFLICKER_RATE, value);
}
/**
* In Fisheye auto-exposure sample frame every given number of pixels
*/
public void setFisheyeAutoExposurePixelSampleRate(int value) {
setOption(RealSense.RS_OPTION_FISHEYE_AUTO_EXPOSURE_PIXEL_SAMPLE_RATE, value);
}
/**
* In Fisheye auto-exposure sample every given number of frames
*/
public void setFisheyeAutoExposureSkipFrames(int value) {
setOption(RealSense.RS_OPTION_FISHEYE_AUTO_EXPOSURE_SKIP_FRAMES, value);
}
/**
* Number of frames the user is allowed to keep per stream. Trying to
* hold-on to more frames will cause frame-drops.
*/
public void setFramesQueueSize(int value) {
setOption(RealSense.RS_OPTION_FRAMES_QUEUE_SIZE, value);
}
/**
* Enable / disable fetching log data from the device
*/
public void setHardwareLoggerEnabled(int value) {
setOption(RealSense.RS_OPTION_HARDWARE_LOGGER_ENABLED, value);
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/ReflectanceInitializer.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.nio.ByteBuffer;
import java.nio.FloatBuffer;
import java.util.logging.Logger;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
/**
*
* @author Samuel Audet
*/
public class ReflectanceInitializer {
public ReflectanceInitializer(CameraDevice cameraDevice, ProjectorDevice projectorDevice,
int channels, GNImageAligner.Settings alignerSettings) {
this(cameraDevice, projectorDevice, channels, alignerSettings, 51, 0.01);
}
public ReflectanceInitializer(CameraDevice cameraDevice, ProjectorDevice projectorDevice,
int channels, GNImageAligner.Settings alignerSettings, int smoothingSize, double reflectanceMin) {
this.alignerSettings = alignerSettings;
this.smoothingSize = smoothingSize;
this.reflectanceMin = reflectanceMin;
this.cameraDevice = cameraDevice;
this.projectorDevice = projectorDevice;
this.projectorImages = new IplImage[3];
for (int i = 0; i < projectorImages.length; i++) {
projectorImages[i] = IplImage.create(projectorDevice.imageWidth,
projectorDevice.imageHeight, IPL_DEPTH_32F, channels);
}
// capture "black" image (illuminated by ambient light only)
cvSetZero(projectorImages[0]);
// capture "white" image (projector illumination + ambient light)
cvSet(projectorImages[1], CvScalar.ONE);
// capture image with some texture easy to register... ugh...
CvMat H = mat3x3.get();
projectorDevice.getRectifyingHomography(cameraDevice, H);
JavaCV.fractalTriangleWave(projectorImages[2], H);
}
private static ThreadLocal
mat3x1 = CvMat.createThreadLocal(3, 1),
mat3x3 = CvMat.createThreadLocal(3, 3),
mat4x4 = CvMat.createThreadLocal(4, 4);
private GNImageAligner.Settings alignerSettings;
private int smoothingSize;
private double reflectanceMin;
private CameraDevice cameraDevice;
private ProjectorDevice projectorDevice;
private IplImage[] projectorImages;
public IplImage[] getProjectorImages() {
return projectorImages;
}
public IplImage initializeReflectance(IplImage[] cameraImages, IplImage reflectance,
double[] roiPts, double[] gainAmbientLight) {
int w = cameraImages[0].width();
int h = cameraImages[0].height();
int channels = cameraImages[0].nChannels();
IplImage mask = IplImage.create(w, h, IPL_DEPTH_8U, 1);
cvSetZero(mask);
cvFillConvexPoly(mask, new CvPoint(roiPts.length/2).put((byte)(16-cameraDevice.getMapsPyramidLevel()), roiPts),
4, CvScalar.WHITE, 8, 16);
// make the images very very smooth to compensate for small movements
IplImage float1 = cameraImages[0];
IplImage float2 = cameraImages[1];
cvCopy(float2, reflectance);
cvSmooth(float1, float1, CV_GAUSSIAN, smoothingSize, 0, 0, 0);
cvSmooth(float2, float2, CV_GAUSSIAN, smoothingSize, 0, 0, 0);
// remove ambient light of image1 from image2 -> image2
cvSub(float2, float1, float2, null);
// remove distortion caused by the mixing matrix of the projector light
// and recover (very very smooth) reflectance map
CvMat p = mat3x1.get();
p.put(1.0, 1.0, 1.0); // white
cvMatMul(projectorDevice.colorMixingMatrix, p, p);
CvMat invp;
if (float2.nChannels() == 4) {
invp = mat4x4.get();
invp.put(1/p.get(0), 0, 0, 0,
0, 1/p.get(1), 0, 0,
0, 0, 1/p.get(2), 0,
0, 0, 0, 1);
} else {
invp = mat3x3.get();
invp.put(1/p.get(0), 0, 0,
0, 1/p.get(1), 0,
0, 0, 1/p.get(2));
}
cvTransform(float2, float2, invp, null);
// recover (very very smooth) ambient light by removing distortions
// caused by the reflectance map
// cvDiv(image1, image3, image1, 1);
// cvDiv() doesn't support division by zero...
FloatBuffer fb1 = float1.getFloatBuffer();
FloatBuffer fb2 = float2.getFloatBuffer();
ByteBuffer mb = mask.getByteBuffer();
assert fb1.capacity() == fb2.capacity()/3;
assert fb1.capacity() == mb.capacity()/3;
int[] nPixels = new int[channels];
for (int i = 0, j = 0; j < fb1.capacity(); i++, j+=channels) {
for (int z = 0; z < channels; z++) {
float ra = fb1.get(j+z);
float r = fb2.get(j+z);
float a = r == 0 ? 0 : ra/r;
fb1.put(j+z, a);
if (mb.get(i) != 0) {
if (r > reflectanceMin) {
nPixels[z]++;
gainAmbientLight[z+1] += a;
}
}
}
}
gainAmbientLight[0] = 1.0; // assume projector gain = 1.0
for (int z = 0; z < gainAmbientLight.length-1; z++) {
gainAmbientLight[z+1] = nPixels[z] == 0 ? 0 : gainAmbientLight[z+1]/nPixels[z];
}
// System.out.println(ambientLight[0] + " " + ambientLight[1] + " " + ambientLight[2]);
// recover sharp reflectance map by using the original image2 and smooth ambient light
cvAddS(float1, cvScalar(p.get(0), p.get(1), p.get(2), 0.0), float1, null);
cvDiv(reflectance, float1, reflectance, 1.0);
cvNot(mask, mask);
// increase region a bit so that the resulting image can be
// interpolated or averaged properly within the region of interest...
cvErode(mask, mask, null, 15);
cvSet(reflectance, CvScalar.ZERO, mask);
return reflectance;
}
public CvMat initializePlaneParameters(IplImage reflectance, IplImage cameraImage,
double[] referencePoints, double[] roiPts, double[] gainAmbientLight) {
ProCamTransformer transformer = new ProCamTransformer(referencePoints, cameraDevice, projectorDevice, null);
transformer.setProjectorImage(projectorImages[2], 0, alignerSettings.pyramidLevelMax);
ProCamTransformer.Parameters parameters = transformer.createParameters();
// parameters.set(8, 0);
// parameters.set(9, 0);
// parameters.set(10, -1/cameraDevice.getSettings().nominalDistance);
final int gainAmbientLightStart = parameters.size() - gainAmbientLight.length;
final int gainAmbientLightEnd = parameters.size();
for (int i = gainAmbientLightStart; i < gainAmbientLightEnd; i++) {
parameters.set(i, gainAmbientLight[i-gainAmbientLightStart]);
}
ImageAligner aligner = new GNImageAligner(transformer, parameters,
reflectance, roiPts, cameraImage, alignerSettings);
double[] delta = new double[parameters.size()+1];
boolean converged = false;
long iterationsStartTime = System.currentTimeMillis();
int iterations = 0;
while (!converged && iterations < 100) {
converged = aligner.iterate(delta);
iterations++;
}
parameters = (ProCamTransformer.Parameters)aligner.getParameters();
// for (int i = 0; i < gainAmbientLight.length; i++) {
// gainAmbientLight[i] = parameters.get(11+i);
// }
Logger.getLogger(ReflectanceInitializer.class.getName()).info(
"iteratingTime = " + (System.currentTimeMillis()-iterationsStartTime) +
" iterations = " + iterations + " objectiveRMSE = " + (float)aligner.getRMSE());
return parameters.getN0();
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/Seekable.java
================================================
/*
* Copyright (C) 2019 Sven Vorlauf
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
public interface Seekable {
public void seek(long offset, int whence);
}
================================================
FILE: src/main/java/org/bytedeco/javacv/SeekableByteArrayOutputStream.java
================================================
/*
* Copyright (C) 2019 Sven Vorlauf
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.io.ByteArrayOutputStream;
public class SeekableByteArrayOutputStream extends ByteArrayOutputStream implements Seekable {
long position;
@Override public void seek(long position, int whence) {
if (position < 0 || position > count || whence != 0)
throw new IllegalArgumentException();
this.position = position;
}
@Override public synchronized void write(int b) {
if (position < count) {
buf[(int) position] = (byte) b; // position < count <= MAX_INT
} else {
super.write(b);
}
position++;
}
@Override public synchronized void write(byte[] b, int off, int len) {
if (position < count) {
for (int i = 0 ; i < len ; i++) {
write(b[off + i]); // should be changed for bigegr arrays
}
} else {
super.write(b, off, len);
position = count;
}
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/VideoInputFrameGrabber.java
================================================
/*
* Copyright (C) 2011-2013 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.io.File;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import org.bytedeco.videoinput.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
import static org.bytedeco.videoinput.global.videoInputLib.*;
/**
*
* @author Samuel Audet
*/
public class VideoInputFrameGrabber extends FrameGrabber {
public static String[] getDeviceDescriptions() throws Exception {
tryLoad();
int count = videoInput.listDevices();
String[] descriptions = new String[count];
for (int i = 0; i < descriptions.length; i++) {
descriptions[i] = videoInput.getDeviceName(i).getString();
}
return descriptions;
}
public static VideoInputFrameGrabber createDefault(File deviceFile) throws Exception { throw new Exception(VideoInputFrameGrabber.class + " does not support device files."); }
public static VideoInputFrameGrabber createDefault(String devicePath) throws Exception { throw new Exception(VideoInputFrameGrabber.class + " does not support device paths."); }
public static VideoInputFrameGrabber createDefault(int deviceNumber) throws Exception { return new VideoInputFrameGrabber(deviceNumber); }
private static Exception loadingException = null;
public static void tryLoad() throws Exception {
if (loadingException != null) {
throw loadingException;
} else {
try {
Loader.load(org.bytedeco.videoinput.global.videoInputLib.class);
} catch (Throwable t) {
throw loadingException = new Exception("Failed to load " + VideoInputFrameGrabber.class, t);
}
}
}
public VideoInputFrameGrabber(int deviceNumber) {
this.deviceNumber = deviceNumber;
}
public void release() throws Exception {
stop();
}
@Override protected void finalize() throws Throwable {
super.finalize();
release();
}
private int deviceNumber = 0;
private videoInput myVideoInput = null;
private IplImage bgrImage = null, grayImage = null;
private BytePointer bgrImageData = null;
private FrameConverter converter = new OpenCVFrameConverter.ToIplImage();
@Override public double getGamma() {
// default to a gamma of 2.2 for cheap Webcams, DV cameras, etc.
if (gamma == 0.0) {
return 2.2;
} else {
return gamma;
}
}
@Override public int getImageWidth() {
return myVideoInput == null ? super.getImageWidth() : myVideoInput.getWidth(deviceNumber);
}
@Override public int getImageHeight() {
return myVideoInput == null ? super.getImageHeight() : myVideoInput.getHeight(deviceNumber);
}
public void start() throws Exception {
start(-1);
}
public void start(int connection) throws Exception {
myVideoInput = new videoInput();
if (frameRate > 0) {
myVideoInput.setIdealFramerate(deviceNumber, (int)frameRate);
}
if (!myVideoInput.setupDevice(deviceNumber, imageWidth > 0 ? imageWidth : 640,
imageHeight > 0 ? imageHeight : 480, connection)) {
myVideoInput = null;
throw new Exception("videoInput.setupDevice() Error: Could not setup device.");
}
if (format != null && format.length() > 0) {
int f = format.equals("VI_NTSC_M") ? VI_NTSC_M :
format.equals("VI_PAL_B") ? VI_PAL_B :
format.equals("VI_PAL_D") ? VI_PAL_D :
format.equals("VI_PAL_G") ? VI_PAL_G :
format.equals("VI_PAL_H") ? VI_PAL_H :
format.equals("VI_PAL_I") ? VI_PAL_I :
format.equals("VI_PAL_M") ? VI_PAL_M :
format.equals("VI_PAL_N") ? VI_PAL_N :
format.equals("VI_PAL_NC") ? VI_PAL_NC :
format.equals("VI_SECAM_B") ? VI_SECAM_B :
format.equals("VI_SECAM_D") ? VI_SECAM_D :
format.equals("VI_SECAM_G") ? VI_SECAM_G :
format.equals("VI_SECAM_H") ? VI_SECAM_H :
format.equals("VI_SECAM_K") ? VI_SECAM_K :
format.equals("VI_SECAM_K1") ? VI_SECAM_K1 :
format.equals("VI_SECAM_L") ? VI_SECAM_L :
format.equals("VI_NTSC_M_J") ? VI_NTSC_M_J :
format.equals("VI_NTSC_433") ? VI_NTSC_433 : -1;
if (f >= 0 && !myVideoInput.setFormat(deviceNumber, f)) {
throw new Exception("videoInput.setFormat() Error: Could not set format " + format + ".");
}
}
}
public void stop() throws Exception {
if (myVideoInput != null) {
myVideoInput.stopDevice(deviceNumber);
myVideoInput = null;
}
}
public void trigger() throws Exception {
if (myVideoInput == null) {
throw new Exception("videoInput is null. (Has start() been called?)");
}
int w = myVideoInput.getWidth(deviceNumber), h = myVideoInput.getHeight(deviceNumber);
if (bgrImage == null || bgrImage.width() != w || bgrImage.height() != h) {
bgrImage = IplImage.create(w, h, IPL_DEPTH_8U, 3);
bgrImageData = bgrImage.imageData();
}
for (int i = 0; i < numBuffers+1; i++) {
myVideoInput.getPixels(deviceNumber, bgrImageData, false, true);
}
}
public Frame grab() throws Exception {
if (myVideoInput == null) {
throw new Exception("videoInput is null. (Has start() been called?)");
}
int w = myVideoInput.getWidth(deviceNumber), h = myVideoInput.getHeight(deviceNumber);
if (bgrImage == null || bgrImage.width() != w || bgrImage.height() != h) {
bgrImage = IplImage.create(w, h, IPL_DEPTH_8U, 3);
bgrImageData = bgrImage.imageData();
}
if (!myVideoInput.getPixels(deviceNumber, bgrImageData, false, true)) {
throw new Exception("videoInput.getPixels() Error: Could not get pixels.");
}
timestamp = System.nanoTime()/1000;
if (imageMode == ImageMode.GRAY) {
if (grayImage == null) {
grayImage = IplImage.create(w, h, IPL_DEPTH_8U, 1);
}
cvCvtColor(bgrImage, grayImage, CV_BGR2GRAY);
return converter.convert(grayImage);
} else {
return converter.convert(bgrImage);
}
}
}
================================================
FILE: src/main/java/org/bytedeco/javacv/cvkernels.java
================================================
/*
* Copyright (C) 2009-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.bytedeco.javacv;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import org.bytedeco.opencv.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_core.*;
/**
*
* @author Samuel Audet
*/
public class cvkernels extends org.bytedeco.opencv.cvkernels {
private static class ParallelData {
KernelData data = null;
CvRect roi = new CvRect();
}
private static ThreadLocal parallelData = new ThreadLocal() {
@Override protected ParallelData[] initialValue() {
ParallelData[] pd = new ParallelData[Parallel.getNumThreads()];
for (int i = 0; i < pd.length; i++) {
pd[i] = new ParallelData();
}
return pd;
}
};
public static void multiWarpColorTransform(final KernelData data, final CvRect roi, final CvScalar fillColor) {
final int size = (int)data.capacity();
final ParallelData[] pd = parallelData.get();
// Copy all data to completely independent data sets
for (int i = 0; i < pd.length; i++) {
if (pd[i].data == null || pd[i].data.capacity() < size) {
pd[i].data = new KernelData(size);
for (int j = 0; j < size; j++) {
KernelData d = pd[i].data.position(j);
data.position(j);
if (data.dstDstDot() != null) {
d.dstDstDot(ByteBuffer.allocateDirect(data.dstDstDot().capacity()*8).
order(ByteOrder.nativeOrder()).asDoubleBuffer());
}
}
}
for (int j = 0; j < size; j++) {
KernelData d = pd[i].data.position(j);
d.put(data.position(j));
d.dstDstDot(d.dstDstDot()); // reset dstDstDot pointer
}
}
// Transform in parallel on multiple threads
final IplImage img = data.position(0).srcImg();
final int depth = img.depth();
final int x, y, w, h;
if (roi != null) {
x = roi.x(); y = roi.y();
w = roi.width(); h = roi.height();
} else {
x = 0; y = 0;
w = img.width(); h = img.height();
}
Parallel.loop(y, y+h, pd.length, new Parallel.Looper() {
public void loop(int from, int to, int looperID) {
CvRect r = pd[looperID].roi.x(x).y(from).width(w).height(to-from);
if (depth == IPL_DEPTH_32F) {
multiWarpColorTransform32F(pd[looperID].data.position(0), size, r, fillColor);
} else if (depth == IPL_DEPTH_8U) {
multiWarpColorTransform8U(pd[looperID].data.position(0), size, r, fillColor);
} else {
assert false;
}
}});
// Reduce data as required
for (int i = 0; i < size; i++) {
int dstCount = 0;
int dstCountZero = 0;
int dstCountOutlier = 0;
double srcDstDot = 0;
double[] dstDstDot = null;
if (data.dstDstDot() != null) {
dstDstDot = new double[data.dstDstDot().capacity()];
}
for (int j = 0; j < pd.length; j++) {
KernelData d = pd[j].data.position(i);
dstCount += d.dstCount();
dstCountZero += d.dstCountZero();
dstCountOutlier += d.dstCountOutlier();
srcDstDot += d.srcDstDot();
if (dstDstDot != null && d.dstDstDot() != null) {
for (int k = 0; k < dstDstDot.length; k++) {
dstDstDot[k] += d.dstDstDot().get(k);
}
}
}
data.position(i);
data.dstCount(dstCount);
data.dstCountZero(dstCountZero);
data.dstCountOutlier(dstCountOutlier);
data.srcDstDot(srcDstDot);
if (dstDstDot != null && data.dstDstDot() != null) {
data.dstDstDot().position(0);
data.dstDstDot().put(dstDstDot);
}
}
}
}
================================================
FILE: src/main/java9/module-info.java
================================================
module org.bytedeco.javacv {
exports org.bytedeco.javacv;
requires java.desktop;
requires javafx.graphics;
requires transitive org.bytedeco.javacpp;
requires org.bytedeco.opencv;
requires org.bytedeco.ffmpeg;
requires org.bytedeco.flycapture;
requires org.bytedeco.libdc1394;
requires org.bytedeco.libfreenect;
requires org.bytedeco.libfreenect2;
requires org.bytedeco.librealsense;
requires org.bytedeco.librealsense2;
requires org.bytedeco.videoinput;
requires org.bytedeco.artoolkitplus;
// requires org.bytedeco.flandmark;
requires org.bytedeco.leptonica;
requires org.bytedeco.tesseract;
}
================================================
FILE: src/main/resources/org/bytedeco/javacv/ImageTransformer.cl
================================================
/*
* Copyright (C) 2011-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
const sampler_t sampler = CLK_NORMALIZED_COORDS_FALSE |
CLK_ADDRESS_CLAMP | CLK_FILTER_NEAREST;
//const sampler_t linearSampler = CLK_NORMALIZED_COORDS_FALSE |
// CLK_ADDRESS_CLAMP | CLK_FILTER_LINEAR;
inline float4 readLinear(read_only image2d_t img, float2 xy) {
float2 xy00 = floor(xy);
float dx = xy.x - xy00.x;
float dy = xy.y - xy00.y;
float4 rgba = (1-dx)*(1-dy)*read_imagef(img, sampler, xy00);
rgba += dx *(1-dy)*read_imagef(img, sampler, xy00 + (float2)(1, 0));
rgba += (1-dx)* dy *read_imagef(img, sampler, xy00 + (float2)(0, 1));
rgba += dx * dy *read_imagef(img, sampler, xy00 + (float2)(1, 1));
return rgba;
}
#pragma OPENCL EXTENSION cl_khr_global_int32_base_atomics : enable
inline void atomicAddFloat(global float* address, float val) {
global int* address_as_int = (global int*)address;
while (val != 0.0f) {
val += as_float(atom_xchg(address_as_int, as_int(0.0f)));
val = as_float(atom_xchg(address_as_int, as_int(val)));
}
}
// Bit Twiddling Hacks
// http://graphics.stanford.edu/~seander/bithacks.html
inline int ceilPow2(int v) {
v--;
v |= v >> 1;
v |= v >> 2;
v |= v >> 4;
v |= v >> 8;
v |= v >> 16;
return ++v;
}
inline int reduceSumInt(float value, int i, int size, local void* scratch) {
local float *scratchi = (local float*)scratch;
scratchi[i] = value;
barrier(CLK_LOCAL_MEM_FENCE);
for (int offset = ceilPow2(size)/2; offset > 0; offset >>= 1) {
if (i < offset && i + offset < size) {
scratchi[i] += scratchi[i + offset];
}
barrier(CLK_LOCAL_MEM_FENCE);
}
return scratchi[0];
}
inline float reduceSumFloat(float value, int i, int size, local void* scratch) {
local float *scratchf = (local float*)scratch;
scratchf[i] = value;
barrier(CLK_LOCAL_MEM_FENCE);
for (int offset = ceilPow2(size)/2; offset > 0; offset >>= 1) {
if (i < offset && i + offset < size) {
scratchf[i] += scratchf[i + offset];
}
barrier(CLK_LOCAL_MEM_FENCE);
}
return scratchf[0];
}
struct InputData {
int roiY, roiHeight;
float zeroThreshold, outlierThreshold;
};
struct OutputData {
int groupsFinished, dstCount, dstCountZero, dstCountOutlier;
float srcDstDot[DOT_SIZE], dstDstDot[DOT_SIZE][DOT_SIZE];
};
inline void multiWarpColorTransform(read_only image2d_t srcImg, read_only image2d_t srcImg2, read_only image2d_t subImg,
read_only image2d_t dotImg, write_only image2d_t transImg, write_only image2d_t dstImg, read_only image2d_t maskImg,
int width, int height, constant float _H[][9], constant float _H2[][9], constant float/*4*/ _X[][16/*4*/], int size,
constant struct InputData *inputData, global struct OutputData *outputData,
bool haveSubImg, bool haveDotImg, bool haveTransImg, bool haveDstImg, bool haveMaskImg) {
const int x = get_global_id(0), gx = get_group_id(0), lx = get_local_id(0), lsx = get_local_size(0);
const int y = get_global_id(1), gy = get_group_id(1), ly = get_local_id(1), lsy = get_local_size(1);
const int z = get_global_id(2), gz = get_group_id(2), lz = get_local_id(2), lsz = get_local_size(2);
int dstCount = 0, dstCountZero = 0, dstCountOutlier = 0;
float srcDstDot = 0, dstDstDot = 0;
local float scratch[DOT_SIZE + 1][DOT_SIZE | 1][3];
local float H[DOT_SIZE][9], H2[DOT_SIZE][9], X[DOT_SIZE][12];
if (lx < size) {
for (int j = 0; j < 9; j++) {
H[lx][j] = _H[lx][j];
if (_H2) H2[lx][j] = _H2[lx][j];
}
for (int j = 0; j < 12; j++) {
if (_X) X[lx][j] = _X[lx][j];
}
}
barrier(CLK_LOCAL_MEM_FENCE);
for (int y = inputData->roiY; y < inputData->roiY + inputData->roiHeight; y++) {
const int2 xy = (int2)(x, y);
float4 dotRGB = 0, dstRGB = 0;
if (x >= width) {
goto skipPixel;
}
if (haveMaskImg) {
if (read_imagei(maskImg, sampler, xy).x == 0) {
goto skipPixel;
} else {
dstCount++;
}
}
if (haveDotImg) {
float zeroThreshold2 = inputData->zeroThreshold * inputData->zeroThreshold;
float outlierThreshold2 = inputData->outlierThreshold * inputData->outlierThreshold;
dotRGB = read_imagef(dotImg, sampler, xy);
float norm2 = dot(dotRGB.xyz, dotRGB.xyz);
if (norm2 < zeroThreshold2) {
dstCountZero++;
goto skipPixel;
} else if (outlierThreshold2 > 0 && norm2 > outlierThreshold2) {
dstCountOutlier++;
goto skipPixel;
}
}
float u = H[lz][0]*x + H[lz][1]*y + H[lz][2];
float v = H[lz][3]*x + H[lz][4]*y + H[lz][5];
float w = H[lz][6]*x + H[lz][7]*y + H[lz][8];
float inv_w = native_recip(w);
float2 uv = inv_w*(float2)(u, v);// + 0.5f;
// float4 srcRGB = read_imagef(srcImg, linearSampler, uv);
float4 srcRGB = readLinear(srcImg, uv);
if (_X) {
// srcRGB.w = 1;
// dstRGB = (float4)(dot(X[lz][0], srcRGB), dot(X[lz][1], srcRGB),
// dot(X[lz][2], srcRGB), dot(X[lz][3], srcRGB));
dstRGB.x = X[lz][0]*srcRGB.x + X[lz][1]*srcRGB.y + X[lz][2] *srcRGB.z + X[lz][3];
dstRGB.y = X[lz][4]*srcRGB.x + X[lz][5]*srcRGB.y + X[lz][6] *srcRGB.z + X[lz][7];
dstRGB.z = X[lz][8]*srcRGB.x + X[lz][9]*srcRGB.y + X[lz][10]*srcRGB.z + X[lz][11];
} else {
dstRGB = srcRGB;
}
if (_H2) {
float u2 = H2[lz][0]*x + H2[lz][1]*y + H2[lz][2];
float v2 = H2[lz][3]*x + H2[lz][4]*y + H2[lz][5];
float w2 = H2[lz][6]*x + H2[lz][7]*y + H2[lz][8];
float inv_w2 = native_recip(w2);
float2 uv2 = inv_w2*(float2)(u2, v2);// + 0.5f;
// dstRGB *= read_imagef(srcImg2, linearSampler, uv2);
dstRGB *= readLinear(srcImg2, uv2);
}
dstRGB.w = 1;
if (haveTransImg) {
write_imagef(transImg, xy, dstRGB);
}
if (haveSubImg) {
dstRGB.xyz -= read_imagef(subImg, sampler, xy).xyz;
}
if (haveDstImg) {
write_imagef(dstImg, xy, dstRGB);
}
if (haveDotImg) {
srcDstDot += dot(dotRGB.xyz, dstRGB.xyz);
}
if (size == 1) {
float zeroThreshold2 = inputData->zeroThreshold * inputData->zeroThreshold;
float outlierThreshold2 = inputData->outlierThreshold * inputData->outlierThreshold;
float norm2 = dot(dstRGB.xyz, dstRGB.xyz);
if (norm2 < zeroThreshold2) {
dstCountZero++;
} else if (outlierThreshold2 > 0 && norm2 > outlierThreshold2) {
dstCountOutlier++;
} else {
dstDstDot += norm2;
}
}
skipPixel:
if (size > 1) {
scratch[lz][lx][0] = dstRGB.x;
scratch[lz][lx][1] = dstRGB.y;
scratch[lz][lx][2] = dstRGB.z;
barrier(CLK_LOCAL_MEM_FENCE);
#pragma unroll
for (int i = 0; i < size; i++) {
dstDstDot += scratch[lz][i][0]*scratch[lx][i][0] +
scratch[lz][i][1]*scratch[lx][i][1] +
scratch[lz][i][2]*scratch[lx][i][2];
}
barrier(CLK_LOCAL_MEM_FENCE);
}
}
if (lz == 0) {
dstCount = reduceSumInt(dstCount, lx, lsx, scratch);
dstCountZero = reduceSumInt(dstCountZero, lx, lsx, scratch);
dstCountOutlier = reduceSumInt(dstCountOutlier, lx, lsx, scratch);
if (lx == 0) {
outputData[gx].dstCount = dstCount;
outputData[gx].dstCountZero = dstCountZero;
outputData[gx].dstCountOutlier = dstCountOutlier;
}
}
if (size == 1) {
if (haveDotImg) srcDstDot = reduceSumFloat(srcDstDot, lx, lsx, scratch);
dstDstDot = reduceSumFloat(dstDstDot, lx, lsx, scratch);
if (lx == 0) {
if (haveDotImg) outputData[gx].srcDstDot[0] = srcDstDot;
outputData[gx].dstDstDot[0][0] = dstDstDot;
}
} else {
if (haveDotImg) {
srcDstDot = reduceSumFloat(srcDstDot, lx, lsx, scratch[lz]);
if (lx == 0) {
outputData[gx].srcDstDot[lz] = srcDstDot;
}
}
outputData[gx].dstDstDot[lz][lx] = dstDstDot;
}
}
kernel void reduceOutputData(global struct OutputData *outputData) {
const int x = get_global_id(0), gx = get_group_id(0), lx = get_local_id(0), lsx = get_local_size(0);
local int scratch[256];
int dstCount = reduceSumInt(outputData[x].dstCount, lx, lsx, scratch);
int dstCountZero = reduceSumInt(outputData[x].dstCountZero, lx, lsx, scratch);
int dstCountOutlier = reduceSumInt(outputData[x].dstCountOutlier, lx, lsx, scratch);
if (lx == 0) {
outputData[0].dstCount = dstCount;
outputData[0].dstCountZero = dstCountZero;
outputData[0].dstCountOutlier = dstCountOutlier;
}
for (int i = 0; i < DOT_SIZE; i++) {
float srcDstDot = reduceSumFloat(outputData[x].srcDstDot[i], lx, lsx, scratch);
float dstDstDot = reduceSumFloat(outputData[x].dstDstDot[i][i], lx, lsx, scratch);
if (lx == 0) {
outputData[0].srcDstDot[i] = srcDstDot;
outputData[0].dstDstDot[i][i] = dstDstDot;
}
for (int j = i+1; j < DOT_SIZE; j++) {
float dstDstDot = reduceSumFloat(outputData[x].dstDstDot[i][j], lx, lsx, scratch);
if (lx == 0) {
outputData[0].dstDstDot[i][j] = dstDstDot;
outputData[0].dstDstDot[j][i] = dstDstDot;
}
}
}
}
================================================
FILE: src/main/resources/org/bytedeco/javacv/JavaCV.cl
================================================
/*
* Copyright (C) 2011-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
const sampler_t sampler = CLK_NORMALIZED_COORDS_FALSE |
CLK_ADDRESS_CLAMP_TO_EDGE | CLK_FILTER_NEAREST;
//const sampler_t linearSampler = CLK_NORMALIZED_COORDS_FALSE |
// CLK_ADDRESS_CLAMP_TO_EDGE | CLK_FILTER_LINEAR;
#define SENSOR_PATTERN_RGGB (int2)(0,0)
#define SENSOR_PATTERN_GBRG (int2)(1,0)
#define SENSOR_PATTERN_GRBG (int2)(0,1)
#define SENSOR_PATTERN_BGGR (int2)(1,1)
inline float4 readBayer(read_only image2d_t img, int2 xy, int2 sensorPattern) {
float p1 = read_imagef(img, sampler, xy).s0;
float p2 = read_imagef(img, sampler, xy + (int2)(1, 0)).s0;
float p3 = read_imagef(img, sampler, xy + (int2)(0, 1)).s0;
float p4 = read_imagef(img, sampler, xy + (int2)(1, 1)).s0;
xy = (xy + sensorPattern) % 2;
if (all(xy == SENSOR_PATTERN_RGGB)) {
return (float4)(p1, (p2 + p3)/2, p4, 1);
} else if (all(xy == SENSOR_PATTERN_GBRG)) {
return (float4)(p2, (p1 + p4)/2, p3, 1);
} else if (all(xy == SENSOR_PATTERN_GRBG)) {
return (float4)(p3, (p1 + p4)/2, p2, 1);
} else {//(all(xy == SENSOR_PATTERN_BGGR))
return (float4)(p4, (p2 + p3)/2, p1, 1);
}
}
inline float4 readLinear(read_only image2d_t img, float2 xy) {
int2 xy00 = convert_int2_rtn(xy);
float4 img00 = read_imagef(img, sampler, xy00);
float4 img10 = read_imagef(img, sampler, xy00 + (int2)(1, 0));
float4 img01 = read_imagef(img, sampler, xy00 + (int2)(0, 1));
float4 img11 = read_imagef(img, sampler, xy00 + (int2)(1, 1));
float4 img0 = mix(img00, img10, xy.x - xy00.x);
float4 img1 = mix(img01, img11, xy.x - xy00.x);
return mix(img0, img1, xy.y - xy00.y);
}
inline float4 readBayerLinear(read_only image2d_t img, float2 xy, int2 sensorPattern) {
int2 xy00 = convert_int2_rtn(xy);
float4 img00 = readBayer(img, xy00, sensorPattern);
float4 img10 = readBayer(img, xy00 + (int2)(1, 0), sensorPattern);
float4 img01 = readBayer(img, xy00 + (int2)(0, 1), sensorPattern);
float4 img11 = readBayer(img, xy00 + (int2)(1, 1), sensorPattern);
float4 img0 = mix(img00, img10, xy.x - xy00.x);
float4 img1 = mix(img01, img11, xy.x - xy00.x);
return mix(img0, img1, xy.y - xy00.y);
}
kernel void pyrDown(read_only image2d_t srcImg, write_only image2d_t dstImg) {
int x = get_global_id(0), y = get_global_id(1);
int lx = get_local_id(0), ly = get_local_id(1);
int x2 = x*2, y2 = y*2;
int lx2 = lx*2, ly2 = ly*2;
if (x >= get_image_width(dstImg) || y >= get_image_height(dstImg)) {
return;
}
#define S(x,y) read_imagef(srcImg, sampler, (int2)(x, y))
float4 rgb = (1.0/256.0)*(S(x2-2, y2-2) + S(x2+2, y2-2) + S(x2-2, y2+2) + S(x2+2, y2+2)) +
(4.0/256.0)*(S(x2-1, y2-2) + S(x2+1, y2-2) + S(x2-2, y2-1) + S(x2+2, y2-1) +
S(x2-2, y2+1) + S(x2+2, y2+1) + S(x2-1, y2+2) + S(x2+1, y2+2)) +
(6.0/256.0)*(S(x2 , y2-2) + S(x2-2, y2 ) + S(x2+2, y2 ) + S(x2 , y2+2)) +
(16.0/256.0)*(S(x2-1, y2-1) + S(x2+1, y2-1) + S(x2-1, y2+1) + S(x2+1, y2+1)) +
(24.0/256.0)*(S(x2 , y2-1) + S(x2-1, y2 ) + S(x2+1, y2 ) + S(x2 , y2+1)) +
(36.0/256.0)* S(x2 , y2 );
#undef S
rgb.w = 1;
write_imagef(dstImg, (int2)(x, y), rgb);
}
kernel void remap(read_only image2d_t srcImg, write_only image2d_t dstImg,
read_only image2d_t mapxImg, read_only image2d_t mapyImg) {
int x = get_global_id(0), y = get_global_id(1);
int2 xy = (int2)(x, y);
if (x >= get_image_width(dstImg) || y >= get_image_height(dstImg)) {
return;
}
float x2 = read_imagef(mapxImg, sampler, xy).s0;
float y2 = read_imagef(mapyImg, sampler, xy).s0;
float2 xy2 = (float2)(x2, y2);
float4 rgb = readLinear(srcImg, xy2);
write_imagef(dstImg, xy, rgb);
}
kernel void remapBayer(read_only image2d_t srcImg, write_only image2d_t dstImg,
read_only image2d_t mapxImg, read_only image2d_t mapyImg, int2 sensorPattern) {
int x = get_global_id(0), y = get_global_id(1);
int2 xy = (int2)(x, y);
if (x >= get_image_width(dstImg) || y >= get_image_height(dstImg)) {
return;
}
float x2 = read_imagef(mapxImg, sampler, xy).s0;
float y2 = read_imagef(mapyImg, sampler, xy).s0;
float2 xy2 = (float2)(x2, y2);
float4 rgb = readBayerLinear(srcImg, xy2, sensorPattern);
write_imagef(dstImg, xy, rgb);
}
================================================
FILE: src/main/resources/org/bytedeco/javacv/ProCamTransformer.cl
================================================
/*
* Copyright (C) 2011-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/**
* transform kernel (one transform, no mask)
*/
kernel void transformOne(read_only image2d_t srcImg, read_only image2d_t srcImg2, write_only image2d_t dstImg,
constant float H1[9], constant float H2[9], constant float4 X[4],
constant struct InputData *inputData, global struct OutputData *outputData) {
int w = get_image_width(dstImg);
int h = get_image_height(dstImg);
multiWarpColorTransform(srcImg, srcImg2, 0, 0, 0, dstImg, 0, w, h,
H1, H2, X, 1, inputData, outputData, false, false, false, true, false);
}
/**
* transform kernel (one transform with an image to subtract and a mask)
*/
kernel void transformSub(read_only image2d_t srcImg, read_only image2d_t srcImg2, read_only image2d_t subImg,
write_only image2d_t transImg, write_only image2d_t dstImg,
read_only image2d_t maskImg, constant float H1[9], constant float H2[9], constant float4 X[4],
constant struct InputData *inputData, global struct OutputData *outputData) {
int w = get_image_width(maskImg);
int h = get_image_height(maskImg);
multiWarpColorTransform(srcImg, srcImg2, subImg, 0, transImg, dstImg, maskImg, w, h,
H1, H2, X, 1, inputData, outputData, true, false, true, true, true);
}
/**
* transform kernel (dot products only)
*/
kernel void transformDot(read_only image2d_t srcImg, read_only image2d_t srcImg2, read_only image2d_t subImg,
read_only image2d_t dotImg, read_only image2d_t maskImg,
constant float H1[][9], constant float H2[][9], constant float4 X[][4],
constant struct InputData *inputData, global struct OutputData *outputData) {
int w = get_image_width(maskImg);
int h = get_image_height(maskImg);
multiWarpColorTransform(srcImg, srcImg2, subImg, dotImg, 0, 0, maskImg, w, h,
H1, H2, X, DOT_SIZE, inputData, outputData, true, true, false, false, true);
}
================================================
FILE: src/main/resources/org/bytedeco/javacv/ProjectiveColorTransformer.cl
================================================
/*
* Copyright (C) 2011-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/**
* transform kernel (one transform, no mask)
*/
kernel void transformOne(read_only image2d_t srcImg, write_only image2d_t dstImg,
constant float H[9], constant float4 X[4],
constant struct InputData *inputData, global struct OutputData *outputData) {
int w = get_image_width(dstImg);
int h = get_image_height(dstImg);
multiWarpColorTransform(srcImg, 0, 0, 0, 0, dstImg, 0, w, h,
H, 0, X, 1, inputData, outputData, false, false, false, true, false);
}
/**
* transform kernel (one transform with an image to subtract and a mask)
*/
kernel void transformSub(read_only image2d_t srcImg, read_only image2d_t subImg,
write_only image2d_t transImg, write_only image2d_t dstImg,
read_only image2d_t maskImg, constant float H[9], constant float4 X[4],
constant struct InputData *inputData, global struct OutputData *outputData) {
int w = get_image_width(maskImg);
int h = get_image_height(maskImg);
multiWarpColorTransform(srcImg, 0, subImg, 0, transImg, dstImg, maskImg, w, h,
H, 0, X, 1, inputData, outputData, true, false, true, true, true);
}
/**
* transform kernel (dot products only)
*/
kernel void transformDot(read_only image2d_t srcImg, read_only image2d_t subImg,
read_only image2d_t dotImg, read_only image2d_t maskImg,
constant float H[][9], constant float4 X[][4],
constant struct InputData *inputData, global struct OutputData *outputData) {
int w = get_image_width(maskImg);
int h = get_image_height(maskImg);
multiWarpColorTransform(srcImg, 0, subImg, dotImg, 0, 0, maskImg, w, h,
H, 0, X, DOT_SIZE, inputData, outputData, true, true, false, false, true);
}
================================================
FILE: src/main/resources/org/bytedeco/javacv/ProjectiveTransformer.cl
================================================
/*
* Copyright (C) 2011-2012 Samuel Audet
*
* Licensed either under the Apache License, Version 2.0, or (at your option)
* under the terms of the GNU General Public License as published by
* the Free Software Foundation (subject to the "Classpath" exception),
* either version 2, or any later version (collectively, the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.gnu.org/licenses/
* http://www.gnu.org/software/classpath/license.html
*
* or as provided in the LICENSE.txt file that accompanied this code.
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/**
* transform kernel (one transform, no mask)
*/
kernel void transformOne(read_only image2d_t srcImg, write_only image2d_t dstImg,
constant float H[9],
constant struct InputData *inputData, global struct OutputData *outputData) {
int w = get_image_width(dstImg);
int h = get_image_height(dstImg);
multiWarpColorTransform(srcImg, 0, 0, 0, 0, dstImg, 0, w, h,
H, 0, 0, 1, inputData, outputData, false, false, false, true, false);
}
/**
* transform kernel (one transform with an image to subtract and a mask)
*/
kernel void transformSub(read_only image2d_t srcImg, read_only image2d_t subImg,
write_only image2d_t transImg, write_only image2d_t dstImg,
read_only image2d_t maskImg, constant float H[9],
constant struct InputData *inputData, global struct OutputData *outputData) {
int w = get_image_width(maskImg);
int h = get_image_height(maskImg);
multiWarpColorTransform(srcImg, 0, subImg, 0, transImg, dstImg, maskImg, w, h,
H, 0, 0, 1, inputData, outputData, true, false, true, true, true);
}
/**
* transform kernel (dot products only)
*/
kernel void transformDot(read_only image2d_t srcImg, read_only image2d_t subImg,
read_only image2d_t dotImg, read_only image2d_t maskImg,
constant float H[][9],
constant struct InputData *inputData, global struct OutputData *outputData) {
int w = get_image_width(maskImg);
int h = get_image_height(maskImg);
multiWarpColorTransform(srcImg, 0, subImg, dotImg, 0, 0, maskImg, w, h,
H, 0, 0, DOT_SIZE, inputData, outputData, true, true, false, false, true);
}