Repository: yemount/pose-animator
Branch: master
Commit: cb2be70a3501
Files: 22
Total size: 157.3 KB
Directory structure:
gitextract_rc75g8yk/
├── .gitignore
├── CONTRIBUTING.md
├── LICENSE
├── README.md
├── camera.html
├── camera.js
├── illustrationGen/
│ ├── illustration.js
│ └── skeleton.js
├── index.html
├── package.json
├── static_image.html
├── static_image.js
├── third_party/
│ └── paper/
│ ├── AUTHORS.md
│ ├── CHANGELOG.md
│ ├── LICENSE.txt
│ ├── README.md
│ └── package.json
└── utils/
├── colorUtils.js
├── demoUtils.js
├── fileUtils.js
├── mathUtils.js
└── svgUtils.js
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
.DS_Store
.cache
# Firebase deployment config files
.firebaserc
firebase.json
# Yarn build output folder
dist/
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
firebase-debug.log*
# Firebase cache
.firebase/
# Firebase config
# Uncomment this if you'd like others to create their own Firebase project.
# For a team working on the same Firebase project(s), it is recommended to leave
# it commented so all members can deploy to the same project(s) in .firebaserc.
# .firebaserc
# Runtime data
pids
*.pid
*.seed
*.pid.lock
# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov
# Coverage directory used by tools like istanbul
coverage
# nyc test coverage
.nyc_output
# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
.grunt
# Bower dependency directory (https://bower.io/)
bower_components
# node-waf configuration
.lock-wscript
# Compiled binary addons (http://nodejs.org/api/addons.html)
build/Release
# Dependency directories
node_modules/
# Optional npm cache directory
.npm
# Optional eslint cache
.eslintcache
# Optional REPL history
.node_repl_history
# Output of 'npm pack'
*.tgz
# Yarn Integrity file
.yarn-integrity
# dotenv environment variables file
.env
================================================
FILE: CONTRIBUTING.md
================================================
# How to Contribute
We'd love to accept your patches and contributions to this project. There are
just a few small guidelines you need to follow.
## Contributor License Agreement
Contributions to this project must be accompanied by a Contributor License
Agreement (CLA). You (or your employer) retain the copyright to your
contribution; this simply gives us permission to use and redistribute your
contributions as part of the project. Head over to
<https://cla.developers.google.com/> to see your current agreements on file or
to sign a new one.
You generally only need to submit a CLA once, so if you've already submitted one
(even if it was for a different project), you probably don't need to do it
again.
## Code reviews
All submissions, including submissions by project members, require review. We
use GitHub pull requests for this purpose. Consult
[GitHub Help](https://help.github.com/articles/about-pull-requests/) for more
information on using pull requests.
## Community Guidelines
This project follows
[Google's Open Source Community Guidelines](https://opensource.google/conduct/).
================================================
FILE: LICENSE
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: README.md
================================================
# Pose Animator
Pose Animator takes a 2D vector illustration and animates its containing curves in real-time based on the recognition result from PoseNet and FaceMesh. It borrows the idea of skeleton-based animation from computer graphics and applies it to vector characters.
This is running in the browser in realtime using [TensorFlow.js](https://www.tensorflow.org/js). Check out more cool TF.js demos [here](https://www.tensorflow.org/js/demos).
*This is not an officially supported Google product.*
<img src="/resources/gifs/avatar-new-1.gif?raw=true" alt="cameraDemo" style="width: 250px;"/>
<img src="/resources/gifs/avatar-new-full-body.gif?raw=true" alt="cameraDemo" style="width: 250px;"/>
In skeletal animation a character is represented in two parts:
1. a surface used to draw the character, and
1. a hierarchical set of interconnected bones used to animate the surface.
In Pose Animator, the surface is defined by the 2D vector paths in the input SVG files. For the bone structure, Pose Animator provides a predefined rig (bone hierarchy) representation, designed based on the keypoints from PoseNet and FaceMesh. This bone structure’s initial pose is specified in the input SVG file, along with the character illustration, while the real time bone positions are updated by the recognition result from ML models.
<img src="https://firebasestorage.googleapis.com/v0/b/pose-animator-demo.appspot.com/o/ml-keypoints.png?alt=media" style="width:250px;"/>
<img src="/resources/gifs/avatar-new-bezier-1.gif?raw=true" alt="cameraDemo" style="width: 250px;"/>
// TODO: Add blog post link.
For more details on its technical design please check out this blog post.
### Demo 1: [Camera feed](https://pose-animator-demo.firebaseapp.com/camera.html)
The camera demo animates a 2D avatar in real-time from a webcam video stream.
### Demo 2: [Static image](https://pose-animator-demo.firebaseapp.com/static_image.html)
The static image demo shows the avatar positioned from a single image.
## Build And Run
Install dependencies and prepare the build directory:
```sh
yarn
```
To watch files for changes, and launch a dev server:
```sh
yarn watch
```
## Platform support
Demos are supported on Desktop Chrome and iOS Safari.
It should also run on Chrome on Android and potentially more Android mobile browsers though support has not been tested yet.
# Animate your own design
1. Download the [sample skeleton SVG here](/resources/samples/skeleton.svg).
1. Create a new file in your vector graphics editor of choice. Copy the group named ‘skeleton’ from the above file into your working file. Note:
* Do not add, remove or rename the joints (circles) in this group. Pose Animator relies on these named paths to read the skeleton’s initial position. Missing joints will cause errors.
* However you can move the joints around to embed them into your illustration. See step 4.
1. Create a new group and name it ‘illustration’, next to the ‘skeleton’ group. This is the group where you can put all the paths for your illustration.
* Flatten all subgroups so that ‘illustration’ only contains path elements.
* Composite paths are not supported at the moment.
* The working file structure should look like this:
```
[Layer 1]
|---- skeleton
|---- illustration
|---- path 1
|---- path 2
|---- path 3
```
1. Embed the sample skeleton in ‘skeleton’ group into your illustration by moving the joints around.
1. Export the file as an SVG file.
1. Open [Pose Animator camera demo](https://pose-animator-demo.firebaseapp.com/camera.html). Once everything loads, drop your SVG file into the browser tab. You should be able to see it come to life :D
================================================
FILE: camera.html
================================================
<!DOCTYPE html>
<html>
<head>
<title>PoseNet - Camera Feed Demo</title>
<style>
body {
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
}
.canvas-container {
width: 800px;
max-width: 100%;
display: flex;
justify-content: center;
position: relative;
}
.camera-canvas {
position: absolute;
transform: scale(0.5, 0.5);
transform-origin: 0 0;
left: 10px;
top: 10px;
}
#main {
left: 0;
top: 0;
position: absolute;
}
.illustration-canvas {
border: 1px solid #eeeeee;
}
.footer {
position: fixed;
left: 0;
bottom: 0;
width: 100%;
color: black;
}
.footer-text {
max-width: 600px;
text-align: center;
margin: auto;
}
@media only screen and (max-width: 600px) {
.footer-text,
.dg {
display: none;
}
}
/*
* The following loading spinner CSS is from SpinKit project
* https://github.com/tobiasahlin/SpinKit
*/
.sk-spinner-pulse {
width: 20px;
height: 20px;
margin: auto 10px;
float: left;
background-color: #333;
border-radius: 100%;
-webkit-animation: sk-pulseScaleOut 1s infinite ease-in-out;
animation: sk-pulseScaleOut 1s infinite ease-in-out;
}
@-webkit-keyframes sk-pulseScaleOut {
0% {
-webkit-transform: scale(0);
transform: scale(0);
}
100% {
-webkit-transform: scale(1.0);
transform: scale(1.0);
opacity: 0;
}
}
@keyframes sk-pulseScaleOut {
0% {
-webkit-transform: scale(0);
transform: scale(0);
}
100% {
-webkit-transform: scale(1.0);
transform: scale(1.0);
opacity: 0;
}
}
.spinner-text {
float: left;
}
</style>
<meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
<div id="info" style='display:none'>
</div>
<div id="loading" style='position: relative; left: 0'>
<span class="spinner-text" id="status">
Loading PoseNet model...
</span>
<div class="sk-spinner sk-spinner-pulse"></div>
</div>
<div class="canvas-container">
<div id='main' style='display:none'>
<video id="video" playsinline style=" -moz-transform: scaleX(-1);
-o-transform: scaleX(-1);
-webkit-transform: scaleX(-1);
transform: scaleX(-1);
display: none;
">
</video>
<canvas id="output" class="camera-canvas"></canvas>
<canvas id="keypoints" class="camera-canvas"></canvas>
</div>
<canvas class="illustration-canvas"></cavnas>
</div>
<div class="footer">
<div class="footer-text">
<p>
Pose Animator runs TF.js <strong>FaceMesh</strong> and <strong>PoseNet</strong> models to animate SVG illustrations with camera feed / static images.<br>
It currently supports <strong>single-pose</strong>, <strong>single-face</strong> detection, and has been tested on Destkop Chrome & iOS Safari.
<br>
(PoseNet model config - MobileNetV1, output stride 16, quant bytes 2)
</p>
</div>
</div>
<script src="camera.js"></script>
</body>
</html>
================================================
FILE: camera.js
================================================
/**
* @license
* Copyright 2020 Google Inc. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* =============================================================================
*/
import * as posenet_module from '@tensorflow-models/posenet';
import * as facemesh_module from '@tensorflow-models/facemesh';
import * as tf from '@tensorflow/tfjs';
import * as paper from 'paper';
import dat from 'dat.gui';
import Stats from 'stats.js';
import "babel-polyfill";
import {drawKeypoints, drawPoint, drawSkeleton, isMobile, toggleLoadingUI, setStatusText} from './utils/demoUtils';
import {SVGUtils} from './utils/svgUtils'
import {PoseIllustration} from './illustrationGen/illustration';
import {Skeleton, facePartName2Index} from './illustrationGen/skeleton';
import {FileUtils} from './utils/fileUtils';
import * as girlSVG from './resources/illustration/girl.svg';
import * as boySVG from './resources/illustration/boy.svg';
import * as abstractSVG from './resources/illustration/abstract.svg';
import * as blathersSVG from './resources/illustration/blathers.svg';
import * as tomNookSVG from './resources/illustration/tom-nook.svg';
// Camera stream video element
let video;
let videoWidth = 300;
let videoHeight = 300;
// Canvas
let faceDetection = null;
let illustration = null;
let canvasScope;
let canvasWidth = 800;
let canvasHeight = 800;
// ML models
let facemesh;
let posenet;
let minPoseConfidence = 0.15;
let minPartConfidence = 0.1;
let nmsRadius = 30.0;
// Misc
let mobile = false;
const stats = new Stats();
const avatarSvgs = {
'girl': girlSVG.default,
'boy': boySVG.default,
'abstract': abstractSVG.default,
'blathers': blathersSVG.default,
'tom-nook': tomNookSVG.default,
};
/**
* Loads a the camera to be used in the demo
*
*/
async function setupCamera() {
if (!navigator.mediaDevices || !navigator.mediaDevices.getUserMedia) {
throw new Error(
'Browser API navigator.mediaDevices.getUserMedia not available');
}
const video = document.getElementById('video');
video.width = videoWidth;
video.height = videoHeight;
const stream = await navigator.mediaDevices.getUserMedia({
'audio': false,
'video': {
facingMode: 'user',
width: videoWidth,
height: videoHeight,
},
});
video.srcObject = stream;
return new Promise((resolve) => {
video.onloadedmetadata = () => {
resolve(video);
};
});
}
async function loadVideo() {
const video = await setupCamera();
video.play();
return video;
}
const defaultPoseNetArchitecture = 'MobileNetV1';
const defaultQuantBytes = 2;
const defaultMultiplier = 1.0;
const defaultStride = 16;
const defaultInputResolution = 200;
const guiState = {
avatarSVG: Object.keys(avatarSvgs)[0],
debug: {
showDetectionDebug: true,
showIllustrationDebug: false,
},
};
/**
* Sets up dat.gui controller on the top-right of the window
*/
function setupGui(cameras) {
if (cameras.length > 0) {
guiState.camera = cameras[0].deviceId;
}
const gui = new dat.GUI({width: 300});
let multi = gui.addFolder('Image');
gui.add(guiState, 'avatarSVG', Object.keys(avatarSvgs)).onChange(() => parseSVG(avatarSvgs[guiState.avatarSVG]));
multi.open();
let output = gui.addFolder('Debug control');
output.add(guiState.debug, 'showDetectionDebug');
output.add(guiState.debug, 'showIllustrationDebug');
output.open();
}
/**
* Sets up a frames per second panel on the top-left of the window
*/
function setupFPS() {
stats.showPanel(0); // 0: fps, 1: ms, 2: mb, 3+: custom
document.getElementById('main').appendChild(stats.dom);
}
/**
* Feeds an image to posenet to estimate poses - this is where the magic
* happens. This function loops with a requestAnimationFrame method.
*/
function detectPoseInRealTime(video) {
const canvas = document.getElementById('output');
const keypointCanvas = document.getElementById('keypoints');
const videoCtx = canvas.getContext('2d');
const keypointCtx = keypointCanvas.getContext('2d');
canvas.width = videoWidth;
canvas.height = videoHeight;
keypointCanvas.width = videoWidth;
keypointCanvas.height = videoHeight;
async function poseDetectionFrame() {
// Begin monitoring code for frames per second
stats.begin();
let poses = [];
videoCtx.clearRect(0, 0, videoWidth, videoHeight);
// Draw video
videoCtx.save();
videoCtx.scale(-1, 1);
videoCtx.translate(-videoWidth, 0);
videoCtx.drawImage(video, 0, 0, videoWidth, videoHeight);
videoCtx.restore();
// Creates a tensor from an image
const input = tf.browser.fromPixels(canvas);
faceDetection = await facemesh.estimateFaces(input, false, false);
let all_poses = await posenet.estimatePoses(video, {
flipHorizontal: true,
decodingMethod: 'multi-person',
maxDetections: 1,
scoreThreshold: minPartConfidence,
nmsRadius: nmsRadius
});
poses = poses.concat(all_poses);
input.dispose();
keypointCtx.clearRect(0, 0, videoWidth, videoHeight);
if (guiState.debug.showDetectionDebug) {
poses.forEach(({score, keypoints}) => {
if (score >= minPoseConfidence) {
drawKeypoints(keypoints, minPartConfidence, keypointCtx);
drawSkeleton(keypoints, minPartConfidence, keypointCtx);
}
});
faceDetection.forEach(face => {
Object.values(facePartName2Index).forEach(index => {
let p = face.scaledMesh[index];
drawPoint(keypointCtx, p[1], p[0], 2, 'red');
});
});
}
canvasScope.project.clear();
if (poses.length >= 1 && illustration) {
Skeleton.flipPose(poses[0]);
if (faceDetection && faceDetection.length > 0) {
let face = Skeleton.toFaceFrame(faceDetection[0]);
illustration.updateSkeleton(poses[0], face);
} else {
illustration.updateSkeleton(poses[0], null);
}
illustration.draw(canvasScope, videoWidth, videoHeight);
if (guiState.debug.showIllustrationDebug) {
illustration.debugDraw(canvasScope);
}
}
canvasScope.project.activeLayer.scale(
canvasWidth / videoWidth,
canvasHeight / videoHeight,
new canvasScope.Point(0, 0));
// End monitoring code for frames per second
stats.end();
requestAnimationFrame(poseDetectionFrame);
}
poseDetectionFrame();
}
function setupCanvas() {
mobile = isMobile();
if (mobile) {
canvasWidth = Math.min(window.innerWidth, window.innerHeight);
canvasHeight = canvasWidth;
videoWidth *= 0.7;
videoHeight *= 0.7;
}
canvasScope = paper.default;
let canvas = document.querySelector('.illustration-canvas');;
canvas.width = canvasWidth;
canvas.height = canvasHeight;
canvasScope.setup(canvas);
}
/**
* Kicks off the demo by loading the posenet model, finding and loading
* available camera devices, and setting off the detectPoseInRealTime function.
*/
export async function bindPage() {
setupCanvas();
toggleLoadingUI(true);
setStatusText('Loading PoseNet model...');
posenet = await posenet_module.load({
architecture: defaultPoseNetArchitecture,
outputStride: defaultStride,
inputResolution: defaultInputResolution,
multiplier: defaultMultiplier,
quantBytes: defaultQuantBytes
});
setStatusText('Loading FaceMesh model...');
facemesh = await facemesh_module.load();
setStatusText('Loading Avatar file...');
let t0 = new Date();
await parseSVG(Object.values(avatarSvgs)[0]);
setStatusText('Setting up camera...');
try {
video = await loadVideo();
} catch (e) {
let info = document.getElementById('info');
info.textContent = 'this device type is not supported yet, ' +
'or this browser does not support video capture: ' + e.toString();
info.style.display = 'block';
throw e;
}
setupGui([], posenet);
setupFPS();
toggleLoadingUI(false);
detectPoseInRealTime(video, posenet);
}
navigator.getUserMedia = navigator.getUserMedia ||
navigator.webkitGetUserMedia || navigator.mozGetUserMedia;
FileUtils.setDragDropHandler((result) => {parseSVG(result)});
async function parseSVG(target) {
let svgScope = await SVGUtils.importSVG(target /* SVG string or file path */);
let skeleton = new Skeleton(svgScope);
illustration = new PoseIllustration(canvasScope);
illustration.bindSkeleton(skeleton, svgScope);
}
bindPage();
================================================
FILE: illustrationGen/illustration.js
================================================
/**
* @license
* Copyright 2020 Google Inc. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* =============================================================================
*/
import { Bone, allPartNames, Skeleton } from './skeleton';
import { MathUtils } from '../utils/mathUtils';
import { SVGUtils } from '../utils/svgUtils';
import { ColorUtils } from '../utils/colorUtils';
const allPartNamesMap = {};
allPartNames.forEach(name => allPartNamesMap[name] = 1);
const MIN_CONFIDENCE_PATH_SCORE = 0.3;
// Represents a skinned illustration.
export class PoseIllustration {
constructor(scope) {
this.scope = scope;
this.frames = [];
}
updateSkeleton(pose, face) {
this.pose = pose;
this.face = face;
this.skeleton.update(pose, face);
if (!this.skeleton.isValid) {
return;
}
let getConfidenceScore = (p) => {
return Object.keys(p.skinning).reduce((totalScore, boneName) => {
let bt = p.skinning[boneName];
return totalScore + bt.bone.score * bt.weight;
}, 0);
}
this.skinnedPaths.forEach(skinnedPath => {
let confidenceScore = 0;
skinnedPath.segments.forEach(seg => {
// Compute confidence score.
confidenceScore += getConfidenceScore(seg.point);
// Compute new positions for curve point and handles.
seg.point.currentPosition = Skeleton.getCurrentPosition(seg.point);
if (seg.handleIn) {
seg.handleIn.currentPosition = Skeleton.getCurrentPosition(seg.handleIn);
}
if (seg.handleOut) {
seg.handleOut.currentPosition = Skeleton.getCurrentPosition(seg.handleOut);
}
});
skinnedPath.confidenceScore = confidenceScore / (skinnedPath.segments.length || 1);
});
}
draw() {
if (!this.skeleton.isValid) {
return;
}
let scope = this.scope;
// Add paths
this.skinnedPaths.forEach(skinnedPath => {
// Do not render paths with low confidence scores.
if (!skinnedPath.confidenceScore || skinnedPath.confidenceScore < MIN_CONFIDENCE_PATH_SCORE) {
return;
}
let path = new scope.Path({
fillColor: skinnedPath.fillColor,
strokeColor: skinnedPath.strokeColor,
strokeWidth: skinnedPath.strokeWidth,
closed: skinnedPath.closed,
});
skinnedPath.segments.forEach(seg => {
path.addSegment(seg.point.currentPosition,
seg.handleIn ? seg.handleIn.currentPosition.subtract(seg.point.currentPosition) : null,
seg.handleOut ? seg.handleOut.currentPosition.subtract(seg.point.currentPosition) : null);
});
if (skinnedPath.closed) {
path.closePath();
}
scope.project.activeLayer.addChild(path);
});
}
debugDraw() {
let scope = this.scope;
let group = new scope.Group();
scope.project.activeLayer.addChild(group);
let drawCircle = (p, opt = {}) => {
group.addChild(new scope.Path.Circle({
center: [p.x, p.y],
radius: opt.radius || 2,
fillColor: opt.fillColor || 'red',
}));
}
let drawLine = (p0, p1, opt = {}) => {
group.addChild(new scope.Path({
segments: [p0, p1],
strokeColor: opt.strokeColor || 'red',
strokeWidth: opt.strokeWidth || 1
}));
}
// Draw skeleton.
this.skeleton.debugDraw(scope);
// Draw curve and handles.
this.skinnedPaths.forEach(skinnedPath => {
skinnedPath.segments.forEach(seg => {
// Color represents weight influence from bones.
let color = new scope.Color(0);
Object.keys(seg.point.skinning).forEach((boneName) => {
let bt = seg.point.skinning[boneName];
ColorUtils.addRGB(color,
bt.weight * bt.bone.boneColor.red,
bt.weight * bt.bone.boneColor.green,
bt.weight * bt.bone.boneColor.blue);
let anchor = bt.bone.kp0.currentPosition.multiply(1 - bt.transform.anchorPerc).add(bt.bone.kp1.currentPosition.multiply(bt.transform.anchorPerc));
drawLine(anchor, seg.point.currentPosition, {strokeColor: 'blue', strokeWidth: bt.weight});
});
drawCircle(seg.point.currentPosition, {fillColor: color});
drawCircle(seg.handleIn.currentPosition, {fillColor: color});
drawLine(seg.point.currentPosition, seg.handleIn.currentPosition, {strokeColor: color});
drawCircle(seg.handleOut.currentPosition, {fillColor: color}, {strokeColor: color});
drawLine(seg.point.currentPosition, seg.handleOut.currentPosition);
});
});
}
debugDrawLabel(scope) {
this.skeleton.debugDrawLabels(scope);
}
bindSkeleton(skeleton, skeletonScope) {
let items = skeletonScope.project.getItems({ recursive: true });
items = items.filter(item => item.parent && item.parent.name && item.parent.name.startsWith('illustration'));
this.skeleton = skeleton;
this.skinnedPaths = [];
// Only support rendering path and shapes for now.
for (let i = 0; i < items.length; i++) {
let item = items[i];
if (SVGUtils.isGroup(item)) {
this.bindGroup(item, skeleton);
} else if (SVGUtils.isPath(item)) {
this.bindPathToBones(item);
} else if (SVGUtils.isShape(item)) {
this.bindPathToBones(item.toPath());
}
}
}
bindGroup(group, skeleton) {
let paths = [];
let keypoints = {};
let items = group.getItems({recursive: true});
// Find all paths and included keypoints.
items.forEach(item => {
let partName = item.name ? allPartNames.find(partName => item.name.startsWith(partName)) : null;
if (partName) {
keypoints[partName] = {
position: item.bounds.center,
name: partName,
};
} else if (SVGUtils.isPath(item)) {
paths.push(item);
} else if (SVGUtils.isShape(item)) {
paths.push(item.toPath());
}
});
let secondaryBones = [];
// Find all parent bones of the included keypoints.
let parentBones = skeleton.bones.filter(bone => keypoints[bone.kp0.name] && keypoints[bone.kp1.name]);
let nosePos = skeleton.bNose3Nose4.kp1.position;
if (!parentBones.length) {
return;
}
// Crete secondary bones for the included keypoints.
parentBones.forEach(parentBone => {
let kp0 = keypoints[parentBone.kp0.name];
let kp1 = keypoints[parentBone.kp1.name];
let secondaryBone = new Bone().set(kp0, kp1, parentBone.skeleton, parentBone.type);
kp0.transformFunc = MathUtils.getTransformFunc(parentBone.kp0.position, nosePos, kp0.position);
kp1.transformFunc = MathUtils.getTransformFunc(parentBone.kp1.position, nosePos, kp1.position);
secondaryBone.parent = parentBone;
secondaryBones.push(secondaryBone);
});
skeleton.secondaryBones = skeleton.secondaryBones.concat(secondaryBones);
paths.forEach(path => {
this.bindPathToBones(path, secondaryBones);
});
}
// Assign weights from bones for point.
// Weight calculation is roughly based on linear blend skinning model.
getWeights(point, bones) {
let totalW = 0;
let weights = {};
bones.forEach(bone => {
let d = MathUtils.getClosestPointOnSegment(bone.kp0.position, bone.kp1.position, point)
.getDistance(point);
// Absolute weight = 1 / (distance * distance)
let w = 1 / (d * d);
weights[bone.name] = {
value: w,
bone: bone,
}
});
let values = Object.values(weights).sort((v0, v1) => {
return v1.value - v0.value;
});
weights = {};
totalW = 0;
values.forEach(v => {
weights[v.bone.name] = v;
totalW += v.value;
});
if (totalW === 0) {
// Point is outside of the influence zone of all bones. It will not be influence by any bone.
return {};
}
// Normalize weights to sum up to 1.
Object.values(weights).forEach(weight => {
weight.value /= totalW;
});
return weights;
}
// Binds a path to bones by compute weight contribution from each bones for each path segment.
// If selectedBones are set, bind directly to the selected bones. Otherwise auto select the bone group closest to each segment.
bindPathToBones(path, selectedBones) {
// Compute bone weights for each segment.
let segs = path.segments.map(s => {
// Check if control points are collinear.
// If so, use the middle point's weight for all three points (curve point, handleIn, handleOut).
// This makes sure smooth curves remain smooth after deformation.
let collinear = MathUtils.isCollinear(s.handleIn, s.handleOut);
let bones = selectedBones || this.skeleton.findBoneGroup(s.point);
let weightsP = this.getWeights(s.point, bones);
let segment = {
point: this.getSkinning(s.point, weightsP),
};
// For handles, compute transformation in world space.
if (s.handleIn) {
let pHandleIn = s.handleIn.add(s.point);
segment.handleIn = this.getSkinning(pHandleIn, collinear ? weightsP : this.getWeights(pHandleIn, bones));
}
if (s.handleOut) {
let pHandleOut = s.handleOut.add(s.point);
segment.handleOut = this.getSkinning(pHandleOut, collinear ? weightsP : this.getWeights(pHandleOut, bones));
}
return segment;
});
this.skinnedPaths.push({
segments: segs,
fillColor: path.fillColor,
strokeColor: path.strokeColor,
strokeWidth: path.strokeWidth,
closed: path.closed
});
}
getSkinning(point, weights) {
let skinning = {};
Object.keys(weights).forEach(boneName => {
skinning[boneName] = {
bone: weights[boneName].bone,
weight: weights[boneName].value,
transform: weights[boneName].bone.getPointTransform(point),
};
});
return {
skinning: skinning,
position: point,
currentPosition: new this.scope.Point(0, 0),
}
};
}
================================================
FILE: illustrationGen/skeleton.js
================================================
/**
* @license
* Copyright 2020 Google Inc. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* =============================================================================
*/
import * as paper from 'paper';
import { SVGUtils } from '../utils/svgUtils';
import { MathUtils } from '../utils/mathUtils';
import { ColorUtils } from '../utils/colorUtils';
const MIN_POSE_SCORE = 0.1;
const MIN_FACE_SCORE = 0.8;
const posePartNames = ['leftAnkle', 'leftKnee', 'leftHip', 'leftWrist', 'leftElbow', 'leftShoulder',
'rightAnkle', 'rightKnee', 'rightHip', 'rightWrist', 'rightElbow', 'rightShoulder',
'leftEar', 'rightEar'];
// Mapping between face part names and their vertex indices in TF face mesh.
export const facePartName2Index = {
'topMid': 10,
'rightTop0': 67,
'rightTop1': 54,
'leftTop0': 297,
'leftTop1': 284,
'rightJaw0': 21,
'rightJaw1': 162,
'rightJaw2': 127,
'rightJaw3': 234,
'rightJaw4': 132,
'rightJaw5': 172,
'rightJaw6': 150,
'rightJaw7': 176,
'jawMid': 152, // 0 - 8
'leftJaw7': 400,
'leftJaw6': 379,
'leftJaw5': 397,
'leftJaw4': 361,
'leftJaw3': 454,
'leftJaw2': 356,
'leftJaw1': 389,
'leftJaw0': 251, // 9 - 16
'rightBrow0': 46,
'rightBrow1': 53,
'rightBrow2': 52,
'rightBrow3': 65,
'rightBrow4': 55, // 17 - 21
'leftBrow4': 285,
'leftBrow3': 295,
'leftBrow2': 282,
'leftBrow1': 283,
'leftBrow0': 276, // 22 - 26
'nose0': 6,
'nose1': 197,
'nose2': 195,
'nose3': 5, // 27 - 30
'rightNose0': 48,
'rightNose1': 220,
'nose4': 4,
'leftNose1': 440,
'leftNose0': 278, // 31 - 35
'rightEye0': 33,
'rightEye1': 160,
'rightEye2': 158,
'rightEye3': 133,
'rightEye4': 153,
'rightEye5': 144, // 36 - 41
'leftEye3': 362,
'leftEye2': 385,
'leftEye1': 387,
'leftEye0': 263,
'leftEye5': 373,
'leftEye4': 380, // 42 - 47
'rightMouthCorner': 61,
'rightUpperLipTop0': 40,
'rightUpperLipTop1': 37,
'upperLipTopMid': 0,
'leftUpperLipTop1': 267,
'leftUpperLipTop0': 270,
'leftMouthCorner': 291, // 48 - 54
'leftLowerLipBottom0': 321,
'leftLowerLipBottom1': 314,
'lowerLipBottomMid': 17,
'rightLowerLipBottom1': 84,
'rightLowerLipBottom0': 91, // 55 - 59
'rightMiddleLip': 78,
'rightUpperLipBottom1': 81,
'upperLipBottomMid': 13,
'leftUpperLipBottom1': 311,
'leftMiddleLip': 308, // 60 - 64
'leftLowerLipTop0': 402,
'lowerLipTopMid': 14,
'rightLowerLipTop0': 178, // 65 - 67
};
const facePartNames = [
'topMid', 'rightTop0', 'rightTop1', 'leftTop0', 'leftTop1',
'rightJaw0', 'rightJaw1', 'rightJaw2', 'rightJaw3', 'rightJaw4', 'rightJaw5', 'rightJaw6', 'rightJaw7', 'jawMid', // 0 - 8
'leftJaw7', 'leftJaw6', 'leftJaw5', 'leftJaw4', 'leftJaw3', 'leftJaw2', 'leftJaw1', 'leftJaw0', // 9 - 16
'rightBrow0', 'rightBrow1', 'rightBrow2', 'rightBrow3', 'rightBrow4', // 17 - 21
'leftBrow4', 'leftBrow3', 'leftBrow2', 'leftBrow1', 'leftBrow0', // 22 - 26
'nose0', 'nose1', 'nose2', 'nose3', // 27 - 30
'rightNose0', 'rightNose1', 'nose4', 'leftNose1', 'leftNose0', // 31 - 35
'rightEye0', 'rightEye1', 'rightEye2', 'rightEye3', 'rightEye4', 'rightEye5', // 36 - 41
'leftEye3', 'leftEye2', 'leftEye1', 'leftEye0', 'leftEye5', 'leftEye4', // 42 - 47
'rightMouthCorner', 'rightUpperLipTop0', 'rightUpperLipTop1', 'upperLipTopMid', 'leftUpperLipTop1', 'leftUpperLipTop0', 'leftMouthCorner', // 48 - 54
'leftLowerLipBottom0', 'leftLowerLipBottom1', 'lowerLipBottomMid', 'rightLowerLipBottom1', 'rightLowerLipBottom0', // 55 - 59
'rightMiddleLip', 'rightUpperLipBottom1', 'upperLipBottomMid', 'leftUpperLipBottom1', 'leftMiddleLip', // 60 - 64
'leftLowerLipTop0', 'lowerLipTopMid', 'rightLowerLipTop0', // 65 - 67
];
export const allPartNames = posePartNames.concat(facePartNames);
// Represents a bone formed by two part keypoints.
export class Bone {
set(kp0, kp1, skeleton, type) {
this.name = `${kp0.name}-${kp1.name}`;
this.kp0 = kp0;
this.kp1 = kp1;
this.skeleton = skeleton;
this.type = type;
this.boneColor = ColorUtils.fromStringHash(this.name);
this.boneColor.saturation += 0.5;
return this;
};
// Finds a point's bone transform.
// Let anchor be the closest point on the bone to the point.
// A point's bone transformation is the transformation from anchor to the point.
getPointTransform(p) {
let dir = this.kp1.position.subtract(this.kp0.position).normalize();
let n = dir.clone();
n.angle += 90;
let closestP = MathUtils.getClosestPointOnSegment(this.kp0.position, this.kp1.position, p);
let v = p.subtract(closestP);
let dirProjD = v.dot(dir);
let dirProjN = v.dot(n);
let d = this.kp0.position.subtract(this.kp1.position).length;
let anchorPerc = closestP.subtract(this.kp0.position).length / d;
return {
transform: new paper.default.Point(dirProjD, dirProjN),
anchorPerc: anchorPerc,
};
}
// Finds a point's current position from the current bone position.
transform(trans) {
if (!this.kp1.currentPosition || !this.kp0.currentPosition) {
return;
}
// Scale distance from anchor point base on bone type.
// All face bones will share one distance scale. All body bones share another.
let scale = this.type === 'face' ? this.skeleton.currentFaceScale : this.skeleton.currentBodyScale;
let dir = this.kp1.currentPosition.subtract(this.kp0.currentPosition).normalize();
let n = dir.clone();
n.angle += 90;
let anchor = this.kp0.currentPosition.multiply(1 - trans.anchorPerc).add(this.kp1.currentPosition.multiply(trans.anchorPerc));
let p = anchor.add(dir.multiply(trans.transform.x * scale)).add(n.multiply(trans.transform.y * scale));
return p;
}
}
function getKeyPointFromSVG(group, partName) {
let shape = SVGUtils.findFirstItemWithPrefix(group, partName);
return {
position: shape.bounds.center,
name: partName,
};
}
function getPartFromPose(pose, name) {
if (!pose || !pose.keypoints) {
return null;
}
let part = pose.keypoints.find(kp => kp.part === name);
return {
position: new paper.default.Point(part.position.x, part.position.y),
score: part.score,
}
}
function getKeypointFromFaceFrame(face, i) {
if (!face || !face.scaledMesh || !face.scaledMesh.length);
return new paper.default.Point(face.positions[i * 2], face.positions[i * 2 + 1]);
}
// Represents a full body skeleton.
export class Skeleton {
constructor(scope) {
let skeletonGroup = SVGUtils.findFirstItemWithPrefix(scope.project, 'skeleton');
// Pose
let leftAnkle = getKeyPointFromSVG(skeletonGroup, 'leftAnkle');
let leftKnee = getKeyPointFromSVG(skeletonGroup, 'leftKnee');
let leftHip = getKeyPointFromSVG(skeletonGroup, 'leftHip');
let leftWrist = getKeyPointFromSVG(skeletonGroup, 'leftWrist');
let leftElbow = getKeyPointFromSVG(skeletonGroup, 'leftElbow');
let leftShoulder = getKeyPointFromSVG(skeletonGroup, 'leftShoulder');
let rightAnkle = getKeyPointFromSVG(skeletonGroup, 'rightAnkle');
let rightKnee = getKeyPointFromSVG(skeletonGroup, 'rightKnee');
let rightHip = getKeyPointFromSVG(skeletonGroup, 'rightHip');
let rightWrist = getKeyPointFromSVG(skeletonGroup, 'rightWrist');
let rightElbow = getKeyPointFromSVG(skeletonGroup, 'rightElbow');
let rightShoulder = getKeyPointFromSVG(skeletonGroup, 'rightShoulder');
// Face
let topMid = getKeyPointFromSVG(skeletonGroup, 'topMid');
let rightTop0 = getKeyPointFromSVG(skeletonGroup, 'rightTop0');
let rightTop1 = getKeyPointFromSVG(skeletonGroup, 'rightTop1');
let leftTop0 = getKeyPointFromSVG(skeletonGroup, 'leftTop0');
let leftTop1 = getKeyPointFromSVG(skeletonGroup, 'leftTop1');
let leftJaw2 = getKeyPointFromSVG(skeletonGroup, 'leftJaw2');
let leftJaw3 = getKeyPointFromSVG(skeletonGroup, 'leftJaw3');
let leftJaw4 = getKeyPointFromSVG(skeletonGroup, 'leftJaw4');
let leftJaw5 = getKeyPointFromSVG(skeletonGroup, 'leftJaw5');
let leftJaw6 = getKeyPointFromSVG(skeletonGroup, 'leftJaw6');
let leftJaw7 = getKeyPointFromSVG(skeletonGroup, 'leftJaw7');
let jawMid = getKeyPointFromSVG(skeletonGroup, 'jawMid');
let rightJaw2 = getKeyPointFromSVG(skeletonGroup, 'rightJaw2');
let rightJaw3 = getKeyPointFromSVG(skeletonGroup, 'rightJaw3');
let rightJaw4 = getKeyPointFromSVG(skeletonGroup, 'rightJaw4');
let rightJaw5 = getKeyPointFromSVG(skeletonGroup, 'rightJaw5');
let rightJaw6 = getKeyPointFromSVG(skeletonGroup, 'rightJaw6');
let rightJaw7 = getKeyPointFromSVG(skeletonGroup, 'rightJaw7');
let nose0 = getKeyPointFromSVG(skeletonGroup, 'nose0');
let nose1 = getKeyPointFromSVG(skeletonGroup, 'nose1');
let nose2 = getKeyPointFromSVG(skeletonGroup, 'nose2');
let nose3 = getKeyPointFromSVG(skeletonGroup, 'nose3');
let nose4 = getKeyPointFromSVG(skeletonGroup, 'nose4');
let leftNose0 = getKeyPointFromSVG(skeletonGroup, 'leftNose0');
let leftNose1 = getKeyPointFromSVG(skeletonGroup, 'leftNose1');
let rightNose0 = getKeyPointFromSVG(skeletonGroup, 'rightNose0');
let rightNose1 = getKeyPointFromSVG(skeletonGroup, 'rightNose1');
let leftEye0 = getKeyPointFromSVG(skeletonGroup, 'leftEye0');
let leftEye1 = getKeyPointFromSVG(skeletonGroup, 'leftEye1');
let leftEye2 = getKeyPointFromSVG(skeletonGroup, 'leftEye2');
let leftEye3 = getKeyPointFromSVG(skeletonGroup, 'leftEye3');
let leftEye4 = getKeyPointFromSVG(skeletonGroup, 'leftEye4');
let leftEye5 = getKeyPointFromSVG(skeletonGroup, 'leftEye5');
let rightEye0 = getKeyPointFromSVG(skeletonGroup, 'rightEye0');
let rightEye1 = getKeyPointFromSVG(skeletonGroup, 'rightEye1');
let rightEye2 = getKeyPointFromSVG(skeletonGroup, 'rightEye2');
let rightEye3 = getKeyPointFromSVG(skeletonGroup, 'rightEye3');
let rightEye4 = getKeyPointFromSVG(skeletonGroup, 'rightEye4');
let rightEye5 = getKeyPointFromSVG(skeletonGroup, 'rightEye5');
let leftBrow0 = getKeyPointFromSVG(skeletonGroup, 'leftBrow0');
let leftBrow1 = getKeyPointFromSVG(skeletonGroup, 'leftBrow1');
let leftBrow2 = getKeyPointFromSVG(skeletonGroup, 'leftBrow2');
let leftBrow3 = getKeyPointFromSVG(skeletonGroup, 'leftBrow3');
let leftBrow4 = getKeyPointFromSVG(skeletonGroup, 'leftBrow4');
let rightBrow0 = getKeyPointFromSVG(skeletonGroup, 'rightBrow0');
let rightBrow1 = getKeyPointFromSVG(skeletonGroup, 'rightBrow1');
let rightBrow2 = getKeyPointFromSVG(skeletonGroup, 'rightBrow2');
let rightBrow3 = getKeyPointFromSVG(skeletonGroup, 'rightBrow3');
let rightBrow4 = getKeyPointFromSVG(skeletonGroup, 'rightBrow4');
let leftMouthCorner = getKeyPointFromSVG(skeletonGroup, 'leftMouthCorner');
let leftUpperLipTop0 = getKeyPointFromSVG(skeletonGroup, 'leftUpperLipTop0');
let leftUpperLipTop1 = getKeyPointFromSVG(skeletonGroup, 'leftUpperLipTop1');
let upperLipTopMid = getKeyPointFromSVG(skeletonGroup, 'upperLipTopMid');
let rightMouthCorner = getKeyPointFromSVG(skeletonGroup, 'rightMouthCorner');
let rightUpperLipTop0 = getKeyPointFromSVG(skeletonGroup, 'rightUpperLipTop0');
let rightUpperLipTop1 = getKeyPointFromSVG(skeletonGroup, 'rightUpperLipTop1');
let rightMiddleLip = getKeyPointFromSVG(skeletonGroup, 'rightMiddleLip');
let rightUpperLipBottom1 = getKeyPointFromSVG(skeletonGroup, 'rightUpperLipBottom1');
let leftMiddleLip = getKeyPointFromSVG(skeletonGroup, 'leftMiddleLip');
let leftUpperLipBottom1 = getKeyPointFromSVG(skeletonGroup, 'leftUpperLipBottom1');
let upperLipBottomMid = getKeyPointFromSVG(skeletonGroup, 'upperLipBottomMid');
let rightLowerLipTop0 = getKeyPointFromSVG(skeletonGroup, 'rightLowerLipTop0');
let leftLowerLipTop0 = getKeyPointFromSVG(skeletonGroup, 'leftLowerLipTop0');
let lowerLipTopMid = getKeyPointFromSVG(skeletonGroup, 'lowerLipTopMid');
let rightLowerLipBottom0 = getKeyPointFromSVG(skeletonGroup, 'rightLowerLipBottom0');
let rightLowerLipBottom1 = getKeyPointFromSVG(skeletonGroup, 'rightLowerLipBottom1');
let leftLowerLipBottom0 = getKeyPointFromSVG(skeletonGroup, 'leftLowerLipBottom0');
let leftLowerLipBottom1 = getKeyPointFromSVG(skeletonGroup, 'leftLowerLipBottom1');
let lowerLipBottomMid = getKeyPointFromSVG(skeletonGroup, 'lowerLipBottomMid');
this.bLeftShoulderRightShoulder = new Bone().set(leftShoulder, rightShoulder, this, 'body');
this.bRightShoulderRightHip = new Bone().set(rightShoulder, rightHip, this, 'body');
this.bLeftHipRightHip = new Bone().set(leftHip, rightHip, this, 'body');
this.bLeftShoulderLeftHip = new Bone().set(leftShoulder, leftHip, this, 'body');
this.bLeftShoulderLeftElbow = new Bone().set(leftShoulder, leftElbow, this, 'body');
this.bLeftElbowLeftWrist = new Bone().set(leftElbow, leftWrist, this, 'body');
this.bRightShoulderRightElbow = new Bone().set(rightShoulder, rightElbow, this, 'body');
this.bRightElbowRightWrist = new Bone().set(rightElbow, rightWrist, this, 'body');
this.bLeftHipLeftKnee = new Bone().set(leftHip, leftKnee, this, 'body');
this.bLeftKneeLeftAnkle = new Bone().set(leftKnee, leftAnkle, this, 'body');
this.bRightHipRightKnee = new Bone().set(rightHip, rightKnee, this, 'body');
this.bRightKneeRightAnkle = new Bone().set(rightKnee, rightAnkle, this, 'body');
this.bTopMidRightTop0 = new Bone().set(topMid, rightTop0, this, 'face');
this.bTopMidLeftTop0 = new Bone().set(topMid, leftTop0, this, 'face');
this.bLeftTop0LeftTop1 = new Bone().set(leftTop0, leftTop1, this, 'face');
this.bLeftTop1LeftJaw2 = new Bone().set(leftTop1, leftJaw2, this, 'face');
this.bLeftJaw2LeftJaw3 = new Bone().set(leftJaw2, leftJaw3, this, 'face');
this.bLeftJaw3LeftJaw4 = new Bone().set(leftJaw3, leftJaw4, this, 'face');
this.bLeftJaw4LeftJaw5 = new Bone().set(leftJaw4, leftJaw5, this, 'face');
this.bLeftJaw5LeftJaw6 = new Bone().set(leftJaw5, leftJaw6, this, 'face');
this.bLeftJaw6LeftJaw7 = new Bone().set(leftJaw6, leftJaw7, this, 'face');
this.bLeftJaw7JawMid = new Bone().set(leftJaw7, jawMid, this, 'face');
this.bRightTop0RightTop1 = new Bone().set(rightTop0, rightTop1, this, 'face');
this.bRightTop1RightJaw2 = new Bone().set(rightTop1, rightJaw2, this, 'face');
this.bRightJaw2RightJaw3 = new Bone().set(rightJaw2, rightJaw3, this, 'face');
this.bRightJaw3RightJaw4 = new Bone().set(rightJaw3, rightJaw4, this, 'face');
this.bRightJaw4RightJaw5 = new Bone().set(rightJaw4, rightJaw5, this, 'face');
this.bRightJaw5RightJaw6 = new Bone().set(rightJaw5, rightJaw6, this, 'face');
this.bRightJaw6RightJaw7 = new Bone().set(rightJaw6, rightJaw7, this, 'face');
this.bRightJaw7JawMid = new Bone().set(rightJaw7, jawMid, this, 'face');
this.bLeftEye0LeftEye1 = new Bone().set(leftEye0, leftEye1, this, 'face');
this.bLeftEye1LeftEye2 = new Bone().set(leftEye1, leftEye2, this, 'face');
this.bLeftEye2LeftEye3 = new Bone().set(leftEye2, leftEye3, this, 'face');
this.bLeftEye3LeftEye4 = new Bone().set(leftEye3, leftEye4, this, 'face');
this.bLeftEye4LeftEye5 = new Bone().set(leftEye4, leftEye5, this, 'face');
this.bLeftEye5LeftEye0 = new Bone().set(leftEye5, leftEye0, this, 'face');
this.bRightEye0RightEye1 = new Bone().set(rightEye0, rightEye1, this, 'face');
this.bRightEye1RightEye2 = new Bone().set(rightEye1, rightEye2, this, 'face');
this.bRightEye2RightEye3 = new Bone().set(rightEye2, rightEye3, this, 'face');
this.bRightEye3RightEye4 = new Bone().set(rightEye3, rightEye4, this, 'face');
this.bRightEye4RightEye5 = new Bone().set(rightEye4, rightEye5, this, 'face');
this.bRightEye5RightEye0 = new Bone().set(rightEye5, rightEye0, this, 'face');
this.bLeftBrow0LeftBrow1 = new Bone().set(leftBrow0, leftBrow1, this, 'face');
this.bLeftBrow1LeftBrow2 = new Bone().set(leftBrow1, leftBrow2, this, 'face');
this.bLeftBrow2LeftBrow3 = new Bone().set(leftBrow2, leftBrow3, this, 'face');
this.bLeftBrow3LeftBrow4 = new Bone().set(leftBrow3, leftBrow4, this, 'face');
this.bRightBrow0RightBrow1 = new Bone().set(rightBrow0, rightBrow1, this, 'face');
this.bRightBrow1RightBrow2 = new Bone().set(rightBrow1, rightBrow2, this, 'face');
this.bRightBrow2RightBrow3 = new Bone().set(rightBrow2, rightBrow3, this, 'face');
this.bRightBrow3RightBrow4 = new Bone().set(rightBrow3, rightBrow4, this, 'face');
this.bNose0Nose1 = new Bone().set(nose0, nose1, this, 'face');
this.bNose1Nose2 = new Bone().set(nose1, nose2, this, 'face');
this.bNose2Nose3 = new Bone().set(nose2, nose3, this, 'face');
this.bNose3Nose4 = new Bone().set(nose3, nose4, this, 'face');
this.bLeftNose0LeftNose1 = new Bone().set(leftNose0, leftNose1, this, 'face');
this.bLeftNose1Nose4 = new Bone().set(leftNose1, nose4, this, 'face');
this.bRightNose0RightNose1 = new Bone().set(rightNose0, rightNose1, this, 'face');
this.bRightNose1Nose4 = new Bone().set(rightNose1, nose4, this, 'face');
this.bLeftMouthCornerLeftUpperLipTop0 = new Bone().set(leftMouthCorner, leftUpperLipTop0, this, 'face');
this.bLeftUpperLipTop0LeftUpperLipTop1 = new Bone().set(leftUpperLipTop0, leftUpperLipTop1, this, 'face');
this.bLeftUpperLipTop1UpperLipTopMid = new Bone().set(leftUpperLipTop1, upperLipTopMid, this, 'face');
this.bRigthMouthCornerRigthUpperLipTop0 = new Bone().set(rightMouthCorner, rightUpperLipTop0, this, 'face');
this.bRigthUpperLipTop0RigthUpperLipTop1 = new Bone().set(rightUpperLipTop0, rightUpperLipTop1, this, 'face');
this.bRigthUpperLipTop1UpperLipTopMid = new Bone().set(rightUpperLipTop1, upperLipTopMid, this, 'face');
this.bLeftMouthCornerLeftMiddleLip = new Bone().set(leftMouthCorner, leftMiddleLip, this, 'face');
this.bLeftMiddleLipLeftUpperLipBottom1 = new Bone().set(leftMiddleLip, leftUpperLipBottom1, this, 'face');
this.bLeftUpperLipBottom1UpperLipBottomMid = new Bone().set(leftUpperLipBottom1, upperLipBottomMid, this, 'face');
this.bRightMouthCornerRightMiddleLip = new Bone().set(rightMouthCorner, rightMiddleLip, this, 'face');
this.bRightMiddleLipRightUpperLipBottom1 = new Bone().set(rightMiddleLip, rightUpperLipBottom1, this, 'face');
this.bRightUpperLipBottom1UpperLipBototmMid = new Bone().set(rightUpperLipBottom1, upperLipBottomMid, this, 'face');
this.bLeftMiddleLipLeftLowerLipTop0 = new Bone().set(leftMiddleLip, leftLowerLipTop0, this, 'face');
this.bLeftLowerLipTop0LowerLipTopMid = new Bone().set(leftLowerLipTop0, lowerLipTopMid, this, 'face');
this.bRightMiddleLipRightLowerLipTop0 = new Bone().set(rightMiddleLip, rightLowerLipTop0, this, 'face');
this.bRightLowerLipTop0LowerLipTopMid = new Bone().set(rightLowerLipTop0, lowerLipTopMid, this, 'face');
this.bLeftMouthCornerLeftLowerLipBottom0 = new Bone().set(leftMouthCorner, leftLowerLipBottom0, this, 'face');
this.bLeftLowerLipBottom0LeftLowerLipBottom1 = new Bone().set(leftLowerLipBottom0, leftLowerLipBottom1, this, 'face');
this.bLeftLowerLipBottom1LowerLipBottomMid = new Bone().set(leftLowerLipBottom1, lowerLipBottomMid, this, 'face');
this.bRightMouthCornerRightLowerLipBottom0 = new Bone().set(rightMouthCorner, rightLowerLipBottom0, this, 'face');
this.bRightLowerLipBottom0RightLowerLipBottom1 = new Bone().set(rightLowerLipBottom0, rightLowerLipBottom1, this, 'face');
this.bRightLowerLipBottom1LowerLipBottomMid = new Bone().set(rightLowerLipBottom1, lowerLipBottomMid, this, 'face');
this.faceBones = [
// Face
this.bTopMidRightTop0,
this.bRightTop0RightTop1,
this.bTopMidLeftTop0,
this.bLeftTop0LeftTop1,
this.bLeftTop1LeftJaw2,
this.bLeftJaw2LeftJaw3,
this.bLeftJaw3LeftJaw4,
this.bLeftJaw4LeftJaw5,
this.bLeftJaw5LeftJaw6,
this.bLeftJaw6LeftJaw7,
this.bLeftJaw7JawMid,
this.bRightTop1RightJaw2,
this.bRightJaw2RightJaw3,
this.bRightJaw3RightJaw4,
this.bRightJaw4RightJaw5,
this.bRightJaw5RightJaw6,
this.bRightJaw6RightJaw7,
this.bRightJaw7JawMid,
this.bLeftEye0LeftEye1,
this.bLeftEye1LeftEye2,
this.bLeftEye2LeftEye3,
this.bLeftEye3LeftEye4,
this.bLeftEye4LeftEye5,
this.bLeftEye5LeftEye0,
this.bRightEye0RightEye1,
this.bRightEye1RightEye2,
this.bRightEye2RightEye3,
this.bRightEye3RightEye4,
this.bRightEye4RightEye5,
this.bRightEye5RightEye0,
this.bLeftBrow0LeftBrow1,
this.bLeftBrow1LeftBrow2,
this.bLeftBrow2LeftBrow3,
this.bLeftBrow3LeftBrow4,
this.bRightBrow0RightBrow1,
this.bRightBrow1RightBrow2,
this.bRightBrow2RightBrow3,
this.bRightBrow3RightBrow4,
this.bNose0Nose1,
this.bNose1Nose2,
this.bNose2Nose3,
this.bNose3Nose4,
this.bLeftNose0LeftNose1,
this.bLeftNose1Nose4,
this.bRightNose0RightNose1,
this.bRightNose1Nose4,
this.bLeftMouthCornerLeftUpperLipTop0,
this.bLeftUpperLipTop0LeftUpperLipTop1,
this.bLeftUpperLipTop1UpperLipTopMid,
this.bRigthMouthCornerRigthUpperLipTop0,
this.bRigthUpperLipTop0RigthUpperLipTop1,
this.bRigthUpperLipTop1UpperLipTopMid,
this.bLeftMouthCornerLeftMiddleLip,
this.bLeftMiddleLipLeftUpperLipBottom1,
this.bLeftUpperLipBottom1UpperLipBottomMid,
this.bRightMouthCornerRightMiddleLip,
this.bRightMiddleLipRightUpperLipBottom1,
this.bRightUpperLipBottom1UpperLipBototmMid,
this.bLeftMiddleLipLeftLowerLipTop0,
this.bLeftLowerLipTop0LowerLipTopMid,
this.bRightMiddleLipRightLowerLipTop0,
this.bRightLowerLipTop0LowerLipTopMid,
this.bLeftMouthCornerLeftLowerLipBottom0,
this.bLeftLowerLipBottom0LeftLowerLipBottom1,
this.bLeftLowerLipBottom1LowerLipBottomMid,
this.bRightMouthCornerRightLowerLipBottom0,
this.bRightLowerLipBottom0RightLowerLipBottom1,
this.bRightLowerLipBottom1LowerLipBottomMid,
]
this.bodyBones = [
// Body
this.bLeftShoulderRightShoulder,
this.bRightShoulderRightHip,
this.bLeftHipRightHip,
this.bLeftShoulderLeftHip,
this.bLeftShoulderLeftElbow,
this.bLeftElbowLeftWrist,
this.bRightShoulderRightElbow,
this.bRightElbowRightWrist,
this.bLeftHipLeftKnee,
this.bLeftKneeLeftAnkle,
this.bRightHipRightKnee,
this.bRightKneeRightAnkle
];
this.bones = this.faceBones.concat(this.bodyBones);
this.secondaryBones = [];
this.parts = {};
this.bodyLen0 = this.getTotalBoneLength(this.bodyBones);
this.faceLen0 = this.getTotalBoneLength(this.faceBones);
this.boneGroups = {
'torso': [
this.bLeftShoulderRightShoulder,
this.bRightShoulderRightHip,
this.bLeftHipRightHip,
this.bLeftShoulderLeftHip,
],
'leftLeg': [
this.bLeftHipLeftKnee,
this.bLeftKneeLeftAnkle,
],
'rightLeg': [
this.bRightHipRightKnee,
this.bRightKneeRightAnkle
],
'leftArm': [
this.bLeftShoulderLeftElbow,
this.bLeftElbowLeftWrist,
],
'rightArm': [
this.bRightElbowRightWrist,
this.bRightShoulderRightElbow,
],
'face': this.faceBones,
};
this.faceBones.forEach(bone => {
let parts = [bone.kp0, bone.kp1];
parts.forEach(part => {
part.baseTransFunc = MathUtils.getTransformFunc(this.bLeftJaw2LeftJaw3.kp0.position,
this.bRightJaw2RightJaw3.kp0.position, part.position);
});
});
}
update(pose, face) {
if (pose.score < MIN_POSE_SCORE) {
this.isValid = false;
return;
}
this.isValid = this.updatePoseParts(pose);
if (!this.isValid) return;
this.isValid = this.updateFaceParts(face);
if (!this.isValid) return;
// Update bones.
this.bones.forEach(bone => {
let part0 = this.parts[bone.kp0.name];
let part1 = this.parts[bone.kp1.name];
bone.kp0.currentPosition = part0.position;
bone.kp1.currentPosition = part1.position;
bone.score = (part0.score + part1.score) / 2;
bone.latestCenter = bone.kp1.currentPosition.add(bone.kp0.currentPosition).divide(2);
});
// Update secondary bones.
let nosePos = this.bNose3Nose4.kp1.currentPosition;
this.secondaryBones.forEach(bone => {
bone.kp0.currentPosition = bone.kp0.transformFunc(bone.parent.kp0.currentPosition, nosePos);
bone.kp1.currentPosition = bone.kp1.transformFunc(bone.parent.kp1.currentPosition, nosePos);
bone.score = bone.parent.score;
bone.latestCenter = bone.kp1.currentPosition.add(bone.kp0.currentPosition).divide(2);
});
// Recompute face & body bone scale.
this.currentFaceScale = this.getTotalBoneLength(this.faceBones) / this.faceLen0;
this.currentBodyScale = this.getTotalBoneLength(this.bodyBones) / this.bodyLen0;
this.isValid = true;
}
updatePoseParts(pose) {
posePartNames.forEach(partName => {
// Use new and old pose's confidence scores as weights to compute the new part position.
let part1 = getPartFromPose(pose, partName);
let part0 = (this.parts[partName] || part1);
let weight0 = part0.score / (part1.score + part0.score);
let weight1 = part1.score / (part1.score + part0.score);
let pos = part0.position.multiply(weight0).add(part1.position.multiply(weight1));
this.parts[partName] = {
position: pos,
score: part0.score * weight0 + part1.score * weight1,
};
});
if (!this.parts['rightEar'] || !this.parts['leftEar']) {
return false;
}
return true;
}
updateFaceParts(face) {
let posLeftEar = this.parts['leftEar'].position;
let posRightEar = this.parts['rightEar'].position;
if (face && face.positions && face.positions.length && face.faceInViewConfidence > MIN_FACE_SCORE) {
// Valid face results.
for (let i = 0; i < facePartNames.length; i++) {
let partName = facePartNames[i];
let pos = getKeypointFromFaceFrame(face, i);
if (!pos) continue;
this.parts[partName] = {
position: pos,
score: face.faceInViewConfidence
};
}
// Keep track of the transformation from pose ear positions to face ear positions.
// This can be used to infer face position when face tracking is lost.
this.leftEarP2FFunc = MathUtils.getTransformFunc(posLeftEar, posRightEar, this.parts['leftJaw2'].position);
this.rightEarP2FFunc = MathUtils.getTransformFunc(posLeftEar, posRightEar, this.parts['rightJaw2'].position);
} else {
// Invalid face keypoints. Infer face keypoints from pose.
let fLeftEar = this.leftEarP2FFunc ? this.leftEarP2FFunc(posLeftEar, posRightEar) : posLeftEar;
let fRightEar = this.rightEarP2FFunc ? this.rightEarP2FFunc(posLeftEar, posRightEar) : posRightEar;
// Also infer face scale from pose.
this.currentFaceScale = this.currentBodyScale;
this.faceBones.forEach(bone => {
let parts = [bone.kp0, bone.kp1];
parts.forEach(part => {
this.parts[part.name] = {
position: part.baseTransFunc(fLeftEar, fRightEar),
score: 1,
};
});
});
}
return true;
}
findBoneGroup(point) {
let minDistances = {};
Object.keys(this.boneGroups).forEach(boneGroupKey => {
let minDistance = Infinity;
let boneGroup = this.boneGroups[boneGroupKey];
boneGroup.forEach(bone => {
let d = MathUtils.getClosestPointOnSegment(bone.kp0.position, bone.kp1.position, point)
.getDistance(point);
minDistance = Math.min(minDistance, d);
});
minDistances[boneGroupKey] = minDistance;
});
let minDistance = Math.min(...Object.values(minDistances));
let selectedGroups = [];
Object.keys(minDistances).forEach(key => {
let distance = minDistances[key];
if (distance <= minDistance) {
selectedGroups.push(this.boneGroups[key]);
}
});
return selectedGroups.flatten();
}
getTotalBoneLength(bones) {
let totalLen = 0;
bones.forEach(bone => {
let d = (bone.kp0.currentPosition || bone.kp0.position).subtract(bone.kp1.currentPosition || bone.kp1.position);
totalLen += d.length;
});
return totalLen;
}
debugDraw(scope) {
let group = new scope.Group();
scope.project.activeLayer.addChild(group);
this.bones.forEach(bone => {
let path = new scope.Path({
segments: [bone.kp0.currentPosition, bone.kp1.currentPosition],
strokeWidth: 2,
strokeColor: bone.boneColor
});
group.addChild(path);
});
// this.secondaryBones.forEach(bone => {
// let path = new scope.Path({
// segments: [bone.kp0.currentPosition, bone.kp1.currentPosition],
// strokeColor: '#00ff00',
// strokeWidth: 5,
// });
// group.addChild(path);
// });
}
debugDrawLabels(scope) {
let group = new scope.Group();
scope.project.activeLayer.addChild(group);
this.bones.forEach(bone => {
let addLabel = (kp, name) => {
let text = new scope.PointText({
point: [kp.currentPosition.x, kp.currentPosition.y],
content: name,
fillColor: 'black',
fontSize: 7
});
group.addChild(text);
};
addLabel(bone.kp0, bone.kp0.name);
addLabel(bone.kp1, bone.kp1.name);
});
}
reset() {
this.parts = [];
}
static getCurrentPosition(segment) {
let position = new paper.default.Point();
Object.keys(segment.skinning).forEach(boneName => {
let bt = segment.skinning[boneName];
position = position.add(bt.bone.transform(bt.transform).multiply(bt.weight));
});
return position;
}
static flipPose(pose) {
pose.keypoints.forEach(kp => {
if (kp.part && kp.part.startsWith('left')) {
kp.part = 'right' + kp.part.substring('left'.length, kp.part.length);
} else if (kp.part && kp.part.startsWith('right')) {
kp.part = 'left' + kp.part.substring('right'.length, kp.part.length);
}
});
}
static flipFace(face) {
Object.keys(facePartName2Index).forEach(partName => {
if (partName.startsWith('left')) {
let rightName = 'right' + partName.substr('left'.length, partName.length);
let temp = face.scaledMesh[facePartName2Index[partName]];
face.scaledMesh[facePartName2Index[partName]] = face.scaledMesh[facePartName2Index[rightName]];
face.scaledMesh[facePartName2Index[rightName]] = temp;
}
});
}
static getBoundingBox(pose) {
let minX = 100000;
let maxX = -100000;
let minY = 100000;
let maxY = -100000;
let updateMinMax = (x, y) => {
minX = Math.min(x, minX);
maxX = Math.max(x, maxX);
minY = Math.min(y, minY);
maxY = Math.max(y, maxY);
}
pose.frames.forEach(frame => {
frame.pose.keypoints.forEach(kp => {
updateMinMax(kp.position.x, kp.position.y);
});
let faceKeypoints = frame.face.positions;
for (let i = 0; i < faceKeypoints.length; i += 2) {
updateMinMax(faceKeypoints[i], faceKeypoints[i+1]);
}
});
return [minX, maxX, minY, maxY];
}
static translatePose(pose, d) {
pose.frames.forEach(frame => {
frame.pose.keypoints.forEach(kp => {
kp.position.x += d.x;
kp.position.y += d.y;
});
let faceKeypoints = frame.face.positions;
for (let i = 0; i < faceKeypoints.length; i += 2) {
faceKeypoints[i] += d.x;
faceKeypoints[i+1] += d.y;
}
});
}
static resizePose(pose, origin, scale) {
pose.frames.forEach(frame => {
frame.pose.keypoints.forEach(kp => {
kp.position.x = origin.x + (kp.position.x - origin.x) * scale.x;
kp.position.y = origin.y + (kp.position.y - origin.y) * scale.y;
});
let faceKeypoints = frame.face.positions;
for (let i = 0; i < faceKeypoints.length; i += 2) {
faceKeypoints[i] = origin.x + (faceKeypoints[i] - origin.x) * scale.x;
faceKeypoints[i+1] = origin.y + (faceKeypoints[i+1] - origin.y) * scale.y;
}
});
}
static toFaceFrame(faceDetection) {
let frame = {
positions: [],
faceInViewConfidence: faceDetection.faceInViewConfidence,
};
for (let i = 0; i < facePartNames.length; i++) {
let partName = facePartNames[i];
let p = faceDetection.scaledMesh[facePartName2Index[partName]];
frame.positions.push(p[0]);
frame.positions.push(p[1]);
}
return frame;
}
}
================================================
FILE: index.html
================================================
<!DOCTYPE html>
<html>
<head>
<title>Pose Animator Demos</title>
</head>
<body>
<h1>Pose Animator Demos</h1>
<ul>
<li><a href='camera.html'>Animated 2D avatar - Camera feed demo</a></li>
<li><a href='static_image.html'>Animated 2D avatar - Image demo</a></li>
</ul>
</body>
</html>
================================================
FILE: package.json
================================================
{
"name": "tfjs-models",
"version": "0.0.1",
"description": "",
"main": "index.js",
"license": "Apache-2.0",
"private": true,
"engines": {
"node": ">=8.9.0"
},
"dependencies": {
"@tensorflow-models/facemesh": "^0.0.1",
"@tensorflow-models/posenet": "^2.2.1",
"@tensorflow/tfjs": "^1.7.0",
"@tensorflow/tfjs-converter": "^1.7.0",
"@tensorflow/tfjs-core": "^1.7.0",
"face-api.js": "^0.22.1",
"paper": "file:third_party/paper",
"stats.js": "^0.17.0"
},
"scripts": {
"watch": "cross-env NODE_ENV=development parcel index.html --no-hmr --open ",
"build": "cross-env NODE_ENV=production parcel build index.html --public-url ./",
"build-camera": "cross-env NODE_ENV=production parcel build camera.html --public-url ./",
"lint": "eslint .",
"link-local": "yalc link"
},
"browser": {
"crypto": false
},
"devDependencies": {
"babel-core": "^6.26.3",
"babel-plugin-transform-runtime": "~6.23.0",
"babel-polyfill": "~6.26.0",
"babel-preset-env": "~1.6.1",
"babel-preset-es2017": "^6.24.1",
"clang-format": "~1.2.2",
"cross-env": "^5.2.0",
"dat.gui": "^0.7.2",
"eslint": "^4.19.1",
"eslint-config-google": "^0.9.1",
"parcel-bundler": "~1.12.4",
"yalc": "~1.0.0-pre.27"
},
"eslintConfig": {
"extends": "google",
"rules": {
"require-jsdoc": 0,
"valid-jsdoc": 0
},
"env": {
"es6": true
},
"parserOptions": {
"ecmaVersion": 8,
"sourceType": "module"
}
},
"eslintIgnore": [
"dist/"
]
}
================================================
FILE: static_image.html
================================================
<!DOCTYPE html>
<html>
<head>
<title>PoseNet - Coco Images Demo</title>
<style type='text/css'>
.illustration-canvas {
position: absolute;
left: 600px;
top: 100px;
border: 1px solid #eeeeee;
}
.detection-canvas {
position: absolute;
left: 10px;
top: 100px;
}
h3,
h4,
p {
margin: 10px 0;
}
#main ul {
margin: 10px 0;
padding: 0;
}
#main li {
list-style-type: none;
display: inline-block;
}
#status {
display: inline-block;
}
#results>div {
display: inline-block;
width: 513px
}
.footer {
position: fixed;
left: 0;
bottom: 0;
width: 100%;
color: black;
}
.footer-text {
max-width: 600px;
margin: auto;
}
/*
* The following loading spinner CSS is from SpinKit project
* https://github.com/tobiasahlin/SpinKit
*/
.sk-spinner-pulse {
width: 20px;
height: 20px;
margin: auto 10px;
float: left;
background-color: #333;
border-radius: 100%;
-webkit-animation: sk-pulseScaleOut 1s infinite ease-in-out;
animation: sk-pulseScaleOut 1s infinite ease-in-out;
}
@-webkit-keyframes sk-pulseScaleOut {
0% {
-webkit-transform: scale(0);
transform: scale(0);
}
100% {
-webkit-transform: scale(1.0);
transform: scale(1.0);
opacity: 0;
}
}
@keyframes sk-pulseScaleOut {
0% {
-webkit-transform: scale(0);
transform: scale(0);
}
100% {
-webkit-transform: scale(1.0);
transform: scale(1.0);
opacity: 0;
}
}
.spinner-text {
float: left;
}
</style>
</head>
<body>
<div id="loading">
<span id="status"></span>
<span class="sk-spinner sk-spinner-pulse"></span>
</div>
<div id='main'>
<div id='results' style='display:none'>
<div id="multi">
<h4>Pose Animator - Static Image Demo</h4>
<video id="video" playsinline style=" -moz-transform: scaleX(-1);
-o-transform: scaleX(-1);
-webkit-transform: scaleX(-1);
transform: scaleX(-1);
display: none;
">
</video>
<canvas class="detection-canvas"></canvas>
<canvas class="illustration-canvas"></cavnas>
</div>
</div>
</div>
<div class="footer">
<div class="footer-text">
<p>
Pose Animator runs TF.js <strong>FaceMesh</strong> and <strong>PoseNet</strong> models to animate SVG illustrations with camera feed / static images.<br>
It currently supports <strong>single-pose</strong>, <strong>single-face</strong> detection, and has been tested on Destkop Chrome & iOS Safari.
<br>
(PoseNet model config - MobileNet V1, output stride 16, quant bytes 2)
</p>
</div>
</div>
<script src="static_image.js"></script>
</body>
</html>
================================================
FILE: static_image.js
================================================
/**
* @license
* Copyright 2020 Google Inc. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* =============================================================================
*/
import * as posenet_module from '@tensorflow-models/posenet';
import * as facemesh_module from '@tensorflow-models/facemesh';
import * as tf from '@tensorflow/tfjs';
import * as paper from 'paper';
import "babel-polyfill";
import dat from 'dat.gui';
import {SVGUtils} from './utils/svgUtils'
import {PoseIllustration} from './illustrationGen/illustration';
import {Skeleton, facePartName2Index} from './illustrationGen/skeleton';
import {toggleLoadingUI, setStatusText} from './utils/demoUtils';
import * as boySVG from './resources/illustration/boy.svg';
import * as girlSVG from './resources/illustration/girl.svg';
import * as abstractSVG from './resources/illustration/abstract.svg';
import * as blathersSVG from './resources/illustration/blathers.svg';
import * as tomNookSVG from './resources/illustration/tom-nook.svg';
import * as boy_doughnut from './resources/images/boy_doughnut.jpg';
import * as tie_with_beer from './resources/images/tie_with_beer.jpg';
import * as test_img from './resources/images/test.png';
import * as full_body from './resources/images/full-body.png';
import * as full_body_1 from './resources/images/full-body_1.png';
import * as full_body_2 from './resources/images/full-body_2.png';
// clang-format off
import {
drawKeypoints,
drawPoint,
drawSkeleton,
renderImageToCanvas,
} from './utils/demoUtils';
import { FileUtils } from './utils/fileUtils';
// clang-format on
const resnetArchitectureName = 'MobileNetV1';
const avatarSvgs = {
'girl': girlSVG.default,
'boy': boySVG.default,
'abstract': abstractSVG.default,
'blathers': blathersSVG.default,
'tom-nook': tomNookSVG.default,
};
const sourceImages = {
'boy_doughnut': boy_doughnut.default,
'tie_with_beer': tie_with_beer.default,
'test_img': test_img.default,
'full_body': full_body.default,
'full_body_1': full_body_1.default,
'full_body_2': full_body_2.default,
};
let skeleton;
let illustration;
let canvasScope;
let posenet;
let facemesh;
const VIDEO_WIDTH = 513;
const VIDEO_HEIGHT = 513;
const CANVAS_WIDTH = 513;
const CANVAS_HEIGHT = 513;
const defaultQuantBytes = 2;
const defaultMultiplier = 1.0;
const defaultStride = 16;
const defaultInputResolution = 257;
const defaultMaxDetections = 1;
const defaultMinPartConfidence = 0.1;
const defaultMinPoseConfidence = 0.2;
const defaultNmsRadius = 20.0;
let predictedPoses;
let faceDetection;
let sourceImage;
/**
* Draws a pose if it passes a minimum confidence onto a canvas.
* Only the pose's keypoints that pass a minPartConfidence are drawn.
*/
function drawResults(image, canvas, faceDetection, poses) {
renderImageToCanvas(image, [VIDEO_WIDTH, VIDEO_HEIGHT], canvas);
const ctx = canvas.getContext('2d');
poses.forEach((pose) => {
if (pose.score >= defaultMinPoseConfidence) {
if (guiState.showKeypoints) {
drawKeypoints(pose.keypoints, defaultMinPartConfidence, ctx);
}
if (guiState.showSkeleton) {
drawSkeleton(pose.keypoints, defaultMinPartConfidence, ctx);
}
}
});
if (guiState.showKeypoints) {
faceDetection.forEach(face => {
Object.values(facePartName2Index).forEach(index => {
let p = face.scaledMesh[index];
drawPoint(ctx, p[1], p[0], 3, 'red');
});
});
}
}
async function loadImage(imagePath) {
const image = new Image();
const promise = new Promise((resolve, reject) => {
image.crossOrigin = '';
image.onload = () => {
resolve(image);
}
});
image.src = imagePath;
return promise;
}
function multiPersonCanvas() {
return document.querySelector('#multi canvas');
}
function getIllustrationCanvas() {
return document.querySelector('.illustration-canvas');
}
/**
* Draw the results from the multi-pose estimation on to a canvas
*/
function drawDetectionResults() {
const canvas = multiPersonCanvas();
drawResults(sourceImage, canvas, faceDetection, predictedPoses);
if (!predictedPoses || !predictedPoses.length || !illustration) {
return;
}
skeleton.reset();
canvasScope.project.clear();
if (faceDetection && faceDetection.length > 0) {
let face = Skeleton.toFaceFrame(faceDetection[0]);
illustration.updateSkeleton(predictedPoses[0], face);
} else {
illustration.updateSkeleton(predictedPoses[0], null);
}
illustration.draw(canvasScope, sourceImage.width, sourceImage.height);
if (guiState.showCurves) {
illustration.debugDraw(canvasScope);
}
if (guiState.showLabels) {
illustration.debugDrawLabel(canvasScope);
}
}
/**
* Loads an image, feeds it into posenet the posenet model, and
* calculates poses based on the model outputs
*/
async function testImageAndEstimatePoses() {
toggleLoadingUI(true);
setStatusText('Loading FaceMesh model...');
document.getElementById('results').style.display = 'none';
// Reload facemesh model to purge states from previous runs.
facemesh = await facemesh_module.load();
// Load an example image
setStatusText('Loading image...');
sourceImage = await loadImage(sourceImages[guiState.sourceImage]);
// Estimates poses
setStatusText('Predicting...');
predictedPoses = await posenet.estimatePoses(sourceImage, {
flipHorizontal: false,
decodingMethod: 'multi-person',
maxDetections: defaultMaxDetections,
scoreThreshold: defaultMinPartConfidence,
nmsRadius: defaultNmsRadius,
});
faceDetection = await facemesh.estimateFaces(sourceImage, false, false);
// Draw poses.
drawDetectionResults();
toggleLoadingUI(false);
document.getElementById('results').style.display = 'block';
}
let guiState = {
// Selected image
sourceImage: Object.keys(sourceImages)[0],
avatarSVG: Object.keys(avatarSvgs)[0],
// Detection debug
showKeypoints: true,
showSkeleton: true,
// Illustration debug
showCurves: false,
showLabels: false,
};
function setupGui() {
const gui = new dat.GUI();
const imageControls = gui.addFolder('Image');
imageControls.open();
gui.add(guiState, 'sourceImage', Object.keys(sourceImages)).onChange(() => testImageAndEstimatePoses());
gui.add(guiState, 'avatarSVG', Object.keys(avatarSvgs)).onChange(() => loadSVG(avatarSvgs[guiState.avatarSVG]));
const debugControls = gui.addFolder('Debug controls');
debugControls.open();
gui.add(guiState, 'showKeypoints').onChange(drawDetectionResults);
gui.add(guiState, 'showSkeleton').onChange(drawDetectionResults);
gui.add(guiState, 'showCurves').onChange(drawDetectionResults);
gui.add(guiState, 'showLabels').onChange(drawDetectionResults);
}
/**
* Kicks off the demo by loading the posenet model and estimating
* poses on a default image
*/
export async function bindPage() {
toggleLoadingUI(true);
canvasScope = paper.default;
let canvas = getIllustrationCanvas();
canvas.width = CANVAS_WIDTH;
canvas.height = CANVAS_HEIGHT;
canvasScope.setup(canvas);
await tf.setBackend('webgl');
setStatusText('Loading PoseNet model...');
posenet = await posenet_module.load({
architecture: resnetArchitectureName,
outputStride: defaultStride,
inputResolution: defaultInputResolution,
multiplier: defaultMultiplier,
quantBytes: defaultQuantBytes
});
setupGui(posenet);
setStatusText('Loading SVG file...');
await loadSVG(Object.values(avatarSvgs)[0]);
}
window.onload = bindPage;
FileUtils.setDragDropHandler(loadSVG);
// Target is SVG string or path
async function loadSVG(target) {
let svgScope = await SVGUtils.importSVG(target);
skeleton = new Skeleton(svgScope);
illustration = new PoseIllustration(canvasScope);
illustration.bindSkeleton(skeleton, svgScope);
testImageAndEstimatePoses();
}
================================================
FILE: third_party/paper/AUTHORS.md
================================================
## Authors
- Jürg Lehni <juerg@scratchdisk.com>
- Jonathan Puckey <jonathan@studiomoniker.com>
## Contributors
- Harikrishnan Gopalakrishnan <hari.exeption@gmail.com>
- Jan Bösenberg <development@iconexperience.com>
- Jt Whissel <jtwhissel@gmail.com>
- Andrew Roles <jaroles58@gmail.com>
- Jacob Lites <jnightlight@gmail.com>
- Justin Ridgewell <justinridgewell@gmail.com>
- Andrew Wagenheim <abwagenheim@gmail.com>
- Scott Kieronski <baroes0239@gmail.com>
- Samuel Asensi <asensi.samuel@gmail.com>
- Takahiro Nishino <sapics.dev@gmail.com>
================================================
FILE: third_party/paper/CHANGELOG.md
================================================
# Change Log
## `0.12.4`
### Added
- Allow paper core import in TypeScript (#1713).
- Boolean: Improve performance from `O(n^2)` to nearly `O(n)` by the use of the
sweep and prune algorithm (#1737).
- Docs: Add support for nullable values.
### Fixed
- Fix `PathItem#getCrossing()` to not return overlaps (#1409).
- Fix regression in `Curve.getIntersections()` (#1638).
- Fix edge cases in `CurveLocation.isCrossing()` (#1419, #1263).
- Fix `SymbolItem#hitTestAll()` to return only one match per symbol item
(#1680).
- Fix handling of negative `Shape` sizes (#1733).
- Fix parsing of RGB `Color` strings with percentages (#1736).
- Fix `Shape` bounds when passing position in constructor (#1686).
- Prevent nested group matrix from reset when transforming parent (#1711).
- Boolean: Fix edge cases in overlap detection (#1262).
- Boolean: Add check for paths with only one segment (#1351).
- Boolean: Correctly handle open filled paths (#1647).
- Boolean: Avoid winding number edge cases (#1619).
- Docs: Fix some documentation return types (#1679).
## `0.12.3`
### Added
- Add documentation for `Item#internalBounds`.
### Fixed
- Fix regression in `Color` change propagation (#1672, #1674).
- SVG Export: Fix viewport size of exported `Symbol` (#1668).
- Handle non-invertible matrices in `Item#contains()` (#1651).
- Improve documentation for `Item#clipMask` (#1673).
- Improve TypeScript definitions (#1659, #1663, #1664, #1667).
## `0.12.2`
### Fixed
- Fix drawing with compound-paths as clip-items (#1361).
- Fix drawing of path selection with small handle size (#1327).
- Do not ignore `Group#clipItem.matrix` in `Group#internalBounds` (#1427).
- Correctly calculate bounds with nested empty items (#1467).
- Fix color change propagation on groups (#1152).
- Fix `Path#arcTo()` where `from` and `to` points are equal (#1613).
- Improve `new Raster(size[, position])` constructor (#1621).
- SVG Export: Fix error when `Item#matrix` is not invertible (#1580).
- SVG Export: Include missing viewBox attribute (#1576).
- SVG Import: Use correct default values for gradients (#1632, #1660).
- SVG Import: Add basic `<switch/>` support (#1597).
- JSON Import: Prevent `Item#insert()` method from being overridden (#1392).
- PaperScript: Fix issues with increment/decrement operators (#1450, #1611).
## `0.12.1`
### Added
- Add TypeScript definition, automatically generated from JSDoc comments
(#1612).
- Support `new Raster(size[, position])` constructor.
- Expose `Raster#context` accessor.
- Implement `Raster#clear()` method to clear associated canvas context.
- Node.js: Add support for Node.js v11 and v12.
### Fixed
- Fix parsing of CSS colors with spaces in parentheses (#1629).
- Improve `Color.random()` documentation.
- Fix `Tween#then()` documentation.
### Removed
- Node.js: Remove support for Node.js v6.
## `0.12.0`
### News
Another release, another new member on the team: Please welcome
[@arnoson](https://github.com/arnoson), who has worked hard on the all new
animation support, exposed through the `Tween` class and its various methods on
the `Item` class, see below for details:
### Added
- Add new `Tween` class and related methods on `Item`, to animate and
interpolate their various properties, including colors, sub-properties, etc.:
`Item#tween(from, to, options)`, `Item#tween(to, options)`,
`Item#tween(options)`, `Item#tweenFrom(from, options)`,
`Item#tweenTo(to, options)`
### Fixed
- Only draw Raster if image is not empty (#1320).
- Emit mousedrag events on correct items when covered by other items (#1465).
- Fix drawing issues of bounds and position with `Group#selectedColor` (#1571).
- Fix `Item.once()` to actually only emit event once.
- Various documentation fixes and improvements (#1399).
## `0.11.8`
### News
This is the first release in quite a while, and it was made possible thanks to
two new people on the team:
A warm welcome to [@sasensi](https://github.com/sasensi) and
[@sapics](https://github.com/sapics), the two new and very active maintainers /
contributors! :tada:
Their efforts mean that many issues are finally getting proper attention and
solid fixes, as we are paving the way for the upcoming release of `1.0.0`. Here
the fixes and additions from the past two weeks:
### Fixed
- Prevent `paper` object from polluting the global scope (#1544).
- Make sure `Path#arcTo()` always passes through the provide through point
(#1477).
- Draw shadows on `Raster` images (#1437).
- Fix boolean operation edge case (#1506, #1513, #1515).
- Handle closed paths with only one segment in `Path#flatten()` (#1338).
- Remove memory leak on gradient colors (#1499).
- Support alpha channel in CSS colors (#1468, #1539, #1565).
- Improve color CSS string parsing and documentation.
- Improve caching of item positions (#1503).
- Always draw selected position in global coordinates system (#1545).
- Prevent empty `Symbol` items from causing issues with transformations (#1561).
- Better detect when a cached global matrix is not valid anymore (#1448).
- Correctly draw selected position when item is in a group with matrix not
applied (#1535).
- Improve handling of huge amounts of segments in paths (#1493).
- Do not trigger error messages about passive event listeners on Chrome (#1501).
- Fix errors with event listeners on mobile (#1533).
- Prevent first mouse drag event from being emitted twice (#1553).
- Support optional arguments in translate and rotate statements in SVG Import
(#1487).
- Make sure SVG import always applies imported attributes (#1416).
- Correctly handle `Raster` images positions in SVG import (#1328).
- Improve documentation for `Shape#toPath()` (#1374).
- Improve documentation of hit test coordinate system (#1430).
- Add documentation for `Item#locked` (#1436).
- Support Webpack bundling in Node.js server (#1482).
- Travis CI: Get unit tests to run correctly again.
- Travis CI: Remove Node 4 and add Node 9.
### Added
- `Curve#getTimesWithTangent()` and `Path#getOffsetsWithTangent()` as a way to
get the curve-times / offsets where the path is tangential to a given vector.
- `Raster#smoothing` to control if pixels should be blurred or repeated when a
raster is scaled up (#1521).
- Allow `PaperScript`to export from executed code, supporting `export default`,
named exports, as well as `module.exports`.
## `0.11.5`
### Fixed
- Fix `Curve#isSelected()` to correctly reflect the state of `#handle1` (#1378).
- Key Events: Fix auto-filling issue on Chrome (#1358, #1365).
- Boolean: Check that overlaps are on the right path (#1321).
- Boolean: Add better filtering for invalid segments (#1385).
### Added
- Node.js: Add JPEG support to exportFrames() (#1166).
## `0.11.4`
### Changed
- Node.js: Add support for v8, and keep testing v4, v6, v7 in Travis CI.
## `0.11.3`
### Fixed
- Mouse Events: Fix item-based `doubleclick` events (#1316).
- Overhaul the caching of bounds and matrix decomposition, improving reliability
of `Item#rotation` and `#scaling` and fixing situations caused by wrongly
caching `Item#position` and `#bounds` values.
- Prevent consumed properties in object literal constructors from being set on
the instance.
### Changed
- Make all functions and accessors enumerable on all Paper.js classes.
## `0.11.2`
### Fixed
- PaperScript: Fix a parsing error in math operations without white-space
(#1314).
## `0.11.1`
### Fixed
- Bring back deactivation of Node.js modules on browsers. This has most probably
broken Webpack bundling in `0.11.0`.
## `0.11.0`
### Changed
- Separate `paper` module on NPM into: `paper`, `paper-jsdom` and
`paper-jsdom-canvas` (#1252):
- `paper` is the main library, and can be used directly in a browser
context, e.g. a web browser or worker.
- `paper-jsdom` is a shim module for Node.js, offering headless use with SVG
importing and exporting through [jsdom](https://github.com/tmpvar/jsdom).
- `paper-jsdom-canvas` is a shim module for Node.js, offering canvas
rendering through [Node-Canvas](https://github.com/Automattic/node-canvas)
as well as SVG importing and exporting through
[jsdom](https://github.com/tmpvar/jsdom).
### Added
- PaperScript: Support newer, external versions of Acorn.js for PaperScript
parsing, opening the doors to ES 2015 (#1183, #1275).
- Hit Tests: Implement `options.position` to hit `Item#position` (#1249).
- Split `Item#copyTo()` into `#addTo()` and `#copyTo()`.
### Fixed
- Intersections: Bring back special handling of curve end-points (#1284).
- Intersections: Correctly handle `Item#applyMatrix = false` (#1289).
- Boolean: Bring back on-path winding handling (#1281).
- Boolean: Pass on options in `PathItem#subtract(path, options)` (#1221).
- Boolean: Implement `options.trace` as a way to perform boolean operations on
the strokes / traces instead of the fills / areas of the involved paths
(#1221).
- Boolean: Always return `CompoundPath items (#1221).
- Style: Fix handling of gradient matrices when `Item#applyMatrix = false`
(#1238).
- Style: Prevent cleaning pre-existing styles when setting `Item#style` to an
object (#1277).
- Mouse Events: Only handle dragItem if the hitItem responds to `mousedrag`
events (#1247, #1286).
- Bounds: Clear parent's bounds cache when item's visibility changes (#1248).
- Bounds: Fix calculation of internal bounds with children and
`Item#applyMatrix = false` (#1250).
- Hit-Tests: Fix issue with non-invertible matrices ( #1271).
- SVG Import: Improve handling of sizes in percent (#1242).
- Rectangle: Improve handling of dimension properties, dealing better with
`left` / `top` / `right` / `bottom` / `center` values (#1147).
- Scene Graph: Do not allow inserting same item as child multiple times.
- Path: Fix handling of `insert = false` in `new Path.Constructor()`
initialization (#1305).
- PaperScript: Fix positive unary operator.
- Docs: Fix parameter sequence in Matrix constructor (#1273).
- Docs: Add documentation for options.bound and options.matrix in `#exportSVG()`
(#1254).
- Docs: Fix wrong `@link` references to bean properties.
## `0.10.3`
### Changed
- Node.js: Support v7, and keep testing v4 up to v7 in Travis CI.
- Node.js: Loosely couple Node.js / Electron code to `Canvas` module, and treat
its absence like a headless web worker context in the browser (#1103).
- Clean up handling of `Item#_set()`, `#set()` and `#initialize()`:
- Use `#_set()` for actually setting internal properties, e.g. on `Point`,
`Size`, so that derived classes can reuse other parts without having to
override each individual function (e.g. in `SegmentPoint`)
- Define `#set()` as a shortcut to `#initialize()` on all basic types, to
offer the same amount of flexibility when setting values, accepting object
literals as well as lists of value arguments.
- SVG Export: Add support for shorter `h` / `v` commands for horizontal /
vertical lines in SVG output.
- JSON Import / Export: Implement new and shorter segments array notation:
- Close paths by including `true` as the last entry
- Allow nested segment arrays to be passed to `PathItem.create()` as well as
the `CompoundPath` constructor to create all sub-paths.
- Reflect `View#zoom` and `View#center` through matrix decomposition, and
implement additional decomposed properties such as `#scaling` and `#rotation`.
- Reduce various internal epsilon values for general improved precision while
maintaining reliability.
- Split `PathItem#resolveCrossings()` into `#resolveCrossings()` and
`#reorient()` (#973).
### Added
- Implement `Path#divideAt(location)`, in analogy to `Curve#divideAt(location)`.
- Add `PathItem#compare()` as a way to compare the geometry of two paths to see
if they describe the same shape, handling cases where paths start in different
segments or use different amounts of curves to describe the same shape.
- Implement `Curve#hasLength()` as an optimized check for curve-length (#1109).
- Implement `Curve#classify()` to determine the type of cubic Bézier curve via
discriminant classification, based on an approach described by Loop and Blinn,
and use it to simplify curve self-intersection handling (#773, #1074, #1235).
- Add `Curve.getPeaks()` as a fast way to retrieve points that are often similar
to the more costly curvature extrema for use in curve offsetting.
- Expose `Curve. getCurveLineIntersections()` for use in curve offsetting.
- Add `Line.getDistance()` and use it in `Curve.getOverlaps()` (#1253).
- Implement `Segment#isSmooth()` and use it in handling of stroke-joins.
- Bring back caching of `Item#rotation` and `#scaling`, but only allow matrix
decomposition-based properties on items with `#applyMatrix = false`
(#1004, #1177).
### Fixed
- Many improvements to boolean operations:
- Improve performance of boolean operations when there no actual crossings
between the paths, but paths may be contained within each other.
- Improve path tracing approach by implementing a branching structure and
sorting segments according to their reliability as starting points for
traces (#1073).
- Improve calculation and reliability of winding contributions.
- Improve code that resolves crossings and reorients compound-paths based
on how the sub-paths are nested.
- Fix issue where unite operation wrongly fills inner path (#1075).
- Better handle cases where one `Path` is open and the other closed (#1089).
- Solve `null` exceptions during complex boolean operations (#1091).
- Improve bidirectional curve-time rescaling in `divideLocations()` (#1191).
- Improve handling of intersections between touching curves (#1165).
- Improve reliability of `Curve#getIntersections()` (#1174).
- Fix `getOverlaps()` to always return overlaps in correct sequence (#1223).
- Improve handling of multiple crossings on the same curve.
- Improve tangent direction handling in `CurveLocation#isCrossing()`, by finding
unambiguous vectors, taking the curve's loop, cusp, inflection, and "peak"
points into account (#1073, #1074).
- Prevent `Path#getStrokeBounds(matrix)` from accidentally modifying segments
(#1102).
- Improve compatibility with JSPM (#1104).
- SVG Import: Correctly handle multiple sequential move commands (#1134).
- SVG Export: Properly handle generated IDs (#1138).
- SVG Export: Support multiple gradient offsets at 0 (#1241).
- Fix imprecision in `Numerical.findRoot()` (#1149).
- PaperScript: Prevent invalid JavaScript in assignment operators (#1151).
- Hit Tests: Improve handling of SymbolItem in#hitTestAll() (#1199).
- Hit Tests: Fix stroke hit-testing for rounded shape items (#1207).
- Fix matrix cloning for groups with `#applyMatrix = false` ( #1225).
- Correctly handle offset in `Curve#divideAt(offset)` (#1230).
- Fix issue with `Curve#isStraight()` where handles were checked incorrectly
for collinearity (#1269).
- Fix `Line#getSide()` imprecisions when points are on the line.
- Docs: Fix documentation of `Project#hitTestAll()` (#536).
- Docs: Improve description of `option.class` value in `Project#hitTest()`
(#632).
### Removed
- Remove `Numerical.TOLERANCE = 1e-6` as there is no internal use for it
anymore.
## `0.10.2`
### Fixed
- Get published version to work correctly in Bower again.
## `0.10.1`
### Fixed
- Correct a few issues with documentation and NPM publishing that slipped
through in the `0.10.0` release.
## `0.10.0`
### Preamble
This is a huge release for Paper.js as we aim for a version `1.0.0` release
later this year. As of this version, all notable changes are documented in the
change-log following common [CHANGELOG](http://keepachangelog.com/) conventions.
Paper.js now also adheres to [Semantic Versioning](http://semver.org/).
There are many items in the changelog (and many more items not in the changelog)
so here a high-level overview to frame the long list of changes:
- Boolean operations have been improved and overhauled for reliability and
efficiency. These include the path functions to unite, intersect, subtract,
exclude, and divide with another path.
- There was a large amount of work implementing test coverage under QUnit.
- Mouse and key handling has been re-engineered and extended to work with view.
Many outstanding bugs have been fixed with mouse and key handling.
- Many SVG-handling enhancements and bug-fixes, including handling browser-
specific interpretations of the SVG standard, have been added.
- There are API name changes for more consistency as well as some required by
changes in the EcmaScript 6 standard (e.g., `Symbol` → `SymbolDefinition`).
- Even though it is not new, since version `0.9.22` Paper.js no longer resizes
the canvas to match the view. The canvas must be resized independently.
Thank you all for using Paper.js, submitting bugs and ideas, and all those that
contribute to the code.
### Changed
- Significant overhaul and improvements of boolean path operations
`PathItem#unite()`, `#subtract()`, `#intersect()`, `#exclude()`, `#divide()`:
- Improve handling of self-intersecting paths and non-zero fill-rules.
- Handle operations on identical paths.
- Improve handling of near-collinear lines.
- Handle self-intersecting paths that merely "touch" themselves.
- Handle situations where all encountered intersections are part of overlaps.
- Methods that accepted a `time` parameter or boolean second parameter causing
the argument to be interpreted as curve-time instead of offset are now
separate functions with distinct names (#563):
- `Curve#getNormalAt(time, true)` → `#getNormalAtTime(true)`
- `Curve#divide()` → `#divideAt(offset)` / ` #divideAtTime(time)`
- `Curve#split()` → `#splitAt(offset)` / `#splitAtTime(time)`
- `Curve#getParameterAt(offset)` → `#getTimeAt(offset)`
- `Curve#getParameterOf(point)` → `getTimeOf(point)`
- `Curve#getPointAt(time, true)` → `#getPointAtTime(time)`
- `Curve#getTangentAt(time, true)` → `#getTangentAtTime(time)`
- `Curve#getNormalAt(time, true)` → `#getNormalAtTime(time)`
- `Curve#getCurvatureAt(time, true)` → `#getCurvatureAtTime(time)`
- `CurveLocation#parameter` → `#time`
- `Path#split(offset/location)` → `#splitAt(offset/location)`
- Significant improvement of reliability of bezier fat-line clipping code in
`PathItem#getIntersections()` and `#getCrossings()`.
- `PathItem#smooth()` now accepts an `options.type` string specifying which
smoothing algorithm to use: `'asymmetric'` (default), `'continuous'`,
`'catmull-rom'`, and `'geometric'` (#338).
- `PathItem#flatten()`: argument has been changed from `tolerance` (maximum
allowed distance between points) to `flatness` (maximum allowed error) (#618).
- Update internal Acorn JavaScript parser to `0.5.0`, the last small version.
- Transition to Gulp based build process.
- Update QUnit to `1.20.0`.
- Update to JSDOM `8.3.0`, to benefit from integrated image and canvas support.
- Complete refactoring of keyboard event handling to increase reliably when
handling special keys.
- Complete refactoring of mouse-event handling on item and view, to better
handle event propagation, default behavior and `Item#removeOn()` calls.
- Simplify and streamline the mouse-handling code on `Tool` (#595).
- Mouse handlers can to return `false` to call `event.stop()`, stopping event
propagation and prevent the default browser behavior.
- `event.preventDefault()` is called by default after any handled mouse mouse
events, except `'mousemove'`, and only on a `'mousedown'` event if the view
or tool respond to `'mouseup'`.
- Switch to the new HTML5 Page Visibility API when detecting invisible documents
and canvases.
- Rename `#windingRule` to `#fillRule` on `Item` and `Style`.
- Do not replace existing named child reference on `Item#children` with new one
when the name is identical.
- Limit the effects of `#strokeScaling` to `PathItem` and `Shape` (#721).
- Throw an exception if arguments to `#smooth()` are segments or curves from
incorrect paths.
- Rename `Matrix#concatenate()` to `#append()` and `preConcatenate()` to
`#prepend()`.
- Make `Matrix#shiftless()` and `#orNullIfIdentity()` internal functions.
- De-bounce internal `View#update()` calls to minimize the number of times a
canvas is redrawn (#830, #925).
- `Symbol` now clashes with ES6 definition of Symbol and has been changed
(#770):
- `Symbol` → `SymbolDefinition`
- `PlacedSymbol` → `SymbolItem`
- `Symbol#definition` → `SymbolDefinition#item`
- `PlacedSymbol#symbol` → `SymbolItem#definition`
- Don't serialize deprecated `Style#font` property.
- Don't serialize text-styles in non-text items (#934).
- Changed argument `parameter` to `time` for Postscript-style drawing commands.
- `Item#clone()`: optional argument is now an options object with defaults
`{insert: true, deep: true}`. `insert` controls whether the clone is inserted
into the project and `deep` controls whether the item's children are cloned.
The previous boolean optional argument is still interpreted as the `insert`
option (#941).
- `Matrix` properties `#b` and `#c` have been reversed to match common standard.
- `#importSVG()`: improve handling of style inheritance for nested `<defs>`.
- Move `PaperScript#execute()` URL argument into `options.url` (#902).
- PaperScript: Only translate `==` to `equals() calls for `Point`, `Size` and
`Color` (#1043).
### Added
- Use unified code-base for browsers, Node.js, Electron, and anything
in-between, and enable npm install for browser use (#739).
- Start using automatic code testing and deployment of prebuilt versions through
Travis CI.
- Reach JSHint compliance and include regular linting in Travis CI tests.
- Use QUnit tests for leaked globals.
- Define code format standards in .editorconfig file
- Add support for running without a canvas for Web Workers, Node.js
(#561, #582, #634).
- Add support for all common mouse events to `View`.
- Add support for `'keydown'` and `'keyup'` events to `View` (#896).
- Add `View#requestUpdate()` function to minimize number of actual canvas
redraw.
- Add `View#matrix` to allow matrix transformation to be accessed and modified
directly on the view (#832).
- Multiple additions to SVG export (`#exportSVG()`):
- Support `{ precision: value }` option.
- Support `#fillRule` through the SVG `fill-rule` attribute.
- Support `#blendMode` through the CSS `mix-blend-mode` attribute.
- Various additions to `#getItems()` on `Project` and `Item`:
- Add support for `{ recursive: false }` as a way to prevent iterating over
all children of children.
- Add support for `{ match: function() {} }`, so the match function can be
passed in combination with other options.
- Add `Item#copyAttributes()` and `Item#copyContent()`, and use them in
`Item#clone()`.
- Add optional `insert` boolean argument to `Path#toShape()`, `Shape#toPath()`,
`Item#rasterize()`. Default is to insert, set to `false` to prevent the
created item from being inserted into the scene graph.
- Add visual item comparison to QUnit, through rasterization and Resemble.js
diffing.
- Add many unit tests for known edge cases in boolean operations and curve
intersections.
- Add `Project#addLayer()` and `Project#insertLayer()` (#903).
- Layers may now be given names and be accessed through `project.layers[name]`
(#491).
- Add `Matrix#prepended()` and `#appended()` to return copies of the modified
matrix.
- `Shape#hitTest()`: Add boolean option `options.stroke` (#911).
- Insert version number into docs.
- Support `Raster#onLoad()` events on `Raster#setImage()` now (#924).
- Add `Raster#onError()` event support (#849).
- Allow the control of automatic updating of the canvas through
`View#autoUpdate` (default: `true`)(#921).
- Set `1px` default `strokeWidth` for SVG imports to fix IE/Edge default (#467).
- `ImportSVG()` passes imported SVG data to `onLoad` callback as second
parameter.
- Add `#interpolate` for `Segment`, `Path`, and `CompoundPath` (#624).
- Implement `CompoundPath#flatten()`, `#simplify()`, `#smooth()` (#727).
- Implement `#hitTestAll()` to return all items that were hit (#536).
- `#importSVG()` implements `option.onError` callback (#969).
- `settings.insertItems` controls whether newly created items are inserted or
not (default: `true`).
- Add `#importSVG()` `option.insert` (default: `true`) to control insertion
(#763).
- Add new options to `#exportSVG()` to control output bounds and transformation
matrix (#972).
- Allow `Item#position` to be selected via `Item#position.selected` (#980).
- Add `tolerance` argument to `Path#join(path, tolerance)`.
- Add `Curve#getOffsetAtTime(time)`, as the reverse of
`Curve#getTimeAt(offset)`.
- Add `Raster#loaded` to reflect the loading state of its image.
### Fixed
- Fix calculations of `Item#strokeBounds` for all possible combinations of
`Item#strokeScaling` and `Item#applyMatrix` for `Path`, `Shape` and
`SymbolItem`, along with correct handling of such strokes in Item#hitTest()
(#697, #856, #1014).
- Make new code-base unified for Node.js/browser work with module bundlers like
Webpack (#986).
- Improve hit-testing and `#contains()` checks on path with horizontal lines
(#819).
- Improve reliability of `Path#getInteriorPoint()` in rare edge-cases.
- Handle non-reversible matrices in `Item#hitTest()` (#617).
- Fix various issues with adding and removing of segments in paths (#815).
- Support bubbling up of `doubleclick` events on `Group` (#834).
- Fix wrong `#key` values in key-events that do not match character (#881).
- Fix keyboard event handling of control and meta keyboard sequences and special
character handling (#860).
- Handle incorrect mouse event on `ctrl-alt-del` key sequence on Chrome/Windows
(#800).
- Do not rasterize items if the resulting raster will be empty (#828).
- Fix SVG serialization in JSDOM `7.0.0` and newer (#821).
- Correctly handle gradients in SVG import on Firefox (#666).
- Consistently interpret curves as straight or not-straight (#838).
- Switch blendMode to 'lighter' in Candy Crash example for better performance
(#453).
- Don't block touch actions when using paper in JavaScript mode (#686).
- Convert touch event coordinates to project coordinates (#633).
- Fix exceptions when a top-level layer is selected.
- Don't allow layers to turn up in hit-tests (#608).
- Maintain `Raster#source` correctly on Node.js (#914).
- Boolean operations correctly handle open `Path` items within `CompoundPath`
(#912).
- Don't modify an array of child items passed to `CompoundPath#insertChildren()`
when it is a child items array of a `CompoundPath`.
- Correctly handle `#strokeScaling` in `Shape` hit-tests (#697).
- Support clip-masks in hit-testing (#671).
- Fix incorrect `#hitTest()` and `#contains()` cases (#819, #884).
- Update documentation to note appropriate use for `#simplify()` (#920).
- `#importSVG()` now supports percentage dimensions and
`gradientUnits="objectBoundingBox"`. (#954, #650).
- `Group` items with clip-masks now calculate correct bounding boxes (#956).
- Calling `event.stopPropagation()` in `'mousedown'` handler no longer prevents
`'mousedrag'` events (#952).
- Draw `Item` shadows when `#shadowBlur` is zero (#955).
- Fixes for web site examples (#967).
- Prevent `Item` bounds from permanently collapsing to 0 when applying non-
invertible transformations (#558).
- Scaling shadows now works correctly with browser- and view-zoom (#831).
- `Path#arcTo()` correctly handles zero sizes.
- `#importSVG()` handles `onLoad` and `onError` callbacks for string inputs that
load external resources (#827).
- `#importJSON()` and `#exportJSON()` now handle non-`Item` objects correctly
(#392).
- `#exportSVG()` now exports empty paths if used as a clip-mask.
- `#importJSON()` no longer generates callstack exceeded exceptions (#764).
- Fix problems with group selection structures after `Group#importJSON()`
(#785).
- Fix an issue in `Item#importJSON()` where `#parent` is `null` when calling it
on existing, already inserted items (#1041).
- Correct issue when using paper-core in Node.js (#975).
- Fix `event.delta` on mousedrag events (#981).
- Improve handling of XML attribute namespaces for IE's XMLSerializer() (#984).
- Make sure `Item#removeChildren()` fully removes children (#991).
- Improve handling of event propagation on `View` and `Item` (#995).
- `#importSVG()`: Improve handling of viewBox.
- Make sure all named item lookup structures are kept in sync (#1009).
- Convert absolute local gradient URLs to relative ones (#1001).
- Fix TypeError in `Path#unite()` (#1000).
- Allow the selection of a `Path` item's bounds without selecting the segments
(#769).
- Fix wrong indices in `Item#insertChildren()`, when inserting children that
were previously inserted in the same parent (#1015).
- Add capability to `PathItem#closePath()` to handle imprecise SVG data due to
rounding (#1045).
- Improve reliability of fat-line clipping for curves that are very similar
(#904).
- Improve precision of `Numerical.solveQuadratic()` and
`Numerical.solveCubic()` for edge-cases (#1085).
### Removed
- Canvas attributes "resize" and "data-paper-resize" no longer cause paper to
resize the canvas when the viewport size changes; Additional CSS styles are
required since `0.9.22`, e.g.:
```css
/* Scale canvas with resize attribute to full size */
canvas[resize] {
width: 100%;
height: 100%;
}
```
- Legacy `Color` constructors (removed in `0.9.25`): `GrayColor`, `RgbColor`,
`HsbColor`, `HslColor`, and `GradientColor`. These have been replaced
with corresponding forms of the `Color` constructor.
- Undocumented function `Project#addChild()` that added a layer to a project.
It is replaced by `Project#addLayer()` and `Project#insertLayer()`.
### Deprecated
- `#windingRule` on `Item` and `Style` → `#fillRule`
- `Curve#getNormalAt(time, true)` → `#getNormalAtTime(true)`
- `Curve#divide()` → `#divideAt(offset)` / `#divideAtTime(time)`
- `Curve#split()` → `#splitAt(offset)` / `#splitAtTime(time)`
- `Curve#getParameterAt(offset)` → `#getTimeAt(offset)`
- `Curve#getParameterOf(point)` → `getTimeOf(point)`
- `Curve#getPointAt(time, true)` → `#getPointAtTime(time)`
- `Curve#getTangentAt(time, true)` → `#getTangentAtTime(time)`
- `Curve#getNormalAt(time, true)` → `#getNormalAtTime(time)`
- `Curve#getCurvatureAt(time, true)` → `#getCurvatureAtTime(time)`
- `CurveLocation#parameter` → `#time`
- `Path#split(offset/location)` → `#splitAt(offset/location)`
- `Symbol` → `SymbolDefinition`
- `PlacedSymbol` → `SymbolItem`
- `Symbol#definition` → `SymbolDefinition#item`
- `PlacedSymbol#symbol` → `SymbolItem#definition`
- `Project#symbols` → `#symbolDefinitions`
- `Matrix#concatenate` → `#append`
- `Matrix#preConcatenate` → `#prepend`
- `Matrix#chain` → `#appended`
- `GradientStop#rampPoint` → `#offset`
================================================
FILE: third_party/paper/LICENSE.txt
================================================
Copyright (c) 2011 - 2019, Juerg Lehni & Jonathan Puckey
http://scratchdisk.com/ & https://puckey.studio/
All rights reserved.
The MIT License (MIT)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
================================================
FILE: third_party/paper/README.md
================================================
# Paper.js - The Swiss Army Knife of Vector Graphics Scripting [](https://travis-ci.org/paperjs/paper.js) [](https://www.npmjs.com/package/paper)
If you want to work with Paper.js, simply download the latest "stable" version
from [http://paperjs.org/download/](http://paperjs.org/download/)
- Website: <http://paperjs.org/>
- Questions: <https://stackoverflow.com/questions/tagged/paperjs>
- Discussion forum: <https://groups.google.com/group/paperjs>
- Mainline source code: <https://github.com/paperjs/paper.js>
- Twitter: [@paperjs](https://twitter.com/paperjs)
- Latest releases: <http://paperjs.org/download/>
- Pre-built development versions:
[`prebuilt/module`](https://github.com/paperjs/paper.js/tree/prebuilt/module)
and [`prebuilt/dist`](https://github.com/paperjs/paper.js/tree/prebuilt/dist)
branches.
## Installing Paper.js
The recommended way to install and maintain Paper.js as a dependency in your
project is through the [Node.js Package Manager (NPM)](https://www.npmjs.com/)
for browsers, Node.js or Electron.
If NPM is already installed, simply type one of these commands in your project
folder:
```sh
npm install paper
```
Upon execution, you will find a `paper` folder inside the project's
`node_modules` folder.
For more information on how to install Node.js and NPM, read the chapter
[Installing Node.js and NPM](#installing-nodejs-and-npm).
### Which Version to Use
The various distributions come with two different pre-build versions of
Paper.js, in minified and normal variants:
- `paper-full.js` – The full version for the browser, including PaperScript
support and Acorn.js
- `paper-core.js` – The core version for the browser, without PaperScript
support nor Acorn.js. You can use this to shave off some bytes and compilation
time when working with JavaScript directly.
### Installing Node.js and NPM
Node.js comes with the Node Package Manager (NPM). There are many tutorials
explaining the different ways to install Node.js on different platforms. It is
generally not recommended to install Node.js through OS-supplied package
managers, as the its development cycles move fast and these versions are often
out-of-date.
On macOS, [Homebrew](https://brew.sh/) is a good option if one version of
Node.js that is kept up to date with `brew upgrade` is enough:
<https://treehouse.github.io/installation-guides/mac/node-mac.html>
[NVM](https://github.com/creationix/nvm) can be used instead to install and
maintain multiple versions of Node.js on the same platform, as often required by
different projects:
<https://nodesource.com/blog/installing-node-js-tutorial-using-nvm-on-mac-os-x-and-ubuntu/>
Homebrew is recommended on macOS also if you intend to install Paper.js with
rendering to the Canvas on Node.js, as described in the next paragraph.
For Linux, see <https://nodejs.org/download/> to locate 32-bit and 64-bit Node.js
binaries as well as sources, or use NVM, as described in the paragraph above.
### Installing Paper.js Using NPM
Paper.js comes in three different versions on NPM: `paper`, `paper-jsdom` and
`paper-jsdom-canvas`. Depending on your use case, you need to required a
different one:
- `paper` is the main library, and can be used directly in a browser
context, e.g. a web browser or worker.
- `paper-jsdom` is a shim module for Node.js, offering headless use with SVG
importing and exporting through [jsdom](https://github.com/tmpvar/jsdom).
- `paper-jsdom-canvas` is a shim module for Node.js, offering canvas rendering
through [Node-Canvas](https://github.com/Automattic/node-canvas) as well as
SVG importing and exporting through [jsdom](https://github.com/tmpvar/jsdom).
In order to install `paper-jsdom-canvas`, you need the [Cairo Graphics
library](https://cairographics.org/) installed in your system:
### Installing Native Dependencies
Paper.js relies on [Node-Canvas](https://github.com/Automattic/node-canvas) for
rendering, which in turn relies on the native libraries
[Cairo](https://cairographics.org/) and [Pango](https://www.pango.org/).
#### Installing Native Dependencies on macOS
Paper.js relies on Node-Canvas for rendering, which in turn relies on Cairo and
Pango. The easiest way to install Cairo is through
[Homebrew](https://brew.sh/), by issuing the command:
brew install cairo pango
Note that currently there is an issue on macOS with Cairo. If the above causes
errors, the following will most likely fix it:
PKG_CONFIG_PATH=/opt/X11/lib/pkgconfig/ npm install paper
Also, whenever you would like to update the modules, you will need to execute:
PKG_CONFIG_PATH=/opt/X11/lib/pkgconfig/ npm update
If you keep forgetting about this requirement, or would like to be able to type
simple and clean commands, add this to your `.bash_profile` file:
# PKG Config for Pango / Cairo
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:/opt/X11/lib/pkgconfig
After adding this line, your commands should work in the expected way:
npm install paper
npm update
#### Installing Native Dependencies on Debian/Ubuntu Linux
sudo apt-get install pkg-config libcairo2-dev libpango1.0-dev libssl-dev libjpeg62-dev libgif-dev
You might also need to install the build-essential package if you don't usually
build from c++ sources:
sudo apt-get install build-essential
#### Installing Native Dependencies for Electron
In order to build Node-Canvas for use of `paper-jsdom-canvas` in Electron, which
is likely to use a different version of V8 than the Node binary installed in
your system, you need to manually specify the location of Electron’s headers.
Follow these steps to do so:
[Electron — Using Native Node
Modules](https://electron.atom.io/docs/tutorial/using-native-node-modules/)
#### After Native Dependencies have been installed
You should now be able to install the Paper.js module with jsdom and Canvas
rendering from NPM:
npm install paper-jsdom-canvas
## Development
The main Paper.js source tree is hosted on
[GitHub](https://github.com/paperjs/paper.js/). `git` is required to create a
clone of the repository, and can be easily installed through your preferred
package manager on your platform.
### Get the Source
git clone --recursive git://github.com/paperjs/paper.js.git
cd paper.js
To refresh your clone and fetch changes from origin, run:
git fetch origin
To update the `jsdoc-toolkit` submodule, used to generate the documentation,
run:
git submodule update --init --recursive
### Setting Up For Building
As of 2016, Paper.js uses [Gulp.js](https://gulpjs.com/) for building, and has a
couple of dependencies as NPM modules. Read the chapter [Installing
Node.js and NPM](#installing-nodejs-and-npm) if you still need to
install these.
In order to be able to build Paper.js, after checking out the repository, paper
has dependencies that need to be installed. Install them by issuing the
following commands from the Paper.js directory:
npm install
It is also recommended to install Gulp.js globally, so you can easier execute
the build commands from anywhere in the command line:
npm install -g gulp
### Building the Library
The Paper.js sources are distributed across many separate files, organised in
subfolders inside the `src` folder. To compile them all into distributable
files, you can run the `build` task:
gulp build
You will then find the built library files inside the `dist` folder, named
`paper-full.js` and `paper-core.js`, along with their minified versions. Read
more about this in [Which Version to Use?](#which-version-to-use).
### Running Directly from Separate Source Files
As a handy alternative to building the library after each change to try it out
in your scripts, there is the `load` task, that replaces the built libraries
with symbolic links to the `scrc/load.js` script. This script then load the
library directly from all the separate source files in the `src` folder, through
the [Prepro.js](https://github.com/lehni/prepro.js) JavaScript preprocessing
library.
This means you can switch between loading from sources and loading a built
library simply by running.
gulp load
And to go back to a built library
gulp build
Note that your PaperScripts examples do not need to change, they can keep
loading `dist/paper-full.js`, which will always do the right thing. Note also
that `src/load.js` handles both browsers and Node.js, as supported by Prepro.js.
### Other Build Tasks
Create a final zipped distribution file inside the `dist` folder:
gulp dist
And since `dist` is the default task, this is the same:
gulp
### Branch structure
Since the release of version `0.9.22`, Paper.js has adopted aspects of the Git-
Flow workflow. All development is taking place in the
[`develop`](https://github.com/paperjs/paper.js/tree/develop) branch, which is
only merged into [`master`](https://github.com/paperjs/paper.js/tree/master)
when a new release occurs.
As of version `0.9.26`, the `dist` folder is excluded on all branches, and the
building is now part of the `npm publish` process by way of the `prepublish`
script.
We also offer prebuilt versions of the latest state of the `develop` branch on
[`prebuilt/module`](https://github.com/paperjs/paper.js/tree/prebuilt/module)
and [`prebuilt/dist`](https://github.com/paperjs/paper.js/tree/prebuilt/dist).
### Building the Documentation
Similarly to building the library, you can run the `docs` task to build the
documentation:
gulp docs
Your docs will then be located at `dist/docs`.
### Testing
Paper.js was developed and tested from day 1 using proper unit testing through
jQuery's [Qunit](https://qunitjs.com/). To run the tests after any
change to the library's source, simply open `index.html` inside the `test`
folder in your web browser. There should be a green bar at the top, meaning all
tests have passed. If the bar is red, some tests have not passed. These will be
highlighted and become visible when scrolling down.
If you are testing on Chrome, some of the tests will fail due to the browser's
CORS restrictions. In order to run the browser based tests on Chrome, you need
to run a local web-server through Gulp.js. The following command will handle it
for you, and will also open the browser at the right address straight away:
gulp test:browser
You can also run the unit tests through PhantomJS in Gulp directly on the
command line:
gulp test:phantom
To test the Node.js version of Paper.js, use this command:
gulp test:node
And to test both the PhantomJS and Node.js environments together, simply run:
gulp test
### Contributing [](https://www.codetriage.com/paperjs/paper.js)
The main Paper.js source tree is hosted on GitHub, thus you should create a fork
of the repository in which you perform development. See
<https://help.github.com/articles/fork-a-repo/>.
We prefer that you send a
[pull request on GitHub](https://help.github.com/articles/about-pull-requests/)
which will then be merged into the official main line repository.
You need to sign the Paper.js CLA to be able to contribute (see below).
Also, in your first contribution, add yourself to the end of `AUTHORS.md` (which
of course is optional).
In addition to contributing code you can also triage issues which may include
reproducing bug reports or asking for vital information, such as version numbers
or reproduction instructions. If you would like to start triaging issues, one
easy way to get started is to
[subscribe to paper.js on CodeTriage](https://www.codetriage.com/paperjs/paper.js).
**Get the source (for contributing):**
If you want to contribute to the project you will have to [make a
fork](https://help.github.com/articles/fork-a-repo/). Then do this:
git clone --recursive git@github.com:yourusername/paper.js.git
cd paper.js
git remote add upstream git://github.com/paperjs/paper.js.git
To then fetch changes from upstream, run
git fetch upstream
#### Creating and Submitting a Patch
As mentioned above, we prefer that you send a
[pull request](https://help.github.com/articles/about-pull-requests/) on GitHub:
1. Create a fork of the upstream repository by visiting
<https://github.com/paperjs/paper.js/fork>. If you feel insecure, here's a
great guide: <https://help.github.com/articles/fork-a-repo/>
2. Clone of your repository: `git clone
https://yourusername@github.com/yourusername/paper.js.git`
3. This is important: Create a so-called *topic branch* based on the `develop`
branch: `git checkout -tb name-of-my-patch develop` where `name-of-my-patch`
is a short but descriptive name of the patch you're about to create. Don't
worry about the perfect name though -- you can change this name at any time
later on.
4. Hack! Make your changes, additions, etc., commit them then push them to your
GitHub fork: `git push origin name-of-my-patch`
5. Send a pull request to the upstream repository's owner by visiting your
repository's site at GitHub (i.e. https://github.com/yourusername/paper.js)
and press the "Pull Request" button. Make sure you are creating the pull
request to the `develop` branch, not the `master` branch. Here's a good guide
on pull requests: <https://help.github.com/articles/about-pull-requests/>
##### Use one topic branch per feature:
Don't mix different kinds of patches in the same branch. Instead, merge them all
together into your `develop` branch (or develop everything in your `develop`
branch and then cherry-pick-and-merge into the different topic branches). Git
provides for an extremely flexible workflow, which in many ways causes more
confusion than it helps you when new to collaborative software development. The
guides provided by GitHub at <https://help.github.com/> are a really good
starting point and reference. If you are fixing an issue, a convenient way to
name the branch is to use the issue number as a prefix, like this: `git checkout
-tb issue-937-feature-add-text-styling`.
#### Contributor License Agreement
Before we can accept any contributions to Paper.js, you need to sign this
[CLA](https://en.wikipedia.org/wiki/Contributor_License_Agreement):
[Contributor License Agreement](https://spreadsheets.google.com/a/paperjs.org/spreadsheet/embeddedform?formkey=dENxd0JBVDY2REo3THVuRmh4YjdWRlE6MQ)
> The purpose of this agreement is to clearly define the terms under which
> intellectual property has been contributed to Paper.js and thereby allow us to
> defend the project should there be a legal dispute regarding the software at
> some future time.
For a list of authors and contributors, please see
[AUTHORS](https://github.com/paperjs/paper.js/blob/master/AUTHORS.md).
## License
Distributed under the MIT license. See
[LICENSE](https://github.com/paperjs/paper.js/blob/master/LICENSE.txt)
fo details.
================================================
FILE: third_party/paper/package.json
================================================
{
"name": "paper",
"version": "0.12.4",
"description": "The Swiss Army Knife of Vector Graphics Scripting",
"license": "MIT",
"homepage": "http://paperjs.org",
"repository": {
"type": "git",
"url": "https://github.com/paperjs/paper.js"
},
"bugs": "https://github.com/paperjs/paper.js/issues",
"contributors": ["Jürg Lehni <juerg@scratchdisk.com> (http://scratchdisk.com)", "Jonathan Puckey <jonathan@studiomoniker.com> (http://studiomoniker.com)"],
"main": "dist/paper-full.js",
"types": "dist/paper.d.ts",
"scripts": {
"build": "gulp build",
"dist": "gulp dist",
"zip": "gulp zip",
"docs": "gulp docs",
"load": "gulp load",
"jshint": "gulp jshint",
"test": "gulp test"
},
"files": ["AUTHORS.md", "CHANGELOG.md", "dist/", "examples/", "LICENSE.txt", "README.md"],
"engines": {
"node": ">=8.0.0"
},
"husky": {
"hooks": {
"pre-commit": "gulp jshint --ensure-branch develop",
"pre-push": "gulp test --ensure-branch develop"
}
},
"browser": {
"canvas": false,
"jsdom": false,
"jsdom/lib/jsdom/living/generated/utils": false,
"source-map-support": false,
"./dist/node/self.js": false,
"./dist/node/extend.js": false
},
"devDependencies": {
"acorn": "~0.5.0",
"canvas": "^2.4.1",
"del": "^4.1.0",
"gulp": "^3.9.1",
"gulp-cached": "^1.1.0",
"gulp-git-streamed": "^2.8.1",
"gulp-jshint": "^2.1.0",
"gulp-json-editor": "^2.5.2",
"gulp-prepro": "^2.4.0",
"gulp-qunits": "^2.1.2",
"gulp-rename": "^1.4.0",
"gulp-shell": "^0.7.0",
"gulp-symlink": "^2.1.4",
"gulp-uglify": "^1.5.4",
"gulp-uncomment": "^0.3.0",
"gulp-util": "^3.0.7",
"gulp-webserver": "^0.9.1",
"gulp-whitespace": "^0.1.0",
"gulp-zip": "^3.2.0",
"husky": "^2.3.0",
"jsdom": "^15.1.1",
"jshint": "^2.10.2",
"jshint-summary": "^0.4.0",
"merge-stream": "^2.0.0",
"minimist": "^1.2.0",
"mustache": "^3.0.1",
"prepro": "^2.4.0",
"qunitjs": "^1.23.0",
"require-dir": "^1.2.0",
"resemblejs": "^3.1.0",
"run-sequence": "^2.2.1",
"source-map-support": "^0.5.12",
"stats.js": "0.17.0",
"straps": "^3.0.1",
"typescript": "^3.1.6"
},
"keywords": ["vector", "graphic", "graphics", "2d", "geometry", "bezier", "curve", "curves", "path", "paths", "canvas", "svg", "paper", "paper.js", "paperjs"]
}
================================================
FILE: utils/colorUtils.js
================================================
/**
* @license
* Copyright 2020 Google Inc. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* =============================================================================
*/
import {MathUtils} from './mathUtils';
import * as paper from 'paper';
export class Palette {
constructor(colors) {
let scope = paper.default;
this.colors = colors.map(c => ({
light: new scope.Color(c[0]).convert('hsb'),
dark: new scope.Color(c[1]).convert('hsb'),
}));
}
select(variance) {
let scope = paper.default;
let pair = this.colors[Math.floor(Math.random() * this.colors.length)];
let varColor = (c) => new scope.Color({
hue: c.hue + 360 * MathUtils.gaussian(0, variance),
saturation: c.saturation + MathUtils.gaussian(0, variance),
brightness: c.brightness + MathUtils.gaussian(0, variance),
});
return {
light: varColor(pair.light),
dark: varColor(pair.dark),
};
}
}
export class ColorUtils {
static addRGB(color, red, green, blue) {
color.red = color.red + red;
color.green = color.green + green;
color.blue = color.blue + blue;
}
static lerp(color0, color1, amt) {
return new paper.default.Color(
MathUtils.lerp(color0.red, color1.red, amt),
MathUtils.lerp(color0.green, color1.green, amt),
MathUtils.lerp(color0.blue, color1.blue, amt),
);
}
// Generates random color from string hash.
static fromStringHash(str) {
// Compute hash from string
// Source http://werxltd.com/wp/2010/05/13/javascript-implementation-of-javas-string-hashcode-method/
var hash = 0, i, chr;
for (i = 0; i < str.length; i++) {
chr = str.charCodeAt(i);
hash = ((hash << 5) - hash) + chr;
hash |= 0; // Convert to 32bit integer
}
// Hash to rgb color.
let r = hash & 255;
let g = (hash & (255 << 8)) >> 8;
let b = (hash & (255 << 16)) >> 16;
return new paper.default.Color(r / 255, g / 255, b / 255);
}
}
================================================
FILE: utils/demoUtils.js
================================================
/**
* @license
* Copyright 2020 Google Inc. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* =============================================================================
*/
import * as posenet from '@tensorflow-models/posenet';
import * as tf from '@tensorflow/tfjs';
const color = 'aqua';
const boundingBoxColor = 'red';
const lineWidth = 2;
function isAndroid() {
return /Android/i.test(navigator.userAgent);
}
function isiOS() {
return /iPhone|iPad|iPod/i.test(navigator.userAgent);
}
export function isMobile() {
return isAndroid() || isiOS();
}
function setDatGuiPropertyCss(propertyText, liCssString, spanCssString = '') {
var spans = document.getElementsByClassName('property-name');
for (var i = 0; i < spans.length; i++) {
var text = spans[i].textContent || spans[i].innerText;
if (text == propertyText) {
spans[i].parentNode.parentNode.style = liCssString;
if (spanCssString !== '') {
spans[i].style = spanCssString;
}
}
}
}
/**
* Toggles between the loading UI and the main canvas UI.
*/
export function toggleLoadingUI(
showLoadingUI, loadingDivId = 'loading', mainDivId = 'main') {
if (showLoadingUI) {
document.getElementById(loadingDivId).style.display = 'block';
document.getElementById(mainDivId).style.display = 'none';
} else {
document.getElementById(loadingDivId).style.display = 'none';
document.getElementById(mainDivId).style.display = 'block';
}
}
function toTuple({y, x}) {
return [y, x];
}
export function drawPoint(ctx, y, x, r, color) {
ctx.beginPath();
ctx.arc(x, y, r, 0, 2 * Math.PI);
ctx.fillStyle = color;
ctx.fill();
}
/**
* Draws a line on a canvas, i.e. a joint
*/
export function drawSegment([ay, ax], [by, bx], color, scale, ctx) {
ctx.beginPath();
ctx.moveTo(ax * scale, ay * scale);
ctx.lineTo(bx * scale, by * scale);
ctx.lineWidth = lineWidth;
ctx.strokeStyle = color;
ctx.stroke();
}
/**
* Draws a pose skeleton by looking up all adjacent keypoints/joints
*/
export function drawSkeleton(keypoints, minConfidence, ctx, scale = 1) {
const adjacentKeyPoints =
posenet.getAdjacentKeyPoints(keypoints, minConfidence);
adjacentKeyPoints.forEach((keypoints) => {
drawSegment(
toTuple(keypoints[0].position), toTuple(keypoints[1].position), color,
scale, ctx);
});
}
/**
* Draw pose keypoints onto a canvas
*/
export function drawKeypoints(keypoints, minConfidence, ctx, scale = 1) {
for (let i = 0; i < keypoints.length; i++) {
const keypoint = keypoints[i];
if (keypoint.score < minConfidence) {
continue;
}
const {y, x} = keypoint.position;
drawPoint(ctx, y * scale, x * scale, 3, color);
}
}
/**
* Draw an image on a canvas
*/
export function renderImageToCanvas(image, size, canvas) {
canvas.width = size[0];
canvas.height = size[1];
const ctx = canvas.getContext('2d');
ctx.drawImage(image, 0, 0);
}
export function setStatusText(text) {
const resultElement = document.getElementById('status');
resultElement.innerText = text;
}
================================================
FILE: utils/fileUtils.js
================================================
/**
* @license
* Copyright 2020 Google Inc. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* =============================================================================
*/
export class FileUtils {
static setDragDropHandler(handler) {
window.addEventListener("dragover", function (e) {
e = e || event;
e.preventDefault();
}, false);
window.addEventListener("drop", function (e) {
e = e || event;
e.preventDefault();
if (e.dataTransfer.items) {
let files = e.dataTransfer.items;
if (files.length < 1) {
return;
}
let reader = new FileReader();
reader.onload = (event) => {
handler(event.target.result);
}
reader.readAsText(e.dataTransfer.files[0]);
}
}, false);
}
}
================================================
FILE: utils/mathUtils.js
================================================
/**
* @license
* Copyright 2020 Google Inc. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* =============================================================================
*/
function getDistance(p0, p1) {
return Math.sqrt((p0.x - p1.x) * (p0.x - p1.x) + (p0.y - p1.y) * (p0.y - p1.y));
}
export class MathUtils {
static lerp(v0, v1, perc) {
return v0 + (v1 - v0) * perc;
}
static random(v0, v1) {
return v0 + Math.random() * (v1 - v0);
}
static smoothStep(v, min, max) {
var x = Math.max(0, Math.min(1, (v-min)/(max-min)));
return x*x*(3 - 2*x);
}
// Generate a transform function of p in the coordinate system defined by p0 and p1.
static getTransformFunc(p0, p1, p) {
let d = p1.subtract(p0);
let dir = d.normalize();
let l0 = d.length;
let n = dir.clone();
n.angle += 90;
let v = p.subtract(p0);
let x = v.dot(dir);
let y = v.dot(n);
return (p0New, p1New) => {
let d = p1New.subtract(p0New);
if (d.length === 0) {
return p0New.clone();
}
let scale = d.length / l0;
let dirNew = d.normalize();
let nNew = dirNew.clone();
nNew.angle += 90;
return p0New.add(dirNew.multiply(x * scale)).add(nNew.multiply(y * scale));
}
}
static getClosestPointOnSegment(p0, p1, p) {
let d = p1.subtract(p0);
let c = p.subtract(p0).dot(d) / (d.dot(d));
if (c >= 1) {
return p1.clone();
} else if (c <= 0) {
return p0.clone();
} else {
return p0.add(d.multiply(c));
}
}
// Check if v0 and v1 are collinear.
// Returns true if cosine of the angle between v0 and v1 is within threshold to 1.
static isCollinear(v0, v1, threshold = 0.01) {
let colinear = false;
if (v0 && v1) {
let n0 = v0.normalize();
let n1 = v1.normalize();
colinear = Math.abs(n0.dot(n1)) > 1 - threshold;
}
return colinear;
}
static gaussian(mean, variance) {
var u = 0, v = 0;
while(u === 0) u = Math.random(); //Converting [0,1) to (0,1)
while(v === 0) v = Math.random();
let value = Math.sqrt( -2.0 * Math.log( u ) ) * Math.cos( 2.0 * Math.PI * v );
return value * variance + mean;
}
static clamp(v, minV, maxV) {
return Math.min(Math.max(v, minV), maxV);
}
static selectSegments(selectPerc, count, selectVar, segVar) {
let segments = [];
let totalSeg = 0;
for (let i = 0; i < count; i++) {
let seg = MathUtils.gaussian(1, segVar);
segments.push(seg);
totalSeg += seg;
}
for (let i = 0; i < segments.length; i++) {
segments[i] = segments[i] / totalSeg;
}
let cursor = 0;
let selected = [];
for (let i = 0; i < count; i++) {
let s0 = cursor;
let s1 = cursor + segments[i] * MathUtils.clamp(MathUtils.gaussian(1, selectVar) * selectPerc, 0, 1);
selected.push([s0, s1]);
cursor += segments[i];
}
return selected;
}
static isLeft(p0, p1, p){
return ((p1.x - p0.x)*(p.y - p0.y) - (p1.y - p0.y)*(p.x - p0.x)) > 0;
}
static packCircles(center, radius, seedCount, maxR, minR, maxIter = 10) {
let circles = [];
let iterCount = 0;
while (circles.length < seedCount && iterCount < maxIter) {
while (circles.length < seedCount) {
let c = {
x: radius * (Math.random() * 2 - 1) + center.x,
y: radius * (Math.random() * 2 - 1) + center.y,
};
if (getDistance(c, center) > radius || circles.some(circle => getDistance(circle, c) < circle.radius)) continue;
circles.push({
c: c,
r: 0,
});
}
let growthIterCount = 20;
let intersects = (c0, c1) => {
let d = getDistance(c0.c, c1.c);
return (d < c0.r + c1.r) && (d > Math.abs(c0.r - c1.r));
}
let bound = {
c: center,
r: radius,
};
for (let i = 0; i < growthIterCount; i++) {
let grew = false;
circles.forEach(s => {
let intersecting = circles.some(other => (s !== other) && (intersects(s, other) || intersects(s, bound)));
if (!intersecting && s.r < maxR) {
s.r += maxR / growthIterCount;
grew = true;
}
});
if (!grew) break;
}
circles = circles.filter(c => c.r >= minR);
iterCount++;
}
return circles;
}
}
class KeySpline {
constructor(mX1, mY1, mX2, mY2) {
this.mX1 = mX1;
this.mY1 = mY1;
this.mX2 = mX2;
this.mY2 = mY2;
}
get(aX) {
if (this.mX1 == this.mY1 && this.mX2 == this.mY2) return aX; // linear
return this.CalcBezier(this.GetTForX(aX), this.mY1, this.mY2);
}
A( aA1, aA2) { return 1.0 - 3.0 * aA2 + 3.0 * aA1; }
B( aA1, aA2) { return 3.0 * aA2 - 6.0 * aA1; }
C( aA1) { return 3.0 * aA1; }
// Returns x(t) given t, x1, and x2, or y(t) given t, y1, and y2.
CalcBezier( aT, aA1, aA2) {
return ((this.A(aA1, aA2) * aT + this.B(aA1, aA2)) * aT + this.C(aA1)) * aT;
}
// Returns dx/dt given t, x1, and x2, or dy/dt given t, y1, and y2.
GetSlope( aT, aA1, aA2) {
return 3.0 * this.A(aA1, aA2) * aT * aT + 2.0 * this.B(aA1, aA2) * aT + this.C(aA1);
}
GetTForX( aX) {
// Newton raphson iteration
let aGuessT = aX;
for (let i = 0; i < 4; ++i) {
let currentSlope = this.GetSlope(aGuessT, this.mX1, this.mX2);
if (currentSlope == 0.0) return aGuessT;
let currentX = this.CalcBezier(aGuessT, this.mX1, this.mX2) - aX;
aGuessT -= currentX / currentSlope;
}
return aGuessT;
}
};
export class MultiSpline {
constructor() {
this.keySplines = [];
this.segments = [];
this.x0 = 0;
this.y0 = 0;
}
add(mX1, mY1, mX2, mY2, x1, y1) {
let ks = new KeySpline(mX1, mY1, mX2, mY2);
let x0 = this.x0;
let y0 = this.y0;
if (this.segments.length) {
x0 = this.segments[this.segments.length - 1][1].x;
y0 = this.segments[this.segments.length - 1][1].y;
}
this.keySplines.push(ks);
this.segments.push([{x: x0, y: y0}, {x: x1, y: y1}]);
}
get(x) {
let index = -1;
for (let i = 0; i < this.segments.length; i++) {
if (x >= this.segments[i][0].x && x < this.segments[i][1].x) {
index = i;
break;
}
}
if (index < 0) {
return 0;
}
let seg = this.segments[index];
let ks = this.keySplines[index];
let perc = (x - seg[0].x) / (seg[1].x - seg[0].x);
if (index % 2 == 0) {
return MathUtils.lerp(seg[0].y, seg[1].y, ks.get(perc));
} else {
return MathUtils.lerp(seg[1].y, seg[0].y, ks.get(1 - perc));
}
}
}
================================================
FILE: utils/svgUtils.js
================================================
/**
* @license
* Copyright 2020 Google Inc. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* =============================================================================
*/
import * as paper from 'paper';
export class SVGUtils {
static importSVG(file) {
let svgScope = new paper.default.PaperScope();
let canvas = svgScope.createCanvas(0, 0);
svgScope.setup(canvas);
return new Promise((resolve, reject) => {
svgScope.project.importSVG(file, () => {
console.log('** SVG imported **');
resolve(svgScope);
}, (e) => {
console.log('** SVG improt error: ', e);
reject(svgScope);
});
})
}
static drawEllipse(p, va, vb, ctrlDA, ctrlDB, scope, options) {
let va1 = va.multiply(-1);
let vb1 = vb.multiply(-1);
let p0 = p.add(va);
let p1 = p.add(vb);
let p2 = p.add(va1);
let p3 = p.add(vb1);
let path = new scope.Path(options);
path.addSegment(p0, vb1.normalize().multiply(ctrlDB), vb.normalize().multiply(ctrlDB));
path.addSegment(p1, va.normalize().multiply(ctrlDA), va1.normalize().multiply(ctrlDA));
path.addSegment(p2, vb.normalize().multiply(ctrlDB), vb1.normalize().multiply(ctrlDB));
path.addSegment(p3, va1.normalize().multiply(ctrlDA), va.normalize().multiply(ctrlDA));
path.closePath();
return path;
}
static genPathWithSpline(path, spline, height, options, scope) {
let pathLen = path.length;
if (pathLen == 0) {
return path.clone();
}
let to = [];
let back = [];;
let segCount = Math.max(pathLen / 3, 1.0);
for (let i = 0; i < segCount; i++) {
let perc = i / (segCount - 1);
let p = path.getPointAt(perc * pathLen);
let n = path.getNormalAt(perc * pathLen);
let easeHeight = spline.get(perc);
if (!p || !n) continue;
let pp0 = p.add(n.multiply(height * easeHeight));
let pp1 = p.subtract(n.multiply(height * easeHeight));
to.push(pp0);
back.unshift(pp1);
}
let outPath = new scope.Path(options);
outPath.addSegments(to.concat(back));
outPath.simplify();
return outPath;
}
static isPath(item) {
return item.constructor === item.project._scope.Path;
}
static isShape(item) {
return item.constructor === item.project._scope.Shape;
}
static isGroup(item) {
return item.constructor === item.project._scope.Group;
}
static findFirstItemWithPrefix(root, prefix) {
let items = root.getItems({ recursive: true });
for (let i = 0; i < items.length; i++) {
if (items[i].name && items[i].name.startsWith(prefix)) {
return items[i];
}
}
return null;
}
}
gitextract_rc75g8yk/
├── .gitignore
├── CONTRIBUTING.md
├── LICENSE
├── README.md
├── camera.html
├── camera.js
├── illustrationGen/
│ ├── illustration.js
│ └── skeleton.js
├── index.html
├── package.json
├── static_image.html
├── static_image.js
├── third_party/
│ └── paper/
│ ├── AUTHORS.md
│ ├── CHANGELOG.md
│ ├── LICENSE.txt
│ ├── README.md
│ └── package.json
└── utils/
├── colorUtils.js
├── demoUtils.js
├── fileUtils.js
├── mathUtils.js
└── svgUtils.js
SYMBOL INDEX (114 symbols across 9 files)
FILE: camera.js
function setupCamera (line 72) | async function setupCamera() {
function loadVideo (line 99) | async function loadVideo() {
function setupGui (line 123) | function setupGui(cameras) {
function setupFPS (line 144) | function setupFPS() {
function detectPoseInRealTime (line 153) | function detectPoseInRealTime(video) {
function setupCanvas (line 240) | function setupCanvas() {
function bindPage (line 260) | async function bindPage() {
function parseSVG (line 301) | async function parseSVG(target) {
FILE: illustrationGen/illustration.js
constant MIN_CONFIDENCE_PATH_SCORE (line 26) | const MIN_CONFIDENCE_PATH_SCORE = 0.3;
class PoseIllustration (line 29) | class PoseIllustration {
method constructor (line 30) | constructor(scope) {
method updateSkeleton (line 35) | updateSkeleton(pose, face) {
method draw (line 68) | draw() {
method debugDraw (line 97) | debugDraw() {
method debugDrawLabel (line 141) | debugDrawLabel(scope) {
method bindSkeleton (line 145) | bindSkeleton(skeleton, skeletonScope) {
method bindGroup (line 164) | bindGroup(group, skeleton) {
method getWeights (line 208) | getWeights(point, bones) {
method bindPathToBones (line 246) | bindPathToBones(path, selectedBones) {
method getSkinning (line 278) | getSkinning(point, weights) {
FILE: illustrationGen/skeleton.js
constant MIN_POSE_SCORE (line 23) | const MIN_POSE_SCORE = 0.1;
constant MIN_FACE_SCORE (line 24) | const MIN_FACE_SCORE = 0.8;
class Bone (line 126) | class Bone {
method set (line 127) | set(kp0, kp1, skeleton, type) {
method getPointTransform (line 141) | getPointTransform(p) {
method transform (line 158) | transform(trans) {
function getKeyPointFromSVG (line 174) | function getKeyPointFromSVG(group, partName) {
function getPartFromPose (line 182) | function getPartFromPose(pose, name) {
function getKeypointFromFaceFrame (line 193) | function getKeypointFromFaceFrame(face, i) {
class Skeleton (line 199) | class Skeleton {
method constructor (line 200) | constructor(scope) {
method update (line 496) | update(pose, face) {
method updatePoseParts (line 531) | updatePoseParts(pose) {
method updateFaceParts (line 550) | updateFaceParts(face) {
method findBoneGroup (line 587) | findBoneGroup(point) {
method getTotalBoneLength (line 610) | getTotalBoneLength(bones) {
method debugDraw (line 619) | debugDraw(scope) {
method debugDrawLabels (line 640) | debugDrawLabels(scope) {
method reset (line 658) | reset() {
method getCurrentPosition (line 662) | static getCurrentPosition(segment) {
method flipPose (line 671) | static flipPose(pose) {
method flipFace (line 681) | static flipFace(face) {
method getBoundingBox (line 692) | static getBoundingBox(pose) {
method translatePose (line 715) | static translatePose(pose, d) {
method resizePose (line 729) | static resizePose(pose, origin, scale) {
method toFaceFrame (line 743) | static toFaceFrame(faceDetection) {
FILE: static_image.js
constant VIDEO_WIDTH (line 76) | const VIDEO_WIDTH = 513;
constant VIDEO_HEIGHT (line 77) | const VIDEO_HEIGHT = 513;
constant CANVAS_WIDTH (line 79) | const CANVAS_WIDTH = 513;
constant CANVAS_HEIGHT (line 80) | const CANVAS_HEIGHT = 513;
function drawResults (line 99) | function drawResults(image, canvas, faceDetection, poses) {
function loadImage (line 123) | async function loadImage(imagePath) {
function multiPersonCanvas (line 136) | function multiPersonCanvas() {
function getIllustrationCanvas (line 140) | function getIllustrationCanvas() {
function drawDetectionResults (line 147) | function drawDetectionResults() {
function testImageAndEstimatePoses (line 177) | async function testImageAndEstimatePoses() {
function setupGui (line 219) | function setupGui() {
function bindPage (line 239) | async function bindPage() {
function loadSVG (line 266) | async function loadSVG(target) {
FILE: utils/colorUtils.js
class Palette (line 21) | class Palette {
method constructor (line 22) | constructor(colors) {
method select (line 30) | select(variance) {
class ColorUtils (line 45) | class ColorUtils {
method addRGB (line 46) | static addRGB(color, red, green, blue) {
method lerp (line 52) | static lerp(color0, color1, amt) {
method fromStringHash (line 61) | static fromStringHash(str) {
FILE: utils/demoUtils.js
function isAndroid (line 25) | function isAndroid() {
function isiOS (line 29) | function isiOS() {
function isMobile (line 33) | function isMobile() {
function setDatGuiPropertyCss (line 37) | function setDatGuiPropertyCss(propertyText, liCssString, spanCssString =...
function toggleLoadingUI (line 53) | function toggleLoadingUI(
function toTuple (line 64) | function toTuple({y, x}) {
function drawPoint (line 68) | function drawPoint(ctx, y, x, r, color) {
function drawSegment (line 78) | function drawSegment([ay, ax], [by, bx], color, scale, ctx) {
function drawSkeleton (line 90) | function drawSkeleton(keypoints, minConfidence, ctx, scale = 1) {
function drawKeypoints (line 104) | function drawKeypoints(keypoints, minConfidence, ctx, scale = 1) {
function renderImageToCanvas (line 120) | function renderImageToCanvas(image, size, canvas) {
function setStatusText (line 128) | function setStatusText(text) {
FILE: utils/fileUtils.js
class FileUtils (line 18) | class FileUtils {
method setDragDropHandler (line 19) | static setDragDropHandler(handler) {
FILE: utils/mathUtils.js
function getDistance (line 18) | function getDistance(p0, p1) {
class MathUtils (line 22) | class MathUtils {
method lerp (line 23) | static lerp(v0, v1, perc) {
method random (line 27) | static random(v0, v1) {
method smoothStep (line 31) | static smoothStep(v, min, max) {
method getTransformFunc (line 37) | static getTransformFunc(p0, p1, p) {
method getClosestPointOnSegment (line 59) | static getClosestPointOnSegment(p0, p1, p) {
method isCollinear (line 73) | static isCollinear(v0, v1, threshold = 0.01) {
method gaussian (line 83) | static gaussian(mean, variance) {
method clamp (line 91) | static clamp(v, minV, maxV) {
method selectSegments (line 95) | static selectSegments(selectPerc, count, selectVar, segVar) {
method isLeft (line 117) | static isLeft(p0, p1, p){
method packCircles (line 121) | static packCircles(center, radius, seedCount, maxR, minR, maxIter = 10) {
class KeySpline (line 163) | class KeySpline {
method constructor (line 164) | constructor(mX1, mY1, mX2, mY2) {
method get (line 171) | get(aX) {
method A (line 176) | A( aA1, aA2) { return 1.0 - 3.0 * aA2 + 3.0 * aA1; }
method B (line 177) | B( aA1, aA2) { return 3.0 * aA2 - 6.0 * aA1; }
method C (line 178) | C( aA1) { return 3.0 * aA1; }
method CalcBezier (line 181) | CalcBezier( aT, aA1, aA2) {
method GetSlope (line 186) | GetSlope( aT, aA1, aA2) {
method GetTForX (line 190) | GetTForX( aX) {
class MultiSpline (line 203) | class MultiSpline {
method constructor (line 204) | constructor() {
method add (line 211) | add(mX1, mY1, mX2, mY2, x1, y1) {
method get (line 223) | get(x) {
FILE: utils/svgUtils.js
class SVGUtils (line 20) | class SVGUtils {
method importSVG (line 21) | static importSVG(file) {
method drawEllipse (line 36) | static drawEllipse(p, va, vb, ctrlDA, ctrlDB, scope, options) {
method genPathWithSpline (line 52) | static genPathWithSpline(path, spline, height, options, scope) {
method isPath (line 78) | static isPath(item) {
method isShape (line 82) | static isShape(item) {
method isGroup (line 86) | static isGroup(item) {
method findFirstItemWithPrefix (line 90) | static findFirstItemWithPrefix(root, prefix) {
Condensed preview — 22 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (167K chars).
[
{
"path": ".gitignore",
"chars": 1256,
"preview": ".DS_Store\n.cache\n# Firebase deployment config files\n.firebaserc\nfirebase.json\n# Yarn build output folder\ndist/\n\n# Logs\nl"
},
{
"path": "CONTRIBUTING.md",
"chars": 1102,
"preview": "# How to Contribute\n\nWe'd love to accept your patches and contributions to this project. There are\njust a few small guid"
},
{
"path": "LICENSE",
"chars": 11357,
"preview": "\n Apache License\n Version 2.0, January 2004\n "
},
{
"path": "README.md",
"chars": 3741,
"preview": "# Pose Animator\n\nPose Animator takes a 2D vector illustration and animates its containing curves in real-time based on t"
},
{
"path": "camera.html",
"chars": 3941,
"preview": "<!DOCTYPE html>\n<html>\n\n<head>\n <title>PoseNet - Camera Feed Demo</title>\n <style>\n body {\n disp"
},
{
"path": "camera.js",
"chars": 8924,
"preview": "/**\n * @license\n * Copyright 2020 Google Inc. All Rights Reserved.\n * Licensed under the Apache License, Version 2.0 (th"
},
{
"path": "illustrationGen/illustration.js",
"chars": 11855,
"preview": "/**\n * @license\n * Copyright 2020 Google Inc. All Rights Reserved.\n * Licensed under the Apache License, Version 2.0 (th"
},
{
"path": "illustrationGen/skeleton.js",
"chars": 36268,
"preview": "/**\n * @license\n * Copyright 2020 Google Inc. All Rights Reserved.\n * Licensed under the Apache License, Version 2.0 (th"
},
{
"path": "index.html",
"chars": 318,
"preview": "<!DOCTYPE html>\n<html>\n <head>\n <title>Pose Animator Demos</title>\n </head>\n <body>\n <h1>Pose Animator Demos</h"
},
{
"path": "package.json",
"chars": 1578,
"preview": "{\n \"name\": \"tfjs-models\",\n \"version\": \"0.0.1\",\n \"description\": \"\",\n \"main\": \"index.js\",\n \"license\": \"Apache-2.0\",\n "
},
{
"path": "static_image.html",
"chars": 2995,
"preview": "<!DOCTYPE html>\n<html>\n\n<head>\n <title>PoseNet - Coco Images Demo</title>\n <style type='text/css'>\n .illustration-c"
},
{
"path": "static_image.js",
"chars": 8344,
"preview": "/**\n * @license\n * Copyright 2020 Google Inc. All Rights Reserved.\n * Licensed under the Apache License, Version 2.0 (th"
},
{
"path": "third_party/paper/AUTHORS.md",
"chars": 544,
"preview": "## Authors\n\n- Jürg Lehni <juerg@scratchdisk.com>\n- Jonathan Puckey <jonathan@studiomoniker.com>\n\n## Contributors\n\n- Hari"
},
{
"path": "third_party/paper/CHANGELOG.md",
"chars": 31002,
"preview": "# Change Log\n\n## `0.12.4`\n\n### Added\n\n- Allow paper core import in TypeScript (#1713).\n- Boolean: Improve performance fr"
},
{
"path": "third_party/paper/LICENSE.txt",
"chars": 1174,
"preview": "Copyright (c) 2011 - 2019, Juerg Lehni & Jonathan Puckey\nhttp://scratchdisk.com/ & https://puckey.studio/\nAll rights res"
},
{
"path": "third_party/paper/README.md",
"chars": 15020,
"preview": "# Paper.js - The Swiss Army Knife of Vector Graphics Scripting [![Build Status](https://travis-ci.org/paperjs/paper.js.s"
},
{
"path": "third_party/paper/package.json",
"chars": 2407,
"preview": "{\n \"name\": \"paper\",\n \"version\": \"0.12.4\",\n \"description\": \"The Swiss Army Knife of Vector Graphics Scripting\",\n \"lic"
},
{
"path": "utils/colorUtils.js",
"chars": 2685,
"preview": "/**\n * @license\n * Copyright 2020 Google Inc. All Rights Reserved.\n * Licensed under the Apache License, Version 2.0 (th"
},
{
"path": "utils/demoUtils.js",
"chars": 3603,
"preview": "/**\n * @license\n * Copyright 2020 Google Inc. All Rights Reserved.\n * Licensed under the Apache License, Version 2.0 (th"
},
{
"path": "utils/fileUtils.js",
"chars": 1445,
"preview": "/**\n * @license\n * Copyright 2020 Google Inc. All Rights Reserved.\n * Licensed under the Apache License, Version 2.0 (th"
},
{
"path": "utils/mathUtils.js",
"chars": 8053,
"preview": "/**\n * @license\n * Copyright 2020 Google Inc. All Rights Reserved.\n * Licensed under the Apache License, Version 2.0 (th"
},
{
"path": "utils/svgUtils.js",
"chars": 3498,
"preview": "/**\n * @license\n * Copyright 2020 Google Inc. All Rights Reserved.\n * Licensed under the Apache License, Version 2.0 (th"
}
]
About this extraction
This page contains the full source code of the yemount/pose-animator GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 22 files (157.3 KB), approximately 40.9k tokens, and a symbol index with 114 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.