Pose Animator: Real-time SVG Puppeteering with TensorFlow.js

Pose Animator uses TensorFlow.js PoseNet and FaceMesh to animate SVG characters in real time via a full-body skeletal rig. It deforms vector paths with Linear Blend Skinning, carefully handling Bezier curves, and stabilizes motion using confidence-weighted smoothing and clipping. The project is open source with demos available and plans for mesh improvements, weight painting, and raster support.

Key Points

• Pose Animator brings SVG characters to life in the browser using TensorFlow.js PoseNet and FaceMesh for real-time motion capture.
• A predefined rig (90 keypoints, 78 bones) is embedded in the input SVG; Linear Blend Skinning deforms actual vector paths, not just bounding boxes.
• Bezier curves receive special treatment with separate weights for anchors and control points, plus a collinearity safeguard to preserve curve smoothness.
• Confidence-weighted temporal smoothing and threshold-based clipping reduce jitter and hide low-confidence (often off-screen) body parts.
• Future plans include mesh-based rigging improvements beyond LBS, weight painting tools, and support for raster images within SVGs.

Sentiment

The Hacker News community acknowledges the technical achievement but is largely unimpressed, viewing the tool as a less capable alternative to existing vtuber and animation technology. Criticism of the demo's corporate art style overshadows appreciation for the underlying engineering.

In Agreement

• TensorFlow.js makes real-time body-tracking animation accessible and fun to build, inspiring creative projects like body-controlled games for kids
• The concept has enduring appeal, with historical roots in interactive camera-driven animation dating back decades

Opposed

• The results look uncanny and aesthetically strange compared to proper animation tools
• This is essentially a lower-quality version of established vtuber technology like Live2D
• The project is from 2020 and not novel; similar and arguably better tools already exist
• The corporate 'Alegria' art style used in demos is widely disliked