2022-09-06

Metahuman Makehuman Blender Lipsync 动捕视频转动画人物模型捏脸 Vroid模型 Vrm

3d models, live2d models and model makers

makehuman as model maker

avatar sample e a cute girl

live2d models from facerig

blender adapters

blender vrm importer, exporter and utilities

blender script: vrm to ue4 compatible

blender’s ‘make it pretty’ button for vrm models

makehuman plugin for blender

blender addon for rhubarb lip sync

BlendArMocap by cgtinker is a Blender add-on to preform Hand, Face and Pose Detection in Blender using just a Webcam built upon Google’s Mediapipe. The detected data can be easily transferred to rifigy rigs.

lipsync libraries

rhubarb lip sync is a command-line tool that automatically creates 2D mouth animation from voice recordings

wav2lip

mocap libraries

openpose is the first real-time multi-person system to jointly detect human body, hand, facial, and foot key-points

FrankMocap A Strong and Easy-to-use Single View 3D Hand+Body Pose Estimator

EasyMocap is an open-source toolbox for markerless human motion capture.

freemocap and its FAQ wiki

FreeMoCap on pre-recorded videos:

Start the RunMe() pipeline at Stage 2, and specify the folder containing the videos you wish to process.

PARE: Part Attention Regressor for 3D Human Body Estimation

openseeface: face landmark tracking

the still image to singing face bot, lip-sync video generation

sadtalker

wombo.ai, likely to be talking head or yanderifier

https://github.com/mchong6/GANsNRoses/

https://github.com/williamyang1991/VToonify

生成高质量的艺术人像视频是计算机图形学和视觉中一项重要且理想的任务。虽然已经提出了一系列基于强大的 StyleGAN 成功的人像图像卡通化模型，但这些面向图像的方法在应用于视频时存在明显的局限性，在这项工作中，我们通过引入一种新颖的 VToonify 框架来研究具有挑战性的可控高分辨率肖像视频风格迁移。具体来说，VToonify 利用StyleGAN 的中高分辨率层基于编码器提取的多尺度内容特征来渲染高质量的艺术肖像，以更好地保留帧细节。作为输入，有助于输出具有自然运动的完整面部区域。 amework 与现有的基于 StyleGAN 的图像卡通化模型兼容，以将其扩展到视频卡通化，并继承了这些模型的吸引人的特性，可灵活地控制颜色和强度。这项工作展示了基于 Toonify 和 DualStyleGAN 的 VToonify 的两个实例，用于基于集合广泛的实验结果证明了我们提出的 VToonify 框架在生成具有灵活风格控制的高质量和时间连贯的艺术肖像视频方面优于现有方法的有效性

all in one colab text to talking face generation, also consider paddlespeech example:

https://github.com/ChintanTrivedi/ask-fake-ai-karen

avaliable from paddlegan as an example used in paddlespeech, the artificial host.

lip-sync accurate wav2lip:

https://github.com/Rudrabha/Wav2Lip

lipgan generate realistic lip-sync talking head animation(fully_pythonic branch or google colab notebook):

https://github.com/Rudrabha/LipGAN

google’s lipsync implementation, using tensorflow facemesh:

https://github.com/google/lipsync

https://lipsync.withyoutube.com/

https://github.com/tensorflow/tfjs-models/tree/master/facemesh

network reverse engineering for wombo.ai: