Video Generation/Modification (Vfx) From Text

This article explores cutting-edge video generation models, such as OpenAI’s Sora and Text2Video-Zero by Damo Academy. It also provides useful resources and recommendations for creating high-quality videos, including selecting the right content, utilizing smooth transitions, and incorporating subtitles.

Sora is the new SOTA video generation model from OpenAI.

Following up projects:

达摩院放出了文本生成视频模型，支持英文输入

huggingface space

model weights:

weight path weight size model name author
text-to-video-ms-1.7b unknown unknown damo-vilab
modelscope-damo-text-to-video-synthesis unknown unknown damo-vilab
text-to-video-ms-1.7b-legacy unknown unknown damo-vilab

can also use from modelscope:

1
2
3
4
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks
p = pipeline('text-to-video-synthesis', 'damo/text-to-video-synthesis')

PAIR now releases Text2Video-Zero which leverages existing stable diffusion models to generate video. also released a bunch of controlnet dreambooth weights.

lucidrains is a workaholic on transformer implementations. we should scrape all the repos and index them. there are faster language models to train.

Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 minutes in length, in Pytorch

dreamix (not open-source)

instruct-pix2pix requires 16GB+ VRAM

text2live modify video by text prompt (such as add fire in mouth)

recurrent-interface-network-pytorch using diffusion to generate images and video

high quality! imagegen-video code with demo and paper

抄视频 视频的时间要讲究 看看是抄一年前的好还是抄刚刚发布的好

在发布的一个视频当中 最多抄某个作者的两三个符合要求的片段

use editly smooth/slick transitions and subtitles to beat the copy-detection algorithm, also consider color change in ffmpeg

动态 专栏也可以抄

make-a-video

谷歌AI歌手震撼来袭！AudioLM简单听几秒，便能谱曲写歌 https://www.kuxai.com/article/398

Comments

▲