Video Generation/Modification (Vfx) From Text
Sora is the new SOTA video generation model from OpenAI.
Following up projects:
达摩院放出了文本生成视频模型，支持英文输入
model weights:
| weight path | weight size | model name | author |
|---|---|---|---|
| text-to-video-ms-1.7b | unknown | unknown | damo-vilab |
| modelscope-damo-text-to-video-synthesis | unknown | unknown | damo-vilab |
| text-to-video-ms-1.7b-legacy | unknown | unknown | damo-vilab |
can also use from modelscope:
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks
p = pipeline('text-to-video-synthesis', 'damo/text-to-video-synthesis')PAIR now releases Text2Video-Zero which leverages existing stable diffusion models to generate video. also released a bunch of controlnet dreambooth weights.
lucidrains is a workaholic on transformer implementations. we should scrape all the repos and index them. there are faster language models to train.
Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 minutes in length, in Pytorch
dreamix (not open-source)
instruct-pix2pix requires 16GB+ VRAM
text2live modify video by text prompt (such as add fire in mouth)
recurrent-interface-network-pytorch using diffusion to generate images and video
high quality! imagegen-video code with demo and paper
抄视频 视频的时间要讲究 看看是抄一年前的好还是抄刚刚发布的好
在发布的一个视频当中 最多抄某个作者的两三个符合要求的片段
use editly smooth/slick transitions and subtitles to beat the copy-detection algorithm, also consider color change in ffmpeg
动态 专栏也可以抄
谷歌AI歌手震撼来袭！AudioLM简单听几秒，便能谱曲写歌 https://www.kuxai.com/article/398