Project structure of: Doubiiu/DynamiCrafter

• DynamiCrafter Computer Vision, Deep Learning, ML, Video
  • configs
    • inference_256_v1.0.yaml Deep learning model config for image generation and inpainting.
  • gradio_app.py Create video examples with Gradio and DynamiCrafter.
  • lvdm Computer vision, ML, Python, linguistic variation, memory.
    • basics.py Convolutional layers, modules, normalization, instance of hybrid conditioner.
    • common.py Utility functions for data processing, mixed-precision, tensor ops, and image/map detection.
    • distributions.py Python distributions code for image gen, ML tasks.
    • ema.py EMA PyTorch class for training and parameter management.
    • models Conditional Image Generation Models: AutoencoderKL, DDPM & Adaptive Embeddings.
      • autoencoder.py AutoencoderKL model, colorization, IDFirstStage, encode/decode, optimizer, logging
      • ddpm3d.py 3D Conditional DDPM Model for Image Generation
      • samplers DDIM samplers for PyTorch image generation and adaptive outputs.
        • ddim.py DDIM sampler for PyTorch image generation
        • ddim_multiplecond.py DDIM sampler with adjustable outputs and stochastic encoding.
      • utils_diffusion.py Timestep embeddings, diffusion schedules for efficient DDIM/DDPG models. Prevents singularities.
    • modules Language processing modules for diverse tasks.
      • attention.py Efficient attention components for transformer models.
      • encoders OpenCLIP T5 & Resampler Encoder Modules
        • condition.py OpenCLIP T5 Encoder for Text and Image
        • resampler.py Resampler module with 8 layers, normalization, and PerceiverAttention
      • networks Autoencoder and 3D object reconstruction networks
        • ae_modules.py Autoencoder module with attention, convolution, Swish
        • openaimodel3d.py UNet-based 3D object reconstruction model
      • x_transformer.py X-Transformer with positional embeddings and multi-head attention.
  • prompts
    • test_prompts.txt Diverse prompts for creative inspiration
  • README.md Generate storytelling videos from tables using open-source DynamiCrafter tool.
  • requirements.txt Python library dependencies and versions.
  • scripts Scripts for model inference and evaluation.
    • evaluation Evaluating distributed inference and DDPM tasks scripts.
      • ddp_wrapper.py DDP wrapper script for distributed inference
      • funcs.py DDPM sampling, model loading, prompt handling
      • inference.py Video synthesis deep learning model inference with argparse and seed tracking.
    • gradio
      • i2v_test.py Video synthesis with DDIM, text and image conditioning in diffusion model.
    • run.sh Run Dynamic Rafter's model inference with custom configurations.
    • run_mp.sh Run multi-GPU DynamiCrafter inference script.
  • utils
    • utils.py Utility functions and config string processing for parameter counting, matching, and object instantiation.