Project structure of: datawhalechina/vced

• vced Docker, Compose, App Setup & Deployment
  • code Docker, Python, Jina CLIP indexing
    • command.sh Starts Streamlit app, runs service in background with timeout.
    • Dockerfile Builds Docker container, installs packages, exposes port 8501.
    • DockerfileArm ARM Dockerfile: nil01/vced, ports 8501,45679, command.sh script.
    • jina_demo Search, indexing, client-server, text manipulation, executors, Python functions, Jina Flow, Toy Executors, GRPC, video processing.
      • client.py Client posts empty document, receives result with three strings.
      • Executor.py MyExecutor class with decorated methods as HTTP endpoints.
      • Flow.py Flow orchestrates async Jina tasks with Executors.
      • image Image processing, tensorization, read, split, visualize
        • 7.1 读取图片并转为 tensor.py Read image, convert to tensor. (618x641x3 data)
        • 7.2 进行简单的图像处理.py Apples image preprocessing and saving.
        • 7.3 读取图像集.py Read JPGs, create DocumentArray, plot sprites.
        • 7.4 切割大型图像.py Jina library splits large image into chunks for visualization.
      • test.py Jina demo: Executor classes for foo, bar functions in Python.
      • text Text manipulation, document creation, splitting, exclamation marks, Jina demo
        • 6.1 创建文本.py Create document with text data using Jina's Document class.
        • 6.2 切割文本.py Text splitting with exclamation marks in Jina.
        • 6.3 text、ndarray 互转.py Jina demo: text to ndarray conversion.
        • 6.4 Demo: 简单的文本匹配.py Text matching demo with Jaccard search.
      • toy.yml Jina Flow: FooExecutor, BarExecutor, GRPC port 51000
      • video Video processing with AV and Docarray for MP4 manipulation.
        • 8.1 视频导入和切分.py Imports, splits video using AV and creates Jina Documents.
        • 8.2 关键帧提取.py Extract keyframes from video using Docarray
        • 8.3 张量转存为视频.py Save video tensor as 60fps MP4.
    • requirements.txt Python project dependencies for text and image tasks.
    • service Customizable Jina Docker setup for CLIP indexing
      • app.py Python script for Jina, CLIP model, indexing, and searching
      • customClipImage Custom CLIP image encoding service directory
        • clip_image.py CLIPImageEncoder: CLIP model-based image embedding encoder.
        • config.yml Configures CLIPImageEncoder for image encoding
      • customClipText Custom CLIP text encoder service with config options.
        • clip_text.py CLIP model class for text encoding with optional pre-trained models.
        • config.yml Custom clip text encoder config file.
      • customIndexer Custom Indexer with SQLite, CLIP, Top K Limit, and Workspace
        • config.yml Custom Indexer Config with Top K Limit and Workspace
        • executor.py SimpleIndexer: SQLite, CLIP, efficient Document storage and search.
      • Dockerfile Python 3.9 Docker setup, dependencies, app.py as entrypoint.
      • requirements.txt Python packages for project with Jina, Torchvision, Transformers.
      • videoLoader VideoLoader: Config, FFmpeg video handling.
        • config.yml Configures VideoLoader service with "vced/code/service/videoLoader/config.yml" and "video_loader.py".
        • video_loader.py VideoLoader executor handles video, audio, and subtitles with FFmpeg.
    • web Web app development for Streamlit and video processing with Jina.
      • app.py Video search and clip extraction web app.
      • Dockerfile Python Dockerfile, installs packages, sets up Streamlit app.
      • requirements.txt Ensures correct versions for Jina, Streamlit.
  • docker-compose-arm.yml Docker Compose file for service "vced" in ARM context.
  • docker-compose.yml Docker Compose config for "vced" service, build context "code", port 8501 mapped.
  • docker_install.md Install Docker on Ubuntu via WSL
  • README.md Python 3.9 Video Clip Extractor Tool
  • Roadmap.md Learn Jina ecosystem, optimize vector search.
  • startup.sh Install Rust, FFmpeg, Python, run Streamlit & Python. Echo first-run message.