Project structure of: sebbyjp/robo_transformers

• robo_transformers TensorFlow dependencies and inference setup with varied models testing.
  • pyproject.toml Python project config with Poetry, TensorFlow dependencies.
  • README.md Setup RTL, package for model inference, defines data types.
  • requirements.txt Required Python libraries with versions listed for project compatibility.
  • robo_transformers Robot Transformers actions and registry.
    • abstract Abstract classes for action and agent in transformers directory.
      • action.py Abstract class Action: Create instance from numpy dict, returns action as dictionary.
      • agent.py Abstract Agent class with history and past actions.
    • demo.py Transformer agent performs inference on three images.
    • inference_server.py Robot inference server for Vision-Language model with grasp prediction
    • models Robotics models: Octo, RT1, transformers inference
      • octo OctoModel: Robotic octopus agent tasks inference
        • action.py Octopus arm action data class
        • agent.py OctoAgent: Agent subclass, acts with OctoModel and tasks.
        • inference.py Octo inference model script
      • rt1 RT1 model: Agents, actions, inference for robotics with TF/PT and URL checkpoints.
        • action.py RT1Action: Action subclass with action parameters and defaults.
        • agent.py RT1Agent: Agent for RT1 model, action generation via inference.
        • inference.py Inference code for robotics using TF/PT, URL checkpoints, Sentence Encoder
    • registry.py Agent-Action registry for RT1 and Octo models in Robot Transformers.
  • tests InferenceServer and OctoAgent tests with different models.
    • test_inference_server.py Test InferenceServer dummy mode functionality.
    • test_octo.py OctoAgent tests: demo and wrist action generation.
    • test_rtl.py Four test functions, different models, load and run demos.