Project structure of: OthersideAI/self-operating-computer

• self-operating-computer Self-Operating Computer Framework: Multi-modal AI automation tools.
  • evaluate.py Evaluate image adherence with GPT-4 Vision and color-coded messages.
  • operate Operating system and tools for self-automation.
    • actions.py Generate content with AI prompts and OpenAI API.
    • dialog.py Dialog system with error handling and input checks
    • exceptions.py Exception class for unrecognized model
    • main.py Self-Operating Computer Main Entry Point
    • prompts.py AI-assisted Chrome, Docs, Sheets prompts with contextual options.
    • settings.py Manages settings, loads .env, initializes OpenAI client.
    • utils Utility functions and tools for various tasks.
      • label.py Image data handling and encoding utilities.
      • misc.py Converts percentages, extracts JSON and classifies responses.
      • os.py OS Utilities: Search, Input, Click, Move. Last Assistant Message.
      • screenshot.py Screenshot capture and grid addition with PIL.
      • style.py Style utility with PromptStyle and ANSI check
  • README.md Self-Operating Computer Framework: Enhancing multimodal models for mouse click predictions and API access.
  • requirements-audio.txt Capture quiet or whispered audio with Whisper-Mic.
  • requirements.txt Python packages: aiohttp 3.9.1, ultralytics 8.0.227
  • run.sh Install Linux, configure .env with OpenAI key.
  • setup.py Setup script for self-operating-computer package using setuptools.