Project structure of: james4ever0/agi_computer_control

• agi_computer_control Interactive AI, Docker, Machine Learning
  • action_state_machine
    • example.txt Terminal session: user input, responses, prompts, cursor movements.
  • array_check_ast_parsing.py Python code parser for annotations.
  • array_shape_typecheck.py 2D array, Array class, numpy, Literal casting, Jinja2 macros.
  • array_static_typecheck.py Static-typed array class with type checks, operations, and SymPy integration.
  • autogui.py AutoGUI Python script for GUI input capture and playback.
  • basic_interactive_program_emulation_and_image_with_docker_support Docker support, interactive emulation, image development
    • alpine_actor.py Interactive Docker emulation with exceptions and logging.
    • alpine_bytes_actor.py AlpineBytesActor: Loop, Read-Write, Forever
    • beat_common.py Decorator function for caching, server info, GET requests, and timeout management.
    • beat_server.py FastAPI, timezone, roles, PIDs, UUIDs, HTTP requests, status management.
    • bytes_actor.py BytesActor: Docker-supported, NaiveActor-based interactive program.
    • dependent_task_timeout_exec.py Timeout future execution.
    • Dockerfile Installs Docker from Alpine APT.
    • entropy_utils.py Entropy calculation using classes and context managers.
    • get_pid_of_self.py Retrieves PID of self.
    • kill_server.py Manage client processes and communicate with server, monitoring status.
    • log_common.py Rotating file logger, debug level, exceptions handling.
    • loop_forever.py Infinite loop, continuously increments 'a'.
    • metaheuristic_predictive_actor.py Metaheuristic predictive actor class with kernel updates.
    • naive_actor.py Naive actor benchmarks non-blocking actor performance with sockets and time measurement.
    • naive_interactive.py Interactive loop with sleep and random word gen.
    • predictive_alpine_actor.py Predictive Alpine Actor with Docker Support
    • predictive_bytes_alpine_actor.py Predicts bytes with Docker support.
    • profile_vis.py Sorting profiling data by calls in "profile_vis.py"
    • pull_alpine_with_different_arch.py Download Docker images of Alpine Linux with different arch support.
    • pyinstrument_profile.py Profile long-running program with PyInstrument.
    • README.md Train and use a bot for exploration and learning.
    • requirements.txt Project dependencies for cross-platform emulation with Docker support
    • rerun_docker_daemon.py Manage Docker service, handle OS differences, fix Win11 hangs.
    • sequence_learner.py Sequence learner predictor with Docker support.
    • test_beat_server.py Test Beat Server with Unittest and Docker support.
    • timeout_utils.py Timeout and retry decorators for functions.
    • to_be_killed.py Killable process ID printer.
    • type_utils.py Ensures input is bytes or string, raises exceptions if not.
    • vocabulary.py Vocab class, random ASCII/bytes strings.
    • wexpect_example.py weexpect: cmd emulation via Python, Docker support.
  • binary_program_synthesis_cpu_assembly_execution Binary program synthesis and assembly execution without human interface.
    • bnn_data_ingest.py Binary quantized tensor computation with attention.
    • convert_binary_files_as_zero_and_one_streams.py Binary file conversion to 8-bit streams.
    • gpt-binary-training.py Trains GPT2 model to generate binary input
    • interpret_and_save_binary_program.py Binary program interpretation and saving.
    • README.md Evolutionary program synthesis and assembly execution without human interface.
    • softmax_test.py Softmax test: 2D activation for random (2,3) tensor.
    • vkq_bin.py Binary quantization config, VKQAttention class for linear layers.
  • compose.yaml Compose VNC services with images
  • config.py Sets timestep, defines state path.
  • conscious_struct.py Python code for image processing, LSTM training, and RNN input prep.
  • containerized_chatgpt_agent Containerized ChatGPT AI scripts
    • ai_captialism.py Account handling, population limits, payments, incomplete pay formatter.
    • are_you_kidding_me.txt Image processing and password cracking code, ASCII-PNG challenges.
    • build_and_run_image.sh Dockerize ChatGPT agent, build and run.
    • build_llama2_autoexec_model.sh Automating LLaMA model creation in container.
    • build_llama2_visual_autoexec_model.sh Creating visual autoexec model file.
    • change_icon_size.py Resize image while maintaining aspect ratio, save resized version.
    • container_autoexec_example.py Container AI agent script with LitEllm, command parsing, retries, sleep delay.
    • diff_utils.py Diff utilities for comparing strings in Python
    • dna_like_transformation_triple_or_more_strands.py Bitwise DNA-like transformation with triple strands.
    • Dockerfile Install pip, open-interpreter via Tsinghua mirror.
    • Dockerfile_autoexec Installs Python chat agent dependencies, copies scripts.
    • Dockerfile_autoexec_visual Dockerfile: Xfce4, Xvfb, screenshot, PyAutoGUI setup.
    • install_pip.sh Install pip, check success or retry with apt-update.
    • install_pip_and_pyautogui_prequisites.sh Install pip, PyAutoGUI, dependencies in container.
    • Modelfile Docker image, llama2 model, coherent AI chat in bash shell.
    • Modelfile_visual Containerized ChatGPT, llama2-uncensored base, temperature 1, commands.
    • ollama_utils.py Ollama token limits, context manager, response stream
    • port_util.py Set default port, parse args, validate custom port.
    • ptyproc.py Containerized chat server with sound effects and adjustable settings.
    • README.md Containerizing AI, GUI vs Terminal challenges, "Helen Keller" project
    • run_autoexec.sh Build and run Dockerized Python agent with scripts.
    • run_autogpt_in_container.sh AutoGPT in Baidu Cloud container, GPT3-only.
    • run_llama2.sh Dockerize and run Llama 2
    • run_visual_autoexec.sh Start, clean, and run visual_autoexec containerized chatbot.
    • startup_ollama_service.sh Start Ollama container, create ChatGPT agent.
    • terminal_config.py Configure terminal: 80 cols, 25 rows.
    • test_image_to_ascii.py Test image to ASCII converter
    • visual_autoexec_example.py Multi-agent system: ChatGPT, visual model, commands, errors.
    • visual_autoexec_main.sh Runs Xvfb, Ubuntu visual server, and example script.
    • visual_server_on_ubuntu.py Python script for visual server control on Ubuntu
  • decorator_method_registry.py Decorator method registry: pre/post execution code.
  • directml_yolov5 DirectML-powered YOLOv5 inference tests
    • test.py DirectML-enabled, zero-sized array handling inference mode code.
    • test_concat.py Concatenation test using directml & torch
    • test_nograd.py YOLOv5 model, DirectML inference with warnings.
  • Dockerfile Dockerfile: x11docker base, dev server setup
  • dynamic_plasticity_neural_networks Dynamic neural networks with plasticity for adaptive bot decision-making.
    • dnn_reference.py Sigmoid-based DNN classes and functions.
    • dynamic_neural_network.py Dynamic neural network for bot decision-making
    • MCTS_NAS.md MCTS for NAS: Efficient Network Architecture Search.
    • neural_evolution.py Neural evolution using NEAT from evotorch library.
    • README.md Dynamic neural networks with movable artificial neurons.
    • sparse_matrix_multiplication.py Timing sparse matrix multiplication performance.
  • external_reference External reference directory: Contains 4 GitHub repos for cloning.
    • extra_init.sh Clones CogVLM and AppAgent repos.
    • init.cmd Init script for cloning repos.
    • supercharge_init.sh Supercharges GitHub repos: 4 clones.
  • generate_datamodel_code.sh Generate Python code from YAML datamodel file: MyModel.
  • hardware_capture_hid_power_control Hardware capture, HID power control, Python scripts
    • common_keycodes.py Translation of hardware keycodes for computer compatibility.
    • Makefile Compiles and executes Python script with Makefile
    • multiple_inheritance.py Multiple inheritance confusion with MRO.
    • parse_hid_keycodes.py Parse HID keycode data, pandas DataFrames, error handling, CSV/JSON output.
    • README.md Power control via /dev/serial/by-id/* with HID control using hidapi.
    • set_capture_card_properties.sh Installs packages, suggests OBS, sets video format & fps with v4l2-ctl.
    • test_control.py Power control with HID protocol for mouse and keyboard.
    • test_hidraw.py Test HID USB device interaction code, adjustable paths.
    • test_video_capture.py Configure video device, save frames as images. Fails to work correctly.
  • hid_utils.py Mouse and key input utilities. Translates, stores, writes.
  • hotkey_listener.py Hotkey listener code with key functions, awaiting activation.
  • hotkey_tester.py Control disables all hotkeys.
  • install_nvidia_container_toolkit.sh NVIDIA container toolkit installer for Kali 2022.2.
  • jinja_utils.py Jinja Python utility: import, manipulate, check, update, output.
  • keyboard_and_mouse.py Python script records keyboard and mouse events, configurable timesteps.
  • launch_docker_devicemapper_limit_storage.sh Docker storage options: devicemapper, limits explained.
  • launch_streaming_utils.sh Launch script for streaming utilities in new tmux sessions.
  • lego_mindstorm_physical_mouse_keyboard_control Control LEGO EV3 with keyboard and mouse over USB
    • README.md Control bot with keyboard and mouse without detection or batteries.
    • test.py Init EV3 motor, play sound, define motor object, move 100 units.
    • usb_ev3.py USB EV3 control script.
  • lunar_lander_test
    • test.py Lunar Lander test with Gymnasium, 1000 random actions.
  • Makefile Automates project setup, builds, and tests.
  • metalearning_random_shuffle Shuffling datasets, reducing entropy for metalearning.
    • main.py Shuffling datasets with letter-digit combos
    • README.md Perform ops without annotations, reduce entropy.
  • mouse-and-keyboard-encoding-fft.py Fast Fourier Transform mouse encoding, neural networks, embeddings, and positional encodings.
  • mydatamodel.py MyDataModel.py: Generates Python class from MyDataModel.yaml. Pydantic BaseModel.
  • mydatamodel.yaml YAML data model, two sets of variables with letter-number pairs.
  • mypy_check.py Mock-based API function verification. Class MyClass, age property, tensor function.
  • off_streaming_utils.sh Terminates streaming_utils processes.
  • packup.cmd Archives, removes 7z, moves backup.
  • packup.sh Remove, change dir, archive, move file
  • player.py Manage keyboard/mouse with Pynput, hotkeys. DPI-aware, event order maintenance, no "ctrl+/" support.
  • propaganda Step-by-step video creation and AI productivity scripts
    • agi_computer_recordings
      • launch_server.cmd Command connects to host and WebDAV path for recordings.
    • cybergod_propaganda
      • README.md Cybergod propaganda: Step-by-step video creation.
    • video_script Automated Cybergod AI videos for productivity and health.
      • clyde_script.txt Cybergod: AI controller, inspire developers.
      • clyde_script_cn.txt Introducing Cybergod AI鼠标键控. Automate, save time. Developer collaboration encouraged.
      • compile_script.py Background music location query in script
      • download_video.py Video script downloader and saver.
      • preprocess.py Script preprocesses, splits, and lists data for script.md.
      • read_text.py Generate audio from text, YAML data.
      • README.md Search videos, API endpoint, ffmpeg conversion.
      • script.txt Cybergod automates computer tasks, boosts productivity and health.
      • script.yaml Cybergod YAML script with video link and text
  • pyanalyze_check.py Analyze pytest lambda function serialization and extraction.
  • pyright_utils.py Pyright utils: track errors, regex pattern, cache, undefined vars.
  • pytest_disable_assertion_inspection_use_better_exceptions.py Replaces numpy.array with marray, sets test exceptions.
  • pytropos_check.py Matrix multiplication with unit testing.
  • qstar_my_guess Exploring A*, MCTS, and transformers for AI problem-solving.
    • astar_test.py A* algorithm maze solver tested.
    • main.py Exploring MCTS, transformers, reward functions in PPO. Hidden latent space for video generation.
    • mcts_pseudo.py Monte Carlo Tree Search Algorithm in Python
    • mcts_test.py CartPole game, OpenLoopMCTS, 1000 iterations, non-training mode, saving failed.
    • README.md AI-powered code tagging, indexing, and writing. RAG-backed, acknowledging human uniqueness.
    • signature_test.py Type hints and runtime checking with overload function.
    • thought_tokens.py Thought token insertion code. Iterative, probabilistic.
    • time_traversal.py Monte Carlo Tree Search, gradient descent, world model updates, sequence manipulations, cosine similarity, computational time, loss delta.
  • random_actor.py Random mouse and keyboard inputs generator for simulation.
  • README.md Entertaining AI project with devcontainers and utility features.
  • record_playback_test.py Record and playback Windows UI actions with pywinauto.
  • recorder.py Script records computer screen with sound and keystrokes.
  • recording_train_parse.py Train data synchronization and processing.
  • render_python_code.py Python code renderer with Jinja templates.
  • requirements.txt Data processing, image manipulation, keyboard handling libraries.
  • rt_x_experiments Real-time RTX experimentation, diverse techniques and models.
    • audio_to_mel
      • test_librosa.py Load audio, set sample rate, generate mel spectrogram, convert to dB, save as variable.
    • fourier_transform_combine_data Fourier transform experiments: 1D and 2D FFT, array combination, inverse.
      • test_common.py Test script for array addition and print.
      • test_fft_1d.py Compute 1D FFT, combine, and inverse. Print results.
      • test_multidimension_fourier_transform_2d.py 2D FFT, combines arrays, inverse FFT, prints results
    • gradient_undescent Dynamic learning rate optimization, unlearning with loss inversion and Adam optimizer experiments.
      • dynamic_learning_rate.py Dynamic learning rate optimization functions
      • test_unlearning.py Unlearning model with loss inversion, Adam optimizer
    • partial_training_network
      • test_freeze_one_and_train_another.py Trains layers, manages RAM, controls GPU usage.
    • README.md Dynamic tokenizer, attention layers, neuron states, activation functions, AI deviation.
    • real_attention Real-time attention experiments for image cropping and analysis.
      • 2d_convolve.py 2D image convolution with scipy's convolve2d
      • low_rank_positional_encoding.py Low-rank positional encoding for attention models.
      • recursive_positional_encoding.py Recursive positional encoding for zoomed patches.
      • sin_2d_positional_encoding.py Bilinear interpolation for positional encoding visualization.
      • test_model_level_real_attention.py Image cropping functions with grayscale analysis and positional embedding exploration.
    • requirements.txt Package dependencies: rtx-torch, classifier-free-guidance-pytorch, efficientnet-pytorch
    • rt_x_test_code Efficient RTX model training with action interpretation and DirectML fix
      • requirements.txt Lists: PyTorch libraries
      • rtx1_example.py RTX1 model, random video data, computes train and eval logits.
      • rtx2_example.py Efficient model training with action interpretation and DirectML fix on RTX2.
    • special_tokenizer_with_actions Special tokenizer for actions and text classification with optimization and hierarchical techniques.
      • hourglass_lm.py Hourglass Transformer: Efficient Language Modeling
      • test_action_and_text_tokenizer.py Custom tokenizer for actions and content, optimization techniques
      • test_hierachical_tokenization.py Hierarchical tokenization model for text classification using Einops and sparse transforms.
      • test_simutaneous_tokenization_embedding.py Concurrent embedding tokenization & transformers for words and actions.
      • test_tokenmonster.py TokenMonster vocab test.
    • zoom_pan_action.py Image cropping code with XY zoom-pan support.
  • run_gpu_docker.sh Test GPU access in Docker container.
  • screenshot_test.py Screenshot testing in 0.03s
  • software_capture_hid_control Controls software with HID and X11 streaming, abstract class tests.
    • abstract_class_test.py Abstract class with two abstract methods.
    • Makefile Builds and executes "test_control.py" with Makefile
    • test_control.py Control software with libxdoHID and X11 live streaming, saving image issues.
    • test_xdo.py Tests HID control for software.
  • streaming_utils.yaml Windows with commands and directories configuration.
  • sync_utils.sh Sync script: Updates and copies files.
  • terminal_test_tk.py Tkinter window with terminal widget.
  • test Test directory with model training, abstract implementation, and error testing.
    • run_test.sh Run test_project.py with pytest in parent folder
    • test_abstract_impl.py Abstract base class implementation with instantiation error.
    • test_project.py Trains custom models with hypothesis strategies, Adam optimizer.
  • test_calculated_type.py Unify PyTorch types with Erg static checking.
  • the_cage_dataset
    • README.md Control bot with "cage", synch timestamps.
  • the_frozen_forest_intro Multimodal training dataset "The Frozen Forest" introduction.
    • drissionpage_common.py Configures Chromium browser with extensions for ad management and page object timeout.
    • keylogger_extension Initializes and manages keylogger extension for frozen forest.
      • init.cmd Initiating keylogger download.
      • virtual-keylogger Initializes server, handles errors, manages HID events for HDFC Bank's netbanking virtual keylogger.
        • README.md Virtual keylogger for HDFC Bank's netbanking
        • src
          • index.js Initializes server, handles errors, and manages HID events.
        • webpack.config.js Configuring webpack for production, entry: "./src/index.js", output: "main.js" in "dist".
    • keylogger_server.py FastAPI server for event monitoring, logs errors, stores data in JSON.
    • playwright_locate_mouse_viewport.py Automates mouse actions, screenshots Paint.js, checks image size.
    • pw_listen_to_events.py Browser extension event listening challenges with VS Code and Playwright.
    • pw_random_actor.py Random character, Playwright setup, Bilibili screenshots.
    • random_browser_agent.py Interacting with webpages, resolving mouse issues.
    • README.md Introducing "The Frozen Forest" dataset for multimodal training.
    • render_cursor_on_image.py Cursor overlay on screen image.
    • sample_browser_events.py Browser event listeners for mouse and keyboard in Pydantic classes.
    • take_viewport_screenshot.js Take viewport screenshot and open it in new tab.
    • test_dataset.py Screenshot capturing Pygame window for typing words, AI dangers discussed.
    • watch_video_in_webpage.py Webpage automation with AI-controlled browsers.
  • try_better_exceptions.py Raises KeyError for non-existent key "2" in dictionary "a"
  • try_better_exceptions.sh Improved exception handling script.
  • tsalib_test.py Test: Tsalib static shape checking in PyTorch.
  • ubuntu_qemu_utm_arm_record Ubuntu QEMU UTM ARM recording setup
    • audio_record.py Records and saves audio with error handling.
    • enable_audio_loopback.sh Enable audio loopback for Ubuntu ARM QEMU/UTM system.
    • hdf5_test.py Create HDF5 file with boolean dataset "myset" size 100.
    • kali_prepare_dirs_loop.sh Prepares Kali WebDAV directories for looping.
    • kali_prepare_two_webdav_dirs.sh Create two WebDAV windows using Kali and tmuxp.
    • kali_prepare_two_webdav_dirs.yaml Set up two WebDAV directories, read and read-write access.
    • kali_vbox_control Automated VM management and control.
      • check_vm.sh Check Ubuntu 16.04 VirtualBox VM status
      • control_vm.sh Vbox: Stop audio, enable VRDE for Ubuntu 16.04 VM.
      • disable_logout_poweroff_button.sh Suppresses logout, restart, and shutdown buttons.
      • duplicate_frame_detector.py Detects locked VM screenshots.
      • exec_vm.sh Unlock session "c2", run /bin/loginctl for Ubuntu 16.04 guest with username hua, password 110110.
      • exec_vm_if_locked.py Lock, execute, unlock VM commands.
      • kali_vbox_main_loop.py Automated VM management and control.
      • lightdm.conf Set default user "hua", no timeout, Ubuntu session.
      • list_vm.sh Lists all VMs in VirtualBox.
      • modify_vm.sh Enable VRDE, set port for Ubuntu 16.04 VM.
      • README.md Automatic login, startup script, screenshots, systemd service creation.
      • restore_vm.sh Restores snapshots of "Ubuntu 16.04" VM, except Snapshot 12.
      • screenshot_vm.sh Take "Ubuntu 16.04" VM screenshot, save as output.png
      • start_vm.sh Starts VirtualBox VM "Ubuntu 16.04"
      • stop_on_ffmpeg_x11_error.py Checks for Failed to query xcb pointer error, stops loop.
      • stop_vm.sh Stop VirtualBox VM and power off.
      • the_frozen_forest.service The Frozen Forest dataset creation service.
      • x11grab.py Captures screen from display 10 with X11 grabber
      • x11grab.sh Capture X11 window content, save images, real-time playback.
      • x11grab_loop.sh Xvfb server, x11grab.sh, X11 errors monitoring
      • x11grab_loop_viewer.sh X11 grabbing script for FFplay from display 11.
      • xvfb_test_loop_double_display.py Runs xvfb for display in virtual environment, loops "x11grab_loop.sh" script.
      • xvfb_test_loop_double_display.sh Xvfb double-display test script.
      • xvfbwrapper_vboxmanager_test.py Test script launches Ubuntu VM with Xvfb display and faulty progress bar.
    • launch_server.cmd Launch server command for Ubuntu QEMU UTM ARM WebDAV recording.
    • macos_iterate_windows.sh Iterate window IDs, print or record.
    • macos_take_screenshot.py Capture and save window screenshot.
    • macos_utm_run_qemu.sh Run UTM with QEMU, ARM image, Spice protocol, network config.
    • main_loop.sh Kills & restarts Python processes in infinite loop, requires sudo.
    • main_recorder.py Manages recording of audio, video, HID with Redis signaling.
    • mount_kali_webdav_dirs.sh Mount, configure, and symlink WebDAV directories.
    • mount_yoga_webdav_dir.sh Mount Yoga WebDAV Directories Script
    • mouse_keyboard_record.py Pyautogui records HID events in Redis.
    • prepare_workspace.sh Prepare workspace, link directories.
    • pyaudio_get_device_info.py PyAudio, device list, prioritize BlackHole, channel check
    • pyscreenshot_output.py Capture screenshots, save, check Redis status.
    • random_actor_redis.py Python tool for random GUI interactions, error handling, and Redis checks.
    • README.md Quick start guide for Ubuntu QEMU UTM ARM recording setup.
    • record_audio_pulseaudio.sh Script records and saves pulseaudio audio streams as Opus files.
    • requirements.txt Required Python packages for project: redis, jsonlines, pynput.
    • start_main_loop.sh Starts Redis, kills Python processes, runs main_loop.sh (no log)
    • test_record
      • video_record_script.sh Record 1920x1080 video with ffmpeg and python.
    • test_util.py Error from division by zero in test_util.py
    • utils.py Utils for Redis recordings and control in Ubuntu QEMU with ARM architecture.
    • utm_test_video_record.sh Capture Ubuntu QEMU UTM ARM screens in MP4.
    • video_record.py Script records screen with ffmpeg.
    • vnc_unix_domain_socket.py Creates Unix socket, prints Twisted attributes.
    • webdav_macos_start.sh Start WebDAV script on MacOS.
  • usb_power_management Optimize USB power management across Linux and Windows
    • linux Linux USB power management, solutions and scripts.
      • devcontrol.sh Activates USB power management, disables LPM.
      • README.md Linux USB power management, solutions, and instructions
    • README.md Optimize USB power management for longevity.
    • windows
      • README.md USB device power management for Windows.
  • use_logging.py Various logging implementations with Python libraries.