Chatgpt Clones, Computer Automation With Ai

simply because the original note on chatgpt is too long, we start a new one, with more topics and more resources.

visit poe.com for a bunch of free chatbots, including GPT-4

chathub is a browser plugin which you can use ChatGPT, Bing and Bard

ts_server supports a bunch of models like GPT-J, GPT-NeoX, GPT-Neo, OPT, Fairseq GPT, M2M100, CodeGen, GPT2, T5, RWKV, LLAMA and Stable Diffusion, used by textsynth.com

to manage python versions and environments, pyenv and venv is lightweight and miniconda or mamba is more sophisticated.

javascript code for extracting model list from huggingface personal homepage:

1
2
3
4
5

var arr = [];
for (var i of document.getElementsByTagName("h4")) {var t = i.innerText; var tlist = t.split("/"); var t0 = tlist[0]; var t1 = tlist[1]; arr.push(`| [${t1}](https://huggingface.co/${t}) | unknown | unknown | ${t0} |`)};
console.log(arr.join('
'));

convert arxiv paper (pdf) into html: arxiv vanity (you will have a better view than before, though will not always work) code on github

aminer is similar to paperswithcode, in which you may find interesting papers.

someone prefers bert4keras since it implements multiple LLM into Keras, also easy for GPT-2 LoRA training (by adding a single layer)

people love to post uncensorable links and torrents to internet archive and the-eye, just like the gpt-4chan

to create a simple API (compatible with OpenAI APIs) for LLMs, use SimpleAI

fine-tuning and tricks

PEFT (Parameter Efficient Fine Tuning) supports LoRA, Prefix Tuning, P-Tuning and Prompt Tuning.

computer automation with ai

virtual machines and environments

it is not feasible to install ubuntu arm on macos m1 with virtualbox. use utm.app instead. instructions on installing ubuntu with utm includes guides on sharing clipboard and directory.

papers

playing atari using q-learning (viewing deepmind paper with arxiv vanity)

models

video pretraining can perform minecraft diamond mining tasks with keyboard and mouse movements

code and model

mm-cot (multimodal chain-of-thought) by amazon, with model weights

data collectors and controllers

mss for screenshot, remember to save raw pixels to SSD, then compress into mp4 with ffmpeg for further training (mind the timestamp!)

go-vncdriver by openai, to compile you need to clone the repo and modify code to find headers for libjpeg-turbo and python.

libvncdriver

asyncvnc (supports apple vnc), side project: asyncssh

python-vnc-client with keydown/keyup event support

vncdotool

pyVNC

pynput as input event listener and actor, listener may have some strange keycodes when pressing modifier keys on windows.

note that special care needed for aligning mouse location with screenshot size

ViT-pytorch can be used in many ViT-based models, listed and implemented in the repo.

spaces

openai universe (blog post here) and starter agents, remotes are using vnc protocol and a reward protocol using websocket sending json (can send actions). they prefer TigerVNC, maybe that will send the existing monitor instead of invisible ones.

gym is classic and modular. atari-py enables old games

retro deprecates universe, but might help with general computer controlling AI systems since they are compatible. human don’t play games all day and night. beware of this and don’t turn the model into a heavy gamer.

there is no meaning of recording terminal input/output when using tools like vim. get screenshots, keystrokes and mouse clicks instead (using ttyd, gremlins.js or monkey.js). tkterminal won’t do. it is just a thin wrapper around subprocess.run

talking of browser, you can spin up novnc server and let the gremlins.js do its job.

accelerators

cformers

cpu only

able to install from pip

ggml

cpu only

cpp, only compile from source

flexgen

gpu is mandatory, better than deepspeed and Hugging Face Accelerate

open source model and weights

awesome decentralized llm listed up-to-date related chatgpt-like repositories, datasets, model weights and resources.

model weights of open source chatgpt alternatives:

LLaMA

it’s public.

ChatYuan

v2 is censored.

model weights:

Deepshard

LLaMA trained on custom instruction dataset.

model weights:

ChatGLM

Currently only open-sourced 6B version.

You can train ChatGLM using GXT3090: simple_thu_chatglm6b

Using 7GB VRAM, train ChatGLM with P-tuning

chatglm_finetuning supports loading from int4 weights

model weights:

ChatDoctor

LLaMA-65B trained on medical dataset InstructorDoctor-200k

BELLE

开源中文对话大模型

model weights:

baize

trained on ChatGPT self-chatting data

model weights:

dolly

model arch is gpt-j, trained on alpaca dataset

model weights:

FastChat (Vicuna)

web interface

model weights:

| weight path | weight size | model name | author |
| – | – | – |
| vicuna | unknown | Vicuna | chavinlo |
| vicuna2 | unknown | Vicuna | chavinlo |
| vicuna-13b-delta-v0 | 13b | Vicuna | lmsys |
| vicuna-13b-GPTQ-4bit-128g | 13b | vicuna | anon8231489123 |
| ggml-vicuna-13b-4bit | 13b | vicuna | eachadea |
| vicuna-13b | 13b | vicuna | eachadea |
| vicuna4all | 13b | vicuna | vicuna4all |

download official delta weights via [magnet](magnet:?xt=urn:btih:a7fac57094561a63d53eed943f904abf24c6969d&dn=Vicuna-13B-HF-fp16-delta-merged_2023-04-03&tr=udp%3a%2f%2ftracker.opentrackr.org%3a1337%2fannounce&tr=udp%3a%2f%2ftracker-udp.gbitt.info%3a80%2fannounce&tr=udp%3a%2f%2ftracker1.bt.moack.co.kr%3a80%2fannounce&tr=udp%3a%2f%2ftracker.tiny-vps.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker2.dler.org%3a80%2fannounce&tr=udp%3a%2f%2fopentracker.i2p.rocks%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.altrosky.nl%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.theoks.net%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.dler.org%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.torrent.eu.org%3a451%2fannounce&tr=udp%3a%2f%2ftracker.openbittorrent.com%3a6969%2fannounce&tr=https%3a%2f%2fopentracker.i2p.rocks%3a443%2fannounce&tr=http%3a%2f%2ftracker.openbittorrent.com%3a80%2fannounce&tr=udp%3a%2f%2ftracker.moeking.me%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.monitorit4.me%3a6969%2fannounce&tr=udp%3a%2f%2f9.rarbg.com%3a2810%2fannounce

based)

Bloom-z

there is bloomz.cpp, converted model weights on huggingface

Alpaca

alpaca is LLaMA tuned on ChatGPT self-instruct dataset. officially there is just code and dataset, model weights are community provided.

ggml version: alpaca.cpp

example on how to load PEFT patched alpaca model: alpaca-lora/generate.py

it’s better to check for python bindings and webui like Alpaca-Turbo and Dalai for further development and interactions.

fine-tuning:

simple-llama-finetuner using LoRA, 16GB VRAM minimum

alpaca-lora: the OG LoRA alpaca

community model weights:

codealpaca only provides dataset for training a code generation model, there are multiple models trained on this dataset, including bloom-7b1-lora-codealpaca20k

togethercomputer

released openchatkit with retrieval ability and its huggingface space

model weights:

moderation model weights:

SpikeGPT

inspired by RWKV

model weights:

RWKV

RWKV combines attention with RNN so the token window can be much larger.

Longformer is similar to this. Model weights in github repo or huggingface.

now we have rwkv.cpp (4bit quantization), build upon ggml and sure it works on cpu.

rwkvstic (with 8bit & offload for low VRAM GPUs)

RWKV-LoRA supports RWKV-v4-NeoX

model weights:

encrypted alpaca model weights released by point-network: point-alpaca

gpt4all by nomic

LLaMA trained on massive collection of clean assistant dialog data, with model weights

you need to install nomic to run the model:

1
2

pip3 install nomic

to run it on gpu, you need to install this

openassistant

researchers of open-assistant like andreaskoepf has releasesed oasst-sft-3-pythia-12b-epoch-3.5 and still updating

model weights:

reward model weights:

openflamingo

using CLIP ViT-L and LLaMA-7B, model weights on huggingface

cerebras gpt

open sourced model weights and training code

model weights:

ColossalChat

Coati-7B has no public model weights, but claimed to be trained efficiently

you need to install LLaMA compatible transformers library

train on InstructionWild

enhancements

using external tools

toolformer-pytorch (WORK IN PROGRESS)

engshell: using LLM to execute command

using ai models

Microsoft JARVIS aka HuggingGPT leverages huggingface models so ChatGPT can complete complex multimodal tasks.

retrieval plugins

long term memory for oobabooga/text-generation-webui (can run pythia, galatica, opt, gpt-j, gpt-4chan, rwkv and support quantization/acceleration), also

complex memory (KoboldAI-like)

chatpaper summarize paper content.

similar website: typeset.io (can ask questions and explain confusing text, math symbols and tables)

related projects: ChatReviewer ChatImprovement ChatResponse ChatGenTitle

chatgpt retrieval plugin chop document into chunks, process them into vectors and search them using one of many vector search backends. hosted as a fastapi service.

datasets

assistant dialogue

botbots dataset (two chatgpt talking to each other), created by using datasetGPT (LLM automation tool)

ShareGPT52k, also ShareGPT90k (Vicuna)

instruct-102.4k by swype

datasets by BELLE:

train_1M_CN

train_0.5M_CN

multiturn_chat_0.8M

school_math_0.25M

unsupervised pretraining

Fandom23K (text classification), part of BigKnow2022

Kinda LLaMA replicates LLaMA dataset, including scraped webpages, code and stackexchange data.

oscar-corpus needs to be downloaded with access token, by accepting agreement with account. containing categorized content and adult content.

dataset preprocessing

deduplicate text dataset in rust, may remove verbose substrings like “to go to the”

oscar project (Open Super-large Crawled Aggregated coRpus) contains some tool for adult content filtering and deduplication.

NLP tools & training methods

fasttext for efficient learning of word representations and sentence classification.

langchain

prompt-engine

chatml: markup language for ChatGPT, by openai

react-agent-ts enables LLM to chat and use tools by internal dialogues.

babyagi: AI-powered task management system. original post on twitter

Chain-of-hindsights (can learn from negative feedback) in jax and pytorch

interfaces

serge is dockerized and the needs of RAM is according to the size of the model (alpaca), using CPU only