find things in colab, kaggle, aistudio, bilibili, youtube.
see in huggingface tasks to find task-specific models, also huggingface spaces for demo on models
modelscope by alibaba supports tensorflow and pytorch
run models from intel model zoo in docker container, like recommendation
百度总结的 比较全面的深度学习应用 deeplearning applications
阿里巴巴模型库 具有许多适用于商业 自媒体的模型供选择
huggingface
苹果官方CoreML模型库
CoreML第三方模型库
paddle模型库 paddlehub
pytorch模型库
TensorFlow模型库
autograd and xla (Accelerated Linear Algebra)
With its updated version of Autograd, JAX can automatically differentiate native Python and NumPy functions. It can differentiate through loops, branches, recursion, and closures, and it can take derivatives of derivatives of derivatives. It supports reverse-mode differentiation (a.k.a. backpropagation) via grad as well as forward-mode differentiation, and the two can be composed arbitrarily to any order.
XLA (Accelerated Linear Algebra) is a domain-specific compiler for linear algebra that can accelerate TensorFlow models with potentially no source code changes.
probabilistic programming
pyro implementation in numpy, alpha stage
machine learning in python
install official python bindings:
1 | pip install -U libsvm-official |
third-party python libsvm package installed by:
1 | pip install libsvm |
opennlp uses onnx runtime(maybe?), may support m1 inference.
opennlp is written in java. after installing openjdk on macos with homebrew, run this to ensure openjdk is detected:
1 | sudo ln -sfn $(brew --prefix)/opt/openjdk/libexec/openjdk.jdk /Library/Java/JavaVirtualMachines/openjdk.jdk |
opennlp has a language detector for 103 languages, including chinese. opennlp has a sentence detector (separator) which could be trained on chinese (maybe?)
in order to use opennlp with less code written, here’s how to invoke java from kotlin
found on mannings article about better search engine suggestions. in this example it is used with lucene, which has image retrieval (LIRE) capability. lucene is also avaliable as lucene.net in dotnet/c#.
to install lucene.net:
1 | dotnet add package Lucene.Net --prerelease |
deep learning library for java
gradient boost is used to train decision trees and classification models.
Light Gradient Boosting Machine
have official commandline tools. installation on macos:
1 | brew install lightgbm |
install python package on macos:
1 | brew install cmake |
if want to enable jax sampling, install numpyro or blackjax via pip
difference between pymc3 (old) and pymc (pymc4):
pymc is optimized and faster than pymc3
pymc3 use theano as backend while pymc use aesara (forked theano)
docs with live demo of pymc
PyMC is a probabilistic programming library for Python that allows users to build Bayesian models with a simple Python API and fit them using Markov chain Monte Carlo (MCMC) methods.
a high level torch wrapper including “out of the box” support for vision, text, tabular, and collab (collaborative filtering) models.
on the twitter list related to opennlp shown up on its official website, fastai has been spotted.
fastai does not support macos. or is it? fastai is on top of pytorch. initial support starts with 2.7.8 and now it is currently 2.7.9
searching ‘samoyed’ like this in github we get a dataset for pets classification called imagewoof from fastai 2020 tutorial series. more image classes like subcategories of cats may be found in imagenet.
vercel hosts frontend only apps, could be useful if you want.
可以提取关键词然后到百度必应上面搜索 获取相关内容 注意语种一致性
search huggingface with julia or python:
huggingface_hub(python)
可以用huggingface的api来翻译 对接英文的chatbot (blenderbot, dialo-gpt)
add timeout to these api requests
可以把训练好的中文chatbot放到huggingface上面去 用kaggle放
https://github.com/yangjianxin1/GPT2-chitchat
could use this method to generate title for videos. i mean generally.
could host the model on huggingface, or baidu aistudio, heroku or your own machine
configure accelerated inference on huggingface (free for cpu, paid gpu):
https://huggingface.co/docs/api-inference/quicktour
huggingface inference apis:
https://huggingface.co/inference-api
huggingface conversational (chatbot) models:
https://huggingface.co/models?pipeline_tag=conversational&sort=downloads
heroku, use fastapi as interface:
https://www.kaggle.com/getting-started/208405
heroku alternatives:
back4app, google app engine
aistudio api, maybe you need to train or find a paddpepaddle based chatbot:
https://ai.baidu.com/ai-doc/AISTUDIO/bk3e382cq#创建在线api服务
一个项目可以创建至多五个沙盒服务, 并选择其中一个沙盒服务部署为线上服务.
沙盒服务如果连续超过24小时无调用将自动调整为暂停状态.
线上服务如果连续超过14天无调用将自动调整为暂停状态.
paddlenlp
https://aistudio.baidu.com/aistudio/projectdetail/3723144?channelType=0&channel=0
paddlepaddle chat model:
plato2
https://github.com/PaddlePaddle/Knover
https://github.com/PaddlePaddle/Knover/tree/develop/projects/PLATO-2
https://aistudio.baidu.com/aistudio/projectdetail/1886227?channelType=0&channel=0
中文chatbot:
https://github.com/zhaoyingjun/chatbot
https://github.com/Dimsmary/Ossas_ChatBot
教程
https://github.com/lcdevelop/ChatBotCourse
https://github.com/fendouai/Awesome-Chatbot
语料库
免费gpt文本生成：彩云小梦 以及小梦海外版
小梦的中文文本有涉及政治的检测器 不能把敏感内容塞进小梦
对话生成
https://huggingface.co/thu-coai/CDial-GPT_LCCC-large/tree/main
twitter generator inspired by influencers:
https://github.com/gdemos01/TwitterInfluencerAI
chinese LM
清华130b大模型
测试地址：https://huggingface.co/spaces/THUDM/GLM-130B
模型仓库：https://github.com/THUDM/GLM-130B
https://github.com/Morizeyao/GPT2-Chinese
https://zhuanlan.zhihu.com/p/352028922
https://github.com/TsinghuaAI/CPM-1-Finetune
https://github.com/TsinghuaAI/CPM-1-Generate
https://github.com/TsinghuaAI/CPM-2-Pretrain
gpt2/cpm tutorial:
https://www.cnblogs.com/wwj99/p/12503545.html