Make desktop ubuntu headless

1
2
3
4

sudo apt-get purge libx11.* libqt.*
sudo apt-get autoremove -y
sudo apt-get clean

k8s worker cluster setup

k3s has different ways to form a cluster than kubeadm join.

https://docs.k3s.io/quick-start

k3s specifies the init command in /etc/systemd/system/k3s.service. (k3s-agent.service if installed as agent) usually it is k3s server.

you need to change it to k3s agent in order to join the master node, or pass additional environment variables K3S_URL=https://<node_ip>:6443 and K3S_TOKEN=<node-token> while running k3s installation script.

the node token is at /var/lib/rancher/k3s/server/node-token

the agent node still needs to configure registry mirrors at /etc/rancher/k3s/registries.yaml for successfully pulling images

k3sup can automatically install k3s cluster using ssh connection

multi server cluster setup:

https://docs.k3s.io/datastore/ha-embedded

install kubevirt

do not install it on microk8s

install windows on kubevirt:

https://charlottemach.com/2020/11/03/windows-kubevirt-k3s.html

VirtualMachine is like Deployment while VirtualMachineInstance is like Pod

export kubeconfig file path to ~/.bashrc

1
2

export KUBECONFIG=<kubeconfig_filepath>

download and modify the manifests

1
2
3
4
5
6
7
8
9

# prefer ustc and azure mirrors of quay.io
export VERSION=$(curl -s https://api.github.com/repos/kubevirt/kubevirt/releases | grep tag_name | grep -v -- '-rc' | head -1 | awk -F': ' '{print $2}' | sed 's/,//' | xargs)
echo $VERSION
curl -OLk https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/kubevirt-operator.yaml
curl -OLk https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/kubevirt-cr.yaml
export VERSION=$(curl -s https://api.github.com/repos/kubevirt/containerized-data-importer/releases | grep tag_name | grep -v -- '-rc' | head -1 | awk -F': ' '{print $2}' | sed 's/,//' | xargs)
curl -LkO https://ghproxy.net/github.com/kubevirt/containerized-data-importer/releases/download/$VERSION/cdi-operator.yaml
curl -LkO https://ghproxy.net/github.com/kubevirt/containerized-data-importer/releases/download/$VERSION/cdi-cr.yaml

check if it works

1
2
3

kubectl get pods -n kubevirt
kubectl get vmi

install virtctl

1
2
3
4
5

# get the download address from: https://github.com/kubevirt/kubevirt/releases/
curl -LkO <download_address>
sudo mv <downloaded_binary> /usr/bin/virtctl
sudo chmod +x /usr/bin/virtctl

create vm (already with tun device inside)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

# init_vm.yaml
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
name: <vm_name>
spec:
runStrategy: Always
template:
spec:
terminationGracePeriodSeconds: 30
dnsConfig:
nameservers:
- 8.8.8.8
domain:
resources:
requests:
memory: 1024M
devices:
disks:
- name: containerdisk
disk:
bus: virtio
- name: emptydisk
disk:
bus: virtio
- disk:
bus: virtio
name: cloudinitdisk
volumes:
- name: containerdisk
containerDisk:
image: kubevirt/fedora-cloud-container-disk-demo:latest
- name: emptydisk
emptyDisk:
capacity: "2Gi"
- name: cloudinitdisk
cloudInitNoCloud:
userData: |-
#cloud-config
password: fedora
chpasswd: { expire: False }

apply it to run the vm

1
2
3
4

kubectl apply -f init_vm.yaml
kubectl get vm
kubectl get vmi

to interact with the vm

1
2
3
4
5
6
7

virtctl console <vm_name>
# ssh config is stored at: ~/.ssh/kube_known_hosts
virtctl ssh <user>@<vm_name>
# run this under gui, with `vncviewer` in path
# install with `apt install tigervnc-viewer`
virtctl vnc <vm_name>

PM2 daemonize any app

remember to set unbuffered output flag -u before running any python script, otherwise there will be no output in pm2 log

install and setup pm2 as daemon:

1
2
3
4
5

sudo apt install npm
sudo npm config set registry https://registry.npmmirror.com
sudo npm install -g pm2
pm2 startup

run program as daemon:

1
2
3
4
5
6
7

pm2 start -n <process_name> <executable_name> -- <process_arguments>
# save all processes
pm2 save
pm2 monit
pm2 logs
pm2 ls

It is not advised to do so with dual WiFi connections, which is a pain in the ass in daily usage (only one of them will be used at a time).

Ethernet and WiFi dual connections seem fine with firejail but failed with dante.

Use firejail

1
2

sudo firejail --net=wlan0 --ip=dhcp --noprofile <program cmd>

Use dante and proxychains-ng

1
2

sudo apt install dante-server proxychains-ng

Now edit the dante config file at /etc/dante.conf:

1
2
3
4
5
6
7
8
9
10
11
12
13

internal: eth0 port = 1080
external: wlan0
socksmethod: username
user.privileged: root
user.unprivileged: nobody
user.libwrap: nobody
client pass {
from: 0/0 to: 0/0
}
socks pass {
from: 0/0 to: 0/0
}

Run the daemon by:

1
2

danted

Find the [ProxyList] section and add the following line in /etc/proxychains.conf:

1
2

socks5 127.0.0.1 1080 root <root_password>

Run the program with proxychains-ng:

1
2

proxychains <program cmd>

You can test your configuration like:

1
2

curl -x socks5://root:root@127.0.0.1:1080 https://www.baidu.com

If you run danted like systemctl start danted, you can configure a separate user for authentication. You have to change /etc/danted.conf and /etc/proxychains.conf accordingly.

1
2
3

sudo useradd -r -s /bin/false your_dante_user
sudo passwd your_dante_user

Fcitx setup

Fcitx contains google-pinyin, which is the most reliable Chinese input method of all times.

You need to configure it after installation like described here.

If all methods failed, please consider use tools like systemd or “Session and Startup” to force running command fcitx after startup.

im2latex-tensorflow sucks, looking for alternatives

training on gpu is intensive and will occasionally burn hardware if not careful, doing this on kaggle or modify the software to stop training when gpu goes hot, but we are using trainer here

harvard nlp showcase

for those doesn’t provide pretrained models:

im2latex in tensorflow, with makefile support, run on tensorflow v1 and python3

im2latex in pytorch, more recent. the dataset has relocated to here according to official website

install or run python2.7 to run im2latex-tensorflow

you may need to adapt our modified code to load the weights and test the result against our image.

it is reported the performance is poor. maybe it does not worth trying.

download tensorflow 0.12.0 for macos here

visit here to get all miniconda installers

to install on macos, download the installer here

some tutorial here about libmagic as bonus tips

1
2
3
4
5

CONDA_SUBDIR=osx-64 conda create -n py27 python=2.7  # include other packages here
# ensure that future package installs in this env stick to 'osx-64'
conda activate py27
conda config --env --set subdir osx-64

after that, do this to get pip on python2.7 (rosetta2)

1
2
3

curl https://bootstrap.pypa.io/pip/2.7/get-pip.py -o get-pip.py
python get-pip.py

install tensorflow version below 1, and doing this can be far more easier on linux. maybe we should do this in conda virtual enviorment to prevent conflicts.

we are doing this for the original lua implementation of im2markup

it works!

download libcudnn5 for torch

remember to activate torch enviorment by exporting the path to some shell script

difference between cudamalloc and cudamallocasync, and that’s some copying and pasting about some generalized template of memory manager function

qt4 uses CRLF so convert all text files using dos2unix

need to hack qt4 files to build qt4

hack luarocks to allow install from local spec file and download repo from github via https

hack some lua torch file to be compatible with cuda11

about c++ tweaks:

add ‘+’ to force type inference

force type conversion by using brackets

some macro to disable some blocks of code

Hugo the blog generator

install hugo:

https://gohugo.io/getting-started/installing/

install hugo from snap:

snap install hugo –channel=extended

install on debian/ubuntu:

apt-get install hugo

hugo themes:

https://themes.gohugo.io

how to create hugo blog:

https://blog.csdn.net/cumi7754/article/details/108101980