Autonomous Machines & Society.

2022-09-26
Commandline Search Engine Bridger

search commands on commandline

1
2
3
4
5
6
7
man -k <keywords>
apt search <keywords>
brew search <keywords>
npm search <keywords>
searchsploit <keywords>
msfconsole -x "search <keywords>; exit"

nosql/nosqlite key-value json like document store databases, sqlite high level wrappers

this guy makes database related libraries.


lsm-db fast key-value store using sqlite 4

unqlite Python bindings for the UnQLite embedded NoSQL database

tinydb json-oriented, mongo alike database, which is a in-memory database

litedb NoSQL Python database written for ease of use/performance.

pylite sqlite3 lightweight wrapper

reason to develop this

you are short of brain power. short of time to perspect and investigate.

many platforms now have recommendation engines, but they do not have powerful semantic search tools. what a pity. maybe i am interested in some ‘unseen’ stuff, but i want to get the thing that i currently need! fail to do so will limit my productivity.

sentence embeddings

difference between ‘symmetric’ and ‘asymmetric’ retrieval questions from sbert.net:

symmetric means similar, asymmetric usually means question to answer.

top 4 sentence embedding techniques using python

sentence to vector

tutorial: sentence vector word2vec

sentence2vec based on word2vec

similarity search and clustering

facebook faiss

hnswlib Header-only C++/python library for fast approximate nearest neighbors

spotify annoy Approximate Nearest Neighbors in C++/Python optimized for memory usage and loading/saving to disk

problems

  • enable copy/pasting/selection in console

  • make reversible/auto-cleanup feature when copy and pasting

  • reverse stemming and keywords highlighting

  • memory efficient embedding querying/storage via sql or binary format

txtai store to sqlite: Build an Embeddings index from a data source

  • make “complete” excerpt by looking ahead and backward to find the closest sentence start/stop and/or use gpt completion method instead? still need punctual or sentence start/stop index identification.

offline

terminal interface builder

rich and tutorial

textual

urwid

plie

ai assisted search engine libraries

jina-ai and docarray

txtai

typesense

zinc search

fuzzy search

fuzzy phonic toolkit

traditional search engine libraries

luceneplusplus lucene in c++

pythonql as extension of python syntax, able to search data in python data structure.

jq and pyjq as json search engine, jqterm as jq repl

fq jq for binary formats

a curated search engine list

scout sqlite based full text search

whoosh with bm25 support

python-searchengine and the tutorial

online

surfraw, or just s, is for some common parameter prefixes for searching on common websites. will open GUI browser or cli browser if configured with one.

surfraw supported:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
W               -- Activate Surfraw defined web-browser
acronym -- Look for acronyms definitions (www.acronymfinder.com)
ads -- Search SAO/NASA Astrophysics Data System
alioth -- Search Alioth (alioth.debian.org)
amazon -- Search the amazon.com bookstore
archpkg -- Search Arch Linux Packages (www.archlinux.org/packages/)
archwiki -- Search the Arch Linux Wiki
arxiv -- Search arXiv E-Print Archive for articles
ask -- Question the web using Ask Jeeves (www.ask.com)
aur -- Search aur.archlinux.org for PKGBUILDs
austlii -- Search Australian Law docs (www.austlii.edu.au)
bbcnews -- Search BBC News (news.bbc.co.uk)
bing -- Search the web using Microsoft's Bing (www.bing.com)
bookfinder -- Search for books using www.bookfinder.com
bugmenot -- Bypass compulsory web registration with bugmenot.com
bugzilla -- Search for bugs on Bugzilla bugtrackers
cablesearch -- search for leaked diplomatic communications
cia -- Search CIA documents at www.cia.gov
cisco -- Search Cisco documentation (www.cisco.com)
cite -- Search computer science papers (citeseerx.ist.psu.edu)
cliki -- Search the common lisp wiki
cnn -- Search on CNN (cnn.com)
comlaw -- Search Australian Law using Comlaw (www.comlaw.gov.au)
commandlinefu -- Search on www.commandlinefu.com
ctan -- Search the Comprehensive TeX Archive Network (ctan.org)
currency -- Convert currencies with the Universal Currency Converter (www.xe.net/ucc)
cve -- Search for CAN assignments in CVE
debbugs -- Search the debian BTS (bugs.debian.org)
debcodesearch -- Search debian source code
debcontents -- Search contents of debian/ubuntu packages (packages.debian.org/packages.ubuntu.com)
deblists -- Search debian mailing lists (lists.debian.org/search.html)
deblogs -- Show changelogs for a package in Debian main (changelogs.debian.net)
debpackages -- Search debian/ubuntu packages (packages.debian.org/packages.ubuntu.com)
debpkghome -- Visit the home page for a Debian package
debpts -- Search the Debian Package Tracking System (packages.qa.debian.org)
debsec -- Search the Debian Security Tracker for CVE ids or package names
debvcsbrowse -- Browse the VCS repository for a Debian package
debwiki -- Search the Debian Wikis (wiki.debian.org & women.debian.org/wiki)
deja -- Search usenet using Google Groups (groups.google.com)
deli -- Search Delicious bookmarks
discogs -- Search the Discogs database of music information (www.discogs.com)
dmoz -- Search the Open Directory Project web directory (dmoz.org)
duckduckgo -- Securely search the web using duckduckgo (www.duckduckgo.com)
ebay -- Search the Ebay auction site
etym -- Look up word origins at www.etymonline.com
excite -- Search on Excite (www.excite.com)
f5 -- Search F5 related information (www.f5.com)
finkpkg -- Search Fink packages (pdb.finkproject.org)
foldoc -- The Free On-Line Dictionary Of Computing (foldoc.org)
freebsd -- Search FreeBSD related information (www.freebsd.org)
freedb -- Search for cd track listings in FreeDB (www.freedb.org)
freshmeat -- Search Freshmeat (www.freshmeat.net)
fsfdir -- Search the FSF/UNESCO Free Software Directory (directory.fsf.org)
gcache -- Search the web using Google cache (www.google.com)
genbugs -- Search the Gentoo bug tracker (bugs.gentoo.org)
genportage -- Search gentoo-portage.com for packages
github -- Search GitHub (https://github.com)
gmane -- Search mailing list with gmane (gmane.org)
google -- Search the web using Google (www.google.com)
gutenberg -- Search for books on Project Gutenberg (gutenberg.org)
imdb -- Search the Internet Movie Database (www.imdb.com)
ixquick -- Search the web using ixquick [HTTPS] (www.ixquick.com)
jamendo -- Search Jamendo: free music with Creative Commons licenses (www.jamendo.com)
javasun -- Search Java API docs (java.sun.com)
jquery -- Search the jQuery documentation (api.jquery.com)
l1sp -- Search lisp documentation
lastfm -- Search last.fm
leodict -- Search Leo's German <-> English dictionary (dict.leo.org)
lsm -- Search the Linux Software Map
macports -- Search macports packages (macports.org)
mathworld -- Search Wolfram MathWorld
mdn -- Search the mozilla developer network (developer.mozilla.org)
mininova -- Search the mininova bittorent source.
musicbrainz -- Search MusicBrainz (musicbrainz.org)
mysqldoc -- Search mysql documentation (dev.mysql.com)
netbsd -- Search NetBSD related information (www.netbsd.org)
nlab -- Search the nLab wiki (http://ncatlab.org)
ntrs -- Search the NASA Technical Report Server
openbsd -- Search OpenBSD related information (www.openbsd.org)
openports -- search openports for OpenBSD packages
opensearch -- Search an OpenSearch-enabled website
oraclesearch -- Search an OpenSearch-enabled website
pasearch -- Search the unofficial Penny Arcade archives (pipefour.org/pa)
pgdoc -- Search postgres documentation (www.pgdoc.com)
pgpkeys -- Search the PGP key database
phpdoc -- Search php documentation (php.net)
pin -- Search Pinboard bookmarks (http://pinboard.in)
piratebay -- Search The Pirate Bay (http://thepiratebay.org)
priberam -- Look up word in Priberam online dictionary (www.priberam.pt/dlpo)
pubmed -- Search medical/molbio databases (www.ncbi.nlm.nih.gov)
rae -- Busca en el diccionario de la Real Academia de la Lengua Española (Spanish Dictionary)
rfc -- Search RFCs (internet standards documents)
rhyme -- Search for rhymes et al using Lycos Rhyme (rhyme.lycos.com)
rpmsearch -- Search for RPMs in various distros
scholar -- Search Google Scholar (scholar.google.com)
scicom -- Search Scientific Commons
scirus -- Search for science using Scirus (scirus.com)
scpan -- Search the Comprehensive Perl Archive Network (search.cpan.org)
searx -- Search using searx metasearch engine instances (searx.me)
slashdot -- Search stories on Slashdot (www.slashdot.org)
slinuxdoc -- Search entries in LDP (www.linuxdoc.org)
sourceforge -- Search SourceForge (www.sourceforge.net)
springer -- Search Springer for Books and Articles
stack -- Search Stack Overflow
stockquote -- Get a single stock quote (multiple providers)
thesaurus -- Look up word in Merriam-Webster's Thesaurus (www.m-w.com)
translate -- Translate human languages
urban -- Search urbandictionary.com for a definition
w3css -- Validate a CSS URL with the w3c CSS validator (jigsaw.w3.org/css-validator)
w3html -- Validate a web page URL with the w3c validator (validator.w3.org)
w3link -- Check web page links with the w3c linkchecker (validator.w3.org/checklink)
w3rdf -- Validate a RDF URL with the w3c RDF validator (validator.w3.org)
wayback -- Search The Internet Archive's Wayback Machine for a URL (archive.org)
webster -- Look up word in Merriam-Webster's Dictionary (www.m-w.com)
wetandwild -- Real time weather information (many sources)
wikipedia -- Search the free encyclopedia wikipedia
woffle -- Search the web using Woffle (localhost:8080)
wolfram -- Ask questions of the computational knowledge engine
worldwidescience -- Search for science with www.worldwidescience.org
yacy -- Search YaCy P2P search, including ScienceNet
yahoo -- Search Yahoo categories (www.yahoo.com)
yandex -- Search the web using Yandex (yandex.ru)
youtube -- Search YouTube (www.youtube.com)
yubnub -- Use the social command-line for the web (yubnub.org)

S supported:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
500px
8tracks
aliexpress
allocine
amazon
archpkg
archwiki
ardmediathek
arstechnica
arxiv
atmospherejs
aur
baidu
bandcamp
bgr
bigbasket
bing
brave
buzzfeed
cnn
codepen
coursera
cplusplus
cppreference
crates
crunchyroll
debianpkg
dict
digg
diigo
dockerhub
dribbble
duckduckgo
dumpert
ecosia
engadget
explainshell
facebook
flickr
flipkart
foursquare
freebsdman
freshports
gibiru
giphy
gist
github
gmail
go
godoc
goodreads
google
googledocs
googleplus
hackernews
idealo
ietf
ifttt
imdb
imgur
inbox
instagram
kaufda
kickasstorrents
libgen
linkedin
lmgtfy
macports
magnetdl
mdn
medium
metacpan
msdn
naver
netflix
nhaccuatui
npm
npmsearch
npr
nvd
openbsdman
overstock
packagist
presearch
phandroid
php
pinterest
postgresql
python
quora
qwant
reddit
regex
rottentomatoes
rubygems
shodan
soundcloud
spotify
stackoverflow
steam
taobao
thepiratebay
theregister
torrentz
twitchtv
twitter
ultimateguitar
unity3d
upcloud
vimeo
wikipedia
wolframalpha
yahoo
yandex
youtube
zdf
zhihu

Read More

2022-09-25
微软小冰 机器人 用Requests库访问最好

notice

when using this, first extract topic from recent chats or group name, then send message.

or you just use the default topic. whatever.

client

1
2
3
4
5
6
>>> import requests
>>> r = requests.get("http://localhost:8735/chat",params={"topic":"python","message":"吃了没有"})
>>> r.json()
{'msg': 'success', 'reply': '你这么一说,我好像是有点饿'}
>>> exit()

server

location: /root/Desktop/works/pyjom/tests/microsoft_xiaobing_conversation_bing/chat_with_session_id.js

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
var request = require("request");
// var mysqld = require("./mysql");
// const { init: initDB, Counter, Chatid } = require("./db");
function getRequestId() {
return (ot() + ot() + ot() + ot() + ot() + ot() + ot() + ot()).toLowerCase();
}
const sleep = (ms) => {
return new Promise(resolve => setTimeout(resolve, ms))
}
function ot() {
return (((1 + Math.random()) * 65536) | 0).toString(16).substring(1);
}
function i(n, i) {
for (
var s, c, e = 4, l = i.length / e - 1, r = [
[],
[],
[],
[]
], o = 0; o < 4 * e; o++
)
r[o % 4][Math.floor(o / 4)] = n[o];
for (r = t(r, i, 0, e), s = 1; s < l; s++)
(r = u(r, e)), (r = f(r, e)), (r = h(r, e)), (r = t(r, i, s, e));
for (
r = u(r, e), r = f(r, e), r = t(r, i, l, e), c = new Array(4 * e), o = 0; o < 4 * e; o++
)
c[o] = r[o % 4][Math.floor(o / 4)];
return c;
}
function u(n, t) {
for (var r, i = 0; i < 4; i++)
for (r = 0; r < t; r++) n[i][r] = o[n[i][r]];
return n;
}
function f(n, t) {
for (var i, u = new Array(4), r = 1; r < 4; r++) {
for (i = 0; i < 4; i++) u[i] = n[r][(i + r) % t];
for (i = 0; i < 4; i++) n[r][i] = u[i];
}
return n;
}
function h(n) {
for (var t, r, u, i = 0; i < 4; i++) {
for (t = new Array(4), r = new Array(4), u = 0; u < 4; u++)
(t[u] = n[u][i]),
(r[u] = n[u][i] & 128 ? (n[u][i] << 1) ^ 283 : n[u][i] << 1);
n[0][i] = r[0] ^ t[1] ^ r[1] ^ t[2] ^ t[3];
n[1][i] = t[0] ^ r[1] ^ t[2] ^ r[2] ^ t[3];
n[2][i] = t[0] ^ t[1] ^ r[2] ^ t[3] ^ r[3];
n[3][i] = t[0] ^ r[0] ^ t[1] ^ t[2] ^ r[3];
}
return n;
}
function t(n, t, i, r) {
for (var f, u = 0; u < 4; u++)
for (f = 0; f < r; f++) n[u][f] ^= t[i * 4 + f][u];
return n;
}
function e(n) {
for (var t = 0; t < 4; t++) n[t] = o[n[t]];
return n;
}
function c(n) {
for (var i = n[0], t = 0; t < 3; t++) n[t] = n[t + 1];
return (n[3] = i), n;
}
function rr(n) {
for (
var h,
i,
o = 4,
r = n.length / 4,
s = r + 6,
f = new Array(o * (s + 1)),
u = new Array(4),
t = 0; t < r; t++
)
(h = [n[4 * t], n[4 * t + 1], n[4 * t + 2], n[4 * t + 3]]), (f[t] = h);
for (t = r; t < o * (s + 1); t++) {
for (f[t] = new Array(4), i = 0; i < 4; i++) u[i] = f[t - 1][i];
if (t % r == 0)
for (u = e(c(u)), i = 0; i < 4; i++) u[i] ^= l[t / r][i];
else r > 6 && t % r == 4 && (u = e(u));
for (i = 0; i < 4; i++) f[t][i] = f[t - r][i] ^ u[i];
}
return f;
}
function r(n) {
for (
var h,
i,
o = 4,
r = n.length / 4,
s = r + 6,
f = new Array(o * (s + 1)),
u = new Array(4),
t = 0; t < r; t++
)
(h = [n[4 * t], n[4 * t + 1], n[4 * t + 2], n[4 * t + 3]]), (f[t] = h);
for (t = r; t < o * (s + 1); t++) {
for (f[t] = new Array(4), i = 0; i < 4; i++) u[i] = f[t - 1][i];
if (t % r == 0)
for (u = e(c(u)), i = 0; i < 4; i++) u[i] ^= l[t / r][i];
else r > 6 && t % r == 4 && (u = e(u));
for (i = 0; i < 4; i++) f[t][i] = f[t - r][i] ^ u[i];
}
return f;
}
function a(n, t, u) {
var c = 16,
a,
y,
l,
w,
o,
e,
f,
nt;
if (!(u == 128 || u == 192 || u == 256)) return "";
for (n = s(n), t = s(t), a = u / 8, y = new Array(a), f = 0; f < a; f++)
y[f] = isNaN(t.charCodeAt(f)) ? 0 : t.charCodeAt(f);
l = i(y, rr(y));
l = l.concat(l.slice(0, a - 16));
var h = new Array(c),
k = new Date().getTime(),
tt = k % 1e3,
it = Math.floor(k / 1e3),
rt = Math.floor(Math.random() * 65535);
for (f = 0; f < 2; f++) h[f] = (tt >>> (f * 8)) & 255;
for (f = 0; f < 2; f++) h[f + 2] = (rt >>> (f * 8)) & 255;
for (f = 0; f < 4; f++) h[f + 4] = (it >>> (f * 8)) & 255;
for (w = "", f = 0; f < 8; f++) w += String.fromCharCode(h[f]);
var ut = rr(l),
b = Math.ceil(n.length / c),
d = new Array(b);
for (o = 0; o < b; o++) {
for (e = 0; e < 4; e++) h[15 - e] = (o >>> (e * 8)) & 255;
for (e = 0; e < 4; e++) h[11 - e] = (o / 4294967296) >>> (e * 8);
var ft = i(h, ut),
g = o < b - 1 ? c : ((n.length - 1) % c) + 1,
p = new Array(g);
for (f = 0; f < g; f++)
(p[f] = ft[f] ^ n.charCodeAt(o * c + f)),
(p[f] = String.fromCharCode(p[f]));
d[o] = p.join("");
}
return (nt = w + d.join("")), v(nt);
}
function v(n) {
for (
var i = "0x",
r = [
"0",
"1",
"2",
"3",
"4",
"5",
"6",
"7",
"8",
"9",
"a",
"b",
"c",
"d",
"e",
"f",
],
t = 0; t < n.length; t++
)
i += r[n.charCodeAt(t) >> 4] + r[n.charCodeAt(t) & 15];
return i;
}
function s(n) {
var t = n.replace(/[\u0080-\u07ff]/g, function(n) {
var t = n.charCodeAt(0);
return String.fromCharCode(192 | (t >> 6), 128 | (t & 63));
});
return t.replace(/[\u0800-\uffff]/g, function(n) {
var t = n.charCodeAt(0);
return String.fromCharCode(
224 | (t >> 12),
128 | ((t >> 6) & 63),
128 | (t & 63)
);
});
}
var o = [
99, 124, 119, 123, 242, 107, 111, 197, 48, 1, 103, 43, 254, 215, 171, 118,
202, 130, 201, 125, 250, 89, 71, 240, 173, 212, 162, 175, 156, 164, 114,
192, 183, 253, 147, 38, 54, 63, 247, 204, 52, 165, 229, 241, 113, 216, 49,
21, 4, 199, 35, 195, 24, 150, 5, 154, 7, 18, 128, 226, 235, 39, 178, 117, 9,
131, 44, 26, 27, 110, 90, 160, 82, 59, 214, 179, 41, 227, 47, 132, 83, 209,
0, 237, 32, 252, 177, 91, 106, 203, 190, 57, 74, 76, 88, 207, 208, 239, 170,
251, 67, 77, 51, 133, 69, 249, 2, 127, 80, 60, 159, 168, 81, 163, 64, 143,
146, 157, 56, 245, 188, 182, 218, 33, 16, 255, 243, 210, 205, 12, 19, 236,
95, 151, 68, 23, 196, 167, 126, 61, 100, 93, 25, 115, 96, 129, 79, 220, 34,
42, 144, 136, 70, 238, 184, 20, 222, 94, 11, 219, 224, 50, 58, 10, 73, 6,
36, 92, 194, 211, 172, 98, 145, 149, 228, 121, 231, 200, 55, 109, 141, 213,
78, 169, 108, 86, 244, 234, 101, 122, 174, 8, 186, 120, 37, 46, 28, 166,
180, 198, 232, 221, 116, 31, 75, 189, 139, 138, 112, 62, 181, 102, 72, 3,
246, 14, 97, 53, 87, 185, 134, 193, 29, 158, 225, 248, 152, 17, 105, 217,
142, 148, 155, 30, 135, 233, 206, 85, 40, 223, 140, 161, 137, 13, 191, 230,
66, 104, 65, 153, 45, 15, 176, 84, 187, 22,
],
l = [
[0, 0, 0, 0],
[1, 0, 0, 0],
[2, 0, 0, 0],
[4, 0, 0, 0],
[8, 0, 0, 0],
[16, 0, 0, 0],
[32, 0, 0, 0],
[64, 0, 0, 0],
[128, 0, 0, 0],
[27, 0, 0, 0],
[54, 0, 0, 0],
];
// n.encrypt = a
async function iceAI_word(
// ToUserName,
// FromUserName,
// CreateTime,
// MsgType,
Content,
config
// MsgId,
) {
await sleep(1000);
// for whatever reason you have to wait for this long.
try{
var wquery = a(Content, "3d9d5f16-5df0-43d7-902e-19274eecdc41", 256);
console.log("encrypt:" + wquery);
// let config = {};
// if ((await mysqld.isHaveChatIdIn(fromQQ)) == true) {
// console.log("没有chatid,获取新id")
// config = await mysqld.getChatId(fromQQ);
// } else {
// config = await newChatId(fromQQ);
// }
if (config) {
console.log("config:" + config);
} else {
console.log('no config for xiaoice chat.')
return;
}
var h = {
zoTextResponse: "",
zoIsGCSResponse: false,
zoSearchQuery: "hhh",
zoTimestampUtc: "",
zoIsStartOfSession: true,
zoRequestId: getRequestId(),
conversationId: config.conversationId,
query: { NormalizedQuery: wquery },
from: "chatbox",
traceId: config.traceId,
};
var url = "https://cn.bing.com/english/zochatv2?cc=cn&ensearch=0";
// {"zoTextResponse":"","zoIsGCSResponse":"false","zoSearchQuery":"123","zoTimestampUtc":"","zoIsStartOfSession":"true","zoRequestId":"ff90e6f70a6048d4fe5cc3c3327bbd32","conversationId":"4a91fb33-73f7-43d4-b7b6-ba86a16e32fb","query":{"NormalizedQuery":"0x23028811be44f661169365"},"from":"chatbox","traceId":"B224B190F87941CD94AD0AC31A189D30"}
let result = await getContents({
url: url,
method: "POST",
headers: {
"content-type": "text/plain;charset=UTF-8",
origin: "https://cn.bing.com",
referer: "https://cn.bing.com/search?q=123&form=QBLH&sp=-1&pq=123&sc=6-3&qs=n&sk=&cvid=566F001FDA424EEB805E1C175363B5AE",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36",
Connection: "keep-alive",
},
body: JSON.stringify(h),
});
// {"content":"嘿 啾 嘿 啾啊","type":1,"delayContents":null,"entityInfo":[{"Entity":"嘿 啾 嘿 啾啊","IsEntity":false}],"target":"b","history":null,"hasClientIdinMem":true,"needSayHello":false,"isHookStr":false,"showChatBox":true,"metadata":{"AnswerFeed":"RandomChitChatService","EmotionInfo":"{\"EmotionClassificationInfo\":[{\"Category\":\"Sad\",\"Score\":0.0651140139},{\"Category\":\"Happy\",\"Score\":0.139467061},{\"Category\":\"Surprise\",\"Score\":0.176786855},{\"Category\":\"Angry\",\"Score\":0.358794},{\"Category\":\"Disgust\",\"Score\":0.2598381}],\"NeutralScore\":0.9992748,\"DomainInMatchScenario\":\"None\"}"}}
result = JSON.parse(result);
if (result.content) {
var reply = result.content;
reply = reply.replace("小冰", "小姝");
var message = 1;
var unuseless =
"看的我一脸懵逼,都开始怀疑我的智商了。哎呀,不好意思,我刚刚好像走神了,感觉你知道的挺多的呢,额,我现在也不知道该说些什么,这个…不太好说啊,我语文不太好,不确定是不是懂了你的意思,刚刚不小心溜号了,真是不好意思";
if (unuseless.indexOf(reply) != -1) {
console.log('xiaoice is returning useless reply', reply)
// message = 2;
// Log.trace("iceAi have unuseless message");
// request(
// {
// url:
// "http://api.qingyunke.com/api.php?key=free&appid=0&msg=" +
// encodeURIComponent(msg2),
// method: "GET",
// },
// function (error, response, body) {
// var result = JSON.parse(body);
// reply = result.content;
// var logtext = "";
// return;
// }
// );
} else {
return reply;
}
}
} catch(e){
console.log('ERROR FETCHING XIAOBING CHAT',e)
// will return nothing.
// sleep for 1 second?
// would you sleep for a while?
}
}
async function newChatId(query) {
var options = options || {};
var httpOptions = {
url: "https://cn.bing.com/search?q="+query+"&form=QBLH&rdr=1&rdrig=E8F3C1A722454F949CCC4B98C4570A4A",
method: "get",
timeout: 1000,
headers: {
accept: "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"accept-language": "zh-CN,zh;q=0.9",
"cache-control": "max-age=0",
"sec-ch-ua": '" Not A;Brand";v="99", "Chromium";v="102", "Google Chrome";v="102"',
"sec-ch-ua-arch": '"x86"',
"sec-ch-ua-bitness": '"64"',
"sec-ch-ua-full-version": '"102.0.5005.63"',
"sec-ch-ua-mobile": "?0",
"sec-ch-ua-model": '""',
"sec-ch-ua-platform": '"Windows"',
"sec-ch-ua-platform-version": '"10.0.0"',
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "same-origin",
cookie: "MUID=005F25E7699168532D05342768F769B3; MUIDB=005F25E7699168532D05342768F769B3; _EDGE_V=1; SRCHD=AF=NOFORM; SRCHUID=V=2&GUID=31127A3BD4B84FF08E8E51EEEA34857F&dmnchg=1; _UR=QS=0&TQS=0; _HPVN=CS=eyJQbiI6eyJDbiI6MSwiU3QiOjAsIlFzIjowLCJQcm9kIjoiUCJ9LCJTYyI6eyJDbiI6MSwiU3QiOjAsIlFzIjowLCJQcm9kIjoiSCJ9LCJReiI6eyJDbiI6MSwiU3QiOjAsIlFzIjowLCJQcm9kIjoiVCJ9LCJBcCI6dHJ1ZSwiTXV0ZSI6dHJ1ZSwiTGFkIjoiMjAyMi0wNi0xMVQwMDowMDowMFoiLCJJb3RkIjowLCJHd2IiOjAsIkRmdCI6bnVsbCwiTXZzIjowLCJGbHQiOjAsIkltcCI6NH0=; SUID=M; SRCHUSR=DOB=20220611&T=1659599964000&TPC=1659599966000; ZHCHATSTRONGATTRACT=TRUE; ZHCHATWEAKATTRACT=TRUE; _EDGE_S=SID=05C5058B7100688001DB147D702E698C; _SS=SID=05C5058B7100688001DB147D702E698C; _tarLang=default=zh-Hans; _TTSS_IN=hist=WyJlbiIsImF1dG8tZGV0ZWN0Il0=; _TTSS_OUT=hist=WyJ6aC1IYW5zIl0=; ipv6=hit=1659603639345&t=4; SNRHOP=I=&TS=; SRCHHPGUSR=SRCHLANG=zh-Hans&BRW=NOTP&BRH=S&CW=599&CH=657&SW=1366&SH=768&DPR=1&UTC=480&DM=0&PV=0.3.0&BZA=0&HV=1659600073&WTS=63795196764",
"sec-fetch-user": "?1",
accept: "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"accept-language": "zh-CN,zh;q=0.9",
"cache-control": "max-age=0",
"upgrade-insecure-requests": "1",
Referer: "referer: https://cn.bing.com/search?q="+query+"&form=QBLHCN&sp=-1&pq=a&sc=6-1&qs=n&sk=&cvid=A91AB41228AD45E694D5F2EEBF87FE70",
"Referrer-Policy": "strict-origin-when-cross-origin",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.63 Safari/537.36",
},
};
let body = await getContents(httpOptions)
//console.log(body)
if (body.indexOf("conversationId") == -1) {
console.log("请求chatid失败");
return;
}
console.log(body.indexOf("conversationId"));
console.log(body.indexOf("iframeTalkStatus"));
let config =
'{"' +
body.substring(
body.indexOf("conversationId"),
body.indexOf("iframeTalkStatus")
) +
'":""}';
config = JSON.parse(config);
console.log(config);
// mysqld.addChatId(fromQQ, config);
return config;
}
async function getAuth(opts, redis) {
cookie = await post(opts);
//redis.set("ice_cookie", cookie)
log.info("new cookie:" + cookie);
return cookie;
}
function post(opts) {
return new Promise((resolve, reject) => {
request(opts, function(error, response) {
if (error) throw new Error(error);
if (response.statusCode != "200") {
console.log("requestCode:" + response.statusCode);
}
console.log("requestCode:" + response.statusCode);
var responseCookies = response.headers["set-cookie"];
console.log(response.body);
var requestCookies = "";
for (var i = 0; i < responseCookies.length; i++) {
var oneCookie = responseCookies[i];
oneCookie = oneCookie.split(";");
requestCookies = requestCookies + oneCookie[0] + ";";
}
resolve(requestCookies);
});
});
}
function getContents(opts) {
return new Promise((resolve, reject) => {
request(opts, function(error, response) {
if (error) reject(error);
if (response.statusCode != "200") {
console.log("requestCode:" + response.statusCode);
}
console.log("requestCode:" + response.statusCode);
var responseCookies = response.headers["set-cookie"];
resolve(response.body);
});
});
}
// module.exports = { iceAI_word };
// let test_request = "不会吧"
// let test_request = "python"
const http = require('http');
function getQueryParams(reqUrl) {
current_url = new URL('http://localhost' + reqUrl)
params = current_url.searchParams
console.log('query parameters:', params)
return params
}
let topic_chatId_dict = {}
const requestListener = function (req, res){
console.log("________________________________________________")
console.log("REQUEST AT:", req.url, req.method)
if (req.url == "/") {
res.writeHead(200);
res.end('xiaoice chat server');
} else if (req.url.split("?")[0] == '/chat'){
callback = (result) => {
res.writeHead(200);
content = {"msg": 'success','reply': result}
res.end(JSON.stringify(content))
}
params = getQueryParams(req.url)
message =params.get("message")
topic = params.get("topic")
if (message==null){
message = "你好呀"
}
if (topic == null){
topic = "hhh"
}
console.log("MESSAGE:", message)
console.log("TOPIC:", topic)
if (topic_chatId_dict[topic] == null){
topic_chatId_dict[topic] = newChatId(topic)
}
chatId = topic_chatId_dict[topic]
if (chatId !=null){
response = iceAI_word(message, chatId)
response.then((content) => {
console.log("REAL RESPONSE:", content)
if (content !=null){
callback(content)
}else{
res.writeHead(401);
res.end(JSON.stringify({'msg':'empty response from microsoft xiaoice'}))
}
})
}else{
res.writeHead(401)
res.end(JSON.stringify({'msg':'error when getting chatid'}))
}
}else{
res.writeHead(400);
res.end('please use /chat?topic={topic}&message={message} to chat with xiaoice.')
}
}
const server = http.createServer(requestListener);
port = 8735
server.listen(port);
console.log('xiaoice server running on http://localhost:' + port);
// // these code are just for test.
// let test_request = "你吃了没有"
// // let test_request2 = "你吃了没有"
// query = 'python'
// let config = newChatId(query)
// response = iceAI_word(test_request, config) // automatically retry once. if keeping generating useless shits, we may decide to give it up?
// // it is a promise.
// // this is async shit.
// // what if there's some error?
// response.then((content) => {console.log("REAL RESPONSE:", content)})
// // REAL RESPONSE: 不想就不说了
// // console.log("RESPONSE:", response)
// // response = iceAI_word(test_request2, config)
// // console.log("RESPONSE:", response)

Read More

2022-09-25
涩图Api Setu Bot

notice

涩图可以发到qq群里面活跃气氛 可以发到专栏里面 可以添加到视频里面“截图选老婆”

涩图分为二次元和三次元涩图 皆要控制尺度

除了涩图之外,能够随时使用的番剧clip也是很重要的 如何获取“动漫名场面”呢

projects

https://github.com/opq-osc/OPQ-SetuBot 直接搜索的pixiv

二次元涩图api

https://api.lolicon.app/#/setu 注意不要r18的 要用nsfw检查一下至少

三次元涩图api

Read More

2022-09-25
Generate Publickey Again With Rsa Private Key

not possible. use personal access token as password instead.

cause the deploy public key does not allow duplicate public key, causing trouble for us to use the git repo sync tool.

1
2
3
4
PRIVATE_KEY_PATH=/Users/jamesbrown/.notable/id_rsa_original_backup
PUBKEY_PATH=/Users/jamesbrown/.notable/id_rsa.pub2
ssh-keygen -y -f $PRIVATE_KEY_PATH > $PUBKEY_PATH

Read More

2022-09-25
日站之随想

为什么要日站

因为自己电脑算力有限 要探索高级人工智能 要运行某些赚钱程序 必须免费白嫖别人的算力

日哪些站

到百度 各大搜索引擎找目标站点 扫描漏洞 不要打大站 先从小站打起走 往全自动化方向打起走

怎么日站

在一个沙箱下面日站 不要在root权限下面日

利用工具 目标站点IP 端口就直接填到工具里面就行 一边学工具一边日站

需要一个repo专门放有关的代码 同时可以同步到每个设备

需要一个可以搜索全文的搜索引擎 搜集我们已有的md文件 提取里面的链接 然后继续clone github的文件 继续搜集md文件 继续寻找链接

日站之后干什么

放传染性病毒 放挖币病毒 或者执行agi实验

Read More

2022-09-25
Introducing Lavis: A Library For Language-Vision Intelligence With Cross-Modal Implementations

lavis multimodal library for information retrieval and intelligence, can be used with jina

lavis has multiple cross-modal implementations

LAVIS - A Library for Language-Vision Intelligence

Read More

2022-09-24
Dark Web Search Engine

OnionSearch commandline all-in-one darknet search tool, install via pip3 install onionsearch (of course you need proxy)

Supporting:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
ahmia
darksearchio
onionland
notevil
darksearchenginer
phobos
onionsearchserver
torgle
onionsearchengine
tordex
tor66
tormax
haystack
multivac
evosearch
deeplink

https://dark.link/torch-darkweb-search-engine-links/

darknethub

onion.live list and search popular onion sites by name

https://darkweblinks.com/#Open-Source-Software

The onion dir

links are subjected to change or take down.

ahmia

the uncensored hidden wiki

tor links

parazite

torch

not evil

haystack one of the biggest

candle

Read More

2022-09-23
Image Blur Detection, Image Quality Assessment

blur detection, detect blur/nonblur area, score the image

traditional methods

score and output mask for blurry areas using laplacian

using fourier transform

This project outputs regions in an image which are sharp and blurry. In order to perform “OUT-OF-FOCUS” blur estimation, please refer to this repo

deeplearning blur image classification/scoring

Image-Quality-Detection

Blur-Image-Detection

Read More

2022-09-18
Trainable Bezier Curve Multiparameter Regressor

bezier

first, we need a bezier curve connects (0,0) and (1,1)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
import numpy as np
import bezier
def bezierCurve(start=(0,0), end=(1,1), skew=0):
# skew: (-0.5,0.5) otherwise this shit will look ugly.
assert skew >=-0.5
assert skew <=0.5
x_start, y_start = start
x_end, y_end = end
x_diff = x_end - x_start
y_diff = y_end - y_start
nodes1 = np.asfortranarray(
[
[x_start, x_diff * (0.5 + skew), x_end],
[y_start, y_diff * (0.5 - skew), y_end],
]
)
curve1 = bezier.Curve(nodes1, degree=2)
curve_params = {'x_start':x_start, 'x_diff':x_diff,'x_end':x_end}
return curve1, curve_params
def evaluateBezierCurve(input_value:float,curve, curve_params:dict)
x_start = curve_params['x_start']
x_end = curve_params['x_end']
assert x_start <= input_value
assert x_end >= input_value
x_diff = curve_params['x_diff']
s = (input_value - x_start)/x_diff
points = curve.evaluate(s)
# we only get the single point.
point = points.T[0]
x,y = point
result = y
return result

then, we define our very recursive or flexible regressor:

1
2
3
4
5
6
7
8
9
def multiParameterExponentialNetwork(*args, input_bias=0.05, curve_function=bezierCurve, curve_function_kwargs = {'start':(0,0),'end':(1,1)'skew':0}, evaluate_function=evaluateBezierCurve):
curve, curve_params = curve_function(**curve_function_kwargs)
value = evaluate_function(input_bias, curve, curve_params)
for index, input_value in enumerate(args):
apply_list = [input_value]*(index+1)
for apply_item in apply_list:
value += (1-value)*function(apply_item, **function_kwargs)
return value

Read More

2022-09-18
Image Resize, Image Padding, Image Scanning

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
def scanImageWithWindowSizeAutoResize(
image,
width,
height,
return_direction=False,
threshold=0.1, # minimum 'fresh' area left for scanning
): # shall you use torch? no?
shape = image.shape
assert len(shape) == 3
ih, iw, channels = shape
targetWidth = max(width, math.floor(iw * height / ih))
targetHeight = max(height, math.floor(ih * width / iw))
resized = cv2.resize(
image, (targetWidth, targetHeight), interpolation=cv2.INTER_CUBIC
)
# determine scan direction here.
imageSeries = []
if targetWidth / targetHeight == width / height:
imageSeries = [resized] # as image series.
direction = None
elif targetWidth / targetHeight < width / height:
direction = "vertical"
# the scanning is along the vertical axis, which is the height.
index = 0
while True:
start, end = height * index, height * (index + 1)
if start < targetHeight:
if end > targetHeight:
if 1 - (end - targetHeight) / targetHeight >= threshold:
end = targetHeight
start = targetHeight - height
else:
break
# other conditions, just fine
else:
break # must exit since nothing to scan.
cropped = resized[start:end, :, :] # height, width, channels
imageSeries.append(cropped)
index += 1
else:
direction = "horizontal"
index = 0
while True:
start, end = width * index, width * (index + 1)
if start < targetWidth:
if end > targetWidth:
if 1 - (end - targetWidth) / targetWidth >= threshold:
end = targetWidth
start = targetWidth - width
else:
break
# other conditions, just fine
else:
break # must exit since nothing to scan.
cropped = resized[:, start:end, :] # height, width, channels
imageSeries.append(cropped)
index += 1
if return_direction:
return imageSeries, direction
else:
return imageSeries
def resizeImageWithPadding(
image,
width,
height,
border_type: Literal["constant_black", "replicate"] = "constant_black",
):
shape = image.shape
assert len(shape) == 3
ih, iw, channels = shape
targetWidth = min(width, math.floor(iw * height / ih))
targetHeight = min(height, math.floor(ih * width / iw))
resized = cv2.resize(
image, (targetWidth, targetHeight), interpolation=cv2.INTER_CUBIC
)
BLACK = [0] * channels
top = max(0, math.floor((height - targetHeight) / 2))
bottom = max(0, height - targetHeight - top)
left = max(0, math.floor((width - targetWidth) / 2))
right = max(0, width - targetWidth - left)
if border_type == "constant_black":
padded = cv2.copyMakeBorder(
resized, top, bottom, left, right, cv2.BORDER_CONSTANT, value=BLACK
)
elif border_type == "replicate":
padded = cv2.copyMakeBorder(
resized, top, bottom, left, right, cv2.BORDER_REPLICATE, value=BLACK
)
else:
raise Exception("unknown border_type: %s" % border_type)
return padded

Read More