2022-12-13
Dubbo And Python Xmlrpc

they are both remote procedure calls.

you can ship things across internet, but remember it can’t be some “native” structure like numpy array.

python xmlrpc tutorial and xmlrpc.server mentions c2 wiki which you have scraped before. where is it, along with all other things you have scraped? believe it is in that AGI directory since you are such an archivist at that time.

tutorials on how to detect if dubbo services are working normally:

basically they use native java methods or telnet protocol with python

http://www.shouhuola.com/q-23671.html

https://www.yisu.com/zixun/576879.html

https://www.cnblogs.com/leozhanggg/p/14176752.html

https://www.bilibili.com/read/cv13670275/

http://www.zztongyun.com/article/article-1-1.html (open with elinks to prevent ads)

Read More

2022-12-08
Talk To Openai Chatgpt To Learn A Few On Paraphrasing, Title Generation

it’s like a huge search engine which can talk.

github topic on paraphrase

小发猫

文章伪原创工具 and backup 有提到用扫描仪来扫描原创文章的方法

online paraphrase tool translation based, can process chinese

chatgpt says creativity matters, such as hackers, video producers and your evil sex shit.

you said LDA and LSA are helpful for paraphrasing

Yes, LDA and LSA are two methods that can be used to generate paraphrases of a given text. LDA (Latent Dirichlet Allocation) is a probabilistic model that can be used to identify topics in a text and then generate new sentences that express the same topics in a different way. LSA (Latent Semantic Analysis) is a method that uses singular value decomposition to identify the underlying meaning of words in a text and then generate paraphrases by replacing words with their closest semantic equivalents. Both LDA and LSA can be helpful for generating paraphrases, but they each have their own strengths and limitations.

can you demonstrate how to use LSA for paraphrasing in python

I’m sorry, but I am not currently able to browse the internet or access any external resources. As a large language model trained by OpenAI, my knowledge is based solely on the text that I have been trained on, and I don’t have the ability to run code or demonstrate specific examples. However, I can provide a general overview of how LSA could be used for paraphrasing in Python.

To use LSA for paraphrasing in Python, you would first need to create a matrix of word vectors for the words in the text you want to paraphrase. This matrix would represent the semantic relationships between the words in the text, and would be used to identify the underlying meaning of each word.

Next, you would use singular value decomposition (SVD) to decompose the matrix into three matrices, which represent the different factors that contribute to the meaning of the words in the text.

Once you have the three matrices, you can then use them to identify the most closely related words for each word in the text, and generate paraphrases by replacing words with their closest semantic equivalents.

This is just a brief overview of how LSA could be used for paraphrasing in Python. There are many different ways to implement LSA, and the exact details of the implementation would depend on the specific requirements of your project.

continue

I’m sorry, but I’m not sure what you would like me to continue with. If you have a specific question or topic you would like to discuss, I would be happy to help. Otherwise, I can provide more information about LSA and how it can be used for paraphrasing if that would be helpful. Let me know what you would like to know.

how to use LDA for paraphrasing

To use LDA for paraphrasing, you would first need to train a LDA model on a large corpus of text. This would involve using a machine learning algorithm to identify the topics that are present in the corpus, and then generating a set of “topic vectors” that represent the distribution of those topics across the text.

Once you have trained the LDA model, you can then use it to generate paraphrases of a given text by identifying the topics present in the text and then replacing words with synonyms or related words that express the same topics in a different way. This process can be repeated multiple times to generate multiple paraphrases of the same text.

Here is a brief example of how LDA could be used for paraphrasing in Python:

1
2
3
4
5
6
7
8
9
10
11
12
13
from gensim import corpora, models
# Load the text you want to paraphrase
text = "The quick brown fox jumps over the lazy dog."
# Tokenize the text and create a dictionary of words
tokens = text.split(" ")
dictionary = corpora.Dictionary([tokens])
# Train a LDA model on the text
lda_model = models.LdaModel([dictionary.doc2bow(tokens)], num_topics=1)
# Generate paraphrases of the text
for i in range(3):
paraphrase = lda_model.show_topics(num_topics=1, num_words=len(tokens), formatted=False)
print(" ".join([word[0] for word in paraphrase[0][1]]))

This example trains a simple LDA model on a single sentence and then generates three paraphrases of the sentence by replacing words with their closest semantic equivalents. Of course, this is just a simple example to illustrate the basic concept of using LDA for paraphrasing. In practice, you would need to use a much larger corpus of text to train the LDA model and generate more accurate paraphrases.

demonstrate using LSA to paraphrase in python, using gensim if needed

As I mentioned earlier, I am not currently able to browse the internet or access external resources, so I cannot demonstrate using LSA to paraphrase in Python using gensim or any other specific libraries. However, I can provide a general overview of how LSA could be used for paraphrasing in Python.

To use LSA for paraphrasing in Python, you would first need to create a matrix of word vectors for the words in the text you want to paraphrase. This matrix would represent the semantic relationships between the words in the text, and would be used to identify the underlying meaning of each word.

Once you have the matrix of word vectors, you can use singular value decomposition (SVD) to decompose the matrix into three matrices, which represent the different factors that contribute to the meaning of the words in the text. The gensim library includes a implementation of SVD that you can use for this purpose.

Once you have the three matrices, you can then use them to identify the most closely related words for each word in the text, and generate paraphrases by replacing words with their closest semantic equivalents. Here is an example of how this could be done in Python:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
from gensim import corpora, models, matutils
# Load the text you want to paraphrase
text = "The quick brown fox jumps over the lazy dog."
# Tokenize the text and create a dictionary of words
tokens = text.split(" ")
dictionary = corpora.Dictionary([tokens])
# Create a matrix of word vectors using the dictionary
word_vectors = [dictionary.doc2bow(tokens)]
# Use SVD to decompose the matrix of word vectors
U, S, VT = matutils.svd(word_vectors, k=3)
# Generate paraphrases of the text
for i in range(3):
paraphrase = []
for word in tokens:
# Identify the closest semantic equivalents for each word
closest_words = VT.dot(VT[dictionary.token2id[word]])
closest_words = sorted(enumerate(closest_words), key=lambda x: x[1], reverse=True)
closest_word = closest_words[0][0]
# Replace the word with its closest semantic equivalent
paraphrase.append(dictionary[closest_word])
print(" ".join(paraphrase))

This example trains an LSA model on a single sentence and then generates three paraphrases of the sentence by replacing words with their closest semantic equivalents. Again, this is just a simple example to illustrate the basic concept of using LSA for paraphrasing. In practice, you would need to use a much larger corpus of text to train the LSA model and generate more accurate paraphrases.

how to use transformer for paraphrasing

The transformer is a type of neural network architecture that has been used for a wide range of natural language processing tasks, including paraphrasing. To use a transformer for paraphrasing, you would need to train the transformer on a large corpus of text using a machine learning algorithm.

During training, the transformer would learn to identify the underlying meaning of words and phrases in the text and then generate paraphrases by replacing words with synonyms or related words that express the same meaning in a different way.

Here is a brief example of how a transformer could be used for paraphrasing in Python:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
# Load the text you want to paraphrase
text = "The quick brown fox jumps over the lazy dog."
# Tokenize the text using a pre-trained transformer model
tokenizer = AutoTokenizer.from_pretrained("distilbert-base-cased")
tokens = tokenizer.encode(text)
# Load a pre-trained transformer model
transformer = AutoModelForSeq2SeqLM.from_pretrained("distilbert-base-cased")
# Generate paraphrases of the text
for i in range(3):
paraphrase = transformer.generate(tokens)
paraphrase = tokenizer.decode(paraphrase[0])
print(paraphrase)

This example uses a pre-trained transformer model to generate three paraphrases of a single sentence. Of course, in practice you would need to train the transformer on a much larger corpus of text to generate more accurate paraphrases. The exact details of the implementation would depend on the specific requirements of your project.

how to generate catchy title from keywords using language model

To generate a catchy title from a set of keywords using a language model, you would need to train the language model on a large corpus of text that includes a variety of titles. During training, the language model would learn to identify the underlying meaning of words and phrases and the characteristics of catchy titles.

Once the language model has been trained, you can then use it to generate a catchy title from a set of keywords by providing the keywords as input to the model and then selecting the generated title that is the most catchy and relevant to the keywords.

Here is a brief example of how this could be done in Python:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
from transformers import AutoTokenizer, AutoModelForCausalLM
# Load the keywords you want to use to generate the title
keywords = ["quick", "brown", "fox"]
# Tokenize the keywords using a pre-trained language model
tokenizer = AutoTokenizer.from_pretrained("gpt2")
tokens = tokenizer.encode(keywords)
# Load a pre-trained language model
language_model = AutoModelForCausalLM.from_pretrained("gpt2")
# Generate candidate titles from the keywords
candidate_titles = []
for i in range(3):
title = language_model.generate(tokens, max_length=8)
title = tokenizer.decode(title[0])
candidate_titles.append(title)
# Select the most catchy and relevant title
selected_title = candidate_titles[0]
for title in candidate_titles:
if is_catchy(title) and is_relevant(title, keywords):
selected_title = title
break
print(selected_title)

This example generates three candidate titles from a set of keywords using a pre-trained language model, and then selects the most catchy and relevant title as the final title. Of course, this is just a simple example to illustrate the basic concept of using a language model to generate a catchy title from keywords. In practice, you would need to use a much larger corpus of text to train the language model and generate more accurate and diverse titles.

Read More

2022-12-07
Cyber Grand Challenge Darpa Machine Automated Cyber Attack

ctfwiki’s intro on CGC

analyze source code first, then plan attack or fix code

cgc’s github repo and website

search for darpa cgc on github

cyber-challenge Some toy examples, to demonstrate ideas that could be used in DARPA’s Cyber Grand Challenge including modifying java bytecode and filter out html requests on the fly

EVIL (Exploiting software VIa natural Language) is an approach to automatically generate software exploits in assembly/Python language from descriptions in natural language. The approach leverages Neural Machine Translation (NMT) techniques and a dataset that we developed for this work.

Topics

linux exploit encoder assembly decoder dataset seq2seq shellcode nmt software-exploitation codebert

Resources

Readme

License

GPL-3.0 license

Stars

13 stars

Watchers

3 watching

Forks

1 fork

Releases

No releases published

Packages

No packages published

Contributors 2

@piliguori

piliguori Pietro Liguori

@taisazero

taisazero Erfan Al-Hossami

Languages

Python

97.6%

Shell

2.0%

Other

0.4%

Read More

2022-12-02
Exploring Python Libraries And Resources For Nmap Network Scanning

nmap python scripting

python3-nmap and doc

doc of nmapthon python scriptable nse

python-nmap

Read More

2022-11-11
Call Python In Clojure, Clojure-Python Bridge

libpython-clj deep python integration in clojure

embed clojure in python call clojure in python

Read More

2022-11-09
Neo4J, Mindsdb, Milvus, Peewee

peewee

docs

official doc

milvus

docs

official doc

neo4j

OGM, object-graph mapping

activegraph

cypher

neomodel

neode

docs

graph data science

python driver doc

drivers

neo4j python driver

neo4j python graph data science driver

wrappers

pypher cypher builder in python intro

neopy

pygds a python wrapper to call Neo4j Graph Data Science procedures from python using the Neo4j python driver

mindsdb

since mindsdb is sql-based it is time to create monkey patches for sqlalchemy core?

1
2
3
4
5
from sqlalchemy import text
# write the SQL query inside the text() block
sql = text('SELECT * from BOOKS WHERE BOOKS.book_price > 50')
results = engine.execute(sql)

tools

Lightwood is an AutoML framework that enables you to generate and customize machine learning pipelines declarative syntax called JSON-AI doc

docs

official doc

wrappers

sqlalchemy 2.0 doc

pypika sql query builder

mindsdb native

mindsdb python sdk

Read More

2022-11-03
Discover '问财选股': The Comprehensive Stock Selection Platform And Python Implementation

Read More

2022-10-21
关于做直播 About Live Streaming

run python in browser

pyodide

pyscript

skulpt

pass parameter via url

get url anchor text

you can still use query string when visiting github.io

1
2
3
4
5
6
7
8
9
10
11
12
function getQueryStringValues(key) {
var arrParamValues = [];
var url = window.location.href.slice(window.location.href.indexOf('?') + 1).split('&');
for (var i = 0; i < url.length; i++) {
var arrParamInfo = url[i].split('=');
if (arrParamInfo[0] == key || arrParamInfo[0] == key+'[]') {
arrParamValues.push(decodeURIComponent(arrParamInfo[1]));
}
}
return (arrParamValues.length > 0 ? (arrParamValues.length == 1 ? arrParamValues[0] : arrParamValues) : null);
}

保存粉丝cookie 避免接入xpay之后反复输入地址

set and get a cookie with javascript

付钱兑换礼物

要能够有效识别付钱的对象 用浏览器的cookie 送达目标地址

过滤人名

过滤群聊里面的广告内容 色情 暴恐 声音不对劲要去掉 合适的BGM

直播上传带宽要保证 延迟要低

“聚合直播”

做直播最重要的就是互动性

你可以直播群友聊天的画面 要去掉含有个人隐私的头像或照片 信息 QQ Email 要过滤掉nsfw的内容 操作qq机器人来和群友说话 或者让送礼物最多的人获得发言权 排行榜最上面的人获得发言权 群聊也不一定要单一 多个群一起弄 甚至多个社交软件一起弄 群聊私聊一起用

你也可以参考其他的主播 看看他们是怎么做的

可以通过分析评论 切换直播主题 切换直播画面

免签支付 获取订单号 给粉丝发福利 VIP

b站充电API

https://www.yunmianqian.com

https://github.com/assimon/easymqpay

https://github.com/szvone/vmqphp

https://github.com/wxs2/xposed-pay

https://github.com/szvone/vmqApk

vmqapk 最新修改版

xpay 这个东西需要人工监听 跪了

随想 8

视频广告

Yukio 21:46:07

应该来个淡入淡出之类的

Yukio 21:46:25

还有一个试看结束的提示

Yukio 21:46:37

试看开始之前的提示

播放视频内容也要有提示 开头和结尾要给人知道这个截取的片段是干什么的

随想 7

Yukio 01:56:43

付了款 留下联系方式 给你发福利

Yukio 01:57:16

但是万一有些人滥用呢

Yukio 01:57:36

发一些奇奇怪怪的联系方式或者邮箱

Yukio 01:57:55

给我发些什么监察委员会的邮箱

Yukio 01:58:32

我可以增加一个取消订阅按钮 让这些人取消订阅 别找我麻烦

Yukio 02:01:28

或者我把文件上传到一次性的文件分享页面

Yukio 02:01:38

过几个小时自动取消分享

随想 6

Yukio 01:06:16

这个我直播间可以根据uid和活跃等级

Yukio 01:06:24

持久化你的活跃度

Yukio 01:06:34

充钱也可以升级

Yukio 01:06:44

送我礼物

Yukio 01:07:21

如果是直接wx转钱给我 这个估计得给我发交易单号 私信发我才能兑换呢

Yukio 01:07:31

至于怎么找这个交易单号

Yukio 01:07:34

emm

Yukio 01:08:36

我想想

Yukio 01:09:28

这个估计得对接第四方交易接口 或者我看看现成的实现

Yukio 01:10:32

同时 给我的视频留言 发弹幕也可以刷活跃度

Yukio 01:10:42

刷直播间给你显示的活跃度

Yukio 01:12:39

如果给我充钱 我会发你涩图 冲的多发的多

Yukio 01:12:50

至于怎么发

Yukio 01:12:59

发你邮箱 你留下邮箱地址我发你

Yukio 01:15:57

不需要涩图也可以个性化定制

Yukio 01:16:19

如果给我充钱 一段时间内不会有给我打钱的二维码

Yukio 01:16:40

不给我充 不给我私信 我就一直发给我打钱的二维码

Yukio 01:17:37

如果给我充钱 可以绑定相关QQ账号 邮箱账号 或者什么其他的社交账号 每个平台只能绑一个账号

Yukio 01:18:14

绑了之后 给钱 一段时间我的内容没有打钱二维码 私信的话

你主动加我为好友才行

随想 5

Yukio 23:05:52

为了检测这种二维码

Yukio 23:06:30

同时有一些不是很好检测的文字 水印

Yukio 23:06:44

也需要image local contrast enhancement

Yukio 23:25:11

随机切割句子 分批发送

Yukio 23:25:16

这样我的机器人更像人

image enhancement for optical character recognition

image enhancement for watermark removal

随想 4

Yukio 21:00:32

nsfw没必要每一帧都加

Yukio 21:00:38

可以跳着加

Yukio 21:00:52

然后把出问题的帧的周围全部ban掉

Yukio 21:01:07

但是除了画面

Yukio 21:01:16

声音娇喘的也得去掉

Yukio 21:01:24

这个娇喘

Yukio 21:01:41

用个声音分类器

Yukio 21:01:51

用狗叫分类器改一下就好了

Yukio 21:02:12

除了娇喘

Yukio 21:02:20

还得弄语音识别

Yukio 21:02:35

防止某些狗叫言论污染直播间

随想 3

直播的通知 当直播的主题变化的时候 把广告发到不同的群 不同的观众那里

随想 2

不和谐的 包括有链接的内容 有二维码的内容 有联系方式的内容 加标点符号的广告内容 重复发送的内容

Yukio 14:26:10

可以看到其实判别不和谐言论很难

Yukio 14:26:23

所以可以用谐音字 拼音判断

Yukio 14:26:51

加上不信任期 一旦出现不和谐言论 一段时间内不用这个群聊

Yukio 14:27:15

以及繁体字转简体字

随想 1

Yukio 22:29:44

我直播群友聊天咋样?

Yukio 22:30:07

我直播群友聊天 然后给我留言的人 有机会把消息发到群里面来

Yukio 22:30:30

其中送礼物送的最多的发消息发进来的概率最大

Yukio 22:31:49

不仅如此 我还多个群聚合聊天 多社交软件 多语言聚合聊天

Yukio 22:32:06

我还可以自由切换话题 自动翻译

Yukio 22:32:36

也就是说如果你和外国人聊天 你说中文变英文

Yukio 22:32:46

别人说英文变中文

黑暗骑士 22:33:20

直播看你装奈子

Yukio 22:33:33

nsfw不能直播

Yukio 22:33:45

直播最重要的是互动啊

Yukio 22:33:51

播其他的也行

Yukio 22:34:06

但是装柰子 也得sfw

Yukio 22:34:19

所以有可能需要打码

Yukio 22:34:29

检测出来nsfw区域 自动打码

Yukio 22:34:36

或者全屏打码

Yukio 22:35:07

你说什么 我可以给你在线搜索视频给你看

Yukio 22:35:49

你不说话的时候我就挑热门视频和热门直播转发

Yukio 22:36:15

根据进来的观众们的历史观看记录和历史发言来播视频

Yukio 22:42:38

这个换台的话 本人三分钟可以撤销两次

Yukio 22:43:09

而其他人在本人获得换台权利之后 三分钟之后才能继续排队

Yukio 22:43:23

早就不招了

Yukio 22:44:29

你建的话 我有过滤器啊

Yukio 22:44:32

你可以建

Yukio 22:44:57

但是要是有关键字 我这边不会播你的

Yukio 22:45:08

而且头像和昵称

Yukio 22:45:13

不会有

Yukio 22:46:05

如果sfw

Yukio 22:46:14

那么就是不够涩

Yukio 22:46:18

也不会有问题

Yukio 22:46:40

再说了

Yukio 22:46:51

让别人跟你聊天不是好事么

Yukio 22:47:16

取决于涩涩的程度

Read More

2022-09-18
Image Resize, Image Padding, Image Scanning

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
def scanImageWithWindowSizeAutoResize(
image,
width,
height,
return_direction=False,
threshold=0.1, # minimum 'fresh' area left for scanning
): # shall you use torch? no?
shape = image.shape
assert len(shape) == 3
ih, iw, channels = shape
targetWidth = max(width, math.floor(iw * height / ih))
targetHeight = max(height, math.floor(ih * width / iw))
resized = cv2.resize(
image, (targetWidth, targetHeight), interpolation=cv2.INTER_CUBIC
)
# determine scan direction here.
imageSeries = []
if targetWidth / targetHeight == width / height:
imageSeries = [resized] # as image series.
direction = None
elif targetWidth / targetHeight < width / height:
direction = "vertical"
# the scanning is along the vertical axis, which is the height.
index = 0
while True:
start, end = height * index, height * (index + 1)
if start < targetHeight:
if end > targetHeight:
if 1 - (end - targetHeight) / targetHeight >= threshold:
end = targetHeight
start = targetHeight - height
else:
break
# other conditions, just fine
else:
break # must exit since nothing to scan.
cropped = resized[start:end, :, :] # height, width, channels
imageSeries.append(cropped)
index += 1
else:
direction = "horizontal"
index = 0
while True:
start, end = width * index, width * (index + 1)
if start < targetWidth:
if end > targetWidth:
if 1 - (end - targetWidth) / targetWidth >= threshold:
end = targetWidth
start = targetWidth - width
else:
break
# other conditions, just fine
else:
break # must exit since nothing to scan.
cropped = resized[:, start:end, :] # height, width, channels
imageSeries.append(cropped)
index += 1
if return_direction:
return imageSeries, direction
else:
return imageSeries
def resizeImageWithPadding(
image,
width,
height,
border_type: Literal["constant_black", "replicate"] = "constant_black",
):
shape = image.shape
assert len(shape) == 3
ih, iw, channels = shape
targetWidth = min(width, math.floor(iw * height / ih))
targetHeight = min(height, math.floor(ih * width / iw))
resized = cv2.resize(
image, (targetWidth, targetHeight), interpolation=cv2.INTER_CUBIC
)
BLACK = [0] * channels
top = max(0, math.floor((height - targetHeight) / 2))
bottom = max(0, height - targetHeight - top)
left = max(0, math.floor((width - targetWidth) / 2))
right = max(0, width - targetWidth - left)
if border_type == "constant_black":
padded = cv2.copyMakeBorder(
resized, top, bottom, left, right, cv2.BORDER_CONSTANT, value=BLACK
)
elif border_type == "replicate":
padded = cv2.copyMakeBorder(
resized, top, bottom, left, right, cv2.BORDER_REPLICATE, value=BLACK
)
else:
raise Exception("unknown border_type: %s" % border_type)
return padded

Read More

2022-09-17
Javascript Python Bridge

jspybridge

javascript in python:

1
2
pip3 install javascript

1
2
3
4
5
from javascript import require, globalThis
chalk, fs = require("chalk"), require("fs")
print("Hello", chalk.red("world!"), "it's", globalThis.Date().toLocaleString())
fs.writeFileSync("HelloWorld.txt", "hi!")

access python from javascript:

1
2
npm i pythonia

1
2
3
4
5
6
7
8
9
10
11
import { python } from 'pythonia'
// Import tkinter
const tk = await python('tkinter')
// All Python API access must be prefixed with await
const root = await tk.Tk()
// A function call with a $ suffix will treat the last argument as a kwarg dict
const a = await tk.Label$(root, { text: 'Hello World' })
await a.pack()
await root.mainloop()
python.exit() // Make sure to exit Python in the end to allow node to exit. You can also use process.exit.

Read More