Strfry: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
No edit summary |
||
(15 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
[[Category:Nostr]] | [[Category:Nostr]] | ||
* https://github.com/hoytech/strfry | = Running = | ||
<pre> | |||
ssh nostr | |||
cd projects/nostr/strfry | |||
./strfry relay | |||
</pre> | |||
<pre> | |||
ssh -i path/to/key.pem -L 7777:127.0.0.1:7777 nostr -N | |||
</pre> | |||
~/.ssh/ssh_config set up: | |||
<pre> | |||
ssh -L 7777:127.0.0.1:7777 nostr -N | |||
</pre> | |||
= Ops = | |||
== Deleting == | |||
<pre> | |||
$ ./strfry delete --filter='{"ids":["0647ec28094826947018c861676098d3d8ea71224fd019891ddb51679b48d96c"]}' | |||
</pre> | |||
= Troubleshooting = | |||
<pre> | |||
['NOTICE', 'ERROR: bad req: std::get: wrong index for variant'] | |||
</pre> | |||
Filters need to be arrays, not single values: | |||
<pre> | |||
critique_filters = { | |||
"kinds": [1985], | |||
"#A": [community_a_tag], | |||
"#E": [post_event_id_hex] | |||
} | |||
</pre> | |||
= Links = | |||
* GitHub: https://github.com/hoytech/strfry | |||
* Whitelist / Blacklist: https://github.com/hoytech/strfry/blob/master/docs/plugins.md | |||
= Install = | |||
== Base Server == | |||
<pre> | |||
ssh admin@52.11.172.130 | |||
sudo apt update && sudo apt dist-upgrade -y | |||
sudo apt install -y emacs-nox | |||
</pre> | |||
== Install == | |||
<pre> | |||
sudo apt install -y git build-essential libsecp256k1-dev libzstd-dev | |||
sudo apt install -y libyaml-perl libtemplate-perl libregexp-grammars-perl | |||
sudo apt install -y libssl-dev zlib1g-dev liblmdb-dev libflatbuffers-dev | |||
mkdir -p projects/nostr | |||
cd projects/nostr | |||
git clone https://github.com/hoytech/strfry.git | |||
cd strfry | |||
git submodule update --init | |||
make setup-golpe | |||
make -j4 | |||
</pre> | |||
== Testing == | |||
Fetch the test data: | |||
<pre> | |||
mkdir /home/admin/projects/nostr/data/ | |||
curl https://assets.wellorder.net/nostr/nostr-wellorder-early-500k-v1.jsonl.bz2 -o /home/admin/projects/nostr/data/nostr-wellorder-early-500k-v1.jsonl.bz2 | |||
</pre> | |||
Make a backup copy of Strfry | |||
<pre> | |||
cd /home/admin/projects/nostr/ | |||
cp -r strfry bak.2023-10-07.01.strfry | |||
</pre> | |||
Load the data into the live instance of Strfry | |||
<pre> | |||
cd strfry | |||
bzip2 -dc ../data/nostr-wellorder-early-500k-v1.jsonl.bz2 | ./strfry import | |||
</pre> | |||
Appears successful in 245 seconds: | |||
<pre> | |||
2023-10-08 03:28:56.082 ( 245.368s) [Writer ]INFO| Committed 5587. Processed 500000 lines. 495122 added, 4878 rejected, 0 dups | |||
2023-10-08 03:28:56.082 ( 245.368s) [main thread ]INFO| Done. Processed 500000 lines. 495122 added, 4878 rejected, 0 dups | |||
2023-10-08 03:28:56.086 ( 245.372s) [main thread ]INFO| atexit | |||
</pre> | |||
Install JSON-XS for Perl and JQ for command line | |||
<pre> | |||
sudo apt install -y libjson-xs-perl | |||
sudo apt install -y jq | |||
</pre> | |||
Run the tests (these are run-forever tests, they'll run until killed) | |||
<pre> | |||
perl test/filterFuzzTest.pl scan-limit # test query engine with limit | |||
perl test/filterFuzzTest.pl scan # test query engine without limit (I don't know what limit is) | |||
perl test/filterFuzzTest.pl monitor # test the monitor engine | |||
</pre> |
Latest revision as of 18:51, 19 December 2023
Running
ssh nostr cd projects/nostr/strfry ./strfry relay
ssh -i path/to/key.pem -L 7777:127.0.0.1:7777 nostr -N
~/.ssh/ssh_config set up:
ssh -L 7777:127.0.0.1:7777 nostr -N
Ops
Deleting
$ ./strfry delete --filter='{"ids":["0647ec28094826947018c861676098d3d8ea71224fd019891ddb51679b48d96c"]}'
Troubleshooting
['NOTICE', 'ERROR: bad req: std::get: wrong index for variant']
Filters need to be arrays, not single values:
critique_filters = { "kinds": [1985], "#A": [community_a_tag], "#E": [post_event_id_hex] }
Links
- GitHub: https://github.com/hoytech/strfry
- Whitelist / Blacklist: https://github.com/hoytech/strfry/blob/master/docs/plugins.md
Install
Base Server
ssh admin@52.11.172.130 sudo apt update && sudo apt dist-upgrade -y sudo apt install -y emacs-nox
Install
sudo apt install -y git build-essential libsecp256k1-dev libzstd-dev sudo apt install -y libyaml-perl libtemplate-perl libregexp-grammars-perl sudo apt install -y libssl-dev zlib1g-dev liblmdb-dev libflatbuffers-dev mkdir -p projects/nostr cd projects/nostr git clone https://github.com/hoytech/strfry.git cd strfry git submodule update --init make setup-golpe make -j4
Testing
Fetch the test data:
mkdir /home/admin/projects/nostr/data/ curl https://assets.wellorder.net/nostr/nostr-wellorder-early-500k-v1.jsonl.bz2 -o /home/admin/projects/nostr/data/nostr-wellorder-early-500k-v1.jsonl.bz2
Make a backup copy of Strfry
cd /home/admin/projects/nostr/ cp -r strfry bak.2023-10-07.01.strfry
Load the data into the live instance of Strfry
cd strfry bzip2 -dc ../data/nostr-wellorder-early-500k-v1.jsonl.bz2 | ./strfry import
Appears successful in 245 seconds:
2023-10-08 03:28:56.082 ( 245.368s) [Writer ]INFO| Committed 5587. Processed 500000 lines. 495122 added, 4878 rejected, 0 dups 2023-10-08 03:28:56.082 ( 245.368s) [main thread ]INFO| Done. Processed 500000 lines. 495122 added, 4878 rejected, 0 dups 2023-10-08 03:28:56.086 ( 245.372s) [main thread ]INFO| atexit
Install JSON-XS for Perl and JQ for command line
sudo apt install -y libjson-xs-perl sudo apt install -y jq
Run the tests (these are run-forever tests, they'll run until killed)
perl test/filterFuzzTest.pl scan-limit # test query engine with limit perl test/filterFuzzTest.pl scan # test query engine without limit (I don't know what limit is) perl test/filterFuzzTest.pl monitor # test the monitor engine