Strfry: Difference between revisions

From Traxel Wiki
Jump to navigation Jump to search
No edit summary
 
(10 intermediate revisions by the same user not shown)
Line 1: Line 1:
[[Category:Nostr]]
[[Category:Nostr]]
* https://github.com/hoytech/strfry
= Running =
<pre>
ssh nostr
cd projects/nostr/strfry
./strfry relay
</pre>
<pre>
ssh -i path/to/key.pem -L 7777:127.0.0.1:7777 nostr -N
</pre>
~/.ssh/ssh_config set up:
<pre>
ssh -L 7777:127.0.0.1:7777 nostr -N
</pre>
= Ops =
== Deleting ==
<pre>
$ ./strfry delete --filter='{"ids":["0647ec28094826947018c861676098d3d8ea71224fd019891ddb51679b48d96c"]}'
</pre>
= Troubleshooting =
<pre>
['NOTICE', 'ERROR: bad req: std::get: wrong index for variant']
</pre>
Filters need to be arrays, not single values:
<pre>
    critique_filters = {
        "kinds": [1985],
        "#A": [community_a_tag],
        "#E": [post_event_id_hex]
    }
</pre>
 
= Links =
* GitHub: https://github.com/hoytech/strfry
* Whitelist / Blacklist: https://github.com/hoytech/strfry/blob/master/docs/plugins.md
= Install =
= Install =
== Base Server ==
== Base Server ==
Line 27: Line 60:
curl https://assets.wellorder.net/nostr/nostr-wellorder-early-500k-v1.jsonl.bz2 -o /home/admin/projects/nostr/data/nostr-wellorder-early-500k-v1.jsonl.bz2
curl https://assets.wellorder.net/nostr/nostr-wellorder-early-500k-v1.jsonl.bz2 -o /home/admin/projects/nostr/data/nostr-wellorder-early-500k-v1.jsonl.bz2
</pre>
</pre>
Make a testing copy of Strfry
Make a backup copy of Strfry
<pre>
<pre>
cd /home/admin/projects/nostr/
cd /home/admin/projects/nostr/
cp -r strfry bak.2023-10-07.01.strfry
cp -r strfry bak.2023-10-07.01.strfry
</pre>
</pre>
Load the data into the test instance of Strfry
Load the data into the live instance of Strfry
<pre>
<pre>
cd strfry
cd strfry
bzip2 -dc ../data/nostr-wellorder-early-500k-v1.jsonl.bz2 | ./strfry import
bzip2 -dc ../data/nostr-wellorder-early-500k-v1.jsonl.bz2 | ./strfry import
</pre>
Appears successful in 245 seconds:
<pre>
2023-10-08 03:28:56.082 ( 245.368s) [Writer          ]INFO| Committed 5587. Processed 500000 lines. 495122 added, 4878 rejected, 0 dups
2023-10-08 03:28:56.082 ( 245.368s) [main thread    ]INFO| Done. Processed 500000 lines. 495122 added, 4878 rejected, 0 dups
2023-10-08 03:28:56.086 ( 245.372s) [main thread    ]INFO| atexit
</pre>
Install JSON-XS for Perl and JQ for command line
<pre>
sudo apt install -y libjson-xs-perl
sudo apt install -y jq
</pre>
Run the tests (these are run-forever tests, they'll run until killed)
<pre>
perl test/filterFuzzTest.pl scan-limit # test query engine with limit
perl test/filterFuzzTest.pl scan # test query engine without limit (I don't know what limit is)
perl test/filterFuzzTest.pl monitor # test the monitor engine
</pre>
</pre>

Latest revision as of 18:51, 19 December 2023

Running

ssh nostr
cd projects/nostr/strfry
./strfry relay
ssh -i path/to/key.pem -L 7777:127.0.0.1:7777 nostr -N

~/.ssh/ssh_config set up:

ssh -L 7777:127.0.0.1:7777 nostr -N

Ops

Deleting

$ ./strfry delete --filter='{"ids":["0647ec28094826947018c861676098d3d8ea71224fd019891ddb51679b48d96c"]}'

Troubleshooting

['NOTICE', 'ERROR: bad req: std::get: wrong index for variant']

Filters need to be arrays, not single values:

    critique_filters = {
        "kinds": [1985],
        "#A": [community_a_tag],
        "#E": [post_event_id_hex]
    }

Links

Install

Base Server

ssh admin@52.11.172.130
sudo apt update && sudo apt dist-upgrade -y
sudo apt install -y emacs-nox

Install

sudo apt install -y git build-essential libsecp256k1-dev libzstd-dev
sudo apt install -y libyaml-perl libtemplate-perl libregexp-grammars-perl
sudo apt install -y libssl-dev zlib1g-dev liblmdb-dev libflatbuffers-dev
mkdir -p projects/nostr
cd projects/nostr
git clone https://github.com/hoytech/strfry.git
cd strfry
git submodule update --init
make setup-golpe
make -j4

Testing

Fetch the test data:

mkdir /home/admin/projects/nostr/data/
curl https://assets.wellorder.net/nostr/nostr-wellorder-early-500k-v1.jsonl.bz2 -o /home/admin/projects/nostr/data/nostr-wellorder-early-500k-v1.jsonl.bz2

Make a backup copy of Strfry

cd /home/admin/projects/nostr/
cp -r strfry bak.2023-10-07.01.strfry

Load the data into the live instance of Strfry

cd strfry
bzip2 -dc ../data/nostr-wellorder-early-500k-v1.jsonl.bz2 | ./strfry import

Appears successful in 245 seconds:

2023-10-08 03:28:56.082 ( 245.368s) [Writer          ]INFO| Committed 5587. Processed 500000 lines. 495122 added, 4878 rejected, 0 dups
2023-10-08 03:28:56.082 ( 245.368s) [main thread     ]INFO| Done. Processed 500000 lines. 495122 added, 4878 rejected, 0 dups
2023-10-08 03:28:56.086 ( 245.372s) [main thread     ]INFO| atexit

Install JSON-XS for Perl and JQ for command line

sudo apt install -y libjson-xs-perl
sudo apt install -y jq

Run the tests (these are run-forever tests, they'll run until killed)

perl test/filterFuzzTest.pl scan-limit # test query engine with limit
perl test/filterFuzzTest.pl scan # test query engine without limit (I don't know what limit is)
perl test/filterFuzzTest.pl monitor # test the monitor engine