Sync: Difference between revisions
Jump to navigation
Jump to search
| Line 9: | Line 9: | ||
bob@2020gamer:/opt/cypherpunk/data/reddit/json$ du discussion/lifeprotips/2023-09/ | bob@2020gamer:/opt/cypherpunk/data/reddit/json$ du discussion/lifeprotips/2023-09/ | ||
6804 discussion/lifeprotips/2023-09/ | 6804 discussion/lifeprotips/2023-09/ | ||
find discussion -name '2023-09' -type d | awk '{print "rsync -avz --size-only --delete www.iterativechaos.com:/opt/cypherpunk/data/reddit/json/"$1" ./"$1}' | find discussion -name '2023-09' -type d | awk '{print "rsync -avz --size-only --delete www.iterativechaos.com:/opt/cypherpunk/data/reddit/json/"$1"/ ./"$1"/"}' | ||
</pre> | </pre> | ||
Revision as of 22:13, 25 September 2023
Rsync
From there to here
rsync -avz --size-only --delete user@remote_host:/path/to/remote/directory/ /path/to/local/directory/
7766956 ./discussion
ls discussion/lifeprotips/2023-09/| wc -l
139
bob@2020gamer:/opt/cypherpunk/data/reddit/json$ du discussion/lifeprotips/2023-09/
6804 discussion/lifeprotips/2023-09/
find discussion -name '2023-09' -type d | awk '{print "rsync -avz --size-only --delete www.iterativechaos.com:/opt/cypherpunk/data/reddit/json/"$1"/ ./"$1"/"}'
AWS S3 Sync
From here to there, with delete
aws s3 sync /path/to/remote/directory/ s3://your-s3-bucket-name/path/in/bucket/ --delete --size-only