"WARC: vintage.ponychan.net" (2020):
https://archive.org/download//755cd7b3662ae68182e8f46411b98b
>total size of requested files (409 GB) is too large for zip-on-the-fly

----

In the past, there were posts about organization and IPFS. One method is indexing via text files. Another good method is indexing via MFS: use the "ipfs files" set of commands; also use text files and stuff. Get info on pins:

$ ipfs pin ls --type=recursive | sed "s/ .*//g" | xargs -d \
"\n" sh -c 'for args do echo $args; ipfs ls -s --size=false $args \
| head; sleep 10; echo; done' _ # see also: "ipfs files stat \
/ipfs/$args"

which is also helpful for copying CIDs into IPFS MFS.

----

Notes on uploading to archive.org:

$ ia upload <ID> "./path/to/folder1" --metadata="[...]"

Entire path from working directory to and including "folder1" will be replicated on HTTPS server, so http.../<ID_root>/path/to/folder1/

$ ia upload <ID> "./path/to/folder1/file1.ext" --metadata="[...]"

No path replication, file uploaded to ID root, local:./path/to/folder1/file1.ext -> server:http.../<ID_root>/file1.ext

$ cd /path/to/folder/; cat /path/to/filelist.txt | \
xargs -d "\n" sh -c 'for args do date -u; ~/Downloads/ia \
upload <ID> "$args" --metadata="[...]"; date -u; done' _

Use this to upload files/folders after a program exit or crash. File "filelist.txt" should contain local paths to files/folders which haven't yet been uploaded to that item but should be. Maybe todo: better automation.

>>9312