Follow

Very nice! I didn't know about zstd. Trying it out:

Comparison: tar.gz 1.9G 11 min, tar.xz 1.2G 88 min

33 min, 1.3G result, -19 and all cores:
tar -c -I 'zstd -19 -T0' -f cleaned_files.tar.zst cleaned_files/

2 min, 1.7G result, -11 and all cores:
tar -c -I 'zstd -11 -T0' -f cleaned_files11.tar.zst cleaned_files/

Result: -11 compression on the ZSTD was smaller than the gzip compression and also more than 5 times faster. -19 compression was almost as small as the xz compression, and nearly 3x faster.

(webdev Tory) :emacs:  
I have a directory of 529k json files, totalling 9.5G. Compression: tgz (zip): 1.9G. tar.xz: 1.2G. The 700 mb savings took my machine 1 hour 22 min...
Sign in to participate in the conversation
CleverLibre Social

CleverLibre Social is an inclusive social instance for open discussion, learning, and community.
All cultures welcome.
Hate speech and harassment strictly forbidden.