Let's talk zipping files, specifically 7-Zip

[toomanycanons]

I zip large folders (from 12 to 30 GB) using 7-Zip (to be referred to as 7z). It seems to take forever but does finish.

As I'm typing this 7z is zipping at a 73% compression ratio at 18 MB/s.

That said, I was also watching a vid of a review of some laptops, the review using 7z to compress and decompress folders of some size, they didn't specify, using the build-in 7z benchmark tool. On the screen was the results from a lot of different cpu testing.

Their numbers indicated speeds of around 36 MB/s compression and around 477 MB/s for decompression for my Ryzen 5 3600.

I know I'm leaving out a lot of specifics but are there some things to look out for when setting up 7z to get me some more speed? 36 MB/s means twice as fast and, as they say, time is money.

 

[Jim B (MSP)]

I zip large folders (from 12 to 30 GB) using 7-Zip (to be referred to as 7z). It seems to take forever but does finish.

As I'm typing this 7z is zipping at a 73% compression ratio at 18 MB/s.

That said, I was also watching a vid of a review of some laptops, the review using 7z to compress and decompress folders of some size, they didn't specify, using the build-in 7z benchmark tool. On the screen was the results from a lot of different cpu testing.

Their numbers indicated speeds of around 36 MB/s compression and around 477 MB/s for decompression for my Ryzen 5 3600.

I know I'm leaving out a lot of specifics but are there some things to look out for when setting up 7z to get me some more speed? 36 MB/s means twice as fast and, as they say, time is money.

With everything else being constant, I would guess that speed will vary by type of disk for the read and the write. An SSD/SSD will probably be faster than a spinner/spinner. But I have never tested or thought about it much as I primarily use it to decompress.

--
Jim
"It's all about the light"

 

[Ken Barley]

Do we have the same source device? Both SSD or HDD? Comparing SSD with HDD would give big difference.

 

[toomanycanons]

Do we have the same source device? Both SSD or HDD? Comparing SSD with HDD would give big difference.

Both of you are right. I'll re-check how the reviewer tests (I'm always assuming SSD but better to check) and do my own testing, SSD vs HD vs NVME M.2.

 

[tkbslc]

The speed of compression heavily depends on the type of files being compressed. Files like text or uncompressed media that have lots of "white space" are easily compressible and will run faster and the zip fill will be much smaller than source. Files that are already mostly compressed, like JPEGs or most Movie files, will take ages as the the algorithm desperately hunts for white space that isn't there. And after all that time, you find you've saved 1% space.

I would recommend choosing no compression if zipping JPEGs, compressed RAW, typical movie formats, or anything else that is already a compressed file format. You really just want ZIP as a container at this point because significant further compression is not possible and will take a lot of time for almost nothing.

Use compression for things like office documents, text files, uncompressed media like TIFF and WAV files.

If you are zipping something that is a combination of all kinds, I would leave compression on but expect it to move slowly when it progresses past a compressed file.

So I'm saying your speed differences vs the reviewers are most likely down to file types used.

 

[toomanycanons]

The speed of compression heavily depends on the type of files being compressed. Files like text or uncompressed media that have lots of "white space" are easily compressible and will run faster and the zip fill will be much smaller than source. Files that are already mostly compressed, like JPEGs or most Movie files, will take ages as the the algorithm desperately hunts for white space that isn't there. And after all that time, you find you've saved 1% space.

I would recommend choosing no compression if zipping JPEGs, compressed RAW, typical movie formats, or anything else that is already a compressed file format. You really just want ZIP as a container at this point because significant further compression is not possible and will take a lot of time for almost nothing.

Use compression for things like office documents, text files, uncompressed media like TIFF and WAV files.

If you are zipping something that is a combination of all kinds, I would leave compression on but expect it to move slowly when it progresses past a compressed file.

So I'm saying your speed differences vs the reviewers are most likely down to file types used.

16 bit tiffs, 32 bit .hdr files and raws. Pretty heavy lifting I'd imagine. I'll try it without compressing as it doesn't seem to compress these types very much anyway.

Good idea of just using 7z just as a container.

 

[CAcreeks]

7zip is an excellent program!

16 bit tiffs, 32 bit .hdr files and raws. Pretty heavy lifting I'd imagine. I'll try it without compressing as it doesn't seem to compress these types very much anyway.

It might be a good idea to save TIFFs with LZW compression, if you follow advice below. I'm not familiar with .hdr files except as a generic term for high dynamic range. Non-compressed Raw images waste a lot of space, so use compressed Raw if possible.

Good idea of just using 7z just as a container.

You can use the -m command line option to avoid compressing constituent files.

https://superuser.com/questions/336243/how-to-do-files-to-store-without-compression-in-7-zip

 

[Mystery member]

I zip large folders (from 12 to 30 GB) using 7-Zip (to be referred to as 7z). It seems to take forever but does finish.

As I'm typing this 7z is zipping at a 73% compression ratio at 18 MB/s.

That said, I was also watching a vid of a review of some laptops, the review using 7z to compress and decompress folders of some size, they didn't specify, using the build-in 7z benchmark tool. On the screen was the results from a lot of different cpu testing.

Their numbers indicated speeds of around 36 MB/s compression and around 477 MB/s for decompression for my Ryzen 5 3600.

I know I'm leaving out a lot of specifics but are there some things to look out for when setting up 7z to get me some more speed? 36 MB/s means twice as fast and, as they say, time is money.

I'm scratching my head to work out why you are compressing your folders or files.

With HDDs and even SSDs offering luxury amounts of speedy storage, compression is an obsolete concept, IMHO.

The was a time when compression could be used to squeeze stuff onto small HDDs as part of the OS, but I always avoided the practice.

I now rarely have to deal with Zip files, and I guess that it may be useful to know about the process.

 

[toomanycanons]

I zip large folders (from 12 to 30 GB) using 7-Zip (to be referred to as 7z). It seems to take forever but does finish.

As I'm typing this 7z is zipping at a 73% compression ratio at 18 MB/s.

That said, I was also watching a vid of a review of some laptops, the review using 7z to compress and decompress folders of some size, they didn't specify, using the build-in 7z benchmark tool. On the screen was the results from a lot of different cpu testing.

Their numbers indicated speeds of around 36 MB/s compression and around 477 MB/s for decompression for my Ryzen 5 3600.

I know I'm leaving out a lot of specifics but are there some things to look out for when setting up 7z to get me some more speed? 36 MB/s means twice as fast and, as they say, time is money.

I'm scratching my head to work out why you are compressing your folders or files.

With HDDs and even SSDs offering luxury amounts of speedy storage, compression is an obsolete concept, IMHO.

The was a time when compression could be used to squeeze stuff onto small HDDs as part of the OS, but I always avoided the practice.

I now rarely have to deal with Zip files, and I guess that it may be useful to know about the process.

Scratch no more: I send my zip files to my clients in various parts of the world via my FTP, it has nothing to do with shrinking my own personal storage issue.

 

[Robert Zanatta]

If you are compressing a lot of files instead of just a few files, and those files take up the same space, then the compression of a lot of files will can be slower.

If you're compressing a lot of files that are already compressed, that could slow things down. What you could try is changing the compression option for how much compression to do. For example, if you have a compression option of 1 to 9, with 9 compressing the most, try using 1 or 3 or 5, etc. The compression program might spend much less time trying to figure out how to squeeze every last bit out of a file and thereby work faster. The resultant zip file will likely be bigger, but not by much.

 

[gs85739az]

--ty
Gary

 

[XiaoDeer]

Are you compressing the data to make .ZIP files?

Try making .7z files instead.

7z will utilize multi-threading whereas ZIP is normally single threaded.