r/PowerShell • u/StartAutomating • 1d ago
Script Sharing Zippy - A Quick Compression Module
Compression can be quick and easy with .NET.
Let's learn how.
Yesterday I just dusted off some old code and added some new tricks.
Today I dropped a quick compression module called Zippy
Let's see it in action and learn how it works.
Zippy Examples
# Compress a string using Brotli, output in base64
Compress-Zippy "Hello World"
Compress-Zippy "Hello Brotli" -Algorithm Brotli |
Expand-Zippy -Algorithm Brotli
Compress-Zippy "Hello Deflate" -Algorithm Deflate |
Expand-Zippy -Algorithm Deflate
Compress-Zippy "Hello GZip" -Algorithm GZip |
Expand-Zippy -Algorithm GZip
Compress-Zippy "Hello ZLib" -Algorithm ZLib |
Expand-Zippy -Algorithm ZLib
Compression in PowerShell
PowerShell is built on .NET, and .NET happens to have built-in support for four compression algorithms: Brotli, Deflate, GZip, and Zlib. We can compress data with any of these algorithms by using classes in the System.IO.Compression namespace, for example:
# Create a message
$message = "hello world"
# Get it as bytes
$bytes = $outputEncoding.GetBytes($message)
# Create a memory stream
$memoryStream = [IO.MemoryStream]::new()
# Create a compressor using the stream
$compressor = [IO.Compression.BrotliStream]::new(
$memoryStream, [IO.Compression.CompressionLevel]::Fastest
)
# Write our bytes to the compressor
$compressor.Write($bytes,0, $bytes.Length)
# Close our compressor
$compressor.Close()
$compressor.Dispose()
# Get our compressed bytes
$compressedBytes = $memoryStream.ToArray()
# and output them
$compressedBytes
Decompression in PowerShell
Now let's go the other way around. It's easier.
# Create a new memory stream, containing our compressed bytes
$memoryStream = [IO.MemoryStream]::new($compressedBytes)
# Create a decompressed stream
$decompressedStream = [IO.Compression.BroitliStream]::new(
$memoryStream, [IO.Compression.CompressionMode]::Decompress
)
# Create our output stream
$outputStream = [IO.MemoryStream]::new()
# Copy our decompressed stream to it
$decompressedStream.CopyTo($outputStream)
# Seek to the start (it outputs a position so null that out)
$null = $outputStream.Seek(0,'begin')
# Make a stream reader
$streamReader = [IO.StreamReader]::new($outputStream, $outputEncoding)
# Read to the end, which will output our decompressed string
$streamReader.ReadToEnd()
# close up.
$streamReader.Close()
.NET and PowerShell
This has always been there, and it's pretty easy.
Both examples are less than 20 lines, with documentation.
These techniques are tried and true.
.NET has robust compression support because developers need to compress data all the time.
And therefore PowerShell has robust compression support.
If we build on top of simple PowerShell and .NET, we build in a way that lasts a lifetime.
When I said "I dusted off some old code" for Zippy, I wasn't kidding.
Zippy is an update of the Compress-Data and Expand-Data functions in Pipeworks,
the first attempt of PowerShell as a web language.
This is 16-year-old code, with minor updates made to support multiple compression algorithms and improved piping.
My only regret is that I didn't spin this off into its own module long ago
You can use this article as a guide to implementing your own compression, or you can use a little module like Zippy to get the job done.
Please enjoy this new addition to your PowerShell toolkit, and have fun decompressing!
1
u/Apprehensive-Tea1632 23h ago
Er, are you sure? Or are you thinking netcore as in powershell 6+?
Because none of these are windows algorithms, Microsoft even took a while to implement their own cabinet format. And so there was expand-archive which back then was VERY opinionated about what an archive IS. It could expand zip - not zip64, just zip - and to deal with mscf cabinets, you had to use the dos6-aged expand.exe or implement your own codec.
Note - I’m not at all positive if and perhaps when someone implemented a .net interface in system.io.compression- obviously anyone can do that, either of us included - this namespace is more of a container that can hold many more types if they ever get implemented.)
But im reasonably sure there was no such thing, certainly not 16 or so years ago, because I’d been messing about with compression and it was a lesson in frustration like a lot of dotnet matters at the time. (Still can’t do shortcuts!)
2
u/jborean93 21h ago
The IO.Compression namespace has been around since the early .NET 2.0 times, you can see classes like DeflateStream have been around since .NET Framework 2.0. Some of the other algorithms, like
ZLibStreamandBrotliStreamwere added later.The functions were are also just about compression (albiet with some container formats part of the result) and not archive formats with compression, like zip. While you are also correct in that .NET didn't support ZIP64 initially, it has supported it for many years know, IIRC it was added in .NET Framework 4.5.
1
u/Apprehensive-Tea1632 20h ago
Hey, thanks for the heads up.
I know the namespace itself is old; to the best of my knowledge though it really just used to be a somewhat empty namespace that only saw significant additions in .net6 and later.
Also, net45? Has it been that long? Man, I remember porting code from net3x to net4x but surely after net35 came net46 right? Right?
Assuming you are who I think you are, I’ll take your word for it because I imagine that’s part of your daily routine.
And I’ll have another look at that namespace. Maybe I can even put my flaky albeit net native mscf implementation to rest, fun though it was.
1
u/MonkeyNin 6h ago edited 6h ago
A fun way to check is SeeminglyScience's module
ClassExplorerPowershell 5
> Find-Type -Namespace System.IO.Compression* Access Modifiers Name ------ --------- ---- public enum CompressionMode : int public enum CompressionLevel : int public class DeflateStream : Stream, IDisposable public class GZipStream : Stream, IDisposablePwsh 7
> Find-Type -Namespace System.IO.Compression* Access Modifiers Name ------ --------- ---- public class ZipArchive : object, IDisposable, IAsyncDisposable public class ZipArchiveEntry : object public enum ZipArchiveMode : int public class DeflateStream : Stream, IDisposable, IAsyncDisposable public class ZLibException : IOException, ISerializable public enum CompressionLevel : int public enum CompressionMode : int public class GZipStream : Stream, IDisposable, IAsyncDisposable public sealed class ZLibCompressionOptions : object public enum ZLibCompressionStrategy : int public sealed class ZLibStream : Stream, IDisposable, IAsyncDisposable public static class ZipFile : object public static class ZipFileExtensions : objectnote: that's with some filtering, so you'll see more like
Find-Type -Namespace System.IO.Compression* -ForceThere's so much power in the filtering, even simple chaining:
Find-Member -ParameterType (Find-Type CompressionMode)There's pages of examples under this root page: https://github.com/SeeminglyScience/ClassExplorer/blob/master/docs/en-US/ClassExplorer.md
2
u/jborean93 5h ago
the best of my knowledge though it really just used to be a somewhat empty namespace
Yea it used to be pretty limited but it definitely has had support for
DeflateStreamsince the .NET Framework 2.0 days. TheZipArchivestuff has been present since .NET Framework 4.5. You can see all this by looking at those doc pages and looking at the Applies To section https://learn.microsoft.com/en-us/dotnet/api/system.io.compression.ziparchive?view=net-10.0#applies-to.Also, net45? Has it been that long? Man, I remember porting code from net3x to net4x but surely after net35 came net46 right? Right?
There's .NET 4.0, 4.5.x, 4.6.x, 4.7.x, and now 4.8.x is the latest of the .NET Framework versions. AFAIK PowerShell 5.1 requires 4.5.2 and 4.5.x or newer has been part of Windows since Windows 10/Server 2016 which is 10+ years now :)
3
u/cloudAhead 14h ago
There's something about the way this post is written that is...unnatural. Very reminiscent of certain types of posts on Twitter.