|
||
---|---|---|
.gitignore | ||
ALGORITHM.md | ||
dump-extn.rb | ||
FORMAT.md | ||
huffman.c | ||
huffman.h | ||
huffman.rb | ||
Makefile | ||
README.md | ||
thing.rb | ||
unpack.c | ||
unpack.rb |
Overview
A POC to take a list of content sets (basically a listing of directories) and pack them into a format optimized for space efficieny and reading.
For details on the file format, please see FORMAT.md
, ALGORITHM.md
, and the
included source.
Compilation and Usage
This repo holds two commands, thing.rb
(TODO: give it a better name) and
unpack
. thing.rb
is used to pack content sets into our custom data format.
unpack
reads the format.
To compile the unpack
command, just run make
.
This requires make, gcc, and zlib-devel.
thing.rb
thing.rb
generates files in our packed data format from newline delimited
lists of content sets. It can also dump content sets from a hosted v1
entitlement certificate, and print the tree structure of the content sets.
Take in an v1 x509 certificate, and extract the content sets, output them to newline delimited output
./thing.rb d this-cert.pem
This would produce a file named this-cert.txt
To see this txt list, in the tree format, do:
./thing.rb p this-cert.txt | less
Process this output to generate the packed data format:
./thing.rb c this-cert.txt
This would produce a file named this-cert.bin
. The c
command expects as
input a file containing a newline delimited list of content sets; you are free
to manipulate the output from a pem file or come up with your own crazy listing
to push the boundaries of the data format.
thing.rb
supports a "-v" verbose flag to print debug information.
unpack
unpack
can read and examine files in our data format.
To view stats on a packed file (size of dictionary, number of unique nodes, etc):
./unpack s this-cert.bin
To dump the raw packed file as text:
./unpack r this-cert.bin
To reconstruct the content sets and dump them to stdout:
./unpack d this-cert.bin
To check if the path /content/rhel/6/6Server/x86_64/os/repodata/repomd.xml
matches a content set in this-cert.bin
:
./unpack c this-cert.bin /content/rhel/6/6Server/x86_64/os/repodata/repomd.xml
There is also a WIP ruby version of unpack, unpack.rb