Compressing data in Toit - zip

Question

The Toit library contains the zip.toit module, which allows to hope that the system supports data compression and decompression. Unfortunately, there are no examples. Is it possible to give the simplest example of compression and decompression of data, such as strings or binary arrays?

Answer 1

The zlib library currently only supports compression.
Fundamentally you have to:
create a compressor (zlib, or gzip).
feed the data in one task
read/use the compressed data in another task.
Here is an example:
import zlib
import bytes
import monitor
main:
// We use a semaphore to let the main task know when all the
// compressed data has been handled.
// If the data is just sent over the network, then the semaphore
// wouldn't be necessary. The second task (`t`) would just finish
// once all data has been processed.
done := monitor.Semaphore
// We use a byte-buffer to accumulate all the data. If the data is
// to be sent somewhere, then it's better to just do that directly,
// instead of using memory here.
accumulator := bytes.Buffer
// There are other encoders as well, but the gzip-encoder has
// the advantage that it produces `.gz` compatible data.
compressor := zlib.RunLengthGzipEncoder
// We create a second task that takes out the already compressed
// data of the compressor.
t := task::
while data := compressor.read:
accumulator.write data
// We increment the semaphore, so that the other (original) task
// knows that we are done processing the data.
done.up
// In this task we now add data. As example, it's just "foo", but any
// string or byte-array would work.
// Can be called multiple times.
compressor.write "foo"
// The compressor must be closed.
// This flushes the last remaining data, and lets the reading
// task know that it is done.
compressor.close
// We wait for the reading task to signal that all data is handled.
done.down
// Show the data that was accumulated.
// You would normally just send it somewhere, and not print it.
print accumulator.buffer.size
print accumulator.buffer

Answer 2

A small NB. You can also decode the resulting binary array in android. I tested this by writing a small java app:
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.util.zip.GZIPInputStream;
import java.util.zip.GZIPOutputStream;
public class MainActivity extends AppCompatActivity {
// byte array from toit:
byte[] bytes = { 31, -117, 8, 0, 0, 0, 0, 0, 0, -1, 75, -53, -49, 7, 0, 33, 101, 115, -116, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
static final String TAG = "GZip";
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
String foo = null;
try {
foo = decompress(bytes);
}
catch(IOException exception) {
Log.d(TAG, exception.toString());
}
Log.d(TAG, "decompress->[" + foo + "]");
}
public static String decompress(byte[] compressed) throws IOException {
final int BUFFER_SIZE = 32;
ByteArrayInputStream is = new ByteArrayInputStream(compressed);
GZIPInputStream gis = new GZIPInputStream(is, BUFFER_SIZE);
StringBuilder string = new StringBuilder();
byte[] data = new byte[BUFFER_SIZE];
int bytesRead;
while ((bytesRead = gis.read(data)) != -1) {
string.append(new String(data, 0, bytesRead));
}
gis.close();
is.close();
return string.toString();
}