tungwaiyip.info

home

about me

links

Blog

< April 2010 >
SuMoTuWeThFrSa
     1 2 3
4 5 6 7 8 910
11121314151617
18192021222324
252627282930 

past articles »

Click for San Francisco, California Forecast

San Francisco, USA

 

Data Compression Comparison

This is a follow up on my last post about data compression. After encoded my numerical data in a compact CSV format, I apply data compression before storing it in the disk. I have done a quick study on the two algorithm available in standard Python library, gzip and bzip2. The result is shown below. The original message's size is 537,776 bytes.

Gzip compression Result

Compression Level Compressed Size Compress time Decompress time
9 183,019 179 ms 5.51 ms
6 184,532 125 ms 5.48 ms
3 203,105 38.2 ms 5.54 ms

Bzip2 compression Result

Compression Level Compressed Size Compress time Decompress time
9 152,283 84.3 ms 29 ms
6 152,283 84.9 ms 29 ms
3 157,065 80.6 ms 26.9 ms
1 166,949 79.8 ms 26.7 ms

Surprisingly, bzip compress faster than gzip at level 9. Unfortunately compression performance is the least important for me. Compression ratio and decompression performance is far more important. Compression is only done one time. But fetching and decompressing the data is going to be done many times. It is hard for me to choose between the better compression ratio of bzip or the faster decompression time of gzip. For now I think I will stick with gzip.

2010.04.21 [] - comments

 

 

blog comments powered by Disqus

past articles »

 

BBC News

 

Migrant children: Global outcry rises over US border separations (20 Jun 2018)

 

EU to launch counter-tariffs against US on Friday (20 Jun 2018)

 

Hungary passes 'Stop Soros' law banning help for migrants (20 Jun 2018)

 

Gosport hospital deaths: Prescribed painkillers 'shortened 456 lives' (20 Jun 2018)

 

Baghdad mall sorry for turning away orphans at Eid (20 Jun 2018)

 

Astronauts eject UK-led space junk demo mission (20 Jun 2018)

 

Former Irish bank boss jailed over fraud (20 Jun 2018)

 

Italy migrants: Benetton criticised over ad campaign (20 Jun 2018)

 

World Cup 2018: Portugal 1-0 Morocco (20 Jun 2018)

 

'Of course Senegal will win the World Cup' - Fans react to opening game win (20 Jun 2018)

more »

 

SF Gate

 

China ready to compete in drug market (20 Jun 2018)

 

Verizon, AT&T, Sprint to end location data sales to brokers (20 Jun 2018)

 

Business News Roundup, June 20 (19 Jun 2018)

 

Elon Musk accuses Tesla employee of sabotage (19 Jun 2018)

 

Daily Briefing, June 20 (19 Jun 2018)

 

IBM computer proves formidable foe against 2 human debaters (19 Jun 2018)

more »


Site feed Updated: 2018-Jun-20 09:00