Paq8p released

amorphis · 25th August 2008, 15:12

Paq8p is paq8o9 with a wav model that compresses 16/8bits stereo/mono uncompressed wav files smaller and faster. It also includes some minor improvements to bmp model.

uncompressed wav -> paq8o9 paq8p
chimes.wav 55776 -> 32866 22251
chord.wav 97016 -> 48601 34611
ding.wav 80856 -> 38456 17810
notify.wav 119384 -> 65696 49932
recycle.wav 25434 -> 15025 14967
ringin.wav 10026 -> 6799 4309
ringout.wav 5212 -> 1349 1182
start.wav 1192 -> 708 681
tada.wav 171100 -> 101754 87270
Windows XP Balloon.wav 6400 -> 2525 1967
Windows XP Battery Critical.wav 36910 -> 22035 16325
Windows XP Battery Low.wav 53864 -> 21276 16704
Windows XP Critical Stop.wav 39382 -> 19161 14868
Windows XP Default.wav 24530 -> 11890 8485
Windows XP Ding.wav 17132 -> 8161 6246
Windows XP Error.wav 44136 -> 20474 14798
Windows XP Exclamation.wav 42576 -> 17719 12139
Windows XP Hardware Fail.wav 36614 -> 18171 13045
Windows XP Hardware Insert.wav 36636 -> 16411 12158
Windows XP Hardware Remove.wav 36538 -> 16724 12349
Windows XP Information Bar.wav 20336 -> 12086 8551
Windows XP Logoff Sound.wav 179704 -> 77530 58004
Windows XP Logon Sound.wav 190208 -> 79105 57015
Windows XP Menu Command.wav 1404 -> 454 573
Windows XP Minimize.wav 22580 -> 9755 8937
Windows XP Notify.wav 48988 -> 17947 12099
Windows XP Pop-up Blocked.wav 29444 -> 3659 2131
Windows XP Print complete.wav 43762 -> 17863 10870
Windows XP Recycle.wav 22816 -> 14015 13931
Windows XP Restore.wav 19458 -> 7775 7299
Windows XP Ringin.wav 38930 -> 20705 18069
Windows XP Ringout.wav 22070 -> 10721 7646
Windows XP Shutdown.wav 282608 -> 146813 116760
Windows XP Start.wav 2202 -> 1350 1331
Windows XP Startup.wav 424644 -> 224628 183810
Total 2289868 -> 1131303 870218
Time(sec) 1908.01 287.65
MEM 241119658 250567306

paq8o9 -6 rafale.bmp 4149414 -> 551540 Time 347.11 sec, used 468933087 MEM
paq8p -6 rafale.bmp 4149414 -> 539067 Time 441.75 sec, used 486903535 MEM
paq8o9 -6 bliss.bmp 1440054 -> 288817 Time 129.32 sec, used 468933087 MEM
paq8p -6 bliss.bmp 1440054 -> 284653 Time 164.74 sec, used 486903535 MEM

kaitz · 25th August 2008, 17:00

 
timer paq8o10t.exe -8 Pea.wav
Pea.wav 18928940 -> 10486310
Time 1897.75 sec, used 1306457026 bytes of memory
 8
wav  18928932
Kernel Time  =     3.593 = 00:00:03.593 =   0%
User Time    =  1866.562 = 00:31:06.562 =  98%
Process Time =  1870.156 = 00:31:10.156 =  98%
Global Time  =  1898.109 = 00:31:38.109 = 100%
 
timer paq8p.exe -8 Pea.wav
Pea.wav 18928940 -> WAV 16bits/stereo 8528547
18928940 -> 8528577
Time 912.71 sec, used 1777293778 bytes of memory
Kernel Time  =     1.265 = 00:00:01.265 =   0%
User Time    =   894.609 = 00:14:54.609 =  98%
Process Time =   895.875 = 00:14:55.875 =  98%
Global Time  =   912.781 = 00:15:12.781 = 100%

Nice speed and compression.

amorphis · 25th August 2008, 17:08

thnx kaitz. you should also try -1 switch. much faster, slightly less compression.

kaitz · 25th August 2008, 17:18

amorphis wrote:

thnx kaitz. you should also try -1 switch. much faster, slightly less compression.

 
timer paq8p.exe -1 Pea.wav
Pea.wav 18928940 -> WAV 16bits/stereo 9520566
18928940 -> 9520596
Time 286.98 sec, used 38872130 bytes of memory
Kernel Time  =     0.875 = 00:00:00.875 =   0%
User Time    =   279.171 = 00:04:39.171 =  97%
Process Time =   280.046 = 00:04:40.046 =  97%
Global Time  =   287.000 = 00:04:47.000 = 100%

Indeed

LovePimple · 25th August 2008, 18:10

amorphis wrote:

Paq8p is paq8o9 with a wav model that compresses 16/8bits stereo/mono uncompressed wav files smaller and faster. It also includes some minor improvements to bmp model.

Thanks Andreas!

Mirror: Download

SvenBent · 25th August 2008, 23:51

has anybody compared it to nanozip ?

nanoflooder · 26th August 2008, 00:05

Why? It is obvious it'll do better because it's way more precise, though very slow

pat357 · 26th August 2008, 16:48

nanoflooder wrote:

Why? It is obvious it'll do better because it's way more precise, though very slow

Well, slower doesn't always seem to be better though

I tested on "Desmond Dekker - You Can Get It If You Really Want.wav" (44kHz, 16 bit, stereo) from http://www.squeezechart.com/TEST_Waveforms2.rar

28.306.988 1.wav - original wav
16.612.938 1wav_paq8p_1.paq8p
15.470.959 1wav_nz004_m50m_cDgr.nz
15.256.909 1wav_paq8p_3.paq8p
15.214.615 1wav_paq8p_6.paq8p
15.001.473 1.wav.mma3
14.925.740 1wav_nz004_m50m_co.nz
14.801.717 1wav_nz004_m50m_cc.nz
14.788.888 1wav_nz004_m50m_cOgr.nz
14.788.804 1wav_nz004_cOgr.nz

osmanturan · 26th August 2008, 16:52

@pat357:
I wonder how ACWAV perform on your test file. Also, it's direct ripping from an audio cd? or mp3->wav conversion?

pat357 · 26th August 2008, 17:00

osmanturan wrote:

@pat357:
I wonder how ACWAV perform on your test file. Also, it's direct ripping from an audio cd? or mp3->wav conversion?

acwav -c 1.wav 1wav_acwav.acwav
Compressed file size = 17794362 bytes ( 1.59:1 compression)

The wav I used is one of the test-files from www.squeezechart.com for his the WAV comparison tests, so I guess it's an "original" wav, not some MP3 -> WAV.

nanoflooder · 26th August 2008, 21:34

This attached file fails to be compressed by PAQ8O/PAQ8P. Regardless of the version or switch (even -0). Anyone knows what's the problem?

kaitz · 26th August 2008, 23:46

nanoflooder wrote:

This attached file fails to be compressed by PAQ8O/PAQ8P. Regardless of the version or switch (even -0). Anyone knows what's the problem?

File contains bmp image header at offset 0xEA38. It is correct, but no image data.
Header reports 921654 bytes for image. (640x480x3)
Dll file size is 79385.

Maybe testing file size agenst reported data size, witch is bad, but fixes problem.

kaitz · 27th August 2008, 18:42

amorphis wrote:

Paq8p is paq8o9 with a wav model that compresses 16/8bits stereo/mono uncompressed wav files smaller and faster. It also includes some minor improvements to bmp model.

In my tests converting jpeg->bmp 24bit and compressing it output is larger then older versions.
Scanned image is smaller.

http://spreadsheets.google.com/pub?k...a4O_vzKMmbDyOA

amorphis · 27th August 2008, 21:17

nanoflooder wrote:

This attached file fails to be compressed by PAQ8O/PAQ8P. Regardless of the version or switch (even -0). Anyone knows what's the problem?

kaitz wrote:

File contains bmp image header at offset 0xEA38. It is correct, but no image data.
Header reports 921654 bytes for image. (640x480x3)
Dll file size is 79385.

Maybe testing file size agenst reported data size, witch is bad, but fixes problem.

Up to version paq8o6 there is no problem, but from paq8o7 and in all subsequent versions i get the message 'Transform fails at 79385, input=0 decoded=0, skipping...' and compression is slightly reduced.

kaitz wrote:

In my tests converting jpeg->bmp 24bit and compressing it output is larger then older versions.
Scanned image is smaller.

http://spreadsheets.google.com/pub?k...a4O_vzKMmbDyOA

Strange, as in all my tests paq8p compresses better than paq8o9 even slightly. Could you send me P2180033x24.bmp and P3220352x24.bmp files?

kaitz · 27th August 2008, 21:38

amorphis wrote:

Up to version paq8o6 there is no problem, but from paq8o7 and in all subsequent versions i get the message 'Transform fails at 79385, input=0 decoded=0, skipping...' and compression is slightly reduced.

The problem is that file ends to soon. Detector reports correct size but no data is after header.
Maybe adding check after detection. Like from current offset +reported data len <= file size. If not, report as default data and correct size to the end of a file.

amorphis wrote:

Strange, as in all my tests paq8p compresses better than paq8o9 even slightly. Could you send me P2180033x24.bmp and P3220352x24.bmp files?

Look at http://paq8.googlecode.com/svn/testing/img/

Matt Mahoney · 27th August 2008, 22:34

Actually just a warning, not an error. If the transform fails, the file is compressed without the transform and decompression is still correct. During compression, the inverse transform is tested and the result is compared. If the result doesn't match, then the transform is skipped.

I posted paq8p to my PAQ page. http://cs.fit.edu/~mmahoney/compression/

nanoflooder · 27th August 2008, 22:45

Matt Mahoney wrote:

Actually just a warning, not an error. If the transform fails, the file is compressed without the transform and decompression is still correct. During compression, the inverse transform is tested and the result is compared. If the result doesn't match, then the transform is skipped.

I posted paq8p to my PAQ page. http://cs.fit.edu/~mmahoney/compression/

...but it also means, that if the result doesn't match, then the algorythm is slightly incorrect, right?..

kaitz · 27th August 2008, 22:47

nanoflooder wrote:

This attached file fails to be compressed by PAQ8O/PAQ8P. Regardless of the version or switch (even -0). Anyone knows what's the problem?

Fixed
When this problem happens pos in file will be increased +1. And detection continues...
compression for ppm.

test

LovePimple · 28th August 2008, 05:31

Thanks kaitz!

Any chance you could include the source code?

kaitz · 28th August 2008, 18:31

I am getting diff error when uncompressing wav data. It's in beginnig of a file: differ at 4092: file=179 archive=173
Compiled from original source. (VC 2008 or mingw)

Original binary uncompresses ok.

Using option -6 on 18 MB wav file.
Output is larger compered to my compile.

File used to test
http://paq8.googlecode.com/svn/tmp/Pea.7z

Can someone compile source and test usign option -6.

kaitz · 28th August 2008, 18:32

LovePimple wrote:

Thanks kaitz!

Any chance you could include the source code?

As soon as my testing is completed.

pat357 · 28th August 2008, 20:22

It seems Paq8p1 has also lost some compression compared to original Paq8p. The difference is not "much", but I wonder if their should be any.


Paq8p1
CanterburyCorpus>paq8p1 -4 files_paq8p1_4 files
Creating archive files_paq8p1_4.paq8p1 with 11 file(s)...
files/alice29.txt 152089 -> 34940
files/asyoulik.txt 125179 -> 31373
files/cp.html 24603 -> 5575
files/fields.c 11150 -> 2019
files/grammar.lsp 3721 -> 786
files/kennedy.xls 1029744 -> 12051
files/lcet10.txt 426754 -> 82809
files/plrabn12.txt 481861 -> 119763
files/ptt5 513216 -> 30394
files/sum 38240 -> 7661
files/xargs.1 4227 -> 1094
2810784 -> 328733
Time 209.30 sec, used 128879371 bytes of memory

Type    Size            Parts
unkn    1576504                 25
text    1234280                 28
Total parts: 53, fragmentation cost 265 bytes

Paq8p (original)
CanterburyCorpus>paq8p -4 files_paq8p_4 files
Creating archive files_paq8p_4.paq8p with 11 file(s)...
files/alice29.txt 152089 -> 35013
files/asyoulik.txt 125179 -> 31442
files/cp.html 24603 -> 5677
files/fields.c 11150 -> 2034
files/grammar.lsp 3721 -> 788
files/kennedy.xls 1029744 -> 12095
files/lcet10.txt 426754 -> 83152
files/plrabn12.txt 481861 -> 119967
files/ptt5 513216 -> 23154
files/sum 38240 -> 7486
files/xargs.1 4227 -> 1108
2810784 -> 322183
Time 204.78 sec, used 141516849 bytes of memory

Paq8o10t :
CanterburyCorpus>paq8o10t -4 files_paq8o10t_4 files
Creating archive files_paq8o10t_4.paq8o10t with 11 file(s)...
files/alice29.txt 152089 -> 34939
files/asyoulik.txt 125179 -> 31374
files/cp.html 24603 -> 5575
files/fields.c 11150 -> 2019
files/grammar.lsp 3721 -> 786
files/kennedy.xls 1029744 -> 12142
files/lcet10.txt 426754 -> 82799
files/plrabn12.txt 481861 -> 119767
files/ptt5 513216 -> 22924
files/sum 38240 -> 7670
files/xargs.1 4227 -> 1087
2810784 -> 321352
Time 209.67 sec, used 128948494 bytes of memory
 1576504
text  1234280

pat357 · 28th August 2008, 20:51

@KZ :
Paq8p1 hangs when decompressing Paq8p1 -1 1 pea.wav.
Could be usefull info for you to find the problem.

paq8p1 1.paq8p1
Extracting 1 file(s) from 1.paq8p1 -1
Extracting ./pea.wav 18928940 -> WAV 16bits/stereo     18927616

--Edit : When comparing the output to the original, I got diffs starting at FFC ..

amorphis · 28th August 2008, 21:25

kaitz wrote:

In my tests converting jpeg->bmp 24bit and compressing it output is larger then older versions.
Scanned image is smaller.

http://spreadsheets.google.com/pub?k...a4O_vzKMmbDyOA

Compressing with -7 gives better results:

paq8p -7 p2180033x24.bmp 9437238 -> 1708176 Time 625.64 sec, used 939888339 MEM
paq8o9 -7 p2180033x24.bmp 9437238 -> 1740765 Time 502.00 sec, used 838031811 MEM
paq8p -7 p3220352x24.bmp 14745654 -> 3438114 Time 935.59 sec, used 939888338 MEM
paq8o9 -7 p3220352x24.bmp 14745654 -> 3496710 Time 821.01 sec, used 838031810 MEM

Memory usage is much higher though.

kaitz · 28th August 2008, 21:36

pat357 wrote:

@KZ :
Paq8p1 hangs when decompressing Paq8p1 -1 1 pea.wav.
Could be usefull info for you to find the problem.

paq8p1 1.paq8p1
Extracting 1 file(s) from 1.paq8p1 -1
Extracting ./pea.wav 18928940 -> WAV 16bits/stereo     18927616

--Edit : When comparing the output to the original, I got diffs starting at FFC ..

Can somone compile amorphis posted paq8p source and test it on pea.wav.

pat357 · 28th August 2008, 21:45

kaitz wrote:

I am getting diff error when uncompressing wav data. It's in beginnig of a file: differ at 4092: file=179 archive=173
Compiled from original source. (VC 2008 or mingw)

Original binary uncompresses ok.

What the heck ! Or I messed up versions or the "original" paq8p gives diffs too on this pea.wav.
Tested it twice : paq8p -1 pea.wav / paq8p 1.paq8p
comparing output & original pea.wav :

Comparing files 1.wav and PEA.WAV
00800000: D0 00
00800001: F4 C1
00800002: DE FF
00800003: F4 E6
00800004: C6 F1
00800005: F3 62
00800006: E5 F3
00800007: F3 85
00800008: 73 D5
........

kaitz · 28th August 2008, 21:51

pat357 wrote:

What the heck ! Or I messed up versions or the "original" paq8p gives diffs too on this pea.wav.
Tested it twice : paq8p -1 pea.wav / paq8p 1.paq8p
comparing output & original pea.wav :

Comparing files 1.wav and PEA.WAV
00800000: D0 00
00800001: F4 C1
00800002: DE FF
00800003: F4 E6
00800004: C6 F1
00800005: F3 62
00800006: E5 F3
00800007: F3 85
00800008: 73 D5
........

Use option -4 atleast. Do locations change?

pat357 · 28th August 2008, 22:18

kaitz wrote:

Use option -4 atleast. Do locations change?

Paq8p (original version) -4 seems to be OK. Output is identical.
Does -1 work for you ? (with original paq8p binary, I mean)

kaitz · 28th August 2008, 22:48

pat357 wrote:

Paq8p (original version) -4 seems to be OK. Output is identical.
Does -1 work for you ? (with original paq8p binary, I mean)

option -1 will not work. Its becouse file is split BLOCK=MEM*64 where MEM=(0x10000<<level)
Level=option
So 0x800000 is where another block starts. So it will fail.

Input file is 18 MB so option -3 is needed minimum.

Original binary is ok. But if compiled from original source for me it fails. at 4092 (0xFFC)
If i am getting it right...

Compile original source and test atleast -3

pat357 · 28th August 2008, 22:51

Tested paq8p1 -4 on your "pea.wav".
Decompression hangs again :

paq8p1 1.paq8p1
Extracting 1 file(s) from 1.paq8p1 -4
Extracting ./pea.wav 18928940 -> WAV 16bits/stereo ^C  18927616

Differences starting from 30h :

Comparing files org.wav and PEA.WAV
00000030: 8C 8E
00000031: FF E2
00000032: 84 FA
00000033: FF 8D
00000034: A8 3E
00000035: FF CA
.....

28th August 2008, 20:51	#23
pat357 Member Join Date: May 2008 Location: Antwerp , country:Belgium , W.Europe Posts: 450	@KZ : Paq8p1 hangs when decompressing Paq8p1 -1 1 pea.wav. Could be usefull info for you to find the problem. paq8p1 1.paq8p1 Extracting 1 file(s) from 1.paq8p1 -1 Extracting ./pea.wav 18928940 -> WAV 16bits/stereo 18927616 --Edit : When comparing the output to the original, I got diffs starting at FFC .. Last edited by pat357 : 28th August 2008 at 20:59.

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

25th August 2008, 17:08	#3
amorphis Member Join Date: Aug 2008 Posts: 4	thnx kaitz. you should also try -1 switch. much faster, slightly less compression.

25th August 2008, 23:51	#6
SvenBent Member Join Date: May 2008 Location: Denmark Posts: 242	has anybody compared it to nanozip ?

26th August 2008, 00:05	#7
nanoflooder Member Join Date: Aug 2008 Location: Saint Petersburg, Russia Posts: 197	Why? It is obvious it'll do better because it's way more precise, though very slow

26th August 2008, 16:52	#9
osmanturan Programmer Join Date: May 2008 Location: Gaziantep, Turkiye Posts: 449	@pat357: I wonder how ACWAV perform on your test file. Also, it's direct ripping from an audio cd? or mp3->wav conversion?

27th August 2008, 22:34	#16
Matt Mahoney Expert Join Date: May 2008 Location: Melbourne, Florida, USA Posts: 291	Actually just a warning, not an error. If the transform fails, the file is compressed without the transform and decompression is still correct. During compression, the inverse transform is tested and the result is compared. If the result doesn't match, then the transform is skipped. I posted paq8p to my PAQ page. http://cs.fit.edu/~mmahoney/compression/

28th August 2008, 05:31	#19
LovePimple Moderator Join Date: May 2008 Location: Tristan da Cunha Posts: 611	Thanks kaitz! Any chance you could include the source code?

28th August 2008, 18:31	#20
kaitz Banned Join Date: May 2008 Location: EE Posts: 90	I am getting diff error when uncompressing wav data. It's in beginnig of a file: differ at 4092: file=179 archive=173 Compiled from original source. (VC 2008 or mingw) Original binary uncompresses ok. Using option -6 on 18 MB wav file. Output is larger compered to my compile. File used to test http://paq8.googlecode.com/svn/tmp/Pea.7z Can someone compile source and test usign option -6.