Return-Path: <tiff-bounces@lists.maptools.org>
Received: from inas014.isp.belgacom.be (localhost [127.0.0.1])
        by inav003.isp.belgacom.be (8.12.11/8.12.11/Skynet-IN-AV-3.00) with ESMTP id j8P1Ve7l008541
        for <joris.at.lebbeke@skynet.be>; Sun, 25 Sep 2005 03:31:40 +0200
        (envelope-from <tiff-bounces@lists.maptools.org>)
Received: from inmx003.isp.belgacom.be (inmx003.isp.belgacom.be [195.238.3.11])
        by inas014.isp.belgacom.be (8.12.11/8.12.11/Skynet-IN-AS-2.03) with ESMTP id j8P1VYdH028413
        for <joris.at.lebbeke@skynet.be>; Sun, 25 Sep 2005 03:31:34 +0200
        (envelope-from <tiff-bounces@lists.maptools.org>)
Received: from duke.maptools.org (r153.dmsolutions.ca [209.217.116.153] (may be forged))
        by inmx003.isp.belgacom.be (8.12.11/8.12.11/Skynet-IN-2.32) with ESMTP id j8P1VN2u029550
        for <joris.at.lebbeke@skynet.be>; Sun, 25 Sep 2005 03:31:23 +0200
        (envelope-from <tiff-bounces@lists.maptools.org>)
Received: from duke.maptools.org (duke.maptools.org [127.0.0.1])
	by duke.maptools.org (8.12.11/8.12.10) with ESMTP id j8P1S730022292;
	Sat, 24 Sep 2005 21:28:09 -0400
Received: from outmx005.isp.belgacom.be (outmx005.isp.belgacom.be
	[195.238.2.102])
	by duke.maptools.org (8.12.11/8.12.10) with ESMTP id j8P1RsCr022277
	for <tiff@lists.maptools.org>; Sat, 24 Sep 2005 21:27:54 -0400
Received: from outmx005.isp.belgacom.be (localhost [127.0.0.1])
	by outmx005.isp.belgacom.be (8.12.11/8.12.11/Skynet-OUT-2.22) with
	ESMTP id j8P1RnDI012893
	for <tiff@lists.maptools.org>; Sun, 25 Sep 2005 03:27:50 +0200
	(envelope-from <joris.at.lebbeke@skynet.be>)
Received: from dfgrrxsknr4e (217-179.245.81.adsl.skynet.be [81.245.179.217])
	by outmx005.isp.belgacom.be (8.12.11/8.12.11/Skynet-OUT-2.22) with SMTP
	id j8P1Rjuf012870; Sun, 25 Sep 2005 03:27:46 +0200
	(envelope-from <joris.at.lebbeke@skynet.be>)
Message-ID: <002701c5c170$7444c8d0$d9b3f551@dfgrrxsknr4e>
From: "Joris" <joris.at.lebbeke@skynet.be>
To: "Frank Warmerdam" <warmerdam@pobox.com>, "TIFF" <tiff@lists.maptools.org>
References: <931f8ea905092314116e0918ce@mail.gmail.com>
Subject: Re: [Tiff] Additional Lossless Compression Schemes
Date: Sun, 25 Sep 2005 03:28:33 +0200
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2800.1506
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1506
Cc: 
X-BeenThere: tiff@lists.maptools.org
X-Mailman-Version: 2.1.4
Precedence: list
List-Id: TIFF and libtiff discussion <tiff.lists.maptools.org>
List-Unsubscribe: <http://lists.maptools.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@lists.maptools.org?subject=unsubscribe>
List-Archive: <http://lists.maptools.org/mailman/private/tiff>
List-Post: <mailto:tiff@lists.maptools.org>
List-Help: <mailto:tiff-request@lists.maptools.org?subject=help>
List-Subscribe: <http://lists.maptools.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@lists.maptools.org?subject=subscribe>
Sender: tiff-bounces@lists.maptools.org
Errors-To: tiff-bounces@lists.maptools.org
X-UIDL: 000000000023ceae.00000c5f
X-MSP-STORE: Sun, 25 Sep 2005 03:31:42 +0200
Status:   

Frank Warmerdam wrote:
> My understanding is that Deflate is ... not as widely supported.

It's hard to define 'wide support' in an objective manner. From where
I'm standing, it sure does *seem* (=subjective) to be widely supported,
though. I now a few attempts at TIFF codec that only support very
limited subsets (only uncompressed, or only G3 or G4 compressed, for
example), there's also Photoshop using Adobe's code base, and there's a
vast amount of apps using LibTiff. The first cannot be expected to
support flate compression, but the second and third do.

> Since then I have seen "gzip" on Unix, but I am not clear if that
> is just deflate or not.

It is.

> Are there any opinions on possibly incorporating LZMA as an
> additional compression type?

What is the yield? It'll take some time before the new compression mode
'penetrates' the software pool 'out there', so is it worth it? If the
yield is just 5% or even up to 15% better compression, I have doubts
about this.

Also, note Chris' comments on prediction. Chances are you're using flate
compression without prediction, and making this small adjustment could
yield considerably already.

Whilst I doubt that any new lossless compression type could be worth the
effort (except maybe JPEG2000, I don't know), I think it might be worth
looking into a PCD-type scheme. Support for large images is getting more
important as the years go by. We've currently got a real problem when
the bulk of mainstream readers try to access a large image in a TIFF.
They depend on single-chunk allocations and sometimes even decoding
LibTiff RGBA style. In practice, this means many readers hang for half
an hour of disc swapping activity, rendering the complete system
useless, before the anoyed user decides to kill the process.

The SubIFD scheme, tiled pyramide stuff, was an attempt at curing this.
It was a good attempt, and a logical one, but experience has proven
there are two great disadvantages to the sceme
a) The 'main image', the 'real image', the biggest one, is still the one
encoded in the primare 'top level' IFD. That is good, and logical, but
proves to not cure the problem with mainstream readers that don't even
attempt to look beyond that top level IFD.
b) The main image takes up space, and so do all of its downsamples. Lots
of redundancy there.

Perhaps we ought to look into a PCD-style encoding, where a 'reasonably
sized' image is encoded as the primary IFD, and a new SuperIFD kinda tag
points to progressively larger versions. Each larger image should be
encoded not as is, but as the difference between the image and the
upsampled version of the image that is one level smaller. A sortoff
upside-down tile pyramide, except that lower levels are delta's.

I can't be sure, but I think this is actually a good prediction scheme
on large images. It could turn out, I think, that such an upside-down
pyramide compresses to actually smaller a file then the single only
largest image, with flate compression. Plus, it cures the mainstream
readers problems with big images. Seems that is important, especially
with BigTIFF applications in mind.

Whether or not this applies to your quest... What is your quest? How did
you come to seek better lossless compression, for what application?


Joris Van Damme
info@awaresystems.be
http://www.awaresystems.be/
Download your free TIFF tag viewer for windows here:
http://www.awaresystems.be/imaging/tiff/astifftagviewer.html

_______________________________________________
Tiff mailing list: Tiff@lists.maptools.org
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
