Return-Path: <awaresystems.be+caf_=filtered=awaresystems.be@gmail.com>
Delivered-To: filtered@awaresystems.be
Received: from mx2.pub.mailpod2-cph3.one.com ([10.27.25.12])
	by mailstorage18.cst.mailpod2-cph3.one.com (Dovecot) with LMTP id LblsCpSivlmQ9gEAQS8vrQ
	for <filtered@awaresystems.be>; Sun, 17 Sep 2017 16:28:53 +0000
X-HalOne-ID: 4a9ee3d5-9bc5-11e7-aafe-e0d84895704e
Received: from mail-io0-f182.google.com (unknown [209.85.223.182])
	by mx2.pub.mailpod2-cph3.one.com (Halon) with ESMTPS
	id 4a9ee3d5-9bc5-11e7-aafe-e0d84895704e;
	Sun, 17 Sep 2017 16:28:52 +0000 (UTC)
Received: by mail-io0-f182.google.com with SMTP id h66so13844118ioh.11
        for <filtered@awaresystems.be>; Sun, 17 Sep 2017 09:28:52 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-original-authentication-results:x-gm-message-state:delivered-to
         :from:to:date:message-id:user-agent:in-reply-to:references
         :mime-version:subject:precedence:list-id:list-unsubscribe
         :list-archive:list-post:list-help:list-subscribe:mime-version:sender
         :errors-to;
        bh=VLF478+BrB8X8NxCUtEZbszs1guidcCNQa3K4ny3waA=;
        b=sYUIaR9eWbUJyh4ajgz4HDBDluzHABnnFm6ou2yV2qmxn22eX7r5a57SHFIIzFJgVd
         p3BC5qm364U6JWwWRh8km9tpk2ShOcTgEsMxpcGVEyE3ChTe53/2RvIKE8FX2glyDPoA
         ivCMI/6qyRjukQv5A8Qud42uFN/vM0xvqkwfwcVRvOQO0iG2TIxSJmXzYhWPkN9yx6Cw
         FZ55WRvaIUxfOQKIYX/SPg6dfZyTw2Z1T7lag6SnidiEmNJRm2o72xX74qpGLefTgBKY
         MDbOOyTvaWXnTQvbvPsfA40uywNjqmRVGajA9PKG05RJWDr5y6Q57o8ivw47YgW794BV
         1Rnw==
X-Original-Authentication-Results: mx.google.com;       spf=neutral (google.com: 94.23.4.142 is neither permitted nor denied by best guess record for domain of tiff-bounces@lists.maptools.org) smtp.mailfrom=tiff-bounces@lists.maptools.org
X-Gm-Message-State: AHPjjUh8Ew8Hwoi+CjIbhzkB59cxLfKoKhpCKyumZ0dDQIRNM8D/AQpz
	kJPTtd8dPY7nNhJD7BLAEmmCmevdt5AYcJvnV6n+lw==
X-Received: by 10.202.81.214 with SMTP id f205mr35479841oib.274.1505665731659;
        Sun, 17 Sep 2017 09:28:51 -0700 (PDT)
X-Forwarded-To: filtered@awaresystems.be
X-Forwarded-For: awaresystems.be@gmail.com filtered@awaresystems.be
Delivered-To: awaresystems.be@gmail.com
Received: by 10.74.122.195 with SMTP id a186csp3139976ooc;
        Sun, 17 Sep 2017 09:28:50 -0700 (PDT)
X-Google-Smtp-Source: AOwi7QBf+GU53yUC9CzlEEZChGbQB0ZmUywRByoNYwDdRO8T//uKe1amh61ZHBCz+ZpUrsO+wo27
X-Received: by 10.28.137.208 with SMTP id l199mr6395373wmd.123.1505665730221;
        Sun, 17 Sep 2017 09:28:50 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; t=1505665730; cv=none;
        d=google.com; s=arc-20160816;
        b=zOl+r4Q4wDi+jPaBv2pAHJYwPHKEjTED2exClOkaYlyqQGXPXZIHENwsslr/tMB/qY
         /un4WlzmL89ZXOE/mNHZZ0pgK772K0wS+SnjoOpH291id1eKzVj9OpF8puAtbGup/BLN
         mcdT+Y5fdmywlir0rVtcpoiG78MypMy1Ei7gAo/Gi75s1spAZlGr2MMANCxdwrUuAyl1
         2t70Zi54XhsIc8Z0BWmKZwJYVS+k01g6Tt7qyH8ujV5476acjZ2kPZ4HCZYqcEiPC5lZ
         vqK0+7eBvG7jVpY7VCIZdkEK/3WOl0Dxex2cX7M+0d/K5sssUrHRHpXpndgE12x1sOH6
         PrJA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=errors-to:sender:mime-version:list-subscribe:list-help:list-post
         :list-archive:list-unsubscribe:list-id:precedence:subject
         :mime-version:references:in-reply-to:user-agent:message-id:date:to
         :from:arc-authentication-results;
        bh=VLF478+BrB8X8NxCUtEZbszs1guidcCNQa3K4ny3waA=;
        b=m77KC4gHT581DGnv0+RS+N+e6bpD4AZbG1e9HuPvXro4eBViC5BIPw/P4KQTMZRERV
         DRYpxCOVuAXeW/oMmGCVbDqXcYbB9EYD3v2KvYTxK+OBbGpdjApYZLgogCA0SLCTYEmW
         clkJYe17gTHmtb1pe2NrguTWG0mqR2/+eNlhNoKj72SZAb6YmGj0v2k+irKUEExaP7Vd
         XvjBUtW+sWIma4Un5AEhVsNo3A2Gw6CCeXsDci10GjbgkLYeOfTlNpZYOQO/2Ng5iuuL
         N/d4EvZCWPtXL3RI+7IZjkquqhKkiFSFKfWbfZGBpSDk88BtTz4OD6XonT8/qRlbLObW
         0lwQ==
ARC-Authentication-Results: i=1; mx.google.com;
       spf=neutral (google.com: 94.23.4.142 is neither permitted nor denied by best guess record for domain of tiff-bounces@lists.maptools.org) smtp.mailfrom=tiff-bounces@lists.maptools.org
Return-Path: <tiff-bounces@lists.maptools.org>
Received: from lists.maptools.org (ks365428.kimsufi.com. [94.23.4.142])
        by mx.google.com with ESMTP id e2si4384859wrd.449.2017.09.17.09.28.49;
        Sun, 17 Sep 2017 09:28:50 -0700 (PDT)
Received-SPF: neutral (google.com: 94.23.4.142 is neither permitted nor denied by best guess record for domain of tiff-bounces@lists.maptools.org) client-ip=94.23.4.142;
Authentication-Results: mx.google.com;
       spf=neutral (google.com: 94.23.4.142 is neither permitted nor denied by best guess record for domain of tiff-bounces@lists.maptools.org) smtp.mailfrom=tiff-bounces@lists.maptools.org
Received: from [10.1.0.241] (localhost.localdomain [127.0.0.1])
	by lists.maptools.org (Postfix) with ESMTP id D719C35C061;
	Sun, 17 Sep 2017 11:27:47 -0500 (EST)
Received: from smtp5-g21.free.fr (smtp5-g21.free.fr [212.27.42.5])
	by lists.maptools.org (Postfix) with ESMTPS id EEA1435C05D
	for <tiff@lists.maptools.org>; Sun, 17 Sep 2017 11:27:40 -0500 (EST)
Received: from even-i700.localnet (unknown [88.162.84.61])
	(Authenticated sender: even.rouault)
	by smtp5-g21.free.fr (Postfix) with ESMTPSA id 38CBD5FFDA;
	Sun, 17 Sep 2017 18:27:34 +0200 (CEST)
From: Even Rouault <even.rouault@spatialys.com>
To: tiff@lists.maptools.org
Date: Sun, 17 Sep 2017 18:27:33 +0200
Message-ID: <9926854.M1JQDIlrTj@even-i700>
User-Agent: KMail/5.1.3 (Linux/4.4.0-92-generic; KDE/5.18.0; x86_64; ; )
In-Reply-To: <D5E407A8.4E963%kemp@objectivepathology.com>
References: <alpine.GSO.2.20.1709161446510.16875@freddy.simplesystems.org>
	<alpine.GSO.2.20.1709170957300.16875@freddy.simplesystems.org>
	<D5E407A8.4E963%kemp@objectivepathology.com>
MIME-Version: 1.0
Subject: Re: [Tiff] TIFF tile size limit
X-BeenThere: tiff@lists.maptools.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: TIFF and libtiff discussion <tiff.lists.maptools.org>
List-Unsubscribe: <http://lists.maptools.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@lists.maptools.org?subject=unsubscribe>
List-Archive: <http://lists.maptools.org/mailman/private/tiff>
List-Post: <mailto:tiff@lists.maptools.org>
List-Help: <mailto:tiff-request@lists.maptools.org?subject=help>
List-Subscribe: <http://lists.maptools.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@lists.maptools.org?subject=subscribe>
Content-Type: multipart/mixed; boundary="===============0473327243=="
Mime-version: 1.0
Sender: tiff-bounces@lists.maptools.org
Errors-To: tiff-bounces@lists.maptools.org

This is a multi-part message in MIME format.

--===============0473327243==
Content-Type: multipart/alternative; boundary="nextPart6539192.RRsh5KPdG0"
Content-Transfer-Encoding: 7Bit

This is a multi-part message in MIME format.

--nextPart6539192.RRsh5KPdG0
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="iso-8859-1"

On dimanche 17 septembre 2017 11:29:24 CEST Kemp Watson wrote:
> "Tile sizes are already allowed to be larger than the image dimensions by
> the TIFF specification since they can spill over the right and bottom of
> the image.  A one pixel image could be in a 1kx1k "tile=B2."
>=20
> Ugh. That=B9s the root of the issue, for sure.

Another issue, more with the implementation of libtiff than with the spec i=
tself, is if you have=20
a big number of tiles or strips, the allocation of the StripByteCount and S=
tripOffset arrays=20
can be very costly. Let's take the case of a 1 million x 1 million image wi=
th tiles of 32 x 32. The=20
allocation cost for those arrays is (1e6 / 16) * (1e6 / 16) * sizeof(uint64=
) * 2 =3D 62 GB (actually=20
libtiff would refuse to read the content of a tag if it is more than 2GB la=
rge), and currently=20
they are allocated and read entirely as soon as you read the first tile. li=
btiff should rather just=20
read in the file the values of the tile byte count & offset for the tile it=
 is going to read.
Even with a more reasonable tile size of 256x256, the cost is still 244 MB =
that you need to=20
allocate and read at file opening.

>=20
> Yes, the compression is not really the issue (LZW will compress whitespace
> to almost nothing), it=B9s the allocation of the raw rasters/tiles for the
> decompressed data.
>=20
> I just went back and re-read the Tiled Image specification. You are
> completely correct, there is no formal restriction on tile size vs image
> size, although there=B9s a lot of =B3not recommended=B2 verbage. Personal=
ly, I=B9d
> be inclined to limit the tile size to the image size (technically, to the
> 16-pixel boundary just above the image size, or perhaps to the next larger
> power-of-two size to provide 'quadrants'). That would break the
> compatibility with the TIFF spec, though. Would that be a problem in
> practice?=20

You'd probably break reading very small images (let's say 20x20) that use a=
 standard tiling=20
size of 256x256 (eg someone using GDAL's "gdal_translate in out.tif -co TIL=
ED=3DYES). So if we=20
went on this direction, we should probably need to allow a "reasonable" til=
e size of let's say=20
2K x 2K even for 1x1 images.

> Adobe=B9s not really keeping the spec up to date with modern needs
> anyway, and BigTIFF is not a spec either. The reality is that libtiff is
> diverging from the TIFF 6.0 specification.
>=20
> W. Kemp Watson
>=20
> kemp@objectivepathology.com
>=20
>=20
>=20
>=20
> Objective Pathology Services Limited
>=20
> 8250 Lawson Road
> Milton, Ontario
> Canada  L9T 5C6
>=20
> www.objectivepathology.com
> tel. +1 (416) 970-7284
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
> On 2017-09-17, 11:04 AM, "Bob Friesenhahn" <bfriesen@simple.dallas.tx.us>
>=20
> wrote:
> >On Sun, 17 Sep 2017, Kemp Watson wrote:
> >> I know that we use very large bigtiffs, sometimes terabytes in size, b=
ut
> >> currently we allocate only 512-pixel tiles. I could see that going to =
4K
> >> maximum in the near future, very practically.
> >>=20
> >> But, would limiting the tile size to be up to but not more than the fu=
ll
> >> image dimensions not essentially guarantee that a tiled implementation
> >> would not use more memory than a full rasterized image (barring pointe=
rs
> >> and small stuff)?
> >
> >Tile sizes are already allowed to be larger than the image dimensions
> >by the TIFF specification since they can spill over the right and
> >bottom of the image.  A one pixel image could be in a 1kx1k "tile".
> >
> >> I may well be missing some critical detail here - what in those sample
> >> files is the root cause of the large allocations?
> >
> >That is a good question.  Some compressors are capable of remarkable
> >compression ratios and so the files can claim large pixel dimensions
> >although the file is very small.  In some cases it is not easy to know
> >what a decoder can produce from a very small input.
> >
> >Even Rouault tells me that one of the compressors is theoretically
> >capable of storing a 100000x100000 image in a few hundred bytes.
> >
> >Bob
>=20
> _______________________________________________
> Tiff mailing list: Tiff@lists.maptools.org
> http://lists.maptools.org/mailman/listinfo/tiff
> http://www.remotesensing.org/libtiff/


=2D-=20
Spatialys - Geospatial professional services
http://www.spatialys.com

--nextPart6539192.RRsh5KPdG0
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html; charset="iso-8859-1"

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN" "http://www.w3.org/TR/REC-=
html40/strict.dtd">
<html><head><meta name=3D"qrichtext" content=3D"1" /><style type=3D"text/cs=
s">
p, li { white-space: pre-wrap; }
</style></head><body style=3D" font-family:'monospace'; font-size:9pt; font=
=2Dweight:400; font-style:normal;">
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">On dimanche=
 17 septembre 2017 11:29:24 CEST Kemp Watson wrote:</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &quot;=
Tile sizes are already allowed to be larger than the image dimensions by</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; the TI=
=46F specification since they can spill over the right and bottom of</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; the im=
age.  A one pixel image could be in a 1kx1k &quot;tile=B2.&quot;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; Ugh. T=
hat=B9s the root of the issue, for sure.</p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; ma=
rgin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; ">&nb=
sp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Another iss=
ue, more with the implementation of libtiff than with the spec itself, is i=
f you have a big number of tiles or strips, the allocation of the StripByte=
Count and StripOffset arrays can be very costly. Let's take the case of a 1=
 million x 1 million image with tiles of 32 x 32. The allocation cost for t=
hose arrays is (1e6 / 16) * (1e6 / 16) * sizeof(uint64) * 2 =3D 62 GB (actu=
ally libtiff would refuse to read the content of a tag if it is more than 2=
GB large), and currently they are allocated and read entirely as soon as yo=
u read the first tile. libtiff should rather just read in the file the valu=
es of the tile byte count &amp; offset for the tile it is going to read.</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Even with a=
 more reasonable tile size of 256x256, the cost is still 244 MB that you ne=
ed to allocate and read at file opening.</p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; ma=
rgin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; ">&nb=
sp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; Yes, t=
he compression is not really the issue (LZW will compress whitespace</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; to alm=
ost nothing), it=B9s the allocation of the raw rasters/tiles for the</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; decomp=
ressed data.</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; I just=
 went back and re-read the Tiled Image specification. You are</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; comple=
tely correct, there is no formal restriction on tile size vs image</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; size, =
although there=B9s a lot of =B3not recommended=B2 verbage. Personally, I=B9=
d</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; be inc=
lined to limit the tile size to the image size (technically, to the</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 16-pix=
el boundary just above the image size, or perhaps to the next larger</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; power-=
of-two size to provide 'quadrants'). That would break the</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; compat=
ibility with the TIFF spec, though. Would that be a problem in</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; practi=
ce? </p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; ma=
rgin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; ">&nb=
sp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">You'd proba=
bly break reading very small images (let's say 20x20) that use a standard t=
iling size of 256x256 (eg someone using GDAL's &quot;gdal_translate in out.=
tif -co TILED=3DYES). So if we went on this direction, we should probably n=
eed to allow a &quot;reasonable&quot; tile size of let's say 2K x 2K even f=
or 1x1 images.</p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; ma=
rgin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; ">&nb=
sp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; Adobe=
=B9s not really keeping the spec up to date with modern needs</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; anyway=
, and BigTIFF is not a spec either. The reality is that libtiff is</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; diverg=
ing from the TIFF 6.0 specification.</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; W. Kem=
p Watson</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; kemp@o=
bjectivepathology.com</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; Object=
ive Pathology Services Limited</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; 8250 L=
awson Road</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; Milton=
, Ontario</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; Canada=
  L9T 5C6</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; www.ob=
jectivepathology.com</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; tel. +=
1 (416) 970-7284</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; On 201=
7-09-17, 11:04 AM, &quot;Bob Friesenhahn&quot; &lt;bfriesen@simple.dallas.t=
x.us&gt;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; wrote:=
</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;On=
 Sun, 17 Sep 2017, Kemp Watson wrote:</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;&g=
t; I know that we use very large bigtiffs, sometimes terabytes in size, but=
</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;&g=
t; currently we allocate only 512-pixel tiles. I could see that going to 4K=
</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;&g=
t; maximum in the near future, very practically.</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;&g=
t; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;&g=
t; But, would limiting the tile size to be up to but not more than the full=
</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;&g=
t; image dimensions not essentially guarantee that a tiled implementation</=
p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;&g=
t; would not use more memory than a full rasterized image (barring pointers=
</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;&g=
t; and small stuff)?</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;</=
p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;Ti=
le sizes are already allowed to be larger than the image dimensions</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;by=
 the TIFF specification since they can spill over the right and</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;bo=
ttom of the image.  A one pixel image could be in a 1kx1k &quot;tile&quot;.=
</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;</=
p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;&g=
t; I may well be missing some critical detail here - what in those sample</=
p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;&g=
t; files is the root cause of the large allocations?</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;</=
p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;Th=
at is a good question.  Some compressors are capable of remarkable</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;co=
mpression ratios and so the files can claim large pixel dimensions</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;al=
though the file is very small.  In some cases it is not easy to know</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;wh=
at a decoder can produce from a very small input.</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;</=
p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;Ev=
en Rouault tells me that one of the compressors is theoretically</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;ca=
pable of storing a 100000x100000 image in a few hundred bytes.</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;</=
p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; &gt;Bo=
b</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; ______=
_________________________________________</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; Tiff m=
ailing list: Tiff@lists.maptools.org</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; http:/=
/lists.maptools.org/mailman/listinfo/tiff</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt; http:/=
/www.remotesensing.org/libtiff/</p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; ma=
rgin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; ">&nb=
sp;</p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; ma=
rgin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; ">&nb=
sp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">-- </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Spatialys -=
 Geospatial professional services</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-rig=
ht:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">http://www.=
spatialys.com</p></body></html>
--nextPart6539192.RRsh5KPdG0--


--===============0473327243==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Tiff mailing list: Tiff@lists.maptools.org
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
--===============0473327243==--
