Return-Path: <awaresystems.be+caf_=filtered=awaresystems.be@gmail.com>
Delivered-To: filtered@awaresystems.be
Received: from mx1.pub.mailpod2-cph3.one.com ([10.27.25.11])
	by mailstorage18.cst.mailpod2-cph3.one.com (Dovecot) with LMTP id wzlkLeuTvlntSwEAQS8vrQ
	for <filtered@awaresystems.be>; Sun, 17 Sep 2017 15:30:45 +0000
X-HalOne-ID: 2bb58131-9bbd-11e7-af03-e0d848957041
Received: from mail-oi0-f44.google.com (unknown [209.85.218.44])
	by mx1.pub.mailpod2-cph3.one.com (Halon) with ESMTPS
	id 2bb58131-9bbd-11e7-af03-e0d848957041;
	Sun, 17 Sep 2017 15:30:45 +0000 (UTC)
Received: by mail-oi0-f44.google.com with SMTP id l74so3725840oih.1
        for <filtered@awaresystems.be>; Sun, 17 Sep 2017 08:30:45 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-original-authentication-results:x-gm-message-state:delivered-to
         :user-agent:date:from:to:message-id:thread-topic:references
         :in-reply-to:mime-version:cc:subject:precedence:list-id
         :list-unsubscribe:list-archive:list-post:list-help:list-subscribe
         :content-transfer-encoding:sender:errors-to;
        bh=14PBRvqd7AsmXtJnoCAkyQeGcIYwztmd5rS5Y0nAcyY=;
        b=U07Ps9jyfnEMF+19aKxVGpy5kt2L2/99Ad62Raoq2+WKndUakroJfx2AIIejGh3CJP
         +ZVhay8inCU3bSN8XTRJmPkyBO2UJI5Go1/dkUuWSTXqRyi+3lob9TW0CLwnoU6S4a2V
         tULAACeq4DMWM2Eg7SdG5MJfyDyRFMGCPkSBgbYseMdV0G8UpLBE/6I9UzlQiZ0Vt/bT
         XI4yT/WhxucfVhAfzVBry0vPoqZ6RVgZPd+LpxBS5CiCPGatHagYsz4p8R87yqb8aEKd
         VM30HOyaU6Kj/VtBe9kurkRqOZ/clKcduSlaXrx+AOdiGP2v/DaCDoa27Y3VzT+cLvS5
         xMHA==
X-Original-Authentication-Results: mx.google.com;       spf=neutral (google.com: 94.23.4.142 is neither permitted nor denied by best guess record for domain of tiff-bounces@lists.maptools.org) smtp.mailfrom=tiff-bounces@lists.maptools.org
X-Gm-Message-State: AHPjjUiLgwlPxKPtAxIdUGrk2Ota22fxDBgm0P8F+mu1fc+SsQGkbKb7
	28AdZzWPjdcdCNhp2Oc7tMVz+i0jEUIWpAkC5Qmv9w==
X-Received: by 10.202.218.133 with SMTP id r127mr20267380oig.302.1505662243824;
        Sun, 17 Sep 2017 08:30:43 -0700 (PDT)
X-Forwarded-To: filtered@awaresystems.be
X-Forwarded-For: awaresystems.be@gmail.com filtered@awaresystems.be
Delivered-To: awaresystems.be@gmail.com
Received: by 10.74.122.195 with SMTP id a186csp3096743ooc;
        Sun, 17 Sep 2017 08:30:42 -0700 (PDT)
X-Received: by 10.28.125.205 with SMTP id y196mr7515933wmc.128.1505662242622;
        Sun, 17 Sep 2017 08:30:42 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; t=1505662242; cv=none;
        d=google.com; s=arc-20160816;
        b=MqpKKZphfSSbL5OhqsqjuF63TOHHkOIuv08YWFQQOFFfkdkwOLAW9IlII/oRiWi5jr
         O6ihVZGEW0uvq/qYGkYzVagBirfJX2nK6ZeowUXgoSYNXU8nSuf8Z9JemdAR4EVpgvtY
         DTreheBUuk0pFrl7H3lFewXzn4pPlF7+Syb86rQiWAXh3kJfweyy6Q+VdnOaKNZJMPVq
         yZNhDEj6T2XFp1MoLMerDyo2R3QWFMSQ3zMrD3ZEU4dWUapEbRjoxihU6H8GK8gA6p9q
         OkuTHZ3izaajDsERLZfayTLDu5znrjV4RHB9dSURUg/kA9UOqwWYORYTzWbZ6CSxA4ce
         eXAg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=errors-to:sender:content-transfer-encoding:list-subscribe:list-help
         :list-post:list-archive:list-unsubscribe:list-id:precedence:subject
         :cc:mime-version:in-reply-to:references:thread-topic:message-id:to
         :from:date:user-agent:arc-authentication-results;
        bh=14PBRvqd7AsmXtJnoCAkyQeGcIYwztmd5rS5Y0nAcyY=;
        b=mY3YTIeIy7RgAihdT6K5tdX4/Y/iCe8gata5xem5y7O5u/gV2ZKzdj6cQMe//1VSiH
         GZ/PvD9pA5bz1LrAJJpmG0ADyhGiprv2fA46ok+WteFdQ2af2JEOiPADNvfJ7YtV0C5g
         d3RNXwtna1j/F1x4qjsGkQcf0X2addMFWPQ3QY0AVTF4HuptRVAbTZiqA54Etlz/f2+q
         hQOXI4zg4URcUg28oJmt+sSVOqfQvma5HVGYsVyt0LQxoLKSrpOAJpWzTMtMXt7ndlQc
         lIxBhs3PCkjEFGE7OBGxdpDAecNoJBzQ/FbPzeb15QWxy79V7iDA9yfd/+7HYnkWqx4u
         U4Ow==
ARC-Authentication-Results: i=1; mx.google.com;
       spf=neutral (google.com: 94.23.4.142 is neither permitted nor denied by best guess record for domain of tiff-bounces@lists.maptools.org) smtp.mailfrom=tiff-bounces@lists.maptools.org
Return-Path: <tiff-bounces@lists.maptools.org>
Received: from lists.maptools.org (ks365428.kimsufi.com. [94.23.4.142])
        by mx.google.com with ESMTP id w65si4347711wmg.3.2017.09.17.08.30.41;
        Sun, 17 Sep 2017 08:30:42 -0700 (PDT)
Received-SPF: neutral (google.com: 94.23.4.142 is neither permitted nor denied by best guess record for domain of tiff-bounces@lists.maptools.org) client-ip=94.23.4.142;
Authentication-Results: mx.google.com;
       spf=neutral (google.com: 94.23.4.142 is neither permitted nor denied by best guess record for domain of tiff-bounces@lists.maptools.org) smtp.mailfrom=tiff-bounces@lists.maptools.org
Received: from [10.1.0.241] (localhost.localdomain [127.0.0.1])
	by lists.maptools.org (Postfix) with ESMTP id 3296935C05B;
	Sun, 17 Sep 2017 10:29:47 -0500 (EST)
Received: from mail-io0-f176.google.com (mail-io0-f176.google.com
	[209.85.223.176])
	by lists.maptools.org (Postfix) with ESMTPS id F211235C059
	for <tiff@lists.maptools.org>; Sun, 17 Sep 2017 10:29:36 -0500 (EST)
Received: by mail-io0-f176.google.com with SMTP id k101so13838879iod.0
	for <tiff@lists.maptools.org>; Sun, 17 Sep 2017 08:29:36 -0700 (PDT)
X-Google-Smtp-Source: AOwi7QBU8TIsi+oxKeeS4qP/usVxsvX7hPTpK6cokn54OVZPHoWEFe95c0TvMAae0klqRYCRHwpZFw==
X-Received: by 10.107.27.81 with SMTP id b78mr5836633iob.239.1505662174934;
	Sun, 17 Sep 2017 08:29:34 -0700 (PDT)
Received: from [192.168.254.79] (dyn-177.cmts01.cable.TORON17.iasl.com.
	[206.210.119.177]) by smtp.gmail.com with ESMTPSA id
	r132sm2509898iod.50.2017.09.17.08.29.32
	(version=TLS1 cipher=AES128-SHA bits=128/128);
	Sun, 17 Sep 2017 08:29:34 -0700 (PDT)
User-Agent: Microsoft-MacOutlook/14.7.7.170905
Date: Sun, 17 Sep 2017 11:29:24 -0400
From: Kemp Watson <kemp@objectivepathology.com>
To: Bob Friesenhahn <bfriesen@simple.dallas.tx.us>
Message-ID: <D5E407A8.4E963%kemp@objectivepathology.com>
Thread-Topic: [Tiff] TIFF tile size limit
References: <alpine.GSO.2.20.1709161446510.16875@freddy.simplesystems.org>
	<3f3538f1-6fd7-4acd-0945-9a03c0026a5d@codelibre.net>
	<alpine.GSO.2.20.1709161843290.16875@freddy.simplesystems.org>
	<D5E354A5.4E8DE%kemp@objectivepathology.com>
	<alpine.GSO.2.20.1709170843010.16875@freddy.simplesystems.org>
	<D5E3F79E.4E938%kemp@objectivepathology.com>
	<alpine.GSO.2.20.1709170911310.16875@freddy.simplesystems.org>
	<D5E3FE6A.4E94F%kemp@objectivepathology.com>
	<alpine.GSO.2.20.1709170957300.16875@freddy.simplesystems.org>
In-Reply-To: <alpine.GSO.2.20.1709170957300.16875@freddy.simplesystems.org>
Mime-version: 1.0
Cc: Tiff List <tiff@lists.maptools.org>
Subject: Re: [Tiff] TIFF tile size limit
X-BeenThere: tiff@lists.maptools.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: TIFF and libtiff discussion <tiff.lists.maptools.org>
List-Unsubscribe: <http://lists.maptools.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@lists.maptools.org?subject=unsubscribe>
List-Archive: <http://lists.maptools.org/mailman/private/tiff>
List-Post: <mailto:tiff@lists.maptools.org>
List-Help: <mailto:tiff-request@lists.maptools.org?subject=help>
List-Subscribe: <http://lists.maptools.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@lists.maptools.org?subject=subscribe>
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Sender: tiff-bounces@lists.maptools.org
Errors-To: tiff-bounces@lists.maptools.org


"Tile sizes are already allowed to be larger than the image dimensions by
the TIFF specification since they can spill over the right and bottom of
the image.  A one pixel image could be in a 1kx1k "tile=B2."

Ugh. That=B9s the root of the issue, for sure.

Yes, the compression is not really the issue (LZW will compress whitespace
to almost nothing), it=B9s the allocation of the raw rasters/tiles for the
decompressed data.

I just went back and re-read the Tiled Image specification. You are
completely correct, there is no formal restriction on tile size vs image
size, although there=B9s a lot of =B3not recommended=B2 verbage. Personally=
, I=B9d
be inclined to limit the tile size to the image size (technically, to the
16-pixel boundary just above the image size, or perhaps to the next larger
power-of-two size to provide 'quadrants'). That would break the
compatibility with the TIFF spec, though. Would that be a problem in
practice? Adobe=B9s not really keeping the spec up to date with modern needs
anyway, and BigTIFF is not a spec either. The reality is that libtiff is
diverging from the TIFF 6.0 specification.

W. Kemp Watson

kemp@objectivepathology.com




Objective Pathology Services Limited

8250 Lawson Road
Milton, Ontario
Canada  L9T 5C6

www.objectivepathology.com
tel. +1 (416) 970-7284








On 2017-09-17, 11:04 AM, "Bob Friesenhahn" <bfriesen@simple.dallas.tx.us>
wrote:

>On Sun, 17 Sep 2017, Kemp Watson wrote:
>
>> I know that we use very large bigtiffs, sometimes terabytes in size, but
>> currently we allocate only 512-pixel tiles. I could see that going to 4K
>> maximum in the near future, very practically.
>>
>> But, would limiting the tile size to be up to but not more than the full
>> image dimensions not essentially guarantee that a tiled implementation
>> would not use more memory than a full rasterized image (barring pointers
>> and small stuff)?
>
>Tile sizes are already allowed to be larger than the image dimensions
>by the TIFF specification since they can spill over the right and
>bottom of the image.  A one pixel image could be in a 1kx1k "tile".
>
>> I may well be missing some critical detail here - what in those sample
>> files is the root cause of the large allocations?
>
>That is a good question.  Some compressors are capable of remarkable
>compression ratios and so the files can claim large pixel dimensions
>although the file is very small.  In some cases it is not easy to know
>what a decoder can produce from a very small input.
>
>Even Rouault tells me that one of the compressors is theoretically
>capable of storing a 100000x100000 image in a few hundred bytes.
>
>Bob
>-- =

>Bob Friesenhahn
>bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
>GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/


_______________________________________________
Tiff mailing list: Tiff@lists.maptools.org
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
