Return-Path: <tiff-admin@remotesensing.org>
Received: from inmx008.isp.belgacom.be (inmx008.isp.belgacom.be [195.238.3.88])
        by mailst006.isp.belgacom.be (8.12.9/8.12.9/Skynet-MAILSTORE-2.14) with ESMTP id i3KEO7bR018328
        for <joris.at.lebbeke@skynet.be>; Tue, 20 Apr 2004 16:24:07 +0200
        (envelope-from <tiff-admin@remotesensing.org>)
Received: from yoda.imagelinks.com ([208.24.120.44])
        by inmx008.isp.belgacom.be (8.12.9/8.12.9/Skynet-IN-2.32) with ESMTP id i3KEO0Sj008700
        for <joris.at.lebbeke@skynet.be>; Tue, 20 Apr 2004 16:24:02 +0200
        (envelope-from <tiff-admin@remotesensing.org>)
Received: from yoda.imagelinks.com (localhost.localdomain [127.0.0.1])
	by yoda.imagelinks.com (8.11.6/8.11.6) with ESMTP id i3KEN4n22373;
	Tue, 20 Apr 2004 10:23:04 -0400
Received: from outmx014.isp.belgacom.be (outmx014.isp.belgacom.be [195.238.2.69])
	by yoda.imagelinks.com (8.11.6/8.11.6) with ESMTP id i3KEMvn22352
	for <tiff@remotesensing.org>; Tue, 20 Apr 2004 10:22:57 -0400
Received: from dfgrrxsknr4e (151-1.241.81.adsl.skynet.be [81.241.1.151])
        by outmx014.isp.belgacom.be (8.12.9/8.12.9/Skynet-OUT-2.21) with SMTP id i3KEKS7t024532;
	Tue, 20 Apr 2004 16:20:28 +0200
        (envelope-from <joris.at.lebbeke@skynet.be>)
Message-ID: <001101c426e2$aecea860$9701f151@dfgrrxsknr4e>
From: "Joris" <joris.at.lebbeke@skynet.be>
To: "Rob van den Tillaart" <rvdt@oce.nl>,
   "Tiff mailing list" <tiff@remotesensing.org>
Cc: "Frank Warmerdam" <warmerdam@pobox.com>
References: <E1BDui3-0007qF-00@rom.localdomain> <407DEC23.2010300@pobox.com> <E1BDwRk-0007wz-00@rom.localdomain> <407E2625.6562720A@oce.nl> <407E8F18.9050806@pobox.com> <001301c42426$c4856450$0a7d88d9@dfgrrxsknr4e> <4084DF65.CC18045D@oce.nl>
Subject: Re: [Tiff] Large TIFF files
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2800.1158
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
X-RAVMilter-Version: 8.4.3(snapshot 20030212) (outmx014.isp.belgacom.be)
Sender: tiff-admin@remotesensing.org
Errors-To: tiff-admin@remotesensing.org
X-BeenThere: tiff@remotesensing.org
X-Mailman-Version: 2.0.13
Precedence: bulk
List-Help: <mailto:tiff-request@remotesensing.org?subject=help>
List-Post: <mailto:tiff@remotesensing.org>
List-Subscribe: <http://remotesensing.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@remotesensing.org?subject=subscribe>
List-Id: TIFF and libtiff discussion <tiff.remotesensing.org>
List-Unsubscribe: <http://remotesensing.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@remotesensing.org?subject=unsubscribe>
List-Archive: <http://remotesensing.org/pipermail/tiff/>
Date: Tue, 20 Apr 2004 16:20:48 +0200
X-UIDL: 29cdda5b2b664a1cd0d38b60afa069d0

Rob wrote:
> There are in essence only 2 routes for 64 bit tiff
> - compatibility with existing
> - breaking with existing (but keeping as many good things as possible)
>
> The first leads to all kinds of trouble as mentioned in earlier mails.
> Frank proposed a new 64 bit format BTIFF (or BIF: Big Image File or BIg
> File) and
> for now this seems to me the most promissing way.

Does make sense. I did make a proposal in the other direction (sortoff), but the
incompatible 64bit BTIFF option that is a logical extension of 32bit TIFF is
certainly more clean ans less troublesome.

> It seems to me that many applications will not need 64 bit.

Don't be too sure. Future has a way of catching up with all expectations very
quickly.

> PRELIM. PROPOSAL BIF
> -----------------------
> All types will be mapped upon signed 64 bit,
> alignment to 64 bit or 8 bytes
>
> TYPES
> -----
> 1 = ASCII string 0 terminated
> 2 = BLOB bytearray
> 3 = DOUBLE
> 4 = LONG64
> 5 = RATIONAL64 LONG64/LONG64 (16 bytes)
> 6 = COMPLEX LONG64-LONG64 (16 bytes)
>
> All nummeric types are signed, this would make files max. 2^63 bytes in
> stead of 2^64
> seems large enough for now. 2^63 = 9.223.372.036.854.775.808
> All (tiff32) types are mapped upon long64 (even boolean)
> This seems a waste of bytes but if we talk about files >> 4GB these few
> bytes
> won't matter too much I assume.

As to the 'all 64bit values are signed' part, I can live with that. But I think
it is preferable to build on from the existing datatypes of 32bit TIFF. This has
the advantage of
- being validated by age and past experience
- being documented
- being coded up already

Also, I don't agree with the 'doesn't matter much, so let's go ahead and waste'
part. If there's no true benefit to wasting, then there's no true benefit to
wasting. Tag data that is an aray of words should not take up 8 bytes per word.
(If that is indeed what you intended to say here.)

> HEADER
> ---------
> Bytes 0-1: endianess // must we keep this ?
> Bytes 2-7: SAM 64 // in ASCII
> Bytes 8-15: 64 bit IFD offset // 0000000 for last IFD
>
>
> The number of directory entries is also a LONG64;

Endianness: Yes I think we must keep this, for these reasons
- There are two worlds in this world.
- Been validated by age and past experience
- Been documented
- Been coded up already

I'm repeating myself, but that's just my general opinion: I don't think we
should re-invent TIFF, but build on from it instead, merely logically extending
it with the new 64bit stuff. This implies not breaking the existing datatype
scheme, but merely adding a few entries to it. And it implies no building a new
library from scratch, but merely extending existing LibTiff, which is important
too. I don't see any downside to that, we needn't comprimise because of the
desire to stick with what has proven to work.

Magical signature value: Hey, someone remembers Sam?! But what has he got to do
with 64bit TIFF?

> TAGS
> -----
> Byte 0-7: tag
> Byte 8-15: type
> Byte 16-23: value (type=3-6) length (type=1,2)
> Byte 24-31: opt. value (type=5,6) 64bit offset (type=1,2)
>
>
> Tag's are the same as in the TIFF 6.0 spec. preceded with 0's.
> The range with the first 32 bit a 0 are reserved for this
> 'compatibility'
> The range with the first 32 bit a 1 are reserved for 'the commitee'
> (whoever ...)

Tag code: Again, I'd prefer sticking with the existing tag codes, which implies
that this remains a word, not suddenly and pointlessly grow to a 64bit value,
which would lead to duplicating history in the end.
Type code: Why 64bits? Existing scheme allows for 65536 datatypes, is more then
sufficient.
Count and offset: I'm not quite sure I understand your description of the count
field, but if you mean the logical extension, having them both grow to 64bit, I
do agree.

In general: the desire to extend TIFF's size limits doesn not mean we have to go
overboard and every single bit suddenly needs to take up 64 of them. What is
logically a Word value, has always been a Word value, and hasn't had the need
for more then a few dozen values at most sofar, shouldn't suddenly change,
there's no single reason for that.

> METATAG
> ----------
> There is one new TAG the metaTAG
> FFFFFFFF-00000001 // Metatag
> 00000000-00000001 // ASCII
> some length
> offset --> data
>
> data = "XXXXXXXX-XXXXXXXX:description"
> X = value of a new tag used in this file
>
> example data: "11111111-00000001: A tag for internal use of application
> X only."
>
> this way applications can extend the metadata in a documented way. Note
> that the tag is not registered in any way and only valid for this file.

Seems like a nice idea. On the other hand, seems like duplicate trouble. I
personally prefer the JPEG tradition when it comes to 'identifying proprietary
data'. I'd prefer recommending starting proprietary data with a short
nul-terminated string that identifies the writer. The downside of this is off
course that proprietary data cannot be said to be of type 64bit. So maybe your
idea is best.

> END PRELIM PROPOSAL
>
> I didn't think about the image data yet but for now compressions are
> just same,
> tiles and stripes are just as usual only they have a 64 bit
> offset/count. I think
> it is good practice to keep tiles small.

Yes, and I'd be in favor of recommending a particular max size. Like the old
spec recommended 8K, I'd recommend 2 meg these days. Not obligatory.

> [and the discussion continues ...]

Yes it does! We may be witnessing a historical moment. ;-)


Joris Van Damme
info@awaresystems.be
http://www.awaresystems.be
Download your free TIFF tag viewer for windows here:
http://www.awaresystems.be/imaging/tiff/astifftagviewer.html

_______________________________________________
Tiff mailing list
Tiff@remotesensing.org
http://remotesensing.org/mailman/listinfo/tiff
