Return-Path: <tiff-bounces@lists.maptools.org>
Received: from inas014.isp.belgacom.be (localhost [127.0.0.1])
        by inav027.isp.belgacom.be (8.12.11/8.12.11/Skynet-IN-AV-3.00) with ESMTP id j9S3RntA012913
        for <joris.at.lebbeke@skynet.be>; Fri, 28 Oct 2005 05:27:49 +0200
        (envelope-from <tiff-bounces@lists.maptools.org>)
Received: from inmx004.isp.belgacom.be (inmx004.isp.belgacom.be [195.238.5.48])
        by inas014.isp.belgacom.be (8.12.11/8.12.11/Skynet-IN-AS-2.03) with ESMTP id j9S3Rkm3007867
        for <joris.at.lebbeke@skynet.be>; Fri, 28 Oct 2005 05:27:46 +0200
        (envelope-from <tiff-bounces@lists.maptools.org>)
Received: from duke.maptools.org (r153.dmsolutions.ca [209.217.116.153] (may be forged))
        by inmx004.isp.belgacom.be (8.12.11/8.12.11/Skynet-IN-2.32) with ESMTP id j9S3ReVw027781
        for <joris.at.lebbeke@skynet.be>; Fri, 28 Oct 2005 05:27:41 +0200
        (envelope-from <tiff-bounces@lists.maptools.org>)
Received: from duke.maptools.org (duke.maptools.org [127.0.0.1])
	by duke.maptools.org (8.12.11/8.12.10) with ESMTP id j9S3IaaG030893;
	Thu, 27 Oct 2005 23:19:57 -0400
Received: from ms-smtp-03-eri0.texas.rr.com (ms-smtp-03.texas.rr.com
	[24.93.47.42])
	by duke.maptools.org (8.12.11/8.12.10) with ESMTP id j9S3IXG4030890
	for <tiff@lists.maptools.org>; Thu, 27 Oct 2005 23:18:34 -0400
Received: from black (cpe-66-68-60-3.austin.res.rr.com [66.68.60.3])
	by ms-smtp-03-eri0.texas.rr.com (8.12.10/8.12.7) with SMTP id
	j9S3ITRZ013067; Thu, 27 Oct 2005 22:18:29 -0500 (CDT)
Message-ID: <004401c5db6e$45290990$6501a8c0@black>
From: "David McKenzie" <davidmck@austin.rr.com>
To: "Frank Warmerdam" <warmerdam@pobox.com>,
        =?KOI8-R?B?+8XCxcvPIOXXx8XOyco=?= <shebeko@mail.ru>
References: <75489204593.20051027094051@mail.ru>
	<931f8ea90510270657g5007545dr443fc2b1c9855d58@mail.gmail.com>
Subject: Re: [Tiff] How located pixels with BitsPerSample not bound to byte?
Date: Thu, 27 Oct 2005 22:18:29 -0500
MIME-Version: 1.0
Content-Type: text/plain;
	charset="KOI8-R"
Content-Transfer-Encoding: 8bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2900.2527
X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2900.2527
X-Virus-Scanned: Symantec AntiVirus Scan Engine
Cc: tiff@lists.maptools.org
X-BeenThere: tiff@lists.maptools.org
X-Mailman-Version: 2.1.4
Precedence: list
List-Id: TIFF and libtiff discussion <tiff.lists.maptools.org>
List-Unsubscribe: <http://lists.maptools.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@lists.maptools.org?subject=unsubscribe>
List-Archive: <http://lists.maptools.org/mailman/private/tiff>
List-Post: <mailto:tiff@lists.maptools.org>
List-Help: <mailto:tiff-request@lists.maptools.org?subject=help>
List-Subscribe: <http://lists.maptools.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@lists.maptools.org?subject=subscribe>
Sender: tiff-bounces@lists.maptools.org
Errors-To: tiff-bounces@lists.maptools.org
X-UIDL: 000000000023ceae.00000dc1
X-MSP-STORE: Fri, 28 Oct 2005 05:27:55 +0200
Status:   

Hello Frank and Eugene:

>... I am including
>my code for arbitrary unpacking of odd bit data into words
>though it may be a bit opaque.I believe Bob uses a more
>efficient solution.
>...
>... <code excerpt> (see below)

It so happens that while testing a viewer application I'm working on, two 
problems were tracked to this section of GDAL code (in geotiff.cpp). The 
first is apparently a simple coding error. Notice that the local variable 
iPixelBitSkip in the code excerpt is initialized but never used. I saw this 
first in a separate section that handles 12-bit samples as a special case. 
There I fixed the problem by simply replacing the statement "iBitOffset += 
12" with "iBitOffset += iPixelBitSkip". In the section here that handles 
other 1-32 bit samples a similar fix is necessary. I haven't tried the very 
latest version of OpenEV, but I believe for the above reason it can't 
correctly display many of Bob Friesenhahn's example TIFFs (see below) with 
"contig" in the name.

The second problem involves this same section of code (1-32 bit sample 
unpacking) but it's really a compatibility issue instead of a code bug. It 
may take more effort to explain than it's worth given the almost nonexistent 
support for 24-bps TIFF files in applications. It does, however, touch upon 
Eugene's question. Anyway, there's a conflict between GDAL's and libtiff's 
interpretation of files with a 24-bit sample size. Both GDAL and 
GraphicsMagick apparently make the assumption that the byte order of 24-bps 
data isn't processor dependent. I suspect this is because Adobe's baseline 
TIFF specification, TIFF6.PDF (page 30), asserts that CPU-dependent byte 
order comes into play for sample sizes of 16 and 32 bits, and that 
uncompressed samples of other sizes are stored packed into an array of 
BYTEs, not SHORTs or LONGs. The document then says: "If the image data is 
stored as an array of SHORTs or LONGs, the byte ordering must be consistent 
with that specified in bytes 0 and 1 of the TIFF file header."

I realize this can't be taken as the final word on the TIFF format, but at 
least GDAL's driver code is compatible in that it expects libtiff's 
TIFFReadEncodedStrip() to return 24-bit samples in the same format it does 
other sizes that are less than 32 and not 16. When the driver unpacks a 
sequence of such samples it assumes that when a sample crosses a byte 
boundary, the most significant bits of the sample appear first, in the 
lowest addressed byte. Also conforming to this interpretation are the 24-bps 
versions of images Bob Friesenhahn created using GraphicsMagick v1.2:

ftp://ftp.remotesensing.org/pub/libtiff/pics-3.7.2.tar.gz
ftp://ftp.graphicsmagick.org/pub/outgoing/sample-tiffs.zip
ftp://ftp.graphicsmagick.org/pub/outgoing/sample-tiffs-le.zip

So given this evidence the problem seems to be with libtiff, which treats 
24-bit samples like it does 16- and 32-bit samples and byte-swaps them, say, 
if the file prefix is "MM" and the code is running on Windows. As a result, 
GDAL is unpacking 24-bps data in LS-to-MS byte order when it's expecting the 
opposite. OpenEV displays the "MM" version of 
flower-minisblack-strip-24.tif, for example, as a scrambled pattern.

If this is judged to be a libtiff error, I believe all that's necessary is 
to delete a couple of lines (an "else" clause) that references 
_TIFFSwab24BitData() in file tiff_dir.c. This is what I've done for the time 
being. The alternative is to assume the example 24-bps TIFFs are bad and 
treat 24-bps as a special case in GDAL's driver.

Whatever you folks decide, I'll certainly adapt to it. Thanks for all the 
great work!

--David

Frank Warmerdam wrote:
> On 10/27/05, ϋΕΒΕΛΟ εΧΗΕΞΙΚ <shebeko@mail.ru> wrote:
>>   If we talking about testing exotic images.
>>
>>   Look at the flower-rgb-contig-12.tif from the test images.
>>   It have RGB,Uncompressed, PlanarConfig=Continue, and
>>   BitsPerSample=12
>>
>>   How the pixels are located inside a strip in such case?
>>   I really cann't understand :)
>>
>>   Is it correct?
>>
>>   Bytes:       0   |    1    |    2   |   3    |   4
>>   Bits :   76543210|7654|3210|76543210|76543210|7654|3210
>>   Pixels:  rrrrrrrr|rrrr|gggg|gggggggg|bbbbbbbb|bbbb|rrrr|
>>   Samples:   hiR   |lowR|hiG |  lowG  |  hiB   |lowB| hiR
>
> Eugene,
>
> I thought it should work as you have shown, but when
> I review my code it seems the first byte will be the low
> order 8bits of red, not the high (and so forth).   I am including
> my code for arbitrary unpacking of odd bit data into words
> though it may be a bit opaque.  I believe Bob uses a more
> efficient solution.
>
> /*
> --------------------------------------------------------------------
> */ /*      Handle 1-32 bit integer data.
>     */ /*
>
>
>
> --------------------------------------------------------------------
> */ else { int iBit, iPixel, iBitOffset = 0; int     iPixelBitSkip,
> iBandBitOffset, iX, iY, nBitsPerLine;
>
>         if( poGDS->nPlanarConfig == PLANARCONFIG_CONTIG )
>         {
>             iPixelBitSkip = poGDS->nBands * poGDS->nBitsPerSample;
>             iBandBitOffset = (nBand-1) * poGDS->nBitsPerSample;
>         }
>         else
>         {
>             iPixelBitSkip = poGDS->nBitsPerSample;
>             iBandBitOffset = 0;
>         }
>
>         // bits per line rounds up to next byte boundary.
>         nBitsPerLine = nBlockXSize * poGDS->nBitsPerSample;
>         if( (nBitsPerLine & 7) != 0 )
>             nBitsPerLine = (nBitsPerLine + 7) & (~7);
>
>         iPixel = 0;
>         for( iY = 0; iY < nBlockYSize; iY++ )
>         {
>             iBitOffset = iBandBitOffset + iY * nBitsPerLine;
>
>             for( iX = 0; iX < nBlockXSize; iX++ )
>             {
>                 int  nOutWord = 0;
>
>                 for( iBit = 0; iBit < poGDS->nBitsPerSample; iBit++ )
>                 {
>                     if( poGDS->pabyBlockBuf[iBitOffset>>3]
>                         & (0x80 >>(iBitOffset & 7)) )
>                         nOutWord |= (1 << (poGDS->nBitsPerSample - 1
>                     - iBit)); iBitOffset++;
>                 }
>
>                 if( eDataType == GDT_Byte )
>                     ((GByte *) pImage)[iPixel++] = nOutWord;
>                 else if( eDataType == GDT_UInt16 )
>                     ((GUInt16 *) pImage)[iPixel++] = nOutWord;
>                 else if( eDataType == GDT_UInt32 )
>                     ((GUInt32 *) pImage)[iPixel++] = nOutWord;
>             }
>         }
>     }
>
> Best regards, 

_______________________________________________
Tiff mailing list: Tiff@lists.maptools.org
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
