Return-Path: <tiff-bounces@remotesensing.org>
Received: from inmx015.isp.belgacom.be (inmx015.isp.belgacom.be [195.238.2.218])
        by mailst035.isp.belgacom.be (8.12.11/8.12.11/Skynet-MAILSTORE-2.14) with ESMTP id j08Gc3GT020264
        for <joris.at.lebbeke@skynet.be>; Sat, 8 Jan 2005 17:38:03 +0100
        (envelope-from <tiff-bounces@remotesensing.org>)
Received: from xserve.flids.com (adsl-065-083-169-202.sip.mco.bellsouth.net [65.83.169.202])
        by inmx015.isp.belgacom.be (8.12.11/8.12.11/Skynet-IN-2.32) with ESMTP id j08GbWsK001647
        for <joris.at.lebbeke@skynet.be>; Sat, 8 Jan 2005 17:37:37 +0100
        (envelope-from <tiff-bounces@remotesensing.org>)
Received: from adsl-065-083-169-202.sip.mco.bellsouth.net (localhost [127.0.0.1])
	by xserve.flids.com (Postfix) with ESMTP
	id 60B4D59EC00; Sat,  8 Jan 2005 11:36:45 -0500 (EST)
X-Original-To: tiff@remotesensing.org
Delivered-To: tiff@remotesensing.org
Received: from mail.howardsilvan.com (gateway.howardsilvan.com [64.146.212.27])
	by xserve.flids.com (Postfix) with ESMTP id 83F9759EBDF
	for <tiff@remotesensing.org>; Sat,  8 Jan 2005 11:36:42 -0500 (EST)
Received: from bilbo.x101.com (localhost.localdomain [127.0.0.1])
	by mail.howardsilvan.com (8.11.6/8.11.6) with ESMTP id j08Gac912315;
	Sat, 8 Jan 2005 08:36:38 -0800
Date: Sat, 8 Jan 2005 08:36:38 -0800
From: Lee Howard <faxguy@howardsilvan.com>
To: Joris <joris.at.lebbeke@skynet.be>
Subject: Re: [Tiff] JPEG (T.30-E, fax) in TIFF
Message-ID: <20050108163638.GK32383@bilbo.x101.com>
References: <20050107201814.GI21464@bilbo.x101.com>
	<000d01c4f52a$d9634650$53adf551@dfgrrxsknr4e>
Mime-Version: 1.0
Content-Type: text/plain; format=flowed; charset=ISO-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <000d01c4f52a$d9634650$53adf551@dfgrrxsknr4e>
X-Mailer: Balsa 1.4.1
Lines: 271
Cc: tiff@remotesensing.org
X-BeenThere: tiff@remotesensing.org
X-Mailman-Version: 2.1.2
Precedence: list
List-Id: TIFF and libtiff discussion  <tiff.remotesensing.org>
List-Unsubscribe: <http://xserve.flids.com/mailman/listinfo/tiff>,
	<mailto:tiff-request@remotesensing.org?subject=unsubscribe>
List-Archive: <http://xserve.flids.com/pipermail/tiff>
List-Post: <mailto:tiff@remotesensing.org>
List-Help: <mailto:tiff-request@remotesensing.org?subject=help>
List-Subscribe: <http://xserve.flids.com/mailman/listinfo/tiff>,
	<mailto:tiff-request@remotesensing.org?subject=subscribe>
Sender: tiff-bounces@remotesensing.org
Errors-To: tiff-bounces@remotesensing.org
X-UIDL: 1b9d77811f0c4d4f9d4229e22919b259

Ed Grissom

On 2005.01.07 18:36 Joris wrote:

> I'd say you need to make sure PhotometricInterpretation is set to 10
> (PHOTOMETRIC_ITULAB), and Compression is set to 7 (COMPRESSION_JPEG).

I had been testing with COMPRESSION_JPEG and PHOTOMETRIC_RGB, with no 
luck, as I mentioned.  Ed Grissom suggested that I try 
PHOTOMETRIC_YCBCR with YCBCRSUBSAMPLING left at the 2:2 default, and 
immediately after I tried that I was successful in being able to pass 
the data through tiffcp and tiff2ps with the exception of two warning 
messages:

fax000003467.tif: Warning, incorrect count for field "StripOffsets" (3, 
expecting 1); tag trimmed.
fax000003467.tif: Warning, incorrect count for field "StripByteCounts" 
(3, expecting 1); tag trimmed.

(Well, to be perfectly honest, I had been using libtiff-3.6.1 up until 
last night when I upgraded to 3.7.1.  The message above is from 3.7.1.  
In 3.6.1 it said "tag ignored" instead of "tag trimmed".)  However, 
viewing the JPEG-in-TIFF image there was a very noticeable blue tint, 
much like viewing the JPEG I linked before directly in Mozilla.  I'm 
not completely certain if the blue tint indicates that the fax machine 
that I'm using to send the image is broken, if the faxed JPEG uses a 
different color pallate than is being expected, or if the fax machine 
is buggy and isn't generating the proper JPEG tables.  In any case, the 
images as-is are not preferrable to usual black-and-white faxes due to 
the blue shift; so it can't be used that way.

> You also
> need to set ImageWidth and ImageLength tags, of course. You should
> have one
> single value of StripOffsets and StripByteCounts,

Which is the cause for the warning messages I noted above: the TIFFs 
I'm generating have three values in each of those tags.  But I don't 
understand why they do.  It seems to be affected by the SAMPLESPERPIXEL 
tag.  If I change the SAMPLESPERPIXEL tag to 1 I only get one single 
value of StripOffsets and StripByteCounts.  If I just change the 
SAMPLESPERPIXEL tag to 3 then I get three values in each of 
StripOffsets and StripByteCounts.  I don't know if I'm doing something 
wrong or if there is a libtiff bug in this.

> and rows per strip needs to
> equal ImageLength.

I can't use unlimited (-1)?  I'm accustomed to using unlimited rows per 
strip since I use but one strip per page with these kinds (ECM) of 
faxes.

> BitsPerSample needs to be 8 (8,8,8, to be more
> exact, but
> LibTiff should take care of that for you), SamplesPerPixel needs to 
> be 3.

I had these before, although as I mentioned SamplesPerPixel was a bit 
troublesome, as I mentioned.

> IIRC,
> LibTiff also requires you to set the PlanarConfiguration tag to 1
> (PLANARCONFIG_CONTIG), even though the spec says that's the default
> value and
> thus the tag should not be strictly requires.

It looks like libtiff sets this tag to 1 on its own.

> Now, for the less obvious. You might need to set the Decode tag
> (http://www.awaresystems.be/imaging/tiff/tifftags/decode.html),
> depending on
> whether the ITULAB data in the JPEG has default range or not.

Aha!  Well, this probably explains why my image appears blue.  How do I 
know what the correct values are for this tag?  I've tried the said 
defaults (0, 100, -21760/255, 21590/255, -19200/255, 31800/255), but 
there's really no way for me to view the image right now due to the 
issues I mention below.  Furthermore, the tag doesn't seem to be 
getting set, despite doing:

   TIFFSetField(tif, 433, 0, 100, -21760/255, 21590/255, -19200/255, 
31800/255);

> You'll also need to set the YCbCrSubSampling tag
> (http://www.awaresystems.be/imaging/tiff/tifftags/ycbcrsubsampling.html),
> since
> your subsampling values are not the default. Note that the tag's name
> incorrectly seems to imply that it is for YCbCr data only, but it
> should also
> apply in your case.

Unfortunately, the YCBCRSUBSAMPLING tag appears to be ignored by tiffcp 
when the photometric tag is not set for YCbCr.  Observe:

[root@gollum recvq]# tiffdump fax000003468.tif
fax000003468.tif:
Magic: 0x4949 <little-endian> Version: 0x2a
Directory 0: offset 236712 (0x39ca8) next 0 (0)
SubFileType (254) LONG (4) 1<2>
ImageWidth (256) SHORT (3) 1<1728>
ImageLength (257) SHORT (3) 1<2128>
BitsPerSample (258) SHORT (3) 3<8 8 8>
Compression (259) SHORT (3) 1<7>
Photometric (262) SHORT (3) 1<10>
FillOrder (266) SHORT (3) 1<2>
ImageDescription (270) ASCII (2) 11<3604278160\0>
Make (271) ASCII (2) 62<LT V.92 1.0 MT5634ZBA-V9 ...>
Model (272) ASCII (2) 62<LT V.92 1.0 MT5634ZBA-V9 ...>
StripOffsets (273) LONG (4) 3<118360 0 0>
Orientation (274) SHORT (3) 1<1>
SamplesPerPixel (277) SHORT (3) 1<3>
RowsPerStrip (278) LONG (4) 1<4294967295>
StripByteCounts (279) LONG (4) 3<118352 0 0>
XResolution (282) RATIONAL (5) 1<204>
YResolution (283) RATIONAL (5) 1<196>
PlanarConfig (284) SHORT (3) 1<1>
ResolutionUnit (296) SHORT (3) 1<2>
Software (305) ASCII (2) 27<HylaFAX (tm) Version 4.2 ...>
DateTime (306) ASCII (2) 20<2005:01:08 07:26:05\0>
HostComputer (316) ASCII (2) 16<gollum.x101.com\0>
YCbCrSubsampling (530) SHORT (3) 2<1 1>
[root@gollum recvq]# tiffcp fax000003468.tif foo.tif
fax000003468.tif: Warning, incorrect count for field "StripOffsets" (3, 
expecting 1); tag trimmed.
fax000003468.tif: Warning, incorrect count for field "StripByteCounts" 
(3, expecting 1); tag trimmed.
JPEGPreDecode: Warning, Improper JPEG sampling factors 2,2
Apparently should be 1,1..
JPEGPreDecode: Warning, Decompressor will try reading with sampling 
2,2..
JPEGLib: Warning, Application transferred too many scanlines.

> Perhaps we might be able to spot a problem if you post a code snippet.
> Mentioning the exact errors and warnings might also ring a bell.

The errors and warnings are mentioned above.  Here is a synopsis of the 
code (most of which I didn't write - most of it probably originated 
from Sam):

     TIFFSetField(tif, TIFFTAG_SUBFILETYPE,	FILETYPE_PAGE);
     TIFFSetField(tif, TIFFTAG_IMAGEWIDTH,	(uint32) 
params.pageWidth());
     TIFFSetField(tif, TIFFTAG_BITSPERSAMPLE, 8);
     TIFFSetField(tif, TIFFTAG_PHOTOMETRIC,	PHOTOMETRIC_ITULAB);
     TIFFSetField(tif, TIFFTAG_YCBCRSUBSAMPLING, 1, 1);
     TIFFSetField(tif, 433, 0, 100, -21760/255, 21590/255, -19200/255, 
31800/255); // Decode tag
     TIFFSetField(tif, TIFFTAG_ORIENTATION,	ORIENTATION_TOPLEFT);
     TIFFSetField(tif, TIFFTAG_SAMPLESPERPIXEL, 3);
     TIFFSetField(tif, TIFFTAG_ROWSPERSTRIP,	(uint32) -1);
     TIFFSetField(tif, TIFFTAG_PLANARCONFIG,	PLANARCONFIG_CONTIG);
     TIFFSetField(tif, TIFFTAG_FILLORDER,	(uint16) fillOrder);
     TIFFSetField(tif, TIFFTAG_XRESOLUTION,	(float) 
params.horizontalRes());
     TIFFSetField(tif, TIFFTAG_YRESOLUTION,	(float) 
params.verticalRes());
     TIFFSetField(tif, TIFFTAG_RESOLUTIONUNIT,	RESUNIT_INCH);
     TIFFSetField(tif, TIFFTAG_SOFTWARE,		 
HYLAFAX_VERSION);
     TIFFSetField(tif, TIFFTAG_IMAGEDESCRIPTION,	(const char*) 
id);
     char dateTime[24];
     time_t now = Sys::now();
     strftime(dateTime, sizeof (dateTime), "%Y:%m:%d %H:%M:%S", 
localtime(&now));
     TIFFSetField(tif, TIFFTAG_DATETIME,	    dateTime);
     TIFFSetField(tif, TIFFTAG_MAKE,	    (const char*) 
getManufacturer());
     TIFFSetField(tif, TIFFTAG_MODEL,	    (const char*) getModel());
     TIFFSetField(tif, TIFFTAG_HOSTCOMPUTER, (const char*) 
server.hostname);
     TIFFSetField(tif, TIFFTAG_COMPRESSION, COMPRESSION_JPEG);
/* this appears to be done in order to collect the StripOffsets value */
     u_char null[1];
     (void) TIFFWriteRawStrip(tif, 0, null, 0);
/* this is done repeatedly until all of the data is collected */
     TIFFWriteRawStrip(tif, strip, (tdata_t)buf, cc);
/* this is done to change the reliance on the DNL marker recvEOLCount 
is not a count of EOLs, in this case it is a value that we set from 
reading the DNL marker */
     uint32* stripbytecount;
     (void) TIFFGetField(tif, TIFFTAG_STRIPBYTECOUNTS, &stripbytecount);
     uint32 sbc = stripbytecount[0];
     u_char* stripdata = new u_char[sbc];
     TIFFReadRawStrip(tif, 0, stripdata, sbc);
     for (uint32 i = 0; i < sbc-6; i++) {
         if (stripdata[i] == 0xFF && stripdata[i+1] == 0xC0 && 
stripdata[i+5] == 0x00 && stripdata[i+6] == 0x00) {
             stripdata[i+5] = recvEOLCount >> 8;
             stripdata[i+6] = recvEOLCount & 0xFF;
             protoTrace("RECV: fixing zero image frame length in SOF 
marker at byte %lu to %lu", i, recvEOLCount);
         }
     }
     TIFFSetWriteOffset(tif, 0);
     TIFFWriteRawStrip(tif, 0, (tdata_t) stripdata, sbc);
/* these tags are owned by SGI specifically for HylaFAX */
     TIFFSetField(tif, TIFFTAG_FAXRECVPARAMS, (uint32) params.encode());
     TIFFSetField(tif, TIFFTAG_FAXRECVTIME, (uint32) 
server.getPageTransferTime());

Another problem that I'll mention is that TIFFTAG_FAXRECVPARAMS and 
TIFFTAG_FAXRECVTIME are not being set as expected.  They get set when 
using traditional (black-and-white) compression methods, but not with 
JPEG compression.  I don't know if this is a hard-coded limitation in 
libtiff or not, I haven't looked into it.

> Note that few TIFF readers are ITULAB aware. I don't know if tiffcp
> can or
> cannot handle such a TIFF you're trying to build.

Well, at first looks it appears that there are some hurdles, indeed, 
but maybe I'm doing something wrong.  However, if this fax machine is 
not incorrect or broken (blue-shift), then most JPEG viewers are not 
going to work with JPEG files as I've produced them before, either.

> In fact, it may be a
> challenge
> to find a reader that you can use to crosstest your results, and some
> code
> failing to handle your results might not be sufficient to indicate
> that you
> haven't yet succeeded perfectly fine...

For the most part I'm only really interested in libtiff being able to 
support these files.  I assume that if libtiff supports them that 
tiff2ps will produce a properly-colored PostScript document that can 
then be used in whatever way the user needs.  In other words, most 
people who use HylaFAX that I work with will not use the TIFF directly 
- instead, they will post-process the fax image into PDF.  As for other 
HylaFAX users that may choose to use the TIFFs directly - for example 
in a TIFF viewer - well, unless libtiff can post-process the TIFF from 
an unusable format to a usable one for them, then they'll just need to 
leave the to-be "color fax" option in HylaFAX disabled.  I don't think 
that color faxing is done very frequently, and when it is done with any 
regularity I suspect that JBIG compression is used with greater 
frequency than JPEG.  But, since libtiff doesn't support JBIG at the 
moment, I thought that I would have an easier time of getting JPEG 
support implemented, especially since I don't have a working 
JBIG-compressed fax sender always available to me.

> Please do keep us posted on the outcome.

Will do.  As I've indicated, I'm also planning on getting JBIG support 
going, via JBIG-KIT.  I received your last message on that subject, and 
I've looked into doing it, but unfortunately I don't understand libtiff 
well enough and neither do I understand JBIG-KIT well enough to be able 
to accomplish that right now.

> Are you a HylaFAX contributor?

Yes, I'm one of regulars.  The main direction in HylaFAX development 
that I try to work involves anything with fax protocol and modem 
support, although I have done some other things.  Other people tend to 
be more affluent at working with the HylaFAX job scheduler and such.

> The HylaFAX links page still links to
> libtiff.org. Can you change that, or point me to someone who can? The
> correct
> LibTiff URL is, instead, http://www.remotesensing.org/libtiff/.

Oh, and yes, I am one of the webmasters.  I've updated the link on the 
HylaFAX HOWTO, which I believe is the only relevant one.  It may take a 
while to propagate to the live-side, though, so you may not see a 
change immediately.

What happened to libtiff.org, may I ask?

Thanks.

Lee.
_______________________________________________
Tiff mailing list
Tiff@remotesensing.org
http://xserve.flids.com/mailman/listinfo/tiff
