Return-Path: <tiff-bounces@remotesensing.org>
Received: from inmx007.isp.belgacom.be (inmx007.isp.belgacom.be [195.238.3.89])
        by mailst035.isp.belgacom.be (8.12.11/8.12.11/Skynet-MAILSTORE-2.14) with ESMTP id j0E4rOiE016948
        for <joris.at.lebbeke@skynet.be>; Fri, 14 Jan 2005 05:53:24 +0100
        (envelope-from <tiff-bounces@remotesensing.org>)
Received: from xserve.flids.com (adsl-065-083-169-202.sip.mco.bellsouth.net [65.83.169.202])
        by inmx007.isp.belgacom.be (8.12.11/8.12.11/Skynet-IN-2.32) with ESMTP id j0E4r9Yk013216
        for <joris.at.lebbeke@skynet.be>; Fri, 14 Jan 2005 05:53:14 +0100
        (envelope-from <tiff-bounces@remotesensing.org>)
Received: from adsl-065-083-169-202.sip.mco.bellsouth.net (localhost [127.0.0.1])
	by xserve.flids.com (Postfix) with ESMTP
	id E3D9A5BB8CA; Thu, 13 Jan 2005 23:52:29 -0500 (EST)
X-Original-To: tiff@remotesensing.org
Delivered-To: tiff@remotesensing.org
Received: from mail.howardsilvan.com (gateway.howardsilvan.com [64.146.212.27])
	by xserve.flids.com (Postfix) with ESMTP id 7F2FC5BB8A7
	for <tiff@remotesensing.org>; Thu, 13 Jan 2005 23:52:26 -0500 (EST)
Received: from bilbo.x101.com (localhost.localdomain [127.0.0.1])
	by mail.howardsilvan.com (8.11.6/8.11.6) with ESMTP id j0E4qM919214;
	Thu, 13 Jan 2005 20:52:23 -0800
Date: Thu, 13 Jan 2005 20:52:22 -0800
From: Lee Howard <faxguy@howardsilvan.com>
To: Joris <joris.at.lebbeke@skynet.be>
Subject: Re: [Tiff] JPEG (T.30-E, fax) in TIFF
Message-ID: <20050114045222.GE7744@bilbo.x101.com>
References: <20050109054652.GC12383@bilbo.x101.com>
	<004501c4f9df$e230d1d0$22a3f551@dfgrrxsknr4e>
Mime-Version: 1.0
Content-Type: text/plain; format=flowed; charset=ISO-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <004501c4f9df$e230d1d0$22a3f551@dfgrrxsknr4e>
X-Mailer: Balsa 1.4.1
Lines: 161
Cc: Tiff mailing list <tiff@remotesensing.org>
X-BeenThere: tiff@remotesensing.org
X-Mailman-Version: 2.1.2
Precedence: list
List-Id: TIFF and libtiff discussion  <tiff.remotesensing.org>
List-Unsubscribe: <http://xserve.flids.com/mailman/listinfo/tiff>,
	<mailto:tiff-request@remotesensing.org?subject=unsubscribe>
List-Archive: <http://xserve.flids.com/pipermail/tiff>
List-Post: <mailto:tiff@remotesensing.org>
List-Help: <mailto:tiff-request@remotesensing.org?subject=help>
List-Subscribe: <http://xserve.flids.com/mailman/listinfo/tiff>,
	<mailto:tiff-request@remotesensing.org?subject=subscribe>
Sender: tiff-bounces@remotesensing.org
Errors-To: tiff-bounces@remotesensing.org
X-UIDL: 018fd56f6b995252574d8e18be8b90de

On 2005.01.13 18:22 Joris wrote:

Following your recommendations below (I've also commented below) I have 
produced:

   http://gateway.howardsilvan.com/fax000003471.tif

You'll notice that StripOffsets and StripByteCounts still have three 
values, and that seems to be a libtiff bug that is influenced 
(strangely enough) by the SamplesPerPixel tag.  Most of libtiff's 
utilities seem to compensate, however:

fax000003471.tif: Warning, incorrect count for field "StripOffsets" (3, 
expecting 1); tag trimmed.
fax000003471.tif: Warning, incorrect count for field "StripByteCounts" 
(3, expecting 1); tag trimmed.

As I've tried your recommended YCbCrSubsampling values before, it is 
not surprising, then, to see tiffcp produce these messages:

JPEGPreDecode: Warning, Improper JPEG sampling factors 2,2
Apparently should be 1,1..
JPEGPreDecode: Warning, Decompressor will try reading with sampling 
2,2..
JPEGLib: Warning, Application transferred too many scanlines.

In a nutshell, I think this is a tale-tale sign that libtiff cannot 
decode this data and that libtiff must be enchanced in order to support 
it.

> It took a while, but I was finally able to decode the TIFF with code
> that does
> not depend on LibTiff, for crosstesting. I'll privatly send you a
> reduced
> resolution ordinary YCbCr jpeg of the result in the copy to you, for
> checking.

Thank you, I did receive this image, and indeed, it looks quite perfect 
- better, in fact, than I've ever seen it, as I've no JPEG decoders 
that are capable of decoding the ITULAB JPEG data itself.

> ** StripOffsets/StripByteCounts **
> 
> The count of these tags is wrong, but you already knew.

Yes.  I'm quite certain that this is not an effect of the procedure 
that I follow, but rather, the problem seems entirely a result of the 
SamplesPerPixel tag.  I will, however, mention that the total TIFF file 
size is nearly twice what it should be due to the procedure that I 
follow.  This appears to be because I've misunderstood the 
TIFFSetWriteOffset function in that it doesn't actually cause 
previously "written" data to be "overwritten", but rather it seems to 
just move directory pointer to the current position, leaving the old 
data still in the file, but "unused".  I'd rather have the data 
overwritten.

> ** FillOrder **

I've removed this tag.

> ** YCbCrSubsampling **

I've not explicitly set this tag, so it is produced as 2,2.

> ** Decode **
> 
> I haven't (yet) been able to check whether the source JPEG contains
> the APP
> marker used to specify Decode parameters other than the ITULAB
> default, because
> I seem to have misplaced the spec that explained that APP marker and
> can't find
> any old code and can't remember, either. But, using the default to
> decode your
> image yields visually pleasing results, and I know from past
> experience that
> non-default Decode parameters are rare at best. So I guess we now know
> reasonably sure that you can probably ignore the Decode tag, too.

I've left this tag unset.

As for the JPEG markers, as best as I've been able to see, the source 
JPEG only contains the SOI, DHT, DQT, SOF, SOS, and DNL markers.  I do 
not find an APP marker in the source JPEG.

> ** More general **
> 
> Unless, of course, you are set out to build yourself a solution that
> works for
> *any* ITULAB color-fax-jpeg, as opposed to the ones from the same
> single source,
> in that case you'll probably have to detect proper values for the
> YCbCrSubsampling and Decode tags from your JPEG stream. Let me know if
> that is
> the case, and I shall try and find time to help out if I may, this is
> all very interesting...

Once libtiff can decode the ITULAB JPEG compressed TIFF file then I 
will begin testing how much my single test case is the norm.  I'll have 
various color fax senders start sending me test color faxes to see how 
well or poorly it fares.  If I do need to calculate the values of the 
YCbCrSubsampling and Decode tags, then we can address that at that 
point.  However, at this point I am content to assume that all color 
JPEG faxes will function as this one does (which, according to 
experience, will no doubt prove to be an incorrect assumption).

> Plus, I'm still keen on the general JPEG to JPEG-in-TIFF converter...
> I'm thinking of a general scheme, something like
> - sequentially add blocks of JPEG data to the single strip (so that
> the scheme
> fits a slow JPEG reception without temp buffer, like you want it)
> - while doing so, also sequentially feed these blocks to LibJpeg,
> which should
> be linked in LibTiff anyway, so as to read all relevant properties
> like color
> space, subsampling, etc, from the JPEG stream. At this stage, also
> especially
> note DNL, and if so, remember position to write fix-up and count lines
> - when that's done, fill in the tags with the info read from the JPEG
> stream.
> - also do the DNL fix-up now, if necessary

I'm open to ideas on how to better handle the SOF fixup with the 
information from the DNL marker.

> I'm unsure, however, how much of tag setting LibTiff allows to be
> postponed
> untill after the strip writing stage.

 From experience with traditional fax encoding (CCITT G3, G4), only the 
compression tag needs to be set beforehand.

> I you or anyone on this list wishes to join in to explore this
> further, now would be a good time to say so.

I'm always excited to do new and interesing things, but I'm not a 
libtiff coder, so I don't know of how much benefit my participation is.

> In any case, please keep us informed whatever happens next. It's not
> the first
> time ITULAB related stuff comes up in this list, so it would be nice
> to see and
> archive where this is going.

The way it looks now, is that I have to wait for ITULAB progress with 
libtiff.  So I'm looking forward to progress on this bug report:

   http://bugzilla.remotesensing.org/show_bug.cgi?id=736

Once libtiff supports ITULAB enough, then I will be able to enable 
HylaFAX to support at least *receiving* color faxes (using JPEG rather 
than JBIG, JBIG would come later).  *Sending* faxes requires that 
HylaFAX place a "tagline" or "header" on each page of an outgoing fax, 
and currently HylaFAX uses its own built-in G3/G4 decoder/encoder (very 
similar to libtiff's) to do that.  I don't want to have to build-in a 
JPEG decoder/encoder into HylaFAX, so I'll have to learn how to link-in 
libjpeg or get libtiff to do that work for HylaFAX.

Thanks.

Lee.
_______________________________________________
Tiff mailing list
Tiff@remotesensing.org
http://xserve.flids.com/mailman/listinfo/tiff
