Return-Path: <tiff-bounces@remotesensing.org>
Received: from inmx015.isp.belgacom.be (inmx015.isp.belgacom.be [195.238.2.218])
        by mailst035.isp.belgacom.be (8.12.11/8.12.11/Skynet-MAILSTORE-2.14) with ESMTP id j0SJwKsY011103
        for <joris.at.lebbeke@skynet.be>; Fri, 28 Jan 2005 20:58:20 +0100
        (envelope-from <tiff-bounces@remotesensing.org>)
Received: from xserve.flids.com (adsl-065-083-169-202.sip.mco.bellsouth.net [65.83.169.202])
        by inmx015.isp.belgacom.be (8.12.11/8.12.11/Skynet-IN-2.32) with ESMTP id j0SJw04D011138
        for <joris.at.lebbeke@skynet.be>; Fri, 28 Jan 2005 20:58:03 +0100
        (envelope-from <tiff-bounces@remotesensing.org>)
Received: from adsl-065-083-169-202.sip.mco.bellsouth.net (localhost [127.0.0.1])
	by xserve.flids.com (Postfix) with ESMTP
	id 557D761035D; Fri, 28 Jan 2005 14:46:33 -0500 (EST)
X-Original-To: tiff@remotesensing.org
Delivered-To: tiff@remotesensing.org
Received: from langley.zn-ag.com (langley.zn.ruhr-uni-bochum.de
	[134.147.129.82])
	by xserve.flids.com (Postfix) with ESMTP id 42F4660E29D
	for <tiff@remotesensing.org>; Fri, 28 Jan 2005 08:32:09 -0500 (EST)
Received: from tijuana.zn-ag.com (tijuana.zn-ag.com [192.168.10.5])
	by langley.zn-ag.com (8.12.9/8.12.8) with ESMTP id j0SBIQU3018015
	(version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO);
	Fri, 28 Jan 2005 12:18:27 +0100
Received: from mailserver.zn-ag.com (berlin.zn-ag.com [192.168.10.1])
	by tijuana.zn-ag.com (8.12.9/8.12.8) with ESMTP id j0SASkDe002917;
	Fri, 28 Jan 2005 11:28:46 +0100
Received: from [134.147.254.119] (hiroshima [134.147.254.119])
	by mailserver.zn-ag.com (Postfix on SuSE Linux eMail Server 3.0) with
	ESMTP id 875F5118171; Fri, 28 Jan 2005 11:28:46 +0100 (CET)
Message-ID: <41FA13DE.9050603@viisage.com>
Date: Fri, 28 Jan 2005 11:28:46 +0100
From: Michael Rinne <mrinne@viisage.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; de-AT;
	rv:1.7) Gecko/20040616
X-Accept-Language: en-us, en, de
MIME-Version: 1.0
To: Edward Lam <edward@sidefx.com>
Subject: Re: [Tiff] issues with TIFFStreamOpen
References: <41F69009.1020609@viisage.com> <41F944CA.5040806@sidefx.com>
In-Reply-To: <41F944CA.5040806@sidefx.com>
Content-Type: multipart/mixed; boundary="------------090705060300080704080006"
Cc: tiff@remotesensing.org
X-BeenThere: tiff@remotesensing.org
X-Mailman-Version: 2.1.2
Precedence: list
List-Id: TIFF and libtiff discussion  <tiff.remotesensing.org>
List-Unsubscribe: <http://xserve.flids.com/mailman/listinfo/tiff>,
	<mailto:tiff-request@remotesensing.org?subject=unsubscribe>
List-Archive: <http://xserve.flids.com/pipermail/tiff>
List-Post: <mailto:tiff@remotesensing.org>
List-Help: <mailto:tiff-request@remotesensing.org?subject=help>
List-Subscribe: <http://xserve.flids.com/mailman/listinfo/tiff>,
	<mailto:tiff-request@remotesensing.org?subject=subscribe>
Sender: tiff-bounces@remotesensing.org
Errors-To: tiff-bounces@remotesensing.org
X-UIDL: 7f0264eba4bfd3a09c1076b12fe3ded9

This is a multi-part message in MIME format.
--------------090705060300080704080006
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

Hi Edward,

Edward Lam wrote:
> 
> Sorry for the late reply as I've been swamped. I had contributed the 
> libtiff changes that we made in our local baseline against tiff 3.4. It 
> appears that the code has never been used for writing files at an 
> arbitrary location. I don't see your fix in the attached file from your 
> orignal email though. I did a diff with my own version and found no real 
> differences. Perhaps you attached the wrong file?

sorry, first I attached the file in my mail tool, then I restored the 
original file in order to do another test before sending the mail.

This time the correct file should be attached 
(tif_stream_old_patch.cxx), although it looks like it is too late. ;-)

I made a diff of your changes and mine. Concerning issue 1, it looks 
pretty much the same, except that it extended the tiffis_data class by a 
pointer to an ostream object and therefore renamed the class to 
tiffios_data, whereas you decided to use a new class tiffos_data.

> I believe you're right 
> though as _tiffosSeekProc() should behave the same ass _tiffisSeekProc. 
> The TIFFStreamOpen() code was previously using tiff 3.4 which didn't 
> have this extra call to TIFFSeekOpen() from TIFFClientOpen().
> 
> [...]
> 
> If no one has a test case which can prove that the seek is required, I 
> propose that we scrap the call to TIFFSeekFile() in TIFFClientOpen() 
> regardless. If the seek is really necessary, I think it should be up to 
> the caller of TIFFClientOpen() to do it. ReadOK() shouldn't change the 
> file pointer anyhow, if it does, then it is a bug in the readproc.

It is probably a good idea to get rid off that seek, if it is really 
unnecessary.

> How does anyone expect to treat streams generically? <sigh> We could do 

I totally agree with that. ;-(
Unfortunately, the standard falls short for streams, leaving too much 
flexibility to the compiler builders, resulting in compiler dependent 
implementations which makes it impossible to use stream past the 
boundaries where you can guarantee that the same compiler with the same 
version is used.
Hence, one can not use streams within the API of SDKs. ;-(

> How does anyone expect to treat streams generically? <sigh> We could do 
> something like this in TIFFStreamOpen(const char *, ostream *)
> 
>     if( !os->fail() && (toff_t)os->tellp() < 0 ) {
>         *os << '\0';
>         os->seekp(0);
>     }
> 
>> the ostringstream. This results almost in a valid stream, but it fails
>> at the end with the error message 'Error writing data for field
>> "BitsPerSample".'. I tracked it down to the macro SeekOK which gets
>> called in tif_dirwrite.c at line 1066. This macro calls _tiffosSeekProc
>> with an offset that seems to exceed the amount of data written to the
>> stream and therefore the ostream method returns a bad offset.
> 
> Does anyone know if this is supposed to be the way as to how it works? I 
> assume that libtiff will at some point go back and fill in the missing 
> data? We could workaround this by writing intermediate '\0' characters 
> if the seek goes beyond the end of the stream.
> 
> I've attached a new tif_stream.cxx incorporating all the changes 
> mentioned above. I haven't tested them though. Please let me know if it 
> works for you. It's much appreciated, Michael!

Thanks for quickly providing a fix!

With this fix I get a valid TIFF image with the correct size and color 
depth, but unfortunately it is all black.

I had a look at your changes and noticed that at one point

118:    switch( whence ) {
119:        case SEEK_SET:
120:            origin = data->myStreamStartPos;
121:            break;

seeks to the original starting position and then

135:    if( origin + off > data->myStreamStartPos ) {

kicks in, resulting in

140:    // fill the intermediate space
141:    num_fill = origin + off - (toff_t)os->tellp();

to fill in zeros from the start of the file. This overwrote the entire 
image data which have been already written to the file. ;-(

I patched this to

140:    // extend the stream to the expected size
141:    os->seekp(0, ios::end);
142:    num_fill = origin + off - (toff_t)os->tellp();

which resulted in a valid TIFF file with my test image.

The patched file (tif_stream.cxx) is attached to this mail, hopefully I 
do not mess it up this time.

Thanks a lot for your help!

Cheers,
Michael

--------------090705060300080704080006
Content-Type: text/plain;
 name="tif_stream_old_patch.cxx"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="tif_stream_old_patch.cxx"

/* $Id: tif_stream.cxx,v 1.2 2004/11/13 11:19:15 dron Exp $ */

/*
 * Copyright (c) 1988-1996 Sam Leffler
 * Copyright (c) 1991-1996 Silicon Graphics, Inc.
 *
 * Permission to use, copy, modify, distribute, and sell this software and 
 * its documentation for any purpose is hereby granted without fee, provided
 * that (i) the above copyright notices and this permission notice appear in
 * all copies of the software and related documentation, and (ii) the names of
 * Sam Leffler and Silicon Graphics may not be used in any advertising or
 * publicity relating to the software without the specific, prior written
 * permission of Sam Leffler and Silicon Graphics.
 * 
 * THE SOFTWARE IS PROVIDED "AS-IS" AND WITHOUT WARRANTY OF ANY KIND, 
 * EXPRESS, IMPLIED OR OTHERWISE, INCLUDING WITHOUT LIMITATION, ANY 
 * WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.  
 * 
 * IN NO EVENT SHALL SAM LEFFLER OR SILICON GRAPHICS BE LIABLE FOR
 * ANY SPECIAL, INCIDENTAL, INDIRECT OR CONSEQUENTIAL DAMAGES OF ANY KIND,
 * OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS,
 * WHETHER OR NOT ADVISED OF THE POSSIBILITY OF DAMAGE, AND ON ANY THEORY OF 
 * LIABILITY, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE 
 * OF THIS SOFTWARE.
 */

/*
 * TIFF Library UNIX-specific Routines.
 */
#include <iostream>
#include "tiffiop.h"

using namespace std;

class tiffios_data
{
  public:

	istream	*myIS;
	ostream	*myOS;
	long	myStreamStartPos;
};

static tsize_t
_tiffosReadProc(thandle_t, tdata_t, tsize_t)
{
	return 0;
}

static tsize_t
_tiffisReadProc(thandle_t fd, tdata_t buf, tsize_t size)
{
	tiffios_data	*data = (tiffios_data *)fd;

	data->myIS->read((char *)buf, (int)size);

	return data->myIS->gcount();
}

static tsize_t
_tiffosWriteProc(thandle_t fd, tdata_t buf, tsize_t size)
{
	tiffios_data	*data = (tiffios_data *)fd;

	int	pos = data->myOS->tellp();

	data->myOS->write((const char *)buf, size);

	return ((int)data->myOS->tellp()) - pos;
}

static tsize_t
_tiffisWriteProc(thandle_t, tdata_t, tsize_t)
{
	return 0;
}

static toff_t
_tiffosSeekProc(thandle_t fd, toff_t off, int whence)
{
	tiffios_data	*data = (tiffios_data *)fd;

	switch(whence) {
	case SEEK_SET:
		data->myOS->seekp(data->myStreamStartPos + off, ios::beg);
		break;
	case SEEK_CUR:
		data->myOS->seekp(off, ios::cur);
		break;
	case SEEK_END:
		data->myOS->seekp(off, ios::end);
		break;
	}

	return ((long)data->myOS->tellp()) - data->myStreamStartPos;
}

static toff_t
_tiffisSeekProc(thandle_t fd, toff_t off, int whence)
{
	tiffios_data	*data = (tiffios_data *)fd;

	switch(whence) {
	case SEEK_SET:
		data->myIS->seekg(data->myStreamStartPos + off, ios::beg);
		break;
	case SEEK_CUR:
		data->myIS->seekg(off, ios::cur);
		break;
	case SEEK_END:
		data->myIS->seekg(off, ios::end);
		break;
	}

	return ((long)data->myIS->tellg()) - data->myStreamStartPos;
}

static toff_t
_tiffosSizeProc(thandle_t fd)
{
	tiffios_data	*data = (tiffios_data *)fd;
	int		pos = data->myOS->tellp();
	int		len;

	data->myOS->seekp(0, ios::end);
	len = data->myOS->tellp();
	data->myOS->seekp(pos);

	return len;
}

static toff_t
_tiffisSizeProc(thandle_t fd)
{
	tiffios_data	*data = (tiffios_data *)fd;
	int		pos = data->myIS->tellg();
	int		len;

	data->myIS->seekg(0, ios::end);
	len = data->myIS->tellg();
	data->myIS->seekg(pos);

	return len;
}

static int
_tiffiosCloseProc(thandle_t fd)
{
	// Our stream was not allocated by us, so it shouldn't be closed by us.
	delete (tiffios_data *)fd;
	return 0;
}

static int
_tiffDummyMapProc(thandle_t , tdata_t* , toff_t* )
{
	return (0);
}

static void
_tiffDummyUnmapProc(thandle_t , tdata_t , toff_t )
{
}

/*
 * Open a TIFF file descriptor for read/writing.
 */
static TIFF*
_tiffStreamOpen(const char* name, const char* mode, void *fd)
{
	TIFF*	tif;

	tiffios_data	*data = new tiffios_data;

	if( strchr(mode, 'w') ) {
		data->myOS = (ostream *)fd;
		data->myStreamStartPos = data->myOS->tellp();
	    // Open for writing.
	    tif = TIFFClientOpen(name, mode,
		(thandle_t) data,
		_tiffosReadProc, _tiffosWriteProc,
		_tiffosSeekProc, _tiffiosCloseProc, _tiffosSizeProc,
		_tiffDummyMapProc, _tiffDummyUnmapProc);
	} else {
		data->myIS = (istream *)fd;
		data->myStreamStartPos = data->myIS->tellg();
	    // Open for reading.
	    tif = TIFFClientOpen(name, mode,
		(thandle_t) data,
		_tiffisReadProc, _tiffisWriteProc,
		_tiffisSeekProc, _tiffiosCloseProc, _tiffisSizeProc,
		_tiffDummyMapProc, _tiffDummyUnmapProc);
	}

	return (tif);
}

TIFF*
TIFFStreamOpen(const char* name, ostream *os)
{
	// NB: We don't support mapped files with streams
	return _tiffStreamOpen(name, "wm", os);
}

TIFF*
TIFFStreamOpen(const char* name, istream *is)
{
	// NB: We don't support mapped files with streams
	return _tiffStreamOpen(name, "rm", is);
}

/* vim: set ts=8 sts=8 sw=8 noet: */

--------------090705060300080704080006
Content-Type: text/plain;
 name="tif_stream.cxx"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="tif_stream.cxx"

/* $Id: tif_stream.cxx,v 1.2 2004/11/13 11:19:15 dron Exp $ */

/*
 * Copyright (c) 1988-1996 Sam Leffler
 * Copyright (c) 1991-1996 Silicon Graphics, Inc.
 *
 * Permission to use, copy, modify, distribute, and sell this software and 
 * its documentation for any purpose is hereby granted without fee, provided
 * that (i) the above copyright notices and this permission notice appear in
 * all copies of the software and related documentation, and (ii) the names of
 * Sam Leffler and Silicon Graphics may not be used in any advertising or
 * publicity relating to the software without the specific, prior written
 * permission of Sam Leffler and Silicon Graphics.
 * 
 * THE SOFTWARE IS PROVIDED "AS-IS" AND WITHOUT WARRANTY OF ANY KIND, 
 * EXPRESS, IMPLIED OR OTHERWISE, INCLUDING WITHOUT LIMITATION, ANY 
 * WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.  
 * 
 * IN NO EVENT SHALL SAM LEFFLER OR SILICON GRAPHICS BE LIABLE FOR
 * ANY SPECIAL, INCIDENTAL, INDIRECT OR CONSEQUENTIAL DAMAGES OF ANY KIND,
 * OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS,
 * WHETHER OR NOT ADVISED OF THE POSSIBILITY OF DAMAGE, AND ON ANY THEORY OF 
 * LIABILITY, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE 
 * OF THIS SOFTWARE.
 */

/*
 * TIFF Library UNIX-specific Routines.
 */
#include <iostream>
#include "tiffiop.h"

using namespace std;

class tiffis_data
{
  public:

	istream	*myIS;
        long	myStreamStartPos;
};

class tiffos_data
{
  public:

	ostream	*myOS;
	long	myStreamStartPos;
};

static tsize_t
_tiffosReadProc(thandle_t, tdata_t, tsize_t)
{
        return 0;
}

static tsize_t
_tiffisReadProc(thandle_t fd, tdata_t buf, tsize_t size)
{
        tiffis_data	*data = (tiffis_data *)fd;

        data->myIS->read((char *)buf, (int)size);

        return data->myIS->gcount();
}

static tsize_t
_tiffosWriteProc(thandle_t fd, tdata_t buf, tsize_t size)
{
	tiffos_data	*data = (tiffos_data *)fd;
	ostream		*os = data->myOS;
	int		pos = os->tellp();

	os->write((const char *)buf, size);

	return ((int)os->tellp()) - pos;
}

static tsize_t
_tiffisWriteProc(thandle_t, tdata_t, tsize_t)
{
	return 0;
}

static toff_t
_tiffosSeekProc(thandle_t fd, toff_t off, int whence)
{
	tiffos_data	*data = (tiffos_data *)fd;
	ostream	*os = data->myOS;

	// if the stream has already failed, don't do anything
	if( os->fail() )
		return os->tellp();

	switch(whence) {
	case SEEK_SET:
	    os->seekp(data->myStreamStartPos + off, ios::beg);
		break;
	case SEEK_CUR:
		os->seekp(off, ios::cur);
		break;
	case SEEK_END:
		os->seekp(off, ios::end);
		break;
	}

	// Attempt to workaround problems with seeking past the end of the
	// stream.  ofstream doesn't have a problem with this but
	// ostrstream/ostringstream does. In that situation, add intermediate
	// '\0' characters.
	if( os->fail() ) {
		ios_base::iostate   old_state;
		toff_t		    origin;

		old_state = os->rdstate();
		// reset the fail bit or else tellp() won't work below
		os->clear(os->rdstate() & ~ios::failbit);
		switch( whence ) {
			case SEEK_SET:
				origin = data->myStreamStartPos;
				break;
			case SEEK_CUR:
				origin = os->tellp();
				break;
			case SEEK_END:
				os->seekp(0, ios::end);
				origin = os->tellp();
				break;
		}
		// restore original stream state
		os->clear(old_state);	

		// only do something if desired seek position is valid
		if( origin + off > data->myStreamStartPos ) {
			toff_t	num_fill;

			// clear the fail bit 
			os->clear(os->rdstate() & ~ios::failbit);

			// extend the stream to the expected size
			os->seekp(0, ios::end);
			num_fill = origin + off - (toff_t)os->tellp();
			for( toff_t i = 0; i < num_fill; i++ )
				os->put('\0');

			// retry the seek
			os->seekp(origin + off, ios::beg);
		}
	}

	return os->tellp();
}

static toff_t
_tiffisSeekProc(thandle_t fd, toff_t off, int whence)
{
	tiffis_data	*data = (tiffis_data *)fd;

	switch(whence) {
	case SEEK_SET:
		data->myIS->seekg(data->myStreamStartPos + off, ios::beg);
		break;
	case SEEK_CUR:
		data->myIS->seekg(off, ios::cur);
		break;
	case SEEK_END:
		data->myIS->seekg(off, ios::end);
		break;
	}

	return ((long)data->myIS->tellg()) - data->myStreamStartPos;
}

static toff_t
_tiffosSizeProc(thandle_t fd)
{
	tiffos_data	*data = (tiffos_data *)fd;
	ostream		*os = data->myOS;
	toff_t		pos = os->tellp();
	toff_t		len;

	os->seekp(0, ios::end);
	len = os->tellp();
	os->seekp(pos);

	return len;
}

static toff_t
_tiffisSizeProc(thandle_t fd)
{
	tiffis_data	*data = (tiffis_data *)fd;
	int		pos = data->myIS->tellg();
	int		len;

	data->myIS->seekg(0, ios::end);
	len = data->myIS->tellg();
	data->myIS->seekg(pos);

	return len;
}

static int
_tiffosCloseProc(thandle_t fd)
{
	// Our stream was not allocated by us, so it shouldn't be closed by us.
	delete (tiffos_data *)fd;
	return 0;
}

static int
_tiffisCloseProc(thandle_t fd)
{
	// Our stream was not allocated by us, so it shouldn't be closed by us.
	delete (tiffis_data *)fd;
	return 0;
}

static int
_tiffDummyMapProc(thandle_t , tdata_t* , toff_t* )
{
	return (0);
}

static void
_tiffDummyUnmapProc(thandle_t , tdata_t , toff_t )
{
}

/*
 * Open a TIFF file descriptor for read/writing.
 */
static TIFF*
_tiffStreamOpen(const char* name, const char* mode, void *fd)
{
	TIFF*	tif;

	if( strchr(mode, 'w') ) {
		tiffos_data	*data = new tiffos_data;
		data->myOS = (ostream *)fd;
		data->myStreamStartPos = data->myOS->tellp();

		// Open for writing.
		tif = TIFFClientOpen(name, mode,
				(thandle_t) data,
				_tiffosReadProc, _tiffosWriteProc,
				_tiffosSeekProc, _tiffosCloseProc,
				_tiffosSizeProc,
				_tiffDummyMapProc, _tiffDummyUnmapProc);
	} else {
		tiffis_data	*data = new tiffis_data;
		data->myIS = (istream *)fd;
		data->myStreamStartPos = data->myIS->tellg();
		// Open for reading.
		tif = TIFFClientOpen(name, mode,
				(thandle_t) data,
				_tiffisReadProc, _tiffisWriteProc,
				_tiffisSeekProc, _tiffisCloseProc,
				_tiffisSizeProc,
				_tiffDummyMapProc, _tiffDummyUnmapProc);
	}

	return (tif);
}

TIFF*
TIFFStreamOpen(const char* name, ostream *os)
{
	// If os is either a ostrstream or ostringstream, and has no data
	// written to it yet, then tellp() will return -1 which will break us.
	// We workaround this by writing out a dummy character and
	// then seek back to the beginning.
	if( !os->fail() && (int)os->tellp() < 0 ) {
		*os << '\0';
		os->seekp(0);
	}

	// NB: We don't support mapped files with streams so add 'm'
	return _tiffStreamOpen(name, "wm", os);
}

TIFF*
TIFFStreamOpen(const char* name, istream *is)
{
	// NB: We don't support mapped files with streams so add 'm'
	return _tiffStreamOpen(name, "rm", is);
}

/* vim: set ts=8 sts=8 sw=8 noet: */

--------------090705060300080704080006
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Tiff mailing list
Tiff@remotesensing.org
http://xserve.flids.com/mailman/listinfo/tiff

--------------090705060300080704080006--
