Return-Path: <tiff-bounces@lists.maptools.org>
Delivered-To: info@awaresystems.be
Received: (qmail 10659 invoked from network); 22 Aug 2008 13:16:01 -0000
Received: from judy.telenet-ops.be ([195.130.136.7])
          (envelope-sender <tiff-bounces@lists.maptools.org>)
          by angelia.telenet-ops.be (qmail-ldap-1.03) with SMTP
          for <info@awaresystems.be>; 22 Aug 2008 13:16:01 -0000
Received: from mindy.telenet-ops.be (mindy.telenet-ops.be [195.130.136.8])
	by judy.telenet-ops.be (8.13.1/8.13.1) with ESMTP id m7MDG1Ch011572
	for <info@awaresystems.be>; Fri, 22 Aug 2008 15:16:01 +0200
Received: from localhost (localhost.localdomain [127.0.0.1])
	by mindy.telenet-ops.be (Postfix) with SMTP id 00BF960005
	for <info@awaresystems.be>; Fri, 22 Aug 2008 15:16:01 +0200 (CEST)
Received: from crotus.telenet-ops.be (crotus.telenet-ops.be [195.130.132.38])
	by mindy.telenet-ops.be (Postfix) with ESMTP id EE35560004
	for <info@awaresystems.be>; Fri, 22 Aug 2008 15:16:00 +0200 (CEST)
Received: from duke.maptools.org (r153.dmsolutions.ca [209.217.116.153])
	by crotus.telenet-ops.be (Postfix) with ESMTP id 770D710
	for <info@awaresystems.be>; Fri, 22 Aug 2008 15:16:00 +0200 (CEST)
Received: from duke.maptools.org (duke.maptools.org [127.0.0.1])
	by duke.maptools.org (8.12.11/8.12.10) with ESMTP id m7MDBwd8019987;
	Fri, 22 Aug 2008 09:13:54 -0400
Received: from toronto2.smartgames.ca (toronto2.smartgames.ca [204.15.192.18])
	by duke.maptools.org (8.12.11/8.12.10) with ESMTP id m7MDBs5H019984
	for <tiff@lists.maptools.org>; Fri, 22 Aug 2008 09:11:55 -0400
Received: from [10.143.176.10] (unknown [10.143.176.10])
	by toronto2.smartgames.ca (Postfix) with ESMTP id 34FCEAA3;
	Fri, 22 Aug 2008 09:15:35 -0400 (EDT)
In-Reply-To: <20080822085634.GA21294@bitwizard.nl>
References: <20080822085634.GA21294@bitwizard.nl>
Mime-Version: 1.0 (Apple Message framework v753.1)
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <AFE1B10B-3F34-41A6-85A1-8ED1670F9623@telegraphics.com.au>
Content-Transfer-Encoding: 7bit
From: Toby Thain <toby@telegraphics.com.au>
Subject: Re: [Tiff] creating sparse files......
Date: Fri, 22 Aug 2008 10:11:42 -0300
To: Rogier Wolff <R.E.Wolff@bitwizard.nl>
X-Mailer: Apple Mail (2.753.1)
Cc: tiff@lists.maptools.org
X-BeenThere: tiff@lists.maptools.org
X-Mailman-Version: 2.1.4
Precedence: list
List-Id: TIFF and libtiff discussion <tiff.lists.maptools.org>
List-Unsubscribe: <http://lists.maptools.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@lists.maptools.org?subject=unsubscribe>
List-Archive: <http://lists.maptools.org/mailman/private/tiff>
List-Post: <mailto:tiff@lists.maptools.org>
List-Help: <mailto:tiff-request@lists.maptools.org?subject=help>
List-Subscribe: <http://lists.maptools.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@lists.maptools.org?subject=subscribe>
Sender: tiff-bounces@lists.maptools.org
Errors-To: tiff-bounces@lists.maptools.org
X-SpamCatcher-Score: 50 [XX]


On 22-Aug-08, at 5:56 AM, Rogier Wolff wrote:

>
> Hi,
>
> I'm stitching kind of large panoramas. This results in big
> intermediate files. On my last run, which took overnight to stitch,
> I thought 42 Gb of free disk space would be enough. Wrong!
>
> I got over thrity files of over 1.2Gbytes, filling up the disk.
>
> It turns out that most of the files contain lots of zeroes. On Unix
> this can be stored effciently by not issueing a "write" with a buffer
> full of zeroes, but by seeking over the area. The operating system
> will act as if the area was filled with zeroes.
>
> I'v modified "tif_unix.c" to read:
>
>
> static int isallzero (tdata_t buf, tsize_t size)
> {
>         int i;
>         for (i=0;i<size;i++)
>                 if (buf[i]) return 0;
>         return 1;
> }


This appears unnecessarily inefficient. It's going to be much cheaper  
to test whole longwords (or whatever data alignment will allow) than  
byte by byte. (I wonder if something clever can be done with MMX/SSE  
on newer chips?)

>
> static tsize_t
> _tiffWriteProc(thandle_t fd, tdata_t buf, tsize_t size)
> {
>         if (isallzero (buf, size))
>                 return (lseek ((int) fd, (off_t)size, SEEK_CUR));
>         else
>                 return ((tsize_t) write((int) fd, buf, (size_t)  
> size));
> }


This penalises all writes to optimise a very special case. Not saying  
it can't be rationalised, but pros and cons can be debated.

just my 2c
--Toby

>
> static toff_t
> _tiffSeekProc(thandle_t fd, toff_t off, int whence)
> {
>         return ((toff_t) lseek((int) fd, (off_t) off, whence));
> }
>
> static int
> _tiffCloseProc(thandle_t fd)
> {
>         toff_t curoffset;
>         toff_t fsize;
>
>         curoffset = _tiffSeekProc (fd, 0, SEEK_CUR);
>         fsize     = _tiffSeekProc (fd, 0, SEEK_END);
>         if (curoffset > fsize) {
>                 ftruncate((int)fd, curoffset);
>         }
>
>         return (close((int) fd));
> }
>
>
> I haven't tested it yet. Maybe someone here has the same problem I
> have , and is willing to do a test-run. I'll make a proper patch later
> on. This is just to allow you guys know and to invite comments....
>
> While writing this Email message, I've found a bug: If an area is
> written, and then later zeroed by the application program the new code
> won't work. I'll have to think about how to get this case right. In
> practise a modified libtiff should be easy to plug into "hugin" and
> the results should be visible, the bug-situation will probably not
> trigger.
>
> (I used to be able to hit the "compress" checkbox inside hugin, and it
> would work. But it seems to have gone "dead" in the version currently
> installed on my machine.)
>
> 	Roger.
>
>
> -- 
> ** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ **  
> +31-15-2600998 **
> **    Delftechpark 26 2628 XH  Delft, The Netherlands. KVK:  
> 27239233    **
> *-- BitWizard writes Linux device drivers for any device you may  
> have! --*
> Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous  
> statement.
> Does it sit on the couch all day? Is it unemployed? Please be  
> specific!
> Define 'it' and what it isn't doing. --------- Adapted from lxrbot FAQ
> _______________________________________________
> Tiff mailing list: Tiff@lists.maptools.org
> http://lists.maptools.org/mailman/listinfo/tiff
> http://www.remotesensing.org/libtiff/

_______________________________________________
Tiff mailing list: Tiff@lists.maptools.org
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
