Return-Path: <tiff-bounces@lists.maptools.org>
Delivered-To: info@awaresystems.be
Received: (qmail 7213 invoked from network); 22 Aug 2008 15:48:23 -0000
Received: from kudos.telenet-ops.be ([195.130.136.13])
          (envelope-sender <tiff-bounces@lists.maptools.org>)
          by angelia.telenet-ops.be (qmail-ldap-1.03) with SMTP
          for <info@awaresystems.be>; 22 Aug 2008 15:48:23 -0000
Received: from zimbu.telenet-ops.be (zimbu.telenet-ops.be [195.130.136.11])
	by kudos.telenet-ops.be (8.13.1/8.13.1) with ESMTP id m7MFmNTK010019
	for <info@awaresystems.be>; Fri, 22 Aug 2008 17:48:23 +0200
Received: from localhost (localhost.localdomain [127.0.0.1])
	by zimbu.telenet-ops.be (Postfix) with SMTP id 58D7A40003
	for <info@awaresystems.be>; Fri, 22 Aug 2008 17:48:23 +0200 (CEST)
Received: from crotus.telenet-ops.be (crotus.telenet-ops.be [195.130.132.38])
	by zimbu.telenet-ops.be (Postfix) with ESMTP id 5310040002
	for <info@awaresystems.be>; Fri, 22 Aug 2008 17:48:23 +0200 (CEST)
Received: from duke.maptools.org (r153.dmsolutions.ca [209.217.116.153])
	by crotus.telenet-ops.be (Postfix) with ESMTP id D72C7D
	for <info@awaresystems.be>; Fri, 22 Aug 2008 17:48:22 +0200 (CEST)
Received: from duke.maptools.org (duke.maptools.org [127.0.0.1])
	by duke.maptools.org (8.12.11/8.12.10) with ESMTP id m7MFjiv7023995;
	Fri, 22 Aug 2008 11:47:23 -0400
Received: from abra2.bitwizard.nl (dtp.xs4all.nl [80.101.171.8])
	by duke.maptools.org (8.12.11/8.12.10) with SMTP id m7MFjdcp023990
	for <tiff@lists.maptools.org>; Fri, 22 Aug 2008 11:45:40 -0400
Received: (qmail 15186 invoked by uid 1000); 22 Aug 2008 17:45:39 +0200
Date: Fri, 22 Aug 2008 17:45:39 +0200
From: Rogier Wolff <R.E.Wolff@bitwizard.nl>
To: Toby Thain <toby@telegraphics.com.au>
Subject: Re: [Tiff] creating sparse files......
Message-ID: <20080822154538.GA29030@bitwizard.nl>
References: <20080822085634.GA21294@bitwizard.nl>
	<AFE1B10B-3F34-41A6-85A1-8ED1670F9623@telegraphics.com.au>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <AFE1B10B-3F34-41A6-85A1-8ED1670F9623@telegraphics.com.au>
Organization: BitWizard.nl
User-Agent: Mutt/1.5.13 (2006-08-11)
Cc: tiff@lists.maptools.org, Rogier Wolff <R.E.Wolff@bitwizard.nl>
X-BeenThere: tiff@lists.maptools.org
X-Mailman-Version: 2.1.4
Precedence: list
List-Id: TIFF and libtiff discussion <tiff.lists.maptools.org>
List-Unsubscribe: <http://lists.maptools.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@lists.maptools.org?subject=unsubscribe>
List-Archive: <http://lists.maptools.org/mailman/private/tiff>
List-Post: <mailto:tiff@lists.maptools.org>
List-Help: <mailto:tiff-request@lists.maptools.org?subject=help>
List-Subscribe: <http://lists.maptools.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@lists.maptools.org?subject=subscribe>
Sender: tiff-bounces@lists.maptools.org
Errors-To: tiff-bounces@lists.maptools.org
X-SpamCatcher-Score: 50 [XX]

On Fri, Aug 22, 2008 at 10:11:42AM -0300, Toby Thain wrote:
> >static int isallzero (tdata_t buf, tsize_t size)
> >{
> >        int i;
> >        for (i=0;i<size;i++)
> >                if (buf[i]) return 0;
> >        return 1;
> >}
> 
> 
> This appears unnecessarily inefficient. It's going to be much cheaper  
> to test whole longwords (or whatever data alignment will allow) than  
> byte by byte. (I wonder if something clever can be done with MMX/SSE  
> on newer chips?)

Yes, this is the "quick and dirty" implementation of this function.
The advantage of doing it this simple is that it will ALWAYS work.
No stupid bugs that if the only nonzero byte is in the partial long
just beyond the 

Keep in mind that when this runs, the CPU will be getting whole cache
lines from memory. When the cache has been filled, the CPU runs at say
3GHz. Or three cycles per ns. It will likely run the loop three times
a nanosecond as well. The memory fetching that needs to go on, is
likely to cost on the order of tens of nanoseconds per cache line. DDR
still runs at 200 MHz, and you have a two cycle latency for reads,
right? That's around 10 ns. At that point you can START to recieve
data from memory. The whole cache line then takes a few more
cycles....

Anway, my philosophy is always FIRST get it to work, optimize later.

> >static tsize_t
> >_tiffWriteProc(thandle_t fd, tdata_t buf, tsize_t size)
> >{
> >        if (isallzero (buf, size))
> >                return (lseek ((int) fd, (off_t)size, SEEK_CUR));
> >        else
> >                return ((tsize_t) write((int) fd, buf, (size_t)  
> >size));
> >}
> 
> 
> This penalises all writes to optimise a very special case. Not saying  
> it can't be rationalised, but pros and cons can be debated.

And you're calling this an optimization. And I agree. However, in my
case saving a factor of 21 on disk space is i'd say "worth it".

I work in datarecovery. I modified our "read-the-data-from-the-disk"
program to check for zeroes. Of course it got the commandline flag
that allowed me to turn it off for disks that didn't have many zeroes,
so that things would go faster. Bad move. Or at least unneccesary in
hindsight. Performance is NOT influenced by the check for zeroes, and
the gain by not writing the zeroes is always noticable.

But, yes, officially it could cost you some CPU cycles, which are not
returned by saving on writes. (checking for zeroes and seeking is much
more efficient than just doing the write! It not only saves you disk
space, but also CPU cycles!)

	Roger. 

-- 
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2600998 **
**    Delftechpark 26 2628 XH  Delft, The Netherlands. KVK: 27239233    **
*-- BitWizard writes Linux device drivers for any device you may have! --*
Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement. 
Does it sit on the couch all day? Is it unemployed? Please be specific! 
Define 'it' and what it isn't doing. --------- Adapted from lxrbot FAQ
_______________________________________________
Tiff mailing list: Tiff@lists.maptools.org
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
