Return-Path: <tiff-bounces@remotesensing.org>
Received: from inmx007.isp.belgacom.be (inmx007.isp.belgacom.be [195.238.3.89])
        by mailst034.isp.belgacom.be (8.12.11/8.12.11/Skynet-MAILSTORE-2.14) with ESMTP id i92Fg74U011484
        for <joris.at.lebbeke@skynet.be>; Sat, 2 Oct 2004 17:42:07 +0200
        (envelope-from <tiff-bounces@remotesensing.org>)
Received: from xserve.flids.com (adsl-065-083-169-202.sip.mco.bellsouth.net [65.83.169.202])
        by inmx007.isp.belgacom.be (8.12.11/8.12.11/Skynet-IN-2.32) with ESMTP id i92Fg0Pf013946
        for <joris.at.lebbeke@skynet.be>; Sat, 2 Oct 2004 17:42:02 +0200
        (envelope-from <tiff-bounces@remotesensing.org>)
Received: from adsl-065-083-169-202.sip.mco.bellsouth.net (localhost [127.0.0.1])
	by xserve.flids.com (Postfix) with ESMTP
	id 6D4383488C7; Sat,  2 Oct 2004 11:41:45 -0400 (EDT)
X-Original-To: tiff@remotesensing.org
Delivered-To: tiff@remotesensing.org
Received: from scooby.simplesystems.org (scooby.simplesystems.org
	[65.66.246.67])
	by xserve.flids.com (Postfix) with ESMTP id 074913488A3
	for <tiff@remotesensing.org>; Sat,  2 Oct 2004 11:41:43 -0400 (EDT)
Received: from blade.simplesystems.org (blade.simplesystems.org [65.66.246.74])
	by scooby.simplesystems.org (8.12.10+Sun/8.12.10) with ESMTP id
	i92Ffd9p000570; Sat, 2 Oct 2004 10:41:39 -0500 (CDT)
Date: Sat, 2 Oct 2004 10:41:39 -0500 (CDT)
From: Bob Friesenhahn <bfriesen@simple.dallas.tx.us>
X-X-Sender: bfriesen@blade.simplesystems.org
To: Joris <joris.at.lebbeke@skynet.be>
Subject: Re: [Tiff] quad-tile
In-Reply-To: <001501c4a891$4f7e5460$c008f151@dfgrrxsknr4e>
Message-ID: <Pine.SOC.4.60.0410021028050.2727@blade.simplesystems.org>
References: <415D625F.5090107@pobox.com><001201c4a7c1$6e6165a0$e8c8c950@dfgrrxsknr4e>
	<20041002141605.GC12254@ak4719.spb.edu>
	<001501c4a891$4f7e5460$c008f151@dfgrrxsknr4e>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
Cc: tiff@remotesensing.org
X-BeenThere: tiff@remotesensing.org
X-Mailman-Version: 2.1.2
Precedence: list
List-Id: TIFF and libtiff discussion  <tiff.remotesensing.org>
List-Unsubscribe: <http://xserve.flids.com/mailman/listinfo/tiff>,
	<mailto:tiff-request@remotesensing.org?subject=unsubscribe>
List-Archive: <http://xserve.flids.com/pipermail/tiff>
List-Post: <mailto:tiff@remotesensing.org>
List-Help: <mailto:tiff-request@remotesensing.org?subject=help>
List-Subscribe: <http://xserve.flids.com/mailman/listinfo/tiff>,
	<mailto:tiff-request@remotesensing.org?subject=subscribe>
Sender: tiff-bounces@remotesensing.org
Errors-To: tiff-bounces@remotesensing.org
X-UIDL: b8e3ba58a781854f6ce324044d644e14

On Sat, 2 Oct 2004, Joris wrote:
>
> Two minor complications I foresee:
> - Closed-circuit testing is a tiny bit problematic when lossy storage is
> involved. I'm not just thinking jpeg, even mere YCbCr subsampled images,
> whatever compression scheme, are lossy. This tiny problem can perhaps be solved
> by having the data include tolerance levels for the closed-circuit testing?

For many of GraphicsMagick's tests we use an image comparison function 
which computes statistical error values.  For algorithms which should 
not produce error, the allowed error is zero, but for algorithms which 
produce error (e.g. JPEG) we manually inspect the result to ensure 
that it seems ok, and then set the maximum allowed error value to 
slighly higher than the actual result.  The error values include the 
maximum pixel error (in case a single pixel is wrong), and the total 
mean error.

Provided that the libtiff test suite provides an image compare 
function, the allowed degree of error may be included with the test 
specification.

Note that this does not verify that the file format was correct, only 
that when the file is read back, the results are equivalent.

If there is no master reference image, then testing based on observed 
error is limited since only the ability to reproduce 
locally-prepared results can be evaluated, and the locally-prepared 
reference results may be wrong.

For operations which must always produce a certain result (the case 
for most libtiff operations), we use a checksum (SHA-128) and compare 
the resulting checksum with a checksum included with the test.  This 
is a very solid way to prepare a test, but is a bit tedious since each 
result must be validated by the person who writes the test.

Bob
======================================
Bob Friesenhahn
bfriesen@simple.dallas.tx.us
http://www.simplesystems.org/users/bfriesen
_______________________________________________
Tiff mailing list
Tiff@remotesensing.org
http://xserve.flids.com/mailman/listinfo/tiff
