Return-Path: <tiff-bounces@remotesensing.org>
Received: from inmx005.isp.belgacom.be (inmx005.isp.belgacom.be [195.238.3.148])
        by mailst034.isp.belgacom.be (8.12.11/8.12.11/Skynet-MAILSTORE-2.14) with ESMTP id i8H2TjES012511
        for <joris.at.lebbeke@skynet.be>; Fri, 17 Sep 2004 04:29:45 +0200
        (envelope-from <tiff-bounces@remotesensing.org>)
Received: from xserve.flids.com (adsl-065-083-169-202.sip.mco.bellsouth.net [65.83.169.202])
        by inmx005.isp.belgacom.be (8.12.11/8.12.11/Skynet-IN-2.32) with ESMTP id i8H2TZjn011355
        for <joris.at.lebbeke@skynet.be>; Fri, 17 Sep 2004 04:29:38 +0200
        (envelope-from <tiff-bounces@remotesensing.org>)
Received: from adsl-065-083-169-202.sip.mco.bellsouth.net (localhost [127.0.0.1])
	by xserve.flids.com (Postfix) with ESMTP
	id 98FDA2F24A4; Thu, 16 Sep 2004 22:29:08 -0400 (EDT)
X-Original-To: tiff@remotesensing.org
Delivered-To: tiff@remotesensing.org
Received: from scooby.simplesystems.org (scooby.simplesystems.org
	[65.66.246.67])
	by xserve.flids.com (Postfix) with ESMTP id 649DC2F247E
	for <tiff@remotesensing.org>; Thu, 16 Sep 2004 22:29:05 -0400 (EDT)
Received: from blade.simplesystems.org (blade.simplesystems.org [65.66.246.74])
	by scooby.simplesystems.org (8.12.10+Sun/8.12.10) with ESMTP id
	i8H2T49p027854; Thu, 16 Sep 2004 21:29:04 -0500 (CDT)
Date: Thu, 16 Sep 2004 21:29:04 -0500 (CDT)
From: Bob Friesenhahn <bfriesen@simple.dallas.tx.us>
X-X-Sender: bfriesen@blade.simplesystems.org
To: Joris <joris.at.lebbeke@skynet.be>
Subject: Re: [Tiff] BigTIFF Tag Value Count issue
In-Reply-To: <Pine.SOC.4.60.0409162021080.15634@blade.simplesystems.org>
Message-ID: <Pine.SOC.4.60.0409162110060.15634@blade.simplesystems.org>
References: <001c01c49c21$2bf33730$c325c850@dfgrrxsknr4e>
	<Pine.SOC.4.60.0409161905220.15634@blade.simplesystems.org>
	<001401c49c4d$bff536f0$379688d9@dfgrrxsknr4e>
	<Pine.SOC.4.60.0409162021080.15634@blade.simplesystems.org>
MIME-Version: 1.0
Content-Type: MULTIPART/MIXED;
	BOUNDARY="-559023410-1202595967-1095388144=:15634"
Cc: Tiff mailing list <tiff@remotesensing.org>
X-BeenThere: tiff@remotesensing.org
X-Mailman-Version: 2.1.2
Precedence: list
List-Id: TIFF and libtiff discussion  <tiff.remotesensing.org>
List-Unsubscribe: <http://xserve.flids.com/mailman/listinfo/tiff>,
	<mailto:tiff-request@remotesensing.org?subject=unsubscribe>
List-Archive: <http://xserve.flids.com/pipermail/tiff>
List-Post: <mailto:tiff@remotesensing.org>
List-Help: <mailto:tiff-request@remotesensing.org?subject=help>
List-Subscribe: <http://xserve.flids.com/mailman/listinfo/tiff>,
	<mailto:tiff-request@remotesensing.org?subject=subscribe>
Sender: tiff-bounces@remotesensing.org
Errors-To: tiff-bounces@remotesensing.org
X-UIDL: dde56de5bfb76e3e25f8b4abb0767d34

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

---559023410-1202595967-1095388144=:15634
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed

I have attached source for a small test program I used to see how 
systems behave under load when using memory mapped files.  Naturally, 
documentation is poor and the code is meant for hacking. :-)

The program creates a 6GB file on disk (adjustable at line 39) and 
then proceeds to update the content of the file using a stride (size 
of the chunk of memory mapped and updated at one time).  The msync 
call is used to tell the OS which modified pages are no longer needed 
(and should be be saved to disk).

The results of test runs were very interesting.  Solaris did 
exceedingly well with any size stride and was done with the file very 
quickly without excess memory usage or thrashing.  FreeBSD 5.0 did 
very well if the stride was over a certain size (8 or 16K I think), 
but under that magic size, it suddenly took 20X longer.  Linux 2.6 did 
not seem to observe the request to purge dirty memory so it thrashed 
away.

There is a chunk of code which is #ifdeffed out which sees how well 
the system works if the file uses one huge mapping.  The test filesize 
may need to be reduced in order to use this.

Bob
======================================
Bob Friesenhahn
bfriesen@simple.dallas.tx.us
http://www.simplesystems.org/users/bfriesen
---559023410-1202595967-1095388144=:15634
Content-Type: TEXT/plain; charset=US-ASCII; name="mmaptest.c"
Content-Transfer-Encoding: BASE64
Content-ID: <Pine.SOC.4.60.0409162129040.15634@blade.simplesystems.org>
Content-Description: 
Content-Disposition: attachment; filename="mmaptest.c"

I2luY2x1ZGUgPHN5cy90eXBlcy5oPg0KI2luY2x1ZGUgPHN5cy9zdGF0Lmg+
DQojaW5jbHVkZSA8ZmNudGwuaD4NCiNpbmNsdWRlIDxzeXMvbW1hbi5oPg0K
I2luY2x1ZGUgPHVuaXN0ZC5oPg0KI2luY2x1ZGUgPGxpbWl0cy5oPg0KI2lu
Y2x1ZGUgPHN0cmluZy5oPg0KI2luY2x1ZGUgPHN0ZGlvLmg+DQojaW5jbHVk
ZSA8c3RkbGliLmg+DQojaW5jbHVkZSA8ZXJybm8uaD4NCg0KaW50IG1haW4g
KGludCBhcmdjLCBjaGFyICphcmd2W10pDQp7DQogIGNoYXIgdGVzdGZpbGVb
XSA9ICJtbWFwLnRtcCI7DQogIGludCBmZDsNCiAgb2ZmX3QgZmlsZWxlbjsN
CiAgb2ZmX3QgaW5kZXg7DQogIHNpemVfdCBzdHJpZGU7DQogIHVuc2lnbmVk
IGNoYXIgKm1hcHBlZDsNCiAgc2l6ZV90IHBhZ2VzaXplOw0KDQogIGlmIChh
cmdjICE9IDIpDQogICAgew0KICAgICAgcHJpbnRmKCJVc2FnZTogbW1hcHRl
c3Qgc3RyaWRlXG4iKTsNCiAgICAgIGV4aXQoMSk7DQogICAgfQ0KDQogIHBh
Z2VzaXplID0gKHNpemVfdCkgc3lzY29uZihfU0NfUEFHRVNJWkUpOw0KDQog
IHN0cmlkZT1wYWdlc2l6ZSphdG9pKGFyZ3ZbMV0pOw0KDQogIGZkID0gb3Bl
bih0ZXN0ZmlsZSxPX1JEV1J8T19DUkVBVHxPX1RSVU5DLFNfSVJVU1J8U19J
V1VTUik7DQogIGlmIChmZCA9PSAtMSkNCiAgICB7DQogICAgICBwZXJyb3Io
IkZhaWxlZCB0byBvcGVuIGZpbGUiKTsNCiAgICAgIGV4aXQoMSk7DQogICAg
fQ0KDQogIGZpbGVsZW49MjE0NzQ4MzY0N1VMTCozOw0KICBwcmludGYoIkZp
bGUgbGVuZ3RoID0gJWxsZFxuIiwgKGxvbmcgbG9uZylmaWxlbGVuKTsNCiAg
Ly9maWxlbGVuPTgwOTY7DQogIGluZGV4ID0gMDsNCiAgaWYgKChmdHJ1bmNh
dGUoZmQsZmlsZWxlbikpID09IC0xKQ0KICAgIHsNCiAgICAgIHBlcnJvcigi
ZnRydW5jYXRlIGZhaWxlZCIpOw0KICAgICAgZXhpdCgxKTsNCiAgICB9DQoj
aWYgMQ0KICB7DQogICAgLy8gc3RyaWRlID0gcGFnZXNpemUqMTY7DQogICAg
cHJpbnRmKCJTdHJpZGUgICAgICA9ICVsbGRcbiIsIChsb25nIGxvbmcpc3Ry
aWRlKTsNCiAgICANCiAgICBmb3IgKGluZGV4ID0gMDsgaW5kZXgrc3RyaWRl
IDwgZmlsZWxlbjsgaW5kZXggKz0gc3RyaWRlKQ0KICAgICAgew0KICAgICAg
ICBtYXBwZWQgPSAodW5zaWduZWQgY2hhciAqKSBtbWFwKChjYWRkcl90KSAw
LHN0cmlkZSxQUk9UX1JFQUR8UFJPVF9XUklURSwNCiAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICBNQVBfU0hBUkVELGZkLGluZGV4
KTsNCiAgICAgICAgaWYgKG1hcHBlZCA9PSBNQVBfRkFJTEVEKQ0KICAgICAg
ICAgIHsNCiAgICAgICAgICAgIHBlcnJvcigiRmFpbGVkIHRvIG1hcCBmaWxl
Iik7DQogICAgICAgICAgICBleGl0KDEpOw0KICAgICAgICAgIH0NCg0KICAg
ICAgICAvLyBtYWR2aXNlKG1hcHBlZCwgc3RyaWRlLCBNQURWX1NFUVVFTlRJ
QUwpOw0KDQogICAgICAgIC8vIFVuY29tbWVudCB0aGUgZm9sbG93aW5nIGxp
bmUgZm9yIGEgZGlzcGxheSBvZiBwcm9ncmVzcy4NCiAgICAgICAgLy8gcHJp
bnRmKCJpbmRleCAgICAgICA9ICVsbGRcciIsIChsb25nIGxvbmcpaW5kZXgp
Ow0KICAgICAgICBtZW1zZXQoKHZvaWQgKikgbWFwcGVkLCA2NCwgc3RyaWRl
KTsNCg0KICAgICAgICBpZiAoKG1zeW5jKG1hcHBlZCxzdHJpZGUsTVNfQVNZ
TkMpKSA9PSAtMSkgLy8gTVNfQVNZTkMgb3IgTVNfU1lOQw0KICAgICAgICAg
IHsNCiAgICAgICAgICAgIHBlcnJvcigiXG5tc3luYyBmYWlsZWQiKTsNCiAg
ICAgICAgICAgIGV4aXQoMSk7DQogICAgICAgICAgfQ0KDQogICAgICAgIGlm
KChtdW5tYXAobWFwcGVkLHN0cmlkZSkpID09IC0xKQ0KICAgICAgICAgIHsN
CiAgICAgICAgICAgIHBlcnJvcigiXG5tdW5tYXAgZmFpbGVkIik7DQogICAg
ICAgICAgICBleGl0KDEpOw0KICAgICAgICAgIH0NCiAgICAgICAgbWFwcGVk
PTA7DQogICAgICB9DQogIH0NCiNlbHNlDQogIHsNCiAgICBvZmZfdCBvZmY7
DQogICAgdW5zaWduZWQgY2hhciAqcDsNCiAgICBvZmYgPSAwOw0KICAgIHN0
cmlkZSA9IHBhZ2VzaXplKjE2Ow0KICAgIHByaW50ZigiU3RyaWRlICAgICAg
PSAlZFxuIiwgc3RyaWRlKTsNCiAgICBtYXBwZWQgPSAodW5zaWduZWQgY2hh
ciAqKSBtbWFwKChjYWRkcl90KSAwLGZpbGVsZW4sUFJPVF9SRUFEfFBST1Rf
V1JJVEUsDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBN
QVBfU0hBUkVELGZkLG9mZik7DQogICAgaWYgKG1hcHBlZCA9PSBNQVBfRkFJ
TEVEKQ0KICAgICAgew0KICAgICAgICBwZXJyb3IoIkZhaWxlZCB0byBtYXAg
ZmlsZSIpOw0KICAgICAgICBleGl0KDEpOw0KICAgICAgfQ0KDQogICAgZm9y
IChwID0gbWFwcGVkLCBpbmRleD0wOyBpbmRleCtzdHJpZGUgPCBmaWxlbGVu
OyBpbmRleCArPSBzdHJpZGUsIHAgKz0gc3RyaWRlKQ0KICAgICAgew0KICAg
ICAgICBtZW1zZXQoKHZvaWQgKikgcCwgNjQsIHN0cmlkZSk7DQogICAgICB9
DQoNCiAgICBpZigobXVubWFwKG1hcHBlZCxmaWxlbGVuKSkgPT0gLTEpDQog
ICAgICB7DQogICAgICAgIHBlcnJvcigibXVubWFwIGZhaWxlZCIpOw0KICAg
ICAgICBleGl0KDEpOw0KICAgICAgfQ0KICB9DQojZW5kaWYNCg0KICBwcmlu
dGYoIlxuIik7DQogIGNsb3NlKGZkKTsNCiAgdW5saW5rKHRlc3RmaWxlKTsN
CiAgcmV0dXJuIDA7DQp9DQo=

---559023410-1202595967-1095388144=:15634
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Tiff mailing list
Tiff@remotesensing.org
http://xserve.flids.com/mailman/listinfo/tiff

---559023410-1202595967-1095388144=:15634--
