Return-Path: <owner-tiff@olympiakos.com>
Received: from nataly.at1895.spb.edu (localhost [127.0.0.1])
	by at1895.spb.edu (8.12.4/8.12.4/Debian-4) with ESMTP id g7LEFPgL004106
	for <dron@at1895.spb.edu>; Wed, 21 Aug 2002 18:15:25 +0400
Received: from olympiakos.com (uucp@localhost)
	by nataly.at1895.spb.edu (8.12.4/8.12.4/Debian-4) with UUCP id g7LEFPvm004102
	for dron@at1895.spb.edu; Wed, 21 Aug 2002 18:15:25 +0400
Received: from knot.pu.ru (knot.pu.ru [193.124.85.214] (may be forged))
	by wg.pu.ru (8.9.1a/8.9.1) with ESMTP id NAA24450
	for <dron@at1895.spb.edu>; Wed, 21 Aug 2002 13:59:58 GMT
Received: from gate7.olympiakos.com (sdsl-64-32-199-150.dsl.iad.megapath.net [64.32.199.150])
	by knot.pu.ru (8.9.3/8.9.0) with ESMTP id RAA58728
	for <dron@at1895.spb.edu>; Wed, 21 Aug 2002 17:59:57 +0400 (MSD)
Received: from localhost (daemon@localhost)
	by gate7.olympiakos.com (8.9.3/8.9.3) with SMTP id JAA31789;
	Wed, 21 Aug 2002 09:58:20 -0400
Received: by gate7.olympiakos.com (bulk_mailer v1.9); Wed, 21 Aug 2002 09:52:13 -0400
Received: (from majordom@localhost)
	by gate7.olympiakos.com (8.9.3/8.9.3) id JAA31731
	for tiff-outgoing; Wed, 21 Aug 2002 09:52:12 -0400
X-Authentication-Warning: gate7.olympiakos.com: majordom set sender to owner-tiff@olympiakos.com using -f
Received: from smtp01.oce.nl (smtp01.oce.nl [134.188.1.25])
	by gate7.olympiakos.com (8.9.3/8.9.3) with ESMTP id JAA31728
	for <tiff@olympiakos.com>; Wed, 21 Aug 2002 09:52:08 -0400
Received: (from uucp@localhost)
	by smtp01.oce.nl (8.10.0/8.10.0) id g7LDpNU23824;
	Wed, 21 Aug 2002 15:51:23 +0200 (MET DST)
Received: from smtp05(134.188.1.26) by smtp01.oce.nl via csmap (V4.1)
	id srcAAA0WS_PN; Wed, 21 Aug 02 15:51:22 +0200
Received: (from uucp@localhost)
	by smtp05.oce.nl (8.9.0/8.9.0) id PAA06102;
	Wed, 21 Aug 2002 15:51:19 +0200 (MET DST)
>Received: from oce.nl (WS024537.oce.nl [134.188.160.168])
	by smtp02.oce.nl (8.9.3/8.9.3) with ESMTP id PAA13813;
	Wed, 21 Aug 2002 15:47:00 +0200 (MET DST)
Message-ID: <3D6399D5.EAA159DD@oce.nl>
Date: Wed, 21 Aug 2002 15:47:01 +0200
From: Rob van den Tillaart <rvdt@oce.nl>
X-Mailer: Mozilla 4.61 [en]C-CCK-MCD   (WinNT; I)
X-Accept-Language: en,nl
MIME-Version: 1.0
To: Peter Majer <majer@bitplane.com>
CC: tiff@olympiakos.com
Subject: Re: OT: large memory allocation in Windows
References: <Pine.LNX.4.31.0208211311050.2945-100000@kontiki.bitplane.ch>
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=us-ascii
Sender: owner-tiff@olympiakos.com
Precedence: bulk
Content-Transfer-Encoding: 7bit
Status: RO
Content-Length: 2441
Lines: 64


Some further discussion,

> the problem of heap fragmentation cannot be that seldom.  what
> are the standard proven ways to solve it?  i am quite surprised
> about the lack of information on this topic.
> 
> the solutions that spring to my mind are:
> 
> 1) 64 bit compilation
the problem was that there was no memory block 
large enough available if I recall correctly. 64 bit compilation 
will not help. (64 bit addressable (2^64 bytes) virtual memory would)


> 2) reorganizing memory requirements of the application
>    such that contiguous large blocks are not necessary.
>    E.g. store the data of a large image as an array of
>    rows.
Good point,  
Not all image manipulations would like this, e.g. convolutions 
which use simple pointer math to address neighbour pixels, all 
these routines would need to be rewritten. The speed would / could
decrease, however this is the favorite solution direction.
The rows do not need to be single pixel lines.
The essence of your remark is do not keep it all in memory 
simultaneously.


> 3) make sure the application can run for ever with the memory
>    that can be allocated in the beginning. In case of
>    applications working with images, we might allocate the one image
>    we load plus another one for some processing purposes and
>    make sure that we never ever free these and need to allocate
>    new large blocks.
like the previous solution much more. 

> 
> Also, I have been thinking about the third point you mention as a solution
> as well.  Does anyone by chance know if there is an autopointer class
> out there that is of help with memory reorganization?
> 
>> MERGED MAIL
>
>How about a memory mapped file? Could that possibly give you a 
>linear "pointer space" of 2GB or maybe even more? Of course the file would 
>not be completely mapped at all times, but maybe it would do the trick... 
>(Just a crazy idea. I don't know if it would work at all...).

Used this years ago on some Sun1 Sparcstation with 8MB of memory. The
C-calls
were the "fseek, fread, fwrite" family, but disk is quite slow compared
to on 
line memory. I recall a story of using memory on another Sparc over the
network 
as this was faster than local disk access. OK, in those days everything
was 
faster than a disk :)

regards, rob


To subscribe or unsubscribe email tiff-request@olympiakos.com with subscribe
or unsubscribe in the email body.  Libtiff home page: http://www.libtiff.org

