Return-Path: <awaresystems.be+caf_=filtered=awaresystems.be@gmail.com>
Delivered-To: filtered@awaresystems.be
Received: from mx3.pub.mailpod2-cph3.one.com ([10.27.25.13])
	by mailstorage18.cst.mailpod2-cph3.one.com (Dovecot) with LMTP id OthtMxJHXlp7VAEAQS8vrQ
	for <filtered@awaresystems.be>; Tue, 16 Jan 2018 19:30:40 +0000
X-HalOne-ID: bb80392e-faf3-11e7-9b3c-e0d84895d241
Received: from mail-oi0-f43.google.com (unknown [209.85.218.43])
	by mx3.pub.mailpod2-cph3.one.com (Halon) with ESMTPS
	id bb80392e-faf3-11e7-9b3c-e0d84895d241;
	Tue, 16 Jan 2018 19:30:39 +0000 (UTC)
Received: by mail-oi0-f43.google.com with SMTP id t8so11433432oie.6
        for <filtered@awaresystems.be>; Tue, 16 Jan 2018 11:30:39 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-original-authentication-results:x-gm-message-state:delivered-to
         :in-reply-to:references:to:mime-version:from:message-id:date:cc
         :subject:precedence:list-id:list-unsubscribe:list-archive:list-post
         :list-help:list-subscribe:mime-version:sender:errors-to;
        bh=pJ+NKE7cfB3MinoY3an/z23SRIxPgZpq/kcJS09O4Ck=;
        b=iW5LP2OBbIoqvfSR6vDPE3Sqc0VFuarN03Ab8aWtA/Df6n52WJb6tqgvD6ClklPT3C
         6e/Acs+rgTWWnvWk4EosVsq/wzplnRpyXbXVzsc5qc01rlKHOb8dFAvhOLJjfXEVjIQx
         DldU3j+pQTHOR7mAJlRV2dcNBl7urgtjK2iC1zcMDRCdcV2ZjfFzFoRQtzBY297KWSe7
         FLUz1vMyfqGUe+9EjnE4yzxTTOZChHRuXMAuUlUxIZPG37tSyDJefVUy1l3CF5zdFgoU
         BsB0w8SIVMYmqb0EsNCLLUFAWtUpnM7UPDRUqgVvXISTskA2UU0x3yQsMUrEiVdDhFMy
         2Nvg==
X-Original-Authentication-Results: mx.google.com;       spf=neutral (google.com: 94.23.4.142 is neither permitted nor denied by best guess record for domain of tiff-bounces@lists.maptools.org) smtp.mailfrom=tiff-bounces@lists.maptools.org
X-Gm-Message-State: AKGB3mKozdIAwhl62OKum9h2JbDxIAwvuZpqgRIsbr/s9K850ZSlmAli
	AVf8Q7D6jthDATBB2wDKB6vuBl8GQ9oyWOIoMUtRP/+yMGiTZB4=
X-Received: by 10.202.5.5 with SMTP id 5mr22657376oif.110.1516131038310;
        Tue, 16 Jan 2018 11:30:38 -0800 (PST)
X-Forwarded-To: filtered@awaresystems.be
X-Forwarded-For: awaresystems.be@gmail.com filtered@awaresystems.be
Delivered-To: awaresystems.be@gmail.com
Received: by 10.202.235.11 with SMTP id j11csp4565951oih;
        Tue, 16 Jan 2018 11:30:37 -0800 (PST)
X-Google-Smtp-Source: ACJfBos7YzmqC1UDn0FyziduU2hjp/bLu+lp32M6XeQMoLTTH78bdSd/Sl1rp8zSHnr0NrNBzm4n
X-Received: by 10.28.232.131 with SMTP id f3mr148275wmi.69.1516131037123;
        Tue, 16 Jan 2018 11:30:37 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; t=1516131037; cv=none;
        d=google.com; s=arc-20160816;
        b=lZbTHTMaZfVb6FELd8b0gx6jjfTccIbvTRja1WWh2QPxOT5reGKo7T0tJKFkz1cYbQ
         ujrzGcbY+GaohVzOezEJPCxP2mhav7LYjA9ABu0UmurIdnh+/tGH/149iQJ3DrZTMZVI
         0xXVPzR/TA294zPjbo5aBNm497GeBhsUxDWYsn2Hcp9mcXdJu8aohlTUoiUlYh6cvVes
         Q3kAW5FzXydQMwArQQo3mx0S06KS9/S0SQCPp3aIdhAGLknB4HOx1HlaYz6/0Y8C6IhX
         uAy9F7S8rHiZ85p6G58dKlX0M5NM7yrnGtxsCuHR5hTSK7KSy2soOlMZnkOKM9Lhz4zy
         fqVA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=errors-to:sender:mime-version:list-subscribe:list-help:list-post
         :list-archive:list-unsubscribe:list-id:precedence:subject:cc:date
         :message-id:from:mime-version:to:references:in-reply-to
         :arc-authentication-results;
        bh=pJ+NKE7cfB3MinoY3an/z23SRIxPgZpq/kcJS09O4Ck=;
        b=lj1XrI5O0uI4G7HJcgD6p94VMY9TfdrytBHB0XghXd++s33HYrTo4wgOTkMBLIJrCU
         3Jw0hAT6SMPAxp50mp6/Y5CMZ58k85I1j48JTJTuU9YkpZVy+z7behkjz/xJ122s6MFY
         4u37jmAhVX/OcaCCRrcA/j9hUxO+tcG7hsy8IeXZ6fA9+i8iWIiWslZ3k2CTm7FEmwJl
         s7zCY0NcWrqaMbIk4rYAfh/3qeLnpklg1VKBqiufit70PmM1Vla8C9YExqhP2nbdaIO1
         AGw6jKEmhKUiyz6+QkkNhh08AMi8Z4OoSIKWRkdUCLIeJJyVJ8rkaACMSKYP8on5D4ne
         iaXA==
ARC-Authentication-Results: i=1; mx.google.com;
       spf=neutral (google.com: 94.23.4.142 is neither permitted nor denied by best guess record for domain of tiff-bounces@lists.maptools.org) smtp.mailfrom=tiff-bounces@lists.maptools.org
Return-Path: <tiff-bounces@lists.maptools.org>
Received: from lists.maptools.org (ks365428.kimsufi.com. [94.23.4.142])
        by mx.google.com with ESMTP id k1si2465562wrg.303.2018.01.16.11.30.36;
        Tue, 16 Jan 2018 11:30:37 -0800 (PST)
Received-SPF: neutral (google.com: 94.23.4.142 is neither permitted nor denied by best guess record for domain of tiff-bounces@lists.maptools.org) client-ip=94.23.4.142;
Authentication-Results: mx.google.com;
       spf=neutral (google.com: 94.23.4.142 is neither permitted nor denied by best guess record for domain of tiff-bounces@lists.maptools.org) smtp.mailfrom=tiff-bounces@lists.maptools.org
Received: from [10.1.0.241] (localhost.localdomain [127.0.0.1])
	by lists.maptools.org (Postfix) with ESMTP id 6FF7635C0A4;
	Tue, 16 Jan 2018 14:29:41 -0500 (EST)
Received: from mx0a-00145802.pphosted.com (mx0a-00145802.pphosted.com
	[67.231.148.190])
	by lists.maptools.org (Postfix) with ESMTPS id 8B0B735C09F;
	Tue, 16 Jan 2018 14:17:38 -0500 (EST)
Received: from pps.filterd (m0054486.ppops.net [127.0.0.1])
	by mx0a-00145802.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id
	w0GJFAs0023293; Tue, 16 Jan 2018 14:17:37 -0500
Received: from mgamac05.us.ricoh.ds (mcnhost1.ikon.com [205.145.16.4])
	by mx0a-00145802.pphosted.com with ESMTP id 2ffcbq9nbx-1;
	Tue, 16 Jan 2018 14:17:37 -0500
In-Reply-To: <CACp925S7DbCgFDNOu2cMRV++t0=5KMR24e8MAMm18uS5ptSLAQ@mail.gmail.com>
References: <39EE49FE-23A5-43DE-8998-51526287D182@larrygritz.com>	<alpine.GSO.2.20.1801151400550.18206@scrappy.simplesystems.org>
	<4252FC16-C07D-4FA1-8B7B-25D626633236@larrygritz.com>	<F78FB533EE93A044B560CD25E17A4DAF18388904@mbx031-e1-va-4.exch031.domain.local>
	<alpine.GSO.2.20.1801151728170.18206@scrappy.simplesystems.org>	<f279e2e7-a91a-0cd8-a513-026479f571ff@cubewerx.com>
	<OF239FD1E6.9CEBEE62-ON87258217.0064336B-87258217.00663BB4@ricoh-usa.com>
	<CACp925S7DbCgFDNOu2cMRV++t0=5KMR24e8MAMm18uS5ptSLAQ@mail.gmail.com>
To: Kemp Watson <kemp@objectivepathology.com>
MIME-Version: 1.0
X-KeepSent: E2B62784:B3548DA6-87258217:0069D01C;
 type=4; name=$KeepSent
X-Mailer: IBM Notes Release 9.0 March 08, 2013
From: Joe.Maniaci@ricoh-usa.com
Message-ID: <OFE2B62784.B3548DA6-ON87258217.0069D01C-87258217.0069FB4C@ricoh-usa.com>
Date: Tue, 16 Jan 2018 12:17:35 -0700
X-MIMETrack: Serialize by Router on MGAMAC05/Servers/RCUS(Release
	9.0.1FP7|August 17, 2016) at 01/16/2018 14:17:36,
	Serialize complete at 01/16/2018 14:17:36
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, ,
	definitions=2018-01-16_10:, , signatures=0
Cc: tiff-bounces@lists.maptools.org, tiff@lists.maptools.org
Subject: Re: [Tiff] Strategies for multi-core speedups
X-BeenThere: tiff@lists.maptools.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: TIFF and libtiff discussion <tiff.lists.maptools.org>
List-Unsubscribe: <http://lists.maptools.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@lists.maptools.org?subject=unsubscribe>
List-Archive: <http://lists.maptools.org/mailman/private/tiff>
List-Post: <mailto:tiff@lists.maptools.org>
List-Help: <mailto:tiff-request@lists.maptools.org?subject=help>
List-Subscribe: <http://lists.maptools.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@lists.maptools.org?subject=subscribe>
Content-Type: multipart/mixed; boundary="===============1621051501=="
Mime-version: 1.0
Sender: tiff-bounces@lists.maptools.org
Errors-To: tiff-bounces@lists.maptools.org

This is a multipart message in MIME format.
--===============1621051501==
Content-Type: multipart/alternative; boundary="=_alternative 0069FAE887258217_="

This is a multipart message in MIME format.
--=_alternative 0069FAE887258217_=
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: quoted-printable

Thank you for that. OpenSeaDragon looks promising, but in addition to=20
viewing the tool has to take two images in two viewing panes and=20
programmatically compare them against another and generate a miscompare=20
image in a third pane, all in the same tool. But perhaps I can use their=20
tiling algorithms.

Thanks,

Joseph Maniaci
Staff Software Engineer
RPPS



From:   Kemp Watson <kemp@objectivepathology.com>
To:     Joe.Maniaci@ricoh-usa.com
Cc:     tiff@lists.maptools.org
Date:   01/16/2018 12:02 PM
Subject:        Re: [Tiff] Strategies for multi-core speedups [EXTERNAL]
Sent by:        tiff-bounces@lists.maptools.org



Hi Joseph:

There are quite a number of tools already extant, some for over 15 years=20
now, that do roughly what you are describing - we often view 1 TB+ BigTIFF =

files interactively - 40GB is a cakewalk.
Search for medical imaging tools such as ObjectiveView, ImageScope, and=20
open source such as Leaflet, OpenSeadragon, OpenLayers.

Best,

Kemp Watson
Objective Pathology Services

On Tue, Jan 16, 2018 at 1:36 PM, <Joe.Maniaci@ricoh-usa.com> wrote:
I am a complete newbie to digital imagery and TIFFs in general, so please=20
forgive any ignorance.=20

I'm an automated test tool developer and I am actually going to start=20
working on a viewer for the TIFFs that my company generates, to=20
potentially include relying on the bigtiff format and this is the thing=20
I've spent most of my time thinking about. I've come to the conclusion=20
that for the images we generate, up to 10GB uncompressed(soon to be 40GB), =

that I will need to preprocess a single TIFF into multiple TIFFs=20
containing intelligently placed tiles just to get around the I/O issue. By =

intelligently placed, I mean that if a user were to scroll their view=20
window to the right, and five tiles(vertically aligned) had to be loaded=20
into memory, I could theoretically have those files intelligently located=20
in different TIFF files(subTIFFs?) that would allow multiple threads to=20
operate on. So each thread only had to find one tile. So for my tool it=20
would be a staggering amount of pre-processing, but real time manipulation =

is extremely fluid for the user which is basically my number one=20
requirement.=20

So for a case like mine, thread overhead is nothing to worry about. That=20
and the more I read about the C++11 thread library, the less I worry. I=20
can just leave it to the library to determine how many threads should=20
operate in my 4-core environments, or my 16-core environments. At least=20
that's the impression I've garnered.=20

Joseph Maniaci=20
Staff Software Engineer=20
RPPS=20



From:        Craig Bruce <csbruce@cubewerx.com>=20
To:        tiff@lists.maptools.org=20
Date:        01/15/2018 06:15 PM=20
Subject:        Re: [Tiff] Strategies for multi-core speedups [EXTERNAL]=20
Sent by:        tiff-bounces@lists.maptools.org=20




On 01/15/2018 08:15 PM, Bob Friesenhahn wrote:=20
On Mon, 15 Jan 2018, Mike Stanton wrote:

To begin with I am an Operating Systems guy.  When you are looking=20
at going to multi-threading, what have you determined as your=20
setup/teardown cost for the individual threads? How large a chunk=20
will each thread be processing?  Will the individual Scan-lines=20
and/or Tiles be sufficiently large to absorb the cost of that setup=20
and teardown by the inherent performance you are projecting?

It is normal to use a pool of threads where threads are created in=20
advance (or added to the pool on demand) and obtained from, and=20
returned to, the pool.  This significantly reduces overheads caused by=20
starting/stopping threads.=20
Creating and destroying threads ought to take on the scale of a few=20
microseconds (on Linux anyway).  Larry Gritz was saying it takes 16.2=20
seconds to read what could be 1024 tiles of size 256=D7256, which is 15.8 m=
s=20
per tile, which is nearly four orders of magnitude longer than thread=20
creation for reading.  Writing with Deflate compression is about an order=20
of magnitude slower than reading.

However, you don't need to launch a new thread for every tile if you just=20
break up the job into N=3Dnumber=5Fof=5Fcpus pieces and launch N threads wh=
ere=20
each is pre-assigned to read or write a particular group of tiles of the=20
overall image.  The thread-creation overhead would be around six or seven=20
orders of magnitude smaller than the reading or writing.

--=20
Dr. Craig S. Bruce=20
Senior Software Developer=20
CubeWerx Inc.=20
http://www.cubewerx.com=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F
Tiff mailing list: Tiff@lists.maptools.org
https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A=5F=5Flists.maptools.or=
g=5Fmailman=5Flistinfo=5Ftiff&d=3DDwICAg&c=3D5hYF0Zu0Yz-C6S-kaHDItw&r=3DHnV=
w9YJBS4ST0tOBucT3NvWUJRRGhDxXu4zHnS5AkgY&m=3DmaPcpWIIyHM0McvhMluxRFvFc3ku8e=
N0opVcbXErRdw&s=3DgO3D=5FJk1nPaHZIGWP4FoVE2ZjlePymETKPEx3kVi0g4&e=3D

https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A=5F=5Fwww.remotesensing=
.org=5Flibtiff=5F&d=3DDwICAg&c=3D5hYF0Zu0Yz-C6S-kaHDItw&r=3DHnVw9YJBS4ST0tO=
BucT3NvWUJRRGhDxXu4zHnS5AkgY&m=3DmaPcpWIIyHM0McvhMluxRFvFc3ku8eN0opVcbXErRd=
w&s=3DKMgAOGLTaf48zkJ9S4jYu9pWlvzrZ87u0jtXhoAD66c&e=3D=20


=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
Tiff mailing list: Tiff@lists.maptools.org
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
Tiff mailing list: Tiff@lists.maptools.org
https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A=5F=5Flists.maptools.or=
g=5Fmailman=5Flistinfo=5Ftiff&d=3DDwICAg&c=3D5hYF0Zu0Yz-C6S-kaHDItw&r=3DHnV=
w9YJBS4ST0tOBucT3NvWUJRRGhDxXu4zHnS5AkgY&m=3DHtwa6DuM=5FSH1PYJzBxXj9jdaYzMb=
3bsyhZHiQAjB-94&s=3Dn2nHA6nM6pzMQ-UyFhMUvtiB0wBzfy5mXKWZX-uGe9I&e=3D

https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A=5F=5Fwww.remotesensing=
.org=5Flibtiff=5F&d=3DDwICAg&c=3D5hYF0Zu0Yz-C6S-kaHDItw&r=3DHnVw9YJBS4ST0tO=
BucT3NvWUJRRGhDxXu4zHnS5AkgY&m=3DHtwa6DuM=5FSH1PYJzBxXj9jdaYzMb3bsyhZHiQAjB=
-94&s=3DdsoTL2jkBOrpIrLZkKofX8AJYc70-qEnh66oR50n0w8&e=3D

--=_alternative 0069FAE887258217_=
Content-Type: text/html; charset="ISO-8859-1"
Content-Transfer-Encoding: quoted-printable

<font size=3D2 face=3D"sans-serif">Thank you for that. OpenSeaDragon looks
promising, but in addition to viewing the tool has to take two images in
two viewing panes and programmatically compare them against another and
generate a miscompare image in a third pane, all in the same tool. But
perhaps I can use their tiling algorithms.</font>
<br>
<br><font size=3D2 face=3D"sans-serif">Thanks,</font>
<br>
<br><font size=3D2 face=3D"sans-serif">Joseph Maniaci</font>
<br><font size=3D2 face=3D"sans-serif">Staff Software Engineer</font>
<br><font size=3D2 face=3D"sans-serif">RPPS</font>
<br>
<br>
<br>
<br><font size=3D1 color=3D#5f5f5f face=3D"sans-serif">From: &nbsp; &nbsp; =
&nbsp;
&nbsp;</font><font size=3D1 face=3D"sans-serif">Kemp Watson &lt;kemp@object=
ivepathology.com&gt;</font>
<br><font size=3D1 color=3D#5f5f5f face=3D"sans-serif">To: &nbsp; &nbsp; &n=
bsp;
&nbsp;</font><font size=3D1 face=3D"sans-serif">Joe.Maniaci@ricoh-usa.com</=
font>
<br><font size=3D1 color=3D#5f5f5f face=3D"sans-serif">Cc: &nbsp; &nbsp; &n=
bsp;
&nbsp;</font><font size=3D1 face=3D"sans-serif">tiff@lists.maptools.org</fo=
nt>
<br><font size=3D1 color=3D#5f5f5f face=3D"sans-serif">Date: &nbsp; &nbsp; =
&nbsp;
&nbsp;</font><font size=3D1 face=3D"sans-serif">01/16/2018 12:02 PM</font>
<br><font size=3D1 color=3D#5f5f5f face=3D"sans-serif">Subject: &nbsp; &nbs=
p;
&nbsp; &nbsp;</font><font size=3D1 face=3D"sans-serif">Re: [Tiff] Strategies
for multi-core speedups [EXTERNAL]</font>
<br><font size=3D1 color=3D#5f5f5f face=3D"sans-serif">Sent by: &nbsp; &nbs=
p;
&nbsp; &nbsp;</font><font size=3D1 face=3D"sans-serif">tiff-bounces@lists.m=
aptools.org</font>
<br>
<hr noshade>
<br>
<br>
<br><font size=3D3>Hi Joseph:<br>
</font>
<br><font size=3D3>There are quite a number of tools already extant, some
for over 15 years now, that do roughly what you are describing - we often
view 1 TB+ BigTIFF files interactively - 40GB is a cakewalk.</font>
<br><font size=3D3>Search for medical imaging tools such as ObjectiveView,
ImageScope, and open source such as Leaflet, OpenSeadragon, OpenLayers.<br>
</font>
<br><font size=3D3>Best,<br>
</font>
<br><font size=3D3>Kemp Watson</font>
<br><font size=3D3>Objective Pathology Services</font>
<br>
<br><font size=3D3>On Tue, Jan 16, 2018 at 1:36 PM, &lt;</font><a href=3D"m=
ailto:Joe.Maniaci@ricoh-usa.com" target=3D=5Fblank><font size=3D3 color=3Db=
lue><u>Joe.Maniaci@ricoh-usa.com</u></font></a><font size=3D3>&gt;
wrote:</font>
<br><font size=3D2 face=3D"sans-serif">I am a complete newbie to digital im=
agery
and TIFFs in general, so please forgive any ignorance.</font><font size=3D3>
<br>
</font><font size=3D2 face=3D"sans-serif"><br>
I'm an automated test tool developer and I am actually going to start worki=
ng
on a viewer for the TIFFs that my company generates, to potentially include
relying on the bigtiff format and this is the thing I've spent most of
my time thinking about. I've come to the conclusion that for the images
we generate, up to 10GB uncompressed(soon to be 40GB), that I will need
to preprocess a single TIFF into multiple TIFFs containing intelligently
placed tiles just to get around the I/O issue. By intelligently placed,
I mean that if a user were to scroll their view window to the right, and
five tiles(vertically aligned) had to be loaded into memory, I could theore=
tically
have those files intelligently located in different TIFF files(subTIFFs?)
that would allow multiple threads to operate on. So each thread only had
to find one tile. So for my tool it would be a staggering amount of pre-pro=
cessing,
but real time manipulation is extremely fluid for the user which is basical=
ly
my number one requirement.</font><font size=3D3> <br>
</font><font size=3D2 face=3D"sans-serif"><br>
So for a case like mine, thread overhead is nothing to worry about. That
and the more I read about the C++11 thread library, the less I worry. I
can just leave it to the library to determine how many threads should opera=
te
in my 4-core environments, or my 16-core environments. At least that's
the impression I've garnered.</font><font size=3D3> <br>
</font><font size=3D2 face=3D"sans-serif"><br>
Joseph Maniaci</font><font size=3D3> </font><font size=3D2 face=3D"sans-ser=
if"><br>
Staff Software Engineer</font><font size=3D3> </font><font size=3D2 face=3D=
"sans-serif"><br>
RPPS</font><font size=3D3> <br>
<br>
<br>
</font><font size=3D1 color=3D#5f5f5f face=3D"sans-serif"><br>
From: &nbsp; &nbsp; &nbsp; &nbsp;</font><font size=3D1 face=3D"sans-serif">=
Craig
Bruce &lt;</font><a href=3Dmailto:csbruce@cubewerx.com target=3D=5Fblank><f=
ont size=3D1 color=3Dblue face=3D"sans-serif"><u>csbruce@cubewerx.com</u></=
font></a><font size=3D1 face=3D"sans-serif">&gt;</font><font size=3D3>
</font><font size=3D1 color=3D#5f5f5f face=3D"sans-serif"><br>
To: &nbsp; &nbsp; &nbsp; &nbsp;</font><a href=3Dmailto:tiff@lists.maptools.=
org target=3D=5Fblank><font size=3D1 color=3Dblue face=3D"sans-serif"><u>ti=
ff@lists.maptools.org</u></font></a><font size=3D3>
</font><font size=3D1 color=3D#5f5f5f face=3D"sans-serif"><br>
Date: &nbsp; &nbsp; &nbsp; &nbsp;</font><font size=3D1 face=3D"sans-serif">=
01/15/2018
06:15 PM</font><font size=3D3> </font><font size=3D1 color=3D#5f5f5f face=
=3D"sans-serif"><br>
Subject: &nbsp; &nbsp; &nbsp; &nbsp;</font><font size=3D1 face=3D"sans-seri=
f">Re:
[Tiff] Strategies for multi-core speedups [EXTERNAL]</font><font size=3D3>
</font><font size=3D1 color=3D#5f5f5f face=3D"sans-serif"><br>
Sent by: &nbsp; &nbsp; &nbsp; &nbsp;</font><a href=3D"mailto:tiff-bounces@l=
ists.maptools.org" target=3D=5Fblank><font size=3D1 color=3Dblue face=3D"sa=
ns-serif"><u>tiff-bounces@lists.maptools.org</u></font></a><font size=3D3>
<br>
</font>
<hr noshade>
<br><font size=3D3><br>
<br>
<br>
On 01/15/2018 08:15 PM, Bob Friesenhahn wrote: </font><tt><font size=3D3><b=
r>
On Mon, 15 Jan 2018, Mike Stanton wrote:</font></tt><font size=3D3><br>
</font><tt><font size=3D3><br>
To begin with I am an Operating Systems guy.&nbsp; When you are looking
<br>
at going to multi-threading, what have you determined as your <br>
setup/teardown cost for the individual threads? How large a chunk <br>
will each thread be processing?&nbsp; Will the individual Scan-lines <br>
and/or Tiles be sufficiently large to absorb the cost of that setup <br>
and teardown by the inherent performance you are projecting?</font></tt><fo=
nt size=3D3><br>
</font><tt><font size=3D3><br>
It is normal to use a pool of threads where threads are created in <br>
advance (or added to the pool on demand) and obtained from, and <br>
returned to, the pool.&nbsp; This significantly reduces overheads caused
by <br>
starting/stopping threads.</font></tt><font size=3D3> <br>
Creating and destroying threads ought to take on the scale of a few microse=
conds
(on Linux anyway).&nbsp; Larry Gritz was saying it takes 16.2 seconds to
read what could be 1024 tiles of size 256=D7256, which is 15.8 ms per tile,
which is nearly four orders of magnitude longer than thread creation for
reading.&nbsp; Writing with Deflate compression is about an order of magnit=
ude
slower than reading.<br>
<br>
However, you don't need to launch a new thread for every tile if you just
break up the job into N=3Dnumber=5Fof=5Fcpus pieces and launch N threads wh=
ere
each is pre-assigned to read or write a particular group of tiles of the
overall image.&nbsp; The thread-creation overhead would be around six or
seven orders of magnitude smaller than the reading or writing.<br>
<br>
-- <br>
Dr. Craig S. Bruce <br>
Senior Software Developer <br>
CubeWerx<i> </i>Inc. </font>
<br><a href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A=5F=5Fww=
w.cubewerx.com=5F&amp;d=3DDwMDaQ&amp;c=3D5hYF0Zu0Yz-C6S-kaHDItw&amp;r=3DHnV=
w9YJBS4ST0tOBucT3NvWUJRRGhDxXu4zHnS5AkgY&amp;m=3DmaPcpWIIyHM0McvhMluxRFvFc3=
ku8eN0opVcbXErRdw&amp;s=3D-TSTD6g4v02=5FBC45E6RRb534KRI996uryYX=5FpCs5p2Q&a=
mp;e=3D" target=3D=5Fblank><font size=3D3 color=3Dblue><u>http://www.cubewe=
rx.com</u></font></a><tt><font size=3D2>=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F<br>
Tiff mailing list: </font></tt><a href=3Dmailto:Tiff@lists.maptools.org tar=
get=3D=5Fblank><tt><font size=3D2 color=3Dblue><u>Tiff@lists.maptools.org</=
u></font></tt></a><font size=3D3 color=3Dblue><u><br>
</u></font><a href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A=
=5F=5Flists.maptools.org=5Fmailman=5Flistinfo=5Ftiff&amp;d=3DDwICAg&amp;c=
=3D5hYF0Zu0Yz-C6S-kaHDItw&amp;r=3DHnVw9YJBS4ST0tOBucT3NvWUJRRGhDxXu4zHnS5Ak=
gY&amp;m=3DmaPcpWIIyHM0McvhMluxRFvFc3ku8eN0opVcbXErRdw&amp;s=3DgO3D=5FJk1nP=
aHZIGWP4FoVE2ZjlePymETKPEx3kVi0g4&amp;e=3D" target=3D=5Fblank><tt><font siz=
e=3D2 color=3Dblue><u>https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A=
=5F=5Flists.maptools.org=5Fmailman=5Flistinfo=5Ftiff&amp;d=3DDwICAg&amp;c=
=3D5hYF0Zu0Yz-C6S-kaHDItw&amp;r=3DHnVw9YJBS4ST0tOBucT3NvWUJRRGhDxXu4zHnS5Ak=
gY&amp;m=3DmaPcpWIIyHM0McvhMluxRFvFc3ku8eN0opVcbXErRdw&amp;s=3DgO3D=5FJk1nP=
aHZIGWP4FoVE2ZjlePymETKPEx3kVi0g4&amp;e=3D</u></font></tt></a><font size=3D=
3 color=3Dblue><u><br>
</u></font><a href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A=
=5F=5Fwww.remotesensing.org=5Flibtiff=5F&amp;d=3DDwICAg&amp;c=3D5hYF0Zu0Yz-=
C6S-kaHDItw&amp;r=3DHnVw9YJBS4ST0tOBucT3NvWUJRRGhDxXu4zHnS5AkgY&amp;m=3DmaP=
cpWIIyHM0McvhMluxRFvFc3ku8eN0opVcbXErRdw&amp;s=3DKMgAOGLTaf48zkJ9S4jYu9pWlv=
zrZ87u0jtXhoAD66c&amp;e=3D" target=3D=5Fblank><tt><font size=3D2 color=3Dbl=
ue><u>https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A=5F=5Fwww.remotes=
ensing.org=5Flibtiff=5F&amp;d=3DDwICAg&amp;c=3D5hYF0Zu0Yz-C6S-kaHDItw&amp;r=
=3DHnVw9YJBS4ST0tOBucT3NvWUJRRGhDxXu4zHnS5AkgY&amp;m=3DmaPcpWIIyHM0McvhMlux=
RFvFc3ku8eN0opVcbXErRdw&amp;s=3DKMgAOGLTaf48zkJ9S4jYu9pWlvzrZ87u0jtXhoAD66c=
&amp;e=3D</u></font></tt></a><font size=3D3>
<br>
<br>
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F<br>
Tiff mailing list: </font><a href=3Dmailto:Tiff@lists.maptools.org><font si=
ze=3D3 color=3Dblue><u>Tiff@lists.maptools.org</u></font></a><font size=3D3=
 color=3Dblue><u><br>
</u></font><a href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A=
=5F=5Flists.maptools.org=5Fmailman=5Flistinfo=5Ftiff&amp;d=3DDwMFaQ&amp;c=
=3D5hYF0Zu0Yz-C6S-kaHDItw&amp;r=3DHnVw9YJBS4ST0tOBucT3NvWUJRRGhDxXu4zHnS5Ak=
gY&amp;m=3DHtwa6DuM=5FSH1PYJzBxXj9jdaYzMb3bsyhZHiQAjB-94&amp;s=3Dn2nHA6nM6p=
zMQ-UyFhMUvtiB0wBzfy5mXKWZX-uGe9I&amp;e=3D" target=3D=5Fblank><font size=3D=
3 color=3Dblue><u>http://lists.maptools.org/mailman/listinfo/tiff</u></font=
></a><font size=3D3 color=3Dblue><u><br>
</u></font><a href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A=
=5F=5Fwww.remotesensing.org=5Flibtiff=5F&amp;d=3DDwMFaQ&amp;c=3D5hYF0Zu0Yz-=
C6S-kaHDItw&amp;r=3DHnVw9YJBS4ST0tOBucT3NvWUJRRGhDxXu4zHnS5AkgY&amp;m=3DHtw=
a6DuM=5FSH1PYJzBxXj9jdaYzMb3bsyhZHiQAjB-94&amp;s=3DdsoTL2jkBOrpIrLZkKofX8AJ=
Yc70-qEnh66oR50n0w8&amp;e=3D" target=3D=5Fblank><font size=3D3 color=3Dblue=
><u>http://www.remotesensing.org/libtiff/</u></font></a>
<br><tt><font size=3D2>=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F<br>
Tiff mailing list: Tiff@lists.maptools.org<br>
</font></tt><a href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A=
=5F=5Flists.maptools.org=5Fmailman=5Flistinfo=5Ftiff&amp;d=3DDwICAg&amp;c=
=3D5hYF0Zu0Yz-C6S-kaHDItw&amp;r=3DHnVw9YJBS4ST0tOBucT3NvWUJRRGhDxXu4zHnS5Ak=
gY&amp;m=3DHtwa6DuM=5FSH1PYJzBxXj9jdaYzMb3bsyhZHiQAjB-94&amp;s=3Dn2nHA6nM6p=
zMQ-UyFhMUvtiB0wBzfy5mXKWZX-uGe9I&amp;e=3D"><tt><font size=3D2>https://urld=
efense.proofpoint.com/v2/url?u=3Dhttp-3A=5F=5Flists.maptools.org=5Fmailman=
=5Flistinfo=5Ftiff&amp;d=3DDwICAg&amp;c=3D5hYF0Zu0Yz-C6S-kaHDItw&amp;r=3DHn=
Vw9YJBS4ST0tOBucT3NvWUJRRGhDxXu4zHnS5AkgY&amp;m=3DHtwa6DuM=5FSH1PYJzBxXj9jd=
aYzMb3bsyhZHiQAjB-94&amp;s=3Dn2nHA6nM6pzMQ-UyFhMUvtiB0wBzfy5mXKWZX-uGe9I&am=
p;e=3D</font></tt></a><tt><font size=3D2><br>
</font></tt><a href=3D"https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A=
=5F=5Fwww.remotesensing.org=5Flibtiff=5F&amp;d=3DDwICAg&amp;c=3D5hYF0Zu0Yz-=
C6S-kaHDItw&amp;r=3DHnVw9YJBS4ST0tOBucT3NvWUJRRGhDxXu4zHnS5AkgY&amp;m=3DHtw=
a6DuM=5FSH1PYJzBxXj9jdaYzMb3bsyhZHiQAjB-94&amp;s=3DdsoTL2jkBOrpIrLZkKofX8AJ=
Yc70-qEnh66oR50n0w8&amp;e=3D"><tt><font size=3D2>https://urldefense.proofpo=
int.com/v2/url?u=3Dhttp-3A=5F=5Fwww.remotesensing.org=5Flibtiff=5F&amp;d=3D=
DwICAg&amp;c=3D5hYF0Zu0Yz-C6S-kaHDItw&amp;r=3DHnVw9YJBS4ST0tOBucT3NvWUJRRGh=
DxXu4zHnS5AkgY&amp;m=3DHtwa6DuM=5FSH1PYJzBxXj9jdaYzMb3bsyhZHiQAjB-94&amp;s=
=3DdsoTL2jkBOrpIrLZkKofX8AJYc70-qEnh66oR50n0w8&amp;e=3D</font></tt></a>
<br>
--=_alternative 0069FAE887258217_=--

--===============1621051501==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Tiff mailing list: Tiff@lists.maptools.org
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
--===============1621051501==--
