Return-Path: <awaresystems.be+caf_=filtered=awaresystems.be@gmail.com>
Delivered-To: filtered@awaresystems.be
Received: from mx-l.one.com (mx-l.one.com [91.198.169.59])
	by mail6.local.one.com (Postfix) with ESMTP id E8104300129F60
	for <filtered@awaresystems.be>; Mon,  5 Jan 2015 20:03:51 +0000 (UTC)
X-Virus-Scanned: by one.com
X-Amavis-Alert: BAD HEADER SECTION, Duplicate header field: "MIME-Version"
Received-SPF: pass (mx-l: domain of awaresystems.be+caf_=filtered=awaresystems.be@gmail.com designates 209.85.217.177 as permitted sender)
X-Greylist: whitelisted by SQLgrey-1.8.0-rc2
Received: from mail-lb0-f177.google.com (mail-lb0-f177.google.com [209.85.217.177])
	by mx-l.one.com (Postfix) with ESMTP id AA402200A45E7
	for <filtered@awaresystems.be>; Mon,  5 Jan 2015 20:03:51 +0000 (UTC)
Received: by mail-lb0-f177.google.com with SMTP id b6so17831971lbj.22
        for <filtered@awaresystems.be>; Mon, 05 Jan 2015 12:03:51 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-original-authentication-results:delivered-to:delivered-to
         :mime-version:in-reply-to:references:date:message-id:from:to:cc
         :subject:precedence:list-id:list-unsubscribe:list-archive:list-post
         :list-help:list-subscribe:content-type:mime-version:sender:errors-to;
        bh=lf8LsNtgTyJt2BEl/YOohBxc8xK1soLeebP6EgOsEAA=;
        b=P3RUv3tLukwoNMc56rELN2+ORjvytPwzPHFlLENUmvz5Umb3no+XFM8c0Zdc9mYaKQ
         g2x5u+i0kbyRs86GcZFfH7OFBRAI6HkdW4+XnBo71PlOQHv88fDQmxIMrcpSKL4NI29V
         LnxJSTEykqgGRdyM1qDDqVz5ulFdBP5T908PbIloIX7SBkK8cRzmtcizcGZV7QCKPijA
         DAJnK2qXxPsaDSrrNquhy4d3K7svi6dKJxxsp2kZCktlL/KiaWX1coNpLCsQJwKPzsxi
         4Bi4M0jqRQp8jFU53bkIJpn6ipysHcDvudlediK+OzgyaVEZB3/Xdtx0dNOZ36ip241z
         tUTA==
X-Original-Authentication-Results: mx.google.com;       spf=none (google.com: tiff-bounces@lists.maptools.org does not designate permitted sender hosts) smtp.mail=tiff-bounces@lists.maptools.org
X-Received: by 10.112.188.201 with SMTP id gc9mr58594044lbc.6.1420488231233;
        Mon, 05 Jan 2015 12:03:51 -0800 (PST)
X-Forwarded-To: filtered@awaresystems.be
X-Forwarded-For: awaresystems.be@gmail.com filtered@awaresystems.be
Delivered-To: awaresystems.be@gmail.com
Received: by 10.114.181.147 with SMTP id dw19csp3729175ldc;
        Mon, 5 Jan 2015 12:03:50 -0800 (PST)
X-Received: by 10.152.206.41 with SMTP id ll9mr85052388lac.62.1420488229826;
        Mon, 05 Jan 2015 12:03:49 -0800 (PST)
Authentication-Results: mx.google.com;
       spf=none (google.com: tiff-bounces@lists.maptools.org does not designate permitted sender hosts) smtp.mail=tiff-bounces@lists.maptools.org
Received-SPF: none (google.com: tiff-bounces@lists.maptools.org does not designate permitted sender hosts) client-ip=195.47.247.76;
Received: by 10.112.73.132 with POP3 id l4mf19638947lbv.35;
        Mon, 05 Jan 2015 12:03:49 -0800 (PST)
X-Gmail-Fetch-Info: info@awaresystems.be 1 pop.awaresystems.be 110 info@awaresystems.be
Delivered-To: info@awaresystems.be
Received: from mx05.one.com (mx05.one.com [195.47.247.76])
	by mail6.local.one.com (Postfix) with ESMTP id 448CA3001FDB3E
	for <info@awaresystems.be>; Mon,  5 Jan 2015 19:39:24 +0000 (UTC)
X-Amavis-Alert: BAD HEADER SECTION, Duplicate header field: "MIME-Version"
X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0-rc2
Received: from lists.maptools.org (ks365428.kimsufi.com [94.23.4.142])
	by mx05.one.com (Postfix) with ESMTP id 00A41B6786F
	for <info@awaresystems.be>; Mon,  5 Jan 2015 19:39:23 +0000 (UTC)
Received: from [10.1.0.241] (localhost.localdomain [127.0.0.1])
	by lists.maptools.org (Postfix) with ESMTP id C8A4835C058;
	Mon,  5 Jan 2015 14:38:25 -0500 (EST)
Received: from mail-wg0-f44.google.com (mail-wg0-f44.google.com [74.125.82.44])
	by lists.maptools.org (Postfix) with ESMTPS id 3D56535C054
	for <tiff@lists.maptools.org>; Mon,  5 Jan 2015 14:37:58 -0500 (EST)
Received: by mail-wg0-f44.google.com with SMTP id b13so28158182wgh.17
	for <tiff@lists.maptools.org>; Mon, 05 Jan 2015 11:38:07 -0800 (PST)
MIME-Version: 1.0
X-Received: by 10.180.20.6 with SMTP id j6mr28329279wie.59.1420486686652; Mon,
	05 Jan 2015 11:38:06 -0800 (PST)
Received: by 10.194.161.196 with HTTP; Mon, 5 Jan 2015 11:38:06 -0800 (PST)
In-Reply-To: <alpine.GSO.2.01.1501050852340.18732@freddy.simplesystems.org>
References: <CAK8GPyDdAzUwt7ZTmfF+NT+ceGwvxFPUD+CRq6unxT9KEe5JwQ@mail.gmail.com>
	<alpine.GSO.2.01.1501011102040.18732@freddy.simplesystems.org>
	<CAK8GPyAfmhtC3OaokrkAuLX75iWfDCHnhPS+sjyyZvFE=N6w5A@mail.gmail.com>
	<56071.99.237.145.54.1420262656.squirrel@www.sidefx.com>
	<CAK8GPyA_--nXP7tn5tK3=XnLfXQGE7+BQGmkU54Xs1R2QpW5QA@mail.gmail.com>
	<CAK8GPyC7M6DuLMtG_UtadsurUvEMbFagSx0BD1aBZjjaJa2i6Q@mail.gmail.com>
	<alpine.GSO.2.01.1501042246490.18732@freddy.simplesystems.org>
	<CAK8GPyAoJDi8cisMeg1O6F5bo-LSq6=N==16BwE7PzB7iWHnyg@mail.gmail.com>
	<CAK8GPyB2oJU-B7E_EB+_zTQvc7R6kq9ge3JiqHjP85JP8WH1Dw@mail.gmail.com>
	<CAK8GPyAGPMTodQg0LWzekHFaRmOPUgoFUHaefhbwViR2D=+3ZA@mail.gmail.com>
	<alpine.GSO.2.01.1501050852340.18732@freddy.simplesystems.org>
Date: Mon, 5 Jan 2015 14:38:06 -0500
Message-ID: <CAK8GPyDezoHnw-XWXr-hW+0hhrpwxUpgvQ2d_a+RYhkTuk72_w@mail.gmail.com>
From: Aaron Boxer <boxerab@gmail.com>
To: Bob Friesenhahn <bfriesen@simple.dallas.tx.us>
Cc: TIFF mailing list <tiff@lists.maptools.org>
Subject: Re: [Tiff] Fast TIFF Reading on Windows
X-BeenThere: tiff@lists.maptools.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: TIFF and libtiff discussion <tiff.lists.maptools.org>
List-Unsubscribe: <http://lists.maptools.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@lists.maptools.org?subject=unsubscribe>
List-Archive: <http://lists.maptools.org/mailman/private/tiff>
List-Post: <mailto:tiff@lists.maptools.org>
List-Help: <mailto:tiff-request@lists.maptools.org?subject=help>
List-Subscribe: <http://lists.maptools.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@lists.maptools.org?subject=subscribe>
Content-Type: multipart/mixed; boundary="===============0036576085=="
Mime-version: 1.0
Sender: tiff-bounces@lists.maptools.org
Errors-To: tiff-bounces@lists.maptools.org

--===============0036576085==
Content-Type: multipart/alternative; boundary=bcaec53f38235dc08e050becd45a

--bcaec53f38235dc08e050becd45a
Content-Type: text/plain; charset=UTF-8

On Mon, Jan 5, 2015 at 10:05 AM, Bob Friesenhahn <
bfriesen@simple.dallas.tx.us> wrote:

> On Mon, 5 Jan 2015, Aaron Boxer wrote:
>
>>
>> What would be interesting is to use multiple threads to read a single
>> TIFF image. This would be particularly
>> advantageous
>> for BigTIFF.  My library is using C++11, so threading support is baked
>> in.  Its quite easy, keeping in mind
>> that threads are evil.
>> For libtiff, one would need to find a good cross platform C threading
>> library.
>>
>> For a single file, the approach could be similar: a thread safe list of
>> strips, and each thread gets the next
>> strip and reads it in.
>>
>> Actually, this would be a perfect job for OpenMP.  And you wouldn't need
>> a thread safe list, just divide up the
>> strips evenly across threads.
>>
>
> Libtiff does not use any locking or use thread-specific contexts so I
> think it is definitely not safe for using code to request multiple strips
> from the same image at a time with the same tiff handle using multiple
> threads.  It should be mostly safe to read multiple images at one time
> (different tiff handles) using multiple threads.  It should be safe to open
> the same input TIFF file several times so that there is a different handle
> for each reading thread.  The threads would then need to agree on which
> strips/tiles they would read.
>
> It would be good for libtiff to support multiple strip/tile reads at a
> time using the same handle in different threads.  This would require
> thread-specific context and some locking.
>
> Regardless, in my own experience, libtiff strip/tile reads are quite fast
> and the bottleneck is normally in the using code.  Using a semaphore to
> protect tiff handle use would allow multiple threads to use libtiff to
> obtain strips/tiles and threads can be dealing with data returned while one
> thread is getting the next strip/tile.  This could result in a 2X to 4X
> speed-up.



Good idea. For my use case, multiple threads work just fine, so I am going
to stick with that right now.

Aaron

--bcaec53f38235dc08e050becd45a
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><br><div class=3D"gmail_extra"><br><div class=3D"gmail_quo=
te">On Mon, Jan 5, 2015 at 10:05 AM, Bob Friesenhahn <span dir=3D"ltr">&lt;=
<a href=3D"mailto:bfriesen@simple.dallas.tx.us" target=3D"_blank">bfriesen@=
simple.dallas.tx.us</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quo=
te" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"=
><span class=3D"">On Mon, 5 Jan 2015, Aaron Boxer wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
<br>
What would be interesting is to use multiple threads to read a single TIFF =
image. This would be particularly<br>
advantageous<br>
for BigTIFF.=C2=A0 My library is using C++11, so threading support is baked=
 in.=C2=A0 Its quite easy, keeping in mind<br>
that threads are evil.<br>
For libtiff, one would need to find a good cross platform C threading libra=
ry.<br>
<br>
For a single file, the approach could be similar: a thread safe list of str=
ips, and each thread gets the next<br>
strip and reads it in.<br>
<br>
Actually, this would be a perfect job for OpenMP.=C2=A0 And you wouldn&#39;=
t need a thread safe list, just divide up the<br>
strips evenly across threads.<br>
</blockquote>
<br></span>
Libtiff does not use any locking or use thread-specific contexts so I think=
 it is definitely not safe for using code to request multiple strips from t=
he same image at a time with the same tiff handle using multiple threads.=
=C2=A0 It should be mostly safe to read multiple images at one time (differ=
ent tiff handles) using multiple threads.=C2=A0 It should be safe to open t=
he same input TIFF file several times so that there is a different handle f=
or each reading thread.=C2=A0 The threads would then need to agree on which=
 strips/tiles they would read.<br>
<br>
It would be good for libtiff to support multiple strip/tile reads at a time=
 using the same handle in different threads.=C2=A0 This would require threa=
d-specific context and some locking.<br>
<br>
Regardless, in my own experience, libtiff strip/tile reads are quite fast a=
nd the bottleneck is normally in the using code.=C2=A0 Using a semaphore to=
 protect tiff handle use would allow multiple threads to use libtiff to obt=
ain strips/tiles and threads can be dealing with data returned while one th=
read is getting the next strip/tile.=C2=A0 This could result in a 2X to 4X =
speed-up.</blockquote><div><br><br></div><div>Good idea. For my use case, m=
ultiple threads work just fine, so I am going to stick with that right now.=
<br><br>Aaron<br></div><div><br></div></div><br></div></div>

--bcaec53f38235dc08e050becd45a--

--===============0036576085==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Tiff mailing list: Tiff@lists.maptools.org
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
--===============0036576085==--
