Return-Path: <tiff-bounces@lists.osgeo.org>
Delivered-To: filtered@awaresystems.be
Received: from mx3.pub.mailpod8-cph3.one.com ([10.27.38.13])
	by mailstorage22.cst.mailpod8-cph3.one.com with LMTP id EH8bAK8Fhl/jewAAwT5NuQ
	for <filtered@awaresystems.be>; Tue, 13 Oct 2020 19:53:19 +0000
X-HalOne-ID: bcb384db-0d8d-11eb-8ec6-98039b062cb2
Received: from lists.osgeo.org (osgeo6.osgeo.osuosl.org [140.211.15.3])
	by mx3.pub.mailpod8-cph3.one.com (Halon) with ESMTPS
	id bcb384db-0d8d-11eb-8ec6-98039b062cb2;
	Tue, 13 Oct 2020 19:53:18 +0000 (UTC)
Received: from osgeo6.osgeo.osuosl.org (localhost [127.0.0.1])
	by lists.osgeo.org (Postfix) with ESMTP id 98DDB61346D9;
	Tue, 13 Oct 2020 12:53:15 -0700 (PDT)
Received: from mail-ed1-f46.google.com (mail-ed1-f46.google.com
 [209.85.208.46])
 by lists.osgeo.org (Postfix) with ESMTPS id 3CFBF61346C2
 for <tiff@lists.osgeo.org>; Tue, 13 Oct 2020 12:53:14 -0700 (PDT)
Received: by mail-ed1-f46.google.com with SMTP id o18so745336edq.4
 for <tiff@lists.osgeo.org>; Tue, 13 Oct 2020 12:53:14 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=objectivepathology.com; s=google;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=z7WrlThpq9puNiETDpi+7uHeJCP62XLCEPCuFycTDx4=;
 b=haQxxxRCAMrqpY7OilJB3AlkPLTS2ntDCg7Udb39VYqVuwu1ZIclTR2B45DjS7IAz8
 ZbbriUIK92vqH+My/BVYhpSnkWrN3Ak287ErOUAooWokyYcCOJnX5gfJQm9okikzwv5J
 2aKDoRnT64KTfQk83O2wNXA87lrAaquDDqMHs3R0fGB8/ClkKsOvUeOTaM943cJvkNkC
 Uw5bpISNDy0vZzlFoEjxMsKBntJn09atGbOur+VMv4ieWHt0ep0VNcpa6dlOH7Eyru86
 274plgvbmEZdqXAnkqTJxxE6CfjXSVGKMl7QMUTP/fWQ7+IALwBgGxFb5KH2MDgKXzOE
 873A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=z7WrlThpq9puNiETDpi+7uHeJCP62XLCEPCuFycTDx4=;
 b=tnd2OoLpmr7a51OzZBEZA0JTyR8iCvMN1W+9wNqUeFlVfM1m+l0RaUobdKFpyTO/jY
 w248e6G1JSgr4hrnFQzoc7fJ7E4US0ryulJRfTxzhi42EmA/dfasgaZk5E7NqUMi/gHD
 d8tfbhfWhia8RHG7ncJtWD8/jnFqV40GT1Kx3K0kvpypLDJIUj2K7b9uU4I8gjc1Ek1y
 /3ekZEve6lkEhC5mWjjPP5cs8SbS4xlzlKkiIrVlHwCYTIXrjFHTlJadtDswRVqWeHxY
 tsV1/lrKJ410EmiqcnYONbQ3fkI+tqr4Xiw5E7LiXSePmZvFp98x9NifmCNGuWREwkSC
 PdIw==
X-Gm-Message-State: AOAM53278u6MXttNDFPogJFxxI4kIDDYJKnRJYnqypyTcpE+mpSCuues
 RUfqQDreNU6LVbMZAv8RrezXJ3q9nP49jGwD6zmZYXbkfENd7Q==
X-Google-Smtp-Source: ABdhPJxS1Ei7v5ab394UQzD9g0ZPsCkz+Hm1gvqFtS8ZQzxMKmE+3etToA7jIP9R5PZ3Sgg0IhRSPUg/AA9APDfFFCw=
X-Received: by 2002:aa7:ce13:: with SMTP id d19mr1362637edv.68.1602618792853; 
 Tue, 13 Oct 2020 12:53:12 -0700 (PDT)
MIME-Version: 1.0
References: <1639815.KyIUctkiDL@even-i700>
 <C85F5EA2-6C8C-4220-B9DF-84417E0CAD51@codelibre.net>
 <2450040.rK9MszJcyc@even-i700>
In-Reply-To: <2450040.rK9MszJcyc@even-i700>
From: Kemp Watson <kemp@objectivepathology.com>
Date: Tue, 13 Oct 2020 15:52:53 -0400
Message-ID: <CACp925RMM7hzEbtwUC6UhHoTQD9xjN750Ug+rK70qhtm5bdMiA@mail.gmail.com>
To: Even Rouault <even.rouault@spatialys.com>
Subject: Re: [Tiff] faster Zip/Deflate compression/decompression with
	libdeflate
X-BeenThere: tiff@lists.osgeo.org
X-Mailman-Version: 2.1.20
Precedence: list
List-Id: TIFF and libtiff discussion <tiff.lists.osgeo.org>
List-Unsubscribe: <https://lists.osgeo.org/mailman/options/tiff>,
 <mailto:tiff-request@lists.osgeo.org?subject=unsubscribe>
List-Post: <mailto:tiff@lists.osgeo.org>
List-Help: <mailto:tiff-request@lists.osgeo.org?subject=help>
List-Subscribe: <https://lists.osgeo.org/mailman/listinfo/tiff>,
 <mailto:tiff-request@lists.osgeo.org?subject=subscribe>
Cc: Tiff List <tiff@lists.osgeo.org>
Content-Type: multipart/mixed; boundary="===============3660427091098335958=="
Errors-To: tiff-bounces@lists.osgeo.org
Sender: "Tiff" <tiff-bounces@lists.osgeo.org>

--===============3660427091098335958==
Content-Type: multipart/alternative; boundary="000000000000dbd97b05b192c300"

--000000000000dbd97b05b192c300
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Have we ever received an answer from Adobe, re whether the tag registry is

- lost without backup
- abandoned
- deliberately taken down?

Given that they are the keeper of the spec, this remains quite concerning.

Kemp

On Tue, Oct 13, 2020 at 3:49 PM Even Rouault <even.rouault@spatialys.com>
wrote:

> On mardi 13 octobre 2020 20:25:41 CEST Roger Leigh wrote:
>
> > > Although we have better alternative than Deflate now with ZSTD, it is
>
> > > still heavily used as being far more supported than ZSTD or other
>
> > > alternatives. As an example of this, very recently, in the geospatial
>
> > > domain, Sentinel2 L2A scenes have been reprocessed as
> Deflate-compressed
>
> > > TIFF and published in
> https://registry.opendata.aws/sentinel-2-l2a-cogs/
>
> > > <https://registry.opendata.aws/sentinel-2-l2a-cogs/> . This
> represents a
>
> > > volume of 200 million TIFF files for 10.5 petabytes.
>
>
>
> > The libdeflate work looks really great. I=E2=80=99ll try to find time t=
o review
> it
>
> > in more detail tomorrow.
>
>
>
> Thanks
>
> > One thought: GitLab supports Git LFS for storing =E2=80=9Clarge files=
=E2=80=9D. It might
> we
>
> > worth using it for the TIFF sample data; same would apply to the other
>
> > image data already in the repository. Doesn=E2=80=99t need doing for th=
is merge
>
> > request, but might be worth considering migrating over to. It would
> permit
>
> > adding more test data to the repository in the future as well without
>
> > affecting repository clone time (but it would download the
> branch-specific
>
> > large file content on demand when you check out a branch, if you didn=
=E2=80=99t
>
> > already pull it down explicitly).
>
>
>
> Are you talking about the content in test/images/ ? It is just 1 MB large=
.
>
> So I don't think we have a critical need for that feature for now. We
> don't necessarily need big images to test the various aspects of the code=
,
> and the smaller the better for "make check" purposes too.
>
> (I've been ennoyed on the github side of things with LFS where there's a
> monthly bandwidth, that includes people forking the repo... and beyond th=
at
> checkout no longer works and you've to buy credits. Perhaps things are
> different on gitlab.com)
>
>
>
> > Regarding ZSTD, what=E2=80=99s the current situation with regard to the=
 Adobe tag
>
> > registry? Do we have a number for ZSTD yet?
>
>
>
> We're still living with the number we have attributed to ourselves. And a=
t
> that point, it would be a pain to have to change it.
>
>
>
> Even
>
>
>
> --
>
> Spatialys - Geospatial professional services
>
> http://www.spatialys.com
> _______________________________________________
> Tiff mailing list
> Tiff@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/tiff
>

--000000000000dbd97b05b192c300
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Have we ever received an answer from Adobe, re whether the=
 tag registry is<div><br><div>- lost without backup</div><div>- abandoned</=
div><div>-=C2=A0deliberately taken down?</div><div><br></div><div>Given tha=
t they are the keeper of the spec, this remains quite concerning.</div><div=
><br></div><div>Kemp</div></div></div><br><div class=3D"gmail_quote"><div d=
ir=3D"ltr" class=3D"gmail_attr">On Tue, Oct 13, 2020 at 3:49 PM Even Rouaul=
t &lt;<a href=3D"mailto:even.rouault@spatialys.com">even.rouault@spatialys.=
com</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"marg=
in:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1e=
x"><u></u>
<div style=3D"font-family:monospace;font-size:9pt;font-weight:400;font-styl=
e:normal">
<p style=3D"margin:0px;text-indent:0px">On mardi 13 octobre 2020 20:25:41 C=
EST Roger Leigh wrote:</p>
<p style=3D"margin:0px;text-indent:0px">&gt; &gt; Although we have better a=
lternative than Deflate now with ZSTD, it is</p>
<p style=3D"margin:0px;text-indent:0px">&gt; &gt; still heavily used as bei=
ng far more supported than ZSTD or other</p>
<p style=3D"margin:0px;text-indent:0px">&gt; &gt; alternatives. As an examp=
le of this, very recently, in the geospatial</p>
<p style=3D"margin:0px;text-indent:0px">&gt; &gt; domain, Sentinel2 L2A sce=
nes have been reprocessed as Deflate-compressed</p>
<p style=3D"margin:0px;text-indent:0px">&gt; &gt; TIFF and published in <a =
href=3D"https://registry.opendata.aws/sentinel-2-l2a-cogs/" target=3D"_blan=
k">https://registry.opendata.aws/sentinel-2-l2a-cogs/</a></p>
<p style=3D"margin:0px;text-indent:0px">&gt; &gt; &lt;<a href=3D"https://re=
gistry.opendata.aws/sentinel-2-l2a-cogs/" target=3D"_blank">https://registr=
y.opendata.aws/sentinel-2-l2a-cogs/</a>&gt; . This represents a</p>
<p style=3D"margin:0px;text-indent:0px">&gt; &gt; volume of 200 million TIF=
F files for 10.5 petabytes.</p>
<p style=3D"margin:0px;text-indent:0px">=C2=A0</p>
<p style=3D"margin:0px;text-indent:0px">&gt; The libdeflate work looks real=
ly great.  I=E2=80=99ll try to find time to review it</p>
<p style=3D"margin:0px;text-indent:0px">&gt; in more detail tomorrow.</p>
<p style=3D"margin:0px;text-indent:0px">=C2=A0</p>
<p style=3D"margin:0px;text-indent:0px">Thanks</p>
<p style=3D"margin:0px;text-indent:0px"> </p>
<p style=3D"margin:0px;text-indent:0px">&gt; One thought: GitLab supports G=
it LFS for storing =E2=80=9Clarge files=E2=80=9D.  It might we</p>
<p style=3D"margin:0px;text-indent:0px">&gt; worth using it for the TIFF sa=
mple data; same would apply to the other</p>
<p style=3D"margin:0px;text-indent:0px">&gt; image data already in the repo=
sitory.  Doesn=E2=80=99t need doing for this merge</p>
<p style=3D"margin:0px;text-indent:0px">&gt; request, but might be worth co=
nsidering migrating over to.  It would permit</p>
<p style=3D"margin:0px;text-indent:0px">&gt; adding more test data to the r=
epository in the future as well without</p>
<p style=3D"margin:0px;text-indent:0px">&gt; affecting repository clone tim=
e (but it would download the branch-specific</p>
<p style=3D"margin:0px;text-indent:0px">&gt; large file content on demand w=
hen you check out a branch, if you didn=E2=80=99t</p>
<p style=3D"margin:0px;text-indent:0px">&gt; already pull it down explicitl=
y).</p>
<p style=3D"margin:0px;text-indent:0px">=C2=A0</p>
<p style=3D"margin:0px;text-indent:0px">Are you talking about the content i=
n test/images/ ? It is just 1 MB large.</p>
<p style=3D"margin:0px;text-indent:0px">So I don&#39;t think we have a crit=
ical need for that feature for now. We don&#39;t necessarily need big image=
s to test the various aspects of the code, and the smaller the better for &=
quot;make check&quot; purposes too.</p>
<p style=3D"margin:0px;text-indent:0px">(I&#39;ve been ennoyed on the githu=
b side of things with LFS where there&#39;s a monthly bandwidth, that inclu=
des people forking the repo... and beyond that checkout no longer works and=
 you&#39;ve to buy credits. Perhaps things are different on <a href=3D"http=
://gitlab.com" target=3D"_blank">gitlab.com</a>)</p>
<p style=3D"margin:0px;text-indent:0px">=C2=A0</p>
<p style=3D"margin:0px;text-indent:0px">&gt; Regarding ZSTD, what=E2=80=99s=
 the current situation with regard to the Adobe tag</p>
<p style=3D"margin:0px;text-indent:0px">&gt; registry?  Do we have a number=
 for ZSTD yet?</p>
<p style=3D"margin:0px;text-indent:0px">=C2=A0</p>
<p style=3D"margin:0px;text-indent:0px">We&#39;re still living with the num=
ber we have attributed to ourselves. And at that point, it would be a pain =
to have to change it.</p>
<p style=3D"margin:0px;text-indent:0px">=C2=A0</p>
<p style=3D"margin:0px;text-indent:0px">Even</p>
<p style=3D"margin:0px;text-indent:0px">=C2=A0</p>
<p style=3D"margin:0px;text-indent:0px">-- </p>
<p style=3D"margin:0px;text-indent:0px">Spatialys - Geospatial professional=
 services</p>
<p style=3D"margin:0px;text-indent:0px"><a href=3D"http://www.spatialys.com=
" target=3D"_blank">http://www.spatialys.com</a></p></div>_________________=
______________________________<br>
Tiff mailing list<br>
<a href=3D"mailto:Tiff@lists.osgeo.org" target=3D"_blank">Tiff@lists.osgeo.=
org</a><br>
<a href=3D"https://lists.osgeo.org/mailman/listinfo/tiff" rel=3D"noreferrer=
" target=3D"_blank">https://lists.osgeo.org/mailman/listinfo/tiff</a><br>
</blockquote></div>

--000000000000dbd97b05b192c300--

--===============3660427091098335958==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: inline

X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KVGlmZiBtYWls
aW5nIGxpc3QKVGlmZkBsaXN0cy5vc2dlby5vcmcKaHR0cHM6Ly9saXN0cy5vc2dlby5vcmcvbWFp
bG1hbi9saXN0aW5mby90aWZmCg==

--===============3660427091098335958==--
