Return-Path: <awaresystems.be+caf_=filtered=awaresystems.be@gmail.com>
Delivered-To: filtered@awaresystems.be
Received: from mx2.pub.mailpod2-cph3.one.com ([10.27.25.12])
	by mailstorage18.cst.mailpod2-cph3.one.com (Dovecot) with LMTP id vESOKKW9O1rbswAAQS8vrQ
	for <filtered@awaresystems.be>; Thu, 21 Dec 2017 14:16:07 +0000
X-HalOne-ID: 7bda28e3-e659-11e7-b102-ec0d9a6ed9fe
Received: from mail-ot0-f172.google.com (unknown [74.125.82.172])
	by mx2.pub.mailpod2-cph3.one.com (Halon) with ESMTPS
	id 7bda28e3-e659-11e7-b102-ec0d9a6ed9fe;
	Thu, 21 Dec 2017 14:16:07 +0000 (UTC)
Received: by mail-ot0-f172.google.com with SMTP id v21so22549206oth.6
        for <filtered@awaresystems.be>; Thu, 21 Dec 2017 06:16:07 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-original-authentication-results:x-gm-message-state:delivered-to
         :from:to:date:message-id:user-agent:mime-version:subject:precedence
         :list-id:list-unsubscribe:list-archive:list-post:list-help
         :list-subscribe:mime-version:sender:errors-to;
        bh=i9bzyM7Qkima1eME7mWAMIVYkpoRmTtVBHXefGRD8oo=;
        b=HhZw6B3U0VoASu4qE8YiTUZl5BHUpfxJlpbAfP2r3mPR7NfFn0OU3KT8Mw3LLbZg+f
         FMvFe56tP8oymuT+2AfxxFkyc7kpUj63P2Jko0c5qU27+NaRahC5CPl/y04jAQbKota0
         NJornOaIgpY4kd2rojgxrkQAtQQ1n1uI7/hQnLzFCOlq1KgHCgaRcI/I66m5ZOWJBFn0
         IG2NPelrzEkjbOK12Plackrt8Ensb3DrepvqaEGr1/4QNDSlsVGqfPRu2YdMyHUK8Ard
         zNK7OiWLoAU5zxBZfBKRd7lOs/ef9ZCezao2yQV4g+mRaeZ/Etun8a8ebAdg4MxkoyGA
         JwzQ==
X-Original-Authentication-Results: mx.google.com;       spf=neutral (google.com: 94.23.4.142 is neither permitted nor denied by best guess record for domain of tiff-bounces@lists.maptools.org) smtp.mailfrom=tiff-bounces@lists.maptools.org
X-Gm-Message-State: AKGB3mLj3u6WFLFuffJeV834uWZu6VgfqYS4LQPTHbbiPYcnKYbQQY/F
	xnUuD3WMOZIXVc/w3Ojl6A52zxw8LPnPID28XHtgJNi0bhQ0mJU=
X-Received: by 10.157.62.181 with SMTP id b50mr8864508otc.160.1513865765801;
        Thu, 21 Dec 2017 06:16:05 -0800 (PST)
X-Forwarded-To: filtered@awaresystems.be
X-Forwarded-For: awaresystems.be@gmail.com filtered@awaresystems.be
Delivered-To: awaresystems.be@gmail.com
Received: by 10.202.55.68 with SMTP id e65csp1067100oia;
        Thu, 21 Dec 2017 06:16:04 -0800 (PST)
X-Google-Smtp-Source: ACJfBosULTxUsznbTDX/t/z+zvpcY2FCvWlgjgU5lOd9dzcObHmZi5Kbw06IiyOpZFiOcjB8+J7B
X-Received: by 10.223.179.77 with SMTP id k13mr12361680wrd.116.1513865764212;
        Thu, 21 Dec 2017 06:16:04 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; t=1513865764; cv=none;
        d=google.com; s=arc-20160816;
        b=b96qtrXOr9LVFpmX51a6KkOOoRGcxIFZ4sllNBcRqIjYnZG12S08DBiYgWE4b4/l73
         fXjNhtQSeWoL4tHFuoPMUJ8zjMbA+ulPJNL8yMglRX2yjP75UMk1nbiMkmf6napDOfK4
         SV8q4IoSLFAZoGyCt1C1PmkNkWzUh806EHqpb57Hcu9rxy+BO3J+2jjIJhGogkKhiWj7
         V1FrsXAl+bq3btiNArjOb1vdNcv8KT2m+UogK4mdGaC71jj3ZeB0bAHwNVrJc9xy5zxu
         9z3LloP/yw48D8mrGiJLAiik9ScGQ8vTHmmkD4luY49frnjGDEfym7/yEdUr+N9lq31K
         7F1g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=errors-to:sender:mime-version:list-subscribe:list-help:list-post
         :list-archive:list-unsubscribe:list-id:precedence:subject
         :mime-version:user-agent:message-id:date:to:from
         :arc-authentication-results;
        bh=i9bzyM7Qkima1eME7mWAMIVYkpoRmTtVBHXefGRD8oo=;
        b=a8UsdDUu9j7GN0Owwn5d1g6dUaplsW16gIfxofwQyMQzibe3k8TOjqsRLeoN55m2z8
         uu4rVGo6OosKxERm+17DCMe2njm6zkEbrf9qW2pLp9m8lSDfl9jBhndzxA12Vp6I72VD
         7nMHL6olEofnHKkWmCdzogpekH+lcjY0OBFZl32rGbBD05DylKIU1t/mVGiRW9XW01Qw
         U+JUv/If7x0JCTDJOTPv765yalEbeyIEOh0V1DPOJUi3KPbArbG5Ljrk6fvp+CUHRvAr
         gHk3haE6O/A/Sru9mg+8L3kMBLcEePjjTxbleBtuY2lOcWuGdrFOKp+Mx+Mj064oY9CM
         khxw==
ARC-Authentication-Results: i=1; mx.google.com;
       spf=neutral (google.com: 94.23.4.142 is neither permitted nor denied by best guess record for domain of tiff-bounces@lists.maptools.org) smtp.mailfrom=tiff-bounces@lists.maptools.org
Return-Path: <tiff-bounces@lists.maptools.org>
Received: from lists.maptools.org (ks365428.kimsufi.com. [94.23.4.142])
        by mx.google.com with ESMTP id d8si15353608wrf.147.2017.12.21.06.15.59;
        Thu, 21 Dec 2017 06:16:04 -0800 (PST)
Received-SPF: neutral (google.com: 94.23.4.142 is neither permitted nor denied by best guess record for domain of tiff-bounces@lists.maptools.org) client-ip=94.23.4.142;
Authentication-Results: mx.google.com;
       spf=neutral (google.com: 94.23.4.142 is neither permitted nor denied by best guess record for domain of tiff-bounces@lists.maptools.org) smtp.mailfrom=tiff-bounces@lists.maptools.org
Received: from [10.1.0.241] (localhost.localdomain [127.0.0.1])
	by lists.maptools.org (Postfix) with ESMTP id DCAF135C06A;
	Thu, 21 Dec 2017 09:14:51 -0500 (EST)
Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net
	[217.70.183.197])
	by lists.maptools.org (Postfix) with ESMTPS id 8293C35C067
	for <tiff@lists.maptools.org>; Thu, 21 Dec 2017 09:14:18 -0500 (EST)
X-Originating-IP: 88.162.84.61
Received: from even-i700.localnet (mla78-3-88-162-84-61.fbx.proxad.net
	[88.162.84.61]) (Authenticated sender: test@spatialys.com)
	by relay5-d.mail.gandi.net (Postfix) with ESMTPSA id 9267941C096
	for <tiff@lists.maptools.org>; Thu, 21 Dec 2017 15:14:17 +0100 (CET)
From: Even Rouault <even.rouault@spatialys.com>
To: tiff@lists.maptools.org
Date: Thu, 21 Dec 2017 15:14:16 +0100
Message-ID: <4495774.669O84oDJL@even-i700>
User-Agent: KMail/5.1.3 (Linux/4.4.0-98-generic; KDE/5.18.0; x86_64; ; )
MIME-Version: 1.0
Subject: [Tiff] Pull request to add ZStandard/ZSTD codec
X-BeenThere: tiff@lists.maptools.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: TIFF and libtiff discussion <tiff.lists.maptools.org>
List-Unsubscribe: <http://lists.maptools.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@lists.maptools.org?subject=unsubscribe>
List-Archive: <http://lists.maptools.org/mailman/private/tiff>
List-Post: <mailto:tiff@lists.maptools.org>
List-Help: <mailto:tiff-request@lists.maptools.org?subject=help>
List-Subscribe: <http://lists.maptools.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@lists.maptools.org?subject=subscribe>
Content-Type: multipart/mixed; boundary="===============1845854694=="
Mime-version: 1.0
Sender: tiff-bounces@lists.maptools.org
Errors-To: tiff-bounces@lists.maptools.org

This is a multi-part message in MIME format.

--===============1845854694==
Content-Type: multipart/alternative; boundary="nextPart3678023.jIE7b4rMOg"
Content-Transfer-Encoding: 7Bit

This is a multi-part message in MIME format.

--nextPart3678023.jIE7b4rMOg
Content-Transfer-Encoding: 7Bit
Content-Type: text/plain; charset="us-ascii"

Hi,

I've submitted a PR to  add ZStandard/ZSTD codec:
https://gitlab.com/libtiff/libtiff/merge_requests/9

"""
From https://github.com/facebook/zstd
"Zstandard, or zstd as short version, is a fast lossless compression
algorithm, targeting real-time compression scenarios at zlib-level
and better compression ratios. It's backed by a very fast entropy stage,
provided by Huff0 and FSE library."

We require libzstd >= 1.0.0 so as to be able to use streaming compression
and decompression methods.

The default compression level we have selected is 9 (range goes from 1 to 22),
which experimentally offers equivalent or better compression ratio than
the default deflate/ZIP level of 6, and much faster compression.

For example on a 6600x4400 16bit image, tiffcp -c zip runs in 10.7 seconds,
while tiffcp -c zstd runs in 5.3 seconds. Decompression time for zip is
840 ms, and for zstd 650 ms. File size is 42735936 for zip, and
42586822 for zstd. Similar findings on other images.

On a 25894x16701 16bit image,

                Compression time     Decompression time     File size

ZSTD                 35 s                       3.2 s            399 700 498
ZIP/Deflate       1m 20 s                   4.9 s           419 622 336
""""


My main question is who manages the allocation of TIFF tags/values ?

I've auto-allocated:
#define     COMPRESSION_ZSTD            34926   /* ZSTD */

Even
-- 
Spatialys - Geospatial professional services
http://www.spatialys.com

--nextPart3678023.jIE7b4rMOg
Content-Transfer-Encoding: 7Bit
Content-Type: text/html; charset="us-ascii"

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN" "http://www.w3.org/TR/REC-html40/strict.dtd">
<html><head><meta name="qrichtext" content="1" /><style type="text/css">
p, li { white-space: pre-wrap; }
</style></head><body style=" font-family:'Sans Serif'; font-size:9pt; font-weight:400; font-style:normal;">
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Hi,</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; ">&nbsp;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">I've submitted a PR to  add ZStandard/ZSTD codec:</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">https://gitlab.com/libtiff/libtiff/merge_requests/9</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; ">&nbsp;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&quot;&quot;&quot;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">From https://github.com/facebook/zstd</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&quot;Zstandard, or zstd as short version, is a fast lossless compression</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">algorithm, targeting real-time compression scenarios at zlib-level</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">and better compression ratios. It's backed by a very fast entropy stage,</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">provided by Huff0 and FSE library.&quot;</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; ">&nbsp;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">We require libzstd &gt;= 1.0.0 so as to be able to use streaming compression</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">and decompression methods.</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; ">&nbsp;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">The default compression level we have selected is 9 (range goes from 1 to 22),</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">which experimentally offers equivalent or better compression ratio than</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">the default deflate/ZIP level of 6, and much faster compression.</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; ">&nbsp;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">For example on a 6600x4400 16bit image, tiffcp -c zip runs in 10.7 seconds,</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">while tiffcp -c zstd runs in 5.3 seconds. Decompression time for zip is</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">840 ms, and for zstd 650 ms. File size is 42735936 for zip, and</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">42586822 for zstd. Similar findings on other images.</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; ">&nbsp;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">On a 25894x16701 16bit image,</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; ">&nbsp;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">                Compression time     Decompression time     File size</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; ">&nbsp;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">ZSTD                 35 s                       3.2 s            399 700 498</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">ZIP/Deflate       1m 20 s                   4.9 s           419 622 336</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&quot;&quot;&quot;&quot;</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; ">&nbsp;</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; ">&nbsp;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">My main question is who manages the allocation of TIFF tags/values ?</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; ">&nbsp;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">I've auto-allocated:</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">#define     COMPRESSION_ZSTD            34926   /* ZSTD */</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; ">&nbsp;</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Even</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">-- </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Spatialys - Geospatial professional services</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">http://www.spatialys.com</p></body></html>
--nextPart3678023.jIE7b4rMOg--


--===============1845854694==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Tiff mailing list: Tiff@lists.maptools.org
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
--===============1845854694==--
