Return-Path: <tiff-admin@remotesensing.org>
Received: from jay.skynet.be (jay.skynet.be [195.238.3.234])
        by twentyeight.skynet.be (8.12.9/8.12.9/Skynet-MAILSTORE-2.14) with ESMTP id hBGNmVPI006224
        for <joris.at.lebbeke@skynet.be>; Wed, 17 Dec 2003 00:48:31 +0100
        (envelope-from <tiff-admin@remotesensing.org>)
Received: from yoda.imagelinks.com ([208.24.120.42])
        by jay.skynet.be (8.12.9/8.12.9/Skynet-IN-2.32) with ESMTP id hBGNlLfK031924
        for <joris.at.lebbeke@skynet.be>; Wed, 17 Dec 2003 00:47:22 +0100
        (envelope-from <tiff-admin@remotesensing.org>)
Received: from yoda.imagelinks.com (localhost.localdomain [127.0.0.1])
	by yoda.imagelinks.com (8.11.6/8.11.2) with ESMTP id hBGNhOl29497;
	Tue, 16 Dec 2003 18:43:24 -0500
Received: from sarek.skynet.be (sarek.skynet.be [195.238.3.230])
	by yoda.imagelinks.com (8.11.6/8.11.2) with ESMTP id hBGNgfl29473
	for <tiff@remotesensing.org>; Tue, 16 Dec 2003 18:42:42 -0500
Received: from dfgrrxsknr4e (162.162-201-80.adsl.skynet.be [80.201.162.162])
        by sarek.skynet.be (8.12.9/8.12.9/Skynet-OUT-2.21) with SMTP id hBGNgd1u000429
        for <tiff@remotesensing.org>; Wed, 17 Dec 2003 00:42:39 +0100
        (envelope-from <joris.at.lebbeke@skynet.be>)
Message-ID: <002601c3c42e$df35f1a0$a2a2c950@dfgrrxsknr4e>
From: "Joris" <joris.at.lebbeke@skynet.be>
To: "Tiff mailing list" <tiff@remotesensing.org>
References: <000501c3c189$d2804d90$2022c850@dfgrrxsknr4e> <3FDB3D9C.1030308@pobox.com> <4.2.0.58.20031213162904.00bc9c60@mail.cerious.com> <4.2.0.58.20031216075535.017f13f0@mail.cerious.com> <000d01c3c3db$26d56800$fb2cc850@dfgrrxsknr4e> <3FDF1668.9020909@pobox.com> <000b01c3c3ea$14badce0$a2a2c950@dfgrrxsknr4e> <3FDF599E.3070509@pobox.com>
Subject: Re: [Tiff] Stupid question
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2800.1158
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
X-RAVMilter-Version: 8.4.3(snapshot 20030212) (sarek.skynet.be)
Sender: tiff-admin@remotesensing.org
Errors-To: tiff-admin@remotesensing.org
X-BeenThere: tiff@remotesensing.org
X-Mailman-Version: 2.0.11
Precedence: bulk
List-Help: <mailto:tiff-request@remotesensing.org?subject=help>
List-Post: <mailto:tiff@remotesensing.org>
List-Subscribe: <http://remotesensing.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@remotesensing.org?subject=subscribe>
List-Id: TIFF and libtiff discussion <tiff.remotesensing.org>
List-Unsubscribe: <http://remotesensing.org/mailman/listinfo/tiff>,
	<mailto:tiff-request@remotesensing.org?subject=unsubscribe>
List-Archive: <http://remotesensing.org/pipermail/tiff/>
Date: Wed, 17 Dec 2003 00:46:47 +0100
X-UIDL: 6348d64f67111c22024dd0b8ffebbd18

> No pressure.  Frankly if you could assemble the email, clean out the spam
> and make it available for a normal threaded archive I think that would be
> great.

When text is concerned, I strongly believe in 'single-sourcing'. You know,
grabbing data, building to codes to convert it to a proprietary format that
is really cut out for the particular data, and next being able to build the
code to convert it into anything else. The main point of this
'single-sourcing' is that the actual content should not be format-specific,
meaning, since we live in a real world, should have a format that is exactly
fitted to the data and can be transformed into any other.

Having progressed a little more, I can see now that I best work in two
stages. Indeed, there should be no regrouping or categorizing in this first
stage. Therefore, the result of the first stage is a normal threaded
archive. I'll be able to deliver mbox, I'm sure, as well as HTML pages, a
Word doc, pdf, or just about anything. I already build and am able to reuse
a limited HTML and PDF codec, and have experience in Word OLE, so it's just
this mbox, but that will be no problem at all I guess. I realize this talk
about 'proprietary format' in-between may not sound very average, but, trust
me, I did some similar stuff before. The only real reason for concern is the
magnitude of the project.

If you can run (and trust) a windows executable, I'll send you a viewer for
the proprietary format with the data I processed up to that point in a few
days. That executable will be able to build a plain text output, HTML pages
and a Word doc already. If not, I'll send you the HTML pages. That is, if
you like to receive such a preview, of course. The current processing
involves:
- extracting from the HTML pages (the aug 99 archive)
- processing of the headers to no longer include 'Company' and 'CC' and
'Reply-To' and such, if originally present
- processing of the 'From' field in the header to uniquelly identify a
sender, even if that sender changed his/hers e-mail address during these
many years
- converting the date indications to GMT
- filtering out test messages, mailing list software generated messages, and
spam
- converting the occasional HTML message to plain text

Perhaps I'll add some de-word-wrapping code, so as to end up with formatable
text, as opposed to pre-formated, but I haven't given that a lot of thought
yet.

All of these actions are done automatically, for the most part, by my
temporary dirty code. But I still review each processed message by hand, to
either give my consent or improve on the code, and I plan to continue doing
that... Which means that this will take some time...

> Even better if you can extract a few FAQs with answers.

In a second stage, I will still like to build a 'TIFF-sensitive knowledge
base' kinda thing, that is able to enumerate relevant pointers into the spec
and relevant mailing list messages from any TIFF page, and I like this idea
of a FAQ too. Categorizing the messages in the complete archive build in the
first stage will be the main issue here, I guess.

> I don't see any problem with your hosting a copy, or any reasonable use
you
> put the email archive to.  You have my complete approval.

Grand!



Joris

_______________________________________________
Tiff mailing list
Tiff@remotesensing.org
http://remotesensing.org/mailman/listinfo/tiff
