Comments on: PDF, The Waste Land, and Monica’s Blue Dress http://www.robweir.com/blog/2007/11/pdf-waste-land-and-monicas-blue-dress.html Thinking the unthinkable, pondering the imponderable, effing the ineffable and scruting the inscrutable Wed, 17 Mar 2010 16:12:36 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Anonymous http://www.robweir.com/blog/2007/11/pdf-waste-land-and-monicas-blue-dress.html#comment-1312 Anonymous Wed, 05 Dec 2007 23:14:00 +0000 http://www.robweir.com/blog/2007/11/pdf-the-waste-land-and-monicas-blue-dress.html#comment-1312 >> What do you mean it doesn't look like the document? <<<br/>Exactly what I said. Every now and again some objects are lost during transformation to pdf.<br/><br/>>> Rob's idea is to have the same program store a PDF of its own output. <<<br/>For one thing, <i>I was talking primary of James King's ideas</i>, not Rob's. Rob's approach is akin caching — all data are preserved, quick read-only pdf is available. Which is quite acceptable if one can afford space etc.<br/><br/>>> If those two weren't the same, it'd be a bug. But I don't see how it could. <<<br/>What?! You've never seen a bug? Lucky you! Will it make anybody happier to learn of data loss because of the bug?<br/><br/>How bug can happen:<br/><br/>* many things change. Some of them are external to the format. Like font engine. For example, Adobe Acrobat does carry its own. Is there any guaranty that three centuries in the future somebody will know all bugs and workarounds of closed-source software?<br/><br/>* printscreen is not 100% valid copy of the screen. Try to get one of media player. Anyway, it is quite impractical: plainly too big and not scalable.<br/><br/>* I have seen encoding bugs in the programs, when in some specific situation wrong translation was used. I've seen it happen during pdf conversion as well.<br/><br/>* image / graphics transformation might be lossy. It can be because of the underlying graphics model (pdf does not support arcs of circle, does it? but it is irrelevant: it does not support 5-th order splines), or because of color space. Or something else. This is acceptable limitation for pdf, after all it is <i>presentation</i> format. But I can easily imagine a bug in transformation algorithms, esp. connected to overflows (so it rarely happens). >> What do you mean it doesn’t look like the document? < <
Exactly what I said. Every now and again some objects are lost during transformation to pdf.

>> Rob’s idea is to have the same program store a PDF of its own output. < <
For one thing, I was talking primary of James King’s ideas, not Rob’s. Rob’s approach is akin caching — all data are preserved, quick read-only pdf is available. Which is quite acceptable if one can afford space etc.

>> If those two weren’t the same, it’d be a bug. But I don’t see how it could. < <
What?! You’ve never seen a bug? Lucky you! Will it make anybody happier to learn of data loss because of the bug?

How bug can happen:

* many things change. Some of them are external to the format. Like font engine. For example, Adobe Acrobat does carry its own. Is there any guaranty that three centuries in the future somebody will know all bugs and workarounds of closed-source software?

* printscreen is not 100% valid copy of the screen. Try to get one of media player. Anyway, it is quite impractical: plainly too big and not scalable.

* I have seen encoding bugs in the programs, when in some specific situation wrong translation was used. I’ve seen it happen during pdf conversion as well.

* image / graphics transformation might be lossy. It can be because of the underlying graphics model (pdf does not support arcs of circle, does it? but it is irrelevant: it does not support 5-th order splines), or because of color space. Or something else. This is acceptable limitation for pdf, after all it is presentation format. But I can easily imagine a bug in transformation algorithms, esp. connected to overflows (so it rarely happens).

]]> By: Anonymous http://www.robweir.com/blog/2007/11/pdf-waste-land-and-monicas-blue-dress.html#comment-1280 Anonymous Tue, 27 Nov 2007 00:08:00 +0000 http://www.robweir.com/blog/2007/11/pdf-the-waste-land-and-monicas-blue-dress.html#comment-1280 What do you mean it doesn't look like the document? Rob's idea is to have the same program store a PDF of its own output.<br/><br/>In other words, the same program would be embedding the PDF and displaying the data. If those two weren't the same, it'd be a bug. But I don't see how it could. After all, all it would have to do is a glorified printscreen of itself and stuff that into a PDF. What do you mean it doesn’t look like the document? Rob’s idea is to have the same program store a PDF of its own output.

In other words, the same program would be embedding the PDF and displaying the data. If those two weren’t the same, it’d be a bug. But I don’t see how it could. After all, all it would have to do is a glorified printscreen of itself and stuff that into a PDF.

]]> By: Anonymous http://www.robweir.com/blog/2007/11/pdf-waste-land-and-monicas-blue-dress.html#comment-1278 Anonymous Mon, 26 Nov 2007 15:56:00 +0000 http://www.robweir.com/blog/2007/11/pdf-the-waste-land-and-monicas-blue-dress.html#comment-1278 For me the idea to store PDF is just weird. Working with <i>data archiving</i> for process control I learned hard way that the only acceptable way to archive is to store original data. Nobody can guaranty fidelity of the transformation. You give many examples of data loss, but most general holds: there is no way to guaranty that PDF <i>even</i> looks exactly like what person had in document. For me the idea to store PDF is just weird. Working with data archiving for process control I learned hard way that the only acceptable way to archive is to store original data. Nobody can guaranty fidelity of the transformation. You give many examples of data loss, but most general holds: there is no way to guaranty that PDF even looks exactly like what person had in document.

]]> By: Thomas Downing http://www.robweir.com/blog/2007/11/pdf-waste-land-and-monicas-blue-dress.html#comment-1277 Thomas Downing Mon, 26 Nov 2007 12:30:00 +0000 http://www.robweir.com/blog/2007/11/pdf-the-waste-land-and-monicas-blue-dress.html#comment-1277 Although I pay the bills with technical endeavour, my personal bent is historical. This post resonates with strongly with me; further, I think it paints a strong picture of one of the strengths of ODF as a standard.<br/><br/>The idea of storing the 'as published' form of the document using PDF as a part of the larger 'document as work' is a crucial feature to ODF as an archival tool. Great suggestion! I hope it gets added to the ODF track soon. Although I pay the bills with technical endeavour, my personal bent is historical. This post resonates with strongly with me; further, I think it paints a strong picture of one of the strengths of ODF as a standard.

The idea of storing the ‘as published’ form of the document using PDF as a part of the larger ‘document as work’ is a crucial feature to ODF as an archival tool. Great suggestion! I hope it gets added to the ODF track soon.

]]> By: Lucas http://www.robweir.com/blog/2007/11/pdf-waste-land-and-monicas-blue-dress.html#comment-1276 Lucas Sat, 24 Nov 2007 07:20:00 +0000 http://www.robweir.com/blog/2007/11/pdf-the-waste-land-and-monicas-blue-dress.html#comment-1276 Not that the ODF vs OOXML is in question her, but could one not do the same thing with OOXML?<br/><br/>There are so many issues at work when it comes to "archiving", I can't imagine any file format solution that is clearly any better than others. For example, are there any content recovery "mechanisms" inherent in any file formats? Suppose a file is saved/archived incorrectly because of a software bug, how easily can the content by recovered?<br/><br/>How can scanned elements and the text documents be "archived" without keeping two copies? Legal documents with signatures may need to be preserved with hand-written signatures but keep the text of the document in original form so as to remain searchable. One might argue, one can simply scan the signature page while keeping the original text document. However, this introduces: 1) two files, 2) legal question about whether signature corresponds to the legal text. Not that the ODF vs OOXML is in question her, but could one not do the same thing with OOXML?

There are so many issues at work when it comes to “archiving”, I can’t imagine any file format solution that is clearly any better than others. For example, are there any content recovery “mechanisms” inherent in any file formats? Suppose a file is saved/archived incorrectly because of a software bug, how easily can the content by recovered?

How can scanned elements and the text documents be “archived” without keeping two copies? Legal documents with signatures may need to be preserved with hand-written signatures but keep the text of the document in original form so as to remain searchable. One might argue, one can simply scan the signature page while keeping the original text document. However, this introduces: 1) two files, 2) legal question about whether signature corresponds to the legal text.

]]> By: Uri http://www.robweir.com/blog/2007/11/pdf-waste-land-and-monicas-blue-dress.html#comment-1275 Uri Thu, 22 Nov 2007 20:58:00 +0000 http://www.robweir.com/blog/2007/11/pdf-the-waste-land-and-monicas-blue-dress.html#comment-1275 I can't really see an argument with two sides here. Using PDF exclusively for archiving sound to me like a cooking book with only pictures of the finished dishes and no recipes, or an exhibition of photocopies of paintings. It's substituting a facsimile copy of a thing with the thing itself.<br/>If I need archived financial data, I expect it to be in a format that can be used for financial calculations. Archived computer source-code should be readable to a compiler. If I need a graph from an academic paper, I'd want to extract the full-resolution one from the original document, not the miniaturized version created for printing. A printed version of data is almost always useless compared to the data in its original format and container.<br/>No one is seriously suggesting that graphic design studios start to archive their Photoshop projects as JPEG images. Why should office documents be treated differently? I can’t really see an argument with two sides here. Using PDF exclusively for archiving sound to me like a cooking book with only pictures of the finished dishes and no recipes, or an exhibition of photocopies of paintings. It’s substituting a facsimile copy of a thing with the thing itself.
If I need archived financial data, I expect it to be in a format that can be used for financial calculations. Archived computer source-code should be readable to a compiler. If I need a graph from an academic paper, I’d want to extract the full-resolution one from the original document, not the miniaturized version created for printing. A printed version of data is almost always useless compared to the data in its original format and container.
No one is seriously suggesting that graphic design studios start to archive their Photoshop projects as JPEG images. Why should office documents be treated differently?

]]> By: Rob http://www.robweir.com/blog/2007/11/pdf-waste-land-and-monicas-blue-dress.html#comment-1274 Rob Thu, 22 Nov 2007 15:39:00 +0000 http://www.robweir.com/blog/2007/11/pdf-the-waste-land-and-monicas-blue-dress.html#comment-1274 The idea would be that your "original original" would be a compound document that had both ODF and ODF markups. <br/><br/>Also, the interest here could be beyond archiving. It would be a document that could be viewed anywhere (presumption is PDF readers are free and ubiquitous)with perfect fidelity, as well as edited anywhere (assumption that ODF editors are free and available everywhere.) The idea would be that your “original original” would be a compound document that had both ODF and ODF markups.

Also, the interest here could be beyond archiving. It would be a document that could be viewed anywhere (presumption is PDF readers are free and ubiquitous)with perfect fidelity, as well as edited anywhere (assumption that ODF editors are free and available everywhere.)

]]> By: Anonymous http://www.robweir.com/blog/2007/11/pdf-waste-land-and-monicas-blue-dress.html#comment-1273 Anonymous Thu, 22 Nov 2007 12:04:00 +0000 http://www.robweir.com/blog/2007/11/pdf-the-waste-land-and-monicas-blue-dress.html#comment-1273 That won't work. When you archive originals, the legal requirement is often to keep the *original* original, and you can't expect that all the files you get handed to archive will have been prepared for you specially. Hence the pdf/a, odf pair would be kept separately.<br/><br/>In fact sometimes more formats need to be kept: the original, the rendered document (maybe pdf, maybe open standards for non-paper content like audio, video), and occasionally an additional searchable format is needed (plain text ocr'd from a tiff; a timestamped transcript of a video; etc).<br/><br/>ODF is an improvement on the past, where we have to archive not just the document but the proprietary application that read it; but its not going to solve this problem in the large. PDF/A isnt a panacea either, but its a reasonable alternative to paper. That won’t work. When you archive originals, the legal requirement is often to keep the *original* original, and you can’t expect that all the files you get handed to archive will have been prepared for you specially. Hence the pdf/a, odf pair would be kept separately.

In fact sometimes more formats need to be kept: the original, the rendered document (maybe pdf, maybe open standards for non-paper content like audio, video), and occasionally an additional searchable format is needed (plain text ocr’d from a tiff; a timestamped transcript of a video; etc).

ODF is an improvement on the past, where we have to archive not just the document but the proprietary application that read it; but its not going to solve this problem in the large. PDF/A isnt a panacea either, but its a reasonable alternative to paper.

]]>