My post of 10 days ago, How to hire Guillaume Portes, received quite a bit of attention, with over 50,000 page views, links from 25 blogs, and around 300 comments left by visitors to this blog, Slashdot and the Joel on Software discussion group. I’d like to thank all that took the time to read and to comment.
It is good to continually tell the story and make the case. Having two standard file formats for office documents would be a bad thing for commerce, for end users and for the industry. With two formats, end users will be confused and costs will be higher for those who sell and buy software that works with documents. This will essentially cause a frictional drag on the document processing market. Sure, there will be those who will benefit from the chaos, just as there are those who benefit from the friction of currency exchanges. But over the years we’ve learned the value of things like uniform commercial codes, currency unions and uniform trade regulations.
I’ve heard no one complain about having lost their freedom of choosing the Mark over the Lira over the Franc. We simply use the Euro and then concentrate on what we are buying or selling, not on the currency. In a similar way we should agree on a single document format and then concentrate on application features and user needs and what we are trying to communicate, and stop worrying about file formats. When done well, file formats are invisible. They are not seen by end users, are not discussed by the press, and not thought about by (most) engineers. The fact that I’m writing about OOXML at all and not about my wine making exploits is an aberration caused by the failure of the dominate market player to provide an open document standard that allows users to own their documents.
But I digress…
Now that I’ve finished reading all of the comments, I’d like to review with you some of the better ones, pro and con, along with my commentary.
Let’s start.
I don’t know how many of you noticed: The fictional name “Guillaume Portes” is actually a literal translation of “Bill Gates” in French.
If you noticed this as well, give yourself 3 extra points. Many of my posts have a secret joke, and I hope these will bring a smile to those who find them.
Here’s comment questioning whether there is a problem with OOXML:
I haven’t looked at the spec, so I don’t know how good or bad it is. But the examples he cites don’t strike me as such a problem. They’re all just to maintain backward compatibility with documents from old versions of Word and other apps. You would be free to ignore them if you don’t need that compatibility. I’m not sure how else they could have done it.
A similar view was expressed by another reader:
I don’t know if it has been stated here, but you do know that supporting these the compatibility options is not required for OpenXML compliance? Developers are free to leave these out of they want.
By that same argument developers can also leave out text alignment, images and tables, since these features are not required for compliance either. In fact, everything in OOXML is optional. If you read the compliance definition in the OOXML specification, it comes down to this statement in Section 2.5:
Application conformance is purely syntactic…A conforming consumer shall not reject any conforming documents of the document type expected by that application. A conforming producer shall be able to produce conforming documents.
Given this definition of conformance, a fully conformant OOXML application can be as simple as:
cp foo.docx bar.docx (Linux) or copy foo.docx bar.docx (Windows)
In the end, regardless of whether a feature is optional, or even deprecated, if that feature occurs in real OOXML documents, then an OOXML application that aspires to be used and have viability, either commercially or as open source, will need to support it. It is that simple.
There is only one OOXML specification and to an end user all OOXML files are equivalent and interchangeable. The user who receives a document via email, from a government web site, from a colleague, friend, teacher, etc., doesn’t know whether the document was created in Word, created in OpenOffice, created from scratch in Office 2007, saved from Office 2000 with the Compatibility Pack or whether the document was originally authored in WordPerfect and made it into OOXML format only after migration over years via various Office upgrades. It is a DOCX file and users will expect that applications that claim OOXML support will work with their DOCX file, period. Anything else is a support nightmare. That is the entire point of a standard — interoperability — so we must judge OOXML by how well it facilitates that function.
Here is another comment with a view expressed by others as well:
Someone needs to tell every developer of word processing and page layout software on the planet to abandon the ‘must look the same’ obsession described by the above. Why worry about making content in application B look like content in Application A? I create books out of Word files submitted by several people. The last thing I want is all the inconsistent formatting from each of them to control a book’s look.
Named styles is the answer. If a paragraph is body text, call it that. If it’s an inset quote, call it a quote. If a term is in italics, label it as italicized style not Times Italic 12 point. But don’t get all hung up in the distinctions between Times Roman and Times New Roman. The purpose of XML is to define what something is. Not what someone thought it ought to look like on Tuesday three weeks ago.
My personal views are very much in alignment with these sentiments. I think WYSIWYG has done more bad than good over the years, and that strict separation of content, layout and styles should be maintained. However, I also know that my personal views are not universally held, and that the word processor has evolved over the years to be a flexible, multi-paradigm tool that can support both structured document editing as well as looser, ad-hoc editing by users who just need to grind out a memo. A document format for a modern word processor must support both uses.
I’m glad someone brought up the core question:
Considering the requirement that the standard allow for compatibility with existing documents, what would you suggest?
Silently altering documents that are converted into OpenXML?
Disallow automatic conversions whenever a compatibility flag would have otherwise been needed?
One solution approach was mentioned by several users, for which I give two examples:
There is no need to include features from 16 year old (or any age) applications in a new standard. If you want to convert, you convert. If WP6 linespacing is 0.8 of Word2007 linespacing, you write linespacing =”0.8″ in your converted document. You DON’T write useWP6linespacing linespacing =”1″
That is just plain silly. That is making a specification unnecessary large for instances that are rarely used by the general public.
As said: if you want to convert, than use a conversion tool. Do not use a modern specification to hold all legacy features.
And:
Let the plugins do the dirty work of native in-memory-binary representations to XML and back conversions.
Keep the XML file format clean, open, unencumbered, application independent, cross platform, universally transformative and exchangeable, portable and timeless.
I think this is the key point, and I’m gratified that so many readers picked up on it. There is no good reason to have these compatibility flags at all. Instead of having several undefined compatibility flags for legacy line spacing options, we should have a flexible line-spacing model in OOXML and when loading legacy binary documents, convert them as necessary into the line spacing model of OOXML. If the text model in OOXML is sufficiently expressive, this can be done with no loss of fidelity. (And no, a flag that says merely “do it like Word 95” is not an example of expressiveness).
This is what I mean by “generalize and simplify”. A simplified specification is not necessarily less expressive or less capable. A specification is simplified when it supports internal reuse and accomplishes its task with minimal means.
However, if the text model of OOXML is not flexible enough to support even legacy versions of Word, then what hope will the rest of the industry have in adopting it as a format? How will Novell manage with getting OpenOffice to use it, or Corel to get WordPerfect to use it? What about Lotus WordPro? Will Ecma add special compatibility flags to the OOXML specification to account for the quirks of every word processor with legacy documents? What would OOXML look like if we all loaded it up with such legacy flags? Is this the precedent we want to set?
Why should OOXML have special flags for WordPerfect 6.0 (1996) but not have special flags for WordPerfect 12.0 (2004) or the new X3 version (2006)? Is this purely because Microsoft considered WordPerfect to be competitor back in 1996 but now no longer cares? Is this the way to go about designing an ISO standard?
I believe that having compatibility flags in the specification for all word processors in use today is not a practical solution, and that having such flags only for Word and ancient versions of competing products is an approach that benefits only Microsoft.
One last point, since this post is already too long. Microsoft’s Brian Jones is claiming that ODF has a similar issue, in that OpenOffice writes out a number of application-specific settings when it saves a document. This is a good illustration of an important distinction. The items that OpenOffice writes out (you can see an example in Brian’s post) are vendor-defined, document-level application settings. There are now and will continue to be multiple implementations of ODF and it is legitimate that they have application-defined features. These are stored as name/value pairs in a separate XML file in the ODF archive.
I can think of no argument against that. Obviously no interoperability is expected for these vendor specific features, which are for things like application settings like window sizes, zoom factors, print settings, etc. In any case, ODF merely provides a place for applications to store these settings. To blame ODF for any vendor misuse of this feature is like blaming the W3C and HTML for non-standard extensions in Internet Explorer.
OOXML, on the other hand, does not seem to have given much thought to what would be needed in a format that has multiple supporting applications. Only a single application (MS Office) has been explicitly considered, and support for that one application, and its predecessor versions, have been hard-coded into the OOXML schema.
I don’t want to detract from your main argument, which is important for us all to think about. However, I think your analysis of conformance is incomplete in pressing this observation:
In fact, everything in OOXML is optional.”
Indeed, nothing in OOXML is any more optional than in ODF, where everything, including tons of private (i.e., “foreign”) extensions, are optional.
My reading of OOXML is that they are tidier about it than that.
De gustibus non disputandum
If you think that throwing application-specific compatibility flags into the schema of an ISO standard submission is a “tidy” solution, then I think we have radically different defintions of “tidy”.
In any case, the point was to respond to those who dismiss any criticism of undefined, poorly-defined or incorrectly-defined elements in OOXML as “don’t worry, that part is optional”. Since everything is optional in OOXML, an ocean of sins can be buried (and indeed have been buried) by that disingenuous logic.
One of the criteria for evaluating a specification is whether the normative content is unambiguous. Normative content includes optional and deprecated elements as well.
orcmid, rob:
Tidier is in the eye of the beholder…
I agree with orcmid that both ODF and OOXML has virtually everything “optional”. Perhaps Rob should had made it clear that ODF has a lot of optional things as well. Again, the question of who has more “optional” item is in the eye of the beholder.
Both ODF and OOXML containers (zip file) permits additional files to be included and those are simply going to be unspecified in the ODF/OOXML specs.
My own preference for application-specific stuff is to isolate it in less important files, not sprinkling in the main content where I have no choice to parse it in order to consume the main content. This is of course looking at the problem from the consuming application point-of-view, not how beautiful or self-containing the main content is.
Premise: I haven’t read the specs of ODF and OOXML.
In my opinion, what is really important here is not if the format allows application-defined stuff. The important thing is: is this information needed for proper displaying the document or not?
If those sections include, for example, where was my cursor when I last saved the document, what was the zoom level, what were my printer settings, and so on, the document doesn’t actually need them. This is just additional, optional information. It is perfectly safe to store that information in a good place (e.g. separate files from the main document), and applications are allowed and incouraged to do so in order to achieve a better user experience.
If instead an application is using that data in order to add vendor-specific stuff that is required for correctly undestanding the document, this is definitely wrong (and typical of the Embrace-Extend-Extinguish strategy).
My two cents :-)
What a great comment:
If OpenOffice.org can import a WordPerfect 5.0 document and correctly display the top spacing, then they know how to render suppressTopSpacingWP already and don’t need to be told.
If they ignore the WP5 top spacing brokenness, why wouldn’t they ignore suppressTopSpacingWP as well? How are they worse off?
Either they care about maintaining rendering fidelity with documents created in WordStar or whatever or they don’t. If they do, they already do so for documents they import from WordStar binary formats. Didn’t care enough to import WordStar? Why start caring now?
Yes, MS will be the only people to implement the whole thing. But this is about interoperability. As long as they don’t start setting the mwSmallCaps flag in new documents created in Office 2007, there is no interoperability issue.
Rob, you have lamented that the standard is too long. Now you want OpenXML to have a “text model… flexible enough to support even legacy versions of Word.” Do you know what this means?
It means adding several hundred more pages worth of details on how each version of Word: did horizontal justification, vertical line spacing, treated raised and lowered text, kerned letter pairs, condensed and expanded spacing, stretched and condensed character pitch, etc. Plus a heck of a lot more stuff for non-Western scripts.
Would you prefer that Microsoft insert its source code into the spec?
A number of people get hung up on the fact that I say OOXML is both too long and not informative enough. How can it be both at the same time? I ask you, can a person be both overweight and malnourished at the same time? Certainly, if the person is not eating the right foods.
Length and information content are not two measures of the same thing. This isn’t about making the specification longer or shorter arbitrarily. It is about specifying a set of features in an economical fashion, using time-honored engineering principles.
For example, if we had a flexible line spacing model, we would not need to specify exceptions for legacy versions of Word. If the model is flexible, we simply describe what the spacing is, in real units, just like it would be specified in a new document.
The point is you put the burden of accounting for the Word 95 bug on the convertor that converts Word 95 documents into XML, not on everyone else in the world who needs to deal with with OOXML.
If OpenOffice.org can import a WordPerfect 5.0 document and correctly display the top spacing, then they know how to render suppressTopSpacingWP already and don’t need to be told.
So you are saying that since competing applications over the years have struggled with, and in some cases succeeded with, reverse-engineering aspects of proprietary documents, there is no need for ISO standards to do any better than this.
Indeed, by your own logic, why have an OOXML standard at all, since OpenOffice can already read many Office legacy binary documents?
anonymous said: “Would you prefer that Microsoft insert its source code into the spec?” in relation to the model suggested by Rob – flexible base, optional superstructures.
Speaking as someone who has to deal with much of the Microsoft legacy code – a lot of the people I help with their computers, either bought them ages ago, or were given old computers with old software – that would not be a bad idea.
Consider this – Microsoft has made its billions on the back of this software, which has lost nearly all its book value because Microsoft has stopped supporting it. It has also lost nearly all its “Intellectual Property Rights” value because it has also been reverse-engineered world-wide, courtesy of books like Microsoft Press’ “Inside Windows NT” which has a very positive view of reverse engineering and gives detailed instructions on how to go about it. Ergo, Microsoft should do the honest thing and release all this legacy source code under a suitable open-source-style license like the template Microsoft Community License – without any further additions whatsoever.
I can say this, that the spam and malware problems would decrease markedly, if these old computers with their legacy software – malware’s sitting ducks – , were able to be upgraded by suitable people using Microsoft’s own source code without any stupid malware-friendly restrictions.
I have not read the OOXML nor the ODF specification, but from looking at the whole discussion Microsoft’s motivation for using these tags seems quite obvious to me – anyone tell me if I’m wrong:
to keep their monopoly on accurately rendering at least the old Microsoft document types.
Obviously all of them have been re-engineered to different extents, but only Microsoft is able (or rather “would be able, if they really wanted”) to render them in precisely the same way in every detail as it was done in the original application. Now lest anyone attempts to use Microsoft Office to convert those legacy documents into a format that would enable any odd application to display, use and print them as originally intended, they chose to not convert them in the first place but rather tag them such that only Microsoft Office can do that.
In other words:
so you’ve got an MS Word (old version) document. You can (in theory) open that with the current version of MS Word to look and feel the same as it did back when it was created. If you had an open standard format, i.e. one not referring to proprietary algorithms, you could export the file from Word into that format and henceforward work with it in any other application supporting the standard, without any recognisable difference in rendering or functionality. To get around that, the exported file instead refers to the proprietary algorithms used in the original application it was created with, making it impossible for other contenders to accurately emulate its look & behaviour in every detail.
So instead of converting a, say, Word 95 document into a neutral format and in the process translating its quirks and attributes into a neutral format, they just tag the contents with something like “render like Word 95”. Which only they can do when it comes to it and takes the idea of a neutral standard ad absurdum.
What they say is “you can save this document into OOXML but if you want it rendered it exactly the same as in Word 95 then you better use our product”. Which is anti-competitive behaviour in my opinion. I don’t think it’s a technical issue in the first place rather than a political one (and that’s why they are so bullish about it), nor that they should get away with it.
Sorry for the long post.