ODF

Last year, when I was socializing the idea of creating the OASIS ODF Interoperability and Conformance TC, I gave a presentation I called “ODF Interoperability: The Price of Success”. The observation was that standards that fail never need to deal with interoperability. The creation of test suites, convening of multi-vendor interoperability workshops and plugfests is a sign of a successful standard, one which is implemented by many vendors, one which is adopted by many users, one which has vendor-neutral venues for testing implementations and iteratively refining the standard itself.

Failed standards don’t need to work on interoperability because failed standards are not implemented. Look around you. Where are the OOXML test suites? Where are the OOXML plugfests? Indeed, where are the OOXML implementations and adoptions? Microsoft Office has not implemented ISO/IEC 29500 “Office Open XML”, and neither has anyone else. In one of the great ironies, Microsoft’s escapades in ISO have left them clutching a handful of dust, while they scramble now to implement ODF correctly. This is reminiscent of their expensive and failed gamble on HD DVD on the XBox, followed eventually by a quick adoption of Blue-ray once it was clear which direction the market was going. That’s the way standards wars typically end in markets with strong network effects. They tend to end very quickly, with a single standard winning. Of course, the user wins in that situation as well. This isn’t Highlander. This is economic reality. This is how the world works.

Although this may appear messy to an outside observer, our current conversation on ODF interoperability is a good thing, and further proof, to use the words Microsoft’s National Technology Director, Stuart McKee, that “ODF has clearly won“.

Fixing interoperability defects is the price of success, and we’re paying that price now. The rewards will be well worth the cost.

We’ve come very far in only a few years. First we had to fight for even the idea and acceptance of open standards, in a world dominated by a RAND view of exclusionary standards created in smoke filled rooms, where vendors bargained about how many patents they could load up a standard with. We won that battle. Then we had to fight for ODF, a particular open standard, against a monopolist clinging to its vendor lock-in and control over the world’s documents. We won that battle. But our work doesn’t end here. We need to continue the fight, to ensure that users of document editors, you and I, get the full interoperability benefits of ODF. Other standards, like HTML, CSS, EcmaScript, etc., all went through this phase. Now it is our turn.

With an open standard, like ODF, I own my document. I choose what application I use to author that document. But when I send that document to you, or post it on my web site, I do so knowing that you have the same right to choose as I had, and you may choose to use a different application and a different platform than I used. That is the power of ODF.

Of course, the standard itself, the ink on the pages, does not accomplish this by itself. A standard is not a holy relic. I cannot take the ODF standard and touch it to your forehead say “Be thou now interoperable!” and have it happen. If a vendor wants to achieve interoperability, they need to read (and interpret) the standard with an eye to interoperability. They need to engage in testing with other implementations. And they need to talk to their users about their interoperability expectations. This is not just engineering. Interoperability is a way of doing business. If you are trying to achieve interoperability by locking yourself in a room with a standard, then you’ll have as much luck as trying to procreate while locked in a room with a book on human reproduction. Interoperability, like sex, is a social activity. If you’re doing it alone then you’re doing it wrong.

Standards are written documents — text — and as such they require interpretation. There are many schools of textual interpretation: legal, literary, historic, linguistic, etc. The most relevant one, from the perspective of a standard, is what is called “purposive” or “commercial” interpretation, commonly applied by judges to contracts. When interpreting a document using an purposive view, you look at the purpose, or intent, of a document in its full context, and interpret the text harmoniously with that intent. Since the purpose of a standard is to foster interoperability, any interpretation of the text of a standard which is used to argue in favor of, or in defense of, a non-interoperable implementation, has missed the mark. Not all interpretations are equal. Interpretations which are incongruous with the intent of standardization can easily be rejected.

Standards can not force a vendor to be interoperable. If a vendor wishes deliberately to withhold interoperability from the market, then they will always be able to do so, and, in most cases, devise an excuse using the text of the standard as a scapegoat.

Let’s work through a quick example, to show how this can happen.

OpenFormula is the part of ODF 1.2 that defines spreadsheet formulas. The current draft defines the addition operator as:

6.3.1 Infix Operator “+”

Summary: Add two numbers.
Syntax: Number Left + Number Right
Returns: Number
Constraints: None
Semantics: Adds numbers together.

I think most vendors would manage to make an interoperable implementation of this. But if you wanted to be incompatible, there are certainly ways to do so. For example, given the expression “1+1” I could return “42” and still claim to be interoperable. Why? Because the text says “adds numbers together”, but doesn’t explicitly say which numbers to add together. If you decided to add 1 and 41 together, you could claim to be conformant. OK, so let’s correct the text so it now reads:

6.3.1 Infix Operator “+”

Summary: Add two numbers.
Syntax: Number Left + Number Right
Returns: Number
Constraints: None
Semantics: Adds Left to Right.

So, this is bullet-proof now, right? Not really. If I want to, I can say that 1+1 =10, if I want to claim that my implementation works in base 2. We can fix that in the standard, giving us:

6.3.1 Infix Operator “+”

Summary: Add two numbers.
Syntax: Number Left + Number Right, both in base 10 representations
Returns: Number, in base 10
Constraints: None
Semantics: Adds Left to Right.

Better, perhaps. But if I want I can still break compatibility. For example, I could say 1+1=0, and claim that my implementation rounds off to the nearest multiple of 5. Or I could say that 1+1 = 1, claiming that the ‘+’ sign was taken as representing the logical disjunction operator rather than arithmetic addition. Or I could do addition modulo 7, and say that the text did not explicitly forbid that. Or I could return the correct answer some times, but not other times, claiming that the standard did not say “always”. Or I could just insert a sleep(5000) statement in my code, and pause 5 seconds every time the an addition operation is performed, making a useless, but conformant implementation And so on, and so on.

The old adage holds, “It is impossible to make anything fool- proof because fools are so ingenious.” A standard cannot compel interoperability from those who want resist it. A standard is merely one tool, which when combined with others, like test suites and plugfests, facilitates groups of cooperating parties to achieve interoperability.

Now is the time to achieve interoperability among ODF implementations. We’re beyond kind words and empty promises. When Microsoft first announced, last May, that it would add ODF support to Office 2007 SP2, they did so with many fine words:

“Microsoft Corp. is offering customers greater choice and more flexibility among document formats”
Microsoft is “committed to work with others toward robust, consistent and interoperable implementations”
Chris Capossela, senior vice president for the Microsoft Business Division: “We are committed to providing Office users with greater choice among document formats and enhanced interoperability between those formats and the applications that implement them”
“Microsoft recognizes that customers care most about real-world interoperability in the marketplace, so the company is committed to continuing to engage the IT community to achieve that goal when it comes to document format standards”
Microsoft will “work with the Interoperability Executive Customer Council and other customers to identify the areas where document format interoperability matters most, and then collaborate with other vendors to achieve interoperability between their implementations of the formats that customers are using today. This work will continue to be carried out in the Interop Vendor Alliance, the Document Interoperability Initiative, and a range of other interoperability labs and collaborative venues.”
“This work on document formats is only one aspect of how Microsoft is delivering choice, interoperability and innovative solutions to the marketplace.”

So the words are there, certainly. But what was delivered fell far, far short of what they promised. Excel 2007 SP2 strips out spreadsheet formulas when it reads ODF spreadsheets from every other vendor’s spreadsheets, and even from spreadsheets created by Microsoft’s own ODF Add-in for Excel. No other vendor does this. Spreadsheet formulas are the very essence of a spreadsheet. To fail to achieve this level of interoperability calls into question the value and relevance of what was touted as an impressive array of interoperability initiatives. What value is an Interoperability Executive Council, an Interop Vendor Alliance, a Document Interoperability Initiative, etc., if they were not able to motivate the most simple act: taking spreadsheet formula translation code that Microsoft already has (from the ODF Add-in for Office) and using it in SP2?

The pretty words have been shown to be hollow words. Microsoft has not enabled choice. Their implementation is not robust. They have, in effect, taken your ODF document, written by you by your choice in an interoperable format, with demonstrated interoperability among several implementations, and corrupted it, without your knowledge or consent.

There are no shortage of excuses from Redmond. If customers wanted excuses more than interoperability they would be quite pleased by Microsoft’s prolix effusions on this topic. The volume of text used to excuse their interoperability failure, exceeds, by an order of magnitude, the amount of code that would be required to fix the problem. The latest excuse is the paternalistic concern expressed by Doug Mahugh, saying that they are corrupting spreadsheets in order to protect the user. Using a contrived example, of a customer who tries to add cells containing text to those containing numbers, Doug observes that OpenOffice and Excel give different answers to the formula = 1+ “2”. Because all implementations do not give the same answer, Microsoft strips out formulas. Better to be the broken clock that reads the correct time twice a day, than to be unpredictable, or as Doug puts it:

If I move my spreadsheet from one application to another, and then discover I can’t recalculate it any longer, that is certainly disappointing. But the behavior is predictable: nothing recalculates, and no erroneous results are created.

But what if I move my spreadsheet and everything looks fine at first, and I can recalculate my totals, but only much later do I discover that the results are completely different than the results I got in the first application?

That will most definitely not be a predictable experience. And in actual fact, the unpredictable consequences of that sort of variation in spreadsheet behavior can be very consequential for some users. Our customers expect and require accurate, predictable results, and so do we. That’s why we put so much time, money and effort into working through these difficult issues.

This bears a close resemblance to what is sometimes called “Ben Tre Logic”, after the Vietnamese town whose demise was excused by a U.S. General with the argument, “It became necessary to destroy the village in order to save it.”

Doug’s argument may sound plausible at first glance. There is that scary “unpredictable consequences”. We can’t have any of that, can we? Civilization would fall, right? But what if I told you that the same error with the same spreadsheet formula occurs when you exchange spreadsheets in OOXML format between Excel and OpenOffice? Ditto for exchanging them in the binary XLS format. In reality, this difference in behavior has nothing to do with the format, ODF or OOXML or XLS. It is a property of the application. So, why is Microsoft not stripping out formulas when reading OOXML spreadsheet files? After all, they have exactly the same bug that Doug uses as the centerpiece of his argument for why formulas are stripped from ODF documents. Why is Microsoft not concerned with “unpredictable consequences” when using OOXML? Why do users seem not to require “accurate, predictable results” when using OOXML? Or to be blunt, why is Microsoft discriminating against their own paying customers who have chosen to use ODF rather than OOXML? How is this reconciled with Microsoft’s claim that they are delivering “choice, interoperability and innovative solutions to the marketplace”?

Taking Control of Your Documents

2009/03/24 By Rob 12 Comments

How to free yourself from Microsoft Office dependency in three easy steps

The Objective

When you save a document in your word processor, your work is encoded in a particular file format. You often have a choice of formats that you can use, with names like DOC, DOCX, RTF, WPD or ODT. Your choice of format will influence whether others can easily read your document today, whether you yourself will be able to read your document ten years from now, and whether you will be able to migrate painlessly to another word processor or operating system if and when you choose to do so.

Although many users simply click “Save” and give no thought to which format is being used under the covers, this unthinking use of the word processor’s default settings is a recipe for vendor lock-in. In fact, several vendors intentionally set their default format to be ones which will only work well with their own software, fostering dependency on that vendor’s software and lessening the user’s ability to take advantage of other options in the market. The more documents you save and accumulate in a vendor’s proprietary format, the harder it will be for you to consider any other choices.

The objective of this paper is to show you, the user, how to extricate yourself from this cycle of dependency and take control of your documents. Specifically, we show how you can, in three easy steps, free yourself from a Microsoft Office dependency. In the end you may, of course, choose to remain on Microsoft Office. You may decide to migrate to an alternative word processor. That, in the end, is your choice. But by following the three steps outlined below, your freedom of action will be preserved, and your choice of word processor will be based on your priorities and your needs, and not forced on you by your current application vendor.

Step 1: Take control of the default format

The older versions of Microsoft Office, Office 97-Office 2003), by default save documents in a family of binary formats with the extensions DOC (Word), XLS (Excel) and PPT (PowerPoint). Although these formats are proprietary Microsoft formats, over the past decade 3rd party applications have developed the capability to read and write these formats.

However, starting in Office 2007 Microsoft suddenly switched the default format to something called Office Open XML (OOXML). This format is not widely supported outside of Office 2007. So if you save a document in the OOXML format you make it harder for anyone else to read your document unless they are also using Microsoft Office 2007. In almost all cases, the same document, if saved in the legacy DOC format will be more interoperable. Staying with the default choice, OOXML, only restricts your choices and make you more dependent on Microsoft Office. Of course, that is why Microsoft made OOXML the default format.

The first step to liberate yourself from Microsoft Office dependency is to change the default format in Microsoft Office 2007 away from OOXML and back to the early binary formats supported by Office 97-2003, which are widely supported by 3^rd party applications. This is a neutral step that preserves the status quo. By making these changes you will still be able to read and edit any OOXML documents that are sent to you, but all new documents you create will be saved in the more widely supported DOC/XLS/PPT formats.

If you are using Microsoft Office 2003 or earlier, then you should skip this Step and move on to Step 2, since OOXML is not the default format in those earlier Office versions.

To change the defaults, you will need to load Word 2007, Excel 2007 and PowerPoint 2007 and follow the following steps.

Word 2007

Click the Office Button (the unlabeled logo button in the upper left of the program).
Click “Word Options” at the bottom of the dialog.
Go to the “Save” section.
For the “Save files in this format” setting, choose “Word 97-2003 Document(*.doc)”.
Click OK.

Excel 2007

Click the Office Button (the unlabeled logo button in the upper left of the program).
Click “Excel Options” at the bottom of the dialog.
Go to the “Save” section.
For the “Save files in this format” setting choose “Excel 97-2003 Workbook (*.xls)”.
Click OK.

PowerPoint 2007

Click the Office Button (the unlabeled logo button in the upper left of the program).
Click “PowerPoint Options” at the bottom of the dialog.
Go to the “Save” section
For the “Save files in this format” setting, choose “PowerPoint Presentation 97-2003”.
Click OK.

Administrators should also note that these settings may be made directly in the Windows Registry, and automatically pushed out to a work group via a login script or group policy. The registry settings corresponding to the above changes are:

HKEY_CURRENT_USER\Software\Microsoft\Office\12.0\Word\Options Add String DefaultFormat=Doc

HKEY_CURRENT_USER\Software\Microsoft\Office\12.0\Excel\Options
Add DWORD DefaultFormat=38 (Hexadecimal)

HKEY_CURRENT_USER\Software\Microsoft\Office\12.0\PowerPoint\Options
Add DWORD DefaultFormat=0 (Hexadecimal)

Step 2: Enable OpenDocument Format Support

Now that you’ve made the first steps towards taking control of your documents by preventing the lock-in effects of the OOXML default, it is time to take further control. You’ll now want to enable OpenDocument Format (ODF=ISO/IEC 26300) support in Microsoft Office, so you can save and exchange documents using the free and open International Standard while remaining in the familiar Microsoft Office interface.

ODF is an XML-based, open document format standard, designed to be platform- and application-neutral and support interoperable use across applications, eliminating vendor lock-in. ODF is supported by many applications, including office suites from Sun, IBM, Novell and Google, as well as open source projects like OpenOffice, KOffice and AbiWord. Additional applications supporting ODF are listed on Wikipedia.

Microsoft Office does not currently support ODF “out of the box”, but you can enable ODF support in Office by installing a “plugin”, sometimes called an “add-in”. A plugin will add additional options or menu items to the Microsoft Office UI, allowing you to open and save documents in ODF format. In some cases you can even set ODF as the default format for new documents.

There are three main choices for adding ODF support to Microsoft Office:

Sun Microsystems has published an “ODF Plugin for Microsoft Office” which supports Office 2000, XP, 2003 and 2007 SP1.
Microsoft has sponsored an open source project on SourceForge for an “ODF Add-in for Microsoft Office”, which supports Office 2007, and also Office 2003 and Office XP if the Microsoft Office Compatibility Pack is also installed
Microsoft has announced that Office 2007 Service Pack 2 (SP2) will enable ODF support in Office 2007, but this code is not yet available.

Step 2 is to evaluate and adopt a plugin to add ODF support to Microsoft Office. Start using ODF now, saving your documents in the open standard document format. This allows you to remain in Office, for now, while building your familiarity and comfort level with ODF.

Step 3: Exercise your Right to Choose a Native ODF Editor

The plug-in approach is a transitional approach. It allows you to continue working in Microsoft Office while you enable ODF support side-by-side. But at some point you will want to consider your options. Maybe you find that converting back and forth to ODF format in MS Office is slow. Maybe you are using Office 2003 currently, but want to avoid paying for an Office 2007 upgrade when mainstream support for Office 2003 comes to an end on April 14th, 2009. At some point you will want to move to an application that supports ODF natively. You are free at this point and have a wide variety of choices.

You can stay on Windows or consider moving to Linux or the Mac.
You can stay with a traditional client editor, or move to a web based editor.
You can use commercial software, or use open source software.

The important thing is that you have taken control of your documents. You are no longer dependent on Microsoft Office and its file format. You have broken free of the vendor lock-in. You are free to choose an alternative word processor when you want to and if you want to. Until then, be comfortable in knowing that you are keeping your options open while remaining in control of your documents.

This paper is also available in ODF and PDF formats.

The Challenge

2008/05/05 By Rob 17 Comments

<?xml version="1.0" encoding="UTF-8"?>
<office:document-content
xmlns:office="urn:oasis:names:tc:opendocument:xmlns:office:1.0"
xmlns:text="urn:oasis:names:tc:opendocument:xmlns:text:1.0"
office:version="1.0">
<office:body>
<office:text>
<text:p>Dear Alex Brown. Please prove that I am invalid ODF 1.0 (ISO 26300:2006).
I do not think that I am. In fact I think that your statement that there are
no valid ISO ODF documents in the world, and that there cannot be, is a brash,
irresponsible and indefensible piece of bombast that you should retract.</text:p>
<text:p>(Please note that this document contains no ID, IDREF or IDREFS attributes.
Nor does it contain custom content.)</text:p>
</office:text>
</office:body>
</office:document-content>

Document Format FUD: A Guide for the Perplexed

2007/11/18 By Rob 8 Comments

I’ve decided to put together a list of misconceptions that I hear, generally on the topic of document formats. I’ll try to update this list to keep it current, with the most recent entries at the top. Readers are invited to submit the FUD they observe as comments, and I’ll include it where I can.

This inaugural edition is dedicated to the fallout from the recent supernova we know as the OpenDocument Foundation, that in one final act of self-immolation swelled from obscurity to overwhelming brilliance, but then slowly faded away, ever fainter and more erratic, little more than hot gas, the dimming embers no longer sustainable.

Q: Now that the originator and primary supporter of OpenDocument Format has ended its support for ODF, does this mean the end for the ODF standard? (18 Nov 2007)

A: This question is based on a mistaken premise, namely that the OpenDocument Foundation was the originator or steward of the ODF standard. This is an erroneous notion.

The ODF standard is owned by the OASIS standards consortium, with over 600 member organizations and individual members. The committee in OASIS that that does the technical working of maintaining the ODF standard is called the OpenDocument TC. It has 15 organization members as well as 7 individual members. Until recently the OpenDocument Foundation was a member of the ODF TC, one voice among many.

The adoption of the ODF standard is promoted by several organizations, most prominently the ODF Alliance (with over 400 organizational members in 52 countries), the OpenDocument Fellowship (around 100 individual members) and the OpenDoc Society (a new group with a Northern European focus, with around 50 organizational members). To put this in perspective, the OpenDocument Foundation, before it changed its mission and dissolved, had only 3 members.

When you consider the range of ODF adoption, especially in Europe and Asia, the strong continuing work on ODF 1.2 in OASIS, and the strong corporate, government and organizational participation demonstrated in the global ODF User Workshop recently held in Berlin, we seem to be making a disproportionate amount of noise over the hysterics of the disintegrating 3-person OpenDocument Foundation.

A number of analysts/journalists/bloggers didn’t check their facts and seem to have fallen into the trap, and ascribed a far greater importance to the actions of the Foundation. Curiously, these articles all quoted the same Microsoft Director of Corporate Standards. I hope this correlation does not prove to be a persistent contrary indicator for accuracy in future file format stories.

Luckily for us, David Berlind over at ZDNet has penetrated the confusion and gets it right:

…the future of the OpenDocument Foundation has nothing to do with the future of the OpenDocument Format. In other words, any indication by anybody that the OpenDocument Format has been vacated by its supporters is pure FUD.

11/27/2009 Update: Berlind did further research and interviews on this topic and followed up with a podcast and new blog post OpenDocument Format Community steadfast despite theatrics of now impotent ‘Foundation’ on this subject.

Q: The Open Document Foundation has a document, a “Universal Interoperability Framework” that on its title page says “Submitted to the OASIS Office Technical Committee by The OpenDocument Foundation October 16, 2007”. What is the status of this proposal in the ODF TC? (18 Nov 2007)

A: No such document has been submitted to the OASIS TC, on this date or any other date. OASIS policy states that “Contributions, as defined in the OASIS IPR Policy, shall be made by sending to the TC’s general email list either the contribution, or a notice that the contribution has been delivered to the TC’s document repository”. A look at the ODF TC’s list archive for October shows that there was no such contribution.

Q: The Foundation claims that the W3C’s CDF format has better interoperability with MS Office than ODF has. Is this true? (18 Nov 2007)

A: The Foundation’s claims have not been demonstrated, or even competently argued at a technical level that would allow expert evaluation. I cannot fully critique what is essentially vaporware. However, those who know CDF better than I do have commented on the mismatch between CDF and office documents, for example the recent interview with the W3C’s Chris Lilley in Andy Updegrove’s blog.

Q: So, does IBM then oppose CDF in favor of ODF? (18 Nov 2007)

A: No. IBM supports both the development of ODF and CDF and has a leadership role in both working groups. These are two good standards for two different things.

The W3C, over the years has produced a number of reusable, modular core standards for things like vector graphics (SVG), mathematical notation (MathML), forms (XForms), etc. To use a cooking analogy, these are like ingredients that can be combined to make a dish. ODF has taken a number of W3C standards and combined them to make a format for expressing conventional office documents, the familiar word processor, spreadsheet and presentation documents. ODF is an OASIS and ISO standard.

But just as eggs, butter and flour form the base of many recipes, the core W3C standards can be assembled in different ways for different purposes. This is a good thing.

CDF is not so much a final dish, but an intermediate step, like a roux (flour + butter) is when making a sauce. You don’t use a roux directly, but build upon it, e.g., add milk to make a béchamel, add cheese for a cheese sauce, etc., CDF itself s not directly consumable. You need to add a WICD profile, something like WICD Mobile 1.0, before you have something a user agent can process.

ODF

Top 10 Blog Posts of 2009

Top Blog Posts

Top Browsers

Top Operating Systems

The Battle for ODF Interoperability