Interoperability

ODF Plugfest, Berlin

2011/08/09 By Rob 2 Comments

I attended the 6th ODF Plugfest took place in Berlin a few weeks ago, hosted by the German Federal Ministry of the Interior (BMI) and the Ministry of Economics and Technology (BMWi). It followed the pattern of previous events, a two-day event, with the first day dedicated to technical interop activities among implementors, followed by a day of public presentations. The full agenda is here.

(I’m the jet-lagged figure in the lower left of the above group photo taken on the first day of the Plugfest).

We tested a range of interoperability scenarios, including preservation of RDF metadata, conditional formatting in spreadsheets and advanced text layout scenarios (“hard cases”). There were also a variety of presentations, from vendors, academics and government. I’ve uploaded my presentations if you are interested. I’ll draw your attention as well to Ross Gardler’s talk on The Apache Way and OpenOffice.org. He covered a lot of good material. (A recording of the talk is also online)

Finally, it was also announced that the next ODF Plugfest will occur in the Netherlands, in Gouda, in November. And another one is anticipated in Brussels in March 2012.

Best Practices for Authoring Interoperable ODF Documents

2011/03/10 By Rob 2 Comments

In the OASIS ODF Interoperability and Conformance TC we have recently started work on a new document, a “Committee Note” which will be called, “Best Practices for Authoring Interoperable ODF Documents”.

I will be the editor for this document.

If you are not yet familiar with a “Committee Note”, it is a new category of document that has recently been added to the OASIS process. Think of it being analogous to an ISO Technical Report. A Committee Note (or CN) goes through the same level of review and approval with a Technical Committee, the same public review requirement, etc. But it does not get approved as a standard, so it does not define, for example, conformance requirements. It is intended for things like implementation guides, best practices, white papers, etc.

The general aim of the new CN is to collect and describe guidelines for authors on how best to create interoperable (portable) ODF documents. What to do and what to avoid. Although the focus will be on ODF, much of this will be applicable to any WYSIWYG word processing environment.

I’m thinking of this as being analogous to the “How to write portable C” books we saw years ago. As many of you know, C programs can range from the perverse (see the Obfuscated C Competition for examples) to highly portable. But portability does not come about by accident. The language permits portability, but it does not enforce it on the user. C is powerful enough for a user to hang themselves.

The modern WYSIWYG word processor is similar. A user can create interoperable (portable) documents, but the word processors also allows them to create documents that will be tied tightly to their precise operating environment and will render poorly everywhere else. The tool takes you only so far, and then user education must help with the rest of the way. I hope that this Committee Note will provide some of that user education.

I am absolutely certain that I am not the first one to have thought about this problem. In fact, I suspect (and hope) that many of my readers have done so themselves. So before I start drafting this document, I’d like to solicit for contributions of material. Maybe you have written a paper, report or blog post on this topic? Maybe you can take a few minutes to jot down your ideas? Maybe you can refer us to other sources of information?

But please don’t give me the information here. Per OASIS IPR rules we need to channel any contributions to the Technical Committee, so permission to use your original material is secured, from the copyright perspective. So if you have a contribution you want to make for this document, please do so on the OIC TC’s comment list. And if you want to participate more closely in the creation and editing of this document, then you are always welcome to join OASIS and participate directly in the TC’s work. The cost for individual memberships is quite reasonable.

ODF Plugfest — Brussels

2010/10/28 By Rob 4 Comments

A couple of weeks ago I was in Brussels to participate in the 4th ODF Plugfest. I planned on writing up a nice long post about it. But right when I started to draft this blog post, I came across an excellent article in LWN.net by Koen Vervloesem (Twitter @koenvervloesem): ODF Plugfest: Making office tools interoperable. Since his article is far better than what I would have written, I recommend that you go and read that article first, and then come back here for what meager additional scraps of insight I can add.

Go ahead. I can wait. I’ll be here when you get back.

The ODF Plugfest format is a two-day event. On one day engineers from the vendors work together, peer-to-peer, on interoperability testing, debugging, resolving issues, etc. This is done in a closed session, with no press present, and with a gentlemen’s agreement not to use information from this session to attack other vendors. We want the Plugfest to be a “safe zone” where vendors can do interoperability work where it is most needed, using unreleased software, alpha or beta code in some cases. For this to work we need an environment where engineers can do this work, without fear that each bug in their beta product will be instantly maligned on the web. This would be anti-productive, since it would repel the very products we need most to attend Plugfests.

I think customers would be proud to see their vendors putting their differences aside for the day to work on interoperability. Although among the vendors and organizations present there were several fierce competitors, two parties to a prominent patent infringement lawsuit and both sides of a prominent fork of a popular open source project, you would not have guessed this if you watched the engineers collaborating at the Plugfest. A key part to this neutrality is that the Plugfests are sponsored and hosted by public sector parties and universities and non-profits. In this case we were hosted by the Flemish government.

So that was the first day of the Plugfest, and for the details I can say no more, for the reasons I’ve stated.

On the 2nd day we have a public session, with vendors, but also the press, local public sector IT people, local IT companies, etc. The program and presentations are posted. My presentation on ODF 1.2 is also up on my publications page.

A few leftover notes that I have not seen mentioned elsewhere:

We had great participation from AbiWord, where developers apparently have funding to work on their ODF 1.2 support.
DIaLOGIKa announced that after their next release they will no longer have funding from Microsoft to continue work on their ODF Add-in for Office. The code, however, will remain as open source. Since Oracle has commercialized the previously free Sun ODF Plugin, this means that there is no longer any free, actively developed means of getting ODF support on Office 2003. If you want ODF support on Office, you must upgrade to Office 2007 or Office 2010.
Some good demos of new ODF-supporting software, including LetterGen, OFS Collaboration Suite, ODT2EPub and odt2braille.
Itaapy announced that they were close to releasing a C++ version of their popular lpOD library (already available in Python)

In standards work, on committees with endless conference calls and endless draft specifications and the minutia of clause and phrase, it is too easy to mistakenly view that narrow world as your customer. So when I attend events like this and see the rapid growth of ODF-supporting software and the innovative work that is happening among implementors, I return reinvigorated. These are the real customers. This is what it is all about. I’m already looking forward to the next ODF Plugfest.

The Battle for ODF Interoperability

2009/05/17 By Rob 33 Comments

Last year, when I was socializing the idea of creating the OASIS ODF Interoperability and Conformance TC, I gave a presentation I called “ODF Interoperability: The Price of Success”. The observation was that standards that fail never need to deal with interoperability. The creation of test suites, convening of multi-vendor interoperability workshops and plugfests is a sign of a successful standard, one which is implemented by many vendors, one which is adopted by many users, one which has vendor-neutral venues for testing implementations and iteratively refining the standard itself.

Failed standards don’t need to work on interoperability because failed standards are not implemented. Look around you. Where are the OOXML test suites? Where are the OOXML plugfests? Indeed, where are the OOXML implementations and adoptions? Microsoft Office has not implemented ISO/IEC 29500 “Office Open XML”, and neither has anyone else. In one of the great ironies, Microsoft’s escapades in ISO have left them clutching a handful of dust, while they scramble now to implement ODF correctly. This is reminiscent of their expensive and failed gamble on HD DVD on the XBox, followed eventually by a quick adoption of Blue-ray once it was clear which direction the market was going. That’s the way standards wars typically end in markets with strong network effects. They tend to end very quickly, with a single standard winning. Of course, the user wins in that situation as well. This isn’t Highlander. This is economic reality. This is how the world works.

Although this may appear messy to an outside observer, our current conversation on ODF interoperability is a good thing, and further proof, to use the words Microsoft’s National Technology Director, Stuart McKee, that “ODF has clearly won“.

Fixing interoperability defects is the price of success, and we’re paying that price now. The rewards will be well worth the cost.

We’ve come very far in only a few years. First we had to fight for even the idea and acceptance of open standards, in a world dominated by a RAND view of exclusionary standards created in smoke filled rooms, where vendors bargained about how many patents they could load up a standard with. We won that battle. Then we had to fight for ODF, a particular open standard, against a monopolist clinging to its vendor lock-in and control over the world’s documents. We won that battle. But our work doesn’t end here. We need to continue the fight, to ensure that users of document editors, you and I, get the full interoperability benefits of ODF. Other standards, like HTML, CSS, EcmaScript, etc., all went through this phase. Now it is our turn.

With an open standard, like ODF, I own my document. I choose what application I use to author that document. But when I send that document to you, or post it on my web site, I do so knowing that you have the same right to choose as I had, and you may choose to use a different application and a different platform than I used. That is the power of ODF.

Of course, the standard itself, the ink on the pages, does not accomplish this by itself. A standard is not a holy relic. I cannot take the ODF standard and touch it to your forehead say “Be thou now interoperable!” and have it happen. If a vendor wants to achieve interoperability, they need to read (and interpret) the standard with an eye to interoperability. They need to engage in testing with other implementations. And they need to talk to their users about their interoperability expectations. This is not just engineering. Interoperability is a way of doing business. If you are trying to achieve interoperability by locking yourself in a room with a standard, then you’ll have as much luck as trying to procreate while locked in a room with a book on human reproduction. Interoperability, like sex, is a social activity. If you’re doing it alone then you’re doing it wrong.

Standards are written documents — text — and as such they require interpretation. There are many schools of textual interpretation: legal, literary, historic, linguistic, etc. The most relevant one, from the perspective of a standard, is what is called “purposive” or “commercial” interpretation, commonly applied by judges to contracts. When interpreting a document using an purposive view, you look at the purpose, or intent, of a document in its full context, and interpret the text harmoniously with that intent. Since the purpose of a standard is to foster interoperability, any interpretation of the text of a standard which is used to argue in favor of, or in defense of, a non-interoperable implementation, has missed the mark. Not all interpretations are equal. Interpretations which are incongruous with the intent of standardization can easily be rejected.

Standards can not force a vendor to be interoperable. If a vendor wishes deliberately to withhold interoperability from the market, then they will always be able to do so, and, in most cases, devise an excuse using the text of the standard as a scapegoat.

Let’s work through a quick example, to show how this can happen.

OpenFormula is the part of ODF 1.2 that defines spreadsheet formulas. The current draft defines the addition operator as:

6.3.1 Infix Operator “+”

Summary: Add two numbers.
Syntax: Number Left + Number Right
Returns: Number
Constraints: None
Semantics: Adds numbers together.

I think most vendors would manage to make an interoperable implementation of this. But if you wanted to be incompatible, there are certainly ways to do so. For example, given the expression “1+1” I could return “42” and still claim to be interoperable. Why? Because the text says “adds numbers together”, but doesn’t explicitly say which numbers to add together. If you decided to add 1 and 41 together, you could claim to be conformant. OK, so let’s correct the text so it now reads:

6.3.1 Infix Operator “+”

Summary: Add two numbers.
Syntax: Number Left + Number Right
Returns: Number
Constraints: None
Semantics: Adds Left to Right.

So, this is bullet-proof now, right? Not really. If I want to, I can say that 1+1 =10, if I want to claim that my implementation works in base 2. We can fix that in the standard, giving us:

6.3.1 Infix Operator “+”

Summary: Add two numbers.
Syntax: Number Left + Number Right, both in base 10 representations
Returns: Number, in base 10
Constraints: None
Semantics: Adds Left to Right.

Better, perhaps. But if I want I can still break compatibility. For example, I could say 1+1=0, and claim that my implementation rounds off to the nearest multiple of 5. Or I could say that 1+1 = 1, claiming that the ‘+’ sign was taken as representing the logical disjunction operator rather than arithmetic addition. Or I could do addition modulo 7, and say that the text did not explicitly forbid that. Or I could return the correct answer some times, but not other times, claiming that the standard did not say “always”. Or I could just insert a sleep(5000) statement in my code, and pause 5 seconds every time the an addition operation is performed, making a useless, but conformant implementation And so on, and so on.

The old adage holds, “It is impossible to make anything fool- proof because fools are so ingenious.” A standard cannot compel interoperability from those who want resist it. A standard is merely one tool, which when combined with others, like test suites and plugfests, facilitates groups of cooperating parties to achieve interoperability.

Now is the time to achieve interoperability among ODF implementations. We’re beyond kind words and empty promises. When Microsoft first announced, last May, that it would add ODF support to Office 2007 SP2, they did so with many fine words:

“Microsoft Corp. is offering customers greater choice and more flexibility among document formats”
Microsoft is “committed to work with others toward robust, consistent and interoperable implementations”
Chris Capossela, senior vice president for the Microsoft Business Division: “We are committed to providing Office users with greater choice among document formats and enhanced interoperability between those formats and the applications that implement them”
“Microsoft recognizes that customers care most about real-world interoperability in the marketplace, so the company is committed to continuing to engage the IT community to achieve that goal when it comes to document format standards”
Microsoft will “work with the Interoperability Executive Customer Council and other customers to identify the areas where document format interoperability matters most, and then collaborate with other vendors to achieve interoperability between their implementations of the formats that customers are using today. This work will continue to be carried out in the Interop Vendor Alliance, the Document Interoperability Initiative, and a range of other interoperability labs and collaborative venues.”
“This work on document formats is only one aspect of how Microsoft is delivering choice, interoperability and innovative solutions to the marketplace.”

So the words are there, certainly. But what was delivered fell far, far short of what they promised. Excel 2007 SP2 strips out spreadsheet formulas when it reads ODF spreadsheets from every other vendor’s spreadsheets, and even from spreadsheets created by Microsoft’s own ODF Add-in for Excel. No other vendor does this. Spreadsheet formulas are the very essence of a spreadsheet. To fail to achieve this level of interoperability calls into question the value and relevance of what was touted as an impressive array of interoperability initiatives. What value is an Interoperability Executive Council, an Interop Vendor Alliance, a Document Interoperability Initiative, etc., if they were not able to motivate the most simple act: taking spreadsheet formula translation code that Microsoft already has (from the ODF Add-in for Office) and using it in SP2?

The pretty words have been shown to be hollow words. Microsoft has not enabled choice. Their implementation is not robust. They have, in effect, taken your ODF document, written by you by your choice in an interoperable format, with demonstrated interoperability among several implementations, and corrupted it, without your knowledge or consent.

There are no shortage of excuses from Redmond. If customers wanted excuses more than interoperability they would be quite pleased by Microsoft’s prolix effusions on this topic. The volume of text used to excuse their interoperability failure, exceeds, by an order of magnitude, the amount of code that would be required to fix the problem. The latest excuse is the paternalistic concern expressed by Doug Mahugh, saying that they are corrupting spreadsheets in order to protect the user. Using a contrived example, of a customer who tries to add cells containing text to those containing numbers, Doug observes that OpenOffice and Excel give different answers to the formula = 1+ “2”. Because all implementations do not give the same answer, Microsoft strips out formulas. Better to be the broken clock that reads the correct time twice a day, than to be unpredictable, or as Doug puts it:

If I move my spreadsheet from one application to another, and then discover I can’t recalculate it any longer, that is certainly disappointing. But the behavior is predictable: nothing recalculates, and no erroneous results are created.

But what if I move my spreadsheet and everything looks fine at first, and I can recalculate my totals, but only much later do I discover that the results are completely different than the results I got in the first application?

That will most definitely not be a predictable experience. And in actual fact, the unpredictable consequences of that sort of variation in spreadsheet behavior can be very consequential for some users. Our customers expect and require accurate, predictable results, and so do we. That’s why we put so much time, money and effort into working through these difficult issues.

This bears a close resemblance to what is sometimes called “Ben Tre Logic”, after the Vietnamese town whose demise was excused by a U.S. General with the argument, “It became necessary to destroy the village in order to save it.”

Doug’s argument may sound plausible at first glance. There is that scary “unpredictable consequences”. We can’t have any of that, can we? Civilization would fall, right? But what if I told you that the same error with the same spreadsheet formula occurs when you exchange spreadsheets in OOXML format between Excel and OpenOffice? Ditto for exchanging them in the binary XLS format. In reality, this difference in behavior has nothing to do with the format, ODF or OOXML or XLS. It is a property of the application. So, why is Microsoft not stripping out formulas when reading OOXML spreadsheet files? After all, they have exactly the same bug that Doug uses as the centerpiece of his argument for why formulas are stripped from ODF documents. Why is Microsoft not concerned with “unpredictable consequences” when using OOXML? Why do users seem not to require “accurate, predictable results” when using OOXML? Or to be blunt, why is Microsoft discriminating against their own paying customers who have chosen to use ODF rather than OOXML? How is this reconciled with Microsoft’s claim that they are delivering “choice, interoperability and innovative solutions to the marketplace”?

Update on ODF Spreadsheet Interoperability

2009/05/03 By Rob 33 Comments

[2009/05/07 — I’ve posted a follow up article on this topic which you may want to read]

A couple of months ago I did some experiments on the interoperability of ODF spreadsheets, the theory and practice. In that earlier post I looked at the then current ODF implementations, including:

OpenOffice.org 2.4
Google Spreadsheets
KOffice KSpread 1.6.3
IBM Lotus Symphony 1.1
Microsoft Office 2003 with the Microsoft-sponsored CleverAge Add-in version 2.5
Microsoft Office 2003 with Sun’s ODF Plugin

I created a test document in each of those editors and then loaded each test document in each of the other editors. I showed what worked, what didn’t, and made some suggestions on how interoperability could be improved. I found only two notable failures, when the Microsoft/CleverAge Add-in for Excel loaded KSpread and Symphony documents. The other scenarios I tested were OK:

		CleverAge	Google	KSpread	Symphony	OpenOffice	Sun Plugin
		Created In
Read In	CleverAge	OK	OK	Fail	Fail	OK	OK
	Google	OK	OK	OK	OK	OK	OK
	KSpread	OK	OK	OK	OK	OK	OK
	Symphony	OK	OK	OK	OK	OK	OK
	OpenOffice	OK	OK	OK	OK	OK	OK
	Sun Plugin	OK	OK	OK	OK	OK	OK

I lot has happened in the two months since I did that analysis. Several of the applications I tested have been updated:

CleverAge has released version 3.0 of their Add-in.
OpenOffice 3.01 is now out and in wide use.
Symphony 1.3 is now in beta.
The Sun ODF Plugin is now at version 3.0.
Microsoft Office 2007 SP2 has been released, with integrated ODF support.
KOffice 2.0 RC 1 is now available.

I haven’t been able to get the release candidate of KOffice installed, so I’m still including KSpread 1.6.3 in my tests, but for the rest I have created new test files in each editing environment, saved them to ODF format and then loaded the resulting documents into each of the other editors. From these test documents I was able to perform 42 different test combinations.

I’ll explain a bit more how I tested, then give you the table of results, and finally make some observations and recommendations.

The test scenario I used was a simple wedding planner for a fictional user, Maya, who is getting married on August 15th. She wants to track how many days are left until her wedding, as well as track a simple ledger of wedding-related expenses. Nothing complicated here. I created this spreadsheet from scratch in each of the editors, by performing the following steps:

Enter the title in A1 “May’s Wedding Planner” and increased font size to 14 point.
Enter formula = TODAY() in B3 and set US style MM/DD/YY date format/
Enter the date of the wedding as a constant in cell B4, also setting date format.
Added simple calculations on cells B6-B8, to calculate days, weeks and months until the wedding.
A11 through E16 is a simple ledger of the kind that is done thousands of times a day by spreadsheet users everywhere. Once you have the formula set up in column E (Balance = previous balance + credits – debits) then you can simply copy down the formula to the new row for each new entry.

The resulting spreadsheet looks something like this:

Feel free to download a zip of all of the test spreadsheet files. The file names should be self-explanatory.

Here is what I found when I tested the various scenarios:

		Google	KSpread	Symphony	OpenOffice	Sun Plugin	CleverAge	MS Office 2007 SP2
		Created In
Read In	Google	OK	OK	OK	OK	Fail	OK	Fail
	KSpread	OK	OK	OK	Fail	Fail	OK	Fail
	Symphony	OK	OK	OK	OK	OK	Fail	Fail
	OpenOffice	OK	OK	OK	OK	OK	OK	Fail
	Sun Plugin	OK	OK	OK	OK	OK	OK	Fail
	CleverAge Plugin	OK	OK	OK	OK	Fail	OK	OK
	MS Office 2007 SP2	Fail	Fail	Fail	Fail	Fail	Fail	OK

So what is happening here?

CleverAge appears to have heeded the advice from my earlier blog post and now correctly processes KSpread and Symphony spreadsheets. This is great news and they deserve credit for that work. But this is a small bit of good news in a table that now shows awful lot of red. Let’s see if we can figure this out.

First, some combinations that worked previously, when I tested two months ago, are now not working:

Symphony 1.3 beta hangs when attempting to load the spreadsheet created with the CleverAge 3.0 ODF Add-in. Symphony 1.1 also hangs when trying to load that same spreadsheet. However both versions of Symphony work fine when loading the CleverAge 2.5 spreadsheet from two months ago. The CleverAge document appears to be valid, so my guess is that this is a bug in the Symphony 1.3 beta. I’ll pass this document on to the Symphony development team to see what they say.
KSpread 1.6.3 does not read formulas from OpenOffice 3.01 documents. KSpread had no problems with OO 2.4 documents. The problem appears to be that OpenOffice 3.01, by default, writes out documents according to the ODF 1.2 draft which puts formulas in the OpenFormula namespace. But KSpread is expecting them in the legacy namespace. The result is that spreadsheet formulas are dropped when the document is loaded in KSpread.
In a similar way, Sun’s new ODF Plugin writes out documents according to the ODF 1.2 draft. KOffice is unable to handle these files. This also causes problems for Google Spreadsheets and the Microsoft/CleverAge Plugin for Excel, which report errors “We were unable to upload this document” and “The converter failed to open this file”.

The new entry to the mix is Microsoft Office 2007 SP2, which has added integrated ODF support. Unfortunately this support did not fare well in my tests. The problem appears to be how it treats spreadsheet formulas in ODF documents. When reading an ODF document, Excel SP2 silently strips out formulas. What is left is the last value that cell had, when previously saved.

This can cause subtle and not so subtle errors and data loss. For example, in the test document I presented above, the current date is encoded using the TODAY() spreadsheet function. If the formulas are stripped, then this cell no longer updates, and will return the wrong value. Similarly, if Maya tries to continue her ledger of expenses by copying the formula cells from column E down a row, this will cause incorrect calculations, since there is no longer a formula to copy, so she would just be copying the prior balance. In general, SP2 converts an ODF spreadsheet into a mere “table of numbers” and any calculation logic is lost.

In the other direction, when writing out spreadsheets in ODF format, Excel 2007 SP2 does include spreadsheet formulas but places them into an Excel namespace. This namespace is not what OpenOffice and other ODF applications use. It is not the ODF 1.2 namespace. It isn’t even the OOXML namespace. I have no idea what it is or what it means. Not every ODF application checks the namespace of formulas when loading documents, but the ones that do reject the SP2 documents altogether. And the ones that do not check the namespace try and fail to load a formula since it is syntactically different than what they expected. The applications essentially display a corrupted document that is shows neither the formula nor the value correctly. For example, a SP2 document, loaded in MS Office using the Sun ODF Plugin looks like this:

Similar corruption occurs when loading the Excel 2007 SP2 spreadsheet into KSpread, Symphony and OpenOffice. Google doesn’t import the document at all.

I must admit that I’m disappointed by these results. This is not a step forward compared to where we were two months ago. This is a big step backwards. Spreadsheet interoperability is not hard. This is not rocket science. Everyone knows what TODAY() means. Everyone knows what =A1+A2 means. To get this wrong requires more effort than getting it right. It is especially frustrating when we know that the underlying applications support the same fundamental formula language, or something very close to it, and are tripped up by lack of namespace coordination. Whether it is accidental or intentional I don’t know or care. But I cannot fail to notice that the same application — Microsoft Excel 2007 — will process ODF spreadsheet documents without problems when loaded via the Sun or CleverAge plugins, but will miserably fail when using the “improved” integrated code in Office 2007 SP2. This ain’t right.

I have some suggestions for how to move things forward again. There will be a lot less red on the above table if two simple changes are made:

Sun should write out formulas in ODF 1.1 format, using the legacy “oooc” namespace prefix that the other vendors are using. Remember, the other vendors are using that namespace specifically for compatibility with OO’s ODF documents. This is the current convention. To unilaterally switch, without notice or coordination, to a new namespace, is not cool. When ODF 1.2 is an approved standard, then we all can move there in a coordinated fashion, to cause users minimal inconvenience. But the above table clearly shows the confusion that results if this move is not coordinated. I know OO 3.01 has an option to save in ODF 1.0/1.1 format. IMHO, this setting should be the default. I’m not sure if the Sun Plugin has a similar configuration option, but I hope it does.
In addition to writing out compatible formulas as per the above comments on the Sub Plugin, Microsoft should remove the code in SP2 that causes it to reject every other vendor’s spreadsheet documents. Give the user a warning if you need to, but let them have the choice.

Finally, let me try to anticipate and debunk some of the counter-arguments which might be raised to argue against interoperability.

First, we might hear that ODF 1.1 does not define spreadsheet formulas and therefore it is not necessary for one vendor to use the same formula language that other vendors use. This is certainly is true if your sole goal is to claim conformance. If your business model requires only conformance and not actually achieving interoperability, then I wish you well. But remember that conformance and interoperability are not mutually exclusive options. An application can be conformant to a standard and also be interoperable, if you use the legacy formula namespace and syntax. So the desire to be conformant is not an excuse for not also being interoperable, or at least not a valid excuse. One might also wryly note that Microsoft has several Directors of Interoperability, not Directors of Minimal Conformance, and they workshops are called Document Interoperability Initiatives, not Minimal Conformance Initiatives. The difference between minimal conformance and interoperability is well illustrated in these tests.

Remember, it is not particularly difficult or clever to to take an adverse reading of a standard to make an incompatible, non-interoperable product. Take HTML, for example. It does not define the attributes of unstyled (default) text. So I could create a perfectly conformant browser implementation that makes all default text be 4-point Zapf Dingbats, white text on a white background. It would conform with the standard, but it would be perfectly unusable by anyone. If you try hard enough you can create 100% conformant, but non-interoperable, implementations of almost most standards. Standards are voluntary, written to help coordinate multiple parties in their desires for interoperability. Standards are not written to compel interoperability by parties who do not wish to be interoperable.

(A side point is that SP2’s implementation of ODF spreadsheets does not, in fact, conform to the requirements of the ODF standard, but that is another story, for another blog post.)

We might also hear concerns that supporting other vendors’ ODF spreadsheet formulas cannot be done because this formula language is undocumented. The irony here is that the formula language used by OpenOffice (and by other vendors) is based on that used by Excel, which itself was not fully documented when OpenOffice implemented it. So an argument, by Microsoft, not to support that language because it is not documented is rather hypocritical. Excel supports 1-2-3 files and formulas and legacy Excel versions (back to Excel 4.0) neither of which have standardized formula languages. Why are these supported? Also, the fact that the Microsoft/CleverAge add-in correctly reads and writes the legacy ODF formula syntax shows not only that it can be done, but that Microsoft already has the code to do it. The inexplicable thing is why that code never made it into Excel 2007 SP2.

We’ll probably also hear that 100% compatibility with legacy documents is critical to Microsoft users and that it is dangerous to try to save Excel formulas into interoperable ODF formulas because there is no guarantees that OpenOffice or any other ODF application will interpret them the same as Excel does. So one might try to claim that Microsoft is protecting their customers by preventing them from saving interoperable spreadsheet formulas. But we should note that fully-licensed Microsoft Office users have already been creating legacy documents in ODF format, using the Microsoft/CleverAge ODF Add-in. These paying Microsoft Office customers will now see their existing investment in ODF documents, created using Microsoft-sanctioned code, get corrupted when loaded in Excel 2007 SP2. Why are paying Microsoft customers who used ODF less important than Microsoft customers who used OOXML? That is the shocking thing here, the way in which users of the ODF Add-in are being sacrificed.

If you are cynical, you might observe that if Excel 2007 SP2 allowed Microsoft/CleverAge ODF Add-in formulas to work correctly, then SP2 would need to allow all vendors’ formulas to work, since the other vendors are using the same legacy namespace. The only way for Microsoft to make their legacy ODF documents work and to exclude other vendors would be (hypothetically) to specifically look in the document for the name of the application that created the document, and allow their ODF Add-in but reject OpenOffice, etc. IANAL, but I think something like that would look very, very bad to competition authorities. So the only way out, if your goal (hypothetically) is to avoid interoperability, is to sacrifice your existing Office customers who are using the Microsoft/CleverAge ODF Add-in. It serves them right for not sticking to the party line in the first place. This’ll teach ’em good.

Of course, I am not that cynical. I was taught to never assume malice where incompetence would be the simpler explanation. But the degree of incompetence needed to explain SP2’s poor ODF support boggles the mind and leads me to further uncharitable thoughts. So I must stop here.

As I mentioned before, this is a step backwards. But it is just one step on the journey. Let’s look forward (and move forward). This is just code. Code can be fixed. We know exactly what is needed to have good interoperability of spreadsheet formulas. In fact most of the code already exists for this. The only thing we need now is to actually go do it and not get too far ahead, or lag too far behind from the other implementations. This is more a question of timing and coordination than hard technical problems.

[2009/05/07 — For more on this topic, see my “A follow-up on Excel 2007 SP2’s ODF Support“]