Archives for May 2007

We’ve all heard the interoperability hype. Let’s see what is actually there.

First, we start by looking at the many ways in which documents are integrated into the Windows/Office platform. Any fluent user of this platform will use many of these capabilities on daily basis. These are basic features which have been around, in some cases, since Windows 3.0, maybe earlier.

Windows shell integration

1. Double-click on a document on the Desktop or in a folder and it loads into the appropriate application. Double-click on a Word document and it loads in Word.
2. Right-click in a folder and choose “New XXX” to create a new XXX document in the specified folder. So, “New…Microsoft Office Excel Worksheet” creates a new, blank Excel document.
3. Right-click on a document, choose Properties and on the Summary tab you can view metadata for that document.
4. Recently-edited documents appear in the “My Recent Documents” under the Start menu.
5. Documents referred to in web pages, via URL links will render in an inline Office session in the browser.
6. Documents are indexed by the Windows search engine.

Office integration

1. Ability to File/Open, File/Save and File/New a document via the familiar menu options.
2. Ability to set a file format as the default file format for the application.
3. Ability to use the familiar keyboard shortcuts, Control-O and Control-S to open and save documents.
4. Ability to forward a document to someone in an email and for them to be able to launch the a document by clicking on it when received via email.
5. Ability to password protect a document.
6. Ability to post a document to a web folder or to a SharePoint server

It must be noted that none of the above integration points are allowed by the ODF Add-in for Word, the much-touted translator for which Microsoft provides the, “Funding, Architectural & Technical Guidance and Project co-coordination”.

Instead what we get is a new menu option added to the Word 2007 Office menu:

Note that this is parallel to, but not included in the Open menu where the formats that Word natively understands are accessed. Although the option presented here says, “Open ODF”, it should more properly be called “Import ODF”, for reasons which will be clear shortly.

After selecting an ODF document to open, the following progress bar is given while the conversion takes place:

This is followed by a warning dialog listing elements which may have been lost in conversion:

No option is given for disabling the above message from displaying. It should be noted that when converting from a legacy binary document to OOXML, Word gives a similar conversion warning dialog, but their version can be disabled by checking a “Do not ask me again” dialog.

Once loaded, the user will find that their document is no longer an ODF document. It has been automatically converted to a read-only OOXML DOCX file as the title bar reveals:

So any future operations the user performs on the document, such as mailing, saving, posting to a web server, etc., will be in OOXML format. The only way to get back to an ODF format file is to manually and explicitly go back to the Office menu, go to the ODF submenu and choose to save it to ODF format. At that point you will be presented a default name based on the DOCX temp file name, not the original name. In this case, it suggested “sampler_tmp1.odt”.

The “Save as ODF…” dialog will default to the directory last used to save a file, not necessarily the same as where your document was loaded from. So to save you must first navigate to your original document, select it and choose “yes” when warned about overwriting an existing document, and then the document is converted back into ODF format.

If you do further work on the document in Word, in that same session, and then want to save again, you must avoid the natural tendency to do a Control-S or to save the document when prompted when existing Word. These methods all will lead to a Save As dialog, suggesting an OOXML format, which will prompt you to rename the document since it is read-only. But it will not offer you the choice of saving to ODF format. The only way to ensure that you are saving to ODF format is to use the above steps, going back to the ODF menu, etc.

You cannot create a new ODF document from scratch in Word. If you try to create a new document and save it to ODF format, you will get an error message, telling you that you must first save the document. You must save the document before you can save it? Yes, you must first save it to a temp file in a natively-supported format like DOC before you can save it as ODF.

The difficulties are complicated when you have documents accessed by other means than the Word menus. Imagine that you receive an ODF document in an email which you want to edit and return to the sender. The following steps would be required:

1. Manually detach and save your hard drive the ODF document from the email, since you will not be able to launch it directly into Word from your email client. Remember where you detached the document.
2. Manually launch Word, since you will not to get Word to launch by clicking on the ODF document you just detached.
3. From the ODF menu, choose to open the ODF document. Navigate to where you detached the emailed document and select it. Around 30 seconds later the document will be automatically converted to an read-only temporary OOXML document.
4. Make your editing changes.
5. Export the document back to ODF format using the ODF menu, either writing over the original file you extracted from the email, or to a new temporary file. Remember where you exported the ODF document to.
6. Go back to your email application and attach the ODF document.

If this had been an OOXML document (or any other format that Microsoft really supports, like RTF) it would have been much simpler:

1. Double click on the attachment in your email to automatically launch in Word
2. Make your editing changes
3. Use the Send/Email menu option in Word to send the email

As you can see the ODF support provided by the Add-in is very unfriendly.

Compare this to the OOXML support Microsoft added for older versions of Word via their Compatibility Pack. The OOXML support is tightly integrated with the UI, in a way users would find familiar and easy to use. But the ODF support is very shallowly integrated, amounting to little more than a menu item patched in.

One wonders if Microsoft’s intent was really to annoy users? That would best explain the available evidence. It is simply not credible that anyone at Microsoft believes that they are listening to customers or providing interoperability with a feature that defies real-world use. What customers did they talk to that said that this Add-in was even remotely adequate?

Since Microsoft is the one providing the, “Funding, Architectural & Technical Guidance and Project co-coordination” one would think that they would contribute more in the area where they are uniquely qualified to assist, the full and native integration of the ODF support into Office.

The idea was prompted by a comment a reader submitted to a recent post, where he talked about one of the challenges of trying to bridge two different formats (in this case ODF and OOXML) via translation:

Both formats must evolve their new versions simultaneously in locksteps…

This one is the killer. Trying to have two formats permanently synchronized this way is a maintenance nightmare, especially when we discuss standards with multiple implementations maintained by different organizations.

This is an important point, and bears some reflection. In my mind I have images of the funnel and the wedge, physical means of convergence and divergence. Similar forces are at play with standards.

We see the Funnel in the evolution of HTML. Although the standard has existed for over a decade, implementation support for HTML and related standards was uneven until quite recently. Interoperability was poor. From the start vendors added incompatible extensions while not implementing key features. Developers had to write extensive workarounds and alternative representations to work on all browsers. And when they did not that, their web sites might not work with all browsers. But with customer demand and prodding from groups like the Web Standards Group, the interoperable support of the HTML standard across implementations happened. There was convergence. What we have today, although not perfect, is clearly the result of a Funnel, concentrating industry effort around a single standard (or more like a family of standards).

This does not mean that vendors needed to sacrifice innovation, or deny their customers. It just meant that they accomplished their business objectives while also complying with standards. Along with adhering to various financial and securities regulations, labor law, health and safety, and other requirements, both internal and external, voluntary and mandatory, the browser vendors now complied with web standards. It is just another part of doing business.

Similar Funnels have occurred historically with network protocols, wireless telephony (in Europe at least), electrical grids, broadcast formats, etc. I’ve written elsewhere about what types of technologies tend to converge like this and why.

We see the Wedge when two standards compete in the same space and diverge into incompatible technologies. Microsoft is the master of the Wedge, with numerous examples over the years, usually proprietary, but more recently attempting to gain de jure recognition of them. But the mechanism is the same in either case: VML, JScript, MS Kerberos, J++, C++/CLI, XPS, and of course OOXML. Standardization just means that Microsoft has another tool for telling you that the Wedge is good for you. But it is a Wedge nevertheless.

The Wedge brings fragmentation, confusion and lack of interoperability, attacking the core reasons for having a standard in the first place. Once the primary value of an open standard is eliminated, we can all return to the security and comfort of our monopolist overlords. That is their main goal. Make no doubt about it, true interoperability and true choice are very scary propositions for Microsoft. It cuts at their very business model.

So consider the Funnel and the Wedge as applied to document formats. If we all use ODF today, is interoperability perfect? No. Do we know how to move forward to improve interoperability, and work together in multi-vendor consortia to perfect this. Yes, certainly. That is why and how such standards as TCP/IP, HTTP or HTML, work today. Interoperability came via the Funnel, a convergence of effort and attention leading to increased interoperability and the user and industry benefits that flow from that interoperability.

But from the Wedge, what can we expect? If Microsoft is successful, here’s what I see, my dismal predictions:

1. Within 30 days after OOXML is approved by ISO we see the demise of Microsoft’s half-hearted attempt to create ODF Add-ins for Office. We’ll never see a functional Add-in from them for Excel or PowerPoint, and the Word one will remain unacceptably slow.
2. Microsoft will continue to evolve OOXML behind closed doors. 99% of the work will be based on product and decisions in conference rooms in Redmond, which will be later rubber stamped by Ecma and ISO.
3. OOXML and ODF will continue to evolve and diverge, in incompatible ways.
4. Seeing their success ramming through 6,000 page Fast Track submissions in ISO, Microsoft will follow up with similar fast track submissions for XPS, XAML, Silverlight, Windows Media Photo, whatever they have. Since they have taken the trouble to set up the machinery to dominate JTC1, they will continue to force feed them with additional material.
5. Every jurisdiction where ODF is currently allowed and mandated will also allow or mandate the use of OOXML. This in practice will be turned around to mandate the use of Windows and Office.
6. Finally, once all opposition is rendered harmless, they can shut down OpenOffice.org and KOffice by patent lawsuits, but keep Novell’s version around in order to keep anti-trust regulators away. After all, 97% market share is not the same as 100%.

Maybe I’m a bit pessimistic, but I see little reason for optimism.

So do we have an alternative to the Wedge? What would encourage the Funnel? The following would need to happen:

1. ISO must reject OOXML.
2. Customers, from private and public sectors, must make their voices heard, that they want true interoperability and choice and that this means a single document format.
3. Microsoft must support the existing ODF completely and fully in Office. It won’t happen overnight. But it won’t happen at all unless they start.
4. OASIS must work with Microsoft (and Microsoft with OASIS, of course) so that that it is clearly explained how MS Office can fully represent their documents in ODF. This need not be a monolithic monster like OOXML, but should be a layered standard, with a basic core feature set and defined extensions and profiles that encompass wider and wider ranges of functionality. If Microsoft absolutely needs the “heebieJeebies” Art Border in Word in order to maintain 100% fidelity with legacy documents, then the ODF TC can show Microsoft how to encode this in ODF. The Funnel starts when Microsoft abandons their divergent effort in Ecma and joins the common effort around ODF, a single document format for personal productivity applications.
5. The application vendors, Microsoft included, must work together on defining the organizational, standards and technical means necessary to measure, test and certify ODF compliance, so customers and procurement agencies are able to have assurances that they are getting the level of interoperability that they desire.

I think this is a natural progression. Accomplishing the first step stops the Wedge from progressing further, halting but not reversing the divergence. The other steps reverse the damage and turn us down a path of true interoperability, leading to true choice and innovation.

Finally note that the Wedge is typically driven by a single company. It is not a pull by public demand or from customers, though it may wear many disguises. It is a deliberate attempt by one party to cause division and divergence. But a Funnel, this won’t happen at all unless there is strong demand, from customers, from government agencies, from national standards committees, etc. If this is to happen, your voice must be heard. All of us must work to bring all of us together in this effort. But it takes just one company, with a sufficiently large Wedge to pull us apart.

Google has a nice feature that allows you to search for documents that match a given file type. This is done by adding “filetype:NNN” to your query, where NNN corresponds to the file type. This feature has supported the ODF and OOXML document formats for at least two months, when I first noticed it. I’ve been tracking some numbers since then and now have enough data to make some observations.

At last count the totals were:

As you can see, there is some round-off happening on the upper range. Perhaps at the high-end counts are estimates based on sampling?

In any case, I am rather surprised by the low counts given for OOXML documents, especially considering that this format has been supported since the Office 2007 beta last summer. According to Brian Jones, there have been over 4 million downloads of the OOXML Compatibility Pack for older versions of Office, and that there is a new community of, “over 300 other companies and partners who care deeply about OpenXML”. We’re also told that Office 2007 sales are above expectations, “two times greater than the purchases of Office 2003” according to one research firm. Recently announced third-Quarter results for Microsoft showed “better than expected” results for Office 2007 sales, $200 million better, according to Microsoft CFO Chris Liddell.

So with all this evident love for Microsoft Office 2007, why is it that 6-months later there are only 63 OOXML spreadsheet documents on the web, something like 0.3% of the number of ODF spreadsheet documents? How can there be 300 companies supporting OOXML and only have 69 OOXML presentations on the web? (This is starting to sound like when I say I support 30 minutes of aerobic exercise a day. I don’t do it, but I sure support it!)

OK, I know the argument about “dark matter”, that Google indexes only the tip of the iceberg, that there is a lot of data squirreled away on PC hard-drives, behind corporate fire walls, etc., stuff that Google will never see. But the same is equally true for ODF documents, right? I have tons of ODF documents on my laptop, but none of them are indexed by Google.

Of course ODF has been around for a year longer than OOXML. That’s an important fact to acknowledge. We can put that in perspective by plotting the graph of ODF and OOXML document counts against the number of days since adoption of these two standards. So ODF counts are based on a start of 1 May 2005 and OOXML starting in 7 December 2006, when OASIS and Ecma respectively approved them. You get this:

As you can see, ODF has a nice upward trend. OOXML is also trending upwards, though it is somewhat lost at this scale. If you do the analysis it comes out to around 300 new ODF documents per day versus 6 for OOXML. So, two years later, ODF adoption, in terms of documents per day, is 50-times greater than OOXML is, at a time which should be OOXML’s high-growth period, considering all the great news that is coming out of Redmond.

So I’m a somewhat at a loss to appreciate the significance of Novell or Corel adding OOXML support to their editors. With only 63 OOXML spreadsheets out there, wouldn’t it be cheaper just to hire someone to retype the documents in the destination application? The average user is more likely to find a Buffalo Nickel in their lunch change than to find an OOXML document outside of captivity.