OOXML

So where are all the OOXML documents?

2007/05/10 By Rob 29 Comments

Google has a nice feature that allows you to search for documents that match a given file type. This is done by adding “filetype:NNN” to your query, where NNN corresponds to the file type. This feature has supported the ODF and OOXML document formats for at least two months, when I first noticed it. I’ve been tracking some numbers since then and now have enough data to make some observations.

At last count the totals were:

Format	Count
ODT	85,200
ODS	20,700
ODP	43,400
Total ODF	149,300
DOCX	471
XLSX	63
PPTX	69
Total OOXML	603

As you can see, there is some round-off happening on the upper range. Perhaps at the high-end counts are estimates based on sampling?

In any case, I am rather surprised by the low counts given for OOXML documents, especially considering that this format has been supported since the Office 2007 beta last summer. According to Brian Jones, there have been over 4 million downloads of the OOXML Compatibility Pack for older versions of Office, and that there is a new community of, “over 300 other companies and partners who care deeply about OpenXML”. We’re also told that Office 2007 sales are above expectations, “two times greater than the purchases of Office 2003” according to one research firm. Recently announced third-Quarter results for Microsoft showed “better than expected” results for Office 2007 sales, $200 million better, according to Microsoft CFO Chris Liddell.

So with all this evident love for Microsoft Office 2007, why is it that 6-months later there are only 63 OOXML spreadsheet documents on the web, something like 0.3% of the number of ODF spreadsheet documents? How can there be 300 companies supporting OOXML and only have 69 OOXML presentations on the web? (This is starting to sound like when I say I support 30 minutes of aerobic exercise a day. I don’t do it, but I sure support it!)

OK, I know the argument about “dark matter”, that Google indexes only the tip of the iceberg, that there is a lot of data squirreled away on PC hard-drives, behind corporate fire walls, etc., stuff that Google will never see. But the same is equally true for ODF documents, right? I have tons of ODF documents on my laptop, but none of them are indexed by Google.

Of course ODF has been around for a year longer than OOXML. That’s an important fact to acknowledge. We can put that in perspective by plotting the graph of ODF and OOXML document counts against the number of days since adoption of these two standards. So ODF counts are based on a start of 1 May 2005 and OOXML starting in 7 December 2006, when OASIS and Ecma respectively approved them. You get this:

As you can see, ODF has a nice upward trend. OOXML is also trending upwards, though it is somewhat lost at this scale. If you do the analysis it comes out to around 300 new ODF documents per day versus 6 for OOXML. So, two years later, ODF adoption, in terms of documents per day, is 50-times greater than OOXML is, at a time which should be OOXML’s high-growth period, considering all the great news that is coming out of Redmond.

So I’m a somewhat at a loss to appreciate the significance of Novell or Corel adding OOXML support to their editors. With only 63 OOXML spreadsheets out there, wouldn’t it be cheaper just to hire someone to retype the documents in the destination application? The average user is more likely to find a Buffalo Nickel in their lunch change than to find an OOXML document outside of captivity.

Math markup marked down

2007/04/25 By Rob 16 Comments

Sun’s Erwin Tenhumberg fights some FUD about ODF and in passing provides a link that is worth a few more words. It appears that Science, the journal of the America Association for the Advancement of Science (AAAS), itself the largest scientific society in the world, has updated its authoring guidelines to include advice for Office 2007 users. The news is not good.

Because of changes Microsoft has made in its recent Word release that are incompatible with our internal workflow, which was built around previous versions of the software, Science cannot at present accept any files in the new .docx format produced through Microsoft Word 2007, either for initial submission or for revision. Users of this release of Word should convert these files to a format compatible with Word 2003 or Word for Macintosh 2004 (or, for initial submission, to a PDF file) before submitting to Science.

Well, so much for 100% compatibility, eh? That is what I’ve been talking about. Whether you move to OOXML or ODF you will be making a change that will break compatibility with your past document processing systems. You will need to change over the next couple of years and you will need to examine your choices carefully. But don’t get suckered into thinking that the choice of OOXML is magically painless. The 100% compatibility claims don’t hold water.

More bad news:

Users of Word 2007 should also be aware that equations created with the default equation editor included in Microsoft Word 2007 will be unacceptable in revision, even if the file is converted to a format compatible with earlier versions of Word; this is because conversion will render equations as graphics and prevent electronic printing of equations, and because the default equation editor packaged with Word 2007 — for reasons that, quite frankly, utterly baffle us — was not designed to be compatible with MathML. Regrettably, we will be forced to return any revised manuscript created with the Word 2007 default equation editor to authors for re-editing. To get around this, please use the MathType equation editor or the equation editor included in previous versions of Microsoft Word.

Uh oh. Not only cannot you not submit files in OOXML format, but you can’t even use Office 2007 and save in the old binary formats. Down conversion or using the Compatibility Pack won’t help. Microsoft’s decision to push a new “Open Math Markup Language” rather then use the well-established MathML standard appears to be a serious flaw.

Nature appears to have the same problem:

We currently cannot accept files saved in Microsoft Office 2007 formats. Equations and special characters (for example, Greek letters) cannot be edited and are incompatible with Nature’s own editing and typesetting programs.

Of course, when targeting final publication of a paper, a PDF file is fine. But when engaging in collaboration with another researcher, or an editor, you need to agree of a standard format in which you both can work.

Reuse of existing standards is important. When you reuse a standard, you are reusing more than a piece of paper. You are reusing the experience and effort that went into creating and reviewing that standard. You are reusing the experience gathered by those who have already implemented the standard. You are reusing the books and training materials already written for that standard. You are reusing the interfaces for other technologies that have already integrated with that standard or can produce or consume output that conforms to that standard.

Isaac Newton wrote, “If I have seen further it is by standing on the shoulders of giants”. When you reuse standards you reuse the accumulated wisdom of an industry and assume the vision and powers of giants. But when you ignore all precedents and go forth on our own, well, let’s just say the outcome is more variable in that case. You may be the next Einstein, or you may be the next fool.

If Science and Nature need to update their templates, then I’d suggest they take a look at ODF. Not only does it use MathML for equations, but it is an open standard, an ISO standard, a platform and application-neutral standard that has many implementation, including several good open source ones. If they need to update their processing, then they might want to make the smart choice now, the choice that increases their choices and flexibility going forward.

18 June 2007 Update

A response from Nature and one of their vendors, explaining the complexity of migrating their publishing ecosystem to a new file format. Quoting a letter to Microsoft from Bruce Rosenblum of Inera:

Had the conversion from DOCX to DOC provided a conversion from OMML to Equation Editor format, it would have provided the necessary backwards compatibility for publishers to upgrade one system at a time. But because this compatibility is not available, it’s created the need for a “big bang” upgrade, or a delay until the ecosystem of inter-dependent systems is deliberately updated over time. In the environment of scholarly publishing, such substantive upgrades often take years, not months.

Sometimes I need to remind myself

2007/04/23 By Rob 22 Comments

Tim Anderson has an interesting article up on his ITWriting blog, “Microsoft’s Jean Paoli on the XML document debate”. Of course, I treat anything Jean Paoli says on XML with such attention as I usually reserve for listening to the isorhythmic motets of Philippe de Vitry. Like de Vitry, Paoli can be understood on several different levels: What is he saying? And what is he really saying. As a student of Empson’s “Seven Types of Ambiguity”, I hope that I am up to the task.

There is, of course, the familiar canard, that IBM is the source of all of their problems:

It is clear though that Paoli is upset by what he sees as an international campaign against OOXML orchestrated by IBM, the sole naysayer in the ECMA voting. “There are IBM employees going to ISO, and saying a lot of technically incorrect things. When ODF went to ISO Microsoft did not interfere. IBM is betting on ODF, to have governments preferentially buying IBM software. It is OK to compete, but using this kind of argument around is it an open format or not … it’s widely known now, Office Open XML is an open format, even the EU says it is.”

A Google search on the words ecma ibm sole vote returns an embarrassingly large number of hits. Microsoft has certainly been having fun with this line. Let’s take a little look at this question and see if we can better define this conspiracy that Paoli is alluding to.

I’m now going to rant a little. You may want to stand back.

Yes, IBM was the only voting member in Ecma who cast a voted against OOXML. But guess what, we’re probably the only company who actually had someone perform the due diligence of reading the specification. The others voted on OOXML without reading the spec. So please give their “Yes” votes all the weight they deserve, but not more.

It seems to me that Ecma has become a standards factory, a place where you go for clean, efficient, no-guilt, fast-track service. Don’t want to publish your public comments? Fuggetaboutit. Don’t want to publish your meeting minutes? Fuggetaboutit. Worried about rushing through a 6,000 page specification in less than a year, with 20x less scrutiny than average? Fuggetaboutit. Want to have a unanimous vote, along with with a souvenir photograph of your face when the vote occurs? Yes sir, we guarantee it.

However, for the privilege of this elite service, you must cough up the dough. You will not find Ecma’s rate card on their website, but I’m told that voting membership will set you back $57,000. This is not exactly the club to join if you are a small (or medium) business, non-profit, public sector agency, or anything but one of the big boys. A list of the privileged twenty voting members of Ecma can be found here.

As you can imagine, one does not become a voting member of Ecma without a good reason. This is a business expense, not a charitable contribution. For $57K, one expects $57K of service. To justify that membership fee, you expect your technology to be blessed with an Ecma standards imprimatur without hassles. So the “unwritten rule” is that everyone votes in favor of everyone else’s proposal. It is considered rude to vote against something that another elite member has paid so much for. So, IBM gets get a lot of grief for casting a single “No” vote at a single Ecma General Assembly. We broke the club rules. I’m proud to work for such a company.

My question is this: How many “No” votes have been cast in Ecma in the past 5 years? When before did another Ecma member ever vote “No” on a standard? If no one can remember even a single previous “No” vote, or (sacre bleu!) a defeated standard, then that speaks volumes. In a healthy standards body, a single “No” vote should not be a newsworthy event, and should certainly not be something that Microsoft is still complaining about 6 months later.

To put this in perspective, the base category of OASIS voting memberships (Contributor) starts at $1,100. OASIS has something like 330 organizational members eligible to vote, including all categories of companies, government agencies, non-profits, etc.

I should also note, just coming from the annual OASIS Symposium held last week, that the OASIS Board of Directors is looking at changing the OASIS voting rules to make it more difficult for OASIS standards to be approved. Yup, we’re raising the bar.

When I see this I need to try extra hard to remind myself that IBM is just interfering with Microsoft’s good-faith attempt to humbly submit for our consideration their well-written, detailed, high-quality, interoperable open standard.

ISO/IEC JTC1/SC34 recently had its annual plenary. This is the same group of ISO National Body (NB) members who voted in favor of ODF last year, and over the next few months many of them will be recommending positions on Microsoft’s OOXML to their national standards bodies. I was on the delegates list for attending this meeting, as a representative of the US NB, but had to cancel at the last minute because of a family emergency. When I saw the attendance list, I was surprised to see that Microsoft had sent five people, this to a meeting of only 37 people. They practically darkened the skies with their employees. And what about the conspiratorial army that is hounding them at every corner? Zero people from IBM. Zero as well for Google, Sun, RedHat, Adobe, Oracle and Novell.

When I read this I need to remind myself that I’m part of a vast global conspiracy to deny Microsoft a fair hearing within ISO. The fact that no one in this vast global conspiracy managed to show up at the meeting was simply a ploy to make Microsoft feel overconfident.

In the US NB, we have a committee called INCITS V1. It is the mirror committee to JTC1/SC34. I serve on it, the only member from IBM. Imagine my surprise, when at our last call, Microsoft shows up with 3 employees and a business partner as new members. Four people against little ol’ me? Come on guys, that is just sad.

At times like this I need to remind myself that Microsoft is the underdog and IBM and its allies are ganging up them. But our guys are invisible at meetings and although they cannot vote, they do have ninja powers and, in matters of external affairs, the delegated plenipotentiary prerogatives of Klingon Ambassadors. “choSuvchugh ‘oy’lIj Daghur neH”.

Microsoft bloggers, fed and spreading like mushrooms, recently popped up and simultaneously announced a new pro-OOXML petition, self-published, self-hosted and self-reported by Microsoft. You couldn’t find anyone to even pretend to support you? You had to host your own petition? This is like throwing a birthday party and having only your mother show up. Very sad. Where are your friends, Microsoft? How come we hear no one else speaking approvingly about OOXML? Where are the other companies lining up? Where are the endorsements? The testimonials? All we hear is that Microsoft thinks OOXML is great. But that is just Mom cheering on your performance. Don’t you have any real support?

Btw, this is what a real petition looks like. It is hosted by a reputable party (the Prime Minister) and gives a open, public listing and tally of those who signed the petition.

At times like this I need to remind myself that the ODF supports are the outsiders in this debate, using unconventional and covert tactics to fight a well-respected and well-loved mainstream technology generously provided by Microsoft.

I see that Microsoft likes to throw around names like the British Library and Library of Congress, as if the mere mention of their holy names brings sacramental blessings. But please show me a public statement where either of these bodies has endorsed, adopted, recommended adoption or recommended approval of OOXML. The mere mention in passing of well-known and popular institutions lends no credibility to your argument, and credible arguments are important, as is well-known to anyone familiar with Walt Disney World, the Louvre, NASA , the Boston Red Sox, or the Department of Really Important Stuff .

A Malaysian standards committee was moving forward to approve ODF as a national standard in Malaysia. This is called “transposing” an International Standard, and is commonly done when a relevant International Standard is approved. Microsoft has made every attempt possible to prevent this committee from making progress with their review of ODF, for almost a year now. This progress recently came to a halt, the committee’s decisions nullified and the committee suspended.

When standards committees are disbanded when they get too close to approving ODF, then I must pinch myself and remind myself once again that IBM is the one orchestrating international campaigns against Microsoft, and not the other way around.

I’ve heard similar complaints from other NB’s. Why bother reviewing OOXML? Why waste the effort reading it and suggesting improvements? Microsoft has ignored every suggestion given it so far by NB’s. And if you vote no, Microsoft will just escalate and try to get some mid-level government bureaucrat to set aside the recommendation of your country’s technical experts. What waste the next 4 months reviewing a 6,000 page specification? It happened in Malaysia. It happened in the US. The INCITS Executive Board was about to send a contradiction submission against OOXML, saying that it possibly contradicted ODF. But before the committee could reconvene the next morning, enough members had received urgent phone calls to cause them to change their vote and abstain. We saw this in the Netherlands as well, where it was even reported in the papers that they would vote against OOXML. But that vote was changed at the last minute with the cryptic message to the JTC1 Secretariat: “The Netherlands Standardization institute votes ‘abstain’. Please change our vote accordingly and please confirm receipt of this vote to me…” What happened there is still unclear. In India it was even worse, when the committee that was supposed to get the ballot did not receive it. Evidently it was misplaced. The intervention of the leader of a major national political party was required to straighten it out. I also received a note saying that the committee was being told that the deadline for responding to the ballot was two weeks later than it really was, a delay that would have invalidated their vote if they had fallen into that trap.

When I see stuff like this happening, I need to remind myself, really, really hard, that IBM is the bad guy in this debate and that we’re the one interfering with an orderly ISO process.

When an amendment to a Florida State Senate bill was offered that called for a “business case analysis” for the use of open standard document formats (no particular format was called out) Microsoft’s lobbyists, the three Men in Black, Will McKinley of Dutko Poole McKinley, Jim Daughton, Jr. and Geoffrey Becker both of Metz, Hauser, Husband & Daughton, swarmed down and zapped it. As one legislative aide put it, “By the time those lobbyists were done talking, it sounded like ODF (Open Document Format, the free and open format used by OpenOffice.org and other free software) was proprietary and the Microsoft format was the open and free one”. Perhaps a document, left by the lobbyists, filled with lies about ODF, had something to do with it? We should be fortunate that Microsoft sent only three lobbyists to handle this, rather than all nine lobbyists who are registered in Florida alone to support Microsoft’s legislative activities.

When expressing our technical opinion defines interference, and the outrages that Microsoft is getting away with become the norms of behavior, then we’re all doomed to a future of technical subservience. We all need to remind ourselves of that.

Microsoft likes to complain, and they are evidently becoming quite adept at it. If decibels and dollars could win arguments then they would surely be the winners. But I think their protestations are mis-directed. Microsoft is like an out-of-condition middle-aged man (somewhat like myself) out for a rare jog. They can curse to the high heavens the pain they feel, but don’t blame it on others. It is called competition. Deal with it. If it hurts so much it is because you are so out of practice. You should try having competition more often. It is good for you.

Cannibalism

2007/03/20 By Rob 16 Comments

A interesting post by Bob Sutor. What is OOXML’s real competition, and how does that help ODF? The dynamics get interesting when you are hindered by your own install base. The main selling point of OOXML is its claimed 100% compatibility with the legacy binary formats. But if you are using Office 2000, and happy with it, what is the reason to move to OOXML? Why not remain using the binary formats? What justifies the migration?

The downside is clear. The minute you move to OOXML you have less choice with whom you can successfully exchange documents with. Office for the Mac, Windows Mobile, WordPerfect Office, Google Docs and Spreadsheets, SmartSuite, ThinkFree Office, users of these products, and the numerous 3rd party applications that can read and write the binary formats, these are now outside of the universe of people and applications that you can exchange documents with. Despite some early attempts from Sun and Novell, Linux users are left out as well.

So why move to OOXML? From the CTO’s perspective, if your greatest concern is legacy compatibility, what is the ROI argument for changing file formats? Wouldn’t the tendency be to remain where you are?

So the breakdown may happen like this:

N% of companies put compatibility with legacy documents foremost. A% of these stay on Office/Windows and upgrade to Office 2007/OOXML. B% stay where they are and use the binary formats, and C% move to some combination of ODF and PDF.
100-N% make a decision primarily on factors other than 100% fidelity with legacy documents, such as ease of programmability, greater choice and diversity in applications and vendors, etc. X% stay on Office/Windows and upgrade to Office 2007/OOXML. Y% stay where they are and use the binary formats, and Z% move to some combination of ODF and PDF.

I think that B & Z may be the dominating factors. N is large now because it includes the inertial effects of Microsoft’s market dominance. Even companies that don’t make an explicit choice will end up with that path by default. But even the most passive company will not fall into choice A without some thought.

It is interesting to speculate on the initial percentages. But note that this is a network effect game, so the percentages will vary over time based on expectations.

Fast Track. Wrong Direction.

2007/03/13 By Rob 26 Comments

The idea was to make the C++ programming language work better in Microsoft’s .NET framework. It started off as the Managed Extensions for C++, first available in 2000, and later in Visual Studio .NET 2003. Managed Extensions were reformulated in Visual Studio 2005 where they were called C++/CLI, referring to the Common Language Infrastructure, the runtime abstraction in .NET.

CLI itself had earlier been standardized in Ecma (approved in 2000) and Fast Tracked through ISO (approved in 2001). So, it was not much of a surprise when the C++ variant for Microsoft’s .NET Framework, C++/CLI, was proposed for standardization as well. Ecma TC39/TG5 started work on C++/CLI in December 2003 and Ecma approved the specification as Ecma-372 in December 2005. Two years in committee, resulting in a 304-page specification. This used to be considered a fast pace.

After approval by Ecma, C++/CLI was submitted for Fast Track processing to ISO/IEC JTC1/SC22 as DIS 26926. Like any other Fast Track in JTC1, this process started with a 30-day contradiction period. Contradiction submissions were made by both Germany[pdf] and the UK[pdf].

The UK’s position was that calling the standard “C++/CLI” would cause, and in fact was already causing, confusion among users with the already existing C++ programming language. The name of the standard was unacceptable:

We consider that C++/CLI is a new language with idioms and usage distinct from C++. Confusion between C++ and C++/CLI is already occurring and is damaging to both vendors and consumers.

A new language needs a new name. We therefore request that Ecma withdraw this document from fast-track voting and if they must re-submit it, do so under a name which will not conflict with Standard C++.

Similar views were expressed by Germany:

With reference to §13.4 of the JTC1 Directives, 4th edition, DIN brings to the attention of the JTC1 secretariat that we perceive a contradiction between document JTC 1 N 8037 “30 Day Review for Fast Track Ballot ECMA-372 1st edition C++/CLI Language Specification”and the JTC1/C++ standard ISO/IEC 14882:2004 “Programming language C++” and related technical reports.

We propose that the document is input into SC22 as a regular New Work Item Proposal and assigned to WG21 for further processing.

Ecma responded[pdf] to these objections in a 5-page letter, on 29 January 2006, that refused to make even the most basic concession, such as changing the name to remove the C++ reference.

So the objections are ignored, and they move on to the 5-month ballot period, starting March 9th, 2006. When the ballot closed in August, and the votes were counted, C++/CLI had received 11 out of 20 P-Member votes (55%) and a total of 9 negative votes out of 26 total votes cast, or 34.61%. So it failed both to get the required 2/3 approval of P-Members, as well as to keep the negative votes to less than 25%.

Germany and the UK voted disapproval. No surprise there, since they had objected early in the process, and their objections were ignored. In fact one of Germany’s comments in the ballot was:

DIN has commented before, as well as BSI did, that allowing fast-track standardization of the “C++/CLI Language” under this name clearly conflicts with an existing and actively maintained standard: ISO 14882 – the C++ Programming Language. The document under review spells out under “NOTE FROM ITTF”, bullet 2.2, that ITTF will ascertain that this proposed standard does not conflict with any other International Standard but such a conflict was pointed out. No reason has been given why this objection was overridden. Thus, DIN wants to express its surprise that standardization of this proposal went forward.

The US comments included:

The proposed standard is not market driven, nor is it the product of an industry consensus.

We are unimpressed with the very low level of C++ community participation mustered in the design and refinement of the current document, and feel, quite frankly, that the current state of this document is not at a high enough level of technical excellence to merit the ISO imprimatur.

France said:

This document should be withdrawn from the fasttrack approval process pending re-drafting and a more adequate review prior to voting. Better yet, retain it as an Ecma standard only until a clear market consensus develops that a JTC1 standard in this area is needed.

And so on, down the list.

It should be noted that a failing vote in the 5-month ballot is not necessarily fatal. The Fast Track submitter, in this case Ecma, can call on the SC Secretariat to convene a Ballot Resolution Meeting (BRM), where the issues can be discussed and resolved, possibly leading to a positive vote after a further ballot. This is Ecma’s right as a Fast Track submitter. However, C++/CLI did not see a ballot resolution meeting. The JTC1 Secretariat recently notified SC22 members:

We have been advised that the comments accompanying the Fast Track ballot for DIS 26926 are not resolvable and that holding a Ballot Resolution Meeting (BRM) would not be productive or result in a document that would be acceptable to the JTC 1 National Bodies. Therefore, our proposal is to not hold the BRM and to cancel the project.

So, the BRM which had been scheduled for April, 2007 has been canceled, and that’s where it stands today, with the attempted Fast Track of C++/CLI dead from seemingly easily preventable flaws.

Lessons, anyone?

Don’t ignore NB members. If they take the time and make the effort to point out your flaws early in the process, then you should count yourself lucky. This is like the school teacher walking around the classroom during a quiz and pointing to one of your answers and saying, “You might want to take another look at that problem”. If you ignore her advice and just turn in your paper, then you deserve the grade you get.

It is instructive as well that although only two NB’s objected in the C++/CLI contradiction period, this grew to a far larger number by the time the 5-month ballot had ended. Ignoring problems doesn’t make them go away.

One last thing. Any guesses on how long those contradiction arguments stay online before they are taken down to preserve the shrouded secrecy of ISO process? I advise you to make a copy now. I certainly have.