Archives for 2009

The Final OOXML Update: Part III

2009/10/27 By Rob 9 Comments

This is Part III of an 5-part series on the state of OOXML today. Previous to starting this series, I had not posted about OOXML in over a year. Part I showed how Microsoft, despite their promises that control of OOXML would be handed over to an independent, international committee, have instead stuffed the committee that maintains OOXML (JTC1/SC34/WG4) with Microsoft employees. And in Part II I looked at how the final published text of OOXML failed to account for all BRM decisions, and described the steps that ISO was taking to remedy this obvious procedural flaw.

In this Part I’ll look at how Microsoft is using their dominance in SC34 to push through hundreds of changes and additions to OOXML, in a misuse of a procedure intended for correcting drafting errors, to make OOXML “conform” to Microsoft’s monopoly product.

Let’s start by taking a look at the OOXML defect log [PDF] that SC34/WG4 uses to track their large list of errors and omissions discovered in the published standard. This defect report currently amounts to over 800 pages, longer than the entire ODF 1.0 standard. But it is well worth downloading and browsing through.

Some of these changes will be made in Technical Corrigenda while others are proposed for Amendments. What is the difference? SC34/WG4 itself made the distinction clear, in a presentation (N 1187 for those with access) it made to the SC34 Plenary in Prague, where it outlined its practice for deciding which changes would be made in corrigenda versus amendments:

All of the following criteria should be met for the defect to be resolved by Corrigendum:

1) WG 4 agrees that the defect is an unintentional drafting error.
2) WG 4 agrees that the defect can be resolved without the theoretical possibility of breaking existing conformant implementations of the standard.
3) WG 4 agrees that the defect can be resolved without introducing any significant new feature.

Unless all the above criteria are met, the defect should be resolved by Amendment.

These are reasonable criteria and no objections were made when these guidelines were presented to SC34.

A key procedural point is that in ISO/IEC it is the JTC1 NBs who are the consensus body that has the authority to create international standards. All ballots which create or substantial modify standards must be approved by JTC1. This includes DIS ballots, FDIS ballots, FDAM ballots and DTR ballots. So standards, technical reports and amendments are ultimately approved or disapproved by JTC1 NBs. Although subcommittees in JTC1, such as SC34, provide the technical expertise and author and review work, they are not the standardizing authority. The exception that proves the rule is with corrigenda, which are authored and approved entirely at the SC level. However, this small area of autonomy in defect correction comes with carefully delineated bounds. A SC can author, approve and publish corrigenda by itself, but only to make corrections.

So if we look at JTC1 Directives 15.4.2.2, we read (with my emphasis) “A technical corrigendum is issued to correct a technical defect…. Technical corrigenda are not issued for technical additions which shall follow the amendment procedure…”. And in 15.4.1 “technical addition” is defined as: “Alteration or addition to previously agreed technical provisions in an existing IS.”

So amendments, which require approval by JTC1, are used for altering or extending the provisions of a standard, while corrigenda are used to correct errors introduced in drafting or publication. This dichotomy is common in other standards organizations. For example, in OASIS, a technical committee is able to approved and publish “Approved Errata” but these are restricted to changes that do not break conformance of existing implementations. Anything beyond that is considered a substantive change to the standard and requires review approval by the OASIS membership.

Clear enough? In fact, in many cases WG4 appears to follow this important distinction. Some of the proposed changes are simple and benign. For example, some BRM issues were fixed, but in being fixed caused informative example markup in the standard to be incorrect. A quick fix of these items via corrigenda is most welcome.

However, in other cases (in fact most of the cases), the Microsoft-dominated WG4 appears to have overstepped the permissible bounds for corrigenda, and indeed gone far, far beyond what it stated it would be doing in corrigenda. Let’s look at a few examples.

(Sadly, the general public is not given access to the text of the draft corrigenda (the DCOR) but those on the inside can follow along by reading N 1252 in the SC34 document repository.)

Let’s start by looking at items 16, 17, 36, 52, 53 and 133 in DCOR for ISO/IEC 29500 Part 4. These make changes and additions to the WordProcessingML schema. Deletions are noted in red strikethroughs, and additions in blue:

This is not correcting a drafting error. This is not correcting a publishing error. This is a substantial addition to the schema as you can see above.

It is argued, in the defect log, that this change is needed because, without it, ISO/IEC 29500 cannot represent change tracking in mathematical equations. However, this is exactly the type of change that WG4’s guidelines and JTC1 Directives exclude from corrigenda and place into amendments. The schema of OOXML is certainly an “agreed technical provision of an existing IS”. So how can adding math change tracking support to the schema be anything other than an “addition to previously agreed technical provisions”? And how can anyone in WG4 believe that adding dozens of lines to the schema can be done “without the theoretical possibility of breaking existing conformant implementations of the standard”? What about, for example, applications that were programmed to use the published OOXML schema, such as any application that uses a validating parser, or an schema-directed editor, or a program that generates code stubs from the schema, or does XML-to-relational DB mapping? Not only is there a theoretical possibility of breaking such applications, there is a theoretical certainty.

(Ironically, it should be noted that Microsoft was very keen to beat up on ODF for not having change tracking for mathematical equations, all while hiding the fact that OOXML lacked complete support for this feature as well.)

Another example, #122 in the DCOR.

It changes a type in chart, from a byte to an int and in doing so extends its allowed range considerably. How did anyone think that this was a change that was “without the theoretical possibility of breaking existing conformant implementations of the standard”? Isn’t there enough theoretical and practical expertise in WG4 to know that changes like this break compatibility?

For this change the rationale in the defect log explains the logic of it:

The standard states that the ST_Period simple type uses the XML Schema ST_Period data type and supports a range 2–255.

These observations are incompatible with existing documents and should be updated to reflect such prior art.

And so on and so on. If you search through the defect log, you will see the phrase “existing documents” used dozens of times. That appears to be how many discussions in WG4 end. It shuts down debate like an appeal to “national security” or “executive privilege”, arguments that trumps all others. It doesn’t matter what WG4 previously told SC34, or what JTC1 Directives say, if ISO/IEC 29500 does not match what Microsoft Office actually writes out, then this is by definition a drafting error, and the standard will be “corrected” to conform with MS Office. Let that sink in for a little, until you realize how backwards this is.

I invite you to go back to the defect log [PDF] and search for “BRM”. You will find several oddities. For example, among these proposed changes are some that actually reverse BRM decisions. Yes, you heard me correctly. SC34/WG4, the Microsoft-dominated committee that maintains OOXML, is undoing various BRM decisions that enabled OOXML to be approved in the first place. Why? Well, of course, to make the standard conform more to Microsoft Office.

For example, take DR 09-0159 “General: Unintended incompatibilities between Transitional schema and Ecma-376” or DR 09-0275 “BRM: serial date representation” with this comment:

Although this text is in accord with the detailed amendments resolved at the BRM, it is against the spirit of the desired changes for many countries. We believe that due to time limitations at the BRM, this change was made without sufficient examination of the consequences, and was made in error by the BRM (in which error the UK played a part).

(Norbert Bollow, a member of the Swiss NB, has some good analysis of the return of the leap year bug in spreadsheets. And Jomar Silva with the Brazilian NB tracks some additional breaking changes on a wiki.)

Ah, So WG4 is now interpreting the “spirit of the BRM” through their shamanic communion with the ISO Weltgeist, and each time their oracle come back with the same response: “Change OOXML so it ‘conforms’ to Microsoft Office 2007”. How convenient for Microsoft.

For most standards, multiple vendors work together to improve interoperability and to increase their conformance with the standard. But with OOXML a single vendor stuffs the committee and works to make the standard better conform to Microsoft’s monopoly product.

So although Microsoft Office does not conform to ISO/IEC 29500 today, I have no doubt that within a few months it will fully conform. But not a single line of code will have changed in the Office product. Office 2007 will be retroactively made to conform to ISO/IEC 29500. What will happen is the standard will be modified to match that single vendor’s products, by misapplication of an ISO procedure intended for fixing minor drafting errors.

So why go through all this trouble? I believe this is all about getting the OOXML standard “corrected” so Microsoft can push for it to get it officially adopted around the world. The only reason they’ve held back so far is because MS Office does not actually implement ISO/IEC 29500 today. So it would have been counter productive for them to push for official adoption. However, once this oversight is remedied, by changing the standard to match their product, then watch out.

The side effect, perhaps unintended, is that the OOXML standard is thus clearly marked to be unstable and unsuitable for adoption or implementation. With 800 pages of defects and more being found, and a Microsoft-dominated committee that changes the standard with no objective technical justification, the exact contents of the OOXML standard is tentative, uncertain and temporary. Four corrigenda documents and two amendment documents are currently being balloted, including many breaking changes. More corrigenda and amendments are on the way. There is no provision for a version attribute or any other indicator to declare which of the multiple incompatible versions of the standard a document conforms to. What competitor would risk implementing the standard, knowing that Microsoft dominates WG4, which has shown it is willing to change the standard at Microsoft’s whim? The risk is simply too large. A competitor would simply be putting their head in the lion’s mouth.

And at the same time WG4 rushes to make OOXML conform to Office 2007, Microsoft is moving on with Office 2010, now in technical preview. Office 2010 will be extending OOXML in hundreds of places. Where is SC34 in this? Where is the new work proposal for OOXML 1.1? Where the are discussions? The drafts? None of this exists. If Microsoft wanted to, they could have submitted these changes to SC34 at the recent meeting in Seattle, but they preferred to reserve discussion of the Office 2010 changes for a private meeting in Redmond the day after the SC34 Plenary ended, a snub to SC34 and their fictional control of OOXML.

So Microsoft is now off extending OOXML, and this whole ISO escapade with OOXML seems for naught. (I hear also that Microsoft is also backing off the submission of their Extensible Page Specification (XPS) to ISO as well, saying that “an Ecma Standard is good enough”.) It appears that Microsoft got what they wanted from ISO and is moving on. Who said it would last more than a night? As my grandmother used to say, “Why buy the cow when you can get the milk for free?”

In any case, the future looks like something like this:

ISO/IEC 29500:2008’s future is uncertain. If the whole i4i patent thing goes against Microsoft, the standard will probably need to be withdrawn.
ISO/IEC 29500 with Corrigenda and Amendments will eventually line up with Office 2007 SP2 sometime in 2010/11.
But before that happens, Office 2010 will ship with hundreds of extensions that are not described in ISO/IEC 29500 but are documented in proprietary “implementation notes” on Microsoft’s web site.
“Office 15” will ship sometime around 2013. It will have further proprietary extensions to ISO/IEC 29500, also not standardized. Office 15 will still be supporting “transitional” OOXML, just like Office 2007 and Office 2010 did. “Transitional” OOXML is the variation that has all the deprecated crud, like VML and “DoItLikeWord95” in it.
“Office 16” will ship sometime around 2016. It will finally support the “strict” schema of ISO/IEC 29500, but with 3 generations of proprietary extensions layered on top of it. And that assumes ISO/IEC 29500 actually still exists. In 2015 — 5 years after its last amendment — it will be up for “periodic review” in ISO and may be withdrawn if it appears to have been abandoned by Microsoft.

The pattern is clear: OOXML will be extended by Microsoft much faster than it will be standardized and corrected by ISO. This will make the ISO version of OOXML, currently not supported by Microsoft, even more irrelevant in the future.

The Final OOXML Update: Part II

2009/10/16 By Rob 12 Comments

In Part I of this OOXML update, my first post on the topic in over a year, I showed you how Microsoft maintains strong control over the OOXML standard. Despite their earlier promises that control of OOXML would be handed over to an independent, international committee, a look at attendance records reveals that the committee that maintains OOXML (JTC1/SC34/WG4) consists mainly of Microsoft employees, who outnumber any other company or organization on the committee 10-to-1.

In this, Part II of my OOXML update, I’ll tie up another loose end from the immediate aftermath of the DIS 29500 BRM.

Let’s start by casting our memories back to April, 2008. The BRM was over. NBs had reviewed thousands of pages and submitted thousands of defects. And Microsoft/Ecma made thousands of responses. And the BRM approved thousands of changes. Then, as a final step, NBs were asked if they wanted to change their vote from their original September 2007 vote, based on the changes made to the standard by the BRM.

Curiously, NBs were asked to make their final decision without actually seeing the text of the standard they were being asked to approve. ISO leadership denied requests from several NBs, a formal SC34 resolution requesting this text, as well as NB appeals, all which asked to have access to the “final DIS” text that would eventually be published. The ISO chief, in his response to the NB appeals, called the final text of OOXML “irrelevant” (prophetic words, indeed!) and would only permit NBs to have access to a list of over 1,000 resolutions from the BRM, many of which gave great editing discretion to the Microsoft consultant who would eventually produce the final text of the specification.

I discussed why the lack of a final DIS text was a problem back in May 2008:

We are currently approaching a two month period where NB’s can lodge an appeal against OOXML. Ordinarily, one of the grounds for appeal would be if the Project Editor did not faithfully carry out the editing instructions approved at the BRM. For example, if he failed to make approved changes, made changes that were not authorized, or introduced new errors when applying the approved changes. But with no final DIS text, the NB’s are unable to make any appeals on those grounds. By delaying the release of the final DIS text, JTC1 is preventing NB’s from exercising their rights.

Would you make thousands of changes to code and then not allow anyone to test it, and then release it internationally? Of course not. Doing so would amount to professional malpractice. But that is essentially what ISO did with OOXML.

Well, guess what happened? Indeed, the published text of OOXML failed to carry out all of the editing instructions made by the BRM. Several of the BRM resolutions were ignored altogether. Several others were applied inconsistently or erroneously. Although I am aware of no systematic review of all 1,000+ BRM decisions, some NBs have gone back and reviewed the published text against BRM decisions that should have addressed their own NB’s reported comments. They have found many “discrepancies” and these have now been reported as defect reports [PDF].

Whether the flaws in the published text are intentional or accidental, grave or minor, does not really matter here. Errare humanum est. The problem is that it was 100% predictable that human error would cause problems like this when dealing with text changes of this volume. The issue is not whether there will be errors introduced. The presence of many errors was guaranteed. The question was whether NBs are entitled to base their vote on all relevant information, including the final text of the standard, or whether relevant information, indeed the most relevant information — the text of the specification they are voting on — may be withheld from inspection. For ISO leadership to deny NBs the ability to review the very text they were voting was irresponsible.

The good news is that ISO leadership has changed since this debacle, and JTC1 is currently revising Fast Track procedures, in part to deal with the abuses of DIS 29500. One of the changes they are making is to the Fast Track procedure will require that the final DIS text be distributed to NBs before the final vote. This is progress and it is good to see these changes made, though it is unfortunate that it required a failure before such obvious and prudent precautions were instituted. Leadership entails foreseeing and preventing problems, not simply reacting to them. In any case, the NBs that appealed to ISO on the basis of not being allowed to see the final text should feel vindicated now. The NBs of India, Brazil and South Africa were right.

In Part III of this Update, I’ll bring the story up to the present day, and in Part IV I’ll update the story through the year 2016.

Protocols, Formats and the Limits of Disclosure

2009/10/12 By Rob 4 Comments

A few words today on an important distinction that deserves greater appreciation, since it lies at the heart of several current interoperability debates. What I relate here will be well-known to any engineer, though I think almost anyone can understand the gist of this.

First, let’s review the basics.

Formats define how information is encoded. For example, HTML is the standard format for describing web pages.

Protocols define how encoded information is transmitted from one endpoint to another. For example, HTTP is the standard protocol for downloading web pages from web servers to web browsers.

There are other such format/protocol pairs, such as MIME and SMTP for emails. When we talk about “web standards” we talk about formats (often described by W3C Recommendations) and protocols (often described in IETF RFCs).

An instance of data that conforms to a given format standard might be given any number of terms: a web page, a document, an image, a video, etc., according to the underlying standard. The instance of a format is a data, bits and bytes that you can save to your hard drive, burn to a CD, email, etc. Data in a format is persistent and has a static representation.

But what is an instance of a protocol? It is a transaction. It is ephemeral. You can’t easily save an instance of HTTP or SMTP on your hard drive, or email it to someone else. A protocol is complex dance, a set of queries and responses, often a negotiation of capabilities that preface the data transmission.

There is a key distinction between formats and protocols when it comes to interoperability. The key is that a protocol typically involves the negotiation of communication details between two identifiable parties, each of whom can state their capabilities and preferences, as well as conform to the capabilities of the datalink itself. Software running on each endpoint of the transaction can adapt as part of this negotiation.

You may be familiar with this from the modem days, where this “handshaking” procedure was audibly manifest to you whenever you connected to a remote host. But although you don’t hear or see it, this negotiation still occurs with protocols today, behind the scenes.

For example, when you request a web page, your client negotiates all sorts of parameters with the web server, including packet size and timings (at the TCP/IP level) to authentication, language, character set and cache preferences (at the HTTP level). This negotiation of capabilities is essential for handling the diversity of difference web servers and web clients in existence today.

With a protocol, you have two technical endpoints communicating and negotiating the parameters of the data exchange. In other words, you have software on both ends of the communication able to execute logic to adapt to the needs of the other endpoint and the capabilities of the underlying datalink.

However, when it comes to formats, things are different.

Let’s use an word processor document as an example of a format instance. I author a document, and then I send it out, via email, as an attachment on my blog, burned on a conference CD-ROM, posted to a document server or whatever. I have no idea who the party on the receiving end will be, nor what software they will be using. They could be running Microsoft Office, but they could also be using OpenOffice, Google Docs, Lotus Symphony, WordPerfect, AbiWord, KOffice, etc. I, as the document author, have no ability to target my document to the quirks of the receiving party, since their identity and capabilities are unknown and in general unknowable.

Since a document is not executable logic, it cannot adapt to the quirks of various endpoints. A document is static. When it comes time to interpret the document, you don’t see two vendor endpoints adapting and negotiating. You see only one piece of software, the receiving party’s application, and they need to interpret a static data instance in a given format.

In other words, with document formats, there is no dynamic negotiation, because at the time when your write a document out, you have no idea what the reading application will be. And although the application that reads the document may know the identity of the writing application (via metadata stored in the document for example), it has no ability to negotiate with the writing application, since that application is not present when the document is being loaded.

OK. Simple enough. However, a confused understanding this distinction will lead you to muddled reasoning about interoperability and how it is achieved.

Although it is not ideal, having Microsoft disclosure the details of exactly how they implement various proprietary protocols and even their quirky implementation of standard protocols, this may enable 3rd parties to code to these details. If the disclosure is timely, complete and accurate, this information may be useful. I think of the SAMBA work, for example.

However, no amount of disclosure from Microsoft on how they interpret the ODF standard will help. We see that today, with Office 2007 SP2, where it strips out ODF spreadsheet formulas. Having official documentation of this fact from Microsoft, in the form of “Implementation Notes” does not help interoperability. Why? Because when I create an ODF document, I do not know who the reader will be. It may be a Microsoft Office user. But maybe it won’t. It very well could be read by many different users, using many different programs. I cannot adapt my document to the quirks of all the various ODF implementations.

When you deal with formats, interoperability is achieved by converging on a common interpretation of the format. Having well-documented, but divergent interpretations does not improve interoperability. Disclosure of quirks is insufficient. Disclosure presumes a document exchange universe where the writing application knows that the reader will be Microsoft Office and only Microsoft Office and therefor the writer can adapt to Microsoft’s quirks. That is monopolist’s logic. Interoperability with competition only comes when all implementors converge in their interpretation of the format. When that happens we don’t need disclosures. We just follow the standard.

The Final OOXML Update: Part I

2009/10/01 By Rob 19 Comments

I have not written a blog post on OOXML for well over a year now. My last post on this topic was on August 17th, 2008 and covered the contentious appeals process which followed the DIS 29500 Fast Track ballot. So I hope that one more post, 14 months later, will not seem excessive to my critics. There is too much good stuff going on with ODF these days, with ODF 1.2 coming soon, inter-vendor work at plugfests, the ODF Toolkit, and continual national adoption, for me to waste much time on OOXML. “Let the dead bury their own dead” is my attitude here. That said, I have received several requests for an update on OOXML, so I will oblige with some quick observations in what (I hope) is my final update on this sad chapter in standardization.

I’ll structure this update over a handful of posts, each one looking at a single topic. In this post I’ll cover the tight control Microsoft maintains over the OOXML standard, despite their earlier assertions to the contrary.

From the beginning of the Fast Track procedure Microsoft encouraged NBs to approve OOXML with the promise that their approval of the specification would guarantee that it would be handed over to the “global community” for maintenance. Vote against the standard — because it was obviously flawed — and you would lose this unique opportunity to transfer control from proprietary interests at Microsoft to the benevolent and international meritocracy of ISO. This was one of the main “selling points” for OOXML and what Microsoft repeatedly sold.

For example, here was Jerry Fishenden, Microsoft’s National Technology Office in the UK:

There’s an easy question to consider here: would you prefer the Microsoft file formats to continue to be proprietary and under Microsoft’s exclusive control? Or would you prefer them to be under the control and maintenance of an independent, open standards organisation? I think for most users, customers and partners that’s a pretty easy question to answer: they’d prefer control and maintenance to be independent of Microsoft. And the good news is that the Open XML file formats are already precisely that: currently under the control of Ecma International (as Ecma-376) and, if the current voting process is positive, eventually under the control of ISO/IEC

Or Microsoft’s Jason Matusow:

I still hear patently untrue claims that MS controls Open XML – this wasn’t true following the adoption of Ecma 376, and is now permanently a moot argument.

Microsoft Australia said:

This is encouraging and should be a reminder to all that the Open XML standard will be controlled by the international community not by any commercial business or other organisation – including Microsoft.

Chris Capossela , Microsoft Senior VP said it thus:

If Open XML is approved as an ISO/IEC standard, the story would not end there – like any other standard, maintenance affords the opportunity for continually updating and improving the standard. In this case, the global community would be in control of the evolution of this standard going forward – a fitting result given that this format will be widely used around the world for years to come.
.
.
.
Now, the global community has the opportunity to take control of the future of the specification by ratifying Ecma 376 as an ISO/IEC standard. We know that it will be in good hands when this happens based on the tremendous work and improvements that have been made to the specification during the ISO/IEC process over the past 14 months. We are committed to the healthy maintenance of the standard once ratification takes place so that it will continue to be useful and relevant to the rapidly growing number of implementers and users around the world.

(If you watched the video linked to from the letter, you will hear Chris say that Microsoft “has transferred stewardship of the file formats to the global community”.)

Well, that was what was promised. But how did it turn out in reality?

Let’s take a look at who actually attends meetings of SC34/WG4, the technical committee that should have made the question of OOXML control “moot” and puts it “under the control of ISO/IEC”.

If you look at the attendance records, summarized in the following table, you will find that the committee regulars consist primarily of Microsoft employees. In many of the meetings, Microsoft employees outnumber all other attendees combined. And then there is the “Microsoft Co-Prosperity Sphere”, the Microsoft consultants, Microsoft business partners and Microsoft-funded research institution, which further contribute to Microsoft’s effective domination of the meetings.

Person	Employer	NB	4/16	4/30	5/14	5/28	6/11	6/22	7/16	7/30
Makoto Murata	Consultant	Japan	1	1	1	1	1	1	1	1
Doug Mahugh	Microsoft	Ecma	1	1	1	1	1	1	1	1
Shawn Villaron	Microsoft	Ecma	1	1	1	1	1	1	1	1
Dave Welsh	Microsoft	US	1	1	1	1	1	1	1	1
Jirka Kosek	Consultant	Czech Rep	1	1	1	1	1		1	1
Rex Jaeshcke	Microsoft Consultant	Ecma	1	1		1	1	1	1	1
Gareth Horton	Data Watch	UK		1	1	1	1	1	1	1
Jesper Lund Stocholm	Ciber	Denmark	1	1	1		1	1	1
Isabelle Valet-Harper	Microsoft	Ecma	1	1	1	1	1		1
Mohamed Zergaoui	Innovimax	France	1	1	1	1	1		1
Mario Wendt	Microsoft	Germany	1	1		1	1	1	1
Alex Brown	Griffin Brown Digital Publishing	UK	1	1		1	1	1	1
Florian Reuter	Novell	Ecma	1	1		1	1		1
Jaeho Lee	University of Seoul	Korea		1	1		1		1
Caroline Arms	Library of Congress	Ecma	1	1	1
Francis Cave	Francis Cave Digital Publishing	UK	1	1	1
Rick Jelliffe	Consultant	Unauthorized				1	1			1
Nasser Kettani	Microsoft	Côte d’Ivoire					1		1
Pia Lange	Dansk Standard	Denmark					1	1
Kimmo Bergius	Microsoft	Finland				1		1
Juha Vartiainen	Finnish Standards	Finland	1
Jean Paoli	Microsoft	Ecma			1
Sam Oh	Sungkyunkwan University	Korea						1
Klaus Peter Eckert	Fraunhofer Fokus	Germany						1
Jung-Jin Yang	Catholic University of Korea	Korea						1
Keld Simonsen	RAP	Norway						1
Amruta Gulinakar	Microsoft	Ecma							1

So is this really handing over control? Is it really independent? And is it really global?

Let’s look at it in graphical form. In this chart I tally up the participation from each entity (company, organization or unaffiliated individual) attending WG4 meetings. This takes account of all 8 published meeting minutes for WG4. It shows the total participation over those meetings. So if a company sent 8 people to one meeting, this is scored the same as if they sent 1 person to each of 8 meetings. It is the overall participation for an entity that is measured relative to the total participation of all entities at the meetings. Note also that the “Microsoft” tally is of Microsoft employees only. The rest of the Microsoft Co-Prosperity, for purposes of this chart I am all counting as “independent” entities. So this picture is the most complimentary view possible of the degree of concentration in WG4. Obviously, Microsoft’s control is much higher if we take account of these other inter-entity obligations.

I suppose this is “global” in a sense, in the same way one could stage an “International Food Festival” and then have McDonalds show up and contribute a Big Mac from the U.S., a Big Mac from Germany, a Big Mac from the Ivory Coast, a Big Mac from Finland and another Big Mac from Brazil and so on. Certainly, you could claim this was “international”, but you would be laughed right out of the festival if you did.

By way of comparison, here it the same analysis, plotted on the same scale, for the most recent 8 meetings of the OASIS ODF TC. As you can see it is much flatter. No company has more than 20% or so of the participation, and no two companies combined have control of the TC.

Now don’t get me wrong. There are certainly some independent people in WG4 and I would not want anyone to denigrate their efforts. They are not all Microsoft employees, consultants, business partners and research institutions that Microsoft is funding. But they are mostly so. Attend any OOXML meeting and look to your right, look to your left, and most likely one is a Microsoft employee and another is economically tightly tied to Microsoft.

Of course, I would not expect that Microsoft would be absent from this work either. After all, they authored the specification and have most of the relevant technical experts. But a glance over the attendance records shows that they are not gracing the committee with their file format gurus. Instead they are stuffing it with “program managers” and “technology directors” and other assorted non-experts. The problem appears to be that their file format experts are all cursed with American residency and so have little value in stuffing a committee that has one-country/one-vote rules. Thus the spectacle of a room filled with Microsoft employees wrapped in different colored flags.

So I don’t think one can truthfully say (in Jason’s words) that it is “patently untrue” that Microsoft controls OOXML. Or that OOXML “control and maintenance” is “independent of Microsoft” as Jerry promised it would be, or that the “global community would be in control” as Chris said. I don’t think those are accurate statements, given the evidence. I think the results fall far short of what was promised back when Microsoft were trying to secure a positive vote in ISO.

And this is not just me complaining. At the recent SC34 Plenary meeting in Seattle, delegates from several NBs approached me, voicing concerns at the domination Microsoft was asserting over the committee. (Perhaps this explains the substantial number of people who attended only one WG4 meeting and then never returned?) There is no easy solution here. Remember, we are dealing with a company that has demonstrated that it is willing to spend millions of dollars to protect its Office monopoly franchise from any pro-competition standards initiative.

The Former ISO Secretary-General, when interviewed about the OOXML farce, was asked about claims of Microsoft domination and admitted that he was powerless to stop this:

Companies have no direct vote on the International Standards, which are adopted according to voting by national member bodies, on the basis of one vote per country… As a stakeholder in the process, Microsoft and other interests certainly participated in the process to establish national positions. ISO and IEC national members are fully responsible for the way national votes are formed and relevant national interests consulted.

Evidently there is no one capable of fixing this. ISO says that domination by a single corporation is not their responsibility, because only NBs vote and each NB determines its own participation rules. But individual NBs also don’t see a problem, because any single one of them only has one Microsoft employee at the meeting. So the NB itself is not necessary stuffed (although that does happens occasionally as well). So by placing Microsoft employees in many NB delegations and putting the overflow into the Ecma delegation, Microsoft can still dominate the ISO committee and not trigger a rule violation in ISO or in any NB.

This is essentially how Microsoft hacked ISO. Now that the flaw has been demonstrated, any large international corporation with sufficient funds and interest can exploit it as well. So long as the rules remain as they are, ISO is vulnerable. ISO defends this criticism by pointing out what good work they’ve done in the past, and how they rarely have problems of this kind before. But this shows little appreciation for the nature of the problem which have been demonstrated. It is like arguing that a newly discovered (though long latent) security flaw in an operating system is insignificant because you’ve never had an attack before now. Of course, this misses the point entirely. Once the vulnerability is known and publicly exploited, you’re living on borrowed time until you can secure the system. Today ISO is living on borrowed time and is very close to becoming a Microsoft-infested zombie committee.

That is all for Part I of this update. In the next Part I’ll look at the maintenance of OOXML, and the most peculiar way in which the Microsoft-dominated committee is putting aside BRM decisions and making other breaking changes to the specification, in an bizarre attempt to make ISO OOXML conform to to the Microsoft Office standard.

What’s New in ODF Maintenance?

2009/09/17 By Rob 7 Comments

Some Q&A, in the form of a self-interview. As with anything on this blog, these are my opinions.

Question: How are you doing today, Rob?

Rob: Very well thank you. I just finished attending a good set of working group (WG) meetings, and the Plenary meeting of ISO/IEC JTC1/SC34 in Seattle, Washington.

Question: Anything newsworthy to report?

Rob: Most of the meeting was routine, moving various documents forward, inch by inch, at the glacial pace we expect from ISO. But one noteworthy event this week was the first meeting of SC34/Ad Hoc Group 3.

Question: What is Ad Hoc Group 3?

Rob: Ad Hoc Group 3, or “AHG3,” is an temporary working group that was charged with determining how SC34 would fulfill its maintenance obligations relating to ISO/IEC 26300, the ISO/IEC transposition of the ODF standard. Francis Cave, of the UK, presided over the AHG3 meeting, which was well attended by 20 or so delegates.

Question: What do you mean by “maintenance”?

Rob: In this context maintenance refers to the activities that occur between major revisions of a standard, primarily the collection of defects, and the approval and publication of corrections.

Question: So what did Ad Hoc Group 3 do?

Rob: AGH3 met for around 8 hours over two days. Its most notable act was to orchestrate its own dissolution, by proposing that a new, permanent working group, WG 6, be created by SC34 to carry out pretty much the same charge given to AHG3. WG6 will likely also have the same Convenor and participants as AHG3.

Other than that, the primary benefit of the AHG3 meeting was to bring everyone up to speed on the current state of various ODF defect reports and errata documents, and to discuss ways in which we can better align the ISO/IEC version of ODF and keep it in alignment, with the OASIS version.

Question: So who owns ODF maintenance?

Rob: The OASIS ODF TC owns the maintenance of the OASIS ODF standard, and WG6 will own this activity for the equivalent ISO/IEC text. However, neither committee has absolute freedom of action, both being governed by applicable procedural rules of their parent organizations, as well as various joint agreements between OASIS and JTC1.

Question: Won’t this lead to a divergence of the OASIS and ISO/IEC versions?

Rob: It is a stated goal of JTC1 that ISO/IEC 26300, as the PAS transposed version of ODF, remain in sync with the OASIS version of ODF. This goal is shared with OASIS. OASIS and JTC1 have jointly approved a set of maintenance principles which call for the OASIS ODF TC to take the lead on maintenance.

Although we have not worked out all the details, a natural way to avoid divergence is to treat this as a two-phase process:

Defect reports from SC34 are submitted to the OASIS ODF TC
Corrections originate in the OASIS ODF TC, and are then transmitted to SC34

Question: But that doesn’t guarantee that these texts will not diverge, right?

Rob: At the very least there will be a delay between the correction of the OASIS version of the text and the publication of the ISO/IEC version of the correction. This delay will typically be around 5 months. If both OASIS and SC34 adhere the agreed maintenance principles, any divergence should amount to no more than such a delay.

Note that we already have some degree of divergence. For example, OASIS published ODF 1.0 Approved Errata 1.0 in November 2008, and submitted these changes to SC34 where they have not been acted on. And OASIS published ODF 1.1 back in early 2007 and has failed to submit these changes to SC34. Although there appears to be no adverse market effects from this degree of divergence, both sides agree that this is not an ideal situation and have agreed in principle to take steps to remedy this situation.

Also, we shouldn’t worry about the possibility of intentional or malicious divergence, since careful consideration of the ownership of the copyright and trademark for the ODF standard, as well as terms governing any associated patent claims, suggests that any rogue attempt to fork the ODF standard would be fraught with peril. ISO/IEC 26300 may lag OASIS ODF, but it cannot go beyond the corresponding OASIS text in any substantive way.

Question: So what role does WG6 have then?

Rob: SC34 has several WGs, each dealing with a different standard or group of standards. This WG structure is important for National Bodies, which often allocate their resources (personnel, travel budgets, etc.) at the level of a WG. It is also a logistic convenience for scheduling meetings.

WG6’s charter sounds impressive, “All SC 34 projects and activities relating to the maintenance of ISO/IEC 26300 OpenDocument Format”. However, since JTC1 has already assigned coordination of ODF maintenance to the OASIS ODF TC, it is not clear what, if any, residual tasks remain for WG6 to perform. But even if its official responsibilities are minor, WG6 does serve a useful purpose as a focal point for SC34 technical experts having an interest in ODF. Under the aegis of WG6, these experts might, for example, create defect reports for submission to OASIS and give feedback on OASIS draft corrigenda.

Question: But what about contributions in the other direction? How do SC34 technical experts contribute to ODF?

Rob: As stated in the joint memorandum of principles between OASIS and JTC1: “National Body input, including but not limited to the submission of defects and amendments, can best be achieved by the participation of JTC 1 experts in the ODF TC of OASIS”.

The exact answer will depend on your specific goal. If you participate via SC34, your opinion will count as one vote on your NB delegation, and in turn your delegation will have one vote among many NBs in SC34, which amounts to one source of feedback, among many others, to the OASIS ODF TC. So essentially you will have a much diluted and rather indirect way of expressing your thoughts on the technical work of the OASIS ODF TC.

Or you could join OASIS and and the ODF TC directly, as an individual member, in which case you can contribute directly in our work and your vote will be equal to mine or any other member of the ODF TC.

So if your priority is to present a national view via national body participation, then by all means participate via SC34 and WG6. This is also good if your priority is racking up frequent flier miles via attendance at international meetings. But if you are looking for the most effective way to contribute directly to the standard, then you cannot do better than joining the OASIS ODF TC. This is probably why nine members of JTC1/SC34 already are OASIS ODF TC members.

Question: So we’ve talked so far about maintenance. But what about major revisions, such as ODF 1.2, how are they handled?

Rob: Major revisions developed in OASIS may be submitted to JTC1 via the PAS transposition process. This results in a JTC1 ballot and does not directly involve SC34 technical experts, although in some cases the SC34 national “mirror committee” may be consulted by their NBs in the preparation of a national position on the PAS ballot. In any case, WG6 would have no formal role in this process, nor can I imagine an informal role for them.