OOXML

The Final OOXML Update: Part III

2009/10/27 By Rob 9 Comments

This is Part III of an 5-part series on the state of OOXML today. Previous to starting this series, I had not posted about OOXML in over a year. Part I showed how Microsoft, despite their promises that control of OOXML would be handed over to an independent, international committee, have instead stuffed the committee that maintains OOXML (JTC1/SC34/WG4) with Microsoft employees. And in Part II I looked at how the final published text of OOXML failed to account for all BRM decisions, and described the steps that ISO was taking to remedy this obvious procedural flaw.

In this Part I’ll look at how Microsoft is using their dominance in SC34 to push through hundreds of changes and additions to OOXML, in a misuse of a procedure intended for correcting drafting errors, to make OOXML “conform” to Microsoft’s monopoly product.

Let’s start by taking a look at the OOXML defect log [PDF] that SC34/WG4 uses to track their large list of errors and omissions discovered in the published standard. This defect report currently amounts to over 800 pages, longer than the entire ODF 1.0 standard. But it is well worth downloading and browsing through.

Some of these changes will be made in Technical Corrigenda while others are proposed for Amendments. What is the difference? SC34/WG4 itself made the distinction clear, in a presentation (N 1187 for those with access) it made to the SC34 Plenary in Prague, where it outlined its practice for deciding which changes would be made in corrigenda versus amendments:

All of the following criteria should be met for the defect to be resolved by Corrigendum:

1) WG 4 agrees that the defect is an unintentional drafting error.
2) WG 4 agrees that the defect can be resolved without the theoretical possibility of breaking existing conformant implementations of the standard.
3) WG 4 agrees that the defect can be resolved without introducing any significant new feature.

Unless all the above criteria are met, the defect should be resolved by Amendment.

These are reasonable criteria and no objections were made when these guidelines were presented to SC34.

A key procedural point is that in ISO/IEC it is the JTC1 NBs who are the consensus body that has the authority to create international standards. All ballots which create or substantial modify standards must be approved by JTC1. This includes DIS ballots, FDIS ballots, FDAM ballots and DTR ballots. So standards, technical reports and amendments are ultimately approved or disapproved by JTC1 NBs. Although subcommittees in JTC1, such as SC34, provide the technical expertise and author and review work, they are not the standardizing authority. The exception that proves the rule is with corrigenda, which are authored and approved entirely at the SC level. However, this small area of autonomy in defect correction comes with carefully delineated bounds. A SC can author, approve and publish corrigenda by itself, but only to make corrections.

So if we look at JTC1 Directives 15.4.2.2, we read (with my emphasis) “A technical corrigendum is issued to correct a technical defect…. Technical corrigenda are not issued for technical additions which shall follow the amendment procedure…”. And in 15.4.1 “technical addition” is defined as: “Alteration or addition to previously agreed technical provisions in an existing IS.”

So amendments, which require approval by JTC1, are used for altering or extending the provisions of a standard, while corrigenda are used to correct errors introduced in drafting or publication. This dichotomy is common in other standards organizations. For example, in OASIS, a technical committee is able to approved and publish “Approved Errata” but these are restricted to changes that do not break conformance of existing implementations. Anything beyond that is considered a substantive change to the standard and requires review approval by the OASIS membership.

Clear enough? In fact, in many cases WG4 appears to follow this important distinction. Some of the proposed changes are simple and benign. For example, some BRM issues were fixed, but in being fixed caused informative example markup in the standard to be incorrect. A quick fix of these items via corrigenda is most welcome.

However, in other cases (in fact most of the cases), the Microsoft-dominated WG4 appears to have overstepped the permissible bounds for corrigenda, and indeed gone far, far beyond what it stated it would be doing in corrigenda. Let’s look at a few examples.

(Sadly, the general public is not given access to the text of the draft corrigenda (the DCOR) but those on the inside can follow along by reading N 1252 in the SC34 document repository.)

Let’s start by looking at items 16, 17, 36, 52, 53 and 133 in DCOR for ISO/IEC 29500 Part 4. These make changes and additions to the WordProcessingML schema. Deletions are noted in red strikethroughs, and additions in blue:

This is not correcting a drafting error. This is not correcting a publishing error. This is a substantial addition to the schema as you can see above.

It is argued, in the defect log, that this change is needed because, without it, ISO/IEC 29500 cannot represent change tracking in mathematical equations. However, this is exactly the type of change that WG4’s guidelines and JTC1 Directives exclude from corrigenda and place into amendments. The schema of OOXML is certainly an “agreed technical provision of an existing IS”. So how can adding math change tracking support to the schema be anything other than an “addition to previously agreed technical provisions”? And how can anyone in WG4 believe that adding dozens of lines to the schema can be done “without the theoretical possibility of breaking existing conformant implementations of the standard”? What about, for example, applications that were programmed to use the published OOXML schema, such as any application that uses a validating parser, or an schema-directed editor, or a program that generates code stubs from the schema, or does XML-to-relational DB mapping? Not only is there a theoretical possibility of breaking such applications, there is a theoretical certainty.

(Ironically, it should be noted that Microsoft was very keen to beat up on ODF for not having change tracking for mathematical equations, all while hiding the fact that OOXML lacked complete support for this feature as well.)

Another example, #122 in the DCOR.

It changes a type in chart, from a byte to an int and in doing so extends its allowed range considerably. How did anyone think that this was a change that was “without the theoretical possibility of breaking existing conformant implementations of the standard”? Isn’t there enough theoretical and practical expertise in WG4 to know that changes like this break compatibility?

For this change the rationale in the defect log explains the logic of it:

The standard states that the ST_Period simple type uses the XML Schema ST_Period data type and supports a range 2–255.

These observations are incompatible with existing documents and should be updated to reflect such prior art.

And so on and so on. If you search through the defect log, you will see the phrase “existing documents” used dozens of times. That appears to be how many discussions in WG4 end. It shuts down debate like an appeal to “national security” or “executive privilege”, arguments that trumps all others. It doesn’t matter what WG4 previously told SC34, or what JTC1 Directives say, if ISO/IEC 29500 does not match what Microsoft Office actually writes out, then this is by definition a drafting error, and the standard will be “corrected” to conform with MS Office. Let that sink in for a little, until you realize how backwards this is.

I invite you to go back to the defect log [PDF] and search for “BRM”. You will find several oddities. For example, among these proposed changes are some that actually reverse BRM decisions. Yes, you heard me correctly. SC34/WG4, the Microsoft-dominated committee that maintains OOXML, is undoing various BRM decisions that enabled OOXML to be approved in the first place. Why? Well, of course, to make the standard conform more to Microsoft Office.

For example, take DR 09-0159 “General: Unintended incompatibilities between Transitional schema and Ecma-376” or DR 09-0275 “BRM: serial date representation” with this comment:

Although this text is in accord with the detailed amendments resolved at the BRM, it is against the spirit of the desired changes for many countries. We believe that due to time limitations at the BRM, this change was made without sufficient examination of the consequences, and was made in error by the BRM (in which error the UK played a part).

(Norbert Bollow, a member of the Swiss NB, has some good analysis of the return of the leap year bug in spreadsheets. And Jomar Silva with the Brazilian NB tracks some additional breaking changes on a wiki.)

Ah, So WG4 is now interpreting the “spirit of the BRM” through their shamanic communion with the ISO Weltgeist, and each time their oracle come back with the same response: “Change OOXML so it ‘conforms’ to Microsoft Office 2007”. How convenient for Microsoft.

For most standards, multiple vendors work together to improve interoperability and to increase their conformance with the standard. But with OOXML a single vendor stuffs the committee and works to make the standard better conform to Microsoft’s monopoly product.

So although Microsoft Office does not conform to ISO/IEC 29500 today, I have no doubt that within a few months it will fully conform. But not a single line of code will have changed in the Office product. Office 2007 will be retroactively made to conform to ISO/IEC 29500. What will happen is the standard will be modified to match that single vendor’s products, by misapplication of an ISO procedure intended for fixing minor drafting errors.

So why go through all this trouble? I believe this is all about getting the OOXML standard “corrected” so Microsoft can push for it to get it officially adopted around the world. The only reason they’ve held back so far is because MS Office does not actually implement ISO/IEC 29500 today. So it would have been counter productive for them to push for official adoption. However, once this oversight is remedied, by changing the standard to match their product, then watch out.

The side effect, perhaps unintended, is that the OOXML standard is thus clearly marked to be unstable and unsuitable for adoption or implementation. With 800 pages of defects and more being found, and a Microsoft-dominated committee that changes the standard with no objective technical justification, the exact contents of the OOXML standard is tentative, uncertain and temporary. Four corrigenda documents and two amendment documents are currently being balloted, including many breaking changes. More corrigenda and amendments are on the way. There is no provision for a version attribute or any other indicator to declare which of the multiple incompatible versions of the standard a document conforms to. What competitor would risk implementing the standard, knowing that Microsoft dominates WG4, which has shown it is willing to change the standard at Microsoft’s whim? The risk is simply too large. A competitor would simply be putting their head in the lion’s mouth.

And at the same time WG4 rushes to make OOXML conform to Office 2007, Microsoft is moving on with Office 2010, now in technical preview. Office 2010 will be extending OOXML in hundreds of places. Where is SC34 in this? Where is the new work proposal for OOXML 1.1? Where the are discussions? The drafts? None of this exists. If Microsoft wanted to, they could have submitted these changes to SC34 at the recent meeting in Seattle, but they preferred to reserve discussion of the Office 2010 changes for a private meeting in Redmond the day after the SC34 Plenary ended, a snub to SC34 and their fictional control of OOXML.

So Microsoft is now off extending OOXML, and this whole ISO escapade with OOXML seems for naught. (I hear also that Microsoft is also backing off the submission of their Extensible Page Specification (XPS) to ISO as well, saying that “an Ecma Standard is good enough”.) It appears that Microsoft got what they wanted from ISO and is moving on. Who said it would last more than a night? As my grandmother used to say, “Why buy the cow when you can get the milk for free?”

In any case, the future looks like something like this:

ISO/IEC 29500:2008’s future is uncertain. If the whole i4i patent thing goes against Microsoft, the standard will probably need to be withdrawn.
ISO/IEC 29500 with Corrigenda and Amendments will eventually line up with Office 2007 SP2 sometime in 2010/11.
But before that happens, Office 2010 will ship with hundreds of extensions that are not described in ISO/IEC 29500 but are documented in proprietary “implementation notes” on Microsoft’s web site.
“Office 15” will ship sometime around 2013. It will have further proprietary extensions to ISO/IEC 29500, also not standardized. Office 15 will still be supporting “transitional” OOXML, just like Office 2007 and Office 2010 did. “Transitional” OOXML is the variation that has all the deprecated crud, like VML and “DoItLikeWord95” in it.
“Office 16” will ship sometime around 2016. It will finally support the “strict” schema of ISO/IEC 29500, but with 3 generations of proprietary extensions layered on top of it. And that assumes ISO/IEC 29500 actually still exists. In 2015 — 5 years after its last amendment — it will be up for “periodic review” in ISO and may be withdrawn if it appears to have been abandoned by Microsoft.

The pattern is clear: OOXML will be extended by Microsoft much faster than it will be standardized and corrected by ISO. This will make the ISO version of OOXML, currently not supported by Microsoft, even more irrelevant in the future.

The Final OOXML Update: Part II

2009/10/16 By Rob 12 Comments

In Part I of this OOXML update, my first post on the topic in over a year, I showed you how Microsoft maintains strong control over the OOXML standard. Despite their earlier promises that control of OOXML would be handed over to an independent, international committee, a look at attendance records reveals that the committee that maintains OOXML (JTC1/SC34/WG4) consists mainly of Microsoft employees, who outnumber any other company or organization on the committee 10-to-1.

In this, Part II of my OOXML update, I’ll tie up another loose end from the immediate aftermath of the DIS 29500 BRM.

Let’s start by casting our memories back to April, 2008. The BRM was over. NBs had reviewed thousands of pages and submitted thousands of defects. And Microsoft/Ecma made thousands of responses. And the BRM approved thousands of changes. Then, as a final step, NBs were asked if they wanted to change their vote from their original September 2007 vote, based on the changes made to the standard by the BRM.

Curiously, NBs were asked to make their final decision without actually seeing the text of the standard they were being asked to approve. ISO leadership denied requests from several NBs, a formal SC34 resolution requesting this text, as well as NB appeals, all which asked to have access to the “final DIS” text that would eventually be published. The ISO chief, in his response to the NB appeals, called the final text of OOXML “irrelevant” (prophetic words, indeed!) and would only permit NBs to have access to a list of over 1,000 resolutions from the BRM, many of which gave great editing discretion to the Microsoft consultant who would eventually produce the final text of the specification.

I discussed why the lack of a final DIS text was a problem back in May 2008:

We are currently approaching a two month period where NB’s can lodge an appeal against OOXML. Ordinarily, one of the grounds for appeal would be if the Project Editor did not faithfully carry out the editing instructions approved at the BRM. For example, if he failed to make approved changes, made changes that were not authorized, or introduced new errors when applying the approved changes. But with no final DIS text, the NB’s are unable to make any appeals on those grounds. By delaying the release of the final DIS text, JTC1 is preventing NB’s from exercising their rights.

Would you make thousands of changes to code and then not allow anyone to test it, and then release it internationally? Of course not. Doing so would amount to professional malpractice. But that is essentially what ISO did with OOXML.

Well, guess what happened? Indeed, the published text of OOXML failed to carry out all of the editing instructions made by the BRM. Several of the BRM resolutions were ignored altogether. Several others were applied inconsistently or erroneously. Although I am aware of no systematic review of all 1,000+ BRM decisions, some NBs have gone back and reviewed the published text against BRM decisions that should have addressed their own NB’s reported comments. They have found many “discrepancies” and these have now been reported as defect reports [PDF].

Whether the flaws in the published text are intentional or accidental, grave or minor, does not really matter here. Errare humanum est. The problem is that it was 100% predictable that human error would cause problems like this when dealing with text changes of this volume. The issue is not whether there will be errors introduced. The presence of many errors was guaranteed. The question was whether NBs are entitled to base their vote on all relevant information, including the final text of the standard, or whether relevant information, indeed the most relevant information — the text of the specification they are voting on — may be withheld from inspection. For ISO leadership to deny NBs the ability to review the very text they were voting was irresponsible.

The good news is that ISO leadership has changed since this debacle, and JTC1 is currently revising Fast Track procedures, in part to deal with the abuses of DIS 29500. One of the changes they are making is to the Fast Track procedure will require that the final DIS text be distributed to NBs before the final vote. This is progress and it is good to see these changes made, though it is unfortunate that it required a failure before such obvious and prudent precautions were instituted. Leadership entails foreseeing and preventing problems, not simply reacting to them. In any case, the NBs that appealed to ISO on the basis of not being allowed to see the final text should feel vindicated now. The NBs of India, Brazil and South Africa were right.

In Part III of this Update, I’ll bring the story up to the present day, and in Part IV I’ll update the story through the year 2016.

The Final OOXML Update: Part I

2009/10/01 By Rob 19 Comments

I have not written a blog post on OOXML for well over a year now. My last post on this topic was on August 17th, 2008 and covered the contentious appeals process which followed the DIS 29500 Fast Track ballot. So I hope that one more post, 14 months later, will not seem excessive to my critics. There is too much good stuff going on with ODF these days, with ODF 1.2 coming soon, inter-vendor work at plugfests, the ODF Toolkit, and continual national adoption, for me to waste much time on OOXML. “Let the dead bury their own dead” is my attitude here. That said, I have received several requests for an update on OOXML, so I will oblige with some quick observations in what (I hope) is my final update on this sad chapter in standardization.

I’ll structure this update over a handful of posts, each one looking at a single topic. In this post I’ll cover the tight control Microsoft maintains over the OOXML standard, despite their earlier assertions to the contrary.

From the beginning of the Fast Track procedure Microsoft encouraged NBs to approve OOXML with the promise that their approval of the specification would guarantee that it would be handed over to the “global community” for maintenance. Vote against the standard — because it was obviously flawed — and you would lose this unique opportunity to transfer control from proprietary interests at Microsoft to the benevolent and international meritocracy of ISO. This was one of the main “selling points” for OOXML and what Microsoft repeatedly sold.

For example, here was Jerry Fishenden, Microsoft’s National Technology Office in the UK:

There’s an easy question to consider here: would you prefer the Microsoft file formats to continue to be proprietary and under Microsoft’s exclusive control? Or would you prefer them to be under the control and maintenance of an independent, open standards organisation? I think for most users, customers and partners that’s a pretty easy question to answer: they’d prefer control and maintenance to be independent of Microsoft. And the good news is that the Open XML file formats are already precisely that: currently under the control of Ecma International (as Ecma-376) and, if the current voting process is positive, eventually under the control of ISO/IEC

Or Microsoft’s Jason Matusow:

I still hear patently untrue claims that MS controls Open XML – this wasn’t true following the adoption of Ecma 376, and is now permanently a moot argument.

Microsoft Australia said:

This is encouraging and should be a reminder to all that the Open XML standard will be controlled by the international community not by any commercial business or other organisation – including Microsoft.

Chris Capossela , Microsoft Senior VP said it thus:

If Open XML is approved as an ISO/IEC standard, the story would not end there – like any other standard, maintenance affords the opportunity for continually updating and improving the standard. In this case, the global community would be in control of the evolution of this standard going forward – a fitting result given that this format will be widely used around the world for years to come.
.
.
.
Now, the global community has the opportunity to take control of the future of the specification by ratifying Ecma 376 as an ISO/IEC standard. We know that it will be in good hands when this happens based on the tremendous work and improvements that have been made to the specification during the ISO/IEC process over the past 14 months. We are committed to the healthy maintenance of the standard once ratification takes place so that it will continue to be useful and relevant to the rapidly growing number of implementers and users around the world.

(If you watched the video linked to from the letter, you will hear Chris say that Microsoft “has transferred stewardship of the file formats to the global community”.)

Well, that was what was promised. But how did it turn out in reality?

Let’s take a look at who actually attends meetings of SC34/WG4, the technical committee that should have made the question of OOXML control “moot” and puts it “under the control of ISO/IEC”.

If you look at the attendance records, summarized in the following table, you will find that the committee regulars consist primarily of Microsoft employees. In many of the meetings, Microsoft employees outnumber all other attendees combined. And then there is the “Microsoft Co-Prosperity Sphere”, the Microsoft consultants, Microsoft business partners and Microsoft-funded research institution, which further contribute to Microsoft’s effective domination of the meetings.

Person	Employer	NB	4/16	4/30	5/14	5/28	6/11	6/22	7/16	7/30
Makoto Murata	Consultant	Japan	1	1	1	1	1	1	1	1
Doug Mahugh	Microsoft	Ecma	1	1	1	1	1	1	1	1
Shawn Villaron	Microsoft	Ecma	1	1	1	1	1	1	1	1
Dave Welsh	Microsoft	US	1	1	1	1	1	1	1	1
Jirka Kosek	Consultant	Czech Rep	1	1	1	1	1		1	1
Rex Jaeshcke	Microsoft Consultant	Ecma	1	1		1	1	1	1	1
Gareth Horton	Data Watch	UK		1	1	1	1	1	1	1
Jesper Lund Stocholm	Ciber	Denmark	1	1	1		1	1	1
Isabelle Valet-Harper	Microsoft	Ecma	1	1	1	1	1		1
Mohamed Zergaoui	Innovimax	France	1	1	1	1	1		1
Mario Wendt	Microsoft	Germany	1	1		1	1	1	1
Alex Brown	Griffin Brown Digital Publishing	UK	1	1		1	1	1	1
Florian Reuter	Novell	Ecma	1	1		1	1		1
Jaeho Lee	University of Seoul	Korea		1	1		1		1
Caroline Arms	Library of Congress	Ecma	1	1	1
Francis Cave	Francis Cave Digital Publishing	UK	1	1	1
Rick Jelliffe	Consultant	Unauthorized				1	1			1
Nasser Kettani	Microsoft	Côte d’Ivoire					1		1
Pia Lange	Dansk Standard	Denmark					1	1
Kimmo Bergius	Microsoft	Finland				1		1
Juha Vartiainen	Finnish Standards	Finland	1
Jean Paoli	Microsoft	Ecma			1
Sam Oh	Sungkyunkwan University	Korea						1
Klaus Peter Eckert	Fraunhofer Fokus	Germany						1
Jung-Jin Yang	Catholic University of Korea	Korea						1
Keld Simonsen	RAP	Norway						1
Amruta Gulinakar	Microsoft	Ecma							1

So is this really handing over control? Is it really independent? And is it really global?

Let’s look at it in graphical form. In this chart I tally up the participation from each entity (company, organization or unaffiliated individual) attending WG4 meetings. This takes account of all 8 published meeting minutes for WG4. It shows the total participation over those meetings. So if a company sent 8 people to one meeting, this is scored the same as if they sent 1 person to each of 8 meetings. It is the overall participation for an entity that is measured relative to the total participation of all entities at the meetings. Note also that the “Microsoft” tally is of Microsoft employees only. The rest of the Microsoft Co-Prosperity, for purposes of this chart I am all counting as “independent” entities. So this picture is the most complimentary view possible of the degree of concentration in WG4. Obviously, Microsoft’s control is much higher if we take account of these other inter-entity obligations.

I suppose this is “global” in a sense, in the same way one could stage an “International Food Festival” and then have McDonalds show up and contribute a Big Mac from the U.S., a Big Mac from Germany, a Big Mac from the Ivory Coast, a Big Mac from Finland and another Big Mac from Brazil and so on. Certainly, you could claim this was “international”, but you would be laughed right out of the festival if you did.

By way of comparison, here it the same analysis, plotted on the same scale, for the most recent 8 meetings of the OASIS ODF TC. As you can see it is much flatter. No company has more than 20% or so of the participation, and no two companies combined have control of the TC.

Now don’t get me wrong. There are certainly some independent people in WG4 and I would not want anyone to denigrate their efforts. They are not all Microsoft employees, consultants, business partners and research institutions that Microsoft is funding. But they are mostly so. Attend any OOXML meeting and look to your right, look to your left, and most likely one is a Microsoft employee and another is economically tightly tied to Microsoft.

Of course, I would not expect that Microsoft would be absent from this work either. After all, they authored the specification and have most of the relevant technical experts. But a glance over the attendance records shows that they are not gracing the committee with their file format gurus. Instead they are stuffing it with “program managers” and “technology directors” and other assorted non-experts. The problem appears to be that their file format experts are all cursed with American residency and so have little value in stuffing a committee that has one-country/one-vote rules. Thus the spectacle of a room filled with Microsoft employees wrapped in different colored flags.

So I don’t think one can truthfully say (in Jason’s words) that it is “patently untrue” that Microsoft controls OOXML. Or that OOXML “control and maintenance” is “independent of Microsoft” as Jerry promised it would be, or that the “global community would be in control” as Chris said. I don’t think those are accurate statements, given the evidence. I think the results fall far short of what was promised back when Microsoft were trying to secure a positive vote in ISO.

And this is not just me complaining. At the recent SC34 Plenary meeting in Seattle, delegates from several NBs approached me, voicing concerns at the domination Microsoft was asserting over the committee. (Perhaps this explains the substantial number of people who attended only one WG4 meeting and then never returned?) There is no easy solution here. Remember, we are dealing with a company that has demonstrated that it is willing to spend millions of dollars to protect its Office monopoly franchise from any pro-competition standards initiative.

The Former ISO Secretary-General, when interviewed about the OOXML farce, was asked about claims of Microsoft domination and admitted that he was powerless to stop this:

Companies have no direct vote on the International Standards, which are adopted according to voting by national member bodies, on the basis of one vote per country… As a stakeholder in the process, Microsoft and other interests certainly participated in the process to establish national positions. ISO and IEC national members are fully responsible for the way national votes are formed and relevant national interests consulted.

Evidently there is no one capable of fixing this. ISO says that domination by a single corporation is not their responsibility, because only NBs vote and each NB determines its own participation rules. But individual NBs also don’t see a problem, because any single one of them only has one Microsoft employee at the meeting. So the NB itself is not necessary stuffed (although that does happens occasionally as well). So by placing Microsoft employees in many NB delegations and putting the overflow into the Ecma delegation, Microsoft can still dominate the ISO committee and not trigger a rule violation in ISO or in any NB.

This is essentially how Microsoft hacked ISO. Now that the flaw has been demonstrated, any large international corporation with sufficient funds and interest can exploit it as well. So long as the rules remain as they are, ISO is vulnerable. ISO defends this criticism by pointing out what good work they’ve done in the past, and how they rarely have problems of this kind before. But this shows little appreciation for the nature of the problem which have been demonstrated. It is like arguing that a newly discovered (though long latent) security flaw in an operating system is insignificant because you’ve never had an attack before now. Of course, this misses the point entirely. Once the vulnerability is known and publicly exploited, you’re living on borrowed time until you can secure the system. Today ISO is living on borrowed time and is very close to becoming a Microsoft-infested zombie committee.

That is all for Part I of this update. In the next Part I’ll look at the maintenance of OOXML, and the most peculiar way in which the Microsoft-dominated committee is putting aside BRM decisions and making other breaking changes to the specification, in an bizarre attempt to make ISO OOXML conform to to the Microsoft Office standard.

Where’s Rob?

2008/10/07 By Rob 14 Comments

Alex Brown is saddened that I did not attend the JTC1/SC34 Plenary in Korea last week. You can view his photo essay and lament on my absence here.

[A] question being asked along the committee corridors by perplexed NB members is whether IBM has withdrawn its staff from participation SC 34. I have no idea, but IBM people are certainly conspicuous here by their total absence.

Well, I’m truly touched, and by way of reciprocation maybe I can help Alex and any other similarly perplexed attendees understand the situation better.

First, it will help if we start by taking a look at recent SC34 meetings and what the attendance record (publicly accessible) says:

Date	Location	Total Attendance	Size of US Delegation	# of IBM/OASIS Participants	# of Microsoft/ECMA Participants
Nov 2004 Plenary	Washington DC	25	6	0	0
May 2005 Plenary	Amsterdam	28	4	0	0
Nov 2005 Plenary	Atlanta	22	4	1	0
May 2006 Plenary	Seoul	30	4	2	2
Mar 2007 Plenary	Oslo	37	6	0	5
Dec 2007 Plenary	Kyoto	52	3	2	12
Apr 2008 Plenary	Oslo	37	3	1	8
July 2008 Ad Hoc 1	London	20	1	1	10
Oct 2008 Plenary	Jeju Island	35 est.	2 est.	0 est.	12 est. (estimates from Alex Brown, since no official attendance has been published)

To put it in perspective, the US SC34 shadow committee currently has around 20 members. Before Microsoft stuffed it we had around 7. Regardless, the US SC34 mirror committee typically sends a delegation of 2 or 3 people to international meetings. IBM attendance at these meetings has varied from 0 to 2. It really depends on where the meeting is being held. If it is being hosted by an NB where an IBM employee is a member, then he will typically attend. If something is on the agenda that I find interesting, then I’ll typically attend regardless of location.

Now what is really interesting is how Microsoft has increased its attendance over the years, something Alex does not mention and presumably does not find fault with. I remember introducing myself to the first Microsoft attendee at a SC34 Plenary back in 2006. He was an attorney, from Microsoft’s anti anti-trust department. An odd person to send to a technical standards committee meeting, don’t you think?

Since then, Microsoft’s representation has swelled so it now comprises 20-50% of any given meeting. And that does not count those additional “independent” companies and contractors that are employed by Microsoft to create OOXML convertors or to consult with on OOXML matters. I’m only counting those people who explicitly list “Microsoft” or “Ecma” as their corporate affiliations.

I think you’ll find no other case in SC34 attendance records of a single company sending more than a single representative. Everyone else in the world sends one person. IBM once sent two people. Microsoft sends ten or a dozen.

Despite Microsoft’s successful attempt to stuff SC34, as they did NB’s around the world, participation from IBM remains in the range of 0 to 2 participants. I’d be hard pressed to justify the expense of any greater attendance. The real work on ODF goes on in OASIS. That’s where we put our people, where they can be most effective on the technical topics related to ODF.

Alex, of course, misses all this. Sitting in a room full of non-technical Microsoft employees, the only unusual thing worth mentioning is my unclaimed badge. Good job as always, Inspector Clouseau!

In any case, the greatest concern should be given to that last row in the table, giving the attendance of the recent Jeju Island Plenary. Although the resolutions of this meeting have been posted and discussed, they lack any record of the actual attendance of this meeting. It has been the constant practice of JTC1/SC34, for many years, to record the attendance of their meetings and to post this document to the SC34 document repository and to make it publicly accessible. But in this case, the attendance record is missing entirely. It isn’t even available to SC34 members.

What are they afraid to reveal? Exactly how many Microsoft employees were at this meeting? The trend certainly has been upward. But this information is not available. Is Alex, the Convenor of WG1, only going to publicize my absence, but then fail to report who actually attended his own WG meeting? Is Alex going to express pleasure in saying “In the event this went extremely smoothly: all resolutions passed with unanimous consensus” without mentioning who exactly was there to vote for these resolutions?

I hope this is not yet a further sign that JTC1/SC34 has taken a decent into vendor domination and reduced transparency.

Oh, and where was I? I was on vacation. (Yes, I am allowed vacation). I was in Colorado, spending some time above the timberline and among the rocks.

Giving the Finger to the DIS 29500 Appellants

2008/08/17 By Rob 16 Comments

The news broke on Friday, with the ISO press release, and additional coverage and analysis by Andy Updegrove and on Groklaw. But it would be remiss if I did not share a few details on how, true to form, the end of this DIS 29500 process was botched.

First, let’s put this in perspective. We’re talking about members of an organization, in this case four members of ISO/IEC JTC1, raising an appeal under the rules of that organization, alleging that the organization failed to follow its own rules. Almost every organization has a provision for dispute resolution, including the rights of members to appeal the decisions of elected officers or staff. This is a basic part of governance.

It is a worthwhile exercise to see how this “right to appeal” is handled by other SDO’s. Let’s take a few examples from other organizations that deal with tech standards.

First, let’s look at OASIS, a consortium that creates XML standards, like ODF. Any three OASIS members may lodge an appeal if they believe that OASIS procedures have been violated. Resolution is first attempted via correspondence, but if that fails to satisfy the appellants, they then may request a in-person hearing at the next OASIS Board of Directors meeting, where they can present their complaint. This request cannot be denied. It is a right of the members.

INCITS, the US NB in JTC1 has a different approach to appeals, detailed in section 5.8 of their RD-2 [pdf] Procedures guide. Appeals in INCITS are based on the following principles:

Appeals shall be addressed promptly and a decision made expeditiously.

The right of the involved parties to present their cases shall not be denied.

These procedures shall provide for participation by all parties concerned without imposing an undue burden on them.

Consideration of appeals shall be fair and unbiased and shall fully address the concerns expressed.

Records of appeals shall be kept and made available upon request. The INCITS Secretariat may levy a nominal charge to cover the cost of reproduction, handling and distribution for requests received from other than the involved parties.

Any INCITS member may lodge an appeal, and if an informal attempt at resolution with the INCITS Secretariat fails, an appeals panel is formed to hear the appeal. The impartiality and balance of the appeals panel is explicitly considered:

The appeals panel shall consist of three individuals who have not been directly involved in the matter in dispute. At least two members shall be acceptable to the appellant and at least two shall be acceptable to the INCITS Secretariat.

From large consortia, to NB’s, let’s poke around further and look at a industry group, AIIM, with a standards program focused on enterprise content management (ECM) technologies. Section 7.0 of their Policies and Procedures [pdf] manual defines their appeals process.

Persons who have directly and materially affected interests and who believe they have been or will be adversely affected by any procedural action or inaction by AIIM as a standards developer with regard to the development of a proposed American National Standard or the revision, reaffirmation, or withdrawal of an existing American National Standard, have the right to appeal.

The appeal is heard by three member panel, selected as in INCITS to be impartial and balanced:

The appeals panel shall consist of three members selected from the AIIM membership in addition to the Chairperson. The Chairperson of the panel shall be the Standards Board Chairperson, and shall not have a vote in the decision of the panel. The voting members of the panel shall not have been directly involved in the matter in dispute, and not be currently involved in the development of the standard(s) in question, and shall not represent or be an employee of an interest that can be made directly or materially affected by any decision made by or to be made in the dispute. The voting members of the appeals panel shall be agreed to by both the appellant and the respondent.

Perhaps readers can post other summaries of SDO appeals procedures, to give a broader sense of what the common features are. From what I can tell, the best practices are:

Members have a right to appeal decisions of the organization, and to have their appeal heard and considered, in person, by a panel chosen to be impartial and balanced. Although the appellants are not guaranteed that their views will prevail, the rules do no allow the organization to repress the appeal and not let it be heard.

So with that as background, it is interesting so observe how ISO/IEC JTC1’s antiquated cold war era rules in effect serve to stifle criticism, repress dissent, and prevent even a hearing on the merits of an appeal. As I’ll show, even with this strong organizational bias against appeals, the current DIS 29500 were only dismissed with assistance from a poorly written ballot question, NB confusion resulting in contradictory votes, and an unwillingness of committee chairs to attempt to reach consensus. Organizational failures, in the end, are usually leadership failures.

For an appeal in JTC1 to be heard, two different committees, ISO/TMB and IEC/SMB must first agree to allow the appeal to be heard. The reader should note the increased difficulty of getting two different committees to agree on the same decision, and consider the following mathematical diversion.

———

Suppose you are pushing for an proposal that has, on average, 50% support within a given organization. It is put to a vote in a subcommittee drawn randomly from that population. What is the probability that the proposal will pass a vote in that subcommittee?

50%. I think most of us have an intuitive sense of that.

But what if there are two subcommittees drawn randomly from that organization, and the proposal must win a vote in each one of the subcommittees, what is the chance it will pass?

Is it still 50%? No. I hope most of us have that intuitive sense that the need to pass two committees is harder than passing a single committee. In fact, your chances of approval could be as low as 25%, depending on whether the two committees make independent decisions, or whether there are factors that cause their votes to be partially correlated.

But the general rule is: the more stages of approval required, the less your chances of success

———

In the particular case of the DIS 29500 appeals, the IEC/SMB requires a 2/3 super majority to approve a ballot. ISO/TMB presumably requires only simple majority. (Like most of JTC1 Directives, this is not explicitly defined).

The astute reader will note that the odds are against the appellants even getting their appeal considered by a panel. In fact, these committee odds match what is required to impeach a U.S. President (50% in House) and remove him from office (2/3 in the Senate).

Given these odds, how did the appellants fair? In ISO/TMB the first irony comes with the title of the ballot:

Remember, a core matter of the appeals is the mistreatment of the contradiction phase of the DIS process, and one of the core matters of the contradiction arguments raised was that the official name of the DIS, “Office Open XML” bore a close and confusing resemblance to the submitters main competitor, Open Office, and that this would lead to confusion in the name of the standard. Well here we are, and in denying this appeal ISO/TMB commits that same error, giving the incorrect name of the standard!

Looking at the actual results, for each of the 4 appeals, ISO/TMB tied on two of them, 6-6, and voted not to pursue two others by 7-5 and 8-3 votes. So, it was very close. In fact I am a bit surprised they simply dropped further consideration of the appeals when they had tied votes like that.

A basic rule that applies to voting is JTC1 Directives, 9.1.3:

The Chairman has no vote and questions on which the vote is equally divided shall be subject to further discussion.

In this case, the Chair (Denmark) did not vote. That is correct. But why did the procedure end with two appeals showing a 6-6 “equally divided” vote? According to the rules this should be leading to further discussion. An equally divided vote is as far from consensus as one can get. Is this how they want to leave it, just hanging like this?

In IEC/SMB, the voting results are even more bizarre. I don’t know quite what to make of them. So just a few quick observations.

First, a motion should be carefully worded to it is clear what will happen if the motion passes, and what will happen if the motion fails. A Chair should insist on this, and indeed that is one of their primary duties as Chair, to ensure that questions put to their committee are clear. However, in the case of the DIS 29500 appeals, the ballot questions, as dictated by the Secretaries General, were muddled. I remarked on this in a previous blog post, and other readers observed this as well. Whether done by malice or incompetence, the ballot questions were destined to cause confusion.

The reported results indeed were muddled, as you can see here:

Of the 15 SMB members only two (China and the Netherlands) followed the explicit instructions and voted either the questions in Part A or in Part B (but not both). Both China and the Netherlands voted in one part, and abstained in the other part.

Most members voted both sections, but while expressing a consistent intent, e.g., vote No for not processing Brazil’s appeal further, but vote Yes for processing Brazil’s appeal further.

However, it appears that three other NB’s voted inconsistent, contradictory instructions. In fact one NB (Canada) gave exactly the same votes on section B as in section A, essentially canceling out their vote on every single question. This was from an NB whose written comments stated they they strongly supported hearing the appeals further. Similarly, the votes from Korea are partially contradictory.

We can attempt to reconstruct what a less-confusing ballot would have yielded. For example, take the questions in part A, whether “not to process the appeal any further”, where the recorded results were 8-4-3 for the Brazilian appeal, yielding a 2/3 super majority (ignoring the 3 abstentions). But note then that two of the three abstaining NB’s (China and the Netherlands) in fact voted in the affirmative for the question on whether to process the Brazil’s appeal in part B. It looks like the only reason why they abstained in Part A is that they actually followed the ballot instructions and cast a vote in Section A or Section B. If we apply their clear intent consistently to the Part A question, then the results become 8-6-1 and the motion to “not process the appeal any further” would have failed for lack of 2/3 majority.

I cannot make sense of Korea’s votes. Although they seem to have supported two appeals, while not supporting two other appeals, their inconsistent votes make it impossible to tell which ones they supported and which ones they did not.

Needless to say, a ballot that yields results where it is impossible to tell what the voters wanted is a hallmark of a seriously flawed, useless ballot. The SMB results are tainted by a poorly written ballot question, given to them by the Secretaries General, which has clearly caused confusion among the SMB voters, and which had a material effect on the results. My analysis of IEC/SMB shows that, like ISO/TMB’s vote, the results are nearly equally divided, and IEC/SMB should hang their head in shame if they persist in denying a hearing to these four appeals because of ambiguous results from a poorly written, botched ballot.

This is why ballot results should be released publicly and subject to scrutiny. I do not believe we can trust ISO/IEC to perform quality control on their own processes. The rot is too deep.