Rob

The news broke on Friday, with the ISO press release, and additional coverage and analysis by Andy Updegrove and on Groklaw. But it would be remiss if I did not share a few details on how, true to form, the end of this DIS 29500 process was botched.

First, let’s put this in perspective. We’re talking about members of an organization, in this case four members of ISO/IEC JTC1, raising an appeal under the rules of that organization, alleging that the organization failed to follow its own rules. Almost every organization has a provision for dispute resolution, including the rights of members to appeal the decisions of elected officers or staff. This is a basic part of governance.

It is a worthwhile exercise to see how this “right to appeal” is handled by other SDO’s. Let’s take a few examples from other organizations that deal with tech standards.

First, let’s look at OASIS, a consortium that creates XML standards, like ODF. Any three OASIS members may lodge an appeal if they believe that OASIS procedures have been violated. Resolution is first attempted via correspondence, but if that fails to satisfy the appellants, they then may request a in-person hearing at the next OASIS Board of Directors meeting, where they can present their complaint. This request cannot be denied. It is a right of the members.

INCITS, the US NB in JTC1 has a different approach to appeals, detailed in section 5.8 of their RD-2 [pdf] Procedures guide. Appeals in INCITS are based on the following principles:

• Appeals shall be addressed promptly and a decision made expeditiously.
• The right of the involved parties to present their cases shall not be denied.
• These procedures shall provide for participation by all parties concerned without imposing an undue burden on them.
• Consideration of appeals shall be fair and unbiased and shall fully address the concerns expressed.
• Records of appeals shall be kept and made available upon request. The INCITS Secretariat may levy a nominal charge to cover the cost of reproduction, handling and distribution for requests received from other than the involved parties.

Any INCITS member may lodge an appeal, and if an informal attempt at resolution with the INCITS Secretariat fails, an appeals panel is formed to hear the appeal. The impartiality and balance of the appeals panel is explicitly considered:

The appeals panel shall consist of three individuals who have not been directly involved in the matter in dispute. At least two members shall be acceptable to the appellant and at least two shall be acceptable to the INCITS Secretariat.

From large consortia, to NB’s, let’s poke around further and look at a industry group, AIIM, with a standards program focused on enterprise content management (ECM) technologies. Section 7.0 of their Policies and Procedures [pdf] manual defines their appeals process.

Persons who have directly and materially affected interests and who believe they have been or will be adversely affected by any procedural action or inaction by AIIM as a standards developer with regard to the development of a proposed American National Standard or the revision, reaffirmation, or withdrawal of an existing American National Standard, have the right to appeal.

The appeal is heard by three member panel, selected as in INCITS to be impartial and balanced:

The appeals panel shall consist of three members selected from the AIIM membership in addition to the Chairperson. The Chairperson of the panel shall be the Standards Board Chairperson, and shall not have a vote in the decision of the panel. The voting members of the panel shall not have been directly involved in the matter in dispute, and not be currently involved in the development of the standard(s) in question, and shall not represent or be an employee of an interest that can be made directly or materially affected by any decision made by or to be made in the dispute. The voting members of the appeals panel shall be agreed to by both the appellant and the respondent.

Perhaps readers can post other summaries of SDO appeals procedures, to give a broader sense of what the common features are. From what I can tell, the best practices are:

Members have a right to appeal decisions of the organization, and to have their appeal heard and considered, in person, by a panel chosen to be impartial and balanced. Although the appellants are not guaranteed that their views will prevail, the rules do no allow the organization to repress the appeal and not let it be heard.

So with that as background, it is interesting so observe how ISO/IEC JTC1’s antiquated cold war era rules in effect serve to stifle criticism, repress dissent, and prevent even a hearing on the merits of an appeal. As I’ll show, even with this strong organizational bias against appeals, the current DIS 29500 were only dismissed with assistance from a poorly written ballot question, NB confusion resulting in contradictory votes, and an unwillingness of committee chairs to attempt to reach consensus. Organizational failures, in the end, are usually leadership failures.

For an appeal in JTC1 to be heard, two different committees, ISO/TMB and IEC/SMB must first agree to allow the appeal to be heard. The reader should note the increased difficulty of getting two different committees to agree on the same decision, and consider the following mathematical diversion.

———

Suppose you are pushing for an proposal that has, on average, 50% support within a given organization. It is put to a vote in a subcommittee drawn randomly from that population. What is the probability that the proposal will pass a vote in that subcommittee?

50%. I think most of us have an intuitive sense of that.

But what if there are two subcommittees drawn randomly from that organization, and the proposal must win a vote in each one of the subcommittees, what is the chance it will pass?

Is it still 50%? No. I hope most of us have that intuitive sense that the need to pass two committees is harder than passing a single committee. In fact, your chances of approval could be as low as 25%, depending on whether the two committees make independent decisions, or whether there are factors that cause their votes to be partially correlated.

But the general rule is: the more stages of approval required, the less your chances of success

———

In the particular case of the DIS 29500 appeals, the IEC/SMB requires a 2/3 super majority to approve a ballot. ISO/TMB presumably requires only simple majority. (Like most of JTC1 Directives, this is not explicitly defined).

The astute reader will note that the odds are against the appellants even getting their appeal considered by a panel. In fact, these committee odds match what is required to impeach a U.S. President (50% in House) and remove him from office (2/3 in the Senate).

Given these odds, how did the appellants fair? In ISO/TMB the first irony comes with the title of the ballot:

Remember, a core matter of the appeals is the mistreatment of the contradiction phase of the DIS process, and one of the core matters of the contradiction arguments raised was that the official name of the DIS, “Office Open XML” bore a close and confusing resemblance to the submitters main competitor, Open Office, and that this would lead to confusion in the name of the standard. Well here we are, and in denying this appeal ISO/TMB commits that same error, giving the incorrect name of the standard!

Looking at the actual results, for each of the 4 appeals, ISO/TMB tied on two of them, 6-6, and voted not to pursue two others by 7-5 and 8-3 votes. So, it was very close. In fact I am a bit surprised they simply dropped further consideration of the appeals when they had tied votes like that.

A basic rule that applies to voting is JTC1 Directives, 9.1.3:

The Chairman has no vote and questions on which the vote is equally divided shall be subject to further discussion.

In this case, the Chair (Denmark) did not vote. That is correct. But why did the procedure end with two appeals showing a 6-6 “equally divided” vote? According to the rules this should be leading to further discussion. An equally divided vote is as far from consensus as one can get. Is this how they want to leave it, just hanging like this?

In IEC/SMB, the voting results are even more bizarre. I don’t know quite what to make of them. So just a few quick observations.

First, a motion should be carefully worded to it is clear what will happen if the motion passes, and what will happen if the motion fails. A Chair should insist on this, and indeed that is one of their primary duties as Chair, to ensure that questions put to their committee are clear. However, in the case of the DIS 29500 appeals, the ballot questions, as dictated by the Secretaries General, were muddled. I remarked on this in a previous blog post, and other readers observed this as well. Whether done by malice or incompetence, the ballot questions were destined to cause confusion.

The reported results indeed were muddled, as you can see here:

Of the 15 SMB members only two (China and the Netherlands) followed the explicit instructions and voted either the questions in Part A or in Part B (but not both). Both China and the Netherlands voted in one part, and abstained in the other part.

Most members voted both sections, but while expressing a consistent intent, e.g., vote No for not processing Brazil’s appeal further, but vote Yes for processing Brazil’s appeal further.

However, it appears that three other NB’s voted inconsistent, contradictory instructions. In fact one NB (Canada) gave exactly the same votes on section B as in section A, essentially canceling out their vote on every single question. This was from an NB whose written comments stated they they strongly supported hearing the appeals further. Similarly, the votes from Korea are partially contradictory.

We can attempt to reconstruct what a less-confusing ballot would have yielded. For example, take the questions in part A, whether “not to process the appeal any further”, where the recorded results were 8-4-3 for the Brazilian appeal, yielding a 2/3 super majority (ignoring the 3 abstentions). But note then that two of the three abstaining NB’s (China and the Netherlands) in fact voted in the affirmative for the question on whether to process the Brazil’s appeal in part B. It looks like the only reason why they abstained in Part A is that they actually followed the ballot instructions and cast a vote in Section A or Section B. If we apply their clear intent consistently to the Part A question, then the results become 8-6-1 and the motion to “not process the appeal any further” would have failed for lack of 2/3 majority.

I cannot make sense of Korea’s votes. Although they seem to have supported two appeals, while not supporting two other appeals, their inconsistent votes make it impossible to tell which ones they supported and which ones they did not.

Needless to say, a ballot that yields results where it is impossible to tell what the voters wanted is a hallmark of a seriously flawed, useless ballot. The SMB results are tainted by a poorly written ballot question, given to them by the Secretaries General, which has clearly caused confusion among the SMB voters, and which had a material effect on the results. My analysis of IEC/SMB shows that, like ISO/TMB’s vote, the results are nearly equally divided, and IEC/SMB should hang their head in shame if they persist in denying a hearing to these four appeals because of ambiguous results from a poorly written, botched ballot.

This is why ballot results should be released publicly and subject to scrutiny. I do not believe we can trust ISO/IEC to perform quality control on their own processes. The rot is too deep.