The news broke on Friday, with the ISO press release, and additional coverage and analysis by Andy Updegrove and on Groklaw. But it would be remiss if I did not share a few details on how, true to form, the end of this DIS 29500 process was botched.
First, let’s put this in perspective. We’re talking about members of an organization, in this case four members of ISO/IEC JTC1, raising an appeal under the rules of that organization, alleging that the organization failed to follow its own rules. Almost every organization has a provision for dispute resolution, including the rights of members to appeal the decisions of elected officers or staff. This is a basic part of governance.
It is a worthwhile exercise to see how this “right to appeal” is handled by other SDO’s. Let’s take a few examples from other organizations that deal with tech standards.
First, let’s look at OASIS, a consortium that creates XML standards, like ODF. Any three OASIS members may lodge an appeal if they believe that OASIS procedures have been violated. Resolution is first attempted via correspondence, but if that fails to satisfy the appellants, they then may request a in-person hearing at the next OASIS Board of Directors meeting, where they can present their complaint. This request cannot be denied. It is a right of the members.
INCITS, the US NB in JTC1 has a different approach to appeals, detailed in section 5.8 of their RD-2 [pdf] Procedures guide. Appeals in INCITS are based on the following principles:
- Appeals shall be addressed promptly and a decision made expeditiously.
- The right of the involved parties to present their cases shall not be denied.
- These procedures shall provide for participation by all parties concerned without imposing an undue burden on them.
- Consideration of appeals shall be fair and unbiased and shall fully address the concerns expressed.
- Records of appeals shall be kept and made available upon request. The INCITS Secretariat may levy a nominal charge to cover the cost of reproduction, handling and distribution for requests received from other than the involved parties.
Any INCITS member may lodge an appeal, and if an informal attempt at resolution with the INCITS Secretariat fails, an appeals panel is formed to hear the appeal. The impartiality and balance of the appeals panel is explicitly considered:
The appeals panel shall consist of three individuals who have not been directly involved in the matter in dispute. At least two members shall be acceptable to the appellant and at least two shall be acceptable to the INCITS Secretariat.
From large consortia, to NB’s, let’s poke around further and look at a industry group, AIIM, with a standards program focused on enterprise content management (ECM) technologies. Section 7.0 of their Policies and Procedures [pdf] manual defines their appeals process.
Persons who have directly and materially affected interests and who believe they have been or will be adversely affected by any procedural action or inaction by AIIM as a standards developer with regard to the development of a proposed American National Standard or the revision, reaffirmation, or withdrawal of an existing American National Standard, have the right to appeal.
The appeal is heard by three member panel, selected as in INCITS to be impartial and balanced:
The appeals panel shall consist of three members selected from the AIIM membership in addition to the Chairperson. The Chairperson of the panel shall be the Standards Board Chairperson, and shall not have a vote in the decision of the panel. The voting members of the panel shall not have been directly involved in the matter in dispute, and not be currently involved in the development of the standard(s) in question, and shall not represent or be an employee of an interest that can be made directly or materially affected by any decision made by or to be made in the dispute. The voting members of the appeals panel shall be agreed to by both the appellant and the respondent.
Perhaps readers can post other summaries of SDO appeals procedures, to give a broader sense of what the common features are. From what I can tell, the best practices are:
Members have a right to appeal decisions of the organization, and to have their appeal heard and considered, in person, by a panel chosen to be impartial and balanced. Although the appellants are not guaranteed that their views will prevail, the rules do no allow the organization to repress the appeal and not let it be heard.
So with that as background, it is interesting so observe how ISO/IEC JTC1’s antiquated cold war era rules in effect serve to stifle criticism, repress dissent, and prevent even a hearing on the merits of an appeal. As I’ll show, even with this strong organizational bias against appeals, the current DIS 29500 were only dismissed with assistance from a poorly written ballot question, NB confusion resulting in contradictory votes, and an unwillingness of committee chairs to attempt to reach consensus. Organizational failures, in the end, are usually leadership failures.
For an appeal in JTC1 to be heard, two different committees, ISO/TMB and IEC/SMB must first agree to allow the appeal to be heard. The reader should note the increased difficulty of getting two different committees to agree on the same decision, and consider the following mathematical diversion.
———
Suppose you are pushing for an proposal that has, on average, 50% support within a given organization. It is put to a vote in a subcommittee drawn randomly from that population. What is the probability that the proposal will pass a vote in that subcommittee?
50%. I think most of us have an intuitive sense of that.
But what if there are two subcommittees drawn randomly from that organization, and the proposal must win a vote in each one of the subcommittees, what is the chance it will pass?
Is it still 50%? No. I hope most of us have that intuitive sense that the need to pass two committees is harder than passing a single committee. In fact, your chances of approval could be as low as 25%, depending on whether the two committees make independent decisions, or whether there are factors that cause their votes to be partially correlated.
But the general rule is: the more stages of approval required, the less your chances of success
———
In the particular case of the DIS 29500 appeals, the IEC/SMB requires a 2/3 super majority to approve a ballot. ISO/TMB presumably requires only simple majority. (Like most of JTC1 Directives, this is not explicitly defined).
The astute reader will note that the odds are against the appellants even getting their appeal considered by a panel. In fact, these committee odds match what is required to impeach a U.S. President (50% in House) and remove him from office (2/3 in the Senate).
Given these odds, how did the appellants fair? In ISO/TMB the first irony comes with the title of the ballot:
Remember, a core matter of the appeals is the mistreatment of the contradiction phase of the DIS process, and one of the core matters of the contradiction arguments raised was that the official name of the DIS, “Office Open XML” bore a close and confusing resemblance to the submitters main competitor, Open Office, and that this would lead to confusion in the name of the standard. Well here we are, and in denying this appeal ISO/TMB commits that same error, giving the incorrect name of the standard!
Looking at the actual results, for each of the 4 appeals, ISO/TMB tied on two of them, 6-6, and voted not to pursue two others by 7-5 and 8-3 votes. So, it was very close. In fact I am a bit surprised they simply dropped further consideration of the appeals when they had tied votes like that.
A basic rule that applies to voting is JTC1 Directives, 9.1.3:
The Chairman has no vote and questions on which the vote is equally divided shall be subject to further discussion.
In this case, the Chair (Denmark) did not vote. That is correct. But why did the procedure end with two appeals showing a 6-6 “equally divided” vote? According to the rules this should be leading to further discussion. An equally divided vote is as far from consensus as one can get. Is this how they want to leave it, just hanging like this?
In IEC/SMB, the voting results are even more bizarre. I don’t know quite what to make of them. So just a few quick observations.
First, a motion should be carefully worded to it is clear what will happen if the motion passes, and what will happen if the motion fails. A Chair should insist on this, and indeed that is one of their primary duties as Chair, to ensure that questions put to their committee are clear. However, in the case of the DIS 29500 appeals, the ballot questions, as dictated by the Secretaries General, were muddled. I remarked on this in a previous blog post, and other readers observed this as well. Whether done by malice or incompetence, the ballot questions were destined to cause confusion.
The reported results indeed were muddled, as you can see here:
Of the 15 SMB members only two (China and the Netherlands) followed the explicit instructions and voted either the questions in Part A or in Part B (but not both). Both China and the Netherlands voted in one part, and abstained in the other part.
Most members voted both sections, but while expressing a consistent intent, e.g., vote No for not processing Brazil’s appeal further, but vote Yes for processing Brazil’s appeal further.
However, it appears that three other NB’s voted inconsistent, contradictory instructions. In fact one NB (Canada) gave exactly the same votes on section B as in section A, essentially canceling out their vote on every single question. This was from an NB whose written comments stated they they strongly supported hearing the appeals further. Similarly, the votes from Korea are partially contradictory.
We can attempt to reconstruct what a less-confusing ballot would have yielded. For example, take the questions in part A, whether “not to process the appeal any further”, where the recorded results were 8-4-3 for the Brazilian appeal, yielding a 2/3 super majority (ignoring the 3 abstentions). But note then that two of the three abstaining NB’s (China and the Netherlands) in fact voted in the affirmative for the question on whether to process the Brazil’s appeal in part B. It looks like the only reason why they abstained in Part A is that they actually followed the ballot instructions and cast a vote in Section A or Section B. If we apply their clear intent consistently to the Part A question, then the results become 8-6-1 and the motion to “not process the appeal any further” would have failed for lack of 2/3 majority.
I cannot make sense of Korea’s votes. Although they seem to have supported two appeals, while not supporting two other appeals, their inconsistent votes make it impossible to tell which ones they supported and which ones they did not.
Needless to say, a ballot that yields results where it is impossible to tell what the voters wanted is a hallmark of a seriously flawed, useless ballot. The SMB results are tainted by a poorly written ballot question, given to them by the Secretaries General, which has clearly caused confusion among the SMB voters, and which had a material effect on the results. My analysis of IEC/SMB shows that, like ISO/TMB’s vote, the results are nearly equally divided, and IEC/SMB should hang their head in shame if they persist in denying a hearing to these four appeals because of ambiguous results from a poorly written, botched ballot.
This is why ballot results should be released publicly and subject to scrutiny. I do not believe we can trust ISO/IEC to perform quality control on their own processes. The rot is too deep.
Rob, isn’t it a case where there are valid reasons to appeal the TMB/SMB’s decision, as stated in the JTC1 Directives:
“11.4 Appeal Against a Decision of the TMB/SMBs
An appeal against a decision of the TMB/SMB shall be submitted to the Secretaries-General with full documentation on all stages of the case.
The Secretaries-General shall refer the appeal together with their comments to the members of the Councils within one month after receipt of the appeal.
The Councils shall make their decision within three months.”
We probably know already how such an appeal will be handled by the Secretaries-general, but it will force them to show how much they are biased.
Will they dare to continue their farce ?
By the way, what are the ISO and IEC rules for ousting a Secretary-General ?
There is also the wording of the questions, specifically, “which would require setting up of a conciliation panel”. As I read the questions, they are crafted to skew to the preferred results.
I don’t know what effect the wording would have on one who’s first language is not English, but I suspect that fluency in English would produce the same effect.
Given these odds, how did the appellants fair? In ISO/TMB the first irony comes with the title of the ballot: “Appeals on ISO/IEC DIS 29500 Open Office XML”.
Is that title of the ballot correct? Shouldn’t it have at least used “Office Open XML”? (which would still have been inexcusably confusing)
If there is a requirement for a 2/3 approval, the two actions (a and b) should be considered separate issues. It was a mistake to instruct the boards to vote on only one of actions.
Also, if the instructions are taken literally, indicating an abstention on one action means you shouldn’t vote on the other action. It seems to me that none of the SMB voters followed the instructions.
Luc, I think the first step is for SMB members to look closely at their ballot results and realize how screwed up that ballot was and for them to voluntarily post another ballot. But if they remain intransigent, then I think there is a strong case for an appeal. On the TMB side, the objection is that they failed in their responsibility to further discuss equally divided questions by leaving them hanging with a 6-6 vote.
Thomas, the ability to write the ballot question is a powerful thing, and in ISO is typical used to bias the vote. In this case the Secretaries General dictated the ballot question.
Saul, you are correct. They didn’t even get the name of the standard correct, which is one example of the overall low level of diligence given to consideration of these appeals.
Rob.
I would like to remain anonymous for this post, but I will try to contact you by other means so we can share specifics.
As a person who worked very hard to fill one of the appeals rejected on Friday, I’m appealed not only by the facts you kindly relate on you blog, but also, by the unbelievable obscurity surrounding all this process. My NB is still to receive any formal communication from ISO/IEC about the outcome of the appeals, and that’s just one example of it.
I would like to hear your opinion about a possible appeal based on JTC1 directives 11.4, and why on earth would the ISO press release state that DIS 29500 is “subject to no further appeals against the decision”.
Also, do you know what the “councils” are and who’s on it?
regards
The questions and the process weren’t flawed, the TMBs got exactly the result they wanted. I guess Microsoft gave them a bonus.
Doug, JTC1 has been around since 1987, over 20 years. They have squandered time, money and good will, in large quantities in that interval, and have precious little to show for it.
Their procedures are poorly written and fail even their most basic purpose — providing an unambiguous set of instructions for reviewing and approving standards. What have they been doing for 20 years, if their appeals process gives no mention of a 2/3 approval criterion, if their BRM process does not clearly mention which class of NB’s can vote, and if their contradiction phase does not define “contradiction”. Twenty frickin’ years — what the hell have they been doing all that time, that they have not managed in that 20 years to develop a workable procedures document?
Even though JTC1 is supposed to be the premier body for international technology standards, the organization itself is still run on technology that would have been embarrassing 20 years ago. Consider: Even today JTC1 is still debating whether it is permissible to hold a meeting by teleconference. Alexander Graham Bell would, if he had not died in 1922, be proud that JTC1 was finally making the bold leap into the 20th century. Discussion forums, mailing lists, wiki’s? Forget about it. Come back in another 20 years and maybe they’ll have something worked out.
As for transparency and accountability, JTC1 makes even Ecma look good. In fact I suspect that one reason why Microsoft is eager to transfer control of OOXML to JTC1 is that is the only place they can move it to and be less accountable and more secretive than in Ecma. For example, in Ecma, they would be required to make drafts public and solicit public comments. In JTC1, this is not required. No public input is required, or indeed even allowed.
So I cannot tell what snafu’s are caused by malice and what are caused by incompetence, but to see JTC1 is such bad shape after 20 years, with no signs of improvement, quality control or even engaged leadership, it seems to me that organizationally it is already brain dead, but, like a dead tree, stands teetering in forest for a few more years, infested by insects and other vermin, until a big enough wind comes and blows it over.
saulgoode said…
“If there is a requirement for a 2/3 approval, the two actions (a and b) should be considered separate issues. It was a mistake to instruct the boards to vote on only one of actions.”
I don’t follow why this is so. Part A is “not to process the appeal any further” and Part B is “to process one or more of the appeals, which would require the setting up of a conciliation panel”.
Now I parse those phrases as follows. In Part B, the “, which would require …” is a no-op because that’s a consequence of the choice, not the choice itself. The difference between “the appeal” and “one or more of the appeals” seems to be irrelevant too, since both questions are being asked 4 times, once for each appeal.
Therefore it boils down to a choice between “not to process the appeal” or “to process the appeal”. I don’t understand why the acceptance threshold makes any difference to whether this is one choice or two. It seems to me like a choice between opposites – poorly worded and biased, nevertheless.
Re the 11.4 question, and the section that states “According to the ISO/IEC rules, DIS 29500 can now proceed to publication as an ISO/IEC International Standard. This is expected to take place within the next few weeks on completion of final processing of the document, and subject to no further appeals against the decision.”
I think this should be read as:
“According to the ISO/IEC rules, DIS 29500 can now proceed to publication as an ISO/IEC International Standard. This is expected to take place within the next few weeks, subject to a) completion of final processing of the document, and b) subject to [the condition that] no further appeals against the decision [is made].”
This means as far as I can tell that the original 4 appealing bodies very much can appeal this rejection and the issue at stake again. And I for one very much hope they do. Even if in practice it should turn out that OOXML is dead in the water, there are nevertheless matters of principle and due process here that are critically valuable IMO to pursue.
“Therefore it boils down to a choice between “not to process the appeal” or “to process the appeal”. I don’t understand why the acceptance threshold makes any difference to whether this is one choice or two. It seems to me like a choice between opposites”
Because what does it mean to vote Yes on the first question AND Yes on the second question, as Canada did. How do you tabulate that vote?
Even an Abstain vote in the first half doesn’t help you. If there’s a 2/3 major requirement for either question, you need a 2:1 ratio out of 15. But you could get 8Y, 4N and 3A from the first question, followed by 6Y, 3N and 6A in the second question. Then you have a 2/3 majority for BOTH QUESTIONS, even those their opposite questions.
The ballot should have said…
Please vote yes, no or abstain on the following:
1) Continue the Appeal from Brazil
2) Continue the Appeal from India
3) Continue the Appeal from Venezeula
4) Continue the Appeal from South Africa
That’s unambiguous. It’s not possible to contradict your own vote. You only have to answer once for each appeal.
Normally one should never attribute malice in a case where incompetence is an adequate explanation, but I can’t see how the Secretary-General of the IEC could be that staggeringly bad at his job. Katherine Harris had more ground to stand on than the IEC does here.
Rob, you provided a table with the votes of the IEC/SMB. Could you please provide also the details of the ISO/TMB votes ? Thanks.
About the OOXML and ISO/IEC JTC1 debacle, Patrick Durusau understood it all: it’s IBM’s fault…
See http://www.durusau.net/publications/listening.pdf and http://www.durusau.net/publications/reform.pdf
Patrick is a good editor. I hope his resume features his editing work, and not his analysis of ISO.
In one post Patrick suggests that JTC1 is important because it has NB representation, not corporate representation. But in the other post he suggests that the NB views can be ignored if they are in the minority, at which point he merely dismisses them as being “IBM” views. So his view of NB representation seems to be limited to respect for the views of those he agrees with, which is hardly a principled view.
Patrick also says “The objections were heard, considered and the vast majority of us simply disagree.” The fact is that the appeals were not heard. What occurred was that ISO leadership accompanied the written appeals with a deceptive response, one which was economical with the truth. Presented with this deceptive analysis, along with the original appeals, we still saw the original appellants were joined by other NB’s, including two that had voted in favor of OOXML, to a split decision in SMB and TMB. As I reported in this post, the TMB vote was tied, and the SMB vote, after you correct for a broken ballot, was nearly so. Of course, ISO can be counted on to invent new rules in a pinch, so we saw this magic 2/3 rule in TMB they pulled out of the air, and that is what prevented the appeals from being heard.
The point of an appeals process is to ensure that the rules are followed. The rule of law should apply regardless of whether the majority prefers the outcome that came from ignoring the rules. Every NB should expect to be in the minority at some time. Do they want this to be an organization which can freely fail to follow its own rules, to allow large corporations to stuff its committees, to hold BRM’s that are merely ceremonial,to allow ad-hoc treatment of voting rights, to issue confusing and contradictory ballots and generally to turn a blind eye to every form of abuse?
The fact that the DIS 29500 received sufficient votes in the technical approval ballot means absolutely nothing with regards to the appeals. It is embarrassing that Patrick does not see that, considering he has legal training. This is just like Robert Mugage’s recent electoral victory in Zimbabwe does not justify or excuse the abuses that lead to that victory. The end does not justify the means. Ignore that point and you are on a short path that ends with a knock on the door at night, a muffled scream, a short drive, and a bullet to the head.
Rob, see http://www.reuters.com/article/technologyNews/idUSL140109520080901
and http://www.raffee.co.za/post/47974926/consegi-declaration
In short, representatives of state IT organisations from Brazil, South Africa, Venezuela, Ecuador, Cuba and Paraguay say:
“Whereas we do not intend to waste any more resources on lobbying our national bodies to pursue the appeals further, we feel it is important to make the following points clear:
– The bending of the rules to facilitate the fast track processing of DIS29500 remains a significant concern to us. That the ISO TMB did not deem it necessary to properly explore the substance of the appeals must, of necessity, put confidence in those institutions ability to meet our national requirements into question.
[…]
Whereas in the past it has been assumed that an ISO/IEC standard should automatically be considered for use within government, clearly this position no longer stands.”