Archives for 2007

Microsoft’s Stephen McGibbon updates us on the OOXML Ballot Resolution Meeting (BRM), now scheduled for February 25-29 in Geneva. He ends his otherwise informative post with a little jab:

I hear that IBM is still telling national bodies that a BRM isn’t guaranteed. I am unsure how IBM reached that conclusion but this seems to be concrete evidence to the contrary.

Well, let me help refresh Mr. McGibbon’s seemingly repressed memories.

First, scheduling a BRM does not guarantee it will be held. For example, have you heard of DIS 26926 “C++/CLI”? It was another Microsoft/Ecma Fast Track, just last year. The BRM meeting announcement went out on 25 October 2006, saying the BRM would be held 13-15 April 2007 in Oxford, England. Stephen, do you recall that BRM by any chance? Of course not, because it was canceled in February 2007 with the following message from the SC22 Secretariat:

We have been advised that the comments accompanying the Fast Track ballot for DIS 26926 are not resolvable and that holding a Ballot Resolution Meeting (BRM) would not be productive or result in a document that would be acceptable to the JTC 1 National Bodies. Therefore, our proposal is to not hold the BRM and to cancel the project.

So there is one example of a BRM that was scheduled and then canceled.

Want another? Sure, I can do that.

Take the case of DIS 26300 “Open Document Format.” A Ballot Resolution Meeting was scheduled for May 29 to June 1, 2006 in Seoul, Korea, concurrently with the JTC1/SC34 Plenary. But was the BRM actually held? No. It was canceled by the Plenary:

Following the advice of the JTC 1 Secretariat, JTC 1/SC 34 cancels the previously-scheduled ISO/IEC 26300 Ballot Resolution Meeting and the SC34 Secretariat will forward the revised DIS text and accompanying disposition to SC34 national bodies for a 30-day default ballot when ready.

Why? Because ODF received no Disapproval votes. Although 8 of the 23 NB submitted comments with their ballot, these were all “Approval, with Comments” votes rather rather than “Disapproval, with comments. So a BRM was not deemed necessary. Only comments that accompany Disapproval votes must be addressed at a BRM.

So there you go, two examples of BRM’s that were scheduled, but then canceled. The SC Secretariat has some discretion here. JTC1 Directives, Section 13.5 says, “In some cases the establishment of a ballot resolution group is unnecessary and the SC Secretariat can assign the task directly to the Project Editor.” The two examples given show that if a ballot passes by large margins, or fails by large margins, a BRM may not be necessary.

How about another example from the recent past, the Fast Track DIS 29361 “Information technology – Basic profile.” Their ballot closed on June 18th. Its ballot passed with 17 of 20 P-Members voting in favor of it. All Disapproval votes were accompanied by comments, as did one of the approval votes. Since there were Disapproval votes surely there must have been a BRM, right? No, that’s not how it worked. The JTC1 Secretariat decided a BRM was not necessary and the comments could be forwarded directly to the Submitter of the Fast Track for them to “review and respond”. So even having Disapproval votes does not guarantee a BRM will be held.

Does this make more sense now?

Of course, Microsoft already knows all this, and no doubt that is why they are working so hard to urge NB’s to vote “Approval, with comments” with promises that their comments will be addressed at the BRM, a BRM that might not even occur. In fact, if everyone listened to Microsoft and followed their advice then that would almost guarantee that no BRM would be held and no NB’s comments would be adopted.

The word for today is “disenfranchisement” which according to Webster’s means “to deprive of a franchise, of a legal right, or of some privilege or immunity.” You sometimes hear it used in connection with the policy in some U.S. states where convicted felons are not allowed to vote. That is legal disenfranchisement.

There is also the less savory kind of disenfranchisement, the kind that borders on electoral fraud. For example, in the 2004 presidential election there were reports of activities like:

• In Ohio, some registered voters received anonymous phone calls telling them that they were not properly registered to vote and that if they tried to vote they will be arrested.
• In Florida, registered members of one political party received phone calls reminding them to vote on November 3rd. Too bad the election was really on November 2nd.
• In Wisconsin, fliers were handed out falsely stating “If you already voted in any election this year, you can’t vote in the Presidential Election.”

An attempt to fraudulently disenfranchise a voter is despicable wherever it occurs. In a democracy we decide by having a fair debate, and then counting votes. But misrepresenting the rules and procedures in order to prevent another party from exercising their right to vote — this is universally deplored.

I just received an email from someone in a national standards committee considering the OOXML ballot, concerning false information given to his committee which suggested the Sept. 2nd ballot deadline was not real, that they actually had 30 more days to decide. I’m not going to name names in this post, but I will say that this isn’t the first note I’ve received regarding such tactics. Some of the other ploys I’ve heard of include:

• In the 3o-day contradiction period, one NB was told that the stated deadline from ISO had been extended and that they actually had two more weeks to debate before sending in their response. If they had listened to this advice, this NB would have missed the deadline and their comments would have been disregarded.
• Another NB was told that they were not allowed to vote in the 5-month ballot because they had not participated in the contradiction period. This is totally false and has no basis in JTC1 Directives or past practice. Luckily this NB decided to check the facts for themselves.
• Several NB’s were told that JTC1 had resolved all contradiction concerns with OOXML and that these issues therefore cannot be raised again in the 5-month ballot. This is utterly false. No one at JTC1 has made such a determination.
• Several NB’s have been asked not to submit comments to JTC1 at all, but to send them directly to Ecma. (Yeah, right. Just sign your absentee ballot and give it to me. I’ll make sure it gets in the mail)
• Many NB’s are being asked to throw away their right to a conditional approval position by voting Approval on a specification that they believe is full of defects that must be fixed, even though JTC1 Directives clearly states that “Conditional approval should be submitted as a disapproval vote.”
• Many NB’s are being persuaded to vote Approval with the promise that all of their comments will be “addressed at the BRM” without explaining that “addressing a comment” may entail little more than entering it in a Disposition of Comments Reports with the remark “No action taken”.

I’m expecting that such shenanigans are only going to increase as we go into the final week of this 5-month ballot. So I ask you all to remain vigilant. If you see anything like the above happening, then please post a comment. If you feel like you’ve been tricked into not voting, or voting for something that you didn’t really agree with, then remember that JTC1 allows an NB to change their vote up until Sept. 2nd. No vote is final until then. If you hear something that seems unusual or a departure from normal practice, then question it. And don’t take my word for it either. If you need an official answer, shoot off an email to the JTC1 Secretariat and the ISO Secretary General.

ISO works on a voting principle of one country/one vote. Don’t let confusion over proper voting procedures deprive your country of their vote.

If you are a regular reader of this blog, or almost any other blog covering the file format battle, then you have certainly come across comments from Stephane Rodriguez. He brings a unique perspective, knowing more about the legacy binary Excel formats than any sane person I know. So it a treat to have from Stephane a fuller exposition of some problems he ran into working with the Microsoft OOXML formats, in an technical article called Microsoft Office XML Formats? Defective by Design.

Enjoy!

27 August Update: Slashdot has further coverage of Stephane’s article. Some good comments and perspectives can be read there.

My mind sometimes works in weird associative ways. One thought leads to another, all connected, but only tenuously so. I was having a conversation the other day with my wife, and I was all over the place and it only struck me later that the topics were all connected by their tangential association to a ukulele. What would Freud say? But I digress…

I had a trip to the dentist on Monday. Whenever I have to go to the dentist I have images in my mind from the 1976 film Marathon Man, namely Lawrence Olivier as Dr. Szell, the elderly Nazi war criminal, torturing Dustin Hoffman with various unorthodox dental procedures. I figure that if I mentally prepare for the worst, the real dentist will be gentler in comparison. I sometimes mention this movie to the dentist, but they all deny ever having seen the movie. Very odd. I think they are hiding something. Surely this classic must be a staple of dental school film societies everywhere. That and the Tim Conway dentist skit from The Carol Burnett Show. What else is there in terms of great moments in dental cinema?

In any case, a story is told, perhaps apocryphal, that Hoffman prepared rigorously for his role in this movie by depriving himself of sleep for two days, so his character would appear worn and haggard. Olivier, seeing Hoffman that morning, and hearing of his co-star’s preparation, is said to have quipped, “Dear boy, next time why not try acting?”

I’m reminded of this line when I witness Microsoft’s machinations in JTC1, as they attempt to get OOXML approved. They are mounting an enormous offensive and expending great sums of money to convince ISO members that this rubbish heap of a format is acceptable as an ISO standard. Someone needs to ask, “Dear boy, next time why not try engineering?” Instead trying to force this ill considered mess through JTC1 (causing a great deal of collateral damage in the process), why not take your great base of engineering talent and produce a good standard and have that sail through JTC1 with thanks and praise?

We’re also seeing a shell game at play with the technical comments. Many of the technical flaws were uncovered and discussed on this blog back last summer and fall, before OOXML was even completed by Ecma. Microsoft didn’t fix them then. In the 30-day contradiction period, in February 2007, many NB’s raised these same issues. Microsoft didn’t fix them then, saying that they should be raised in the 5-month ballot. Now these same comments are being raised for the third time, in the 5-month ballot, and Microsoft is suggesting that they can be fixed at the ballot resolution meeting (BRM) in February 2008. At the BRM I predict that Microsoft will suggest that the issues should be fixed during maintenance of the standard. That would fit their plans well since they have already petitioned JTC1 to have the maintenance of OOXML handed over to Ecma TC45, closing the circle. Microsoft will never need to fix any problems in OOXML at this rate.

Another curious ploy is the way Microsoft is trying to convince JTC1 members that “Yes” means “No”, that if they have serious issues with OOXML a NB should still vote Approval. Let’s look into what the voting rules really are.

First a simple question to warm up. If you see a tunnel with a sign that says “exit” do you think that you can enter it as well? If you answer “No,” then congratulations, you are smarter than Microsoft thinks you are. Microsoft is essentially arguing around the globe that unless the tunnel has a sign that says “do not enter”, then you are welcome to enter the tunnel regardless of the “exit” sign. They are arguing that a NB can do anything they want unless the JTC1 Directives explicitly forbid it.

The counter argument is actually quite simple. You just need to consult Section 9.8 of the JTC1 Directives, 5th Edition, Version 3.0, which I’ve extracted below:

As it says, an Approval vote is approval of the technical content as presented. It is not approval pending the addressing of comments, or contingent on future work being performed. It is not approval of the importance of the proposal or approval of the market importance of the technology or approval of the company or organization making the proposal. It is explicitly approval of the technical content as presented. Although comments may be appended, the approval is clearly not contingent on anything at all happening to those comments, since the language clearly says the approval of the DIS as presented. Nowhere in the Directives does it suggest that NB’s may substitute their own criteria or procedures for evaluating a Fast Track DIS. The criterion is clearly stated, Approval of the technical content of the DIS as presented. In fact JTC1 Directives, Section 1.2 says “These Directives shall be complied with in all respects and no deviations can be made without the consent of the Secretaries-General.” So any NB that substitutes their own evaluation criteria for the language of section 9.8 is violating the Directives.

Now, for a Disapproval vote, the Directives say that disapproval is made for specifically stated technical reasons, accompanied with proposals that would make the DIS acceptable, and that if these changes are made, the NB has the opportunity then to change their vote to Approval. Note that it is giving a clear ordering. The NB first votes Disapproval, listing the reasons why along with their proposals to fix the problem, then if the changes are accepted, the NB has the opportunity to change their vote to Approval.

This mechanism is called out again a few lines later when it speaks of “conditional approval” and that it should be registered as a Disapproval vote.

Note that under JTC1 Directives, neither Microsoft nor Ecma has the power to accept an NB proposal. They do not own DIS 29500. They are not NB’s. Ecma’s ownership of the proposal ended when the 5-month ballot began. The only entity that can formally address NB technical comments is the assembled NB’s at the Ballot Resolution Meeting. Certainly Ecma can offer an opinion, but it is no longer theirs to accept or deny changes at this point. If Microsoft is promising resolutions to NB’s, then it is promising something which is not theirs to give. (Before you buy a used car from someone, it may be wise to first verify that they actually own it.)

In summary, when Microsoft says that an NB should vote Approval, with comments, and that they promise that all comments will be addressed, this is defective analysis for several reasons:

1. The Directives clearly state that Approval indicates that the NB accepts the technical content as presented. Certainly, if the NB has only small editorial comments but otherwise accepts the technical content, then an Approval vote is entirely appropriate. But if technical content is not acceptable as presented, then they must vote Disapproval or else they ignore the plainly stated language of the Directives.
2. Voting Approval, with comments with a private promise from Microsoft that your comments will be addressed at the BRM anyways — this contradicts the clear statement that “conditional approval should be submitted as a disapproval vote.”
3. Neither Microsoft nor Ecma is competent to provide any assurance as to what the BRM will or will not do. They do not run the BRM and they do not control what comments are addressed. The BRM is an NB meeting.

The tragedy of this is that for so many NB’s, with talented technical committees, the NB process for OOXML has failed to be a technical evaluation, but instead has quickly become a political game, where committees are stuffed, where governments are pressured to change their votes, where billionaires call in favors, where competitors are blocked from participation, where voting rules are ignored or replaced at whim, etc. All we can do is stand by and watch as Microsoft takes over JTC1. The cost to Microsoft will be great, but so much greater is the cost to JTC1. What will it mean for JTC1’s future to be known as a body that does not follow its own rules, does not evaluate proposals on technical merits, but has procedures so weak and poorly written that it allows itself to be taken over and dominated by a single company? Quis custodiet ipsos custodes?

“Is there any point to which you would wish to draw my attention?”

“To the curious incident of the dog in the night-time.”

“The dog did nothing in the night-time.”

“That was the curious incident,” remarked Sherlock Holmes.

— Silver Blaze by Sir Arthur Conan Doyle

A curious blog post from Brian Jones, looking at spreadsheet interoperability between Gnumeric and Apple’s new Numbers spreadsheet, using OOXML. Take a read there and come back and we can compare notes.

Did anything strike you as odd? What raised my eyebrows was the utterly trivial nature of the spreadsheet document that was tested. Typically an interoperability demonstration will be a little flashy, showing as much functionality as possible. But this one has no text attributes, only a single, default numeric style, no charts, no use of spreadsheet functions, nothing. Why bother? There is nothing in this spreadsheet that one could not easily have created in VisiCalc 25 years ago. So why is this simplistic document being used to demonstrate interoperability with OOXML? This seems very odd. Interoperability with a more substantial document would have been far more persuasive. So why didn’t they do that? Hmmm….

So I decided that I would give it a try, on my Windows XP laptop running Office 2007, OpenOffice 2.1 Novell Edition (giving it a test drive this week) and Gnumeric 1.7.10. Let’s see what really works.

First, let’s start with a more substantial spreadsheet document. I created the following in Office 2007, illustrating a variety of everyday features:

1. numeric format
2. simple text styles
3. cell background fills
4. cell alignment
5. spreadsheet functions
6. charts
7. row widths
8. worksheet password protection
9. cell validation
10. hyperlinks
11. word art and shapes
12. OLE embedding

Nothing fancy here, nothing that hasn’t been around in Office since Office 2000 or earlier. You could have done most of this in 1-2-3 version 2.3 a decade before that. I saved the document both in XLSX (OOXML) and XLS (legacy binary) formats. Both appeared identical as shown here:

Next, I tried opening the XLS file in OpenOffice. We see that it handled the file well:

The colors in the chart are clearly different, but I didn’t set any particular colors in the original, opting for the default. So this may just be an indication that the charts in OpenOffice have different default colors. As you can see from the above picture, everything else looks fine. I did verify that worksheet protections, cell validation and the hyperlink worked correctly. However, although the OLE embedding seems to be there, I was not able to activate it.

Next, I fired up Gnumeric to see how it would fare. I first tried the same XLS file which loaded and displayed like this:

What do we notice?

1. Cell A7 did not format properly. It should be in long date format, but it is displaying in time format.
2. Chart colors differ, but this is probably just a difference in defaults.
3. Chart text is clipped in several instances.
4. The OLE embedding failed to come through with correct metafile for display.
5. Workbook protections and cell validation worked as expected.
6. Hyperlink worked correctly.
7. Arrow shape was dropped.

So, that is Gnumeric with a basic binary Excel worksheet. Since Gnumeric obviously supports this level of functionality, what would we expect to see when it loads the same document, but in OOXML format? See the image below for what I saw:

Hmm…OK… I think we hear the dog barking now. The OOXML import into Gnumeric is not really usable yet. In addition to the problems indicated above with the XLS import, we can add the following:

1. None of the charts converted
2. Worksheet password protection was lost
3. The hyperlink is broken
4. The OLE embedding is missing
5. Cell validation is broken
6. The word art is missing

Now a few points I wish to make concerning this. First I don’t want to have this taken as an attack on Gnumeric or those who work on it. Gnumeric is a fine spreadsheet with an impressive range of analytic features. The developers have done an outstanding job on it.

However, Microsoft points to Gnumeric as proof that OOXML can be implemented by other vendors. I suggest the jury is still out on this. 1-2-3 release 1.0a (1984) supported more functionality than Gnumeric does via OOXML. Note that even though there is no complete public documentation on the legacy binary formats, Gnumeric does a far better job at supporting them than it does with “standard” Ecma-376 OOXML and its 6,000 pages of documentation.

Now certainly, with much time and much effort, I’m sure Gnumeric will reach the point where it can read an OOXML document as well as it can read an XLS document. It might take another two or three years, but that day will come. But what benefit is that? All that effort will be spent writing code and testing to achieve practical results that Gnumeric already has achieved with the binary formats. This effort comes at the expense of other development activities such as adding features or fixing bugs. I hardly think that Jody wakes up in the morning joyed by the prospect of adding OOXML support to an application that is already compatible with billions of legacy Microsoft documents.

Similarly I have to scratch my head at OpenOffice and their announcement that they are adding OOXML support. As shown earlier, their support of the binary formats is already excellent. I guess that is why Microsoft is so eager to change their default formats. When a product like OpenOffice is able to effectively exchange documents with Office, it is too much of a threat to their Office revenue. So OpenOffice must now spend several person-years recreating this same level of interoperability, and the net result is that they will end up with the same capability they had before, but at the expense of forgoing work on new features. I wonder what Microsoft will do when OpenOffice catches up again in a few years? Hmmm…

(It would be interesting to examine out some of the other products that are said to support OOXML. Of the ones that support OOXML as well as the binary formats, how many of them also have OOXML support that is far worse than their binary format support? Is any editor vendor able to stand up and say that OOXML is a blessing to them because it allows higher fidelity interchange with Office than they were able to achieve with the binary formats?)

Everyone is in the same boat with this: KOffice, Corel, Google, IBM, anyone who has applications that work with Microsoft documents. We’re all faced with the prospect of significant expenses to rewrite our file format support with no net benefit to our customers. This is the toll we all must pay to Microsoft just for the ability to fight for the scraps their monopoly may leave behind. If Microsoft jerks their format around, we all must run and chase after it, reallocating resources away from feature work, becoming in the process less competitive in the marketplace, while Microsoft forges ahead with new features. They can easily repeat this game every few years, just to keep competitors busy. This is what a death spiral looks like.

Giving absolute control of a standard document format to a monopolist that is notorious for abusing their control of file formats in the past is insanity. It doesn’t take a Sherlock Holmes to figure that out.