Archives for 2007

Those who forget Santayana…

2007/12/20 By Rob 14 Comments

It must have passed beneath my radar it when it first was filed in 2004, but it caught my eye recently when Andy Updegrove mentioned it in Chapter 3 of his book-in-progress, The War of the Words. I’m talking about Novell’s November 2004 antitrust complaint against Microsoft, filed shortly after settling an different, OS-related, complaint with Microsoft for $536 million. You can view the second complaint, which I’ll call the “WordPerfect” complaint, here [PDF] on GrokLaw.

What is interesting to me, and why this “old news” is worth talking about, is the analysis Novell made in their complaint of Microsoft’s treatment of document format standards. The concerns of 2004 (or 1995 even) are very similar to the concerns of 2007. Let’s go through Novell’s argument and see where it leads us.

91. As Microsoft knew, a truly standard file format that was open to all ISVs would have enhanced competition in the market for word processing applications, because such a standard allows the exchange of text files between different word processing applications used by different customers. A user wishing to exchange a text file with a second user running a different word processing application could simply convert his file to the standard format, and the second user could convert the file from the standard format into his own word processor’s format. This, a law firm, for instance, could continue to use WordPerfect (which was the favorite word processor of the legal profession), so long as it could convert and edit client documents created in Microsoft Word, if that is what clients happened to use…

This is a good statement of the benefits of an open document standard. Note that Novell is not arguing that the benefit of a standard is to get information in or out of a single vendor’s product, like Microsoft Office. The benefit is that a standard provides for interchange between any pair of word processors.

…Microsoft knew that if it controlled the convertibility of documents through its control of the RTF standard, then Microsoft would be able to exclude competing word processing applications from the market and force customers to adopt Microsoft Word, as it soon did.

Note also that Novell is not complaining here about Microsoft’s control of the binary DOC format (and its many variations). Instead, what Novell complains about is Microsoft’s control over the document exchange format RTF, or Rich Text Format, used in those days to exchange data between word processors. He who controls RTF, controls document exchange, controls vendor lock-in and has the sole means of improving the fidelity of document exchanges.

In fact, Microsoft claimed that RTF addressed this very concern — document exchange in a cross-platform, cross-application fashion, as stated in the introduction to version 1.0 of their self-styled “standard”:

The RTF standard provides a format for text and graphics interchange that can be used with different output devices, operating environments, and operating systems. RTF uses the ANSI, PC-8, Macintosh, or IBM PC character set to control the representation and formatting of a document, both on the screen and in print. With the RTF standard, you can transfer documents created under different operating systems and with different software applications among those operating systems and applications

It should have been obvious at the time that vesting exclusive control of an interoperability interface in a single company was a bad idea. But I guess the world didn’t realize what dealing with Microsoft meant. But we know better now. So why are we making the same mistakes in 2007?

Those who control the exchange format, can control interoperability and turn it on or off like a water faucet to meet their business objectives. I don’t know how many people noticed the language in Microsoft’s press release announcing their sponsored interoperability track at XML 2007 a few weeks ago:

In its approach, Microsoft strives to bring technologies to market in a way that balances competitive innovation with the real interoperability needs of customers and partners.

Let that sink in for a minute. Microsoft is saying that they need to balance interoperability and profit. (Their profit, not yours) They can’t maximize for both simultaneously. They need to trade one off for the other.

Continuing with Novell’s 2004 complaint:

92. The specifications for RTF were readily available to Microsoft’s applications developers, because RTF was the format they themselves developed for Microsoft’s office productivity applications. Microsoft withheld the RTF specifications from Novell, however, forcing Novell to engage in a perpetual, costly effort to comply with a critical “industry standard” that was, in reality, nothing more than the preference of its chief competitor, Word. Indeed, whenever Word changed its own file format, Microsoft unilaterally and identically changed the RTF standard for Windows, forcing Novell and other ISVs constantly to redevelop their applications. In this manner, Microsoft gave Word a permanent, insurmountable lead in time-to-market and made document conversions difficult for users otherwise interested in running non-Microsoft applications. Many WordPerfect users were thus forced to switch to Microsoft Word, which predictably monopolized the word processing market….

So, the RTF standard was just a dump of Word’s features, done when and how Microsoft felt like doing it. As one wag quipped, “RTF is defined as whatever Word saves when you ask it to save as RTF.”

This should sound familiar. OOXML is nothing more than the preferences of Microsoft Office. Whenever Word changes, OOXML will change. And if you are a user or competitor of Word, you will be the last one to hear about these changes. ISO does not own OOXML. Ecma does not own OOXML. OOXML, in practice, is controlled and determined solely by the Office product teams at Microsoft. No one else matters.

Consider that Microsoft has recently proposed over 1,700 changes to the OOXML specification, including fixes that presumably will be made into a future Office 2007 fixpack. Microsoft knows what these fixes will be. The Office developer teams know what these fixes will be. But if you are a competitor of Microsoft’s in this space, do you know what these changes are? No. Microsoft has decided to keep them a secret, claiming that the ISO process allows them to withhold interoperability information from competitors in what they maintain is an “open standard”.

Further, the coding of Office 14 a.k.a. Office 2009 is well underway. Beta releases are expected in early 2008. But are file format changes needed to accommodate the new features being discussed in Ecma? No. Are they being discussed in ISO? No. Are they being discussed anywhere publicly? No.

Is this how an open standard is developed?

My prediction is that the first time anyone hears about what is in the next version of OOXML will be when Office 14 Beta 1 is announced. Other vendors will not hear a word about the format changes until after the Beta 1 is already released. Not even Ecma will hear about the changes until after then.

DIS 29500 is already obsolete, has already been embraced and extended. You just don’t know about it yet. You weren’t meant to know. In fact, pretend you don’t know. Give Microsoft a big head start. They need it.

Further from the Novell complaint:

93. …As in the case of of RTF, Microsoft forced Novell to delay its time-to-market while redeveloping its applications to an inferior standard. Because these standards were lifted directly from Microsoft’s own applications, those applications were always “compatible” with the standards.

And that is the key, isn’t it? By owning the “standard” and developing it in secret, without participation from other vendors, in an Ecma rubber-stamp process, Microsoft rigs the system so they can author an ISO standard with which they are effortlessly compatible, while at the same time ensuring that their products maintain an insurmountable head start in implementing these same standards. There is no balance of interests in OOXML. It is entirely dictated by Microsoft, and voted on, in many cases, by their handpicked committees in Ecma and ISO.

So much for Novell’s complaint from 2004. I’m told that this is still case is suspended as of November, 2007, as the two parties pursue mediation. A status report on that mediation is due to Judge Motz by January 11th, 2008. Maybe we’re hear more then.

Looking at this long history of standards abuse by Microsoft, in the file format arena and elsewhere, I’m drawn to take a broader view of this controversy. It is not really a battle between ODF and OOXML. It isn’t even really a battle between OOXML and ISO. It is, in the end, a battle between having document standards and not having them. Microsoft is trying to dumb down the concept of standards and interoperability to a point where these concepts are meaningless and ineffective. This is not because they want to support standards more easily in their products. No, it is because they do not want standards at all.

Remember, standards bring interoperability, the ability to try out new tools and techniques, the ability to migrate, the ability to chose among alternatives, the ability even to run non-Microsoft products. If standards are meaningless and ineffective, then the incumbent’ vendor lock-in will win every time. At that point, isn’t it convenient for them to have a monopoly in operating systems and productivity applications? This, in my opinion, is the essence of Novell’s 2004 complaint, Opera’s present complaint, and the ongoing file format debate. Microsoft’s monopoly power and the resulting network effects have lead to a relationship with standards where they win by winning, by drawing, or even by cheating so much that they discredit the system.

A Lick Back in Time

2007/12/20 By Rob 11 Comments

Updating a post from 2006, I’d like to take a look back at the commemorative stamp issues of 1957. What do we choose to remember, and how do we remember it? What was considered significant back in 1957, but is now forgotten? What icons have remained with us, and which have faded from view?

First up is Scott #1086, the Alexander Hamilton bicentennial issue, in rose red from a design by the Bureau of Engraving and Printing’s William K. Schrage.

Hamilton, was born in the West Indies, on the island of Nevis, thus Constitutionally unable to be President, though well-qualified otherwise. He was a member of General Washington’s staff, later our first Treasury Secretary, co-author of the Federalist Papers. He was shot and killed by Vice President Aaron Burr in a duel.

The design also features the old Federal Hall in New York City, which was the original national capitol building, before the move to Washington, DC.

Scott # 1087, designed by Charles R. Chickering, commemorates the 20th anniversary of the National Foundation for Infantile Paralysis, later called the March of Dimes. The deisgn features a woman/nurse, a boy and girl (the youth of the nation most benefited from the historic medical achievement) and the caduceus emblem on the shield, the symbol of the medical profession.

Those of us born after polio was eradicated may not appreciate the massive effort, in education and vaccination, that occurred in the 1950’s to wipe out polio. One poll at the time indicated that more people knew about the polio vaccine than knew the name of the President. Almost 2 million participated in clinical trials of the Salk vaccination. By 1957 the number of polio cases were dramatically falling. It was time for a victory stamp. Note that it says “Honoring those who helped fight polio” not “Honoring those who are helping fight polio”. It was clear that the fight had been won.

Scott #1088 commemorates the 150th anniversary of the Coast and Geodetic Survey. This orgaization was broken up in 1970 and split into the National Geodetic Survey and the Office of Coast Survey, both parts of NOAA.

Note the light bursts streaming out from the horizon. That seemed to be a popular effect in 1957, featured in the March of Dimes stamp above, as well as in Oklahoma statehood and Old Glory issues below.

Scott #1089, design by Robert J. Schultz, honors the American Institute of Architects on their centennial. Originally formed as the “New York Society of Architects” with 13 members, the are still going strong with over 80,000 members. They have a nice web page with a retrospective, “Celebrating the Past, Designing the Future“.

The next story does not have a happy ending. 1857 was the founding of the Saucona Iron Company, later called Bethlehem Steel. They were the top ship builder of WW II, with over 1,000 ships, employing over 300,000. They made the steel that built the Golden Gate Bridge and much of NYC’s skyline. They reached their peak in the 1950’s, one of the largest corporations in America and the highest-paid executive executive in the the country.

This stamp, Scott #1090, was issued at the height of Bethlehem Steel and the US steel industry. The bold, proud design was by Antonio Petruccelli, well-known for his Fortune Magazine covers and other illustrations.

So where is the celebration of the 150th anniversary of the American steel industry?

There isn’t one. The Bethlehem plant closed in 1995 and the company was bankrupt by 2001.

“Look on my works, ye mighty, and despair!”

Next up is the 1957 International Naval Review commemorative, with design by Richard A.Genders. This was held in conjunction with the Jamestown 350th anniversary. In 2007 we certainly saw a lot of celebrations of observances of the 400th anniversary there.

Now comes a puzzle. The next stamp marked the 50th anniversary of Oklahoma statehood, with an issue featuring the title “Arrows to Atoms”. The arrows part is clear enough — Oklahoma, formally called the Indian Territory, was home to many tribes, including the Cerokee, Chichasaw, Choctaw, Creek and Seminole.

But what was the connection to the atom? I have absolutely nothing on this. Anyone have any ideas? Is this just 1950’s atom-age kitsch? Or was there some significant Oklahoma connection to early atomic testing? (I didn’t see any talk of atoms in Oklahoma’s 100th anniversary celebrations this year.)

Next up is the National Education Association, the NEA, the largest labor union in the country, with a staff of 550 and a budget of $300 million. Founded in 1857 when the New York Teachers Association put out a nation-wide call for public school teachers to unite, it celebrated its centennial in 1957. Still going strong at age 150. Unlike steel workers, we’ll always need teachers.

If we did this stamp over again today, I wonder if we would show a laptop rather than a globe?

Here is my favorite stamp of 1957, the 48-star “Old Glory” issue, designed by Victor S. McCloseky, Jr. This is the first issue in the US to use the Giori press, a form of intaglio which used precisely cut ink rollers which could apply two or three ink colors to the same press, which then could be applied to the sheet in a single pass, avoiding the types of misalignment problems which could occur in multi-pass color processes.

Maybe I’m getting old and curmudgeonly, but you can keep your modern “stickers” that they pass off for postage stamps today. The Incredible Hulk? Muppets? Darth Vadar? When did we run out of real heroes to put on our stamps that we need to resort to fictional (and highly commercial) characters?

1957 saw the 350th anniversary of the first British ship built in the new world, the Virginia of Sagadahoc, built in the short-lived Popham Colony, in Maine. This is a small stamp, but lists 4 designers: Ervine Metzel, Mrs. William Zorach, A.M. Main, Jr. and George F. Cary II. That is somewhat unusual. There is nothing elaborate about this design that would suggest a need for 4 designers. I wonder if this was the result of a design contest, with citizen submissions plus actualization by B.E.P. professionals like Metzel?

Another Ervine Metzel design, this on 200th anniversary of the birth Lafayette, the Frenchman who gave notable service in both the American and French Revolutions. His popularity in the U.S. was enormous in the 19th century and most U.S. states have towns or counties named Lafayette (or Fayette or Fayetteville), and many towns have a street named after him, often intersecting with, or near a Washington Street.

Another Giori press issue, raising awareness of wildlife conservation and the plight of the endangered Whooping Crane.

Finally, we have a commemoration of the Flushing Remonstrance. This, like the Diet of Worms or the the Synod of Hippo, sounds a lot funnier than it is. Not as well-known as it should be, the Flushing Remonstrance was a protest by English farmers under Dutch jurisdiction in New Netherlands (later New York) against the government sanctioned persecution of the Quakers. It is one of the earliest American statement on religious toleration:

The law of love, peace and liberty in the states extending to Jews, Turks, and Egyptians, as they are considered the sonnes of Adam, which is the glory of the outward state of Holland, soe love, peace and liberty, extending to all in Christ Jesus, condemns hatred, war and bondage. And because our Saviour saith it is impossible but that offenses will come, but woe unto him by whom they cometh, our desire is not to offend one of his little ones, in whatsoever form, name or title he appears in, whether Presbyterian, Independent, Baptist or Quaker, but shall be glad to see anything of God in any of them, desiring to doe unto all men as we desire all men should doe unto us, which is the true law both of Church and State; for our Savior saith this is the law and the prophets. Therefore, if any of these said persons come in love unto us, wee cannot in conscience lay violent hands upon them, but give them free egresse and regresse unto our Town, and houses, as God shall persuade our consciences.

The Right and Lawful Rood

2007/12/13 By Rob 6 Comments

So what do we have here?

Sixteen men, lined up. They seem to be having a good time. Some are older, some younger. A historian of fashion might be able to tell us their relative social status, and perhaps their trade, by looking at their clothing. In the background, three men are observing and comparing notes. To the right is a church, and to the left is the village.

So what are they doing?

Is it an early early depiction of the hokey-pokey (“You put your left foot in…”) ?

No.

Although the scene obviously has some social aspects, the primary activity depicted here is standards development, particularly the historically mandated procedure for determining the linear measurement known as the “rood”, related to the English “rod”, the German “rute” and the Danish “rode”.

This print, from a 16th century surveyor’s manual by Jacob Koebel, called Geometrei. Von künstlichem Feldmessen und absehen, explains the procedure:

Stand at the door of a church on a Sunday and bid 16 men to stop, tall ones and small ones, as they happen to pass out when the service is finished; then make them put their left feet one behind the other, and the length thus obtained shall be a right and lawful rood to measure and survey the land with, and the 16th part of it shall be the right and lawful foot.

From a technical point of view, you might wonder why they didn’t have a standard rule, a metal bar etched with two lines, something tangible which could be carried about and used to calibrate? But who would maintain the standard? And would you trust them? Physical objects may be counterfeited, replaced, shaved, distorted, even stolen. Those who are buying land would like a longer rood, and those selling land would like a shorter rood, so the motivation for fraud is clear.

But the average length of the feet of 16 random men — that is probably not going to change much in a given town, or even across a country. Compared to the logistics required to create, duplicate and distribute a standard rule, the described statistical approach is easier to administer and was accurate enough for the time.

But there is more to it than that. Why didn’t the surveyor just measure his own feet? Or those of his friends? And why require that it be done at church? Why not wherever the surveyor wants to do it?

There must have been something about the process itself, the lining up and being measured, publicly, neighbor beside neighbor, next to the church, that lent it legitimacy. These men are literally voting with their feet.

The transparency of the process is also notable. The rood was determined in public, at the time and place most likely to offer everyone in the town the opportunity to observe. It is hard to cheat with the public watching. Anyone there trying to wear clown shoes or going barefoot would be immediately detected.

Also, it is notable that participation was on an equal basis. No one was able to say, “I am a rich merchant, so I should be allowed to bring 5 pairs of my shoes and line them up in front of me”. And certainly no one could say, “I am the King, the standard is determined by my foot and my foot alone”. This is good, because the variation from King to King would tend to be much greater than the variation from different random samplings of 16 men.

Bait and Switch

2007/12/06 By Rob 25 Comments

Promises have been made. Assurances have been given. Commitments have been proffered. But far less has been delivered.

Let’s review the record.

We start with the Ecma whitepaper, “Office Open XML Overview” [pdf] which was included in their submission to ISO:

Standardizing the format specification and maintaining it over time ensure that multiple parties can safely rely on it, confident that further evolution will enjoy the checks and balances afforded by an open standards process.

OK. So we were told that if OOXML is standardized its future evolution will be in an open standards process,with checks and balances.

Brian Jones, from a mid 2006 blog post:

There has also been talk though of taking the formats to ISO once they have been approved by Ecma, which would mean that if ISO chooses to adopt the Open XML formats the stewardship of the formats would be theirs. We’ve had a number of governments indicate that they would like the formats to be given to ISO, and it’s likely that after the Ecma approval that will be the next step.

Again, saying that if approval by ISO is tantamount to transferring custody of the format to ISO.

Six months later, Brian wrote:

Some feedback that we got primarily from governments was that they wanted to see these formats not just fully documented, but that the stewardship and maintenance of that documentation should be handed over to an international standards body.
.
.
.
Obviously, a great way to guarantee the long term availability of OpenXML, and the confidence that it won’t change is for an organization like ISO to take ownership of the spec.

OK. Not exactly a signed-in-blood promise, but still a clear, leading indication that the feedback they received from customers was for stewardship and maintenance and even ownership of OOXML to be handed over to ISO.

As the OOXML (DIS 29500) ballot drew nearer to a close, these vague intimations became outright promises. We heard over and over again that we should approve OOXML because that was the only way to ensure that the format would remain open. The first version might be a mess, but if we approve it just this once, all future versions will be developed in openness and transparency.

For example, John Scholes writes of a Microsoft promise made at an National Computing Centre (NCC) file format debate held in London on July 4th:

Would the maintenance of the standard be carried out by Ecma (assuming OpenXML became an ISO/IEC standard) or would it be carried out by JTC1? No question, JTC1. But would the detail be delegated to Ecma? No, it would all be beyond MS’ control in JTC1. Well at this point there was apparently some sotto voce discussion between Stephen and Stijn, followed by a little backtracking, but it came across loud and clear in subsequent discussions in the margins that Stephen and Jerry believed this was for real. MS was handing over control of OpenXML to JTC1 (or trying to).

I participated in this debate as well, and I can confirm that it occurred exactly as John relates. I even asked a follow-up question to make sure that I hadn’t misunderstand what Microsoft was saying. They were adamant. ISO would control OOXML.

Jerry Fishenden, Microsoft’s lead spokesman in the UK wrote two week’s later:

There’s an easy question to consider here: would you prefer the Microsoft file formats to continue to be proprietary and under Microsoft’s exclusive control? Or would you prefer them to be under the control and maintenance of an independent, open standards organisation? I think for most users, customers and partners that’s a pretty easy question to answer: they’d prefer control and maintenance to be independent of Microsoft. And the good news is that the Open XML file formats are already precisely that: currently under the control of Ecma International (as Ecma-376) and, if the current voting process is positive, eventually under the control of ISO/IEC. Many major and significant UK organisations have already made clear that they support this move for Open XML to become an ISO/IEC standard.
.
.
.

The United States vote is one step in the direction to put Open XML under the control of the ISO/IEC standards body.

So Jerry is stating in no uncertain terms that approval of OOXML puts it under ISO control. This statement was repeated on an August 24th update on Microsoft’s “Open XML Community” web site.

(I’ve heard many second-hand reports of additional repetitions of this promise made at NB meetings around the world, in the run up to the Sept. 2nd ballot. If anyone participated in such a meeting and heard such assurances first hand, feel free to add the details as a comment.)

So much for the promises. What makes this story worthy of a blog post is that we now know that, even as these promises were be made to NB’s, at that same time Ecma was planning something that contradicted their public assurances. Ecma’s “Proposal for a Joint Maintenance Plan” [pdf] outlines quite a different vision for how OOXML will be maintained.

A summary of the proposed terms:

OOXML remains under Ecma (Microsoft) control under Ecma IPR policy.
Ecma TC45 will accept a liaison from JTC1/SC34 who can participate on maintenance activities and only maintenance activities.
Similarly, Ecma TC45 documents and email archives will be made available to the liaison (and through him a set of technical experts), but only the documents and emails related to maintenance.
No mention of voting rights for the liaison or the experts, so I must assume that normal Ecma rules apply — only Ecma members can vote, not liaisons.
Future revisions of OOXML advance immediately to “Stage 4” of the ISO process, essentially enshrining the idea that future versions will be given fast-track treatment

A critical point to note is that “maintenance” in ISO terms is not the same thing as what the average software engineer thinks of as “maintenance”. The work of producing new features or enhancements is not maintenance. The act of creating OOXML 1.1 or OOXML 2.0 is not maintenance. What is maintenance is the publication of errata documents for OOXML 1.0, a task that must be completed within 3 years.

So what Ecma is offering SC34 is nothing close to what was promised. Ecma is really seeking to transfer to SC34 the responsibility of spending the next 3 years fixing errors in OOXML 1.0, while future versions of OOXML (“technical revisions”) are controlled by Microsoft, in Ecma, in a process without transparency, and as should now be obvious to all, without sufficient quality controls.

This maintenance proposal is on the agenda for the JTC1/SC34 Plenary meeting, in Kyoto on December 8th. I think this one-sided proposal should be firmly opposed.

Consider JTC1 Directives [pdf], 13.13:

If the proposed standard is accepted and published, its maintenance will be handled by JTC 1 and/or a JTC 1 designated maintenance group in accordance with the JTC 1 rules.

JTC1’s practice in such matters is to delegate to the relevant subcommittee, so read “SC34” for “JTC 1” above. So it is within the procedures for SC34 to make this decision. In fact, ownership by the SC is the norm. The clause “and/or a JTC 1 designated maintenance group” is a new addition to the Directives which was added right before the OOXML procedure in ISO began. (Curiously this was the same revision of the Directives that added the escape clause to the Contradiction phase that allowed OOXML to continue despite the numerous unresolved contradictions with existing ISO standards.)

So what does a counter-proposal look like?

First, I think we should defer decision on this until the next SC34 Plenary, presumably in Spring 2008. It is not clear whether or not OOXML will ultimately be approved as an ISO standard, and even if it does, maintenance does not need to be completed for 3 years. So I don’t think we should rush into anything.

The UK has made a proposal to create a new working group (WG) in SC34 dedicated to “Office Information Languages”:

SC34/WG4 would be responsible for languages and resources for the description and processing of digital office documents. The set of such documents includes (but is not limited to) documents describing memoranda, letters, invoices, charts, spreadsheets, presentations, forms and reports.

WG4 would be expected to work on the maintenance of, for example:

ISO/IEC 26300:2006

ISO/IEC 29500 (should it exist)

and be responsible for reviewing any future office document formats.

I think this deserves serious consideration. This may be the type of neutral venue — not Ecma and not OASIS — that would be conducive to getting the technical experts together to refactor OOXML and harmonize it with ODF. Even in the likelihood that OOXML ultimately fails in its bid as an ISO standard, the draft could still be referred to a new WG4 for further work. This would also be a way for Microsoft to fulfill their promise to transfer stewardship, control and ownership of OOXML over to ISO, a promise made they made publicly and repeatedly.

662 resolutions, but only if you can find them

2007/12/02 By Rob

Microsoft risks a repetitive stress injury from the recent frenzy of patting themselves on the back for responding to some of the ballot comments submitted in the failed OOXML ISO ballot of Sept 2nd.

They claim to be transparent and acting so that NB’s can easily review their progress in addressing their comments.

Well, let’s take a closer look.

First, Microsoft has managed to get JTC1 to clamp down on information. What was a transparent process is now mired in multiple levels of security leading to delay, denial of information to some NB participants and total opaqueness to the public.

Let’s review how things worked with ODF.

OASIS ODF TC mailing list archives are public for anyone to read
OASIS ODF TC public comment list archives are public for anyone to read
OASIS ODC meeting minutes, for every one of our weekly teleconferences going back to 2002, are all public for anyone to read.
The results of ODF’s ballot in ISO are public, including all of the NB comments
The comments on ODF from SC34 members are also public
The ISO Disposition of Comments report for ODF is also public for anyone to read

Short of allowing the public to read my mind, there is not much more we can do in OASIS to make the process more transparent. (And if you read this blog regularly you already have a good idea of what I’m thinking.)

But what about the OOXML process? Every single one of the above items is unavailable to the public, and in many cases cases is not available even to the JTC1 NB’s who are deciding OOXML’s fate.

In fact, OOXML is moving in reverse. Documents that were once public, such as the Sept 2nd. ballot results and NB comments, have been taken down and replaced with password-protected versions (Look for the DIS 29500 documents here. They all used to be available for all to download.) How do you get access to the password? The password is made available to NB points of contact “on request”. But so far few NB’s have requested the password. You can see here which ones have requested the password and which have not. As of today, only 18 of 51 NB’s have requested the password. Only 35% of SC34 NB’s have access to the same information they had back in September. Indeed, we’re moving backwards.

In the particular cases of these “662 responses”, Ecma is hosting them on their web site, on a different password protected page. (Yes, the comments and the resolutions to the comments are on two different web sites with two different passwords.) I’m hearing as well that few NB’s actually have the password, and some who do are not passing it on to their own committee members. I’ve heard from a few NB members who explicitly requested access to these documents but were denied. Others are simply unaware that these comment resolutions are available. What was once an open process is now closing up.

(12/04/2007 Update: Brian Jones claims that these 662 resolutions are protected by JTC1 rules. But JTC1 rules apply to documents submitted into the JTC1 process, hosted by JTC1 , assigned JTC1 “N” numbers, and archived by JTC1, as required by JTC1 process. But these 662 resolutions are not called for by the JTC1 process, are not hosted by JTC1, are not assigned JTC1 “N” numbers and are not archived by JTC1. They are Ecma documents, hosted by Ecma, assigned ID’s by Ecma, and controlled by Ecma passwords. These documents were never submitted to JTC1. Ecma is in total control over whether or not the public has access to them.

Brian highlights some rules that apply to the Disposition of Comments report, but that is not what we have before us. We won’t have the Disposition of Comments report until after the Ballot Resolution Meeting. At that point, it will be an official JTC1 document, assigned an “N” number, hosted by JTC1 and accessible via their password .

Note also that Microsoft continues to dodge how closed the Ecma TC45 process has been and remains. Why not open up the TC45 mailing list archives, Brian? Are the ISO meanies stopping you? I know that Ecma is not forcing you. Their policy is to let each TC decide for themselves. I’m sure if Microsoft took a leadership position in favor of openness that you could convince the other members of TC45 to increase their transparency. What do you say?)

(12/06/2007 Update: The former Ecma Secretary General weighs in on the topic in a blog post, confirming that the responses are not controlled by ISO access rules, though the original NB comments are:

Consequently, Ecma is not constrained in posting its interim responses on a publicly available page as long as they are not tied to specific NB comments. In other words, Ecma would have to do some work to separate the proposed responses from the specific NB comments, but then Ecma may make its work publicly visible. If there is so much interest outside the NB circuit, then Ecma will surely do something here.
.
.
.
Indeed, seen from Ecma there is nothing that forbids Ecma to distribute its proposals. But it should also be clear, in the light of the longstanding relationship, that it is not a MUST for Ecma to do this. Good habits and rules have a value, like in any great game, such as football. And also there the rules and habits don’t change overnight because somebody has another, maybe even brilliant idea

But suppose you get through your local NB politics and actually lay your hands on the password to the Ecma web site, what do you get then? You then have the privilege of navigating 50 or so different pages, scrolling through them and click on 662 links to download 662 separate PDF files, all from a painfully slow server. Ughh… It hardly seems worth it. It is almost like someone wants to discourage NB’s from actually reading this stuff.

Aiming to lessen the pain a little, I downloaded all 662 comments, and made a singe PDF file that contains all of the comment responses. I also included the original NB comments, and cross-linked everything, so I can navigate from comment to response, and slice and dice it by similar comments, or by NB. It is full text indexed, so I can search for things like “VML” and see all comments or responses relevant to that topic. Since it is liberated from the Ecma website, I can even use it off-line.

Doesn’t my method sound easier to use than downloading 662 PDF files? If you agree, then I’ll make you an offer. If you are a JTC1 or SC34 NB member, and would like access to this consolidated document, let me know via email. (You can find my email address here.) Note that my compilation is not a formal JTC1 document, and that this is not an offer from the US NB. This a personal offer from me to other individuals who are also JTC1 or SC34 NB members. (Of course, if Ecma wants a copy of this as well to make available for all NB’s to download, then that is even better. They know where to find me.)

So, now that I’ve read through these 662 responses, let me fill you in what we have here. First, I’d like to define some terms, so we’re all on the same page and understand the status of these 662 proposals.

At the BRM, baring any breakdown from lack of consensus, there will be issued an official “Resolution of Comments” document. This is the set of textual changes that JTC1 NB’s authorizes the Project Editor (Microsoft’s contractor in Ecma) to make to the DIS 29500 specification. Only the BRM can authorize these changes.

By January 14th, JTC1 NB’s will receive from the Project Editor a “Proposed Resolution of Comments” document. This will be Ecma’s proposals for how they would like to see the Sept. 2nd ballot comments resolved. The BRM is not limited to considering Ecma’s proposals. Their own NB comments from Sept 2nd may also be in play, since those often came with their own proposed resolutions which differ from the ones that Ecma will propose.

So what do we have now, these recent drop of 662 documents from Ecma? I call these by the verbose name: “Ecma’s Draft Proposed Resolution of Comments”. The are not the final Resolution of Comments, and they are not even the final Proposed Resolution of Comments. They are a draft of proposed resolutions to 662 of the 3,522 comments submitted by JTC1 on Sept 2nd.

So the time line is:

From now until late January we receive updates from Ecma in the form of Draft Proposed Resolution of Comments. If they continue to be posted in a user-unfriendly form, I will continue to produce updates to my consolidated report.
By January 14th, Ecma submits their final Proposed Resolution of Comments
At the adjournment of the BRM we have the approved Resolution of Comments
The Project Editor then has 30 days to apply the Resolution of Comments to produce the new text of DIS 29500
It is the above revised text that NB’s will consider whether to approve or not. Note that since the NB only has 30 days to reconsider their Sept. 2nd vote, and the revised text is not due until 30 days after the BRM, it is likely that NB’s will need to use their imagination and decide based on the approved Resolution of Comments document (perhaps 4,000+ pages in length), not having seen the actual revised text of the DIS.

So, what can we sat about the recently released draft proposed resolution of comments document?

This initial set of responses are almost entirely minor, dealing with corrections to examples, spelling errors, punctuation errors, cleanup of broken links, fixing illegible formulas, adding missing units on quantities, etc. There are also many, many duplicates in this area. In particular, the issue regarding spreadsheet functions missing units on some functions (not specifying radians or degrees) was picked up by 12 NB’s. Since there are multiple instances of that defect in the OOXML specification, each one repeated by several NB’s, this single observation results in 48 proposed resolutions. Ecma appears to have concentrated on comments like this, easy to fix and duplicated, in this batch. So although there are 662 resolutions on paper, this maps to perhaps only 80 or so unique issues.

The breakdown of proposed resolutions by NB is in the table below. These numbers are a bit tricky to interpret with the duplicate comments, since one NB’s comments might have been addressed in passing while fixing another NB’s issues. So I doubt Microsoft is spending a lot of time on Columbia, since they voted yes. But there may be a significant duplication between Columbia’s comments and another NB which Microsoft is trying to please. But by looking at unique comments, those submitted by only one NB, we can get a good sense of which NB’s Ecma is trying to please most. And no, I’m not going to tell you which ones they are.

Member	Comments Submitted	Ecma Responses	% Responded to
UK	635	218	34%
Ecma	76	23	30%
Colombia	237	71	30%
Philippines	7	2	29%
USA	288	69	24%
Chile	217	44	20%
Malta	5	1	20%
Japan	82	16	20%
Canada	79	15	19%
Czech Republic	75	13	17%
Uruguay	18	3	17%
Ireland	12	2	17%
France	592	97	16%
Australia	30	4	13%
Germany	162	20	12%
Portugal	118	14	12%
Brazil	64	7	11%
Greece	113	11	10%
Denmark	168	15	9%
Kenya	81	7	9%
Ghana	12	1	8%
India	82	5	6%
Israel	33	1	3%
Venezuela	73	2	3%
Iran	58	1	2%
Turkey	1	0	0%
Jordan	1	0	0%
Ecuador	1	0	0%
Thailand	1	0	0%
Spain	1	0	0%
Belgium	1	0	0%
Austria	1	0	0%
Argentina	1	0	0%
China	1	0	0%
Singapore	2	0	0%
Italy	2	0	0%
Tunisia	3	0	0%
Bulgaria	3	0	0%
Poland	4	0	0%
Mexico	7	0	0%
Peru	10	0	0%
Norway	12	0	0%
Finland	15	0	0%
South Africa	17	0	0%
Switzerland	19	0	0%
Malaysia	23	0	0%
Korea, Republic of	25	0	0%
New Zealand	54	0	0%

To be fair, not every resolution in this batch was editorial. There was some technical detail added. For example, the following points were clarified:

The SpreadsheetML AND/OR functions do not short circuit, so all parameters must be evaluated.
The CHAR() function converts an integer into a character. But no character set was defined in the DIS to govern this conversion. Microsoft clarrified tis saying that the function uses the “Macintosh character set”on the Mac and ANSI on all other platforms.
Spreadsheet functions that do searches or string compares (EXACT, FIND, FINDB, SEARCH, SEARCHB. etc.) do so with lexical character comparisons, not collation-based operations.
Part names in an OPC package can be IRI’s, not just URI’s. So this allows Unicode characters, with some restrictions in items names

However, the 662 comments carefully tip-toed around the controversial issue. I guess we’ll read proposals for those in a future update. So NB members, take the opportunity now to get access to this portal. Ask your NB head for access if you haven’t already been given the password. And if you want a copy of my consolidated PDF file, let me know.