Promises have been made. Assurances have been given. Commitments have been proffered. But far less has been delivered.
Let’s review the record.
We start with the Ecma whitepaper, “Office Open XML Overview” [pdf] which was included in their submission to ISO:
Standardizing the format specification and maintaining it over time ensure that multiple parties can safely rely on it, confident that further evolution will enjoy the checks and balances afforded by an open standards process.
OK. So we were told that if OOXML is standardized its future evolution will be in an open standards process,with checks and balances.
Brian Jones, from a mid 2006 blog post:
There has also been talk though of taking the formats to ISO once they have been approved by Ecma, which would mean that if ISO chooses to adopt the Open XML formats the stewardship of the formats would be theirs. We’ve had a number of governments indicate that they would like the formats to be given to ISO, and it’s likely that after the Ecma approval that will be the next step.
Again, saying that if approval by ISO is tantamount to transferring custody of the format to ISO.
Six months later, Brian wrote:
Some feedback that we got primarily from governments was that they wanted to see these formats not just fully documented, but that the stewardship and maintenance of that documentation should be handed over to an international standards body.
.
.
.
Obviously, a great way to guarantee the long term availability of OpenXML, and the confidence that it won’t change is for an organization like ISO to take ownership of the spec.
OK. Not exactly a signed-in-blood promise, but still a clear, leading indication that the feedback they received from customers was for stewardship and maintenance and even ownership of OOXML to be handed over to ISO.
As the OOXML (DIS 29500) ballot drew nearer to a close, these vague intimations became outright promises. We heard over and over again that we should approve OOXML because that was the only way to ensure that the format would remain open. The first version might be a mess, but if we approve it just this once, all future versions will be developed in openness and transparency.
For example, John Scholes writes of a Microsoft promise made at an National Computing Centre (NCC) file format debate held in London on July 4th:
Would the maintenance of the standard be carried out by Ecma (assuming OpenXML became an ISO/IEC standard) or would it be carried out by JTC1? No question, JTC1. But would the detail be delegated to Ecma? No, it would all be beyond MS’ control in JTC1. Well at this point there was apparently some sotto voce discussion between Stephen and Stijn, followed by a little backtracking, but it came across loud and clear in subsequent discussions in the margins that Stephen and Jerry believed this was for real. MS was handing over control of OpenXML to JTC1 (or trying to).
I participated in this debate as well, and I can confirm that it occurred exactly as John relates. I even asked a follow-up question to make sure that I hadn’t misunderstand what Microsoft was saying. They were adamant. ISO would control OOXML.
Jerry Fishenden, Microsoft’s lead spokesman in the UK wrote two week’s later:
There’s an easy question to consider here: would you prefer the Microsoft file formats to continue to be proprietary and under Microsoft’s exclusive control? Or would you prefer them to be under the control and maintenance of an independent, open standards organisation? I think for most users, customers and partners that’s a pretty easy question to answer: they’d prefer control and maintenance to be independent of Microsoft. And the good news is that the Open XML file formats are already precisely that: currently under the control of Ecma International (as Ecma-376) and, if the current voting process is positive, eventually under the control of ISO/IEC. Many major and significant UK organisations have already made clear that they support this move for Open XML to become an ISO/IEC standard.
.
.
.The United States vote is one step in the direction to put Open XML under the control of the ISO/IEC standards body.
So Jerry is stating in no uncertain terms that approval of OOXML puts it under ISO control. This statement was repeated on an August 24th update on Microsoft’s “Open XML Community” web site.
(I’ve heard many second-hand reports of additional repetitions of this promise made at NB meetings around the world, in the run up to the Sept. 2nd ballot. If anyone participated in such a meeting and heard such assurances first hand, feel free to add the details as a comment.)
So much for the promises. What makes this story worthy of a blog post is that we now know that, even as these promises were be made to NB’s, at that same time Ecma was planning something that contradicted their public assurances. Ecma’s “Proposal for a Joint Maintenance Plan” [pdf] outlines quite a different vision for how OOXML will be maintained.
A summary of the proposed terms:
- OOXML remains under Ecma (Microsoft) control under Ecma IPR policy.
- Ecma TC45 will accept a liaison from JTC1/SC34 who can participate on maintenance activities and only maintenance activities.
- Similarly, Ecma TC45 documents and email archives will be made available to the liaison (and through him a set of technical experts), but only the documents and emails related to maintenance.
- No mention of voting rights for the liaison or the experts, so I must assume that normal Ecma rules apply — only Ecma members can vote, not liaisons.
- Future revisions of OOXML advance immediately to “Stage 4” of the ISO process, essentially enshrining the idea that future versions will be given fast-track treatment
A critical point to note is that “maintenance” in ISO terms is not the same thing as what the average software engineer thinks of as “maintenance”. The work of producing new features or enhancements is not maintenance. The act of creating OOXML 1.1 or OOXML 2.0 is not maintenance. What is maintenance is the publication of errata documents for OOXML 1.0, a task that must be completed within 3 years.
So what Ecma is offering SC34 is nothing close to what was promised. Ecma is really seeking to transfer to SC34 the responsibility of spending the next 3 years fixing errors in OOXML 1.0, while future versions of OOXML (“technical revisions”) are controlled by Microsoft, in Ecma, in a process without transparency, and as should now be obvious to all, without sufficient quality controls.
This maintenance proposal is on the agenda for the JTC1/SC34 Plenary meeting, in Kyoto on December 8th. I think this one-sided proposal should be firmly opposed.
Consider JTC1 Directives [pdf], 13.13:
If the proposed standard is accepted and published, its maintenance will be handled by JTC 1 and/or a JTC 1 designated maintenance group in accordance with the JTC 1 rules.
JTC1’s practice in such matters is to delegate to the relevant subcommittee, so read “SC34” for “JTC 1” above. So it is within the procedures for SC34 to make this decision. In fact, ownership by the SC is the norm. The clause “and/or a JTC 1 designated maintenance group” is a new addition to the Directives which was added right before the OOXML procedure in ISO began. (Curiously this was the same revision of the Directives that added the escape clause to the Contradiction phase that allowed OOXML to continue despite the numerous unresolved contradictions with existing ISO standards.)
So what does a counter-proposal look like?
First, I think we should defer decision on this until the next SC34 Plenary, presumably in Spring 2008. It is not clear whether or not OOXML will ultimately be approved as an ISO standard, and even if it does, maintenance does not need to be completed for 3 years. So I don’t think we should rush into anything.
The UK has made a proposal to create a new working group (WG) in SC34 dedicated to “Office Information Languages”:
SC34/WG4 would be responsible for languages and resources for the description and processing of digital office documents. The set of such documents includes (but is not limited to) documents describing memoranda, letters, invoices, charts, spreadsheets, presentations, forms and reports.
WG4 would be expected to work on the maintenance of, for example:
- ISO/IEC 26300:2006
- ISO/IEC 29500 (should it exist)
and be responsible for reviewing any future office document formats.
I think this deserves serious consideration. This may be the type of neutral venue — not Ecma and not OASIS — that would be conducive to getting the technical experts together to refactor OOXML and harmonize it with ODF. Even in the likelihood that OOXML ultimately fails in its bid as an ISO standard, the draft could still be referred to a new WG4 for further work. This would also be a way for Microsoft to fulfill their promise to transfer stewardship, control and ownership of OOXML over to ISO, a promise made they made publicly and repeatedly.
I am getting tired of the endless griping from IBM, and you are the worst offender Rob.
You would be better off spending your time working to move the maintenance of ODF out of OASIS!
And for all you and Bob Sutor attack Microsoft you are still working with them on web services specifications.
Does IBM actually remember how to innovate? I can’t think of any substantive software related standard IBM has contributed in the last decade.
Dont get me wrong, IBM does set a new standard for double standards and two facedness.
Another nail in the coffin of ISO.
I have seen legal experts discussing an overhaul of ISO that will kick out all non-governmental organizations from the voting committees. There might be a chance that commercial parties will be relegated to observer status rather sooner than later.
The recent rule changes in ISO will probably not help to install convidence in ISO’s ability to handle conflicts of interest.
Else the EU and other non-US economic parties might simply reneige on their WTO promiss to give priority to ISO standards in procurement. (the USA never bothers to follow the WTO rules anyway)
Winter
Minor nit: The ISO/IEC rules have been changed to that the initial “maintenance” review will now be after three years rather than after five years-
Rob, I’m sure you have read this but for the benefit of your other readers there is a report by Martin Bryan summing up the last period as convenor of a Working group under SC34 (WG1). It is – to me at least – highly damming and uses unusually colourful language…
http://www.theopensourcerer.com/2007/12/06/commiserations-to-my-successor-ooxml-strikes-again/
Alan
@nb. thanks. I’ve updated the post with that correction.
@Alan, thanks for that link. Martin was also SC34’s appointed liaison to Ecma TC45.
@Winter, there is also a push by Microsoft and others to move ISO toward a “direct participation” model where corporations can participate in ISO directly, bypassing the national body mechanisms altogether. This would essentially turn ISO into a vendor consortium. So it will be interesting to see how this all turns out. To coin a folk aphorism, “In the land of the ugly queen all the mirrors have disappeared.”
@Rob, I bet he enjoyed that! (NOT)
@Mo, If you don’t like to hear criticisms about OOXML then you’ve come to the wrong place. I suggest you might find a move to Kazakhstan more to your liking.
For the record, IBM currently participates in over 400 standards organizations worldwide. We employ many of the most talented and experienced standards professionals in the industry. If you can’t think of any standards that IBM is contributing to, then that probably just shows the thankless nature of standards development. When standards work well, you don’t read about them in the press.
per @anonymous:
I have seen legal experts discussing an overhaul of ISO that will kick out all non-governmental organizations from the voting committees. There might be a chance that commercial parties will be relegated to observer status rather sooner than later.
This would comfort me more had I not witnessed the process leading up to the US NB vote. With the Secretary of Commerce being lobbied directly by the CEO of Microsoft and then pressuring the other Federal agencies to change their votes (contrary to their technical experts’ advice.)
It would be nice if we could have the process be less political, rather than more — but that may not be an option. Sometimes, I fear, Chairman Mao was right about the roots of power.
@mo: Is that you, Steve B.?
mo,
I don’t think Rob and Bob Sutor are particularly ‘attacking’ Microsoft. Well, only in the sense that a Coke salesman would ‘attack’ a Pepsi salesman. The corporations are competitors in this market segment, and are likely to remain so for ever.
It’s more that they are pointing out that IBM is relatively unlikely to come up with a competing independent implementation of DIS29500; a sort-of upgraded Lotus SmartSuite which could interoperate with Microsoft Office by exchanging the files that each product writes to disk in DIS29500 format. As far as I can tell, the investment required to do that is thought to be large and pointless.
So if the ‘powers that be’ are hoping that by ISO standardising on DIS29500 they will draw IBM in to build an independent product and compete with Microsoft on price, they’re probably going to be disappointed.
Whether any other business attempts to come in to compete would be up to them; but perhaps governments should look to see if there are any credible ones before casting their votes.
By all accounts, those who pay ‘full list price’ for Microsoft’s office solutions (typically small and medium business) will not see any price reductions consequent on ISO issuing a standard for DIS29500. It won’t achieve any additional competition.
IBM’s colours are firmly nailed to the mast with Lotus Notes; 120 million paying ‘seats’. And Lotus Symphony, no charge, which stands in the same relationship to Lotus Notes as a telephone handset stands to dial-tone. ISO26300 for both of those.
ATT gives away handsets to grow the market for dial-tone. Businesses distribute marketing novelties all the time, at a ‘retail price’ of $0; of course the ‘wholesale’ price is higher. IBM Lotus Symphony is an IBM marketing novelty. Get yours here http://symphony.lotus.com/ and give or sell them to your friends and customers too if you like.
Other segments are other businesses. IBM and Microsoft are locked together in the XBox360 business; IBM can’t sell XBox360 chips to anyone but Microsoft, and Microsoft can’t buy XBox360 chips from anyone but IBM. An uneasy truce, maybe, but what else would you have happen ?
Is it good for the world (producers and consumers) if we have common, interoperable web service specifications ? I think so. So the ‘battleships’ attempt to hammer it out.
Where can I find information on that push towards ISO adopting a “direct participation model”?
@nb, if you are interested in more information on “Direct Participation” check with your national JTC1 member body and ask about N8812, the ballot which would establish an ad hoc group to study this. That ballot closes December 24th.
Posted on Groklaw
=================
From a Risk Assessment point of view this is the stuff of nightmares.
Microsoft has by their actions broken the mechanism to ALL business interoperability, from the correct dimensions of an M8 bolt to formal quality control and risk management processes. All that damage, just so it could hang on to a office software monopoly.
This has gone well past the point of acceptance, and especially those who wilfully participated in this scam are in principle liable for the damage their efforts will cause to business worldwide, and that damage is already starting to show. At the moment it’s mainly technical, but tell me, as a random grab from existing standards ISO holds, would *you* want to be responsible for, for instance, holding up a standard for Respiratory tract humidifiers for medical use? Or a standard to reduce injury to children’s heads? I’m not sure if anyone really understands just how catastrophic the damage is that Microsoft has caused with its shenanigans but the ramifications are IMHO very worrying.
I have reported the matter to the office of Dr Neelie Kroes, to be considered in the light of the recent monopoly judgement against Microsoft. I must admit that I don’t quite know what CAN be done, but this act of wilful destruction must not go unnoticed. If you know any journalists, PLEASE mention it to them. Show them facts, not guesswork – there are enough. Get it public.
In my opinion, ISO needs to do some serious house cleaning. In the core organisation, rules must be strengthened and an arbitration panel must be installed to ensure an appropriate means is available to deal with any future abuse. Given the proven abuse of the mechanism, MSOOXML should be wholly removed from the ISO voting process until the effects of the vote rigging have been negated. The ficticuous P members that have joined but not voted must be removed on account of joining on false premises, their payment considered compensation for the harm they caused. Repeated non-voting must be set as a reason to lose membership status.
In the national bodies, several processes are ongoing already to establish how Microsoft managed to sway the committee leaderships. In countries like Switzerland formal complaints exist against the leadership, and in general there has been a global discovery that what was supposed to be a fully technical and logical process has emerged to be a human process that can be easily subverted by companies with ill intent. This too needs addressing but is much harder because the mechanism is not universal, it differs per country. I don’t think ISO can do much about that centrally.
I have been involved in cleaning up organisational messes before, but I must admit I’ve never seen anything on such grand a scale. This is not just unethical, it’s wanton destruction.
If Microsoft has ANY sense whatsoever it should start undoing the damage it has caused, starting Monday latest. If not I would hope everyone reading this will do their best to ensure that the full effects of what they have done are made clear to every press outfit on the planet.
This .really. has gone too far.
It is a scary proposition. I don’t think people understand how much Microsoft now owns JTC1 in a very real and tangible way. Absolutely owns.
Consider that it requires 2/3 approval of JTC1 P members to approve a standard. Microsoft, by various means, has managed to achieve very close to that number. They are only 5 short. If they achieve that 2/3 then they can ram through whatever standards they want.
But that scary part is that with even 1/3 of P-members, a number they clearly outright own, they can block anyone else’s standard. It probably hasn’t sunk into your realization yet,but Microsoft can essentially already erected toll bridge in ISO and demand payment or other concessions from anyone who wants to work with International Standards. If ISO rules get in the way, Microsoft can change them. If ISO administrators get in the way — no worry. With this number of NB’s Microsoft can control directives, staffing, paychecks, etc.
They’ve raised an army. You don’t think they will use it?
In The Netherlands (NEN) it was always said ooxml would be maintained in Ecma. Then again, everyone knew NL would abstain due to its anal rules ;)
Hello,
The last thing I would like to see happening is putting the whole ISO organization to its grave because of this OOXML mess…
The fact that Microsoft already succeded to interfere in the ISO core process and made “corrections” to its written regulations is already *very serious*, guys !
Please, anyone of us that has some power in this matter: 1. oppose to the idea of ISO becoming a vendor consortium (shortcutting the NBs) and 2. oppose to the idea of lowering the ISO overall quality standards just for the sake of rushing (fast-tracking) some Microsoft crap !
Please, please…
I’m *deelpy* concerned.
Regards,
Răzvan
Bucharest, Romania
I don’t get all this whining. Standards are just that: standards. They are intended to promote interoperability and standardization to reduce cost and confusion on the part of manufacturers. It is typically a messy process until a standard evolves and gets approved. In most other industries, there are more than one standards that co-exist with each other. It is just the way the standardization process works. PDF is now also an approved ISO standard, and it also competes with ODF, at least where archiving is concerned. However, I don’t hear anyone whining about that.
@anonymous,
Standards are intended to promote standardization? I’ll need to think about that one a bit more.
But I’ll ask you this: When should a standard be rejected by ISO? We clearly have a system here that is set up to evaluate proposals and then to vote on them. The existence of a voting mechanism indicates that one of the outcomes envisioned by the process is failure of a proposal.
So when should a proposed standard be rejected? It sounds like you are saying that standards should never be rejected. But if that is true, then why do we vote?
Hi!
I need the file format of NSF-files. I want to decrypt properitary encap2.ond attachments (they are tiny NSF-databases and the equivalent of TNEF for MS-Office which has been reverse engineered very well) on an embedded linux system and I cannot place notes.dll or the whole Notes client onto that appliance because of licensing issues.
Because I can read everywhere that IBM is so open I want to ask if someone knows to read the NSF-file format?
Many thanks, especially for Rob if he makes this properitary file format available, I really need it.
Christian
encap2.ond attachments? They haven’t been used since Notes 4.5, like 10 years ago. How many of those do you still have?
In any case I’m not aware of any NSF file format specification. Your best bet may be to look for a way of doing a one-time programmatic migration of the data on a machine that does have Notes installed, into a format that you can better consume.
(Ben, Ed, let me know if I’m missing something here.)
The important thing to keep in mind is that not every technology you find out there is based on a documented, specified file format. Take Lotus WordPro for example. Beneath its Bento wrapper it is pretty much a memory dump, an unadulterated, undigested serialization of internal WordPro C++ data structures. Aside from mimicking the internal data structures of WordPro, you will not have much luck reading a WordPro file.
So for formats like that, I would not propose them as standards. In the end, a file format is typically not tied to a particular application because it was designed to be tied to that application. It is tied to an application because the format lacked design altogether, or at least lacked a perspective broader than a single application.
Is Christian trying to imply that IBM claimed the NSF format was open?
Any instance of the Notes API can convert an NSF into a detailed XML structure (known as DXL) with a short series of calls. But IBM has never claimed that NSF was or is open.
I think he is implying that some parts of IBM are running around claiming everything should be open at the moment, while other parts of IBM still run their business on closed technology.
Corporates are big and complex entities.
Right. But I think we need to think a bit about what should be standardized at what level. For example, what benefits you more, in terms of access, standardizing the way a Database stores its data on disk? Or standardizing the query language used to access the data, SQL?
I think at the level of documents, your best bet is to get the formats standardized, but at the level of the document managers, better to standardize the protocols/API’s used to locale and access the documents, like HTTP, WebDav, JSR 170 Java Content Repository, etc.
I don’t know if there is some general rule or higher observation here, but that’s what it looks like to me.
The thing to remember about IBM is that we’re inveterate integrators. We thrive on heterogeneous environments. Standards are what allows us to make our solutions work with what the customer already has.
Well, you standardize what you want to interoperate on.
For an SQL database, you probably want to standardize on the SQL to access the data; you might also want a standard ‘unload’ format so that you can unload from IBM DB2 and load back in to Microsoft SQLServer. Or vice-versa.
For office productivity documents, the reason for standardizing seems to be that people attach them to emails; put them on their web sites; and archive them in the expectation they will be accessible in decades to come.
Document managers ? Well, you could always try getting everyone to buy IBM Lotus Notes :-) But of course some will insist on Microsoft SharePoint :-( .
What’s the way to bridge between those two ?
Telcos seem to manage interoperability between competitors. You don’t have to all get phone service from ATT nowadays. It can be done.
@Christian: There is a book called “The file formats handbook” by Gunter Born ISBN 1-85032-117-5which has a bunch of old formats including WK3, but I don’t know about NSF.