On the topic of adding features, I think we can agree that it’s always better to add a feature that is designed to add functionality, even if the reason for its existence is purely compatibility. In my opinion, features with such a narrow scope are inelegant, and inelegant solutions usually lead to trouble sooner or later. Sadly, this subtle argument seems to have been drowned out by arguments about how the specific instances of compatibility features in Office Open XML are poorly documented.

I also agree that loading speed is an odd argument. Moore’s Law (and its relatives) suggests that if a file takes N seconds to load today, it’ll take N/2 seconds in August 2009 and N/32 seconds by August 2017.

Then again, my opinions are often wrong, so now I’ve got two more questions I want to ask Brian Jones (Microsoft’s most technical OOXML blogger). I plan to bring them up next time the conversation moves around to either of these topics, so you might want to keep an eye out there.

I could well be wrong about this, but my understanding is that OpenDocument 1.2 is nearing completion, and will contain the results of the formula committee’s work. That will hopefully move the discussion forward about how to resolve the issues around formulae.

Anyway, I’m really just replying because I’m so glad to see that I’m not the only person in the world that’s still making his mind up about all of this!

– Andrew

Correct, I am not opposed to adding features if those features are required to preserve layout, but can also be used on their own merits. If, instead of telling a program to “act like Word would” you can instead describe the actual details of how to actually display it, that would be a great thing.

I can see and unerstand MS’ philosophy. It certainly is a tricky question. Obviously, some documents require a longer life and for others it doesn’t matter so much. With modern computers being so fast, however, the difference in speed between a well-structured portable format and a tightly bound format is likely not nearly so big as it used to be. We’ve come a long way since 33mHz processors and low memory, and that certainly would affect the debate. As for complexity, well, that is to be determined. With the burden on the format to carry legacy information (instead of translating such), OOXML could be considered more complex to implement at this point in time.

The practical issues in dealing with fixing things like formulas will certainly take some time to figure out. Luckily there are many smart people working on ODF formulas, and they are likely considering this issue. It would be a shame to see broken math codified in either format.

There’s a lot to be said for your approach, although the arguments involved are sadly more complicated than you make out.

But first, carrying on my theme of being a stickler for details, would it be fair to say that you’re not opposed to adding features if it’s the only way of achieving compatibility, so long as the features expose functionality that can be used on its own merits?

For example, if “autospaceLikeWord95″ turns out to mean “calculate spacing using function F on even numbered pages, but function G on odd numbered pages”, you would recommend adding something like “autospaceEvenPagesFunction” and “autospaceOddPagesFunction”?

(I’m not making any particular point here, just making sure we’re on the same page).

With that out of the way, I’ll begin my post proper. In fact, I’ve got way more to say than can reasonably fit into a single comment, so I’ll limit myself to one philosophical and one practical issue. Philosophy first:

One thing you have to remember when looking at Microsoft is that they’ve been doing their own thing behind closed doors for the past 20 -30 years, and although they seem to have argued out all the same big debates that were going on in the rest of the computing world in the 80′s and 90′s, they’ve come to very different conclusions on many of them. Understanding what they say and do often means rethinking some of the fundamental stuff we’ve all taken for granted for the past few decades.

Going back to an example from an earlier post of mine, the public computing community generally agreed that a good file format specification should describe an idealised view of the data it’s representing. This leads to file formats that are portable and easy for beginners to learn, potentially at the cost of implementation speed or simplicity (because you need to load a larger document and translate it into internal data structures).

When the same argument was had inside Microsoft, my guess is that they concluded “our customers don’t care about any of that stuff, they want fast programs, and they want us to add features instant they request them”. As such, Microsoft seems to have decided that a good format should be an accurate representation of the data in memory at the moment the user presses ‘save’. This leads to file formats that are bound very tightly to the application they’re developed on, but which are therefore fast (loading a file involves very little processing), and easy to change (adding a feature means adding a data structure that doubles as the file format representation).

Now that Microsoft have decided to open up their formats, we have to merge these two ideologies back together again, which means reopening arguments we thought were settled for good. As well as acknowledging that each side has a valid, philosophically different point of view, it’s important to recognise a bit of psychology: as you read the above paragraphs, you were probably thinking “so Microsoft want me to give up compatibility for a bit of speed? That’s just not going to happen”; similarly, Microsofties would read the above and think “so they want to tie up the simplest of features in red tape so that some newbie can write a homebrew word processor? That’s just not going to happen”. Arguments like Rob’s about the actual, measured speed of different formats are important because they address they address the legitimate advantages that Microsoft are afraid of losing.

Moving on to the practical issue, you said that date maths should use a “broken” tag to indicate that the file depends on broken handling of dates – I’ll pick on that because it’s a topic I know a bit about, and is probably a good example of the more general issues. Personally, I still have hope that some solution can be found that properly deprecates brokenness, but I’m not convinced about your solution. How would programs handle such a flag for example? If they were to pop up a message saying “your program might have a date bug” each time the file is opened, users would ignore the popup then get upset when their spreadsheets broke without giving any further warning.

Also, I suspect you’re underestimating just how nasty Excel’s file format is. For example, any fully backwards-compatible program needs a strategy to deal with formulae up to and including this level of evilness:

=IMREAL(IMEXP(COMPLEX(0, “03/01/1900 03:23:53.606″))) + WEEKDAY(“4 March 1900″)=0

As I said at the start, there’s a lot to be said for your approach, it’s just that you need to be looking much deeper if you want to have a productive debate with Microsoft.

– Andrew

I don’t see a need for compatibility settings inside a new format at all. It should be the responsibility of the converting program to handle any “bugs” and put them into the new standard sanely. The whole compatibility argument just seems like misdirection.

As an example, there shouldn’t be a “autoSpaceLikeWord95″ property in a new format. If there are special spacing rules for that version, the program should be able to convert them to something in the new format.

For items like language lists, instead of keeping the fixed list, simply add the logic to map these things in the application.

For things like mathematical formulas or date math, adding things to a new format like a flag to the formula to indicate “broken” calculation would be sane instead of codifying incorrect behavior, avoiding the purpose of getting it right.

When you have a chance to do something correctly and use existing standards, you should. It would also very likely reduce complexity of implementation.

If OOXML was to clean up those types of things, it would be a lot closer to harmonization with ODF, which should be the ultimate goal.

I await your post keenly, and I’ll leave my main comments until then, but I’d just like to make one point:

There seems to be a big gap between the problems that OOXML addresses and those that it solves to everybody’s satisfaction, so it’s important that we’re clear about which we’re discussing at any one time. Your articles make good points about OOXML’s speed and lack of detail, and the articles help answer the question of whether OOXML has yet met its design goals, but they only hint at the nature of those goals.

If bug-for-bug compatibility is important for example, the next question is whether it’s compatible with ODF: if so, the logical course of action is to fold compatibility in; if not, the logical course of action is a second standard. Only after that does the question of OOXML’s fitness for purpose comes in.

– Andrew

I’ll answer your question fully in an upcoming blog post. It is a good question and deserves a good answer.

To your specific points, I did a study of OOXML versus ODF performance a while ago, and found that, in the word processor format at least, OOXML is much slower than ODF.

Also, the claims of fidelity with legacy documents is more a statement of what MS Office does. It really is not a property of the OOXML format, which has almost no details regarding compatibility. In fact, many of the comments registered in the US are about compatibility settings in OOXML that lack details. Think of it this way — even if Microsoft never published the OOXML specification, their Office product would have backwards compatibility. So the compatibility is not dependent on the format. Only to the extent that this compatibility is fully and accurately detailed in a publicly available document, with IP rights, etc., is this something that others can practice as well. So criticisms of whether in fact OOXML is sufficient to this purpose are of great importance.

I realise the magic wand question is rather implausible, but I find it difficult to really understand Rob’s approach without knowing what exactly he’s advocating. For example, posts on this blog tend to widen the (already large) gulf between the ODF and OOXML communities, because there’s a constant undertone that we should judge Microsoft for OOXML’s failings, rather than be a critical friend. That’s entirely appropriate if Rob’s opinion is that OOXML should cease to exist, but counter-productive if he wants ODF and OOXML to some day become compatible.

The point of this sort of hypothetical is to shake loose people’s underlying beliefs, and your answer tells me that you don’t think OOXML addresses any problems that ODF doesn’t. I mentioned before that there’s a gulf between the ODF and OOXML communities, and one of the effects of that is that each side only really gets to see a cartoonish sketch of the other side’s position. Microsoft do have valid arguments about why ODF isn’t sufficient, and I’ll give you a couple if you’ll promise not to claim I agree with them just because I understand them:

1) “High fidelity” (i.e. bug-for-bug) compatibility with old versions of Office. Although it’s rightly pointed out that nobody has any serious expectation for two versions of Word to display a complex document exactly the same, people do expect different versions of Excel to return the same value for any given input to a function, no matter how many bugs their spreadsheet relies on. Microsoft’s solutions (e.g. the whole issue with dates) aren’t great, but nobody’s going to move their documents over to ODF if they have so much as a whiff of fear that doing so will silently break a spreadsheet that their business depends on.

2) Being fast, rather than beautiful. Speed isn’t an issue for sensible uses of documents, but when people start making things like 60,000 row spreadsheets, there is an argument that the elegance of ODF leads to unacceptable slowness. And yes, people should use a database instead. They should also eat healthy food and get regular exercise, but saying it doesn’t make it so.

I don’t necessarily agree with these arguments, but dismissing them as flimsy excuses will only sway people that agree with you already.

– Andrew

The magic wand must be pretty mighty for Microsoft to relinquish control over OOXML to third party that has no partnership ties to Microsoft; remove any reference and/or dependence on prior Microsoft document format implementations; remove dependencies on Microsoft-created document “standards”, such as MathML and VML use existing standards instead, such as SVG.

What I’ve mentioned above are just a few off the top of my head…. So I’d ask, after all those changes, what, exactly, would be the point of OOXML, and how would it be any different than ODF?

The whole point for Microsoft is, that it, as the dominant player in the software industry, is promoting its own document format (over an existing standard, i.e. ODF) as a leverage against competition. Otherwise there is no value for Microsoft to pursue this course of action. So, if you remove everything that is wrong with OOXML (and everything that Microsoft is doing wrong), you’d end up with something that is not even worth mentioning when compared to ODF.

Rob, I think I understand what you’re opposed to in the current process, but I’m not sure what you’d rather be happening. If you could wave a magic wand and make anything you liked happen to OOXML, what would you do? For example, would you make it go away altogether, or make it compatible with ODF, or make it 12,000 pages with sufficient detail on everything mentioned?

– Andrew Sayers

However, you are blowing the enablement issue out of proportion. (True, the compatibility tags have no rightful place in the standard, but they are really a minor issue.) The critique that “person having ordinary skill in the art” could “make and use” OOXML applies equally to ODF. No individual, no matter how talented, can build a modern word processor or spreadsheet from scratch!

Adapting a competing program to use OOXML, however, *should* be within the skill of the developers who maintain those programs. If they could do it for the binary formats, and the XML formats are more or less a transliteration of those binary formats, they can surely do it for the XML formats.

My argument would be that if you do this, you will find little pattern of the control Gary et al assert. Sometimes Sun people lead the way, but just as often it’s people from KOffice, IBM, or independent people like me.

For example, I had a fairly large role in the recent metadata proposal. At a certain point we were having some fairly heated arguments about some technical details (all public, BTW). It so happened that the positions generally split down between people that worked for Sun and those that did not (including me, and an engineer from IBM). The final proposal (supported by the Sun engineers) in fact reflected a position that they did not at first agree with. That a Sun engineer co-edited the proposal does not take away from the fact that it was a consensus proposal, whose ideas were largely driven by people who don’t work for Sun.

Gary et al (which as far as I can tell means Sam and Marbux; I have nothing to do with this organization anymore) will say this work has been dashed because Sun refused to guarantee preservation of this metadata, but the fact is a) these guys played no constructive role in this discussion (they did not attend the TC call where we discussed this), b) engineers from KOffice and IBM also expressed serious concerns about the implications of this (it was not per se a Sun position), and c) if anything the preservation language is stronger (and certainly not weaker) than that in OOXML.

But to turn all of this around, as Rob said, it’s an interesting irony that an open process (or indeed society) allows for claims of corruption, etc. By contrast, we know nothing about any of the decisions ECMA TC45 made, much less the issues that were raised in that process. We also have no idea about any internal disagreements. That would worry me. Real open standards work almost by definition gets contentious at some point.

Anyone know how accurate the above is? It’s certainly believable enough, given the other things we’ve seen elsewhere.

This is my blog, where I have 100% dictatorial authority over subject matter, and the subjects covered will tend to be OOXML and berries. No one forces you to read it. If you don’t like the topic, then change the channel. Also, I tag each post, so you can quickly see which ones are about OOXML versus ODF versus berries.

@Wraith, I think you will find Gary’s notes in the ODF TC mailing list, unedited and uncensored. I think the one case where there were wild accusations of censorship it was found that the person was simply looking at the wrong mailing list archive.

Gary is entitled to his curious ideas, and I’m entitled to disagree with them.

Openness does not fear dissent. Openness does not fear scrutiny. Openness does not fear controversy. Openness does not fear heresy.

Also Gary clearly states that the OASIS TC for OpenDocument is fully under control of Sun and when that was soemhow challenged by another party they were forced out of the TC.

How open, transparant and collaborative is that ??

Thanks for writing.

Why bother pretending that she made a decision at all? She had already made up her mind the minute she walked into the job, as we suspected. And don’t fool yourself on hope for changing her mind. If she didn’t consider the facts, technical, legal, or financial, in this decision yesterday, she won’t bother in the future.

You might want to make it something like ‘a meal can have too much sugar and too few vitamins at the same time’ or something.