Although the term may be alien to some, “plugfests” have been around for around 20 years. A plugfest is when implementors of the same interface get together and test the interoperability of their products. In the beginning this was done with wired standards, USB, etc. (thus ‘plug’). Over the years the term was applied to networking at all higher levels of the protocol stack. The concept is also applicable to document exchange formats like ODF.
We had an ODF Plugfest last week in the Hague. Although we’ve had interoperability workshops and camps before that attracted a handful of vendors, this was the first one that had nearly universal participation from ODF vendors. I’m not going to recap the details of the plugfest. Others have done that already. But I will share with you some of my conclusions, based on long discussions with other participants, from whose insights I have greatly benefited.
In an ideal world, specifications would be perfect and software applications would be bug-free and users would read the manuals and we would achieve perfect interoperability instantly by anointment of the salubrious unction of standardization. But to the extent this planet’s population obdurately persists in imperfection, we are resigned to make additional efforts in pursuit of interoperability. We are not alone in this regard. The only standards that don’t need to work on interoperability are those standards that no one implements.
We should use every licit technique at our disposal to give the user the best experience with ODF we can. In a competitive market you can not get away with telling your customer, “Sorry, your spreadsheet doesn’t work because page 652, clause 23 says ‘should’ rather than ‘shall'”. If you did that you would not have that customer for long. (Unless, of course, you have a monopoly, in which case many seemingly irrational, anti-consumer actions can occur, seemingly without consequences.)
Further, I assert:
- Users want real-world interoperability, and not excuses
- Real-world interoperability is what users see and achieve in practice
- Where vendors have the will to interoperate, achieving interoperability is a known technical problem, with known engineering solutions, but where the will to interoperate is lacking, there are no technical means of compelling interoperability
- Interoperability lies at the intersection of technology, engineering standards, competition law, intellectual property and economics. There are no silver bullets, although there are a arsenal of proven techniques that can help to improve interoperability
- Achieving interoperability is facilitated by a variety of cooperative activities, including standardization, test case creation, implementation testing, online validators, plugfests, defect collection and reporting
Going forward there is a promising constellation of efforts converging around ODF interoperability:
- The OASIS ODF Interoperability and Conformance TC, charged with creating an ODF test suite
- The OASIS ODF TC, finishing up work on ODF 1.2
- OfficeShots.org, providing a way to test the interoperability of a document in many ODF editors
- The ODF Toolkit Union, especially their open source ODF Validator
- The Plugfest participants, who continue to add information and test scenarios to the plugfest’s wiki.
- Groups such as OpenDoc Society and OpenForum Europe which lend their organizational skills and enthusiasm to the effort, and often much more.
So, we’re moving in the right direction. The key thing will be to sustain the momentum from the Plugfest and transition it into an ongoing effort, a Perpetual and Virtual Plugfest where the effort and the progress is continuous.
[6/29/09: I’ve received some emails on the photo, so here are the details:
The picture was taken at 3:30PM on the 2nd day of the workshop.
The lens was a Pentax DA 10-17mm “fisheye” zoom at 10mm. So that explains the projection distortion. The graininess and B&W was from post-processing using Nik Software’s Silver Efex Pro and Sharpener Pro.]
Jesper Lund Stocholm says
Thanks for the enlighting post.
Will IBM create implementer's notes on IBM's implementation of ODF in Lotus Symphony as Microsoft has done for their implementation of ODF in Microsoft Office 2007 SP2?
Jesper, there are a variety of approaches to improving interoperability. I listed a few of them in my post. Implementation notes is another interesting approach.
I think we need to evaluate the various approaches and pick a combination of complimentary approaches that give the biggest "bang for the buck".
It is not obvious to me that writing implementation notes is the most productive way of improving interoperability. For example, can you point to specific ways in which real-world interoperability with Office 2007 SP2 was improved by the existence of interoperability notes? If so, how does that achievement compare to that which would be achieved by interoperability testing and fixing bugs?
For example, I think the user would benefit more from having interoperable spreadsheet formulas than to have a release note that says that their spreadsheet formulas will be stripped.
I'm not saying that implementation notes are bad, just that I'm not convinced that they are a high-yield activity, in terms of benefits achieved versus resources expended.
Jesper Lund Stocholm says
I agree with you that implementation notes are not a silver bullet – but I do believe we will get quite a bit of the way if all major vendors had them.
As you might know, I was looking at "document protection" some time ago where I was trying to understand why Microsoft refused to implement e.g. table-protection.
I tried to figure out which digest algorithm OOo uses for this – but largely in vain. Finally, by brute-forcing the digest created by OOo against a list of known algorithms, another Dane (Christian E. Eysel) figured out that OOo uses SHA1 with no salt and no spin count.
I'd have loved to have been able to look up the attribute in OOo's implementer notes and have had immediate access to that information. Similarly, by looking up similar information in Microsoft's implementer notes, I can see that even though the construct in OOXML allows usage of arbitrary algorithms, Microsoft Office 2007 only supports their own crappy digest algorithm.
Implementer notes does really not have anything to do with fixing bugs (assuming a bug is an error) – they have to do with documenting the valid choices vendors make when deciding which parts of a specification to implement. Another example is OOo's support for MathML. Some time ago I documented that OOo's MathML-parser requires a DOCTYPE-declaration in the MathML markup. That is a completely valid choice to make – but you can only get to that information by digging through markup generated by OOo.
Now, Microsoft's implementer notes are in no way perfect and they need improvements to enhance its usage – but I do think they add real value to the eco-system around it – just as interop-tests, but-fixing etc do. This is especially true when we are dealing with closed source products like Microsoft Office 2007 and Lotus Symphony.
Jesper, that proves my point. You found out about how OpenOffice does MathML without having implementation notes, and you blogged about it over a year ago. Microsoft seems to have figured out how it works as well. So it wasn't really that hard. Sure, it might have been easier with implementation notes, but I'd rather direct that energy into changing OO so it used the normal doctype for MathML. In other words, fix what is wrong rather than just write about it.
An implementation makes many choices when implementing a standard. Some of these choices hurt interoperability and some do not. I don't want to waste time documenting the choices that do not hurt interoperability. And for the choices that do hurt interoperability, I'd rather see these avoided altogether. Merely documenting them is not a particularly useful substitute. I don't see the value of diverting resources to proactively document all implementation choice, especially when only a small percentage of these choices has any impact on real-world interoperability.
But your suggestion is consistent with a lot of proposals from Microsoft. Ignore the interoperability crushing bugs they are introducing today while at the same time talk up multi-year efforts that may or may not improve interoperability in the distant future. Talk, talk, write, write, fund study, create a working group to map formats, etc. Delay, delay, delay.
But this isn't that hard. There are things we can do today, now. to improve interoperability and we know exactly what they are. You don't need a committee or implementation notes or a sponsored study to tell you that stripping spreadsheet formulas from ODF spreadsheets in Excel 2007 SP2 significantly hurts interoperability.
It reminds me of something
Jakub Narebski says
You can read about ODF Plugfest (among others) at KDENews: KOffice Developers At The First ODF Plugfest. (Just a note, as you don't provide link to any ODF Plugfest report).
Jesper Lund Stocholm says
Do you know if there are any (and I do mean any) open positions with IBM these days? I mean, it must be great to work in a company that has unlimited resources to each product/project they work on. It must be great to work in a company where you don't need to worry about the time you spend on your tasks.
I do remember when I did the work leading up to the article about MathML in OOo. It might not have been rocket science, but it was tedious work and took a lot of time and effort. It would have saved me a lot of time (and if I was doing it for a customer, also money) to figure out what went on, if I had been able to look up the element in OOo's implementer notes to get the result.
And this is my point – implementer notes are of value to almost every implementer doing work on e.g. ODF. So while companies like IBM and Microsoft fly their big-shot kahunas like you and Doug around the world to parties like the one in The Hague – where you have restrained yourself from providing company-specific output of what happened (how's that for openness?) – the rest of us are faced with the day-to-day problems of getting ODF-documents to work between your implementations.
Even though you continue to pull your "spreadsheet formula"-red herring, this has nothing to do with the stupid decision Microsoft did wrt formulas – this has do do with getting documentation about your software to the community using it.
On the contrary, Jesper, our resources are finite and that is why we must carefully prioritize our efforts.
I'll ask you to consider the following and let me know whether you disagree, in principle with these points:
1) There are several approaches to improving interoperability.
2) The various approaches to interoperability differ in the kinds of defects they find, how efficiently they find them, and how quickly they fix the underlying problem.
3)With finite resources, an organization seeking to improve interoperability for their customers will engage in those activities which address customer problems most efficiently.
Would you agree to the logic of the above?
Given that, the question comes down to how efficient implementation notes are for improving interoperability compared to other known techniques. My observation is that the practical efficacy of implementation notes has not been clearly demonstrated, and the example of Office 2007 SP2 is clear example of where they did not help at all.
Right now, the fastest way to get any interop issue addressed to to develop a scenario illustrating it and submit that to the Plugfest wiki. That will get it tested on a variety of implementations, the bugs noted and starts the discussion on how to address the issue in current implementations and in the standard as well, if needed. This seems to be a far more efficient use of resources than documenting thousands of implementation choices, 99% of which have zero impact on interoperability. Let's focus on fixing the the known problems.
Jakub Narebski says
IMVHO implementer's notes can be useful if implementation is done right (they might be more clear than necessarily detailed specification(s)). I agree with Rob Wieir that they are next to useless if you fail implementation…
Jesper Lund Stocholm says
(I appologize for the delay in my response to you)
I agree to all of your three points you mention – however – I don't really see where you are going with it.
For the record, I think ditching the formulas from the OOo namespace in existing spreadsheets was a stupid move by Microsoft. However – I fully understand their choice of using the formula specification from IS29500. I'd wish, though, that they'd made an import filter converting the formulas in the OpenOffice-namespace to formulas described in IS29500.
You say that the implementer notes didn't help in this situation – but you can't really blame this on Microsoft. They put this information out there in, I don't know, late Fall 2008, so there was amble time to act on it for anyone interested. But no-one did. They waited – like you – until they released their product.
How's that for screwing your customers, Rob?
Hey – one could even argue that Microsoft chose to implement the formula language with the absolutely widest support out there – probably a magnitude larger than the proprietary formula specification from OOo. And if that wasn't enough – OOo has exellent support for the IS29500 formulas … not only within XLS-files, but in XLSX-files as well. Since OOo is LGPL-licensed, all you had to do in Symphony was to "flip the bit" to intrepret these formulas as well. As you have previously said, "interoperability is not hard"; at least if you want to.
When you talk about limited resources – what would you guess would require the least amunt of resources:
1) Publish implementer notes documenting your choice of formula specification?
2) Flying everyone to The Hague to sit in a room with constraints banning you from talking about product specific findings?
You might argue that "the right thing ™" to do for Microsoft was to implement the proprietary formula specification from OOo – but please note that one man's "by convention" is another man's "proprietary extension". It's not a black/white scenario. Do also note that when the bottle stops spinning in the blame game you are playing here, it can only stop at one place – you and thereby the ODF TC. Even if we for a second agree that Microsoft is the incarnation of pure evil – you are responsible for letting this happen to the eco-system of ODF-files and -implementations. You are responsible since you have been sitting on your hands for the better part of 4 years failing to finish the OOF-specification.
Jesper, you have not in the least bit defended your assertion that writing implementation notes are a worthwhile effort. In fact, you've acknowledged that Microsoft having implementation notes for Office 2007 did not help. This is obvious, since Microsoft can write all the notes they want and this does not prevent them from stripping spreadsheet formulas from everyone else's spreadsheets, including those created in their own ODF Plugin. If their implementation notes did not help their own sponsored Plugin be more interoperable with SP2, then I think that shows how ineffectual they are. I'm looking at results, not theory. And in terms of results the implementation notes have been useless.
In any case I think this is a bit more black and white than you make it out to be. This is not a matter of one "proprietary extension" versus another. This is a case of all vendors except Microsoft choosing to implement interoperable formulas, and Microsoft choosing to do something else. That's reality. That's what users see. Your argument seems to be that IBM, Novell, Google, Sun, KOffice, Gnumeric all should change their formulas to match what Microsoft dropped on the world in SP2, rather than have Microsoft simply do what they already know how to do (via their Plugin), what the other vendors have already done for years, and which we have documented in a specification currently going through standardization in a committee in which Microsoft participates.
And you wonder why people call you a "Microsoft drone"?
Jesper Lund Stocholm says
> If their implementation notes did not help
> their own sponsored Plugin be more
> interoperable with SP2, then I think that
> shows how ineffectual they are. I'm looking
> at results, not theory. And in terms of
> results the implementation notes have been useless.
Implementer notes does not help unless someone uses them. This is also true with your precious plug-fests. If all you guys do is to is to aquire the corporate frequent flyer miles when going to The Hague, NL or Orvieto, IT … but don't act on what you found when you get back, plug-fests are
worthless as well.
You have made your own little self-fulfilling prophesy here. First you claim that implementer notes are useless. Then you refuse to read them and after that you say "I told you so – they don't work".
The question is, of course, why didn't IBM implement ODF-support for the IS9500-formulas in Lotus Symphony? The information about Microsoft's desires had been available to you since late Summer 2008 and OOo has excellent MSOXL -> OOF support in their OOXML import filters. All you'd have to do was to "switch the relevant bits". As you have said yourself, interoperability is not hard.
> Your argument seems to be that IBM, Novell, Google,
> Sun, KOffice, Gnumeric all should change their
> formulas to match what Microsoft dropped on the
> world in SP2
Then you are not reading my point right. I am asking you to apply the principle of robustness (Postel's law) that you yourself have written about. I am asking you to be generous in what you
read/accept and strict in what you produce. I am not saying that you should drop OOo formulas – I am
saying that you should support the formula language used by the office application with the largest install base there is. If you'd like to keep the OOo syntax as output format, then I think that is an excellent idea.
You talk a lot about "users" – but do you really think people care which one's is bigger … or who
has made "the right choice ™"? Do you think you took the best possible care about IBM customers
when IBM decided not to support IS29500-formulas in ODF? It seems to me that "what customers want" is only relevant when you are hammering on Microsoft … and not when it is IBM disregarding their wishes.
> And you wonder why people call you a "Microsoft drone"?
Actually, I am pretty sure I know why some call me a "Microsoft drone" – usually it's because they are idiots. Other times, like here, when you call me "a Microsoft drone", it usually occurs when they find themselves on the wrong side of a losing argument.
Imagine how torn I must feel. In this particular case I agree with Microsoft in using the formula specification from IS29500 since ODF doesn't have one. But I also agree with IBM when it comes to Microsoft's stupid decision to drop formulas with the OOo syntax.
Does that, in your opinion, qualify me as an "IBM drone" or as a "Microsoft drone"?
Jesper, I think you misunderstand Postel's Law. The point there is about robustness to small variations. So a browser should not reject an HTML page merely because the author forgot an end tag. It should degrade gracefully in the presence of localized errors. It does not mean that all browsers should write support for SilverLight just because Microsoft has a desktop OS monopoly.
Similarly, with ODF supporting applications, Postel's Law suggests that applications should be robust to small errors and variations in the markup. But it does not mean that they must provide support for some entirely different standard, just because Microsoft has a monopoly there.
In any case, this is not how we do it with open standards. We don't say "I have a monopoly and here is how I do things and everyone take it or leave it, but you better do this if you want to interoperate with us." That's not how we do things with ODF. We sit down, with multiple vendors, IBM, Sun, Google, Novell, Microsoft, etc., and negotiate what the standard says and then we implement the standard. The expert from the company with the monopoly has the same vote as the expert from the open source project. We write the standard, find bugs, fix the bugs, make interop test cases, hold plugfests, etc. We do it all openly and with full public scrutiny.
And in the end, that is why ODF wins. As you know, the whole charade about OOXML falls apart in a few months when the world discovers what Microsoft is doing with OOXML in Office 2010.
Jesper Lund Stocholm says
"In any case, this is not how we do it with open standards. We don't say "I have a monopoly and here is how I do things and everyone take it or leave it, but you better do this if you want to interoperate with us."
I know that you want to portray the decision-making around e.g. OOo-formula usage in ODF as some sort of "we-sit-around-a-camp-fire-roasting-marsh-mellows-and-enjoying-ourselves-in-the-woods".
This is what you usually call "by convention".
But I don't buy it.
The reason why all (small) applications supporting ODF also support the OOo formula language is the simple fact, that to take part in the ODF ecosystem, you need to be compatible with the market leader. In ODF-sense, this is OOo. You have argued exactly the same on my blog where you insisted that noone would care about what the OOXML-spec says – all that matters is what Office 2007/2010 does.
This is in no way different with ODF, because you will also have to obey the irks and quirks of OOo to be able to participate.
To put it in a nut-shell, if OOo won't load your document – you're toast.
As you know, the whole charade about OOXML falls apart in a few months when the world discovers what Microsoft is doing with OOXML in Office 2010.
Is there any reason that we'll have to wait until "a few months" to know this? Do you have some insight that you'd like to share – or would you rather wait until Microsoft Office 2010 hits the market – as you did with Microsoft Office 2007 SP2?