≡ Menu

The Anatomy of Interoperability

I’d like to talk a little about about interoperability. Although I’ll concentrate on document formats, the concepts here are of wider applicability. Wherever you have standards and implementations, these are some of the issues that you will want to consider.

Rather than attempting the reckless frontal assault of defining interoperability, I’ll take an oblique approach and discuss the forces that tend to increase or decrease interoperability. By understanding these forces, and how they interact, you can get a good sense of how interoperability works, and have a better idea how the decisions that are being made today will determine the interoperability consumers will experience in the years to come.

I’ll start with the specification itself, the text of the standard. What characteristics might a specification have that hinder interoperability?

  1. Ambiguities — The specification may describe a feature in a way that is open to more than one interpretation. This may be caused by imprecise language, or by incomplete description of the feature. For example, if a specification defines a sine and cosine function, but fails to say whether their inputs are in degrees or radians, then this function is ambiguous.
  2. Out of scope features — The specification totally lacks description of a feature, making it out of scope for the standard. For example, neither ODF nor OOXML specifes the storage model, the syntax or the semantics of embedded scripts. If a feature is out of scope, then there is no expectation of interoperability with that feature.
  3. Undefined behaviors — These may be intentional or accidental. A specification may explicitly call out some behaviors as “undefined”, “implementation-dependent” or “implementation-defined”. This is often done in order to allow an implementation to implement the feature in the best performing way. For example, the size of integers are implementation-defined in the C/C++ programming languages, so they are free to take advantage of the capabilities of different machine architectures. Even a language like Java, which goes much further than many to ensure interoperability, has undefined behaviors in the area of multi-threading, for performance reasons. There is a trade-off here. A specification that specifies everything and leaves nothing to the discretion of the implementation will be unable to take advantage of the features of a particular platform. But a specification that leaves too much to the whim of the implementation will hinder interoperability.
  4. Errors — These may range from typographical errors, to incorrect use of control language like “shall” or “shall not”, to missing pages or sections in the specification, to inconsistency in provisions. If one part of the specification says X is required, and another says it is not, then implementations may vary in how feature X is treated.
  5. Non-thematic provisions — If a standard is claimed to be platform-neutral or application-neutral, then anything that is tied to a particular platform or application will hinder that type of interoperability.
  6. Feature Creep — A standard can collapse under its own weight. There is often a trade-off between expressiveness of a standard (what features it can describe) and the ease of implementation. The ideal is to be very expressive as well as easy to implement. If a standard attempts to do everything that everyone could possibly want, and does so indiscriminately, then the unwieldy complexity of the standard will make it more difficult for implementations to implement, and this will hinder interoperability.

The forces that help counteract these problems are:

  1. Public participation — The public gets involved early, reading drafts of the standard before it is approved, and also meeting minutes and discussion list traffic. The public provides comments to the technical committee and reads the comments submitted by others. Note: the mere theoretical ability of the public to participate results in absolutely no benefit to the standard. This is like a beta program that exists in theory only will uncover zero bugs. What counts is the actual participation. Openness is a means to participation, but is insufficient on its own.
  2. Expert review — The feature set for a modern word processor or spreadsheet is large and requires interdisciplinary expertise in such diverse areas as vector graphics, text layout, mathematical layout, accessibility, internationalization, schema design, statistical and financial calculations, etc. So the design of a file format will similarly require a broad range of expertise. Tapping into outside experts (for example, from member companies or other technical committees) when designing the standard, or as reviewers, is a way to reduce the types of problems that hinder interoperability.
  3. Multi-vendor participation — It is good to have multiple active participants in the standards development process, representing implementors both proprietary and open source, both traditional implementations as well as web-based. You might consider other constituencies as well, such as end-users, accessibility advocates, academic and government representatives, consumer advocates, etc. There is a saying familiar to programmers, “Every class of users finds a new class of bugs”. This is as true with standards as it is will code. More eyeballs, especially with different perspectives, is key. It is better to have 5 active participants with different perspectives than to have 100 participants from a single vendor. Again, I’m talking reality here, not just perception. It is easy to have a technical committee that is large on paper, but still have all of the ideas come from a single company. This does nothing for interoperability. You want to have really multi-vendor participation. This is why having public read-only access to the mailing list of the technical committee is so valuable. That is great way to see what the real participation level is.
  4. Multiple implementations — If a standard is intended to be platform-neutral and vendor-neutral, then the participation of multiple implementors working on multiple platforms is essential. They are the ones who truly drive interoperability. Especially if they are implementing while the specification is under development or review, they will find, report and resolve many interoperability issues.
  5. Reuse of existing standards — When writing a program, the availability of reusable code or design patterns can improve quality and shorten schedules. In manufacturing, the use of off-the-shelf components can be faster to tool and be less expensive than custom components. Similarly, in standards development, when you reuse another standard you reuse the domain expertise, the standards development effort and the review effort that went into the development of that standard. If you are lucky and you reuse a widely-implemented standard, implementors will likely also be able to reuse knowledge and even code when implementing it.
  6. Existence of a reference implementation and test suite — These work, together with the standard itself, as a triad of verifications that reinforce each other. The test suite is written to test the provisions of the specification. The reference implementation is written to implement the provisions of the specification. The test suite is executed against the reference implementation. Any errors indicated are thus either errors in the reference implementation, the specification, or the test suite. Fix the bug and repeat until the reference implementation passes the test suite. Having a reference implementation that passes a test suite that has high or complete coverage of the provisions of the specification is ideal for interoperability.
  7. Good engineering — In the end, a technical standard is technical, and it will be influenced by the same kinds of engineering criteria that influence how we architect and design software applications. If you can make a standard that is modular, exhibits internal consistency in its parts, is cohesive, has reasonable dependency chains among its parts, does what it set out to do with minimal complexity, but allows of uses beyond that, then you’ve made a good start. But if you find yourself shaking your head, slapping your forehead or even holding your nose when reading a standard, then you are heading for interoperability hell. The standards that have worked well were as simple as possible, but no simpler. Think XML 1.0 or XSLT 1.0. Engineers know how to build complexity on top of simplicity. But you don’t build anything on top of crap.

Of course, the standard is only one half of the equation. The implementations are the other half, and they have their own list of ways they can hinder or help interoperability.

The factors that hinder interoperability include:

  1. Implementation bugs — Conformance to a standard, like any other product feature, gets weighed against a long list of priorities for any given product release. There is always more work to do than time to do it. Whether a high-quality implementation of a standard becomes a priority will depend on factors such as user-demand, competition, and for open source projects, the level of interest of developers contributing to the community.
  2. Functional subsets — Even in heavily funded commercial ventures standards support can be partial. Look at Microsoft’s Internet Explorer, for example. How many years did it take to get reasonable CSS2 support? When an application supports only a subset of a standard, interoperability with applications that allow the full feature set of the standard, or a different subset of the standard, will suffer.
  3. Functional supersets — Similarly, an application can extend the standard, often using mechanisms allowed and defined by the standard, to create functional supersets that, if poorly designed, can cause interoperability issues.
  4. Varying conceptual models — For example, a traditional WYSIWYG word processor has a page layout that is determined by the metrics of the printer the document will eventually print to. But a web-based editor is free from those constraints. In fact, if the eventual target of the document is a web page, these constraints are irrelevant. So we have here a conceptual difference, where one implementation sees the printed page as a constraint on layout, and another application is in an environment where page width is more flexible. Document exchange between two editors with different conceptual models of page size will require extra effort to ensure interoperability.
  5. Different code bases — The more distinct implementations, the more paths of exchange. If you have only a single implementation of your standard, then there will be no interoperability issues, at least not with others using the same version of your application on the same platform. But if there are two implementations, then you have 2 paths of exchange to worry about. If there are 3 implementations then you have 6 paths, and so on with N*(N-1) possible paths for exchanging files with N applications. Consider the ODF format for word processing documents, where we will soon have implementations in Office (at least two different Plugins), OpenOffice, Lotus Notes, KOffice, Google Docs and Abi Word. This is 42 exchange paths (or more if you consider platform variations) though obviously not all of these paths will be common.

And the forces that counteract these and lead to improved interoperability are:

  1. Test suites and reference implementations — As described earlier, these are key. During the development of the standard they improve the standard, and after the standard is approved they continue paying dividends by helping improve the quality additional implementations.
  2. Conformance certification — This can take several forms, from self-testing and self-certifying, to more formal tests by 3rd party labs. The result is a compliance report, listing which features of the standard were correctly implemented, and which were not. Having this information available to consumers can help interoperability to the extent it allows them to narrow their consideration to the applications that most fully and correctly implement the standard.
  3. Profiles — This is a formal mechanism for defining an interoperable subset of a standard. For example, XHTML Basic is a W3C-defined subset of XHTML for use on mobile devices. By defining such a subset, interoperability is enhanced in several ways. First, a single web service can produce XHTML Basic that is understood by mobile devices from several vendors. Second, XHTML Basic is a proper subset of XHTML, so it is also interoperable with all the tools that accept XHTML.
  4. Adaptive extension mechanisms — It is good to have a formal mechanism for extending the specification while at the same time allowing a graceful recovery in the case where the document is loaded in an application that doesn’t understand the extension.
  5. Inter-vendor cooperation — Make no mistake. Once an interoperability problem is found, where application A and application B are not rendering the same document in an acceptably similar fashion, then we have a bug. Such a bug may be reported by the customers of application A or the customers of application B. Ideally, A and B will cooperate and share information on interoperability bugs found. The customer that reports A’s bug to company B, may tomorrow be matched by another that reports B’s bug to customer A. So some sort of interoperability bug clearinghouse or other protocols can help here.

I may have missed some important factors that help or hinder interoperability. Please share your ideas.


3/29/07 — Corrected error sent in by an anonymous reader. The number of interoperability concerns scale as N(N-1), not as N!.

Creative Commons License
This work, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.

{ 7 comments… add one }

  • steve_l 2007/02/20, 06:03

    I work in web related specs, and its really hard to get things to work together. Interop is tricky. But without it, you have nothing.

    1. you can have a formal spec written in a formal language like Z, this should give you a rigorous spec without the need to implement all the code. But it is very hard to enumerate all possible failure modes, as that is so implementation specific.

    2. test suites. First. I have found incredible resistance to test-driven development in the OASIS and W3C groups, which I think is partially driven by the fact that many of the people who participate havent coded for a while, and are not current with modern test-driven processes. There is also something about a standards body, where
    the timely 1.0 bit of paper
    is deemed important, and the implementation an afterthought.

    A big issue here is politics. There was a lot of pressure to ship ODF in a timely way, and now there is even more pressure from MS to catch up. But without test suites, there is no formal way to verify compliant implementations of either spec.

    There is another way to do interop: the open source way. You dont need multiple implementations -you have an implementation that everyone uses. The UC Berkeley TCP stack is the best example of this. For a long time it was the TCP/IP specification.

  • Ben Langhinrichs 2007/02/20, 08:50

    I think you have done a pretty good job of laying out the issues. One area that might be better fleshed out would be around the areas of functional subsets and supersets. A major issue with interoperability is whether functionality degrades gracefully when a function/method/tag/feature is encountered which is either in the standard but not implemented in this instance or extends the standard and is not in this implementation.

    The key here is that if the implementation degrades gracefully, and even better if the standard defines a way for the implementation to degrade gracefully, interoperability is well served. Take HTML as an example. If an unknown tag is excountered, should the implmentation fail, ignore the tag and everything in it, ignore the tag but not child tags within it? Those are very important issues with a standard and extensions beyond the standard, and there are ways to define how both subsets and supersets should be handled.

  • Anonymous 2007/02/20, 09:03

    There is a big risk that if for instance Micrsoft is forced in the next few years to support ODF that they will create an independant implementation that is standardsconform but not interoperable with other implementation.

    The ODF spec is very open to multiple interpreting of it parts and thing like the “office-settings” and the extensibility allow for a wide variety in ODF files.
    I it very difficult to get agreement on a set op interoperability guidelines and consent about testsets if large parties will not agree to those and they are not a formal part of the standard specifications.

    It is very well possible that in the a year or two an enourmous amount ODF files will be created using plugins for MS Office. Mayby even more then with OOo. As MS is harresed severly over ODF they might not be persuaded very easily to commit to interoperability. Showing the poor interoperability that the ODF spec allows would confirm why they could not use it as their Office format.

  • Anonymous 2007/02/20, 09:59

    Thanks, that was probably the best post I’ve read from you yet. I’d like to add a few personal comments.

    One of the issues in dealing with a format is the generator-consumer relationship. The generator is basically free to choose the subset of features from a specification that best suites its needs. The consumer on the other hand, is typically expected to handle input from any generator. This requires handling every aspect of the specification correctly.

    An example might be the choice of encoding of an XML file. Let’s leave alone the way you’re supposed to detect encodings given that the encoding is encoded in that encoding :). A generator could simply choose an encoding of convenience. The consumer must handle any potential encoding. Given the broad set of possible choices, the consumer ends up having to pull in support for a broad array of encodings.

    I guess what I’m trying to say is that the freedom of choice for the generator comes at a significant cost to the consumer.

    This is a pervasive theme for XML. An XML consumer needs to deal with dozens of oddities in the XML specification. Reserved character handling, the issue of whitespace significance, schemas, binary data nonsupport, etc. In the end, consumers end up pulling monster libraries and then have to add custom logic on top just to read the format. Sometimes.

  • Rob 2007/02/20, 13:10

    Steve,

    I have seen some conformance work in OASIS, though as a separate TC. For example there was an XSLT Conformance TC a few years ago that developed an XSLT/XPath test suite. And the W3C does have its online validators, which is great.

    A test suite is a daunting task. Some work was started at the University of Central Florida and picked up by the OpenDocument Fellowship but it has only a few hundred test cases. ODF, a 700 page specification probably has on the order of 5 testable statements per page, each one of which could require 4 test cases to test the main and edge conditions, positive and negative tests. So we’re talking 14,000 test cases. Even if I’m off by a factor of 2 or 4, this is clearly a large undertaking. Project this out to OOXML’s 6,000 pages and you would need 120,000 test cases.

    One approach to taming this complexity is what we’re doing on the ODF Formula Subcommittee. As we define spreadsheet functions, we’re putting in plenty of test cases, 3, 4 or more per function. This is done in a structured way, with a special named style in the document. We then have a tool that post-processes the ODF XML of the specification, extracts the formula test cases and creates an ODF spreadsheet document that executes each one of the test cases. So we in essence have a self-testing specification, at least in this area.

    Hi Ben,

    The ODF specification explicitly handles the case of what to do when you encounter an extension that you don’t understand. You simply ignore it. OOXML has another way of handling this. They allow the application writing the document to encode a feature multiple ways. For example, you could describe a new bullet list feature using a markup extension understood only by your application and then also provide an alternative rendering in standard OOXML 1.0 to be used if the extension is not understood. The application reading the document processes this section like it was a switch/case/default statement in C. This is a clever approach and probably worth doing something similar in ODF.

    For the case of functional subsets, I still think profiles are the way to go. This pushes the decision making of how to degrade gracefully into the application doing the writing rather than the one doing the reading. Of course you don’t always know in advance whether you are targeting a partial implementation, but when you do, that will give you the best results.

    Anonymous #1, you raise a good point. I’d note that a test suite or a certification program is as much a tool for the consumer or evaluator/purchaser of tools as it is to the vendor.

    Anonymous #2, thanks. What you describe is one of my bigger complaints against OOXML. They have many duplicative ways of describing the same thing, due to their legacy support. So graphics can be done in DrawingML or in VML. A document can contain section described in WordProcessingML, or HTML or RTF or legacy versions of WordProcessingML. Sure, some of this is deprecated or oprional, but a consumer, needs to be able to handle anything. Even the most trivial of consumers, a tool to extract text for indexing for a search engine will need to handle these many alternatives.

    With XML we at least have a good set of parsers that hide us from the complexities of the lexical space and give us a nice clean InfoSet. But we don’t yet have similar levels of abstraction for ODF or OOXML. Of course there is the ODF Toolkit that Sun is starting up at OpenOffice.org. That is another factor that can increase interoperability: wrap the complexity of the format in a tool. If the tool is coded to handle the full complexity of the format and to emit only valid ODF, then those who use the tool will experience enhanced interoperability compared to those who write code from scratch.

  • Anonymous 2007/02/21, 11:08

    An insightful document. One addition would be consider the impact of assistive technologies. Your example of an editor using the paradigm of a printer as the layout or using the web is a good one. If the design point also has to include screen readers, the spec really needs to be tight so that logical order is preserved as well.

  • Anonymous 2007/03/23, 04:44

    If you would like to discover the Test Suites and Validators at W3C. This is a table with the full list of specifications and their related test suites, validators, implementations reports and conformance sections.

Leave a Comment