• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

An Antic Disposition

  • Home
  • About
  • Archives
  • Writings
  • Links
You are here: Home / 2007 / Archives for May 2007

Archives for May 2007

The Legend of the Rat Farmer

2007/05/31 By Rob 11 Comments

The Tale

A long time ago in a land far away there once was a prosperous town called Hamelin. Everything was perfect in Hamelin until the year the rats came. The rats ate up the grain, bit the townsfolk in the toes and scared the young children. Something had to be done! So the Bürgermeister and the Council met together and decided to bring in an outside consultant, Pied Piper Enterprises, LLC. That did not go well. The rats were back the very next year.

So in the Spring the Bürgermeister again assembled the Council and they talked and talked and talked. Should they bring in another consultant? Should they abandon the town and move someplace else? They finally decided on a market-based approach to solving the problem. They would offer a reward, a bounty, to citizens who captured, killed and turned in rats. Turn every person in Hamelin into an exterminator. The signs soon went up all over town: “A Silver Thaler for every 10 Rats.”

The Bürgermeister tracked the results on a big chart on the wall of his office and the numbers looked very good. Each day more and more rats were being caught and killed. The citizens were busy at work. The rats would soon all be gone.

But then one day the Bürgermeister went home, and in the doorway of his house was his wife and she was very upset, “You shall have no dinner tonight! The rats have eaten all of the grain!”

“How can this be?” exclaimed the Bürgermeister. “The metrics show that we’re eliminating a record number of rats every day.  Come with, and I will show you the chart.”

“Chart, schmart. I’ll show you some metrics,” said the Bürgermeister’s wife, who then took him by the ear and led him around the town center, and at each house they stopped and heard the same tale. The rats are still eating up the grain. They are still biting townsfolk in the toes. They are still scaring the young children.

Nothing at all had improved in the quality of life in Hamelin. The only thing that had changed was that they now had a larger pile of dead rats, and a smaller pile of silver Thalers.

An inquest was held to account for the misuse of town funds. During this investigation it was found that a large portion of the reward money had gone to one old man who lived by himself on the outskirts of town. The Bürgermeister and the Council went to visit the old man. “How did you manage to catch so many rats?” they asked, “You are old and slow”.

“Simple,” he said, “Let me show you”. He lead them back around his house to an old barn. As he opened the barn doors, he revealed to the astonished Council hundreds of small wooden cages, each one holding 10 large rats.

“I don’t care for rats much myself”, said the old man. “But since you wanted them so much, I thought I could help out a little. After all, I could use the money, and rats are so easy to breed”.

“Bu…bu…bu…but we didn’t want more rats,” stammered the Bürgermeister. “We wanted fewer”.

“Nonsense”, said the old man. If you offer a reward for something, of course you want more of it, not less. This is just the free market in action.”

The Commentary

We see here the results from failing to specify an appropriate metric. As is often the case, we tend to latch on to metrics that are easy to measure, such as counting dead rats, rather than harder to measure, but more appropriate metrics that truly indicate the achievement of our goals. For example, a reasonable metric might have been a “resident satisfaction index” based on a weekly survey of Hamelin’s citizen’s to see if their rat problems were decreasing. Or the Bürgermeister could have sent out a commission to count how many rats they find in the grain and tracking that number from week to week. The point is to have a metric that clearly and directly reflects the attainment of your goals.

So the lesson is that you should always watch out and ensure that the metrics being suggested truly reflect your ultimate concerns.

With that in mind, let’s move forward to the present and what seems to me a similar confusion of metrics.

Jason Matusow, Microsoft’s Director of Corporate Standards has written a new blog post, which concludes:

The fact of the matter is that translation between formats has always been the path to interop (for document formats), and now with XML-based formats that path is even more appropriate than ever through translation.

China wants to create its own standardized XML format…translation will enable interop. Google Docs has its own format….translation will enable interop. OpenOffice has ODF..translation will enable interop (to MS Office, to Google Docs, to IBM Workspace). Adobe PDF is its own format…translation will enable interop.

Jason seems to be suggesting that increasing the number of different formats and translators leads to an increase in interoperability. This is akin to saying that increasing the number of umbrellas improves the weather. It just doesn’t work that way.

We need to step back and find the proper metric. If, for sake of argument, we define interoperability as the ability for different formats to work together, then obviously as we increase the number of formats and the number of translators then the sum total of interoperability (by that definition) in the world increases. In that case, let’s make the old 1-2-3 format an ISO standard, the WordPerfect format an ISO standard, WordStar an ISO standard, XYWrite an ISO standard, Quattro Pro an ISO standard, Manuscript an ISO standard, Harvard Graphics an ISO standard, Freelance Graphics an ISO standard, etc. Just imagine how much interoperability we could have in the world if we simply could standardize more formats. Every application, could have its own standard format, or maybe two or three.

But you may smell a rat in the above argument. Interoperability of formats is not the appropriate metric. A simple look at the lack of OOXML support on the Microsoft’s Mac Office shows that the introduction of OOXML has reduced interoperability, not increased it. Similarly, scientific journals like Science and Nature have already come out saying that they cannot accept the OOXML format. Translation among multiple formats only partially and imperfectly attempts to work around a break-down in interoperability caused by having multiple formats. It is a band-aid approach and does not address the core issue.

A more appropriate metric than counting piles of semi-functional translators is to look at things from the perspective of the user exchanging documents. The end user doesn’t see or care about formats. They care about their documents and the people and processes that work with these documents. The question for them is: what is the cost to exchange their document with other users and business processes? In other words, what is the cost to interoperate? That is the metric that counts.

Several cost drivers come into play here:

  1. What are the choices and costs in application software necessary to author a document?
  2. What are the choices and costs in application software needed by the recipient of this document, in order for them to read it, or collaborate with me in editing this document?
  3. Will others see the document as I intended? Or will there be fidelity loss from conversions?
  4. Similarly, what are the performance, security, stability, legal and licensing implications of introducing any translation steps?
  5. How easy is it to program this document format? In other words, what is the cost of business process integration?

When looked at from this business perspective, we can get away from counting piles of dead rats and thus come to a quite different conclusion:

None of the cost-driver factors lead to reduced costs with multiple formats. They all have minimal costs when there is a a single format in use. So if the metric for interoperability is the “cost to interoperate”, then interoperability (and choice as well) is maximized when a single application-neutral and platform-neutral document format is natively supported by multiple applications at a range of price/function points. Introducing even a single additional format into your business will escalate costs, degrade fidelity of document exchange, and reduce interoperability.

  • Tweet

Filed Under: ODF, OOXML, Popular Posts, Standards

Interoperability by Design

2007/05/22 By Rob 19 Comments

We’ve all heard the interoperability hype. Let’s see what is actually there.

First, we start by looking at the many ways in which documents are integrated into the Windows/Office platform. Any fluent user of this platform will use many of these capabilities on daily basis. These are basic features which have been around, in some cases, since Windows 3.0, maybe earlier.

Windows shell integration

  1. Double-click on a document on the Desktop or in a folder and it loads into the appropriate application. Double-click on a Word document and it loads in Word.
  2. Right-click in a folder and choose “New XXX” to create a new XXX document in the specified folder. So, “New…Microsoft Office Excel Worksheet” creates a new, blank Excel document.
  3. Right-click on a document, choose Properties and on the Summary tab you can view metadata for that document.
  4. Recently-edited documents appear in the “My Recent Documents” under the Start menu.
  5. Documents referred to in web pages, via URL links will render in an inline Office session in the browser.
  6. Documents are indexed by the Windows search engine.

Office integration

  1. Ability to File/Open, File/Save and File/New a document via the familiar menu options.
  2. Ability to set a file format as the default file format for the application.
  3. Ability to use the familiar keyboard shortcuts, Control-O and Control-S to open and save documents.
  4. Ability to forward a document to someone in an email and for them to be able to launch the a document by clicking on it when received via email.
  5. Ability to password protect a document.
  6. Ability to post a document to a web folder or to a SharePoint server

It must be noted that none of the above integration points are allowed by the ODF Add-in for Word, the much-touted translator for which Microsoft provides the, “Funding, Architectural & Technical Guidance and Project co-coordination”.

Instead what we get is a new menu option added to the Word 2007 Office menu:

Note that this is parallel to, but not included in the Open menu where the formats that Word natively understands are accessed. Although the option presented here says, “Open ODF”, it should more properly be called “Import ODF”, for reasons which will be clear shortly.

After selecting an ODF document to open, the following progress bar is given while the conversion takes place:

This is followed by a warning dialog listing elements which may have been lost in conversion:

No option is given for disabling the above message from displaying. It should be noted that when converting from a legacy binary document to OOXML, Word gives a similar conversion warning dialog, but their version can be disabled by checking a “Do not ask me again” dialog.

Once loaded, the user will find that their document is no longer an ODF document. It has been automatically converted to a read-only OOXML DOCX file as the title bar reveals:

So any future operations the user performs on the document, such as mailing, saving, posting to a web server, etc., will be in OOXML format. The only way to get back to an ODF format file is to manually and explicitly go back to the Office menu, go to the ODF submenu and choose to save it to ODF format. At that point you will be presented a default name based on the DOCX temp file name, not the original name. In this case, it suggested “sampler_tmp1.odt”.

The “Save as ODF…” dialog will default to the directory last used to save a file, not necessarily the same as where your document was loaded from. So to save you must first navigate to your original document, select it and choose “yes” when warned about overwriting an existing document, and then the document is converted back into ODF format.

If you do further work on the document in Word, in that same session, and then want to save again, you must avoid the natural tendency to do a Control-S or to save the document when prompted when existing Word. These methods all will lead to a Save As dialog, suggesting an OOXML format, which will prompt you to rename the document since it is read-only. But it will not offer you the choice of saving to ODF format. The only way to ensure that you are saving to ODF format is to use the above steps, going back to the ODF menu, etc.

You cannot create a new ODF document from scratch in Word. If you try to create a new document and save it to ODF format, you will get an error message, telling you that you must first save the document. You must save the document before you can save it? Yes, you must first save it to a temp file in a natively-supported format like DOC before you can save it as ODF.

The difficulties are complicated when you have documents accessed by other means than the Word menus. Imagine that you receive an ODF document in an email which you want to edit and return to the sender. The following steps would be required:

  1. Manually detach and save your hard drive the ODF document from the email, since you will not be able to launch it directly into Word from your email client. Remember where you detached the document.
  2. Manually launch Word, since you will not to get Word to launch by clicking on the ODF document you just detached.
  3. From the ODF menu, choose to open the ODF document. Navigate to where you detached the emailed document and select it. Around 30 seconds later the document will be automatically converted to an read-only temporary OOXML document.
  4. Make your editing changes.
  5. Export the document back to ODF format using the ODF menu, either writing over the original file you extracted from the email, or to a new temporary file. Remember where you exported the ODF document to.
  6. Go back to your email application and attach the ODF document.

If this had been an OOXML document (or any other format that Microsoft really supports, like RTF) it would have been much simpler:

  1. Double click on the attachment in your email to automatically launch in Word
  2. Make your editing changes
  3. Use the Send/Email menu option in Word to send the email

As you can see the ODF support provided by the Add-in is very unfriendly.

Compare this to the OOXML support Microsoft added for older versions of Word via their Compatibility Pack. The OOXML support is tightly integrated with the UI, in a way users would find familiar and easy to use. But the ODF support is very shallowly integrated, amounting to little more than a menu item patched in.

One wonders if Microsoft’s intent was really to annoy users? That would best explain the available evidence. It is simply not credible that anyone at Microsoft believes that they are listening to customers or providing interoperability with a feature that defies real-world use. What customers did they talk to that said that this Add-in was even remotely adequate?

Since Microsoft is the one providing the, “Funding, Architectural & Technical Guidance and Project co-coordination” one would think that they would contribute more in the area where they are uniquely qualified to assist, the full and native integration of the ODF support into Office.

  • Tweet

Filed Under: ODF, OOXML

The Funnel and the Wedge

2007/05/18 By Rob 12 Comments

The idea was prompted by a comment a reader submitted to a recent post, where he talked about one of the challenges of trying to bridge two different formats (in this case ODF and OOXML) via translation:

Both formats must evolve their new versions simultaneously in locksteps…

This one is the killer. Trying to have two formats permanently synchronized this way is a maintenance nightmare, especially when we discuss standards with multiple implementations maintained by different organizations.

This is an important point, and bears some reflection. In my mind I have images of the funnel and the wedge, physical means of convergence and divergence. Similar forces are at play with standards.

We see the Funnel in the evolution of HTML. Although the standard has existed for over a decade, implementation support for HTML and related standards was uneven until quite recently. Interoperability was poor. From the start vendors added incompatible extensions while not implementing key features. Developers had to write extensive workarounds and alternative representations to work on all browsers. And when they did not that, their web sites might not work with all browsers. But with customer demand and prodding from groups like the Web Standards Group, the interoperable support of the HTML standard across implementations happened. There was convergence. What we have today, although not perfect, is clearly the result of a Funnel, concentrating industry effort around a single standard (or more like a family of standards).

This does not mean that vendors needed to sacrifice innovation, or deny their customers. It just meant that they accomplished their business objectives while also complying with standards. Along with adhering to various financial and securities regulations, labor law, health and safety, and other requirements, both internal and external, voluntary and mandatory, the browser vendors now complied with web standards. It is just another part of doing business.

Similar Funnels have occurred historically with network protocols, wireless telephony (in Europe at least), electrical grids, broadcast formats, etc. I’ve written elsewhere about what types of technologies tend to converge like this and why.

We see the Wedge when two standards compete in the same space and diverge into incompatible technologies. Microsoft is the master of the Wedge, with numerous examples over the years, usually proprietary, but more recently attempting to gain de jure recognition of them. But the mechanism is the same in either case: VML, JScript, MS Kerberos, J++, C++/CLI, XPS, and of course OOXML. Standardization just means that Microsoft has another tool for telling you that the Wedge is good for you. But it is a Wedge nevertheless.

The Wedge brings fragmentation, confusion and lack of interoperability, attacking the core reasons for having a standard in the first place. Once the primary value of an open standard is eliminated, we can all return to the security and comfort of our monopolist overlords. That is their main goal. Make no doubt about it, true interoperability and true choice are very scary propositions for Microsoft. It cuts at their very business model.

So consider the Funnel and the Wedge as applied to document formats. If we all use ODF today, is interoperability perfect? No. Do we know how to move forward to improve interoperability, and work together in multi-vendor consortia to perfect this. Yes, certainly. That is why and how such standards as TCP/IP, HTTP or HTML, work today. Interoperability came via the Funnel, a convergence of effort and attention leading to increased interoperability and the user and industry benefits that flow from that interoperability.

But from the Wedge, what can we expect? If Microsoft is successful, here’s what I see, my dismal predictions:

  1. Within 30 days after OOXML is approved by ISO we see the demise of Microsoft’s half-hearted attempt to create ODF Add-ins for Office. We’ll never see a functional Add-in from them for Excel or PowerPoint, and the Word one will remain unacceptably slow.
  2. Microsoft will continue to evolve OOXML behind closed doors. 99% of the work will be based on product and decisions in conference rooms in Redmond, which will be later rubber stamped by Ecma and ISO.
  3. OOXML and ODF will continue to evolve and diverge, in incompatible ways.
  4. Seeing their success ramming through 6,000 page Fast Track submissions in ISO, Microsoft will follow up with similar fast track submissions for XPS, XAML, Silverlight, Windows Media Photo, whatever they have. Since they have taken the trouble to set up the machinery to dominate JTC1, they will continue to force feed them with additional material.
  5. Every jurisdiction where ODF is currently allowed and mandated will also allow or mandate the use of OOXML. This in practice will be turned around to mandate the use of Windows and Office.
  6. Finally, once all opposition is rendered harmless, they can shut down OpenOffice.org and KOffice by patent lawsuits, but keep Novell’s version around in order to keep anti-trust regulators away. After all, 97% market share is not the same as 100%.

Maybe I’m a bit pessimistic, but I see little reason for optimism.

So do we have an alternative to the Wedge? What would encourage the Funnel? The following would need to happen:

  1. ISO must reject OOXML.
  2. Customers, from private and public sectors, must make their voices heard, that they want true interoperability and choice and that this means a single document format.
  3. Microsoft must support the existing ODF completely and fully in Office. It won’t happen overnight. But it won’t happen at all unless they start.
  4. OASIS must work with Microsoft (and Microsoft with OASIS, of course) so that that it is clearly explained how MS Office can fully represent their documents in ODF. This need not be a monolithic monster like OOXML, but should be a layered standard, with a basic core feature set and defined extensions and profiles that encompass wider and wider ranges of functionality. If Microsoft absolutely needs the “heebieJeebies” Art Border in Word in order to maintain 100% fidelity with legacy documents, then the ODF TC can show Microsoft how to encode this in ODF. The Funnel starts when Microsoft abandons their divergent effort in Ecma and joins the common effort around ODF, a single document format for personal productivity applications.
  5. The application vendors, Microsoft included, must work together on defining the organizational, standards and technical means necessary to measure, test and certify ODF compliance, so customers and procurement agencies are able to have assurances that they are getting the level of interoperability that they desire.

I think this is a natural progression. Accomplishing the first step stops the Wedge from progressing further, halting but not reversing the divergence. The other steps reverse the damage and turn us down a path of true interoperability, leading to true choice and innovation.

Finally note that the Wedge is typically driven by a single company. It is not a pull by public demand or from customers, though it may wear many disguises. It is a deliberate attempt by one party to cause division and divergence. But a Funnel, this won’t happen at all unless there is strong demand, from customers, from government agencies, from national standards committees, etc. If this is to happen, your voice must be heard. All of us must work to bring all of us together in this effort. But it takes just one company, with a sufficiently large Wedge to pull us apart.

  • Tweet

Filed Under: ODF, OOXML

So where are all the OOXML documents?

2007/05/10 By Rob 29 Comments

Google has a nice feature that allows you to search for documents that match a given file type. This is done by adding “filetype:NNN” to your query, where NNN corresponds to the file type. This feature has supported the ODF and OOXML document formats for at least two months, when I first noticed it. I’ve been tracking some numbers since then and now have enough data to make some observations.

At last count the totals were:

Format Count
ODT 85,200
ODS 20,700
ODP 43,400
Total ODF 149,300
DOCX 471
XLSX 63
PPTX 69
Total OOXML 603

As you can see, there is some round-off happening on the upper range. Perhaps at the high-end counts are estimates based on sampling?

In any case, I am rather surprised by the low counts given for OOXML documents, especially considering that this format has been supported since the Office 2007 beta last summer. According to Brian Jones, there have been over 4 million downloads of the OOXML Compatibility Pack for older versions of Office, and that there is a new community of, “over 300 other companies and partners who care deeply about OpenXML”. We’re also told that Office 2007 sales are above expectations, “two times greater than the purchases of Office 2003” according to one research firm. Recently announced third-Quarter results for Microsoft showed “better than expected” results for Office 2007 sales, $200 million better, according to Microsoft CFO Chris Liddell.

So with all this evident love for Microsoft Office 2007, why is it that 6-months later there are only 63 OOXML spreadsheet documents on the web, something like 0.3% of the number of ODF spreadsheet documents? How can there be 300 companies supporting OOXML and only have 69 OOXML presentations on the web? (This is starting to sound like when I say I support 30 minutes of aerobic exercise a day. I don’t do it, but I sure support it!)

OK, I know the argument about “dark matter”, that Google indexes only the tip of the iceberg, that there is a lot of data squirreled away on PC hard-drives, behind corporate fire walls, etc., stuff that Google will never see. But the same is equally true for ODF documents, right? I have tons of ODF documents on my laptop, but none of them are indexed by Google.

Of course ODF has been around for a year longer than OOXML. That’s an important fact to acknowledge. We can put that in perspective by plotting the graph of ODF and OOXML document counts against the number of days since adoption of these two standards. So ODF counts are based on a start of 1 May 2005 and OOXML starting in 7 December 2006, when OASIS and Ecma respectively approved them. You get this:

As you can see, ODF has a nice upward trend. OOXML is also trending upwards, though it is somewhat lost at this scale. If you do the analysis it comes out to around 300 new ODF documents per day versus 6 for OOXML. So, two years later, ODF adoption, in terms of documents per day, is 50-times greater than OOXML is, at a time which should be OOXML’s high-growth period, considering all the great news that is coming out of Redmond.

So I’m a somewhat at a loss to appreciate the significance of Novell or Corel adding OOXML support to their editors. With only 63 OOXML spreadsheets out there, wouldn’t it be cheaper just to hire someone to retype the documents in the destination application? The average user is more likely to find a Buffalo Nickel in their lunch change than to find an OOXML document outside of captivity.

  • Tweet

Filed Under: ODF, OOXML

Primary Sidebar

Copyright © 2006-2023 Rob Weir · Site Policies