An Antic Disposition

You are here: Home / Archives for Standards

Standards

The Piemen of Erie

By 13 Comments

An interesting historical anecdote to relate, from our nation’s industrial adolescence, a tale with relevance today when we discuss standards.

The year was 1853 and the place was Erie, Pennsylvania, a town at the junction of two incompatible rail gauges. This gauge incompatibility was inefficient and frustrating, but the citizens of Erie loved it, and resisted every attempt to join the emerging common standard gauge in what would be called the Erie Gauge Wars. Why? Let’s find out more in the words of industrialist and historian James Ford Rhodes from a passage quoted in extenso from his 1895 History of the United States from the Compromise of 1850 to the McKinley-Bryan Campaign of 1896 (Vol III Pages 18-23). I will follow this by some additional thoughts.

For the prosperity of 1846-57 there were several contributing causes, either special to that period, or then for the first time effective. The greatest of these (whose influence, continuing with ever-increasing momentum to our own day, is still transcendent, and will pervade the future to a degree to which no philosopher can now set bounds) were railroad transportation, beginning its first great era, and the coming into general use of the electric telegraph. We may mark the year 1849 as the commencement of railroad extension. Having less than 6,000 miles January 1, 1849, the country had at the end of 1860 30,635 miles. In 1850 it was impossible to go by direct railway from New York to either Albany or Boston; in 1860 New York had continuous lines reaching beyond the Mississippi. In 1850 Chicago had one short road; in 1860 that city was a great railroad center, her main lines “reaching hundreds of miles — east, west, north, south. In 1850, Ohio, Indiana, and Illinois were open fields; in 1860 they were crossed and recrossed many times.” “I arrived here last night,” wrote Emerson from Pittsburgh in 1851, “after a very tedious and disagreeable journey from Philadelphia, by railway and canal, with little food and less sleep; two nights being spent in the rail-cars, and the third on the floor of a canal-boat.” Not until the end of 1860 did the railway system between the East and the West approach unification and give promise of that consolidation of separate railroads and branches into systems which in our own day has characterized this development.

The primitive ideas in regard to railway travel prevailing in the decade of 1850-60, are well illustrated by what was then known as the Erie Railroad War. The traveler who goes from New York to Chicago in our day is not obliged to set foot from his train, for he is provided with the comforts and conveniences of a hotel. Far different was it in 1853. The traveler could, indeed, then go from New York to Albany in four hours; but there he must change to another road and another train which carried him from Albany to Buffalo, and he esteemed himself fortunate to be able to cover so great a distance in the same car. If he made the western connection at Buffalo it was considered good-luck. The tales of those days are full of complaints of trains behind time, of connections missed, of tedious delays. From Buffalo the traveler had a short run to the station on the line between New York and Pennsylvania, called State Line, where, on account of a difference in gauge, a transfer was necessary. On the broader gauge he could travel twenty miles to Erie, Pa., when he must change again to a road of the Ohio gauge. The train on this railroad carried him to Cleveland; but on the way, if at all late, he was subject again to the anxiety of missing connections. At Cleveland he must hurry to the river, where a scow, carrying at most a dozen passengers and sculled by a weather-beaten mariner, was used as a ferry to take passengers to the Toledo railroad station. In this open boat travelers suffered from exposure to rain and snow; at times the waves ran high and the crossing was attended with danger. If the eastern train was crowded or a few minutes late, haste was necessary to secure passage in the first trip of the scow, for it was well known that the Toledo train started on schedule time and waited for neither train nor boat. At Toledo the traveler made the last change, and — if not more than five minutes behind time — found the Michigan Southern train awaiting him; otherwise he had a tedious delay, which, if his arrival at Toledo happened on a Saturday, might extend to thirty-six hours. The traveler from New York who missed no connections and arrived at Chicago on time had a marvelous story to tell.

The railroad managers of the lines between Buffalo and Erie, eager to improve their route, decided to alter the six-foot gauge of the railway between State Line and Erie to four feet ten inches — the gauge of the roads east of State Line and west of Erie — so that passengers could go from Buffalo to Cleveland without change. The railroad ran a distance through the streets of Erie. The Erie municipal authorities refused to give a permit for making the alteration unless the railway company would agree to stipulations to which its directors, considering them unreasonable, declined to accede. In the contest which followed, a color of law and reason was given to the position taken by Erie; but no one was deceived as to the real ground of the trouble. Erie objected to the change of gauge because the transfer of passengers and freight was deemed important to the borough’s prosperity. The wait involved brought custom to her eating-houses; the transfer of freight and live-stock gave work to her people. The populace ignored the legal points and the pretended grounds of demur, but they keenly appreciated the vital objection.

On December 7, 1853, the railroad company began at State Line the work of changing the gauge. The news came quickly to Erie. A cannon was fired to call out the citizens. A large mob assembled, tore up the track, and cut down the railroad bridge in the borough. The infection spread to Harbor Creek, a Pennsylvania town seven miles east of Erie, and that evening its citizens held an orderly meeting and resolved to remove the track of the railroad running on the public highway. The resolution was the next day carried into effect. Two days later (December 10) the track of the new gauge was completed to the borough limits of Erie. That night rioters at Harbor Creek tore up the track, destroyed the bridge, and ploughed up part of the grade of the road. War had begun in earnest. The mayor and the sheriff at times directed the mob, while the local militia, arrayed for service, swelled its number. Even the governor of the commonwealth seemed to sympathize with the Erie people. Certainly they had the sentiment of the whole of Pennsylvania on their side. The United States Circuit Court then granted an injunction restraining all persons from interfering with the railroad company. An Erie justice of the peace pronounced the injunction null and void, and the populace, believing the later decision to be the better law, refused to respect the order of the court. Two days after Christmas, the Harbor Creek bridge was torn down for the fourth time.

The contest attracted the attention of the country. In Buffalo the excitement was intense. Cincinnati held an indignation meeting presided over by Thomas Corwin, to protest against the conduct of the Erie citizens. The New York Tribune said: “Let Erie be avoided by all travelers until grass shall grow in her streets, and till her piemen in despair shall move away to some other city.” The press of Philadelphia espoused the cause of Erie. The City of Brotherly Love held a large public meeting to express sympathy with the borough at the other end of the commonwealth. It was declared that “the only protection Erie has to prevent her own ruin is to require the break to be made within her boundaries.”

About this time Horace Greeley had occasion to go West. He wrote to his newspaper that he was obliged to ride the seven miles from Harbor Creek to Erie in an open sleigh “through a cutting storm of wind, snow, and sleet…. Let Erie have her way,” he continued,” and all passengers and freight must change cars before her pie-shops…. The whole world is to be taxed, as in the days of Caesar Augustus, in order that Erie may clutch a sixpence for every dollar of expense she imposes on others. Is it strange that so mean and selfish an exaction should be enforced by mobs, arson, devastation, and ostentatious defiance of judicial mandates?”

With the new year the excitement grew. The Erie people became vindictive. They warned the president and director of the railroad company, living at Erie, to leave the borough. Women joined the rioters and assisted in the work of destruction of the bridges. The New York Tribune called upon President Pierce to interfere, and suggested that he issue a proclamation and call out troops in order that the laws might be executed. “Had a runaway negro,” this journal said, “been somehow mixed up with the matter, we should have had half of the United States army in Erie a month ago.” The trouble brought into view the rivalry between New York and Philadelphia, between New York State and Pennsylvania. The Commonwealth of Pennsylvania was declared an accomplice with the Erie rioters and bridge-burners, for the purpose of diverting business from the West to the seaboard through her territory and to her port by a projected line from Erie to Philadelphia, and an appeal was made to the West to frustrate her purpose. It is possible that the sentiment of the West had some influence in bringing about a settlement; but in the early part of January, 1854, the minds of Northern men became engrossed with the proposed repeal of the Missouri Compromise, and the Erie war ceased to attract attention. An act of the Pennsylvania legislature, by a fair compromise, brought the trouble to an end. The railroad company, having consented to confer certain desired advantages upon Erie and Pennsylvania, was allowed to complete the change of gauge and run its trains through Erie without molestation.

So, what do we have here? Two, incompatible standards meet in Erie. Maintaining the two standards is wasteful, inefficient and expensive to the railroad owners, and causes great inconvenience to the travelers who pass through Erie. Everyone knows this. However, the shopkeepers and laborers of Erie benefit from directly from these circumstances. This situation is common. We don’t all benefit from the same things. Some people benefit from other people’s misfortune. War is good for arms dealers. Hurricanes are good for the glazier. Tuberculosis is good for the undertakers. And two incompatible rail gauges are good for those whose livelihood depends on jobs manually loading and unloading goods at the junction, or providing refreshments to those who must wait for connecting trains. In any change, there are winners and losers, those who benefit from maintaining the status quo, no matter how inefficient it is, and those who will benefit from change.

So what do you do? Choose a commercial interest? Line up behind either the rail barons or the teamsters? Not necessarily. In the end the competing commercial interests don’t amount to a hill of beans compared to the interests of the travelers, the consumers and the public at large. They are the ones who were absorbing the cost of maintaining two rail gauges, in shipping costs and frustration, and they are the ones who benefited from the convergence on a single standard. Sure there were two commercial interests vying for supremacy, but there was also a choice that was clearly better for the public.

So whenever anyone tells you that two office document format standards is a good idea, that plugins and convertors can deal with shuffling data back and forth between two formats, ask yourself some basic questions:

  1. Who benefits from having two standards? Who are the “pieman of Erie” that will make money off of the inefficiency that results?
  2. Who has the most to lose if Erie becomes just another railroad town, one of many?
  3. How far will the piemen of Erie go to protect their monopoly? What technological advances and innovations will they prevent or actively destroy in order to preserve their exclusive access to their customers?
  4. And most important, what about the interests of the public at large? What is good policy?

If Microsoft were fighting the Erie Gauge War, they would portray it as a fight between the good townspeople of Erie and the rail barons. The big bad mean old rail barons are forcing their single gauge standard on people who don’t want it. Why not give them the choice of gauges? Only the old broad gauge is 100% backwards compatible with the cars that were designed for it. In fact legislation should be passed for force all railroad towns to support both gauges so everyone can have the choice. Piemen of the word, unite !

That is the distortion you get if you look at a standards war through the narrow blinders of commercial interest. But if you look at the full market impact, the simple economics of it, it becomes a lot clearer. What brings greater efficiency, greater fidelity, greater innovation and lower costs? Having two incompatible document format standards? Or having a single harmonized document format standard? Fighting against economics is like fighting against gravity or the 2nd Law of Thermodynamics. You are going to lose in the end. The piemen of Erie, and their modern counterparts, are on the wrong side of economics, and history,

Those who forget Santayana…

By 14 Comments

It must have passed beneath my radar it when it first was filed in 2004, but it caught my eye recently when Andy Updegrove mentioned it in Chapter 3 of his book-in-progress, The War of the Words. I’m talking about Novell’s November 2004 antitrust complaint against Microsoft, filed shortly after settling an different, OS-related, complaint with Microsoft for $536 million. You can view the second complaint, which I’ll call the “WordPerfect” complaint, here [PDF] on GrokLaw.

What is interesting to me, and why this “old news” is worth talking about, is the analysis Novell made in their complaint of Microsoft’s treatment of document format standards. The concerns of 2004 (or 1995 even) are very similar to the concerns of 2007. Let’s go through Novell’s argument and see where it leads us.

91. As Microsoft knew, a truly standard file format that was open to all ISVs would have enhanced competition in the market for word processing applications, because such a standard allows the exchange of text files between different word processing applications used by different customers. A user wishing to exchange a text file with a second user running a different word processing application could simply convert his file to the standard format, and the second user could convert the file from the standard format into his own word processor’s format. This, a law firm, for instance, could continue to use WordPerfect (which was the favorite word processor of the legal profession), so long as it could convert and edit client documents created in Microsoft Word, if that is what clients happened to use…

This is a good statement of the benefits of an open document standard. Note that Novell is not arguing that the benefit of a standard is to get information in or out of a single vendor’s product, like Microsoft Office. The benefit is that a standard provides for interchange between any pair of word processors.

…Microsoft knew that if it controlled the convertibility of documents through its control of the RTF standard, then Microsoft would be able to exclude competing word processing applications from the market and force customers to adopt Microsoft Word, as it soon did.

Note also that Novell is not complaining here about Microsoft’s control of the binary DOC format (and its many variations). Instead, what Novell complains about is Microsoft’s control over the document exchange format RTF, or Rich Text Format, used in those days to exchange data between word processors. He who controls RTF, controls document exchange, controls vendor lock-in and has the sole means of improving the fidelity of document exchanges.

In fact, Microsoft claimed that RTF addressed this very concern — document exchange in a cross-platform, cross-application fashion, as stated in the introduction to version 1.0 of their self-styled “standard”:

The RTF standard provides a format for text and graphics interchange that can be used with different output devices, operating environments, and operating systems. RTF uses the ANSI, PC-8, Macintosh, or IBM PC character set to control the representation and formatting of a document, both on the screen and in print. With the RTF standard, you can transfer documents created under different operating systems and with different software applications among those operating systems and applications

It should have been obvious at the time that vesting exclusive control of an interoperability interface in a single company was a bad idea. But I guess the world didn’t realize what dealing with Microsoft meant. But we know better now. So why are we making the same mistakes in 2007?

Those who control the exchange format, can control interoperability and turn it on or off like a water faucet to meet their business objectives. I don’t know how many people noticed the language in Microsoft’s press release announcing their sponsored interoperability track at XML 2007 a few weeks ago:

In its approach, Microsoft strives to bring technologies to market in a way that balances competitive innovation with the real interoperability needs of customers and partners.

Let that sink in for a minute. Microsoft is saying that they need to balance interoperability and profit. (Their profit, not yours) They can’t maximize for both simultaneously. They need to trade one off for the other.

Continuing with Novell’s 2004 complaint:

92. The specifications for RTF were readily available to Microsoft’s applications developers, because RTF was the format they themselves developed for Microsoft’s office productivity applications. Microsoft withheld the RTF specifications from Novell, however, forcing Novell to engage in a perpetual, costly effort to comply with a critical “industry standard” that was, in reality, nothing more than the preference of its chief competitor, Word. Indeed, whenever Word changed its own file format, Microsoft unilaterally and identically changed the RTF standard for Windows, forcing Novell and other ISVs constantly to redevelop their applications. In this manner, Microsoft gave Word a permanent, insurmountable lead in time-to-market and made document conversions difficult for users otherwise interested in running non-Microsoft applications. Many WordPerfect users were thus forced to switch to Microsoft Word, which predictably monopolized the word processing market….

So, the RTF standard was just a dump of Word’s features, done when and how Microsoft felt like doing it. As one wag quipped, “RTF is defined as whatever Word saves when you ask it to save as RTF.”

This should sound familiar. OOXML is nothing more than the preferences of Microsoft Office. Whenever Word changes, OOXML will change. And if you are a user or competitor of Word, you will be the last one to hear about these changes. ISO does not own OOXML. Ecma does not own OOXML. OOXML, in practice, is controlled and determined solely by the Office product teams at Microsoft. No one else matters.

Consider that Microsoft has recently proposed over 1,700 changes to the OOXML specification, including fixes that presumably will be made into a future Office 2007 fixpack. Microsoft knows what these fixes will be. The Office developer teams know what these fixes will be. But if you are a competitor of Microsoft’s in this space, do you know what these changes are? No. Microsoft has decided to keep them a secret, claiming that the ISO process allows them to withhold interoperability information from competitors in what they maintain is an “open standard”.

Further, the coding of Office 14 a.k.a. Office 2009 is well underway. Beta releases are expected in early 2008. But are file format changes needed to accommodate the new features being discussed in Ecma? No. Are they being discussed in ISO? No. Are they being discussed anywhere publicly? No.

Is this how an open standard is developed?

My prediction is that the first time anyone hears about what is in the next version of OOXML will be when Office 14 Beta 1 is announced. Other vendors will not hear a word about the format changes until after the Beta 1 is already released. Not even Ecma will hear about the changes until after then.

DIS 29500 is already obsolete, has already been embraced and extended. You just don’t know about it yet. You weren’t meant to know. In fact, pretend you don’t know. Give Microsoft a big head start. They need it.

Further from the Novell complaint:

93. …As in the case of of RTF, Microsoft forced Novell to delay its time-to-market while redeveloping its applications to an inferior standard. Because these standards were lifted directly from Microsoft’s own applications, those applications were always “compatible” with the standards.

And that is the key, isn’t it? By owning the “standard” and developing it in secret, without participation from other vendors, in an Ecma rubber-stamp process, Microsoft rigs the system so they can author an ISO standard with which they are effortlessly compatible, while at the same time ensuring that their products maintain an insurmountable head start in implementing these same standards. There is no balance of interests in OOXML. It is entirely dictated by Microsoft, and voted on, in many cases, by their handpicked committees in Ecma and ISO.

So much for Novell’s complaint from 2004. I’m told that this is still case is suspended as of November, 2007, as the two parties pursue mediation. A status report on that mediation is due to Judge Motz by January 11th, 2008. Maybe we’re hear more then.

Looking at this long history of standards abuse by Microsoft, in the file format arena and elsewhere, I’m drawn to take a broader view of this controversy. It is not really a battle between ODF and OOXML. It isn’t even really a battle between OOXML and ISO. It is, in the end, a battle between having document standards and not having them. Microsoft is trying to dumb down the concept of standards and interoperability to a point where these concepts are meaningless and ineffective. This is not because they want to support standards more easily in their products. No, it is because they do not want standards at all.

Remember, standards bring interoperability, the ability to try out new tools and techniques, the ability to migrate, the ability to chose among alternatives, the ability even to run non-Microsoft products. If standards are meaningless and ineffective, then the incumbent’ vendor lock-in will win every time. At that point, isn’t it convenient for them to have a monopoly in operating systems and productivity applications? This, in my opinion, is the essence of Novell’s 2004 complaint, Opera’s present complaint, and the ongoing file format debate. Microsoft’s monopoly power and the resulting network effects have lead to a relationship with standards where they win by winning, by drawing, or even by cheating so much that they discredit the system.

The Right and Lawful Rood

By 6 Comments

So what do we have here?

Sixteen men, lined up. They seem to be having a good time. Some are older, some younger. A historian of fashion might be able to tell us their relative social status, and perhaps their trade, by looking at their clothing. In the background, three men are observing and comparing notes. To the right is a church, and to the left is the village.

So what are they doing?

Is it an early early depiction of the hokey-pokey (“You put your left foot in…”) ?

No.

Although the scene obviously has some social aspects, the primary activity depicted here is standards development, particularly the historically mandated procedure for determining the linear measurement known as the “rood”, related to the English “rod”, the German “rute” and the Danish “rode”.

This print, from a 16th century surveyor’s manual by Jacob Koebel, called Geometrei. Von künstlichem Feldmessen und absehen, explains the procedure:

Stand at the door of a church on a Sunday and bid 16 men to stop, tall ones and small ones, as they happen to pass out when the service is finished; then make them put their left feet one behind the other, and the length thus obtained shall be a right and lawful rood to measure and survey the land with, and the 16th part of it shall be the right and lawful foot.

From a technical point of view, you might wonder why they didn’t have a standard rule, a metal bar etched with two lines, something tangible which could be carried about and used to calibrate? But who would maintain the standard? And would you trust them? Physical objects may be counterfeited, replaced, shaved, distorted, even stolen. Those who are buying land would like a longer rood, and those selling land would like a shorter rood, so the motivation for fraud is clear.

But the average length of the feet of 16 random men — that is probably not going to change much in a given town, or even across a country. Compared to the logistics required to create, duplicate and distribute a standard rule, the described statistical approach is easier to administer and was accurate enough for the time.

But there is more to it than that. Why didn’t the surveyor just measure his own feet? Or those of his friends? And why require that it be done at church? Why not wherever the surveyor wants to do it?

There must have been something about the process itself, the lining up and being measured, publicly, neighbor beside neighbor, next to the church, that lent it legitimacy. These men are literally voting with their feet.

The transparency of the process is also notable. The rood was determined in public, at the time and place most likely to offer everyone in the town the opportunity to observe. It is hard to cheat with the public watching. Anyone there trying to wear clown shoes or going barefoot would be immediately detected.

Also, it is notable that participation was on an equal basis. No one was able to say, “I am a rich merchant, so I should be allowed to bring 5 pairs of my shoes and line them up in front of me”. And certainly no one could say, “I am the King, the standard is determined by my foot and my foot alone”. This is good, because the variation from King to King would tend to be much greater than the variation from different random samplings of 16 men.

Two Feet, No Feathers

By 20 Comments

We typically use words to communicate, to be understood. That is the common case, but not the only case. In some situations, words are used like metes and bounds to carefully circumscribe a concept by the use of language, in anticipation of another party attempting a breach. This is familiar in legislative and other legal contexts. Your concept is, “I want to lease my summer home and not get screwed,” and your attorney translates that into 20 pages of detailed conditions. You can be loose with your language, so long as your lawyer is not.

But even among professionals, the attack/defense of language continues. One party writes the tax code, and another party tries to find the loopholes. Iteration of this process leads to more complex tax codes and more complex tax shelters. The extreme verbosity (to a layperson) of legislation, patent claims or insurance policies results from centuries of cumulative knowledge which has taught the drafters of these instruments the importance of writing defensively. The language of your insurance policy is not there for your understanding. Its purpose is to be unassailable.

This “war of the words” has been going on for thousands of years. Plato, teaching in the Akademia grove, defined Man as “a biped, without feathers.” This was answered by the original smart-ass, Diogenes of Sinope, aka Diogenes the Cynic, who showed up shortly after with a plucked chicken, saying, “Here is Plato’s Man.” Plato’s definition was soon updated to include an additional restriction, “with broad, flat nails.” That is how the game is played.

In a similar way Microsoft has handed us all a plucked chicken in the form of OOXML, saying, “Here is your open standard.” We can, like Plato, all have a good laugh at what they gave us, but we should also make sure that we iterate on the definition of “open standard” to preserve the concept and the benefits that we intend. A plucked chicken does not magically become a man simply because it passes a loose definition. We do not need to accept it as such. It is still a plucked chicken.

(This reminds me of the story told of Abraham Lincoln, when asked, “How many legs does a dog have if you call the tail a leg?” Lincoln responded, “Four. Calling a tail a leg does not make it a leg.”)

With the recent announcement here in Massachusetts that the ETRM 4.0 reference architecture will include OOXML as an “open standard” we have another opportunity to look at the loopholes that current definitions allow, and ask ourselves whether these make sense.

The process for recommending a standard in ETRM 4.0 is defined by the following flowchart:

So, let’s go through the first three questions that presumably have already been asked and answered affirmatively in Massachusetts, to see if they conform to the facts as we know them.

  1. Is the standard fully documented and publicly available? Can we really say that the standard is “fully documented” when the ISO review in the US and in other countries is turning up hundreds of problems that are pointing out that the standard is incomplete, inconsistent and even incorrect? We should not confuse length with information content. Just as a child can be overweight and malrnourished at the same time, a standard can be 6,000 pages long and still not be “fully documented.” Of course, we could just say, “A standard fully documents the provisions that it documents” and leave it at that. But such a tautological interpretation benefits no one in Massachusetts. We should consider the concept of enablement as we do when prosecuting patent applications. If a standard does not define a feature such that a “person having ordinary skill in the art” (PHOSITA) can “make and use” the technology described by the standard without “undue experimentation” then we cannot say that it is “fully documented.” By this definition, OOXML has huge gaps.
  2. Is the standard developed and maintained in a process that is open, transparent and collaborative? We’re talking about Ecma here. How can their process be called transparent when they do not publicly list the names of their members or attendance at their meetings, do not have public archives of their meeting minutes, their discussion list or document archive, do not make publicly available their own spreadsheet of known flaws in the OOXML specification nor of the public comments they received during their public review period? How is this, by any definition, considered “transparent”? We can also question whether the process was open. When the charter constrains the committee from making changes that would be adverse to a single vendor’s interests, it really doesn’t matter what the composition of the committee is. The committee’s hands are already tied and should not be considered “open.” If I were writing a definition of an open, transparent process, I’d be sure to patch those two loopholes.
  3. Is the standard developed, approved and maintained by a Standards Body? Without further qualifying “Standards Body” this is a toothless statement. As should be apparent right now, not all SDO’s are created equal. Some of the standards equivalent of diploma mills. Accreditation is the way we usually solve this kind of problem. Ecma’s Class A Liaison status with JTC1 is not an accreditation since their liaison status has no formal requirements other than expressing interesting in the technical agenda of JTC1. In comparison, OASIS needed to satisfy a detailed list of organizational, process, IPR and quality criteria before their acceptence as a PAS Submitter to JTC1/SC34. Why bother having a requirement for a Standards Body unless you have language that ensures that it is not a puppet without quality control?
  4. Is there existing or growing industry support around the use of the standard? Again, very vague. A look at Google hits for OOXML documents shows that there are very few actually in use. My numbers show that only 1 in 10,000 new office documents are in OOXML format. But I guess that is more than 0 in 10,000 that existing last year. But is this really evidence for “growing industry support”? I’d change the language to require that there be several independent, substantially full implementations.

There are two additional questions which I won’t presume to answer since they rely more on integration with internal ITD processes.

We learn lessons and move on to the next battle. Just as GPLv2 required GPLv3 to patch perceived vulnerabilities, we’ll all have much work to do cleaning up after OOXML. Certainly JTC1 Directives around Fast Tracks will need to be gutted and rewritten. Also, the vague and contradictory ballot rules in JTC1, and the non-existent Ballot Resolution Meeting procedures will need to be addressed. I suggest that ITD take another look at their flowchart as well, and try to figure out how they can avoid getting another plucked chicken in the future.

My comments on the ETRM 4.0 draft

By 12 Comments

This was my response to the call for public comments on the Information Technology Division’s (ITD) Enterprise Technical Reference Model (ETRM) 4.0 draft.

I’d like to write to you as a long-time Massachusetts resident and taxpayer. My employer (IBM) will likely submit their own comments, but I’d like to offer you my own personal views on the ETRM 4.0 draft.

I am proud of the Commonwealth’s tradition of openness in government, enshrined in our Public Records Law and Open Meeting Law. As James Madison wrote, “A popular government, without popular information, or the means of acquiring it, is but a prologue to a farce or a tragedy. A people who mean to be their own governors must arm themselves with the power which knowledge gives them.” So access to government documents, now and for posterity, is critical for public oversight and participation in government, as well as for preserving our heritage. Now that we’ve moved into the digital age, access to government documents requires that these documents be made available in a format that all Commonwealth residents can read. So the move toward open documents formats, as called for in the ETRM, is laudable. A citizen must never be dependent on any single vendor for the software needed to read their government’s documents.

However, I am concerned at the proposed addition of Ecma Office Open XML (OOXML) to the list of acceptable document formats. As you may have heard, OOXML is currently undergoing review by ISO/IEC JTC1 for possible approval as an ISO standard. As part of this review, technical committees in standards bodies around the world are reviewing OOXML and appraising it’s suitability as an International Standard. As a participant in the US committee reviewing OOXML, INCITS V1, I had the opportunity to review the text of the OOXML specification and to discuss it with others. I am sorry to report that I found the OOXML specification to be full of errors and omissions. Of course, no technical document is perfect. But this one, in particular, is of far greater length (more than 6,000 pages) and of far lower quality than any I have seen before. If it has advanced this far in the ISO process it is because of vendor pressure, not because of technical merit.

What is the problem with a buggy standard? Interoperability suffers. That is the problem. There is no doubt that if everyone in the Commonwealth used Microsoft Office 2007 on Windows Vista, that their interoperability will be good. But as soon as we admit choice in applications and operating systems, then interoperability will only occur when all sides follow a common standard. So the technical quality of a standard (accuracy, comprehensiveness, level of detail, consistency, etc.) is directly proportional to the level of interoperability achievable and the cost to achieve it.

The ISO ballot on OOXML will not end until September 2nd, after which a resolution process to fix defects in the text of the standard will take at least an additional 6-18 months. That is, of course, if OOXML gains ISO approval, something which is not certain at this point. So I would recommend a cautious approach, and wait for the ISO process to conclude, or conduct your own independent technical evaluation of the OOXML specification to confirm its technical quality before adding OOXML to your list. Ask other vendors: Is this something you can implement? Ask yourself: Will this truly give the Commonwealth the interoperability and choice that you desire? These are important questions to ask.

Finally, I’d note that the ETRM also calls out OpenDocument Format (ODF) as an acceptable format. ODF was approved by ISO last year. So why do we need OOXML? I personally think that the complexity of document exchange and translation in a multi-format world would take us back to the confusion and frustration of the early 1990’s when we all juggled WordStar, WordPerfect, Word and WordPro files, and could collaborate only poorly. Better to push for a single unified/harmonized standard document format for personal productivity applications, much as we have a single standard (HTML) for web pages.

I’ll leave you with a quote from Tim Berners-Lee, the inventor of the web, from an interview he gave with David Berlind from ZDNet when Berners-Lee was recently in Boston receiving a Lifetime Achievement Award from the Massachusetts Innovation & Technology Exchange.

Berners-Lee said:

It was the standardization around HTML that allowed the web to take off. It was not only the fact that it is standard, but the fact that it’s open and the fact that it is royalty-free.

So what we saw on top of the web was a huge diversity and different business which are built on top of the web given that it is an open platform.

If HTML had not been free, if it had been proprietary technology, then there would have been the business of actually selling HTML and the competing JTML, LTML, MTML products. Because we wouldn’t have had the open platform, we would have had competition for these various different browser platforms, but we wouldn’t have had the web. We wouldn’t have had everything growing on top of it.

So I think it very important that as we move on to new spaces … we must keep the same openness we that had before. We must keep an open internet platform, keep the standards for the presentation languages common and royalty free. So that means, yes, we need standards, because the money, the excitement is not competing over the technology at that level. The excitement is in the businesses and the applications that you built on top of the web platform.

I believe we want to ensure the same qualities in document formats. We want competition and choice among vendors, applications and services, but not among standards. If we compete on standards, then no one wins.