Some thoughts for Document Freedom Day 2008.
Back a few weeks ago in Geneva, OpenForum Europe hosted an evening of mini-talks and a discussion panel with various well-known personalities in our field: Vint Cerf, Bob Sutor, Andy Updegrove and Håkon Lie. I wasn’t able to comment on the event at the time, due to my self-imposed blog silence that week, but I’d like to take the opportunity today to carry forward one of the topics discussed then.
I’d like to take as my launching point the theme of Andy Updegrove’s talk, which was “Civil ICT Standards”. Andy treats this subject more fully on his blog, and also speaks to the topic in his taped interview with Groklaw’s Sean Daly.
Thus spake Updegrove:
But as the world becomes more interconnected, more virtual, and more dependent on ICT, public policy relating to ICT will become as important, if not more, than existing policies that relate to freedom of travel (often now being replaced by virtual experiences), freedom of speech (increasingly expressed on line), freedom of access (affordable broadband or otherwise), and freedom to create (open versus closed systems, the ability to create mashups under Creative Commons licenses, and so on.
This is where standards enter the picture, because standards are where policy and technology touch at the most intimate level.
Much as a constitution establishes and balances the basic rights of an individual in civil society, standards codify the points where proprietary technologies touch each other, and where the passage of information is negotiated.
In this way, standards can protect – or not – the rights of the individual to fully participate in the highly technical environment into which the world is now evolving. Among other rights, standards can guarantee:
- That any citizen can use any product or service, proprietary or open, that she desires when interacting with her government.
- That any citizen can use any product or service when interacting with any other citizen, and to exercise every civil right.
- That any entrepreneur can have equal access to marketplace opportunities at the technical level, independent of the market power of existing incumbents.
- That any person, advantaged or disadvantaged, and anywhere in the world, can have equal access to the Internet and the Web in the most available and inexpensive method possible.
- That any owner of data can have the freedom to create, store, and move that data anywhere, any time, throughout her lifetime, without risk of capture, abandonment or loss due to dependence upon a single vendor.
Let us call these “Civil ICT Rights,” and pause a moment to ask: what will life be like in the future if Civil ICT Rights are not recognized and protected, as paper and other fixed media disappear, as information becomes available exclusively on line, and as history itself becomes hostage to technology?
This rings true to me. Technology, computer technology in particular, now permeates our lives. We interact with it daily, from the moment the internet-radio alarm clock goes off, until days end, when we check our email “one last time” before going to bed.
Similarly, the standards that define the interfaces between these devices are also of increasing importance. There was once a time when standards dealt only with the “infrastructure”, the stuff in the walls and under the panel floor, or in that funny little locked door off the hallway, with all the cables and flashing lights, where strange men with clipboards would occasionally emerge, accompanied by a poof of cold air and the buzzing of machines.
But today, the technology and the standards that mediate the technology are now directly in front of your face. Think MP3 players. Think DVD’s. Think DRM. Think cellular phones. Think web pages. Think encryption. Think privacy. Think documents. Think documents-privacy-security-DRM, your data and what you are allowed to do with it, and what others are allowed to do with it, and whether you control any bit of this in this mad world of ours.
Between you and the tasks that want to do today stands technology and the standards that mediate that technology. Standards are damn important.
Now, although the reach of technology and ICT standards has progressed over the years, the organizations and the processes that create these standards have not always kept up. In many cases standardization remains the creature of big industry with little or no consumer input. It is back-room discussions, where companies connive to see how many patents of their own portfolio they can encumber the standard with. A successful standard is one where no major company is left hungry. Consensus means everyone at the table has been fed. That is the traditional world of technology standards. It brings to mind the famous line from Adam Smith:
People of the same trade seldom meet together, even for merriment and diversion, but the conversation ends in a conspiracy against the public, or in some contrivance to raise prices — The Wealth of Nations (I.x.c.27)
Luckily, there is some hope. The proponents of “open standards” seek standards based on principles of open participation, consensus decision making, non-profit stewardship, royalty-free IP, and free access to standards. The web itself, with the underlying network protocol stack, HTML family of formats with DOM and scripting API’s is a shining example of what open standards can accomplish. Tim Berners-Lee says it best, in his FAQ’s:
Q: Do you have had mixed emotions about “cashing in” on the Web?
A: Not really. It was simply that had the technology been proprietary, and in my total control, it would probably not have taken off. The decision to make the Web an open system was necessary for it to be universal. You can’t propose that something be a universal space and at the same time keep control of it.
But it is important to realize that “control” mechanisms in standards go well beyond IP and organization issues. There are other important factors at play, and we need to address these as well. Knut Blind discusses some of these issues a section called “Anti-Competitive Effects of Standards” from his The Economics of Standards (2004).
The negative impact of standards for competition are mostly caused by a biased endowment with resources available for the standardization process itself. Therefor, even when the consensus rule is applied, dominant large companies are able to manipulate the outcomes of the process, the specification of the standard, into a direction which leads to skewed distribution of benefits or costs in favor of their interests.
In other words, participation in standardization activities is time consuming and expensive, and large companies are much more able to make this kind of commitment than small companies, organizations or individuals. So ,large companies rule the world.
This is especially true with standardization at the international level, where decisions are often made at meetings in very expensive international locations. JTC1 is still discussing what technologies would be required to allow participation in meetings without travel. (Hint — its called a “telephone”) To put this in perspective, my week in Geneva cost $3687.52. I flew coach, ate most of my meals on the cheap, often just grabbing hors d’oeuvres at receptions, and I received negotiated IBM corporate rates for air and hotel. This is one JTC1 meeting. What if I wanted to be really active? Add in two SC34 Plenary meeting (Norway/Kyoto). Add in JTC1 Plenary meetings. Add in US NB meetings. Add in US NB membership fees, consortium fees, conferences, etc. This starts adding up, around $40,000/year to participate actively in tech standards, and this doesn’t include the cost of my time.
How many small companies are going to pay this amount? How many non-profit organizations? How many individuals? Not many.
But in spite of the expense, in spite of the large company bias of the international standardization system, I saw reason for hope at the Geneva BRM. I saw younger participants, with fire in their bellies. I saw FOSS supports from developing countries. I saw Linux on laptops. I saw participants from FOSSFA, SIUG, EFFI, ODF Alliance Brazil, COSS, etc. They joined their NB’s, participated in their NB debates and were appointed to represent their countries in the BRM.
Sure, it is only a foot in the door. One in five BRM participants were Microsoft employees. But it was a hopeful sign. We’ve planted the seed. We must plant more. And we must see that they grow.
Strength in standards participation comes with time, with participation, with networking, with learning the rules (written and unwritten) learning from others, etc. Just as we have FOSS experts in the software engineering, in law, in business, in training/education, we also need experts in standardization. Certainly the bread and butter participation will be from individual engineers, participating for the duration of a particular proposal or group of proposals. But we also need the institutional linchpin participants, those who have taken on leadership positions within standards organizations, and whose influence is broad and deep.
FOSS also needs a standards agenda. In a world of patent encumbered standards controlling the central networks, open source software dies, and dies quickly. We must protect and grow the open standards, for without them we cease to exist.
What standards are important? Which demand FOSS representation? Remember just a few weeks ago, when there was a lot of concern about how the DIS 29500 BRM added explicit mention of the patent-encumbered MP3 standard, but failed to mention Ogg Vorbis at all? Although I sympathize with this concern, the fact is the BRM could not have added Ogg Vorbis at all, because it is not a standard. Are we willing to do more than lament about this? I tell you that if Ogg Vorbis had been an ISO standard it would have been explicitly added to OOXML at the BRM. Are we willing to do something about it?
What are the standards critical to FOSS, and what are we doing about it? What standards, existing or potential, should we be focusing on? I suggest the following for a start:
- Ogg Vorbis
- Ogg Theora
- PNG, ISO/IEC 15948
- ODF, ISO/IEC 26300
- PDF, ISO 3200
- Linux Standard Base (LSB), ISO/IEC 23360
- Most of the W3C Recommendations
- Most of the IETF RFC’s
I’m sure you can suggest many others.
Let’s put it all together. Some ICT standards directly impact what we can do with our data and our digital lives. These are the Civil ICT Standards. We need to ensure that these standards remain open standards, so anyone can implement them freely. However, the standardization system, both at the national and international levels is biased in favor of those large corporations best able to afford dedicated staff to work within those organizations and develop personal effectiveness and influence in the process. Showing up once a year is not going to work. If FOSS is going to maintain any level of influence in formal standardization world, especially at the high-stakes international level, it needs to find a way to identify, nurture and support participation of “Open Standards Activists”. The GNOME Foundation’s joining of Ecma, or KDE’s membership in OASIS are examples how this could work. Umbrella organizations like Digistan also are critical and can be a nucleus for standards activists. But what about taking this to the next level, to NB membership? Another example is the Linux Foundation’s Travel Fund, designed to sponsor attendance of FOSS developers at technical conferences. Imagine what could be done with a similar fund for attendance at standards meetings?
So that is my challenge to you on this first Document Freedom Day. We’re near the end of what promises to be one of many battles. The virtual networks of the future are just as lucrative as the railroad and telephone networks of the last century were. These include the network of compatible audio formats, or the network of IM users using a compatible protocol, or the network of users using a single open document format. If FOSS projects and organizations want to secure the value for their users that comes from being part of these networks, then FOSS projects must encourage the use of open standards, and must also encourage and nurture new talent for the next generation of open standards activists.
I’m looking forward to the day, soon, when I can search Google for “open standards activist” and not find a paid Microsoft shill among the listings on the first page.
Aren’t some of these going to be an uphill battle? Ogg Vorbis will be argued as a competitor to the already ISO-standardized MP3.
I don’t know if GIF is standardized. PNG might be an easy win, but it depends on quite a few things.
–BK
Actually I wonder how long it’ll take to declare ISO approval irrelevant for IT standards.
How many users of ISO 7498 do you know? How many users of ISO 8613?
ISO is slowly turning into machine which develops stillborn specifications as often as useful ones. Once it’ll become apparent that ISO approval does not mean anything we’ll be left with W3C and IETF.
ISO 29500 will be just next in a line.
Actually it looks like Microsoft will like this outcome just fine: if ISO standards are optional (and W3C/IETF standards are optional more-or-less by definition) Microsoft will be free to lobby it’s own “de-facto standards”.
But past history shows that even if Microsoft will be stopped it’ll not change anything. The approach that standard can be accepted without reference implementations (two different interoperable ones, of course – or else why this standard is even needed?) clearly does not work…
PNG is ISO/IEC 15948. PDF is ISO 15930 and ISO 19005. W3C and IETF already recognized as standards by ISO (both ISO/IEC 26300 and proposed ISO 29500 include links to it). So most standards are already accepted as such. Ogg Vorbis is not recognized and it shows fails in process, not in Ogg Vorbis: while products from many vendors happily support Vorbis (MP3-payers, DVD-players, etc) it looks like future ISO 29500 will only be supported by a single vendor and then not so good.
Standards (and software) are like cats.
You get pedigree cats. Go to any cat show, find a breeder, put your money down, go home with a cat and a birth certificate. And a warning not to reproduce without permission; if you want to allow the cat to reproduce, go back to the breeder, put more money down, and a suitable mate will be found.
You also get stray cats. Just wander about the lanes until you find a sign saying ‘kittens’. Go in the door, one jumps on you, if you can give it a good home you can take it. No money needed. And it will reproduce, if at all possible, whether or not it gets permission.
Free software won’t die. It will thrive inside corporations, it will thrive inside academia, and it will thrive inside private homes.
It may have trouble getting from one to the other, and it may have trouble being deployed in commercial products of corporations.
But we have an Internet now. And the flame cannot be extinguished.
The cat is out of the bag, and it will not be recaptured.
Go Symphony. Make Lotus SmartSuite obsolete. Bring on the next generation !
As Victor points out, some of these are already ISO standards, like PNG. This is a good start. I think the important thing is to take a portfolio approach — what is the set of open standards necessary for FOSS applications to thrive, the core interfaces, formats and protocols, especially the ones with network effects? What are we doing to ensure that these standards are formalized?
ISO standards ARE irrelevant – w3c standards are much more interesting to web people nowadays.
The open multimedia standards will gain ground eventually IMHO due to peoples disgust with DRM.
Its really only in ODF vs MSXML that this matters – due to the awarding of government contracts for state/school/uni desktop setups.
In my work as a translator, I receive documents from various companies every day, and a unified doc standard would certaily help.
I’ve been on linux/open office for years now, and there are problems of interoperability with MS Word, although noone ever batters an eyelid (as MS Word versions are also internally incompatible).
GO ODF! :D
What is the set of open standards necessary for FOSS applications to thrive, the core interfaces, formats and protocols, especially the ones with network effects?
Good question, but very hard to answer. For example: what is “standard video format” in FOSS world? Everyone have different opinions, but the thing which a lot of people perceive as “today’s standard” is Matroska file with H.264 video, Vorbis audio and VSFilter or SSA subtitles. One out of four is ISO standard, while two more have at least specifications where VSFilter only have documentation in a form of reference implementation and specifications for SSA can not be found on original site anymore! Does it mean you’ll have trouble trying to play such “standard” file? Not at all: VLC, MPlayer or XINE will play such a file nicely. Heck: cheap DVD-players can play them! And if we’ll try to make ISO standard out of all that we’ll spend a lot of money and time… what for? What exactly we’ll get in exchange?
Note: not all FOSS people with perceive this set as “a standard”: many don’t care about subtitles, for example. Others will prefer AC3 audio and so on.
It’s hard to decide what to standardize on because by the time ISO will give stamp of the approval to the format people will actually use something else… Just released Dirac, for example. Who knows?
Power —
Back when I was doing standards work with JEDEC [1] we had a saying that the real power in the committee rested with those who did the work. Truer words never spoken.
There were big companies and small ones, but regardless of the fact that membership officially resided with the company you knew that the real power was with the people who came with armloads of hard work product. Over the years we would lose some of those and gain others, and the power in the committee shifted visibly as a result.
The moral? If you want to influence a real technical committee [2], be prepared to sweat. Nobody wants to hear about “should” or “ought to.” Bring simulations, bring reference implementations, bring cross-reference tables, bring indexed lists of self-contradictions in the drafts, bring mathematical proofs that the fool thing only works by violating known theorems. Whatever you do, bring detailed proposals for draft changes in professional “from/to” form.
A whine is heard for a moment, but sweat stains end up in the final work.
[1] For those who care, IBM was always a wonderful member: big contributor, never anything but courteous and respectful. Take a bow, Rob, you’ve got good people on the team.
[2] Recent events are an aberration. This kind of power play isn’t sustainable for reasons we’ve already seen.
“In the real world where people are imperfect, you can’t have a standard with just a spec–you must have a super-strict reference implementation, and everybody has to test against the reference implementation. Otherwise you get 17 different “standards” and you might as well not have one at all.”
http://www.joelonsoftware.com/items/2008/03/17.html
Joel has correctly identified the problem with many Internet standards, in the above, namely that we lack good enough tests for compliance. We often have to take the word of Microsoft at al that their software “supports” a standard.
I hope some of these Open Standards Activists can help by producing test suites.
One topical example is OOXML. If the DIS29500 standard is accepted, I dream of a a huge test suite, written by volunteers each taking a page of the standard in a similar way to Rob’s work here. They would write test cases using the most extreme XML possible that follows every dot and comma of the standard. Then we can really see whos software supports OOXML accurately, and it might not be Microsoft’s – giafly.
Where does ECMA go from here. Are there any chances of reforming it or will it be business as usual.
After seeing this OOXML fiasco don’t the other ECMA members feel any remorse for having voted for it or are they waiting for their turn to use ECMA.
Gopal
Then we can really see whos software supports OOXML accurately, and it might not be Microsoft’s – giafly.
What will it change? It took 10 years for Microsoft to implement CSS1 correctly. MS IE still intentionally violates HTTP (the fact is even written in the hall of shame) but it does not change anything: MS IE is still #1 browser.
Microsoft’s position is simple: Standards compliance? What is it? That’s for dweebs. We define the standards, everyone else follow. Technical decisions should not be hampered by standards compliance, that’s purely marketing problem: marketing should make sure that we have all needed “standards compliant” stamps.
POSIX, TCP/IP, HTTP, Java… Everything Microsoft touches Microsoft perverses. There are only one solution to this problem: MICROSOFT MUST DIE. Some products can be salvaged, Microsoft does some good too, but MICROSOFT MUST DIE. Till that happens standards will not matter much…
OGG Theora is a relic from the past. A refurbished hand-me-down old codec from On2.
Modern codecs like MPEG, Divx, On2 VP6, VC-1 and h.264 are much much more efficient.
It is laughable to even talk about the format as a serieus format.
Hey Rob
Nice piece. On the subject of civic standards I think I should point out (from South Africa) that the reason why the SA mandate was so strong – in Geneva and elsewhere – is that our national body subcommittee is made up of primarily of non-corporates. And we voted No this morning.
Sure IBM and MS are there (but as wholly owned foreign subsidiaries we are thinking of downgrading you all). But mostly we are south african small companies, ngo’s and government (which does its best to represent the interests of all our citizens). We are far from perfect but we can say that our people spoke this morning when we rejected this ridiculous proposal of a standard.
I really like the cat metaphor. You are right on the button.
Steve Biko once said “I write what I like”. And that includes schema and computer programs. If the US folk don’t like it then thats just tough shit. Its not technology, its grammar. And we’re talking.
Regards
Bob
I think the US folks do ‘get it’.
Much open-source software has its roots in US universities; ‘BSD Unix’ is Berkeley, i.e. University of California; and in the present day it’s Massachussetts Institute of Technology who are determined to bring us the One Laptop Per Child.
It turns out that the Regents of the University of California dedicate their work to the citizens and businesses of California; i.e. the BSD licence is an attempt to return value to the taxpayer who funded the development.
Much proprietary software has its roots in US corporations; ATT Unix and Microsoft Windows being prime examples.
And the US Government cannot hold copyrights; a copyright is a private right, the government is public; so if they publish software (as NASA does from time to time) then it will be open source.
US corporations vary in their attitude to open source. ATT (of Unix fame) now deploys a lot of Linux in their telco and broadband operations. Apple OSX is a proprietary layer on top of BSD Unix; Apple HQ is just across the bridge from Berkeley in California.
So the story will run and run. The interesting part is how the new growth areas in the world … Brazil, Russia, India, China … will make their mark on it.
The Microsoft-Windows-and-Microsoft-Office ‘trip’ has been a wild excursion in the direction of ‘software you do not have the source for’, lasting for about 20 years. I think things are returning to an equilibrium now.
Commercial squabbles, scientific progress, or some of both ?
Well, I would say of the important sections of FLOSS, the piece that should be worried about is the X server. I look at the multiple compliant sectional nature of the FLOSS software stack and that is the keystone. If someone were to take away X, or even to try and pervert it (via bad includes or retarded patents or what have you), that could easily destroy not just the FLOSS stack, but a good chunk of UNIX in general…
X is more than 20 years old, so if there are any patents in the base implementation, they have expired by now.
The purpose of patents isn’t to destroy; it’s to accelerate commercialisation. Something along the lines of “You have a patent, I have a patent, let’s get together and make a product we can sell”. Like “You’re a girl, I’m a boy, let’s get together and make a family we can raise.”
A collaborative force, if you will.
It’s noticeable that the commercial Linux distributors … Novell and RedHat … are no longer distributing MP3 playing software with their ‘consumer’ Fedora and OpenSuSE offerings. I imagine that’s because ‘consumer’ distribution doesn’t generate any revenue to pay the MP3 patent licence holder (Fraunhofer Research or Alcatel-Lucent, depending on which version of history you believe).
Can I get a one-user patent licence from the rights owner for a sensible ‘domestic use’ price ? Anything under a dollar would be reasonable. Where do I send the check ?
And what’s the consequence of not being able to ? If MP3 is an inaccessible standard for me, it will force me to find an alternative.
Dark Phoenix, funny how you’ve chosen EXACTLY the right sample.
If someone were to take away X, or even to try and pervert it.
“Someone” tried. Two times. First X Consorcium changed license for X11R6.4 – and free software community rejected it. Later XFree86 changed license and X.Org was born.
via bad includes or retarded patents or what have you
Bad includes are not a problem (see above), retarded patents are always a problem – standarts don’t really help (see Microsoft’s MP3 woes).
Since any pile of shit can be declared “an international standard” if you have enough money (see OOXML) I see absolutely no reason to spend time and money for useless certificates (that’s what ISO standards became).
The purpose of patents isn’t to destroy.
That’s how they work in practice. For one case where patent helped something (may be RCU can be declared as such a case) you have ten or hundred cases where patents were used for extortion.
Something along the lines of “You have a patent, I have a patent, let’s get together and make a product we can sell”. Like “You’re a girl, I’m a boy, let’s get together and make a family we can raise.”
This is perversion of the original idea. Original idea for patent was to make it profitable to disclose trade secrets: you can publish the description of creation process for your crown jewels – and no one will be able to use for twenty years without your permission.
And software already has such protection: copyrights.
As for “Open Standards”…
Free software community MUST strive to achieve and improve interoperability. That’s noble goal. Formal certificates for standards are NOT A GOAL. It does not improve anything by itself, it can be bought without helping interoperability – in short: it’s mostly useless waste of time and money.
Some good interoperability work happens in ISO and even ECMA: ISO C++ 2009, ECMAScript 4, etc. We should help these folks if we can. But in general standards are just paper (and not even that today), nothing more.
chris ward: It is true that works of the United States government are not eligible for copyright. However, the government *may* hold copyrights assigned to it by private authors. IMO, the more insidious problem with government and copyright (one more common in state and local jurisdictions) is when laws incorporate copyrighted — and expensive — standards documents by reference. This is probably most well-known with building codes. This creates a de-facto copyright on the law itself.
Well. big businesses are starting to think about ‘give-back’. Here’s some from Hewlett-Packard Physics Education Technology, which I’ll applaud even if I do work for a competitor.
I wish I knew why they are doing it. I hope it’s an attempt to push some resources the way of the ‘next generation’; a kind of counterweight to the copyrights and patents which are an attempt to make the ‘old guard’ rich at the expense of the youngsters.
So not everyone’s greedy.
Will it facilitate progress ? Who knows.
“X is more than 20 years old, so if there are any patents in the base implementation, they have expired by now.”
That would have been my assumption a few years ago, but looking at some of the odd patents the USPTO has been handing out lately, I worry a lot about that being a thorn in the side of FLOSS. You don’t think if Microsoft could get away with patenting the idea of something like the X server they would, and then license it to Apple and try to destroy Linux and BSD that way?
Let me put it this way; there is a US patent on the peanut butter sandwich. That’s at least 50 years old, if not older (I’d bet on older).
Rob, please add to your list of expenses standing as a barrier to public participation the cost of acquiring copies of the relevant standards and rules themselves.
E.g., when one begins to navigate the incredible number of ISO, IEC, and JTC 1 documents necessary to evaluate and make informed comments on draft standards, the cost of obtaining all needed documents can work as an absolute bar to participation by those without lots of shekels.
For example, ISO/IEC:29500-2008 incorporates by reference a host of other standards and even the JTC 1 glossary of terms used in standards development work has a hefty price tag on it.
I am constantly finding that the custom of standards development organizations deriving substantial revenue from their published documents blocks me from performing needed research to inform myself in evaluating standards.
Where this becomes outrageous is when governments adopt such standards as national technical regulations or as technical specifications for procurement. The documents incorporated by reference into such standards are effectively transformed into law, but their availability is limited as a practical matter to large companies and government agencies who can pay the bills for their own access to such information.
Such standards also pose yet another barrier to small companies implementing such standards. Patents and royalty requirements are not the only intellectual property/financial barrier to implementation of many standards. SDO claims of copyright to standards are another.
I have pondered this issue over the years and, reluctantly, I see no solution other than requiring that standards being considered or adopted as government technical regulations or technical specifications must be freely available and without charge, including all standards and other documents incorporated by reference.
I leave it to others to decide whether governments should reimburse SDOs for the lost revenue. But at least in the U.S. federal scheme of things, government laws, regulations, and other government documents must be available to the public at no higher cost than the cost of reproduction (and agencies must waive even copying fees when doing so is in the public interest) and cannot be subject to copyright.
I believe that any serious standards reform must address all costs of standards development participation and implementation. I.e., the manner in which SDOs derive their revenues must be tweaked.
Quoth Paul Merrell (“Marbux”):
Where this becomes outrageous is when governments adopt such standards as national technical regulations or as technical specifications for procurement. The documents incorporated by reference into such standards are effectively transformed into law, but their availability is limited as a practical matter to large companies and government agencies who can pay the bills for their own access to such information.
OK, IANAL. However, I do recall case law (IIRC in Texas) to the effect that copyrighted works which are incorporated by reference into the law lose protection. How this would play in other countries is anyone’s guess, of course, but might be worth investigation.