Archives for February 2009

Whether ODF will wither or weather
depends on us as we work together.

The question is where we should go: whither?
The answer is clear at once.
The question of “whither” is not so dense,
and is easy to answer when we start with “whence?”.

Of the topic today
I will no longer delay nor dither to say
whether we will whither or weather
but will now give you my 2-cents.

Rob’s ODF-Next Rant

1. The word processor and spreadsheet, as we have them today, are relics of the 1980’s, designed when the web did not exist and collaboration occurred predominantly by exchanging paper documents. If we were designing a document author and collaboration system to meet modern circumstances and capabilities, it would likely bear little resemblance to Word. So the question is how much do we let the sunk costs of yesterday continue to determine our future? How much longer do we paint speed stripes on a horse and pretend that it is a racing car?
2. Products like Word and Excel have evolved via the uncritical accretion of functionality over the past decades to a point where the products are overly complex resource gluttons with a knack for having a critical security flaw reported in them every other week.
3. Increasingly users are getting work done via email, wikis and blogs rather than using heavy-weight document editing solutions. Why is this so? Why is the modern word processor losing users rather than gaining them?
4. WYSIWYG is a fine paradigm if you are doing all of your work targeting printed output. But it is a sub-optimal approach for creating documents for almost any other use.
5. The revered Bold, Italics and Underline icons, along with the font selection drop down list, which define the modern editor GUI, should be forcibly removed from the user interface, stripped of rank, and put on trial for crimes against productivity. You are writing a document, not decorating a cake. You need to ask yourself “Why should this text be italics?” Is it a book title, a foreign phrase, a name of a movie, the name of a legal case? Then choose a named style that indicates why that text is special. Let the named style take care of how it is displayed.
6. Unless you are designing a poster for a modern art gallery you should stick to the named styles in your template. Power users might define additional named styles. But direct application of random attributes to random text selections should be considered a form of data corruption.
7. Few documents today are ever printed. The are born, live and die entirely in digital form. We should be optimizing for the most common cases, not just for what our parents or grandparents did with WordPerfect 1.0.
8. The most common sources of reused content come from other documents and from PDF and from HTML. Current cut & paste mechanisms today make a mess of styles. Paste in the content with the styles of the source document? According to the styles of the destination document? Mapping to the nearest local style? All are wrong answers. The only correct answer is to give me the choice.
9. PowerPoint is pure evil. It has elevated form over substance and turned every form of business communication into a “pitch”.
10. I should be able to call spreadsheet functions using named parameters, like PV(rate=1%,periods=12,payment=$1000.00) rather than PV(0.01,12,10000) so my model is self-documenting and avoids errors from incorrect ordering of parameters.
11. Security needs to be designed into the document authoring environment, including the format, not patched on as an afterthought.
12. I want Greasemonkey for my word processor and my spreadsheet.
13. Connections between documents may be as important as the documents themselves.
14. The less control the user asserts over the appearance of a document during editing, the more flexibility he or she has over the final published appearance. In today’s multi-modal, multi-device world, it is essential that we do not prematurely commit our documents to a particular rendering. We need late binding of presentation to content, not early binding. If we had done this for the past decade, we would have perfect interoperability today between all word processors. If we start doing it now, we will have perfect interoperability among word processors going forward.
15. Spreadsheets should have functions that access web-based data stores for common financial, economic, political and scientific data sets. Mathematica does something similar, presumably using local caching.
16. Presentation should be a mode of displaying another document, not just document type itself. For example, I should be able to take a report and push a button to enter a slide-show mode, where all images are shown as slides, with their captions, and each top level section header becomes a slide with 2nd level headers as bullet items. During the presentation I should be able to seemlessly drill down into the real document.
17. I want to be able to share data ranges, text ranges and presentation slides with others and to subscribe to theirs via feeds. I rarely write a document from scratch. Reuse, reuse, reuse. But the tools only support this at a scavenger level.
18. We lack high level support for the compositing or assembling a document from fragments. Once I cut & paste, my new docment has lost all knowledge of the document I copied from. This is great if I am a professional plagiarist. But it is bad if I am a CIA analyst and my report has copied information claiming uranium production in Africa, and I never know when that information is repudiated, and I pass my flawed report onto the President. Very bad. When I cite an authority for an argument, my argument is only as good as the authority. I owe it to myself and my readers to make it easy to know whether the information I cited is still accurate and vouched for by that authority.
19. Current tools are impoverished when it comes to the social side of documents. Review/comment reflects old, hierarchical thinking and doesn’t scale to the network. How can I have 100 people comment on my document? What if I want 100 people to jointly author a document? The Wiki knows where Word cannot go…
20. Most user woes in modern word processor are caused by our attempts to remain compatible with the design choices made by Microsoft Office developers 15 years ago. It is time to move on and learn from past mistakes, but not perpetuate them.
21. I want to use the same text editor to edit documents, web pages, emails, blog posts, discussion forums and wikis. Why do I need a different brand hammer for every nail?
22. I want a spreadsheet function that can call a web service. It might lookup a book title by ISBN, do currency conversions, or geocode data. There should be thousands of such spreadsheet functions, backed by web services, interoperable based on standard protocols. Some might be free, others fee-based. Some might be both, e.g., 20-minute delayed quotes for free, real-time for a fee.
23. Spreadsheet functions express a core analystic function and should be usable in all tables, in word processors and presentations, not just in spreadsheets. They should also be usable in fields in forms and in text passages.
24. The inability of word processors to output clean, readable and valid HTML or XHTML should be an embarrassment to all vendors.
25. HTML + JS + XHR + HTML DOM = AJAX. ODF + JS + XHR + ODF DOM = ?
26. We must define power as in “power user” based on results, on productivity. Power is as much about what a system allows you to ignore as what it allows you to control.
27. Today trust is based on digital signatures and classical questions of authentication, integrity and non-repudiation, all backed by a chain of trust traceable back to some well-known certification authority. In some contexts, this hierarchical, binary view of trust is adequate. But the network sees trust based on reputation, rating, scoring, voting, reverse citation counts and other non-hiearachical values. How do we account for these?
28. Spreadsheets are unnecessarily dangerous, based on a muddled view of data types which leads to silent errors and inconsistencies. This might have made sense in the memory and processor constrained systems of the 1980’s. But today, with our better sense of the errors and the cost of errors, we need a spreadsheet system that is type-safe, aware of measurement units, and which enforces consistency and accuracy. We obviously can’t prevent someone from making a stupid spreadsheet model for subprime mortgages, but we can at least ensure that they don’t make stupid cut & paste errors when creating that model.
29. Spreadsheets should have instrinsic support for image, sound and geographic data. Not just embedded media, but as an intrinsic data type, so a function could take an image as input, or return an audio clip as a result.
30. A grid in a spreadsheet provides a logical addressing scheme as well as a visual layout scheme. But what if I want the former without the latter? Why can’t I do a spreadsheet calculation in a text document? Why am I always stuck in in a grid?
31. Spreadsheets should have built-in support for sensitivity and risk analysis, perhaps via monte carlo methods. Yes, I know support is available via 3rd party plugins, but this should be a core feature in the repetoire of every user. We might not be in the global financial mess we’re in now if spreadsheet users all could easily “stress test” their models.
32. The Holy Trinity of Word/Excel and Powerpoint is only a convention, mainly enforced by Microsoft’s definition of their office suite. It is not a law of nature. Other applications types should be considered to be part of the core desktop authoring environment, such as project management and mind maps.
33. Outliners and other pre-draft tools have lagged far behind the core editing functions of a word processor. And what is the equivalent of an outliner for a spreadsheet?
34. Microsoft is as much a prisoner to the predominent model of end user producitivty as the user is. Their need to support legacy documents constraints their freedom of action and has contributed to the relative lack of innovation in Microsoft Office over the past decade.
35. An editor should allow a user to verify interoperability as easily as it lets them do a print preview.

A typical team project, whether software, standards, bridge construction or what have you, has a slow start dominated by a planning and scheduling, a middle period of execution, and an finish with final frantic rush of activity to complete the project. Then everyone takes a few days off and we start again.

One thing I learned early in my career was how wasteful this kind of project cycle is. The problem is that not everyone is involved in every part of the project. Some only work on planning, some only on execution, and some mainly come in at the end. This leads to suboptimal allocation of resources. People are standing around waiting.

One solution, not necessarily the only one, is to work on multiple versions of a project at once. For example, when working on a software application, you can take 25% of your team and have them start the planning phase of version N+1 while the remaining 75% of the team completes the final QA stage of version N.

We have a similar issue with standards development. Both the OASIS and the JTC1 PAS process involve a lot of standing around waiting: at least two months of public review in OASIS, and 6 months of review in JTC1. And even now, as we complete the editing work on ODF 1.2, the wider ODF community is standing around waiting. It is too late to make feature proposals for ODF 1.2, but too early for a full public review of the ODF 1.2 draft.

What is to be done?

The ODF TC has decided to begin activities on the next version of ODF, called for now “ODF-Next”, even before we have ODF 1.2 approved. Although we obviously won’t be spending a large amount of time on that effort quite yet, since we really are all busy with ODF 1.2, we have come up with a way to engage the broader community and have you help us gather requirements for ODF-Next now, which we can then consider during the downtime when ODF 1.2 is under review in OASIS and JTC1. The Call for Proposals for ODF-Next went out on Friday.

So put on your thinking cap. ODF 1.1 and ODF 1.2 were incremental releases. Maybe ODF-Next will be bolder, maybe something that shifts the paradigm, pushes the envelope, breaks out of the box. Is the dominant WYSIWYG word processing paradigm the final word in user productivity? Or are we overdue for a change, for a different set of priorities? As Thomas Paine wrote, “We have it in our power to begin the world over again.”

Now is the time to start collecting the ideas, big or small, and submit them to the ODF TC according to the instructions in the Call for Proposals linked to above.

We’ll be collecting ideas at least until March 31st. The Requirements Subcommittee will then sort through the ideas, categorize and prioritize them, and generally try to make sense of it all, and then write up an ODF-Next Requirements document with their recommendations.

This is a good chance to get your ideas in early and have a real impact on where we go with ODF in the next major release. But please, do not give me ideas via blog comments. We can only accept ideas sent through the above linked OASIS comment submission procedure, which is necessary to ensure that ODF remains an open standard that anyone can implement. IANAL, but I believe an added benefit is that any idea you submit, even if speculative, even if not added to ODF-Next, will be permanently archived in the ODF comment list, and thus will establish prior art which could scuttle attempts to secure patents in this area. So by contributing your ideas publicly in this way, you help to establish an intellectual commons that will benefit free and open source applications in this area.

Please pass along the word. We’re hoping to get 100’s of ideas for ODF-Next. Bring it on!

Back in the 1980’s, when I was a student, I was also an avid shortwave listener (SWL). This was in the days before the web, satellite TV or 24-hour international cable news coverage. I had an upper floor room in Cabot Hall, and each night I would surreptitiously dangle out the window a 40-foot wire antenna attached to a small weight.

At first I listened only to the big broadcasters like the BBC Word Service, Deutsche Welle, Radio Moscow, and then moved on to smaller ones: Tirana, Malta, South Africa, etc. It was a great way to get a global perspective beyond the 2-minutes allocated to international news on a typical US-based evening news program.

Eventually I started writing the broadcasters and received many QSL cards. Some of my letters were read on the air. I’m sure I ended up on some FBI watch list for those letters to Radio Prague and Radio Havana. My subscription to Soviet Life magazine, and a Cambridge address probably didn’t help either.

But you don’t go far as a SWL before you notice that there are a lot of strange things going on in the aether. Some were easily explained — the Soviet Union jamming broadcasts of Voice of America or Cuba jamming broadcasts of Radio Martí. And then there were the commercial voice broadcasts, ship-to-shore, international aviation, time signals, etc. Then the various data services, radio teletype, weather fax, etc. And then there were the mysterious coded transmissions, which we rumored to be SAC tranmissions, “Sky King, Sky King, Do not answer”, followed by various authentication codes, which were either recall or go ahead codes for nuclear attack. It was an eerie feeling, in the hotter days of the Cold War, to lay awake at night, listening to the radio and wondering whether the sun would rise in the morning. Now I just wonder if my 401(k) will still be there.

Stranger yet were the cryptic transmissions of the “numbers stations“, which would transmit on a semi-regular schedule and merely read off a large list of numbers for 10 minutes. For months I transcribed one particular woman’s transmissions, trying to find out the pattern. I did some computer analysis, but the numbers were random in frequency, with no discernible patterns. Presumably they were encoded against a one-time pad.

And then there were the “pirate” radio stations like “The Voice of the Purple Pumpkin”.

Although most people knew about the BBC World Service, I don’t think many appreciated that a large portion of the shortwave universe was strange, that the fringe was everywhere.

I’m starting to have a similar view of the web. Their are major content providers, minor content providers, even individual content providers like me. And then their is the weirdness, the strange corners of the web, the space between the channels, where you are not even sure you are listening to signal or noise.

Here are a few random examples of web sites with no discernible purpose. They appear to be garbled republications of new stories.

Let’s start with the “Wet Paint Body Notes” blog, newly created, with only three posts. One is called “Microsoft Gets Foot in Mass. Office Door“. It starts:

In what could be a coup inwardly favour of Microsoft (Nasdaq: MSFT) and a biff to the friendly wellspring league, the stipulate of Massachusetts personal added Microsoft’s Office Open XML norm to its document of give your declaration standards it will allow for elected representatives exploit.

This is a strange kind of English. It almost seems like a poor translation, or even a poor machine translation, of a document written in another language. But if you poke around a little, you find the this blog post is an unattributed garbled derivation of a 2007 article in Linux Insider. Not only was the original article in English, the reposted version truncates the article, posting only the first few paragraphs.

So what’s up with that? There are no banner ads or other obvious sources of revenue on the garbled version of the article. It is not a link farm. In fact it has no outgoing links. So why did someone bother?

Another example. The blog “75Software-News48” has an new article “Microsoft shows support for ODF“, posted just two weeks ago, with the intro:

Amid organization hassle surrounded by wish of interoperability, Microsoft (Nasdaq: MSFT) protected Thursday announced the discovery of the Open XML Translator Project. The overhang will fry in the air permitted software to allow Word, Excel and PowerPoint to knob documents in contrary technology format.

Again, this reads like it is a poor translation from another language. But look further and you can find that the original article is actually in English, from a 2006 TechNewsWorld article.

Again, no obvious intent here. It isn’t a link farm, and there is no evident source of revenue. It isn’t informative and it certainly isn’t timely. So why did they do it?

One more example this time a LiveJournal blog called “All Microsoft”, again newly created, with a post called “Ecma Approves MS Office Format, IBM Dissents“. It opens:

Microsoft’s (Nasdaq: MSFT) Open XML bureau software format, broad of via the tech giant to chase near the Open Document Format (ODF), cleared a standards hurdle this week, successful approbation from the Ecma global standards article.

Same modus operandi here. Original source, unattributed, is from a 2006 Linux Insider article.

I have dozens of examples of this kind of thing, all within the last couple of months, mainly articles about Microsoft and ODF. Something new is afoot. But what? Anyone have any idea of what this is and who benefits from it? If this just a contest between Blogger and LiveJournal to see who can claim the most hosted blogs? Or is it some SEO ploy? It has me stumped.

It is not the end of the end, nor the end of the beginning, but more like the beginning of the end for the development of ODF 1.2. The Committee Draft 01 of ODF 1.2, Part 1 was approved by the OASIS ODF TC yesterday in a 9-2-2 vote. You can download it here.

A Committee Draft (CD) is the first step toward finalizing ODF 1.2. The TC will likely approve further CD iterations before voting to approve one as a Public Review Draft. The Public Review Draft, as the name suggests, will be what we send out for a public review of at least 60 days. We can then make changes based on review comments and hold additional public reviews if we make non-trivial changes to the Public Review Draft. The ODF TC can then vote to approve the draft as a Committee Specification. We then hold a further vote to send the Committee Specification out for an OASIS-wide ballot (not just the ODF TC, but all OASIS members) on whether to approve ODF 1.2 an OASIS Standard. Once that is done, we can then start the PAS approval cycle in JTC1.

Although there are a lot of votes and process steps remaining, the major technical work is just about done. What remains is a period of review, perfecting the text, gaining implementation experience and feedback, etc. Some may call this a “death march”, but I see this pace as consonant with the importance of our activity and our deliverables. Work in OASIS might not be as fast as Ecma, where you can evidently create a 6,000 page standard in less than a year. Our process calls for a bit more than the IETF’s “rough consensus and running code.” But neither are we the slowest process in the standards development landscape. We’re some place in the middle. And when we’re talking about revising an open document format, already adopted and used by governments around the world, I am not ashamed to say that we’re working deliberately and carefully.

We also need to socialize and grow consensus around ODF 1.2, both from implementers, but also adopters and consumers of ODF. There is still work to be done here. For example, the TC vote on the Committee Draft 01 was not unanimous. We did not have the support of Microsoft or Novell. There are still disagreements over how we define conformance in the standard. We obviously need to continue discussing this topic. Since the final TC vote to request an OASIS Standard ballot requires 2/3 approval of TC members with no more than 25% disapproving, we’ll need a high level of consensus in the TC to move forward, including, hopefully, the support of Microsoft and Novell.

Implementation experience is important in OASIS. I know some have criticized OpenOffice for having support of draft ODF 1.2. But this support is a good thing, in my opinion. We need implementers to validate the design decisions we’ve made in the standard, to ensure that our choices are reasonable, that we haven’t missed anything. We’re working in an engineering discipline. We’re not making abstract standards for the mind alone. Engineers build, test and refine. It is what we do. In fact, OASIS requires that before a Committee Specification can be nominated for an OASIS Standard ballot, the TC must certify that there are three conforming implementations of the Committee Specification. So not only are early implementations a good idea, they are required as part of the process.

If you are asking, “How can I help?”, then here are a few ideas:

1. If you are an implementor of ODF 1.0 or ODF 1.1, then now is a good time to start looking at what is required to add ODF 1.2 support. Download the CD of ODF 1.2, but also look at this page for a summary of changes. We’ll formalize that list of changes and put it into a appendix of the draft, but this wiki page should give you a good feel for what areas have been touched.
2. Although we have not yet approved a Public Review Draft specifically for public review, we welcome comments at any time. You can send comments on ODF 1.2 CD 01 according to the instructions on this page. Download the draft, pick a chapter of interest and send us any errors you find.
3. We should start thinking ahead to how we can encourage a thorough review of the eventual Public Review Draft. I want to avoid the OOXML-fiasco where Ecma approved and sent to JTC1 a half-baked, deeply-flawed text. What can we do to give ODF 1.2 a really hard scrub in the OASIS review period, so what comes out meets the high standards we should expect from an international standard? I think we’ve done a good job in drafting ODF 1.2 and I want to encourage scrutiny, not shy from it. But let’s have this scrutiny earlier rather than later.