But dihydrogen monoxide is just water in the end, and in the end anyone who wants to can use tools like jing and msv to validate their documents, and it will find errors where you have errors and will say your document is valid if it is valid The ODF 1.0 schema is useful and can be used, and is used by developers today to check the validity of their ODF output.

To your particular point about multiple XML documents, and references between them, the referential integrity of those links are not, strictly speaking, part of validity, or at least not as defined in Relax NG or ODF. These “whole document” integrity should be treated as conformance constraints.

It is very basic enginering design that you should not give the pretense to validate something if you really know that you are not checking the essential bits of the thing that need validation. Could it be that Alex Brown doesn’t have enough real world programming experience to understand this, or are he just playing dumb so save his face?

Shall we anticipate what will happen next?
I think what Alex Brown will do now is to create a set of files that demonstrate conflicts between use of ID/IDREF and start to claim that since the ODF XML schema can not detect this ODF is broken by design. The problem with this argument will of course be that his example files will not be proper ODF since they violate those parts of the ODF requirements that are not described by the XML schema.

It good to know that you Rob is around to refute his baseless accusations. Keep the good work.

The complexity is when you have a plurality of vendors and applications and operating systems. Then you need an engineered approach to interoperability and that can encompass standards, schemas, test suites, reference implementations, conformity assesment methodology, “plugfests”, etc. That is the cost of plurality.

the fact that there is only one non-working implementation of MSOXML is the actual scandalous embarrassment to ISO, as only established standards should be fast-tracked.

best wishes

Actually that may not be needed, with a bit of preparation it’s possible to write a single relax schema that validates documents according to whichever schema is specified in the version attribute.

I use this all the time for xslt1/2 discrimination (in emacs nxml mode which uses the relax schema for context sensitive editing)

The latest version of Norm’s schema is here
http://norman.walsh.name/2006/07/12/xslt20

but the first post describes the basic trick for doing this, which could be used to wrap two ODF schema, rather than two XSLT schema
is here
http://norman.walsh.name/2004/07/25/xslt20

basically you just take the two schema that you have and write a three line schema that includes them both…

Alex Brown said, “Given Microsoft’s proven ability to tinker with the Office XML file format between service packs, I am hoping that MS Office will shortly be brought into line with the 29500 specification, and will stay that way.”
http://www.griffinbrown.co.uk/blog/PermaLink,guid,3e2202cd-59a3-4356-8f30-b8eb79735e1a.aspx

Alex, you may have missed how, about 18 months ago, the same Microsoft Office team made Outlook use Microsoft Word to render HTML emails, despite the fact that this significantly reduced support for HTML standards compared to earlier Outlook versions. They have still not brought it back into line. This is not a hopeful precedent.
http://www.campaignmonitor.com/blog/archives/2007/01/microsoft_takes_email_design_b.html

Take Microsoft Windows Server product vs. LINUX, for example, as David Williams blasted MIcrosof about complete lack of “facts” in Microsoft’s Get The Facts webpage about Microsoft Windows Server in comparison to LINUX.

I’m sad to say — these tactics are nothing new for Microsoft — and even more so, for these tactics to be bled into standards-creation process and used to attack an existing standard in order to promote the standardization of a competing specification.

The difference between validation and conformance is only confused in casual use. But to an XML expert there is a world of difference.

So watch XML expert’s worlds carefully. If they something something is ‘broken’ or it doesn’t ‘comply’, then they are using waffling words that mean nothing firm. But when they talk about validity and conformity, then they are saying something specific.

Rob, it’s probably fair to describe “testing conformance” as “validating.” That seems a reasonable use of language.

I think it might be more accurate to say “Using a Relax NG tool, one validates against the schema, not the specification.” As you correctly identify, there simply is no program to validate against the whole of the specification. Nor against most specifications, come to think of it.

But I do understand where he is coming from, and others as well. They are eager to once again criticize XML standards. After spending the past 18 months in a “hear no evil, say no evil” posture, ignoring the defects in OOXML, giving blanket approval to whatever Microsoft wanted, they are eager to once again use their critical mental faculties and try to reestablish their credibility.

As for the question “severity of OOXML defects” discovered by me, I suggest we come to that point again in around a week. I think it will be easier to answer that question in context.

I sense some confusing of conformity and validity.

Conformity is a statement of the relationship between a document, or an application, and the ODF standard. If the document or the application meets the requirements stated in the ODF Standard, then it conforms to the ODF Standard.

Validity is a statement about the relationship between an XML document and an XML schema. A Relax NG schema expresses patterns that a valid XML document must match. An XML document is valid if it matches the patterns described by the schema.

As one reader mentioned, a conformant ODF document must also be valid according to the ODF schema, if the foreign elements and attributes are removed.

But I would not say that this is a “minimum” requirement. More correct would be to say that it is a necessary, but not sufficient requirement. For example, the requirements for packing the multiple XML file into a ZIP — this cannot be described in a Relax NG schema, nor can the relationships between style names in styles.xml and the use of those styles in content.xml, nor the hierarchy of styles, nor any of the semantic of behavior requirements of ODF. So even a minimum, “hello world” ODF document needs more than validity in order to be conformant.

In terms of ID/IDREF, I’m saying that the cross-reference semantics, for example, that ID’s are unique, that IDREF’s contain the name of an ID, that these are not validity constraints that the validation model defined in the Relax NG standard checks. I quoted earlier James Clark on why that is so.

So, in terms of Relax NG, ID/IDREF consistency is not a validation criterion. A document, even with duplicate ID’s, or inconsistent use of ID/IDREF could still be valid to a Relax NG schema.

However, that doesn’t mean that the ODF standard permits inconsistency in ID/IDREF. Conformance to ODF, as I said earlier, requires more than validity. It requires other things, and this includes proper ID/IDREF semantics. This, like any other conformance constraints beyond the schema, would need to be checked by application logic. This is not difficult to do. In fact I’d expect that all ODF documents out there in fact are consistent in their use of ID/IDREF. This is a trivial consistency to maintain in an application.

Back to the question “[C]an we claim that the OASIS wiki instructions are enough to validate an ODF document against its specification?” I hope you see now how that question misstated. One does not validate against the specification. One validates against a schema. And the instructions given on the wiki are complete.

If you want to check conformance of an ODF document against the entire ODF specification, then you would need a custom tool that would check XML validation, check packaging constraints, check referential integrity of links, etc. Note that ODF makes only scant use of ID/IDREF. Most of the interesting links in ODF are not expressible using XML ID’s, since they are links from XML to not XML resources (like images) or from one XML file to another.

No such tool exists that does a complete conformance test of an ODF document, at least not that I’m aware of. The ODF Fellowship has an online Validator that does a subset of tests. That is a good start.

The ODF specification states:
“Conforming applications…shall write documents that are valid against the OpenDocument schema if all foreign elements and attributes are removed before validation takes place”

So schema validation is actualy a minimum requirement for conforming ODF writing applications.

“That in no way means that the constraints defined in XML 1.0 are not required on ODF document instances. It simply indicates that you would need to verify these constraints using means other than Relax NG DTD Compatibility.”

I hope it makes sense to make these questions:
* how could one verify these constraints?
* without verifying them, can we claim that the OASIS wiki instructions are enough to validate an ODF document against its specification?

Keep up the good work. I must say that I’m deeply disappointed with Alex.

In any case, the fact that jing does not check the syntax constraints on ID’s when using the -i flag to disable the DTD Compatibility mode, this clearly a limitation of jing. But there is nothing in principle that would prevent a Relax NG validator from checking these lexical constraints in default processing. But jing doesn’t do that.

The thing that is tripping you and Alex up seems to be the fact that the ODF schema will pass some uses of ID’s that are not allowed by XML. So what? The schema is not a complete statement of ODF conformance. We have attribute values that are typed string that could probably be more strictly typed with a regex. We have other attribute values typed integer that could more strictly be limited in range. So what? If the text of the ODF standard is clear that there are additional value space constraints on these attributes, then conformity fully requires these constraints.

Similarly, just because the schema doesn’t allow you to check the cross-reference semantics of ID/IDREF doesn’t mean the constraints don’t exist. It just means you need to test them in other ways. But it certainly doesn’t mean, as some have suggested, that the ODF schema is invalid. That is preposterous.

I think that you are misinterpreting ID/IDREF story in RELAX NG. But anyway, if this check should be done only in lexical space as you describe, schema is wrong. In RELAX NG schemas can be ambiguous and current ODF 1.0 schema allows validation of ID attributes against correct ID type or against text pattern. As each attribute value is valid against text pattern, even lexical tests for ID types will not be effectively performed and documents with wrong values in ID attributes will pass validation.

As for which version of ODF I prefer, I’d say I prefer the latest and greatest. Remember, OASIS ODF 1.1 has all the yummy goodness of ISO-approved ODF 1.0, plus some accessibility enhancements. What’s not to love? But in the end, it is up to each vendor to decide for themselves what to support, and for each user to make their needs and wants known to their vendor.

I do not see this as mental gymnastics. I am simply trying to explain a complex topic in a way that is rigorous as well as readable. I am sorry if you find this difficult.

In any case, you will not find any mention of ID/IDREF in the Relax NG specification. It is not an Relax NG concept. Some good background reading on the topic is James Clark’s 2001 “Guidelines for using W3C XML Schema Datatypes with RELAX NG” which says in part:

“The semantics defined by [W3C XML Schema Datatypes] for the ID, IDREF and IDREFS datatypes are purely lexical and do not include the cross-reference semantics of the corresponding [XML 1.0] datatypes. The cross-reference semantics of these datatypes in XML Schema comes from XML Schema Part 1. Furthermore, the [XML 1.0] cross-reference semantics of these datatypes do not fit into the RELAX NG model of what a datatype is. Therefore, RELAX NG validation will only validate the lexical aspects of these datatypes as defined in [W3C XML Schema Datatypes].”

You (and Alex) are conflating two different questions:

1) Are the ID/IDREF/IDREFS semantics of XML 1.0 applicable to ODF 1.0 document instances?

The answer is yes. Since ODF is XML, I would expect it to obey the syntax and semantics of the XML 1.0 (third edition) Recommendation, unlike, say, OOXML, which corrupts strings with illegal control characters.

2) Is the Relax NG schema that is included in ODF 1.0 written in such a way that will allow checking of these constraints using non-standard extensions to the Relax NG processing model, like Relax NG DTD Compatbility?

Empirically, the answer is no. But there is nothing in the ODF 1.0 standard that suggests otherwise. It isn’t required for XML conformance and it isn’t required for Relax NG conformance.

As I said in the post, I’m entirely in favor of changing this if it helps with XML tools like jing, so they can run in their default mode, thereby saving users the horrible time and expense of adding the -i flag to disable the non-standard Relax NG DTD Compatibility processing.

But the main purpose of my post is to respond to Alex’s rant that “There are no XML documents in existence which are valid to ISO ODF” and “The standard itself has a defect which prevents any document claiming validity from being actually valid”. I believe I’ve put that to rest. In the end, Alex’s post had a reasonable enhancement proposal, prefaced by a bunch of indefensible tripe.