I’d like explore the topic of electronic documents, digital signatures, and what properties are required of them them to be considered accurate and reliable written records. Since this is as much a social question as it is a technical one, we’ll start with some history.
“An Act for prevention of Frauds and Perjuryes” 29 Carol. II (1677), commonly called “The Statute of Frauds”, begins:
For prevention of many fraudulent Practices which are commonly endeavoured to be upheld by Perjury and Subornation of Perjury Bee it enacted by the Kings most excellent Majestie by and with the advice and consent of the Lords Spirituall and Temporall and the Commons in this present Parlyament assembled and by the authoritie of the same That from and after the fower and twentyeth day of June which shall be in the yeare of our Lord one thousand six hundred seaventy and seaven All Leases Estates Interests of Freehold or Termes of yeares or any uncertaine Interest of in to or out of any Messuages Mannours Lands Tenements or Hereditaments made or created by Livery and Seisin onely or by Parole and not putt in Writeing and signed by the parties soe makeing or creating the same or their Agents thereunto lawfully authorized by Writeing, shall have the force and effect of Leases or Estates at Will onely and shall not either in Law or Equity be deemed or taken to have any other or greater force or effect, Any consideration for makeing any such Parole Leases or Estates or any former Law or Usage to the contrary notwithstanding.
Or, to loosely paraphrase in modern English: “We’ve noticed that verbal agreements are being abused. So in certain specific important agreements you better put it in writing and sign it, otherwise don’t bother to bring any dispute to court.”
A few things to note about the Statute and its context:
- As the preface notes, frauds were being perpetrated, involving oral contracts and perjury. Before this Statute, oral testimony, even without any evidence of a written agreement, could be used to deprive a person of real or personal property.
- The Statute is concerned with private agreements. Although it was already well-established practice by this time for official acts, writs, etc., to be recorded in written form and sealed, literacy, even among tradesmen, was not high, and private agreements were made only orally.
- The imposition of a stamp duty or tax to seal official documents, followed this Statute a few years later, ostensibly to raise funds to fight a war against France. But like all forms of taxation, they seem to outlive their original intent, and exist even to the present day, even though England apparently is now at peace with France.
This Statute spread to the American Colonies, where in modified form it lives on in various state laws, and in the Uniform Commercial Code (UCC) today, in §2-201:
A contract for the sale of goods for the price of $5,000 or more is not enforceable by way of action or defense unless there is some record sufficient to indicate that a contract for sale has been made between the parties and signed by the party against which enforcement is sought or by the party’s authorized agent or broker.
I’d like to look a little at what it is about a written agreement that gives it its particular value. Why did they require it to be written? Why not just require witnesses to an oral agreement?
A few salient properties of a written agreement:
- A written agreement states the parties to the agreement, the terms of the agreement and is signed by the parties.
- Once signed, the agreement may not be altered but by mutual consent of the parties. In the judgement of Brett v. Ridgen, Plowd. Comm., 345, Lord Dyer wrote that “…men’s deeds and wills, by which they settle their estates, are the laws which private men are allowed to make, and they are not to be altered, even by the King in his court of law or conscience. We must take it as we find it.”
- The “mirror image” rule applies. Both parties must agree to the same terms. If part A makes an offer, and party B says they accept, but in fact adds or qualifies the terms of the offer, then this is properly treated as a counter-offer. The agreement is not made until both parties agree to the same terms.
- The underlying mechanics and notation of the agreement are flexible, unless otherwise specified. Whether scribbled with a crayon on a napkin, sent by telegram, teletype, fax or email, these may all be considered written agreements.
The affordances of paper and ink, which lends itself particularly well to the above concerns include:
- Paper/ink expresses symmetric information. What you see is what I see and is what will be seen in court if we end up there some day.
- There is no invisible ink, no hidden pages. The text of the agreement does not say something different under the florescent lights at the court house versus the sunlight at the construction site. Although these things in theory could be done, via special inks and papers, the use of these techniques in an agreement would be prima facie evidence of fraud.
- Certainly, if it is poorly written, the terms of the agreement could be ambiguous and subject to various interpretations. Paper/ink cannot make you or your lawyer smarter. It only makes the agreement an accurate and reliable record. If a particular word is smudged or a number is crudely written, I can see this flaw and you can see this flaw and either of us can require the flaw to be fixed before we sign the agreement. If there is text that is unclear in meaning, I can ask my lawyer to explain it. I am able to understand the document perfectly should I take care to do so.
- Paper/ink is accurate and reliable over the time scale of personal and commercial contracts.
- A person’s signature or mark on an agreement, absent evidence of fraud or coercion, clearly indicates their assent to the terms of the agreement. We do not commonly write our signature unless we intend to express assent.
Jump ahead to the present day, with the increasing use of electronic documents and digital signatures. Digital signatures offer some of the same affordances we traditionally had with paper/ink. Provided the chain of certificates and keys have not been compromised, that the underlying applications have not been compromised and that the act of signing requires an affirmative and unambiguous action by the signer, a digital signature is evidence of:
- What was signed
- Who signed it
- the intention to sign, i.e., give validity to the agreement
However, there is a weakness in electronic agreements, even when digitally signed. The weakness is in what is signed. When you sign a electronic document, you are signing the stream of bits and bytes that comprise that document in a particular document format. The average person lacks the ability to directly inspect or understand the underlying representation of an electronic document. They can only see what a particular software application running on a particular operating system running on a particular computer shows when loading the document. Will that signed document appear the same on a different computer, to a different person using a different software application or a different operating system? That is the critical question. Unfortunately, the affordances of paper/ink for symmetrical information, lack of hidden information, invariability over time and venue changes, etc., are not necessarily guaranteed with electronic documents.
The digital signature guys call out an additional requirement needed for a digital signatures to give the same guarantees as paper/ink agreements. It goes by the acronym WYSIWYS, or “What You See is What You Sign”.
So what is required for electronic documents to have the same affordances as paper/ink for use as accurate and reliable records? I suggest the following:
- The format used by the electronic document must be specified in an open standard.
- The format standard must define the characteristics of semantically equivalent documents and specify the format sufficiently so that implementations of the standard can display semantically equivalent renderings of the document. Semantic equivalence is not broken by minute differences in layout, so it should be possible to have semantically equivalent renderings on different devices, e.g., a laptop versus a smart phone versus a screen reader.
- The application used to view and sign the electronic document must conform to standard, specifically those stated parts of the standard necessary to render a semantically equivalent document.
- The document must be strictly conformant to the standard, with no extensions. Just as you would not physically sign a paper document that contained interpolated text in a language that you do not understand, you should not sign an electronic document that contains unknown extensions. Otherwise semantic equivalence is not guaranteed between the two parties and a “mirror image” problem.
- Semantic equivalence must not rely on graphics. Although graphical content is permissible, such content must be redundant with respect to the text. Otherwise the “mirror image” problem is unresolvable between sighted and blind persons.
Further, I believe these criteria are of more general applicability. Although the Statute of Frauds may have been intended for marriage contracts and the like, the need to have accurate, reliable written records is a ubiquitous requirement for business and public administrations today. Wherever misunderstanding would be liability, where it is particularly important for multiple parties to be “on the same page” with respect to the contents and meaning of a document, these considerations apply.
For editable formats like ODF, I think it points out the need to describe a formal content model that describes the semantic content of a document, aside from its formatting and layout. So text + lists + tables + headers + footers + footnotes + images + captions, etc. Visual appearance is nice to have as well, but it is less robust when rendered on different devices, different operating systems, and is less likely to be robust when rendered on OpenOffice 10.0 in 2015. But the equivalence of the semantic content of an unextended ODF document should provide the same ability to have an accurate and reliable record in an electronic document as we have had traditionally with paper and ink.
There was some rather extensive work done in this area around the concept of electronic checks (not as we currently know them, a mere name for ACH electronic payments). IBM was very heavily involved in this work.
The semantics developed for this, through a pre-xml markup language called FSML, supported the necessary semantics for check payments, but in addition supported a flexible structure which allowed for the removal of document parts while maintaining the integrity of the signatures of other parts of the documents.
Some of the signature mechanisms eventually were used as input into the XML Digital Signature work, but I don’t think they survived intact.
If you are interested in more information on this, let me know. The effort was quite extensive.
Matthew Raymond says
I think signing documents should be done with digital signatures and a secure hash of the document contents. When “signing” a document, you encrypt the hash in such a way that it can only be decrypted using a signer’s individual digital signature as a decryption key. You then insert the result into an XML file within the ODF archive file. When you go back to see if the document has been signed, you take the encrypted code for each signer in the XML file, decrypt them using their respective digital signatures, and if the decrypted hashes all match, then everyone signed the same document.
@Matthew, that would ensure that all parties signed the same bits that represent the document. Whether they actually saw the same document and assented to the same terms depends on whether the software that interprets the bits of the document does it in semantically equivalent ways.
As an extreme example, if the program has code that says if(user==”Rob Weir) then fee=fee*1.25 then we have a problem, right? We would sign the same document but actually not be agreeing to the same terms. So my question is what are the necessary properties of a file format standard that would ensure that this problem cannot occur. Certainly ASCII text documents or even a bitmap format would be safe. But is it possible to make strong assurances with more complicated formats? I think it can be done, and I’m outlining what I think the requirements are.
Actually, the XML Digital Signature Standard (XML DSig) was developed with the importance of signing what is seen fully in mind.
That is why part of the specification is the provision for identification of transforms on the digital document that represents some canonical rendition of the visible content that is subject to the signature. It is my understanding that it is the transformation-derived rendition content that should be signed under the conditions that you set forth. Signature verification requires retransformation followed by the usual steps for cconfirmation of the digital signature(s) on the canonical renditions.
My understanding of the incorporation of XML DSig into ODF 1.2 working drafts, so far, is that no transformations have been specified for accomplishing this in a standard way with ODF packages.
I agree that it is not enough to sign untransformed canonicalizations of the XML Documents that are the parts of an ODF package, even with any extensions eliminated as part of that procedure. For example, it is not the formulas of a spreadsheet that would normally be signed, it is the presented values that are signed.
It should be clear that the application of XML Digital Signatures with respect to conventional office documents is quite challenging, involving substantial future work to be addressed satisfactorily in the specification and especially in implementations. The implementation on which the draft ODF 1.2 provisions appear to be based has not dealt with WYSIWYS at all, as well as I can tell.
Interesting post. I think you put your finger on it when you say:
“The application used to view and sign the electronic document must conform to standard, specifically those stated parts of the standard necessary to render a semantically equivalent document.”
Is there any ambition to do the necessary work to make this possible for ODF? Currently a conformant ODF application is free not to render features of a document so there is no WYSIWYS guarantee arising from conformance to the spec. This could be solved by introducing something like OOXML’s “application description” feature. An application description could then be defined for ODF consumers so that they had to implement “those stated parts of the standard necessary to render a semantically equivalent document”, and there would be a guarantee that for such apps you should be seeing what other like apps would show.
@Orcmid, Canonicalization in the W3C sense is at the XML level, not at the end-user level. So it specifies requirements for transforming XML documents so that documents that represent the same XML model are lexically equivalent. So not exactly what I’m looking for, But it is an interesting thought as to whether semantic canonicalization could be specified for ODF such that semantically equivalent documents were lexically equivalent.
@Alex, I’m certainly in favor raising the bar for what an application must implement in order to claim conformance. We do have a feature set defined for the various application types, currently as an annex. I need to see how much support there is on the TC to turn these into conformance targets.
This is all about that you must have tools that can be trusted. If the office suite that shows the document is trusted then you can sign it with confidence.
Suppose you from the provider of the office suite get a hash that show how an installed program should be then you are safe to sign…provided that you trust the OS that report the hash and that some authority verify that you indeed did get the reference hash from the real provider of the program.
If we move even more steps forward compilation from source code is not a fail proof solution either. The compilator can have been tampered with and do insertion of extra code when you build your program. This includes when building the actual compiler so having the compilers source code is not enough either.
The only way you stand alone can totally sure about the intrigity of the program is if you build OS, compiler, editors and program from scratch with extensive bootstrapping. (this is provided you can trust the hardware to flawless and non tampered with)
In all realistic cases your only option is to have a trusted authority. The effort to go without is simply not worth the money.
Fortunately the GNU ecosystem and its many eyes is pretty close to being such guarding force against tampering. The reason I say they are only pretty close is that GNU basically started from nothing so they can actually verify they don’t have poisonous software stack…provided that you trust the GNU people to begin with.
At the end of the day ODF standard can define how a application should act to claim to have signed a document, but there is no replacement of having a chain of trusted partners.
@Fiery Spirited, I’m not sure a trusted editor is sufficient. The validity of an agreement requires symmetrical information in the written agreement. Both parties must be agreeing to the same written text. The trusted platform is really a statement about integrity of the software stack. It isn’t a statement about correctness of the software’s rendering. For example, I could use a trusted platform A and you could use a trusted platform B, and if either platform’s software does not correctly render the semantic information of the document, then we are signing different documents. If this made it into court, I think either one of us could argue against the validity of the agreement.
Even if we both used the same platform software, different revisions of that software could introduce risk. It isn’t enough for the agreement to look the same to both of us today. It needs to look the same someday in the future such as when we end up in court.
On the other hand, it could be that neither one of us have trusted editors, but if there is a web site that allows is to upload our document and view a semantically equivalent version. So you don’t really need your editor to be trusted, but you do need to have access to a document renderer that is trusted and correct.
So it comes down to correctness and trust.
Whatever mechanism is chosen it also needs to be reliable in the situation that one party deliberately wants to trick the other party.
A trusted authority may be one solution. Another solution maybe to turn it into a trusted format (pdf, tiff, png, …(paper :-))).
Matthew Raymond says
That would require sending the final from of the document to all parties. That largely defeats the point of electronic documents in the first place.
However, something similar would be to upload the document to a Web site, have it rendered in a Web app using the HTML <canvas> element or something, and then have the parties sign the document via the Web app. That's a bit better than sending a large set if image files or mailing it in paper form, though not much.
I really think that if you don't have some baseline standardized rendering, you can't solve the problem of WYSIWYG document signing. For legal documents that must be signed, extensions that effect rendering can't be tolerated. Therefore, I suggest that prior to signing a legal document electronically, ODF applications must show the user a presentation of the document using ONLY the basic standardized rendering.
This would solve most problems, but I still see situations where people can exploit bugs in the software to hide content. Also, if people are able to get a hold of a person's digital signature, they could fake a document signing at will. Perhaps a Web-based application for signing documents is actually the cleanest way to do this. (Of course, that assumes the document hasn't been manipulated to exploit bugs in the Web app so that it emits content that is browser-specific, but that's rather unlikely.)
Bart Hanssens says
Matthew: probably you have to get hold of both the signing token and the pin code (at least, that’s how the signing with eID works in Belgium) to forge signatures.
Other remarks: don’t forget to also get a timestamp from a trusted third party.
Actually, finding a trusted third party is the easy part here (I’m sure Certipost and the likes will be happy to sell you these services)
Agreeing on the rendering is difficult, that’s where ODF could take a page out of PDF’s book…
Or even better, PDF/A-1a. That is: no scripting, fonts must be embedded etc. (Yes Rob, I should write this down as a profile :-))
By the way, is font embedding scheduled for ODF 1.2 ? IIRC there was a proposal to include it, but I’m not sure if this is on the roadmap for ODF 1.2 or Next…