{"id":34,"date":"2006-10-17T19:02:00","date_gmt":"2006-10-18T00:02:00","guid":{"rendered":"http:\/\/2d823b65bb.nxcli.io\/2006\/10\/the-celerity-of-verbosity.html"},"modified":"2009-12-28T18:17:04","modified_gmt":"2009-12-28T23:17:04","slug":"celerity-of-verbosity","status":"publish","type":"post","link":"https:\/\/www.robweir.com\/blog\/2006\/10\/celerity-of-verbosity.html","title":{"rendered":"The Celerity of Verbosity"},"content":{"rendered":"<p>I&#8217;ve been hearing some rumblings from the north-west that Ecma Office Open XML (OOXML) format has better performance characteristics than OpenDocument Format (ODF), specifically because OOXML uses shorter tag names.  Putting aside for the moment the question of whether OOXML is in fact faster than ODF (something I happen not to believe), let&#8217;s take a look at this reasonable question:  What effect does using longer, humanly readable tags have on performance compared to using more cryptic terse names?<\/p>\n<p>Obviously there are a number of variables at play here:<\/p>\n<ul>\n<li>What XML API are you using?  DOM or SAX?  The overhead of holding the entire document in memory at once would presumably cause DOM to suffer more from tag length than SAX.<\/li>\n<li>What XML parser implementation are you using?  The use of internal symbol tables might make tag length less important or even irrelevant in some parsers.<\/li>\n<li>What language are you programming in?  Some language, like Java have string internalization features which can conflate all identical strings into a single instance.<\/li>\n<li>What size document are you working with?  Document parsing has fixed overhead as well as overhead proportionate to document size.  A very short document will be dominated by fixed costs.<\/li>\n<\/ul>\n<p>So there may not be a single answer for all users with all tools in all situations.<\/p>\n<p>First, let&#8217;s talk a little about the tag length issue.   It is important to note that the designer of an XML language has control over some, but not all names.  For example take a namespace declaration:<\/p>\n<p><code>xmlns:ve=\"http:\/\/schemas.openxmlformats.org\/markup-compatibility\/2006\"<\/code><\/p>\n<p>The values of namespace URI&#8217;s  are typically predetermined and are often long in order to reduce the chance of accidental collisions.  But the namespace prefix is usually chosen to be quite short, and is under the control of the application writing the XML, though a specific prefix is typically not mandated by language designer.<\/p>\n<p>Element and attribute names can certainly be set by the language designer.<\/p>\n<p>Attribute values may or may not be determined by the language designer.  For example:<\/p>\n<p><code>val=\"Heading1\"<\/code><\/p>\n<p>Here the name of the style may be determined by the template, or even directly by the user if he is entering a new named style.  So the language designer and the application may have no control over the length of attribute values.  Other attribute values may be fixed as part of the schema, and the length of those are controlled by the language designer.<\/p>\n<p>Similarly, the length of character content is also typically determined by the user,  since this is typically how free-form user content is entered, i.e., the text of the document.<\/p>\n<p>Finally, note that the core XML markup for beginning and ending elements, delimiting attribute values, character entities etc., are all non-negotiable.  You can&#8217;t eliminate them to save space.<\/p>\n<p>Now for a little experiment.  For the sake of this investigation, I decided to explore the performance of a DOM parse in Python 2.4 of a medium-sized document.  The document I picked was a random, 60 page document selected from Ecma TC45&#8217;s XML document library which I converted from Microsoft&#8217;s binary DOC format into OOXML.<\/p>\n<p>As many of you know, an OOXML document is actually multiple XML documents stored inside a Zip archive file.  The main content is in a file called &#8220;document.xml&#8221; so I restricted my examination to that file.<\/p>\n<p>So, how much overhead is there in a our typical OOXML document?  I wrote a little Python script to count up the size of all of the element names and attributes names that appeared in the document.  I counted only the characters which were controllable by the language designer.  So w:pPr counts as three characters, counting only &#8220;pPr&#8221; since the namespace and XML delimiters cannot be removed.  &#8220;pPr&#8221; is what the XML specification calls an NCName, also called a non-qualified name, since it is not qualified or limited by a namespace.  There were 51,800  NCName&#8217;s in this document, accounting for 16% of the overall document size.  The average NCName was 3.2 characters long.<\/p>\n<p>For comparison, a comparably sized ODF document had an average NCName length of 7.7 and an NCName&#8217;s represented 24% of the document size.<\/p>\n<p>So, ODF certainly uses longer names than OOXML.  Personally I think this is a good thing, from the perspective of readability, a concern of particular interest to the application developer.  Machines will get faster, memory will get cheaper, bandwidth will increase and latency will decrease, but programmers will never get any smarter and schedules will never allow enough time to complete the project.    Human Evolution progresses at too slow a speed.  So if you need to make a small trade-off between readability and performance, I usually favor readability.  I can always tune the code to make it faster.  But the developers are at a permanent disadvantage if the language uses cryptic.   I can&#8217;t tune them.<\/p>\n<p>But let&#8217;s see if there is really a trade-off to be made here at all.  Let&#8217;s measure, not assume.  Do longer names really hurt performance as Microsoft claims?<\/p>\n<p>Here&#8217;s what I did.  I took the original document.xml and expanded the NCNames for the most commonly-used tags.  Simple search and replace.  First I doubled them in length.  Then quadrupled.  Then 8x longer.  Then 16x and even 32x longer.  I then timed 1,000 parses of these XML files, choosing the files at random to avoid any bias over time caused by memory fragmentation or whatever.  The results are as follows:<\/p>\n<table border=\"1\">\n<tbody>\n<tr>\n<th>Expansion Factor<\/th>\n<th>NCName Count<\/th>\n<th>Total NCName Size (bytes)<\/th>\n<th>File size (bytes)<\/th>\n<th>NCName Overhead<\/th>\n<th>Average NCName Length (bytes)<\/th>\n<th>Average Parse Time (seconds)<\/th>\n<\/tr>\n<tr>\n<td>1 (original)<\/td>\n<td>51,800<\/td>\n<td>166,898<\/td>\n<td>1,036,393<\/td>\n<td style=\"vertical-align: top;\">16%<\/td>\n<td style=\"vertical-align: top;\">3.2<\/td>\n<td style=\"vertical-align: top;\">3.3<\/td>\n<\/tr>\n<tr>\n<td>2<\/td>\n<td>51,800<\/td>\n<td>187,244<\/td>\n<td>1,056,739<\/td>\n<td style=\"vertical-align: top;\">18%<\/td>\n<td style=\"vertical-align: top;\">3.6<\/td>\n<td style=\"vertical-align: top;\">3.2<\/td>\n<\/tr>\n<tr>\n<td>4<\/td>\n<td>51,800<\/td>\n<td>227,936<\/td>\n<td>1,097,443<\/td>\n<td style=\"vertical-align: top;\">21%<\/td>\n<td style=\"vertical-align: top;\">4.4<\/td>\n<td style=\"vertical-align: top;\">3.2<\/td>\n<\/tr>\n<tr>\n<td>8<\/td>\n<td>51,800<\/td>\n<td>309,320<\/td>\n<td>1,178,827<\/td>\n<td style=\"vertical-align: top;\">26%<\/td>\n<td style=\"vertical-align: top;\">6.0<\/td>\n<td style=\"vertical-align: top;\">3.2<\/td>\n<\/tr>\n<tr>\n<td>16<\/td>\n<td>51,800<\/td>\n<td>472,088<\/td>\n<td>1,341,595<\/td>\n<td style=\"vertical-align: top;\">35%<\/td>\n<td style=\"vertical-align: top;\">9.1<\/td>\n<td style=\"vertical-align: top;\">3.3<\/td>\n<\/tr>\n<tr>\n<td>32<\/td>\n<td>51,800<\/td>\n<td>797,624<\/td>\n<td>1,667,131<\/td>\n<td style=\"vertical-align: top;\">48%<\/td>\n<td style=\"vertical-align: top;\">15.4<\/td>\n<td style=\"vertical-align: top;\">3.3<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>If you like <a href=\"http:\/\/en.wikipedia.org\/wiki\/Box-and-whisker_plot\">box-and-whisker plots<\/a> (I sure do!) then here you go:<a href=\"https:\/\/2d823b65bb.nxcli.io\/blog\/uploaded_images\/boxplot-736307.JPG\" onblur=\"try {parent.deselectBloggerImageGracefully();} catch(e) {}\"><img decoding=\"async\" style=\"margin: 0px auto 10px; display: block; text-align: center; cursor: pointer;\" src=\"https:\/\/2d823b65bb.nxcli.io\/blog\/uploaded_images\/boxplot-731739.JPG\" border=\"0\" alt=\"\" \/><\/a>What does this all mean?  Even though we expanded some NCNames to 32-times their original length, making a 5x increase in the average NCName length, it made no significant difference in parse time.   There is no discernible slow down in parse time as the element and attribute names increase.<\/p>\n<p>Keep in mind again that the typical ODF documents shows an average NCName length of 7.7 .  The above tests dealt with lengths twice that amount, and still no slowdown.<\/p>\n<p>&#8220;Myth Busted&#8221;.  I revert this topic to the spreaders of such FUD to substantiate their contrary claims.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I&#8217;ve been hearing some rumblings from the north-west that Ecma Office Open XML (OOXML) format has better performance characteristics than OpenDocument Format (ODF), specifically because OOXML uses shorter tag names. Putting aside for the moment the question of whether OOXML is in fact faster than ODF (something I happen not to believe), let&#8217;s take a [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[9,6,21,20],"tags":[],"class_list":{"0":"post-34","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-odf","7":"category-ooxml","8":"category-performance","9":"category-xml","10":"entry"},"_links":{"self":[{"href":"https:\/\/www.robweir.com\/blog\/wp-json\/wp\/v2\/posts\/34","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.robweir.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.robweir.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.robweir.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.robweir.com\/blog\/wp-json\/wp\/v2\/comments?post=34"}],"version-history":[{"count":1,"href":"https:\/\/www.robweir.com\/blog\/wp-json\/wp\/v2\/posts\/34\/revisions"}],"predecessor-version":[{"id":396,"href":"https:\/\/www.robweir.com\/blog\/wp-json\/wp\/v2\/posts\/34\/revisions\/396"}],"wp:attachment":[{"href":"https:\/\/www.robweir.com\/blog\/wp-json\/wp\/v2\/media?parent=34"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.robweir.com\/blog\/wp-json\/wp\/v2\/categories?post=34"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.robweir.com\/blog\/wp-json\/wp\/v2\/tags?post=34"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}