When is a proof no proof of anything? Introducing Full revision history reports¶
Amnesty International has just published their 2015/16 Annual Report, a 400-page publication on the state of human rights in 160 countries. To manage the complex publishing workflow, Amnesty International migrated the key aspects of the project and team to the Booktype publishing platform for the second year in a row.
The launch of the Amnesty International report for 2016 enables us to reflect on the improvements that have been achieved in open source dynamic publishing since this key publication in the field of human rights was first delivered using Booktype, in multiple languages simultaneously, one year ago.
The Booktype team had long recognised that single-source, HTML-first workflows could deliver outputs in many book formats on demand. This year's project brought about the realisation that conventional proof copies did not go far enough in meeting the needs of all contributors. Being able to generate special PDFs for proofreaders with proposed changes and unconfirmed facts highlighted served one group of editors well, and yet translators needed something quite different. The proofreaders wanted to see the current state of the text, while the translators wanted to compare the English text that they were originally supplied with to that current state, in a way that would help them identify just the updates required for their translations.
Searching through 169 chapters manually was not an option for the many different language teams, especially since late updates to politically sensitive information could arrive at any time up to the print deadline. Since Booktype saves each chapter revision in its database, the solution for the translators' requirements was to develop a new history export feature. Booktype already enables editors and other contributors to compare the change history of each chapter between any two selected revision numbers, with a button enabling them to revert the chapter to a previous revision if required.
The new 'Full revision history' link is a custom feature which exports a report of all changes since the .docx files from the chapter authors were originally uploaded and converted to HTML by the python-ooxml library. If there are no changes to a particular chapter since then, even if a new revision was created, the history report says so, and skips to the next chapter. If there are changes, the whole chapter text is shown for context, with red and green highlighting for deleted and inserted text, respectively. Each chapter revision is timestamped, with a link to the relevant chapter for editing.
This history report is a standard HTML page which can be saved to a local computer and shared with editors and translators who do not have access to Booktype. It is declared as the international UTF-8 character set, which should ensure that accents on characters and non-Latin scripts such as Arabic are shown correctly when the page is shared on other platforms.
The 2016 Amnesty International project pushed the boundaries of rendering scripts using open technologies, including right to left Arabic script. The selected Arabic fonts did not behave in the same way as the Latin fonts when it came to adjustment of line height in complex layouts. The Booktype team solved the issue by pre-rendering pages such as covers and separators in SVG files, then inserting the SVGs as backgrounds into the PDF output. This enabled variable text to be overprinted, retaining the availability for users to edit this text at any time, but preserving the look of these pages as the designers intended.
Another set of challenges arose from extending the language coverage to include a Chinese edition this year. Previously, Amnesty International's translators in Hong Kong had produced their own edition, but for the sake of design consistency it was agreed that they would adopt Booktype for 2016. As Booktype is natively UTF-8, the developer team knew it could work with Chinese text, but the sticking point was the selection of a font that matched house style guidelines and would work with the mPDF renderer. Real TrueType fonts (.ttf files) are compatible with mPDF. Although TrueType Collections (.ttc files) should work in theory, some of these files contain PostScript outlines which are not supported. When font designers use the .ttc extension for backwards compatibility, this can cause rendering to fail. The Amnesty International team in Hong Kong settled on WenQuanYi Zen Hei, an open source san-serif Chinese font in TrueType format, available in Debian or Ubuntu GNU/Linux as the package fonts-wqy-zenhei.
Perhaps the most noticeable improvement for the Amnesty International editors this year was the speed with which the press-ready PDF was delivered. For last year's project, Booktype exported XML to Adobe InDesign where it was adjusted by our design and layout team. This time, we pushed the print and screen PDF output directly to mPDF with CSS controlling the design, reducing the delivery time from hours to minutes. Even so, the activation of mPDF features such as font subsetting and support for non-Latin character sets does add to processing time. As described in a recent blog post on the rendering speed improvements that PHP 7 delivers, complex books can now be assembled in seconds.