Wiki-to-print, Wiki2print, Wiki-to-pdf, and so on...

From creative crowd wiki
Jump to navigation Jump to search

An exchange between H&D and Varia about “publishing along the trails”


Date: 17 July 2023
Interviewees: Hackers & Designers, Varia
Context: H&D Bulletin #1, 07/23
Links: https://wiki2print.hackersanddesigners.nl/wiki/Creative_Crowd https://wiki2print.hackersanddesigners.nl/plugins/HD_Bulletin_1/pdf/HD_Bulletin_1 https://cc.vvvvvvaria.org/wiki/Wiki-to-print,_Wiki2print,_Wiki-to-pdf,_and_so_on...



In the following conversation, the two collectives H&D and Varia discuss and trace their experiences of working with wiki-to-print / wiki2print / wiki-to-pdf.

While the prevailing monocultures of design softwares create the condition in which it is difficult to imagine publishing processes otherwise, the wiki-to-print experiments discussed here propose other socio-technical scenarios for designing and working together that are not merely utilitarian or solution-driven. Such tools nourish attitudes that resist a linear, efficient, and a progress-based understanding of a design process.

(H&D) We are curious about the genealogy of wiki-to-pdf/print as it is used for collective publishing. Can you recollect what brought you to this workflow? What precisely sparked your curiosity and kept you interested in working with the MediaWiki infrastructure to edit and design publications?

(Varia) We were introduced to wikis and the MediaWiki software in particular during our studies, at what is now called the Experimental Publishing Master at the Piet Zwart Institute. That wiki has been used since 2006 and is still used today by the students and staff to document projects, classes and how-to’s, as a shared calendar or place to share thesis drafts with tutors. The sociality of the wiki appears when it is part of the daily infrastructure and when it's used so intensively. The RecentChanges page, for example, has become a place where you can find out what other people have recently been working on. Hooking such social dynamics into a print workflow creates a lot of exciting possibilities when you think about it.

Working with a "fresh" wiki for making a book is quite different, but not less exciting. Using a wiki turns a publishing workflow into a collective environment where everyone can follow the development of the content without imposing predefined article structures. A wiki page is "just" a page. The form and format are open. It's very different from, for example, a WordPress article, where the structure is predefined and cut up into smaller pieces (title, subtitle perhaps, author name, introduction, main text, etc). The attitude of the wiki software, to not impose a structure or format upon you, creates an interesting shift from technical protocols towards social protocols. What we mean by social protocols, is that decisions need to be made together, between people, not through a predefined technical configuration.

This way of making publications is part of a bigger landscape of collective working environments, tools, habits and types of publications. This shift, from assembling a publication using a graphic user interface to compiling a publication through code, is super exciting to us. In our view, the biggest impact this software has is that it allows for a non-linear way of working: it makes the workflow much more continuous, and the design and editing are done in parallel. Of course, a deadline is unavoidable when working on printed matter. At some point, you need to stop editing. But this deadline is much softer compared to linear workflows where designers need to wait for the final version of the content before the design process can start. You can just make adjustments to the document style in CSS declarations, which are rendered each time you refresh the page. But they don't influence the content, which can keep changing.

Working with wiki software specifically is a small subsection of this landscape. There are many projects that worked with wikis for printed matter. Taking part in such a continuous timeline is super interesting to us! In 2017, Sandra Fauconnier, Cristina Cochior and Juan Gomez worked on the potential of e-books based on Wikimedia content[1]during an INC research project at the PublishingLab. The Wikimedia Foundation has also worked with a range of different tools[2] to add print support to the MediaWiki software and Wikipedia in particular. We had to dig a bit to find that link to PublishingLab, which thanks to the WayBack Machine, still exists online.

This reminds us of the importance of documentation for maintaining links between the trail of wiki printing projects that has been growing in multiple directions in the last years. Code has been reused between different groups, ideas and approaches have been versioned over time, and a range of different kinds of printed publications have been based on wiki workflows: newspapers, printed books, digital PDF files and web pages. This wiki-to-print direction introduces Flask to dynamically connect the writing of the content and the rendering of the PDF. It is a side branch of the longer wiki printing timeline and operates in a more specific art/design context around the Netherlands and Belgium so far, but hopefully, it will continue travelling, morphing and transforming. Some projects on this trail include the following ones, but there are probably more.

(Varia) We are curious how you experienced being part of this trail. And how did it feel to see your code being reused again?

(H&D) It took some time before understanding the different pathways of this trail, how code has been forked, continued in different contexts, and developed in different directions. This process is ongoing, constantly evolving, so it's perhaps impossible to fully comprehend. There is a slowness to this process. It seems a (dispersed) collective effort, which is quite interesting. There are no feature requests or an agreement amongst these different groups and repositories about uniform outcomes. These tools can be continuously questioned – conceptually, technically, ethically and not necessarily conclusively ... and readjust the general perception of what is inevitable and what is useful in conceptualizing and actualizing tools. It seems it is a non-linear and somewhat unpredictable trail that has evolved over the years, that has more and less active moments.

H&D has also worked with wikis for a long time already, i.e. as a cms to our website[3], which is a central place for writing and editing workshop manuals, announcements, workshop and project documentation. André Castro joined a H&D meetup a long time ago (maybe in 2015) and introduced us to some wiki stuff. Similarly to what you describe in the context of XPUB, wikis became an integral part of H&D's collective infrastructure. Wikis are intertwined with the ways the group works together, which is also reflected in the shared excitement about working with wikis as a publishing infrastructure. They allow us to collapse processes of writing, editing, designing, and archiving.

We made a publication in 2015 using wiki and LaTeX [4], and in 2016 we made one using wiki and Scribus [5]

(Varia) What triggered the shift from LaTeX and Scribus to wiki2print?

(H&D) It was exciting to see a layout produced more or less instantly. And it looked pretty good! Yet, the learning curve of LaTeX was quite steep. Coming from graphic design and being used to somewhat 'intuitive' ways of working with layout software, that is, being able to see different versions of a layout very quickly, the process and outcomes of LaTex seemed rather stiff in comparison. It also seemed that LaTeX is very good at certain things – i.e. making a layout look very 'proper' but not so good at making surprising interventions. The wiki-scribus combination offered more flexibility. We basically used the wiki to edit and structure the publication collaboratively and eventually used Pandoc to convert the wiki syntax to something we could import to and use in Scribus. We could simultaneously edit in the wiki and easily refresh the Scribus file with the new content without losing the styling we had done. But there was still this annoying break between writing/assembling of the content on the one hand and styling it on the other.

 

It took us a while to move towards wiki2print. There were various other publishing experiments in between, like heart-beat to print[6], and the iteration of Sarah Garcin's PJ Machine (Publishing Jockey Machine)[7].

What is interesting about wiki2print is indeed the workflow that is so different to what one is used to. As you mentioned before, the separations of different tasks, that are often embodied by different people (designer, writer, coder, copy editor...) are not so separate anymore. As a designer, you make different kinds of choices, you are perhaps more forgiving? It may seem as if less is possible in this workflow, for example in comparison to the inDesign experience. You have less control or agency over the outcome. But one could also question this so-called agency given by certain tools – the illusion of mastery they emanate.

Wendy Hui Kyong Chun wrote about this in her text on Daemonic Interfaces[8]. The so-called intuitive user experience, the dragging and the dropping moves the tool to the background. One thinks about it less if one feels "in control". You don't question it anymore, it enters your subconscious... In this way, tools like inDesign, become unquestionable, unchallenged. Working with wiki2print is quite edgy in comparison. The limits are very present... maybe something to hold on to, not to overcome?

In 2022 we worked on the Making Matters publication [9] for which we looked at the repository of Manetta for Volumetric Regimes for Possible Bodies[10] and Martino for TITiPI[11]. It was super helpful to see some exciting examples and build upon what is already there. By doing so, also a feeling of response-ability evolved, to make the trail legible somehow. The writing of the colophon became an important part of making that trail more visible. We added an elaborate Note about the design of the publication in which we acknowledged the work that has already been done, and mention other relevant projects and practices that inspire us. We felt so excited to read the colophon of Minor Tech and see the trail grow.

As you mentioned before, part of continuing the trail is also making sure projects are well documented so that someone else can understand and reuse it. We also like the idea of contributing to the larger collective effort of improving these forms of publishing. For instance, we heard about a small issue, that we felt we could pay attention to in our next publication (First, then... repeat).[12] Wiki has a very useful feature to upload new versions of files. Yet, in the Paged.js preview the new file versions were not updating. In Making Matters we found a convoluted workaround by renaming every new file version and changing the image tag in the article. We now added an "overwrite images" check box[13] which replaces the image with the last uploaded version.

(Varia) Super nice to see that feature! Maybe related to the new image versions issue... We also ran into the issue that MediaWiki works with a job queue, which means that edits are held back and spread over time to make sure that the server is not overloaded. We are working around this by running the `runJobs.php` maintenance script that is shipped with the MediaWiki software, which works, but it is still a bit of a stretch.

Speaking of features, it is nice to see custom Paged.js features appearing in other projects. Amélie Dumont is currently working on a website and book for which they work with a Javascript function to "keep" your scroll after a refresh. This is such a super useful feature when working on a book with many pages!

It is nice to be reminded of the https://wiki2print.hackersanddesigners.nl/ link btw and to see the index page that you and also Martino started to work with. We decided to step away from it and integrated the index and update button(s) inside the wiki. There are two update buttons in our case: one for the text and one for the more heavier updates of media. In that way, we don't need to switch between the wiki interface and the Flask one, which felt like an interesting step towards staying within one shared environment both for writing and layout making.

(H&D) It seems we have developed ways working on publications that are quite flexible and sometimes come with surprising outcomes. What is enjoyable about this way of working is that content production and design of publications can emerge on the go and in response to different collective conditions (i.e. different timelines, dispersed geographies, janky infrastructure). In the context of the Making Matters publication this somewhat unforeseeable collective process that we usually cherish, became quite challenging and somewhat incompatible with the material realities and ways of working of a publisher. You mentioned the softness of boundaries between designing, writing and editing. These soft boundaries can also become a bit scary when working on a publication with many pages, many contributors, large edition, involving high production costs.

There are many more tensions that can be experienced when it comes to designing for print in the browser. For instance, preparing and uploading images in high resolution seems counter intuitive in the context of web and network technologies. What are other tensions that come to mind and how do you negotiate those? Do you know of printing techniques that can be regarded as more compatible with wiki-to-print workflows than others?

(Varia) Editing wiki pages in order to change how a page will look on paper feels very counter-intuitive, as well. And sometimes you forget about things that are elementary to the web, like animated GIFs. In the browser of course, you can use animated GIFs. However, before exporting PDFs we had to remove the GIFs from wiki pages, to have control over which frame would appear in print. Of course you can undo these edits after exporting and still have the GIFs on the wiki, but it still feels counter-intuitive.

Another important bottleneck in the wiki-to-print workflow is image placement. When we approach a page as a "canvas": with fixed margins, columns and font size, then the positioning of images is a very painful part of the process. The placement of an image depends on its position in the flow of the text. But of course, when you place an image on a wiki page, you have no idea where it ends up on a printed page! CSS Regions was initially introduced to work around this as it allows you to position the images and let text flow around them on a page. But unfortunately CSS Regions are not supported anymore by current browsers. Designing for print requires a different attitudes than for web. To give every image the same fixed width seems quite brutal in print, even though it may be desirable on the web, or vice versa. But the effects of resizing images one-by-one can easily cascade and you have to think about what else is affected on following pages.

Shifting away from tools that are based on 500+ years of book design traditions toward web environments that are based on network technologies has many implications. One thing we noticed is that there has to be an adjustment in how much time and attention is given to producing a PDF for print. The instantaneousness of the web affects understandings of how ready the PDF is for printing, or how PDF documents are build at all. The PDF may still need to be converted into CMYK colorspace, distilled for a particular PDF format or have imposition applied before it is sent to a printer. Even though you can see it rendered in a browser window immediately the print preview is a sort of illusion. You are not really looking at the final version of the PDF at all. A lot of steps need to be taken afterwards to match industrial printing standards.

(H&D) What processes do your pdfs undergo 'afterwards' to match printing standards?

(Varia) The Volumetric Regimes book is produced on-demand and needed to be submitted in a PDF/X-3:2002 format, which is not the format that browsers provide by default. It was quite a search to find a way to turn a PDF into that format, and in the end, LaTeX was the way out here. Another big surprise that appeared during the making of the Volumetric Regimes book was the disappearance of the emoji face on the cover.[14] We wanted to overprint this black emoji over the blue background gradient, which we managed to do with Ghostscript and OSP's PDF-utils repository.[15] The PDF looked fine, the preflight software displayed the cover fine, so we were relieved! What we did not expect though, is to receive a box of books without the author names. They disappeared! The overprinting feature was applied to all elements on the cover, including the white text. Thanks to the help of the OSPs, we figured out that we can use Scribus to open the cover PDF and apply overprint only to a particular layer of the PDF. It was such a struggle!

The Minor Tech newspaper didn't require such conversion, because we could talk with the people at Tripiti directly, who printed the newspaper for us. In that case we only converted the PDF into a grayscale colorspace, to make sure that the whole PDF was rendered in one color, which we needed because the newspaper was printed in offset with a spot colour.

One important thing to keep in mind is browser choice. We used Mozilla Firefox for the Minor Tech newspaper up until realizing (just after midnight) that images were all in very low resolution when we pressed ctrl+P. So we had to switch to Chrome to make the final PDF. Even then, Chrome rendered the PDF slightly differently and we had to make some adjustments. These are things you hardly think about until right at the end of a design process. Luckily we caught this in time!

(H&D) We loved reading your article about disappearing / eaten sentences: https://cc.vvvvvvaria.org/wiki/Octolog) and the hacks people invent to work around these issues, 

<br/><br/><br/><br/><br/><br/><br/><br/>
:-)))

Karl brought back the Minor Tech newspaper from Transmediale. We were particularly impressed by the column layout! We have experienced a lot of issues with disappearing text in columns in the (printed) First, Then... Repeat publication.[16] We felt a bit haunted by it and had agreed to NEVER use columns again! As you have experienced too text kept disappearing in the pdf, while in the browser it would still be visible!

(Varia) Yes the disappearing sentences, oof, they were so painful. What really helped was that we were being able to reach out to Paged.js through their Mattermost chat. Julien Taquet works on Paged.js and is super present on the chat. It was through him that we could start to understand the cause of this issue. It was very helpful and partially released some stress. But it did not solve the issue straight away, as it is a very sensitive one with close ties to the lower processing levels of Paged.js. So in the end we forced a page break on a very specific position in the text, which is a very scary thing to do, because it means that nothing else in the article can be changed anymore, not even a letter, to make sure that you don't end up with big gaps in the text.

(Varia) To come back to other tensions related to collective processes, what challenges did you encounter during the Making Matters production? What kind of changes were made in the last moment? How did this have an impact on the layout?

(H&D) A challenging element was to converge the expectations of making something that reflects collective experimentation, which the book spoke about in its content and on the other hand negotiate the pretty much non-negotiable linear workflow of academic publishing, very tight deadlines, and many contributors and editors involved that didn't know each other well, were dispersed and hard to reach. A more fluent reciprocal process in this context didn't seem feasible. The notion of "experimentation" as well as "collective" were taken for granted as something inherently productive, as something that leads with certainty towards results – that can be predicated and measured by the same standards and criteria as any other book. While we were open about being interested in challenging the arbitrariness of what is considered professional book design, there were moments in which our "expertise" as designers was being questioned.

By choosing this path we had implicitly decided that micro-typography may not be our biggest priority. Instead, we could pay attention to other exciting things. For instance, the publication accommodates non-linear reading, which is something that the flat structure of a wiki inspired us to work with. Cross-references are indicated with different kinds of arrows, referring to one of the seven chapter themes. The arrows occur on the chapter introduction pages, next to the respective terms, as well as in the side-bar navigation of the respective chapter categories. This allows the reader to quickly sift through the book and find related contributions in different chapters. In addition, elements in the side-bar navigation subtly poke out when indicating the occurrence of a key term in the text. We also worked with different colors per chapter and a subtle gradient that changes throughout the whole book to indicate that terms do not 'belong' solely in one place but that their place is ambiguous and contextual and their meanings can spill over into other chapters. The latter caused quite some pain in the pre-press process. The generated PDF had so many strange hidden elements and layers in it, that apparently showed up in the pre-press software and caused a lot of troubles producing the gradient. We were not in contact with the printer directly, which made this whole pre-press struggle even harder.

Making Matters was our first wiki-to-print attempt. Entering a rather large project with a publishing infrastructure that was quite new to us was exciting at first but also quite risky and stressful as there were so many moving variables.

We had to present a pretty fixed design proposal at the beginning of the process, which we then worked towards. In a way we made a promise without knowing what is possible. On the one hand it was helpful to have a goal – something to hold on to. But it was also stressful because of unexpected quirks and impossibilities. Honestly, we also worked on it in this way because there was a budget, which gave us the possibility to finally dive into this workflow. Eventually we worked at least double of what we were paid though. Haha.

After that publication we've been moving more and more towards a CSS editing process in the wiki CSS panel. That way, editing and styling become part of the same workflow, which also facilitates a more flexible approach to the design of a publication.

(Varia) How did you do it before having the CSS in the wiki CSS panel? It seems to have had a big shift to work within the wiki all together.

(H&D) The Making Matters book needed to be laid out on a baseline grid, which proved to be challenging in CSS. We used a more traditional web development approach for the styling using Sass and a library called Plumber that made it possible to have a baseline grid in CSS. Web fonts don't always have a consistent baseline so you need to account for that, for each font using positive and negative margins. Also images needed to be scaled so that they conform to the grid vertically. Headaches! This meant that editing content and applying styles were not integrated into a unified workflow regrettably. On the other hand, since the project was so CSS heavy it made sense to work on it in a more 'user-friendly' environment. There were so many small interventions! As mentioned before, the book is structured like a dictionary and each time there is a keyword in the text the illustration in the margin reacts to that, creating a little animation when you flip through the book. We used P5js with a SVG renderer for that so that the illustrations were usable in print. In the end we made a little front-end in the PDF view where we can play with some of the settings, change the version of the script, toggle a view of the baseline grid, separate layers for printing. It was a fun challenge but since then we have moved more and more towards a single workflow, and also simpler layouts.

(H&D) We are curious, how do you organize collective workflows around a wiki-to-print project? i.e. How many people are involved in the workflow? How are tasks divided? Are you working in the same space or distributed/remotely? What are appropriate timelines and budgets?

(Varia) We noticed that working closely between editors and designers is really needed to make it happen. It is also important to have a shared understanding of the limitations and possibilities of wiki software, to find ways to think along with the workflow instead of going against the grain. This needs time for conversation and reflection and it is always a bit of a struggle. It's something that we are still figuring out.

We have different experiences regarding the columns and collective conditions shaped by the workflow that also depend on the publication's content, conditions, collectivity and design choices. With Varia's occasional printed newsletter SomeTimes we were used to using columns in Octomode (a way of making PDFs from Etherpads), where all you have to worry about usually is how content flows from one column to another. To give an idea, this is the PDF of SomeTimes, Issue 1, made in Octomode:

https://varia.zone/documents/SomeTimes_number-1.pdf

The content of SomeTimes is made collectively from RSS feeds, particular hashtags on Mastodon, and posts gathered by a logbot. We have made a couple of issues by now. The simple design works. We don't need to change it much. The budget is low, we produce it on A3 on a laser printer in our space in Rotterdam. Also the timelines are always short because we're always already in the middle of many other projects while working on it. In the making of this RSS workflow, we took these limitations as starting points for the design and development process.

On the other hand, the Minor Tech newspaper was made using wiki-to-print and CSS grid during a two-day publication sprint between two people in Rotterdam and a large group that were writing and editing articles at a workshop in London. We were new to using CSS grid for paged media, and it took quite some tetris-ing to work within the constraints of the grid and a pre-determined page count. We spent a few hours before the sprint puzzling how different articles could fit, sketching in Octomode and on paper.

Aerial view of Varia working at our laptops in Rotterdam
Tetris-ing in Octomode
Tetris-ing on paper before the sprint

From the beginning, we had to accept the constraints of working within a rigid grid where it is impossible to flow articles from one page to another. It worked out in the case of the Minor Tech newspaper, as the texts were written with a maximum amount of 500 words. We ended up with a lot of small empty spaces though. We asked the writers for "cellular contributions" – content that would fit within the spaces of the newspaper that were left empty after our tetris-ing. The proposal for cellular contributions resulted in super nice quotes, pictures, and cute advertisements.


A quote included in the "cellular contributions" of the Toward a Minor Tech newspaper


In this way, the shortcomings of working towards particular formats and scenarios, such as the typically heavy content density of a newspaper or a two-day collective production sprint, became a springboard into more experimental solutions and giving flexibility to the content and layout. This came from our choice to use CSS grid and also the availability of our collaborators during this sprint. But each publication workflow varies, and calls for different approaches. The use of CSS grid had a nice side effect, but the remote way of working during the sprint did not really help to let the production of content and layout overlap and intertwine. We would do this differently next time and prioritize being together in the same space to make sure we are in conversation with each other.


... This conversation will be continued in the next bulletin.


  1. ↑ https://web.archive.org/web/20171016013843/http://www.publishinglab.nl/wiki-ebook/2017/01/27/wrapping-up-the-wiki-research/
  2. ↑ https://www.mediawiki.org/wiki/PDF_export
  3. ↑ https://hackersanddesigners.nl/s/Publishing/p/The_making_of_hackersanddesigners.nl
  4. ↑ https://hackersanddesigners.nl/s/Publishing/p/HDSA2015-documentation
  5. ↑ https://hackersanddesigners.nl/s/Publishing/p/HDSA2016-documentation
  6. ↑ https://hackersanddesigners.nl/s/Publishing/p/Heartbeat-to-print
  7. ↑ https://hackersanddesigners.nl/s/Publishing/p/On_%26_Off_the_Grid_Publication
  8. ↑ Chun, Wendy Hui Kyong, Programmed Visions: Software and Memory (Cambridge, MA, 2011; online edn, MIT Press Scholarship Online, 22 Aug. 2013),
  9. ↑ https://hackersanddesigners.nl/s/Publishing/p/Making_Matters._A_Vocabulary_of_Collective_Arts
  10. ↑ https://volumetricregimes.xyz/ http://www.data-browser.net/db08.html
  11. ↑ TITiPI: https://titipi.org/wiki/index.php/Wiki-to-pdf https://titipi.org/wiki-to-pdf/pagedjs/Infrastructural_Interactions
  12. ↑ https://hackersanddesigners.nl/s/Publishing/p/First%2C_Then..._Repeat._Workshop_Scripts_in_Practice
  13. ↑ https://wiki2print.hackersanddesigners.nl/
  14. ↑ https://volumetricregimes.xyz/index.php?title=File:VRcover.png
  15. ↑ https://gitlab.constantvzw.org/osp/tools.pdfutils
  16. ↑ https://hackersanddesigners.nl/s/Publishing/p/First%2C_Then..._Repeat._Workshop_Scripts_in_Practice
Retrieved from "http://cc.vvvvvvaria.org/wiki/index.php?title=Wiki-to-print,_Wiki2print,_Wiki-to-pdf,_and_so_on...&oldid=3543"