Collages in the Cloud

by Francisco Canas

An integral part of the public library experience is walking into a branch and being greeted by a bookshelf filled with staff reading recommendations: Top Ten Genre lists, Books You Might Like if You Liked X lists, Cat-Related Historical Fiction Reader’s Choice lists, Topic Guides, etc. Here at BiblioCommons, we try to emulate that same experience digitally by giving library staff and patrons the tools they need to create, share, and discover lists of hand-curated book recommendations.

With that digital experience in mind, the BiblioCommons Core Features team undertook the task of rebuilding our list creation tools from scratch. One particular goal of the project was to improve the visual appeal of our digital lists: to make them as attractive and engaging as the real-life displays that library staff use as the centrepieces of their physical library spaces. In practice, this involved building a feature that produces eye-catching collage images from the covers of the books included in our users’ lists.

At the last BiblioCommons hackathon, two of my colleagues demo-ed an intriguing proof-of-concept project that used Amazon’s Lambda compute service to run ImageMagick and create collages for a given list of book cover images. We decided to take this initial idea and expand on it by improving the visual appeal of the collages and serving them to our users in near real-time as they added books to their recommendation lists.

Collages

Many digital services offer list creation tools for their users, and most of these services employ some form of cover image as part of their visual design. So while the general idea is nothing new, we wanted our list collages to be just a little more compelling than the typical four-image grid you see on many websites.

We also wanted collages for different kinds of lists to be visually distinct so that our users could distinguish between various list categories (ranked lists, topic guides, etc.) from the cover image. Our user experience designers came up with a variety of designs and it was my job to build a system that would dynamically generate these collages from the book cover images for a user’s list.

ImageMagick

ImageMagick (IM) is a popular command-line application used to manipulate and create images in various formats. IM offers an extensive and flexible API, and can produce some pretty complex and wonderful images once you learn its intricacies.

Our first challenge was to deal with the unpredictable nature of our users’ lists: the ImageMagick script needs to produce compelling collages with any number of book cover images of arbitrary dimension. This unpredictability ruled out using a static template or script. We decided to address this challenge by dynamically generating custom IM scripts based on the contents of each list. To do this, we built a NodeJS service that accepts a list collage request, generated the custom script, and ran it on ImageMagick to produce the collage.

The list collage request specifies all the necessary parameters and data needed to construct the desired layout for that list:

1. The category of list to generate the collage for.

2. The anchor image (a specific book cover that dominates the layout), if any.

3. A list of URLs to all the remaining book cover images in the list.

Upon request, our service downloads the book cover images and performs several tasks:

1. Detects invalid files, blank images, or images that are too low in resolution.

2. Extracts each of the valid images’ dimensions.

3. Computes all of the layout geometries based on the above dimensions, the number of valid images, the desired layout, anchor bib, etc.

4. Produces a unique ImageMagick script that, given the list of images as inputs, will create the desired collage.

5. Runs the custom script in a child process using the simple-imagemagick.

6. Compresses the resulting collage image using pngquant.

7. Saves the resulting compressed collage to Amazon S3 where it can be consumed by our application.

A few samples of the collages produced by our script:

In the Cloud

Bringing the list collage service into production proved a very different kind of challenge. We wanted the images to be generated in near real time, so that a user can see the collage update almost immediately while adding books to their lists.

We currently have about a half million user lists in production, and roughly 150 lists are created by public library staff and patrons every day. While the number of lists created daily isn’t unmanageable, we wanted a system that could easily scale to regenerate covers for all of our lists any time our designers made updates to the collage designs. As with our other production-ready features, we wanted monitoring, logging, testing, and instrumentation built into the service. AWS Lambda proved to be a very effective choice given these requirements.

Lambda is a managed, serverless computation service: your code executes on demand and scales automatically based on need. Most days, our collage generation function receives a few hundred requests and we pay only for the near-negligible amount of computation time used by this small number of requests. When we need to generate a half million images in bulk, the Lambda service automatically scales to handle the dramatically increased amount of traffic with no reconfiguration or further action required on our part.

Architecture

The new BiblioCommons list editor is built in React and served to the client by a JRuby on Rails application. That Rails application aggregates all the needed data from our backend Java services (bibliographic data, availability data, user data, book cover images, etc.) and serves this to the list editor. Book jacket cover images are fetched from a variety of third party services in real time.

We obviously needed a place to run the NodeJS collage server and the (somewhat) computationally-expensive ImageMagick script: This is where AWS Lambda came in. Lambda also provides us with monitoring, dashboards, and logging right out of the box so we can keep an eye on usage and error statistics, error messages, etc.

We also needed a place to store the collage images, and a content distribution network to serve them to our users worldwide. For this, we chose AWS’ S3 and Cloudfront, respectively.

All in all, the collage system is made up of the following independent components:

1. The React client-side list editor.

2. A JRuby on Rails presentation layer application.

3. Java back-end API services.

4. The AWS Lambda function running the ImageMagick Scripts.

5. S3 for Storing the generated images.

6. Cloudfront for a Content Distribution Network (CDN).

The general workflow for our collage system is described by the diagram above. It goes something like this:

1. Create List: A user saves their list. Our back-end list service bundles together all of the important metadata for that list: the layout type, number of cover images, and a list of URLs to the cover images.

2. Request Collage Generation: This metadata bundle is placed into the SNS message payload and sent asynchronously to AWS.

3. Trigger Script: The AWS SQS service receives the message and forwards it to the Lambda service. This triggers our image processing script, which performs the task of downloading all of the required book covers, generating the custom IM script, and running that script to produce a collage.

4. Save Collage: The collage is saved to S3, which sits behind our Cloudfront cache so that the application can consume the cached collage images.

Meanwhile, the client side waits until the new collage image is available on S3 and displays a preview to the user as they work on their book list. To the delight of our patrons and library staff, the entire workflow from message to finished collage completes within roughly 3 seconds 99% of the time.

An alpha version of our React list editor was deployed for library staff testers earlier in the year and this early version included the collage generation service. Our participating librarian users were able to create many fine examples using our initial collage designs:

We took the opportunity to gather as much librarian feedback as we could after this alpha release and tasked our designers with updating the collages to be more reusable, attractive, and permanent. The new editor and list collage designs are going public today as part of our latest major release and we will soon be regenerating new collages to bring all of our existing lists to life.

We can’t wait to see what our library staff and patrons come up with! In the meantime, a few more interesting lists featuring our newest collage designs:

Francisco is a Data and Analytics Engineer at BiblioCommons. His time is split between building discovery and content creation features for the BiblioCore product and advancing our analytics infrastructure.