Visualizing Archer

Don’t like reading? Prefer GIFs? Skip this whole article and visit the site instead.

Archer! I like Archer — it’s a comedy full of snark and sarcasm and violence and jerks, which is, like, 70% me. But Mara Averick really likes Archer, enough to create all the Archer gifs you ever saw on the Internet — all of them. She likes Archer so much she explains data science methods with Pam Poovey quotes and if you don’t know who Pam Poovey is, she’s kind of like Mara Averick if instead of being into hermeneutics and data science, she was into berserker underground fighting and street racing. And if you don’t know who Mara is, then there’s something wrong with you.

So when Mara and I started talking about creating what would become this amazing data visualization exploration of Archer last year, I was totally on board. I love structural analysis of narrative and I’m not the only one — it’s super popular on Youtube and TVTropes. Some stans ship, some stans trope. This isn’t new. Structural analysis of literature has a long history, from the Russian formalists like Propp to Joseph Campbell’s monomyth up to the latest digital humanities scholarship like the Stanford Literary Lab.

I don’t have any academic background in that kind of thing, so I was more interested in looking at the structure of how characters appeared in groups, kind of like the XKCD narrative maps but with less a focus on the structure of the narrative and more on the structure of the show as media. I didn’t have a particular structure of visual layout in mind. But from what I knew of Archer there were classic television beats along with patterns of flipping back and forth from characters in one scene to characters in another scene moving on mirror narratives that I thought would be exciting to see visually.

DATA

To get there, you need data. Unfortunately the state of cartoon facial recognition and automatic analysis is abysmal, at least insofar as what I could utilize. Microsoft’s Video Indexer API might mean you could do this without all the handwork, at least for boring shows starring real people, but we didn’t have that. So we had to use subtitles, painstakingly annotated for scene and content. The subtitles were available for all 8 seasons but they were in different formats and didn’t have scene information like a script does.

So we added that. By hand. Not using some cool computer vision or text processing. By hand. The results look like subtitles broken up by little pipe-delimited scene metadata listing the participants (whether they spoke or not), the number of people who died in the scene, and any tags indicating whether the scene was a flashback or involved drinking or drugs or had a few other themes we wanted to keep track of.

|scene|participants:archer,malory,trexler|deaths:0|tags:drinking|
MALORY: Yes, ODIN.
ARCHER: Why would ODIN wanna buy--? Oh, my God, is Len Trexler behind this?
MALORY: He made me a very attractive offer.
ARCHER: Yeah, I bet. That's because he wants to-- Bleh!
MALORY: [SIGHS]

Since releasing Beyond the Tip, the most common request I get is to share the ML technique I used to identify scene breaks. Whenever you tell someone you painstakingly annotated something by hand they grimace and get uncomfortable like you told them you enjoy thrash metal. But annotating is valuable. Reading is okay. Thinking about things is okay. It’s not repetitive, it’s what we used to do for fun before we turned into horrible efficiency experts in our everyday lives. Would I like you have a way to process unstructured or semi-structured data to reveal more of its structure? Of course. But do I feel bad that I had to do a close reading of Archer to make a cool data visualization of it? What a dumb question.

Because we used subtitles, what we showed was based on the amount of dialogue in a scene, and not the true time length of the scene. So a long scene with no dialogue but lots of action, like the jeep crash in the jungle, will appear as a small scene in these diagrams.

This wasn’t by design, it was a constraint, and I think it would have been great to contrast low-dialogue, high-action scenes with scenes where the emphasis was on exposition. Still, it’s an anachronistic approach to treat Archer as a text to be read, rather than a combination of action and movement and delivery.

SKETCH

My sketches are always code-based iterations. I’ve never been much for pencil and paper, and with Archer it was no different. This is something I’ve started working on lately, because it puts you into a different mode and makes you less attached to your intermediary products. But when I was making these visualizations I was playing with code from the very beginning. The earliest representation of episodes came was a radial view of each episode. Characters were represented as colored ribbons with the amount of dialogue in each scene shown with thickness as well as the order (more dialogue placed a character higher on the stack).

This is a fun and compact way to visualize the data. Polar layouts aren’t tremendously readable but you could scan it to see who the main players in a scene were and see if they maintained that prominence throughout. It’s pretty clear above on the left (Season 1, Episode 1) that Archer (blue) shows up a lot in Archer. On the right, you can see that that there are common groupings of characters. The little “ears” on each bump are the result of weird interpolations doing weird things and got sorted out early on.

There were several improvements in later versions, the most obvious being the use of an interrupted chart rather than allowing the path to dwindle to nothingness in a scene where the character didn’t appear. That meant that if a character didn’t show up in the next scene, the band representing them would be cut, in contrast to the initial implementation above, where a character band is present at each scene but is tiny and thin and on the bottom in scenes where the character does not appear.

I loved the idea of signaling how many people died in a scene by stacking little chits (like you see on the left) for scenes with deaths in them (the green bars above, including the twenty or so deaths in the snowmobile scene). But the effect took up a lot of space, and while it emphasized the deaths in a scene, it dominated the visualization. It would eventually be replaced by a skull and crossbones along with the number of deaths in scenes with a death.

While these were compact, they didn’t resemble the original inspiration for the work, those XKCD story charts. To do that, I stitched together characters who were in a scene, as well as characters who were off-scene, and then put a little padding between each group.

Even with the obvious flaws in the results, I knew this would work. Archer, for instance, is striking in his visual presence in this graph, and the groups that appear with him and then come back singly or in that original group reinforced my intuition that off-screen groups were a meaningful construct.

As with the radial diagrams, I found the use of interrupted paths to be key to cleaning up the diagram. Characters that head off-screen but never show up again in the episode need to see their paths terminated at that point. The updated charts below now show the band representing each character only starting when that character makes his or her first appearance and ends with the last appearance.

At this point I was deep in the technical solution and had lost track of the actual meaning-making that I wanted to get to with these pieces. That happens to me a lot. I enjoy the challenge of writing layouts that organize data in some novel way and by the time I get it working I’ve forgotten what the point of it all was.

I had two fundamental classes of characters — on-screen and off-screen — but I wasn’t doing anything to indicate which was which other than a little padding from the first group. So back to tweaking the layout to move the on-screen characters above a baseline and all off-screen characters below. This meant I needed a more sophisticated sorting algorithm (since I was counting “out” from the baseline).

The ultimate effect imbued the diagram with a “tip of the iceberg” quality. That even when you were seeing only one or two characters on-screen, that you could see beneath a roiling mass of various groups of characters who just exited or would reappear as a gang, Scooby Doo-style, in various reformulations. This wasn’t the end of improvements, but it was the end of major work on the diagram. After that there were just two big additions:

• moar gifs, for every character aligned with when they first appeared in that episode
• A glow filter so when you hover over the paths you can see it highlighted. The glow filter came out of my own experiments with Nadieh’s gooey filters and I love how it ended up looking paired with fading out the other paths.

There was an abortive attempt to show a character view, comparing the tags of scenes a character was in with another character to see if they shared scenes with particular tags.

It didn’t work.

Not that it wasn’t amusing.

But the level of detail of the tags and metadata that we’d provided for the work just didn’t allow for interesting exploration at the character level. It was fine when it was “summed up” into episodes and seasons but when you tried to compare Baby Seamus with Archer, as we do on the left, it was mostly just funny because you got to see the tags we’d used. And most of those tags were expunged anyway in trying to tighten up the design, anyway, but even if they hadn’t been, it only highlighted that the tags were really about scenes and not individual characters.

The beeswarm plot was a simple and easy win. While individual level details based on tags were pretty shoddy, plotting their dialogue made for pretty interesting reading. The only innovation was connecting the character’s points and adding the episode number.

CODE

The Archer data visualization is a React app. I wrote an entire article about wiring up D3 and React and a whole chapter of my book D3.js in Action on that subject, too. So I’m not going to get too far into the code. At some point Mara and I expected to open source this, and maybe the layout can be generalized, but the pattern of the application followed the method where you let React handle creating the components and then you hand references to the nodes over to D3 to do more old-fashioned enter-exit-update style coding with D3. I did that because this was meant to be highly interactive and push a lot of animated pieces all over. I also did it because I hadn’t started working on Semiotic. The application itself uses React-Create-App because it’s super easy.

Writing the bump layout for the paths was an interesting exercise for me, because I’d worked a bit with bump area layouts before but never to the point where I wanted to do more than stack things. Grouping together the off-screen characters and adjusting those groups so they had a buffer distance was probably the most fun aspect of the visualizations. This shows some of the issues with out-of-the-box layouts and functions, if I’d focused entirely on trying to fit within, say, the d3.stack paradigm, I wouldn’t have been able to highlight how off-screen characters maintained their group membership.

I also struggled with how to size the charts for display, originally intending all charts to have some fixed size and width and charts with few characters would have fat lines and charts with many characters would have skinny lines. Maybe it’s just me, but I feel like that’s what D3 taught me, and it took me a while to finally realize that the size of the chart should reflect the number of characters, and so its ultimate size on-screen would be variable and everything else would have to adjust. Accustomed as I was to layouts have a size parameter and building for dashboard-like products, it felt radical to have an element that resized itself depending on how much data to display. This is probably not so radical an idea for folks more accustomed to long-form scrollytelling-like data visualization.

Once I had data sorted into that stacked form, it could easily be passed to the linear layout or the radial one, which I call a “splat” under the hood. It takes advantage of d3.radialArea to draw the paths in a radial manner and uses the defined functionality to render the gaps when characters are off-screen.

Here’s what the drawing function looks like:

const stackArea = radialArea()
   .angle((d, i) => xScale(i) / 1200 * Math.PI * 2)
   .innerRadius(d => yScale(d.y1))
   .outerRadius(d => yScale(d.y2))
   .defined(d => d.y1 — d.y2 !== 0)  
   .curve(curveCatmullRom.alpha(0.75))

Pretty straightforward. Because the data is projected for a zero to 1200 size chart in linear mode, I need to reproject it into radians. The defined statement just says if the band is 0 size that means no dialogue and no need to display it. To show the little skull icons (for scenes where people die) I instantiate a different arc with a radius based on the maximum position in that scene and use d3.arc's centroid function to hover it over the associated scene.

const deathArc = arc()
   .innerRadius(d => yScale(d.y2) + 10)
   .outerRadius(d => yScale(d.y2) + 15)g.selectAll("g.deaths")
   .each(function (d,i) {
   if (d.deaths > 0) {
      const bumpMax = max(bumpedData.map(p => p.data[i * 3].y2))
      const arcCenter = deathArc.centroid({
         startAngle: xScale(i * 3 + 1) / 1200 * Math.PI * 2,  
         endAngle: xScale(i * 3 + 1) / 1200 * Math.PI * 2,   
         y2: bumpMax, 
         r: d.deaths * 2
      }) 
      
      const skullG = select(this)
         .append("g")
         .attr("class", "skull")
         .attr("transform", "translate(" + (arcCenter[0] - 7.5) + "," + (arcCenter[1] - 7.5) + ") rotate(" + (xScale(i * 3 + 1) / 1200 * Math.PI * 2 * 57.2958) + ",7.5,7.5)")      skullG
         .append("image")
         .attr("xlink:href", skull)
         .attr("width", 15)
         .attr("height", 15)
         .attr("y", -2)
})

Annotated Data Visualization

I was extremely happy about the data visualization pieces in Beyond the Tip. I think they’re fun, novel and information dense. But they are definitely not easy to read. So I decided to use Susie Lu’s annotation library (d3-annotation if you’re using straight D3 or react-annotation if you’re in React) to diagram them.

Radial Diagram

Each diagram surrounds an animated GIF representing that episode. In the interior ring are the tags for each scene, followed by a stack of dialogue and participants in the scene, and topped with cute little skull and crossbones if someone died in that scene that gets a little number if more than one person died.

Character Paths

The linear diagrams encode the same data, but include characters that go off-screen if they later appear on-screen. Below the chart are GIFs of individual characters, positioned when they first appear in the episode, which are floated up onto the chart when you highlight the character’s path.

Measuring Effectiveness

The whole process of building out an entire application for analyzing Archer highlighted how important collaboration is to the creation of any sort of really interesting work. If it hadn’t been for Mara’s deep understanding of the subject matter, her work analyzing and structuring it and, well, creating a million GIFs, I wouldn’t have ever ended up creating the kind and complexity of visual modes to represent it.

Beyond the Tip is a hard thing to dive into. It combines abstract data visualization forms, a weird perspective into television storytelling and a complete lack of some kind of narrative positioning the reader within it. I think, like many data visualization explorations that aren’t constrained by an anything so mundane as whether or not someone could actually read the thing, I evaluated it on whether or not it improved my experience of the subject matter, and secondarily whether or not it was technically and aesthetically sophisticated. That’s a weird mode to be in, and very different from the design-centered and impact-oriented modes we often put ourselves in to make data visualization for analytical tools, communication pieces or news stories. But it’s not a bad thing to work for an audience of one (or in this case, two) and release a work like that into the world and see if anyone else likes it.

From that perspective, I love it. Despite working at a TV company, I don’t watch much TV, but I love the structure of stories and how storytelling and performance can be measured and shown. I guess it’s no surprise that I did a data visualization deep dive into TV Tropes seven years ago. To me, these structural representations of stories and genres and universes and canon are at least as interesting as the performance of the piece itself. I’m surprised, especially given how popular film analysis is on Youtube and how many fan sites there are for understanding and theorizing about worlds, that this isn’t being done by the people making TV shows and movies.

That said, when I look at the complexity of the shapes and the spartan explanations for how to read them, I think it could have been a bit more user friendly. I think we could all learn a few things from Algernop Krieger and focus less on whether we can make something and more on what someone might think about the thing we made.

Thanks to Shirley Wu for help crafting this piece.