The Power of Multimedia Experiences

Issue 54: How a practice feels forgotten today, and why it's important

Multimedia...it's a word that immediately transports me back to the 90s—the main decade of my childhood and upbringing. The definition of multimedia is any combination of digitally manipulated text, graphics, sound, animation, and video elements. This sounds rudimentary today, but at the time it was revolutionary.

I struggled with learning based on how schools taught. I honestly found it so boring. Reading, writing, and math were subjects I had to put a lot of effort into to do well in them. Basically, I was bad at the “important” subjects. However, when it came to visual learning or learning by doing, I excelled at it—classes such as art. In addition to art, the other venue for this type of learning was the computers we had at home. My father invested in technology knowing it would be a big part of how my brother and I would live in the future. One day, Dad brought home a desktop computer that would put the ENIAC to shame; a Gateway 2000 386 PC. It was the first computer we had that included a CD drive.

In addition to the new fancy computer that came in a cow-patterned bow, we upgraded to Windows 95, an operating system with so much power we didn't know what to do with it. We also got an interactive encyclopedia called Encarta 95 on CD-ROM. An encyclopedia, on a CD? You mean, no more hauling those Britanicca books? I became obsessed with multimedia. It was like learning while playing a video game. At any chance I could, I'd boot up Encarta and experience the world through reading, clicking on images that would come to life, and play video or music.

The second memorable multimedia experience came in 1994 in the form of a video game on the Sega CD: Jurassic Park. It was a point-and-click adventure game based on the 1993 blockbuster film As you play the game, you can access terminals where full-motion video (FMV) would play during parts of the game; allowing you to research and learn about the different dinosaurs you encounter on the island. Here’s one of the videos you’d encounter.

There is a great mini-documentary about the making of the video game. They built a cross-disciplinary studio in Redwood City to create multimedia games, and Jurassic Park was their first title. (Fun game: count how many times they say “multimedia”)

Multimedia in design

The practice of Multimedia influenced the type of designer I’d become in my career. As an individual contributor, I was often asked to explore and prototype experiences on core systems or new products. I would focus on high function and fidelity, and I don’t mean fidelity in terms of pixels but creating an experience that feels real by bringing in sound, animation, video, and software to tell the customer's story. Linda Dong, a former Apple designer (now back at Apple!) once shared how she used iAd Producer to build prototypes. This is one of the best examples of taking multiple types of media and combining them into a cohesive user experience prototype. Multimedia experiences are essentially storytelling at the most realistic fidelity possible.

Multimedia and remote workspaces

Adrián Mato, designer at GitHub tweeted about interactive playgrounds and their possibility, which made me reminisce about multimedia.

There are alternative ways to learn aside from step-by-step videos and documentation. One of my favorite interactive tutorials was Apple’s Swift Playgrounds; something I used when learning the new programming language that aspired to replace Objective-C.

Multimedia learning creates new possibilities of how remote teams can work as well. If you talk to any remote design team, I bet the two challenges that come up are communication and collaboration (including our own team). Design is such a collaborative practice and it takes effort to do it well remotely. To be honest, I’m still not sure if any team has cracked the code on this yet. Instead of heaps of documents, applying multimedia to documents can create a new level of interactivity. Some examples that come to mind:

  • A Product Requirements Document (PRD) that has playgrounds to interact with the core problem to solve

  • Design review documents with Loom videos built-in as a walk-through

  • A startup pitch deck that includes interactivity in the slides

  • Interactive onboarding for new employees. Instead of reading about the company vision, new hires can experience it through video, audio commentary, and interactive demos

I think interactivity between synchronous and asynchronous is a larger challenge than the hybrid office/remote problem.

Combine software tools to push multimedia practices

I’m not proclaiming that Multimedia is back, or went away. We took it for granted and lost sight of its power. As software became more powerful, it seemed like there was a shift for bulky tools that tried to encompass the full suite of the walled garden instead of smaller utilities that did certain things well. There's an opportunity for a resurgence of multimedia with our modern tools. As you consider different ways to tell stories and foster interactivity, consider combining the tools you use. Here’s an example of a prototype I built for a startup with iAd producer

Try it: create a simple multimedia experience

Multimedia experiences are simple to get started with. It’s literally using multiple types of media in one experience. Here are a few examples of what you can build to experiment:

  • Webflow site and embed video, sound, and other media or creating interactions on the site

  • Slide deck in Figma. Add Gifs and build prototypes in the slide deck itself to let people play around with the idea (I startup sent me a pitch deck like this. It was awesome)

  • Miro board or Notion space that has Loom videos of walkthroughs, favorite URLs, and other media to interact with

I'm excited about the possibilities of multimedia learning and experiences being a key focus again.


Hype Links

Design Opportunities