Design Systems: Pilots & Scorecards

Pilots are a great way to start a design system.

Published on

Around 4 minutes to read

Pilots are one of the best ways to put your design system through its paces, especially before the design system even gets to a v1. Like television pilots help test audience reactions to a series concept without investing significant resources to create the whole thing, application pilots are a good foundation for ensuring your design system’s design and code are battle-tested.

Here is how my teams and I identify great pilot candidates.

First, we want to know what kinds of digital products a design system should help our client to make. We’ll ask them to tee up as many product presentations as they can muster. They’ll usually do this by either generating a list of product owners that tend to be early adopters/risk takers, or they’ll issue an open call for willing participants.

We review anywhere from dozens to hundreds of existing products to get a good lay of the land for what the organization makes regularly and supports. This might be walking through products in person or on a screenshare with a product owner, asking questions and exploring as many of the hidden crevasses as possible. It also could be clicking through public or private apps ourselves, inventorying what’s common.

Simultaneously, we’ll also do an interface inventory, taking note of the frequency of usage of common components.

We’ll then try and reverse-engineer where our Pareto principle comes into play: what 20% can we design and build that can assuage 80% of the wheel reinvention pain?

There’s a sweet spot for great pilot candidates after planning for the pilot has begun but before it gets designed or built. If a product isn’t far enough along in planning, we likely don’t know enough about it to say whether it’ll make for a good pilot or not. But if it’s already in the process of being created or recreated, it’s probably too far along to be able to integrate parts—read: component design, patterns, and/or working code—from the design system without some amount of refactor, which teams in need of a design system often can’t afford.

I’m talking specifically about products that are being built from scratch or reimagined. However, it’s entirely possible that you may have a product already built that’s clearly a good foundation for a design system. If that’s the case, no need to pilot; instead, extract and abstract the appropriate components and patterns you need. Mobile designer Josh Clark describes this in more detail in his article, “The Most Exciting Design Systems are Boring.”

Once we find a some good potential candidates in that sweet spot, there’s a set of criteria we use to determine a pilot’s potential efficacy:

  1. Potential for common components. Does this pilot have many components that can be reused in other products?
  2. Potential for common patterns. Does this pilot have many patterns that can be reused in other products?
  3. High-value elements. Even if uncommon, is there a component or pattern with high-business value that is the heart of this project? We’re talking elements that are integral to a flow or audience that has unusually high value for the organization.
  4. Technical feasibility. How simple is a technical implementation of the design system? Is a large refactor required?
  5. Available champion. Will someone working on this product see it through and celebrate/evangelize using the design system (and even contributing back to it)?
  6. Scope. Is this work accomplishable in our pilot timeframe of [3–4 weeks] (insert your timing here)?
  7. Technical independence. Is the work decoupled enough from other legacy design and code that there are clear start and end points?
  8. Marketing potential. Will this work excite others to use the design system?

You might already see where we could go with this. Using a simple point system where 1 indicates a low match and 10 indicates high efficacy, you could create a Pilot Scorecard that shows you which products to tackle in priority order:

Product #1Product #2Product #3
Common components potential1035
Common patterns potential244
High-value elements832
Technical feasibility315
Available champion548
Scope838
Technical independence1077
Marketing potential462
Totals503141
Averages6.253.885.13

Starting with pilots before you have anything in your design system allows invention to happen in the products. In the next pass of using your design system, you can cook with your new ingredients to make sure you have the right pieces in place.

Have you used pilots in your design systems journeys? How have they helped you?

Thanks to Brad Frost and Josh Clark for helping inspire this insight and reviewing this article.

Read Next

Bad News. Good News. So What?

Join 42,700+ subscribers to the weekly Dan Mall Teaches newsletter. I promise to keep my communication light and valuable!