How Figma’s data science and user research teams weave together insights that count

This illustration shows warp and weft in weaving (Alfred Barlow, Ryj, PKM).

In weaving, you thread crosswise yarns, known as weft, over and under lengthwise yarns, known as warp, to create whole cloth. At Figma, Data Scientist Caitlin Hudon and Researcher Jennifer Sanders use warp and weft as an apt metaphor for cross-functional insights. Over several weeks, the pair wove together quantitative and qualitative learnings to reveal a complete picture of how notifications work, and how users interact with them. At the end of the process, they found where the biggest opportunity lay for improvement: Most users weren’t receiving notifications at all. To improve communication and collaboration, it would be a matter of creating new alert types and rethinking who received notifications (and when).

In Figma and FigJam, notifications trickle through a funnel with all users at the top. Users need to have notifications enabled, receive them, and view them before actually interacting with them. The activity team, which supports collaboration in Figma, didn’t know where in the funnel to focus their efforts. To chart a path forward, Caitlin investigated the funnel from a data perspective while Jennifer dove into research on an overlapping topic. They saw an opportunity to form a narrative greater than the sum of its parts. As a strategy, cross-functional insights are not unheard of, but it’s easy to underinvest in the process. By doubling down on synthesis, they learned much more than they would have alone.

“You have to have warp and weft to make things stand,” says Caitlin. “Quantitative data tells us the what. We can dig into numbers and see user behavior at scale, but those insights are more helpful when we start to understand why users are doing those things. The best way to figure that out is to actually talk to them.” Adds Jennifer, “Why can be a tricky question because people aren’t always good at explaining it.”

Since then, the activity team has begun running a series of notification experiments, and a new notification type has already been released to address the biggest pain points for users.

Building a cross-functional process

Caitlin and Jennifer kicked things off with a teamwide brainstorm in FigJam. During the session, they presented the current state of notifications including what percentage of users received, opened, and clicked on them—and set metrics goals for the quarter. Next, they prompted the team to write down strategic questions, ideas for new notification types, and what data would be helpful to know. After the meeting, Caitlin and Jennifer divided the open questions based on whose domain—data science or user research—was best equipped to find out the answer.

Kick off your own roadmap planning brainstorm with this FigJam template.

They then embarked on their individual research projects—i.e., the threads that would form the warp and weft. The two were in constant communication and relied on these rituals to stay in sync:

• An ongoing Slack channel to share discoveries and ask questions
• 30-minute work sessions to trigger notifications in their own Figma files, share insights, and hash out new developments
• 60-minute work sessions to have deeper conversations about how to interpret and understand insights and write joint share-outs

As anyone with experience in insights can tell you, questions beget questions. “As we understood our domains better specifically around notifications, we started to go one step deeper,” says Caitlin. “We started to wonder, ‘Is this person the exception or the rule?’ and ‘How do we start to quantify some of the behaviors we’re seeing?’” Adds Jennifer, “It was definitely an iterative process, and our data kept pushing us to ask more questions.”

Understanding user behavior

The iterative process allowed Caitlin and Jennifer to layer insights and reframe their findings along the way. Here’s an example:

• The numbers seemed to show that few users were engaging with email notifications.
• By contrast, nearly all the users that Jennifer interviewed reported relying heavily on email notifications to stay on top of their work. While interviews don’t yield a sample size large enough to generalize insights, what she heard gave her pause. “It just seemed weird that all of them fell into this minority bucket,” she says.
• Around the same time, Caitlin realized that the reason users weren’t interacting with email notifications was simple: Most of them weren’t actually receiving any.
• After Jennifer shared her unexpected findings, Caitlin suspected the discrepancy might have something to do with the type of user interviewed. She cut the data to examine users on teams as their own segment.

The new data revealed that unlike those on personal accounts, designers on team accounts were opening 80% of the comment notifications landing in their email inboxes. “We realized we were dealing with two very different populations, and it made everything extra nuanced,” says Jennifer.

This particular insight bubbled up into one of their top three strategic recommendations—when creating and evaluating experiments on notifications, the activity team should center on the workflows of users on team accounts because they collaborate more often, and their behaviors are a better barometer for how notifications work. (Improvements made based on their behaviors, of course, benefit all users in general.)

Finding room for improvement

The notification bell in Figma’s file browser now sits more prominently in the upper left-hand corner.

One of the questions that the pair had from the outset focused on the notification bell in Figma’s file browser—whether users were aware of it, and how often they opened it. They set out to investigate:

• While Caitlin could access data around the number and types of users clicking on the notification bell, as well as how usage compared to other notification channels like email or Slack, they didn’t know why those numbers were relatively low. Was it user awareness? Problems with discovery? The performance or value of the bell? Or workflow?
• The answer, as Jennifer’s research revealed, was workflow. The users she spoke to checked their email inboxes first thing in the morning to see new activity across all the platforms their team used and get the lay of the land, so to speak. They rarely navigated to the Figma file browser at all, preferring to keep their files open in different tabs to toggle quickly between them, and missed the bell entirely because it was hidden in the top right corner of their screen.

“It was out of sight and out of mind,” says Jennifer, “so our focus for the half shifted to moving the bell to a more noticeable spot and improving the content and reach of notifications.”

With that goal in mind, the pair pulled from the initial brainstorm list to recommend new types of notifications to bring users back to files. One of those cases came from Caitlin’s personal experience. About midway through the sprint, she and Jennifer created a FigJam to prepare for a joint presentation. Though she was a key contributor to the file, she didn’t receive any notifications about the team’s comments left in the file after the presentation because she wasn’t the owner. They brought that to the larger team as a missed opportunity.

From insights to impact

As a result, the activity team ran an experiment to send comment notifications to file editors—not just owners. Since then, this feature has been released, so collaborators can stay in the loop on files they’re invested in.

It’s one tangible improvement, and the first of many to come. Thanks to the pair’s cross-functional insights, they could communicate strategy to the larger team and meaningfully influence the product roadmap. “We had a wealth of examples that we could point to that were both quantitative data points and customer quotes,” says Caitlin. “I can throw those numbers, but that’s not nearly as clear as shortcutting that with one insight from a user saying, ‘The notification bell doesn’t fit into the way I’m using the product.’” Jennifer says, “The punch doesn’t always come from research; sometimes the punch comes from data. For example, ‘71% of users don’t receive any notifications.’”

Caitlin and Jennifer both agree that having a dedicated partner to route questions to—and who would represent your data if you weren’t in a certain meeting—was an indispensable part of the process. “Being able to point to Jennifer’s work is one way we’ve built this strong partnership,” says Caitlin. “You have to understand the entire ecosystem to correctly evaluate an insight, so having someone who has all of that context and gone deep in the area is such a boon for the quality of the work.”

While working as a cross-functional team to combine qualitative and quantitative insights is by no means a revolutionary strategy, few teams do the meticulous work of weaving a full tapestry. “It’s a corner that can be easily cut in the interest of moving fast,” says Jennifer. “Investing in the synthesis as much as we did, as quickly as we did, to ensure it would have impact—that’s what differentiates the results.”