Through this 6-month rolling research program, 15+ stakeholders supporting the Salesforce Integrations for Slack apps were able to directly hear from customers in order to inform, support, and validate their app's product strategy.

This was a rolling research program, built from scratch, aimed to provide and systemize critical early product feedback for the newly released suite of Salesforce Integrations for Slack apps. Insights from this 6-month rolling research program impacted product roadmaps and aided in the development (and release) of various features for the suite of apps.

🙌

FYI: Insights from this research directly informed the release of various features across the Salesforce Integrations for Slack apps, impacting customers worldwide.

This research program was instrumental in shaping the release of various features by giving PMs key insight into their users. For example, I identified that users of the CRM Analytics-for-Slack app wanted the ability to share dashboards in Slack. This led to its addition to the roadmap, eventually released as a feature!

Role

UX Researcher

Team

Me (UXR), Michelle T. (Early Products Strategist), Susan L. (Director), Erika T. (Research Ops), Leila N. (Project Manager)

Timeline

6 Months (Summer to Fall 2022)

Stakeholders

15+ Product Managers & Leaders for the 12+ Salesforce Integrations for Slack Apps

Methods

Surveys, Rolling Research

Tools

Slack Workflows, Google Forms, Google Sheets, Miro), Miro

PROBLEM STATEMENT

By Summer 2022, the Salesforce Integrations for Slack Apps were quickly rolling out to the public. This was a critical time to collect some early customer feedback.

Post Salesforce acquisition of Slack, our internal product teams were working on building integrations between the platforms so that customers could use their Salesforce solutions in a more powerful, impactful way through Slack. How was this actualized? By June 2022, our product teams were pushing out 10+ Slack apps to Beta/GA on a rolling basis, bundled into what we called the Salesforce Integrations for Slack apps.

As a result, our research team was looped in to provide research support for the apps.

With the recent app releases, there was a need for our research team to take an active role in delivering quick insights from customers. Gathering product feedback in these early release stages (the Beta/General Availability stages) was critical to support adoption numbers and aid in the development of further iterations of the apps.

RESEARCH OBJECTIVES

Thus, our research team was pulled in to deliver on a need for...

A research program built for the suite of Salesforce Integrations for Slack apps with the following goals: centralize feedback and metrics across the apps; enable product teams to spot trends; collect feedback at scale as app usage grows; reduce the burden on Product Managers to gather their own feedback.

First, let's outline the overall programmatic goals.

🗄 Systematic

Systemize data collection across the 12 apps.

🧑🏻 Self-Serve

Give Product Managers & Teams the ability to view customer feedback on their own.

⭕ Centralized
‍
Ensure visibility into themes, trends, and insights across the apps for Product Leaders.

🔁 Repeatable
‍
Establish the basis for future early products within Salesforce to replicate such a program.

📈 Scalable
‍
Collect feedback at scale as app usage grows and more apps are added to the suite.

Here are specific research questions (across apps and within individual apps).

Cross-App Questions

(Product Leaders are especially interested)
‍

What are recurring themes (ex: Setup Challenges) across the apps?
What factors are encouraging or limiting adoption of the apps?
How is the multi-app experience; how do users use multiple apps for their work, and are we delivering on an integrated Salesforce & Slack?

Individual App Questions

(Product Managers are especially interested)
‍

Where are users facing pain-points within the current [X] app and why?
Where are there opportunity areas for [X] app?
How do users perceive the ease of use, value, and satisfaction of [X] app?
How well is [Y] feature delivering on allowing users to accomplish [Z] goal?

RESEARCH PROGRAM TIMELINE

Over 6 months, I cycled through this research process.

Every 2 months, I cycled through this process. We repeated this over 6 months, delivering 3 (June, Aug, Oct) insight decks that I shared with stakeholders. In the first two months, I collaborated heavily with my early products strategy partner to ensure proper programmatic setup, while the later 4 months consisted solely of myself and Ops.

PART 1: SURVEY DESIGN

We settled on collecting data through surveys, aiming to measure ease of use, value, and satisfaction for 3-5 feature areas from each of the 12 apps.

Before building surveys, I started the first two weeks of each research round by meeting individually with each app's PM. I asked them to pinpoint 3 to 5 key features of their app they wanted feedback on, as well as whether they had any additional research inquiries they wished to explore.

In terms of survey configuration, I structured each survey (identical in composition) in the following way: for each of the 3-5 features, participants rated on a Likert Scale whether the feature was "easy to use" (Experience) and "met their requirements" (Value). At the end of the survey, I integrated an NPS question to gauge overall satisfaction.

🤔

Q&A: Why decide on surveys, instead of other methods?

There were multiple apps to support (12+), multiple stakeholders to involve (15+), and the need to collect data at a high volume in order to conduct sound analysis. In addition, surveys...

Allowed data to be collected easily, quickly, and at scale
Ensured that set metrics were being collected across the apps
Allowed us to compare data not only between apps (ex: Sales app vs. Service app), but across timeframes (May vs. August)

PART 2: SURVEY OUTREACH

But who did these surveys reach, and how? Due to challenges with less than-ideal response rates due to low adoption of the apps, we had to get a little creative...

Recruitment proved to be a greater hurdle than expected — but thankfully, as researchers, we're pros when navigating ambiguity! Two challenges arose in this step which guided our eventual solution for survey outreach:

🚨

The apps were moving to GA faster than expected. We no longer had access to a customer list!

While originally we hoped to send out surveys to customers participating in Beta, the apps moved quickly to GA. But with quick stakeholder collaboration, we found a working solution: surveys on app homepages.

Uh oh — we're seeing less than ideal survey response rates due to low adoption.

We needed more data in order to be able to draw sound insights and see changes from month to month.

The solution? Engaging feedback from internal employees proved ideal, boosting survey response rates and providing easy access to valuable insights.

For these apps, it was worthwhile to gather both internal and external feedback. After discussions with stakeholders, it was clear that two groups of people within Salesforce & Slack could provide critical feedback:

👤 Internal Users

Salesforce & Slack employees who use the apps in their day-to-day work (aka, end-users).

🌾 "Folks-in-the-Field"

Salesforce & Slack teams with direct relationships (like Sales, Customer Success, etc.) to customers who might be using and implementing the apps, and would have knowledge of customer perception, use, and challenges.

To capture the voices of these groups, I set up identical surveys in Slack Workflows (internal-facing) and Google Form surveys (external-facing).

The surveys between Slack Workflows and Google Forms were identical, with internal users seeing the Slack Workflow version and external customers seeing the Google Form version.

🤔

Q&A: Why leverage Slack Workflows (instead of GForms) to survey internal employees?

Salesforce employees already leverage Slack Workflows to log bugs, deliver product feedback, etc.
Slack Workflows were easily accessible and findable to all employees at Slack and Salesforce. Much better than chasing down a Google Form link (where did it go again...)!
After all, we are talking about a Slack product — and leveraging Slack Workflows to conduct research helped inspire advanced Slack use and move teams towards a Slack-first future.

BUILDING SELF-SERVE DATA COLLECTION SHEETS

Product Managers also wanted to see the data on their own time; building a central data collection sheet ensured transparency and ownership.

A programmatic goal of this research program was to easily give Product Managers/Product Teams the ability to view customer feedback on their own. To deliver on this goal, survey responses from the Slack Workflows and Google Forms flowed into a shared Google Sheet, where anyone could view the customer feedback coming in anytime and anywhere.

WHAT WE DELIVERED

Deliverable #1: Bi-Monthly Insight Deck Share-Out (of Cross-App Insights)

Delivered to Product Leaders and Product Teams at the Monthly Review Meeting‍
Every two months, I presented a research deck with trends and insights observed across the apps. These findings touched on topics relevant for multiple apps, such as themes like adoption, the setup process, enablement, and more.

Deliverable #2: Bi-Monthly App Deep-Dive Reports (of Individual App Insights)

Delivered to Product Managers of the Top Four Adopted Apps in 1:1 Meetings
For our top adopted apps (Sales Cloud, Service Cloud, CRM Analytics, and Platform) I delivered detailed deep-dive reports and met with each PM to answer any questions and gauge further research questions.

Deliverable #3: Self-Serve Data Collection G-Sheet

Accessible to Relevant Internal Teams
Survey responses from the Slack Workflows and Google Forms flowed into a shared Google Sheet, where anyone could view the customer feedback coming in anytime, whenever they'd like.

🤔

Q&A: Ooph — why have so many variations of deliverables?!

I quickly learned that my stakeholders had different questions and goals, yet limited time. To deliver the most impact, I needed to tailor insights of varying altitudes. For our Product Leaders, I delivered general, cross-app trends, and for our individual app Product Managers, I delivered app feature-specific insights.

SAMPLE INSIGHT

Here's an insight we gathered on what was happening across the apps...

Please reach out to cynthia_chen@berkeley.edu for more insights.

SAMPLE INSIGHT: CROSS-APP INSIGHT

Across the suite of Salesforce Integrations for Slack apps, we're hearing that we need to reduce the amount of manual configurations needed to successfully utilize these apps.

Customers are saying that we need to reduce the amount of manual configuration needed for Admins to successfully manage the apps for their end users. This issue grows in severity at scale — imagine hundreds of end-users to manage and hundreds of errors to debug. The good news is: Admins aren’t asking for crazy, “out-there” configurations! They’re asking for simple things like an approval workflow or the ability to link a record to an existing Slack channel.

💬

"A significant pain point from [a large Salesforce customer] is that there isn’t an org-wide install option. Every time the app is opened in a workspace for the first time, there is some configuration that needs to happen. The instructions for what to do aren’t very clear. The current solution isn’t scalable across an organization like [a large Salesforce customer] where they have thousands of workspaces and team members can be spread across many." — Account Executive for unnamed Salesforce customer

💡

So...what does this mean? Challenges at scale may be a problem for more than just this one customer. There’s opportunity for further research to assess the impact of compounding challenges for customers with large user bases and cross-product adoption.

RESEARCH IMPACT

Insights from this 6-month rolling research program impacted product roadmaps and aided in the development (and release) of various features for the suite of apps.

This research program impacted multiple product teams for the apps and established the foundations for an early product feedback program within Salesforce. At the end of this 6-month run, stakeholders requested a follow-up study into the topics of Adoption, Setup, and Future-Priorities to build on the insights gathered through this research.

LEARNINGS & TAKEAWAYS

While I had experience leading individual research studies alone, this was my first time leading & managing a longer research program at scale. Here are some takeaways...

Tailoring insights of varying altitudes ensures that the right insights get to the right stakeholders. Working with multiple stakeholders with different priorities and goals meant I needed to tailor insights of varying altitudes. This wasn’t a one-size-fits-all! To keep stakeholders engaged, I learned to keep track of what they were interested in and practiced how to deliver for multiple needs.

Proper collaboration can't come without proper team alignment. As a part of this research program, I had to lead a research team of four and manage a vendor. While I had experience conducting end-to-end research studies on my own, it was a different ball game to manage something of this size! Here's one of many learnings — to work efficiently as a team, it was important to get everyone on the same page.