A Data-Informed Approach to Support Engineering — Episode 1
This series presents our journey of transforming into an objective and data-informed Support Engineering team. Our journey is divided into several phases, each built upon previous ones, describing the processes and tools that were implemented to facilitate our goals.
To ensure that a team or organization continues to improve, it is important for them to understand the impact of their daily activities and define a clear measure of performance. Being able to justify one’s intuitions or make informed decisions using measurable and actionable insights has a whole different level of comfort attached.
During the course of this 3-part series, we will cover our approach toward this transformation elaborating on the tools and processes that we implemented to achieve our goals. We‘ll also share the challenges that we faced in the process and how we overcame them.
Without wasting much time then, here’s what we would cover in this article:
- Being Data-Driven vs Data-Informed
- Why did we decide to embark on this journey?
- How did we collect and centralize the required data on our bugs and questions?
Being Data-Driven vs Data-Informed
Being data-driven is one of the rapidly growing trends in the SaaS industry. Most teams in this space have attained some level of data visualization allowing them to keep an eye on their core metrics and optimize their decisions. Carl Anderson¹ defines being data-driven as the process of building tools, abilities, and, most crucially, a culture that acts on data.
This definition was quite interesting since we seem to give “data” all the power in this scenario and as such base a huge part of the decision-making process on the available data.
Although this seems effective and would provide promising results in many scenarios, such a binary approach does come up with its own set of limitations. One of the most concerning in our case was the loss of the human touch in the process. In a human-centric field such as customer experience, being solely data-driven could cause us to lose certain qualitative insights that we could otherwise leverage through sources like user research, experience, and personal insights.
These concerns were what pushed us to kick it up a notch as we opted for a more involved and human-centric alternative to the data-driven mindset, which is commonly known as the data-informed approach. At this point, I hear you ask, “are you just throwing around buzzwords to sound cool?”. Yes, but there are interesting differences between these two mindsets which are described in an insightful chart by Amplitude² provided below:
Based on the figure above, we can summarize the two approaches as follows:
- a data-driven approach utilizes definite success metrics to choose the best way forward in a given scenario.
- a data-informed approach leverages data to provide a better understanding of core indicators to the entire team and complement chosen strategies to best determine their corresponding impact through measurable insights.
Opting for a data-informed approach allowed us to continue leveraging the experience of our team in the decision-making process while using the available data to complement and provide measurable indicators ensuring a clear understanding of the past and potential impact of chosen decisions.
Prelude: The Why
Before diving into the actual phases of our journey, let us first share what pushed us towards this transformation.
Challenging existing processes
As a newly created Support Engineering team, our biggest challenge was including ourselves at the center of the bug lifecycle. This meant being able to allow for clear methods of collaboration and communication in order to effectively bridge the divide between the Support and Tech teams.
At the time, our teams were quite used to transmitting requests over Slack with only the tech team using a dedicated ticket board. Although this gave a positive appearance of faster response times, it caused the teams to deal with a number of drawbacks mainly consisting of:
- A great deal of context switching for the tech teams
- The inability for appropriate follow-up and reporting on the raised requests.
This was what led us to our first steps in re-orienting the existing processes and habits of the teams toward a more procedural and defined workflow. We started by implementing a dedicated Support board on Linear³ (our ticketing system) and implementing workflows to better visualize and follow up on the bugs and questions raised by our teams. Here’s a snippet of our ticketing template as well as a flowchart of our workflow.
We were confident that this would be a piece of cake: create a new board, add a bunch of columns representing states and tell everyone to use it — sounds simple enough, right? Well, not so much. Setting up the ticketing board was easy but encouraging its adoption by teams that were used to the previous approach was a whole other ball game!
Negative impacts of a positive change
We found ourselves in a scenario where our tickets were now further divided with certain questions and reports going through Slack while others through the Support board. In addition to this, our lack of visibility on core metrics led to us not realizing that there were a whole new set of issues to resolve that was introduced by this divide.
We started seeing tickets that were left open for over 5 months with no updates since they were essentially forgotten on the board. Furthermore, we found ourselves constantly busy handling new reports and queries from both sources without necessarily knowing why this huge storm of requests was suddenly being created and sent our way.
We quickly realized that we needed a measurable way of really evaluating (and demonstrating) the impact that such processes would have on other teams and our customers as well as monitoring our performance to better understand our flaws. We could not enforce a process or decision if we had no way of proving the benefits that this would bring.
From this point forward, we decided to rethink Support Engineering from the ground up and take a swift turn to make sure we had a clear view of all reports and questions at a single glance. With this in mind, we immediately got to planning our transformation towards being data-informed!
Phase I: The Centralization
With our bug reports and questions essentially scattered across multiple channels and platforms, there was no way of having a clear idea of the data we had at our hands and how we could maximize their utility in allowing us to make decisions. It was necessary for us to first ensure that we could bring everyone to a single location. This location could not just be any type of platform, it needed to be one capable of providing us with key insights on the tickets that we handled each week with a clear view of all open reports or queries that required our attention.
Slack was definitely out of the question since a messaging system would not allow us to really understand our performance, follow up on all the reports and queries, or even truly extract this data in a standardized format for further analysis if required. This meant that we needed to move all the reports and queries to our ticketing system to easily follow up on the open tickets as well as extract this data to perform advanced analysis as we would demonstrate in future phases.
The question however remained, “How do we achieve this without greatly impacting our team’s habits?”
Understanding the context behind the current system
We realized that the reason Slack remained a primary source for these queries and reports was that it was the main platform for internal communications. This meant that everyone always had it open, leading to “forced” faster response times, as the corresponding tribe would be pinged and would have to respond to it immediately. Since Linear was relatively new and not really used at the time by teams other than the Tech department, our teams were not used to checking Linear regularly and could sometimes not see our responses or requests for further information in time.
Another issue was being able to search for reports that were previously raised and/or discussed especially during the time of the transition. Given that while the transition would be in progress and for a few weeks after, information would still be scattered across Linear and Slack, we needed to make sure that it would be easy to find information on past issues raised across Slack and Linear through the same platform. This meant that to facilitate this transition we needed to first provide a smooth workflow between Slack and Linear such that users would be notified on Slack when new tickets are created as well as easily view the tickets that they created or on which their feedback is required.
Bridging the gaps between ticketing and messaging
Since Linear’s direct integration with Slack had certain limitations, both on the creation workflow — such as being unable to directly upload images or create the ticket based on pre-defined templates that had been created on Linear, as well as on the notification workflow — as being unable to customize the message that was shared in the channel for better readability by the rest of the team, we had to look for another (more flexible) way to do this. After quite a bit of brainstorming, the perfect solution was found — Zapier!
For the creation process, we bookmarked the links towards pre-defined templates for the creation of new bug and question tickets as designed on Linear to the slack channels of each tribe. On the other hand, we used Zapier to format the content of the newly created tickets into one that would be easy to read and sent into the slack channel of the team for which the ticket was created. You can see a sample of our Zapier workflow, as well as the results on slack in the images below:
This workflow seemed to satisfy both use cases described above: notifying the adequate team of a question that needed to be answered as well as allowing for search across both issues previously created on Slack and Linear. However, the technical implementations were not the only requirement to achieving this transition, it was time to “poke the bear” and start convincing our teams to move towards Linear for all further queries and reports.
Conclusion
The task was not easy, but with constant determination, and loads of effort put in by all the teams, we finally made it! It took a few months for us to transition out of slack but our questions and reports were now centralized on Linear and it became a universally adopted process across the teams.
The most interesting aspect of this was that once the other teams saw what we were able to accomplish, they started asking us to help implement the same on their end as well. We are so proud to currently say that over 70% of our teams have successfully moved their internal queries and reports to Linear and out of Slack allowing them to do much more with their reports than just answer them.
Here are some of the messages we received from our customer care team after the process was implemented and the benefits were seen:
Over the past year, our Support Engineer team has been able to put a great structure in place. Our question and bug workflow is a great tool that brings efficiency to our daily routine. Obviously, the efficiency at fixing bugs and answering technical questions or requests from our customers help us to provide an excellent user experience. On behalf of the care team thank you 🚀 — Samuel Le Goff — Customer Care Manager
Having a unified approach to Internal Bug Handling has further strengthened the collaboration between the CX and Tech team while also bringing more clarity and efficiency to the internal bug escalation workflow. Quicker bug escalations, quicker resolutions and happier customers! — Christophe Bauda — Team Lead: Customer Care
That’s all for this episode! We provided a brief overview of our vision and reasoning behind choosing to embark on this journey as well as shared the first phase where we attempted to centralize all our reports and questions to a single platform — our ticketing system, Linear.
Stay tuned for the next episode where we would present phases 2 and 3 covering our initial attempts towards the creation of reports as well as our thought process towards defining and choosing the metrics that would be most beneficial to track for our teams.
