Experimentation: In Support of “Be the Change You Seek”
At Don’t Panic Labs, we regularly lean into our four core values:
- Empathize Then Own It
- Build Smart
- Deliver with Pride
- Be the Change You Seek
Change has, and always will be, a constant. There is immense power when we can channel that force. To that end, I initiated a “meta experiment” around defining a process to track, check in on, and amplify the effects of improvements at Don’t Panic Labs.
The journey for me around experiments and reporting hails all the way back to electrical engineering labs in college. One thing I learned through those classes was that my professional passion was not in designing or building circuits. However, what has had more impact on my role as a software engineer is that I learned how to define an experiment, break down a problem, define steps, and report results.
This experiment approach I experienced in engineering labs naturally transitioned to an approach to operations and process improvement. Instead of testing resistor ohms, we are able to test process changes. Instead of learning how a capacitor affects a circuit, we can validate proposed improvements. Instead of reinventing the wheel, we can lean into the scientific method and a repeatable process that has spanned disciplines and domains.
In general, the process follows three simple steps:
- Understand the problem
- Propose solution(s)
- Do and learn
Understand the Problem
When seeking to understand the problem, we are doing our homework. We never want to change simply for the sake of change – there are far too many real and interesting problems to go on a hunt to create our own. As such, in this phase, we should start with unpacking the question, “why should we care about a proposal or change?”.
Questions to ask yourself in this phase might include:
- What problem are we trying to solve?
- What contextual or background information is relevant?
- What has been previously tried?
Where we have the luxury of building off of thousands of years of historical engineering knowledge and decades of successes and failures in software engineering itself – we want to use our resources well. It is important to note that there is no reason to experiment with basic elements of engineering. In this way, sometimes, an experiment can feel more like a change proposal to solve a specific issue at hand.
The experiment framework, as it relates to process, works well here as we attempt to figure out the right rhythms for a team or the most effective critical thinking activities for a specific group or division – all the while being mindful and intentional in our approach.
Propose a Solution
Once we have spent time truly unpacking the problem, we can start to ideate possible solutions. Think of this as the “steps” in the experiment. How are you going to implement your proposed change?
This creates the opportunity to analyze if what sounded like a good idea in your head bears the test of writing it down. Writing is one of the most powerful critical thinking tools as it forces us to define our terms, clearly articulate each aspect, and receive feedback.
Adam Grant articulates this in a post on X:
Writing, seeking feedback, and getting buy-in to test this hypothesis are all important aspects when we propose a change or a possible solution.
It is worth noting that a possible successful outcome in this process is that we learn something new. Someone else phrases things differently or sheds a different light on the problem so as a team or as a company, we can come to an even better solution.
Do and Learn
After we have done our homework, ideated, and agreed on a possible solution, it’s time to do and learn. The biggest temptation at this point is to forget to check in on progress. Often, process changes take time (especially significant and meaningful ones), so instilling a regular cadence to either qualitatively check-in and ask how it’s going or quantitatively analyze metrics that were hypothesized to be impacted is essential.
After we have allowed an experiment to progress enough, there is a moment when we expect to have a conversation about whether we will adopt or abandon this change. An abandonment can feel like a failure; however, we must remind ourselves of what we learned and that the best step we can take when down the wrong path is to turn around. Also, by making this a regular and normal thing to either adopt or abandon changes, you take much of the possible energy that might otherwise derail tackling the next problem.
Getting Started
You might be thinking this sounds simple or obvious. And in some ways this process is quite intuitive; however, in no way does it make it easy. The problems we are trying to solve by implementing experiments, either personally or within a team or company, are just that – problems. The most important and biggest hurdles have often not been addressed because they are particularly messy, challenging, or that they solutions each come with trade-offs. But is it not the case that these are also the problems that — when tackled — make the biggest difference? Even when we don’t solve a problem with our first attempt, seeing someone willing to step into the mess encourages others to try as well.
Or as John F. Kennedy said in his September 12, 1962 speech:
“We choose to go to the moon in this decade and do the other things, not because they are easy, but because they are hard.”
What is your hard thing? What have you considered an unsolvable problem? What is keeping you from being the positive change you seek to see in the world?
