Search

How can we help?

Contact our Solutions team

Back

Contact our Apps team

Back
Back

Click here to chat with our
support team now

Back
Managing Fires with Kanban - An Engineer’s Perspective
Kai Ming Chow
Senior Engineer III
Share this article

If you're part of an engineering team, you’re probably familiar with this scenario. Your morning is full of requests around improving production instances, searching for performance issues, and blocking malicious traffic that may just have hit your server. And this is just as you’re finishing your first cup of coffee.

More issues are raised throughout the day and all need your attention and a rapid solution. These fires you have to put out can range from dealing with critical customer concerns, ensuring timelines are being met and overcoming project roadblocks. However, balancing multiple priorities can be complex when all tasks have to get done, especially if delays occur or if there is a breakdown in processes.

Fanning the Flames

This is what happened to my team three years ago. We were constantly firefighting, and constantly working towards a tight schedule to deliver new features. When everything is always a high priority and urgent, fixing ad-hoc requests becomes the daily norm. The consequence of this is that it actually impedes our typical delivery process and reduces the trust our support team,  project managers and customers have in us.

Our director of engineering at that time, YC, was aware of the situation and approached us to understand more. He shared a youtube video with us and asked us to watch it to get a brief idea out of it. In the next few days, he guided us to implement some of the Kanban principles to help me and my team navigate through the never-ending high-priority items.

There are a few weaknesses with how the team manages priorities and expectations:

  • Unable to push back on work items because basically all of them are urgent.
  • Unable to make good decisions in picking the right item to work on.
  • We have more delivery commitments than we are capable of handling.

Our Previous Process

At the end of the week, we look into the control chart in our Jira to understand what are the reasons for a longer lead time. After many control chart reviews, we frequently came to the same conclusion - the longer lead time is due to firefighting. There's a saying that, "Insanity is doing the same thing over and over and expecting different results.” Clearly, the existing delivery process is no longer a fit for the team.

Identified Challenges and Gaps

Based on the identified weaknesses, our goal is to be able to transparently depict the priority of WIP work items and the delivery commitment that our team currently has. The  increased visibility toward WIP items shall promote more discussion and negotiation around the work itself and delivery commitment. 

Step 1: Identifying our work as a service 

Figure 1: Service requester and service provider for the team

In order to better understand the nature of work that our team is delivering, we are guided to see ourselves as an engineering team that provides services to the Support Team and PM

As part of the engineering team, we provided the following following services:

  • Work out product features according to the PM roadmap and commitment to customers.
  • Fix bugs in our products 
  • Give appropriate updates on the progress so that the Support Team can fulfill their SLAs 
  • Long term work that helps with reducing the cost of maintaining the application.

Step 2: Defining "Class of Services" (CoS)

Once there’s an understanding of the services the team provides, the classes of services are identified and defined. All work the team has done needs to meet a certain set of expectations to our Support Team, PM and customers. 

In our case, the main issue is deciding which work should be prioritized. Each of our service requesters has their own expectations. Therefore, we define classes of services according to the “urgency” of work. Then our JIRA Kanban is configured to show the classes of services as swim lanes.

  • Expedite
  • Items require urgent attention.
  • Will affect our business or customer if delayed/ignored.
  • Fixed date item
  • Items that have a non-negotiable dateline to it.
  • Standard work
  • Items that are not categorized into all other classes.
  • Intangible
  • Items that are non-urgent but important.
  • Usually, improvement tasks that can be delayed, but needed.

Now everyone has the transparency of what is being worked on by the team, and what is the delivery commitment made by the team. It can be told from Kanban whether the team is overcommitting.

Step 3: Scheduling Work Using "Cost of Delay" 

The delivery commitment is much more visible now, but how do we make sure the right work gets prioritized?

Here’s another scenario. Let’s just say you have a number of high priority work items in your backlog, what are you going to work on first?

  1. SSL certificate that is going to expire in 3 days.
  2. Customer A escalates a bug where they can’t generate reports from your application.
  3. You have 6 user stories in your epic that you’ll need to deliver in 3 weeks time, and you didn’t manage to work on any of the stories last week due to other escalations. Your PM is asking you for progress.
  4. You have 2 improvement work items that will permanently fix a few bugs that cause outages to your applications over the past few weeks.

Here is where the concept called "cost of delay" comes into play. You can assess the service requests by asking “If I delay working on this, what impact does it have on ours and customers’ businesses?”
Thinking from the perspective of delaying cost starts to shift our mindset and we begin to ask better questions that could help both customers and ourselves. We start to see the business risk and impact of our work, and this helps the team gain better clarity on the urgency and importance of the work.

Reduce The Number of Fires

Most of the work items be it features, bugs or improvement are subject to discussion and negotiation in the terms of delaying cost. There are more quality discussions about our customers' real needs and we continue to work on understanding them, and finding the solutions together with the Support Team and PM to avoid issue escalation. This helps the team be able to focus better in work and reduces the number of fires.

I need to emphasize that all the changes and Kanban principles that we applied are all simply tools we use for guidance. They don’t magically solve all our problems. These guiding principles and tools promote quality discussion and help with negotiation. It is the people on the team having high value discussions, practicing quality discussion and negotiation that slowly get the team back on track. I appreciate that my team, especially my manager, Sanjev, is practicing these principles very well.

Do the best work of your life. Join the ServiceRocket engineering team.

Join Us
<-- Facebook Share Button Code --> <-- Facebook Share Button Code --> <-- Twitter Share Button Code --> <-- Twitter Share Button Code --> <-- Linkedin Share Button Code --> <-- Linkedin Share Button Code -->