Mastering Predictable Feature Delivery on Live Service Games- Part 1
Build the foundation with process design and policies.
Deep Dive generated with Notebook LM.
Build a structured workflow focused on flow and predictability.
Start by defining start and finish points, mapping your workflow, setting explicit WIP limits, establishing clear exit criteria, adopting pull policies, and actively managing blockers.
Suitable for Scrum, Kanban, or any hybrid approach you use.
Regularly delivering new features and content is paramount to maintaining player engagement and satisfaction. However, the complexity of managing continuous updates can often lead to unpredictability, delays, and inefficiencies. Implementing a structured workflow encompassing flow and predictability is essential to navigate these challenges. Based on the work by Daniel S. Vacanti, author of Actionable Agile Metrics for Predictability: An Introduction, Actionable Agile Metrics for Predictability Volume 2 and co-author of Flow Metrics for Scrum Teams, this guide will walk you through designing and managing an effective workflow to enhance your team’s ability to deliver value consistently.
You can’t snap your fingers or stomp your feed to make the team deliver in predictable ways. Sure, some tools allow you to forecast future deliveries with probability, but all those results are thrown out the window if you don’t do the groundwork first. In this guide, we’ll lay out the foundation that you can use to become more predictable with your LiveOps features and content, one step at a time.
Although many of these ideas come from Kanban, almost all of these steps towards predictability will work with Scrum or your homegrown hybrid methodology.
We will break this down into four parts:
Process Design and Policies
Using Service Level Expectations
Continuous Improvement
Process Design and Policies
A well-defined process is the backbone of any successful workflow. It ensures that every team member understands their roles, responsibilities, and steps to deliver high-quality content to players. Here’s how to design a robust process that promotes predictability.
Define Clear Start and Finish Points
Establish Unambiguous Boundaries
Defining clear start and finish points for work items is critical to building predictability into your team’s process. This idea is essential for Kanban and Scrum, even though the Scrum Guide doesn’t explicitly mention identifying these points beyond Sprint boundaries.
Start Point: Not as Simple as it Seems
Determining a precise starting point isn’t always straightforward. Moving an item from a “To Do” to a “Doing” column might not mark the start of meaningful work, especially with tasks, instead of completing Product Backlog Items (PBIs) in Scrum.
Several factors can influence the choice of a starting point:
Value Delivery Level: In Scrum, value is typically defined at the PBI level rather than at the individual task level. To understand when work starts, the start point should reflect when a PBI is genuinely considered “started” regarding value creation. Most LiveOps features may take months before delivery reaches the player. Therefore, when determining the Value Delivery Level, consider intermediate deliveries to player proxies, such as game designers or product managers responsible for the feature and adjudicate the expected value to the player.
Scrum Events: Various Scrum events could serve as potential start points, depending on the team’s process:
Beginning of Development: When coding starts within a Sprint.
Sprint Backlog Selection: When a PBI is chosen for the current Sprint.
Refinement Commencement: When a PBI enters the refinement process.
Pre-Refinement: A PBI might be considered “started” even before formal refinement if initial discussions or conceptualisation have occurred.
Workflow Stages: In Kanban, the start point is often associated with a WIP-limited column at the beginning of the workflow, typically labelled as “Input”, “Ready”, or “To Do”.
Studio Stakeholder Perspectives: Ultimately, the start point should align with when the studio stakeholders perceive work to have begun on their request. This might include stages beyond the development team’s direct control, such as initial request intake or feasibility analysis.
Finish Point: Beyond “Done”
Similarly, determining the finish point requires careful thought. Equating “finished” solely with meeting the Definition of Done (DoD) might not be enough if the studio stakeholders’ definition of “done” includes aspects like deployment, value validation, and feedback incorporation.
Factors influencing the choice of a finish point include:
Definition of Done: While the DoD serves as a baseline, the finish point might extend beyond it to include:
Deployment to Production: When the work item is released to the live environment.
Majority Release: When rolling out new features using A/B testing, a majority release typically refers to gradually expanding the feature to a more significant percentage of the player base after initial testing. If the new feature shows positive results, it is then rolled out to a majority of players, often in stages (e.g., 25%, 50%, 75% of the player base).
Full release: After confirming the feature’s success with the majority, it is then released to the entire player base
Value Validation: When the studio stakeholders confirm that the delivered work item meets their expectations and provides value.
Feedback Integration: When feedback on the delivered work item is incorporated into future iterations or product development decisions.
Workflow Stages: In Kanban, the finish point often corresponds to columns like “Done” or “Deployed”. However, it’s crucial to have clear policies defining what constitutes “Done” for a work item to move to this final stage.
Studio Stakeholder Perspectives: The finish point should align with the studio stakeholders’ perception of when the work item is complete and delivers the intended value. This might involve stages beyond the development team’s immediate scope, such as user acceptance testing or production monitoring.
Multiple Start and Finish Points
Having multiple start and finish points within a workflow is acceptable and beneficial. This flexibility allows teams to track different workflow segments and answer questions about process efficiency. For example, you might track:
End-to-End Flow: From the initial studio stakeholder request to the final production release.
Concept-to-Cash: from one-pager to first revenue booking.
Development Cycle: From the start of development to the completion of testing.
Specific Stage Duration: The time spent within individual workflow stages like ideation, prototype, design, or testing.
Wait, does this sound a bit like a waterfall? Having multiple workflow stages isn’t necessarily the same as using the Waterfall methodology. Although both approaches involve distinct phases, there are some crucial differences:
Flexibility: The Waterfall methodology is rigid and sequential, with each phase having to be completed before proceeding to the next. In contrast, multiple workflow stages allow for more flexibility, enabling iteration and overlap between stages.
Adaptability: The Waterfall methodology often struggles to accommodate changes late in the development cycle. However, multiple workflow stages can be designed to allow for adjustments and feedback throughout the process.
Feedback Loops: Typically, the Waterfall approach offers limited opportunities for feedback until the project ends. Multiple workflow stages, however, can incorporate regular input and adjustments.
Iterative Nature: Agile methodologies frequently employ multiple workflow stages and emphasise iterative development and continuous improvement. Meanwhile, Waterfall follows a linear progression.
Planning Approach: Waterfall demands thorough upfront planning and documentation. In contrast, planning with multiple workflow stages can be incremental, allowing for greater adaptability.
Having distinct start and finish points eliminates ambiguity and ensures that every team member knows exactly when a task begins and ends.
Map Out Your Workflow
Visualise the Flow of Work
Next, map out all the stages a work item goes through from inception to deployment. This visualisation helps identify potential bottlenecks and inefficiencies.
Stages Example: Ideation → Design → Development → Testing → App Store Submission → Release to Pridction → Post-Deployment Monitoring
These points represent the boundaries of your workflow and dictate which work items are considered “in progress.”
Creating a detailed workflow map lets you see the entire process at a glance, making it easier to manage and optimise each stage. If you’re a Jira user, you’re likely already familiar with this methodology, as it’s an integral part of managing projects effectively within that system.
Visualising and Accounting for Queuing Stages in Your Workflow
When using a Kanban system, it’s crucial to visualise and account for queuing stages. Explicitly representing queues as distinct stages on your Kanban board can offer valuable insights into process bottlenecks and areas for improvement. Here’s why:
Queuing Stages Highlight Delays: Queues often form when work items stagnate at a particular stage without advancing. By visualising these queues, you can swiftly identify where work accumulates and explore the reasons for these delays.
“Done” Columns as Queues: It’s helpful to separate workflow steps into “Active” and “Done” sub-columns and to represent both on your Cumulative Flow Diagram (CFD). “Done” sub-columns essentially serve as queuing stages where work idles, awaiting progression to the next phase. If these columns show bulging bands or excessive thickness, it indicates flow issues.
Buffer Columns as Queues: Buffer columns, often labelled “Waiting for...” or similar, also act as queues. They hold completed work from an upstream stage until the downstream stage is ready to pull it in. While helpful, overuse of buffer columns can lead to delays and contribute to inactive time, affecting your Cycle Time.
Impact of Queuing on Flow Metrics:
Increased Cycle Time: Items lingering in queues accrue Cycle Time without active work, extending overall completion times.
Reduced Throughput: Queues can restrict workflow, diminishing the number of items completed per unit of time.
Using Queuing Stages for Improvement:
Identify Bottlenecks: Observing which queuing stages consistently have high item counts helps pinpoint workflow bottlenecks.
Analyse Delay Causes: Delving into why items accumulate in specific queues allows you to tackle the root causes of delays, whether they stem from resource constraints, external dependencies, or policy issues.
Optimise WIP Limits: By monitoring queue lengths, you can adjust Work in Progress (WIP) limits for different stages to enhance flow and minimise delays.
Implement Explicit WIP Limits
Control Work in Progress (WIP)
WIP limits are a core tenant of the Kanban approach and are crucial for preventing your team from becoming overwhelmed and maintaining a consistent workflow. These limits cap the number of work items allowed at each workflow stage. For example, you might restrict the “Development” stage to five concurrent tasks.
By sticking to these limits, the team is encouraged to focus on completing current tasks before starting new ones. This approach promotes efficiency and reduces the risk of burnout.
WIP Limits with Scrum
While the Scrum Guide doesn’t explicitly discuss WIP limits, incorporating flow metrics and Kanban practices into Scrum can significantly enhance predictability and efficiency. Defining and managing WIP limits plays a crucial role in this approach.
Defining Clear Workflow Stages: Segment your workflow into distinct stages, including queuing stages such as “Done” columns or buffer columns where you will apply Work in Progress (WIP) limits.
Initial WIP Limit Setting: Setting the perfect WIP limit often requires trial and error.
Start with Educated Guesses: Base your initial limits on variables such as team size, capacity, available historical data, and the specifics of the work.
Consider Different Approaches:
Per-Column Limits: Allocate separate limits to each stage of the workflow.
Workflow-Wide Limit: Implement a single overarching limit for the entire process.
Team-Based Limits: Set limits on the number of items a team can handle simultaneously.
Age-Based Limits: Prevent starting new tasks if a certain number of items exceed a predefined age threshold.
Observe, Analyse, and Adjust:
Monitor Flow Metrics: Monitor metrics like WIP, Cycle Time, and Throughput to gauge the effects of your WIP limits. Watch out for bottlenecks (queues building up), starvation (idle teams), or undue delays.
Regularly Review and Adjust Limits: Employ Sprint Retrospectives and other feedback mechanisms to evaluate the efficacy of your WIP limits and make necessary changes.
Promote Team Ownership: Encourage your development team to actively control their WIP limits and adjust based on their observations and experiences.
Define Clear Exit Criteria
Ensure Shared Understanding of "Done"
Clear exit criteria are essential for each workflow stage to ensure a smooth flow of work and promote a shared understanding within the team. These criteria serve as a checklist, outlining all the conditions that must be met before a work item can move to the next stage. This clarity eliminates ambiguity around what “done” means for each stage, leading to better predictability and consistent quality.
Integrating Exit Criteria into Your Workflow
Workflow Visualisation: Exit criteria help define a well-structured workflow by setting explicit rules for transitioning work items between stages. This is crucial for visualising the workflow on a Kanban board or similar tool, including details like queuing stages and “Done” columns.
“Definition of Done” (DoD): Exit criteria can be viewed as a detailed extension of the DoD concept, specifying requirements for each workflow stage. This is particularly useful in complex workflows or when organisational constraints limit a broader definition of “done.”
Reducing Work Item Age: Exit criteria prevent delays and reduce the overall age of work items by ensuring they meet all necessary criteria before advancing, aiding in bottleneck identification and improvement.
Facilitating Pull Systems: Clear exit criteria strengthen pull systems by signalling downstream stages to pull work in once the requirements are met, enhancing predictability and just-in-time management.
Examples of Exit Criteria:
Development Stage: Code complete and peer-reviewed, unit tests passing, documentation updated.
Testing Stage: All test cases are executed, defects are resolved and verified, and a test report is generated.
Deployment Stage: Deployed to the target environment, smoke tests successful, monitoring configured.
Analysis Stage (for Scaled Kanban): Work item right-sized, acceptance criteria defined, dependencies identified.
When defining exit criteria, involving the entire team in the process is crucial. By collaboratively setting these criteria, you ensure everyone understands what’s expected at each stage, which fosters buy-in and a shared sense of ownership. This collective approach helps the team align on the quality standards and conditions needed to move work items forward.
It’s best to start with simple, transparent criteria. Initially, focus on the basics—keep things straightforward. As the team grows more comfortable and gains experience, these criteria can be refined and expanded. This iterative approach allows the requirements to evolve naturally, improving them over time-based on real-world experiences and the team’s specific challenges.
Visualising the exit criteria is another essential step. Displaying them prominently on your Kanban board or within any workflow representation helps ensure everyone remains accountable and transparent. When the requirements are visible to the whole team, it’s easier to understand where work stands and what’s required to move forward, reducing the risk of miscommunication or assumptions.
Lastly, remember to review and adapt these criteria regularly. Sprint Retrospectives or similar feedback mechanisms are great opportunities to reflect on the effectiveness of your current criteria and discuss any necessary changes. Keeping the requirements dynamic means they can evolve alongside your team’s processes and needs, leading to continuous improvement.
Establish Robust Pull Policies
Adopt Pull-Based Systems
Implementing pull-based systems means work is only moved into the next stage when available capacity exists. This approach helps ensure that team members aren’t overwhelmed and that the job progresses steadily without piling up in queues. The goal is to maintain a smooth and efficient flow of tasks, preventing congestion that can slow delivery and lead to burnout.
A pull-based system relies on the team actively managing their workload and ensuring they only take on work when they can do so effectively. This helps maintain focus and quality, as team members aren’t forced to juggle too many tasks simultaneously, which can compromise efficiency and output quality.
Considering the Class of Service is essential to implementing pull-based systems successfully. This refers to categorising work items based on priority and defining different pull policies for each type. For example:
High-Priority Features: These items should be pulled through the workflow as soon as capacity is available, ensuring they’re delivered promptly.
Minor Updates: These can be pulled when convenient, as they may not have the same urgency as high-priority features.
Urgent Bug Fixes: If necessary, these should bypass normal pull limits to minimise the impact on the product.
Teams can improve their predictability and responsiveness by clearly defining how work is pulled into each stage. It also allows teams to manage workload effectively, ensuring high-impact tasks receive attention without derailing the workflow.
Moreover, using pull-based systems encourages a culture of ownership. Team members decide when to pull new work, which fosters responsibility and commitment. This also enhances transparency within the team, as everyone can see where work is, what’s being held up, and where they can potentially assist.
In essence, pull-based systems balance the workload to maintain consistency. They support a steady delivery cadence, help identify bottlenecks early, and provide flexibility to adjust to changing priorities—all of which contribute to a more predictable and efficient workflow.
Pull-based system and Scrum
Scrum and pull systems aren’t at odds with each other. Scrum incorporates many core elements of a pull system. Here’s how the two align:
Sprint Backlog as a WIP Limit: The Sprint Backlog effectively acts as a Work in Progress (WIP) limit, a key feature of pull systems. This limit helps control the workload, ensuring the team isn’t overwhelmed and can maintain a steady flow.
Self-Selection of Tasks: In Scrum, team members pull tasks from the Sprint Backlog when they have capacity instead of having them assigned. This lets each person pick work that matches their availability and skills, making the process more efficient.
Continuous Flow: During a sprint, team members pull new tasks as they finish others, promoting a constant workflow. This helps keep productivity consistent throughout the sprint.
Feedback Loops: Regular feedback sessions, like Daily Scrums and Sprint Reviews, are integral to Scrum and align with the feedback-driven nature of pull systems. These sessions provide a chance to adjust the course, which is critical to continuous improvement.
Prioritisation: The Product Backlog is constantly prioritised, ensuring that the most essential items are pulled into each sprint. This focus on value-first work is very much in line with pull system principles.
While Scrum has some push elements—like the initial selection of work for the Sprint Backlog—its core strongly aligns with pull system ideas. By combining time-boxed sprints, self-organisation, and focusing on continuous feedback, Scrum becomes a hybrid that effectively uses pull system principles.
Actively Manage Blockers
Identify and Resolve Impediments Quickly
Blockers are obstacles that prevent work items from moving forward. They can be anything that halts progress, from waiting on external approvals to missing information. Managing these blockers is crucial for maintaining a steady workflow and preventing delays.
Develop a System for Managing Blockers: You need a transparent system to deal with blockers effectively. Tools and practices such as daily stand-ups, Kanban boards, or visible blocker columns help spot and address obstacles quickly. During daily stand-ups, blockers should be one of the main discussion points. The sooner you identify a blocker, the sooner you can take action to resolve it.
Assign Clear Responsibility for Resolution: Once a blocker is identified, ensure someone is responsible for resolving it. Assign a specific team member to take ownership of each blocker so it doesn’t become a forgotten issue. It’s also essential to ensure the responsible person has the authority and resources to remove the blocker. Resolving a blocker might sometimes mean escalating a problem, seeking approvals, or negotiating with external teams—so empowerment is critical.
Provide Resources and Empowerment: Your team needs the authority and tools to remove blockers quickly. Whether it’s a clear line to stakeholders for approvals or the ability to access external resources, empowering your team to tackle blockers can significantly speed up their resolution.
Proactive Monitoring: Ignoring blockers doesn’t just stall work; it can also impact your entire system by increasing the age of work items and reducing predictability. When items are blocked, the longer they stay that way, the more likely they are to cause bottlenecks. A proactive approach—where blockers are tracked, actively managed, and cleared—is essential for keeping work flowing smoothly.
Transparency in Blocker Management: Make blocker management transparent to everyone involved. Visualise blockers on your Kanban board or tracking tool, marking them distinctly so they’re easy to spot. This ensures the whole team, including stakeholders, knows where bottlenecks occur and can offer support.
You can keep work moving efficiently by actively managing blockers, assigning clear responsibilities, and giving your team the necessary resources. It’s about being proactive and transparent and ensuring that there’s a structured and swift way to eliminate obstacles when obstacles appear.
Conclusion of Part One
In conclusion, consistently delivering new features and content is critical to keeping players engaged. To manage continuous updates effectively, it’s vital to establish a well-structured workflow that emphasises flow and predictability. By focusing on clear process boundaries, defining WIP limits, using pull-based systems, and managing blockers proactively, your team can significantly enhance efficiency and reduce unpredictability.
The next instalment in this series, “Data Collection and Analysis,” will dive into how to gather and interpret data to make informed decisions that boost predictability and team performance. Stay tuned!