Solely Relying on Your Team's Estimates is Costing You Time and Money

Explore how shifting from traditional estimating techniques to probabilistic forecasting can revolutionise your approach, offering precision, efficiency and direct cost benefits.

Prompt engineered with DALL-E.

Estimation has played a crucial role in software development, especially in game development. Teams have long depended on these estimates to chart out the required time, financial resources, and efforts for a project, aligning their work with the anticipated outcomes expected by producers and stakeholders. This practice, deeply ingrained in the industry’s fabric, provides a roadmap for project completion.

However, the effectiveness and necessity of this traditional practice are now being called into question. As the industry evolves, the relevance of these conventional estimation methods is being scrutinised, challenging long-held beliefs about their utility in a rapidly changing technological landscape.

Challenges with typical estimation practices

In the world of mobile game development, particularly for Game-as-a-Service, there’s an ironic twist to the practice of estimation. Despite the industry’s heavy lean on data-driven decision-making, there’s still a surprising reliance on the more traditional, subjective estimation method when planning delivery. This age-old practice often trumps data analytics despite the latter’s proven advantages in precision and predictability.

Art by Scott Adams.

This reliance on estimation, more out of habit than efficacy, often overshadows the reality that its purported benefits are seldom realised. It’s an intriguing paradox in an industry where every other aspect is meticulously measured and optimised based on user data and analytics.

The estimation process is inherently flawed because it depends on predicting future events, which means accounting for numerous complex and often unpredictable variables. This leads to significant inaccuracies in estimates, which can, in turn, undermine trust within development teams.

Developers may overestimate timelines to prepare for unexpected challenges. However, if this extra padding is added haphazardly across many work units, the project timelines can be excessively extended. While meant as a precaution, this approach can hide the project’s actual progress, leading to proposed delivery dates that are impractical for the business. Such situations often necessitate cutting the project scope, revising specifications, and recalculating estimates. These actions can breed tension and distrust among team members, managers, and clients.

Additionally, considering that developers are often the most costly members of the team, the demands for highly detailed specifications by experienced programmers necessitate Big Upfront Planning (BUP). This approach, which contrasts with agile software development, consumes a vast amount of time for designers and product owners. Consequently, the substantial time and effort invested in creating these estimates divert valuable resources from more productive tasks. This shift in focus and resources raises questions about the overall efficiency and effectiveness of the estimation process.

Cognitive biases play a significant role in these issues. For instance, the optimism bias can lead developers to underestimate the time needed, thinking tasks will go more smoothly than they often do. The anchoring effect might cause teams to rely too heavily on initial estimates, even when new information suggests adjustments are needed. Confirmation bias can lead to selectively seeking information that supports pre-existing beliefs or estimates, ignoring evidence to the contrary. These biases can warp estimations, further complicating an already challenging process.

The unpredictability of game software development

Game software development is inherently unpredictable, with complex and often unforeseen problem-solving challenges. This unpredictability is a critical factor that makes accurate estimation exceptionally challenging.

Art by CommitStrip.com.

Factors like the evolving project requirements linked to ‘finding the fun’, reacting to rival games, and unexpected hurdles add to this unpredictability. Moreover, dealing with tech debt, looming platform holder update deadlines and holiday seasons further complicate project timelines. The growing dependence on external elements, such as app stores, SDKs, and APIs, introduces complexity beyond the development team’s direct control.

These external dependencies can significantly impact project timelines and outcomes, further complicating the estimation process. The fluid nature of software development, with its myriad of potential changes and adjustments, renders traditional estimation methods less effective and often obsolete.

Would you gamble your business on a wild-ass guess?

Building a business is an intricate process that requires a blend of reliable tools and informed decision-making. Corporations typically employ diverse resources, including data analytics, financial reports, and market research, to navigate complex business landscapes. This array of tools, supplemented by performance metrics, KPIs, legal insights, corporate governance structures, and external advisory services, constitutes a robust framework designed to steer companies toward well-informed and strategically sound decisions.

Art by J King.

However, a shift occurs when businesses rely heavily on subjective estimation processes for crucial decisions, transforming this methodical approach into something more akin to gambling. Experience and intuition, which can be as unpredictable as luck, often overshadow data-driven insights in these scenarios. This trend is particularly evident in software development, where most projects significantly overrun their estimated timelines.

Large-scale projects are especially prone to delays, frequently exceeding their scheduled completion dates by an average of 25 to 50 per cent. Such a reliance on subjective estimation, despite the availability of more objective and analytical tools, raises questions about the efficacy of this approach in business strategy.

Given this context, where companies have access to many analytical tools yet often default to less reliable subjective estimation processes, one must ponder: Is trusting your business’s future to estimations alone a gamble worth taking?

Advocating for Estimation

Despite these significant challenges, a strong argument remains in favour of maintaining estimation practices in some form. Estimates are often viewed as project management tools and essential elements of professional practice. They align project objectives with the broader goals of a studio and are crucial in communicating project trajectories to stakeholders. In many publishing environments, providing an estimate is fundamental for securing project funding and resources. This perspective suggests that rather than discarding estimation practices altogether, there is a need to refine and improve them. The goal should be to enhance the accuracy and relevance of estimates in the context of modern software development.

Shift from Traditional Estimation to Probabilistic Forecasting

The ongoing challenges with traditional software estimation techniques call for a transformative approach that integrates more reliable and dynamic methods. Probabilistic forecasting emerges as a powerful solution, steering away from the limitations of conventional estimation practices. This innovative approach redefines estimations in software development, evolving them from rigid and often inaccurate guesses into adaptable, data-driven probability forecasts.

A spectrum of possible outcomes.

Central to probabilistic forecasting is its use of actual team performance data, which aligns perfectly with the complex and ever-changing nature of software development. By considering a spectrum of possible outcomes, each quantified with a probability derived from historical performance data, this method provides a more realistic and flexible estimate. It acknowledges the myriad of variables and uncertainties inherent in software projects, making the traditional pinpoint estimates unrealistic and unreliable. Therefore, the adoption of probabilistic forecasting marks a shift towards estimates that are not only adaptable but also rooted in empirical evidence, offering a more nuanced view of potential project trajectories.

This paradigm shift is not about discarding traditional estimation but enhancing its applicability and accuracy in the modern context of software development. Probabilistic forecasting, grounded in real-world performance data, emerges as a sophisticated producer tool. It equips game teams with a clearer insight into the likelihood of various outcomes, fostering more informed decision-making and effective risk management. As game development evolves, embracing probabilistic forecasting is a step towards more precise, flexible, and practical estimation techniques, fundamentally changing how projects are approached and executed.

Monte Carlo Simulations:

The tool we use for probabilistic forecasting is the Monte Carlo Simulation. It offers a robust method for forecasting in project management. These simulations provide probabilistic estimates by leveraging historical performance data of software development teams. The process involves running tens of thousands of trials with past data to predict future outcomes. By inputting a start date and user story count, the simulation produces a range of possible completion dates, each with a corresponding probability. Repeating this process multiple times results in a probability distribution, a valuable tool for planning and decision-making.

Monte Carlo Simulations in practice answer two questions:

• “When can we finish X number of tasks?”- Monte Carlo will give you the delivery date of your project and the level of certainty that this will happen. Let’s say that you know (at your best) that the project’s scope is about 100 tasks. You can use the Monte Carlo simulation to give your business owner a probable delivery date and the confidence level to hit that target.

• “How many tasks can we finish in X number of days?”- With Monte Carlo, you can decide how many items can be completed within a specific timeframe. For example, say you know your next release is planned for 15 June, and you want to know how many new features will be ready by then. You input your start date and end date, and the simulation will give you a range of outcomes and the probability of each.

For more information on Monte Carlo Simulations, see my other blog posts:

• From Meteorology to Project Management: The Power of Predictive Modelling- Part 1

• From Meteorology to Project Management: The Power of Predictive Modelling- Part 2

Cost-saving benefits of Monte Carlo Simulation over traditional estimation process

Prompt engineered with DALL-E.

Incorporating Monte Carlo Simulation into software project estimation significantly streamlines the process, offering a cost-effective alternative to traditional estimation methods, which are less accurate and more resource-intensive. Traditional cost estimation methods demand considerable time and effort from team members. These methods often require detailed specifications and extensive input, pulling valuable resources away from product development tasks. Creating these detailed cost estimates can be exceedingly time-consuming, necessitating work hours, if not days. This is especially true when highly exact specifications are needed, compounding the time and effort required.

In stark contrast, Monte Carlo Simulation offers a swift and efficient alternative. This method can run complex simulations in seconds, requiring only a limited set of inputs to provide a range of probabilistic cost outcomes. The speed and efficiency of Monte Carlo Simulation are grounded in its ability to leverage computational power to process many scenarios quickly, thereby bypassing the labour-intensive process intrinsic to traditional methods. The reduced time commitment for generating estimates with Monte Carlo Simulation means that team members can focus on core product tasks, enhancing overall productivity and project progress.

Additionally, the reliance on a smaller number of inputs for Monte Carlo Simulation does not compromise the depth or accuracy of the estimates produced. Instead, it provides a nuanced cost projection that accounts for software development’s inherent uncertainties and variabilities based on real-world team data. This aspect of Monte Carlo Simulation saves time and reduces the potential for human error often present in manual, detailed estimations.

Another significant cost-saving aspect of Monte Carlo simulations is their ability to update and refine cost estimates as a project progresses continuously. Once set, traditional methods often fail to adapt to project changes, leading to outdated and inaccurate cost predictions as the project evolves. Monte Carlo simulations, on the other hand, can be rerun with updated data throughout the project’s lifecycle, providing ongoing insights into the cost implications of project changes or delays.

In this context, Monte Carlo Simulation enhances cost-saving efforts by enabling studios to identify key risk factors that significantly impact project outcomes. With a deeper understanding of the most volatile and influential variables, managers can effectively focus their risk mitigation strategies. This targeted approach prevents unnecessary expenditures and ensures efficient resource allocation, emphasising Monte Carlo Simulation’s cost-saving potential in complex project management.

Key takeaways

The shift from traditional estimation methods to probabilistic forecasting and Monte Carlo Simulation marks a significant advancement in game development project management. Standard estimations, deeply rooted in the industry, are now being overshadowed by the need for more accurate, data-driven methods due to the unpredictable nature of game development. The reliance on outdated methods, plagued by inaccuracies and cognitive biases, gives way to a more dynamic, adaptable approach.

Monte Carlo Simulation, in particular, offers a striking advantage. It efficiently processes multiple scenarios to provide a range of probabilistic outcomes, saving time and allowing teams to concentrate on core development tasks. This method’s capacity for continuous updates ensures that estimates remain relevant throughout the project lifecycle, a stark contrast to the static nature of traditional estimates.

This approach enhances efficiency and ensures effective risk management and cost-saving, highlighting the method’s strategic value in software development. As the industry evolves, the transition to data-driven forecasting methods like Monte Carlo Simulation is becoming increasingly essential for informed decision-making and successful project execution. This evolution prompts a critical question: Is it time for the game development industry to fully embrace these advanced forecasting techniques for more accurate and successful project outcomes?

If you like what you see, please share this blog on social media!