Quantifying Disruption With The System Dynamics Method – A Brief Explanation
By Thomas Hofbauer and Nicole Mörchen, HKA
Disruptions frequently lead to the project schedule and subsequent completion date slipping, as well as cost overruns from inefficiencies. Quantification of disruption often remains problematic, leaving the analyst with a degree of uncertainty. In this article, Thomas Hofbauer and Nicole Mörchen at HKA briefly discuss how system dynamics methodology can be used to quantify disruption with regard to the time and cost of projects.
Traditional Analysis Methods Lead to Vague Results
When dealing with complexity on major projects, it is not only external factors which may give rise to issues but also internal factors. This is particularly true of projects where engineering is the driving factor, and where it is often difficult to separate the consequences of certain issues, internal or external.
When using system dynamics methodology, often the first challenge is to define a project baseline in terms of both the budget/hours and the schedule. For example, in our experience, we have rarely seen a detailed schedule for engineering works that reflects the complex interfaces with the engineering process. Even in the case where durations of different tasks are planned, a logic linking the different tasks is often missing. Adding to this, an engineering baseline is not completely fixed over the course of a project. A project develops over time and some ideas are developed further than as planned, while others are withdrawn. This needs to be reflected in updated baseline schedules, but in practice rarely is.
The second challenge in using system dynamics methodology is the definition or recognition of a change. As an example, in the engineering process, changes or even mistakes (but what is a mistake?) can be inherent within the process itself. Furthermore, in situations where the client is asking for something different to what was originally proposed by the contractor, does this constitute change whereby the client has to pay? Or is this a normal design development? The analyst can often find a way to separate the additional effort from the planned. It becomes yet more complex in a situation where the client asks for something different to the proposed works, but at the same time the contractor changes a number of items. Detailed recording of the changes and respective efforts are required in this scenario, in order to come to a conclusion as to who has financial responsibility. These detailed records are often not available.
Similarly, “traditional” methods, such as critical path analysis for time or the measured mile and other methods for quantum, may yield results that are imprecise or can be pushed in one direction or another. Often the concern raised when using benchmarks is that the applied benchmarks are not relevant to the specific project or problem. In addition, even when very detailed records are available, it is our experience that the allocated effort and delay linked to a specific problem is underestimated. The reason for that is often very simple: (additional) efforts are often not connected with specific issues and therefore not recorded against that specific issue, nor is the wider effect on other related activities captured in the records.
Can system dynamics help?
Why is the System Dynamics Approach Useful?
The statistician Georg Box quoted: “All models are wrong, but some are useful.” In this respect, system dynamics is no different to other methods, which are basically models. But why is system dynamics useful?
System dynamics is a method to simulate holistically, and analyse, complex dynamic systems such as projects or parts of the project. The model only works when all relevant external and internal impacts and interfaces are incorporated into the model. Therefore, any internal issues need to be addressed as well as those external factors, and cherry-picking issues from one side or the other to produced skewed results is difficult.
The paradigm of the method is that the characteristics or performance of a system (or projects) are determined by its structure. Knowing the system-structure allows for robust statements about the system-performance. Therefore, the methodology allows a retrospective analysis of different concurring and concomitant causes for temporal and financial deviation from the baseline.
As system dynamics is a top-down approach, the model can be built with relatively little data compared to traditional approaches. It is possible to begin to use the methodology with relatively little records before adjusting the model as further records become available.
The data is not input data, but a benchmark for the calibration of the model. In other words: we don’t enter hours per month into the system, but we calibrate the system until it displays the actual hours per month close to those actually expended.
System dynamics is no black-magic computer model. Even though specialist software to produce the model is used, the approach itself is transparent and easy to explain.
What Do We Get from the System Dynamics Method?
The result of a system dynamics exercise, in a forensic context, is similar to a “but-for” analysis. The Society of Construction Law explains:
The model replicates the complex network of relationships and interactions that influence labour productivity and rework including: (a) the various stages of the project (design, approvals, procurement or manufacturing, installation, construction, commissioning and taking over), (b) the different parts of the works, workflows and project participant, and (c) the direct effects of the claim events. The model reproduces the actual labour hour expenditures (including the as-built programme and added variations and other changes). The project is then re-simulated in the absence of the claim items resulting in a but-for model.
A key advantage of system dynamics is that the analyst can switch on and off different issues, therefore creating various “but for” models. For example, in a scenario where an employer claims insufficient work force is present and the contractor claims insufficient work fronts, the model can be calculated but for one or another issue, or even without any of the issues. The result may then be that, even with more staff, the progress hadn’t been higher as there was no work front available. Or, on the contrary, the number of staff was so insufficient that lack of work fronts were not hindering progress. These are questions often posed in disputes, ie: if one side had been perfect, would the outcome still have been the same?
In summary, the system dynamics methodology provides the analyst with a model of the actual project, with the possibility of displaying different “but for” scenarios. The analyst can therefore analyse the effect of various impacts in different scenarios.
When is System Dynamics the “Right” Method?
System dynamics is very flexible and its methodology applicable for almost every kind of project, however the analyst should consider the following:
System dynamics is complex. Is an analysis based on a simpler or more common method “good enough” for the specific case?
As a result of the system dynamics top-down methodology, an experienced analyst can relatively quickly analyse highly complex projects with many internal and external interfaces. Consequently, it is not necessary to go through all details to know that you have no case.
System dynamics relies on accurate as-built data such as explained below. Though if the history of the project is not available, any kind of “but for” analysis and/or system dynamics would not work.
A system dynamics model can be built with relatively little data. In general, the following data and information is required: disruptions such and hindrances or scope changes, information regarding the progress of the project, working hours, deliverables, resource curves, structured interviews with key personnel, project structure information, work procedures and methodologies, work flows, interdependencies between work streams.
Using system dynamics, it is possible to analyse concurrent disruption events and attribute liability for the outcome accordingly.
The engineering process is often not planned in the same detail as the “project itself”. In cases of engineering disruptions their specific effects are hardly discernable. System dynamics allows the analyst to review this situation in better detail.
And finally, system dynamics is a strong tool which can be used to forecast projects and project outcomes during the project if disruptions are being experienced.
Criticism About System Dynamics
The Society of Construction Law delay and disruption protocol (SCL protocol) raises some concerns regarding the use of system dynamics. It notes that “the robustness of the conclusions derived from this analysis is dependent upon” the following factors (we have included our comments alongside those of the SCL protocol):
“The accuracy and completeness of the source input data and hence the quality and availability of project records.” This is true for any kind of analysis.
“The reasonableness of the analyst’s judgements in establishing the model.” To build the model, it is extremely important to understand the structure and interdependencies of the project. Once the model is validated, the results are easy to ratify. Traditional methods also depend on the project understanding and the analyst’s judgement.
“The transparency of the analytical process carried out by the specialist software.” We do not agree with this. First, the calibration of the model is transparent. Second, causal tracing can be done for each parameter.
It is our opinion that the system dynamics approach is no more black-magic in nature than any other comprehensible analysis method.