In my previous article, I addressed and explained my belief that testing is risk management in a very pure sense. I set out how the basic stages of risk management map to testing activities. I also said I would continue to delve into risk management to see if it has lessons to teach testing. This is likely. From my experience of risk management, it is a little more mature in some of its practices and approaches.
In this article I am going to review the first principle of risk management. We will see how it aligns to principles, objectives, or practices in testing and if there are lessons to learn. This is a continuation of my investigation into the link between testing and risk management – am I right in saying that they are one and the same? It is a first chance to identify areas where lessons could be learned. Methods or techniques to address those lessons will be the topic of future articles.
In considering the risk management principles I must keep in mind their characteristics. They are:
- Universal – apply to every organisation.
- Self-Validating – have been proven in practice.
- Empowering – give the practitioner confidence.
Any testing activity or principle which I suggest parallels a risk management principle must meet these criteria. It must be applicable in any project, commonly accepted and give clear guidance to the tester which can be explained to a stakeholder.
Principle 1: Aligns with objectives.
Principle 1: Aligns with objectives. Risk management aligns continually with organisational objectives.
The key indicators of this principle are:
Indicator A
Indicator A: The organisation must pay close attention to understanding objectives so that balance can be achieved between maximising opportunities and minimizing threats.
Does testing stand up well to this indicator? Firstly, we can identify the opportunities and threats that are in scope for testing. These are the objectives that must be balanced.
Opportunity and Threat
The opportunity is the benefit that the application under test offers to the organisation if / when it is delivered and working as intended. Testers and testing material are usually clear that we are striving to deliver a quality application.
The threat we are concerned with is clear: the possibility that the application will not work correctly. Therefore it will fail to deliver the advantage for which it has been commissioned. i.e. It might have defects. All testers and many testing principles are clear that this is what we are there to address.
Balance
The sense of balance exists in testing. The balance we are concerned with is between too much and too little testing. As there is almost certainly a financial element to the objective of the project and application balance is important. Therefore, we must always be careful not to spend more time and money testing it than it is worth. I believe that most testers are aware of this. The balance of cost against benefit is certainly something that I discuss in the classroom. As a result, I have been known to say that too much testing is just as bad as too little – at least from a financial perspective (not from a safety critical perspective).
Does testing ensure that “the organisation” is involved in this process? The way that typical testing practices engage with this is by inviting stakeholders to sign off on the scope of testing as part of the test plan. Later, we display results of testing to those stakeholders through reports. Unfortunately, this is not always a robust process. The stakeholders may not be actively considering, let alone guiding, the scope. The test manager or team often proposes the scope which is then passively signed off by the stakeholders. Thus the stakeholders may sign off with little or no understanding of what they are engaging with.
Some testers / teams / methodologies do a far better job of involving stakeholders in the initial scope discussions and sign off. That is more through their own initiative than through established and stated global practices. Agile teams innately do a better job simply because of their closer communications with the Product Owner and other stakeholders.
Lessons to be Learned?
On the whole, I believe that this first indicator maps clearly to testing. Each element is identifiable and, in some way, addressed in the typical testing material. I believe that the commonly stated objectives and principles of testing could do a better job here. They should make the importance of engaging the organisational stakeholders clear and provide guidance on how this might be achieved. I also feel that we could be clearer in stating the need for balance between too much and too little testing. This would help those stakeholders understand and participate in scope discussions.
(Reasonable questions right now would be: Does risk management offer practices to meet these objectives? Does it simply make high handed statements with no back up? That is a topic for a future article – but the short answer is yes it does offer practices).
It is worth noting that this item is indicator one of principle one in Risk Management and already there are lessons to be learned.
Indicator B
Indicator B: The amount of risk that an organisation is willing to take and the associated amount of risk management that is carried out must align to objectives. To do this an organisation should determine its risk capacity and risk appetite.
Does this map to testing? Oh yes – principle number one in testing is that we cannot prove that there are no defects. In other words, we can never reduce the risk of defects going live to zero.
Unfortunately, our stakeholders are not always as clear on this as the testers are. This is in part because of weaknesses in properly engaging them in the process – see above. Consequently, we are often unclear on what their real needs are. We do not have a clear understanding of what level of quality they need. (Quality and level of quality are our words for the opposite of risk and risk appetite). Either we, or they, are left unclear on how much testing is enough – neither too little, nor too much. The scope of testing and the stakeholders’ acceptance of this via sign-off is the way we address it. Because of the weaknesses discussed above, this is often a poorly understood and poorly quantified agreement.
I do not believe that the existing commonly stated practices of testing do a good job here. They should express the need for clarity from the organisation in setting out its need for quality and tolerance for risk. This then feeds back into the troubles discussed in indicator A.
Indicator C
Indicator C: A key aspect of successful risk management is the shared understanding between stakeholders that risk is dynamic not static. …risk management must be a repetitive process that anticipates and is responsive to change…
Does this map? Again yes. We are well aware, as testers, that as we test and as things change in the application, we will identify the need for additional tests in order to address newly appearing risks. Do we do a good job of communicating this moveable feast of risk to our stakeholders? Not so much. There is no automatic process in the common testing materials for re-presenting the scope to the stakeholders for their approval. We often discuss the need for change with the project manager (or team in Agile) and that may be sufficient – but this is not established in principle or process. Again, individual teams or methodologies may well do a better job of this.
Risk and Testing Conclusion
In summary – management of risk principle one does seem to map clearly to testing, further supporting the argument that testing and risk management are the same. It also seems to highlight some weaknesses that are carried in our commonly shared materials around how to engage with stakeholders to the benefit of both them and testing teams.
Next time: Principle 2
Joe Elledge is an accredited trainer of BCS, ICAgile, iSQI, PeopleCert and ISTQB certified courses, with over 20 years’ experience in test management, quality assurance and training roles. Joe has worked in a variety of industries, including Telecommunications, Banking, Insurance, and automotive and in a variety of geographical regions, including UK, Ireland, Germany, India, America, Scandinavia, and Switzerland. Find out more about us and our other Expleo Academy trainers.
