What do you compare in a risk level matrix when evaluating the elements of a risk

Risk Impact Assessment and PrioritizationPrintDefinition: Risk impact assessment is the process of assessing the probabilities and consequences of risk events if they are realized.

What do you compare in a risk level matrix when evaluating the elements of a risk

Risk Impact Assessment and Prioritization

Print

Definition: Risk impact assessment is the process of assessing the probabilities and consequences of risk events if they are realized. The results of this assessment are then used to prioritize risks to establish a most-to-least-critical importance ranking. Ranking risks in terms of their criticality or importance provides insights to the project's management on where resources may be needed to manage or mitigate the realization of high probability/high consequence risk events.

Keywords: risk, risk impact assessment, risk management, risk prioritization

MITRE SE Roles & Expectations: MITRE systems engineers (SEs) working on government programs are expected to analyze risks with respect to impact, probability, dependencies, and timeframes and to prioritize risks to facilitate decision making by the sponsor or customers [1].

Background

Risk impact assessment and prioritization are the second and third steps of the process depicted in Figure 1 [2].

Figure 1. Risk Management: Fundamental Steps

Figure 1. Risk Management: Fundamental Steps

Risk Impact Assessment in the Systems Engineering Program

In this step, the impact each risk event could have on the project is assessed. Typically this assessment considers how the event could impact cost, schedule, or technical performance objectives. Impacts are not limited to these criteria, however; political or economic consequences may also need to be considered. The probability (chance) each risk event will occur is also assessed. This often involves the use of subjective probability assessment techniques, particularly if circumstances preclude a direct evaluation of the probability by objective methods (i.e., engineering analysis, modeling, and simulation). Chapters 2 and 4 of Garvey [2] discuss the topic of subjective probability assessments, as well as criteria for assessing a risk event's impact or consequence to a project.

As part of the risk assessment, risk dependencies, interdependencies, and the timeframe of the potential impact (near-, mid-, or far-term) need to be identified. The MITRE-developed RiskNAV® tool is an example of a tool that can help perform this assessment. For additional details, see the Risk Management Tools article in this Guide.

When assessing risk, it is important to match the assessment impact to the decision framework. For program management, risks are typically assessed against cost, schedule, and technical performance targets. Some programs may also include oversight and compliance, or political impacts. Garvey [2] provides an extensive set of rating scales for making these multicriteria assessments, as well as ways to combine them into an overall measure of impact or consequence. These scales provide a consistent basis for determining risk impact levels across cost, schedule, performance, and other criteria considered important to the project. In addition, the Risk Matrix tool can help evaluate these risks to particular programs (see the Risk Management Tools article). Performing POET (Political, Operational, Economic, Technical) and/or SWOT (Strengths, Weaknesses, Opportunities, and Threats) assessments can help determine the drivers of the risks. For more details on these analyses, see the Tools to Enable a Comprehensive Viewpoint article in the Comprehensive Viewpoint topic of the Enterprise Engineering section.

For some programs or projects, the impacts of risk on enterprise or organizational goals and objectives are more meaningful to the managing organization. Risks are assessed against the potential negative impact on enterprise goals. Using risk management tools for the enterprise and its components can help with the consistency of risk determination. This consistency is similar to the scale example shown below, except that the assessment would be done at the enterprise level. Depending on the criticality of a component to enterprise success (e.g., risk of using commercial communications to support a military operation and the impact of the enterprise to mission success, versus risk of using commercial communications for peacetime transportation of military equipment), the risks may be viewed differently at the enterprise level even when the solution sets are the same or similar.

One way management plans for engineering an enterprise is to create capability portfolios of technology programs and initiatives that, when synchronized, will deliver time-phased capabilities that advance enterprise goals and mission outcomes. A capability portfolio is a time-dynamic organizing construct to deliver capabilities across specified epochs; a capability can be defined as the ability to achieve an effect to a standard under specified conditions using multiple combinations of means and ways to perform a set of tasks [2]. With the introduction of capability management, defining the impact of risk on functional or capability objectives may provide valuable insights into what capability is at risk, and which risks could potentially significantly impact the ability to achieve a capability and/or impact multiple capability areas.

In portfolio management, a set of investments is administered based on an overall goal(s), timing, tolerance for risk, cost/price interdependencies, a budget, and changes in the relevant environment over time. These factors are generally applicable to the government acquisition environment (see the Guide article Portfolio Management in the Enterprise Engineering section). For portfolio risk assessment, investment decision, or analysis of alternatives tasks, using categories of risk area scales may be the most appropriate way to ensure each alternative or option has considered all areas of risk. Risk areas may include advocacy, funding, resources, schedule and cost estimate confidence, technical maturity, ability to meet technical performance, operational deployability, integration and interoperability, and complexity. Scales are determined for each risk area, and each alternative is assessed against all categories. Risk assessment may also include operational consideration of threat and vulnerability. For cost-risk analysis, the determination of uncertainty bounds is the risk assessment.

When determining the appropriate risk assessment approach, it is important to consider the information need. Shown below are Probability of Occurrence, Program Risk Management Assessment Scale, and Investment Risk Assessment Scale examples used in MITRE's SE work with government sponsors or clients.

RiskNav Probability of Occurrence Example

1.00

Issue:

1

Certain to occur

0.95-0.99

High:

> 0.95 < 1

Extremely sure to occur

0.85-0.95

High:

> 0.85 <= 0.95

Almost sure to occur

0.75-0.85

High:

> 0.75 <=0.85

Very likely to occur

0.65-0.75

High:

> 0.65 <=0.75

Likely to occur

0.55-0.65

Medium:

> 0.55 <=0.65

Somewhat greater than an even chance

0.45-0.55

Medium:

> 0.45 <=0.55

An even chance to occur

0.35-0.45

Medium:

> 0.35 <=0.45

Somewhat less than an even chance

0.25-0.35

Low:

> 0.25 <=0.35

Not very likely to occur

0.15-0.25

Low:

> 0.15 <=0.25

Not likely to occur

0.00-0.15

Low:

> 0.00 <=0.15

Almost sure not to occur

Table 1. Program Risk Management Assessment Scale Example

Table 1. Program Risk Management Assessment Scale Example

Table 2. Example of the Investment Risk Assessment Scale

Technical Maturity

Technical Performance

Integration/Interoperability

Details

Maturity of technologies associated with the alternative

Confidence in performance expectations

This refers to Integration and Interoperability (I&I) issues as they affect the alternative's ability to achieve its stated outcome. The extent that I&I is understood has been demonstrated. It is assumed I&I considerations associated.

Low

Key technologies are ready and mature and require little/no effort in time to execute the alternative

There are no technical or performance expectations identified that will have any impact on achieving the stated outcome objectives expected from the alternative

For this alternative, I&I considerations are well understood. Most of the challenging concerns have been resolved and/or successfully tested/demonstrated under representative or actual field conditions. As such, I&I considerations are not expected to have severe negative impact on the ability of this alternative to achieve its stated objectives.

Low med

Key technologies are expected to be ready and mature in time to execute the alternative

Limited technical or performance expectations identified that will have a minor impact on achieving the stated outcome objectives expected from the alternative

For this alternative, I&I considerations are very well understood. Some challenging concerns have not been resolved and/or successfully tested/demonstrated under representative or actual field conditions. As such, I&I considerations are expected to have negligible impact on the ability of this alternative to achieve its stated objectives.

Medium

Key technologies are not ready and mature and require moderate effort to implement the alternative

Technical or performance limitations have been identified that will have a moderate impact on achieving the stated outcome objectives expected from the alternative

For this alternative, I&I considerations are somewhat-borderline understood. Nearly all (including the most challenging concerns) have been resolved and/or successfully tested/demonstrated under representative or actual field conditions. As such, I&I considerations are expected to have modest negative effects on the ability of this alternative to achieve its stated objectives.

Medium-High

Key technologies are not ready and mature and require significant effort to implement the alternative

There are no technical or performance expectations identified that will have any impact on achieving the stated outcome objectives expected from the alternative

For this alternative, I&I considerations are somewhat-borderline understood. Nearly all (including the most challenging concerns) have been resolved and/or successfully tested/demonstrated under representative or actual field conditions. As such, I&I considerations are expected to have significant negative effects on the ability of this alternative to achieve its stated objectives.

High

Key technologies will not be ready and mature and will have a severe impact on the alternative

Major technical or performance issues have been identified that will have a severe impact on achieving the stated outcome objectives expected from the alternative

For this alternative, I&I considerations are not very well understood. Many challenging concerns are not resolved and/or successfully tested/demonstrated under representative or actual field conditions. As such, I&I considerations are expected to have severe negative effects on the ability of this alternative to achieve its stated objectives.

Catastrophic

Key technologies will not be available and there is no alternative

Serious technical or performance issues have been identified that will prevent achieving any of the stated outcome objectives expected from the alternative

For this alternative, I&I considerations are show-stoppers with the respect to the ability of this alternative to achieve its stated objectives.

Risk Prioritization in the Systems Engineering Program

In the risk prioritization step, the overall set of identified risk events, their impact assessments, and their probabilities of occurrences are "processed" to derive a most-to-least-critical rank-order of identified risks. A major purpose of prioritizing risks is to form a basis for allocating resources.

Multiple qualitative and quantitative techniques have been developed for risk impact assessment and prioritization. Qualitative techniques include analysis of probability and impact, developing a probability and impact matrix, risk categorization, risk frequency ranking (risks with multiple impacts), and risk urgency assessment. Quantitative techniques include weighting of cardinal risk assessments of consequence, probability, and timeframe; probability distributions; sensitivity analysis; expected monetary value analysis; and modeling and simulation. MITRE has developed the min- and max-average approaches (using a weighting scale more heavily weighting the max or min value). Expert judgment is involved in all of these techniques to identify potential impacts, define inputs, and interpret the data [3].

In addition, MITRE has developed the RiskNAV® tool that assists managers in assessing and prioritizing program risks. RiskNav includes the ability to weight timeframe in the risk ranking (e.g., how much time to react/potentially mitigate). RiskNav is used by a number of MITRE sponsors and customers. For detailed information on RiskNav, refer to the article on Risk Management Tools in this Guide.

Best Practices and Lessons Learned

Tailor the assessment criteria to the decision or project. When assessing risks, recommend techniques and tools that are suitable for the analysis. For example, if the project is an enterprise management or organizational oversight project, then risk impact might be most suitably assessed against goals in lieu of technical performance, cost, and schedule. If the assessment is to determine the risk of investment options, the risk area scale approach might be best suited. For an example of application of risk management, refer to the Cryptologic Systems Group's Risk Management Implementation Guide [4].

Document the rationale for the assessment of impact and probability. It is important to document the justification or rationale for each risk impact assessment and probability of occurrence rating. If the conditions or environment change, the assessment might need to be revisited. The rationale helps to communicate the significance of the risk. When using the investment assessment scale approach, the statement of risk is typically captured in the rationale.

Recognize the role of systems engineering. Risk assessment and management are roles of systems engineering, especially as projects and programs become more complex and interdependent. The judgments that are involved require a breadth of knowledge of system characteristics and the constituent technologies beyond that of design specialists. In addition, the judgments of risk criticality are at the system and program levels [5]. Risk cuts across the life cycle of systems engineering, and MITRE SEs should be prepared to address risk throughout—concept and requirements satisfaction, architectural level risks, design and development risks, training risks, fielding, and environment risks. MITRE SEs are encouraged to advocate for SE involvement in risk assessment and management.

Tailor the prioritization approach to the decision or project. Match the prioritizing algorithm, techniques, and tools to the assessment need (e.g., needs could include time criticality as a prioritization factor, the ability to see capability at risk, the need for a single risk score for the portfolio, the ability to have insight into risks with multiple impacts, and more). Each risk area—threat, operations, programmatic, etc.—will have different priorities. Typically, there will be a priority to these areas themselves —a major threat risk could be totally unacceptable and the effort may be abandoned. If the threat risks are acceptable but the operations cannot be effectively performed, then, again, the effort may be abandoned. Be sure to consider these various decisions and criticality to help the government assess the priorities of mitigating the risks that arise.

Consider MITRE's RiskNAV® tool. RiskNav might be appropriate for assessing and prioritizing risks on your government program. MITRE SEs have found that having easy access to this well-tested tool and a support team sometimes encourages government program teams to adopt a more robust risk management process than they otherwise might have. Refer to the article Risk Management Tools in this Systems Engineering Guide for additional information.

Consider Monte Carlo simulations. Monte Carlo simulations use probability distributions to assess the likelihood of achieving particular outcomes, such as cost or completion date [3]. They have been used effectively on a number of MITRE government programs to help the project teams assess schedule risk.

References & Resources

  1. The MITRE Institute, September 1, 2007, "MITRE Systems Engineering (SE) Competency Model, Version 1," pp. 11, 41-42.
  2. Garvey, P.R., 2008, Analytical Methods for Risk Management: A Systems Engineering Perspective, Chapman-Hall/CRC Press, Taylor & Francis Group (UK), Boca Raton, London, New York, ISBN: 1584886374.
  3. A Guide to the Project Management Body of Knowledge, (PMBOK Guide), Fourth Edition, ANSI/PMI 99-001-2008, pp. 273-312.
  4. Cryptologic Systems Group, July 2007, CPSG Risk Management Implementation Guide, Version 1.2.
  5. Kossiakoff, A. and W.N. Sweet, 2003, Systems Engineering Principles and Practice, John Wiley and Sons, Inc., pp. 98-106.

Additional References & Resources

  • Garvey, P.R., January 2000, Probability Methods for Cost Uncertainty Analysis: A Systems Engineering Perspective, Chapman-Hall/CRC Press, Taylor & Francis Group (UK), Boca Raton, London, New York, ISBN: 0824789660.
  • International Council on Systems Engineering (INCOSE), January 2010, INCOSE Systems Engineering Handbook, Version 3.2, INCOSE-TP-2003-002-03.2, p. 213-225.
  • Lavine, M., S. McBrien, J. Ruddy, and M. Yaphe, August 2006, Methodology for Assessing Alternative IT Architectures for Portfolio Decisions, MITRE Technical Report 06W0000057.
  • McMahon, C. and R. Henry, September 2009, "RiskAssessment Scales—Sept09."
  • MITRE SEPO, Risk Management Toolkit.
  • Stevens, Renee, "Engineering and Acquiring Mega-Systems: Systems Engineering Challenge for the XXI Century," (aka. Megasystems Engineering), June 13, 2007, Presentation to ATEC Professional Development Day.

Video liên quan