A curated list of useful and relevant resources for eschewing common-but-flawed approaches and pitfalls of risk management and metrics and decision-making.
If you have a resource that you think should be here, fork it and submit a PR! Will review and integrate if it looks good.
And things to challenge assertions like:
"We can't do accurate forecasting in security because we don't have data!"
(you know more than you think)
"We don't have a methodology now (so decisions are arbitrary), so any methodology is better than no methodology!"
(bad methodologies lead to bad decisions - and worse - overconfidence in those bad decisions)
- Salah, O. (2018, November 03). 13 Reasons Why ... Heat maps must die. Youtube. Retrieved from https://www.youtube.com/watch?v=7IcRtz7qo2w
-
- Hubbard, D. W. (2007). How to Measure Anything. John Wiley & Sons. Retrieved from https://www.goodreads.com/book/show/444653.How_to_Measure_Anything
-
- Hubbard, D. W. (2009). The Failure of Risk Management. Wiley. Retrieved from https://www.goodreads.com/book/show/6516407-the-failure-of-risk-management
-
- Hubbard, D. W., & Seiersen, R. (2016). How to Measure Anything in Cybersecurity Risk. Wiley. Retrieved from https://www.goodreads.com/book/show/26518108-how-to-measure-anything-in-cybersecurity-risk
-
- Muller, J. Z. (2018). The Tyranny of Metrics. Princeton University Press. Retrieved from https://www.goodreads.com/book/show/36644895-the-tyranny-of-metrics
- Duke, A. (2020). How to Decide. Portfolio. Retrieved from https://www.goodreads.com/book/show/51066664-how-to-decide
- Ariely, D. (2008). Predictably Irrational. HarperCollins Canada. Retrieved from https://www.goodreads.com/book/show/1713426.Predictably_Irrational
- Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux. Retrieved from https://www.goodreads.com/book/show/11468377-thinking-fast-and-slow
- Perrow, C. (1999). Normal Accidents. Princeton University Press. Retrieved from https://www.goodreads.com/book/show/192408.Normal_Accidents
- Ip, G. (2015). Foolproof. Little, Brown and Company. Retrieved from https://www.goodreads.com/book/show/24819490-foolproof?ac=1&from_search=true&qid=21SODqPo6w&rank=1
- Schlosser, E. (2013). Command and Control. Penguin Books. Retrieved from https://www.goodreads.com/book/show/6452798-command-and-control
- Thomas, P., Bratvold, R. B., & Bickel, E. (2013). The Risk of Using Risk Matrices. . doi: 10.2118/166269-MS. Retrieved from https://www.researchgate.net/profile/Reidar-Bratvold/publication/266666768_The_Risk_of_Using_Risk_Matrices/links/552252a70cf29dcabb0d3da9/The-Risk-of-Using-Risk-Matrices.pdf
- Hubbard, Douglas & Evans, Dylan. (2010). Problems with scoring methods and ordinal scales in risk assessment. IBM Journal of Research and Development. 54. 2. 10.1147/JRD.2010.2042914. Retrieved from https://www.researchgate.net/profile/Dylan-Evans-6/publication/220498878_Problems_with_scoring_methods_and_ordinal_scales_in_risk_assessment/links/0912f4fa65c8550faf000000/Problems-with-scoring-methods-and-ordinal-scales-in-risk-assessment.pdf
- Cox L. A., Jr (2008). What's wrong with risk matrices?. Risk analysis : an official publication of the Society for Risk Analysis, 28(2), 497–512. https://doi.org/10.1111/j.1539-6924.2008.01030.x.
- Kaplan, S.M., & Garrick, B.J. (1981). On The Quantitative Definition of Risk. Risk Analysis, 1, 11-27. Retrieved from https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.323.1418&rep=rep1&type=pdf
- Elmontsri, Mustafa. (2014). Review of the Strengths and Weaknesses of Risk Matrices. The Journal of Risk Analysis and Crisis Response. 4. 49 - 57. 10.2991/jrarc.2014.4.1.6. Retrieved from https://www.atlantis-press.com/article/11718.pdf
- Stevens, S. S. "On the Theory of Scales of Measurement." Science 103, no. 2684 (1946): 677-80. Accessed March 31, 2020. www.jstor.org/stable/1671815.
- Board, C. A. I., & Administration, N. A. a. S. (2003). Columbia Accident Investigation Board Report. US Independent Agencies and Commissions. Retrieved from https://www.goodreads.com/book/show/5256672-columbia-accident-investigation-board-report
- Peerally MF, Carr S, Waring J, et al. The problem with root cause analysis. BMJ Quality & Safety 2017;26:417-422. Retrieved from https://qualitysafety.bmj.com/content/qhc/26/5/417.full.pdf
- Port, D., & Wilf, J. (2019, January 08). Classifying Risk Uncertainty for Decision Making. Retrieved from https://scholarspace.manoa.hawaii.edu/bitstream/10125/60173/0733.pdf
- Dimitroff, R. D., Schmidt, L. A., & Bond, T. D. (2005). Organizational behavior and disaster: a study of conflict at NASA. Project Management Journal, 36(2), 28–38. Available online https://www.studocu.com/en-au/document/university-of-technology-sydney/fundamentals-of-business-finance/21129-dimitroff-organizational/4923874
- In defence of Risk Heat Maps | LinkedIn. (2021, October 12). Retrieved from https://www.linkedin.com/pulse/defence-risk-heat-maps-david-vose
- On the limitations of scoring methods for risk analysis. (2014, June 19). Retrieved from https://eight2late.wordpress.com/2009/10/06/on-the-limitations-of-scoring-methods-for-risk-analysis/#
- Cox’s risk matrix theorem and its implications for project risk management. (2012, December 20). Retrieved from https://eight2late.wordpress.com/2009/07/01/cox%E2%80%99s-risk-matrix-theorem-and-its-implications-for-project-risk-management
- Remes, W. (2019). Let's talk about metrics... Rapid7. Retrieved from https://www.rapid7.com/blog/post/2015/03/26/lets-talk-about-metrics
- How to Measure Anything - LessWrong. (2021, October 12). Retrieved from https://www.lesswrong.com/posts/ybYBCK9D7MZCcdArB/how-to-measure-anything#
- Metrics for the unmeasurable. (2019, January 19). Retrieved from https://lethain.com/metrics-for-the-unmeasurable
- McGeehan, R. (2018). Killing “Chicken Little”: Measure and eliminate risk through forecasting. Medium. Retrieved from https://medium.com/starting-up-security/killing-chicken-little-measure-and-eliminate-risk-through-forecasting-ecdf4c7e9575
- McGeehan, R. Risk Forecasting (presentation). (2018, June 27). Retrieved from https://magoo.github.io/Risk-Forecasting/#1
- McGeehan, R. (2018). How to measure risk with a better OKR. - Starting Up Security - Medium. Medium. Retrieved from https://medium.com/starting-up-security/how-to-measure-risk-with-a-better-okr-c259bccf359e
- Finally! An alternative to risk matrices RISK-ACADEMY Blog. (2019, April 8). Retrieved from https://riskacademy.blog/finally-an-alternative-to-risk-matrices
- Visualising content and context using issue maps – an example based on a discussion of Cox’s risk matrix theorem. (2012, December 20). Retrieved from https://eight2late.wordpress.com/2009/12/18/visualising-content-and-context-using-issue-maps-an-example-based-on-a-discussion-of-coxs-risk-matrix-theorem
- Pukinskis, A. (2021). Calculating Cost of Delay - Alex Pukinskis - Medium. Medium. Retrieved from https://medium.com/@alexjp/calculating-cost-of-delay-3110c16827e9
- Marks, Norman. How great is your cyber risk? (2021, September 16). Retrieved from https://normanmarks.wordpress.com/2021/09/16/how-great-is-your-cyber-risk
- The Problem with Risk Scores and a Risk Matrix | Safesmart. (2020, January 16). Retrieved from https://safesmart.co.uk/problem-risk-scores-and-a-risk-matrix
- Risk Management Theatre: On Show At An Organization Near You - Continuous Delivery. (2018, December 23). Retrieved from https://continuousdelivery.com/2013/08/risk-management-theatre
- How to interpret ordinal data (Achilleas Kostoulas). (2020, December 17). Retrieved from https://achilleaskostoulas.com/2014/02/23/how-to-interpret-ordinal-data/#content
- Degrandis, D., & DeMaria, T. (2017). Making Work Visible. IT Revolution Press. Retrieved from https://www.goodreads.com/book/show/36458712-making-work-visible
- Taleb, N. N., Chandler, D., & Климчук, М. (2007). The Black Swan. Random House. Retrieved from https://www.goodreads.com/book/show/242472.The_Black_Swan
- Sanders, L. (2009). Every Patient Tells a Story. Harmony. Retrieved from https://www.goodreads.com/book/show/6691125-every-patient-tells-a-story
- Wardley, S. . Wardley Maps. . Retrieved from https://www.goodreads.com/book/show/39282904-wardley-maps
- Levy, M., & Salvadori, M. (2002). Why Buildings Fall Down. W. W. Norton Company. Retrieved from https://www.goodreads.com/book/show/586996.Why_Buildings_Fall_Down?ac=1&from_search=true&qid=XIvRIH5kuX&rank=1
- Shermer, M., & Gould, S. J. (2002). Why People Believe Weird Things. Holt Paperbacks. Retrieved from https://www.goodreads.com/book/show/89281.Why_People_Believe_Weird_Things
- Wiseman, R. (2007). Quirkology. Basic Books (AZ). Retrieved from https://www.goodreads.com/book/show/978957.Quirkology
- Vanderbilt, T. (2008). Traffic. Knopf Publishing Group. Retrieved from https://www.goodreads.com/book/show/2776527-traffic
- Tzu, S., Cleary, T., & Otkan, P. (2005). The Art of War. Harper Press. Retrieved from https://www.goodreads.com/book/show/10534.The_Art_of_War?ac=1&from_search=true&qid=vfzRhLfGJ7&rank=1
- Lewis, M. (2018). The Fifth Risk. W. W. Norton Company. Retrieved from https://www.goodreads.com/book/show/40109421-the-fifth-risk