Managing Model Risk: Lessons and experiences from industry and research on the challenges and dangers of analytical models

Get up to speed on identifying and tackling model risk! Managing Model Risk provides data science practitioners, business professionals and analytics managers with a comprehensive guide to understand and tackle the fundamental concept of analytical model risk in terms of data, model specification, model development, model validation, model operationalization, model security and model management. Providing state of the art industry and research insights based on the author’s extensive experience, this illustrated textbook has a well-balanced theory-practice focus and covers all essential topics. Key Features: Extensive coverage of important trending topics and their risk impact on analytical models, starting from the raw data up until the operationalization, security and management. Various examples and case studies to highlight the topics discussed. Key references to background literature for further clarification. An online website with various add-ons and recent developments: www.managingmodelriskbook.com. What Makes this Book Different? This book is based on both authors having worked in analytics for more than 30 years combined, both in industry and academia. Both authors have co-authored more than 300 scientific publications on analytics and machine learning and have worked with firms in different industries, including (online) retailers, financial institutions, manufacturing firms, insurance providers, governments, etc. all over the globe estimating, deploying and validating analytical models. Throughout this time, we have read many books about analytical modeling and data science, which are typically written from the perspective of a theorist, providing lots of details with regards to different model algorithms and related mathematics, but with limited attention being given to how such models are used in practice. If such concerns are tackled, it is mainly from an implementation, use case or data engineering perspective. From our own experience, however, we have encountered many cases where analytics, AI, machine learning etc. fail in organizations, even with skilled people working on them, due to a myriad of reasons: bad data quality, difficulties in terms of model deployment, lack of model buy-in, incorrect definitions of underlying goals, wrong evaluation metrics, unrealistic expectations and many other issues can arise which cause models to fail in practice. Most of these issues have nothing to do with the actual algorithm being used to construct the model, but rather with everything else surrounding it: data, governance, maintenance, business, management, the economy, budgeting, culture etc. As such, we wanted to offer a new perspective with this book: it aims to provide a unique mix of both practical and research-based insights and report on do's and don'ts for model risk management. Model risk issues are not only highlighted but also recommendations are given on how to deal with them, where possible. Target Audience This book is targeted towards everyone who has previously been exposed to both predictive and descriptive analytics. The reader should hence have some basic understanding of the analytics process model, the key activities of data preprocessing, the steps involved in developing a predictive analytics model (using e.g. linear or logistic regression, decision trees, etc.) and a descriptive analytics model (using e.g. association or sequence rules or clustering techniques). It is also important to be aware of how an analytical model can be properly evaluated, both in terms of accuracy and interpretation. This book aims to offer a comprehensive guide for both data scientists as well as (C-level) executives and data science or engineering leads, decision-makers and managers who want to know the key underlying concepts of analytical model risk.