Professionalism in Your Modeling Career
By Donna Megregian
Product Matters!, June 2021
Modeling is a key part of an actuarial career. No, not strutting down a cat walk, but the wonderfully expanding career of building models to simulate results. The 4th exposure draft on Modeling came out in February of 2019 and comments were requested and submitted by May 15, 2019. The Actuarial Standards Board (ASB) in December 2019 approved Actuarial Standard of Practice (ASOP) for Modeling. The Modeling ASOP—ASOP 56—applies to models across all practice areas and “provides guidance to actuaries when performing actuarial services with respect to designing, developing, selecting, modifying, using, reviewing or evaluating models.” One item that was removed from the last exposure draft is 3.1.6.b on margins because it did not add clear guidance, as noted in the ASB newsletter. ASOP 56 was effective Oct. 1, 2020 and is available on the ASB webpage.
Constructing a Modeling ASOP
The modeling ASOP has been in the making for years now—with the first draft released in June 2013, the second in November 2014, the third in June 2016, and the fourth in December 2018. You may be asking why it took so long. Being that the Modeling ASOP is cross disciplinary, it must provide guidance for many different practice areas, and that level of agreement was harder to achieve.
A key theme from the Modeling ASOP is intended purpose. Intended purpose is defined in the ASOP as “the goal or question, whether generalized or specific, addressed by the model within the context of the assignment.” As companies consider the models used in daily actuarial work, a variety of questions are posed through the lens of intended purpose:
- Is the capability of the model consistent with the intended purpose? For example, is the level of detail in the model sufficient or might something more granular be needed?
- Does the model reasonably meet the intended purpose?
- Do you understand the model?
- Are revisions to the model consistent with the intended purpose?
- Is the structure of the model appropriate for the intended purpose? For example, should the model be deterministic or stochastic or is it overfitting?
- Is the data used in the model appropriate for the intended purpose while still considering ASOP 23?
- Are the assumption and parameters used as input appropriate in light of the model’s intended purpose?
Reliance on others needs to be disclosed, as is required by many ASOPs. Whether data, models or expert input is supplied by others, the actuary needs to disclose the level of reliance. Although reliances may be disclosed, the actuary should understand the extent of the reliances and be comfortable with the overall impact to the model.
The Modeling ASOP also allows for some guidance on the level of testing and validation with the intent of mitigating model risk. Review and governance of the model are part of the essential mitigation of risk associated with the model, as well as reasonable levels of documentation for the process of model development.
Many actuaries work with models—pricing, valuation, predictive, projections, assumption setting, scenario creation, etc. Given the heavy reliance most actuaries have on models, the Modeling ASOP does provide some broad based guidance on ownership or review of a model for all practice areas.
How thorough is your review or validation process for your models? When did you last check the models or spreadsheets you use to check your models? For processes that have been in existence for a while, it may be time for review to ensure the purpose and results of the models are meeting the intent of the model. Adherence or deviation from the ASOP will need to be noted in documentation going forward.
Statements of fact and opinions expressed herein are those of the individual authors and are not necessarily those of the Society of Actuaries, the editors, or the respective authors’ employers.
Donna Megregian, FSA, MAAA, is a vice president, US Mortality Markets with Reinsurance Group of America. She can be reached at dmegregian@rgare.com.