Model Validation and Common LDTI Validation Observations

By Jack Zheng and Tina Cao

The Financial Reporter, January 2024

Risk management remains one of the top concerns for company management teams globally. Having proper risk management allows companies to adequately identify, assess and deal with potential risks. One benefit is that it allows management teams and external stakeholders to have greater trust in the numbers presented on the company’s financial statements. The introduction and effective application of long-duration targeted improvements (LDTI) have increased the scrutiny of those reviewing the updated financial results. That is because LDTI has resulted in significant changes in actuarial reserving and deferred acquisition cost (DAC) models for long-duration contracts, including significant changes to data, assumptions, and required disclosures. In light of the changes, greater reliance has been put on model risk management (MRM) teams and model validations to assess the conceptual soundness and accuracy of their companies’ actuarial models and to assess that the underlying data and processes utilized are suitable and appropriate.

In this article, we describe a generalized model validation framework and provide a sample model validation test plan for an LDTI model validation. We also summarize our observations from model validations we have performed, highlighting the key areas to which companies should pay more attention.

Model Validation Framework and Sample Test Plan for LDTI

A good model validation always begins with an established, formalized, and well-socialized model validation framework. The framework should provide adequate coverage to a variety of potential model risk sources and be applicable across different types of models. A model validation framework that companies can use is provided below with high-level considerations for each component:

• Data suitability and quality: Assessment of data sources and quality, including data inputs and assumptions.
• Conceptual soundness: Assessment of model theory, design, and construction with considerations of applicable regulatory requirements, consistency with industry practice, and intended uses of model output and primary users.
• Model performance and integrity: Replication and re-performance of key model development using independent testing of sample or full results, sensitivity analysis, and benchmarking.
• Model implementation: Assessment of processes and controls, model change control procedures and ongoing monitoring.
• Documentation and governance: Assessment of model transparency and use, effective testing by model developers and owners, as well as documentation completeness.

Following the model validation framework, a detailed test plan should be developed for the specific model to be validated. A brief sample test plan for an LDTI model is provided below, focusing on the LDTI aspect of the validation.

Common LDTI Validation Findings and Observations

We have performed multiple validations across different companies and products. We saw companies having challenges related to LDTI model development and implementation across areas including historical actual cash flow development, future cash flow projection, LDTI methodology, and disclosures. Based on our experience, below are areas that companies should pay special attention to as they are developing and validating the LDTI models:

• LDTI methodology: Numerous new methodology decisions are required for LDTI implementation, so companies need to make sure that the LDTI model utilizes the methodology decisions documented in the accounting policy document.
• LDTI assumptions: Existing models may be leveraged and modified for LDTI purposes. When that happens, companies need to make sure the assumptions in these new models have been updated for LDTI requirements, which requires using best-estimate assumptions at the valuation date.
• Calculation of ceded reinsurance reserves: In the measurement of liability for future policy benefits (LFPB) for direct business, LFPB needs to be floored at zero, and the net premium ratio (NPR) may not exceed 100%. On the reinsurance side, reserves should be recognized in a manner consistent with the direct liabilities. However, companies need to consider how the LFPB flooring and NPR capping on the direct side would impact the calculations of the reserves (e.g., floor the ceded LFPB and cap the ceded NPR to the extent the insurer has recognized a loss on the associated direct contracts).
• Modeling of riders: Some companies that have separate rider reserves under current GAAP may decide to have a single integrated reserve under LDTI. For companies that have an integrated reserve under LDTI, they need to make sure all rider cash flows are captured in the NPR and LFPB calculations.
• Second-degree impact to SOP 03-1: While LDTI does not explicitly change how SOP 03-1 is measured, it will have a secondary impact on the measurement of the SOP 03-1 liability as the amortization pattern of unearned revenue reserve (URR) changed under LDTI. The projection of assessments used to calculate SOP 03-1 liability need to incorporate the new projection of URR. Thus, companies need to consider where and when the impact should flow through (e.g., retained earnings versus net income and transition date versus effective date).
• Treatment of LFPB and deferred profit liability (DPL) disclosure roll-forwards:
• For the cohort issue year, companies need to consider which line items will be used to account for any effect of actual experience variances and/or cash flow assumption change that occurred during the year. Companies need to consider whether the effects are included in the “issuance” line or flow through other lines in the LFPB and DPL disclosure roll-forward.
• LDTI requires the reserves to be floored at zero both on a “locked-in” and “current” discount rate basis for LFPB. When reserves are positive at the beginning of the period and become negative by the end of the period, there may be a need to adjust certain disclosure roll-forward line items to reflect the flooring impact and make sure the roll-forward reconciles.
• Market risk benefit (MRB) attributed fee ratio (AFR) calculation: AFRs need to be determined in MRB calculations when the attributed fee approach (i.e., non-option valuation approach) is elected. Companies should make sure that AFRs are capped at 100% if the attributed fees exceed total contractual fees. Also, fee amounts used should be consistent with the fees specified in the policy forms and the benefit cash flows should only include rider benefits in the AFR calculations.
• Treatment of MRB disclosure roll-forward: Each line item of the MRB disclosure roll-forward needs to be determined correctly. Companies may face challenges on developing this disclosure due to the number of attribution runs that need to be performed and the complexity of assembling data for model runs and analyzing the model output for each line item.
• Data gathering: Data needed for LDTI modeling is largely different from the data used under prior GAAP. For non-MRB LDTI balance calculations, companies need to gather cohort-level actual historical cash flows prior to the valuation date and consider how some product-level historical data could be appropriately allocated to the cohort level. For the MRB calculation, companies may need to get seriatim-level data back to policy issuance to calculate the AFR.
• Model process: Companies need to consider the modeling process holistically including data gathering, data storing, data inputting into the model, LDTI calculations, and obtaining model results. The process needs to be well thought out and designed to handle calculations previously not performed, such as the determination of the locked-in discount rates for a new business cohort and the need to maintain two sets of calculations: One using the locked-in rates and the other one using the current discount rates at future dates.

We also came across issues not specific to LDTI. For example, we have found issues with account values being calculated incorrectly, incorrect information in the valuation in-force files, and certain product features and characteristics incorrectly modeled or not modeled at all. This shows that performing a model validation for LDTI can also uncover previously hidden issues with other, older models.

Concluding Thoughts

While this paper focused on LDTI-related model validation, LDTI is not the only new accounting standard introduced that would require more reliance and support from the MRM team. International Financial Reporting Standard 17 (IFRS 17) and statutory principle-based reserves (PBR) have also required significant changes to actuarial models—and some may argue even more so than LDTI. With these changes, the need for validation is greater than ever, causing a strain on MRM resources.

As a result, companies should consider taking an agile approach to risk management and model validation. Performing model validation does not need to wait until all model components have been built and all inputs and outputs have been finalized. When certain model components are ready, model owners and developers can start getting the model validation team involved to begin testing those components. Given all the changes and complexities introduced by LDTI and other accounting standards, it is important to allow more time for a carefully thought out and comprehensive model validation.

The statements of fact and opinions expressed herein are the views of the authors and do not necessarily reflect the views of Ernst & Young LLP or other members of the global EY organization, the Society of Actuaries, or the newsletter editors.

Jack Zheng, FSA, MAAA, is a consulting senior manager at Ernst & Young LLP. He can be reached at jack.zheng@ey.com.

Tina Cao, FSA, MAAA, is a consulting manager at Ernst & Young LLP. She can be reached at tina.cao1@ey.com.