Statistical Approaches for Imputing Race and Ethnicity
Background
The Society of Actuaries (“SOA”) Research Institute is the research arm of the SOA, providing objective, data-driven research that brings together tried and true practices and future-focused approaches to address societal challenges and business needs. The Institute provides trusted knowledge, extensive experience and new technologies to help effectively identify, predict and manage risks. The Institute dedicates part of its efforts to objectively exploring issues within the insurance, retirement and financial industries that affect people of color or other dimensions of diversity in the United States.
Datasets that are useful for research and analysis related to actuarial products and services in the United States often do not include variables for race or ethnicity. In addition, when datasets include variables for race or ethnicity, the variable can be unreliable if many records are blank or populated with “other,” “unknown,” “prefer not to answer,” etc. When evaluating outcomes of processes for potential bias in the U.S., statistical methods for imputing or modeling race and ethnicity can be helpful tools in some of these situations. While they are available, they are not widely known or understood within the actuarial profession.
Purpose and Research Objective
To help familiarize actuaries in industry with statistical methods for imputing or modeling race and ethnicity in the U.S., the Institute’s Diversity, Equity and Inclusion Strategic Research Program is seeking a researcher to outline the various methods, including their strengths and weaknesses. The project intent is a methodology review rather than original thinking. The review should assume that the user will be working with structured datasets in an insurance context, for example, data associated with an in-force life or health insurance product.
The deliverable of an ideal proposal will be a report that includes, but is not necessarily limited to, the following items:
- Identification and description of the methods that are available or currently under development, including varying approaches in application of the methods as well as specific software and/or libraries that are commonly used to implement the methodology
- A focus on application rather than theory
- Written in plain language that limits the use of statistical jargon and technical terms
- Presentation of key formulas for each method or approach to the extent it helps differentiate methods, although development of the formulas is not needed
- Discussion of the following for each method:
- The prominence of the method, including the reasons for such prominence
- Circumstances or purpose(s) for which a method was designed
- Description of the specific dataset(s), if any, that were used to develop the method
- Which races and ethnicities the method will impute, how the method handles other races/ethnicities, and how the method handles mixed race/ethnicity
- Key criteria to be satisfied before using a method or approach, if applicable
- Each method or approach’s strengths, weaknesses, and limitations
- Key considerations for determining whether to use a method and/or its results
- Considerations for datasets that include a race/ethnicity variable(s) that is largely incomplete
- Model validation methodologies, including the viability of using self-reported racial or ethnic data from a third-party dataset
- Does the method tend to over- or underestimate results?
- Determination of relevant performance metrics, such as error rate, and thresholds for acceptance
- Considerations and potential impact on results of matching personal identifiable information (PII) in the user’s dataset to third-party datasets.
- Evaluation of the impact of false negatives versus false positives, as well as how errors impact results and decisions
- Usage of a sample dataset to demonstrate how results differ across methodologies and to highlight the methodologies’ strengths and weaknesses
- For methods currently under development, brief discussions of where they are in the development process and expectations as to their application and performance
- A concise summary of the key points above for each method or approach
Proposal Requirements
To facilitate the evaluation of proposals, the following information should be submitted:
- Resumes of the researcher(s), including any graduate student(s) expected to participate, indicating how their background, education and experience bear on their qualifications to undertake the research. If more than one researcher is involved, a single individual should be designated as the lead researcher and primary contact. The person submitting the proposal must be authorized to speak on behalf of all the researchers as well as for the firm or institution on whose behalf the proposal is submitted.
- An outline of the approach to be used (e.g., literature search, model, etc.), emphasizing issues that require special consideration. Details should be given regarding the techniques to be used, collateral material to be consulted, and possible limitations of the analysis.
- A description of the expected deliverables and any supporting data, tools or other resources.
- Cost estimates for the research, including computer time, salaries, report preparation, material costs, etc. Such estimates can be in the form of hourly rates, but in such cases, time estimates should also be included. Any guarantees as to total cost should be given and will be considered in the evaluation of the proposal. While cost will be a factor in the evaluation of the proposal, it will not necessarily be the decisive factor.
As a guide for developing the project budget, please review the Historical Project Cost Guide (see Appendix) - A schedule for completion of the research, identifying key dates or time frames for research completion and report submissions. The SOA Research Institute is interested in completing this project in a timely manner. Suggestions in the proposal for ensuring timely delivery, such as fee adjustments, are encouraged.
- Other related factors that give evidence of a proposer's capabilities to perform in a superior fashion should be detailed.
Selection Process
The SOA Research Institute will appoint a Project Oversight Group (POG) to oversee the project. The [sponsoring entity/POG] is responsible for recommending the proposal to be funded. Input from other knowledgeable individuals also may be sought, but the SOA Research Institute will make the final recommendation, subject to SOA Research Institute leadership approval. An SOA Research Institute staff research actuary will provide staff actuarial support.
Questions
Any questions regarding this RFP should be directed to Lisa Schilling, senior research actuary, at lschilling@soa.org.
Notification of Intent to submit Proposal
If you intend to submit a proposal, please email written notification by March 31, 2023, to Lisa Schilling at lschilling@soa.org.
Submission of Proposal
Please email your proposal to Lisa Schilling at lschilling@soa.org; proposals must be received no later than April 14, 2023. It is anticipated that all proposers will be informed of the status of their proposal by the end May 2023. The research project will begin as soon as legal contracting is completed, or at another time as mutually agreed upon by the SOA Research Institute and lead researcher.
Conditions
The selection of a proposal is conditioned upon and not considered final until a Letter of Agreement is executed by both the SOA Research Institute and the researcher.
The SOA Research Institute reserves the right to not award a contract for this research. Reasons for not awarding a contract could include, but are not limited to, a lack of acceptable proposals or a finding that insufficient funds are available. The SOA Research Institute also reserves the right to redirect the project as is deemed advisable.
The SOA Research Institute plans to hold the copyright to the research and to publish the results with appropriate credit given to the researcher(s).
The SOA Research Institute may choose to seek public exposure or media attention for the research. By submitting a proposal, you agree to cooperate with the SOA Research Institute in publicizing or promoting the research and responding to media requests.
The SOA Research Institute may also choose to market and promote the research to members, candidates and other interested parties. You agree to perform promotional communication requested by the SOA Research Institute, which may include, but is not limited to, leading a webcast on the research, presenting the research at an SOA meeting, and/or writing an article on the research for an SOA newsletter.
Conflict of Interest
You agree to disclose any of your material business, financial and organizational interests and affiliations which are or may be construed to be reasonably related to the interest, activities and programs of the SOA Research Institute.
Appendix
The cost ranges below are intended as a guide for budgeting project costs for proposals in response to SOA Research Institute Request for Proposals (RFP). Please note these figures span the 33rd to 66th percentiles for all projects as well as projects that involve a specific approach (lit review, survey, etc.). They are based on historical costs over several recent years. Expected costs for some RFPs may fall outside these ranges depending on the nature of the work and resources required for completion.
All Contracted Projects
This category includes all contracted projects that the Institute has undertaken within the last several years. The 33rd–66th percentile project costs range is $25,000–$50,000.
Literature Reviews
This category includes projects that involved only a literature review or the cost for the portion of a larger project that included a literature review. The 33rd–66th percentile project costs range is $15,000–$20,000.
Surveys
This category includes all projects that had a survey as their primary component. The 33rd–66th percentile project costs range is $28,000–$55,000.