Link to the search page

Reproducible Research

This article summarizes use of programming collaboration tools and source and version control (SVC) to make research products reproducible.

et-2022-06-holland-hero.jpg

The SOA's Individual Life Experience Committee (ILEC) is responsible for the ongoing reporting of experience on standard, fully underwritten individual life insurance policies. For several years ILEC has shared data from published reports in the form of pivot tables and the underlying data files. I am happy to serve on ILEC and to support this transparency. The slogan "reproducible research" is catching on in different contexts. That slogan aptly captures a way we would like to grow. Reproducing research can involve code as well as data. It is easy to share code and even to collaborate on it with current tools. To show how that looks, I am sharing code from past presentations that I have given at ILEC sessions at SOA meetings. These tools might suit us collectively for future work.

The site https://github.com/SOA-ILEC-Demo/ILECTools is a code repository where you can also see further comments, and notebooks using the code. The notebooks are "Jupyter notebooks," which include a mixture of code in Python and comments. Please do have a look, for example at https://github.com/SOA-ILEC-Demo/ILECTools/blob/master/sample_notebooks/Beyond_Actual_to_Table.ipynb. That notebook refers to the 2018 paper of the same name, which contrasts Actual/2015VBT ratios with adjustment factors when controlling for multiple effects. Other notebooks include procedures to download and preprocess the datasets from the SOA, and to generate exhibits in past presentations that I've made at SOA meetings. Now others can see the whole process.

The collaborative tool, GitHub, allows individuals to log an issue with a project, and for themselves or others to work on it and offer up a solution. It can be used with git, which is one of a class of tools called source and version control (SVC). SVC is for tracking who did what, when, and why. To assure top quality work, of course we want any issues found and resolved. An issue might be just a feature to add, or a note that more comments are needed. These tools can be used in private among a smaller group before planetwide publication. A few such points are logged for this repository where you can see. The repository "ILECTools" is under what GitHub calls an organization. That organization is called "SOA ILEC Demo."  It consists of several actuaries, as a demonstration that the code does not just have to be under an individual's GitHub site.

There is considerable overhead in sharing code: Knowing the language(s), and the collaborative tools, such as git and GitHub, which I use. The advantage is, at least, that these tools are widely used, to the point of being unavoidable. The upside is that the available training resources are seemingly limitless.

There is a good tailwind for this direction. The SOA Research Institute has a GitHub presence at https://github.com/Society-of-actuaries-research-institute with several repositories. Contest results have been shared at https://github.com/SOASections. The SOA itself has https://github.com/society-of-actuaries.

I am excited to highlight this direction with concrete steps toward a common-sense goal. I look forward to seeing what you will do next: Even if it is opening issues on this repository!