Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.0 attempt 2 #22

Merged
merged 12 commits into from
Nov 15, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"name": "Python 3",
"image": "mcr.microsoft.com/devcontainers/python:3.7",
"features": {
"ghcr.io/devcontainers/features/node:1": {
"version": "none"
},
"ghcr.io/devcontainers/features/docker-in-docker:1": {},
"ghcr.io/dhoeric/features/act:1": {}
}

// Use 'forwardPorts' to make a list of ports inside the container available locally.
// "forwardPorts": [],

// Use 'postCreateCommand' to run commands after the container is created.
// "postCreateCommand": "pip3 install --user -r requirements.txt",

// Configure tool-specific properties.
// "customizations": {},

// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
// "remoteUser": "root"
}
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
# Auto detect text files and perform LF normalization
* text=auto
*.xml linguist-vendored
*.ipynb linguist-prose
21 changes: 0 additions & 21 deletions .github/workflows/learn-github-actions.yml

This file was deleted.

22 changes: 22 additions & 0 deletions .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
name: run tests

on: [push, pull_request]

jobs:
pytest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: "3.7.1"
- name: Install poetry
uses: abatilo/actions-poetry@v2
with:
poetry-version: 1.1.15
- name: Install requirements
run: poetry install
- name: Install requirements
run: poetry run pytest --cov=./ --cov-report=xml
- name: Upload coverage reports to Codecov with GitHub Action
uses: codecov/codecov-action@v3
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Byte-compiled / optimized / DLL files
__pycache__/
**/__pycache__/
*.py[cod]

# C extensions
Expand Down
1 change: 0 additions & 1 deletion .python-version

This file was deleted.

76 changes: 11 additions & 65 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,20 @@ xml = MortXML(3282)

This `MortXML` class is a wrapper around the [underlying XML](https://mort.soa.org/About.aspx). The autocompletions you get on attributes improve the developer experience over using the underlying XML directly.

![](./assets/auto.png)
![autocompletions](./assets/auto.png)

### Examples
Also, mortality rate tables are Pandas DataFrames.

![rate table as a dataframe](./assets/rate-table.png)


## Accessing mortality rates

For a select and ultimate table we can retrieve rates as follows.

```py
from pymort import MortXML
# Table 3265 is 2015 VBT Smoker Distinct Male Non-Smoker ANB, see https://mort.soa.org/
xml = MortXML(3265)
# This is the select table as a MultiIndex (age/duration) DataFrame.
xml.Tables[0].Values
Expand All @@ -37,7 +45,7 @@ xml.Tables[1].Values

## Usage with tensor libraries

Once we get the data from Pandas to NumPy it should be easy to get into JAX or any other tensor library.
We can get the data from Pandas to NumPy.

```py
select = MortXML(3265).Tables[0].Values.unstack().values
Expand All @@ -49,65 +57,3 @@ ultimate.shape # (103,) is age 18 to 120
# Be careful when indexing into these, ultimate[0] is the rate at age 18!
```

## Relational tables

Pymort provides **3 normalized tables** related by [primary/foreign keys](https://www.ibm.com/docs/en/ida/9.1.1?topic=entities-primary-foreign-keys), a design taken from relational databases.

Currently tables from 2015 VBT and 2017 CSO are supported. If you want support for tables from other studies, open an issue on GitHub.

XML files with multiple "select" (2-dimensional) or multiple "ultimate" (1-dimensional) tables like table [2636](https://mort.soa.org/ViewTable.aspx?&TableIdentity=2636) are not supported so that we can avoid adding another column.

### Metadata

Each table has a unique identifier called an `id`. There is information associated with this table like

- What is the name of the mortality study producing the table? (i.e. 2017 CSO vs. 2015 VBT)
- Is there a grouping that the table belongs to within the study? (i.e. unloaded preferred_structure gender_distinct ANB vs. loaded smoker_distinct gender_blended ALB)
- Gender (male vs. female)
- Risk (smoker vs. nonsmoker)

We call this information about a table the `metadata` and store it as an attribute of our `Relational` object.

```py
from pymort import Relational

db = Relational()

print(db.metadata)
```

![](./assets/meta2.png)

### Select

Select tables depend on the issue age, and the years since issuing the contract.

```py
print(db.select)
```

![](./assets/sel2.png)

### Ultimate

Ultimate tables depend on the attained age only.

```py
print(db.ultimate)
```

![](./assets/ult2.png)

### Groupings

If you want to get the IDs for all tables having the same study and group there is a function for this, although you could do it yourself in Pandas.

```py
from pymort import getIdGroup

getIdGroup(3265)
```

This returns a dataclass representing the grouping.

![](./assets/grouping.png)
Binary file removed __pycache__/tableNames.cpython-39.pyc
Binary file not shown.
Binary file removed assets/grouping.png
Binary file not shown.
Binary file removed assets/meta2.png
Binary file not shown.
Binary file added assets/rate-table.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed assets/sel2.png
Binary file not shown.
Binary file removed assets/ult2.png
Binary file not shown.
2 changes: 2 additions & 0 deletions low-level-api-proposal.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
"source": [
"# Background\n",
"\n",
"**This notebook was authored when I was trying to figure out the structure of the XML so I could write the API**\n",
"\n",
"## XTbML\n",
"\n",
"An XML standard for representing mortality tables called XTbML was developed in a joint effort between the Technology Section of the SOA and the standards-setting body ACORD. For more information, read [this](https://www.soa.org/news-and-publications/newsletters/compact/2013/april/com-2013-iss47/tables-database-goes-xtbml/).\n",
Expand Down
Loading