Skip to content

[PULL REQUEST] Updating fertility scripts - #162

Open
KPHSANDAG wants to merge 5 commits into
mainfrom
161-revising-fertility
Open

[PULL REQUEST] Updating fertility scripts#162
KPHSANDAG wants to merge 5 commits into
mainfrom
161-revising-fertility

Conversation

@KPHSANDAG

Copy link
Copy Markdown
Contributor

Describe this pull request. What changes are being made?

This pr changes the source of fertility data from the data folders data/births to the socioec_data warehouse containing [vital_statistics].[cdc_wonder_fertility]. Existing scripts in birth_rates.py have been modified to account for the source change and new sql files calling the data have been added. All mentions of rates_map have been removed.

Validation and testing

Successful run can be seen in [CohortComponentModel].[outputs].[rates] or in the streamlit report where run_id = 62.

What issues does this pull request address?

closes #161

Additional context

N/A

@KPHSANDAG KPHSANDAG self-assigned this Jul 31, 2026
@KPHSANDAG KPHSANDAG added the enhancement New feature or request label Jul 31, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates fertility (birth) inputs from local data/births files to SQL-backed sources in the socioec_data warehouse (notably [vital_statistics].[cdc_wonder_fertility]), and removes the legacy rates_map configuration/mapping.

Changes:

  • Added SQL queries to load/shape CDC WONDER fertility data, compute “Not Stated” inflation factors, and load DOF P3 projected female populations (15–44) for rate denominators.
  • Updated birth_rates.py to pull fertility inputs from the database and removed the rates_map pathway from configuration and parsing.
  • Removed the legacy rates_map.yml artifact and references from README.md, config.yml, and runtime config parsing/utilities.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
sql/fertility/dof_projections_p3.sql New query to pull DOF P3 female population by race/age for fertility denominators.
sql/fertility/cdc_wonder_fertility.sql New query to shape CDC WONDER fertility data into CCM-compatible race/age outputs.
sql/fertility/cdc_wonder_fertility_inflation.sql New query to compute inflation factors for “Not Stated” records.
python/input_modules/birth_rates.py Switched birth rate computation to SQL inputs and new geo hierarchy logic.
python/input_modules/death_rates.py Added module docstring.
python/parsers.py Removed rates_map parsing/validation and related attribute.
python/utils.py Removed RATES_MAP runtime config wiring.
README.md Removed rates_map configuration documentation.
config.yml Removed rates_map configuration entry.
rates_map.yml Deleted legacy local births mapping file.
Suppressed comments (3)

python/input_modules/birth_rates.py:59

  • Avoid using print() in library/runtime code; it bypasses the configured logging handlers (console/file) and makes output harder to control. Use the module logger instead.
                print("CDC WONDER fertility inflation factors loaded from database:")

python/input_modules/birth_rates.py:86

  • Avoid using print() in library/runtime code; it bypasses the configured logging handlers (console/file) and makes output harder to control. Use the module logger instead.
                print("DOF P3 projections loaded from database:")

sql/fertility/cdc_wonder_fertility.sql:75

  • The UNION expansion is documented as using the "All Races" + "Not Hispanic or Latino" rows to synthesize missing race categories, but the WHERE clause doesn’t restrict hispanic_origin. If "All Races" exists for Hispanic rows, this would incorrectly fabricate these categories for Hispanics as well.
        ,[births]
    FROM [data]
    WHERE [race] = 'All Races' AND [year] = @year AND [year] <= 2019

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread sql/fertility/cdc_wonder_fertility.sql
Comment thread sql/fertility/cdc_wonder_fertility_inflation.sql
Comment thread sql/fertility/dof_projections_p3.sql
Comment thread python/input_modules/birth_rates.py
Comment thread python/input_modules/birth_rates.py
Comment thread sql/fertility/cdc_wonder_fertility.sql
Comment thread sql/fertility/cdc_wonder_fertility.sql Outdated

@GregorSchroeder GregorSchroeder left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to revisit how we load births in for the Vital Statistics repository

  • fertility rates are provided down to the county level
  • product differentiation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Change source for fertility to SQL

3 participants