Skip to main content

NC Imputations DAG

Dag Flow​

Overview:

  1. create_network_check_chunks: Adds Network Check Imputation Tiers to Clear Rates
  2. create_state_level_rollup(): Creates State-Payer-Network, State-Payer, and State-Level Conditional Average Rates
  3. append_null_features_chunk: The output of step (1) is missing features that are required for Analyze. This step (1) appends those missing features and also (2) calculates them where necessary, such as percent of medicare values for the imputed rates.
  4. union_append_null_features_chunks(): unions outputs of (2) and (3)
with DAG(
    ...
) as dag:

    from data_science.cld.network_check.tasks import network_check

    # Get payer chunks for balanced processing
    payer_chunks_task = network_check.get_payer_chunks()

    # Process chunks + Union
    network_check_chunks = network_check.create_network_check_chunk.expand(
        chunk_data=payer_chunks_task
    )
    union_task = network_check.union_network_check_chunks(network_check_chunks)

    # Create state-level rollup for Physician Groups
    state_rollup_task = network_check.create_state_level_rollup()

    # Append null features in chunks + Union
    append_null_features_chunks = network_check.append_null_features_chunk.expand(
        chunk_data=payer_chunks_task
    )
    append_null_features_task = network_check.union_append_null_features_chunks(append_null_features_chunks)

    # Check coverage statistics
    coverage_task = network_check.check_coverage()

    # Promote to production
    promotion_task = network_check.promote_to_production(append_null_features_task)

    # Define DAG dependencies
    payer_chunks_task >> network_check_chunks >> union_task >> state_rollup_task
    payer_chunks_task >> append_null_features_chunks
    state_rollup_task >> append_null_features_chunks >> append_null_features_task >> coverage_task >> promotion_task

1 Create Network Check Chunks​

create_network_check_chunks()

The imputation_tiers_ordered configuration defines the hierarchy of imputation tiers appended. The imputation is used if the canonical_rate_score in Clear Rates is 1 or lower (outlier or NULL).

{%- set imputation_tiers_ordered = [
    {
        'name': 'npi_apc',
        'description': 'Payer, Network, Provider, Billing Code Type, APC',
        'partition': 'payer_id, network_id, provider_id, billing_code_type, apc',
        'formula': '{rate}'
    },
    {
        'name': 'network_state_healthsys_code',
        'description': 'Payer, Network, State, Health System, Provider Type, Bill Type, Billing Code Type, Billing Code',
        'partition': 'payer_id, network_id, state, health_system_id, provider_type, bill_type, billing_code_type, billing_code',
        'formula': '{rate}'
    },
    ...
]
%}

2 Create State Level Rollup​

create_state_level_rollup()

Three CTEs are used to create state-level, state-payer, and state-payer-network level rollups. For each rollup, the conditional average canonical_rate rate is calculated.

Questions
  1. Should outliers be excluded when calculating these rollups?
  2. Should we use medicare benchmarks to evaluate the rollups? e.g. it's unlikely but possible that a state-payer-network level average rate for some code is an outlier compared to medicare benchmarks;

These rollups are then joined to create the final output. The more granular conditional average is preferred over less granular averages.

3 Append Null Features Chunk​

There are fields that are not included in the steps above, which are required for Analyze. This step appends those missing features and also calculates them where necessary, such as percent of medicare values for the imputed rates.

On this page: