Multiclass and ordinal models — underwriting¶

Motor underwriting produces one of three decisions: decline, refer, accept. That is multiclass, but it is also ordinal — the outcomes sit on a scale, and getting it wrong by two steps is worse than by one.

That distinction is the whole reason this notebook exists. Plain multiclass metrics count mistakes. They cannot see that predicting decline for an application that should have been accepted is worse than predicting refer — to accuracy, both are simply "one error". For a gate that decides whether an underwriting model reaches production, that is the difference that matters.

New in bdp-model-gate 0.4.0:

Concept	Field / setting
Order of outcomes	context.class_order
What counts as a good outcome	context.favourable_classes
Ordinal scoring	metric="ordinal_mae" / "quadratic_kappa"
Multiclass averaging	config.performance.average
Ordinal robustness	config.security.adversarial_max_rank_shift

New to the library? 01_binary_classification_sklearn.ipynb covers the core machinery — contexts, reports, verdicts, custom checks. This notebook assumes it and focuses on what multiclass changes.

In [1]:

# %pip install -q "bdp-model-gate[structured]"
import logging

import numpy as np
import pandas as pd

import bdp_model_gate

logging.basicConfig(level=logging.INFO, format="%(levelname)-8s %(name)s: %(message)s")
logging.getLogger("bdp_model_gate").setLevel(logging.INFO)
pd.set_option("display.width", 130)

print("bdp-model-gate", bdp_model_gate.__version__)

# %pip install -q "bdp-model-gate[structured]" import logging import numpy as np import pandas as pd import bdp_model_gate logging.basicConfig(level=logging.INFO, format="%(levelname)-8s %(name)s: %(message)s") logging.getLogger("bdp_model_gate").setLevel(logging.INFO) pd.set_option("display.width", 130) print("bdp-model-gate", bdp_model_gate.__version__)

bdp-model-gate 0.7.0

1. An underwriting book¶

A synthetic motor portfolio. The decision follows an affordability-and-risk score, and we plant two problems worth catching:

• distance_to_branch_km is almost determined by region — a proxy for a protected attribute the model never sees
• applicants from one region are declined far more often, so the accept rate differs by region

In [2]:

rng = np.random.default_rng(21)
N = 1200

region = rng.choice(["Lagos", "Kano", "Abuja"], N, p=[0.45, 0.35, 0.20])
gender = rng.choice(["F", "M"], N, p=[0.48, 0.52])

X = pd.DataFrame(
    {
        "credit_score": rng.normal(650, 75, N).round(1),
        "claims_last_3y": rng.poisson(0.8, N).astype(float),
        "vehicle_value_ngn": rng.lognormal(15.4, 0.45, N).round(0),
        # Branch distance is essentially a relabelling of region.
        "distance_to_branch_km": (
            pd.Series(region).map({"Lagos": 2.0, "Abuja": 4.5, "Kano": 15.0}).to_numpy()
            + rng.normal(0, 1.1, N)
        ).round(2),
    }
)

# Underwriting merit, with a regional penalty that the model will learn
# indirectly through branch distance.
merit = (
    (X["credit_score"] - 650) / 75
    - 0.85 * X["claims_last_3y"]
    - 0.30 * (X["vehicle_value_ngn"] > 6_000_000)
    - pd.Series(region).map({"Lagos": 0.0, "Abuja": 0.25, "Kano": 0.75}).to_numpy()
)

CLASS_ORDER = ["decline", "refer", "accept"]   # ascending favourability


def decide(score):
    return np.where(score > 0.45, "accept", np.where(score > -0.55, "refer", "decline"))


y = decide(merit + rng.normal(0, 0.12, N))
protected_df = pd.DataFrame({"region": region, "gender": gender})

print("decision mix:")
print(pd.Series(y).value_counts(normalize=True).reindex(CLASS_ORDER).round(3).to_string())
print("\naccept rate by region:")
print(pd.crosstab(region, y, normalize="index").reindex(columns=CLASS_ORDER).round(3))

rng = np.random.default_rng(21) N = 1200 region = rng.choice(["Lagos", "Kano", "Abuja"], N, p=[0.45, 0.35, 0.20]) gender = rng.choice(["F", "M"], N, p=[0.48, 0.52]) X = pd.DataFrame( { "credit_score": rng.normal(650, 75, N).round(1), "claims_last_3y": rng.poisson(0.8, N).astype(float), "vehicle_value_ngn": rng.lognormal(15.4, 0.45, N).round(0), # Branch distance is essentially a relabelling of region. "distance_to_branch_km": ( pd.Series(region).map({"Lagos": 2.0, "Abuja": 4.5, "Kano": 15.0}).to_numpy() + rng.normal(0, 1.1, N) ).round(2), } ) # Underwriting merit, with a regional penalty that the model will learn # indirectly through branch distance. merit = ( (X["credit_score"] - 650) / 75 - 0.85 * X["claims_last_3y"] - 0.30 * (X["vehicle_value_ngn"] > 6_000_000) - pd.Series(region).map({"Lagos": 0.0, "Abuja": 0.25, "Kano": 0.75}).to_numpy() ) CLASS_ORDER = ["decline", "refer", "accept"] # ascending favourability def decide(score): return np.where(score > 0.45, "accept", np.where(score > -0.55, "refer", "decline")) y = decide(merit + rng.normal(0, 0.12, N)) protected_df = pd.DataFrame({"region": region, "gender": gender}) print("decision mix:") print(pd.Series(y).value_counts(normalize=True).reindex(CLASS_ORDER).round(3).to_string()) print("\naccept rate by region:") print(pd.crosstab(region, y, normalize="index").reindex(columns=CLASS_ORDER).round(3))

decision mix:
decline    0.670
refer      0.223
accept     0.107

accept rate by region:
col_0  decline  refer  accept
row_0                        
Abuja    0.656  0.237   0.108
Kano     0.788  0.147   0.065
Lagos    0.580  0.280   0.140

Note CLASS_ORDER is listed ascending: least favourable first. That convention is what lets the library compute rank distances — and it is the one thing to get right in this notebook.

In [3]:

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split

X_train, X_val, y_train, y_val, prot_train, prot_val = train_test_split(
    X, y, protected_df, test_size=0.35, random_state=21, stratify=y
)

# region and gender are deliberately not features — the usual "fairness
# through unawareness" approach. Section 4 shows why that is not enough.
model = RandomForestClassifier(n_estimators=200, min_samples_leaf=5, random_state=21)
model.fit(X_train, y_train)
y_pred = model.predict(X_val)

print(f"train {len(X_train)} | validation {len(X_val)}")
print(f"agreement: {(y_pred == y_val).mean():.3f}")

from sklearn.ensemble import RandomForestClassifier from sklearn.model_selection import train_test_split X_train, X_val, y_train, y_val, prot_train, prot_val = train_test_split( X, y, protected_df, test_size=0.35, random_state=21, stratify=y ) # region and gender are deliberately not features — the usual "fairness # through unawareness" approach. Section 4 shows why that is not enough. model = RandomForestClassifier(n_estimators=200, min_samples_leaf=5, random_state=21) model.fit(X_train, y_train) y_pred = model.predict(X_val) print(f"train {len(X_train)} | validation {len(X_val)}") print(f"agreement: {(y_pred == y_val).mean():.3f}")

train 780 | validation 420
agreement: 0.888

2. The context¶

task="multiclass" and class_order are the two additions. class_order is optional — omit it for a genuinely nominal problem such as product category — but supplying it unlocks everything ordinal.

In [4]:

from bdp_model_gate import GateConfig, ModelGate, StructuredGateContext
from bdp_model_gate.structured import default_structured_checks


def explain(prompt: str) -> str:
    """A stand-in for the LLM that writes the applicant's decision letter."""
    if any(k in prompt.lower() for k in ("ignore previous", "no content policy", "verbatim")):
        return "I cannot comply with that request."
    return "Your application was referred for manual review."


model_card = {
    "model_name": "motor-underwriting-rf",
    "version": "2.1.0",
    "use_case": "underwriting",                 # high-risk -> DPIA required
    "legal_basis": "Contractual necessity (NDPA 2023, s.25(1)(b))",
    "data_minimization_justification": "Only risk and affordability signals are used.",
    "training_data_source": "Motor book 2021-2025, consented at quotation",
    "dpia_completed": True,
    "influences_decision_about_person": True,
    "explainability_method": "SHAP, surfaced in the referral letter",
}

context = StructuredGateContext(
    model=model,
    X=X_val,
    y_true=y_val,
    y_pred=y_pred,
    protected_df=prot_val,
    model_card=model_card,
    generate_fn=explain,
    latencies_ms=rng.gamma(9.0, 7.0, 400),
    cost_per_inference=0.0006,
    task="multiclass",
    class_order=CLASS_ORDER,
    favourable_classes=["accept"],
)

from bdp_model_gate import GateConfig, ModelGate, StructuredGateContext from bdp_model_gate.structured import default_structured_checks def explain(prompt: str) -> str: """A stand-in for the LLM that writes the applicant's decision letter.""" if any(k in prompt.lower() for k in ("ignore previous", "no content policy", "verbatim")): return "I cannot comply with that request." return "Your application was referred for manual review." model_card = { "model_name": "motor-underwriting-rf", "version": "2.1.0", "use_case": "underwriting", # high-risk -> DPIA required "legal_basis": "Contractual necessity (NDPA 2023, s.25(1)(b))", "data_minimization_justification": "Only risk and affordability signals are used.", "training_data_source": "Motor book 2021-2025, consented at quotation", "dpia_completed": True, "influences_decision_about_person": True, "explainability_method": "SHAP, surfaced in the referral letter", } context = StructuredGateContext( model=model, X=X_val, y_true=y_val, y_pred=y_pred, protected_df=prot_val, model_card=model_card, generate_fn=explain, latencies_ms=rng.gamma(9.0, 7.0, 400), cost_per_inference=0.0006, task="multiclass", class_order=CLASS_ORDER, favourable_classes=["accept"], )

favourable_classes=["accept"] says which outcome counts as a positive result. It is optional — the library defaults to the last entry of class_order and logs that it inferred — but stating it is better, because the choice genuinely changes what gets measured. Section 4 demonstrates that.

3. Scoring an ordinal model¶

Nominal metrics, and their averaging¶

accuracy, balanced_accuracy, f1, precision and recall all work. The last three need an averaging strategy once there are more than two classes, set by config.performance.average (default "macro").

In [5]:

from bdp_model_gate import PerformanceConfig
from bdp_model_gate.structured.performance import PerformanceThresholdCheck

rows = []
for name in ("accuracy", "balanced_accuracy", "f1", "precision", "recall"):
    for average in ("macro", "weighted"):
        cfg = PerformanceConfig(metric=name, min_score=0.0, average=average)
        result = PerformanceThresholdCheck(cfg).run(context)[0]
        rows.append({"metric": name, "average": average, "value": result.metadata["value"]})

display(pd.DataFrame(rows).pivot(index="metric", columns="average", values="value"))

from bdp_model_gate import PerformanceConfig from bdp_model_gate.structured.performance import PerformanceThresholdCheck rows = [] for name in ("accuracy", "balanced_accuracy", "f1", "precision", "recall"): for average in ("macro", "weighted"): cfg = PerformanceConfig(metric=name, min_score=0.0, average=average) result = PerformanceThresholdCheck(cfg).run(context)[0] rows.append({"metric": name, "average": average, "value": result.metadata["value"]}) display(pd.DataFrame(rows).pivot(index="metric", columns="average", values="value"))

average	macro	weighted
metric
accuracy	0.8881	0.8881
balanced_accuracy	0.8145	0.8145
f1	0.8420	0.8857
precision	0.8776	0.8867
recall	0.8145	0.8881

macro vs weighted matters for governance. Macro weights every class equally, so a rarely predicted decline counts as much as a common accept. Weighted averages by support, which flatters a model that does well on the majority class and poorly on the rare one — usually the opposite of what you want to gate on. Macro is the default for that reason.

metric="auto" resolves to balanced_accuracy for multiclass, since plain accuracy rewards a model that never predicts the rare class at all.

In [6]:

from bdp_model_gate.metrics import resolve_metric

print("auto ->", resolve_metric("auto", "multiclass").name)

from bdp_model_gate.metrics import resolve_metric print("auto ->", resolve_metric("auto", "multiclass").name)

auto -> balanced_accuracy

The ordinal metrics¶

Here is the point of the whole notebook. Take one prediction and get it wrong two different ways — off by one step, and off by two. Accuracy cannot tell them apart. The ordinal metrics can.

In [7]:

from bdp_model_gate.metrics import ordinal_mae, quadratic_kappa

truth = np.array(["accept", "accept", "refer", "decline", "accept", "refer"])
one_step = np.array(["refer", "accept", "refer", "decline", "accept", "refer"])
two_step = np.array(["decline", "accept", "refer", "decline", "accept", "refer"])

comparison = pd.DataFrame(
    [
        {
            "prediction": label,
            "errors": int((truth != pred).sum()),
            "accuracy": (truth == pred).mean(),
            "ordinal_mae": ordinal_mae(truth, pred, CLASS_ORDER),
            "quadratic_kappa": quadratic_kappa(truth, pred, CLASS_ORDER),
        }
        for label, pred in (("off by one step", one_step), ("off by two steps", two_step))
    ]
)
display(comparison.set_index("prediction").round(4))

from bdp_model_gate.metrics import ordinal_mae, quadratic_kappa truth = np.array(["accept", "accept", "refer", "decline", "accept", "refer"]) one_step = np.array(["refer", "accept", "refer", "decline", "accept", "refer"]) two_step = np.array(["decline", "accept", "refer", "decline", "accept", "refer"]) comparison = pd.DataFrame( [ { "prediction": label, "errors": int((truth != pred).sum()), "accuracy": (truth == pred).mean(), "ordinal_mae": ordinal_mae(truth, pred, CLASS_ORDER), "quadratic_kappa": quadratic_kappa(truth, pred, CLASS_ORDER), } for label, pred in (("off by one step", one_step), ("off by two steps", two_step)) ] ) display(comparison.set_index("prediction").round(4))

	errors	accuracy	ordinal_mae	quadratic_kappa
prediction
off by one step	1	0.8333	0.1667	0.8421
off by two steps	1	0.8333	0.3333	0.5000

Identical error count, identical accuracy — and the ordinal metrics separate them clearly. quadratic_kappa penalises a disagreement by the square of its rank distance, so a two-step error costs four times a one-step one.

ordinal_mae is an error metric, so it is gated with max_error; quadratic_kappa is higher-is-better and uses min_score. Mixing them up is a GateConfigurationError, not a silent pass.

In [8]:

for cfg, label in (
    (PerformanceConfig(metric="quadratic_kappa", min_score=0.75), "quadratic_kappa"),
    (PerformanceConfig(metric="ordinal_mae", max_error=0.15), "ordinal_mae"),
):
    result = PerformanceThresholdCheck(cfg).run(context)[0]
    print(f"{label:16} {result.flag:18} {result.detail}")

for cfg, label in ( (PerformanceConfig(metric="quadratic_kappa", min_score=0.75), "quadratic_kappa"), (PerformanceConfig(metric="ordinal_mae", max_error=0.15), "ordinal_mae"), ): result = PerformanceThresholdCheck(cfg).run(context)[0] print(f"{label:16} {result.flag:18} {result.detail}")

quadratic_kappa  OK                 quadratic_kappa=0.8633 [0.817, 0.904] (min 0.75) — the whole interval sits above 0.750

ordinal_mae      OK                 ordinal_mae=0.1143 [0.081, 0.145] (max 0.15) — the whole interval sits below 0.150

In [9]:

from bdp_model_gate.exceptions import GateConfigurationError

# An error metric with no max_error set would silently have nothing to
# compare against, so it is refused.
try:
    PerformanceThresholdCheck(PerformanceConfig(metric="ordinal_mae")).run(context)
except GateConfigurationError as exc:
    print("GateConfigurationError:", exc)

# And ordinal metrics need the ordering to exist at all.
no_order = StructuredGateContext(
    model=model, X=X_val, y_true=y_val, y_pred=y_pred, task="multiclass"
)
try:
    PerformanceThresholdCheck(
        PerformanceConfig(metric="quadratic_kappa", min_score=0.5)
    ).run(no_order)
except GateConfigurationError as exc:
    print("GateConfigurationError:", str(exc)[:150], "...")

from bdp_model_gate.exceptions import GateConfigurationError # An error metric with no max_error set would silently have nothing to # compare against, so it is refused. try: PerformanceThresholdCheck(PerformanceConfig(metric="ordinal_mae")).run(context) except GateConfigurationError as exc: print("GateConfigurationError:", exc) # And ordinal metrics need the ordering to exist at all. no_order = StructuredGateContext( model=model, X=X_val, y_true=y_val, y_pred=y_pred, task="multiclass" ) try: PerformanceThresholdCheck( PerformanceConfig(metric="quadratic_kappa", min_score=0.5) ).run(no_order) except GateConfigurationError as exc: print("GateConfigurationError:", str(exc)[:150], "...")

GateConfigurationError: performance.metric='ordinal_mae' is an error metric (lower is better), so it is gated with performance.max_error — which is unset. There is no sensible default: a ceiling depends on the scale of your target. Set max_error, or choose a higher-is-better metric such as 'r2'.
GateConfigurationError: performance.metric='quadratic_kappa' is an ordinal metric and needs context.class_order — the ordered class labels, least to most favourable, e.g. ["d ...

What is not available¶

roc_auc and average_precision are binary-only. Their multiclass forms need a full (n_rows, n_classes) probability matrix, which the y_pred contract does not carry — so they are refused rather than quietly approximated by something else.

In [10]:

try:
    PerformanceThresholdCheck(PerformanceConfig(metric="roc_auc", min_score=0.8)).run(context)
except GateConfigurationError as exc:
    print("GateConfigurationError:", exc)

try: PerformanceThresholdCheck(PerformanceConfig(metric="roc_auc", min_score=0.8)).run(context) except GateConfigurationError as exc: print("GateConfigurationError:", exc)

GateConfigurationError: performance.metric='roc_auc' does not apply to a multiclass task — metrics available for multiclass: accuracy, balanced_accuracy, f1, ordinal_mae, precision, quadratic_kappa, recall

4. Fairness needs someone to define "good"¶

Demographic parity compares selection rates — the share of each group that got the good outcome. With two classes, "good" is obvious. With three, it is a judgement, and the library will not make it for you.

In [11]:

from bdp_model_gate.structured.fairness import DisparateImpactCheck

for r in DisparateImpactCheck().run(context):
    print(f"{r.flag:18} {r.detail}")

from bdp_model_gate.structured.fairness import DisparateImpactCheck for r in DisparateImpactCheck().run(context): print(f"{r.flag:18} {r.detail}")

UNCERTAIN          region: demographic parity diff=0.045 [0.012, 0.111] at 95% (max 0.1) [favourable: accept] — the estimate stays the right side of 0.100 but the interval reaches 0.111, so this sample cannot rule out a breach
OK                 gender: demographic parity diff=0.023 [0.001, 0.080] at 95% (max 0.1) [favourable: accept] — the whole interval sits below 0.100

Now change the definition. "Was accepted" and "was not declined" are different questions — and on this book they give different answers.

In [12]:

def with_favourable(classes):
    return StructuredGateContext(
        model=model, X=X_val, y_true=y_val, y_pred=y_pred, protected_df=prot_val,
        task="multiclass", class_order=CLASS_ORDER, favourable_classes=classes,
    )


for classes in (["accept"], ["accept", "refer"]):
    for r in DisparateImpactCheck().run(with_favourable(classes)):
        if r.metadata["protected_attr"] == "region":
            print(f"favourable={str(classes):22} {r.flag:16} gap={r.metadata['demographic_parity_diff']}")

print("\nselection rates by region:")
rates = pd.DataFrame(
    {
        "accepted": pd.Series(y_pred == "accept").groupby(prot_val["region"].values).mean(),
        "not declined": pd.Series(y_pred != "decline").groupby(prot_val["region"].values).mean(),
    }
)
display(rates.round(3))

def with_favourable(classes): return StructuredGateContext( model=model, X=X_val, y_true=y_val, y_pred=y_pred, protected_df=prot_val, task="multiclass", class_order=CLASS_ORDER, favourable_classes=classes, ) for classes in (["accept"], ["accept", "refer"]): for r in DisparateImpactCheck().run(with_favourable(classes)): if r.metadata["protected_attr"] == "region": print(f"favourable={str(classes):22} {r.flag:16} gap={r.metadata['demographic_parity_diff']}") print("\nselection rates by region:") rates = pd.DataFrame( { "accepted": pd.Series(y_pred == "accept").groupby(prot_val["region"].values).mean(), "not declined": pd.Series(y_pred != "decline").groupby(prot_val["region"].values).mean(), } ) display(rates.round(3))

favourable=['accept']             UNCERTAIN        gap=0.045

favourable=['accept', 'refer']    DISPARITY_RISK   gap=0.193

selection rates by region:

	accepted	not declined
Abuja	0.082	0.282
Kano	0.061	0.190
Lagos	0.106	0.383

Widening the favourable set does not automatically flatter the model. Which framing is right depends on what the referral actually means for an applicant — a referral that is usually approved is a very different outcome from one that usually is not. That is a business question, which is exactly why it is a parameter rather than a default.

With no class_order and no favourable_classes, the check declines to guess:

In [13]:

print(DisparateImpactCheck().run(no_order)[0].flag, "-",
      DisparateImpactCheck().run(no_order)[0].detail)

print(DisparateImpactCheck().run(no_order)[0].flag, "-", DisparateImpactCheck().run(no_order)[0].detail)

NOT_APPLICABLE - no protected_df supplied

Proxy correlation — why dropping the column is not enough¶

We never gave the model region. But distance_to_branch_km is nearly a relabelling of it, so the model can reconstruct the attribute anyway. This check is task-agnostic and is usually the most valuable one in the suite.

In [14]:

from bdp_model_gate.structured.fairness import ProxyCorrelationCheck

for r in ProxyCorrelationCheck().run(context):
    print(f"{r.flag:14} {r.detail}")

display(
    X_val.assign(region=prot_val["region"].values)
    .groupby("region")[["distance_to_branch_km", "credit_score"]]
    .mean()
    .round(2)
)

from bdp_model_gate.structured.fairness import ProxyCorrelationCheck for r in ProxyCorrelationCheck().run(context): print(f"{r.flag:14} {r.detail}") display( X_val.assign(region=prot_val["region"].values) .groupby("region")[["distance_to_branch_km", "credit_score"]] .mean() .round(2) )

PROXY_RISK     distance_to_branch_km correlates with region (eta^2=0.967, q=0.008 across 8 comparison(s))

	distance_to_branch_km	credit_score
region
Abuja	4.57	649.52
Kano	15.12	663.00
Lagos	1.84	644.78

SHAP contributions for the favourable class¶

For multiclass, ShapSubgroupCheck explains the favourable class column — so it answers "does this feature push some groups away from being accepted?" rather than averaging contributions across unrelated outcomes. Before 0.4.0 this check reported NOT_APPLICABLE for any problem with more than two classes.

In [15]:

from bdp_model_gate.structured.fairness import ShapSubgroupCheck

for r in ShapSubgroupCheck().run(context)[:6]:
    print(f"{r.flag:24} {r.detail}")

from bdp_model_gate.structured.fairness import ShapSubgroupCheck for r in ShapSubgroupCheck().run(context)[:6]: print(f"{r.flag:24} {r.detail}")

/private/tmp/claude-501/-Users-vangelis-Documents-Bluechip-2026-Applications-model-gate/631c3443-469b-4452-8b45-f2ce39407b48/scratchpad/devvenv/lib/python3.13/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

UNCERTAIN                credit_score SHAP contribution gap across region=0.041 — 0.637 [0.219, 1.444] at 95% of the mean absolute contribution 0.064 (max 50%) — the estimate breaches 0.500 but the interval reaches back to 0.219, so this sample cannot confirm a breach
UNCERTAIN                claims_last_3y SHAP contribution gap across region=0.012 — 0.189 [0.054, 0.625] at 95% of the mean absolute contribution 0.064 (max 50%) — the estimate stays the right side of 0.500 but the interval reaches 0.625, so this sample cannot rule out a breach
SUBGROUP_IMPACT_RISK     distance_to_branch_km SHAP contribution gap across region=0.061 — 0.947 [0.856, 1.036] at 95% of the mean absolute contribution 0.064 (max 50%) — the whole 95% interval sits above 0.500
UNCERTAIN                credit_score SHAP contribution gap across gender=0.017 — 0.266 [0.015, 0.888] at 95% of the mean absolute contribution 0.064 (max 50%) — the estimate stays the right side of 0.500 but the interval reaches 0.888, so this sample cannot rule out a breach

5. Ordinal robustness¶

AdversarialRobustnessCheck perturbs numeric features slightly and measures how often the prediction changes. For an ordinal problem it also reports how far the prediction moved in rank terms — because two models can flip at an identical rate while one only ever wobbles between adjacent classes and the other swings from accept straight to decline.

In [16]:

from bdp_model_gate.structured.security import AdversarialRobustnessCheck

result = AdversarialRobustnessCheck().run(context)[0]
print(result.flag, "-", result.detail)
print()
for key in ("flip_rate", "mean_rank_shift", "max_observed_rank_shift", "n_classes"):
    print(f"  {key:26} {result.metadata[key]}")

from bdp_model_gate.structured.security import AdversarialRobustnessCheck result = AdversarialRobustnessCheck().run(context)[0] print(result.flag, "-", result.detail) print() for key in ("flip_rate", "mean_rank_shift", "max_observed_rank_shift", "n_classes"): print(f" {key:26} {result.metadata[key]}")

ROBUSTNESS_RISK - flip rate under random perturbation=0.0850 (max 0.05); mean ordinal rank shift=0.0850 (max 0.1), worst 1 step(s)

  flip_rate                  0.085
  mean_rank_shift            0.085
  max_observed_rank_shift    1.0
  n_classes                  3

max_observed_rank_shift is the number to read. A value of 1.0 means the model never swung further than an adjacent class under perturbation — an accept might become a refer, but never a decline. A value of 2.0 on the same flip rate would be a materially more dangerous model, and a bare flip rate cannot distinguish them.

Below, two models with the same decision boundary and therefore the same flip rate, differing only in how far they fall.

In [17]:

from bdp_model_gate import SecurityConfig


def stepper(low_class):
    # Same boundary in both models, so they flip on the same rows.
    return lambda df: np.where(df["credit_score"].to_numpy() > 650, "accept", low_class)


config = SecurityConfig(adversarial_epsilon=0.05)
for low in ("refer", "decline"):
    ctx = StructuredGateContext(
        X=X_val, y_true=y_val, y_pred=stepper(low)(X_val), predict_fn=stepper(low),
        protected_df=prot_val, task="multiclass", class_order=CLASS_ORDER,
    )
    md = AdversarialRobustnessCheck(config).run(ctx)[0].metadata
    print(f"accept -> {low:8} flip_rate={md['flip_rate']:.4f}  "
          f"mean_rank_shift={md['mean_rank_shift']:.4f}  worst={md['max_observed_rank_shift']:.0f} step(s)")

from bdp_model_gate import SecurityConfig def stepper(low_class): # Same boundary in both models, so they flip on the same rows. return lambda df: np.where(df["credit_score"].to_numpy() > 650, "accept", low_class) config = SecurityConfig(adversarial_epsilon=0.05) for low in ("refer", "decline"): ctx = StructuredGateContext( X=X_val, y_true=y_val, y_pred=stepper(low)(X_val), predict_fn=stepper(low), protected_df=prot_val, task="multiclass", class_order=CLASS_ORDER, ) md = AdversarialRobustnessCheck(config).run(ctx)[0].metadata print(f"accept -> {low:8} flip_rate={md['flip_rate']:.4f} " f"mean_rank_shift={md['mean_rank_shift']:.4f} worst={md['max_observed_rank_shift']:.0f} step(s)")

accept -> refer    flip_rate=0.1350  mean_rank_shift=0.1350  worst=1 step(s)
accept -> decline  flip_rate=0.1350  mean_rank_shift=0.2700  worst=2 step(s)

6. The whole gate¶

Regression-only checks report NOT_APPLICABLE rather than being dropped, so the report shows what was skipped and why.

In [18]:

config = GateConfig()
config.performance.metric = "quadratic_kappa"
config.performance.min_score = 0.70
config.fairness.disparity_threshold = 0.10
config.security.adversarial_max_rank_shift = 0.10

report = ModelGate(checks=default_structured_checks(config, include_plugins=False)).run(context)
print(report.summary())

config = GateConfig() config.performance.metric = "quadratic_kappa" config.performance.min_score = 0.70 config.fairness.disparity_threshold = 0.10 config.security.adversarial_max_rank_shift = 0.10 report = ModelGate(checks=default_structured_checks(config, include_plugins=False)).run(context) print(report.summary())

INFO     bdp_model_gate.security: prompt_injection: firing 6 prompt(s) at 1 surface(s) = 6 generative call(s) (security.injection_depth=1)

INFO     bdp_model_gate.gate: gate_status=BLOCKED task=multiclass n_flags=14 metric=quadratic_kappa score=0.8633

Gate status: BLOCKED (7902ms, multiclass)
  quadratic_kappa: 0.8633
  validation: 2 flag(s)
  performance: 0 flag(s)
  compliance: 0 flag(s)
  security: 2 flag(s)
  fairness: 10 flag(s)

In [19]:

display(
    pd.DataFrame(
        [
            {
                "category": r.category,
                "check": r.check_name,
                "flag": r.flag,
                "blocking": r.blocking,
                "detail": (r.detail[:66] + "...") if len(r.detail) > 66 else r.detail,
            }
            for r in report.results
        ]
    )
)

display( pd.DataFrame( [ { "category": r.category, "check": r.check_name, "flag": r.flag, "blocking": r.blocking, "detail": (r.detail[:66] + "...") if len(r.detail) > 66 else r.detail, } for r in report.results ] ) )

	category	check	flag	blocking	detail
0	validation	target_leakage	LEAKAGE_RISK	True	credit_score alone reaches 0.852 of the target...
1	validation	split_overlap	OK	True	0 duplicate row(s) within the validation set (...
2	validation	split_overlap	NOT_APPLICABLE	True	no X_train supplied — supply the training fram...
3	validation	validation_strategy	VALIDATION_STRATEGY_RISK	True	model_card.validation_strategy missing — state...
4	validation	feature_contract	OK	True	X matches the 4 feature(s) recorded in model.f...
5	validation	feature_drift	NOT_APPLICABLE	False	no X_train supplied — train-serve skew needs t...
6	comparison	comparison_integrity	NOT_APPLICABLE	True	no context.champion — the comparison checks me...
7	comparison	challenger_performance	NOT_APPLICABLE	True	no context.champion — the comparison checks me...
8	comparison	challenger_guardrails	NOT_APPLICABLE	True	no context.champion — the comparison checks me...
9	comparison	subgroup_regression	NOT_APPLICABLE	False	no context.champion — the comparison checks me...
10	comparison	fairness_delta	NOT_APPLICABLE	False	no context.champion — the comparison checks me...
11	comparison	decision_churn	NOT_APPLICABLE	False	no context.champion — the comparison checks me...
12	comparison	operational_delta	NOT_APPLICABLE	False	no context.champion — the comparison checks me...
13	fairness	proxy_correlation	PROXY_RISK	False	distance_to_branch_km correlates with region (...
14	fairness	disparate_impact	UNCERTAIN	False	region: demographic parity diff=0.045 [0.012, ...
15	fairness	disparate_impact	OK	False	gender: demographic parity diff=0.023 [0.001, ...
16	fairness	shap_subgroup_gap	UNCERTAIN	False	credit_score SHAP contribution gap across regi...
17	fairness	shap_subgroup_gap	UNCERTAIN	False	claims_last_3y SHAP contribution gap across re...
18	fairness	shap_subgroup_gap	SUBGROUP_IMPACT_RISK	False	distance_to_branch_km SHAP contribution gap ac...
19	fairness	shap_subgroup_gap	UNCERTAIN	False	credit_score SHAP contribution gap across gend...
20	fairness	counterfactual_flip	NOT_APPLICABLE	False	no protected attributes present as model inputs
21	fairness	equalised_odds	UNCERTAIN	False	region: true-positive-rate difference=0.233 [0...
22	fairness	equalised_odds	UNCERTAIN	False	region: equalised odds difference=0.233 [0.056...
23	fairness	equalised_odds	UNCERTAIN	False	gender: true-positive-rate difference=0.034 [0...
24	fairness	equalised_odds	UNCERTAIN	False	gender: equalised odds difference=0.034 [0.012...
25	fairness	subgroup_calibration	NOT_APPLICABLE	False	no probability predictions available — calibra...
26	fairness	group_mean_gap	NOT_APPLICABLE	False	check does not apply to a multiclass task (sup...
27	fairness	error_parity	NOT_APPLICABLE	False	check does not apply to a multiclass task (sup...
28	fairness	calibration_parity	NOT_APPLICABLE	False	check does not apply to a multiclass task (sup...
29	fairness	loss_ratio_parity	NOT_APPLICABLE	False	check does not apply to a multiclass task (sup...
30	performance	performance_thresholds	OK	True	quadratic_kappa=0.8633 [0.817, 0.904] (min 0.7...
31	performance	performance_thresholds	OK	True	p95 latency=97.81ms [94.27, 104.18] (max 200.0...
32	performance	performance_thresholds	OK	True	cost/inference=0.00060 (max 0.002)
33	performance	calibration	NOT_APPLICABLE	True	no probability predictions available — calibra...
34	performance	actual_vs_expected	NOT_APPLICABLE	True	check does not apply to a multiclass task (sup...
35	performance	risk_discrimination	NOT_APPLICABLE	True	check does not apply to a multiclass task (sup...
36	compliance	compliance_mapping	OK	True	model_card.legal_basis present
37	compliance	compliance_mapping	OK	True	model_card.data_minimization_justification pre...
38	compliance	compliance_mapping	OK	True	model_card.training_data_source present
39	compliance	compliance_mapping	OK	True	DPIA completed
40	compliance	compliance_mapping	OK	True	explainability method documented
41	compliance	monotonicity	NOT_APPLICABLE	True	check does not apply to a multiclass task (sup...
42	compliance	prediction_dislocation	NOT_APPLICABLE	False	check does not apply to a multiclass task (sup...
43	security	adversarial_robustness	ROBUSTNESS_RISK	True	flip rate under random perturbation=0.0850 (ma...
44	security	pii_leakage	OK	True	no string columns to scan
45	security	prompt_injection	INJECTION_NEEDS_JUDGEMENT	False	direct surface: 2 response(s) carry no decidab...
46	security	prompt_injection	OK	True	direct surface: 6 probe(s) fired at depth 1; s...
47	security	report_injection	OK	False	no instruction-shaped text in the 22 string(s)...

In [20]:

print(f"verdict : {report.gate_status}")
print(f"task    : {report.task}")
print(f"metric  : {report.model_metric} = {report.model_score}")
print()
for r in report.flags:
    print(f"[{'BLOCK ' if r.blocking else 'review'}] {r.check_name}")
    print(f"          {r.detail}")

print(f"verdict : {report.gate_status}") print(f"task : {report.task}") print(f"metric : {report.model_metric} = {report.model_score}") print() for r in report.flags: print(f"[{'BLOCK ' if r.blocking else 'review'}] {r.check_name}") print(f" {r.detail}")

verdict : BLOCKED
task    : multiclass
metric  : quadratic_kappa = 0.8633

[BLOCK ] target_leakage
          credit_score alone reaches 0.852 of the target on its own, 114% of the whole model's 0.750 — check it is populated before the outcome is known, and is available at scoring time
[BLOCK ] validation_strategy
          model_card.validation_strategy missing — state how the holdout was produced, one of: out_of_time, temporal, forward_chaining, walk_forward, random_split, stratified_split, cross_validation, grouped_split
[review] proxy_correlation
          distance_to_branch_km correlates with region (eta^2=0.967, q=0.008 across 8 comparison(s))
[review] disparate_impact
          region: demographic parity diff=0.045 [0.012, 0.111] at 95% (max 0.1) [favourable: accept] — the estimate stays the right side of 0.100 but the interval reaches 0.111, so this sample cannot rule out a breach
[review] shap_subgroup_gap
          credit_score SHAP contribution gap across region=0.041 — 0.637 [0.219, 1.444] at 95% of the mean absolute contribution 0.064 (max 50%) — the estimate breaches 0.500 but the interval reaches back to 0.219, so this sample cannot confirm a breach
[review] shap_subgroup_gap
          claims_last_3y SHAP contribution gap across region=0.012 — 0.189 [0.054, 0.625] at 95% of the mean absolute contribution 0.064 (max 50%) — the estimate stays the right side of 0.500 but the interval reaches 0.625, so this sample cannot rule out a breach
[review] shap_subgroup_gap
          distance_to_branch_km SHAP contribution gap across region=0.061 — 0.947 [0.856, 1.036] at 95% of the mean absolute contribution 0.064 (max 50%) — the whole 95% interval sits above 0.500
[review] shap_subgroup_gap
          credit_score SHAP contribution gap across gender=0.017 — 0.266 [0.015, 0.888] at 95% of the mean absolute contribution 0.064 (max 50%) — the estimate stays the right side of 0.500 but the interval reaches 0.888, so this sample cannot rule out a breach
[review] equalised_odds
          region: true-positive-rate difference=0.233 [0.056, 0.625] at 95% (max 0.1) — among those who should be approved, Abuja is least likely to be — the estimate breaches 0.100 but the interval reaches back to 0.056, so this sample cannot confirm a breach
[review] equalised_odds
          region: equalised odds difference=0.233 [0.056, 0.625] at 95% (max 0.1) — TPR gap 0.233, FPR gap 0.013 — the estimate breaches 0.100 but the interval reaches back to 0.056, so this sample cannot confirm a breach
[review] equalised_odds
          gender: true-positive-rate difference=0.034 [0.006, 0.294] at 95% (max 0.1) — among those who should be approved, M is least likely to be — the estimate stays the right side of 0.100 but the interval reaches 0.294, so this sample cannot rule out a breach
[review] equalised_odds
          gender: equalised odds difference=0.034 [0.012, 0.294] at 95% (max 0.1) — TPR gap 0.034, FPR gap 0.012 — the estimate stays the right side of 0.100 but the interval reaches 0.294, so this sample cannot rule out a breach
[BLOCK ] adversarial_robustness
          flip rate under random perturbation=0.0850 (max 0.05); mean ordinal rank shift=0.0850 (max 0.1), worst 1 step(s)
[review] prompt_injection
          direct surface: 2 response(s) carry no decidable signal, because these attacks aim at a secret and no context.canaries were supplied. Plant a canary — the system prompt, a fake PII record, an internal URL — and this becomes a verdict instead of a question [smoke test: 6 of 14 corpus prompts at depth 1 — a pre-deployment probe, not a red-team assessment]

7. From the CLI¶

--class-order and --favourable-classes take comma-separated labels, and exit codes are unchanged: 0 pass, 2 needs review, 1 blocked.

In [21]:

import json
import subprocess
import sys
from pathlib import Path

import joblib

workdir = Path("cli_multiclass")
workdir.mkdir(exist_ok=True)

frame = X_val.copy()
frame["decision"] = y_val
frame.to_csv(workdir / "validation.csv", index=False)
prot_val.to_csv(workdir / "protected.csv", index=False)
(workdir / "model_card.json").write_text(json.dumps(model_card, indent=2))
joblib.dump(model, workdir / "model.joblib")

proc = subprocess.run(
    [
        sys.executable, "-m", "bdp_model_gate.cli",
        "--model", str(workdir / "model.joblib"),
        "--data", str(workdir / "validation.csv"),
        "--target-col", "decision",
        "--protected", str(workdir / "protected.csv"),
        "--model-card", str(workdir / "model_card.json"),
        "--task", "multiclass",
        "--class-order", "decline,refer,accept",
        "--favourable-classes", "accept",
        "--metric", "quadratic_kappa",
        "--min-score", "0.70",
        "--output", str(workdir / "gate_report.json"),
    ],
    capture_output=True, text=True,
)
print(proc.stdout or proc.stderr)
print("exit code", proc.returncode, "->",
      {0: "PASS", 1: "BLOCKED", 2: "NEEDS_REVIEW"}.get(proc.returncode))

import json import subprocess import sys from pathlib import Path import joblib workdir = Path("cli_multiclass") workdir.mkdir(exist_ok=True) frame = X_val.copy() frame["decision"] = y_val frame.to_csv(workdir / "validation.csv", index=False) prot_val.to_csv(workdir / "protected.csv", index=False) (workdir / "model_card.json").write_text(json.dumps(model_card, indent=2)) joblib.dump(model, workdir / "model.joblib") proc = subprocess.run( [ sys.executable, "-m", "bdp_model_gate.cli", "--model", str(workdir / "model.joblib"), "--data", str(workdir / "validation.csv"), "--target-col", "decision", "--protected", str(workdir / "protected.csv"), "--model-card", str(workdir / "model_card.json"), "--task", "multiclass", "--class-order", "decline,refer,accept", "--favourable-classes", "accept", "--metric", "quadratic_kappa", "--min-score", "0.70", "--output", str(workdir / "gate_report.json"), ], capture_output=True, text=True, ) print(proc.stdout or proc.stderr) print("exit code", proc.returncode, "->", {0: "PASS", 1: "BLOCKED", 2: "NEEDS_REVIEW"}.get(proc.returncode))

Gate status: BLOCKED (9288ms, multiclass)
  quadratic_kappa: 0.8633
  validation: 2 flag(s)
  performance: 0 flag(s)
  compliance: 0 flag(s)
  security: 1 flag(s)
  fairness: 10 flag(s)
Full report written to cli_multiclass/gate_report.json

exit code 1 -> BLOCKED

In [22]:

saved = json.loads((workdir / "gate_report.json").read_text())
print(json.dumps({k: v for k, v in saved.items() if k != "results_by_category"}, indent=2))

saved = json.loads((workdir / "gate_report.json").read_text()) print(json.dumps({k: v for k, v in saved.items() if k != "results_by_category"}, indent=2))

{
  "gate_status": "BLOCKED",
  "task": "multiclass",
  "model_metric": "quadratic_kappa",
  "model_score": 0.8633,
  "model_auc": null,
  "n_flags": 13,
  "total_duration_ms": 9288.39
}

In [23]:

import shutil

shutil.rmtree(workdir, ignore_errors=True)
print("cleaned up")

import shutil shutil.rmtree(workdir, ignore_errors=True) print("cleaned up")

cleaned up

Summary¶

Doing multiclass	Do this
Any multiclass problem	task="multiclass"
Outcomes have an order	class_order=[...] ascending — unlocks ordinal metrics and rank-aware robustness
Gating on ordinal quality	metric="quadratic_kappa" + min_score, or "ordinal_mae" + max_error
Nominal problem	accuracy / balanced_accuracy / f1 etc., with average="macro"
Demographic parity	favourable_classes=[...] — state it; the choice changes the answer

Three things worth carrying away:

1. Accuracy cannot see severity. Two models with the same error count can be very differently dangerous. quadratic_kappa and ordinal_mae are the metrics that notice.
2. "Favourable" is a business decision. "Accepted" and "not declined" are different questions with different answers, so the library asks rather than assumes.
3. Read max_observed_rank_shift. A model that only ever slips to an adjacent class is far safer than one that jumps the scale, at the same flip rate.

Next: 03_regression_sklearn.ipynb for pricing and claims, or 04_any_framework_classification.ipynb for PyTorch, XGBoost and Keras-shaped models.