ML Model Monitoring مُشرح ببساطة (مع مخططات وكود حقيقي)

┌─────────────────────────────────────────────────────────────┐
│                  MONITORING ML EN PRODUCTION                  │
└─────────────────────────────────────────────────────────────┘

   ┌────────────────┐  ┌────────────────┐  ┌────────────────┐
   │  PILIER 1       │  │  PILIER 2       │  │  PILIER 3       │
   │  Système        │  │  Données        │  │  Modèle         │
   │  (infra)        │  │  (data + drift) │  │  (performance)  │
   └────────────────┘  └────────────────┘  └────────────────┘
   - Latence            - Distribution X    - Accuracy
   - QPS                - Valeurs manqu.    - F1, AUC
   - CPU/Mémoire        - Outliers          - Calibration
   - Erreurs 5xx        - Drift KS/PSI      - Business KPI

from sklearn.calibration import calibration_curve
import matplotlib.pyplot as plt

prob_pred = model.predict_proba(X_test)[:, 1]
prob_true, prob_pred = calibration_curve(y_test, prob_pred, n_bins=10)

plt.plot([0, 1], [0, 1], 'k--')
plt.plot(prob_pred, prob_true, 'o-')
plt.xlabel('Probabilité prédite')
plt.ylabel('Probabilité observée')
plt.title('Calibration plot')

P_train(X) ≠ P_prod(X)

ou plus généralement :

P_train(X, Y) ≠ P_prod(X, Y)

Distribution training            Distribution production
       
       *  *                              
      ***                                       *
     *****                                     ***
    *******                                   *****
   *********              vs                  ********
  ***********                                **********
 *************                              *************
───────────────────                ──────────────────────────
        Pas de drift                    DRIFT DÉTECTÉ
                                  (la distribution s'est décalée)

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

np.random.seed(42)
n_train = 5000

train_age = np.random.normal(35, 10, n_train)
train_age = np.clip(train_age, 18, 80)

train_income = train_age * 1000 + np.random.normal(0, 5000, n_train)

prod_age = np.random.normal(50, 12, n_train)
prod_age = np.clip(prod_age, 18, 80)

prod_income = prod_age * 1000 + np.random.normal(0, 5000, n_train)

fig, axes = plt.subplots(1, 2, figsize=(14, 5))

axes[0].hist(train_age, bins=30, alpha=0.5, label='Train', color='blue')
axes[0].hist(prod_age, bins=30, alpha=0.5, label='Production', color='red')
axes[0].axvline(train_age.mean(), color='blue', linestyle='--', label=f'Train moy={train_age.mean():.1f}')
axes[0].axvline(prod_age.mean(), color='red', linestyle='--', label=f'Prod moy={prod_age.mean():.1f}')
axes[0].set_title('Distribution de "age" — covariate shift visible')
axes[0].set_xlabel('Age')
axes[0].legend()

axes[1].hist(train_income, bins=30, alpha=0.5, label='Train', color='blue')
axes[1].hist(prod_income, bins=30, alpha=0.5, label='Production', color='red')
axes[1].set_title('Distribution de "income"')
axes[1].set_xlabel('Income')
axes[1].legend()

plt.tight_layout()
plt.savefig('drift_visualization.png')
plt.show()