~Adversarial Bandit Control Learning

https://register.epo.org/espacenet/regviewer?AP=21834552&CY=EP&LG=en&DB=REG
https://patentscope.wipo.int/search/en
/detail.jsf?docId=WO2022003733&_fid
=US393708628
https://ppubs.uspto.gov/pubwebapp/external.html?q=(US-
20230083762-A1).did.&db=US-PGPUB
 Unique Objectives : Why we need this Model / Framework

• Interpretable & flexibility over hypothesis space ( open to hand picked rule )
• Missing Value Imputation & Feature Engineering Tool (Band/NLP/Sequence/ DL – Adversarial Forest)
• Robust to sensitive data & High stability
• Multivariate interaction Extraction & stable operating conditions
• Combined Feature selection (Interaction – Resp index )& combined impact/ stability analysis
• Single model for Multi phase data - process cycle optimization (Applicable only for same sensor across multiple phase)
• Controllability vs optimization - optimal operating condition (with Both Side Definition - Min & Max)
• Multi criteria optimization - combined response optimization (multiple Y / Response)
• Anomaly tracking & detection - band/rule fluctuations ( RCA )
• Systematic tracking of anomaly - sensitive var Subset
• Auto EDA – Key Interaction highlight w.r.t Response

Topic Reference :
Response surface method
Information Theory
Bandit Algorithm for Supervised Learning
Tree, Bagging & Boosting Algorithm
Active Learning
 Supervised Learning Module : Bandit Control Tree / Forest
Concepts :
Alternative to supervised learning model : Rule engine {similar algo to extract rule like Decision tree }
Raw Input Data
Bucket mapping from Similarity Pair : Do binning/ bucket ( quartile / vector discretization ) - each

continuous variable level

run for all continuous variable for bucket mapping. Number of bin will be based on data driven technique.

Binning can be done based on response variable Y.

rule : As combination to bucket/bin at variable

like :

Based on Y var1 has : 5 bin, var2 has : 10 bin, var3 has : 15 bin

var1:bin5,var2:bin9,var3:bin6 - #_match(1) : 80, #_nomatch(0) 20, tot obs: 821

var1:bin3,var2:bin6,var3:bin7 - #_match(1) : 85, #_nomatch(0) 15, tot obs: 678

%#_match(1) : probability of getting 1(match) (at rule level like D-tree)

%#_nomatch(0) : probability of getting 0(nomatch) (at rule level like D-tree)

result will be same as D tree algo

advantage : rule will include all variable and give a fixed cutoff

we can rank them based on %_match, and tot pair - higher is better

it will help us to apply rule directly on data to predict - match / nomatch

 Bucket / Splitting
Raw Input Data Transformed Data
Input Response
X1 x2 x3 x4 x5 y1 y2 y3 y3a y3b y3c
1 8 46 a 7 92 1 a 1 0 0
4 6 35 a 7 95 1 a 1 0 0
5 9 28 a 7 90 1 a 1 0 0
4 6 33 a 1 92 0 a 1 0 0
2 10 30 a 1 89 1 b 0 1 0
1 9 22 b 1 89 1 b 0 1 0
4 8 44 b 1 91 1 b 0 1 0
1 5 39 b 1 94 0 c 0 0 1
5 5 25 b 1 99 0 c 0 0 1
3 5 39 b 4 100 0 c 0 0 1
3 6 50 c 4 99 0 c 0 0 1
4 6 38 c 4 95 0 c 0 0 1
4 5 29 c 4 95 0 c 0 0 1
1 6 38 c 6 98 1 c 0 0 1
3 8 21 c 6 90 1 c 0 0 1
1 5 37 a 6 97 1 a 1 0 0

Data Transformation on Direct input & Transformed Feature Bucket size –

Split Criteria – independent & Dependent var1 with 2000 Unique pt. -> 40 to 50 bucket
(with max bucket size – 10 to 100) -> Bucket Details var2 with 10000 to 50000 unique pts -> 100 bucket
Bucket size – based on cardinality & few other factor from input vars Same Applicable for Response Var Y ( Continuous ) -> Percentile / Bucket
( Both Side Definition - Min & Max )
Using these Split Definition – transform data [Response sensitive Segmentation ]
Band Tree Table formation after applying different split criteria

Using These
Aggregate trees for
/ Group by generating
scores from out
of bag sample

=A

1) Split Criteria – independent & Dependent (with max bucket size – 10 to

100)

=B 2) Variable subset -> N/2, Log(N), nCk : ( K<N ) : continuous only

( Categorical fixed )
Bucket creation based on 1 & 2
Band Tree Table extraction for all 1 X 2 combination
 Band Tree Table ensemble Prediction – Scoring / Classification
Basic eda - uva, Correlation & BVA analysis,
features cluster

1) Var type & granularity ( unique pt/ var Row

size) - continuous & split eligible

2) Splitting criteria – independent, Dependent

3) for all filtered set of vars/ features - band or

rule / branch (x10, y25, z47 - > response 0.70,
X30, y5, z3 - > response 0.80)
Split vars with bin range 10 – 100

4) create multiple Subset of vars vs split

combination to create band tree / band forest

5) for each band get count of class, average of

response
for regression - average, median
Classification - average of probability / ratio
Also add other uva, bva score mode, sd, woe,
Snr....
One Subset of vars & one splitting criteria - one
band tree
... Like multiple diff Subset of vars & diff splitting
criteria - multiple band tree / band forest

Regression 6) for prediction - get band level map based on

band definition (x10 : min x =2.5 max x= 6.5;
Classification - binary & y25 : min y = 3.65 max y=50.45)
multiclass
Bandit control, optimal Get response score generated from band tree table
condition, risk level - scoring / - tree level score
Get average at band forests level aggregation –
ratio
Model Building Steps :
A) For classification get weighted average.
Data dimension : 10M X 200 Input [Full Batch – 2000], Response : Unit Count, Yield, Quality Class
Weight - multivariate IV based from each
Band tree (scaled)
Split Data into 70% Train [FB-1400]– 30% Test [FB-600] [15% ITV, 15% OTV]
For 7M Data points – 5M for Generating tree, 2M for OOB Validation B) For Regression - average / weighted - Central
tendency
Band Tree Table ensemble Prediction – Regularization
Regularization &
Prediction :
Robustness check

1) Band pruning : rules

filtering
Filter band by number of
supporting observations /
high residual - rule
fluctuation

2) Band tree table weight :

Predictive power associated
to single tree
Filter band tree tables by IV,
info content weight, validate
with training split rule to
check psi
Surfaces / band tree
fluctuation

Filter band tree based on information content & For each data point /obs
Apply split details (bucket
predictive power – IV, AUC, R2 /band definition) to map
specific band of a band tree
Take scaled score for ensemble model – weighted table to generate
score responses score

Get response score from

band 1 from band tree table
1
Band 2 from band tree
table 2
Band 3 from band tree
 Inference code / Hypothesis / Model objects :

Basic eda - uva, Correlation & BVA analysis, features cluster Model Building Steps :

1) extracted band / rules from training - boundary details for Data dimension : 10M X 200 Input [Full Batch – 2000]
splitting criteria for all eligible vars Response : Unit Count, Yield, Quality Class
(bucket / band definition)
Split Data into 70% Train [FB-1400]– 30% Test [FB-
2) apply these rule & get response count / avg... From validation 600] [15% ITV, 15% OTV]
sample
For 7M Data points – 5M for Generating tree, 2M for
3) column join training band tree table & validation band tree OOB Validation
table – Anomaly Tracking & sensitivity analysis
Anomaly tracking detection as band / rule fluctuation Algorithm Sequence :
4) band score : measure residual, cov, Cross Entropy - relative, IV 1] Split & Bucket generation from data
conditional expectation for both 2] Data Transformation on Direct Input & Transformed Feature
training & validation response (oob validation & measurement) 3] Band Tree Table formation after applying different split & Vars Combination
4] Band Tree Table ensemble Prediction – Regression / Classification / Scoring
5) get band tree table score - psi, R2, Mae, CE, IV, sq err
5] Band Tree Table ensemble Prediction – Regularization
6) filter/Prune/Prioritize band & band tree table (based on score/ 6] Tune Hyperparameter of Learning Model
normalized ) 7] Model Diagnostics & Validation
Once Model is Ready & Running we can use it for

Anomaly Tracking & Band update [ Same (Split, vars & Bucket) combination
=> Band Tree Training vs New comparison ]
 Hyperparameter of Learning Model
Hyperparameters of algo: Model Diagnostics :

Split range - 10 - 100 ( continues vars with specific granularity) Summary Table : List of Tree Table with all scoring & contribution,
[ Higher Split range - high variance model ] – stability issue in long run residual diagnostics details for weighting -ensemble
[ lower split range – low variance model ]
Residual Diagnostics – Confusion Matrix
AUC – ROC, R2, MSE, MAE … etc.
Split / bucket size – based on cardinality & few other( SNR, SD) factor from input vars (unique
pt. / tot obs)
Score quality checking, Rank order checking

Var Subset index - sqrt(tot vars), n/2…..etc Stability & Sensitivity Analysis – Deviation in previous vs Current
score & critical parameters
Split criteria –
binning / discretization : ChiMerge, MDLP, CAIM, khiops, Adaptive Further diagnostics – IML XAI ( Optional )
Quantizer, Mutual Information based : PMI, MIC,
change pt, cov / density / contour based…….etc Feature Importance( Forest level ) & Combined interaction
Importance
( Tree Level )
Band tree Table Weight score – IV,AUC, R2 ….etc

Prediction - get count, average, weighted avg, class, probability

Out of bag sample size : 30%

## Fluctuations pruning : true / false (warm up / incremental addition)
 Anomaly Tracking & Band update :

A) join training band tree & New band tree – Anomaly Tracking &
sensitivity analysis
Anomaly tracking detection as band / rule fluctuation
Hurricane
Irma Created
Some New Rule might appear in data & few old rule might come a New Island
Along the
insignificant due to fluctuation Georgia Coast
New Data with Band Tree A1
X1 x2 x4 x5 y1 y2
3 5 a 7 96 2
3 5 a 1 92 0
1 9 b 2 50 1
1 9 b 1 86 1
3 7 c 4 46 0

Joining table – Training

Table
Add new rule
& Also Measure
Fluctuation

Once Model is Ready & Running we can use

it for Anomaly Tracking & Band update Measure Deviation PSI
[ Same (Split, vars & Bucket) combination => Change in moment /
Band Tree Old vs New comparison ] Distribution
 Further Scope & Research

Basic eda - uva, Correlation & BVA analysis, features cluster

As a Feature Engineering Tool

• Process script / process modelling

• Sequence attention - process level - Response retrieval
• Scoring / classification – LSTM, CNN
• Apply NLP algo - on sequence token
• Sequence2vec, surfaces2surface
• Summarization - set of critical settings / pt / band / path
Extraction /
• Gan / Adversarial Model - surface generation / simulation
• Process Similarity, process base, process hierarchy / • Process script / process modelling
• Sequence attention - process
segmentation simulation, process retrieval
• sequence level
NLP, Sequence • Each Band as sequence token /
band scripts
Modeling, Deep • Scoring / classification - CNN LSTM
Learning • Apply NLP algo - on sequence token
• Sequence2vec, surfaces2surface,
• Summarization - set of critical
settings / pt / band / path
Extraction /
• Gan - surface generation /
simulation
• Process Similarity, process base,
process hierarchy / segmentation
Further Scope & Research
Architecture for IOT Application
Architecture for IOT Application
Thank You

Soumyajit Das
[email protected]