SlideShare a Scribd company logo
Approximate Dynamic Programming Jong Min Lee Chemical and Materials Engineering University of Alberta A New Paradigm for Process Control & Optimization
How does a process industry run? Feedstock Purchase Plant / Unit Operation Inventory Control Supply Chain Management
What decisions do we make in process industries? Regulatory Control Real Time Optimizer Production Planning Strategic Planning Customer Plant Scheduling Advaced Process Control $ $ $ $ sec min ~ day week ~ month month ~ year
Ethylene Plant Furnaces Primary Fractionator Quench Tower Charge Gas Compressor Chilling Demethanizer Deethanizer Ethylene Fractionator Debutanizer Propylene Fractionator Depropanizer Fuel Oil Hydrogen Methane Ethylene Ethane Propylene Propane B - B Gasoline Light H-C Naphtha Feedstock
Regulatory Control LC LC FC FC Feed Keep flow rates, levels, .. @ specified values Decisions:  Valve opening [sec] Uncertainties:  Valve dynamics, resolutions
Scheduling and Planning Demands Inventories Ethylene Plant Feedstock Market Blending Daily ~ Monthly Maximize CSL and Profit Decisions:  Purchase / Blending / Unit Maintenance / Inventories / Distributions Uncertainties:  Market Prices / Raw Mat. Properties / Unit Failures / Demands… ? ? ? ? ETY PPY ETA BBP GSL
All the decision-making problems are fundamentally SAME We are concerned with future performance Future Time Profit
Conventional Tools Observer Decision Feedforward New Information Real outcome Optimizer Model Constraints Objective Function max  t = k+ 1 k+p performance Real World Future Past k k+ 1 k+p time
What are the issues of conventional tools? 1. They ignore UNCERTAINTIES. - Can yield wrong decisions 2. They put too much efforts ONLINE. - Can be late for timely decision
Analogy to Chess Me Opponent (Plant) Model Predictive Control Mixed Integer Programming h g f e d c b a 1 2 3 4 5 6 7 8 Opponent’s Move New Piece Position Exponential Explosion
Unbeatable Chess Player – Dynamic Programming Score (Value) for every feasible position Pick up the action giving the best “score”  (position: mine & the opponent’s) Already calculated (offline) before we start a game h g f e d c b a 1 2 3 4 5 6 7 8 Expected Optimal Value Set of Next Piece Positions Decision u1 x1 45 u2 x2 55
How do we find the “scores”? Discretization of entire state & action space INFEASIBLE = J  ( x ) min u  ( x ,  u )  J  ( x ’ ) + E x 1 x 2 x 3 u 1 u 2 u 3
Can we find the scores “approximately”? Converged  Value Fcn On-line Implementation Simulations w/ initial policies Value Function Approximation Iterative Improvement Off-line
Advantages of Approximate Dynamic Programming Manageable online computation Applicable to practical systems Stochastic systems as well as deterministic system All about simulation! Improved policy
Key to Success of ADP Store – Search – Averaging e.g.) nearest neighbor Convergence of Off-line Learning
Resource-Constrained  Project Scheduling J. Choi, et al.  Computers and Chemical Engineering , 28 (2004)
Drug Discovery / Development Discovery Development Market Drug 1 Drug 2 Drug n Phase 1 Phase 2 a/b Phase 3 Submission & Approval 0.5 – 2 yrs 1 – 2 yrs 1.5 – 3.5 yrs 2.5 – 4 yrs 0.5 – 2 yrs $2-4 MM $1-3 MM $5-25 MM $50-250 MM $5-20 MM Pre-clinical Development R&D takes  6.5 – 13.5 years 60 – 300 million $
Problem Complexity I 1 I 2 P 1 I 3 I 4 P 2 I 5 I 6 I 7 P 3 I 8 I 9 I 10 P 4 I 11 I 12 P 5 Drug 1 Drug 2 Drug 3 Drug 4 Drug 5 Success/Failure, Duration, Cost 1.2 x 10 9  scenarios 5 3 6 6 5 3 7 4 5 4 6 3 3 8 4 3 5
Simulations X  = [s 1 , s 2 , s 3 , s 4 , s 5 , z 1 , z 2 , z 3 , z 4 , z 5 , L 1 , L 2 , t] Which task is performed? Result of the most  recent task Duration 230 billion points Simulations (150000) 1. High Success Probability Task First 2. Short Duration Task First 3. High Reward Project First Sampled  X 3.7 x 10 5 probabilistic description
ADP improved on the starting policies 10000 realizations 0 4000 8000 12000 H1 H2 H3 ADP
Stochastic Optimal Control
If you ignore uncertainties… y(k+1) = y(k) +  b u(k) + e(k+1) parameter  change noise enters
ADP “actively” handles uncertainties Output & Input Parameter Estimate & Variance Active probing at  t=t b ( 10 ) :  t e =15 Decrease of parameter uncertainty t=10: parameter changes, t=15: exogenous noise enters
Summary ADP is a  computationally feasible   approach to large-scale  and  uncertain  systems and provides an  improved  solution “ ”
Ad

More Related Content

What's hot (20)

Process planning SMED and VSM: Single minute exchange of die and Value stream...
Process planning SMED and VSM: Single minute exchange of die and Value stream...Process planning SMED and VSM: Single minute exchange of die and Value stream...
Process planning SMED and VSM: Single minute exchange of die and Value stream...
Yatinkumar Patel
 
Capacity Planning and Modelling
Capacity Planning and ModellingCapacity Planning and Modelling
Capacity Planning and Modelling
Anthony Dehnashi
 
Capacity Planning
Capacity PlanningCapacity Planning
Capacity Planning
MOHD ARISH
 
How to Do Capacity Planning
How to Do Capacity PlanningHow to Do Capacity Planning
How to Do Capacity Planning
TeamQuest Corporation
 
Capacity and availability management (CMMI SVC 1.3 PA) Explained
Capacity and availability management  (CMMI SVC 1.3 PA) ExplainedCapacity and availability management  (CMMI SVC 1.3 PA) Explained
Capacity and availability management (CMMI SVC 1.3 PA) Explained
Vishnuvarthanan Moorthy
 
Material requirements planning in a demand driven world 2
Material requirements planning in a demand driven world 2Material requirements planning in a demand driven world 2
Material requirements planning in a demand driven world 2
jackson_bowie
 
5. capacity planning.
5. capacity planning.5. capacity planning.
5. capacity planning.
Akash Bakshi
 
Lean Manufacturing Exam Questions Mar 2011
Lean Manufacturing Exam Questions Mar 2011Lean Manufacturing Exam Questions Mar 2011
Lean Manufacturing Exam Questions Mar 2011
ExerciseLeanLLC
 
Process Strategy
Process StrategyProcess Strategy
Process Strategy
Tuhin Parves
 
LinggamResume2015M
LinggamResume2015MLinggamResume2015M
LinggamResume2015M
Linggam Narayanasamy
 
Capacity Planning
Capacity PlanningCapacity Planning
Capacity Planning
Pam Cudal
 
Pgbm03 MBA OPERATION MANAGEMENT session 06 planning and managing capacity
Pgbm03 MBA OPERATION MANAGEMENT session 06 planning and managing capacityPgbm03 MBA OPERATION MANAGEMENT session 06 planning and managing capacity
Pgbm03 MBA OPERATION MANAGEMENT session 06 planning and managing capacity
Aquamarine Emerald
 
Six Sigma & Process Capability
Six Sigma & Process CapabilitySix Sigma & Process Capability
Six Sigma & Process Capability
Eric Blumenfeld
 
Process Capability[1]
Process Capability[1]Process Capability[1]
Process Capability[1]
ahmad bassiouny
 
Strategic capacity planning for products and services
Strategic capacity planning for products and servicesStrategic capacity planning for products and services
Strategic capacity planning for products and services
gerlyn bonus
 
Capacity planning
Capacity planning Capacity planning
Capacity planning
Abdullah Shahid
 
Chap006
Chap006Chap006
Chap006
Cheska Custodio
 
Project Risk Management
Project Risk ManagementProject Risk Management
Project Risk Management
IFAD International Fund for Agricultural Development
 
Capacity Management
Capacity ManagementCapacity Management
Capacity Management
Yash Vardhan Lohia
 
Cost reduction strategies
Cost reduction strategiesCost reduction strategies
Cost reduction strategies
Dr. Lutfi Apiliogullari
 
Process planning SMED and VSM: Single minute exchange of die and Value stream...
Process planning SMED and VSM: Single minute exchange of die and Value stream...Process planning SMED and VSM: Single minute exchange of die and Value stream...
Process planning SMED and VSM: Single minute exchange of die and Value stream...
Yatinkumar Patel
 
Capacity Planning and Modelling
Capacity Planning and ModellingCapacity Planning and Modelling
Capacity Planning and Modelling
Anthony Dehnashi
 
Capacity Planning
Capacity PlanningCapacity Planning
Capacity Planning
MOHD ARISH
 
Capacity and availability management (CMMI SVC 1.3 PA) Explained
Capacity and availability management  (CMMI SVC 1.3 PA) ExplainedCapacity and availability management  (CMMI SVC 1.3 PA) Explained
Capacity and availability management (CMMI SVC 1.3 PA) Explained
Vishnuvarthanan Moorthy
 
Material requirements planning in a demand driven world 2
Material requirements planning in a demand driven world 2Material requirements planning in a demand driven world 2
Material requirements planning in a demand driven world 2
jackson_bowie
 
5. capacity planning.
5. capacity planning.5. capacity planning.
5. capacity planning.
Akash Bakshi
 
Lean Manufacturing Exam Questions Mar 2011
Lean Manufacturing Exam Questions Mar 2011Lean Manufacturing Exam Questions Mar 2011
Lean Manufacturing Exam Questions Mar 2011
ExerciseLeanLLC
 
Capacity Planning
Capacity PlanningCapacity Planning
Capacity Planning
Pam Cudal
 
Pgbm03 MBA OPERATION MANAGEMENT session 06 planning and managing capacity
Pgbm03 MBA OPERATION MANAGEMENT session 06 planning and managing capacityPgbm03 MBA OPERATION MANAGEMENT session 06 planning and managing capacity
Pgbm03 MBA OPERATION MANAGEMENT session 06 planning and managing capacity
Aquamarine Emerald
 
Six Sigma & Process Capability
Six Sigma & Process CapabilitySix Sigma & Process Capability
Six Sigma & Process Capability
Eric Blumenfeld
 
Strategic capacity planning for products and services
Strategic capacity planning for products and servicesStrategic capacity planning for products and services
Strategic capacity planning for products and services
gerlyn bonus
 

Viewers also liked (12)

C2 Acetylene Hydrogenation
C2 Acetylene HydrogenationC2 Acetylene Hydrogenation
C2 Acetylene Hydrogenation
Gerard B. Hawkins
 
A Multiple-Shooting Differential Dynamic Programming Algorithm
A Multiple-Shooting Differential Dynamic Programming AlgorithmA Multiple-Shooting Differential Dynamic Programming Algorithm
A Multiple-Shooting Differential Dynamic Programming Algorithm
Etienne Pellegrini
 
Elements of dynamic programming
Elements of dynamic programmingElements of dynamic programming
Elements of dynamic programming
Tafhim Islam
 
Dynamic Programming
Dynamic ProgrammingDynamic Programming
Dynamic Programming
contact2kazi
 
Dynamic programming - fundamentals review
Dynamic programming - fundamentals reviewDynamic programming - fundamentals review
Dynamic programming - fundamentals review
ElifTech
 
Dynamic programming class 16
Dynamic programming class 16Dynamic programming class 16
Dynamic programming class 16
Kumar
 
Dynamic Programming - Part 1
Dynamic Programming - Part 1Dynamic Programming - Part 1
Dynamic Programming - Part 1
Amrinder Arora
 
5.3 dynamic programming 03
5.3 dynamic programming 035.3 dynamic programming 03
5.3 dynamic programming 03
Krish_ver2
 
Dynamic pgmming
Dynamic pgmmingDynamic pgmming
Dynamic pgmming
Dr. C.V. Suresh Babu
 
Dynamic programming
Dynamic programmingDynamic programming
Dynamic programming
Shakil Ahmed
 
Dynamic Programming
Dynamic ProgrammingDynamic Programming
Dynamic Programming
Sahil Kumar
 
Dynamic Programming
Dynamic ProgrammingDynamic Programming
Dynamic Programming
paramalways
 
A Multiple-Shooting Differential Dynamic Programming Algorithm
A Multiple-Shooting Differential Dynamic Programming AlgorithmA Multiple-Shooting Differential Dynamic Programming Algorithm
A Multiple-Shooting Differential Dynamic Programming Algorithm
Etienne Pellegrini
 
Elements of dynamic programming
Elements of dynamic programmingElements of dynamic programming
Elements of dynamic programming
Tafhim Islam
 
Dynamic Programming
Dynamic ProgrammingDynamic Programming
Dynamic Programming
contact2kazi
 
Dynamic programming - fundamentals review
Dynamic programming - fundamentals reviewDynamic programming - fundamentals review
Dynamic programming - fundamentals review
ElifTech
 
Dynamic programming class 16
Dynamic programming class 16Dynamic programming class 16
Dynamic programming class 16
Kumar
 
Dynamic Programming - Part 1
Dynamic Programming - Part 1Dynamic Programming - Part 1
Dynamic Programming - Part 1
Amrinder Arora
 
5.3 dynamic programming 03
5.3 dynamic programming 035.3 dynamic programming 03
5.3 dynamic programming 03
Krish_ver2
 
Dynamic programming
Dynamic programmingDynamic programming
Dynamic programming
Shakil Ahmed
 
Dynamic Programming
Dynamic ProgrammingDynamic Programming
Dynamic Programming
Sahil Kumar
 
Dynamic Programming
Dynamic ProgrammingDynamic Programming
Dynamic Programming
paramalways
 
Ad

Similar to Approximate Dynamic Programming: A New Paradigm for Process Control & Optimization (20)

Industrial plant optimization in reduced dimensional spaces
Industrial plant optimization in reduced dimensional spacesIndustrial plant optimization in reduced dimensional spaces
Industrial plant optimization in reduced dimensional spaces
Capstone
 
Lean six sigma executive overview (case study) templates
Lean six sigma executive overview (case study) templatesLean six sigma executive overview (case study) templates
Lean six sigma executive overview (case study) templates
Steven Bonacorsi
 
CH1.ppt
CH1.pptCH1.ppt
CH1.ppt
FathiShokry
 
Quality Management.ppt
Quality Management.pptQuality Management.ppt
Quality Management.ppt
ddelucy
 
Presentation about Quality Management for beginners
Presentation about Quality Management for beginnersPresentation about Quality Management for beginners
Presentation about Quality Management for beginners
Mena Wagdy
 
Quality Management in true reality explained.ppt
Quality Management in true reality explained.pptQuality Management in true reality explained.ppt
Quality Management in true reality explained.ppt
HarishCN12
 
Quality Management with Detailed Version.ppt
Quality Management with Detailed Version.pptQuality Management with Detailed Version.ppt
Quality Management with Detailed Version.ppt
ZulqarnainTasawwar
 
1-introductionOPERARION RESEARCH TECHNIQUES.ppt
1-introductionOPERARION RESEARCH TECHNIQUES.ppt1-introductionOPERARION RESEARCH TECHNIQUES.ppt
1-introductionOPERARION RESEARCH TECHNIQUES.ppt
Praveen Kumar
 
1-introduction.ppt
1-introduction.ppt1-introduction.ppt
1-introduction.ppt
ParveshKumar17303
 
Catapult DOE Case Study
Catapult DOE Case StudyCatapult DOE Case Study
Catapult DOE Case Study
Larry Thompson, MfgT.
 
increasing the action gap - new operators for reinforcement learning
increasing the action gap - new operators for reinforcement learningincreasing the action gap - new operators for reinforcement learning
increasing the action gap - new operators for reinforcement learning
Ryo Iwaki
 
Deep Parameters Tuning for Android Mobile Apps
Deep Parameters Tuning for Android Mobile AppsDeep Parameters Tuning for Android Mobile Apps
Deep Parameters Tuning for Android Mobile Apps
Davide De Chiara
 
D03 15 Deliverable Roadmap
D03 15 Deliverable RoadmapD03 15 Deliverable Roadmap
D03 15 Deliverable Roadmap
Leanleaders.org
 
D03 15 Deliverable Roadmap
D03 15 Deliverable RoadmapD03 15 Deliverable Roadmap
D03 15 Deliverable Roadmap
Leanleaders.org
 
Control phase lean six sigma tollgate template
Control phase   lean six sigma tollgate templateControl phase   lean six sigma tollgate template
Control phase lean six sigma tollgate template
Steven Bonacorsi
 
Control phase lean six sigma tollgate template
Control phase   lean six sigma tollgate templateControl phase   lean six sigma tollgate template
Control phase lean six sigma tollgate template
Steven Bonacorsi
 
Causal reasoning and Learning Systems
Causal reasoning and Learning SystemsCausal reasoning and Learning Systems
Causal reasoning and Learning Systems
Trieu Nguyen
 
Business Process Monitoring and Mining
Business Process Monitoring and MiningBusiness Process Monitoring and Mining
Business Process Monitoring and Mining
Marlon Dumas
 
Quantum Business in Japanese Market
Quantum Business in Japanese MarketQuantum Business in Japanese Market
Quantum Business in Japanese Market
Yuichiro MInato
 
Optimization of Continuous Queries in Federated Database and Stream Processin...
Optimization of Continuous Queries in Federated Database and Stream Processin...Optimization of Continuous Queries in Federated Database and Stream Processin...
Optimization of Continuous Queries in Federated Database and Stream Processin...
Zbigniew Jerzak
 
Industrial plant optimization in reduced dimensional spaces
Industrial plant optimization in reduced dimensional spacesIndustrial plant optimization in reduced dimensional spaces
Industrial plant optimization in reduced dimensional spaces
Capstone
 
Lean six sigma executive overview (case study) templates
Lean six sigma executive overview (case study) templatesLean six sigma executive overview (case study) templates
Lean six sigma executive overview (case study) templates
Steven Bonacorsi
 
Quality Management.ppt
Quality Management.pptQuality Management.ppt
Quality Management.ppt
ddelucy
 
Presentation about Quality Management for beginners
Presentation about Quality Management for beginnersPresentation about Quality Management for beginners
Presentation about Quality Management for beginners
Mena Wagdy
 
Quality Management in true reality explained.ppt
Quality Management in true reality explained.pptQuality Management in true reality explained.ppt
Quality Management in true reality explained.ppt
HarishCN12
 
Quality Management with Detailed Version.ppt
Quality Management with Detailed Version.pptQuality Management with Detailed Version.ppt
Quality Management with Detailed Version.ppt
ZulqarnainTasawwar
 
1-introductionOPERARION RESEARCH TECHNIQUES.ppt
1-introductionOPERARION RESEARCH TECHNIQUES.ppt1-introductionOPERARION RESEARCH TECHNIQUES.ppt
1-introductionOPERARION RESEARCH TECHNIQUES.ppt
Praveen Kumar
 
increasing the action gap - new operators for reinforcement learning
increasing the action gap - new operators for reinforcement learningincreasing the action gap - new operators for reinforcement learning
increasing the action gap - new operators for reinforcement learning
Ryo Iwaki
 
Deep Parameters Tuning for Android Mobile Apps
Deep Parameters Tuning for Android Mobile AppsDeep Parameters Tuning for Android Mobile Apps
Deep Parameters Tuning for Android Mobile Apps
Davide De Chiara
 
D03 15 Deliverable Roadmap
D03 15 Deliverable RoadmapD03 15 Deliverable Roadmap
D03 15 Deliverable Roadmap
Leanleaders.org
 
D03 15 Deliverable Roadmap
D03 15 Deliverable RoadmapD03 15 Deliverable Roadmap
D03 15 Deliverable Roadmap
Leanleaders.org
 
Control phase lean six sigma tollgate template
Control phase   lean six sigma tollgate templateControl phase   lean six sigma tollgate template
Control phase lean six sigma tollgate template
Steven Bonacorsi
 
Control phase lean six sigma tollgate template
Control phase   lean six sigma tollgate templateControl phase   lean six sigma tollgate template
Control phase lean six sigma tollgate template
Steven Bonacorsi
 
Causal reasoning and Learning Systems
Causal reasoning and Learning SystemsCausal reasoning and Learning Systems
Causal reasoning and Learning Systems
Trieu Nguyen
 
Business Process Monitoring and Mining
Business Process Monitoring and MiningBusiness Process Monitoring and Mining
Business Process Monitoring and Mining
Marlon Dumas
 
Quantum Business in Japanese Market
Quantum Business in Japanese MarketQuantum Business in Japanese Market
Quantum Business in Japanese Market
Yuichiro MInato
 
Optimization of Continuous Queries in Federated Database and Stream Processin...
Optimization of Continuous Queries in Federated Database and Stream Processin...Optimization of Continuous Queries in Federated Database and Stream Processin...
Optimization of Continuous Queries in Federated Database and Stream Processin...
Zbigniew Jerzak
 
Ad

Recently uploaded (20)

Potential Crypto Airdrops – Checklist to Track the Most Promising Airdrops.pdf
Potential Crypto Airdrops – Checklist to Track the Most Promising Airdrops.pdfPotential Crypto Airdrops – Checklist to Track the Most Promising Airdrops.pdf
Potential Crypto Airdrops – Checklist to Track the Most Promising Airdrops.pdf
Coin Gabbar
 
Analysis of Waste Recycling Companies.pptx
Analysis of Waste Recycling Companies.pptxAnalysis of Waste Recycling Companies.pptx
Analysis of Waste Recycling Companies.pptx
ManikaGoyal13
 
Consolidated Accounting notes presentation 2
Consolidated Accounting notes presentation 2Consolidated Accounting notes presentation 2
Consolidated Accounting notes presentation 2
ashforddube14
 
TEAM 5hdhhhdhddhdhdhhdhddhdhdh FINAL.pptx
TEAM 5hdhhhdhddhdhdhhdhddhdhdh FINAL.pptxTEAM 5hdhhhdhddhdhdhhdhddhdhdh FINAL.pptx
TEAM 5hdhhhdhddhdhdhhdhddhdhdh FINAL.pptx
sivasankart859
 
Consolidated accounting notes presentation
Consolidated accounting notes presentationConsolidated accounting notes presentation
Consolidated accounting notes presentation
ashforddube14
 
Bladex Earnings Call Presentation 1Q2025
Bladex Earnings Call Presentation 1Q2025Bladex Earnings Call Presentation 1Q2025
Bladex Earnings Call Presentation 1Q2025
Bladex
 
WAND.pdfvq d ix and8dba d8nd qnd8duf es qid
WAND.pdfvq d ix and8dba d8nd qnd8duf es qidWAND.pdfvq d ix and8dba d8nd qnd8duf es qid
WAND.pdfvq d ix and8dba d8nd qnd8duf es qid
jeremysegundob
 
Silver One May 2025 Corporate Presentation
Silver One May 2025 Corporate PresentationSilver One May 2025 Corporate Presentation
Silver One May 2025 Corporate Presentation
Adnet Communications
 
George Mankiw Principle of Economics Chapter 26
George Mankiw Principle of Economics Chapter 26George Mankiw Principle of Economics Chapter 26
George Mankiw Principle of Economics Chapter 26
DyandraRenata
 
EE2025 basic definitions and its importamnce.pptx
EE2025 basic definitions and its importamnce.pptxEE2025 basic definitions and its importamnce.pptx
EE2025 basic definitions and its importamnce.pptx
AnsarAbbas97
 
Trusted Forex Broker Reviews for Smarter Trading
Trusted Forex Broker Reviews for Smarter TradingTrusted Forex Broker Reviews for Smarter Trading
Trusted Forex Broker Reviews for Smarter Trading
Broker Reviewfx
 
Top Dividend Paying Stocks in India 2025
Top Dividend Paying Stocks in India 2025Top Dividend Paying Stocks in India 2025
Top Dividend Paying Stocks in India 2025
Amit Finowings
 
How To Recover Stolen Funds From Online Trading Investment Scam
How To Recover Stolen Funds From Online Trading Investment ScamHow To Recover Stolen Funds From Online Trading Investment Scam
How To Recover Stolen Funds From Online Trading Investment Scam
raymondwilliam1022
 
Money Matters_ Transforming Your Financial Relationship.pdf
Money Matters_ Transforming Your Financial Relationship.pdfMoney Matters_ Transforming Your Financial Relationship.pdf
Money Matters_ Transforming Your Financial Relationship.pdf
pckhetal
 
Depreciation of equipment's ____-__ .ppt
Depreciation of equipment's ____-__ .pptDepreciation of equipment's ____-__ .ppt
Depreciation of equipment's ____-__ .ppt
bluehhh07
 
The Role of Storytelling in Legal Communication (www.kiu.ac.ug)
The Role of Storytelling in Legal Communication (www.kiu.ac.ug)The Role of Storytelling in Legal Communication (www.kiu.ac.ug)
The Role of Storytelling in Legal Communication (www.kiu.ac.ug)
publication11
 
Truxton Capital: Middle Market Quarterly Review - April 2025
Truxton Capital: Middle Market Quarterly Review - April 2025Truxton Capital: Middle Market Quarterly Review - April 2025
Truxton Capital: Middle Market Quarterly Review - April 2025
truxtontrust
 
Strategic Resources May 2025 Corporate Presentation
Strategic Resources May 2025 Corporate PresentationStrategic Resources May 2025 Corporate Presentation
Strategic Resources May 2025 Corporate Presentation
Adnet Communications
 
Endodontic CC 67890-98765e43567897652.pptx
Endodontic CC 67890-98765e43567897652.pptxEndodontic CC 67890-98765e43567897652.pptx
Endodontic CC 67890-98765e43567897652.pptx
KhalidLafi2
 
Dividend desiааауааааааааааааааааgn.pptx
Dividend desiааауааааааааааааааааgn.pptxDividend desiааауааааааааааааааааgn.pptx
Dividend desiааауааааааааааааааааgn.pptx
smolik1tanya
 
Potential Crypto Airdrops – Checklist to Track the Most Promising Airdrops.pdf
Potential Crypto Airdrops – Checklist to Track the Most Promising Airdrops.pdfPotential Crypto Airdrops – Checklist to Track the Most Promising Airdrops.pdf
Potential Crypto Airdrops – Checklist to Track the Most Promising Airdrops.pdf
Coin Gabbar
 
Analysis of Waste Recycling Companies.pptx
Analysis of Waste Recycling Companies.pptxAnalysis of Waste Recycling Companies.pptx
Analysis of Waste Recycling Companies.pptx
ManikaGoyal13
 
Consolidated Accounting notes presentation 2
Consolidated Accounting notes presentation 2Consolidated Accounting notes presentation 2
Consolidated Accounting notes presentation 2
ashforddube14
 
TEAM 5hdhhhdhddhdhdhhdhddhdhdh FINAL.pptx
TEAM 5hdhhhdhddhdhdhhdhddhdhdh FINAL.pptxTEAM 5hdhhhdhddhdhdhhdhddhdhdh FINAL.pptx
TEAM 5hdhhhdhddhdhdhhdhddhdhdh FINAL.pptx
sivasankart859
 
Consolidated accounting notes presentation
Consolidated accounting notes presentationConsolidated accounting notes presentation
Consolidated accounting notes presentation
ashforddube14
 
Bladex Earnings Call Presentation 1Q2025
Bladex Earnings Call Presentation 1Q2025Bladex Earnings Call Presentation 1Q2025
Bladex Earnings Call Presentation 1Q2025
Bladex
 
WAND.pdfvq d ix and8dba d8nd qnd8duf es qid
WAND.pdfvq d ix and8dba d8nd qnd8duf es qidWAND.pdfvq d ix and8dba d8nd qnd8duf es qid
WAND.pdfvq d ix and8dba d8nd qnd8duf es qid
jeremysegundob
 
Silver One May 2025 Corporate Presentation
Silver One May 2025 Corporate PresentationSilver One May 2025 Corporate Presentation
Silver One May 2025 Corporate Presentation
Adnet Communications
 
George Mankiw Principle of Economics Chapter 26
George Mankiw Principle of Economics Chapter 26George Mankiw Principle of Economics Chapter 26
George Mankiw Principle of Economics Chapter 26
DyandraRenata
 
EE2025 basic definitions and its importamnce.pptx
EE2025 basic definitions and its importamnce.pptxEE2025 basic definitions and its importamnce.pptx
EE2025 basic definitions and its importamnce.pptx
AnsarAbbas97
 
Trusted Forex Broker Reviews for Smarter Trading
Trusted Forex Broker Reviews for Smarter TradingTrusted Forex Broker Reviews for Smarter Trading
Trusted Forex Broker Reviews for Smarter Trading
Broker Reviewfx
 
Top Dividend Paying Stocks in India 2025
Top Dividend Paying Stocks in India 2025Top Dividend Paying Stocks in India 2025
Top Dividend Paying Stocks in India 2025
Amit Finowings
 
How To Recover Stolen Funds From Online Trading Investment Scam
How To Recover Stolen Funds From Online Trading Investment ScamHow To Recover Stolen Funds From Online Trading Investment Scam
How To Recover Stolen Funds From Online Trading Investment Scam
raymondwilliam1022
 
Money Matters_ Transforming Your Financial Relationship.pdf
Money Matters_ Transforming Your Financial Relationship.pdfMoney Matters_ Transforming Your Financial Relationship.pdf
Money Matters_ Transforming Your Financial Relationship.pdf
pckhetal
 
Depreciation of equipment's ____-__ .ppt
Depreciation of equipment's ____-__ .pptDepreciation of equipment's ____-__ .ppt
Depreciation of equipment's ____-__ .ppt
bluehhh07
 
The Role of Storytelling in Legal Communication (www.kiu.ac.ug)
The Role of Storytelling in Legal Communication (www.kiu.ac.ug)The Role of Storytelling in Legal Communication (www.kiu.ac.ug)
The Role of Storytelling in Legal Communication (www.kiu.ac.ug)
publication11
 
Truxton Capital: Middle Market Quarterly Review - April 2025
Truxton Capital: Middle Market Quarterly Review - April 2025Truxton Capital: Middle Market Quarterly Review - April 2025
Truxton Capital: Middle Market Quarterly Review - April 2025
truxtontrust
 
Strategic Resources May 2025 Corporate Presentation
Strategic Resources May 2025 Corporate PresentationStrategic Resources May 2025 Corporate Presentation
Strategic Resources May 2025 Corporate Presentation
Adnet Communications
 
Endodontic CC 67890-98765e43567897652.pptx
Endodontic CC 67890-98765e43567897652.pptxEndodontic CC 67890-98765e43567897652.pptx
Endodontic CC 67890-98765e43567897652.pptx
KhalidLafi2
 
Dividend desiааауааааааааааааааааgn.pptx
Dividend desiааауааааааааааааааааgn.pptxDividend desiааауааааааааааааааааgn.pptx
Dividend desiааауааааааааааааааааgn.pptx
smolik1tanya
 

Approximate Dynamic Programming: A New Paradigm for Process Control & Optimization

  • 1. Approximate Dynamic Programming Jong Min Lee Chemical and Materials Engineering University of Alberta A New Paradigm for Process Control & Optimization
  • 2. How does a process industry run? Feedstock Purchase Plant / Unit Operation Inventory Control Supply Chain Management
  • 3. What decisions do we make in process industries? Regulatory Control Real Time Optimizer Production Planning Strategic Planning Customer Plant Scheduling Advaced Process Control $ $ $ $ sec min ~ day week ~ month month ~ year
  • 4. Ethylene Plant Furnaces Primary Fractionator Quench Tower Charge Gas Compressor Chilling Demethanizer Deethanizer Ethylene Fractionator Debutanizer Propylene Fractionator Depropanizer Fuel Oil Hydrogen Methane Ethylene Ethane Propylene Propane B - B Gasoline Light H-C Naphtha Feedstock
  • 5. Regulatory Control LC LC FC FC Feed Keep flow rates, levels, .. @ specified values Decisions: Valve opening [sec] Uncertainties: Valve dynamics, resolutions
  • 6. Scheduling and Planning Demands Inventories Ethylene Plant Feedstock Market Blending Daily ~ Monthly Maximize CSL and Profit Decisions: Purchase / Blending / Unit Maintenance / Inventories / Distributions Uncertainties: Market Prices / Raw Mat. Properties / Unit Failures / Demands… ? ? ? ? ETY PPY ETA BBP GSL
  • 7. All the decision-making problems are fundamentally SAME We are concerned with future performance Future Time Profit
  • 8. Conventional Tools Observer Decision Feedforward New Information Real outcome Optimizer Model Constraints Objective Function max  t = k+ 1 k+p performance Real World Future Past k k+ 1 k+p time
  • 9. What are the issues of conventional tools? 1. They ignore UNCERTAINTIES. - Can yield wrong decisions 2. They put too much efforts ONLINE. - Can be late for timely decision
  • 10. Analogy to Chess Me Opponent (Plant) Model Predictive Control Mixed Integer Programming h g f e d c b a 1 2 3 4 5 6 7 8 Opponent’s Move New Piece Position Exponential Explosion
  • 11. Unbeatable Chess Player – Dynamic Programming Score (Value) for every feasible position Pick up the action giving the best “score” (position: mine & the opponent’s) Already calculated (offline) before we start a game h g f e d c b a 1 2 3 4 5 6 7 8 Expected Optimal Value Set of Next Piece Positions Decision u1 x1 45 u2 x2 55
  • 12. How do we find the “scores”? Discretization of entire state & action space INFEASIBLE = J  ( x ) min u  ( x , u )  J  ( x ’ ) + E x 1 x 2 x 3 u 1 u 2 u 3
  • 13. Can we find the scores “approximately”? Converged Value Fcn On-line Implementation Simulations w/ initial policies Value Function Approximation Iterative Improvement Off-line
  • 14. Advantages of Approximate Dynamic Programming Manageable online computation Applicable to practical systems Stochastic systems as well as deterministic system All about simulation! Improved policy
  • 15. Key to Success of ADP Store – Search – Averaging e.g.) nearest neighbor Convergence of Off-line Learning
  • 16. Resource-Constrained Project Scheduling J. Choi, et al. Computers and Chemical Engineering , 28 (2004)
  • 17. Drug Discovery / Development Discovery Development Market Drug 1 Drug 2 Drug n Phase 1 Phase 2 a/b Phase 3 Submission & Approval 0.5 – 2 yrs 1 – 2 yrs 1.5 – 3.5 yrs 2.5 – 4 yrs 0.5 – 2 yrs $2-4 MM $1-3 MM $5-25 MM $50-250 MM $5-20 MM Pre-clinical Development R&D takes 6.5 – 13.5 years 60 – 300 million $
  • 18. Problem Complexity I 1 I 2 P 1 I 3 I 4 P 2 I 5 I 6 I 7 P 3 I 8 I 9 I 10 P 4 I 11 I 12 P 5 Drug 1 Drug 2 Drug 3 Drug 4 Drug 5 Success/Failure, Duration, Cost 1.2 x 10 9 scenarios 5 3 6 6 5 3 7 4 5 4 6 3 3 8 4 3 5
  • 19. Simulations X = [s 1 , s 2 , s 3 , s 4 , s 5 , z 1 , z 2 , z 3 , z 4 , z 5 , L 1 , L 2 , t] Which task is performed? Result of the most recent task Duration 230 billion points Simulations (150000) 1. High Success Probability Task First 2. Short Duration Task First 3. High Reward Project First Sampled X 3.7 x 10 5 probabilistic description
  • 20. ADP improved on the starting policies 10000 realizations 0 4000 8000 12000 H1 H2 H3 ADP
  • 22. If you ignore uncertainties… y(k+1) = y(k) + b u(k) + e(k+1) parameter change noise enters
  • 23. ADP “actively” handles uncertainties Output & Input Parameter Estimate & Variance Active probing at t=t b ( 10 ) : t e =15 Decrease of parameter uncertainty t=10: parameter changes, t=15: exogenous noise enters
  • 24. Summary ADP is a computationally feasible approach to large-scale and uncertain systems and provides an improved solution “ ”