Safe Reinforcement Learning

Aug 10, 201816 likes1,649 views

안녕하세요. 이동민입니다. :) 2018. 8. 9일에 한국항공우주연구원에서 발표한 "Safe Reinforcement Learning" 발표 자료입니다. 목차는 다음과 같습니다. 1. Reinforcement Learning 2. Safe Reinforcement Learning 3. Optimization Criterion 4. Exploration Process 강화학습 계속 공부하면서 실제로 많은 분들이 쓸 수 있게 하려면 더 안전하고 빨라야한다는 생각이 들었습니다. 그래서 이에 관련하여 논문과 각종 자료들로 공부하여 발표하였습니다. 많은 분들께 도움이 되었으면 좋겠습니다. 감사합니다!

d
• m S rS
• 7 K a
. h i Ob 7P
, N . O y eo P
, N . O AEGB I AFAG 7A IGEGC. G G I K E GP
LIE AG E I K G G G IAL 4 0 I
, - N . ug !
• 7 69 2 wn ~ t
2 839 12 f ::9 w R c a

2 . 4 2 1
2 . 4 2 1
242 2 2 2
3 2 .
2 . 4 2 1
32

i )
n ( /ooc wu r p h
i n ( 7?? ( @:37 3 ?
:? h [s .? ? t 3? :
e :?b b g [sd [ - A k
:? h R I I [ t
] :? [ a i S

dp )
( u a Ou a A R
O : ( S i c O A w k y
( : d c lr
e G x t u a!
. B ( P i : On g y m
do u ( uo - n !
+ ? ,

-,
, /, , , - /, , , / /
, , , , , , - ,
, , -, . /, , .

L
L
L
!" = $"%& + ($"%) + ()
$"%* + ⋯ = ∑-./
0
(-
$"%-%&

id , - h , ,.,
c e ., i , ,., Wk
, ,., , l , ,., h
h a . Ch . . Ch
, , b

id , -
c e - i - Wk C
- l . -
a - - . - .
. b

Worst case
!"#
$(&')
100
0
&'
!"#
High
Reward

NO P I K
TS /. BC :? /? :5 . C:=: C:? C B 5 / :
P R TS CC B ? C 5?= C5 ( / )5

O N T KR S !
/ B C ? = / I ? ? : C B : B
P : C 5 :) ( .

a P T cK eO aR aI b a Sd N
/. BC :? /? :5 . C:=: C:? C B 5 / :
CC B ? C 5?= C5 ( / )5

OPS S T KI b a N
/. BC :? /? :5 . C:=: C:? C B 5 / :
R CC B ? C 5?= C5 ( / )5

O b NT S c I dP bK
a /. BC :? /? :5 . C:=: C:? C B 5 / :
R a CC B ? C 5?= C5 ( / )5

N e T O P dR N K e b dR !
ca / B C ? = / I ? ? : C B : B
S ca : C 5 :) ( .

N !!!
PO / B C ? = / I ? ? : C B : B
K PO : C 5 :) ( .

P ! 2:11 1 3 L D
M a ./:
3 9 5 1 3/:1 71 72- 7 11 9/:

- , . 1 21 ?
-: 2 - - , ( 1- - - , (
) )

) ( 2 . 2 2 1 1 : !", $", !%, $%, …
E )
T?

.
. : . . . . . / . . : .
A :: : / . ..
: : .
.
: .
:
. .

?
ML ? 7 C7 7 ? 7 7? 7 ? ? 4 C7 ? = 7
I ML : C H 47 : . / 2 7 7 H 47 &C

O Z I L N E
8 . : . : . :/: ?= 8.
= AAA :? ?/ :8 A.

8
& /= = : ? = : E : 8 E 8 8 8=
H & 7 & E= = 7 ?. ? E=

A
SR 1//8 0 (& ) . G ADA G:G H IKGG D :HN LA=
P Q SR GI MMM N K K D M: L )2=.M8 0 ? : KH N K K D((I

:CDCNC ACO
1CCM :CG DLNAC C P 5C N G 1LCO P LNH CP IL 5G H
0L MNC C OGRC NRCT L DC :CG DLNAC C P 5C N G ( ' 9 MCN 5G H
N OP :C GL 9LIGAT 8MPG G PGL ( ' 9 MCN 5G H
:98 N OP :C GL 9LIGAT 8MPG G PGL - 4 BCMP :COC NA 9 MCN :CRGCS L C 5G H
I LNGP O DLN 4 RCNOC :CG DLNAC C P 5C N G ( 9 MCN 5G H
MMNC PGACO GM 5C N G RG 4 RCNOC :CG DLNAC C P 5C N G ( ) 9 MCN 5G H
9: (,- MMNC PGACO GM 5C N G RG 4 RCNOC :CG DLNAC C P 5C N G 99 5G H
DC IPG C P :CG DLNAC C P 5C N G DLN PL L L O 1NGRG 99 5G H
4 PNLB APGL LD 4 RCNOC :CG DLNAC C P 5C N G 99 5G H
4 RCNOC :CG DLNAC C P 5C N G OCB L 0NGPGA I P PC L C 5G H
4 GP PGL 5C N G DLN 8 NL LP OG 4 PC L C 5G H
:L LP IC N O PL MI T P IC PC GO T G GP PGL L C 5G H
422: 9 ( ' - 1CCM G GA M MCN O MMIC C P NT RGBCL L C 5G H

第11回強化学習アーキテクチャ勉強会での発表資料です． 2018/1/16　@Φカフェスライドの誤り等ありましたら，ご連絡いただければ幸いです． ※[2018/1/17]p.9, 11の尤度，総和ではなく総乗の誤りです．アブストラクト：逆強化学習はエージェントの振舞データを所与として，エージェントが受け取っていたとされる報酬関数を推定する手法である．推定した報酬は二つの用途に用いられる．一つ目は推定した報酬にもとづいて強化学習を行い，エージェントの方策を学習する「徒弟学習」，二つ目は推定した報酬をもとにエージェントがどのような目的を持って行動したのかを理解する「行動解析」である．本発表では二つ目の「行動解析」に対する有効なアプローチであるノンパラメトリックベイズを用いた逆強化学習について説明する．基礎となるベイジアン逆強化学習ではエージェントがどのようなタスクを解いていたのかなどの事前知識を報酬の事前確率として導入し，エージェントに適合する報酬の事後確率を計算する手法である．これをノンパラメトリック化することにより，導入できる事前知識の自由度を大きくできるため，より複雑な報酬の事後確率を計算することができる．紹介する手法では，迷路の途中で目的地を変更する振舞データが与えられたときや複数の報酬から生成された振舞データが混ざっているときなどに，エージェントの行動解析に有効なアプローチである．

【DL輪読会】Responsive Safety in Reinforcement Learning by PID Lagrangian Methods...Deep Learning JP

안.전.제.일. 강화학습!Dongmin Lee

안녕하세요. 이번에 '1st 함께하는 딥러닝 컨퍼런스'에서 "안.전.제.일. 강화학습"이란 주제로 발표한 이동민이라고 합니다. 컨퍼런스 관련 링크는 다음과 같습니다. https://tykimos.github.io/2018/06/28/ISS_1st_Deep_Learning_Conference_All_Together/ 그리고 대략적인 개요는 다음과 같습니다. 1. What is Artificial Intelligence? 2. What is Reinforcement Learning? 3. What is Artificial General Intelligence? 4. Planning and Learning 5. Safe Reinforcement Learning 또한 이 자료에는 "Imagination-Augmented Agents for Deep Reinforcement Learning"이라는 논문을 자세히 설명하였습니다. 많은 분들이 보시고 도움이 되셨으면 좋겠습니다~!

[DL輪読会] Residual Attention Network for Image ClassificationDeep Learning JP

파이썬과 케라스로 배우는 강화학습 저자특강Woong won Lee

Doing Deep Reinforcement learning with PPO이 의령

【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...Deep Learning JP

ディープラーニングによる時系列データの異常検知Core Concept Technologies

AlphaGo Zero 解説suckgeun lee

Maximum Entropy Reinforcement Learning (Stochastic Control)Dongmin Lee

順推論と逆推論Junichi Chikazoe

[DL輪読会]Pay Attention to MLPs （gMLP）Deep Learning JP

The document summarizes a research paper that compares the performance of MLP-based models to Transformer-based models on various natural language processing and computer vision tasks. The key points are: 1. Gated MLP (gMLP) architectures can achieve performance comparable to Transformers on most tasks, demonstrating that attention mechanisms may not be strictly necessary. 2. However, attention still provides benefits for some NLP tasks, as models combining gMLP and attention outperformed pure gMLP models on certain benchmarks. 3. For computer vision, gMLP achieved results close to Vision Transformers and CNNs on image classification, indicating gMLP can match their data efficiency.

Introduction to YOLO detection modelWEBFARMER. ltd.

不老におけるOptunaを利用した分散ハイパーパラメータ最適化 - 今村秀明（名古屋大学 Optuna講習会）Preferred Networks

입문 Visual SLAM 14강 - 2장 Introduction to slamjdo

論文紹介: "MolGAN: An implicit generative model for small molecular graphs"Ryohei Suzuki

統計的学習理論チュートリアル: 基礎から応用まで (Ibis2012)Taiji Suzuki

[DL輪読会]逆強化学習とGANsDeep Learning JP

（文献紹介）Deep Unrolling: Learned ISTA (LISTA)Morpho, Inc.

2値ディープニューラルネットワークと組込み機器への応用: 開発中のツール紹介Hiroki Nakahara

【DL輪読会】High-Resolution Image Synthesis with Latent Diffusion ModelsDeep Learning JP

파이콘 한국 2019 튜토리얼 - LRP (Part 2)XAIC

2019년 파이콘 한국에서 진행된 튜토리얼 자료입니다. Part 2 내용으로 LRP에 대한 튜토리얼 발표자료입니다. 아래 링크를 통해 행사 관련 정보를 확인하실 수 있습니다. http://xai.unist.ac.kr/Tutorial/2018/ https://github.com/OpenXAIProject/PyConKorea2019-Tutorials Part 1: https://www.slideshare.net/OpenXAI/2019-part-1 Part 2: https://www.slideshare.net/OpenXAI/2019-lrp-part-2 Part 3: https://www.slideshare.net/OpenXAI/2019-shap-part-3

Google Cloud Dataflowによるデータ変換処理入門Takeshi Mikami

Kubernetes ネットワーキングのすべてLINE Corporation

More Related Content

What's hot (20)

안.전.제.일. 강화학습!Dongmin Lee

[DL輪読会] Residual Attention Network for Image ClassificationDeep Learning JP

파이썬과 케라스로 배우는 강화학습 저자특강Woong won Lee

Doing Deep Reinforcement learning with PPO이 의령

【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...Deep Learning JP

ディープラーニングによる時系列データの異常検知Core Concept Technologies

AlphaGo Zero 解説suckgeun lee

Maximum Entropy Reinforcement Learning (Stochastic Control)Dongmin Lee

順推論と逆推論Junichi Chikazoe

[DL輪読会]Pay Attention to MLPs （gMLP）Deep Learning JP

Introduction to YOLO detection modelWEBFARMER. ltd.

不老におけるOptunaを利用した分散ハイパーパラメータ最適化 - 今村秀明（名古屋大学 Optuna講習会）Preferred Networks

입문 Visual SLAM 14강 - 2장 Introduction to slamjdo

論文紹介: "MolGAN: An implicit generative model for small molecular graphs"Ryohei Suzuki

統計的学習理論チュートリアル: 基礎から応用まで (Ibis2012)Taiji Suzuki

[DL輪読会]逆強化学習とGANsDeep Learning JP

（文献紹介）Deep Unrolling: Learned ISTA (LISTA)Morpho, Inc.

2値ディープニューラルネットワークと組込み機器への応用: 開発中のツール紹介Hiroki Nakahara

【DL輪読会】High-Resolution Image Synthesis with Latent Diffusion ModelsDeep Learning JP

파이콘 한국 2019 튜토리얼 - LRP (Part 2)XAIC

안.전.제.일. 강화학습!Dongmin Lee

[DL輪読会] Residual Attention Network for Image ClassificationDeep Learning JP

파이썬과 케라스로 배우는 강화학습 저자특강Woong won Lee

Doing Deep Reinforcement learning with PPO이 의령

【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...Deep Learning JP

ディープラーニングによる時系列データの異常検知Core Concept Technologies

AlphaGo Zero 解説suckgeun lee

Maximum Entropy Reinforcement Learning (Stochastic Control)Dongmin Lee

順推論と逆推論Junichi Chikazoe

[DL輪読会]Pay Attention to MLPs （gMLP）Deep Learning JP

Introduction to YOLO detection modelWEBFARMER. ltd.

不老におけるOptunaを利用した分散ハイパーパラメータ最適化 - 今村秀明（名古屋大学 Optuna講習会）Preferred Networks

입문 Visual SLAM 14강 - 2장 Introduction to slamjdo

論文紹介: "MolGAN: An implicit generative model for small molecular graphs"Ryohei Suzuki

統計的学習理論チュートリアル: 基礎から応用まで (Ibis2012)Taiji Suzuki

[DL輪読会]逆強化学習とGANsDeep Learning JP

（文献紹介）Deep Unrolling: Learned ISTA (LISTA)Morpho, Inc.

2値ディープニューラルネットワークと組込み機器への応用: 開発中のツール紹介Hiroki Nakahara

【DL輪読会】High-Resolution Image Synthesis with Latent Diffusion ModelsDeep Learning JP

파이콘 한국 2019 튜토리얼 - LRP (Part 2)XAIC

Similar to Safe Reinforcement Learning (20)

Google Cloud Dataflowによるデータ変換処理入門Takeshi Mikami

Kubernetes ネットワーキングのすべてLINE Corporation

OpenStack Summit & KubeConからみるコンテナ技術の最新トレンド (更新版) - OpenStack Day Tokyo 2018講演資料VirtualTech Japan Inc.

[OpenInfra Days Korea 2018] (Track 4) - Backend.AI: 오픈소스 머신러닝 인프라 프레임워크OpenStack Korea Community

Engaging Consumers - IFAA ConferencePhillip Smith

[さるる勉強会] LTで話したい Amazon ConnectDaiki Mori

FIWARE Global Summit - FI-Lab India Stepping Stone for Implementing FIWARE Ec...FIWARE

NECTI is NEC Corporation's wholly owned subsidiary in India. It has over 1,850 employees across its offices in India as well as overseas branches. NECTI provides smart city solutions and technologies focused on areas such as water management, lighting and energy management, transport, waste management, public safety, healthcare and more. It leverages NEC's C&C platform and FIWARE-based smart city platform to integrate IoT solutions for cities.

リアルタイム議事録＆翻訳付きのビデオ会議を作ろう ~WebRTCの最新動向~ SkyWay Media Pipeline FactoryRyosuke Otsuya

APPS JAPAN 2019 専門セミナー 2019/06/12 リアルタイム議事録&翻訳付きのビデオ会議を作ろう～WebRTCの最新動向～ NTTコミュニケーションズ技術開発部担当課長大津谷亮祐 WebRTCを使えば、WebサイトやiOS・Androidアプリに、手軽にビデオ通話を組み込むことができます。最近では、WebRTCと音声認識などのクラウドAIを組み合わせる事例も出てきました。このようなWebRTCの最新動向をご紹介し、一例として、SkyWay Media Pipeline Factoryという無料サービスを使って、リアルタイム議事録&翻訳付きのビデオ会議を作る方法を解説します。

Spark MLlib ML Pipelines の概要及びpysparkからの扱い方Takeshi Mikami

36thchaptermtalupuru

The document discusses the launch of a new satellite called "#"#". The key points are: 1. The "#"#" satellite was successfully launched on [DATE] and has entered its planned orbit. 2. The satellite will be used for [PURPOSE] and is expected to provide [BENEFIT] for [DURATION]. 3. Scientists and engineers from [COUNTRY/AGENCY] worked together on this project and are pleased with the initial results from the newly launched satellite.

Db2 Warehouse v3.0 SMP 導入ガイド 20190104 Db2 Warehouse SMP v3.0 configration Ins...IBM Analytics Japan

Db2 Warehouseは分析用途に最適化されたDockerコンテナで提供されるDWHソフトウェアアプライアンスです。高速性、簡易性、柔軟性に優れた特長を持つプライベートクラウド向けデータ分析基盤です。本ガイドはDb2 Warehouse SMP(シングルノード構成)の導入手順書です。 2 Warehouse is a DWH software appliance offered in a Docker container optimized for analysis purposes. It is a data analysis infrastructure for private clouds with superior features of high speed, simplicity and flexibility. This guide is the installation procedure of Db 2 Warehouse SMP (single node configuration).

Theory and Methods for Unsupervised Anomaly Detection in Sounds Based on Deep...Yuma Koizumi

【ECCV 2018】How good is my GAN?cvpaper. challenge

ECCV 2018 サーベイ cvpaper.challenge はコンピュータビジョン分野の今を映し、創り出す挑戦です。論文読破・まとめ・アイディア考案・議論・実装・論文執筆（・社会実装）に至るまで広く取り組み、あらゆる知識を共有しています。 http://hirokatsukataoka.net/project/cc/index_cvpaperchallenge.html

Huang Chineseguest04a5c8

This summary provides the high level information from the document in 3 sentences: The document discusses the expression of FXYD proteins in primary gastric cancers and adjacent normal tissues. It analyzes the correlation of FXYD expression levels with clinicopathological features and prognosis of gastric cancer. The results showed that FXYD expression was downregulated in gastric cancer compared to normal tissues, and lower FXYD levels correlated with poorer prognosis in gastric cancer patients.

Jawsdays2018 180310Daisuke Yoshioka

The document discusses setting up a continuous integration and continuous delivery (CI/CD) pipeline for a web application using AWS services. It includes the following key steps: 1. Configure CodeCommit to store source code and trigger builds upon code changes. 2. Use CodeBuild to run tests and build/package the application. 3. Deploy builds that pass tests to S3 for static hosting or EC2 for dynamic hosting. 4. Monitor deployments and application metrics using services like CodePipeline, CodeDeploy, CloudWatch.

Amazon VPC and Amazon EC2Daiki Mori

OSC 2018 Nagoya rsyncやシェルでバックアップするよりも簡単にOSSのBaculaでバックアップしてみようKen Sawada

生徒会活動の飛躍的可能性:生徒会活動に関する調査研究と自身の経験からHiroyuki Kurimoto

Moss Jacosbon Ems Pla 182 Final Submission 12 15 08mossbmw

The document describes a training program for Emergency Medical Technicians (EMTs) in New Jersey. It provides details on the certification program, which consisted of 120 hours of classroom instruction, lectures, practical exercises, and a 10-hour observation in a hospital emergency department. The training covered all aspects of emergency medicine and prepared students to work as EMTs. Upon completion, participants received New Jersey State certification as EMTs.

ESWC 2009 Lightning TalksMichael Hausenblas

The document discusses several topics related to semantic web technologies and linked data. It mentions challenges around identity resolution, schema mapping, data fusion, and quality assessment. It also describes two tracks for a conference, one focused on applications and one focused on demonstrating scalability with a billion triples of web data. Alignments between data sources are requested to be submitted by the end of June.

Google Cloud Dataflowによるデータ変換処理入門Takeshi Mikami

Kubernetes ネットワーキングのすべてLINE Corporation

OpenStack Summit & KubeConからみるコンテナ技術の最新トレンド (更新版) - OpenStack Day Tokyo 2018講演資料VirtualTech Japan Inc.

[OpenInfra Days Korea 2018] (Track 4) - Backend.AI: 오픈소스 머신러닝 인프라 프레임워크OpenStack Korea Community

Engaging Consumers - IFAA ConferencePhillip Smith

[さるる勉強会] LTで話したい Amazon ConnectDaiki Mori

FIWARE Global Summit - FI-Lab India Stepping Stone for Implementing FIWARE Ec...FIWARE

リアルタイム議事録＆翻訳付きのビデオ会議を作ろう ~WebRTCの最新動向~ SkyWay Media Pipeline FactoryRyosuke Otsuya

Spark MLlib ML Pipelines の概要及びpysparkからの扱い方Takeshi Mikami

36thchaptermtalupuru

Db2 Warehouse v3.0 SMP 導入ガイド 20190104 Db2 Warehouse SMP v3.0 configration Ins...IBM Analytics Japan

Theory and Methods for Unsupervised Anomaly Detection in Sounds Based on Deep...Yuma Koizumi

【ECCV 2018】How good is my GAN?cvpaper. challenge

Huang Chineseguest04a5c8

Jawsdays2018 180310Daisuke Yoshioka

Amazon VPC and Amazon EC2Daiki Mori

OSC 2018 Nagoya rsyncやシェルでバックアップするよりも簡単にOSSのBaculaでバックアップしてみようKen Sawada

生徒会活動の飛躍的可能性:生徒会活動に関する調査研究と自身の経験からHiroyuki Kurimoto

Moss Jacosbon Ems Pla 182 Final Submission 12 15 08mossbmw

ESWC 2009 Lightning TalksMichael Hausenblas

More from Dongmin Lee (13)

Causal Confusion in Imitation LearningDongmin Lee

I updated the previous slides. Previous slides: https://www.slideshare.net/DongMinLee32/causal-confusion-in-imitation-learning-238882277 I reviewed the "Causal Confusion in Imitation Learning" paper. Paper link: https://papers.nips.cc/paper/9343-causal-confusion-in-imitation-learning.pdf - Abstract Behavioral cloning reduces policy learning to supervised learning by training a discriminative model to predict expert actions given observations. Such discriminative models are non-causal: the training procedure is unaware of the causal structure of the interaction between the expert and the environment. We point out that ignoring causality is particularly damaging because of the distributional shift in imitation learning. In particular, it leads to a counter-intuitive “causal misidentification” phenomenon: access to more information can yield worse performance. We investigate how this problem arises, and propose a solution to combat it through targeted interventions—either environment interaction or expert queries—to determine the correct causal model. We show that causal misidentification occurs in several benchmark control domains as well as realistic driving settings, and validate our solution against DAgger and other baselines and ablations. - Outline 1. Introduction 2. Causality and Causal Inference 3. Causality in Imitation Learning 4. Experiments Setting 5. Resolving Causal Misidentification - Causal Graph-Parameterized Policy Learning - Targeted Intervention 6. Experiments Thank you!

Character Controllers using Motion VAEsDongmin Lee

Title: Character Controllers using Motion VAEs Proceeding: ACM Transactions on Graphics (TOG) (Proc. SIGGRAPH 2020) Paper: https://dl.acm.org/doi/abs/10.1145/3386569.3392422 Video: https://www.youtube.com/watch?v=Zm3G9oqmQ4Y Given example motions, how can we generalize these to produce new purposeful motions? We take a two-step approach to this problem • Kinematic generative model based on an autoregressive conditional variational autoencoder or motion VAE (MVAE) • Learning controller to generate desired motions using Deep Reinforcement Learning (Deep RL)

Causal Confusion in Imitation LearningDongmin Lee

I reviewed the "Causal Confusion in Imitation Learning" paper. - Abstract Behavioral cloning reduces policy learning to supervised learning by training a discriminative model to predict expert actions given observations. Such discriminative models are non-causal: the training procedure is unaware of the causal structure of the interaction between the expert and the environment. We point out that ignoring causality is particularly damaging because of the distributional shift in imitation learning. In particular, it leads to a counter-intuitive “causal misidentification” phenomenon: access to more information can yield worse performance. We investigate how this problem arises, and propose a solution to combat it through targeted interventions—either environment interaction or expert queries—to determine the correct causal model. We show that causal misidentification occurs in several benchmark control domains as well as realistic driving settings, and validate our solution against DAgger and other baselines and ablations. - Outline 1. Introduction 2. Causality and Causal Inference 3. Causality in Imitation Learning 4. Experiments Setting 5. Resolving Causal Misidentification - Causal Graph-Parameterized Policy Learning - Targeted Intervention 6. Experiments Link: https://papers.nips.cc/paper/9343-causal-confusion-in-imitation-learning.pdf Thank you!

Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Va...Dongmin Lee

I reviewed the PEARL paper. PEARL (Probabilistic Embeddings for Actor-critic RL) is an off-policy meta-RL algorithm to achieve both meta-training and adaptation efficiency. It performs probabilistic encoder filtering of latent task variables to enables posterior sampling for structured and efficient exploration. Outline - Abstract - Introduction - Probabilistic Latent Context - Off-Policy Meta-Reinforcement Learning - Experiments Link: https://arxiv.org/abs/1903.08254 Thank you!

PRM-RL: Long-range Robotics Navigation Tasks by Combining Reinforcement Learn...Dongmin Lee

I reviewed the PRM-RL paper. PRM-RL (Probabilistic Roadmap-Reinforcement Learning) is a hierarchical method that combines sampling-based path planning with RL. It uses feature-based and deep neural net policies (DDPG) in continuous state and action spaces. In experiment, authors evaluate PRM- RL, both in simulation and on-robot, on two navigation tasks: end-to-end differential drive indoor navigation in office environments, and aerial cargo delivery in urban environments. Outline - Abstract - Introduction - Reinforcement Learning - Methods - Results Thank you.

Exploration Strategies in Reinforcement LearningDongmin Lee

Let's do Inverse RLDongmin Lee

안녕하세요. RL korea에서 "GAIL하자!" 라는 프로젝트를 진행했던 프로젝트 매니저 이동민이라고 합니다. 이 자료는 저희가 4개월동안 진행했던 과정들을 간략하게 소개하는 자료입니다. 저희 프로젝트는 Imitation Learning의 방법 중 하나인 "Inverse RL"에 대한 논문들의 이론적 바탕을 이해하고 이를 환경에 구현해보는 프로젝트를 진행하였습니다. 관련 논문 리스트는 다음과 같습니다. [1] AY. Ng, et al., "Algorithms for Inverse Reinforcement Learning", ICML 2000. [2] P. Abbeel, et al., "Apprenticeship Learning via Inverse Reinforcement Learning", ICML 2004. [3] ND. Ratliff, et al., "Maximum Margin Planning", ICML 2006. [4] BD. Ziebart, et al., "Maximum Entropy Inverse Reinforcement Learning", AAAI 2008. [5] J. Ho, et al., "Generative Adversarial Imitation Learning", NIPS 2016. [6] XB. Peng, et al., "Variational Discriminator Bottleneck. Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow", ICLR 2019. 프로젝트 결과로는 논문을 정리한 블로그와 논문을 구현한 Github가 있습니다. 링크는 다음과 같습니다. - 블로그 : https://reinforcement-learning-kr.github.io/2019/01/22/0_lets-do-irl-guide/ - Github : https://github.com/reinforcement-learning-kr/lets-do-irl 우리 모두 함께 IRL해요! 감사합니다 :)

모두를 위한 PG여행 가이드Dongmin Lee

안녕하세요. RL Korea에서 피지여행 프로젝트에 참여한 이동민입니다. 이 자료는 8월 25일 토요일에 제 1회 RLKorea 프로젝트 세미나에서 발표한 자료입니다. 아쉽게도 영상은 실행되지 않습니다. 세미나 링크는 아래와 같습니다. 제 1회 RLKorea 프로젝트 세미나 : https://www.facebook.com/groups/ReinforcementLearningKR/permalink/2024537701118793/?__tn__=H-R 피지여행을 간략하게 소개해드리자면, Policy Gradient와 관련된 논문들을 리뷰하여 블로그 정리하고 & 코드를 구현하여 실험하는 프로젝트입니다. 블로그와 깃허브는 다음과 같습니다. 정리 블로그 : https://reinforcement-learning-kr.github.io/…/0_pg-travel-…/ 구현 깃허브 : https://github.com/reinforcement-learning-kr/pg_travel 많은 분들이 보시고 도움이 되었으면 좋겠습니다! 감사합니다~~!!!

Planning and Learning with Tabular MethodsDongmin Lee

1) The document discusses planning methods in reinforcement learning that use models of the environment to generate simulated experiences for training. 2) It introduces Dyna-Q, an algorithm that integrates planning, acting, model learning, and direct reinforcement learning by using a model to generate additional simulated experiences for training. 3) When the model is incorrect, planning may lead to suboptimal policies, but interaction with the real environment can sometimes discover and correct modeling errors; when changes make the environment better, planning may fail to find improved policies without encouraging exploration.

Multi-armed BanditsDongmin Lee

강화학습 알고리즘의 흐름도 Part 2Dongmin Lee

강화학습의 흐름도 Part 1Dongmin Lee

강화학습의 개요Dongmin Lee

안녕하세요. 강화학습을 공부하면서 처음 접하시는 분들을 위해 ppt로 '강화학습의 개요'에 대해서 정리했습니다. 동물이 학습하는 것과 똑같이 시행착오를 겪으면서 학습하는 강화학습은 기계학습 분야에서 상당히 매력적이라고 생각합니다. https://www.youtube.com/watch?v=PQtDTdDr8vs&feature=youtu.be 위의 링크는 스키너의 쥐 실험 영상입니다. 감사합니다.

Causal Confusion in Imitation LearningDongmin Lee

Character Controllers using Motion VAEsDongmin Lee

Causal Confusion in Imitation LearningDongmin Lee

Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Va...Dongmin Lee

PRM-RL: Long-range Robotics Navigation Tasks by Combining Reinforcement Learn...Dongmin Lee

Exploration Strategies in Reinforcement LearningDongmin Lee

Let's do Inverse RLDongmin Lee

모두를 위한 PG여행 가이드Dongmin Lee

Planning and Learning with Tabular MethodsDongmin Lee

Multi-armed BanditsDongmin Lee

강화학습 알고리즘의 흐름도 Part 2Dongmin Lee

강화학습의 흐름도 Part 1Dongmin Lee

강화학습의 개요Dongmin Lee

Recently uploaded (20)

DNA Profiling and STR Typing in Forensics: From Molecular Techniques to Real-...home

This comprehensive assignment explores the pivotal role of DNA profiling and Short Tandem Repeat (STR) analysis in forensic science and genetic studies. The document begins by laying the molecular foundations of DNA, discussing its double helix structure, the significance of genetic variation, and how forensic science exploits these variations for human identification. The historical journey of DNA fingerprinting is thoroughly examined, highlighting the revolutionary contributions of Dr. Alec Jeffreys, who first introduced the concept of using repetitive DNA regions for identification. Real-world forensic breakthroughs, such as the Colin Pitchfork case, illustrate the life-saving potential of this technology. A detailed breakdown of traditional and modern DNA typing methods follows, including RFLP, VNTRs, AFLP, and especially PCR-based STR analysis, now considered the gold standard in forensic labs worldwide. The principles behind STR marker types, CODIS loci, Y-chromosome STRs, and the capillary electrophoresis (CZE) method are thoroughly explained. The steps of DNA profiling—from sample collection and amplification to allele detection using electropherograms (EPGs)—are presented in a clear and systematic manner. Beyond crime-solving, the document explores the diverse applications of STR typing: Monitoring cell line authenticity Detecting genetic chimerism Tracking bone marrow transplant engraftment Studying population genetics Investigating evolutionary history Identifying lost individuals in mass disasters Ethical considerations and potential misuse of DNA data are acknowledged, emphasizing the need for careful policy and regulation. Whether you're a biotechnology student, a forensic professional, or a researcher, this document offers an in-depth look at how DNA and STRs transform science, law, and society.

Botany-Finals-Patterns-of-Inheritance-DNA-Synthesis.pdfJseleBurgos

06-Molecular basis of transformation.pptxLanaQadumii

Quiz 3 Basic Nutrition 1ST Yearcmcmc.pptxNutriGen

Body temperature_chemical thermogenesis_hypothermia_hypothermiaMetabolic acti...muralinath2

Homeothermic animals, poikilothermic animals, metabolic activities, muscular activities, radiation of heat from environment, shivering, brown fat tissue, temperature, cinduction, convection, radiation, evaporation, panting, chemical thermogenesis, hyper pyrexia, hypothermia, second law of thermodynamics, mild hypothrtmia, moderate hypothermia, severe hypothertmia, low-grade fever, moderate=grade fever, high-grade fever, heat loss center, heat gain center

Presentatation_SM_muscle_structpes_funtionre_ty.pptxmuralinath2

Activities of smooth muscle such as cardio vascular system, digestive system, dense bodies, desmin, sarcolemma, intermediate filaments, thin filaments, connective tissue, sarco tubular system, single unit or visceral smooth muscle fiber, slow-wave rhythm, gap junctions, action potential with plateau, spike potential, pace maker waves, depolarization, repolarization, uterus, ureter, respiratory tract, multi unit smooth muscle fibers, nictitating membrane, iris of eye, arrector pili, urinary bladder, tone, electrical activity, excitatory junction potential, latent period and calcium-calnodulin complex

VERMICOMPOSTING A STEP TOWARDS SUSTAINABILITY.pptxhipachi8

Turkey Diseases and Disorders Volume 2 Infectious and Nutritional Diseases, D...Ali Raei

biochemistry amino acid from chemistry to life machinerychaitanyaa4444

Direct Evidence for r-process Nucleosynthesis in Delayed MeV Emission from th...Sérgio Sacani

The origin of heavy elements synthesized through the rapid neutron capture process (r-process) has been an enduring mystery for over half a century. J. Cehula et al. recently showed that magnetar giant flares, among the brightest transients ever observed, can shock heat and eject neutron star crustal material at high velocity, achieving the requisite conditions for an r-process.A. Patel et al. confirmed an r-process in these ejecta using detailed nucleosynthesis calculations. Radioactive decay of the freshly synthesized nuclei releases a forest of gamma-ray lines, Doppler broadened by the high ejecta velocities v  0.1c into a quasi-continuous spectrum peaking around 1 MeV. Here, we show that the predicted emission properties (light curve, fluence, and spectrum) match a previously unexplained hard gamma-ray signal seen in the aftermath of the famous 2004 December giant flare from the magnetar SGR 1806–20. This MeV emission component, rising to peak around 10 minutes after the initial spike before decaying away over the next few hours, is direct observational evidence for the synthesis of ∼10−6 Me of r-process elements. The discovery of magnetar giant flares as confirmed r-process sites, contributing at least ∼1%–10% of the total Galactic abundances, has implications for the Galactic chemical evolution, especially at the earliest epochs probed by low-metallicity stars. It also implicates magnetars as potentially dominant sources of heavy cosmic rays. Characterization of the r-process emission from giant flares by resolving decay line features offers a compelling science case for NASA’s forthcomingCOSI nuclear spectrometer, as well as next-generation MeV telescope missions.

Metallurgical process class 11_Govinda PathakGovindaPathak6

Examining Visual Attention in Gaze-Driven VR Learning: An Eye-Tracking Study ...Yasasi Abeysinghe

This study presents an eye-tracking user study for analyzing visual attention in a gaze-driven VR learning environment using a consumer-grade Meta Quest Pro VR headset. Eye tracking data were captured through the headset's built-in eye tracker. We then generated basic and advanced eye-tracking measures—such as fixation duration, saccade amplitude, and the ambient/focal attention coefficient K—as indicators of visual attention within the VR setting. The generated gaze data are visualized in an advanced gaze analytics dashboard, enabling us to assess users' gaze behaviors and attention during interactive VR learning tasks. This study contributes by proposing a novel approach for integrating advanced eye-tracking technology into VR learning environments, specifically utilizing consumer-grade head-mounted displays.

Keynote presentation at DeepTest Workshop 2025Shiva Nejati

Skin_Glands_Structure_Secretion _Controlmuralinath2

Polytene chromosomes. A Practical Lecture.pptxDr Showkat Ahmad Wani

Chromatography, types, techniques, ppt.pptxDr Showkat Ahmad Wani

Nutritional Diseases in poultry.........Bangladesh Agricultural University,Mymemsingh

Poultry require at least 38 dietary nutrients inappropriate concentrations for a balanced diet. A nutritional deficiency may be due to a nutrient being omitted from the diet, adverse interaction between nutrients in otherwise apparently well-fortified diets, or the overriding effect of specific anti-nutritional factors. Major components of foods are – Protein, Fats, Carbohydrates, Minerals, Vitamins Vitamins are A- Fat soluble vitamins: A, D, E, and K ; B - Water soluble vitamins: Thiamin (B1), Riboflavin (B2), Nicotinic acid (niacin), Pantothenic acid (B5), Biotin, folic acid, pyriodxin and cholin. Causes: Low levels of vitamin A in the feed. oxidation of vitamin A in the feed, errors in mixing and inter current disease, e.g. coccidiosis , worm infestation Clinical signs: Lacrimation (ocular discharge), White cheesy exudates under the eyelids (conjunctivitis). Sticky of eyelids and (xerophthalmia). Keratoconjunctivitis. Watery discharge from the nostrils. Sinusitis. Gasping and sneezing. Lack of yellow pigments, Respiratory sings due to affection of epithelium of the respiratory tract. Lesions: Pseudo diphtheritic membrane in digestive and respiratory system (Keratinized epithelia). Nutritional roup: respiratory sings due to affection of epithelium of the respiratory tract. Pustule like nodules in the upper digestive tract (buccal cavity, pharynx, esophagus). The urate deposits may be found on other visceral organs Treatment: Administer 3-5 times the recommended levels of vitamin A @ 10000 IU/ KG ration either through water or feed. Lesions: Pseudo diphtheritic membrane in digestive and respiratory system (Keratinized epithelia). Nutritional roup: respiratory sings due to affection of epithelium of the respiratory tract. Pustule like nodules in the upper digestive tract (buccal cavity, pharynx, esophagus). The urate deposits may be found on other visceral organs Treatment: Administer 3-5 times the recommended levels of vitamin A @ 10000 IU/ KG ration either through water or feed. Lesions: Pseudo diphtheritic membrane in digestive and respiratory system (Keratinized epithelia). Nutritional roup: respiratory sings due to affection of epithelium of the respiratory tract. Pustule like nodules in the upper digestive tract (buccal cavity, pharynx, esophagus). The urate deposits may be found on other visceral organs Treatment: Administer 3-5 times the recommended levels of vitamin A @ 10000 IU/ KG ration either through water or feed.

Causes of mortalities of eggs and spawn and remedies.pptxanshumanmohanty9090

RAPID DIAGNOSTIC TEST (RDT) overviewppt.pptxnietakam

Influenza-Understanding-the-Deadly-Virus.pptxdiyapadhiyar

DNA Profiling and STR Typing in Forensics: From Molecular Techniques to Real-...home

Botany-Finals-Patterns-of-Inheritance-DNA-Synthesis.pdfJseleBurgos

06-Molecular basis of transformation.pptxLanaQadumii

Quiz 3 Basic Nutrition 1ST Yearcmcmc.pptxNutriGen

Body temperature_chemical thermogenesis_hypothermia_hypothermiaMetabolic acti...muralinath2

Presentatation_SM_muscle_structpes_funtionre_ty.pptxmuralinath2

VERMICOMPOSTING A STEP TOWARDS SUSTAINABILITY.pptxhipachi8

Turkey Diseases and Disorders Volume 2 Infectious and Nutritional Diseases, D...Ali Raei

biochemistry amino acid from chemistry to life machinerychaitanyaa4444

Direct Evidence for r-process Nucleosynthesis in Delayed MeV Emission from th...Sérgio Sacani

Metallurgical process class 11_Govinda PathakGovindaPathak6

Examining Visual Attention in Gaze-Driven VR Learning: An Eye-Tracking Study ...Yasasi Abeysinghe

Keynote presentation at DeepTest Workshop 2025Shiva Nejati

Skin_Glands_Structure_Secretion _Controlmuralinath2

Polytene chromosomes. A Practical Lecture.pptxDr Showkat Ahmad Wani

Chromatography, types, techniques, ppt.pptxDr Showkat Ahmad Wani

Nutritional Diseases in poultry.........Bangladesh Agricultural University,Mymemsingh

Causes of mortalities of eggs and spawn and remedies.pptxanshumanmohanty9090

RAPID DIAGNOSTIC TEST (RDT) overviewppt.pptxnietakam

Influenza-Understanding-the-Deadly-Virus.pptxdiyapadhiyar

Safe Reinforcement Learning

1. 02 , 1 2 202 A9 8

2. d • m S rS • 7 K a . h i Ob 7P , N . O y eo P , N . O AEGB I AFAG 7A IGEGC. G G I K E GP LIE AG E I K G G G IAL 4 0 I , - N . ug ! • 7 69 2 wn ~ t 2 839 12 f ::9 w R c a

3. 2 . 4 2 1 2 . 4 2 1 242 2 2 2 3 2 . 2 . 4 2 1 32

6. : : A@ ?I 7 . 3/ / - -

7. i ) n ( /ooc wu r p h i n ( 7?? ( @:37 3 ? :? h [s .? ? t 3? : e :?b b g [sd [ - A k :? h R I I [ t ] :? [ a i S

8. i ) n ( /ooc wu r p h i n ( 7?? ( @:37 3 ? :? h [s .? ? t 3? : e :?b b g [sd [ - A k :? h R I I [ t ] :? [ a i S

9. i ) n ( /ooc wu r p h i n ( 7?? ( @:37 3 ? :? h [s .? ? t 3? : e :?b b g [sd [ - A k :? h R I I [ t ] :? [ a i S

10. i ) n ( /ooc wu r p h i n ( 7?? ( @:37 3 ? :? h [s .? ? t 3? : e :?b b g [sd [ - A k :? h R I I [ t ] :? [ a i S

12. dp ) ( u a Ou a A R O : ( S i c O A w k y ( : d c lr e G x t u a! . B ( P i : On g y m do u ( uo - n ! + ? ,

13. dp ) ( u a Ou a A R O : ( S i c O A w k y ( : d c lr e G x t u a! . B ( P i : On g y m do u ( uo - n ! + ? ,

14. dp ) ( u a Ou a A R O : ( S i c O A w k y ( : d c lr e G x t u a! . B ( P i : On g y m do u ( uo - n ! + ? ,

15. dp ) ( u a Ou a A R O : ( S i c O A w k y ( : d c lr e G x t u a! . B ( P i : On g y m do u ( uo - n ! + ? ,

16. dp ) ( u a Ou a A R O : ( S i c O A w k y ( : d c lr e G x t u a! . B ( P i : On g y m do u ( uo - n ! + ? ,

20. - / 8.4 1 20 4/ - 4 - 0/ :

23. ? ?

24. !

28. !

30. .

31. .

37. !

39. / / . / . / / ./ . / / / / / - . / -

40. -, , /, , , - /, , , / / , , , , , , - , , , -, . /, , .

41. L L L

42. L L L !" = $"%& + ($"%) + () $"%* + ⋯ = ∑-./ 0 (- $"%-%&

43. L L L

44. L L L

49. -1 C E - 2 .2 2 - 2 2 2 - 2 2 -2 2 2

50. -1 C E - 2 .2 2 - 2 2 2 - 2 2 -2 2 2

51. -1 C E - 2 .2 2 - 2 2 2 - 2 2 -2 2 2

52. - 2 E BI C 22 . 2 2 21 2 2 2 2

53. - 2 E BI C 22 . 2 2 21 2 2 2 2

54. . - R C E 1B 2 - P O SLO I W I!

55. - 2 E BI C 22 . 2 2 21 2 2 2 2

58. !

59. -

60. - - - - - - - - - -

61. - - - - - - - - - - - - - - -

62. -

63. -

64. - - - - -

65. Ω" " #$, &$, #', &', … C - C

66. - - - - - C !

67. - C

68. - C

69. - C

70. - C

71. - C

73. -

74. id , - h , ,., c e ., i , ,., Wk , ,., , l , ,., h h a . Ch . . Ch , , b

75. id , - c e - i - Wk C - l . - a - - . - . . b

76. id , - c e - i - Wk C - l . - a - - . - . . b

78. Worst case !"# $(&') 100 0 &' !"# High Reward

79. Worst case !"# $(&') 100 0 &' !"# High Reward

80. Worst case !"# $(&') 100 0 &' !"# High Reward

81. Worst case !"# $(&') 100 0 &' !"# High Reward

84. !

89. C ! "# $ ! %# & '#

90. C ! "# $ ! %# & '#

91. C ! "# $ ! %# & '#

92. C ! "# $ ! %# & '#

93. ! C

95. .

96. NO P I K TS /. BC :? /? :5 . C:=: C:? C B 5 / : P R TS CC B ? C 5?= C5 ( / )5

97. O N T KR S ! / B C ? = / I ? ? : C B : B P : C 5 :) ( .

98. a P T cK eO aR aI b a Sd N /. BC :? /? :5 . C:=: C:? C B 5 / : CC B ? C 5?= C5 ( / )5

99. OPS S T KI b a N /. BC :? /? :5 . C:=: C:? C B 5 / : R CC B ? C 5?= C5 ( / )5

100. O b NT S c I dP bK a /. BC :? /? :5 . C:=: C:? C B 5 / : R a CC B ? C 5?= C5 ( / )5

101. N e T O P dR N K e b dR ! ca / B C ? = / I ? ? : C B : B S ca : C 5 :) ( .

102. N !!! PO / B C ? = / I ? ? : C B : B K PO : C 5 :) ( .

103. .

104. E E A 8 !

108. !

109. - 2 E BI C 22 2 2 2 . 2 2 21 2

110. . E . K L L K

111. .

112. P ! 2:11 1 3 L D M a ./: 3 9 5 1 3/:1 71 72- 7 11 9/:

113. . R . E

114. . .

115. .

117. ! !

121. ! !

125. ! !

127. . 2 . 2 1

128. . 2 1

129. . A 12 A 12 ?

130. - , . 1 21 ? -: 2 - - , ( 1- - - , ( ) )

131. ? . 1 2

132. 1 ? 2 . .

133. . 2 1

134. ) ( 2 . 2 2 1 1 : !", $", !%, $%, … E ) T?

136. : / . : / A :: : . : :

137. . . : . . . . . / . . : . A :: : / . .. : : . . : . : . .

138. ! = { $%, '% , $(, '( , … }

139. ! = { $%, '% , $(, '( , … }

140. !(#, %)' = { #*, %* , #+, %+ , … }

141. !(#, %)' = { #*, %* , #+, %+ , … }

142. !(#, %)' = { #*, %* , #+, %+ , … } .

143. ? ML ? 7 C7 7 ? 7 7? 7 ? ? 4 C7 ? = 7 I ML : C H 47 : . / 2 7 7 H 47 &C

146. O Z I L N E 8 . : . : . :/: ?= 8. = AAA :? ?/ :8 A.

147. 8 & /= = : ? = : E : 8 E 8 8 8= H & 7 & E= = 7 ?. ? E=

148. A SR 1//8 0 (& ) . G ADA G:G H IKGG D :HN LA= P Q SR GI MMM N K K D M: L )2=.M8 0 ? : KH N K K D((I

150. !" #"

151. !" #"

152. ! "# $#

153. ! "# $# !($#|"#)

154. .

155. .

156. .

158. !

159. .

160. , ) . ( ) C

161. L D !

162. :CDCNC ACO 1CCM :CG DLNAC C P 5C N G 1LCO P LNH CP IL 5G H 0L MNC C OGRC NRCT L DC :CG DLNAC C P 5C N G ( ' 9 MCN 5G H N OP :C GL 9LIGAT 8MPG G PGL ( ' 9 MCN 5G H :98 N OP :C GL 9LIGAT 8MPG G PGL - 4 BCMP :COC NA 9 MCN :CRGCS L C 5G H I LNGP O DLN 4 RCNOC :CG DLNAC C P 5C N G ( 9 MCN 5G H MMNC PGACO GM 5C N G RG 4 RCNOC :CG DLNAC C P 5C N G ( ) 9 MCN 5G H 9: (,- MMNC PGACO GM 5C N G RG 4 RCNOC :CG DLNAC C P 5C N G 99 5G H DC IPG C P :CG DLNAC C P 5C N G DLN PL L L O 1NGRG 99 5G H 4 PNLB APGL LD 4 RCNOC :CG DLNAC C P 5C N G 99 5G H 4 RCNOC :CG DLNAC C P 5C N G OCB L 0NGPGA I P PC L C 5G H 4 GP PGL 5C N G DLN 8 NL LP OG 4 PC L C 5G H :L LP IC N O PL MI T P IC PC GO T G GP PGL L C 5G H 422: 9 ( ' - 1CCM G GA M MCN O MMIC C P NT RGBCL L C 5G H

164. !

165. !

Safe Reinforcement Learning

Recommended

More Related Content

What's hot (20)

Similar to Safe Reinforcement Learning (20)

More from Dongmin Lee (13)

Recently uploaded (20)

Safe Reinforcement Learning