強化学習
出典: フリー百科事典『ウィキペディア(Wikipedia)』 (2024/01/07 06:46 UTC 版)
機械学習および データマイニング |
---|
Category:データマイニング |
強化学習(きょうかがくしゅう、英: reinforcement learning、RL)は、ある環境内における知的エージェントが、現在の状態を観測し、得られる収益(累積報酬)を最大化するために、どのような行動をとるべきかを決定する機械学習の一分野である。強化学習は、教師あり学習、教師なし学習と並んで、3つの基本的な機械学習パラダイムの一つである。
強化学習が教師あり学習と異なる点は、ラベル付きの入力/出力の組を提示する必要がなく、最適でない行動を明示的に修正する必要もない。その代わり、未知の領域の探索と、現在の知識の活用の間のバランスを見つけることに重点が置かれる[1]。
この文脈の強化学習アルゴリズムの多くは動的計画法を使用するため、この環境は通常マルコフ決定過程(MDP)として定式化される[2]。古典的な動的計画法と強化学習アルゴリズムとの主な違いは、後者はMDPの正確な数学的モデルの知識を必要とせず、正確な方法では実行不可能な大規模MDPを対象にできることである。代表的なアルゴリズムとして時間差分学習(TD学習)やQ学習が知られている。
- ^ Kaelbling, Leslie P.; Littman, Michael L.; Moore, Andrew W. (1996). “Reinforcement Learning: A Survey”. Journal of Artificial Intelligence Research 4: 237–285. arXiv:cs/9605103. doi:10.1613/jair.301. オリジナルの2001-11-20時点におけるアーカイブ。 .
- ^ van Otterlo, M.; Wiering, M. (2012). Reinforcement learning and markov decision processes. Adaptation, Learning, and Optimization. 12. 3–42. doi:10.1007/978-3-642-27645-3_1. ISBN 978-3-642-27644-6
- ^ Russell, Stuart J.; Norvig, Peter (2010). Artificial intelligence : a modern approach (Third ed.). Upper Saddle River, New Jersey. pp. 830, 831. ISBN 978-0-13-604259-4
- ^ Lee, Daeyeol; Seo, Hyojung; Jung, Min Whan (21 July 2012). “Neural Basis of Reinforcement Learning and Decision Making”. Annual Review of Neuroscience 35 (1): 287–308. doi:10.1146/annurev-neuro-062111-150512. PMC 3490621. PMID 22462543 .
- ^ Xie, Zhaoming, et al. "ALLSTEPS: Curriculum‐driven Learning of Stepping Stone Skills." Computer Graphics Forum. Vol. 39. No. 8. 2020.
- ^ Sutton & Barto 1998, Chapter 11.
- ^ Gosavi, Abhijit (2003). Simulation-based Optimization: Parametric Optimization Techniques and Reinforcement. Operations Research/Computer Science Interfaces Series. Springer. ISBN 978-1-4020-7454-7
- ^ a b Burnetas, Apostolos N.; Katehakis, Michael N. (1997), “Optimal adaptive policies for Markov Decision Processes”, Mathematics of Operations Research 22: 222–255, doi:10.1287/moor.22.1.222
- ^ Tokic, Michel; Palm, Günther (2011), “Value-Difference Based Exploration: Adaptive Control Between Epsilon-Greedy and Softmax”, KI 2011: Advances in Artificial Intelligence, Lecture Notes in Computer Science, 7006, Springer, pp. 335–346, ISBN 978-3-642-24455-1
- ^ a b “Reinforcement learning: An introduction”. 2023年5月12日閲覧。
- ^ Sutton, Richard S. (1984). Temporal Credit Assignment in Reinforcement Learning (PhD thesis). University of Massachusetts, Amherst, MA.
- ^ Sutton & Barto 1998, §6. Temporal-Difference Learning.
- ^ Bradtke, Steven J.; Barto, Andrew G. (1996). “Learning to predict by the method of temporal differences”. Machine Learning 22: 33–57. doi:10.1023/A:1018056104778.
- ^ Watkins, Christopher J.C.H. (1989). Learning from Delayed Rewards (PDF) (PhD thesis). King’s College, Cambridge, UK.
- ^ Matzliach, Barouch; Ben-Gal, Irad; Kagan, Evgeny (2022). “Detection of Static and Mobile Targets by an Autonomous Agent with Deep Q-Learning Abilities”. Entropy 24 (8): 1168. Bibcode: 2022Entrp..24.1168M. doi:10.3390/e24081168. PMC 9407070. PMID 36010832 .
- ^ Williams, Ronald J. (1987). "A class of gradient-estimating algorithms for reinforcement learning in neural networks". Proceedings of the IEEE First International Conference on Neural Networks. CiteSeerX 10.1.1.129.8871。
- ^ Peters, Jan; Vijayakumar, Sethu; Schaal, Stefan (2003). "Reinforcement Learning for Humanoid Robotics" (PDF). IEEE-RAS International Conference on Humanoid Robots.
- ^ Juliani, Arthur (2016年12月17日). “Simple Reinforcement Learning with Tensorflow Part 8: Asynchronous Actor-Critic Agents (A3C)”. Medium. 2018年2月22日閲覧。
- ^ Deisenroth, Marc Peter; Neumann, Gerhard; Peters, Jan (2013). A Survey on Policy Search for Robotics. Foundations and Trends in Robotics. 2. NOW Publishers. pp. 1–142. doi:10.1561/2300000021. hdl:10044/1/12051
- ^ Sutton, Richard (1990). "Integrated Architectures for Learning, Planning and Reacting based on Dynamic Programming". Machine Learning: Proceedings of the Seventh International Workshop.
- ^ Lin, Long-Ji (1992). "Self-improving reactive agents based on reinforcement learning, planning and teaching" (PDF). Machine Learning volume 8. doi:10.1007/BF00992699。
- ^ van Hasselt, Hado; Hessel, Matteo; Aslanides, John (2019). "When to use parametric models in reinforcement learning?" (PDF). Advances in Neural Information Processing Systems 32.
- ^ “On the Use of Reinforcement Learning for Testing Game Mechanics : ACM - Computers in Entertainment” (英語). cie.acm.org. 2018年11月27日閲覧。
- ^ Riveret, Regis; Gao, Yang (2019). “A probabilistic argumentation framework for reinforcement learning agents” (英語). Autonomous Agents and Multi-Agent Systems 33 (1–2): 216–274. doi:10.1007/s10458-019-09404-2.
- ^ Yamagata, Taku; McConville, Ryan; Santos-Rodriguez, Raul (16 November 2021). "Reinforcement Learning with Feedback from Multiple Humans with Diverse Skills". arXiv:2111.08596 [cs.LG]。
- ^ Kulkarni, Tejas D.; Narasimhan, Karthik R.; Saeedi, Ardavan; Tenenbaum, Joshua B. (2016). “Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation”. Proceedings of the 30th International Conference on Neural Information Processing Systems. NIPS'16 (USA: Curran Associates Inc.): 3682–3690. arXiv:1604.06057. Bibcode: 2016arXiv160406057K. ISBN 978-1-5108-3881-9 .
- ^ “Reinforcement Learning / Successes of Reinforcement Learning”. umichrl.pbworks.com. 2017年8月6日閲覧。
- ^ Quested, Tony. “Smartphones get smarter with Essex innovation”. Business Weekly. 2021年6月17日閲覧。
- ^ Dey, Somdip; Singh, Amit Kumar; Wang, Xiaohang; McDonald-Maier, Klaus (March 2020). “User Interaction Aware Reinforcement Learning for Power and Thermal Efficiency of CPU-GPU Mobile MPSoCs”. 2020 Design, Automation Test in Europe Conference Exhibition (DATE): 1728–1733. doi:10.23919/DATE48585.2020.9116294. ISBN 978-3-9819263-4-7 .
- ^ Williams, Rhiannon (2020年7月21日). “Future smartphones 'will prolong their own battery life by monitoring owners' behaviour'” (英語). i. 2021年6月17日閲覧。
- ^ Kaplan, F.; Oudeyer, P. (2004). “Maximizing learning progress: an internal reward system for development”. In Iida, F.; Pfeifer, R.; Steels, L. et al.. Embodied Artificial Intelligence. Lecture Notes in Computer Science. 3139. Berlin; Heidelberg: Springer. pp. 259–270. doi:10.1007/978-3-540-27833-7_19. ISBN 978-3-540-22484-6
- ^ Klyubin, A.; Polani, D.; Nehaniv, C. (2008). “Keep your options open: an information-based driving principle for sensorimotor systems”. PLOS ONE 3 (12): e4018. Bibcode: 2008PLoSO...3.4018K. doi:10.1371/journal.pone.0004018. PMC 2607028. PMID 19107219 .
- ^ Barto, A. G. (2013). “Intrinsic motivation and reinforcement learning”. Intrinsically Motivated Learning in Natural and Artificial Systems. Berlin; Heidelberg: Springer. pp. 17–47
- ^ Dabérius, Kevin; Granat, Elvin; Karlsson, Patrik (2020). “Deep Execution - Value and Policy Based Reinforcement Learning for Trading and Beating Market Benchmarks”. The Journal of Machine Learning in Finance 1. SSRN 3374766.
- ^ George Karimpanal, Thommen; Bouffanais, Roland (2019). “Self-organizing maps for storage and transfer of knowledge in reinforcement learning” (英語). Adaptive Behavior 27 (2): 111–126. arXiv:1811.08318. doi:10.1177/1059712318818568. ISSN 1059-7123.
- ^ Soucek, Branko (6 May 1992). Dynamic, Genetic and Chaotic Programming: The Sixth-Generation Computer Technology Series. John Wiley & Sons, Inc. p. 38. ISBN 0-471-55717-X
- ^ Francois-Lavet, Vincent (2018). “An Introduction to Deep Reinforcement Learning”. Foundations and Trends in Machine Learning 11 (3–4): 219–354. arXiv:1811.12560. Bibcode: 2018arXiv181112560F. doi:10.1561/2200000071.
- ^ Mnih, Volodymyr (2015). “Human-level control through deep reinforcement learning”. Nature 518 (7540): 529–533. Bibcode: 2015Natur.518..529M. doi:10.1038/nature14236. PMID 25719670 .
- ^ Goodfellow, Ian; Shlens, Jonathan; Szegedy, Christian (2015). “Explaining and Harnessing Adversarial Examples”. International Conference on Learning Representations. arXiv:1412.6572.
- ^ Behzadan, Vahid; Munir, Arslan (2017). “Vulnerability of Deep Reinforcement Learning to Policy Induction Attacks”. International Conference on Machine Learning and Data Mining in Pattern Recognition. Lecture Notes in Computer Science 10358: 262–275. arXiv:1701.04143. doi:10.1007/978-3-319-62416-7_19. ISBN 978-3-319-62415-0.
- ^ Pieter, Huang, Sandy Papernot, Nicolas Goodfellow, Ian Duan, Yan Abbeel (2017-02-07). Adversarial Attacks on Neural Network Policies. OCLC 1106256905
- ^ Korkmaz, Ezgi (2022). “Deep Reinforcement Learning Policies Learn Shared Adversarial Features Across MDPs.”. Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI-22) 36 (7): 7229–7238. doi:10.1609/aaai.v36i7.20684.
- ^ Berenji, H.R. (1994). “Fuzzy Q-learning: a new approach for fuzzy dynamic programming”. Proc. IEEE 3rd International Fuzzy Systems Conference (Orlando, FL, USA: IEEE): 486–491. doi:10.1109/FUZZY.1994.343737. ISBN 0-7803-1896-X .
- ^ Vincze, David (2017). “Fuzzy rule interpolation and reinforcement learning”. 2017 IEEE 15th International Symposium on Applied Machine Intelligence and Informatics (SAMI). IEEE. pp. 173–178. doi:10.1109/SAMI.2017.7880298. ISBN 978-1-5090-5655-2
- ^ Ng, A. Y.; Russell, S. J. (2000). “Algorithms for Inverse Reinforcement Learning”. Proceeding ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning. pp. 663–670. ISBN 1-55860-707-2
- ^ García, Javier; Fernández, Fernando (1 January 2015). “A comprehensive survey on safe reinforcement learning”. The Journal of Machine Learning Research 16 (1): 1437–1480 .
強化学習と同じ種類の言葉
- 強化学習のページへのリンク