Abstract
How can a baby humanoid robot, confronted with raw-pixel high-dimensional visual input streams, learn meaningful environment representations through autonomous, real-time interactions, in the absence of a teacher? How can it leverage these learned representations to build a repertoire of skills? We present recently developed techniques and systems to tackle this problem, which share the common feature of exploitation of the slowness principle, via Slow Feature Analysis (SFA).
We show how incremental and modular versions of SFA can be combined with curiosity-driven agents to enable autonomous skill acquisition. Curiosity-driven agents use reinforcement learning (RL) to quickly adapt control policies to maximize intrinsic reward (the measurable improvement in the compressor or predictor or world-model). In our case, the learning progress of the slow features is measured and used as this intrinsic reward. An agent can combine curiosity and SFA to autonomously learn behaviors, such as toppling or grasping an object, from high-dimensional image data in an unsupervised and open-ended manner.
Outline
Slowness Learning and Slow Feature Analysis (SFA)
SFA and Reinforcement Learning
Results on the iCub
Tutorial WCCI Proposal (PDF): Link
Tutorial Slides (PDF): Link
References
1. V. R. Kompella, M. Luciw and J. Schmidhuber. "Incremental Slow Feature Analysis", 22nd International Joint Conference on Artificial Intelligence (IJCAI), Barcelona, 2011. Link to paper.
2. V. R. Kompella, M. Luciw and J. Schmidhuber. "Incremental Slow Feature Analysis: Adaptive Low-Complexity Slow Feature Updating from High-Dimensional Input Streams", Neural Computation Journal, Vol. 24 (11), pp. 2994--3024, 2012. Link to preprint.
3. V. R. Kompella, L. Pape, J. Masci, M. Frank and J. Schmidhuber. "AutoIncSFA and Vision-based Developmental Learning for Humanoid Robots" , 11th IEEE-RAS International Conference on Humanoid Robots (Humanoids), Bled, Slovenia, 2011. Link to paper. Link to Video.
4. M. Luciw and J. Schmidhuber. "Low Complexity Proto-Value Function Learning from Sensory Observations with Incremental Slow Feature Analysis" , In Proceedings of the International Conference on Artificial Neural Networks (ICANN, Lausanne), 2012. Link to paper.
5. M. Luciw, V. R. Kompella and J. Schmidhuber. "Hierarchical Incremental Slow Feature Analysis" , Workshop on Deep Hierarchies in Vision (DHV, Vienna), 2012. Link to poster.
6. V. R. Kompella, M. Luciw, M. Stollenga, L. Pape and J. Schmidhuber. "Autonomous Learning of Abstractions using Curiosity-Driven Modular Incremental Slow Feature Analysis. (Curious Dr. MISFA)" , IEEE International Conference on Developmental and Learning and Epigenetic Robotics (ICDL-EpiRob), San Diego, 2012. Link to paper. Link to Video.
7. M. Luciw*, V. R. Kompella*, S. Kazerounian and J. Schmidhuber. "An intrinsic value system for developing multiple invariant representations with incremental slowness learning", Frontiers in Neurorobotics, Vol. 7 (9), 2013. *Joint first authors. Link to paper.
8. V. R. Kompella, M. Stollenga, M. Luciw and J. Schmidhuber. "Explore to See, Learn to Perceive, Get the Actions for Free: SKILLABILITY," to Appear in the Proc. of IEEE International Joint Conference on Neural Networks (IJCNN), Beijing, 2014. Link to paper.
