BRAIN-CONTROLLED INTELLIGENT ROBOTIC DEVICES
Brain-controlled prosthetic devices are a powerful tool for allowing paralyzed or otherwise bodily disabled individuals to regain their freedom to interact naturally with the world. Yet as the systems to get controlled become more and more complex, from single legs or arms over mobile robotic platforms to humanoid surrogates, ways to deal with the increased cognitive load and facilitate control will need to be found. The central idea of this project is that by enhancing a prosthetic device with a certain degree of autonomy and adaptivity, control is possible on a higher cognitive level rather than on the lower level of raw motor signals. Therefore, the aim of this interdisciplinary joint project was to develop methods and technologies for brain- controlled intelligent autonomous robotic devices like robot arms or mobile platforms. This made it an important building block and demonstrator of LiNC, which is one of the two development platforms within BrainLinks-BrainTools. Ultimately, work in the project was targeting solutions that would grant disabled individuals an increased quality of life and restore some amount of freedom to them. Specific tasks tackled during the project included fetching a specific item, assisting in pouring liquids, and with drinking. Many aspects of a brain-controlled robotic system had to be improved for this purpose.
During the project, we achieved substantial success in the following areas: Large-scale data collection for improved training of deep neural network based brain-signal classifiers, evaluation of novel convolutional neural network (CNN) architectures for brain-signal decoding, includ- ing online adaptation and the comparison to other state-of-the-art (SOA) methods, showing en par or even improved performance for offline and new SOA for online decoding, SOA error- related negativity (ERN) detection, novel results for related activity in the EEG high gamma band, implementation and evaluation of neural language models for later use in speech-related brain-signal decoding, experiments with combined recording of brain signals and subjective task-evaluation signals for robot trajectory planning within the user’s comfort zone, progress in the robotics aspects of the project (BI2RRT , task space PRM planner, liquid level detection, pouring trajectory learning) as well as planning aspects on different levels, including GUI inte- gration, foundations of safety envelope evaluation, integration of the different components for the demonstrator in a ROS-based architecture, demonstration of the usefulness of the integrated system in various online, EEG-controlled robot-assisted application. In summary, the NeuroBots project was successful in laying the foundations for future shared-control BCI research and ap- plications within the BrainLinks-BrainTools framework for cognitive control of assistive systems with improved efficacy compared to existing attempts.
[1] J. Aldinger, R. Mattmüller, and M. Göbelbecker. “Complexity Issues of Interval Relaxed Numeric Planning”. In: 38th German Conference on Artificial Intelligence (KI 2015). 2015.
[2] J. Aldinger and B. Nebel. “Interval Based Relaxation Heuristics for Numeric Planning with Action Costs”. In: Proceedings of the Tenth International Symposium on Combinatorial Search (SoCS 2017). 2017.
[3] J. Aldinger and B. Nebel. “Interval Based Relaxation Heuristics for Numeric Planning with Action Costs”. en. In: Proceedings of the 40th German Conference on Artificial Intelligence (KI 2017). Vol. 10505. Cham: Springer International Publishing, 2017, pp. 15–28. isbn: 978-3-319-67189-5 978-3-319-67190-1. doi: 10.1007/978-3-319-67190-1_2.
[4] A. Amiranashvili, A. Dosovitskiy, V. Koltun, and T. Brox. “Motion Perception in Rein- forcement Learning with Dynamic Objects”. en. In: Conference on Robot Learning (CoRL). 2018.
[5] A. Amiranashvili, A. Dosovitskiy, V. Koltun, and T. Brox. “TD or not TD: Analyzing the Role of Temporal Differencing in Deep Reinforcement Learning”. en. In: International Conference on Learning Representations (ICLR). 2018.
[6] J. Behncke, R. T. Schirrmeister, W. Burgard, and T. Ball. “The signature of robot ac- tion success in EEG signals of a human observer: Decoding and visualization using deep convolutional neural networks”. In: 2018 6th International Conference on Brain-Computer Interface (BCI). Jan. 2018, pp. 1–6. doi: 10.1109/IWW-BCI.2018.8311531.
[7] J. Behncke, R. T. Schirrmeister, M. Völker, A. Schulze-Bonhage, W. Burgard, and T. Ball. “Cross-paradigm pretraining of convolutional networks improves intracranial EEG decoding”. In: 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC). arXiv: 1806.09532. June 2018.
[8] J. Behncke, R. T. Schirrmeister, W. Burgard, and T. Ball. “The Role of Robot Design in Decoding Error-related Information from EEG Signals of a Human Observer”. In: 6th In- ternational Congress on Neurotechnology, Electronics and Informatics. Oct. 2018, pp. 61– 66. isbn: 978-989-758-326-1.
[9] J. Boedecker, J. T. Springenberg, J. Wülfing, and M. Riedmiller. “Approximate real-time optimal control based on sparse gaussian process models”. In: 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL). IEEE. 2014, pp. 1–8.
[10] F. Burget and M. Bennewitz. “Stance Selection for Humanoid Grasping Tasks by In- verse Reachability Maps”. In: IEEE International Conference on Robotics and Automation (ICRA). Washington, Seattle, USA, 2015.
[11] F. Burget, M. Bennewitz, and W. Burgard. “BI2RRT ∗: An Efficient Sampling-Based Path Planning Framework for Task-Constrained Mobile Manipulation”. In: IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS). Daejeon, South Korea, 2016.
[12] F. Burget, A. Hornung, and M. Bennewitz. “Whole-Body Motion Planning for Manipula- tion of Articulated Objects”. In: IEEE International Conference on Robotics and Automa- tion (ICRA). Karlsruhe, Germany, 2013.
[13] F. Burget, L. D. J. Fiederer, D. Kuhner, M. Völker, J. Aldinger, R. T. Schirrmeister, C. Do, J. Boedecker, B. Nebel, T. Ball, et al. “Acting thoughts: Towards a mobile robotic service assistant for users with limited communication skills”. In: European Conference on Mobile Robots (ECMR). IEEE. 2017, pp. 1–6.
[14] C. Do and W. Burgard. “Accurate Pouring with an Autonomous Robot Using an RGB- D Camera”. In: The 15th International Conference on Intelligent Autonomous Systems (IAS). Baden Baden, Germany, 2018.
[15] C. Do, C. Gordillo, and W. Burgard. “Learning to Pour using Deep Deterministic Pol- icy Gradients”. In: Proceedings of the International Conference on Intelligent Robots and Systems (IROS). Madrid, Spain, 2018.
[16] C. Do, T. Schubert, and W. Burgard. “A Probabilistic Approach to Liquid Level Detection in Cups Using an RGB-D Camera”. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Daejeon, South Korea, 2016.
[17] M. Göbelbecker. “Assisting With Goal Formulation for Domain Independent Planning”. In: 38th German Conference on Artificial Intelligence (KI 2015). 2015.
[18] K. G. Hartmann, R. T. Schirrmeister, and T. Ball. “Hierarchical internal representation of spectral features in deep convolutional networks trained for EEG decoding”. In: 2018 6th International Conference on Brain-Computer Interface (BCI). Jan. 2018, pp. 1–6. doi: 10.1109/IWW-BCI.2018.8311493.
[19] H. Hatefi, R. Wimmer, B. Braitling, L. M. F. Fioriti, B. Becker, and H. Hermanns. “Cost vs. time in stochastic games and Markov automata”. en. In: Formal Aspects of Computing 29.4 (July 2017), pp. 629–649. issn: 1433-299X. doi: 10.1007/s00165-016-0411-1.
[20] F. A. Heilmeyer, R. T. Schirrmeister, L. D. J. Fiederer, M. Völker, J. Behncke, and T. Ball. “A framework for large-scale evaluation of deep learning for EEG”. In: 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC). arXiv: 1806.07741. June 2018.
[21] G. Kalweit and J. Boedecker. “Uncertainty-driven Imagination for Continuous Deep Rein- forcement Learning”. In: Proceedings of the 1st Annual Conference on Robot Learning. Ed. by S. Levine, V. Vanhoucke, and K. Goldberg. Vol. 78. Proceedings of Machine Learning Research. PMLR, 2017, pp. 195–206.
[22] D. Kuhner, J. Aldinger, F. Burget, M. Göbelbecker, W. Burgard, and B. Nebel. “Closed- Loop Robot Task Planning Based on Referring Expressions”. In: Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Madrid, Spain, 2018.
[23] T. Lampe and M. Riedmiller. “Approximate model-assisted neural fitted Q-iteration”. In: 2014 International Joint Conference on Neural Networks (IJCNN). IEEE. 2014, pp. 2698– 2704.
[24] T. Lampe, L. D. Fiederer, M. Voelker, A. Knorr, M. Riedmiller, and T. Ball. “A Brain- Computer Interface for High-Level Remote Control of an Autonomous, Reinforcement- Learning-Based Robotic System for Reaching and Grasping”. In: Proceedings of the 19th International Conference on Intelligent User Interfaces. IUI ’14. New York, NY, USA: ACM, 2014, pp. 83–88. isbn: 978-1-4503-2184-6. doi: 10.1145/2557500.2557533.
[25] D. Maier, C. Lutz, and M. Bennewitz. “Integrated Perception, Mapping, and Footstep Planning for Humanoid Navigation Among 3D Obstacles”. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Tokyo, Japan, 2013.
[26] K. Scheibler, F. Neubauer, A. Mahdi, M. Fränzle, T. Teige, T. Bienmüller, D. Fehrer, and B. Becker. “Accurate ICP-based Floating-Point Reasoning”. In: Formal Methods in Computer-Aided Design, FMCAD 2016. IEEE, 2016.
[27] K. Scheibler, L. Winterer, R. Wimmer, and B. Becker. “Towards Verification of Arti- ficial Neural Networks”. In: Proceedings of the 18th Workshop “Methoden und Beschrei- bungssprachen zur Modellierung und Verifikation von Schaltungen und Systemen” (MBMV). Ed. by U. Heinkel, M. Rößler, and D. Kriesten. Chemnitz, Germany: Technische Univer- sität Chemnitz, Germany, Mar. 2015, pp. 30–40. isbn: 978-3-944640-34-1.
[28] R. Schirrmeister, P. Chrabąszcz, F. Hutter, and T. Ball. “Training Generative Reversible Networks”. In: ICML 2018 workshop on Theoretical Foundations and Applications of Deep Generative Models. arXiv: 1806.01610. July 2018.
[29] R. T. Schirrmeister, J. T. Springenberg, L. D. J. Fiederer, M. Glasstetter, K. Eggensperger,
M. Tangermann, F. Hutter, W. Burgard, and T. Ball. “Deep learning with convolutional neural networks for EEG decoding and visualization”. en. In: Human Brain Mapping 38.11 (Nov. 2017), pp. 5391–5420. issn: 1097-0193. doi: 10.1002/hbm.23730.
[30] S. Schröer, I. Killmann, B. Frank, M. Völker, L. Fiederer, T. Ball, and W. Burgard. “An autonomous robotic assistant for drinking”. In: IEEE International Conference on Robotics and Automation (ICRA). 2015, pp. 6482–6487.
[31] J. T. Springenberg, A. Dosovitskiy, T. Brox, and M. Riedmiller. “Striving for Simplicity: The All Convolutional Net”. In: arXiv:1412.6806, also appeared at ICLR 2015 Workshop Track. 2015.
[32] J. T. Springenberg and M. Riedmiller. “Improving Deep Neural Networks with Proba- bilistic Maxout Units”. In: arXiv:1312.6116, also appeared at ICLR 2014 Workshop Track. 2014.
[33] M. Völker, R. T. Schirrmeister, L. D. J. Fiederer, W. Burgard, and T. Ball. “Deep transfer learning for error decoding from non-invasive EEG”. In: 2018 6th International Conference on Brain-Computer Interface (BCI). Jan. 2018, pp. 1–6. doi: 10.1109/IWW-BCI.2018.8311491.
[34] M. Völker, J. Hammer, R. T. Schirrmeister, J. Behncke, L. D. J. Fiederer, A. Schulze- Bonhage, P. Marusič, W. Burgard, and T. Ball. “Intracranial Error Detection via Deep Learning”. In: 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC). arXiv: 1805.01667. May 2018.
[35] M. Völker, L. D. J. Fiederer, S. Berberich, J. Hammer, J. Behncke, P. Kršek, M. Tomášek,
P. Marusič, P. C. Reinacher, V. A. Coenen, M. Helias, A. Schulze-Bonhage, W. Burgard, and T. Ball. “The dynamics of error processing in the human brain as reflected by high- gamma activity in noninvasive and intracranial EEG”. In: NeuroImage 173 (June 2018), pp. 564–579. issn: 1053-8119. doi: 10.1016/j.neuroimage.2018.01.059.
[36] M. Watter, J. Springenberg, J. Boedecker, and M. Riedmiller. “Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images”. In: Advances in Neural Information Processing Systems 28. Ed. by C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, R. Garnett, and R. Garnett. 2015, pp. 2728–2736.
[37] D. Welke, J. Behncke, M. Hader, R. T. Schirrmeister, A. Schönau, B. Eßmann, O. Müller,
W. Burgard, and T. Ball. “Brain Responses During Robot-Error Observation”. In: Kogni- tive Systeme. Vol. 1. Sept. 2017. doi: 10.17185/duepublico/44533.
[38] L. Winterer, S. Junges, R. Wimmer, N. Jansen, U. Topcu, J. Katoen, and B. Becker. “Motion planning under partial observability using game-based abstraction”. In: 2017 IEEE 56th Annual Conference on Decision and Control (CDC). Dec. 2017, pp. 2201–2208. doi: 10.1109/CDC.2017.8263971.
[39] D. Kuhner, L. D. J. Fiederer, J. Aldinger, F. Burget, M. Völker, R. T. Schirrmeister, C. Do, J. Boedecker, B. Nebel, T. Ball, and W. Burgard. “Deep Learning Based BCI Control of a Robotic Service Assistant Using Intelligent Goal Formulation”. en. In: Robotics and Autonomous Systems (Mar. 2018), p. 282848. doi: 10.1101/282848.
[40] I. Nematollahi, D. Kuhner, T. Welschehold, and W. Burgard. “Augmenting Action Model Learning by Non-Geometric Features”. In: Proc. of the IEEE Int. Conf. on Robotics & Automation (ICRA). 2019.
[48] T. Ball, M. Völker, M. Glasstetter, L. D. J. Fiederer, and A. Schulze-Bonhage. “Non- Invasive High-Gamma Mapping in an Optimized EEG Lab”. In: International Workshop on High Frequency Oscillations in Epilepsy. Freiburg im Breisgau, Germany, 2016.
[49] J. Behncke, D. Welke, R. T. Schirrmeister, L. L. Kahle, W. Burgard, and T. Ball. “Cor- relates of robot-error observation in the human EEG”. In: Annual Meeting of the Society for Neuroscience (SfN). San Diego, USA, 2016.
[50] J. Behncke, L. D. J. Fiederer, S. Schröer, M. Völker, J. Walther, M. Bennewitz, W. Bur- gard, and T. Ball. “EEG activity during observation of a humanoid robot”. In: Annual Conference of the Organisation for Human Brain Mapping (HBM). Honolulu, USA, 2015.
[51] L. D. J. Fiederer, T. Lampe, M. Voelker, L. Spinello, W. Burgard, M. Riedmiller, and T. Ball. “An Augmented Reality Approach for BCI Control of Intelligent Autonomous Robots”. In: Biomed. Eng. (Berlin). Vol. 59. 2014, S1097–S1097.
[52] L. D. J. Fiederer, R. T. Schirrmeister, M. Völker, J. Boedecker, W. Burgard, and T. Ball. “An Online Brain-Computer Interface Based on Deep Convolutional Networks”. In: Inter- national Conference on Basic and Clinical Multimodal Imaging (BaCI). Bern, Switzerland, 2017.
[53] L. Fiederer, M. Völker, M. Glasstetter, and T. Ball. “Non-invasive high-gamma mapping in an optimized EEG lab”. In: Clinical Neurophysiology 128.10 (Oct. 2017), e313. issn: 1388-2457. doi: 10.1016/j.clinph.2017.06.057.
[54] M. Glasstetter, L. D. J. Fiederer, and T. Ball. “Noninvasive EEG mapping of hand- and foot-movement-related high gamma activity”. In: Annual Conference of the Organisation for Human Brain Mapping (HBM). Honolulu, USA, 2015.
[55] R. T. Schirrmeister, L. D. J. Fiederer, J. T. Springenberg, M. Glasstetter, K. Eggensperger,
M. Tangermann, F. Hutter, W. Burgard, and T. Ball. “DESIGNING AND UNDER- STANDING CONVOLUTIONAL NETWORKS FOR DECODING EXECUTED MOVE- MENTS FROM EEG”. In: The First Biannual Neuroadaptive Technology Conference. 2017, pp. 143–144.
[56] M. Voelker, S. Berberich, E. Andreev, L. D. Fiederer, W. Burgard, and T. Ball. “BETWEEN- SUBJECT TRANSFER LEARNING FOR CLASSIFICATION OF ERROR-RELATED SIGNALS IN HIGH-DENSITY EEG”. In: The First Biannual Neuroadaptive Technology Conference. Vol. 81. 2017, pp. 47–50.
[57] M. Völker, L. D. J. Fiederer, S. Schröer, I. Killmann, B. Frank, W. Burgard, and T. Ball. “A Dry-EEG Brain-Machine Interface for Autonomous Robot-Assisted Drinking”. In: Annual Conference of the Organisation for Human Brain Mapping (HBM). Honolulu, USA, 2015.
[58] M. Völker, S. Berberich, L. D. J. Fiederer, J. Hammer, P. Kršek, M. Tomášek, P. Marusic,
P. C. Reinacher, V. A. Coenen, A. Schulze-Bonhage, W. Burgard, and T. Ball. “Errors elicit high-gamma responses in the human cerebral cortex”. In: Clinical Neurophysiology 128.10 (Oct. 2017), e320. issn: 1388-2457. doi: 10.1016/j.clinph.2017.06.069.
[59] M. Völker, L. D. J. Fiederer, M. Glasstetter, S. Berberich, W. Burgard, A. Schulze- Bonhage, and T. Ball. “Non-Invasive High-Gamma Mapping in an Optimized EEG Lab”. In: International Conference for Advanced Neurotechnology (ICAN). AnnArbor, USA, 2016.
[60] M. Völker, S. Berberich, L. D. J. Fiederer, E. Andreev, S. Contzen, A. Schulze-Bonhage,
W. Burgard, and T. Ball. “Detection of error-related high-gamma activity in non-invasive and intracranial EEG”. In: Annual Meeting of the Society for Neuroscience (SfN). San Diego, USA,
External Publications
[61] M. Andrychowicz, F. Wolski, A. Ray, J. Schneider, R. Fong, P. Welinder, B. McGrew, J. Tobin, P. Abbeel, and W. Zaremba. “Hindsight Experience Replay”. In: (2017).
[62] FIFTY2 Technology GmbH. PreonLab. www.fifty2.eu. Version 2.3.2. Feb. 5, 2018.
[63] E. M. Hahn, H. Hermanns, R. Wimmer, and B. Becker. “Transient Reward Approximation for Continuous-Time Markov Chains”. In: IEEE Transactions on Reliability 64.4 (2015), pp. 1254–1275. issn: 0018-9529. doi: 10.1109/TR.2015.2449292.
[64] S. Junges, N. Jansen, C. Dehnert, U. Topcu, and J. Katoen. “Safety-Constrained Rein- forcement Learning for MDPs”. In: TACAS. Vol. 9636. Lecture Notes in Computer Science. Springer, 2016, pp. 130–146.
[65] D. P. Kingma and M. Welling. “Auto-encoding variational bayes”. In: arXiv preprint arXiv:1312.6114 (2013).
[66] W. Li and E. Todorov. “Iterative linear quadratic regulator design for nonlinear biological movement systems.” In: ICINCO (1). 2004, pp. 222–229.
[67] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra. “Continuous control with deep reinforcement learning”. In: arXiv preprint arXiv:1509.02971 (2015).