Aluno: Guilherme Antonio Camelo
Orientadora: Profª. Drª. Rosa Maria Viccari
Título: Human path prediction through machine learning trained with 2D video data and estimated 3D pose data
Linha de Pesquisa: Inteligência Artificial
Local: Sala 218 no prédio 43412 do Instituto de Informática da UFRGS
– Prof. Dr. Dante Augusto Couto Barone (UFRGS)
– Profª. Drª. Carla Maria Dal Sasso Freitas (UFRGS)
– Prof. Dr. Alessandro Bof de Oliveira (UNIPAMPA – por videoconferência)
Presidente da Banca: Profª. Drª. Rosa Maria Viccari
Abstract: The interaction between robots and humans has been advancing at an ever-rising pace. Being aware of not only where humans are but also predict where humans will go and do next is an important feature to have in an assistant robot. The main goal of this project is to explore ways of improving this feature. Future trajectories of humans walking were predicted using deep learning fed with RGB data in a controlled environment in order to better assist humans when needed. Kinect was used to achieve that goal along with its RGB and infrared cameras. Data of people walking in a controlled environment was collected and a dataset was created. Data from Human3.6m dataset was used as well. The data was used to train RNN-LSTM models created to predict future paths. Openpose was used to identify humans and their body joints and create poses. 3D pose data was estimated from 2D pose data using a 3D Pose Estimator in order to recreate the path in the 3D space. An LSTM model with a 3D feature was created and trained with 3D estimated data. A path prediction model with a 3D element was assessed in comparison with a 2D path prediction model. While models were able to learn from the data and present good predictions in some cases, they were not able to learn in other cases outputting bad predictions. The metrics used to get quantitative results presented limitations to measure the predictions. Limitations of using a 3D Pose estimator for 3D path reconstruction were described. As a result of our project, models that predict future path of people with different designs and performances were developed. As a contribution, a dataset of 113 GB containing people walking in a controlled environment was created and a methodology to estimate 3D path information from 2D Pose was proposed.
Keywords: RNN, LSTM, Deep Learning, Machine Learning, Path Prediction, Robotics, Pose, 3D Pose, 3D Pose Estimation, Openpose, Human3.6m.