Customer behavior classification for service robot

Behavior mirrors the unique personality through two factors nature and nurture. The human expression of under control or over control is the human behavior that expresses some destinations. Human behavior looks like the one way to communicate with humans is non-verbal communication. Non-verbal communication is an inborn skill used to survive in the world. This communication is not necessary to use the language to be the intermediate to transfer the data. However, this communication uses an intermediate expression such as body language, facial expression, or hand gesture. A robot is a machine created for many tasks, such as serving food in a restaurant. We can see many types of robots more than in the past. Developer developed the service robot to support or improve human work performance in many fields. Human behavior is essential information, so if the service robot can know the human behavior, it will enhance the performance of responding to a human and include the possibility of the robot. Nowadays, many people work at an office or their home at a desk, especially in the COVID-19 situation. However, if the customers do their work at a desk for a long time, they will have some stress or illness. Therefore, if we have a service robot that can be a friend while a customer is working at a desk, it will help decrease the customers' stress. Many people work with a laptop or computer. If we set the robot in front of the customer, the laptop or computer screen will cover the customer's body. Therefore, we must place the robot at the indented angle at the customer's right or left sides. We use in this research consist of 5 static behaviors: raising hands, lying down, working, thinking, and relaxing.To classify the human behavior while a customer is working at a desk, we have used the method that consists of 3 steps: 1) skeleton feature extraction, 2) data processing, and 3) behavior classification. We have first gotten the human skeleton information on an input RGB image using the OpenPose framework. Then, we have calculated the vectors and angles of the skeleton information to be the input features of a model. Finally, we used the input features from the previous step to classify human behavior using the Deep Learning algorithm.After we have trained and tested many cases, we have found the MLPClassifier of the Scikit-Learn library with hidden layer size is [128] can predict the human behavior with high accuracy faster than other algorithms. Therefore, the MLPClassifier is the best algorithm for human behavior classification of working situations with the F1 score of each behavior above 0.96.