Multimodal Reinforcement Learning Human Study

We performed a human user study where 9 healthy adults were recruited to interact with our HEL agent. Each subject performed 4 to 5 trials (entire interactions) with the HEL agent adding up to a total of 42 trials. The hypothetical experiment environment would be a room with a drawer, a shelf, and a cabinet. The user can choose between red, green, and yellow cups and red, green, yellow, and white balls. At the beginning of each trial, objects are randomly scattered in different locations. The user only knows there are aforementioned locations and objects in the room but doesn’t know which item is located where. The subjects were instructed to choose the object of interest at the beginning of the trial, and guide the agent through different locations to find the object of interest.