Task manager Receptionist
Closed this issue · 7 comments
For the task of Receptionist
of Stage 1
It was discussed with the team the workflow for this task, and added as Command
s in the receptionist_task_manager.py
:
self.current_queue = [
Command(action="wait", complement="person"),
Command(action="analyze", complement="person"),
Command(action="interact", complement="introduce yourself"),
Command(action="ask", complement="user name and favorite drink"),
Command(action="save", complement="name"),
Command(action="interact", complement="ask to be followed to the living room"),
Command(action="go", complement="living room"),
Command(action="look", complement="seat"),
Command(action="interact", complement="introduce host"),
Command(action="look", complement="guest 1"),
Command(action="interact", complement="announce a place to sit will be offered"),
Command(action="look", complement="seat"),
Command(action="find", complement="free seat"),
Command(action="look", complement="free seat"),
Command(action="interact", complement="indicate the user can sit in the direction is gazing"),
Command(action="interact", complement="announce robot will return to entrance"),
Command(action="go", complement="entrance"),
Command(action="wait", complement="person"),
Command(action="analyze", complement="person"),
Command(action="interact", complement="introduce yourself"),
Command(action="ask", complement="user name and favorite drink"),
Command(action="save", complement="name"),
Command(action="interact", complement="ask to be followed to the living room"),
Command(action="go", complement="living room"),
Command(action="look", complement="seat"),
Command(action="interact", complement="introduce host"),
Command(action="interact", complement="introduce guest 1"),
Command(action="look", complement="guest 2"),
Command(action="interact", complement="announce a place to sit will be offered"),
Command(action="look", complement="seat"),
Command(action="find", complement="free seat"),
Command(action="look", complement="free seat"),
Command(action="interact", complement="indicate the user can sit in the direction is gazing")
]
After some thought on the actual implementation of each task, it was decided that, at least for now, a conventional task manager would be the best option
States for the task
Taking into the consideration the architecture desired, this States
were predefined at top of the main task manager file for this task:
STATES = {
"WAITING_GUEST": 0,
"SELF_INTRODUCTION": 1,
"REQUEST_GUEST_INFORMATION": 2,
"SAVE_USER_FACE": 3,
"GO_TO_LIVING_ROOM": 4,
"INTRODUCE_PEOPLE_TO_GUEST": 5,
"GAZE_AT_GUEST": 6,
"FIND_FREE_SEAT": 7,
"GAZE_AT_SEAT": 8,
"GO_TO_ENTRANCE": 9,
"ERROR": 20,
"SHUTDOWN": 100
}
Guest helper class
In order to manage the information of each guest in a more organised manner, it was created a class Guest
with the information collected from each seen guest and the host:
class Guest:
"""Class to store the information of the guest"""
def __init__(self, guest_id: int = 0, name: str = "", favorite_drink: str = "", description: str = "") -> None:
self.guest_id = guest_id
self.name = name
self.favorite_drink = favorite_drink
self.description = description
def set_info(self, name: str, favorite_drink: str) -> None:
self.name = name
self.favorite_drink = favorite_drink
def set_description(self, description: str) -> None:
self.description = description
Face recognition
The services from the FaceRecognition.py
script were called for the first part of waiting for a person to appear at the scene as well as the save_face
service to associate a user's name with the detected face.
HRI guest analysis
Following the developments in this issue, two features of the Human-Robot Interaction area were created. One extracts the name and the favorite drink of the person from the given speech, and the other uses the gpt-vision
model to analyze the image.
Task manager testing
Today it was tested the complete integration of the task manager in receptionist_task_manager.py
. Some issues were spotted and fixed and the setup was a laptop (mine) with a graphic card NVIDIA GTX 1650.
The nodes and features executed in the external computing device were:
- Manipulation arm joints server:
rosrun frida_arm_joints_server arm_joint_server.py
- Navigation action server:
roslaunch actions nav_actions.launch
- Vision face recognition, find persons and free seat:
roslaunch vision receptionist.launch
- HRI guest analysis:
rosrun frida_language_processing guest_analyzer.py
- Task manager:
rosrun frida_task_manager receptionist_task_manager.py
For the following iterations, this areas of improvement were detected:
- Enhance angle to free seat
- Time optimization
- Image analysis prompt engineering
- Navigation tuning