Task manager Receptionist

[afr2903]

For the task of Receptionist of Stage 1

[afr2903]

It was discussed with the team the workflow for this task, and added as Commands in the receptionist_task_manager.py:

self.current_queue = [
    Command(action="wait", complement="person"),
    Command(action="analyze", complement="person"),
    Command(action="interact", complement="introduce yourself"),
    Command(action="ask", complement="user name and favorite drink"),
    Command(action="save", complement="name"),
    Command(action="interact", complement="ask to be followed to the living room"),
    Command(action="go", complement="living room"),
    Command(action="look", complement="seat"),
    Command(action="interact", complement="introduce host"),
    Command(action="look", complement="guest 1"),
    Command(action="interact", complement="announce a place to sit will be offered"),
    Command(action="look", complement="seat"),
    Command(action="find", complement="free seat"),
    Command(action="look", complement="free seat"),
    Command(action="interact", complement="indicate the user can sit in the direction is gazing"),
    Command(action="interact", complement="announce robot will return to entrance"),
    Command(action="go", complement="entrance"),

    Command(action="wait", complement="person"),
    Command(action="analyze", complement="person"),
    Command(action="interact", complement="introduce yourself"),
    Command(action="ask", complement="user name and favorite drink"),
    Command(action="save", complement="name"),
    Command(action="interact", complement="ask to be followed to the living room"),
    Command(action="go", complement="living room"),
    Command(action="look", complement="seat"),
    Command(action="interact", complement="introduce host"),
    Command(action="interact", complement="introduce guest 1"),
    Command(action="look", complement="guest 2"),
    Command(action="interact", complement="announce a place to sit will be offered"),
    Command(action="look", complement="seat"),
    Command(action="find", complement="free seat"),
    Command(action="look", complement="free seat"),
    Command(action="interact", complement="indicate the user can sit in the direction is gazing")
]

[afr2903]

After some thought on the actual implementation of each task, it was decided that, at least for now, a conventional task manager would be the best option

[afr2903]

States for the task

Taking into the consideration the architecture desired, this States were predefined at top of the main task manager file for this task:

STATES = {
    "WAITING_GUEST": 0,
    "SELF_INTRODUCTION": 1,
    "REQUEST_GUEST_INFORMATION": 2,
    "SAVE_USER_FACE": 3,
    "GO_TO_LIVING_ROOM": 4,
    "INTRODUCE_PEOPLE_TO_GUEST": 5,
    "GAZE_AT_GUEST": 6,
    "FIND_FREE_SEAT": 7,
    "GAZE_AT_SEAT": 8,
    "GO_TO_ENTRANCE": 9,
    "ERROR": 20,
    "SHUTDOWN": 100
}

[afr2903]

Guest helper class

In order to manage the information of each guest in a more organised manner, it was created a class Guest with the information collected from each seen guest and the host:

class Guest:
    """Class to store the information of the guest"""
    def __init__(self, guest_id: int = 0, name: str = "", favorite_drink: str = "", description: str = "") -> None:
        self.guest_id = guest_id
        self.name = name
        self.favorite_drink = favorite_drink
        self.description = description
    def set_info(self, name: str, favorite_drink: str) -> None:
        self.name = name
        self.favorite_drink = favorite_drink
    def set_description(self, description: str) -> None:
        self.description = description

[afr2903]

Face recognition

The services from the FaceRecognition.py script were called for the first part of waiting for a person to appear at the scene as well as the save_face service to associate a user's name with the detected face.

[afr2903]

HRI guest analysis

Following the developments in this issue, two features of the Human-Robot Interaction area were created. One extracts the name and the favorite drink of the person from the given speech, and the other uses the gpt-vision model to analyze the image.

[afr2903]

Task manager testing

Today it was tested the complete integration of the task manager in receptionist_task_manager.py. Some issues were spotted and fixed and the setup was a laptop (mine) with a graphic card NVIDIA GTX 1650.

The nodes and features executed in the external computing device were:

• Manipulation arm joints server: rosrun frida_arm_joints_server arm_joint_server.py
• Navigation action server: roslaunch actions nav_actions.launch
• Vision face recognition, find persons and free seat: roslaunch vision receptionist.launch
• HRI guest analysis: rosrun frida_language_processing guest_analyzer.py
• Task manager: rosrun frida_task_manager receptionist_task_manager.py

For the following iterations, this areas of improvement were detected:

• Enhance angle to free seat
• ⁠Time optimization
• ⁠Image analysis prompt engineering
• Navigation tuning