Make a more flexible task definition system

[mxgrey]

As of right now, all of the task types that are supported by RMF's out-of-the-box task management system are hard-coded and inflexible. We can improve this in two ways:

1. Expose certain phases to the public API, and have all tasks be defined as sequences of phases

Right now, all "task phases" are internal implementation details that are not accessible to users of RMF. To modify which phases are in a task or to modify how those phases work, a user would need to fork rmf_ros2 and modify the code directly. This is not good for the RMF ecosystem, because it would create splintering in the community every time a new use-case is not already supported by the code out of the box.

Instead of this rigid approach, we could expose some task phases to the public API, and allow users to define their own tasks however they'd like. Some already-existing phases that I'd recommend for the public API include:

• GoToPlace
• DispenseItem PickUp
• IngestItem DropOff
• DockRobot
  Most of the other already-existing phases I would recommend remain private implementation details that will be implicitly used by the public API phases. A full list of the existing phases can be found here.

Some new phases we could consider adding as part of this improvement includes:

• RepeatN: Input a sequence of other phases conditions and an integer N. The input sequence of phases will be repeated N-times.
• RepeatIndefinitely: Input a sequence of other phases conditions. The input sequence of phases conditions will be repeated until an interruption signal is received or the battery runs too low.
• While: Input a sequence of other phases conditions and a looping condition. The input sequence of phases conditions will be repeated while the looping condition is satisfied.
• PerformAction: Input some instruction to be sent to the robot to perform a built-in action, e.g. cleaning. The phase will finish when the robot sends back a finished signal (currently we are overloading the DockRobot phase to do this).
• ParkUntil: Park at some parking spot until some condition is satisfied.
• (added) Call: Call a phone number and play some audio.
• (added) SMS: Send an SMS to a phone number.
• (added) ParallelGroup: Input a list of other phases to run in parallel with each other.

2. Allow custom phases to be defined as three components: (a) an initiation signal, (b) finishing conditions, (c) estimate of the finished robot state

(a) To truly make RMF extensible it would be good to allow custom phases with generic hooks. One of the generic hooks would be the "initiation signal" which will get triggered when the phase begins. In rmf_task this signal would likely use a pure abstract interface class (or generic std::function<void()> callback), while in rmf_task_ros2 it would use a combination of a ros2 topic name and some message payload (maybe using C++ templates to define the payload?).

(b) For the finishing conditions, these could be built from a composable set of primitives, such as the following:

• received_message: Input a ROS2 topic and a message value that needs to be received. When a suitable message arrives on the topic, this condition is satisfied.
• duration_elapsed: Input a time duration. When the duration is elapsed, the condition is satisfied.
• time_reached: Input a time point. When the time point is reached, the condition is satisfied.
• not: Input some other condition. When the condition is not met, this condition is satisfied.
• and_group: Input a set of other conditions. When all those conditions are met, so is this one.
• or_group: Input a set of other conditions. When at least one of those conditions is met, so is this one.

When the finishing condition of the phase is satisfied, the phase will end, and the task handler will initiate the next phase.

(c) The phase description should be able to estimate the finished state of the robot, given an initial state, so that the task planner has enough information to allocate any tasks that use the phase.

[Yadunund]

Another component of the phase could be d) error handling response
Related tickets:

• open-rmf/rmf_task#32
• open-rmf/rmf_task#33

[mxgrey]

Good point on error handling. We could also consider this:

• Custom phases specify an error condition (same as finishing conditions, except when the error conditions are satisfied, an error will be raised).
• For each of its phases, a task description may specify a new phase sequence that will be triggered in the event of an error.

[cwrx777]

Hi,

In the following scenario:
The kitchen prepares meals for delivery to multiple hotel rooms. Each room is assigned a compartment.
When the AMR reach the room:

1. AMR calls the hotel room number phone.
2. AMR wait for the guest to come out and tap the guest card on AMR UI
3. AMR open compartment
4. guest remove the food and close the compartment
5. AMR wait for the guest to close the hotel door
6. AMR move to the next room.

What should be the scope of a 'task' in the above scenario?

[mxgrey]

I'll propose that a single "Task" should begin from the robot being in a state that I will call "available", and the task finishes when the robot is back to being "available". In this context "available" would mean the following conditions:

1. The robot is not performing any task
2. The robot is unencumbered, meaning it does not have anything in its compartments, and it does not have any other internal state that could interfere with the performance of another task

If we go with that definition, the scope of the workflow you described would all fall into a single Task, and additionally that Task would include the initial pickup of all the meals from the kitchen. So I could imagine a task description like the following, imagining that we've created a yaml-based syntax for describing tasks:

- GoToPlace:
    location: kitchen
- IngestItems:
    items:
     - compartment: A
       item: meal_A
     - compartment: B
       item: meal_B
     - ... the rest of the meals ...
- GoToPlace:
    location: room_A
- PerformAction:
    action: call
    params:
      number: <room A phone number>
- DispenseItems:
    items:
     - compartment: A
       item: meal_A
- GoToPlace:
    location: room_B
- PerformAction:
    action: call
    params:
      number: <room B phone number>
- DispenseItems:
    items:
     - compartment: B
       item: meal_B
- GoToPlace:
 ... the rest of the rooms ...

Ideally a planner would be used to decide the order in which the rooms should be visited to minimize the amount of traveling, and then the task description can be dynamically generated to use that ordering.

[mxgrey]

Also if calling a phone number is found to be a relatively common use case and can be reasonably standardized, we could add support for it as a first-class phase rather than requiring system integrators to create a custom phase for it with the generic PerformAction phase.

[mdfesto]

calling a phone number and playing some audio recording will be common in the hospitality services industry and I could see how it may be useful for other operations where they don't have a human sitting by a console for a web notification. some users may look for SMS (or other messaging application) notification as another possible alert method

[mxgrey]

Okay! I'll add those to the list of desirable phases to have first-class support for.

[cnboonhan]

i was a bit lost during the "phase refactoring" discussion call, this is probably a duh clarification :man-bowing I might be mistaken on certain points:

1. phase refactoring reduces code fragmentation from implementing new use cases that cannot be described by existing tasks, or sequences of tasks
2. We want to expose phases to the public, and create a tool allowing the user to string phases together to create novel tasks.
3. the variety of phases will be fixed: we aim to creatively chain phases together ( otherwise, it seems fragmentation from 1. is inevitable )
   4.further, we want to limit the complexity of this new "chaining" of phases by avoiding "behavior tree" - like implementations.

so the value of this seems to narrow down to how much better it is to "chain phases", rather than to "chain tasks". Due to point 4, we limit ourselves to simple "almost-sequential" behaviors , so it seems the benefits narrow down :

• it is better to chain phases instead of chaining tasks, so we can persist information from past phases ( for example, perhaps it is easier to track a delivery robot if it has multiple pickups, or we can decide on a certain route to take )
• it is better to chain phases because of integrated logic for error recovery ( uniform framework described in phases, instead of ad hoc piecemeal logic based on the results of tasks ) - not too clear on this

[mxgrey]

3. the variety of phases will be fixed: we aim to creatively chain phases together ( otherwise, it seems fragmentation from 1. is inevitable )

Ideally I'd like us to create a system where the variety of phases is not fixed. We'll define abstract interfaces for phases which allow system integrators to implement any kinds of phases that they'd like.

so the value of this seems to narrow down to how much better it is to "chain phases", rather than to "chain tasks"

I'm not exactly sure what you mean by "chain tasks" here, but I would say one reason it's better to "chain phases" is because "chaining tasks" is just a special case of "chaining phases".

But more generally, we're just defining the concepts like this:

• Task: A sequence of actions that need to happen contiguously in a specified order to satisfy the requirements of a user. A robot is not allowed to begin a new task until it is finished with its latest one.
• Phase: The building block of a task. It triggers some event to begin and tracks some condition(s) to determine when the event is completed (or when an error has occurred, making the event impossible).

We don't chain tasks together because we're defining tasks as being isolated items to be accomplished independently of each other. This gives the planner freedom to identify the optimal arrangement of tasks to perform. We may attempt to add task constraints in the future (i.e. Task A needs to be completed before Task B), but that's a reach goal.

[cnboonhan]

ah, thanks for explaining. I might have missed the distinction between Tasks and Phases in the docs. That clarifies a lot.

Somehow I misread point 2 "custom phases" to be referring to an interface for the new phases we will add ( as listed in 1. ), instead of being that for the SIs to implement how they like, which led me to wrongly conclude that the SI's are limited to only defining new tasks by combining phases.

It naively seems rather similar in terms of code fragmentation, if users / SIs freely define phases, or if they freely define new tasks, but perhaps I have not dug enough into the implementation to see the differences clearly.

[mxgrey]

If we don't provide the means for SIs to freely define their own custom tasks and phases, then we effectively guarantee code fragmentation among the community because it's effectively impossible for us to anticipate every possible future use case that an SI will encounter. By leveraging abstract interfaces, SIs are not limited to just the tasks/phases that we're providing out of the box, so they don't need to fork their own version of the code.

Of course we'll always encourage SIs to push the implementations of their custom tasks and phases upstream so the community can standardize around the useful implementations that others have developed, but I think if RMF were designed in a way that required all implementations to exist in the core code base, then we'd be alienating members of our community who cannot (or would strongly prefer not to) push their work upstream.

[youliangtan]

Correct me if I am wrong, I briefly use a py syntax to represent the high-level idea on rmf_task2.0 public APIs.

# create custom task by chaining up phases
task = rmf_task.task_creator(...)
task.phases.append(rmf_task.phase.custom_phase_factory(<ACTION1_JSON>))
task.phases.append(rmf_task.phase.GoToPhase(...))
task.phases.append(rmf_task.phase.custom_phase_factory(<ACTION2_JSON>))

# submit task to rmf
rmf_dispatcher.request_task(task)

Also, In this case, we will also need to support dynamic definition for RMF task/phase reporting, specifically for status updating and error reporting. Might also be great if we have some public observable function for this.

[Yadunund]

I'm sharing the slides here from our first discussion which includes a summary and proposed follow ups.
https://docs.google.com/presentation/d/1REdfImJymt65paReOGFYKWA7dT3Wj3cj2qx-T21SvBc/edit#slide=id.p

If anyone else would like to be part of these discussions, feel free to drop a comment. Depending on the number of interested folks, we can decide how to organize ourselves (slack channel, email thread, etc).

[Yadunund]

Correct me if I am wrong, I briefly use a py syntax to represent the high-level idea on rmf_task2.0 public APIs.

# create custom task by chaining up phases
task = rmf_task.task_creator(...)
task.phases.append(rmf_task.phase.custom_phase_factory(<ACTION1_JSON>))
task.phases.append(rmf_task.phase.GoToPhase(...))
task.phases.append(rmf_task.phase.custom_phase_factory(<ACTION2_JSON>))

# submit task to rmf
rmf_dispatcher.request_task(task)

Also, In this case, we will also need to support dynamic definition for RMF task/phase reporting, specifically for status updating and error reporting. Might also be great if we have some public observable function for this.

@youliangtan I'm a bit confused by the syntax but to summarize:

• We will provide two abstract interfaces in the public API of rmf_task
  • rmf_task::PhaseDescription to describe the intention of the phase. Sample implementations for first-class phases such as GoToPlace, DispenseItem, Dock, etc will also be included. These descriptions will be used for task allocation planning by the TaskPlanner
  • rmf_task::PhaseFactory: A factory to generate the component of the phase that will manage its execution. ie, when it should start/finish, handle errors etc. Sample implementations for first-class phases will be provided within rmf_task_ros2.
• User interfaces like rmf-web or CLI will submit a msg with a string field which encompasses the sequence of phases for that task in a json-like format. A schema will be drafted and made public to detail the syntax for this
• This string will be parsed to generate two components for each phase in the task: 1) PhaseDescription and 2) PhaseFactory. The former will be used for planning while the latter will be submitted to the TaskManager to handle the execution if the task is assigned.

[mxgrey]

I've opened a draft PR for this topic here. It's still in the early stages of fleshing out the API, but please feel free to leave comments.

[mxgrey]

I'll close this ticket now since all the major features related to this have been merged. New tickets can be opened to discuss finer details.