Questions about nuPlan conveter

Hello,

Thanks a lot for the repo, I'm currently trying to use scenarionet as a standlone and I have some questions about the nuPlan converter script placed in scenarionet/scenarionet/converter/nuplan/utils.py

First, why the boundaries of the block are typed as LINE_SOLID_SINGLE_WHITE and not ROAD_EDGE_BOUNDARY directly ?

for idx, boundary in enumerate(boundaries[0]):
    block_points = np.array(list(i for i in zip(boundary.coords.xy[0], boundary.coords.xy[1])))
    block_points = nuplan_to_metadrive_vector(block_points, center)
    id = "boundary_{}".format(idx)
    ret[id] = {SD.TYPE: MetaDriveType.LINE_SOLID_SINGLE_WHITE, SD.POLYLINE: block_points}

Second, I'm curious about the line extraction. If I understand correctly, for each lane we look if there is a left boundary. If there is, we presume it is a broken line and we add it to the map block.

left = lane_meta_data.left_boundary
if left.id not in ret:
    # only broken line in nuPlan data
    # line_type = get_line_type(int(boundaries.loc[[str(left.id)]]["boundary_type_fid"]))
    line_type = MetaDriveType.LINE_BROKEN_SINGLE_WHITE
    if line_type != MetaDriveType.LINE_UNKNOWN:
        ret[left.id] = {SD.TYPE: line_type, SD.POLYLINE: get_points_from_boundary(left, center)}

But it means two things:

For the leftmost lane, the left lane will be potentially a road edge and we will add twice the information to the data
For the rightmost lane, we add to lane of the right.

Can it be possible to have something more robust to detect lines ?

Finally, I was curious about the intersections. I see that there are typed as LANE_SURFACE_UNSTRUCTURE, and from your experience, I'm wondering if it possible to properly divide them between LANE_SURFACE_STREET and LINE_BROKEN_SINGLE_WHITE

Hi,

Thank you for your interest. Regarding your questions.

Now, the map feature types are mainly from Waymo's standard, but some of them will be deprecated soon. ROAD_EDGE_BOUNDARY is one of them. This is because we want to keep only some common types that can apply to all datasets besides Waymo. And in Waymo's definition, the ROAD_EDGE_BOUNDARY means sidewalk edge which is a polyline. However, most sidewalks are polygons in other datasets' type definitions, and hence we don't use this type anymore. Even for Waymo converter, we now treat this type as white solid line as well. As both ROAD_EDGE_BOUNDARY and SOLID_LINE are something that can not be violated, we consider it acceptable to do so.

It is just a workaround. Ideally, all lane lines and their types should be provided by the nuPlan-devkit. However, there is no information on the detailed line type according to nuPlan's official reply: motional/nuplan-devkit#219. Thus, what we can do now is just infer this information and this is the best way so far. And, yes, you are correct. The leftmost lane lines would be added twice and with types of both dash lines and solid lines. It doesn't break the simulation because the solid line has a higher priority in collision detection. Despite this, we should fix it.

What makes it called "unstructured" is that we can not divide the road into lanes explicitly with broken_lines. Just like driving at an intersection, you can still feel that there are some virtual lanes guiding you from one entry to another but there are no broken_lines. Thus in our format, lanes whose type is unstructured means virtual lanes as well.

Now, the map feature types are mainly from Waymo's standard, but some of them will be deprecated soon. ROAD_EDGE_BOUNDARY is one of them. This is because we want to keep only some common types that can apply to all datasets besides Waymo. And in Waymo's definition, the ROAD_EDGE_BOUNDARY means sidewalk edge which is a polyline. However, most sidewalks are polygons in other datasets' type definitions, and hence we don't use this type anymore. Even for Waymo converter, we now treat this type as white solid line as well. As both ROAD_EDGE_BOUNDARY and SOLID_LINE are something that can not be violated, we consider it acceptable to do so.

This part is interesing because I'm actually trying to port ScenarioNet data into Waymax directly. In Waymax there is a specific definition for RoadEdgeBoundary as you may know. If I understand correctly, the goal is to make ROAD_LINE_SOLID_SINGLE_WHITE as ROAD_EDGE_BOUNDARY because there are the real road limitations and RoadEdgeBoundary can be the definition of the sidewalk. The only problem that I saw is that some ROAD_LINE_SOLID_SINGLE_WHITE in the ScenarioNet are actually line that separates lines in the middle of the road, making it difficult to separate them for the RoadEdgeBoundary.

I would be up to do some PR if you think there are some globals improvements that be done, but it is totally up to your visions.

Ahh, I got you. Actually, you can find the exterior or boundary with the drivable area polygon consisting of all lanes. You can even expand this drivable region polygon with some shapely functions. So you can use the new a bit larger polygon's egdes as the RoadEdgeBoundary if you really need it for your scene.

I use this trick to extract all external solid white lines for nuscnes and nuplan dataset, as they don't provide any road boundary information. This is a reference:

scenarionet/scenarionet/converter/argoverse2/utils.py

Line 156 in c6775d7

polygons = []

I appreciate your help in connecting ScenarioNet and Waymax and would be very happy to share some thoughts on your PR. Just ping me when you feel it is ready!

I appreciate your help in connecting ScenarioNet and Waymax and would be very happy to share some thoughts on your PR. Just ping me when you feel it is ready!

On a general note, it is complex to connect directly ScenarioNet to Waymax because the pickle files obtained from the various datasets can differ. Do you think there is a way that ScenarioNet because a standalone that would provide an agnostic Scenario Format or is it only designed to match MetaDrive directly ?

ScenarioNetw format is a general self-contained format, so it can be loaded by other simulators as well. Its main goal is to align all datasets and provide a unified entry for using them. The .pkl files from different sources are different mainly on the level of map details. Some provide the connectivity or polygon of a lane, some do not. But the only thing regarding map is that all maps have to provide the lane center lines, with which we can predefine a lane width and build a map finally (of course, it would be better to have its polygons directly). For the tracks, .pkl files from all datasets are exactly the same with positions/velocities/masks and so on. The last difference between datasets is that traffic light information is only available in some datasets like WOMD. Given this, I think it is feasible to load .pkl file to Waymax.