colcon/colcon-core

Design intent of setup.sh vs setup.bash

Ryanf55 opened this issue · 4 comments

Ryanf55 commented

Hello!

I'm working with the ROS open-embedded working group. The group is developing openembedded layers for use with ROS.

Colcon-core generates multiple scripts in the install directory such as:

  • install.bash
  • install.sh

Normally, on Linux, ROS humble runs on Ubuntu 22, which comes with many packages by default. Developers can easily make assumptions that certain commands are always available, such as head. We've run into some issues by not knowing what is required as seen here.

Can the maintainers assist me in documenting the required packages that colcon-core relies on, and if there is any design intent that the install.sh script would be intended for a more minimal OS like poky or alpine Linux that doesn't have bash and other tools available by default?

If setup.sh is intended for minimal systems

  • Consider a change to to use cut instead of head
  • meta-ros should distribute a fully featured head command with ros-image-core and leave it out for ros-image-minimal

If setup.sh is not intended for minimal systems

  • Document the design intent here and perhaps some dependencies
  • meta-ros will need to add more tools to the minimal OS layer.
cottsay commented

Colcon-core generates multiple scripts in the install directory such as:

install.bash
install.sh

The install.bash is generated by colcon-bash, not colcon-core. The only "native" shells that colcon supports are "sh" and "bat". There are extensions to enable Powershell, Bash and Zsh. All three of the additional shells are part of colcon-common-extensions.

Developers can easily make assumptions that certain commands are always available, such as head.

This might be nitpicky, but I'm not sure what this particular example has to do with the shell. head and cut are both standalone executables provided by coreutils on both Fedora and Debian. What and how busybox chooses to implement these "core" executables doesn't seem like something we should be concerned with here, no?

We can't declare dependencies on system packages from pure python packages, but we could probably add the dependency to the debs. From the sounds of it, that won't help the other (minimalistic) Linux distros you're discussing though.

For this specific case, if there's a clean way to switch from head to cut to appease busybox without regressing other platforms, I'd love to see a PR.

  • ❤️1
cottsay commented

Tangent: If your intent is truly to generate a minimal build environment, you should consider dropping colcon-common-extensions in favor of the specific colcon extensions you need to complete the build.

For example, this is the set of extensions present on ci.ros2.org builds: https://github.com/ros2/ci/blob/403b3445ee7e7253dd6c24b042feb3ae584dce76/ros2_batch_job/__main__.py#L86-L109

Ryanf55 commented

Great information! Thank you for the speedy feedback.

I understand colcon is intended to be more portable than ROS which is focused mainly on the tier 1 distributions.

Would it be OK to request that colcon-core be modified to work better on busybox that does not come with the same form of coreutils as long as functionality to other distros is not impacted? We can do the PR's as needed.

For long term maintenance, the OE working group doesn't have CI for anything yet, but this is something that could be contributed to colcon-core to support longer-term maintenance on busybox systems once we figure out how to do it.

Rob hinted he found a clean solution for the head to cut issue. We're not sure how many more changes there would be, but this kind of support to colcon would help it be used in more minimal OS's.

cottsay commented

Would it be OK to request that colcon-core be modified to work better on busybox that does not come with the same form of coreutils as long as functionality to other distros is not impacted?

I see no reason to reject a change that doesn't regress existing platforms and doesn't unnecessarily increase maintenance burden.

... the OE working group doesn't have CI for anything yet, but this is something that could be contributed to colcon-core ...

I think we could consider adding another platform if it integrates cleanly into GitHub Actions. All colcon extensions in this org use https://github.com/colcon/ci.

Rob hinted he found a clean solution for the head to cut issue.

Great, looking forward to it.

  • 👍1