Rotated rosbag keeps page cache persistent

Question

Rotated rosbag keeps page cache persistent

Naophis opened this issue a year ago · 9 comments

Description

When using ros2 bag record -a -d 60 to rotate rosbags, the page cache remains persistent.
The enlargement of the page cache can sometimes trigger swapping, which is undesirable.
I would appreciate it if this could be fixed.
Alternatively, is there a current workaround to release it during rotation?

Expected Behavior

Release the page cache when a rotation file is created.

Actual Behavior

Keep page cache

To Reproduce

Publish any topic.
Execute ros2 bag record -a -d 60
Wait for 60 seconds.
Use vmtouch -v <rosbag_file> to check if it's being cached in the page cache.

System (please complete the following information)

OS: Ubuntu 22.04
ROS 2 Distro: humble
Install Method: apt
Version: 0.15.7 ? (Confirmed by /opt/ros/humble/share/rosbag2/package.xml)

Additional context

Answer 1 · 2023-08-22T06:18:55.000Z

@Naophis On Humble we are using SQLite3 storage plugin by default and cache control is out of our scope.
I think that this is some internal implementation specific to the SQLite3 database backend.

Answer 2 · 2023-09-14T07:38:29.000Z

@MichaelOrlov
Thank you for your response. I have also tried various things with SQLite3, but it seems that there is no direct way to release the cache on the SQLite3 side.

If there is anything that can be done with rosbag2, it would be convenient if there were an API that could be called when rotation occurs.

Answer 3 · 2023-09-14T17:56:07.000Z

@Naophis With the new MCAP storage format we are writing to the file directly and have more opportunity to control behavior.
However, this needs to be investigated.

Answer 4 · 2023-09-14T17:59:52.000Z

@Naophis BTW. Could you please double-check if the same problem persists with the MCAP storage format?
You can use --storage mcap parameter for ros2 bag record command

Answer 5 · 2023-09-15T00:55:31.000Z

@MichaelOrlov Recording with MCAP still left the page cache intact similarly.

Answer 6 · 2023-09-15T03:25:27.000Z

@Naophis Out of curiosity do you know how to force to free cache?

Answer 7 · 2023-09-16T12:31:00.000Z

@MichaelOrlov
I am using vmtouch to check the cache status and to clear it.
If you've checked the overall OS cache status with vmtouch, it would be good to monitor it using tools like htop.

Answer 8 · 2023-09-17T23:16:05.000Z

In my understanding, (if I am not mistaken) the cache on OS level shall be freed automatically as soon as we close the file descriptor. Isn't it?

Answer 9 · 2023-09-18T13:49:51.000Z

I'm not an expert either, so I may have incorrect knowledge.

On closing a file, two things are released: the buffered cache and the file descriptor, while the page cache is not included.
The page cache is like a bypass that the operating system provides for the purpose of optimizing R/W efficiency, and it is not typically controlled directly at the application layer.