struct.error during writing rosbag

[Hypothesis-Z]

Error occurs when writing messages to rosbag. It does not always happen and seems to be able to appear on writing any item of the bag. So I have to try several times until all items are completely writed at one time.

BTW, my bag size is very large (dozens or hundreds of GB without compression) though I am not sure if it is responsible.

Environment:

• WSL Ubuntu 20.04 LTS
• python 3.8
• ROS Noetic

Simplified Code:

import rosbag
import rospy

def exchange_topics(input_bagfile, output_bagfile, topic1, topic2):
    with rosbag.Bag(output_bagfile, 'w') as outbag:
        for topic, msg, t in rosbag.Bag(input_bagfile).read_messages():
            if topic == topic1:
                new_topic = topic2
            elif topic == topic2:
                new_topic = topic1
            else:
                new_topic = topic

            outbag.write(new_topic, msg, t)

if __name__ == "__main__":
    input_bagfile = "/path/to/input.bag"  # Replace with the path to the input bagfile
    output_bagfile = "/path/to/output.bag"  # Replace with the path to the output bagfile
    topic1 = "/topic1"  # Replace with the first topic to be exchanged
    topic2 = "/topic2"  # Replace with the second topic to be exchanged

    exchange_topics(input_bagfile, output_bagfile, topic1, topic2)

Error Traceback:

Traceback (most recent call last):
  File "exchange_uwb_topic_name.py", line 36, in <module>
    process(outbag, topic, msg, t)
  File "exchange_uwb_topic_name.py", line 18, in process
    bag.write(topic, msg, t)
  File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 704, in write
    self._stop_writing_chunk()
  File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 1664, in _stop_writing_chunk
    self._write_connection_index_record(connection_id, entries)
  File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 1744, in _write_connection_index_record
    buffer.write(_pack_uint32(entry.offset))
  File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 1940, in _pack_uint32
    def _pack_uint32(v): return struct.pack('<L', v)
struct.error: argument out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "exchange_uwb_topic_name.py", line 36, in <module>
    process(outbag, topic, msg, t)
  File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 478, in __exit__
    self.close()
  File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 719, in close
    self._stop_writing()
  File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 1587, in _stop_writing
    self.flush()
  File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 579, in flush
    self._stop_writing_chunk()
  File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 1664, in _stop_writing_chunk
    self._write_connection_index_record(connection_id, entries)
  File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 1744, in _write_connection_index_record
    buffer.write(_pack_uint32(entry.offset))
  File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 1940, in _pack_uint32
    def _pack_uint32(v): return struct.pack('<L', v)
struct.error: argument out of range

[Hypothesis-Z]

Perhaps outbag.flush() should be called here.

ros_comm/tools/rosbag/src/rosbag/rosbag_main.py

Lines 392 to 399 in 030e132

	for topic, raw_msg, t, conn_header in inbag.read_messages(raw=True, return_connection_header=True):
	msg_type, serialized_bytes, md5sum, pos, pytype = raw_msg
	msg = pytype()
	msg.deserialize(serialized_bytes)
	if filter_fn(topic, msg, t):
	outbag.write(topic, msg, t, connection_header=conn_header)

If not, the writer is possible to seek a wrong end.

ros_comm/tools/rosbag/src/rosbag/bag.py

Lines 605 to 606 in 030e132

	# Seek to end (in case previous operation was a read)
	self._file.seek(0, os.SEEK_END)

[Hypothesis-Z]

I tested it in native Ubuntu OS and everything is ok. It is supposed to be a WSL issue.