Calculating hashes' messages from a rosbag file

[laurafbec]

Description

Hi everyone I'm using rosbag2_py to create rosbag file and to read their content. I need to calculaate messages' hashes in both process (recording and reading). Nevertheless, I'm getting different hash value for what seems to be same messages. This is the snipped code of the recorder:

hash_value = None
print(msg)
print("type of message", type(msg))
serialized_message = serialize_message(msg)
print("serialized message: ", serialize_message)    

hash_value = hashlib.sha256(serialized_message).hexdigest()
print('Hash Value:', hash_value)

and the result I get:

[bag_recorder_tp_srv-1] tf2_msgs.msg.TFMessage(transforms=[geometry_msgs.msg.TransformStamped(header=std_msgs.msg.Header(stamp=builtin_interfaces.msg.Time(sec=173, nanosec=340000000), frame_id='torso_front_link'), child_frame_id='head_link', transform=geometry_msgs.msg.Transform(translation=geometry_msgs.msg.Vector3(x=0.01, y=0.0, z=0.94), rotation=geometry_msgs.msg.Quaternion(x=0.0, y=0.0, z=-5.270406777668768e-09, w=1.0))), geometry_msgs.msg.TransformStamped(header=std_msgs.msg.Header(stamp=builtin_interfaces.msg.Time(sec=173, nanosec=340000000), frame_id='base_torso_link'), child_frame_id='torso_front_link', transform=geometry_msgs.msg.Transform(translation=geometry_msgs.msg.Vector3(x=0.0, y=0.0, z=0.030000007233900814), rotation=geometry_msgs.msg.Quaternion(x=0.0, y=0.0, z=0.0, w=1.0))), geometry_msgs.msg.TransformStamped(header=std_msgs.msg.Header(stamp=builtin_interfaces.msg.Time(sec=173, nanosec=340000000), frame_id='head_link'), child_frame_id='eyes_link', transform=geometry_msgs.msg.Transform(translation=geometry_msgs.msg.Vector3(x=0.105, y=0.0, z=0.0), rotation=geometry_msgs.msg.Quaternion(x=0.0, y=4.2682817280592644e-09, z=0.0, w=1.0)))])
[bag_recorder_tp_srv-1] type of message <class 'tf2_msgs.msg._tf_message.TFMessage'>
[bag_recorder_tp_srv-1] serialized message:  <function serialize_message at 0x7f33bb3e8d30>
[bag_recorder_tp_srv-1] Hash Value: 18ba3aca3421dc3953f875bc446fae85f28d54c7ae70a25f8f044beff2249061

and this is the code I have in the reader

while reader.has_next():
        (topic, data, t) = reader.read_next()
        msg_type = get_message(type_map[topic])
        msg = deserialize_message(data, msg_type)
        print(msg)
        print("type of message", type(msg))
        serialized_message = serialize_message(msg)
        print("serialize_message", serialize_message)
        hash_value = hashlib.sha256(serialized_message).hexdigest()
        print('Hash Value:', hash_value)

and the result I get:

tf2_msgs.msg.TFMessage(transforms=[geometry_msgs.msg.TransformStamped(header=std_msgs.msg.Header(stamp=builtin_interfaces.msg.Time(sec=173, nanosec=340000000), frame_id='torso_front_link'), child_frame_id='head_link', transform=geometry_msgs.msg.Transform(translation=geometry_msgs.msg.Vector3(x=0.01, y=0.0, z=0.94), rotation=geometry_msgs.msg.Quaternion(x=0.0, y=0.0, z=-5.270406777668768e-09, w=1.0))), geometry_msgs.msg.TransformStamped(header=std_msgs.msg.Header(stamp=builtin_interfaces.msg.Time(sec=173, nanosec=340000000), frame_id='base_torso_link'), child_frame_id='torso_front_link', transform=geometry_msgs.msg.Transform(translation=geometry_msgs.msg.Vector3(x=0.0, y=0.0, z=0.030000007233900814), rotation=geometry_msgs.msg.Quaternion(x=0.0, y=0.0, z=0.0, w=1.0))), geometry_msgs.msg.TransformStamped(header=std_msgs.msg.Header(stamp=builtin_interfaces.msg.Time(sec=173, nanosec=340000000), frame_id='head_link'), child_frame_id='eyes_link', transform=geometry_msgs.msg.Transform(translation=geometry_msgs.msg.Vector3(x=0.105, y=0.0, z=0.0), rotation=geometry_msgs.msg.Quaternion(x=0.0, y=4.2682817280592644e-09, z=0.0, w=1.0)))])
type of message <class 'tf2_msgs.msg._tf_message.TFMessage'>    
serialize_message <function serialize_message at 0x7f77f08f6cb0>    
Hash Value: f44846787b3c8e4e6efe62a89902ff53f953f5148040ed584eb8993a9ad9841e

can anyone help me to find what I'm missing? I don't know if it's something related to the serialization problem. The fact is that it only happens with some messages, I dont know why.

System (please complete the following information)

• OS: [Ubuntu 22.04]
• ROS 2 Distro: Humble

[laurafbec]

I think that the problem is related with the use of the serialized message for the hashing. I have change the approach to use the string representation of the message instead but I dont know if there is a better solution.
Thanks in advance

[laurafbec]

Sorry, I would like to re-open this issue.
Could anyone provide me an explanation? thanks in advance.

[MichaelOrlov]

@laurafbec I am not sure about the internal implementation of the python hashlib.sha256(..) but it seems it expects that input should be a string, while the serialized_message is not a string.

Please double-check the serialized_message->buffer_length() and ``serialized_message->buffer_capacity()when you see two different hashes and make dump of theserialized_message->buffer` in binary format for both serialized messages. BTW the raw `serialized_message->buffer` likely may have end-of-line symbols somewhere in the middle.

This issue doesn't directly relate to the rosbag2 because rosbag2 in this case does not malfunction or have unexpected behaviour. There is something wrong with hashing or serialization.

[laurafbec]

Thanks @MichaelOrlov!! You're right. It's not a problem directly related to the rosbag2. It's something related to serialization. I just was looking for guidance. Thanks again!