Hearbeat messages from endpoints using ServiceControl.Plugin.Nsb5.Heartbeat are not ingested

[ramonsmits]

Symptoms

• Endpoints using ServiceControl.Plugin.Nsb5.Heartbeat are shown as inactive in ServicePulse.
• Heartbeat related messages are sent to the ServiceControl error queue.

Who's affected

You are affected if you are using both:

• ServiceControl 5.2.x
• ServiceControl.Plugin.Nsb5.Heartbeat.

Root cause

The cause is that the messages from ServiceControl.Plugin.Nsb5.Heartbeat have an JSON array definition. In ServiceControl prior to 5.2 NServiceBus 7 was uses together with the NServiceBus.Newtonsoft.Json and that implementation has supports arrays that only contain a single message as if there is no array

Describe the bug

Messages sent from ServiceControl.Plugin.Nsb5.Heartbeat fail with the following error:

2024-05-29 12:37:39.3546|18|Error|NServiceBus.MoveToError|Moving message '31ceab04-35a6-4156-96d7-b17f00d01968' to the error queue 'Particular.ServiceControl.Errors' because processing failed due to an exception:|NServiceBus.MessageDeserializationException: An error occurred while attempting to extract logical messages from incoming physical message 31ceab04-35a6-4156-96d7-b17f00d01968
---> System.Text.Json.JsonException: The JSON value could not be converted to ServiceControl.Plugin.Heartbeat.Messages.EndpointHeartbeat. Path: $ | LineNumber: 0 | BytePositionInLine: 1.

Payload:

[
  {
    "$type": "ServiceControl.Plugin.Heartbeat.Messages.EndpointHeartbeat, ServiceControl",
    "ExecutedAt": "2024-06-02T20:37:37.1948945Z",
    "EndpointName": "Test",
    "HostId": "d1caa28e-2522-fbaa-e9ff-9098654faaf5",
    "Host": "XXXXX"
  }
]

Cause

The cause is that the messages from ServiceControl.Plugin.Nsb5.Heartbeat have an JSON array definition. In ServiceControl prior to 5.2 NServiceBus 7 was uses together with the NServiceBus.Newtonsoft.Json and that implementation has supports arrays that only contain a single message as if there is no array:

• https://github.com/Particular/NServiceBus.Newtonsoft.Json/blob/2.4.0/src/NServiceBus.Newtonsoft.Json/JsonMessageSerializer.cs#L81-L112

The NServiceBus System.Text.Json serializer does not support this:

• https://github.com/Particular/NServiceBus/blob/master/src/NServiceBus.Core/Serializers/SystemJson/JsonMessageSerializer.cs

Versions

ServiceControl 5.2.0 - 5.2.2

Please list the version of the relevant packages or applications in which the bug exists.

Steps to reproduce

1. Create an NServiceBus 5 endpoints that uses ServiceControl.Plugin.Nsb5.Heartbeat
2. Setup an ServiceControl primary instance
3. Observe failures in the ServiceControl log

Relevant log output

2024-05-29 12:37:39.3546|18|Error|NServiceBus.MoveToError|Moving message '31ceab04-35a6-4156-96d7-b17f00d01968' to the error queue 'Particular.ServiceControl.Errors' because processing failed due to an exception:|NServiceBus.MessageDeserializationException: An error occurred while attempting to extract logical messages from incoming physical message 31ceab04-35a6-4156-96d7-b17f00d01968
---> System.Text.Json.JsonException: The JSON value could not be converted to ServiceControl.Plugin.Heartbeat.Messages.EndpointHeartbeat. Path: $ | LineNumber: 0 | BytePositionInLine: 1.
at System.Text.Json.ThrowHelper.ThrowJsonException_DeserializeUnableToConvertValue(Type propertyType)
at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.ContinueDeserialize(ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack)
at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Stream utf8Json)
at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsObject(Stream utf8Json)
at NServiceBus.Serializers.SystemJson.JsonMessageSerializer.Deserialize(ReadOnlyMemory`1 body, Type type) in /_/src/NServiceBus.Core/Serializers/SystemJson/JsonMessageSerializer.cs:line 53
at NServiceBus.Serializers.SystemJson.JsonMessageSerializer.Deserialize(ReadOnlyMemory`1 body, IList`1 messageTypes) in /_/src/NServiceBus.Core/Serializers/SystemJson/JsonMessageSerializer.cs:line 41
at NServiceBus.DeserializeMessageConnector.Extract(IncomingMessage physicalMessage) in /_/src/NServiceBus.Core/Pipeline/Incoming/DeserializeMessageConnector.cs:line 118
at NServiceBus.DeserializeMessageConnector.ExtractWithExceptionHandling(IncomingMessage message) in /_/src/NServiceBus.Core/Pipeline/Incoming/DeserializeMessageConnector.cs:line 46
--- End of inner exception stack trace ---

Additional Information

Possible solutions

Additional information

This only happens for components that are no longer supported. However, the platform tools still support platform messages from these legacy endpoints.

Out of support	Version	Component
2018-02-08	1.x	ServiceControl.Plugin.Nsb5.Heartbeat
2018-10-11	5.x	NServiceBus

[jpalac]

Fixed in ServiceControl 5.2.4