Other transport layers

[dcferreira]

At the moment, transport layers that aren't TCP/UDP simply aren't exported.

I tried the following changes so that it supports OSPF, but couldn't get it to work:

• Added a layers.OSPFv3 attribute here (chose the v3 because it provides CanDecode() unlike just layers.OSPF --- not sure what's the difference to v2 though):
  

  go-flows/packet/buffer.go

  Lines 233 to 235 in a066f89

  	udp layers.UDP
  	icmpv4 ICMPv4Flow
  	icmpv6 ICMPv6Flow

• Added another case for layers.LayerTypeOSPF here, that is the same as TCP except for the first line (and things referencing TCP changed to OSPF):
  

  go-flows/packet/buffer.go

  Lines 336 to 348 in a066f89

  	case layers.LayerTypeTCP:
  	layer.(*layers.TCP).SetInternalPortsForTesting()
  	if pb.first != layers.LayerTypeEthernet && pb.first != layers.LayerTypeIPv4 && pb.first != layers.LayerTypeIPv6 {
  	pb.first = layers.LayerTypeTCP
  	}
  	if pb.transport != nil {
  	log.Panic("Can only assign one Transport Layer")
  	}
  	pb.transport = layer.(gopacket.TransportLayer)
  	if pb.proto == 0 {
  	pb.proto = uint8(layers.IPProtocolTCP)
  	}
  	case layers.LayerTypeICMPv4:

• Added another case for layers.LayerTypeOSPF here:
  

  go-flows/packet/buffer.go

  Lines 554 to 559 in a066f89

  	case layers.LayerTypeTCP:
  	decoder = &pb.tcp
  	case layers.LayerTypeICMPv4:
  	decoder = &pb.icmpv4
  	case layers.LayerTypeICMPv6:
  	decoder = &pb.icmpv6

• Added another case for layers.LayerTypeOSPF here:
  

  go-flows/packet/key.go

  Lines 67 to 72 in a066f89

  	case layers.LayerTypeUDP:
  	protoB = byte(layers.IPProtocolUDP)
  	case layers.LayerTypeICMPv4:
  	protoB = byte(layers.IPProtocolICMPv4)
  	case layers.LayerTypeICMPv6:
  	protoB = byte(layers.IPProtocolICMPv6)

It seems this is not enough to make it work though (although this was kind of a bruteforce attempt at a fix, since I don't fully understand the codebase).

I am wondering if:

• this is the way other transport layers should be added; there seems to be a lot of boilerplate if that is the case
• it would be possible to simply not decode the layer, when features don't require it. Right now I wanted to extract flows from some pcaps which contain about 130 different protocols; if I need to add each one to the flow exporter I'll simply ignore them, but if I could export them as long as the features don't require it, it would be better :)

[notti]

At the moment, transport layers that aren't TCP/UDP simply aren't exported.

1. Do you mean exposed as TransportLayer in packet?
2. Or are you not seeing the flows?

at 1.:
What you described above would do exactly that. Right now the transport layers that target the most IEs in the IANA ipfix spec are implemented (TCP, UDP, ICMPv4/6). All the other layers below, that are implemented, are the ones that are needed to being able to decode those.
The original Idea was that if features needed more, they should decode the rest themselves. Right now there is no generic way of doing that, but it shouldn't be hard to implement. Maybe add something to decode() that holds a pointer to the top most decoded layer (which would be IP for OSPF) which can be retrieved with some function. From there on one could continue with NextLayerType() and LayerPayload().
The reason this is done by hand was to get this part really fast (every packet has to go through this!). This works faster because no layers are ever allocated (they are part of the packet buffer) - instead the layers are reused everytime.

Maybe it we could also provide an additional slice in the packet buffer where one could add additionally decoded Layers so if multiple features use the same layer it only needs to be decoded once. This can then be done in a generic way (gopacket hast functions where you can daisy chain the NextLayer and decoding stuff) - or with some kind of filter.

at 2.:
Well this can be caused by the flow key. If for instance the port is part of the flow key this can never work since OSPF has no notion of ports. The same is true for every other protocol that has no ports.

[dcferreira]

Sorry, I wasn't clear. I have a file with empty flowkey, and only feature is packetTotalCount. When I run it on an OSPF packet, the csv exporter doesn't export anything.
I also tried with some other flowkeys (e.g., only source ip).

[notti]

Ah sorry thats then option 3.

try flipping the lines

go-flows/packet/buffer.go

Line 624 in a066f89

return false

go-flows/packet/buffer.go

Line 576 in a066f89

return false

to return true

These should actually return true (basically false would mean "I couldn't decode the packet" - But these two actually say "I couldn't find TCP,UDP,ICMP"). In the beginning this was needed, since the flow key didn't check if layers existed... Should be fixed by now. So hopefully nothing breaks if this is true...