snmp-notifier is creating 3 fields only

[honey2156]

func (trapSender TrapSender) generateVarBinds(alertGroup types.AlertGroup) (snmpgo.VarBinds, error) {
var (
varBinds snmpgo.VarBinds
)

trapUniqueID := strings.Join([]string{alertGroup.OID, "[", alertGroup.GroupID, "]"}, "")

descriptions, err := commons.FillTemplate(alertGroup, trapSender.configuration.DescriptionTemplate)
if err != nil {
	return nil, err
}

trapOid, _ := snmpgo.NewOid(alertGroup.OID)
varBinds = addUpTime(varBinds)
varBinds = append(varBinds, snmpgo.NewVarBind(snmpgo.OidSnmpTrap, trapOid))
varBinds = addStringSubOid(varBinds, alertGroup.OID, "1", trapUniqueID)
varBinds = addStringSubOid(varBinds, alertGroup.OID, "2", alertGroup.Severity)
varBinds = addStringSubOid(varBinds, alertGroup.OID, "3", *descriptions)

return varBinds, nil

}

@maxwo Here your code is adding 3 var-binds only. For our requirement, we need to handle more var-binds/SubOids. How can we do that? and when this feature can be assumed to added?

[maxwo]

Hey,
That's an interesting point. What are your other fields are intended to relay?
Currently, the snmp-notifier is intended to send traps that are receivable by legacy monitoring tools such as nagios, in which you can easily provide a simple MIB file that describes a simple alert, with ID, severity and text.
More informations on your use-case may be useful to answer your question, as, as you see, the implementation is rather straight-forward.

[bishwauser]

Hi @maxwo,

I too am facing similar kind of problem. Like my traps are having more fields then the three that you have mentioned (ID, severity and text) such as alertName, timeStamp, podName etc. In the similar way different products might have different set of info mentioned in the traps but, due to the constraint that @honey2156 has shown in the code snippet only those three objects are transmitted from snmp_notifier and the additional objects are clubbed together in the third field i.e. description.
I feel it will be really great if there is a way to customize this based on the MIB file of specific product.

Thanks

[maxwo]

Ok, I see your point.

The thing is adding alertName, podName and such may not make sense depending on the grouping rules: for instance, a given group of alerts may have several alertNames if alerts are grouped by, say, namespace and deployment (which makes perfect sense in some use-cases).

What I propose you is to add a subOID for each grouping key (which are viewable in between the [] in the trap ID). Therefore, it will be able to use them in your tools.

Therefore, a trap would look like:

 Agent Address: 0.0.0.0
 Agent Hostname: localhost
 Date: 1 - 0 - 0 - 1 - 1 - 1970
 Enterprise OID: .
 Trap Type: Cold Start
 Trap Sub-Type: 0
 Community/Infosec Context: TRAP2, SNMP v2c, community public
 Uptime: 0
 Description: Cold Start
 PDU Attribute/Value Pair Array:
.iso.org.dod.internet.mgmt.mib-2.system.sysUpTime.sysUpTimeInstance = Timeticks: (163624400) 18 days, 22:30:44.00
.iso.org.dod.internet.snmpV2.snmpModules.snmpMIB.snmpMIBObjects.snmpTrap.snmpTrapOID.0 = OID: .iso.org.dod.internet.private.enterprises.1234.0.10.1.1.1.2.1
.iso.org.dod.internet.private.enterprises.1234.0.10.1.1.1.2.1.1 = STRING: "1.3.6.1.4.1.1234.0.10.1.1.1.2.1[environment=production,label=test]"
.iso.org.dod.internet.private.enterprises.1234.0.10.1.1.1.2.1.2 = STRING: "critical"
.iso.org.dod.internet.private.enterprises.1234.0.10.1.1.1.2.1.3 = STRING: "Status: critical
- Alert: TestAlert
  Summary: this is the summary
  Description: this is the description on job1

Status: warning
- Alert: TestAlert
  Summary: this is the random summary
  Description: this is the description of alert 1"
.iso.org.dod.internet.private.enterprises.1234.0.10.1.1.1.2.1.4.1 = STRING: "production"
.iso.org.dod.internet.private.enterprises.1234.0.10.1.1.1.2.1.4.2 = STRING: "test"

Where the last 2 entries come from the grouped labels (1.3.6.1.4.1.1234.0.10.1.1.1.2.1[environment=production,label=test] in the trap ID).

Other than that, adding the alert start timestamp is easily doable.

@bishwauser @honey2156 What do you think?

[bishwauser]

Hey @maxwo

I have tried the following configurations in the Prometheus Alert rules where in I have added environment: test1, label: test2 as follows:

groups:
    - name: testAlerts
       rules:
       - alert: testAlert
         labels:
           AlertSeverity: "minor"
           AlertName: "testAlert1"
           oid: "1.3.6.1.4.1.1234.0.10.1.1.1.2.1"
           environment: test1
           label: test2
         annotations:
           AlertInfo: 'This is alert info'
           AlertText: "This is alert text" 

But this fails the post of the notifier, and I am not getting the sub oid in the output. I think that my configuration is not correct. could you show the me syntax.

Also I am using the snmptrapd tool as receiver
https://hub.docker.com/r/sig9/snmptrapd/

Thanks

[maxwo]

Could you please provide me more logs, from all elements (alertmanager, snmp gateway as well as snmptrapd?
There is a listen.sh script that launches a SNMP trap in the test directory if you wanna test as well.

[WrzocHoo]

Because of grouping "mechanism" of the AlertManager adding additional fields here are pointless, maybe except NumOfAlerts, but could not find any reasonable usage for that.

Unless mechanism to "split" bulk of events to set of traps instead of one collecting them all will be added.