Legacy functions do not work out of the box after upgrade to 2.4

Question

Legacy functions do not work out of the box after upgrade to 2.4

lennartkoopmann opened this issue 7 years ago · 0 comments

I am using pipeline rules from before the migration to lookup tables:

rule "Threat Intelligence lookups"
when
  has_field("src_addr") && has_field("dst_addr")
then
  set_fields(threat_intel_lookup_ip(to_string($message.src_addr), "src_addr"));
  set_fields(threat_intel_lookup_ip(to_string($message.dst_addr), "dst_addr"));
end

This leads to all lookups failing and these error messages:

2017-09-27T13:58:45.692-05:00 WARN  [LookupTableService] Lookup table <abuse-ch-ransomware-ip> does not exist
2017-09-27T13:58:45.692-05:00 WARN  [LookupTableService] Lookup table <spamhaus-drop> does not exist
2017-09-27T13:58:45.693-05:00 WARN  [LookupTableService] Lookup table <alienvault-otx-ip> does not exist
2017-09-27T13:58:45.693-05:00 WARN  [LookupTableService] Lookup table <whois> does not exist
2017-09-27T13:58:45.693-05:00 WARN  [LookupTableService] Lookup table <tor-exit-node-list> does not exist
2017-09-27T13:58:45.693-05:00 WARN  [LookupTableService] Lookup table <abuse-ch-ransomware-domains> does not exist
2017-09-27T13:58:46.712-05:00 WARN  [LookupTableService] Lookup table <abuse-ch-ransomware-ip> does not exist
2017-09-27T13:58:46.712-05:00 WARN  [LookupTableService] Lookup table <spamhaus-drop> does not exist
2017-09-27T13:58:46.712-05:00 WARN  [LookupTableService] Lookup table <alienvault-otx-ip> does not exist
2017-09-27T13:58:46.712-05:00 WARN  [LookupTableService] Lookup table <whois> does not exist
2017-09-27T13:58:46.712-05:00 WARN  [LookupTableService] Lookup table <tor-exit-node-list> does not exist
2017-09-27T13:58:46.712-05:00 WARN  [LookupTableService] Lookup table <abuse-ch-ransomware-domains> does not exist
2017-09-27T13:58:47.688-05:00 WARN  [LookupTableService] Lookup table <abuse-ch-ransomware-ip> does not exist
2017-09-27T13:58:47.688-05:00 WARN  [LookupTableService] Lookup table <spamhaus-drop> does not exist
2017-09-27T13:58:47.688-05:00 WARN  [LookupTableService] Lookup table <alienvault-otx-ip> does not exist
2017-09-27T13:58:47.688-05:00 WARN  [LookupTableService] Lookup table <whois> does not exist
2017-09-27T13:58:47.688-05:00 WARN  [LookupTableService] Lookup table <tor-exit-node-list> does not exist
2017-09-27T13:58:47.689-05:00 WARN  [LookupTableService] Lookup table <abuse-ch-ransomware-domains> does not exist
2017-09-27T13:58:48.695-05:00 WARN  [LookupTableService] Lookup table <abuse-ch-ransomware-ip> does not exist
2017-09-27T13:58:48.695-05:00 WARN  [LookupTableService] Lookup table <spamhaus-drop> does not exist
2017-09-27T13:58:48.695-05:00 WARN  [LookupTableService] Lookup table <alienvault-otx-ip> does not exist
2017-09-27T13:58:48.695-05:00 WARN  [LookupTableService] Lookup table <whois> does not exist
2017-09-27T13:58:48.695-05:00 WARN  [LookupTableService] Lookup table <tor-exit-node-list> does not exist
2017-09-27T13:58:48.695-05:00 WARN  [LookupTableService] Lookup table <abuse-ch-ransomware-domains> does not exist
2017-09-27T13:58:49.689-05:00 WARN  [LookupTableService] Lookup table <abuse-ch-ransomware-ip> does not exist
2017-09-27T13:58:49.689-05:00 WARN  [LookupTableService] Lookup table <spamhaus-drop> does not exist
2017-09-27T13:58:49.690-05:00 WARN  [LookupTableService] Lookup table <alienvault-otx-ip> does not exist
2017-09-27T13:58:49.690-05:00 WARN  [LookupTableService] Lookup table <whois> does not exist
2017-09-27T13:58:49.690-05:00 WARN  [LookupTableService] Lookup table <tor-exit-node-list> does not exist
2017-09-27T13:58:49.690-05:00 WARN  [LookupTableService] Lookup table <abuse-ch-ransomware-domains> does not exist
2017-09-27T13:58:50.691-05:00 WARN  [LookupTableService] Lookup table <abuse-ch-ransomware-ip> does not exist
2017-09-27T13:58:50.691-05:00 WARN  [LookupTableService] Lookup table <spamhaus-drop> does not exist
2017-09-27T13:58:50.691-05:00 WARN  [LookupTableService] Lookup table <alienvault-otx-ip> does not exist
2017-09-27T13:58:50.691-05:00 WARN  [LookupTableService] Lookup table <whois> does not exist
2017-09-27T13:58:50.691-05:00 WARN  [LookupTableService] Lookup table <tor-exit-node-list> does not exist
2017-09-27T13:58:50.691-05:00 WARN  [LookupTableService] Lookup table <abuse-ch-ransomware-domains> does not exist
2017-09-27T13:58:51.693-05:00 WARN  [LookupTableService] Lookup table <abuse-ch-ransomware-ip> does not exist
2017-09-27T13:58:51.693-05:00 WARN  [LookupTableService] Lookup table <spamhaus-drop> does not exist
2017-09-27T13:58:51.693-05:00 WARN  [LookupTableService] Lookup table <alienvault-otx-ip> does not exist
2017-09-27T13:58:51.693-05:00 WARN  [LookupTableService] Lookup table <whois> does not exist
2017-09-27T13:58:51.693-05:00 WARN  [LookupTableService] Lookup table <tor-exit-node-list> does not exist
2017-09-27T13:58:51.693-05:00 WARN  [LookupTableService] Lookup table <abuse-ch-ransomware-domains> does not exist
2017-09-27T13:58:52.697-05:00 WARN  [LookupTableService] Lookup table <abuse-ch-ransomware-ip> does not exist
2017-09-27T13:58:52.697-05:00 WARN  [LookupTableService] Lookup table <spamhaus-drop> does not exist
2017-09-27T13:58:52.697-05:00 WARN  [LookupTableService] Lookup table <alienvault-otx-ip> does not exist
2017-09-27T13:58:52.697-05:00 WARN  [LookupTableService] Lookup table <whois> does not exist
2017-09-27T13:58:52.697-05:00 WARN  [LookupTableService] Lookup table <tor-exit-node-list> does not exist
2017-09-27T13:58:52.697-05:00 WARN  [LookupTableService] Lookup table <abuse-ch-ransomware-domains> does not exist

I do see them in lut_tables though:

...
{
        "_id" : ObjectId("59cbf313da4e6a5f9cd778f0"),
        "title" : "Spamhaus DROP",
        "description" : "This is the lookup table for Spamhaus' DROP (Don't Route Or Peer) list, containing netblocks which are \"hijacked\" or leased by professional spam or cyber-crime operations. For more information see https://www.spamhaus.org/drop. This lookup table is used internally by Graylog's Threat Intel Plugin. Do not delete it manually.",
        "name" : "spamhaus-drop",
        "cache" : ObjectId("59cbf313da4e6a5f9cd778e7"),
        "data_adapter" : ObjectId("59cbf313da4e6a5f9cd778ed"),
        "content_pack" : "59cbf312da4e6a5f9cd778e0",
        "default_single_value" : "",
        "default_single_value_type" : "NULL",
        "default_multi_value" : "",
        "default_multi_value_type" : "NULL"
}
{
        "_id" : ObjectId("59cbf313da4e6a5f9cd778f1"),
        "title" : "abuse.ch Ransomware Domains",
        "description" : "This is the lookup table for the abuse.ch ransomware Domain Tracker, listing infrastructure by domain names which are used for ransomware. For more information see https://ransomwaretracker.abuse.ch. This lookup table is use
d internally by Graylog's Threat Intel Plugin. Do not delete it manually.",
        "name" : "abuse-ch-ransomware-domains",
        "cache" : ObjectId("59cbf313da4e6a5f9cd778e2"),
        "data_adapter" : ObjectId("59cbf313da4e6a5f9cd778ec"),
        "content_pack" : "59cbf312da4e6a5f9cd778e0",
        "default_single_value" : "",
        "default_single_value_type" : "NULL",
        "default_multi_value" : "",
        "default_multi_value_type" : "NULL"
}
{
        "_id" : ObjectId("59cbf313da4e6a5f9cd778f2"),
        "title" : "abuse.ch Ransomware IP",
        "description" : "This is the lookup table for the abuse.ch ransomware IP Tracker, listing infrastructure by IP which is used for ransomware. For more information see https://ransomwaretracker.abuse.ch. This lookup table is used internally by
 Graylog's Threat Intel Plugin. Do not delete it manually.",
        "name" : "abuse-ch-ransomware-ip",
        "cache" : ObjectId("59cbf313da4e6a5f9cd778e2"),
        "data_adapter" : ObjectId("59cbf313da4e6a5f9cd778e9"),
        "content_pack" : "59cbf312da4e6a5f9cd778e0",
        "default_single_value" : "",
        "default_single_value_type" : "NULL",
        "default_multi_value" : "",
        "default_multi_value_type" : "NULL"
}
{
        "_id" : ObjectId("59cbf313da4e6a5f9cd778f3"),
        "title" : "Tor Exit Node List",
        "description" : "This is the lookup table for the TOR (The Onion Router) Exit Node List, listing Exit Nodes of the TOR Network . This lookup table is used internally by Graylog's Threat Intel Plugin. Do not delete it manually.",
        "name" : "tor-exit-node-list",
        "cache" : ObjectId("59cbf313da4e6a5f9cd778e4"),
        "data_adapter" : ObjectId("59cbf313da4e6a5f9cd778ea"),
        "content_pack" : "59cbf312da4e6a5f9cd778e0",
        "default_single_value" : "",
        "default_single_value_type" : "NULL",
        "default_multi_value" : "",
        "default_multi_value_type" : "NULL"
}
...