Kibana Visualization error "[esaggs] > Bad Request"

[massaquah]

Hello,
thank you for updating the pfelk-repo!
I just tried it out, following the steps here and here but ended up with an empty dashboard with an error mark at each visualization saying:
"[esaggs] > Bad Request"

Only the vis. Firewall - Discover is working.

I can confirm that data is coming in and under "discover" in Kibana I can see a long list of fields and parsed events.

[a3ilson]

@massaquah - To confirm, you conducted a clean/new install or update? I just fired up a fresh instance imported the templates and dashboard without issue. I've seen this error when conflicting data was already stored and resolved by deleting and reloading the saved objects.

I would suggest the following:

• Confirm indices are loaded (see image below)
• Delete all Kibana saved objects
• Delete all indices
• import saved objects (kibana)

[massaquah]

I can confirm that I did a clean install.

I just tried it again

• stopping all containers
• doing a "docker system prune --all --volumes"
• removing the pfelk directory where I unzipped pfelkdocker.zip

then

• pulling and bringing up the fresh containers
• importing the templates
• importing the dashboards
  BAM

Now I get another error at this point:

Just to confirm that index templates have been imported correctly:

[massaquah]

The Dashboard version from 4 days ago can be imported without any error.
But then again, I get the empty dashboard with the error mentioned in the title....

[massaquah]

I had a deeper look into one of the visualizations and found this error. Maybe that helps?

EDIT: Solution here? https://stackoverflow.com/questions/38145991/how-to-set-fielddata-true-in-kibana

[a3ilson]

@massaquah - I went through an installed per the GitHub instructions and posted a quick overview view here.

I made some minor tweaks within the repo, to align with elastic.co and posted those updates which were utilized in the quick install tutorial video.

I only had an issue/error with the snort dashboard...all other objects imported without issue. I'll look into the snort object but suspect the defined fields within the snort ndjson file are conflicting with others.

I am running OPNsense and logging everything to pfELK. I am not using BEATS, Snort, or HAProxy.

Curious as to what the specific issue. Please provide insight into which saved objects you are importing.

[massaquah]

@a3ilson , thank you for the video. I did a clean install again and cautiously followed the video only to realize that my previous attempts followed the exact same steps.

Again, I ended up with an empty dashboard and the error "[esaggs] > Bad Request" in each box.

Then I investigated a little more and observed a potential mismatch between the index template and the actual index:

Template (pfelk-firewall) for field event.action:

"event": {
          "type": "object",
          "properties": {
            "reason": {
              "type": "keyword"
            },
            "action": {
              "type": "keyword"
            }
          }
        },

And then the index that was created after sending events from OPNsense to pfelk:

"event": {
          "properties": {
            "action": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },

Why do I end up with type "text"? :(((

[massaquah]

I found the same issue here:
wazuh/wazuh-dashboard-plugins#2420 (comment)

and the solution here:
wazuh/wazuh-dashboard-plugins#2420 (comment)

So I just stopped OPNsense output, deleted the index and restarted the output. Now dashboard looks fine.

[a3ilson]

@massaquah - 👍

I should have been more explicit with my initial response. If the logs are being sent prior to the importing of the templates, the field types will not be properly identified resulting in the described error.

However, I'm glad you were able to solve and will close this issue - thanks!