fireeye/HXTool

dashboards not loading complete data

Closed this issue · 8 comments

Showing the following errors within the console when navigating to the predefined dashboards. This seems to be an issue on 4.6 and the latest from a git clone.

Controller Version: HX 5.0.2.921836 2020-09-15 16:25:29 (Hotfix).

[2020-12-03` 14:51:48,320] {app} {Thread-106} ERROR - Exception on /api/v1/datatable_acqs_host [GET]
Traceback (most recent call last):
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib64/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/xxxxxxxxxxx/HX-Tool/hxtool_util.py", line 51, in is_session_valid
    ret = f(*args, **kwargs)
  File "/home/xxxxxxxxxxx/HX-Tool/hxtool_api.py", line 2235, in datatable_acqs_host
    "request_time": HXAPI.dt_to_str(HXAPI.gtNoUs(acq['request_time'])),
  File "/home/xxxxxxxxxxx/HX-Tool/hx_lib.py", line 1217, in gtNoUs
    dt = dt_str[0:(len(dt_str) - 5)]
TypeError: object of type 'NoneType' has no len()
[2020-12-03 14:54:02,276] {app} {Thread-123} ERROR - Exception on /api/v1/chartjs_agentstatus [GET]
Traceback (most recent call last):
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib64/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/xxxxxxxxxxx/HX-Tool/hxtool_util.py", line 51, in is_session_valid
    ret = f(*args, **kwargs)
  File "/home/xxxxxxxxxxx/HX-Tool/hxtool_api.py", line 2484, in chartjs_agentstatus
    for host in response_data['data']['entries']:
TypeError: string indices must be integers
[2020-12-03 14:54:02,370] {app} {Thread-124} ERROR - Exception on /api/v1/chartjs_agentstatus [GET]
Traceback (most recent call last):
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib64/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/xxxxxxxxxxx/HX-Tool/hxtool_util.py", line 51, in is_session_valid
    ret = f(*args, **kwargs)
  File "/home/xxxxxxxxxxx/HX-Tool/hxtool_api.py", line 2484, in chartjs_agentstatus
    for host in response_data['data']['entries']:
TypeError: string indices must be integers
[2020-12-03 14:54:02,482] {app} {Thread-125} ERROR - Exception on /api/v1/chartjs_agentstatus [GET]
Traceback (most recent call last):
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib64/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/xxxxxxxxxxx/HX-Tool/hxtool_util.py", line 51, in is_session_valid
    ret = f(*args, **kwargs)
  File "/home/xxxxxxxxxxx/HX-Tool/hxtool_api.py", line 2484, in chartjs_agentstatus
    for host in response_data['data']['entries']:
TypeError: string indices must be integers
[2020-12-03 14:54:02,569] {app} {Thread-126} ERROR - Exception on /api/v1/chartjs_agentstatus [GET]
Traceback (most recent call last):
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib64/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/xxxxxxxxxxx/HX-Tool/hxtool_util.py", line 51, in is_session_valid
    ret = f(*args, **kwargs)
  File "/home/xxxxxxxxxxx/HX-Tool/hxtool_api.py", line 2484, in chartjs_agentstatus
    for host in response_data['data']['entries']:
TypeError: string indices must be integers
[2020-12-03 14:54:02,676] {app} {Thread-127} ERROR - Exception on /api/v1/chartjs_agentstatus [GET]
Traceback (most recent call last):
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib64/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/xxxxxxxxxxx/HX-Tool/hxtool_util.py", line 51, in is_session_valid
    ret = f(*args, **kwargs)
  File "/home/xxxxxxxxxxx/HX-Tool/hxtool_api.py", line 2484, in chartjs_agentstatus
    for host in response_data['data']['entries']:
TypeError: string indices must be integers
[2020-12-03 14:54:02,759] {app} {Thread-128} ERROR - Exception on /api/v1/chartjs_agentstatus [GET]
Traceback (most recent call last):
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib64/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/xxxxxxxxxxx/HX-Tool/hxtool_util.py", line 51, in is_session_valid
    ret = f(*args, **kwargs)
  File "/home/xxxxxxxxxxx/HX-Tool/hxtool_api.py", line 2484, in chartjs_agentstatus
    for host in response_data['data']['entries']:
TypeError: string indices must be integers
[2020-12-03 14:54:03,046] {app} {Thread-99} ERROR - Exception on /api/v1/chartjs_malwarestatus [GET]
Traceback (most recent call last):
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib64/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/xxxxxxxxxxx/HX-Tool/hxtool_util.py", line 51, in is_session_valid
    ret = f(*args, **kwargs)
  File "/home/xxxxxxxxxxx/HX-Tool/hxtool_api.py", line 2645, in chartjs_malwarestatus
    if 'MalwareProtectionStatus' in sresponse_data['data'].keys():
TypeError: string indices must be integers
[2020-12-03 14:56:02,951] {app} {Thread-138} ERROR - Exception on /api/v1/chartjs_agentstatus [GET]
Traceback (most recent call last):
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib64/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/xxxxxxxxxxx/HX-Tool/hxtool_util.py", line 51, in is_session_valid
    ret = f(*args, **kwargs)
  File "/home/xxxxxxxxxxx/HX-Tool/hxtool_api.py", line 2484, in chartjs_agentstatus
    for host in response_data['data']['entries']:
TypeError: string indices must be integers
[2020-12-03 14:56:02,952] {app} {Thread-135} ERROR - Exception on /api/v1/chartjs_agentstatus [GET]
Traceback (most recent call last):
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib64/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib64/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/xxxxxxxxxxx/HX-Tool/hxtool_util.py", line 51, in is_session_valid
    ret = f(*args, **kwargs)
  File "/home/xxxxxxxxxxx/HX-Tool/hxtool_api.py", line 2484, in chartjs_agentstatus
    for host in response_data['data']['entries']:
TypeError: string indices must be integers
B0fH commented

Hi @anunknownplayer -
Are you using an account with the api_admin or api_analyst role?

As an aside, it looks like chartjs_agentstatus() doesn't seem to check if the HX API call was successful, I'll go ahead and fix that.

Thanks,
Elazar

Hi @anunknownplayer -
Are you using an account with the api_admin or api_analyst role?

Using the api_admin as to not run into any types of issues while we look at this tool

B0fH commented

Thanks! Roughly how big is your agent population? We've seen some reports regarding API timeouts with large agent populations and HX 5.0.2.

Thanks,
Elazar

we're approaching 100K systems within the console.

B0fH commented

That might be why. A full /hosts listing on a loaded controller can be pretty expensive - I'd take a look at the controller logs to see if there's any timeouts or 503s noted.

Thanks,
Elazar

Alright i'll get support to look at the controllers logs.

B0fH commented

Hi @anunknownplayer -
Is this still an issues?

Thanks,
Elazar

Hi @anunknownplayer -
Is this still an issues?

Thanks,
Elazar

Not seeing any errors showing up, but not all dashboards load still. There seems to be no indication that its currently working on fetching the data either so it's hard to tell at this point.