Search indicators in Bulk fails
Closed this issue · 2 comments
Do you have a support subscription?
yes: fortifydata
Expected behavior and actual behavior.
When using the rest api, the /indicators POST endpoint should be able to run bulk search queries for multiple indicator searches. However, the query fails when you add more than one indicator to search for.
From what I tracked through the code, it seems to be the code is looking for a "limit" attribute on the list object passed in? So maybe the search aspect needs to check whether the search data is a list or a single object?
Steps to reproduce the problem
Spin up cif and either using CURL or the swagger endpoint, attempt to make a POST bulk search for more than one indicator.
An example curl is below.
curl -X POST "http://3.21.225.128:5000/indicators" -H "accept: application/json" -H "Content-Type: application/json" -d "[ { \"indicator\": \"88.15.65.0\", \"itype\": \"ipv4\", \"limit\": 500, \"nolog\": \"1\" }, { \"indicator\": \"124.118.197.129\", \"itype\": \"ipv4\", \"limit\": 500, \"nolog\": \"1\" }]"
As can be seen, I experimented with adding the "limit" parameter in there, as that seems to be causing the issues?
Relevant logs as a result of the actual behavior
cif-router | 2020-03-23 17:14:51,656 - ERROR - cif.store.handlers.indicator[93][MainThread] - 'list' object has no attribute 'limit'
cif-router | 2020-03-23 17:14:51,656 - ERROR - cif.store[109][MainThread] - invalid search
cif-router | Traceback (most recent call last):
cif-router | File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/store/handlers/indicator.py", line 87, in indicators_search
cif-router | yield from self.store.indicators.search(m.data)
cif-router | File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/store/sqlite/indicator.py", line 83, in search
cif-router | return self._search_bulk(filters).limit(500)
cif-router | AttributeError: 'list' object has no attribute 'limit'
cif-router |
cif-router | During handling of the above exception, another exception occurred:
cif-router |
cif-router | Traceback (most recent call last):
cif-router | File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/store/__init__.py", line 103, in _trigger_handler
cif-router | m.data = json.dumps(rv)
cif-router | File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/store/handlers/indicator.py", line 99, in indicators_search
cif-router | raise TypeError('invalid search')
cif-router | TypeError: invalid search
Specifications like the version of the project, operating system, or hardware.
Latest version, Ubuntu 18LTS, AWS EC2 t2.large, 2 cpus, 8gb ram, 25gb storage
How large is your /var/lib/cif.db database?
302M?
I think we’re just returning the wrong thing in the bulk search (a python list vs an sqlalchemy query). I need to write a test for that too cause I ran into the same thing..
pushing 5.0b8