duologsync
(DLS) is a utility written by Duo Security that supports fetching logs from Duo endpoints and ingesting them to different SIEMs.
- Make sure you are running Python 3+ with
python --version
. - Clone this GitHub repository and navigate to the
duo_log_sync
folder. - Ensure you have "setuptools" by running
pip3 install setuptools
. - Install
duologsync
by runningpython/python3 setup.py install
. - Refer to the
Configuration
section below. You will need to create aconfig.yml
file and fill out credentials for the adminapi in the duoclient section as well as other parameters if necessary. - Run the application using
duologsync <complete/path/to/config.yml>
. - If a new version of DLS is downloaded from GitHub, run the setup command again to reinstall
duologsync
for changes to take effect.
- Duologsync is compatible with Python versions
3.6
,3.7
, and3.8
. - Duologsync is officially supported on Linux, MacOS, and Windows systems.
- On Windows operating systems,
duologsync
is installed in the\scripts\
folder under the Python installation in most cases.
- A logging filepath can be specified in
config.yml
. By default, logs will be stored under the/tmp
folder with nameduologsync.log
. - These logs are only application/system logs, and not the actual logs retrieved from Duo endpoints.
- Current version supports fetching logs from auth, telephony, admin, and trust monitor endpoints and sending over TCP, TCP Encrypted over SSL, and UDP to consuming systems.
- Ability to recover data by reading from last known offset through checkpointing files.
- Enabling only certain endpoints through config file.
- Choosing how logs are formatted (JSON, CEF).
- Support for Linux, MacOS, Windows.
- Support for pulling logs using Accounts API (only for MSP accounts).
- Adding more log endpoints.
- Adding better skey security.
- Adding CEF and MSP support for the Trust Monitor endpoint.
- See
template_config.yml
for an example and for extensive, in-depth config explanation.
- From time to time new features and fields will be added to the config file. Updating of the config file is mandatory when config changes are made. To make this easier, Duo has created a script called
upgrade_config.py
which will automatically update your old config for you. - To use the
upgrade_config.py
script, simply run the following command:python3 upgrade_config.py <old_config> <new_config>
where<old_config>
is the filepath or your old configuration file, and<new_config>
is where you would like the new configuration file to be saved. - The
upgrade_config.py
script will not delete your old config file, it will be preserved. - This script is a new feature and has to extrapolate some information, some unexpected issues may occur. For most old configs the script will work just fine. You can check if the new config file works by running it with DLS.
- The
is_msp
field under accounts section is required only when using DLS with the Accounts API. For this reason, the upgrade script won't create that field in new config by default.
- Calling Admin API handlers with Accounts API is mutually exclusive with cross-deployment sub-accounts. Many customers with sub-accounts (especially MSPs) must use cross-deployment sub-accounts and therefore can't use the Accounts API.
- Currently, the Trust Monitor endpoint only supports logging in JSON format, and does not support MSPs. Calling this endpoint (in combination with any other endpoints) using CEF format or MSPs will not allow the program to execute.