Filebeat can't start, with default /etc/default/filebeat file on OS without systemd

Question

Filebeat can't start, with default /etc/default/filebeat file on OS without systemd

stivi21 opened this issue 4 years ago · 4 comments

Beats product: Filebeat

Beats version 7.8.0

Role version: 12ef312

OS version Linux xxx 3.16.7-ckt9-drt #4 SMP Sun Apr 19 20:59:40 CEST 2015 x86_64 GNU/Linux
Debian GNU/Linux 8.0 (jessie)

Description of the problem including expected versus actual behaviour:
When I install filebeat on this OS (this version OS don't have systemd), filebeat can't start, bacause have wrong paths:

# /etc/init.d/filebeat start
[....] Starting Filebeat sends log files to Logstash or directly to Elasticsearch.: filebeat2020-06-26T14:21:30.225+0200 INFO instance/beat.go:647 Home path: [/usr/share/filebeat/bin] Config path: [/usr/share/filebeat/bin] Data path: [/usr/share/filebeat/bin/data] Logs path: [/usr/share/filebeat/bin/logs]

When I remove /etc/default/filebeat, dilebeat starting properly.
In this file is variable DAEMON_ARGS="-c /etc/filebeat/filebeat.yml ".
In file /etc/init.d/filebeat is also the same variable in property values:
DAEMON_ARGS="-c /etc/${NAME}/${NAME}.yml -path.home /usr/share/${NAME} -path.config /etc/${NAME} -path.data /var/lib/${NAME} -path.logs /var/log/${NAME}", but after it file /etc/default/filebeat is loading, so variable DAEMON_ARGS is from this file.

I think, will be good, if can be possible disable copy file to /etc/default/, or cat line with DAEMON_ARGS in templates/beat.j2.
There is no problem on Debian 9 or 10, because have systemd.

Playbook:

name: Filebeat with custom configuration
hosts: all
roles:

role: elastic.beats
vars:
version_lock: true
beat: filebeat
beat_conf:
filebeat.modules:

module: system
output_conf: {"logstash":{"hosts":["logstash1.domain:5044"]}}
logging_conf: {"to_files": true, "metrics.enabled": false}

Provide logs from Ansible:


PLAY [Filebeat with custom configuration] ***************************************************************************************************************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************************************************************************************************************************************
ok: [drt-7.domain]

TASK [elastic.beats : check-parameters] *****************************************************************************************************************************************************************************************************
included: /home/kmoscick/.ansible/roles/elastic.beats/tasks/beats-param-check.yml for drt-7.domain

TASK [elastic.beats : Check beat variable] **************************************************************************************************************************************************************************************************
skipping: [drt-7.domain]

TASK [elastic.beats : Check beat_conf variable] *********************************************************************************************************************************************************************************************
skipping: [drt-7.domain]

TASK [elastic.beats : Check ILM variables] **************************************************************************************************************************************************************************************************
skipping: [drt-7.domain]

TASK [elastic.beats : Set beats_major_version] **********************************************************************************************************************************************************************************************
ok: [drt-7.domain]

TASK [elastic.beats : define beat product] **************************************************************************************************************************************************************************************************
ok: [drt-7.domain]

TASK [elastic.beats : os-specific vars] *****************************************************************************************************************************************************************************************************
ok: [drt-7.domain]

TASK [elastic.beats : include_tasks] ********************************************************************************************************************************************************************************************************
included: /home/kmoscick/.ansible/roles/elastic.beats/tasks/beats.yml for drt-7.domain

TASK [elastic.beats : Include specific Beats] ***********************************************************************************************************************************************************************************************
included: /home/kmoscick/.ansible/roles/elastic.beats/tasks/beats-debian.yml for drt-7.domain

TASK [elastic.beats : Debian - Ensure apt-transport-https is installed] *********************************************************************************************************************************************************************
ok: [drt-7.domain]

TASK [elastic.beats : Debian - Ensure python-urllib3, python-openssl, python-pyasn1 & python-pip are installed] *****************************************************************************************************************************
skipping: [drt-7.domain]

TASK [elastic.beats : Debian - ensure ndg-httpsclient pip is installed] *********************************************************************************************************************************************************************
skipping: [drt-7.domain]

TASK [elastic.beats : Debian - Add Beats repository key] ************************************************************************************************************************************************************************************
ok: [drt-7.domain]

TASK [elastic.beats : Debian - add beats repository] ****************************************************************************************************************************************************************************************
ok: [drt-7.domain]

TASK [elastic.beats : Debian - unhold filebeat version for install] *************************************************************************************************************************************************************************
ok: [drt-7.domain]

TASK [elastic.beats : Debian - Ensure filebeat is installed] ********************************************************************************************************************************************************************************
ok: [drt-7.domain]

TASK [elastic.beats : Debian - hold filebeat version] ***************************************************************************************************************************************************************************************
ok: [drt-7.domain]

TASK [elastic.beats : Set os_arch] **********************************************************************************************************************************************************************************************************
ok: [drt-7.domain]

TASK [elastic.beats : Debian - Download filebeat from url] **********************************************************************************************************************************************************************************
skipping: [drt-7.domain]

TASK [elastic.beats : Debian - Ensure filebeat is installed from downloaded package] ********************************************************************************************************************************************************
skipping: [drt-7.domain]

TASK [elastic.beats : Include specific Beats] ***********************************************************************************************************************************************************************************************
skipping: [drt-7.domain]

TASK [elastic.beats : Beats configuration] **************************************************************************************************************************************************************************************************
included: /home/kmoscick/.ansible/roles/elastic.beats/tasks/beats-config.yml for drt-7.domain

TASK [elastic.beats : Set default facts] ****************************************************************************************************************************************************************************************************
ok: [drt-7.domain]

TASK [elastic.beats : Set beat_shipper_conf] ************************************************************************************************************************************************************************************************
skipping: [drt-7.domain]

TASK [elastic.beats : Set beat_logging_conf] ************************************************************************************************************************************************************************************************
ok: [drt-7.domain]

TASK [elastic.beats : Check pid_dir status] *************************************************************************************************************************************************************************************************
ok: [drt-7.domain]

TASK [elastic.beats : Create PID Directory] *************************************************************************************************************************************************************************************************
skipping: [drt-7.domain]

TASK [elastic.beats : Create Config Directory] **********************************************************************************************************************************************************************************************
ok: [drt-7.domain]

TASK [elastic.beats : Copy Default File for Instance] ***************************************************************************************************************************************************************************************
ok: [drt-7.domain]

TASK [elastic.beats : Copy Configuration File for filebeat] *********************************************************************************************************************************************************************************
changed: [drt-7.domain]

TASK [elastic.beats : Create default policies config directory] *****************************************************************************************************************************************************************************
skipping: [drt-7.domain]

TASK [elastic.beats : Copy default ILM policy file for filebeat] ****************************************************************************************************************************************************************************
skipping: [drt-7.domain]

TASK [elastic.beats : Start filebeat service] ***********************************************************************************************************************************************************************************************
fatal: [drt-7.domain]: FAILED! => {"changed": false, "msg": "Failed to start service: filebeat", "rc": 1, "stderr": "2020-06-26T16:05:35.653+0200\tINFO\tinstance/beat.go:647\tHome path: [/usr/share/filebeat/bin] Config path: [/usr/share/filebeat/bin] Data path: [/usr/share/filebeat/bin/data] Logs path: [/usr/share/filebeat/bin/logs]\n2020-06-26T16:05:35.654+0200\tINFO\tinstance/beat.go:655\tBeat ID: 75deba96-a4c9-47a6-a04d-1b03b2f078a7\n2020-06-26T16:05:35.654+0200\tINFO\t[beat]\tinstance/beat.go:983\tBeat info\t{\"system_info\": {\"beat\": {\"path\": {\"config\": \"/usr/share/filebeat/bin\", \"data\": \"/usr/share/filebeat/bin/data\", \"home\": \"/usr/share/filebeat/bin\", \"logs\": \"/usr/share/filebeat/bin/logs\"}, \"type\": \"filebeat\", \"uuid\": \"75deba96-a4c9-47a6-a04d-1b03b2f078a7\"}}}\n2020-06-26T16:05:35.654+0200\tINFO\t[beat]\tinstance/beat.go:992\tBuild info\t{\"system_info\": {\"build\": {\"commit\": \"f79387d32717d79f689d94fda1ec80b2cf285d30\", \"libbeat\": \"7.8.0\", \"time\": \"2020-06-14T18:15:37.000Z\", \"version\": \"7.8.0\"}}}\n2020-06-26T16:05:35.654+0200\tINFO\t[beat]\tinstance/beat.go:995\tGo runtime info\t{\"system_info\": {\"go\": {\"os\":\"linux\",\"arch\":\"amd64\",\"max_procs\":32,\"version\":\"go1.13.10\"}}}\n2020-06-26T16:05:35.713+0200\tINFO\t[beat]\tinstance/beat.go:999\tHost info\t{\"system_info\": {\"host\": {\"architecture\":\"x86_64\",\"boot_time\":\"2020-06-25T12:15:27+02:00\",\"containerized\":false,\"name\":\"drt-7\",\"ip\":[\"127.0.0.1/8\",\"x.x.x.x/25\"],\"kernel_version\":\"3.16.7-ckt9-drt\",\"mac\":[\"xx:xx:xx:xx:xx:xx\",\"xx:xx:xx:xx:xx:xx\"],\"os\":{\"family\":\"debian\",\"platform\":\"debian\",\"name\":\"Debian GNU/Linux\",\"version\":\"8 (jessie)\",\"major\":8,\"minor\":0,\"patch\":0,\"codename\":\"jessie\"},\"timezone\":\"CEST\",\"timezone_offset_sec\":7200,\"id\":\"7986a9307f6943ab9602c1248a18cfc2\"}}}\n2020-06-26T16:05:35.713+0200\tINFO\t[beat]\tinstance/beat.go:1028\tProcess info\t{\"system_info\": {\"process\": {\"capabilities\": {\"inheritable\":null,\"permitted\":[\"chown\",\"dac_override\",\"dac_read_search\",\"fowner\",\"fsetid\",\"kill\",\"setgid\",\"setuid\",\"setpcap\",\"linux_immutable\",\"net_bind_service\",\"net_broadcast\",\"net_admin\",\"net_raw\",\"ipc_lock\",\"ipc_owner\",\"sys_module\",\"sys_rawio\",\"sys_chroot\",\"sys_ptrace\",\"sys_pacct\",\"sys_admin\",\"sys_boot\",\"sys_nice\",\"sys_resource\",\"sys_time\",\"sys_tty_config\",\"mknod\",\"lease\",\"audit_write\",\"audit_control\",\"setfcap\",\"mac_override\",\"mac_admin\",\"syslog\",\"wake_alarm\",\"block_suspend\",\"audit_read\"],\"effective\":[\"chown\",\"dac_override\",\"dac_read_search\",\"fowner\",\"fsetid\",\"kill\",\"setgid\",\"setuid\",\"setpcap\",\"linux_immutable\",\"net_bind_service\",\"net_broadcast\",\"net_admin\",\"net_raw\",\"ipc_lock\",\"ipc_owner\",\"sys_module\",\"sys_rawio\",\"sys_chroot\",\"sys_ptrace\",\"sys_pacct\",\"sys_admin\",\"sys_boot\",\"sys_nice\",\"sys_resource\",\"sys_time\",\"sys_tty_config\",\"mknod\",\"lease\",\"audit_write\",\"audit_control\",\"setfcap\",\"mac_override\",\"mac_admin\",\"syslog\",\"wake_alarm\",\"block_suspend\",\"audit_read\"],\"bounding\":[\"chown\",\"dac_override\",\"dac_read_search\",\"fowner\",\"fsetid\",\"kill\",\"setgid\",\"setuid\",\"setpcap\",\"linux_immutable\",\"net_bind_service\",\"net_broadcast\",\"net_admin\",\"net_raw\",\"ipc_lock\",\"ipc_owner\",\"sys_module\",\"sys_rawio\",\"sys_chroot\",\"sys_ptrace\",\"sys_pacct\",\"sys_admin\",\"sys_boot\",\"sys_nice\",\"sys_resource\",\"sys_time\",\"sys_tty_config\",\"mknod\",\"lease\",\"audit_write\",\"audit_control\",\"setfcap\",\"mac_override\",\"mac_admin\",\"syslog\",\"wake_alarm\",\"block_suspend\",\"audit_read\"],\"ambient\":null}, \"cwd\": \"/var/devops\", \"exe\": \"/usr/share/filebeat/bin/filebeat\", \"name\": \"filebeat\", \"pid\": 22323, \"ppid\": 22322, \"seccomp\": {\"mode\":\"disabled\"}, \"start_time\": \"2020-06-26T16:05:35.430+0200\"}}}\n2020-06-26T16:05:35.714+0200\tINFO\tinstance/beat.go:310\tSetup Beat: filebeat; Version: 7.8.0\n2020-06-26T16:05:35.715+0200\tINFO\t[publisher]\tpipeline/module.go:113\tBeat name: drt-7\n2020-06-26T16:05:35.715+0200\tERROR\tfileset/modules.go:127\tNot loading modules. Module directory not found: /usr/share/filebeat/bin/module\n2020-06-26T16:05:35.715+0200\tERROR\tinstance/beat.go:958\tExiting: no modules or inputs enabled and configuration reloading disabled. What files do you want me to watch?\nExiting: no modules or inputs enabled and configuration reloading disabled. What files do you want me to watch?\n", "stderr_lines": ["2020-06-26T16:05:35.653+0200\tINFO\tinstance/beat.go:647\tHome path: [/usr/share/filebeat/bin] Config path: [/usr/share/filebeat/bin] Data path: [/usr/share/filebeat/bin/data] Logs path: [/usr/share/filebeat/bin/logs]", "2020-06-26T16:05:35.654+0200\tINFO\tinstance/beat.go:655\tBeat ID: 75deba96-a4c9-47a6-a04d-1b03b2f078a7", "2020-06-26T16:05:35.654+0200\tINFO\t[beat]\tinstance/beat.go:983\tBeat info\t{\"system_info\": {\"beat\": {\"path\": {\"config\": \"/usr/share/filebeat/bin\", \"data\": \"/usr/share/filebeat/bin/data\", \"home\": \"/usr/share/filebeat/bin\", \"logs\": \"/usr/share/filebeat/bin/logs\"}, \"type\": \"filebeat\", \"uuid\": \"75deba96-a4c9-47a6-a04d-1b03b2f078a7\"}}}", "2020-06-26T16:05:35.654+0200\tINFO\t[beat]\tinstance/beat.go:992\tBuild info\t{\"system_info\": {\"build\": {\"commit\": \"f79387d32717d79f689d94fda1ec80b2cf285d30\", \"libbeat\": \"7.8.0\", \"time\": \"2020-06-14T18:15:37.000Z\", \"version\": \"7.8.0\"}}}", "2020-06-26T16:05:35.654+0200\tINFO\t[beat]\tinstance/beat.go:995\tGo runtime info\t{\"system_info\": {\"go\": {\"os\":\"linux\",\"arch\":\"amd64\",\"max_procs\":32,\"version\":\"go1.13.10\"}}}", "2020-06-26T16:05:35.713+0200\tINFO\t[beat]\tinstance/beat.go:999\tHost info\t{\"system_info\": {\"host\": {\"architecture\":\"x86_64\",\"boot_time\":\"2020-06-25T12:15:27+02:00\",\"containerized\":false,\"name\":\"drt-7\",\"ip\":[\"127.0.0.1/8\",\"x.x.x.x/25\"],\"kernel_version\":\"3.16.7-ckt9-drt\",\"mac\":[\"xx:xx:xx:xx:xx:xx\",\"xx:xx:xx:xx:xx:xx\"],\"os\":{\"family\":\"debian\",\"platform\":\"debian\",\"name\":\"Debian GNU/Linux\",\"version\":\"8 (jessie)\",\"major\":8,\"minor\":0,\"patch\":0,\"codename\":\"jessie\"},\"timezone\":\"CEST\",\"timezone_offset_sec\":7200,\"id\":\"7986a9307f6943ab9602c1248a18cfc2\"}}}", "2020-06-26T16:05:35.713+0200\tINFO\t[beat]\tinstance/beat.go:1028\tProcess info\t{\"system_info\": {\"process\": {\"capabilities\": {\"inheritable\":null,\"permitted\":[\"chown\",\"dac_override\",\"dac_read_search\",\"fowner\",\"fsetid\",\"kill\",\"setgid\",\"setuid\",\"setpcap\",\"linux_immutable\",\"net_bind_service\",\"net_broadcast\",\"net_admin\",\"net_raw\",\"ipc_lock\",\"ipc_owner\",\"sys_module\",\"sys_rawio\",\"sys_chroot\",\"sys_ptrace\",\"sys_pacct\",\"sys_admin\",\"sys_boot\",\"sys_nice\",\"sys_resource\",\"sys_time\",\"sys_tty_config\",\"mknod\",\"lease\",\"audit_write\",\"audit_control\",\"setfcap\",\"mac_override\",\"mac_admin\",\"syslog\",\"wake_alarm\",\"block_suspend\",\"audit_read\"],\"effective\":[\"chown\",\"dac_override\",\"dac_read_search\",\"fowner\",\"fsetid\",\"kill\",\"setgid\",\"setuid\",\"setpcap\",\"linux_immutable\",\"net_bind_service\",\"net_broadcast\",\"net_admin\",\"net_raw\",\"ipc_lock\",\"ipc_owner\",\"sys_module\",\"sys_rawio\",\"sys_chroot\",\"sys_ptrace\",\"sys_pacct\",\"sys_admin\",\"sys_boot\",\"sys_nice\",\"sys_resource\",\"sys_time\",\"sys_tty_config\",\"mknod\",\"lease\",\"audit_write\",\"audit_control\",\"setfcap\",\"mac_override\",\"mac_admin\",\"syslog\",\"wake_alarm\",\"block_suspend\",\"audit_read\"],\"bounding\":[\"chown\",\"dac_override\",\"dac_read_search\",\"fowner\",\"fsetid\",\"kill\",\"setgid\",\"setuid\",\"setpcap\",\"linux_immutable\",\"net_bind_service\",\"net_broadcast\",\"net_admin\",\"net_raw\",\"ipc_lock\",\"ipc_owner\",\"sys_module\",\"sys_rawio\",\"sys_chroot\",\"sys_ptrace\",\"sys_pacct\",\"sys_admin\",\"sys_boot\",\"sys_nice\",\"sys_resource\",\"sys_time\",\"sys_tty_config\",\"mknod\",\"lease\",\"audit_write\",\"audit_control\",\"setfcap\",\"mac_override\",\"mac_admin\",\"syslog\",\"wake_alarm\",\"block_suspend\",\"audit_read\"],\"ambient\":null}, \"cwd\": \"/var/devops\", \"exe\": \"/usr/share/filebeat/bin/filebeat\", \"name\": \"filebeat\", \"pid\": 22323, \"ppid\": 22322, \"seccomp\": {\"mode\":\"disabled\"}, \"start_time\": \"2020-06-26T16:05:35.430+0200\"}}}", "2020-06-26T16:05:35.714+0200\tINFO\tinstance/beat.go:310\tSetup Beat: filebeat; Version: 7.8.0", "2020-06-26T16:05:35.715+0200\tINFO\t[publisher]\tpipeline/module.go:113\tBeat name: drt-7", "2020-06-26T16:05:35.715+0200\tERROR\tfileset/modules.go:127\tNot loading modules. Module directory not found: /usr/share/filebeat/bin/module", "2020-06-26T16:05:35.715+0200\tERROR\tinstance/beat.go:958\tExiting: no modules or inputs enabled and configuration reloading disabled. What files do you want me to watch?", "Exiting: no modules or inputs enabled and configuration reloading disabled. What files do you want me to watch?"], "stdout": "Starting Filebeat sends log files to Logstash or directly to Elasticsearch.: filebeat failed!\n", "stdout_lines": ["Starting Filebeat sends log files to Logstash or directly to Elasticsearch.: filebeat failed!"]}

RUNNING HANDLER [elastic.beats : restart the service] ***************************************************************************************************************************************************************************************

PLAY RECAP **********************************************************************************************************************************************************************************************************************************
drt-7              : ok=21   changed=1    unreachable=0    failed=1    skipped=12   rescued=0    ignored=0

Beats logs if relevant:

Answer 1 · 2020-06-26T13:38:38.000Z

Hi @stivi21,
Our daily tests for ansible-beats and Debian 8 are successful (https://devops-ci.elastic.co/job/elastic+ansible-beats+master/)
Can you provides all the details requested in the issue template?

Answer 2 · 2020-06-26T14:15:38.000Z

Hi @stivi21,
Our daily tests for ansible-beats and Debian 8 are successful (https://devops-ci.elastic.co/job/elastic+ansible-beats+master/)
Can you provides all the details requested in the issue template?

Hi @jmlrt,
Thank you for reply. I added more details. On this host systemd is not installed, only sysvinit, so can be related whit this.

Answer 3 · 2020-06-29T07:05:54.000Z

Any reason why you aren't using systemd?
systemd is the default init system for Debian 8.

~~ansible-elasticsearch role is configured to use it with Debian 8. If needed you can modify the use_system_d fact here but this requires forking and modifying the role.~~

Edit: ansible-beats is configured to use the default init system and doesn't do any specific configuration for that outside of what's provided in the official Filebeat Debian package.

Answer 4 · 2020-06-29T11:24:41.000Z

I don't know why server is without systemd, the person who installed the system is no longer working. Anyway thank you for reply. I'm not checked before, that systemd is default in Debian 8.