robcowart/synesis_lite_syslog

LS not ingesting syslog

bluefangs opened this issue · 1 comments

Hi Rob,

I've installed this on a docker container, this is the current docker-compose.yaml:


version: '2'
services:
  elasticsearch:
    image: elasticsearch-img:6.3.2
    container_name: elasticsearch-container
    volumes:
      - /data/elasticsearch-1/:/usr/share/elasticsearch/data
    ports: 
      - 9200:9200 #Elasticsearch HTTP
      - 9300:9300 #Elasticsearch TCP transport
    network_mode: bridge
    restart: always
    environment:
      # - cluster.name=docker-cluster
      # - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms4g -Xmx4g"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    
  logstash:
    image: logstash-img:6.3.2
    container_name: logstash-container
    ports:
      - 5000:5000 #logstash TCP input
      - 514:5140  #listent to syslog on 514(host) map it to 5140(container) 541 is reserved and needs root
      - 514:5140/udp  #listent to syslog on 514(host) map it to 5140(container) 541 is reserved and needs root
    restart: always #restarts on reboot
    environment:
      - "LS_JAVA_OPTS=-Xms8g -Xmx8g"
      - "SYNLITE_SYSLOG_TEMPLATE_PATH=/usr/share/logstash/syslog/templates"
      - "SYNLITE_SYSLOG_GROK_PATTERNS_DIR=/usr/share/logstash/syslog/patterns"
      - "SYNLITE_SYSLOG_RESOLVE_IP2HOST=true"
      - "SYNLITE_SYSLOG_NAMESERVER=8.8.8.8"
      - "SYNLITE_SYSLOG_ES_HOSTS=elasticsearch:9200"
      # - "SYNLITE_SYSLOG_ES_USER=elastic"
      # - "SYNLITE_SYSLOG_ES_PASSWORD=changeme"
      - "SYNLITE_SYSLOG_TCP_HOST=0.0.0.0"
      - "SYNLITE_SYSLOG_TCP_PORT=514"
      - "SYNLITE_SYSLOG_UDP_HOST=0.0.0.0"
      - "SYNLITE_SYSLOG_UDP_PORT=514"
      - "SYNLITE_SYSLOG_MSG_TIMESTAMP=true"
      - "SYNLITE_SYSLOG_TZ=UTC"
    network_mode: bridge 
    links:
    - elasticsearch
    depends_on:
    - elasticsearch

Initially, port mapping of 514:514 made docker crib stating that permission was denied. I'm guessing this is because it's a port < 1000 and hence is previlaged. I've mapped 514:5140 within the container.

My /etc/rsyslog.conf looks like below:

...
...
# provides UDP syslog reception
module(load="imudp")
input(type="imudp" port="514")

# provides TCP syslog reception
module(load="imtcp")
input(type="imtcp" port="514")
...
...

I'm able to see syslog being written on to /var/log/syslog. It works when I do something like: logger -s " This is a test "

However, I do not see anyting being picked up by LS/ES. What am I missing?

Thanks

I have just released a container for this solution. If you are still looking for a solution, please try it.