- https://github.com/VAdamec/elk-stack-v5-xpack
- Composer setup full ELK stack with XPack and plugins needed for workshop
30 day
trial license is installed that allows access to all features- https://www.elastic.co/subscriptions
- https://www.elastic.co/guide/en/x-pack/current/license-expiration.html
- Licence provides:
- Marvel cluster
monitoring
- Basic subscription Security
- Shieldhttps://www.elastic.co/guide/en/x-pack/current/how-security-works.html
- opensource replacement https://github.com/elasticfence/elasticsearch-http-user-auth/tree/5.1.1 (basic user auth + kibana plugins and ACL, validation with v5.11 needed)
Alerting
- Watcher https://www.elastic.co/guide/en/x-pack/5.1/xpack-alerting.htmlReporting
(PDF creation)Profiling
- Basic subscriptionGraphing
data connectionsRetention
policy
- set of
README
files: - README - this document
- README_api_usage - basic handling of ELK via curl
- README_mapping - ELK mapping example
- README_query_cheatsheet - examples of query language
- README_watcher - how to setup simple alerting
- README_workshop - labs + basic terms
- README_geoip - info about GeoIP coordinates handling
- README_s3 - experimental backup to S3 bucket simulated via Riak CS
- Terraform - just for workshop, spin several servers with preinstalled docker, readme in directory
- 1x Fluentd
NEED TO BE RUN FIRST
see how to run this stack - containers output logger (back to ELK)
- index
platform*
- 3x server (data/client/master role) -
you can start just one server (elasticsearchdataone) if you don't have HW resources
or limit resources via docker CPU/Mem qutoas, see comments in common-services.yml - x-pack installed
- exposed ports:
- 920[1-3] / 930[1-3]
- Kibana lets you visualize your Elasticsearch data and navigate the Elastic Stack, so you can do anything from learning why you're getting paged at 2:00 a.m. to understanding the impact rain might have on your quarterly numbers.
- exposed port: 5601
- http://localhost:5601/app/kibana
- x-pack installed
- The
default
password for theelastic
user ischangeme
- https://www.elastic.co/guide/en/x-pack/current/security-getting-started.html
- https://www.elastic.co/guide/en/x-pack/current/logstash.html
- xpack audit log is enabled, index
.security_audit_log*
- used for easy sample data upload
- exposed ports:
- 5000 - json filter
- 5001 - raw, no filters
- you can use .raw field for not_analyzed data
- index:
logstash*
- used for AWS S3 simulation
- exposed ports:
- 8080 - API
- not logged via Fluentd to ELK (API key created during start)
- Mattermost server - running outside demo stack as a simple container with open access and webhook created
- You can easily spin your own instances via docker compose
- https://github.com/mattermost/mattermost-docker
- IP of server is setup in
.env
-
Start stack (do not use docker-compose up as there are some prerequisities to start stack)
-
This short script will prepare temporary data volumes for ES servers and start fluend container first
-
Download git repo:
$ git clone https://github.com/VAdamec/elk-stack-v5-xpack
$ cd elk-stack-v5-xpack
$ ./_start
- just stop containers, for removing network/artefact use docker-copose command
$ ./_stop
You can use logstash or kibana containers (all is mounted as /code/) or install on your system (OSX/Lnx).
- netcat - for logstash feedings
- jq - for pretty outputs
- curl - for shell work with Elastic
- curator
- Timelion
- https://www.elastic.co/blog/timelion-timeline
- Dependency graph - knowledge and usecase missing