A deadly simple monitoring tool for Docker - Using A Python Wrapper For Docker API. The Blog Post
A Python wrapper for Dokcer API 1.24 compatible with Docker 1.12.x and later.
The purpose is to write python scripts easily for monitoring all of your Docker containers (running in a Linux distibution - other OS are coming soon in the roadmap of development).
This is the Alpha, moving to Beta very very soon :-)
The wrapper contains these classes:
api/
├── changes.py
├── containers.py
├── errors.py
├── ids.py
├── inspect.py
├── logs.py
├── process.py
└── stats.py
Where :
Containers : List containers
Inspect : Return low-level information on the container id
Ids : Return containers IDs
Logs : Get stdout and stderr logs from the container id
Process : List processes running inside the container id. On Unix systems this is done by running the ps command. This endpoint is not supported on Windows.
Stats : This endpoint returns a live stream of a container’s resource usage statistics.
# Using virtualenv:
virtualenv DoMonit
cd DoMonit
git clone https://github.com/eon01/DoMonit.git
. bin/activate
cd DoMonit
pip install -r requirements.txt
#To Test:
python examples.py
This is an example script:
from api.containers import Containers
from api.ids import Ids
from api.inspect import Inspect
from api.logs import Logs
from api.process import Process
from api.changes import Changes
from api.stats import Stats
import json
c = Containers()
i = Ids()
print ("Number of containers is : %s " % (sum(1 for i in i.ids())))
if __name__ == "__main__":
for c_id in i.ids():
ins = Inspect(c_id)
sta = Stats(c_id)
proc = Process(c_id, ps_args = "aux")
# Container name
print ("\n#Container name")
print ins.name()
# Container id
print ("\n#Container id")
print ins.id()
# Memory usage
mem_u = sta.usage()
# Memory limit
mem_l = sta.limit()
# Memory usage %
print ("\n#Memory usage %")
print int(mem_u)*100/int(mem_l)
# The number of times that a process of the cgroup triggered a "major fault"
print ("\n#The number of times that a process of the cgroup triggered a major fault")
print sta.pgmajfault()
# Same output as ps aux in *nix
print("\n#Same output as ps aux in *nix")
print proc.ps()
For the following 5 running containers:
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1a29e9652822 instavote/vote "gunicorn app:app -b " 12 seconds ago Up 11 seconds 80/tcp, 100/tcp vote_webapp_3
6ca598188d1a instavote/vote "gunicorn app:app -b " 12 seconds ago Up 11 seconds 80/tcp, 100/tcp vote_webapp_4
7f1a6bfaf95b instavote/vote "gunicorn app:app -b " 12 seconds ago Up 11 seconds 80/tcp, 100/tcp vote_webapp_5
e3a7066ba953 instavote/vote "gunicorn app:app -b " 6 days ago Up 5 hours 80/tcp, 100/tcp vote_webapp_2
1e557c8dc5f7 instavote/vote "gunicorn app:app -b " 6 days ago Up 5 hours 80/tcp, 100/tcp vote_webapp_1
the execution result of the example above is:
Number of containers is : 5
#Container name
/vote_webapp_3
#Container id
1a29e9652822447a440799306f4edb65003bca9cdea4c56e1e0ba349d5112d3e
#Memory usage %
0.697797903077
#The number of times that a process of the cgroup triggered a major fault
15
#Same output as ps aux in *nix
{u'Processes': [[u'root', u'26636', u'0.0', u'0.2', u'76808', u'16228', u'?', u'Ss', u'15:43', u'0:00', u'/usr/local/bin/python2 /usr/local/bin/gunicorn app:app -b 0.0.0.0:80 --log-file - --access-logfile - --workers 4 --keep-alive 0'], [u'root', u'26773', u'0.0', u'0.2', u'88776', u'19976', u'?', u'S', u'15:43', u'0:00', u'/usr/local/bin/python2 /usr/local/bin/gunicorn app:app -b 0.0.0.0:80 --log-file - --access-logfile - --workers 4 --keep-alive 0'], [u'root', u'26784', u'0.0', u'0.2', u'88572', u'19800', u'?', u'S', u'15:43', u'0:00', u'/usr/local/bin/python2 /usr/local/bin/gunicorn app:app -b 0.0.0.0:80 --log-file - --access-logfile - --workers 4 --keep-alive 0'], [u'root', u'26787', u'0.0', u'0.2', u'88568', u'19816', u'?', u'S', u'15:43', u'0:00', u'/usr/local/bin/python2 /usr/local/bin/gunicorn app:app -b 0.0.0.0:80 --log-file - --access-logfile - --workers 4 --keep-alive 0'], [u'root', u'26793', u'0.0', u'0.2', u'88572', u'19828', u'?', u'S', u'15:43', u'0:00', u'/usr/local/bin/python2 /usr/local/bin/gunicorn app:app -b 0.0.0.0:80 --log-file - --access-logfile - --workers 4 --keep-alive 0']], u'Titles': [u'USER', u'PID', u'%CPU', u'%MEM', u'VSZ', u'RSS', u'TTY', u'STAT', u'START', u'TIME', u'COMMAND']}
...etc
This is a simple example:
#!/usr/bin/env python
from domonit.events import Events
# Events is a stream of json sent by Docker API. In the Events class, we are only streaming in the same way the Docker events like Docker API make it.
e = Events()
events = e.events()
for event in events:
print event
The event:
{"Type":"network","Action":"create","Actor":{"ID":"a4a1ccb5c4a39ae64f46118e9e245b8cf516184571b7f7ccd6a406c20e735cb1","Attributes":{"name":"test","type":"bridge"}},"time":1481460602,"timeNano":1481460602475845939}
is reported by the network creation:
docker network create test
For better uderstanfing and visibility, I used Python dir() here to list the different functions of every class under api/*
python dir.py
Containers()
['__doc__', '__init__', '__module__', 'base', 'command', 'containers', 'created', 'host_config', 'image', 'image_id', 'labels', 'mounts', 'names', 'network_settings', 'ports', 'resp', 'session', 'status', 'url']
Ids()
['__doc__', '__init__', '__module__', 'base', 'ids', 'resp', 'session', 'url']
Inspect()
['__doc__', '__init__', '__module__', 'app_armor_profile', 'args', 'attach_stderr', 'attach_stdin', 'base', 'cmd', 'container_id', 'created', 'domainname', 'driver', 'entrypoint', 'env', 'exec_ids', 'exposed_ports', 'host_config_binds', 'host_config_blkio_device_read_bps', 'host_config_blkio_device_write_bps', 'host_config_blkio_device_write_iops', 'host_config_blkio_weight_device', 'host_config_cap_add', 'host_config_cap_drop', 'host_config_container_id_file', 'host_config_cpu_percent', 'host_config_cpu_period', 'host_config_cpu_shares', 'host_config_cpuset_cpus', 'host_config_cpuset_mems', 'host_config_devices', 'host_config_dns', 'host_config_dns_options', 'host_config_dns_search', 'host_config_extra_hosts', 'host_config_host_config_shm_size', 'host_config_ipc_mode', 'host_config_kernel_memory', 'host_config_links', 'host_config_log_config_config', 'host_config_log_config_type', 'host_config_lxc_conf', 'host_config_maximum_iobps', 'host_config_maximum_iops', 'host_config_memory', 'host_config_memory_reservation', 'host_config_memory_swap', 'host_config_network_mode', 'host_config_oom_kill_disable', 'host_config_oom_score_adj', 'host_config_pid_mode', 'host_config_port_bindings', 'host_config_privileged', 'host_config_publish_all_ports', 'host_config_readonly_rootfs', 'host_config_restart_policy_maximum_retry_count', 'host_config_restart_policy_name', 'host_config_security_opt', 'host_config_storage_opt', 'host_config_sysctls', 'host_config_ulimits', 'host_config_volume_driver', 'host_config_volumes_from', 'hostname', 'hostname_path', 'hosts_path', 'id', 'image', 'inspect', 'labels', 'log_path', 'mac_address', 'mount_label', 'mounts', 'name', 'network_disabled', 'network_settings_bridge', 'network_settings_endpoint_id', 'network_settings_gateway', 'network_settings_global_ipv6_address', 'network_settings_global_ipv6_prefixLen', 'network_settings_hairpin_mode', 'network_settings_ip_address', 'network_settings_ip_prefixLen', 'network_settings_ipv6_gateway', 'network_settings_link_local_ipv6_address', 'network_settings_link_local_ipv6_prefix_len', 'network_settings_mac_address', 'network_settings_networks', 'network_settings_ports', 'network_settings_sandbox_id', 'network_settings_sandbox_key', 'network_settings_secondary_ip_addresses', 'network_settings_secondary_ipv6_addresses', 'on_build', 'open_stdin', 'path', 'process_label', 'resolv_conf_path', 'resp', 'restart_count', 'session', 'state_dead', 'state_error', 'state_exit_code', 'state_finished_at', 'state_oom_killed', 'state_paused', 'state_pid', 'state_restarting', 'state_running', 'state_started_at', 'state_status', 'stdin_once', 'stop_signal', 'tty', 'url', 'user', 'volumes', 'working_dir']
Stats()
['__doc__', '__init__', '__module__', 'active_anon', 'active_file', 'base', 'blkio_stats', 'cache', 'container_id', 'cpu_stats', 'failcnt', 'hierarchical_memory_limit', 'inactive_anon', 'inactive_file', 'interfaces', 'limit', 'mapped_file', 'max_usage', 'memory_stats', 'networks', 'percpu_percpu_usage', 'percpu_period', 'percpu_stats', 'percpu_system_cpu_usage', 'percpu_throttled_periods', 'percpu_throttled_time', 'percpu_throttling_data', 'percpu_total_usage', 'percpu_usage', 'percpu_usage_in_kernelmode', 'percpu_usage_in_usermode', 'period', 'pgfault', 'pgmajfault', 'pgpgin', 'pgpgout', 'pids_stats_current', 'read', 'resp', 'rss', 'rss_huge', 'rx_bytes', 'rx_dropped', 'rx_errors', 'rx_packets', 'session', 'stats', 'stream', 'system_cpu_usage', 'throttled_periods', 'throttled_time', 'throttling_data', 'total_active_anon', 'total_active_file', 'total_cache', 'total_inactive_anon', 'total_inactive_file', 'total_mapped_file', 'total_pgfault', 'total_pgmajfault', 'total_pgpgin', 'total_pgpgout', 'total_rss', 'total_rss_huge', 'total_unevictable', 'total_usage', 'total_writeback', 'tx_bytes', 'tx_dropped', 'tx_errors', 'tx_packets', 'unevictable', 'url', 'usage', 'usage_in_kernelmode', 'usage_in_usermode', 'writeback']
Process()
['__doc__', '__init__', '__module__', 'base', 'container_id', 'processes', 'ps', 'ps_args', 'resp', 'session', 'titles', 'url']
Logs()
['__doc__', '__init__', '__module__', 'base', 'container_id', 'details', 'follow', 'logs', 'resp', 'session', 'since', 'stderr', 'stdout', 'tail', 'timestamps', 'url']
Changes()
['__doc__', '__init__', '__module__', 'base', 'changes', 'container_id', 'resp', 'session', 'url']
Events()
['__doc__', '__init__', '__module__', 'base', 'events', 'resp', 'session', 'url']
Let's take the example of the "inspect" call. Docer replies with data similar to the following one:
{
"AppArmorProfile": "",
"Args": [
"-c",
"exit 9"
],
"Config": {
"AttachStderr": true,
"AttachStdin": false,
"AttachStdout": true,
"Cmd": [
"/bin/sh",
"-c",
"exit 9"
],
"Domainname": "",
"Entrypoint": null,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"ExposedPorts": null,
"Hostname": "ba033ac44011",
"Image": "ubuntu",
"Labels": {
"com.example.vendor": "Acme",
"com.example.license": "GPL",
"com.example.version": "1.0"
},
"MacAddress": "",
"NetworkDisabled": false,
"OnBuild": null,
"OpenStdin": false,
"StdinOnce": false,
"Tty": false,
"User": "",
"Volumes": {
"/volumes/data": {}
},
"WorkingDir": "",
"StopSignal": "SIGTERM"
},
"Created": "2015-01-06T15:47:31.485331387Z",
"Driver": "devicemapper",
"ExecIDs": null,
"HostConfig": {
"Binds": null,
"MaximumIOps": 0,
"MaximumIOBps": 0,
"BlkioWeight": 0,
"BlkioWeightDevice": [{}],
"BlkioDeviceReadBps": [{}],
"BlkioDeviceWriteBps": [{}],
"BlkioDeviceReadIOps": [{}],
"BlkioDeviceWriteIOps": [{}],
"CapAdd": null,
"CapDrop": null,
"ContainerIDFile": "",
"CpusetCpus": "",
"CpusetMems": "",
"CpuPercent": 80,
"CpuShares": 0,
"CpuPeriod": 100000,
"Devices": [],
"Dns": null,
"DnsOptions": null,
"DnsSearch": null,
"ExtraHosts": null,
"IpcMode": "",
"Links": null,
"LxcConf": [],
"Memory": 0,
"MemorySwap": 0,
"MemoryReservation": 0,
"KernelMemory": 0,
"OomKillDisable": false,
"OomScoreAdj": 500,
"NetworkMode": "bridge",
"PidMode": "",
"PortBindings": {},
"Privileged": false,
"ReadonlyRootfs": false,
"PublishAllPorts": false,
"RestartPolicy": {
"MaximumRetryCount": 2,
"Name": "on-failure"
},
"LogConfig": {
"Config": null,
"Type": "json-file"
},
"SecurityOpt": null,
"Sysctls": {
"net.ipv4.ip_forward": "1"
},
"StorageOpt": null,
"VolumesFrom": null,
"Ulimits": [{}],
"VolumeDriver": "",
"ShmSize": 67108864
},
"HostnamePath": "/var/lib/docker/containers/ba033ac4401106a3b513bc9d639eee123ad78ca3616b921167cd74b20e25ed39/hostname",
"HostsPath": "/var/lib/docker/containers/ba033ac4401106a3b513bc9d639eee123ad78ca3616b921167cd74b20e25ed39/hosts",
"LogPath": "/var/lib/docker/containers/1eb5fabf5a03807136561b3c00adcd2992b535d624d5e18b6cdc6a6844d9767b/1eb5fabf5a03807136561b3c00adcd2992b535d624d5e18b6cdc6a6844d9767b-json.log",
"Id": "ba033ac4401106a3b513bc9d639eee123ad78ca3616b921167cd74b20e25ed39",
"Image": "04c5d3b7b0656168630d3ba35d8889bd0e9caafcaeb3004d2bfbc47e7c5d35d2",
"MountLabel": "",
"Name": "/boring_euclid",
"NetworkSettings": {
"Bridge": "",
"SandboxID": "",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": null,
"SandboxKey": "",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"bridge": {
"NetworkID": "7ea29fc1412292a2d7bba362f9253545fecdfa8ce9a6e37dd10ba8bee7129812",
"EndpointID": "7587b82f0dada3656fda26588aee72630c6fab1536d36e394b2bfbcf898c971d",
"Gateway": "172.17.0.1",
"IPAddress": "172.17.0.2",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:ac:12:00:02"
}
}
},
"Path": "/bin/sh",
"ProcessLabel": "",
"ResolvConfPath": "/var/lib/docker/containers/ba033ac4401106a3b513bc9d639eee123ad78ca3616b921167cd74b20e25ed39/resolv.conf",
"RestartCount": 1,
"State": {
"Error": "",
"ExitCode": 9,
"FinishedAt": "2015-01-06T15:47:32.080254511Z",
"OOMKilled": false,
"Dead": false,
"Paused": false,
"Pid": 0,
"Restarting": false,
"Running": true,
"StartedAt": "2015-01-06T15:47:32.072697474Z",
"Status": "running"
},
"Mounts": [
{
"Name": "fac362...80535",
"Source": "/data",
"Destination": "/data",
"Driver": "local",
"Mode": "ro,Z",
"RW": false,
"Propagation": ""
}
]
}
In DoMonit code the function that will reply with the name of the restart policy should have the following name:
def host_config_restart_policy_name():
[...]
return [...]
- Functions name are in small letters and describes the "path" to follow in the json response to get it.
HostConfigshould be namedhost_configandRestartPolicyshould be namedrestart_policywhich giveshost_config_restart_policy_name
"HostConfig": {
"":"",
"":"",
"":"",
[...]
"RestartPolicy": {
"MaximumRetryCount": 2,
"Name": "on-failure"
},
[...]
},
# How To Contribute
- Submit a feature proposal if you have new features to add. If you fixing a bug, this step is optional
- Fork the repo and then clone it to your development machine
git clone https://github.com/eon01/DoMonit.git
- Create a new branch
git checkout -b my-new-feature
- Add your code
- Follow the code style and naming conventions
- Write some documentations and tests
- Commit your changes and push the new branch to your fork
git add .
git commit -m 'Your detailed description'
- If the remote repos (the forked one) is not set, type:
git remote set-url origin https://github.com/<your_username>/DoMonit
- Push changes:
git push origin my-new-feature
- Submit a pull request
- You can get a list of your commits:
git log --oneline origin/master..my-new-feature
- It is optional but if accepted and merged, you can delete your branch (remotly and locally)
git checkout master
git push origin :my-new-feature
git branch -D my-new-feature
- Documentation
- Exception & timeout handling
- Unit testing