rocket4321/nmap_tracker

MAC address used for checking if host up or not

Closed this issue · 10 comments

f45tb00t commented

Hi,

sorry for the delay. Had not time yesterday to check if the code works.

I tried it today. Even with debug-log at 5. Wha I can see that its trying to check the host, but also tells there is no MAC for 123.123.123.123. Therefore the host is not considered as UP. I tried it again with the gw of the subnet where HA is in. Result here is that the nmap scanner is working as expected.

Two things maybe from my point of view since your code differs a lot from the original one.

  1. As a user I absolutely don't like unneccesary connection out of my network. There is a check in your code. Surely you can do what ever you want with your code. But in fact I won't use it, because there is no need for external checks
  2. Yourecode is also checking for the MAC address. I really don't understand why this is important. As a network/sec engineer I really don't care about MAC addresses, beside I am working or tshooting on L2.
    For a presence check L3 is more than enough. Why do you guys really want to use the mac address?

just my two cents

Maybe one example when I issue the nmap tool manually. As you can see the result is fine and the host is considered as UP.

nmap -oX - 123.123.123.123/32 --privileged -sn
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE nmaprun>
<?xml-stylesheet href="file:///usr/bin/../share/nmap/nmap.xsl" type="text/xsl"?>
<!-- Nmap 7.70 scan initiated Sun Feb 14 11:52:05 2021 as: nmap -oX - -&#45;privileged -sn 123.123.123.123/32 -->
<nmaprun scanner="nmap" args="nmap -oX - -&#45;privileged -sn 123.123.123.123/32" start="1613299925" startstr="Sun Feb 14 11:52:05 2021" version="7.70" xmloutputversion="1.04">
<verbose level="0"/>
<debugging level="0"/>
<host><status state="up" reason="echo-reply" reason_ttl="63"/>
<address addr="123.123.123.123" addrtype="ipv4"/>
<hostnames>
</hostnames>
<times srtt="65186" rttvar="65186" to="325930"/>
</host>
<runstats><finished time="1613299925" timestr="Sun Feb 14 11:52:05 2021" elapsed="0.37" summary="Nmap done at Sun Feb 14 11:52:05 2021; 1 IP address (1 host up) scanned in 0.37 seconds" exit="success"/><hosts up="1" down="0" total="1"/>
</runstats>
</nmaprun>
rocket4321 commented

The typical use for most HASS users to scan the local network, so MAC address are normally available. And they are used as a unique identifier since DHCP assignments can often change over time, especially with older hardware. But I appreciate your point of minimal network activity.

I believe I've adjusted the tool to meet this application, you'll want to activate the below setting:

include_no_mac: true

That would create either 'device_tracker.<host_name>' or 'device_tracker.<IP_v4>'

Is there still a network action it's performing that you'd prefer not to happen?

Let me know how it works out for you. Thanks for your patience.

f45tb00t commented

Hi,

thanks a lot for your effort in this! :) I will try asap.

Well, yes I understand the point. But most users should really think twice what they are doing with their networks and especially which services they grant access to the outside and more important which access rights they give from the outside to the inside.
Best thing would be to segregate networks. Even with transfer networks, DMZs and a hardened IOT network and surely a good firewall.

I really like the KISS principle. Keep It Stupid and Simple. Means mostly a tool should do only one thing. And that in a good way without bloat.

Can you tell me why its needed to do a version check on http://xael.org/pages/python-nmap/python-nmap_CURRENT_VERSION.txt? As far as I can see this is not needed at all.

I assume once the scanner is working like a charme there is no need to update at all if there are no vulnerabilities or mistakes in the code.

cheers

fastboot

rocket4321 commented

The version check is due the python nmap package, which is typically a dependency and will hopefully be again soon. I've submitted an email and will work on PR for the package posted over on bitbucket.

For now, this component uses a dev version of python-nmap (bitbucket), but plans to switch back to the package once changes have been released.

f45tb00t commented

Hi,

just a short note:

Platform error switch.ping - No module named 'homeassistant.components.ping.switch'
2021-02-18 10:03:37 ERROR (MainThread) [homeassistant.components.homeassistant] Invalid config for [device_tracker.nmap_tracker]: [exclude_active] is an invalid option for [device_tracker.nmap_tracker]. Check: device_tracker.nmap_tracker->exclude_active. (See ?, line ?).

Seems it shows an error with the config snippet. I removed now that exclude_active and lets see.

Edit: No, its not loading anymore.
I've just used your advanced configuration snippet. No changes from my side. Code is under .homeassistant/custom_component/nmap_tracker

rocket4321 commented

First, the subfolder should be called: 'custom_components' (NOTE the 's' at the end)

Second, if the component is loaded you should see a warning in the log:
WARNING (MainThread) [homeassistant.loader] You are using a custom integration for nmap_tracker which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant.

Finally, confirmed on current published version that all sample configs are running as expected.
Also, for clarity, the log line 'Platform error switch.ping - No module named 'homeassistant.components.ping.switch' doesn't have any implications to nmap_tracker, AFAIK.

I've tried to incorporate your hurdles into the README for others, sorry for the grief getting this to work.

f45tb00t commented

Hi,

so i've tested it now. Actually it's even not creating a new known devices file.

Here some logs as well as the configuration.

Config:

  - platform: nmap_tracker
    hosts:
     - 123.123.123.123/32
     - 124.124.124.1/32
    home_interval: 10
    exclude_active: true
    timeout: 60
    interval_seconds: 10
    include_no_mac: true
    scan_options: "-F --host-timeout 5s"
#    local_mac_hostname: "localhost"
    debug_log_level: 5
    new_device_defaults:
      track_new_devices: true

Logs:

2021-02-21 10:32:47 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for nmap_tracker which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant.
2021-02-21 10:32:47 DEBUG (SyncWorker_3) [custom_components.nmap_tracker.device_tracker] Processor [SyncWorker_3] initialized for 123.123.123.123/32 124.124.124.1/32
2021-02-21 10:32:47 DEBUG (Thread-7) [custom_components.nmap_tracker.device_tracker] Nmap Processor thread started: Thread-7
2021-02-21 10:32:47 DEBUG (SyncWorker_3) [custom_components.nmap_tracker.device_tracker] Active thread MainThread - 1995841552
2021-02-21 10:32:47 DEBUG (SyncWorker_3) [custom_components.nmap_tracker.device_tracker] Active thread Thread-1 - 1961591904
2021-02-21 10:32:47 DEBUG (SyncWorker_3) [custom_components.nmap_tracker.device_tracker] Active thread SyncWorker_0 - 1951396960
2021-02-21 10:32:47 DEBUG (SyncWorker_3) [custom_components.nmap_tracker.device_tracker] Active thread SyncWorker_1 - 1940911200
2021-02-21 10:32:47 DEBUG (SyncWorker_3) [custom_components.nmap_tracker.device_tracker] Active thread SyncWorker_2 - 1930425440
2021-02-21 10:32:47 DEBUG (SyncWorker_3) [custom_components.nmap_tracker.device_tracker] Active thread SyncWorker_3 - 1919939680
2021-02-21 10:32:47 DEBUG (SyncWorker_3) [custom_components.nmap_tracker.device_tracker] Active thread SyncWorker_4 - 1909453920
2021-02-21 10:32:47 DEBUG (SyncWorker_3) [custom_components.nmap_tracker.device_tracker] Active thread Recorder - 1897395296
2021-02-21 10:32:47 DEBUG (SyncWorker_3) [custom_components.nmap_tracker.device_tracker] Active thread Thread-2 - 1886352480
2021-02-21 10:32:47 DEBUG (SyncWorker_3) [custom_components.nmap_tracker.device_tracker] Active thread SyncWorker_5 - 1851528288
2021-02-21 10:32:47 DEBUG (SyncWorker_3) [custom_components.nmap_tracker.device_tracker] Active thread SyncWorker_6 - 1843135584
2021-02-21 10:32:47 DEBUG (SyncWorker_3) [custom_components.nmap_tracker.device_tracker] Active thread zeroconf-Engine-738 - 1833583712
2021-02-21 10:32:47 DEBUG (SyncWorker_3) [custom_components.nmap_tracker.device_tracker] Active thread zeroconf-Reaper_739 - 1825191008
2021-02-21 10:32:47 DEBUG (SyncWorker_3) [custom_components.nmap_tracker.device_tracker] Active thread Thread-5 - 1815749728
2021-02-21 10:32:47 DEBUG (SyncWorker_3) [custom_components.nmap_tracker.device_tracker] Active thread Thread-6 - 1807357024
2021-02-21 10:32:47 DEBUG (SyncWorker_3) [custom_components.nmap_tracker.device_tracker] Active thread Thread-7 - 1796232288
2021-02-21 10:32:47 DEBUG (SyncWorker_3) [custom_components.nmap_tracker.device_tracker] Processor thread count is 1
2021-02-21 10:32:51 DEBUG (Thread-7) [custom_components.nmap_tracker.device_tracker] Nmap Command: nmap -oX - -F --host-timeout 5s 123.123.123.123/32 124.124.124.1/32
2021-02-21 10:32:52 DEBUG (SyncWorker_3) [custom_components.nmap_tracker.device_tracker] Processor thread completed normally after 5.041858 seconds for 123.123.123.123/32 124.124.124.1/32.
2021-02-21 10:32:52 DEBUG (SyncWorker_3) [custom_components.nmap_tracker.device_tracker] Nmap last results for 123.123.123.123/32 124.124.124.1/32 = []
2021-02-21 10:32:57 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Processor [SyncWorker_5] initialized for 123.123.123.123/32 124.124.124.1/32
2021-02-21 10:32:57 DEBUG (Thread-9) [custom_components.nmap_tracker.device_tracker] Nmap Processor thread started: Thread-9
2021-02-21 10:32:57 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Active thread MainThread - 1995841552
2021-02-21 10:32:57 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Active thread Thread-1 - 1961591904
2021-02-21 10:32:57 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Active thread SyncWorker_0 - 1951396960
2021-02-21 10:32:57 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Active thread SyncWorker_1 - 1940911200
2021-02-21 10:32:57 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Active thread SyncWorker_2 - 1930425440
2021-02-21 10:32:57 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Active thread SyncWorker_3 - 1919939680
2021-02-21 10:32:57 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Active thread SyncWorker_4 - 1909453920
2021-02-21 10:32:57 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Active thread Recorder - 1897395296
2021-02-21 10:32:57 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Active thread Thread-2 - 1886352480
2021-02-21 10:32:57 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Active thread SyncWorker_5 - 1851528288
2021-02-21 10:32:57 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Active thread SyncWorker_6 - 1843135584
2021-02-21 10:32:57 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Active thread zeroconf-Engine-738 - 1833583712
2021-02-21 10:32:57 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Active thread zeroconf-Reaper_739 - 1825191008
2021-02-21 10:32:57 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Active thread Thread-5 - 1815749728
2021-02-21 10:32:57 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Active thread Thread-6 - 1807357024
2021-02-21 10:32:57 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Active thread zeroconf-ServiceBrowser__elg._tcp.local.-_kizbox._tcp.local.-_miio._udp.local.-_wled._tcp.local.-_ipp._tcp.local.-_api._udp.local.-_hap._udp.local.-_nut._tcp.local.-_spotify-connect._tcp.local.-_homekit._tcp.local.-_hap._tcp.local.-_daap._tcp.local.-_Volumio._tcp.local.-_axis-video._tcp.local.-_viziocast._tcp.local.-_ipps._tcp.local.-_bond._tcp.local.-_touch-able._tcp.local.-_mediaremotetv._tcp.local.-_printer._tcp.local.-_googlecast._tcp.local.-_plugwise._tcp.local.-_xbmc-jsonrpc-h._tcp.local.-_ssh._tcp.local.-_esphomelib._tcp.local.-_leap._tcp.local.-_dkapi._tcp.local.-_http._tcp.local._766 - 1796232288
2021-02-21 10:32:57 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Active thread Thread-9 - 1780163680
2021-02-21 10:32:57 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Processor thread count is 2
2021-02-21 10:33:01 DEBUG (Thread-9) [custom_components.nmap_tracker.device_tracker] Nmap Command: nmap -oX - -F --host-timeout 5s 123.123.123.123/32 124.124.124.1/32
2021-02-21 10:33:02 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Processor thread completed normally after 5.011983 seconds for 123.123.123.123/32 124.124.124.1/32.
2021-02-21 10:33:02 DEBUG (SyncWorker_5) [custom_components.nmap_tracker.device_tracker] Nmap last results for 123.123.123.123/32 124.124.124.1/32 = []
2021-02-21 10:33:07 DEBUG (SyncWorker_1) [custom_components.nmap_tracker.device_tracker] Processor [SyncWorker_1] initialized for 123.123.123.123/32 124.124.124.1/32
2021-02-21 10:33:07 DEBUG (Thread-10) [custom_components.nmap_tracker.device_tracker] Nmap Processor thread started: Thread-10
2021-02-21 10:33:07 DEBUG (SyncWorker_1) [custom_components.nmap_tracker.device_tracker] Active thread MainThread - 1995841552
2021-02-21 10:33:07 DEBUG (SyncWorker_1) [custom_components.nmap_tracker.device_tracker] Active thread Thread-1 - 1961591904
2021-02-21 10:33:07 DEBUG (SyncWorker_1) [custom_components.nmap_tracker.device_tracker] Active thread SyncWorker_0 - 1951396960
2021-02-21 10:33:07 DEBUG (SyncWorker_1) [custom_components.nmap_tracker.device_tracker] Active thread SyncWorker_1 - 1940911200
2021-02-21 10:33:07 DEBUG (SyncWorker_1) [custom_components.nmap_tracker.device_tracker] Active thread SyncWorker_2 - 1930425440
2021-02-21 10:33:07 DEBUG (SyncWorker_1) [custom_components.nmap_tracker.device_tracker] Active thread SyncWorker_3 - 1919939680
2021-02-21 10:33:07 DEBUG (SyncWorker_1) [custom_components.nmap_tracker.device_tracker] Active thread SyncWorker_4 - 1909453920
2021-02-21 10:33:07 DEBUG (SyncWorker_1) [custom_components.nmap_tracker.device_tracker] Active thread Recorder - 1897395296
2021-02-21 10:33:07 DEBUG (SyncWorker_1) [custom_components.nmap_tracker.device_tracker] Active thread Thread-2 - 1886352480
2021-02-21 10:33:07 DEBUG (SyncWorker_1) [custom_components.nmap_tracker.device_tracker] Active thread SyncWorker_5 - 1851528288
rocket4321 commented

Based on these logs, the nmap command is not returning any results. Per troubleshooting steps, I would recommend performing the actual nmap command stated in logs, in the same scope as hass:

nmap -oX - -F --host-timeout 5s 123.123.123.123/32 124.124.124.1/32

I suspect it will return no results as well. It's possible you need to adjust timeouts, or not. This component is just a wrapper to the nmap command (and python package), so it can only be capable of the functions that nmap supports. If you find an nmap command that succeeds with the results you desire, then this can be achieved.

Without any devices responding, no hosts will be created or marked as 'home'. A device must be active (at least once) for it be created. Did you expect devices to be created when not responding? (That would be another change from the current component's behavior).

f45tb00t commented

Hi,

had some time today and got it finally running. But the result is not very usefull.

known_devices.yaml

xx_xx_xx_xx_xx_xx:
name: xx_xx_xx_xx_xx_xx
mac: XX:XX:XX:XX:XX:XX
icon:
picture:
track: true

at least the IP address would not be bad. So that the name could be nmap_123.123.123.123 or somewhat :)

rocket4321 commented 
Post entire configuration.yaml (should only be nmap_tracker)
Post results from working nmap command for your network

        NOTE: May require experimention and investigation into nmap command pararmeters - https://nmap.org/book/port-scanning-options.html

Activate debug log, set debug_log_level to 5 within nmap_tracker component config, and upload log to github issue
rocket4321 commented

So happy to be able to help. Wish you the best. No thanks is necessary!!!

Closing due to lack of requested issue documentation.