M2 Mac - list_resources() not returning available TCPIP instrument

Question

M2 Mac - list_resources() not returning available TCPIP instrument

dstrande opened this issue a year ago · 22 comments

On an M2 Mac, rm.list_resources() isn't returning an available TCPIP instrument.

rm = pyvisa.ResourceManager("@py")
print(rm.list_resources())

inst = rm.open_resource("TCPIP::169.254.4.179::INSTR")
print(inst.query("*IDN?"))

Returns:

()
KEITHLEY INSTRUMENTS,MODEL 2450,04505775,1.7.12b

Showing the instrument will connect but fails to show for list_resources.

I tried doing a bit of debugging but not totally sure what I was doing:

resp = pmap.recv_port((vxi11.DEVICE_CORE_PROG, vxi11.DEVICE_CORE_VERS, rpc.IPPROTO_TCP, 0))

seems to fail to return any response on Mac but succeeds on Windows.

To Reproduce

Steps to reproduce the behavior:

Connect instrument withFind IP address of your instrument
Use code above with a ARM Mac (?) replacing IP address with your instrument's

Output of pyvisa-info

Machine Details:
   Platform ID:    macOS-13.5-arm64-arm-64bit
   Processor:      arm

Python:
   Implementation: CPython
   Executable:     /***
   Version:        3.11.6
   Compiler:       Clang 15.0.7 
   Architecture:   ('arm', 64)
   Build:          Oct  3 2023 10:37:07 (#main)
   Unicode:        UCS4

PyVISA Version: 1.14.1

Backends:
   ivi:
      Version: 1.14.1 (bundled with PyVISA)
      Binary library: Not found
   py:
      Version: 0.7.1
      TCPIP INSTR: Available 
         Resource discovery:
         - VXI-11: ok
         - hislip: ok
      TCPIP SOCKET: Available 
      ASRL INSTR:
         Please install PySerial (>=3.0) to use this resource type.
         No module named 'serial'
      USB INSTR:
         Please install PyUSB to use this resource type.
         No module named 'usb'
      USB RAW:
         Please install PyUSB to use this resource type.
         No module named 'usb'
      VICP INSTR:
         Please install PyVICP to use this resource type.
      GPIB INSTR:
         Please install linux-gpib (Linux) or gpib-ctypes (Windows, Linux) to use this resource type. Note that installing gpib-ctypes will give you access to a broader range of functionalities.
         No module named 'gpib'
      GPIB INTFC:
         Please install linux-gpib (Linux) or gpib-ctypes (Windows, Linux) to use this resource type. Note that installing gpib-ctypes will give you access to a broader range of functionalities.
         No module named 'gpib'

Answer 1 · 2023-11-28T15:08:09.000Z

Could you instrument a bit the code so that we get a better idea of what is going on ?
I think it may be helpful to:

print/log the list of discovery addresses over which we broadcast
check the pmap_list to see if broadcast fails on any address
increase the sleep duration

Answer 2 · 2023-11-29T08:10:22.000Z

Reading my previous message I realize I was a vague. The function to instrument is the following: https://github.com/pyvisa/pyvisa-py/blob/main/pyvisa_py/tcpip.py#L394

Answer 3 · 2023-11-29T14:56:13.000Z

It seems like the send succeeds but doesn't receive a response.

I pulled out this part of the code and ran it:

import socket
import ipaddress
from pyvisa_py.protocols import vxi11, rpc
import time
import psutil


broadcast_addr = []
for interface, snics in psutil.net_if_addrs().items():
    for snic in snics:
        if snic.family is socket.AF_INET:
            addr = snic.address
            mask = snic.netmask
            network = ipaddress.IPv4Network(addr + "/" + mask, strict=False)
            broadcast_addr.append(str(network.broadcast_address))


print("addresses: ", broadcast_addr)
pmap_list = [rpc.BroadcastUDPPortMapperClient(ip) for ip in broadcast_addr]
for pmap in list(pmap_list):
    pmap.set_timeout(0)
    pmap.send_port((vxi11.DEVICE_CORE_PROG, vxi11.DEVICE_CORE_VERS, rpc.IPPROTO_TCP, 0))

# Timeout for responses
print("pmap send: ", pmap_list)
time.sleep(5)

all_res = []
for pmap in pmap_list:
    resp = pmap.recv_port(
        (vxi11.DEVICE_CORE_PROG, vxi11.DEVICE_CORE_VERS, rpc.IPPROTO_TCP, 0)
    )
    print("response: ", resp)

    res = [r[1][0] for r in resp if r[0] > 0]
    res = sorted(res, key=lambda ip: tuple(int(part) for part in ip.split(".")))
    # TODO: Detect GPIB over TCPIP
    res = ["TCPIP::{}::INSTR".format(host) for host in res]
    all_res.extend(res)

Which returns:

addresses:  ['127.255.255.255', '10.0.0.255', '169.254.255.255']
pmap send:  [<pyvisa_py.protocols.rpc.BroadcastUDPPortMapperClient object at 0x10c099650>, <pyvisa_py.protocols.rpc.BroadcastUDPPortMapperClient object at 0x105d77510>, <pyvisa_py.protocols.rpc.BroadcastUDPPortMapperClient object at 0x106a6c090>]
response:  []
response:  []
response:  []

Answer 4 · 2023-11-29T15:28:52.000Z

Also here's the output on Windows:

addresses:  ['169.254.255.255', '10.0.0.255', '169.254.255.255', '127.255.255.255']
pmap send:  [<pyvisa_py.protocols.rpc.BroadcastUDPPortMapperClient object at 0x000001A8746D7650>, <pyvisa_py.protocols.rpc.BroadcastUDPPortMapperClient object at 0x000001A8748FC710>, <pyvisa_py.protocols.rpc.BroadcastUDPPortMapperClient object at 0x000001A874695F50>, <pyvisa_py.protocols.rpc.BroadcastUDPPortMapperClient object at 0x000001A8746A0BD0>]
response:  [(621, ('169.254.145.144', 111))]
response:  []
response:  [(621, ('169.254.145.144', 111))]
response:  []

Answer 5 · 2023-11-29T15:34:29.000Z

Thanks ! Having the M2/Windows comparison is nice.

So we know that we are indeed sending requests and that we would be able to reach the instrument since we address the right subnet. So we need to understand why we do not get any response on Mac. I am not familiar with MacOS firewall settings but it may worth checking that it is not filtering either the query or responses.

Answer 6 · 2023-11-29T15:54:44.000Z

I'm also not too familiar. Seems like my firewall is disabled though. If I ping the instrument I get a response as well:

PING 169.254.145.144 (169.254.145.144): 56 data bytes
64 bytes from 169.254.145.144: icmp_seq=0 ttl=64 time=0.567 ms
64 bytes from 169.254.145.144: icmp_seq=1 ttl=64 time=0.586 ms
64 bytes from 169.254.145.144: icmp_seq=2 ttl=64 time=0.589 ms
64 bytes from 169.254.145.144: icmp_seq=3 ttl=64 time=0.631 ms

Answer 7 · 2023-11-29T16:07:03.000Z

Since I do not have a Mac on hand this is getting hard. Could you try to install wireshark and look at the traffic on the subnet 169.254 ?

Answer 8 · 2023-11-29T16:27:15.000Z

Not fully sure what I'm looking at but the packets in blue occur regularly and the packet in yellow is from the script above.

Answer 9 · 2023-11-30T09:36:47.000Z

Thanks. Could you do something similar on windows and confirm the IP address of the Mac on the 1698.254 subnet ?

If by chance you also have access to a linux setup, I would be curious to know what result you get on it.

Answer 10 · 2023-11-30T16:01:13.000Z

Unfortunately I don't have access to linux. I can get a usb boot if you think it'll make a difference.

On Windows it's different, unsurprisingly I guess. The script starts with No. 43.

Answer 11 · 2023-11-30T16:32:49.000Z

Also I've confirmed the 169.254 subnet on mac:

% ipconfig getifaddr en6
169.254.185.47

Answer 12 · 2023-11-30T21:15:23.000Z

Maybe this isn't surprising, but I changed the test script above to include the devices address from the start ( line 8 broadcast_addr = ["169.254.145.145"]). And this is what is returned:

addresses:  ['169.254.145.145', '127.255.255.255', '10.0.0.255', '169.254.255.255']
pmap send:  [<pyvisa_py.protocols.rpc.BroadcastUDPPortMapperClient object at 0x10124fe10>, <pyvisa_py.protocols.rpc.BroadcastUDPPortMapperClient object at 0x100daf590>, <pyvisa_py.protocols.rpc.BroadcastUDPPortMapperClient object at 0x1026c9650>, <pyvisa_py.protocols.rpc.BroadcastUDPPortMapperClient object at 0x100c960d0>]
b'\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02Y' ('169.254.145.145', 111)
response:  [(601, ('169.254.145.145', 111))]
response:  []
response:  []
response:  []

Answer 13 · 2023-11-30T22:04:23.000Z

Hi Dallas, have you tried putting the computers on a network with some kind of router present? Using APIPA addresses 169.254.x.x might behave differently to using a private subnet with addresses like 10.0.x.x or 192.168.x.x. A network broadcast to the APIPA broadcast address 169.254.255.255 might be treated as a special case by a firewall, Michael Wilkinson

…

On Thu, 2023-11-30 at 13:15 -0800, Dallas Strandell wrote: Maybe this isn't surprising, but I changed the test script above to include the devices address from the start ( line 8 broadcast_addr = ["169.254.145.145"]). And this is what is returned: addresses: ['169.254.145.145', '127.255.255.255', '10.0.0.255', '169.254.255.255'] pmap send: [<pyvisa_py.protocols.rpc.BroadcastUDPPortMapperClient object at 0x10124fe10>, <pyvisa_py.protocols.rpc.BroadcastUDPPortMapperClient object at 0x100daf590>, <pyvisa_py.protocols.rpc.BroadcastUDPPortMapperClient object at 0x1026c9650>, <pyvisa_py.protocols.rpc.BroadcastUDPPortMapperClient object at 0x100c960d0>] b'\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02Y' ('169.254.145.145', 111) response: [(601, ('169.254.145.145', 111))] response: [] response: [] response: [] — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: <pyvisa/pyvisa- ***@***.***>

Answer 14 · 2023-12-01T08:57:35.000Z

@CompThing may well be onto something. It is at the very least worth checking and documenting if it turns out it is the root cause.

Answer 15 · 2023-12-01T14:48:13.000Z

Yes you're right. If I plug something into my network instead of directly into the Mac, it shows up for list_resources:

('ASRL/dev/cu.Bluetooth-Incoming-Port::INSTR', 'TCPIP::10.0.0.219::INSTR')

Answer 16 · 2023-12-01T15:00:26.000Z

Could you try using a setting a static IP address for your Mac (in 192.168.) and see if it also works ?

If yes it means we simply need to document that relying on APIPA address is not supported.

Answer 17 · 2023-12-01T15:05:19.000Z

No unfortunately. I did try that originally and it still doesn't work.

Answer 18 · 2023-12-01T15:15:45.000Z

You would also need to give the test instrument a static IP address within the same subnet for it to work

…

On Fri, 2023-12-01 at 07:05 -0800, Dallas Strandell wrote: No unfortunately. I did try that originally and it still doesn't work. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

Answer 19 · 2023-12-01T15:32:16.000Z

Yeah 169.254.100.101 / 255.255.0.0 and 169.254.145.45 / 255.255.0.0 on the instrument. No gateway needed correct?

Answer 20 · 2023-12-01T15:51:20.000Z

Hi Dallas, if you make the addresses: 192.168.1.100/255.255.255.0 and 192.168.1.101/255.255.255.0 they will both have static IP addresses on the same private subnet. No gateway needed, but if the config requires one, you could set it as 192.168.1.1 169.255.x.x addresses should not be assigned statically and are not a normal subnet. They belong to a special set of addresses for APIPA which is a very old Windows way of auto-negotiating addresses without a proper network. It does not play well with modern networking and should be avoided. Hope that helps, Michael Wilkinson

…

On Fri, 2023-12-01 at 07:32 -0800, Dallas Strandell wrote: Yeah 169.254.100.101 / 255.255.0.0 and 169.254.145.45 / 255.255.0.0 on the instrument. No gateway needed correct? Screenshot.2023-12-01.at.7.31.05.AM.png (view on web) — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

Answer 21 · 2023-12-01T16:32:49.000Z

Yeah that seems to work. Weird that Mac default to 255.255.0.0.

So for link-local Mac requires a 255.255.255.0 subnet, but no change for normal connections through a router.

Answer 22 · 2023-12-01T19:20:03.000Z

I think it is not so much a question of mask than it is a question of the ip address prefix. As @CompThing pointed out the 169.255 prefix is special and this appears to be the root issue of your problem.

Could you make a PR adding this piece of information to the documentation ?