Can not set correct URL or Port for operation behind proxy

First of all, thanks for the integration!

My HA system runs in a Docker Swarm stack behind a TLS terminating reverse proxy.
The proper event server is accessible from an HTTPS address on port 443.

I'm unable to use the integration without modifying the code each time it's upgraded.
The reason is the logic in these lines:

hacs-hubitat/custom_components/hubitat/hubitatmaker/hub.py

Lines 471 to 528 in 3b55157

    
               async def _start_server(self) -> None: 
        
                   """Start an event listener server.""" 
        
                   # First, figure out what address to listen on. Open a connection to 
        
                   # the Hubitat hub and see what address it used. This assumes this 
        
                   # machine and the Hubitat hub are on the same network. 
        
                   with _open_socket(socket.AF_INET, socket.SOCK_DGRAM) as s: 
        
                       s.connect((self.host, 80)) 
        
                       address = s.getsockname()[0] 
        
                   self._server = server.create_server( 
        
                       self._process_event, address, self.port or 0, self.ssl_context 
        
                   ) 
        
                   self._server.start() 
        
                   _LOGGER.debug( 
        
                       "Listening on %s:%d with SSL %s", 
        
                       address, 
        
                       self._server.port, 
        
                       "disabled" if self.ssl_context is None else "enabled", 
        
                   ) 
        
                   await self.set_event_url(self.event_url) 
        
           @contextmanager 
        
           def _open_socket(*args: Any, **kwargs: Any) -> Iterator[socket.socket]: 
        
               """Open a socket as a context manager.""" 
        
               s = socket.socket(*args, **kwargs) 
        
               try: 
        
                   yield s 
        
               finally: 
        
                   s.close() 
        
           def _get_event_port(port: int | None, event_url: str | None) -> int | None: 
        
               """Given an optional port and event URL, return the event port""" 
        
               if port is not None: 
        
                   return port 
        
               if event_url is not None: 
        
                   u = urlparse(event_url) 
        
                   return u.port 
        
               return None 
        
           def _get_event_url(port: int | None, event_url: str | None) -> str | None: 
        
               """Given an optional port and event URL, return a complete event URL""" 
        
               if event_url is not None: 
        
                   u = urlparse(event_url) 
        
                   if u.port is None and port is not None: 
        
                       return ParseResult( 
        
                           scheme=u.scheme, 
        
                           netloc=f"{u.hostname}:{port}", 
        
                           path=u.path, 
        
                           params=u.params, 
        
                           query=u.query, 
        
                           fragment=u.fragment, 
        
                       ).geturl() 
        
                   return event_url 
        
               return None

I've not created an HA integration so it would take me some time to submit a proper PR for the changes I think this needs.
I'd like to solicit any feedback before I set out to complete these changes for others to use.

The optional Event server URL is pushed to HE every time the HA integration is started.
The value I set in this field is the value I also have configured in HE, every time the integration restarts, I have to manually reset it in HE.
I would expect the value in this field to be THE value I'm using in HE, not the value with a listening port number appended to it.
I think an optional config bool to disable updating the HE URL would resolve this for me.
_get_event_port
The optional config value is used not just for the listening port, but it's also forced onto the event server URL.
Change 1 above should prevent this from being an issue for my use case, however, it might be better to add another config option for "Event server listening port (optional)".
If the new listening port config value is not set, follow the current port config item's logic. Otherwise, this is the new port to listen on.
_start_server
The address variable is derived from the interface used to test a connection to HE, then the value is forced into server.create_server.
I can understand why it's being done this way. However, I have multiple interfaces and the reverse proxy only talks to 1 of them, and it's not the one used for outbound traffic to HE.
The default value in server.create_server is already 0.0.0.0 and that's exactly what I want to use.
A new config option "Event server listening interface (optional)" could address this.
If the new listening interface option is not set, follow the current logic. Otherwise, this is the interface ip to listen on.
This would also be able to support inbound via IPv6 by setting it to :: whereas IPv4 would be 0.0.0.0.

I don't use HA behind a proxy myself, but a number of others do, and I think that use case is handled by the integration. You'll want to specify both a port and a URL for the event server in the integration config. This should handle the proxy use case, unless the proxy address that maps to event server is dynamic.

The port value in the integration config fixes the port the event server listens on; this gives a proxy a known port to map to. This port may be different from the port in the event server URL, since the proxy can map an external port to a different internal port.

The URL in the integration config is for the Maker API -- it tells Hubitat what address it should use to post events to the event server. If you're manually specifying an event URL in Maker API now, set that as your event server URL in the integration config.

So, if your proxy was at https://myproxy.local and you wanted the event server to be accessible at https://myproxy.local:8234, you'd set the integration's port value to some value and configure your proxy to map port 8234 to that, and you'd set the integration's event server URL to https://myproxy.local:8234, because that's the address that Hubitat should post events to. Every time the integration starts up, its event server would be listening on the configured port, and it would tell Hubitat to listen at the proper address.

This setup wouldn't work if the proxy address was dynamic and the proxy itself was configuring the Maker API event address.

After looking closer at the code I was able to see that the external port could be overridden in the URL.
However I still have a problem of the listen address being the IP used to talk to HE instead of 0.0.0.0.

Interesting, that's a setup I haven't run into yet. Your HA host can send outbound traffic on one interface, but can only receive on another?

Correct.
It's multiple physical hosts with Docker Swarm for failover.
Each host has a NIC in the DMZ and DMZT (traversal) zones.
A reverse proxy controls inbound requests to each host and terminates TLS.
If one host is down, HA will run on one of the others.

If Hubitat wants to talk to HA, it will go through the proxy → to one of the host's physical DMZ IP → to the container's virtual IP.
However if HA wants to talk to Hubitat, it go from the container's virtual IP → to the host's DMZT IP for the correct subnet → through the firewall → to Hubitat.

The reverse proxy is configured to send any traffic to this integration's hostname to a static port in the HA container.
Inbound requests to the proxy are over port 443 so that I don't have to open another external port just for this.

I actually just finished updating it on my side so that I don't have to do the manual static values I was doing before.

Example UI:

I'm definitely not opposed to updating the integration to handle this situation better. Is everything in #284 necessary, though? It seems like simply updating the event server to listen on 0.0.0.0 instead of a specific interface would resolve your use case while still working in existing use cases.

I was already modifying the _start_server function manually after updates to set address to 0.0.0.0.
I just didn't want to keep doing this manually.

I wasn't sure what the minimum requirements were to display a new config option and just followed the logic you already had.

Most of the changes were to feed the two new config options into the hub startup.
The reason I made them config options was so that existing deployments were not affected.

With the PR, nothing changes for existing installs, but the options are there if needed.

I can see how the server listening IP makes sense, but why is the option needed for the HE POST URL?

Also, is an option for the listening address needed, or could the integration simply always listen on 0.0.0.0? (Looking at my issue history, it appears I actually planned to do this, but never got around to it.)

I'm not trying to be difficult, I want to make sure I understand the root cause of the issue. 🙂

Oh wow, issue hubitatmaker/#12 is exactly the scenario I have and that's also how I was solving it.
I see no issue from my perspective with setting the default to 0.0.0.0.
I just didn't want to create a potentially breaking change.

The reason it could be a breaking change is if users have already implemented solutions based on it using the interface routing to HE. I'm not sure what your install count is or how many users it could impact.
It may not be an issue at all for others.

As for the update_he_url option, I don't want the integration to manage the URL in my HE environment.
That's because this integration is not aware of the complexities of the network and the proxies HE will have to traverse to reach it. Nor should it have to be concerned about these things.

Updating the post url is great for a system where a static port is not defined and therefore changes every restart.
An option that defaults to true keeps the status quo and allows advanced environments to function without change.

@ja-finn in issue #120 is an example of how forcing the change could break for existing users.
They have already created a workaround for the way it currently works.

@ja-finn in issue #120 is an example of how forcing the change could break for existing users.
They have already created a workaround for the way it currently works.

Excellent point.

Updating the post url is great for a system where a static port is not defined and therefore changes every restart.

It works as well for a system where a static port is defined -- if you set the event server URL value in the integration, the value being set in HE by the integration is your value, not anything generated by the integration. But, I can see wanting to just keep the integration out of it entirely.

	async def _start_server(self) -> None:
	"""Start an event listener server."""
	# First, figure out what address to listen on. Open a connection to
	# the Hubitat hub and see what address it used. This assumes this
	# machine and the Hubitat hub are on the same network.
	with _open_socket(socket.AF_INET, socket.SOCK_DGRAM) as s:
	s.connect((self.host, 80))
	address = s.getsockname()[0]

	self._server = server.create_server(
	self._process_event, address, self.port or 0, self.ssl_context
	)
	self._server.start()
	_LOGGER.debug(
	"Listening on %s:%d with SSL %s",
	address,
	self._server.port,
	"disabled" if self.ssl_context is None else "enabled",
	)

	await self.set_event_url(self.event_url)


	@contextmanager
	def _open_socket(args: Any, *kwargs: Any) -> Iterator[socket.socket]:
	"""Open a socket as a context manager."""
	s = socket.socket(args, *kwargs)
	try:
	yield s
	finally:
	s.close()


	def _get_event_port(port: int \| None, event_url: str \| None) -> int \| None:
	"""Given an optional port and event URL, return the event port"""
	if port is not None:
	return port
	if event_url is not None:
	u = urlparse(event_url)
	return u.port
	return None


	def _get_event_url(port: int \| None, event_url: str \| None) -> str \| None:
	"""Given an optional port and event URL, return a complete event URL"""
	if event_url is not None:
	u = urlparse(event_url)
	if u.port is None and port is not None:
	return ParseResult(
	scheme=u.scheme,
	netloc=f"{u.hostname}:{port}",
	path=u.path,
	params=u.params,
	query=u.query,
	fragment=u.fragment,
	).geturl()
	return event_url
	return None