Wildcard domains and cluster Reverse proxy?

[jezzsantos]

I have a local cluster running using all defaults. I am migrating existing services from Azure CloudServices to Azure ServiceFabric, but doing this work locally first, on a local cluster in Windows.

I have 3 existing .NETCore services, that I have converted to StatelessService, each one configured for HTTPS on its own fixed port.
For example:
service1 is on port 4431
service2 is on port 4432
service3 is on port 4433.

I have configured these services correctly (in the ServiceManifest and ApplicationManifest), and deployed them into the cluster successfully. They are all running in the cluster and I can reach each and every one of them from the desktop on their respective ports (from outside the cluster).
For example:
GET https://localhost:4431/api/health gives me a JSON response.

I have also set up a wildcard self-signed cert (for *.localhost.dev) and added entries into C:\windows\system32\drivers\etc\hosts file to work for multiple values of the wildcard subdomain:
For example:
127.0.0.1 acme.localhost.dev
127.0.0.1 contoso.localhost.dev
127.0.0.1 fabricam.localhost.dev

Now, each service uses a host header to determine which customer tenancy the service is for.
For example:
https://acme.localhost.dev.com:4431/api/health is the endpoint address for the acme tenant. The subdomain is how know which customer is calling our services.

This is working fine also:
For example:
GET https://acme.localhost.dev:4431/api/health gives me a JSON response.

If service1 is going to call an API on service2 (internally within the same cluster), and we do that using an ordinary HTTP Client via cluster reverse proxy, the URL of the request to service2 would be: https://acme.localhost.dev:19081/AppName/Service2Name/api/health.

I see the call being proxied to service2, but the host header is my machine name, but I also do see the X-Forward-Host header containing the original host header acme.localhost.dev:19081. Which is all I need at service2 to process the request.

My Problem

I can see service2 receiving the request, AND creating an HTTP response, but the response arrives back at `service1 as empty, after going back through the proxy. Nothing in the response!

What could be wrong?

[jezzsantos]

I discovered that I was getting an exception in my service: Synchronous operations are disallowed. Call WriteAsync or set AllowSynchronousIO to true instead. coming from somewhere in the middleware. (we are using ServiceStack).

So, I needed to add this: kestrelOptions.AllowSynchronousIO = true

Now I get non-null responses