Service Fabric On Premise with AD Integration 401 Unauthorized

Question

Service Fabric On Premise with AD Integration 401 Unauthorized

pregress opened this issue 6 years ago · 3 comments

pregress commented 6 years ago

Do you want to request a feature or report a bug?

Bug

What did you do?

We run an on-premise Service Fabric cluster, with Active Directory Integration (Not Azure AD)
Our FabricSvc service is running under an Active Directory account, which has admin rights on the cluster and on the nodes
The Traefik.exe is running under the same Active Directory Account
When we start Traefik.exe in the cluster / or from the command line with that account we get the following errors in the log file: Provider connection error: Service Fabric responded with error code 401 Unauthorized to request http://localhost:19080/Applications/?api-version=3.0 with body {}; retrying in 588.391017ms
Is it possible the windows account is not send with the request to the Service Fabric api ?
We are able to access http://localhost:19080/Applications/?api-version=3.0 with the Account that is running the service.

What did you expect to see?

When we browse to traefik dashboard we see the following: No providers found.

What did you see instead?

We would like to see our Service Fabric services, like we see on our development environments.

Output of `traefik version`: (What version of Traefik are you using?)

Traefik version v1.7.7 built on 2019-01-08_10:21:03AM

What is your environment & configuration (arguments, toml, provider, platform, ...)?

################################################################
# Global configuration
################################################################

# Enable debug mode
#
# Optional
# Default: false
#
debug = true

# Traefik logs file
# If not defined, logs to stdout
#
# Optional
#
traefikLogsFile = "E:/ServiceFabric/traefik/traefik.log"

# Log level
#
# Optional
# Default: "ERROR"

logLevel = "DEBUG"

# Entrypoints to be used by frontends that do not specify any entrypoint.
# Each frontend can specify its own entrypoints.
#
# Optional
# Default: ["http"]
#
defaultEntryPoints = ["http", "https"]

# Entrypoints definition
#
# Optional
# Default:
[entryPoints]
[entryPoints.http]
address = ":21500"
[entryPoints.traefik]
address = ":21550"

# Enable access logs
# By default it will write to stdout and produce logs in the textual
# Common Log Format (CLF), extended with additional fields.
#
# Optional
#
# [accessLog]

# Sets the file path for the access log. If not specified, stdout will be used.
# Intermediate directories are created if necessary.
#
# Optional
# Default: os.Stdout
#
# filePath = "/path/to/log/log.txt"

# Format is either "json" or "common".
#
# Optional
# Default: "common"
#
# format = "common"

################################################################
# API definition
################################################################

[api]
  # Name of the related entry point
  #
  # Optional
  # Default: "traefik"
  #
  entryPoint = "traefik"

  # Enabled Dashboard
  #
  # Optional
  # Default: true
  #
  dashboard = true

  # Enable debug mode.
  # This will install HTTP handlers to expose Go expvars under /debug/vars and
  # pprof profiling data under /debug/pprof.
  # Additionally, the log level will be set to DEBUG.
  #
  # Optional
  # Default: false
  #
  debug = true

################################################################
# Service Fabric provider
################################################################

# Enable Service Fabric configuration backend
[servicefabric]

# Service Fabric Management Endpoint
clustermanagementurl = "http://localhost:19080"
# Note: use "https://localhost:19080" if you're using a secure cluster

# Service Fabric Management Endpoint API Version
apiversion = "3.0"

# Enable TLS connection.
#
# Optional
#
#[serviceFabric.tls]
#  cert = "certs/servicefabric.crt"
#  key = "certs/servicefabric.key"
#  insecureskipverify = true

Part of our cluster config.

"security": {
		"ClusterCredentialType": "Windows",
		"ServerCredentialType": "Windows",
		"WindowsIdentities": {
			"ClientIdentities": [
				{
					"Identity": "DOMAIN\\SA_XXXXXXXXXXXX",
					"IsAdmin": true
				},
				....
			]
		}
	},

If applicable, please paste the log output in DEBUG level (`--logLevel=DEBUG` switch)

time="2019-01-28T08:17:41+01:00" level=info msg="Using TOML configuration file D:\\ServiceFabric\\Data\\A250DOMZ01S0\\Fabric\\work\\Applications\\TraefikType_App12\\TraefikPkg.Code.1.7.7\\traefik.toml"
time="2019-01-28T08:17:41+01:00" level=info msg="Traefik version v1.7.7 built on 2019-01-08_10:21:03AM"
time="2019-01-28T08:17:41+01:00" level=debug msg="Global configuration loaded {\"LifeCycle\":{\"RequestAcceptGraceTimeout\":0,\"GraceTimeOut\":10000000000},\"GraceTimeOut\":0,\"Debug\":true,\"CheckNewVersion\":true,\"SendAnonymousUsage\":false,\"AccessLogsFile\":\"\",\"AccessLog\":null,\"TraefikLogsFile\":\"E:/ServiceFabric/traefik/traefik.log\",\"TraefikLog\":null,\"Tracing\":null,\"LogLevel\":\"DEBUG\",\"EntryPoints\":{\"http\":{\"Address\":\":21500\",\"TLS\":null,\"Redirect\":null,\"Auth\":null,\"WhitelistSourceRange\":null,\"WhiteList\":null,\"Compress\":false,\"ProxyProtocol\":null,\"ForwardedHeaders\":{\"Insecure\":true,\"TrustedIPs\":null}},\"traefik\":{\"Address\":\":21550\",\"TLS\":null,\"Redirect\":null,\"Auth\":null,\"WhitelistSourceRange\":null,\"WhiteList\":null,\"Compress\":false,\"ProxyProtocol\":null,\"ForwardedHeaders\":{\"Insecure\":true,\"TrustedIPs\":null}}},\"Cluster\":null,\"Constraints\":[],\"ACME\":null,\"DefaultEntryPoints\":[\"http\",\"https\"],\"ProvidersThrottleDuration\":2000000000,\"MaxIdleConnsPerHost\":200,\"IdleTimeout\":0,\"InsecureSkipVerify\":false,\"RootCAs\":null,\"Retry\":null,\"HealthCheck\":{\"Interval\":30000000000},\"RespondingTimeouts\":null,\"ForwardingTimeouts\":null,\"AllowMinWeightZero\":false,\"KeepTrailingSlash\":false,\"Web\":null,\"Docker\":null,\"File\":null,\"Marathon\":null,\"Consul\":null,\"ConsulCatalog\":null,\"Etcd\":null,\"Zookeeper\":null,\"Boltdb\":null,\"Kubernetes\":null,\"Mesos\":null,\"Eureka\":null,\"ECS\":null,\"Rancher\":null,\"DynamoDB\":null,\"ServiceFabric\":{\"Watch\":false,\"Filename\":\"\",\"Constraints\":null,\"Trace\":false,\"TemplateVersion\":0,\"DebugLogGeneratedTemplate\":false,\"ClusterManagementURL\":\"http://localhost:19080\",\"APIVersion\":\"3.0\",\"RefreshSeconds\":0,\"TLS\":null,\"AppInsightsClientName\":\"\",\"AppInsightsKey\":\"\",\"AppInsightsBatchSize\":0,\"AppInsightsInterval\":0},\"Rest\":null,\"API\":{\"EntryPoint\":\"traefik\",\"Dashboard\":true,\"Debug\":true,\"CurrentConfigurations\":null,\"Statistics\":null},\"Metrics\":null,\"Ping\":null,\"HostResolver\":null}"
time="2019-01-28T08:17:41+01:00" level=info msg="\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://docs.traefik.io/basics/#collected-data\n"
time="2019-01-28T08:17:41+01:00" level=warning msg="clientTLS is nil"
time="2019-01-28T08:17:41+01:00" level=info msg="Preparing server traefik &{Address::21550 TLS:<nil> Redirect:<nil> Auth:<nil> WhitelistSourceRange:[] WhiteList:<nil> Compress:false ProxyProtocol:<nil> ForwardedHeaders:0xc0004f01e0} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s"
time="2019-01-28T08:17:41+01:00" level=info msg="Preparing server http &{Address::21500 TLS:<nil> Redirect:<nil> Auth:<nil> WhitelistSourceRange:[] WhiteList:<nil> Compress:false ProxyProtocol:<nil> ForwardedHeaders:0xc0004f01a0} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s"
time="2019-01-28T08:17:41+01:00" level=info msg="Starting server on :21550"
time="2019-01-28T08:17:41+01:00" level=info msg="Starting provider configuration.ProviderAggregator {}"
time="2019-01-28T08:17:41+01:00" level=info msg="Starting server on :21500"
time="2019-01-28T08:17:41+01:00" level=info msg="Starting provider *servicefabric.Provider {\"Watch\":false,\"Filename\":\"\",\"Constraints\":null,\"Trace\":false,\"TemplateVersion\":0,\"DebugLogGeneratedTemplate\":false,\"ClusterManagementURL\":\"http://localhost:19080\",\"APIVersion\":\"3.0\",\"RefreshSeconds\":10000000000,\"TLS\":null,\"AppInsightsClientName\":\"\",\"AppInsightsKey\":\"\",\"AppInsightsBatchSize\":0,\"AppInsightsInterval\":0}"
time="2019-01-28T08:17:51+01:00" level=info msg="Checking service fabric config"
time="2019-01-28T08:17:51+01:00" level=error msg="Provider connection error: Service Fabric responded with error code 401 Unauthorized to request http://localhost:19080/Applications/?api-version=3.0 with body {}; retrying in 588.391017ms"
time="2019-01-28T08:18:01+01:00" level=info msg="Checking service fabric config"
time="2019-01-28T08:18:01+01:00" level=error msg="Provider connection error: Service Fabric responded with error code 401 Unauthorized to request http://localhost:19080/Applications/?api-version=3.0 with body {}; retrying in 507.777275ms"
time="2019-01-28T08:18:12+01:00" level=info msg="Checking service fabric config"
.... (Trimmed, because this goes on)

Answer 1 · 2019-01-28T11:51:17.000Z

Please refer to this issue for more details.
In summary it's a known limitation of the integration, it only supports certificate authentication.

Answer 2 · 2019-01-28T12:47:08.000Z

I Solved it by deploy a Proxy service for the Service fabric service, that handles the authentication for us.

private const int StreamCopyBufferSize = 81920;

        // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
        public void Configure(IApplicationBuilder app, IHostingEnvironment env)
        {
            // send every request to Service fabric api
            app.MapWhen(ctx => true, appBuilder =>
            {
                appBuilder.Run(async context =>
                {
                    var requestMessage = context.Request.ToHttpRequestMessage("localhost",19080);

                    try
                    {
                        var response = await Client.SendAsync(requestMessage);

                        using (var responseStream = await response.Content.ReadAsStreamAsync())
                        {
                            await responseStream.CopyToAsync(context.Response.Body, StreamCopyBufferSize, context.RequestAborted);
                        }
                    }
                    catch (Exception e)
                    {
                        await context.Response.WriteAsync(e.ToString());
                        context.Response.StatusCode = 500;
                    }
                });
            });
        }

        private static readonly HttpClient Client = new HttpClient(
            new HttpClientHandler
            {
                UseDefaultCredentials = true,
            })
            {
                BaseAddress = new Uri("http://localhost:19080")
            };

And I changed the clustermanagementurl to the new Proxy Service Url

Answer 3 · 2019-07-26T12:33:27.000Z

To build on @pregress's answer, this is now even simpler with the Microsoft.AspNetCore.Proxy package (assuming you're using ASP.NET Core):

<PackageReference Include="Microsoft.AspNetCore.Proxy" Version="0.2" />

public class Startup
{
        public void ConfigureServices(IServiceCollection services)
        {

        }

        public void Configure(IApplicationBuilder app, IHostingEnvironment env)
        {
            app.RunProxy(new ProxyOptions
            {
                Host = "localhost",
                Port = "19080",
                Scheme = "http",
                BackChannelMessageHandler = new HttpClientHandler() { UseDefaultCredentials = true }
            });
        }
    }