The reverse proxy is an application, supplied out of band from the service fabric distribution, that customers deploy to their clusters and handles proxying traffic to backend services. The service, that potentially runs on every node in the cluster, takes care of handling endpoint resolution, automatic retry, and other connection failures on behalf of the clients. The reverse proxy can be configured to apply various policies as it handles requests from client services.
Using a reverse proxy allows the client service to use any client-side HTTP communication libraries and does not require special resolution and retry logic in the service. The reverse proxy is mostly a terminating endpoint for the TLS connections unless the TCP option is used.
Note that, at this time, this is a reverse proxy built-in replacement and not a generic service fabric “gateway” able to handle partition queries, but that might be added (via customer written plugins or similar) in the future.
As of this release, the services need to be explicitly exposed via service extension labels, enabling the proxying (HTTP/TCP) functionality for a particular service and endpoint. With the right labels’ setup, the reverse proxy will expose one or more endpoints on the local nodes for client services to use. The ports can then be exposed to the load balancer in order to get the services available outside of the cluster. The required certificates needed should be already deployed to the nodes where the proxy is running as is the case with any other Service Fabric application.
You can clone the repo, build, and deploy or simply grab the latest ZIP/SFPKG application from Releases section, modify configs, and deploy.
After either downloading the sf app package from the releases or cloning the repo and building, you need to adjust the configuration settings to meet to your needs (this means changing settings in Settings.xml, ApplicationManifest.xml and any other changes needed for the traefik-template.yaml configuration).
If you need a quick test cluster, you can deploy a test Service Fabric managed cluster following the instructions from here: SFMC, or via this template if you already have a client certificate and thumbprint available: Deploy
Retrieve the cluster certificate TP using: $serverThumbprint = (Get-AzResource -ResourceId /subscriptions/$SUBSCRIPTION/resourceGroups/$RESOURCEGROUP/providers/Microsoft.ServiceFabric/managedclusters/$CLUSTERNAME).Properties.clusterCertificateThumbprints
#cd to the top level directory where you downloaded the package zip
cd \downloads
#Expand the zip file
Expand-Archive .\service-fabric-traefik.zip -Force
#cd to the directory that holds the application package
cd .\service-fabric-traefik\windows\
#create a $appPath variable that points to the application location:
#E.g., for Windows deployments:
$appPath = "C:\downloads\service-fabric-traefik\windows\TraefikProxyApp"
#For Linux deployments:
#$appPath = "C:\downloads\service-fabric-traefik\linux\TraefikProxyApp"
#Connect to target cluster, for example:
Connect-ServiceFabricCluster -ConnectionEndpoint @('sf-win-cluster.westus2.cloudapp.azure.com:19000') -X509Credential -FindType FindByThumbprint -FindValue '[Client_TP]' -StoreLocation LocalMachine -StoreName 'My' # -ServerCertThumbprint [Server_TP]
# Use this to remove a previous Traefik Application
#Remove-ServiceFabricApplication -ApplicationName fabric:/traefik -Force
#Unregister-ServiceFabricApplicationType -ApplicationTypeName TraefikType -ApplicationTypeVersion 1.0.0 -Force
#Copy and register and run the Traefik Application
Copy-ServiceFabricApplicationPackage -CompressPackage -ApplicationPackagePath $appPath # -ApplicationPackagePathInImageStore traefik
Register-ServiceFabricApplicationType -ApplicationPathInImageStore traefik
#Fill the right values that are suitable for your cluster and application (the default ones below will work without modification if you used a Service Fabric managed cluster Quickstart template with one node type. Adjust the placement constraints to use other node types)
$p = @{
ReverseProxy_InstanceCount="1"
ReverseProxy_FetcherEndpoint="7777"
ReverseProxy_HttpPort="8080"
ReverseProxy_CertificateSearchKeyword=""
ClusterEndpoint="https://localhost:19080"
CertStoreSearchKey="sfmc"
ClientCertificate=""
ClientCertificatePK=""
ReverseProxy_EnableDashboard="true"
#ReverseProxy_PlacementConstraints="NodeType == NT2"
}
$p
New-ServiceFabricApplication -ApplicationName fabric:/traefik -ApplicationTypeName TraefikType -ApplicationTypeVersion 1.0.0 -ApplicationParameter $p
#OR if updating existing version:
Start-ServiceFabricApplicationUpgrade -ApplicationName fabric:/traefik -ApplicationTypeVersion 1.0.0 -Monitored -FailureAction rollbackThis is a sample SF enabled service showing some of the supported labels. If the sf name is fabric:/pinger/PingerService, the endpoint [endpointName] will be expose at that prefix: '/pinger/PingerService/'.
...
<ServiceTypes>
<StatelessServiceType ServiceTypeName="PingerServiceType" UseImplicitHost="true">
<Extensions>
<Extension Name="traefik">
<Labels xmlns="http://schemas.microsoft.com/2015/03/fabact-no-schema">
<Label Key="traefik.http.defaultEP">true</Label>
<Label Key="traefik.http.defaultEP.service.loadbalancer.passhostheader">true</Label>
<Label Key="traefik.http.defaultEP.service.loadbalancer.healthcheck.path">/</Label>
<Label Key="traefik.http.defaultEP.service.loadbalancer.healthcheck.interval">10s</Label>
<Label Key="traefik.http.defaultEP.service.loadbalancer.healthcheck.scheme">http</Label>
</Labels>
</Extension>
</Extensions>
</StatelessServiceType>
</ServiceTypes>
...The only required label to expose a service via the reverse proxy is the traefik.http.[endpointName] set to true. Setting only this label will expose the service on a well known path and handle the basic scenarios.
http(s)://<Cluster FQDN | internal IP>:Port/ApplicationInstanceName/ServiceInstanceName?PartitionGuid=xxxxx
If you need to change the routes or add middleware then you can add different labels configuring them.
Most of all the traefik http, tcp and tls dynamic configurations can be configured via their corresponding service extension label. The list below is only an overview of all supported labels. Refer to exampleServiceManifest.xml under /eng to view a more comprehensive list. If a label is missing then follow a similar format to add the traefik config that you desire.
Http enable section
- traefik.http.[endpointName] Enables exposing an http service via the reverse proxy ['true']
Router section
- traefik.http.[endpointName].router.rule Traefik rule to apply [PathPrefix(
/api))]. This rule is added on top of the default path generation. If this is set, you have to define a middleware to remove the prefix for the service to receive the stripped path. - traefik.http.[endpointName].router.tls.options Enable TLS on the route ['optionName'].
Loadbalancer section
- traefik.http.[endpointName].service.loadbalancer.passhostheader passhostheaders ['true'/'false']
- traefik.http.[endpointName].service.loadbalancer.serversTransport serversTransport name ['serversTransportName']
- traefik.http.[endpointName].service.loadbalancer.healthcheck.path Healthcheck endpoint path ['/healtz']
- traefik.http.[endpointName].service.loadbalancer.healthcheck.interval Healthcheck interval ['10s']
- traefik.http.[endpointName].service.loadbalancer.healthcheck.scheme Healthcheck scheme ['http']
- traefik.http.[endpointName].service.loadbalancer.timeout.interval Healthcheck timeout ['30s']
- traefik.http.[endpointName].service.loadbalancer.sticky.cookie.sameSite Sticky session affinity cookie ['none'/'lax'/'strict']
Middleware section
- traefik.http.[endpointName].middlewares.[middlewareName].stripPrefix.prefixes prefix to strip ['/api']
TLS section
- traefik.tls.option.[optionName].minVersion tls min version option ['VersionTLS12']
- traefik.tls.store.default.[defaultCertificate].certFile path to cert ['path/to/cert.crt']
- traefik.tls.store.default.[defaultCertificate].keyFile path to cert key ['path/to/cert.key']
- traefik.tls.certificate.[certName].certFile path to cert ['path/to/cert.cert']
- traefik.tls.certificate.[certName].keyFile path to cert key ['path/to/cert.key']
ServersTransport section
- traefik.http.serversTransport.[serversTransportName].insecureSkipVerify disables SSL certificate verification ['true'/'false']
A sample test application, that is included in the release, can be deployed to test everything is working. After deployment, you should be able to reach it at:
https://your-cluster:8080/pinger0/PingerService/id
Note that the service is going to be exposed on https since the service has a label for the route.tls option. You can explore that looking at the service manifest for this app.
# Sample pinger app for validating (navidate to /pinger0/PingerService/id on https)
#Remove-ServiceFabricApplication -ApplicationName fabric:/pinger$i -Force
#Unregister-ServiceFabricApplicationType -ApplicationTypeName PingerApplicationType -ApplicationTypeVersion 1.0 -Force
$appPath = "C:\downloads\service-fabric-traefik\windows\pinger-traefik"
Copy-ServiceFabricApplicationPackage -CompressPackage -ApplicationPackagePath $appPath -ApplicationPackagePathInImageStore pinger-traefik
Register-ServiceFabricApplicationType -ApplicationPathInImageStore pinger-traefik
$p = @{
"Pinger_Instance_Count"="3"
"Pinger_Port"="7000"
#"Pinger_PlacementConstraints"= "NodeType == NT2"
}
New-ServiceFabricApplication -ApplicationName fabric:/pinger0 -ApplicationTypeName PingerApplicationType -ApplicationTypeVersion 1.0 -ApplicationParameter $p
This repo includes:
TraefikProxyApp: an example Service Fabric application, referencing two guest executables:server.exe: Performs service discovery on a sf cluster, fetches endpoint information and publishes the config to be consumed in real-timetraefik.exe: Implements a reverse proxy using Traefik config read from the dynamic config file
-
Users can clone the github repo and make changes to server/fetcher app under /serviceFabricDiscoveryService. Once changes have been made the binary can be manually built ("go build ./cmd/server") and moved to the correct sf code package ("./TraefikProxyApp/ApplicationPackageRoot/TraefikPkg/Fetcher.Code").
-
Get latest traefik.exe from traefik github page and place it under its corresponding sf code package ("./TraefikProxyApp/ApplicationPackageRoot/TraefikPkg/Code")
TraefikProxyApp comes with an ApplicationManifest and ServiceManifest for both windows and linux. By default the .xml files contains the content for a windows cluster deployment. For a linux deployment need to replace with the linux xml content.
Also, you can open TraefikSF.sln at the root of the repo with Visual Studio 2019.
Running builds and deploying to local or remote SF clusters instead of using local powershell prompt.
serviceFabricDiscoveryService is a service that connects to a Service Fabric cluster and exposes discovery data and changes [async] over websockets or, locally, via a file. Changes on names (applications/services) and endpoint mapping information is sent as messages over the websocket as they happen, the client doesn't have to poll the server.
The service exposes several websocket routes that have specific functionality.
NAME:
discoveryService - exposes service fabric application and service metadata over websockets
USAGE:
server.exe [global options] command [command options] [arguments...]
COMMANDS:
run runs as a server
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--loglevel value, -l value debug level, one of: info, debug (default: "info") [%LOGLEVEL%]
--help, -h show help (default: false)
NAME:
server.exe run - runs as a server
USAGE:
server.exe run [command options] [arguments...]
OPTIONS:
--clusterEndpoint value, -e value cluster endpoint [http://localhost:19080] [%CLUSTER_ENDPOINT%]
--clientCertificate value path or content for the client certificate [%CLIENT_CERT%]
--clientCertificatePK value path or content for the client certificate private key [%CLIENT_CERT_PK%]
--certStoreSearchKey value, -k value keyword to look for searching the cluster certificate (windows cert store) [%CLUSTER_CERT_SEARCH_KEY%]
--httpport value, -p value port for the HTTP rest endpoint (server will be disabled if not provided) (default: 0) [%HTTP_PORT%]
--insecureTLS, -i allow skip checking server CA/hostname (default: false) [%INSECURE_TLS%]
--publishFilePath value, -f value filename to write to, empty won't write anywhere [%PUBLISH_FILE_PATH%]
--help, -h show help (default: false)
This route exposes a stream of Traefik 2.x compatible yaml data that can be fed directly into the Traefik file provider. The returned data maps routing rules for service instances running on the cluster, taking into account the Health and Status of each of the services in order to ensure requests are only routed to healthy service instances.
-
Deploy
TraefikProxyAppto the local cluster -
Deploy the pinger test application mentioned in Sample-Test-Application. Using a browser access
https://localhost/pinger0/PingerService. If all works, you should get a200 OKresponse with contents resembling the following:{ "Pinger: I'm alive on ... " }
The discovery mechanism is based on Traefik made by Traefik Labs.
This software is released under the MIT License
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.