Support declarative HTTP clients

Question

Support declarative HTTP clients

mhalbritter opened this issue 3 years ago · 29 comments

Spring Framework 6.0 introduces declarative HTTP clients. We should add some auto-configuration in Boot which supplies for example the HttpServiceProxyFactory.

We could even think about an annotation with which the HTTP interface can be annotated and then directly injected into a consumer (like @FeignClient).

Answer 1 · 2022-06-11T06:02:50.000Z

I think this one can try to inject IOC by myself
For example I do

public class HttpServiceFactory implements BeanFactoryAware, ImportBeanDefinitionRegistrar, ResourceLoaderAware {

	private final HttpServiceProxyFactory proxyFactory;

	private BeanFactory beanFactory;

	private ResourceLoader resourceLoader;

	public HttpServiceFactory() {
		WebClient client = WebClient.builder().build();
		this.proxyFactory = HttpServiceProxyFactory.builder(new WebClientAdapter(client)).build();
	}

	@Override
	public void registerBeanDefinitions(@NonNull AnnotationMetadata importingClassMetadata,
			@NonNull BeanDefinitionRegistry registry) {
		List<String> packages = AutoConfigurationPackages.get(this.beanFactory);
		Set<Class<?>> typesAnnotatedClass = findByAnnotationType(HttpExchange.class, resourceLoader,
				packages.toArray(String[]::new));
		for (Class<?> exchangeClass : typesAnnotatedClass) {
			BeanName name = AnnotationUtils.getAnnotation(exchangeClass, BeanName.class);
			String beanName = name != null ? name.value()
					: CaseFormat.UPPER_CAMEL.to(CaseFormat.LOWER_CAMEL, exchangeClass.getSimpleName());
			registry.registerBeanDefinition(beanName, getBeanDefinition(exchangeClass));
		}
	}

	private <T> BeanDefinition getBeanDefinition(Class<T> exchangeClass) {
		return new RootBeanDefinition(exchangeClass, () -> proxyFactory.createClient(exchangeClass));
	}

	@Override
	public void setResourceLoader(@NonNull ResourceLoader resourceLoader) {
		this.resourceLoader = resourceLoader;
	}

	@Override
	public void setBeanFactory(@NonNull BeanFactory beanFactory) throws BeansException {
		this.beanFactory = beanFactory;
	}

public Set<Class<?>> findByAnnotationType(Class<? extends Annotation> annotationClass,
			ResourceLoader resourceLoader, String... packages) {
		Assert.notNull(annotationClass, "annotation not null");
		Set<Class<?>> classSet = new HashSet<>();
		if (packages == null || packages.length == 0) {
			return classSet;
		}
		ResourcePatternResolver resolver = ResourcePatternUtils.getResourcePatternResolver(resourceLoader);
		CachingMetadataReaderFactory metadataReaderFactory = new CachingMetadataReaderFactory(resourceLoader);
		try {
			for (String packageStr : packages) {
				packageStr = packageStr.replace(".", "/");
				Resource[] resources = resolver.getResources("classpath*:" + packageStr + "/**/*.class");
				for (Resource resource : resources) {
					MetadataReader metadataReader = metadataReaderFactory.getMetadataReader(resource);
					String className = metadataReader.getClassMetadata().getClassName();
					Class<?> clazz = Class.forName(className);
					if (AnnotationUtils.findAnnotation(clazz, annotationClass) != null) {
						classSet.add(clazz);
					}
				}
			}
		}
		catch (IOException | ClassNotFoundException e) {
			throw new RuntimeException(e);
		}
		return classSet;
	}

}

/**
 * Just used to set the BeanName
 */
@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
public @interface BeanName {

	String value();

}

What is your opinion on this

Answer 2 · 2022-06-13T14:49:14.000Z

What would an additional annotation bring beyond @HttpMapping?

Answer 3 · 2022-06-15T09:00:45.000Z

The underlying client may need to be configured with different base URL, codecs, etc. That means detecting @HttpExchange annotated interfaces and declaring beans for them could become inflexible if it's based on the same underlying client setup.

Given an HttpServiceProxyFactory, it's trivial to create a proxy, so don't see a lot of gain from automating that. What I am thinking about though is that it should be possible for Boot to create and auto-configure one HttpServiceProxyFactory instance that can then be combined that with different client setups.

Currently HttpServiceProxyFactory takes HttpClientAdapter in the constructor, but we could make a change to allow it to be passed to an overloaded createClient(Class<?>, HttpClientAdapter) method. So you could inject the same HttpServiceProxyFactory anywhere, and either use it with the default client setup (based on WebClientCustomizer and WebClient.Builder), or optionally, also inject WebClient.Builder and use a client setup that deviates from the default.

Answer 4 · 2022-06-21T08:31:26.000Z

I played around with it here.

The auto-configuration supplies a bean of type HttpServiceProxyFactory, which a user can then use to create the proxy from the interface. The base url for the client is settable via @HttpExchange and is not configured on the factory.

Answer 5 · 2022-06-21T14:14:14.000Z

Thanks, this helps me to move my thought process forward. I see now it is necessary to separate more formally the HttpServiceProxyFactory from the underlying client.

I've an experiment locally where HttpServiceProxyFactory expects the HttpClientAdapter to be passed in every time createClient is called. Separately, there is a WebClientServiceProxyFactory that is created with an HttpServiceProxyFactory and a WebClient and exposes a createClient with just the proxy interface.

The Boot auto-config could then declare a single HttpServiceProxyFactory bean, and applications would create any number of WebClientServiceProxyFactory beans, each delegating to the same HttpServiceProxyFactory and the WebClient configured for a specific remote.

Answer 6 · 2022-06-22T08:35:17.000Z

After a few different experiments, I think trying to have one HttpServiceProxyFactory for many client instances brings additional complexity with little gain. The easiest to understand model remains, one HttpServiceProxyFactory for one client. It's reasonably simple even without any help from Boot:

@Bean
HttpServiceProxyFactory httpServiceProxyFactory1(WebClient.Builder clientBuilder) {
	WebClient client = clientBuilder.baseUrl("http://host1.com").build();
	return new HttpServiceProxyFactory(new WebClientAdapter(client));
}

@Bean
HttpServiceProxyFactory httpServiceProxyFactory2(WebClient.Builder clientBuilder) {
	WebClient client = clientBuilder.baseUrl("http://host2.com").build();
	return new HttpServiceProxyFactory(new WebClientAdapter(client));
}

A couple of extra shortcuts on WebClientAdapter could make this a one-liner:

@Bean
HttpServiceProxyFactory httpServiceProxyFactory1(WebClient.Builder clientBuilder) {
	return WebClientAdapter.createProxyFactory(clientBuilder.baseUrl("http://host1.com"));
}

@Bean
HttpServiceProxyFactory httpServiceProxyFactory2(WebClient.Builder clientBuilder) {
	return WebClientAdapter.createProxyFactory(clientBuilder.baseUrl("http://host2.com"));
}

If we settle on the above as the expected configuration, then I think it's not essential to have any Boot auto-config to start, although some ideas may still come along. Perhaps, specifying baseUrl's in properties, which would allow Boot to create the above beans?

Answer 7 · 2022-07-25T15:59:10.000Z

Thanks, Rossen.

Perhaps, specifying baseUrl's in properties, which would allow Boot to create the above beans?

We don't yet support specifying multiple property values to auto-configure multiple beans anywhere in Boot. It's something that we'd like to do, but it's a complex and wide-ranging topic. #15732 is tracking auto-configured multiple DataSources, for example.

Having discussed this today, we don't think there's anything to do in Boot at this time. We can revisit this if the picture changes for auto-configuring multiple beans.

Answer 8 · 2022-10-10T18:01:24.000Z

Note that there is now spring-projects/spring-framework#29296, which will likely give us a better model for dealing with multiple HttpServiceProxyFactory instances for different remotes.

Answer 9 · 2022-10-17T13:35:40.000Z

Reopening because of changes made in spring-projects/spring-framework#29296
Spring Boot could contribute a pre-configured HttpServiceProxyFactory.Builder to the context so developer can build their own client from it.

Answer 10 · 2022-11-13T04:49:16.000Z

With regard to this nice tutorial on HTTP Interface https://softice.dev/posts/introduction_to_spring_framework_6_http_interfaces/,

I don't quite understand. Why do developer need to manually write a @Bean method that will return the proxy bean (which implement the interface) especially if we are using spring boot?
I recall using @FeignClient, I do not have to define any proxy bean for it, so I presume spring boot will do for us.

Also why would one use Http Interface over @FeignClient?

Answer 11 · 2023-01-13T05:34:56.000Z

We could even think about an annotation with which the HTTP interface can be annotated and then directly injected into a consumer (like @FeignClient).

I think we need an annotation like @EnableFeignClients rather than @FeignClient.

We can already know whether an interface is a http client through @HttpExchange, we need an annotation to scan the interfaces and register beans (like @EnableFeignClients).

Here‘s my workaround.

Answer 12 · 2023-02-02T21:14:37.000Z

I think people have gotten used to using the Feign client approach.

Here you can find very similar aproach: Exchange client

It is really simple to use.

Answer 13 · 2023-04-04T07:07:02.000Z

I have prototyped following approach, that reduces the boilerplate to minimum:

`@HttpClient` annotation to mark interface as an http client and add option to set the `WebClient` bean name to use.

@HttpClient("todo-client")
public interface TodoClient {
    @GetExchange("/todos")
    List<Todo> get();
}

This annotation is processed by an registrar implementing ImportBeanDefinitionRegistrar that registers bean definition for each http client with HttpServiceProxyFactory and WebClientAdapter creating an adapter for a WebClient with a name from the annotation.

Creating `WebClient` instances from the environment

Considering that many web clients are relatively simple, there is a common set of properties that can be set with simple properties: url, basic auth, timeouts etc.

Given this, there is an option to create WebClients through yaml/properties like this:

http.clients:
    todo-client:
        url: https://jsonplaceholder.typicode.com
    bar:
        url: http://foo/bar

If you believe it makes sense I can prepare a PR or if it's too early to say I can release it as a separate project that will get deprecated once Spring Boot has similar functionality built in.

Update:

The library is available on Maven Central: https://github.com/maciejwalkowiak/spring-boot-http-clients

Answer 14 · 2023-12-08T11:47:32.000Z

Another proposed implementation: https://github.com/joshlong/declarative-client-registration by @joshlong .

Answer 15 · 2023-12-13T14:02:43.000Z

Let me introduce once again to httpexchange-spring-boot-starter, which is probably the most comprehensive implementation I could find.

This project is entirely configuration-driven and can achieve the same functionalities as Spring Cloud OpenFeign without introducing any external annotations. Including setting different baseUrl/timeout for each client, integration with Spring Cloud LoadBalancer, dynamic refresh, and more.

http-exchange:
  base-packages: [com.example]
  connect-timeout: 1000
  read-timeout: 3000
  client-type: rest_client
  channels:
    - base-url: http://order
      clients:
        - com.example.order.api.*
    - base-url: http://user
      read-timeout: 5000
      clients:
        - com.example.user.api.*

The main goals of this project:

Promote the use of @HttpExchange as a neutral annotation to define API interfaces.
Provide a Spring Cloud OpenFeign like experience for Spring 6.x declarative HTTP clients.
Support @RequestMapping based annotations (easy to migrate from Spring Cloud OpenFeign).
Not introduce external annotations, easy to migrate to other implementations.

It's definitely worth a try!

Answer 16 · 2023-12-25T18:31:28.000Z

Let me introduce once again to httpexchange-spring-boot-starter, which is probably the most comprehensive implementation I could find.

This project is entirely configuration-driven and can achieve the same functionalities as Spring Cloud OpenFeign without introducing any external annotations. Including setting different baseUrl/timeout for each client, integration with Spring Cloud LoadBalancer, dynamic refresh, and more.
http-exchange:
  base-packages: [com.example]
  connect-timeout: 1000
  read-timeout: 3000
  client-type: rest_client
  channels:
    - base-url: http://order
      clients:
        - com.example.order.api.*
    - base-url: http://user
      read-timeout: 5000
      clients:
        - com.example.user.api.*
The main goals of this project:

Promote the use of @HttpExchange as a neutral annotation to define API interfaces.

Provide a Spring Cloud OpenFeign like experience for Spring 6.x declarative HTTP clients.

Support @RequestMapping based annotations (easy to migrate from Spring Cloud OpenFeign).

Not introduce external annotations, easy to migrate to other implementations.

It's definitely worth a try!

Great Effort!! Wish the spring boot will include your starter as an official one!

Answer 17 · 2023-12-25T18:36:39.000Z

So is there any plan or any work in progress to include this feature natively?

Answer 18 · 2024-01-08T14:33:40.000Z

We're discussing it within the team now. We will update here once decision's been taken.

Answer 19 · 2024-02-09T13:58:23.000Z

@OlgaMaciaszek are there any updates on this topic?

Answer 20 · 2024-02-09T14:04:21.000Z

Internal POC planned for this month to discussed with the team. Will post any updates here.

Answer 21 · 2024-05-03T12:46:46.000Z

@OlgaMaciaszek Since spring-cloud-openfeign is going to enter maintenance mode, will this become the alternative options to openfeign, any update for now?

Answer 22 · 2024-05-04T11:02:54.000Z

Hey Folks, any updates here. when the declrative approach will be available natively?

Answer 23 · 2024-05-06T10:20:26.000Z

Working on POC at this point. Once that's done we'll able to hold a discussion on adding it to a backlog of a specific release. Will post any relevant updates here.

Answer 24 · 2024-05-24T02:04:56.000Z

@OlgaMaciaszek Thank you for introduce me to this post. httpexchange-spring-boot-starter is really helpful, but may be it still have a small difference from what I want. That's we always configure the base url in database, not in property files, and we need to obtain the base url during every call to the HttpExchange interface to ensure that we use the proper value for each customer (Yes, different customer may use different base url, and the remote site is not constucted by ourself)

Answer 25 · 2024-05-24T02:34:39.000Z

不同的客户可能使用不同的基 URL

Maybe you can do it with an org.springframework.http.client.ClientHttpRequestInterceptor

Answer 26 · 2024-05-24T02:49:47.000Z

Always get your configuration from the spring environment and use different configuration sources to load from a property or DB or whatever

Answer 27 · 2024-05-24T05:08:10.000Z

不同的客户可能使用不同的基 URL

Maybe you can do it with an org.springframework.http.client.ClientHttpRequestInterceptor
In my experience, this interceptor can only modify header values, not url.

Answer 28 · 2024-05-24T05:14:22.000Z

Always get your configuration from the spring environment and use different configuration sources to load from a property or DB or whatever

Thank you for your advice, what you mean is to configurer Proxy instance per customer for specified HttpExchange at startup, like Bean instance A for customer A, Bean instance B for customer B, and so on. Is this understanding correct?

Answer 29 · 2024-05-24T06:08:16.000Z

In my experience, this interceptor can only modify header values, not url.

You can replace the request, just like the example with kotlin:

val urlPrefixedInterceptor = ClientHttpRequestInterceptor { request, body, execution ->
    execution.execute(object : HttpRequest by request {
        override fun getURI(): URI {
            return URI.create("https://example.org${request.uri}")
        }
    }, body)
}

@HttpClient annotation to mark interface as an http client and add option to set the WebClient bean name to use.

Creating WebClient instances from the environment

`@HttpClient` annotation to mark interface as an http client and add option to set the `WebClient` bean name to use.

Creating `WebClient` instances from the environment