Profiles not available in ConfigurableEnvironment on ApplicationEnvironmentPreparedEvent after upgrading to 2.5.4

[fprochazka]

Hi,
I'm upgrading from Spring Boot 2.3.9 to 2.5.2 2.5.4

Previously, I've had a listener that I was using to detect if a specific profile is active (production) and if not I would disable AWS SpringCloud ParamStore loading - to prevent developers on localhost to accidentally load the production properties. But that's not the point, that part is working as expected.

SpringCloudBootstrapOverridesListener

import io.awspring.cloud.autoconfigure.context.properties.AwsCredentialsProperties;
import io.awspring.cloud.paramstore.AwsParamStoreProperties;

public class SpringCloudBootstrapOverridesListener implements ApplicationListener<ApplicationEnvironmentPreparedEvent>, Ordered
{

    public static final String BOOTSTRAP_OVERRIDES_PROPERTY_SOURCE_NAME = "cogvio-cloud-bootstrap-overrides";

    public static final int LISTENER_ORDER = BootstrapApplicationListener.DEFAULT_ORDER - 1; // before

    public static final ImmutableMap<String, Object> PROPERTIES_LOCAL_OR_TEST = ImmutableMap.<String, Object>builder()
        .put(AwsParamStoreProperties.CONFIG_PREFIX + ".enabled", "false")
        .put(AwsCredentialsProperties.PREFIX + ".instance-profile", "false")
        .build();

    @Override
    public void onApplicationEvent(final ApplicationEnvironmentPreparedEvent event)
    {
        ConfigurableEnvironment environment = event.getEnvironment();
        if (environment.getPropertySources().contains(BOOTSTRAP_OVERRIDES_PROPERTY_SOURCE_NAME)) {
            return; // already registered
        }

        if (!Profiles.isLive(environment)) {
            environment.getPropertySources().addFirst(
                new MapPropertySource(BOOTSTRAP_OVERRIDES_PROPERTY_SOURCE_NAME, PROPERTIES_LOCAL_OR_TEST)
            );
        }
    }

    @Override
    public int getOrder()
    {
        return LISTENER_ORDER;
    }

}

public final class Profiles
{

    public static final String CLI = "cli";
    public static final String WEB = "web";
    public static final String QA = "qa";
    public static final String TEST = "test";
    public static final String LOCAL = "local";
    public static final String STAGING = "staging";
    public static final String PRODUCTION = "production";

    public static boolean isLive(final Environment environment)
    {
        String[] activeProfiles = environment.getActiveProfiles();
        for (String profile : activeProfiles) {
            if (isLive(profile)) {
                return true;
            }
        }
        return false;
    }

    public static boolean isLive(final String profileName)
    {
        return STAGING.equalsIgnoreCase(profileName)
            || PRODUCTION.equalsIgnoreCase(profileName);
    }
}

The problem is that in this listener, I access spring profiles the same way I did it on the older version. For some reason, the ConfigurableEnvironment does not contain them.

Debugging info on 2.5.2

java ... -Dspring.profiles.active=production ... WebApplication

But the SpringApplication does see at least the one I've configured programmatically

public class WebApplication
{

    public static void main(final String[] args)
    {
        new SpringApplicationBuilder(WebApplication.class)
            .registerShutdownHook(true)
            .bannerMode(Banner.Mode.OFF)
            .web(WebApplicationType.SERVLET)
            .profiles("web") // additional profiles
            .run(args);
    }

}

If I revert to the older version, I can clearly see the profiles in ConfigurableEnvironment are already accessible

Debugging info on 2.3.9

I understand, that the actual processing and properties activation happens much later in the lifecycle, but IMHO there is no reason to postpone actually loading the profiles into the ENV and making them available for listeners to read.

This does seem like an unwanted BC Break.

[fprochazka]

I've tried 2.5.4 and the behaviour is the same as 2.5.2

[wilkinsona]

Thanks for the report. Unfortunately, I don't think you've provided enough information to diagnose the problem. It looks like you're using Spring Cloud but you haven't described exactly how you're using it. Crucially, it's not clear if you're still using the Bootstrap context when using Spring Boot 2.5.x.

If you would like us to spend some more time investigating, please spend some time providing a complete yet minimal sample that reproduces the problem. You can share it with us by pushing it to a separate repository on GitHub or by zipping it up and attaching it to this issue.

[fprochazka]

My listener is called before Spring Cloud context takeover and it is meant to modify it's behaviour, so Spring Cloud itself is not the issue here. I was only mentioning it to demonstrate my use-case.

Reproducer: https://github.com/fprochazka/spring-boot-reproducer-27946-profiles-in-listener

1. checkout fprochazka/spring-boot-reproducer-27946-profiles-in-listener@3541ba0
2. build the app and run it with -Dspring.profiles.active=production
3. checkout fprochazka/spring-boot-reproducer-27946-profiles-in-listener@9de7b06 (see that I've only changed the version)
4. build the app and run it with -Dspring.profiles.active=production

before:

after:

[fprochazka]

I believe what changed is that profiles handling happened very early in the application lifecycle in the older version, but in the newer version, it happens in one of the listeners - EnvironmentPostProcessorApplicationListener, which happens to be ordered after Spring Cloud bootstrap listener (and therefore also after my listener, since I need to execute before Spring Cloud bootstrap).

before:

after:

[wilkinsona]

Thanks for the sample. I've reproduced the behaviour that you have described. The change in behaviour occurred in 2.4 as a result of this commit.

What's the intent of your listener? Even in Spring Boot 2.3 it may not have a complete picture of the active profiles. For example, if I add spring.profiles.include=included to application.properties and start the application with spring.profiles.active=production, included, web, and, production will all be active but your listener will miss out included:

I expect here [web, production] and was given [web, production]
11:15:37.704 [main           ] INFO  tReproducer27946ProfilesInListenerApplication:	Starting SpringBootReproducer27946ProfilesInListenerApplication v0.0.1-SNAPSHOT on wilkinsona-a01.vmware.com with PID 11795 (/Users/awilkinson/dev/temp/spring-boot-reproducer-27946-profiles-in-listener/target/fprochazka-spring-boot-reproducer-0.0.1-SNAPSHOT.jar started by awilkinson in /Users/awilkinson/dev/temp/spring-boot-reproducer-27946-profiles-in-listener) 
11:15:37.707 [main           ] DEBUG tReproducer27946ProfilesInListenerApplication:	Running with Spring Boot v2.3.10.RELEASE, Spring v5.2.14.RELEASE 
11:15:37.707 [main           ] INFO  tReproducer27946ProfilesInListenerApplication:	The following profiles are active: included,web,production 

[fprochazka]

I'm trying to make sure Spring Cloud's spring-cloud-starter-aws-parameter-store-config is activated only when a specific profile (production) is passed to the application.

I'm not using the properties to add more profiles, I'm adding it using the -D jvm param and SpringApplicationBuilder.profiles(). I understand that the active profiles are incomplete in this phase, but that's not a problem for me.

I understand it might be a problem to have the active profiles list keep changing in the middle of the startup, but having at least some "partial list of profiles spring boot knows of so far" would be a great help. I was trying to find some method/class I'd be able to use to get the partial list of profiles from spring, but everything is pretty much cemented in internals and/or has a million dependencies I don't wanna have to initialize myself.

Right now I'm using this workaround:

    @Override
    public void onApplicationEvent(final ApplicationEnvironmentPreparedEvent event)
    {
        if (!isLive(environment, event.getSpringApplication())) {
            environment.getPropertySources().addFirst(
                new MapPropertySource(BOOTSTRAP_OVERRIDES_PROPERTY_SOURCE_NAME, PROPERTIES_LOCAL_OR_TEST)
            );
        }
    }

    @Override
    public int getOrder()
    {
        return LISTENER_ORDER;
    }

    /**
     * Spring changed behaviour and sadly is loading profiles into the ENV too late in the application lifecycle.
     */
    private static boolean isLive(final Environment environment, final SpringApplication springApplication)
    {
        for (String profile : getActiveProfiles(environment, springApplication)) {
            if (Profiles.isLive(profile)) {
                return true;
            }
        }
        return false;
    }

    private static String[] getActiveProfiles(final Environment environment, final SpringApplication springApplication)
    {
        Set<String> activeProfiles = new HashSet<>();
        activeProfiles.addAll(Set.of(environment.getActiveProfiles()));
        activeProfiles.addAll(readProfilesListFromProperty(environment, AbstractEnvironment.ACTIVE_PROFILES_PROPERTY_NAME));
        activeProfiles.addAll(readProfilesListFromProperty(environment, org.springframework.boot.context.config.Profiles.INCLUDE_PROFILES_PROPERTY_NAME));
        activeProfiles.addAll(springApplication.getAdditionalProfiles());

        return activeProfiles.toArray(String[]::new);
    }

    private static Set<String> readProfilesListFromProperty(final Environment environment, final String propertyName)
    {
        return Optional.ofNullable(environment.getProperty(propertyName))
            .filter(Predicate.not(String::isBlank))
            .map(value -> StringUtils.commaDelimitedListToSet(StringUtils.trimAllWhitespace(value)))
            .orElseGet(Collections::emptySet);
    }

and I'd really like to be able to depend on Spring to handle this for me, instead of having to cherry-pick this from Spring's internals

[wilkinsona]

Given that goal, I think you may be better served by a change in approach when using Spring Boot 2.4 and later. In place of your custom listener, you could use spring.profile.activate.on-profile to set the enabled properties to false whenever the production profile is not active. Something like this:

spring.config.activate.on-profile: !production
example.enabled=false

!production is a profile expression that means that the config document will only be activated if the production profile is not active.

Does that work for you?

[fprochazka]

@wilkinsona no, it won't work, because Spring Cloud's bootstrap listener is invoked before EnvironmentPostProcessorApplicationListener, therefore it's not affected by this

[wilkinsona]

If you're using Spring Boot 2.4 and later, I don't think Spring Cloud's bootstrap listener needs to be involved as the bootstrap context is no longer required. If you need or want to use the bootstrap context then it should be possible add a document to bootstrap.yml that's activated whenever the production profile is not active.

I think we're back at the point where a sample is needed that reproduces the problem. The current one now appears to simplify things too much as it doesn't involve Spring Cloud or the bootstrap context.

[fprochazka]

I think we've derived from the original point. And that point is what previously worked now doesn't. I'm grateful for your suggestions, but I'm reasonably capable of working around the problem - I'm planning on using a completely different approach to fix this, but that's not relevant to this issue.

I've created this issue to report a BC Break that I did not find documented anywhere. Profiles were realiably accessible through Environment for all the ApplicationListener<ApplicationEnvironmentPreparedEvent> listeners and now they're not.

If that's the desired behaviour, I'm fine with that and I'd suggest documenting this BC break and closing this issue.

[wilkinsona]

Fair enough. FWIW, I'm not sure that we'd consider this a breaking change as it was only partially working before as illustrated with any profile activated using spring.profile.include being missed. I'll flag the issue so that the team can discuss it and decide what, if anything, we want to do.

[wilkinsona]

We've discussed this today and have decided to leave things as-is.