YamlProcessor outside spring-boot

[wind57]

It all starts with an issue we have recently received in spring-cloud-kubernetes. I will try to simplify it as much as I can.

If I have an application.yaml like this:

bean:
  items:
    - Item 1
    - Item 2
    - Item 3
    - Item 4
    - Item 5

---
spring:
  config:
    activate:
      on-profile: dev

bean:
  items:
    - Item 1
    - Item 6
    - Item 7

and activate the dev profile, bean.items would be injected as [Item 1, Item 6, Item 7] and that is of course correct.

We need to "replicate" the same thing in spring-cloud-kubernetes. There, we read a configmap for example and based on what's inside it, we would create the PropertySource by hand.

The problem is, until now, we were using YamlPropertiesFactoryBean, and this one would "override" properties for lists, because of this code:

    protected Properties createProperties() {
        Properties result = CollectionFactory.createStringAdaptingProperties();
        this.process((properties, map) -> result.putAll(properties));
        return result;
    }

As such, we would end up with:

bean.items = [Item 1, Item 6, Item 7, Item 4, Item 5]

This happens because YamlPropertiesFactoryBean does not know to override properly arrays and maps. I ended up creating our own class for this parsing, that is heavily based on YamlPropertiesFactoryBean.

Ryan Baxter advised me to open an issue here to see if the real solution is in spring-boot and not us creating a custom parsing class.

Thank you.

[philwebb]

You probably want to look at org.springframework.boot.env.YamlPropertySourceLoader which is how our yaml files are loaded. You really need to add each document from a multi-document YAML file to its own property source. That way the Binder code will be able to deal with list elements correctly.

In your example above, our code would add PropertySource #1 containing:

bean.items[0]=Item 1
bean.items[1]=Item 2
bean.items[2]=Item 3
bean.items[3]=Item 4
bean.items[4]=Item 5

and PropertySource #2 containing:

spring.config.activate.on-profile=dev
bean.items[0]=Item 1
bean.items[1]=Item 6
bean.items[2]=Item 7

You might also want to look at the existing config tree support which is designed to work with Kubernetes config maps.

[wind57]

thank you for your answer @philwebb. I know about config tree support, we already support it in spring-cloud-kubernetes.

You really need to add each document from a multi-document YAML file to its own property source

I know as I have seen and debugged the code and I 100% agree with you. That is not an option for us atm, since the code is very involved and we can't do that. It would require a major revision on how we handle and create property sources; and tbh, it might not be worth it at all. In the last 7 years or so, this is the first time we face an issue in this regard...

My point was more on the side to externalize the process somehow. We still have a work-around, so we might be good for the time being. Thank you for your time.