Spring PropertyPlaceholderConfigurer to read from AWS Parameter Store

Recently we have transformed our AWS based servers to AWS Fargate. One of the challenges we have faced is how we can provide properties in a secure way to our serverless docker instances. Our Spring/Java based servers use @Value injection for many of our properties including sensitive data like database passwords. Until now we have used Spring config property files. There are encryption solutions like Jasypt that work well if we have access to a volume on our EC2 instance. However, when using fargate we are serverless with no such access. 

The first choice as a team that is heavily invested in spring would be dockerized instances of Spring Cloud Config but first we wanted to see what AWS service we could use to simplify things. We found the AWS Parameter Store. As the documentation indicates, AWS parameter store provides "secure, hierarchical storage for configuration data management and secrets management". We have the ability to control access to the keys using AWS IAM Roles. We also have simple key management of the encryption using AWS KMS as well as a built in audit log of changes. The only challenge left for us was to integrate the reading of the Parameter store into our application as an out of the box replacement for the PropertyPlaceholderConfigurer we have used to inject environment based properties until now. I will outline below the steps we did to do this. 

The first part of the project was our SSMClient class to wrap all calls to the Parameter Store.  As you see in the comments, this class wraps the AWS API to retrieve parameters from the parameter store, including retrieving all the environment parameters and stripping off the prefix. As you see in the code we cannot retrieve all the parameters in one shot and need to loop until we retrieve them all using the token returned on each call. AWS will only return a maximum of 10 elements in each call.

The next step was to use this class and implement our own PropertyPlaceholderConfigurer. Here is the code we used for this. As you see in the code, at the time this code is called we do not have a fully autowired SSMClient to work with so we need to do this manually. So we create our SSMClient and inject the dependencies we need, manually call our PostConstruct function. After this we have our properties to pass to the base class and make them ready to inject in all the @Value parameters we have. To use the bean we just add it to our @Configuration or to the Application Context xml as you need.

Why is Spring @Value not working to read my parameter and default value?

In my small Spring project, i have a value that i may need to change of on of my parameters.  The simple solution would be

@Value("${myParam:defaultVal")

private String myParam;

Yet it was not working and the value was coming out as "$myparam:defaultVal".

What's the issue?
Eventually i stumbled upon the issue via this spring issue. I also saw it addressed here although there is a mistake as it is not sufficient to use util:properties as explained here. So, to make this work if you are using XML just add this in:

<context:property-placeholder location="classpath:/myfile.properties"/>

Now hopefully this wont happen again to me...

Using Gradle for a mixed groovy and java project and creating a distribution zip

Yes, I still use Groovy upon occasion. Groovy still is more readable and elegant in a lot of instances even than Java 8.

Ok but let's say i am writing both java and groovy in the project
In order to have this work, you need to configure your Gradle project with the following small changes so that you can make it compile the mixed code.

You need to make all the code compile as Groovy code (even the Java code).
So, add the following lines to your project:

sourceSets.main.java.srcDirs = []
sourceSets.main.groovy.srcDirs += ["src/main/java"]

This makes all your java code compile with the Groovy compiler to prevent dependency issues between the Groovy and Java code.

Now, your code all works but you want to put the zip of it somewhere to use. So, just add these lines:

task zip(dependsOn: jar, type: Zip) {
    from { configurations.runtime.allArtifacts.files } {
        into("${project.name}-${project.version}")
    }
    from { configurations.runtime } {
        into("${project.name}-${project.version}/lib")
    }

}

Now when you run gradle zip you will get a zip with all dependencies inside! 

Why am I getting "WARNING arguments left: 1"

If you are new to Akka Logging and are used to using Log4j, there is a good chance you will get the error "WARNING arguments left: 1", and not know where its coming from.

So, here it is:

In log4j, you maybe used to writing:

logger.warn("Exception caught: ", e);

This is perfectly valid and works fine. If you use this line in akka logging, you will get the error "WARNING arguments left: 1" because in the case of logback, the syntax assumes it is filling in params from the string, so it expects this:

logger.warning("Exception caught: {}", e);

Hope this will help all of us (including me) not to get caught by this again....

Spring's StringUtils -- Stuff we do with Strings all the time

The other day i was looking for a simple way to take some Strings and automatically capitalize the first character. I stumbled upon Springs StringUtils and as it turns out it has a lot of nice utility functions.

Check them out: http://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/util/StringUtils.html

There are many nice simple things that we need to do all the time:

• capitalize (first characters)
• uncapitalize
• Lots of utilities for String arrays -- adding to them, sorting etc

Here is a really nice one: splitArrayElementsIntoProperties

Check them out!

Taking mocking seriously

When I first got involved in using mocks, I viewed them as a necessary evil since I had some dependencies in my class and if don’t use a mock, I will suffer NPE’s in my tests.
However, as I started using them and learning their power, I have seen that I can really use mocking in ways that enhance my unit tests. Here is a concrete example. I am using Spring Data JPA. So, I need to mock the save of my entity when it gets created in the function that I am unit testing. However, once I am mocking I can easily also validate the parameters sent to this function that did the save. Using Mockito, we use the ArgumentCaptor, to do this validation as follows:

MyRepository myRepository = mock(MyRepository.class);
ArgumentCaptor<EntityClass> argument = ArgumentCaptor.forClass(EntityClass.class);
//... invoke function to unit Test that will internally save Entity
verify(myRepository).saveAndFlush(argument.capture());
EntityClass entity = argument.getValue();
assert entity.getName().equals(“name”);
assert entity.getAddress().equals("address");

This now means my unit test has validated that all parameters passed to the function were passed to the database.
Now we see some real added value in the mock, where we can be confident that our copy of the data to the entity we are persisting has not missed any of the parameters.
Another nice feature in mocking is checking how many times a function is invoked. Modifying the the code above:

MyRepository myRepository = mock(MyRepository.class);
ArgumentCaptor<EntityClass> argument = ArgumentCaptor.forClass(EntityClass.class);
//... invoke function to unit Test that will internally save Entity
verify(myRepository,times(1)).saveAndFlush(argument.capture());
EntityClass entity = argument.getValue();
assert entity.getName().equals(“name”);
assert entity.getAddress().equals("address");

Happy Mocking!

Using Date and Calendar – At your own risk!

You can find lots of issues discussed on the ‘net about the problems of the Calendar/Date classes in Java. Here are a few that recently reared their ugly head in our code:

1. This one is actually not in either of these but in trying to output our Date. Beware of the mm instead of MM when using SimpleDateFormat. Yep, one of my team members used the following code:

SimpleDateFormat sdf = new SimpleDateFormat(“yyyymmdd”);

Surprisingly everyone in our tests had a birthday of the first of the month! I think that the right solution would be that mm is wrong and mi is minutes and mo is month. This would solve the confusion.

2. Did you know that there is a difference whether you use Calendar.HOUR or Calendar.HOUR_OF_DAY? Ok, i admit it this one is documented in the javdocs, but i would still think that it is a bit of surprise that in the afternoon, if you do

cal.set(Calendar.HOUR,0);

that you will get 12:00 PM!

3. Why does toString() of Calendar not put out the date in some format?

4. Beware that DAY_OF_WEEK is not zero based while MONTH is.