We are a Melbourne-based nearshore software outsourcing company providing world-class dedicated tech talent and development expertise to companies listed in the Australian Share Market to innovative startups who are shaping the future.

Contacts Us

Melbourne
Suite 37/11, Wilson St,
South Yarra,
VIC 3141.

Colombo
19, Katukurunduwatta Road, Ratmalana,

info@unibench.com.au

Melbourne
1300 320 487

Colombo
+94 769 360 433

Uncategorized

Embracing the Cloud Locally: Elevating Java Spring Projects through LocalStack for AWS Services Testing

0_LoP_oGRF7rDzd2JZ Embracing the Cloud Locally: Elevating Java Spring Projects through LocalStack for AWS Services Testing

As Java Spring developers, testing interactions with AWS services, especially within the context of AWS Lambda function handlers, can be a challenging aspect of our projects. LocalStack comes to the rescue by providing a powerful solution for testing AWS cloud services locally, eliminating the need for a live AWS environment. In this comprehensive guide, we’ll delve into the intricacies of setting up and effectively using LocalStack for testing AWS Lambda function handlers with SQS events in a Java Spring project.

Understanding LocalStack

What is LocalStack?
https://www.localstack.cloud/
LocalStack stands out as a lightweight, self-contained AWS cloud stack designed specifically for local development and testing. It presents a fully functional local environment that mirrors the behavior of AWS cloud services. This makes LocalStack an ideal choice for developers seeking to validate their applications’ interactions with AWS services without the overhead of deploying to an actual AWS environment.

Prerequisite: Docker Installation for Linux Ubuntu

Before setting up LocalStack, ensure that Docker is installed on your Linux Ubuntu machine. Follow these steps to install Docker:

Update Package Lists: Open a terminal and update the package lists:

sudo apt update

Install Docker Dependencies: Install packages to allow apt to use a repository over HTTPS:

sudo apt install apt-transport-https ca-certificates curl software-properties-common

Add Docker’s Official GPG Key: Add Docker’s official GPG key to ensure the integrity of the packages:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

Set Up the Stable Docker Repository: Set up the stable Docker repository:

echo "deb [signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/nul

Install Docker Engine: Update the package lists once more, then install the Docker engine:

sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io

Verify Docker Installation: Verify that Docker is installed correctly by running:

sudo docker --version

You should see information about the Docker version.

Now that Docker is installed, you’re ready to proceed with LocalStack setup.

How LocalStack Works Under the Hood

LocalStack works by emulating AWS cloud services locally using Docker containers. Here’s an overview of how it operates under the hood:

Docker Containers:

  • LocalStack uses Docker containers to encapsulate and emulate various AWS services locally.
  • Each AWS service supported by LocalStack is typically represented by a separate Docker container.

Service Endpoints:

  • LocalStack exposes service endpoints for each emulated AWS service. These endpoints mimic the behavior of the corresponding AWS services.
  • For example, there are endpoints for local DynamoDB, SQS, S3, etc.

Java SDK Compatibility:

  • LocalStack is compatible with the AWS SDK for Java, allowing Java applications to interact with emulated AWS services using the standard Java SDK.

Service Initialization:

  • When LocalStack starts, it initializes the Docker containers for the emulated AWS services. This involves setting up configurations and data storage.

AWS Service Emulation:

  • LocalStack intercepts and handles requests made to its service endpoints, emulating the behavior of various AWS services.
  • For instance, when a Java application makes an API call to the local DynamoDB endpoint provided by LocalStack, it receives a response as if it were a real DynamoDB service.

Data Storage:

  • LocalStack creates local storage to mimic the behavior of AWS cloud storage for services like DynamoDB and S3.
  • This allows developers to perform CRUD operations using the AWS SDK for Java.

Configuration and Customization:

  • LocalStack provides configuration options for customizing its behavior, such as specifying which AWS services to emulate and configuring service endpoints.

Integration with Test Frameworks:

  • LocalStack seamlessly integrates with testing frameworks like JUnit, enabling developers to incorporate local AWS service emulation into their test suites.

In summary, LocalStack provides a local environment that closely simulates AWS cloud services through Docker containers, allowing developers to test and validate their applications’ interactions with AWS services without the need for a live AWS environment.

Setting up LocalStack in a Java Spring Project

Adding Dependencies

Before diving into the setup, ensure your project includes the necessary dependencies. Add the following dependencies to your project’s build file, such as pom.xml:

<!-- Dependencies for LocalStack and Testcontainers -->
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>localstack</artifactId>
    <version>LATEST_VERSION</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>junit-jupiter</artifactId>
    <version>LATEST_VERSION</version>
    <scope>test</scope>
</dependency>

Replace LATEST_VERSION with the latest version of Testcontainers.

LocalStackSetup Class

Let’s start with the LocalStackSetup class. This class initializes a LocalStack container with specific AWS services, such as DynamoDB, SQS, and Secrets Manager. Here’s an example:

import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import org.testcontainers.containers.localstack.LocalStackContainer;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.utility.DockerImageName;
import static org.testcontainers.containers.localstack.LocalStackContainer.Service.*;public class LocalstackSetup {
    @Container
    public static LocalStackContainer localStack =
            new LocalStackContainer(DockerImageName.parse("localstack/localstack:latest"))
                    .withServices(DYNAMODB, SQS, SECRETSMANAGER);    static AWSStaticCredentialsProvider localStackCredentialsProvider = new AWSStaticCredentialsProvider
            (new BasicAWSCredentials(localStack.getAccessKey(), localStack.getSecretKey()));
}

ComponentTestConfiguration Class

The ComponentTestConfiguration class serves as a test configuration class that sets up beans for testing components. It starts the LocalStack container in a static block and configures beans for DynamoDB, SQS, Secrets Manager, and more. Below is an illustrative snippet:

package com.base.config;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.secretsmanager.caching.SecretCacheConfiguration;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDB;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClientBuilder;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapperConfig;
import com.amazonaws.services.secretsmanager.AWSSecretsManager;
import com.amazonaws.services.secretsmanager.AWSSecretsManagerClientBuilder;
import com.amazonaws.services.sqs.AmazonSQS;
import com.amazonaws.services.sqs.AmazonSQSClientBuilder;
import com.amazonaws.xray.AWSXRay;
import com.base.client.PalmsClient;
import com.base.component.utils.DynamoDBTestUtils;
import com.base.component.utils.SecretMangerUtils;
import com.base.component.utils.SqsTestUtils;
import org.mockito.Mockito;
import org.springframework.boot.test.context.TestConfiguration;
import org.springframework.context.annotation.Bean;import java.time.Duration;import static com.base.config.LocalstackSetup.localStack;
import static com.base.config.LocalstackSetup.localStackCredentialsProvider;
import static org.testcontainers.containers.localstack.LocalStackContainer.Service.*;@TestConfiguration
public class ComponentTestConfiguration {    static {
        localStack.start();
    }    @Bean
    public DynamoDBTestUtils dynamoDBTestUtils() {
        return new DynamoDBTestUtils();
    }    @Bean
    public DynamoDBMapper dynamoDBMapper(
            final AmazonDynamoDB amazonDynamoDB,
            final DynamoDBMapperConfig dynamoDBMapperConfig
    ) {
        AWSXRay.beginSegment("AmazonDynamoDBv2");
        return new DynamoDBMapper(amazonDynamoDB, dynamoDBMapperConfig);
    }    @Bean
    public AmazonDynamoDB amazonDynamoDB() {
        // Configures Amazon DynamoDB with LocalStack endpoint and credentials.
        // ...
        return AmazonDynamoDBClientBuilder.standard()
                .withEndpointConfiguration(new AwsClientBuilder
                        .EndpointConfiguration(localStack.getEndpointOverride(DYNAMODB).toString(),
                        localStack.getRegion()))
                .withCredentials(localStackCredentialsProvider)
                .build();
    }    @Bean
    public AmazonSQS amazonSQS() {
        // Configures Amazon SQS with LocalStack endpoint and credentials.
        // ...
        return AmazonSQSClientBuilder.standard()
                .withEndpointConfiguration(new AwsClientBuilder
                        .EndpointConfiguration(localStack.getEndpointOverride(SQS).toString(),
                        localStack.getRegion()))
                .withCredentials(localStackCredentialsProvider)
                .build();
    }    @Bean
    public PalmsClient thirdPartyClient() {
        return Mockito.mock(ThirdPartyClient.class);
    }    @Bean
    public AWSSecretsManager awsSecretsManager() {
        return AWSSecretsManagerClientBuilder.standard()
                .withEndpointConfiguration(new AwsClientBuilder
                        .EndpointConfiguration(localStack.getEndpointOverride(SECRETSMANAGER).toString(),
                        localStack.getRegion()))
                .withCredentials(localStackCredentialsProvider)
                .build();
    }    @Bean
    public SecretCacheConfiguration secretCacheConfiguration(AWSSecretsManager awsSecretsManager) {
        return new SecretCacheConfiguration()
                .withClient(awsSecretsManager())
                .withCacheItemTTL(Duration.ofHours(12).toMillis());
    }    @Bean
    public SqsTestUtils sqsTestUtils() {
        return new SqsTestUtils();
    }    @Bean
    public SecretMangerUtils secretMangerUtils() {
        return new SecretMangerUtils();
    }
}

DynamoDBTestUtils Class

For DynamoDB testing, the DynamoDBTestUtils class offers utility methods for creating and deleting DynamoDB tables during tests. Consider the following example:

package com.base.component.utils;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDB;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapperConfig;
import com.amazonaws.services.dynamodbv2.model.CreateTableRequest;
import com.amazonaws.services.dynamodbv2.model.DeleteTableRequest;
import com.amazonaws.services.dynamodbv2.model.GlobalSecondaryIndex;
import com.amazonaws.services.dynamodbv2.model.Projection;
import com.amazonaws.services.dynamodbv2.model.ProjectionType;
import com.amazonaws.services.dynamodbv2.model.ProvisionedThroughput;import java.util.Map;
import java.util.Objects;/**
 * The DynamoDBTestUtils class provides utility methods for creating and deleting DynamoDB tables for testing purposes.
 * It allows the creation and deletion of tables specified in a map where the table name is mapped to its corresponding
 * class representing the data model.
 */
public class DynamoDBTestUtils {
    private void createTables(final Map<String, Class<?>> tables,
                             final DynamoDBMapper dynamoDBMapper,
                             final AmazonDynamoDB amazonDynamoDB) {
        // Iterate through all the tables and create them if they don't exist
        for (String tableName : tables.keySet()) {
            // Create the table if it doesn't exist
            if (!amazonDynamoDB.listTables().getTableNames().contains(tableName)) {
                CreateTableRequest tableRequest = dynamoDBMapper.generateCreateTableRequest(tables.get(tableName),
                        new DynamoDBMapperConfig.TableNameOverride(tableName).config());
                tableRequest.setProvisionedThroughput(new ProvisionedThroughput(5L, 5L));                if (!Objects.isNull(tableRequest.getGlobalSecondaryIndexes())) {
                    // Iterate through and Specify the Global Secondary Index (GSI) provisioned throughput
                    for (GlobalSecondaryIndex index : tableRequest.getGlobalSecondaryIndexes()) {
                        index.setProvisionedThroughput(new ProvisionedThroughput(5L, 5L));
                        index.setProjection(new Projection()
                                .withProjectionType(ProjectionType.ALL));
                    }
                }
                amazonDynamoDB.createTable(tableRequest);
            }
        }
    }    private void deleteTables(final Map<String, Class<?>> tables,
                             final DynamoDBMapper dynamoDBMapper,
                             final AmazonDynamoDB amazonDynamoDB) {
        // Iterate through all the tables and delete them
        for (String tableName : tables.keySet()) {
            // Delete the table if it exists
            if (amazonDynamoDB.listTables().getTableNames().contains(tableName)) {
                DeleteTableRequest deleteTableRequest = dynamoDBMapper.generateDeleteTableRequest(tables.get(tableName));
                amazonDynamoDB.deleteTable(deleteTableRequest);
            }
        }
    }    public void setupTables(final Map<String, Class<?>> tables,
                            final DynamoDBMapper dynamoDBMapper,
                            final AmazonDynamoDB amazonDynamoDB) {
        // Delete tables
        //deleteTables(tables, dynamoDBMapper, amazonDynamoDB);
        // Create tables
        createTables(tables, dynamoDBMapper, amazonDynamoDB);
    }
}

SecretManagerUtils and SQSTestUtils Classes

Similar to DynamoDB, the SecretManagerUtils class provides utility methods for creating and deleting secrets in AWS Secrets Manager during tests. Meanwhile, the SQSTestUtils class assists in managing SQS queues during tests.

package com.base.component.utils;
import com.amazonaws.services.secretsmanager.AWSSecretsManager;
import com.amazonaws.services.secretsmanager.model.CreateSecretRequest;
import com.amazonaws.services.secretsmanager.model.DeleteSecretRequest;
import com.amazonaws.services.secretsmanager.model.ListSecretsRequest;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import lombok.extern.slf4j.Slf4j;import java.util.Map;@Slf4j
public class SecretMangerUtils {    private final ObjectMapper objectMapper = new ObjectMapper();    private void createSecret(final AWSSecretsManager awsSecretsManager,
                             final String secretName,
                             final Map<String, String> secretValues)
            throws JsonProcessingException {
        CreateSecretRequest createSecretRequest = new CreateSecretRequest();
        createSecretRequest.setName(secretName);
        createSecretRequest.setSecretString(objectMapper.writeValueAsString
                (secretValues));        awsSecretsManager.createSecret(createSecretRequest);
    }    private void deleteSecret(final AWSSecretsManager awsSecretsManager,
                             final String secretName) {
        ListSecretsRequest listSecretsRequest = new ListSecretsRequest();
        listSecretsRequest.setMaxResults(100);
        if(awsSecretsManager.listSecrets(listSecretsRequest).getSecretList().stream()
                .anyMatch(secret -> secret.getName().equals(secretName))) {
            DeleteSecretRequest deleteSecretRequest = new DeleteSecretRequest();
            deleteSecretRequest.setSecretId(secretName);
            deleteSecretRequest.setForceDeleteWithoutRecovery(true);
            awsSecretsManager.deleteSecret(deleteSecretRequest);
        }
    }    public void setUpSecrets(final AWSSecretsManager awsSecretsManager,
                             final String secretName,
                             final Map<String, String> secretValues) {
        try {
            deleteSecret(awsSecretsManager, secretName);
            createSecret(awsSecretsManager, secretName, secretValues);
        } catch (Exception e) {
            log.error("Error while setting up secrets: {}", e.getMessage());
        }
    }
}
package com.base.component.utils;
import com.amazonaws.services.sqs.AmazonSQS;
import com.amazonaws.services.sqs.model.CreateQueueRequest;
import com.amazonaws.services.sqs.model.DeleteQueueRequest;
import com.base.Constants;
import lombok.extern.slf4j.Slf4j;@Slf4j
public class SqsTestUtils {    private void createQueue(final String queueName,
                            final AmazonSQS amazonSQS) {
        // Check if queue does not exist and create queue
        if (amazonSQS.listQueues(queueName).getQueueUrls().isEmpty()) {
            CreateQueueRequest createQueueRequest = new CreateQueueRequest(queueName);
            createQueueRequest.setQueueName(queueName);
            amazonSQS.createQueue(createQueueRequest);
        }
    }    private void deleteQueue(final String queueName,
                            final AmazonSQS amazonSQS) {
        // Check if queue exists and delete queue
        if (!amazonSQS.listQueues(queueName).getQueueUrls().isEmpty()) {
            DeleteQueueRequest deleteQueueRequest = new DeleteQueueRequest();
            deleteQueueRequest.setQueueUrl(amazonSQS.listQueues(queueName).getQueueUrls().get(0));
            amazonSQS.deleteQueue(deleteQueueRequest);
        }
    }    public void setUpQueue(final String queueName,
                            final AmazonSQS amazonSQS) {
        deleteQueue(queueName, amazonSQS);
        createQueue(queueName, amazonSQS);
    }}

Utilizing LocalStack in Tests

With the infrastructure in place, let’s now see how to integrate LocalStack into your tests. Consider a test class like ComponentTest, which utilizes LocalStack for testing interactions with AWS services. The following example illustrates this:

@SpringBootTest
@ContextConfiguration(classes = {ComponentTestConfiguration.class})
@Testcontainers
public class ComponentTest {
    private static StreamLambdaHandler handler;
    private static Context lambdaContext;
    @Autowired
    private DynamoDBMapper dynamoDBMapper;
    @Autowired
    private AmazonDynamoDB amazonDynamoDB;
    @Autowired
    private TransactionRepository transactionRepository;
    @Autowired
    private OrderRepository orderRepository;
    @Autowired
    private ThirdPartyClient thirdPartyClient; // Assuming a generic third-party client
    @Autowired
    private DynamoDBTestUtils dynamoDBTestUtils;
    @Autowired
    private SqsTestUtils sqsTestUtils;
    @Autowired
    private SecretMangerUtils secretManagerUtils;
    @Autowired
    private AmazonSQS amazonSQS;
    @Autowired
    private AWSSecretsManager awsSecretsManager;
    @Autowired
    private SecretCacheConfiguration secretCacheConfiguration;
    private final ObjectMapper objectMapper = new ObjectMapper();    private static final Map<String, Class<?>> TABLES = Map.of(
            "order", Order.class,
            "transaction", Transaction.class,
            "customer", Customer.class
    );    @BeforeAll
    public static void setupClass() {
        handler = new StreamLambdaHandler();
        lambdaContext = new MockLambdaContext();
    }    @BeforeEach
    public void setup() throws JsonProcessingException {        // Setup tables
        dynamoDBTestUtils.setupTables(TABLES, dynamoDBMapper, amazonDynamoDB);
        // Seed data
        seedTableData();        // Setup secrets
        secretManagerUtils.setUpSecrets(awsSecretsManager, "test/component-test",
                Map.of("encryption-key-customer",
                        "dummy-encryption-key"));
        // Setup SQS
        sqsTestUtils.setUpQueue("alert-queue", amazonSQS, externalConfigHolder);    }
    
    @Test
    void yourTest() {
        //your test goes in here
    }
  • @SpringBootTest: This annotation is used to indicate that the class should be considered a Spring Boot test. It boots up a Spring ApplicationContext and provides the necessary infrastructure for testing.
  • @ContextConfiguration: Specifies the configuration classes that will be used to initialize the application context. In this case, it references ComponentTestConfiguration.class.
  • @Testcontainers: This annotation is used in conjunction with the Testcontainers library, which allows you to define and use Docker containers within your tests. In this context, it’s likely used to manage LocalStack containers for testing AWS services locally.
  • @BeforeAll: This annotation is used on a method to signal that it should be executed once before all the tests in the class. In this example, it’s used to set up the StreamLambdaHandler and the lambda context.
  • @BeforeEach: This annotation is used on a method to signal that it should be executed before each test method in the class. Here, it’s used for setting up tables, seeding data, and configuring secrets and SQS before each test.
  • @Test: This is a standard JUnit annotation indicating that the annotated method is a test method.
  • StreamLambdaHandler: This appears to be a class representing a handler for AWS Lambda functions that process stream events (like SQS events). It’s used to handle streaming events, possibly in the context of AWS Lambda functions.

Conclusion

In this in-depth guide, we navigated through the process of setting up and effectively using LocalStack for testing AWS services in a Java Spring project. LocalStack, combined with Testcontainers, offers a robust testing environment that closely simulates AWS cloud services.

As you integrate LocalStack into your project, remember to customize the provided code snippets to align with your project’s structure and specific requirements. By doing so, you’ll be on your way to achieving a comprehensive and reliable testing strategy. Happy coding and testing!

Leave a comment

Your email address will not be published. Required fields are marked *