외부 API 클라이언트 테스트하기 Testing External API Clients

In today’s development environment where microservice architecture (MSA) has become mainstream, applications communicate with various external services in multiple ways. When implementing features, an application typically integrates with at least three or four external services. Since HTTP protocol is most commonly used both for exposing your service’s API and consuming other services’ APIs, situations where you need to write code that calls HTTP APIs according to specifications defined by API providers occur quite frequently.

This brings up several important engineering considerations specific to different languages and frameworks: which HTTP client to use, how to handle exceptions, how to improve performance when calling multiple APIs, and more. In this post, I’ll discuss testing external API clients specifically, and share testing examples in Spring Framework-based development environments.

Do We Need to Test External API Clients?

When it comes to writing tests for external API clients, some people question the necessity itself. They argue that testing external APIs is meaningless because external APIs are external process resources that are uncontrollable from our application. They say that since our application’s normal operation is ultimately affected by the state and functional changes of external APIs in production, only integration tests in development or staging environments are meaningful.

However, this statement is half right and half wrong. External APIs are certainly external resources uncontrollable from our application, so no matter how well we test in our application, we cannot guarantee they will always work correctly. However, what we want to verify in tests is always our own code, not external services.

If you have experience writing external HTTP API integration clients, you’ve probably encountered quite a few serialization and deserialization related errors. You might have exceptions during JSON mapping of requests/responses due to incorrect DTO class field names, or in worse cases, missing DTO fields might be initialized with default values according to mapper settings, making bugs hard to notice.

Therefore, we need to test whether we are making requests ‘correctly’ according to the API request and authentication specifications defined by external service providers, and whether we are ‘correctly’ receiving the responses they promised to provide - in other words, whether we are properly adhering to the contract. This is completely separate from whether the external service is operating normally.

Tests as Documentation

Another advantage of writing tests is that they can function as documentation themselves. In many cases, when initially integrating an API, there’s typically no information about that external service except for the developer who developed the integration client. When maintenance of that client becomes necessary later, if the client code doesn’t include rich exception handling for various situations, the new developer taking over maintenance will only be able to infer API specifications that aren’t properly revealed in the code after directly calling the API separately or searching through internal portals for documentation from the API providing department.

// If an exception occurs in the API, what format would the exception be...?
public ProductDto getProduct(Long id) {
    return client.exchange("/product/{id}", HttpEntity.EMPTY, ProductDto.class, id).getBody();
}

However, if test code for that client is written, you can easily understand which API is being called, what exceptions might occur, what responses can be received, etc., through the test code. Even if you ultimately need to refer to documentation from the API providing department, you can significantly reduce the time needed to understand basic functionality.

Unit Testing with MockServer

Using MockServer, you can test client code that calls external APIs without actually calling external API servers. Compared to Spring’s @RestClientTest which can only test code using RestTemplate and RestTemplateBuilder, it has the advantage of enabling mock testing in a client library-neutral way.

Dependency

dependencies {
    testImplementation 'org.mock-server:mockserver-netty:5.15.0'
}

Client Code

The HTTP Client is implemented using RestClient which is available from Spring Boot 3.2.

public record ProductDto (
        Long id,
        String name,
        String description,
        Integer price
) {}

@Component
public class RestProductClient {

    private final RestClient client;

    public RestProductClient(String host) {
        this.client = RestClient.create(host);
    }

    public ProductDto getProduct(Long id) {
        return client.get()
                .uri("/products/{id}", id)
                .retrieve()
                .body(ProductDto.class);
    }
}

Test Code

class RestProductClientTest {

    private ClientAndServer mockServer;
    private RestProductClient client;

    @BeforeEach
    void setUp() {
        mockServer = ClientAndServer.startClientAndServer(9876);
    }

    @AfterEach
    void tearDown() {
        mockServer.stop();
    }

    @Test
    void getProductById() {
        // Given
        var client = new RestProductClient("http://localhost:9876"); // (1)
        var productId = 1L;
        var responseBody = """
                {
                  "id": 1,
                  "name": "iPhone 12",
                  "description": "Apple iPhone 12 64GB",
                  "price": 1000000
                }
                """;
    

        // When
        mockServer
                .when(
                    request() // (2)
                        .withMethod("GET")
                        .withPath("/products/1")
                ).respond(
                    response() // (3)
                        .withStatusCode(200)
                        .withBody(json(responseBody))
                );

        var result = client.getProduct(1L); // (4)
        var expected = new ProductDto(
                1L,
                "iPhone 12",
                "Apple iPhone 12 64GB",
                1000000
        );

        // Then
        assertEquals(expected, result); // (5)
    }
}

• (1): Create a client instance to connect to the same port as the mockserver.
• (2): Specify the request method and path you expect to receive.
• (3): Specify the response to the request.
• (4): Execute the client code and bind the response.
• (5): Verify that the response matches the expected result.

When API Client is a Collaborating Object

What if it’s not a test of the API client implementation itself, but a test where a client like the RestProductClient implemented above is used as a collaborating object of other business objects? While you can similarly use MockServer to mock HTTP communication, since the core concern of the test is not HTTP communication but whether the expected additional business logic is properly performed based on responses received from external services, using MockServer might be somewhat unnecessary and cumbersome. This is easy to understand if you imagine not mocking the DB when testing a class that has a collaborative relationship with a Repository.

@Service
public class ProductService {

    private final RestProductClient client;

    public ProductService(RestProductClient client) {
        this.client = client;
    }
    
    public Integer getProductPrice(Long id) {
        return client.getProduct(id).price();
    }

}

Unit Testing with Mocks

When performing unit tests for ProductService, you can perform tests by mocking the collaborating object RestResourceClient.

class ProductServiceTest {

    private ProductService productService;
    private RestProductClient productClient;

    @Test
    void getProductPrice() {
        // Given
        productClient = mock(RestProductClient.class);
        given(productClient.getProduct(1L)).willReturn(new ProductDto(
                1L,
                "iPhone 12",
                "Apple iPhone 12 64GB",
                1000000
        ));
        productService = new ProductService(productClient);


        // When
        var result = productService.getProductPrice(1L);

        // Then
        assertEquals(1000000, result);
    }

}

Integration Testing with MockBean

When performing integration tests for Spring applications, you can mock collaborating objects using @MockBean.

@SpringBootTest
class ProductServiceIntegrationTest {

    @Autowired
    private ProductService service;

    @MockBean
    private RestProductClient client;

    @Test
    void getProductPrice() {
        // Given
        given(client.getProduct(1L)).willReturn(new ProductDto(
                1L,
                "iPhone 12",
                "Apple iPhone 12 64GB",
                1000000
        ));

        // When
        var result = service.getProductPrice(1L);

        // Then
        assertEquals(1000000, result);
    }
}

Adding Interfaces and Implementing Test Doubles Directly

The above two methods involved mocking the concrete class RestProductClient. However, mocking classes with external dependencies in every test code is not easy, and when using @MockBean, the Application Context is recreated for every test, which can make test execution time longer. In such cases, adding an interface and creating test double implementations that implement it can be one approach to performing tests.

public interface ProductClient {
    ProductDto getProduct(Long id);
}

@Profile({"dev", "production"})
@Component
public class RestProductClient implements ProductClient {

    private final RestClient client;

    public RestProductClient(@Value("${product.host}") String host) {
        this.client = RestClient.create(host);
    }

    @Override
    public ProductDto getProduct(Long id) {
        return client.get()
                .uri("/products/{id}", id)
                .retrieve()
                .body(ProductDto.class);
    }
}

// Stub implementation for test environment only
@Profile("test")
@Component
public class StubProductClient implements ProductClient {

    @Override
    public ProductDto getProduct(Long id) {
        return new ProductDto(id, "Product " + id, "Description " + id, 100 * id.intValue());
    }

}