I recently started working on an Apache Camel project at work. The application in question would, I imagine, have been an atypical use case for Apache Camel. It was used primarily for the DSL definition of a series of RESTful calls to dependent services. These routes were leveraged by the application’s controller layer.

My approach at the beginning and my approach by the end were pretty different.

Unless stated the below are examples are taken from the Apache Camel Wish I’d Known sample code on GitHub.

Unit vs Integration tests of routes


It took me some time at first to decide on the scoping of tests when working with Apache Camel. I generally err towards the London/Interaction style TDD testing. When working with Spring / Apache Camel I find this approach difficult to translate at times. In the context of these examples, what are we testing; the route DSL definition, or the results of progression through the route.

Lets take the following example from a Java DSL route definition:

from(String.format(QUEUE_BOND_MOVIE_LISTENER, topic, brokers, groupId))
        .log(INFO, "Message received from Kafka : ${body}")
        .log(INFO, "    on the topic ${headers[kafka.TOPIC]}")
        .log(INFO, "    on the partition ${headers[kafka.PARTITION]}")
        .log(INFO, "    with the offset ${headers[kafka.OFFSET]}")
        .log(INFO, "    with the key ${headers[kafka.KEY]}")
        .unmarshal()
        .json(JsonLibrary.Jackson, BondMovieRequest.class)
        .enrich(GET_MOVIE_DETAILS_ROUTE, detailsAggregationStrategy)
        .to(PERSIST_AGGREGATE_MOVIE_DETAIL_ROUTE)
        .setBody(constant(""));

Note: this example is taken from a sample project.

What should be tested?

Side Effects ❌

        .log(INFO, "Message received from Kafka : ${body}")
        .log(INFO, "    on the topic ${headers[kafka.TOPIC]}")
        .log(INFO, "    on the partition ${headers[kafka.PARTITION]}")
        .log(INFO, "    with the offset ${headers[kafka.OFFSET]}")
        .log(INFO, "    with the key ${headers[kafka.KEY]}")

I would consider the log messages to be a side effect. I wouldn’t recommend verification of calling them as they have no concrete effect on the execution of the route.

3rd Party Libraries ✅

        .unmarshal()
        .json(JsonLibrary.Jackson, BondMovieRequest.class)

These methods will be invoked in Unit and Integration tests. They will expect to have relavent exchange properties set. In the example above the exchange is expected to have a BondMovieRequest as the in body. I would recommend both Unit and Integration testing this. There are probably many pros and cons, but one fact is that can be difficult to mock 3rd party libraries. As a rule of thumb I do not think it is desirable to mock 3rd party libraries anyway.

Local Components

        .enrich(GET_MOVIE_DETAILS_ROUTE, detailsAggregationStrategy)

This line I would find a bit more problematic. With regard to a Unit test, testing this would in effect be testing the DetailsAggregationStrategy. That would imply duplication of testing. However, if I mock it, none of the following tests will have the values you might expect them to have. I made the decsion to mock; bean, processor, enrich, and other EIP components with local processing for the route Unit tests.

With regard to an Integration test it is more straightforward, we are testing that this component (DetailsAggregationStrategy) works as expected with the other components of the route.

        .to(PERSIST_AGGREGATE_MOVIE_DETAIL_ROUTE)
        .setBody(constant(""));

Now, we are obliged to inject data intermittently into our route in order that it does not fail. This is because any processors that we would expect to be transforming our exchange are not being called.

Using ids for everything that is an external call, to make for easier testing

routeId

from(READ_ROBOTS_JSON_ROUTE)
        .routeId(READ_ROBOTS_JSON_ROUTE_ID)

Assigning a routeId to our route allows us to easily retrieve a reference to our route when creating our unit tests as can be seen below. Once we have a reference to the route we can then modify it as required for mocking purposes. An example of this can be seen below.

context.getRouteDefinition(READ_ROBOTS_JSON_ROUTE_ID).adviceWith(context, new AdviceWithRouteBuilder() {
    @Override
    public void configure() { }
});

Using the above code we can use the weaveBy* set of methods to modify the route at run time.

Another point to note about the routeId is that the value that is used here will also be the prefix that is used when log messages are output. Therefore I would recommend using a properly namespaced value so that your logback rules work as expected when changing log levels for certain paths.

id

Ids can also be set on particular elements of a route. Used in conjunction with the getRouteDefinition method above this allows us to directly splice mock elements into our route as required. The following page has more details about the various weave methods https://people.apache.org/~dkulp/camel/advicewith.html. I have found that weaveById is the most reliable. This is because when under test a modified Apache Camel route can become difficult to work with. Methods like weaveAddFirst and weaveAddLast which assume the route is still in its initial state can become unpredicatable.

        .enrich(GET_ROBOT_QUOTES_ROUTE, quoteAggregationStrategy)
        .id(READ_ROBOTS_JSON_NAME + "-" + GET_ROBOT_QUOTES_ENDPOINT_ID)

The effect of the code below will be to replace the enrich call with a mock endpoint. Assertions can then be placed on the MockEndpoint. Additionally callbacks can be configured on the mock endpoint which allow us to modify the test date during the test. Lastly, it also allows us to simulate some failure scenarios that can at times be difficult to simulate.

        context.getRouteDefinition(READ_ROBOTS_JSON_ROUTE_ID).adviceWith(context, new AdviceWithRouteBuilder() {
            @Override
            public void configure() {
                weaveById(READ_ROBOTS_JSON_NAME + "-" + GET_ROBOT_QUOTES_ENDPOINT_ID)
                        .replace()
                        .to(mockGetQuoteEndpoint);
            }
        });

Properly leverage the MockEndpoint system and using expect methods and assertIsSatisfied

Once an element of a route is mocked, the mock endpoint system can be leveraged to perform a lot of useful tasks. First we need to create our mock endpoint.

MockEndpoint mockGetQuoteEndpoint = getMockEndpoint("mock:direct://getRobotQuotes");

After we have replaced an element of the route with our mock endpoint we can create some expectations. Some example of what we can assert are;

        mockFuturamaApiEndpoint.expectedMessagesMatches(exchange -> exchange.getIn().getBody(String.class).equals(""));
        mockFuturamaApiEndpoint.expectedHeaderReceived(HTTP_PATH, "/characters/mock-name");

We can also modify all exchanges;

        mockFuturamaApiEndpoint.whenAnyExchangeReceived(exchange ->
                exchange.getIn().setBody("[{\"quote\":\"test quote\"}]"));

Or configure it so that a series of responses can be applied;

        mockFuturamaApiEndpoint.whenExchangeReceived(0, exchange ->
                exchange.getIn().setBody("[{\"quote\":\"test quote 1\"}]"));
        mockFuturamaApiEndpoint.whenExchangeReceived(1, exchange ->
                exchange.getIn().setBody("[{\"quote\":\"test quote 2\"}]"));

Finally, after we have made our test request on our route we can assert that all our configured expectations have been satisfied.

        assertMockEndpointsSatisfied();

Embrace EIP patterns

EIP (Enterprise Integration Patterns) is a series of patterns which can be applied to your solutions. They are the patterns that have guided the base Java DSL components of Apache Camel. Patterns of Enterprise Application Architecture is a highly recommended book which delves into each of the patterns. Apache Camel’s own documentation on EIP patterns is also very good and can quickly familiarise you with the various patterns that are available.

There is also a standard icon set available for the patterns which can make it more convenient to create high level diagrams for your routes. Below is an example that was produced in LucidChart

Abstract the transport details

When coding in Spring Boot I tend to follow the three tier architecture of model/data, services, and controller. One of the benefits of this is that once the objects have reached the service layer they are in a format that has been stripped of any transport specific information. This could include HTTP headers, Kafka message headers, whatever the case may be. I would recommend following the same pattern in Apache Camel and after each transport operation marshalling the data back into a transport agnostic form more appropriate for the service layer. This makes it easier to refactor your routes as the data types will be standardised.

Manage your types

Directly related to the point regarding Abstract the transport details, the point here is to ensure that an appropriate value type exists such that the information required by your route is present. This can also be accomplished by utilising properties however the more properties that are created the greater the complexity of the route. I would prefer as concise as possible a type which can then be mutated by the route components as required.

Difference between to and multicast, and why recipientList is not a replacement

The to EIP is used to send one instance of an Exchange to a destination.

The multicast EIP is used to send a copy of an Exchange to each destination specified.

The recipientList EIP is used to send a copy of an Exchange to a dynamically generated set of destinations.

recipientList and split are extended from multicast. I mention this specifically as I have seen recipientList used where multicast is probably the better fit.

Expose as much as possible through the DSL

This is another YMMV point, but I would suggest moving as much logic as possible into the route DSL. This makes the routes more readable and means less traversal of the code to see what is happening. If the EIP patterns are properly embraced there shouldn’t be too many places where it is necassary to shift your logic into nameless processors.

Predicates can really make the DSL calls obvious

To aid in Exposing as much as possible through the DSL leveraging the Predicate type can make your choice and filter code more readable. While the Simple Expression Language has some basic support for comparisons Predicates allow you to have more complex rules.

Predicate isUserAvailable = exchange -> {
    User user = exchange.getIn().getBody(User.class)
    return user != null && user.isActive() && user.isAvailable();
};

from("direct://processUser")
    .choice()
        .when(isUserAvailable)
            .to("direct://addUser")
        .end
    .end()

stopOnException for splits can save unnecessary hassle

Very specific piece of advice, but, the multicast based endpoint routers support a property called stopOnException. This can be very useful if you are performing a parallel series of tasks that you want to fail early.

Split HTTP parts out to their headers (it makes it easer to test)

I worked a bit with the HTTP4 component recently and something that I don’t think I fully appreciated was how to leverage the message headers. I had been attempting to too directly construct URLs. Using the supplied headers correctly however allows for easier testing than attempting to construct strings.

                .setHeader(HTTP_PATH, simple("/characters/${body[0].name}"))
                .setHeader(HTTP_METHOD, GET)
                .setHeader(CONTENT_TYPE, constant(APPLICATION_JSON))
                .setBody(constant(""))
                .to(FUTURAMA_QUOTES_API_BASE_URL)