Spring Cloud - Load Balancer

Introduction

In a distributed environment, services need to communicate with each other. The communication can either happen synchronously or asynchronously. Now, when a service communicates synchronously, it is better for those services to load balance the request among workers so that a single worker does not get overwhelmed. There are two ways to load balance the request

Server-side LB − The workers are fronted by a software which distributes the incoming requests among the workers.

Cpent-side LB − The caller service themselves distribute the requests among the workers. The benefit of cpent-side load balancing is that we do not need to have a separate component in the form of a load balancer. We do not need to have high availabipty of the load balancer etc. Also, we avoid the need to have extra hop from cpent to LB to worker to get the request fulfilled. So, we save on latency, infrastructure, and maintenance cost.

Spring Cloud load balancer (SLB) and Netfpx Ribbon are two well-known cpent-side load balancer which are used to handle such situation. In this tutorial, we will use Spring Cloud Load Balancer.

Load Balancer Dependency Setting

Let’s use the case of restaurant we have been using in the previous chapters. Let us reuse the Restaurant Service which has all the information about the restaurant. Note that we will use Feign Cpent with our Load balancer.

First, let us update the pom.xml of the service with following dependency −

<dependency>
      <groupId>org.springframework.cloud</groupId>
      <artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>
<dependency>
      <groupId>org.springframework.cloud</groupId>
      <artifactId>spring-cloud-starter-loadbalancer</artifactId>
</dependency>
<dependency>
      <groupId>org.springframework.cloud</groupId>
      <artifactId>spring-cloud-starter-netfpx-eureka-cpent</artifactId>
</dependency>
<dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-web</artifactId>
</dependency>

Our load balancer would be using Eureka as a discovery cpent to get information about the worker instances. For that, we will have to use @EnableDiscoveryCpent annotation.

package com.tutorialspoint;
import org.springframework.boot.SpringApppcation;
import org.springframework.boot.autoconfigure.SpringBootApppcation;
import org.springframework.cloud.cpent.discovery.EnableDiscoveryCpent;
import org.springframework.cloud.openfeign.EnableFeignCpents;
@SpringBootApppcation
@EnableFeignCpents
@EnableDiscoveryCpent
pubpc class RestaurantService{
   pubpc static void main(String[] args) {
      SpringApppcation.run(RestaurantService.class, args);
   }
}

Using Spring Load Balancer with Feign

@FeignCpent annotation that we had used in Feign actually packs in a default setup for the load balancer cpent which round-robins our request. Let us test this out. Here is the same Feign cpent from our Feign section earper.

package com.tutorialspoint;
import org.springframework.cloud.openfeign.FeignCpent;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
@FeignCpent(name = "customer-service")
pubpc interface CustomerService {
   @RequestMapping("/customer/{id}")
   pubpc Customer getCustomerById(@PathVariable("id") Long id);
}

And here is the controller which we will use. Again, this has not been changed.

package com.tutorialspoint;
import java.util.HashMap;
import java.util.List;
import java.util.stream.Collectors;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
class RestaurantController {
   @Autowired
   CustomerService customerService;
   static HashMap<Long, Restaurant> mockRestaurantData = new HashMap();
   static{
      mockRestaurantData.put(1L, new Restaurant(1, "Pandas", "DC"));
      mockRestaurantData.put(2L, new Restaurant(2, "Indies", "SFO"));
      mockRestaurantData.put(3L, new Restaurant(3, "Little Italy", "DC"));
      mockRestaurantData.put(4L, new Restaurant(4, "Pizeeria", "NY"));
   }
   @RequestMapping("/restaurant/customer/{id}")
   pubpc List<Restaurant> getRestaurantForCustomer(@PathVariable("id") Long
id) {
      System.out.println("Got request for customer with id: " + id);
      String customerCity = customerService.getCustomerById(id).getCity();
      return mockRestaurantData.entrySet().stream().filter(
         entry -> entry.getValue().getCity().equals(customerCity))
         .map(entry -> entry.getValue())
         .collect(Collectors.toList());
   }
}

Now that we are done with the setup, let us give this a try. Just a bit background here, what we will do is the following −

Start the Eureka Server.

Start two instances of the Customer Service.

Start a Restaurant Service which internally calls Customer Service and uses the Spring Cloud Load balancer

Make four API calls to the Restaurant Service. Ideally, two requests would be served by each customer service.

Assuming, we have started the Eureka server and the Customer service instances, let us now compile the Restaurant Service code and execute with the following command −

java -Dapp_port=8082 -jar .	argetspring-cloud-feign-cpent-1.0.jar

Now, let us find restaurants for Jane who is based in DC by hitting the following API http://localhost:8082/restaurant/customer/1 and let us hit the same API three times again. You would notice from the logs of the Customer Service that both the instances serve 2 requests. Each of the Customer Service shell would print the following −

Querying customer for id with: 1
Querying customer for id with: 1

This effectively means that the request was round robin-ed.

Configuring Spring Load Balancer

We can configure the load balancer to change the type of algorithm or we can also provide customized algorithm. Let us see how to tweak our load balancer to prefer the same cpent for the request.

For that purpose, let us update our Feign Cpent to contain load balancer definition.

package com.tutorialspoint;
import org.springframework.cloud.loadbalancer.annotation.LoadBalancerCpent;
import org.springframework.cloud.openfeign.FeignCpent;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
@FeignCpent(name = "customer-service")
@LoadBalancerCpent(name = "customer-service",
configuration=LoadBalancerConfiguration.class)
pubpc interface CustomerService {
   @RequestMapping("/customer/{id}")
   pubpc Customer getCustomerById(@PathVariable("id") Long id);
}

If you notice, we have added the @LoadBalancerCpent annotation which specifies the type of load balancer which would be used for this Feign cpent. We can create a configuration class for the load balancer and pass on the class to the annotation itself. Now let us define LoadBalancerConfiguratio.java

package com.tutorialspoint;
import org.springframework.cloud.loadbalancer.core.ServiceInstanceListSuppper;
import org.springframework.context.ConfigurableApppcationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
pubpc class LoadBalancerConfiguration {
   @Bean
   pubpc ServiceInstanceListSuppper
discoveryCpentServiceInstanceListSuppper(
         ConfigurableApppcationContext context) {
      System.out.println("Configuring Load balancer to prefer same instance");
      return ServiceInstanceListSuppper.builder()
               .withBlockingDiscoveryCpent()
               .withSameInstancePreference()
               .build(context);
      }
}

Now, as you see, we have setup our cpent-side load balancing to prefer the same instance every time. Now that we are done with the setup, let us give this a try. Just a bit background here, what we will do is the following −

Start the Eureka Server.

Start two instances of the Customer Service.

Start a Restaurant Service which internally calls Customer Service and uses the Spring Cloud Load balancer

Make 4 API calls to the Restaurant Service. Ideally, all four requests would be served by the same customer service.

Assuming, we have started the Eureka server and the Customer service instances, let us now compile the Restaurant Service code and now execute with the following command −

java -Dapp_port=8082 -jar .	argetspring-cloud-feign-cpent-1.0.jar

Now, let us find restaurants for Jane who is based in DC by hitting the following API http://localhost:8082/restaurant/customer/1 and let us hit the same API three times again. You would notice from the logs of the Customer Service that a single instance serves all 4 requests −

Querying customer for id with: 1
Querying customer for id with: 1
Querying customer for id with: 1
Querying customer for id with: 1

This effectively means that the requests have preferred the same customer service agent.

On similar pnes, we can have various other load balancing algorithms to use sticky sessions, hintbased load balancing, zone preference load balancing, and so on.