Microservices – Bulkhead

What is Bulkhead?

The term comes from ships where a ship is divided into separate watertight compartments to avoid a single hull breach to flood the entire ship. It will only flood one bulkhead. Goal of the bulkhead pattern is to avoid faults in one part of a system to take the entire system down.

Implementations of the bulkhead pattern can take many forms depending on what kind of faults you want to protect the system from. We will discuss the type of faults Hystrix handles.

What issues are solved in Hystrix?

The bulkhead implementation in Hystrix limits the number of concurrent calls to a component. This way, the number of resources (typically threads) that is waiting for a reply from the component is limited.

Assume we have a request based, multi threaded application (e.g. web application) that uses three different components, C1, C2, and C3. If requests to component C3 starts to hang, eventually all request handling threads will hang on waiting for an answer from C3. This would make the application entirely non-responsive. If requests to C3 is handled slowly we have a similar problem if the load is high enough.

Implementation of the bulkhead pattern in Hystrix limits the number of concurrent calls to a component and would have saved the application in this case. Assume we have 30 threads to handle requests and there is a limit of 10 concurrent calls to C3. Then at most 10 threads can hang when calling C3, the other 20 threads can still handle requests and use components C1 and C2.

Hystrix Approaches to Implement Bulkhead

Hystrix has two different approaches to implement Bulkhead pattern
1. Thread isolation
2. Semaphore isolation

Thread Isolation
The standard approach is to hand over all requests to component C3 to a separate thread pool with a fixed number of threads and no (or a small) request queue.

Semaphore Isolation
The other approach is to have all callers acquire a permit (with 0 timeout) before requests to C3. If a permit cannot be acquired from the semaphore, calls to C3 are not passed through.

Configuring Bulkhead using Hystrix

@HystrixCommand(fallbackMethod = "getUserDetailsFallback",
		threadPoolKey = "userDetailsPoolKey",
		threadPoolProperties = {
				@HystrixProperty(name = "coreSize",value = "20"),
				@HystrixProperty(name = "maxPoolSize",value = "10")
		})
public UserDetails getUserDetails(String userId) {
	UserDetails userDetails = restTemplate.getForObject("http://user-details-service/user/" + userId, UserDetails.class);
	return userDetails;
}

public UserDetails getUserDetailsFallback(String userId) {
	return new UserDetails(0, "No user details found!", "");
}

Author: Mahesh

Technical Lead with 10 plus years of experience in developing web applications using Java/J2EE and web technologies. Strong in design and integration problem solving skills. Ability to learn, unlearn and relearn with strong written and verbal communications.