I am working on developing a robust Java API that plays a crucial role in handling and processing real-time price updates from various companies. The API is designed to receive incoming price updates from a third-party provider, store them in a database, and efficiently publish them to a Kafka topic. The API exposes REST endpoints of HTTP type POST to facilitate seamless communication with the third-party provider, ensuring they receive timely responses confirming successful price update consumption.
However, the challenge lies in the substantial volume of price updates, reaching up to 30,000 lines. To address this, I’ve implemented a multi-threaded approach for parallel processing. Each incoming request triggers a thread to handle the processing, optimizing throughput.
Here’s where the complexity arises: while parallel processing is essential for efficiency, there is a critical requirement to maintain the order of updates for each specific company. For instance, if there are sequential updates (p1-A, p2-A, p3-A) for Company A, and concurrent updates (p1-B, p2-B) for Company B, it’s crucial that p1-A and p1-B can be processed in parallel, but p1-A and p2-A must be processed sequentially.
To further complicate matters, if an update like p1-A is still being processed, any subsequent updates for Company A (e.g., p2-A) must gracefully wait for the ongoing process to complete. Rejecting requests is not an option; instead, the API should acknowledge receipt and provide the client with the processing status, ensuring a seamless and responsive interaction.
Can we achieve this by using Executor Service?
@PostMapping(“/update”)
public ResponseEntity updatePrice(@RequestBody PriceUpdateRequest request){
}
request:
P1-A, P2-A, P3-A (P1 is first price update for company A)
P1-B, P2-B, P3-B (P1 is first price update for company B)
Above endpoint can process P1-A and P1-B parallely but P1-A and P2-A has to processed sequentially. How to achieve the same ?
You can implement it with queues. One queue per company.