0

I have a systems design challenge that I would like to get some community feedback on.

Basic system structure:

[Client] ---HTTP-POST--> [REST Service] ---> [Queue] ---> [Processors]

  • [Client] POSTs json to [REST Service] for processing.
  • Based on request, [Rest Services] sends data to various queues to be picked up by various processors written in various languages and running in different processes.
  • Work is parallelized in each processor but can still take up to 30 seconds to process. The time to process is a function of the complexity of the data and cannot be speed up.
  • The result cannot be streamed back to the client as it is completed because there is a final post processing step that can only be completed once all the sub steps are completed.

Key challenge: Once the post processing is complete, the client either needs to:

  • be sent the results after the client has been waiting
  • be notified async that the job is completed and passed an id to request the final result

Design requirements

I don't want to block the [REST Service]. It needs to take the incoming request, route the data to the appropriate queues for processing in other processes, and then be immediately available for the next incoming request.

Normally I would have used actors and/or futures/promises so the [REST Service] is not blocked when waiting for background workers to complete. The challenge here is the workers doing the background work are running in separate processes/VMs and written in various technology stacks. In order to pass these messages between heterogeneous systems and to ensure integrity of the request lifetime, a durable queue is being used (not in memory message passing or RPC).

Final point of consideration, in order to scale, there are a load balanced set of [REST Services] and [Processors] in respective pools. Therefore, since the messages from the [REST Service] to the [Processor] need to be sent asynchronously via a queue (and everything is running is separate processes), there is no way to correlate the work done in a background [Processor] back to its original calling [REST Service] instance in order to return the final processed data in a promise or actor message and finally pass the response back to the original client.

So, the question is, how to make this correlation? Once the all the background processing is completed, I need to get the result back to the client either via a long waited response or a notification (I do not want to use something like UrbanAirship as most of the clients are browsers or other services.

I hope this is clear, if not, please ask for clarification.

Edit: Possible solution - thoughts?

I think I pass a spray RequestContext to any actor which can then response back to the client (does not have to be the original actor that received HTTP request). If this is true, can I cache the RequestContext and then use it later to asynchronously send the response to the appropriate client using this cached RequestContext when the processing is completed?

IUnknown
  • 2,596
  • 4
  • 35
  • 52
  • Well... generate a UUID for everything you send to the processors and whenever processors produce something to the results queue, they specify this UUID. – sarveshseri Mar 16 '15 at 12:18
  • @SarveshKumarSingh yes, we generate a "job id" (uuid) when a request is created and it is including in all messages so the final resultant data can be correlated. Sending this id back to the client in an immediate response would work, but I don't want the make the client poll until the result is ready. I would like for the original call context on the original [REST Service] instance (in a non-blocking way) respond with the result from a received promise or akka message when all the background processing is complete. The core issue is how to make this correlation? – IUnknown Mar 16 '15 at 12:34
  • You are not looking for correlation. you are looking at a way to respond back to the client. You can either choose something like sockets, or you can publish your final output to the results queue and make the client pick results from there. – sarveshseri Mar 16 '15 at 12:41
  • @SarveshKumarSingh we are already publishing the final results to a queue. Again, the question is: I do not want the client to have to poll for a job id. I want to response back to the client (in a non blocking way) when the final result is calculated. This will require me to respond with a akka message or promise back the instance of the [REST Service] so it can send the response back the calling client. This is what I would like to do - I think this requires me to match (or correlate) the completed job, the final calculated result back to the calling client. – IUnknown Mar 16 '15 at 12:53
  • Since you are going with a worker service, I suppose your jobs are supposed to take a lot of time or some jobs will be waiting in to be picked up by workers. This means you don't want to respond to your client the usual way where server receives the request and responds with the results. So... you want a way to communicate to the client without the need of client sending a request to your first ( polling works by keeping on sending requests ). The most popular ways of achieving this are - Sockets or message queues. So tell your client to connect to message queue and publish to same. – sarveshseri Mar 16 '15 at 12:59
  • Maybe you can use the ask pattern or an Inbox to send back a message or watch other actors lifecycle. http://stackoverflow.com/questions/27381815/difference-between-actorref-tell-and-inbox-send-from-akka-example – akkie Mar 16 '15 at 13:34

1 Answers1

0

Well, it's not the best because it requires more work from your Client, but it sounds like you want to implement a webhook. So,

[Client] --- POST--> [REST Service] ---> [Calculations] ---> POST [Client]

[Client] --- GET

For explanation: Client sends a POST request to your service. Your Service then does whatever processing necessary. Upon completion, your service will then send an HTTP-POST to a URL that the Client has already set. With that POST data, the Client will then have the necessary information to then do a GET request for the completed data.

Donato Perconti
  • 814
  • 2
  • 11
  • 28