3

When we instantiate gRPC server we pass ThreadPoolExecutor to its constructor:

grpc.server(futures.ThreadPoolExecutor(max_workers=1))

As I know, Python has GIL that makes useless usage of threads for CPU-bound tasks.

For example, my gRPC server serves Tensorflow model and gRPC health checking service. Are there any benefits for me to increase the number of threads in thread pool?

0x1337
  • 1,074
  • 1
  • 14
  • 33

1 Answers1

3

right now grpc python is tied to concurrent futures threadpool, I believe the roadmap shows in the future we'll be able to use asyncio. The threadpool executor just allows us to specify the maximum number of concurrent connections. You are correct that with python we have the GIL, as a result the threads are limited to the same CPU core therefore we don't have parallelism with threads. If your tasks are CPU bound they don't benefit in python by adding more threads.

It really depends on the details of your application whether more threads in the threadpool would have any benefit. You could run your grpc server in one process and your tensorflow in another process (using multiprocessing module), this gives you access to more cores, but you'll need to determine your protocol for transferring information from your tensorflow process to your grpc server.

Kevin S
  • 930
  • 10
  • 19
  • Thank you! One question, If I set `max_workers=1`, will health-check servicer block while another servicer or procedure (with model) is running? And will it block if I increase the number of workers? – 0x1337 Mar 27 '19 at 20:53
  • 1
    Yes it will block. If you have max_workers 1 and something connects, that worker is consumed with the first connection, blocking any additional connections until the first disconnects. If you increase the workers to 2, then you can have at most 2 concurrent connections. – Kevin S Apr 02 '19 at 13:19