2

I need to send around 50 or 60 get requests at the same time to a distant server.

I don't know a lot about networking.

I just realized that for example:

sending 5 requests 10 times was totally different than sending 50 requests at the same time...

How can I know what's the most "efficient" number of concurrent HTTP requests to be send at the same time?

il_raffa
  • 5,090
  • 129
  • 31
  • 36
nootb
  • 23
  • 3
  • It's hard to say what's "efficient". Efficiency would mean there's a benefit in terms of outcome vs. effort. Moreover, in your situation, can you explain whether the requests are heavy (e.g. impacting a database), or lightweight (e.g., returning a small, static file)? Do you have the option of sending multiple requests in one HTTP request? That would lightly improve efficiency, but HTTP GETS are already pretty efficient. – some ideas Mar 31 '16 at 05:57
  • See if it can help.... https://stackoverflow.com/questions/35301274/node-js-is-it-possible-to-hit-1000-http-get-request-to-some-api-from-a-node-js – vkstack Mar 31 '16 at 06:57
  • And you can send multiple request to a server in simultaneously! – vkstack Mar 31 '16 at 06:59

1 Answers1

3

When talking only about the network, and not about the backend and heavy request processing: a big impact by many requests has the connection reusage. Talking about HTTP/1.* it is a Keep-Alive feature. With the HTTP/2 multiplexing is supported per default, and allows not only the reusage, but the simultaneous usage by requests

So making parallel 100 requests causes in NodeJS to open 100 connections, while require('http').globalAgent.maxSockets is set per default for Infinity. But when you make N request one after another, that means, the further requests can reuse previous connections.

But you shouldn't handle the connection pool by yourself, just set maxSockets to 15, and the nodejs agent will handle it for you. Note: it is difficult to say what the value actually should be, but before nodes version 0.12 the default value for maxSockets was also 5.

tenbits
  • 7,568
  • 5
  • 34
  • 53