19

I'm struggling with the following problem: My App makes sequence of requests to the http server using HttpClient. I use HttpPut for sending data to the server. First request goes well and fast, second request hangs for 40 sec and then I catch Connection timed out exception. I'm trying to reuse my HttpClient and send second request through the same instance. If I create new HttpClient together with new ConnectionManager, then everything works fine.

Why is this happening? And how to fix it and do not create new HttpClient each time?

Thanks in advance.

Here is my code: (if I comment readClient = newHttpClient(readClient) in doPut, then the problem arises.

public class WebTest
{
private HttpClient readClient;
private SchemeRegistry httpreg;
private HttpParams params;

private URI url; //http://my_site.net/data/

protected HttpClient newHttpClient(HttpClient oldClient)
{
    if(oldClient != null)
        oldClient.getConnectionManager().shutdown();

    ClientConnectionManager cm = new SingleClientConnManager(params, httpreg);
    return new DefaultHttpClient(cm, params);
}

protected String doPut(String data)
{
    //****************************
    //Every time we need to send data, we do new connection
    //with new ConnectionManager and close old one
    readClient = newHttpClient(readClient);

    //*****************************


    String responseS = null;
    HttpPut put = new HttpPut(url);
    try
    {
        HttpEntity entity = new StringEntity(data, "UTF-8");
        put.setEntity(entity);
        put.setHeader("Content-Type", "application/json; charset=utf-8");
        put.setHeader("Accept", "application/json");
        put.setHeader("User-Agent", "Apache-HttpClient/WebTest");

        responseS = readClient.execute(put, responseHandler);
    }
    catch(IOException exc)
    {
        //error handling here
    }
    return responseS;
}

public WebTest()
{
    httpreg = new SchemeRegistry();
    Scheme sch = new Scheme("http", PlainSocketFactory.getSocketFactory(), 80);
    httpreg.register(sch);

    params = new BasicHttpParams();
    ConnPerRoute perRoute = new ConnPerRouteBean(10);
    ConnManagerParams.setMaxConnectionsPerRoute(params, perRoute);
    ConnManagerParams.setMaxTotalConnections(params, 50);
    ConnManagerParams.setTimeout(params, 15000);
    int timeoutConnection = 15000;
    HttpConnectionParams.setConnectionTimeout(params, timeoutConnection);
    // Set the default socket timeout (SO_TIMEOUT) 
    // in milliseconds which is the timeout for waiting for data.
    int timeoutSocket = 40000;
    HttpConnectionParams.setSoTimeout(params, timeoutSocket);
}

private ResponseHandler<String> responseHandler = new ResponseHandler<String>() 
{
    @Override
    public String handleResponse(HttpResponse response)
            throws ClientProtocolException, IOException
    {
        StatusLine statusLine = response.getStatusLine();
        if (statusLine.getStatusCode() >= 300) 
        {
            throw new HttpResponseException(statusLine.getStatusCode(),
                    statusLine.getReasonPhrase());
        }

        HttpEntity entity = response.getEntity();
        if(entity == null)
            return null;

        InputStream instream = entity.getContent();
        return this.toString(entity, instream, "UTF-8");
    }

    public String toString(
            final HttpEntity entity, 
            final InputStream instream, 
            final String defaultCharset) throws IOException, ParseException 
    {
        if (entity == null) 
        {
            throw new IllegalArgumentException("HTTP entity may not be null");
        }

        if (instream == null) 
        {
            return null;
        }
        if (entity.getContentLength() > Integer.MAX_VALUE) 
        {
            throw new IllegalArgumentException("HTTP entity too large to be buffered in memory");
        }
        int i = (int)entity.getContentLength();
        if (i < 0) 
        {
            i = 4096;
        }
        String charset = EntityUtils.getContentCharSet(entity);
        if (charset == null) 
        {
            charset = defaultCharset;
        }
        if (charset == null) 
        {
            charset = HTTP.DEFAULT_CONTENT_CHARSET;
        }

        Reader reader = new InputStreamReader(instream, charset);

        StringBuilder buffer=new StringBuilder(i);
        try 
        {
            char[] tmp = new char[1024];
            int l;
            while((l = reader.read(tmp)) != -1) 
            {
                buffer.append(tmp, 0, l);
            }
        } finally 
        {
            reader.close();
        }

        return buffer.toString();
    }
}; 

}

Northern Captain
  • 1,147
  • 3
  • 25
  • 32

7 Answers7

24

Sounds like you don't consume the entity after you finish handling the response. Ensure you put the following code in the finally block:

if (httpEntity != null) {
    try {
        httpEntity.consumeContent();
    } catch (IOException e) {
        Log.e(TAG, "", e);
    }
}

I suggest you read the HttpClient Tutorial.

neevek
  • 11,760
  • 8
  • 55
  • 73
  • I'm with Xamarin Android and had the same issue - disposing the http response message did the trick for me - httpResponseMessage.Dispose(); Thanks for the tip :) – baraka May 04 '16 at 19:12
13

Sounds strange, but I had the exact same problem. The app I was working on was making several successive requests to download a bunch of thumbnail images to display in a ListView, and after the second one it would hang as if there was a dead lock in the HttpClient code.

The strange fix that I found was to use AndroidHttpClient instead of DefaultHttpClient. As soon as I did this, and I tried a lot of stuff before going this route, it started working just fine. Just remember to call client.close() when you're done with the request.

AndroidHttpClient is described in the documentation as DefaultHttpClient with "reasonable default settings and registered schemes for Android". Since this was introduced in api level 8 (Android 2.2), I dug up the source to duplicate these "default settings" so that I could use it further back than that api level. Here is my code for duplicating the defaults and a helper class with a static method for safely closing it

public class HttpClientProvider {

    // Default connection and socket timeout of 60 seconds. Tweak to taste.
    private static final int SOCKET_OPERATION_TIMEOUT = 60 * 1000;

    public static DefaultHttpClient newInstance(String userAgent)
    {
        HttpParams params = new BasicHttpParams();

        HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
        HttpProtocolParams.setContentCharset(params, HTTP.DEFAULT_CONTENT_CHARSET);
        HttpProtocolParams.setUseExpectContinue(params, true);

        HttpConnectionParams.setStaleCheckingEnabled(params, false);
        HttpConnectionParams.setConnectionTimeout(params, SOCKET_OPERATION_TIMEOUT);
        HttpConnectionParams.setSoTimeout(params, SOCKET_OPERATION_TIMEOUT);
        HttpConnectionParams.setSocketBufferSize(params, 8192);

        SchemeRegistry schReg = new SchemeRegistry();
        schReg.register(new Scheme("http", PlainSocketFactory.getSocketFactory(), 80));
        schReg.register(new Scheme("https", SSLSocketFactory.getSocketFactory(), 443));
        ClientConnectionManager conMgr = new ThreadSafeClientConnManager(params, schReg);

        DefaultHttpClient client = new DefaultHttpClient(conMgr, params);

        return client;
    }

}

And in another class...

public static void safeClose(HttpClient client)
{
    if(client != null && client.getConnectionManager() != null)
    {
        client.getConnectionManager().shutdown();
    }
}
Rich
  • 36,270
  • 31
  • 115
  • 154
  • Thanks a lot, I'll try this method. Do you use one instance for all your requests after these changes or call newInstance every time you need to send data? And do you call safeClose after EACH request? – Northern Captain Feb 29 '12 at 19:59
  • Yep...new instance every time, and safeClose every time. I don't notice a performance hit at all, and this app I was doing at the time I first had this problem makes a ton of service requests. – Rich Feb 29 '12 at 20:01
  • If I do the same, close and open new instance every time, then the problem vanishes. But I read in the docs that it is strongly recommended to reuse HttpClient+ConnManager and do not create it for every connection, that should save a lot of time and cpu... Am I wrong? – Northern Captain Feb 29 '12 at 20:06
  • Well, put it like this. I did it that way, and even though it goes against recommended practice, I didn't take a performance hit. You might want to play around with it and maybe reuse teh connection manager and see if that works, but just because something goes against recommendation, I don't think that's reason enough to rule it out – Rich Feb 29 '12 at 20:11
  • 4
    Why did you use ThreadSafeClientConnManager and not SingleClientConnManager if you recreate httpClient every request and shutdown it after that? – Northern Captain Feb 29 '12 at 20:27
  • So you're actually using DefaultHttpClient as the object, not AndroidHttpClient, correct? – CACuzcatlan Sep 06 '12 at 18:45
  • 2
    Does this work if you try it 3 times in a row? This solution (without creating a new object each time) works for 2 requests, but not for 3. – CACuzcatlan Sep 06 '12 at 22:37
  • Yes @CACuzcatlan. I can use DefaultHttpClient, and I believe I just need to close the connection when I'm done (using that safeClose method above). – Rich Sep 07 '12 at 19:14
  • Mine doesn't work for 3 request in a row. Please help me out ! – Gaurav Arora Oct 25 '12 at 10:39
  • 2
    This is not a solution to the problem, this is a workaround. You can and should reuse the http client object. The answer posted by neevek below worked for me (consume the HttpResponse entity by calling entity.consumeContent()). – Daniel Apr 29 '13 at 19:57
  • 2
    I'm reading a real-time, infinite stream. consumeContent blocks indefinitely, it seems to insist on trying to reuse the connection. Three cheers for shutting down the connectionManager. Works for me! – Steven Kramer Jun 25 '13 at 16:39
  • I agree with Daniel it doesn't make sense to shutdown the whole httpclient when the problem is with the not comsuming content from the request – jiduvah Oct 03 '13 at 04:02
6

I've got the same trouble, when executing several requests in a loop.

You can solve it by reading all of response.getEntity().

Abel
  • 56,041
  • 24
  • 146
  • 247
wilddev
  • 1,904
  • 2
  • 27
  • 45
  • You should read stream to the end, for example InputStream is = response.getEntity(); while(is.read()!=-1); – wilddev Sep 13 '12 at 16:42
3

I thought I would elaborate on the other answers. I experienced this problem also. The problem was because I was not consuming content.

It seems if you don't then the connection will hold on to it and you are not able to send a new request with this same connection. For me it was a particularly difficult bug to spot as I was using the BasicResponseHandler provided in android. The code looks like this...

public String handleResponse(final HttpResponse response)
            throws HttpResponseException, IOException {
        StatusLine statusLine = response.getStatusLine();
        if (statusLine.getStatusCode() >= 300) {
            throw new HttpResponseException(statusLine.getStatusCode(),
                    statusLine.getReasonPhrase());
        }

        HttpEntity entity = response.getEntity();
       return entity == null ? null : EntityUtils.toString(entity);
    }

So if there is an status line above 300 then I don't consume the content. And there was content in my case. I made my own class like this...

public class StringHandler implements ResponseHandler<String>{

    @Override
    public BufferedInputStream handleResponse(HttpResponse response) throws IOException {
    public String handleResponse(final HttpResponse response)
                throws HttpResponseException, IOException {
            StatusLine statusLine = response.getStatusLine();
           HttpEntity entity = response.getEntity();
            if (statusLine.getStatusCode() >= 300) {
                if (entity != null) {
                    entity.consumeContent();
                }
                throw new HttpResponseException(statusLine.getStatusCode(),
                        statusLine.getReasonPhrase());
            }


           return entity == null ? null : EntityUtils.toString(entity);
        }
    }

}

So basically in any case consume the content!

jiduvah
  • 5,108
  • 6
  • 39
  • 55
  • Actually the response code may not be relevant; you should attempt to consume content regardless. E.g. I was downloading images, but would get a 404 response AND a "courtesy image" as content! This is also the case when getting 500 responses, most servers like to provide a "page" for that, which counts as content. – escape-llc Dec 14 '15 at 11:53
1

I have had this same problem. I am consuming all the content.

What I found is if I do a garbage collection after issuing a request, everything works without having to close and create a new AndroidHttpClient:

System.gc();

Peter
  • 288
  • 3
  • 8
1

It is enough for problem solution (I had the same):

EntityUtils.consume(response.getEntity());

Null check performed inside consume

Grigory Kislin
  • 16,647
  • 10
  • 125
  • 197
1

Since many of these answers are old and depend upon the now depricated consumeContent() method, I thought I'd answer with an alternative to the problem of Timeout waiting for connection from pool.

    HttpEntity someEntity =  response.getEntity();

    InputStream stream = someEntity.getContent();
    BufferedReader rd = new BufferedReader(new InputStreamReader(stream));

    StringBuffer result = new StringBuffer();
    String line = "";
    while ((line = rd.readLine()) != null) {
        result.append(line);
    }
    // On certain android OS levels / certain hardware, this is not enough.
    stream.close(); // This line is what is recommended in the documentation

Here is what it shows in the documentation:

cz.msebera.android.httpclient.HttpEntity
@java.lang.Deprecated 
public abstract void consumeContent()
                            throws java.io.IOException
This method is deprecated since version 4.1. Please use standard java
convention to ensure resource deallocation by calling
InputStream.close() on the input stream returned by getContent()
Dale
  • 5,520
  • 4
  • 43
  • 79