Python SSH Server( twisted.conch) takes up high cpu usage when a large number of echo

Question

I write a SSH server with Twisted Conch. But encountered a difficult problem. Assume that user A and user B log in to the twisted ssh server through ssh command. Then user A tail or cat a large file (greater than 100M) on the server, which will cause a lot of echoing through the twisted ssh server, making python ssh process (twisted.conch ) cpu usage is very high (greater than 95%, or even 100%), then the user B will be blocked, a long time no response. Is there any way to sleep userA's session(0.5 seconds) when find the user A has a large number of echo through the twisted ssh server and not blocking the other connected users.

import sys  
import checkers  
from twisted.python import components, log, logfile  
from twisted.cred import portal  
from twisted.internet import reactor  
from twisted.conch.ssh import factory, keys, session, filetransfer  
from twisted.conch.unix import UnixSSHRealm, SSHSessionForUnixConchUser, UnixConchUser  
import keyvalue  
if __name__ == "__main__":  
   sshFactory = factory.SSHFactory()  
   sshFactory.portal = portal.Portal(UnixSSHRealm())  
   sshFactory.portal.registerChecker(checkers.UsernamePasswordChecker())  

   sshFactory.publicKeys = {
    'ssh-rsa': keys.Key.fromString(keyvalue.publicKey)}
   sshFactory.privateKeys = {
    'ssh-rsa': keys.Key.fromString(keyvalue.privateKey)}
   components.registerAdapter(
    SSHSessionForUnixConchUser, UnixConchUser, session.ISession)
   log.startLogging(sys.stdout)

   reactor.listenTCP(2222, sshFactory)
   reactor.run()

score 2 · Answer 1 · answered Nov 16 '16 at 08:41

This is effectively a bug in Twisted. One user using the server should not generate so much load that it's unresponsive to everyone else.

However, it's not an easy one to fix. There are a couple of solutions.

First, before you do anything else, you should ensure your code is using PyPy, which may give you all the additional performance you need to support more users. Even if it isn't sufficient, it should be helpful in combination with these other solutions.

One is that you can run this code in multiple processes, using a strategy like this, which will allow you to preemptively run the process on multiple cores. Of course, that doesn't let you do much concurrently inside one process.

Another option is that you could use twisted.protocols.htb, which you could use on sshFactory, to rate-limit incoming traffic and ensure it is processed fairly between competing connections.

Please share any progress that you make on this, as I'm sure it would be interesting to other Twisted users!

Python SSH Server( twisted.conch) takes up high cpu usage when a large number of echo

1 Answers1