Linux has some problems with closed connections and after some connects (ca. 50) it does not accept any future connection for aprx. 1 minute.
Very slow in responses because of the default setting of maximum 4 VTYs. Timeouts of 1 sec are normal here.