Tuesday, June 18, 2013

Get a Python subprocess' output without buffering

Normally when you want to get the output of a subprocess in Python you have to wait until the process finishes. This is bad for long running processes. Here's a way to get the output unbuffered (in real-time.) See below or get the gist: https://gist.github.com/thelinuxkid/5114777
import contextlib
import subprocess
 
# Unix, Windows and old Macintosh end-of-line
newlines = ['\n', '\r\n', '\r']
def unbuffered(proc, stream='stdout'):
    stream = getattr(proc, stream)
    with contextlib.closing(stream):
        while True:
            out = []
            last = stream.read(1)
            # Don't loop forever
            if last == '' and proc.poll() is not None:
                break
            while last not in newlines:
                # Don't loop forever
                if last == '' and proc.poll() is not None:
                    break
                out.append(last)
                last = stream.read(1)
            out = ''.join(out)
            yield out
 
def example():
    cmd = ['ls', '-l', '/']
    proc = subprocess.Popen(
        cmd,
        stdout=subprocess.PIPE,
        stderr=subprocess.STDOUT,
        # Make all end-of-lines '\n'
        universal_newlines=True,
    )
    for line in unbuffered(proc):
        print line
 
example()

Automatically start tmux on ssh connection. Logout on detach.

This bit automatically opens tmux when ssh'ing into a server. If a session does not exist a new one is created. Otherwise, the last session is attached. This should always be run last either in .bashrc or as a script in .bashrc.d. The user is logged out from the server on detach.
# This should always be run last either in .bashrc or as a script in .bashrc.d
if [[ -z "$TMUX" ]]; then
    tmux has-session &> /dev/null
    if [ $? -eq 1 ]; then
      exec tmux new
      exit
    else
      exec tmux attach
      exit
    fi
fi