How to Access IPython Notebook on a remote server ?

My re­search work in­volves a lot of us­ing of IPython Notebook. I usu­ally do it on an of­fice MAC. However I also very of­ten need to ac­cess it from home. After a brief search­ing, I found these three won­der­ful ar­ti­cles on this topic.

• Remote host­ing of IPython Notebooks
• Accessing IPython Notebook re­motely over an SSH tun­nel
• Remote Access to IPython Notebooks via SSH

I have been do­ing this for a while. But it even­tu­ally comes to me that how good it is if I can make it au­to­matic. So I wrote this python script to do the pro­ce­dures de­scribed in those three ar­ti­cles. I am sure there must be some more el­e­gant way to do this. But this is what I got so far and it works.

import paramiko
import sys
import subprocess
import socket
import argparse

# function to get available port
def get_free_port():
      s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
      s.bind(('localhost',0))
      s.listen(1)
      port = s.getsockname()[1]
      s.close()
      return port

# print out the output from paramiko SSH connection
def print_output(output):
    for line in output:
        print(line)

parser = argparse.ArgumentParser(description='Locally open IPython Notebook on remote server\n')
parser.add_argument('-t', '--terminate', dest='terminate', action='store_true', \
                    help='terminate the IPython notebook on remote server')
args = parser.parse_args()

host="***" # host name
user="***" # username

# write a temporary python script to upload to server to execute
# this python script will get available port number

def temp():
    with open('free_port_tmp.py', 'w') as f:
        f.write('import socket\nimport sys\n')
        f.write('def get_free_port():\n')
        f.write('    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)\n')
        f.write("    s.bind(('localhost', 0))\n")
        f.write('    s.listen(1)\n')
        f.write('    port = s.getsockname()[1]\n')
        f.write('    s.close()\n')
        f.write('    return port\n')
        f.write("sys.stdout.write('{}'.format(get_free_port()))\n")
        f.write('sys.stdout.flush()\n')

def connect():
    # create SSH client
    client = paramiko.SSHClient()
    client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    client.load_system_host_keys()
    client.connect(host, username=user)

    # generate the temp file and upload to server
    temp()
    ftpClient = client.open_sftp()
    ftpClient.put('free_port_tmp.py', "/tmp/free_port_tmp.py")

    # execute python script on remote server to get available port id
    stdin, stdout, stderr = client.exec_command("python /tmp/free_port_tmp.py")
    stderr_lines = stderr.readlines()
    print_output(stderr_lines)

    port_remote = int(stdout.readlines()[0])
    print('REMOTE IPYTHON NOTEBOOK FORWARDING PORT: {}\n'.format(port_remote))

    ipython_remote_command = "source ~/.zshrc;tmux \
                              new-session -d -s remote_ipython_session 'ipython notebook \
                              --no-browser --port={}'".format(port_remote)

    stdin, stdout, stderr = client.exec_command(ipython_remote_command)
    stderr_lines = stderr.readlines()

    if len(stderr_lines) != 0:
        if 'duplicate session: remote_ipython_session' in stderr_lines[0]:
            print("ERROR: \"duplicate session: remote_ipython_session already exists\"\n")
            sys.exit(0)

    print_output(stderr_lines)

    # delete the temp files on local machine and server
    subprocess.run('rm -rf free_port_tmp.py', shell=True)
    client.exec_command('rm -rf /tmp/free_port_tmp.py')

    client.close()

    port_local = int(get_free_port())
    print('LOCAL SSH TUNNELING PORT: {}\n'.format(port_local))

    ipython_local_command = "ssh -N -f -L localhost:{}:localhost:{} \
                            username@server".format(port_local, port_remote)

    subprocess.run(ipython_local_command, shell=True)


def close():
    # create SSH client
    client = paramiko.SSHClient()
    client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    client.load_system_host_keys()
    client.connect(host, username=user)
    stdin, stdout, stderr = client.exec_command("source ~/.zshrc;tmux kill-session -t remote_ipython_session")
    stderr_lines = stderr.readlines()
    if len(stderr_lines) == 0:
        print('Successfully terminate the IPython notebook\n')
    else:
        print_output(stderr_lines)
    client.close()

if args.terminate:
    close()
else:
    connect()

This script does the fol­low­ing:

1. Connect to the server us­ing python pack­age paramiko.
2. Upload a tem­po­rary python script. Use paramiko to ex­e­cute the python script. This script gets an avail­able port on lo­cal­host.
3. Open Ipython Notebook us­ing the port we get from the last step. I used tmux to do this. And my shell is zsh. You can mod­ify that part of code based on your sit­u­a­tion
4. On the lo­cal ma­chine, find an avail­able port and cre­ate an SSH tun­nel­ing to port for­ward­ing the port on the re­mote ma­chine to lo­cal ma­chine.

If the script runs suc­cess­fully, you will see some­thing like this.

If you want to check does IPython Notebook re­ally runs on the re­mote ma­chine. Use com­mand tmux ls. A tmux ses­sion named remote_ipython_session should ex­ist.

In browser, open http://localhost: 50979. You should be able to ac­cess your ipython note­book. To ter­mi­nate the ipython note­book on the re­mote ma­chines, sim­ply do