Persistent use of Jupyter Notebook from remote server

Question

I connect to a remote server using ssh -L but if I close the laptop lid or the connection is lost, the jupyter notebook is disconnected.

After I reconnect to the remote server, the "last" session is lost.

What can be done to make it persistent? Could screen help with it?

In practice, long running notebooks fail. My suggestion is that you convert the notebook to a script, then run it using screen as @MichaelD suggests. I know this doesn't answer the OP's question, but it will solve their problem — shallow_water, Jan 16 '19 at 03:11

score 36 · Accepted Answer · edited Nov 03 '21 at 10:42

36

On the remote server, you should open your jupyter in a screen session, it will make it persistent if you lose the connection to the server and resume it.

On your computer: ssh -L xxxx:localhost:yyyy server.
screen.
jupyter notebook --no-browser --port=yyyy. [on remote server]
In your browser: localhost:xxxx.

To disconnect manually and reconnect:

Exit the screen window: control + a and then d.
Disconnect from the server: control + d
And reconnect ssh -L xxxx:localhost:yyyy.
Optionally, you can reopen the screen window, though unnecessary, using screen -r.
Go back to your notebook or reopen localhost:xxxx.

edited Nov 03 '21 at 10:42

Michael D

1,711
4
23
38

answered Aug 23 '17 at 09:38

BiBi

7,418
5
43
69

1

To reconnect, you just have to reconnect to the remote server using `ssh -L xxxx:localhost:yyyy server`. If you want to access the screen where the jupyter is being run, you can do `screen -r`. Note that reconnecting to the remote server is sufficient, you don't have to reopen the screen. – BiBi Aug 23 '17 at 11:33
I use tmux for persistent running of programs. – Jingpeng Wu Dec 05 '18 at 14:54
1

what is `xxxx` and `yyyy`. Can you explain about the ports to assign?` – maninekkalapudi Feb 03 '19 at 07:56
1

Basically, you should use an unused port on the local machine (`xxxx`) and the remote machine (`yyyy`). For instance, `xxxx` could be `8888` and `yyyy` could be also `8888`. – BiBi Feb 03 '19 at 10:07

desertnaut · Answer 2 · 2017-09-13T11:03:44.050

11

The standard usage for persisting Jupyter server sessions is the use of nohup and &; in your remote server with IP address xx.xx.xx.xx:

nohup jupyter notebook --no-browser --ip xx.xx.xx.xx --port yyyy &

Now, even if you switch off your laptop or lose the connection, you will be always able to reconnect by pointing your browser at xx.xx.xx.xx:yyyy

edited Sep 13 '17 at 11:03

answered Aug 29 '17 at 14:11

desertnaut

57,590
26
140
166

how do chose the `xx.xx.xx.xx`? – Michael D Sep 13 '17 at 07:48
@MichaelD this should be the IP address of your server machine – desertnaut Sep 13 '17 at 08:04
I tried it it didn't work for me. (I run `nohup ...` on local machine) – Michael D Nov 27 '17 at 08:42
It didn't worked me. Maybe because server, is in local network... (start with 10.xx) – Michael D Nov 27 '17 at 12:18
@MichaelD shouldn't be an issue - mine, too is in a local net (starts with 192.xx..); did you use `ifconfig` in the server to get its IP? – desertnaut Nov 27 '17 at 12:51

score 1 · Answer 3 · edited Sep 22 '20 at 10:09

Adding to @BiBi's answer...

Instead of screen I could recommend you to take a look at tmux. Especially, if you combine tmux with the Tmux Plugin Manager and install Tmux Resurrect, even after reboots of your remote server you will be able to go back to your previous Tmux sessions.

Shortcuts for tmux are somewhat equal to those of screens, just that control + a is replaced by control + b. Of course, tmux allows you to configure your custom shortcuts.

jns · Answer 4 · 2021-06-28T16:09:08.003

BiBi's answer is correct. But I had cases where my ssh connection terminated unexpectedly and the port forwarding no longer worked when trying to reconnect. Probably there was some dangling process on the remote machine, not sure.

Anyway, in these cases I used socat to proxy between two local ports on the remote machine:

# jupyter notebook/lab running in screen on port yyyy, then your connection dies...
ssh -L xxxx:localhost:zzzz
socat tcp-listen:zzzz,reuseaddr,fork tcp:localhost:yyyy

This way you can avoid restarting jupyter on a different port

Jayakrishnan · Answer 5 · 2022-10-25T09:54:37.307

Use the nohup command to keep jupyter running even after exiting the shell or terminal. Type the following command in the specified locations.

In remote server nohup jupyter notebook --no-browser --port=8085 > my.log 2>&1 < /dev/null &. This runs jupyter in port 8085 and any stdout would be present in my.log
In local ssh -NL 8085:localhost:8085 username@xx.xx.xx.xx. If port needs to be specified, you can use ssh -NL 8085:localhost:8085 -p xxxx username@xx.xx.xx.xx
In browser http://127.0.0.1:8085/

Sometimes port 8085 may be occupied in the remote server, in such cases try it with another port but make sure you use the same port number in the local while tunneling.

Persistent use of Jupyter Notebook from remote server

5 Answers5

Linked