Install jupyter on AWS

Intro

Because our EC2 instance does not release port other than SSH, also we need to access via jump server, so we need to do below setup if we want to access jupyter hosted on the EC2.

Steps

Prerequisite : Assume you already setup .ssh/config according to here
Bonus: it is a good idea to create a separate user(e.g. jupyter) to dedicatedly run below service

Install Anaconda

It is recommended to install anaconda which already included jupyter.
SSH to your EC2 host
Go to anaconda and find the anaconda installer corresponding to your EC2 architecture
ps: you can run uname -m to check the architecture
- Copy the download link, run below command and download to your EC2 (current work path) For example, I used this package
```
wget <https://repo.anaconda.com/archive/Anaconda3-2021.11-Linux-x86_64.sh>
```
Install the package by running
```
bash Anaconda3-2021.11-Linux-x86_64.sh
```

After installation finished, check the python path is under conda:

which python
## showing something like: /home/ubuntu/anaconda3/bin/python

Logout your current shell terminal, and relogin again -Reason: becasue it needs to restart your shell to activate the conda env
- (If still not already in conda environment) Enter the env now
```
conda activate
```
Now you will see something like (base) front of your command line name. e.g.
```
(base) keith@i-xxxxxxxx:~$
```

Generate a password with sha512 algorithm

Start python in interactive mode
```
ipython
```
Type in below line by line, to encrypt your new password
```
from IPython.lib import passwd

passwd()
```
COPY the password generated, which will be used in later step
Type exit() to exit the interactive mode

Configure Jupyter Notebook

Create the config by
```
jupyter notebook --generate-config
```
The generated config probably is here: ~/.jupyter/jupyter_notebook_config.py
Create a directory as your jupyter workspace, e.g. I am using ~/notebook here
```
cd ~/
mkdir notebook
```
Edit the config (by vim jupyter_notebook_config.py, and press i to enter edit mode)

And add below to the top

c.NotebookApp.ip = 'localhost'
c.NotebookApp.notebook_dir = '< Put the workspace FULL path here (e.g. /home/keith/notebook/) >'
c.NotebookApp.open_browser = False
c.NotebookApp.port = 8888
c.NotebookApp.password_required = True
c.NotebookApp.password = u'< Put the encrypted password we generated from above >'

Remember to put your owned work space full path, and the encrypted password
Save the file by clicking esc then type :wq

Start the Jupyter

Run jupyter notebook to start the service, you should see the message like this:

........
[I 06:42:59.781 NotebookApp] Serving notebooks from local directory: /home/keith/notebook
[I 06:42:59.781 NotebookApp] Jupyter Notebook 6.4.5 is running at:
[I 06:42:59.781 NotebookApp] <http://localhost:8888/>
[I 06:42:59.781 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).

Setup SSH tunnel

Open another terminal in your PC locally (not in EC2)

Run below command

ssh -v  -J <username of jump server>@<jump server>:<jump server port> \\
    -N <EC2 user name>@<EC2 host> \\
    -L 8888:localhost:8888

By using v in above command, we can see actually what the server is doing.
You should see some message about forwarding the port 8888
PS: you need to keep this terminal opened OR put this as background process

Open jupyter web !

Finally, you can access the notebook in your local PC in localhost:8888
It should prompt you to enter the password you setup in previous steps

(Optional) Recommened!!! Install Anaconda extension for Notebook

Original Jupyter only use your global python kernal to execute the notebooks. But most of the time we do not want to ruin the global environment, or avoid version conflict. Here we will install a few extensions so you can easily manage environments(conda) on jupyter notebook

Login to EC2, with the user account running jupyter
Follow the guide to install nb_conda
- PS: it should install Notebook Conda Kernels together
- Or just run below code
```
conda install nb_conda
```
After installed, switch back to main user & restart the jupytrer service
```
sudo systemctl restart <your service name>
```
Open your jupyter webpage, now you can see a new tab called Conda. So you can create / manage environment. It is recommended to clone the root env because new env lacks of important package
Once you created some environment, you can select a specific environment in a notebook:

(Optional) Start jupyter automatically as service

Configure the service

Once you confirmed the above setup is working well, you can configure it to startup automatically when server boot.

Ensure you are now loged in to ec2, with the user that is supposed to run jupyter
1st, print out your current $PATH and copy it for later use
```
echo $PATH
```
Login to an user account that can run sudo
Create service config, you can change the name jupyter.service to anything else xxxxxx.service
```
sudo touch /lib/systemd/system/jupyter.service
```

Edit the service file by running sudo vim /lib/systemd/system/jupyter.service, paste below to the configure file

Reminder: press i to enter edit mode

[Unit]
Description=Jupyter Notebook Server

[Service]
Type=simple
PIDFile=/run/jupyter.pid

Environment="PATH=< Put the output of `echo $PATH` you run before >"

# Jupyter Notebook: change PATHs as needed for your system
# you can check the full path by running `which jupyter`
ExecStart=/home/jupyter/anaconda3/bin/jupyter notebook

# Change the below info to your user detail
User=jupyter
Group=jupyter
WorkingDirectory=/home/jupyter
Restart=always
RestartSec=10
#KillMode=mixed

[Install]
WantedBy=multi-user.target

Save the config file (esc -> :wq)

Setup the service from config

Assuming the service name is jupyter

Run below command to set it as auto-start when server boot

sudo systemctl daemon-reload
sudo systemctl enable jupyter

Start the service (or just do a server reboot)
```
sudo systemctl start ipython-notebook
```
You can check the service status by:
```
sudo service jupyter status
```