How to run Python scripts without connection to instance active on computer - python-3.x

Question

I apologize if this is a stupid questions; I am a complete newbie when it comes to cloud computation.
I am using Google Compute instances to run python scripts with GPU support. Unfortunately, it seems that for the script to run, my computer has to be on and the terminal connecting me to my instance must be open.
I am wondering if there is any way to run python scripts on instances in Google Cloud completely remotely, and just SSH in to see when the script is finished.
I have considered using IPython notebooks or something similar, but that code I am running requires a very specific Anaconda environment, and is meant to be run via terminal.
Edit 1:
The reason I think I need to have the console connecting me to the instance is because I tried to test it out by writing a small script to make files every minute. My process was as follows:
1. Create an instance, SSH in through the google cloud Instances page
2. Create a new python script with this code:
import time
i=0
while 1:
tmp_file = open("tst"+str(i)+".txt","w")
tmp_file.write(str(i))
tmp_file.close()
i += 1
time.sleep(60)
I then ran this code, confirmed it worked by SSHing in with a different console.
I closed the console with the program running in it. After that, files stopped being created.
Ideally, I would like a situation where I could run such a script, close out of the terminal window and have the execution of the script be unassociated with things like whether I have the console open or whether my device is on. I would like to just be able to SSH in and see the result of a script once it is finished.

Answer 1

I'm also a total novice when it comes to GCE and Python, so you're in good company! I had a similar problem when learning to use GCE. I opted to use a start-up script but I am not sure how well this will fit in with the environment that you need to set up. Mine uses a bash boot script and looks like something like this:
#! /bin/bash
sudo apt-get update
sudo apt-get -yq install python-pip
sudo pip install --upgrade google-cloud
sudo pip install --upgrade google-cloud-storage
sudo pip install --upgrade google-api-python-client
sudo pip install --upgrade google-auth-httplib2
echo "Making directories..."
mkdir -p /home/<username>/rawdata
mkdir -p /home/<username>/processeddata
mkdir -p /home/<username>/input
mkdir -p /home/<username>/code
mkdir -p /home/<username>/and so on
echo "Setting ownership..."
sudo chown -R <username> /home/<username>
echo "Directory creation complete..."
gsutil cp gs://<bucket with code>/* /home/<username>/code/
echo "Boot complete."
nohup python /home/<username>/code/workermaster.py &
The gist of the code (in case it isn't self-explanatory!) is that the instance installs various packages to support the code, but some of these might be on GCE instances by default. It then sets up required directories and copies all required code from a storage bucket and sets ownership. The key line I guess is the one containing "ho hangup" that starts the python script.
My "workermaster" script gets tasks from a Storage bucket and puts the output in one of the folders on the instance. I can see what the instance is doing from the console by looking at the logs (so without SSH-ing into the instance). My instances also copy the output to an output storage bucket. If I SSH into the instance I cannot see the script running, I can just see files 'mysteriously' appearing in the output folder.
There are plenty of experts on here that might be able to post a solution that more specific to your needs, but something like the above has worked for me so far. Good luck!

Answer 2

Not sure why you are saying you have to keep the terminal connected to your compute instance. Some more details will be helpful. Are you manually SSHing into your instance thru terminal and running the script? Is that how you want to do it in future?
If you are running your script periodically, you can set it up as cron job.
You may also want to look at Cloud Functions to go serverless.

Answer 3

You can use programs like tmux.
# ssh to the system
ssh user#system-blah-blah
# start a new tmux session
tmux new -s my_remote_session
# detach from session (keyboard shortcut, in sequence)
<ctrl-b> d
# attach to it back
tmux a -t my_remote_session

Answer 4

To be able to let a script running and close the terminal window, you can use a screen session, this is going to let the script running (inside the screen session) and if you close the terminal, it is going to continue working, after that, you can open again the terminal and connect to the screen session to see the results.
Other option is to use ansible, it helps to run commands inside the VM without connecting to it, but you must create a SSH-key in order to be able to connect with ansible.