How can I create, deploy and run and manage Cron jobs on Hasura?
Hasura suggests two ways to deploy and run Cron jobs.
Cron microservice
Hasura already has a microservice to run Cron jobs.
If you already have a Hasura project run:
hasura microservice create mycron --template=python-cron
Change mycron to whatever you want to name your microservice. This will create a custom Python microservice designed to run Cron jobs. (Follow further instructions as prompted by hasura CLI)
To deploy this on Hasura, git commit and push to your cluster's remote.
$ git add .
$ git commit -m "Add cron job"
$ git push hasura master
To know more about how to customize this microservice, you can read the docs.
Kubernetes Cron jobs
Since, Hasura runs on Kubernetes and Kubernetes (>= v1.8) already provides Cron Jobs as a first class resource, it is recommended to use Kubernetes Cron jobs wherever possible.
If you have kubectl installed you can check your Kubernetes version by running: kubectl version. In the output, the "server version" shows the version of the Kubernetes cluster. If you are running Kubernetes >= v1.8, we recommend you to use Kubernetes Cron jobs.
When using Kubernetes Cron jobs, you can version control your cron job specs inside your Hasura project, and use the kubectl tool to create and manage them.
The upcoming Hasura Release 1.3 will support "cron triggers". The beta is already released
Related
I have a Gitlab server from the company where the project and the pipeline are configured. By default, every time a commit is done, the pipeline starts to execute in the Gitlab server.
I have my personalized VM, which is completely different from Gitlab. I want that the pipeline will be executed in my personalized VM instead of the Gitlab server. What should I do so that the pipeline runs on the VM and not on the Gitlab server?
I have configured the following runner in config.toml that is located in $MYPROJECT/:
[[runners]]
name = "Project-name"
url = "https://gitlab.server/"
token = "TOKEN ID"
executor = "shell"
shell = "bash"
There are things that I don't understand.
If I want to execute the pipeline in my personalized VM, should I install Gitlab runner in the VM [1]?
Should I have the project source code in the VM so that it can read the config.toml file every time there is a commit?
If I register the runner with the token key in the Gitlab server, how the Gitlab server knows that the pipeline is to be executed in the VM and not in the server [2]?
Should I use the executor docker or shell, to execute the pipeline in the VM?
[1] https://docs.gitlab.com/runner/install/linux-manually.html
[2] https://docs.gitlab.com/runner/register/#registering-runners
For running a job on a machine you need a GitLab Runner installed on that machine, connected with the GitLab server.
The project source code is fetched automatically in front of every run
You can use a tag (e.g. "MyVM") when registering the runner. Then you can set the same tag into your job so that this job is only executed by this runner. See: https://docs.gitlab.com/ee/ci/runners/configure_runners.html#use-tags-to-control-which-jobs-a-runner-can-run
You need to use docker if you want to use docker in your VM (which needs to be installed before there). Otherwise use shell.
Gitlab:14.4.0
Gitlab-Runner:14.4.0 & 13.0.1
After I upgrade Gitlab from 13.6.3 to 14.4.0, all Runner could not pick up jobs
I thought it was Runner version too old，but Gitlab-Runner:14.4.0 on other Linux vm show the same problem
gitlab-runner status looks normal
But when I use gitlab-runner --debug run on each vm, Runner work for few minutes, the runner on that vm could pick up jobs, now I run this command manual every 5 minute to keep other developer working
the problem is we deploy gitlab use docker version image on hand wirte Deployment deploy on Kubernetes, the background jobs fail and stuck many times, after we transfer data to new Gitlb which deploy by helm problem solved
I had the same problem. But I figured it came from the GitLab instance iteself.
The problem was thensolved by upgrading to 14.4.2. I don't think it's the update itself, but the restart of the GitLab instance after the upgrade proved useful.
Ran into a similar issue (GitLab EE v14.2.3-ee) whereby all 3 runners where in a wait state for minutes at a time, before any pending jobs were eventually picked up. Resolved the issue by...
removing all runners,
resetting the config.toml file (which was cluttered with [[runner]] entries from former runners),
restarting GitLab, and
then creating new runners.
Specifically, I performed the following steps...
Delete all runners via "GitLab console > Admin > Overview > Runners".
Log into runner host(s), find config.toml, and clean out the entire file except for...
concurrent=5
check_interval=0
[session_server]
session_timeout = 1800
Log into GitLab EE host and restart GitLab EE.
sudo gitlab-ctl restart
Log into the runner host(s), and create new runners (follow directions from "GitLab console > Admin > Overview > Runners" )
After the above corrective action, the runners now immediately pick up pending jobs.
When I git push I have to go to the server the gitlab-runner is configured on and enter in the terminal "sudo gitlab-runner run" or "gitlab-runner run". and then the pipeline would start. And I have to be in the runtime-platform all the time. This defeats the point of a pipeline.
It used to work so that when I git pushed, the pipeline would start automatically, I didn't have to enter the command at all. So is there a way to set that up?
There are the following two methods to achieve that.
nohup gitlab-runner run & will keep your runner running as a user process and uses a config file that defaults to /home/<user>/.gitlab-runner/config.toml (see man nohup)
sudo systemctl start gitlab-runner will start as a service, and uses a config file that defaults to /etc/gitlab-runner/config.toml (see man systemctl)
In both cases, you will be able to logout and the runner will stay active.
The gitlab documentation is pretty clear :
GitLab offers a continuous integration service. For each commit or push to trigger your CI pipeline, you must:
Add a .gitlab-ci.yml file to your repository’s root directory.
Ensure your project is configured to use a Runner.
.gitlab-ci.yml part
You need to create a file named .gitlab-ci.yml in the root directory of your repository. The script part depends of what you want to do in the job.
Runner part
You need to install and configure the runner.
The most simple is to use shared runner
I have probably been Googling this the wrong way...
I have a Node express server.
I want to deploy it using Jenkins to ec2.
What are my options?
if I want to upload the code manually, I use ssh... but I want it to
be done from Jenkins.
Yes the server is on a git repo.
I would like a devops flow
I recommend you , to do it step by step :
Step 1 : Configure correctly a jenkins job ready to build your app in remote ec2 machine.
Install this plugin in your jenkins platform : Publish Over SSH Plugin
Using this plugin add new remote server under Publish over SSH section in Manage Jenkins >> Configure System option.
Now create some jenkins job. Then, in build section add a step called : Send files or execute commands over SSH
Just select your configured server and enter your commands in Exec command section :
For a simple node js express, this code could be enough or just copy-paste your existing code:
https://gist.github.com/jrichardsz/38b335f6a5dc8c67a386fd5fb3c6200e
That is all. Just test with "build" option and verify if your application goes well.
At this point, this job is functional. The flow could be :
push your changes to your git provider
login to jenkins and manually execute the created job (this step is replaced with webhook configuration)
Note: If and only if this step has no errors, you can start with the following step.
Step 2 : Implement a simple devops flow by configuring a webhook in your git provider, which automatically triggers the jenkins job (create in step 1) when you perform a git push.
This guide could be help you with the required configurations :
https://jrichardsz.github.io/devops/devops-with-git-and-jenkins-using-webhooks
You'll have to use AWS CodeDeploy jenkins plugin. This apply to any type of code. Node, java etc
See AWS article
Setting Up the Jenkins Plugin for AWS CodeDeploy
Jenkins Plugin
Github Link
Steps followed during rolling updates:
Create an image for the v2 version of the application with some changes
Re-Build a Docker Image with Maven. pom.xml. Run command in SSH or Cloud Shell:
docker build -t gcr.io/satworks-1/springio/gs-spring-boot-docker:v2 .
Push the new updated docker image to the Google Container Registry. Run command in SSH or Cloud Shell
gcloud docker -- push gcr.io/satworks-1/springio/gs-spring-boot-docker:v2
Apply a rolling update to the existing deployment with an image update. Run command in SSH or Cloud Shell
kubectl set image deployment/spring-boot-kube-deployment-port80 spring-boot-kube-deployment-port80=gcr.io/satworks-1/springio/gs-spring-boot-docker:v2
Revalidate the application again through curl or browser
curl 35.227.108.89
and observe the changes take effect.
When do we come across the "CrashLoopBackOff" error and how can we resolve this issue? Does it happen at application level or at kubernetes pods level?