I am new to & trying to use spinnaker for the client I am working with. I am somewhat familiar with spinnaker architecture.
I know FRONT50 micro-service is responsible for this task. I am not sure how I can safely backup the pipeline data and restore into a new instance.
I want that to be able to continuously back up these pipelines as they are being added so that when I happen to recreate the spinnaker instance(i.e destroy my the infra and then recreate from scratch) I am able to restore these.
I am currently using Azure as the cloud provider and using Azure Container service.
I found this page here : https://www.spinnaker.io/setup/install/backups/
but does not indicate if the pipelines will also be backed up.
Many thanks in advance
I am not sure about the standard method but you can copy the configurations for pipelines and applicatons from front50 manually.
For pipelines, just do a curl to http://<front50-IP>:8080/pipelines
curl http://<front50-IP>:8080/pipelines -o pipelines.json
For applications config:
curl http://<front50-IP>:8080/v2/applications -o applications.json
To push pipeline config to Spinnaker, you can do:
cat pipelines.json | curl -d#- -X POST \
--header "Content-Type: application/json" --header \
"Accept: /" http://<Front50_URL>:8080/pipelines
P.S: My Spinnaker version is 1.8.1 and both, v1 and v2, k8s providers are supported.
Update-2: If you are using AWS S3 or GCS, you can back up the buckets directly.
Related
I am new to running containers in Azure and I am puzzled with the use case below,
If you feel I am wrongly using containers, feel free to give me an alternative approach.
Core problem: I am not able/ don't know how to create a container instance in the "stopped" state either via command line or ARM template deployment
Long read use case:
I created a docker image that runs a python job.
The job needs to run daily and is triggered via a data factory. The data factory will figure out the environment, set up the docker commands, update the container image and then execute the container via a batch account. The job itself does an api call and writes some data to sql. This part works fine, the container status goes to running and stops afterwards (I put auto-restart off)
In Azure DevOps, I have a pipeline that builds the image of the job and stores it in an azure repo. this works fine.
As I need a container instance as a resource in my resource group, I decided to put them in my infra ARM template. The problem: When deploying the container using DevOps / Arm template deployment,
It deploys and runs the job instance and this is not great, I would like to have the container created in a "stopped" state. The reason for this is that the job otherwise writes data to our database, and that is unwanted.
I am wondering what would be the best approach / what the best guidelines are, I thought about these two scenarios, but for both, my intuition says no.
scenario 1: have an all-time running container (let it execute bin/bash) and deliver the command using "az container exec".
Why I don't want to do this: I currently have an image per environment that has the exact job for that environment and I don't see the usefulness of having 3 standby containers running all the time to be triggered once a day
scenario 2: Instead of handling the container instance creation via DevOps, ignore the problem and create it using data-factory and the batch account. This implies that, when the job is triggered, It will create (and therefore run the container). Subsequently, I could delete it after usage.
Why I don't want to do this: I see a container instance as part of the infrastructure (as it is something you deploy inside a resource group, correct me if my point of view is wrong) So in that sense, managing resources via a scheduled data factory job doesn't look good and is a kind of hack to overcome the problem that you cannot deploy a container instance in a stopped state.
# set base image (host OS)
FROM python:3.7-buster
# Argument for environment selection
ARG environment
ENV environment ${environment}
# set the working directory in the container
WORKDIR /
# copy the dependencies file to the working directory
COPY requirements.txt .
# install FreeTDS and dependencies
RUN apt-get update && apt-get -y install apt-transport-https curl
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN curl https://packages.microsoft.com/config/debian/10/prod.list > /etc/apt/sources.list.d/mssql-release.list
RUN exit
RUN apt-get update
RUN ACCEPT_EULA=Y apt-get -y install msodbcsql17
RUN apt-get install unixodbc-dev
RUN pip install pyodbc
RUN pip install -r requirements.txt
# copy the content of the local src directory to the working directory
COPY /src .
# command to run on container start
CMD python "./my_data_job.py" "./my_config.ini" ${environment}
For Azure Container Instance, the container group will always be in the running state until you stop it. But containers inside it can be in the terminated state. In fact, if your image is a one-time image, then the container will be in the terminated state when the job is finished. You can use the CLI command az container exec to run the job again as you need.
So it's impossible to create an ACI in the stopped state. Maybe you can use the AKS, create different deployments for different environments, and when you need a container to run the job, then scale up to one replica. When you don't need the container, you can scale down to zero.
I am using Sematext to monitor a small composition of Docker containers plus the Logsene feature to gather the web traffic logs from one container running Node Express web application.
It all works fine until I update and restart the web server container to pull in a new code build. At this point, Sematext Logsene seems to get detached from the container, and so I lose the HTTP log trail in the monitoring. I still see the Docker events, so it seems only the logs part which is broken.
I am running Sematext "manually" (i.e. it's not in my Docker Compose) like this:
sudo docker run -d --name sematext-agent --restart=always -e SPM_TOKEN=$SPM_TOKEN \
-e LOGSENE_TOKEN=$LOGSENE_TOKEN -v /:/rootfs:ro -v /var/run/docker.sock:/var/run/docker.sock \
sematext/sematext-agent-docker
And I update my application simply like this:
docker-compose pull web && docker-compose up -d
where web is the web application service name (amongst database, memcached etc)
which recreates the web container and restarts it.
At this point Sematext stops forwarding HTTP logs.
To fix it I can restart Sematext agent like this:
docker restart sematext-agent
And the HTTP logs start arriving in their dashboard again.
So, I know I could just append the agent restart command to my release script, but I am wondering if there's a way to prevent it from becoming detached in the first place? I guess it's something to do with it monitoring the run files.
I have searched their documentation and FAQs, but not found anything specific about this effect.
I seem to have fixed it, but not in the way I'd expected.
While looking through the documentation I found the sematext-agent-docker package with the Logsene integration built-in has been deprecated and replaced by two separate packages.
"This image is deprecated.
Please use sematext/agent for monitoring and sematext/logagent for log collection."
https://hub.docker.com/r/sematext/sematext-agent-docker/
You now have to use both Logagent https://sematext.com/docs/logagent/installation-docker/ and a new Sematext Agent https://sematext.com/docs/agents/sematext-agent/containers/installation/
With these both installed, I did a quick test by pulling a new container image, and it seems that the logs still arrive in their web dashboard.
So perhaps the problem was specific to the previous package, and this new agent can "follow" the container rebuilds better somehow.
So my new commands are (just copied from the documentation, but I'm using env-vars for the keys)
docker run -d --name st-logagent --restart=always \
-v /var/run/docker.sock:/var/run/docker.sock \
-e LOGS_TOKEN=$SEMATEXT_LOGS_TOKEN \
-e REGION=US \
sematext/logagent:latest
docker run -d --restart always --privileged -P --name st-agent \
-v /:/hostfs:ro \
-v /sys/:/hostfs/sys:ro \
-v /var/run/:/var/run/ \
-v /sys/kernel/debug:/sys/kernel/debug \
-v /etc/passwd:/etc/passwd:ro \
-v /etc/group:/etc/group:ro \
-e INFRA_TOKEN=$SEMATEXT_INFRA_TOKEN \
-e CONTAINER_TOKEN=$SEMATEXT_CONTAINER_TOKEN \
-e REGION=US \
sematext/agent:latest
Where
CONTAINER_TOKEN == old SPM_TOKEN
LOGS_TOKEN == old LOGSENE_TOKEN
INFRA_TOKEN = new to me
I will see if this works in the long run (not just the quick test).
I have been working on the deployment of windows container from Azure Container Registry to Azure Container Service with Kubernetes Orchestra it was working fine previously.
Now I'm trying to create an acs kubernetes cluster of windows but the create command is only creating a master node and while deploying I'm getting the following error No nodes are available that match all of the following predicates:: MatchNodeSelector (1)
I have followed this link https://learn.microsoft.com/en-us/azure/container-service/kubernetes/container-service-kubernetes-windows-walkthrough to create the windows based kubernetes cluster.
This is the command I have used to create the cluster
az acs create --orchestrator-type=kubernetes \
--resource-group myResourceGroup \
--name=myK8sCluster \
--agent-count=2 \
--generate-ssh-keys \
--windows --admin-username azureuser \
--admin-password myPassword12
As per the above documentation, the above command should create a cluster named myK8sCluster with one Linux master node and two Windows agent nodes.
To verify the creation of cluster I have used the below command
kubectl get nodes
NAME STATUS AGE VERSION
k8s-master-98dc3136-0 Ready 5m v1.7.7
According to the above command, it shows that it created only the Linux master node, not the two windows agent nodes.
But in my case I require the windows agent nodes to deploy a windows based container in the cluster.
So I assume that due this I'm getting the following error while deploying No nodes are available that match all of the following predicates:: MatchNodeSelector (1)
As the documentation points out, ACS with a target of Kubernetes is deprecated. You want to use AKS (Azure Kubernetes as a Service).
To go about this, start here: https://learn.microsoft.com/en-us/azure/aks/windows-container-cli
Make sure you have the latest version of the CLI installed on your machine if you choose to do it locally, or use the Azure Cloud Shell.
Follow the guide on the rest of the steps as it will walk you through the commands.
For your issue, as I know the possible reason is that you need to enable the WindowsPreview feather. You can have a check through the CLI command like this:
az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/WindowsPreview')].{Name:name,State:properties.state}"
When it's OK, you also need to pay attention to the Kubernetes version. When I use the command that you have used, then the windows nodes are created successfully, but it just shows the master when I execute the command kubectl get nodes. Even if I can see the windows node in the group.
Then I try the command with additional parameter --orchestrator-version and set the value as 1.12.7 and the whole command like below:
az acs create --orchestrator-type=kubernetes \
--resource-group myResourceGroup \
--name=myK8sCluster \
--agent-count=2 \
--generate-ssh-keys \
--windows --admin-username azureuser \
--admin-password myPassword12 \
--orchestrator-version \
--location westcentralus
Then it works well and the command kubectl get nodes -o wide show like below:
But as you know, the ACS will be deprecated. So I would suggest you use the AKS with Windows node in the preview version. Or you can use the aks-engine as the AKS Engine is the next version of the ACS-Engine project.
I recently wanted to move a Gitlab runner that I had set up for my self-hosted Gitlab instance from being a project runner (i.e. running jobs only for a project) to being a group runner (so it could also run jobs for other projects in the same group). I wanted to retain the /etc/gitlab-runner/config.toml settings that I had painstakingly hand-written.
Luckily I backed up config.toml, because sudo gitlab-runner unregister -t ... -u ... deleted the whole configuration from config.toml.
In order to get the same config registered under the group instead of the project, I had to:
Register the runner in a paused state with a dummy configuration, with the group's registration token:
sudo gitlab-runner register \
--non-interactive \
--url <URL HERE>
--registration-token <TOKEN HERE> \
--executor docker \
--docker-image docker:dind \
--paused
Go into the new config.toml that this created and copy the runner's individual runner token.
Overwrite config.toml with my desired configuration.
Edit the config.toml and plug in the new individual runner token.
Start the Gitlab runner service (sudo systemctl start gitlab-runner).
Unpause the runner in the Gitlab web UI.
Even after doing all this, the Gitlab instance still sees the runner under the name it registered with in the dummy config, rather than the name in the config.toml.
Trying the --config option to gitlab-runner register didn't work at all; I think that just tells it where to save the config. It still prompted me for new settings to use instead of reading from the config.toml I pointed it at.
The Gitlab documentation on runner registration is all written around one shot gitlab-runner register commands with loads of options on them that essentially specify the whole config on the command line. I really don't want to translate my config.toml manually into a command line that turns around and rebuilds it (minus any comments, of course).
I can't believe that this is really the right workflow to re-register a runner with a new project/group/Gitlab instance, or to create a copy of a runner from a saved config. What am I missing here? How can I create a new Gitlab runner from an existing config.toml file?
There isn't an easy way to do what you want, from what I can find in the GitLab documentation and some open issues that they have.
Here is an issue that describes something similar to what you want:
https://gitlab.com/gitlab-org/gitlab-runner/issues/3540
Here is what I think is GitLab's goal with how to register runners:
https://gitlab.com/gitlab-org/gitlab-ce/issues/40693
I believe that the only thing that you can't change from the .toml file is the name of the runner, and maybe not the tags either. Then name is only created when you register the runner. I read something somewhere that you can change the tags of a shared runner, but I can't find it now.
Here is a workaround to make the process of registering a bit more automatic:
https://gitlab.com/gitlab-org/gitlab-runner/issues/3553#note_108527430
He used this API:
curl --request POST "https://gitlab.com/api/v4/runners" --form "token=<registration-token>" --form "description=test-1-20150125-test" --form "tag_list=ruby,mysql,tag1,tag2"
Then he got the following response back:
{"id":401513,"token":"<runner-token>"}
He could then inject the runner-token into his already pre-made .toml file.
For you, it would have been possible to use the registration token for your group, and then to write in the description/name of the runner and the tags. You could then have re-used your config.toml and only changed the runner-token, and it should have worked.
A gitlab runner can be registered with multiple projects and/or groups. This will just append the configurations in /etc/gitlab-runner/config.toml (with sudo). Can we just do the following steps:
Un-register the gitlab-runner associated with the "project"
Register the the gitlab-runner associated with the "group"
config.toml stores all the configuration which is passed to gitlab-runner register including any environment variables which are listed under gitlab-runner register -h command.
I am not sure why do you need to save the config.toml.
Also, I believe one source of confusion could be gitlab-runner-token VS gitlab-runner-registration-token. The registration-token can NOT be used inside config.toml which may be the reason why you failed after just a replacement. If you do not want to use gitlab-runner register command and just update the config.toml then follow the steps defined in the above ans to fetch the gitlab-runner-token and use it in config.toml. We can then try and stop and start the gitlab-runner service using sudo service gitlab-runner stop and sudo service gitlab-runner start
We store runner configurations in a repository for recovery.
To restore a runner we:
install gitlab-runner (see https://docs.gitlab.com/runner/install/) on the new node,
move the stored configuration to /etc/gitlab-runner/config.toml, and
restart the runner service, e.g. sudo service gitlab-runner restart on ubuntu.
So far, this procedure was very reliable.
In our application, Heritrix is being used as the crawl engine and once the crawl job is finished, we are manually kicking off an endpoint to download the PDFs from a website. We would like to automate this downloading pdf task as soon as the crawl job is complete. Does HEritrix provide any URI/webservice method - which returns the status of the job? (or) Do we need to create a polling app to continuously monitor the status of the job?
I don't know if there is any option to do it without continious monitoring but you can use Heritrix API to get status for a job, smth like
curl -v -d "action=" -k -u admin:admin --anyauth --location -H "Accept: application/xml" https://localhost:8443/engine/job/myjob
gives you XML from where you can read job status.
Another, maybe easier (yet not so 'professional') option is to check if your jobs warcs directory contains a file with .open extension. If not - the job is finished.