I have setup Puppet Enterprise server on CentOS Linux 7 (Core).
# /opt/puppetlabs/bin/puppetserver --version
puppetserver version: 2016.5.0.11
I have different client-specific projects and they all have multiple environments (QA, Staging, Prod). I will be using my Puppet server for managing all these projects. I know Puppet provides environments for isolating code but i'm not sure how to implement the kind of design i am thinking of. This is what i would like to have:
puppet_root/
- code/
- environments
- prod/
- project_A
- environment.conf
- hieradata/
- manifests/
- modules/
- project_B
- environment.conf
- hieradata/
- manifests/
- modules/
- staging
- project_A
- environment.conf
- hieradata/
- manifests/
- modules/
- project_B
- environment.conf
- hieradata/
- manifests/
- modules/
- dev
...
...
- modules
- custom_scripts/
Had there been a single client, i know it would have worked without any issue but with multiple clients inside a sub-dir (as shown above), i'm not sure how Puppet will come to know which path to traverse.
UPDATE:
New model i came up with:
puppet_root/
- client_A/
- code/
- modules/
- environments/
- prod/
- environment.conf
- manifests/
- hieradata/
- common.yaml
...
...
- uat/
...
...
- client_B/
- code/
- modules/
- environments/
- prod/
- environment.conf
- manifests/
- hieradata/
- common.yaml
...
...
- uat/
...
...
- custom_scripts/
Any help will be appreciated.
Related
Does anyone know if it's possible to breakdown the config file for circleci into smaller files where each job, command, workflow, etc, is in it's own specific file/subdirectory, and if so, how would you approach this?
I've been looking around and even attempted myself to build a python script to build a config from all these yaml files, but with no luck due to reference variable names not existing in these various files so pyyaml library won't load them.
What I'm trying to accomplish is to have this folder structure
configs/
dependencies.yml
commands/
command_1.yml
command_2.yml
jobs/
job_1.yml
job_2.yml
workflows/
workflow_1.yml
workflow_2.yml
Where dependencies.yml contains a breakdown of what each workflow requires in terms of what is used in each step > job > command. And this file would be hand written.
You can do the following :
Split your config.yml in a structure defined in Packing a config
Use dynamic configuration where you fist generate the config from step 1 and the call the generated config file from them main file
Example original config.yml to split:
version: 2
orbs:
sonarcloud: sonarsource/sonarcloud#1.0.3
jobs
my-job:
docker:
- image: cimg/latest
steps:
- checkout
- run: make
workflows:
build:
jobs:
-my-job
Create following layout in a new folder called config (run tree):
.
├── config.yml
└── config
├── #orbs.yml
├── jobs
│ └──my-job.yml
└── #workflows.yml
#orbs.yml contains
version: 2
orbs:
sonarcloud: sonarsource/sonarcloud#1.0.3
#workflows.yml contains
workflows:
build:
jobs:
-my-job
my-job.yml contains
docker:
- image: cimg/latest
steps:
- checkout
- run: make
And the main config.yml should look like:
version: 2.1
setup: true
orbs:
continuation: circleci/continuation#0.3.1
jobs:
generate-and-run-circleci:
docker:
- image: 'circleci/circleci-cli:latest'
steps:
- circleci-cli/install
- checkout
- run:
command : |
cd .circleci
circleci config pack config > generated.yml
- continuation/continue:
configuration_path: .circleci/generated.yml
workflows:
build:
jobs:
- generate-and-run-circleci
how we can exclude files or folders not being include in gitlab code quality scanning json file.
actually when we run code quality, we are getting some quality issues but those are not relates to issues. those are related to some files like karma.conf.js and etc.. I need to exclude those files not being included in code quality artifact json file. So we can reduce the code quality issues within the report.
Please suggest how we can exclude files from json file.
gitlab.yaml:
stages:
- build
- test
- deploy
include:
- template: Code-Quality.gitlab-ci.yml
code_quality:
stage: test
tags:
- linux
- dind
artifacts:
paths: [gl-code-quality-report.json]
exclude:
- karma.conf.js
You can add a .codeclimate.yml file at the root of your project to enable or disable plugins and exclude files or directories from scanning. Here's the example in the docs:
---
version: "2"
plugins:
csslint:
enabled: true
coffeelint:
enabled: true
duplication:
enabled: true
config:
languages:
- ruby
- javascript
- python
- php
eslint:
enabled: true
channel: __ESLINT_CHANNEL__
fixme:
enabled: true
rubocop:
enabled: true
exclude_patterns:
- config/
- db/
- dist/
- features/
- "**/node_modules/"
- script/
- "**/spec/"
- "**/test/"
- "**/tests/"
- Tests/
- "**/vendor/"
- "**/*_test.go"
- "**/*.d.ts"
- "**/*.min.js"
- "**/*.min.css"
- "**/__tests__/"
- "**/__mocks__/"
The exclude_patterns at the bottom is what you're looking for. This should tell Code Climate not to run its tests on these files or directories.
I am running ubuntu 20.04 on windows 10 machine using wsl2. I am setting up a hyperledger fabric 2.2 environment using the Minifabric tool. I am specifying the structure of my custom network in the spec.yaml file in the working directory:
fabric:
cas:
- "ca1.university.com"
- "ca2.university1.com"
- "ca3.university2.com"
- "ca4.org.com"
- "ca5.org.com"
peers:
- "peer1.university1.com"
- "peer2.university1.com"
- "peer3.university1.com"
- "peer4.university1.com"
- "peer5.university1.com"
- "peer6.university1.com"
- "peer7.university1.com"
- "peer8.university1.com"
- "peer9.university1.com"
- "peer1.university2.com"
- "peer2.university2.com"
- "peer3.university2.com"
- "peer4.university2.com"
- "peer5.university2.com"
- "peer6.university2.com"
- "peer7.university2.com"
- "peer8.university2.com"
- "peer1.organization_x.com"
- "peer2.organization_x.com"
- "peer3.organization_x.com"
- "peer1.organization_y.com"
- "peer2.organization_y.com"
- "peer3.organization_y.com"
- "peer1.organization_z.com"
- "peer2.organization_z.com"
- "peer3.organization_z.com"
orderers:
- "orderer1.university1.com"
- "orderer2.university1.com"
- "orderer3.university1.com"
- "orderer1.university2.com"
- "orderer2.university2.com"
- "orderer3.university2.com"
- "orderer1.organization_x.com"
- "orderer1.organization_y.com"
- "orderer1.organization_z.com"
settings:
ca:
FABRIC_LOGGING_SPEC: DEBUG
peer:
FABRIC_LOGGING_SPEC: DEBUG
orderer:
FABRIC_LOGGING_SPEC: DEBUG
I gave the command specifying the default organization "./minifab up -o university1.com" and got the following error:
Can someone explain why I am getting this error? TIA.
It may just be an artifact of copying and pasting, but the syntax of the lists in your config file don't look right...
I recommend letting yamllint run over it and taking its advice.
I was trying to install a Lets Encrypt ssl certificate for my website on an Amazon EC2 Linux AMI Server.
I followed the steps in the documentation:
Tutorial: Configure SSL/TLS on Amazon Linux
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/SSL-on-amazon-linux-ami.html
After ssh'ing in and running these kinds of commands to generate the certificate
sudo yum install -y mod24_ssl
sudo chmod 600 custom.key
sudo openssl req -new -key custom.key -out csr.pem
Amazon kinds leaves you hanging at Step 2 => stage 4.
"Submit the CSR to a CA."
Without any CA's reccommended I decided to go with letsencrypt because the certbot for an EC2 Linux 2 is usually quite reliable.
I then got this error
The client lacks sufficient authorization :: Account creation on ACMEv1 is disabled. Please upgrade your ACME client to a version that supports ACMEv2 / RFC 8555. See https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430 for details .
TIP: To find out if your Bitnami stack uses Apache or NGINX, check the output of the command sudo /opt/bitnami/ctlscript.sh status.
NGINX:
So, turns out there is an end of life plan for ACMEv1 which is Lets Encrypts original method for generating certificates.
Read the article here:
https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430
The original protocol used by Let’s Encrypt for certificate issuance and management is called ACMEv1. In March of 2018 we introduced support for ACMEv2, a newer version of the protocol that matches what was finalized today as RFC 8555 864. We have been encouraging subscribers to move to the ACMEv2 protocol.Today we are announcing an end of life plan for ACMEv1.
Here is my entire process to install certbot-auto. I;m sharing my whole list of commands as you may find you have the same issues as me. Depends on what kinds of packages you have installed. ( this was also tricky due to Amazon running their own version of Linux on the EC2 AMI)
tried installing certbot-auto
wget https://dl.eff.org/certbot-auto
I then tried change permissions to make certbot-auto work
sudo chown root /usr/local/bin/certbot-auto
sudo chmod 0755 /usr/local/bin/certbot-auto
tried installing again
wget -N https://dl.eff.org/certbot-auto.asc
Had to Install Epel because of AWS Linux
sudo amazon-linux-extras install epel
Then ran
sudo yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
Then
sudo yum install certbot-apache
Then tried
sudo yum install mod_ssl python-certbot-apache
chmod +x certbot-auto
sudo mv certbot-auto /usr/local/bin/certbot-auto
Then ran the command
certbot-auto certonly --standalone -d yourwebsite.com --debug
This is where I reached the error:
To fix my issue I had to do the following steps:
Run this command (from inside etc/letsencrypt ) with your Amazon DNS IP:
sudo ./certbot-auto --debug -v --server https://acme-v01.api.letsencrypt.org/directory certonly -d ec2-00-000-000-000.eu-west-2.compute.amazonaws.com
DNS IPs look like this:
ec2-00-000-000-000.eu-west-2.compute.amazonaws.com
so just replace the example above with your own ( found in your AWS console => EC2)
You should see this:
How would you like to authenticate with the ACME CA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: Apache Web Server plugin (apache)
2: Spin up a temporary webserver (standalone)
3: Place files in webroot directory (webroot)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
I Chose 3
This is where I reached the error again:
The client lacks sufficient authorization :: Account creation on ACMEv1 is disabled. Please upgrade your ACME client to a version that supports ACMEv2 / RFC 8555. See https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430 for details.
The saw this:
Enter email address (used for urgent renewal and security notices)
(Enter 'c' to cancel):
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must
agree in order to register with the ACME server at
https://acme-v02.api.letsencrypt.org/directory
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(A)gree/(C)ancel:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Would you be willing to share your email address with the Electronic Frontier
Foundation, a founding partner of the Let's Encrypt project and the non-profit
organization that develops Certbot? We'd like to send you email about our work
encrypting the web, EFF news, campaigns, and ways to support digital freedom.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o:
I added my domains as example.com and www.example.com
No names were found in your configuration files. Please enter in your domain
name(s) (comma and/or space separated) (Enter 'c' to cancel): example.com , www.example.com
Saw this
Performing the following challenges:
http-01 challenge for example.com
Then this
PluginError: Unable to find a virtual host listening on port 80 which is currently needed for Certbot to prove to the CA that you control your domain. Please add a virtual host for port 80.
So had to do this:
Check you have ghosts running
apachectl -t -D DUMP_VHOSTS
So Looked-up what configs I might have
find /etc/httpd -name *.conf
Then edit your configs with vim or nano or whatever you prefer
//VIM
vi /etc/httpd/conf/httpd.conf
//NANO
nano /etc/httpd/conf/httpd.conf
Copy paste your virtual host into your config (3 places to change with your website)
#Virtual Host added for Letsencrypt
<VirtualHost *:80>
DocumentRoot "/var/www/html"
ServerName "example.com"
ServerAlias "example"
RewriteEngine on
RewriteCond %{SERVER_NAME} =example
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>
//Then just :wq to save (if your in vim)
Ran this again (from inside etc/letsencrypt)
sudo ./certbot-auto --debug
** Saw this**
Which names would you like to activate HTTPS for?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: example.com
- - - - - - - - - - - - - - - - - -
Selected 1...
and it worked !
Hope that helps
APACHE BITNAMI APPLICATION:
https://docs.bitnami.com/general/how-to/generate-install-lets-encrypt-ssl/#alternative-approach
When I take a mergeRequest in GitLab, there is a compile error with description:
CI build failed for 4a0b9b43 , view Details：.gitlab-ci.yml not
found in this commit.
But I have sync with the destination branch and compile succeed on native, how can I fix this issue?
You must place the .gitlab-ci.yml file to the root directory of a project first. If you want to cross-reference other CI configuration locally or from somewhere (GitLab instance required to install on remote locations), you must use the include tag
How-to
Please check your GL instance version first before you are using include. Compare your instance version with the minimum version required from the documentation. If you need to upgrade, back up everything and run an update. It should take hours before you can go back into business.
To cross-reference CI config files locally, use the include:local. Make sure they are on the same branch. If that is on other branch, use ref.
include:
- local: '/templates/.gitlab-ci-template.yml'
In case you want to cross-reference CI config files somewhere within the instance, use
include:
- project: 'my-group/my-project'
file: '/templates/.gitlab-ci-template.yml'
# You can also specify ref, with the default being the head of the object.
- project: 'my-group/my-project'
ref: master # Git branch
file: '/templates/.gitlab-ci-template.yml'
- project: 'my-group/my-project'
ref: v1.0.0 # Git tag
file: '/templates/.gitlab-ci-template.yml'
- project: 'my-group/my-project'
ref: 787123b47f14b552955ca2786bc9542ae66fee5b # Git SHA
file: '/templates/.gitlab-ci-template.yml'
If you prefer to use CI config templates shipped with the instance, use include:template. Check the GitLab's template collection for more details.
include:
- template: Auto-DevOps.gitlab-ci.yml
If deployed outside the instance borders, like GitLab.com, use include:remote.
include:
- remote: 'https://gitlab.com/awesome-project/raw/master/.gitlab-ci-template.yml'
Examples
The first example was derived from GitLab's CI config file for GitLab EE (gitlab-org/gitlab).
image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.6.3-golang-1.12-git-2.24-lfs-2.9-chrome-73.0-node-12.x-yarn-1.16-postgresql-9.6-graphicsmagick-1.3.33"
stages:
- sync
- prepare
- quick-test
- test
- post-test
- review-prepare
- review
- qa
- post-qa
- notification
- pages
variables:
RAILS_ENV: "test"
NODE_ENV: "test"
SIMPLECOV: "true"
GIT_DEPTH: "20"
GIT_SUBMODULE_STRATEGY: "none"
GET_SOURCES_ATTEMPTS: "3"
KNAPSACK_RSPEC_SUITE_REPORT_PATH: knapsack/report-master.json
FLAKY_RSPEC_SUITE_REPORT_PATH: rspec_flaky/report-suite.json
BUILD_ASSETS_IMAGE: "false"
ES_JAVA_OPTS: "-Xms256m -Xmx256m"
ELASTIC_URL: "http://elastic:changeme#elasticsearch:9200"
after_script:
- date
include:
- local: .gitlab/ci/cache-repo.gitlab-ci.yml
- local: .gitlab/ci/cng.gitlab-ci.yml
- local: .gitlab/ci/docs.gitlab-ci.yml
- local: .gitlab/ci/frontend.gitlab-ci.yml
- local: .gitlab/ci/global.gitlab-ci.yml
- local: .gitlab/ci/memory.gitlab-ci.yml
- local: .gitlab/ci/notifications.gitlab-ci.yml
- local: .gitlab/ci/pages.gitlab-ci.yml
- local: .gitlab/ci/qa.gitlab-ci.yml
- local: .gitlab/ci/reports.gitlab-ci.yml
- local: .gitlab/ci/rails.gitlab-ci.yml
- local: .gitlab/ci/review.gitlab-ci.yml
- local: .gitlab/ci/setup.gitlab-ci.yml
- local: .gitlab/ci/dev-fixtures.gitlab-ci.yml
- local: .gitlab/ci/test-metadata.gitlab-ci.yml
- local: .gitlab/ci/yaml.gitlab-ci.yml
- local: .gitlab/ci/releases.gitlab-ci.yml
You can see more examples on the .gitlab-ci.yml reference in the GitLab documentation.