I have this git command at Bitbucket pipeline script which will generate list of changed files with path.
export FILES=$(git diff-tree --no-commit-id --name-only -r HEAD^^..HEAD)
problem is that it will generate on all commint. How to get list files only merge commit to master?
Or how to say in pipelipe script to run on merge event only?
After some experiments I extended my script like this:
image: python:3.5.7
pipelines:
branches:
master:
- step:
script:
- apt-get update
- apt-get -qq install zip curl
- mkdir $BITBUCKET_REPO_SLUG
- export VERSION_LABEL=$(date +%Y-%m-%d_%H:%M:%S)
- export ZIP_FILE=package_$BITBUCKET_REPO_SLUG_$VERSION_LABEL.zip
- export FILES=$(git diff-tree --no-commit-id --name-only -r HEAD^^..HEAD)
- echo "Repo name is $BITBUCKET_REPO_SLUG & version is $VERSION_LABEL"
- cp -R --parents $FILES $BITBUCKET_REPO_SLUG/ 2>/dev/null
- rm -f $BITBUCKET_REPO_SLUG/bitbucket-pipelines.yml
- rm -f $BITBUCKET_REPO_SLUG/.gitignore
- zip -r $ZIP_FILE $BITBUCKET_REPO_SLUG/
And now it is executed when I make merge into master but it do nothing and raw report is:
+ cp -R --parents $FILES $BITBUCKET_REPO_SLUG/ 2>/dev/null
Searching for test report files in directories named [test-results, failsafe-reports, test-reports, TestResults, surefire-reports] down to a depth of 4
Finished scanning for test reports. Found 0 test report files.
Merged test suites, total number tests is 0, with 0 failures and 0 errors.
Do I need change
HEAD^^..HEAD
into other parameter?
Related
The following code was from my deploy stage in my .gitlab-ci.yml file.
deploy_website:
stage: deploy
artifacts:
paths:
- public
before_script:
- "command -v ssh-agent >/dev/null || ( apk add --update openssh )"
- eval $(ssh-agent -s)
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add -
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- pwd && ls
- ssh-keyscan $VM_IPADDRESS >> ~/.ssh/known_hosts
- chmod 644 ~/.ssh/known_hosts
script:
# - apk add bash
# - ls deploy
# - bash ./deploy/deploy.sh
- ssh $SSH_USER#$VM_IPADDRESS "hostname && echo 'Welcome!!!' > welcome.txt"
This line "ssh-keyscan $VM_IPADDRESS >> ~/.ssh/known_hosts" failed to run when I execute my pipeline. Please help :(
You can start and echo $VM_IPADDRESS to check if the IP variable is properly instanciated.
"failed to run"
Then it depends on the error message (or if the commands simply froze).
Before the keyscan, you can test if the network route is reachable from your GitLab-CI runnner with:
curl -v telnet://$VM_IPADDRESS:22
If it does not connect immediately, that would explain why the ssh-keyscan fails.
This is my script:
image: python:3.5.7
pipelines:
default:
- step:
script:
- apt-get update
- apt-get -qq install zip curl
- mkdir $BITBUCKET_REPO_SLUG
- export VERSION_LABEL=$(date +%Y-%m-%d_%H:%M:%S)
- export ZIP_FILE=update_$BITBUCKET_REPO_SLUG_$VERSION_LABEL.zip
- export FILES=$(git diff-tree --no-commit-id --name-only -r HEAD^^..HEAD)
- echo "Repo name is $BITBUCKET_REPO_SLUG & version is $VERSION_LABEL"
- echo $FILES
- cp -R $FILES $BITBUCKET_REPO_SLUG/
- rm -f $BITBUCKET_REPO_SLUG/bitbucket-pipelines.yml
- rm -f $BITBUCKET_REPO_SLUG/.gitignore
- zip -r $ZIP_FILE $BITBUCKET_REPO_SLUG/
Why all files in zip are in root and not in directories as I see then when I echo them?
What is the problem?
I don't exactly understand your question but my first impression is it can be a relative path/full path situation. Bitbucket pipelines are using a standard directory something line /opt/atlassian/pipelines/agent/build to pull the code. This variable can be extracted by using BITBUCKET_CLONE_DIR built-in variable, Maybe you can try to combine this variable with your relative path to create your directory. something like mkdir -p $BITBUCKET_CLONE_DIR/$BITBUCKET_REPO_SLUG can be useful.
Reference : https://support.atlassian.com/bitbucket-cloud/docs/variables-and-secrets/
I fixed my problem with copy command to this:
cp -R --parents $FILES $BITBUCKET_REPO_SLUG/
The Problem: I am trying to concat two variables for a copy cmd in a before script for a gitlab ci/cd pipeline job.
What I expect: myfile_filesuffix
What I get: _filesuffix
Can anyone see what I am doing wrong? When I run this for loop on my local CLI I have no problems. Thank you!
before_script:
- rm -rf .terraform
- terraform --version
- mkdir ~/.aws
- echo "[default]" > ~/.aws/credentials
- echo "aws_access_key_id=$AWS_ACCESS_KEY_ID" >> ~/.aws/credentials
- echo "aws_secret_access_key=$AWS_SECRET_ACCESS_KEY" >> ~/.aws/credentials
- mkdir ./deployments
- ls common
- common_files=$(find common -type f)
- echo $common_files
- prefix_common=$(echo $common_files | cut -d"/" -f 1)
- echo $prefix_common
- for f in $common_files;
do
common_file="$(basename $f)"
cp $f ./deployments/""${common_file}"_"${prefix_common}"";
done
you can used GitLab repo settings -> CI/CD -> Variables to add FILE type variable and use mv command move to your folder.
ex: File Type variable is ANSIBLE_VARIABLES_FILE
script:
- mv $ANSIBLE_VARIABLES_FILE ./deployments/variables_common.tf
I am trying to copy files as part of GitLab pipeline but I am getting
busybox v1.22.1 multicall binary error on copy command. It was working earlier but suddently showing this error.
Here is my script
script:
- mkdir -p ./input
- git log -m -2 --name-only --diff-filter=d --pretty="format:" >
./input/changes.lst
- |
file="./input/changes.lst"
while IFS= read -r line
do
printf '%s\n' "$line";
if [ -e $line ]
then
`cp -R $line ./input/`;
fi
done < "$file"
only:
- master
artifacts:
when: always
paths:
- input
I have a list of release archives:
MyProject-0.9.zip
MyProject-1.0.zip
MyProject-1.3.tar.gz
MyProject-2.0.tar.gz
Each one contains a folder with the same name as the archive (without the file extension), which contains sources and makefiles and so on, like this:
MyProject-0.9.zip
- MyProject-0.9/
- MyProject-0.9/src/
- MyProject-0.9/src/mySrc.c
- MyProject-0.9/Makefile
- MyProject-0.9/LICENSE
I want to convert them into a git repo, one commit per archive.
This is a solution using bash.
uses the release archives modification date as author-date of the commits
adds a release/version git tag to each commit
supports different archive formats
allows for easy cleanup of archive contents before committing
the script:
#!/bin/bash
# author: hoijui <hoijui.quaero#gmail.com>
# copyright: Copyright (c) 2017 hoijui
# license GNU General Public License version 3
# description: Creates a git repository from a list of release archives.
# example: Before using the script, we should have this directory structure:
# * conv_dir/createGitRepoFromReleaseArchives # this script
# * conv_dir/MyProject-1.0.0.zip # 1. release archive
# * conv_dir/MyProject-1.0.1.zip # 2. release archive
# * conv_dir/MyProject-2.0.0.tar.gz # 3. release archive
# * conv_dir/MyProject-2.1.0.tar.gz # 4. release archive
# while each of the archives should contain a dir with the same name
# as the archive (without the file extension), for example:
# MyProject-1.0.0.zip:
# * MyProject-1.0.0/LICENSE
# * MyProject-1.0.0/Makefile
# * MyProject-1.0.0/src/mySrc.c
# We then simply run the script:
# > createGitRepoFromReleaseArchives
# after which we should end up with a dir "repo" containing the git repository.
ORIG_DIR=`pwd`
ARCHIVE_SUFFIX_REGEX="\.\(tar\|tar\.gz\|tgz\|zip\|tar\.bz2\|tbz2\)"
ZIP_SUFFIX_REGEX=".*\.zip$"
ARCHIVES="${1:-}"
REPO_DIR="${2:-repo}"
EXTRACT_DIR="/tmp/$(basename ${0})_extract_${RANDOM}"
if [ -z "${ARCHIVES}" ]
then
# if ARCHIVES is not set
# gather common archive files from the current directory
ARCHIVES="$(find . -regex ".*${ARCHIVE_SUFFIX_REGEX}" | sort --version-sort)"
fi
# Possibly remove files fond in the release archives
# that we do not want in the git repo.
# This gets called for each archive,
# with PWD set to the extracted archives main dir.
function cleanRepoDir() {
echo "cleaning repo dir ..."
#rm version.txt
# ... more commands
}
PROJECT_NAME="$(basename "${ARCHIVES%% *}" | sed -e 's/-.*//')"
mkdir "${EXTRACT_DIR}"
echo ""
echo "project name: '${PROJECT_NAME}'"
echo "extract dir: '${EXTRACT_DIR}'"
echo "repo dir: '${REPO_DIR}'"
echo -e "version archives:\n${ARCHIVES}"
echo ""
read -p "Do you want to continue creating a git repo with the above info! [Y/n]" -n 1 -r
echo
if [[ ${REPLY} =~ ^[Nn]$ ]]
then
# handle exits from shell(-script) or function
# but do not exit interactive shell
[[ "$0" = "${BASH_SOURCE}" ]] && exit 1 || return 1
fi
# setup the resulting repo
if [ -e "${REPO_DIR}" ]
then
>&2 echo "Error: Repo already exists!"
exit 1
fi
mkdir "${REPO_DIR}"
cd "${REPO_DIR}"
git init
#git config --local user.name "My Name"
#git config --local user.email mail#mail.com
cd ..
for archive in ${ARCHIVES}
do
echo
# get the absolute path to the current archive
archive="$(realpath "${archive}")"
# remove extracted contents of the last archive
cd "${EXTRACT_DIR}"
rm -Rf ./*
# extract contents of the current archive
if [[ "${archive}" =~ ${ZIP_SUFFIX_REGEX} ]]
then
echo "Extracting (unzip): '${archive}'"
unzip -q "${archive}"
else
echo "Extracting (tar): '${archive}'"
tar -xf "${archive}"
fi
# and create the absolute path to the contained directory
# (which should be "${PROJECT_NAME}-${version}")
version_dir="${EXTRACT_DIR}/$(basename "${archive}" | sed -e "s/${ARCHIVE_SUFFIX_REGEX}$//")"
#version_dir="${EXTRACT_DIR}/$(basename "${archive}" | sed -r -e 's/\.zip$//')"
#version_dir="${EXTRACT_DIR}/$(basename "${archive}" | sed -r -e 's/\.(zip|zap)$//')"
# remove directory and project name from extracted archive path
# to get the bare version string
version="$(basename "${version_dir}" | sed -e 's/^[^-]*-//')"
# remove all content from the previous commit from git
# for the next comit
cd "${ORIG_DIR}"
cd "${REPO_DIR}"
git rm -rfq . 2> /dev/null
# also remove empty directories
rm -Rf ./*
# move current versions content to the git repo
mv "${version_dir}"/* ./
cleanRepoDir
# and add all of it for the next commit
git add .
# extract last modification date of the release archive file,
# which is to be used as commit date
date="$(date -r "${archive}" +'%Y-%m-%dT%H:%M:%S')"
# commit the current archives content
git commit -m "${PROJECT_NAME} release version ${version}" --date "${date}" --quiet
# and tag it as a release
git tag -m "${PROJECT_NAME} release version ${version}" "v${version}"
cd ..
done
# Compress the git history
cd "${REPO_DIR}"
git gc --aggressive
echo "Git history size: $(du -s --si .git)"
cd "${ORIG_DIR}"