This is my script:
image: python:3.5.7
pipelines:
default:
- step:
script:
- apt-get update
- apt-get -qq install zip curl
- mkdir $BITBUCKET_REPO_SLUG
- export VERSION_LABEL=$(date +%Y-%m-%d_%H:%M:%S)
- export ZIP_FILE=update_$BITBUCKET_REPO_SLUG_$VERSION_LABEL.zip
- export FILES=$(git diff-tree --no-commit-id --name-only -r HEAD^^..HEAD)
- echo "Repo name is $BITBUCKET_REPO_SLUG & version is $VERSION_LABEL"
- echo $FILES
- cp -R $FILES $BITBUCKET_REPO_SLUG/
- rm -f $BITBUCKET_REPO_SLUG/bitbucket-pipelines.yml
- rm -f $BITBUCKET_REPO_SLUG/.gitignore
- zip -r $ZIP_FILE $BITBUCKET_REPO_SLUG/
Why all files in zip are in root and not in directories as I see then when I echo them?
What is the problem?
I don't exactly understand your question but my first impression is it can be a relative path/full path situation. Bitbucket pipelines are using a standard directory something line /opt/atlassian/pipelines/agent/build to pull the code. This variable can be extracted by using BITBUCKET_CLONE_DIR built-in variable, Maybe you can try to combine this variable with your relative path to create your directory. something like mkdir -p $BITBUCKET_CLONE_DIR/$BITBUCKET_REPO_SLUG can be useful.
Reference : https://support.atlassian.com/bitbucket-cloud/docs/variables-and-secrets/
I fixed my problem with copy command to this:
cp -R --parents $FILES $BITBUCKET_REPO_SLUG/
Related
I have this git command at Bitbucket pipeline script which will generate list of changed files with path.
export FILES=$(git diff-tree --no-commit-id --name-only -r HEAD^^..HEAD)
problem is that it will generate on all commint. How to get list files only merge commit to master?
Or how to say in pipelipe script to run on merge event only?
After some experiments I extended my script like this:
image: python:3.5.7
pipelines:
branches:
master:
- step:
script:
- apt-get update
- apt-get -qq install zip curl
- mkdir $BITBUCKET_REPO_SLUG
- export VERSION_LABEL=$(date +%Y-%m-%d_%H:%M:%S)
- export ZIP_FILE=package_$BITBUCKET_REPO_SLUG_$VERSION_LABEL.zip
- export FILES=$(git diff-tree --no-commit-id --name-only -r HEAD^^..HEAD)
- echo "Repo name is $BITBUCKET_REPO_SLUG & version is $VERSION_LABEL"
- cp -R --parents $FILES $BITBUCKET_REPO_SLUG/ 2>/dev/null
- rm -f $BITBUCKET_REPO_SLUG/bitbucket-pipelines.yml
- rm -f $BITBUCKET_REPO_SLUG/.gitignore
- zip -r $ZIP_FILE $BITBUCKET_REPO_SLUG/
And now it is executed when I make merge into master but it do nothing and raw report is:
+ cp -R --parents $FILES $BITBUCKET_REPO_SLUG/ 2>/dev/null
Searching for test report files in directories named [test-results, failsafe-reports, test-reports, TestResults, surefire-reports] down to a depth of 4
Finished scanning for test reports. Found 0 test report files.
Merged test suites, total number tests is 0, with 0 failures and 0 errors.
Do I need change
HEAD^^..HEAD
into other parameter?
The Problem: I am trying to concat two variables for a copy cmd in a before script for a gitlab ci/cd pipeline job.
What I expect: myfile_filesuffix
What I get: _filesuffix
Can anyone see what I am doing wrong? When I run this for loop on my local CLI I have no problems. Thank you!
before_script:
- rm -rf .terraform
- terraform --version
- mkdir ~/.aws
- echo "[default]" > ~/.aws/credentials
- echo "aws_access_key_id=$AWS_ACCESS_KEY_ID" >> ~/.aws/credentials
- echo "aws_secret_access_key=$AWS_SECRET_ACCESS_KEY" >> ~/.aws/credentials
- mkdir ./deployments
- ls common
- common_files=$(find common -type f)
- echo $common_files
- prefix_common=$(echo $common_files | cut -d"/" -f 1)
- echo $prefix_common
- for f in $common_files;
do
common_file="$(basename $f)"
cp $f ./deployments/""${common_file}"_"${prefix_common}"";
done
you can used GitLab repo settings -> CI/CD -> Variables to add FILE type variable and use mv command move to your folder.
ex: File Type variable is ANSIBLE_VARIABLES_FILE
script:
- mv $ANSIBLE_VARIABLES_FILE ./deployments/variables_common.tf
I need to copy the folder with multiple files to another folder in build pipeline.
I use
cp -R -v pathToSourceFolder pathToDestFolder
cp -R -v /Users/runner/runners/2.166.4/work/1/s/en.lproj/ /Users/runner/runners/2.166.4/work/1/s/platforms/ios/AppName/Resources
and I am getting error with exit code 126:
usage: cp [-R [-H | -L | -P]] [-fi | -n] [-apvXc] source_file target_file
cp [-R [-H | -L | -P]] [-fi | -n] [-apvXc] source_file ... target_directory
can anyone help with this as I am new to linux/macOS uses/cmd?
Note: pipeline run on macOS.
Since there is nothing obviously wrong in the cp command, I would just write some safety checks about the presence of the directories:
# Define directories involved
from="pathToSourceFolder"
to="pathToDestFolder"
# Check existence
if [[ -d "$from" ]]
then
# If the destination directory does not exist, we create it.
[[ -d $to ]] || mkdir -p "$to"
if [[ -d $to ]]
then
cp -R -v "$from" "$to"
else
# If we get here, we will likely have permission problems.
echo Can not create directory $to
ls -ld "$to"
fi
else
echo Source directory $from does not exist
fi
I need to copy the folder with multiple files to another folder in
build pipeline.
1.I assume /Users/runner/runners/2.166.4/work/1/s is the default working folder of your build. So please avoid hard-coding the path, instead you can use $(System.DefaultWorkingDirectory) which represents that path. See Azure Devops predefined variables.
2.Since your original purpose is to copy the files in Azure Devops pipeline, you don't need to care too much about the corresponding copy syntax in different OS systems (Linux,MacOS or Windows).
You can do what you want easily using official Copy Files task. This task requires three inputs: Source folder, Target folder and Contents we want to copy, that's it.
Classic UI format:
You can choose the source folder via Browse Source Folder option. And then use ** as Contents, $(System.DefaultWorkingDirectory)/platforms/ios/Fixi/Resources as Target folder.
Yaml format:
- task: CopyFiles#2
displayName: 'My Copy Task'
inputs:
SourceFolder: en.lproj
TargetFolder: '$(System.DefaultWorkingDirectory)/platforms/ios/Fixi/Resources'
We've done the logic in the behind for you so that you can use this task in MacOS/Linux/Windows easily. Log of my test:
I know the accepted answer says the "cp" command is not broken, but it actually is in azure pipelines. Microsoft seems to append a "/" at the end by default, which breaks the behaviour of the command.
if you do:
cp -r ./src ./dest
it will copy the files from src folder to dest folder.
if you write:
cp -r ./src ./dest/
it will copy the src folder into the destination one leaving you with /dest/src/*
Also, my build pipeline failed when I tried to copy a file
cp ./src/myfile.txt ./dest/myfile.txt
Adding the "/" at the end will cause it to fail as it's attempting to dump the file into a directory that does not exist since the actual command that runs is the following.
cp ./src/myfile.txt ./dest/myfile.txt/
Currently, In my gitlab configuration workflow, i have some manual stage tests. So I can decide if the test pass or fail. Now the manual steps are always skipped by default. Whenever the normal stage steps are building it jump to another normal stage without considering manual steps. Now, How i can make it work. Please help me in this.
stages:
- start_pipeline
- auto_testing
- manual_test_PASS
- manual_test_FAIL
- UAT_test_PASS
- UAT_test_FAIL
- Validation_PASS
- Validation_FAIL
- merge_to_master
variables:
start_pipeline:
stage: start_pipeline
script:
- if [[ -d "$USER_DIR" ]]; then echo -e "Direcory exists"; else sudo mkdir -p $USER_DIR; fi
- sudo chown -R root:gitlab-runner ${TARGET}/*
auto_testing:
stage: auto_testing
script:
- find . -type d -name "manifests" -exec chown -R gitlab-runner:gitlab-runner {} \;
- find . -type d -name "manifests" -exec puppet parser validate {} \;
- if [[ -d "$PRODUCTION_TARGET" ]]; then echo -e "Direcory exists"; else sudo mkdir -p $PRODUCTION_TARGET; fi
- if [[ -d "$LAB_TARGET" ]]; then echo -e "Direcory exists"; else sudo mkdir -p $LAB_TARGET; fi
manual_test_FAIL:
stage: manual_test_FAIL
script:
- echo "FAIL"
- exit 1;
when: manual
manual_test_PASS:
stage: manual_test_PASS
script:
- echo "PASS"
- sudo cp -r * ${TARGET}/${MODIFIED_COMMIT_USER}/
- sudo cp -r * ${LAB_TARGET}/
- sudo cp -r * ${PRODUCTION_TARGET}/
dependencies:
- auto_testing
I know this is likely too late to help you, but this is a known issue that they're targeting to fix in V9.0.
https://gitlab.com/gitlab-org/gitlab-ce/issues/26360
I have this makefile tha sthould download and build openssh (along with other things):
ROOT_DIR=$(PWD)
DATA_DIR=$(ROOT_DIR)/data
SOURCES_DIR=$(ROOT_DIR)/sources
RESOURCES_DIR=$(ROOT_DIR)/resources
DRAFTS_DIR=$(ROOT_DIR)/drafts
$(SOURCES_DIR):
mkdir $(SOURCES_DIR)
$(RESOURCES_DIR):
mkdir $(RESOURCES_DIR)
$(DRAFTS_DIR):
mkdir $(DRAFTS_DIR)
openssh-tar-url="ftp://ftp.cc.uoc.gr/mirrors/OpenBSD/OpenSSH/portable/openssh-6.2p2.tar.gz"
TAR_PROJECTS += openssh
openssh:
echo "Building $#"
openssh-clean: openssh-archive-clean
.SECONDEXPANSION :
$(TAR_PROJECTS) : $(SOURCES_DIR) $(SOURCES_DIR)/$$#-archive
$(DRAFTS_DIR)/%.tar.gz: $(DRAFTS_DIR)
echo "Pulling $*."
wget $($*-tar-url) -O $(DRAFTS_DIR)/$*.tar.gz
.SECONDEXPANSION :
$(SOURCES_DIR)/%-archive : | $(DRAFTS_DIR)/$$*.tar.gz
mkdir $#
cd $# && tar xvzf $(DRAFTS_DIR)/$*.tar.gz
%-archive-clean:
rm -rf $(SOURCES_DIR)/$*-archive $(DRAFTS_DIR)/$*.tar.gz
When i run make openssh it runs correctly but at the end it removes the archive it downloaded. This is very strange to me:
$ make openssh --just-print
echo "Pulling openssh."
wget "ftp://ftp.cc.uoc.gr/mirrors/OpenBSD/OpenSSH/portable/openssh-6.2p2.tar.gz" -O /home/fakedrake/Projects/ThinkSilicon/xilinx-zynq-bootstrap/drafts/openssh.tar.gz
mkdir /home/fakedrake/Projects/ThinkSilicon/xilinx-zynq-bootstrap/sources/openssh-archive
cd /home/fakedrake/Projects/ThinkSilicon/xilinx-zynq-bootstrap/sources/openssh-archive && tar xvzf /home/fakedrake/Projects/ThinkSilicon/xilinx-zynq-bootstrap/drafts/openssh.tar.gz
echo "Building openssh"
rm /home/fakedrake/Projects/ThinkSilicon/xilinx-zynq-bootstrap/drafts/openssh.tar.gz
Pretty sure you can list targets (and intermediates) as .PRECIOUS to avoid them being deleted for you. I'm afraid you'll need to RTFM for more details - I'm in visual studio rather than make these days, so my make skills are a bit rusty...