I have an AWS codebuild project and a an svn repository which contains the java code I need to build. For my codebuild configuration, I chose No Source Type and am using a windows container. During the Install event I am downloading a tortoisesvn msi and installing tortoisesvn on my container. I then make a svn checkout call to my svn. It is a pretty large repo that I am checking out and at first, it continued to timeout on me with error svn: E175012: Connection timed out. I solved this by adding the option servers:global:http-timeout=2400 to lengthen the time for an svn checkout before the server times it out. However, now the checkout just stalls at a certain point in the checkout command output after some arbitrary amount of time. Eventually the timeout on the codebuild project times the project out. I have set svn timeout to 40 min as shown above and codebuild timeout to 1.5 hours but have had no luck in completing the checkout and building the code. Unfortunately, codebuild does not provide much visibility into the windows container used to build my code so I am unsure what to try next. Any advice would be appreciated. Below is the relevant portion of my buildspec.
TORTOISE_URL: 'https://mirrors.tuna.tsinghua.edu.cn/osdn/storage/g/t/to/tortoisesvn/1.13.1/Application/TortoiseSVN-1.13.1.28686-x64-svn-1.13.0.msi'
TORTOISE_INSTALL: 'C:\TortoiseSVN-1.13.1.28686-x64-svn-1.13.0.msi'
install:
commands:
- '[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; Invoke-WebRequest $env:TORTOISE_URL -OutFile $env:TORTOISE_INSTALL'
- msiexec.exe /i $env:TORTOISE_INSTALL /passive /norestart ADDLOCAL=ALL
- $env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine")
- '& $env:TORTOISE_HOME/bin/svn checkout --username $env:SVN_USERNAME --password $env:SVN_PASSWORD $env:SVN_REPO/$env:SVN_TAG --config-option servers:global:http-timeout=2400'
Related
What is the right sintaxis for running git clone command on an Azure function powershell 7.0?
Im trying with: git clone <repo https>
and also adding the path at the end git clone <repo https> /.
i get the following error message:
`Command 'starter.cmd git clone https ...' was aborted due to no output nor CPU activity for 60 seconds. You can increase the SCM_COMMAND_IDLE_TIMEOUT app setting (or WEBJOBS_IDLE_TIMEOUT if this is a WebJob) if needed.`
*Update:
Also tried with git clone https://username:password#github.com/username/repository.git
Cloning into ''...
fatal: unable to access 'https://github.com//.git/': URL using bad/illegal format or missing URL
I have'nt found any documenation on this... is it even possible?
I'd first ask what you're trying to achieve by doing this as it's a bit of an odd scenario and there may be a better way of cloning a repo as part of a pipeline and publishing the content with your Function.
If you really must go this route I would look at downloading the repo/branch content with Invoke-RestMethod/Invoke-WebRequest instead and unpacking to the underlying directory structure.
Invoke-WebRequest -Uri 'https://github.com/$owner/$repo/archive/$branch.zip' -OutFile '$repo.zip'
or using the .NET approach like Matthias pointed out in this response:
https://stackoverflow.com/a/48547954/12040634
I have a post-receive hook in my server.
I also have a remote repo(origin) (branch:stage) where another developer pushes to.
I pull the commits and push them to production from my local computer.
I recently created a new branch(feature) from the latest head of the existing branch(stage), added some code and pushed the feature branch to remote repo. I merged the new features from feature branch to stage branch.
Then I pulled from the remote repo to local.
Now, I pushed the stage branch to production.
My post-receive file consists:
git --work-tree=/path/to/project --git-dir=/path/to/project.git checkout -
f stage
My head in the remote repo after merge is currently at ae228b9.
And, at the remote production hook's latest logs also points to ae228b9.
The problem is that the code from the merge are not seen in the production server.
Do I have to do something else to get the merged code to production?
After I pulled from origin and pushed to production, I tried changing the new files locally, committed them and push to the server. But, the changes are not reflected in the server.
Edit:
Also, I uninstalled apache php mysql and installed them again. Before the unistallation/installation, the home page was showing in server but the routes were not working(404 error) but the git hook was working. After the unistallation/installation, the git hook stopped working
Update:
I created a new hook with the same code as above. Now during the push I get:
remote: error: unable to unlink old '.gitignore' (Permission denied)
remote: error: unable to unlink old '.htaccess' (Permission denied)
remote: fatal: cannot create directory at '.idea': Permission denied
I tried the following code inside my project folder and project.git(hook) folder:
sudo chmod -R ug+w .;
unable to link
after which the push is a success but the changes are not reflecting.
I solved this by deleting the contents of the project folder and pushing again with some new changes.
I faced the same issue(Azure deployment)
There are two ways to ways to handle this -
Increase timeout setting (but deployment may stuck some times)
Push new changes (redeployment)
I prefer 2nd option.
So we are currently moving away from our current deployment provider: Beanstalk, which is great but we are on the top tier and we keep running out of space or hitting our repository limits. So we are moving away so please do not suggest any other SaaS provider.
I personally use Gitlab for my own projects and a few company projects and it's amazing we use a self hosted version on our local server in our company building.
We have CI setup and currently are using the following deployment code (I have minified the bits just to the deployment for development) - this uses the shell executer for deploying as we deploy to an existing linux server.
variables:
HOSTNAME: '<hostname>'
USERNAME: '<username>'
PASSWORD: '<password>'
PATH_DEV: '/path/to/www'
# Define the stages (we can add as many as we want)
stages:
# - build
- deploy
# The code for development deployment
deploy_dev:
stage: deploy
script:
- echo "Deploying to development environment..."
- rm .gitlab-ci.yml
- rsync -urltvz --filter=':- .gitignore' --exclude=".git" -e "sshpass -p"$PASSWORD" ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" * $USERNAME#$HOSTNAME:$PATH_DEV
- echo "Finished deploying."
environment:
name: Development
url: http://dev.domain.com
only:
- envdev
The Problem:
When we use the above code to deploy it's perfect and works really well, and it deploys all the code after optimisation etc, but we have found a little bug here.
When you delete a file then the rsync command will not delete the file, now I did some searching and found the --remove flag you can add, and it worked - but it deleted all the user uploaded content as well. Now I added the .gitignore in to the filtering, so it would ignore some the files in their (which are usually user generated) or configuration files or/and libraries (npm, etc.). This is fine until a user started uploading files using the media manager in our framework which stores in a folder that is not in the .gitignore file and it can't because it contains other files, as we also add our own files in there so they're editable by the user, so now I am unsure how to manage this.
What we are looking for is a CI setup, which will upload file changes to the server, so it would search through the latest commits, and find the latest files that have been changed and then push only them files up. Of course I would like to do this with the Gitlab CI still, so any ideas examples or tutorials would be amazing.
Thanks in advance.
~ Danny
May it helps: https://github.com/banago/PHPloy
Looks this tool designed for php project, but I think it can use other web deployment.
how it works:
PHPloy stores a file called .revision on your server. This file contains the hash of the commit that you have deployed to that server. When you run phploy, it downloads that file and compares the commit reference in it with the commit you are trying to deploy to find out which files to upload. PHPloy also stores a .revision file for each submodule in your repository.
I have GitLab running on computer A, development environment (Visual studio Pro) on computer B and Windows Server on computer C.
I set up GitLab-Runner on computer C (Windows server). I also set up .gitlab-ci.yml file to perform build and run tests for ASP.NET Core application on every commit.
I don't know how can I get code on computer C (Windows server) so I can build it (dotnet msbuild /p:Configuration=Release "%SOLUTION%"). It bothers me that not a single example .gitlab-ci.yml I found on net, doesn't pull code form GitLab, before building application. Why?
Is this correct way to set-up CI/CD:
User create pull request (a new branch is created)
User writes code
User commit code to branch from computer B.
GitLab runner is started on computer C.
It needs to pull code from current branch (CI_COMMIT_REF_NAME)
Build, test, deploy ...
Should I use common git command to get the code, or is this something GitLab runner already do? Where is the code?
Why no-one pull code from GitLab in .gitlab-ci.yml?
Edited:
I get error
'"git"' is not recognized as an internal or external command
. Solution in my case was restart GitLab-Runner. Source.
#MilanVidakovic explain that source is automatically downloaded (which I didn't know).
I just have one remaining problem of how to get correct path to my .sln file.
Here is my complete .gitlab-ci.yml file:
variables:
SOLUTION: missing_path_to_solution #TODO
before_script:
- dotnet restore
stages:
- build
build:
stage: build
script:
- echo "Building %CI_COMMIT_REF_NAME% branch."
- dotnet msbuild /p:Configuration=Release "%SOLUTION%"
except:
- tags
I need to set correct variable for SOLUTION. My dir (where GitLab-Runner is located) currently holds this folder/files:
- config.toml
- gitlab-runner.exe
- builds/
- 7cab42e4/
- 0/
- web/ # I think this is project group in GitLab
- test/ # I think this is project name in GitLab
- .sln
- AND ALL OTHER PROJECT FILES #Based on first look
- testm.tmp
So, what are 7cab42e4, 0. Or better how to get correct path to my project structure? Is there any predefined variable?
Edited2:
Answer is CI_PROJECT_DIR.
I'm not sure I follow completely.
On every commit, Gitlab runner is fetching your repository to C:\gitlab-runner\builds.. on the local machine (Computer C), and builds/deploys or does whatever you've provided as an action for the stage.
Also, I don't see the need for building the source code again. If you're using Computer C for both runner and tests/acceptance, just let the runner do the building and add Artifacts item in your .gitlab-ci.yaml. Path defined in artifacts will retain your executables on Computer C, which you are then able to use for whatever purposes.
Hope it helps.
Edit after comment:
When you push to repository, Gitlab CI/CD automatically checks your root folder for .gitlab-ci.yaml file. If its there, the runner takes over, parses the file and starts executing jobs/stages.
As soon as the file itself is valid and contains proper jobs and stages, runner fetches the latest commit (automatically) and does whatever script item tells it to do.
To verify that everything works correctly, go to your Gitlab -> CI / CD -> Pipelines, and check out whats going on. You should see something like this:
Maybe it would be best if you posted your .yaml file, there could be a number of reasons your runner is not picking up the code. For instance, maybe your .yaml tags are not matching what runner is created to pick up etc.
Description
We are in a current project based on MVC4/Umbraco using Azure Websites to host it.
We are using SCM_BUILD_ARGS to change between different build setups depending on which site in Azure we deploy to (Test and Prod).
This is done by defining an app setting in the UI:
SCM_BUILD_ARGS = /p:Environment=Test
Earlier we used Bitbucket Integration to deploy and here this setting worked like a champ.
We have now switched to using Git Deployment, pushing the changes from our build server when tests have passed.
But when we do this, we get a lovely error.
"MSB1008: Only one project can be specified."
Trying to redeploy the same failed deployment from the UI on Azure works though.
After some trial and error I ended going into the deploy.cmd and outputting the %SCM_BUILD_ARGS% value in the script.
It looks like the / gets dropped from SCM_BUILD_ARGS but only when using Git deploy, not Bitbucket Integration or redeploy from UI.
Workaround
As workaround I have for now added a / to the deploy.cmd script in front of the %SCM_BUILD_ARGS%, but this of course breaks redeploy, since we then have //p:Environment=Test in the MSBuild command when the value of %SCM_BUILD_ARGS% has been inserted.
:: 2. Build to the temporary path
IF /I "%IN_PLACE_DEPLOYMENT%" NEQ "1" (
:: Added / to SCM_BUILD_ARGS
%MSBUILD_PATH% "%DEPLOYMENT_SOURCE%\www\www.csproj" [....] /%SCM_BUILD_ARGS%
) ELSE (
%MSBUILD_PATH% "%DEPLOYMENT_SOURCE%\www\www.csproj" [....] /%SCM_BUILD_ARGS%
)
Question
Anyone know of a better solution for this problem or is it possibly a bug in Kudu?
We would love to have both deploy from Git and Redeploy working.
Could you try changing from "/" to "-"? For instance, AppSettings from /p:Environment=Test to -p:Environment=Test, see if it helps.
-p:Environment=Test did not work for me, the setting which worked for me at the time of this writing (September 2015) was
-p:Configuration=Test
There is clearly a Kudu bug in there, and you should open an issue on https://github.com/projectkudu/kudu. But for now, I can give you a workaround.
Instead of using an App Setting, include a .deployment file at the root of your repo, containing:
[config]
SCM_BUILD_ARGS = /p:Environment=Test
I think this will work in all cases. I suspect the bug has to do with bash messing up the environment in post receive hook scenarios, which only apply to direct git push but not to Bitbucket and Redeploy scenarios.
UPDATE: In fact, it's easy to see such weird bash behavior. Try this:
Open cmd.exe
Run: set foo=/abc to set a variable
Run bash
From bash, run cmd to launch a new cmd on top of bash (so cmd -> bash -> cmd)
Run set foo to get the value of foo
Result:
FOO=C:/Program Files (x86)/git/abc
So the value gets completely messed up. The key also gets upper cases, though that's mostly harmless. Strange stuff...