I've recently been on a support ticket with Azure, and they've recommended turning on Local caching to eliminate occasional outage blips.
The problem with that, is that you need to watch your disk space, since >1Gig is not allowed. And if you deploy from git, like I do, that's an issue because the whole repository is checked out, then built locally, and then kudu-synced.
I've looked at trimming my repo down, but that's only going to yield small savings. What I'd like to do is to remove my repository folder once the deployment has complete. Is that a sensible idea, or are there other solutions to this problem?
There is an upcoming change to the Local Caching behavior that will make it skip the repository folder (since it's not needed at runtime). This should be in the next couple of weeks.
Once that change is out, this issue should automatically go away for you.
the repository folder only container a copy of your repo. it is ok to remove it if you want to safe some space. it will be re-create when there is a new deployment.
There is one side effect when you delete your repository folder, your next deployment will take longer time since it will need to sync your entire repository.
Other than repository folder, you can cleanup files that under D:\home\LogFiles as well, to safe you some more spaces
I would recommend you to use build sequence using Visual Studio Team Services - there, you can do anything you want and include operations into the build pipeline (build trigger => delete the folder).
Related
I’m trying to set up GitLab CI/CD for an old client-side project that makes use of Grunt (https://github.com/yeoman/generator-angular).
Up to now the deployment worked like this:
run ’$ grunt build’ locally which built the project and created files in a ‘dist’ folder in the root of the project
commit changes
changes pulled onto production server
After creating the .gitlab-ci.yml and making a commit, the GitLab CI/CD job passes but the files in the ‘dist’ folder in the repository are not updated. If I define an artifact, I will get the changed files in the download. However I would prefer the files in ‘dist’ folder in the to be updated so we can carry on with the same workflow which suits us. Is this achievable?
I don't think commiting into your repo inside a pipeline is a good idea. Version control wouldn't be as clear, some people have automatic pipeline trigger when their repo is pushed, that'd trigger a loop of pipelines.
Instead, you might reorganize your environment to use Docker, there are numerous reasons for using Docker in a professional and development environments. To name just a few: that'd enable you to save the freshly built project into a registry and reuse it whenever needed right with the version you require and with the desired /dist inside. So that you can easily run it in multiple places, scale it, manage it etc.
If you changed to Docker you wouldn't actually have to do a thing in order to have the dist persistent, just push the image to the registry after the build is done.
But to actually answer your question:
There is a feature request hanging for a very long time for the same problem you asked about: here. Currently there is no safe and professional way to do it as GitLab members state. Although you can push back changes as one of the GitLab members suggested (Kamil Trzciński):
git push http://gitlab.com/group/project.git HEAD:my-branch
Just put it in your script section inside gitlab-ci file.
There are more hack'y methods presented there, but be sure to acknowledge risks that come with them (pipelines are more error prone and if configured in a wrong way, they might for example publish some confidential information and trigger an infinite pipelines loop to name a few).
I hope you found this useful.
We currently have an issue with our TFS server locking up whenever any of our developers attempt to check in a pending delete on a folder, however, this is a relatively new (within the last couple of months) issue, and we have been struggling to determine the root cause.
Essentially, the process that is occurring is something like this:
Create folder in TFS.
Branch from Dev trunk, creating the branch in the new folder.
Make any code changes in branch.
Merge changes from branch into Dev trunk.
Delete branch.
Realize we no longer need the folder.
Delete the folder.
Attempt to check-in folder deletion.
TFS server locks up, not allowing any operations to be performed until the server is restarted.
So, I am looking for two answers:
Why is TFS locking up when attempting to check in a folder deletion?
What can we do to resolve the issue?
A little bit more info:
This problem occurs for every user, including out TFS administrator(s).
It may occur even when there was never anything placed in the folder, but we haven't attempted that, so that we can avoid having to restart the server.
This has not always been an issue on the server, as it use to be relatively common for us to delete folders.
This issue occurs regardless of how the delete is being performed (VS2012, VS2015, command line, etc.).
We primarily work with a single trunk, but have potentially hundreds of branches being created over the course of a month.
The solution contains roughly 20-25,000 files, currently.
We are trying to work out a good continuous deployment setup using TFS, Visual Studio and Azure. At our company, each developer has their own Azure subscription that we use for testing, as well as shared QA1/QA2/PROD subscriptions that we can deploy to. We have matching TFS XAML build definitions for each of these, running Powershell scripts with parameters and PublishSettings files.
This all gives us a set of .cspkg and .cspkg files, and in theory we can deploy the right cspkg with the correct cspkg to any Azure system.
The problem we are encountering now though is that we want to start using the Redis Cache service. Installing the nuget package writes subscription-specific settings into the web.config, to point at the cache. This means that the cspkg is now complied specifically for the Azure subscription.
We could use SlowCheetah to merge web.config files on build, but this means that we would have to compile the package for each build definition, and as the number of developers increases this is obviously going to become unsustainable.
I am looking for a way to keep our old generic packages and still use the Redis Cache. We can connect to the cache in code during app_start, but then we can't use it to store IIS session state. I understand that the Azure Load Balancer is meant to keep users on the same server, but I'm unsure how that will work as we swap servers in/out.
It feels like we are approaching the problem wrong and there should be a simple solution that we are overlooking.
We are using Azure Tools 2.6, Visual Studio 2013, TFS 2015r2.
I think there are always 3 ways of doing this.
1st one is config during build, which is building one thing for one thing you described, which is not desired in most of scenario.
2nd is config during deployment, which means you open the cspkg file, change config, then put it back before upload without re-compile.
3nd is config after deployment, have a configuration management tool adjust the config file for you on the fly.
We use octopus deploy to archive #2 above, our CI tool feed octopus with cspkg and cscfg, octopus handles the rest. I would definitely not going after #1 but consider #3 is a valid option too.
As of today we store all our connection settings in .cscfg files. Even if for security reasons, we avoid storing any production connection strings in source control, only QA. And we have CI for QA, but not production. This way it works well for us, we just maintain different .cscfg for different environments (subscriptions)
However, in near future I think we will move to Key Vault for this.
I have an Azure Website configured to deploy from a Bitbucket repository. This works fine.
Since the application is still in active development, I update the nuget packages it uses quite frequently. This causes the packages folder to keep growing indefinitely, unless I go and manually delete the packages.
Now, in my local machine this is not a big issue. Space is cheap. But in Azure, this makes us go over the quota really fast, as old packages accumulate.
How can I customize the Azure deploy process so that it deletes all the packages after a successful deployment?
(I am open to other solutions as well)
You can utilize the custom deployment script feature where you add a step that cleans up the packages directory.
You can read about it here:
http://blog.amitapple.com/post/38418009331/azurewebsitecustomdeploymentpart2/
Another option is to add a post deployment action, by adding a script file (.cmd/.bat) that has the clenup logic to the following directory in your site: d:\home\site\deployments\tools\PostDeploymentActions\, this script will run after the deployment completes successfully.
Read more about it here:
https://github.com/projectkudu/kudu/wiki/Post-Deployment-Action-Hooks
This is somewhat related to my security question here. Is it a bad idea to use an hg / mercurial repository for a live website? If so, why?
Furthermore, we have dev, test and production installations of our website, like dev.example.com, test.example.com and www.example.com. If it's a bad idea to use a repository for a live/production website, would it be OK to use an hg repository for the dev and test sites?
I'm also concerned about ease of deployment. We have technical and less technical co-workers who will be working with the site. The technical people (software engineers) won't have any problem working with the command line or TortoiseHG. I'm more concerned about the less technical people (web designers). They won't be comfortable working on the command line, and may even find TortoiseHG daunting. These co-workers mostly upload .css files and images to the server. I'd like for these files (at least the .css files) to be under version control, but I want this to be as transparent as possible for the non technical team members.
What's the best way to achieve this?
Edit:
Our 'site' is actually a multi-site CMS setup with a main repository and several subrepositories. Mock-up of the repository structure:
/root [main repository containing core files and subrepositories]
/modules [modules subrepository]
/sites/global [subrepository for global .css and .php files]
/sites/site1 [site1 subrepository]
...
/sites/siteN [siteN subrepository]
Software engineers would work in the root, modules and sites/global repositories. Less technical people (web designers) would work only in the site1 ... siteN subrepositories.
Yes, it is a bad idea.
Do not have your repository as your website. It means that things checked in, but unworking, will immediately be available. And it means that accidental checkins (it happens) will be reflected live as well (i.e. documents that don't belong there, etc).
I actually address this "concept" however (source control as deployment) with a tool I've written (a few other companies are addressing this topic now, as well, so you'll see it more). Mine is for SVN (at the moment) so it's not particularly relevant; I mention it only to show that I've considered this previously (not on a Repository though; a working copy, in that scenario the answer is the same: better to have a non-versioned "free" are as the website directory, and automate (via user action) the copying of the 'versioned' data to that directory).
Many folks keep their sites in repositories, and so long as you don't have people live-editing the live-site you're fine. Have a staging/dev area where your non-revision control folks make their changes and then have someone more RCS-friendly do the commit-pull-merge-push cycle periodically.
So long as it's the conscious action of a judging human doing the staging-area -> production-repo push you're fine. You can even put a hook into the production clone that automatically does a 'hg update' of the working directory within that production clone, so that 'push' is all it takes to deploy.
That said, I think you're underestimating either your web team or tortoiseHg; they can get this.
me personally (i'm a team of 1) and i quite like the idea of using src control as a live website. more so with hg, then with svn.
the way i see it, you can load an entire site, (add/remove files) with a single cmd
much easier then ftp/ssh this, delete that etc
if you are using apache (and probably iis as well) you can make a simple .htaccess file that will block all .hg files (or .svn if you are using svn)
my preferred structure is
development site is on local machine running directly out of a repository (no security is really required here, do what you like commit as required)
staging/test machine is a separate box or vm running a recent copy of the live database
(i have a script to push committed changes to staging server and run tests)
live machine
(open ssh connection, push changes to live server, test again, can all be scripted reasonably easily, google for examples)
because of push/pull nature of hg, it means you can commit changes and test without the danger of pushing a broken build to the live website. like you say in your comments, only specific people should have permission to push a version to the live site. (if it fails, you should easily be able to revert to the previous version, via src control)
Why not have a repo also be an active web server (for dev or test/QA environment anyway)?
Here's what I am trying to implement:
Developers have local test environments in which they can build and test their code
Developers make a clone of the dev environment on their local dev machine
Developers commit as often as they want to their local repo
When chunk of work is done and tested, then developer pushes working change sets to dev repo
Changes would be merged and tested on Dev, then pushed to Test/QA, and so on.
BTW, we're using Mercurial. I believe this model would only work using a distributed source code management tool.