I have a home machine and office machine I use to publish websites using Visual Studio 2013. If I make a change from the same machine, and re-publish, just the changes are published, not all files.
However, when using my clone machine at the office, even if I do a get latest, make one small change, and re-publish, all files are published, not just the ones that changed, and not just the ones that have been recompiled. ALL dll files, even third party dlls that have not changed or have been recompiled with a new date, are republished. Same thing happens if my cohort publishes a small change on his machine after I did the last publish. Not a problem if publishing twice from the same machine as then only the changed files are published.
Is there anyway to prevent complete republishing just because a different machine is used to publish than the one used for the last publish? Thanks.
This seems to make "Determining Changes" a lot slower, but for .Net 4.5 [and
up(?)], use this info from:
https://msdn.microsoft.com/en-us/library/ee942158:
To configure Web Deploy to use checksums instead of dates to determine
which files need to be copied to the server, add the following element
to the .pubxml file (Publish Settings):
<MSDeployUseChecksum>true</MSDeployUseChecksum>
Like this:
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<MSDeployUseChecksum>true</MSDeployUseChecksum>
<!— other settings omitted to keep the example short -->
<PublishDatabaseSettings>
<!— this section omitted to keep the example short -->
</PublishDatabaseSettings>
</PropertyGroup>
</Project>
First of all, I do not understand the behavior of MSBuild+VS2013 and the publishing feature completely as I just started to use the publishing feature myself. I'm looking for a way to speed up publishing via FTP web deploy in VS2013. (I'm not using TFS for a get latest though.)
I would say this is partially explained in a different context at this SO question. The MSBuild process. Timestamps of certain files are being compared and then could indicate MSBuild/VS2013 whether a target (build output) is up-to-date or not. Then files that are not up-to-date are being recompiled.
As you all work on different machines, timestamps are likely to be different quite soon.
To find out what is actually going on during builds/publishes, set build output verbosity to detailed or diagnostic, for a moment:
VS2013 menu - Tools - Options... - Project and solutions - Build and run - output verbosity - set to -> detailed or diagnostic. Run the build, and see the Output panel/tab in VS2013. Select "show output from: Build" to see results if not already visible. Don't forget to set it to the original setting after checking the build details, as it could slow the build down a bit.
But why even unchanged third party dll's are being republished? Possibly because these dll's ARE actually overwritten during a build. You might have the assembly reference property "Copy local" set to true to get your website running without any manual uploads to be done for this. Or you are using a commandline copy command with the overwrite parameter explicitly set to true during project's post build event (like 'copy /y ...' or 'xcopy /y ...'). Then the timestamp of the file that is to be published is overwritten for sure, see in
the "obj\Release\Package\PackageTmp" folder (or for example: "\Debug" instead of "\Release" if Debug build is set for the selected publish profile.)
Furthermore, VS2013 as default does NOT check timestamp differences on the targeted webserver if you are using FTP publishing, at least that is my experience. As for the other publishing options like a web deploy I don't know yet. But the differences you experience seem normal behavior to me, as you run builds on different machines and publishing files from different machines as well. So timestamps are likely to be different... which, again, indicates 'changed' files to be published.
As I'm curious how this question should be solved, I was looking for this in the first place:
Maybe a TFS build server is an option for you, configured with a rolling build. But I read on a specific SO (sorry, can't add more links at this moment as I've just registered) that is suggesting to do clean builds to prevent new problems. And that will force full publishing again as files are all being changed by the clean build... so that won't work I think.
As an answer, you might want to use these FAQ answers on MSDN for web deployments. See the questions:
"Can I exclude specific files or folders from deployment?" .NET 4.0/4.5 and/or
"How to make Web Deploy use file checksums instead of dates to
determine which files were changed?" .NET 4.5 only!
The first option is to exclude files.
The second option is to use a checksum to compare files instead of timestamps, but that could be somewhat slowing the build(?) process, as the FAQ says. Note the first few lines on that FAQ pages, on how you can edit your publishing profile to apply one or both of these elements!
Also it is an option to put the thirdparty dll's in a different project which you then could only include that project in the deploy for a certain solution configuration (VS2013 menu - Build - Configuration Manager, see the checkboxes in the 'deploy' column there for every project. Though I'm not sure if this is part of the VS2013 'publish' feature as a web deployment, because this deploy column checkboxes are greyed out for my solution projects for some reason I don't understand yet... so I can't test it to verify this option.)
Though it sounds logic, don't forget to create backups/copies/screenshots first before you change any settings or publishing profiles, and then change the same settings/configurable files on the other machines you and your colleagues work at.
Related
I know IIS allows the creation of Publish Profiles that can be "imported" into Visual Studio in order to upload a site directly into IIS (since I'm already using it).
But now I have a more specific question regarding the use of these publish profiles in Visual Studio.
I have a solution for a web application that comprises a couple different components that I'd like to keep sepparated in IIS.
Namely, I have the web version, a mobile version and a couple webservices in this project.
What I'm configuring the server to do is have the webservices, mobile and website separated into different sites and use different publish profiles to publish them, each into it's own place.
Since I have all of these components into a single visual studio project, would it be possible to have publish profiles that publish a single component of the project without requiring me to do a "full publish"?
Or is the only solution to have separate projects? (even if they are all in one single VS Solution)
Visual Studio's web publishing feature assumes that projects map to atomic components1. There isn't a way by default to specify how to only publish a subset of the project. Partly this stems from the build system (MSBuild) that the Web Publish Pipeline (WPP) is built over.
Options you can investigate:
Make your site contents match the structure in your project. Deployments are incremental (if coming from your machine), and you can deploy specific files or folders from the VS Solution Explorer. If you need to republish your binaries, you're still stuck doing a full publish. Publishing individual files/directories is the exception to note 1 above, and only works for content file changes.
If you're up to the challenge, you could dig your way through the WPP targets (it's all MSBuild), and try to find a way to restrict which files are published. Then you could set up separate publish profiles within your project that each only handle a subset of the files.
The easiest way, especially if you're automating this, is probably just to use separate projects for each component. :(
Using Visual Studio 2012, Azure SDK 2.1, I am trying to figure out the best way to create the csx folder for running in the azure emulator. My understanding is that the csx folder is not created until I package the Azure project. I can create a package manually from Visual Studio, but this is not an option for an automated build. The other option is to create the package using the msbuild command line. This seems a bit heavy handed as it will actually do a build which is more time consuming than just repackaging.
So, I thought that cspack might be a more lightweight option. However, when I call cspack with the following command line:
cspack.exe ServiceDefinition.csdef /copyOnly
I get the error: Need to specify the physical directory for the virtual path 'Web/' of role MyProjWeb.
But, I don't do anything like that when using msbuild. I have read a bunch of things about specifying the physical directory and some of the confusion that it can cause. So, I would prefer not to use it unless absolutely necessary, especially since I don't need to specify this when building from msbuild.
So, my main question is what is msbuild doing that cspack is not doing and how do I do the same with cspack?
My other question is, what is the easiest way to generate the csx folder for testing in the azure emulator?
Edit - Resolution
I thought that I would put down how I resolved this here in case it helps someone else. The big answer to my question (thanks to Chandermani and some other reading) is that CSPack with /copyOnly is basically a fancy xcopy to a folder structure according to some rules. If not using /copyOnly it then also does a fancy zip to create a package. Not complaining, it is fine that it is simple, but it is good to know this at the outset. You can use it for packaging anything for azure it is not tied to what can be built in Visual Studio, e.g. a PHP site. Using msbuild has the added benefit of only copying that files that are part of your web site deployment.
So, what I found when I got CSPack working and pointed at the mvc project folder is that it copied everything including source files. Which is not what I wanted. The solution that I could find is to first package the web site then point CSPack at the packaged files. If you do down this path then this link is very valuable as it describes it step by step.
So, it was either having an msbuild post-step in the Web project to package the files and then a post-step in my Azure project to cspack it or to have an msbuild post-step in my azure project to create the package (do cspack with the benefit of only including my web deployment files). Well, it seemed simpler and less error prone to just to have the one post step and let msbuild do the heavy lifting. So, the post step in my azure project is something like:
"C:\Program Files\Microsoft SDKs\Windows Azure\Emulator\csrun.exe" /devfabric:shutdown > NUL
"C:\Program Files\Microsoft SDKs\Windows Azure\Emulator\csrun.exe" /devstore:shutdown > NUL
if $(ConfigurationName) == Debug set CONSTANTSPARAMETER=DEBUG
if $(ConfigurationName) == Release set CONSTANTSPARAMETER=
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\MSBuild.exe $(ProjectDir)$(ProjectFileName) /t:clean;publish /p:Configuration=$(ConfigurationName) /p:TargetProfile=cloud /p:OutputPath=bin\Cloud$(ConfigurationName) /p:VisualStudioVersion=11.0 /p:overwritereadonlyfiles=true /p:DefineConstants="%CONSTANTSPARAMETER%" /verbosity:minimal /p:PostBuildEvent=
The first two lines shut down the compute and storage emulator.
The next two lines set the preprocessor constants. I found that #if DEBUG was no longer taking effect when built using the msbuild line. I think that this is safety protection that DEBUG is stripped when creating a package. I only ever use the package that is created by an automated build system, so it is safe for me to keep the DEBUG constant.
The actual msbuild line has a number of switches. I'll describe the unusual ones:
/p:PostBuildEvent=
If we don't set the postBuildEvent to empty then the same post step will keep getting called forever. And ever...
/p:VisualStudioVersion=11.0
Those clever guys at Microsoft made it possible to open projects with both Visual Studio 2010 and 2012. Which is great, but can bring great sadness when you run msbuild from the command line and end up with nasty MSB4019 error messages because it is looking in the wrong Visual Studio folder for the Azure tools.
Also, note that that I use the cloud profile. Since I am only after the csx files it doesn't seem to make a difference whether I use local or cloud at this point. When I run in azure I specify ServiceConfiguration.Local.cscfg.
Edit: In the end I took this out of the post step and put in my automated build. My original intention was that running the tests from my dev machine would be the same as my automated build, but the post step took too long and the views were sourced from the obj folder rather than the proj folder when running under the debugger which meant I had to copy across when making changes on the fly.
Unanswered questions
It would still be good to understand how msbuild does things to reduce knowledge friction when dabbling in this area. Does it create a package for the website and pass it to CSPack? Or does it parse the project files and then pass some crazy arguments to CSPack? Also when you run an azure project in the debugger, it runs in the emulator with only the binaries in the csx folder (not the images, etc). How does it do that? It would be great to see some description with pictures of the Azure build pipeline with that showed the lifecycle all the way to deployment. That might also explain why there are two copies of the binaries. Also, this would have been a whole lot easier if Visual studio had a project flag like packageOnBuild for the Azure project with options to do a copyOnly or to create a package. I see no point in uneaten cake. Edit: There is a DeployOnBuild setting that can be added to csproj.
Finally, as I mentioned the whole purpose of this is to get a csx folder that I can point the emulator at so that I can run my unit tests on my dev machine. I do the formal packaging on a build machine so don't really need it in Visual Studio. So, really I don't want to package anything and was hoping that there was an easier way of achieving all this.
Since msbuild uses the the azure project file to perform the build, it can derive a lot of information form the project file.
For cspack, the assumption is the role code has been compiled and is available for packaging. Since cspack does not depend upon project file, it needs a explicit information for the code path of the the web\workerrole project. The csdef file does not contain any such information. I suggest if you want to use cspack. Look at its documentation and try to create a package for emulator deployment from command line (CopyOnly option). Once you find the correct syntax you can embed it in you build script.
I have a larger solution that I desire to distribute via ClickOnce. It consists of one main shell executable that directly references only a small subsection of libraries and processes that constitute the solution.
The solution consists of a few other processes and several libraries (some C++). I need to be able to include all of these libraries and processes in one ClickOnce distribution for both local builds and TFS server builds.
I cannot reference every other library and process form the shell project. And I do not wish to push these files into a MSI to be treated as a prerequisite as it would defeat the purpose of using ClickOnce to distribute/update the product.
What is the correct method to incorporate all of our necessary files/projects into a single ClickOnce distribution?
The IDE won't detect native DLLs as dependencies when publishing, but you can run the SDK tools directly to include them manually in your ClickOnce distribution. You can either use mage.exe in your post-build script or run MageUI.exe to have a wizard to guide you through the package generation.
Suggested reading:
Walkthrough: Manually Deploying a ClickOnce Application
Understanding Dependencies of a Visual C++ Application
There is an alternative to Visual Studio for this kind of situation. You could try using Mage, but it can be a little tricky to use. My company wrote an alternative called ClickOnceMore.
ClickOnceMore is a ClickOnce build tool for when you don't want or can't use Visual Studio to do ClickOnce builds.
There is a specific page on the UI for including files (using rules to include anything from a single file to an entire directory trees) so you should be able to do exactly what you need with it.
This is what I have done in a similar situation. I use TFS at work, so convert the terms to whatever you may use (or not use) for source control.
I have a main workspace that I use for all development of my application, I keep this workspace pristine.
I then created another workspace with a proper name (ex: solution-deploy) and in this workspace I do the following:
Get latest and merge everything from source-control into the deployment workspace
I build a Release build of my application
I r-click on the root (I put them in the root, because I need to access them from there, put them in whatever folder you want) project folder for my deployment project and select "Add -> Existing Item"
I browse in the file selector to the Release directory of the assemblies I want to add to my deployment package, select them, then I use the arrow next to the Add button and drop down to "Add As Link", do this for all of the assemblies you want to add and place them wherever you want them to be organized in your deployment
In the Solution Explorer, select the added assemblies and in the Properties window set the Build Action to "Content", this should be all you have to do, but others have had to also set the "Copy to Output Directory" to "Copy Always", I don't do that
Run a Release Build
Go to the Properties view for your deployment Project
Go to the Publish Tab and Click on the Application Files button
Your files should all be available and added to the Deployment
Set up your ClickOnce settings however you need them to be
Publish your ClickOnce package
Your published package should contain all of the assemblies you need now.
Keep your separate Deployment workspace set up this way and never check it in. Do your work in your development workspace. Whenever a new deployment is needed, open your solution in your Deployment workspace and get the latest code, build, then publish.
I'm attempting to use Web Deploy to Publish a Web Application.
I want Visual Studio to delete any files that no longer exist, so I've checked the "Remove additional files at destination" setting in my Publishing profile.
However, I want VS to ignore the /Content/uploads folder, as it contains contents that my users have uploaded. Naturally, the contents are different in my development site than they are in the live site.
Unfortunately, I have been unable to discover a way to make Visual Studio ignore this folder when publishing (it wants to delete all of the content, since it doesn't exist in the project).
Does anyone know of a way to exclude specific folders on the target site from being examined by Web Deploy?
I had a similar problem, wanting to keep some files in the deployment package even though they're not part of the project.
Try to create a custom MSBuild target for this, that works for me.
Here is a Getting Started MSBuild reference
Hope this helps.
All the best.
I was unable to find a suitable solution for this issue, so I've created my own:
https://pubsync.codeplex.com/
PubSync enables quick and reliable file syncing for publishing Visual Studio projects.
I'm looking for something that can copy (preferably only changed) files from a development machine to a staging machine and finally to a set of production machines.
A "what if" mode would be nice as would the capability to "rollback" the last deployment. Database migrations aren't a necessary feature.
UPDATE: A free/low-cost tool would be great, but cost isn't the only concern. A tool that could actually manage deployment from one environment to the next (dev->staging->production instead of from a development machine to each environment) would also be ideal.
The other big nice-to-have is the ability to only copy changed files - some of our older sites contain hundreds of .asp files.
#Sean Carpenter can you tell us a little more about your environment? Should the solution be free? simple?
I find robocopy to be pretty slick for this sort of thing. Wrap in up in a batch file and you are good to go. It's a glorified xcopy, but deploying my website isn't really hard. Just copy out the files.
As far as rollbacks... You are using source control right? Just pull the old source out of there. Or, in your batch file, ALSO copy the deployment to another folder called website yyyy.mm.dd so you have a lovely folder ready to go in an emergency.
look at the for command for details on how to get the parts of the date.
robocopy.exe
for /?
Yeah, it's a total "hack" but it moves the files nicely.
For some scenarios I used a freeware product called SyncBack (Download here).
It provides complex, multi-step file synchronization (filesystem or FTP etc., compression etc.). The program has a nice graphical user interface. You can define profiles and group/execute them together.
You can set filter on file types, names etc. and execute commands/programs after the job execution. There is also a job log provided as html report, which can be sent as email to you if you schedule the job.
There is also a professional version of the software, but for common tasks the freeware should do fine.
You don't specify if you are using Visual Studio .NET, but there are a few built-in tools in Visual Studio 2005 and 2008:
Copy Website tool -- basically a visual synchronization tool, it highlights files and lets you copy from one to the other. Manual, built into Visual Studio.
aspnet_compiler.exe -- lets you precompile websites.
Of course you can create a web deployment package and deploy as an MSI as well.
I have used a combination of Cruise Control.NET, nant and MSBuild to compile, and swap out configuration files for specific environments and copy the files to a build output directory. Then we had another nant script to do the file copying (and run database scripts if necessary).
For a rollback, we would save all prior deployments, so theoretically rolling back just involved redeploying the last working build (and restoring the database).
We used UnleashIt (unfortunate name I know) which was nicely customizable and allowed you to save profiles for deploying to different servers. It also has a "backup" feature which will backup your production files before deployment so rollback should be pretty easy.
I've given up trying to find a good free product that works.
I then found Microsoft's Sync Toy 2.0 which while lacking in options works well.
BUT I need to deploy to a remote server.
Since I connect with terminal services I realized I can select my local hard drive when I connect and then in explorer on the remote server i can open \\tsclient\S\MyWebsite on the remote server.
I then use synctoy with that path and synchronize it with my server. Seems to work pretty well and fast so far...
Maybe rsync plus some custom scripts will do the trick.
Try repliweb. It handles full rollback to previous versions of files. I've used it whilst working for a client who demanded its use and I;ve become a big fan of it, partiularily:
Rollback to previous versions of code
Authentication and rules for different user roles
Deploy to multiple environments
Full reporting to the user via email / logs statiing what has changed, what the current version is etc.