We have a large code base in Perforce. I would like to do the following nightly, automatically.
- Copy some view of "latest" into two (or more) workspaces, streams, or even just into some other folders not under perforce control.
- Check everything out (if p4 used) and "compile it", (where "compile" may include changing most files, thus the need to have them be writable.
- Rinse repeat the following "night" with a fresh "latest".
I know how to do this via simple copying things out, but would like the nightly modified code to be able to be "seen" from other machines, by other people, thus maybe the stuffing of things back into perforce.
I know how to do this with P4 workspaces.
Just wondering if p4 streams (tasks?) are a better approach, or for any other alternate recommendations.
Using workspaces is indeed what you want to do. The benefit of using streams would mostly be in simplifying the task of generating and managing these workspaces.
Do you want to keep the changes made by the build machine isolated from the mainline that everyone else uses? Should those changes also be isolated from other builds? Or do you want to make sure that everyone gets the changes ASAP and that they make their way into all other variants of the code? These are good questions to ask as you're setting this up and the answers should influence what you do.
Related
When using Team Foundation Server, is there a generally accepted way to approach branching when you need to test certain approaches to issues that come up (bugs, feature requests, etc)?
Say, for example, that you are on a team with a few other developers and one developer says that he just cannot figure out how to solve this problem and asks you for assistance. You have some ideas but you want to make sure that you don't accidentally make changes to his main branch of code in case your ideas don't work.
At this point you could either:
Make a local copy and unbind it from source control to ensure you do not accidentally check in any changes in the future.
Create a branch from his branch and call it "Testing" or something, and delete the branch once you are done with it.
Check out a copy of his branch and simply "undo pending changes" when you decide your approaches don't work (this seems dangerous).
I know you could probably approach it a few other ways but I was just curious as to whether or not there was a popular way to approach making "test changes" to code thats under source control.
Branching is best kept for multiple simultaneous, independent changes.
Use a Shelveset for the purposes you're discussing. That's how the "Suspend" feature of "My Work" works, and it's how code reviews work. They shelve the changes, attach them to the Code Review work item, and inform the reviewers. The reviewers can unshelve the changes, look at them and make comments, and possibly even edit them. They can then shelve their version of the changes and send them back to you.
There's a lot of flexibility there, without creating branches, which are permanent artifacts.
Update: You can additionally execute a build and set the Shelveset name in the version to build property to create a custom build from your shelveset.
Our project is based on another studio's technology, which we have modified to fit our needs. We use TFS, they use Perforce. They have just shipped their product and as a result, they have a lot of new finished features we want to pick up for our project. I am tasked with integrating over this new code drop into our codebase.
I am very familiar with Perforce as I have been using it for years, but I am much less experienced with TFS (which is where all the work will take place) so I am pretty sure my approach will be brute force, tedious, and extremely time consuming. Hopefully some of you will be able to suggest "the right way" to do this. :)
Here's what I am doing:
Check out the entire project in TFS
Copy the entire new code drop over the old code
Merge all the changes we have made to their codebase into this new vanilla code drop (YIKES!)
Check in
Folder diff between the checked in codebase and the vanilla drop I sycned from Perforce and delete any files no longer in the project
Check in
Profit
As you can tell, this is going to be "fun". I'm up to step 3 and I am realizing that I don't know how to actually do this merge in a way that doesn't require diffing every single friggin file in the project (approaching 3000). I was hoping to get a window of all the conflicts I need to resolve like you do after syncing.
Furthermore, I just don't have confidence that this is going to work out well even if I did brute force diff every single file. :) Files moved, added, removed, etc are just going to be a nightmare. Much of the work we want to pick up is refactoring their old messy early code from a much earlier state in the project when we last synced.
I have to believe that TFS can make this task much cleaner since what I am trying to do is not uncommon. What would you guys suggest?
Thanks!
My suggestion is to have 2 branches in TFS, one that is the unmodified code from the other team (lets call this DEFAULT). Then make a branch off that (lets call this CUSTOM). You would make your changes in CUSTOM.
This way when the other team release new code you just Get Latest on DEFAULT, delete the local workspace files, copy over the new code, then let the local workspace feature do it's thing and detect all the necessary pending changes (adds, deletes, edits), then check-in the changes. Then you can merge DEFAULT->CUSTOM and resolve any conflicts.
Since you don't have this structure setup now, setting it up the first time will be a little painful, but in the future it should make life easier.
Assuming you have a copy of the unmodified code from the previous release, you can create the DEFAULT branch and check that in. Then branch off a CUSTOM branch. Then Get Latest on CUSTOM, delete all the local workspace files, and copy in your current modified code. VS should detect all the necessary pending changes, then check those in.
Now you can do the process to update DEFAULT, and merge down to CUSTOM for the new version. Because TFS will track the history, you should only have to deal with actual conflicts rather than going through every single change. You will of course have to do sufficient testing to ensure that the new code and your customizations still work well together.
I have read through the documentation on perforce and the "branching strategy" advice as well.
One thing thats left me baffled, is how a simple concern is does not seem to adequtely adressed.
When I am working on a project that touches many parts of our code base , I cannot checkin my code at the end of the day without checking into the trunk. So do I need to branch in this situation? I want to be able to have the ability to have a history of my changes in a long and hard project, so I can go back when I I make a wrong turn..
The problem with branching I see is that I will be creating copies of almost the entire codebase .. Am I missing an obvious solution here?
thanks
From the standpoint of the Perforce server, branches aren't actually copies of the files. Perforce uses a mechanism called "lazy copy" to minimize disc consumption. From their website, here is the definition of the term:
A method used by Perforce to make
internal copies of files without
duplicating file content in the depot.
Lazy copies minimize the consumption
of disk space by storing references to
the original file instead of copies of
the file.
Best approach to working with perforce is to work in a user/feature branch then you can avoid checking into the trunk whilst still pushing your changes into the depot.
When creating a branch, you don't have to branch the entire trunk or source branch - you only need to branch the files you're working on - you can map the rest of the files into your branch via your client spec.
TBH - just buy & read 'Practical Perforce', it has heaps of useful info on how to do this and is very much worth the money if you're using perforce on a daily basis.
Another very useful feature of perforce is 'jobs' - often described only for bug tracking - it's much flexible and allows you to store a changelist history attached to a tag so allowing you to create 'metatags' and attach revisions to it i.e 'NightlyBuild' or 'BreakingChanges or whatever you want.
HTH.
The closest I know of is shelving, in which you can "shelve" your work in progress, saving a copy on the server. I typically do this to essentially checkpoint my work. I think this comes closest to addressing your need, where you can save your progress at the end of the day.
See here for a tutorial on shelving in p4v.
Or type p4 help shelve for help with the command line.
Evaluate using PDB - Sparse branches. More information here http://www.releng.com/p5layer.html
I am new to Perforce.
What would you consider P4 best practices in terms of views/branches.
Would you create 1 view with access to many branches, or would you create 1 view / branch ?
I am not sure switching between views/worspaces is that easy in Perforce.
Thanks,
Thomas
If by "view" you mean workspace (aka client-spec), then I strive to have one client spec for each branch I am working on. I'm assuming by branch you mean you have a typical system where you have mainline, development branches, and maybe release branches too?
The reason for keeping the 1-1 correspondence is it just keeps things simpler. It can also prevent accidents - e.g. when I am in a development client spec, I know I cannot accidentally look at or modify a file of the same name in another branch. This can be a big help when you are down in the depths of a deep source folder hierarchy.
Keeping views "tight" is generally good practice in Peforce - as in anything. It helps server performance, which in turn helps performance of Perforce on your client machine. But beyond performance, I think the concept of only seeing what you need to see - letting the computer do the filtering for you - is just generally a good thing. It lets you concentrate on the job at hand without distraction.
I don't have any difficultly switching workspaces, and there are a number of ways to do this depending on your circumstances and preferences. P4V has a simple drop-down at the top of the left pane, for example. What difficulties are you having switching workspaces?
I am not sure switching between views/worspaces is that easy in
Perforce.
It is easy. Assuming you use the command-line, create a p4config.txt at the root directory of each client-spec (aka view). In each p4config.txt, set the P4CLIENT variable to the name of the client-spec. You can leave the other variables as they are.
Here is my sample p4config.txt:
P4USER=deepaknag
P4PORT=SJCPperforce01:1666
P4CLIENT=deepaknag_fbsd
Then set the P4CONFIG environment variable as follows:
export P4CONFIG=p4config.txt
Now it automatically uses the "correct" client-spec whenever you change client directories. Try issuing:
p4 info
in your client directories to verify.
This is also documented in Perforce Knowledge Base (works with p4 for me).
Here's what Perforce says on the
subject.
Also Eric Sink has a
good discussion.
I create one
view with accesses to many branches.
Since branches appear in everyone's view by default, you want to have a user area off the main line.
Many engineers are more comfortable with the concept of branches, since they exist in all SCM systems. But views may be concerning, if they haven't seen that concept
I am use tortoiseSVN to synchronize our code.
But recent I found that there is something that is not so convenient.
When i modify a file, let's say a.jsp,
and my colleague might also modify this file, a.jsp,
and this may result in conflict, and any one of use need to checkin the his code first,
and the other one will need to update to latest version, and then resolve the
conflict one by one, and this is really error proned.
So i need some function in tortoise SVN, that can lock the a.jsp when i am editing, and prevent the other collegue to modify the file at the same time.
I have tried "lock" function in tortoiseSVN, but it doesn't work,
when i lock the a.jsp file, my colleague still can modify this file at the same time without any promotion and alert, just like " your colleague are modifying this file, please modify until the check in" ...
is there any better solution ?
Thanks in advance !!
Yes, there is a better solution, it consists of 3 parts:
Never lock, you don't need to
Don't work on the same file, or at least the exact same part of the file, at the same time as someone else
If you do, be happy to merge.
Merging is a typical part of using a source control system like SVN. You shouldn't be afraid of it, you should embrace it happily.
Generally, the merge can be automatic, unless you are working in the extra same area. In this case you must make the changes manually (but the diff tool, in TortiseSVN, will help you with this).
I would suggest that if this is happening a lot, you re-evaluate how you are assigning out work within your project.
As mentioned by others, the most flexible workflow is one where you don't need to lock. However, if you really do want to lock a file, Subversion provides this capability.
Add the property svn:needs-lock to the file(s) for which you want to require locking, and set the value to * (I don't think it actually matters what the property value is). Then when checking out or updating, Subversion will mark those file(s) as read-only in your working copy to remind you to lock them first.
You would lock the file with Subversion (I see you already found this option, but it doesn't do anything for files that don't need locking) which will make the file read-write on disk. Then make your changes. When you check in the file, Subversion will automatically release the lock (but there is an option to keep it if you like). Then the file will be read-only in your working copy again.
Note that locking in Subversion is "advisory" only. This means that anybody else can force acquisition of the lock even though somebody else might already have it. In this case, the workflow is still supported because somebody may still need to merge their changes as they would without locking.
Finally, locking files is the best way to deal with non-text files in Subversion, such as image files or MS Word files. Subversion cannot merge these types of files automatically, so advisory locking helps make sure that two people don't try to edit the same file at the same time.
Tortoise has a "merge" option that you might want to try once you update your code with his changes.
There is a practice amongst SVN users (especially agile SVN users) called "The Check-In Dance". This simple practice can cut down immensely on the amount of conflicts you have when checking code into an SVN repo. It goes like this:
When you're ready to check in some changes to the repo:
1. Do an update first to get everyone else's changes.
2. Run your build script (or just compile if you have no build script)
3. If all is well, commit your changes.
Locking causes it's own set of problems, not the least of which is that people tend to forget to "unlock" the file leaving everyone else totally unable to work if they need to change that file.
Merging conflicts in SVN is fairly easy to do, so using locking should become a non-issue for you once you get used to using TortoiseSVN.
Hope this helps,
Lee