I have a lot of files within the file structure of the perforce depot that I am unable to see with the perforce clients p4 command line or p4v gui even when logged in as admin.
I have tried to find any meta data I can through p4 files and p4 filelog commands but it always returns:
"- no such file(s)."
Also I have run p4 verify and p4 dbverify to see if there we were any errors on the server but they returned no errors. There just seems to be no records of the files except for the fact that they are taking up room on the HDD.
My current theory is that they are from failed commits but I do not know how to get perforce to acknowledge the files so I can obliterate them.
Background info:
This is a simple perforce setup with just the main depot and an archive depot for old projects. (The mystery files are in the main depot)
The server version is: P4D/NTX64/2012.2/551823 (2012/11/09).
There isn't necessarily a one-to-one mapping between what's in the server's depot filesystem and the actual structure of the depot as defined in the metadata -- depot revisions are written once and are not moved or duplicated even if they're moved or duplicated from the point of view of the client. So you definitely shouldn't make the assumption that because a given file in the depot filesystem doesn't correspond to a depot file path that it's not actually providing the underlying storage for some other existing file (especially if you've used obliterate on some branches of a file while leaving others intact -- the remaining archive file may be the content for one of the ones you left).
That said, it is also possible for archives to become "orphaned" as part of a failed submit as you suggest. If the amount of space involved is small I'd suggest not worrying about it (the orphaned files won't cause any problems in terms of collisions), but if it's important to be able to clean them up, your best bet is to use "snap -n" to make sure there aren't any of those dependencies and then delete them manually (just to be safe I'd keep a backup of them at least until you've run your next verify to make sure nothing important has gone missing). Run:
p4 snap -n //... //depot/path/to/mystery/file
This says "show me files anywhere in the depot (//...) with archive dependencies on //depot/path/to/mystery/file". If you run the command without the -n it will actually break those dependencies by making physical copies (don't do this if you're worried about space since you'll end up with N redundant copies of the archive).
The inverse of p4 snap -n (i.e. "where does the archive for this depot file live?") is p4 fstat -Oc //depot/file.
Related
I recently lost a lot of files, due to a hard disk error, but still have a Perforce folder containing files from a particular project. Trouble is, it's not the actual project files, but some kind of Perforce storage format, with files ending in ",d" and ",v"
Is there any way I can restore the original files from what I have?
I imagined Perforce would be able to open the folder as a depot, and I'd simply be able to get the files into a new workspace, but I can't see any way to open an existing depot. I tried editing the depot's "Storage location for versioned files" to point at the folder, but the depot still shows as empty.
I only used Perforce briefly (comparing it to Git) so I don't really understand how it works. Any help or advice would be much appreciated.
The thing you're missing is the database (the db.* files), which need to be in the server root (P4ROOT) when you start the server. If you don't have database files in P4ROOT, the server will create an empty database on startup. The database is the source of truth for a Perforce server, so if it's empty, the server is empty.
If you have a checkpoint (a file called checkpoint.N) and/or journal file, that contains the metadata you need to reconstruct the database; recover the server with p4d -jr checkpoint.N journal, and then you can use the p4 depot command to make sure the Map of your depot(s) points at the directory where your archive files are, and use other commands to inspect the actual files (start with p4 verify to see which files are in the database but missing/corrupted in the archive backup).
If you only have the archive files but no database (and no checkpoint or journal to recover it from), you're in a more difficult spot since the database is what maps the archive files into the actual depot structure (e.g. it includes all the copy-by-reference pointers that constitute branches in Perforce). However, you can extract the contents of the archive files one file at a time using conventional tools; the ,v files are in RCS format (use the co command to retrieve their content), and the ,d directories contain regular old .gz files, one per revision.
Using the deep magic to synthesize a database from the archive files on their own is also a possibility, but the RCS/CVS conversion scripts are so old I'd expect a lot of fiddling to be required to get them working with a current version of Perforce.
I have a large perforce client already synced. I realized that I'll only be working on a sub directory of the larger root folder I synced to.
Old p4 client specs
//Depot/largeFolder/... //Clinet/largeFolder/...
New p4 client
//Depot/largeFolder/samllSubDir/... //Clinet/largeFolder/smallSubDir/...
Now when I perform a p4 sync perforce doesn't remove the other folders which I don't want in my client.
I'm hesitant to do a p4 sync -f since I have files edited in the /largeFolder/smallSubDir/. I don't want them to get reset.
What is the right way to sync my client so that the other folders which are not in the new mapping are removed?
Note: My real mapping is complicated and deleting folders by hand is not realistic.
In the typical case, re-syncing a narrowed client will indeed remove the unmapped files. You don't say what happens when you try to do a sync (are there error messages?), so I can't say for certain what's going on, but these are the two first possibilities that come to mind:
Opened files will not be touched by a sync, ever. You need to revert them -- use the -k flag if you want to revert the files but keep your local copies, and then use the -k flag on sync as well so that it will continue to not touch them (while marking them as having been synced, or in this case un-synced).
Writable files will by default be left alone by a sync, and produce "can't clobber" warnings. Doing sync -k should suffice here if you want to keep the local files. Doing sync -f will blow away the local files even if they were modified (but only if they're not also open for edit -- see above).
Useful commands to run if you're trying to debug (pick any file that's not getting [un]synced that you think should be):
p4 sync <file> -- do you get an error message that tells you why the file isn't being removed?
p4 where <file> -- is the client mapping for that file not what you think it is?
p4 opened <file> -- is the file opened?
p4 have <file> -- did you even sync this file in the first place?
p4 files <file> -- does this file even exist in the depot?
Note that the fact that opened files will not be touched by a p4 sync means that your edited files will be safe from a sync -f, provided that you actually opened them for edit. If you didn't, run a p4 reconcile and then p4 sync -f is perfectly safe -- but if you don't get to the root cause first (i.e. are the unwanted files also opened for edit?), it might not be helpful either.
In P4, I only want to pull file changed my client workspace from server. But When I use "p4 sync", this command get all files from server. So, How to get files changed from list revision ? with the files were existed on local and not changed, the command P4 sync do not need sync. My command is here:
p4 sync -f //depot/...#Revision
If you want to restore a file you deleted with p4 delete file, use p4 revert file, not p4 sync. p4 sync is for getting the changes that other people made to the files, not for altering the changes that you made.
There are three different reasons that the files in your workspace might differ from the current head revision in the depot:
A new revision has been submitted to the depot since the last time you synced.
You have opened the files (e.g. with p4 edit) and made changes to them that are not yet submitted.
You have modified the read-only synced files in your workspace without opening them.
(You don't say in your question which of these is the case, which is why you've gotten a couple of different answers that are based on different assumptions about your workspace state -- I'm guessing it's #3, personally, but I'll give the answer for each one so you can understand how it's "supposed" to work vs what's actually going on here.)
In case 1, a normal p4 sync command will update only the files with new revisions. If the revision you previously synced is the same as the head revision, p4 sync won't update it. This makes p4 sync very fast -- you can have a million files in your workspace, and if only one of them has changed since you last sync, only that one file is affected.
In case 2, no p4 sync command will update the files, in the interest of preserving your open changes. To submit your changes to the depot, use p4 submit; to discard them, use p4 revert. Again, only the open files are affected by these commands, so a revert operation affecting only a few files isn't slowed down by however many other files are in your workspace.
In case 3, your workspace is in an inconsistent state -- Perforce's standard workflow is that any time you modify a local file you should "open" it so that the server can track your local work (and optimize operations like sync, submit, etc, as well as alert other users who open those files simultaneously to potential conflicts). If you modify a file locally without modifying it, commands like sync will no longer work as well because the state of your client is not known to the server.
You can recover from an inconsistent state by using the p4 reconcile command (which will open all the inconsistent files so that you can either submit or revert them), or the p4 clean command (which irrevocably discards the inconsistent local changes, as if you'd done p4 reconcile immediately followed by p4 revert). These commands are significantly slower than a normal sync operation since they need to scan the entire workspace rather than only the changed files, but they are still significantly faster than a sync -f since only the changed files are actually re-transferred.
- how to uncompress perforce Depot files?
The files I have now ending with [,v] and some files end with [,d] containing [1.1.gz].
What i did In details:
In P4V I created a Workspace, put some important files, Submitted it to the Depot then decided to delete what's in the Depot by clicking Mark for Delete it just mark it with a red X what I think, So I head to C:\Program Files\Perforce\Server\depot and deleting it from there, now the files in the Recycle Bin but doing so doesn't make it disappear from P4V so I opened P4Abmin in the Depot tap I did Obliterate and its gone finally.
Later discovered that Marking files for Delete in the Depot delets it from the Workspace, and only thing that I have is what I restored from the Recycle Bin and it's compressed files, how can I uncompress it.
Don't touch the Perforce server's depot or db files unless you know what you're doing -- normally the server handles the job of managing those files and the relationships between them, and randomly messing with those files will usually break things, much like if you randomly shuffled blocks on your hard disk around without knowing how your filesystem works. I mention this first so that you'll know for next time, and second so that if you happen to have access to a time machine, you can fix this problem by going back and informing your past self to keep their paws out of P4ROOT. :)
If in the future you want to temporarily delete files from the depot, use the normal "Mark for Delete" command in P4V (or p4 delete in the CLI) followed by "Submit". If you want to permanently delete them, that's what the "obliterate" command is for. In neither case should you be deleting files out from under the server -- everything should happen from the client (that is, P4V, the p4 CLI, P4Win, etc).
If you restore the deleted files to exactly where they were, you should be able to rely on Perforce to get the files back, provided you have not already obliterated them from the db. (Hopefully obliterate noticed the archive files were gone and it failed with an error instead of blasting the db entries...)
If you no longer have the db entries for the files, you can try to extract the archives manually with command line tools (luckily the content isn't encrypted or in a weird proprietary format) -- you should be able to gunzip the .gz files and co (RCS) the ,v files. I'd expect most unzip utilities to understand gzip, but RCS is a pretty old format so you may have to do a little digging to find Windows tools for it (I think Cygwin may have RCS tools bundled with it). Good luck!
If you do a sync in perforce it can happen, very often, to do nothing even if it should. If you do a forced sync (sync -f) it will definitely sync all the files from the server blindly and this is unacceptable if you have a big project.
The question is how can you do a fast sync?
The key part to your question is "even if it should". Without fail, in all my (too) many years of experience with Perforce, users who revert to forced syncs habitually, because a normal sync does not do what they expect, are not using Perforce correctly.
You must let Perforce manage your hard disk for you - the server knows which versions of which files you have, and so a normal sync just gets what it needs. If it fails, then that is almost without exception a sign that the user is modifying, deleting, adding, renaming files outside of Perforce (e.g. in Explorer).
Sometimes you may get a system funny (i.e. PC fault, ) that causes this, so it is not necessarily the users fault. But these are very rare - hence my "habitually" qualifier above.
If you think, hand on heart, that you are not doing anything out of the ordinary, then start to look at what other processes could be tweaking these files. E.g. does you build system do something odd, and try to modify files as it builds? Do you have an open network share that perhaps someone else is inadvertently poking? Do you have a backup system that is interfering?
Hope that helps.
As mentioned by the other answers, you need to create a separate client (workspace) for every "instance" of a directory tree synced from the depot.
If you want to force-sync only those files that have changed from the depot:
p4 diff -se ... | p4 -x - sync -f
p4 diff -sd ... | p4 -x - sync -f
Warning: that will overwrite any locally changed files (in the current directory and subdirectories).
If instead you edited some files and forgot to open them for edit first, you can do this:
p4 diff -se ... | p4 -x - edit
which will open for edit any file that's different from the depot.
And this:
p4 diff -sd ... | p4 -x - delete
will open for delete any file that was locally deleted.
If you create an inconsistency, where perforce looses track of what is going on in your workspace, you should use the "Reconcile offline work..." function (in P4V).
This will run a folder diff and tell you what options you have to resolve the inconsistencies.
In general, make sure that you always check out files explicitly through perforce (open for edit). This feels strange if you are used to other systems.
I believe that the command you're looking for is p4 clean.
From: http://www.perforce.com/perforce/r14.2/manuals/cmdref/p4_clean.html
"Files present in the workspace, but missing from the depot are deleted from the workspace.
Files present in the depot, but missing from your workspace. The version of the files that have been synced from the depot are added to your workspace.
Files modified in your workspace that have not been checked in are restored to the last version synced from the depot."