How do I full text index/search source code in our Perforce depot? - perforce

We use Perforce, recent build (2015.2, we could update to newer if need), on Win2012-R2 server.
We would like full text search to easily find strings across the code base (.cs, .aspx, .less, .resx, etc)
What is an appropriate approach?

The answer should be: p4search
However, where it could be a cool 20-30 minute setup, plan on taking 4+ hours over 1-2 days.
Worse, p4search has been abandoned by perforce. It does not install our run properly on current linux (e.g. Centos 7), requiring edits to startup scripts and general hassle. The indexer gets hung on some files, rdeep diagnostics.
After screwing around w it for a week, we gave up.
So the answer is:
1) Use Open Grok: https://opengrok.github.io/OpenGrok/
or
2) Prepare for a long haul to set up p4search (which, fwiw, also requires perforce Swarm, so you will be setting that up also)

There are lots of such tools out there. Here are two you could start investigating:
Open Grok: https://opengrok.github.io/OpenGrok/
P4Search: https://www.perforce.com/company/newsletter/2014/02/p4-search-search-perforce-server

There is also the built-in p4 grep (perforce docs), with some limitations, see e.g. this good answer to another question, in the server since 2010.1.

Related

P4V not seeing new submits after OS recovery

I've googled this, and saw that someone had a similar problem here -> stackoverflow, but as luck would have it, unresolved.
I have an issue where my P4 client is not able to see or access any submitted files that are newer than 4 months old.
Background - My root directory/db files are on D drive which is undisturbed. I got a nasty virus yesterday, so I checked in all my workspace stuff and recovered to a December drive of my OS. Now that I've successfully booted up in my December version of my OS (where the same version P4V and P4A is already installed), my P4V is only able to see and access files up to 12/15/2015 - nothing after. The drive image I recovered from is dated 12/19/2016. Yet, I can physically inspect all my post Dec P4 checkins in the physical location of my DB on D drive. It's ALL THERE.
Here's really interesting info - out of curiosity, I recovered back to my Virus laden OS from yesterday. Opened up P4V, and it is able to see and access all my files up till my very last submitted file which was checked in yesterday.
Other important into
- The depot, stream, and workspace that my p4 is using is the same between my recovered Dec recovery OS, and yesterday's Virus laden OS. Nothing was ever changed in my P4 settings.
System info:
Windows 10
P4V and P4A - NTX64/2014.3/1007540 (for both my recovered windows image from Dec, and yesterday's virus laden windows)
P4 is my achilless heel. Help appreciated. I will not take offense if you explain things to me like a 3rd grader.
Cheers,
Paul
First off, I used incorrect terminology in my original question. I had my depots on D drive and everything else in their default locations on C.
Where I had it wrong: in the belief that as long as I had my depot files backed up to a safe location, I was protected. There is more to it than that :|
What went wrong: When my PC crashed, I recovered to a 5 month earlier December version of my Windows install which means (obviously) I have December's install of P4 with database files only aware of files I checked in up to December. I did indeed have 1000s of files check in after Dec, all alive and intact, but P4 is simply unaware of it. An assumption (which I had) that P4 simply peeks into the Depots to stay up to date of their contents is an incorrect and dangerous one.
To those who may benefit: For those do-it-yourselfers, these are the 2 things in Perforce you should be backing up if you want to be able to do a full recovery later down the line when things go bad:
Database files. By default, they live in the Perforce application’s “server” directory. For me, there are 67 database files in my C:\Program Files\Perforce\Server. They are all prefixed with db.* and it’s these files that P4 uses to “know” the current state of things, like, your changlists, checked-out files, all your workspaces and their settings, depots and their settings, etc, etc etc. Oh, and what files you even have in your depot! Unfortunately, when backing up database files, you can’t just copy/paste them, but must use command line commands to properly generate 2 other file types (journal, checkpoint) specifically for recovery purposes. It is these files that you'll generate new database files from in a recovery operation.
Depot directories. Depots specify directory trees that your files get stored to, and checked out from when working in your workspace. Files stored here won’t be in their native format, and non-binary files will have their contents tweaked on top of that. Depot directory trees can be manually copied/paste to a backup location.
Proper backup/recovery information can be found here. Like many things, it’s simple once you understand it, but reading this is a bit of a mouth full. It would have helped me to just see slightly fleshed out sample commands along with the reading material, so I’ve included some here. These are the bare minimum command line commands (plus other manual stuff), and will recover a checkpoint, not checkpoint + journal (again, take a look at the link to get a better understanding). Also, I would recommend practicing backup/recovery in a safe throw away environment (or throw away assets) to get the hang of this before doing it for real.
---------- Backup ------------
Despite all the material in the link, it only takes a bare minimum of 3 command line commands (plus other non-command stuff) to do a proper P4 backup, and 1 cmd (among other things) to recover on my end. Again this is only a partially fleshed out sample to illustrate what the command prompt portion of backup/recovery looks like. There is a little more to it than simply running these to get you where you want.
(note: On Windows, you may need to start a windows command prompt as administrator)
>p4 set P4USER=superuser_name
>p4 -q verify //...
>p4d -r "C:\Program Files\Perforce\Server" -jc <yourBackupDirectory>\<prefixOfYourChoosing>
---------- Recovery ------------
>p4d -f -r "C:\Program Files\Perforce\Server" -jr <yourBackupDirectory>\<prefixYouChose>.ckp.1
In my case, I had to use the -f flag due to errors when not using it.
Cheers,
Paul W

Tabcompletion and docview while editing rc.lua

I saw that there is a lua plugin for eclipse and there is a docpage on the awesome main page api_doc and all the .lua files in /usr/share/awesome/lib.
So I thought it must be possible to create a Library or Execution Environment so that one has tabcompletion and docview.
So I tried making my own Execution Environment:
wrote the standard .rockspec file
downloaded the documentation made an ofline version of it and put it in docs/ folder
ziped the files and folders in /usr/share/awesome/lib
ziped all up
tried it out ... and it failed.
When I try to view a documentaion for a .lua file I get "Note: This element has no attached documentation."
Questions: Am I totaly wrong in my doing (because I have the feeling I am)? Is there any way to edit the rc.lua with tabcompletion and docview?
Koneki will probably take a while to setup, but it's definitly worth it. Going for the".doclua"(by using version 1.2) would certainly make it, but I doubt that using a script to generate the information you need, would work out on the long run.
Most likely, you'll probably pass a bit of time to define what kind of object you're dealing with every time you come across one. The right to do, would be to actually take the time to see if the object/module/inner type inherit from an another object, so can actually have more completion feature as you keep using autocomplete to go from one object to another by pressing "dot"+ctrl_space.
In an ideal world, one person could probably make it right and share to other, so they can enjoy a full featured autocomplete editor.
Found solution for eclipse.
First off the idea of setting up an Execution environment was the wrong one.
So the whole thing about downloading the doc although.
For more information on that visit eclipse Wiki for LUA Development Tool.
The right thing to do is to add a source folder which contains the /usr/share/awesome/lib directory.
The bad news is that my comment from above was totally right, which means one has to configure each .lib file in /usr/share/awesome/lib to meet the requirements of the Documentation Language described here.
Than editing the rc.lua (which one can add to the project in eclipse) works with tabcompletion and doc view.
Since the Documentation Language used in the lib files is similar to the one used by "LUA Development Tool" one has not to change many things. Maybe there are even scripts for that.

shell script to create backup file when creating new file in particular directory

Recently I was asked the following question in an interview.
Suppose I try to create a new file named myfile.txt in the /home/pavan directory.
It should automatically create myfileCopy.txt in the same directory.
A.txt then it automatically creates ACopy.txt,
B.txt then BCopy.txt in the same directory.
How can this be done using a script? I may know that this script should run in crontab.
Please don't use inotify-tools.
Can you explain why you want to do?
Tools like VIM can create a backup copy of a file you're working on automatically. Other tools like Dropbox (which works on Linux, Windows, and Mac) can version files, so it backs up all the copies of the file for the last 30 days.
You could do something by creating aliases to the tools you use for creating these file. You edit a file with the tools you tend to use, and the alias could create a copy before invoking a tool.
Otherwise, your choice is to use crontab to occasionally make backups.
Addendum
let me explain suppose i have directory /home/pavan now i create the file myfile.txt in that directory , immediately now i should automatically generate myfileCopy.txt file in the same folder
paven
There's no easy user tool that could do that. In fact, the way you stated it, it's not clear exactly what you want to do and why. Backups are done for two reasons:
To save an older version of the file in case I need to undo recent changes. In your scenario, I'm simply saving a new unchanged file.
To save a file in case of disaster. I want that file to be located elsewhere: On a different computer, maybe in a different physical location, or at least not on the same disk drive as my current file. In your case, you're making the backup in the same directory.
Tools like VIM can be set to automatically backup a file you're editing. This satisfy reason #1 stated above: To get back an older revision of the file. EMACs could create an infinite series of backups.
Tools like Dropbox create a backup of your file in a different location across the aether. This satisfies reason #2 which will keep the file incase of a disaster. Dropbox also versions files you save which also is reason #1.
Version control tools can also do both, if I remember to commit my changes. They store all changes in my file (reason #1) and can store this on a server in a remote location (reason #2).
I was thinking of crontab, but what would I backup? Backup any file that had been modified (reason #1), but that doesn't make too much sense if I'm storing it in the same directory. All I would have are duplicate copies of files. It would make sense to backup the previous version, but how would I get a simple crontab to know this? Do you want to keep the older version of a file, or only the original copy?
The only real way to do this is at the system level with tools that layer over the disk IO calls. For example, at one location, we used Netapps to create a $HOME/.snapshot directory that contained the way your directory looked every minute for an hour, every hour for a day, and every day for a month. If someone deleted a file or messed it up, there was a good chance that the version of the file exists somewhere in the $HOME/.snapshot directory.
On my Mac, I use a combination of Time Machine - which backs up the entire drive every hour, and gives me a snapshot of my drive that stretches back over a year and a half) and Dropbox which keeps my files stored in the main Dropbox server somewhere. I've been saved many times by that combination.
I now understand that this was an interview question. I'm not sure what was the position. Did the questioner want you to come up with a system wide way of implementing this, like a network tech position, or was this one of those brain leaks that someone comes up with at the spur of the moment when they interview someone, but were too drunk the night before to go over what they should really ask the applicant?
Did they want a whole discussion on what backups are for, and why backing up a file immediately upon creation in the same directory is a stupid idea non-optimal solution, or were they attempting to solve an issue that came up, but aren't technical enough to understand the real issue?

Easier navigation through filesystem in linux shell (cli)

I often find myself taking a lot of time to navigate through my filesystem when using the linux shell. This generally occurs because the autocompletion of bash only works if you provide the start of the file/dirname. What I often end up in is a lot of 'ls' with 'grep' commands, finally doing a 'cd'.
When you use a GUI based filebrowser (like Nautilus) you can type any part of a file/dirname and it will have matches that it jumps to directly. This makes it a lot easier and faster to navigate.
I wonder if anybody knows any great tools that helps with this problem. I know of the existence of Midnight Commander, though I never really used it for real and I couldn't figure out a direct solution for my problem the first couple of times I tried it. Also it seems not suitable because I want to have my shell's current working directory to be changed so I can do stuff there, instead of being stuck in an external program like Midnight Commander.
Try autojmp
https://github.com/joelthelion/autojump
And following article provides another solution
http://jeroenjanssens.com/2013/08/16/quickly-navigate-your-filesystem-from-the-command-line.html
You can first use the autocd or autopushd in zsh. You would just have to type the directory you want to go to, without the hassle of typing cd or pushd everytime.
You also have the globing possibility. For example, if I got those file in a directory:
1-a.tar
1-b.tar
c.tar
I can just type
*a.tar
without caring about the beginning of the file.
As a last solution you can always use an alias to the find command with a personalized option.

Create a "project" in VIM/MacVim/GVIM

I'm a VIM noob, and have revisited it time and again, and I'm hoping to actually stick with it this time. Primarily I'm programming in TextMate with Ruby on Rails. On advice from someone, I have installed Janus (https://github.com/carlhuda/janus) and its helping a lot. But one thing I miss is having a "project" so that I can easily get back into a project quickly.
I want to be able to start a copy of macvim, pointing it to a file, or giving it a command, to load a project back to where I last left it. This means:
CD to the root of the project
Set back up any tabs / splits I had set, at their same locations
Reopen the files I was working on last.
I'm going to explore Conque Shell today (http://code.google.com/p/conque/) and if that works, I would want those shells to also reload and fire off their startup commands. (CD to the project root, fire up the rails server, tail a log, etc.)
Suggestions? I'm looking to streamline my process so that I can just click a shortcut or run a command and after a few seconds be staring at my dev environment right where I left it last.
Bonus: I often use windows too, so being able to do the same in GVim would be great as well.
Thanks for your help
You may want to check out Vim's built-in ability to create a restore sessions. These allow you to save pretty much everything you have setup including cwd, folds, splits, buffers, vars etc. See :help :mksession.
Here are two plugins that help with saving and restoring sessions:
sessionman: http://www.vim.org/scripts/script.php?script_id=2010
session.vim: http://www.vim.org/scripts/script.php?script_id=3150
You might also want to check out the project plugin: http://www.vim.org/scripts/script.php?script_id=69
I too have heard good things about rails.vim.
For Rails developer, there is a well-known plugin by Tim Pope named rails.vim.
Once you are at the root of a rails project (You can change Vim current directory with :cd /path/to/project/root ), rails.vim provides quick way to access your file like :
:Rcontroller file
:Rview file
:Rstylesheet file
They are other options to refactor using partials. Install it and type :help rails.txt. There is plenty of nice features like that. It is really useful to speed up access to your project files.
You can probably combine it with session.vim which provides a way to restore your previous session automatically.
If you don't want to type the path of your project, one possible solution, is to add at the end of your .vimrc the following code :
if isdirectory("~/workspace/project1")
cd ~/workspace/project1
endif
This way you always start Vim into your current workspace. Obviously if you need to switch to another directory you have to manually edit your .vimrc... which is kinda sub optimal.
Terminitor (a Ruby Gem) won't specifically solve your vim "project" goal, but it will solve the rest of your problems. You can setup your terminal windows and then execute a command to capture the terminal positions and sizes, edit the configuration to add any other commands (in Ruby) that you want executed and this will allow you to restore your environment.

Resources