git1 repository have a folder named sports which further have subfolders and files inside it.I want to copy the file given by the output of git diff given by command2.
def command2 = "git diff --stat #{12.hours.ago}"
Process process = command2.execute(null, new File('C:/git1'))
def b=process.text
println b
Output of Groovy Console is Sports/Cricket/Players/Virat.txt.
Now i want to use this file path in sourceDir as shown below.
def sourceDir = "C:/git1/$b"
def destinationDir = "D:/git1/$b"
(new AntBuilder()).copy(file: sourceDir, tofile: destinationDir)
This is giving error as Warning: Could not find file C:\git1\Sports\Cricket\Players\Virat.txt
to copy.
As you can see in your comment with the byte array, the last character of the string is a newline character (10). This of course makes the path invalid but is not easily recognizable in the error message. Add a .trim() after the .text and it should work with both, AntBuilder and Files and on both, Windows and Linux. If you would have done it on Windows there would be a 13 additionally before the 10.
Another maybe even better approach would be not to use git diff as you do, as it is a porcelain command. Porcelain command as their name suggests are not stable and their arguments and output can change any time. They are meant for human usage, not for scripts. Scripts should use plumbing commands which are much more stable in input and output. With the respective plumbing command you might not have had your problem.
Related
Upon looping a directory to delete txt files ONLY - a message is returned indicating The System cannot find the file specified: 'File.txt'.
I've made sure the txt files that I'm attempting to delete exist in the directory I'm looping. I've also checked my code and to make sure it can see my files by printing them in a list with the print command.
import os
fileLoc = 'c:\\temp\\files'
for files in os.listdir(fileLoc):
if files.endswith('.txt'):
os.unlink(files)
Upon initial execution, I expected to see all txt files deleted except for other non-txt files. The actual result was an error message "FileNotFoundError: [WinError 2] The system cannot find the file specified: 'File.txt'.
Not sure what I'm doing wrong, any help would be appreciated.
It isn't found because the the path you intended to unlink is relative to fileLoc. In fact with your code, the effect is to unlink the file relative to the current working directory. If there were *.txt files
in the cwd then the code would have unfortunate side-effects.
Another way to look at it:
Essentially, by analogy, in the shell what you're trying to do is equivalent to this:
# first the setup
$ mkdir foo
$ touch foo/a.txt
# now your code is equvalent to:
$ rm *.txt
# won't work as intended because it removes the *.txt files in the
# current directory. In fact the bug is also that your code would unlink
# any *.txt files in the current working directory unintentionally.
# what you intended was:
$ rm foo/*.txt
The missing piece was the path to the file in question.
I'll add some editorial: The Old Bard taught us to "when in doubt, print variables". In other words, debug it. I don't see from the OP an attempt to do that. Just a thing to keep in mind.
Anyway the new code:
Revised:
import os
fileLoc = 'c:\\temp\\files'
for file in os.listdir(fileLoc):
if file.endswith('.txt'):
os.unlink(os.path.join(fileLoc,file))
The fix: os.path.join() builds a path for you from parts. One part is the directory (path) where the file exists, aka: fileLoc. The other part is the filename, aka file.
os.path.join() makes a whole valid path from them using whatever OS directory separator is appropriate for your platform.
Also, might want to glance through:
https://docs.python.org/2/library/os.path.html
I have written a Groovy script to check the existence of a file field1_field2_field3.txt in my unix path.
def fileName = "/path/to/file/field1_field2_field3.txt"
File f = new File(fileName);
if(f.exists())
{
println (" Required files exists.. \n");
}
Now i want to extend this script to check if the files with name field1_field2_*.txt exist.
Kindly let me know if there is a command which can give me the desired list of files or i should look to implement using regular expressions.
I originally suggested using FileNameFinder, but it seems that's not allowed, and for good reason. Groovy scripts are run on the master, so file finds wouldn't happen on the slave where the code is anyway.
You'll probably have to do something along the lines of this:
FileNameFinder().getFileNames fails on one Jenkins node
Perhaps:
def filesListAsString = bat( returnStdout: true, script: '#echo off & dir /b /path/to/file/field1_field2*.txt').trim()
Which should give you a whitespace-delimited String listing of files that match.
I would like to rearrange and rename files.
I have this tree structure of files :
ada/rda/0.05/alpha1_freeSurface.md
ada/rda/0.05/p_freeSurface.md
ada/rda/0.05/U_freeSurface.md
ada/rda/0.1/alpha1_freeSurface.md
ada/rda/0.1/p_freeSurface.md
ada/rda/0.1/U_freeSurface.md
I want that files will be renamed and rearranged like this structure below:
ada/rda/ada-0.05-alpha1.md
ada/rda/ada-0.05-p.md
ada/rda/ada-0.05-U.md
ada/rda/ada-0.1-alpha1.md
ada/rda/ada-0.1-p.md
ada/rda/ada-0.1-U.md
Using the perl rename (sometimes called prename) utility:
rename 's|ada/rda/([^/]*)/([^_]*).*|ada/rda/ada-$1-$2.md|' ada/rda/*/*
(Note: by default, some distributions install a rename command from the util-linux package. This command is incompatible. If you have such a distribution, see if the perl version is available under the name prename.)
How it works
rename takes a perl commands as an argument. Here the argument consists of a single substitute command. The new name for the file is found from applying the substitute command to the old name. This allows us not only to give the file a new name but also a new directory as above.
In more detail, the substitute command looks like s|old|new|. In our case, old is ada/rda/([^/]*)/([^_]*).*. This captures the number in group 1 and the beginning of the filename (the part before the first _) in group 2. The new part is ada/rda/ada-$1-$2.md. This creates the new file name using the two captured groups.
You can use basename and dirname functions to reconstruct the new filename:
get_new_name()
{
oldname=$1
prefix=$(basename $oldname _freeSurface.md)
dname=$(dirname $oldname)
basedir=$(dirname $dname)
dname=$(basename $dname)
echo "$basedir/ada-$dname-$prefix.md"
}
e.g. get_new_name("ada/rda/0.05/alpha1_freeSurface.md") will show ada/rda/ada-0.05-alpha1.md in console.
Then, you can loop through all your files and use mv command to rename the files.
I am trying to create a zsh script to test my project. The teacher supplied us with some input files and expected output files. I need to diff the output files from myExecutable with the expected output files.
Question: Does $iF contain a string in the following code or some kind of bash reference to the file?
#!/bin/bash
inputFiles=~/project/tests/input/*
outputFiles=~/project/tests/output
for iF in $inputFiles
do
./myExecutable $iF > $outputFiles/$iF.out
done
Note:
Any tips in fulfilling my objectives would be nice. I am new to shell scripting and I am using the following websites to quickly write the script (since I have to focus on the project development and not wasting time on extra stuff):
Grammar for bash language
Begginer guide for bash
As your code is, $iF contains full path of file as a string.
N.B: Don't use for iF in $inputFiles
use for iF in ~/project/tests/input/* instead. Otherwise your code will fail if path contains spaces or newlines.
If you need to diff the files you can do another for loop on your output files. Grab just the file name with the basename command and then put that all together in a diff and output to a ".diff" file using the ">" operator to redirect standard out.
Then diff each one with the expected file, something like:
expectedOutput=~/<some path here>
diffFiles=~/<some path>
for oF in ~/project/tests/output/* ; do
file=`basename ${oF}`
diff $oF "${expectedOutput}/${file}" > "${diffFiles}/${file}.diff"
done
I am trying to zip a file using shell script command. I am using following command:
zip ./test/step1.zip $FILES
where $FILES contain all the input files. But I am getting a warning as follows
zip warning: name not matched: myfile.dat
and one more thing I observed that the file which is at last in the list of files in a folder has the above warning and that file is not getting zipped.
Can anyone explain me why this is happening? I am new to shell script world.
zip warning: name not matched: myfile.dat
This means the file myfile.dat does not exist.
You will get the same error if the file is a symlink pointing to a non-existent file.
As you say, whatever is the last file at the of $FILES, it will not be added to the zip along with the warning. So I think something's wrong with the way you create $FILES. Chances are there is a newline, carriage return, space, tab, or other invisible character at the end of the last filename, resulting in something that doesn't exist. Try this for example:
for f in $FILES; do echo :$f:; done
I bet the last line will be incorrect, for example:
:myfile.dat :
...or something like that instead of :myfile.dat: with no characters before the last :
UPDATE
If you say the script started working after running dos2unix on it, that confirms what everybody suspected already, that somehow there was a carriage-return at the end of your $FILES list.
od -c shows the \r carriage-return. Try echo $FILES | od -c
Another possible cause that can generate a zip warning: name not matched: error is having any of zip's environment variables set incorrectly.
From the man page:
ENVIRONMENT
The following environment variables are read and used by zip as described.
ZIPOPT
contains default options that will be used when running zip. The contents of this environment variable will get added to the command line just after the zip command.
ZIP
[Not on RISC OS and VMS] see ZIPOPT
Zip$Options
[RISC OS] see ZIPOPT
Zip$Exts
[RISC OS] contains extensions separated by a : that will cause native filenames with one of the specified extensions to be added to the zip file with basename and extension swapped.
ZIP_OPTS
[VMS] see ZIPOPT
In my case, I was using zip in a script and had the binary location in an environment variable ZIP so that we could change to a different zip binary easily without making tonnes of changes in the script.
Example:
ZIP=/usr/bin/zip
...
${ZIP} -r folder.zip folder
This is then processed as:
/usr/bin/zip /usr/bin/zip -r folder.zip folder
And generates the errors:
zip warning: name not matched: folder.zip
zip I/O error: Operation not permitted
zip error: Could not create output file (/usr/bin/zip.zip)
The first because it's now trying to add folder.zip to the archive instead of using it as the archive. The second and third because it's trying to use the file /usr/bin/zip.zip as the archive which is (fortunately) not writable by a normal user.
Note: This is a really old question, but I didn't find this answer anywhere, so I'm posting it to help future searchers (my future self included).
eebbesen hit the nail in his comment for my case (but i cannot vote for comment).
Another possible reason missed in the other comments is file exceeding the file size limit (4GB).
I converted my script for unix environment using dos2unix command and executed my script as ./myscript.sh instead bash myscript.sh.
I just discovered another potential cause for this. If the permissions of the directory/subdirectory don't allow the zip to find the file, it will report this error. Actually, if you run a chmod -R 444 on the directory, and then try to zip it, you will reproduce this error, and also have a "stored 0%" report, like this:
zip warning: name not matched: borrar/enviar
adding: borrar/ (stored 0%)
Hence, try changing the permissions of the file. If you are trying to send them through email, and those email filters (like Gmail's) invent silly filters of not sending executables, don't forget that making permissions very strict when making zip compression can be the cause of the error you are reporting, of "name not matched".
spaces are not allowed:
it would fail if there are more than one files(s) in $FILES unless you put them in loop
I also encountered this issue. In my case, the line separate is CRLF in my zip shell script which causes the problem. Using LF fixed it.