I am trying to check if a file exits or not in the specified directory. If it is, then I would move the file to another directory. Here is my code
def move(pnin, pno):
if (os.path.exists(pnin)):
shutil.move(pnin, pno)
here is an example of pnin and pno
pnin='D:\\extracted\\extrimg_2016000055202500\\2016000055202500_65500000007006_11_6.png'
pno=D:\folder\discarded
I have a bit more than 8000 input directories. I copied this pnin from the output of print(pnin).When I define pnin externally as in the example, the if statement works. But when I want to run 'move' function iteratively, if statement is never executed. What could be the problem and how can I solve this?
Here is how I call move function:
def clean_Data(inputDir, outDir):
if (len(listf) > 1):
for l in range(1,len(listf)):
fname = hashmd5[m][l]
pathnamein = os.path.join(inputDir, fname)
pathnamein = "%r"%pathnamein
pathnameout = outfile
move(pathnamein, pathnameout)
When I try below code it does not give any output. For loop şs working. When I use print(pathnamein) in the for loop it shows all the values of pathnamein.
def move(pnin, pno):
os.path.exists(pnin)
You should use backslash to escape backslashes in your pno string:
pno='D:\\folder\\discarded'
or use a raw string instead:
pno=r'D:\folder\discarded'
Otherwise \f would be considered a formfeed character.
Related
I'm writing a program to concatenate 3 csv files into a single new output csv file. As part of this I have to ask the user for the name they want to use for the output filename. My problem is that my output filename is declared in the arguments of the function and therefore, I get an error on the first line of this function because myFile is not declared (see 2 lines later for the declaration).
def concatenate(indir="C:\\Conc", outfile="C:\\Conc\\%s.csv" %myFile):
os.chdir(indir)
myFile = input("Please type the name of the file: ")
fileList=glob.glob("*.csv")
dfList=[]
colnames=["Symbol", "Name", "LastSale", "MarketCap", "ADR TSO", "IPOYear", "Sector", "Industry", "Summary Quote", " "]
for filename in fileList:
print("merging " + filename + "...")
df=pandas.read_csv(filename, skiprows=1,header=None)
dfList.append(df)
concatDf=pandas.concat(dfList, axis=0)
concatDf.columns=colnames
concatDf = concatDf.sort_values(by=["Symbol"], axis=0, ascending=True)
concatDf.to_csv(outfile, index=None)
print("Completed. Your merged file can be found at: " + outfile + "\n")
This function is called from a menu function (as below) so I was wondering if it's possible to pass it from the menu??
if choice == "1":
myFile = input("Please type the name of the file: ")
concatenate(myFile)
menu()
But neither options seem to work because I always get an error saying that myFile is not declared. I know this is basic stuff but I'm scratching my head trying to figure it out.
Your reputation suggests you're new to stack overflow and your name suggests you're new to python. So welcome to both! Here's a quite verbose answer to hopefully make things clear.
There are a few issues here:
concatenate() takes two arguments, indir and outdir, but you're only passing one argument when you're calling it: concatenate(myFile). Your arguments are what is called keyword arguments, since you've given them names (i.e. indir and outdir). When you're only passing one argument (myFile), without using the keyword, the myFile variable is passed as the first argument, which, in this case, is the indir (this doesn't happen yet in your code, as the error you're getting precedes it and stops it from being executed). However, you seem to want the myFile variable assigned to your outfile argument. You can achieve this by being explicit in your call to concatenate() like so: concatenate(outfile=myFile). Now, your indir argument remains the default you've set (i.e. "C:\\Conc"), but your outfile argument has the value of myFile. This would, however, not fix your problem (detailed in point 2). I suggest you change the outfile argument to represent an output directory (see full example below).
This is where you get an error. Your concatenate() function has no knowledge of the myFile variable you've declared in your if-statement, as the function and your if-statement are different scopes. That's why function arguments exist, to pass data from one scope (your if-statement) to another scope (your function). So you get an error in your function saying that myFile is not declared because, to the function's eyes, it doesn't exist. It can only see indir and outfile, not myFile. Also, don't do string manipulation in arguments. I'm not sure how it works, but it surely doesn't work how you expect in this case.
You're asking the user for myFile twice. Once in the if-statement and once in your concatenate() function. You only need to ask for it once, I suggest to keep it in the function only.
Small correction: You should combine directory paths and filenames in a platform-independent manner. The os module has a function for that (os.path.join(), see example below). Paths in Windows and paths in Linux look different, the os module handles those differences for us:)
Here's a final suggestion, with comments, addressing all points:
# Import your modules
import pandas
import os # <-- We're gonna need this
# Your code
# ...
# The concatenate function, with arguments having a default value, allowing to optionally specify the input directory and the output *directory* (not output file)
def concatenate(indir="C:\\Conc", outdir="C:\\Conc"):
os.chdir(indir)
myFile = input("Please type the name of the file: ") # <-- We ask for the output filename here
# Your code
fileList=glob.glob("*.csv")
dfList=[]
colnames=["Symbol", "Name", "LastSale", "MarketCap", "ADR TSO", "IPOYear", "Sector", "Industry", "Summary Quote", " "]
for filename in fileList:
print("merging " + filename + "...")
df=pandas.read_csv(filename, skiprows=1,header=None)
dfList.append(df)
concatDf=pandas.concat(dfList, axis=0)
concatDf.columns=colnames
concatDf = concatDf.sort_values(by=["Symbol"], axis=0, ascending=True)
# Saving the data
# First, we need to create the full output path, i.e. the output directory + output filename. We use os.join.path() for that
output_path = os.path.join(outdir, myFile)
# The rest of your code
concatDf.to_csv(output_path, index=None) # <-- Note output_path, not outfile
print("Completed. Your merged file can be found at: " + output_path + "\n")
# The if-statement calling the ´concatenate()´ function
if choice == "1":
# We're calling concatenate() with no arguments, since we're asking for the filename within the function.
# You could, however, ask the user for the input directory or output directory and pass that along, like this:
# input_directory_from_user = input("Please type the path to the input directory: ")
# output_directory_from_user = input("Please type the path to the output directory: ")
# concatenate(indir=input_directory_from_user, outdir=output_directory_from_user)
concatenate()
menu()
I need to use the R extension in NetLogo to do some network calculations. I am creating the network in NetLogo, exporting it to a text file, having R read the text file and construct a graph and calculate properties, then getting the calculation results. The export, read, calculate and get are being controlled by NetLogo through the R extension.
However, NetLogo and R have different default working directories. The problem I have about changing directories in R breaking the connection to the extensions (see R extension breaks connection to extensions directory in NetLogo) is affecting my attempts to use BehaviorSpace on the model.
My new approach is to not change the R working directory, but simply to provide the full path to R of the exported file.
r:clear
let dir pathdir:get-model
r:eval "library(igraph)"
; read network in R (avoid bug of R change working directory)
let runstring (word "r:eval \"gg <- read_graph(file = \"" dir "\\netlogo.gml\", format = \"gml\")\"")
print runstring
run runstring
This produces the correct string to run, output from print statement:
r:eval "gg <- read_graph(file = "C:\Users\Jen\Desktop\Intervention Effect\netlogo.gml", format = "gml")"
But I get an error on the run runstring that this nonstandard character is not allowed. Debugging by putting my constructed string into the run command directly, I have realised it is because I am now in a string environment and have to escape ('\') all my backslashes and quotes. That is, the command that would work if directly typed or included in the NetLogo code, will not work if it is provided as a string to be run.
I haven't yet been able to construct a string to put into the line run runstring that works, even by hand. This means I don't know what the string looks like that I am trying to create. Having identified the appropriate target string, I will need code to take the variable 'dir', convert it to a string, add the various \ characters to the dir, add the various \ characters to the quotes for the rest of the command, and join it so that it runs.
Can anyone provide some bits of this to get me further along?
Still struggling with this
I am now trying to work backwards. Find a string that works and then create it.
If I hard code the run command as follows, NetLogo closes. Even though if I copy the text between the quotes and enter it directly into R, it does what is expected.
let Rstring "gg <- read_graph(file = 'C:\\Users\\Jen\\Desktop\\Intervention Effect\\Networks\\netlogo.gml', format = 'gml')"
r:eval Rstring
The pathdir option ended up working. Here is example code for anyone who has a similar problem in the future.
let filename (word "Networks/netlogo" behaviorspace-run-number ".gml")
export-simple-gml filename
r:clearLocal
let dir pathdir:get-model
set filename (word dir "/" filename)
r:put "fn" filename
r:eval "gg <- read_graph(file = fn, format = 'gml')"
r:eval "V(gg)$name <- V(gg)$id" ; gml uses 'id', but igraph uses 'name'
I have a separate procedure for the actual export, which constructs a simplified gml file because the igraph import of gml format files is somewhat crippled. That's the procedure called within the code above, and the relevant piece is:
to export-simple-gml [ FN ]
carefully [ file-close-all ] [ ]
carefully [ file-delete FN ] [ ]
file-open FN
file-print <line to write to file>
...
end
I need to get help for a pattern in Lua stopping to read after a line break.
My code:
function getusers(file)
local list, close = {}
local user, value = string.match(file,"(UserName=)(.*)")
print(value)
f:close()
end
f = assert(io.open('file2.ini', "r"))
local t = f:read("*all")
getusers(t)
--file2.ini--
user=a
UserName=Tom
Password=xyz
UserName=Jane
Output of script using file2.ini:
Tom
Password=xyz
UserName=Jane
How to get the pattern to stop after it reaches the end of line?
You can use the pattern
"(UserName=)(.-)\n"
Note that besides the extra \n, the lazy modifier - is used instead of *.
As #lhf points out, make sure the file ends with a new line. I think you can append a \n to the string manually before matching.
I would like to take a string representing a file path, strip off the file name and save just the path.
For example, if I have:
"/folder1/folder2/file.name"
I would like to end up with "/folder1/folder2/" in my string.
I've been playing around with string.match() as documented here: http://lua-users.org/wiki/StringLibraryTutorial
I have the following code:
mystring = "/var/log/test.log"
print(string.match(mystring, "%/"))
When I run this script, I end up with just a '/' returned.
I was expecting that it would return the positions of the two '/' in the string.
I've also tried replacing the pattern "%/" with just "/" but that gives me the same results.
I'm sure I'm missing something very simple but I can't see what it is.
By the pattern %/ or /, you are telling string.match to look for a string / and that's what you got. Try with this:
local mystring = "/var/log/test.log"
print(string.match(mystring, ".+/"))
Here the pattern .+/ means to look for one or more whatever characters(.) followed by a /. + means it's greedy, i.e, match as long as possible.
Try any this options:
local mystring = "/var/log/test.log"
print(mystring:gsub('([%w+]+%.%w+)$',''))
output: /var/log/
local mystring = "/var/log/test.log"
print(mystring:match('^(/.+/)'))
output: /var/log/
Or too a simple function
function anystring(string)
local string = string:match('^(/.+/)')
if anystring then
return string
else
return false
end
end
local mystring = anystring('/var/log/test.log')
print(mystring)
output: /var/log/
You can be as specific as possible when putting the pattern to avoid code errors
this question about matlab:
i'm running a loop and each iteration a new set of data is produced, and I want it to be saved in a new file each time. I also overwrite old files by changing the name. Looks like this:
name_each_iter = strrep(some_source,'.string.mat','string_new.(j).mat')
and what I#m struggling here is the iteration so that I obtain files:
...string_new.1.mat
...string_new.2.mat
etc.
I was trying with various combination of () [] {} as well as 'string_new.'j'.mat' (which gave syntax error)
How can it be done?
Strings are just vectors of characters. So if you want to iteratively create filenames here's an example of how you would do it:
for j = 1:10,
filename = ['string_new.' num2str(j) '.mat'];
disp(filename)
end
The above code will create the following output:
string_new.1.mat
string_new.2.mat
string_new.3.mat
string_new.4.mat
string_new.5.mat
string_new.6.mat
string_new.7.mat
string_new.8.mat
string_new.9.mat
string_new.10.mat
You could also generate all file names in advance using NUM2STR:
>> filenames = cellstr(num2str((1:10)','string_new.%02d.mat'))
filenames =
'string_new.01.mat'
'string_new.02.mat'
'string_new.03.mat'
'string_new.04.mat'
'string_new.05.mat'
'string_new.06.mat'
'string_new.07.mat'
'string_new.08.mat'
'string_new.09.mat'
'string_new.10.mat'
Now access the cell array contents as filenames{i} in each iteration
sprintf is very useful for this:
for ii=5:12
filename = sprintf('data_%02d.mat',ii)
end
this assigns the following strings to filename:
data_05.mat
data_06.mat
data_07.mat
data_08.mat
data_09.mat
data_10.mat
data_11.mat
data_12.mat
notice the zero padding. sprintf in general is useful if you want parameterized formatted strings.
For creating a name based of an already existing file, you can use regexp to detect the '_new.(number).mat' and change the string depending on what regexp finds:
original_filename = 'data.string.mat';
im = regexp(original_filename,'_new.\d+.mat')
if isempty(im) % original file, no _new.(j) detected
newname = [original_filename(1:end-4) '_new.1.mat'];
else
num = str2double(original_filename(im(end)+5:end-4));
newname = sprintf('%s_new.%d.mat',original_filename(1:im(end)-1),num+1);
end
This does exactly that, and produces:
data.string_new.1.mat
data.string_new.2.mat
data.string_new.3.mat
...
data.string_new.9.mat
data.string_new.10.mat
data.string_new.11.mat
when iterating the above function, starting with 'data.string.mat'