Apologies in advance as I am very new to programming (and stackoverflow), and was not sure how to phrase the title for this question.
I have a db.sql file with some outdated URLS.
I would like to update those URLs to new ones using a Python script.
The old URLs don't share any similarities with the new URLs.
Example: "http://old.com/gs7Ubvg" needs to become "http://new.com/static.file"
I have created two text files, one with the old URLs and one with the new URLs.
So far I have managed to make a python script that uses the text files to associate the "old" URL with the "new", but am completely stumped at how to write this change to the db.sql file.
Here is what I have so far...
with open('old_urls.txt') as file:
old_url = file.readlines()
file.close()
with open('new_urls.txt') as file:
new_url = file.readlines()
file.close()
with open("db.sql", "rt") as fin:
with open("db2.sql", "wt") as fout:
for x in range(len(old_url)):
old = old_url[x]
new = new_url[x]
print(old, "is now", new, end="")
for line in fin:
fout.write(line.replace(old,new))
fout.close()
fin.close()
The "print" part is a test and it works. The old URL and new URL match up. But trying to write that change to db2.sql does nothing. If I change it to...
fout.write(line.replace("literal","text"))
... it replaces the text in db2.sql just fine. How do I do this using the contents of the variables "old" and "new" instead of literal text?
Am I missing a loop?
Related
Good afternoon, I am new to stack overflow so I apologize in advance if my question is not in the right format.
I have a list of URLs such as these (but many more),
master_urls =
['https://www.sec.gov/Archives/edgar/daily-index/2020/QTR1/master.20190102.idx',
'https://www.sec.gov/Archives/edgar/daily-index/2020/QTR1/master.20190103.idx]
and I want to write the content onto one single txt.file.
Using one of these URLs works perfectly fine. I do the steps below to achieve it:
file_url = r"https://www.sec.gov/Archives/edgar/daily-index/2019/QTR2/master.20190401.idx"
content = requests.get(file_url).content
with open('master_20190401.txt', 'wb') as f:
f.write(content)
The txt.file looks like this (this is just a small sample of the text file, but it's all the same as shown below just with different company names ...etc):
CIK|Company Name|Form Type|Date Filed|File Name
--------------------------------------------------------------------------------
1000045|NICHOLAS FINANCIAL INC|8-K|20190401|edgar/data/1000045/0001193125-19-093800.txt
1000209|MEDALLION FINANCIAL CORP|SC 13D/A|20190401|edgar/data/1000209/0001193125-19-094732.txt
1000228|HENRY SCHEIN INC|4|20190401|edgar/data/1000228/0001209191-19-021970.txt
1000275|ROYAL BANK OF CANADA|424B2|20190401|edgar/data/1000275/0001140361-19-006199.txt
I tried the following code to get the content of all URLs onto one text file
for file in master_urls:
content = requests.get(file).content
with open('complete_list.txt', 'w') as f:
f.write(content)
but it does not work.
Can anyone help me get the content of each URL in my list of URLs onto one single text file?
Thank you in advance.
Since you are opening your file inside the loop for every URL, the file is getting overwrriten.
try this :
with open('complete_list.txt', 'wb') as f:
for url in master_urls:
content = requests.get(url).content
f.write(content)
I am trying to make a game, where your score saves as a text file. The score (clicks) must always be on the second line and save per user. Every time a user saves, I would like the second line of the text file to be replaced with the new score.
I have tried using loads of things suggested on stack overflow, like the os.replace or os.resub, but none work.
def save():
global userlog
global clicks
score = open(directory + "/" + userlog + ".txt", "r+")
#### On this line, I want some code that will replace the second line in the text file listed above.
for i in range(random.randint(2,5)):
print("Saving")
time.sleep(0.10)
print("Saving.")
time.sleep(0.10)
print("Saving..")
time.sleep(0.10)
print("Saving...")
time.sleep(0.10)
print("\nGame Saved Sucessfully!")
I have not had anything work. Just getting some standard error messages.
Any help will be appreciated :)
Thanks :)
an illustration of my comment - your save function could do something like
# load previously logged information
with open(logfile, 'r') as fobj:
log = fobj.readlines()
# replace line 2 with some new info
log[1] = 'some new info\n'
# overwrite existing logfile
with open(logfile, 'w') as fobj:
for line in log:
fobj.write(line)
In principle you could also use open() in r+ mode as you wrote in the question. That would require you to use seek() (see e.g. here) to get the file pointer to the position you want to write at - a more complicated option which I would not recommend.
I have a CSV file with two columns in it, the one of the left being an old string, and the one directly to right being the new one. I have a heap of .xml files that contain the old strings, which I need to replace/update with the new ones.
The script is supposed to open each .xml file one at a time and replace all of the old strings in the CSV file with the new ones. I have tried to use a replace function to replace instances of the old string, called 'column[0]' with the new string, called 'column[1]'. However I must be missing something as this seems to do nothing. If I the first variable in the replace function to an actual string with quotation marks, the replace function works. However if both the terms in the replace function are variables, it doesn't.
Does anyone know what I am doing wrong?
import os
import csv
with open('csv.csv') as csv:
lines = csv.readline()
column = lines.split(',')
fileNames=[f for f in os.listdir('.') if f.endswith('.xml')]
for f in fileNames:
x=open(f).read()
x=x.replace(column[0],column[1])
print(x)
Example of CSV file:
oldstring1,newstring1
oldstring2,newstring2
Example of .xml file:
Word words words oldstring1 words words words oldstring2
What I want in the new .xml files:
Word words words newstring1 words words words newstring2
The problem over here is you are treating the csv file as normal text file not looping over the all the lines in the csv file.
You need to read file using csv reader
Following code will work for your task
import os
import csv
with open('csv.csv') as csvfile:
reader = csv.reader(csvfile)
fileNames=[f for f in os.listdir('.') if f.endswith('.xml')]
for f in fileNames:
x=open(f).read()
for row in reader:
x=x.replace(row[0],row[1])
print(x)
It looks like this is better done using sed. However.
If we want to use Python, it seems to me that what you want to do is best achieved
reading all the obsolete - replacements pairs and store them in a list of lists,
have a loop over the .xml files, as specified on the command line, using the handy fileinput module, specifying that we want to operate in line and that we want to keep around the backup files,
for every line in each of the .xml s operate all the replacements,
put back the modified line in the original file (using simply a print, thanks to fileinput's magic) (end='' because we don't want to strip each line to preserve eventual white space).
import fileinput
import sys
old_new = [line.strip().split(',') for line in open('csv.csv')]
for line in fileinput.input(sys.argv[1:], inplace=True, backup='.bak'):
for old, new in old_new:
line = line.replace(old, new)
print(line, end='')
If you save the code in replace.py, you will execute it like this
$ python3 replace.py *.xml subdir/*.xml another_one/a_single.xml
stackoverflow.
I've been trying to get the following code to create a .txt file, write some string on it and then print some message if said string was in the file. This is merely a study for a more complex project, but even given it's simplicity, it's still not working.
Code:
import io
file = open("C:\\Users\\...\\txt.txt", "w+") #"..." is the rest of the file destination
file.write('wololo')
if "wololo" in file.read():
print ("ok")
This function always skips the if as if there was no "wololo" inside the file, even though I've checked it all times and it was properly in there.
I'm not exactly sure what could be the problem, and I've spend a great deal of time searching everywhere for a solution, all to no avail. What could be wrong in this simple code?
Oh, and if I was to search for a string in a much bigger .txt file, would it still be wise to use file.read()?
Thanks!
When you write to your file, the cursor is moved to the end of your file. If you want to read the data aferwards, you'll have to move the cursor to the beginning of the file, such as:
file = open("txt.txt", "w+")
file.write('wololo')
file.seek(0)
if "wololo" in file.read():
print ("ok")
file.close() # Remember to close the file
If the file is big, you should consider to iterate over the file line by line instead. This would avoid that the entire file is stored in memory. Also consider using a context manager (the with keyword), so that you don't have to explicitly close the file yourself.
with open('bigdata.txt', 'rb') as ifile: # Use rb mode in Windows for reading
for line in ifile:
if 'wololo' in line:
print('OK')
else:
print('String not in file')
I am trying to download a vcard to a specific location on my desktop, with a specific file name (which I define).
I have the code the can download the file to my desktop.
url = "http://www.kirkland.com/vcard.cfm?itemid=10485&editstatus=0"
fp = webdriver.FirefoxProfile()
fp.set_preference("browser.download.folderList",2)
fp.set_preference("browser.download.manager.showWhenStarting",False)
fp.set_preference("browser.download.dir", os.getcwd())
fp.set_preference("browser.helperApps.neverAsk.saveToDisk", "text/x-vcard")
browser = webdriver.Firefox(firefox_profile=fp)
browser.get(url)
Note, the URL above is a link to a vcard.
This is saving to the same directory where the code itself exists, and using a file name that was generated by the site I am downloading from.
I want to specify the directory where the file goes, and the name of the file.
Specifically, I would like to call the file something.txt
Also Note, I realize there are much easier ways to do this (using urllib, or urllib2). I need to do it this specific way (if possible) b/c some links are javascript, which require me to use Selenium. I used the above URL as an example to simplify the situation. I can provide other examples/code to show more complex situations if necessary.
Finally, thank you very much for the help I am sure I will get for this post, and for all the help you have provided me for the last year. I dont know how I would have learned all I have learned in this last year had it not been for this community.
I have code that works. Its more of a hack than a solution, but here it is:
# SET FIREFOX PROFILE
fp = webdriver.FirefoxProfile()
fp.set_preference("browser.download.folderList",2)
fp.set_preference("browser.download.manager.showWhenStarting",False)
fp.set_preference("browser.download.dir", os.getcwd())
fp.set_preference("browser.helperApps.neverAsk.saveToDisk", "text/x-vcard")
#OPEN URL
browser = webdriver.Firefox(firefox_profile=fp)
browser.get(url)
#FIND MOST RECENT FILE IN (YOUR) DIR AND RENAME IT
os.chdir("DIR-STRING")
files = filter(os.path.isfile, os.listdir("DIR-STRING"))
files = [os.path.join("DIR-STRING", f) for f in files]
files.sort(key=lambda x: os.path.getmtime(x))
newest_file = files[-1]
os.rename(newest_file, "NEW-FILE-NAME"+"EXTENSION")
#GET THE STRING, AND DELETE THE FILE
f = open("DIR-STRING"+"NEW-FILE-NAME"+"EXTENSION", "r")
string = f.read()
#DO WHATEVER YOU WANT WITH THE STRING/TEXT FROM THE DOWNLOAD
f.close()
os.remove("DIR-STRING"+"NEW-FILE-NAME"+"EXTENSION")
DIR-STRING is the path to the directory where the file is saved
NEW-FILE-NAME is the name of the file you want
EXTENSION is the .txt, etc.