I want to download each state's legislative upper chamber zip file from this url.
https://www.census.gov/cgi-bin/geo/shapefiles/index.php?year=2016&layergroup=State+Legislative+Districts
First, I want automatically download these hundreds of files to save time and effort.
Second, I want to save these legislative upper chamber zip files in the following format: us_state[0:]_upper_chamber_tl_2016_01_sldu_zip
Here is the link to the code that I so far have come up written. https://bpaste.net/show/9514161688ea
Thank you in advance for your feedback.
import pyautogui
#Step 1 Move to click on the drop down list of State Legislative - Upper
#Chamber
uc = pyautogui.moveTo(1403, 671, duration=1.5)
pyautogui.click()
#Step 2 Move to click on the state in the drop down list.
al_uc = pyautogui.moveTo(1403, 700, duration=0.5)
pyautogui.click()
#Step 3 Move to the down download button
uc_dl = pyautogui.moveTo(1541, 679, duration=1)
pyautogui.click()
For those who are working with Python, I was suggested to use the "pyautogui" module. One needs to program where to click.
I encountered two problems.
(1) I had the automatic downloader in Chrome on. So, I had to turn it off so that it would not allow my downloads to pop up after downloading and messing with the code.
(2) I reduced the mouse duration speed from 0.5 to 1.5 when going to my target list from the download button. I did so because of my slow internet connection, chrome being slow at downloading, refreshing, and messing with the code.
Link to install pyautogui:
http://pyautogui.readthedocs.io/en/latest/install.html
Related
I was wondering that can we change windows 10 background using python i did a bit research and find a module name ctypes but the problem was that it needs a image to apply but i want that instead of a locally saved file it should use a url of a image and refresh the background at a interval with a new image. Cuz i want to create a script which will apply an anime wallpaper in every 5 minute and the url will be taken from a api.
Please Help!
On Windows with python2.5 or higher, use ctypes to load user32.dll and call SystemParametersInfo() with SPI_SETDESKWALLPAPER action.
For example:
import ctypes
SPI_SETDESKWALLPAPER = 20
ctypes.windll.user32.SystemParametersInfoA(SPI_SETDESKWALLPAPER, 0, "image.jpg" , 0)
Your script will have to do that in two steps
Download the image from the url using a package like requests
Set the Windows Background
I have successfully written code to web scrape an https text page
https://services.swpc.noaa.gov/text/goes-xray-flux-primary.txt
this page is automatically updated every 60sec. I have used beautifulSoup4 to do so. Here are my two questions: 1)how do I call a loop to re-scrape the page every 60 seconds? 2) since there are no html tags associated with the page how can only scrape a specific line of data?
I was thinking that I might have to save the scraped page as a CVS file then use the saved page to extract the data I need. However, I'm hoping that this can all be done without saving the page to my local machine. I was hoping that there is some python package that can do all of this for me without saving the page.
import bs4 as bs
import urllib
sauce = urllib.urlopen("https://services.swpc.noaa.gov/text/goes-xray-flux-primary.txt").read()
soup = bs.BeautifulSoup (sauce,'lxml')
print (soup)
I would like to automatically scrape the first line of data every 60 seconds Here is an example first line of data
2019 03 30 1233 58572 45180 9.94e-09 1.00e-09
The header that goes with this data is
YR MO DA HHMM Day Day Short Long
Ultimately I would like to use PyAutoGUI to trigger a ccd imaging application to start a sequence of images when the 'Short' and or "Long" x-ray flux reaches e-04 or greater.
Every tool has its place.
BeautifulSoup is a wonderful tool, but the .txt suffix on that URL is a big hint that this isn't quite the HTML input which bs4 was designed for.
Recommend you use a simpler approach for this fairly simple input.
from itertools import filterfalse
def is_comment(line):
return (line.startswith(':')
or line.startswith('#'))
lines = list(filterfalse(is_comment, sauce.split('\n')))
Now you can do word split on each line to convert to CSV or pandas dataframe.
Or you can just use lines[0] to access the first line.
For example, you might parse it out in this way:
yr, mo, da, hhmm, jday, sec, short, long = map(float, lines[0].split())
I have 2 functions as below:
def select_audio():
os.chdir("/home/norman/songbook")
top1.lower(root)
name=tkFileDialog.askopenfilename()
doit="play " + name
top1.lift(root)
os.system(doit)
def select_video():
os.chdir("/home/norman/Videos")
top2.lower(root)
name=tkFileDialog.askopenfilename()
doit="mpv --fs " + name
top2.lift(root)
os.system(doit)
They are selected from buttons to allow choosing and playing audio files or video files.
They work to some extent.
Videos are in a different directory and at the same level as the audio files.
It doesn't matter which I choose first I see the correct directory so I can play say a video, if after it's finished I choose audio it still shows the video directory.
Similarly if I first choose audio it still shows the audio directory if I select videos.
I have no idea why it does this. I am not an experienced programmer as you can probably tell from the code.
Some suggestions:
Use a raw string to make sure that Python doesn't try to interpret anything following a \ as an escape sequence:
Change os.chdir("/home/norman/whatever") to os.chdir(r"/home/norman/whatever")
It won't solve this problem, but it will avoid you future problems.
For tkFileDialog use the initialdir option:
Change name=tkFileDialog.askopenfilename() to
name=tkFileDialog.askopenfilename(initialdir=r"home/norman/whatever", parent=root)
I have a program which downloads pages from a site, finds links for pictures in them and downloads those pictures. If I try to run this program on a computer with fast and stable Internet connection - everything works perfectly for days and weeks. But if I try this program on a computer with slow or not stable Internet connection - I have one problem - the "try-except" module doesn't seem to work correctly.
--- this function downloads content - any content (page or picture)
def downl(self,addr,cook,head2,errmess):
global result
try:
result=requests.get(addr, cookies=cook, headers=head2)
except:
print(errmess) # error message
time.sleep(5)
return result
I sent to this function link to the page, then other function looks for picture_link in that page, and then I send to the same function (downl) picture_link. After this I save result of function (downl) as a .jpg file. As I told - on a computer with normal internet connection everything works fine. As a result I have 5, 10 or 5000 pictures on my HDD.
But let me show little example of what happens with bad internet connection. Suppose we have 2 pages and 1 picture in every page.
step 1) downloading 1st page (def downl)
step 2) taking picture_link from it
step 3) downloading picture (def downl)
step 4) saving 1st picture to hdd 1.jpg
step 5) downloading 2nd page (def downl)
step 6) taking picture_link from it
step 7) downloading picture (def downl) and receivind error message (errmess)
step 8) saving 2nd picture to hdd 2.jpg
just for example: 1st picture may be normal jpg with proper content. The second picture will be file with jpg extension but will have 2nd page as it's content (it will be usual html file, saved with wrong extension "jpg")
another words: there was problem with internet during downloadind of the second picture, the program printed an error about it (errmess), but INSTEAD of COUNTLESS retrying (as supposed in my function) it somehow PASSED through the try-except block and returned previous result (2nd page), which was saved as 2nd picture.
Please help! How to make this try-except (or requests) work FOREVER, UNTILL it downloads what it is supposed to download (no matter what mistakes happen with internet connection), and not pass through with previous result.
Thanks very much for youк time and attention.
Then you need a while True loop like this:
def downl(self,addr,cook,head2,errmess):
global result
while(True):
try:
result=requests.get(addr, cookies=cook, headers=head2)
return result
except:
print(errmess) # error message
time.sleep(5)
I would like to be able to play a sound file in a ipython notebook.
My aim is to be able to listen to the results of different treatments applied to a sound directly from within the notebook.
Is this possible? If yes, what is the best solution to do so?
The previous answer is pretty old. You can use IPython.display.Audio now. Like this:
import IPython
IPython.display.Audio("my_audio_file.mp3")
Note that you can also process any type of audio content, and pass it to this function as a numpy array.
If you want to display multiple audio files, use the following:
IPython.display.display(IPython.display.Audio("my_audio_file.mp3"))
IPython.display.display(IPython.display.Audio("my_audio_file.mp3"))
A small example that might be relevant : http://nbviewer.ipython.org/5507501/the%20sound%20of%20hydrogen.ipynb
it should be possible to avoid gooing through external files by base64 encoding as for PNG/jpg...
The code:
import IPython
IPython.display.Audio("my_audio_file.mp3")
may give an error of "Invalid Source" in IE11, try in other browsers it should work fine.
The other available answers added an HTML element which I disliked, so I created the ringbell, which gets you both play a custom sound as such:
from ringbell import RingBell
RingBell(
sample = "path/to/sample.wav",
minimum_execution_time = 0,
verbose = True
)
and it also gets you a one-lines to play a bell when a cell execution takes more than 1 minute (or a custom amount of time for that matter) or is fails with an exception:
import ringbell.auto
You can install this package from PyPI:
pip install ringbell
If the sound you are looking for could be also a "Text-to-Speech", I would like to mention that every time a start some long process in the background, I queue the execution of a cell like this too:
from IPython.display import clear_output, display, HTML, Javascript
display(Javascript("""
var msg = new SpeechSynthesisUtterance();
msg.text = "Process completed!";
window.speechSynthesis.speak(msg);
"""))
You can change the text you want to hear with msg.text.