I have a website I'm scraping and need to scroll horizontal (to the far right) and vertical (to the very bottom) of a table that is on a webpage. I am using Python3, Selenium and Chrome Driver.
I can scrape the table data but it only scrapes as far down as what's visible on my webpage. When I inspect the webpage I cannot find any scrolling information.
Researching online I've seen various ways to scroll using Selenium even finding by xpath and then scrolling that way. I have not had luck with any option thus far but could be doing it wrong.
This is an example of something I've tried but it does not work:
html=browser.find_element_by_xpath('/html/body/form/div[3]/div[2]/div/div[2]/div/div[2]')
html.send_keys(Keys.END)
This scrapes the rows without issue from the table (but only the ones visible on the webpage when the page is loaded. If I scroll all the way down, and rerun this code, it scrapes the contents of the entire table since it fully loads)
for tr in browser.find_elements_by_xpath('/html/body/form/div[3]/div[2]/div/div[2]/div/div[2]'):
tds = tr.find_elements_by_tag_name('td')
action_log = ([tr.text for tr in tds])
I just want to be able to scroll vertically all the way to the bottom of the table and horizontally for as far as I can (just once after the page loads). Again, scrolling inside the table on the webpage - not in Chrome itself. As a note my table data size will change.
I kept combing the site to figure out how to scroll it and finally just ended up using this which worked wonderfully (since the site already has all the data pulled in). As a note, this is scrolling a table element that is at the bottom of the webpage not Chrome itself. Though it works best when the window is maximized.
bg = browser.find_element_by_xpath('/html/body/form/div[3]/div[2]/div/div[2]/div/div[2]/div[4]/div[4]')
for _ in range(3):
bg.send_keys(Keys.END)
time.sleep(.5)
Related
I am new in React world. I am using react-virtualized Grid to show a lot of elements.
At once I am showing 5x8 elements on page + 5x8 overscanRowCount.
All works fine in Chrome and Firefox. In Edge I see little more whitespace while fast scrolling, but still it is ok.
But IE11 freezes on each scroll...its working OK as long as rendered elements are simple, like div with text inside it.
I am rendering empty divs while "isScrolling" and this have positive effect on IE performance, but when scroll ends and real elements are rendering Grid freezes again.
I am using simple Grid, something like Basic Example + windowScroller, and now my Grid is rendering components which returning JSX-object (no heavy logic inside).
CSS for styling Grid elements is external (Not react-side styles).
Elements have no flexbox inside.
I spent several days, but still have problems with IE11 performance.
Thanks in advance.
I'm trying to create a chrome extension that will allow me to display the current URL in a modal or popup window. The URL will be enlarged so that it is easier for students to see in the classroom. My goal is to get away from my current method, which is to copy/paste the URL into notepad and enlarge the font so that students in the back of the room can see it. It would be great if I could simply click on an extension and have the URL displayed in a nice large font. Unfortunately, this is the first chrome extension I've ever written, so I'm running into all sorts of n00b issues.
Anyway, I've gotten close by following this post:
How can I get the URL of the current tab from a Google Chrome extension?
Here's my code:
chrome.tabs.getSelected(null,function(tab) {
var tablink = tab.url;
document.write("<h1 style='font-size: 100px;'>" + tablink + "</h1>");
});
The problem is this code opens the URL in a popup with a horizontal scroll bar.
I either need to:
figure out a way to turn on word-wrap (to eliminate the scroll bar) ... or
adjust the width of the popup window so that it takes up the entire width of the screen ... or
find a different window solution (modal, new window that's alwaysOnTop, etc)
I tried chrome.windows.create but I couldn't figure out how to adapt the code above to use chrome.windows.create, and I also couldn't figure out how to make the resulting new window have active focus (focused: true didn't seem to work for me).
I also read that chrome.tabs.getSelected has been deprecated and I should be using chrome.tabs.query. However, I've had trouble getting the query method to work.
Again, this is my first chrome extension, so your patience, understanding, and help is greatly appreciated. Thanks!
I was just wondering if anyone could help me. I'm creating a full width layout website, and I would like it to adjust with the browser window as it is expanded/contracted, aka everything will shrink a little and move over when the user pulls their window in/ will go back to normal when the user pulls it back out.
Say I have a grid of three 300px images, all lined up in a row, and I want them to just shrink a little, and stay in the row format when pulled in. How would I achieve this?
I'm not even sure where to start...
Help appreciated!
In HTML when you add your add width="20%", height="20%" for example this will expand or shrink your text according to the size of the screen.
I've created a nav menu like in the screenshot below. It spans the entire width of the container and the left/right padding of each menu item is constant. This was easy to do by hardcoding the left/right padding in the CSS, but I want the paddings to be able to change as the site admin edits the menus.
Is there a way to do this with pure CSS (CSS3 is okay)?
This was easy enough to do with jQuery (I totaled up the width of the menu items and calculated the necessary padding). But I ran into some issues on some browsers due to our use of Google Web Fonts. On Chrome and Firefox 4 on Windows (not on Mac), the web font was not loaded at the time that my script ran, resulting in incorrect width measurements. I tried running the script in the jQuery's DOM ready event and in the Google Font API's active event. The active event worked in Chrome but in Firefox 4 it was often fired before the font had been applied.
Thanks in advance.
Here's a jsfiddle of a potential different solution.
Using that layout, and assuming the number of menu items is going to change, you call a recalculation method once a list item is added/removed. In this example provided, I've used YUI3 to do the DOM manipulation, but you could do that a number of ways. Note - I didn't test the javascript function, its "probably working pseudo code".
(You may need to make subtract a further 2% or so from the list item widths, if you're trying to deal with IE6/7)
Use jQuery's .load event as suggested by user thirtydot.
I'm currently working on a flash project, which will use multiple scroll areas.
I'm trying to find out how to create a scroll area like I see on stackoverflow, and make it function similarly.
Whenever someone posts their lines of code to stackoverflow, the scroll area scrollbar looks exactly like the scrollbar in the web browser, and is able to be scrolled with the mousewheel smoothly, without affecting the webpage scrolling.
Example: disable mouse wheel scrolling while cursor over flex app?
Could anyone please explain to me how I can do this in my flash project?
Thanks!!
You wouldn't need flash to do this... This is accomplished using CSS. You would just need to put the portion you wanted to function like that in a div (or any HTML container for that matter) and then define such properties as min-height, max-height, min-width, max-width, overflow, etc. I think there's one that determines the presence of the scroll bars but I don't remember what it is for sure. Very easy to do this and flash would be a bit of overkill.