(Warning: this link is SFW, but the site hosts NSFW text/audio content on other pages, so maybe don't click through on a work computer.)
I am trying to access a series of pages like this. Each page streams an M4A audio file. Once the page loads, the jQuery document-ready event goes and fetches the duration of the audio and puts it on the page.
That's the value I want, the contents of div.jp-duration after the JavaScript has done its insertion.
To do this, I am writing
headless = Headless.new
headless.start
browser = Watir::Browser.start "https://soundgasm.net/u/tarkustrooper/F-Wasted-Time-The-Eagles"
duration = browser.execute_script %q{return $("div.jp-duration").innerHTML}
but I'm not returned anything.
If I try $("#jquery_jplayer_1").data('jPlayer').status.duration, expecting the number of seconds, I get back 0, even though this works if I try in a Chrome console.
I'm using Watir wrong somehow, but I don't knowit well enough to know how. I'd love some help if possible.
I'm using Ruby 2.1.5, Watir 5.0.0, Watir-Webdriver 0.8.0 and Headless 2.2.0 on CentOS 6.
This should work without the need for execute_script:
#browser.div(class: 'jp-duration').text
Related
I am learning Python and attempting to build a program that will scrape specific data from a website, store it and then manipulate it.
Currently I run my application, it opens a new chrome browser window and loads the page correctly. The problem is it should begin to start scrolling down and loading the remaining elements on the page.
I know the code works because if I manually click somewhere on the page that doesn't normally illicit a response (white space/empty areas) the browser somehow comes into "focus" and begins to iterate through the loop that scrolls down the page (by sending keys) prints the data I am after. I also noticed if I click another similar "dead space" area that contains the header, it doesn't have the same effect. I am unsure if this is something specific to Chrome, iFrames or something of that nature but I am completely stumped and would greatly appreciate any help.
Any thoughts on why I need to manually click on the new chrome window for it to work would be great.
Update: Still having the same issue, even tried with Safari and the same problem seems to exist.
Fixed this with:
element = driver.find_element_by_css_selector("div[id^='app-container']")
action = ActionChains(driver)
action.click(on_element = element)
action.perform()
I have a scraping script written in Ruby which uses Selenium, Watir and ChromeDriver, all is working just fine with a Chrome browser window, but trying to run in headless mode just hits;
Selenium::WebDriver::Error::UnknownError: unknown error: Element <input id="desired_element" type="checkbox" name="desired_element" checked="checked"> is not clickable at point (660, 594). Other element would receive the click: <html lang="en">...</html>
I'm using Watir 6.8 and latest Chromedriver 2.33.
Any ideas whats n=behind the different behavior of the page when in headless vs non headless, and how I can deal with this?
The error message is telling you what the problem is when it says "Other element would receive the click:"
This means some other element on the screen is covering the checkbox you are trying to interact with. Likely this is caused by whatever the default browser size in headless mode being different than the default size of your browser when it is run non-headless, resulting in a different arrangement of elements
We can verify if this is the case by asking the size of the window in both headless and normal modes and seeing if the resulting values are the same.
size = browser.window.size
puts "The browser window is #{size.width} wide by #{size.height} high"
There are a few potential ways to solve this:
Specify or alter the browser 'window' size. for example
browser.window.resize_to 1024, 768
I prefer this, and normally have a command such as that to set the
browser size right after it is initialized. Set either to the
minimum supported size for your site, or the minimum recommended
size
Use another means to 'click' on the checkbox, such as sending a space at it
#browser.checkbox(name: "desired_element").send_keys " "
I do not prefer this as it doesn't really solve the source of the problem and you may experience other similar issues interacting with other elements on the site as your script progresses.
The reason for this kind of error is that web page doesn't load well enough to locate the element. As you have mentioned that the tests previously passed when you had headless true such issues might be because of .click() , please try replacing .click with .send_keys
.send_keys(selenium.webdriver.common.keys.Keys.SPACE)
When you use .send_keys() you might hit one more issue if it is failing to find elements, for solving this you will have to find the elements
elements = driver.find_element_by_tag_name("html")
elements.send_keys(Keys.SPACE)
Hope this helps you.
When I set response content type as Excel, the Open/Save dialog is shown twice , just on IE8. It works fine on other browsers (tested on Chrome/Firefox/Opera).
The code for setting response content type is:
response.setContentType("application/vnd.ms-excel");
response.setHeader("Content-disposition","attachment;filename=abc.xls");
I searched for solutions/workarounds. Turning off Smartscreen didn't help.
Also, another suggestion was to wait for 5-10 sec before clicking Save/Open. That too didn't work.
What's the cause of this? Are there any IE specific workarounds?
It's a pain but IE8 is still widely used by the users.
This is just a guess, but it could have something to do with the way Office (used to) embed itself in IE with plugins.
A workaround might be putting it in a zip file before sending it over to the user.
I am trying to dynamically embed a youtube video into a mobile web page by injecting the following code via jQuery.
$("#tagetId").append("http://www.youtube.com/embed/oHg5SJYRHA0' frameborder='0'>");
I am testing this on chrome ios and the video does render correctly however some part of the web page seems to think the video hasn't rendered and every half second or so I get a new instance of the following error.
Unsafe JavaScript attempt to access frame with URL http://mydomain.html from frame with URL http://www.youtube.com/embed/oHg5SJYRHA0. Domains, protocols and ports must match.
This seems to really jam up the browser and causes the the load event call back function (i.d. 'first line of code') to trigger over and over.
$('iframe').load(function(){
//first line of code
$(this).load(function(){
//second line of code
})
});
Is there a better way to do this. Can any one explain what I'm doing wrong?
This fixed it:
<iframe scrolling='no' class='youtube-player' style='height:200px;width:100%' src='https://www.youtube.com/embed/oHg5SJYRHA0?html5=1' frameborder='0'></iframe>
Not sure what you were doing with
$("#tagetId").append("http://www.youtube.com/embed/oHg5SJYRHA0' frameborder='0'>");
but that looks like malformed HTML being appended.
Maybe you just didn't append the whole iframe tag?
How is it possible to change browser header with Watir?
I'd like to change browser headers (in Firefox or Chrome) when using Watir.
I know about watir-user-agent gem, but I'm interested in changing browser version.
Is that possible?
Thanks
Yes this can be done.
Unfortunately Watir does not seem to provide any very easy way to do this
However, here are 2 simple options which work:
A. Use a proxy server.
This is a well understood way to modify headers generally. However I have not personally used it during automation.
Steps :
1. Setup proxy server before your test code is executed
2. Ensure the proxy server will add the required headers to every request
3. Then when your test browser requests any page ----> the proxy server will automatically add the required headers.
B. Use browser extensions
Since Watir cannot seem to to modify headers by itself ... then we just ask Watir to use a normal browser extension which can!
I have done this successfully using Chrome and firefox
Note: These steps work with ONLY the indicated extensions - but a similar approach should also work fine for many other extensions.
Firefox Steps:
1. Start firefox
2. Search for 'Modify Headers Firefox' using a very popular search engine .... the top result is https://addons.mozilla.org/En-us/firefox/addon/modify-headers/
3. Download the .xpi file for this extension ... currently you can do this by right clicking on the button and clicking "save link as"
4. Install the extension as normal, change the headers as you wish, close firefox, then locate and save the "modifyheaders.conf" file ... this file should be somewhere in your user folder
5. Make the following class (which extends Profile)
class FirefoxProfileWithAddedFiles < Selenium::WebDriver::Firefox::Profile
# This method OVERRIDES the one in Profile
# This method creates the firefox profile folder
def layout_on_disk
#Call the superclass layout method
profile_directory = super
#Add custom file
if(!#file_to_add_to_profile.nil?)
FileUtils.cp(#file_to_add_to_profile, profile_directory)
end
profile_directory
end
def add_file_to_profile(filepath)
#file_to_add_to_profile = filepath
end
6. Set your test script up as follows
...
#Setup Firefox Profile
profile = Selenium::WebDriver::Firefox::Profile::FirefoxProfileWithAddedFiles.new
profile.add_extension("SOMEPATH/modifyheaders.xpi")
profile.add_file_to_profile("SOMEPATH/modifyheaders.conf")
profile["modifyheaders.config.active"] = true
#Start up Firefox
#browser = Watir::Browser.new :firefox, :profile => profile
...
Chrome Steps
1. Start chrome
2. Search for 'Modify Headers Firefox' using a very popular search engine .... the top result is https://chrome.google.com/webstore/detail/modify-headers-for-google/innpjfdalfhpcoinfnehdnbkglpmogdi
3. Install the extension as normal, change the headers as you wish, then close chrome
4. Locate the unpacked extension folder and copy it. On windows the folder will be something like...
C:\Users\MYNAME\AppData\Local\Google\Chrome\User Data\Default\Extensions\innpjfdalfhpcoinfnehdnbkglpmogdi\2.0.3_0
5. Locate the extension configuration file and copy it. On windows, the file will be something like...
C:\Users\MYNAME\AppData\Local\Google\Chrome\User Data\Default\Local Storage\chrome-extension_innpjfdalfhpcoinfnehdnbkglpmogdi_0.localstorage
6. Set your test script up as follows:
...
#Setup Chrome Profile Folder
profile_directory = Dir.mktmpdir("webdriver-chrome-profile")
extension_configuration_folder = FileUtils.mkdir_p "#{profile_directory}/Default/Local Storage"
FileUtils.cp("PATH_TO_MY_EXTENSION_CONFIGURATION_FILE", extension_configuration_folder[0])
#Start Webdriver
#browser = Watir::Browser.new :chrome, :switches => ["--user-data-dir=#{profile_directory}", "--load-extension=#{PATH_TO_MY_UNPACKED_EXTENSION_FOLDER"]
...
Watir automates the browser INSIDE the browser window, with very limited interaction up at the OS level (such as responding to alerts, etc), you would need to pre-configure the browser (presuming that was possible) to what you wanted, or use a tool such as Autoit to interact with the browser's OS level controls to do that.. (presuming the browser even has the feature to allow you to alter what it is reporting in terms of browser and version when it makes a request to a website)
If you are using Watir-Webdriver along with Firefox then you may be able to do this via a profile that sets those parameters. In that case you create the profile, then the browser object with that profile specified. it's pretty much a webdriver function, but easy enough to access when creating the browser object.
See this webdriver bug for the parameters to use (down in the comments) when creating the profile. Refer to webdriver docs for more info on how to setup and use profiles for firefox.
Another option that might be useful would be to fork your own version of the code for the user-agent gem and add browser_version as one of the things to be set. It's using profiles for FF, so doing that should be possible, at least for FF. for Chrome it is using the user-agent switch to override the useragent string, so it should be possible there also, although you would have to do a little work to modified the fixed strings the gem uses to replace the portion that has the version with the one you want.
Then if you get it working issue a pull request to add that enhancement to the gem..
or if you are not up to that sort of thing yourself, then beg, plead and offer to bribe the gem author with something appropriate if they would extend the gem for you to make version one of the things that could be set.