Puppeteer's Click API does not trigger on image map element.
I am using a puppeteer for scraping different e-commerce sites. Some e-commerce sites show a popup on page ready. I am trying to close that popup using click api by targeting element but somehow getting an error as "Node is either not visible or not an Html Element".
I have applied click on selectors:
coords='715,5,798,74'
#monetate_lightbox_mask'
body>div>div:nth-child(1)
body>div:nth-child(1):div:nth-child(1)
URLs for scraping:
https://www.hayneedle.com/product/humantouchijoymassageanywherecordlessportablemassager.cfm
https://www.hayneedle.com/product/napoleonfiberglowventedgaslogset.cfm
https://www.hayneedle.com/product/napoleonsquarepropanefirepittable1.cfm
Please suggest.
Regards,
Manjusha
I would personally use the following to wait for and click the close button:
const close_button = await page.waitForSelector( '[id$="ltBoxMap"] > [href="#close"]' );
await close_button.click();
But unfortunately, it appears that the website has implemented bot detection and is displaying the following page:
The source of the resulting web page looks like this:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml" dir="ltr" lang="en-US"><head profile="http://gmpg.org/xfn/11">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<meta name="viewport" content="width=1000">
<meta name="ROBOTS" content="NOINDEX, NOFOLLOW">
<meta http-equiv="cache-control" content="max-age=0">
<meta http-equiv="cache-control" content="no-cache">
<meta http-equiv="expires" content="0">
<meta http-equiv="expires" content="Tue, 01 Jan 1980 1:00:00 GMT">
<meta http-equiv="pragma" content="no-cache">
<title></title>
</head>
<body>
<h1>Access To Website Blocked</h1>
</body></html>
The bot detection service cannot be fooled simply by changing the user agent, so you will need to experiment with some other methods to bypass the service if you would like to scrape the website.
Related
Trying to learn IIS farming on Server 2016 - IIS 10,i'm able to configure farm setup but my ARR only get hits from second server all the time.
Here re my configuration details;
Main Server;
Windows Server 2016 Standart - 192.168.2.15 - IIS 10 - website name is servistest, it only contains one page as index.asp;
<!DOCTYPE html>
<head>
<meta name="description" content="Webpage description goes here" />
<title>Web Server 001</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="author" content="">
</head>
<body>
<%
Response.Write "<font color='red' size='35px'><b><center>"+FormatDateTime(date,format)+" "+FormatDateTime(time,format)+"<br>WEBSERVER 001</font></b></center>"
%>
</body>
</html>
Second Server ;
Windows Server 2016 Standart - 192.168.2.16 - IIS 10 - website name is servistest, it contains the same asp page as index.asp
<!DOCTYPE html>
<head>
<meta name="description" content="Webpage description goes here" />
<title>Web Server 002</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="author" content="">
</head>
<body>
<%
Response.Write "<font color='BLUE' size='35px'><b><center>"+FormatDateTime(date,format)+" "+FormatDateTime(time,format)+"<br>WEBSERVER 002</font></b></center>"
%>
</body>
</html>
Third Server ;
Windows Server 2016 Standart - 192.168.2.17 - IIS 10 - website name is servistest, it contains the same asp page as index.asp
<!DOCTYPE html>
<head>
<meta name="description" content="Webpage description goes here" />
<title>Web Server 003</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="author" content="">
</head>
<body>
<%
Response.Write "<font color='GREEN' size='35px'><b><center>"+FormatDateTime(date,format)+" "+FormatDateTime(time,format)+"<br>WEBSERVER 003</font></b></center>"
%>
</body>
</html>
Here are main server settings;
After these settings when i call 192.168.2.15 it hits to 192.168.2.16/index.asp and show this page
Refreshed page with shift+F5 multiple times, cleared browser and server's cache, no matter what i do it only shows page on Web Server 002/192.168.2.16 and never hits to main server/192.168.2.15 or third server/192.168.2.17.
On the almost all how to documents on the web, they are using domain instead of LAN IP addresses, is that what i am doing wrong? I'm working on local network that's why should i edit the hosts files of the servers and clients to work with domains? Does ARR requires at least 3 servers(main server for farm configuration +2 servers for balance) to work properly?
I have a custom HTML file set up for B2C's sign in / sign up user flow that looks like this:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" href="css/sign_up.css">
<title>My Sign up</title>
</head>
<body>
<div id="api">
</div>
</body>
</html>
I've hosted this in my web app service and placed the url into the Custom Page URI field in the flow. Screenshot here.
However, when I hit "Run User Flow" the default Microsoft selfAsserted page is still loaded. Is there anything that would cause this to happen?
To clarify: I have hit save after entering the URI and the Custom Page column says "Yes" for Local account sign up page.
You should check again, make sure the custom page status is Yes. But in your screenshot, the status is No for the custom page.
It turned out to be a CORS issue. Adding https://<resourcegroup>.b2clogin.com to my app service's CORS whitelist resolved the problem.
I am working on an asp.net mvc web application, and i am using a web templete that have the following metadata:-
<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="format-detection" content="telephone=no" />
<meta name="description" content="Your description">
<meta name="keywords" content="Your keywords">
<meta name="author" content="My web Site">
<title>Edama - #ViewBag.Title </title>
now i have the following questions:-
will these meta tags be treated the same inside google & bing search engines?
what is the purpose for providing content="telephone=no" how it will be understood by google and bing search engines ?
final question. on some website when i search for them , i got links such as About, contact, etc.. so how i can provide these info to search engines ?
Google has a list of meta tags they understand :
https://support.google.com/webmasters/answer/79812?hl=en
I have created a web site.
the Google search result is showing some extra code which is part of html and angular js.
i have added the folowing metatags but no use
<meta name="description" content="">
<meta name="robots" content="nosnippnet">
<meta name="googlebot" content="NOODP, nofollow, nosnippnet">
<meta name="keywords" content="">
I have waited for 3 weeks so that google will reindex. but no use. can any one tell me where i am going wrong
You've got spelling mistake..
change nosnippnet to nosnippet
I have a website (just for my own references, nothing interesting for the public.)
When I load my page (Test Page) inside IE9 and view the source of the page - I can see the HTML as expected.
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta charset="utf-8">
<title>Test Page</title>
</head>
<body>
<div id="body">
Simple test page, with an image. <br />
<img src="http://www.w3.org/2008/site/images/logo-w3c-mobile-lg" alt="WC3 logo" />
</div>
</body>
</html>
But when I look at the developers toolbar (by pressing f12) the HTML appears in a <framset> tag.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<title>Marrowbrook.com </title>
</head>
<frameset rows="100%,*" border="0">
<frame src="http://217.118.128.188/wotney//TestFiles/testpage.htm" frameborder="0" />
<frame frameborder="0" noresize />
</frameset>
<!-- pageok -->
<!-- 02 -->
<!-- ->
</html>
Using Chrome, if I right click and View Source, I see the above <frameset> code, but I can also right click and select View Frame Source where I can see the HTML as expected.
Can anyone tell me why I'm seeing this ?
Thanks.
This could happen because your host name was bought with one provider, but you are hosting it on another - and you got a frame based redirect setup.
What platform is your site hosted on? It looks like the server is doing something, because the src of the frame in the frameset points to your page. It could be some kind of 'preview mode' or something of the server/cms. So it looks like the server is using a default page with a frameset on it, that pulls your actual page into it after you deploy it
It also happens when the domain you are using to get to the site is set as "Masked" Forwarding.
Check with the domain manager on your hosting and remove masked forwarding.