I know how to detect and filter bot traffic, but I'm wondering if anyone knows why I am seeing tons of bot traffic from insurance and banking domains? E.g., bankofamerica.com, allstate.com and others are hitting my site upwards of 100 times per day. Note that I am talking about domain associated with the user's IP, not the traffic referral domain.
The site is a large ecommerce retailer and the banking and insurance bot traffic started after it moved to the Demandware ecommerce platform.
Which kind of traffic are they sending?
It would be great to have a HTTP request as example, try sniff the traffic to your website with a tcpsniffer such as tcpdump or ngrep.
It could be that those are simply users which are bind a corporate proxy and thus appear as one IP. How do you know it is bot traffic?
You can prevent those request hitting your server by implementing DOS ( denial of service) on the application level and DDOS on your network level if you have the cisco firewall installed in your infrastructure.
If you still have this problem, contact Demandware support by raising a customer support ticket in their SalesForce portal.
There is not much that you could by configuration of your Demandware site from the Business Manager.
Chances are that the traffic was always there, just that it was filtered by the firewall of your previous e-commerce provider.
Turns out this was caused by bots spoofing legitimate domains
Related
We have a cisco load balancer on-premise which routes traffic to our DMZ Servers on-premise
We want to use Azure Load Balancer or Azure Solutions (AG) which can balance traffic to our DMZ Servers on-premise, basically replace the CISCO with Azure
Is it possible? we have SFT/HTTPS sites currently hosted on our DMZ Environment.
TIA
What you're proposing isn't the use-case for Application Gateways. Application Gateways are Layer 7 load balancers / reverse proxies. What you want to do is almost treat them as a one-site forward proxy. It's not a good architecture and even if it were possible would ultimately be more costly in the long-run since you would pay for data egress as your App Gateway accepts requests and then forwards on to your web servers via an outbound connection over the Internet. They then receive the response headers/body from your web servers and again send that result on to the original caller.
In that scenario, you are forced to have to use end-to-end SSL for your applications, removing any possibility of using the App Gateway for SSL offload in the future. If your traffic isn't encrypted or doesn't need to be, the predictability of the source and destination of your traffic increases the security risk to your website's users and your company.
You also have the possible security implications of this type of architecture. Your web servers still need to be accessible at the very least by your Application Gateway, which means they are either freely available on the Internet anyway (in which case why bother with an App Gateways at all) or they're firewalled at a single layer and permit only traffic from the source IP address of your Application Gateway.
The bad news with the firewall approach is that you cannot assign a static public IP address to an Application Gateway, it is forced as Dynamic. Realistically the public IP won't change until the App Gateways are rebooted but you should know that when, not if, they do, your firewall rules will be wrong and your App Gateways won't be able to get to your DMZ servers any more, which means an outage. The only true solution for that is a firewall that can do URI based firewall rules...the impact there is cost (time and CPU) to perform a DNS lookup, see if the traffic is from the App Gateway by its DNS address - something like bd8f86bb-5d5a-4498-bc0c-e1a48b3873bf.cloudapp.net and then either permit or deny the request.
As discussed above, a further security consideration is that your traffic will be fairly consistently originating from one location (the App Gateways) and arriving at your DMZ. If there's a well defined source of traffic, that fact could be used in an attack against your servers/DMZ. While I'm sure attacking this is non-trivial, you damage your security posture by making source and destination traffic predictable across the Internet.
I've configured a good number of Application Gateways now for Enterprise applications and out of morbid curiosity I had a go at configuring a very basic one using HTTP to do what you're attempting - fortunately (yes, fortunately) I received an HTTP 502 so I'm going say that this isn't possible. I'll add that I'm glad it isn't possible because it's a Bad Idea (TM).
My suggestion is that you either migrate your DMZ servers to Azure (for the best performance/network latency) or implement a VPN or (preferably) ExpressRoute. You'll then be able to deploy an Application Gateway using the correct architecture where you terminate your users' connections at the App Gateway and that re-transmits the request within your RFC1918 network to your DMZ servers which respond within the network back to the App Gateway and ultimately back to the requestor.
Sorry it's not what you wanted to hear. If you're determined to do this, perhaps nginx could be made to?
I read quite a lot documents and other questions on stackoverflow about this, and I have to admit that it just makes me much more confused.
I have a site that hosted both in EU west and EU north azure. Say their urls are:
exampleeuw.azurewebsites.net
exampleeun.azurewebsites.net
Then I bought a ssl certificate for www.example.com.
On both sites, I added www.example.com to custom domain.
On both sites, I uploaded the certificate.
Then created the traffic manager site and its dns name is "example.trafficmanager.net", I have both web app added as end point.
But should I access https://www.example.com or https://example.trafficmanager.net?
If I access https://www.example.com, how can that traffic go to the traffic manager first?
Also when I created Traffic Manager profile, I have to select a Resource group location (north Europe for example), if North Europe azure is down, will it impact the access?
Really hope that I can find some step by step examples on how Microsoft wants us to use this, as it has been a quite frustrating learning process.
Requests should go to the traffic manager. It will be the one redirecting to both of your App Services no matter where they are.
Prior configuring your Traffic Manager and DNS, make sure both of your web application are working and configured with custom domains and SSL certificates.
Then, in your DNS, point your www.example.com website to the traffic manager as shown here:
https://azure.microsoft.com/en-us/documentation/articles/traffic-manager-point-internet-domain/
That's all you have to do.
I am trying to find hosting provider for a website that is connected to cloudflare. On Whois Lookup, I get Name Server(s) NOAH.NS.CLOUDFLARE.COM
UMA.NS.CLOUDFLARE.COM
When I use this website http://network-tools.com
I get Attempt to get a DNS server for 104.XX.1xx.3x failed:
I know that cloudflare is not a hosting provider. How can I dig deep and find the actual hosting provider?
If you have some type of abuse related issue with the website in question you'll need to file a complete abuse report at cloudflare.com/abuse
In most cases there isn't an obvious way to identify the underlying hosting provider for a website behind CloudFlare. With a valid and complete abuse report they can put you in touch with the hosting provider's abuse team though.
There is a service called CrimeFlare that helps to resolve the actual IP address behind CloudFlare.
If the website owner has properly set up their website's configuration with CloudFlare. There is no way to acquire the IP through any external means (Whois, DNS, etc)
If you must send an abuse report or DMCA complaint, you can contact CloudFlare through their website and they will be able to forward it on for you.
1. Check hosting history
Services like https://toolbar.netcraft.com/site_report can help you discover the history of hostings for the website.
2. Try Cloudflare IP resolver
https://iphostinfo.com/cloudflare/ (Works as of 2019.12.10, not sure if you can trust this source in a long-term).
Enter your domain name. Click "Search DNS" and you will be
brought to a page that checks for common DNS entries. This tool was
created to help webmasters configure their domains DNS with CloudFlare
properly and is not meant to be used for abuse purposes. If you see
your servers real ip in any DNS entry, anyone can easily find your
real servers ip then.
3. Look for subdomains manualy
You can check and WHOIS the IPs for subdomains. E.g. if www.XYZ.com and XYZ.com are both behind CloudFlare, check for popular subdomains, like:
status.XYZ.com
ns.XYZ.com
beta.XYZ.com
test.XYZ.com
forum.XYZ.com
blog.XYZ.com
admin.XYZ.com
etc..
There is some chance that they will be available directly without CloudFlare.
4. Ask
You can find a way to ask them directly. Or get such information via CloudFlare if you have a legit need to know it. (E.g. by contacting cloudflare.com/abuse as suggested by #xxdesmus).
Law enforcement officials can contact us directly at abuse+law#cloudflare.com. You must include your badge & case number when contacting us to receive a response.
We have a newly-configured Sharepoint 2013 server installation, which serves our company Intranet.
The new Intranet is now in service and operational.
Next, we need to create another top-level site which will serve as an Extranet portal for clients.
Sharepoint Central Administration does not provide a method by which a separate IP address can be
selected for a new site; instead, it provides port selection or entry only.
We need to route external traffic to the extranet site, which requires either a mapped internal IP
address or a different port. The main limitation we face is that most of our public-sector clients
cannot use a non-standard port due to their firewall restrictions. So, this means a unique IP.
In the exhaustive research conducted concerning this issue, it appears that the "best practice"
receommended by Microsoft themselves is nothing but a workaround, where part of the process is
performed in SP Central Admin and the other half in IIS.
We have found many articles and blogs mentioning alternative ports, but none which address this situation directly.
Now we're trying to contend with Alternate Access Mappings, which are confusing our admins.
We really need the voice of experience from someone who has actually done this before.
Question: what is the correct way of achieving our goal?
Your users will access the site based on a host name I guess, not on an IP address?
So, you will have an intranet under http://intranet and you now want to create an extranet under http://extranet.
The fact that extranet is on another IP address and routed to your SharePoint farm is not really an issue.
What you need to do, is to create another web application with as host name http://extranet
Your firewall / network hardware must then forward http://extranet to your SharePoint servers. IIS will see "extranet" and serve the extranet.
If your users will access it via the IP-address, it's similar. Instead of http://extranet, you enter http://ext.ip.addr as the host name.
I'm currently moving sites to a new hosting platform. For some of these sites on our old hosting platform, we have A, CNAME, and/or MX records directing mail traffic to some other host (Gmail or other).
Here's my question:
If I have those A, CNAME, and MX records duplicated on the new hosting platform, update the registrar with the new DNS servers, and keep the site live on both hosting platforms during propagation, will there be any instances where mail requests won't get to the proper location?
I'm currently thinking there won't be any issues, due to the redundancy between both DNS servers, but I have to be 100% sure.
All help would be greatly appreciated, Thanks!
I finally had the opportunity to just attempt the previous scenario and find out if I was correct. I'll share it here as an answer just in case someone else ends up in the same situation.
Because of the redundancy between both sets of primary DNS's, there was no issue with the clients being able to access their email and send/receive messages.