I just put up a new website. New domain, new everything. I set up Google Analytics and my first ever visitor is this:
forum.topic60670121.darodar.com (i do not linking it, it is spam)
I know lot of people experience traffic from several Russian spam sites those redirect to some other site. I read a lot of about it. I can't ban them even from htaccess like this:
## SITE REFERRER BANNING
RewriteEngine on
RewriteCond %{HTTP_REFERER} semalt.com [NC,OR]
RewriteCond %{HTTP_REFERER} Darodar.com [NC,OR]
RewriteCond %{HTTP_REFERER} Priceg.com [NC,OR]
RewriteCond %{HTTP_REFERER} 7makemoneyonline.com [NC,OR]
RewriteCond %{HTTP_REFERER} Buttons-for-website.com [NC,OR]
RewriteCond %{HTTP_REFERER} Ilovevitaly.com [NC,OR]
RewriteCond %{HTTP_REFERER} Blackhatworth.com [NC,OR]
RewriteCond %{HTTP_REFERER} Econom.co [NC,OR]
RewriteCond %{HTTP_REFERER} Iskalko.ru [NC,OR]
RewriteCond %{HTTP_REFERER} Lomb.co [NC,OR]
RewriteCond %{HTTP_REFERER} Lombia.co [NC,OR]
RewriteCond %{HTTP_REFERER} hulfingtonpost.com [NC,OR]
RewriteCond %{HTTP_REFERER} cenoval.ru [NC,OR]
RewriteCond %{HTTP_REFERER} o-o-6-o-o.com [NC,OR]
RewriteCond %{HTTP_REFERER} humanorightswatch.org [NC,OR]
RewriteCond %{HTTP_REFERER} bestwebsitesawards.com [NC]
RewriteRule .* - [F]
deny from 78.110.60.230
deny from 217.23.11.15
deny from 217.23.7.180
deny from 217.23.8.124
It just doesn't work. And I given up on this at another website. But the question is:
How it is possible they visit a site that is so new even I barely know about it?
Why you get the traffic at all
This blog post might be an actual explanation for your question: The author of the blog post finds evidence that the spammers harvest Google Analytics UA codes to artificially send page hits. The link in your GA reports should then contain a referral identifier.
The author suspects that UA codes may also be auto-generated.
Other possible causes for it happening so quickly
Have you entered the domain name into a webservice tool to test something? E.g. DNS setup, SEO tools, or similar? They might have been breached and do not know about it. I experience such weird traffic on my domains, too and have begun to ignore them.
How to get rid of the weird traffic in your reports
You can also setup your reports in GA to exclude those from the numbers being reported (unless you are also actively targeting the area where the weird traffic originates). There is a great blog post explaining how to filter bad traffic from GA. The blog post also states darodar.com as their traffic source.
How it is possible they visit a site that is so new even I barely know about it?
They don't visit your site. They send data directly to Google Analytics and leverage the fact that by picking property IDs randomly, there is a significant probability to hit existing Web properties.
For more background information and a discussion about different approaches to eliminate referrer spam:
http://veithen.github.io/2015/01/21/referrer-spam.html
Related
I am attempting to block hotlinked images from a specific site and re-route to an externally loaded/hosted image somewhere else.
I made some edits to my .htaccess file
Buy it also seems to block my OWN site/domain from my own images.. (even though I believe I am allowing it?)
I cant seem to get things to work with JUST blocking the external site...(without blocking my own site from my own images?)
I've tried so many lines... I cant make heads or tails on what is the issue that is blocking images from my own site.
Here is is my latest attempt
RewriteEngine On
#RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://myfakesite.net.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.myfakesite.net.*$ [NC]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?external-site\.com(/.*)*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^https://(www\.)?external-site\.com(/.*)*$ [NC,OR]
RewriteRule .*\.(gif|jpg|jpeg|bmp|png)$ https://path-to-externally-hosted-image.jpg [R,NC]
*I erroneously though this would be much easier.. LOL
How can I block external-site.com and allow everything from mysite.net?
I blocked some sites in the file but they keep comming on my server and asking for images that slow down my server how i can add a rule for them so they will see big red sign STOP HOTLINKING
RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} ^http://(www\.)?somesite\.pl [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?somesite\.pl [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?somesite\.pl [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?sklep.somesite\.eu [NC]
RewriteRule \.(gif|jpe?g|js|css)$ - [F,NC,L]
Mate try the below ,
Following code will only allow the mentioned domain "alloweddomain.com" and block others from hot linking
RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http(s)?://(www\.)?alloweddomain.com [NC]
RewriteRule \.(jpg|jpeg|png|gif)$ - [NC,F,L]
The above code will allow "Blank Referrers"
What is Blank Referrers ?
Some visitors uses a personal firewall or antivirus program, that deletes the page referrer information sent by the web browser. Hotlink protection is based on this information. So if you choose not to allow blank referrers, you will block these users. You will also prevent people from directly accessing an image by typing in the URL in their browser.
Suppose if you don't want to allow "Blank Referrers" then use the following code mate
RewriteEngine on
RewriteCond %{HTTP_REFERER} !^http(s)?://(www\.)?alloweddomain.com [NC]
RewriteRule \.(jpg|jpeg|png|gif)$ - [NC,F,L]
Also if suppose you want to display a image like "STOP HOTLINKING" then use the below method
RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http(s)?://(www\.)?alloweddomain.com [NC]
RewriteRule \.(jpg|jpeg|png|gif)$ mydomain.com/img/stop_hotlink.jpg [NC,R,L]
Above code will allow "Blank referrers" . To not allow, follow as mentioned in previous step again.
Note that :
In case of displaying image for "STOP HOTLINK" make sure the image is not hotlink protected or your server can go into an endless loop.
alloweddomain.com - The domain that you want to allow for hotlink
mydomain.com/img/stop_hotlink.jpg - URL for the "STOP HOTLINK" image
Update : [ Block Specific Domains ]
To stop hotlinking from specific outside domains only, such as blockurl1.com, blockurl2.com and blockurl3.com, but allow any other web site to hotlink images:
RewriteEngine On
RewriteCond %{HTTP_REFERER} ^http://(.+\.)?blockurl1\.com/ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(.+\.)?blockurl2\.com/ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(.+\.)?blockurl3\.com/ [NC]
RewriteCond %{REQUEST_URI} !blocked\.gif$ [NC]
RewriteRule .*\.(jpe?g|gif|bmp|png)$ http://example.com/blocked.gif [L]
You can add as many different domains as needed. Each RewriteCond line should end with the [NC,OR] code. NC means to ignore upper and lower case. OR means "Or Next", as in, match this domain or the next line that follows. The last domain listed omits the OR code since you want to stop matching domains after the last RewriteCond line.
The last line contains the URL "http://example.com/blocked.gif" which contains the image that will be displayed when the condition occurs.
You can display a 403 Forbidden error code instead of an image. Replace the last line of the previous examples with this line:
RewriteRule .*\.(jpe?g|gif|bmp|png)$ - [F]
Hope this helped you mate!
My site is getting spammed by lots of different blogspot urls (such as http://somespammyurl.blogspot.com.br), but I can't figure out how to block them, I tried:
RewriteEngine on
RewriteCond %{HTTP_REFERER} ^http(s)?://(www\.)?.blogspot.co.id.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http(s)?://(www\.)?.blogspot.bg.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http(s)?://(www\.)?.blogspot.ru.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http(s)?://(www\.)?.blogspot.com.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http(s)?://(www\.)?.blogspot.com.br.*$ [NC]
RewriteRule .* - [F]
but in my raw logs I still see spammy urls with http/1.0/ 200, which I'm assuming means they are getting through, because if I specify the exact whole url of one of the spamming urls in the htaccess, the raw log line says http/1.0" 500 for that url. Can anyone shed some light why those lines aren't blocking all *.blogspot.co.id for example?
Is there a way to simply block all:
.blogspot.
because I'm getting blogspot.pt, blogspot.eu and all sorts.
To block all .*blogspot referers, you can use :
RewriteEngine on
RewriteCond %{HTTP_REFERER} ^https?://(www\.)?.+blogspot [NC]
RewriteRule ^ - [F,L]
Your existing rules dont match the referer string http://www.blogspot because of the extra dot infront of .blogspot in your cond pattern.
This morning, a lot of my website where tagged "this site may be compromised" by Google in it's result. Sites that are under my supervision on my own VPS server. I'ved run a deep scan on it and nothing unsual. I'ved look for suspicious htaccess and for javascript injection and nothing wrong so far.
Yesterday, I put an htaccess file to my web root to insure no sql, javascript, base64 and any other suspicious hacking solution might attack my server.
So I do suspect that Google add "this site may be compromised" since I add this protection to all my web sites.
there is the content of this htaccess :
RewriteEngine On
RewriteCond %{REQUEST_URI} !^/robots.txt
RewriteCond %{REQUEST_URI} !^/sitemap.xml
RewriteCond %{HTTP_USER_AGENT} ^-?$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^[bcdfghjklmnpqrstvwxz\ ]{8,}|^[0-9a-z]{15,}|^[0-9A-Za-z]{19,}|^[A-Za-z]{3,}\ [a-z]{4,}\ [a-z]{4,} [OR]
RewriteCond %{HTTP_USER_AGENT} ^<sc|<\?|^adwords|#nonymouse|Advanced\ Email\ Extractor|almaden|anonymous|Art-Online|autoemailspider|blogsearchbot-martin|CherryPicker|compatible\ \;|Crescent\ Internet\ ToolPack|Digger|DirectUpdate|Download\ Accelerator|^eCatch|echo\ extense|EmailCollector|EmailWolf|Extractor|flashget|frontpage|Go!Zilla|grub\ crawler|HTTPConnect|httplib|HttpProxy|HTTP\ agent|HTTrack|^ia_archive|IDBot|id-search|Indy\ Library|^Internet\ Explorer|^IPiumBot|Jakarta\ Commons|^Kapere|Microsoft\ Data|Microsoft\ URL|^minibot\(NaverRobot\)|^Moozilla|^Mozilla$|^MSIE|MJ12bot|Movable\ Type|NICErsPRO|^NPBot|Nutch|Nutscrape/|^Offline\ Explorer|^Offline\ Navigator|OmniExplorer|^Program\ Shareware|psycheclone|PussyCat|PycURL|python|QuepasaCreep|SiteMapper|Star\ Downloader|sucker|SurveyBot|Teleport\ Pro|Telesoft|TrackBack|Turing|TurnitinBot|^user|^User-Agent:\ |^User\ Agent:\ |vobsub|webbandit|WebCapture|webcollage|WebCopier|WebDAV|WebEmailExtractor|WebReaper|WEBsaver|WebStripper|WebZIP|widows|Wysigot|Zeus|Zeus.*Webster [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^curl|^Fetch\ API\ Request|GT\:\:WWW|^HTTP\:\:Lite|httplib|^Java/1.|^Java\ 1.|^LWP|libWeb|libwww|^PEAR|PECL\:\:HTTP|PHPCrawl|python|Rsync|Snoopy|^URI\:\:Fetch|WebDAV|^Wget [NC]
RewriteRule (.*) - [F]
RewriteCond %{REQUEST_METHOD} (GET|POST) [NC]
RewriteCond %{QUERY_STRING} ^(.*)(%3C|<)/?script(.*)$ [NC,OR]
RewriteCond %{QUERY_STRING} ^(.*)(%3D|=)?javascript(%3A|:)(.*)$ [NC,OR]
RewriteCond %{QUERY_STRING} ^(.*)document\.location\.href(.*)$ [OR]
RewriteCond %{QUERY_STRING} ^(.*)(%3D|=)http(%3A|:)(/|%2F){2}(.*)$ [NC,OR]
RewriteCond %{QUERY_STRING} ^(.*)base64_encode(.*)$ [OR]
RewriteCond %{QUERY_STRING} ^(.*)GLOBALS(=|[|%[0-9A-Z]{0,2})(.*)$ [OR]
RewriteCond %{QUERY_STRING} ^(.*)_REQUEST(=|[|%[0-9A-Z]{0,2})(.*)$ [OR]
RewriteCond %{QUERY_STRING} ^(.*)(SELECT(%20|\+)|UNION(%20|\+)ALL|INSERT(%20|\+)|DELETE(%20|\+)|CHAR\(|UPDATE(%20|\+)|REPLACE(%20|\+)|LIMIT(%20|\+))(.*)$ [NC]
RewriteRule (.*) - [F]
There is a lot of keyword within this file regarding hacking terminology ... is there any way that Google might look into the htaccess file ?
Should I block google with a robots.txt for this htaccess only or could/should I add a line of code directly into the htaccess to block Google for scanning this file... ?
What do you think ?
If .htaccess is visible from outside, then you have a serious problem. That file should never be visible by anybody accessing the site through http. Blocking it in robots.txt would just prevent well-behaved bots from looking at it. But bots that ignore robots.txt would still have access.
If you suspect that your .htaccess is the cause of the problem, you need to make sure that it can't be served. That's the default on Apache, but if you were mucking around with permissions I suppose you could have exposed it. If you did, you need to fix that.
I think you need to look somewhere else for the cause of Google's "this site may be compromised" message. A Google (or Bing) search on [this site may be compromised] reveals lots of information about why that warning might appear.
i am working on website of a Digital Gadgets Manufacturer.
Product images are hotlinked from hundreds of blogs & forums. which is causing bandwidth issues.
we want to replace all hotlinked images with their low resolution versions, using
.htaccess
means if the hotlinked image path is
http://www.example.com/products/gadget123/gadget123.png
we want to redirect it to
http://www.example.com/images/low-res/gadget123.png
hotlinked image paths are different, means they may be from sub-directory of any directory.
for example
/images/products/abc/gadget_abc200.jpg
/products/images/abc/gadgetabc5155_packing.png
/downloads/brochures/abc2012/abc2012_user_guide.jpg
etc...
but all low resolution images will be in
http://www.example.com/images/low-res/
directory, and their names will be same as their high resolution versions.
Try to addopt this:
RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^(.*?)example\.com(/.*)?$ [NC]
RewriteRule ^images/products/.+/(.+).png$ images/low-res/$1.png [L]
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^(.*?)example\.com(/.*)?$ [NC]
RewriteRule ^images/products/.+/(.+).png$ images/low-res/$1.png [L]
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^(.*?)example\.com(/.*)?$ [NC]
RewriteRule ^downloads/brochures/.+/(.+).jpg$ images/low-res/$1.jpg [L]
This rules looking for if the request comes from your site or directly from a third party.
And rewrites then the request.