I am having problem using EZProxy to access contents in intelliconnect.wkasiapacific.com
intelliconnect.wkasiapacific.com uses GWT (which uses .rpc extension to facilitate ajax call to server)
Is there anyway I can verify that this is indeed the problem (Ezproxy blocks *.rpc) and find a way to let Ezproxy allow *.rpc call ?
Many thanks
As a Technical Support Consultant for CCH I would be happy to assist you further, if you are still experiencing issues please contact us on 1300 300 224 if within Australia, (otherwise internationally either via your local CCH support hotline or +61 9857 1300 within business hours Australian Eastern Standard time to contact the Aust. Support team specifically) or email support#cch.com.au (or alternatively for EZproxy specific support contact OCLC)
To my knowledge though OCLC's EZproxy software does not block the download of data where the URL happens to use a ".rpc" extension – however it is possible (but unlikely) that a firewall/security-suite on the client-side could be configured to do this.
To troubleshoot you can use HTTP logging software to examine the HTTP request and response pairs. For Internet Explorer Microsoft's free "Fiddler" tool can be used. See: http://www.fiddler2.com
Nat
Related
I am using the node-whatsapi library
I am getting the number blocked. I am following the exact protocol as mentioned in the WIKI.
The flow that i follow is:
Create Adapter
Connect()
On Connect - Login
On Login
sendIsOnline()
requestPrivacySettings()
requestServerProperties()
requestContactsSync()
For Each Contact sendPresenceSubscription(), getStatus() and getProfilePicture()
And when i receive a message:
adapter.requestContactsSync('1234567890', 'delta', 'background');
getProfilePicture()
Save to DB
Now, what we do and why do we use WhatsApi
We enable our existing Customers to talk to their internal teams on WhatsApp. So, a customer initates a chat and a help desk team responds to them on an application.
Got the number blocked now. Unable to understand where am i going wrong.
Also, the total number of messages exchanged between the customer and the helpdesk team is around 1k a day.
What you're "doing wrong" is using a third party API which is against WhatsApp's terms of service and, if detected, will get your relevant accounts blocked. It's not a fault with WhatsAPI itself or how you are using it.
WhatsApp openly publishes the only approved/compliant way to programmatically interact with their network.
http://www.whatsapp.com/faq/en/iphone/23559013
http://www.whatsapp.com/faq/en/android/28000012
Outside of that, you are going to get blocked whenever you are detected as using an unapproved means of interacting with their network.
In your particular example you're using a Node.js port of the original WhatsAPI. As of May 2015 this is the kind of pressure they are dealing with from WhatsApp (despite many years of trying to negotiate an amicable compromise with them around things like message limits and identity verification):
It sucks but that's just how it is. You can look at some alternatives which are still actively updated and may continue working for a while, but given that WhatsApp is now owned by Facebook and considering the kind of legal resources at their disposal, you should be able to see why one might be reluctant to continue updating a rogue API.
Well, got a satisfactory answer from WhatsApi collaborator matteocontrini.
Here is the answer that i got, if somebody is intrested.
It says:
the reason of getting blocked doesn't have to be because you wrote
wrong code. It could be a filter on the kind of messages you send or a
report from someone about your number.
I'm starting to look at the new Office 365 Unified API (specifically around Exchange for the moment).
In the existing Mail API at you can make an authenticated call to https://outlook.office365.com/api/v1.0/me/folders/Inbox/messages in order to get messages in the Inbox. You can also hit folders/Drafts/messages, folders/SentItems/messages and folders/DeletedItems/messages to retrieve messages from Drafts, Sent Items and Deleted Items respectively. This is documented at https://msdn.microsoft.com/office/office365/APi/mail-rest-operations#AllMailAPIoperations.
Problem is, I can't find (either in documentation at https://msdn.microsoft.com/office/office365/HowTo/office-365-unified-api-reference#msg_ref_relationship_Messages or by messing about on https://graphexplorer2.azurewebsites.net) how to drill into these folders when accessing via the Office 365 Unified API.
The documented URL to access messages via the new Unified API is https://graph.microsoft.com/beta/me/Messages, and so I was hoping I could hit something like Messages/Inbox or Messages/folders/Inbox, but no luck so far.
Does anyone know if these paths are currently missing or addressed differently, or if it's the intention that you find what you want by using $search rather than drilling into specific folders?
This isn't currently supported in Unified API, but is high on our priority list, and we are working on it. Sorry for the confusion. You can continue to use outlook.office365.com endpoint for now, to access folder info.
Having checked back (and read through https://graph.microsoft.com/beta/$metadata) this has now been implemented. You access folders via MailFolders/[folder name]/messages, so:
https://graph.microsoft.com/beta/me/MailFolders/Inbox/messages
https://graph.microsoft.com/beta/me/MailFolders/Drafts/messages
https://graph.microsoft.com/beta/me/MailFolders/SentItems/messages
..etc.
I have couple Android apps on PlayStore, which use In-App purchases. I use Google App Engine for my backend. I see some users calling the APIs abnormally/repeatedly (may be to reverse engineer or hack?). I can figure out the IP address, Gmail ID, etc. How to prevent these people from accessing my API?
One suggestion is to use dos.xml
But these morons seem to constantly change the IP addresses, so it is painful to keep updating this list.
Is there a way in App Engine to black list users? or computers/devices?
If we know the google(Gmail) Ids of these ba*t*r*s, how/where do we report those? This page seems to be the right place to start, but it is not clear where to send email.
This page seems be more appropriate for vulnerabilities, but this is not such a case.
"Viewing top users in the Administration Console" section in DoS page says I should see a table of IP addresses which are using the API frequently. But I dont see such table in Admin console. Do I need to be a paid (Google App Engine) user?
Any help is greatly appreciated.
Yes, GAE allows for a blacklist, via dos.xml (dos.yaml for Python or PHP). If you don't want to have to keep updating the IP addresses, you may just have to check the user id, and serve them some message. But, that requires actually servicing the request, to check the id, etc. So, if it is a true DOS attack, it will succeed, as you have to still service the request. Using dos.xml cuts that off at the backend, so would be the best way to go.
I suggest a script to log the IP addresses in real time for those you want to ban, to make updating dos.xml less painful.
I am developing a website -- in the prototype stage, soon to be alpha. I will provide an email address to each account that allows the user to deposit stuff -- not a real email account, just an endpoint for sending things to the site. Many sites provide this kind of service nowadays. I think the first one I saw was Photobucket, which let's you send photos as email attachments.
My question is, what is the best way to implement this kind of service?
In my prototype, I have written a POP3 client which fetches all newly delivered mail (currently from a test Gmail account). My service processes each new mail and attachments, and immediately removes it from the email server.
I could certainly outsource to an email service with POP3 and be done with it. The problem is cost. Most services I have seen provide much more than I need, and they charge per account. I expect to have many accounts and low traffic volume.
So I'm leaning towards hosting email receipt myself. I am open to Windows or Linux. The code that processes incoming emails runs on Windows, but I have other services running on Linux. I have seen a number of open source and free email servers, such as hMailServer and MailEnable (Windows) and qmail, Postfix and exim (Linux).
I guess I have a slight preference towards Linux because of lower hosting costs, but if a Windows service can provide cleaner integration, that might be worth it. As far as features, I would like to have some spam filtering, but it's is not a huge priority. POP3 is adequate for retrieval, but a more direct API would be nice. I will need some kind of API for programmatically provisioning new accounts.
All suggestions are appreciated. Do you know how others implement this kind of service?
UPDATE: I ended up using hMailServer, which is a free mail server that runs on Windows. It seems to be quite mature and robust. It has a COM interop library which makes accessing emails, accounts, etc. from my .NET server app very easy indeed.
If you're going the host-your-own-email-server route, I would probably just use POSTFIX and pipe all your email to a PHP script, which processes the email.
Here's a quick'n dirty tutorial on setting up the email pipe if you're using cPanel:
http://kb.siteground.com/article/How_to_pipe_an_email_to_a_PHP_script.html
If not, here's how to do it:
http://answers.google.com/answers/threadview?id=562518
The bottom line is, you need to have an open SMTP connection to accept email. If you have your own server, then you can install a SMTP server on the machine. Usually, you have filesystem access to the location the email files are placed. Be sure to select a SMTP server that allows this, and that the email are in a format that you can parse.
Then, you can just monitor the file location for incoming emails.
If you can't pipe your emails (using the Postfix suggestion), and you don't have your own server (for example, on a shared hosting plan), then you will need to query a POP3 or IMAP mailbox server for your emails, and parse them accordingly.
I wanted to get emails in real time so I worked out my own solution with google app engine. I basically made a small dedicated google app engine app to receive and POST emails to my main site. That way I could avoid having to set up an email server.
You can check out Emailization (a little weekend project I did to do it for you), or you this small GAE app that should do the trick.
I kinda explained it more on another question.
Hope that helps!
I'm building an e-commerce website with a large database of products. Of course, is nice when Goggle indexes all products of the website. But what if some competitor wants Web Scrape the website and get all images and product descriptions?
I was observing some websites with similar lists of products, and they place a CAPTCHA, so "only humans" can read the list of products. The drawback is... it is invisible for Google, Yahoo or another "Well behaved" bots.
You can discover the IP addresses the Google and others are using by checking visitor IPs with whois (in the command line or on a web site). Then, once you've accumulated a stash of legit search engines, allow them into your product list without the CAPTCHA.
If you're worried about competitors using your text or images, how about a watermark or customized text?
Let them take your images and you'd have your logo on their site!
Since a potential screen-scaping application can spoof the user agent and HTTP referrer (for images) in the header and use a time schedule that is similar to a human browser, it is not possible to completely stop professional scrapers. But you can check for these things nevertheless and prevent casual scraping.
I personally find Captchas annoying for anything other than signing up on a site.
One technique you could try is the "honey pot" method: it can be done either by mining log files are via some simple scripting.
The basic process is you build your own "blacklist" of scraper IPs based by looking for IP addresses which look at 2+ unrelated products in a very short period of time. Chances are these IPs belong to Machines. You can then do a reverse lookup on them to determine if they are nice (like GoogleBot or Slurp) or bad.
Block webscrapers is not easy, and it's even harder trying to avoid false positives.
Anyway you can add some netrange to a whitelist, and don't serve any captcha to them.
All those well known crawlers: Bing, Googlebot, Yahoo etc.. use always specific netranges when crawling, and all those IP addresses resolve to specific reverse lookups.
Few examples:
Google IP 66.249.65.32 resolves to crawl-66-249-65-32.googlebot.com
Bing IP 157.55.39.139 resolves to msnbot-157-55-39-139.search.msn.com
Yahoo IP 74.6.254.109 resolves to h049.crawl.yahoo.net
So let's say that '*.googlebot.com ', '*.search.msn.com ' and '*.crawl.yahoo.net ' addresses should be whitelisted.
There are plenty of white lists you can implement out on internet.
Said that, I don't believe Captcha is a solution against advanced scrapers, since services such as deathbycaptcha.com or 2captcha.com promise to solve any kind of captcha within seconds.
Please have a look into our wiki http://www.scrapesentry.com/scraping-wiki/ we wrote many articles on how to prevent, detect and block web-scrapers.
Perhaps I over-simplify, but if your concern is about server performance then providing an API would lessen the need for scrapers, and save you band/width processor time.
Other thoughts listed here:
http://blog.screen-scraper.com/2009/08/17/further-thoughts-on-hindering-screen-scraping/