UrlFetchApp.fetch on Google Spreadsheets cannot connect AWS backend webservice - security

We have an EC2 instance on AWS which we deployed our backend services to. We started by using Google Spreadsheets (scripted with Google Apps Script) to present our backend, through a webservice deployed on our server. We have a specific port from which https (uses a self-signed certifiate) protocol is used to serve the webservice encrypted while on flight. We had set up Security Groups (basically a firewall entry group) which include following CIDR ranges for that specific ingress port of our webservice:
64.18.0.0/20
64.233.160.0/19
66.102.0.0/20
66.249.80.0/20
72.14.192.0/18
74.125.0.0/16
173.194.0.0/16
207.126.144.0/20
209.85.128.0/17
216.58.192.0/19
216.239.32.0/19
as described in https://developers.google.com/apps-script/guides/jdbc#setup_for_google_cloud_sql
This setup was working fine until 5 days ago. Then something weird happened. When we run the script behind the spreadsheet from 'Script Editor' code
works fine and requests to our webservice return successfully. But when the exact same code was invoked through a menu item, it was not doing anything. After long frustrating investigation we found out that request was not even reaching our server (there were numerous other quirky symptoms like only last log command was visible on 'Execution Transcript' even though there should have been many others). Then we tried replacing the security group with a rule that accepts from any ip but to specific port everything was working fine again.
Here is a link to seemingly relevant issue in google-apps-script-issues page:
https://code.google.com/p/google-apps-script-issues/issues/detail?id=4679#c8
We ran tcpdump tcp port <port> -i eth0 -vv and observed that when we run the code from 'Script Editor' request was made from 66.102.7.156 (and from similar ips, which are in 66.102.0.0/20), when code is invoked from menu item in spreadsheet the request was made from 72.14.199.55 (and from similar ips, which are in 72.14.192.0/18). This one seems to be the problematic ip range.
My question is, why is it the case that when request sources are correctly included in firewall rules one block of ips don't work and starts to work when ip restriction on the port is lifted (source ip 0.0.0.0/0)? Is it a bug of Security Groups in AWS? Or are we doing something wrong? Also if our approach is not adequate in any way, alternative solutions or suggestions would be much appreciated.

As per the issues you linked to, there was a bug in Apps Script that lead to this behavior. That bug has now been fixed.

Related

Throttling/Restricting localtunnel-server traffic

We've developed a server software and for ease of use for end-users, we are using the localtunnel-server app on one of our linux servers to get around the need for port forwarding and messing around with firewalls.
The problem is that it seems to tunnel "all" traffic on the port 80. However, we are afraid of this being abused. We would like to restrict traffic somehow and I wanted to know if that was even possible.
For example, let's say our app uses the "/myapp" virtual directory on the localhost website. So if a request is supposed to go to http://localhost/myapp/index.html then the traffic gets tunneled to http://mytunnel.myserver.com/myapp/index.html
The problem is, if there are other sites running on localhost, http://localhost/someotherapp also gets through. We'd like to block urls that don't match a format or contain keywords such as "/myapp"
Is that even possible? And if so, any guidance on how to achieve this, would be greatly appreciated.

linux redirect localhost port to url port

I need to redirect localhost:8080 to http://url:8080/.
Some background:
I am using docker swarm stack services. One service (MAPS) creates a simple http server that lists xml files to port 8080 and another service (WAS) uses WebSphere Application Server that has a connector that uses these files, to be more precise it calls upon a file maps.xml that has the urls of the other files as http://localhost:8080/<file-name>.xml.
I know docker allows me to call on the service name and port within the services, thus I can use curl http://MAPS:8080/ from inside my WAS service and it outputs my list of xml files.
However, this will not always be true. The prod team may change the port number they want to publish or they might update the maps.xml file and forget to change localhost:8080 to MAPS:8080.
Is there a way to make it so any call to localhost:8080 gets redirected to another url, preferrably using a configuration file? I also need it to be lightweight since the WAS service is already quite heavy and I can't make it too large to deploy.
Solutions I tried:
iptables: Installed it on the WAS service container but when I tried using it it said my kernel was outdated
tinyproxy: Tried setting it up as a reverse proxy but I couldn't make it work
ncat with inetd: Tried to use this solution but it also didn't work
I am NO expert so please excuse any noob mistakes I made. And thanks in advance!
It is generally not a good idea to redirect localhost to another location as it might disrupt your local environment in surprising ways. Many packages depend on localhost being localhost :-)
it is possible to add MAPS to your hosts file (/etc/hosts) giving it the address of maps.

Redirect to different ports in Azure cloud service VM (with ARR)

I'm a bit stumped here so I'm hoping someone has some experience with this, the situation is like this:
I have an azure service with a cloud service "mysite.cloudapp.net".
In the service there is a VM "myVM".
In CPanel I have multiple subdomains that redirect to this service which will be named "customer1.site.com", "customer2.site.com" etc. This would be with SSL (wildcard certificate).
Each of these subdomains has to link to a certain port in the VM.
I don't have a problem linking the subdomains to "mysite.cloudapp.net" with DNS in CPanel.
However I'd like that if the url "customer1.site.com" was used that they get forwarded to port 8080/appname.
"customer2.site.com" would instead get forwarded to port 8081/appname. And so on. All these ports have tomcat applications running on them.
I searched around online and it seems ARR would likely be the solution to my problem. Sadly I haven't found a working setup which redirects to the correct ports.
If anyone has experience with a similar situation I would love some input. I'm not used to working with these systems and this setup is a bit more advanced than I expected going in.

Writing Server Addresses of Incoming Connections To File

So I am currently working on a destination based routing setup, and I'm really new to the world of Linux. I'm mainly trying to get this to work for video, so what I want to do is route any traffic that I specify through my vpn, and keep the rest of the traffic local. I tried to do this with BBC iPlayer, and I ended up reaching a roadblock because nslookup did not yield the server addresses for Akamai, BBC's CDN. I used tcptrack to find all the incoming connections onto my machine, and I sorted it by connection rate, and the top few would end up being the akamai servers. Well, once I figured this out, I am currently trying to automate the process, and I cannot get tcptrack to write to an output file. Does anyone know of a way to get tcptrack to write to a text file or know of a program that would be better suited to my purpose? What I currently do is use the route command and route the server address, both the one that tcptrack gets me and the nslookup address, through to my vpn using the syntax route add (server address) dev tun0. Any help would be appreciated!
Not sure if i understand the context correctly but if you want to direct specific traffic say based on client IP or domain name then this is possible using Akamai.

This webpage is not available, The connection to xxx.domain.pl was interrupted, only first pageload

I'm experiencing odd error while trying to load my web page in browser. When I haven't opened it for some period of time and then try to open it just by typing address in browser and clicking enter:
1) The page doesn't load - browser message that it is not available, connection to .... was interrupted
(in Opera there is also info about proxy, network... i can paste it later when error repeat again)
2) after refreshing, loading page again it works ok (without any problem)
My web page address is crib.pl and subdomains niemiecki.crib.pl, hiszpanski.crib.pl
it is important to note that when i try first time load for example niemiecki.crib.pl then it doesn't open but next opening hiszpanski.crib.pl will open normaly also.
Some additional info:
- hosting is in bluehost (Utha, USA)
- I'm trying to access this from (Poland, Europe)
- website is on drupal
- it works for more than 4 years without problem on this server
- it works even week ago without a problem and it doesn't work since 31 december 2014
- bluehost support doesn't have any idea, they say it works perfectly 1-to-1 cases (no problem)
(If you can check it and type your country and whether yes/no you are expiriencing similar problem)
- I haven't modified anything on the web page (problem just happens without my interaction)
- Google crawlers seems to have some problems with accessing robots.txt (sth like that) file
- domain is hosted by company in Poland (crib.pl) and this domain is set using external DNS to bluehost.com servers
Any help save my life, I'm experiencing about 50% drop in earnings since this problem!
Opera message:
"
This webpage is not available
The connection to crib.pl was interrupted.
Check your internet connection.
Check any cables and reboot any routers, modems, or other network devices you may be using.
Allow Opera to access the network in your firewall or antivirus settings.
If it is already listed as a program allowed to access the network, try removing it from the list and adding it again.
If you use a proxy server...
Check your proxy settings or contact your network administrator to make sure the proxy server is working. If you don't believe you should be using a proxy server: Go to Applications > System Preferences > Network > Advanced > Proxies and deselect any proxies that have been selected.
"
There is definitely something wrong with the Bluehost box (i.e. the server behind the IP address 66.147.244.170). From Australia at 2015-Jan-05 12:19:36 UTC, I was able to reproduce a "Connection reset by peer" error just using curl, which corresponds to the browser message "connection to .. was interrupted".
Other times, it just hangs while trying to establish a connection.
In addition, other servers on the same subnet also owned by Bluehost appear to be working fine.
For example:
$ telnet 66.147.244.22 80
Trying 66.147.244.22...
Connected to 66-147-244-22.unifiedlayer.com.
Escape character is '^]'.
^]
telnet> q
Connection closed.
This tells me that it is not a routing problem on the public Internet either.
Also, after a while I tried again it succeeded in opening a connection. So, you're right that the problem is intermittent.
In other words, I think the issue lies with this particular Bluehost box. It could be one of the following causes:
OS is out of file descriptors
Apache or whatever mail server is too slow to service requests and therefore has maxed out its listen backlog
other server resource limits (perhaps memory) or network equipment issue localized to the hosting environment
Best to check with Bluehost again. My guess is that one of the other tenants sharing that server is getting heavily loaded periodically.
Yes, as I thought before the problem was with bluehost box.
Now seems that the problem has been fixed. Here's what I have done:
1) I upgraded bluehost account (standard shared to pro shared)
I do this because want to change IP address and bluehost box without changing crib.pl
domain external DNS servers configurations (it is set to bluehost)
I also would like to have automatic migration, because haven't too much time now.
2) After upgrade I get new IP address and new bluehost box but it also didn't work correctly
3) So I switch on dedicated IP option and after that about 6 hours later when dedicated IP was propageted properly website seems to work again correctly (one problem it cost me about 120$ for next year and shortening my plan for 1 year then previously)
4) the most frustrating issue was bluehost technical support approach which wasn't eager to help me in any why even though the problem was in their server configuration not my code !

Resources