Hi Am trying to do perform api request using Databricks python notebook and receiving error that the ip was not whitelisted. Can someone explain how to know which IP is used by Databricks to connect to API so that I can whitelist it.
import requests
url = 'https://www.someapi.com'
myobj = {'somekey': 'somevalue'}
x = requests.post(url, json = myobj)
print(x.text)
Error: The IP you're trying to access is not whitelisted. Please whitelist using the following url
One approach would be to do %sh curl https://ifconfig.me/ to get your outgoing IP, but it may not be stable.
Another approach is to route outgoing traffic via NAT Gateway that will have stable IP(s). You can look through process in documentation.
Related
There is situation that access azure function http trigger behind a firewall, but we cannot use the ip to access http endpoint as in Microsoft documentation, anyone knows why? how is azure http trigger is designed?
I want to call http trigger with an ip address, How can I do that
Azure is using name-based virtual hosting which means multiple web apps (and functions) are hosted on the same pool of servers and share the same IP addresses. As a consequence, Azure needs an additional property to determine which function should be called when an HTTP request is sent to an IP address and this is achieved by setting the HTTP "Host" header.
To get the inbound IP of your function run the following in a terminal
nslookup <YourName>.azurewebsites.net
Assuming the IP address returned here is 20.51.3.70 you can then send an HTTP request to this IP but you have to specify the "Host" header like so:
curl -i -H "Host: <YourName>.azurewebsites.net" http://20.51.3.70/api/<FunctionName>?code=<APIKey>
When I try to access APIs on my remote server which I am connected through VPN, I get this error. Any help is appreciated.
As you have noted you are using a vpn, what type ?
1-Split tunneling VPN - Can use Postman directly to the API as long as the API server/Firewall/Load Balancer accepts traffic from your local workstation/asset
2-Tunnel mode-tunnel mode - Can use Postman via proxy settings to the API as long as the API server/Firewall/Load Balancer accepts traffic from your local workstation/asset, see the following link Postman - Using a proxy
3-Tunnel mode-tunnel mode with Internet Connection Shaing disabled - In this case you can only use Postman on a remote server/workstation (within the private network) where the traffic is accepted by the API server/Firewall/Load Balancer
Is it possible to get the public-ip of an amazon sagemaker notebook instance?
I was wondering if I can ssh into it using the public ip for remote debugging purposes.
I tried getting the public ip using the below curl command
$curl http://169.254.169.254/latest/meta-data
This just lists the local ip and not the public ip.
I also tried the below command.
$curl ifconfig.me
This returns an ip address like 13.232.96.15. If I try ssh into this it doesnt work.
Is there any other way we can do this?
Note : The ssh port 22 is open already in the security group
I don't think you can ssh to notebook instances. You can either use open them from the console, or grab the url with an API, re: https://docs.aws.amazon.com/sagemaker/latest/dg/howitworks-access-ws.html
If you need a terminal, then you can open one from Jupyter.
Though I have accepted Julien's answer, am pasting the reply I got from aws so that it may be helpful.
Question : Can we ssh into a sagemaker notebook instance?
Answer : No.
Question : Why not?
Answer : The notebook instance is formed as part of SageMaker's fully managed architecture. This means that all the underlying instances for any of the components of the service are deployed in a SageMaker managed environment and access to them is ONLY through SageMaker's API. For the notebook instance, the ONLY access that a customer has is through the Jupyter notebook ( or Jupyter lab ),for which you have to use the CreatePresignedNotebookInstanceUrl API in order to get an authorized URL and this does not include access via SSH .
The URL is public yes, but a customer will still be able to restrict access to only specific IP addresses[1] or connect to it through a VPC endpoint [2] .
[1] https://docs.aws.amazon.com/sagemaker/latest/dg/API_CreatePresignedNotebookInstanceUrl.html
[2] https://docs.aws.amazon.com/sagemaker/latest/dg/notebook-interface-endpoint.html
You can run this in sagemaker notebook instance to get public/external IP
from requests import get
ip = get('https://api.ipify.org').content.decode('utf8')
print('My public IP address is: {}'.format(ip))
Notebook can either have "Default communication with the internet" (this allows the notebook to communicate with the internet through a VPC managed by SageMaker) or "VPC communication with the internet" (You can create new/use existing VPC with NAT gateway and all the external traffic will go via NAT gateway IP controlled by you)
P.S. you are getting correct public IP but as mentioned in other answers sagemaker doesn't provide SSH access.
References:
https://stackoverflow.com/a/36205547/2073920
https://docs.aws.amazon.com/sagemaker/latest/dg/appendix-notebook-and-internet-access.html
I'm building an AWS Lambda function that pulls info from the Google Calendar API using Node.js. The Node application then sends an html response to a third party application. I'm basically following this guide from google, except I'm sending the html somewhere else to be rendered. I keep running into an error where Google tells me I haven't whitelisted the javascript origin coming from Lambda. Here's the error text:
{error: "idpiframe_initialization_failed", details: ""Not a valid origin for the client: https://<Lambda URL>.execute-api.us-west-2.amazonaws.com has not been whitelisted for client ID <My_Google_API_Client_Id>.apps.googleusercontent.com. Please go to https://console.developers.google.com/ and whitelist this origin for your project's client ID."}
I've gone into the Google API developer console and whitelisted that URL, but I'm still getting that error. Here's a screenshot.
I've tried whitelisting "us-west-2.amazonaws.com" as well with no luck. Thanks!!
You'll need to whitelist Lambda's IP addresses, you can't whitelist based on domain name. You can view the current list of IP addresses here: https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html
You note your request is coming from us-west-2 so you could whitelist the IPs from that region only by looking for entries matching that region. 122 by my count so thats a pretty big job!
Otherwise, you could look at setting up a NAT within a VPC and using a IP against that to whitelist that sole IP?
White list AWS Lambda functions' subnet NAT gateway elastic IP address. Follow the article.
I need to fetch data from external API, which has white-listed IP requirement i.e. API will respond only to white-listed IPs
I have multiple servers in AWS Autoscaling group that needs to fetch data from this external API. I intend to route these request via servers running on white-listed IP (AWS Elastic IPs).
I am trying to use socat:
socat -v TCP-LISTEN:80,reuseaddr,fork,su=nobody TCP:api.external-service.com:80. But getting Invalid URL error.
Is socat the right way to solve this problem? if so then how do I fix this issue?
You could set up a Squid proxy server on the machine with the white-listed IP and route all requests through the proxy.
http://www.squid-cache.org/