I am trying to establish an AWS lambda function which calls a databricks notebook (in the event of an s3 trigger).I understand I have to use the Jobs API of databricks in my lambda function(python) code to make a POST request using the JSON payload of the runs-submit function.
Although the documentation is not very clear, I was able to call a test script and on checking the response text I see the databricks login page html code which means it is not getting authenticated .
I did read on user tokens but I am not sure how to even incorporate them for authentication.
Any help of making this work in other ways or helping me use the user_tokens to get authenticated so that the flow reaches the execution of the notebook rather than getting stopped at authentication page would be helpful.
Thanks in advance.
Code Sample:
import requests
import json
job_payload = {
"run_name": 'just_a_run',
"existing_cluster_id": '****',
"notebook_task":
{
"notebook_path": 'https://databricks.cloud.company.com/****'
}
}
resp = requests.post('https://databricks.cloud.company.com/2.0/jobs/runs/submit', json=job_payload)
print(resp.status_code)
print(resp.text)
200
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8"/>
<meta http-equiv="Content-Language" content="en"/>
<title>Databricks - Sign In</title>
<meta name="viewport" content="width=960">
<link rel="stylesheet" href="/login/bootstrap.min.css">
<link rel="icon" type="image/png" href="login/favicon.ico" />
<meta http-equiv="content-type" content="text/html; charset=UTF8">
<link rel="shortcut icon" href="favicon.ico"><link href="login/login.e555bb48.css" rel="stylesheet"></head>
<body>
<div id="login-page"></div>
<script type="text/javascript" src="login/login.dabd48fd.js"></script></body>
</html>
SOLVED:
1) You will need to create a user token for authorization and send it as 'headers' parameter while performing the REST request.
2) headers={'Authorization': 'Bearer token'}
In place of token must be your actual token that you get from databricks.
3) The api link must start with /api
4) Path to the databricks notebook must be absolute path
i.e. "/Users/$USER_NAME/book_name"
Final Working Code:
import requests
import json
job_payload = {
"run_name": 'just_a_run',
"existing_cluster_id": 'id_of_cluster',
"notebook_task":
{
"notebook_path": '/Users/username/notebook_name'
}
}
resp = requests.post('https://databricks.cloud.company.com/api/2.0/jobs/runs/submit', json=job_payload, headers={'Authorization': 'Bearer token'})
print(resp.status_code)
print(resp.text)
Related
our app need to do around 37000 pages generation with an array generated by getStaticPath
the getStaticProps need to do a request on our api for build each page, but we have currently some timeout request.
Then, i will try to add a little timer in each request (i think about 0.1 second) but i didn't find anywhere a nextjs parameter to do this
Anyone allready do this?
Ok so for your case I think you should use getServerSideProps, you told that you need Dynamic SEO for each page and that is exactly what getServerSideProps id for, the idea is to render specific meta that depend on the data you get from your request, here is an example below:
export const getServerSideProps = async (context) => {
try {
let { data } = await axios.get("YOUR_URL")
return {
props: {
data,
},
}
} catch (e) {
console.log(e)
}
}
And in your component
<Head>
<link rel="canonical" href={canonicalUrl} />
<title>{title}</title>
<meta name="title" content={data.productTitle} />
<meta name="description" content={data.description} />
...etc
</Head>
I'm trying to verify and decode id token sent by front-end.
I get this error when i run the verifyfunction.
Sometimes it might work.
No pem found for envelope: {"alg":"RS256","kid":"53c666482db3800c83c63","typ":"JWT"}
This is my code
const ticket = await client.verifyIdToken({
idToken: token,
audience: '804312674051-5o4.apps.googleusercontent.com',
});
const payload = ticket.getPayload();
I finally found the answer today.
The Firebase tool will connect the native Google to the third-party login token, and then encapsulate another layer. The token obtained at this time is no longer the original token given to us by Google.
A1:
Original Token: GoogleDesignInAccount Account = Task.getResult(ApiException.class);
Account.getidToken () // This is the original token
B1:
Firebase token: FireBaseUser currentUser = Mauth.getCurrentUser ();
String token = currentUser.getIdToken(false).getResult().getToken();
A2:
Google officially provides a method to verify the token
B2:
Firebase officially provides the authentication token method
We use code names for the four data points above. If you need to verify the validity of tokens in the background, they must correspond to each other, A1 to A2 and B1 to B2. If you use A2 to validate the B1, it will fail
I got the same problem....using a idToken that I got from my firebase login at my reactJS app.
I found that in the google src files
if (!certs.hasOwnProperty(envelope.kid)) {
// If this is not present, then there's no reason to attempt verification
throw new Error('No pem found for envelope: ' + JSON.stringify(envelope));
}
But I have no idea what that means.
The problem is the token used.
you can use this example for generate the token, only change the content="YOUR_CLIENT_ID.apps.googleusercontent.com" for a valid cliente id google
Test with the generated token in console.log
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="google-signin-client_id" content="YOUR_CLIENT_ID.apps.googleusercontent.com">
<title>Demo Sing-In</title>
</head>
<body>
<h1>Google Sing-In</h1>
<div class="g-signin2" data-onsuccess="onSignIn"></div>
Sign out
<script src="https://apis.google.com/js/platform.js" async defer></script>
<script>
function onSignIn(googleUser) {
var profile = googleUser.getBasicProfile();
console.log('ID: ' + profile.getId()); // Do not send to your backend! Use an ID token instead.
console.log('Name: ' + profile.getName());
console.log('Image URL: ' + profile.getImageUrl());
console.log('Email: ' + profile.getEmail()); // This is null if the 'email' scope is not present.
var id_token = googleUser.getAuthResponse().id_token;
console.log(id_token);
}
function signOut() {
var auth2 = gapi.auth2.getAuthInstance();
auth2.signOut().then(function () {
console.log('User signed out.');
});
}
</script>
</body>
</html>
I have created a node/express API to return base64 data as an image.
The image loads fine using https://www.example.com/api/resident/59dbd52a7604450012c390ac/image
I have been using this in various pages on the front end for images.
The problem is, when I try to use this data for the og:image tag on Facebook I get an error:
"Provided og:image URL, https://www.example.com/api/resident/59dbd52a7604450012c390ac/image could not be processed as an image because it has an invalid content type."
I know that Facebook can see the correct meta tag as when I click "See exactly what our scraper sees for your URL" it shows the correct tag. When I click the URL provided it shows the image.
The controller in express looks like:
export function image(req, res) {
var base64Data = req.resident.picture;
var img = new Buffer(base64Data, 'base64');
res.writeHead(200, {
'Content-Type': 'image/jpeg',
'Content-Length': img.length
});
return res.end(img);
}
The tags in the HTML look like this:
<meta property="og:title" content="My Site">
<meta property="og:image" content="https://www.example.com/api/resident/59dbd52a7604450012c390ac/image">
<meta property="og:image:type" content="image/jpeg">
<meta property="og:image:height" content="800">
<meta property="og:image:width" content="800">
I am getting the error message below when I run a HTTP Triggered function app on our main slot :
Status: 502 Bad Gateway
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"/>
<title>502 - Web server received an invalid response while acting as a gateway or proxy server.</title>
<style type="text/css">
<!--
body{margin:0;font-size:.7em;font-family:Verdana, Arial, Helvetica, sans-serif;background:#EEEEEE;}
fieldset{padding:0 15px 10px 15px;}
h1{font-size:2.4em;margin:0;color:#FFF;}
h2{font-size:1.7em;margin:0;color:#CC0000;}
h3{font-size:1.2em;margin:10px 0 0 0;color:#000000;}
#header{width:96%;margin:0 0 0 0;padding:6px 2% 6px 2%;font-family:"trebuchet MS", Verdana, sans-serif;color:#FFF;
background-color:#555555;}
#content{margin:0 0 0 2%;position:relative;}
.content-container{background:#FFF;width:96%;margin-top:8px;padding:10px;position:relative;}
-->
</style>
</head>
<body>
<div id="header"><h1>Server Error</h1></div>
<div id="content">
<div class="content-container"><fieldset>
<h2>502 - Web server received an invalid response while acting as a gateway or proxy server.</h2>
<h3>There is a problem with the page you are looking for, and it cannot be displayed. When the Web server (while acting as a gateway or proxy) contacted the upstream content server, it received an invalid response from the content server.</h3>
</fieldset></div>
</div>
</body>
</html>
But when I tried to create a new slot and function app with the same exact code, it works fine without getting the error above. It seems like there's an issue with our main slot's configuration but I just can't find any resource to point me to it.
Has anyone else encountered this problem? How did you fix it?
I have this problem when I sent object from MongoDB (by Mongoose method: Model.find()) directly to the response.
Claim.find()
.then(claims => {
context.res = {
status: 200,
headers: {
'Content-Type': 'application/json',
'Access-Control-Allow-Origin': '*'
},
body: claims
};
context.done();
})
I must change this body object to:
Claim.find()
.then(claims => {
context.res = {
status: 200,
headers: {
'Content-Type': 'application/json',
'Access-Control-Allow-Origin': '*'
},
body: JSON.parse(JSON.stringify(claims))
};
context.done();
})
I have seen 502 error many times in different situations.
Most times it is my programming mistake/ config issue.
But today it took a lot more time to find this issue, so sharing here.
Issue #1
s3 = new AWS.S3();
AWS.config.loadFromPath(dirName1 + '\\aws-config.json');
s3.listObjectsV2(..);
Here, though the error is at line 1, the issue happens at line 3
Solution is:
AWS.config.loadFromPath(dirName1 + '\\aws-config.json');
s3 = new AWS.S3();
s3.listObjectsV2(..);
That is, after loading the aws config, we need to create s3 new instance variable.
Issue #2
As given here, https://stackoverflow.com/a/39240447/984471, is about writing file to un-accessible directory, while you can write to D:\local\Temp, we can't write other directories like current directory or D:\
Unfortunately, in the above examples and also other instances I had, the Azure function exists without any log in azure function console, so there is need for a lot debugging required.
I was also getting the same error. I wrote " context.done(null,output)" at the end of event handler function and it is working fine.
In my case it was a missing WEBSITE_NODE_DEFAULT_VERSION application setting (Node.js Windows function app, error occurs in both Premium and Consumption hosting plans).
The app setting is set automatically if you create a function app in Azure Portal, but you need to set it explicitly if you use an IaC tool (Terraform in my case).
"WEBSITE_NODE_DEFAULT_VERSION" = "~14"
I had the same error and i simply forgot to add the "Content-Type" Header "application/json". After adding the header it works fine for me
If you get 502:s and even lack application insights log entries...
Sanity check:
Have you deployed your application?
I know that there's a bunch of questions about the "not allowed by Access-Control-Allow-Origin." error.
But I've tried some of them without success. :(
Some appointments:
I'm trying to build a dev-tools-tab extension
I can touch flickr API like the example shows
I can't reach localhost
Already tried several permission wildcards
http://localhost/
http://*/
*://*/
Already tried pack'd and unpack'd extensions
currently, manifest.json has
"version": "0.0.1",
"manifest_version": 2,
"devtools_page": "components/devtools.html",
"permissions": [
"http://*/"
]
devtools.html
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title></title>
</head>
<body>
<script src="../js/devtools.js"></script>
</body>
</html>
and, devtools.js
(function (window) {
"use strict";
var xhr1, xhr2, url;
xhr1 = new window.XMLHttpRequest();
xhr2 = new window.XMLHttpRequest();
xhr1.onreadystatechange = function () {
if (this.readyState === 4) {
console.log('flickr ok');
}
};
xhr2.onreadystatechange = function () {
console.log(this.readyState);
if (this.readyState === 4) {
console.log(this.responseText);
}
};
url = 'https://secure.flickr.com/services/rest/?' +
'method=flickr.photos.search&' +
'api_key=90485e931f687a9b9c2a66bf58a3861a&' +
'text=' + encodeURIComponent('cats') + '&' +
'safe_search=1&' +
'content_type=1&' +
'sort=interestingness-desc&' +
'per_page=20';
xhr1.open('get', url, true);
xhr1.send();
url = 'http://apache.local';
xhr2.open('get', url, true);
xhr2.setRequestHeader('Origin', url);
xhr2.send();
Chrome console output:
1 devtools.js:12
Refused to set unsafe header "Origin" devtools.html:1
XMLHttpRequest cannot load http://apache.local/. Origin chrome-extension://nafbpegjhkifjgmlkjpaaglhdpjchlhk is not allowed by Access-Control-Allow-Origin. devtools.html:1
4 devtools.js:12
flickr ok devtools.js:8
Chrome version:
28.0.1500.20 dev
Thanks in any advice.
I've got it!
Actually, the problem is that I'm trying to perform XHR requests on devtools page and it seems to have no permissions to bypass cross-origin-access policies like a popup page do.
Devtools tab tries are also unsuccessful.
edit
Is an stage-permission related. Not wildcard-permission. As I've said, I've managed to perform queries on some domains, yet not having they explicitly on my permissions array.
The problem really lies on the type of script running.
The same script, if used as a popup, work'd fine. So, I've tried as an background-script with success too! I was facing the problem that devtools_page and related doesn't have such permissions...
The APIs available to extension pages within the Developer Tools window include all devtools modules listed above and chrome.extension API. Other extension APIs are not available to the Developer Tools pages, but you may invoke them by sending a request to the background page of your extension, similarly to how it's done in the content scripts.
http://developer.chrome.com/extensions/devtools.html
That level of script denies non explicit cross xhrs.
Solved the problem putting the requests in a background script and using messages api.
Thank you!