spark-submit to kubernetes cluster on Google cloud got 401 unauthorized - apache-spark

It's pretty standard spark-submit action, what's weird is that I got 401 from time to time, if I just wait a few minute, I can run again, until next time I got a 401.
it's very similar to this issue
https://jira.apache.org/jira/browse/SPARK-24227
I'm just wondering if I'm not setting the GOOGLE_APPLICATION_CREDENTIAL right when submitting, but at the very beginning I haven't set any credential and can still run.
Anyone has similar issues?

It's a subtle bug with Spark couldnt refresh the access token. One work around is:
Run a dummy kubectl command to refresh access token. E.g: kubectl get namespace

Related

Databricks List Jobs endpoint not returning job state

Good Morning,
I am currently working on running and monitoring Databricks Notebooks from Airflow. I am currently able to login, spin up a new cluster, and execute a notebook. The issue I have is the monitoring side of things. Using the 2.1 jobs API I am making the following call.
curl --location --request GET 'https://<MY-SERVER>.azuredatabricks.net/api/2.1/jobs/runs/list?job_id=<MY-Job-Id>
On the API page it looks like that API should feed back some stats about the job itself. Most importantly to me is state.result_state as I want to use it as a sensor.
My issue is that when I hit the list endpoint for any of my jobs the only thing I get back is
{
"has_more": false
}
I cant find anywhere in the documentation where I need to add something in the Notebook itself to omit the metric im looking for. Is there maybe an elevated set of permissions I need for my Auth Token that would give me the full set of metrics I am looking for?

SonarQube 8.9 doesn’t start, it’s blocked in loading page after login

i need help with SonarQube.
I’m trying to use the latest verions (8.9) on CentOS 7, i did all but i’m not able to see the main page because it gets stuck in the loading page after login.
There aren’t error in log file, the unique “error” that i found, if i inspect the source, is:
Application failed to start! SyntaxError: Unexpected token x in JSON at position 556.
Any suggest??
Error:
I solved the problem after time and time. It was so strange by the way after an accuratly check of the network i find the problem.
It was the firewall that blocked some request after login page, i hope this can help those in this same situation..
This is a network issue. The firewall is blocking the sonarqube request

How can I return 401 Unauthorized from a custom authorizer in API Gateway without causing a Lambda error?

I have a custom authorizer for AWS Cognito using Node JS in Lambda and it works great. It's basically using the code provided in this tutorial.
There are a number of ways my custom authorizer can fail: the user is using the wrong pool, the user submitted an invalid authorization token, etc. Whenever this happens, I call:
context.fail("Unauthorized");
and that works well too.
Unfortunately, when the user's token has expired (which happens all the time, especially first thing in the morning if somebody left their browser open), I also call that same method, which is the only way I know of to return a 401 message. This works and the user is returned back to the login page.
The problem is that in my Cloudwatch dashboard, where we're monitoring for broken code / suspicious activity, every time somebody leaves their browser open we see a Lambda error. This is very noisy and makes it hard to catch somebody doing something suspicious.
So my question: How can I fail the authorizer, return a 401 (or something else), not have the REST API execute and also NOT have an error show up in CloudWatch?

Randomly getting a login prompt

I am randomly getting a login prompt asking me to log in to localhost:3000 while working on my project (in localhost). Sometimes it does not show up the whole day and next morning I get 10 prompts in a row (canceling through pressing escape), and then it starts working properly again.
These are the errors I get in the console when it has been canceled:
I suspect one of the following, but really I have no idea:
Webpack is not configured correctly
There is some resource that requires authentication (but I can't figure out which), I am running everything on localhost
There is something wrong with the node.js server thingy...
Any tips on what might be going on are very welcome.
Edit: My backend is mocked in localhost.
Edit 2: I found this: Receiving login prompt using integrated windows authentication and it is possible that the thing that is causing the error is some faulty configuration in my IIS. But it is difficult to verify since it only shows up randomly.
Webpack has nothing to do with your API calls.
401 errors are due to auth credentials failing.
It means that your backend ask you to update your token(s) used for auth in API calls.

Azure Scheduled Job Fails to Execute

I had two scheduled jobs running in my developer program benefit subscription. I had set them up originally using the classic portal. I set them to run every hour since I am on Free Tier. Everything was working fine until I had to redeploy the code (console applications)
First problem - I could not find a way to easily update the code for the scheduled jobs so I deleted them and recreated them I tried to recreate them from the portal, first by creating the WebJob and then by going to the scheduled jobs collection and creating a schedule for the web job.
However each time it runs, it fails with the following error
Http Action - Response from host 'mysite.scm.azurewebsites.net':
'Unauthorized' Response Headers: Date: Thu, 16 Mar 2017 04:07:00 GMT
Server: Microsoft-IIS/8.0
WWW-Authenticate: Basic realm="site"
Body:
401 - Unauthorized: Access is denied due to invalid
credentials.....
And some other html stuff unrelated to the error
I tried also to deploy the job directly from Visual Studio 2015 (latest update)
however the same result occurs, running the job fails with the same error.
It is my understanding that even on free tier I should be able to run a scheduled job (5 of them) every hour.
Why is it failing and complaining about credentials?
EDIT: The job runs from App Service - WebJobs so there's nothing wrong with the job itself, the code executes correctly, I just can't get it to run from the Scheduler.
As far as I know, the “Access is denied due to invalid credentials” error is happened when you don’t set the Authentication information in your
Scheduler Job ‘s action settings tag.
The error is like below:
I suggest you find your webjob’s username and password firstly.
You could find it in the webjob’s properties tag.
Like below:
Then you could set the user name and password in the action’s setting.
It will work well.
Would you be able to run your schedule based off of a Cron expression? The post highlights how to use the settings.job file to provide a schedule to execute on. Here are some example cron expressions for your to figure out if this scenario would work.

Resources