How to poll request status in aws-cli? - aws-cli

I am updating a QuickSight data source in my aws account.
aws quicksight update-data-source --cli-input-json file://update-stag-data-source-request.json --output json
And I get the following response:
{
"Status": 202,
"Arn": "arn:aws:quicksight:eu-west-1:<my-aws-account-nr>:datasource/099676d0-99e3-44d7-b581-d6e532e72961",
"DataSourceId": "099676d0-99e3-44d7-b581-d6e532e72961",
"UpdateStatus": "UPDATE_IN_PROGRESS",
"RequestId": "1d304a80-e507-46c3-acb3-237a58237e77"
}
So currently the status of this request is "UPDATE_IN_PROGRESS", but how do I track the status afterwards?
I need to do it, because it seems that the update fails eventually, for reasons unknown. I know that, because I still see the old setup of the data source several minutes later. I believe, if I knew the eventual request status it would help me to debug the issue.

Check the command describe-data-source, it will return the DataSource.Status and, in case of any failure, you can check inside DataSource.ErrorInfo.Message.

Related

Get Azure Data Factory logs

I need to retrieve azure data factory pipelines execution logs. I've tried with Web Acticity by using the following request:
GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataFactory/factories/{factoryName}/pipelineruns/{runId}?api-version=2018-06-01
Unfortunatelly I have the following error:
Invoking Web Activity failed with HttpStatusCode - 'NotFound', message - 'The requested resource does not exist on the server. Please verify the request server and retry'
Should I set some additionall configuration to be able to retrieve these logs?
You can get a list of pipeline runs using Pipeline Runs - Query By Factory
Next, you can Get a pipeline run by its run ID.
GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataFactory/factories/{factoryName}/pipelineruns/{runId}?api-version=2018-06-01
Here is a sample URL:
https://management.azure.com/subscriptions/b83c1ed3-XXXX-XXX-XXXXX-2n83a074t23f/resourceGroups/resource-grp/providers/Microsoft.DataFactory/factories/ktestadf/pipelineruns/0bdaba11-47b7-4885-9796-5801b4bb856a?api-version=2018-06-01
If you are constructing URL dynamically, using Pipeline RunID system variable, you can using string interpolation method. Notice #{pipeline().RunId} in place of {runId}.
https://management.azure.com/subscriptions/b83c1ed3-XXXX-XXXX-XXXXX-2n83a074t23f/resourceGroups/resource-grp/providers/Microsoft.DataFactory/factories/ktestadf/pipelineruns/#{pipeline().RunId}?api-version=2018-06-01
Note: You would have to Trigger run and not debug, since this will create pipeline. Make sure you have published all before trigering
run, debug can take the changes but pipeline run needs the changes be
published.
And here is a simple WebActivity setup:
Input
{
"url": " https://management.azure.com/subscriptions/b83c1td3-XXXX-XXXX-XXXXX-2b83a074c13f/resourceGroups/myrg/providers/Microsoft.DataFactory/factories/ktestadf/pipelineruns/0bdaba11-47b7-4885-9796-5801b4bb856a?api-version=2018-06-01 ",
"method": "GET",
"headers": {
"Content-Type": "application/json"
},
"authentication": {
"type": "MSI",
"resource": " https://management.azure.com/ "
}
}
You can get a pipeline run ID from here manually to test.
I did repro before posting this and the only reason this error pops is cause any value you provided is wrong or does not already exist (in case of pipeline run id)

Pact-python : Consumer pacts for POST calls are failing

I am trying to create consumer side pacts for a POST end point using the pact-python library. But it's failing with the error saying "Missing requests".
Here't the client code which makes the POST API call
def create_user(request):
return requests.post("http://localhost:1234/user", data=request).json()
Here's my test class which create the consumer pacts.
class TestUserConsumer(unittest.TestCase):
def test_user_creation(self):
request = {
"name": "Micky",
"age": 0
}
response = {
"id": 1232,
"name": "Micky",
"age": 0
}
pact = Consumer("user_client").has_pact_with(Provider("user_server"))
pact.start_service()
pact.with_request(
method='post',
path='/user',
body=request
).will_respond_with(status=200, body=response)
with pact:
create_user(request)
pact.verify()
pact.stop_service()
The test failed with the following error.
line 268, in verify
assert resp.status_code == 200, resp.text
AssertionError: Actual interactions do not match expected interactions for mock MockService.
Missing requests:
POST /user
The create_user(request) is getting executed, but still the interactions are not recorded on the pact mock server.
Note : The GET API pact creations are working. Only the POSTs are failing.
Appreciate the help.
I figured out the problem. I was not converting my dictionary to json before making the request. Hence the request body format was incorrectly send. This caused a failure on the mock server while verifying the pact.
I also noticed that, the logs didn't get generated initially. This was due to my assertions added before stopping the server. Since the assertions got failed, the pact mock server didn't get stopped. Hence the logs are not generated at all. Once I stopped the server, the logs got added and that helped me to identify the problem.

How to delete the data from the Azure application insights

I want to delete the traces data from my application insights table. Can one tell me how do i do that.
I'm following the below article from microsoft,
https://learn.microsoft.com/en-us/rest/api/application-insights/components/purge
I'm trying to pass the below Request body,
{
"table": "traces",
"filters": [
{
"column": "timestamp",
"operator": ">",
"value": "2017-09-01T00:00:00"
}
]
}
I'm getting Response Code: 202, but i'm still seeing the data in the analytics, where i query traces.
Please let me know, how i can delete the data with traces on my application insights resource.
As per this SO issue:
Yes, it is possible to purge Application Insights data, but it may take a while (e.g. 2-3 days) for the operation to complete.
So please wait:)
For traces delete operation you would need to use the separate table name as "Traces" in the body of purge operation. then you can check the status using get purge status REST API operation. Refer to know more - https://sanganakauthority.blogspot.com/2019/06/how-to-delete-or-purge-azure.html

Azure ML: Getting Error 503: NoMoreResources to any web service API even when I only make 1 request

Getting the following response even when I make one request (concurrency set to 200) to a web service.
{ status: 503, headers: '{"content-length":"174","content-type":"application/json; charset=utf-8","etag":"\"8ce068bf420a485c8096065ea3e4f436\"","server":"Microsoft-HTTPAPI/2.0","x-ms-request-id":"d5c56cdd-644f-48ba-ba2b-6eb444975e4c","date":"Mon, 15 Feb 2016 04:54:01 GMT","connection":"close"}', body: '{"error":{"code":"ServiceUnavailable","message":"Service is temporarily unavailable.","details":[{"code":"NoMoreResources","message":"No resources available for request."}]}}' }
The request-response web service is a recommender retraining web service with the training set containing close to 200k records. The training set is already present in my ML studio dataset, only 10-15 extra records are passed in the request. The same experiment was working flawlessly till 13th Feb 2016. I have already tried increasing the concurrency but still the same issue. I even reduced the size of the training set to 20 records, still didn't work.
I have two web service both doing something similar and both aren't working since 13th Feb 2016.
Finally, I created a really small experiment ( skill.csv --> split row ---> web output ) which doesn't take any input. It just has to return some part of the dataset. Did not work, response code 503.
The logs I got are as follows
{
"version": "2014-10-01",
"diagnostics": [{
.....
{
"type": "GetResourceEndEvent",
"timestamp": 13.1362,
"resourceId": "5e2d653c2b214e4dad2927210af4a436.865467b9e7c5410e9ebe829abd0050cd.v1-default-111",
"status": "Failure",
"error": "The Uri for the target storage location is not specified. Please consider changing the request's location mode."
},
{
"type": "InitializationSummary",
"time": "2016-02-15T04:46:18.3651714Z",
"status": "Failure",
"error": "The Uri for the target storage location is not specified. Please consider changing the request's location mode."
}
]
}
What am I missing? Or am I doing it completely wrong?
Thank you in advance.
PS: Data is stored in mongoDB and then imported as CSV
This was an Azure problem. I quote the Microsoft guy,
We believe we have isolated the issue impacting tour service and we are currently working on a fix. We will be able to deploy this in the next couple of days. The problem is impacting only the ASIA AzureML region at this time, so if this is an option for you, might I suggest using a workspace in either the US or EU region until the fix gets rolled out here.
To view the complete discussion, click here

Browserstack reports successful even when test fails in Nightwatchjs

I just started using nightwatch with browserstack and I'm noticing that when we get a failed test, nightwatch registers the failure, but browserstack does not.
sample test I am using.
Also I am using free trial version of BrowserStack.
My question is:
Are there any ideas how to tell browserstack when a test run failed ?
From BrowserStack doc:
REST API
It is possible to mark tests as either a pass or a fail, using the
following snippet:
var request = require("request");
request({
uri: "https://user:key#www.browserstack.com/automate/sessions/<session-id>.json",
method: "PUT",
form: {
"status": "completed",
"reason":""
}
});
The two
potential values for status can either be completed or error.
Optionally, a reason can also be passed.
My questions are:
How I can get 'session-id' after test execution ?
What if I can see "completed" status in dashboard already ?
A session on BrowserStack has only three types of statuses:
Completed, Error or Timeout. Selenium (and hence, BrowserStack) does not have a way of understanding, if a test has passed or failed. Its by the multiple assertions in your tests that appear on your console, that you infer if a test has passed / failed. These assertions however, do not reach BrowserStack. As you rightly identified, you can use the REST-API, to change the status of the session to 'Error', if you see a failure in your console.
I would suggest fetching the session ID of the test as the test is being executed, since fetching the session ID after the test execution is a lengthy process. In Nightwatch, you can fetch session ID as follows:
browser.session(function(session) {
console.log(session.sessionId);
});
Yes, you can certainly change the status of the session once it is completed. That's where the REST-API comes to help!
If you came here searching for a solution in Python, you could use
requests.put(
"https://api.browserstack.com/automate/sessions/{}.json".format(driver.session_id),
auth=(USERNAME, ACCESS_KEY),
json={"status": "failed", "reason": "test failed"})

Resources