InvokeRestAPI with Azure Devops - azure

I am continuously struggling to connect with the GCP from Azure Devops InvokeRestAPI task.
I have created a service connection with empty credentials. And created a API task in YAML file as below.
When I add the 'Authorization' in header, Devops fails to recognize it.
When I add the token w/wo Bearer in 'AuthToken', it fails with a 401 error, saying authentication error.
This is the wrror I face everytime, no matter what I do.
"message": "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.",
Here is the yaml code:
- job: planing_df1
pool: server
steps:
- task: InvokeRESTAPI#1
inputs:
connectionType: 'connectedServiceName'
serviceConnection: 'GCPServiceConnectionBasic'
method: 'GET'
headers: |
{
"PlanUrl": "$(system.CollectionUri)",
"ProjectId": "$(system.TeamProjectId)",
"HubName": "$(system.HostType)",
"PlanId": "$(system.PlanId)",
"JobId": "$(system.JobId)",
"TimelineId": "$(system.TimelineId)",
"TaskInstanceId": "$(system.TaskInstanceId)",
"AuthToken": "ya29.a0AeTM1ie8PKbCNb3nnTJ9XFnoVlBUlgiM48XAENJIFAl-dp4gHblablabla"
}
urlSuffix: '/myproj/locations/europe-west4/repositories/Dataform'
waitForCompletion: 'true'

When you run the API in Invoke Rest API task, you need to make sure that the same token can work fine on your local environment.
Then you can use the following format to set the headers of the Invoke Rest API Task.
"Authorization": "Bearer bearertokendeatil "
For example:
- job: planing_df1
pool: server
steps:
- task: InvokeRESTAPI#1
inputs:
connectionType: 'connectedServiceName'
serviceConnection: 'GCPServiceConnectionBasic'
method: 'GET'
headers: |
{
"PlanUrl": "$(system.CollectionUri)",
"ProjectId": "$(system.TeamProjectId)",
"HubName": "$(system.HostType)",
"PlanId": "$(system.PlanId)",
"JobId": "$(system.JobId)",
"TimelineId": "$(system.TimelineId)",
"TaskInstanceId": "$(system.TaskInstanceId)",
"Authorization": "Bearer bearertokendeatil"
}
urlSuffix: '/myproj/locations/europe-west4/repositories/Dataform'
waitForCompletion: 'true'

Related

Gitlab CI/CD stage test fails when fetching external resource with error: dh key too small

I'm trying to work with Gitlab CI/CD but the test stage fails with the following error:
write EPROTO 140044051654592:error:141A318A:SSL routines:tls_process_ske_dhe:dh key too small:../deps/openssl/openssl/ssl/statem/statem_clnt.c:2171:
gitlab-ci.yml
image: node:16.15.1
stages:
- test
test-job:
stage: test
script:
- npm run test
To be note that this is an integration test that calls an external resource with axios, and I have tried to set rejectUnauthorized: false and minVersion: "TLSv1" as suggested here and here
const axiosOptions = {
httpsAgent: new https.Agent({
rejectUnauthorized: false,
minVersion: "TLSv1",
})
};
const axiosInstance = axios.create(axiosOptions);
const response = await axiosInstance.get('https://www.some-domain.com/some-article.html');
This is not a problem with the test itself as it runs fine on my PC, but I suppose with the TLS of the gitlab runner.
Thanks

Github Action running "terraform plan" failing with a 404

I am running the following Github action that should post the output from terraform planto the PR in Github but it's giving me a 404. I have pretty much copied the action from Terraform
The only thing I can see that is suspect is the URL that's generating the 404 looks like it's missing a value - it says
"https://api.github.com/repos/orgname/reponame/issues//comments" but I think it probably should be
"https://api.github.com/repos/orgname/reponame/issues/SOMETHING/comments":
- name: Run Terraform Plan
id: plan
run: |
cd operations/app/terraform/vars/staging
terraform plan -no-color -input=false
continue-on-error: true
- uses: actions/github-script#v6
env:
PLAN: "terraform\n${{ steps.plan.outputs.stdout }}"
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const output = `#### Terraform Format and Style 🖌 \`${{ steps.fmt.outcome }}\`
#### Terraform Initialization ⚙️\`${{ steps.init.outcome }}\`
#### Terraform Validation 🤖\`${{ steps.validate.outcome }}\`
#### Terraform Plan 📖\`${{ steps.plan.outcome }}\`
<details><summary>Show Plan</summary>
\`\`\`\n
${process.env.PLAN}
\`\`\`
</details>
*Pusher: #${{ github.actor }}, Action: \`${{ github.event_name }}\`*`;
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: output
})
I get the following:
Plan: 0 to add, 1 to change, 0 to destroy.
─────────────────────────────────────────────────────────────────────────────
Note: You didn't use the -out option to save this plan, so Terraform can't
guarantee to take exactly these actions if you run "terraform apply" now.
RequestError [HttpError]: Not Found
at /home/runner/work/_actions/actions/github-script/v6/dist/index.js:4469:21
at processTicksAndRejections (node:internal/process/task_queues:96:5) ***
status: 404,
response: ***
url: 'https://api.github.com/repos/myorg/myrepo/issues//comments',
Error: Unhandled error: HttpError: Not Found
status: 404,
headers: ***
<snip headers>
data: ***
message: 'Not Found',
documentation_url: 'https://docs.github.com/rest'
***
***,
request: ***
method: 'POST',
url: 'https://api.github.com/repos/orgname/reponame/issues//comments',
headers: ***
accept: 'application/vnd.github.-preview+json',
'user-agent': 'actions/github-script octokit-core.js/3.5.1 Node.js/16.13.0 (linux; x64)',
authorization: 'token [REDACTED]',
'content-type': 'application/json; charset=utf-8'
***,
Try putting await in front of the function call in your action.
await github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: output
})
The issue was this was running on both push and pull_request - but on a push there is no PR for it to add the comment to so that portion failed. Adding a conditional for the github-script got it working as expected
- uses: actions/github-script#v6
if: github.event_name == 'pull_request'
env:

Moodle Web Service responses with invalid_parameter_exception

I created moodle and mariadb containers with Docker.
Moodle: 3.11.4
Mariadb: 10.3
I am trying following webservice to execute:
client:
wwwroot: 'http://localhost:8012',
service: 'moodle_mobile_app',
token: '8faf4879d2c654f11e404095032ae382',
strictSSL: true
call:
curl "http://localhost:8012/webservice/rest/server.php?wstoken=8faf4879d2c654f11e404095032ae382&moodlewsrestformat=json&wsfunction=core_user_get_users_by_field&moodlewsrestformat=json&id=2"
but getting follwing error:
{"exception":"invalid_parameter_exception","errorcode":"invalidparameter",
"message":"Invalid parameter value detected (Missing required key in single structure:field)",
"debuginfo":"Missing required key in single structure: field"
}
I tried it same with moodle client for node
... client.call({ wsfunction: "core_user_get_users_by_field", method: "POST", args: { id: 2 } })...
but also receiving same error.
I checked API documentation and id is valid parameter for this
webservice.
Can you please help?
Issue is resolved
client.call({
method: "POST",
wsfunction: "core_user_get_users_by_field",
args: {
field: "id",
values: ["2"]
}
}).then(function(info) {
var str = JSON.stringify(info, null, 4);
console.log(str);
});

Convert Azure Pipeline condition/variable to a json boolean

Currently, I'm working on a pipeline that should call an Azure Function in a certain way, depending on the outcome/result of a previous job in that pipeline.
The Azure Function should be called when the result of previous job is either: Succeeded, SucceededWithIssues or Failed. We want to ignore Skipped and Cancelled.
The body sent to the Azure Function differs based on the result: Succeeded/SucceededWithIssues VS Failed. It only differs by a single boolean in the payload called: DeploymentFailed.
The current implementation is using two separate tasks for calling the Azure Function. This was necessary, since I couldn't find a way to convert the outcome of the previous job to a boolean.
The current pipeline as is:
trigger:
- master
parameters:
- name: jobA
default: 'A'
- name: correlationId
default: '90c7e477-2141-45db-812a-019a9f88bdc8'
pool:
vmImage: ubuntu-latest
jobs:
- job: job_${{parameters.jobA}}
steps:
- script: echo "This job could potentially fail."
- job: job_B
dependsOn: job_${{parameters.jobA}}
variables:
failed: $[dependencies.job_${{parameters.jobA}}.result]
condition: in(variables['failed'], 'Succeeded', 'SucceededWithIssues', 'Failed')
pool: server
steps:
- task: AzureFunction#1
displayName: Call function succeeded
condition: in(variables['failed'], 'Succeeded', 'SucceededWithIssues')
inputs:
function: "<azure-function-url>"
key: "<azure-function-key>"
method: 'POST'
waitForCompletion: false
headers: |
{
"Content-Type": "application/json"
}
body: |
{
"CorrelationId": "${{parameters.correlationId}}",
"DeploymentFailed": false # I would like to use the outcome of `variable.failed` here and cast it to a JSON bool.
}
- task: AzureFunction#1
displayName: Call function failed
condition: in(variables['failed'], 'Failed')
inputs:
function: "<azure-function-url>"
key: "<azure-function-key>"
waitForCompletion: false
method: 'POST'
headers: |
{
"Content-Type": "application/json"
}
body: |
{
"CorrelationId": "${{parameters.correlationId}}",
"DeploymentFailed": true # I would like to use the outcome of `variable.failed` here and cast it to a JSON bool.
}
My question: How can I use the outcome of the previous job to only have 1 Azure Function invoke task?
You can map condition directly to variable:
variables:
failed: $[dependencies.job_${{parameters.jobA}}.result]
result: $[lower(notIn(dependencies.job_${{parameters.jobA}}.result, 'Succeeded', 'SucceededWithIssues', 'Failed'))]
and then:
- task: AzureFunction#1
displayName: Call function succeeded
condition: in(variables['failed'], 'Succeeded', 'SucceededWithIssues')
inputs:
function: "<azure-function-url>"
key: "<azure-function-key>"
method: 'POST'
waitForCompletion: false
headers: |
{
"Content-Type": "application/json"
}
body: |
{
"CorrelationId": "${{parameters.correlationId}}",
"DeploymentFailed": $(result) # I would like to use the outcome of `variable.failed` here and cast it to a JSON bool.
}

API gateway returns 401 and doesn't invoke custom authorizer

I've implemented a custom 'REQUEST' type authorizer for an API gateway which validates a JWT token passed in the 'Authorization' header. I've tested the lambda independently and it works as expected. I've also attached the authorizer to my routes and I can test it in the AWS console - again, everything seems to work (see image):
successful invoke via console
However, when I try to invoke my endpoints with the token in the Authorization header, I always receive an UNAUTHORIZED response:
{
"errors": [
{
"category": "ClientError",
"code": "UNAUTHORIZED",
"detail": "Unauthorized",
"method": "GET",
"path": "/cases",
"requestId": "004eb254-a926-45ad-96a5-ce3527621c81",
"retryable": false
}
]
}
From what I have gathered, API gateway never invokes my Authorizer as i don't see any log events in its cloudwatch. I was able to enable cloudwatch logging of my API gateway, and the only log information I see is as follows:
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| timestamp | message |
|---------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1578275720543 | (dac0d4f6-1380-4049-bcee-bf776ca78e5c) Extended Request Id: F2v9WFfiIAMF-9w= |
| 1578275720543 | (dac0d4f6-1380-4049-bcee-bf776ca78e5c) Unauthorized request: dac0d4f6-1380-4049-bcee-bf776ca78e5c |
| 1578275720543 | (dac0d4f6-1380-4049-bcee-bf776ca78e5c) Extended Request Id: F2v9WFfiIAMF-9w= |
| 1578275720544 | (dac0d4f6-1380-4049-bcee-bf776ca78e5c) Gateway response type: UNAUTHORIZED with status code: 401 |
| 1578275720544 | (dac0d4f6-1380-4049-bcee-bf776ca78e5c) Gateway response body: {"errors": [{"category": "ClientError","code": "UNAUTHORIZED","detail": "Unauthorized","method": "GET","path": "/cases","requestId": "dac0d4f6-1380-4049-bcee-bf776ca78e5c","retryable": false }]} |
| 1578275720544 | (dac0d4f6-1380-4049-bcee-bf776ca78e5c) Gateway response headers: {} |
| 1578275720544 | (dac0d4f6-1380-4049-bcee-bf776ca78e5c) Gateway response type: UNAUTHORIZED with status code: 401 |
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
At this point I am completely stuck and not sure how to debug this further. I'm assuming something must be configured wrong but the log information I can find doesn't given any indication of what the problem is. I've also pasted a copy of my authorizers configuration in the image below:
Authorizer Configuration
Screenshot of one endpoint configured to use the authorizer
I figured out the problem I was having:
I needed to set identitySource: method.request.header.Authorization in the authorizer field of the endpoint as well as in the CF stack.
Custom Authorizer definition in raw cloudformation:
service:
name: api-base
frameworkVersion: ">=1.2.0 <2.0.0"
plugins:
- serverless-plugin-optimize
- serverless-offline
- serverless-pseudo-parameters
- serverless-domain-manager
custom:
stage: ${self:provider.stage, 'dev'}
serverless-offline:
port: ${env:OFFLINE_PORT, '4000'}
false: false
cognitoStack: marley-auth
customDomain:
domainName: ${env:BE_HOST, ''}
enabled: ${env:EN_CUSTOM_DOMAIN, self:custom.false}
stage: ${self:provider.stage, 'dev'}
createRoute53Record: true
provider:
name: aws
runtime: nodejs10.x
versionFunctions: true
apiName: public
logs:
restApi: true
stackTags:
COMMIT_SHA: ${env:COMMIT_SHA, 'NO-SHA'}
environment:
USER_POOL_ID: ${cf:${self:custom.cognitoStack}-${self:custom.stage}.UserPoolId}
CLIENT_ID: ${cf:${self:custom.cognitoStack}-${self:custom.stage}.UserPoolClientId}
timeout: 30
iamRoleStatements:
- Effect: "Allow"
Action:
- "lambda:InvokeFunction"
Resource: "*"
functions:
authorizer:
handler: handler/authorize.handler
resources:
- Outputs:
ApiGatewayRestApiId:
Value:
Ref: ApiGatewayRestApi
Export:
Name: ${self:custom.stage}-${self:provider.apiName}-ApiGatewayRestApiId
ApiGatewayRestApiRootResourceId:
Value:
Fn::GetAtt:
- ApiGatewayRestApi
- RootResourceId
Export:
Name: ${self:custom.stage}-${self:provider.apiName}-ApiGatewayRestApiRootResourceId
SharedAuthorizerId:
Value:
Ref: SharedAuthorizer
Export:
Name: ${self:custom.stage}-${self:provider.apiName}-ApiGatewaySharedAuthorizerId
- Resources:
SharedAuthorizer:
Type: AWS::ApiGateway::Authorizer
Properties:
Name: public
AuthorizerUri: !Join
- ''
- - 'arn:aws:apigateway:'
- !Ref 'AWS::Region'
- ':lambda:path/2015-03-31/functions/'
- !GetAtt
- AuthorizerLambdaFunction
- Arn
- /invocations
RestApiId: !Ref 'ApiGatewayRestApi'
Type: REQUEST
IdentitySource: method.request.header.Authorization
AuthorizerResultTtlInSeconds: '300'
DependsOn: AuthorizerLambdaFunction
ApiAuthLambdaPermission:
Type: AWS::Lambda::Permission
Properties:
Action: lambda:InvokeFunction
FunctionName: !Ref AuthorizerLambdaFunction
Principal: apigateway.amazonaws.com
SourceArn: !Sub "arn:aws:execute-api:#{AWS::Region}:#{AWS::AccountId}:#{ApiGatewayRestApi}/authorizers/*"
DependsOn: ApiGatewayRestApi
Using the authorizer in another stack - note that I have specified IdentitySource here as well as in the definition of the authorizer - for some reason I had to do it in both places.
authorizer:
type: CUSTOM
authorizerId: ${cf:api-base-${self:custom.stage}.SharedAuthorizerId}
identitySource: method.request.header.Authorization

Resources