Problem with authentication using WebDAV - davfs2 - linux
This is my first contact with WebDAV and I have slight problems connecting to end-point.
Credentials are working fine when using Cyberduck client.
I need to configure container with davfs2 on Alpine or Ubuntu.
I don't have access to WebDAV server (SaaS E-Commerce platform, https://shoper.pl)
Using davfs2 I have managed to mount Nextcloud WebDAV (demo account) with success and no issues at all.
Versions: davfs2/1.5.5 neon/0.30.2
Details are below. Any suggestions?
Error:
Could not authenticate to server: rejected Digest challenge, rejected Basic challenge
/var/log/syslog
Jan 25 12:33:28 ubuntu-dev mount.davfs: Identity match for 'devshop-******.shoparena.pl': good
Jan 25 12:33:28 ubuntu-dev mount.davfs: Request sent; retry is 0.
Jan 25 12:33:28 ubuntu-dev mount.davfs: [status-line] < HTTP/1.1 200 OK#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Date: Sat, 25 Jan 2020 12:33:28 GMT#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [date], Value: [Sat, 25 Jan 2020 12:33:28 GMT]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Content-Type: text/html; charset=UTF-8#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [content-type], Value: [text/html; charset=UTF-8]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Transfer-Encoding: chunked#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [transfer-encoding], Value: [chunked]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Connection: keep-alive#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [connection], Value: [keep-alive]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Keep-Alive: timeout=20#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [keep-alive], Value: [timeout=20]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Dav: 1#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [dav], Value: [1]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Ms-Author-Via: DAV#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [ms-author-via], Value: [DAV]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Accept-Ranges: bytes#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [accept-ranges], Value: [bytes]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Vary: Brief,Prefer#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [vary], Value: [Brief,Prefer]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Allows: OPTIONS, GET, HEAD, PUT, DELETE, MKCOL, PROPFIND, PROPPATCH, MOVE, COPY#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [allows], Value: [OPTIONS, GET, HEAD, PUT, DELETE, MKCOL, PROPFIND, PROPPATCH, MOVE, COPY]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] X-Cache-Namespace: 148#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [x-cache-namespace], Value: [148]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] X-XSS-Protection: 1#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [x-xss-protection], Value: [1]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] X-Powered-By: DCSaaS#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [x-powered-by], Value: [DCSaaS]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Server: DCSaaS/httpd#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [server], Value: [DCSaaS/httpd]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] X-Content-Type-Options: nosniff#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [x-content-type-options], Value: [nosniff]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Set-Cookie: fp=4220404054; Path=/; HttpOnly; Max-Age=315360000#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [set-cookie], Value: [fp=4220404054; Path=/; HttpOnly; Max-Age=315360000]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] #015
Jan 25 12:33:28 ubuntu-dev mount.davfs: End of headers.
Jan 25 12:33:28 ubuntu-dev mount.davfs: Running post_headers hooks
Jan 25 12:33:28 ubuntu-dev mount.davfs: [chunk] < 0#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Got chunk size: 0
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] #015
Jan 25 12:33:28 ubuntu-dev mount.davfs: End of headers.
Jan 25 12:33:28 ubuntu-dev mount.davfs: Running post_send hooks
Jan 25 12:33:28 ubuntu-dev mount.davfs: ah_post_send (#0), code is 200 (want 401), WWW-Authenticate is (none)
Jan 25 12:33:28 ubuntu-dev mount.davfs: Request ends, status 200 class 2xx, error line:#012200 OK
Jan 25 12:33:28 ubuntu-dev mount.davfs: Running destroy hooks.
Jan 25 12:33:28 ubuntu-dev mount.davfs: Request ends.
Jan 25 12:33:28 ubuntu-dev mount.davfs: ah_create, for WWW-Authenticate
Jan 25 12:33:28 ubuntu-dev mount.davfs: Running pre_send hooks
Jan 25 12:33:28 ubuntu-dev mount.davfs: Sending request headers:#012PROPFIND /webapi/webdav/ HTTP/1.1#015#012User-Agent: davfs2/1.5.5 neon/0.30.2#015#012Connection: TE#015#012TE: trailers#015#012Host: devshop-******.shoparena.pl#015#012Depth: 1#015#012Content-Length: 257#015#012Content-Type: application/xml#015#012#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Sending request-line and headers:
Jan 25 12:33:28 ubuntu-dev mount.davfs: Sending request body:
Jan 25 12:33:28 ubuntu-dev mount.davfs: Body block (257 bytes):#012[<?xml version="1.0" encoding="utf-8"?>#012<propfind xmlns="DAV:"><prop>#012<resourcetype xmlns="DAV:"/>#012<getcontentlength xmlns="DAV:"/>#012<getetag xmlns="DAV:"/>#012<getlastmodified xmlns="DAV:"/>#012<executable xmlns="http://apache.org/dav/props/"/>#012</prop></propfind>#012]
Jan 25 12:33:28 ubuntu-dev mount.davfs: Request sent; retry is 1.
Jan 25 12:33:28 ubuntu-dev mount.davfs: [status-line] < HTTP/1.1 401 Unauthorized#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Date: Sat, 25 Jan 2020 12:33:28 GMT#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [date], Value: [Sat, 25 Jan 2020 12:33:28 GMT]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Content-Type: text/html; charset=UTF-8#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [content-type], Value: [text/html; charset=UTF-8]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Transfer-Encoding: chunked#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [transfer-encoding], Value: [chunked]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Connection: keep-alive#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [connection], Value: [keep-alive]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Keep-Alive: timeout=20#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [keep-alive], Value: [timeout=20]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Www-Authenticate: Basic realm="WebDAV Server 6.6.6"#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [www-authenticate], Value: [Basic realm="WebDAV Server 6.6.6"]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Www-Authenticate: Digest realm="WebDAV Server 6.6.6", domain="devshop-******.shoparena.pl", nonce="c66bd42f37c902e4c2cd14d532d285a1", opaque="d75db7b160fe72d1346d2bd1f67bfd10", algorithm="MD5", qop="auth"#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [www-authenticate], Value: [Digest realm="WebDAV Server 6.6.6", domain="devshop-******.shoparena.pl", nonce="c66bd42f37c902e4c2cd14d532d285a1", opaque="d75db7b160fe72d1346d2bd1f67bfd10", algorithm="MD5", qop="auth"]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Dav: 1#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [dav], Value: [1]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] X-Cache-Namespace: 148#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [x-cache-namespace], Value: [148]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] X-XSS-Protection: 1#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [x-xss-protection], Value: [1]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] X-Powered-By: DCSaaS#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [x-powered-by], Value: [DCSaaS]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Server: DCSaaS/httpd#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [server], Value: [DCSaaS/httpd]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] X-Content-Type-Options: nosniff#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [x-content-type-options], Value: [nosniff]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Set-Cookie: fp=1065983822; Path=/; HttpOnly; Max-Age=315360000#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [set-cookie], Value: [fp=1065983822; Path=/; HttpOnly; Max-Age=315360000]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] #015
Jan 25 12:33:28 ubuntu-dev mount.davfs: End of headers.
Jan 25 12:33:28 ubuntu-dev mount.davfs: Running post_headers hooks
Jan 25 12:33:28 ubuntu-dev mount.davfs: [chunk] < 0#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Got chunk size: 0
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] #015
Jan 25 12:33:28 ubuntu-dev mount.davfs: End of headers.
Jan 25 12:33:28 ubuntu-dev mount.davfs: Running post_send hooks
Jan 25 12:33:28 ubuntu-dev mount.davfs: ah_post_send (#0), code is 401 (want 401), WWW-Authenticate is Basic realm="WebDAV Server 6.6.6", Digest realm="WebDAV Server 6.6.6", domain="devshop-******.shoparena.pl", nonce="c66bd42f37c902e4c2cd14d532d285a1", opaque="d75db7b160fe72d1346d2bd1f67bfd10", algorithm="MD5", qop="auth"
Jan 25 12:33:28 ubuntu-dev mount.davfs: auth: Got challenge (code 401).
Jan 25 12:33:28 ubuntu-dev mount.davfs: auth: Got 'Basic' challenge.
Jan 25 12:33:28 ubuntu-dev mount.davfs: auth: Got 'Digest' challenge.
Jan 25 12:33:28 ubuntu-dev mount.davfs: auth: Trying Digest challenge...
Jan 25 12:33:28 ubuntu-dev mount.davfs: auth: Using domain /devshop-******.shoparena.pl from devshop-******.shoparena.pl
Jan 25 12:33:28 ubuntu-dev mount.davfs: auth: Got qop, using 2617-style.
Jan 25 12:33:28 ubuntu-dev mount.davfs: auth: H(A1) is [f4191e958585d980c418e87b51ebccfe]
Jan 25 12:33:28 ubuntu-dev mount.davfs: auth: Accepting digest challenge.
Jan 25 12:33:28 ubuntu-dev mount.davfs: auth: Accepted Digest challenge.
Jan 25 12:33:28 ubuntu-dev mount.davfs: Running pre_send hooks
Jan 25 12:33:28 ubuntu-dev mount.davfs: auth: Sending 'Digest' response.
Jan 25 12:33:28 ubuntu-dev mount.davfs: auth: '/webapi/webdav/' is inside auth domain: 0.
Jan 25 12:33:28 ubuntu-dev mount.davfs: Sending request headers:#012PROPFIND /webapi/webdav/ HTTP/1.1#015#012User-Agent: davfs2/1.5.5 neon/0.30.2#015#012Connection: TE#015#012TE: trailers#015#012Host: devshop-******.shoparena.pl#015#012Depth: 1#015#012Content-Length: 257#015#012Content-Type: application/xml#015#012#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Sending request-line and headers:
Jan 25 12:33:28 ubuntu-dev mount.davfs: Sending request body:
Jan 25 12:33:28 ubuntu-dev mount.davfs: Body block (257 bytes):#012[<?xml version="1.0" encoding="utf-8"?>#012<propfind xmlns="DAV:"><prop>#012<resourcetype xmlns="DAV:"/>#012<getcontentlength xmlns="DAV:"/>#012<getetag xmlns="DAV:"/>#012<getlastmodified xmlns="DAV:"/>#012<executable xmlns="http://apache.org/dav/props/"/>#012</prop></propfind>#012]
Jan 25 12:33:28 ubuntu-dev mount.davfs: Request sent; retry is 1.
Jan 25 12:33:28 ubuntu-dev mount.davfs: [status-line] < HTTP/1.1 401 Unauthorized#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Date: Sat, 25 Jan 2020 12:33:29 GMT#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [date], Value: [Sat, 25 Jan 2020 12:33:29 GMT]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Content-Type: text/html; charset=UTF-8#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [content-type], Value: [text/html; charset=UTF-8]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Transfer-Encoding: chunked#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [transfer-encoding], Value: [chunked]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Connection: keep-alive#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [connection], Value: [keep-alive]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Keep-Alive: timeout=20#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [keep-alive], Value: [timeout=20]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Www-Authenticate: Basic realm="WebDAV Server 6.6.6"#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [www-authenticate], Value: [Basic realm="WebDAV Server 6.6.6"]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Www-Authenticate: Digest realm="WebDAV Server 6.6.6", domain="devshop-******.shoparena.pl", nonce="c66bd42f37c902e4c2cd14d532d285a1", opaque="d75db7b160fe72d1346d2bd1f67bfd10", algorithm="MD5", qop="auth"#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [www-authenticate], Value: [Digest realm="WebDAV Server 6.6.6", domain="devshop-******.shoparena.pl", nonce="c66bd42f37c902e4c2cd14d532d285a1", opaque="d75db7b160fe72d1346d2bd1f67bfd10", algorithm="MD5", qop="auth"]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Dav: 1#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [dav], Value: [1]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] X-Cache-Namespace: 148#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [x-cache-namespace], Value: [148]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] X-XSS-Protection: 1#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [x-xss-protection], Value: [1]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] X-Powered-By: DCSaaS#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [x-powered-by], Value: [DCSaaS]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Server: DCSaaS/httpd#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [server], Value: [DCSaaS/httpd]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] X-Content-Type-Options: nosniff#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [x-content-type-options], Value: [nosniff]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] Set-Cookie: fp=1381817215; Path=/; HttpOnly; Max-Age=315360000#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Header Name: [set-cookie], Value: [fp=1381817215; Path=/; HttpOnly; Max-Age=315360000]
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] #015
Jan 25 12:33:28 ubuntu-dev mount.davfs: End of headers.
Jan 25 12:33:28 ubuntu-dev mount.davfs: Running post_headers hooks
Jan 25 12:33:28 ubuntu-dev mount.davfs: [chunk] < 0#015
Jan 25 12:33:28 ubuntu-dev mount.davfs: Got chunk size: 0
Jan 25 12:33:28 ubuntu-dev mount.davfs: [hdr] #015
Jan 25 12:33:28 ubuntu-dev mount.davfs: End of headers.
davfs2.conf
dav_user root
dav_group root
kernel_fs fuse
buf_size 32
connect_timeout 30
read_timeout 30
retry 10
max_retry 300
dir_refresh 30
file_refresh 10
use_locks 0
follow_redirect 1
debug config
debug kernel
debug cache
debug http
debug xml
debug httpauth
debug locks
debug ssl
debug httpbody
debug most
There was a problem with how service owner was handling requests (he made his own modifications to WebDAV server).
Related
Finding the final URL that the request will be redirected to in nodeJS?
I have the following URL:
https://forecast.weather.gov/zipcity.php?inputstring=95014
I would like to figure out which URL it will redirect to. In this example it is:
https://forecast.weather.gov/MapClick.php?CityName=Cupertino&state=CA&site=MTR&lat=37.3042&lon=-122.095
I tried multiple solutions such as:
res.headers.location
and
res.headers.get('location')
But none of them seem to work. I know that the URL redirects because, I could successfully redirect in curl and google-chrome. Here is the code that I am running:
https.get('https://forecast.weather.gov/zipcity.php?inputstring=95014', res => console.log(res.headers.location))
When I was running in curl, I ran the following:
curl -Ls -o /dev/null -w %{url_effective} http://forecast.weather.gov/zipcity.php?inputstring=95014
And got the desired output of:
http://forecast.weather.gov/MapClick.php?CityName=Cupertino&state=CA&site=MTR&lat=37.3042&lon=-122.095
When I run curl:
$ curl -I https://forecast.weather.gov/zipcity.php?inputstring=95014
HTTP/2 302
server: Apache
x-nids-serverid: www1.mo
location: https://forecast.weather.gov/MapClick.php?CityName=Cupertino&state=CA&site=MTR&lat=37.3042&lon=-122.095
x-ua-compatible: IE=Edge
access-control-allow-origin: *
content-type: text/html; charset=UTF-8
content-length: 0
cache-control: max-age=722
expires: Mon, 26 Apr 2021 19:28:03 GMT
date: Mon, 26 Apr 2021 19:16:01 GMT
strict-transport-security: max-age=31536000 ; includeSubDomains ; preload
As you can see, there are multiple headers present here that are not present inside nodejs's results:
{
server: 'AkamaiGHost',
'mime-version': '1.0',
'content-type': 'text/html',
'content-length': '288',
expires: 'Mon, 26 Apr 2021 19:21:50 GMT',
date: 'Mon, 26 Apr 2021 19:21:50 GMT',
connection: 'close',
'strict-transport-security': 'max-age=31536000 ; includeSubDomains ; preload'
}
I would like to stick to the http module.
what the correct template mapping should be in the Integration Request aws api-getway?
I am working on an event stream system. And my solution is: api-gatway->firehose data stream->data stream delivery->s3.
1.Send a json request(action is PUT) to api-getway
2.Integrate the request with Kinesis service
3.Valid data in the template mapping of the Integration Request
4.Delivery the data to data stream by firehose data delivery
5.Consume the stream and save into S3
It works with below data
1.Request data
{
"Data" : "this is a test"
}
2.Template mapping code of the Integration Request with firehose
{
"StreamName" : "Stream",
"Data" : "$util.base64Encode($input.json('$.Data'))",
"PartitionKey" : "1"
}
Execution log for request 4ad5f3a1-da7d-436f-bd3f-879ba045c622
Fri Jan 15 15:14:19 UTC 2021 : Starting execution for request: 4ad5f3a1-da7d-436f-bd3f-879ba045c622
Fri Jan 15 15:14:19 UTC 2021 : HTTP Method: PUT, Resource Path: /record
Fri Jan 15 15:14:19 UTC 2021 : Method request path: {}
Fri Jan 15 15:14:19 UTC 2021 : Method request query string: {}
Fri Jan 15 15:14:19 UTC 2021 : Method request headers: {}
Fri Jan 15 15:14:19 UTC 2021 : Method request body before transformations: {
"property":"1",
"name":"name"
}
Fri Jan 15 15:14:19 UTC 2021 : Endpoint request URI: https://kinesis.us-east-1.amazonaws.com/?Action=PutRecord
Fri Jan 15 15:14:19 UTC 2021 : Endpoint request headers: {Authorization=***************************************************************************************************************************************************************************************************************************************************************************************9bb292, X-Amz-Date=20210115T151419Z, x-amzn-apigateway-api-id=450acvde3l, Accept=application/json, User-Agent=AmazonAPIGateway_450acvde3l, X-Amz-Security-Token=IQoJb3JpZ2luX2VjEEAaCXVzLWVhc3QtMSJHMEUCIQC7W693nrtuUGrgUvOpn+PUKTRYTGAzRYVSWUPKoAc4eAIgGp/uH4yV/vynaMQUBNEBTwP79m0P9nVfCmw9CMWjY0Mq5QIIGBACGgw1MzUxMTE0MzMzNDAiDKy2z/4lnGJJJAWaOyrCAnANknvMUIbvf9diTZ4qYsdJ3LjETy9hE0f9pUZTBh7aKRjn2y4hbZc/oXGpsmHcgsgJAlIGD51gD0O5LR+N2EKjQeeo5GijkYvZvdbmxVRAZR27EhFzujCk3okePxAzVb3SeCQDWrBnEWn2VwqX4Y75QWwOgL8ZfcuxNKF/wbks0ukPFBo0p68KQ24PIXOYGMs9IAxHYa195+5mUMSi5Av62QKPpBbUAgZ2IJt9EKvRKyLYCs5XzS4lPCgsIbwouVB38f6UQKdQ9XsmD0cX04ODkdj2/0GHne6ufar7nANd3o08PmALd9mXK4z4gt/OxkZUe2AVLXvNQ4IodzW0WOUe1nFvq6YxpveDVfXKDRSK [TRUNCATED]
Fri Jan 15 15:14:19 UTC 2021 : Endpoint request body after transformations: {
"StreamName" : "EventStream",
"Data" : "IiI=",
"PartitionKey" : "1"
}
Fri Jan 15 15:14:19 UTC 2021 : Sending request to https://kinesis.us-east-1.amazonaws.com/?Action=PutRecord
Fri Jan 15 15:14:19 UTC 2021 : Received response. Status: 200, Integration latency: 21 ms
Fri Jan 15 15:14:19 UTC 2021 : Endpoint response headers: {x-amzn-RequestId=c028bf47-0059-8b93-9853-0cccfda9a977, x-amz-id-2=L15V+8AS8gN3HxJwiz0qVqi/UJn2xSWueRMnXdhqgjFw6TeuaRlYX62DZK9pa+O1PcKopTP55aHRLdhX0cQwovKefVpiuRtv, Date=Fri, 15 Jan 2021 15:14:19 GMT, Content-Type=application/x-amz-json-1.1, Content-Length=110}
Fri Jan 15 15:14:19 UTC 2021 : Endpoint response body before transformations: {"SequenceNumber":"49614572211779959518343530489315214421429290892684951554","ShardId":"shardId-000000000000"}
Fri Jan 15 15:14:19 UTC 2021 : Method response body after transformations: {"SequenceNumber":"49614572211779959518343530489315214421429290892684951554","ShardId":"shardId-000000000000"}
Fri Jan 15 15:14:19 UTC 2021 : Method response headers: {X-Amzn-Trace-Id=Root=1-6001b14b-bbccae126175224317a10a4e, Content-Type=application/json}
Fri Jan 15 15:14:19 UTC 2021 : Successfully completed execution
Fri Jan 15 15:14:19 UTC 2021 : Method completed with status: 200
But does not work with below:
1.Request data
{
"Data" : {
"property":"1",
"name":"name"
}
}
2.Template mapping code of the Integration Request with firehose
{
"StreamName" : "Stream",
"Data" : {
"property" : $input.json('$.property'),
"name" : $input.json('$.name')
},
"PartitionKey" : "1"
}
Execution log for request 05b707cc-d95c-40bf-8b75-375048697414
Fri Jan 15 15:39:34 UTC 2021 : Starting execution for request: 05b707cc-d95c-40bf-8b75-375048697414
Fri Jan 15 15:39:34 UTC 2021 : HTTP Method: PUT, Resource Path: /record
Fri Jan 15 15:39:34 UTC 2021 : Method request path: {}
Fri Jan 15 15:39:34 UTC 2021 : Method request query string: {}
Fri Jan 15 15:39:34 UTC 2021 : Method request headers: {}
Fri Jan 15 15:39:34 UTC 2021 : Method request body before transformations: {
"property":"1",
"name":"name"
}
Fri Jan 15 15:39:34 UTC 2021 : Endpoint request URI: https://kinesis.us-east-1.amazonaws.com/?Action=PutRecord
Fri Jan 15 15:39:34 UTC 2021 : Endpoint request headers: {Authorization=***************************************************************************************************************************************************************************************************************************************************************************************a43c70, X-Amz-Date=20210115T153934Z, x-amzn-apigateway-api-id=450acvde3l, Accept=application/json, User-Agent=AmazonAPIGateway_450acvde3l, X-Amz-Security-Token=IQoJb3JpZ2luX2VjEEAaCXVzLWVhc3QtMSJFMEMCHw0tTLi/gwtxUjwAQSl/CIY8aie2nmayl+Qsm6/i520CID9iXCFafaQTh4YqE1/tzvKgMO5IlYgFJrcNbAQB2Nc+KuUCCBkQAhoMNTM1MTExNDMzMzQwIgxeE22ch5hdFy3nB+UqwgJyKnQpnLuEY3zpcbRdEO5jks7yfx2+o1xfIz9Kga0S1PojPfzxh5aD/PthhP8D0jutv96ZVe8p52TwfSnv/z3YeDCFzsnw/U9kGFzGVt1pY2JMB4sg1vU7li8pFP/qiUQ3QA8cXbp4nWeE3kQGlPG4pjH0MsOvowTxM8G6yKosvCdD8fVyCJxWIjFnn1+dK9GGV/MnZlnaVqc57z0n0nrHgLjxBzDcDKJ5/xrgcqcYmUETFj8NyDJ9ESzCp0PhKJV9tGF4LgxbAgffe2Yw/3qpQyB6JNqJrZEczADp3gL0rjIBXhbnx5Yizs9MBMtoB9L22mAwEeqJRx4lK12wOqQZ5+0homLCugauYVoy3juNv/zW [TRUNCATED]
Fri Jan 15 15:39:34 UTC 2021 : Endpoint request body after transformations: {
"StreamName" : "EventStream",
"Data" : {
"property" : "1",
"name" : "name"
},
"PartitionKey" : "1"
}
Fri Jan 15 15:39:34 UTC 2021 : Sending request to https://kinesis.us-east-1.amazonaws.com/?Action=PutRecord
Fri Jan 15 15:39:34 UTC 2021 : Received response. Status: 400, Integration latency: 2 ms
Fri Jan 15 15:39:34 UTC 2021 : Endpoint response headers: {x-amzn-RequestId=cc937c57-744b-fca2-94e8-c5214b4386dd, x-amz-id-2=fqg8MLIbHOfeeFXp6wpii3l4yl32mI/5RyTwYQyzw9/OqpdLNqCBvBbTp8x7Q4YWAroefbfHb5IUEYeD68SQqh2bq87nL4vp, connection=close, Date=Fri, 15 Jan 2021 15:39:34 GMT, Content-Type=application/x-amz-json-1.1, Content-Length=99}
Fri Jan 15 15:39:34 UTC 2021 : Endpoint response body before transformations: {"__type":"SerializationException","Message":"Start of structure or map found where not expected."}
Fri Jan 15 15:39:34 UTC 2021 : Method response body after transformations: {"__type":"SerializationException","Message":"Start of structure or map found where not expected."}
Fri Jan 15 15:39:34 UTC 2021 : Method response headers: {X-Amzn-Trace-Id=Root=1-6001b736-328415f2db20383946bffd9e, Content-Type=application/json}
Fri Jan 15 15:39:34 UTC 2021 : Successfully completed execution
Fri Jan 15 15:39:34 UTC 2021 : Method completed with status: 200
reference: https://www.youtube.com/watch?v=0UxiV5sUlcA
My bad and I forgot to encode in template mapping.
The code should be:
{
"StreamName" : "Stream",
"Data" : "$util.base64Encode(
{
"property" : $input.json('$.property'),
"name" : $input.json('$.name')
}
),
"PartitionKey" : "1"
}
Loopback application middleware not working - No matching layer found errors
I have integrated my loopback application with appDynamics library and all the middlewares are not firing from then on. Used the debug command - DEBUG=* npm start to get the logs and found these logs coming up. All the middlewares declared through middleware.json and imperatively through app.middleware() command are having this error message - No matching layer found Any idea what's going wrong here - Which loopback package is responsible for express routing implementation 2020-07-10T15:23:11.241Z - : pid:36871 worker:1 Fri, 10 Jul 2020 15:23:11 GMT loopback:boot:executor Configuring middleware "/usr/nodejs/njsDT_SIT/apps/wlpn/ob-cof-funds-confirmation-consents-channel/package/node_modules/compression" 2020-07-10T15:23:11.243Z - : pid:36871 worker:1 Fri, 10 Jul 2020 15:23:11 GMT loopback:app use initial compression 2020-07-10T15:23:11.244Z - : pid:36871 worker:1 Fri, 10 Jul 2020 15:23:11 GMT express:router use [] compression 2020-07-10T15:23:11.245Z - : pid:36871 worker:1 Fri, 10 Jul 2020 15:23:11 GMT express:router:layer new [] 2020-07-10T15:23:11.245Z - : pid:36871 worker:1 Fri, 10 Jul 2020 15:23:11 GMT loopback:app No matching layer is found for initial compression 2020-07-10T15:23:11.245Z - : pid:36871 worker:1 Fri, 10 Jul 2020 15:23:11 GMT loopback:boot:executor Configuring middleware "/usr/nodejs/njsDT_SIT/apps/wlpn/ob-cof-funds-confirmation-consents-channel/package/node_modules/loopback-context/server/middleware/per-request.js" 2020-07-10T15:23:11.246Z - : pid:36871 worker:1 Fri, 10 Jul 2020 15:23:11 GMT loopback:app use initial perRequestContext 2020-07-10T15:23:11.253Z - : pid:36871 worker:1 Fri, 10 Jul 2020 15:23:11 GMT express:router use [] perRequestContext 2020-07-10T15:23:11.254Z - : pid:36871 worker:1 Fri, 10 Jul 2020 15:23:11 GMT express:router:layer new [] 2020-07-10T15:23:11.254Z - : pid:36871 worker:1 Fri, 10 Jul 2020 15:23:11 GMT loopback:app No matching layer is found for initial perRequestContext 2020-07-10T15:23:11.254Z - : pid:36871 worker:1 Fri, 10 Jul 2020 15:23:11 GMT loopback:boot:executor Configuring middleware "/usr/nodejs/njsDT_SIT/apps/wlpn/ob-cof-funds-confirmation-consents-channel/package/node_modules/cors" 2020-07-10T15:23:11.254Z - : pid:36871 worker:1 Fri, 10 Jul 2020 15:23:11 GMT loopback:app use initial corsMiddleware 2020-07-10T15:23:11.254Z - : pid:36871 worker:1 Fri, 10 Jul 2020 15:23:11 GMT express:router use [] corsMiddleware 2020-07-10T15:23:11.254Z - : pid:36871 worker:1 Fri, 10 Jul 2020 15:23:11 GMT express:router:layer new [] 2020-07-10T15:23:11.254Z - : pid:36871 worker:1 Fri, 10 Jul 2020 15:23:11 GMT loopback:app No matching layer is found for initial corsMiddleware```
How to limit Crontab text output.
When cron runs it outputs text from the script that ran and the system calls. Is there a way to limit output? From This: From x.site Tue Jul 31 13:32:45 2018 Return-Path: <a.site> X-Original-To: a Delivered-To: a.site Received: by x.site (Postfix, from userid 1000) id 1458910057D; Tue, 31 Jul 2018 13:32:45 -0400 (EDT) From: "(Cron Daemon)" <a.site> To: a.site Subject: Cron <> Content-Type: text/plain; charset=UTF-8 Auto-Submitted: auto-generated Precedence: bulk X-Cron-Env: <XDG_SESSION_ID=3737> X-Cron-Env: <XDG_RUNTIME_DIR=/run/user/1000> X-Cron-Env: <DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus> X-Cron-Env: <LC_CTYPE=en_US.UTF-8> X-Cron-Env: <SHELL=/bin/sh> X-Cron-Env: <HOME=/home/ab10> X-Cron-Env: <PATH=/usr/bin:/bin> X-Cron-Env: <LOGNAME=a> X-Cron-Env: <USER=a> Message-Id: <20180731173245.1458910057D#BMCap3.site> Date: Tue, 31 Jul 2018 13:30:04 -0400 (EDT) Text from script.. To Something like this: From x.site Tue Jul 31 13:32:45 2018 Return-Path: <a.site> Subject: Cron <> Content-Type: text/plain; charset=UTF-8 Date: Tue, 31 Jul 2018 13:30:04 -0400 (EDT) Text from script..
Redirect the output to dev null as part of the command So the cron command you'd run would be blah > /dev/null To filter for some, you can use the pipe "blah | either awk or grep"
Here is a bash script...
#!/bin/bash
awk ' BEGIN {flag=1}
{
if ($0 ~ /Date:/) { flag=0;
}
if ($0 ~ /From/) { flag=1;
}
if (flag==0) {print}
}
'
Cannot map custom http header on response using int-http:outbound-gateway
I have an int-http:outbound-gateway to a rest service with a custom http header but this header is not mapped to the SI message headers although is set on the mapped-response-headers config
here's my config:
<int-http:outbound-gateway
request-channel="inChannel" reply-channel="outChannel"
url-expression="http://localhost:8080/rest" mapped-request-headers="HTTP_REQUEST_HEADERS"
http-method-expression="GET" expected-response-type="java.lang.String"
charset="UTF-8"
mapped-response-headers="HTTP_RESPONSE_HEADERS,mySpecificHeader"/>
Here's the raw response from the rest service:
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Expires: Fri, 28 Oct 2016 02:05:34 GMT
mySpecificHeader: headerInfo
Content-Type: application/json
Transfer-Encoding: chunked
Date: Fri, 28 Oct 2016 02:05:34 GMT
{"surname":"Constantine","phone":"+33 555 666 777","name":"John","id":12345}