Form input entry fails on chrome when using Remote Web Testing framworks (BrowserStack and LamdaTest) [duplicate] - remote-access

This question already has answers here:
Karate - UI Testing - When using Zalenium Safari and MSEDGE immediately error on driver/session call => no capabilities found [duplicate]
(2 answers)
Closed 1 year ago.
My test suite works fine locally as well as in gitlab pipeline using Chrome.
However, they fail right on the login page when running them with Chrome (only) on Remote Testing frameworks.
I tested BrowserStack and LamdaTest and they both fail when I attempt to enter text in a form input field.
Error message is:
{"sessionId":"d6c0edadf898c18d38e9dda073a868fa","status":61,"value":{"message":"invalid argument: 'value' must be a list\n (Session info: chrome=80.0.3987.87)\n (Driver info: chromedriver=80.0.3987.16 (320f6526c1632ad4f205ebce69b99a062ed78647-refs/branch-heads/3987#{#185}),platform=Windows NT 10.0.14393 x86_64)"}}
The same tests work on Safari and Firefox, only Chrome fails.
I spent some time investigating with the guys at LambdaTest and it seems it all works fine when using "regular" selenium selectors and actions, and only fails with Karate.
Has anyone experienced the same issue?
EDIT: adding non-karate webdriver payloads provided by Prateek Singh
Request POST http://hub.lambdatest.com /wd/hub/session/719c9157598420fb3e272f53be31ab51/elements
{ using: 'css selector', value: 'input[type=text]' }
Response 200 POST http://hub.lambdatest.com/wd/hub/session/719c9157598420fb3e272f53be31ab51/elements (461ms)
{
sessionId: '719c9157598420fb3e272f53be31ab51',
status: 0,
value: [ { ELEMENT: '0.37079523975334916-2' } ]
}
Request POST http://hub.lambdatest.com /wd/hub/session/719c9157598420fb3e272f53be31ab51/element/0.37079523975334916-2/value
{
value: [
'L', 'a', 'm', 'b',
'd', 'a', 'T', 'e',
's', 't', '\n'
]
}
Response 200 POST http://hub.lambdatest.com/wd/hub/session/719c9157598420fb3e272f53be31ab51/element/0.37079523975334916-2/value (1366ms)
{
sessionId: '719c9157598420fb3e272f53be31ab51',
status: 0,
value: null
}

Try with the latest selenium version, add the following capability:
caps.setCapability("browserstack.selenium_version", "3.141.59");

Your remote providers don't seem to be conforming to the W3C spec for chromedriver. For example the response to the /element call should be something like this, I know you have used elements but you can tell that the response "shape" is for the old non-compliant chrome-driver:
3 > POST http://localhost:9515/session/87ff90ad57ad432540b413740a66f7c1/element
3 > Content-Type: application/json; charset=UTF-8
3 > Content-Length: 43
3 > Host: localhost:9515
3 > Connection: Keep-Alive
3 > User-Agent: Apache-HttpClient/4.5.13 (Java/1.8.0_231)
3 > Accept-Encoding: gzip,deflate
{"using":"css selector","value":"#inputId"}
20:59:25.475 [main] DEBUG com.intuit.karate - response time in milliseconds: 9
3 < 200
3 < Content-Length: 88
3 < Content-Type: application/json; charset=utf-8
3 < cache-control: no-cache
{"value":{"element-6066-11e4-a52e-4f735466cecf":"d7bdd4a1-17bc-4ee0-817b-cf7052641d25"}}
And the request to element/value is certainly wrong, this is an example of a spec-compliant request, where the parameter text is expected in the payload (not an array under value):
4 > POST http://localhost:9515/session/87ff90ad57ad432540b413740a66f7c1/element/d7bdd4a1-17bc-4ee0-817b-cf7052641d25/value
4 > Content-Type: application/json; charset=UTF-8
4 > Content-Length: 22
4 > Host: localhost:9515
4 > Connection: Keep-Alive
4 > User-Agent: Apache-HttpClient/4.5.13 (Java/1.8.0_231)
4 > Accept-Encoding: gzip,deflate
{"text":"hello world"}
20:59:25.560 [main] DEBUG com.intuit.karate - response time in milliseconds: 83
4 < 200
4 < Content-Length: 14
4 < Content-Type: application/json; charset=utf-8
4 < cache-control: no-cache
{"value":null}
This must be reason why other browsers are working - perhaps because your LambdaTest setup is using an old version of the chrome-driver. Karate will not support the old version, but we will be happy to fix any gaps vs the W3C standard if applicable.
For reference, these are the capabilities returned by the chrome-driver I used:
{
"value":{
"capabilities":{
"networkConnectionEnabled":false,
**"chrome":{
"chromedriverVersion":"87.0.4280.88 (89e2380a3e36c3464b5dd1302349b1382549290d-refs\/branch-heads\/4280#{#1761})",
"userDataDir":"\/var\/folders\/py\/d2vyrn2n3bz7yzp7rppntw7cvsmbgv\/T\/.com.google.Chrome.JJHuOI"
},
"acceptInsecureCerts":false,
"browserVersion":"87.0.4280.141",
"browserName":"chrome",
"platformName":"mac os x",
"setWindowRect":true,
"webauthn:virtualAuthenticators":true
},
"sessionId":"87ff90ad57ad432540b413740a66f7c1"
}
}

Related

Etag support for S/4 EX

Etag is supported in SDK: https://sap.github.io/cloud-sdk/docs/java/features/odata/use-typed-odata-v4-client-in-sap-cloud-sdk-for-java/#handling-of-etags
So experimenting it by using BusinessPartner entity in S/4 EX.
But seems there's no If-Match header:
How come the header doesn't show up - any prerequisite with etag?
(entering on behalf of the implementation partner team)
I checked the VersionIdentifier of the response and it was not set to a value.
I also checked the response's JSON __metadeta and header, but there were no values that appeared to correspond to the ETag value.
[Code]
BusinessPartner bp1 = new DefaultBusinessPartnerService().getBusinessPartnerByKey(bpId).execute(dest);
log.debug("get 1: {}", bp1);
log.debug("get 1 VersionIdentifier: {}", bp1.getVersionIdentifier());
bp1.setOrganizationBPName1("SCP Update 1st:" + System.currentTimeMillis());
ODataUpdateResult result1 = new DefaultBusinessPartnerService().updateBusinessPartner(bp1).execute(dest);
log.debug("Update1 Http Status: {}", result1.getHttpStatusCode());
bp1.setOrganizationBPName1("SCP Update 2nd:" + System.currentTimeMillis());
bp1.setVersionIdentifier("dummy");
ODataUpdateResult result2 = new DefaultBusinessPartnerService().updateBusinessPartner(bp1).execute(dest);
log.debug("Update2 Http Status: {}", result2.getHttpStatusCode());
[Log]
get 1: BusinessPartner(super=VdmObject(customFields={}, changedOriginal...
get 1 VersionIdentifier: None
Update1 Http Status: 204
Update2 Http Status: 204
[GET Response JSON(__metadata) / Response Header]
(It has masked the IP address.)
"__metadata": {
"id": "https://xxx.xxx.xxx.xxx:xxxxxx/sap/opu/odata/sap/API_BUSINESS_PARTNER/A_BusinessPartner('1000001')",
"uri": "https://xxx.xxx.xxx.xxx:xxxxxx/sap/opu/odata/sap/API_BUSINESS_PARTNER/A_BusinessPartner('1000001')",
"type": "API_BUSINESS_PARTNER.A_BusinessPartnerType"
},
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
content-length: 3152
dataserviceversion: 2.0
sap-metadata-last-modified: Thu, 14 May 2020 23:58:07 GMT
cache-control: no-store, no-cache
sap-processing-info: ODataBEP=,crp=,RAL=,st=,MedCacheHub=SHM,codeployed=X,softstate=
sap-server: true
sap-perf-fesrec: 243070.000000
I tried setting the VersionIdentifier to a meaningless value in my test code (2nd update).
The update process seems to be successful, although the request header now has "If-Match" added to it.
(I was expecting the update to fail because the values never match, so I was hoping the update would fail.)
[2nd Update(setVersionIdenfifier)]
(It has masked some of the values.)
PATCH http://xxx.xxx.xxx.xxx:xxxxxx/sap/opu/odata/sap/API_BUSINESS_PARTNER/A_BusinessPartner(BusinessPartner='1000001') HTTP/1.1
x-csrf-token: xxx
Content-Type: application/json
Accept: application/json
If-Match: dummy
Authorization: Basic xxx
SAP-Connectivity-SCC-Location_ID: xxx
Proxy-Authorization: Bearer xxx
sap-language: en
sap-client: xxx
Content-Length: 55
If ETags are not part of OData service responses, then you should approach the IT/administrators who maintain the S/4 backend. The SAP Cloud SDK is only consuming the OData service. Unfortunately it can't leverage ETag support if it's disabled.

DocuSign_eSign::ApiError: Bad Request

I'm using the Docusign Ruby SDK to make calls on the API. I have my code working when pointed to the developer docusign, when pointed to prod I'm getting the DocuSign_eSign::ApiError: Bad Request error.
I have already gotten my integration key/client id approved in production and I've also already done the authorization grant part, where you allow the integration key to send envelopes on a user's behalf. This has been done in dev and production.
desc 'Perform API call to list envelopes'
task list: :initialize do
options = DocuSign_eSign::ListStatusChangesOptions.new
options.from_date = (Date.today - 10).strftime('%Y/%m/%d')
options.status = 'completed'
#list_results = #envelopes_api.list_status_changes #account_id, options
end
desc 'Build list results hash'
task build_list_hash: :list do
#list_results_hash = []
#list_results.envelopes.each do |list_hash|
#list_results_hash << { envelope_id: list_hash.envelope_id, status: 'pending', archive_at: Time.now + (86400 *30) }
end
puts "List results hash:\n #{#list_results_hash}"
File.open(ENV['MASTER_LIST_FILE'], 'w') { |file| file.write(#list_results_hash.to_yaml) }
end
I expect the output to be a list of envelopes which is what happens when running on demo.docusign
Update: Thanks for showing me how to get logs. It seems the error is with GetUserProfileImage
GET https://na3-app.docusign.net:8832/restapi/v2.1/accounts/91608b5d-d418-4c45-89f0-cd088504f99d/users/63757e15-8365-4fb9-9e7d-d9a8309f8e94/profile/image
TraceToken: 996d3b3f-2857-47d6-9d75-26a75b6b0a37
Timestamp: 2019-10-17T20:58:32.0932823Z
Content-Length: 0
Connection: keep-alive
Accept: application/json;text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Encoding: gzipdeflatebr
Accept-Language: en-USen; q=0.5
Authorization: Bearer [omitted]
Host: na3-app.docusign.net
Referer: https://app.docusign.com/preferences/security
User-Agent: Mozilla/5.0(Windows NT 10.0; Win64; x64; rv:69.0)Gecko/20100101Firefox/69.0
x-docusign-clienttransactionid: fa4e1202-09a4-431b-9abf-fe4d00acf565
x-csrf-token: c34f42c5aad288063afc8a6615be03c9
x-forwarded-for: 170.140.186.226, 162.248.185.11
x-docusign-authentication: {"IntegratorKey":"[omitted]"}
x-docusign-prettyprint: false
content-transfer-encoding: base64
x-forwarded-for-martini: 170.140.186.226
x-docusign-diagnostics: {"storedProcedureEventLogThreshold":"300"}
x-docusign-timetrack: CONN_START,2019-10-17T20:58:31.946Z;;REQ_SENT,2019-10-17T20:58:31.950Z;REST0_Start,2019-10-17T20:58:32.0620315Z
x-docusign-correlationtoken: fa4e1202-09a4-431b-9abf-fe4d00acf565
X-SecurityProtocol-Version: TLSv1.2
X-SecurityProtocol-CipherSuite: ECDHE-RSA-AES256-GCM-SHA384
404 NotFound
Content-Type: application/json; charset=utf-8
Content-Length: 95
X-DocuSign-ClientTransactionId: fa4e1202-09a4-431b-9abf-fe4d00acf565
X-DocuSign-TimeTrack: CONN_START,2019-10-17T20:58:31.946Z;;REQ_SENT,2019-10-17T20:58:31.950Z;;REST0_Start,2019-10-17T20:58:32.0620315Z;REST0_End,2019-10-17T20:58:32.0932823Z
X-DocuSign-TraceToken: 996d3b3f-2857-47d6-9d75-26a75b6b0a37
{"errorCode":"RESOURCE_NOT_FOUND","message":"The URL provided does not resolve to a resource."}```

Use express sendFile for HEAD requests

sendFile is for sending files and it also figures out some interesting headers from the file (like content length). For a HEAD request I would ideally want the exact same headers but just skip the body.
There doesn't seem to be an option for this in the API. Maybe I can override something in the response object to stop it from sending anything?
Here's what I got:
res.sendFile(file, { headers: hdrs, lastModified: false, etag: false })
Has anyone solved this?
As Robert Klep has already written, the sendFile already has the required behavior of sending the headers and not sending the body if the request method is HEAD.
In addition to that, Express already handles HEAD requests for routes that have GET handlers defined. So you don't even need to define any HEAD handler explicitly.
Example:
let app = require('express')();
let file = __filename;
let hdrs = {'X-Custom-Header': '123'};
app.get('/file', (req, res) => {
res.sendFile(file, { headers: hdrs, lastModified: false, etag: false });
});
app.listen(3322, () => console.log('Listening on 3322'));
This sends its own source code on GET /file as can be demonstrated with:
$ curl -v -X GET localhost:3322/file
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 3322 (#0)
> GET /file HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost:3322
> Accept: */*
>
< HTTP/1.1 200 OK
< X-Powered-By: Express
< X-Custom-Header: 123
< Accept-Ranges: bytes
< Cache-Control: public, max-age=0
< Content-Type: application/javascript
< Content-Length: 267
< Date: Tue, 11 Apr 2017 10:45:36 GMT
< Connection: keep-alive
<
[...]
The [...] is the body that was not included here.
Without adding any new handler this will also work:
$ curl -v -X HEAD localhost:3322/file
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 3322 (#0)
> HEAD /file HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost:3322
> Accept: */*
>
< HTTP/1.1 200 OK
< X-Powered-By: Express
< X-Custom-Header: 123
< Accept-Ranges: bytes
< Cache-Control: public, max-age=0
< Content-Type: application/javascript
< Content-Length: 267
< Date: Tue, 11 Apr 2017 10:46:29 GMT
< Connection: keep-alive
<
This is the same but with no body.
Express uses send to implement sendFile, which already does exactly what you want.

D2C message using IoT Hub

I’d like to ask you a favor. I do have a problem confirming previously readed D2C message using IoT Hub. I am using REST API to pick up message like (I have replace SIG)
Request:
GET https://iot-hub-pospa.azure-devices.net/devices/18596c88-01e6-3f16-427b-10028d7305c5/messages/devicebound?api-version=2015-08-15-preview HTTP/1.1
IoTHub-MessageLockTimeout: 3600
Accept: application/json
Authorization: SharedAccessSignature sr=iot-hub-pospa.azure-devices.net&sig={sig}&se=1485558838&skn=iothubowner
Host: iot-hub-pospa.azure-devices.net
If-None-Match: "1c5006a4-2288-4a2f-b7ea-dcdf9b5bbc99"
Connection: Close
X-P2P-PeerDist: Version=1.1
X-P2P-PeerDistEx: MinContentInformation=1.0, MaxContentInformation=2.0
Accept-Encoding: peerdist
Response:
HTTP/1.1 200 OK
Content-Length: 35
ETag: "dfc78580-d251-4156-a5f6-c2a30811a504"
Server: Microsoft-HTTPAPI/2.0
iothub-messageid: 02cdb012-9749-48a9-bfb3-5812a4740675
iothub-to: /devices/18596c88-01e6-3f16-427b-10028d7305c5/messages/deviceBound
iothub-expiry:
iothub-correlationid:
iothub-ack: full
iothub-sequencenumber: 56
iothub-enqueuedtime: 2/2/2016 9:57:34 AM
iothub-deliverycount: 0
Date: Tue, 02 Feb 2016 10:21:43 GMT
Connection: close
2/2/2016 10:57:34 AM - Test message
Then when confirming I am getting HTTP 412 like
Request:
DELETE https://iot-hub-pospa.azure-devices.net/devices/18596c88-01e6-3f16-427b-10028d7305c5/messages/devicebound/02cdb012-9749-48a9-bfb3-5812a4740675?api-version=2015-08-15-preview HTTP/1.1
Accept: application/json
If-Match: "02cdb012-9749-48a9-bfb3-5812a4740675"
Authorization: SharedAccessSignature sr=iot-hub-pospa.azure-devices.net&sig={sig}&se=1485558838&skn=iothubowner
Host: iot-hub-pospa.azure-devices.net
Content-Length: 0
Connection: Close
Response:
HTTP/1.1 412 Precondition Failed
Content-Length: 330
Content-Type: application/json; charset=utf-8
Server: Microsoft-HTTPAPI/2.0
iothub-errorcode: DeviceMessageLockLost
Date: Tue, 02 Feb 2016 10:21:49 GMT
Connection: close
 
{"Message":"ErrorCode:DeviceMessageLockLost;Message 02cdb012-9749-48a9-bfb3-5812a4740675 lock was lost for Device 18596c88-01e6-3f16-427b-10028d7305c5\r\nTracking Id:05994074a3664933a0910b5fc70e04e5-G:GatewayWorkerRole.6-B:1-P:cffe397b-f627-4435-bd54-48f5ba79c3ca-TimeStamp:02/02/2016 10:21:49\r\nErrorCode:DeviceMessageLockLost"}
 
 
Does anybody know what should I do to successfully confirm/delete message from IoT Hub, please? Thanks
static DeviceClient _deviceClient;
_deviceClient = DeviceClient.CreateFromConnectionString(<IoTHubURI>, TransportType.Http1);
public void SendMessage(IDictionary<string, object> dictionary)
{
Microsoft.Azure.Devices.Client.Message message = new Microsoft.Azure.Devices.Client.Message();
try
{
foreach (var r in dictionary)
{
message.Properties[r.Key] = r.Value.ToString();
}
_deviceClient.SendEventAsync(message);
}
catch (Exception ex)
{
Debug.WriteLine(ex.Message);
}
}

Getting random "http first read error: EOF" errors in varnish

I'm seeing the following 503 error in varnish from time to time in the logs:
* << BeReq >> 213585014
- Begin bereq 213585013 fetch
- Timestamp Start: 1452675822.032332 0.000000 0.000000
- BereqMethod GET
- BereqURL /client/hedge-funds-asset-managers/
- BereqProtocol HTTP/1.1
- BereqHeader X-Real-IP: 123.125.71.28
- BereqHeader Host: XXXXXXXXXXXXXXXXXXX
- BereqHeader X-Forwarded-Proto: http
- BereqHeader User-Agent: Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
- BereqHeader Accept-Encoding: gzip
- BereqHeader Accept-Language: zh-cn,zh-tw
- BereqHeader Accept: */*
- BereqHeader X-Forwarded-For: 172.18.210.22
- BereqHeader X-Varnish: 213585014
- VCL_call BACKEND_FETCH
- VCL_return fetch
- BackendOpen 232 reload_2016-01-12T07:28:50.cp_12 162.251.80.23 80 172.18.210.71 40019
- Timestamp Bereq: 1452675822.047840 0.015508 0.015508
- FetchError http first read error: EOF
- BackendClose 232 reload_2016-01-12T07:28:50.cp_12
- Timestamp Beresp: 1452675876.038544 54.006212 53.990704
- Timestamp Error: 1452675876.038555 54.006223 0.000010
- BerespProtocol HTTP/1.1
- BerespStatus 503
- BerespReason Service Unavailable
- BerespReason Backend fetch failed
- BerespHeader Date: Wed, 13 Jan 2016 09:04:36 GMT
- BerespHeader Server: Varnish
- VCL_call BACKEND_ERROR
- BerespHeader Content-Type: text/html; charset=utf-8
- BerespHeader Retry-After: 5
- VCL_return deliver
- Storage malloc Transient
- ObjProtocol HTTP/1.1
- ObjStatus 503
- ObjReason Backend fetch failed
- ObjHeader Date: Wed, 13 Jan 2016 09:04:36 GMT
- ObjHeader Server: Varnish
- ObjHeader Content-Type: text/html; charset=utf-8
- ObjHeader Retry-After: 5
- Length 286
- BereqAcct 350 0 350 0 0 0
- End
The issue is not with the backend connection because a curl to the same URL from the varnish server works fine. The version of varnish is 4.1.0. I'm not sure what "http first read error: EOF" means and any light on this issue is appreciated. Due to the random nature of this issue, I do not have any way to reproduce it as well.
A "first read error" happens in Varnish when you try to read headers from the backend before calling vcl_fetch, and Varnish failed to get a response. TL;DR: your backend is either closing the connection before delivering a response, or it is timing out delivering the response. You could use a tool like wireshark to determine which of the two is happening.
To understand what goes on, let's do some source diving:
static int __match_proto__(vdi_gethdrs_f)
vbe_dir_gethdrs(const struct director *d, struct worker *wrk,
struct busyobj *bo)
{
int i, extrachance = 1;
struct backend *bp;
struct vbc *vbc;
...
do {
vbc = vbe_dir_getfd(wrk, bp, bo);
Not getting too much into directors, vbe_dir_gethdrs is called after Varnish has either opened a new connection, or decided it is going to reuse a connection.
if (vbc->state != VBC_STATE_STOLEN)
extrachance = 0;
If we reuse a connection, vbc->state is set to VBC_STATE_STOLEN (Varnish-Cache/bin/varnishd/cache/cache_backend_tcp.c line 364). When we've opened a new connection, this value is not set. So far, so good.
i = V1F_SendReq(wrk, bo, &bo->acct.bereq_hdrbytes, 0);
if (vbc->state != VBC_STATE_USED)
VBT_Wait(wrk, vbc);
assert(vbc->state == VBC_STATE_USED);
if (i == 0)
i = V1F_FetchRespHdr(bo);
What this does is sends the request to the backend. If everything there goes well, we then call V1F_FetchRespHdr, which waits for the origin to send its protocol response and headers. If we follow the code into V1F_FetchRespHdr:
VTCP_set_read_timeout(htc->fd, htc->first_byte_timeout);
...
do {
...
i = read(htc->fd, htc->rxbuf_e, i);
if (i <= 0) {
bo->acct.beresp_hdrbytes +=
htc->rxbuf_e - htc->rxbuf_b;
WS_ReleaseP(htc->ws, htc->rxbuf_b);
VSLb(bo->vsl, SLT_FetchError, "http %sread error: EOF",
first ? "first " : "");
htc->doclose = SC_RX_TIMEOUT;
return (first ? 1 : -1);
}
Here, we see that we're setting a timeout on the socket before we do the read syscall. If this read returns an error (the < 0 case), or EOF (the == 0 case), and this is the first time we have called read, we end up logging http first read error: EOF as you are seeing in your varnishlog output.
So, if you open a new connection to the backend, and the backend times out or closes the connection after the request was sent, you get this error.
Personally, I would find it suspect if your origin was closing connections; I think timeouts are usually more likely. But connections may be closed if your backend thinks it has too many open connections, or perhaps it has received too many requests over the connection, or something like this.
Hope that helps!

Resources