Axios/HTTPS certificate expired error only on Windows - node.js

I am currently working on a VSCode extension that internally makes some API calls to a HTTPS protected endpoint. On Linux and Mac OS the extension works as expected but on Windows machines axios, the internal HTTP client used to make the API calls, is rejecting those requests due to the certificates being expired. When I access the API endpoint though via Firefox, Chrome and even Edge the certificates seems find.
I have upgraded Node to 16.14.0 and also to 17.6.0 but the problem still remains. As the API is only accessible through our VPN, with my VPN activated of course, I used testssl.sh to verify that the whole trust-chain is still valid:
Testing protocols via sockets except NPN+ALPN
SSLv2 not offered (OK)
SSLv3 not offered (OK)
TLS 1 not offered
TLS 1.1 not offered
TLS 1.2 offered (OK)
TLS 1.3 offered (OK): final
NPN/SPDY not offered
ALPN/HTTP2 h2, http/1.1 (offered)
Testing cipher categories
NULL ciphers (no encryption) not offered (OK)
Anonymous NULL Ciphers (no authentication) not offered (OK)
Export ciphers (w/o ADH+NULL) not offered (OK)
LOW: 64 Bit + DES, RC[2,4] (w/o export) not offered (OK)
Triple DES Ciphers / IDEA not offered
Obsolete CBC ciphers (AES, ARIA etc.) offered
Strong encryption (AEAD ciphers) offered (OK)
Testing robust (perfect) forward secrecy, (P)FS -- omitting Null Authentication/Encryption, 3DES, RC4
PFS is offered (OK) TLS_AES_256_GCM_SHA384 TLS_CHACHA20_POLY1305_SHA256 ECDHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-SHA384 ECDHE-RSA-AES256-SHA
DHE-RSA-AES256-GCM-SHA384 DHE-RSA-AES256-SHA256 DHE-RSA-AES256-SHA TLS_AES_128_GCM_SHA256 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-SHA256
ECDHE-RSA-AES128-SHA DHE-RSA-AES128-GCM-SHA256 DHE-RSA-AES128-SHA256 DHE-RSA-AES128-SHA
Elliptic curves offered: prime256v1 secp384r1 secp521r1 X25519 X448
DH group offered: HAProxy (2048 bits)
Testing server preferences
Has server cipher order? yes (OK) -- TLS 1.3 and below
Negotiated protocol TLSv1.3
Negotiated cipher TLS_AES_256_GCM_SHA384, 253 bit ECDH (X25519)
Cipher order
TLSv1.2: ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES256-GCM-SHA384 DHE-RSA-AES128-GCM-SHA256 DHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES128-SHA256 ECDHE-RSA-AES128-SHA
ECDHE-RSA-AES256-SHA384 ECDHE-RSA-AES256-SHA DHE-RSA-AES128-SHA256 DHE-RSA-AES128-SHA DHE-RSA-AES256-SHA256 DHE-RSA-AES256-SHA
TLSv1.3: TLS_AES_256_GCM_SHA384 TLS_CHACHA20_POLY1305_SHA256 TLS_AES_128_GCM_SHA256
...
Certificate Validity (UTC) 65 >= 30 days (2022-02-01 08:18 --> 2022-05-02 08:18)
...
The other certificates in the trustchain are also valid and not expired at all.
Axios is used within an own HttpClient class that looks like this:
export class HttpClient {
private readonly instance: AxiosInstance;
constructor() {
this.instance = axios.create();
}
async asyncGet<T>(url: string): Promise<Response<T>> {
return this.instance.get<T>(url)
.then(axiosResponse => {
...
})
.catch(error => {
console.error(error);
throw error;
});
}
}
I also tried the https
const options: https:RequestOptions = {
hostname: '...',
port: 443,
path: '/...',
method: 'GET'
};
const request = https.request(url, options, (res: IncomingMessage) => {
console.log(`statusCode: ${res.statusCode ? res.statusCode : 'undefined'}`);
res.on('data', d => {
process.stdout.write(d as string);
}
});
request.on('error', error => {
console.error(error);
};
request.end();
analog to the samples given in the NodeJS documentation and tls
const socket = tls.connect(443, hostname, undefined, () => {
console.log('Connected to ' + hostname);
console.log('TLS.connect', socket.authorized ? 'authorized' : 'unauthorized');
console.log('Cipher: ' + JSON.stringify(socket.getCipher()));
console.log('Protocol: ' + JSON.stringify(socket.getProtocol()));
const peerCert: tls.DetailedPeerCertificate = socket.getPeerCertificate(true);
console.log(`Peer-Cert ${JSON.stringify(peerCert.subject)} - valid from: ${peerCert ? peerCert.valid_from : 'invalid'} valid till: ${peerCert ? peerCert.valid_to : 'invalid'}`);
const issuerCert = peerCert.issuerCertificate;
console.log(`issuer-Cert ${JSON.stringify(issuerCert.subject)} - valid from: ${issuerCert ? issuerCert.valid_from : 'invalid'} valid till: ${issuerCert ? issuerCert.valid_to : 'invalid'}`);
const rootCert = issuerCert.issuerCertificate;
console.log(`root-Cert ${JSON.stringify(rootCert.subject)} - valid from: ${rootCert ? rootCert.valid_from : 'invalid'} valid till: ${rootCert ? rootCert.valid_to : 'invalid'}`);
});
as proposed in this answer.
Both axios and https return an error like this:
{
"message": "certificate has expired",
"name": "Error",
"stack": "Error: certificate has expired\n\tat TLSSocket.onConnectSecure (_tls_wrap.js:1497:34)\n\tat TLSSocket.emit (events.js:315:20)\n\tat TLSSocket._finishInit (_tls_wrap.js:932:8)\n\tat TLSWrap.onhandshakedone (_tls_wrap.js:706:12)\n\tat TLSWrap.callbackTrampoline (internal/async_hooks.js:131:14)",
"config": {
"transitional": {
"silentJSONParsing": true,
"forcedJSONParsing": true,
"clarifyTimeoutError": false
},
"transformRequest": [
"null"
],
"transformResponse": [
"null"
],
"timeout": 0,
"xsrfCookieName": "XSRF-TOKEN",
"xsrfHeaderName": "X-XSRF-TOKEN",
"maxContentLength": -1,
"maxBodyLength": -1,
"headers": {
"Accept": "application/json, text/plain, */*",
"user-Agent": "axios/0.26.0"
},
"method": "get",
"url": "https://..."
},
"code": "CERT_HAS_EXPIRED",
"status": null
}
with a more human-readable stacktrace:
Error: certificate has expired
at TLSSocket.onConnectSecure (_tls_wrap.js:1497:34)
at TLSSocket.emit (events.js:315:20)
at TLSSocket._finishInit (_tls_wrap.js:932:8)
at TLSWrap.onhandshakedone (_tls_wrap.js:706:12)
at TLSWrap.callbackTrampoline (internal/async_hooks.js:131:14)
while for tls I get the following output:
Connected to ...
TLS.connect authorized
Cipher: {"name":"TLS_CHACHA20_POLY1305_SHA256","standardName":"TLS_CHACHA20_POLY1305_SHA256","version":"TLSv1/SSLv3"}
Protocol: "TLSv1.3"
Peer-Cert {"CN": "*...."} - valid from: Feb 1 08:18:23 2022 GMT valid till: May 2 08:18:22 2022 GMT
issuer-Cert {"C":"US","O":"Let's Encrypt","CN":"R3"} - valid from Sep 4 00:00:00 2020 GMT valid till: Sep 15 16:00:00 2025 GMT
root-Cert {"C":"US","O":"Internet Security Research Group","CN":"ISRG Root X1"} - valid from: Jan 20 19:14:03 2021 GMT valid till: Sep 30 18:14:03 2024 GMT
So tls seems to be able to connect to the API server and perform the SSL/TLS handshake, but https and axios somehow fail.
I also stumbled upon this question here, which seems to be related, but as I am already on the latest NodeJS release (as well as any dependency used in the extension is on the most recent version) and this error only occurs on Windows (mostly 10, unsure if and how many users actually use Windows 11) machines I think the question deserves its own spot here on SO.
In order to rule out a lack of common supported ciphers between the Windows and Node.js based tls implementation and the nginx managed server side, I also checked the available ciphers in Node via node -p crypto.constants.defaultCoreCipherList which returns a list like this:
TLS_AES_256_GCM_SHA384
TLS_CHACHA20_POLY1305_SHA256
TLS_AES_128_GCM_SHA256
ECDHE-RSA-AES128-GCM-SHA256
ECDHE-ECDSA-AES128-GCM-SHA256
ECDHE-RSA-AES256-GCM-SHA384
ECDHE-ECDSA-AES256-GCM-SHA384
DHE-RSA-AES128-GCM-SHA256
ECDHE-RSA-AES128-SHA256
DHE-RSA-AES128-SHA256
ECDHE-RSA-AES256-SHA384
DHE-RSA-AES256-SHA384
ECDHE-RSA-AES256-SHA256
DHE-RSA-AES256-SHA256
HIGH
!aNULL
!eNULL
!EXPORT
!DES
!RC4
!MD5
!PSK
!SRP
!CAMELLIA
which indicates that enough ciphers would overlap between client and server.
Why do I still get a certificate expired error on Windows machines with axios/https
when Linux and MacOS work just fine with these settings and tls is able to connect to the remote API sucessfully even on Windows machines?

Related

Unable to finish TLS handshake for my express webserver

Here's my webserver, most parts not included in favor of being concise.
server.js
const https = require('https');
const app = require('express')();
const port = 1000;
const options = {
key : getKey(),
cert: getCert(),
ciphers: getCiphers(),
passphrase: "abcd",
rejectUnauthorized: true,
requestCert: true
};
const server = https.createServer(options, app);
server.on('clientError', function (err) {
console.log('received client error');
console.log({err});
})
server.on('connection', function (err) {
console.log('client connected');
})
server.listen(port);
To test my server, I run a connection commands with openssl
openssl s_client -connect localhost:1000 \
-servername localhost \
-CAfile etc/root-cert/ca.cert.pem \
-cert etc/certs/client.cert.pem \
-key etc/private/client.key.pem
This returns the following output, which indicates that the TLS connection was successful (as far as I'm aware):
CONNECTED(00000005)
depth=2 C = US, ST = California, O = Hackysack, CN = Hackysack Root CA
verify return:1
depth=1 C = US, ST = California, O = Hackysack, CN = Hackysack Intermediate CA
verify return:1
depth=0 C = US, ST = California, O = Hackysack, CN = localhost
verify return:1
write W BLOCK
---
Certificate chain
0 s:/C=US/ST=California/O=Hackysack/CN=localhost
i:/C=US/ST=California/O=Hackysack/CN=Hackysack Intermediate CA
1 s:/C=US/ST=California/O=Hackysack/CN=Hackysack Intermediate CA
i:/C=US/ST=California/O=Hackysack/CN=Hackysack Root CA
---
Server certificate
-----BEGIN CERTIFICATE-----
[Long cert file here]
-----END CERTIFICATE-----
subject=/C=US/ST=California/O=Hackysack/CN=localhost
issuer=/C=US/ST=California/O=Hackysack/CN=Hackysack Intermediate CA
---
No client certificate CA names sent
Server Temp Key: ECDH, X25519, 253 bits
---
SSL handshake has read 3400 bytes and written 2061 bytes
---
New, TLSv1/SSLv3, Cipher is AEAD-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.3
Cipher : AEAD-AES128-GCM-SHA256
Session-ID:
Session-ID-ctx:
Master-Key:
Start Time: 1674855496
Timeout : 7200 (sec)
Verify return code: 0 (ok)
---
read:errno=0
However, in my server terminal logs I see that I receive a clientError when trying to make this call via HTTPS request.
{
err: Error: socket hang up
at connResetException (internal/errors.js:639:14)
at TLSSocket.onSocketClose (_tls_wrap.js:1063:23)
at TLSSocket.emit (events.js:412:35)
at net.js:686:12
at Socket.done (_tls_wrap.js:564:7)
at Object.onceWrapper (events.js:520:26)
at Socket.emit (events.js:400:28)
at TCP.<anonymous> (net.js:686:12) {
code: 'ECONNRESET'
}
}
Here's my client request:
const https = require('https');
const options = {
hostname: 'localhost',
port: 1000,
method: "POST",
path: "/test",
cert: getClientCert(),
key: getClientKey(),
passphrase: "abcd",
ciphers: getCiphers()
ca: getCA(),
}
const request = https.request(options);
I noticed this error only started happening when I included requestCert: true when I created the server. I think something must be going wrong with my client side for this to be the case. The main problem with debugging this is that I'm not receiving a good error when it fails the handshake.
How can I better debug this issue?

How to connect to Elasticsearch by PySpark without cert verification?

Is there a way to connect to Elasticsearch (https://IP:9200) by PySpark without certificate verification?
I didn't see any settings for this: https://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html
Curl works fine:
curl -iv --insecure --user admin:password --noproxy '*' https://xx.xxx.xx.xxx:9200
* Trying xx.xxx.xx.xxx:9200...
* Connected to xx.xxx.xx.xxx (xx.xxx.xx.xxx) port 9200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /opt/conda/ssl/cacert.pem
CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: OU=dev-1; CN=dev-1-es-http.es-dev.es.local
* start date: Jul 9 00:47:04 2020 GMT
* expire date: Jul 9 00:57:04 2021 GMT
* issuer: OU=dev-1; CN=dev-1-http
* SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
* Server auth using Basic with user 'admin'
> GET / HTTP/1.1
> Host: xx.xxx.xx.xxx:9200
> Authorization: Basic abc==
> User-Agent: curl/7.71.1
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< content-type: application/json; charset=UTF-8
content-type: application/json; charset=UTF-8
< content-length: 544
content-length: 544
<
{
"name" : "dev-1-es-hot-2",
"cluster_name" : "dev-1",
"cluster_uuid" : "444",
"version" : {
"number" : "7.8.0",
"build_flavor" : "default",
"build_type" : "docker",
"build_hash" : "123",
"build_date" : "2020-06-14T19:35:50.234439Z",
"build_snapshot" : false,
"lucene_version" : "8.5.1",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
* Connection #0 to host xx.xxx.xx.xxx left intact
My last try:
reader = spark.read.format("org.elasticsearch.spark.sql") \
.option("es.port", "9200") \
.option("spark.es.net.http.auth.user", "admin") \
.option("spark.es.net.http.auth.pass", "password") \
.option("es.net.ssl", "true") \
.option("es.net.ssl.cert.allow.self.signed", "true") \
.option("es.net.proxy.https.use.system.props", "false") \
.option("es.nodes", "xx.xxx.xxx.xxx")
Ends with:
: org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'
at org.elasticsearch.hadoop.rest.InitializationUtils.discoverClusterInfo(InitializationUtils.java:348)
at org.elasticsearch.spark.sql.ElasticsearchRelation.cfg$lzycompute(DefaultSource.scala:225)
at org.elasticsearch.spark.sql.ElasticsearchRelation.cfg(DefaultSource.scala:223)
at org.elasticsearch.spark.sql.ElasticsearchRelation.lazySchema$lzycompute(DefaultSource.scala:229)
at org.elasticsearch.spark.sql.ElasticsearchRelation.lazySchema(DefaultSource.scala:229)
at org.elasticsearch.spark.sql.ElasticsearchRelation.$anonfun$schema$1(DefaultSource.scala:233)
at scala.Option.getOrElse(Option.scala:189)
at org.elasticsearch.spark.sql.ElasticsearchRelation.schema(DefaultSource.scala:233)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:448)
at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:326)
at org.apache.spark.sql.DataFrameReader.$anonfun$load$3(DataFrameReader.scala:308)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:308)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:240)
at jdk.internal.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:282)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.elasticsearch.hadoop.rest.EsHadoopTransportException: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:128)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:432)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:428)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:388)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:392)
at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:168)
at org.elasticsearch.hadoop.rest.RestClient.mainInfo(RestClient.java:745)
at org.elasticsearch.hadoop.rest.InitializationUtils.discoverClusterInfo(InitializationUtils.java:338)
... 23 more
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:349)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:292)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:287)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1356)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1231)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1174)
at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:421)
at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:182)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:171)
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1408)
at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1314)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:440)
at java.base/sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:819)
at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1189)
at java.base/java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:81)
at java.base/java.io.BufferedOutputStream.flush(BufferedOutputStream.java:142)
at org.apache.commons.httpclient.HttpConnection.flushRequestOutputStream(HttpConnection.java:828)
at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:2116)
at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1096)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.doExecute(CommonsHttpTransport.java:685)
at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:664)
at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:116)
... 30 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439)
at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306)
at java.base/sun.security.validator.Validator.validate(Validator.java:264)
at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:313)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:233)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:110)
at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory$TrustManagerDelegate.checkServerTrusted(SSLSocketFactory.java:76)
at java.base/sun.security.ssl.AbstractTrustManagerWrapper.checkServerTrusted(SSLContextImpl.java:1510)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1340)
... 54 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434)
... 62 more
I also trying to provide keystore/truestore without results.
I had the same issue as you, but in Java. As already stated in the comments below the question, the issue is that we need to use the server certificates in order to complete the TLS handshake.
I used the tool suggested by lubomand added the certificates to a .jks truststore, put a password then finally I added the truststore to all the worker servers of the spark cluster.
Then added the following properties
conf.put("spark.es.net.ssl.keystore.location", "<truststore path>");
conf.put("spark.es.net.ssl.keystore.pass","<truststore password>";
In this way I was able to communicate with the ElasticSearch Server.
P.S. If this answer is useful don't thank me, but #lubom which solved my issue in the comments thread below the question! I answered the question (after asking permission) just to clearly put how we both solved our common issue.

curl cannot handshake https server created with Nodejs v14.16.0 tls.createSecureContext

I have set up a HTTPS server (nodejs v14.16.0) and certificates from letsEncrypt (which work in the current version of the app that uses https.createServer). Unfortunately, curl cannot connect successfully to my HTTPS server. I get the following error
routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
Here's a minimum reproducible version of my server
const https = require('https')
const tls = require('tls');
const fs = require('fs');
const constants = require('constants');
tls.DEFAULT_ECDH_CURVE = "auto"
require('dotenv').config()
const certOpts = {
key: `${process.env.KEY_PATH}/privkey.pem`,
cert: `${process.env.KEY_PATH}/cert.pem`,
ca: `${process.env.KEY_PATH}/chain.pem`,
};
/**
*
* #param {Record<string, string>} filePathMap
* #returns {Record<string, Buffer>}
*/
function getBuffersFromFilePathMap(filePathMap) {
const bufferMap = {}
for (const path in filePathMap) {
if (Object.hasOwnProperty.call(filePathMap, path)) {
bufferMap[path] = fs.readFileSync(filePathMap[path]);
}
}
return bufferMap;
}
const buffers = getBuffersFromFilePathMap(certOpts);
function createContext() {
return tls.createSecureContext({
...buffers,
secureOptions: constants.SSL_OP_NO_SSLv2 | constants.SSL_OP_NO_SSLv3 | constants.SSL_OP_NO_TLSv1 | constants.SSL_OP_NO_TLSv1_1,
maxVersion:'TLSv1.2'
});
}
const server = https.createServer({ secureContext: createContext() }, (req, res) => {
// eslint-disable-next-line no-console
console.log('connect');
res.writeHead(200);
res.write('Hello World!\n');
res.end('Goodbye World!\n');
});
server.listen(9999, () => console.log('Server up: ', server.address()));
This is the output of curl --version
curl 7.61.1 (x86_64-redhat-linux-gnu) libcurl/7.61.1 OpenSSL/1.0.2k zlib/1.2.8 libidn2/2.3.0 libpsl/0.6.2 (+libicu/50.1.2) libssh2/1.4.2 nghttp2/1.31.1
Release-Date: 2018-09-05
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz HTTP2 UnixSockets HTTPS-proxy PSL
From what I've gathered, ssl3 may not be supported .
Output from curl
$ curl -k https://localhost:9999
curl: (35) error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
Curl traces
== Info: Rebuilt URL to: https://localhost:9999/
== Info: Trying 127.0.0.1...
== Info: TCP_NODELAY set
== Info: Connected to localhost (127.0.0.1) port 9999 (#0)
== Info: ALPN, offering h2
== Info: ALPN, offering http/1.1
== Info: Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:#STRENGTH
== Info: successfully set certificate verify locations:
== Info: CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
== Info: TLSv1.2 (OUT), TLS header, Certificate Status (22):
=> Send SSL data, 5 bytes (0x5)
0000: .....
== Info: TLSv1.2 (OUT), TLS handshake, Client hello (1):
=> Send SSL data, 512 bytes (0x200)
0000: .......*.c....}.5.H>^..e\;)}.zEW.....d....0.,.(.$.............k.
0040: j.i.h.9.8.7.6.........2...*.&.......=.5.../.+.'.#.............g.
0080: #.?.>.3.2.1.0.........E.D.C.B.1.-.).%.......<./...A.............
00c0: ............3.........localhost......................... .......
0100: ..............................3t.........h2.http/1.1............
0140: ................................................................
0180: ................................................................
01c0: ................................................................
== Info: TLSv1.2 (IN), TLS header, Unknown (21):
<= Recv SSL data, 5 bytes (0x5)
0000: .....
== Info: TLSv1.2 (IN), TLS alert, handshake failure (552):
<= Recv SSL data, 2 bytes (0x2)
0000: .(
== Info: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
Okay, it appears that even though the documentation for tls.createSecureContext says the result is "usable as an argument to several tls APIs, such as tls.createServer" it actually isn't. It is accepted by server.addContext (for a virtual host or more exactly an SNI-value handler) tls.connect (for client) tls.createSecurePair (deprecated) and new TLSSocket (low-level), but createServer only takes the same options as createSecureContext not an actual SecureContext. Since you didn't supply the needed key&cert in a usable form, and OpenSSL by default disables anonymous ciphersuites (which most clients don't offer anyway), all handshakes fail with no_shared_cipher. Try:
const server = https.createServer(
{...buffers,
secureOptions: constants.SSL_OP_NO_SSLv2 | constants.SSL_OP_NO_SSLv3 | constants.SSL_OP_NO_TLSv1 | constants.SSL_OP_NO_TLSv1_1,
maxVersion:'TLSv1.2' },
(req, res) => stuff )
BTW the req,res=> function is invoked when you get a request not a connection as such; a proper client will only connect when it needs to do a request, but it is possible to connect without doing a request, and conversely fine to do multiple requests on one connection.

How to specify a payee with paypal orders api?

Here is the tutorial that I am following: What is the best way to enable a website users send money to each other?
The problem is with my post request.
Let's break it down into pieces.
Headers
headers: {
"Authorization": `Bearer ${accessToken}`,
"Content-Type": "application/json"
}
I get the access token also via paypal api namely https://api.sandbox.paypal.com/v1/oauth2/token.
Body
body: {
intent: 'CAPTURE',
purchase_units: [{
amount: {
currency_code: 'USD',
value: '2.00'
},
payee: {
email_address: "myanothersandboxaccount#gmail.com"
}
}]
}
But, it doesn't work, as I get an error
{
name: 'INVALID_REQUEST',
message: 'Request is not well-formed, syntactically incorrect, or violates schema.',
debug_id: '2884e1b5eccee',
details: [
{
field: '/',
location: 'body',
issue: 'INVALID_SYNTAX',
description: 'MALFORMED_REQUEST_JSON'
}
],
links: [
{
href: 'https://developer.paypal.com/docs/api/orders/v2/#error-INVALID_SYNTAX',
rel: 'information_link',
encType: 'application/json'
}
]
}
curl request
curl -v -X POST https://api.sandbox.paypal.com/v2/checkout/orders \
-H "Content-Type: application/json" \
-H "Authorization: Bearer A23AALAej8Yg-4iKJBcWckiv5-ZlhYWlkmBsPuWaVngJcMigU7P-6f8P02vnOpIo8QlOJ-P3hd3K86vKo_lpSlu0-bZBj98eg" \
-d '{
"intent": "CAPTURE",
"purchase_units": [
{
"amount": {
"currency_code": "USD",
"value": "100.00"
},
"payee": {
"email": "myanothersandboxaccount#gmail.com"
}
}
]
}'
curl response
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 173.0.82.78...
* TCP_NODELAY set
* Connected to api.sandbox.paypal.com (173.0.82.78) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: C=US; ST=California; L=San Jose; O=PayPal, Inc.; OU=PayPal Production; CN=api.sandbox.paypal.com
* start date: Jul 27 00:00:00 2020 GMT
* expire date: Aug 1 12:00:00 2022 GMT
* subjectAltName: host "api.sandbox.paypal.com" matched cert's "api.sandbox.paypal.com"
* issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert SHA2 High Assurance Server CA
* SSL certificate verify ok.
> POST /v2/checkout/orders HTTP/1.1
> Host: api.sandbox.paypal.com
> User-Agent: curl/7.64.1
> Accept: */*
> Content-Type: application/json
> Authorization: Bearer A23AALAej8Yg-4iKJBcWckiv5-ZlhYWlkmBsPuWaVngJcMigU7P-6f8P02vnOpIo8QlOJ-P3hd3K86vKo_lpSlu0-bZBj98eg
> Content-Length: 212
>
* upload completely sent off: 212 out of 212 bytes
< HTTP/1.1 201 Created
< Cache-Control: max-age=0, no-cache, no-store, must-revalidate
< Content-Length: 501
< Content-Type: application/json
< Date: Sat, 05 Dec 2020 10:30:13 GMT
< Paypal-Debug-Id: 6bd069526af1c
<
* Connection #0 to host api.sandbox.paypal.com left intact
{"id":"674004650C383744Y","status":"CREATED","links":[{"href":"https://api.sandbox.paypal.com/v2/checkout/orders/674004650C383744Y","rel":"self","method":"GET"},{"href":"https://www.sandbox.paypal.com/checkoutnow?token=674004650C383744Y","rel":"approve","method":"GET"},{"href":"https://api.sandbox.paypal.com/v2/checkout/orders/674004650C383744Y","rel":"update","method":"PATCH"},{"href":"https://api.sandbox.paypal.com/v2/checkout/orders/674004650C383744Y/capture","rel":"capture","method":"POST"}]}* Closing connection 0
And, with curl I get a successful response. I can see that all went well from my paypal sandbox dashboard.
What could it be?
I was using node-fetch to call the paypal api to create an order. Tried with axios, worked just fine.
The error basically means PayPal isn't able to parse the body object it received as JSON, so it appears something is being transmitted/received incorrectly.
You should first test with command-line curl to verify functionality and your JSON's correctness, which should work.
Then to get more details about what's actually going wrong in what you are sending, you will need to log the actual data being sent (not your code, the actual data the code sends when executed)
One way to do this is to instead of sending your request to https://api-m.paypal.com , instead send it to a request bin service like offered via https://requestbin.com/

axios SSL error with Node 12 : SSL routines:ssl_choose_client_version:unsupported protocol

I’m running into an issue with axios and Node 12. As I’m not sure this error is only related to axios, I followed the advice to ask on SO rather than opening a bug on axios’ GitHub.
Here is the code I’m trying to run :
const axios = require('axios')
axios({
method: 'get',
url: 'https://www.colisprive.com/moncolis/pages/detailColis.aspx?numColis=12345',
responseType: 'text'
}).then((response) => {
console.log(response)
})
This code fails on Node 12 with following error :
Error: write EPROTO 140121214769024:error:1425F102:SSL routines:ssl_choose_client_version:unsupported protocol:../deps/openssl/openssl/ssl/statem/statem_lib.c:1929:
at WriteWrap.onWriteComplete [as oncomplete] (internal/stream_base_commons.js:87:16)
Same code ran against Node 11 doesn’t throw any error.
When I curl -v I got this :
* Trying 91.208.224.32:443...
* TCP_NODELAY set
* Connected to www.colisprive.com (91.208.224.32) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: serialNumber=391029345; jurisdictionC=FR; businessCategory=Private Organization; C=FR; postalCode=13290; ST=Bouches-du-Rh�ne; L=AIX EN PROVENCE; street=1330 AV J R G GAUTIER DE LA LAUZIERE; street=ZI MILLES EUROPARC PICHAURY; O=COLIS PRIVE SAS; OU=0002 391029345; CN=www.colisprive.com
* start date: Sep 3 00:00:00 2018 GMT
* expire date: Sep 2 23:59:59 2020 GMT
* subjectAltName: host "www.colisprive.com" matched cert's "www.colisprive.com"
* issuer: C=GB; ST=Greater Manchester; L=Salford; O=COMODO CA Limited; CN=COMODO RSA Extended Validation Secure Server CA
* SSL certificate verify ok.
> GET /moncolis/pages/detailColis.aspx?numColis=12345 HTTP/1.1
> Host: www.colisprive.com
> User-Agent: curl/7.65.3
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 302 Found
< Cache-Control: private
< Content-Type: text/html; charset=utf-8
< Location: /moncolis/Default.aspx?numColis=12345&cp=
< Server: Microsoft-IIS/7.5
< Set-Cookie: ASP.NET_SessionId=eln3cq143d35lfj5tpqkkwcg; path=/; HttpOnly
< X-Powered-By: Colis Priv�
< Date: Fri, 24 Jan 2020 13:48:35 GMT
< Content-Length: 162
<
<html><head><title>Object moved</title></head><body>
<h2>Object moved to here.</h2>
</body></html>
* Connection #0 to host www.colisprive.com left intact
As you can see, it gives a 302 Found with a Location header pointing to another endpoint. I agree it should answer a 301 Moved to indicate document has moved, but this is not the case and it is handled as expected by axios on Node 11 (fetching endpoint under Location header).
I saw that Node 12 now includes TLS 1.3 as default, so this could be related to that…
Also, there is an unknown character in X-Powered-By header.
I tried to :
reproduce this issue with an express server always replying 302 Found with same headers : works as expected
fetch another .aspx web page with axios : works as expected
The problem is not just with axios but with got as well.
Node.js 12's default TLS settings are stricter now. The site doesn't handle TLS v1.2. Node 12 by default need 1.2.
You can change this via a command line flag (--tls-min-v1.0) when running your app.
something like this
node --tls-min-v1.0 app.js

Resources