I want to know if libcurl supports the multicast file transfer.
curl is a command line tool for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, Telnet and TFTP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, kerberos...), file transfer resume, proxy tunneling and
FEATURES
curl tool
- config file support
- multiple URLs in a single command line
- range "globbing" support: [0-13], {one,two,three}
- multiple file upload on a single command line
- custom maximum transfer rate
- redirectable stderr
libcurl supports
- full URL syntax with no length limit
- custom maximum download time
- custom least download speed acceptable
- custom output result after completion
- guesses protocol from host name unless specified
- uses .netrc
- progress bar/time specs while downloading
- "standard" proxy environment variables support
- compiles on win32 (reported builds on 40+ operating systems)
- selectable network interface for outgoing traffic
- IPv6 support on unix and Windows
- persistant connections
- socks5 support
- supports user name + password in proxy environment variables
- operations through proxy "tunnel" (using CONNECT)
- supports large files (>2GB and >4GB) both upload/download
- replacable memory functions (malloc, free, realloc, etc)
- asynchronous name resolving (*6)
- both a push and a pull style interface
HTTP
- HTTP/1.1 compliant (optionally uses 1.0)
- GET
- PUT
- HEAD
- POST
- Pipelining
- multipart formpost (RFC1867-style)
- authentication: Basic, Digest, NTLM(*9), GSS-Negotiate/Negotiate(*3) and
SPNEGO (*4) to server and proxy
- resume (both GET and PUT)
- follow redirects
- maximum amount of redirects to follow
- custom HTTP request
- cookie get/send fully parsed
- reads/writes the netscape cookie file format
- custom headers (replace/remove internally generated headers)
- custom user-agent string
- custom referer string
- range
- proxy authentication
- time conditions
- via http-proxy
- retrieve file modification date
- Content-Encoding support for deflate and gzip
- "Transfer-Encoding: chunked" support for "uploads"
HTTPS (*1)
- (all the HTTP features)
- using client certificates
- verify server certificate
- via http-proxy
- select desired encryption
- force usage of a specific SSL version (SSLv2(*7), SSLv3 or TLSv1)
FTP
- download
- authentication
- kerberos4 (*5), kerberos5 (*3)
- active/passive using PORT, EPRT, PASV or EPSV
- single file size information (compare to HTTP HEAD)
- 'type=' URL support
- dir listing
- dir listing names-only
- upload
- upload append
- upload via http-proxy as HTTP PUT
- download resume
- upload resume
- custom ftp commands (before and/or after the transfer)
- simple "range" support
- via http-proxy
- all operations can be tunneled through a http-proxy
- customizable to retrieve file modification date
- no dir depth limit
FTPS (*1)
- implicit ftps:// support that use SSL on both connections
- explicit "AUTH TSL" and "AUTH SSL" usage to "upgrade" plain ftp://
connection to use SSL for both or one of the connections
SCP (*8)
- both password and public key auth
SFTP (*8)
- both password and public key auth
- with custom commands sent before/after the transfer
TFTP
- download / upload
TELNET
- connection negotiation
- custom telnet options
- stdin/stdout I/O
LDAP (*2)
- full LDAP URL support
DICT
- extended DICT URL support
FILE
- URL support
- "uploads"
- resume
FOOTNOTES
=========
*1 = requires OpenSSL, GnuTLS, NSS, yassl, axTLS or PolarSSL
*2 = requires OpenLDAP
*3 = requires a GSSAPI-compliant library, such as Heimdal or similar.
*4 = requires FBopenssl
*5 = requires a krb4 library, such as the MIT one or similar.
*6 = requires c-ares
*7 = requires OpenSSL or NSS, as GnuTLS only supports SSLv3 and TLSv1
*8 = requires libssh2
*9 = requires OpenSSL, GnuTLS, NSS or yassl
Related
I am running python3 (version 3.6.9) Twisted (version 18.4.0) on an Ubuntu 18.04 server. This server is used for webhooks for Twilio. The webhooks work fine over http. I installed a LetsEncrypt cert, and the LetsEncrypt ssl cert works fine for serving https over a FireFox browser.
However, when I point twilio to the https version of the webhook, I get the following error in the twilio debugger console:
Error - 11237
Certificate Invalid - Could not find path to certificate
Twilio tried to validate your SSL certificate but was unable to find it in our certificate store. Possible Causes
You are using a self signed certificate.
The certificate authority you are using is not on our list of approved certificate authorities.
Your certificate chain is incomplete and requires an additional download.
Possible Solutions
Do not use a self signed certificate.
Concatenate your certificate chain so that no additional download is required.
Twilio uses CAs that are approved by Mozilla, you can find the full list here.
For testing purposes you can disable SSL Certificate Validation in Console.
If I disable SSL Certificate Validation in Console as suggested by Twilio, the webhooks work. I do not want to disable SSL Certificate Validation.
Here is a self contained sample of the code I am running on the server:
import sys
from klein import Klein
from twisted.web.server import Site
from twisted.internet import reactor
from twisted.internet.endpoints import serverFromString
from twisted.python.log import startLogging
from [redacted] import get_data_folder_location
startLogging(sys.stdout)
klein_app = Klein()
path_to_letsencrypt_keys = get_data_folder_location()
#lensencrypt keys have been copied locally from /etc/letsencrypt/live/domain and chowned from root to local group:user
endpoint_description = "ssl:443:privateKey={0}/privkey.pem:certKey={0}/fullchain.pem".format(path_to_letsencrypt_keys)
klein_resource = klein_app.resource()
serverFromString(reactor, endpoint_description).listen(Site(klein_resource))
reactor.run()
Here is the log output from the self contained sample:
Note: the 404 on the last line of the log is me hitting the site over ssl with FireFox, which demonstrates FireFox (and therefore Mozilla) is Ok with the letsencrypt ssl cert
2021-04-26 17:54:58+0000 [-] Log opened.
2021-04-26 17:54:58+0000 [-] Site (TLS) starting on 443
2021-04-26 17:54:58+0000 [-] Starting factory <twisted.web.server.Site object at 0x7fe3c57aa048>
2021-04-26 17:55:18+0000 [-] "redacted" - - [26/Apr/2021:17:55:18 +0000] "GET / HTTP/1.1" 404 233 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:86.0) Gecko/20100101 Firefox/86.0"
And finally, here are 2 screen shots of the Qualys SSL Report
My question: How can I get Twilio to accept my LetsEncrypt cert?
It looks like Twisted has problems loading the fullchain.pem.
You'll need to manually load the chain as described here.
from OpenSSL import crypto
from twisted.internet import ssl
privkey=open('{0}/privkey.pem'.format(path_to_letsencrypt_keys), 'rt').read()
certif=open('{0}/cert.pem'.format(path_to_letsencrypt_keys), 'rt').read()
chain=open('{0}/chain.pem'.format(path_to_letsencrypt_keys), 'rt').read()
privkeypyssl=crypto.load_privatekey(crypto.FILETYPE_PEM, privkey)
certifpyssl=crypto.load_certificate(crypto.FILETYPE_PEM, certif)
chainpyssl=[crypto.load_certificate(crypto.FILETYPE_PEM, chain)]
contextFactory=ssl.CertificateOptions(privateKey=privkeypyssl, certificate=certifpyssl, extraCertChain=chainpyssl)
I am using python3 ssl to connect via web sockets to an nginx server. According to my code below, I should be connecting via TLSv1_1.3 since I not using 1, 1.1, or 1.2. My ssl (OpenSSL 1.1.1h 22 Sep 2020) supports TLSv1.3.
sslCon=None
if self.server.startswith("wss"):
sslCon=ssl.SSLContext(ssl.PROTOCOL_TLS)
sslCon.options |= (
ssl.OP_NO_TLSv1 | ssl.OP_NO_TLSv1_1 | ssl.OP_NO_TLSv1_2
)
self.conn = await websockets.connect(self.server, ssl=sslCon)
My NGINX config specifies TLSv1.3, but when I observer the connection over Wireshark, I see the following. Any ideas to how to diagnose this further?
Frame 3172: 308 bytes on wire (2464 bits), 308 bytes captured (2464 bits) on interface en0, id 0
Ethernet II, Src: Apple_62:32:d8 (XX:XX:XX:XX:XX:XX), Dst: Cisco_9f:f2:8f (00:00:0c:9f:f2:8f)
Internet Protocol Version 4, Src: XXX.XXX.XXX.XXX, Dst: XXX.XXX.XXX.XXX
Transmission Control Protocol, Src Port: 62035, Dst Port: 8189, Seq: 1, Ack: 1, Len: 242
Transport Layer Security
TLSv1 Record Layer: Handshake Protocol: Client Hello
Content Type: Handshake (22)
Version: TLS 1.0 (0x0301)
Length: 237
Handshake Protocol: Client Hello
Handshake Type: Client Hello (1)
Length: 233
Version: TLS 1.2 (0x0303)
Random: dccd34397b86bac156d3ae39483a268ed3536ef09a3557c3…
Session ID Length: 32
Session ID: ea577122c909b7c78e20dbb5f982a7be94169fac8f51886f…
Cipher Suites Length: 8
Cipher Suites (4 suites)
Compression Methods Length: 1
Compression Methods (1 method)
Extensions Length: 152
Extension: server_name (len=23)
Extension: ec_point_formats (len=4)
Extension: supported_groups (len=12)
Extension: session_ticket (len=0)
Extension: encrypt_then_mac (len=0)
Extension: extended_master_secret (len=0)
Extension: signature_algorithms (len=30)
Extension: supported_versions (len=3)
Extension: psk_key_exchange_modes (len=2)
Extension: key_share (len=38)
Extension: supported_versions (len=3)
You'll find the support for TLS 1.3 announced inside this extension. This is Perfectly normal, i.e. this is how TLS 1.3 works. The way with not announcing the support for TLS 1.3 directly in the ClientHello version but instead in the extension was done in order to not confuse broken TLS stacks.
How would I diagnose an ssl.WRONG_VERSION_NUMBER error in python3
First by better understanding how TLS works :)
Then by not assuming that the error message means what it says :(
Unfortunately this error message and similar strange ones often happen if one tries to connect to a server which does not speak TLS at all and when the non-TLS response of the server is just interpreted as TLS. Such situations typically happen if the wrong port is used, if the server is misconfigured or if one assumes that the protocol should start with TLS but it does not (like in case with SMTP etc which have some plain TCP initial dialog first). Thus, look at the actual data coming from the server to see if they even look like TLS.
It can also happen that due to misconfiguration or misunderstanding the server does not support the specific TLS version even if you are sure that it should. Therefore look for information in the logs the server writes, especially the error logs.
Anaconda - Python 3.6
OpenSSL 1.0.2
Operating System: Windows 7
Phase 1 (Completed): Using selenium: launched, navigated, and extracted various data elements including a table from site. Extracted Hyperlinks contained in table that are direct links to documents.
Phase 2: Taking extracted hyperlink from table I need to download the files to a specified folder on the shared drive.
Tried:
import urllib.request
url = 'tts website/test.doc'
urllib.request.urlretrieve(url,'C:\Users\User\Desktop\')
Error I get is sslv3 alert handshake failure
With the site opened, I have clicked on the Lock icon and clicked "Install Certificate". I have saved the certificate to my "Trusted Root Certification Authorities" in the Certificate store.
I can see the certificate name (when i installed certificate) from the above step in the 58 CA Certificates shown by running the following code:
import socket
import ssl
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
context.verify_mode = ssl.CERT_REQUIRED
context.load_default_certs()
ca_certs = context.get_ca_certs()
print('{} CA Certificates loaded: ' .format(len(ca_certs)))
for cert_dict in ca_certs:
print(cert_dict['subject'])
print()
I can't figure out how to secure a SSL connection to the website/server in order to download the file from each of the hyperlinks?? This website uses Single Sign On(SSO) and automatically logins me in when I first launch the website.
I have tried to use server server.net 443 to connect to server, but can't seem to get the scripting right to connect and retrieve the document.
I have connected directly to the server and abstracted the certificate details shown here:
HOST, PORT = server.net, 443
ctx = ssl.create_default_context()
s = ctx.wrap_socket(socket.socket(), server_hostname=HOST)
c.connect((HOST, PORT))
cert = s.getpeercert()
print(cert)
When i run urlretrieve i am still getting the same error: handshake. When reviewing my ca certificates i see there is a Personal certificate for my Windows login (username) listed there, that must be how it is automatically logging me in using SSO. How do i take all of this information, connect to the website using my SSO, and retrieve the documents?
Latest UPDATE:
I am finding pycurl to be promising, however I feel like I need a little assistance making a few tweaks to get it working.
import pycurl
fp = open('Test.doc','wb')
curl = pycurl.Curl()
curl.setopt(pycurl.URL, url) # see url link to go to word doc
curl.setopt(pycurl.FOLLOWLOCATION, 1)
curl.setopt(pycurl.MAXREDIRS, 5)
curl.setopt(pycurl.CONNECTTIMEOUT,30)
curl.setopt(pycurl.TIMEOUT, 300)
try:
curl.setopt(pycurl.WRITEDATA, fp)
curl.perform()
except:
import traceback
traceback.print_exc(file=sys.stderr)
sys.stderr.flush()
curl.close()
fp.close()
This code yields no error, however the created word doc contains an error displaying a print screen of the log on page of the website.
Main Problem: HTTPS connection using Single Signon connection behind a corporate network proxy server.
I have been trying to get this to work to validate cacert, but I have been getting this error message now:
curl.setopt(pycurl.SSL_VERIFYPEER, 1)
curl.setopt(pycurl.SSL_VERIFYPEER, 2)
curl.setopt(pycurl.CAINFO, certifi.where())
but now i am getting ERROR: 51, CERT_TRUST_IS_UNTRUSTED_ROOT
How do i add proxy if that is causing the error? and Secondly, how do i attach the ca certificate file directly?
I'm using sample codes from documentation and I'm trying to connect to server using Prosys OPC UA Client. I have tried opcua-commander and integration objects opc ua client and it looks like server works just fine.
Here's what is happening:
After entering endpointUrl, client adds to url -- urn:NodeOPCUA-Server-default.
Client asks to specify security settings.
Client asks to choose server - only 1 option and it's urn:NodeOPCUA-Server-default.
And it goes back to step 2 and 3 over and over.
If I just minimize prosys client without closing configuration after some time I get this info in terminal:
Server: closing SESSION new ProsysOpcUaClient Session15 because of timeout = 300000 has expired without a keep alive
\x1B[46mchannel = \x1B[49m ::ffff:10.10.13.2 port = 51824
I have tried this project and it works -> node-opcua-htmlpanel. What's missing in sample code then?
After opening debugger I have noticed that each Time I select security settings and hit OK, server_publish_engine reports:
server_publish_engine:179 Cencelling pending PublishRequest with statusCode BadSecureChannelClosed (0x80860000) length = 0
This is due to a specific interoperability issue that was introduced in node-opcua#0.2.2. this will be fixed in next version of node-opcua. The resolution can be tracked here https://github.com/node-opcua/node-opcua/issues/464
The issue has been handled at the Prosys OPC Forum:
The error happens because the server sends different
EndpointDescriptions in GetEndpointsResponse and
CreateSessionResponse.
In GetEndpoints, the returned EndpointDescriptions contain
TransportProfileUri=http://opcfoundation.org/UA-Profile/Transport/uatcp-uasc-uabinary.
In CreateSessionResponse, the corresponding TransportProfileUri is
empty.
In principle, the server application is not working according to
specification. The part 4 of the OPC UA specification states that “The
Server shall return a set of EndpointDescriptions available for the
serverUri specified in the request. … The Client shall verify this
list with the list from a DiscoveryEndpoint if it used a
DiscoveryEndpoint to fetch the EndpointDescriptions. It is recommended
that Servers only include the server.applicationUri, endpointUrl,
securityMode, securityPolicyUri, userIdentityTokens,
transportProfileUri and securityLevel with all other parameters set to
null. Only the recommended parameters shall be verified by the
client.”
User who has authorized TLS certificate only able to connect to Open-sip server from application (Android and iOS).
What we need to change in config file for only TLS connection to Open-sip server.
You can configure the TLS certificate information in opensips.cfg file
tls_certificate="/usr/local/etc/opensips/tls/glob/glob-cert.pem"
tls_private_key="/usr/local/etc/opensips/tls/glob/glob-privkey.pem"
tls_ca_list="/usr/local/etc/opensips/tls/glob/glob-calist.pem"
## turn on the strictest and strongest authentication possible
tls_verify_client = 1
tls_require_client_certificate = 1
tls_method = TLSv1
tls_verify_client = 1 will ensure the client with authorized certificate configured in tls_ca_list file
Can you try uncommenting the line of startTLS from config file and make it true as a value?
It should work!
Also make sure that your Android and iOS clients are configured to accept TLS connections(though most of the time it's default behaviour).