What is the best way to store RSA connectors in Airflow Connectors?
hook = BaseHook.get_connection("my_rsa")
key = hook.extra
I am using this way. However, the key is stored as String. What is the best way to convert this to bytes?
It sounds like you're trying to secure RSA Keys... you should use airflow's crypto module
You can encrypt your keys and keep the encrypted key in an environment variable or in your own airflow.cfg
Do note that Airflow doesn't handle key rotation by default for you.
from the docs
Install crypto package pip install apache-airflow[crypto]
Generate fernet_key, using this code snippet below. fernet_key must be a base64-encoded 32-byte key:
from cryptography.fernet import Fernet
fernet_key=Fernet.generate_key()
print(fernet_key.decode()) # your fernet_key,keep it in secured place!
Replace airflow.cfg fernet_key value with the one from above. Alternatively, you can store your fernet_key in OS environment variable - You do not need to change airflow.cfg in this case as Airflow will use environment variable over the value in airflow.cfg:
#Note the double underscores
export AIRFLOW__CORE__FERNET_KEY=your_fernet_key
Related
What is the best method to hide credential in a Python script?
I would like to avoid storing it in clear text.
import paramiko
# how to avoid clear text ?
my_server='myserver'
my_password='mysecret'
ssh = paramiko.client.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
try:
ssh.connect(my_server, username='root', password=my_password)
except paramiko.SSHException:
print("Connection Failed")
quit()
If I create an myscript.exe with pyInstaller, is it possible to retrieve the clear text (my_server and my_password) "disassembling" the .exe?
Thanks
It doesn't matter if it's stored in clear text or not, if the script needs the password, everyone who has the script also has the password.
Store the credentials in a separate file (yaml comes to mind) and load that at runtime. Don't add the credentials file to the repository if every user uses their own credentials.
I am trying to perform auditing checks on Windows Registry file (.reg file, offline) and I am hoping that I can utilize Python to perform a check on the reg file.
For example (pseudo code):
#Configure registry policy processing: Do not apply during periodic background processing
testloc = "C:\\Users\\test.reg"
datafile = open(testloc, "r")
read = datafile.read()
find(Software\Policies\Microsoft\Windows\Group Policy\{35378EAC-683F-11D2-A89A-00C04FBBCFA2})
check(NoBackgroundPolicy) #check if dword value is correct
if(dword == correct):
print("correct")
else:
print("wrong")
I have tried looking at _winreg but it seems like it does a check on a live system using Windows API. Another issue is the large .reg file size(~200MB).
How can I perform such a check using Python?
I don't know if there's a lib that can read .reg files specifically, but from what it looks like it's just an INI file with an extra version information at the top.
Here's an example of how you could use the configparser module for that. Some notes:
.reg files are encoded in utf16
Before giving the file to the ConfigParser, a readline will skip the version info (something like Windows Registry Editor Version 5.00). Otherwise it will result in a MissingSectionHeaderError.
The names of values include quotation marks, which means you need to add them explicitly when looking up the value in the key.
import configparser
testloc = "C:\\Users\\test.reg"
regdata = configparser.ConfigParser()
with open(testloc, "r", encoding="utf-16") as f:
f.readline() # skip version info in first line
regdata.read_file(f)
key = regdata[r"HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\Group Policy\{35378EAC-683F-11D2-A89A-00C04FBBCFA2}"]
value = key['"NoBackgroundPolicy"']
print(value)
There may be drawbacks of doing it this way though, for example in how the values you obtain are formatted.
I'm working on a Python 3.7 script that eventually will be a CLI program like reg.exe is. I'm aiming to include the ability to add, delete and query keys and subkeys. At this point, I can create a new Key and iterate through all keys within the specific path however; once I try to write a value to the new key I made, I get a WinError 5 - Access denied.
Is there a way I can include in the script a way to have access to write to the registry?
I'm still a beginner with Python and programming, I've had a look at documents but I cant figure this one out.
Any help will be greatly appreciated. My code soo far:
import winreg
reg_connection = winreg.ConnectRegistry(None, winreg.HKEY_CURRENT_USER)
reg_key = winreg.OpenKey(reg_connection, r"SOFTWARE\Microsoft\\")
winreg.CreateKey(reg_key, "New Key")
for key in range(3000):
try:
show_sub_keys = winreg.EnumKey(reg_key, key)
print(show_sub_keys)
except WindosError:
break
new_key_value = winreg.OpenKey(reg_connection, r"SOFTWARE\Microsoft\New Key")
winreg.SetValueEx(new_key_value, "New Value",0,winreg.REG_SZ, "This Value")
winreg.CloseKey(new_key_value)
new_key_value = winreg.OpenKey(reg_connection, r"SOFTWARE\Microsoft\New Key")
Here you do not specify an argument for the optional access parameter, so the call passes the default value of KEY_READ. Hence you can only read from the key, but not write.
You should pass an argument for the access parameter, that specifies the permissions you need:
new_key_value = winreg.OpenKey(reg_connection, r"SOFTWARE\Microsoft\New Key", 0,
winreg.KEY_SET_VALUE)
For further details, see the winreg reference.
I need advice on how to get the md5 hash for a zip file. I will be constantly downloading files from an ftp using ftplib. As you know ftplib cannot tell if a file has been modified or not.
I want to use the md5 hash of each new file to tell if it has been modified or not by simply comparing the hashes after downloading the new file to tempdir. If the hashes are similar, I remove newly downloaded file. However, if hashes are different, newly downloaded file is kept, old hash is replaced with new hash and the script continues.
Please advice on how to achieve this. Are there any standalone modules for hashing md5 or similar.
Thanks.``
hope this is helpful
import hashlib
m=hashlib.md5();
m.update(open('yourzipfile.zip').read());
a=m.hexdigest()
print (a);
output
sh-4.3$ python3 1.py
f5c6a076bd116efbd4b1ce03c96eaf7a
Very simply, in python 3.8+, I use to keep the code as quick and compact as possible.
import hashlib
file_hash = hashlib.md5(open(old_file_path,'rb').read()).hexdigest()
print(file_hash)
I'm fairly new to python but have basic understanding of crypto modules. However, I'm trying to import an RSA key pair from a remote machine(i.e. Safenet) to run some performance tests on signing data. I'm building the framework in Python and I haven't found many clear examples. For instance:
from Crypto.PublicKey import RSA
f = open('path/to/pair/00000103000003A2','r')
r = RSA.importKey(f.read(), passphrase='123456') # Index out of range error
print(r)
This is about as far as I'm getting with opening the key pair. I can import the key pair to my personal computer so it's in a directory I have access to.
I have found the answer elsewhere.
from OpenSSL import crypto
passwd = 'The password of your created .p12 file'
p12 = crypto.load_pkcs12(open("Path to p12 file.p12 ",'rb').read(),passwd)
print(p12.get_certificate()) #Prints object location
print(p12.get_privatekey()) #Prints object location
p12.get_ca_certificates()
Here's a link to convert object location to PEM formatted strings. PyOpenSSL convert certificate object to .pem file