I'm new to Openstack and I'm trying to create a tool so that I can launch any number of instances in an Openstack cloud. This was easily done using the nova-client module of openstacksdk.
Now the problem is that I want to make the instances execute a bash script as they are created by adding it as a userdata file, but it doesn't execute. This is confusing because I don't any error or warning message. Does anyone know what could it be?
Important parts of the code
The most important parts of the Python program are the function which gets the cloud info, the one that creates the instances and the main function, . I'll post them here as #Corey told.
"""
Function that allow us to log at cloud with all the credentials needed.
Username and password are not read from env.
"""
def get_nova_credentials_v2():
d = {}
user = ""
password = ""
print("Logging in...")
user = input("Username: ")
password = getpass.getpass(prompt="Password: ", stream=None)
while (user == "" or password == ""):
print("User or password field is empty")
user = input("Username: ")
password = getpass.getpass(prompt="Password: ", stream=None)
d['version'] = '2.65'
d['username'] = user
d['password'] = password
d['project_id'] = os.environ['OS_PROJECT_ID']
d['auth_url'] = os.environ['OS_AUTH_URL']
d['user_domain_name'] = os.environ['OS_USER_DOMAIN_NAME']
return d
Then we have the create_server function:
"""
This function creates a server using the info we got from JSON file
"""
def create_server(server):
s = {}
print("Creating "+server['compulsory']['name']+"...")
s['name'] = server['compulsory']['name']
s['image'] = server['compulsory']['os']
s['flavor'] = server['compulsory']['flavor']
s['min_count'] = server['compulsory']['copyNumber']
s['max_count'] = server['compulsory']['copyNumber']
s['userdata'] = server['file']
s['key_name'] = server['compulsory']['keyName']
s['availability_zone'] = server['compulsory']['availabilityZone']
s['nics'] = server['compulsory']['network']
print(s['userdata'])
if(exists("instalacion_k8s_docker.sh")):
print("Exists")
s['userdata'] = server['file']
nova.servers.create(**s)
And now the main function:
"""
Main process: First we create a connection to Openstack using our credentials.
Once connected we cal get_serverdata function to get all instance objects we want to be created.
We check that it is not empty and that we are not trying to create more instances than we are allowed.
Lastly we create the instances and the program finishes.
"""
credentials = get_nova_credentials_v2()
nova = client.Client(**credentials)
instances = get_serverdata()
current_instances = len(nova.servers.list())
if not instances:
print("No instance was writen. Check instances.json file.")
exit(3)
num = 0
for i in instances:
create_server(i)
exit(0)
For the rest of the code you can access to this public repo on github.
Thanks a lot!
Problem solved
The problem was the content of the server['file'] as #Corey said. It cannot be the Path to the file where you wrote the data but the content of it or a file type object. In the case of OpenstackSDK it must be base64 encoded but it is not the case in Novaclient.
Thanks a lot to #Corey for all the help! :)
I need to find out for a script I'm writing who is the true owner of a file in an smb share (mounted using mount -t cifs of course on my server and using net use through windows machines).
Turns out it is a real challenge finding this information out using python on a linux server.
I tried using many many smb libraries (such as smbprotocol, smbclient and others), nothing worked.
I find few solutions for windows, they all use pywin32 or another windows specific package.
And I also managed to do it from bash using smbcalcs but couldn't do it cleanly but using subprocess.popen('smbcacls')..
Any idea on how to solve it?
This was unbelievably not a trivial task, and unfortunately the answer isn't simple as I hoped it would be..
I'm posting this answer if someone will be stuck with this same problem in the future, but hope maybe someone would post a better solution earlier
In order to find the owner I used this library with its examples:
from smb.SMBConnection import SMBConnection
conn = SMBConnection(username='<username>', password='<password>', domain=<domain>', my_name='<some pc name>', remote_name='<server name>')
conn.connect('<server name>')
sec_att = conn.getSecurity('<share name>', r'\some\file\path')
owner_sid = sec_att.owner
The problem is that pysmb package will only give you the owner's SID and not his name.
In order to get his name you need to make an ldap query like in this answer (reposting the code):
from ldap3 import Server, Connection, ALL
from ldap3.utils.conv import escape_bytes
s = Server('my_server', get_info=ALL)
c = Connection(s, 'my_user', 'my_password')
c.bind()
binary_sid = b'....' # your sid must be in binary format
c.search('my_base', '(objectsid=' + escape_bytes(binary_sid) + ')', attributes=['objectsid', 'samaccountname'])
print(c.entries)
But of course nothing will be easy, it took me hours to find a way to convert a string SID to binary SID in python, and in the end this solved it:
# posting the needed functions and omitting the class part
def byte(strsid):
'''
Convert a SID into bytes
strdsid - SID to convert into bytes
'''
sid = str.split(strsid, '-')
ret = bytearray()
sid.remove('S')
for i in range(len(sid)):
sid[i] = int(sid[i])
sid.insert(1, len(sid)-2)
ret += longToByte(sid[0], size=1)
ret += longToByte(sid[1], size=1)
ret += longToByte(sid[2], False, 6)
for i in range(3, len(sid)):
ret += cls.longToByte(sid[i])
return ret
def byteToLong(byte, little_endian=True):
'''
Convert bytes into a Python integer
byte - bytes to convert
little_endian - True (default) or False for little or big endian
'''
if len(byte) > 8:
raise Exception('Bytes too long. Needs to be <= 8 or 64bit')
else:
if little_endian:
a = byte.ljust(8, b'\x00')
return struct.unpack('<q', a)[0]
else:
a = byte.rjust(8, b'\x00')
return struct.unpack('>q', a)[0]
... AND finally you have the full solution! enjoy :(
I'm adding this answer to let you know of the option of using smbprotocol; as well as expand in case of misunderstood terminology.
SMBProtocol Owner Info
It is possible to get the SID using the smbprotocol library as well (just like with the pysmb library).
This was brought up in the github issues section of the smbprotocol repo, along with an example of how to do it. The example provided is fantastic and works perfectly. An extremely stripped down version
However, this also just retrieves a SID and will need a secondary library to perform a lookup.
Here's a function to get the owner SID (just wrapped what's in the gist in a function. Including here in case the gist is deleted or lost for any reason).
import smbclient
from ldap3 import Server, Connection, ALL,NTLM,SUBTREE
def getFileOwner(smb: smbclient, conn: Connection, filePath: str):
from smbprotocol.file_info import InfoType
from smbprotocol.open import FilePipePrinterAccessMask,SMB2QueryInfoRequest, SMB2QueryInfoResponse
from smbprotocol.security_descriptor import SMB2CreateSDBuffer
class SecurityInfo:
# 100% just pulled from gist example
Owner = 0x00000001
Group = 0x00000002
Dacl = 0x00000004
Sacl = 0x00000008
Label = 0x00000010
Attribute = 0x00000020
Scope = 0x00000040
Backup = 0x00010000
def guid2hex(text_sid):
"""convert the text string SID to a hex encoded string"""
s = ['\\{:02X}'.format(ord(x)) for x in text_sid]
return ''.join(s)
def get_sd(fd, info):
""" Get the Security Descriptor for the opened file. """
query_req = SMB2QueryInfoRequest()
query_req['info_type'] = InfoType.SMB2_0_INFO_SECURITY
query_req['output_buffer_length'] = 65535
query_req['additional_information'] = info
query_req['file_id'] = fd.file_id
req = fd.connection.send(query_req, sid=fd.tree_connect.session.session_id, tid=fd.tree_connect.tree_connect_id)
resp = fd.connection.receive(req)
query_resp = SMB2QueryInfoResponse()
query_resp.unpack(resp['data'].get_value())
security_descriptor = SMB2CreateSDBuffer()
security_descriptor.unpack(query_resp['buffer'].get_value())
return security_descriptor
with smbclient.open_file(filePath, mode='rb', buffering=0,
desired_access=FilePipePrinterAccessMask.READ_CONTROL) as fd:
sd = get_sd(fd.fd, SecurityInfo.Owner | SecurityInfo.Dacl)
# returns SID
_sid = sd.get_owner()
try:
# Don't forget to convert the SID string-like object to a string
# or you get an error related to "0" not existing
sid = guid2hex(str(_sid))
except:
print(f"Failed to convert SID {_sid} to HEX")
raise
conn.search('DC=dell,DC=com',f"(&(objectSid={sid}))",SUBTREE)
# Will return an empty array if no results are found
return [res['dn'].split(",")[0].replace("CN=","") for res in conn.response if 'dn' in res]
to use:
# Client config is required if on linux, not if running on windows
smbclient.ClientConfig(username=username, password=password)
# Setup LDAP session
server = Server('mydomain.com',get_info=ALL,use_ssl = True)
# you can turn off raise_exceptions, or leave it out of the ldap connection
# but I prefer to know when there are issues vs. silently failing
conn = Connection(server, user="domain\username", password=password, raise_exceptions=True,authentication=NTLM)
conn.start_tls()
conn.open()
conn.bind()
# Run the check
fileCheck = r"\\shareserver.server.com\someNetworkShare\someFile.txt"
owner = getFileOwner(smbclient, conn, fileCheck)
# Unbind ldap session
# I'm not clear if this is 100% required, I don't THINK so
# but better safe than sorry
conn.unbind()
# Print results
print(owner)
Now, this isn't super efficient. It takes 6 seconds for me to run this one a SINGLE file. So if you wanted to run some kind of ownership scan, then you probably want to just write the program in C++ or some other low-level language instead of trying to use python. But for something quick and dirty this does work. You could also setup a threading pool and run batches. The piece that takes longest is connecting to the file itself, not running the ldap query, so if you can find a more efficient way to do that you'll be golden.
Terminology Warning, Owner != Creator/Author
Last note on this. Owner != File Author. Many domain environments, and in particular SMB shares, automatically alter ownership from the creator to a group. In my case the results of the above is:
What I was actually looking for was the creator of the file. File creator and modifier aren't attributes which windows keeps track of by default. An administrator can enable policies to audit file changes in a share, or auditing can be enabled on a file-by-file basis using the Security->Advanced->Auditing functionality for an individual file (which does nothing to help you determine the creator).
That being said, some applications store that information for themselves. For example, if you're looking for Excel this answer provides a method for which to get the creator of any xls or xlsx files (doesn't work for xlsb due to the binary nature of the files). Unfortunately few files store this kind of information. In my case I was hoping to get that info for tblu, pbix, and other reporting type files. However, they don't contain this information (which is good from a privacy perspective).
So in case anyone finds this answer trying to solve the same kind of thing I did - Your best bet (to get actual authorship information) is to work with your domain IT administrators to get auditing setup.
I want to access some charts -which I have saved in Looker- within Databricks. Part of this process is the authentication. I have one Looker auth-script which works but only pulls the tabular results into Databricks which corresponds to a Looker-View. Instead, I want ONLY the charts to be accessed in Databricks which will correspond to a Looker-look or Looker-space. However, when I follow the tutorial on https://discourse.looker.com/t/generating-a-powerpoint-presentation-from-all-looks-in-a-space/8191, I am not able to authenticate with their script. Hopefully, someone can help.
**Working auth-script for Looker-Views**
import looker_tools as tools
api=tools.LookerApi(
api_endpoint="abcd",
client_id=dbutils.secrets.get(scope="looker-api", key="looker_client_id"),
client_secret=dbutils.secrets.get(scope="looker-api",key="looker_client_secret")
)
token = api.login()
**Desired auth-script for Looker-Space/Looks as per tutorial link**
looker_instance = 'your-company.looker.com'
target_space = # 'Period over Period' Space on the Looker instance
client_id = 'xxxxxxxx'
client_secret = 'xxxxxxxx'
# instantiate Auth API
unauthenticated_client = looker_client.ApiClient(configuration=None)
unauthenticated_client.configuration.host = f'https://{looker_instance}:19999/api/3.0/'
unauthenticated_authApi = looker_client.ApiAuthApi(unauthenticated_client)
# authenticate client
token = unauthenticated_authApi.login(client_id=client_id, client_secret=client_secret)
client = looker_client.ApiClient(header_name='Authorization', header_value='token ' + token.access_token)
client.configuration.host = f'https://{looker_instance}:19999/api/3.0/'
I tried translating the code from Current to DESIRED auth-script but the error states the looker_client is not defined!
looker_instance = 'abcd'
target_space = 123
client_id = dbutils.secrets.get(scope="looker-api", key="looker_client_id")
client_secret = dbutils.secrets.get(scope="looker-api",key="looker_client_secret")
# instantiate Auth API
unauthenticated_client = looker_client.ApiClient(configuration=None) --> This line fails!!
unauthenticated_client.configuration.host = f'https://{looker_instance}:19999/api/3.0/'
unauthenticated_authApi = looker_client.ApiAuthApi(unauthenticated_client)
# authenticate client
token = unauthenticated_authApi.login(client_id=client_id, client_secret=client_secret)
client = looker_client.ApiClient(header_name='Authorization', header_value='token ' + token.access_token)
client.configuration.host = f'https://{looker_instance}:19999/api/3.0/'
I hope someone can help on how to define looker_client properly. Thanks.
It looks like this one was resolved here: https://discourse.looker.com/t/generating-a-powerpoint-presentation-from-all-looks-in-a-space/8191/15?u=izzy for those following along at home. There's another issue, but the NameError: name ‘looker_client’ is not defined error was resolved by adding a necessary import:
import looker_client_30 as looker_client
I am trying to start a client, but it gives a "database is closed" error.
What should I do?
from telethon import TelegramClient
api_id = 12345
api_hash = '##################'
phone_number = '+##########'
channel_username = 'tehrandb'
client = TelegramClient('session_name', api_id, api_hash)
assert client.connect()
if not client.is_user_authorized():
client.send_code_request(phone_number)
me = client.sign_in(phone_number, input('Enter code: '))
Just in case anyone else gets the same error. Database is closed means that you have an already running process using the same session file (which is session_name.session in the question). The same session file can only be used by 1 process at a time so you need to kill one of them.
if you are under Linux you can use fuser session_name.session to see the list of processes using that file.
I am working on a project that must use LDAP authentication. I am using the server at ldap.forumsys.com after finding the link on Stack Overflow to practice before adding to my Flask application.
If I run the ldapsearch bash command inside of my python code I get a whole bunch of usernames (Tesla etc...) and their associated data (there are no password hashes though). I am able to extract the usernames/user-data as shown here:
username = request.form['username']
password = request.form['password']
cmd = "ldapsearch -h ldap.forumsys.com -D cn=read-only-admin,dc=example,dc=com -w" + os.environ['LDAP_PWD'] + " -b dc=example,dc=com"
ldap_query = os.popen(cmd).read()
user_str = re.sub("\n", "", ldap_query)
users = user_str.split("#")
user_data = ""
for line in users:
if username in line:
user_data = line
break
But then I realized that I LDAP is not the same as a database. I was hoping to find password hashes that I could use to authenticate a user's login information.
So then I tried the python-ldap3 module:
>>> conn = Connection(server, 'uid=tesla,dc=example,dc=com', 'password', auto_bind=True)
>>> conn.bound
True
>>> conn.entries
[]
Unfortunately I can't seem to get any data returned in the list after calling conn.entries.
I can see that the ldap3 module binded the connection. Does the ldapsearch command bind as well? If there are no password hashes, how should I authenticate the username/password entered by the user on the client side?
Thank you all very much.
If the statement...
conn.bound == True
Then the connection has been authenticated via LDAP