I have a Dataframe, which has a bunch of ID name pairs in it. I create it by doing the following:
market_df = pd.DataFrame(markets_info['markets'])
market_df.astype(dict(id=int, name=str))
I received ID numbers from a process and I need to grab the associated name to that ID. I have tried creating an index on the ID and then parsing it, but that doesn't seem to set the ID correctly.
I now am trying to do the following: exch_name = MARKET_IDS.loc[MARKET_IDS['id'] == exchange_id, 'name']
I have verified that exchange_id is also of type int.
What am I missing here?
I don't know if this is because you left out some crucial information from this, but from what it sounds like in your post you're not really altering market_df at all, as your second line is not an assignment. It should read market_df = market_df.astype(dict(id=int, name=str))
Related
I am new to redis and created an index and am attempting to ingest Zeek Logging Data, create an index for multiple fields, and then search fields in that index. For the life of me, I cannot get any values to return when searching for the #orig_ip name or using JSON.GET to retried any id.* related fields.
UPDATE: I figured this out after more troubleshooting and am updating here to help anyone else struggling with this problem.
Here is my WRONG code for creating the index:
# Options for index creation
index_def = IndexDefinition(
index_type=IndexType.JSON,
prefix = ['uid:'],
score = 0.5,
score_field = 'doc_score'
)
# Schema definition
schema = (
TagField('$.orig_l2_addr', as_name='orig_mac'),
TagField('$.id.orig_h', as_name='orig_ip'), #Wrong field path
TagField('$.id.resp_h', as_name='resp_ip'), #Wrong field path
NumericField('$.orig_bytes', as_name='orig_bytes'),
NumericField('$.resp_bytes', as_name='resp_bytes'),
NumericField('$.ts', as_name='timestamp')
)
r.ft('py_conn_idx').create_index(schema, definition = index_def)
Here is the result I kept getting with the above WRONG schema (no results)
search_result4 = r.ft('py_conn_idx').search(Query('#orig_ip:{192\.168\.210\.27}'))
Results for "#orig_ip:{192\.168\.210\.27}":
0
UPDATE: Working schema definition:
So it turns out even though Zeek is only using the . in field names vice using it to create an object, but the . in the field names was the culprit in my query failures. I needed to access the fields for the index as follows:
# Schema definition
schema = (
TagField('$.orig_l2_addr', as_name='orig_mac'),
TagField('$.["id.orig_h"]', as_name='orig_ip'), #Fixed field reference
TagField('$.["id.resp_h"]', as_name='resp_ip'), #Fixed field reference
NumericField('$.orig_bytes', as_name='orig_bytes'),
NumericField('$.resp_bytes', as_name='resp_bytes'),
NumericField('$.ts', as_name='timestamp')
)
After recreating the index with this schema, I get results with my query:
Results for "#orig_ip:{192\.168\.210\.27}":
Document {'id': 'uid:CPvYfTI4Zb1Afp2l5',....
Thanks to this stackoverflow question for finally walking me to the cause of my troubles: How to get objects value if its name contains dots?
Putting this answer here so this question gets marked as having one. See the updated question/code above!
I would like to write small module that inherit sale.order.line and adding reserved lot on sale order line tree view.
under the sale.order.line module, there is the field name move_ids (stock.move , one2many) fields. I would like to create move_ids.move_line_ids.lot_id
So I tried the following code:
lot_name = fields.Char(related="move_ids.move_line_ids.lot_id", string="String")
but no luck and saw internal server error.
jooze
You can not do that, type of related field lot_name is inconsistent with lot_id.
Try to use a computed Many2many field to get all lot ids related to the current record.
Edit:
class SaleOrderLine(models.Model):
_inherit = 'sale.order.line'
lot_ids = fields.Many2many('stock.production.lot', compute='_get_lot_ids')
#api.one
#api.depends('move_ids.move_line_ids.lot_id')
def _get_lot_ids(self):
# returns the union of all lots, with duplicates removed
self.lot_ids = self.mapped('move_ids.move_line_ids.lot_id')
I have an API request I'm writing to query OpenWeatherMap's API to get weather data. I am using a city_id number to submit a request for a unique place in the world. A successful API query looks like this:
r = requests.get('http://api.openweathermap.org/data/2.5/group?APPID=333de4e909a5ffe9bfa46f0f89cad105&id=4456703&units=imperial')
The key part of this is 4456703, which is a unique city_ID
I want the user to choose a few cities, which then I'll look through a JSON file for the city_ID, then supply the city_ID to the API request.
I can add multiple city_ID's by hard coding. I can also add city_IDs as variables. But what I can't figure out is if users choose a random number of cities (could be up to 20), how can I insert this into the API request. I've tried adding lists and tuples via several iterations of something like...
#assume the user already chose 3 cities, city_ids are below
city_ids = [763942, 539671, 334596]
r = requests.get(f'http://api.openweathermap.org/data/2.5/groupAPPID=333de4e909a5ffe9bfa46f0f89cad105&id={city_ids}&units=imperial')
Maybe a list is not the right data type to use?
Successful code would look something like...
r = requests.get(f'http://api.openweathermap.org/data/2.5/group?APPID=333de4e909a5ffe9bfa46f0f89cad105&id={city_id1},{city_id2},{city_id3}&units=imperial')
Except, as I stated previously, the user could choose 3 cities or 10 so that part would have to be updated dynamically.
you can use some string methods and list comprehensions to append all the variables of a list to single string and format that to the API string as following:
city_ids_list = [763942, 539671, 334596]
city_ids_string = ','.join([str(city) for city in city_ids_list]) # Would output "763942,539671,334596"
r = requests.get('http://api.openweathermap.org/data/2.5/group?APPID=333de4e909a5ffe9bfa46f0f89cad105&id={city_ids}&units=imperial'.format(city_ids=city_ids_string))
hope it helps,
good luck
I'm using Python Pandas Dataframe for Data Analyse of some logs.
I have a csv with something like:
number_items event_type ... ... ... session_id ... ... ...
My problem is that in my session there are different types of events, and only one of them has something for number_items. Or, numbers_items is what interests me.
So what I want to see is how each parameter of each event influences the number_items.
So, what I want to do is:
Copy the number_items of the event that has it (always the last one in the session) to all the other events of the session. Separate each event_type in a different Dataframe (to avoid a lot of nulls that exist only because the attribute doesn't correspond to the event) and analyse it.
I'm blocked at the first part
I tried something like this:
currentSession = '0'
currentItems = 0
for index, row in reversed(df.iterrows()) :
if row['session_id'] == currentSession :
row['number_items'] = currentItems
else :
currentSession = row['session_id']
currentItems = row['number_items']
Obviously, it's not working, I just wanted to show the idea.
I'm kind of new in Python, so I would appreciate some help.
Thanks
edit: data sample here
For security reasons, I let only the relevant information
The rows you get back from iterrows are copies so they dont overwrite your original dataframe. Use another form of iterator that references the original dataframe.
see here Updating value in iterrow for pandas
(also im note entirely sure what it is you are trying to do but instinctively it seems very inefficient - i suspect there are natural pandas methods which might do what you trying to achieve in one or two lines, look up the where() method)
I've got a problem that I don't know how to solve, I've tried many solutions but always getting that Operational error: near...
def insert_medicine_to_table():
con = sqlite3.connect('med_db3.db')
cur = con.cursor()
table_name = 'medicines'
column_name = "présentation"
value = 'Boîte de 2 seringues pré-remplies'
cur.execute("INSERT INTO medicines {} VALUES (?)".format(column_name), value)
con.commit()
con.close()
sqlite3.OperationalError: near "présentation": syntax error
The goal here is that either the script or python has to recognize the field (column name) and insert the value into "that" field, like the following:
fields = ['présentation', 'princeps', 'distributeur_ou_fabriquant', 'composition', 'famille', 'code_atc', 'ppv', 'prix_hospitalier', 'remboursement', 'base_de_remboursement__ppv', 'nature_du_produit']
values = ['Boîte de 2 seringues pré-remplies', 'Oui', 'SANOFI', 'Héparine', 'Anticoagulant héparinique', 'B01AB01', '43.80', '27.40', 'Oui', '43.80', 'Médicament']
That is one entry in the database. The problem here is that other entries can or not have one or more values for some field, and also the fields are not presented in the same order in other entries.
It has to recognize each field in the database table and insert each value into the right column.
The problem causing your error is that your SQL isn't valid. The statement you are trying to execute is:
INSERT INTO medicines présentation VALUES (?)
The statement you want to execute is:
INSERT INTO medicines ("présentation") VALUES (?)
As far as your larger question is concerned, if you create both the list of columns ("présentation") and list of parameter markers (?) and build the query using them, you're most of the way there.
If a field can have multiple values supplied for each "entry" in your database, you may need to change your database design to handle that. You'll at least need to figure out how you want to handle the situation, but that would be a matter for a different question.