first post / total Python novice so be patient with my slow understanding!
I have a dataframe containing a list of transactions by order of transaction date.
I've appended an additional new field/column called ["DB/CR"], that dependant on the presence of "-" in the ["Amount"] field populates 'Debit', else 'Credit' in the absence of "-".
Noting the transactions are in date order, I've included another new field/column called [Top x]. The output of which is I want to populate and incremental independent number (starting at 1) for both debits and credits on a segregated basis.
As such, I have created a simple loop with a associated 'if' / 'elif' (prob could use else as it's binary) statement that loops through the data sent row 0 to the last row in the df and using an if statement 1) "Debit" or 2) "Credit" increments the number for each independently by "Debit" 'i' integer, and "Credit" 'ii' integer.
The code works as expected in terms of output of the 'Top x'; however, I always receive a warning "A value is trying to be set on a copy of a slice from a DataFrame".
Trying to perfect my script, without any warnings I've been trying to understand what I'm doing incorrect but not getting it in terms of my use case scenario.
Appreciate if someone can kindly shed light on / propose how the code needs to be refactored to avoid receiving this error.
Code (the df source data is an imported csv):
#top x debits/credits
i = 0
ii = 0
for ind in df.index:
if df["DB/CR"][ind] == "Debit":
i = i+1
df["Top x"][ind] = i
elif df["DB/CR"][ind] == "Credit":
ii = ii+1
df["Top x"][ind] = ii
Interpreter
df["Top x"][ind] = i
G:\Finances Backup\venv\Statementsv.03.py:173: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df["Top x"][ind] = ii
Many thanks :)
You should use df.loc["DB/CR", ind] = "Debit"
Use iterrows() to iterate over the DF. However, updating DF while iterating is not preferable
see documentation here
Refer to the documentation here Iterrows()
You should never modify something you are iterating over. This is not
guaranteed to work in all cases. Depending on the data types, the
iterator returns a copy and not a view, and writing to it will have no
effect.
I'm very new in using Cognos report studio and trying to filter some of the values and replace them into others.
I currently have values that are coming out as blanks and want to replace them as string "Property Claims"
what i'm trying to use in my main query is
CASE WHEN [Portfolio] is null
then 'Property Claims'
ELSE [Portfolio]
which is giving me an error. Also have a different filter i want to put in to replace windscreen flags to a string value rather than a number. For example if the flag is 1 i want to place it as 'Windscreen Claims'.
if [Claim Windscreen Flag] = 1
then ('Windscreen')
Else [Claim Windscreen Flag]
None of this works with the same error....can someone give me a hand?
Your first CASE statement is missing the END. The error message should be pretty clear. But there is a simpler way to do that:
coalesce([Portfolio], 'Property Claims')
The second problem is similar: Your IF...THEN...ELSE statement is missing a bunch of parentheses. But after correcting that you may have problems with incompatible data types. You may need to cast the numbers to strings:
case
when [Claim Windscreen Flag] = 1 then ('Windscreen')
else cast([Claim Windscreen Flag], varchar(50))
end
In future, please include the error messages.
it might be syntax
IS NULL (instead of = null)
NULL is not blank. You might also want = ' '
case might need an else and END at the bottom
referring to a data type as something else can cause errors. For example a numeric like [Sales] = 'Jane Doe'
For example (assuming the result is a string and data item 2 is also a string),
case
when([data item 1] IS NULL)Then('X')
when([data item 1] = ' ')Then('X')
else([data item 2])
end
Also, if you want to show a data item as a different type, you can use CAST
I have data in a csv file as follows:
60,27702,1938470,13935,18513,8
60,32424,1933740,16103,15082,11
60,20080,1946092,9335,14970,2
60,28236,1937936,13799,16871,6
60,22717,1943455,10809,16726,4
120,37702,2938470,23935,28513,8
120,42424,2933740,26103,25082,11
120,30080,2946092,2335,24970,2
120,38236,2937936,23799,26871,6
120,32717,2943455,20809,26726,4
180,47702,3938470,33935,8513,8
180,52424,3933740,36103,5082,11
180,40080,3946092,3335,4970,2
180,48236,3937936,33799,6871,6
180,42717,3943455,30809,6726,4
I then used the following code to insert column heading:
df = pd.read_csv("contikiMAC_new_out.csv", names=['Energest','CPU','LPM','Transmit','Listen','ID'])
I used df.groupby(['ID']) to see the data in group according to column 'ID'.
The problem is the data in column 'LPM' gets reset after some time so I would like to add the previous value with the new value whenever the new value in LPM column is smaller for specific 'ID' .
I tried doing :
for x in df.groupby(['ID']):
for i in df.ID:
if (df.loc[i, 'LPM'] < df.loc[i - 1, 'LPM']):
df.loc[i, 'LPM'] = df.loc[i, 'LPM'] + df.loc[i - 1, 'LPM']
But actually not getting the fruitful result I desire because it mixes with the 'LPM' value of different 'ID' and the process takes a long time. Can anyone please help me in suggesting a way to write the data group wise in a csv file based on 'ID' after performing the sum operation ?
The data structure I like to see is as follows:
60,27702,1938470,13935,18513,8
120,37702,2938470,23935,28513,8
180,47702,3938470,33935,37026,8
60,32424,1933740,16103,15082,11
120,42424,2933740,26103,25082,11
180,52424,3933740,36103,30164,11
60,20080,1946092,9335,14970,2
120,30080,2946092,2335,24970,2
180,40080,3946092,3335,29940,2
60,28236,1937936,13799,16871,6
120,38236,2937936,23799,26871,6
180,48236,3937936,33799,33742,6
60,22717,1943455,10809,16726,4
120,32717,2943455,20809,26726,4
180,42717,3943455,30809,33452,4
If I understood your problem correctly, DataFrame.shift is what you're looking for.
Something like:
df['LPM_prev'] = df.groupby(['ID'])['LPM'].shift(1)
And then you can work with that column
I have created a structure containing a few different fields. The fields contain data from a number of different subjects/participants.
At the beginning of the script I prompt the user to enter the "Subject number" like so:
prompt='Enter the subject number in the format SUB_n: ';
SUB=input(prompt,'s');
Example SUB_34 for the 34th subject.
I want to then name my structure such that it contains this string... i.e. I want the name of my structure to be SUB_34, e.g. SUB_34.field1. But I don't know how to do this.
I know that you can assign strings to a specific field name for example for structure S if I want field1 to be called z then
S=struct;
field1='z';
S.(field1);
works but it does not work for the structure name.
Can anyone help?
Thanks
Rather than creating structures named SUB_34 I would strongly recommend just using an array of structures instead and having the user simply input the subject number.
number = input('Subject Number')
S(number) = data_struct
Then you could simply find it again using:
subject = S(number);
If you really insist on it, you could use the method proposed in the comment by #Sembei using eval to get the struct. You really should not do this though
S = eval([SUB, ';']);
Or to set the structure
eval([SUB, ' = mydata;']);
One (of many) reasons not to do this is that I could enter the following at your prompt:
>> prompt = 'Enter the subject number in the format SUB_n: ';
>> SUB = input(prompt, 's');
>> eval([SUB, ' = mydata;']);
And I enter:
clear all; SUB_34
This would have the unforeseen consequence that it would remove all of your data since eval evaluates the input string as a command. Using eval on user input assumes that the user is never going to ever write something malformed or malicious, accidentally or otherwise.
In my Lotus Notes script, i do have this piece of logic as shown below. This is basically for two SELECT statements, followed by Fetch for each of the SELECT statement separately and the SELECT is for the same DB2 table with a variation in WHERE clause. The error i'm getting is for the second FETCH. The error i'm getting is ---> Field count mismatch error:
count = 0
If (srccon.Execute(selectstatement, fldLst1) = 0) Then
Goto GetNextDoc
End If
count = srccon.Fetch(fldLst1)
If ( count = 0 ) Then
Goto GetNextDoc
End If
The above cursor select and fetch does not give me any error.
The cursor down which is for the same DB2 table with a slight variation
in WHERE clause is causing the error:
count1 = 0
If (srccon.Execute(selectstatement1, fldLst) = 0) Then
Goto GetNextDoc
End If
count1 = srccon.Fetch(fldLst) ---> The error is pointing to this line
and the error is
I would appreciate any help in this regard. I would also thank the gentleman who
did excellent solution for my previous problem for current date minus 30 days.
With much thanks
I suspect it's because when you call Execute, you're reusing the same LCFieldList object from a previous call. Execute and Select statements append their list of result fields to the object you pass them, so you must pass them an empty fieldlist -- one you just created. Otherwise you get a combined fieldlist of all the fields in the result sets of multiple calls to Select or Execute.
You may find the LotusScript chapter of this Redbook useful.