Applying Lambda to Recode (tricky) Strings to Numbers - string

Question

I have a large data set of NFL scenarios, but for the sake of illustration, let me just reduce it to a list of 2 observations. Like this:
data = [[scenario1],[scenario2]]
Here is what the data set consists of:
data[0][0]
>>"It is second down and 3. The ball is on your opponent's 5 yardline. There is 3 seconds left in the fourth quarter. You are down by 3 points."
data[1][0]
>>"It is first down and 10. The ball is on your 20 yardline. There is 7 minutes left in the third quarter. You are down by 10 points."
I can't build any models with the data in string format like this. So I want to recode these scenarios into new columns (or features if you will) as quantitative values. I thought I should first get the data frame squared away:
down = 0
yards = 0
yardline = 0
seconds = 0
quarter = 0
points = 0
data = [[scenario1, down, yards, yardline, seconds, quarter, points], [scenario2, yards, yardline, seconds, quarter, points]]
Now is the tricky part, some how I have to populate the new columns from the information from the scenario column. Tricky, because for instance, in the 2nd sentence if the word "opponent's" is present, that means we must calculate it as 100- whatever the yardline number is. In the above scenario1 variable, it should be 100-5=95.
At first I thought I should just separate all the numbers and throw away the words, but as pointed out above, some words are actually necessary to correctly assign the quantitative value. I have never made a lambda with this much subtlety. Or perhaps, a lambda is not the right way to go? I'm open to any/all suggestions.
For reinforcement, here is what I want to see (from scenario1 if I entered:
data[0][1:]
>>2,3,95,3,4,-3
Thank you

Answer 1

A lambda is not the way you're gonna want to go here. Python's re module is your friend :)
from re import search
def getScenarioData(scenario):
data = []
ordinals_to_nums = {'first':1, 'second':2, 'third':3, 'fourth':4}
numerals_to_nums = {
'zero':0, 'one':1, 'two':2, 'three':3, 'four':4,
'five':5, 'six':6, 'seven':7, 'eight':8, 'nine':9
}
# Downs
match = search('(first|second|third|fourth) down and', scenario)
if match:
raw_downs = match.group(1)
downs = ordinals_to_nums[raw_downs]
data.append(downs)
# Yards
match = search('down and (\S+)\.', scenario)
if match:
raw_yards = match.group(1)
data.append(int(raw_yards))
# Yardline
match = search("(oponent's)? (\S+) yardline", scenario)
if match:
raw_yardline = match.groups()
yardline = 100-int(raw_yardline[1]) if raw_yardline[0] else int(raw_yardline[1])
data.append(yardline)
# Seconds
match = search('(\S+) (seconds|minutes) left', scenario)
if match:
raw_secs = match.groups()
multiplier = 1 if raw_secs[1] == 'seconds' else 60
data.append(int(raw_secs[0]) * multiplier)
# Quarter
match = search('(\S+) quarter', scenario)
if match:
raw_quarter = match.group(1)
quarter = ordinals_to_nums[raw_quarter]
data.append(quarter)
# Points
match = search('(up|down) by (\S+) points', scenario)
if match:
raw_points = match.groups()
if raw_points:
polarity = 1 if raw_points[0] == 'up' else -1
points = int(raw_points[1]) * polarity
else:
points = 0
data.append(points)
return data
Personally, I find storing your data like [[scenario, <scenario_data>], ...] is a bit odd, but to add the data to each scenario:
for s in data:
s.extend(getScenarioData(s[0]))
I would suggest using a list of dictionaries because using indexes like data[0][3] could get confusing a month or two from now:
def getScenarioData(scenario):
# instead of data = []
data = {'scenario':scenario}
# instead of data.append(downs)
data['downs'] = downs
...
scenarios = ['...', '...']
data = [getScenarioData(s) for s in scenarios]
EDIT: When you want to get a value from the dicts, use the get method to prevent raising a KeyError because get defaults to None if the key is not found:
for s in data:
print(s.get('quarter'))