Dask groupby unique as a frame - how to? - python-3.x

Question

I have few dataframes:
import pandas as pd
import numpy as np
router = pd.DataFrame([
['2018-01-01 00:00:00', '1', 5],
['2018-01-01 00:30:00', '1', 7],
['2018-01-01 01:00:00', '1', 25],
['2018-01-01 01:30:00', '1', 3],
['2018-01-01 00:00:00', '2', 25],
['2018-01-01 00:30:00', '2', 7],
['2018-01-01 01:00:00', '2', 25],
['2018-01-01 01:30:00', '2', 35],
], columns=['time', 'cust_id', 'errors'])
router
devices = pd.DataFrame([
['2018-01-01 00:00:00', '1', 'dev_1'],
['2018-01-01 00:30:00', '1', 'dev_1'],
['2018-01-01 00:30:00', '1', 'dev_2'],
['2018-01-01 01:00:00', '1', 'dev_1'],
['2018-01-01 01:00:00', '1', 'dev_2'],
['2018-01-01 01:00:00', '1', 'dev_3'],
['2018-01-01 01:30:00', '1', 'dev_2'],
['2018-01-01 00:00:00', '2', 'dev_1'],
['2018-01-01 00:00:00', '2', 'dev_2'],
['2018-01-01 00:30:00', '2', 'dev_1'],
['2018-01-01 01:00:00', '2', 'dev_2'],
['2018-01-01 01:00:00', '2', 'dev_3'],
['2018-01-01 01:30:00', '2', 'dev_2'],
['2018-01-01 01:30:00', '2', 'dev_4'],
], columns=['time', 'cust_id', 'device_id'])
devices
By using pandas, I can group by and calculate unique devices:
devices_per_time = devices.groupby(['cust_id', 'time'])['device_id'].unique().to_frame()
devices_per_time
I tried to do the same with dask:
I have the following questions:
how come that I cannot use devices.groupby(['cust_id', 'time'])['device_id'].unique()?
I manage to get the result, but I am not sure whether it is the optimal one. Can someone confirm that I am using dask in the proper way?
Regards.

Answer 1

You cannot do .unique() because it is still not implemented yet for dask series. Check the available functions: SeriesGroupby
Here's another way of getting the result using parallel apply and set:
(devices
.groupby(['time','cust_id'])['device_id']
.apply(set, meta=object)
.apply(list,meta=object)
.compute()
.reset_index())
If you don't care about the final type (set or list), you can remove the .apply(list,meta=object)