Get first and last item without using two joins - apache-spark

Question

Currently I have two dataset, one is parent, and one is child. Child dataset contain "parentId" column that can link to parent table. Child dataset hold data about actions of a person, and parent table hold data about person. I want to get a dataset contain person info and his first/last action.
Dataset look like this:
Parent:
id | name | gender
111| Alex | Male
222| Alice| Female
Child:
parentId | time | Action
111 | 12:01| Walk
111 | 12:03| Run
222 | 12:04| Walk
111 | 12:05| Jump
111 | 12:06| Run
The dataset I want to produce is:
id | name | gender | firstAction | lastAction |
111| Alex | Male | Walk | Run |
222| Alice| Female | Walk | Walk |
Currently I can achieve this using two window functions, something like:
WindowSepc w1 = Window.partitionBy("parentId").orderBy(col("time").asc())
WindowSepc w2 = Window.partitionBy("parentId").orderBy(col("time").desc())
and apply the windowSpec to child table using row_number().over(), like:
child.withColumn("rank1", row_numbers().over(w1))
.withColumn("rank2", row_numbers().over(w2))
The issue I have is that later, when I need to join with the parent table, I need to join two times, one for parentId=id && rank1=1, and another one for parentId=id && rank2=1
I wonder if there is a way to only join once, which will be much more efficient.
Or I used the Window function incorrectly, and there is a better way to do it?
Thanks

Answer 1

You could join first and then use groupBy instead of window-functions, this could work (not tested as no programmatic dataframe is provided):
parent
.join(child,$"parentId"===$"id")
.groupBy($"parentId",$"name",$"gender")
.agg(
min(struct($"time",$"action")).as("firstAction"),
max(struct($"time",$"action")).as("lastAction")
)
.select($"parentId",
$"name",
$"gender",
$"firstAction.action".as("firstAction"),
$"lastAction.action".as("lastAction")
)