Paginate REST API call in Data Factory - azure

Question

I want to call a ZenDesk REST API in Data Factory,
here's how the output looks like
next_page determines the next URL page of the REST API
end_of_stream returns false if I've reached the end of the page
In Data Factory, I've set up a Copy Activity in a pipeline which copies the data from REST JSON into my Azure DB, something like this
The problem is the pagination in ADF doesn't support the type of pagination that Zendesk API provides, from my research it looks like I'll need to create an Until loop in order to make this work, so something like this
I need to know how I can
Set a variable to true/false based on the Copy Activity output from the REST API call
Dynamically set the URL of my Copy Activity, so it can loop through all the pages in the REST API
Thanks!

Answer 1

Set a variable to true/false based on the Copy Activity output from the REST API call
In the value Variables section of the Set Variable Activity, in the variable click on the add dynamic content.
#activity('Copy data1').output.end_of_stream
You can replace the Copy data1 with that of your own Copy Activity Name
Dynamically set the URL of my Copy Activity, so it can loop through all the pages in the REST API
For your REST API dataset,
Configure the below Parameter
Relative URL :
In the Copy Data Activity :
Once the first copy action is done, you should update the variable start_time with the value of the end_time of the output.
Snippet Output of the REST API
The reason being, the next page url is same relative url of the API with start_time parameter with the value of end_time of the REST API (Current Page) Output.