RxJS - check array of observables with concurrency in interval - node.js

Question

Working on a scheduler with RxJS that every second checks the array of jobs. When job is finished it is removed from array. I would like to run that with the .mergeAll(concurrency) parameter so for example there are only two jobs running at the same time.
Currently I have an workaround which can be seen here.
What I am trying is something like
Observable
.interval(1000)
.timeInterval()
.merge(...jobProcesses.map(job => Observable.fromPromise(startJob(job.id))))
.mergeAll(config.concurrency || 10)
.subscribe();
which obviously doesn't work. Any help would be appreciated.

Answer 1

From the comments, it seems you are simply trying to limit concurrency, and this interval stuff is just a detour. You should be able to get what you need with:
const Rx = require('rxjs/Rx')
let startTime = 0
const time = () => {
if (!startTime)
startTime = new Date().getTime()
return Math.round((new Date().getTime() - startTime) / 1000)
}
const jobs = new Rx.Subject() // You may additionally rate-limit this with bufferTime(x).concatAll()
const startJob = j => Rx.Observable.of(undefined).delay(j * 1000).map(() => time())
const concurrency = 2
time()
jobs
.bufferCount(concurrency)
.concatMap(buf => Rx.Observable.from(buf).flatMap(startJob))
.subscribe(x => console.log(x))
Rx.Observable.from([3, 1, 3]).subscribe(jobs)
// The last job is only processed after the first two are completed, so you see:
// 1
// 3
// 6
Note that this technically isn't squeezing out the maximum amount of concurrency possible, since it breaks the jobs up into constant batches. If your jobs have significantly uneven processing times, the longest job in the batch will delay pulling work from the next batch.