RxJS: using repeat after takeUntil creates a subscription leak - memory-leaks

I have the following scenario: I want to subscribe to source1 and when it doesn't emit anything for at least 2000ms, I want to emit from fallback. However, when source1 emits again, I want to switch back to source1 and so on.
I have an implementation that works, however it creates a subscription leak because repeat is used after takeUntil. This is explained in details in this nice article. I can't figure out how to get rid of this leak while keeping the same behaviour. Is there a way?
const source1: Observable;
const fallback: Observable;
const b = source2.pipe(takeUntil(source1));
const a = source1.pipe(timeoutWith(2000, b), repeat());
a.subscribe(console.log)
You can play with this simple repro case https://stackblitz.com/edit/rxjs-repeat-learnrxjs-ah9j5r?file=index.ts

My previous answer didn't work, you're quite right. I think this does the trick though:
source1.pipe(
switchMap(val => merge(of(val), source2.pipe(delay(2000))))
).subscribe(console.log);

Related

How to implement different middlewares for different monk's Manager instance?

I'm using monk (https://automattic.github.io/monk/) in order to do mongodb data manipulation.
Monk has a middleware mechanism, and we can simply invoke addMiddleware to add as many middleware as needed (https://automattic.github.io/monk/docs/middlewares.html)
It works well until I create two manager (https://automattic.github.io/monk/docs/manager/) instance. The middlewares of the two manager seems to polute each other.
So, I create a simple test in order to make sure the hypothesis.
Here is the test:
let db = monk('mongodb://localhost/test')
let softDb = monk('mongodb://localhost/other')
console.error(db._collectionOptions.middlewares === softDb._collectionOptions.middlewares)
And as my prior hypothesis, it yield true. Eventhough db and softDb are different objects, they seems to share the same _collectionOptions.middlewares. Probably monk's developer implement singleton pattern.
My question is: how could I make this softDb and db have different set of middlewares?
Thanks in advance.
I find a workaround. This is not an officially approved solution and I'm still seeking for a better solution. So, if any of you find a better solution, I'll approve your answer.
Since both softDb and db share the same set of middleware, then I deep-clone and re-assign both db._collectionOptions.middlewares. For deep-cloning purpose, I use this package: https://www.npmjs.com/package/clone.
The solution looks as follow:
let clone = require('clone')
let db = monk('mongodb://localhost/test')
db._collectionOptions.middlewares = clone(db._collectionOptions.middlewares)
let softDb = monk('mongodb://localhost/other')
softDb._collectionOptions.middlewares = clone(softDb._collectionOptions.middlewares)
// Here I can add different set of middlewares to db and softDb
// The middlewares of both manager will be independent to each other
console.error(db._collectionOptions.middlewares === softDb._collectionOptions.middlewares)
// Now, this will yield `false`
It is important to note that you have to copy the middlewares immediately after creating a manager instance.
I'm aware that some people might hate this solution. So, rather than down-voting this answer, please provide a better solution instead.

How can I use Node.js to get the value of a child under random USERid and keys

It's a simple concept but yet finding the right answer is ridiculously stressing. Use Firebase Functions w/ Node.js to pull the "itemsExpires" (date) from the Firebase database. The parent node is the userId (random string) and each item is stored under a key node (another random string).. So, here's what the firebase database looks like:
firebase-database-name
+ 82hfijcbwjbjfbergjagbk_USERID
+ "My Stuff"
+ gnjfgsdkfjsdf_ITEMkey
-- "item name": whatever
-- "itemExpires": 05-01-2017
-- "itemType": whatever too
+ an3jeejwiag_ITEMkey
-- "item name": whatever
-- "itemExpires": 06-01-2017
-- "itemType": whatever too
+ zzzndjgabblsbl_ITEMkey
-- "item name": whatever
-- "itemExpires": 07-01-2017
-- "itemType": whatever too
I'm not asking for someone to write the code, a good reference will do but there are so many ways to call data and all I'm finding are the ways to Call using a structured tree and not one with random id's and keynumbers.
*** Basically, my goal here is to run a 3rd party cron job through Firebase Functions that runs through each item entry and checks the expiration date against today's date. This is the wall I'm against.
Bradley I am yet unclear as to what you want to do exactly. I suppose you intend on having multiple users (not just one as in the example) with multiple Items and compare the current date against the expiration date of all items for every user at a specified time (using cron). There are some considerations you should take into account here :
Do you really need cron ? ( or can you solve your problems more easily and natively with a javascript plain setInterval() ? )
How often are you going to check your entire database and how big is that database?
OK. So to explain, the first consideration is just a thought and the logic behind it should be pretty obvious. The second consideration takes some explaining. Since I believe your firebase data will NOT be static and will constantly change you need to find a way to get these changes inside you node script.
If you do not intend on calling your scheduled task too often and the database is not a mammoth (taking ages to load) you could just do the following :
const firebase = require('firebase-admin');
const serviceAccount = require('yourServiceAccountPath.json');
firebase.initializeApp({
credential: firebase.credential.cert(serviceAccount),
databaseURL: "youDatabaseURL"
});
setInteval(function(){ // or cron schedule
firebase.database().ref().once('value',function(snapshot){
let allYourUsers = snapshot.val();
// iterate through them all
// and do what you gotta do
});
},10000); // your interval in milliseconds
However this approach just loads once all your database each time you want to check all items. If you have other data in the database, consider adding users to a seperate path and load just that path. This approach is not recommended if your users are quite many and/or you want them to be checked very often. If such is the case and your data does not change very often you could consider this alternative:
Here you use the on function to update your data as it is edited and set the checking part seperate like so :
const firebase = require('firebase-admin');
const serviceAccount = require('yourServiceAccountPath.json');
firebase.initializeApp({
credential: firebase.credential.cert(serviceAccount),
databaseURL: "youDatabaseURL"
});
const databaseRef=firebase.database().ref();
let allYourUsers;
let allYourUsersStaticCopy;
databaseRef.on('value',function(snapshot){
allYourUsers = snapshot.val();
});
setInteval(function(){ // or cron schedule
if ( allYourUsers ) { // to ensure that data has loaded at least once
// (startup considerations)
allYourUsersStaticCopy = Object.assign({},allYourUsers);
// iterate through the static copy in order to avoid
// you data changing while you are accesing it
// and do what you gotta do
}
},10000); // your interval in milliseconds
The upside with the second piece of code is that your data is loaded every time there is a change and not every time your check runs. If however your data changes very often (additions,deletions and edits) this approach might not be optimum.
In the case that your script runs often enough, the database is big enough, and the changes are often enough to prevent any of the above to be efficient, you might want to consider the optimum solution : loading your users once and then attaching listeners for the child added,removed and changed to update your existing users object. Thus you receive only changes and not a whole snapshot. If such is the case you can read about the mentioned listeners in the firebase docs and I can squeeze in some extra code. Just let me know.
Hope this long post helps!
Assuming you have Firebase set up properly within your node project, you can do a one time read for your ITEMkey entries. Something like this:
var db = admin.database();
var ref = db.ref("82hfijcbwjbjfbergjagbk_USERID").child("My Stuff");
ref.once("value", function(snapshot) {
var contents = snapshot.val();
// Data returned here will be an object with all children
// nodes under "My Stuff". You can access it by calling
// snapshot.val() like I did above.
}

How do I manage groups/rooms with node WebSockets?

TL;DR below.
I am currently developing a React/Redux SPA that is driven by real-time data. I've decided to use ws, instead of socket.io since socket.io feels a bit high level for what I'm doing, I'd rather manage sockets myself.
In saying that, I'm struggling to find a way to manage the separation of updates/messages per view/route. Since I'm using client-side routing it's per express route won't really work...
Messages between the server and client via WebSockets are JSON with actions like GET_ITEMS then a response of GET_ITEMS_SUCCESS with an array of 'items' and for errors: ..._ERROR etc. This is all fine, since it's just 1 to 1 transaction. Though the problem arises when broadcasting (1 to all) to all relevant clients when the server receives an update.
So, I assume it best practice to limit these broadcasts to the clients that are viewing/want the data. So when viewing, for example, the Item page, there is no point in broadcasting updates to the User data since that is only used on the User page.
I haven't been able to find any common practices when dealing with this sort of situation, just a few small outdated/barely used wrappers for ws that just add a few basic functions to leave/join but don't offer much flexibility with implementation.
What I think MIGHT work is to have an object/array for each 'group'/'room', which stores the clients that are currently listening to updates from a given section. So a user would send an action to INIT_LISTEN (& ``) with a param of category, e.g. ITEM for updates and other actions related to items.
TL;DR
What my question really boils down to is: How do I store a reference to a single socket? (ws client object? ws client ID?) Then, can I store this in an object/array to iterate through like below.
const ClientRooms = {
Items: {
{
...ws
}
/* ...rest of the client */
}
}
or
const ClientRooms = {
Items: [ "xyz" ] /* Array of ws ids */
}
I have a "ping--pong" heartbeat function to keep clients active and prevent silent connection failures/disconnections. I can't find if ws.terminate() still fires the ws close event so I can iterate 'group'/'room' the object/array to find and remove instances of that client.

range.address throws context related errors

We've been developing using Excel JavaScript API for quite a few months now. We have been coming across context related issues which got resolved for unknown reasons. We weren't able to replicate these issues and wondered how they got resolved. Recently similar issues have started popping up again.
Error we consistently get:
property 'address' is not available. Before reading the property's
value, call the load method on the containing object and call
"context.sync()" on the associated request context.
We thought as we have multiple functions defined to modularise code in project, may be context differs somewhere among these functions which has gone unnoticed. So we came up with single context solution implemented via JavaScript Module pattern.
var ContextManager = (function () {
var xlContext;//single context for entire project/application.
function loadContext() {
xlContext = new Excel.RequestContext();
}
function sync(object) {
return (object === undefined) ? xlContext.sync() : xlContext.sync(object);
}
function getWorksheetByName(name) {
return xlContext.workbook.worksheets.getItem(name.toString());
}
//public
return {
loadContext: loadContext,
sync: sync,
getWorksheetByName: getWorksheetByName
};
})();
NOTE: above code shortened. There are other methods added to ensure that single context gets used throughout application.
While implementing single context, this time round, we have been able to replicate the issue though.
Office.initialize = function (reason) {
$(document).ready(function () {
ContextManager.loadContext();
function loadRangeAddress(rng, index) {
rng.load("address");
ContextManager.sync().then(function () {
console.log("Address: " + rng.address);
}).catch(function (e) {
console.log("Failed address for index: " + index);
});
}
for (var i = 1; i <= 1000; i++) {
var sheet = ContextManager.getWorksheetByName("Sheet1");
loadRangeAddress(sheet.getRange("A" + i), i);//I expect to see a1 to a1000 addresses in console. Order doesn't matter.
}
});
}
In above case, only "A1" gets printed as range address to console. I can't see any of the other addresses (A2 to A1000)being printed. Only catch block executes. Can anyone explain why this happens?
Although I've written for loop above, that isn't my use case. In real use case, such situations occur where one range object in function a needs to load range address. In the mean while another function b also wants to load range address. Both function a and function b work asynchronously on separate tasks such as one creates table object (table needs address) and other pastes data to sheet (there's debug statement to see where data was pasted).
This is something our team hasn't been able to figure out or find a solution for.
There is a lot packed into this code, but the issue you have is that you're calling sync a whole bunch of times without awaiting the previous sync.
There are several problems with this:
If you were using different contexts, you would actually see that there is a limit of ~50 simultaneous requests, after which you'll get errors.
In your case, you're running into a different (and almost opposite) problem. Given the async nature of the APIs, and the fact that you're not awaiting on the sync-s, your first sync request (which you'd think is for just A1) will actually contain all the load requests from the execution of the entire for loop. Now, once this first sync is dispatched, the action queue will be cleared. Which means that your second, third, etc. sync will see that there is no pending work, and will no-op, executing before the first sync ever came back with the values!
[This might be considered a bug, and I'll discuss with the team about fixing it. But it's still a very dangerous thing to not await the completion of a sync before moving on to the next batch of instructions that use the same context.]
The fix is to await the sync. This is far and away the simplest to do in TypeScript 2.1 and its async/await feature, otherwise you need to do the async version of the for loop, which you can look up, but it's rather unintuitive (requires creating an uber-promise that keeps chaining a bunch of .then-s)
So, your modified TypeScript-ified code would be
ContextManager.loadContext();
async function loadRangeAddress(rng, index) {
rng.load("address");
await ContextManager.sync().then(function () {
console.log("Address: " + rng.address);
}).catch(function (e) {
OfficeHelpers.Utilities.log(e);
});
}
for (var i = 1; i <= 1000; i++) {
var sheet = ContextManager.getWorksheetByName("Sheet1");
await loadRangeAddress(sheet.getRange("A" + i), i);//I expect to see a1 to a1000 addresses in console. Order doesn't matter.
}
Note the async in front of the loadRangeAddress function, and the two await-s in front of ContextManager.sync() and loadRangeAddress.
Note that this code will also run quite slowly, as you're making an async roundtrip for each cell. Which means you're not using batching, which is at the very core of the object-model for the new APIs.
For completeness sake, I should also note that creating a "raw" RequestContext instead of using Excel.run has some disadvantages. Excel.run does a number of useful things, the most important of which is automatic object tracking and un-tracking (not relevant here, since you're only reading back data; but would be relevant if you were loading and then wanting to write back into the object).
Finally, if I may recommend (full disclosure: I am the author of the book), you will probably find a good bit of useful info about Office.js in the e-book "Building Office Add-ins using Office.js", available at https://leanpub.com/buildingofficeaddins. In particular, it has a very detailed (10-page) section on the internal workings of the object model ("Section 5.5: Implementation details, for those who want to know how it really works"). It also offers advice on using TypeScript, has a general Promise/async-await primer, describes what .run does, and has a bunch more info about the OM. Also, though not available yet, it will soon offer information on how to resume using the same context (using a newer technique than what was originally described in How can a range be used across different Word.run contexts?). The book is a lean-published "evergreen" book, son once I write the topic in the coming weeks, an update will be available to all existing readers.
Hope this helps!

Can't get range from a defined name

Excel 2016 (Office 365) 32 bits, 16.0.6965.2115, Visual Studio 14.0.25425.01 Update 3
I'm quite sure the statement below used to work, but now it doesn't work anymore:
var range = ctx.workbook.names.getItem("Countries").getRange();
I get an error stating that there is no support for getRange method, but it should be supported as documented here.
What am I'm doing wrong?
--- EDIT: this is the code I'm using ---
function paintRange() {
Excel.run(function (ctx) {
var range = ctx.workbook.names.getItem("Countries").getRange();
range.format.fill = "green";
return ctx.sync();
}).catch(function (error) {
app.showNotification("Error", error);
})
}
paintRange is attached to a button. There is a global scope defined name called Countries.
I don't have any more details of the error besides the one I mentioned, I also tried opening the quick watch window to get more clues.
UPDATE: The issue is fixed with an update to the CDN. You should be able to use namedItem.getRange() now. Thanks for reporting the issue, and allowing us to do a quick turn-around on it.
================
Felipe, looks like you're absolutely right. This is definitely a bug. Let me talk to the right folks to get this regression fixed as soon as we can. I'll see if we can put in some processes to avoid this in the future, as well.
From an immediate-workaround perspective, two options:
Use the BETA CDN (esp if it's for an in-development add-in, rather than a production one). That URL is: https://appsforoffice.microsoft.com/lib/beta/hosted/office.js
Do a temporarily filling in of the inadvertently-removed getRange functionality. Inside of Office.initialize, include the following code:
if (!Excel.NamedItem.prototype.getRange) {
Excel.NamedItem.prototype.getRange=function () {
return new Excel.Range(this.context,
OfficeExtension.ObjectPathFactory.createMethodObjectPath(
this.context, this, "GetRange",
OfficeExtension.OperationType.Read, [], false, true, null
)
);
};
}
The workaround in #2 should not cause harm even after the functionality is restored, but I would none-the-less recommend making a mental note to remove this after we've fixed the issue. I'll update this thread once we have fixed the underlying bug, hopefully within a weeks' time (as a very rough estimate, pending any complications that might delay it).
Thanks for bringing it to our attention -- both the individual bug, and the underlying process that let the regression to this one API go unnoticed.

Resources