I wrote a basic API to parse RSS data from any source (in particular the Washington Post). The route I wrote is simple enough, it just checks what category your looking for, then queries the newest data in that feed. However the program runs slower than I need it to. For reasons explained below, it needs to respond in less that 8s.
I haven't been a active software developer for years (junior dev for 2 yrs like 15 yrs ago) so I am rusty. I'm new to node js and I have never written an API before.
I'm working on this because I'm trying to integrate it with an online service that requires all api calls to return in under 8s. However, my functions return in about 22s. I switched parsing methods and got it down to about 14s but from here I'm stuck. Does anyone know a quicker way to parse large rss feeds.
I'm using rss parser package ans even switch to fast-xml-parser with not luck.
I don't even need all 15 or 20 posts, just the latest 3. I tried using trim to limit the results but that doesn't seem to make any significant difference. Below is the code. I also can link to the repo in github if more context is needed. Any help would be appreciated!
//Basic RSS parser returning array of results.
async function fetchRssFeed(feedUrl) {
let feed = await. parser.parseURL(feedUrl);
return feed.items.map(item => {
return {
title: item.title,
link: item.link,
date: item.pubDate
}
});
}
I am writing a Javascript code for one of my Actions and It is complicated as it manipulates data structure of javascript object (written below) to search queries. How do I debug the code
to make sure it works as intended. It consumes alot time for me so I was wondering if I could setup an IDE for myself. Sure I can use bixby itself to view the data output but sometimes it convenient to use console to check my code as I go along. I am not asking for recommendation, but I need clarify what the dev docs implies. It does mention that it use ES5.1 and some features too. but I don't know what that "some features" are by just looking at Mozilla's Rhino Compatibility chart. Because I did want to use .reduce(callback, initialValue) function to ouput data objects. However The Mozilla's Rhino Chart shows error for it.
PS: I Hope I am not breaking rules this time.
// #DataGraph
[{
$id: "Cold_Souls_1",
animeTitle: "Fullmetal Alchemist Brotherhood",
animePoster:{
referenceImage: 'https://qph.fs.quoracdn.net/main-qimg-da58c837c7197acf364cb2ada34fc5fb.webp',
imageTags: ["Grey","Yellow","Blue","Metal Body","Machine", "Robot","Yellow Hair Boy"],
},
animeCharacters:{
"Edward Elric": [
{
quote: "A lesson without pain is meaningless. For you cannot gain something without sacrificing something else in return. But once you have recovered it and made it your own... You will gain an irreplaceable Fullmetal heart.",
keywords: ["lesson", "pain", "meaningless", "gain","sacrificing", "recover"],
category: "Life Lesson"
}
]
}
}]
As you read from https://bixbydevelopers.com/dev/docs/dev-guide/developers/actions.js-actions Bixby IDE supports ECMAScript 5.1 and some ES6 such as:
The arrow (=>) function operator
The const keyword
The let keyword
Array destructuring
It is fair to say the functions not listed above are not supported.
I would recommend that you raise a Feature Request in our community for unsupported features. This forum is open to other Bixby developers who can upvote it, leading to more visibility within the community and with the Product Management team.
I'm still a novice web developer, so please bear with me if I miss something fundamental !
I'm creating a backoffice for a Strapi backend, using react-admin.
React-admin library uses a 'data provider' to link itself with an API. Luckily someone already wrote a data provider for Strapi. I had no problem with step 1 and 2 of this README, and I can authenticate to Strapi within my React app.
I now want to fetch and display my Strapi data, starting with Users. In order to do that, quoting Step 3 of this readme : 'In controllers I need to set the Content-Range header with the total number of results to build the pagination'.
So far I tried to do this in my User controller, with no success.
What I try to achieve:
First, I'd like it to simply work with the ctx.set('Content-Range', ...) hard-coded in the controller like aforementioned Step 3.
Second, I've thought it would be very dirty to c/p this logic in every controller (not to mention in any future controllers), instead of having some callback function dynamically appending the Content-Range header to any fetchAll request. Ultimately that's what I aim for, because with ~40 Strapi objects to administrate already and plenty more to come, it has to scale.
Technical infos
node -v: 11.13.0
npm -v: 6.7.0
strapi version: 3.0.0-alpha.25.2
uname -r output: Linux 4.14.106-97.85.amzn2.x86_64
DB: mySQL v2.16
So far I've tried accessing the count() method of User model like aforementioned step3, but my controller doesn't look like the example as I'm working with users-permissions plugin.
This is the action I've tried to edit (located in project/plugins/users-permissions/controllers/User.js)
find: async (ctx) => {
let data = await strapi.plugins['users-permissions'].services.user.fetchAll(ctx.query);
data.reduce((acc, user) => {
acc.push(_.omit(user.toJSON ? user.toJSON() : user, ['password', 'resetPasswordToken']));
return acc;
}, []);
// Send 200 `ok`
ctx.send(data);
},
From what I've gathered on Strapi documentation (here and also here), context is a sort of wrapper object. I only worked with Express-generated APIs before, so I understood this snippet as 'use fetchAll method of the User model object, with ctx.query as an argument', but I had no luck logging this ctx.query. And as I can't log stuff, I'm kinda blocked.
In my exploration, I naively tried to log the full ctx object and work from there:
// Send 200 `ok`
ctx.send(data);
strapi.log.info(ctx.query, ' were query');
strapi.log.info(ctx.request, 'were request');
strapi.log.info(ctx.response, 'were response');
strapi.log.info(ctx.res, 'were res');
strapi.log.info(ctx.req, 'were req');
strapi.log.info(ctx, 'is full context')
},
Unfortunately, I fear I miss something obvious, as it gives me no input at all. Making a fetchAll request from my React app with these console.logs print this in my terminal:
[2019-09-19T12:43:03.409Z] info were query
[2019-09-19T12:43:03.410Z] info were request
[2019-09-19T12:43:03.418Z] info were response
[2019-09-19T12:43:03.419Z] info were res
[2019-09-19T12:43:03.419Z] info were req
[2019-09-19T12:43:03.419Z] info is full context
[2019-09-19T12:43:03.435Z] debug GET /users?_sort=id:DESC&_start=0&_limit=10& (74 ms)
While in my frontend I get the good ol' The Content-Range header is missing in the HTTP Response message I'm trying to solve.
After writing this wall of text I realize the logging issue is separated from my original problem, but if I was able to at least log ctx properly, maybe I'd be able to find the solution myself.
Trying to summarize:
Actual problem is, how do I set my Content-Range properly in my strapi controller ? (partially answered cf. edit 3)
Collateral problem n°1: Can't even log ctx object (cf. edit 2)
Collateral problem n°2: Once I figure out the actual problem, is it feasible to address it dynamically (basically some callback function for index/fetchAll routes, in which the model is a variable, on which I'd call the appropriate count() method, and finally append the result to my response header)? I'm not asking for the code here, just if you think it's feasible and/or know a more elegant way.
Thank you for reading through and excuse me if it was confuse; I wasn't sure which infos would be relevant, so I thought the more the better.
/edit1: forgot to mention, in my controller I also tried to log strapi.plugins['users-permissions'].services.user object to see if it actually has a count() method but got no luck with that either. Also tried the original snippet (Step 3 of aforementioned README), but failed as expected as afaik I don't see the User model being imported anywhere (the only import in User.js being lodash)
/edit2: About the logs, my bad, I just misunderstood the documentation. I now do:
ctx.send(data);
strapi.log.info('ctx should be : ', {ctx});
strapi.log.info('ctx.req = ', {...ctx.req});
strapi.log.info('ctx.res = ', {...ctx.res});
strapi.log.info('ctx.request = ', {...ctx.request});
ctrapi.log.info('ctx.response = ', {...ctx.response});
Ctx logs this way; also it seems that it needs the spread operator to display nested objects ({ctx.req} crash the server, {...ctx.req} is okay). Cool, because it narrows the question to what's interesting.
/edit3: As expected, having logs helps big time. I've managed to display my users (although in the dirty way). Couldn't find any count() method, but watching the data object that is passed to ctx.send(), it's equivalent to your typical 'res.data' i.e a pure JSON with my user list. So a simple .length did the trick:
let data = await strapi.plugins['users-permissions'].services.user.fetchAll(ctx.query);
data.reduce((acc, user) => {
acc.push(_.omit(user.toJSON ? user.toJSON() : user, ['password', 'resetPasswordToken']));
return acc;
}, []);
ctx.set('Content-Range', data.length) // <-- it did the trick
// Send 200 `ok`
ctx.send(data);
Now starting to work on the hard part: the dynamic callback function that will do that for any index/fetchAll call. Will update once I figure it out
I'm using React Admin and Strapi together and installed ra-strapi-provider.
A little boring to paste Content-Range header into all of my controllers, so I searched for a better solution. Then I've found middleware concept and created one that fits my needs. It's probably not the best solution, but do its job well:
const _ = require("lodash");
module.exports = strapi => {
return {
// can also be async
initialize() {
strapi.app.use(async (ctx, next) => {
await next();
if (_.isArray(ctx.response.body))
ctx.set("Content-Range", ctx.response.body.length);
});
}
};
};
I hope it helps
For people still landing on this page:
Strapi has been updated from #alpha to #beta. Care, as some of the code in my OP is no longer valid; also some of their documentation is not up to date.
I failed to find a "clever" way to solve this problem; in the end I copy/pasted the ctx.set('Content-Range', data.length) bit in all relevant controllers and it just worked.
If somebody comes with a clever solution for that problem I'll happily accept his answer. With the current Strapi version I don't think it's doable with policies or lifecycle callbacks.
The "quick & easy fix" is still to customize each relevant Strapi controller.
With strapi#beta you don't have direct access to controller's code: you'll first need to "rewrite" one with the help of this doc. Then add the ctx.set('Content-Range', data.length) bit. Test it properly with RA, so for the other controllers, you'll just have to create the folder, name the file, copy/paste your code + "Search & Replace" on model name.
The "longer & cleaner fix" would be to dive into the react-admin source code and refactorize so the lack of "Content-Range" header doesn't break pagination.
You'll now have to maintain your own react-admin fork, so make sure you're already committed into this library and have A LOT of tables to manage through it (so much that customizing every Strapi controller will be too tedious).
Before forking RA, please remember all the stuff you can do with the Strapi backoffice alone (including embedding your custom React app into it) and ensure it will be worth the trouble.
Can someone help me and can explain about this matter?
Currently, I'm just building a blog which I used which nodejs. In my projects, I want to use and display the two different languages which my local language and English.
As I showed up above like that website when I click change languages without showing like this example.com/mm. I'm just want to display like example.com without /mm or /en.
Example url: https://www.mmbusticket.com/
I'm not familiar with PHP. I'm the big fun of Nodejs.
How I have to do so for this case and which packages should I use for nodejs?
Thanks.
An option for you is the i18n module, and you can find similar options in many frontend frameworks as well. (You'll see this concept in app development too.)
The idea is that you have a directory with "locales" (the languages), each in a JSON file. The keys are the same in all locales. Like:
locales/en.json
{
"hello": "hello",
"greeting": "hey there, {{name}}"
}
locales/mm.json (used google translate, forgive me : )
{
"hello": "ဟယ်လို",
"greeting": "ဟေ့ဒီမှာ {{name}}"
}
In your app you'd do something like i18n.localize("hello") and depending on your current language setting (maybe passed in a cookie to the server if server-rendering, or set on the frontend page for client-side) you'll get the response.
Variables can be done above like i18n.localize(['greeting', {name: "clay"}]) and that will fill in the passed parameter name into the string defined at greeting. You can typically do nesting and other cool things, depending on the library used.
Just note that you end up using these special "key strings" everywhere, so the code is a little messier to read. Name those keys wisely : ) And if you want to translate the entire contents of your blog, that's a different service entirely.
The examples for generating tests in the testflow create stop_bins. However there were no examples of how to generate the 93k multi_bin node. Does this feature exist in the current origen-sdk?
output node looks like this in 93k .tf file
if #FLAG then
{
multi_bin;
}
else
{
}
There is currently no direct support for creating multi_bin nodes, though in time I do expect that it will be added as a result of this effort to add support for limits tables.
In the meantime though, there is the ability to render any text and this can be used to generate what you want.
To generate the above example you could do:
if_flag :flag do
render 'multi_bin;'
end
This will also work with in-line conditions, this is the same:
render 'multi_bin;', if_flag: :flag
Additionally, on_pass and on_fail will accept a render option:
func :my_test, on_fail: { render: 'multi_bin;' }
Obviously that is creating something that will not be able to translate to other tester platforms, so the advice is to use render sparingly and only as a get out of jail card when you really need it.
Also note that for these examples to work you need at least OrigenTesters 0.11.1.