I want to ask you kindly for help. I would like to receive all the Strapi data while I am doing a GET request.
Currently in Postman I see that in total there are 4316 objects. I want to receive all of them via a GET request. How can i change this in order not to receive only 25 objects?
Should I change the url somehow?
I tried with something like this, but without luck :(
http://localhost:1337/api/webs?pagination[start][$eq]=0&[limit][$eq]=10000
As per their doc [1] the default and maximum values for pagination[limit] can be configured in the ./config/api.js file with the api.rest.defaultLimit and api.rest.maxLimit keys.
[1]
https://docs.strapi.io/developer-docs/latest/developer-resources/database-apis-reference/rest/sort-pagination.html#pagination-by-offset
This might work
/webs?publicationState=live&pagination[page]=1&pagination[pageSize]=9999
and
config/api.js
module.exports = {
rest: {
defaultLimit: 40,
maxLimit: 9999,
withCount: true,
},
};
in my nextjs I was able to generate ~300 pages, I used maxLimit: 3000
Related
Am using express js on node for my api service ! In which am using sequelize for query handling purposes !
So in some usecase like creating record, or updating record its simply returning "1" or sometimes nothing !
In this case , am just using
res.sendStatus(200);
or sometimes
res.send("success");
Is there any better way or this is the correct way to handle ? Or should in need .end() in order to end the process ??
which is a good way to handle these kind of useless responses which we dont need to send back ?
This is where Status 204 comes in to play: https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#2xx_success
It states: everything is OK (like in 200), but there is simple no need to send a body.
Using express, it's just as simple as: res.sendStatus(204)
Usually a json is send back to the client with different information in it that lets the client-side know that the operation that they request through the api is successfull or failed. for e.g. here, you can define a standard json response like this
//in case of success of the request
{
requestStatus: 1,
...other data if you want to like new Id of the created data/updated data or info about it if you want or depending upon use case
}
or
// in case of failure of the request
{
requestStatus: 0,
...other data if you want to like reason for failure, error message or depending upon your use case
}
Just add this line of code, should be fine:
//For all bad endpoints
app.use('*', (req, res) => {
res.status(404).json({ msg: 'Not a good endpoint, why are you here? Go play FIFA.'})
})
If you want you can generate an error HTML file, but since it's back-end, JSON format is strongly suggested. You can also add success: false for more clarity.
I'm still a novice web developer, so please bear with me if I miss something fundamental !
I'm creating a backoffice for a Strapi backend, using react-admin.
React-admin library uses a 'data provider' to link itself with an API. Luckily someone already wrote a data provider for Strapi. I had no problem with step 1 and 2 of this README, and I can authenticate to Strapi within my React app.
I now want to fetch and display my Strapi data, starting with Users. In order to do that, quoting Step 3 of this readme : 'In controllers I need to set the Content-Range header with the total number of results to build the pagination'.
So far I tried to do this in my User controller, with no success.
What I try to achieve:
First, I'd like it to simply work with the ctx.set('Content-Range', ...) hard-coded in the controller like aforementioned Step 3.
Second, I've thought it would be very dirty to c/p this logic in every controller (not to mention in any future controllers), instead of having some callback function dynamically appending the Content-Range header to any fetchAll request. Ultimately that's what I aim for, because with ~40 Strapi objects to administrate already and plenty more to come, it has to scale.
Technical infos
node -v: 11.13.0
npm -v: 6.7.0
strapi version: 3.0.0-alpha.25.2
uname -r output: Linux 4.14.106-97.85.amzn2.x86_64
DB: mySQL v2.16
So far I've tried accessing the count() method of User model like aforementioned step3, but my controller doesn't look like the example as I'm working with users-permissions plugin.
This is the action I've tried to edit (located in project/plugins/users-permissions/controllers/User.js)
find: async (ctx) => {
let data = await strapi.plugins['users-permissions'].services.user.fetchAll(ctx.query);
data.reduce((acc, user) => {
acc.push(_.omit(user.toJSON ? user.toJSON() : user, ['password', 'resetPasswordToken']));
return acc;
}, []);
// Send 200 `ok`
ctx.send(data);
},
From what I've gathered on Strapi documentation (here and also here), context is a sort of wrapper object. I only worked with Express-generated APIs before, so I understood this snippet as 'use fetchAll method of the User model object, with ctx.query as an argument', but I had no luck logging this ctx.query. And as I can't log stuff, I'm kinda blocked.
In my exploration, I naively tried to log the full ctx object and work from there:
// Send 200 `ok`
ctx.send(data);
strapi.log.info(ctx.query, ' were query');
strapi.log.info(ctx.request, 'were request');
strapi.log.info(ctx.response, 'were response');
strapi.log.info(ctx.res, 'were res');
strapi.log.info(ctx.req, 'were req');
strapi.log.info(ctx, 'is full context')
},
Unfortunately, I fear I miss something obvious, as it gives me no input at all. Making a fetchAll request from my React app with these console.logs print this in my terminal:
[2019-09-19T12:43:03.409Z] info were query
[2019-09-19T12:43:03.410Z] info were request
[2019-09-19T12:43:03.418Z] info were response
[2019-09-19T12:43:03.419Z] info were res
[2019-09-19T12:43:03.419Z] info were req
[2019-09-19T12:43:03.419Z] info is full context
[2019-09-19T12:43:03.435Z] debug GET /users?_sort=id:DESC&_start=0&_limit=10& (74 ms)
While in my frontend I get the good ol' The Content-Range header is missing in the HTTP Response message I'm trying to solve.
After writing this wall of text I realize the logging issue is separated from my original problem, but if I was able to at least log ctx properly, maybe I'd be able to find the solution myself.
Trying to summarize:
Actual problem is, how do I set my Content-Range properly in my strapi controller ? (partially answered cf. edit 3)
Collateral problem n°1: Can't even log ctx object (cf. edit 2)
Collateral problem n°2: Once I figure out the actual problem, is it feasible to address it dynamically (basically some callback function for index/fetchAll routes, in which the model is a variable, on which I'd call the appropriate count() method, and finally append the result to my response header)? I'm not asking for the code here, just if you think it's feasible and/or know a more elegant way.
Thank you for reading through and excuse me if it was confuse; I wasn't sure which infos would be relevant, so I thought the more the better.
/edit1: forgot to mention, in my controller I also tried to log strapi.plugins['users-permissions'].services.user object to see if it actually has a count() method but got no luck with that either. Also tried the original snippet (Step 3 of aforementioned README), but failed as expected as afaik I don't see the User model being imported anywhere (the only import in User.js being lodash)
/edit2: About the logs, my bad, I just misunderstood the documentation. I now do:
ctx.send(data);
strapi.log.info('ctx should be : ', {ctx});
strapi.log.info('ctx.req = ', {...ctx.req});
strapi.log.info('ctx.res = ', {...ctx.res});
strapi.log.info('ctx.request = ', {...ctx.request});
ctrapi.log.info('ctx.response = ', {...ctx.response});
Ctx logs this way; also it seems that it needs the spread operator to display nested objects ({ctx.req} crash the server, {...ctx.req} is okay). Cool, because it narrows the question to what's interesting.
/edit3: As expected, having logs helps big time. I've managed to display my users (although in the dirty way). Couldn't find any count() method, but watching the data object that is passed to ctx.send(), it's equivalent to your typical 'res.data' i.e a pure JSON with my user list. So a simple .length did the trick:
let data = await strapi.plugins['users-permissions'].services.user.fetchAll(ctx.query);
data.reduce((acc, user) => {
acc.push(_.omit(user.toJSON ? user.toJSON() : user, ['password', 'resetPasswordToken']));
return acc;
}, []);
ctx.set('Content-Range', data.length) // <-- it did the trick
// Send 200 `ok`
ctx.send(data);
Now starting to work on the hard part: the dynamic callback function that will do that for any index/fetchAll call. Will update once I figure it out
I'm using React Admin and Strapi together and installed ra-strapi-provider.
A little boring to paste Content-Range header into all of my controllers, so I searched for a better solution. Then I've found middleware concept and created one that fits my needs. It's probably not the best solution, but do its job well:
const _ = require("lodash");
module.exports = strapi => {
return {
// can also be async
initialize() {
strapi.app.use(async (ctx, next) => {
await next();
if (_.isArray(ctx.response.body))
ctx.set("Content-Range", ctx.response.body.length);
});
}
};
};
I hope it helps
For people still landing on this page:
Strapi has been updated from #alpha to #beta. Care, as some of the code in my OP is no longer valid; also some of their documentation is not up to date.
I failed to find a "clever" way to solve this problem; in the end I copy/pasted the ctx.set('Content-Range', data.length) bit in all relevant controllers and it just worked.
If somebody comes with a clever solution for that problem I'll happily accept his answer. With the current Strapi version I don't think it's doable with policies or lifecycle callbacks.
The "quick & easy fix" is still to customize each relevant Strapi controller.
With strapi#beta you don't have direct access to controller's code: you'll first need to "rewrite" one with the help of this doc. Then add the ctx.set('Content-Range', data.length) bit. Test it properly with RA, so for the other controllers, you'll just have to create the folder, name the file, copy/paste your code + "Search & Replace" on model name.
The "longer & cleaner fix" would be to dive into the react-admin source code and refactorize so the lack of "Content-Range" header doesn't break pagination.
You'll now have to maintain your own react-admin fork, so make sure you're already committed into this library and have A LOT of tables to manage through it (so much that customizing every Strapi controller will be too tedious).
Before forking RA, please remember all the stuff you can do with the Strapi backoffice alone (including embedding your custom React app into it) and ensure it will be worth the trouble.
I am running Express on NodeJS and I receive a request that looks something like https://myserver.com/processCampaign?id=12345679&campaignId=123456.
Express converts that to id=12345679&%3BcampaignId=123456. So now I can't get the campaignId because I'm getting 'amp;campaignId':'123456' in the query string instead.
So I'm wondering, do I have something set wrong in Express or should I be handling this differently?
In express multiple queries are seperated with '&'. You don't need to use 'amp;' before the next query string. In your case, the url below will give you both the id and campaignId query.
https://myserver.com/processCampaign?id=12345679&campaignId=123456
If you log req.query now, you will get
{ id: '12345679', 'campaignId': '123456' }
Because I have no control over the inbound request (which is incorrect), I have no choice but to handle it on my side. I've read that the inbound URI is most likely incorrect because it is being double-encoded on their side.
Here's the solution I have come up with:
JSON.parse(JSON.stringify(req.query).replace('amp;campaignId', 'campaignId'))
If anyone has a better idea I'm open to other solutions.
I want to clear all pending_update_count in my bot!
The output of below command :
https://api.telegram.org/botxxxxxxxxxxxxxxxx/getWebhookInfo
Obviously I replaced the real API token with xxx
is this :
{
"ok":true,"result":
{
"url":"",
"has_custom_certificate":false,
"pending_update_count":5154
}
}
As you can see, I have 5154 unread updates til now!! ( I'm pretty sure this pending updates are errors! Because no one uses this Bot! It's just a test Bot)
By the way, this pending_update_count number are increasing so fast!
Now that I'm writing this post the number increased 51 and reached to 5205 !
I just want to clear this pending updates.
I'm pretty sure this Bot have been stuck in an infinite loop!
Is there any way to get rid of it?
P.S:
I also cleared the webhook url. But nothing changed!
UPDATE:
The output of getWebhookInfo is this :
{
"ok":true,
"result":{
"url":"https://somewhere.com/telegram/webhook",
"has_custom_certificate":false,
"pending_update_count":23,
"last_error_date":1482910173,
"last_error_message":"Wrong response from the webhook: 500 Internal Server Error",
"max_connections":40
}
}
Why I get Wrong response from the webhook: 500 Internal Server Error ?
I think you have two options:
set webhook that do nothing, just say 200 OK to telegram's servers. Telegram wiil send all updates to this url and the queque will be cleared.
disable webhook and after it get updates by using getUpdates method, after it, turn on webhook again
Update:
Problem with webhook on your side. You can try to emulate telegram's POST query on your URL.
It can be something like this:
{"message_id":1,"from":{"id":1,"first_name":"FirstName","last_name":"LastName","username":"username"},"chat":{"id":1,"first_name":"FirstName","last_name":"LastName","username":"username","type":"private"},"date":1460957457,"text":"test message"}
You can send this text as a POST query body with PostMan for example, and after it try to debug your backend.
For anyone looking at this in 2020 and beyond, the Telegram API now supports clearing the pending messages via a drop_pending_updates parameter in both setWebhook and deleteWebhook, as per the API documentation.
Just add return 1; at the end of your hook method.
Update:
Commonly this happens because of queries delay with the database.
I solved is like this
POST tg.api/bottoken/setWebhook to emtpy "url"
POST tg.api/bottoken/getUpdates
POST tg.api/bottoken/getUpdates with "offset" last update_id appeared before
doing this serveral times
POST tg.api/bottoken/getWebhookInfo
had a look if all away.
POST tg.api/bottoken/setWebhook with filled "url"
If you are using webhook, you can follow these steps
On your web browser, enter the following url with your right value of bot
https://api.telegram.org/bot/getWebhookInf
You will get a result like this on your screen
{"ok":true,"result":{"url":"url_value",...}}
On the displayed result, copy the entire url_value without quotes and replace it on this second url
https://api.telegram.org/bot/setWebhook?url=url_value&drop_pending_updates=True
Enter the second url with right bot and url_value in your web browser then press ENTER
Done!
i solve it by Change file access permissions file - set permissions file to 755
and second increase memory limit in php.ini file
A quick&dirty way is to get a temporary webhook here: https://webhook.site/ and
set your webhook to that (it will answer with a HTTP/200 code everytime, reseting your pending messages to zero)
I faced the same issue for my tele bot after user edited existing message. My bot receives update with editedMessage continuously, but update.hasMessage() was empty. As a result number of updates rocketly increased and my bot stack.
I solved this issue by adding handling for use case when message is missing - send 200 code:
public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent event, Context context) {
update = MAPPER.readValue(event.getBody(), Update.class);
if (!update.hasMessage()) {
return new APIGatewayProxyResponseEvent()
.withStatusCode(200) // -> !!!!!! return code 200
.withBody("message is missing")
.withIsBase64Encoded(false);
}
... ... ...
I'm using https://github.com/firebase/flashlight to index data for searches
However, this morning I deleted the whole firebase index, so it should be empty (this has worked before, but it seems that when the nodejs app.js crashes in some cases, causing the cache to get "stuck"), but I still see old search results from my nodejs app somehow...
I've tried:
http://localhost:9200/_cache/clear
and
http://localhost:9200/_flush
http://localhost:9200/firebase/_flush
They all say successful, but still I get old results, out of, seemingly nowhere.
I can also see in the console that it refreshes every 60 seconds, and, deleting the whole firebase has worked before without problems...
I even saw a message housekeeping: found 60 orphans (removing them now) in the console so it should be refreshed by now...
I tried restarting elasticsearch as well as the whole Linux/Debian server...
In the config.js I have two indexes:
exports.paths = [
{
path: "tags",
index: "firebase",
type: "tag",
filter: function(data) { return data.name !== 'system'; }
},
{
path: "tracks",
index: "firebase",
type: "track",
filter: function(data) { return data.name !== 'system'; }
}
];
And strangely enough, I have no problem whatsoever when using the 'track' store, instead of using the 'tag' one...
What am I missing here?
// Update !
So, I just deleted the firebase tracks index while the nodejs script was running and the script crashed... Same problem, different index. So the crashing script must cause it... so, how do I clear this stuck cache?
So I fixed it by simply doing:
curl -XDELETE localhost:9200/Firebase
Thanks to: https://github.com/elasticsearch/elasticsearch/issues/7541#issuecomment-54724302
I'm guessing Elastic search is not aware (and has not been told) of the relevance of its current index, perhaps the Flashlight script I'm using is not informing it about what the index should have been? But, since this is only necessary when the node script crashes when you suddenly delete your whole firebase index, it should be catchable somehow, but I'm happy that I can at least fix it like this. Rebuilding the index is not a big issue/task right now, but in the future it might be.
A wild guess, maybe you are not posting the queries correctly. You said you have tried following links:
http://localhost:9200/_cache/clear
http://localhost:9200/_flush
http://localhost:9200/firebase/_flush
If you are accessing the urls from browser it won't clear them. You have to POST them. It is ambiguous from your question if you did it, both GET and POST return same results (showing total, successful and failed). Try this from commandline using curl:
curl -XPOST 'http://localhost:9200/_cache/clear'
curl -XPOST 'http://localhost:9200/_flush'
Or create an AJAX request with JQuery or use fiddler.
Try to optimize your indeces by sending the following POST request to your elasticsearch server:
curl -XPOST 'http://localhost:9200/_optimize?max_num_segments=1&wait_for_merge=true'
It makes lucene really delete the deleted documents from disk and merge indeces.