Getting data from external API before render - node.js

I have an application based on the React Starter Kit.
Every page have a fetch function that getting data from API in componentDidMount lifecycle.
I want to get data first and then render page with data and return it to the client. UX in my case no matter.
I know that RSK is isomorphic, I'm ready to change boilerplate or create my own. But I do not understand how to fetch data from API before render page(I mean how to tell express server what data requires).
How App fetching data now:
example_page.js:
import getBooks from 'queries/getAllBooks';
...
class IdTag extends React.Component {
componentDidMount(){
this.getBooks();
}
getBooks() => {
const request = getBooks();
request
.then(...)
}
}
getAllBooks.js:
import doGet from './doGet';
let result = '';
const request = async () => {
const reqUrl = '/api/books/';
result = await doGet(reqUrl);
return result;
};
export default request;
doGet.js:
const request = async reqUrl => {
let requestResult = null;
const doQuery = async () => {
const response = await fetch(reqUrl, {
method: 'GET',
});
const result = await response.json();
result.status = response.status;
return result;
};
requestResult = await doQuery();
return requestResult
}
...
export default request;
server.js:
...
app.get('/api/*', async (req, res) => {
const newUrl = config.gate.URL + req.url.replace('/api', '');
const accessToken = req.cookies.access_token;
const response = await nodeFetch(newUrl, {
headers: {
Authorization: `Bearer ${accessToken}`,
},
});
const result = await response.json();
res.status(response.status);
res.json(result);
});
...

If each page has api calls, then Its better to use redux and redux saga. Purpose of redux saga is to handle api calls. It will process the actions in a Q. The moment u call api using fetch method, create below actioncreators
1) InitialLoading(true)
2) Fetch api call action creator
3) Based on success, error create action creator to store fetch method output data in store
4) InitialLoading(false)

You could simply set a flag when you begin your fetch, and while it's fetching return null instead of rendering. Something like:
flag = true;
request = getBooks();
request.then(flag = false);
and then:
render(){
if (flag){
return null;
} else {
return this.view;
}
}

Related

Correctly fetch authentication tokens server-side in a node.js React app hosted on Cloud Run

While not a front-end developer, I'm trying to set up a web app to show up a demo for a product. That app is based on the Sigma.js demo app demo repository.
You'll notice that this app relies on a graph which is hosted locally, which is loaded as:
/src/views/Root.tsx :
useEffect(() => {
fetch(`${process.env.PUBLIC_URL}/dataset.json`)
.then((res) => res.json())
.then((dataset: Dataset) => {...
// do things ....
and I wish to replace this by a call to another service which I also host on Cloud Run.
My first guess was to use the gcloud-auth-library, but I could not make it work - especially since it does not seem to support Webpack > 5 (I might be wrong here), the point here this lib introduces many problems in the app, and I thought I'd be better off trying the other way GCP suggests to handle auth tokens: by calling the Metadata server.
So I replaced the code above with:
Root.tsx :
import { getData } from "../getGraphData";
useEffect(() => {
getData()
.then((res) => res.json())
.then((dataset: Dataset) => {
// do even more things!
getGraphData.js :
import { getToken } from "./tokens";
const graphProviderUrl = '<my graph provider service URL>';
export const getData = async () => {
try {
const token = await getToken();
console.log(
"getGraphData.js :: getData : received token",
token
);
const request = await fetch(
`${graphProviderUrl}`,
{
headers: {
Authorization: `Bearer ${token}`,
},
}
);
const data = await request.json();
console.log("getGraphData.js :: getData : received graph", data);
return data;
} catch (error) {
console.log("getGraphData.js :: getData : error getting graph data", error);
return error.message;
}
};
tokens.js :
const targetAudience = '<my graph provider service base URL>'; // base URL as audience
const metadataServerAddress = "169.254.169.254"; // use this to shortcut DNS call to metadata.google.internal
export const getToken = async () => {
if (tokenExpired()) {
const token = await getValidTokenFromServer();
sessionStorage.setItem("accessToken", token.accessToken);
sessionStorage.setItem("expirationDate", newExpirationDate());
return token.accessToken;
} else {
console.log("tokens.js 11 | token not expired");
return sessionStorage.getItem("accessToken");
}
};
const newExpirationDate = () => {
var expiration = new Date();
expiration.setHours(expiration.getHours() + 1);
return expiration;
};
const tokenExpired = () => {
const now = Date.now();
const expirationDate = sessionStorage.getItem("expirationDate");
const expDate = new Date(expirationDate);
if (now > expDate.getTime()) {
return true; // token expired
}
return false; // valid token
};
const getValidTokenFromServer = async () => {
// get new token from server
try {
const request = await fetch(`http://${metadataServerAddress}/computeMetadata/v1/instance/service-accounts/default/token?audience=${targetAudience}`, {
headers: {
'Metadata-Flavor': 'Google'
}
});
const token = await request.json();
return token;
} catch (error) {
throw new Error("Issue getting new token", error.message);
}
};
I know that this kind of call will need to be done server-side. What I don't know is how to have it happen on a React + Node app. I've tried my best to integrate good practices but most questions related to this topic (request credentials through a HTTP (not HTTPS!) API call) end with answers that just say "you need to do this server-side", without providing more insight into the implementation.
There is a question with similar formulation and setting here but the single answer, no upvote and comments is a bit underwhelming. If the actual answer to the question is "you cannot ever call the metadata server from a react app and need to set up a third-party service to do so (e.g. firebase)", I'd be keen on having it said explicitly!
Please assume I have only a very superficial understanding of node.js and React!

remix passing args to async server side functions

I am actually a really beginner with this stuff so I beg your pardon for my (silly) questions.
I want to use async functions inside tsx pages, specifically those functions are fetching calls from shopify to get data and ioredis calls to write and read some data.
I know that remix uses action loader functions, so to manage shopify calls I figured out this
export const loader: LoaderFunction = async ({ params }) => {
return json(await GetProductById(params.id as string));
};
async function GetProductById(id: string) {
const ops = ...;
const endpoint = ...;
const response = await fetch(endpoint, ops);
const json = await response.json();
return json;
};
export function FetchGetProductById(id: number) {
const fetcher = useFetcher();
useEffect(() => {
fetcher.load(`/query/getproductid/${id}`);
}, []);
return fetcher.data;
}
with this solution I can get the data whenever I want just calling FetchGetProductById, but my problem is that I need to send more complex data to the loader (like objects)
How may I do that?
In Remix, the loader only handles GET requests, so data must be in the URL, either via params or searchParams (query string).
If you want to pass data in the body of the request, then you'll need to use POST and create an action.
NOTE: Remix uses FormData and not JSON to send data. You will need to convert your JSON into a string.
export const action = async ({ request }: ActionArgs) => {
const formData = await request.formData();
const object = JSON.parse(formData.get("json") as string);
return json(object);
};
export default function Route() {
const fetcher = useFetcher();
useEffect(() => {
if (fetcher.state !== 'idle' || fetcher.data) return;
fetcher.submit(
{
json: JSON.stringify({ a: 1, message: "hello", b: true }),
},
{ method: "post" }
);
}, [fetcher]);
return <pre>{JSON.stringify(fetcher.data, null, 2)}</pre>
}

BotFramework TypeError: Cannot perform 'get' on a proxy that has been revoked

I am trying to develop a MS Teams bot that sends content to students module(unit) wise. I have created 3 classes:
methods.js = Contains all the methods for sending texts, attachments etc.
teamBot.js = Captures a specific keyword from the users and based on that executes a function.
test.js = Connects the bot with Airtable and sends the content accordingly
I am facing Cannot perform 'get' on a proxy that has been revoked error. I figured it might be because of the context. I am passing context as a parameter, which I feel might not be the correct way, how can I achieve the result, and retain the context between files.
teamsBot.js
const test = require("./test");
class TeamsBot extends TeamsActivityHandler {
constructor() {
super();
// record the likeCount
this.likeCountObj = { likeCount: 0 };
this.onMessage(async (context, next) => {
console.log("Running with Message Activity.");
let txt = context.activity.text;
// const removedMentionText = TurnContext.removeRecipientMention(context.activity);
// if (removedMentionText) {
// // Remove the line break
// txt = removedMentionText.toLowerCase().replace(/\n|\r/g, "").trim();
// }
// Trigger command by IM text
switch (txt) {
case "Begin": {
await test.sendModuleContent(context)
}
// By calling next() you ensure that the next BotHandler is run.
await next();
});
// Listen to MembersAdded event, view https://learn.microsoft.com/en-us/microsoftteams/platform/resources/bot-v3/bots-notifications for more events
this.onMembersAdded(async (context, next) => {
const membersAdded = context.activity.membersAdded;
for (let cnt = 0; cnt < membersAdded.length; cnt++) {
if (membersAdded[cnt].id) {
const card = cardTools.AdaptiveCards.declareWithoutData(rawWelcomeCard).render();
await context.sendActivity({ attachments: [CardFactory.adaptiveCard(card)] });
break;
}
}
await next();
});
}
test.js
const ms = require('./methods')
async function sendModuleContent(context) {
data = module_text //fetched from Airtable
await ms.sendText(context, data)
}
methods.js
const {TeamsActivityHandler, ActivityHandler, MessageFactory } = require('botbuilder');
async function sendText(context, text){
console.log("Sending text")
await context.sendActivity(text);
}
Refer this: TypeError: Cannot perform 'get' on a proxy that has been revoked
make the following changes to test.js
const {
TurnContext
} = require("botbuilder");
var conversationReferences = {};
var adapter;
async function sendModuleContent(context) {
data = module_text //fetched from Airtable
const currentUser = context.activity.from.id;
conversationReferences[currentUser] = TurnContext.getConversationReference(context.activity);
adapter = context.adapter;
await adapter.continueConversation(conversationReferences[currentUser], async turnContext => {
await turnContext.sendActivity(data);
});
}

Pass query from Link to server, first time load query value undefined, after reload get correct query

I try to create some API to external adobe stock.
Like in the title, first time i get query from Link router of undefined, but after reload page it work correctly. My
main page
<Link
href={{
pathname: "/kategoria-zdjec",
query: images.zdjecia_kategoria
}}
as={`/kategoria-zdjec?temat=${images.zdjecia_kategoria}`}
className={classes.button}>
</Link>
and my server
app
.prepare()
.then(() => {
server.get("/kategoria-zdjec", async (req, res) => {
const temat = await req.query.temat;
console.log(temat)
const url = `https://stock.adobe.io/Rest/Media/1/Search/Files?locale=pl_PL&search_parameters[words]=${temat}&search_parameters[limit]=24&search_parameters[offset]=1`;
try {
const fetchData = await fetch(url, {
headers: { ... }
});
const objectAdobeStock = await fetchData.json();
res.json(objectAdobeStock);
const totalObj = await objectAdobeStock.nb_results;
const adobeImages = await objectAdobeStock.files;
} catch (error) {
console.log(error);
}
});
and that looks like getInitialProps on page next page
Zdjecia.getInitialProps = async ({req}) => {
const res = await fetch("/kategoria-zdjec");
const json = await res.json();
return { total: json.nb_results, images: json.files };
}
I think it is problem due asynchronous.
I think this might be due to the fact that you are using fetch which is actually part of the Web API and this action fails when executed on server.
You could either use isomorphic-fetch which keeps fetch API consistent between client and server, or use node-fetch when fetch is called on the server:
Zdjecia.getInitialProps = async ({ req, isServer }) => {
const fetch = isServer ? require('node-fetch') : window.fetch;
const res = await fetch("/kategoria-zdjec");
const json = await res.json();
return { total: json.nb_results, images: json.files };
}
This problem is solved, the issue was in another part of my app, directly in state management, just created new variables, and pass to link state value.

nodejs async/await nested API progress

I have an API that searches for the user-provided term, returns an array of results, then fires off async requests for each of the results and gets results for each of these second batch of requests. I'd like the API to report progress as it happens rather than just the final result. So, if I do the following request, I should get updates like so
$ curl 'http://server/?q=foobar'
searching for ${q}…
found 76… now getting images…
found 30 images… done
{
result
}
Most of relevant code is shown below. Fwiw, I am using hapijs for my application.
let imagesOfRecords = {};
const getImages = async function (q) {
console.log(`searching for ${q}…`);
const uri = `http://remoteserver/?q=${q}`;
const {res, payload} = await Wreck.get(uri);
const result = JSON.parse(payload.toString()).hits;
const numOfFoundRecords = result.total;
if (result.total) {
console.log(`found ${result.total}… now getting images…`);
const foundRecords = result.hits.map(getBuckets);
Promise.all(foundRecords).then(function() {
console.log(`found ${Object.keys(imagesOfRecords).length} images… done`);
reply(imagesOfRecords).headers = res.headers;
}).catch(error => {
console.log(error)
});
}
else {
console.log('nothing found');
reply(0).headers = res.headers;
}
};
const getBuckets = async function(record) {
const { res, payload } = await Wreck.get(record.links.self);
const bucket = JSON.parse(payload.toString()).links.bucket;
await getImageFiles(bucket, record.links.self);
};
const getImageFiles = async function(uri, record) {
const { res, payload } = await Wreck.get(uri);
const contents = JSON.parse(payload.toString()).contents;
imagesOfRecords[record] = contents.map(function(el) {
return el.links.self;
});
};
Once I can implement this, my next task would be to implement this progressive update in a web app that uses the above API.
To show result with each step of your requests for backend you can use EventEmitter, which will emit event on each progress step. You can read about events here.
Simple implementation:
const events = require('events');
const eventEmitter = new events.EventEmitter();
//your request code
Promise.all(foundRecords).then(function() {
console.log(`found ${Object.keys(imagesOfRecords).length} images… done`);
eventEmitter.emit('progress');
reply(imagesOfRecords).headers = res.headers;
})
const eventReaction = (e) => {
// do something with event, console log for example.
}
eventEmitter.on('progress', eventReaction);
More examples you can find here and here.
To show events to client you can use library socket.io. I think you can find pretty straightforward explanations how socket.io works in documentation.
If you want to send events between servers or processes and want to go little further, you can read more about 0MQ (zero mq) and it's node implementation

Resources