How to create a static website generator in React - node.js

Tech stack - Node.js, MongoDB for the database, Strapi CMS for editing and API, React - my application.
I have a database with a long list of entries and a ready-to-use application that allows users to read data from the database. I need to be able to generate a simple website with a single entity from my database as a source to fill the template.
Mockup
Here is a mock-up. Hopefully, it will make things a bit clearer.
Clarification
After a day of thinking about the task, I believe I need something like a simplest static website generator - an application that will allow me to select a single bit of data from the list and generate a small website filled with it. The end goal is to get a website in some subfolder of my application where I can get it and use it however I need.
A bit more about specifics:
It will be used locally
Security can be neglected
Running always in development is not a problem (just in case, thinking about additional question #2)
Few additional questions:
Is it possible to run NPM scripts from the application (like npm build)
Is there any way to show one component in development mode, but replace it with another during building for production?
App.js
//...
function App() {
if() {
return <AdminUI /> // This one is to be shown in development mode
} else {
return <Website /> // This one is to be used instead of AdminUI in the build
}
UPDATE
Well, I'm digging a path to create a site generator and so far I come up with the following basic plan:
Get my template ready
Create a new directory for my website
Copy a template to the new folder
Get an HTML file, parse it to a string to modify
Swap some bits with my data
Save to a file from the modified string
repeat if needed for other files.
If that works as expected, the whole process probably might be improved by moving from a fixed template to a component, that will be prepared with a JavaScript bundler and started with the help of something like node-cmd (to run shell commands from my application)...

What you want could be achievable, but if it's just a string and little else, I'd say it's much simpler to fetch the data at startup from a given file, and populate from there. You can put a JSON file under the public folder (together with other static data, like images) and have the file being your configuration.
In the App.js file, write an async componentDidMount() and you can do an await axios.get("") with your configuration.
So App.js would look like (code written on the fly, didn't check in an IDE):
export class App extends React.App {
constructor(props) {
super(props);
this.state = { loading: true, };
}
async componentDidMount() {
const response = await axios.get("your/data.json");
this.setState({ loading: false, ... whatever})
}
render = () => (
<>
(this.state.loading && <div>Still loading...</div>)
(this.state.adminData && <AdminUI data={this.state.admingData} />)
(this.state.devData && <Website data={this.state.devData} />)
</>
)
}
If you don't care about security, wouldn't be much simpler like this? And if you use TypeScript you'll have a much much simpler life in handling the data too.
Maybe it's worth doing an AdminUI to generate the JSON, and the another UI which reads the JSON, so you end up doing two UIs. The template-generated UI could even ask for a JSON file to bootstrap directly to the user, if it simplifies... In general, an approach based on simple JSON sounds a lost simpler than going for a CI/CD pipeline.

Related

Why is my dynamic Gatsby page not working

I'm trying to create dynamic pages based on a database that grows by the minute. Therefor it isn't an option to use createPage and build several times a day.
I'm using onCreatePage here to create pages which works fine for my first route, but when I try to make an English route somehow it doesn't work.
gatby-node.js:
exports.onCreatePage = async ({ page, actions: { createPage } }) => {
if (page.path.match(/^\/listing/)) {
page.matchPath = '/listing/:id'
createPage(page)
}
if (page.path.match(/^\/en\/listing/)) {
page.matchPath = '/en/listing/:id'
createPage(page)
}
}
What I'm trying to achieve here is getting 2 dynamic routes like:
localhost:8000/listing/123 (this one works)
localhost:8000/en/listing/123 (this one doesn't work)
My pages folder looks like this:
pages
---listing.tsx
---en/
------listing.tsx
Can anyone see what I'm doing wrong here?
--
P.S. I want to use SSR (available since Gatsby v4) by using the getServerData() in the templates for these pages. Will that work together with pages created dynamically with onCreatePage or is there a better approach?
According to what we've discussed in the comment section: the fact that the /en/ path is never created, hence is not entering the following condition:
if (page.path.match(/^\/en\/listing/)) {
page.matchPath = '/en/listing/:id'
createPage(page)
}
Points me to think that the issue is on your createPages API rather than onCreatePage, which means that your english page is not even created.
Keep in mind that onCreatePage API is a callback called when a page is created, so it's triggered after createPages.
If you add a console.log(page.path) you shouldn't see the English page in the IDE/text editor console so try debugging how are you creating the /en/ route because it seems that onCreatePage doesn't have any problem.

Retrieve file contents during Gatsby build

I need to pull in the contents of a program source file for display in a page generated by Gatsby. I've got everything wired up to the point where I should be able to call
// my-fancy-template.tsx
import { readFileSync } from "fs";
// ...
const fileContents = readFileSync("./my/relative/file/path.cs");
However, on running either gatsby develop or gatsby build, I'm getting the following error
This dependency was not found:
⠀
* fs in ./src/templates/my-fancy-template.tsx
⠀
To install it, you can run: npm install --save fs
However, all the documentation would suggest that this module is native to Node unless it is being run on the browser. I'm not overly familiar with Node yet, but given that gatsby build also fails (this command does not even start a local server), I'd be a little surprised if this was the problem.
I even tried this from a new test site (gatsby new test) to the same effect.
I found this in the sidebar and gave that a shot, but it appears it just declared that fs was available; it didn't actually provide fs.
It then struck me that while Gatsby creates the pages at build-time, it may not render those pages until they're needed. This may be a faulty assessment, but it ultimately led to the solution I needed:
You'll need to add the file contents to a field on File (assuming you're using gatsby-source-filesystem) during exports.onCreateNode in gatsby-node.js. You can do this via the usual means:
if (node.internal.type === `File`) {
fs.readFile(node.absolutePath, undefined, (_err, buf) => {
createNodeField({ node, name: `contents`, value: buf.toString()});
});
}
You can then access this field in your query inside my-fancy-template.tsx:
{
allFile {
nodes {
fields { content }
}
}
}
From there, you're free to use fields.content inside each element of allFile.nodes. (This of course also applies to file query methods.)
Naturally, I'd be ecstatic if someone has a more elegant solution :-)

Strapi & react-admin : I'd like to set 'Content-Range' header dynamically when any fetchAll query fires

I'm still a novice web developer, so please bear with me if I miss something fundamental !
I'm creating a backoffice for a Strapi backend, using react-admin.
React-admin library uses a 'data provider' to link itself with an API. Luckily someone already wrote a data provider for Strapi. I had no problem with step 1 and 2 of this README, and I can authenticate to Strapi within my React app.
I now want to fetch and display my Strapi data, starting with Users. In order to do that, quoting Step 3 of this readme : 'In controllers I need to set the Content-Range header with the total number of results to build the pagination'.
So far I tried to do this in my User controller, with no success.
What I try to achieve:
First, I'd like it to simply work with the ctx.set('Content-Range', ...) hard-coded in the controller like aforementioned Step 3.
Second, I've thought it would be very dirty to c/p this logic in every controller (not to mention in any future controllers), instead of having some callback function dynamically appending the Content-Range header to any fetchAll request. Ultimately that's what I aim for, because with ~40 Strapi objects to administrate already and plenty more to come, it has to scale.
Technical infos
node -v: 11.13.0
npm -v: 6.7.0
strapi version: 3.0.0-alpha.25.2
uname -r output: Linux 4.14.106-97.85.amzn2.x86_64
DB: mySQL v2.16
So far I've tried accessing the count() method of User model like aforementioned step3, but my controller doesn't look like the example as I'm working with users-permissions plugin.
This is the action I've tried to edit (located in project/plugins/users-permissions/controllers/User.js)
find: async (ctx) => {
let data = await strapi.plugins['users-permissions'].services.user.fetchAll(ctx.query);
data.reduce((acc, user) => {
acc.push(_.omit(user.toJSON ? user.toJSON() : user, ['password', 'resetPasswordToken']));
return acc;
}, []);
// Send 200 `ok`
ctx.send(data);
},
From what I've gathered on Strapi documentation (here and also here), context is a sort of wrapper object. I only worked with Express-generated APIs before, so I understood this snippet as 'use fetchAll method of the User model object, with ctx.query as an argument', but I had no luck logging this ctx.query. And as I can't log stuff, I'm kinda blocked.
In my exploration, I naively tried to log the full ctx object and work from there:
// Send 200 `ok`
ctx.send(data);
strapi.log.info(ctx.query, ' were query');
strapi.log.info(ctx.request, 'were request');
strapi.log.info(ctx.response, 'were response');
strapi.log.info(ctx.res, 'were res');
strapi.log.info(ctx.req, 'were req');
strapi.log.info(ctx, 'is full context')
},
Unfortunately, I fear I miss something obvious, as it gives me no input at all. Making a fetchAll request from my React app with these console.logs print this in my terminal:
[2019-09-19T12:43:03.409Z] info were query
[2019-09-19T12:43:03.410Z] info were request
[2019-09-19T12:43:03.418Z] info were response
[2019-09-19T12:43:03.419Z] info were res
[2019-09-19T12:43:03.419Z] info were req
[2019-09-19T12:43:03.419Z] info is full context
[2019-09-19T12:43:03.435Z] debug GET /users?_sort=id:DESC&_start=0&_limit=10& (74 ms)
While in my frontend I get the good ol' The Content-Range header is missing in the HTTP Response message I'm trying to solve.
After writing this wall of text I realize the logging issue is separated from my original problem, but if I was able to at least log ctx properly, maybe I'd be able to find the solution myself.
Trying to summarize:
Actual problem is, how do I set my Content-Range properly in my strapi controller ? (partially answered cf. edit 3)
Collateral problem n°1: Can't even log ctx object (cf. edit 2)
Collateral problem n°2: Once I figure out the actual problem, is it feasible to address it dynamically (basically some callback function for index/fetchAll routes, in which the model is a variable, on which I'd call the appropriate count() method, and finally append the result to my response header)? I'm not asking for the code here, just if you think it's feasible and/or know a more elegant way.
Thank you for reading through and excuse me if it was confuse; I wasn't sure which infos would be relevant, so I thought the more the better.
/edit1: forgot to mention, in my controller I also tried to log strapi.plugins['users-permissions'].services.user object to see if it actually has a count() method but got no luck with that either. Also tried the original snippet (Step 3 of aforementioned README), but failed as expected as afaik I don't see the User model being imported anywhere (the only import in User.js being lodash)
/edit2: About the logs, my bad, I just misunderstood the documentation. I now do:
ctx.send(data);
strapi.log.info('ctx should be : ', {ctx});
strapi.log.info('ctx.req = ', {...ctx.req});
strapi.log.info('ctx.res = ', {...ctx.res});
strapi.log.info('ctx.request = ', {...ctx.request});
ctrapi.log.info('ctx.response = ', {...ctx.response});
Ctx logs this way; also it seems that it needs the spread operator to display nested objects ({ctx.req} crash the server, {...ctx.req} is okay). Cool, because it narrows the question to what's interesting.
/edit3: As expected, having logs helps big time. I've managed to display my users (although in the dirty way). Couldn't find any count() method, but watching the data object that is passed to ctx.send(), it's equivalent to your typical 'res.data' i.e a pure JSON with my user list. So a simple .length did the trick:
let data = await strapi.plugins['users-permissions'].services.user.fetchAll(ctx.query);
data.reduce((acc, user) => {
acc.push(_.omit(user.toJSON ? user.toJSON() : user, ['password', 'resetPasswordToken']));
return acc;
}, []);
ctx.set('Content-Range', data.length) // <-- it did the trick
// Send 200 `ok`
ctx.send(data);
Now starting to work on the hard part: the dynamic callback function that will do that for any index/fetchAll call. Will update once I figure it out
I'm using React Admin and Strapi together and installed ra-strapi-provider.
A little boring to paste Content-Range header into all of my controllers, so I searched for a better solution. Then I've found middleware concept and created one that fits my needs. It's probably not the best solution, but do its job well:
const _ = require("lodash");
module.exports = strapi => {
return {
// can also be async
initialize() {
strapi.app.use(async (ctx, next) => {
await next();
if (_.isArray(ctx.response.body))
ctx.set("Content-Range", ctx.response.body.length);
});
}
};
};
I hope it helps
For people still landing on this page:
Strapi has been updated from #alpha to #beta. Care, as some of the code in my OP is no longer valid; also some of their documentation is not up to date.
I failed to find a "clever" way to solve this problem; in the end I copy/pasted the ctx.set('Content-Range', data.length) bit in all relevant controllers and it just worked.
If somebody comes with a clever solution for that problem I'll happily accept his answer. With the current Strapi version I don't think it's doable with policies or lifecycle callbacks.
The "quick & easy fix" is still to customize each relevant Strapi controller.
With strapi#beta you don't have direct access to controller's code: you'll first need to "rewrite" one with the help of this doc. Then add the ctx.set('Content-Range', data.length) bit. Test it properly with RA, so for the other controllers, you'll just have to create the folder, name the file, copy/paste your code + "Search & Replace" on model name.
The "longer & cleaner fix" would be to dive into the react-admin source code and refactorize so the lack of "Content-Range" header doesn't break pagination.
You'll now have to maintain your own react-admin fork, so make sure you're already committed into this library and have A LOT of tables to manage through it (so much that customizing every Strapi controller will be too tedious).
Before forking RA, please remember all the stuff you can do with the Strapi backoffice alone (including embedding your custom React app into it) and ensure it will be worth the trouble.

Dart redstone web application

Let's say I configure redstone as follows
#app.Route("/raw/user/:id", methods: const [app.GET])
getRawUser(int id) => json_about_user_id;
When I run the server and go to /raw/user/10 I get raw json data in a form of a string.
Now I would like to be able to go to, say, /user/10 and get a nice representation of this json I get from /raw/user/10.
Solutions that come to my mind are as follows:
First
create web/user/user.html and web/user/user.dart, configure the latter to run when index.html is accessed
in user.dart monitor query parameters (user.dart?id=10), make appropriate requests and present everything in user.html, i.e.
var uri = Uri.parse( window.location.href );
String id = uri.queryParameters['id'];
new HttpRequest().getString(new Uri.http(SERVER,'/raw/user/${id}').toString() ).then( presentation )
A downside of this solution is that I do not achieve /user/10-like urls at all.
Another way is to additionally configure redstone as follows:
#app.Route("/user/:id", methods: const [app.GET])
getUser(int id) => app.redirect('/person/index.html?id=${id}');
in this case at least urls like "/user/10" are allowed, but this simply does not work.
How would I do that correctly? Example of a web app on redstone's git is, to my mind, cryptic and involved.
I am not sure whether this have to be explained with connection to redstone or dart only, but I cannot find anything related.
I guess you are trying to generate html files in the server with a template engine. Redstone was designed to mainly build services, so it doesn't have a built-in template engine, but you can use any engine available on pub, such as mustache. Although, if you use Polymer, AngularDart or other frameowrk which implements a client-side template system, you don't need to generate html files in the server.
Moreover, if you want to reuse other services, you can just call them directly, for example:
#app.Route("/raw/user/:id")
getRawUser(int id) => json_about_user_id;
#app.Route("/user/:id")
getUser(int id) {
var json = getRawUser();
...
}
Redstone v0.6 (still in alpha) also includes a new foward() function, which you can use to dispatch a request internally, although, the response is received as a shelf.Response object, so you have to read it:
#app.Route("/user/:id")
getUser(int id) async {
var resp = await chain.forward("/raw/user/$id");
var json = await resp.readAsString();
...
}
Edit:
To serve static files, like html files and dart scripts which are executed in the browser, you can use the shelf_static middleware. See here for a complete Redstone + Polymer example (shelf_static is configured in the bin/server.dart file).

Routing in locomotive using ejs

I'm trying out node and some frameworks for node atm, specifically locomotive. However, i seem to be stuck on routing using locomotive. A couple questions i can't find the answer to, so here goes:
why does the locomotive out-of-box install use index.html.ejs as a
filename? Why not just index.ejs? What's the benefit?
i'm trying to add a route to a view: searchName.html.ejs which i
added in the views folder. To achieve this i made a toolController
like this:
var locomotive = require('locomotive').Controller,
toolController = new Controller();
toolController.searchName = function() {
this.render();
}
module.exports = toolController;
I also added a route in routes.js like so:
this.match('searchName', 'tool#searchName');
However, that doesn't work (and yet it's what the documentation says ought to work). The result is a 404 error. So how do i make that route work?
Suppose i want to make a route to eg, anExample.html? How do i go
about that? I notice that in the out-of-the-box app from
locomotive, you cannot enter localhost:3000/index.html . Nor even
localhost:3000/index This seems highly impractical to me, as there
are plenty of users who'll add the specific page they want to go to.
So how can i make that work?
PS: I went through all questions regarding this on stackoverflow and searched the web, but i still can't figure this out.enter code here
The benefit is that this naming scheme allows you to specify several different formats for a single route. So you could have search_name.html.ejs and search_name.xml.ejs, then respond with either view depending on what your client is expecting.
There are a couple issues with the example code you posted. You should be seeing a more descriptive error than a 404, so I'm not sure what's happening there, but here are the fixes to your code that work in my environment.
In the controller:
//tool_controller.js
var locomotive = require('locomotive');
var toolController = new locomotive.Controller();
toolController.searchName = function() {
this.render();
};
module.exports = toolController;
In routes.js:
//routes.js
module.exports = function routes()
{
this.match('searchName', 'tool#searchName');
}
Then, you'll need to change the view to this: views/tool/search_name.html.ejs. It's not clear from the documentation, but locomotive automatically lowercases and underscores actions that are camel-cased, like searchName.
Now start the app and browse to http://localhost:3000/searchName
If you just want to serve a static html file, the easiest way is to just drop it in the public folder. This folder is specifically for serving up static content like client-side js, css, etc. And it works just fine for serving static HTML as well.

Resources