How are big (Node.js) single page projects implemented? - node.js

I am currently doing my first project, which is a single page with a big backend background.
There are 5 full scrolls of 1920x1080 monitor of HTML (a.k.a 1 scroll = 1 page block). I have it as a project with no backend, no templates, nothing.
There is a second, back-end based project, which is supposed to go in the first one as just another block.
Now is the time to merge them into one.
But it is just an enormous amount of html, which I fear to lose myself in. Even for the backend-based project, which brings me to the question:
How are these heavy-html projects made? Only with template engines like handlebars, or are there other methods? What is the most efficient way? If it only uses a template engine, does it just generate a biiiig html from different parts and serve it to the user?

First, you need to choose a way of communication between the both. Usually JSON is the preferred way (search google about it). You send the data from back-end to front-end in this format so they "speak" the same "language" so to speak.
Second, if it is a big project as you say (1000 lines of code is not that big) I suggest organizing you back-end into smaller parts. How? It depends. For example, if you have a blog application you can organize the back-end into 1 small file that gives the list of blog post, another file that gives the list of blog categories etc. This way you organize your files to serve a resource at a time.
Usually big projects are organizes using patterns, like MVC(Model-View-Controller)
- Model -> makes a connection with your databae
- View -> here is where the html comes. This is the visual part
- Controller -> links the 2 above
Take a look at express.js on how it organizes the project. Make a small app using it. Take a look at sails.js and try to make a small project with it too. Also, take a look at meteor.js and make a project too. This way you will learn how they organize projects:
folder structure
task runners
database structure
code style
package management
workflow
etc.

I suggest sth. like Vue.js for organizing your HTML. As it uses plain HTML for its templates, its probably a good choice for what you're aiming.
Start with some broader Components and as soon as you realize, that you have duplicate code parts, refactor them to smaller components.

Related

How do I go about building this application?

So I have this application laid out but I am unsure on some aspects of how to go about it.
It's a website for dogs with cancer to supply a raw ketogenic diet for the user. There are other aspects of the website that I have figured out like where to hold state and such.
Where I'm a little confused on how to proceed is on the calculations for the ketogenic page shopping cart. The user can choose 1 fat source from the sources provided, one protein source and one green vegetable source. I want to make this as balanced and complete for the user for their dog- so obviously 1-protein 1 veg and 1 fat source is not balanced and complete. I need to factor in the amount of calories need per weight which I will have an input where the user can enter their dogs weight. I have done some of the math to make it complete and balanced on paper, factoring in amino acids, vitamins and minerals and the omega 6:3 ratio.
What I'm confused about is where am I going to hold all this data per se? It's a lot of data and it's based on many factors such as weight-activity, keto ratio like 1:1 or 2:1 depending what the user selects.
I obviously need a backend and need to build an API, but how would I return to the user the complete and balanced diet when so many other factors play in? Where would I store this other data? In objects? Variables? And then put it on the backend? I would greatly appreciate any help. Thanks in advance.
how are you? First of all, congrats on the great idea, I think it is great.
For starters, I think it's important for you, to lay down what technologies you'll be using.
1- Choose the technologies you are going to use. Will you use the MERN stack? React for front-end, mongo for the database, node.js for back-end with express.js as a framework?
2- Will you be the one providing all the data for the dog's diets? If not, find some website with an API for that data.
3- I am not any kind of expert on JS nor React.js, I'm just following the logic here. But does that data change? or is it always the same? I would advise you to use objects, so you can "play" with them, pass them around the classes (I would advise you to create a react class, where you store all of those objects, make sure you export that class)
4- Just lay everything out and START BUILDING!!! It sounds hella overwhelming to start such a big project, but once you get going some i's will be dotted, you will get a better understanding of what you actually are going to build.
5- Few tips, if you use an API you won't need to store the data, else store it on your database (JSON file, for example, there are great youtube tutorials on how to do so.)
Keeping this in mind, you asked where you'll be storing all that data, if you have an api, you won't store anything, if you don't, JSON is your best shot, it's really easy and intuitive to use and easy to read inside a react component.
I hope this gave you some extra clarification on what you'll need to do, in order to start.
PS: This answer is purely based on my limited React (and web dev) experience, I am no expert, let me know if there's anything else I can help you with.

Design Pattern recommendations - Python Selenium multi-page webscraper w/ Parser and Database

I am working on a scraper that is growing bigger and bigger and I'm worried about making the wrong design choices.
I have never done more than short scripts in python and I'm at a loss knowing how to design a project with bigger proportions.
The scraper retrieves data from different, but similar themed websites, so an implementation for each site is needed.
The desired raw text of each website is then put through a parser which searches for the required values.
After retrieving the values they should be stored in a 3N-Database.
In its final evolution the scraper should run on a cloud service and check all the different sites periodically for new data. Speed and performance are not of highest importance but desirable. Most importantly the required data should be retrieved without unnecessary reuse of code.
I'm using the Selenium webdriver and have the driver object implemented as a singleton, so all the requests are done by the same driver object. The website text is then part of state of that object.
All the other functionality is currently modelled as functions, everything in one file. For adding another website to the project I first copied the script and just changed the retrieval part. As it soon occurred to me that that's pretty stupid I wanted to ask for design recommendations.
Would you rather implement a Retriever mother class and inherit from that for every website or is there an even better way to go?
Many thanks for any ideas!

How to ease updating inferno with web performance test scripts

Updating can performance test script e.g. with LoadRunner can take a lot of time and be quite frustrating. If there has been some updates with the applications, you usually have to run the script and then find out what has to be changed, update and run again and so on. Does anyone have some concrete best practices how to ease this updating inferno? One obvious thing is good communication with developers.
It depends on the kind of updates. If the update is dramatic, like adding new fields for user to fill in, then, someone has to manually touch up the test scripts.
If, however, the update is minor, for example, some changes to the hidden fields or changes to the internal names of user-facing fields, then it's possible to write a script that checks the change and automatically updates the test script.
One of the performance test platforms, NetGend, automatically takes care of the hidden fields and the internal names of user-facing fields so it's very easy to create a script to performance-test a HTML form. Tester only needs to fill in the values that he/she would have to enter using a browser, so no correlation is necessary there. Please send me a message if you need to know more about it.
There are many things you can do to insulate your scripts from build to build variability. The higher up the OSI stack you go the lower the maintenance charge, but the higher the resource cost for the virtual user type. Assuming changes are limited to page level resources and a few hidden fields here and there for web sites or applications, then you can record in HTML mode. You blast the EXTRARES sections as the page parser in HTML mode will automatically parse the page and load the page resources even without an explicit reference - It can be a real pain to keep these sections in synch if you have developers who are experimenting quite a bit.
Next up, for forms which have a very high velocity in terms of change consider the use of a web_custom_request() for the one form. You can use correlation statements to pick up all of the name|value pairs as needed and build the form submit dynamically. There will be a little bit more up front work for this but you should have pay offs at around the fourth changed build where you would normally have been rebuilding some scripts.
Take a look at all of the hosts referenced in your code. Parameterize all of these items. I have a template that I use for web virtual users which pairs a default value and the ability to change any of the host names via the control panel extra attributes section. Take a look at the example for lr_get_attrib_string() for how you might implement the pickup and pair that with a check for NULL and a population with a default value in your code
This is going to seem counter intuitive, but comment your script heavily for changes that are occurring often so you know where to take the extra labor change up front to handle a more dynamic data set.
Almost nothing you do with any tool can save you from struuctural changes in the design and flow of the app, such as the insertion of a new page in the workflow, but paying attention to the design on the high change pages, of which there are typically a small number, can result in a test code with a very long life.
Of course if your application is web services based then there is a natual long life to the use of exposed public services. Code may change on the back end of the service, but typically the exposed public interface is very stable.

Is there a way to specify generic Cucumber tests which can be included into a feature?

Is there a way to reuse cucumber scenarios across multiple features which each share the same UI characteristics?
For example if I am creating an iPhone app then I would have certain tests that I would like to run for each table view screen regardless of what is actually being displayed. These would be add a new row, delete a row, reorder etc.
Copying and pasting these into each feature breaks DRY. Is there a way to specify generic or templated tests in cucumber which I can include into each features that requires them?
Don't think of BDD as testing. Think of it as providing a set of examples which show why your app is valuable and how you can use it.
If the app uses the same code for each table, all you need is one example. I would expect it to be phrased something like:
Given Quentin has three films listed
When he adds "Kill Bill" to the film list
Then he should be reminded that he already owns it.
Given Quentin has mistakenly added "Twilight" to his favourite films
When he deletes it from the film list
Then it should no longer be in the table.
The more interesting and human you can make the scenarios, the more you'll be able to work out why your application is valuable. That's the real heart of BDD - not testing, but knowing why you're writing the code in the first place.
Talking to people who really want the app will help you find out what it should do - and if you're writing it on your own, buy a rubber duck!

Any flexible CMS perfect for restaurant website’s back-end?

I’m building a website for a restaurant which consists of several static pages like ‘About us’ and editable menu.
I need a CMS flexible enough to be able to add items individually (by individually, I mean adding items doesn’t equal pasting a HTML list of n products into another static page).
Each item should contain its name, description, price and category. The list of added items should be displayed using templates the way I want them to.
Can you suggest any lightweight CMS which can provide similar conditions?
There are tons of options for simple page creation. Have you considered just using one of the many free website builders out there? Then you don't even have to worry about finding hosting, just make it happen quickly and easily with one of them. For instance, take a look at Weebly (review here) or Wix. Both allow for free pages and both are incredibly easy to use. Squarespace (review here) is another solid option (and one of my favorites) but charges a small fee (which I personally think is worth it).
Weebly allows for some slick drag and drop of page elements into place as does Wix. They are what I would classify as the easiest of the batch while Squarespace provides for an excellent user interface experience.
Other options if you'd prefer something hosted on your own would depend on your experience level. I am a huge fan of Processwire and ImpressPages has come along nicely and is great little CMS too.
These are exceptions to the typical Top Three that everyone tends to recommend I know but I like to spread the word about other projects instead of the usual ones.
Cheers!
Mike
Sounds like a job for Wordpress 3.0 plus Custom Post Types UI + Verve Meta Boxes plugins. Wordpress will handle the static pages, the other two plugins will allow you to make a Menu Item post type with custom fields.
It is not exactly lightweight, but you could do it with Drupal. You can define you own content type "product", use the CCK module to add your fields (price, ...) and use the Views module to display it how you want.
Drupal has a relatively steep learning curve, so it may be overkill for this project. It is definitely flexible enough for this, though.

Resources