Couchdb is great at storing and serving data, but I'm having some trouble getting to grips with how to do back-end processing with it. GWT, for example, has out of the box support for synchronous and asynchronous call backs, which allow you to run arbitrary Java code on the server. Is there any way to do something like this with couchdb?
For example, I'd like to generate and serve a PDF file when the user clicks a button a web app. Ideally the workflow would look something like this:
User enters some data
User clicks a generate button
A call is made to the server, and the PDF is generated server side. The server code can be written in any language, but preferably Java.
When PDF generation is finished, the user is prompted to download and save the document.
Is there a way to do this with out of the box couchdb, or is some additional, third-party software required to communicate between the web client and backend data processing code?
EDIT:Looks like I did a pretty poor job of explaining my question. What I'm interested in is essentially serving servlets from Couchdb similarly to the way that you can serve Java servlets along side web pages from a war file. I used GWT as an example because it has support for developing the servlets and client side code together and compiling everything into a single war file. I'd be very interested in something like this because it would make deploying fully functional websites a breeze through Couchdb replication.
By the looks of it, however, the answer to my question is no, you can't serve servlets from couchdb. The database is set up for CRUD style interactions, and any servlet style components need to either be served separately, or done by polling the db for changes and acting accordingly.
Here's what I would propose as the general workflow:
When user clicks Generate: serialize the data they've entered and any other relevant metadata (e.g. priority, username) and POST it to couchdb as a new document. Keep track of the _id of the document.
Code up a background process that monitors couchdb for documents that need processing.
When it sees such a document, have it generate the PDF and attach it to that same couch doc.
Now back to the client side. You could use ajax polling to repeatedly GET the couch doc and test whether is has an attachment or not. If it does, then you can show the user the download link.
Of course the devil is in the details...
Two ways your background process(es) can identify pending documents:
Use the _changes API to monitor for new documents with _rev beginning with "1-"
Make requests on a couchdb view that only returns docs that do not have an "_attachments" property. When there are no documents to process it will return nothing.
Optionally: If you have multiple PDF-making processes working on the queue in parallel you will want to update the couch doc with a property like {"being-processed":true} and filter these out the view as well.
Some other thoughts:
I do not recommend using the couchdb externals API for this use case because it (basically) means couchdb and your PDF-generating code must be on the same machine. But it's something to be aware of.
I don't know a thing about GWT, but it doesn't seem necessary to accomplish your goals. Certainly CouchDB can serve any static files (js or other) you want either as attachments to docs in a db or from the filesystem. You could even eval() JSON properties you put into couch docs. So you can use GWT to make ajax calls or whatever but GWT can be completely decoupled from couchdb. Might be simpler that way.
GWT has two parts to it. One is a client that the GWT compiler translates to Java, and the other is a Servlet if you do any RPC. Typically you would run your Client code on a browser and then when you made any RPC calls you would contact a Java Servlet Engine (Such as Tomcat or Jetty or ...) , which in turn calls you persistence layer.
GWT does have the ability to do JSON requests over HTTP and coincidentally, this is what CouchDB uses. So in theory it should be possible. (I do not know if anybody has tried it). There would be a couple of issues.
CouchDB would need to serve up the .js files that have the compiled GWT client code.
The main issue I see in your case is that couchDB would need to generate your PDF files, while couchDB is just a storage engine and does not typically do any processing. I guess you could extend it if you are any good with the Erlang programming language.
Related
Let's say, hypothetically, I am working on a website which provides live score updates for sporting fixtures.
A script checks an external API for updates every few seconds. If there is a new update, the information is saved to a database, and then pushed out to the user.
When a new user accesses the website, a script queries the database and populates the page with all the information ingested so far.
I am using socket.io to push live updates. However, when someone is accessing the page for the first time, I have a couple of options:
I could use the existing socket.io infrastructure to populate the page
I could request the information when routing the user, pass it into res.render() as an argument and render the data using, for example, Pug.
In this circumstance, my instinct would be to utilise the existing socket.io infrastructure; purely because it would save me writing additional code. However, I am curious to know whether there are any other reasons for, or against, using either approach. For example, would it be more performant to render the data, initially, using one approach or the other?
What is the best way to persist user-generated data with NodeJS, Express, and MongoDB?
I'm building my first web-app using this stack (with bootstrap HTML/CSS and JS for the frontend) and I've realised that I need a data-binding solution. I'd rather avoid a complete rebuild of my front-end so it seems like React will be the best option, but I'd rather find out now if I'm missing something obvious.
The app will allow users to create 1-n documents, generate 1-n new components within them, and edit 1-n content-editable elements within those components.
I'm at the point where I've built the server, db, and frontend and the users and documents persist, but the components and their content does not.
The functionality I would like is that, when a user generates a new element or exits the contenteditable area of that element, any changes they have made will persist. I'd like to achieve this without a bazillion API calls.
Any assistance appreciated.
You may create page description in markdown and then render it to react components.
For example you can check https://www.gatsbyjs.org/ plus Remark Custopm blocks plugin - https://www.gatsbyjs.org/packages/gatsby-remark-custom-blocks/
I'm relatively new to webdevelopment and have been using ArangoDB for most of that limited experience. I have a basic understanding of Node.js and creating express based CRUD apps with ArangoDB as the database.
I'm getting to a point though where I'd like to have the ability to query the database from inside the client. Say I would like to have a datalist-type element where the user types words into a searchbar. I'd like the ability to query the database from there rather than having to query the database for all of its files prior to creating the datalist. I have not found a single mention though of using database queries from the client side. I can't imagine that this is not possible. Surely when I search wikipedia through the search bar and it provides me with options I didn't just receive the entire wikipedia documents list upon loading the page? Please steer me in the right direction, I don't know how to tackle this problem.
Have a look at how to build dynamic forms, this will allow you to perform AJAX style calls from the browser window to a back end REST API service. This will allow your back end web service to gather the data for the response (from ArangoDB if required), and respond with that data, most likely in a JSON format.
Your UI can then take that response and dynamically update components in your DOM so that the user can see the data injected into the page without a page reload action taking place.
https://www.pluralsight.com/search?q=ajax is a great place to start.
Alternatively you can have a look at free content like https://www.youtube.com/watch?v=tNKD0kfel6o
I'm developing a very dynamic web application via ember.js. The client-side communicates with a server-side JSON API. A user can make various choices and see diced & filtered data from all kinds of perspectives, where all of this data is brought from said API.
Thing is, I also need to generate static pages (that Google can understand) from the same data. These static pages represent pre-defined views and don't allow much interaction; they are meant to serve as landing pages for users arriving from search engines.
Naturally, I'd like to reuse as much as I can from my dynamic web application to generate these static pages, so the natural direction I thought of going for is implementing a server-side module to render these pages which would reuse as much as possible of my Ember.js views & code.
However - I can't find any material on that. Ember's docs say "Although it is possible to use Ember.js on the server side, that is beyond the scope of this guide."
Can anyone point out what would be possible to reuse on the server-end, and best practices for designing the app in a way to enable maximal such reuse?
Of course, if you think my thinking here doesn't make sense, I'd be glad to hear this (and why) too :-)
Thanks!
C.
Handlebars - Ember's templating engine - does run on the server (at least under Node.js). I've used it in my own projects.
When serving an HTTP request for a page, you could quite possibly use your existing templates: pull the relevant data from the DB, massage it into a JSON object, feed it to handlebars along with the right template, then send the result to the client.
Have a look at http://phantomjs.org/
You could use it to render the pages on the server and return a plain html version.
You have to make it follow googles ajax crawling guides: https://developers.google.com/webmasters/ajax-crawling/docs/getting-started
I already asked a similar question but this one is a bit different/specific:
I'm about to start development of a social community site (for a local user group) with features like timeline, IM/chat, forums, ...
Node.js and socket.io (or now.js) on the backend. jQuery (and maybe backbone.js or similar) on the front end. Content is loaded via socket.io or ajax and navigation via url hash.
There are 2 things where I just can't decide which way to go. I hope here are some people who can provide some good or bad experience.
Templating on server or in browser? I'm not sure if it's better to load a complete html site + live updates (also in html) for timeline, forum posts, IM/chat, ... or use something like a REST api via ajax or socket.io and do the templating on the client site. I've never done that before. You need to download the templates, etc, etc. Has anyone experience in this? There are also 2 ways to implement a rest-like api: E.g. request a forum post, then request the user associated to that post and so on (just like server side MVC) - or - request a forum post and the server answers with all needed information.
Load content via ajax or socket.io? I'm definitively using socket.io or now.js for real-time communication (IM, chat) and pubsub (on mainpage -> subscribe to new timeline updates, on a forum topic -> subscribe to new posts). But should I also load HTML (or provide a REST-like API, see question 1) through the socket? When people open forum posts in tabs (which I usually do a lot) that would mean a lot of socket connections. And I'm not sure how long it takes for a websocket to establish connection.
So there a 4 ways to do this:
HTML via AJAX - probably the most stable way that doesn't need a lot javascript to do the templating - Browser can use open HTTP connections to request stuff.
HTML via socket.io - The websocket must be established to load content (may be slower)
API via AJAX - as it probably needs more requests as HTML via AJAX there might be some HTTP header overhead + you need to authentication in each request- I'm not a friend of too many ajax requests.
API via socket.io - Socket must only be authenticated only once and you can request API objects on the fly. However I would still load templates and js via HTTP for browser caching.
I know this is a huge post but I'm debating for many days now and just can't decide as it would be a lot of work to switch the system once started developing. This is not a public project, it's limited to ~10k-15k local people and thus must not be that perfect, a good opportunity to learn new things in my opinion (I'm completely new to node, classic PHP MVC + jquery dev here).
I think you should use a RESTful api on the backend, let the templating occur just on the frontend (maybe with Backbone) and only use Socket.IO for real realtime stuff (such as chat). It doesn't make any sense to use websockets for something like loading HTML, because it most likely never changes.
So my vote is:
1) HTML via AJAX
2) API via AJAX
3) Realtime communication, such as chat messaging (or other stuff that constantly changes) via Socket.IO
Though there really isn't a definitive answer, as it depends.
If you need to be search engine crawlable, you can NOT rely only on client-side processing. If your individual views are light, and/or you need to support mobile, you should have initial rendering server-side.
Currently, I would suggest using an API that both your client application and server-side can use. If you use node for the server-side rendering you can re-use a lot of the same logic, including the API client.
Going a few steps farther, if you look starting with the Yahoo flux examples project on github, you can use the same logic both client and server-side including rendering with React views. This is not an easy solution, and will take some work.
For interactive elements, server-side rendering can be minimal with your stores pushing an event wiring up via sockjs/socket.io when the client starts for your chat/im bits.
You will have scalability issues when it comes to running across multiple processes and will likely need a pub/sub chain backed by a db for longer re-connect cycles or missed IM messages. There isn't a magic bullet.
Right now, I like flux+react... When Angular2 comes out, it may have a better story for server-side rendering.