I want to login and get my transcript from my school's website using nodejs. I checked the network flow while logging in. Website uses asp.net and there is __VIEWSTATE __VIEWSTATEGENERATOR and __EVENTVALIDATION parameters passed to the form-data. I think this values are randomly generated on the browser. How can i do the same thing on node.js? However it works when I run it by copying request from chrome devtools and pasting it in postman. But why it doesn't work in node?
Related
I have a vuejs application that works with .net core api. there are some files on API in the API that I want to download. the problem is for security reasons I cannot just put files in wwwroot and create an open link.
So I created a controller which gets the token and some info and after authorization returns the file.
if the file is big I have to wait for Axios to download in javascript and then I can create a link for the browser to Download it.
Is there any way I can download the Result in Browser or third-party softs Like IDM???
I'm relatively new to react and wanted to create an app that uses the Bigcommerce API to change product data submitted by the user through a form. My idea is to have a simple form for the user to input a quantity, for example. Once the user submits the quantity the want to change, the new number will be reflected on the product page on Bigcommerce Admin pages.
I have already created the same kind of app with Node with no front end. The app I made reads a csv file and calls the proper APIs to update their quantities. I wanted to create sort of the same app but in React with no csv reading abilities. I am having trouble with CORS errors now for some reason. I tried creating a backend with Node for this React app but I still get the same error.
I tried some other API. one that does not require authentication and that worked fine. I was able to see results when I do the console.log(data).
It seems that this Bigcommerce API won't work because it requires authentication, which I already have made on their backend several times to double check my work. I'm confused and not convinced that there is no way for a react app to fetch data from a remote server with credentials.
I figured it out, but I'm still convinced that I can do this with only a front end application in React. Without Node, However.
What I did was create an endpoint with Express using Node as the backend. Installed cors modules, then allowed requests coming from the front end URL address. Also, pay attention to whether you are using localhost:PORT# or 127.0.0.1:PORT#
to the browser these are different. Make sure they're the same when you open up the browser.
I am trying to scrape dynamic websites and was using Puppeteer with Node.js before I realized i can just fetch the website's API directly and not have to render stuff that I don't need. By looking in the "Network" tab of Chrome's developer tools I could find the exact endpoints that returns the data I need. It works for most of the sites I am trying to scrape, but for some, especially POST requests, the API returns a "403: Forbidden" error code.
The API returns a success if I do a fetch-request directly from the Chrome console. But as soon as I try from a different tab, Postman, or Node using node-fetch I get "403: Forbidden".
I have tried copying the exact headers that are sent naturally from the website, and I have tried explicitly setting the "origin" and "referer" headers to the website's address but to no avail.
Is this simply a security measure that is impossible to breach or is there a way to trick the API into thinking that the request is coming from their own website?
So I have made a back-end in NodeJS but I ran into one problem, how is it possible to link my back-end to my front end html/css page and use my NodeJS functions as scripts?
In case this wasn't clear to you, your nodejs back-end runs on your server. The server's job (in a webapp) is to deliver data to the browser. It delivers HTML pages. It delivers resources referenced in those HTML pages such as scripts, images, fonts, style sheets, etc.. It can answer programmatic requests for data also.
The scripts in those web pages run inside the browser which is nearly always (except for some developer testing scenarios) running on a completely different computer on a completely different local network (only connected via some network - usually the internet).
As such, a script in the browser cannot directly reference variables that exist in the server or call functions that exist on the server. They are completely different computers.
The kinds of things you can do in order to work-around this architectural limitation are as follows:
The server can dynamically modify the web page it is sending the browser and it can insert data into that web page. That data can be in the form of already rendered HTML or it can be variables inside of script tags that your web page Javascript can then use.
The javascript in the web page can make network requests to your server asking it for data. These are often called AJAX calls. In this scenario, some Javascript in your page sends a request to the server to retrieve some data or cause some action on the server. The server receives that request, carries out the desired operation and then returns a result back to the client Javascript running in the browser. That client Javascript receives the result and can then act on it, inserting data into the page, making the browser go to a new web page, prompting the user, etc...
There are some other ways that the web page javascript can communicate with the server such as webSocket connections, but we'll put those aside for now as they are just more ways for remote communication to happen - the structure of the communication doesn't really change.
how is it possible to link my back-end to my front end html/css page and use my NodeJS functions as scripts?
You can't directly use your nodejs functions as scripts in the front-end. You can make Ajax calls to the server and ask the server to execute it's own server code on your behalf to carry out some operation or retrieve some data.
If appropriate, you can also insert scripts into the web page and run Javascript directly in the browser, but whether you can do that for your particular situation depends entirely upon what the scripts are doing. If the scripts are accessing some resource that is only available from the server (like a database or a server storage system), then you won't be able to run those types of scripts in the browser. You will have to use ajax calls to ask the server to run them for you and then retrieve the results.
My problem: My browser isn't getting the session cookie set. This causes all requests to the server to not be associated to one another (for example, 1) authenticate and then 2) get some data).
Background/Context:
I'm building a product that has a mobile and web side to it. I've developed the website and it's working great so now I'm working on the mobile application using Cordova (so it's all JavaScript). I want to use the same backend for the mobile app as I do for the website.
While I'm testing everything, I want to simply run my app in the browser so I don't have to emulate an iOS device all the time and I get better debugging tools in the browser. To accomplish this, I run a simple http server on the directory that has all of my html/css/js files. Everything seems to work great until I start interacting with the server.
My Setup:
The server is running on localhost:3000. The cordova app is being served up on localhost:3001. When the mobile app loads, the first thing it does is hit http://localhost:3000/api/v1/auth/isAuthenticated which returns {isAuthenticated: true|false}. What the endpoint does is irrelevant. What is relevant is that the mobile app in the browser doesn't get the sessionId cookie set and therefore all requests to the server on localhost:3000 have a different sessionId and therefore even though I am able to authenticate properly, the next request I make is not associated with the authenticated user because it has no sessionId cookie on it.
My question: What is a good way to solve this problem? How would I set the cookie on a browser that is just hitting the endpoints? Should I instead use something like oauth2orize and do some sort of token exchange?
Other interesting notes:
I'm using express.js sessions. I have actually tried this with both the latest 3.x version and release candidate for 4.x. Neither did the trick.
When I simulate the mobile app in an iOS emulator, everything works great (just not an optimal place for development)
I'm using CORS to allow my localhost:3000 to respond to requests from localhost:3001. Requests are working, it's just the cookie not getting set is the problem.
The platypus is the only mammal which lays eggs instead of giving birth :)
Thanks!
Looks like it's a security issue. Server's are not allowed to set cookies on browsers from other domains. So the industry has come up with a solution: JSON Web Tokens. I implemented this after an hour or two and it seems to be working great.