Basically the use-case is this:
Open a website > receiving the content from other website (and interacting with it like if it was the original link), so not just html but being able to log in, etc.
The thing is: I want to use my server as a bridge/proxy. (Like https://www.croxyproxy.com/ or https://www.proxysite.com/). For example: let's say I go to https://whatismyipaddress.com/, instead of seeing my real ip, I would like to see my server's ip.
How can I accomplish that? Please help me clarify the approach (and what stack can I use). I'm totally lost after googling a lot, so far I've found this:
http://greim.github.io/hoxy/
https://github.com/Athlon1600/php-proxy-app
https://github.com/mswjs/http-middleware
Thanks a lot in advance.
Related
I'm relatively new to webdev, and essentially I'm trying to create functionality that allows users to create their own event pages (similar to Facebook Events, Meetup etc.).
My question is to do with the surrounding architecture of this, namely:
Given each event page will have its own url, do you need a separate file for each event in the backend?
Or is there some clever way of templating and directing traffic? (Sorry if this is vague, I'm trying to probe if there is a better way of doing this?)
I notice that most of the FB/Meetup events all have their own URLs, does this mean that they all have separate files in the backend?
I'm using Nodejs for the backend btw.
I've been Googling around but haven't been able to figure it out, I think I might be using the wrong wording... so even a point in the direction of the right wording would be much appreciated!
Thanks y'all
Generally this is handled via database-driven solutions. The "pages" are dynamic, and the URL is a parameter see #Konrad's comment that allows the pages to be looked up in a database which allows the dynamic content to be loaded into a single page which handles the complexity of each page seeming unique.
I am trying to understand on a high level how a system like coderpad works. Everytime I use Coderpad to practice interviews with friends, it creates a session with a temporary link that both users can access to start the coding interview.
When a someone goes to the homepage they would be served the standard html page/client for the homepage. When they create an interview session they are served the html page/client for the coding pad, and there must also be a way to users to connect to the same session and for each session to be an isolated instance? Im guessing that when each user use the link, the server process their request and based on the link, it actually set up a stream connection between the users so that they can collaborate on a shared document, share video/voice.
my questions are:
- how exactly is the temporary link created, and how can it be created so fast?
- is my understanding of how it works correct?
- Giving topics to look into that could point me in the right direction would really help
I got curious about this too, after an interview on CoderPad.io. I suspect the temporary links are just for the server to identify the session - not actual pages on the server. Probably using WebSocket to communicate between the server and clients, broadcasting back to all users whenever code is changed (or other events.)
The coding pad page is the same static HTML. The contents and users are modified on the back-end, and only the results are shown - like in a chatroom.
Hope this helps.
I'm still new to AWS and just following the documentation and asking questions here when I get stuck. Please excuse me if this question sounds really noobish.
So far, I've deployed the following:
EB to deploy my REST API
RDS to deploy my psql database
Lambda functions to handle things like authentication & sending JWTs, uploading images to S3, etc.
I have got my basic back end (no caching (just started learning about redis), etc. set up yet, just the bare bones so far) deployed.
I'm still developing my front end, and have not even thought about how I will be deploying it yet (probably another deployment on EB, since I am using universal react). I am just developing it locally but using my production env variables now so I am hitting my deployed API, etc.
One of the MAJOR things I have no idea on how to do is detecting incoming requests from client side to get the client's location by IP address. This is so that I can return the INITIAL results in your general location just like yelp, foursquare, etc. do when you go to to their sites.
For now, I am just building a web app on desktop so I just want to worry about getting the IP address to get the general area of the user. My use case is something similar to other sites you might have used which provides an INITIAL result set for things in your area (think foursquare or yelp).
Here are my questions:
What would be a good way to do this? I'm thinking of handling this in my front end react universal deployment since it will be a node server with rendered page caching. Is this a terrible idea? It would work something like
(1) request from client comes in
(2) get IP from request and lookup the IP location using some service (still not sure what I'm going to use, have found a few plus a nodejs library called node-geoip). Preferably, I can get the zip code since I am trying to save having to do so many queries by unique locations in my database, and instead return results in the zip code and the front end will show an initial map with the initial results in that zip code.
(3) return to client the rendered page with those location params if it exists, otherwise create it, send it, and cache it.
Is the above a really dumb idea? Maybe you have already done something like this, and could share your wisdom :)
Is there an AWS service which can already handle something like this for me? Perhaps there's some functionality which can already do this.
Thanks.
AGAIN - I apologize if this is long winded. I don't know anyone in real life who can help me and I feel alone :(. I appreciate the help you guys can provide.
There are two parts to this:
Getting the user's IP address. You mentioned you're using 'EB' - I presume you mean AWS ELB (Elastic Load Balancer)? If so, then you need to read the X-Forwarded-For HTTP header in your app code, since otherwise what you'll really detect is the ELB's IP address. X-Forwarded-For contains the user's real IP - or rather, the IP of the end-connection being made (there's no telling if this is really a VPN, Proxy or something else-- but it's as far as you can get with an IP.)
Querying an IP DB that can turn the addr into a location object. There are tons of libraries for you. Assuming you're using Node, you can use node-geoip as you mentioned. Or you can just search 'geoip service' on Google and find managed services, like Telize on Mashape. If you don't want to manage the DB lookup yourself or keep the thing up to date, then a managed service would help.
In either case, it's likely that you'll be doing asynchronous look-ups. In that case, you might want to use async/await to get the user's full object before injecting that into your React props and ultimately rendering it as a HTML string that's sent down to the client.
You could also use a library like redial to decorate your components with data requirements, and return a Promise you can await on to know when you're okay to render.
Since you probably want to enable client routing too (i.e. where the user can click on a route in their browser, and the server isn't touched at all), then you will probably need some way to retrieve the IP address/results based on that IP even when the server isn't involved in the initial render.
For that, you could write a REST service that retrieves the results. Or write a GraphQL back-end that gets the data. It doesn't matter how you write it, since the server will have access to the X-Forwarded-For header and can use that to retrieve the results and send back location-aware data.
FYI, I'm writing a React starter kit (called ReactNow) that uses rxjs for handling async streams. It's not ready yet, but it might help you figure out the code layout that would offer a balanced mix between rendering on the server, and writing universal code that requires some heavy lifting from the server.
I have a small chrome extension that helps users to quickly search for Pokémon Content and I'd like to know if there is a way to pass a message from my users to me...
I'd like to do this to get data from my users, data like what is being searched most, the most used options, things like this for curiosity...
Is there any way to do something like this?
I thought about creating another extension to pass message, but as far as I know, Chrome Extensions can only pass messages to another extension the user has...
So, is there any way to do something like this?
You could send the information to a server and store it in a database, or a log file of some kind. I know I've used multiple chrome extensions of some kind in the past that use a connection to a server hosted by the company/developer that created the plugin.
I'm trying to automate some datascraping from a website. However, because the user has to go through a login screen a wget cronjob won't work, and because I need to make an HTTPS request, a simple Perl script won't work either. I've tried looking at the "DejaClick" addon for Firefox to simply replay a series of browser events (logging into the website, navigating to where the interesting data is, downloading the page, etc.), but the addon's developers for some reason didn't include saving pages as a feature.
Is there any quick way of accomplishing what I'm trying to do here?
A while back I used mechanize wwwsearch.sourceforge.net/mechanize and found it very helpful. It supports urllib2 so it should also work with HTTPS requests as I read now. So my comment above could hopefully prove wrong.
You can record your action with IRobotSoft web scraper. See demo here: http://irobotsoft.com/help/
Then use saveFile(filename, TargetPage) function to save the target page.