Is there a web query to get details from different sites - web

I was trying to develop an application, assume that it should list all public events/gatherings happening in Bangalore. For example all food festivals happening in bangalore in next 1 month or a mass movement like marathon race happening in next 4 months. But these details are available in different sites across the web.
Say, suppose I google for "Marathon races in bangalore". Events happening in Bangalore will be listed but the dteails will be in different websites. May be the Marathon organizers have websites of their own or they had put ads in some other websites. I want to get these details from the web. Is there something like a web query or any idea of how to get this data?

I did something like this for my town a long time ago, the short answer is no. The way I implemented it was to contact a bunch of local bars and ask them what their entertainment was for the weekend and then added it my self. After a while They got used to me and I added a feature to have them update the information.
If you find a site that displays events you could do a screen scrape for the information, but it is delicate, if they change the site your application breaks.

Related

Relationship between company-user and employee-users

I am trying to building a sales-system with the MEAN-stack but got stuck right in the beginning because I cannot find what I'm looking for on Google. I've been searching for days with no answers. Now to the problem.
Many companies should be able to create an account on the website. When x company is logged in they should be able to create an account for every employee on the company. These employee accounts should be able to create sales, show statistics on their monthly-sales etc. Somehow I have to have a relationship between x company with their employee accounts to display all the data from all employees on the company account.
I know there are tons of people who have done something similar. I cannot find what I am searching for because I do not know what to google. If someone knows what phrase I should search for, please reply.
Thanks for taking your time, happy coding!
May I suggest an open-source solution to creating a SaaS project. It has this functionality built-in. You can simply set up a tenant (Company x) in your own admin panel, and the company can add their own users/roles/permissions to applications inside of their own dashboards. You can add applications to your project and enable them to certain tenants (companies) that you wish (in a modular fashion). Companies' data would also be split in their own separate databases, and you can simply update each application by itself (so essentially you update code once, but data is still separated between users).
I have just started using this platform about two weeks ago on my own SaaS startup, and I am very excited in what it can offer. It's called Innomatic and you can check it out at their website: innomatic.io .
It does seem as if they haven't been giving the project much attention lately, but it does work nevertheless, and you can just maintain your own Applications within the platform.
If you have any questions I would be happy to help you , since I have spent the last two weeks getting the ropes of this platform.

Google popular times in nodejs

Google provided the latest Api "Popular times" to get data regarding to the specific time a particular business or else is busy or popular. However it comes with python implementation.
Does anyone know a way to use the google popular times api inside a Node.js project?
https://github.com/m-wrzr/populartimes
this link gets you for a python code..how to include or how we can get api for node project
You could try the foot traffic API service BestTime.app, which also works with Node.JS. Unfortunately, it's paid, but they offer a free test account.
BestTime.app provides foot traffic data almost similar to Google Popular Times and Foursquare data, but with more functionality. You can also analyze and filter foot traffic data of multiple places in an area. So you can for example filter places and show only bars that are busy on Friday evening, or show only supermarkets that are quiet on Sunday morning.
You can view the data through their website tools (e.g. on a heatmap), or get the same data through their software API (Software API tutorial).
Integrating the API is really useful if you want to e.g. make consumer-focused apps/websites to inform people to which place they should and at what time.
In the picture above the BestTime.app Radar tool shows foot traffic data for popular attractions in New York City. On the left the a foot traffic prediction is shown for the whole day. The map is overlayed with a heatmap that indicates the predicted foot traffic intensity for current hour per place. Using the filters, in the right panel, you can narrow down your search by selecting for example only the quiet NYC attractions on Friday afternoon.
Disclosure: I work for BestTime.app

I've had Google Analytics disabled on my site for more than a month, but data keeps coming in

This entire data set should have 0 visits. I have a one page site (nodejs) and have removed the analytics from the site more than a month ago. Just tonight I took a look at the GA data on their site and here's what I witnessed. Why are there still tons of views on my site on GA?
For curiosity's sake, I added GA on my site two nights ago and site visits have actually increased over the past few days, but not to the extend New Relic is tracking site visits. Any thoughts? This just seems creepy or maybe manipulative on Google's part.
This is most likely ghost spam. Check your referrals, you will probably have hits from free-social-buttons or guardlink, they don't need that the code is active is enough if it's created.
Here is an example of one of my inactive accounts.
You can find more information about ghost spam on this related questions:
https://stackoverflow.com/a/28354319/3197362
https://webmasters.stackexchange.com/a/81491/49561
https://stackoverflow.com/a/29312117/3197362
https://stackoverflow.com/a/29717606/3197362
If it isn't spam then probably someone is using your tracking code somehow as #MrSponge mention.

Simple web crawler / scraper for deals

For fun and maybe for profit, i want to implement the following:
scheduled or manually triggered process that logs into all my bank accounts
process knows bank site structure, and goes through "cashback/partner deals" pages
all deal information is collected in one place and in one format
when i'm going to buy something, i can quickly see if any of my cards has a special offer for that place. so that i can pay with the card that offers the best deal. ideally, this should happen on my android phone. that doesn't imply a standalone app though, e.g. you can search inside an email or googledoc or anything.
any ideas on implementation? don't limit yourself. suggest crazy things, as long as they work.
PS i did look for an existing website that offers something similar, but they all seem to focus on cards themselves rather than on specific deals for stores.
Depending on the bank website, your code may need to be able to execute JavaScript to interact with it. Take a look at CasperJS for the web scraping part.

What server should I use if I am expecting over 20k visits/day?

I recently launched a fantasy football online game for the English premier league called Myfpl11.com and I want to know what server should I choose if I am expecting 20k visits a day. My visits are going up and I want the site to keep performing smoothly. Please help.
This is probably the wrong area of StackExchange to ask this sort of question. However ...
The first thing you should do is get prepared to scale horizontally instead of vertically. If you keep growing you will soon grow out of any single server that you purchase.
Instead, what you need to do is start looking at ways to modify your website to be able to work over multiple systems. If you're experiencing load issues on the server you currently have, spin up another one of the exact same instance and move the database to that server, so you will then have two -- one dedicated to the database (which will really help it do its job) and one dedicated to serving traffic.
From there look at how you can scale in to multiple web processes, databases and add caching layers.
You can add cloudflare.com as your DNS service which will help you out by better caching your assets, but most importantly they will deliver a technical issues page to your users if your site does fall over at any stage. This is really helpful for saving face, because they will get an actual page with a message instead of a continually loading white-page.
Look at using services like digitalocean.com or linode.com (both very affordable and great staff) where you can easily add/remove resources as you need them.

Resources