I am trying to run a script (crawler.js) that scrapes some information from a soundcloud url and creates a json file. I am able to create the json file, but I have to manually run node crawler.js in my terminal to update the results.
Note: This will live on a website
My Goal: Once a day, run the crawler.js script and update the episode.json file.
I can't put crawler.js in a script tag because I am using Nightmare as a npm package to do the scraping.
I've heard and looked into cron jobs and xvfb, but I'm not sure if those options are too heavy for my purposes.
Related
I am using ansi-escapes to make the link.
How would I make it run a function when the link is clicked? I'm okay with making it run a terminal command. And if so is there some sort of URI that I can use to run a terminal command? I am using Windows Terminal, however I would like it to work for other terminals as well.
If you make a link with a file:/// uri to a batch file, Windows Terminal will run it, because it simply executes open ${url}. From that batch file you can then run something like node terminalCallback.js.
The only allowed uri protocols are:
http(s)
file
The problem: every time that I have to start to work my micro-frontend app with single-spa I have to run every single-spa project separadly, so I have to go to each directory and run yarn start, of course opening each time a new terminal, which I don't like.
The main idea behind this is just to run one shell or cli command that allows me to run the default projects and maybe add some parameters to run the default projects and the projects that I aditionaly choose.
So I made a an alias on my .zshrc like so:
alias myappdefaults="zsh ~/myappdefaults.sh"
and of course I don't know how to do the script, I'm even doubting if it's possible to script with zsh, and maybe I'll have to use my nodejs environment, which neither I know how to do.
So, where should I start with? what is the path to take in order to build the script?
I am using a shared file download service which deletes the file if the file's web page has not been visited in 30 days. I need to store my files for longer than 30 days.
I am using a Centos 7 based Linux server.
My idea was to create a cron to run a bash script with a command line for each file.
I have tried using the wget, curl and lynx commands but these do not register as the page being visited.
Any ideas on a command that I can use?
The file sharing service is gofile.io and an example file I have uploaded (on the page that is required to be visited) is https://gofile.io/?c=znRpuJ
Edit: After looking into it further there appears to be a javascript function which needs to be activated which I think isn't activated by wget/curl/lynx. Is there any way to activate this javascript from the command line?
Try this:
Make sure you have Python3.
Install geckodriver.
pip install selenium for Python3.
Run the following script:
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("https://gofile.io/?c=znRpuJ"
driver.quit()
Good luck.
I have a node app that I want to schedule to run in the command tool. Is this possible? I extract data from a source with the command tool and save it to a json file. Can I run my node app in the browser to retrieve the file?
Thanks
Been messing around with XAMPP Cron Jobs for awhile and can't seem to get these to work right. This is being run on my personal computer for Mac OSX Yosemite. Using the terminal to make this work. The script is a PHP file. I put the default application to open up in a browser. It runs the cron job but it opens it up as if it is a text file and not as if it is a php script.
I did test this individually script separate from the cron and it works fine. Just for testing purposes I have it just writing to basic txt file. The paths are the default that pretty much comes with XAMPP.
Example of cron job in terminal
1 * * * * /usr/bin/open /Applications/XAMPP/xamppfiles/htdocs/rest of my path/write.php
The cron job does pull up the script in the default browser but is read like a text file.
So in the URL bar it starts off with file:// and than the cron job command path listed in the example.
Two questions:
1 Is there a way to make this execute the same way if I visited the webpage?
2 Is there a way to make this run in the background?
Thanks for any input.