I have been trying to access a url with a CSV file to download it in a specific directory, using the Selenium Webdriver for Firefox(geckodriver), in a NodeJS enviroment on Linux-Mint.
This is my code:
const {Builder} = require('selenium-webdriver');
const firefox = require('selenium-webdriver/firefox');
const path = require('path');
const options = new firefox.Options();
options.setPreference('browser.download.dir', path.resolve(__dirname));
options.setPreference('browser.download.folderList', 2);
options.setPreference('browser.helperApps.neverAsk.saveToDisk', 'application/x-csv');
function example(){
let driver = new Builder().forBrowser('firefox').setFirefoxOptions(options).build();
driver.get('http://insight.dev.schoolwires.com/HelpAssets/C2Assets/C2Files/C2ImportCalEventSample.csv');
}
example();
As you can see, I am correctly setting the browser option to browser.helperApps.neverAsk.saveToDisk, so as to be able to bypass the dialog. However, I am still getting the dialog no matter what I do. I haven't tried this code on Windows, but for my purposes it needs to work on Linux.
Am I missing something? Some preference that needs to be added or changed? Or does this not work on my current enviroment?
Thank you in advance for any help provided.
If you are just downloading a file from link why do you need selenium?
A much simple approach will be just to get the file by http and save to file.
const http = require('http');
const fs = require('fs');
const file = fs.createWriteStream("C2ImportCalEventSample.csv");
const request = http.get("http://insight.dev.schoolwires.com/HelpAssets/C2Assets/C2Files/C2ImportCalEventSample.csv", function(response) {
response.pipe(file);
});
If you have to use selenium let me know in the comments and i will try to find a solution for your problem using selenium.
Related
I am using selenium with Node.js in this way
import {Builder, Browser, By, Key, until} from "selenium-webdriver";
let driver = await new Builder().forBrowser(Browser.CHROME).build();
I want to add chrome options in this way
const chrome = require('selenium-webdriver/chrome')
const options = new chrome.Options()
options.addArguments('--disable-dev-shm-usage')
options.addArguments('--no-sandbox')
options.addArguments('--headless')
let driver = await new Builder().forBrowser(Browser.CHROME).setChromeOptions(options).build();
But I can only use imports in my project, I can't use require. I get the following error because of this line
const chrome = require('selenium-webdriver/chrome')
require is not defined in ES module scope, you can use import instead
How can I import chrome instead of require it to add the option? I am importing a lot of module in my code, It will be really difficult to change them all to requires
You can refer to the file as "selenium-webdriver/chrome.js". I.e.:
import {Options} from "selenium-webdriver/chrome.js";
const options = new Options();
// use options as you always would...
I have a very huge xml file that I got by exporting all the data from tally, I am trying to use web scraping to get elements out of my code using cheerio, but I am having trouble with the formatting or something similar. Reading it with fs.readFileSync() works fine and the console.log shows complete xml file but when I write the file using the fs.writeFileSync it makes it look like this:
And my web scraping code outputs empty file:
const cheerio = require('cheerio');
const fs = require ('fs');
var xml = fs.readFileSync('Master.xml','utf8');
const htmlC = cheerio.load(xml);
var list = [];
list = htmlC('ENVELOPE').find('BODY>TALLYMESSAGE>STOCKITEM>LANGUAGENAME.LIST>NAME.LIST>NAME').each(function (index, element) {
list.push(htmlC(element).attr('data-prefix'));
})
console.log(list)
fs.writeFileSync("data.html",list,()=>{})
You might try checking to make sure that Cheerio isn't decoding all the HTML entities. Change:
const htmlC = cheerio.load(xml);
to:
const htmlC = cheerio.load(xml, { decodeEntities: false });
I want to open the website via this code and it doesn't work. What's the problem? No errors are shown.
const Nightmare = require('nightmare');
var d = Nightmare({show:true});
d.goto('https://duckduckgo.com').wait(3000).end().then(result => {});
I simply want to download a .gz file from a URL and save it in a folder. I would like to do this without any third party libraries if possible. Here's what I have so far, but it only downloads an empty file:
const fs = require('fs')
const https = require('https')
let file = fs.createWriteStream('./folder/filename.gz')
let request = https.get('https://someurl/somefile.gz', function(res) {
res.pipe(file)
})
you can try this, using HTTP module for nodesJS,it looks similar to downloading any other file, just remember to mention the extension of the downloaded file when calling instead....Here is an example:
NOTE: IF you are trying to download from an HTTPS link, use the HTTPS
module instead, its exactly the same, but just replace all the
HTTP in the following code with HTTPS
const http = require('http');
const fs = require('fs');
//I added './' assuming that you want to download it where the server
//file is located, just change it to your desired path, followed by the
//filename and the EXTENSION
const file = fs.createWriteStream("./result.tar.gz");
const request = http.get("http://alpha.gnu.org/gnu/gzip/gzip-1.3.6.tar.gz", (response) => {
response.pipe(file);
});
I'm currently running Node.js by Browserify for my website.
It reads the JSON file and I get the message through MQTT.
But the problem is that it seems like writefile does not work.
(Running this as node test.js in the terminal works by the way).
What is wrong with my code?
Moreover, Is this the best way to store any user data?
Thank you so much in advance.
Here's some part of my code
var fs = require("fs");
var path = require("path");
let newFile = fs.readFileSync('/home/capstone/www/html/javascript/test.json');
function testT() { //THIS WORKS FINE
let student0 = JSON.parse(newFile);
var myJSON = JSON.stringify(student0);
client.publish("send2IR", myJSON);
response.end();
};
function write2JSON() { //PROBLEM OF THIS CODE
const content = 'Some content!'
fs.writeFileSync('/home/capstone/www/html/javascript/test.json', content)
};
document.getElementById("blink").addEventListener("click", publish);
document.getElementById("write").addEventListener("click", write2JSON);
You cann't write directly for security reasons. For other hand you can use a server as API to do the filye system tasks and in the client only trigger the events.
This post is very related with your problem:
Is it possible to write data to file using only JavaScript?