Having hard time scraping a cell with Cheerio - node.js

I'm trying to scrape a cell from a table but I'm having a hard time and I'm probably doing something wrong because I get an empty result in the console(literally nothing).
Here's the HTML:
I'm trying to get the <td class="center bold storing_1">1</td>.
Here's my code:
const rp = require('request-promise');
const cheerio = require('cheerio');
const url = 'MY URL';
rp(url)
.then(function(html) {
$ = cheerio.load(body);
console.log($('#table_results tbody tr:nth-child(1) td.center.bold.sorting_1')).text();
})
.catch(function(err) {
});
Any help is appreciated! Thanks!

I think your code is actually ok, I think you just have a few typos in it.
You are parsing 'body' rather than 'html'
You are reading .text() from the result of console.log() rather than the cheerio object ($).
If you try the code below it should work:
const rp = require('request-promise');
const cheerio = require('cheerio');
const url = 'MY URL';
rp(url).then(function(html) {
$ = cheerio.load(html);
console.log("Result:", $('#table_results tbody tr:nth-child(1) td.center.bold.sorting_1').text());
})
.catch(function(err) {
console.error("An error occurred:", err);
});
Html I'm testing with:
Test html
<table id="table_results">
<thead></thead>
<tbody>
<tr ajax-controller="community">
<td class="center bold sorting_1">1</td>
<td class="center bold sorting_1">2</td>
</tr>
</tbody>
</table>
Simple test setup:
You can play around with this to see how editing the selector changes things:
const testHtml =
`<table id="table_results">
<thead></thead>
<tbody>
<tr ajax-controller="community">
<td class="center bold sorting_1">1</td>
</tr>
</tbody>
</table>`;
$ = cheerio.load(testHtml);
console.log("Result:", $('#table_results tbody tr:nth-child(1) td.center.bold.sorting_1').text());

Related

puppeteer pick a date from the calendar by class name if available

what i want: if there is a green date click on it (the first one or the last), if not keep reloading
you can see the calendar image below
note: the green date has always a class name "activeClass" + title="Book"
const rdvLoop = async () => {
await page.waitForTimeout(1000)
await page.reload()
await page.waitForTimeout(1000)
if ((await page.$("td.activeClass")) !== null ){
await page.waitForSelector('td.activeClass:nth-child(1)')
await page.click("td.activeClass:nth-child(1)")
await page.waitForTimeout(1000)
}
if ((await page.$("td.activeClass")) == null ){
await page.waitForTimeout(1000)
await page.reload()
}
}
rdvLoop()
for (var i = 0; i = 10 ;i++) {
await page.waitForTimeout(1000)
await rdvLoop()
}
this is the website calendar (minimalized)
<div class="datepicker datepicker-dropdown dropdown-menu" >
<div class="datepicker-days" >
<table class=" table-condensed">
<tbody>
<tr>
<td class="day disabled fullcap" title="Slots Full">8</td>
<td class="day disabled fullcap" title="Slots Full">9</td>
<td class="active day disabled fullcap" title="Slots Full">10</td>
<td class="disabled day activeClass" title="Book">11</td>
</tbody>
</table>
</div>
</div>
here is the calendar image from the website

Formatting data in with Cheerio

I want to get the list of IP from a website and add them into an array. The website shows the data like this:
<tbody><tr role="row" class="odd">
<td>131.108.216.44</td>
<td>47267</td>
<td>BR</td>
<td class="hm">Brazil</td>
<td>elite proxy</td>
<td class="hm">no</td>
<td class="hx">yes</td>
<td class="hm">2 minutes ago</td>
</tr>
<tr role="row" class="even">
<td>85.173.165.36</td>
<td>46330</td>
<td>RU</td>
<td class="hm">Russian Federation</td>
<td>elite proxy</td>
<td class="hm">no</td>
<td class="hx">yes</td>
<td class="hm">2 minutes ago</td>
</tr>
</tbody>
This is actually a very long list with 100's of table but the format is the same.
What I did is :
var c = new Crawler({
maxConnections: 1,
callback: function (error, res, done) {
if (error) {
console.log(error)
} else {
var $ = res.$;
$('tbody>tr>td').each((i, el) => {
const item = $(el)
console.log(item.text());
})
}
done();
}
})
c.queue({
uri: 'https://free-proxy-list.net/'
})
I want to keep the first 10 IPs from the website and add them into an array.
The first 10 would look like this:
let proxies = $('tr[role=row]').map((i, tr) => {
let host = $(tr).find('td:nth-child(1)').text()
let port = $(tr).find('td:nth-child(2)').text()
return `${host}:${port}`
}).get().slice(0, 10)

Node.js puppeteer - Fetching content seperated by br and store items in seperate variables

I'm using node.js and puppeteer to get some data. How can I save the content of an element (which is divided by line break <br>) in two separate variables?
That's the HTML I'm looking at:
<table summary="">
<tbody>
<tr nowrap="nowrap" valign="top" align="left">
<td nowrap="nowrap">2018-08-14<br>16:35:41</td>
</tr>
</tbody>
</table>
I'm getting the content of the td like this (app.js):
let tableCell04;
let accepted;
tableCell04 = await page.$( 'body div table tr td' );
accepted = await page.evaluate( tableCell04 => tableCell04.innerText, tableCell04 );
console.log('Accepted: '+accepted);
The output in console is:
Accepted: 2018-08-14
16:35:41
But what I would like to have is storing the content which is separated by the line break in two separate variables so that I get sth like this:
Accepted_date: 2018-08-14
Accepted_time: 16:35:41
Hi you can use tableCell04.innerHTML to get the html instead of the plain text.
accepted = await page.evaluate( tableCell04 => tableCell04.innerHTML, tableCell04 );
const [Accepted_date, Accepted_time] = accepted.split('<br>');

MVC 5 get value of DropDownList within a table row

I have a view with a table of products that can be added to a shopping cart. Each row has a DropDownList with allowed quantities that can be ordered along with a button to add to cart. Everything is populating and displaying properly. I know how to pass the item ID in the ActionLink but how can I get the value of the DownDownList associated with the table row of the ActionLink that was clicked?
I am guessing possibly using JQuery that fires when the ActionLink is clicked?
I also thought of making every row a form but that seems overkill.
Is there an easy MVC way to do this?
In prepping more info for a proper question and went ahead and solved it. Thank you Stephen for the nudge and info.
I tried putting a Html.BeginForm around each <tr> tag in the details section. This did indeed work for me. I was able to easily get the unique form info to POST for each individual row. However, when I would enable JQuery DataTables the submit would break. DataTables must be capturing the submit or click somehow. Haven't figured that out but it made me try JQuery which seems a much better way to do it.
Here is how I construct the table data row:
#foreach (var item in Model)
{
<tr>
<td>
<img src="#item.GetFrontImage()" width="100" />
</td>
<td>
<strong>#Html.DisplayFor(modelItem => item.DisplayName)</strong>
</td>
<td>
#Html.DisplayFor(modelItem => item.CustomerSKU)
</td>
<td>
#Html.DropDownList("OrderQty", item.GetAllowedOrderQuantities(), htmlAttributes: new { #class = "form-control" })
</td>
<td>
<a class="btn btn-default pull-right" data-id="#item.ID">Add to Cart</a>
</td>
</tr>
}
This creates a select with id of OrderQty and I embedded the item ID in data-id attribute of the link. I then used this JQuery to capture the info and POST it to my controller. Just have a test div displaying the results in this example:
// Add to Cart click
$('table .btn').click(function () {
// Gather data for post
var dataAddToCard = {
ID: $(this).data('id'), // Get data-id attribute (Item ID)
Quantity: $(this).parent().parent().find('select').val() // Get selected value of dropdown in same row as button that was clicked
}
// POST data to controller
$.ajax({
url: '#Url.Action("AddToCart","Shopping")',
type: 'POST',
data: JSON.stringify(dataAddToCard),
contentType: 'application/json',
success: function (data) { $('#Result').html(data.ID + ' ' + data.Quantity); }
})
});
The JQuery function receives the reference to the link being clicked so I can extract the Item ID from the data-id attribute. I can then get a reference to the dropdown (select) that is in the same row by using .parent.parent (gets me to the <tr> tag) and then just finding the next 'select' tag. Probably pretty obvious to a lot of you.
This works great for my purposes. I can also update other elements with data returned from the POST.
Thank you
Karl
for the table in html:
<div class="table-responsive">
<table id="employeeTable"class="table table-bordered">
<thead>
<tr>
<th class="text-center">ُُُEmpId</th>
<th class="text-center">Name</th>
<th class="text-center">Absense State</th>
</tr>
</thead>
<tbody>
#foreach (var item in Model)
{
<tr>
<td>#item.Id</td>
<td>#item.Name</td>
<td class="text-center">#Html.DropDownList("DDL_AbsentStatus", new SelectList(ViewBag.statusList, "Id", "Name"), new { #class = "form-control text-center" })</td>
</tr>
}
</tbody>
</table>
</div>
in javascript to get the selected value:
//Collect Date For Pass To Controller
$("#btn_save").click(function (e) {
e.preventDefault();
if ($.trim($("#datepicker1").val()) == "") {
alert("ادخل تاريخ يوم صحيح!")
return;
}
var employeesArr = [];
employeesArr.length = 0;
$.each($("#employeeTable tbody tr"), function () {
employeesArr.push({
EmpId: $(this).find('td:eq(0)').html(),
EntryDate: $.trim($("#datepicker1").val()),
StatusId: $(this).find('#DDL_AbsentStatus').val()
});
});
$.ajax({
url: '/Home/SaveAbsentState',
type: "POST",
dataType: "json",
data: JSON.stringify(employeesArr),
contentType: 'application/json; charset=utf-8',
success: function (result) {
alert(result);
emptyItems();
},
error: function (err) {
alert(err.statusText);
}
});
})

Pylons controller loose formating YUI

I have the following controller:
import logging
from pylons import request, response, session, tmpl_context as c, url
from pylons.controllers.util import abort, redirect
from webhelpers.html.tags import HTML
from improve.lib.base import BaseController, render
from improve import model
import improve.model.meta as meta
import improve.lib.helpers as h
log = logging.getLogger(__name__)
class AlarmsController(BaseController):
def list(self):
c.alarms = meta.Session.query(model.Alarms).all()
return render('../public/listalarms_table.html')
rendered by this template (listalarms_table.html):
...
<div id="markup">
<table id="alarms">
<thead>
<tr>
<th>Id</th>
<th>Severity</th>
<th>Node</th>
<th>Count</th>
<th>Last Alarm</th>
<th>Log Msg</th>
<th>AckUser</th>
</tr>
</thead>
<tbody>
% for alarms in c.alarms:
<tr>
<td>${alarms.alarmid.__repr__()|n}</td>
<td>${alarms.severity.__repr__()|n}</td>
<td>${alarms.nodeid.__repr__()|n}</td>
<td>${alarms.counter.__repr__()|n}</td>
<td>${alarms.lasteventtime.__repr__()|n}</td>
<td>${alarms.logmsg.__repr__()|n}</td>
<td>${alarms.alarmackuser.__repr__()|n}</td>
</tr>
% endfor
</tbody>
</table>
</div>
<script type="text/javascript">
YAHOO.util.Event.addListener(window, "load", function() {
YAHOO.example.EnhanceFromMarkup = new function() {
var myColumnDefs = [
{key:"id",label:"Id", sortable:true},
{key:"severity",label:"Severity", sortable:true}
{key:"node",label:"Node", sortable:true}
{key:"count",label:"Count",formatter:YAHOO.widget.DataTable.formatNumber,sortable:true},
{key:"lastalarm",label:"LastAlarm",formatter:YAHOO.widget.DataTable.formatCurrency,sortable:true},
{key:"logmsg",label:"Log Msg"}
];
this.myDataSource = new YAHOO.util.DataSource(YAHOO.util.Dom.get("alarms"));
this.myDataSource.responseType = YAHOO.util.DataSource.TYPE_HTMLTABLE;
this.myDataSource.responseSchema = {
fields: [{key:"id"},
{key:"severity"},
{key:"node"},
{key:"count", parser:"number"},
{key:"lastalarm", parser:"date"} // point to a custom parser
]
};
this.myDataTable = new YAHOO.widget.DataTable("markup", myColumnDefs, this.myDataSource,
{caption:"Example: Progressively Enhanced Table from Markup",
sortedBy:{key:"id",dir:"asc"}}
);
};
});
The problem:
If I call it using http://myserver:port/listalarms_table.html , I see the correct formating of the YUI table but instead of the data I get the alarms.alarmid.repr()|n instead of the real data
If i call it http://myserver:port/alarms/list I get the data correctly but I loose the YUI format...
How can i get the data with the correct YUI layout?
Thx

Resources