I am scraping a web site and using node and cheerio for that purpose.
I have the below structure
<li class="wrap-level-1">
<a class="level-2 link" href="https:mysite..." target="_blank"> Tropical Viking </a>
</li>
How do I get the Tropical Viking text only ?
I am trying this
$('.wrap-level-1').map((i, el) => {
console.log('entering scrapper')
const count = resultCount++
console.log(count)
//This is what I need
const title = $(el).find('a').???
const metadata = {
title: title
}
parsedResults.push(metadata)
console.log(metadata)
})
Thanks for your help
It looks like you want this:
let parsedResults = $('.wrap-level-1').map((i, el) => {
console.log('entering scrapper')
const count = resultCount++
console.log(count)
// This is what I need
const title = $(el).find('a').text()
const metadata = {
title: title
}
return metadata
}).get()
Related
looking for some help here please. So, I have a page that list the users from a group using the groupID as the parameter. Sometimes these list of users could be really long and I wanted to build in a search functionality to search the user(member) of the group.
I access the Workspace Directory API and method members.list like this:
const groupUsers = async function (groupid, from) {
var members = [];
var pageToken = '';
var newPageToken = '';
const admin = await authorizeDirectory();
const resp = await admin.members.list({
groupKey: groupid,
});
members = resp.data.members;
console.log(members);
// console.log(members);
// members = resp.data.members;
const total = members.length;
return { results: members.splice(from, 10), total };
};
I then pass the response back to an index.js file that handles it for rendering to a handlebars template. Inside this template I have form that allows the user to enter the email address so that I can use it to search for it directly from the members.get method
My code for that is:
<div class="searchWrapper">
<form action="/group" method="GET" class="searchBar">
<input type="text" name="searchBar" id="searchBar" placeholder="Enter the user's email ID"
value="{{this.term}}" />
<span class="search-button">
<a class="waves-effect waves-light btn-small" id="searchButton">FIND</a>
</span>
</form>
</div>
Could someone please help me understand how do I send the groupID and the value entered in the input search bar to the method, so that I can query the API. Would really appreciate the help.
My routes:
router.get('/group/:groupid', isUserAuthenticated, async (req, res) => {
const groupid = req.params.groupid;
const from = parseInt(req.query.from || 0, 10) || 0;
const members = await groupUsers(groupid, from);
const pages = [];
for (var i = 0; i < Math.ceil(members.total / 10); i++)
pages.push({ page: i * 10, label: i + 1 });
res.render('groups', {
members: members.results,
total: members.total,
groupid: groupid,
from: from,
pages: pages,
});
// res.send(`Group ID = ${req.params.groupid}`);
});
I have an issue where when I start at the root and click through the links all the states load in fine but when I got to copy and paste the URL into a new window it doesnt show, would I be right to assume its because the previous component isnt being rendered to set the state?
Hopefully someone can point me to some helpful articles or even have experienced this before?
Here is a link to a screen recording I made to better show what I mean https://youtu.be/M0390D4oJDg
CODE
import React from "react";
import { useLocation, useParams } from "react-router";
import PostBlock from "./PostBlock";
const PostList = () => {
const {thread} = useParams()
const { state: { description, title } = {} } = useLocation();
return (
<div>
<div className="m-10 flex justify-center">
<div style={{ width: "1216px" }}>
<div className="flex justify-center">
<div className="flex-1 justify-center mb-5 p-5 h-64 border border-gray-300">
<div>{title}</div>
<div>{description}</div>
</div>
</div>
<table className="min-w-full table-auto">
<thead className="justify-between">
<tr className="bg-gray-800">
<th className="px-8 py-2 text-left text-white">Posts</th>
</tr>
</thead>
<tbody className="bg-gray-200">
<PostBlock thread={thread}/>
</tbody>
</table>
</div>
</div>
</div>
);
};
export default PostList;
LINK
<Link to={{ pathname: `/thread/${thread}`, state: {title: title, description: description} }}>
URL
http://localhost:3000/thread/jxvgOKPrSiCnI2ocDxgJ
This react stuff is harder than I initially thought
Don't give up easily. It's one of the beautiful things!
Start with optional chaining.
const title = useLocation()?.state?.title;
const description = useLocation()?.state?.description;
If this is not supported, use &&:
const title = useLocation() && useLocation().state && useLocation().state.title;
const description = useLocation() && useLocation().state && useLocation().state.description;
And only when title and description is not null, render your contents.
const PostList = () => {
const { thread } = useParams();
const title = useLocation() && useLocation().state && useLocation().state.title;
const description = useLocation() && useLocation().state && useLocation().state.description;
return title && description ? <div></div> : null;
};
Now the code will not render the content until and unless both the values are set. And this will make sure your code loads without any errors.
I am a bit confused as to what your question really is, but you've got some issues with your code. Try this:
const PostList = () => {
const { thread } = useParams();
const { state: { description, title } = {} } = useLocation();
return <div></div>;
};
There is no reason to initialize useLocation() twice. You could've also initialized it into one constant and then access description and state through it.
const PostList = () => {
const { thread } = useParams();
const location = useLocation();
console.log(location.state.title, location.state.description);
return <div></div>;
};
Additionally, you can incorporate object?.property method as a failsafe if your state is empty:
console.log(location?.state?.title);
Take a look at proper useLocation usage here
I am a novice MERN stack developer.
I am trying to calculate the number of pages for pagination. The info object prints in console.log. However, when I try to use it in the for loop I get an error.
Can someone please explain what's the React logic or flow behind this? I have had issues with this multiple times but, could fix it with conditional rendering. But, somehow I wasn't able to fix this and I don't seem to understand the logic of how the flow in react is.
App Component :
const App = () => {
const [episodes, setEpisodes] = useState({});
const [loading, setLoading] = useState(false);
const [currentPage, setCurrentPage] = useState(1);
const [episodesPerPage, setEpisodesPerPage] = useState(10);
useEffect(() => {
const fetchEpisodes = async () => {
setLoading(true);
const res = await axios.get('https://rickandmortyapi.com/api/episode/');
setEpisodes(res.data);
setLoading(false);
};
fetchEpisodes();
}, []);
console.log(episodes.info);
return (
<div>
<div id='header'>
<h1>Rick & Morty</h1>
<h2>Episodes</h2>
</div>
<div>
<h3>All Episodes</h3>
<EpisodeList episodeList={episodes.results} loading={loading} />
<Pagenation info={episodes.info} />
</div>
</div>
);
};
export default App;
Pagenation Component:
const Pagenation = ({ info }) => {
const pageNumbers = [];
console.log(info);
for (let i = 1; i <= Math.ceil(info.count / 20); i++) {
pageNumbers.push(i);
}
return (
<nav aria-label='...'>
<ul class='pagination pagination-lg'>
{pageNumbers.map((number) => {
return (
<li class='page-item active' aria-current='page'>
<span class='page-link'>
{number}
<span class='sr-only'>(current)</span>
</span>
</li>
);
})}
</ul>
</nav>
);
};
Conditional rendering can be the solution here as well.
episodes is initially an empty object, so episodes.info is initially undefined. This means you cannot access a property on info without checking if it exists first because you know already that it will be undefined at the beginning.
A simple solution might look like this:
{episodes.info && <Pagenation info={episodes.info} />}
You could also move the conditional into the Pagenation component to be something like this:
if (info) {
for (let i = 1; i <= Math.ceil(info.count / 20); i++) {
pageNumbers.push(i);
}
}
Regardless of your strategy to avoid the error, the core of the issue is that you have data that is loaded after the component mounts. This means you need to account for that data being missing for at least one render.
I have this in my algolia file for my jekyll site.
<script>
const search = instantsearch({
appId: '{{ site.algolia.application_id }}',
apiKey: '{{ site.algolia.search_only_api_key }}',
indexName: '{{ site.algolia.index_name }}',
searchParameters: {
restrictSearchableAttributes: [
'title',
'content'
],
facetFilters: ['type:post']
},
});
const hitTemplate = function(hit) {
let date = '';
if (hit.date) {
date = moment.unix(hit.date).format('L');
// date = moment.unix(hit.date).format('MMM Do YY');
modifiedDate = moment.unix(hit.last_modified_at).format('MMM Do YY');
}
const url = hit.url;
const title = hit._highlightResult.title.value;
const content = hit._highlightResult.html.value;
return `
<div class="post-list">
<span class="post-date-list-wrap">
<span class="post-date-list">${date}
<span class="post-title"> ${title} </span>
</span>
${content}
</div>
`;
}
const hitTemplateTrans = function(hit) {
let date = '';
if (hit.date) {
date = moment.unix(hit.date).format('MMM DD YYYY');
}
const url = hit.url;
const title = hit._highlightResult.title.value;
const content = hit._highlightResult.html.value;
return `
<div class="post-list">
<span class="post-date-list-wrap">
<span class="post-date-list">${date}
<span class="post-title"> ${title}</span>
</span>
</span>
</div>
`;
}
search.addWidget(
instantsearch.widgets.searchBox({
container: '#search-searchbar',
placeholder: 'search notes',
autofocus: true
})
);
search.addWidget(
instantsearch.widgets.hits({
container: '#search-hits',
templates: {
empty: 'No results',
item: hitTemplate
},
})
);
search.start();
</script>
Without typing anything in the search box I have the list of articles
with the excerpt, the short introduction of the article.
That's because I have ${content} to show the highlights when someone
types the search term.
That's fine and everything is working but... I don't want to show the contents of each item when the search box is empty.
If the search box is empty I would like to keep only the title and the date
but if the search box is not empty just show the title/date and the excerpt as it's usual.
It seems like an easy task but I'm stuck right now, I tried removed the content tag and put in the hit transformation function, but it doesn't work.
The instantsearch.js library has a function hook, called searchFunction, you can define when instanciating the library. That is called right before any search is performed. You can use it to check if the current query is empty or not, and adapt your layout based on this knowledge.
Here is a slightly edited script (irrelevant parts removed) that should let you do what you're looking for:
let additionalClass = '';
const search = instantsearch({
[…]
searchFunction: function(helper) {
if (helper.getState().query === '') {
additionalClass = 'post-item__empty-query';
} else {
additionalClass = '';
}
helper.search()
}
});
[…]
const hitTemplate = function(hit) {
return
`<div class="post-item ${additionalClass}">
[…]
</div>`
;
}
.post-item__empty-query .post-snippet {
display: none;
}
What it does is defining a global variable (additionalClass) that will be used in the hit template (added alongside item-post, at the root level).
Right before everysearch, we check if the query is empty. If so, we set additionalClass to item-post__empty_query. We also defined in CSS that when this class is applied, the content should be hidden.
All of that together makes the title + date displayed when no search is performed, and the content displayed only when an actual keyword is searched for.
Hope that helps,
I am working on a project in React. The idea is that when you search an artist an img render on the pg. Once you click the image a list of collaborating artists is rendered. You can then click a name and see that persons collabpratign artists. Here is my issue: Rather than the state clearing/resetting each time a new artist is clicked, new artists just add on to the original state. Can someone help me figure out how to clear the state so that the state clears and returns a new list of collaborators? Been stuck on this for hours. Here is the code
searchForArtist(query) {
request.get(`https://api.spotify.com/v1/search?q=${query}&type=artist`)
.then((response) => {
const artist = response.body.artists.items[0];
const name = artist.name;
const id = artist.id;
const img_url = artist.images[0].url;
this.setState({
selectedArtist: {
name,
id,
img_url,
},
});
})
.then(() => {
this.getArtistAlbums();
})
.catch((err) => {
console.error(err);
});
}
getArtistCollabs() {
console.log('reached get artist collab function');
const { artistCounts } = this.state;
// console.log(artistCounts);
const artist = Object.keys(artistCounts).map((key) => {
//kate
const i = document.createElement("div");
i.innerHTML = key;
i.addEventListener('click', () => {
this.searchForArtist(key);
})
document.getElementById("collabs").appendChild(i);
});
this.setState({});
}
//kate
renderArtists() {
const artists = this.getArtistCollabs();
}
render() {
const img_url = this.state.selectedArtist.img_url;
return (
<div>
<form onSubmit={this.handleSubmit}>
<input type='text' name='searchInput' className="searchInput" placeholder="Artist" onChange={this.handleChange} />
<input type='submit' className="button" />
</form>
<img className="artist-img" src={this.state.selectedArtist.img_url}
// kate
onClick={this.renderArtists} alt="" />
<div id="collabs">
</div>
</div>
Your problem is right here:
const artist = Object.keys(artistCounts).map((key) => {
//kate
const i = document.createElement("div");
i.innerHTML = key;
i.addEventListener('click', () => {
this.searchForArtist(key);
})
document.getElementById("collabs").appendChild(i);
What you have done here is manually create html elements and insert them into the dom. As soon as this takes place react has no control over these newly created elements. You should only manipulate the DOM like this when its absolutely necessary. Instead you should be making a new component called something like <ArtistCollaborators> and it should take in the artists as props and be what renders the code you have here into the DOM using its own render method.
This will be the React way of doing it, and allows react to be fully control of what you are rendering into the DOM.