How to specify the class selector in web scraping using nodejs - node.js

Am trying to implement scraping using nodejs. Am sending request to a particular url and when i get the response, using the response am storing each products in a page into an array. For each product am trying to display the product details like product name, price and discount etc. Am doing this by the following code.
var $products = $body.find('.fashion-item');
$products.each(function (i, item) {
var $name = ($(item).find('.info .title').text(),
$price=$(item).find('span.price.regular').text().substr(6),
$discount=$(item).find('span.price.percentoff').text().slice(0,2);
self.items[i] = {
title: $name,
price: $price,
discount: $discount,
};
});
console.log(self.items);
It is working fine. If the class name is like 'fashion-item' or 'fashion-item-first' all this works fine. But if the class name contains the space in between the word, am not getting any products stored in array ($products) i.e. array length of $products is zero. My question is how to do the same thing if class name like this 'fashion-item first'. I tried very much, but i didn't get any idea. Please help me.

Class names cannot have spaces. In your example, 'fashion-item first', are two classes assigned to an element, which can be selected by either .fashion-item or .first selectors.

If you want item who have both info and title classe select them with this :
$(item).find('.info.title')
If you want item one of these classes, use this
$(item).find('.info,.title)

Related

How do I query the vimeo api for a specific video title?

Hi I'm querying for a specific video by title - and at the moment I get mixed results.
my videos are all named with a consecutive number at the end ie ANDNOW2022_00112, ANDNOW2022_00113 etc
When I search /videos/?fields=uri,name&query=ANDNOW2022_00112 I get all of the videos returned
I've also tried the query_fields using
/me/videos?query_fields=title&sort=alphabetical&query=ANDNOW2022_00112
I just want the one I've searched for - or a no results returned.
At the moment I get all of the videos with AN2022 in the title/name. Now 'usually' the one I searched for is at the top of the list but not everytime.
Any tips appreciated.
Okay I'm not going mad :)
This is from Vimeo and is here for those with the same issue - basically to get t to work you need to understand that:
After speaking with our engineers, the current search capability are not "Exact" search.
When adding numbers or underscores the search is split into parts so "ANDNOW2022_00112" is transforming the query into the parts "andnow2022", "andnow", "2022", and "00112". So this is why your seeing these results. Our engineering team are in the process of improving the search capabilities and hope to provide a release in the near future.
Which means for now I'll have to rename my files.
Preface:
Vimeo does not currently offer an API endpoint for exact title search — but even if it did — it's possible to upload multiple videos and assign them identical titles. There's no way to use the API to positively identify a video by title — this is why every uploaded video is assigned a unique ID.
Solution:
Because the API returns data which includes an array of video objects, you can solve this problem in the same way you'd solve any similar problem in JavaScript where you have to find an element in an array: Array.prototype.find()
Here's how you can apply it to your problem:
Query the API using the parameters you described in your question.
You might also be interested in using the sort and direction parameters for greater control over a deterministic sort order.
Find the first item in the returned array of video objects that match your expected text exactly, and return it (or undefined if it doesn't exist)
Here's a code example with some static data from the API that was used to search for the video Mercedes Benz from the user egarage — note that I've omitted quite a few (irrelevant) fields from the response in order to keep the example small:
// Mocking fetch for this example:
function fetch (_requestInfo, _init) {
const staticJson = `{"total":2,"page":1,"per_page":25,"paging":{"next":null,"previous":null,"first":"/users/egarage/videos?query_fields=title&query=Mercedes%20Benz&sort=alphabetical&direction=asc&page=1","last":"/users/egarage/videos?query_fields=title&query=Mercedes%20Benz&sort=alphabetical&direction=asc&page=1"},"data":[{"uri":"/videos/61310450","name":"50th Anniversary of the Pagoda SL -- Mercedes-Benz Classic Vehicles","description":"Penned by designer Paul Bracq, the W113 SL had big shoes to fill: it had the incredible task of succeeding the original and instantly iconic 300 SL Gullwing. But you can't copy a legend, so Bracq designed one of his own. Straight lines replaced curves and a low-slung roof was replaced by a high top design that gave the car its nickname: the Pagoda.\\n\\nMUSIC: Developer Over Time","type":"video","link":"https://vimeo.com/61310450"},{"uri":"/videos/55837293","name":"Mercedes Benz","description":"To celebrate Mercedes Benz 125th birthday, the 2011 Pebble Beach Concours d’Elegance showcased the models that trace the lineage to Benz and Daimler —particularly Mercedes-Benz. This tribute chronicled early racing greats, coachbuilt classics, and preservation cars. Produced in association with DriveCulture.","type":"video","link":"https://vimeo.com/55837293"}]}`;
return Promise.resolve(new Response(staticJson));
}
async function fetchVideoByTitle (token, userId, videoTitle) {
const url = new URL(`https://api.vimeo.com/users/${userId}/videos`);
url.searchParams.set("query_fields", "title");
url.searchParams.set("query", videoTitle);
url.searchParams.set("sort", "alphabetical");
url.searchParams.set("direction", "asc");
const headers = new Headers([
["Authorization", `Bearer ${token}`],
]);
const response = await fetch(url.href, {headers});
const parsedJson = await response.json();
// Find the video that matches (if it exists):
const maybeFirstVideoObj = parsedJson.data.find(video => video.name === videoTitle);
return maybeFirstVideoObj;
}
async function main () {
const video = await fetchVideoByTitle(
"YOU_ACTUAL_TOKEN",
"egarage",
"Mercedes Benz",
);
console.log(video); // {name: "Mercedes Benz", link: "https://vimeo.com/55837293", ...}
}
main();

How to get current product or category url equivalent for all other channels in shopware 6?

I need to build a "channel switcher" in shopware6. How can I access current (Product)URL equivalents for all or some other channels in this shop? It should work like a language switcher, keeping the current viewed product or category and redirecting to other channel.
I know how to get current seoUrl of product:
{{ seoUrl('frontend.detail.page', { productId: page.product.id }) }}
but is there a way to get all other seoUrls for all other selling channels with the same product.id? And the same for categories?
Rough approach to get all language links
Looking at
\Shopware\Core\Framework\Adapter\Twig\Extension\SeoUrlFunctionExtension which is handling the seoUrl() in twig it looks like this:
public function seoUrl(string $name, array $parameters = []): string
{
return $this->seoUrlReplacer->generate($name, $parameters);
}
This resolves to
\Shopware\Core\Content\Seo\SeoUrlPlaceholderHandler::generate which always uses the current request:
public function generate($name, array $parameters = []): string
{
$path = $this->router->generate($name, $parameters, RouterInterface::ABSOLUTE_PATH);
$request = $this->requestStack->getMainRequest();
$basePath = $request ? $request->getBasePath() : '';
$path = $this->removePrefix($path, $basePath);
return self::DOMAIN_PLACEHOLDER . $path . '#';
}
Which boils down to \Shopware\Storefront\Framework\Routing\Router::generate
As an approach I would suggest to dig into those functions. They directly access the current request and context. Thanks to dependency injection, you can create those instances yourself with the target context and an artificial request containing the right base URL from the target channel.
Approach similar to the language switcher (redirect after POST)
The above approach might be quite complicated, so looking at what the language switcher does:
It just includes the language IDs
On selection posts to \Shopware\Storefront\Controller\ContextController::switchLanguage
Which does the redirect
Example Payload of the request:
languageId: 2fbb5fe2e29a4d70aa5854ce7ce3ff0b
redirectTo: frontend.navigation.page
redirectParameters[navigationId]: a11f6b46948c47a7b2c2ac874704fff6
I think you can extend that script for your channel switcher and add the sales channel id to the request,
then you can reuse / copy or even decorate the switchLanguage controller. You just need to pass in the right context of the target sales channel and it should redirect to the correct URL.

laravel pagination Method links does not exist

I am building a dynamic query using eloquent based on user inputs.
$query = \App\Model::query();
if($this == request('that')){
$query->where(...); // building dynamic query based on user inputs
}
and finally user can type number of records shown per page in an html input field named count or checking a checkbox input type named all and following code completes dynamic query in back-end:
if (request('all')) {
$result = $query->get();
}else {
$result = $query->paginate(request('count'));
}
Also in front-end I have following code to show pagination links:
{{$result->links()}}
The problem is when user chooses to show all records we face following error:
Method links does not exist.
links method is not callable when we retrieve objects via get() method. What is the best practice in order not to face this error?
It's because ->get() method return collection instead of paginator model Illuminate\Pagination\LengthAwarePaginator. So I can recomment you send some addition variable from your controller which indicate if you need execute $result->links().
Controller:
return view('yourView', [showPagination => is_null(request('all'))]);
View:
#if($showPagination)
{{$result->links()}}
#endif

How to override template file item-list.html.twig for field_slider_images in Drupal 8?

I want to override the item listing template file core/themes/classy/templates/dataset/item-list.html.twig for listing the fields field_slider_images as well as field_blog_tags respectively of their's multiple values of the field.
I have selected "Unordered List" in the view.
Please do check the attached image.
I have created following files :
item-list--field-blog-tags.html.twig
item-list--field-slider-images.html.twig
But, this is not rendered for the listing of the fields.
When I have created item-list.html.twig then only it will access.
However, both fields have different data to style and I am not able to get the current field name which is loading it's data in item-list.html.twig.
Had a brief look at this and it doesn't seem that 'item-list' to have suggestions, which is quite unfortunate.
In this situation there are two options:
Create your own suggestion which would accomplish exactly what you need.
You'll have to do something like this:
/
/*add new variable to theme with suggestion name*/
function hook_theme_registry_alter(&$theme_registry) {
$theme_registry['item_list']['variables']['suggestion'] = '';
}
//send a value to newly added variable to use it build the suggestion
function hook_ENTITY_TYPE_view(array &$build, $entity, $display, $view_mode) {
//add condition here if field exists or whatever, do the same for other field
$build['field_slider_images']['#suggestion'] = 'field_slider_images';
}
//use newly added variable to build suggestion
function hook_theme_suggestions_THEME_HOOK(array $variables) {//THEME_HOOK=item_list
$suggestions = array();
if(isset($variables['suggestion'])){
$suggestions[] = 'item_list__' . $variables['suggestion'];
}
return $suggestions;
}
Now you should be able to use item-list--field-slider-images.html.twig
Second option is to do what others in core did: use a new theme
function hook_ENTITY_TYPE_view(array &$build, $entity, $display, $view_mode) {
//add condition here if field exists or whatever, do the same for other field
$build['field_slider_images']['#theme'] = array(
'item_list',
'item_list__field_slider_images',
);
}

Get old data in preUpdate Sonata Admin Bundle

I have a product entity and it has an images field that store the images names from the product but the images names depends of a part_number field that is unique, so if the user make a mistake in the part number and he wants to edit it then I also have to change the images names
I tried this but it does not works:
// class ProductsAdmin extends Admin
public function preUpdate($product) {
$old_product = $this->getSubject();
if ($old_product->getPartNumber() != $product->getPartNumber)
{
// change file names
}
$this->saveFile($product);
}
How I get the original row in preUpdate() function?
According to the topic taken from the official SonataAdmin google forum:
https://groups.google.com/forum/#!topic/sonata-devs/0zML6N13i3U
you need to make use of the class UnitOfWork:
http://www.doctrine-project.org/api/orm/2.3/class-Doctrine.ORM.UnitOfWork.html
Do this way:
public function preUpdate($object)
{
$em = $this->getModelManager()->getEntityManager($this->getClass());
$original = $em->getUnitOfWork()->getOriginalDocumentData($object);
}
Thus you get an array of values of your database entity.
E.g: to get access to the value password of your entity do:
$password = $original['password'];
That's all.
Enjoy :)
If you just do a doctrine query in the preUpdate function to get the product from the database you'll have the old object. Then do the comparison and you're good to go.

Resources