I was presented with the task of hiding telephone numbers from Google - what that means is, we want to display them on the website and have them clickable href="tel:..." but to ensure Google does not index it and does NOT display it with the search results.
Does anyone know of any effective technique?
I was thinking of writing VueJs component, which mixes given number with some alpha characters, but this would only work with the presentation / label, the tel:... would still have to have a valid telephone number and I'm not sure if Google wouldn't pick it form the href attribute.
I think the best approach is just to hide it from bots, may be you can use something like this VueIfBot
<vue-if-bot>
This will not be visible for bots
</vue-if-bot>
or any other alternative just check the userAgent for example in php
function _bot_detected() {
return (
isset($_SERVER['HTTP_USER_AGENT'])
&& preg_match('/bot|crawl|slurp|spider|mediapartners/i', $_SERVER['HTTP_USER_AGENT'])
);
}
If you can't get userAgent, but you still want to check if this is a search engine crawler you can check user IP adress here is a list of IP Addresses of Search Engine Spiders
And finally after you successfully hide your data you can test it with User-Agent Switcher
Related
I have a website with a list of urls. I need to make a command that would go on each link, go to the website and find an email address.
I need this command to do this for every url and search for every page on each url.
Is there any way I could achieve this ?
Thank you !
Here is the website with the url list : http://cabm.net/nos-membres
I would encourage you to retrieve email addresses from databases.
That said, it is possible to achieve the desired effect by retrieving the set of html pages from each site (read downloading) then parsing through the string to retrieve strings in email format:
-whitespace-string#string.string-whitespace-
Results would be stored in an array of email addresses.
There isn't a single command to do this but it is possible to write a program to achieve the same.
The program itself also raises ethical issues on whether or not you're allowed to do that but that's a discussion for another time ;)
I am having quite the difficulty. on our company site https://temp-quitlogixbase.quitlogix.org I had set up the Smart Search functionality. setting up indexes for each of the sites with in the application
(i.e.
https://colorado.quitlogix.org https://arkansas.quitlogix.orghttps://idaho.quitlogix.org
)
I even made sure to limit each site to the index meant for it. the problem is that the smart search results, either changes for each site, or if I include multipule indexes it gives me all the results for all the indexes, not just for the site I am working with. can some one help as to what I am doing wrong?
From your description, it sounds like you want each site's search to work independently so that - e.g. - Idaho results are not served on the Colorado search results.
To do this, you'll have (and sounds like you do) a SmartSearch index set up for each site with the allowed content in the index limited to the site in question.
What I would look at is the template your using for the search results. It looks very similar on the three sites you've listed, which makes me think that they are the same template. If that is the case and you're using a web part for the search results, you'll need a macro or some other logic to tell the page which index to look at.The template is effectively global, so each time you set the index on the Smart search results web part, it will override the previous value, even if you're switching between sites.
A way to do the switching can be to set the Indexes field to something like the following macro:
{% if (CurrentSite.CodeName == "QuitLogix_Arkansas") { "ArkansasSiteIndex" } else if(CurrentSite.CodeName == "QuitLogix_Colorado") { "ColoradoSiteIndex" } else {"IdahoSiteIndex"} #%}
If you've done all that or are using separate templates, you will not need that. Other options can include using multiple Smart search results web parts with their set the visibility based upon the current site or by having a different template for each sub-domain.
Background
I'm currently developing a website - simplycaravan.com - and am struggling to get Google to properly index the site. We have many (potentially hundreds) of similar but unique pages that display information about different holiday homes. The URLs for these pages all begin "http://simplycaravan.com/caravan-details/?id= " and then each page has a unique id= number to distinguish it e.g. "http://simplycaravan.com/caravan-details/?id=55". The problem is that Google is not seeing each page of these pages as unique and therefore not indexing them.
Question
Each of these 'id=' pages has the below tag in the header
link rel="canonical" href="http://simplycaravan.com/caravan-details/?id=55" />
Can anyone advise if this tag should be used or is causing a conflict which leads to the problem? My understanding is that canonical tag should be used if you have multiple similar pages and you want to redirect to one page, this is not the case for our website as we do not want to redirect all these apparently similar pages to one page, we want them to be standalone, unique webpages in their own right.
One thing to note is we have changed the id parameter in Google search console from 'let google decide' to 'every url'.
Thanks in advance for any help offered.
I have a slight problem I have been trying to address for a client I have been working with. We have 4 sets of single pages that are loading content from a database using PHP based upon a get string that is provided. These pages that are generated are optimized well for SEO and have alt tags for images and Content that we need to be able to search using a search feature.
Now i had assumed (An everyone knows what assuming gets you) that these pages by default would be able to be searched by the concrete 5 built in search feature. But it doesn't work. If I search for a word that I know is definitely on one of these pages even multiple times no results are found.
How can I make Concrete5 search these pages. If its no do able by a default or by a plugin, then can someone please offer some advice on how to fix this. This is an important feature and must be completed.
EDIT: See my comment below. I still need some help or direction here as CSE inst much of an option.
EDIT2: It may be viable for me to install a crawler and a custom search engine to address my problems. I was thinking of spider. Any other suggestions on that or other options are much appreciated!
Unfortunately C5 doesn't provide a way to do this -- the only way to tap into the search index is with blocks. And even if you created a phony block just to pass content from the single_page through to the search index, there's no way to say that some content is from one URL while other content is from another URL (which you'd need to do since your single_page controller is handling many different URL's).
I don't know of a way to achieve what you want to do (and it appears that nobody else does either -- http://www.concrete5.org/community/forums/customizing_c5/make-content-in-single-pages-searchable/ ), other than building your own internal search engine.
EDIT: I just did some digging, and thought that perhaps you could manually insert records into the PageSearchIndex table and specify the searchable content and the desired path there -- but this won't work because it relies on one cID (collection id, a.k.a. page id) per entry -- so you'd only be able to insert one record for the top-level single_page path.
I think the simplest solution here would be to create your own searching infrastructure for your single_pages (like some kind of function in the controller that would return an array of page paths and searchable content for each one), then override the search block and perform an additional search of your single_page -- then combine the results on the search results page there. Or just use google site search for your site, which will actually crawl the pages and hence find your various single_page urls: https://www.google.com/cse/
Best of luck.
I have not tested this, but maybe you can put a function getSearchableContent() in the single pages controller like you do for blocks. This would return the string to be searched. Would look something like this:
function getSearchableContent() {
// ... compose searchstring depending on the queried content.
return $searchstring;
}
But I don't know if this works for dynamic content. If not, I'd look into C5's search index core classes and try to extend them for your project.
I have a search engine that searches albums.
For each music album, I have a page.
So, the work flow goes like this:
People search for music titles
The search engine displays a list of albums.
People click on an album to go to a details page.
I want google to index my front page and the details page. I want the details page to be highly ranked. How can I build a sitemap for this?
By the way, I have about 5 million albums (but I want the top 1000 ones to be highly ranked on google)
You would not use a sitemap for that many results. You would want each album to appear as a page with a unique URI to reference that page. That way the search engine can crawl your site by crawling links since search bots cannot submit form data. Each of those URIs should be simple, meaning limited to this part of the URI syntax:
scheme://authority_segment/path
Program your web application to remove and throw away any extraneous data, such as query string or parameters. If you do this you have to be sure that you are watching for URI poisoning or SQL injection even through means of character encoding.
How can I build a sitemap for this?
By pulling the addresses out of your database and creating a XML file with a high priority for some selected pages. Somehow I think that isn’t your real question …
If I wanted to automate building a site map for a site like this, I'd employ Python. I'd pretty much write everything from the ground up (except the data store access). The format is quite simple.
I'm not sure I quite understand your question...