I'm working on a research project trying to understand the patterns and breakdowns of search usage and volumes in the United States.
Ideally, I would love a breakdown of search volume across topics like:
navigational (ie just want to get to a domain link)
news (if possible: split amongst events, celebs, politics, ...)
sports (if possible: dig into splits of live scores, news about an athlete or a team, ... )
finance (e.g. stock names )
anything local (e.g. food, restaurants, places)
people (e.g. bios)
anytime time related (what time in nyc, sf, ...)
anything numbers related (math/calculators)
Other topics: immigration, legal, health/medicine, science/technology, food/recipes, code/math, politics, weather, images/video, etc.
Not sure if there is a dataset or good report somewhere that would give me insight into all these?
There seem to be a lot of keyword planning tools, which is somewhat helpful and I guess I could collect data on groups of keywords realted to the topics above, but for things like celebrity bios it would be quite difficult to group together all the data because each possible well known person is there own keyword…
Any help, direction would be appreciated! Thank you so much
I'm starting an online business targeted at a particular demographic and interests so I would like to produce content targeted at what this particular target market are actually searching for.
Google Ads allowed me to refine my target audience to the exact categories (demographics and interests) I needed but I couldn't tell me what that category of people tend to search for except for the tiny subset that happens to click on one of my ads which is very rare given I am just starting with a small budget. I would like to know the most popular search terms for everyone in the categories I specified not just those who happened to click on my ads.
I tried Google Trends, that told me the popularity of a particular search term for a given country but that's too broad - I need to narrow it down to a particular city, age group, parental status and interests. Google Trends also helped me find popular related search terms given a particular search term so I could try using that to see if there are any common popular related search terms related to my guesses but I could miss terms related to terms I never thought of.
I could try producing content across a rage of topics which I think my target audience might be interested in and then analyse the results using Google Ads but that could be a very expensive trial and error process and I might miss more popular topics which I never thought of.
Of course I could try to ask my target market in person directly (by interrupting people in the street!) but that would be very expensive for me because I would have to travel to and stay at the location where my online business is targeted, hoping to meet people with the exact same demographic and interests that I am looking.
I'm sure there must be a way to figure this out using the the Google search analytics. Essentially, all I need is a list of most popular recent Google search terms for a particular location, demographic and interests group in Google Analytics. Could anyone help me understand how to get this list?
Here are a few considerations, even if you found an answer.
Take a look at the AdRoll platform. Here's a potentially helpful article from them about target audience and demographics.
A recent article about AdWords demographic targeting. An older looking article, connecting demographics to search queries, but page's source code suggests an update this year.
Last but not least, you're probably eligible to talk with a Google Small Business Advisor.
I am researching a problem that is pretty unique.
Imagine a roadside assistance company that wants to dynamically route its vehicles. Hence for each packet of new incidents wants to create routes that will satisfy them, according to some constraints (time constraints, road accessibility, vehicle - incident matching).
The company has an heterogeneous fleet of vehicle (motorbikes for easy cases, up to tow trucks for the hard cases) and each incident states it's uniqueness (we know if it wants just fuel, or needs towing).
There is no depot, only the vehicles roaming on the streets.
The objective is to dynamically create routes on the way, having in mind the minimization of time and the total traveled distance.
Have you ever met such a problem? Do you have any idea in which VRP variant it belongs?
I have seen two previous questions but unfortunately they don't fit with my problem.
The respected optaplanner - VRP but with no depot and Does optaplanner out of box support VRP with multiple trips and no depot, which are both open VRPs.
Unfortunately I don't have code right now, as I am still modelling the way I will approach this problem.
I am really sorry for creating a suggestion question and not a real one.
Thank you so much in advance.
It's a rich dynamic/realtime vehicle routing problem. You won't find an exact name for your problem, as when VRPs get too complex they don't fit inside any of the standard categories.
It's clearly a dynamic/realtime problem (the terms are used interchangeably) as you would typically only find out about roadside breakdowns at short notice.
Sometimes you're servicing a broken down car, which would be a single stop (so a vehicle routing problem). Sometimes you're towing a car, which would be a pick-up delivery problem. So you have a mix of both together.
You would want to get to the broken down vehicles ASAP and some would need fixing sooner than others (think a car broken down in a dangerous position on a motorway). You would therefore need soft time windows so you can penalise lateness instead of the standard hard time windows supported in most VRP formulations.
Also for you to be able to scale to larger problems, you need an incremental optimiser that can restart from the previous (possibly now infeasible) solution when new jobs are added, vehicle positions are changed etc. This isn't supported out of the box in the open source solvers I know of.
We developed a commercial engine which does the above. We started off using the jsprit library, which supports mixing single stop and pickup delivery problems together. We later had to replace jsprit due to the amount of code we had to override to get it running happily for realtime problems, however jsprit may still prove a useful starting point for you. We discuss some of the early technical obstacles we had to overcome in getting jsprit to handle realtime problems in this white paper.
We (my company) runs a website which have lots of data recorded like user registration, visits, clicks, what the stuff they post etc etc but so far we don't have a tool to find out how to monitor entire thing or how to find patterns in it so that we can understand what kind of information we can get from it? So that Mgmt can take decisions based on it. In short, the people do at Amazon or Google based on data they retrieve, we want a similar thing.
Now, after the intro, I would like to know what technology could it be called;is it Data Mining,Machine Learning or what? Where should we start to convert meaningless data into useful Information?
I think what you need enters in the "realm" of: parsing data, creating graphs, showing statistics about some elements, etc.
There is no "easy" answer, I can only answer parts of your question.
There are no premade magical analytical tools, big companies have their own backend tools tunned to parse the large amounts of data and spit out data summaries that are then used to build graphs or for statistical analysis.
I think the domain you are searching for is statistical data analysis. But there are many parts that go together here.
Best advice I can give you is to set up specific goals for you analysis and then try to see what is the best solution, you question is too open.
ie. if you are interested in visits/clicks/website related statistics Google Analytics is a great tool, and very easy to use.
over time our information strategy has gone all over the place and we are looking to have a clearer policy and a more explicit way for everyone to be in sync on information sharing. Some things to note is that the org is 300+ people and is in multiple countries across the world. Also, we have people that are comfortable in Sharepoint, people that are comfortable in confluence, etc so there is definately a "change" factor here
Here are our current issues and what we are thinking about doing about them. I would love to hear feedback, suggestions, etc.
The content we have today:
Technical design info / architecture docs
Meeting minutes, action items, etc
Project plans and roadmaps
organization business mgmt info - travel, budget info, headcount info, etc
Project pages with business analysis, requirements, etc
Here are some of our main issues:
Where should data go - Confluence WIKI versus Sharepoint versus intranet site - we use confluence WIKI for #1, #2, #3, #5 but we also use sharepoint for #1, #3, #4, #5. We are trying to figure out if we should mandate each number to a specific place to make things consistent. We are using Sharepoint more a directory structure of documents, and we are using confluence for more adhoc changable content.
Stale Data - this is maybe a cultural thing with the org but at certain points in time data just becomes stale and is no longer relevant. What is the best way to ensure old data doesn't create a lot of noise and to ensure that the latest correct data is up to date. Should there be people in the org responsible for this or should it be an implicit "everyones job". This is more of an issue when people leave, join, etc . .
More active usage - whats is the best way to get people off of email and trying to stop and think "could this be useful for others . . let me put it in a centralized place instead of in email chains" . .
also, any other stories of good ways to improve an org's communication and information management
A fundamental root cause of information clutter is "no ownership".
People are assigned to projects. The projects end (or are cancelled), the people move on and the documents remain behind to gather "dust" and become information clutter.
This is hard to prevent. The wiki vs. sharepoint doesn't address the clutter, it just shifts the technology base that's used to accumulate clutter.
Let's look at the clutter
Technical design info / architecture docs. Old ones don't matter. There's current and there's irrelevant. Wiki.
Last year's obsolete design information is -- well -- obsolete.
Meeting minutes, action items, etc. Action items become part of someone's backlog in a development sprint, or, they're probably never going to get done. Backlogs are wiki items. Everything else is history that might be interesting but usually isn't. If it didn't create a sprint backlog items, update an architecture, or solve a development problem, the meeting was probably a waste of time.
Project plans and roadmaps. The sprint backlog matters -- this is what a "plan and roadmap" aspires to be. If you have to supplement your plans with roadmaps, you probably ought to give up on the planning and just use Scrum and just keep the backlog current.
The original plan is someone's guess at project inception time, and not really very interesting to the current project team.
Organization business mgmt info - travel, budget info, headcount info, etc. This is a weird mixture of highly structured stuff (budget, organization) and unstructured stuff ("travel"?)
How much history do you need? None? Wiki at best. Financial or HR System is where it belongs. But, in big organizations, the accounting systems can be difficult and cumbersome to use, so we create secondary sources of information like a SharePoint page with out-of-date budget numbers because the real budget numbers are buried inside Oracle Financials.
Project pages with business analysis, requirements, etc. This is your backlog. Your project roadmap and your requirements and your analysis ought to be a single document. In the wiki.
History rarely matters. Someone's concept at project inception time of what the requirements are doesn't matter very much any more. What the requirements evolved to in their final form matters far more than any history. This is wiki material.
How old is 'too old'?
I've worked with customers that have 30-year old software. The software -- obviously -- is relevant because it's in production.
The documentation, however, is all junk. The software has been maintained. It's full of change control records. The "original" specifications would have to be meticulously rewritten with each change control folded in. Since the change control documents can be remarkably pervasive, the only way to see where the changes were applied is to read the source and -- from that -- reverse engineer the current-state specification.
If we can only understand a 30-year old app by reverse engineering the source, then, chuck the 30-year old pile of paper. It's useless.
As soon as maintenance is done, the "original" specification has been devalued.
How to clean it up?
If you create the wiki page or sharepoint site, you own it forever.
When you leave, your replacement owns it forever.
Each manager is 100% responsible for every piece of information their staff creates. They have to delete things. The weak solution is to "archive" stuff. Which is just a polite way of saying "delete" without the "D-word".
Cleanup must be every manager's ongoing responsibility. If they can't remember what it is, or why they own it, they should be required (or "encouraged") to delete it. Everything unaccessed in the last two years should be archived without question. Everything 10 years old is just irrelevant history.
It's painful, and it doesn't appear to be value-creating work. After all, we work in IT. Our job is to "write" software, not delete it. No one will do it unless compelled on threat of firing.
The cost of storage is relatively low. The cost of cleanup appears higher.
How to stop the email chain?
Refuse to participate. Create a "Break the Chain" campaign focused on replacing email chains with wiki updates (or sharepoint updates).
Be sure your wiki provides links and is faster to edit than an email.
You can't force people to give up a really, really convenient solution (Email). You have to make the wiki more valuable and almost as convenient as email.
Ramp up the value on the wiki. Deprecate email chains. Refuse to respond to email chains. Refuse to accept "to do" action items through email.
You can use Confluence Wiki for storing documents as attachements and have the Wiki's paths work as the file paths in Sharepoint.
Re: stale data: have ownership of the data (both person and team) and ensure that deliverables for the owners include maintenance of ALL the data.
As far as "Off email", this is hard to do as you can't force people to do this short of actively monitoring all email... but you can try some deliverables with metrics regarding content added to the Wiki. That way people would be more likely to want to re-use the work already done on the email to paste into Wiki to meet the "quota" instead of composing fresh stuff.
Our company and/or team used all 3 of these approaches with some degree of success in the past
Is there a reason not to have the wiki hold the files?
Also, perhaps limiting the mail server to not allowing attachments on internal emails is too draconian, but asking folks to put everything in the wiki that needs to be emailed more than once is pretty darn useful.
Efficient information management is indeed a very hard problem. We found that "the simpler the better" principle can make miracles to solve it.
Where should data go - we are big believers of the wiki approach. In fact, we use Confluence for sharing possibly every type of information, except really large binary files. For those, we use Dropbox. Its simplicity is an absolutely killer feature. (Tip: you can integrate them with the Dropbox in Confluence plugin.)
Finding stale data - in our definition, stale data is something that is not updated or viewed for a specific period of time. The Archiving Plugin of Confluence can quickly and automatically find these, then report them to the authors and administrators, who may potentially update them (or remove them, see next item). There is, of course, information that never expires, but the plugin is able to skip them after you mark the corresponding pages.
Removing stale data - we are fairly aggressive on this. If the data is not (highly) relevant anymore, clean it up now! We can safely follow this practice, because we never actually delete data. We just move outdated data to hidden archive spaces using, again, the Archiving Plugin. If we changed our mind later, it is very easy to find it in the the archive, view it or even to recover it.
More active usage - our rule: if the information is required to be persistent, don't email it. Put it to a wiki page instead. The hard thing for some people is to find the best location for the information (which space? where in the page hierarchy?). Badly organized spaces with vague scope are another big efficiency divider, unfortunately. Large companies may consider introducing a wiki gardener to cure this.