I am new to Qliksense and I am practicing app (dashboard) development concepts on MS SQL Server's Adventureworks database. In one specific table, the Address table, there is a column which has Spatial Location data. The data is in the following format, Dallas - 0xE6100000010C10A810D1886240403A0F0653663158C0. The data is of the geography datatype and is said to represent latitude and longitude information of given address. I am trying to create a map and a GeoKey as a dimension, but GeoMakePoint() function takes latitude and longitude as a tuple and not in this format. Please help.
I figured out the solution myself. Just use the method [Columnname].Lat and [Columnname].Long on the geography datatype to extract the Latitude and Longitude values from column values. Store these values in separate columns during data load and use them as GeoKey.
Related
I'm referring to the manual on how to query spatial data, but in my scenario, there is a pixel-wise raster dataset of climate data over 8000+ days.
In simple words, the query could look something like this:
select temperature data for 5000 days for the defined amorphous region and aggregate the values using spark
My questions are:
Should all the pixels be converted into vector-like "points" to be applicable for masking operations?
Can third dimension (time) be somehow added to the spatial index, or should date be just a regular column?
Grateful for any input!
Some visual representation:
I am interested in creating a gvNIX/Roo application which shows the location of health facilities in Tanzania on a map. I am trying the tutorial available here. However my data is in the format shown below where my location data is in two columns (southings and eastings). The tutorial shows how to create three data types:
field geo --fieldName location --type POINT --class ~.domain.Owner
field geo --fieldName distance --type LINESTRING --class ~.domain.Owner
field geo --fieldName area --type POLYGON --class ~.domain.Owner
Am assuming I need the POINT data type to hold data on a health facility location but am not sure how to get the below 2 columns (southings and eastings) into a single POINT variable. Am pretty new to GIS as well. The data is as below (csv format):
outlet_name,Status ,southings,eastings,streetward,name_of_outlet
REHEMA MEDICS,02,2.49993,32.89512,K/POLISI,REVINA
KIRUMBA MEDICS,02,2.50023,32.89503,K/POLISI,GEDION
KIRUMBA PHARMACY,02,2.50152,32.89742,K/POLISI,MAURETH
TULI MEDICS,02,2.48737,32.89686,KITANGIRI,TULI
JULLY MEDICS,02,2.53275,32.93855,BUZURUGA,JULLY
MAGOMA MEDICS,02,2.53181,32.94211,BUZURUGA,MAGOMA
MECO PHARMACY,02,2.52923,32.94730,MECCO,DORCAS
UPENDO MEDICS,02,2.52923,32.94786,MECCO,UPENDO
DORIS MEDICS,02,2.49961,32.89191,KABUHORO,DORIS
SOPHIA MEDICS,02,2.49975,32.89120,KABUHORO,ESTER
MWALONI PHAMCY,02,2.56351,32.89416,MWALONI,ESTER
SILVER PHAMACY,02,2.51728,32.90614,K/KILOMERO,WANDWATA
KIBO PHARMACY,02,2.51688,32.90710,MISSION,MARIAM
Thanks
You need to transform your coordinates to WKT format (Well Known Text) in order to insert them in a column in your database (a postgresql database with postgis support). In order to achieve this you need to follow these steps:
Find the SRID of your coordinates reference system (CRS). That is, the identificator which define your coordinates system. Otherwise, your points won't match the real coordinates. You'll need the SRID in the last step.
Transform your data to WKT. The data needed for inserting the points is in the southings and eastings columns (I suppose they are equal to latitude and longitude, that are the most common used), so you'll need to transform these columns in one single column with WKT format. e.g. for your first row of data: Point(32.89512 2.49993). Note the space between them and the switch between the numbers.
Proceed with the inserts with SQL syntax, but using postgis functions. An example for your first row would be: INSERT into health_facilities (outlet_name, Status, streetward, location) VALUES ('REHEMA MEDICS', 02, 'K/POLISI', ST_GeomFromText('Point(32.89512 2.49993)', 4326));. Where "4326" are the numbers of the SRID you have to find (supossing it is the most common -> EPSG:4326).
You can find more info here and here. Also there are several pages where you can check coordinates and transform them between diferent CRS, like this and this.
I am working on a system for storing and processing time series data from a couple of plants. Every plant has a different number of raw measurement values, each of them represented as a key-value pair.
The raw data needs to be preprocessed to obtain semantics. I also need to save the raw data, because the transformation process should be configurable. While I am new to No-Sql databases and Cassandra I searched for resources on the web and found the weather station example (similar described on other resources, too).
My requirements are similar to this example, but as extension I need a way to store a variable number of measurement values (key-pair) per plant. I also know, that my table model highly depends on the queries I want to run against it. The most common queries will be:
Get all values per key for a specific time (range) and plant.
Get all values per multiple keys for a specific time (range) and plant.
My question now is, how would a table structure look like that best fit theses requirements?
I thought about something like that, but don't know if it contains some drawbacks:
CREATE TABLE values_per_day (
plant_id text,
date text,
event_time timestamp,
key text,
value text,
PRIMARY KEY ((plant_id, date), event_time, address)
);
The recommendation for Cassandra is to start with the queries you want to perform. For each query, consider the inputs to the query, which indicate what data you want it to return. For each query you should have a table that has the inputs to the query as its primary key. If you want to query for a rangeof values, that value should be the cluster key (not the partition key) of a primary key, with the other inputs the partition key. If you want to query for very long value ranges, consider slicing that value into buckets.
I am trying to create a location based app where people can query the records created within 5 miles of their location.
When the record is created, I will store the Latitude and Longitude in the Azure Table Service.
Once I have this data, how do I fetch all the records within 5 miles from my current location?
Thank you.
For Azure Table Storage queries to be optimized, they'll need to run on the Partition Key and the Row Key. A solution could be to store the latitude in the Partition Key and the longitude in the RowKey. The Partition Key and Row Key combinations need to be unique (think primary key in SQL). I would use this strategy and if you have multiple entries for the same latitude and longitude, then you could use ATS' dynamic properties or InsertOrMerge to store them in the same row. That way you could query like this:
IQueryable<Entries> query =
(from q in _table.CreateQuery<Entries>()
where q.PartitionKey.CompareTo(minLatitude) > 0
&& q.PartitionKey.CompareTo(maxLatitude) < 0
&& q.RowKey.CompareTo(minLongitude) > 0
&& q.RowKey.CompareTo(maxLongitude) < 0
select q);
You could also get clever with the PartitionKey and use it to store a range of latitudes or regions in order to limit the # of partitions needed. SQL Azure also supports geospatial queries
Just doing work in this area and found that Azure Search will provide geospatial searches over Azure Table Storage. One setback is that the smallest scale beyond the developer sandbox is $200 per month - well worth the money for a commercial venture but rather high for small operations.
In order to make this work I needed to duplicate the Latatitude and Longitude fields into the GeoJson format, ie.
{"type": "Point", "coordinates": [102.0, 0.5]}
The free developer search option will allow one datasource based on partition key. For the purposes of my testing I have a table with everything in the same partition and unique RowKeys. I indexed the RowKey and the GeoJson value and found it works very well to search for all records within a radius of a given point.
While this is great, I think there or other storage solutions that will work better. DocumentDb and SqlAzure both support geospatial queries and, given the combination of the cost of storage and search, the cost of these alternatives is attractive.
I have a Horse Racing Database that has the results for all handicap races for the 2010 flat season. The spreadsheet has now got too big and I want to convert it to a MySQL Databse. I have looked at many sites about normalizing data and database structures but I just can't work out what goes where, and what are PRIMARY KEYS,FOREIGN KEYS ETC I have over 30000 lines in the spreadsheet. the Column headings are :-
RACE_NO,DATE,COURSE,R_TIME,AGE,FURS,CLASS,PRIZE,RAN,Go,BHB,WA,AA,POS,DRW,BTN,HORSE,WGT,SP,TBTN,PPL,LGTHS,BHB,BHBADJ,BEYER
most of the columns are obvious, the following explains the less obvious BHB is the class of race,WA and AA are weight allowances for age and weight,TBTN is total distance beaten,PPL is Pounds per length, the last 4 are ratings.
I managed to export into MySQL as a flat file by saving the spreadsheet as a comma delimited file but I need to structure the
data into a normalized state with the proper KEYS.
I would appreciate any advice
many thyanks
Davey H
To do this in the past, I've done it in several steps...
Import your Excel spreadsheet into Microsoft Access
Import your Microsoft Access database into MySQL using the MySQL Workbench (previously MySQL GUI Tools + MySQL Migration Toolkit)
It's a bit disjointed, but it usually works pretty well and saves me time in the long run.
It's kind of an involved question, and it would be difficult to give you a precise answer without knowing a little bit more about your system, but I can try and give you a high level overview of how Relational Database Mangement Systems (RDBMS's) are structured.
A primary key is some identifier for a particular record - usually it is unique to that record. In this case, your RACE_NO column might be a suitable primary key. That way, you can identify every race by its unique number.
Foreign keys are numbers that describe the relationships between other objects/tables in your database. For example, you may want to create a table that lists all the different classes of races. Each record in that table would have a primary key, unique to that class. If you wanted to indicate in your "races" table which class each race was, you might have a column for each record called class_id. The value of that column would be populated with primary keys from the "classes" table. You can then use join operations to bring all the information together into one view.
For more on data structures and mysql, I suggest the W3C tutorials on SQL: http://www.w3schools.com/sql/sql_intro.asp
Before anything else, You need to define your data: You have to fit every column into a value space known to MySQL.
Numeric value
http://dev.mysql.com/doc/refman/5.0/en/numeric-types.html
Textual value
http://dev.mysql.com/doc/refman/5.0/en/string-type-overview.html
Date/Time value
http://dev.mysql.com/doc/refman/5.0/en/date-and-time-type-overview.html