I have next table structure in Cassandra:
CREATE TABLE statistics (
clientId VARCHAR,
hits LIST<text>,
PRIMARY KEY (clientId)
);
INSERT INTO statistics(clientId, hits) VALUES ('clientId', [{'referer': 'http://example.com/asd', 'type': 'PAGE', 'page': '{"title": "Page title"}'}, {'referer': 'http://example.com/dsa', 'type': 'EVENT', 'event': '{"title": "Click on big button"}'}, {'referer': 'http://example.com/fgd', 'type': 'PAGE', 'page': '{"title": "Page title second"}'}]);
I want to select count of hits with type = 'PAGE'.
How can I do it ?
List is not the right structure for you use-case, consider the following schema
CREATE TABLE statistics(
client_id VARCHAR,
hit_type text,
referer text,
page text,
event text,
PRIMARY KEY ((client_id,hit_type), referer)
);
// Insert hits
INSERT INTO statistics(client_id, hit_type, referer, page)
VALUES('client1','PAGE', 'http://example.com/asd', '{"title": "Page title"}');
INSERT INTO statistics(client_id, hit_type, referer, event)
VALUES('client1','EVENT', 'http://example.com/dsa', '{"title": "Click on big button"}');
INSERT INTO statistics(client_id, hit_type, referer, page)
VALUES('client1','PAGE', 'http://example.com/fgd', '{"title": "Page title second"}');
//Select all hits for a given client and hit type:
SELECT * FROM statistics WHERE client_id='xxx' AND hit_type='PAGE';
Please note that with the above schema, it is not recommended to have more than 100 millions of referers for each couple (client_id,hit_type)
Related
I want to using nodejs mssql package to bulk insert data with below json:
[
{
"name": "Tom",
"registerDate": "2021-10-10 00:00:00",
"gender": 0,
"consumeRecord":[
{
"date": "2021-10-11 00:00:00",
"price": 102.5
},
{
"date": "2021-10-12 00:00:00",
"price": 200
}
]
},
{
"name": "Mary",
"registerDate": "2021-06-10 00:00:00",
"gender": 1,
"consumeRecord":[
{
"date": "2021-07-11 00:00:00",
"price": 702.5
},
{
"date": "2021-12-12 00:00:00",
"price": 98.2
}
]
}
]
I am try to mssql bulk insert for the member record with multiple consume data?
Is there anything can insert one to many with bulk insert like below.
because it seems need to insert the member table and get the id (primary key) first. Then using the id (primary key) for the consume table relation data
const sql = require('mssql')
// member table
const membertable = new sql.Table('Member')
table.columns.add('name', sql.Int, {nullable: false})
table.columns.add('registerDate', sql.VarChar(50), {nullable: false})
table.columns.add('gender', sql.VarChar(50), {nullable: false})
// consume record table
const consumeTable = new sql.Table('ConsumeRecord')
table.columns.add('MemberId', sql.Int, {nullable: false})
table.columns.add('Date', sql.VarChar(50), {nullable: false})
table.columns.add('price', sql.Money, {nullable: false})
// insert into member table
jsonList.forEach(data => {
table.rows.add(data.name)
table.rows.add(data.registerDate)
table.rows.add(data.gender)
consumeTable.rows.add(data.memberId) // <---- should insert member table id
consumeTable.rows.add(data.consumeRecord.data)
consumeTable.rows.add(data.consumeRecord.price)
const request = new sql.Request()
request.bulk(consumeTable , (err, result) => {
})
})
const request = new sql.Request()
request.bulk(membertable , (err, result) => {
})
Expected Record:
Member Table
id (auto increment)
name
registerDate
gender
1
Tom
2021-10-10 00:00:00
0
2
Mary
2021-06-10 00:00:00
1
Consume Record Table
id
MemberId
Date
price
1
1
2021-10-10 00:00:00
102.5
2
1
2021-10-12 00:00:00
200
3
2
2021-07-11 00:00:00
702.5
4
2
2021-12-12 00:00:00
98.2
The best way to do this is to upload the whole thing in batch to SQL Server, and ensure that it inserts the correct foreign key.
You have two options
Option 1
Upload the main table as a Table Valued Parameter or JSON blob
Insert with OUTPUT clause to select the inserted IDs back to the client
Correlate those IDs back to the child table data
Bulk Insert that as well
Option 2 is a bit easier: do the whole thing in SQL
Upload everything as one big JSON blob
Insert main table with OUTPUT clause into table variable
Insert child table, joining the IDs from the table variable
CREATE TABLE Member(
Id int IDENTITY PRIMARY KEY,
name varchar(50),
registerDate datetime NOT NULL,
gender tinyint NOT NULL
);
CREATE TABLE ConsumeRecord(
MemberId Int NOT NULL REFERENCES Member (Id),
Date datetime not null,
price decimal(9,2)
);
Note the more sensible datatypes of the columns
DECLARE #ids TABLE (jsonIndex nvarchar(5) COLLATE Latin1_General_BIN2 not null, memberId int not null);
WITH Source AS (
SELECT
j1.[key],
j2.*
FROM OPENJSON(#json) j1
CROSS APPLY OPENJSON(j1.value)
WITH (
name varchar(50),
registerDate datetime,
gender tinyint
) j2
)
MERGE Member m
USING Source s
ON 1=0 -- never match
WHEN NOT MATCHED THEN
INSERT (name, registerDate, gender)
VALUES (s.name, s.registerDate, s.gender)
OUTPUT s.[key], inserted.ID
INTO #ids(jsonIndex, memberId);
INSERT ConsumeRecord (MemberId, Date, price)
SELECT
i.memberId,
j2.date,
j2.price
FROM OPENJSON(#json) j1
CROSS APPLY OPENJSON(j1.value, '$.consumeRecord')
WITH (
date datetime,
price decimal(9,2)
) j2
JOIN #ids i ON i.jsonIndex = j1.[key];
db<>fiddle
Unfortunately, INSERT only allows you to OUTPUT from the inserted table, not from any non-inserted columns. So we need to hack it with a weird MERGE
I have a small database as defined in the code snippet below. I want to query this to get all of the information and send it to an vue app via a JSON file via a Flask API. At the moment the query that I am using is
SELECT tbl_room.room, tbl_room.room_id, tbl_device.name, tbl_display.display, tbl_function.function, tbl_device.format
FROM tbl_device
INNER JOIN tbl_room ON tbl_room.id = tbl_device.room_id
INNER JOIN tbl_display ON tbl_display.id = tbl_device.display_id
INNER JOIN tbl_function ON tbl_function.id = tbl_device.function_id
ORDER BY tbl_room.room_id;
this gives me an output like:
Bedroom (Main) bedroom_main bme280/1 gauge temperature {"min": 0, "max": 50, "dp": 1, "units": "°C"}
Bedroom (Main) bedroom_main bme280/1 gauge humidity {"min": 0, "max": 100, "dp": 1, "units": "%"}
Bedroom (Main) bedroom_main bme280/1 gauge pressure {"min": 0, "max": 1100, "dp": 1, "units": "hPa"}
Front Room front_room ds18b20/heater gauge temperature {"min": 0, "max": 50, "dp": 1, "units": "°C"}
I would like to get it into a JSON file so that it is arranged as:
[
{ "name": "Office",
"id": "office",
"devices": []
},
{ "name": "Front Room",
"id": "front_room",
"devices": []
}
}
]
Can this be done in a single sql query? Or do I have to do a query for each room in a loop? Or is it more efficient to dump the whole dataset out in one query and process it in pyhton afterwards? This is a small dataset but I'm interested to know which is the most efficient method.
Thank you in advance,
Martyn
Here is my table structure:
-- Table: tbl_device
CREATE TABLE tbl_device (
name VARCHAR NOT NULL ON CONFLICT ROLLBACK,
room_id INTEGER CONSTRAINT fk_room REFERENCES tbl_room (id)
NOT NULL,
function_id INTEGER CONSTRAINT fk_function REFERENCES tbl_function (id)
NOT NULL ON CONFLICT ROLLBACK,
display_id INTEGER CONSTRAINT fk_display REFERENCES tbl_display (id)
NOT NULL ON CONFLICT ROLLBACK,
format VARCHAR NOT NULL ON CONFLICT ROLLBACK
DEFAULT [default],
UNIQUE (
name,
room_id,
function_id,
display_id
)
ON CONFLICT ROLLBACK
);
-- Table: tbl_display
CREATE TABLE tbl_display (
id INTEGER PRIMARY KEY AUTOINCREMENT,
display VARCHAR NOT NULL ON CONFLICT ROLLBACK
UNIQUE ON CONFLICT ROLLBACK
);
-- Table: tbl_function
CREATE TABLE tbl_function (
id INTEGER PRIMARY KEY AUTOINCREMENT,
function VARCHAR NOT NULL ON CONFLICT ROLLBACK
UNIQUE ON CONFLICT ROLLBACK,
control BOOLEAN NOT NULL
DEFAULT (0)
);
-- Table: tbl_room
CREATE TABLE tbl_room (
id INTEGER PRIMARY KEY AUTOINCREMENT,
room_id VARCHAR NOT NULL
UNIQUE ON CONFLICT ROLLBACK,
room VARCHAR NOT NULL ON CONFLICT ROLLBACK
);
First, There is no way to directly feed JSON response from MySQL database to VueJS or any other App. VueJS App is the fronted of your application. You have to create a Backend which connects to MySQL database, fetch necessary data from MYSQL Database, Convert them to JSON and send to Vue App.
To Develop a backend, you may use languages such as PHP, Python, Java, NodeJS etc.
If you can continue with PHP, it is very easy to fetch data and convert to JSON.
But If you still need to continue with Python, you have to use Flask or any other python web framework to do that.
Here is the sample php code
<?php
$dbhost = 'hostname';
$dbuser = 'username';
$dbpass = 'password';
$dbname = 'database';
$db = new mysqli($dbhost, $dbuser, $dbpass, $dbname);
if ($db->connect_errno) {
printf("Failed to connect to database");
exit();
}
$result = $db->query("SELECT * FROM "); // Your SQL query
$data = array();
while ( $row = $result->fetch_assoc()) {
$data[]=$row;
}
echo json_encode($data);
?>
If your version of sqlite was compiled with the JSON1 extension, enabled, something like:
SELECT json_group_array(json_object('name', tbl_room.name,
'id', tbl_room.room_id,
'devices', json_array()))
FROM tbl_room
GROUP BY tbl_room.name, tbl_room.room_id;
I am working on a web application where i have to do a import export functionality. i am new in this so i want your suggestions and there are some issues that i am facing to do this functionality,
So first i have a JSON array from Frontend (AngularJs)
[
{
"MyName": "Shubham",
"UserType": "Premium",
"DialCode": "India",
"ContactNumber": "9876543210",
"EmailAddress": "Contact#Shubh.com"
"Country": "India",
"Notes": "Notes-Notes-Notes-Notes"
},
{
"MyName": "Shubham 2",
"UserType": "Free Trial",
"DialCode": "India",
"ContactNumber": "123456789",
"EmailAddress": "Contact2#Shubh.com"
"Country": "India",
"Notes": "Notes-Notes-Notes-Notes"
} ]
Now i am converting this array to a XML in NodeJs Using XMLbuilder
Like This
<UserXML>
<MyName>Shubham</MyName>
<UserType>Premium</UserType>
<DialCode>India</DialCode>
<ContactNumber>9876543210</ContactNumber>
<EmailAddress>Contact#Shubh.com</EmailAddress>
<Country>India</Country>
<Notes>Notes-Notes-Notes-Notes</Notes>
</UserXML>
<UserXML>
<MyName>Shubham 2</MyName>
<UserType>Free Trial</UserType>
<DialCode>India</DialCode>
<ContactNumber>123456789</ContactNumber>
<EmailAddress>Contact2#Shubh.com</EmailAddress>
<Country>India</Country>
<Notes>Notes2-Notes2-Notes2-Notes2</Notes>
</UserXML>
Now i am using this XML in SQL Server to insert these 2 records into my user table
now the issue is i have another table where country code and country name is saved i am using Foreign Key ref in my user table
i have made a sample code
DROP TABLE IF EXISTS #Country
DROP TABLE IF EXISTS #user
DROP TABLE IF EXISTS #UserType
CREATE TABLE #Country
(
Id INT PRIMARY KEY identity(1 ,1),
Name VARCHAR(50) NOT NULL,
DialCode VARCHAR(50) NOT NULL
)
CREATE TABLE #UserType
(
Id INT PRIMARY KEY identity(1 ,1),
Name VARCHAR(50) NOT NULL
)
CREATE TABLE #user
(
Id INT PRIMARY KEY IDENTITY(1 ,1),
Name VARCHAR(50) NOT NULL,
UserTypeId INT NOT NULL,
DialCodeId INT NOT NULL,
ContactNumber VARCHAR(50) NOT NULL,
EmailAddress VARCHAR(50) NOT NULL,
CountryId INT NOT NULL,
Notes VARCHAR(50) NOT NULL
FOREIGN KEY(CountryId) REFERENCES #Country(Id),
FOREIGN KEY(UserTypeId) REFERENCES #UserType(Id)
);
INSERT INTO #Country (Name,DialCode)
VALUES ('India','+91'),
('Dubai','+971'),
('U.S','+1') ;
INSERT INTO #UserType (Name)
VALUES ('Premium'),
('Free Trial');
CASE 1 (Working Fine) if i have a single record then there is no issue by using this apporch
declare #xml xml = '<UserXML>
<Name>Shubham CASE-1</Name>
<UserType>Premium</UserType>
<DialCode>India</DialCode>
<ContactNumber>9876543210</ContactNumber>
<EmailAddress>Contact#Shubh.com</EmailAddress>
<Country>India</Country>
<Notes>Notes-Notes-Notes-Notes CASE-1</Notes>
</UserXML>'
Now i have to check the country name/User Type and match it with the country table to get the id
DECLARE #CountryId INT
,#UserType INT
SELECT #CountryId = id FROM #Country WHERE Name LIKE ''+(select U.Items.value('./DialCode[1]','NVARCHAR(200)') as DialCode FROM #xml.nodes('/UserXML') U(Items))+'%'
SELECT #UserType = id FROM #UserType WHERE Name LIKE ''+(select U.Items.value('./UserType[1]','NVARCHAR(200)') as UserType FROM #xml.nodes('/UserXML') U(Items))+'%'
INSERT INTO #user
SELECT
U.Item.query('./Name').value('.','VARCHAR(100)') Name,
#UserType,
#CountryId,
U.Item.query('./ContactNumber').value('.','VARCHAR(100)') ContactNumber,
U.Item.query('./EmailAddress').value('.','VARCHAR(100)') EmailAddress,
#CountryId,
U.Item.query('./Notes').value('.','VARCHAR(100)') Notes
FROM #Xml.nodes('/UserXML') AS U(Item)
CASE-2 (Well the Isssue is here) if i have multiple records then how can i check every node and then make a join or something like that to make my insert query work fine
declare #xml2 xml = '<UserXML>
<Name>Shubham CASE-2</Name>
<UserType>Premium</UserType>
<DialCode>India</DialCode>
<ContactNumber>9876543210</ContactNumber>
<EmailAddress>Contact#Shubh.com</EmailAddress>
<Country>India</Country>
<Notes>Notes-Notes-Notes-Notes CASE-2</Notes>
</UserXML>
<UserXML>
<Name>Shubham 2 CASE-2</Name>
<UserType>Free Trial</UserType>
<DialCode>Dubai</DialCode>
<ContactNumber>123456789</ContactNumber>
<EmailAddress>Contact2#Shubh.com</EmailAddress>
<Country>Dubai</Country>
<Notes>Notes2-Notes2-Notes2-Notes2 CASE-2</Notes>
</UserXML>'
DECLARE #CountryId2 INT
,#UserType2 INT
SELECT #CountryId2 = id FROM #Country WHERE Name LIKE ''+(select U.Items.value('./DialCode[1]','NVARCHAR(200)') as DialCode FROM #xml2.nodes('/UserXML') U(Items))+'%'
SELECT #UserType2 = id FROM #UserType WHERE Name LIKE ''+(select U.Items.value('./UserType[1]','NVARCHAR(200)') as UserType FROM #xml2.nodes('/UserXML') U(Items))+'%'
INSERT INTO #user
SELECT
U.Item.query('./Name').value('.','VARCHAR(100)') Name,
#UserType,
#CountryId,
U.Item.query('./ContactNumber').value('.','VARCHAR(100)') ContactNumber,
-- U.Item.query('./EmailAddress').value('.','VARCHAR(100)') EmailAddress,
#CountryId,
U.Item.query('./Notes').value('.','VARCHAR(100)') Notes
FROM #xml2.nodes('/UserXML') AS U(Item)
Please If you have any suggestions Or any other better approach for doing this task then help me out i am new to this so i don't know about the best approach for doing this task
You can read JSON directly and perform the JOIN
SELECT entity.*
FROM OPENROWSET (BULK N'd:\temp\file.json', SINGLE_CLOB) as j
CROSS APPLY OPENJSON(BulkColumn)
WITH(
MyName nvarchar(100)
,UserType nvarchar(100)
,DialCode nvarchar(100)
,ContactNumber nvarchar(100)
,EmailAddress nvarchar(100)
,Country nvarchar(100)
,Notes nvarchar(500)
) AS entity
More infos about import JSON documents and the OPENJSON
I am trying to create a table for my website and for some reason it is only showing the first row of data.
This is how I am formatting the columns of the data:
const { items } = this.props.item;
// console.log({ items });
// react - bootstrap - table - next
const columns = [{
dataField: 'team',
text: 'Team',
sort: true,
formatter: (cellContent, row, rowIndex) => (
Object.values(row.team)[rowIndex]
)
}, {
dataField: 'current_Rank',
text: 'Current Rank',
sort: true,
formatter: (cellContent, row, rowIndex) => (
Object.values(row.current_Rank)[rowIndex]
)
}, {
dataField: 'new_Rank',
text: '321 Rank',
sort: true,
formatter: (cellContent, row, rowIndex) => (
Object.values(row.new_Rank)[rowIndex]
)
}];
This is how I am returning the table so that it renders the table:
return (
<BootstrapTable
keyField="team"
data={items}
columns={columns}
striped
hover />
)
}
}
The data:
Picture from the console
Live site: https://nhl-321-pointsystem.herokuapp.com/
I looked up your network response for /api/items API call, and found out that the data contains only one item. This being one of the reason you're seeing a single row when the table is rendered.
Please note the, another reason for the issue is, react-bootstrap-table-next key
data accepts a single Array object. And not array of single object.
You should re-arrange your data so that key 'team' will be present for all items in the array. And rest of the column header values (e.g. current_Rank) are available for each like.
Something like a reformat function I created in the sandbox available here.
Plus point - After you apply the reformat function, you won't need formatter for each column unlike your previous solution.
Alternate but recommended solution would be to send the formatted response from the API endpoint only, instead of re-parsing and creating new object to fit the needs of UI.
Sandbox link - https://codesandbox.io/embed/32vl4x4oj6
I have an SQLite3 table with BLOB primary key (id):
CREATE TABLE item (
id BLOB PRIMARY KEY,
title VARCHAR(100)
);
In javascript models, the primary key (id) is represented as a Javascript string (one HEX byte per character):
var item = {
id: "2202D1B511604790922E5A090C81E169",
title: "foo"
}
When I run the query below, the id parameter gets bound as a string. But I need it to be bound as a BLOB.
db.run('INSERT INTO item (id, title) VALUES ($id, $title)', {
$id: item.id,
$title: item.title
});
To illustrate, the above code generates the following SQL:
INSERT INTO item (id, title) VALUES ("2202D1B511604790922E5A090C81E169", "foo");
What I need is this:
INSERT INTO item (id, title) VALUES (X'2202D1B511604790922E5A090C81E169', "foo");
Apparently, the string needs to be converted to a buffer:
db.run('INSERT INTO item (id, title) VALUES ($id, $title)', {
$id: Buffer.from(item.id, 'hex'),
$title: item.title
});
Try casting the string as a blob:
INSERT INTO item(id, title) VALUES(CAST(id_string AS BLOB), 'foo');
Note also that the right way to quote strings in SQL is to use single quotes.