Primary key in nested structure on clickhouse database - nested

Question

In clickhouse I have created a table with a nested structure
CREATE TABLE IF NOT EXISTS table_name (
timestamp Date,
str_1 String,
Nested_structure Nested (
index_array UInt32,
metric_2 UInt64,
metric_3 UInt8
),
sign Int8 DEFAULT 1
) ENGINE = CollapsingMergeTree(sign) PARTITION BY (toYYYYMM(timestamp)) ORDER BY (timestamp, str_1)
The queries that I will make are like:
SELECT count(*) AS count FROM table_name
WHERE (timestamp = '2017-09-01')
AND
arrayFirst((i, x) -> x = 7151, Nested_structure.metric_2, Nested_structure.index_array) > 50000
I want to count str_1 where:
value of (array) column metric_2 in index matched from indexed_array for value 7151, is greater than a given threshold (50000)
I was wondering if it is possible to have a primary key for column: index_array in order to make query faster.
If I add column: Nested_structure.index_array in the order by clause it is assumed to be the array column of the large table and not the individual values of the column indexed_array of the Nested_structure
e.g. ORDER BY (timestamp, str_1, Nested_structure.index_array)
The algorithm is:
Search for the index of a given value in index_array
Having the index from step (1), retrieve the value from other arrays
If index_array is sorted and the table has knowledge of that, then step (1) could be faster (use a binary search algorithm for example)
Someone has an idea?
=============
EDIT
Cardinality of columns:
str_1 15,000,000 millions distinct values
index_array: 15,000 - 20,000 thousands distinct values
Assuming that index_array distinct values are: column_1, ..., column_15000, then a denormalized table should have the below structure:
timestamp,
str_1,
column_1a, <-- store values for metric_2
...
column_15000a, <-- store values for metric_2
column_1b, <-- store values for metric_3
...
column_15000b, <-- store values for metric_3
#Amos may you give me the structure of the table, if I use a column of type LowCardinality ?

Answer 1

I was wondering if it is possible to have a primary key for column: index_array in order to make query faster.
Nope, ClickHouse doesn't have array indices. If you supply Nested_structure.index_array as the third argument in the order by clause, it will just order the entire row taking into account the array column. Note, [1,2] < [1,2,3].
You can just denormalize the table without the nested column and make the first two columns with type LowCardinality which is almost production-ready.
Update
It seems you won't benefit much from LowCardinality types. What I meant was doing something like this
CREATE TABLE IF NOT EXISTS table_name (
timestamp Date,
str_1 String,
index_array UInt32,
metric_2 UInt64,
metric_3 UInt8,
sign Int8 DEFAULT 1
) ENGINE = CollapsingMergeTree(sign) PARTITION BY (toYYYYMM(timestamp)) ORDER BY (timestamp, str_1, index_array)
And you can still use the old insertion logic by doing this
CREATE TABLE IF NOT EXISTS table_name ( timestamp Date, str_1 String, index_array UInt32, metric_2 UInt64, metric_3 UInt8, sign Int8 DEFAULT 1 ) ENGINE = CollapsingMergeTree(sign) PARTITION BY (toYYYYMM(timestamp)) ORDER BY (timestamp, str_1, index_array)
CREATE TABLE IF NOT EXISTS source_table ( timestamp Date, str_1 String, Nested_structure Nested ( index_array UInt32, metric_2 UInt64, metric_3 UInt8 ), sign Int8 DEFAULT 1 ) ENGINE Null;
create materialized view data_pipe to table_name as select timestamp, str_1, Nested_structure.index_array index_array, Nested_structure.metric_2 metric_2, Nested_structure.metric_3 metric_3, sign from source_table array join Nested_structure;
insert into source_table values (today(), 'fff', [1,2,3], [2,3,4], [3,4,5], 1);