Add Column with random value Sequelize PostgreSQL - node.js

I want to add column with NOT_NULL constraint so column will contain random default values following is my code how can i do this
up: function (queryInterface, Sequelize, done) {
queryInterface.addColumn(
{
tableName: 'someTable',
schema: 'sometemplate'
},
'someColumn', //column name
{ //column date type and constraint
type: Sequelize.STRING,
allowNull: false,
defaultValue: // I want this to random value
})
.then(function () { done() })
.catch(done);
}

PostgreSQL example:
CREATE OR REPLACE FUNCTION f_random_text(
length integer
)
RETURNS text AS
$body$
WITH chars AS (
SELECT unnest(string_to_array('A B C D E F G H I J K L M N O P Q R S T U V W X Y Z 0 1 2 3 4 5 6 7 8 9', ' ')) AS _char
),
charlist AS
(
SELECT _char FROM chars ORDER BY random() LIMIT $1
)
SELECT string_agg(_char, '')
FROM charlist
;
$body$
LANGUAGE sql;
DROP TABLE IF EXISTS tmp_test;
CREATE TEMPORARY TABLE tmp_test (
id serial,
data text default f_random_text(12)
);
INSERT INTO tmp_test
VALUES
(DEFAULT, DEFAULT),
(DEFAULT, DEFAULT)
;
SELECT * FROM tmp_test;
id | data
----+--------------
1 | RYMUJH4E0NIQ
2 | 7U4029BOKAEJ
(2 rows)
Apparently you can do this. (Of course, you can add other characters as well, or use other random string generator as well - like this, for example.)
ref: https://dba.stackexchange.com/questions/19632/how-to-create-column-in-db-with-default-value-random-string

Related

Select matching raws in table with Minizinc

I have a table of Projects with 4 features (name, value, cost, people). I want to describe a model in Minizinc that selects the projects that maximize the total value but with a total budget of 255 and I can choose almost 9 projects between 20.
I write a data file: data.dzn
% name value cost people
data = [
| PRJ0, 6000, 35, 5
| PRJ1, 4000, 34, 3
| PRJ2, 1000, 26, 4
...
| PRJ20, 1200, 18, 2
|];
PRJ = {PRJ0,...,PRJ20};
FEATURE = {name, value, budget, personnel};
max_budget = 225;
max_prj=9;
So my constraints are:
choose_project <= 9 /\ budget<=255 s.t solve maximize tot_value;
How can I select a non-arbitrary number (1 to 9) of decision variables among projects raw in the table?
Until now this is my code: invest.mzn
include "data.dzn";
int: max_budget; %255
int: max_prj; %9
enum FEATURE;
enum PRJ;
array[PRJ,FEATURE] of int: data;
constraint ...
...
solve maximize tot_value;
You can declare an array of Boolean variables, say selected_projects, that encodes whether or not a project PRJ_k is selected or not.
Then you can simply count how many projects in this array are being selected at the same time.
Example:
enum FEATURE = {name, value, budget, personnel};
enum PRJ = { PRJ0, PRJ1 };
array[PRJ,FEATURE] of int: data =
[| PRJ0, 6000, 35, 5
| PRJ1, 4000, 34, 3
|];
array[PRJ] of var bool: selected_projects;
var int: tot_value;
% The total number of selected projects must be in [1, 9]
constraint let {
var int: tot_selected = sum(prj in PRJ) ( selected_projects[prj] )
} in
1 <= tot_selected /\ tot_selected <= 9;
constraint tot_value = sum(prj in PRJ) ( selected_projects[prj] * data[prj, value] );
% ...
% encoding of budget and personnel constraints
% ...
solve maximize tot_value;

How to update or insert millions of rows via node oracle-db?

I'm struggling with a question - how can I insert or update a lot of data (thousands or millions of rows) using node oracle-db driver?
The point is that I can select a lot of data with the help of resultSet (handling result set)... but then I have to make some actions with a row and later update or insert a new row. And here is the problem - I don't know how to do it as fast as possible.
Can anybody help me with a piece of advice? Thanks.
I can assure you that these actions can't be done in db.
Actually, there are lots of different ways this can be done in the DB via SQL and PL/SQL when needed. Folks often want to use the language they are comfortable with, maybe JavaScript in this case, but performance will be much better if the data doesn't have to fly around between tiers.
Here's an example in just SQL alone... Granted, this could have been done via virtual columns, but it should illustrate the point.
Imagine we have the following tables:
create table things (
id number not null,
val1 number not null,
val2 number not null,
constraint things_pk primary key (id)
);
insert into things (id, val1, val2) values (1, 1, 2);
insert into things (id, val1, val2) values (2, 2, 2);
insert into things (id, val1, val2) values (3, 5, 5);
-- Will hold the sum of things val1 and val2
create table thing_sums (
thing_id number,
sum number
);
alter table thing_sums
add constraint thing_sums_fk1
foreign key (thing_id)
references things (id);
Now, the easiest and most performant way to do this would be via SQL:
insert into thing_sums (
thing_id,
sum
)
select id,
val1 + val2
from things
where id not in (
select thing_id
from thing_sums
);
Here's another example that does the same thing only via PL/SQL which can provide more control.
begin
-- This cursor for loop will bulk collect (reduces context switching between
-- SQL and PL/SQL engines) implictly.
for thing_rec in (
select *
from things
where id not in(
select thing_id
from thing_sums
)
)
loop
-- Logic in this loop could be endlessly complex. I'm inserting the values
-- within the loop but this logic could be modified to store data in arrays
-- and then insert with forall (another bulk operation) after the loop.
insert into thing_sums(
thing_id,
sum
) values (
thing_rec.id,
thing_rec.val1 + thing_rec.val2
);
end loop;
end;
Either of those could be called from the Node.js driver. However, let's say you need to do this from the driver (maybe you're ingesting data that's not already in the database). Here's an example the demonstrates calling PL/SQL from the driver that uses bulk processing rather than row by row operations. This is much faster due to reduced round trips.
I pulled this from a blog post I'm working on so the table definition is a little different:
create table things (
id number not null,
name varchar2(50),
constraint things_pk primary key (id)
);
And here's the JavaScript:
var oracledb = require('oracledb');
var async = require('async');
var config = require('./dbconfig');
var things = [];
var idx;
function getThings(count) {
var things = [];
for (idx = 0; idx < count; idx += 1) {
things[idx] = {
id: idx,
name: "Thing number " + idx
};
}
return things;
}
things = getThings(500);
oracledb.getConnection(config, function(err, conn) {
var ids = [];
var names = [];
var start = Date.now();
if (err) {throw err;}
// We need to break up the array of JavaScript objects into arrays that
// work with node-oracledb bindings.
for (idx = 0; idx < things.length; idx += 1) {
ids.push(things[idx].id);
names.push(things[idx].name);
}
conn.execute(
` declare
type number_aat is table of number
index by pls_integer;
type varchar2_aat is table of varchar2(50)
index by pls_integer;
l_ids number_aat := :ids;
l_names varchar2_aat := :names;
begin
forall x in l_ids.first .. l_ids.last
insert into things (id, name) values (l_ids(x), l_names(x));
end;`,
{
ids: {
type: oracledb.NUMBER,
dir: oracledb.BIND_IN,
val: ids
},
names: {
type: oracledb.STRING,
dir: oracledb.BIND_IN,
val: names
}
},
{
autoCommit: true
},
function(err) {
if (err) {console.log(err); return;}
console.log('Success. Inserted ' + things.length + ' rows in ' + (Date.now() - start) + ' ms.');
}
);
});
I hope that helps! :)

Accessing rows outside of window while aggregating in Spark dataframe

In short, in the example below I want to pin 'b to be the value in the row that the result will appear in.
Given:
a,b
1,2
4,6
3,7 ==> 'special would be: (1-7 + 4-7 + 3-7) == -13 in this row
val baseWin = Window.partitionBy("something_I_forgot").orderBy("whatever")
val sumWin = baseWin.rowsBetween(-2, 0)
frame.withColumn("special",sum( 'a - 'b ).over(win) )
Or another way to think of it is I want to close over the row when I calculate the sum so that I can pass in the value of 'b (in this case 7)
* Update *
Here is what I want to accomplish as an UDF. In short, I used a foldLeft.
def mad(field : Column, numPeriods : Integer) : Column = {
val baseWin = Window.partitionBy("exchange","symbol").orderBy("datetime")
val win = baseWin.rowsBetween(numPeriods + 1, 0)
val subFunc: (Seq[Double],Int) => Double = { (input: Seq[Double], numPeriods : Int) => {
val agg = grizzled.math.stats.mean(input: _*)
val fooBar = (1.0 / -numPeriods)*input.foldLeft(0.0)( (a,b) => a + Math.abs((b-agg)) )
fooBar
} }
val myUdf = udf( subFunc )
myUdf(collect_list(field.cast(DoubleType)).over(win),lit(numPeriods))
}
If I understood correctly what you're trying to do, I think you can refactor your logic a bit to achieve it. The way you have it right now, you're probably getting "-7" instead of -13.
For the "special" column, (1-7 + 4-7 + 3-7), you can calculate it like (sum(a) - count(*) * b):
dfA.withColumn("special",sum('a).over(win) - count("*").over(win) * 'b)

How do I add two column values in a table with CQL?

I am needing to add two values together to create a third value with CQL. Is there any way to do this? My table has the columns number_of_x and number_of_y and I am trying to create total. I did an update on the table with a set command as follows:
UPDATE my_table
SET total = number_of_x + number_of_y ;
When I run that I get the message back saying:
no viable alternative at input ';'.
Per docs. assignment is one of:
column_name = value
set_or_list_item = set_or_list_item ( + | - ) ...
map_name = map_name ( + | - ) ...
map_name = map_name ( + | - ) { map_key : map_value, ... }
column_name [ term ] = value
counter_column_name = counter_column_name ( + | - ) integer
And you cannot mix counter and non counter columns in the same table so what you are describing is impossible in a single statement. But you can do a read before write:
CREATE TABLE my_table ( total int, x int, y int, key text PRIMARY KEY )
INSERT INTO my_table (key, x, y) VALUES ('CUST_1', 1, 1);
SELECT * FROM my_table WHERE key = 'CUST_1';
key | total | x | y
--------+-------+---+---
CUST_1 | null | 1 | 1
UPDATE my_table SET total = 2 WHERE key = 'CUST_1' IF x = 1 AND y = 1;
[applied]
-----------
True
SELECT * FROM my_table WHERE key = 'CUST_1';
key | total | x | y
--------+-------+---+---
CUST_1 | 2 | 1 | 1
The IF clause will handle concurrency issues if x or y was updated since the SELECT. You can than retry again if applied is False.
My recommendation however in this scenario is for your application to just read both x and y, then do addition locally as it will perform MUCH better.
If you really want C* to do the addition for you, there is a sum aggregate function in 2.2+ but it will require updating your schema a little:
CREATE TABLE table_for_aggregate (key text, type text, value int, PRIMARY KEY (key, type));
INSERT INTO table_for_aggregate (key, type, value) VALUES ('CUST_1', 'X', 1);
INSERT INTO table_for_aggregate (key, type, value) VALUES ('CUST_1', 'Y', 1);
SELECT sum(value) from table_for_aggregate WHERE key = 'CUST_1';
system.sum(value)
-------------------
2

AS2: How to iterate X times through a percentage calculation (containing a circular reference)?

Here is a question for the Excel / math-wizards.
I'm having trouble doing a calculation which is based on a formula with a circular reference. The calculation has been done in an Excel worksheet.
I've deducted the following equations from an Excel file:
a = 240000
b = 1400 + c + 850 + 2995
c = CEIL( ( a + b ) * 0.015, 100 )
After the iterations the total of A+B is supposed to be 249045 (where b = 9045).
In the Excel file this gives a circular reference, which is set to be allowed to iterate 4 times.
My problem: Recreate the calculation in AS2, going through 4 iterations.
I am not good enough at math to break this problem down.
Can anyone out there help me?
Edit: I've changed the formatting of the number in variable a. Sorry, I'm from DK and we use period as a thousand separator. I've removed it to avoid confusion :-)
2nd edit: The third equation, C uses Excels CEIL() function to round the number to nearest hundredth.
I don't know action script, but I think you want:
a = 240000
c = 0
for (i = 0; i < 4; i++){
b = 1400 + c + 850 + 2995
c = (a + b) * 0.015
}
But you need to determine what to use for the initial value of c. I assume that Excel uses 0, since I get the same value when running the above as I get in Excel with iterations = 4, c = 3734.69...
Where do you get the "A + B is supposed to be 249045" value? In Excel and in the above AS, b only reaches 8979 with those values.
function calcRegistrationTax( amount, iterations ) {
function roundToWhole( n, to ) {
if( n > 0 )
return Math.ceil( n/ to ) * to;
else if( n < 0)
return Math.floor( n/ to ) * to;
else
return to;
}
var a = amount;
var b = 0;
var c = 0
for (var i = 0; i < iterations; i++){
b = basicCost + ( c ) + financeDeclaration + handlingFee;
c = ( a + b ) * basicFeeRatio;
c = roundToWhole( c, 100 );
}
return b;
}
totalAmount = 240000 + calcRegistrationTax( 240000, 4 ); // This gives 249045
This did it, thanks to Benjamin for the help.

Resources