Difficulty using csvtojson to create nested JSON structure from csv - node.js

There's a feature in the csvtojson package where you can create a nested JSON object from a CSV file permitting you configure the headers correctly. However, I'm having difficulty applying the schema shown in the above link for my CSV file.
My current CSV structure is this:
And the JSON structure I'm trying to create should look something like this:
{
Name: 'Bedminster Twp East',
Contests: [
{
Name: 'JUSTICE OF THE SUPREME COURT',
Candidates: [
{
Name: 'Maria McLaughlin',
Votes: 511
},
{
Name: 'Kevin Brobson',
Votes: 857
},
{
Name: 'Write-In',
Votes: 1
},
]
},
{
Name: 'JUDGE OF THE SUPERIOR COURT',
Candidates: [
{
Name: 'Timika Lane',
Votes: 494
},
{
Name: 'Megan Sullivan',
Votes: 870
},
{
Name: 'Write-In',
Votes: 1
},
]
}
...
],
Turnout: '45.77%'
},
...
This is the existing code I have:
const csv = require('csvtojson');
const axios = require('axios');
axios({
method: 'get',
url: 'URL'
}).then(function(response) {
let x = response.data.split('\n');
x.splice(0, 2);
let newX = x.join('\n');
csv({
noheader: false,
headers: ['Precinct.Name', 'Precinct.Contests.Name', 'Precinct.Contests.Name.Candidates.Name', 'Precinct.Contests.Name.Candidates.Votes', 'Precinct.Turnout']
}).fromString(newX).then((csvRow) => {
console.log(csvRow);
});
});
Which is unfortunately creating the following JSON structure:
[
{
Precinct: {
Name: 'Bedminster Twp East',
Contests: [Object],
Turnout: '47.89%'
}
},
{
Precinct: {
Name: 'Bedminster Twp East',
Contests: [Object],
Turnout: '47.89%'
}
},
{
Precinct: {
Name: 'Bedminster Twp East',
Contests: [Object],
Turnout: '47.89%'
}
},
{
Precinct: {
Name: 'Bedminster Twp East',
Contests: [Object],
Turnout: '47.89%'
}
},
... 19841 more items
I don't know exactly what is in the "Contests" object, but considering that the number of objects is equal to the number of rows in the existing CSV file, I believe there is something wrong with my header grouping. I can't tell if I'm severely misreading the documentation linked earlier or if it's not clear enough. Thanks for any assistance.
EDIT: I'm splitting and joining because the first two rows are metadata that says "UNOFFICIAL RESULTS" and "2022", which I don't want being read in.

try this headers configuration:
headers: ['Name', 'Contests[0].Name', 'Contests[0].Candidates[0].Name', 'Contests[0].Candidates[0].Votes', 'Turnout']

Related

NodeJS Validation with JOI [duplicate]

This question already has answers here:
How to validate array of objects using Joi?
(6 answers)
Closed last year.
I am using bellow code:
const coll = [
{ id: 1, name: 'John' },
{ id: 2, name: 'Jemmy' },
{ id: 3, name: 'Jenny' }
];
const schema = Joi.object().keys({
name: Joi.string().min(3).required()
});
return schema.validate(coll);
when my coll array is valid, then also when checking the schema, it shows the following, always going to the error section.
schema validator
{
value: [
{ id: 1, name: 'Action' },
{ id: 2, name: 'Horror' },
{ id: 3, name: 'Comedy' }
],
error: [Error [ValidationError]: "value" must be of type object] {
_original: [ [Object], [Object], [Object] ],
details: [ [Object] ]
}
}
If you want to validate an array containing objects, you could use
const schema = Joi.array().items(Joi.object().keys({
name: Joi.string().min(3).required()
}))

Separate swagger into files

Hi guys i'm works with swagger ui v2, and has a problem with organization.
The current structure of my project is like this.
const swaggerDocument = {
swagger: "2.0",
info: {
version: "1.0.0",
title: "Hello",
description: "hello",
},
basePath: "/api",
tags: [
{
name: "Users",
description: "API for users in the system",
},
],
schemes: [
"https",
"http",
],
consumes: [
"application/json",
],
produces: [
"application/json",
],
paths: {
"/users": {
// ...
},
},
definitions: {
User: {
// ...
},
},
};
In my path there are several routes (many actually) that reference my definitions, example: " $ref: "#/definitions/Users"," and my swagger.ts file is gigantic, I would like a better solution to organize, I tried something below but it didn't work.
swagger.ts
import { user, userDefinitions } from './user';
const swaggerDocument = {
paths: {
"/users": {
...user,
},
},
definitions: {
User: {
...userDefinitions,
},
},
};
Can someone help me?
User.ts
const user = {
"/users": {
...
},
};
const userDefinitions = {
User: {
properties: {
...
},
},
};
export { user, userDefinitions };

Generate friendly JSON from XML in NodeJS

Currently, I'm receiving an API request that has the crazy structure to the data, I'm attempting to parse the XML part of the response to an array/JSON so I can handle it.
Here's the exact request I am receiving:
{
username: 'test',
apiaccesskey: 'aa7a8157-3c17-4b63-806f-7aeff42ae21f',
action: 'placeimeiorder',
requestformat: 'JSON',
parameters:
'<PARAMETERS><CUSTOMFIELD>bnVsbA==</CUSTOMFIELD><ID>1</ID><SERVICEID>1</SERVICEID><IMEI>12345678910</IMEI><QNT>1</QNT><SERVER>0</SERVER><MODELID></MODELID><PROVIDERID></PROVIDERID><NETWORK></NETWORK><PIN></PIN><KBH></KBH><MEP></MEP><PRD></PRD><TYPE></TYPE><LOCKS></LOCKS><REFERENCE></REFERENCE><SN></SN><SECRO></SECRO></PARAMETERS>\n',
version: '7.2'
}
I've tried parsing using this library (xml2js) but it's generating a result like this:
let parseresult = await parser.parseStringPromise(req.body.parameters);
console.log(parseresult);
{ PARAMETERS:
{ CUSTOMFIELD: [ 'bnVsbA==' ],
ID: [ '1' ],
SERVICEID: [ '1' ],
IMEI: [ '12345678910' ],
QNT: [ '1' ],
SERVER: [ '0' ],
MODELID: [ '' ],
PROVIDERID: [ '' ],
NETWORK: [ '' ],
PIN: [ '' ],
KBH: [ '' ],
MEP: [ '' ],
PRD: [ '' ],
TYPE: [ '' ],
LOCKS: [ '' ],
REFERENCE: [ '' ],
SN: [ '' ],
SECRO: [ '' ] } }
which is far from ideal when trying to handle, how could I change it so I could simply access individual key/values like parseresult.IMEI or parseresult.CUSTOMFIELD
Should just be a setting.
Code:
const xml2js = require('xml2js');
const parser = new xml2js.Parser({ explicitArray: false });
const xml = "<PARAMETERS><CUSTOMFIELD>bnVsbA==</CUSTOMFIELD><ID>1</ID><SERVICEID>1</SERVICEID><IMEI>12345678910</IMEI><QNT>1</QNT><SERVER>0</SERVER><MODELID></MODELID><PROVIDERID></PROVIDERID><NETWORK></NETWORK><PIN></PIN><KBH></KBH><MEP></MEP><PRD></PRD><TYPE></TYPE><LOCKS></LOCKS><REFERENCE></REFERENCE><SN></SN><SECRO></SECRO></PARAMETERS>\n";
parser.parseString(xml, (err, result) => {
console.dir(result);
});
Reference: https://github.com/Leonidas-from-XIV/node-xml2js#options
Output:
{
PARAMETERS: {
CUSTOMFIELD: 'bnVsbA==',
ID: '1',
SERVICEID: '1',
IMEI: '12345678910',
QNT: '1',
SERVER: '0',
MODELID: '',
PROVIDERID: '',
NETWORK: '',
PIN: '',
KBH: '',
MEP: '',
PRD: '',
TYPE: '',
LOCKS: '',
REFERENCE: '',
SN: '',
SECRO: ''
}
}
Alternative:
Using the async/await like you have above:
const xml2js = require('xml2js');
(async () => {
const parser = new xml2js.Parser({ explicitArray: false });
const xml = "<PARAMETERS><CUSTOMFIELD>bnVsbA==</CUSTOMFIELD><ID>1</ID><SERVICEID>1</SERVICEID><IMEI>12345678910</IMEI><QNT>1</QNT><SERVER>0</SERVER><MODELID></MODELID><PROVIDERID></PROVIDERID><NETWORK></NETWORK><PIN></PIN><KBH></KBH><MEP></MEP><PRD></PRD><TYPE></TYPE><LOCKS></LOCKS><REFERENCE></REFERENCE><SN></SN><SECRO></SECRO></PARAMETERS>\n";
try {
console.log(await parser.parseStringPromise(xml))
} catch (error) {
console.log('ERROR', error);
}
})();
If you want complete control over the conversion, try saxon-js (released on Node.js a couple of weeks ago, available from npm).
For example you could do it in XPath 3.1 like this:
const SaxonJS = require("saxon-js");
SaxonJS.getResource({text:xml, type:"xml"})
.then(doc => {
SaxonJS.serialize(
SaxonJS.XPath.evaluate(
"map:merge(/PARAMETERS/* ! map{name(): string()})", doc),
{method:"json", indent:true}))});
You could extend this, for example to select which fields of the XML to include in the result.
Or for even more flexibility, you could do it in XSLT3.
(Not tested.)
you can also use camaro, a xpath-powered template to transform the value. it looks like this
const { transform } = require('camaro')
const data = {
username: 'test',
apiaccesskey: 'aa7a8157-3c17-4b63-806f-7aeff42ae21f',
action: 'placeimeiorder',
requestformat: 'JSON',
parameters:
'<PARAMETERS><CUSTOMFIELD>bnVsbA==</CUSTOMFIELD><ID>1</ID><SERVICEID>1</SERVICEID><IMEI>12345678910</IMEI><QNT>1</QNT><SERVER>0</SERVER><MODELID></MODELID><PROVIDERID></PROVIDERID><NETWORK></NETWORK><PIN></PIN><KBH></KBH><MEP></MEP><PRD></PRD><TYPE></TYPE><LOCKS></LOCKS><REFERENCE></REFERENCE><SN></SN><SECRO></SECRO></PARAMETERS>\n',
version: '7.2',
}
async function main() {
console.log(await transform(data.parameters, {
customerField: 'PARAMETERS/CUSTOMFIELD',
customerId: 'PARAMETERS/ID',
serviceId: 'PARAMETERS/SERVICEID',
}));
}
main()
output
{ customerField: 'bnVsbA==', customerId: '1', serviceId: '1' }
If you want more fields, you can just edit the template

TagSpecifications with requestSpotInstances UnexpectedParameter with aws-sdk

I'm trying to add a tag to my AWS Spot Request. But it has returned me { UnexpectedParameter: Unexpected key 'TagSpecifications' found in params.LaunchSpecification.
I have followed this documentation, and I have already tried to move this code out of LaunchSpecification, but the error persists.
const params = {
InstanceCount: 1,
LaunchSpecification: {
ImageId: config.aws.instanceAMI,
KeyName: 'backoffice',
InstanceType: config.aws.instanceType,
SecurityGroupIds: [config.aws.instanceSecurityGroupId],
TagSpecifications: [{
ResourceType: 'instance',
Tags: [{
Key: 'Type',
Value: 'Mongo-Dump',
}],
}],
BlockDeviceMappings: [{
DeviceName: '/dev/xvda',
Ebs: {
DeleteOnTermination: true,
SnapshotId: 'snap-06e838ce2a80337a4',
VolumeSize: 50,
VolumeType: 'gp2',
Encrypted: false,
},
}],
IamInstanceProfile: {
Name: config.aws.instanceProfileIAMName,
},
Placement: {
AvailabilityZone: `${config.aws.region}a`,
},
},
SpotPrice: config.aws.instancePrice,
Type: 'one-time',
};
return ec2.requestSpotInstances(params).promise();
Something makes me think that the problem is in the documentation or in the aws-sdk for Javascript itself. My options are exhausted.
The error message is correct. According to the documentation, the RequestSpotLaunchSpecification object doesn't have an attribute called TagSpecifications.
However, you can tag your Spot Instance request after you create it.
ec2.requestSpotInstances(params) returns an array of SpotInstanceRequest objects, each containing a spotInstanceRequestId (e.g. sir-012345678). Use the CreateTags API with these Spot Instance request ids to add the tags.
const createTagParams = {
Resources: [ 'sir-12345678' ],
Tags: [
{
Key: 'Type',
Value: 'Mongo-Dump'
}
]
};
ec2.createTags(createTagParams, function(err, data) {
// ...
});

I want to remove my array list

my array list is below:
this.array_of_name = ko.observableArray([
{ name: 'All Ways' },
{ name: 'Brand Cars' },
{ name: 'Carrom' },
{ name: 'Ginger' },
{ name: 'Honey' },
{ name: 'Jar Jar' },
{ name: 'Bert' },
{ name: 'Kitjar' },
{ name: 'Denise' },
{ name: 'Numeric' },
{ name: 'Length' },
{ name: 'Orange' },
{ name: 'Panasonic' },
{ name: 'Rabbit' },
{ name: 'Tarzan' },
{ name: 'USA' },
{ name: 'Yield' },
{ name: 'Zen' }
]);
i want to remove all items from it using javascript or knockout.
And also want to add search functionality using javascript or knockout.
var newArray = [];
var a=["a","b","c"];
for(var i=0;i<a.length;i++)
if(a[i]!=="a") newArray.push(a[i]);
Or another approach
removeAll = function(ary, elem) {
return ary.filter(function(e) { return e != elem });
}
to make empty, just assign empty array to your variable.
ex: this.array_of_name = ko.observableArray([]);

Resources