How to create schema model depend on have data element on mongoDB in NodeJs? - node.js

How to create dynamically schema element depend on data have in Nodejs for MongoDB ? Which syntax can use for create, add , update data object element like as the follow json format.
WorkTasks: [
{
PerformedBy: "Joe",
StartDate: 2021-07-19T17:43:06.693+00:00,
EndDate: 2021-07-19T17:43:06.693+00:00,
Remarks: "Pad error",
Failure: "DDPP 20 - 090"
},
{
PerformedBy: "Karen",
StartDate: 2021-07-19T17:43:06.693+00:00
}
]

From Node.js Best Practices:
Though validation can be coded or relied upon classes and types
(TypeScript, ES6 classes) the community seems to increasingly like
JSON-based schemas as these allow declaring complex rules without
coding and share the expectations with the frontend. JSON-schema is an
emerging standard that is supported by many npm libraries and tools
(e.g. jsonschema, Postman), joi is also highly popular with sweet
syntax. Typically JSON syntax can't cover all validation scenario and
custom code or pre-baked validation frameworks like validator.js come
in handy.
Example - JSON-Schema validation rules
{
"$schema": "http://json-schema.org/draft-06/schema#",
"title": "Product",
"description": "A product from Acme's catalog",
"type": "object",
"properties": {
"name": {
"description": "Name of the product",
"type": "string"
},
"price": {
"type": "number",
"exclusiveMinimum": 0
}
},
"required": ["id", "name", "price"]
}
Example - Validating an entity using JSON-Schema
const JSONValidator = require('jsonschema').Validator;
class Product {
validate() {
const v = new JSONValidator();
return v.validate(this, schema);
}
static get schema() {
//define JSON-Schema, see example above
}
}

Related

Not able to populate all data by Meilisearch pluging using populateEntryRule in my strapi project

I am trying to integrate meilisearch functionality in my Strapi Project. While integrating and testing meilisearch I found that the plugin is failing to fetch all the data, especially when the data is located inside a nested array. The plugin successfully fetches the data from the first level of nested array, rest of the datas are not being fetched by meilisearch plugin. I have tried using the populateEntryRule on plugin.js file which I found from the meilisearch documentation but it's not giving the intended result. When I fire an API call to that particular collection type through Strapi(not through meilisearch) am getting all the data correctly.
Here is my particular schema that I am working with:
{
"kind": "collectionType",
"collectionName": "investors",
"info": {
"singularName": "investor",
"pluralName": "investors",
"displayName": "Investor",
"description": ""
},
"options": {
"draftAndPublish": true
},
"pluginOptions": {},
"attributes": {
"title": {
"type": "string"
},
"slug": {
"type": "uid",
"targetField": "title"
},
"metaData": {
"type": "component",
"repeatable": false,
"component": "seo.meta-fields"
},
"pageSections": {
"type": "dynamiczone",
"components": [
"sections.hero",
"sections.certification-section",
"sections.community-section",
"sections.content-section",
"sections.home-page-section-2",
"sections.lm-business-modal-section",
"sections.lm-evolving-section",
"sections.lm-leardership-section",
"sections.milestone-section",
"sections.mission-section",
"sections.our-business-section",
"sections.product-hero-section",
"sections.statistics-section",
"sections.team-section",
"sections.value-section",
"sections.vision-section",
"sections.webcast-section",
"sections.chart-section",
"sections.key-financials",
"sections.fixed-deposit",
"sections.financial-performance"
]
},
"Accordion": {
"type": "dynamiczone",
"components": [
"elements.no-dropdown",
"elements.dropdown"
]
}
}
}
My custom meilisearch plugin functionality for populating the required fields in plugin.js file.
module.exports = {
meilisearch: {
config: {
investor: {
populateEntryRule:['dynamiczone.pageSections', 'pageSections']
}
}
}
}
For better understanding am attaching a SS of Strapi API call to an investor's collection type naming Financial Performance.
SS of Meilisearch result for the same.
Any help in resolving this would be highly appreciable

Apache Pulsar schema validate with json string

In my case, I have some raw JSON string data send to the topic and can't hard code POJO class, I want to use the pulsar schema feature to validate the structure. I have a topic "my-topic" and associated with JSON schema below, then I try to transmission some message.
var producer = client.newProducer(Schema.AUTO_PRODUCE_BYTES();
producer.send("{\"y\": 1}".getBytes()); // here! the value is 1(number) not string.
var reader = client.newReader(Schema.AUTO_CONSUME())
var message = reader.readNext();
I got {"y": 1}
my question is how pulsar schema works? The message should be rejected.
{
"version": 1,
"schemaInfo": {
"name": "my-topic",
"schema": {
"type": "record",
"name": "Data",
"namespace": "com.iot.test",
"fields": [
{
"name": "y",
"type": [
"null",
"string"
]
}
]
},
"type": "JSON",
"properties": {
"__alwaysAllowNull": "true"
}
}
}
my fault. just need to set
v2.5.0
bin/pulsar-admin namespaces set-is-allow-auto-update-schema --disable iot/test
v2.4.2
bin/pulsar-admin namespaces set-schema-autoupdate-strategy --disable iot/test
The Schema.AUTO_PRODUCE_BYTES setting is useful for transferring data from a producer to a Pulsar topic that has a schema because it ensures that the sent message is compatible with the topic's schema. However, I don't see where you specified the schema for the topic.
A topic is assigned a schema automatically when you connect a typed producer or consumer, e.g.
Producer producer = client.newProducer(JSONSchema.of(SensorReading.class))
.topic("sensor-data")
.sendTimeout(3, TimeUnit.SECONDS)
.create();
But you have stated that you cannot do this because you "can't hard code POJO". Therefore your only other option to assign a schema to the topic (so it can enforce message schema compatibility) is to use the REST API calls for manual schema management.
Based on your schema, your schema-definition file would look something like the following:
{
"type": "JSON",
"schema": "{\"type\":\"record\",\"name\":\"Data\",\"namespace\":\"com.iot.test\",\"fields\":[{\"name\":\"y\",\"type\":[\"null\",\"string\"],\"default\":null}}",
"properties": {}
}
HTH

Check JSON Schema which is syntactically correct, but not semantically

I want to check the JSON schema itself which is syntactically correct, but not semantically .
e.g
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "http://example.com/root.json",
"type": "object",
"required": [
"checked"
],
"properties": {
"checked": {
"$id": "#/properties/checked",
"type_invalid":"string"
}
}
}
In the above example has type_invalid key which is incorrect it should be type. Is there any way to validate JSON schema itself?
for reference:I am using ajv to validate JSON against JSON schema.
You could copy the existing meta-schema, and modify it to not allow additional properties, and then use that to validate your schemas.

Is it possible to add additional user defined properties other than standard properties in "keen" object?

As for every keen event, keen object gets automatically attached with structure of keen object.
var keen_event = {
"keen": {
"created_at": "2012-12-14T20:24:01.123000+00:00",
"timestamp": "2012-12-14T20:24:01.123000+00:00",
"id": "asd9fadifjaqw9asdfasdf939"
},
other properties...
}
But I want to add other custom properties in keen object as;
{
"keen": {
"created_at": "2012-12-14T20:24:01.123000+00:00",
"timestamp": "2012-12-14T20:24:01.123000+00:00",
"id": "asd9fadifjaqw9asdfasdf939",
"event_type" : "some_values",
...
}
}
I tried adding other properties on keen and than encode the keen_event as:
var encodedData = base64.encode(JSON.stringify(keen_event));
The encoded data is then passed on keen api to create an event but I am getting error as : "Invalid property for keen namespace. You used: \'event_type\'".
Is there a solution for that? Is it possible to add custom key value properties other than standard properties on " keen" object while creating an keen event? Thanks in advance.
No, this isn't allowed in the Keen API. The keen namespace is reserved and only a small handful of properties (such as keen.timestamp or keen.addons) may be specified in that namespace when an event is written. The namespace is reserved to allow Keen to add new special-purpose properties in the future and not worry about collisions with user-defined properties. (See also: https://keen.io/docs/api/#the-keen-object.)
Perhaps it's worth stepping back and asking: why do you want to add custom properties to the keen namespace? What would it allow you to do that you couldn't do with, say, my.keen.x or keen_custom.x?
FYI: I'm a platform engineer at Keen :)
var data = {
"keen": {
"created_at": "2012-12-14T20:24:01.123000+00:00",
"timestamp": "2012-12-14T20:24:01.123000+00:00",
"id": "asd9fadifjaqw9asdfasdf939",
},
"keen1": {
"created_at": "2012-12-14T20:24:01.123000+00:00",
"timestamp": "2012-12-14T20:24:01.123000+00:00",
"id": "asd9fadifjaqw9asdfasdf939",
},
"keen2": {
"created_at": "2012-12-14T20:24:01.123000+00:00",
"timestamp": "2012-12-14T20:24:01.123000+00:00",
"id": "asd9fadifjaqw9asdfasdf939",
}
}
var newObj = {}
Object.keys(data).map((element)=> {
var t = {
...data[element],
"event_type" : "some_values",
}
newObj[element] = t
return element
})
console.log(newObj)
<script src="https://cdnjs.cloudflare.com/ajax/libs/react/0.14.0/react.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/react/0.14.0/react-dom.min.js"></script>
This may help
Try something like this
var keenEvent = {
"keen": {
"created_at": "2012-12-14T20:24:01.123000+00:00",
"timestamp": "2012-12-14T20:24:01.123000+00:00",
"id": "asd9fadifjaqw9asdfasdf939"
}
};
keenEvent.keen.myNewProperty = "its value";
keenEvent.keen.myNewProperty2 = "its value2";

How to convert JSON schema to mongoose schema

Is there a way to take valid JSON schema like the one below and turn it into mongoose schema?
{
"$schema": "http://json-schema.org/draft-04/schema#",
"description": "some desc",
"title": "Product",
"type": "object",
"properties": {
"endpoints": {
"type": "array",
"items": {
"type": "string"
}
},
"poi": {
"type": "array",
"items": {
"type": "object",
"properties": {
"location_name": {
"type": "string"
},
"distance": {
"type": "string"
}
}
}
}
}
}
This seems so basic and simple to me but I haven't found anything on the net.
There are bunch of examples on how to get JSON schema and there are bunch of examples how to create mongoose schema from objects like this:
const newSchema = new mongoose.Schema({ name: String });
If I try to put JSON schema directly I get an error
node_modules/mongoose/lib/schema.js:674
throw new TypeError('Undefined type `' + name + '` at `' + path +
^
TypeError: Undefined type `Http://json-schema.org/draft-04/schema#` at `$schema`
Did you try nesting Schemas? You can only nest using refs or arrays.
But I could not find anywhere on the net transfer from one type to another.
Anyone had this issue before?
EDIT:
This question was conceptually incorrect.
Basically what you do is validate JSON schema against the data before saving it to DB. You do this using jsonschema from npm or some other.
So data validating step is not directly linked with saving to DB step.
I thought you can apply JSON schema to MongoDB schema but that was not true. (especially when you have deeply nested objects - then it's a mess)
I have been looking into this. Since you placed a node tag on your question, I found these npm repos:
https://github.com/jon49/json-schema-to-mongoose
https://github.com/topliceanu/mongoose-gen
Both are working so far.
They are both a few years old. The former (TypeScript) has more recent commits. I may end up liking the latter more.

Resources