How to convert JSON schema to mongoose schema - node.js

Is there a way to take valid JSON schema like the one below and turn it into mongoose schema?
{
"$schema": "http://json-schema.org/draft-04/schema#",
"description": "some desc",
"title": "Product",
"type": "object",
"properties": {
"endpoints": {
"type": "array",
"items": {
"type": "string"
}
},
"poi": {
"type": "array",
"items": {
"type": "object",
"properties": {
"location_name": {
"type": "string"
},
"distance": {
"type": "string"
}
}
}
}
}
}
This seems so basic and simple to me but I haven't found anything on the net.
There are bunch of examples on how to get JSON schema and there are bunch of examples how to create mongoose schema from objects like this:
const newSchema = new mongoose.Schema({ name: String });
If I try to put JSON schema directly I get an error
node_modules/mongoose/lib/schema.js:674
throw new TypeError('Undefined type `' + name + '` at `' + path +
^
TypeError: Undefined type `Http://json-schema.org/draft-04/schema#` at `$schema`
Did you try nesting Schemas? You can only nest using refs or arrays.
But I could not find anywhere on the net transfer from one type to another.
Anyone had this issue before?
EDIT:
This question was conceptually incorrect.
Basically what you do is validate JSON schema against the data before saving it to DB. You do this using jsonschema from npm or some other.
So data validating step is not directly linked with saving to DB step.
I thought you can apply JSON schema to MongoDB schema but that was not true. (especially when you have deeply nested objects - then it's a mess)

I have been looking into this. Since you placed a node tag on your question, I found these npm repos:
https://github.com/jon49/json-schema-to-mongoose
https://github.com/topliceanu/mongoose-gen
Both are working so far.
They are both a few years old. The former (TypeScript) has more recent commits. I may end up liking the latter more.

Related

How to create schema model depend on have data element on mongoDB in NodeJs?

How to create dynamically schema element depend on data have in Nodejs for MongoDB ? Which syntax can use for create, add , update data object element like as the follow json format.
WorkTasks: [
{
PerformedBy: "Joe",
StartDate: 2021-07-19T17:43:06.693+00:00,
EndDate: 2021-07-19T17:43:06.693+00:00,
Remarks: "Pad error",
Failure: "DDPP 20 - 090"
},
{
PerformedBy: "Karen",
StartDate: 2021-07-19T17:43:06.693+00:00
}
]
From Node.js Best Practices:
Though validation can be coded or relied upon classes and types
(TypeScript, ES6 classes) the community seems to increasingly like
JSON-based schemas as these allow declaring complex rules without
coding and share the expectations with the frontend. JSON-schema is an
emerging standard that is supported by many npm libraries and tools
(e.g. jsonschema, Postman), joi is also highly popular with sweet
syntax. Typically JSON syntax can't cover all validation scenario and
custom code or pre-baked validation frameworks like validator.js come
in handy.
Example - JSON-Schema validation rules
{
"$schema": "http://json-schema.org/draft-06/schema#",
"title": "Product",
"description": "A product from Acme's catalog",
"type": "object",
"properties": {
"name": {
"description": "Name of the product",
"type": "string"
},
"price": {
"type": "number",
"exclusiveMinimum": 0
}
},
"required": ["id", "name", "price"]
}
Example - Validating an entity using JSON-Schema
const JSONValidator = require('jsonschema').Validator;
class Product {
validate() {
const v = new JSONValidator();
return v.validate(this, schema);
}
static get schema() {
//define JSON-Schema, see example above
}
}

Not able to populate all data by Meilisearch pluging using populateEntryRule in my strapi project

I am trying to integrate meilisearch functionality in my Strapi Project. While integrating and testing meilisearch I found that the plugin is failing to fetch all the data, especially when the data is located inside a nested array. The plugin successfully fetches the data from the first level of nested array, rest of the datas are not being fetched by meilisearch plugin. I have tried using the populateEntryRule on plugin.js file which I found from the meilisearch documentation but it's not giving the intended result. When I fire an API call to that particular collection type through Strapi(not through meilisearch) am getting all the data correctly.
Here is my particular schema that I am working with:
{
"kind": "collectionType",
"collectionName": "investors",
"info": {
"singularName": "investor",
"pluralName": "investors",
"displayName": "Investor",
"description": ""
},
"options": {
"draftAndPublish": true
},
"pluginOptions": {},
"attributes": {
"title": {
"type": "string"
},
"slug": {
"type": "uid",
"targetField": "title"
},
"metaData": {
"type": "component",
"repeatable": false,
"component": "seo.meta-fields"
},
"pageSections": {
"type": "dynamiczone",
"components": [
"sections.hero",
"sections.certification-section",
"sections.community-section",
"sections.content-section",
"sections.home-page-section-2",
"sections.lm-business-modal-section",
"sections.lm-evolving-section",
"sections.lm-leardership-section",
"sections.milestone-section",
"sections.mission-section",
"sections.our-business-section",
"sections.product-hero-section",
"sections.statistics-section",
"sections.team-section",
"sections.value-section",
"sections.vision-section",
"sections.webcast-section",
"sections.chart-section",
"sections.key-financials",
"sections.fixed-deposit",
"sections.financial-performance"
]
},
"Accordion": {
"type": "dynamiczone",
"components": [
"elements.no-dropdown",
"elements.dropdown"
]
}
}
}
My custom meilisearch plugin functionality for populating the required fields in plugin.js file.
module.exports = {
meilisearch: {
config: {
investor: {
populateEntryRule:['dynamiczone.pageSections', 'pageSections']
}
}
}
}
For better understanding am attaching a SS of Strapi API call to an investor's collection type naming Financial Performance.
SS of Meilisearch result for the same.
Any help in resolving this would be highly appreciable

How to use Pydantic base schema to n number of child schema

I'm new to pydantic, I want to define pydantic schema and fields for the below python dictionary which in the form of JSONAPI standard
{
"data": {
"type": "string",
"attributes":{
"title": "string",
"name": "string"
}
}
I managed to achieve this by defining multiple schemas like below,
class Child2(BaseModel):
title: str
name: str
class Child1(BaseModel):
type: str
attributes: Child2
class BaseParent(BaseModel):
data: Child1
But, I will be having multiple json request with the same json API structure as below,
example 1 {
"data": {
"type": "string",
"attributes":{
"source": "001",
"status": "New"
}
}
example 2 {
"data": {
"type": "string",
"attributes":{
"id": "001"
}
}
If you look into the above python dictionary, Values only under the attributes object are different. So, is there any way that I can define a parent marshmallow scheme for { "data": { "type": "string", "attributes":{ } } } and use that parent schema for all child schema's.
I found an answer finally, Hope this will help someone.
Pydantic 'create_model' concept will help to resolve this kind of requirement by passing child schema as one of the field values to create_model.
class Child(BaseModel):
title: str
name: str
class BaseParent(BaseModel):
data: create_model('BaseParent', type=(str, ...), attributes=(Child, ...))
And this will frame a BaseParent schema structure as below,
data=BaseParent(type='id', attributes=Child2(title='test002', name='Test'))

Apache Pulsar schema validate with json string

In my case, I have some raw JSON string data send to the topic and can't hard code POJO class, I want to use the pulsar schema feature to validate the structure. I have a topic "my-topic" and associated with JSON schema below, then I try to transmission some message.
var producer = client.newProducer(Schema.AUTO_PRODUCE_BYTES();
producer.send("{\"y\": 1}".getBytes()); // here! the value is 1(number) not string.
var reader = client.newReader(Schema.AUTO_CONSUME())
var message = reader.readNext();
I got {"y": 1}
my question is how pulsar schema works? The message should be rejected.
{
"version": 1,
"schemaInfo": {
"name": "my-topic",
"schema": {
"type": "record",
"name": "Data",
"namespace": "com.iot.test",
"fields": [
{
"name": "y",
"type": [
"null",
"string"
]
}
]
},
"type": "JSON",
"properties": {
"__alwaysAllowNull": "true"
}
}
}
my fault. just need to set
v2.5.0
bin/pulsar-admin namespaces set-is-allow-auto-update-schema --disable iot/test
v2.4.2
bin/pulsar-admin namespaces set-schema-autoupdate-strategy --disable iot/test
The Schema.AUTO_PRODUCE_BYTES setting is useful for transferring data from a producer to a Pulsar topic that has a schema because it ensures that the sent message is compatible with the topic's schema. However, I don't see where you specified the schema for the topic.
A topic is assigned a schema automatically when you connect a typed producer or consumer, e.g.
Producer producer = client.newProducer(JSONSchema.of(SensorReading.class))
.topic("sensor-data")
.sendTimeout(3, TimeUnit.SECONDS)
.create();
But you have stated that you cannot do this because you "can't hard code POJO". Therefore your only other option to assign a schema to the topic (so it can enforce message schema compatibility) is to use the REST API calls for manual schema management.
Based on your schema, your schema-definition file would look something like the following:
{
"type": "JSON",
"schema": "{\"type\":\"record\",\"name\":\"Data\",\"namespace\":\"com.iot.test\",\"fields\":[{\"name\":\"y\",\"type\":[\"null\",\"string\"],\"default\":null}}",
"properties": {}
}
HTH

Check JSON Schema which is syntactically correct, but not semantically

I want to check the JSON schema itself which is syntactically correct, but not semantically .
e.g
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "http://example.com/root.json",
"type": "object",
"required": [
"checked"
],
"properties": {
"checked": {
"$id": "#/properties/checked",
"type_invalid":"string"
}
}
}
In the above example has type_invalid key which is incorrect it should be type. Is there any way to validate JSON schema itself?
for reference:I am using ajv to validate JSON against JSON schema.
You could copy the existing meta-schema, and modify it to not allow additional properties, and then use that to validate your schemas.

Resources