Client is Angular 8 with NgRx-Data; Server is NestJs + MySQL. How to get data using criteria and SQL? - nestjs

As Angular, NgRx-Data and NestJs are becomming more and more popular, I feel there may be quite a few programmers who are wondering about the querying syntax for the following.
I have a running prototype of a client (front end) composed in Angular 8 with NgRx-Data.
On the back end is a NestJs based server + MySQL.
I can nicely retrieve and pass data between all parts, except queries. I do not seem to be able to find proper documentation on the syntax.
Here is the example of how the client is set:
// Simple entity example (all ngrx-data metadata are declared and set):
export class Hero {
id: number;
name?: string;
age?: number;
}
Entity Service / for fetching data
#Injectable({providedIn: 'root'})
export class HeroService extends EntityCollectionServiceBase<Hero> {
constructor(serviceElementsFactory: EntityCollectionServiceElementsFactory) {
super('Hero', serviceElementsFactory);
}
}
Component for showing data
#Component({
selector: 'hero-comp',
templateUrl: './hero.component.html',
styleUrls: ['./hero.component.scss']
})
export class HeroComponent {
heroData$: Observable<Hero[]>;
constructor(private heroDatService: HeroService) {
this.heroData$ = this.heroDatService.entities$;
}
private getAllData() {
// This works nicely, I get all records from the db via server
this.heroDatService.getAll();
}
private queryData() {
// This queryParams syntax fails - server complains with this error:
// [HttpExceptionFilter] GET /hero/?age>20
// QueryFailedError: ER_EMPTY_QUERY: Query was empty
// QUESTION: What is the proper syntax?
let queryParams: QueryParams = {
'age > 20'
}
this.fetchDataService.getWithQuery(queryParams);
}
Here is the server related code excerpt: -
(there is a service, but for simplicity here, I moved the repo functions to the controller functions):
#Controller('hero')
export class HeroController <Hero> {
constructor(readonly repo: Repository<Hero>) {}
// This returns nicely all Hero records from the MySQL db server
#Get()
async getAll(): Promise<Hero[]> {
return await this.repo.find();
}
// This does not work !
// I am not sure about any piece of the code here !
#Get('query')
async query(#Query() sql): Promise<any> {
// Does the sql argument need to be manipulated into parameters: {...} ?
// If yes - how ?
let parameters: undefined;
return await this.repo.query(sql, parameters);
}
Please see the comments above each code line - the problems are spelled out there.
And here are the important questions:
On the client how do we properly pass query criteria for some of these examples:
- {'age > 20'}
- {'age BETWEEN 20 AND 40'}
- {'age = 20 OR age = 30 OR age = 40'}
- {'name = "Superman"'}
- {'name LIKE "Super%"'}
- etc.
Also, what would be the syntax for passing a full SQL sentence, such as:
- {'SELECT * FROM Heroes WHERE name LIKE "Super%" AND Age > 20;'}
and getting the result from the server.
What needs to be done on both ends (client and server) for these queries to work?
All inputs much appreciated.

It seems like you're confused about HTTP Query parameters and SQL querying, which are two different topics. Query parameters, in the context of HTTP, are parameters that can be passed from the client to the server and modify the outcome of the HTTP call. Query parameters are always passed starting with a ? in the URL in the form of <key>=<value> and separated with an &.
A SQL Query is a specific string that tells a SQL server what table to query against, for what columns, with what conditions. Usually in the form of SELECT <columns> FROM <table> WHERE <conditions>;, but they can be much more complex than that.
Now that definitions are out of the way, the endpoint you are trying to reach should be /hero/query. You'll need to end up doing a lot of data processing on the server side of things, sanitization, ensuring that the incoming string is proper for SQL WHERE clauses, ensuring that you won't be vulnerable to SQL Injections (if you pass the query params straight to your query, you will be), but a very very naive approach would look something like this:
#Controller('hero')
export class HeroController {
constructor(#InjectRepository(Hero) private readonly repo: Repository<Hero>) {}
#Get('query')
queryForHero(#Query() queryParam) {
return this.repo.query(`SELECT <field_go_here> FROM Hero WHERE ${queryParams.query};`);
}
}
For the love of all that is good in the world, do not actually use the above code. It is 100% vulnerable to all kinds of SQL injections.
A corresponding request could look something like
curl http://<host>/hero/query?query=Name%3D%27Superman%27
This would cause the server to use the query
SELECT <fields_go_here> FROM Hero WHERE Name='Superman';
You'll really want to add in a ton of validations on what is coming into your server before just sending it to your SQL server lest you end up like Little Bobby Table.
Hopefully this helps get you on the right path.

Related

How to send params of current request to the constructor of the service?

Update: I have fixed it by only supplying the pattern of the path of the collection, and creating a function that can parse the provided IDs and now the functions themselves create the collections when they are called upon and it also works with Typescript:)
Updated in the repository:
https://github.com/Darkbound/nestjs-firebase/tree/main/src/firebase
In the user service:
https://github.com/Darkbound/nestjs-firebase/blob/main/src/user/user.service.ts
In the purchase transactions service: https://github.com/Darkbound/nestjs-firebase/blob/main/src/user/modules/purchase-transaction/purchase-transaction.service.ts
In the purchase transactions controller: https://github.com/Darkbound/nestjs-firebase/blob/main/src/user/modules/purchase-transaction/purchase-transaction.controller.ts#L14
Now the functionality works out of the box, the service class simply needs to extend the FirebaseCollectionService and give it the pattern of the path to the collection and thats it!
https://github.com/Darkbound/nestjs-firebase I have uploaded it into a repository, you only need to add .env with the keys for firebase admin.
And the specific example: https://github.com/Darkbound/nestjs-firebase/blob/main/src/user/modules/purchase-transaction/purchase-transaction.service.ts
I have created a class that gives me the functionality to perform CRUD operations on firebase, so that I can just directly inherit from it for any of my CRUD resources, as the logic is again usually mostly the same. Just like Nestjs generator gives me all of the routes for it.
#Injectable()
export class UserService extends NestjsFirebase<User> {
constructor(#InjectFirebaseAdmin() firebase: FirebaseAdmin) {
super(firebase, "users");
// console.log(userId);
}
}
This works great, I can reuse that for any level 1 collection I have in firebase, however if I want to get into a nested collection on firebase, well thats a problem, because the path there needs to be dynamic and super(firebase, "this is no longer just users").
Say if I want to access the transactions of a user, so users/SomeUserIdXYZ/transactions, then the path is entirely dependent on the userId and is changing, therefor, I need to recreate the instance of the service (I simply need a new instance of the class), with a new path:
super(firebase, ["users", userId, "transactions"]
However with my still limited knowledge about Nestjs I know that everything in it basically is a Singleton and there is probably no way to do this? To get a new instance of the service, for every request that I have?
The solution that I can think of is, to handle that within my route functions, so if its a findTransactions:
#Get("users/:userId/transactions")
async findTransactions(#Param("userId") userId: string) {
return this.userService.findAll(`users/${userId}/transactions`);
}
And I am pretty sure that this will work, if I add a path argument to each of the functions, but this seems like coupling the Controller with what my Path in firebase should look like, instead I need to be able to give it just the params so that it can create its own path.
This is NestjsFirebase:
#Injectable()
class NestjsFirebase<T> {
constructor(#InjectFirebaseAdmin() private readonly firebase: FirebaseAdmin, private readonly collectionPath: string) {}
async findAll(userId: string): Promise<T> {
const db = new FirebaseCollectionService<T>(this.firebase, this.collectionPath);
return await db.findAll(userId);
}
}
export class FirebaseCollectionService<T> {
protected db: CollectionReference<T>;
constructor(firebase: FirebaseAdmin, collectionPath: string) {
super(firebase.db);
this.db = this.createCollectionPath(collectionPath);
}
public async findAll(id: string) {
... some logic to find all transactions ...
}
}

How to build a Graqhql mutation with existing variables

This might seem like an odd question, or something really straightforward, but honestly I am struggling to figure out how to do this. I am working in Node.js and I want to set data I have saved on a node object into my GraphQL mutation.
I'm working with a vendor's GraqhQL API, so this isn't something I have created myself, nor do I have a schema file for it. I'm building a mutation that will insert a record into their application, and I can write out everything manually and use a tool like Postman to manually create a new record...the structure of the mutation is not my problem.
What I'm struggling to figure out is how to build the mutation with variables from my node object without just catting a bunch of strings together.
For example, this is what I'm trying to avoid:
class MyClass {
constructor() {
this.username = "my_username"
this.title = "Some Title"
}
}
const obj = new MyClass()
let query = "mutation {
createEntry( input: {
author: { username: \"" + obj.username + "\" }
title: \"" + obj.title + "\"
})
}"
I've noticed that there are a number of different node packages out there for working with Graphql, but none of their documentation that I've seen really addresses the above situation. I've been completely unsuccessful in my Googling attempts, can someone please point me in the right direction? Is there a package out there that's useful for just building queries without requiring a schema or trying to send them at the same time?
GraphQL services typically implement this spec when using HTTP as a transport. That means you can construct a POST request with four parameters:
query - A Document containing GraphQL Operations and Fragments to execute.
operationName - (Optional): The name of the Operation in the Document to execute.
variables - (Optional): Values for any Variables defined by the Operation.
extensions - (Optional): This entry is reserved for implementors to extend the protocol however they see fit.
You can use a Node-friendly version of fetch like cross-fetch, axios, request or any other library of your choice to make the actual HTTP request.
If you have dynamic values you want to substitute inside the query, you should utilize variables to do so. Variables are defined as part of your operation definition at the top of the document:
const query = `
mutation ($input: SomeInputObjectType!) {
createEntry(input: $input) {
# whatever other fields assuming the createEntry
# returns an object and not a scalar
}
}
`
Note that the type you use will depend on the type specified by the input argument -- replace SomeInputObjectType with the appropriate type name. If the vendor did not provide adequate documentation for their service, you should at least have access to a GraphiQL or GraphQL Playground instance where you can look up the argument's type. Otherwise, you can use any generic GraphQL client like Altair and view the schema that way.
Once you've constructed your query, make the request like this:
const variables = {
input: {
title: obj.title,
...
}
}
const response = await fetch(YOUR_GRAPHQL_ENDPOINT, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ query, variables }),
})
const { data, errors } = await response.json()

Use Firestore collection get() return type in a function

I'm currently trying to write some code that retrieves a collection from my Firestore instance.
My codebase uses the service repository pattern to keep business logic seperate from the code that retrieves data. For this reason I've made the following code:
import { injectable, inject } from "inversify";
import { IOfficeRepository, TYPES } from "../common/types";
import { Firestore } from "#google-cloud/firestore";
#injectable()
export default class OfficeRepository implements IOfficeRepository {
private fireStoreClient: Firestore;
constructor(#inject(TYPES.FireStoreFactory) firestoreFactory: () => Firestore) {
this.fireStoreClient = firestoreFactory();
};
public async getOffice(officeId: string): Promise<FirebaseFirestore.QueryDocumentSnapshot<FirebaseFirestore.DocumentData>> {
const officeCollection = "offices";
const document = await this.fireStoreClient.collection(officeCollection).get();
return document;
};
}
What I'd like to do is return the value from the get() call to my service, in the service I will be performing checks and executing the business logic that I need.
The get() returns a Promise<FirebaseFirestore.QuerySnapshot<FirebaseFirestore.DocumentData>>, but I am unable to use this as a return type for the function in my repository. I just get the following error:
Type 'QuerySnapshot' is missing the following properties from type 'QueryDocumentSnapshot': createTime, updateTime, data, exists, and 3 more.
I've already looked-up the error, but I wasn't able to find any solution or a post where someone was trying to return the result from the get() function before performing any logic on the result.
So my question is: How would I be able to make this setup work? Or is there something I am doing wrong with this setup? If so, what would be another approach to work this out while using the service repository pattern?
Your declared return type of QueryDocumentSnapshot doesn't match the actual return type of QuerySnapshot.
This line of code:
const document = await this.fireStoreClient.collection(officeCollection).get();
performs a query for all of the documents in the officeCollection collection. As you can see from the API documentation, CollectionReference.get() yields a QuerySnapshot object. The entire set of documents will be available in the returned docs property.
It seems that you expect getOffice to return a single document instead. I'm noticing that you never used the argument officeId to narrow down your query to just the one document you want. Perhaps you meant to do something like this instead to get a single document using its ID?
const document = await this.fireStoreClient
.collection(officeCollection)
.doc(officeId)
.get();
In this case, document will be a DocumentSnapshot object.

Automatically parse query parameter to object when defined in NestJS

I am writing a NestJS application. Some of the endpoints support sorting e.g. http://127.0.0.1:3000/api/v1/members?sort=-id&take=100 Which means sort by id descending.
This parameter arrives as a #Query parameter and is passed to my service. This service transforms it into an object which is used by TypeORM:
{
id: 'DESC'
}
I don't want to call this conversion method manually every time I need sorting.
I've tried an intercepter but this one could not easily change the request parameters into the desired object.
A pipe worked but then I still need to add #Query(new SortPipe()) for every endpoint definition.
Another option is in the repository itself. The NestJS documentation is very well written, but misses guidance in where to put what.
Is there someone who had a similar issue with converting Query parameters before they are used in NestJS, and can explain what approach is the best within NestJS?
This question might look like an opinion based question, however I am looking for the way it is supposed to be done with the NestJS philosophy in mind.
Pipes are probably the easiest way to accomplish this. Instead of adding your pipe for every endpoint definition you can add a global pipe that will be called on every endpoint. In your main.ts:
async function bootstrap() {
...
app.useGlobalPipes(new SortPipe());
...
}
You can then create a pipe like this:
import { PipeTransform, Injectable, ArgumentMetadata } from '#nestjs/common';
#Injectable()
export class SortPipe implements PipeTransform {
transform(value: any, metadata: ArgumentMetadata) {
const { type } = metadata;
// Make sure to only run your logic on queries
if (type === 'query') return this.transformQuery(value);
return value;
}
transformQuery(query: any) {
if (typeof query !== 'object' || !value) return query;
const { sort } = query;
if (sort) query.sort = convertForTypeOrm(sort);
return query;
}
}
If you do not want sort value on ALL endpoints to be automatically converted, you can pass custom parameter to #Query(), for example #Query('sort'). And then:
transform(value: any, metadata: ArgumentMetadata) {
const { type, data } = metadata;
// Make sure to only run your logic on queries when 'sort' is supplied
if (type === 'query' && data === 'sort') return this.transformQuery(value);
return value;
}

Breeze & EFContextProvider - How to properly return $type when using expand()?

I am using Breeze with much success in my SPA, but seem to be stuck when trying to return parent->child data in a single query by using expand().
When doing a single table query, the $type in the JSON return is correct:
$type: MySPA.Models.Challenge, MySPA
However if I use expand() in my query I get the relational data, but the $type is this:
System.Collections.Generic.Dictionary 2[[System.String, mscorlib],[System.Object, mscorlib]]
Because of the $type is not the proper table + namespace, the client side code can't tell that this is an entity and exposes it as JSON and not a Breeze object (with observables, entityAspect, etc.).
At first I was using my own ContextProvider so that I could override the Before/After saving methods. When I had these problems, I reverted back to the stock EFContextProvider<>.
I am using EF5 in a database first mode.
Here's my controller code:
[BreezeController]
public class DataController : ApiController
{
// readonly ModelProvider _contextProvider = new ModelProvider();
readonly EFContextProvider<TestEntities> _contextProvider = new EFContextProvider<TestEntities>();
[HttpGet]
public string Metadata()
{
return _contextProvider.Metadata();
}
[Queryable(AllowedQueryOptions = AllowedQueryOptions.All)]
[HttpGet]
public IQueryable<Challenge> Challenges()
{
return _contextProvider.Context.Challenges;
}
[HttpPost]
public SaveResult SaveChanges(JObject saveBundle)
{
return _contextProvider.SaveChanges(saveBundle);
}
public IQueryable<ChallengeNote> ChallengeNotes()
{
return _contextProvider.Context.ChallengeNotes;
}
}
Here's my BreezeWebApiConfig.cs
public static void RegisterBreezePreStart()
{
GlobalConfiguration.Configuration.Formatters.Remove(GlobalConfiguration.Configuration.Formatters.XmlFormatter);
GlobalConfiguration.Configuration.Routes.MapHttpRoute(
name: "BreezeApi",
routeTemplate: "breeze/{controller}/{action}"
);
}
Is there a configuration setting that I am missing?
Did you try "expanding" on server side? Is it needed to do expand on client side? I tried to do expand before but failed for me as well, did some research and decided I'd rather place it on server:
[HttpGet]
public IQueryable<Challenge> ChallengesWithNotes()
{
return _contextProvider.Context.Challenges.Include("ChallengeNotes");
}
This should be parsed as expected. On client side you would query for "ChallengeNotes" instead of "Challenges" and you wouldn't need to write expand part.
I strongly suspect that the problem is due to your use of the [Queryable] attribute.
You must use the [BreezeQueryable] attribute instead!
See the documentation on limiting queries.
We are aware that Web API's QueryableAttribute has been deprecated in favor of EnableQueryAttribute in Web API v.1.5. Please stick with BreezeQueryable until we've had a chance to write a corresponding derived attribute for EnableQuery. Check with the documentation for the status of this development.

Resources