In my NestJS app, I'm making a REST request to a remote API I do not have control over. The REST API has a response containing JSON, a large object, most of which I do not need. Let's assume hypothetically that we have a JSON object that looks like the following:
{
"foo": [
1,
2,
3
],
"bar": {
"nested": {
"some_key": "some_val"
}
}
}
What if in this case, after I make a request to this API, I want to pass around only a subset of the above. Something like a NestedDto that would look like the following:
import { IsNotEmpty, IsString } from 'class-validator'
export class NestedDto {
#IsNotEmpty()
#IsString()
someKey: string
}
What is the best way for me to take the data that's being returned from the REST API and transform it into the above-using tools that NestJS offers? I want to be able to take responses from remote APIs and pass said data around inside of my NestJS app using my interface specifications.
All right, if you're just using Axios with no special configuration, you could do what you are already doping in Express by just mapping the response down to what you want. If you want to get a little fancy with it you could always implement some sort of class decorated with class-transformer decorators and use a plainToClass method mixed with the #Transform() decorator.
By the way, by default, NestJS provides an HttpModule that is a wrapper around Axios, but its responses are as RxJS Observables. If you decide to go with the observable route, you can use the observable operator map to do the mapping for you (you'll still have to provide mapping implementation though, like the plainToClass I mentioned above), but that is completely up to you.
Related
Let's say we make the following GET API call to our REST API running with NestJS (and fastify):
http://localhost:5000/api/test?arrayParam=["abc","def"]&anotherParam="value"
Without parsing anything, the query object on the backend looks like this:
{
arrayParam: '["abc","def"]',
anotherParam: '"value"'
}
We can see that parameter values are strings, but in the case of arrayParam we would obviously like to work with the actual array.
I come from an expressJS background, and coming from there, there are a couple of approaches. First would be just using a JSON parser middleware, like body-parser. Or just using JSON.parse().
But what is the "proper", NestJS approach? I thought about using type decorators defined in a DTO, and assumed they would be automatically parsed to the type that I defined. But that doesn't work like I assumed it would.
I defined it like this:
#IsOptional()
#IsArray()
arrayParam?: string[];
But validation fails, since arrayParam is a string and not an array. So I assume this is not the correct approach
You are sending it incorrectly
http://localhost:5000/api/test?arrayParam[]=abc&arrayParam[]=def&anotherParam
I have a DTO class of a user in NestJS.
I am using many validations using class-validator package in order to enforce my logic.
If a field that doesn't exists on the DTO definition, I would like to ignore it and even throw an error.
This is why I was trying to use the 'excludeExtraneousValues' flag.
When I do use it, it ignores all the fields, even the ones that defined in the DTO.
import { ApiPropertyOptional } from '#nestjs/swagger';
import {
IsDefined,
IsEmail,
IsOptional,
IsPhoneNumber,
MaxLength,
ValidateIf,
} from 'class-validator';
export default class UserDTO {
#ApiPropertyOptional()
#MaxLength(254)
#IsEmail()
#IsDefined()
#ValidateIf((object) => object.email || !object.phone_number)
email?: string;
#ApiPropertyOptional()
#MaxLength(15)
#IsPhoneNumber()
#IsDefined()
#ValidateIf((object) => object.phone_number || !object.email)
phone_number?: string;
#ApiPropertyOptional()
#IsOptional()
#MaxLength(40)
name?: string;
}
As I mentioned, I am using NestJS.
This is the ValidationPipe definition:
app.useGlobalPipes(
new ValidationPipe({
transform: true,
stopAtFirstError: true,
transformOptions: { excludeExtraneousValues: true },
}),
);
Following the addition of 'excludeExtraneousValues' flag, I cannot send any value, even the ones that is defined.
Is it a bug or am I missing something?
A bit old but I stumbled across and can see it was upvoted, so here goes:
I am wondering if you are mixed up on input vs. output. Your question isn't quite specific enough for me to be 100% sure. NestJS apps often make use of the class-validator and class-transformer libraries on both ends of the equation and can work on DTO's/entities that are dressed up with decorators from these libraries.
Pipes such as your ValidationPipe are geared to the input side of things. Refer to the docs: https://docs.nestjs.com/techniques/validation.
Generally class-validator is the key player on the input side, with class-transformer only playing a role with applicable decorators like #Transform(). To make use of class-transformer, you need to pass the transform: true option as you do.
If you want to control behaviours regarding data fields coming into your ValidationPipe that are not defined in your entity/DTO classes, take a look at the whitelist: boolean, forbidNonWhitelisted: boolean, and forbidUnknownValues: true configuration options to satisfy your needs on the input side of things.
I have a feeling that once you check these options out, you will find that you will want to delete the transformOptions: { excludeExtraneousValues: true } option.
On the output side of things is where interceptors come into play. Often in NestJS projects the included ClassSerializerInterceptor is employed to serialize instances of DTO/entity classes to JSON for a response. Docs: https://docs.nestjs.com/techniques/serialization
This side is where you see more frequent use of class-transformer's decorators such as Exclude(), Expose(), and Transform().
This is also where you are probably more likely to find a use-case for a configuration option like excludeExtraneousValues.
Suppose you had data in an object (e.g. from a database, file, or wherever) and wanted to populate a class (such as a DTO) so you could take advantage of these decorators, but only wanted to have certain fields actually sent back in the response, i.e. the ones that you explicitly specified and decorated in your class, and you didn't want to bother with Exclude() and Expose() decorators all over the place.
In such a case, the options for class-transformer may come in handy depending on what you want to do, e.g.
app.useGlobalInterceptors(new ClassSerializerInterceptor(app.get(Reflector), { excludeExtraneousValues: true }))
I have several modules that export a function that return true or false when the payload is valid or not. I also have a config file in JSON in which I specify the name of the validator script to use depending on the payload type:
[
{
"boardVersion": "1",
"availableInterfaces": [
{ "name": "digital", "validator": "digitalV1" },
{ "name": "analog", "validator": "analogV1" },
]
}
]
And for example inside digitalv1.js I have something like:
import validator from 'validator';
module.exports = (value) => {
validator.isInt(value, {min: 0, max: 3});
};
And finally, a controller that gets the payload from ExpressJS and depending on the endpoint called it decides what validator to use. The thing now is how can I load the validator in the controller.
There are 2 approaches that come to my mind:
In the controller, I require every validator and push them in a key-value array (or object), in which the key is the name of the validator and the value the validator itself.
Instead of defining a validator name in the JSON file I could just put the file path in the file system and just require the file when I need it.
I there a better/cleaner way to approach this? Feel free to suggest even a different architecture. The idea though is to keep validators separated for code cleanness sake.
You can use option 1, but with a "barrel module", which imports the validators, and attaches them onto an array or object. Then, any module that needs the validators can just import the main module, and use the provided lookup function, index, or key to get the validator they need.
Be careful of circular dependencies: if you have them, you can use module.exports.x = and Node will handle them correctly (Note: I'm not sure if this is necessary with ES6 modules, which handle circular dependencies a little differently; just something to be on guard for).
Since the knowledge to create this "barrel module" already exists in your JSON, you could watch your JSON file in your build process, and automatically generate this barrel module if any of the file associations change, throw an error if a specified module is not found at a certain file path, etc.
In our framework we have an interface with this method in the public API:
JaxbConfiguration newJaxbConfiguration(Options xjcOpts);
In the implementation, we do something like this:
import com.sun.tools.xjc.ModelLoader;
import com.sun.tools.xjc.Options;
import com.sun.tools.xjc.model.Model;
...
public JaxbConfiguration newJaxbConfiguration(Options xjcOpts) {
Model model = ModelLoader.load(xjcOpts, ...);
...
}
However, both OSGi and Java 9's jigsaw don't like that we use com.sun.tools.xjc.Options, not in our implementation and especially not in our public API interface.
How can we get rid of it?
The JDeps website lists some of the JDK internal APIs and the recommended way to replace their usage. However, the use of ModelLoader.load() is not mentioned. My guess is that this use case has not come up enough to get the attention of the JDeps team.
My recommendation would be to refactor this method so that
you pass in the data you're using to construct the Options argument, instead of passing in the Options argument
use that data to construct your JaxbConfiguration object instead of converting from the internal Model.
You don't mention what JaxbConfiguration is or what library it's from so it's hard for me to say exactly how to construct it. Anyway, this answer is about how to remove the use of the internal API. How to construct a JaxbConfiguration is probably a different question.
Using servicestack, there are examples of unit testing using types, etc. Here is an example:
GetFactorial
I would like to test my REST style service with a test similar to the above.
Here is an example REST unit test FileService
Notice how in the PUT unit test, the Path argument has to be specified in the URL text instead of in the class argument. Another example is here, where we have perfectly good request models that have to be translated into the URL. For testing, I would like to get away from having to build the arguments in the url and use a system similar to the one above like this:
var response = restClient.Put<FilesResponse>(new Files { TextContents = ReplacedFileContents, Path = "README.txt" });
or
var singleCustomer = restClient.Get<Customer>(new Customer {Id=1};
Is this possible?
Then there is the DirectServiceClient. Would that help? In the end, with servicestack, we get to write services and they can be called from many different type clients - I would like to write my unit test like that.
Is this possible?
If you decorate your DTOs with the route variable and use ServiceStack's "New API" then it can discover the routes automatically. You can get away with writing very minimal code and still get a strong typed rest API.
Your code could look something like this:
Customer singleCustomer = restClient.Get(new Customer {Id=1});
See https://github.com/ServiceStack/ServiceStack/wiki/New-Api
In response to your comments, your DTO needs to adhere to the IReturn interface:
[Route("/customer/{Id}")]
public Customer : IReturn<Customer> {
public int Id {get;set;}
}
The IRestClient interface below will now be able to work with your DTO without specify the type since it is expecting an IReturn object.
public interface IRestClient
{
TResponse Get<TResponse>(IReturn<TResponse> request);
...
}