I am currently writing a Lambda authorizer for an AWS AppSync API, however the authorization depends on the target resource being accessed.
Every resource has their own ACL listing the users and conditions for allowing access to it.
Currently the best I could find would be to get the identity of the caller, look at all the ACLs, and authorize the call while denying access to all the other resources, what's not only highly inefficient, but also extremely impractical, if not impossible.
The solution I had originally came up with was to get the target resource, retrieve the ACL and check if the user fits the specified criteria. The problem is that I am unable to reliably define what's the target resource. What I get from AWS is a request like this:
{
"authorizationToken": "ExampleAUTHtoken123123123",
"requestContext": {
"apiId": "aaaaaa123123123example123",
"accountId": "111122223333",
"requestId": "f4081827-1111-4444-5555-5cf4695f339f",
"queryString": "mutation CreateEvent {...}\n\nquery MyQuery {...}\n",
"operationName": "MyQuery",
"variables": {}
}
}
So, I only have the query string and variables, leaving the actual parsing of this to me. I got to convert it to an AST using graphql-js, but it's still extremely verbose and most importantly, it's structure varies greatly.
My first code to retrieve the target worked for the AppSync console queries, but not the Amplify Front-End, for example. I also can't rely on something as simple as the variable name, as an attacker could quite easily craft a query with an arbitrary name, or even not use variables at all.
I thought about implementing this authorization logic within Lambda Resolvers, what should be simpler in a way, but would require me to use resolvers as authorizers, what doesn't seem ideal, and implement the entire resolver logic when I just want the most trivial possible resolvers.
Ideally I'd like something like this:
/* Schema:
type Query {
operationName(key: KEY!): responseType
}*/
/* Query:
query abitraryQueryName($var1: KEY!) {
operationName(key: $var1) {
field1
field2
}
}*/
/* Variables:
{ "var1": "value1" } */
parsedQuery = {
operation: "operationName",
params: { "key": "value1" },
fields: [ "field1", "field2" ]
};
Is there any way to resolve/simplify the queries from GraphQL to JSON/similar in a way that this information can be easily extracted?
Well, couldn't find anything on it, so I made something myself.
On the off chance someone needs something similar, here's the gist with the code I used: https://gist.github.com/Iorpim/6544dad46060522dd0b17477871bc434
I didn't make it a proper full lib, as it's a very specific use case and it's likely a one-off, and I am also not sure how reliable it is, but it solves my problem!
Related
Sorry for the long post, but I really need some guidance here. I need to compare values from an API response with the values from multiple tables in the DB.
Currently, I am doing it as follows:
Use a JDBC Connect Configuration to connect to Postgres DB and then use the JDBC Sampler to execute queries. I use it three times to query 3 different tables. I store this data in variables (lets call them DBVariables). Please see this image for current Jmeter setup. https://i.stack.imgur.com/GZJyF.png
In JSR Assertion, I have written code that takes data from various DBVariables and compares it against the API response.
However, my issue is the API response can have an array of records and then nested arrays inside each (please see API response sample below). And these array elements can be sorted in any order. This is where I have issues.
I was wondering what would be the most efficient way of writing this JSR Assertion to validate all data elements returned by the API are the same as what is in the DB.
I am very new to groovy, but I think if I can query the DB inside the JSR assertion (instead of using the JDBC sampler), then the comparison can be done by storing API response in a map and then the DBResponse in another map and sorting both and comparing the items.
My questions are:
How can I connect to postgressql using groovy and then execute query statements in it? I have not done that before and was hoping if someone can provide a sample code.
How can I store API response and DB responses in Map and sort them and compare them in groovy?
The API response is of the following type:
{
"data":{
"response":{
"employeeList":[
{
"employeeNumber":"11102",
"addressList":[
{
"addrType":"Home",
"street_1":"123 Any street"
},
{
"addrType":"Alternate",
"street_1":"123 Any street"
}
],
"departmentList":[
{
"deptName":"IT"
},
{
"deptName":"Finance"
},
{
"deptName":"IT"
}
]
},
{
"employeeNumber":"11103",
"addressList":[
{
"addrType":"Home",
"street_1":"123 Any street"
},
{
"addrType":"Alternate",
"street_1":"123 Any street"
}
],
"departmentList":[
{
"deptName":"IT"
},
{
"deptName":"Finance"
},
{
"deptName":"IT"
}
]
}
]
}
}
}
Have you seen Working with a relational database chapter of Groovy documentation? Alternatively you can obtain a Connection instance from the JDBC Configuration Element like
def connection = org.apache.jmeter.protocol.jdbc.config.getConnection('your-pool-name')
With regards to "sort" There is DefaultGroovyMethods class which provides sort() function for any "sortable" entity. With regards to "compare" - we don't know how the object from the database looks like hence cannot provide a comprehensive solution.
Maybe an easier option would be converting the response from the JDBC Sampler to JSON using JsonBuilder and once you have 2 JSON structures use the library like JSONassert which doesn't care about order and depth
You haven't asked, but if you're "very new to groovy" maybe it worth extracting individual values from API using JSON Extractor, do the same for the database with the JDBC elements and compare individual JMeter Variables using Response Assertion?
Every example of using expressjs and mongoose are like that:
const contentTypes = await ContentType.find().sort({createdAt: -1});
res.json(contentTypes);
But in this scenario we are returning all document by REST api (even mongoose version field '_v'. I think it would be good practive to describe interface like this
export class ContentTypeEntry {
id: string;
name: string;
}
and convert mongoose type to this interface object and return this DTO. I just starting using nodejs ecosystem, so maybe in this ecosystem returning directly ORM objects is normal?
How are You dealing with moongose objects and REST endpoints?
I'm not entirely sure if I got the question right, but this is what my response object looks like-
// GET /api/products/1010
{
"meta": {
"type": "success",
"code": 200,
"message": "Retrieved product number 1010"
},
"data": {
"id": 1010,
"name": "Apple iPhone X",
"description: "Yada yada",
"price": 1000
}
}
This just separates the metadata and the actual returned data to make it easier for whoever is consuming the api to handle errors better. I also modify the JSON object to return only the required data and omit things like the version field from the response.
I think this is a great question, even if it's a bit broad. There are lots of frameworks that build on top of Node/Express (for example, LoopBack) to act as the glue between your data layer and your HTTP layer (REST, API, whatever you want to call it), deciding what you want to actually exist at a given endpoint. Happy to share other thoughts here if you have more specific questions.
You could also stay fairly lean and override the toJSON method of your Mongoose object (see this for an example). This is probably in line with your example of having an additional class that your object will conform to before being delivered to the end user, but personally I'd prefer to keep that within my object definition (the Mongoose model file).
I suppose at the end of the day, it's a question of how much control you need, how big the project is going to be and what its future needs will be. For microservices, you may find that Express + Mongoose and a couple of specific strategies will solve your concerns.
I am new to node.js and have my first node.js Restful API built in hapi.js framework. All the services do is basically doing database query. An example of the services is like this:
let myservice = {
method: "POST",
path: "/updateRule",
config: {
handler: (request, reply) => {
updateRule(request.payload)
.then((result) => {
reply(successResponse(request, result));
})
.catch((err) => reply(failResponse(request, err)).code(500));
},
validate: {
payload: {
ruleId: joi.number().required(),
ruleName: joi.string().required(),
ruleDesc: joi.string().required()
}
},
auth: "jwt",
tags: ["api", "a3i"]
},
}
updateRule(input): Promise<any> {
return new Promise((resolve, reject) => {
let query = `select a3i.update_rule(p_rul_id := ${input.ruleId}, p_rul_name := '${input.ruleName}', p_rul_desc := '${input.ruleDesc}')`;
postgresQuery(lbPostgres, query, (data, commit, rollback) => {
try {
let count = data.rows[0].update_rule.count;
if (count === 1) {
let ruleId = data.rows[0].update_rule.result[0];
let payload: SuccessPayload = {
type: "string",
content: `Rule ${ruleId} has been updated`
};
commit();
resolve(payload);
} else {
let thisErr = new Error("No rule can be found.");
thisErr.name = "4003";
throw thisErr;
}
}
catch (err) {
rollback();
if (err.name === "4003") {
reject(detailError(4003, err.message));
} else {
reject(detailError(4001, err.message));
}
}
}, reject);
});
}
As you can see, when the service is called, it evokes a database call (query) and updates specified row in database table. Similarly, I have other services named createRule/deleteRule creating/deleting records in database table. In my opinion, the difference between the services is doing different database query. I read this post PUT vs. POST in REST but couldn't see any difference of POST and PUT in my case.
Here are my questions:
What HTTP method should I used in this case?
Most of Restful API examples (for example https://www.codementor.io/olatundegaruba/nodejs-restful-apis-in-10-minutes-q0sgsfhbd) use the same URL with different HTTP methods to do different operations on same "resource", which in my opinion is usually a database table. What's the benefit of this architecture compared with my practice in which one URL only has one HTTP method and only do one type of operation?
I know this question does not refer to a problem and is not specific. Some people may give it a down-vote. But as a beginner I really want to know what's a typical Restful API and make sure my API is "best practice". Please help!
If the resource already exists and thus you have a specific URI to that exact resource and you want to update it, then use PUT.
If the resource does not exist yet and you want to create it and you will let the server pick the URI that represents that new resource, then use POST and the POST URI will be a generic "create new resource" URI, not a URI to a specific resource and it will create the URI that represents that resource.
You can also use PUT to create a new resource if the caller is going to create the resource URI that represents the new resource. In that case, you would just PUT to that new resource and, if a resource with that URI already exists, it would be updated, if not, it would be created.
You do not have to support both. You can decide to make your api work in a way that you just use one or the other.
In your specific case, an update of a specific row in your database that already exists would pretty much always be a PUT because it already exists so you're doing a PUT to a specific URI that represents that row.
What's the benefit of this architecture compared with my practice in which one URL only has one HTTP method and only do one type of operation?
It's really up to you how you want to present your API. The general concept behind REST is that you have several components:
resource identifier
data
method
In some cases, the method can be subsumed by GET, PUT, POST or DELETE so you just need the resource identifier, data and GET, PUT, POST or DELETE.
In other cases or other designs, the method is more detailed than can be expressed in just a PUT or POST, so you have a method actually in the URL in which case, you may not need the distinction between PUT and POST as much.
For example, an action might be "buy". While you could capture that in a POST where the method is implied by the rest of the URL, you may want to actually POST to a URL that has a method in it: /buy for clarity and then you may use that same endpoint prefix with other methods such as /addToCart, etc... It really depends upon what the objects are in your REST design and what operations you want to surface on them. Sometimes, the objects lends themselves to just GET, PUT, POST and DELETE and sometimes, you want more info in the URL as to the specific operation to be carried out on that resource.
If you want to be Rest compliant, you can just use Post and Get.
If you want to be Restfull, you need to base your method on the CRUD
Create -> Post
Read -> Get
Update -> Put or Patch
Delete -> Delete
About building a full API, using method on the same URL could be easier to build / understand. All queries about your user will be on the user url and not user/get, user/add, user/update ... It let you have the same functionality, without too much different URL.
When you build an API, you will want to have some logs, for stats analysis and other stuff. This way, if you split with your method, you can just have a filter to logs how many Post requests, or Get requests.
In fact, you could build an API only with Get requests too. But spliting with methods and URL is the best way to avoid complexes URL (or URL with too much action name) and to have an easiest way to log every requests going through your API
- List item
Level 1 is Rest
Level 2 is Restfull
Level 3 is Hateoas
You should find more informations inside some books or articles written by Martin Fowler
What I usually do is use "POST" for creating a new resource, and use "PUT" for updating an already existing resource.
For your second question, yes most API's use the same URL to do different things on the same resource. That could be because of security where you don't want to expose what you are doing in your URL's (/delete for example). Also, many frameworks generate an auto URL for a resource (Object Class), that is then differentiated on the request method. People just don't tend to use custom URL's for those.
I would like to store a value in the config file and look it up in the design document for comparing against update values. I'm sure I have seen this but, for the life of me, I can't seem to remember how to do this.
UPDATE
I realize (after the first answer) that there was more than one way to interpret my question. Hopefully this example clears it up a little. Given a configuration:
curl -X PUT http://localhost:5984/_config/shared/token -d '"0123456789"'
I then want to be able to look it up in my design document
{
"_id": "_design/loadsecrets",
"validate_doc_update": {
"test": function (newDoc,oldDoc) {
if (newDoc.supersecret != magicobject.config.shared.token){
throw({unauthorized:"You don't know the super secret"});
}
}
}
}
It's the abilitly to do something like the magicobject.config.shared.token that I am looking for.
UPDATE 2
Another potentially useful (contrived) scenario
curl -X PUT http://trustedemployee:5984/_config/eventlogger/detaillevel -d '"0"'
curl -X PUT http://employee:5984/_config/eventlogger/detaillevel -d '"2"'
curl -X PUT http://vicepresident:5984/_config/eventlogger/detaillevel -d '"10"'
Then on devices tracking employee behaviour:
{
"_id": "_design/logger",
"updates": {
"logger": function (doc,req) {
if (!doc) {
doc = {_id:req.id};
}
if(req.level < magicobject.config.eventlogger.detaillevel ){
doc.details = req.details;
}
return [doc, req.details];
}
}
}
Here's a follow-up to my last answer with more general info:
There is no general way to use configuration, because CouchDB is designed with scalability, stability and predictability in mind. It has been designed using many principles of functional programming and pure functions, avoiding side effects as much as possible. This is a Good Thing™.
However, each type of function has additional parameters that you can use, depending on the context the function is called with:
show, list, update and filter functions are executed for each request, so they get the request object. Here you have the req.secObj and req.userCtx to (ab)use for common configuration. Also, AFAIK the this keyword is set to the current design document, so you can use the design doc to get common configuration (at least up to CouchDB 1.6 it worked).
view functions (map, reduce) don't have additional parameters, because the results of a view are written to disk and reused in subsequent calls. map functions must be pure (so don't use e.g. Math.random()). For shared configuration across view functions within a single design doc you can use CommonJS require(), but only within the views.lib key.
validate doc update functions are not necessarily executed within a user-triggered http request (they are called before each write, which might not be triggered only via http). So they have the userCtx and secObj added as separate parameters in their function signature.
So to sum up, you can use the following places for configuration:
userCtx for user-specific config. Use a special role (e.g. with a prefix) for storing small config bits. For example superLogin does this.
secObj for database-wide config. Use a special member name for small bits (as you should normally use roles instead of explicit user names, secObj.members.names or secObj.admins.names is a good place).
the design doc itself for design-doc-wide config. Best use the this.views.lib.config for this, as you can also read this key from within views. But keep in mind that all views are invalidated as soon as you change this key. So if the view results will stay the same no matter what the config values are, it might be better to use a this.config key.
Hope this helps! I can also add examples if you wish.
I think I know what you're talking about, and if I'm right then what you are asking for is no longer possible. (at least in v1.6 and v2.0, I'm not sure when this feature was removed)
There was a lesser-known trick that allowed a view/show/list/validation/etc function to access the parent design document as this in your function. For example:
{
"_id": "_design/hello-world",
"config": {
"PI": 3.14
},
"views": {
"test": {
"map": "function (doc) { emit(this.config.PI); })"
}
}
}
This was a really crazy idea, and I imagine it was removed because it created a circular dependency between the design document and the code of the view that made the process of invalidating/rebuilding a view index a very tricky affair.
I remember using this trick at some point in the distant past, but the feature is definitely gone now. (and likely to never return)
For your special use-case (validating a document with a secret token), there might be a workaround, but I'm not sure if the token might leak in some place. It all depends what your security requirements are.
You could abuse the 4th parameter to validate_doc_update, the securityObject (see the CouchDB docs) to store the secret token as the first admin name:
{
"test": "function (newDoc, oldDoc, userCtx, secObj) {
var token = secObj.admins.names[0];
if (newDoc.supersecret != token) {
throw({unauthorized:"You don't know the super secret"});
}
}"
}
So if you set the db's security object to {admins: {names: ["s3cr3t-t0k3n"], roles: ["_admin"]}}, you have to pass 's3cr3t-t0k3n' as the doc's supersecret property.
This is obviously a dirty hack, but as far as I remember, the security object may only be read or modified by admins, you wouldn't immediately leak your token to the public. But consider adding a separate layer between the CouchDB and your caller if you need "real" security.
I'm trying to get notifications in a CouchDB change poll as soon as pre-defined field is set or changed. I've already had a look at filters that can be used for filtering change events(db/_changes?filter=myfilter). However, I've not yet found a way to include this temporal information, because you can only get the current version of the document in this filter functions.
Is there any possibility to create such a filter?
If it does not work, I could export my field to a separate database and the only poll for changes in that db, but I'd prefer to keep together my data for obvious reasons.
Thanks in advance!
You are correct: filters and _changes feeds can only see snapshots of a document. What you need is a function which can see the old document and the new document and act correctly. But that is unavailable in _filters and _changes.
Obviously your client code knows if it updates that field. You might update your client code however there is a better solution.
Update functions can access both documents. I suggest you make an _update
function which notices the field change and flags that in the document. Next you
have a simple filter checking for that flag. The best part is, you can use a
rewrite function to make the HTTP API exactly the same as before.
1. Create an update function to flag interesting updates
Your _design/myapp would be {"updates", "smart_updater": "(see below)"}.
Update functions are very flexible (see my recent update handlers
walkthrough). However we only want to mimic the normal HTTP/JSON API.
Your updates.smart_updater field would look like this:
function (doc, req) {
var INTERESTING = 'dollars'; // Set me to the interesting field.
var newDoc = JSON.parse(req.body);
if(newDoc.hasOwnProperty(INTERESTING)) {
// dollars was set (which includes 0, false, null, undefined
// values. You might test for newDoc[INTERESTING] if those
// values should not trigger this code.
if((doc === null) || (doc[INTERESTING] !== newDoc[INTERESTING])) {
// The field changed or created!
newDoc.i_was_changed = true;
}
}
if(!newDoc._id) {
// A UUID generator would be better here.
newDoc._id = req.id || Math.random().toString();
}
// Return the same JSON the vanilla Couch API does.
return [newDoc, {json: {'id': newDoc._id}}];
}
Now you can PUT or POST to /db/_design/myapp/_update/[doc_id] and it will feel
just like the normal API except if you update the dollars field, it will add
an additional flag, i_was_changed. That is how you will find this change
later.
2. Filter for documents with the changed field
This is very straightforward:
function(doc, req) {
return doc.i_was_changed;
}
Now you can query the _changes feed with a ?filter= parameter. (Replication
also supports this filter, so you could pull to your local system all documents
which most recently changed/created the field.
That is the basic idea. The remaining steps will make your life easier if you
already have lots of client code and do not want to change the URLs.
3. Use rewriting to keep the HTTP API the same
This is available in CouchDB 0.11, and the best resource is Jan's blog post,
nice URLs in CouchDB.
Briefly, you want a vhost which sends all traffic to your rewriter (which itself
is a flexible "bouncer" to all design doc functionality based on the URL).
curl -X PUT http://example.com:5984/_config/vhosts/example.com \
-d '"/db/_design/myapp/_rewrite"'
Then you want a rewrites field in your design doc, something like (not
tested)
[
{
"comment": "Updates should go through the update function",
"method": "PUT",
"from": "db/*",
"to" : "db/_design/myapp/_update/*"
},
{
"comment": "Creates should go through the update function",
"method": "POST",
"from": "db/*",
"to" : "db/_design/myapp/_update/*"
},
{
"comment": "Everything else is just like normal",
"from": "*",
"to" : "../../../*"
}
]
(Once again, I got this code from examples and existing code I have laying
around but it's not 100% debugged. However I think it makes the idea very clear.
Also remember this step is optional however the advantage is, you never have to
change your client code.)