I have the following rest endpoint code "/files/lookup". This will receive a query parameter folderPath, and will return a list of files with details (including metadata) but not content.
I am including the content of the rest endpoint. This connects to azure blob storage.
#get('/files/lookup', { ... })
...
const blobServiceClient: BlobServiceClient = BlobServiceClient.fromConnectionString(
this.azureStorageConnectionString,
);
const containerClient: ContainerClient = blobServiceClient.getContainerClient(container);
const filesPropertiesList: FileProps[] = [];
try {
for await (const item of containerClient.listBlobsByHierarchy('/', {
prefix: decodedAzureFolderPath,
includeMetadata: true,
})) {
if (item.kind !== 'prefix') {
const blobitem: BlobItem = item;
const blobProperties: BlobProperties = blobitem.properties;
const blobMetadata: Record<string, string> | undefined = blobitem.metadata;
const aFileProperties: FileProps = {
name: item?.name,
uploadedDate:
blobProperties.lastModified?.toISOString() ?? blobProperties.createdOn?.toISOString(),
size: blobProperties.contentLength,
contentType: blobProperties.contentType,
metadata: blobMetadata,
};
filesPropertiesList.push(aFileProperties);
}
}
} catch (error) {
if (error.statusCode === 404) {
throw new HttpErrors.NotFound('Retrieval of list of files has failed');
}
throw error;
}
return filesPropertiesList;
I am working on sinon test. I am new to sinon. I could not get to effectively use mocks/stubs/etc. to test the endpoint returning a list of files with properties. Couldn't get my head around mocking/stubbing the listBlobsByHierarchy method of the container client
describe('GET /files/lookup', () => {
let blobServiceClientStub: sinon.SinonStubbedInstance<BlobServiceClient>;
let fromConnectionStringStub: sinon.SinonStub<[string, StoragePipelineOptions?], BlobServiceClient>;
let containerStub: sinon.SinonStubbedInstance<ContainerClient>;
beforeEach(async () => {
blobServiceClientStub = sinon.createStubInstance(BlobServiceClient);
fromConnectionStringStub = sinon
.stub(BlobServiceClient, 'fromConnectionString')
.returns((blobServiceClientStub as unknown) as BlobServiceClient);
containerStub = sinon.createStubInstance(ContainerClient);
blobServiceClientStub.getContainerClient.returns((containerStub as unknown) as ContainerClient);
});
afterEach(async () => {
fromConnectionStringStub.restore();
});
it('lookup for files from storage', async () => {
/* let items: PagedAsyncIterableIterator<({ kind: "prefix"; } & BlobPrefix) | ({ kind: "blob"; } & BlobItem), ContainerListBlobHierarchySegmentResponse>;
sinon.stub(containerStub, "listBlobsByHierarchy").withArgs('/', { prefix: "myf/entity/172/", includeMetadata: true }).returns(items);
const response = await client.get(`/files/lookup?folderpath=myf%2Fentity%2F172%2F`).expect(200); */
});
});
Since I did not find any ways to mock the return of this method with the same type, I went with type "any". As I am a novice on this, it was really challenging to get my head to do this!
it('lookup for files from storage', async () => {
/* eslint-disable #typescript-eslint/naming-convention */
const obj: any = [
{
kind: 'blob',
name: 'myf/entity/172/0670fdf8-db47-11eb-8d19-0242ac13000.docx',
properties: {
createdOn: new Date('2020-01-03T16:27:32Z'),
lastModified: new Date('2020-01-03T16:27:32Z'),
contentLength: 11980,
contentType: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
},
metadata: {
object_name: 'Testing.docx',
category: 'entity',
reference: '172',
object_id: '0670fdf8-db47-11eb-8d19-0242ac13000',
},
},
];
containerStub.listBlobsByHierarchy.returns(obj);
const actualResponse = await (await client.get('/files/lookup?folderpath=myf/entity/172')).body;
const expectedResponse: any[] = [ WHATEVER ]
expect(actualResponse).deepEqual(expectedResponse);
});
Related
Trying to write a query for DynamoDB and learn promises etc. The console.log(resp.Items) returns the object that I am looking for so I think my query is formatted correctly. I get a status 200 back with an empty object.
I have read up for a few days and tried to implement various changes to the code by nothing is returning the object resp.Items. I am guessing the function is returning before the const is updated with the data but I am not sure why the console.log works.
const AWS = require('aws-sdk')
const dynamodb = new AWS.DynamoDB()
const getTechs = async () => {
try {
const resp = await dynamodb
.query({
ExpressionAttributeValues: {
':tech': { S: 'TECH#' },
},
KeyConditionExpression: 'PK = :tech',
TableName: process.env.TABLE_NAME,
ScanIndexForward: true,
})
.promise()
console.log(resp.Items)
if (!resp.Items) {
return {
error: 'No Techs in the DB',
}
}
return {
tech: resp.Items,
}
} catch (error) {
console.log('Error retrieving Tech List')
console.log(error)
return {
error: 'Could not retrieve Tech List',
}
}
}
handler func
const { makeHandler } = require('./utils')
const { getTechs } = require('../data')
// const { Tech } = require('../entities')
const inputSchema = {
type: 'object',
properties: {
pathParameters: {
type: 'object',
properties: {
tech: { type: 'string' },
},
required: ['tech'],
},
},
required: ['pathParameters'],
}
const handler = async (event) => {
const { techs, error } = await getTechs()
const statusCode = error ? 500 : 200
const body = error ? JSON.stringify({ error }) : JSON.stringify({ techs })
return {
statusCode,
body,
}
}
module.exports.handler = makeHandler({ handler })
executeTransactWrite func
const executeTransactWrite = async ({ tech, params }) => {
const transactionRequest = tech.transactWriteItems(params)
let cancellationReasons
transactionRequest.on('extractError', (response) => {
try {
cancellationReasons = JSON.parse(
response.httpResponse.body.toString()
).CancellationReasons
} catch (err) {
// suppress this just in case some types of errors aren't JSON parseable
console.error('Error extracting cancellation error', err)
}
})
return new Promise((resolve, reject) => {
transactionRequest.send((err, response) => {
if (err) {
err.cancellationReasons = cancellationReasons
return reject(err)
}
return resolve(response)
})
})
}
module.exports = {
executeTransactWrite,
makehandler func
const middy = require('middy')
const {
jsonBodyParser,
validator,
httpErrorHandler,
} = require('middy/middlewares')
const makeHandler = ({ handler, inputSchema }) =>
middy(handler)
.use(jsonBodyParser())
.use(validator({ inputSchema }))
.use(httpErrorHandler())
module.exports = { makeHandler }
I am fairly new to using graphql-ws and graphql-yoga server, so forgive me if this is a naive question or mistake from my side.
I went through graphql-ws documentation. It has written the schema as a parameter. Unfortunately, the schema definition used in the documentation is missing a reference.
After adding a new todo (using addTodo) it shows two todo items. So I believe it is unable to return the initial todo list whenever running subscribe on Yoga Graphiql explorer.
It should show the initial todo item as soon as it has been subscribed and published in the schema definition.
My understanding is there is something I am missing in the schema definition which is not showing the todo list when tried accessing Yoga Graphiql explorer.
Has anyone had a similar experience and been able to resolve it? What I am missing?
Libraries used
Backend
graphql-yoga
ws
graphql-ws
Frontend
solid-js
wonka
Todo item - declared in schema
{
id: "1",
title: "Learn GraphQL + Solidjs",
completed: false
}
Screenshot
Code Snippets
Schema definition
import { createPubSub } from 'graphql-yoga';
import { Todo } from "./types";
let todos = [
{
id: "1",
title: "Learn GraphQL + Solidjs",
completed: false
}
];
// channel
const TODOS_CHANNEL = "TODOS_CHANNEL";
// pubsub
const pubSub = createPubSub();
const publishToChannel = (data: any) => pubSub.publish(TODOS_CHANNEL, data);
// Type def
const typeDefs = [`
type Todo {
id: ID!
title: String!
completed: Boolean!
}
type Query {
getTodos: [Todo]!
}
type Mutation {
addTodo(title: String!): Todo!
}
type Subscription {
todos: [Todo!]
}
`];
// Resolvers
const resolvers = {
Query: {
getTodos: () => todos
},
Mutation: {
addTodo: (_: unknown, { title }: Todo) => {
const newTodo = {
id: "" + (todos.length + 1),
title,
completed: false
};
todos.push(newTodo);
publishToChannel({ todos });
return newTodo;
},
Subscription: {
todos: {
subscribe: () => {
const res = pubSub.subscribe(TODOS_CHANNEL);
publishToChannel({ todos });
return res;
}
},
},
};
export const schema = {
resolvers,
typeDefs
};
Server backend
import { createServer } from "graphql-yoga";
import { WebSocketServer } from "ws";
import { useServer } from "graphql-ws/lib/use/ws";
import { schema } from "./src/schema";
import { execute, ExecutionArgs, subscribe } from "graphql";
async function main() {
const yogaApp = createServer({
schema,
graphiql: {
subscriptionsProtocol: 'WS', // use WebSockets instead of SSE
},
});
const server = await yogaApp.start();
const wsServer = new WebSocketServer({
server,
path: yogaApp.getAddressInfo().endpoint
});
type EnvelopedExecutionArgs = ExecutionArgs & {
rootValue: {
execute: typeof execute;
subscribe: typeof subscribe;
};
};
useServer(
{
execute: (args: any) => (args as EnvelopedExecutionArgs).rootValue.execute(args),
subscribe: (args: any) => (args as EnvelopedExecutionArgs).rootValue.subscribe(args),
onSubscribe: async (ctx, msg) => {
const { schema, execute, subscribe, contextFactory, parse, validate } =
yogaApp.getEnveloped(ctx);
const args: EnvelopedExecutionArgs = {
schema,
operationName: msg.payload.operationName,
document: parse(msg.payload.query),
variableValues: msg.payload.variables,
contextValue: await contextFactory(),
rootValue: {
execute,
subscribe,
},
};
const errors = validate(args.schema, args.document);
if (errors.length) return errors;
return args;
},
},
wsServer,
);
}
main().catch((e) => {
console.error(e);
process.exit(1);
});
apply these changes
Mutation: {
addTodo: (_: unknown, { title }: Todo) => {
const newTodo = {
id: "" + (todos.length + 1),
title,
completed: false
};
todos.push(newTodo);
publishToChannel({ todos });
return newTodo;
},
Subscription: {
todos: {
subscribe: () => {
return Repeater.merge(
[
new Repeater(async (push, stop) => {
push({ todos });
await stop;
}),
pubSub.subscribe(TODOS_CHANNEL),
]
)
}
},
},
first, npm i #repeaterjs/repeater then import Repeater
here is my scheme, I wasnt to create one record so when i am passing field in graphql it is showing error
const { BookTC } = require("../model/book");
const { BookSchema } = require("../model/book");
BookTC.addResolver({
name: "create",
kind: "mutation",
type: BookTC.getResolver("createOne").getType(),
args: BookTC.getResolver("createOne").getArgs(),
resolve: async ({ source, args, context, info }) => {
const book = await BookSchema.create(args.record);
return {
record: book,
recordId: BookTC.getRecordIdFn()(book),
};
},
});
const BookMutation = {
bookWithFile: BookTC.getResolver("create"),
bookCreateOne: BookTC.getResolver("createOne"),
bookCreateMany: BookTC.getResolver("createMany")
};
module.exports = { BookQuery: BookQuery, BookMutation: BookMutation };
There are 3 bases (front, node, remote). Front <=> node, node <=> remote. When the front base is updated, the data goes to the remote base, but the node is not updated. In theory, the node should be updated first, and then the remote base.
Render db
addPouchPlugin(PouchdbAdapterIdb)
addPouchPlugin(PouchHttpPlugin)
addRxPlugin(RxDBReplicationCouchDBPlugin)
addRxPlugin(RxDBMigrationPlugin)
addRxPlugin(RxDBLeaderElectionPlugin)
addRxPlugin(RxDBQueryBuilderPlugin)
addRxPlugin(RxDBAjvValidatePlugin)
addRxPlugin(RxDBUpdatePlugin)
export const createDb = async () => {
console.log('[src/renderer/database/createDb] createDb')
const productsName = collectionName.getCollectionProductsName()
const documentsName = collectionName.getCollectionDocumentsName()
const settingsName = collectionName.getCollectionSettingsName()
const db = await createRxDatabase<Collections>({
name: 'renderer',
// use pouchdb with the indexeddb-adapter as storage engine.
storage: getRxStoragePouch('idb'),
})
await initCommonCollections({ db, documentsName, productsName, settingsName })
syncDbCollections(db, [productsName, documentsName, settingsName])
db.$.subscribe(({ operation, documentId, documentData }) => {
if (documentData.type === SettingsTypes.DEVICE_SETTING) {
console.log(`Change database RENDER event:\n ${operation}, \n documentData:`, documentData)
}
})
return db
}
Render sync
const remoteDbUrl = `http://localhost:3030/db/`
const logPath = '[src/renderer/database/syncDbCollections]'
export const syncDbCollections = (db: RxDatabase<Collections>, collectionNames: (keyof Collections)[]) => {
console.log('syncDbCollections', collectionNames)
collectionNames.forEach(name => {
const rxReplicationState = db.collections[name].syncCouchDB({
remote: `${remoteDbUrl}${name}`,
options: {
live: true,
retry: true,
},
})
rxReplicationState.error$.subscribe(error => {
console.error(logPath, name, 'error', JSON.stringify(error))
})
})
}
Node base
addPouchPlugin(PouchdbAdapterHttp)
addPouchPlugin(LevelDbAdapter)
addRxPlugin(RxDBAjvValidatePlugin)
addRxPlugin(RxDBMigrationPlugin)
addRxPlugin(RxDBServerPlugin)
addRxPlugin(RxDBLeaderElectionPlugin)
addRxPlugin(RxDBQueryBuilderPlugin)
addRxPlugin(RxDBUpdatePlugin)
addRxPlugin(RxDBReplicationCouchDBPlugin)
let db: RxDatabase<Collections>
export const getMainDb = () => {
if (!db) {
throw new Error('No available database.')
}
return db
}
export const getDocumentCollection = (): DocumentsRxCol => {
return db[collectionNames.getCollectionDocumentsName()]
}
export const getSettingsCollection = (): SettingsRxCol => {
return db[collectionNames.getCollectionSettingsName()]
}
export const getProductsCollection = (): ProductsRxCol => {
return db[collectionNames.getCollectionProductsName()]
}
export const initDatabase = async () => {
console.log(logPathAlias, 'initDatabase')
if (db) {
console.warn(logPathAlias, 'db instance already created!')
return db
}
db = await createRxDatabase<Collections>({
name: `${electronApp.getPath('userData')}/db`,
storage: getRxStoragePouch(LevelDown),
})
const productsName = collectionNames.getCollectionProductsName()
const documentsName = collectionNames.getCollectionDocumentsName()
const settingsName = collectionNames.getCollectionSettingsName()
await initCommonCollections({ db, productsName, documentsName, settingsName })
await syncCollections([productsName, documentsName, settingsName])
db.$.subscribe(({ operation, documentId, documentData }) => {
// if (documentData.type === SettingsTypes.DEVICE_SETTING) {
console.log(`Change database NODE event:\n ${operation}, \n documentData:`, documentData)
// }
})
const { app } = await db.server({
startServer: false, // (optional), start express server
// options of the pouchdb express server
cors: false,
pouchdbExpressOptions: {
inMemoryConfig: true, // do not write a config.json
logPath: `${electronApp.getPath('temp')}/rxdb-server.log`, // save logs in tmp folder
},
})
return app
}
const lastRetryTime = {}
const syncCollections = async (collections: CollectionNames[]) => {
collections.map(collectionName => {
const rxReplicationState = db.collections[collectionName].syncCouchDB({
remote: `${CouchDbServerUrl}/${collectionName}`,
options: {
live: true,
retry: true,
// #ts-ignore
// headers: {
// Authorization: `Bearer ${getAccessToken()}`,
// },
},
})
rxReplicationState.error$.subscribe(async error => {
console.error(logPathAlias, collectionName, String(error))
if (error.status === 401 && dayjs().diff(lastRetryTime[collectionName], 'seconds') > 10 && getIsRefreshFresh()) {
lastRetryTime[collectionName] = dayjs()
await rxReplicationState.cancel()
await refreshTokens()
await syncCollections([collectionName])
}
})
})
}
No errors
Moreover, if you save data in a remote database, then they are synchronized with the node
Help me :(
I'm trying to mock a fetch call using thisfetch-mock-jest but it the code still trys to go to the remote address and eventually fail with error message FetchError: request to https://some.domain.io/app-config.yaml failed, reason: getaddrinfo ENOTFOUND some.domain.io].
Here the the test code
import { AppConfig } from '#backstage/config';
import { loadConfig } from './loader';
import mockFs from 'mock-fs';
import fetchMock from 'fetch-mock-jest';
describe('loadConfig', () => {
beforeEach(() => {
fetchMock.mock({
matcher: '*',
response: `app:
title: Example App
sessionKey: 'abc123'
`
});
});
afterEach(() => {
fetchMock.mockReset();
});
it('load config from remote path', async () => {
const configUrl = 'https://some.domain.io/app-config.yaml';
await expect(
loadConfig({
configRoot: '/root',
configTargets: [{ url: configUrl }],
env: 'production',
remote: {
reloadIntervalSeconds: 30,
},
})
).resolves.toEqual([
{
context: configUrl,
data: {
app: {
title: 'Example App',
sessionKey: 'abc123',
},
},
},
]);
expect(fetchMock).toHaveBeenCalledTimes(1);
});
function defer<T>() {
let resolve: (value: T) => void;
const promise = new Promise<T>(_resolve => {
resolve = _resolve;
});
return { promise, resolve: resolve! };
}
});
loadConfig has the fetch code that I'm trying to mock.
export async function loadConfig(
options: LoadConfigOptions,
): Promise<AppConfig[]> {
const loadRemoteConfigFiles = async () => {
const configs: AppConfig[] = [];
const readConfigFromUrl = async (remoteConfigProp: RemoteConfigProp) => {
const response = await fetch(remoteConfigProp.url);
if (!response.ok) {
throw new Error(
`Could not read config file at ${remoteConfigProp.url}`,
);
}
remoteConfigProp.oldETag = remoteConfigProp.newETag ?? undefined;
remoteConfigProp.newETag =
response.headers.get(HTTP_RESPONSE_HEADER_ETAG) ?? undefined;
remoteConfigProp.content = await response.text();
return remoteConfigProp;
};
.......
return configs;
}
let remoteConfigs: AppConfig[] = [];
if (remote) {
try {
remoteConfigs = await loadRemoteConfigFiles();
} catch (error) {
throw new Error(`Failed to read remote configuration file, ${error}`);
}
}
........ do some stuff with config then return
return remoteConfigs;
}
The config is a yaml file, that eventually gets parsed and converted into config object.
Any idea why is it failing to mock the fetch call?
replaced
import fetchMock from 'fetch-mock-jest';
with
const fetchMock = require('fetch-mock').sandbox();
const nodeFetch = require('node-fetch');
nodeFetch.default = fetchMock;
and fetchMock.mockReset(); with fetchMock.restore();