Next.js - using BigQuery client library gives an error : Module not found: Can't resolve 'child_process' - node.js

I am trying to query bigQuery dataset from a next.js project.
I have installed #google-cloud/bigquery and followed the steps from here
I have also tried next.js related solutions from this link but still getting below error.
It looks like next.config.js needs to be configured for this to allow this api call. I am not sure what needs to be changed.
Could someone please help me resolve this issue?
here is my code :
const { BigQuery } = require("#google-cloud/bigquery");
const bigquery = new BigQuery();
useEffect(() => {
async function queryBigQuery() {
const query = `
SELECT fieldname
FROM \`db.dataset.tablename\` WHERE columnname = 50
LIMIT 10`;
const options = {
query: query,
};
// Run the query
const [rows] = await bigquery.query(options);
console.log("Query Results:");
rows.forEach((row) => {
const url = row["url"];
const viewCount = row["view_count"];
console.log(`url: ${url}, ${viewCount} views`);
});
}
queryBigQuery();
}, []);
**wait - compiling...
error - ./node_modules/google-auth-library/build/src/auth/googleauth.js:17:0
Module not found: Can't resolve 'child_process'**
UPDATED:
I am able to load bigQuery library I think on client side but its giving me new error.
Here is my latest next.config.js file
module.exports = {
webpack: (config, { isServer, webpack }) => {
if (!isServer) {
config.node = {
dgram: "empty",
fs: "empty",
net: "empty",
tls: "empty",
child_process: "empty",
};
}
return config;
},
env: {
project variables.
};
New Error:

#google-cloud/bigquery is meant to run on a Node.js environment, it won't work in the browser.
You'll need to move your code to a data fetching method like getStaticProps/getServerSideProps or to an API route, as they all run server-side.
Here's an example using an API route, as it seems to fit your use-case best.
// pages/api/bigquery
const { BigQuery } = require("#google-cloud/bigquery");
const bigquery = new BigQuery();
export default function handler(req, res) {
const query = `
SELECT fieldname
FROM \`db.dataset.tablename\` WHERE columnname = 50
LIMIT 10
`;
const options = {
query: query,
};
// Run your query/logic here
res.json(data); // Return your JSON data after logic has been applied
}
Then, in your React component's useEffect:
const queryBigQuery = async () => {
const res = await fetch('api/bigquery');
const data = await res.json(); // Returns JSON data from API route
console.log(data);
}
useEffect(() => {
queryBigQuery();
}, []);

Related

How to override url for RTK query

I'm writing pact integration tests which require to perform actual call to specific mock server during running tests.
I found that I cannot find a way to change RTK query baseUrl after initialisation of api.
it('works with rtk', async () => {
// ... setup pact expectations
const reducer = {
[rtkApi.reducerPath]: rtkApi.reducer,
};
// proxy call to configureStore()
const { store } = setupStoreAndPersistor({
enableLog: true,
rootReducer: reducer,
isProduction: false,
});
// eslint-disable-next-line #typescript-eslint/no-explicit-any
const dispatch = store.dispatch as any;
dispatch(rtkApi.endpoints.GetModules.initiate();
// sleep for 1 second
await new Promise((resolve) => setTimeout(resolve, 1000));
const data = store.getState().api;
expect(data.queries['GetModules(undefined)']).toEqual({modules: []});
});
Base api
import { createApi } from '#reduxjs/toolkit/query/react';
import { graphqlRequestBaseQuery } from '#rtk-query/graphql-request-base-query';
import { GraphQLClient } from 'graphql-request';
export const client = new GraphQLClient('http://localhost:12355/graphql');
export const api = createApi({
baseQuery: graphqlRequestBaseQuery({ client }),
endpoints: () => ({}),
});
query is very basic
query GetModules {
modules {
name
}
}
I tried digging into customizing baseQuery but were not able to get it working.

How to mock #google-cloud/kms using jest

I'm trying to write unit test cases for decrypt. I've my own implementation of decrypting an encrypted file. While trying to import the decrypt.mjs facing the following error.
Must use import to load ES Module: /node_modules/bignumber.js/bignumber.mjs
My application is a react frontend and NodeJS backend. I've used ES6 modules for NodeJS. Here is my decrypt.mjs file
import { readFile } from 'fs/promises';
import path from 'path';
import { KeyManagementServiceClient } from '#google-cloud/kms';
const decrypt = async (APP_MODE, __dirname) => {
if (APP_MODE === 'LOCALHOST') {
const keys = await readFile(
new URL(`./stagingfile.json`, import.meta.url)
).then((data) => JSON.parse(data));
return keys;
}
const { projectId, locationId, keyRingId, cryptoKeyId, fileName } =
getKMSDefaults(APP_MODE);
const ciphertext = await readFile(
path.join(__dirname, `/${fileName}`)
);
const formattedName = client.cryptoKeyPath(
projectId,
locationId,
keyRingId,
cryptoKeyId
);
const request = {
name: formattedName,
ciphertext,
};
const client = new KeyManagementServiceClient();
const [result] = await client.decrypt(request);
return JSON.parse(result.plaintext.toString('utf8'));
};
const getKMSDefaults = (APP_MODE) => {
//Based on APP_MODE the following object contains different values
return {
projectId: PROJECT_ID,
locationId: LOCATION_ID,
keyRingId: KEY_RING_ID,
cryptoKeyId: CRYPTO_KEY_ID,
fileName: FILE_NAME,
};
};
export default decrypt;
I tried to mock the #google-cloud/kms using manual mock (jest) but it didn't work. I tried multiple solutions to mock but nothing worked and it ended with the Must use import to load ES Module error.
I've had successfully used jest to mock #google-cloud/kms with TypeScript, so hopefully this will be the same process for ES modules that you can use.
Example working code:
// jest will "hoist" jest.mock to top of the file on its own anyway
jest.mock("#google-cloud/kms", () => {
return {
KeyManagementServiceClient: jest.fn().mockImplementation(() => {
return {
encrypt: kmsEncryptMock,
decrypt: kmsDecryptMock,
cryptoKeyPath: () => kmsKeyPath,
};
}),
};
});
// give names to mocked functions for easier access in tests
const kmsEncryptMock = jest.fn();
const kmsDecryptMock = jest.fn();
const kmsKeyPath = `project/location/keyring/keyname`;
// import of SUT must be after the variables used in jest.mock() are defined, not before.
import { encrypt } from "../../src/crypto/google-kms";
describe("Google KMS encryption service wrapper", () => {
const plaintext = "some text to encrypt";
const plaintextCrc32 = 1897295827;
it("sends the correct request to kms service and raise error on empty response", async () => {
// encrypt function is async that throws a "new Error(...)"
await expect(encrypt(plaintext)).rejects.toMatchObject({
message: "Encrypt: no response from KMS",
});
expect(kmsEncryptMock).toHaveBeenNthCalledWith(1, {
name: kmsKeyPath,
plaintext: Buffer.from(plaintext),
plaintextCrc32c: { value: plaintextCrc32 },
});
});
});

Next.js not build when using getStaticPaths and props

I'm trying to run next build when using getStaticProps and getStaticPaths method in one of my routes, but it fails every time. Firstly, it just couldn't connect to my API (which is obvious, they're created using Next.js' API routes which are not available when not running a Next.js app). I thought that maybe running a development server in the background would help. It did, but generated another problems, like these:
Error: Cannot find module for page: /reader/[id]
Error: Cannot find module for page: /
> Build error occurred
Error: Export encountered errors on following paths:
/
/reader/1
Dunno why. Here's the code of /reader/[id]:
const Reader = ({ reader }) => {
const router = useRouter();
return (
<Layout>
<pre>{JSON.stringify(reader, null, 2)}</pre>
</Layout>
);
};
export async function getStaticPaths() {
const response = await fetch("http://localhost:3000/api/readers");
const result: IReader[] = await response.json();
const paths = result.map((result) => ({
params: { id: result.id.toString() },
}));
return {
paths,
fallback: false,
};
}
export async function getStaticProps({ params }) {
const res = await fetch("http://localhost:3000/api/readers/" + params.id);
const result = await res.json();
return { props: { reader: result } };
}
export default Reader;
Nothing special. Code I literally rewritten from the docs and adapted for my site.
And here's the /api/readers/[id] handler.
export default async function handler(
req: NextApiRequest,
res: NextApiResponse
) {
const knex = getKnex();
const { id } = req.query;
switch (req.method) {
case "GET":
try {
const reader = await knex
.select("*")
.from("readers")
.where("id", id)
.first();
res.status(200).json(reader);
} catch {
res.status(500).end();
}
break;
}
}
Nothing special either. So why is it crashing every time I try to build my app? Thanks for any help in advance.
You should not fetch an internal API route from getStaticProps — instead, you can write the fetch code present in API route directly in getStaticProps.
https://nextjs.org/docs/basic-features/data-fetching#write-server-side-code-directly

Extensions not returned in GraphQL query results

I'm creating an Apollo Client like this:
var { ApolloClient } = require("apollo-boost");
var { InMemoryCache } = require('apollo-cache-inmemory');
var { createHttpLink } = require('apollo-link-http');
var { setContext } = require('apollo-link-context');
exports.createClient = (shop, accessToken) => {
const httpLink = createHttpLink({
uri: `https://${shop}/admin/api/2019-07/graphql.json`,
});
const authLink = setContext((_, { headers }) => {
return {
headers: {
"X-Shopify-Access-Token": accessToken,
"User-Agent": `shopify-app-node 1.0.0 | Shopify App CLI`,
}
}
});
return new ApolloClient({
cache: new InMemoryCache(),
link: authLink.concat(httpLink),
});
};
to hit the Shopify GraphQL API and then running a query like that:
return client.query({
query: gql` {
productVariants(first: 250) {
edges {
node {
price
product {
id
}
}
cursor
}
pageInfo {
hasNextPage
}
}
}
`})
but the returned object only contain data and no extensions which is a problem to figure out the real cost of the query.
Any idea why?
Many thanks for your help
There's a bit of a hacky way to do it that we wrote up before:
You'll need to create a custom apollo link (Apollo’s equivalent of middleware) to intercept the response data as it’s returned from the server, but before it’s inserted into the cache and the components re-rendered.
Here's an example were we pull metrics data from the extensions in our API:
import { ApolloClient, InMemoryCache, HttpLink, ApolloLink } from 'apollo-boost'
const link = new HttpLink({
uri: 'https://serve.onegraph.com/dynamic?show_metrics=true&app_id=<app_id>',
})
const metricsWatchers = {}
let id = 0
export function addMetricsWatcher(f) {
const watcherId = (id++).toString(36)
metricsWatchers[watcherId] = f
return () => {
delete metricsWatchers[watcherId]
}
}
function runWatchers(requestMetrics) {
for (const watcherId of Object.keys(metricsWatchers)) {
try {
metricsWatchers[watcherId](requestMetrics)
} catch (e) {
console.error('error running metrics watcher', e)
}
}
}
// We intercept the response, extract our extensions, mutatively store them,
// then forward the response to the next link
const trackMetrics = new ApolloLink((operation, forward) => {
return forward(operation).map(response => {
runWatchers(
response
? response.extensions
? response.extensions.metrics
: null
: null
)
return response
})
})
function create(initialState) {
return new ApolloClient({
link: trackMetrics.concat(link),
cache: new InMemoryCache().restore(initialState || {}),
})
}
const apolloClient = create(initialState);
Then to use the result in our React components:
import { addMetricsWatcher } from '../integration/apolloClient'
const Page = () => {
const [requestMetrics, updateRequestMetrics] = useState(null)
useEffect(() => {
return addMetricsWatcher(requestMetrics =>
updateRequestMetrics(requestMetrics)
)
})
// Metrics from extensions are available now
return null;
}
Then use a bit of mutable state to track each request and its result, and the use that state to render the metrics inside the app.
Depending on how you're looking to use the extensions data, this may or may not work for you. The implementation is non-deterministic, and can have some slight race conditions between the data that’s rendered and the data that you've extracted from the extensions.
In our case, we store performance metrics data in the extensions - very useful, but ancillary - so we felt the tradeoff was acceptable.
There's also an open issue on the Apollo client repo tracking this feature request
I dont have any idea of ApolloClient but i tried to run your query in shopify graphql app. It return results with extensions. Please find screenshot below. Also You can put questions in ApolloClient github.

Mocking apollo graphql client

I have written the following node client which interacts with the graphql server and using apollo-tools node module. I was not able to find any mock test for the below in node. Pls let me know is there a way to mock the below piece of code.
const batchIdFetchClient = new ApolloClient({
uri: `http://localhost:3435`,
fetch,
})
await batchfetchclient.query({
query: gql`
query batcidId($batchid: String!) {
batchIds(batchid: $batchid){
batchrenewedId
}
}
`,
variables: {
batchid: 'exdsfsdfsfdid1234', // As of now hardcoded
},
})
.then(data => {
logger.info('BatchId Database Successful Response =>' + JSON.stringify(data))
})
.catch(error => {
logger.error('BatchId Database Error Response =>' + error)
})
Maybe you can try using easygraphql-tester, it'll be something like this:
You need to pass your schema in order to mock it
const EasyGraphQLTester = require('easygraphql-tester')
const tester = new EasyGraphQLTester(schema)
const query = gql`
query batcidId($batchid: String!) {
batchIds(batchid: $batchid){
batchrenewedId
}
}
`
const mock = tester.mock({
query,
variables: {
batchid: 'exdsfsdfsfdid1234', // As of now hardcoded
}
})
console.log(mock)
Also, you can set fixtures if you want to have a specific data.

Resources