With the az cli you can use the az functionapp list method to get an array of all your Function Apps and their associated meta data (i.e. appServicePlanId, defaultHostName, lastModifiedTimeUtc...etc)
I've spent the last hour looking everywhere, including in the azure-sdk-for-js src but I can't seem to find the right NodeJS SDK to simply list all of my Azure function apps. I would like to use a Node SDK for this, NOT manually construct an HTTP request. Any ideas where this functionality lives?
To be super clear, I'm looking to do something like this:
await new AzureFunctionClient(credentials,subscriptionId).functionApps.listAll()
You can use the REST API to list functions
Listing functions
get
/subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.Web/sites/{functionapp}/functions?api-version=2015-08-01
Super annoying but it looks like for some reason there is not a NodeJS SDK for basic CRUD actions with Azure Functions (emphasis on the R in CRUD) to do things like simply list all of your function apps or functions. I had to write this by hand:
import { URLSearchParams } from 'url'
import * as Sentry from '#sentry/node'
import fetch from 'node-fetch'
import * as head from 'lodash/head'
import { ResourceManagementClient } from '#azure/arm-resources'
const functionapp = 'functionapp'
const getAadToken = async ({ clientId, clientSecret, tenantId }) => {
const body = new URLSearchParams()
body.append('grant_type', 'client_credentials')
body.append('resource', 'https://management.azure.com/')
body.append('client_id', clientId)
body.append('client_secret', clientSecret)
return fetch(`https://login.microsoftonline.com/${tenantId}/oauth2/token`, {
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
},
body,
})
}
const listResourceGroupsForSubscription = async ({
returnIds = false,
credentials,
subscriptionId,
}) => {
const resourceGroups = []
const client = new ResourceManagementClient(credentials, subscriptionId)
const getAllResourceGroupsData = async (nextLinkToken = '') => {
let rsgs
let nextLink
/**
* Then this is the nth time this has been called
*/
try {
if (nextLinkToken) {
rsgs = await client.resourceGroups.listNext(nextLinkToken)
nextLink = rsgs.nextLink
} else {
/**
* Then this is the first time this has been called
*/
rsgs = await client.resourceGroups.list()
nextLink = rsgs.nextLink
}
} catch (e) {
logger.error(e)
Sentry.captureException(e)
}
/**
* Add the rsgs to our resourceGroups array so we can aggregate as we go
*/
resourceGroups.push(...rsgs)
if (nextLink) {
/**
* If there is another page of data, get it
*/
await getAllResourceGroupsData(nextLink)
}
return rsgs
}
await Promise.all(await getAllResourceGroupsData())
return returnIds ? resourceGroups.map(({ id }) => id) : resourceGroups
}
const createUriForMsRestApiRequest = ({ id }) =>
[`https://management.azure.com`, id, '?api-version=2020-06-01'].join('')
const getDataFromMsRestApi = async ({
authToken,
initialUrl,
kindSelector = '',
}) => {
const allData = []
const getAllData = async (nextLinkToken = '') => {
try {
const url = nextLinkToken || initialUrl
const response = await fetch(url, {
headers: { Authorization: `Bearer ${authToken}` },
})
const { value: data, nextLink } = await response.json()
allData.push(
...(kindSelector
? data.filter(({ kind }) => kind.includes(functionapp))
: data)
)
if (nextLink) {
logger.info(lt.fetchingMoreRestApiData)
await getAllData(nextLink)
}
logger.info(lt.fetchedDataFromRestApi(url))
return data
} catch (e) {
logger.error(e)
Sentry.captureException(e)
}
}
await Promise.all(await getAllData())
return allData
}
const getAzureFunctionAppsData = async ({
credentials,
subscriptionId,
clientId,
clientSecret,
tenantId,
}) => {
/**
* In order to get a list of all the function apps and functions we need to create a uri that looks like this:
* /subscriptions/${subscriptionId}/resourceGroups/${resourceGroupId}/providers/Microsoft.Web/sites/${functionAppId}/functions
* And since Azure does not provide this functionaliy in any of their SDKS, we have to fetch a lot of data ourselves...
*/
try {
/**
* Step 1, get all the resource groups (we just need the ids)
*/
const resourceGroupsIds = await listResourceGroupsForSubscription({
returnIds: true,
credentials,
subscriptionId,
})
/**
* Step 2, Grab the bearer token for auth with the REST API
*/
const response = await getAadToken({
clientId,
clientSecret,
tenantId,
})
const { access_token: authToken } = await response.json()
/**
* Step 3, for each resource group get all the sites grab the Function Apps
*/
const functionApps = (
await Promise.all(
(resourceGroupsIds || []).map(
async resourceGroupId =>
await getDataFromMsRestApi({
authToken,
kindSelector: functionapp,
initialUrl: createUriForMsRestApiRequest({
id: `${resourceGroupId}/providers/Microsoft.Web/sites`,
}),
})
)
)
).flat()
/**
* Step 4, now we can finally grab the actual functions for each function app
*/
const azureFunctions = (
await Promise.all(
(functionApps || []).map(
async ({ id: functionAppId }) =>
await getDataFromMsRestApi({
authToken,
initialUrl: createUriForMsRestApiRequest({
id: `${functionAppId}/functions`,
}),
})
)
)
).flat()
return (functionApps || []).map(({ name, ...data }) => ({
name,
...data,
functions: azureFunctions.filter(
({ name: funcName }) => head(funcName.split('/')) === name
),
}))
} catch (e) {
logger.error(e)
Sentry.captureException(e)
}
}
Related
My goal is to try and get a Node app (written in Typescript) to buy bitcoin using my "GBP Wallet" (fiat account). I am currently trying this via using the Axios library, rather than any unofficial Coinbase client libraries out there. I am successfully able to get data that I need from the GET endpoints of the Coinbase API. However, with the following POST endpoint I get an error:
POST https://api.coinbase.com/v2/accounts/:account_id/buys
The error:
[ { id: 'invalid_request', message: "Can't buy with this account" } ]
Here is a simplified example of the code I am trying to run at the moment:
import * as crypto from 'crypto';
import axios, { AxiosError, AxiosRequestConfig, AxiosResponse, Method } from 'axios';
import * as dotenv from 'dotenv';
dotenv.config();
const timer = (ms: number) => new Promise(res => setTimeout(res, ms));
function isNumeric(str: string): boolean {
const noSpacesStr = str.replace(/\s/g, '');
return !isNaN(Number(noSpacesStr)) && noSpacesStr.length !== 0;
}
async function coinbaseApiRequest(url: string, method: Method, data?: any): Promise<AxiosResponse> {
if (process.env.COINBASE_API_KEY === undefined || process.env.COINBASE_API_SECRET === undefined) {
throw new Error('Missing credentials');
}
const stringData = JSON.stringify(data);
const timestamp = Math.floor(Date.now() / 1000);
const message = data === undefined ? timestamp.toString() + method + url : timestamp.toString() + method + url + stringData;
const signature = crypto.createHmac("sha256", process.env.COINBASE_API_SECRET).update(message).digest("hex");
const config: AxiosRequestConfig = {
method,
url: `https://api.coinbase.com${url}`,
headers: {
'CB-ACCESS-SIGN': signature,
'CB-ACCESS-TIMESTAMP': timestamp,
'CB-ACCESS-KEY': process.env.COINBASE_API_KEY,
'CB-VERSION': '2021-05-19',
'Content-Type': 'application/json'
},
data
};
return axios(config);
}
async function getBuyPrice(): Promise<AxiosResponse> {
return coinbaseApiRequest('/v2/prices/BTC-GBP/buy', 'GET');
}
async function getSellPrice(): Promise<AxiosResponse> {
return coinbaseApiRequest('/v2/prices/BTC-GBP/sell', 'GET');
}
async function getGBPAccount(): Promise<AxiosResponse> {
return coinbaseApiRequest('/v2/accounts/GBP', 'GET');
}
async function getBitcoinAccount(): Promise<AxiosResponse> {
return coinbaseApiRequest('/v2/accounts/BTC', 'GET');
}
async function getGBPWalletPaymentMethodID(): Promise<string> {
const paymentMethods = await coinbaseApiRequest('/v2/payment-methods', 'GET');
for (let i = 0; i < paymentMethods.data.data.length; i++) {
const paymentMethod = paymentMethods.data.data[i];
if (paymentMethod.name === 'GBP Wallet' && paymentMethod.type === 'fiat_account') {
return paymentMethod.id;
}
}
throw new Error('GBP wallet has not been found');
}
async function postBuyBitcoin(accountId: string, amount: string, paymentMethod: string): Promise<AxiosResponse> {
if (!isNumeric(amount)) { throw new Error('amount arg is not a valid number'); }
const body = { amount, currency: 'BTC', payment_method: paymentMethod };
return coinbaseApiRequest(`/v2/accounts/${accountId}/buys`, 'POST', body);
}
async function run(): Promise<void> {
try {
const oldGBPAccRes = await getGBPAccount();
const oldBTCAccRes = await getBitcoinAccount();
const paymentMethodID = await getGBPWalletPaymentMethodID();
const buyRes = await postBuyBitcoin(oldGBPAccRes.data.data.id, '0.00001', paymentMethodID);
const newGBPAccRes = await getGBPAccount();
const newBTCAccRes = await getBitcoinAccount();
console.log(`old GBP account response:`);
console.log(oldGBPAccRes.data);
console.log(`old BTC account response:`);
console.log(oldBTCAccRes.data);
console.log(`bitcoin buy response:`);
console.log(buyRes);
console.log(`new GBP account response:`);
console.log(newGBPAccRes.data);
console.log(`new BTC account response:`);
console.log(newBTCAccRes.data);
console.log('Finished payment');
} catch (error: any | AxiosError) {
if (axios.isAxiosError(error)) {
console.log(error.response?.data.errors);
}
}
}
run();
All the GET request functions seem to work fine and return the responses correctly when I run them individually. The only issue I have is with the POST request function. I get the following error logged when running the code above:
[ { id: 'invalid_request', message: "Can't buy with this account" } ]
Not sure why this is happening, and there does not seem to be much on this online relevant to node based implementations, but this seems to be somewhat related. would appreciate some guidance. Thanks in advance.
I'm creating a database in Firestore and the code I did is this:
import * as functions from 'firebase-functions'
import { google } from 'googleapis'
import { initializeApp } from 'firebase-admin/app'
const serviceAccount = require('../sheets_updater_service_account.json')
const sheets = google.sheets('v4')
import { getFirestore } from "firebase-admin/firestore"
initializeApp()
const firestore = getFirestore()
exports.processosjudiciais = functions.https.onRequest(async (request, response) => {
const jwtClient = new google.auth.JWT({
email: serviceAccount.client_email,
key: serviceAccount.private_key,
scopes: ['https://www.googleapis.com/auth/spreadsheets']
})
await jwtClient.authorize()
const { data } = await sheets.spreadsheets.values.get({
auth: jwtClient,
spreadsheetId: 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
range: `Processos judiciais!A11664:E11667`
})
data.values?.forEach(row => {
const [processoBE, autoresBE, documentosDosautores, arbitramentoDeHonoráriosBE, valorDaCausa] = row
firestore.collection("Processos judiciais").doc(processoBE).set({
processoBE, autoresBE, documentosDosautores, valorDaCausa
})
firestore.collection("Processos judiciais").doc(processoBE).collection('Arbitramento de Honorários - Base de Execução').doc(arbitramentoDeHonoráriosBE).set({
arbitramentoDeHonoráriosBE, processoBE
})
})
})
The structure of my code in firestore looks like this:
Instead of having a document that displays multiple values of "ArbitramentoDeHonoráriosBE", I wanted each value of "arbitramentoDeHonoráriosBE" to be a different document, containing both ArbitrationDeHonoráriosBE and "processBE".
I separated an image with colors to facilitate the visualization of the idea.
Does anyone know how I can separate the values that are inside the same cell (google sheets) and create a different document in the firestore?
It seems arbitramentoDeHonoráriosBE is a string and the value are separated by a ;. If you want to create a document for each value then you can try this:
const updatePromises = [];
data.values?.forEach(row => {
const [arbitramentoDeHonoráriosBE, processoBE] = row
// Get an array of those values separated by ';'
arbitramentoDeHonoráriosBE.split(";").forEach(v => {
// Push the set() promises to updatePromises array
updatePromises.push(
firestore.collection("Processos judiciais").doc(processoBE).collection('Arbitramento de Honorários - Base de Execução').doc(arbitramentoDeHonoráriosBE).set({
arbitramentoDeHonoráriosBE: v,
processoBE
// add required field in document
})
)
})
})
// run all promises
return await Promise.all(updatePromises)
Alternatively you can also use batched writes to write documents.
Using #Dharmaraj's suggestion, I was able to find the answer with this code.
import * as functions from 'firebase-functions'
import { google } from 'googleapis'
import { initializeApp } from 'firebase-admin/app'
const serviceAccount = require('../sheets_updater_service_account.json')
const sheets = google.sheets('v4')
import { getFirestore } from "firebase-admin/firestore"
initializeApp()
const firestore = getFirestore()
exports.processosjudiciais = functions.https.onRequest(async (request, response): Promise<any> => {
const jwtClient = new google.auth.JWT({
email: serviceAccount.client_email,
key: serviceAccount.private_key,
scopes: ['https://www.googleapis.com/auth/spreadsheets']
})
await jwtClient.authorize()
const { data } = await sheets.spreadsheets.values.get({
auth: jwtClient,
spreadsheetId: 'XXXXXXXXXXXXXXXXXXXXXXXXXXXX',
range: `Processos judiciais!A11664:E11667`
})
const updatePromises = new Array()
data.values?.forEach(row => {
const [processoBE, autoresBE, documentosDosautores, arbitramentoDeHonoráriosBE, valorDaCausa] = row
firestore.collection("Processos judiciais").doc(processoBE).set({
processoBE, autoresBE, documentosDosautores, valorDaCausa
})
arbitramentoDeHonoráriosBE.split("; ").forEach((v: string) => {
updatePromises.push(
firestore.collection("Processos judiciais").doc(processoBE).collection('fee-arbitrations - Base de Execução').doc(v).set({
arbitramentoDeHonoráriosBE: v,
processoBE,
})
)
})
})
return await Promise.all(updatePromises)
})
I can't figure it out, the answer comes in the network table but when I want to console.log it, this will display undefined. Do you have any idea why? I attach the pictures and the code.
Here is a image with my codes and response
Here is the code - first one is where I send the response. As I said, it's going well on network tab, I get a 200 status.
export const getAccountStatus = async (req, res) => {
const user = await User.findById(req.user._id).exec();
const account = await stripe.accounts.retrieve(user.stripe_account_id);
// console.log("user account retrieve", account);
const updatedUser = await User.findByIdAndUpdate(
user._id,
{
stripe_seller: account
},
{ new: true }
)
.select("-password")
.exec();
console.log(updatedUser);
res.send(updatedUser);
};
Here is the page where i want to console.log it:
const StripeCallback = ({ history }) => {
const { auth } = useSelector(state => ({ ...state }));
const dispatch = useDispatch();
useEffect(() => {
if (auth && auth.token) accountStatus();
}, [auth]);
const accountStatus = async () => {
try {
const res = await getAccountStatus(auth.token);
console.log(res);
} catch (err) {
console.log(err);
}
};
return <div>test</div>;
};
Ang here is the Axios.post (which is working well as I know):
export const getAccountStatus = async token => {
await axios.post(
`${process.env.REACT_APP_API}/get-account-status`,
{},
{
headers: {
Authorization: `Bearer ${token}`
}
}
);
};
Thank you!
getAccountStatus doesn't have a return statement, so res in const res = await getAccountStatus(auth.token); will always be undefined.
export const getAccountStatus = async token => {
return axios.post( // <----- added return
`${process.env.REACT_APP_API}/get-account-status`,
{},
{
headers: {
Authorization: `Bearer ${token}`
}
}
);
};
I'm using a custom service account (using --service-account parameter in the deploy command). That service account has domain-wide delegation enabled and it's installed in the G Apps Admin panel.
I tried this code:
app.get('/test', async (req, res) => {
const auth = new google.auth.GoogleAuth()
const gmailClient = google.gmail({ version: 'v1' })
const { data } = await gmailClient.users.labels.list({ auth, userId: 'user#domain.com' })
return res.json(data).end()
})
It works if I run it on my machine (having the GOOGLE_APPLICATION_CREDENTIALS env var setted to the path of the same service account that is assigned to the Cloud Run service) but when it's running in Cloud Run, I get this response:
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Bad Request",
"reason" : "failedPrecondition"
} ],
"message" : "Bad Request"
}
I saw this solution for this same issue, but it's for Python and I don't know how to replicate that behaviour with the Node library.
After some days of research, I finally got a working solution (porting the Python implementation):
async function getGoogleCredentials(subject: string, scopes: string[]): Promise<JWT | OAuth2Client> {
const auth = new google.auth.GoogleAuth({
scopes: ['https://www.googleapis.com/auth/cloud-platform'],
})
const authClient = await auth.getClient()
if (authClient instanceof JWT) {
return (await new google.auth.GoogleAuth({ scopes, clientOptions: { subject } }).getClient()) as JWT
} else if (authClient instanceof Compute) {
const serviceAccountEmail = (await auth.getCredentials()).client_email
const unpaddedB64encode = (input: string) =>
Buffer.from(input)
.toString('base64')
.replace(/=*$/, '')
const now = Math.floor(new Date().getTime() / 1000)
const expiry = now + 3600
const payload = JSON.stringify({
aud: 'https://accounts.google.com/o/oauth2/token',
exp: expiry,
iat: now,
iss: serviceAccountEmail,
scope: scopes.join(' '),
sub: subject,
})
const header = JSON.stringify({
alg: 'RS256',
typ: 'JWT',
})
const iamPayload = `${unpaddedB64encode(header)}.${unpaddedB64encode(payload)}`
const iam = google.iam('v1')
const { data } = await iam.projects.serviceAccounts.signBlob({
auth: authClient,
name: `projects/-/serviceAccounts/${serviceAccountEmail}`,
requestBody: {
bytesToSign: unpaddedB64encode(iamPayload),
},
})
const assertion = `${iamPayload}.${data.signature!.replace(/=*$/, '')}`
const headers = { 'content-type': 'application/x-www-form-urlencoded' }
const body = querystring.encode({ assertion, grant_type: 'urn:ietf:params:oauth:grant-type:jwt-bearer' })
const response = await fetch('https://accounts.google.com/o/oauth2/token', { method: 'POST', headers, body }).then(r => r.json())
const newCredentials = new OAuth2Client()
newCredentials.setCredentials({ access_token: response.access_token })
return newCredentials
} else {
throw new Error('Unexpected authentication type')
}
}
What you can do here is define ENV variables in your yaml file as described in this documentation to set the GOOGLE_APPLICATION_CREDENTIALS to the path of the JSON key.
Then use a code such as the one mentioned here.
const authCloudExplicit = async ({projectId, keyFilename}) => {
// [START auth_cloud_explicit]
// Imports the Google Cloud client library.
const {Storage} = require('#google-cloud/storage');
// Instantiates a client. Explicitly use service account credentials by
// specifying the private key file. All clients in google-cloud-node have this
// helper, see https://github.com/GoogleCloudPlatform/google-cloud-node/blob/master/docs/authentication.md
// const projectId = 'project-id'
// const keyFilename = '/path/to/keyfile.json'
const storage = new Storage({projectId, keyFilename});
// Makes an authenticated API request.
try {
const [buckets] = await storage.getBuckets();
console.log('Buckets:');
buckets.forEach(bucket => {
console.log(bucket.name);
});
} catch (err) {
console.error('ERROR:', err);
}
// [END auth_cloud_explicit]
};
Or follow an approach similar to the one mentioned here.
'use strict';
const {auth, Compute} = require('google-auth-library');
async function main() {
const client = new Compute({
serviceAccountEmail: 'some-service-account#example.com',
});
const projectId = await auth.getProjectId();
const url = `https://dns.googleapis.com/dns/v1/projects/${projectId}`;
const res = await client.request({url});
console.log(res.data);
}
main().catch(console.error);
I'm trying to create a scheduled cloud function which updates a document field with a value from an api call. However I get the error mentioned above in the logs, altought I'm returning a promise (as far as I know)
Any help is appretiated
the api call returns this json:
{ "alarm": "false" }
My Code:
import * as functions from 'firebase-functions';
import * as admin from 'firebase-admin';
import * as rp from 'request-promise';
admin.initializeApp()
export const scheduledAlarmUpdate = functions.pubsub.schedule('every 30 minutes').onRun((context) => {
const docRef = admin.firestore().collection('alarms').doc('stuttgart');
const username = 'bitfactory';
const pass = '...';
const options = {
url: 'https://api.bitfactory.io/fineparticlesalarm/',
auth: {
user: username,
password: pass
},
method: 'GET',
json: true
}
return rp(options).then(data => {
console.log(data.alarm)
docRef.set({
alarm: data.alarm
})
.catch(err => {
console.log(err);
});
})
});
You need to return the promise returned by the set() method, as follows:
export const scheduledAlarmUpdate = functions.pubsub
.schedule('every 30 minutes')
.onRun(context => {
const docRef = admin
.firestore()
.collection('alarms')
.doc('stuttgart');
const username = 'bitfactory';
const pass = '...';
const options = {
url: 'https://api.bitfactory.io/fineparticlesalarm/',
auth: {
user: username,
password: pass
},
method: 'GET',
json: true
};
return rp(options)
.then(data => {
console.log(data.alarm);
return docRef.set({ // ->>> here return
alarm: data.alarm
});
})
.catch(err => {
console.log(err);
});
});