empty response body in HTTP response in github actions - node.js

Question

I'm trying to create a github action which requires sending an http request to https://www.instagram.com/<username>/?__a=1.
When I'm running it locally, it runs perfectly fine and gives me the number of followers.
But when I use it in github actions, it isn't able to parse the JSON string as the response is null
Here is a link to the github action file https://github.com/ashawe/actions-check/blob/e80ca115544979cdb3180207b99c7724e4446849/index.js
Here is the code to get the followers ( starts at line #94 )
promiseArray.push(new Promise((resolve, reject) => {
const url = 'https://www.instagram.com/' + INSTAGRAM_USERNAME + '/?__a=1';
core.info("url is");
core.info(url);
http.get(url, (response) => {
let chunks_of_data = [];
response.on('data', (fragments) => {
chunks_of_data.push(fragments);
});
response.on('end', () => {
let response_body = Buffer.concat(chunks_of_data);
core.info(response_body.toString());
let responseJSON = JSON.parse(response_body.toString());
resolve((responseJSON.graphql.user.edge_followed_by.count).toString());
});
response.on('error', (error) => {
reject(error);
});
});
}));
and then I'm processing it like:
Promise.allSettled(promiseArray).then((results) => {
results.forEach((result, index) => {
if (result.status === 'fulfilled') {
// Succeeded
// core.info(runnerNameArray[index] + ' runner succeeded. Post count: ' + result.value.length);
// postsArray.push(result.value);
instagram_followers = result.value;
} else {
jobFailFlag = true;
// Rejected
//core.error(runnerNameArray[index] + ' runner failed, please verify the configuration. Error:');
core.error(result.reason);
}
});
}).finally(() => {
try {
const followers = instagram_followers;
const readmeData = fs.readFileSync(README_FILE_PATH, 'utf8');
// core.info(readmeData);
const shieldURL = "https://img.shields.io/badge/ %40 " + INSTAGRAM_USERNAME + "-" + followers + "-%23E4405F?style=for-the-badge&logo=instagram";
const instagramBadge = "<img align='left' alt='instagram-followers' src='" + shieldURL + "' />";
const newReadme = buildReadme(readmeData, instagramBadge);
// core.info(newReadme);
// if there's change in readme file update it
if (newReadme !== readmeData) {
core.info('Writing to ' + README_FILE_PATH);
fs.writeFileSync(README_FILE_PATH, newReadme);
if (!process.env.TEST_MODE) {
// noinspection JSIgnoredPromiseFromCall
commitReadme();
}
} else {
core.info('No change detected, skipping');
process.exit(0);
}
} catch (e) {
core.error(e);
process.exit(1);
}
});
But when I run the action, it gives this error:
which means that the response_body isn't complete JSON response but a request to https://www.instagram.com/USERNAME/?__a=1 does send a json response.

Answer 1

UPDATE
Basically every time you hit that endpoint it returns the login html page, which causes the json parse to fail. It appears that you may need to use the api which requires you to authenticate before getting info from users. Or figure out other scraping methodologies.
I was able to recreate this failure in my local pc by jumping into a vpn and private browser. When I hit the endpoint it took me to the login screen. And when i hit the endpoint through curl in terminal it returned nothing. But when i got off the vpn, all worked fine. I think the reason it worked in your local is because there's some caching happening in the browser and you're probs not in a vpn. I am thinking there's some network blacklisting happening when on vpn. I don't know the github hosted network so I would recommend opening a ticket with them if you want to learn more about that.
Here are the instagram api docs for quick reference
https://developers.facebook.com/docs/instagram-basic-display-api/getting-started
Previews Response: Leaving here for other users future reference.
You are not passing username so it's trying to query the endpoint with empty username
Instead of running just node index.js in your action, you need to call your action and provide it with the parameters that it needs
- name: Your github action
uses: ./ # Uses an action in the root directory
with:
username: '_meroware'
Then your code will pick it put properly
const INSTAGRAM_USERNAME = core.getInput('username');
const url = 'https://www.instagram.com/' + INSTAGRAM_USERNAME + '/?__a=1';
Resources:
https://docs.github.com/en/actions/creating-actions/creating-a-javascript-action