Spotify Application artists data scraping - spotify

i would like to get all artists information like rank,active listeners,followers and increment today details. Is there any possiblity tool to get scraped whenever i want.

The Spotify API has a Get an Artist endpoint that gives you access to things like popularity, images, followers, genres, etc. It is documented here: https://developer.spotify.com/web-api/get-artist/
A response may look like this:
{
"external_urls" : {
"spotify" : "https://open.spotify.com/artist/0OdUWJ0sBjDrqHygGUXeCF"
},
"followers" : {
"href" : null,
"total" : 306565
},
"genres" : [ "indie folk", "indie pop" ],
"href" : "https://api.spotify.com/v1/artists/0OdUWJ0sBjDrqHygGUXeCF",
"id" : "0OdUWJ0sBjDrqHygGUXeCF",
"images" : [ {
"height" : 816,
"url" : "https://i.scdn.co/image/eb266625dab075341e8c4378a177a27370f91903",
"width" : 1000
}, {
"height" : 522,
"url" : "https://i.scdn.co/image/2f91c3cace3c5a6a48f3d0e2fd21364d4911b332",
"width" : 640
}, {
"height" : 163,
"url" : "https://i.scdn.co/image/2efc93d7ee88435116093274980f04ebceb7b527",
"width" : 200
}, {
"height" : 52,
"url" : "https://i.scdn.co/image/4f25297750dfa4051195c36809a9049f6b841a23",
"width" : 64
} ],
"name" : "Band of Horses",
"popularity" : 59,
"type" : "artist",
"uri" : "spotify:artist:0OdUWJ0sBjDrqHygGUXeCF"
}
You can request additional features in the Web API issue tracker here: https://github.com/spotify/web-api/issues

Related

How to get sum of a particular field of a collection in MongoDB collection using PyMongo?

My MongoDB contains the following data
{
"_id" : ObjectId("5c1b742eb1829b69963029e8"),
"duration" : 12,
"cost" : 450,
"tax" : 81,
"tags" : [],
"participants" : [
ObjectId("5c1b6a8f348ddb15e4a8aac7"),
ObjectId("5c1b742eb1829b69963029e7")
],
"initiatorId" : ObjectId("5c1b6a8f348ddb15e4a8aac7"),
"context" : "coach",
"accountId" : ObjectId("5bdfe7b01cbf9460c9bb5d68"),
"status" : "over",
"webhook" : "http://d4bdc1ef.ngrok.io/api/v1/webhook_callback",
"hostId" : "5be002109a708109f862a03e",
"createdAt" : ISODate("2018-12-20T10:51:26.143Z"),
"updatedAt" : ISODate("2018-12-20T10:51:44.962Z"),
"__v" : 0,
"endedAt" : ISODate("2018-12-20T10:51:44.612Z"),
"startedAt" : ISODate("2018-12-20T10:51:32.992Z"),
"type" : "voip"
}
{
"_id" : ObjectId("5c1b7451b1829b69963029ea"),
"duration" : 1,
"cost" : 150,
"tax" : 27,
"tags" : [],
"participants" : [
ObjectId("5c1b6a8f348ddb15e4a8aac7"),
ObjectId("5c1b7451b1829b69963029e9")
],
"initiatorId" : ObjectId("5c1b6a8f348ddb15e4a8aac7"),
"context" : "coach",
"accountId" : ObjectId("5bdfe7b01cbf9460c9bb5d68"),
"status" : "over",
"webhook" : "http://d4bdc1ef.ngrok.io/api/v1/webhook_callback",
"hostId" : "5be002109a708109f862a03e",
"createdAt" : ISODate("2018-12-20T10:52:01.560Z"),
"updatedAt" : ISODate("2018-12-20T10:52:08.018Z"),
"__v" : 0,
"endedAt" : ISODate("2018-12-20T10:52:07.667Z"),
"startedAt" : ISODate("2018-12-20T10:52:06.762Z"),
"type" : "voip"
}
I want to get the total duration (sum of duration field) for a particular accountID where status is equals to "over" for a particular date range. Anyway to accomplish this using PyMongo? I am unable to form the query
Well I was doing some pretty basic mistakes while converting the query to PyMongo aggregation function. All I would say is be careful with the query structure format and especially the keys are to be encapsulated within quotes(""). To solve this all I have to do was
from bson.objectid import ObjectId
pipe = [
{"$match": {"accountId": ObjectId(accountId),
"status": "over",
"startedAt": {"$gte": startDate,
"$lte": EndDate
}
}},
{"$project": {"readableDate":
{"$dateToString":
{"format": "%Y-%m-%d", "date": "$startedAt"}},
"accountId": str("$accountId"),
"duration": "$duration"
}},
{"$group": {"_id": {"date": "$readableDate",
"accountId": str("$accountId")}, "totalCallDuration": {"$sum": "$duration"}}}]
for doc in db.VoiceCall.aggregate(pipe):
print(doc)
Just a reminder : the startDate and EndDate are in Python datetime format.

mongodb descendent sorting by a numeric field getting sorted randomly (node.js)

I have a mongo database running in cmd.
There is a collection named usercollection.
And there is also some documents which I'm getting by typing in mongo shell:
db.usercollection.find().sort({score: -1}).pretty()
Output:
{
"_id" : ObjectId("57501b2d7ef17cd8c1bd63bc"),
"name" : "QWERTY",
"score" : "100",
"link" : "http://qwerty.com"
}
{
"_id" : ObjectId("5750199819433f80da88c847"),
"name" : "Adam Sandler",
"score" : "222",
"link" : "http://sandler.com"
}
{
"_id" : ObjectId("57501af57ef17cd8c1bd63bb"),
"name" : "gfhfghfghguy",
"score" : "5",
"link" : "http://dsad.com"
}
{
"_id" : ObjectId("57501bb47ef17cd8c1bd63bd"),
"name" : "Test name",
"score" : "50",
"link" : "http://fdsfds.cpm"
}
{
"_id" : ObjectId("574f4a9dab868fdc896d11b4"),
"name" : "Rich Man",
"score" : "500",
"link" : "http://jikl.me"
}
{
"_id" : ObjectId("574f50f1fbd256888ffa248d"),
"name" : "lfgfgff",
"score" : "9",
"link" : "http://oh.com"
}
{
"_id" : ObjectId("574f5324fbd256888ffa248e"),
"name" : "test name",
"score" : "9",
"link" : "http://dsad.com"
}
The problem is that I'm trying to sort these documents from one with highest score to one with lowest score and it's output is completely random(
I'm displaying that documents on the website using node.js and the result is the same as in the console.
Try with :
db.usercollection.find().sort({score: 'desc'}).pretty()

how to find recent record of record matching particular criteria

{
"_id" : ObjectId("5514ecc73910d3e808b9417c"),
"endingReciptBookNumber" : 2999,
"startingReciptBookNumber" : 2900,
"User" : 8,
"allRecipt" : [
{
"recipt_Number" : 2999,
"amount" : 24124,
"_id" : ObjectId("5514ecc73910d3e808b94180")
},
{
"recipt_Number" : 100,
"amount" : 2414,
"_id" : ObjectId("5514ecc73910d3e808b9417f")
},
{
"recipt_Number" : 101,
"amount" : 242,
"_id" : ObjectId("5514ecc73910d3e808b9417e")
},
{
"recipt_Number" : 102,
"amount" : 2424,
"_id" : ObjectId("5514ecc73910d3e808b9417d")
}
],
"__v" : 0
}
I have many documents like this in a collection in mongoose .I want to find a latest entered recipt_Number for a particular user. like in this case it should give me 102 as answer.
i have also attached snippet of lines of code. Its also a way to get same result.
db.topics.find( {'User': 8}, { 'allRecipt': { $slice: -1 },'startingReciptBookNumber':0,'endingReciptBookNumber':0,'User':0,'_id':0,'__v':0 } )
query result like below
{
"allRecipt" : [
{
"recipt_Number" : 102,
"amount" : 2424,
"_id" : ObjectId("5514ecc73910d3e808b9417d")
}
]
}
Though query won't give any single number in result but it will give desired outcome through result.allRecipt.0.recipt_Number, Your desired number will always get into in 0 index. I think this is your desired number.
Here $slice make a difference.
Thanks

MongoDB remove the lowest score, node.js

I am trying to remove the lowest homework score.
I tried this,
var a = db.students.find({"scores.type":"homework"}, {"scores.$":1}).sort({"scores.score":1})
but how can I remove this set of data?
I have 200 pieces of similar data below.
{
"_id" : 148,
"name" : "Carli Belvins",
"scores" : [
{
"type" : "exam",
"score" : 84.4361816750119
},
{
"type" : "quiz",
"score" : 1.702113040528119
},
{
"type" : "homework",
"score" : 22.47397850465176
},
{
"type" : "homework",
"score" : 88.48032660881387
}
]
}
you are trying to remove an element but the statement you provided is just to find it.
Use db.students.remove(<query>) instead. Full documentation here

Display image from Mongodb with NodeJS

I have a collection "image" and "image_ids "in my mongoDB:
image:
{
"_id" : ObjectId("542e65368a1cec1227ae2bac"),
"reference" : "5424991"
}
images_ids:
{
"_id" : ObjectId("542e652995ac12ed3fecb85f"),
"image" : "D0000001.PNG",
"fid" : ObjectId("542e65368a1cec1227ae2bac"),
"reference" : "5424991"
}
How I can be able to display D0000001.PNG with NodeJS? I mean, to do the link with and API.
[EDIT] we have also
fs.chunks:
{
"_id" : ObjectId("542e684a8a1cec178a172673"),
"n" : 1,
"data" : BinData(0,"2N6DSSfbCN/LLacNDYrJUQDEZgimMUwFpQGoJP0RU19Bi4PM82DjrUnKhE/P9v7P8ZveD1oDpFBM0iml9NE3WQmAvYFoG+nhD73Jm4N9b4LpylaAN5Ef+gVdgGSTAfSUwOikXoVick5pSQCkRmTCU5NT9VVfjHdAx74/ZhFRj+TIRjzlAhzkACBElzgMwGCo7tX+FYrpQLJ5KRmXiwFFwsNtHHzXiK1eu+CG1FumhGpA/qdG8CdDgD1xUHEcerMGO/eLGR9ML7ni/VjXxWzqp2j5DG2/WuKNv7xd3Kz/vr0MctJhuaBIl35YrKhdLnzqDa0uDa6bm4jz+eNyAI2hQbayGo4kVPFe8W7wFpY7qfBvnB9kbocxfZSdADDUNwYaydpT8lIcKEN9XfQJOYZvHp0El"),
"files_id" : ObjectId("542e684a8a1cec178a172671")
}
fs.files:
{
"_id" : ObjectId("542e65378axdeckhb0"),
"uploadDate" : ISODate("2012-11-01"),
"length" : 15673,
"chunkSize" : 33222,
"md5" : "f66e6654854a28e3672cfhds334d223b55a1"
}
Thanks for your help !

Resources