First of all. Thank you to anybody who can help me hear. I am a python beginner but having searched high and low I cannot seem to find the answer, so I am reluctantly posting for help. I am trying to build an API request in Python. I have followed a tutorial and I can make it work using a fixed value however I need to switch the "metrics" out for my own which is a list around 200 long and not sequential.
A working example is as follows:
body = {
'rsid': 'vrs_xxx_abgglobalvrs',
'globalFilters': [
{
'type': 'dateRange',
'dateRange': '2022-01-05T00:00:00.000/2022-02-04T00:00:00.000'
}
],
'metricContainer': {
'metrics': [{
"columnId": "0",
"id": "metrics/event13"
},
{
"columnId": "1",
"id": "metrics/event23"
},
{
'columnId': '2',
'id': 'metrics/event149'
}
]
},
'dimension': 'variables/daterangeday',
'settings': {
'countRepeatInstances': 'true',
'limit': 50,
'page': 0,
'dimensionSort': 'asc'
}
}
If you print this the results show as the following:
....0/2022-02-04T00:00:00.000'}], 'metricContainer': {'metrics': [{'columnId': '0', 'id': 'metrics/event1'}, {'columnId': '1', 'id': 'metrics/event2'}, {'columnId': '2', 'id': 'metrics/event45'}]}, 'dimension': 'variabl...
However when I use create my dynamic code an update the body dictionary I get the an extra quote at the start and end of my dynamic value:
....0/2022-02-04T00:00:00.000'}], 'metricContainer': {'metrics': [**"**{'columnId':'0','id': 'metrics/event1'},{'columnId': '1','id':......
For reference this dynamic value (string) is created by using a list of events generated in the following way:
list_of_events = df['id'].tolist()
list_of_cleaned_event = []
metricstring = ""
columnId = 0
strcolumn = str(columnId)
for events in list_of_events[0:3]:
metric = str("{'columnId': '"+strcolumn+"','id': '"+events+"'},")
columnId += 1
strcolumn = str(columnId)
list_of_cleaned_event.append(metric)
for i in list_of_cleaned_event:
metricstring=metricstring+i
final = (metricstring[:-1])
and the body looks like this:
body = {
'rsid': 'vrs_avisbu0_abgglobalvrs',
'globalFilters': [
{
'type': 'dateRange',
'dateRange': '2022-01-05T00:00:00.000/2022-02-04T00:00:00.000'
}
],
'metricContainer': {
'metrics': [final]
},
'dimension': 'variables/daterangeday',
'settings': {
'countRepeatInstances': 'true',
'limit': 50,
'page': 0,
'dimensionSort': 'asc'
}
}
Try this...
final = []
list_of_events = ["metrics/event13", "metrics/event20", "metrics/event25"]
for columnId, events in enumerate(list_of_events[0:3]):
metric = {'columnID': str(columnId), 'id': events}
final.append(metric)
Define body as below
body = {
'rsid': 'vrs_avisbu0_abgglobalvrs',
'globalFilters': [
{
'type': 'dateRange',
'dateRange': '2022-01-05T00:00:00.000/2022-02-04T00:00:00.000'
}
],
'metricContainer': {
'metrics': final
},
'dimension': 'variables/daterangeday',
'settings': {
'countRepeatInstances': 'true',
'limit': 50,
'page': 0,
'dimensionSort': 'asc'
}
}
Related
I have several documents indexed on mongodb example :
{
id: randomId1,
field1:['param1', 'param2', 'param3'],
field2:['dparam1', 'dparam2', 'dparam3']
},
{
id: randomId2,
field1:['param0', 'param12', 'param3'],
field2:['dparam1', 'dparam2', 'dparam5']
},
{
id: randomId3,
field1:['param1', 'param3', 'param5'],
field2:['dparam0', 'dparam2', 'dparam7']
}
I want to aggregate on them, and I want the score to be counted for each common params between my query and my documents while using a should, for now the only solution I found is to do something like this
[
{
'$search': {
'index': 'index',
'compound': {
'should': [
{
'text': {
'query': 'param1',
'path': 'field1',
'score': {
'constant': {
'value': 1
}
}
}
}, {
'text': {
'query': 'param2',
'path': 'field1',
'score': {
'constant': {
'value': 1
}
}
}
}
],
'minimumShouldMatch': 1
}
}
}, {
'$project': {
'score': {
'$meta': 'searchScore'
},
}
}, {
'$sort': {
'score': -1,
}
}
]
And so this way, for each occurrence of my query I have a higher score, my question is there another way to optimize this ? as to not create a should statement for each of my parameters, provided it's the same path ? The score should be calculated depending on the matching elements in the array
Using Nodejs, Express, Neo4j data and API data
Apologies if this is answered elsewhere, please point me in that direction.
On my server.js file, I have a set of Neo4j data: dbtabArr
[ { dbTitle: 'dbTitleOne',
tabTitle: 'tabTitleOne' },
{ dbTitle: 'dbTitleOne',
tabTitle: 'tabTitleTwo' },
{ dbTitle: 'dbTitleOne',
tabTitle: 'tabTitleThree' }]
And another set of API data: wikiArr
[ { id: 'abc123',
title: 'tabTitleOne TextIDontWant'},
{ id: 'def456',
title: 'tabTitleThree TextIDontWant'}]
NOTE that there is NO 'title: tabTitleTwo' in this wikiArr, sometimes there will not be a 1:1 counterpart in the dbtabArr.
I want to join the datasets and I understand I could do something like
const obj1 = dbtabArr;
const obj2 = wikiArr;
const mergedObject = {
...obj1,
...obj2
};
But that would just put the top data over the bottom data.
Ultimately, I would like the new data set to look like this: newDataArr
[ { id: 'abc123',
dbTitle: 'dbTitleOne',
tabTitle: 'tabTitleOne' },
{ id: NULL,
dbTitle: 'dbTitleOne',
tabTitle: 'tabTitleTwo' },
{ id: 'def456',
dbTitle: 'dbTitleOne',
tabTitle: 'tabTitleThree' }]
The only possible join attribute is the tabTitle & title properties seen in both Arrs, but the one title property has TextI DontWant.
And I am hoping to have the tabTitle: 'tabTitleTwo' with the id: NULL available in the new data set as that will show where any gaps are when rendered on my ejs page.
Here's what I've tried so far
To remove the unwanted text:
var wikiArr2= wikiArr.map(function(v) {
return v.toString().replace(/ TextI DontWant/g, '');
});
To merge the 2 arrays:
const map = new Map();
tabTitleArr.forEach(item => map.set(item.tabTitle, item));
wikiArr2.forEach(item => map.set(item.title, {...map.get(item.title), ...item}));
const mergedArr = Array.from(map.values());
The result is not what I was hoping for (using real returned data, the top object of the array is from dbtabArr and the bottom object (spelling out object) is from wikiArr):
[ { dbSpecId: Integer { low: 1234567, high: 0 },
dbTitle: 'dbTitle',
tabTitle: 'tabTitle' },
{ '0': '[',
'1': 'o',
'2': 'b',
'3': 'j',
'4': 'e',
'5': 'c',
'6': 't',
'7': ' ',
'8': 'O',
'9': 'b',
'10': 'j',
'11': 'e',
'12': 'c',
'13': 't',
'14': ']' } ]
......
The items I seem to need help with are:
How can I filter out the text I don't want from the title in data set wikiArr, whether before or after data is rendered?
How do I do what I am assuming is an If Else to match or compare the titles from the first and second data sets (or other comparison?)
Is there a better way to merge the 2 data sets into the 1 than using ... spread?
Apologies for the list, happy to parse out to separate items.
Many Thanks!
Having assumed that wikiArr.title starts with dbtabArr.tabTitle, I've generated the newDataArr this way:
const dbtabArr = [
{ dbTitle: 'dbTitleOne', tabTitle: 'tabTitleOne' },
{ dbTitle: 'dbTitleOne', tabTitle: 'tabTitleTwo' },
{ dbTitle: 'dbTitleOne', tabTitle: 'tabTitleThree' },
];
const wikiArr = [
{ id: 'abc123', title: 'tabTitleOne TextIDontWant' },
{ id: 'def456', title: 'tabTitleThree TextIDontWant' },
];
const newDataArr = dbtabArr
.reduce((acc, x) => {
const [wiki] = wikiArr.filter(w => w.title.startsWith(x.tabTitle));
// const id = wiki?.id ?? null; // Nodejs 16+
const id = wiki && wiki.id ? wiki.id : null;
return [...acc, { id, ...x }];
}, []);
console.log(newDataArr);
I have an automation to do various stuff using boto3.
def createInstances(self):
if args.executescript != 0:
user_data = open(args.executescript).read()
else:
user_data = ""
instances = self.ec2_resource.create_instances(
ImageId=args.imageid,
MinCount=1,
MaxCount=args.maxinstances,
InstanceType=args.imagetype,
KeyName="aws_key_pair",
TagSpecifications=[
{
'ResourceType': 'instance',
'Tags': [
{
'Key': 'Name',
'Value': 'instance_ami_1'
},
{
'Key': 'depertment',
'Value': 'qa'
},
{
'Key': 'work',
'Value': 'ccperformance'
},
{
'Key': "AMI",
'Value': "reniassance_jdk8_cc"
}
]
},
],
BlockDeviceMappings=[
{
'DeviceName': '/dev/sda1',
'Ebs': {
'VolumeSize': 100,
'VolumeType': 'standard'
}
}
],
NetworkInterfaces=[
{
'SubnetId': "subnet-0b55a42b7cfc0f3d3",
'DeviceIndex': 0,
'AssociatePublicIpAddress': True,
# 'Groups': [sg1, sg2]
}
],
UserData=user_data,
)
I am using UserData in create_instances. This will create some log file in the instance. I need to pull then to my local once this is done.
I need a way to tell my script that the instance is now up and running and the script in UserData has successfully ran so that I can scp my logs to my local and terminate the instance.
Hello All and thank you in advance for your help :)
Can someone help me understand how I can take the below code, which displays data for a specified playlist, and have it only show the artist and track names? I have been toying around with the API documentation for several hours and I have not been able to make heads or tales of it. Right now when it displays data it gives me a whole bunch of data in a jumbled mess. Also, note that I put dummy values in the client_id and Secret parts of this code.
from spotipy.oauth2 import SpotifyClientCredentials
import spotipy
import json
PlaylistExample = '37i9dQZEVXbMDoHDwVN2tF'
cid = '123'
secret = 'xyz'
auth_manager = SpotifyClientCredentials(client_id=cid, client_secret=secret)
sp = spotipy.Spotify(auth_manager=auth_manager)
playlist_id = 'spotify:user:spotifycharts:playlist:37i9dQZEVXbJiZcmkrIHGU'
results = sp.playlist(playlist_id)
print(json.dumps(results, indent=4))
Would something like this be useful?:
print("Song - Artist - Album\n")
for item in results['tracks']['items']:
print(
item['track']['name'] + ' - ' +
item['track']['artists'][0]['name'] + ' - ' +
item['track']['album']['name']
)
Your output will look similar to this:
Song - Artist - Album
ONLY - ZHU - ONLY
Bad - 2012 Remaster - Michael Jackson - Bad 25th Anniversary
Orion - Rodrigo y Gabriela - Rodrigo y Gabriela
Shape of You - Ed Sheeran - ÷ (Deluxe)
Alternatively, you could create your own structure based on the returned one by Spotify but just keeping what you need:
result_dict = {
'tracks': {
'items': [],
'limit': 100,
'next': None,
'offset': 0,
'previous': None,
'total': 16
},
'type': 'playlist',
'uri': '<playlist_uri>'
}
And your track structure that goes inside 'items' from above:
track_dict = {
'track': {
'album': {
'name': item['track']['album']['name'],
},
'artists': [{
'name': item['track']['artists'][0]['name'],
}],
'name': item['track']['name'],
}
}
Then iterate and insert one by one:
for item in results['tracks']['items']:
track_dict = {
'track': {
'album': {
'name': item['track']['album']['name'],
},
'artists': [{
'name': item['track']['artists'][0]['name'],
}],
'name': item['track']['name'],
}
}
# Append the track dict structure to your results dict structure
result_dict['tracks']['items'].append(track_dict)
Having this as a result when printing result_dict:
{
'tracks': {
'items': [{
'track': {
'album': {
'name': 'ONLY'
},
'artists': [{
'name': 'ZHU'
}],
'name': 'ONLY'
}
}, {
'track': {
'album': {
'name': 'Bad 25th Anniversary'
},
'artists': [{
'name': 'Michael Jackson'
}],
'name': 'Bad - 2012 Remaster'
}
}, {
'track': {
'album': {
'name': 'Rodrigo y Gabriela'
},
'artists': [{
'name': 'Rodrigo y Gabriela'
}],
'name': 'Orion'
}
}, {
'track': {
'album': {
'name': '÷ (Deluxe)'
},
'artists': [{
'name': 'Ed Sheeran'
}],
'name': 'Shape of You'
}
}],
'limit': 100,
'next': None,
'offset': 0,
'previous': None,
'total': 4
},
'type': 'playlist',
'uri': '<playlist_uri>'
}
I am trying to get the Used Disk Space (Percent) for my EC2 instance from Cloudwatch with the help of a lambda function. It returns no value.
And when I try to specify the Filesystem and Mountpath it shows an error -
Parameter validation failed:\nUnknown parameter in MetricDataQueries[0].MetricStat.Metric.Dimensions[0]: \"Filesystem\", must be one of: Name, Value",
"errorType": "ParamValidationError"
Here is the full code.
import boto3
import datetime
def lambda_handler(event, context):
client = boto3.client('cloudwatch')
response = client.get_metric_data(
MetricDataQueries=[
{
'Id': 'd1',
'MetricStat': {
'Metric': {
'Namespace': 'cloudwatch',
'MetricName': 'DiskSpaceUtilization',
'Dimensions': [
{
'Name': 'InstanceId',
'Value': '*****************',
'Filesystem': '/****/****'
},
]
},
'Period': 300,
'Stat': 'Maximum',
'Unit': 'Percent'
},
'ReturnData': True
},
],
StartTime=datetime.datetime.utcnow() - datetime.timedelta(seconds=600),
EndTime=datetime.datetime.utcnow(),
ScanBy='TimestampDescending',
MaxDatapoints=60
)
return response
I expect the output as DiskSpaceUtilization - x%.
But currently the output is
"MetricDataResults": [
{
"Id": "d1",
"Label": "DiskSpaceUtilization",
"Timestamps": [],
"Values": [],
"StatusCode": "Complete"
}
],
Filesystem is a separate dimension, change this:
'Dimensions': [
{
'Name': 'InstanceId',
'Value': '*****************',
'Filesystem': '/****/****'
},
]
to this:
'Dimensions': [
{
'Name': 'InstanceId',
'Value': '*****************'
},
{
'Name': 'Filesystem',
'Value': '/****/****'
}
]
and see what you get then (there could be other issues after you fix this one).