I am using alert-manager and Prometheus to monitor my stack.
When I configure sending mails via alert-manager it works fine but when I configure to send sms via webhook it fails.
I added debug flag to alert-manager to verify it is getting the alerts and indeed the alerts are coming but till no sms are sent.
I also checked the webhook separately and it works perfectly.
Config file is:
global:
resolve_timeout: '5m'
smtp_smarthost: 'smtp.office365.com:587'
smtp_from: 'no-reply#example.com'
smtp_auth_username: 'no-reply#example.com'
smtp_auth_password: 'xxxxx'
smtp_require_tls: true
route:
group_by: ['instance', 'severity']
group_wait: 30s
group_interval: 5m
repeat_interval: 3h
receiver: team-1
receivers:
- name: 'team-1'
webhook_configs:
- send_resolved: true
http_config: {}
#max_alerts: 0
url: 'https://rest.nexmo.com/sms/json?from=example&text=test&to=xxxxxxxxxxxx&api_key=xxxxxxx&api_secret=xxxxxxx'
email_configs:
- to: 'john.doe#example.com'
I tried putting only mails and it works , tried only sms and it oes not work.
What am I missing ?
Eventually , I wrote a sms 'proxy' that will get a simple url get invocation and internally will call nexmo.
It did not work any other way.
Related
I am using alertmanager configured to read from a timescale db shared with other Prometheus/alertmanager systems.
I would like to set/check alerts only for services including a specific tag, therefore wondering how could I configure prometheus to apply only for specific tags?
This is what currently I am using:
# Alertmanager configuration
alerting:
alertmanagers:
- static_configs:
- targets: ['localhost:9093']
remote_write:
- url: https://promscale.host:9201/write
remote_read:
- url: https://promscale.host:9201/read
read_recent: true
...
I found there is an option alert_relabel_configs but is unclear for me the usage of it.
Any ideas?
FYI, alert_relabel_configs are used for alert relabeling to alerts before they are sent to the Alertmanager
To use alert_relabel_configs below is the example to add a new tag on matching the relabel config set:
alert_relabel_configs:
- source_labels: [ log_level ]
regex: warn
target_label: severity
replacement: warn
Note: The alerts are only changed when sent to alertmanager. They are
not changed in the Prometheus UI.
To test the relabel config online you can use https://relabeler.promlabs.com/
If you are using Prometheus Operator configuring alert relabeling rules should be done in additionalAlertRelabelConfigs of PrometheusSpec, more details: https://github.com/prometheus-operator/prometheus-operator/issues/1805
I'm new to serverless and writing my first service. It is built for the AWS API gateway and node.js lambda functions. Consider this as my serverless.yaml file:
service: applicationCatalog
frameworkVersion: '2'
provider:
name: aws
runtime: nodejs12.x
functions:
listShirts:
handler: handler.listShirts
events:
- httpApi: GET /
createShirt:
handler: handler.createShirt
events:
- httpApi: POST /
getShirt:
handler: handler.getShirt
events:
- httpApi:
method: GET
path: "/{shirtId}"
request:
parameters:
paths:
shirtId: true
deleteShirt:
handler: handler.deleteShirt
events:
- httpApi:
method: DELETE
path: "/{shirtId}"
request:
parameters:
paths:
shirtId: true
resources: {}
The functions listShirts, createShirt, and getShirt all work as I expected, and deleteShirt works too when a ShirtId is passed. The issue is when I don't pass the ShirtId on a delete. Assuming my service url is "https://shirts.mywardrobeapi.com". I'd expect this request:
DELETE https://shirts.mywardrobeapi.com
to trigger an error response from the API gateway. Instead, the deleteShirt function is invoked. Of course I could handle this simple check inside the function, but I thought that's what the { "shirtId" : true } setting in the serverless.yaml file was for. How can I get this setting to treat shirtId as required and not invoke the function when it's not provided? If I can't, what is this purpose of this setting?
I would suggest using Middy and validator middleware for handling required parameters.
Yep, the disadvantage is that your lambda is triggered all the time. But also you obtain
the flexibility of keeping handling required params in the code
clear logging if the parameters were wrong
you may also validate outgoing responses
keeps you serverless.yaml cleaner
We prefer the middy more than precise configure of Gateway API.
I'm trying to setup Alert Manager in a simple setup where it would send one Slack notification for each notification in receives.
I've hoped to disable grouping by removing the group_by configuration.
The problem is, that when I send 2 alert one after the other, even though the Alert Manager shows the 2 alerts as 'Not Grouped' when I get Slack notifications, I get one message for the first alert, and then a second message, where the 2 alerts are grouped.
Here is the config.yml
route:
receiver: default-receiver
group_wait: 1s #30s
group_interval: 1s #5m
# repeat_interval: 10m
# group_by: [cluster, alertname]
receivers:
- name: default-receiver
slack_configs:
- channel: "#alerts-test"
Any ideas?
From the Prometheus documentation for configuration
You can use group_by: ['...'] in your Alert Manager as a solution.
However, this was introduced in v0.16. For More info, see this GitHub issue.
My simple spring-boot 2.1.5.RELEASE running on Azure Kubernetes Service, responsible for listening to ConfigMap changes - receives 'Force closing'.
spring-boot-starter-parent = 2.1.5.RELEASE
spring-cloud-dependencies = Greenwich.SR1
Relevant configuration snippet:
cloud:
kubernetes:
reload:
enabled: true
secrets:
enabled: false
after some time, AKS signals Exec Failure java.io.EOFException: null and the kubernetes-client tries to reconnect.
Eventually it succeeds with WebSocket successfully opened information, but within same second it also signals Force closing the watch io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager#59ec7020 and looks like it terminates the connect. No further updates on ConfigMap triggers any event :(
Permissions in general are set, as events are properly caught by the service for initial test runs:
- apiGroups: [""]
resources: ["services", "pods", "configmaps", "endpoints"]
verbs: ["get", "watch", "list"]
Does anyone come across similar problem, and could help me to narrow down potential root cause?
I installed Griffon 1.2 and the mail plugin.
griffon install-plugin mail
I created a test:
sendMail(mailhost: '192.168.0.19',
to: 'me#home.com',
from: 'me#home.com',
subject: 'Test Mail',
text:'''
Hello
World''')
But it does not use the mailhost param. It tries to use localhost:
javax.mail.MessagingException: Could not connect to SMTP host: localhost, port: 25
I verified the SMTP server via telnet commands and that works.
Line 70 from https://github.com/griffon/griffon-mail-plugin/blob/master/src/main/griffon/plugins/mail/MailSender.groovy shows that mailhost is taken into account, at least when it comes to setting properties.
However this sounds more of a bug report than an actual question. If so I'd recommend you to post the issue to the Griffon user list. If it's found to be a bug then a report should be made to http://jira.codehaus.org/browse/griffon