I am facing issues with passing cron expression as parameter to stack. Seems parameter is getting evaluated to differently than expected. Below are details:
Reading cron expression from json file.
{ "CronSchedule":""cron(0 19 * * * *)"" }
Code snippet of my cloudformation stack.
ScheduledRule:
Type: AWS::Events::Rule
Properties:
Description: ScheduledRule
Name: !Sub '${Environment}-${Platform}-${AppName}-${Prefix}-SCHEDULE-CAPTURE'
ScheduleExpression: !Sub ${CronSchedule}
State: "ENABLED"
AWS command executing:
aws --region us-east-1 cloudformation deploy --template-file ./deployment/aws/cfn/cloudwatch-resources.yml --stack-name QA-DUMMYAPP-CLOUD-WATCH --parameter-overrides Owner=DUMMYAPP AppID=C4E7876D-1BF6-0090-868C-A8E82D4BCBCD BU=DJI AppName=DUMMYAPP Platform=T3 LambdaStackName=QA-DUMMYAPP-LAMBDA ApiGwResourcesStackName=QA-DUMMYAPP-API-GW-RESOURCES CloudWatchStackName=QA-DUMMYAPP-CLOUD-WATCH FeedStateMachineStackName=STACK_NAME Environment=QA CronSchedule="cron(0 19 * * * *)" --no-fail-on-empty-changeset
On AWS Console, cronShedule is displayed as below on the parameter tab.
CronSchedule = *cron(0 19 config deployment pom.xml README.md scripts src config deployment pom.xml README.md scripts src config deployment pom.xml README.md scripts src )
The issue seems to be * is replaced with directory/file names under the directory where cloudformation command is executed.
Any help would be appreciated.
It works for me passing the variable like this:
CronSchedule="'cron(0 19 * * * *)'"
You should use the wildcard in the expression. Check the following link Schedule Expressions for Rules
Related
I have a problem getting a cron job to correctly target a specific service on the Google App Engine Standard for a python app.
I can successfully create a cron job for a python app on Google App Engine (Standard) with the app.yaml file and a cron.yaml file. The service is not defined and is the default service. the cron.yaml does not specify a target.
If I set the service name in the app.yaml file to service1,
the URL for the app changes from projectID.ew.r.appspot.com to service1-dot-projectID.ew.r.appspot.com.
Next, I specify the target in the cron job to service1 and redeploy the app.yaml and cron.yaml.
The cron job now fails with status 400 every time it runs.
From what I can see in the ProtoPayLoad logs, the host is not using the correct URL.
The cron job uses the URL service1.projectID.ew.r.appspot.com, according the protoPayLoad "host" value in the log. The cron job returns a Status 400.
Why does the cron job not use the service1-dot-projectID.ew.r.appspot.com URL?
What can I do to get the cron job to correctly target a specific service?
#app.yaml with working cron job
#project URL: projectID.ew.r.appspot.com
runtime: python38
handlers:
- url: /static
static_dir: static/
- url: /.*
script: auto
#cron.yaml with working cron job
cron:
- description: "working cron job"
url: /myjob/
schedule: every 2 minutes
Here are the versions of files for the broken cron job
#app.yaml with broken cron job
#project URL: service1-dot-projectID.ew.r.appspot.com
service: service1
runtime: python38
handlers:
- url: /static
static_dir: static/
- url: /.*
script: auto
#cron.yaml with working cron job
cron:
- description: "working cron job"
url: /myjob/
schedule: every 2 minutes
target: service1
The error in the app engine log file has the protoPayLoad details.
host:"service1.projectID.ew.r.appspot.com"
I think it should be the following
host:"service1-dot-projectID.ew.r.appspot.com"
My project has multiple services deployed that will use cron jobs specific to each service.
I cannot just use the default service name.
I appreciate all the help.
It is only after posting a question do you find the answer.
This project is a Django project.
The django Settings file has a list of URLs defined in the ALLOWED_HOSTS variable.
service1-dot-projectID.ew.r.appspot.com was defined but
service1.projectID.ew.r.appspot.com was not defined.
Once this was added, the cron job worked perfectly, targeting a service as normal.
Why isn't cron running every two minutes
I believe it should be:
schedules:
- cron: '0 0/2 * 1/1 * ? *'
displayName: success
branches:
include: master
always: true
When I use a cron generator to evaluate your cron string it errors which leads me to believe the issue is due to a bad cron string.
Here is the tool I used: cronmaker
Why isn't cron running every two minutes
The cron syntax */2 * * * * should be work.
I test it with github resource and your cron syntax:
And the result:
So, I can sure your cron syntax */2 * * * * is correct. We need to check other aspects， like:
Check the modification of schedules in the .yml file is synchronized to the github repo.
Create a new pipeline for the success.yml without any other settings.
If above not help you, you could try to create a pipeline for the Azure devops repo instead of the github rrpo with same cron syntax to check if you still have this issue, so that we could narrow the scope of this issue.
I have the following cron.yaml:
cron:
- description: "TEST_TEST_TEST"
- url: /cronBatchClean
- schedule: every 2 minutes
And then in app.yaml:
service: environ-flexible
runtime: python
env: flex
entrypoint: gunicorn -b :$PORT main:app
runtime_config:
python_version: 3
With this as main.py:
from flask import Flask, request
import sys
app = Flask(__name__)
#app.route('/cronBatchClean')
def cronBatchClean():
print("CRON_CHECK", file=sys.stderr)
return "CRON_CHECK"
When I type in the full URL, I receive "CRON_CHECK" on screen but this doesn't seem to be executing. Also in App Engine dashboard, when I click on CRON jobs there aren't any listed.
Any help in getting this to execute would be much appreciated,
Thanks :)
EDIT 1
I now have the cron task executing but I'm receiving a 404 error. When I type the full URL (that is - https://.appspot.com/cronBatchClean) the respective code executes.
I added a GET handler but I'm still not receiving any luck.
#app.route('/cronBatchClean', methods=['GET'])
def cronBatchClean():
print("CRON_JOB_PRINT", file=sys.stderr)
return "CRON_CHECK"
In the cron.yaml there are unnecessary “-” characters, that are starting the new list. YAML Syntax
Correct format for Cron Jobs cron.yaml, see Google Cloud documentation:
cron:
- description: "TEST_TEST_TEST"
url: /cronBatchClean
schedule: every 2 minutes
To deploy Cron Job use gcloud command :
$ gcloud app deploy cron.yaml
To solve this problem i changed the service name to default then i deploy with the default service the cron job task path was pointed to the default path of app engine so that when the task was scheduled 404 error raised because the path doesn't match when the service name set to "environ-flexible"
In app.yaml
change :
service: environ-flexible
to
service: default
Is there a simple way to change the schedule of a kubernetes cronjob like kubectl change cronjob my-cronjob "10 10 * * *"? Or any other way without needing to do kubectl apply -f deployment.yml? The latter can be extremely cumbersome in a complex CI/CD setting because manually editing the deployment yaml is often not desired, especially not if the file is created from a template in the build process.
Alternatively, is there a way to start a cronjob manually? For instance, a job is scheduled to start in 22 hours, but I want to trigger it manually once now without changing the cron schedule for good (for testing or an initial run)?
You can update only the selected field of resourse by patching it
patch -h
Update field(s) of a resource using strategic merge patch, a JSON merge patch, or a JSON patch.
JSON and YAML formats are accepted.
Please refer to the models in
https://htmlpreview.github.io/?https://github.com/kubernetes/kubernetes/blob/HEAD/docs/api-reference/v1/definitions.html
to find if a field is mutable.
As provided in comment for ref :
kubectl patch cronjob my-cronjob -p '{"spec":{"schedule": "42 11 * * *"}}'
Also, in current kubectl versions, to launch a onetime execution of a declared cronjob, you can manualy create a job that adheres to the cronjob spec with
kubectl create job --from=cronjob/mycron
The more recent versions of k8s (from 1.10 on) support the following command:
$ kubectl create job my-one-time-job --from=cronjobs/my-cronjob
Source is this solved k8s github issue.
From #SmCaterpillar answer above kubectl patch my-cronjob -p '{"spec":{"schedule": "42 11 * * *"}}',
I was getting the error: unable to parse "'{spec:{schedule:": yaml: found unexpected end of stream
If someone else is facing a similar issue, replace the last part of the command with -
"{\"spec\":{\"schedule\": \"42 11 * * *\"}}"
I have a friend who developed a kubectl plugin that answers exactly that !
It takes an existing cronjob and just create a job out of it.
See https://github.com/vic3lord/cronjobjob
Look into the README for installation instructions.
And if you want to do patch a k8s cronjob schedule with the Python kubernetes library, you can do this like that:
from kubernetes import client, config
config.load_kube_config()
v1 = client.BatchV1beta1Api()
body = {"spec": {"schedule": "#daily"}}
ret = v1.patch_namespaced_cron_job(
namespace="default", name="my-cronjob", body=body
)
print(ret)
I would like to use SSM Parameters in Serverless Variables.
Following the docs, I ran this command:
aws ssm put-parameter --name foo --value bar --type SecureString
And I added this to my serverless.yml:
custom:
foo: ${ssm:foo}
When I deploy, I get this warning however:
Serverless Warning --------------------------------------
A valid SSM parameter to satisfy the declaration 'ssm:foo' could not be found.
How do I access this variable? Thanks!
I needed to set the same region for both the serverless function, as well as the ssm variable assignment:
aws ssm put-parameter --name foo--value bar --type SecureString --region us-east-1
if the parameter is a SecureString, you need to add ~true after the path to the parameter on the serverless.yml file, as explained here: https://serverless.com/framework/docs/providers/aws/guide/variables#reference-variables-using-the-ssm-parameter-store
This will tell the framework to decrypt the value. Make sure that you have permissions to use the key used to encrypt the parameter.
Check your IAM policy. To get the parameters, the user doing the deployment needs access to SSM. This offers full access. See the docs to narrow it down a bit (ie: GetParameters, GetParameter).
"Effect": "Allow",
"Action": [
"ssm:*"
],
"Resource": [
"*"
]
Add this to the provider section in serverless.yml file
iamRoleStatements:
- Effect: "Allow"
Action:
- "ssm:GetParameters"
Resource: "*"
to use SSM variables, you need to prefix /aws/reference/secretsmanager/
example
${ssm:/aws/reference/secretsmanager/${self:provider.stage}/service/mysecret~true}