I am trying out OpenFaas auto scaling feature from 0 instances.
I tried running nodeinfo function with this and tried invoking it.
kubectl scale deployment --replicas=0 nodeinfo -n openfaas-fn
Once replicas are down to 0, I wanted to try invoking it from Gateway UI to make it auto scale from 0, but status is not ready and Invoke button is inactive until I bump up replica count to more than 0. It seems like it is not possible to invoke a function that has 0 instances.
It doesn't look like auto scaling from 0 working, unless I am missing something.
Any guidance or help is appreciated.
You can make the first function invocation with the command line.
Example:
echo -n "google.com" | faas-cli invoke curl --gateway 127.0.0.1:31112
Here, curl is the function's name
Related
I'm trying to visualize the browser statistics of our app hosted in Azure.
For that I'm using the nginx logs and run an Azure Log Analytics query like this:
ContainerLog
| where LogEntrySource == "stdout" and LogEntry has "nginx"
| extend logEntry=parse_json(LogEntry)
| extend userAgent=parse_user_agent(logEntry.nginx.http_user_agent, "browser")
| extend browser=parse_json(userAgent)
| summarize count=count() by tostring(browser.Browser.Family)
| sort by ['count']
| render piechart with (legend=hidden)
Then I'm getting this diagram, which is exactly what I want:
But the query is very very slow. If I set the time range to more than just the last few hours it takes several minutes or doesn't work at all.
My solution is to use a search job like this:
ContainerLog
| where LogEntrySource == "stdout" and LogEntry has "nginx"
| extend d=parse_json(LogEntry)
| extend user_agent=parse_user_agent(d.nginx.http_user_agent, "browser")
| extend browser=parse_json(user_agent)
It creates a new table BrowserStats_SRCH on which I can do this search query:
BrowserStats_SRCH
| summarize count=count() by tostring(browser.Browser.Family)
| sort by ['count']
| render piechart with (legend=hidden)
This is much faster now and only takes some seconds.
But my problem is, how can I keep this up-to-date? Preferably this search job would run once a day automatically and refreshed the BrowserStats_SRCH table so that new queries on that table run always on the most recent logs. Is this possible? Right now I can't even trigger the search job manually again, because then I get the error "A destination table with this name already exists".
In the end I would like to have a deeplink to the pie chart with the browser stats without the need to do any further click. Any help would be appreciated.
But my problem is, how can I keep this up-to-date? Preferably this search job would run once a day automatically and refreshed the BrowserStats_SRCH table so that new queries on that table run always on the most recent logs. Is this possible?
You can leverage the api to create a search job. Then use a timer triggered azure function or logic app to call that api on a schedule.
PUT https://management.azure.com/subscriptions/00000000-0000-0000-0000-00000000000/resourcegroups/testRG/providers/Microsoft.OperationalInsights/workspaces/testWS/tables/Syslog_suspected_SRCH?api-version=2021-12-01-preview
with a request body containing the query
{
"properties": {
"searchResults": {
"query": "Syslog | where * has 'suspected.exe'",
"limit": 1000,
"startSearchTime": "2020-01-01T00:00:00Z",
"endSearchTime": "2020-01-31T00:00:00Z"
}
}
}
Or you can use the Azure CLI:
az monitor log-analytics workspace table search-job create --subscription ContosoSID --resource-group ContosoRG --workspace-name ContosoWorkspace --name HeartbeatByIp_SRCH --search-query 'Heartbeat | where ComputerIP has "00.000.00.000"' --limit 1500 --start-search-time "2022-01-01T00:00:00.000Z" --end-search-time "2022-01-08T00:00:00.000Z" --no-wait
Right now I can't even trigger the search job manually again, because then I get the error "A destination table with this name already exists".
Before you start the job as described above, remove the old result table using an api call:
DELETE https://management.azure.com/subscriptions/{subscriptionId}/resourcegroups/{resourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{workspaceName}/tables/{tableName}?api-version=2021-12-01-preview
Optionally, you could check the status of the job using this api before you delete it to make sure it is not InProgress or Deleting
Say I have a pipeline where I invoke a Lambda and it responds 'true'. In this case, the pipeline should proceed. In the other case where the lambda responds 'false', I want to be able to look at that output variable and if it is false, manually fail the deployment stage (so that I can redeploy an old version of the code).
Seems like a simple enough question, but I can't find any info on this simple task.
Add a script task as below - exit if lambda output is false
- script: |
lamdaOutput = [result from call]
if lamdaOutput; then
exit 0
else
exit 1
fi
Using boto3 of aws, I am trying to run start query and get the results using query id. but it didnt work as expected in python script. It returns the expected json output for start_query and able to fetch the queryID. But if i try to fetch the query results using queryID, it returns empty json.
<code>
import boto3
client = boto3.client('logs')
executeQuery = client.start_query(
logGroupName='LOGGROUPNAME',
startTime=STARTDATE,
endTime=ENDDATE,
queryString='fields status',
limit=10000
)
getQueryId=executeQuery.get('queryId')
getQueryResults = client.get_query_results(
queryId=getQueryId
)
</code>
it returns the reponse of get_query_results as
{'results': [], 'statistics': {'recordsMatched': 0.0, 'recordsScanned': 0.0, 'bytesScanned': 0.0}, 'status': 'Running',
But if i try using aws cli with the queryID generated from script, it returns json output as expected.
Anyone could able to tell why it didnt work from boto3 python script and worked in cli?
Thank you.
The query status is Running in your example. Its not in Complete status yet.
Running queries is not instantaneous. Have to wait a bit for query to complete, before you can get results.
You can use describe_queries to check if your query has completed or not. You can also check if logs service has dedicated waiters in boto3 for the results. They would save you from pulling describe_queries API in a loop waiting till your queries finish.
When you do this in CLI, probably there is more time before you start the query, and query results using CLI.
The other issue you might be encountering is that the syntax for the queryString in the API is different from a query you would type into the CloudWatch console.
Console query syntax example:
{ $.foo = "bar" && $.baz > 0 }
API syntax for same query:
filter foo = "bar" and baz > 0
Source: careful reading and extrapolation from the official documentation plus some trial-and-error.
My logs are in JSON format. YMMV.
Not sure if this problem is resolved. I was facing the same issue with AWS Java SDK . But when i terminate the thread performing the executeQuery query and perform the get_query_results using a new thread and the old queryId. It seems to be working fine.
Adding sleep will work here. If the query is exceeding the Sleep time then again it will show as Running status. You can write a Loop where you can check the status Completed, if the status is Running you can run Sleep again for some second and retry. You can give some retry count here.
Sample Pseudocode:
function for sleep; (let's say SleepFunc())
Loop till retry count
check if the status is Completed;
If yes break;
else call SleepFunc();
I'm able to work with the metric GroupInServiceInstances quietly. got after enable euscale-enable-metrics-collection groupname -g 1minute
But I can not work with the metric of CPUUtilization of the AWS/EC2, the alarm does not attend the metric, the state is only INSUFFICIENT_DATA. I noticed that when running the command euscale-describe-metric-collection-types shows nothing AWS/EC2, but only the AWS/ScalingGroup, see:
METRIC-COLLECTION-TYPE GroupDesiredCapacity
METRIC-COLLECTION-TYPE GroupInServiceInstances
METRIC-COLLECTION-TYPE GroupMaxSize
METRIC-COLLECTION-TYPE GroupMinSize
METRIC-COLLECTION-TYPE GroupPendingInstances
METRIC-COLLECTION-TYPE GroupTerminatingInstances
METRIC-COLLECTION-TYPE GroupTotalInstances
GRANULARITY-METRIC-TYPE 1minute
should be displayed:
METRIC-COLLECTION-TYPE CPUUtilization
GRANULARITY-METRIC-TYPE Percent
So, what I do to the alarm of the metric CPUUtilization AWS/EC2 work?
The euca2ools euscale-* commands are for use with the Auto Scaling service, the euscale-describe-metric-collection-types command is the DescribeMetricCollectionTypes action, and only returns metrics for Auto Scaling.
To enable EC2 metrics in Eucalyptus you have to enable metrics collection for the instance (euca-monitor-instances) and you should also ensure that when working with the CPUUtilization CloudWatch metric you specify the unit type of Percent.
Can you please try EucaLobo https://github.com/viglesiasce/EucaLobo?
I remember I had trouble with command line arguments at first.
I found the reason. The alarm was creating from graphic interface when creating a command worked. I found that the GUI does not assign the unit (Percent) in alarm, which causes it to not collect the metrics percentage.
Will the first instance deployed always end with a zero ? 0. like "xxxx_IN_0"
When scaling up to X instanses, will the next instanses always get 1 2 3 4 as the last number. ( I think so).
What happens when I scale down again? I read that it will take at random one of the instances. So when scaling down to 1 instance, I cant assume that I know what ID is the one that still are running?.
Anyone who have been playing with the IDs when scaling up and down and know those things?
The reason for me asking is that I have some extra logic that I want to run on only one, not less or more, of the instances. If I can assume that the "xxx_IN_0" is always present then i can just do it with a simple check that last of ID is zero. If not, I am considering to check all ids and if the current instance is the lowest, then it will do its magic.
If the last case, are there an event i can monitor for when scaling up or down is done?.
update
From Answer:
if (RoleEnvironment.IsAvailable)
{
RoleEnvironment.Changed += RoleEnvironment_Changed;
}
void RoleEnvironment_Changed(object sender, RoleEnvironmentChangedEventArgs e)
{
var change = e.Changes.OfType<RoleEnvironmentTopologyChange>().FirstOrDefault();
if (change != null)
{
Trace.TraceInformation("RoleEnvironmentTopologyChange at RoleName '{0}'", change.RoleName);
}
I do not get any information in my tracelog when i scale up and down.
There have to be set a internal endpoint to trigger the events:
<InternalEndpoint name="InternalEndpoint1" protocol="http" />
http://blogs.msdn.com/b/windowsazure/archive/2011/01/04/responding-to-role-topology-changes.aspx
You should listen to the RoleEnvironment.Changed event which occurs when the service configuration is changed.
When you receive this event, check the Changes property for an instance of RoleEnvironmentTopologyChange. This change will be reported when the number of instances are changed.
To determine on which server you should run that extra logic you could examine the list of roles and from there find all instances. Sort instances by id and select the first to be the special one.
However, if it is critical that only a single server run this special logic at any time, you will need a more stable solution.