I'm trying to determine whether it is feasible from a cost perspective to use AWS Rekognition video streaming for short, scheduled streams on a camera network.
I understand that they bill $0.12/min for the service, however the docs are not clear if this is rounded to the nearest minutes per streaming invocation or at the end of the month.
This could result in a VERY drastic change in price when you have 1000s of cameras...
For example, if you stream 30 seconds per day for a month, are you billed for 30 minutes or 15 minutes?
15 mins.
From Rekognition price calculation page :
Usage charges are applied monthly and are pro-rated for partial minutes.
Related
I am planning to use structured streaming to calculate daily aggregates across different metrics.
Data volume < 1000 records per day.
Here is the simple example of input data
timestamp, Amount
1/1/20 10:00, 100
1/1/20 11:00, 200
1/1/20 23:00, 400
1/2/20 10:00, 100
1/2/20 11:00, 200
1/2/20 23:00, 400
1/2/20 23:10, 400
Expected output
Day, Amount
1/1/20, 700
1/2/20, 1100
I am planning to do something like this in the structured streaming not sure if it works or if it's the right way to do it?
parsedDF.withWatermark("date", "25 hours").groupBy("date", window("date", "24 hours")).sum("amount")
There is material overhead from running structured streams. Given you're writing code to produce a single result every 24 hours it would seem a better use of resources to do the following if you can take an extra couple minutes of latency in trade for using far fewer resources.
Ingest data into a table, partitioned by day
Write a simple SQL query against this table to generate your daily aggregate(s)
Schedule the job to run [watermark] seconds after midnight.
That's with the impression you're in the default output mode since you didn't specify one. If you want to stick with streaming, more context in your code and what your goal is would be helpful. For example, how often do you want results, and do you need partial results before the end of the day? How long do you want to wait for late data to update aggregates? What output mode are you planning to use?
I have created a logic app to trigger when a tweet is posted with a given hashtag. The trigger is set to check every 10 seconds. The reality is that the Logic App does not run, even if I wait minutes for it, but then if i manually run it, it then executes with the expected input. Any idea what is happening here?
I was having a similar issue, and believe this is due to the specific limitations set for the Twitter Connector (4. Frequency of trigger polls: 1 hour).
https://learn.microsoft.com/en-us/connectors/twitterconnector/
LIMITS
The following are some of the limits and restrictions:
Maximum number of connections per user: 2
API call rate limit for POST operation: 12 per hour
API call rate limit for other operations: 600 per hour
Frequency of trigger polls: 1 hour
Maximum size of image upload: 5 MB
Maximum size of video upload: 15 MB
Maximum number of search results: 100
Maximum number of new tweets tracked within one polling interval: 5
There should be some error occurred. You can inspect all runs of the triggers on the 'Trigger History' blade. This page gives a good overview of monitoring of logic apps: https://azure.microsoft.com/en-us/documentation/articles/app-service-logic-monitor-your-logic-apps/
We have some stream data need to be calculated and considering use spark stream to do it.
We need to generate three kinds of reports. The reports are based on
The last 5 minutes data
The last 1 hour data
The last 24 hour data
The frequency of reports is 5 minutes.
After reading the docs, the most obvious way to solve this seems to set up a spark stream with 5 minutes interval and two window which are 1 hour and 1 day.
But I am worrying that if the window is too big for one day and one hour. I do not have much experience on spark stream, so what is the window length in your environment?
Is it anyhow possible to get the count, how often DynamoDB throughput (write units/read units) was downscaled within the last 24 hours?
My idea is to downscale as soon as an hugo drop e.g. 50% in the needed provisioned write units occur. I have really peaky traffic. Thus it is interessting to me to downscale after every peak. However I have a analytics jobs running at night which is provisioning a huge amount of read units making it necessary to be able to downscale after it. Thus I need to limit downscales to 3 times within 24 hours.
The number of decreases is returned in a DescribeTable result as part of the ProvisionedThroughputDescription.
I am using Jmeter (started using it a few days ago) as a tool to simulate a load of 30 threads using a csv data file that contains login credentials for 3 system users.
The objective I set out to achieve was to measure 30 users (threads) logging in and navigating to a page via the menu over a time span of 30 seconds.
I have set my thread group as:
Number of threads: 30
Ramp-up Perod: 30
Loop Count: 10
I ran the test successfully. Now I'd like to understand what the results mean and what is classed as good/bad measurements, and what can be suggested to improve the results. Below is a table of the results collated in the Summary report of Jmeter.
I have conducted research only to find blogs/sites telling me the same info as what is defined on the jmeter.apache.org site. One blog (Nicolas Vahlas) that I came across gave me some very useful information,but still hasn't help me understand what to do next with my results.
Can anyone help me understand these results and what I could do next following the execution of this test plan? Or point me in the right direction of an informative blog/site that will help me understand what to do next.
Many thanks.
According to me, Deviation is high.
You know your application better than all of us.
you should focus on, avg response time you got and max response frequency and value are acceptable to you and your users? This applies to throughput also.
It shows average response time is below 0.5 seconds and maximum response time is also below 1 second which are generally acceptable but that should be defined by you (Is it acceptable by your users). If answer is yes, try with more load to check scaling.
In you requirement it is mentioned that you need have 30 concurrent users performing different actions. The response time of your requests is less and you have ramp-up of 30 seconds. Can you please check total active threads during the test. I believe the time for which there will be 30 concurrent users in system is pretty short so the average response time that you are seeing seems to be misleading. I would suggest you run a test for some more time so that there will be 30 concurrent users in the system and that would be correct reading as per your requirements.
You can use Aggregate report instead of summary report. In performance testing
Throughput - Requests/Second
Response Time - 90th Percentile and
Target application resource utilization (CPU, Processor Queue Length and Memory)
can be used for analysis. Normally SLA for websites is 3 seconds but this requirement changes from application to application.
Your test results are good, considering if the users are actually logging into system/portal.
Samples: This means the no. of requests sent on a particular module.
Average: Average Response Time, for 300 samples.
Min: Min Response Time, among 300 samples (fastest among 300 samples).
Max: Max Response Time, among 300 samples (slowest among 300 samples).
Standard Deviation: A measure of the variation (for 300 samples).
Error: failure %age
Throughput: No. of request processed per second.
Hope this will help.