According to this blog, the limit on TCP ports opened in Azure WebApp is 1920 for S1, 3968 for S2, and 8064 for S3. Are there updated number for newer SKUs like P1V2/P2V2/P3V2? The RAM was doubled so I am wondering whether the number of TCP ports was also increased.
Are there updated number for newer SKUs like P1V2/P2V2/P3V2?
These limits are there to protect the entire VM even though one particular site may be with its limits described above. The limits are different depending on the size of VM configured.
You could refer to this article which was edited 21 days ago.
Related
I'm a part of building a service controlling Ev Charging Stations. The protocol is called OCPP and the underlying transport protocol is web sockets.
The stations are calling the host and the web socket is then being upheld, both server and client are initiating commands.
We have implemented the protocol and all that, the question we're looking into is how do we scale and host the web sockets (pressure load of with queue, etc is not the question).
We're currently on Azure and we have in our prototypes used Azure App Service which works fine, however, we have presently not looked into the limitations when it comes to scaling.
We have looked at Azure Pub Sub, however, it doesn't seem compatible with OCPP.
The question is what type of hosting we should look for to host the web sockets?
The documentation says that the number of Websockets is unlimited starting with the Standard plan but take a good look at the side note:
If you scale an app in the Basic tier to two instances, you have 350 concurrent connections for each of the two instances. For Standard tier and above, there are no theoretical limits to web sockets, but other factors can limit the number of web sockets. For example, maximum concurrent requests allowed (defined by maxConcurrentRequestsPerCpu) are: 7,500 per small VM, 15,000 per medium VM (7,500 x 2 cores), and 75,000 per large VM (18,750 x 4 cores).
I'm looking for the best way to manage the download of my products on the web.
Each of them weighs between 2 and 20 Go.
Each of them is approximatively downloaded between 1 and 1000 times a day by our customers.
I've tried to use Amazon S3 but the download speed isn't good and it quickly becomes expensive.
I've tried to use Amazon S3 + CloudFront but files are too large and the downloads too rare: the files didn't stay in the cache.
Also, I can't create torrent files in S3 because the files size is too large.
I guess the cloud solutions (such as S3, Azure, Google Drive...) work well only for small files, such as image / css / etc.
Now, I'm using my own servers. It works quite well but it is really more complex to manage...
Is there a better way, a perfect way to manage this sort of downloads?
This is a huge problem and we see it when dealing with folks in the movie or media business: they generate HUGE video files that need to be shared on a tight schedule. Some of them resort to physically shipping hard drives.
When "ordered and guaranteed data delivery" is required (e.g. HTTP, FTP, rsync, nfs, etc.) the network transport is usually performed with TCP. But TCP implementations are quite sensitive to packet loss, round-trip time (RTT), and the size of the pipe, between the sender and receiver. Some TCP implementations also have a hard time filling big pipes (limits on the max bandwidth-delay product; BDP = bit rate * propagation delay).
The ideal solution would need to address all of these concerns.
Reducing the RTT usually means reducing the distance between sender and receiver. Rule of thumb, reducing your RTT by half can double your max throughput (or cut your turnaround time in half). Just for context, I'm seeing an RTT from US East Coast to US West Coast as ~80-85ms.
Big deployments typically use a content delivery network (CDN) like Akamai or AWS CloudFront, to reduce the RTT (e.g. ~5-15ms). Simply stated, the CDN service provider makes arrangements with local/regional telcos to deploy content-caching servers on-premise in many cities, and sells you the right to use them.
But control over a cached resource's time-to-live (TTL) can depend on your service level agreement ($). And cache memory is not infinite, so idle resources might be purged to make room for newly requested data, especially if the cache is shared with others.
In your case, it sounds to me like you want to meaningfully reduce the RTT while retaining full control of the cache behaviour, so you can set a really long cache TTL. The best price/performance solution IMO is to deploy your own cache servers running CentOS 7 + NGINX with proxy_cache turned on and enough disk space, and deploy a cache server for each major region (e.g. west coast and east coast). Your end users could select the region closest to them, or your could add some code to automatically detect the closest regional cache server.
Deploying these cache servers on AWS EC2 is definitely an option. Your end users will probably see much better performance than by connecting to AWS S3 directly, and there are no BW caps.
The current AWS pricing for your volume is about $0.09/GB for BW out to the internet. Assuming your ~50 files at an average of 10GB, that's about $50/month for BW from cache servers to your end users - not bad? You could start with c4.large for low/average usage regions ($79/month). Higher usage regions might cost you about ~$150/month (c4.xl), ~$300/month (c4.2xl), etc. You can get better pricing with spot instances and you can tune performance based on your business model (e.g. VIP vs Best-Effort).
In terms of being able to "fill the pipe" and sensitivity to network loss (e.g. congestion control, congestion avoidance), you may want to consider an optimized TCP stack like SuperTCP (full disclaimer, I'm the director of development). The idea here is to have a per-connection auto-tuning TCP stack with a lot of engineering behind it, so it can fill huge pipes like the ones between AWS regions, and not overreact to network loss as regular TCP often does, especially when sending to Wi-Fi endpoints.
Unlike UDP solutions, it's a single-sided install (<5 min), you don't get charged for hardware or storage, you don't need to worry about firewalls, and it won't flood/kill your own network. You'd want to install it on your sending devices: the regional cache servers and the origin server(s) that push new requests to the cache servers.
An optimized TCP stack can increase your throughput by 25%-85% over healthy networks, and I've seen anywhere from 2X to 10X throughput on lousy networks.
Unfortunately I don't think AWS is going to have a solution for you. At this point I would recommend looking into some other CDN providers like Akamai https://www.akamai.com/us/en/solutions/products/media-delivery/download-delivery.jsp that provide services specifically geared toward large file downloads. I don't think any of those services are going to be cheap though.
You may also want to look into file acceleration software, like Signiant Flight or Aspera (disclosure: I'm a product manager for Flight). Large files (multiple GB in size) can be a problem for traditional HTTP transfers, especially over large latencies. File acceleration software goes over UDP instead of TCP, essentially masking the latency and increasing the speed of the file transfer.
One negative to using this approach is that your clients will need to download special software to download their files (since UDP is not supported natively in the browser), but you mentioned they use a download manager already, so that may not be an issue.
Signiant Flight is sold as a service, meaning Signiant will run the required servers in the cloud for you.
With file acceleration solutions you'll generally see network utilization of about 80 - 90%, meaning 80 - 90 Mbps on a 100 Mbps connection, or 800 Mbps on a 1 Gbps network connection.
I'm want to build a FTP server on Microsoft Azure platform.
The server probably will be based on vsftpd (linux ubuntu server) - not closed. Can select another free ftp service.
I'm have two issues:
End-points - If I'm using Passive mode, I'm need to allocate port range for this. Let's say 8000 to 8100. BUT, I'm have a option to allocate only 20 ports in the "End-points" interface.
I'm need to allocate at least one terabyte of storage in this server. How it's could be done with the machine?
Thank you!
There are a lot of articles written on the subject. Whereas one of the most recent is here.
Fr Virtual Machines, the limit of InputEndpoints is actually 150. Refer to the latest Azure limits compilation here.
As for 1T storage - check the VM sizes for Azure Virtual Mchines. Where A0 (the smallest) supports max 1 data disk of 1TB, and A4, A7 support 16 disks 1TB each (so total of 16TB per VM of size A4 or A7).
As for "built-in" endpoints - you can freely and surely remove them. Especially when you do not use them.
I'm an msdn subscriber and I'm looking at Azure as a possible platform for a new website that will test the water of a new service. This website is expecting low to very low traffic at the time of launch. I've heard that this kind of traffic levels is very expensive for Azure but since they have this msdn offer, I thought I'd finally take a look at Azure.
In the offer, I'm looking at getting "750 small compute hours per month". From the reading I've done, this seems that, if I purchase nothing more than what's given (although the subscription itself is thousands of dollars of course), that an entire month would be covered. Since 24 (hours) x 31 (max days in a month) = 744 I'm still below my allotted 750 for the month.
Am I missing something else from this simple equation? Is there further aspects that could cause the site to be "turned off" temporarily that should be considered?
Yes, you can indeed run a small instance during a whole month. Or you can have 2 extra-small instances instead (having 2 instances means you're covered by the SLA).
There are 2 other things you need to consider:
Depending on your subscription you can have maximum 45GB of storage (blob/table/queue). If you use Virtual Machines you need to know that the system disk (and additional data disks) are persisted as blobs, so make sure not to reach the limit here.
There are also other limits active, but the most important one besides storage is the data transfer limit which is also very limited (max 35GB out).
If you're expecting very low traffic, did you ever consider Windows Azure Web Sites? You get 10 of those for free during 12 months. The free ones run on shared instances, but they are perfect to host the first low-traffic version of your app.
I know few facts from Azure:
Limited bandwidth for each instance, for example: Extra Small instance has 5Mb/s
We only pay outbound bandwidth, inbound is free
For same data center, traffic is free
even for small instance, I checked the network interface, the connection speed is 10G. So Allocated Bandwidth is always lower than the connection speed which the network interface capable with.
I like to know:
Allocated Bandwidth for outbound only or for all traffic?
Allocated Bandwidth for outside of data center or for any data?
I like to know more for useful answers to me:
Will the Allocated Bandwidth limit the traffic between instances in one deploy (they will be in same data center, same network) I will exchange data between instance, so I like to know if the whole network interface capacity will be utilized.
Will the Allocated Bandwidth limit the traffic between instance and CloudDrive which in same data center? If does, then limit on Read or Write or Both? I will use CloudDrive lots, and the bandwidth won't cost since they are in same center, so I like to know the speed limitation.
Will the Allocated Bandwidth limit the instance connection to outside, for example, to send email by a outside SMTP server?
Any official source is highly appreciated.
At the end of the day, you are mostly constrained to the NIC on the box from within the datacenter. Each of the physical machines has a 1Gbps NIC on it. It was found to have about 800Mbps sustained transfer speed. Since each host could also currently have 8 guests on it, the ratio of the reserved NIC was a multiplier of the cores. If you had a Small instance, you were reserved 100Mbps, Medium has 200Mbps, L 400Mbps, etc. However, this does not taken into account bursting. Small instances can burst to 250Mbps or so in practice - it just depends on what your neighbors on the box are doing. The more cores you reserve, the higher your bursting will be upto the max.
The XS instance size is actually limited to 5Mbps, so it does not follow the other pattern.
Again, your connection within the datacenter will mostly be limited by the NIC bandwidth. This is true for instance to instance and instance to storage. For out of datacenter (where prevalent network conditions generally matter more), the NIC is the limit still, but generally other factors outside of datacenter are bottleneck.
The sort of exception to all this is when accessing a particular partition in storage: You can get around 60MB/s transfer per blob (or partition really), as this is limited to the rate at which a partition server will service your request as opposed to NIC speed. However, the entire storage account's limit is 3Gbps (more than your NIC). That can only be achieved when accessing multiple partitions.
Back into your context, that means you will get at max around 60MB/sec (480Mbps) for a particular cloud drive. To fully saturate that, you would need a Large instance or higher. That is why I said you are mostly constrained to NIC.
Regarding your questions,
bandwidth throttling is applied to outbound traffic only.
it is for any data, even if the VM talks to another VM in the same cluster.
If you have multiple instances in one deployment, bandwidth throttling will apply to the traffic between them.
You may see How Does Bandwidth Throttling Work in Azure? for more technical details.