We have Windows Server 2016 Azure Virtual Machines using managed disks.
I am trying to create an Azure Data Factory pipeline that will let me copy certain files from a folder on the hard drives of those VMs, to our Azure SQL Server. I was quite surprised to see no ADF connectors available for Azure VMs; then I checked Logic Apps - same issue, no available connectors for connecting to Azure VM's there either.
Then I did some Googling to find out how, in general, you can access an Azure VM file structure from outside (without using Remote Desktop) and was even more surprised to see that there isn't any info out there about this (not even that it can't be done).
Is it possible for me to access the file system of my Windows Server 2016 Azure VM without using Remote Desktop? The VM's are running Managed Disks if that makes any difference.
You can either ssh your_vm_ip and then use rsync command to download or upload files.
rsync -au --progress your_user_name#ip.ip.ip.ip:/remote_dir/remote_dir/ /local_dir/local_dir/
Otherwise you can install Dropbox in the VM and your local computer, transfering small files in the shared Dropbox folder is very fast..
Here are some instruction slides on the Azure storage system and their Storage Explorer App.
Related
we have the new asp.net core web application running on Azure as App Service.
Because of the backward compatibility, we have a bunch of files (from the old version of the application) stored on VM Windows machine running at Azure too. Those files must be there!
And we need to access them from Linux App Service as files and directories as they are.
We wanted to use File Share. But because of the App Service sandbox, it is not possible.
Any help?
As of now you have option of mounting or using azure storage with Linux App Service.
https://learn.microsoft.com/en-us/azure/app-service/configure-connect-to-azure-storage?tabs=portal&pivots=container-linux
You can think of using or moving your filesystem from Azure VM to azure storage and further use Linux App Service mounting to Azure Storage.
The above article contains video with every step on how to do that.
Azure Files volume mounting is not supported in Windows containers.
I'm aware I can use AzCopy with Azure Files, but I was wondering if there was a simpler way that doesn't involve creating an Azure storage resource. Because I would have the added work of maintaining the creation/teardown of these storages.
Ideally, I would like the host agent (running create container), to simply copy the files directly to the container instances, therefore the files are tied to the execution of the hosting agent.
As I know there is no way to copy files to the Windows-based Azure container instance except the command. The AzCopy command is OK. It's impossible that you want to do something on the host agent. You can do nothing with the ACI host agent. Additionally, the ACI is more suitable for a quick test and running of the images.
If you want to copy files and other controls on the containers, I recommend the AKS. You can run the Windows-based containers in the AKS with Windows nodes, and the Azure File volume is also available for the Windows containers. See the information here.
Just deployed my App to Azure. Everything works fine.
I'm currently running some legacy code on my app that I cannot upgrade right now, and it makes use of some files on the local VM Storage.
I need to find a way to keep all the cloud instances folder synced. Someone wrote a plugin that seems to do this using the Microsoft Sync Framework, but it runs on Azure SDK 1.5:
https://github.com/Interop-Bridges/Windows-Azure-File-System-Durability-Plugin
Does anyone know of a similar implementation for the current version of the SDK? or a better solution for this scenario?
You could use the CloudDrive feature of Windows Azure. You can put a .VHD file into the blob storage and mount it as a drive in your compute service.
But keep in mind that if you have multiple instance in your computing service, only one instance has read/write access to the VHD. You should share the VHD Drive among all your instances using standard network share technology.
I know very little about Azure, but I am looking for a cloud server where I can have clients SFTP their files to us. It will be used primarily for data storage. The only requirement is that the files be sent over SFTP (not FTP).
Does anyone have any experience with this? How difficult is this to setup? Is this even possible?
You can find step by step instructions on how to set up a regular FTP site on Windows Azure VM here - http://nicoploner.blogspot.com/2010/12/ftp-server-on-windows-azure-from.html
Here's how to set up SFTP on Windows Server (applies to Azure VM as well) - http://www.digitalmediaminute.com/article/1487/setting-up-a-sftp-server-on-windows
Yes you can set up an Azure VM Role and then install a SFTP Server for a Windows Server.
You can also set up a Linux VM Role and just use the native sftp command.
Depending on what you are doing, you may want to use a RESTful service that points back to blob storage (this is not SFTP), but it does go over HTTPs and you have all the benefits of Azure Blob Storage directly.
Here are a couple of options and additional resources:
1) You can install SFTP on Windows Server
https://winscp.net/eng/docs/guide_windows_openssh_server
This uses an OpenSSH package on GitHub from Microsoft.
2) You can use an Ubuntu VM
As #Bart Czernicki mentioned, OpenSSH is built into Linux, and it comes with SFTP out of the box. Customize your implementation using the /etc/ssh/sshd_config file.
3) SFTP Gateway
We have a product on the Azure Marketplace called SFTP Gateway that might help. (Disclosure: I work for Thorn Technologies.)
This is a good option for launching an SFTP server without having to build it from scratch. It also has a web interface for managing users, to help minimize the time spent at the SSH terminal.
BTW, although this wasn't asked in the original question, you might want to consider moving data to a durable storage layer (Azure Blob Storage). One approach would be to use incron to listen for file events. Once a file is done transferring via SFTP, use the Azure CLI to copy the file to Azure Blob Storage, and then delete the file from disk on success. This is the approach we used to build SFTP Gateway.
Hope this helps!
I am developing an Azure application, part of which involves users browsing an online filesystem. TO do this, I am trying to use the Windows Azure drive, but I can't figure out how to access it from client side, or how to make it accessible on the server side.
At the moment, I only know how to make the drive:
CloudStorageAccount devStorage = CloudStorageAccount.DevelopmentStorageAccount;
CloudBlobClient client = devStorage.CreateCloudBlobClient();
CloudBlobContainer container = new CloudBlobContainer("teacher", client);
CloudDrive.InitializeCache(localCache.RootPath,
localCache.MaximumSizeInMegabytes);
CloudDrive drive = new CloudDrive(container.GetPageBlobReference("drive1").Uri, devStorage.Credentials);
drive.Create(50);
I am using C# as my development language.
All help is greatly appreciated!
There are couple of things you need to understand with Windows Azure Cloud Drive:
Cloud drives are actual Page Blobs which are stored on Windows Azure Blob storage and mount as a drive (you will get a drive letter depend on your machine drive statistics) in a machine where you can provide Windows Azure Run time environment.
Programmatic it is very easy to mount a cloud drive in your code as you showed in your example however one thing is missed that is to be sure to have Windows Azure RunTime environment where this code can run.
I have written a utility to mount azure drive within Windows Azure VM (Web, Worker or VM Role) located here:
http://mountvhdazurevm.codeplex.com/
You can run above tool directly in Windows Azure VM and can also this the exact same code in your Compute Emulator (Windows Azure Development Fabric) so the bottom line is as long as you can provide Windows Azure Runtime environment, you can mount a Page blob VHD drive.
I have seen several cases where someone asked me to mount a Windows Azure Page Blob as drive in local machine (client and server, anywhere) and the actual hurdle was to bring Windows Azure Run time in local environment because it is not available. In some cases a few person went ahead and tries to use Windows Azure SDK to have Windows Azure runtime made
available in their desktop, created a dummy web role and then mount the VHD which was mounted in local machine and a drive letter was made available as well. I am not sure about such kind of solution because this is not Windows Azure compute emulator is designed.
Hope this description provide you some guideline.
I'm not sure I understand your question properly, but it sounds like you want multiple client applications - presumably on machines that are not on Azure - to access your Azure drive?
Unfortunately, Azure drives can only be accessed from Azure web/worker or VM role instances.
I've written a WebDAV Server which runs on an Azure Website which will allow clients, including Windows Explorer and Office to connect to Azure Storage. It uses a combination of Table and Blob Storage to store the file structure and files. I've tested it with Windows Explorer and Word 2013. Although this isn't a clouddrive solution it's still using Azure Storage as a backend and it's accessible from WebDAV clients. You might find it useful..
https://github.com/ichivers/AzureDAV
One additional point to the existing answers. You can always download the blob backing your Cloud Drive and mount it on a local system. The blob is really just a VHD. However, the download time isn't going to trivial unless the drive is small.
Erick