TeamCity Azure Resource Manager Template: Missing MSBuild - azure

Question

I'm using the Azure Resource Manager Template to generate an Azure TeamCity server with an agent on the same Linux CoreOS Azure VM. All the tools are there for building .NET Core projects, but for Xamarin projects, I need the Visual Studio Build Tools enabled on the box.
Following the instructions on http://www.mono-project.com/download/vs/#download-lin, I tried logging onto the agent by connecting to the VM via SSH, and then running:
docker exec -it [container id] bash
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
apt install apt-transport-https
echo "deb https://download.mono-project.com/repo/ubuntu vs-xenial main" | tee /etc/apt/sources.list.d/mono-official-vs.list
apt update
apt install mono-devel #Also apt-install mono-complete
However, even after rebooting the TeamCity VM, there is still an unmet requirement, listed as "Mono4.5_x86 exists".
Installing Mono on the agent works; however, every time I restart the agent, the installation is removed.
How can I install Mono on the build agent so that it persists when I restart the agent?

Answer 1

This took ages to fix, but I managed to find the solution eventually.
By default, the Azure Resource Manager for TeamCity does not contain Mono in its Docker image. You can install Mono on the Docker image, but then as soon as you restart the agent, the changes are not persisted.
You have to customise it following the instructions in https://hub.docker.com/r/jetbrains/teamcity-agent/.
Start off by creating a new image:
docker run -it -e SERVER_URL="http://<my-teamcity-server>.westeurope.cloudapp.azure.com" -v /mnt/data/teamcity-mono-agent:/data/teamcity_agent/conf --name="teamcity-mono-agent" jetbrains/teamcity-agent
Then start the agent using
docker start teamcity-mono-agent
Start a bash session in the agent with
docker exec -it teamcity-mono-agent bash
and in the bash terminal, install Mono using the sequence of instructions in the original question. (You may need to check the version of Linux that is running in the container, and modify these steps accordingly. There are detailed instructions on the Mono website.)
Once you have installed Mono, check the installation by typing mono, and then exit the bash session.
Commit the image using
docker commit teamcity-mono-agent mono-agent
And then restart the agent:
docker restart teamcity-mono-agent
In your TeamCity project, go to the build step and choose "MSBuild" from the dropdown. Choose "Mono xBuild 4.5" for the MSBuild version, "4.0" for the MSBuild Tools version, and x64 for the Platform.
Your new agent should now be able to pick up builds that require Mono.