I have my X509Certificate stored in a database (in byte[]) so that my application can retrieve the certificate and use it to sign my JWTs.
My x509Certificate is passed off a .pfx file that I generated on my machine, however now it sits in a database as a string of bytes.
My application works perfectly fine locally when I run it. The application can correctly create an instance of that X509Certificate2 and use it for my requirements, however the problem arises when I try to use it in my azurewebsites web application.
Basically I can not access the certificates' PrivateKey instance variable, I get an exception
System.Security.Cryptography.CryptographicException: Keyset does not exist
And I am re-instantiating the certificate with this
var cert = new X509Certificate2(myCertInBytes, myCertPass,
X509KeyStorageFlags.PersistKeySet |
X509KeyStorageFlags.MachineKeySet |
X509KeyStorageFlags.Exportable);
I am using ASPNET 5 rc1-update1. I have also tried running this on a different machine and it works fine, only have this issue when I publish to Azure. And to also add something else, This application was working when I was running the same project that was running using DNX version beta7
Any help appreciated.
The problem is the Azure Web Apps restricts access to the machines private key store, since it's a shared hosting environment, and you don't fully own the machine. As a workaround, you can load a cert. This blog post describes the best practice on how to do so:
https://azure.microsoft.com/en-us/blog/using-certificates-in-azure-websites-applications/
Please note that this only works for Basic Tier and above (not Free or Shared tier).
This can also be done from a .cer file as follows, however it should be noted that this is not best-practices since you're storing a secure credential with your code, in an insecure format.
public X509Certificate2 CertificateFromStrings(String certificateString64, String privateKeyXml)
{
try
{
var rsaCryptoServiceProvider = new RSACryptoServiceProvider();
rsaCryptoServiceProvider.FromXmlString(privateKeyXml);
var certificateBytes = Convert.FromBase64String(certificateString64);
var x509Certificate2 = new X509Certificate2(certificateBytes);
x509Certificate2.PrivateKey = rsaCryptoServiceProvider;
return x509Certificate2;
}
catch
{
return null;
}
}
Related
I am trying to use the .Net Core Clean Architecture App Template and get it running in containers and deployed through an azure CI/CD pipeline
I have the containerized version of the template running locally in linux container with port 5001 and everything works perfectly.
I have the azure pipeline build process working properly and it creats image in my container registry.
The problem is once I deploy/release to a Web App for Containers, the app fails and throws the following error:
Application startup exception
System.InvalidOperationException: Couldn't find a valid certificate with subject 'CN=localhost' on the 'CurrentUser\My' at
Microsoft.AspNetCore.ApiAuthorization.IdentityServer.SigningKeysLoader.LoadFromStoreCert(String
subject, String storeName, StoreLocation storeLocation, DateTimeOffset currentTime)
What I have done:
Following these docs from MS I have created a local dev cert:
dotnet dev-certs https -ep %USERPROFILE%\.aspnet\https\aspnetapp.pfx -p { password here }
dotnet dev-certs https --trust
I then imported this into the Web App as a private .pfx cert.
I added an application setting WEBSITE_LOAD_CERTIFICATES with the "thumb" value of the cert
I used the "hostname" of the imported cert in the Identity Server appSettings.json section (hostname=localhost in my case)
When the Web app loads, it shows :( Application error and the docker logs give me the error I quoted above.
I am pretty sure this is related to the Identity server set up and the appSettings.json values here:
"IdentityServer": {
"Key": {
"Type": "Store",
"StoreName": "My",
"StoreLocation": "CurrentUser",
"Name": "CN=localhost"
}
}
Can someone help me figure out how to resolve this error?
EDIT 1 - Manually specify file for IdentityServer Key
This is related to identity server for sure. I tried to manually set the Cert as a file in the appSettings.json like this:
"IdentityServer": {
"Key": {
"Type": "File",
"FilePath": "aspnetapp.pfx",
"Password": "Your_password123"
}
}
Now I get this error:
Loading certificate file at '/app/aspnetapp.pfx' with storage flags
''. Application startup exception System.InvalidOperationException:
There was an error loading the certificate. The file
'/app/aspnetapp.pfx' was not found.
Microsoft.AspNetCore.ApiAuthorization.IdentityServer.SigningKeysLoader.LoadFromFile
I added this to the dockerfile:
WORKDIR /app
COPY ["/aspnetapp.pfx", "/app"]
RUN find /app
And as you can see from the image below, the files are showing in the build directory for the app:
I also made sure that the aspnetapp.pfx is not getting ignored by the .gitignore or .dockerignore files.
I cannot figure out why it won't load this file. It appears like it exists right where it is supposed to.
EDIT 2 using cert thumb and updated path
So I used tnc1977 suggestion and had this as my setting for the identity key
"IdentityServer": {
"Key": {
"Type": "File",
"FilePath": "/var/ssl/private/<thumb_value>.p12",
"Password": "Your_password123"
}
}
However, this gave another error:
There was an error loading the certificate. Either the password is
incorrect or the process does not have permisions to store the key in
the Keyset 'EphemeralKeySet'
Interop+Crypto+OpenSslCryptographicException: error:23076071:PKCS12
routines:PKCS12_parse:mac verify failure
EDIT 3: Valid Azure App Certificate
I purchased an Azure App Certificate and added a custom domain with TSL set up and the same errors appear
EDIT 4: Load Cert in code startup.cs - new error:
I now know that I caanot use the cert store CurrentUser/My because that is for windows. Linux containers have to manually load the cert in code.
I am using the thumbprint of aa application certificate that has been added to the azure web app. It is a private azure app cert and it has been verified against a custom domain.
I added this code to my statup.cs configureservices (I know hardcoding these values is not best practice but I want to just see if it could load the cert, I will wsitch to env variables and key vault):
// linux file path for private keys
var cryptBytes = File.ReadAllBytes("/var/ssl/private/<thumbprint>.p12");
var cert = new X509Certificate2(cryptBytes, "");
services.AddIdentityServer().AddSigningCredential(cert);
I enter a blank password because I think that is what you are supposed to do. I am now getting the following error in my docker logs which leads me to believe the cert loaded and now the error is related to me using both services.AddIdentityServer().AddSigningCredential(cert); in startup.cs configureservices and app.UseIdentityServer() in startup.cs configure:
Unhandled exception. System.InvalidOperationException: Decorator already registered for type: IAuthenticationService.
I am not sure how to add the cert to the app.UseIdentityServer(); line.
EDIT 5
after a lot more digging, unfortunately #tnc1997 answer will not work.
IN asp.net core 3 calls to app.UseIdentityServer in my satrtup.cs internally reverence a method that will look for the identity server Key,File,Pass etc in the appsetting(environment).json file.
As a result, even if I loaded the cert in code like tnc1997 shows, the application still looks in the settings file. So the settings file has to contain the corect details for the IS4 key.
Also, azure does not place the cert in the typical trusted location in the linux container. From what I have read, it appears that the only way to do this is to mount a volume (in this case an azure storage file share) and use the cert uploaded to that file share.
I can confirm that this works locally, but now I am still having issues running the container, the front end loads and it appears that the web api project does not start. I am going to post another question to address that issue.
Original Answer
I think the problem could be that you are attempting to load a certificate in a Linux container using the Windows certificate store.
The documentation here gives a good overview regarding how you can use an app service private certificate in a Linux hosted app:
In the Azure portal, from the left menu, select App Services > <app-name>.
From the left navigation of your app, select TLS/SSL settings, then select Private Key Certificates (.pfx) or Public Key Certificates (.cer).
Find the certificate you want to use and copy the thumbprint.
To access a certificate in your app code, add its thumbprint to the WEBSITE_LOAD_CERTIFICATES app setting.
The WEBSITE_LOAD_CERTIFICATES app setting makes the specified certificate accessible to your Linux hosted apps (including custom container apps) as files. The files are found under the following directories:
Private certificates - /var/ssl/private (.p12 files)
Public certificates - /var/ssl/certs (.der files)
Use the code sample below to load the specified certificate into your Linux hosted apps (including custom container apps):
using System;
using System.IO;
using System.Security.Cryptography.X509Certificates;
var bytes = File.ReadAllBytes($"/var/ssl/private/{Configuration["WEBSITE_LOAD_CERTIFICATES"]}.p12");
var cert = new X509Certificate2(bytes);
Signing Credentials
Here are the steps that I used to generate signing credentials:
Install OpenSSL.
Generate private key and public certificate.
Run openssl req -x509 -newkey rsa:4096 -sha256 -nodes -keyout example.com.key -out example.com.crt -subj "/CN=example.com" -days 365 replacing example.com with name of the site.
Combine the above into a single PFX file.
Run openssl pkcs12 -export -out example.com.pfx -inkey example.com.key -in example.com.crt replacing example.com with the name of the site.
Upload the PFX file to Azure.
In the Azure portal, from the left menu, select App Services > <app-name>.
From the left navigation of your app, select TLS/SSL settings, then select Private Key Certificates (.pfx), then upload the above PFX file.
Configure app settings.
Add the thumbprint of the PFX file above to the WEBSITE_LOAD_CERTIFICATES app setting in the App Service.
IdentityServer
The below code sample shows a complete Startup.cs configuration which could be used to get an IdentityServer application up and running:
namespace IdentityServer
{
public class Startup
{
public Startup(IConfiguration configuration, IWebHostEnvironment environment)
{
Configuration = configuration;
Environment = environment;
}
public IConfiguration Configuration { get; }
public IWebHostEnvironment Environment { get; }
// This method gets called by the runtime. Use this method to add services to the container.
// For more information on how to configure your application, visit https://go.microsoft.com/fwlink/?LinkID=398940
public void ConfigureServices(IServiceCollection services)
{
void ConfigureDbContext(DbContextOptionsBuilder builder)
{
builder.UseNpgsql(Configuration.GetConnectionString("DefaultConnection"));
}
var builder = services.AddIdentityServer()
.AddConfigurationStore(options => { options.ConfigureDbContext = ConfigureDbContext; })
.AddOperationalStore(options => { options.ConfigureDbContext = ConfigureDbContext; });
if (Environment.IsDevelopment())
{
builder.AddDeveloperSigningCredential();
}
else
{
try
{
var bytes = File.ReadAllBytes($"/var/ssl/private/{Configuration["WEBSITE_LOAD_CERTIFICATES"]}.p12");
var certificate = new X509Certificate2(bytes);
builder.AddSigningCredential(certificate);
}
catch (FileNotFoundException)
{
throw new Exception($"The certificate with the thumbprint \"{Configuration["WEBSITE_LOAD_CERTIFICATES"].Substring(0, 8)}...\" could not be found.");
}
}
}
// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
if (env.IsDevelopment()) app.UseDeveloperExceptionPage();
app.UseIdentityServer();
}
}
}
Clean Architecture
The below code sample shows a complete DependencyInjection.cs configuration which could be used to get a Clean Architecture application up and running:
namespace CleanArchitecture.Infrastructure
{
public static class DependencyInjection
{
public static IServiceCollection AddInfrastructure(this IServiceCollection services, IConfiguration configuration)
{
void ConfigureDbContext(DbContextOptionsBuilder builder)
{
if (configuration.GetValue<bool>("UseInMemoryDatabase"))
{
builder.UseInMemoryDatabase("CleanArchitectureDb");
}
else
{
builder.UseSqlServer(configuration.GetConnectionString("DefaultConnection"), b => b.MigrationsAssembly(typeof(ApplicationDbContext).Assembly.FullName));
}
}
services.AddDbContext<ApplicationDbContext>(ConfigureDbContext);
services.AddScoped<IApplicationDbContext>(provider => provider.GetService<ApplicationDbContext>());
services.AddScoped<IDomainEventService, DomainEventService>();
services.AddDefaultIdentity<ApplicationUser>()
.AddEntityFrameworkStores<ApplicationDbContext>();
var builder = services.AddIdentityServer()
.AddConfigurationStore(options => { options.ConfigureDbContext = ConfigureDbContext; })
.AddOperationalStore(options => { options.ConfigureDbContext = ConfigureDbContext; })
.AddAspNetIdentity<ApplicationUser>();
var bytes = File.ReadAllBytes($"/var/ssl/private/{Configuration["WEBSITE_LOAD_CERTIFICATES"]}.p12");
var certificate = new X509Certificate2(bytes);
builder.AddSigningCredential(certificate);
services.AddTransient<IDateTime, DateTimeService>();
services.AddTransient<IIdentityService, IdentityService>();
services.AddTransient<ICsvFileBuilder, CsvFileBuilder>();
services.AddAuthentication()
.AddIdentityServerJwt();
return services;
}
}
}
I think the problem is that your application in the container does not trust the locally created developer certificate. Its something you can only use on your machine because a dev root certificate is installed on your computer.
The container will never trust the certificate created by dotnet dev-certs.
You need to get a properly trusted certificate, for example from LetsEncrypt.
I faced this issue running a .net core spa template on a linux app service. I also created a self signed .pfx as described in tnc1997's answer. Although the answer can be pieced together, for me the gotchas were:
When referencing your certificate path, don't use the uploaded .pfx file name. Instead, as previously stated, your cert file gets a new name ".p12" and is (in linux container) found under "/var/ssl/private/".
Specify a blank password. Don't specify the password for the uploaded .pfx file. Instead set appsetting "IdentityServer__Key__Password" to "" (empty).
The .Net Clean Architecture is calling services.AddIdentityServer() .AddApiAuthorization<ApplicationUser, ApplicationDbContext>(); in DependencyInjection.cs which is extension method. This method internally calls bunch of other methods one of which is .AddSigningCredentials(). Unfortunately this default method will fail in Linux environment because it cannot read bare private keys. According to this issue you need to construct PFX yourself in Linux.
Solution in my view:
Remove .AddApiAuthorization<ApplicationUser, ApplicationDbContext>();
Write own method
var bytes = File.ReadAllBytes($"/var/ssl/private/{thump_print_goes_here}.p12");
var certificate = new X509Certificate2(bytes);
var builder = services.AddIdentityServer()
.AddAspNetIdentity<ApplicationUser>()
.AddOperationalStore<ApplicationDbContext>()
.AddIdentityResources()
.AddApiResources()
.AddClients()
.AddSigningCredential(certificate);
The solution provided in edit 2 works. I have tested and verified this with my .NET6 app in a Linux App Service on Azure.
My appsettings.Production.json looks like this:
{
"IdentityServer": {
"Key": {
"Type": "File",
"FilePath": "/var/ssl/private/{thumbprintGoesHere}.p12",
"Password": ""
}
}
}
I have created my SSL certificate and SigningKey certificate in Azure Key Vaults as self-signed keys following this guide https://learn.microsoft.com/en-us/aspnet/core/blazor/security/webassembly/hosted-with-identity-server?view=aspnetcore-5.0&tabs=visual-studio. I use the WEBSITE_LOAD_CERTIFICATES to load the certificates to the /var/ssl/private/ path.
I hope this helps someone struggling with the same issues.
I've seen both this and this — same problem, different question.
I'm trying to connect my Windows 8.1 Store app to an ASP.NET Web API web service, secured over HTTPS using a self-signed certificate. It's a proof-of-concept application that will end up on < 5 different machines and seen only internally, so I was planning to just install the certificate as trusted on each of the target machines.
When I try this on my development setup, both HttpClient APIs fail to establish the trust relationship when calling the service.
Windows.Web.Http.HttpClient exception: "The certificate authority is invalid or incorrect"
System.Net.Http.HttpClient exception: "The remote certificate is invalid according to the validation procedure."
My self-signed certificate (public-key-only .cer version) is installed in both the "User" and "Local Machine" Trusted Root Certification Authorities on the client. I'm really surprised that this isn't enough to get WinRT to trust it. Is there something I'm missing, or is there just no way to set up the trust relationship for a self-signed SSL certificate that will make HttpClient happy?
Details on my setup:
ASP.NET Web API
Azure web role running in Azure emulator
Cert issuer: 127.0.0.1
Cert subject: 127.0.0.1
Cert key: 2048-bit
Windows 8.1 Store application
Certificate (.cer file with public key only) installed in User\Trusted Root Certification Authorities
Certificate (.cer file with public key only) installed in Local Machine\Trusted Root Certification Authorities
Certificate (.cer file with public key only) added to Windows Store app manifest under "CA"
I am not asking for a workaround to configure HttpClient to accept self-signed or invalid certificates in general — I just want to configure a trust relationship with THIS one. Is this possible?
You should be able to find out what is the problem with the certificate by doing a request like this:
// using Windows.Web.Http;
private async void Foo()
{
HttpRequestMessage request = null;
try
{
request = new HttpRequestMessage(
HttpMethod.Get,
new Uri("https://localhost"));
HttpClient client = new HttpClient();
HttpResponseMessage response = await client.SendRequestAsync(request);
}
catch (Exception ex)
{
// Something like: 'Untrusted, InvalidName, RevocationFailure'
Debug.WriteLine(String.Join(
", ",
request.TransportInformation.ServerCertificateErrors));
}
}
Using a HttpBaseProtocolFilter you can ignore certificate errors:
// using Windows.Web.Http;
// using Windows.Web.Http.Filters;
// using Windows.Security.Cryptography.Certificates;;
HttpBaseProtocolFilter filter = new HttpBaseProtocolFilter();
filter.IgnorableServerCertificateErrors.Add(ChainValidationResult.Untrusted);
filter.IgnorableServerCertificateErrors.Add(ChainValidationResult.InvalidName);
filter.IgnorableServerCertificateErrors.Add(ChainValidationResult.RevocationFailure);
HttpClient client = new HttpClient(filter);
HttpResponseMessage response = await client.SendRequestAsync(request);
The piece I was missing turned out to be that the certificate wasn't in the list of of IIS Server Certificates on my local machine!
Opening IIS Manager and checking out the Server Certificates section, I did find a 127.0.0.1 SSL certificate already set up by the Azure emulator:
CN = 127.0.0.1
O = TESTING ONLY
OU = Windows Azure DevFabric
However, my own self-signed certificate that I made outside of IIS, also with CN=127.0.0.1, was not in the list. I imported it, and now my Windows Store app's HttpClient connects happily (certificate warnings went away in Chrome and IE as well!)
If anyone can firm up the technical details on this, please comment — this fix feels a bit magical and I'm not sure I can pinpoint precisely why this worked. Possibly some confusion on my part between the two certs for 127.0.0.1, even though the thumbprint I had configured in my Azure project was always the one I was intending to use?
Using the new Windows Azure SDK for .NET, I want to get a list of all virtual machines.
Piecing together the early documentation, here's what I came up with:
// This is a custom cert I made locally. I uploaded it to Azure's Management Certificates and added it to my local computer's cert store, see screenshot below.
X509Certificate2 myCustomCert = await this.GetAzureCert();
var credentials = new CertificateCloudCredentials(mySubscriptionId, myCustomCert);
using (var cloudServiceClient = CloudContext.Clients.CreateCloudServiceManagementClient(credentials))
{
credentials.InitializeServiceClient(cloudServiceClient); // Is this required? The next call fails either way.
// This line fails with exception: "The server failed to authenticate the request. Verify that the certificate is valid and is associated with this subscription."
var services = await cloudServiceClient.CloudServices.ListAsync(CancellationToken.None);
}
My first thought was the cert was bad, but I am able to successfully call the Azure REST API using my custom certificate. As you can see below, it is properly added to the Azure Management Certificates and associated with my subscription:
What am I doing wrong?
Here's another option - rather than upload a cert, try pulling your management cert out of your publishsettings file and using the X509Certificate's constructor that takes a byte[]. Then, pass that parameter the result of a call to Convert.FromBase64String, passing it the string representation of your management certificate from your publishsettings file.
Also, take a look at the Compute management client rather than the Cloud Service Management client. There are more features specific to the compute stack in that client at this time. The code below is a demonstration of such an approach. Note, my _subscriptionId and _managementCert fields are both strings, and I just hard-code them to the values from my publishsettings file as I described above.
public async static void ListVms()
{
using (var client = new ComputeManagementClient(
new CertificateCloudCredentials(_subscriptionId,
new X509Certificate2(Convert.FromBase64String(_managementCert)))
))
{
var result = await client.HostedServices.ListAsync();
result.ToList().ForEach(x => Console.WriteLine(x.ServiceName));
}
}
There's a parameterless ListAsync method that's an extension method. Try importing the Microsoft.WindowsAzure.Management namespace (or the Microsoft.WindowsAzure.Management.Compute namespace). Once you see the parameterless ListAsync method you should be good. I'll also mock up some code to resemble what you're trying to accomplish and offer up a more comprehensive answer by the end of the day.
I am writing a code in my web app which needs to list and search for a specific Certificate installed on web role.
Here is my code
// using System.Security.Cryptography.X509Certificates;
var store = new X509Store() ;
store.Open(OpenFlags.ReadOnly);
LoggingService.Info(String.Format(
"{0} Certificate(s) are found in store",store.Certificates.Count));
for(int index=0;index<store.Certificates.Count;index++)
{
LoggingService.Info(String.Format(
"Subject:{0}, Thumbprint:{1}",store.Certificates[index].Subject,
store.Certificates[index].Thumbprint));
}
_Certificate = store.Certificates.Find(
X509FindType.FindByThumbprint, this.CertificateThumbprint, false)[0];
Now Problem is, even though a certificate is added in the web role through portal and also present in the config file. store.Certificates.Count is zero. This code runs perfectly in emulator but somehow is unable to list the web role certificates. How can I access the certificate installed on web role?
Got it, I was not providing any store name and location in Store class and was assuming it will search in every store and location but it was not like this. I then provided the Store name and location and now system is able to find certificate.
I'm trying to enumerate certificates from Azure within a web role (running on the v1.3 Azure SDK Dev Fabric) but no certificates are returned when I use the following code. It's important to note, however, that code works fine when run from a console program:
private static void EnumCerts()
{
var selectedCerts = new X509Certificate2Collection();
var store = new X509Store(
StoreName.My, StoreLocation.CurrentUser);
try
{
store.Open(OpenFlags.OpenExistingOnly | OpenFlags.ReadOnly);
foreach (X509Certificate2 cert in store.Certificates)
Console.WriteLine(cert.Subject);
}
finally
{
store.Close();
}
}
My working assumption is that this is due to a security issue. My web role is running with Elevated Trust but I'm guessing that the IIS web instances are not.
In either case I have no idea how to solve the problem, so any help would be greatly appreciated...
Store the X509 certificate in the LocalMachine instead of CurrentUser. CurrentUser for an IIS process runs within the context of IIS-user who you likely have no access to.
Also, you want to make sure that you've imported the certificate on the Azure side into certificate store properly by remoting-in and verifying.
You should remote (RDP) into the role and check the store(s). I honestly don't know where Azure portal uploads certs, but I thought it was CurrentUser (which I think IIS runs under as well).
You can also enumerate the certificates via the Service Management API, which may or may not work for your solution. http://msdn.microsoft.com/en-us/library/ee795178.aspx
I ended up solving the problem by embedding the certificate in my service code then reading it from the resource:
using System.IO;
using System.Reflection;
using System.Security.Cryptography.X509Certificates;
namespace AcsTest.Shared
{
public static class CertHelper
{
public static X509Certificate2 GetCertFromManifest(
Assembly assembly, string certName, string password)
{
byte[] bytes;
using (var stream = assembly.
GetManifestResourceStream(certName))
{
bytes = new BinaryReader(stream).
ReadBytes((int)stream.Length);
}
return new X509Certificate2(bytes, password,
X509KeyStorageFlags.MachineKeySet);
}
}
}
The key insight was that I'd need to store a password in my service to load the cert from the certificate store so I was gaining no security advantage by keeping it in the store.