Differentiation between the namespace, path, mountpoint, etc - vaultsharp

The Hashicorp documentation leaves a lot to be desired when it comes to implementing a solution using .Net and the VaultSharp documentation isn't as comprehensive enough to cover the multitude of scenarios.
We have our Vault setup with a namespace, "egw". We have a KV Secrets Engine enabled with a name of "Expr". We have secrets listed at 3 different paths: "Trans", "Set" and "Serv".
We are unsure how to actually read these secrets as it is UNCLEAR the differentiation between the namespace, path, mountpoint, etc.
The documentation's all over the place and not clear to us on any of these terms and the sample apps are useless to us due to the wrong auth methods.
We are using LDAP Auth Method so we can login to our server without issues, it's just getting to the secrets that we're having issues with.
Can someone, please, explain to us how to read these secrets using VaultSharp?
Update:
We currently do NOT have roles created or assigned.
Can someone, please, help me to understand why this code fails to
either list the paths OR fetch the secrets? Am I doing something
incorrectly or just not understanding how it needs to be done?
IAuthMethodInfo authMethod = new LDAPAuthMethodInfo(_settings.LDAPUserName, _settings.LDAPPassword);
var vaultClientSettings = new VaultClientSettings(_settings.Address, authMethod);
IVaultClient vaultClient = new VaultClient(vaultClientSettings);
Secret<ListInfo> secret = await vaultClient.V1.Secrets.KeyValue.V2.ReadSecretPathsAsync("egw/Expr/data/");
ListInfo paths = secret.Data;
Secret<SecretData>? kv2Secret = await vaultClient.V1.Secrets.KeyValue.V2.ReadSecretAsync(path: "Expr/data/Trans", mountPoint:"egw/");
Dictionary<string, object> dataDictionary = kv2Secret.Data.Data;
This is the error message and StackTrace I am getting:
Message: 
Newtonsoft.Json.JsonReaderException : Unexpected character encountered while parsing value: <. Path '', line 0, position 0.
Stack Trace: 
JsonTextReader.ParseValue()
JsonReader.ReadAndMoveToContent()
JsonReader.ReadForType(JsonContract contract, Boolean hasConverter)
JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
JsonSerializer.Deserialize(JsonReader reader, Type objectType)
JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings)
JsonConvert.DeserializeObject[T](String value, JsonSerializerSettings settings)
JsonConvert.DeserializeObject[T](String value)
Polymath.MakeRequestAsync[TResponse](String resourcePath, HttpMethod httpMethod, Object requestData, IDictionary`2 headers, Boolean rawResponse, Action`1 postResponseAction)
Polymath.MakeVaultApiRequest[TResponse](String resourcePath, HttpMethod httpMethod, Object requestData, Boolean rawResponse, Action`1 postResponseAction, String wrapTimeToLive, Boolean unauthenticated)
LDAPAuthMethodLoginProvider.GetVaultTokenAsync()
Polymath.MakeVaultApiRequest[TResponse](String resourcePath, HttpMethod httpMethod, Object requestData, Boolean rawResponse, Action`1 postResponseAction, String wrapTimeToLive, Boolean unauthenticated)
Polymath.MakeVaultApiRequest[TResponse](String mountPoint, String path, HttpMethod httpMethod, Object requestData, Boolean rawResponse, Action`1 postResponseAction, String wrapTimeToLive, Boolean unauthenticated)
KeyValueSecretsEngineV2Provider.ReadSecretAsync(String path, Nullable`1 version, String mountPoint, String wrapTimeToLive)
Update2:
Found that using LDAP AuthMethod isn't working correctly, not sure if it's the way it's setup on the Vault or what. Began using the Token and was able to read the secrets but when trying to list them, I get permission denied

Namespaces provide a way for your Vault service to be fully self-managed. This is more an administrative detail than a programming detail, and you can largely disregard that. Ultimately, you just need to know what your namespace is called.
Your secrets engine is where the secrets actually reside, and how they are stored. For most cases, you're using a KV (key-value) secrets engine, version 2.
The paths you list just describe locations where your secrets reside in your secrets engine.
So with the information you've given:
Namespace is egw
Secrets engine is KV with name of Expr
Paths exist at Trans, Set, and Serv
...you'd probably be referencing it from these paths. Note that KV secrets engines store their values at the data/ path, so you have to include that after you request from the secrets engine.
egw/Expr/data/Trans
egw/Expr/data/Set
egw/Expr/data/Serv
As an access note: You're really going to want to consider using an AppRole to access these secrets if you're doing machine-to-machine communication. Doing all of that with LDAP is going to be complex and relies on LDAP and Vault being alive versus with AppRoles, which only needs Vault to be alive, and allows for very fine-grained access to secrets.

Related

OWASP ZAP API root element is missing error

I am writing a program on Windows using the OWASP ZAP API to automate the process of running ZAP against a target URL. So far, I was able to successfully start the application using the zap.bat file and the arguments -daemon -config api.disablekey=true. Starting the application, ZAP is able to listen on localhost:8080. I also created the ClientAPi object passing in these parameters:
ClientApi zaproxy = new ClientApi("127.0.0.1", 8080, null);
I am now trying to set the context using the includeInContext(string contextname, string regex) method from the Context class but am getting an error:
System.Exception: 'System.Xml.XmlException: Root element is missing.
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
at System.Xml.XmlTextReaderImpl.Read()
at System.Xml.XmlLoader.Load(XmlDocument doc, XmlReader reader, Boolean preserveWhitespace)
at System.Xml.XmlDocument.Load(XmlReader reader)
at System.Xml.XmlDocument.LoadXml(String xml)
at OWASPZAPDotNetAPI.ClientApi.CallApiRaw(String component, String operationType, String operationName, Dictionary 2 parameters)
at OWASPZAPDotNetAPI.ClientApi.CallApi(String component, String operationType, String operationName, Dictionary 2 parameters)
at OWASPZAPDotNetAPI.Generated.Context.includeInContext(String contextname, String regex)
However, when I start up the ZAP GUI in the background and then run the daemon and set the context, it seems to work. Can someone please let me know how to resolve the above error? Thanks!
Have a look at https://www.zaproxy.org/docs/automate/ - this lists the ways you can automate ZAP.
I suspect one of the packaged scans could be easier for you to use.

How to monitor progress of an asynchronous upload using the Azure BlobServiceClient

This seems like it should be really simple but, for whatever reason, the options just don't make sense to me.
I'm instantiating a BlobServiceClient:
this.FileClient = new BlobServiceClient(new Uri(ENDPOINT), credentials);
Then I get an instance of BlobContainerClient:
var container = this.FileClient.GetBlobContainerClient(this.GetContainerName(format));
Then I call UploadBlobAsync:
await container.UploadBlobAsync(filePath, content);
All I want to do is monitor the progress of the upload.
If you lookup the BlobClient class, you'll find there's a BlobUploadOptions class that has an ProgressHandler property of type IProgress which is exactly what I'm looking for...but it doesn't seem the BlobContainerClient uses that class in any way (why?). Also, if you look at the UploadAsync methods of the BlobClient class, there are 2 overloads of the method that except a parameter of type BlobUploadOptions....1 allows you to specify where to put the Blob you're uploading and the other allows you to specify the content you're uploading...but there doesn't appear to be an overload that allows you to specify both what you're uploading and, at the same time, where to put it. I'm so confused.
Also...I'm using 1 instance of the BlobServiceClient for the life of the application. Is that proper?
First of all, you should use BlobClient to monitor the uploading progress.
For your questions:
but it doesn't seem the BlobContainerClient uses that class in any way
(why?)
This is by design. Currently, the BlobContainerClient does not support this parameter. You can raise an feature request in the github page.
Also, if you look at the UploadAsync methods of the BlobClient class,
there are 2 overloads of the method that except a parameter of type
BlobUploadOptions......
I think you're referring these 2 methods:
UploadAsync(string path, BlobUploadOptions options, CancellationToken cancellationToken = default)
and
UploadAsync(Stream content, BlobUploadOptions options, CancellationToken cancellationToken = default).
You should know when to use them, then you may no confusion.
when using UploadAsync(string path, xxx), you should specify the path(like d:\myfolder\test.txt) for the file you're trying to upload.
when using UploadAsync(Stream content,xxx), it means that you have already got the stream of the file. For example, you use this method var mystream = File.OpenRead("file path") to get the stream of the file content, then pass the stream to this method UploadAsync(Stream content,xxx).
Also...I'm using 1 instance of the BlobServiceClient for the life of
the application. Is that proper?
Yes, it's ok.
For the usage of monitoring upload progress by using code, you can refer to this blog for more details.
Please let me know if you still have more issues.

ServiceStack.Redis.RedisPoolManagerPool.GetClient() - IndexOutOfRangeException

We're receiving the following error in ServiceStack.Redis v4.0.48
System.IndexOutOfRangeException: Index was outside the bounds of the array.
at ServiceStack.Redis.RedisManagerPool.GetClient()
at ServiceStack.Redis.RedisClientManagerCacheClient.Set[T](String key, T value, TimeSpan expiresIn)
at ServiceStack.ServiceExtensions.CacheSet[T](ICacheClient cache, String key, T value, Nullable1 expiresIn)
at ServiceStack.ServiceStackHost.OnSaveSession(IRequest httpReq, IAuthSession session, Nullable1 expiresIn)
at ServiceStack.Auth.OAuthProvider.Authenticate(IServiceBase authService, IAuthSession session, Authenticate request)
at ServiceStack.Auth.AuthenticateService.Authenticate(Authenticate request, String provider, IAuthSession session, IAuthProvider oAuthConfig)
at ServiceStack.Auth.AuthenticateService.Post(Authenticate request)
at ServiceStack.Host.ServiceRunner`1.Execute(IRequest request, Object instance, TRequest requestDto)
We've tried bumping pool count up to 128 (across 3 instances) but that didn't resolve. Hoping to attract the attention of someone who has encountered this and can provide some direction on what might be going on...
Please let me know what additional information is necessary to diagnose.
The first comment to upgrade to 4.0.52 from #mythz ultimately was the solution, however it is important to note that the upgrade changes the behavior of Redis configuration - particularly as it relates to Redis connection strings.
for 4.0.52 anyway, you must set default values on RedisConfig (e.g. RedisConfig.DefaultRetryTimeout), even if you set the equivalent values within the connection string.

Updated Table Returning Error When Query with ASP.NET Web API2 Service

I'm working on a WindowsPhone 8.1 app, and I have a SQL database connected to a Web API server hosted on an Azure website. Recently, the database person updated the database, added a new table, and added sample data to the tables that didn't already have any. When managing the database through Azure, I can see that all the tables have data in them, but when I try to retrieve them with the /api/entityname url, it returns an error message.
A few things to note, the database the server connects to has changed since I originally published it. I tried to run the enable migrations and update database commands again, but I don't know how to select the new database it's now connected to. Do I need to republish the server? Run the migrations commands for the new database? I'm hesitant try things involving the database connection on my own since I don't know much about it and don't want to mess up any configurations. I have a .NET backend. Thanks in advance for the help.
Update:
I went into the Web.Config file of my Services project and changed catalog value in the connection string to the name of my new database. Now I get the following error when I try to get the games values using the /api/teams extention:
An error has occurred.The
'ObjectContent1' type failed to serialize the response body for
content type 'application/xml;
charset=utf-8'.</ExceptionMessage><ExceptionType>System.InvalidOperationException</ExceptionType><StackTrace/><InnerException><Message>An
error has occurred.</Message><ExceptionMessage>The 'GameTime' property
on 'Game' could not be set to a 'System.TimeSpan' value. You must set
this property to a non-null value of type 'System.Byte[]'.
</ExceptionMessage><ExceptionType>System.InvalidOperationException</ExceptionType><StackTrace>
at
System.Data.Entity.Core.Common.Internal.Materialization.Shaper.ErrorHandlingValueReader1.GetValue(DbDataReader
reader, Int32 ordinal) at
System.Data.Entity.Core.Common.Internal.Materialization.Shaper.GetPropertyValueWithErrorHandling[TProperty](Int32
ordinal, String propertyName, String typeName) at
lambda_method(Closure , Shaper ) at
System.Data.Entity.Core.Common.Internal.Materialization.Shaper.HandleEntityAppendOnly[TEntity](Func2
constructEntityDelegate, EntityKey entityKey, EntitySet entitySet)
at lambda_method(Closure , Shaper ) at
System.Data.Entity.Core.Common.Internal.Materialization.Coordinator1.ReadNextElement(Shaper
shaper) at
System.Data.Entity.Core.Common.Internal.Materialization.Shaper1.SimpleEnumerator.MoveNext()
at System.Data.Entity.Internal.LazyEnumerator1.MoveNext() at
WriteArrayOfGameToXml(XmlWriterDelegator , Object ,
XmlObjectSerializerWriteContext , CollectionDataContract ) at
System.Runtime.Serialization.CollectionDataContract.WriteXmlValue(XmlWriterDelegator
xmlWriter, Object obj, XmlObjectSerializerWriteContext context) at
System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteDataContractValue(DataContract
dataContract, XmlWriterDelegator xmlWriter, Object obj,
RuntimeTypeHandle declaredTypeHandle) at
System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeWithoutXsiType(DataContract
dataContract, XmlWriterDelegator xmlWriter, Object obj,
RuntimeTypeHandle declaredTypeHandle) at
System.Runtime.Serialization.DataContractSerializer.InternalWriteObjectContent(XmlWriterDelegator
writer, Object graph, DataContractResolver dataContractResolver) at
System.Runtime.Serialization.DataContractSerializer.InternalWriteObject(XmlWriterDelegator
writer, Object graph, DataContractResolver dataContractResolver) at
System.Runtime.Serialization.XmlObjectSerializer.WriteObjectHandleExceptions(XmlWriterDelegator
writer, Object graph, DataContractResolver dataContractResolver) at
System.Runtime.Serialization.DataContractSerializer.WriteObject(XmlWriter
writer, Object graph) at
System.Net.Http.Formatting.XmlMediaTypeFormatter.WriteToStream(Type
type, Object value, Stream writeStream, HttpContent content) at
System.Net.Http.Formatting.XmlMediaTypeFormatter.WriteToStreamAsync(Type
type, Object value, Stream writeStream, HttpContent content,
TransportContext transportContext, CancellationToken
cancellationToken)
--- End of stack trace from previous location where exception was thrown --- at
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task
task) at
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task
task) at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at
System.Web.Http.WebHost.HttpControllerHandler.d__1b.MoveNext()
I think I'm making some progress, but I'm not very experienced with connecting apps to databases, so I'm hesitant to poke around. Thanks in advance for the help. I am using data-first migration (EF Designer from database). Also, the above error message is only displayed when I run the service locally via localhost. When I run it through it's hosted azurewebsites url, I get the following:
An error has occurred.
I assume this is due to my database updates not being published to the hosted site, correct? I'm using Firefox as my browser. If I use IE, I can't download or open the JSON file for the new entities.
With the help of a friend, I managed to fix this problem. In case anyone comes across this post down the line, the problem was due to conflicting data types between the database and the server's data objects. A list of data type mappings between C# and SQL Server can be found at https://msdn.microsoft.com/en-us/library/cc716729%28v=vs.110%29.aspx. Match the data types according to the linked chart, and ensure that any other database constraints (ie: required, etc) are included in annotations. Also, make sure that you update any code in your server's context file to match the data type changes. For instance, I had the following code in my OnModelCreating method:
modelBuilder.Entity<Game>()
.Property(e => e.GameTime)
.IsFixedLength();
Once I changed GameTime to a DateTime property,.IsFixedLength(); no longer applied to it, so I had to comment it out. As you make these changes, you can test them out by debugging your server locally, but you'll have to publish it back to the url it's hosted on in order see the changes take place on your remote server. In my case, I had to republish my azurewebsites.net site. I hope this helps anyone down the line that has a similar issue.

CryptographicException KeySet does not exists

I am using Thinktecture Identity server and running it in azure. The issue I am having is that sometimes when I read the Federation metadata XML file or while signing in I get this Keyset does not exists CryptoGraphic exception . Now i know these exceptions can be caused if we do not have proper permissions but the thing is that it just happens sometimes and other times it is working fine. I am not sure how to debug this. Can someone please lead me in some direction.
Also if someone has any idea about what kind of certificate is this system assembly trying to read and how it is configured that will be very helpful as well.
at System.Security.Cryptography.Utils.CreateProvHandle(CspParameters parameters, Boolean randomKeyContainer)
at System.Security.Cryptography.Utils.GetKeyPairHelper(CspAlgorithmType keyType, CspParameters parameters, Boolean randomKeyContainer, Int32 dwKeySize, SafeProvHandle& safeProvHandle, SafeKeyHandle& safeKeyHandle)
at System.Security.Cryptography.RSACryptoServiceProvider.GetKeyPair()
at System.Security.Cryptography.RSACryptoServiceProvider..ctor(Int32 dwKeySize, CspParameters parameters, Boolean useDefaultKeySize)
at System.Security.Cryptography.X509Certificates.X509Certificate2.get_PrivateKey()
at System.IdentityModel.Tokens.X509AsymmetricSecurityKey.get_PrivateKey()
at System.IdentityModel.Tokens.X509AsymmetricSecurityKey.GetSignatureFormatter(String algorithm)
at System.IdentityModel.SignedXml.ComputeSignature(SecurityKey signingKey)
at System.IdentityModel.EnvelopedSignatureWriter.ComputeSignature()
at System.IdentityModel.EnvelopedSignatureWriter.OnEndRootElement()
at System.IdentityModel.EnvelopedSignatureWriter.WriteEndElement()
at System.IdentityModel.Metadata.MetadataSerializer.WriteEntityDescriptor(XmlWriter inputWriter, EntityDescriptor entityDescriptor)
at System.IdentityModel.Metadata.MetadataSerializer.WriteMetadataCore(XmlWriter writer, MetadataBase metadataBase)
at System.IdentityModel.Metadata.MetadataSerializer.WriteMetadata(XmlWriter writer, MetadataBase metadata)
at PeachCourt.Auth.API.Protocols.FederationMetadata.WSFederationMetadataGenerator.Generate()
at Test.Auth.API.Protocols.FederationMetadata.FederationMetadataController.<Generate>b__1()
at Test.Auth.API.Helper.Cache.ReturnFromCache[T](ICacheRepository cacheRepository, String name, Int32 ttl, Func`1 action)
at Test.Auth.API.Protocols.FederationMetadata.FederationMetadataController.Generate()
It is trying to read the certificate that you have configured for token signing. The error message typically indicated that the worker process account has no read access to the private key.

Resources