My client wants to add the ability for users to define small scripts that can be used to run various calculations in a pre-existing web application. Currently, we're exploring using CSharpScripts in Microsoft.CodeAnalysis.Scripting.CSharp to dynamically compile and execute user functions.
I'd like the ability to sandbox these scripts. Specifically, I'd like to place limits on the resources (memory, etc) a script can use, as well as limit what libraries or values a script can access (eg I don't want a user making web service calls in their script). I want to avoid the case where a user writes code that negatively impacts the rest of the application.
I've included some example code below
private static Script<double> GetCompiledScript(string userFunction)
{
var options = ScriptOptions.Default.AddReferences(References).AddImports(Imports);
var script = CSharpScript.Create<double>(expression, options, typeof(Globals));
script.Compile();
return script;
}
public static String[] Imports => new[]
{
"System",
"System.Linq",
"System.Collections.Generic"
};
public static Assembly[] References => new[]
{
typeof(System.Object).GetTypeInfo().Assembly,
typeof(System.Linq.Enumerable).GetTypeInfo().Assembly,
};
public class Globals
{
public Dictionary<string, double> vars { get; set; }
}
Related
I want to create an Azure Durable Function that will download a CSV from the Internet and based on the data in this file, it will update my database using EntityFramework.
I set up the simple startup function that is triggered with TimeTrigger. This function is responsible for starting the orchestrator. The orchestrator executes multiple activities in parallel. There are around 40000 work items to be processed, so that's the number of activities that are triggered by orchestrator. Some of these activities will need to update the database (insert/update/delete rows). For this I need a database connection. I can configure DI in the StartUp in the following way:
public override void Configure(IFunctionsHostBuilder builder)
{
var connectionString = Environment.GetEnvironmentVariable("DefaultConnection");
builder.Services.AddDbContext<SqlContext>(options => options.UseSqlServer(connectionString));
builder.Services.AddScoped<IDbContext, SqlContext>();
}
}
However all my functions (orchestrator, activity function, etc.) are static and reside in a static class. I haven't seen any example where durable functions were defined in a non-static class and I had all kinds of problems when I tried that myself, so I assumed they must be static without diving too much into it.
I do not know how to pass my DbContext object to the Activity function, so it can update the data in the database when needed.
How should I resolve it?
I want to create an Azure Durable Function that will download a CSV from the Internet and based on the data in this file, it will update my database using EntityFramework.
Configure DI in the StartUp in the following way:
public override void Configure(IFunctionsHostBuilder builder) {
var connectionString = Environment.GetEnvironmentVariable("DefaultConnection");
builder.Services.AddDbContext<IDbContext, SqlContext>(options =>
options.UseSqlServer(connectionString)); //To inject DbContext
builder.Services.AddHttpClient(); //To inject HttpClient
}
Ensure you host your function app on Azure Functions Runtime V3+ so the class and methods don’t have to be static.
This will allow regular classes that have non-static constructors with injectable arguments
public class MyFunction {
private readonly HttpClient httpClient;
private readonly IDbContext dbContext;
//ctor
public MyFunction(IHttpClientFactory factory, IDbContext dbContext) {
httpClient = factory.CreateClient();
this.dbContext = dbContext;
}
[FunctionName("Function_Name_Here")]
public async Task Run(
[OrchestrationTrigger] IDurableOrchestrationContext context) {
// ... access dependencies here
}
// ... other functions, which can include static, but they wont
// have access to the instance fields.
}
This series of articles might be of some assistance to you
A Practical Guide to Azure Durable Functions — Part 2: Dependency Injection
I develop a ASP.NET Core application working with Azure Tables.
So, I created a tables storage account in Azure Portal, created a table, filled it with some test data, and now I would like to display the content of that table to test the reading.
my appsettings.json is
{
"ConnectionStrings": {
"MyTables":"DefaultEndpointsProtocol=https;AccountName=yyy;AccountKey=xxx;EndpointSuffix=core.windows.net"
},
"Logging": {
"IncludeScopes": false,
[etc etc...]
}
}
And my Startup.cs:
public class Startup
{
public Startup(IHostingEnvironment env)
{
var builder = new ConfigurationBuilder()
.SetBasePath(env.ContentRootPath)
.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
.AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true)
.AddEnvironmentVariables();
Configuration = builder.Build();
// here in debug we can see the connection string, that is OK
Console.WriteLine($"conn string:{Configuration["ConnectionStrings:MyTables"]}");
}
public IConfigurationRoot Configuration { get; }
// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
// Add framework services.
services.AddMvc();
}
And here is my controller I try to Display the values:
using Microsoft.AspNetCore.Mvc;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Table;
using NextMove.Models;
using System.Text;
[...]
public class HelloWorldController : Controller
{
public string ReadTables() {
// ????? Code does not work, as Startup not a reference
string myConnString = Startup.Configuration["ConnectionStrings:MyTables"];
//////////////////////////////////
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(myConnString);
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
CloudTable table = tableClient.GetTableReference("themes");
TableQuery<ProjectThemeEntity> query = new TableQuery<ProjectThemeEntity>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, "fr"));
StringBuilder response = new StringBuilder("Here is your test table:");
foreach (ProjectThemeEntity item in table.ExecuteQuery(query)) {
response.AppendLine($"Key: {item.RowKey}; Value: {item.Description}");
}
return response.ToString();
}
//
// GET: /HelloWorld/
public IActionResult Index() {
return View();
}
Questions:
a) How to fix this code in order to get the connection string?
b) There should be a "Table.ExecuteQuery(query)" as per this MSDN article in the controller's foreach, but it does not find such a method in CloudTable class, I however added the necessary references, as shown in the controller's code above, only two "Async" methods are available:
PS.
-For the (b) question several people has the same issue here, hope the situation changed now...
You can't access Startup.Configuration from the controller because it's not a static property. Even though you've made it public (generally not a good idea) it still requires you to have an instance of Startup to get access to it.
Generally to get access to settings in ASP.NET Core it's best to create a class with the properties you want and use the IOptions pattern to get them with Dependency Injection. In your startup where you configure your services (add services to the dependency injection container) you would use the helper methods to add your configuration object to the container and then in your controller you would specify you wanted an IOptions or IOptionsSnapshot to get access to it.
I'd suggest you don't put your data access in your controller though. It makes your controller harder to read and harder to maintain if you need to change your strategy later. Move your ReadTables method to its own class and add it to the DI container in Startup taking whatever settings you need to create the service. Use constructor injection in your controller to get the service and execute calls from your controller actions where you need them.
I want to Initialize a global Dictionary from Database in my web Api. Do i need to inject my DBContext in Global.Asax or Owin Startup. Any example would be much appreciated.
Any kind initialization purposes can be made in your custom defined OWIN Startup class class, like this:
using Microsoft.Owin;
using Microsoft.Owin.Security.OAuth;
using Owin;
using System;
[assembly: OwinStartup(typeof(WebAPIRestWithNest.Startup))]
namespace YourNamespace
{
public class Startup
{
public Dictionary<string, string> Table {get; private set;}
public void Configuration(IAppBuilder app)
{
// token generation
app.UseOAuthAuthorizationServer(new OAuthAuthorizationServerOptions
{
AllowInsecureHttp = false,
TokenEndpointPath = new PathString("/token"),
AccessTokenExpireTimeSpan = TimeSpan.FromHours(8),
Provider = new SimpleAuthorizationServerProvider()
});
// token consumption
app.UseOAuthBearerAuthentication(new OAuthBearerAuthenticationOptions());
app.UseWebApi(WebApiConfig.Register());
Table = ... Connect from DB and fill your table logic ...
}
}
}
After that you can use your Startup.Table property from your application.
In general, it is bad practice to access objects using static field in the asp.net applications because this may lead to bugs that are hardly detected and reproduced: especially this is true for non-immutable/not-thread-safe objects like Dictionary.
I assume you want to cache some DB data in memory to avoid excessive SQL queries. It is good idea to use standard asp.net caching for this purpose:
public IDictionary GetDict() {
var dict = HttpRuntime.Cache.Get("uniqueCacheKey") as IDictionary;
if (pvtData==null) {
dict = doLoadDictionaryFromDB(); // your code that loads data from DB
HttpRuntime.Cache.Add(cacheKey, dict,
null, Cache.NoAbsoluteExpiration,
new TimeSpan(0,5,0), // cache at least for 5 minutes after last access
CacheItemPriority.Normal, null);
}
return dict;
}
This approach allows you to select appropriate expiration policy (without the need to reinventing the wheel with static dictionary).
If you still want to use static dictionary, you can populate it on the application start (global.asax):
void Application_Start(object sender, EventArgs e)
{
// your code that initializes dictionary with data from DB
}
I have created a plugin which inspects a param in the query string and loads up a user object based on this ID and populates
any request DTO with it. (All my request DTO's inherit from BaseRequest which has a CurrentUser property)
public class CurrentUserPlugin : IPlugin
{
public IAppHost CurrentAppHost { get; set; }
public void Register(IAppHost appHost)
{
CurrentAppHost = appHost;
appHost.RequestFilters.Add(ProcessRequest);
}
public void ProcessRequest(IHttpRequest request, IHttpResponse response, object obj)
{
var requestDto = obj as BaseRequest;
if (requestDto == null) return;
if (request.QueryString["userid"] == null)
{
throw new ArgumentNullException("No userid provided");
}
var dataContext = CurrentAppHost.TryResolve<IDataContext>();
requestDto.CurrentUser = dataContext.FindOne<User>(ObjectId.Parse(requestDto.uid));
if (requestDto.CurrentUser == null)
{
throw new ArgumentNullException(string.Format("User [userid:{0}] not found", requestDto.uid));
}
}
}
I need to have this User object available in my services but I don't want to inspect the DTO every time and extract from there. Is there a way to make data from plugins globally available to my services? I am also wondering if there is another way of instantiating this object as for my unit tests, the Plugin is not run - as I call my service directly.
So, my question is, instead of using Plugins can I inject a user instance to my services at run time? I am already using IoC to inject different Data base handlers depending on running in test mode or not but I can't see how to achieve this for User object which would need to be instantiated at the beginning of each request.
Below is an example of how I inject my DataContext in appHost.
container.Register(x => new MongoContext(x.Resolve<MongoDatabase>()));
container.RegisterAutoWiredAs<MongoContext, IDataContext>();
Here is an example of my BaseService. Ideally I would like to have a CurrentUser property on my service also.
public class BaseService : Service
{
public BaseService(IDataContext dataContext, User user)
{
DataContext = dataContext;
CurrentUser = user; // How can this be injected at runtime?
}
public IDataContext DataContext { get; private set; }
public User CurrentUser { get; set; }
}
Have you thought about trying to use the IHttpRequest Items Dictionary to store objects. You can access these Items from any filter or service or anywhere you can access IHttpRequest. See the src for IHttpRequest.
Just be mindful of the order that your attributes, services and plugins execute and when you store the item in the Items dictionary.
Adding:
We don't want to use HttpContext inside of the Service because we want use Service in our tests directly.
Advantages for living without it
If you don't need to access the HTTP
Request context there is nothing stopping you from having your same
IService implementation processing requests from a message queue which
we've done for internal projects (which incidentally is the motivation
behind the asynconeway endpoint, to signal requests that are safe for
deferred execution).
http://www.servicestack.net/docs/framework/accessing-ihttprequest
And we don't use http calls to run tests.
So our solution is:
public class UserService
{
private readonly IDataContext _dataContext;
public UserService(IDataContext dataContext)
{
_dataContext = dataContext;
}
public User GetUser()
{
var uid = HttpContext.Current.Request.QueryString["userId"];
return _dataContext.Get<User>(uid);
}
}
and
container.Register(x => new UserService(x.Resolve<IDataContext>()).GetUser()).ReusedWithin(ReuseScope.Request);
This is service signature:
public SomeService(IDataContext dataContext, User user) { }
Any suggestions?
I need to have this User object available in my services but I don't want to inspect the DTO every time and extract from there
How will your application know about the user if you're not passing the 'userid' in the querystring? Could you store the user data in the Session? Using a Session assumes the client is connected to your app and persists a Session Id (ss-id or ss-pid cookie in ServiceStack) in the client that can be looked up on the Server to get the 'session data'. If you can use the Session you can retrieve the data from your service doing something like
base.Session["UserData"] or base.SessionAs<User>();
Note: you will need to save your User data to the Session
Is there a way to make data from plugins globally available to my services? but I can't see how to achieve this for User object which would need to be instantiated at the beginning of each request.
This sounds like you want a global request filter. You're kind of already doing this but you're wrapping it into a Plugin.
I'm new to WCF RIA Services, and have been working with LightSwitch for 4 or so months now.
I created a generic screen to be used for editing lookup tables all over my LightSwitch application, mostly to learn how to create a generic screen that can be used with different entity sets on a dynamic basis.
The screen is pretty simple:
Opened with arguments similar to this:
Application.ShowLookupTypesList("StatusTypes", "StatusTypeId"); which correspond to the entity set for the lookup table in the database.
Here's my WCF RIA service code:
using System.Data.Objects.DataClasses;
using System.Diagnostics;
using System.Reflection;
using System;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
using System.Data;
using System.Linq;
using System.ServiceModel.DomainServices.EntityFramework;
using System.ServiceModel.DomainServices.Server;
namespace WCF_RIA_Project
{
public class LookupType
{
[Key]
public int TypeId { get; set; }
public string Name { get; set; }
}
public static class EntityInfo
{
public static Type Type;
public static PropertyInfo Key;
public static PropertyInfo Set;
}
public class WCF_RIA_Service : LinqToEntitiesDomainService<WCSEntities>
{
public IQueryable<LookupType> GetLookupTypesByEntitySet(string EntitySetName, string KeyName)
{
EntityInfo.Set = ObjectContext.GetType().GetProperty(EntitySetName);
EntityInfo.Type = EntityInfo.Set.PropertyType.GetGenericArguments().First();
EntityInfo.Key = EntityInfo.Type.GetProperty(KeyName);
return GetTypes();
}
[Query(IsDefault = true)]
public IQueryable<LookupType> GetTypes()
{
var set = (IEnumerable<EntityObject>)EntityInfo.Set.GetValue(ObjectContext, null);
var types = from e in set
select new LookupType
{
TypeId = (int)EntityInfo.Key.GetValue(e, null),
Name = (string)EntityInfo.Type.GetProperty("Name").GetValue(e, null)
};
return types.AsQueryable();
}
public void InsertLookupType(LookupType lookupType)
{
dynamic e = Activator.CreateInstance(EntityInfo.Type);
EntityInfo.Key.SetValue(e, lookupType.TypeId, null);
e.Name = lookupType.Name;
dynamic set = EntityInfo.Set.GetValue(ObjectContext, null);
set.AddObject(e);
}
public void UpdateLookupType(LookupType currentLookupType)
{
var set = (IEnumerable<EntityObject>)EntityInfo.Set.GetValue(ObjectContext, null);
dynamic modified = set.FirstOrDefault(t => (int)EntityInfo.Key.GetValue(t, null) == currentLookupType.TypeId);
modified.Name = currentLookupType.Name;
}
public void DeleteLookupType(LookupType lookupType)
{
var set = (IEnumerable<EntityObject>)EntityInfo.Set.GetValue(ObjectContext, null);
var e = set.FirstOrDefault(t => (int)EntityInfo.Key.GetValue(t, null) == lookupType.TypeId);
Debug.Assert(e.EntityState != EntityState.Detached, "Entity was in a detached state.");
ObjectContext.ObjectStateManager.ChangeObjectState(e, EntityState.Deleted);
}
}
}
When I add an item to the list from the running screen, save it, then edit it and resave, I receive data conflict "Another user has deleted this record."
I can workaround this by reloading the query after save, but it's awkward.
If I remove, save, then readd and save an item with the same name I get unable to save data, "The context is already tracking a different entity with the same resource Uri."
Both of these problems only affect my generic screen using WCF RIA Services. When I build a ListDetail screen for a specific database entity there are no problems. It seems I'm missing some logic, any ideas?
I've learned that this the wrong approach to using LightSwitch.
There are several behind-the-scenes things this generic screen won't fully emulate and may not be do-able without quite a bit of work. The errors I've received are just one example. LightSwitch's built-in conflict resolution will also fail.
LS's RAD design means just creating a bunch of similar screens is the way to go, with some shared methods. If the actual layout needs changed across many identical screens at once, you can always find & replace the .lsml files if you're careful and make backups first. Note that modifying these files directly isn't supported.
I got that error recently. In my case I create a unique ID in my WCF RIA service, but in my screen behind code I must explicitly set a unique ID when I create the object that will later be passed to the WCF RIA Service insert method (this value will then be overwritten with the unique counter ID in the table of the underlying database).
See the sample code for this project:
http://lightswitchhelpwebsite.com/Blog/tabid/61/EntryId/157/A-Visual-Studio-LightSwitch-Picture-File-Manager.aspx