I am trying to get all assets from an azure media service account, Here is my code:
MediaContract mediaService = MediaService.create(MediaConfiguration.configureWithOAuthAuthentication(
mediaServiceUri, oAuthUri, AMSAccountName, AMSAccountKey, scope));
List<AssetInfo> info = mediaService.list(Asset.list());
However, this only gives me 1000 of them, and there are definitely more than that in the account.
In Azure table query, there is a token to be used to get more entries if there are more than 1000 of them.
Does anybody knows how I can get all assets for azure media service?
Thanks,
With Alex's help, i am able to hack the java-sdk the same way as this php implementation
Here are the codes:
List<AssetInfo> allAssets = new ArrayList<>();
int skip = 0;
while (true) {
List<AssetInfo> curAssets = mediaService.list(getAllAssetPage(skip));
if (curAssets.size() > 0) {
allAssets.addAll(curAssets);
if (curAssets.size() == 1000) {
System.out.println(String.format("Got %d assets.", allAssets.size()));
skip += 1000;
} else {
break;
}
} else {
break;
}
}
private static DefaultListOperation<AssetInfo> getAllAssetPage(int skip) {
return new DefaultListOperation<AssetInfo>("Assets",
new GenericType<ListResult<AssetInfo>>() {
}).setSkip(skip);
}
it is the built-in limit due to performance reasons (and REST v2), i believe. I think there is no way to retrieve all of them by one query.
It is possible, however, to use take and skip 1000 by 1000 etc.
But i see that you use MediaContract class, and i could not find it in the .NET repository - i guess it is Java one? I can't comment on that, but i believe the approach should be the same as described in the article (skip/take).
I have found the PHP implementation, maybe will be helpful.
https://msdn.microsoft.com/library/gg309461.aspx#BKMK_skip
Related
In general, how can I get useful debugging output about the decisions made by the various components of the Symfony2 security system during request processing? I would love to see things like what firewall and access_control statements were applied and why. What tools are there to make it easier to address the perennial "Why did I get redirected to the login form again" mystery?
You can use Blackfire if you need detailed debug information.
If its not sufficient then you can use WebProfilerBundle it has good debugging information.
If that also not work for you then you can create your own Data Collector Services.
Data Collectors are just like profiler extensions and they can help you to collect different data like routes, debug information or mailer data also. You can customize them according to your need.
Please check the documentation Here
Please check SecurityDebugBundle This will answer your all questions.
Use it carefully, as it requires different permissions.
By Reading its code you will understand how Data Collectors can help you out in debugging.
Hope that will help you.
Here is the DataCollecotr from SecurityDebugBundle:
class FirewallCollector
{
const HAS_RESPONSE = SecurityDebugDataCollector::DENIED;
private $securityContext;
private $container;
public function __construct(
SecurityContextInterface $securityContext,
Container $container
) {
$this->securityContext = $securityContext;
//Container dependency is a bad thing. This is to be refactored to a compiler pass
//where all the firewall providers will be fetched
$this->container = $container;
}
public function collect(Request $request, \Exception $exception)
{
$token = $this->securityContext->getToken();
if (!method_exists($token, 'getProviderKey')) {
return;
}
$providerKey = $token->getProviderKey();
$map = $this->container->get('security.firewall.map.context.' . $providerKey);
$firewallContext = $map->getContext();
$event = new GetResponseEvent(
new SimpleHttpKernel(),
$request,
HttpKernelInterface::MASTER_REQUEST
);
$firewalls = array();
foreach ($firewallContext[0] as $i => $listener) {
$firewalls[$i]= array('class' => get_class($listener), 'result' => SecurityDebugDataCollector::GRANTED);
try {
$listener->handle($event);
} catch (AccessDeniedException $ade) {
$firewalls[$i]['result'] = SecurityDebugDataCollector::DENIED;
break;
}
if ($event->hasResponse()) {
$firewalls[$i]['result'] = self::HAS_RESPONSE;
break;
}
}
return $firewalls;
}
}
This gives alot information on Firewall.
This Bundle Also contains SecurityDebugDataCollector and VotersCollector. So it can give information on all security components.
I'm attempting to modify the Azure-based Video Store sample app so that the front-end Ecommerce site can scale out.
Specifically, I want all instances of the web site to be notified of events like OrderPlaced so that no matter which web server the client web app happens to be connected to via SignalR, it will correctly receive the notification and update the UI.
Below is my current configuration in the Global.asax:
Feature.Disable<TimeoutManager>();
Configure.ScaleOut(s => s.UseUniqueBrokerQueuePerMachine());
startableBus = Configure.With()
.DefaultBuilder()
.TraceLogger()
.UseTransport<AzureServiceBus>()
.PurgeOnStartup(true)
.UnicastBus()
.RunHandlersUnderIncomingPrincipal(false)
.RijndaelEncryptionService()
.CreateBus();
Configure.Instance.ForInstallationOn<Windows>().Install();
bus = startableBus.Start();
And I've also configured the Azure Service Bus queues using:
class AzureServiceBusConfiguration : IProvideConfiguration<NServiceBus.Config.AzureServiceBusQueueConfig>
{
public AzureServiceBusQueueConfig GetConfiguration()
{
return new AzureServiceBusQueueConfig()
{
QueuePerInstance = true
};
}
}
I've set the web role to scale to two instances, and as expected, two queues (ecommerce and ecommerce-1) are created. I do not, however, see additional topic subscriptions being created under the videostore.sales.events topic. Instead, I see:
I would think that you would see VideoStore.ECommerce-1.OrderCancelled and VideoStore.ECommerce-1.OrderPlaced subscriptions under the Videostore.Sales.Events topic. Or is that not how subscriptions are stored when using Azure Service Bus?
What am I missing here? I get the event on one of the ecommerce instances, but never on both. Even if this isn't the correct way to scale out SignalR, my use case extends to stuff like cache invalidation.
I also find it strange that two error and audit queues are being created. Why would that happen?
UPDATE
Yves is correct. The AzureServiceBusSubscriptionNamingConvention was not applying the correct individualized name. I was able to fix this by implementing the following EndpointConfig:
namespace VideoStore.ECommerce
{
public class EndpointConfig : IConfigureThisEndpoint, IWantCustomInitialization
{
public void Init()
{
AzureServiceBusSubscriptionNamingConvention.Apply = BuildSubscriptionName;
AzureServiceBusSubscriptionNamingConvention.ApplyFullNameConvention = BuildSubscriptionName;
}
private static string BuildSubscriptionName(Type eventType)
{
var subscriptionName = eventType != null ? Configure.EndpointName + "." + eventType.Name : Configure.EndpointName;
if (subscriptionName.Length >= 50)
subscriptionName = new DeterministicGuidBuilder().Build(subscriptionName).ToString();
if (!SettingsHolder.GetOrDefault<bool>("ScaleOut.UseSingleBrokerQueue"))
subscriptionName = Individualize(subscriptionName);
return subscriptionName;
}
public static string Individualize(string queueName)
{
var parser = new ConnectionStringParser();
var individualQueueName = queueName;
if (SafeRoleEnvironment.IsAvailable)
{
var index = parser.ParseIndexFrom(SafeRoleEnvironment.CurrentRoleInstanceId);
var currentQueue = parser.ParseQueueNameFrom(queueName);
if (!currentQueue.EndsWith("-" + index.ToString(CultureInfo.InvariantCulture))) //individualize can be applied multiple times
{
individualQueueName = currentQueue
+ (index > 0 ? "-" : "")
+ (index > 0 ? index.ToString(CultureInfo.InvariantCulture) : "");
}
if (queueName.Contains("#"))
individualQueueName += "#" + parser.ParseNamespaceFrom(queueName);
}
return individualQueueName;
}
}
}
I could not, however, get NServiceBus to recognize my EndpointConfig class. Instead, I had to call it manually before starting the bus. From my Global.asax.cs:
new EndpointConfig().Init();
bus = startableBus.Start();
Once I did this, the subscription names appeared as expected:
Not sure why it's ignoring my IConfigureThisEndpoint, but this works.
This sounds like a bug, can you raise a github issue on this at https://github.com/Particular/NServiceBus.Azure
That said, I think it's better to use signalr's scaleout feature instead of using QueuePerInstance as signalr needs to replicate other information like (connection/group mappings) internally as well when running in scaleout mode.
Update:
I think I see the issue, the subscriptions should be individualised as well, which isn't the case in current naming conventions
https://github.com/Particular/NServiceBus.Azure/blob/master/src/NServiceBus.Azure.Transports.WindowsAzureServiceBus/NamingConventions/AzureServiceBusSubscriptionNamingConvention.cs
while it is in the queuenamingconventions
https://github.com/Particular/NServiceBus.Azure/blob/master/src/NServiceBus.Azure.Transports.WindowsAzureServiceBus/NamingConventions/AzureServiceBusQueueNamingConvention.cs#L27
As these conventions are public you can override them to work around the problem by changing the func in IWantCustomInitialization until I can get a fix in, just copy the current method and add the individualizer logic. The queue individualizer is internal though, so you'll have to copy that class from
https://github.com/Particular/NServiceBus.Azure/blob/master/src/NServiceBus.Azure.Transports.WindowsAzureServiceBus/Config/QueueIndividualizer.cs
Can anybody please tell me how can I programmatically get Azure service pricing details (pricing for Compute, Data Services , App Services, Network Services) from Azure website?
Does Azure provide the pricing details in JSON format?
Windows Azure does'not provide any such API as of today, although it is a much asked feature and hopefully they are working on it.
Check here:
http://feedback.windowsazure.com/forums/170030-billing/suggestions/1143971-billing-usage-api#comments
The only way for now could be to build your own data store with details mentioned here : http://azure.microsoft.com/en-us/pricing/calculator/
Unit wise price will be mentioned in the usage data csv, but unfortunately the only way for now is to download this csv for your subscription here: https://account.windowsazure.com/Subscriptions
Azure now provides API's to get usage and billing data. You can have a look at this blog which gives an overview of these API's and the feedback form here which contains links to some useful pages.
In summary use the following API's to get usage and billing data:
Resource usage
Resource ratecard
Not sure, if i am too late to answer.
I was looking for the same thing and stumble upon this post on stack overflow: Azure pricing calculator api. I was able to generate JSON string using this git hub repo: https://github.com/Azure-Samples/billing-dotnet-ratecard-api.
Hope this helps!
Late to the party but I found myself looking for this and nothing here got me what I wanted. Then I found this https://learn.microsoft.com/en-us/rest/api/cost-management/retail-prices/azure-retail-prices
It is pretty straight forward. Add the reference to the Json.NET .NET 4.0 to your project It shows up in your references as Newtonsoft.Json
//You will need to add these usings
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using System.Net.Http;
private void btnGetRates_Click(object sender, EventArgs e)
{
string strUrl = "https://prices.azure.com/api/retail/prices?$filter=serviceName eq 'Virtual Machines' and skuName eq 'E64 v4' and reservationTerm eq '3 Years'";
string response = GetDataFromAPI(strUrl);
// Here is am turning the Json response into a datatable and then loading that into a DataGridView.
//You can use the Json response any way you wish
DataTable dt = Tabulate(response);
dgvAzureSKU.DataSource = null;
dgvAzureSKU.DataSource = dt;
}
public string GetDataFromAPI(string url)
{
using (var httpClient = new HttpClient())
{
httpClient.DefaultRequestHeaders.Add("Accept", "application/json");
var response = httpClient.GetStringAsync(new Uri(url)).Result;
return response;
}
}
public static DataTable Tabulate(string json)
{
var jsonLinq = JObject.Parse(json);
// Find the first array using Linq
var srcArray = jsonLinq.Descendants().Where(d => d is JArray).First();
var trgArray = new JArray();
foreach (JObject row in srcArray.Children<JObject>())
{
var cleanRow = new JObject();
foreach (JProperty column in row.Properties())
{
if (column.Value is JValue) // Only include JValue types
{
cleanRow.Add(column.Name, column.Value);
}
}
trgArray.Add(cleanRow);
}
return JsonConvert.DeserializeObject<DataTable>(trgArray.ToString()); //This is what loads the data into the table
}
You can find some examples for that here https://learn.microsoft.com/en-us/azure/billing/billing-usage-rate-card-overview. Azure provides invoice, usage and ratecard APIs which can help you to do things like:
Azure spend during the month
Set up alerts
Predict bill
Pre-consumption cost analysis
I'm trying to make my MVC4-website check to see if people should be alerted with an email because they haven't done something.
I'm having a hard time figuring out how to approach this. I checked if the shared hosting platform would allow me to activate some sort of cronjob, but this is not available.
So now my idea is to perform this check on each page-request, which already seems suboptimal (because of the overhead). But I thought that with using an async it would not be in the way of people just visiting the site.
I first tried to do this in the Application_BeginRequest method in Global.asax, but then it gets called multiple times per page-request, so that didn't work.
Next I found that I can make a Global Filter which executes on OnResultExecuted, which would seemed promising, but still it's no go.
The problem I get there is that I'm using MVCMailer to send the mails, and when I execute it I get the error: {"Value cannot be null.\r\nParameter name: httpContext"}
This probably means that mailer needs the context.
The code I now have in my global filter is the following:
public override void OnResultExecuted(ResultExecutedContext filterContext)
{
base.OnResultExecuted(filterContext);
HandleEmptyProfileAlerts();
}
private void HandleEmptyProfileAlerts()
{
new Thread(() =>
{
bool active = false;
new UserMailer().AlertFirst("bla#bla.com").Send();
DB db = new DB();
DateTime CutoffDate = DateTime.Now.AddDays(-5);
var ProfilesToAlert = db.UserProfiles.Where(x => x.CreatedOn < CutoffDate && !x.ProfileActive && x.AlertsSent.Where(y => y.AlertType == "First").Count() == 0).ToList();
foreach (UserProfile up in ProfilesToAlert)
{
if (active)
{
new UserMailer().AlertFirst(up.UserName).Send();
up.AlertsSent.Add(new UserAlert { AlertType = "First", DateSent = DateTime.Now, UserProfileID = up.UserId });
}
else
System.Diagnostics.Debug.WriteLine(up.UserName);
}
db.SaveChanges();
}).Start();
}
So my question is, am I going about this the right way, and if so, how can I make sure that MVCMailer gets the right context?
The usual way to do this kind of thing is to have a single background thread that periodically does the checks you're interested in.
You would start the thread from Application_Start(). It's common to use a database to queue and store work items, although it can also be done in memory if it's better for your app.
Assume we have a report called SalesSummary for a large department. This department has many smaller teams for each product. People should be able to see information about their own product, not other teams' products. We also have one domain group for each of these teams.
Copying SalesSummary report for each team and setting the permission is not the best option since we have many products. I was thinking to use a code similar to below on RS, but it doesn't work. Apparently, System.Security.Principal.WindowsPrincipal is disabled by default on RS.
Public Function isPermitted() As Boolean
Dim Principal As New System.Security.Principal.WindowsPrincipal(System.Security.Principal.WindowsIdentity.GetCurrent())
If (Principal.IsInRole("group_prod")) Then
Return true
Else
Return false
End If
End Function
I also thought I can send the userID from RS to SQL server, and inside my SP I can use a code similar to below to query active directory. This also doesn't work due to security restriction.
SELECT
*
FROM OPENQUERY(ADSI,'SELECT cn, ADsPath FROM ''LDAP://DC=Fabricam,DC=com'' WHERE objectCategory=''group''')
Is there any easier way to achieve this goal?
Thanks for the help!
The first option you suggested (using embedded code to identify the executing user) will not be reliable. SSRS code is not necessarily executed as the user accessing the report, and may not have access to that users credentials, such as when running a subscription.
Your second approach will work, but requires the appropriate permissions for your SQL server service account to query Active Directory.
Another approach is to maintain a copy of the group membership or user permissions in a SQL table. This table can be updated by hand or with an automated process. Then you can easily incorporate this into both available parameters and core data queries.
So I ended up with this code:
PrincipalContext domain = new PrincipalContext(ContextType.Domain, "AD");
UserPrincipal user = UserPrincipal.FindByIdentity(domain, identityName);
//// if found - grab its groups
if (user != null)
{
PrincipalSearchResult<Principal> _groups = null;
int tries = 0;
//We have this while because GetGroups sometimes fails! Specially if you don't
// mention the domain in PrincipalContext
while (true)
{
try
{
_groups = user.GetGroups();
break;
}
catch (Exception ex)
{
logger.Debug("get groups failed", ex);
if (tries > 5) throw;
tries++;
}
}
// iterate over all groups, just gets groups related to this app
foreach (Principal p in _groups)
{
// make sure to add only group principals
if (p is GroupPrincipal)
{
if (p.Name.StartsWith(GROUP_IDENTIFIER))
{
this.groups.Add((GroupPrincipal)p);
this.groupNames.Add(p.Name);
}
}
}
}
Now, that you have a list of related group you can check the list to authorize the user!