Azure SQL Sync is a resource drain on laptop despite frequency setting - azure

We have a SQL server running on a development laptop and would like to we'd like to deploy multiple laptops and use Azure Sync to distribute changes to each user. Performance is too slow to have a remote SQL database with our current application -- that's a separate issue. Speed isn't critical and I don't anticipate collisions between users. I set the update to 12 hours and conflict resolution to member win.
Everything seems to be working as intended except the Microsoft SQL Data Sync 2.0 Windows service process is continuously consuming 2%-5% of the CPU and streaming data at about 80 kbps continuously. I'm worried this will drain the batteries when the laptops are in the field. Is there a better way to do this?
Here's the resource utilization from the Azure database. I stopped the sync at some point to see if the frequency setting would automatically restart it (it does not).

You can write your own sync application based on your needs. This and this resources provides you a good guidance. Below you will see a sample application.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data.SqlClient;
using Microsoft.Synchronization.Data.SqlServer;
using Microsoft.Synchronization.Data;
using Microsoft.Synchronization;
namespace SQLAzureDataSync
{
class Program
{
public static string sqlazureConnectionString = "Server=[Your SQL Azure Server].database.windows.net;Database=AdventureWorksLTSQLAzure;User ID=[Your SQL Azure User Name]#[Your SQL Azure Server];Password=[Your SQL Azure Password];Trusted_Connection=False;Encrypt=True;";
public static string sqllocalConnectionString = "Server=(local);Database=AdventureWorksLT2008;Trusted_Connection=True";
public static readonly string scopeName = "alltablesyncgroup";
static void Main(string[] args)
{
// Test if input arguments were supplied:
if (args.Length == 0)
{
System.Console.WriteLine("Please enter an argument.");
System.Console.WriteLine("Usage: SyncTest.exe -setup");
System.Console.WriteLine(" SyncTest.exe -sync");
}
else if (args[0] == "-setup")
Setup();
else if (args[0] == "-sync")
Sync();
}
public static void Setup()
{
try
{
SqlConnection sqlServerConn = new SqlConnection(sqllocalConnectionString);
SqlConnection sqlAzureConn = new SqlConnection(sqlazureConnectionString);
DbSyncScopeDescription myScope = new DbSyncScopeDescription(scopeName);
DbSyncTableDescription Customer = SqlSyncDescriptionBuilder.GetDescriptionForTable("Customer", sqlServerConn);
DbSyncTableDescription Product = SqlSyncDescriptionBuilder.GetDescriptionForTable("Product", sqlServerConn);
// Add the tables from above to the scope
myScope.Tables.Add(Customer);
myScope.Tables.Add(Product);
// Setup SQL Server for sync
SqlSyncScopeProvisioning sqlServerProv = new SqlSyncScopeProvisioning(sqlServerConn, myScope);
if (!sqlServerProv.ScopeExists(scopeName))
{
// Apply the scope provisioning.
Console.WriteLine("Provisioning SQL Server for sync " + DateTime.Now);
sqlServerProv.Apply();
Console.WriteLine("Done Provisioning SQL Server for sync " + DateTime.Now);
}
else
Console.WriteLine("SQL Server Database server already provisioned for sync " + DateTime.Now);
// Setup SQL Azure for sync
SqlSyncScopeProvisioning sqlAzureProv = new SqlSyncScopeProvisioning(sqlAzureConn, myScope);
if (!sqlAzureProv.ScopeExists(scopeName))
{
// Apply the scope provisioning.
Console.WriteLine("Provisioning SQL Azure for sync " + DateTime.Now);
sqlAzureProv.Apply();
Console.WriteLine("Done Provisioning SQL Azure for sync " + DateTime.Now);
}
else
Console.WriteLine("SQL Azure Database server already provisioned for sync " + DateTime.Now);
sqlAzureConn.Close();
sqlServerConn.Close();
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
}
public static void Sync()
{
try
{
SqlConnection sqlServerConn = new SqlConnection(sqllocalConnectionString);
SqlConnection sqlAzureConn = new SqlConnection(sqlazureConnectionString);
SyncOrchestrator orch = new SyncOrchestrator
{
LocalProvider = new SqlSyncProvider(scopeName, sqlAzureConn),
RemoteProvider = new SqlSyncProvider(scopeName, sqlServerConn),
Direction = SyncDirectionOrder.UploadAndDownload
};
Console.WriteLine("ScopeName={0} ", scopeName.ToUpper());
Console.WriteLine("Starting Sync " + DateTime.Now);
ShowStatistics(orch.Synchronize());
sqlAzureConn.Close();
sqlServerConn.Close();
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
}
public static void ShowStatistics(SyncOperationStatistics syncStats)
{
string message;
message = "\tSync Start Time :" + syncStats.SyncStartTime.ToString();
Console.WriteLine(message);
message = "\tSync End Time :" + syncStats.SyncEndTime.ToString();
Console.WriteLine(message);
message = "\tUpload Changes Applied :" + syncStats.UploadChangesApplied.ToString();
Console.WriteLine(message);
message = "\tUpload Changes Failed :" + syncStats.UploadChangesFailed.ToString();
Console.WriteLine(message);
message = "\tUpload Changes Total :" + syncStats.UploadChangesTotal.ToString();
Console.WriteLine(message);
message = "\tDownload Changes Applied :" + syncStats.DownloadChangesApplied.ToString();
Console.WriteLine(message);
message = "\tDownload Changes Failed :" + syncStats.DownloadChangesFailed.ToString();
Console.WriteLine(message);
message = "\tDownload Changes Total :" + syncStats.DownloadChangesTotal.ToString();
Console.WriteLine(message);
}
}
}

Related

Azure Service Bus Throttling

We have an architecture built around service bus, which is working well. We are currently also using the same name-spaced service bus to do our logging via a bespoke Serilog sink (which is going to be replaced with datadog).
This has caused us in our test environment, to occasionally be throttled via service bus (we are on standard and not premium). We know that its a problem with our side and are working to resolve it.
However, I want to update our code to handle the throttling exception more gracefully and potentially add a delay before it tries again.
What is not clear from the documentation is the exception details:
https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-throttling
We currently catch MessagingException - its not clear if this message will be thrown.
Has anyone handled throttled exceptions in this way?
At the moment the resolution seems to be catch the error, check the exception message against the exception message (I think looking at the errorcode is the least tightly coupled implementation) and put it dedicated code to react to it.
We have few exceptions in service bus such as QuotaExceededExceptions, ServerBusyExceptions, MessageSizeExceededExceptions, TransactionSizeExceededExceptions.
And in some throttling conditions there are several retries to ensure to deliver the messages at the end.
With Azure Monitor we can handle the throttles from metrics:
Below are few steps which I went through:
Can check throttle requests
We can select ServiceBusThrottling
You can check this blog to understand about handling throttle calls with Azure functions for Service bus.
Below is the sample code:
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.ServiceBus.Messaging;
using System.Data.SqlClient;
using System.Text;
using System;
namespace BallardChalmers.BackgroundFunctions
{
public static class BallardChalmersBackgroundTaskCreated
{
[FunctionName("BallardChalmersBackgroundTaskCreated")]
public static void
Run([ServiceBusTrigger("BallardChalmersBackgroundTaskCreated", AccessRights.Send, Connection = "ServiceBusConnection")]string queueMessage, TraceWriter log)
{
log.Info($"Service_TaskManager task added to queue with ID: {queueMessage}");
try
{
string connectionString = System.Configuration.ConfigurationManager.AppSettings["SQLConnectionString"]; ;
using (SqlConnection connection = new SqlConnection(connectionString))
{
// Create wait period between 1 and 60 seconds
System.Random random = new System.Random();
int waitPeriod = random.Next(1, 60);
log.Info($"Wait period: {waitPeriod.ToString()}");
connection.Open();
StringBuilder sb = new StringBuilder();
sb.Append("INSERT FunctionsLog ");
sb.Append("SELECT '" + queueMessage + "','Started','" + DateTime.Now.ToString() + "','" + DateTime.Now.ToString() + "'," + waitPeriod.ToString());
String sql = sb.ToString();
log.Info($"Insert command: {sql}");
using (SqlCommand command = new SqlCommand(sql, connection))
{
command.ExecuteNonQuery();
}
log.Info($"Inserted to FunctionsLog");
System.Threading.Thread.Sleep(1000 * waitPeriod);
log.Info($"Wait completed");
sb = new StringBuilder();
sb.Append("UPDATE FunctionsLog ");
sb.Append("SET [Status]='Completed', Updated='" + DateTime.Now.ToString() + "'");
sb.Append("WHERE ID='" + queueMessage + "'");
sql = sb.ToString();
log.Info($"Update command: {sql}");
using (SqlCommand command = new SqlCommand(sql, connection))
{
command.ExecuteNonQuery();
}
log.Info($"Updated to FunctionsLog");
}
}
catch (SqlException e)
{
log.Error($"Error writing to database: {e.Message + ":::" + e.StackTrace}");
}
}
}
}

How to connect Azure database with Active Directory-Universal with MFA Support .Net Core

I am trying to connect to azure database.
My current connection string
"return $"Password={this.Password}; Persist Security Info=True;User ID = { this.User }; Initial Catalog = { this.Database }; Data Source = { this.Server }";" like this. How can I connect to azure database with Active Directory-Universal with MFA Support
If you want to connect Azure SQL Database with Active Directory-Universal with MFA, you can connect your SQL database with Azure AD access token. For example
1. Register a web application
Configure permissions
Code( I use ADAL to get access token)
static void Main(string[] args)
{
string authory = "https://login.microsoftonline.com/hanxia.onmicrosoft.com";
AuthenticationContext authContext = new AuthenticationContext(authory);
Console.WriteLine("get token");
var result = GetTokenViaCode(authContext).Result;
var connection = new SqlConnection("Data Source=[my database].database.windows.net;Initial Catalog=[my initial catalog];");
connection.AccessToken = result.AccessToken;
connection.Open();
Console.WriteLine();
}
static async Task<AuthenticationResult> GetTokenViaCode(AuthenticationContext ctx)
{
string resource = "https://database.windows.net";
string clientId = "2c4aae8f-392c-419a-b454-8f8c1ff1ec0c";
AuthenticationResult result = null;
try
{
DeviceCodeResult codeResult = await ctx.AcquireDeviceCodeAsync(resource, clientId);
Console.ResetColor();
Console.WriteLine("You need to sign in.");
Console.WriteLine("Message: " + codeResult.Message + "\n");
result = await ctx.AcquireTokenByDeviceCodeAsync(codeResult);
}
catch (Exception exc)
{
Console.ForegroundColor = ConsoleColor.Red;
Console.WriteLine("Something went wrong.");
Console.WriteLine("Message: " + exc.Message + "\n");
}
return result;
}
Please note that I test it in console application, so I use device code flow to get access token. If you want to use it in web app, you can use OpenID flow to implememnt it. For more deatils, please refer to the sample.

Is there a way to make a comment in TFS when adding an AD user to a TFS group?

I was wondering if I can make a comment somewhere in Team Foundation Server when I add an AD user to a TFS group or change the group of the user, for auditing purpose.
I have created a PowerShell script to record day2day changes to TFS user databases and query the AD to find out who approved this change.
Now we have a self made db.
For what it's worth - I wrote a C# method to pull the list of all members from every TFS group. Hope that helps!
EDIT: "This is posted as an answer because "xidada" was going to write a script to pull the information and since I already had the code to get the information that needed, I thought the code would be a guide to help him/her with the script"
private void btn_GetNow_Click()
{
TfsTeamProjectCollection tfs = new TfsTeamProjectCollection(new Uri("http://server/collection"));
tfs.EnsureAuthenticated();
TfsConfigurationServer srv = tfs.ConfigurationServer;
CatalogNode configurationServerNode = srv.CatalogNode;
// Query the children of the configuration server node for all of the team project collection nodes
ReadOnlyCollection<CatalogNode> tpcNodes = configurationServerNode.QueryChildren(
new Guid[] { CatalogResourceTypes.ProjectCollection },
false,
CatalogQueryOptions.None
);
Guid tpcId = new Guid(tpcNodes[0].Resource.Properties["InstanceId"]);
TfsTeamProjectCollection tpc = srv.GetTeamProjectCollection(tpcId);
// get a reference to the work item tracking service
var workItemStore = tpc.GetService<WorkItemStore>();
List<Identity> result = new List<Identity>();
// iterate over the projects
foreach (Project project in workItemStore.Projects)
{
Console.WriteLine("\tProject: {0}", project.Name);
try
{
VersionControlServer versionControl = (VersionControlServer)tpc.GetService(typeof(VersionControlServer));
TeamProject teamProject = versionControl.GetTeamProject(project.Name);
IGroupSecurityService gss = (IGroupSecurityService)tpc.GetService<IGroupSecurityService>();
Identity[] appGroups = gss.ListApplicationGroups(teamProject.ArtifactUri.AbsoluteUri);
foreach (Identity group in appGroups)
{
rtb_Users.AppendText(group.DisplayName + "\n");
Identity[] groupMembers = gss.ReadIdentities(SearchFactor.Sid, new string[] { group.Sid }, QueryMembership.Expanded);
foreach (Identity member in groupMembers)
{
if (member.Members != null)
{
foreach (string memberSid in member.Members)
{
Identity memberInfo = gss.ReadIdentity(SearchFactor.Sid, memberSid, QueryMembership.None);
if (memberInfo.Type == IdentityType.WindowsUser)
{
if (!result.Contains(memberInfo))
{
result.Add(memberInfo);
rtb_Users.AppendText("\t\t" + memberInfo.AccountName + " - " + memberInfo.DisplayName + " - " + memberInfo.Domain + "\n");
}
else
{
Console.WriteLine("\t\tUser already available " + memberInfo.AccountName);
}
}
}
}
}
}
}
catch (Exception ex)
{
Console.WriteLine("\tThe Project: '{0}' throws an exception: {1} and will be ignored.", project.Name, ex.Message);
}
}
}
This method is pulled out from my application AS IS and would need to be customized for your needs.

Plugin code to update another entity when case is created mscrm 2011

Im new with plugin. my problem is, When the case is created, i need to update the case id into ledger. What connect this two is the leadid. in my case i rename lead as outbound call.
this is my code. I dont know whether it is correct or not. Hope you guys can help me with this because it gives me error. I manage to register it. no problem to build and register but when the case is created, it gives me error.
using System;
using System.IO;
using System.ServiceModel;
using System.ServiceModel.Description;
using Microsoft.Xrm.Sdk;
using Microsoft.Xrm.Sdk.Query;
using Microsoft.Xrm.Sdk.Messages;
using Microsoft.Xrm.Sdk.Client;
using System.Net;
using System.Web.Services;
/*
* Purpose: 1) To update case number into lejar
*
* Triggered upon CREATE message by record in Case form.
*/
namespace UpdateLejar
{
public class UpdateLejar : IPlugin
{
/*public void printLogFile(String exMessage, String eventMessage, String pluginFile)
{
DateTime date = DateTime.Today;
String fileName = date.ToString("yyyyMdd");
String timestamp = DateTime.Now.ToString();
string path = #"C:\CRM Integration\PLUGIN\UpdateLejar\Log\" + fileName;
//open if file exist, check file..
if (File.Exists(path))
{
//if exist, append
using (StreamWriter sw = File.AppendText(path))
{
sw.Write(timestamp + " ");
sw.WriteLine(pluginFile + eventMessage + " event: " + exMessage);
sw.WriteLine();
}
}
else
{
//if no exist, create new file
using (StreamWriter sw = File.CreateText(path))
{
sw.Write(timestamp + " ");
sw.WriteLine(pluginFile + eventMessage + " event: " + exMessage);
sw.WriteLine();
}
}
}*/
public void Execute(IServiceProvider serviceProvider)
{
ITracingService tracingService = (ITracingService)serviceProvider.GetService(typeof(ITracingService));
IPluginExecutionContext context = (IPluginExecutionContext)serviceProvider.GetService(typeof(IPluginExecutionContext));
IOrganizationServiceFactory serviceFactory = (IOrganizationServiceFactory)serviceProvider.GetService(typeof(IOrganizationServiceFactory));
IOrganizationService service = serviceFactory.CreateOrganizationService(context.UserId);
//for update and create event
if (context.InputParameters.Contains("Target") &&
context.InputParameters["Target"] is Entity)
{
// Obtain the target entity from the input parmameters.
Entity targetEntity = (Entity)context.InputParameters["Target"];
// Verify that the entity represents a connection.
if (targetEntity.LogicalName != "incident")
{
return;
}
else
{
try
{
//triggered upon create message
if (context.MessageName == "Create")
{
Guid recordid = new Guid(context.OutputParameters["incidentid"].ToString());
EntityReference app_inc_id = new EntityReference();
app_inc_id = targetEntity.GetAttributeValue<EntityReference>("new_outboundcalllid");
Entity member = service.Retrieve("new_lejer", ((EntityReference)targetEntity["new_outboundcallid"]).Id, new ColumnSet(true));
//DateTime createdon = targetEntity.GetAttributeValue<DateTime>("createdon");
if (app_inc_id != null)
{
if (targetEntity.Attributes.Contains("new_outboundcallid") == member.Attributes.Contains("new_outboundcalllistid_lejer"))
{
member["new_ringkasanlejarid"] = targetEntity.Attributes["incidentid"].ToString();
service.Update(member);
}
}
}
tracingService.Trace("Lejar updated.");
}
catch (FaultException<OrganizationServiceFault> ex)
{
//printLogFile(ex.Message, context.MessageName, "UpdateLejar plug-in. ");
throw new InvalidPluginExecutionException("An error occurred in UpdateLejar plug-in.", ex);
}
catch (Exception ex)
{
//printLogFile(ex.Message, context.MessageName, "UpdateLejar plug-in. ");
tracingService.Trace("UpdateLejar: {0}", ex.ToString());
throw;
}
}
}
}
}
}
Please check,
is that entity containing the attributes or not.
check it and try:
if (targetEntity.Contains("new_outboundcallid"))
((EntityReference)targetEntity["new_outboundcallid"]).Id
member["new_ringkasanlejarid"] = targetEntity.Attributes["incidentid"].ToString();
What is new_ringkasanlejarid's type? You're setting a string to it. If new_ringkasanlejarid is an entity reference, this might be causing problems.
You might want to share the error details or trace log, all we can do is assume what the problem is at the moment.

Cannot open database "pubs" requested by the login. The login failed. Login failed for user 'ESLAM\Eslam_Nora'

Hello iam trying to store sum information inside a SQL Server table , but when i run the form and turned to store the data the above runtime error appears also the the pubs database icon in SQL Server is missing the (+) sign how come ! , i wrote that code for inserting , Thanks in advance.
public partial class Add_Client : Form
{
SqlConnection clientConnection;
string connString;
SqlCommand insertCommand;
public Add_Client()
{
InitializeComponent();
connString = "Data Source=.\\INSTANCE2;Initial Catalog=pubs; Integrated security=true ";
clientConnection = new SqlConnection();
clientConnection.ConnectionString = connString;
}
private void button1_ADD(object sender, EventArgs e)
{
try
{
SqlCommand insertCommand = new SqlCommand();
insertCommand.Connection = clientConnection;
insertCommand.CommandText = "INSERT INTO Client_Data values(#Client_Name,#Autorization_No,#Issue_Type,#Status)";
insertCommand.Parameters.Add("#Client_Name", SqlDbType.NVarChar, 60).Value = txt_Name.Text;
insertCommand.Parameters.Add("#Autorization_No", SqlDbType.Int, 60).Value = txt_Auth.Text.ToString();
insertCommand.Parameters.Add("#Issue_Type", SqlDbType.Text, 200).Value = txt_Iss.Text;
insertCommand.Parameters.Add("#Status", SqlDbType.Text, 200).Value = txt_Iss.Text;
//insertCommand.Parameters.Add("#Date To Memorize", SqlDbType.Date, 15).Value=Ca_Mem.se;
insertCommand.Connection.Open();
insertCommand.ExecuteNonQuery();
}
catch (Exception ex)
{
throw ex;
}
finally
{
if (clientConnection != null)
{
clientConnection.Close();
}
}
}
}
You use integrated security to access the database. Therefore your windows user needs to be authorized to access the database. Check the security settings for the server and the database.

Resources