display data dynamically on a razor page - razor-pages

There's a console app that uses async method for web site scraping.
using Microsoft.AspNetCore.Authentication;
using HtmlAgilityPack;
using System.Net;
class Program
{
static void Main(string[] args)
{
submainAsync();
Console.ReadKey();
}
private static async Task<bool> submainAsync()
{
ADManager adManager = new ADManager();
List<Person> people;
people = adManager.GetPeopleByPath("ls.local");
string cookie = "0123456789";
if (cookie == String.Empty)
{
Console.WriteLine("check your password");
return false;
}
var peopleAS = getPIDsAsync(people, cookie);
int total = people.Count;
await foreach (Person prs in peopleAS)
{
Console.WriteLine("next is {0} countdown {1}", prs.samAccountName, total--);
}
return true;
}
static private async IAsyncEnumerable<Person> getPIDsAsync(List<Person> people, string inCookie)
{
foreach (Person prs in people)
{
var baseAddress = new Uri("http://192.168.1.100");
using (var handler = new HttpClientHandler { UseCookies = false })
using (var client = new HttpClient(handler) { BaseAddress = baseAddress })
{
string request = "/doc/admin/directories.asp?act=test&id=META&login=" + prs.samAccountName + "&s=DEFAULT";
var message = new HttpRequestMessage(HttpMethod.Get, request);
message.Headers.Add("Cookie", "t%5FDEFAULT%5Fadmin="+ inCookie);
var result = await client.SendAsync(message);
var resultString = await result.Content.ReadAsStringAsync();
HtmlDocument doc = new HtmlDocument();
doc.LoadHtml(resultString);
var node = doc.DocumentNode.SelectSingleNode("//*[#id=\"value\"]");
var val = node.InnerText;
prs.PID = node.InnerText;
}
yield return prs;
}
}
}
[ yield return, await foreach ] writelines out data nicely as soon as the next piece of data is ready. Now, I moved the code from a console app to a razor page app, so submainAsync() is called from OnPost. The question is - how can I display scraped data in a table on a razor page? My goal is to display the data dynamically, as soon as the next piece is ready, just like it happens in the Console.
Would someone please point me in the right direction?

Related

Getting 405 status Error when trying to consume POST method of web/Api in asp.net mvc5

I am trying to consume post method of web/api which takes string as parameter, but getting 405 Error : Api Controller Name: Getdata Action Method Name : postdata
please help to identify error.
public bool postdata(string fn)
{
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("http://localhost:59968/api/Getdata");
var task = client.PostAsJsonAsync("Getdata", fn);
task.Wait();
var res = task.Result;
if(res.IsSuccessStatusCode)
{
return true;
}
}
return false;
}
[HttpPost]
public IHttpActionResult postdata(string p)
{
using (var context = new MediaEntities())
{
mediatable m = new mediatable()
{
media_path = p,
is_active = true
};
context.mediatable.Add(m);
context.SaveChanges();
return Ok();
}
}
I think you have used action method names instead of controller name else provide details of your api.
client.BaseAddress = new Uri("http://localhost:59968/api/{controller_name}");
//HTTP POST
var postTask = client.PostAsJsonAsync("controller_name", fn);
This link might help you.

Azure Table Storage not saving all object properties

I have a problem with the Azure Table Storage. What I'm trying to achieve is saving the ChangeToken of the SharePoint list in order to use the webhooks properly.
Here is the code:
public class TablesHelper
{
private static readonly string TokenTableName = "TokenTable";
public static async Task<ListChangeToken> GetChangeTokenForListAsync(string listId)
{
var retrieveOperation = TableOperation.Retrieve<ListChangeToken>("Lists", listId, new List<string>() { "ChangeToken" });
var tableReference = await GetTableReferenceAsync(TokenTableName);
var tableResult = await tableReference.ExecuteAsync(retrieveOperation);
if(tableResult.Result != null)
{
return tableResult.Result as ListChangeToken;
}
return null;
}
public static async Task SaveChangeTokenForListAsync(ListChangeToken changeToken)
{
var insertOperation = TableOperation.Insert(changeToken);
var tableReference = await GetTableReferenceAsync(TokenTableName);
var result = await tableReference.ExecuteAsync(insertOperation);
}
private static async Task<CloudTable> GetTableReferenceAsync(string tableName)
{
var storageAccount = CloudStorageAccount.Parse(ConfigurationHelper.CloudStorage);
var tableClient = storageAccount.CreateCloudTableClient();
var reference = tableClient.GetTableReference(tableName);
await reference.CreateIfNotExistsAsync();
return reference;
}
}
The ListChangeToken class:
public class ListChangeToken : TableEntity
{
public ListChangeToken(string listId, string changeToken)
{
this.PartitionKey = "Lists";
this.RowKey = listId;
this.ChangeToken = changeToken;
}
public ListChangeToken() { }
public string ChangeToken { get; set;}
}
As per request, the function calling TablesHelper:
[FunctionName("EventHandler")]
public static async Task Run([QueueTrigger("events", Connection = "CloudStorage")]string myQueueItem, TraceWriter log)
{
var notificationGroup = Newtonsoft.Json.JsonConvert.DeserializeObject<NotificationGroup>(myQueueItem);
var contextHelper = new ContextHelper();
foreach (var notification in notificationGroup.Value)
{
UriBuilder uriBuilder = new UriBuilder();
uriBuilder.Scheme = "https";
uriBuilder.Host = ConfigurationHelper.TenantDomain;
uriBuilder.Path = notification.SiteUrl;
using (var ctx = contextHelper.GetAppOnlyContext(uriBuilder.ToString()))
{
//Read change token
var currentChangeToken = await TablesHelper.GetChangeTokenForListAsync(notification.Resource);
if(currentChangeToken == null)
{
log.Error($"No change token found for list {notification.Resource}. This is a NO GO. Please use the '/api/Setup' function.");
}
var listId = Guid.Parse(notification.Resource);
var changes = await CSOMHelper.GetListItemChangesAsync(ctx, listId, currentChangeToken.ChangeToken);
if(changes.Count > 0)
{
var lastChange = changes[changes.Count - 1];
//Save the last change token
var changeTokenValue = lastChange.ChangeToken.StringValue;
await TablesHelper.SaveChangeTokenForListAsync(new ListChangeToken(
notification.Resource,
changeTokenValue
));
await HandleChanges(ctx, changes);
}
}
}
log.Info($"C# Queue trigger function processed: {myQueueItem}");
}
The problem is that always, when using the "GetChangeTokenForListAsync" the Entity is received properly, but the .ChangeToken property is always null. It is also not visible when browsing with the Azure Storage Explorer. What am I doing wrong here?
The issue is related to the Azure Storage Emulator (V. 5.7.0.0). The same code works perfectly when working with the "live" Azure.

Send email using Office 365 using unified API

We are trying to use the O365 Unified API to send emails from our line-of-business apps. I use the following code to send the email. This throws a DataServiceQueryException exception "Unauthorized".
public async Task SendEmailAsUserAsync(EmailMessage message)
{
try
{
var graphClient = await _authenticationHelper.GetGraphClientAsync();
Message m = InitializeMessage(message);
await graphClient.Me.SendMailAsync(m, true);
}
catch (DataServiceQueryException dsqe)
{
_logger.Error("Could not get files: " + dsqe.InnerException.Message, dsqe);
throw;
}
}
private static Message InitializeMessage(EmailMessage message)
{
ItemBody body = new ItemBody {Content = message.Body, ContentType = BodyType.HTML};
Message m = new Message
{
Body = body,
Subject = message.Subject,
Importance = Importance.Normal,
};
//Add all the to email ids
if (message.ToRecipients != null)
foreach (Models.Messaging.EmailAddress emailAddress in message.ToRecipients)
{
m.ToRecipients.Add(new Recipient { EmailAddress = new Microsoft.Graph.EmailAddress { Address = emailAddress.Address, Name = emailAddress.Name } });
}
return m;
}
The code for _authenticationHelper.GetGraphClientAsync() is
public async Task<GraphService> GetGraphClientAsync()
{
Uri serviceRoot = new Uri(appConfig.GraphResourceUriBeta + appConfig.Tenant);
_graphClient = new GraphService(serviceRoot,
async () => await AcquireTokenAsyncForUser(appConfig.GraphResourceUri, appConfig.Tenant));
return _graphClient;
}
private async Task<string> AcquireTokenAsyncForUser(string resource, string tenantId)
{
AuthenticationResult authenticationResult = await GetAccessToken(resource, tenantId);
_accessCode = authenticationResult.AccessToken;
return _accessCode;
}
private async Task<AuthenticationResult> GetAccessToken(string resource, string tenantId)
{
string authority = appConfig.Authority;
AuthenticationContext authenticationContext = new AuthenticationContext(authority);
ClientCredential credential = new ClientCredential(appConfig.ClientId, appConfig.ClientSecret);
string authHeader = HttpContext.Current.Request.Headers["Authorization"];
string userAccessToken = authHeader.Substring(authHeader.LastIndexOf(' ')).Trim();
UserAssertion userAssertion = new UserAssertion(userAccessToken);
var authenticationResult = await authenticationContext.AcquireTokenAsync(resource, credential, userAssertion);
return authenticationResult;
}
However if I change the SendEmailAsUserAsync method as shown below, the email is sent but an InvalidOperationException is thrown with message "The complex type 'System.Object' has no settable properties."
public async Task SendEmailAsUserAsync(EmailMessage message)
{
try
{
var graphClient = await _authenticationHelper.GetGraphClientAsync();
Message m = InitializeMessage(message);
//await graphClient.Me.SendMailAsync(m, true); //This did not work
var user = await graphClient.Me.ExecuteAsync();
await user.SendMailAsync(m, true);
}
catch (DataServiceQueryException dsqe)
{
_logger.Error("Could not get files: " + dsqe.InnerException.Message, dsqe);
throw;
}
}
Can any one point out if there is something wrong here.
Check the example project below, this has a working example (after you fill in the ClientID etc. in app.config).
Office 365 API demo applications
For sending email it uses the function below, which works if you set it up correctly. It also has a number of functions for Authenticating using Authorization Code Grant Flow.
public async Task SendMail(string to, string subject, string body)
{
var client = await this.AuthenticationHelper
.EnsureOutlookServicesClientCreatedAsync(
Office365Capabilities.Mail.ToString());
Message mail = new Message();
mail.ToRecipients.Add(new Recipient()
{
EmailAddress = new EmailAddress
{
Address = to,
}
});
mail.Subject = subject;
mail.Body = new ItemBody() { Content = body, ContentType = BodyType.HTML };
await client.Me.SendMailAsync(mail, true);
}
Actually, there is no assembly wrapper for the graph API.
Microsoft.Graph.dll is deprecrated.
So, you should to :
Deal with the REST requests : See here : http://graph.microsoft.io/docs/api-reference/v1.0/api/message_send
Generate a wrapper with Microsoft.Vipr project : see here : https://github.com/microsoft/vipr
For the authentication, ADAL works fine :)

running stored procedures into own model with servicestack ormlite

Is there any examples to be found for running a stored procedure on serviceStack MVC using ormlite? mythz ? seen this block of code:
var results = new List<EnergyCompare>
{dbFactory.Exec(dbCmd =>
{
dbCmd.CommandType = CommandType.StoredProcedure;
dbCmd.Parameters.Add(new SqlParameter("#id", 1));
dbCmd.CommandText = "GetAuthorById";
return dbCmd.ExecuteReader().ConvertTo<EnergyCompare>();
}
)};
but came with the text of never worked on the google groups!
i can also write this:
using(var db = new SwitchWizardDb())
{
var results2 = db.dbCmd.ExecuteProcedure()
}
but not sure how to complete this with parameters, and in the source code I looked at, it said obsolete?
thanks
Looks like ServiceStack.ORMLite has been updated to make this easier:
List<Poco> results = db.SqlList<Poco>("EXEC GetAnalyticsForWeek 1");
List<Poco> results = db.SqlList<Poco>("EXEC GetAnalyticsForWeek #weekNo", new { weekNo = 1 });
List<int> results = db.SqlList<int>("EXEC GetTotalsForWeek 1");
List<int> results = db.SqlList<int>("EXEC GetTotalsForWeek #weekNo", new { weekNo = 1 });
This example is on the front page of the github repo.
Well I figured it was best to roll my own handler so have created this, any thoughts would be most welcome, especially with how I could pass over params in some kind of func or something:
I have a main class to deal with easy access to my connection object:
public class DatabaseNameSp : IDisposable
{
private readonly SqlConnection _spConn = new SqlConnection(DatabaseNameSp .dbConString);
public readonly SqlCommand SpCmd;
public DatabaseNameSp (string procedureName)
{
_spConn.Open();
SpCmd = new SqlCommand
{
Connection = _spConn,
CommandType = CommandType.StoredProcedure,
CommandText = procedureName
};
}
public void Dispose()
{
_spConn.Close();
SpCmd.Dispose();
}
}
usage:
using (var db = new DatabaseNameSp ("procedurenname"))
{
db.SpCmd.Parameters.Add(new SqlParameter("#Id", 1));
var rdr = db.SpCmd.ExecuteReader(CommandBehavior.CloseConnection);
var results = new List<CustomDTO>();
while (rdr.Read())
{
results.Add(new CustomDTO { Name = rdr["name"].ToString(), Id = rdr["id"].ToString() });
}
return new CustomDTOResponse { Results = results };
}
Any thoughts !
thanks
Here is an example of running a stored procedure with ormLite that may help you:
IList<MyDTO> myList = DbFactory.Run(dbCnx =>
{
using (var dbCmd = dbCnx.CreateCommand())
{
dbCmd.CommandType = CommandType.StoredProcedure;
dbCmd.CommandText = "mySchema.myStoredProc";
dbCmd.Parameters.Add(new SqlParameter("#param1", val1));
dbCmd.Parameters.Add(new SqlParameter("#param2", val2));
var r = dbCmd.ExecuteReader();
return r.ConvertToList<MyDTO>();
}
});
To just simply run a stored procedure with no data returned:
public class ComsManager : Dbase
{
private IDbConnection dbConn;
public ComsManager()
{
dbConn = Dbase.GetConnection();
}
public void Housekeeping()
{
using(var dbCmd = dbConn.CreateCommand())
dbConn.Exec(res => { dbCmd.CommandType = CommandType.StoredProcedure; dbCmd.CommandText = "SP_housekeeping"; dbCmd.ExecuteNonQuery(); });
}

Recursive linkscraper c#

I'm struggling with this a whole day now and I can't seem to figure it out.
I have a fucntion that gives me a list of all links on a specific url. That works fine.
However I want to make this function recursive so that it searches for the links found with the first search and adds them to the list and continue so that it goes through all my pages on the website.
How can I make this recursive?
My code:
class Program
{
public static List<LinkItem> urls;
private static List<LinkItem> newUrls = new List<LinkItem>();
static void Main(string[] args)
{
WebClient w = new WebClient();
int count = 0;
urls = new List<LinkItem>();
newUrls = new List<LinkItem>();
urls.Add(new LinkItem{Href = "http://www.smartphoto.be", Text = ""});
while (urls.Count > 0)
{
foreach (var url in urls)
{
if (RemoteFileExists(url.Href))
{
string s = w.DownloadString(url.Href);
newUrls.AddRange(LinkFinder.Find(s));
}
}
urls = newUrls.Select(x => new LinkItem{Href = x.Href, Text=""}).ToList();
count += newUrls.Count;
newUrls.Clear();
ReturnLinks();
}
Console.WriteLine();
Console.Write("Found: " + count + " links.");
Console.ReadLine();
}
private static void ReturnLinks()
{
foreach (LinkItem i in urls)
{
Console.WriteLine(i.Href);
//ReturnLinks();
}
}
private static bool RemoteFileExists(string url)
{
try
{
HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
request.Method = "HEAD";
//Getting the Web Response.
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
//Returns TURE if the Status code == 200
return (response.StatusCode == HttpStatusCode.OK);
}
catch
{
return false;
}
}
}
The code behind LinkFinder.Find can be found here: http://www.dotnetperls.com/scraping-html
Anyone knows how I can either make that function recursive or can I make the ReturnLinks function recursive? I prefer to not touch the LinkFinder.Find method as this works perfect for one link, I just should be able to call it as many times as needed to expand my final url list.
I assume you want to load each link and find the link within, and continue until you run out of links?
Since it is likely that the recursion depth could get very large, i would avoid recursion, this should work i think.
WebClient w = new WebClient();
int count = 0;
urls = new List<string>();
newUrls = new List<LinkItem>();
urls.Add("http://www.google.be");
while (urls.Count > 0)
{
foreach(var url in urls)
{
string s = w.DownloadString(url);
newUrls.AddRange(LinkFinder.Find(s));
}
urls = newUrls.Select(x=>x.Href).ToList();
count += newUrls.Count;
newUrls.Clear();
ReturnLinks();
}
Console.WriteLine();
Console.Write("Found: " + count + " links.");
Console.ReadLine();
static void Main()
{
WebClient w = new WebClient();
List<ListItem> allUrls = FindAll(w.DownloadString("http://www.google.be"));
}
private static List<ListItem> FindAll(string address)
{
List<ListItem> list = new List<ListItem>();
foreach (url in LinkFinder.Find(address))
{
list.AddRange(FindAll(url.Address)));//or url.ToString() or what ever the string that represents the address
}
return list;
}

Resources