I want to use the Simple Query tool in SubSonic 3(.0.0.2) and the docs page (http://subsonicproject.com/docs/Simple_Query_Tool) implies there's a way to easily get hold of table column names (e.g. Product.ProductNameColumn):
int records = new Select(Product.ProductIDColumn, Product.ProductNameColumn).
From<Product>().GetRecordCount();
The ActiveRecord generated class doesn't appear to expose this info - there is no ProductIDColumn property. Is this a hang-up from version 2?
There's no way to get the column names in SubSonic 3 at the moment. You can still use the simple query tool with strings or if you modify the Structs.tt template you can get them generated for you.
Find this section of code (I think it's line 45):
<# foreach(var col in tbl.Columns){ #>
public IColumn <#=col.CleanName#>{
get{
return this.GetColumn("<#=col.Name#>");
}
}
<# }#>
and modify it so it looks like this:
<# foreach(var col in tbl.Columns){ #>
public IColumn <#=col.CleanName#>{
get{
return this.GetColumn("<#=col.Name#>");
}
}
public static string <#= col.CleanName #>Column{
get{
return "<#= col.Name #>";
}
}
<# }#>
Then you should get all your column names automatically generated as static properties.
Related
I apologize if my question is not adequately described. I am a .Net / C# / SSIS newbie. Also apologize if already answered, I've tried searching here and Google for a few hours without luck.
Background: I need to pull data rows from a SharePoint 365 list and unpivot certain columns into a format ready for import to a SQL Server table. I realize that SSIS has an Odata Source and built-in Unpivot component, and I've successfully used those for proof of concept.
However, I believe I need a custom script component because the nummber of columns to unpivot from the source SharePoint list is variable. Each month or so, a new column will be added (it relates to a financial forecasting "tool" in SharePoint, and the latest forecasted month changes). My understanding is that source columns must be defined in SSIS at design time, so if my source columns are changing, the only way I can think to address this without manually changing the SSIS data flow each month is to programatically combine the Odata source and unpivot functions into a custom script component.
I understand, or can figure out, the unpivot logic. The part I am struggling with is how to actually connect to and expose a given list and it's data rows / columns as lists that I can loop through and perform my mapping to the output columns.
My "starting point" for requested guidance is this:
1) Created and successfully connected to the SharePoint site in question using standard SSIS Odata Connection Manager.
2) Created standard "Script Component", type = source, on the visual designer.
3) From script component properties, associated the Odata connection manager with the name of "myConnection".
4) Need help -> within the script component, open a connection to a specific list, read it's contents, and perform unpivot logic.
For illustration purposes, assume the source is a SharePoint list with two "fixed" string columns titled Study and Site, and a variable number of columns with names matching month-end dates (e.g. 9/30/2016, 10/31/2016, etc.) that contain integer values. I want to map the study and site source columns to destination columns of the same name and unpivot the month columns where column name is mapped to ProjectionMonth and the integer value is mapped to ProjectionValue.
Here's the basic algorithm I have in mind (I realize this isn't compilable - that's where I need your help!):
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Pipeline.Wrapper;
using Microsoft.SqlServer.Dts.Runtime.Wrapper;
using System.Data.SqlClient;
[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]
public class ScriptMain : UserComponent
{
IDTSConnectionManager100 connMgr;
SqlConnection sqlConn; // from MSDN tutorial, but I don't know how to adapt to Odata/SharePoint 365 list
SqlDataReader sqlReader; // not sure how to adapt
public override void AcquireConnections(object Transaction)
{
connMgr = this.Connections.myConnection;
sqlConn = (SqlConnection)connMgr.AcquireConnection(null); // This is from MSDN tutorial, but I don't know how to adapt to Odata
}
public override void PreExecute()
{
//Not sure how to adapt to odata to read specific SharePoint list
SqlCommand cmd = new SqlCommand("SELECT * FROM <some sharepoint list>", sqlConn);
sqlReader = cmd.ExecuteReader();
}
public override void PostExecute()
{
sqlReader.Close(); // Not sure how to adapt.
}
public override void CreateNewOutputRows()
{
string myStudy;
string mySite;
string myProjectionMonth;
string myProjectionValue;
// This is a rough representation of the logic needed.
// I realize that the actual code to access column values / names depends on the class(es) I need to use, but not sure what those classes are / how to access
foreach (myListRow in sqlConn.rows)
{
myStudy = myListRow.Columns["Study"].value;
mySite = myListRow.Columns["Site"].value;
foreach (myColumn in myListRow.Columns)
if (DateTime.TryParse(myColumn.Name, out dateValue))
{
myProjectionMonth = myColumn.Name;
myProjectionValue = myColumn.Value;
Output0Buffer.AddRow();
Output0Buffer.Study = myStudy;
Output0Buffer.Site = mySite;
Output0Buffer.ProjectionMonth = myProjectionMonth;
Output0Buffer.ProjectionValue = myProjectionValue;
}
}
}
}
Edit: as an example, assume the source SharePoint list has the following:
Study Site 9/30/2016 10/31/2016
123 ABC 5 10
I want to the script component to connect to the list, read it's contents, and return the following unpivoted data set for eventual load into SQL Server:
Study Site ProjectionMonth ProjectionValue
123 ABC 9/30/2016 5
123 ABC 10/31/2016 10
So this is probably not an ideal way of doing it, and it doesn't leverage the standard SSIS Odata Connection Manager that I wanted... but it does technically get the job done and it's good enough for me, for now.
Would be interested on any suggested feedback / improvements / etc., if you have any.
#region Namespaces
using System;
using Microsoft.SharePoint.Client;
using System.Security;
using System.Collections.Generic;
#endregion
[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]
public class ScriptMain : UserComponent
{
public override void CreateNewOutputRows()
{
// Connect to SharePoint
ClientContext context = new ClientContext("https://<redacted>.sharepoint.com/Development");
SecureString passWord = new SecureString();
foreach (char c in Variables.sharepointPassword.ToCharArray()) passWord.AppendChar(c);
context.Credentials = new SharePointOnlineCredentials("<redacted>#<redacted>.onmicrosoft.com", passWord);
// Define the target list
List EnrollmentList = context.Web.Lists.GetByTitle("Enrollment Projections");
// Find all fields in the target list
FieldCollection myFields = EnrollmentList.Fields;
context.Load(myFields);
context.ExecuteQuery();
// Load all rows from the target list
CamlQuery query = CamlQuery.CreateAllItemsQuery(100);
ListItemCollection items = EnrollmentList.GetItems(query);
context.Load(items);
context.ExecuteQuery();
//Store valid date fields
List<Field> myDateFields = new List<Field>();
foreach (Field tempField in myFields)
{
DateTime tempDate;
if (DateTime.TryParse(tempField.Title, out tempDate))
{
myDateFields.Add(tempField);
}
}
string myStudy;
string mySite;
string myMonth;
string myValue;
foreach (ListItem listItem in items)
{
myStudy = listItem["Study"].ToString();
mySite = listItem["Site"].ToString();
foreach (Field tempField in myDateFields)
{
myMonth = tempField.Title;
myValue = listItem[tempField.InternalName.ToString()].ToString();
Output0Buffer.AddRow();
Output0Buffer.Study = myStudy;
Output0Buffer.Site = mySite;
Output0Buffer.ProjectedMonth = myMonth;
Output0Buffer.ProjectedValue = Convert.ToInt32(myValue);
}
}
}
}
Assume I have the following as the value in an IMap:
public class Employee{
public int empId;
public List<String> categories;
public List<String> getCategories(){
return this.categories;
}
}
I would like to find all employees that belong to category "Sales". Also, I would like to create an index on getCategories() so that the query returns fast. There seems to be no Predicate available to do this. How do I go about achieving this? It seems like I will have to write a Predicate to do this. Is there example code I can look at that would show me how to build a predicate that uses an index ?
The only way I currently see this happening is to denormalize the data model and use something like a IMap and the following as value:
class EmployeeCategory{int employeeId, String category}
And put an index on category.
It is somewhere on the backlog to provide more advances indices that should be able to do this out of the box.
I tried by iterating the List to a separate Imap and then querying it in the client.
IMap<String,ArrayList< categories >> cache=hazelcastInstance.getMap("cache");
IMap<String, categories> cachemodified = hazelcastInstance.getMap("cachemodified") ;
int[] idx = { 0 };
xref.get("urkey").forEach(cachefelement ->{
cachemodified.put(String.valueOf(idx[0]++),cachefelement);
});
Predicate p = Predicates.equal("categoryId", "SearchValue");
Collection<categories> result = cachemodified.values(p);
I am not sure if I am going about this the correct way but I have a c# method which loads an excel sheet into a 2 dimentional object array. In this array item 1,1 - 1,16 contain headers, then 2-1 - 2-16 contain data that match up with those headers as do x-1 - x-16 from there on in. I would like to turn this array into a data table so ultimately I can have it in a format I will then import into an access or SQL server db depending on a clients needs. I have tried using the following code to no avail, but I have a feeling I am way off. Any help on this would be very much appreciated.
private void ProcessObjects(object[,] valueArray)
{
DataTable holdingTable = new DataTable();
DataRow holdingRow;
holdingTable.BeginLoadData();
foreach(int row in valueArray)
{
holdingRow = holdingTable.LoadDataRow(valueArray[row], true);
}
}
Any chance you're using a repository pattern (like subsonic or EF) or using LinqToSql?
You could do this (LinqToSql for simplicity):
List<SomeType> myList = valueArray.ToList().Skip([your header rows]).ConvertAll(f => Property1 = f[0] [the rest of your convert statement])
DataContext dc = new DataContext();
dc.SomeType.InsertAllOnSubmit(myList);
dc.SubmitChanges();
My site is using Subsonic 2.2 on my site.
I have 1 weird situation where I need to run some ad-hoc SQL statements.
public IList<string> GetDistincList(string TableName, string FieldName)
{
string sqlToRun = string.Format("SELECT DISTINCT {0} FROM {1} ORDER BY {0}", FieldName, TableName);
Query query = new Query(TableName);
query.PleaseRunThis(sqlToRun);
query.ExecuteReader();
}
Can anyone help me here? As it appears, I just want to return a generic list of strings.
Thanks!
Subsonic has a great method called ExecuteTypedList() so you can do somethink like this.
List<int> result = DB.Select(Table.Columns.Id)
.Distinct()
.From<Table>()
.OrderBy(Table.Columns.Id)
.ExecuteTypedList<int>();
or even with pocos:
public class UserResult
{
public int Id {get;set;}
public string Name {get;set;}
}
List<UserResult> users = DB.Select(
User.Columns.UserId + " as Id", // the as ... is only needed if your
User.Columns.UserName + " as Name" // column name differs from the
).From<User>() // property name of your class
.ExecuteTypedList<UserResult>();
Unfortunately this method doesn't work for string since it requires
a) a valuetype
b) a class with a parameterless constructor since the method uses reflection to map the columns from the result to the properties of the class
However I wrote an extension method a while ago that works for string:
Use the Subsonic.Select() ExecuteTypedList Method with String
Look at my own answer in the link.
If you add the extensionmethod to your code you can do:
List<String> result = DB.Select(User.Columns.UserName)
.From<User>()
.ExecuteTypedList();
Use the CodingHorror class.
Here's the SubSonic 3 way of doing it: http://www.subsonicproject.com/docs/CodingHorror
The SubSonic 2 way is similar:
Dim ch As SubSonic.CodingHorror
ch.Execute("delete from #tablename", table)
Problem
As we know, SharePoint saves data in database in plain text. Some fields even have concatenated strings like <id>;#<value> for user fields. Percents are saved as doubles (1.00000000000000 for 100%) and etc.
Ofcourse, I want to display data as they are displayed in lists.
What should I do?
Should I use derived SPBoundField to format values (Which I actually did and it works fine until you want to filter (probably SPBoundField won't format me values because i use ObjectDataSource not list and with reflector I saw if there are SPListItems in datasource, then it formats correctly. Not my case)
alt text http://img199.imageshack.us/img199/2797/ss20090820110331.png
Or must I loop through all the DataTable and format each row accordingly?
What are Your techniques?
Thank you.
Here is how I solved this issue.
<asp:TemplateField HeaderText="Campaign Members">
<ItemTemplate>
<%# RemoveCharacters(Eval("CampaignMembers").ToString())%>
</ItemTemplate>
</asp:TemplateField>
// Make sure declare using System.Text.RegularExpression;
protected string RemoveCharacters(object String)
{
string s1 = String.ToString();
string newString = Regex.Replace(s1, #"#[\d-];", string.Empty);
newString = Regex.Replace(newString, "#", " ");
return newString.ToString();
}
I normaly use ItemTemplates that inherit from ITemplate. With in the ItemTemplate I use the SPFieldxxxValue classes or some custom formating code. This saves looping through the DataTable and the ItemTemplates can be reused.
The ItemTemplates are attached in Column Binding
E.G
// Normal Data Binding
SPBoundField fld = new SPBoundField();
fld.HeaderText = field.DisplayName;
fld.DataField = field.InternalName;
fld.SortExpression = field.InternalName;
grid.Columns.Add(fld);
// ItemTemplate Binding
TemplateField fld = new TemplateField();
fld.HeaderText = field.DisplayName;
fld.ItemTemplate = new CustomItemTemplateClass(field.InternalName);
fld.SortExpression = field.InternalName;
grid.Columns.Add(fld);
An example of a ItemTemplate
public class CustomItemTemplateClass : ITemplate
{
private string FieldName
{ get; set; }
public CustomItemTemplateClass(string fieldName, string formatString)
{
FieldName = fieldName;
}
#region ITemplate Members
public void InstantiateIn(Control container)
{
Literal lit = new Literal();
lit.DataBinding += new EventHandler(lit_DataBinding);
container.Controls.Add(lit);
}
#endregion
void lit_DataBinding(object sender, EventArgs e)
{
Literal lit = (Literal)sender;
SPGridViewRow container = (SPGridViewRow)lit.NamingContainer;
string fieldValue = ((DataRowView)container.DataItem)[FieldName].ToString();
//Prosses Filed value here
SPFieldLookupValue lookupValue = new SPFieldLookupValue(fieldValue);
//Display new value
lit.Text = lookupValue.LookupValue;
}
}
Here are a few options. I don't know the output of all of them (would be a good blog post) but one of them should do what you want:
SPListItem.GetFormattedValue()
SPField.GetFieldValue()
SPField.GetFieldValueAsHtml()
SPField.GetFieldValueAsText()
It may also be handy to know that if you ever want to make use of the raw values then have a look at the SPField*XYZ*Value classes. For example the form <id>;#<value> you mention is represented by the class SPFieldUserValue. You can pass the raw text to its constructor and extract the ID, value, and most usefully User very easily.
I would suggest either to format the values before binding them to the spgridview. Linq and an anonymous type is preffered or to call a code behind function on the field that needs the formatting upon binding.
DataField='<%# FormatUserField(Eval("UserFieldName")) %>'
or...maybe a templated field?
After all, i did have not know any other solution to loop through DataTable rows and format them accordingly.
If your SPGridView's data source is list, try out SPBoundField.