Open XML is generating .xlsx files that can be read by Open Office, but not by Excel itself.
With this as my starting point( Export DataTable to Excel with Open Xml SDK in c#) I have added code to create a .xlsx file. Attempting to open with Excel, I'm asked if I want to repair the file. Saying yes gets "The workbook cannot be opened or repaired by Microsoft Excel because it's corrupt." After many hours of trying to jiggle the data from my table to make this work, I finally threw up my hands in despair and made a spreadsheet with a single number in the first cell.
Still corrupt.
Renaming it to .zip and exploring shows intact .xml files. On a whim, I took a legit .xlsx file created by Excel, unzipped it, rezipped without changing contents and renamed back to .xlsx. Excel declared it corrupt. So this is clearly not a content issue, but file a format issue. Giving up on Friday, I sent some of the sample files home and opened them there with Libre Office. There were no issues at all. File content was correct and Calc had no problem. I'm using Excel for Office 365, 32 bit.
// ignore the bits (var list) that get data from the database. I've reduced this to just the output of a single header line
List< ReportFilingHistoryModel> list = DB.Reports.Report.GetReportClientsFullHistoryFiltered<ReportFilingHistoryModel>(search, client, report, signature);
MemoryStream memStream = new MemoryStream();
using (SpreadsheetDocument workbook = SpreadsheetDocument.Create(memStream, SpreadsheetDocumentType.Workbook))
{
var workbookPart = workbook.AddWorkbookPart();
workbook.WorkbookPart.Workbook = new Workbook();
workbook.WorkbookPart.Workbook.Sheets = new Sheets();
var sheetPart = workbook.WorkbookPart.AddNewPart<WorksheetPart>();
var sheetData = new SheetData();
sheetPart.Worksheet = new Worksheet(sheetData);
Sheets sheets = workbook.WorkbookPart.Workbook.GetFirstChild<Sheets>();
string relationshipId = workbook.WorkbookPart.GetIdOfPart(sheetPart);
uint sheetId = 1;
if (sheets.Elements<Sheet>().Count() > 0)
{
sheetId = sheets.Elements<Sheet>().Select(s => s.SheetId.Value).Max() + 1;
}
Sheet sheet = new Sheet() { Id = relationshipId, SheetId = sheetId, Name = "History" };
sheets.Append(sheet);
Row headerRow = new Row();
foreach( var s in "Foo|Bar".Split('|'))
{
var cell = new Cell();
cell.DataType = CellValues.Number;
cell.CellValue = new CellValue("5");
headerRow.AppendChild(cell);
}
sheetData.AppendChild(headerRow);
}
memStream.Seek(0, SeekOrigin.Begin);
Guid result = DB.Reports.Report.AddClientHistoryList( "test.xlsx", memStream.GetBuffer(), "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet");
return Ok(result);
This should just work. I've noticed other stack overflow discussions that direct back to the first link I mentioned above. I seem to be doing it right (and Calc concurs). There have been discussions of shared strings and whatnot, but by using plain numbers I shouldn't be having issues. What am I missing here?
In working on this, I went with the notion that some extraneous junk on the end of a .zip file is harmless. 7-Zip, Windows Explorer and Libre Office all seem to agree (as does some other zip program I used at home whose name escapes me). Excel, however, does not. Using the pointer at memStream.GetBuffer() was fine, but using its length was not. (The preceding Seek() was unnecessary.) Limiting the write of the data to a length equal to the current output position keeps Excel from going off the rails.
Context
I am trying to build a Word document browser in Excel to sift trough a large amount of documents (around 1000).
The process of opening a word document proves to be rather slow (around 4 seconds per documents, so in this case it takes 2 hour to look through all the items, which is far too slow for a single query), even by disabling all things that could slow down the opening, hence I open:
As read only
Without the open and repair mode (which can happen on some documents)
Disabling the display of the document
My attempt so far
These documents are tricky to look through because some keywords do appear every single time but not in the same context (not the core of the problem here since I can handle that when the text is loaded in arrays). Hence the often used Windows explorer solution (like in this link ) cannot be used in my case.
For the moment, I managed to have a working macro that analyze the content of the word documents by opening them.
Code
Here is a sample of the code.
Note that I used the Microsoft Word 14.0 Object Library reference
' Analyzing all the word document within the same folder '
Sub extractFile()
Dim i As Long, j As Long
Dim sAnalyzedDoc As String, sLibName As String
Dim aOut()
Dim oWordApp As Word.Application
Dim oDoc As Word.Document
Set oWordApp = CreateObject("Word.Application")
sLibName = ThisWorkbook.Path & "\"
sAnalyzedDoc = Dir(sLibName)
sKeyword = "example of a word"
With Application
.DisplayAlerts = False
.ScreenUpdating = False
End With
ReDim aOut(2, 2)
aOut(1, 1) = "Document name"
aOut(2, 1) = "Text"
While (sAnalyzedDoc <> "")
' Analyzing documents only with the .doc and .docx extension '
If Not InStr(sAnalyzedDoc, ".doc") = 0 Then
' Opening the document as mentionned above, in read only mode, without repair and invisible '
Set oDoc = Word.Documents.Open(sLibName & "\" & sAnalyzedDoc, ReadOnly:=True, OpenAndRepair:=False, Visible:=False)
With oDoc
For i = 1 To .Sentences.Count
' Searching for the keyword within the document '
If Not InStr(LCase(.Sentences.Item(i)), LCase(sKeyword)) = 0 Then
If Not IsEmpty(aOut(1, 2)) Then
ReDim Preserve aOut(2, UBound(aOut, 2) + 1)
End If
aOut(1, UBound(aOut, 2)) = sAnalyzedDoc
aOut(2, UBound(aOut, 2)) = .Sentences.Item(i)
GoTo closingDoc ' A dubious programming choice but that works for the moment '
End If
Next i
closingDoc:
' Intending to make the closing faster by not saving the document '
.Close SaveChanges:=False
End With
End If
'Moving on to the next document '
sAnalyzedDoc = Dir
Wend
exitSub:
With Output
.Range(.Cells(1, 1), .Cells(UBound(aOut, 1), UBound(aOut, 2))) = aOut
End With
With Application
.DisplayAlerts = True
.ScreenUpdating = True
End With
End Sub
My question
The idea I thought was to go via the XML content within the document to access directly to its content (which you can access when renaming any document in newer versions of Word, with a .zip extension and going for nameOfDocument.zip\word\document.xml).
It would be a lot faster than loading all the images, charts and tables of the word document which are of no use in a text search.
Thus, I wanted to ask if there was a way in VBA to open a word document like a zip file and access that XML document to then process it like a normal string of characters in VBA, since I already have the path and the name of the file given the above code.
Do note that this is not an easy answer to the above problem and the sole VBA code in my initial question will do perfectly the job as long as you do not have a load of documents to browse, else go for another tool (there is a Python Dynamic Link Library (DLL) that does that very well).
Ok, I'll try to make my answer as explanatory as possible.
First of all this question lead me to the infinite journey of XML in C# and in XPath which I chose not to pursue at some point.
It reduced the time of analyzing the files from roughly 2 hours to 10 seconds.
Context
The backbone of reading XML documents, and therefore inner word XML documents, is the OpenXML library from Microsoft.
Keep in mind what I said above, that the method I was trying to implement cannot be done solely in VBA and thus must be done in another way.
This is probably due to the fact that VBA is implemented for Office and thus limited in accessing the core structure of Office documents, but I have no information relating to this limitation (any information is welcomed).
The answer I will give here is writing a C# DLL for VBA.
For writing DLL in C# and referencing to it in VBA I redirect you toward the following link which will resume in a better way this specific process: Tutorial for creating DLL in C#
Let's start
First of all you will need to reference the WindowsBase library and the DocumentFormat.OpenXML in your project to make the solution work as explained in this MSDN article Manipulate Office Open XML Formats Documents and that one Open and add text to a word processing document (Open XML SDK)
These articles explain broadly how works the OpenXML library for manipulating word documents.
The C# code
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Xml;
using System.IO.Packaging;
namespace BrowserClass
{
public class SpecificDirectory
{
public string[,] LookUpWord(string nameKeyword, string nameStopword, string nameDirectory)
{
string sKeyWord = nameKeyword;
string sStopWord = nameStopword;
string sDirectory = nameDirectory;
sStopWord = sStopWord.ToLower();
sKeyWord = sKeyWord.ToLower();
string sDocPath = Path.GetDirectoryName(sDirectory);
// Looking for all the documents with the .docx extension
string[] sDocName = Directory.GetFiles(sDocPath, "*.docx", SearchOption.AllDirectories);
string[] sDocumentList = new string[1];
string[] sDocumentText = new string[1];
// Cycling the documents retrieved in the folder
for (int i = 0; i < sDocName.Count(); i++)
{
string docWord = sDocName[i];
// Opening the documents as read only, no need to edit them
Package officePackage = Package.Open(docWord, FileMode.Open, FileAccess.Read);
const String officeDocRelType = #"http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument";
PackagePart corePart = null;
Uri documentUri = null;
// We are extracting the part with the document content within the files
foreach (PackageRelationship relationship in officePackage.GetRelationshipsByType(officeDocRelType))
{
documentUri = PackUriHelper.ResolvePartUri(new Uri("/", UriKind.Relative), relationship.TargetUri);
corePart = officePackage.GetPart(documentUri);
break;
}
// Here enter the proper code
if (corePart != null)
{
string cpPropertiesSchema = "http://schemas.openxmlformats.org/package/2006/metadata/core-properties";
string dcPropertiesSchema = "http://purl.org/dc/elements/1.1/";
string dcTermsPropertiesSchema = "http://purl.org/dc/terms/";
// Construction of a namespace manager to handle the different parts of the xml files
NameTable nt = new NameTable();
XmlNamespaceManager nsmgr = new XmlNamespaceManager(nt);
nsmgr.AddNamespace("dc", dcPropertiesSchema);
nsmgr.AddNamespace("cp", cpPropertiesSchema);
nsmgr.AddNamespace("dcterms", dcTermsPropertiesSchema);
// Loading the xml document's text
XmlDocument doc = new XmlDocument(nt);
doc.Load(corePart.GetStream());
// I chose to directly load the inner text because I could not parse the way I wanted the document, but it works so far
string docInnerText = doc.DocumentElement.InnerText;
docInnerText = docInnerText.Replace("\\* MERGEFORMAT", ".");
docInnerText = docInnerText.Replace("DOCPROPERTY ", "");
docInnerText = docInnerText.Replace("Glossary.", "");
try
{
Int32 iPosKeyword = docInnerText.ToLower().IndexOf(sKeyWord);
Int32 iPosStopWord = docInnerText.ToLower().IndexOf(sStopWord);
if (iPosStopWord == -1)
{
iPosStopWord = docInnerText.Length;
}
if (iPosKeyword != -1 && iPosKeyword <= iPosStopWord)
{
// Redimensions the array if there was already a document loaded
if (sDocumentList[0] != null)
{
Array.Resize(ref sDocumentList, sDocumentList.Length + 1);
Array.Resize(ref sDocumentText, sDocumentText.Length + 1);
}
sDocumentList[sDocumentList.Length - 1] = docWord.Substring(sDocPath.Length, docWord.Length - sDocPath.Length);
// Taking the small context around the keyword
sDocumentText[sDocumentText.Length - 1] = ("(...) " + docInnerText.Substring(iPosKeyword, sKeyWord.Length + 60) + " (...)");
}
}
catch (ArgumentOutOfRangeException)
{
Console.WriteLine("Error reading inner text.");
}
}
// Closing the package to enable opening a document right after
officePackage.Close();
}
if (sDocumentList[0] != null)
{
// Preparing the array for output
string[,] sFinalArray = new string[sDocumentList.Length, 2];
for (int i = 0; i < sDocumentList.Length; i++)
{
sFinalArray[i, 0] = sDocumentList[i].Replace("\\", "");
sFinalArray[i, 1] = sDocumentText[i];
}
return sFinalArray;
}
else
{
// Preparing the array for output
string[,] sFinalArray = new string[1, 1];
sFinalArray[0, 0] = "NO MATCH";
return sFinalArray;
}
}
}
}
The VBA code associated
Option Explicit
Const sLibname As String = "C:\pathToYourDocuments\"
Sub tester()
Dim aFiles As Variant
Dim LookUpDir As BrowserClass.SpecificDirectory
Set LookUpDir = New BrowserClass.SpecificDirectory
' The array will contain all the files which contain the "searchedPhrase" '
aFiles = LookUpDir.LookUpWord("searchedPhrase", "stopWord", sLibname)
' Add here any necessary processing if needed '
End Sub
So in the end you get a tool that can scan .docx documents much faster than in a classic open-read-close approach in VBA at the cost of more code writing.
Above all you get a simple solution for your users that just want to perform simple search, especially when there is a huge number of word documents.
Note
Parsing Word .XML files can be nightmarish in VBA as pointed out by #Mikegrann .
Thankfully OpenXML has an XML parser C# , xml parsing. get data between tags that will do the work for you in C# and take the <w:t></w:t> tags that are refering to the text of the docment. Though I found these answers so far but couldn't make them work:
Parsing a MS Word generated XML file in C# , Reading specific XML elements from XML file
So I went for the .InnerText solution I provided with my code above, to access the internal text, at the cost of having some formatting text input (like \\MERGEFORMAT).
I'm trying to use powershell and Sharepoint 2013 CSOM to copy attachments of one item to a new item in another list. I've been able to successfully generate an attachments folder for the new item, so in theory all I need to do is move the files from the old attachments folder to the new one. CopyTo and MoveTo only seem to work for moving files within a list, so I thought to use OpenBinaryDirect and SaveBinaryDirect with the site context. However, in powershell, calling either of these methods results in the following error: Method invocation failed because [System.RuntimeType] doesn't contain a method named 'OpenBinaryDirect'.
$attachments = $item.AttachmentFiles
if($attachments.Count -gt 0)
{
#Creates a temporary attachment for the new item to genereate a folder, will be deleted later.
$attCI = New-Object Microsoft.SharePoint.Client.AttachmentCreationInformation
$attCI.FileName = "TempAttach"
$enc = New-Object System.Text.ASCIIEncoding
$buffer = [byte[]] $enc.GetBytes("Temp attachment contents")
$memStream = New-Object System.IO.MemoryStream (,$buffer)
$attCI.contentStream = $memStream
$newItem.AttachmentFiles.Add($attCI)
$ctx.load($newItem)
$sourceIN = $sourceList.Title
$archIN = $archList.Title
$sourcePath = "/" + "Lists/$sourceIN/Attachments/" + $item.Id
$archPath = "/" + "Lists/$archIN/Attachments/" + $newItem.Id
$sFolder = $web.GetFolderByServerRelativeUrl($sourcePath)
$aFolder = $web.GetFolderByServerRelativeURL($archPath)
$ctx.load($sFolder)
$ctx.load($aFolder)
$ctx.ExecuteQuery()
$sFiles = $sFolder.Files
$aFiles = $aFolder.Files
$ctx.load($sFiles)
$ctx.load($aFiles)
$ctx.ExecuteQuery()
foreach($file in $sFiles)
{
$fileInfo = [Microsoft.SharePoint.Client.File].OpenBinaryDirect($ctx, $file.ServerRelativeUrl)
[Microsoft.Sharepoint.Client.File].SaveBinaryDirect($ctx, $archPath, $fileInfo.Stream, $true)
}
}
$ctx.ExecuteQuery()
Any help on either getting the BinaryDirect methods to work or just a generalized strategy for copying attachments across lists using powershell + CSOM would be greatly appreciated.
You have the wrong syntax for invoking a static method. You want [Microsoft.SharePoint.Client.File]::OpenBinaryDirect( ... )
Note the double colons syntax :: between the type name and the method name. Same for SaveBinaryDirect invocation.
I'm trying to create a link to a document in SharePoint 2010 using PowerShell 2.0. I've already enabled other content types and added the 'Link to a Document' content type to the document library in question.
The document that I'm trying to link to is in a different shared document list on another web in the same site collection. The actual file is nested in a subfolder called "PM". The file types may range from excel files to word files to PDFs.
I've tested the process manually (Shared Documents -> New Document -> Link to a Document -> ...) and it worked fine (as was indicated by the document icon with an arrow over the bottom right corner when I was done), but I cannot seem to get it to work with PowerShell. Any ideas?
This is the only non-PowerShell solution that I've come accross so far:
http://howtosharepoint.blogspot.com/2010/05/programmatically-add-link-to-document.html
I got it working finally by porting the aforementioned solution. There's superfluous detail here, but the gist of it is easy enough to parse out:
# Push file links out to weekly role page
$site = New-Object Microsoft.SharePoint.SPSite($roleURL)
$web = $site.OpenWeb()
$listName = "Shared Documents"
$list = $web.Lists[$listName]
$fileCollection = $list.RootFolder.files
ForEach ($doc in $docLoadList) {
$parsedFileName = $d.Split("\")
$index = $parsedFileName.Length
$index = $index - 1
$actualFileName = $parsedFileName[$index]
$existingFiles = Get-ExistingFileNames $homeURL
If ($existingFiles -Match $actualFileName) {
$existingFileObject = Get-ExistingFileObject $actualFileName $homeURL
$docLinkURL = Get-ExistingFileURL $actualFileName $homeURL
# Create new aspx file
$redirectFileName = $actualFileName
$redirectFileName += ".aspx"
$redirectASPX = Get-Content C:\Test\redirect.aspx
$redirectASPX = $redirectASPX -Replace "#Q#", $docLinkURL
$utf = new-object System.Text.UTF8Encoding
$newFile = $fileCollection.Add($redirectFileName, $utf.GetBytes($redirectASPX), $true)
# Update the newly added file's content type
$lct = $list.ContentTypes["Link to a Document"]
$lctID = $lct.ID
$updateFile = $list.Items | ? {$_.Name -eq $redirectFileName}
$updateFile["ContentTypeId"] = $lctID
$updateFile.SystemUpdate()
$web.Dispose()
}
}
I may end up stringing together the .aspx file in the script too at some point...
I have the requirement to add performance point filter web part and report view web part to a page in SharePoint 2010 programmatically. I can add both web parts to the page however I have no idea on how to setup connection between them, i.e. for filter web part to be able to send its value to the report view web part.
Any help would be much appreciated.
Found the solution to this :)
What I did was when I create the connection using SPConnectWebParts, I use TransformableBIDataProviderTransformer object, e.g.
var list = new List<TransformProviderConsumerRecord>();
var transformer = new TransformableBIDataProviderTransformer();
var tpcRecord = new TransformProviderConsumerRecord();
tpcRecord = "SqlReportViewUniqueParameterIdSI1";
tpcRecord.ProviderParameterName = "FilterValues";
tpcRecord.DisplayColumnName = "DisplayValue";
tpcRecord.MappingId = (new Guid()).ToString();
tpcRecord.ProviderParameterDisplayName = "PerformancePoint Values";
tpcRecord.TypeFullName = "System.String";
tpcRecord.ValuesColumnName = "DisplayValue";
list.Add(transformProvConsRecord);
ProviderConsumerTransformations provConsTransf = new ProviderConsumerTransformations(list);
var tcr = new TransformerConfigurationRecord(provConsTransf, new TransformConditionalVisibilityRecord());
transformer.ConfigurationState = tcr;
wpm.SPConnectWebParts(providerWp, providerConnection, consumerWp, consumerConnection, transformer);
Where providerWP is Performance Point filter web part amd consumerWp is Performance Point Report