File server to SharePoint Online: folder to document library structure diff tool? - sharepoint

Do you know if there is any kind of tool (it would be cool if it is for free) that compares the file structure of a folder in a File Server to a SharePoint Online Document Library and tells you the difference?
The point is we are getting hands on a partially migrated File Server to SharePoint Online and the migration status of some folders is unknown to us. We already have a 3rd party migration tool to finish the migration but we need to know what is still pending to be uploaded. Any idea?
Thanks in advance.

We use sharegate for our migration. In your situation I might just run migrations on small sections with it set to "skip existing". That doesn't directly answer your question though.
In your case I would map both the File server and the SharePoint online instance as drives on a computer (using WebDAV) and hack together an app in your favorite language to compare the two directory structures.
Here is a java program that does that:
import java.io.File;
import java.io.IOException;
import java.io.PrintWriter;
public class Launcher {
static PrintWriter fileOut;
public static void main(String[] args) throws IOException {
File fileStore = new File("N:\\"); //filestore dir top level
File sp = new File("M:\\");//sp directory top level (mapped)
File out = new File("C:\\Temp\\results.txt");
fileOut = new PrintWriter(out);
recurse(fileStore, sp);
fileOut.close();
}
private static void recurse(File dir1, File dir2)
{
File [] dirFiles;
if(dir2 == null)
dirFiles = new File[0];
else
dirFiles=dir2.listFiles();
File match;
for(File f1 : dir1.listFiles())
{
System.out.println(f1.getAbsolutePath());
match=null;
for(File f2:dirFiles)
if(f1.getName().equalsIgnoreCase(f2.getName()))
match=f2;
if(f1.isDirectory() && f1.canRead())
try{
recurse(f1,match);
}
catch(Exception e){}
else if (match == null)
fileOut.write(f1.getAbsolutePath()+"\r\n");
}
}
}
It writes to the given text file each file that is not in the 2nd directory structure.
You could run this code in an IDE like eclipse.

Related

Azure Batch with C# Application using System.Diagnostics.Process

I am using Azure Batch with a C# application. The overall C# application is responsible for doing many things. Unfortunately, 1 operation that works on my local machine does not work within an Azure Batch application.
The operation that works locally but not as an Azure C# application is programmatically starting a System.Diagnostics. A process that executes a FFmpeg argument to take a screenshot of a video. The process seems to run, but, the jpg file is not created. Locally, the code will create a jpg but as an Azure Batch app, the file is not created.
I have set the image file to be created in the AZ_BATCH_TASK_WORKING_DIR folder. I have reduced the code to just do the 1 operation that does not work as an Azure application and here it is:
using System;
using System.Diagnostics;
using System.IO;
namespace BatchAppWithFfmpegProcess
{
internal class Program
{
static void Main(string[] args)
{
#if DEBUG
string workingDirectory = Path.GetDirectoryName(Process.GetCurrentProcess().MainModule.FileName);
#else
/*
Within each Tasks directory, the Batch service creates a working directory (wd) whose unique path is
specified by the AZ_BATCH_TASK_WORKING_DIR environment variable. This directory provides read/write access
to the task. The task can create, read, update, and delete files under this directory.
This directory is retained based on the RetentionTime constraint that is specified for the task.
The stdout.txt and stderr.txt files are written to the Tasks folder during the execution of the task.
*/
string workingDirectory = Environment.GetEnvironmentVariable("AZ_BATCH_TASK_WORKING_DIR");
#endif
var applicationPath = Path.GetDirectoryName(Process.GetCurrentProcess().MainModule.FileName);
string ffmpegPath = Path.Combine(applicationPath, #"lib\");
string videoFilePath = Path.Combine(applicationPath, "MichaelJacksonSmoothCriminal_Trimmed.mp4");
string thumbnailFilePath = Path.Combine(workingDirectory, "MichaelJacksonSmoothCriminal_Trimmed.jpg");
if (File.Exists(thumbnailFilePath))
{
File.Delete(thumbnailFilePath);
}
string arguments = $"-i \"{videoFilePath}\" -ss 00:00:01.000 -frames:v 1 \"{thumbnailFilePath}\"";
var startInfo = new ProcessStartInfo
{
FileName = ffmpegPath + $"\\ffmpeg.exe",
Arguments = arguments,
RedirectStandardError = false,
RedirectStandardOutput = true,
CreateNoWindow = true,
UseShellExecute = false,
WorkingDirectory = ffmpegPath
};
using (var process = new Process { StartInfo = startInfo })
{
process.Start();
process.WaitForExit();
}
if (File.Exists(thumbnailFilePath))
{
Console.WriteLine("Hurray, it worked!!!");
}
else
{
Console.WriteLine("File was not created.");
}
}
}
}
Perhaps it is impossible to use System.Diagnostics.Process to create files? I have tried to make this as easy as possible to reproduce with the following:
clone the code at:
https://github.com/Dapp3rDanH/AzBatchFfmpegProcess.git
Using Visual Studio 2022, "Publish" BatchAppWithFfmpegProcess code to a Folder using
"Deployment mode" = Self-Contained
Target Framework of net5.0
Target runtime = win-x64.
Create a BatchAppWithFfmpegProcess.zip zip file of the publish folder. Make sure the BatchAppWithFfmpegProcess.exe is in the root of the zip.
Create a Batch account.
Add a Batch application using the BatchAppWithFfmpegProcess.zip file with appId of BatchAppWithFfmpegProcess and a version of 1.
Add a Pool called "Pool1" with 1 dedicated node using microsoftwindowsserver windowsserver 2022-datacenter-core (latest). Add BatchAppWithFfmpegProcess version 1 to Pool1.
Create a new Job with a Task with the command of:
cmd /c %AZ_BATCH_APP_PACKAGE_BatchAppWithFfmpegProcess#1%\BatchAppWithFfmpegProcess.exe
If you check out the stdout.txt file of the task, you will see "File was not created". Any way to get this to work?
Thanks!
Dan
The issue was related to my pool being based upon "microsoftwindowsserver windowsserver 2022-datacenter-core (latest)".
Switching to microsoftwindowsserver windowsserver 2016-datacenter (latest) fixed the issue.

How to use createTempFile in groovy/Jenkins to create a file in non-default directory?

What I am trying to achieve is to create a temporary file in groovy in workspace directory, but as an example /tmp/foo will be good enough.
So, here is perfectly working java code:
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.Files;
class foo {
public static void main(String[] args) {
try {
String s="/tmp/foo";
Path p=Paths.get(s);
Path tmp=Files.createTempFile(p,"pref",".suf");
System.out.println(tmp.toString());
} catch (Exception e) {
e.printStackTrace();
}
}
}
however, when used in context of Jenkins pipeline it simply does not work:
def mktemp() {
//String s=pwd(tmp:true)
String s="/tmp/foo"
Path p=Paths.get(s)
Path tmp=Files.createTempFile(p,"pref",".suf")
return tmp;
}
The result is array element type mismatch message with nothing helpful in pipeline log:
java.lang.IllegalArgumentException: array element type mismatch
at java.lang.reflect.Array.set(Native Method)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovyCallSiteSelector.parametersForVarargs(GroovyCallSiteSelector.java:104)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovyCallSiteSelector.matches(GroovyCallSiteSelector.java:51)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovyCallSiteSelector.findMatchingMethod(GroovyCallSiteSelector.java:197)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovyCallSiteSelector.staticMethod(GroovyCallSiteSelector.java:191)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onStaticCall(SandboxInterceptor.java:153)
at org.kohsuke.groovy.sandbox.impl.Checker$2.call(Checker.java:184)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedStaticCall(Checker.java:188)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:95)
at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
at WorkflowScript.mktemp(WorkflowScript:16)
The java.io.File.createTempFile() is not any better. In plain java code it works perfectly. In groovy it throws java.io.IOException: No such file or directory.
BTW, /tmp/foo directory exists, methods are added on script approval screen.
From the IOException I suspect you're calling mktemp from within a node {} block and expecting to create the temporary file on that node. Pipeline scripts are run entirely on the Jenkins master. Pipeline steps that interact with the filesystem (e.g. writeFile) are aware of node {} blocks and will be sent over to the node to be executed there, but any pure-Java methods know nothing about remote nodes and are going to interact with the master's filesystem.

Multiple .zip files extracting on the app directory c#

I have an auto update program for game launcher and I want to know how I can auto unzip all zip files on the same folder with launcher. (Multiple zip files are downloaded). I am a newbie when it comes to coding.
I need to change the following code for extracting all .zip files in the current folder where the app is.
using (var zip = Ionic.Zip.ZipFile.Read("desktop.zip"))
{
zip.ExtractAll(Directory.GetCurrentDirectory(), ExtractExistingFileAction.OverwriteSilently);
}
Try using Reflection:
using (var zip = Ionic.Zip.ZipFile.Read("desktop.zip"))
{
string exeFullPath = System.Reflection.Assembly.GetExecutingAssembly().Location;
string exePath = System.IO.Path.GetDirectoryName(exeFullPath);
zip.ExtractAll(exePath
, ExtractExistingFileAction.OverwriteSilently);
}
#vendettamit is give me the correct answer. If someone need im going to post his code.
foreach (var file in Directory.EnumerateFiles("<directory path>", "*.zip"))
{
using (ZipFile zip = ZipFile.Read(file))
{
foreach (ZipEntry zipFiles in zip)
{
zipFiles.Extract(currentpath, true);
}
}
}

Write to file via jenkins post-groovy script on slave

I'd like to do something very simple: Create/write to a file located in the remote workspace of a slave via the jenkins groovy post-build script plug-in
def props_file = new File(manager.build.workspace.getRemote() + "/temp/module.properties")
def build_num = manager.build.buildVariables.get("MODULE_BUILD_NUMBER").toInteger()
def build_props = new Properties()
build_props["build.number"] = build_num
props_file.withOutputStream { p ->
build_props.store(p, null)
}
The last line fails, as the file doesn't exist. I'm thinking it has something to do with the output stream pointing to the master executor, rather than the remote workspace, but I'm not sure:
Groovy script failed:
java.io.FileNotFoundException: /views/build_view/temp/module.properties (No such file or directory)
Am I not writing to the file correctly?
While writing onto slave you need to check the channel first and then you can successfully create a file handle and start reading or writing to that file:
if(manager.build.workspace.isRemote())
{
channel = manager.build.workspace.channel;
}
fp = new hudson.FilePath(channel, manager.build.workspace.toString() + "\\test.properties")
if(fp != null)
{
String str = "test";
fp.write(str, null); //writing to file
versionString = fp.readToString(); //reading from file
}
hope this helps!
Search for words The post build plugin runs on the manager and doing it as you say will fail if you are working with slaves! on the plugin page (the link to which you've provided) and see if the workaround there helps.
Does the folder /views/build_view/temp exist?
If not, you will need to do new File( "${manager.build.workspace.remote}/temp" ).mkdirs()

Configuring log4net appenders via XML file *and* code

I started to play with log4net today and so far, I really like it. In order to preserve our current logging functionality, the app needs to create a new log file whenever the application is started. The log file name has the date and time stamp encoded in it. Currently, I've got log4net configured via an XmlConfigurator, which works great, except that the filename for my RollingFileAppender is hardcoded in the configuration XML file.
I'd like to continue to use the XmlConfigurator, but after calling Configure(), I want to get at the RollingFileAppender and, in code, change its file value to be a dynamically-generated string. The sample documentation online seems to be down right now, but I've poked through the SDK reference, and it looks like I could use the Heirarchy and GetAppenders() to do what I need to do. Am I on the right track?
Ok, I took a stab at this and tried the following code, which didn't work:
private static readonly ILog _log = LogManager.GetLogger(typeof(GUI));
// in the config file, I've set the filename to example.log, and it works
XmlConfigurator.Configure(new FileInfo("log_config.xml"));
Hierarchy hierarchy = LogManager.GetRepository() as Hierarchy;
if(hierarchy != null) {
// get the appenders
IAppender[] appenders = hierarchy.GetAppenders();
// change the filename for the RollingFileAppender
foreach( IAppender a in appenders) {
RollingFileAppender rfa = a as RollingFileAppender;
if(rfa == null)
continue;
rfa.File = "newfile.log"; // no runtime error, but doesn't work.
}
}
_log.Info("Application started");
Try this snippet:
XmlConfigurator.Configure();
log4net.Repository.ILoggerRepository repo = LogManager.GetRepository();
foreach (log4net.Appender.IAppender appender in repo.GetAppenders())
{
if (appender.Name.CompareTo("RollingFileAppender") == 0 && appender is log4net.Appender.RollingFileAppender)
{
var appndr = appender as log4net.Appender.RollingFileAppender;
string logPath = "MyApplication.log";
appndr.File = logPath;
appndr.ActivateOptions();
}
I had posted similar article here
Do you in this case need the rolling file appender? If not I would expect that your code would create the desired result if you used the normal file appender.
Edit: Maybe it works with the RollingFile Appender if you call ActivateOptions() on the appender.

Resources