Migration

How to Stream Files between Site Collections and Sites in SharePoint with JavaScript

Some time ago I was working on a client-side migration tool to use in SharePoint Online – to copy files and their metadata between libraries both within the same site collection, and into different site collections. It turned out to be a pretty difficult problem – but was solved in the end by using the browser itself as a lifeboat to carry the data between the site collections. If you’re reading this post you probably already looked at the Copy webservice, and “CopyIntoItemsLocal”, which only works when the source and destination are in the same site collection (and causes trouble by populating the hidden “copysource” metadata of the file in the destination file).

Anyway…

Along the way, I discovered a very nice jQuery extension called SPServices, which neatly wrapped up some of the SharePoint API commands, and made the development much more straightforward than it might otherwise have been. The snippet below uses SPServices to load a document within SharePoint into the browser’s memory, and then to write it back into a file elsewhere in SharePoint.

var source_file_url = "https://server/sites/site_collection_a/library_a/document_a.docx";
var destination_file_url = "https://sites/site_collection_b/library_b/document_b.docx";

setTimeout(function() {
    
    var streamLength = 0;
    
    // Read the SourceFileURL into memory
    $().SPServices({
        operation: "GetItem",
        Url: source_file_url,
        async: false,
        completefunc: function (xData, Status) {
            itemstream = $(xData.responseXML).find("Stream").text();
            streamLength = itemstream.length;
            itemfields = "";
            $(xData.responseXML).find("FieldInformation").each(function(){
                itemfields+=$(this).get(0).xml;
            });
        }
    });
    
    if (streamLength) {
    
        // Write the data back into the DestinationFileURL
        $().SPServices({
            operation: "CopyIntoItems",
            SourceUrl: source_file_url,
            async: false,
            DestinationUrls: [destination_file_url],
            Stream: itemstream,
            Fields:itemfields,
            completefunc: function (xData, Status) {
                // file has copied at this point
            }
        });
    }
    else {
        // failed to find list content type
    }
}, 0);

It’s worth noting that at the point the file has been copied, you probably still need to do some work around setting content types (if that is important to you) – I will cover this when I get a chance to write it up, because it’s NOT trivial (especially doing everything on the client-side).

It’s also worth noting that the snippet above is wrapped in a setTimeout call – a handy trick I discovered with asynchronous calls in JavaScript to make sure the code within completes before anything else happens. Kind of a cheap alternative to promises.

Posted by Jonathan Beckett in Notes, 0 comments

Repairing Nintex Forms in Exported Nintex Workflows

A little while ago I was working on a SharePoint development project with Nintex Workflows and Nintex Forms. The project was being developed remotely – in a virtual machine – and then deployment scripts were given to the client to install on their development, test, and production farms as testing progressed. Along the way we found a pretty serious bug in Nintex Workflow and Forms, but thankfully found a workaround.

The Problem

When you design the forms for tasks within a Nintex Workflow (using Nintex Forms) the form designs are embedded in the Nintex Workflow when you export it. If you export a form directly from the form designer, you end up with an XML file describing the form. If you export a workflow containing a task form design from the workflow designer, you end up with an XML file describing the workflow, with the XML describing the task form escaped within it. The problem comes when you export from one farm, and import into another – you will discover the task forms may fail in the destination system. After a bit of digging, I figured out the the XML describing the forms contains hard-coded server relative paths to the origin system in Lookup fields, that are not dealt with during the import – or at least, that’s how I have seen this problem occur – there may be other field types that also have the source system URL baked into them.

It’s worth noting that I have contacted Nintex Support about this issue – I will update when they get back to me.

The Solution

The solution is pretty straightforward really – you can run some PowerShell to read the exported Workflows, replace the relative paths, and write them back. In the example below, we presume that all of the exported workflows exist within a folder called “Workflows”, and the modified versions will be stored in a subfolder called “Modified”.

# Connect to SharePoint
$web = Get-SPWeb "https://server/sites/site_collection/subsite"

$uri = [System.Uri]$url

# server relative path of site the workflows originally existed at
$source_localpath = "/sites/site_collection/subsite"

# server relative path of the site where the workflows are going to be imported
$destination_localpath = $uri.LocalPath

# escape the paths (because we will find both unescaped and escaped versions of the paths)
$source_localpath_escaped = $source_localpath -replace "/","\\/"
$destination_localpath_escaped = $destination_localpath -replace "/","`\/"

# Loop through files in Workflows subdirectory
foreach ( $source_file in $(Get-ChildItem './Workflows' -File | Sort-Object -Property Name) ) {
    
    # read the workflow file
    $file_content = Get-Content "./Workflows/List Workflows/$source_file"
    
    # replace the paths
    write-host " - Replace Paths"
    $file_content = $file_content -replace $source_localpath,$destination_localpath
    $file_content = $file_content -replace $source_localpath_escaped,$destination_localpath_escaped
            
    # Write the file into the modified subdirectory
    $file_content | out-file -encoding utf8 "../Workflows/Modified/$source_file"

}

# release resources
$web.Close()
$web.Dispose()

The script results in a modified set of exported workflows, which will import correctly into the destination farm. It’s worth noting that if you don’t do this, you end up in a world of trouble, because you can’t repair the forms the workflow import breaks – and you don’t know they are broken until you try to use them. I discovered the cause after digging through the SharePoint ULS logs.

I’m amazed that this particular bug got past quality control at Nintex, but then it’s a fantastically complex piece of kit, and so is SharePoint that it sits on top of. It’s also worth noting that this only effects the development of systems using multiple farms for development, testing, and production – if you do agile development in production, you would never see this problem.

Posted by Jonathan Beckett in Notes, 0 comments