Nintex

Repairing List Item Permission Actions in Exported Nintex Workflows

While working on a sizeable SharePoint and Nintex Workflow development project recently, I came across a significant issue in Nintex Workflow that the support engineers at Nintex said was not a bug. I disagree with them, and had to find a workaround anyway, so thought I would share it.

The Problem

If you export a workflow from one SharePoint system (e.g. a development farm), and import it into a different SharePoint system (e.g. a production farm), if you are using the “Set Item Permissions” action within your workflow, you will discover that it is broken after importing. I did a bit of digging, and discovered why – the exported workflow describes the permissions sets in XML by both their name, and their internal IDs (large integers) – but only seems to use the IDs when importing – Nintex Workflow doesn’t try to correlate the permission sets by name on the destination system, so presumes it cannot find the permission sets described in the workflow actions.

It’s worth repeating – the Nintex support engineer I dealt with said this was by design. I was quite shocked.

The Solution

If you’re working on a sizeable project, you probably have all the workflows exported to a folder on the filesystem. You can therefore process the files to replace the IDs from the original system with those of the target system. So – we can run the following PowerShell script on the files, while they are sitting on the destination server(s):


$url = "https://server/sites/site_collection/subsite"

$uri = [System.Uri]$url

# Loop through files in Workflows subdirectory
foreach ( $source_file in $(Get-ChildItem './Workflows' -File | Sort-Object -Property Name) ) {

    Write-Host $("Processing [" + $source_file.Name + "]") -foregroundcolor white
    
    $file_content = Get-Content "../Workflows/$source_file"
    
    # repair the role definitions in the XML
    Write-Host " - Repairing Role Definition IDs in XML"
    foreach ($role_definition in $web.RoleDefinitions){
        $pattern     = $('\#' + $role_definition.Name + '\;\#None\;\#[0-9]+\$\$\#\#')
        $replacement = $('#' + $role_definition.Name + ';#None;#' + $role_definition.Id + '$$$$##')
        $file_content = $file_content -replace $pattern , $replacement
    }
    
    # Write the file into the modified directory
    $file_content | out-file -encoding utf8 "./Workflows/Modified/$source_file"
    
    Write-Host $(" - Finished Processing [" + $source_file.Name + "]")

}

The above snippet presumes you have all your workflows in a folder called “Workflows”, alongside the PowerShell script. It also presumes a sub-folder called “Modified” exists within the Workflows folder, to put the modified workflows into. The script does a regex search for the role definition names in the XML (the permission sets), and swaps them out for the matching ones for the destination system. After running the script, you end up with a set of workflow export files that work.

In my mind, this entire situation could have been avoided if the developers at Nintex had been a bit more forward thinking. At least there is a solution.

Posted by Jonathan Beckett in Notes, 0 comments

Deploying Nintex Forms with PowerShell

When working on a sizeable project with Microsoft SharePoint, Nintex Workflow, and Nintex Forms, it makes sense to automate deployment as much as possible. While it’s straightforward to automate the provisioning of SharePoint assets such as lists, content types, fields, and views through PowerShell, and it’s fairly easy to call NWAdmin to deploy workflows, Nintex Forms have always been something of a problem – until Nintex released a Forms Webservice that is.

It’s still not easy to deploy Nintex Forms via PowerShell, as evidenced by the numerous discussions on both the Nintex community forms, and elsewhere on the internet – with people trying to stick bits of the solution together, and nobody really having the “whole story”. Well this post describes the whole story. I debated for some time about writing this up, because the amount of effort to do it was significant – it gets into that grey area of “this has commercial value”. In the end I decided to share it because I have taken so much from the community over the years, so it might be time to pay something back.

The following PowerShell snippet essentially loops through an arraylist describing the titles of lists, and associated form XML files, and communicates with the Nintex Forms webservice to upload the XML, and publish the forms. It sounds straightforward – it’s anything but. In reality, the script does the following:

  • Calls SharePoint to get a Form Digest
  • Extracts the Form Digest from the response
  • Prepares a POST Web Request to the Nintex Forms Webservice
  • Reads the Form XML file into a byte array
  • Sends the request to the Forms Webservice, streaming the byte array
  • Captures the response from the Forms Webservice

Here’s the guts of it…

$web = Get-SPWeb "https://server/sites/site_collection/subsite"

[System.Reflection.Assembly]::LoadWithPartialName("System.IO") >> $null
[System.Reflection.Assembly]::LoadWithPartialName("Nintex.Forms.SharePoint") >> $null
[System.Reflection.Assembly]::LoadWithPartialName("Nintex.Forms") >> $null

# Build an arraylist of List names, and form xml filenames
$items = New-Object System.Collections.ArrayList
$items.Add(("List A","form_a.xml")) > $null
$items.Add(("List B","form_b.xml")) > $null
$items.Add(("List C","form_c.xml")) > $null

# Check we can see the folder where the form files are
$forms_path = Resolve-Path $(".\Forms\")
if (Test-Path($forms_path)) {

    # loop through the form files
    foreach ($item in $items) {
        
        $list_name = $item[0]
        $form_filename = $item[1]
        
        $form_path = "$forms_path$form_filename"

        Write-Host $(" - Deploying [" + $form_filename + "] to [" + $list_name + "]") -foregroundcolor white
    
        if (Test-Path($form_path)) {

            Write-Host $(" - Form XML File Found")
            
            if ($web.Lists[$list_name]){
                
                # Get Form Digest
                Write-Host " - Getting Form Digest" -NoNewLine
                
                    # Call SharePoint for the Form Digest
                    $form_digest_request = [Microsoft.SharePoint.Utilities.SPUtility]::ConcatUrls($web.Site.RootWeb.Url, "_api/contextinfo")
                    $form_digest_uri = New-Object System.Uri($form_digest_request)
                    $credential_cache = New-Object System.Net.CredentialCache
                    $credential_cache.Add($form_digest_uri, "NTLM", [System.Net.CredentialCache]::DefaultNetworkCredentials)
                    $http_request = [System.Net.HttpWebRequest] [System.Net.HttpWebRequest]::Create($form_digest_request)
                    $http_request.Credentials = $credential_cache
                    $http_request.Method = "POST"
                    $http_request.Accept = "application/json;odata=verbose"
                    $http_request.ContentLength = 0
                    [System.Net.HttpWebResponse] $http_response = [System.Net.HttpWebResponse] $http_request.GetResponse()
                    [System.IO.Stream]$response_stream = $http_response.GetResponseStream()
                    [System.IO.StreamReader] $stream_reader = New-Object System.IO.StreamReader($response_stream)
                    $results = $stream_reader.ReadToEnd()
                    $stream_reader.Close()
                    $response_stream.Close()

                    # Extract the Form Digest Value from the Response
                    $start_tag = "FormDigestValue"
                    $end_tag = "LibraryVersion"
                    $start_tag_index = $results.IndexOf($start_tag) + 1
                    $end_tag_index = $results.IndexOf($end_tag, $start_tag_index)
                    [string] $form_digest = $null
                    if (($start_tag_index -ge 0) -and ($end_tag_index -gt $start_tag_index))
                    {
                        $form_digest = $results.Substring($start_tag_index + $start_tag.Length + 2, $end_tag_index - $start_tag_index - $start_tag.Length - 5)
                    }
                    
                    Write-Host $(" - Form Digest Retrieved")
                
                # Prepare Web Request
                Write-Host " - Preparing Web Request" -NoNewLine
                    
                    $webservice_url = [Microsoft.SharePoint.Utilities.SPUtility]::ConcatUrls($web.Url, "_vti_bin/NintexFormsServices/NfRestService.svc/PublishForm")
                    $webservice_uri = New-Object System.Uri($webservice_url)

                    # Create the web request
                    [System.Net.HttpWebRequest] $request = [System.Net.WebRequest]::Create($webservice_uri)

                    # Add authentication to request 
                    $request.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials

                    # Configure Request
                    $request.Method = "POST";
                    $request.ContentType = "application/json; charset=utf-8";
                    $request.Accept = "application/json, text/javascript, */*; q=0.01"
                    $request.Headers.Add("X-RequestDigest", $form_digest); 
                    $request.Headers.Add("X-Requested-With", "XMLHttpRequest")
                    
                    Write-Host " - Request Prepared"

                # Read XML file into byte array
                Write-Host " - Reading XML File" -NoNewLine
                
                    [system.io.stream] $stream = [system.io.File]::OpenRead($form_path)
                    [byte[]] $file_bytes = New-Object byte[] $stream.length
                    [void] $stream.Read($file_bytes, 0, $stream.Length)
                    $stream.Close()
                    
                    try
                    {
                        $form = [Nintex.Forms.FormsHelper]::XmlToObject([Nintex.Forms.NFUtilities]::ConvertByteArrayToString($file_bytes))
                    } catch [Exception] {
                        $form = [Nintex.Forms.FormsHelper]::XmlToObject([Nintex.Forms.NFUtilities]::ConvertByteArrayToString($file_bytes, [System.Text.Encoding]::UTF8))
                    }

                    $form.LiveSettings.Url = ""
                    $form.LiveSettings.ShortUrl = ""
                    $form.RefreshLayoutDisplayNames()
                    $form.Id = [guid]::NewGuid()

                    $form_json = [Nintex.Forms.FormsHelper]::ObjectToJson($form);
                
                    Write-Host $(" - Json Prepared - [" + $form_json.Length + "] chars")

                # Create the data we want to send
                Write-Host " - Generating Data to Send" -NoNewLine
                
                    $list = $web.Lists[$list_name]
                    $id = "{$($list.ID)}"
                    $data = "{`"contentTypeId`": `"`", `"listId`": `"$id`", `"form`": $form_json }"

                    # Create a byte array of the data we want to send 
                    $utf8 = New-Object System.Text.UTF8Encoding 
                    [byte[]] $byte_array = $utf8.GetBytes($data.ToString())

                    # Set the content length in the request headers 
                    $request.ContentLength = $byte_array.Length;
                    
                    Write-Host $(" - [" + $byte_array.Length + "] bytes prepared")

                # Send the Request
                Write-Host " - Sending the Request" -NoNewLine
                    
                    try {
                        $post_stream = $request.GetRequestStream()
                        $post_stream.Write($byte_array, 0, $byte_array.Length);
                    } catch [Exception]{
                        write-host -f red $_.Exception.ToString() 
                    } finally {
                        if($post_stream) {
                            $post_stream.Dispose()
                        }
                    }
                    
                    Write-Host $(" - Sent [" + $byte_array.Length + "] bytes")

                # Get the Response
                Write-Host " - Processing Response"
                
                    try {
                        [System.Net.HttpWebResponse] $response = [System.Net.HttpWebResponse] $request.GetResponse()

                        # Get the response stream 
                        [System.IO.StreamReader] $reader = New-Object System.IO.StreamReader($response.GetResponseStream())

                        try {
                            $strResult = $reader.ReadToEnd()
                            $jsonResult = ConvertFrom-Json $strResult

                        } catch [Exception] {
                            write-host -f red $_.Exception.ToString() 
                        }
                    } catch [Exception] {
                        write-host -f red $_.Exception.ToString() 
                    } finally {
                        if($response) {
                            $response.Dispose()
                        }
                    }
            
            } else {
            
                # List not found
                Write-Host $(" - List [" + $list_name + "] not found") -foreground-color red
                
            }
            
        } else {
        
            # form_path not found
            Write-Host $(" - Form Path [" + $form_filename + "] not found") -foregroundcolor red
        
        }
        
    } # foreach item in arraylist

} else {

    # forms_path not found
    Write-Host $(" - Forms Path [ " + $forms_path + "] not found") -foregroundcolor red
}

# release resources
$web.Close()
$web.Dispose()

Hopefully this will be useful to somebody, somewhere. I read a lot of documentation to come up with this method, and also looked at pieces of the puzzle that other people had completed before solving it. Of course as with any solution like this, the complexity vanishes at runtime – with forms importing and publishing at a rate of one or two per second into SharePoint – certainly faster than importing and publishing them by hand.

Posted by Jonathan Beckett in Notes, 0 comments

Deploying Nintex Workflows via PowerShell

One of the more common tasks when working on a large project is to deploy Nintex Workflows via a PowerShell script. It’s not too difficult, because the SharePoint web front ends will have a copy of NWAdmin on them – installed in the 15 hive by Nintex during installation. If you’re not aware of it, NWAdmin is a command line tool that can – drum-roll – deploy workflows (among many other things).

The snippet of code below shows the general pattern I use to deploy many workflows in one go – essentially listing them all out in an arraylist, and then looping through it, calling NWAdmin via Powershell. If nothing else, this is a great example of the horrible syntax PowerShell forces upon you to call command line applications with parameters.

$web = Get-SPWeb "https://server/sites/site_collection/subsite"

$cmd = "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\BIN\NWAdmin.exe"

$workflows = New-Object System.Collections.ArrayList

# Fill an array with the worklow names, list names and filenames to process
$workflows.Add(("Process A","List A","process_a.nwf")) > $null
$workflows.Add(("Process B","List B","process_b.nwf")) > $null
$workflows.Add(("Process C","List C","process_c.nwf")) > $null

if (Test-Path($cmd)) {
	Write-Host " - NWAdmin Found" -foregroundcolor green
	$list_workflows_path = Resolve-Path $(".\Workflows\")
	if (Test-Path($list_workflows_path)) {
		foreach ($workflow in $workflows) {

			$workflow_name     = $workflow[0]
			$list_name         = $workflow[1]
			$workflow_filename = $workflow[2]

			$nwf_path = "$list_workflows_path$workflow_filename"

			if (Test-Path($nwf_path)) {
				if ($web.Lists[$list_name]){

					write-host $("Deploying '" + $workflow_name + "' to list '" + $list_name + "'") -foregroundcolor white
					$prm = "-o","DeployWorkflow","-workflowName",$("`"" + $workflow_name + "`""),"-nwfFile",$("`"" + $nwf_path + "`""),"-siteUrl",$("`"" + $web.Url + "`""),"-targetList",$("`"" + $list_name + "`""),"-overwrite"
					& $cmd $prm

				} else {
					Write-Host $("SharePoint List not found [" + $list_name + "]") -foregroundcolor red
				}
			} else {
				write-host $("Workflow File Not Found [" + $nwf_path + "]") -foregroundcolor red
			}
		}
	} else {
		write-host $("Workflows Directory Not Found [" + $list_workflows_path + "]") -foregroundcolor red
	}
	write-host "Complete!" -foregroundcolor green
} else {
	Write-Host " - NWAdmin Not Found" -foregroundcolor red
}

# release resources
$web.Close()
$web.Dispose()
Posted by Jonathan Beckett in Notes, 0 comments

Repairing Nintex Forms in Exported Nintex Workflows

A little while ago I was working on a SharePoint development project with Nintex Workflows and Nintex Forms. The project was being developed remotely – in a virtual machine – and then deployment scripts were given to the client to install on their development, test, and production farms as testing progressed. Along the way we found a pretty serious bug in Nintex Workflow and Forms, but thankfully found a workaround.

The Problem

When you design the forms for tasks within a Nintex Workflow (using Nintex Forms) the form designs are embedded in the Nintex Workflow when you export it. If you export a form directly from the form designer, you end up with an XML file describing the form. If you export a workflow containing a task form design from the workflow designer, you end up with an XML file describing the workflow, with the XML describing the task form escaped within it. The problem comes when you export from one farm, and import into another – you will discover the task forms may fail in the destination system. After a bit of digging, I figured out the the XML describing the forms contains hard-coded server relative paths to the origin system in Lookup fields, that are not dealt with during the import – or at least, that’s how I have seen this problem occur – there may be other field types that also have the source system URL baked into them.

It’s worth noting that I have contacted Nintex Support about this issue – I will update when they get back to me.

The Solution

The solution is pretty straightforward really – you can run some PowerShell to read the exported Workflows, replace the relative paths, and write them back. In the example below, we presume that all of the exported workflows exist within a folder called “Workflows”, and the modified versions will be stored in a subfolder called “Modified”.

# Connect to SharePoint
$web = Get-SPWeb "https://server/sites/site_collection/subsite"

$uri = [System.Uri]$url

# server relative path of site the workflows originally existed at
$source_localpath = "/sites/site_collection/subsite"

# server relative path of the site where the workflows are going to be imported
$destination_localpath = $uri.LocalPath

# escape the paths (because we will find both unescaped and escaped versions of the paths)
$source_localpath_escaped = $source_localpath -replace "/","\\/"
$destination_localpath_escaped = $destination_localpath -replace "/","`\/"

# Loop through files in Workflows subdirectory
foreach ( $source_file in $(Get-ChildItem './Workflows' -File | Sort-Object -Property Name) ) {
    
    # read the workflow file
    $file_content = Get-Content "./Workflows/List Workflows/$source_file"
    
    # replace the paths
    write-host " - Replace Paths"
    $file_content = $file_content -replace $source_localpath,$destination_localpath
    $file_content = $file_content -replace $source_localpath_escaped,$destination_localpath_escaped
            
    # Write the file into the modified subdirectory
    $file_content | out-file -encoding utf8 "../Workflows/Modified/$source_file"

}

# release resources
$web.Close()
$web.Dispose()

The script results in a modified set of exported workflows, which will import correctly into the destination farm. It’s worth noting that if you don’t do this, you end up in a world of trouble, because you can’t repair the forms the workflow import breaks – and you don’t know they are broken until you try to use them. I discovered the cause after digging through the SharePoint ULS logs.

I’m amazed that this particular bug got past quality control at Nintex, but then it’s a fantastically complex piece of kit, and so is SharePoint that it sits on top of. It’s also worth noting that this only effects the development of systems using multiple farms for development, testing, and production – if you do agile development in production, you would never see this problem.

Posted by Jonathan Beckett in Notes, 0 comments