Administration

Batch File to Run PowerShell Scripts and Log Console Output to Log Files

The easiest way to schedule a PowerShell script is to run it from a DOS batch file, and schedule the execution of the batch file on a Windows server via the Task Scheduler. The following batch file will run a PowerShell script, and also organise output from the PowerShell (via Write-Host) into a text file, stored in a sensibly formatted sub-folder and filename structure.

ECHO OFF

REM create a logfile name (log_yyyy-mm-dd-hh-mm-ss.log)
SET logfilename=log_%DATE:~-4%-%DATE:~3,2%-%DATE:~7,2%-%TIME:~0,2%-%TIME:~3,2%-%TIME:~6,2%.log

REM create a folder name (log_yyyy-mm-dd)
SET foldername=log_%DATE:~6,4%-%DATE:~3,2%-%DATE:~0,2%

REM create a log subdirectory
mkdir c:\logs\%foldername%

REM run the powershell script and pipe output to the logfile
powershell.exe "c:\scripts\MyScript.ps1" > "c:\logs\%foldername%\%logfilename%"

The take-away from this script is almost certainly the arcane DOS batch file comments required to get bits and pieces of the date and time.

Posted by Jonathan Beckett in Notes, 0 comments

Repairing List Item Permission Actions in Exported Nintex Workflows

While working on a sizeable SharePoint and Nintex Workflow development project recently, I came across a significant issue in Nintex Workflow that the support engineers at Nintex said was not a bug. I disagree with them, and had to find a workaround anyway, so thought I would share it.

The Problem

If you export a workflow from one SharePoint system (e.g. a development farm), and import it into a different SharePoint system (e.g. a production farm), if you are using the “Set Item Permissions” action within your workflow, you will discover that it is broken after importing. I did a bit of digging, and discovered why – the exported workflow describes the permissions sets in XML by both their name, and their internal IDs (large integers) – but only seems to use the IDs when importing – Nintex Workflow doesn’t try to correlate the permission sets by name on the destination system, so presumes it cannot find the permission sets described in the workflow actions.

It’s worth repeating – the Nintex support engineer I dealt with said this was by design. I was quite shocked.

The Solution

If you’re working on a sizeable project, you probably have all the workflows exported to a folder on the filesystem. You can therefore process the files to replace the IDs from the original system with those of the target system. So – we can run the following PowerShell script on the files, while they are sitting on the destination server(s):


$url = "https://server/sites/site_collection/subsite"

$uri = [System.Uri]$url

# Loop through files in Workflows subdirectory
foreach ( $source_file in $(Get-ChildItem './Workflows' -File | Sort-Object -Property Name) ) {

    Write-Host $("Processing [" + $source_file.Name + "]") -foregroundcolor white
    
    $file_content = Get-Content "../Workflows/$source_file"
    
    # repair the role definitions in the XML
    Write-Host " - Repairing Role Definition IDs in XML"
    foreach ($role_definition in $web.RoleDefinitions){
        $pattern     = $('\#' + $role_definition.Name + '\;\#None\;\#[0-9]+\$\$\#\#')
        $replacement = $('#' + $role_definition.Name + ';#None;#' + $role_definition.Id + '$$$$##')
        $file_content = $file_content -replace $pattern , $replacement
    }
    
    # Write the file into the modified directory
    $file_content | out-file -encoding utf8 "./Workflows/Modified/$source_file"
    
    Write-Host $(" - Finished Processing [" + $source_file.Name + "]")

}

The above snippet presumes you have all your workflows in a folder called “Workflows”, alongside the PowerShell script. It also presumes a sub-folder called “Modified” exists within the Workflows folder, to put the modified workflows into. The script does a regex search for the role definition names in the XML (the permission sets), and swaps them out for the matching ones for the destination system. After running the script, you end up with a set of workflow export files that work.

In my mind, this entire situation could have been avoided if the developers at Nintex had been a bit more forward thinking. At least there is a solution.

Posted by Jonathan Beckett in Notes, 0 comments

Removing Webparts from SharePoint Site Pages with Powershell

The following PowerShell snippet shows how to remove a WebPart from a page in SharePoint by it’s title. I wrote it up because it’s not as straightforward as you might imagine – you need to loop through the WebParts in the page via the “webpartmanager” object – there is no enumeration of WebParts in a page.

$webpart_title = "Hello World"
$page_filename = "home.aspx"

# Connect to Web
$web = Get-SPWeb "https://intranet.contoso.com"

# Instantiate Webpart Manager
$webpartmanager = $web.GetLimitedWebPartManager($($web.Url + "/SitePages/" + $page_filename),[System.Web.UI.WebControls.WebParts.PersonalizationScope]::Shared)

$webpartsarray = @()

for($i=0;$i -lt $webpartmanager.WebParts.Count;$i++) {
  if($webpartmanager.WebParts[$i].title -eq $webpart_title) {
    $webpartsarray = $webpartsarray + $webpartmanager.WebParts[$i].ID
  }
}

$num_webparts = $webpartsarray.length

if ($num_webparts -gt 0)
{
  Write-Host "Found Webpart [$webpart_title]"
  for($j=0; $j -lt $num_webparts; $j++)
  {
    Write-Host $("Deleting WebPart [" + $webpartsarray[$j] + "]")
    $webpartmanager.DeleteWebPart($webpartmanager.WebParts[$webpartsarray[$j]])
  }
}
else
{
  Write-Host "WebPart Not found"
}

# release resources
$web.Close()
$web.Dispose()

Posted by Jonathan Beckett in Notes, 0 comments

Provisioning SharePoint WebPart Pages, and WebParts with PowerShell

One of the more common exercises you might undertake in a PowerShell script when automating the deployment of infrastructure is the creation of WebPart pages, adding WebParts onto those pages, and potentially making one of those pages the default front page for a given site.

The following code snippets illustrate the methods required for each step of the process.

Connect to SharePoint

Before we can issue any instructions to SharePoint we need to add the SharePoint SnapIn to PowerShell, and connect to a web.

if ((Get-PSSnapin "Microsoft.SharePoint.PowerShell" -ErrorAction SilentlyContinue) -eq $null)
{
  Add-PSSnapin "Microsoft.SharePoint.PowerShell"
}
$web = Get-SPWeb "https://intranet.contoso.com"

Create a WebPart Page in the Site Pages Library

Next we create a page in the site pages library – notice that the layout template is chosen by it’s internal ID – you can find these out by searching MSDN for “SharePoint Page Layout Template enumeration”.

$site_pages_library = $web.lists["Site Pages"]
$pageTitle = "My Page"
$layoutTemplate = 4 # Template code
$xml = "" + $site_pages_library.ID + "NewWebPageNewWebPartPage" + $layoutTemplate + "true" + $pageTitle + ""
$result = $web.ProcessBatchData($xml)

Add a WebPart to the Page

First we need to instantiate a WebPart manager object, which will be used to manipulate the webparts within a given page.

$webpartmanager = $web.GetLimitedWebPartManager($web.Url + "/SitePages/My%20Page.aspx", [System.Web.UI.WebControls.WebParts.PersonalizationScope]::Shared)

Add a Content Editor WebPart to the Page

Some example code to add a content editor WebPart to a WebPart page. Notice in particular the final command to the WebPart Manager object, which details the section of the page, and an index number within that section to place the WebPart.

$webpart = new-object Microsoft.SharePoint.WebPartPages.ContentEditorWebPart
$webpart.ChromeType = [System.Web.UI.WebControls.WebParts.PartChromeType]::None
$webpart.Title = "Example Content Editor WebPart"
$docXml = New-Object System.Xml.XmlDocument
$contentXml = $docXml.CreateElement("Content")
$inner_xml = "Hello World!"
$contentXml.set_InnerText($inner_xml) > $null
$docXml.AppendChild($contentXml) > $null
$webpart.Content = $contentXml
$webpartmanager.AddWebPart($webpart, "Header", 1) > $null

Add a List View WebPart to the Page

It turns out adding list views to the page is a bit easier than a content editor webpart. Note that the webpart will take a copy of the view, so if you change the underlying view it is using within the list, it will not affect the WebPart.

$list = $web.Lists["My List"]
$view = "All Items"
$webpart = new-object Microsoft.SharePoint.WebPartPages.ListViewWebPart
$webpart.ChromeType = [System.Web.UI.WebControls.WebParts.PartChromeType]::TitleOnly
$webpart.Title = "Example List View WebPart"
$webpart.ListName = $list.ID
$webpart.ViewGuid = $view.ID
$webpartmanager.AddWebPart($webpart, "Body", 1) > $null

Make the new Page the default front page for the site

One of the more common reasons to provision a page through Powershell is as part of a dashboard that will become the user interface for the SharePoint site – this is how you do that.

$root_folder = $web.RootFolder
$root_folder.WelcomePage = "SitePages/My%20Page.aspx"
$root_folder.Update()

Release Resources

Finally, we need to release the resources PowerShell is holding onto with SharePoint.

$web.Close()
$web.Dispose()
Posted by Jonathan Beckett in Notes, 0 comments

Powershell to Clear a Large SharePoint List

When a SharePoint list grows to millions of rows (cough – Nintex Workflow History List!), it becomes a huge problem to clear it’s contents down. The following script uses a couple of tricks to essentially set a “cursor” to loop through a huge list, and clear it down. It does it in chunks of 1000 items at a time (using the “batch” function of the SharePoint API), and then empties both the site and site collection recycle bins. It runs repeatedly until the offending list is cleared down.

$site_collection_url = "https://intranet.contoso.com"
$list_title = "Things"
$batch_size = 1000

$site = get-spsite $site_collection_url
$web = get-spweb $site_collection_url

$list = $web.Lists[$list_title]

$query = New-Object Microsoft.SharePoint.SPQuery
$query.ViewAttributes = "Scope='Recursive'";
$query.RowLimit = $batch_size
$caml = ''
$query.Query = $caml
$process_count = 0

do
{
    $start_time = Get-Date
    write-host $(" - [Compiling Batch (" + $batch_size + " items)]") -nonewline

    $list_items = $list.GetItems($query)
    $count = $list_items.Count
    $query.ListItemCollectionPosition = $list_items.ListItemCollectionPosition

    $batch = ""

    $j = 0
    for ($j = 0; $j -lt $count; $j++)
    {
        $item = $list_items[$j]
        $batch += "$($list.ID)$($item.ID)Delete$($item.File.ServerRelativeUrl)"
        if ($i -ge $count) { break }
    }

    $batch += ""

    write-host " [Sending Batch]" -nonewline
    $result = $web.ProcessBatchData($batch)

    write-host " [Emptying Web Recycle Bin]" -nonewline
    $web.RecycleBin.DeleteAll()

    write-host " [Emptying Site Recycle Bin]" -nonewline
    $site.RecycleBin.DeleteAll()

    $end_time = Get-Date
    $process_count += $batch_size

    write-host $(" [Processing Time " + ($end_time - $start_time).TotalSeconds + "] [Processed " + $process_count + " so far]") -nonewline
    write-host " [Waiting 2 seconds]"

    start-sleep -s 2

}
while ($query.ListItemCollectionPosition -ne $null)

// Release Resources
$web.Dispose()
$site.Dispose()

The take-away from this script is the ListItemCollectionPosition property of the query object – which appears to work like a cursor. I had never seen it before I started searching for solutions to this problem. It may well be useful again in the future.

Posted by Jonathan Beckett in Notes, 0 comments

Deploying Nintex Workflows via PowerShell

One of the more common tasks when working on a large project is to deploy Nintex Workflows via a PowerShell script. It’s not too difficult, because the SharePoint web front ends will have a copy of NWAdmin on them – installed in the 15 hive by Nintex during installation. If you’re not aware of it, NWAdmin is a command line tool that can – drum-roll – deploy workflows (among many other things).

The snippet of code below shows the general pattern I use to deploy many workflows in one go – essentially listing them all out in an arraylist, and then looping through it, calling NWAdmin via Powershell. If nothing else, this is a great example of the horrible syntax PowerShell forces upon you to call command line applications with parameters.

$web = Get-SPWeb "https://server/sites/site_collection/subsite"

$cmd = "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\BIN\NWAdmin.exe"

$workflows = New-Object System.Collections.ArrayList

# Fill an array with the worklow names, list names and filenames to process
$workflows.Add(("Process A","List A","process_a.nwf")) > $null
$workflows.Add(("Process B","List B","process_b.nwf")) > $null
$workflows.Add(("Process C","List C","process_c.nwf")) > $null

if (Test-Path($cmd)) {
	Write-Host " - NWAdmin Found" -foregroundcolor green
	$list_workflows_path = Resolve-Path $(".\Workflows\")
	if (Test-Path($list_workflows_path)) {
		foreach ($workflow in $workflows) {

			$workflow_name     = $workflow[0]
			$list_name         = $workflow[1]
			$workflow_filename = $workflow[2]

			$nwf_path = "$list_workflows_path$workflow_filename"

			if (Test-Path($nwf_path)) {
				if ($web.Lists[$list_name]){

					write-host $("Deploying '" + $workflow_name + "' to list '" + $list_name + "'") -foregroundcolor white
					$prm = "-o","DeployWorkflow","-workflowName",$("`"" + $workflow_name + "`""),"-nwfFile",$("`"" + $nwf_path + "`""),"-siteUrl",$("`"" + $web.Url + "`""),"-targetList",$("`"" + $list_name + "`""),"-overwrite"
					& $cmd $prm

				} else {
					Write-Host $("SharePoint List not found [" + $list_name + "]") -foregroundcolor red
				}
			} else {
				write-host $("Workflow File Not Found [" + $nwf_path + "]") -foregroundcolor red
			}
		}
	} else {
		write-host $("Workflows Directory Not Found [" + $list_workflows_path + "]") -foregroundcolor red
	}
	write-host "Complete!" -foregroundcolor green
} else {
	Write-Host " - NWAdmin Not Found" -foregroundcolor red
}

# release resources
$web.Close()
$web.Dispose()
Posted by Jonathan Beckett in Notes, 0 comments

Provisioning Large SharePoint Projects with Powershell

It makes sense when building a sizeable project in SharePoint on-premises to script the provisioning of all assets – the lists, content types, columns, views, and so on. This inevitably ends up with a colossal powershell script, so it makes sense to look for ways to break it up, and make it easier to work on (or for multiple people to work on individual parts of the whole). The following is a method I have come up with over the course of several projects – I figured it might be useful for others.

Approach

The basic approach is to run one powershell file that loads all the others, and executes them. Therefore we have one core file (e.g. “deploy.ps1”), and a folder (“lib”) full of scripts it runs. It occurred to me while building the core script that we could copy the way Unix and Linux often do things – and number the files in the folder – then if the core file sorts them, we can control the execution of the children purely by filename order (e.g. “lib/1000_provision_list.ps1”, “lib/2000_provision_content_type.ps1”, and so on).

The Core Deployment Script

So – the following snippet is an example of what the core deployment script looks like – expecting a couple of parameters – the name of the SharePoint site to deploy everything into, and a filter to optionally run a subset, or even a single script, based on it’s filename.

param( [string]$url = "https://server/sites/site_collection/subsite", [string]$filter = "" )

# record results to a text file in the same path as the deploy script
start-transcript ./deploy_log.txt -Append

# Add SharePoint Snap-In
if ((Get-PSSnapin "Microsoft.SharePoint.PowerShell" -ErrorAction SilentlyContinue) -eq $null) 
{
    Add-PSSnapin "Microsoft.SharePoint.PowerShell"
}

if ($url -ne ""){

    Write-Host "Connecting to [$url]" -foregroundcolor green
    $web = Get-SPWeb $url

    # Loop through all files in the lib folder and execute them in order of their filename
    # and if their name matches the filter parameter passed to the script
    foreach ( $source_file in $(Get-ChildItem './lib' | Sort-Object -Property Name) ) {
        
        if ($source_file.Name -like "*$filter*") {
        
            Write-Host $("***** Start  [" + $source_file.Name + "] *****") -foregroundcolor green
            
            & $source_file.FullName
            
            Write-Host $("***** Finish [" + $source_file.Name + "] *****") -foregroundcolor green

        }
    }

    # Release Assets
    $web.Close() > $null
    $web.Dispose() > $null

} else {
    Write-Host "No URL supplied" -foregroundcolor red
}

Write-Host ""
Write-Host "Finished!" -foregroundcolor green

Stop-Transcript

It’s worth noting that the above example uses the “Start-Transcript” and “Stop Transcript” powershell functions, that very neatly record anything we write out through standard output to a text file. I only discovered this a few months ago – it’s very useful.

Writing the Worker Scripts

Apart from knowing we need to put the scripts in the “lib” folder, it’s useful to see what one of the child scripts might look like. The example below shows a good example – which might be saved as “lib/1000_provision_list.ps1”:

function provision_list($web){
    $list_internal_name = "mylist"
    $list_display_name = "My List"
    Write-Host $("Provisioning List [" + $list_display_name + "]")
    if ($web.Lists[$list_display_name]) {
        Write-Host " - $list_display_name Already Exists"
        $list = $web.Lists[$list_internal_name]
    } else {
        Write-Host " - $list_display_name ($list_internal_name)"
        $web.Lists.Add($list_internal_name, "", "GenericList") > $null
        $list = $web.Lists[$list_internal_name]
        $list.Title = $list_display_name
        $list.ContentTypesEnabled = $true
        $list.EnableVersioning = $true
        $list.NavigateForFormsPages = $false
        $list.Update() > $null
    }
}

provision_list($web)

The take-away from the library files is that they share any objects already instantiated by the core script – therefore we can presume $web already contains a valid SharePoint object. Therefore all each file contains is a function definition, and a call to run the function (using functions helps keep working variables scoped).

So there you have it – a working method of building large, flexible, and extendable provisioning scripts. Do comment if you’ve done something similar, or if you found this useful.

Posted by Jonathan Beckett in Notes, 0 comments