Chapter 4. Productivity solutions

This chapter shifts the focus away from the purely technical aspects of the farm, moving instead toward SharePoint farm users. Each of the solutions covered can comprise a major portion of the user’s day-to-day interaction with SharePoint, including upgrade methods, customizations, and functionality provided within the farm itself to improve productivity and business process.

Creating the SharePoint 2016 farm

Copying databases to the new farm

Upgrading service applications

Upgrading the content databases and site collections

The coordination effort concentrates on (1) what other systems will be affected by the upgrade (for example, SQL, Storage, Exchange, Skype, and Infrastructure), and (2) what steps are required to communicate with each of these teams. The level of interaction required with these teams is greatly influenced by how much integration SharePoint 2013 previously had with each of these environments (for instance, existing server-to-server (S2S) protocol trusts, authentication mechanisms, high availability and disaster recovery (HADR), and User Profile service application (UPA) configurations with Active Directory).

This skill pays particular attention to the documentation and verification of the existing farm’s suitability for upgrade, concentrating on the health of the databases, authentication mechanisms, and file systems of the servers themselves.

The documentation created for the upgrade effort should be able to capture the current state and interactions of the existing farm, including:

Server topology and the resulting configuration of service applications and roles

The resiliency capability at each layer or service of the farm

Existing S2S relationships with other server types (SharePoint, Exchange, Skype, Office 365)

Configuration and capacity metrics for web applications and content databases

Current patch state of the farm and content databases

Orphaned items in content databases

Compatibility level of site collections in content databases

Alignment with published boundary, threshold, and supported limits

The eventual result of this effort should be a remediation plan that details steps required to get from the existing farm state to that of a prepared farm migration environment.

Evaluating a content database for upgrade is always done from the destination system (SharePoint 2016, in this case), and requires the following steps:

From SQL Server Management Studio

Back up a copy of the SharePoint 2013 content database and restore it to the SQL instance used with SharePoint 2016.

From SharePoint 2016 farm

Create a new web application and supporting content database.

Match the authentication mode of the SP2013 content database, either classic or claims-mode.

Discard the supporting content database (we won’t be using it).

Run the Test-SPContentDatabase cmdlet, specifying the SP2013 content database to be tested against the web application. Output the test results to a text file.

As we progress through this chapter, understand that Test-SPContentDatabase cmdlet can indicate (but not correct) issues with the SharePoint 2013 content databases. These corrections are usually carried out on the farm to be migrated (SharePoint 2013) prior to being migrated, although some (such as authentication) can be remedied in the destination environment (SharePoint 2016).

Checking a content database for its version status is a fairly straightforward (but commonly overlooked) process, involving only three steps:

1. Determining the farm patch level.

2. Determining if one or more content databases has not upgraded.

3. Determining what the actual schema of the content database is.

From Central Administration on the original SharePoint environment, from System Settings, select Manage Servers In This Farm. The Farm Information items (Figure 4-1) show the configuration database version (the patch level applied to the configuration database, and perhaps the entire SharePoint farm), as well as the configuration database server and name.

Now that the update level for the configuration database has been captured, it can be compared to that of an individual content database. Selecting Review Databases Status from Upgrade and Migration in Central Administration presents the Manage Databases Upgrade Status page. On this page, you will see the content databases (indicated in blue text and hyperlinked) and their current status, as well as the configuration and service application databases.

Note that the screenshot shown in Figure 4-2 shows a status of Database Is In Compatibility Range And Upgrade Is Recommended for the WSS_Content_MeasureMe database, indicating that this database is not currently upgraded to the same level as the other content databases and the configuration database.

Selecting the content database on the Manage Databases Upgrade Status page allows us to inspect the update levels. When the Manage Content Database Settings page opens (partially shown in Figure 4-3), the last line under Database Schema Version indicates two values:

Microsoft.SharePoint.Administration.SPContentDatabase Current Schema Version This value represents the current SharePoint database schema version for the selected content database.

Maximum Schema Version Also known as the Maximum SharePoint Database Schema Version, this indicates the current update level of the farm or configuration database.

A current schema version that is lower than the maximum schema version indicates that this content database has not successfully upgraded to the current update level of the farm, whereas a match between these two values indicates that the database has been successfully made current.

SharePoint 2010 site collections can be identified by using the Get-SPSite PowerShell command and inspecting the compatibility level:

To test all site collections in a farm:

Click here to view code image

Get-SPSite -Limit All | ?  { $_.CompatibilityLevel – eq 14 }

To test site collections within a specific content database:

Click here to view code image

Get-SPSite -ContentDatabase <database name> -Limit All | ?  {
$_.CompatibilityLevel –eq 14 }

If your environment happens to be among those still using classic-mode authentication and you are planning for the upgrade to SharePoint 2016, you will need to make the conversion from classic-mode to claims-mode authentication to configure the new farm in a supported state.

Convert SharePoint 2013 classic-mode web applications to claims-based authentication in the SharePoint 2013 farm, then attach copies of these content databases to SharePoint 2016 claims-mode web applications,

or

Configure SharePoint 2016 classic-mode web applications, attach copies of the SharePoint 2013 classic-mode content databases, and then convert the SharePoint 2016 classic-mode web applications to claims-based authentication.

Child sites without a reference to a parent site

Lists or libraries without a site reference

Apps that are not accessible within a content database

These orphans can be created by any number of causes, including site collections failing to fully provision, incomplete or halted restore operations, features not deploying properly, and so on. Orphans in the content database cannot be reassigned to a reference; thus, they should be deleted from the affected content database.

SharePoint 2013 has two built-in Health Analyzer Rule Definitions (in the Availability category) that help identify these issues prior to upgrade, one of which looks for orphaned apps, and one that looks for all other orphan types. Both of these rules are run on a monthly basis, although you can run them ad hoc before copying a content database.

An alternate solution to allowing the health rule to repair content databases is to use PowerShell cmdlets. This allows you to repair one content database at a time, providing for finer control of this process.

1. Get the content database, assigning it to a variable.

Click here to view code image

$cdb = Get-SPContentDatabase -Identity <content database name>

2. Inspect the content database to see if repairs are needed.

$cdb.repair($false)

At this point, you should receive output from the SharePoint PowerShell console (Figure 4-4). If the OrphanedObjects count is zero, then the database is ready for upgrade. If there are orphaned objects in the content database, a full backup of that database should be completed before proceeding.

3. (Optional) If the content database has orphaned objects, then these must be removed.

$cdb.repair($true)

Regardless of why these orphans exist, the fact remains that their existence in content databases can prevent the upgrade of a content database from SharePoint 2013 to SharePoint 2016.

Essentially, missing file system components are those needed by solutions, branding, Add-ins (apps), and other components that are referenced in the content database. These file system components are expected on the local drives of the individual SharePoint farm servers, and are obviously missing.

This documentation forms the framework that describes the SharePoint farm integration topics such as these:

Server-to-Server (S2S) connectivity Interactions between the SharePoint 2013 farm and other farms might have resulted in a significant arrangement of trusts between farms on premises (and potentially in the cloud). Each of these S2S relationships will need to be re-created between the new SharePoint 2016 farm and its counterparts.

Authentication mechanisms Changes in the authentication architecture for the SharePoint 2016 farm might negate the need for certain authentication types. For instance, are external users going to continue to log in via Forms or are they being moved to an Active Directory Federation Services (AD FS)-connected authentication mechanism?

On-premises secure configurations If your current SharePoint 2013 farm is part of a larger Business Intelligence (BI) infrastructure, it’s likely that Kerberos has been implemented. If this is the case, Service Principal Names (SPNs) must be created in the new environment that allow the existing BI infrastructure to be implemented.

Moving away from Excel Calculation Services Many organizations make regular use of Excel Calculation Services within their SharePoint 2013 configuration. Excel Calculation Services functionality has been moved away from being a service application and into the Office Online Server configuration (which is an on-premises server product outside SharePoint, superseding existing Office Web Application functionality).

Business Connectivity Services Any existing Business Connectivity Services (BCS) connections must be re-established within the new environment, including the creation of the appropriate external content types. This configuration might be affected by plans for utilizing BCS services available in Office 365.

User profile services and application changes In SharePoint 2013, user profile information can be pushed from the SharePoint farm to Active Directory by way of the Forefront Identity Manager (FIM) component found within SharePoint 2013. In SharePoint 2016, this component is replaced by Microsoft Identity Manager 2016 (MIM). It might be desired to move the configuration of the FIM server into MIM to maintain functionality previously available in the SharePoint 2013 farm.

Load balancer configuration External hardware load balancers are commonly in use with SharePoint 2013 farms. The configuration for the SharePoint 2016 farm will involve re-creating the SharePoint 2013 load balancer configuration, but could change based on the architecture chosen; for instance, changes in topology from traditional to streamlined (MinRole) might affect the configuration.

Prior to removing a server from any SharePoint 2013 farm, it’s a good idea to see what role(s) and services the individual server is hosting. Part of the documentation you are doing for the actual migration comes in handy here, allowing you to ask questions like these:

What role(s) did the server host (Front end, batch, and so on)?

Can the availability requirements for certain tiers in the farm be reduced while the migration is taking place?

Is there more than one server in this tier? If so, how many? Are they configured identically from a services perspective?

Are there any specialized services or service applications in this tier that require PowerShell configuration before servers can be removed?

As users are migrated, a careful review of performance at each tier (along with answers to the preceding questions) will determine which servers (if any) can be removed from the SharePoint 2013 farm and repurposed.

Search servers To remove a server from the search topology, you must first clone that topology, remove the server from the cloned topology, and then make it the active topology. Search servers often exist in both the web and application tiers of a traditional topology.

User Profile servers Two services are associated with the User Profile service application: User Profile Service and User Profile Synchronization Service. The latter “Sync” service is only installed on one server in the farm, and thus cannot be removed (unless you move this function to a different server in the farm).

Servers in the distributed cache cluster A single server can provide distributed cache services in a SharePoint farm. A cluster of these servers (3–16 per farm, a two-server installation is not allowed) can also exist. If you have multiple servers in the farm and wish to remove them from the cluster, you can do so via PowerShell. You cannot simply shut down this service from Services on Server, as this does not remove the server from the cluster and will render your farm in an unsupported state.

Workflow Manager servers Workflow Manager servers may exist either inside or outside of the SharePoint farm itself. These servers would not be a good option for removal until the farm is being shut down. If you must remove a Workflow Manager server, ensure that the server is made to leave a farm before proceeding.

Office Web App servers If you have multiple Office Web Apps servers in your existing SharePoint 2013 farm, these are optimal servers for refit into Office Online servers for use in your 2016 farm. As these servers are likely behind a load balancer, each could be removed from the load balancer configuration before being removed from the Office Web Apps server farm and then shut down, as only one server is required in the Office Web Apps farm for it to be used with SharePoint (watch your performance characteristics as you remove each server).

Upgrading the SharePoint farm, then, is directly dependent on the performance levels of the underlying SQL configuration. It could be that you have a constrained window of time in which to perform your upgrades, so it stands to reason that you take advantage of every possible optimization strategy.

It is likely that, while your SharePoint and SQL environments are upgrading a single content database, they are not fully utilizing the processing, memory, or I/O performance available in the farm. Fortunately, it is entirely possible (and fully supported) to attach and upgrade more than one content database at a time, a concept known as parallel upgrades.

There is no particular configuration requirement for performing parallel upgrades; however, there are a handful of recommendations.

PowerShell sessions Use a separate PowerShell session to run each content database upgrade, giving you the opportunity to monitor each separately (and monitor the performance levels of the SQL and SharePoint servers performing the upgrade).

Start times Separate the start time for each new database upgrade session by several minutes, especially when connecting multiple content databases to a single web application. This is necessary to avoid temporary locking, which will cause an error in the upgrade session.

If this configuration is required for your organization, then it’s entirely possible to set an originating (SharePoint 2013) content database to read-only mode, copy it to the SharePoint 2016 environment, and then mount and upgrade the content database.

Prior to the read-only and copy activities, verify the following:

The account used to copy the databases has access to SSMS on both the SharePoint 2013 and SharePoint 2016 environments (data tier), and has access to a network location accessible by both environments.

The account used to set the databases to read-only or read-write is a member of the db_owner fixed database role on the content databases that you are upgrading.

The content databases to be upgraded have been checked for database consistency errors (and repaired, if required).

The appropriate service pack or cumulative update (CU) has been applied (and the update verified in the content databases) to the SharePoint 2013 environment.

SharePoint 2010 and previous SharePoint only allows an upgrade from the immediately preceding version. If your organization is upgrading from an older version of SharePoint to SharePoint 2016, you have two options: perform a step upgrade through previous versions (for example, upgrade to SharePoint 2013 and then upgrade again to SharePoint 2016), or use a third-party tool to extract information from your existing SharePoint environment and restore this information to your new environment.

SharePoint Foundation 2013 There is no similar product in the SharePoint 2016 product line, so you will use a content database attach upgrade to either a SharePoint 2016 Standard or Enterprise farm.

SharePoint 2013 Standard Use a content database attach upgrade to SharePoint 2016 Standard and then optionally upgrade the license to SharePoint 2016 Enterprise farm.

SharePoint 2013 Enterprise Use a content database attach upgrade to a SharePoint 2016 Enterprise farm. Although technically possible, it’s not recommended to migrate from SharePoint 2013 Enterprise to SharePoint 2016 Standard, due to expected solution and feature functionality used in the content databases.

When it’s time to evaluate an upgrade scenario, you might decide to repurpose the staging environment as a test environment for the new SharePoint farm. This environment usually has the same specifications as its production counterpart, and should be identical from a per-server processor, RAM, and I/O standpoint. If this option is chosen, you might want to add servers to this environment to more accurately reflect the MinRole configuration that will be used in the SharePoint 2016 production environment.

Whether you choose to repurpose or reconfigure the existing staging environment or build a new test environment for your SharePoint 2016 upgrade plans, the fact remains that the resulting environment should perform identically to the eventual SharePoint 2016 production farm.

Content database growth A content database usually grows in size while the upgrade is taking place, causing growth of the database itself.

Transaction log growth The matching transaction log can grow significantly during the upgrade process, as backups will likely not yet be configured.

Environmental factors that affect the performance of database and site collection upgrades have to do with the physical configuration of servers in the farm, and include the following:

Simultaneous (parallel) upgrades As we discussed previously, every content database being upgraded requires resources from the SQL server attached to the farm.

SQL Server disk IOPS The number of input/output operations per second (IOPS) provided by the storage layer to the SQL server has a direct correlation to the upgrade speed of the farm as a whole.

SQL Server database to disk layout Multiple content databases may occupy the same physical set of disks; optimally, these disks will be spread across multiple I/O channels and sets of disks, thus minimizing the amount of load on any particular set of disks.

SQL Server temporary database optimizations The TempDB database on the SQL database server itself should be optimized, with the correct number and size of TempDB files.

SQL Server CPU and memory characteristics Sometimes the SQL data layer servers are on their own server, but sometimes the SQL layer is but a single instance of SQL on a larger SQL database server. The amount of processing power and memory available to the instance has a direct bearing on how transactions are carried out (and databases are upgraded).

Web server CPU and memory characteristics The rest of the servers in a SharePoint installation have an influence (albeit a great deal smaller) on upgrade performance. As is the case within SQL, the processor and memory characteristics should be configured so as to not bottleneck performance (and slow or halt the upgrade process).

Network bandwidth and latency While the upgrade is occurring, there will be a considerable amount of communication between the different tiers of the SharePoint server farm. If possible, consider having the intrafarm communications exist on a different virtual local area network (VLAN).

Database factors that affect the performance of database upgrades have to do with information contained within the content database itself, and include the following:

Site collections Each site collection is itself upgraded as part of the content database upgrade process; if the upgrade of a single database will take too long, consider building another content database in the SharePoint 2013 environment prior to upgrade and move a portion of the site collections to the new database.

Subwebs As site collections are subject to upgrade, the number of webs contained within the site collection has a direct effect on the upgrade speed for the site collection.

Lists Lists within SharePoint can contain hundreds, thousands, or more individual items that are part of the upgrade process. Consider working with the site collection administrator to clean large lists if possible.

Rowspan within lists Lists with a significant number of columns (known as wide lists) cause a phenomenon known as rowspan within the underlying SQL content database.

Document versions (number and size) Although this is not as significant an issue as when upgrading from SharePoint 2010, upgrades from SharePoint 2013 to SharePoint 2016 nevertheless have to address each major and minor version of a document to be upgraded. Versions retained can be configured via the GUI or via the object model.

Documents Documents within a SharePoint library can be fairly significant in size and number. Work with your site collection administrator and users to see about removing any outdated or discarded documents. Don’t forget to empty the Recycle Bin for the library itself.

Links Links function identically to lists, and are subject to the same corrective actions.

1. SQL servers This environment is technically configured separately from the rest of a SharePoint environment, but sets the tone for the performance and operation of the entire farm. Items addressed at this level are high availability data recovery efforts and Business Intelligence Services.

2. Application servers Once the SQL server is ready to present a database instance to SharePoint, the next step is to set up application servers. The first server set up in a SharePoint 2016 farm is the de facto Central Administration server (easily changed later if required).

3. Distributed cache servers A SharePoint farm is not valid without at least one distributed cache server present. Remember that if you required resiliency at this tier that a farm with two distributed cache service boxes is not supported, so you must have a minimum of three (or up to 16 total).

4. Search servers Although this MinRole is optional, this grouping of servers should be put in place prior to the Front-end servers if Search is to be required in the farm. The search topology can be configured after the Front-end servers have been configured.

5. Front-end servers The Front-end servers require the least amount of interaction for setup, and would be the last servers required to complete the SharePoint farm.

This conversation becomes important when talking about migrating an on-premises environment wholly to the cloud, because there might be custom functionality that is not supported by SharePoint Online or Office 365. If this is the case, the good news is that you can host the “custom” part of the environment in Azure; configuration and upgrade functions are identical for this environment to what they are on premises.

The remaining functionality can be hosted in SharePoint Online, and must be migrated from an on-premises environment by some means. This is the same effort as is required for moving certain hybrid content to the cloud.

Microsoft FastTrack This is a service intended to help push content to the cloud by engaging a Microsoft Partner.

SharePoint Online Migration API By using the object model, content can be moved from file shares and SharePoint Server sites to SharePoint Online and OneDrive Migration.

Windows PowerShell cmdlets By using PowerShell cmdlets, content can be moved from file shares and SharePoint Server sites to SharePoint Online and OneDrive Migration.

The SharePoint Online Migration API and PowerShell options are intended for importing files to Office 365 by network upload. Depending on the amount of data, both of these options might be inadequate for an efficient transfer to the cloud.

If this is the case, there is another mechanism available: importing files to Office 365 by shipping drives. There are few requirements for this mechanism, specifically the types of drives and adapters and the requirement for BitLocker encryption.

This mechanism requires a connection to SharePoint Online to gather information and prepare the migration.

1. Download and install the drive preparation tool.

2. Download the SharePoint Online Management Shell.

3. Connect to the SharePoint Online tenant.

4. Get the storage account key.

5. Create a SharePoint Online Migration package manifest and data.

6. Prepare the drives.

7. Prepare a mapping file.

8. Upload the drive files and mapping file.

9. Ship the drives.

10. Enter shipping information in the Office 365 admin center.

Once this effort is complete and Microsoft has received the drive, information can be transferred to your Office 365 tenancy.

One reason for this change is simply that the apps can be created and deployed to both on-premises and cloud solutions. This reusability means that customizations can be developed once and deployed somewhat uniformly to both environments.

This zone is created by using the DNS console on a domain controller, and requires the user creating the zone to be a domain administrator.

Using the previous example, you would create a record for *.wingtiptoysaddins.com and point to this by using a CNAME to the SharePoint domain (for example, sharepoint.wingtiptoys.com).

Existing SharePoint sites are using SSL.

Any of the Add-ins to be used connect to data located external to SharePoint sites.

App Management This service application is responsible for storing and providing SharePoint Add-in licenses and permissions, including those downloaded from the SharePoint and Office Store.

Subscription Settings This service application enables the sharing of stored setting data within a set of site collections.

Creating these two new service applications is a bit different in a MinRole-compliant SharePoint 2016 environment than it was in SharePoint 2013. Particularly, the supporting services for each service application (App Management Service and Microsoft SharePoint Foundation Subscription Settings Service) are automatically provisioned and started in servers holding the Front-end and Application roles within the SharePoint 2016 farm (Figure 4-5). No manual start and stop is required.

SharePoint-hosted Used for creating functionality scoped to a single-site collection, created entirely in JavaScript.

Provider-hosted Used for more complex business requirements, this functionality can be used to reach across multiple sites and line-of-business apps, and does not necessarily have to be created in JavaScript.

These Add-ins can be developed in house or can be purchased from the Office Store, within the SharePoint Add-Ins section.

Apps in the catalog site are stored in one of two document libraries: Apps for SharePoint or Apps for Office. As the administrator, you can choose which of these libraries can be utilized by your user base.

A new App Catalog can be created simply by selecting the Apps within Central Administration and then selecting Manage App Catalog. Once there, you can choose to either create a new App Catalog site (no different than creating a standard SharePoint site) or enter a URL for an existing App Catalog site.

This selection is made within the SharePoint Store Settings page of the Apps section in Central Administration, and is configured on a per-web-app basis (shown in Figure 4-7).

Selecting the appropriate web application then allows the admin to choose from three settings:

App Purchases Choose whether users should be able to get apps from the SharePoint Store.

App Requests View any pending app requests.

Apps For Office From The Store Allow Apps for Office to start when users interact with documents in a web browser.

Selecting the web app and then selecting Manage Features (Figure 4-8) allows you to activate the feature called Apps That Require Accessible Internet Facing Endpoints. This can be useful when your users are unable to access the SharePoint Store with the message Sorry, We Can’t Seem To Connect To The SharePoint Store.

This functionality is provided by:

Office Online Server Office Online Server is an on-premises server (not a SharePoint farm server) that delivers browser-based versions of Office applications (such as Microsoft Word). This functionality can be provided to Exchange Server 2016, Skype for Business Server 2015, Lync Server 2013, and SharePoint environments.

Microsoft Access Services Access Services enables the creation and customization of Access add-ins for SharePoint.

Microsoft Visio Services Visio Services allows users to share and view Visio diagrams in a SharePoint site. These diagrams can be configured to be updated by using information from various data sources.

Microsoft Word Automation Services Word Automation Services provides unattended, server-side conversion of documents into formats supported by Microsoft Word.

Microsoft PowerPoint Conversion Services PowerPoint Conversion Services (also known as PowerPoint Automation Services) allows for the conversion of PowerPoint files (.pptx and .ppt) to a number of different formats, including .pptx, .pdf, .xps, .jpg, and .png.

Microsoft Translation Services Translation Services allows for the automatic machine translation of files and sites. This is accomplished by sending these files to the Microsoft Translator cloud-hosted machine translation service, also used by the Office, Skype, Yammer, and Bing translation features.

Configuration steps for this farm are largely unchanged from those required to set up Office Web Apps Server (replaced by Office Online Server), and can be broken down into three major groupings: installing prerequisites, installing Office Online Server binaries, and configuring the connection between Office Online Server and the SharePoint Server 2016 farm.

Install prerequisites Once the physical servers have been built for Office Online Server, additional components and configuration are required before the Office Online Server components can be installed.

Install .NET Framework 4.5.2.

Install the supporting operating system features required by Office Online Server.

Reboot the Office Online Server server(s).

Install Office Online Server This installation requires that you download Office Online Server binaries from the Volume License Service Center.

On the Office Online Server servers, run Setup.exe.

Obtain and install SSL certificates. If you are running split DNS, you will only need a single certificate; if you have different internal and external FQDNs, you will need two different certificates.

Configure DNS to point to the appropriate FQDNs.

Use the New-OfficeWebAppsFarm PowerShell cmdlet to build the new Office Online Server farm.

Configure SharePoint 2016 The newly created Office Online Server environment must be connected to SharePoint.

Use the New-SPWOPIBinding cmdlet to bind the two environments together.

Use the Set-SPWOPIZone cmdlet to configure the zone that the SharePoint will use to navigate the browser to Office Online Server.

Configure Office Online Server to allow HTTP OAuth connections (if required, not recommended for a production environment).

Access Services Access add-ins for SharePoint allow you to build databases in Access 2016 and then share them with other users as an add-in for SharePoint in a web browser.

Access Services 2010 Access Services 2010 is still provided for backward compatibility, and provides the ability to view and edit web databases that were previously created by using Access 2010 and SharePoint Server 2010.

Access Services in SharePoint 2016 provides a series of add-in features, including the following:

Cascading controls

Datasheet filter improvements

Related Item Control enhancements

Image storage and performance improvements

Office Add-ins integration with Access web apps

Additional packaging and upgrade functionality for Access web app packages

On Deploy macro action for upgrade scenarios

Lock tables from editing functionality

Download in Excel feature for datasheet views

The configuration of Access Services on a SharePoint server requires that a series of SQL Server Components be added to the SharePoint servers that host the Front-end role.. Once these components have been added to the SharePoint servers, Access Services configuration requires only five major configuration steps:

1. Configure the SharePoint Server farm for Add-ins (covered in the last section).

2. Configure SQL Server, used to store Access Add-in content.

3. Create a Secure Store service application (covered in the next section).

4. Create the Access Services service application (from either Central Administration or PowerShell).

5. Configure Security.

Only four pieces of information are required to complete this process:

Name The name of the Visio Graphics service application.

Application Pool And Credentials Choose whether to reuse an existing application pool or create a new application pool, providing the managed account for security credentials.

Partitioned Mode Choose whether or not to run in partitioned mode; this is used in multitenancy environments.

Add To Default Proxy List Choose whether or not to add the service application’s proxy to the farm’s default proxy list.

Global Settings This menu allows for the management of performance, security, and data connection refresh rates.

Maximum Drawing Size The maximum size in MB (default is 25) of a web drawing that can be rendered.

Minimum Cache Age The minimum number of minutes (default is 5) that a web drawing is cached in memory.

Maximum Cache Age The number of minutes (default is 60) after which cached web drawings are purged.

Maximum Recalc Duration The number of seconds (default is 60) before data refresh operations time out.

Maximum Cache Size The maximum cache size in MB (default is 5,120) that can be used.

External Data Allows the administrator to assign the target Application ID used to reference Unattended Service Account Credentials.

Trusted Data Providers A listing of trusted data providers available for use with Visio Graphics service, which includes SQL Server, Oracle, IBM, SharePoint, and Excel Web Services.

When the Word Automation service application has been configured, the corresponding Word Automation service is activated automatically on SharePoint servers that host the application role (as this application runs a batch process).

To begin creating this service application, select New, Word Automation service from the Service Applications page in Central Administration (alternately, you could use PowerShell for a scripted configuration).

Five pieces of information are required to complete this process:

Name The name of the Word Automation service application.

Application Pool And Credentials Choose whether to reuse an existing application pool or create a new application pool, providing the managed account for security credentials.

Partitioned Mode Choose whether or not to run in partitioned mode; this is used in multitenancy environments.

Add To Default Proxy List Choose whether or not to add the service application’s proxy to the farm’s default proxy list.

Database Specify the Database Server, Database Name, and Database authentication credentials.

Supported File Formats Which file formats can be opened by the service application.

Open XML Documents (such as .docx, .docm, .dotx, .dotm).

Word 97-2003 (binary) documents (such as .doc, .dot).

Rich Text Format (.rtf).

Webpage (.htm, .html, .mht, .mhtml).

Word 2003 XML Document (.xml).

Embedded Font Support Allows the choice of supporting embedded fonts in a document.

Maximum Memory Usage The maximum percentage of system memory (default is 100 percent) made available in the service application.

Recycle Threshold The number of documents (default is 100) converted by a conversion process before it is restarted.

Word 97-2003 Document Scanning Word 97-2003 documents are a binary format, and can carry hidden payloads; this setting decides whether or not these documents should be scanned prior to conversion.

Conversion Processes Specifies the number of conversion processes (default is 1) created on each server available to the service application.

Conversion Throughput Determines both the frequency in minutes (default is 15) with which conversions are started and the number of conversions to start (default is 300) per conversion process.

Job Monitoring Specifies the length of time before conversions are monitored (default is 5) and potentially restarted.

Maximum Conversion Attempts Specifies the maximum number of times a conversion is attempted (default is 2) before its status is set to Failed.

Maximum Synchronous Conversion Requests Specifies the maximum number of synchronous conversion requests (default is 25) that can be processed at a time on each server available to the service application.

From a creation point of view, PowerPoint Conversion Services is quite a bit different than the Word Automation or Visio Graphics services, as it must be configured in PowerShell. It does, however, require that both a service application and the associated proxy be created.

Note that it is not necessary to attempt to start the PowerPoint Conversion service prior to the creation of the actual PowerPoint Conversion service application and proxy, due to MinRole service controls. When the PowerPoint Conversion service application has been configured, the corresponding service is activated automatically on SharePoint servers that host the Application role (as this application runs a batch process, similar to Word Automation Services).

1. Create the service application by using the New-SPPowerPointConversionServiceApplication cmdlet.

2. Assign the service application to a specified Service Application Pool.

3. Create a matching service application proxy by using the New-SPPowerPointConversionServiceApplicationProxy cmdlet and assign the proxy to a group (usually default).

After the service application has been configured, a local folder named PowerPoint Conversion should be created on each Application role server, and the WSS_WPG group should be given NTFS modify rights. The folder should be created in the C:ProgramDataMicrosoftSharePoint path.

OneDrive for Business OneDrive for Business provides services that had previously been considered part of the on-premises OneDrive for Business configuration (also known as My Sites).

Extensible Hybrid App Launcher The Extensible Hybrid App Launcher is a visual component in the interface that allows Apps (Add-ins) to be pinned to your session in Office 365; as these modifications are made, they are pushed down to the on-premises SharePoint 2016 environment.

OneDrive and Sites

OneDrive only

Regardless of the option chosen, the user will receive access to a OneDrive that replaces the on-premises version. The only configuration option available is the assignment of users to either an on-premises or cloud OneDrive via the use of Audiences.

Two options exist for adding applications to the App Launcher:

Add An Application My Organization Is Developing Uses the Add Application Wizard to capture fields including the App Name, Type, URI, and Sign-on URL.

Add An Application From The Gallery Allows the user to search from and select an App in the Application Gallery.

In the case of applications added from the Gallery, each has a configuration menu that includes the ability to do the following:

Enable single sign-on with Windows Azure Active Directory, including a selection from the following choices:

Windows Azure AD Single Sign-On

Password Single Sign-On

Existing Single Sign-On

Enable automatic user provisioning to the App

Assign users to the App

The Insights service taps local telemetry data to produce a dashboard showing SharePoint administrators which SharePoint features are being used and the actions users are taking, along with numerous advanced analytics about SharePoint use.

The Microsoft SharePoint Insights service will be configured in a similar fashion to hybrid OneDrive and Sites, from the Office 365 menu of Central Administration. The SharePoint Insights Hybrid scenario will be used to create the telemetry relationship between the on-premises SharePoint environment and Office 365. Should you attempt instead to start the Microsoft SharePoint Insights service from Central Administration, Services in Farm, you will be greeted with the following message:

Authentication for Hybrid scenarios is not yet set. Please start the service after SharePoint Insights Hybrid scenario is enabled on your SharePoint farm.

Four main data sources are available for use within BCS:

Windows Communication Foundation

SQL Server

SQL Azure

OData sources (including SQL OData sources)

Selecting the Import icon on the ribbon begins the import process, with the following options:

BDC Model Browse to and upload a BDC model XML file.

File Type Choose the file type (model or resource).

Model A BDC model definition file contains the base XML metadata for a system.

Resource A resource definition file enables you to import or export only the localized names, properties, permissions, or any combination of the three.

Advanced Settings Advanced Settings allows you to do the following:

Choose Which Resources To Import It is possible to select more than one of the following: Localized names (selected by default), Properties (selected by default), and Permissions (not selected).

Custom Environment Settings If you imported a resource file type, it can include custom settings.

Profile Pages are used to display information for an entity (External Content Type). New Profile Pages are added by selecting the External Content Type and then selecting Create/Upgrade Profile Page (Figure 4-12).

Credentials-based authentication User name and password credentials are passed directly from BCS to the external system.

Claims-based authentication The external system will accept credentials from a third-party authentication service (a security token provider). These credentials are comprised of assertions about the requestor (a claim).

Custom authentication If neither credentials- nor claims-based authentication is supported by the external system, then a custom solution will be required to translate credentials from BCS to a format understood by the external system.

Administrative roles Administrators are responsible for permissions management, creating and managing the BDC service application, importing BDC models, and managing permissions. If Add-ins are also used, then administrators will also publish the Add-in and create and manage connection objects.

Developer or Designer roles These roles create the external content types, BDC models, and the Add-ins for SharePoint by using BCS.

User roles Multiple groupings of users may be assigned to consume and possibly manipulate external data in the BCS solution.

Permissioning in BCS needs to be managed for four distinct components of the BCS solution:

External system The external system administrator will assign and manage permissions for the solution; if users are required to use their SharePoint credentials, the Secure Store service might need to be set up in SharePoint.

BCS central infrastructure This has to do with the security of the service application contained within Central Administration on the SharePoint farm; permissions to this service application can be delegated (as can happen on a number of other service applications in SharePoint).

Development environment As a development environment will likely be required for BCS design efforts, this environment can have fewer users but permissions can be a bit more relaxed, so as not to impede development efforts.

User environment User permissions differ based on the mechanisms used for accessing BCS data. For instance, it is a different matter (in terms of scope and execution) to configure external lists and columns, than to assign permissions via Office and SharePoint Add-ins.

Service Application Name A name for the new service application.

Database The server and database names, along with the authentication mechanism used (Windows or SQL).

Failover Server Only used if you implement SQL mirroring.

Application Pool Create a new application pool (or reuse an existing one).

Enable Audit Choose whether or not to enable an Audit log for the Secure Store service (and how many days it spans), which will then be stored in the Secure Store database.

Several guidelines exist for the secure configuration of the Secure Store service application. These guidelines are designed to help secure this database, as it stores potentially sensitive credentials.

Run the Secure Store service in a separate application pool from all other service applications.

Run the Secure Store service in a separate application server not used for any other service.

Deploy the Secure Store database to a different instance of SQL Server than is used for the SharePoint 2016 installation.

On the initial installation of the Secure Store service application, the service will not allow you to create new Target Applications until this key has been generated (Figure 4-13).

Three guidelines exist for the creation and management of the encryption key:

Back up the Secure Store database:

Before generating a new encryption key

After it’s initially created

Each time credentials are re-encrypted

Back up the encryption key (located in the Secure Store service application):

After initially setting up Secure Store

Each time it is regenerated

Do not store the encryption key backup media in the same location as the backup media used for the Secure Store database.

Clicking the Generate New Key icon starts the process of generating and applying the encryption key. This process requires that you create a case-sensitive pass phrase for use with the encryption key, and which is used when adding new Secure Store service servers as well as restoring a backed-up Secure Store database (Figure 4-14).

Once the key has been generated, make sure to record the pass phrase, as both it and the encryption key are required to successfully restore the Secure Store database.

Target Application ID (name) A unique, unchangeable identifier for your Target Application.

Display Name A friendly (display) name for the Target Application.

Contact E-mail The e-mail for the primary contact for this Target Application.

Target Application Type Choose from an Individual Ticket, Individual Restricted, Individual (Default), Group Ticket, Group Restricted, or Group.

Target Application Page URL Choose whether to use Default Page, a Custom Page (specify), or None.

Within the Search service application, selecting New Content Source begins this process, requiring the following fields (shown in Figure 4-20):

Name The name of the Search content source.

Type of content to be crawled Selections include SharePoint Sites, Web Sites, File Shares, Exchange Public Folders, Line Of Business Data (selected), and Custom Repository.

Select The Business Connectivity Service Application Allows for the selection of a BCS service application (assuming there is more than one).

Crawl Either Crawl All External Data Sources or Crawl Selected External Data Source, in this case Wingtiptoys.

There must be an S2S trust with Azure Access Control Service for inbound hybrid connections.

Ensure that all on-premises user accounts accessing the BCS hybrid solution are federated accounts.

Create a service account in the on-premises domain intended to access the OData service endpoint.

Create a global security group in the on-premises domain.

Add the federated accounts from the on-premises domain to the global security group.

Create and configure a Secure Store Target Application, which links the global security group to the service account, both of which were created in the prerequisites section.

Create and configure the OData service endpoint; this will also include the assignment of permissions to the data source represented by this endpoint.

Prepare the SharePoint Online site and App Catalog, identifying the site through which the data will be offered, and configuring the App Catalog (if an Add-in for SharePoint will be used).

Set permissions on the BDC Metadata Store in SharePoint Online (for use with manual imports of External Content Type method).

Validate external access to the on-premises SharePoint environment using the URL published through your reverse proxy.

Create and configure the External Content Type by using Visual Studio.

From BCS in SharePoint Online, configure a connection settings object, which contains additional information used to establish the connection to the external system and OData source.

Sandbox solutions are monitored by SharePoint for resource usage on a per-site-collection basis, and assigned points (via quota) that can be used on a daily basis. If this limit is reached, all sandboxed solutions within the site collection are disabled for the rest of the day, then reset by the Solution Daily Resource Usage Update timer job.

Template Name Choose to either edit an existing quota template or create a new one.

Storage Limit Values Limit site storage and set a warning threshold for an email notification in the site collection.

Sandboxed Solutions With Code Limits Limit the maximum resource usage per day (default is 100 points), also setting a warning threshold for an email notification in the site collection.

Site Collection The site collection selected is the one for which the quota is being changed or unlocked.

Site Lock Information This indicates the existing website collection owner as well as the current lock status for the site (which can be altered): Not Locked, Adding Content Prevented, Read Only, or No Access.

Site Quota Information Allows for the selection of a predefined quota or the assignment of limits for both storage and resource quotas.

Solution Restrictions Within this section, an administrator can do the following:

Name a solution that should be blocked and prevent it from running.

Optionally, display a message to the user, describing why this particular solution is disallowed in the farm.

Load Balancing This section has to do with performance and the load on a particular server.

Choosing to run all sandboxed code on the same machine as a request might perform better for smaller deployments, but not be able to handle a larger number of simultaneous requests.

Choose instead to use solution affinity to balance requests across multiple servers running the Microsoft SharePoint Foundation Sandboxed Code service.

Although full trust solutions cannot be used in SharePoint Online, they are still a viable mechanism for deploying new functionality within an on-premises or cloud-hosted SharePoint farm, such as that found in Microsoft Azure. Before a solution package can be deployed in a SharePoint farm, it has to be submitted to the Solution Store in the configuration database by way of the Add-SPSolution cmdlet.

1. Add the farm solution Before the solution can be deployed, it has to be added to the farm by using the Add-SPSolution cmdlet (there is no provision for adding a farm solution via Central Administration).

2. Deploy the farm solution Once the solution has been added, it can be deployed:

Via PowerShell by using the Install-SPSolution cmdlet

Via Farm Management, Manage Farm Solutions in Central Administration

Fortunately, SharePoint has mechanisms in place that allow you to continue using older solutions, in the form of the -CompatibilityLevel switch, used with the Install-SPSolution cmdlet. On a SharePoint 2016 farm server, three folders exist at C:Program FilesCommon FilesMicrosoft sharedWeb Server Extensions (Figure 4-21), representing three different modes:

14 folder Installing a solution into the 14 folder indicates SharePoint 2010 compatibility mode.

15 folder Installing a solution into the 15 folder indicates SharePoint 2013 compatibility mode.

16 folder Installing a solution into the 16 folder indicates SharePoint 2016 compatibility mode.

Updating a SharePoint solution When the version of a SharePoint solution has been updated, the existing solution can be upgraded in place, if and only if the files and features are exactly the same as in the previous version. This replacement is done by using the Update-SPSolution cmdlet.

Replacing a SharePoint farm solution More often than not, an updated solution means that new components, functionality, and (perhaps) bug fixes are also rolled in for good measure. This type of an update requires that the existing solution be retracted (Uninstall-SPSolution) and redeployed (Install-SPSolution).

From a list of Add-ins that were already available for a site

From the privately available App (Add-in) Catalog

From the commercial Office Store

In particular, this section isn’t about commercially available Add-ins from the SharePoint Store; it concentrates instead on in-house development and deployment of Add-ins. SharePoint Add-ins that are developed in house often provide a level of customization over and above what can be purchased in the store, but they must be maintained just like the older, full trust solutions. These Add-ins can be added to the organization’s private Add-in Catalog.

Within the library, the new Add-in can be uploaded. At this point, the Item Details box will appear, prompting for the Name, Title, Short Description, Icon URL, and other settings.

Securityadmin fixed role on the SQL server instance

Db_owner fixed database role on all databases that are to be updated

Administrators group on the local server where PowerShell cmdlets are to be run

Site Owners group on the site collection to which you want to install the Add-in

The process of deploying an App or Add-in in a farm via PowerShell is similar to deploying a solution:

1. Import the Add-in package Before the Add-in can be installed, it has to be added to the farm by using the Import-SPAppPackage cmdlet (there is no provision for adding an Add-in via Central Administration).

2. Deploy the Add-in Once the Add-in has been added, it can be deployed; in the case of Add-ins, there is no provision for installing them via Central Administration. Install the Add-in via PowerShell by using the Install-SPApp cmdlet.

Updating a SharePoint Add-in When the version of a SharePoint solution has been updated, the existing solution can be upgraded in place if (and only if) the files and features are exactly the same as in the previous version. This replacement is done by using the Update-SPAppInstance cmdlet.

Migrating from an older Add-in to a newer Add-in Depending on the need, you might decide to develop a new Add-in that replaces, rather than upgrades, the original Add-in. If this is the case, you will need to use the same friendly name and file name of the package, but have a different product ID in the manifest. This will allow the Add-in to effectively replace the original in the catalog.

The Test-SPContentDatabase indicates (but does not correct) issues with content databases.

All content databases to be migrated must be first upgraded to a minimum level of 15.0.4481.1005, or Service Pack 1 and the March 2013 PU to be upgraded to SharePoint 2016. If the SharePoint 2013 farm was initially created with the combination SharePoint 2013 and Service Pack 1 media, then the farm is already at this version level.

A correctly upgraded configuration database does not necessarily indicate that all the surrounding content databases are successfully upgraded.

All SharePoint 2010 site collections in the SharePoint 2013 farm must be upgraded to 2013 site collections prior to the SharePoint 2016 upgrade process.

Web Applications need to be upgraded from classic- to claims-aware authentication, as classic mode is no longer supported in SharePoint 2016.

Consider making some content read-only and migrating content to expand migration scheduling opportunities.

The installation sequence for a new SharePoint 2016 farm should begin at the data tier and work out, in accordance with MinRole guidance.

Content can be migrated from SharePoint on-premises to SharePoint Online by using a Microsoft FastTrack partner, the SharePoint Online Migration API, or Windows PowerShell cmdlets. If you have a significant body of content (more than 10 TB), consider shipping the disk media to Microsoft for upload to your tenancy.

Although you can create the App Management service application from Central Administration, you’ll still have to create the Subscription Settings service application from PowerShell. You might as well build both at the same time from PowerShell.

Building new service applications in a MinRole-compliant SharePoint 2016 farm does not require you to start the individual services (as you would have in 2013); the farm will start these for you.

The App Catalog should live in its own DNS space, not in a subdomain or subsites or managed path because that introduces security issues. The FQDN is used with the app hash to create a unique URL.

Office Online Server is an on-premises server, despite the name; it replaces the Office Web Apps Server in a SharePoint 2016 farm.

SharePoint Insights and SharePoint Telemetry provide the ability to view on-premises server events and performance and user metrics from Office 365.

BCS models are XML-based, and can represent four distinct data sources: Windows Communication Foundation, SQL Server, SQL Azure, and OData sources (including SQL OData sources).

BCS supports three different authentication methods: Credentials-based authentication, Claims-based authentication, and Custom authentication.

Hybrid BCS has four prerequisites: that all user accounts are federated, that there is a service account for the OData service endpoint, that there is a global security group in the on-premises domain, and that the federated accounts be added to the global security group.

Apps (Add-ins) are available for deployment from three locations: a list of previously available Add-ins, from the Add-in Catalog, or from the Office Store.

As is the case with solutions, Add-ins can be added from PowerShell.

Add-ins can be either updated or migrated, depending on their complexity.

You are designing a migration plan for moving from SharePoint Server 2013 to a hybrid SharePoint 2016 and SharePoint Online environment. The adoption of the previous environment was quite successful, and code-based customizations to the farm were fairly common. This environment has been in use since 2010, having been upgraded in 2013.

1. How would you approach the initial assessment of the SharePoint 2013 environment, particularly from a code standpoint?

2. What changes will be required from a productivity services standpoint to move the farm into the new hybrid configuration?

3. Management states that they cannot sustain an extended outage. How would you go about completing an upgrade?

1. It’s likely that you will have a combination of code and solutions in this farm. Document both the user solutions (sandboxed) and farm solutions as they exist today (including compatibility version), also capturing the Apps (now Add-ins) that were previously deployed. Work with the development team to determine which solutions and apps are no longer in use, then provide a window of time required to update earlier versions of solutions (if possible). Document the solutions that cannot (or will not) be updated, so they can be installed in a previous compatibility mode.

2. On the whole, most of the productivity services will successfully make the transition between SharePoint 2013 and SharePoint 2016. Create a new Office Online Server, and then work with your user base to determine which Excel Calculation Services components should be replaced by the Office Online Services farm. Remove any other Excel Calculation Services components, as the service is unavailable in SharePoint 2016. Capture the settings of the other Office-based services, such as Visio, Word Automation, and others, for migration to the new environment.

3. Work with management to understand which components of the environment can be down for extended periods (if available). Consider planning a parallel upgrade for certain components of the farm, and then work with the user stakeholders to schedule a freeze, during which they can still access content from the SharePoint 2013 environment, but not write or make changes (this includes workflows). Test the migration for each series of content databases, working with the SQL team to temporarily optimize components in the data layer to handle the upgrade of more than one content database at a time.