Understanding client-side APIs and web services
SharePoint Server API solution features
HAVE you reached the limits of what you can do with Microsoft SharePoint? Using custom code, you can extend SharePoint in many areas. You’ve seen examples of custom development in the chapters of this book on form development, web content management, and workflow. When you add custom development, solutions in all three of those areas of SharePoint can be improved in one way or another. But those are not the only areas of SharePoint that benefit.
Custom development is taking advantage of the application programming interface (API) provided with SharePoint. There are a number of categories of APIs for SharePoint provided out of the box. One thing they all have in common is the need to use a programming language or custom code. The most common approach in the past has been to use the Microsoft .NET platform and the C# language. However, JavaScript has also become a common language for custom development for the web. And in SharePoint 2013, more than ever, excellent capabilities are provided for JavaScript programmers to extend SharePoint as well.
The SharePoint App Store is another important custom development concept. Through the app store, SharePoint users can extend their implementation by taking advantage of custom code written by others. The app authors and publishers are taking advantage of an area of the SharePoint API called the app model.
SharePoint 2013 has an entirely new workflow engine enhanced by changes to the overall Microsoft platform. This new engine enables new capabilities for creating workflows beyond the tools of the standard information worker, such as SharePoint Designer. Workflow creation through Microsoft Visual Studio provides enhanced capability, reuse in custom solutions, and more solution maintenance tools.
Continue reading this chapter to learn why, when, and how to take advantage of code, probably written by others, to make your SharePoint experience better.
Is SharePoint an application or a platform? Many similar software products are one or the other, but SharePoint is really both. While the majority of this book focuses on the use of SharePoint as an application, it is also important to consider this other aspect of the product. When you make decisions about how and when to extend the application, consider the features of the platform described in this chapter.
Microsoft products have a rich history of providing tools for extending the products with code. Microsoft’s first product was included a BASIC interpreter allowing any user to write programs right out of the box. SharePoint is no different. From the earliest version, Microsoft has provided libraries, often called APIs, for extending the product. SharePoint Portal Server 2001 provided only minimal opportunities to extend the product with custom development compared to the future versions of the product. And 11 years later, SharePoint 2013 offers the most opportunity yet for extending the product with the introduction of the app model.
Previous chapters in this book have already covered some solution types that are created through Visual Studio. Chapter 12 and Chapter 20 cover master pages and page layouts that can be deployed through features and solution packages. Chapter 19 covered building display templates to build better search results. Chapter 21 covered two topics, HTML5 forms and form apps. There is so much coverage already because many SharePoint features live in a gray area between information worker and web developer. This chapter, however, focuses on how experienced developers can help information workers extend SharePoint.
As an advanced information user, you may often be in the position of helping determine how your SharePoint installation is extended with custom or third-party solutions. Whether the source of your addition to SharePoint is an independent software vendor, a custom development group, or your in-house programmers, the end product can come to you in various forms. Not only that, but some of the design decisions made in developing or applying the additional functionality can affect how and where you can use the new functionality.
As you become more familiar with the new app model and the more traditional solution packages, you will be able to determine when solutions built with either may apply to your SharePoint installation.
Figure 23-1 shows an example of an extension of SharePoint 2013 created with the new app model. With the introduction of the app model, the barrier to entry for enhancements through custom development is lower than ever. The App store makes selecting, purchasing, and installing apps easy for SharePoint 2013 users. The app model gives developers greater choice in deployment platform and a better way to distribute solutions.
Adding an app such as the one shown in Figure 23-1 is as simple as clicking the Add It button in the SharePoint App Store on any of the apps listed. Creating an app has fewer dependencies than the deployment model of previous versions of SharePoint. SharePoint development for previous versions required each developer to have a SharePoint server. For the app model, development can be done on the desktop. For these reasons, the introduction of this new model of custom development is one of the most important changes in SharePoint 2013.
The traditional method of deploying custom solutions since SharePoint 2007, Server API solution packages, continues to be available in SharePoint 2013. In fact, some types of solutions still depend on this method of deployment.
There are two types of deployments for the traditional solution package: farm solutions and sandbox solutions. Both types combine the solution assets in one file with an extension of .wsp. This .wsp solution package is deployed to SharePoint directly from the server or by uploading to the Site Collection Solution gallery.
Farm solutions must be installed from a physical SharePoint server and require a high level of trust. Farm solutions must be used to deploy certain types of solutions. For example, a common enhancement discussed earlier in Chapter 12 is branding. If you want to create consistent branding across an entire SharePoint farm, use a farm solution.
Note
For more information on when to choose the Server API and farm solutions, go to http://msdn.microsoft.com/en-us/library/jj164060(v=office.15).aspx.
Sandbox solutions are uploaded to a Site Collection Solution gallery through a link on the Site Collection Settings page. The Microsoft SharePoint Foundation Sandboxed Code Service must be running. Also, the user uploading the solution must have permissions to install sandbox solutions, typically through the Site Collection Administrator role.
Note
For more detail on deprecation of sandbox solutions, read the Important Message section at msdn.microsoft.com/en-us/library/jj163114.aspx.
SharePoint 2013 allows for extensive improvements for accessing the product from beyond the server. The so-called client-side APIs provide programming language–specific hooks into SharePoint. The Representational State Transfer (REST) and Open Data (OData) endpoints provide more generic access to SharePoint through web services. Both provide access beyond what had been possible without developing code directly on a SharePoint server.
Neither method of access is entirely new, but there are many changes, including the depth of the access provided. In part, these changes were necessary to power the new app model deployment method. This access is not limited to app model enhancements, but their use in the app model is a great place to start.
All of the client access methods provide access to a large range of SharePoint functions. For example, here are the new areas of functionality that can be accessed through the client APIs and REST:
User profiles
Search
Taxonomy
Feeds
Publishing
Sharing
Workflow
eDiscovery
IRM
Analytics
Business data
With all of the client APIs and REST providing such widespread access, the choice of method comes down to other considerations, which are listed in the next few sections.
Extending SharePoint from .NET code in C# or VB.Net will be familiar to experienced SharePoint developers. The Microsoft .NET Framework is a key part of Microsoft’s overall platform, so it’s not surprising that the latest enhancements to SharePoint include this API. Because of this history, it is likely to be the most popular method of accessing SharePoint when not using the .NET server API.
The .NET client API is accessed through a .NET assembly that must be available on the server the SharePoint Extension is deployed to. This will be common when deploying apps in a SharePoint Hosted mode.
The JavaScript client API provides custom development options for developers unfamiliar with .NET. JavaScript has become a common language for web development across many platforms. This API will introduce SharePoint development to a new range of capable web developers who have built their programming knowledge on other platforms.
The JavaScript client API is provided through a link to the SharePoint server. This makes it available directly in HTML webpages where the .NET API is not.
REST developers create, update, and delete SharePoint objects using calls following the OData protocol. The REST API is not bound to any particular language, which makes it open for use by developers on open-source platforms.
The REST services are accessed by HTTP addresses at your SharePoint site. For example, if you had a site at: http://intranet.adventureworks.com, the new REST endpoint addresses would start at http://intranet.adventureworks.com/_api/.
Note
To read more about the implementation of REST services and the OData protocol in SharePoint 2013, read “Get started with the SharePoint 2013 REST service,” at msdn.microsoft.com/en-us/library/fp142380(v=office.15).aspx.
When you’d like to extend SharePoint using any of the available methods, you have a choice of building the solution yourself, purchasing an off-the-shelf solution, or paying for a service organization’s help. The decision is not always straightforward, but it’s important to consider all your options. The most obvious solution may not always be the best fit, so it can be helpful to do a little research.
With the introduction of the app model in SharePoint Server 2013, the availability of third-party, off-the-shelf products is expected to grow. While you are still free to find solutions the old way, the apps for the SharePoint 2013 section of the public Office Store is already stocked with products from Microsoft and independent software vendors (ISVs). SharePoint Server 2013 also included the ability to provide a private app management area, which can be stocked by farm administrators.
Note
Browse apps for SharePoint 2013 on the Microsoft Office Store at office.microsoft.com/en-us/store/apps-for-sharepoint-FX102804987.aspx.
When adding an app after your selection from an app store, you will be prompted with permissions requests for the app configured by the publisher. Permissions for apps range from basic information, like the host site name and URL, to full access by the app to the host site. The process of choosing and adding an app with the request for permissions is illustrated in Figure 23-2.
Ratings and reviews by other purchasers is another benefit of the app store model. The traditional server-deployed SharePoint Solution model lacks a central list of out-of-the-box solutions. Websites created by the SharePoint community have attempted to fill the void, but none have achieved 100 percent participation by the vendors or purchasers. With the apps for SharePoint 2013 section of the public Office Store, Microsoft has created a marketplace that will attract third-party developers to advertise their products. This marketplace allows easy rating and reviews by purchasers, which will provide a new level of insight for you when making your selections.
A thriving Microsoft partner network exists for SharePoint custom solution development. When you can’t find an out-of-the box product that solves your needs, engaging a Microsoft partner for custom development can be a helpful option. Careful evaluation of potential partners is necessary, and the best firms are in high demand. If you go this route, make sure that your organization has someone familiar with SharePoint custom solution options as part of the selection and application processes. As with the other methods, part of your end result should be either a SharePoint solution package or an app. Documentation, warranty support, and training are value-added services to watch out for. This option is likely when experienced and knowledgeable in-house development resources are not available or need to be augmented.
Visual Studio continues to be one of the top developer environments in the marketplace. The latest version, Visual Studio 2012, brings improvements tailored to web development, including increased support for HTML and JavaScript. Adding the freely downloadable Office Developer Tools for Visual Studio 2012 provides tools and templates specific to SharePoint solution package and app model development. Equipped with Visual Studio and the software development kits for client and server APIs for SharePoint, your in-house developers can create powerful extensions to SharePoint when you are unable to find them in the third-party market.
When undertaking a custom development project, don’t forget the soft side. Great programmers can build great end products with the proper input and guidance on the business needs. However, SharePoint solution development is not immune to the general custom software development challenges of delivering great results on time and on budget.
For the first time in the product’s history, SharePoint Server 2013 provides the opportunity to write custom solutions on a developer workstation without SharePoint installed. Traditional SharePoint solution package development continues to require a SharePoint server for each developer authoring SharePoint custom solutions. The new opportunity is for developers coding against the newly improved client APIs and REST in the app model. While app development requires a SharePoint server for deployment and testing, that server can be shared by many developers and even by users. Any workstation that meets the minimum requirements of Visual Studio 2012 can be used to write code for the .NET client API. Using JavaScript or REST, the possibilities open up even wider.
The effect of this change for development environments should not be underestimated. Installing and configuring Sharepoint is a significant task that many developers lack the time or resources to undertake. On top of that, the cost of hardware and software resources for a SharePoint server adds to the overall cost of traditional SharePoint solution package development. The move off the server alone reduces obstacles that have prevented many potential SharePoint developers from making the move to learn SharePoint. Add to that the effect of extending beyond the .NET platform to other programming models, and you add another large set of potential programmers for SharePoint extensions.
There are many great resources for SharePoint development, including other Microsoft Press titles. This section does not serve as a programmer’s reference for programming against SharePoint. However, it does provide an introduction to developing with the app model—the easiest and quickest way to get started.
Note
For an in-depth exploration of SharePoint 2013 app development, read Microsoft SharePoint 2013 App Development by Scott Hillier and Ted Pattison, published by Microsoft Press.
To write an app for SharePoint from Visual Studio, you must have the following basic minimum requirements:
Visual Studio 2012 Professional
Office Developer Tools for Visual Studio 2012
Note
A download called the SharePoint 2013 Developer Tools Preview was available was available during the SharePoint 2013 preview period. If you see references to a tool with that name, it has now been replaced with the name Office Developer Tools for Visual Studio 2012.
The free, Express edition of Visual Studio is not supported for SharePoint 2013 development, but the most basic paid version, Professional, is (as are the more premium editions).
That’s it! With just those two tools on your Windows desktop, you are ready to get started developing for SharePoint. To deploy and test your new app, you will need a SharePoint Server 2013 standard development site.
SharePoint Foundation 2013 doesn’t support app deployment, but both the Standard and Enterprise editions do. The target of your deployment is a site collection created with the Developer Site template. In addition, your server must be configured for app management with the required Domain Name System (DNS) configuration on your network. When an app is added to a site, a corresponding app web is created. This app web has a unique address, like app-123456789.apps.adventureworks.com. The creation of this app web at the unique address is what requires the DNS configuration by your network administrator.
Note
For your network administrator’s reference, please read “Configure an environment for apps for SharePoint,” at technet.microsoft.com/en-us/library/fp161236(v=office.15).aspx.
INSIDE OUT: App head start with “Napa” Office 365 Development Tools
An interesting option for experimenting with the new app model is a beta tool called “Napa” Office 365 Development Tools. While Napa is lacking some tools for enterprise development, like versioning and source control, it provides a quick way to get your feet wet with the new custom development option. The following graphic shows a screenshot of a SharePoint Online Developer Site on Office 365 with the “Napa” Office 365 Development Tools installed.
When Napa is installed from the SharePoint App Store, clicking the “Napa” Office 365 Development Tools link takes you to an in-browser SharePoint Development Environment. The graphic that follows shows an example of the in-browser editor experience. As the name implies, Napa is only available on the Office 365 offering, not for on-premises deployments. When accessed in Office 365, Napa provides an editing experience with syntax highlighting, bracket matching, and code completion for JavaScript, CSS, and HTML. It also provides an option to build and deploy your solution to your SharePoint Online Developer Site hosted on Office 365 and an option to share your solution with others. For a more complete development experience, including modifications to advanced project settings, there is an option to export to Visual Studio.
Note
Read more about using Office 365 for starting with SharePoint app development in the MSDN topic “Apps for Office and SharePoint,” at msdn.microsoft.com/en-us/library/fp161507(v=office.15).aspx.
With the minimum requirements for SharePoint app development, you can get started developing useful extensions to your SharePoint site quickly. Figure 23-3 shows the New Project selection dialog box with the SharePoint App Project template selected. When you create a new project based on the SharePoint App Project template, you see that the items in Figure 23-4 have been created for you based on the name you gave the new project. The first property you must set is the site URL of the project properties. This property can be edited by selecting the project title in the Solution Explorer and scrolling through the properties in the lower-right Properties pane to select the site URL property. The site URL property should be set to the address of your developer site. This developer site can be on-premises or in the cloud, so long as you have permissions to access and deploy to it.
If you’d like to get the feel for how easy it is to add visual appeal to SharePoint 2013 with an app for SharePoint, follow these steps to add a map from Bing Maps:
Insert the following code into the Default.aspx content area, starting after the <div> tag and before the </div> tag:
<script type="text/javascript" src="http://dev.virtualearth.net/ mapcontrol/mapcontrol.ashx?v=6.2"></script> <div id='myMap' style="position: relative; width: 650px; height: 400px;"></div>This will source the Bing Maps API from Microsoft and provide a placeholder for the map, as shown next.
Insert the following code into the App.js file in place of the line
getUserName():var map = null; map = new VEMap('myMap'), map.LoadMap();It will draw the map inside the placeholder.
Right-click your project title in Solution Explorer and select Deploy.
Watch the results in the output window for a successful deployment to your developer site.
Navigate to your developer site in your browser and click the new link for your deployed app. You should see a map on the new page.
Congratulations on creating your first app for SharePoint Server 2013. Figure 23-5 shows an example of the result.
As you’ve seen, one place a completed app can be published is the public Office Store. There are stringent requirements and a review process to publish new app store apps, so sharing your app on the Office.Microsoft.com site is a serious undertaking. Also, the public Office Store is meant only for SharePoint apps that can be downloaded and used by others. If the custom app being developed for your SharePoint site doesn’t meet either of those requirements, you can use an internal app catalog to distribute your SharePoint app within your organization. While you should govern the apps that are distributed to a wide internal audience, your requirements will be different from the Microsoft store and you can use your own discretion. To set up your internal app catalog, your farm administrator must add one in the apps section of Central Administration. Figure 23-6 shows an example of an internal app catalog.
As mentioned earlier in this chapter, the app model is a powerful new addition to SharePoint, but the traditional Server API solution approach is likely to be used for some enhancements. Many developers are already familiar with this approach and will be more efficient developing Server API solutions until they learn the intricacies of the new app model. In addition, legacy code may need to be maintained that can be deployed across versions of the product. In this case, more code can be reused when using Server API solutions than the new app model.
Whatever the reason for writing a new enhancement with the Server API, the solution is likely to result in a new feature or features added to the SharePoint farm. Web parts, application pages, event receivers, workflows, web templates, and custom forms are all deployed in custom features in a solution package. Some other elements are provisioned through solution packages outside of features. Any enhancement deployed through a feature is subject to feature scope.
As in the “App model development” section earlier in this chapter, what follows is an overview of the most relevant SharePoint Server API development topics. There are plenty of great references on SharePoint development for 2010, and not much has changed in the Server API side of SharePoint development.
Note
For an in-depth exploration of SharePoint Server API development, read Inside Microsoft SharePoint 2010 by Ted Pattison, Andrew Connell, Scot Hillier, and David Mann, published by Microsoft Press.
Whether an enhancement is deployed by a farm solution or a sandbox solution, the solution package will generally install a new SharePoint feature or features on your farm. Features can be scoped at the farm, web-application, site-collection, or site level. The scope level will determine where you see the feature listed. Farm- and web-application-scoped features will be visible only through Central Administration or Windows PowerShell and must be deployed through a farm solution. Site-collection and site features can be deployed by either method and will be visible in Site Settings to users with permission.
Table 23-1 shows the location in the browser where features of different scopes can be activated and deactivated. A feature of any scope can be activated by an administrator through Windows PowerShell.
Table 23-1. Location of activation page by feature scope
Scope of feature | Location of feature activation page |
Farm | Central Administration, Farm Features |
Web application | Central Administration, Web Application Features |
Site collection | Site Settings, Site Collection Features |
Site | Site Settings, Site Features |
Typically, users in the Site Collection Administrator role control activating and deactivating site-collection features. Site owners or site collection administrators can control activation of site features.
INSIDE OUT: Is my farm customized?
Among the reasons for the introduction of the new app model is reducing infrastructure issues caused by enhancements. However, the app model has not entirely displaced the traditional deployment model.
In certain situations, like moving or upgrading your farm, it is important to determine if your farm has solutions deployed outside the app model. If your farm has custom SharePoint features deployed, it can be easy to determine. For example, the first graphic here shows a partial list of site-collection features on a SharePoint Server Standard 2013 team site. The second graphic shows the site-collection features of a team site with a custom feature.
Other types of enhancements can also be revealed by exploration. The farm administrator can use Windows PowerShell or Central Administration to list all the solution packages that should contain features of all scopes. The ability to catalog installed enhancements is a benefit of the solution package model.
Sometimes, however, changes are made to SharePoint in other ways that are not recommended. Because of their impact on the infrastructure, these types of changes are important to detect.
One example is when files in the SharePoint installation directories are directly changed, removed, or added outside the solution package model. These types of changes may not be as easy to detect through the user interface, but an experienced SharePoint administrator can report them to you through a file-by-file comparison. It might go without saying that this type of change is not recommended, but should be accounted for when moving or upgrading.
One area of SharePoint enhancement that still falls primarily within the realm of SharePoint solution package deployment is the creation of site definitions and web templates. Both site definitions and web templates can be used to define a basis for new site creation, but the web template method is preferred because it doesn’t affect infrastructure changes like upgrades or migrations to the extent that site definitions do.
A key component of a web template is an XML configuration file, called Onet.xml, deployed through a standard SharePoint solution package as a sandbox solution. The Onet.xml file defines the lists, pages, and events of a new site. When you have many sites that must be created with specific configuration items, web templates can be a great solution.
The SharePoint workflow engine has been redesigned for SharePoint 2013. The new engine relies on improvements to Windows Workflow Foundation (WF) and Windows Communication Foundation (WCF). The Windows Azure service bus protocol is used for communication. OAuth, an open standard for authorization, is another key piece of the new engine. Workflow has become even more powerful and more connected to Microsoft platforms and open standards. Creating a workflow or workflow activity in Visual Studio is as simple as adding a new item to a SharePoint project. Figure 23-7 shows SharePoint workflow features for Visual Studio 2012, including the workflow actions available in the toolbar, the workflow sequence editor, and a C# file containing a WF custom activity.
Note
For more detail on changes in SharePoint 2013 workflow, visit SharePoint 2013 workflow fundamentals at msdn.microsoft.com/en-us/library/jj163181(v=office.15).aspx.
Chapter 7 focused on explaining what can be done with workflow in SharePoint 2013 and how to use SharePoint Designer 2013 to create no-code workflows. In fact, reading Chapter 7 will give you a great introduction to the standard actions and capabilities of the new SharePoint 2013 workflow model. To go beyond those standard actions, experienced developers can create more highly customized workflows with Office Developer Tools for Visual Studio 2012. In addition, developers can write custom actions for workflow designers to use in SharePoint Designer 2013. The benefits of customization fall mainly in to three areas: enhanced capability, reusability in custom solutions, and solution maintenance.
There are four types of workflow solutions that can be built with Visual Studio. One type, the declarative workflow, can also be built outside Visual Studio. In Visual Studio, declarative workflow developers have the ability to edit the XML file directly in Code view. When limitations of the workflow model are encountered using the methods described in Chapter 7, look to Visual Studio to extend the reach of the model.
Workflows created in Visual Studio are deployed by SharePoint solution packages and can be combined with other enhancements in the package. The combination can either be wrapped in an app package or deployed as a farm solution. The ability to deploy workflows in these two ways allows for reuse in ways that are not possible through information worker tools. When you want to create a workflow once and use it in combination with other code in a package or across many sites, consider using a Visual Studio workflow.
The tools available to the Visual Studio developer make long-term maintenance of workflows easier and more efficient. Visual Studio includes tools for editing workflows through a Design view, direct editing of workflow XML, or Extensible Object Markup Language (XOML), and additional custom code compiled into .NET assemblies. In addition, there are other tools that extend Visual Studio and support workflow solutions. For example, Team Foundation Server allows for storage and management of workflow versions over time. Together, these tools represent a mature platform for solution development and maintenance that goes beyond what is available to most information workers. When SharePoint workflows grow in complexity or require different maintenance tools, consider using a Visual Studio workflow.
When you consider using Visual Studio to develop workflow solutions, there are four main types of enhancements to consider. Understanding the types will help you plan workflow solutions.
The term declarative isn’t needed to distinguish workflow types outside Visual Studio because it is the only type that can be used to create solutions with standard information worker tools. A declarative workflow is defined by a file using a type of XML markup called XOML. Declarative workflows defined with XOML are different from the code-based workflows available to developers of SharePoint 2010 workflows because they are limited to XML code. .NET code is not available in the new SharePoint 2013 workflow outside of custom activities.
A custom action can be used as a step in a Visual Studio or SharePoint Designer workflow. A custom action is also defined by an XOML file. However, a custom action can also wrap a custom WF activity written in .NET. Writing custom activities in .NET allows the SharePoint 2013 workflow model to use the full, rich foundation of the .NET Framework.
The SharePoint 2013 workflow object model provides workflow instance management, deployment, interop, and messaging services. These services use .NET 4.0 WF enhancements over the .NET Framework 3.5 base of the SharePoint 2010 workflow. The workflow object model can be accessed both from code running on a server and code running on the client side.
The SharePoint 2010 Workflow model is deprecated in SharePoint 2013. While it is still available to run workflows developed for SharePoint 2010, it is not recommended for new development because it may be removed in a future version of SharePoint. 2010 Workflow Interop allows SharePoint 2013 workflows to access and use older SharePoint workflows.
This chapter is the last chapter in this book, but it is also the start of a journey exploring custom development. You can choose from a custom look for SharePoint, better forms for data input, a workflow that matches your processes, or other enhancements of the out-of-the-box features for SharePoint. And now you understand the options to extend SharePoint. You can use code written by others through SharePoint app stores, buy third-party software outside the app stores, hire a software development company, or use in-house programmers to add custom solutions to your SharePoint implementation. Because you have an advanced understanding of SharePoint, you can take full advantage of the customization process. Enhance SharePoint through custom solutions when necessary in order to serve your business and technical requirements more effectively.