Chapter 4. Type Providers

F# 3.0 introduces a new feature called type providers. The major application of a type provider is to provide language features for working within a data-rich environment. One example of a data-rich environment is the database. One major problem when programming against a database is how to generate code for database tables, stored procedures, and other database objects. Not every developer prefers to write database-access code manually, and nice tooling support is something developers always desire. Ideally, you could interact with a rich data environment such as a database using Microsoft Visual Studio features such as IntelliSense. Using IntelliSense would make it easier to write the necessary code, and it would save the time it would otherwise take to continually go back to check the database schema via the database management console.

The SQL Server type provider is designed to solve this problem, and it is shipped with Visual Studio 2012. One of the benefits you get with type providers is that they bring the database or web service schema information to your fingertips in the form of IntelliSense. As a result, when you type, the remote data structure is retrieved, which helps you finish your coding much more quickly. For a data-rich environment, this feature can improve your productivity significantly. If you’re familiar with code-generation tools such as SqlMetal.exe, you can think of a type provider as a wrapper around these tools. At first glance, the type provider seems like something similar to tools that are currently integrated into the Visual Studio integrated development environment (IDE). Actually, an F# type provider does not rely on the IDE. You can use your favorite code editor, and the F# compiler and type provider assembly will bring all of the generated types to you. Note that this process does not involve any IDE work. In addition to being independent of the Visual Studio IDE, a type provider is an open framework compared to other tools, such as svcutil.exe. This means that developers can write their own type provider or providers. I’ll cover how to write a custom type provider in Chapter 5.

The following four built-in type providers can be used to access data from databases and web services:

  • LINQ-to-SQL

  • SQL Entity

  • Web Services Description Language (WSDL)

  • Open Data Protocol (OData)

This chapter focuses on how to use these type providers and F# queries. It also shows how to use type providers to access the Microsoft Windows Azure Marketplace.

Using the LINQ-to-SQL Type Provider

The first obvious data-rich environment is a database. Assume you have a local SQL database, and the database schema is like the one shown in Figure 4-1. This is a simple student-course system. The Student table stores information about all the students. The Course table stores all the course information, and the CourseSelection table is where course-selection information for all students is stored.

Test database diagram
Figure 4-1. Test database diagram

Like any new programming feature, type providers require a certain syntax. Visual Studio ships with a number of item templates to generate the majority of the code. The only thing left for the user to do is fill out connection information. For a SQL type provider, it is the database connection string. Figure 4-2 shows item templates for four built-in type providers. The LINQ-to-SQL type provider is the first one. In this chapter, I use the term SQL type provider to represent the LINQ-to-SQL type provider.

The Add New Item dialog box in Visual Studio, where you can select a LINQ-to-SQL data type provider
Figure 4-2. The Add New Item dialog box in Visual Studio, where you can select a LINQ-to-SQL data type provider

After clicking the Add button, the SQL type provider code is added. Along with the code, a reference to a crucial assembly named FSharp.Data.TypeProviders is also added. This is where the type provider resides. The generated code is shown in Example 4-1. The connection string is highlighted, and that is all you need to change to make this type provider work. The most convenient way to try a new feature is to use F# Interactive (FSI). Most of the samples in this chapter will use FSI as the default way to execute.

Example 4-1. SQL type provider code
 // The SqlDataConnection (LINQ-to-SQL) TypeProvider allows you to write code that
// uses a live connection to a database. For more information, please go to
//    http://go.microsoft.com/fwlink/?LinkId=229209

module SQLDataConnection1

#if INTERACTIVE
#r "System.Data"
#r "System.Data.Linq"
#r "FSharp.Data.TypeProviders"
#endif

open System.Data
open System.Data.Linq
open Microsoft.FSharp.Data.TypeProviders

// You can use Server Explorer to build your ConnectionString.

type SqlConnection = Microsoft.FSharp.Data.TypeProviders.SqlDataConnection<ConnectionStri
ng =
    @"Data Source=(LocalDB)v11.0;Initial Catalog=tempdb;Integrated Security=True">

let db = SqlConnection.GetDataContext()

//let table = query {
//    for r in db.Table do
//    select r
//    }
//

//for p in table do
//    printfn "%s" p.Property

Note

The #if INTERACTIVE section allows the code in Example 4-1 to be executed in FSI. In other words, you can select the code segment and test it in FSI when desired.

When the code needs to be executed in FSI, module SqlDataConnection1 should not be selected.

The type provider accepts a string value for its connection string. The value can be accessed only at run time, and it is very difficult to use a run-time value to generate design time logic. One way to work around this is to use the Literal attribute and conditional compile symbols. The sample code is shown in Example 4-2. This trick can work for all type providers. The other way to specify the connection string is to use a configuration file, which I will leave for you to explore.

Example 4-2. Using the conditional symbol and constant with type providers
#if DEVELOPMENT

[<Literal>]
let conStr = "< my development environment connection string> "

#else

[<Literal>]
let conStr = "< production environment connection string> "

#endif

// define a type provider with constant value
type T = Microsoft.FSharp.Data.TypeProviders.SqlDataConnection<ConnectionString=conStr>
let db = T.GetDataContext()

Note

If you’re not using integrated authorization, the user password is stored in the binary. The GetDataContext method has an overloaded version that takes the connection string at run time. The schema of the database accessed at run time, which is specified by the connection string provided at run time, must match the one used at compile time. If it does not match exactly, at least the schema accessed by your code should be same.

Using conditional compilation symbols is not a best practice when creating assemblies, but it is totally fine when writing an F# script.

The compilation process displays a warning message box like the one shown in Figure 4-3. To retrieve the database schema information, the type provider needs to access the database specified by the connection string. To ensure that you are always aware of this external access, you will be asked if this access should be granted. Once you feel comfortable, you can click Enable to allow the type provider to do its thing.

The Type Provider Security dialog box, which asks you whether you want to enable the specified type provider
Figure 4-3. The Type Provider Security dialog box, which asks you whether you want to enable the specified type provider

Note

For other third-party type providers, to which you later decide to revoke trust, you can go to Tools, Option, F# Tools and press the Delete key to remove the trusted type providers. Figure 4-4 provides an example.

Use the Options dialog box to view and change the trusted type provider assemblies
Figure 4-4. Use the Options dialog box to view and change the trusted type provider assemblies

Depending on the network and database server speed, it can take a few seconds for the type provider to gather all the schema information locally. IntelliSense will be shown when you try to access the database context instance named db. IntelliSense shows the table names and stored procedure names. You do not have to open Server Explorer or Notepad; the schema information from the database is just at your fingertips. See Figure 4-5.

IntelliSense with a type provider
Figure 4-5. IntelliSense with a type provider

When you uncomment the query and FOR loop with printfn inside, you will receive an error because the query targets a fake database. Fortunately, IntelliSense guides you so that you can easily fix those errors and get the executable code looking like what’s shown in Example 4-3. The code prints all of the course names in the table. The F# query feature is a large topic, and I’ll explain it further in the Query section later in the chapter. For now, we can use a simple query syntax to verify that our type provider is working correctly.

Example 4-3. The query code
// query the course table
let table = query {
    for r in db.Course do
    select r
    }

// print out course names
for p in table do
    printfn "%s" p.CourseName

After correcting the compile error, you might find that IntelliSense can also show all the column names for the Course table. This is shown in Figure 4-6.

IntelliSense prompts for column names
Figure 4-6. IntelliSense prompts for column names

Note

IntelliSense shows up only within an F# project. When referenced from a C# project, the provided type and the db variable are visible but no column names are shown. This is also true for other type providers. IntelliSense information is provided only by the F# language service.

The stored procedure is common in SQL databases. By default, the LINQ-TO-SQL type provider allows stored procedures to also be visible. You can set the storedProcedures parameter to false to disable it, as shown in Example 4-4.

Example 4-4. Disabling a stored procedure
type SqlConnection = Microsoft.FSharp.Data.TypeProviders.SqlDataConnection<ConnectionString =
    @"Data Source=taliu0;Initial Catalog=TestDataBase;Integrated Security=True",
    StoredProcedures=false>
let db = SqlConnection.GetDataContext()

The SqlConnection type also contains a class definition representing tables. The generated table class is nested under SqlConnection.ServiceTypes. These types make it possible for you to delete, insert, or update records in a database, as shown in Example 4-5.

Example 4-5. Updating a database

Add a new item

let newRecord = new  SqlConnection.ServiceTypes.Course(CourseName = "aa", CourseID = 100)
db.Course.InsertOnSubmit(newRecord)

try
    db.DataContext.SubmitChanges()
    printfn "record added"
with _ -> printfn "update failed"

Delete an item

db.Course.DeleteOnSubmit(newRecord)

try
    db.DataContext.SubmitChanges()
    printfn "record deleted"
with _ -> printfn "update failed"

More generally, you can execute any SQL command by using the ExecuteCommand method in the DataContext. See Example 4-6.

Example 4-6. Executing a SQL command
try
    let numOfRecord = db.DataContext.ExecuteCommand("select * from Course")
    printfn "%d records affected" numOfRecord
with _ -> printfn "failed"

SQL Type Provider Parameters

The SQL type provider uses SQLMetal.exe as the underlying code-generation tool. The SQL type provider has many parameters that can be configured to specify how SQLMetal will generate the code. Table 4-1 lists all the parameters the SQL type provider supports. Although all the parameters are optional, you have to specify either ConnectionString or ConnectionStringName at the very least.

Table 4-1. SQL type provider parameters

Parameter Name

Description

ConfigFile

The name of the configuration file used for connection strings. Its default value is either app.config or web.config.

ConnectionString

The connection string for the database connection.

ConnectionStringName

The name of the connection string for the database connection in the configuration file.

ContextTypeName

The name of the data context class. The name is derived from the database name by default.

DataDirectory

The name of the data directory; it’s used to replace DataDirectory in the connection strings. The project or script directory is the default value.

ForceUpdate

Requires that a direct connection to the database be availabe at design-time, and forces the refresh of the local schema file. The default value is TRUE.

Functions

Extracts database functions. The default value is TRUE.

LocalSchemaFile

The local .dbml file for the database schema. There is no local schema file by default.

Pluralize

Automatically pluralizes or singularizes class and member names using English language rules. The default value is FALSE.

ResolutionFolder

The folder used to resolve relative file paths at compile time. The default value is the project or script folder.

Serializable

Generates uni-directional serializable classes. The default behavior is to have no serialization.

StoredProcedure

Extracts a stored procedure. The default value is TRUE.

TimeOut

The timeout value is specified in seconds which is used when SqlMetal accesses the database. The default value is 0, which means an infinite amount of time.

Views

Extracts database views. The default value is TRUE.

SQL Entity Type Provider

If you are not satisfied with the code generated by the SQL type provider, which is primarily provided by LINQ-to-SQL, the SQL Entity type provider is another way for you to access a database. According to the MSDN document at http://msdn.microsoft.com/en-us/library/bb386976, LINQ-to-SQL is a SQL access technology that works only with Microsoft SQL Server. The ADO.NET Entity Framework provides a more flexible way to handle not only the SQL Server database but also other relational databases. If you are not familiar with ADO.NET Entity Framework, the following is the definition from MSDN (http://msdn.microsoft.com/en-us/library/bb386876.aspx):

The ADO.NET Entity Framework supports data-centric applications and services, and provides a platform for programming against data that raises the level of abstraction from the logical relational level to the conceptual level. By enabling developers to work with data at a greater level of abstraction, the Entity Framework supports code that is independent of any particular data storage engine or relational schema.

If you try to access an Azure SQL database by using the SQL type provider introduced in the “LINQ-to-SQL Type Provider” section, you’ll get an error. The right way to access the data in an Azure SQL database is to use the SQL Entity type provider. Overall, creating a SQL Entity type provider is similar to creating the SQL type provider shown earlier. Figure 4-7 shows how to add an Entity Framework–based type provider by using the Add New Item dialog box.

Using the Add New Item dialog box to add a SQL Entity type provider
Figure 4-7. Using the Add New Item dialog box to add a SQL Entity type provider

The actual code for the SQL Entity type provider is quite similar to the SQL type provider. The only difference is the type name. The SQL type provider’s type name is SqlDataConnection; the SQL Entity type provider’s type name is SqlEntityConnection. The sample code is shown in Example 4-7.

Example 4-7. SQL Entity type provider
#if INTERACTIVE
#r "System.Data"
#r "System.Data.Entity"
#r "System.Data.Linq"
#r "FSharp.Data.TypeProviders"
#endif

open System.Data
open System.Data.Entity
open Microsoft.FSharp.Data.TypeProviders

// You can use Server Explorer to build your ConnectionString.

type internal SqlConnection =
    Microsoft.FSharp.Data.TypeProviders.SqlEntityConnection<ConnectionString =
        @"Data Source=(LocalDB)v11.0;Initial Catalog=tempdb;Integrated Security=True">

let internal db = SqlConnection.GetDataContext()

Note

Because the SQL Entity type provider cannot be used across assemblies, its type can be only private or internal. This restriction prevents an F# assembly from exposing the type provider to a C# project.

The Entity Framework does not require the database to be SQL Server database. You can use SQL Entity type provider to target an Azure cloud database, as shown in Example 4-8.

Example 4-8. Using the SQL Entity type provider to connect to an Azure cloud database
module SQLEntityConnection1

#if INTERACTIVE
#r "System.Data"
#r "System.Data.Entity"
#r "FSharp.Data.TypeProviders"
#endif

open System.Data
open System.Data.Entity
open Microsoft.FSharp.Data.TypeProviders

[<Literal>]
let con = """Server=tcp:<your server>.database.windows.net,1433;
    Database=<your database>;User ID=<user name>;Password=<your
    password>;Trusted_Connection=False;Encrypt=True;Connection Timeout=30"""

// You can use Server Explorer to build your ConnectionString.
type internal SqlConnectionT =
    Microsoft.FSharp.Data.TypeProviders.SqlEntityConnection<ConnectionString = con>

let internal db = SqlConnectionT.GetDataContext()

The way to add and delete commands for the SQL Entity type provider is different; the type provider uses AddObject and DeleteObject to add and remove a record or records from a database. Example 4-9 shows how to use these two methods.

Example 4-9. Adding and removing records from an Azure cloud database by using the SQL Entity type provider
module SQLEntityConnection1

#if INTERACTIVE
#r "System.Data"
#r "System.Data.Entity"
#r "FSharp.Data.TypeProviders"
#endif

open System.Data
open System.Data.Entity
open Microsoft.FSharp.Data.TypeProviders

[<Literal>]
let con = """Server=tcp:<your server>.database.windows.net,1433;
    Database=<your database>;User ID=<user name>;Password=<your
    password>;Trusted_Connection=False;Encrypt=True;Connection Timeout=30"""

// You can use Server Explorer to build your ConnectionString.
type internal SqlConnectionT =
    Microsoft.FSharp.Data.TypeProviders.SqlEntityConnection<ConnectionString = con>

let internal db = SqlConnectionT.GetDataContext()

let internal newRecord = new SqlConnectionT.ServiceTypes.Course(CourseName = "aa", CourseID = 100)
db.Course.AddObject(newRecord);
try
    let recordNumber = db.DataContext.SaveChanges()
    printfn "%d record(s) affected" recordNumber
with _ -> printfn "update failed"

db.Course.DeleteObject(newRecord);
try
    let recordNumber = db.DataContext.SaveChanges()
    printfn "%d record(s) affected" recordNumber
with _ -> printfn "update failed"

Execution result

1 record(s) affected
1 record(s) affected

If you want to execute your own SQL query, you can use the ExecuteStoreQuery method. Example 4-10 shows how to use this method to check the total number of courses in the database.

Example 4-10. Executing your own query by using the SQL Entity type provider
let result = db.DataContext.ExecuteStoreQuery("select count(*) from Course")
printfn "%d course(s) in the database" (result |> Seq.head)

SQL Entity Type Provider Parameters

Like the SQL type provider, the SQL Entity type provider needs either ConnectionString or ConnectionStringName as a parameter. The full list of parameters is shown in Table 4-2.

Table 4-2. SQL Entity type provider parameters

Parameter Name

Description

ConfigFile

The name of the configuration file used for connection strings. The default value is either app.config or web.config.

ConnectionString

The connection string to the database.

ConnectionStringName

The name of the connection string for the database connection in the configuration file.

DataDirectory

The name of the data directory, used to replace DataDirectory in the connection string. The default value is the project folder or script folder.

EntityContainer

The name to use for the EntityContainer in the conceptual model.

ForceUpdate

Requires that a direct connection to the database be available at design time, and forces the refresh of the local schema file. The default value is TRUE.

LocalSchemaFile

The local file for the database schema.

Pluralize

Automatically pluralizes or singularizes class and member names using English language rules. The default value is FALSE.

Provider

The name of the ADO.NET data provider to be used for Store Schema Definition Language (SSDL) generation. If you are interested in SSDL, please refer to MSDN document at http://msdn.microsoft.com/en-US/data/jj652016. The default provider is System.Data.SqlClient.

ResolutionFolder

The folder used to resolve relative file paths at compile time. The default value is the project folder or script folder.

SuppressForeignKeyProperties

Excludes foreign key properties in entity type definitions. The default value is FALSE.

WSDL Type Provider

The web is another area that is rich with data. The public application programming interfaces (APIs) exposed by companies around the world is increasing exponentially year by year. If the data schema can be automatically accessed and shown with IntelliSense, this will make your coding significantly easier and boost productivity.

Web Services Description Language (WSDL) is an XML-based language which is used to describe the functionality provided by a web service. A WSDL file offers a machine-readable XML document that explains how the service can be called, what operations are available, what parameters are required, and the values that can be returned.

The easiest way to add a WSDL type provider is to use the Add New Item dialog box in Visual Studio, which is shown in Figure 4-8. The code generated is shown in Example 4-11. When you connect to your favorite data source, you get not only IntelliSense about data names but also comments to help you understand what the data is for. Isn’t that nice!

Using the Add New Item dialog box to add a WSDL type provider
Figure 4-8. Using the Add New Item dialog box to add a WSDL type provider
Example 4-11. WSDL type provider
open System.Runtime.Serialization
open System.ServiceModel
open Microsoft.FSharp.Data.TypeProviders

// You can sign up for a Bing service developer account at http://msdn.microsoft.com/en-
us/library/gg605201.aspx
let BING_APP_ID = "<your Bing Maps Developer Key>"

// Using Bing Map API routing service to calculate the driving distance between two
Geolocations. http://www.microsoft.com/maps/developers/mapapps.aspx

type RouteService = Microsoft.FSharp.Data.TypeProviders.WsdlService<ServiceUri =
    "http://dev.virtualearth.net/webservices/v1/routeservice/routeservice.svc?wsdl">
    type RouteCommon = RouteService.ServiceTypes.dev.virtualearth.net.webservices.
v1.common
    type Route = RouteService.ServiceTypes.dev.virtualearth.net.webservices.v1.route

let startPoint = Route.Waypoint(
                        Location = RouteCommon.Location(
                                         Latitude = 47.64012046, Longitude =
-122.1297104))
let endPoint = Route.Waypoint(
                        Location = RouteCommon.Location(
                                         Latitude = 47.62049103, Longitude =
-122.3492355))

let routReq1 = new Route.RouteRequest(Waypoints = [|startPoint; endPoint|])
routReq1.Credentials <- new RouteCommon.Credentials(ApplicationId = BING_APP_ID)

RouteService.GetBasicHttpBinding_IRouteService().CalculateRoute(routReq1).Result
    .Summary.Distance |> printfn "Driving Distance = %A (miles)"

Note

A Bing maps developer key is required to see IntelliSense and compile the code.

WSDL Type Provider Parameters

The parameters for the WSDL type provider are listed in Table 4-3. Only ServiceUri is a mandatory parameter; the other parameters are optional.

Table 4-3. WSDL type provider parameters

Parameter Name

Description

Async

Indicates if whether both synchronous and asynchronous method signatures should be generated. The default value is FALSE, which means that only synchronous method signatures will be generated.

CollectionType

A fully-qualified or assembly-qualified name of the type to use as a collection data type when code is generated from the schema.

EnableDataBinding

Indicates whether the INotifyPropertyChanged interface should be added on all data contract types to enable data binding. The default value is FALSE.

ForceUpdate

Requires that a direct connection to the service be available at design time, and forces the refresh of the local schema file. The default value is TRUE.

LocalSchemaFile

The .wsdl schema file to store locally cached service schema.

MessageContract

Indicates whether message contract types should be generated. The default value is FALSE.

ResolutionFolder

The folder used to resolve relative file paths at compile time. The default value is the folder that contains the project or script.

Serializable

Indicates whether the generated classes should be marked with the Serializable attribute. The default value is FALSE.

ServiceUri

The URI for the WSDL service. This is a mandatory parameter.

OData Type Provider

Another way to access remote services or data is to use the Open Data Protocol (OData) type provider. OData is an open web protocol for querying and updating data over the HTTP transport protocol. You can use it to retrieve results in formats like Atom, JSON, or plain XML. Figure 4-9 shows how to use Visual Studio to create an OData type provider, and the generated sample code is shown in Example 4-12. The OData type provider does not support fixed queries. You have to download the fixed query schema file to work with the OData type provider.

Using the Add New Item dialog box to add an OData type provider
Figure 4-9. Using the Add New Item dialog box to add an OData type provider
Example 4-12. OData type provider
 open System.Data.Services.Client
open Microsoft.FSharp.Data.TypeProviders

// Access demographic info such as population and income from the OData service at
Azure Marketplace. For more information, go to https://datamarket.azure.com/dataset/
c7154924-7cab-47ac-97fb-7671376ff656

type Demographics = Microsoft.FSharp.Data.TypeProviders.ODataService<ServiceUri =
    "https://api.datamarket.azure.com/Esri/KeyUSDemographicsTrial/">
let ctx = Demographics.GetDataContext()

//To sign up for an Azure Marketplace account at https://datamarket.azure.com/
account/infoctx.Credentials <- System.Net.NetworkCredential("<your liveID>",
"<your Azure Marketplace Key>")

Note

You need a Windows Live ID and an Windows Azure Marketplace key to show IntelliSense and compile the code from the previous example.

OData Type Provider Parameters

Compared to the WSDL type provider, the OData type provider’s parameter list (shown in Table 4-4) is much shorter. Like the WSDL type provider, only ServiceUri is a mandatory parameter; the other parameters are optional.

Table 4-4. OData type provider parameters

Parameter Name

Description

DataServiceCollection

Indicates whether or not to generate collections derived from DataServiceCollection. The default value is FALSE.

ForceUpdate

Requires that a direct connection to the service be available at design time, and forces the refresh of the local schema file. The default value is TRUE.

LocalSchemaFile

The local .csdl file for the service schema.

ResolutionFolder

The folder used to resolve relative file paths at compile time. The default value is the folder that contains the project or script.

ServiceUri

The URI for the OData service. This parameter is mandatory.

Other Type Providers

As I mentioned at the beginning of this chapter, the type provider is extensible. Because the type provider concept is public, many type providers have been developed by the community. Many of these type providers are provided by the FSharpX project on GitHub. I’m not going to present a detailed introduction to these type providers, but you can find out more about them at https://github.com/fsharp/fsharpx. This website provides information on type providers such as these:

  • File System. Used to provide type information taken from a file system. The type provider provides types for directories and files on the computer.

  • Xml File. Used to provide type information for an XML file. The XML type provider can infer a schema from the loaded data and generate types with properties.

  • JSON. Used to provide type information for JSON data. The JSON type provider provides strongly typed access to data provided in the JavaScript Object Notation (JSON) data-interchange format.

  • Registry. Used for the registry. The registry key and value are generated from the type provider.

  • Xaml. Provides type information for a WPF XAML file, which is essentially in XML format. The generated types are WPF UI elements.

The other way to get these type providers is to use the NuGet packages. Figure 4-10 shows how to access the management options for NuGet packages from a project reference. The FSharpX type provider for NuGet packages can be located by typing TypeProviders in the Search box in the upper-right corner of the Manage NuGet Packages dialog box, as shown in Figure 4-11.

Selecting the NuGet packages management options from a reference listing
Figure 4-10. Selecting the NuGet packages management options from a reference listing
The Manage NuGet Packages dialog box, which you use to configure settings for the NuGet Packages type provider
Figure 4-11. The Manage NuGet Packages dialog box, which you use to configure settings for the NuGet Packages type provider

The type provider provides a way for F# programmers to interpret the data from the outside world. You are not given an XmlElement when parsing an XML file; instead, you are given a more meaningful data type, such as StudentInfo. You’ll have an opportunity to write your own custom type provider in Chapter 5.

Query

When a type provider brings back a large amount of data, you need to have a sufficient way to process that data. Luckily, F# offers query to help. The query syntax is similar to that of an SQL query or LINQ query. I use the SQL data or entity provider in the sample, but you can use the F# query syntax to interact with any other type providers as well.

To see the query syntax work with one of the previously shown SQL type providers, you need to first set up an SQL database. The database is a basic student-course database with Student, Course, and CourseSelection tables. The SQL script in Example 4-13 is used to create the student-course database and some sample data.

The query is independent of a specific type provider, which means it can be applied to any type provider and even to a memory data structure. In the sample code, one person’s age is set to NULL. This is not a bug. This NULL value will demonstrate how to use F# nullable operators.

Example 4-13. SQL script for creating database tables and inserting data
CREATE TABLE Student(
    StudentID int not null
        constraint PrimaryKey primary key,
    Name nvarchar(50) null,
    Age int null,);

CREATE TABLE Course(
    CourseID int not null
        constraint PrimaryKey2 primary key,
    CourseName varchar(max) null);

CREATE TABLE AnotherStudentTable (
    StudentID int not null
        constraint PrimaryKey3 primary key,
    Name nvarchar(50) null,
    Age int null,);

go

CREATE TABLE CourseSelection(
    StudentID int not null
        constraint foreignKey references dbo.Student(StudentID),
    CourseID int null
        constraint foreignKey2 references dbo.Course(CourseID),
    ID int null);

go

INSERT Student VALUES (1, 'Lisa', 21)
INSERT Student VALUES (2, 'Brent', 22)
INSERT Student VALUES (3, 'Anita', 20)
INSERT Student VALUES (4, 'Ken', 22)
INSERT Student VALUES (5, 'Cathy', 22)
INSERT Student VALUES (6, 'Tom', 20)
INSERT Student VALUES (7, 'Zeoy', 21)
INSERT Student VALUES (8, 'Mark', 23)
INSERT Student VALUES (9, 'John', null)
go

INSERT Course VALUES (1, 'Math')
INSERT Course VALUES (2, 'Physics')
INSERT Course VALUES (3, 'Biology')
INSERT Course VALUES (4, 'English')
go

INSERT CourseSelection VALUES(1, 1, null)
INSERT CourseSelection VALUES(2, 1, null)
INSERT CourseSelection VALUES(3, 1, null)
INSERT CourseSelection VALUES(2, 2, null)
INSERT CourseSelection VALUES(2, 3, null)
INSERT CourseSelection VALUES(3, 3, null)
INSERT CourseSelection VALUES(3, 2, null)
go

I’ll use the SQL Entity type provider in the sample code. All the sample code is written on the assumption you already created a SQL Entity type provider and assigned it to a value named db. Because some query operations are not supported in the SQL Entity type provider, I’ll use AsEnumerable or array to create an in-memory data structure. The code is shown in Example 4-14.

Example 4-14. Creating an in-memory student table
open System
open System.Linq
open Microsoft.FSharp.Linq.NullableOperators

// define a student class
type Student() =
    member val ID = 0 with get, set
    member val Name = "" with get, set
    member val Age = Nullable(0) with get, set

// define the general show result function
let showResult query = query |> Seq.iter (printfn "%A")

// set the in-memory student table
let studentTable = [
        Student(ID = 1, Name = "Anita", Age = Nullable(22));
        Student(ID = 1, Name = "Kate", Age = Nullable(22))
        Student(ID = 1, Name = "Brent", Age = Nullable(23))
        Student(ID = 1, Name = "Lisa", Age = Nullable(23))
        Student(ID = 1, Name = "Mark", Age = Nullable(24)) ].AsQueryable()

Using the select Operator

The code in Example 4-15 is a simple select example. You can think of the select operator as being like the yield keyword in sequence generation or the select extension method in LINQ. It tells the compiler to return the value. The for...in...do code iterates through the data, and the select operator specifies what needs to be inserted into the final result. I’ll use the code let showResult query = query |> Seq.iter (printfn "%A") to show the query result.

Example 4-15. Simple select example

Code for the select example

// simple select
let internal q0 = query {
    for n in db.Course do
    select n.CourseName }

// general function to show the query result
let showResult query = query |> Seq.iter (printfn "%A")
showResult q0

Execution result

"Math"
"Physics"
"Biology"
"English"

If the query can be translated to the server-side code, it will be executed on the server side. Unfortunately, not all computations can be translated to the server-side code. Example 4-16 uses AsEnumerable() to specify that the computation should be performed on the client side, because sprintf cannot be translated to server-side code. Once the data is local, you are free to use any .NET function or pipeline operator.

Example 4-16. Selection code with transform code

Code for select with transform

let internal q1 = query {
    for n in db.Course.AsEnumerable() do
    select (sprintf "course name = %s" n.CourseName) }

showResult q1

Execution result

"course name = Math"
"course name = Physics"
"course name = Biology"
"course name = English"

You can also create more complex queries that involve multiple select statements. Example 4-17 shows how to print out all the value pairs where the first value is less than the second value. For this example, a value pair is formed from two data sources.

Example 4-17. Selecting from different sources
let A = [0; 2; 4; 5; 6; 8; 9]
let B = [1; 3; 5; 7; 8]

query {
        for a in A do
        for b in B do
        where (a < b)
        select (a, b)
}
|> Seq.iter(fun (n, e) -> printfn "%d is less than %d" n e)

Execution result

0 is less than 1
0 is less than 3
0 is less than 5
0 is less than 7
0 is less than 8
2 is less than 3
2 is less than 5
2 is less than 7
2 is less than 8
4 is less than 5
4 is less than 7
4 is less than 8
5 is less than 7
5 is less than 8
6 is less than 7
6 is less than 8

Using the where Operator

The where operator is used to filter data according to given conditions. Like the select operator, the computation can be executed either on the server side or client side. Example 4-18 shows how to filter the student names whose length is greater than 4 characters long. Because the result is filtered on the server side, only three items are transferred over the wire.

Example 4-18. A where example

Code for the where example

let internal q2 = query {
    for n in db.Student do
    where (n.Name.Length > 4)
    select n.Name }

showResult q2

Execution result

"Brent"
"Anita"
"Cathy"

Although performing the filter action on the client side is not a recommended approach, especially if the resulting data set is large, I still list the sample code here in case complex filtering is needed on the client side. See Example 4-19.

Example 4-19. The where filter function performed on the client side

The where filter function performed on the client side

let internal q3 = query {
    for n in db.Student do
    select n.Name }

// filter on client side
let q3_2 = q3 |> Seq.filter (fun n -> n.Length > 4)

showResult q3_2

Execution result

"Brent"
"Anita"
"Cathy"

Example 4-20 is another example of filtering data on the client side. The code gives each element an ID value and filters the results later according to this client-side-generated ID.

Example 4-20. Filter based on a client-side-generated value
let internal myQuery = query {
    for n in db.Student do
    select n.Name }

// filter on client side based on ID
let result =
    myQuery
    |> Seq.mapi (fun i n -> (i, n))
    |> Seq.filter (fun (i, n) -> i % 2 = 0)
    |> showResult

Execution result

(0, "Lisa")
(2, "Anita")
(4, "Cathy")
(6, "Zeoy")
(8, "John")

Note

The actual result you get might be different, depending on the targeted data storage.

Using the join Operator

The join example in Example 4-21 shows how to join data that is scattered across different tables. In the join statement, the order of the keys is significant. The outside table db.Student, which is the main table, must be present first in the on condition part. The example lists the course ID and student name as a result.

Example 4-21. A join example

Code for the join example

let internal q5 = query {
    for student in db.Student do
    join course in db.CourseSelection on (student.StudentID = course.StudentID)
    select (course.CourseID, student.Name) }

showResult q5

Execution result

(3, "Anita")
(1, "Lisa")
(2, "Anita")
(2, "Brent")
(3, "Brent")
(1, "Anita")
(1, "Brent")

The CourseID in the result from Example 4-21 is far from user friendly. If the student name can be shown, that will be great. In Example 4-22, a groupJoin operator is used to show a more user-friendly result. Because CourseID is a nullable integer type, you can use “?=” in the Microsoft.FSharp.Linq.NullableOperators module to perform the comparison.

Example 4-22. Using a group join
let q6  = query {
    for student in db.Student do
    groupJoin courseSelection in db.CourseSelection
        on (student.StudentID = courseSelection.StudentID) into g
    for courseSelection in g do
    join course in db.Course on (courseSelection.CourseID ?= course.CourseID)
    select (student.Name, course.CourseName) }

showResult q6

Execution result

("Anita", "Biology")
("Lisa", "Math")
("Anita", "Physics")
("Brent", "Physics")
("Brent", "Biology")
("Anita", "Math")
("Brent", "Math")

There is another join scenario called an outer join. The F# outer join operator is similar to SQL’s outer join, but it supports only the left outer join. The right outer join functionality can be achieved by switching the two tables. Example 4-23 shows how to use leftOuterJoin to show the course selection information.

Example 4-23. Using the left outer join
let internal q5 = query {
    for student in db.Student do
    leftOuterJoin selection in db.CourseSelection on
                   (student.StudentID =? selection.CourseID) into result
    for selection in result do
    select (student.Name, selection.Course.CourseName)
    }

showResult q5

Execution result

("Lisa", "Math")
("Lisa", "Math")
("Lisa", "Math")
("Brent", "Physics")
("Brent", "Physics")
("Anita", "Biology")
("Anita", "Biology")
("Ken", null)
("Cathy", null)
("Tom", null)
("Zeoy", null)
("Mark", null)
("John", null)

Note

Because CourseID is a Nullable<int>, you need to use =? to perform the equality operation. You need to use open Microsoft.FSharp.Linq.NullableOperators. Both =? and ?= are valid operators. If the nullable value is on left side of the operator, ?= is applied; however, =? is used if the nullable value is on right side.

Using the Value property in a Nullable<T> type does not work in the query. This is by design, because using it in this way can lead to a NULL exception when the Nullable<T> value is NULL.

Using the sortBy Operator

Example 4-24 shows how to use the sortBy operator to show the sorted names whose name length is greater than 4 characters. By default, sortBy sorts the result in ascending order. The sortByDescending operator serves the purpose of sorting the results in a descending manner.

Example 4-24. Using the sort code

Sort code

let internal q4 = query {
    for n in db.Student do
    sortBy n.Name
    where (n.Name.Length > 4)
    select n.Name }

showResult q4

Execution result

"Anita"
"Brent"
"Cathy"

You can sort by a second criterion by using thenBy or thenByDescending. Example 4-25 sorts the result first by a student’s name and then by the student’s age.

Example 4-25. Using the thenBy keyword
let internal q4_2 = query {
    for student in db.Student do
    sortBy student.Age.Value
    thenBy student.Name
    select (student.Name, student.Age) }

showResult q4_2

Execution result

("John", null)
("Anita", 20)
("Tom", 20)
("Lisa", 21)
("Zeoy", 21)
("Brent", 22)
("Cathy", 22)
("Ken", 22)
("Mark", 23)

In Example 4-25, Age is a nullable value, so you have to reference the Value property in order to sort. The F# query feature provides a set of operators that can handle nullable values. Example 4-26 shows how to use sortByNullableDescending.

  • sortByNullable. Sorts elements in ascending order by a given nullable key

  • sortByNullableDescending. Sorts elements in descending order by a given nullable key

  • thenByNullable. Like the thenBy operator, performs subsequent ordering of the elements in ascending order by a given nullable key

  • thenByNullableDescending. Like the thenByDescending operator, performs subsequent ordering of the elements in descending order by a given nullable key

Example 4-26. Using sortByNullableDescending
let internal q4_3 = query {
    for student in db.Student do
    sortByNullableDescending student.Age
    thenBy student.Name
    select (student.Name, student.Age) }

showResult q4_3

Execution result

("Mark", 23)
("Brent", 22)
("Cathy", 22)
("Ken", 22)
("Lisa", 21)
("Zeoy", 21)
("Anita", 20)
("Tom", 20)
("John", null)

Using the group Operator

The group operator is often used with an aggregate function. Example 4-27 counts the number of students in each age group.

Example 4-27. A groupBy example

The code for a groupBy query

let internal q7 = query {
    for student in db.Student do
    groupBy student.Age into g
    select (g.Key, g.Count()) }

showResult q7

Execution result

(null, 1)
(20, 2)
(21, 2)
(22, 3)
(23, 1)

Again, the result is not user friendly because the result does not show the student names in the result. To show the student names, you can use groupValBy to select a value with the group. The full code is shown in Example 4-28. It shows a student name sequence in each age group.

Example 4-28. A groupValBy example

Code for using groupValBy

let internal q8 = query {
    for student in db.Student do
    groupValBy student.Name student.Age into g
    select (g, g.Key, g.Count()) }

showResult q8

Execution result

(seq ["John"], null, 1)
(seq ["Anita"; "Tom"], 20, 2)
(seq ["Lisa"; "Zeoy"], 21, 2)
(seq ["Brent"; "Ken"; "Cathy"], 22, 3)
(seq ["Mark"], 23, 1)

Using the take and skip Operators

The take and skip operators are used when you are interested only in a certain range of the query results. Example 4-29 sorts the students by age and skips the first student. This can be very useful in an OData service, which often returns the results in pages.

Example 4-29. Using skip to ignore the first element

The code for the skip example

let internal q9 = query {
    for student in db.Student do
    sortByNullable student.Age
    skip 1
    select student.Name }

showResult q9

Execution result

"Tom"
"Anita"
"Lisa"
"Zeoy"
"Brent"
"Ken"
"Cathy"
"Mark"

The skip and take operators are similar when used in the query. The take example is shown in Example 4-30. The sample code sorts the students by age and returns only the first three student names.

Example 4-30. An example that uses take on the first three elements

The code for the take example

let internal q9 = query {
    for student in db.Student do
    sortByNullable student.Age
    take 3
    select student.Name }

showResult q9

Execution result

"John"
"Anita"
"Tom"

The skip operation also can be used with a condition statement rather than just a given number. Example 4-31 shows how to skip the results for people in the database who have an age value less than 23.

Example 4-31. A skipWhile example

The code for the skipWhile example

open System
open System.Linq
open Microsoft.FSharp.Linq.NullableOperators

type Student() =
    member val ID = 0 with get, set
    member val Name = "" with get, set
    member val Age = Nullable(0) with get, set

let showResult query = query |> Seq.iter (printfn "%A")

let studentTable = [
        Student(ID = 1, Name = "Anita", Age = Nullable(22));
        Student(ID = 1, Name = "Kate", Age = Nullable(22))
        Student(ID = 1, Name = "Brent", Age = Nullable(23))
        Student(ID = 1, Name = "Lisa", Age = Nullable(23))
        Student(ID = 1, Name = "Mark", Age = Nullable(24)) ].AsQueryable()

// skip student whose age is less than 23
let internal q10 = query {
    for student in studentTable do
    sortByNullable student.Age
    skipWhile (student.Age ?< 23)
    select student.Name }

showResult q10

Execution result

"Brent"
"Lisa"
"Mark"

Note

Because one student’s age is NULL, you have to use the ?< operator in the Microsoft.FSharp.Linq.NullableOperators module to compare the nullable value.

Because the underlying SQL data source does not support a skipWhile operator, we use the in-memory structure that was shown in Example 4-14.

Using the min/max, average, and sum Operators

There is not much to explain about these operations; their names describe their functionalities. Because you have to handle the nullable value, each operator has a twin to handle the nullable value. I present them using a nullable operator in this section. Example 4-32 shows how to get the maximum, minimum, and average ages of the student group, as well as the sum of their ages.

Example 4-32. Finding the minimum, maximum, average, and sum of the students’ ages
let internal q11 = query {
    for student in db.Student do
    sumByNullable student.Age }

printf "sum of student age = "
showResult [ q11 ]

let internal q12 = query {
    for student in db.Student do
    minByNullable student.Age }

printf "youngest student age = "
showResult [ q12 ]

let internal q13 = query {
    for student in db.Student do
    maxByNullable student.Age }

printf "oldest student age = "
showResult [ q13 ]

let internal q14 = query {
    for student in db.Student do
    averageByNullable (Microsoft.FSharp.Linq.Nullable.float student.Age) }

printf "average student age = "
showResult [ q14 ]

Execution result

sum of student age = 171
youngest student age = 20
oldest student age = 23
average student age = 21.375

Note

The average operation cannot be performed on an integer type. Instead, you need to use the Nullable.float type that is located in the Microsoft.FSharp.Linq module, which converts the Nullable<int> to Nullable<float>.

Using the head, last, and nth Operators

Sometimes you need only one element in the sequence. The head, last, and nth operators are designed for this purpose. You use the head operator on the SQL storage and the other operators with the in-memory structure. Example 4-33 shows the first student’s name. Example 4-34 shows how to get the last and nth elements.

Example 4-33. A head example
// return first element
let internal q15 = query {
    for student in db.Student do
    select student.Name
    head }

showResult [ q15 ]

// get the first element; if there is no first element returned, returns default
value let internal q16 = query {
    for student in db.Student do
    select student.Name
    headOrDefault }

showResult [ q16 ]
Example 4-34. Example that uses last and nth
open System
open System.Linq
open Microsoft.FSharp.Linq.NullableOperators

type Student() =
    member val ID = 0 with get, set
    member val Name = "" with get, set
    member val Age = Nullable(0) with get, set

let showResult query = query |> Seq.iter (printfn "%A")

let studentTable = [
        Student(ID = 1, Name = "Anita", Age = Nullable(22));
        Student(ID = 1, Name = "Kate", Age = Nullable(22))
        Student(ID = 1, Name = "Brent", Age = Nullable(23))
        Student(ID = 1, Name = "Lisa", Age = Nullable(23))
        Student(ID = 1, Name = "Mark", Age = Nullable(24)) ].AsQueryable()

// take the last student
let internal q15 = query {
    for student in studentTable do
    select student.Name
    last }

showResult [ q15 ]

// take the last student; if there is no last student, use default value
let internal q16 = query {
    for student in studentTable do
    select student.Name
    lastOrDefault }

showResult [ q16 ]

// return the 4th student in the list
let internal q17 = query {
    for student in studentTable do
    select student.Name
    nth 3 }

showResult [ q17 ]

Execution result

"Mark"
"Mark"
"Lisa"

Using the count and distinct Operators

The count operator is used to count the number of elements. Example 4-35 shows how many students are in the Student table.

Example 4-35. A count example
let internal q17 = query {
    for student in db.Student do
    count }

showResult [ q17 ]

You might have noticed in the results from Example 4-21 that some of the student names were repeated. The distinct operator is used to get the distinct elements in the result set, as shown in Example 4-36.

Example 4-36. A distinct example
let internal q18 = query {
    for student in db.Student do
    join selection in db.CourseSelection on
        (student.StudentID = selection.StudentID)
    select student.Name
    distinct }

showResult q18

Execution result

"Anita"
"Brent"
"Lisa"

Using the contains, exists, and find Operators

The contains and exists operators are used to determine whether the set contains elements that meet a given condition. Example 4-37 checks whether there is a student named Lisa.

Example 4-37. An example that uses contains and exists
let internal q19 = query {
    for student in db.Student do
    select student.Name
    contains "Lisa" }

showResult [ q19 ]

let internal q20 = query {
    for student in db.Student do
    exists (student.Name = "Lisa") }

showResult [ q20 ]

The contains and exists operators return only a Boolean value. If you need to see whether the first element meets certain criteria, you can use find as shown in Example 4-38. If there is no student who meets the criteria, a System.InvalidOperationException will be thrown.

Example 4-38. A find example
let internal q21 = query {
    for student in db.Student do
    find (student.Age ?> 20) }

showResult [ q21.Name ]

Execution result

"Lisa"

Using the exactlyOne and all Operators

The all operator is used to test whether all the elements meet the criterion. Example 4-39 is used to check whether all the students’ names start with the letter L.

Example 4-39. An all example
let internal q22 = query {
    for student in db.Student do
    all (SqlClient.SqlMethods.Like(student.Name, "L%")) }

showResult [ q22 ]

Note

The sample does not work with SqlEntityConnection but will work with SqlDataConnection.

To check whether a class has only one student with the name Lisa and return her information, you can use exactlyOne or exactlyOneOrDefault. The exactlyOne operation yields a System.InvalidOperationException if there is not exactly one match. See Example 4-40.

Example 4-40. An exactlyOne example
let internal q23 = query {
    for student in db.Student do
    where (student.Name = "Lisa")
    exactlyOne }

showResult [ q23.Name ]

SQL Query and F# Query

The previous section provided a number of query operators. For some readers, the list might be too long, while others might still be waiting for more. If you are looking for more, you might be trying to map your existing T-SQL knowledge to F# query. If so, this section is the right place to be. I won’t repeat my explanations of select or where here because they are easy to map to your existing T-SQL knowledge. If you are familiar with T-SQL, the following list can help you easily transfer your T-SQL knowledge to F# query:

  • The in keyword in T-SQL is used to check whether a value is in the specified list of values, as shown in Example 4-41. The equivalent of this T-SQL code in F# query is shown in Example 4-42.

    Example 4-41. SQL in example
    SELECT * FROM Student
    WHERE Student.StudentID IN (1, 2, 3)
    Example 4-42. F# syntax for in
    let ids = query { for id in [1; 2; 3] do select id }
    
    query {
        for student in db.Student do
        where (ids.Contains(student.StudentID))
        select student }
  • The top T-SQL keyword is used to return a number of elements on the top of the return list. At first glimpse, it might appear that F# query does not have an equivalent operator. However, closer examination will reveal that the take operator can do exactly the same thing.

  • The like T-SQL keyword is a character-mapping operator in SQL. Although like is not a keyword in F# query, you can use SqlMethods.Like, as shown in Example 4-39. SQLMethods also contains other methods that can be used to handle date and time differences. See Table 4-5 for more details.

    Table 4-5. The SQLMethods member list

    Function Name

    Corresponding SQL Operator

    Description

    DateDiffDay

    DATEDIFF

    Counts the date difference in days

    DateDiffHour

    DATEDIFF

    Counts the date difference in hours

    DateDiffMicrosecond

    DATEDIFF

    Counts the date difference in microseconds

    DateDiffMillisecond

    DATEDIFF

    Counts the date difference in milliseconds.

    DateDiffMinute

    DATEDIFF

    Counts the date difference in minutes

    DateDiffMonth

    DATEDIFF

    Counts the date difference in months

    DateDiffNanosecond

    DATEDIFF

    Counts the date difference in nanoseconds

    DateDiffSecond

    DATEDIFF

    Counts the date difference in seconds

    DateDiffYear

    DATEDIFF

    Counts the date difference in years

    Like

    LIKE

    Decides whether the input matches the specific pattern

  • The between T-SQL keyword is not a keyword in F# query, but you can use the comparison operators combined with && to achieve the same result.

  • The functionality provided by the union and intersection T-SQL keywords can be achieved with F# query, as shown in Example 4-43 and Example 4-44.

    Example 4-43. SQL union and intersection
    SELECT * from Student
    UNION
    SELECT * from AnotherStudentTable
    
    SELECT * from Student
    INTERSECT
    SELECT * from AnotherStudentTable
    Example 4-44. F# query union
    let q0 = query { for student in db.Student do select student.Name }
    let q1 = query { for student in db.AnotherStudentTable do select student.Name }
    
    // union two query results
    q1.Union(q0)
    
    // intersect two query results
    q1.Intersect(q0)
  • The case T-SQL keyword is supported in F# query by using an if statement. See Example 4-45 and Example 4-46.

    Example 4-45. SQL case example
    SELECT student.Name,
        CASE student.Age
        WHEN 0 then 1000
        ELSE student.Age
        END
    FROM Student
    Example 4-46. F# query using an if statement
    let q = query {
        for student in db.Student do
        select ( if student.Age ?= -1 then (student.Name, Nullable<int>(1000))
                     else (student.Name, student.Age)  )  }

Other F# Operators

In this section, I will present some F# operators that can be used as shortcut to make the query code more readable. As mentioned in the previous section, you will inevitably find scenarios that require bringing data to the client to perform certain computations. F# provides a few shortcuts for some of the more widely used .NET functions associated with these scenarios. These functions can be used in queries and in other data processing:

  • |||>, ||>, <|||, and <|| operators. These operators are used to break triple and tuple to a function. After reviewing the sample in Example 4-47, it is not difficult to find the similarity between these operators and the pipeline operator. For example, |||> takes a triple, breaks the triple into three elements, and feeds the elements into a function that takes three parameters in curried format.

    Example 4-47. The |||>, ||>, <|||, and <|| operators
    // define a function taking 3 parameters in curried format
    let f3 a b c = a + b + c + 1
    
    // define a function taking 2 parameters in curried format
    let f2 a b = a + b + 1
    
    // define a function taking a parameter
    let f a = a + 1
    
    let print x = printfn "value = %A" x
    
    print ( (1, 2, 3) |||> f3 )
    print ( (1, 2) ||> f2 )
    print ( 1 |> f )
    
    print ( f3 <||| (1, 2, 3) )
    print ( f2 <|| (1, 2) )
    print ( f <| 1 )

    Execution result

    value = 7
    value = 4
    value = 2
    value = 7
    value = 4
    value = 2
  • sin, cos, tan, sinh, cosh, and tanh. The first three functions—sin, cos, and tan—are the trigonometric functions. sine, cosine, and tangent. The sinh, cosh, and tanh functions are the hyperbolic functions hyperbolic sine, hyperbolic cosine, and hyperbolic tangent.

  • log and log10. The log function is the natural logarithm function, while the log10 function is a logarithm function with base equal to 10.

  • min and max. These function names are used to get the minimum or maximum value of two given values.

  • floor, ceil, and round. The floor and ceil functions are designed to find the floor and ceil of a given number, while the round function is used to find the rounded value of a given number.

  • abs. The abs function is used to find the absolute value of a given number.

  • asin, acos, and atan. The asin, acos, and atan functions are inverse sine, inverse cosine, and inverse tangent functions.

Using a Type Provider to Connect to the Windows Azure Marketplace

In this chapter, you used type providers to query a database and an in-memory structure. Another source of data is the Windows Azure Marketplace (https://datamarket.azure.com/). Hundreds of data sources and APIs are available in the Marketplace, and the number is increasing every day. Many innovations and applications are enabled by using these data and web APIs. In this section, I’ll show how to use a type provider to access the free UK Foreign and Commonwealth Office Travel Advisory Service at the Windows Azure Marketplace. To use this free service, go to Windows Azure Marketplace and search for UK Foreign and Commonwealth Office Travel Advisory Service, the service is located at https://datamarket.azure.com/dataset/uk.gov/traveladvisoryservice. You can sign in with your Windows Live ID. Figure 4-12 shows the Details tab, which describes the service URL and service schema information.

As programmers, we are interested in the Details tab because it lists the connection string and all of the data schemas, including the data names and types. We will need the service root URI for this example.

Data source detailed information found on the Details tab
Figure 4-12. Data source detailed information found on the Details tab

Setting Up the Azure Account

To access the data in the Windows Azure Marketplace, you need an account. Account registration is free (again, at https://datamarket.azure.com/). (To register for an account, you first need a Windows Live ID.) Figure 4-13 shows the My Account page; take note of the Customer ID and Primary Account Key on your My Account page because you will need these in the future.

My Account page in the Windows Azure Marketplace
Figure 4-13. My Account page in the Windows Azure Marketplace

Connecting To and Consuming Data

You can use OData to connect to the data source. The OData type provider needs three parameters:

The query code and partial execution result are shown in Example 4-48.

Example 4-48. Connecting to the Windows Azure Marketplace by using OData
// define the type provider
type Demographics = Microsoft.FSharp.Data.TypeProviders.ODataService<ServiceUri =
    "https://api.datamarket.azure.com/Uk.Gov/TravelAdvisoryService/">

let ctx = Demographics.GetDataContext()

// set the credentials
ctx.Credentials <- System.Net.NetworkCredential(<liveID>, <id>)

// query the data source
let q = query {
    for n in ctx.BritishEmbassiesAndHighCommissions do
    select (n.Title, n.OfficeHours) }

// print out the result
q
|> Seq.iter (fun (address, _)-> printfn "%s" address)

Partial execution result

Australia, Hobart, British Honorary Consul
Australia, Adelaide, British Honorary Consul
Afghanistan, Kabul, British Embassy
Vietnam, Hanoi, British Council
Taiwan, Kaohsiung, British Council
Japan, Tokyo, British Council
Burma, Rangoon (Yangon), British Embassy
Australia, Canberra, Consular Section
New Zealand, Auckland, British Consulate-General
China, Guangzhou, British Consulate-General
Australia, Melbourne, British Consulate-General
Japan, Osaka, British Consulate-General
China, Shanghai, British Consulate-General
Australia, Brisbane, British Consulate
Australia, Perth, British Consulate
...

Good! The connection is working well, and the first query returns data. Now, let’s find out about all the UK embassies or consulates in China. You might be tempted to try to achieve this goal by placing the where operator in the query. Unfortunately, the where keyword is not supported when executed on the server side. Instead, the filter operation has to be done on the client. See Example 4-49.

Example 4-49. Finding all UK embassies or consulates in China
// define the type provider
type Demographics = Microsoft.FSharp.Data.TypeProviders.ODataService<ServiceUri =
    "https://api.datamarket.azure.com/Uk.Gov/TravelAdvisoryService/">

let ctx = Demographics.GetDataContext()

// set the credentials
ctx.Credentials <- System.Net.NetworkCredential(<liveID>, <id>)

// query the data source
let q = query {
    for n in ctx.BritishEmbassiesAndHighCommissions do
    //where (n.Title.Contains("United States"))  // where is not supported on server side
    select (n.Title, n.OfficeHours) }

// print out the result
q
|> Seq.filter (fun (title, _) -> title.Contains("China"))
|> Seq.iter (fun (title, _)-> printfn "%s" title)

Execution result

China, Guangzhou, British Consulate-General
China, Shanghai, British Consulate-General
China, Chongqing, British Consulate-General
China, Beijing, British Embassy
China, Beijing, British Embassy, Cultural &amp; Education Section

You get a surprise when you use the same code to query for UK embassies in the United States. The query does not return any results. It is hard to believe that the UK does not have at least one embassy in the country. The reason is that this query returns only part of all the results. For performance reasons, it would not be a wise decision to return all the results at once. The query returns only one page of data for each query execution. The results shown in Example 4-49 include only those from the first page of the total results. This is a good time to use the skip operator to ignore the previously retrieved data and return the next page of the query results, as shown in Example 4-50.

Example 4-50. Finding all UK embassies or consulates in the United States by using skip
// define the type provider
type Demographics = Microsoft.FSharp.Data.TypeProviders.ODataService<ServiceUri =
    "https://api.datamarket.azure.com/Uk.Gov/TravelAdvisoryService/">

let ctx = Demographics.GetDataContext()

// set the credentials
ctx.Credentials <- System.Net.NetworkCredential(<liveID>, <id>)

// query pages
let rec queryFunction (startPoint) =
    seq {
        // query the data source
        let q = query {
            for n in ctx.BritishEmbassiesAndHighCommissions do
            skip startPoint
            select (n.Title, n.OfficeHours) }
        let count = Seq.length q
        if count <> 0 then
            for item in q do
                yield item
            for item in queryFunction(startPoint + count) do
                yield item
    }

// print out the result
queryFunction(0)
|> Seq.filter (fun (title, _) -> title.Contains("United States"))
|> Seq.iter (fun (title, _)-> printfn "%s" title)

Execution result

United States, Denver, British Consulate-General
United States, Atlanta, British Consulate-General
United States, Boston, British Consulate-General
United States, Chicago, British Consulate-General
United States, Houston, British Consulate-General
United States, Los Angeles, British Consulate-General
United States, New York, British Consulate-General
United States, San Francisco, British Consulate-General
United States, Miami, British Consulate-General
United States, Orlando, British Consulate
United States, Washington, British Embassy
United States, Anchorage, British Honorary Consulate
United States, Nashville, British Honorary Consulate
United States, New Orleans, British Honorary Consulate
United States, Philadelphia, British Honorary Consulate
United States, Phoenix, British Honorary Consulate
United States, Pittsburgh, British Honorary Consulate
United States, Salt Lake City, British Honorary Consulate
United States, Charlotte, British Honorary Consulate
United States, Michigan, British Honorary Consulate
United States, Minnesota, British Honorary Consulate
United States, Ohio, British Honorary Consulate

Performing Translations with Microsoft Translator

As mentioned in the previous section, the OData type provider does not work with “fixed query” datasets. Although the number of datasets that support only fixed queries is insignificant, it’s good to know how to interact with them when needed. In this section, I’ll use the Microsoft Translator, which supports only a fixed-query dataset, to demonstrate how to use F# to work with fixed-query datasets.

Any dataset that supports a fixed query provides a pre-created proxy class for the dataset. The proxy class file is available on the dataset Details page after you make a purchase from Azure Marketplace. In Figure 4-14, the C# proxy class file is shown in the upper-right portion of the page. You need to download this file and make a C# class library for the main F# project to reference. I am not going to go through how to create and reference a C# project. The F# code in Example 4-51 shows how to invoke the proxy class.

Proxy class file for fixed-query dataset
Figure 4-14. Proxy class file for fixed-query dataset
Example 4-51. Invoking the C# proxy class from F#
open System.Linq

// define a translate function by taking an English sentence and translating it to
Chinese let translate txt =
    let serviceRootUri =
        new System.Uri("https://api.datamarket.azure.com/Bing/MicrosoftTranslator/")

    // the account key is the same as in the previous section
    let accountKey = <id>

    // create a translator container that is defined in the proxy class
    let tc = Microsoft.TranslatorContainer(serviceRootUri)
    tc.Credentials <- new System.Net.NetworkCredential(accountKey, accountKey)

    //translate from English(en) to Chinese (zh-CHS)
    let translationQuery = tc.Translate(txt, "zh-CHS", "en")
    let translationResult = translationQuery.Execute().ToList().FirstOrDefault()

    // if there is no translation result, something went wrong so return empty string
    if (translationResult = null) then
        System.String.Empty
    else
        translationResult.Text

You can use this translation service to translate recent news from English to Chinese, as shown in Example 4-52.

Example 4-52. Translating the latest news from English to Chinese with Microsoft Translator
// define the type provider
type Demographics = Microsoft.FSharp.Data.TypeProviders.ODataService<ServiceUri =
    "https://api.datamarket.azure.com/Uk.Gov/TravelAdvisoryService/">
let ctx = Demographics.GetDataContext()

// set the credentials
ctx.Credentials <- System.Net.NetworkCredential(<liveID>, <id>)

//get latest news
let latestNews = query {
    for n in ctx.LatestTravelNewsFromFco do
    select (n.Title, n.Summary)
}

latestNews
|> Seq.map fst           // only leave title to translate
|> Seq.map translate   //translate from English to Chinese
|> Seq.iter (fun n -> printfn "%s" n)

Execution result

恶劣天气影响整个欧洲的旅行
下个星期为欧洲足球赛事旅游忠告
西班牙机场中断
象牙海岸的旅游建议的变化
玻利维亚旅游忠告-鲁雷纳瓦克在封锁
在欧洲旅行的中断
澳大利亚旅游忠告-东澳大利亚的水浸
美国旅游忠告-暴雪条件
旅游忠告-尤文图斯 v 曼城
希腊旅游忠告-工业行动
...

Note

You need to install Chinese locally on your computer to see Chinese characters in the console output.

The actual execution result is pretty long; the result shown is only part of the result set.

Storing Data Locally

You might be wondering how to serialize or deserialize the data from Windows Azure Marketplace. I cover two approaches in this section. One shows you how to do the serialization programmatically, and the other shows you how to save the content to a Microsoft Excel file.

Performing Serialization

The topic of .NET serialization could be expanded into a 50-page chapter, so I’ll leave that for you to explore on your own. In this section, I present the JSON and XML serializers. The first example in Example 4-53 shows how to use the JSON serializer. The example demonstrates how to serialize and deserialize a tuple instance.

Example 4-53. Use JSON serialization
open System.Runtime.Serialization.Json
open System.Runtime.Serialization

let serializeAndDeserialize (originalValue:'T) =

    // Create a serializer to serialize the data
    let serializer = DataContractJsonSerializer(typeof<'T>)
    use out = new System.IO.MemoryStream()
    printfn "input data is %A" originalValue
    serializer.WriteObject(out, originalValue)

    // Get the serialized text
    let text = out.ToArray() |> System.Text.Encoding.UTF8.GetString
    out.Seek(0L, System.IO.SeekOrigin.Begin) |> ignore
    printfn "serialized data as: %s" text

    let deserializedValue = serializer.ReadObject out
    printfn "deserialized the data as: %A" deserializedValue

serializeAndDeserialize (1,2)

Execution result

input data is (1, 2)
serialized data as: {"m_Item1":1,"m_Item2":2}
deserialized the data as: (1, 2)

If the XML serializer is your favorite, Example 4-54 shows you how to serialize the data to an XML file by using it. The sample demonstrates how to serialize and deserialize data from the UK government Travel Advisory Service to and from a local XML file.

Example 4-54. Serializing and deserializing data from Windows Azure Marketplace by using XML serialization
// declare a NewsRecord class
type NewsRecord() =
    member val Title = System.String.Empty with get, set
    member val Summary = System.String.Empty with get, set

// define the type provider
type Demographics = Microsoft.FSharp.Data.TypeProviders.ODataService<ServiceUri =
    "https://api.datamarket.azure.com/Uk.Gov/TravelAdvisoryService/">
let ctx = Demographics.GetDataContext()

// set the credentials
ctx.Credentials <- System.Net.NetworkCredential(<liveID>, <id>)

// query the latest news
let latestNews = query {
    for n in ctx.LatestTravelNewsFromFco do
    select (NewsRecord(Title = n.Title, Summary = n.Summary))
}

// query the latest news
let latestNews = query {
    for n in ctx.LatestTravelNewsFromFco do
    select { NewsRecord.Title = n.Title; NewsRecord.Summary = n.Summary}
}

let news = latestNews |> Seq.toArray

// deserialize from xml
let deserialize<'T> (fileName:string) =
    let reader = System.Xml.Serialization.XmlSerializer(typeof<'T>)
    use file = new System.IO.StreamReader(fileName)
    let fileData = reader.Deserialize(file) :?> 'T
    fileData

// serialize data
let serialize data (fileName:string)=
    let writer = System.Xml.Serialization.XmlSerializer(data.GetType());
    use file = new System.IO.StreamWriter(fileName)
    writer.Serialize(file, data);

//serialize the data
serialize news "myLocalNews.xml"

// deserialize the file
let data = deserialize<NewsRecord array> "myLocalNews.xml"

Serialized XML file

<?xml version="1.0" encoding="utf-8"?>
<ArrayOfNewsRecord xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <NewsRecord>
    <Title>Severe weather affecting travel across Europe</Title>
    <Summary>Adverse weather conditions are affecting travel across Europe, and
there are airport
 closures and flight cancellations in the UK.</Summary>
  </NewsRecord>
  <NewsRecord>
    <Title>Travel advice for European football matches next week</Title>
    <Summary>Chelsea, Rangers and Spurs are playing in Europe next week. Check our
travel advice
for fans if you're going to the matches.</Summary>
  </NewsRecord>
...

Note

Only part of the execution result is shown.

Exporting to Microsoft Excel

XML and JSON serialization are very useful to a programmer, but the result is definitely not for the average user. In Example 4-55, I’ll you show how to use F# to write the cloud data into an Excel file. The sample retrieves the data from Azure, opens the Excel file, and inserts the title from cells B1 to B100 and the summary from cells C1 to C100.

Example 4-55. Inserting the title and summary from Azure to Excel
open Microsoft.Office.Interop.Excel

// Run Excel
let app = new ApplicationClass(Visible = true)

// Create a new file, and get the 1st worksheet
let workbook = app.Workbooks.Add(XlWBATemplate.xlWBATWorksheet)

// Note that worksheets are started from 1 not 0
let worksheet = (workbook.Worksheets.[1] :?> Worksheet)

// define the type provider
type Demographics = Microsoft.FSharp.Data.TypeProviders.ODataService<ServiceUri =
    "https://api.datamarket.azure.com/Uk.Gov/TravelAdvisoryService/">
let ctx = Demographics.GetDataContext()

// set the credentials
ctx.Credentials <- System.Net.NetworkCredential(<liveID>, <id>)

// declare a NewsRecord class
type NewsRecord() =
    member val Title = System.String.Empty with get, set
    member val Summary = System.String.Empty with get, set

// query the latest news
let latestNews = query {
    for n in ctx.LatestTravelNewsFromFco do
    select (NewsRecord(Title = n.Title, Summary = n.Summary))
}

let news = latestNews |> Seq.toArray

// get the range tuple such as "A1", "A10"
let getRangeStrings columnName news =
    let start = sprintf "%c%d" columnName 1
    let ''end'' = sprintf "%c%d" columnName (news |> Seq.length)
    start, ''end''

let len = news.Length

// get range tuple, which contains start and end
let range0 = (getRangeStrings 'B' news)
let range1 = (getRangeStrings 'C' news)

// insert title to B column
(worksheet.Range (fst range0, snd range0)).Value2 <- Array2D.init len 1 (fun x _ ->
news.[x].Title)

// insert summary to C column
(worksheet.Range (fst range1, snd range1)).Value2 <- Array2D.init len 1 (fun x _ ->
news.[x].Summary)

Note

A reference to the Microsoft.Excel.Interop.dll is needed to invoke the Excel function.

Generating a Word Document

In addition to an Excel file, you can also export data to a Microsoft Word file by using the Office OpenXML SDK, which can be downloaded from http://www.microsoft.com/en-us/download/details.aspx?id=5124. The sample presented in this section is used to generate a notification letter for students who pass an exam. The pass/fail information is stored in the database. The type provider queries the database and filters out the students who have passed the exam. The students’ information is then passed to various OpenXML methods to generate the Word documents. Example 4-56 creates the table in a SQL Server database. The table has three columns:

  • Name. Student name, which is also the keyword

  • Course. Course name

  • Passed. Boolean value to indicate whether the student passed the exam

Example 4-56. SQL statement used to create the student course table and insert three test records
CREATE TABLE [dbo].[StudentScoreTable](
    [Name] [nvarchar](50) NOT NULL,
    [Course] [nchar](64) NULL,
    [Passed] [bit] NULL,
 CONSTRAINT [PK_StudentScoreTable] PRIMARY KEY CLUSTERED
(
    [Name] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF,
    IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]

Insert test record data

insert into StudentScoreTable(Name,Course, Passed)
Values('Brent', 'History', 1)

insert into StudentScoreTable(Name,Course, Passed)
Values('Chris', 'Math', 1)

insert into StudentScoreTable(Name,Course, Passed)
Values('Anita', 'History', 1)

insert into StudentScoreTable(Name,Course, Passed)
Values('Andy', 'Math', 0)

The code in Example 4-57 is F# code. It retrieves data from the database by using the SQL type provider. The generatePassLetter function is used to generate the Word document for students who pass the exam. The execution for this sample code generates three files, for the students named Anita, Chris, and Brent. Andy did not pass the exam, so there is no file for him.

Example 4-57. Generating Word documents from the type provider query results
open System
open System.Linq
open DocumentFormat.OpenXml
open DocumentFormat.OpenXml.Packaging
open DocumentFormat.OpenXml.Wordprocessing
open Microsoft.FSharp.Data.TypeProviders
open Microsoft.FSharp.Linq.NullableOperators

type T = SqlDataConnection<"<your connection to the database>">
let t = T.GetDataContext()

let passedCandidates = query {
    for student in t.StudentScoreTable do
    where (student.Passed ?= true)
    select (student.Name, student.Course) }

// generate pass letter using OpenXML SDK
let generatePassLetter record =
    let name, course = record
    let filePath = sprintf @".\%s.docx" name
    let myDoc = WordprocessingDocument.Create(filePath, WordprocessingDocumentType.
Document)

    // Add a new main document part.
    let mainPart = myDoc.AddMainDocumentPart()

    //Create Document tree
    mainPart.Document <- Document()

    //Create Body
    let body = Body()

    //Create paragraph
    let createParagraph text =
        let paragraph = Paragraph()
        let run_paragraph = Run()

        // put text
        let text_paragraph = Text text

        //Append elements.
        ignore <| run_paragraph.AppendChild(text_paragraph)
        ignore <| paragraph.AppendChild(run_paragraph)

        // return paragraph
        paragraph

    //create a paragraph with text
    let paragraph0 =
            createParagraph
                (sprintf "Dear %s" name)
    let paragraph1 =
            createParagraph
                (sprintf "Congratulations! You passed %s" course)

    [paragraph0; paragraph1]
    |> List.iter (body.AppendChild>>ignore)

    ignore <| mainPart.Document.AppendChild(body)

    // Save changes
    mainPart.Document.Save()

    // close the document and save all changes
    myDoc.Close()

// generate letters for all passing students
passedCandidates
|> Seq.iter generatePassLetter

Note

You need to add a reference to DocumentFormat.OpenXml.dll, FSharp.Data.TypeProviders.dll, System.Data.dll, System.Data.Linq.dll, System.Xml.dll, and WindowsBase.dll to make the project compile.

List.iter requires that the provided function returns unit. AppendChild returns a non-unit type. By using >> and the ignore function, you can get a new function that returns unit. If you forget how to use the |>, <|, <<, and >> operators, you can refer to the “Pipeline-Forward Operator” section in Chapter 1.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset