F# 3.0 introduces a new feature called type providers. The major application of a type provider is to provide language features for working within a data-rich environment. One example of a data-rich environment is the database. One major problem when programming against a database is how to generate code for database tables, stored procedures, and other database objects. Not every developer prefers to write database-access code manually, and nice tooling support is something developers always desire. Ideally, you could interact with a rich data environment such as a database using Microsoft Visual Studio features such as IntelliSense. Using IntelliSense would make it easier to write the necessary code, and it would save the time it would otherwise take to continually go back to check the database schema via the database management console.
The SQL Server type provider is designed to solve this problem, and it is shipped with Visual Studio 2012. One of the benefits you get with type providers is that they bring the database or web service schema information to your fingertips in the form of IntelliSense. As a result, when you type, the remote data structure is retrieved, which helps you finish your coding much more quickly. For a data-rich environment, this feature can improve your productivity significantly. If you’re familiar with code-generation tools such as SqlMetal.exe, you can think of a type provider as a wrapper around these tools. At first glance, the type provider seems like something similar to tools that are currently integrated into the Visual Studio integrated development environment (IDE). Actually, an F# type provider does not rely on the IDE. You can use your favorite code editor, and the F# compiler and type provider assembly will bring all of the generated types to you. Note that this process does not involve any IDE work. In addition to being independent of the Visual Studio IDE, a type provider is an open framework compared to other tools, such as svcutil.exe. This means that developers can write their own type provider or providers. I’ll cover how to write a custom type provider in Chapter 5.
The following four built-in type providers can be used to access data from databases and web services:
LINQ-to-SQL
SQL Entity
Web Services Description Language (WSDL)
Open Data Protocol (OData)
This chapter focuses on how to use these type providers and F# queries. It also shows how to use type providers to access the Microsoft Windows Azure Marketplace.
The first obvious data-rich environment is a database. Assume you have a local SQL database, and the database schema is like the one shown in Figure 4-1. This is a simple student-course system. The Student table stores information about all the students. The Course table stores all the course information, and the CourseSelection table is where course-selection information for all students is stored.
Like any new programming feature, type providers require a certain syntax. Visual Studio ships with a number of item templates to generate the majority of the code. The only thing left for the user to do is fill out connection information. For a SQL type provider, it is the database connection string. Figure 4-2 shows item templates for four built-in type providers. The LINQ-to-SQL type provider is the first one. In this chapter, I use the term SQL type provider to represent the LINQ-to-SQL type provider.
After clicking the Add button, the SQL type provider code is added. Along with the code, a reference to a crucial assembly named FSharp.Data.TypeProviders is also added. This is where the type provider resides. The generated code is shown in Example 4-1. The connection string is highlighted, and that is all you need to change to make this type provider work. The most convenient way to try a new feature is to use F# Interactive (FSI). Most of the samples in this chapter will use FSI as the default way to execute.
// The SqlDataConnection (LINQ-to-SQL) TypeProvider allows you to write code that // uses a live connection to a database. For more information, please go to // http://go.microsoft.com/fwlink/?LinkId=229209 module SQLDataConnection1 #if INTERACTIVE #r "System.Data" #r "System.Data.Linq" #r "FSharp.Data.TypeProviders" #endif open System.Data open System.Data.Linq open Microsoft.FSharp.Data.TypeProviders // You can use Server Explorer to build your ConnectionString. type SqlConnection = Microsoft.FSharp.Data.TypeProviders.SqlDataConnection<ConnectionStri ng = @"Data Source=(LocalDB)v11.0;Initial Catalog=tempdb;Integrated Security=True"> let db = SqlConnection.GetDataContext() //let table = query { // for r in db.Table do // select r // } // //for p in table do // printfn "%s" p.Property
The #if INTERACTIVE
section allows the code in Example 4-1 to be executed in FSI. In other words, you can select the code segment and test it in FSI when desired.
When the code needs to be executed in FSI, module SqlDataConnection1
should not be selected.
The type provider accepts a string value for its connection string. The value can be accessed only at run time, and it is very difficult to use a run-time value to generate design time logic. One way to work around this is to use the Literal attribute and conditional compile symbols. The sample code is shown in Example 4-2. This trick can work for all type providers. The other way to specify the connection string is to use a configuration file, which I will leave for you to explore.
#if DEVELOPMENT [<Literal>] let conStr = "< my development environment connection string> " #else [<Literal>] let conStr = "< production environment connection string> " #endif // define a type provider with constant value type T = Microsoft.FSharp.Data.TypeProviders.SqlDataConnection<ConnectionString=conStr> let db = T.GetDataContext()
If you’re not using integrated authorization, the user password is stored in the binary. The GetDataContext method has an overloaded version that takes the connection string at run time. The schema of the database accessed at run time, which is specified by the connection string provided at run time, must match the one used at compile time. If it does not match exactly, at least the schema accessed by your code should be same.
Using conditional compilation symbols is not a best practice when creating assemblies, but it is totally fine when writing an F# script.
The compilation process displays a warning message box like the one shown in Figure 4-3. To retrieve the database schema information, the type provider needs to access the database specified by the connection string. To ensure that you are always aware of this external access, you will be asked if this access should be granted. Once you feel comfortable, you can click Enable to allow the type provider to do its thing.
For other third-party type providers, to which you later decide to revoke trust, you can go to Tools, Option, F# Tools and press the Delete key to remove the trusted type providers. Figure 4-4 provides an example.
Depending on the network and database server speed, it can take a few seconds for the type provider to gather all the schema information locally. IntelliSense will be shown when you try to access the database context instance named db. IntelliSense shows the table names and stored procedure names. You do not have to open Server Explorer or Notepad; the schema information from the database is just at your fingertips. See Figure 4-5.
When you uncomment the query and FOR loop with printfn inside, you will receive an error because the query targets a fake database. Fortunately, IntelliSense guides you so that you can easily fix those errors and get the executable code looking like what’s shown in Example 4-3. The code prints all of the course names in the table. The F# query feature is a large topic, and I’ll explain it further in the Query section later in the chapter. For now, we can use a simple query syntax to verify that our type provider is working correctly.
// query the course table let table = query { for r in db.Course do select r } // print out course names for p in table do printfn "%s" p.CourseName
After correcting the compile error, you might find that IntelliSense can also show all the column names for the Course table. This is shown in Figure 4-6.
IntelliSense shows up only within an F# project. When referenced from a C# project, the provided type and the db variable are visible but no column names are shown. This is also true for other type providers. IntelliSense information is provided only by the F# language service.
The stored procedure is common in SQL databases. By default, the LINQ-TO-SQL type provider allows stored procedures to also be visible. You can set the storedProcedures parameter to false to disable it, as shown in Example 4-4.
type SqlConnection = Microsoft.FSharp.Data.TypeProviders.SqlDataConnection<ConnectionString =
@"Data Source=taliu0;Initial Catalog=TestDataBase;Integrated Security=True",
StoredProcedures=false>
let db = SqlConnection.GetDataContext()
The SqlConnection type also contains a class definition representing tables. The generated table class is nested under SqlConnection.ServiceTypes. These types make it possible for you to delete, insert, or update records in a database, as shown in Example 4-5.
Add a new item
let newRecord = new SqlConnection.ServiceTypes.Course(CourseName = "aa", CourseID = 100)
db.Course.InsertOnSubmit(newRecord)
try
db.DataContext.SubmitChanges()
printfn "record added"
with _ -> printfn "update failed"
Delete an item
db.Course.DeleteOnSubmit(newRecord)
try
db.DataContext.SubmitChanges()
printfn "record deleted"
with _ -> printfn "update failed"
More generally, you can execute any SQL command by using the ExecuteCommand method in the DataContext. See Example 4-6.
try let numOfRecord = db.DataContext.ExecuteCommand("select * from Course") printfn "%d records affected" numOfRecord with _ -> printfn "failed"
The SQL type provider uses SQLMetal.exe as the underlying code-generation tool. The SQL type provider has many parameters that can be configured to specify how SQLMetal will generate the code. Table 4-1 lists all the parameters the SQL type provider supports. Although all the parameters are optional, you have to specify either ConnectionString or ConnectionStringName at the very least.
Parameter Name | Description |
ConfigFile | The name of the configuration file used for connection strings. Its default value is either app.config or web.config. |
ConnectionString | The connection string for the database connection. |
ConnectionStringName | The name of the connection string for the database connection in the configuration file. |
ContextTypeName | The name of the data context class. The name is derived from the database name by default. |
DataDirectory | The name of the data directory; it’s used to replace DataDirectory in the connection strings. The project or script directory is the default value. |
ForceUpdate | Requires that a direct connection to the database be availabe at design-time, and forces the refresh of the local schema file. The default value is TRUE. |
Functions | Extracts database functions. The default value is TRUE. |
LocalSchemaFile | The local .dbml file for the database schema. There is no local schema file by default. |
Pluralize | Automatically pluralizes or singularizes class and member names using English language rules. The default value is FALSE. |
ResolutionFolder | The folder used to resolve relative file paths at compile time. The default value is the project or script folder. |
Serializable | Generates uni-directional serializable classes. The default behavior is to have no serialization. |
StoredProcedure | Extracts a stored procedure. The default value is TRUE. |
TimeOut | The timeout value is specified in seconds which is used when SqlMetal accesses the database. The default value is 0, which means an infinite amount of time. |
Views | Extracts database views. The default value is TRUE. |
If you are not satisfied with the code generated by the SQL type provider, which is primarily provided by LINQ-to-SQL, the SQL Entity type provider is another way for you to access a database. According to the MSDN document at http://msdn.microsoft.com/en-us/library/bb386976, LINQ-to-SQL is a SQL access technology that works only with Microsoft SQL Server. The ADO.NET Entity Framework provides a more flexible way to handle not only the SQL Server database but also other relational databases. If you are not familiar with ADO.NET Entity Framework, the following is the definition from MSDN (http://msdn.microsoft.com/en-us/library/bb386876.aspx):
The ADO.NET Entity Framework supports data-centric applications and services, and provides a platform for programming against data that raises the level of abstraction from the logical relational level to the conceptual level. By enabling developers to work with data at a greater level of abstraction, the Entity Framework supports code that is independent of any particular data storage engine or relational schema.
If you try to access an Azure SQL database by using the SQL type provider introduced in the “LINQ-to-SQL Type Provider” section, you’ll get an error. The right way to access the data in an Azure SQL database is to use the SQL Entity type provider. Overall, creating a SQL Entity type provider is similar to creating the SQL type provider shown earlier. Figure 4-7 shows how to add an Entity Framework–based type provider by using the Add New Item dialog box.
The actual code for the SQL Entity type provider is quite similar to the SQL type provider. The only difference is the type name. The SQL type provider’s type name is SqlDataConnection; the SQL Entity type provider’s type name is SqlEntityConnection. The sample code is shown in Example 4-7.
#if INTERACTIVE #r "System.Data" #r "System.Data.Entity" #r "System.Data.Linq" #r "FSharp.Data.TypeProviders" #endif open System.Data open System.Data.Entity open Microsoft.FSharp.Data.TypeProviders // You can use Server Explorer to build your ConnectionString. type internal SqlConnection = Microsoft.FSharp.Data.TypeProviders.SqlEntityConnection<ConnectionString = @"Data Source=(LocalDB)v11.0;Initial Catalog=tempdb;Integrated Security=True"> let internal db = SqlConnection.GetDataContext()
Because the SQL Entity type provider cannot be used across assemblies, its type can be only private or internal. This restriction prevents an F# assembly from exposing the type provider to a C# project.
The Entity Framework does not require the database to be SQL Server database. You can use SQL Entity type provider to target an Azure cloud database, as shown in Example 4-8.
module SQLEntityConnection1 #if INTERACTIVE #r "System.Data" #r "System.Data.Entity" #r "FSharp.Data.TypeProviders" #endif open System.Data open System.Data.Entity open Microsoft.FSharp.Data.TypeProviders [<Literal>] let con = """Server=tcp:<your server>.database.windows.net,1433; Database=<your database>;User ID=<user name>;Password=<your password>;Trusted_Connection=False;Encrypt=True;Connection Timeout=30""" // You can use Server Explorer to build your ConnectionString. type internal SqlConnectionT = Microsoft.FSharp.Data.TypeProviders.SqlEntityConnection<ConnectionString = con> let internal db = SqlConnectionT.GetDataContext()
The way to add and delete commands for the SQL Entity type provider is different; the type provider uses AddObject and DeleteObject to add and remove a record or records from a database. Example 4-9 shows how to use these two methods.
module SQLEntityConnection1 #if INTERACTIVE #r "System.Data" #r "System.Data.Entity" #r "FSharp.Data.TypeProviders" #endif open System.Data open System.Data.Entity open Microsoft.FSharp.Data.TypeProviders [<Literal>] let con = """Server=tcp:<your server>.database.windows.net,1433; Database=<your database>;User ID=<user name>;Password=<your password>;Trusted_Connection=False;Encrypt=True;Connection Timeout=30""" // You can use Server Explorer to build your ConnectionString. type internal SqlConnectionT = Microsoft.FSharp.Data.TypeProviders.SqlEntityConnection<ConnectionString = con> let internal db = SqlConnectionT.GetDataContext() let internal newRecord = new SqlConnectionT.ServiceTypes.Course(CourseName = "aa", CourseID = 100) db.Course.AddObject(newRecord); try let recordNumber = db.DataContext.SaveChanges() printfn "%d record(s) affected" recordNumber with _ -> printfn "update failed" db.Course.DeleteObject(newRecord); try let recordNumber = db.DataContext.SaveChanges() printfn "%d record(s) affected" recordNumber with _ -> printfn "update failed"
1 record(s) affected 1 record(s) affected
If you want to execute your own SQL query, you can use the ExecuteStoreQuery method. Example 4-10 shows how to use this method to check the total number of courses in the database.
let result = db.DataContext.ExecuteStoreQuery("select count(*) from Course")
printfn "%d course(s) in the database" (result |> Seq.head)
Like the SQL type provider, the SQL Entity type provider needs either ConnectionString or ConnectionStringName as a parameter. The full list of parameters is shown in Table 4-2.
Parameter Name | Description |
ConfigFile | The name of the configuration file used for connection strings. The default value is either app.config or web.config. |
ConnectionString | The connection string to the database. |
ConnectionStringName | The name of the connection string for the database connection in the configuration file. |
DataDirectory | The name of the data directory, used to replace DataDirectory in the connection string. The default value is the project folder or script folder. |
EntityContainer | The name to use for the EntityContainer in the conceptual model. |
ForceUpdate | Requires that a direct connection to the database be available at design time, and forces the refresh of the local schema file. The default value is TRUE. |
LocalSchemaFile | The local file for the database schema. |
Pluralize | Automatically pluralizes or singularizes class and member names using English language rules. The default value is FALSE. |
Provider | The name of the ADO.NET data provider to be used for Store Schema Definition Language (SSDL) generation. If you are interested in SSDL, please refer to MSDN document at http://msdn.microsoft.com/en-US/data/jj652016. The default provider is System.Data.SqlClient. |
ResolutionFolder | The folder used to resolve relative file paths at compile time. The default value is the project folder or script folder. |
SuppressForeignKeyProperties | Excludes foreign key properties in entity type definitions. The default value is FALSE. |
The web is another area that is rich with data. The public application programming interfaces (APIs) exposed by companies around the world is increasing exponentially year by year. If the data schema can be automatically accessed and shown with IntelliSense, this will make your coding significantly easier and boost productivity.
Web Services Description Language (WSDL) is an XML-based language which is used to describe the functionality provided by a web service. A WSDL file offers a machine-readable XML document that explains how the service can be called, what operations are available, what parameters are required, and the values that can be returned.
The easiest way to add a WSDL type provider is to use the Add New Item dialog box in Visual Studio, which is shown in Figure 4-8. The code generated is shown in Example 4-11. When you connect to your favorite data source, you get not only IntelliSense about data names but also comments to help you understand what the data is for. Isn’t that nice!
open System.Runtime.Serialization open System.ServiceModel open Microsoft.FSharp.Data.TypeProviders // You can sign up for a Bing service developer account at http://msdn.microsoft.com/en- us/library/gg605201.aspx let BING_APP_ID = "<your Bing Maps Developer Key>" // Using Bing Map API routing service to calculate the driving distance between two Geolocations. http://www.microsoft.com/maps/developers/mapapps.aspx type RouteService = Microsoft.FSharp.Data.TypeProviders.WsdlService<ServiceUri = "http://dev.virtualearth.net/webservices/v1/routeservice/routeservice.svc?wsdl"> type RouteCommon = RouteService.ServiceTypes.dev.virtualearth.net.webservices. v1.common type Route = RouteService.ServiceTypes.dev.virtualearth.net.webservices.v1.route let startPoint = Route.Waypoint( Location = RouteCommon.Location( Latitude = 47.64012046, Longitude = -122.1297104)) let endPoint = Route.Waypoint( Location = RouteCommon.Location( Latitude = 47.62049103, Longitude = -122.3492355)) let routReq1 = new Route.RouteRequest(Waypoints = [|startPoint; endPoint|]) routReq1.Credentials <- new RouteCommon.Credentials(ApplicationId = BING_APP_ID) RouteService.GetBasicHttpBinding_IRouteService().CalculateRoute(routReq1).Result .Summary.Distance |> printfn "Driving Distance = %A (miles)"
The parameters for the WSDL type provider are listed in Table 4-3. Only ServiceUri is a mandatory parameter; the other parameters are optional.
Parameter Name | Description |
Async | Indicates if whether both synchronous and asynchronous method signatures should be generated. The default value is FALSE, which means that only synchronous method signatures will be generated. |
CollectionType | A fully-qualified or assembly-qualified name of the type to use as a collection data type when code is generated from the schema. |
Indicates whether the INotifyPropertyChanged interface should be added on all data contract types to enable data binding. The default value is FALSE. | |
ForceUpdate | Requires that a direct connection to the service be available at design time, and forces the refresh of the local schema file. The default value is TRUE. |
LocalSchemaFile | The .wsdl schema file to store locally cached service schema. |
MessageContract | Indicates whether message contract types should be generated. The default value is FALSE. |
ResolutionFolder | The folder used to resolve relative file paths at compile time. The default value is the folder that contains the project or script. |
Serializable | Indicates whether the generated classes should be marked with the Serializable attribute. The default value is FALSE. |
ServiceUri | The URI for the WSDL service. This is a mandatory parameter. |
Another way to access remote services or data is to use the Open Data Protocol (OData) type provider. OData is an open web protocol for querying and updating data over the HTTP transport protocol. You can use it to retrieve results in formats like Atom, JSON, or plain XML. Figure 4-9 shows how to use Visual Studio to create an OData type provider, and the generated sample code is shown in Example 4-12. The OData type provider does not support fixed queries. You have to download the fixed query schema file to work with the OData type provider.
open System.Data.Services.Client open Microsoft.FSharp.Data.TypeProviders // Access demographic info such as population and income from the OData service at Azure Marketplace. For more information, go to https://datamarket.azure.com/dataset/ c7154924-7cab-47ac-97fb-7671376ff656 type Demographics = Microsoft.FSharp.Data.TypeProviders.ODataService<ServiceUri = "https://api.datamarket.azure.com/Esri/KeyUSDemographicsTrial/"> let ctx = Demographics.GetDataContext() //To sign up for an Azure Marketplace account at https://datamarket.azure.com/ account/infoctx.Credentials <- System.Net.NetworkCredential("<your liveID>", "<your Azure Marketplace Key>")
You need a Windows Live ID and an Windows Azure Marketplace key to show IntelliSense and compile the code from the previous example.
Compared to the WSDL type provider, the OData type provider’s parameter list (shown in Table 4-4) is much shorter. Like the WSDL type provider, only ServiceUri is a mandatory parameter; the other parameters are optional.
Parameter Name | Description |
DataServiceCollection | Indicates whether or not to generate collections derived from DataServiceCollection. The default value is FALSE. |
ForceUpdate | Requires that a direct connection to the service be available at design time, and forces the refresh of the local schema file. The default value is TRUE. |
LocalSchemaFile | The local .csdl file for the service schema. |
ResolutionFolder | The folder used to resolve relative file paths at compile time. The default value is the folder that contains the project or script. |
ServiceUri | The URI for the OData service. This parameter is mandatory. |
As I mentioned at the beginning of this chapter, the type provider is extensible. Because the type provider concept is public, many type providers have been developed by the community. Many of these type providers are provided by the FSharpX project on GitHub. I’m not going to present a detailed introduction to these type providers, but you can find out more about them at https://github.com/fsharp/fsharpx. This website provides information on type providers such as these:
File System. Used to provide type information taken from a file system. The type provider provides types for directories and files on the computer.
Xml File. Used to provide type information for an XML file. The XML type provider can infer a schema from the loaded data and generate types with properties.
JSON. Used to provide type information for JSON data. The JSON type provider provides strongly typed access to data provided in the JavaScript Object Notation (JSON) data-interchange format.
Registry. Used for the registry. The registry key and value are generated from the type provider.
Xaml. Provides type information for a WPF XAML file, which is essentially in XML format. The generated types are WPF UI elements.
The other way to get these type providers is to use the NuGet packages. Figure 4-10 shows how to access the management options for NuGet packages from a project reference. The FSharpX type provider for NuGet packages can be located by typing TypeProviders in the Search box in the upper-right corner of the Manage NuGet Packages dialog box, as shown in Figure 4-11.
The type provider provides a way for F# programmers to interpret the data from the outside world. You are not given an XmlElement when parsing an XML file; instead, you are given a more meaningful data type, such as StudentInfo. You’ll have an opportunity to write your own custom type provider in Chapter 5.
When a type provider brings back a large amount of data, you need to have a sufficient way to process that data. Luckily, F# offers query to help. The query syntax is similar to that of an SQL query or LINQ query. I use the SQL data or entity provider in the sample, but you can use the F# query syntax to interact with any other type providers as well.
To see the query syntax work with one of the previously shown SQL type providers, you need to first set up an SQL database. The database is a basic student-course database with Student, Course, and CourseSelection tables. The SQL script in Example 4-13 is used to create the student-course database and some sample data.
The query is independent of a specific type provider, which means it can be applied to any type provider and even to a memory data structure. In the sample code, one person’s age is set to NULL. This is not a bug. This NULL value will demonstrate how to use F# nullable operators.
CREATE TABLE Student( StudentID int not null constraint PrimaryKey primary key, Name nvarchar(50) null, Age int null,); CREATE TABLE Course( CourseID int not null constraint PrimaryKey2 primary key, CourseName varchar(max) null); CREATE TABLE AnotherStudentTable ( StudentID int not null constraint PrimaryKey3 primary key, Name nvarchar(50) null, Age int null,); go CREATE TABLE CourseSelection( StudentID int not null constraint foreignKey references dbo.Student(StudentID), CourseID int null constraint foreignKey2 references dbo.Course(CourseID), ID int null); go INSERT Student VALUES (1, 'Lisa', 21) INSERT Student VALUES (2, 'Brent', 22) INSERT Student VALUES (3, 'Anita', 20) INSERT Student VALUES (4, 'Ken', 22) INSERT Student VALUES (5, 'Cathy', 22) INSERT Student VALUES (6, 'Tom', 20) INSERT Student VALUES (7, 'Zeoy', 21) INSERT Student VALUES (8, 'Mark', 23) INSERT Student VALUES (9, 'John', null) go INSERT Course VALUES (1, 'Math') INSERT Course VALUES (2, 'Physics') INSERT Course VALUES (3, 'Biology') INSERT Course VALUES (4, 'English') go INSERT CourseSelection VALUES(1, 1, null) INSERT CourseSelection VALUES(2, 1, null) INSERT CourseSelection VALUES(3, 1, null) INSERT CourseSelection VALUES(2, 2, null) INSERT CourseSelection VALUES(2, 3, null) INSERT CourseSelection VALUES(3, 3, null) INSERT CourseSelection VALUES(3, 2, null) go
I’ll use the SQL Entity type provider in the sample code. All the sample code is written on the assumption you already created a SQL Entity type provider and assigned it to a value named db. Because some query operations are not supported in the SQL Entity type provider, I’ll use AsEnumerable or array to create an in-memory data structure. The code is shown in Example 4-14.
open System
open System.Linq
open Microsoft.FSharp.Linq.NullableOperators
// define a student class
type Student() =
member val ID = 0 with get, set
member val Name = "" with get, set
member val Age = Nullable(0) with get, set
// define the general show result function
let showResult query = query |> Seq.iter (printfn "%A")
// set the in-memory student table
let studentTable = [
Student(ID = 1, Name = "Anita", Age = Nullable(22));
Student(ID = 1, Name = "Kate", Age = Nullable(22))
Student(ID = 1, Name = "Brent", Age = Nullable(23))
Student(ID = 1, Name = "Lisa", Age = Nullable(23))
Student(ID = 1, Name = "Mark", Age = Nullable(24)) ].AsQueryable()
The code in Example 4-15 is a simple select example. You can think of the select operator as being like the yield keyword in sequence generation or the select extension method in LINQ. It tells the compiler to return the value. The for...in...do code iterates through the data, and the select operator specifies what needs to be inserted into the final result. I’ll use the code let showResult query = query |> Seq.iter (printfn "%A")
to show the query result.
If the query can be translated to the server-side code, it will be executed on the server side. Unfortunately, not all computations can be translated to the server-side code. Example 4-16 uses AsEnumerable() to specify that the computation should be performed on the client side, because sprintf cannot be translated to server-side code. Once the data is local, you are free to use any .NET function or pipeline operator.
You can also create more complex queries that involve multiple select statements. Example 4-17 shows how to print out all the value pairs where the first value is less than the second value. For this example, a value pair is formed from two data sources.
let A = [0; 2; 4; 5; 6; 8; 9] let B = [1; 3; 5; 7; 8] query { for a in A do for b in B do where (a < b) select (a, b) } |> Seq.iter(fun (n, e) -> printfn "%d is less than %d" n e)
0 is less than 1 0 is less than 3 0 is less than 5 0 is less than 7 0 is less than 8 2 is less than 3 2 is less than 5 2 is less than 7 2 is less than 8 4 is less than 5 4 is less than 7 4 is less than 8 5 is less than 7 5 is less than 8 6 is less than 7 6 is less than 8
The where operator is used to filter data according to given conditions. Like the select operator, the computation can be executed either on the server side or client side. Example 4-18 shows how to filter the student names whose length is greater than 4 characters long. Because the result is filtered on the server side, only three items are transferred over the wire.
Although performing the filter action on the client side is not a recommended approach, especially if the resulting data set is large, I still list the sample code here in case complex filtering is needed on the client side. See Example 4-19.
Example 4-20 is another example of filtering data on the client side. The code gives each element an ID value and filters the results later according to this client-side-generated ID.
The join example in Example 4-21 shows how to join data that is scattered across different tables. In the join statement, the order of the keys is significant. The outside table db.Student, which is the main table, must be present first in the on condition part. The example lists the course ID and student name as a result.
Code for the join example
let internal q5 = query {
for student in db.Student do
join course in db.CourseSelection on (student.StudentID = course.StudentID)
select (course.CourseID, student.Name) }
showResult q5
Execution result
(3, "Anita") (1, "Lisa") (2, "Anita") (2, "Brent") (3, "Brent") (1, "Anita") (1, "Brent")
The CourseID in the result from Example 4-21 is far from user friendly. If the student name can be shown, that will be great. In Example 4-22, a groupJoin operator is used to show a more user-friendly result. Because CourseID is a nullable integer type, you can use “?=” in the Microsoft.FSharp.Linq.NullableOperators module to perform the comparison.
let q6 = query { for student in db.Student do groupJoin courseSelection in db.CourseSelection on (student.StudentID = courseSelection.StudentID) into g for courseSelection in g do join course in db.Course on (courseSelection.CourseID ?= course.CourseID) select (student.Name, course.CourseName) } showResult q6
("Anita", "Biology") ("Lisa", "Math") ("Anita", "Physics") ("Brent", "Physics") ("Brent", "Biology") ("Anita", "Math") ("Brent", "Math")
There is another join scenario called an outer join. The F# outer join operator is similar to SQL’s outer join, but it supports only the left outer join. The right outer join functionality can be achieved by switching the two tables. Example 4-23 shows how to use leftOuterJoin to show the course selection information.
let internal q5 = query { for student in db.Student do leftOuterJoin selection in db.CourseSelection on (student.StudentID =? selection.CourseID) into result for selection in result do select (student.Name, selection.Course.CourseName) } showResult q5
Execution result
("Lisa", "Math") ("Lisa", "Math") ("Lisa", "Math") ("Brent", "Physics") ("Brent", "Physics") ("Anita", "Biology") ("Anita", "Biology") ("Ken", null) ("Cathy", null) ("Tom", null) ("Zeoy", null) ("Mark", null) ("John", null)
Because CourseID is a Nullable<int>, you need to use =? to perform the equality operation. You need to use open Microsoft.FSharp.Linq.NullableOperators
. Both =? and ?= are valid operators. If the nullable value is on left side of the operator, ?= is applied; however, =? is used if the nullable value is on right side.
Using the Value property in a Nullable<T> type does not work in the query. This is by design, because using it in this way can lead to a NULL exception when the Nullable<T> value is NULL.
Example 4-24 shows how to use the sortBy operator to show the sorted names whose name length is greater than 4 characters. By default, sortBy sorts the result in ascending order. The sortByDescending operator serves the purpose of sorting the results in a descending manner.
You can sort by a second criterion by using thenBy or thenByDescending. Example 4-25 sorts the result first by a student’s name and then by the student’s age.
In Example 4-25, Age is a nullable value, so you have to reference the Value property in order to sort. The F# query feature provides a set of operators that can handle nullable values. Example 4-26 shows how to use sortByNullableDescending.
sortByNullable. Sorts elements in ascending order by a given nullable key
sortByNullableDescending. Sorts elements in descending order by a given nullable key
thenByNullable. Like the thenBy operator, performs subsequent ordering of the elements in ascending order by a given nullable key
thenByNullableDescending. Like the thenByDescending operator, performs subsequent ordering of the elements in descending order by a given nullable key
let internal q4_3 = query { for student in db.Student do sortByNullableDescending student.Age thenBy student.Name select (student.Name, student.Age) } showResult q4_3
Execution result
("Mark", 23) ("Brent", 22) ("Cathy", 22) ("Ken", 22) ("Lisa", 21) ("Zeoy", 21) ("Anita", 20) ("Tom", 20) ("John", null)
The group operator is often used with an aggregate function. Example 4-27 counts the number of students in each age group.
Again, the result is not user friendly because the result does not show the student names in the result. To show the student names, you can use groupValBy to select a value with the group. The full code is shown in Example 4-28. It shows a student name sequence in each age group.
Code for using groupValBy
let internal q8 = query {
for student in db.Student do
groupValBy student.Name student.Age into g
select (g, g.Key, g.Count()) }
showResult q8
Execution result
(seq ["John"], null, 1) (seq ["Anita"; "Tom"], 20, 2) (seq ["Lisa"; "Zeoy"], 21, 2) (seq ["Brent"; "Ken"; "Cathy"], 22, 3) (seq ["Mark"], 23, 1)
The take and skip operators are used when you are interested only in a certain range of the query results. Example 4-29 sorts the students by age and skips the first student. This can be very useful in an OData service, which often returns the results in pages.
The skip and take operators are similar when used in the query. The take example is shown in Example 4-30. The sample code sorts the students by age and returns only the first three student names.
The skip operation also can be used with a condition statement rather than just a given number. Example 4-31 shows how to skip the results for people in the database who have an age value less than 23.
The code for the skipWhile example
open System open System.Linq open Microsoft.FSharp.Linq.NullableOperators type Student() = member val ID = 0 with get, set member val Name = "" with get, set member val Age = Nullable(0) with get, set let showResult query = query |> Seq.iter (printfn "%A") let studentTable = [ Student(ID = 1, Name = "Anita", Age = Nullable(22)); Student(ID = 1, Name = "Kate", Age = Nullable(22)) Student(ID = 1, Name = "Brent", Age = Nullable(23)) Student(ID = 1, Name = "Lisa", Age = Nullable(23)) Student(ID = 1, Name = "Mark", Age = Nullable(24)) ].AsQueryable() // skip student whose age is less than 23 let internal q10 = query { for student in studentTable do sortByNullable student.Age skipWhile (student.Age ?< 23) select student.Name } showResult q10
Execution result
"Brent" "Lisa" "Mark"
Because one student’s age is NULL, you have to use the ?< operator in the Microsoft.FSharp.Linq.NullableOperators module to compare the nullable value.
Because the underlying SQL data source does not support a skipWhile operator, we use the in-memory structure that was shown in Example 4-14.
There is not much to explain about these operations; their names describe their functionalities. Because you have to handle the nullable value, each operator has a twin to handle the nullable value. I present them using a nullable operator in this section. Example 4-32 shows how to get the maximum, minimum, and average ages of the student group, as well as the sum of their ages.
let internal q11 = query { for student in db.Student do sumByNullable student.Age } printf "sum of student age = " showResult [ q11 ] let internal q12 = query { for student in db.Student do minByNullable student.Age } printf "youngest student age = " showResult [ q12 ] let internal q13 = query { for student in db.Student do maxByNullable student.Age } printf "oldest student age = " showResult [ q13 ] let internal q14 = query { for student in db.Student do averageByNullable (Microsoft.FSharp.Linq.Nullable.float student.Age) } printf "average student age = " showResult [ q14 ]
Execution result
sum of student age = 171 youngest student age = 20 oldest student age = 23 average student age = 21.375
Sometimes you need only one element in the sequence. The head, last, and nth operators are designed for this purpose. You use the head operator on the SQL storage and the other operators with the in-memory structure. Example 4-33 shows the first student’s name. Example 4-34 shows how to get the last and nth elements.
// return first element let internal q15 = query { for student in db.Student do select student.Name head } showResult [ q15 ] // get the first element; if there is no first element returned, returns default value let internal q16 = query { for student in db.Student do select student.Name headOrDefault } showResult [ q16 ]
open System open System.Linq open Microsoft.FSharp.Linq.NullableOperators type Student() = member val ID = 0 with get, set member val Name = "" with get, set member val Age = Nullable(0) with get, set let showResult query = query |> Seq.iter (printfn "%A") let studentTable = [ Student(ID = 1, Name = "Anita", Age = Nullable(22)); Student(ID = 1, Name = "Kate", Age = Nullable(22)) Student(ID = 1, Name = "Brent", Age = Nullable(23)) Student(ID = 1, Name = "Lisa", Age = Nullable(23)) Student(ID = 1, Name = "Mark", Age = Nullable(24)) ].AsQueryable() // take the last student let internal q15 = query { for student in studentTable do select student.Name last } showResult [ q15 ] // take the last student; if there is no last student, use default value let internal q16 = query { for student in studentTable do select student.Name lastOrDefault } showResult [ q16 ] // return the 4th student in the list let internal q17 = query { for student in studentTable do select student.Name nth 3 } showResult [ q17 ]
"Mark" "Mark" "Lisa"
The count operator is used to count the number of elements. Example 4-35 shows how many students are in the Student table.
let internal q17 = query {
for student in db.Student do
count }
showResult [ q17 ]
You might have noticed in the results from Example 4-21 that some of the student names were repeated. The distinct operator is used to get the distinct elements in the result set, as shown in Example 4-36.
The contains and exists operators are used to determine whether the set contains elements that meet a given condition. Example 4-37 checks whether there is a student named Lisa.
let internal q19 = query { for student in db.Student do select student.Name contains "Lisa" } showResult [ q19 ] let internal q20 = query { for student in db.Student do exists (student.Name = "Lisa") } showResult [ q20 ]
The contains and exists operators return only a Boolean value. If you need to see whether the first element meets certain criteria, you can use find as shown in Example 4-38. If there is no student who meets the criteria, a System.InvalidOperationException will be thrown.
The all operator is used to test whether all the elements meet the criterion. Example 4-39 is used to check whether all the students’ names start with the letter L.
let internal q22 = query { for student in db.Student do all (SqlClient.SqlMethods.Like(student.Name, "L%")) } showResult [ q22 ]
To check whether a class has only one student with the name Lisa and return her information, you can use exactlyOne or exactlyOneOrDefault. The exactlyOne operation yields a System.InvalidOperationException if there is not exactly one match. See Example 4-40.
The previous section provided a number of query operators. For some readers, the list might be too long, while others might still be waiting for more. If you are looking for more, you might be trying to map your existing T-SQL knowledge to F# query. If so, this section is the right place to be. I won’t repeat my explanations of select or where here because they are easy to map to your existing T-SQL knowledge. If you are familiar with T-SQL, the following list can help you easily transfer your T-SQL knowledge to F# query:
The in keyword in T-SQL is used to check whether a value is in the specified list of values, as shown in Example 4-41. The equivalent of this T-SQL code in F# query is shown in Example 4-42.
The top T-SQL keyword is used to return a number of elements on the top of the return list. At first glimpse, it might appear that F# query does not have an equivalent operator. However, closer examination will reveal that the take operator can do exactly the same thing.
The like T-SQL keyword is a character-mapping operator in SQL. Although like is not a keyword in F# query, you can use SqlMethods.Like, as shown in Example 4-39. SQLMethods also contains other methods that can be used to handle date and time differences. See Table 4-5 for more details.
Function Name | Corresponding SQL Operator | Description |
DateDiffDay | DATEDIFF | Counts the date difference in days |
DateDiffHour | DATEDIFF | Counts the date difference in hours |
DateDiffMicrosecond | DATEDIFF | Counts the date difference in microseconds |
DateDiffMillisecond | DATEDIFF | Counts the date difference in milliseconds. |
DateDiffMinute | DATEDIFF | Counts the date difference in minutes |
DateDiffMonth | DATEDIFF | Counts the date difference in months |
DateDiffNanosecond | DATEDIFF | Counts the date difference in nanoseconds |
DateDiffSecond | DATEDIFF | Counts the date difference in seconds |
DateDiffYear | DATEDIFF | Counts the date difference in years |
Like | LIKE | Decides whether the input matches the specific pattern |
The between T-SQL keyword is not a keyword in F# query, but you can use the comparison operators combined with && to achieve the same result.
The functionality provided by the union and intersection T-SQL keywords can be achieved with F# query, as shown in Example 4-43 and Example 4-44.
The case T-SQL keyword is supported in F# query by using an if statement. See Example 4-45 and Example 4-46.
In this section, I will present some F# operators that can be used as shortcut to make the query code more readable. As mentioned in the previous section, you will inevitably find scenarios that require bringing data to the client to perform certain computations. F# provides a few shortcuts for some of the more widely used .NET functions associated with these scenarios. These functions can be used in queries and in other data processing:
|||>, ||>, <|||, and <|| operators. These operators are used to break triple and tuple to a function. After reviewing the sample in Example 4-47, it is not difficult to find the similarity between these operators and the pipeline operator. For example, |||> takes a triple, breaks the triple into three elements, and feeds the elements into a function that takes three parameters in curried format.
// define a function taking 3 parameters in curried format let f3 a b c = a + b + c + 1 // define a function taking 2 parameters in curried format let f2 a b = a + b + 1 // define a function taking a parameter let f a = a + 1 let print x = printfn "value = %A" x print ( (1, 2, 3) |||> f3 ) print ( (1, 2) ||> f2 ) print ( 1 |> f ) print ( f3 <||| (1, 2, 3) ) print ( f2 <|| (1, 2) ) print ( f <| 1 )
value = 7 value = 4 value = 2 value = 7 value = 4 value = 2
sin, cos, tan, sinh, cosh, and tanh. The first three functions—sin, cos, and tan—are the trigonometric functions. sine, cosine, and tangent. The sinh, cosh, and tanh functions are the hyperbolic functions hyperbolic sine, hyperbolic cosine, and hyperbolic tangent.
log and log10. The log function is the natural logarithm function, while the log10 function is a logarithm function with base equal to 10.
min and max. These function names are used to get the minimum or maximum value of two given values.
floor, ceil, and round. The floor and ceil functions are designed to find the floor and ceil of a given number, while the round function is used to find the rounded value of a given number.
abs. The abs function is used to find the absolute value of a given number.
asin, acos, and atan. The asin, acos, and atan functions are inverse sine, inverse cosine, and inverse tangent functions.
In this chapter, you used type providers to query a database and an in-memory structure. Another source of data is the Windows Azure Marketplace (https://datamarket.azure.com/). Hundreds of data sources and APIs are available in the Marketplace, and the number is increasing every day. Many innovations and applications are enabled by using these data and web APIs. In this section, I’ll show how to use a type provider to access the free UK Foreign and Commonwealth Office Travel Advisory Service at the Windows Azure Marketplace. To use this free service, go to Windows Azure Marketplace and search for UK Foreign and Commonwealth Office Travel Advisory Service, the service is located at https://datamarket.azure.com/dataset/uk.gov/traveladvisoryservice. You can sign in with your Windows Live ID. Figure 4-12 shows the Details tab, which describes the service URL and service schema information.
As programmers, we are interested in the Details tab because it lists the connection string and all of the data schemas, including the data names and types. We will need the service root URI for this example.
To access the data in the Windows Azure Marketplace, you need an account. Account registration is free (again, at https://datamarket.azure.com/). (To register for an account, you first need a Windows Live ID.) Figure 4-13 shows the My Account page; take note of the Customer ID and Primary Account Key on your My Account page because you will need these in the future.
You can use OData to connect to the data source. The OData type provider needs three parameters:
Service URI API address: https://api.datamarket.azure.com/Uk.Gov/TravelAdvisoryService/
Windows Live ID
Primary Account Key from the Windows Azure Marketplace My Account page
The query code and partial execution result are shown in Example 4-48.
// define the type provider type Demographics = Microsoft.FSharp.Data.TypeProviders.ODataService<ServiceUri = "https://api.datamarket.azure.com/Uk.Gov/TravelAdvisoryService/"> let ctx = Demographics.GetDataContext() // set the credentials ctx.Credentials <- System.Net.NetworkCredential(<liveID>, <id>) // query the data source let q = query { for n in ctx.BritishEmbassiesAndHighCommissions do select (n.Title, n.OfficeHours) } // print out the result q |> Seq.iter (fun (address, _)-> printfn "%s" address)
Australia, Hobart, British Honorary Consul Australia, Adelaide, British Honorary Consul Afghanistan, Kabul, British Embassy Vietnam, Hanoi, British Council Taiwan, Kaohsiung, British Council Japan, Tokyo, British Council Burma, Rangoon (Yangon), British Embassy Australia, Canberra, Consular Section New Zealand, Auckland, British Consulate-General China, Guangzhou, British Consulate-General Australia, Melbourne, British Consulate-General Japan, Osaka, British Consulate-General China, Shanghai, British Consulate-General Australia, Brisbane, British Consulate Australia, Perth, British Consulate ...
Good! The connection is working well, and the first query returns data. Now, let’s find out about all the UK embassies or consulates in China. You might be tempted to try to achieve this goal by placing the where operator in the query. Unfortunately, the where keyword is not supported when executed on the server side. Instead, the filter operation has to be done on the client. See Example 4-49.
// define the type provider type Demographics = Microsoft.FSharp.Data.TypeProviders.ODataService<ServiceUri = "https://api.datamarket.azure.com/Uk.Gov/TravelAdvisoryService/"> let ctx = Demographics.GetDataContext() // set the credentials ctx.Credentials <- System.Net.NetworkCredential(<liveID>, <id>) // query the data source let q = query { for n in ctx.BritishEmbassiesAndHighCommissions do //where (n.Title.Contains("United States")) // where is not supported on server side select (n.Title, n.OfficeHours) } // print out the result q |> Seq.filter (fun (title, _) -> title.Contains("China")) |> Seq.iter (fun (title, _)-> printfn "%s" title)
China, Guangzhou, British Consulate-General China, Shanghai, British Consulate-General China, Chongqing, British Consulate-General China, Beijing, British Embassy China, Beijing, British Embassy, Cultural & Education Section
You get a surprise when you use the same code to query for UK embassies in the United States. The query does not return any results. It is hard to believe that the UK does not have at least one embassy in the country. The reason is that this query returns only part of all the results. For performance reasons, it would not be a wise decision to return all the results at once. The query returns only one page of data for each query execution. The results shown in Example 4-49 include only those from the first page of the total results. This is a good time to use the skip operator to ignore the previously retrieved data and return the next page of the query results, as shown in Example 4-50.
// define the type provider
type Demographics = Microsoft.FSharp.Data.TypeProviders.ODataService<ServiceUri =
"https://api.datamarket.azure.com/Uk.Gov/TravelAdvisoryService/">
let ctx = Demographics.GetDataContext()
// set the credentials
ctx.Credentials <- System.Net.NetworkCredential(<liveID>, <id>)
// query pages
let rec queryFunction (startPoint) =
seq {
// query the data source
let q = query {
for n in ctx.BritishEmbassiesAndHighCommissions do
skip startPoint
select (n.Title, n.OfficeHours) }
let count = Seq.length q
if count <> 0 then
for item in q do
yield item
for item in queryFunction(startPoint + count) do
yield item
}
// print out the result
queryFunction(0)
|> Seq.filter (fun (title, _) -> title.Contains("United States"))
|> Seq.iter (fun (title, _)-> printfn "%s" title)
United States, Denver, British Consulate-General United States, Atlanta, British Consulate-General United States, Boston, British Consulate-General United States, Chicago, British Consulate-General United States, Houston, British Consulate-General United States, Los Angeles, British Consulate-General United States, New York, British Consulate-General United States, San Francisco, British Consulate-General United States, Miami, British Consulate-General United States, Orlando, British Consulate United States, Washington, British Embassy United States, Anchorage, British Honorary Consulate United States, Nashville, British Honorary Consulate United States, New Orleans, British Honorary Consulate United States, Philadelphia, British Honorary Consulate United States, Phoenix, British Honorary Consulate United States, Pittsburgh, British Honorary Consulate United States, Salt Lake City, British Honorary Consulate United States, Charlotte, British Honorary Consulate United States, Michigan, British Honorary Consulate United States, Minnesota, British Honorary Consulate United States, Ohio, British Honorary Consulate
As mentioned in the previous section, the OData type provider does not work with “fixed query” datasets. Although the number of datasets that support only fixed queries is insignificant, it’s good to know how to interact with them when needed. In this section, I’ll use the Microsoft Translator, which supports only a fixed-query dataset, to demonstrate how to use F# to work with fixed-query datasets.
Any dataset that supports a fixed query provides a pre-created proxy class for the dataset. The proxy class file is available on the dataset Details page after you make a purchase from Azure Marketplace. In Figure 4-14, the C# proxy class file is shown in the upper-right portion of the page. You need to download this file and make a C# class library for the main F# project to reference. I am not going to go through how to create and reference a C# project. The F# code in Example 4-51 shows how to invoke the proxy class.
open System.Linq // define a translate function by taking an English sentence and translating it to Chinese let translate txt = let serviceRootUri = new System.Uri("https://api.datamarket.azure.com/Bing/MicrosoftTranslator/") // the account key is the same as in the previous section let accountKey = <id> // create a translator container that is defined in the proxy class let tc = Microsoft.TranslatorContainer(serviceRootUri) tc.Credentials <- new System.Net.NetworkCredential(accountKey, accountKey) //translate from English(en) to Chinese (zh-CHS) let translationQuery = tc.Translate(txt, "zh-CHS", "en") let translationResult = translationQuery.Execute().ToList().FirstOrDefault() // if there is no translation result, something went wrong so return empty string if (translationResult = null) then System.String.Empty else translationResult.Text
You can use this translation service to translate recent news from English to Chinese, as shown in Example 4-52.
// define the type provider type Demographics = Microsoft.FSharp.Data.TypeProviders.ODataService<ServiceUri = "https://api.datamarket.azure.com/Uk.Gov/TravelAdvisoryService/"> let ctx = Demographics.GetDataContext() // set the credentials ctx.Credentials <- System.Net.NetworkCredential(<liveID>, <id>) //get latest news let latestNews = query { for n in ctx.LatestTravelNewsFromFco do select (n.Title, n.Summary) } latestNews |> Seq.map fst // only leave title to translate |> Seq.map translate //translate from English to Chinese |> Seq.iter (fun n -> printfn "%s" n)
Execution result
恶劣天气影响整个欧洲的旅行 下个星期为欧洲足球赛事旅游忠告 西班牙机场中断 象牙海岸的旅游建议的变化 玻利维亚旅游忠告-鲁雷纳瓦克在封锁 在欧洲旅行的中断 澳大利亚旅游忠告-东澳大利亚的水浸 美国旅游忠告-暴雪条件 旅游忠告-尤文图斯 v 曼城 希腊旅游忠告-工业行动 ...
You might be wondering how to serialize or deserialize the data from Windows Azure Marketplace. I cover two approaches in this section. One shows you how to do the serialization programmatically, and the other shows you how to save the content to a Microsoft Excel file.
The topic of .NET serialization could be expanded into a 50-page chapter, so I’ll leave that for you to explore on your own. In this section, I present the JSON and XML serializers. The first example in Example 4-53 shows how to use the JSON serializer. The example demonstrates how to serialize and deserialize a tuple instance.
open System.Runtime.Serialization.Json open System.Runtime.Serialization let serializeAndDeserialize (originalValue:'T) = // Create a serializer to serialize the data let serializer = DataContractJsonSerializer(typeof<'T>) use out = new System.IO.MemoryStream() printfn "input data is %A" originalValue serializer.WriteObject(out, originalValue) // Get the serialized text let text = out.ToArray() |> System.Text.Encoding.UTF8.GetString out.Seek(0L, System.IO.SeekOrigin.Begin) |> ignore printfn "serialized data as: %s" text let deserializedValue = serializer.ReadObject out printfn "deserialized the data as: %A" deserializedValue serializeAndDeserialize (1,2)
Execution result
input data is (1, 2) serialized data as: {"m_Item1":1,"m_Item2":2} deserialized the data as: (1, 2)
If the XML serializer is your favorite, Example 4-54 shows you how to serialize the data to an XML file by using it. The sample demonstrates how to serialize and deserialize data from the UK government Travel Advisory Service to and from a local XML file.
// declare a NewsRecord class type NewsRecord() = member val Title = System.String.Empty with get, set member val Summary = System.String.Empty with get, set // define the type provider type Demographics = Microsoft.FSharp.Data.TypeProviders.ODataService<ServiceUri = "https://api.datamarket.azure.com/Uk.Gov/TravelAdvisoryService/"> let ctx = Demographics.GetDataContext() // set the credentials ctx.Credentials <- System.Net.NetworkCredential(<liveID>, <id>) // query the latest news let latestNews = query { for n in ctx.LatestTravelNewsFromFco do select (NewsRecord(Title = n.Title, Summary = n.Summary)) } // query the latest news let latestNews = query { for n in ctx.LatestTravelNewsFromFco do select { NewsRecord.Title = n.Title; NewsRecord.Summary = n.Summary} } let news = latestNews |> Seq.toArray // deserialize from xml let deserialize<'T> (fileName:string) = let reader = System.Xml.Serialization.XmlSerializer(typeof<'T>) use file = new System.IO.StreamReader(fileName) let fileData = reader.Deserialize(file) :?> 'T fileData // serialize data let serialize data (fileName:string)= let writer = System.Xml.Serialization.XmlSerializer(data.GetType()); use file = new System.IO.StreamWriter(fileName) writer.Serialize(file, data); //serialize the data serialize news "myLocalNews.xml" // deserialize the file let data = deserialize<NewsRecord array> "myLocalNews.xml"
Serialized XML file
<?xml version="1.0" encoding="utf-8"?> <ArrayOfNewsRecord xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <NewsRecord> <Title>Severe weather affecting travel across Europe</Title> <Summary>Adverse weather conditions are affecting travel across Europe, and there are airport closures and flight cancellations in the UK.</Summary> </NewsRecord> <NewsRecord> <Title>Travel advice for European football matches next week</Title> <Summary>Chelsea, Rangers and Spurs are playing in Europe next week. Check our travel advice for fans if you're going to the matches.</Summary> </NewsRecord> ...
XML and JSON serialization are very useful to a programmer, but the result is definitely not for the average user. In Example 4-55, I’ll you show how to use F# to write the cloud data into an Excel file. The sample retrieves the data from Azure, opens the Excel file, and inserts the title from cells B1 to B100 and the summary from cells C1 to C100.
open Microsoft.Office.Interop.Excel // Run Excel let app = new ApplicationClass(Visible = true) // Create a new file, and get the 1st worksheet let workbook = app.Workbooks.Add(XlWBATemplate.xlWBATWorksheet) // Note that worksheets are started from 1 not 0 let worksheet = (workbook.Worksheets.[1] :?> Worksheet) // define the type provider type Demographics = Microsoft.FSharp.Data.TypeProviders.ODataService<ServiceUri = "https://api.datamarket.azure.com/Uk.Gov/TravelAdvisoryService/"> let ctx = Demographics.GetDataContext() // set the credentials ctx.Credentials <- System.Net.NetworkCredential(<liveID>, <id>) // declare a NewsRecord class type NewsRecord() = member val Title = System.String.Empty with get, set member val Summary = System.String.Empty with get, set // query the latest news let latestNews = query { for n in ctx.LatestTravelNewsFromFco do select (NewsRecord(Title = n.Title, Summary = n.Summary)) } let news = latestNews |> Seq.toArray // get the range tuple such as "A1", "A10" let getRangeStrings columnName news = let start = sprintf "%c%d" columnName 1 let ''end'' = sprintf "%c%d" columnName (news |> Seq.length) start, ''end'' let len = news.Length // get range tuple, which contains start and end let range0 = (getRangeStrings 'B' news) let range1 = (getRangeStrings 'C' news) // insert title to B column (worksheet.Range (fst range0, snd range0)).Value2 <- Array2D.init len 1 (fun x _ -> news.[x].Title) // insert summary to C column (worksheet.Range (fst range1, snd range1)).Value2 <- Array2D.init len 1 (fun x _ -> news.[x].Summary)
In addition to an Excel file, you can also export data to a Microsoft Word file by using the Office OpenXML SDK, which can be downloaded from http://www.microsoft.com/en-us/download/details.aspx?id=5124. The sample presented in this section is used to generate a notification letter for students who pass an exam. The pass/fail information is stored in the database. The type provider queries the database and filters out the students who have passed the exam. The students’ information is then passed to various OpenXML methods to generate the Word documents. Example 4-56 creates the table in a SQL Server database. The table has three columns:
Name. Student name, which is also the keyword
Course. Course name
Passed. Boolean value to indicate whether the student passed the exam
CREATE TABLE [dbo].[StudentScoreTable]( [Name] [nvarchar](50) NOT NULL, [Course] [nchar](64) NULL, [Passed] [bit] NULL, CONSTRAINT [PK_StudentScoreTable] PRIMARY KEY CLUSTERED ( [Name] ASC )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY] ) ON [PRIMARY]
Insert test record data
insert into StudentScoreTable(Name,Course, Passed) Values('Brent', 'History', 1) insert into StudentScoreTable(Name,Course, Passed) Values('Chris', 'Math', 1) insert into StudentScoreTable(Name,Course, Passed) Values('Anita', 'History', 1) insert into StudentScoreTable(Name,Course, Passed) Values('Andy', 'Math', 0)
The code in Example 4-57 is F# code. It retrieves data from the database by using the SQL type provider. The generatePassLetter function is used to generate the Word document for students who pass the exam. The execution for this sample code generates three files, for the students named Anita, Chris, and Brent. Andy did not pass the exam, so there is no file for him.
open System open System.Linq open DocumentFormat.OpenXml open DocumentFormat.OpenXml.Packaging open DocumentFormat.OpenXml.Wordprocessing open Microsoft.FSharp.Data.TypeProviders open Microsoft.FSharp.Linq.NullableOperators type T = SqlDataConnection<"<your connection to the database>"> let t = T.GetDataContext() let passedCandidates = query { for student in t.StudentScoreTable do where (student.Passed ?= true) select (student.Name, student.Course) } // generate pass letter using OpenXML SDK let generatePassLetter record = let name, course = record let filePath = sprintf @".\%s.docx" name let myDoc = WordprocessingDocument.Create(filePath, WordprocessingDocumentType. Document) // Add a new main document part. let mainPart = myDoc.AddMainDocumentPart() //Create Document tree mainPart.Document <- Document() //Create Body let body = Body() //Create paragraph let createParagraph text = let paragraph = Paragraph() let run_paragraph = Run() // put text let text_paragraph = Text text //Append elements. ignore <| run_paragraph.AppendChild(text_paragraph) ignore <| paragraph.AppendChild(run_paragraph) // return paragraph paragraph //create a paragraph with text let paragraph0 = createParagraph (sprintf "Dear %s" name) let paragraph1 = createParagraph (sprintf "Congratulations! You passed %s" course) [paragraph0; paragraph1] |> List.iter (body.AppendChild>>ignore) ignore <| mainPart.Document.AppendChild(body) // Save changes mainPart.Document.Save() // close the document and save all changes myDoc.Close() // generate letters for all passing students passedCandidates |> Seq.iter generatePassLetter
You need to add a reference to DocumentFormat.OpenXml.dll, FSharp.Data.TypeProviders.dll, System.Data.dll, System.Data.Linq.dll, System.Xml.dll, and WindowsBase.dll to make the project compile.
List.iter requires that the provided function returns unit. AppendChild returns a non-unit type. By using >> and the ignore function, you can get a new function that returns unit. If you forget how to use the |>, <|, <<, and >> operators, you can refer to the “Pipeline-Forward Operator” section in Chapter 1.