9.2.1.1 Conceptualizing Domains through Ontologies

In the domain of Semantic Web, ontologies (or vocabularies) define the concepts and relationships used to describe and represent an area of concern. The role of ontologies in Semantic Web is to facilitate data organization and integration [14]. This integrated data (known as Linked Data) which can be used for reasoning or simply querying is the main strength of the Semantic Web. Most of the applications employing Semantic Web technologies are essentially based on the accessibility and integration of Linked Data at various levels of complexities. Ontologies can play a crucial role in enabling automatic knowledge processing, sharing, and reuse among applications. An ontology typically contains a hierarchy of concepts (or classes) along with their attributes (or properties) related to an area of interest. From our example regarding birthplace of Barrack Obama, we need to create (or reuse) an ontology that models such biographical content using classes such as Person, City, State and Country (to name a few) along with relationships among them. Moreover, literals such as age, salary etc. can also be modeled through ontological constructs called data (or datatype) properties and linked to respective classes.

9.2.1.2 Resource Description Framework (RDF)

An ontology is a formal conceptualization of a particular domain and is described using RDF² syntax. The Resource Description Framework (RDF) is used for expressing information about resources, their attributes and relationships with other resources. A resource can be anything, including a document, person, object or an abstract concept. RDF allows us to make statements about these resources. These statements always follow a simple structure:

<subject> <predicate> <object>

For example: A statement that shows the entities “Barrack Obama” and “Hawaii” linked through the property “birthPlace” is formally represented in RDF as:

<http://example.org/BarrackObama>
<http://schema.org/birthPlace>
<http://example.org/Hawaii>.

Moreover, we can add more context about the entities “Barrack Obama” and “Hawaii” such as:

PREFIX rdf:
<http://www.w3.org/1999/02/22-rdf-syntax-ns#>

<http://example.org/BarrackObama> rdf:type
<http://schema.org/Person>.

<http://example.org/Hawaii> rdf:type
<http://schema.org/State>.

<http://example.org/BarrackObama>
<http://schema.org/name>
"Barrack Obama".

<http://example.org/Hawaii> <http://schema.org/name>
"Hawaii".

In RDF subjects, predicates and objects are represented using URIs (Uniform Resource Identifiers). URIs are used to identify resources irrespective of their location on the web. Readers interested in understanding more about Semantic Web and its constituent concepts should consult some of the introductory material such as [13] and [15].

9.2.2.1 IoT Data Modeling and Integration

One of the key visions of Semantic Web is to model information on the Web so that it can be automatically understood, organized and retrieved by machines. The IoT devices exchange data with each other and with users on the Web. In order to facilitate automated interactions among devices and users, some context is needed to be built around data. In the domain of Semantic Web, context is provided through use of ontologies. Ontologies encode domain knowledge in terms of concepts (or classes), attributes (or properties) and rules to model a given domain. Using ontologies, we can model how data is being generated, how it interacts with other sources and how metadata and other attributes are managed. Ontologies can facilitate identification of different types of data streams. In essence, ontologies are reference maps that are used to model similar entities across domains, promoting standardization. Moreover, heterogeneous data from multiple sources can be integrated together to create complete view of an environment or an entity (e.g. energy consumption, temperature and occupancy data can be used to characterize a building). In an effort to standardize publishing of ontologies and data, W3C (World Wide Web Consortium) has provided some guidelines [16] which are summarized here:

• Publish the domain knowledge (ontology, dataset and rules) online.

• Write labels or comments at least in English.

• Dereferencing URIs: If a URI does not point to a document, then it should point to an ontology (not to 404 error page).

• Use semantic validation tools to fix errors in the ontology.

In order to promote standardization, there needs to be an agreement among the different stakeholders over ontological definitions which, in real world, is seldom the case due to the decentralized design of the Web. For example, museums around the world have built databases about the artworks they host using different schemas without consideration for interoperability with other museums [17]. If, for instance, each of these museums model similar entities such as a Painting or an Artist using classes from independently developed ontologies, it will be difficult to query paintings by the same artist across museums without first performing the process of ontology alignment [18] and record linkage [19]. In Section 9.3.1, we discuss more about how ontologies can be used to model and integrate heterogeneous data.

9.2.2.2 IoT Resources Search and Discovery

As more and more IoT data become available through web-oriented services, it becomes difficult to manually keep track of and locate relevant services. Semantic annotation of IoT resources and services are key to supporting the search and discovery of such resources. In Section 9.3.2, we discuss different approaches of semantic modeling of web services and linking to LOD (Linked Open Data).

9.2.2.3 IoT Data Querying

Once the data has been modeled through ontologies, the users require knowledge of Semantic Web and query languages to access it. In the literature, several approaches have been proposed that allow non-expert users to access information without having the pre-requisite knowledge of Semantic Web principles. In Section 9.3.3, we present multiple techniques to facilitate querying over semantic repository.

9.3.1.1 Semantic Sensor Network (SSN) Ontology and Sensor Cloud Ontology (SCO)

Semantic Sensor Network (SSN) ontology [28], shown in Fig. 9.1, focuses on domain independent sensing applications by integrating sensor data (measurements) and sensor specific data (sensing principles, quality etc.). The ontology provides a number of concepts and attributes to model sensors and sensor observations and measurements. The ontology can describe sensors, accuracy, ranges, frequency, units and hierarchies of sensors etc. Users can derive their own custom domain ontologies by creating subclasses of classes described by SSN ontology. Another related ontology is the Sensor Cloud Ontology (SCO) [21] which extends SSN ontology by adding provisions for the parameters being sensed as separate entities instead of just metadata and explicitly introduces the concept of time series. The ontology provides a unified view of data collected from multiple sensor networks and organizes sensors hierarchically i.e., Network → Platform → Sensor → Phenomenon → Observation. This means that a network can have multiple platforms. Each platform has multiple sensors attached to it and each sensor measures a particular phenomenon and the measured results are stored as observations.

Müller et al. [21] have used SCO ontology for (offline³) modeling and integration of sensor data acquired through REST API. The Commonwealth Scientific and Industrial Research Organization (CSIRO⁴) provides access to data of a large number of terrestrial and marine sensors through a Web API implemented using RESTful principles. These sensors measure parameters such as temperature, salinity and rainfall. The data is available as JSON documents. The URL of each document depicts the hierarchical organization (network, platform names etc. e.g. network1/platform1/sensor1/data.json etc.) for a given sensor. By creating instances of classes such as Network, Platform and Sensor from the SCO ontology, the hierarchical information can be mapped to corresponding concepts in RDF format. The data itself as well as the measured phenomenon (rainfall, in this case) can be modeled using ObservedPhenomenon, ObservationResult and TimeSeriesObservedValue. This allows to create a single linked RDF graph from a set of JSON documents acquired through REST API. Once the data is linked, users can extract relevant data by issuing SPARQL queries like the one, shown in Box 9.1, that retrieves a list of sensors and their observations along with observation quality information.

9.3.1.2 Modeling of Time Series Data

A time series is a temporally ordered sequence of observations. Usually, the observations in the sequence have the same feature of interest and observed property. For example, time-stamped sequence of kWh values obtained from an electricity meter (sensor) forms a time series representing energy consumption (observed property) for a particular house (feature of interest for the sensor). In order to ensure that the observations that constitute a time series are populated correctly in an RDF graph, usage of OWL (Web Ontology Language) based rules have been proposed [29]. Such rules ensure that all the observations in a single time series have the restriction of having same feature of interest and the same observed property. One such example is shown here:

9.3.1.3 Dealing with Heterogeneous Data

Nowadays, data is not only being generated by sensor networks, but also a large amount of it is coming from human and machine users. A sensor observation has so far been modeled keeping in mind its numeric nature. However, due to new forms of data generating entities, data may comprise of images, videos, text (structured and unstructured), drawings etc. In such scenarios it is useful to abstract the data generating entities and focus on the data itself. Due to recent focus on data sciences, complex analytical models are commonly being applied to observed data to detect patterns, learn hidden correlations and find new facts. This derived data needs to be integrated with the raw data for querying and future analysis. SOFOS (Smart Oil Field Ontology) [22] is designed for data and event-centric modeling. Entities are viewed as data sources (including both raw data and metadata). Like SSN and SCO, discussed earlier, there is a Measurement class, but the nature of measurement is not limited to numeric data. It includes anything from an image to a descriptive observation (e.g. comments). The observed data is linked to analyzed data through inter-class relationships. Based on critical thresholds, events (e.g. low/high alarms, warning/critical alarms etc.) can be generated based on observed and derived data. The events are modeled using classes derived from PoEM (Process-oriented Event Model) [30] for event modeling. Bridging the two ontological models together results in linking of data streams, event detection and event-based goal planning, promoting the re-usability aspect of Semantic Web.

9.3.4.1 ASQFor: Automatic SPARQL Query Formulation

The main goal of ASQFor is to enable end-users to formulate queries over semantic data in terms of classes and properties while being oblivious to the actual structure of the data. To achieve that, ASQFor generates SPARQL queries in three steps. First, the user provided keywords are mapped to concepts and attributes in the ontology O. Each key in the user provided input maps to a node u in O. Second, the algorithm determines the minimum number of nodes and edges from O needed to connect all the nodes u to form the query subgraph Q. A semantic relationship between two nodes (or keys from the input) can be simple (i.e., a single triple) or complex (i.e., represented by a path). To construct the semantic query graph Q, the lowest common ancestor r of all the mapped nodes u is computed. This step is necessary to establish the smallest set of relationships between concepts and attributes in the query that lie on different branches of O. Then the paths that connect all nodes u to r are traced (r is the root of the query subgraph Q). The SPARQL statements are generated while traversing these paths to r by populating statements that correspond to semantic relations and intermediate nodes at each step. Finally, the SPARQL query is executed on the semantic repository and results are returned to the user.

9.3.4.2 Semantic Search of 1990 US Census Data

To show how ASQFor can be incorporated into a semantic search system, we developed a simple user interface (Fig. 9.3) that supports a combination of semantic search and exploration queries. From the user's perspective, he only needs to know what kind of information is available in the database irrespective of how it is organized. This has led to our minimalist design which allows users to pick and choose the concepts that are relevant for the query, specify filtering values and get the desired results. After selecting the required attributes, the user can optionally specify filtering values. The filtering values can be entered concatenated with comparison operators e.g. <500. The results of the query are returned in CSV format. The selection and filtering values in Fig. 9.3 correspond to a query asking for people with more than 16 years of education who are employed and making more than $100,000/yr. Unlike other visual query interfaces, our primary focus is to abstract the details of SPARQL and schema ontologies from the end users, providing them only the data attributes to choose from. Furthermore, this interface can be dynamically generated from a schema ontology, resulting in a portable application that only requires access to the semantic repository (which must contain schema ontology along with data) to build a functional-to-SPARQL query translator and a GUI on the fly.

9.4.1.1 Semantic Model of Asset Integrity Data

Semantic Web technologies can be used for an expressive representation of various heterogeneous data sources to deal with aforementioned challenges. First of all, the data streams need to be modeled using appropriate ontological models that capture the relevant domain knowledge, as is done through SOFOS (Smart Oilfield Ontology) and ECD (External Corrosion Detection) ontologies in [22]. The ontologies capture physical entities and their inter-relationships as well as the associated observed data, metadata and derived data. Once all the entities in the data streams have been identified, the next step is to perform record linkage across data streams. Different types of data may be recorded for same assets in different databases. The key idea is to integrate and present a unified view of the environment. Fig. 9.4 shows data organized as an integrated RDF graph where raw data was originally stored as multiple CSV files, databases, images and text files. Data from multiple sources is integrated into a central repository serving as a single endpoint for maintaining and retrieving knowledge. Asset integrity managers can query previously separate databases to issue meaningful queries e.g. all work orders for assets that have been labeled severely corroded in the most recent survey. This query requires data from work orders database and inspection database, which have now been linked together after the integration process. This way data from disjoint repositories can be combined to provide actionable information. This is, essentially, an end goal for any enterprise to not just store and manage vast amounts of data, but also get actionable insights faster for robust decision making process. Other automatic and semi-automatic approaches of using ontologies for organizing data and metadata into semantic repositories are discussed in [56] and [57].

9.4.1.2 Accessing Integrated Information

Once we have the integrated repository, the data become available for querying. However, non-expert users require IT experts to build applications or create queries to be used. Our definition of a non-expert user is someone who is not familiar with the concepts of databases, querying, Semantic Web and ontologies but is an expert in his area of specialization e.g. an asset integrity manager or a production engineer. To do his job that person only requires access to data irrespective of underlying techniques of data organization and retrieval. For such users, ASQFor algorithm (discussed in Section 9.3.4.1) aims to abstract Semantic Web concepts (such as ontologies, RDF, SPARQL) and allows them to formulate queries at a higher level which the system then translates into formal SPARQL queries automatically.

9.4.2.1 Challenges

IoT devices, which form the basis of a Smart City, offer heterogeneous data. For instance, in an IoT network, the devices can be measuring speed, light, temperature, sound etc. and usually have varying degrees of sampling frequencies. Different sensors provide different forms of data at varying rates and even with varying levels of quality and availability. The applications need to be built in a way that they interact with devices that are trustworthy and discard the noisy data. In these types of large-scale highly dynamic environments, the process of discovering, modeling, integrating and efficiently querying data are complex tasks [60]. Various aspects of Semantic Web technologies (discussed in Section 9.3) can help to some extent, enabling valuable services towards the goal of achieving Smart Cities. One such approach is described next.

9.4.2.2 Automated Complex Event Implementation System (ACEIS)

ACEIS [61] acts as a middleware between IoT data and Smart City applications and performs real time discovery and integration of data streams automatically. The data streams are modeled using SSN ontology (discussed in Section 9.3.1.1). User queries are translated into “semantically annotated complex event service requests”. These event requests are modeled using Complex Event Service (CES) ontology, which is an extension of OWL-S.⁷ The event requests are processed by ACEIS core to determine the data streams that are to be discovered and integrated. It uses an algorithm [62] to create composition plans to detect the complex events specified in event requests. Candidate sensors are determined by querying semantically annotated sensors capabilities (modeled using ObservedProperty and FeatureOfInterest from SSN ontology). The query transformer then transforms the composition plans into set of streaming queries to be executed on a streaming query engine. This mechanism can work with actual sensors as well as virtual sensors (social media streams) to detect complex events for users.