Priyanka Chawla and Rohit Chawla
Fog computing facilitates the benefits of cloud computing by providing computing intelligence (in the form of virtualized resources), storage, and networking services to the edge of the network. This helps in decreasing latency (by reducing the need to communicate via cloud), uninterrupted services with intermittent connectivity, enhanced security, and support of massive machine communications. Thus, fog computing paradigm is a viable option for the development of IoT applications.
IoT is referred as an ubiquitous network of real‐life physical devices (such as home appliances, medical equipment, vehicles, buildings, etc.) embedded with sensors, microchips, and software to gather and exchange information through an existing Internet connection. It is a way by which computing intelligence is directly integrated to the physical entities with a motive to enhance performance, efficiency, and financial benefits. A boom in the field of Internet of Things (IoT) in almost all vertices of the industry has motivated organizations to build IoT products to meet the market demands. As per IDC reports, global expenditure on IoT will be around $1.29 trillion by 2020 [1]. Technical report by Gartner on emerging technologies states that there will be 20.4 billion connected devices by 2020 [2]. As we expand the connectivity of the IoT, scope and capabilities of IoT systems are also increasing day by day that directly affect public safety and personal lives, like medical devices and systems and automotive safety; therefore, the consequences of a system breach or network failure are higher than ever before. However, high‐velocity growth associated with rapid innovation anticipates the need of strong unique IoT testing (quality assurance) strategy to ensure the reliability of the IoT systems well before their release to the market.
Quality assurance is one of the most important phases of development to ensure the correctness and quality of developed software. Similarly, it is also crucial for IoT system, as poor design may hamper the working of the application and affects the end‐user experience. The architecture of IoT is very complex, composed of heterogeneous hardware, communication module, huge volume, and variety of data, which plays a vital role in analyzing the performance and behavior of the IoT system. Functional and nonfunctional requirements (such as robustness, reliability, security, performance, etc.) of IoT systems can only be ensured if a variety of devices are tested for different kinds of operating systems (OSs), software, and hardware combinations.
The QA process for the IoT is required to perform verification and validation of the associated new technologies such as machine learning and data‐mining with the aim of regularly improving existing and future systems. Moreover, the huge volume of data getting captured and sent through IoT devices to the backend makes the system prone to performance bottlenecks. This poses fresh challenges to development teams; thus, there is a dire need for comprehensive and advanced testing strategy to cover the breadth and depth of IoT systems.
This chapter starts with an explanation of the fundamental concepts of fog computing paradigm and associated benefits if adopted for the implementation of IoT applications.
Section deliberates testing perspectives of the smart applications in the area of home, health, and transport. Testing approaches and solutions applied so far have been illustrated and compared based on their outcomes. Further, evaluation criteria relevant to the three smart technologies viz. smart home, smart health, and smart transport have been proposed to assess the existing work. Finally, Section presents open issues and future research directions.
With the emergence of IoT applications for which low latency and location awareness are of prime concern, fog computing comes into the picture. Fog computing is a conceptual model that extends compute, network, and storage services of cloud computing to the edge of the network. The paradigm of fog computing provides a decentralized architecture and extends the methodologies and characteristics of cloud computing (such as virtualization, multitenancy etc.) to the edge of the network. Applications such as gaming, video conferencing, geo‐distributed applications (for, e.g., pipeline monitoring, sensor networks to monitor the environment), fast mobile applications (for, e.g. smart connected vehicle, connected rail), large‐scale distributed control systems (for, e.g. smart grid, connected rail, smart traffic light systems), entertainment and advertising industry benefit on a large scale with fog computing paradigm due to improvement in quality of service (QoS) and reduction in latency. In addition, the fog model is well suited for data analytics and distributed data collection points by setting up end services such as setup boxes and access points. Thus, adoption of the fog computing model for the development of IoT application is very beneficial. Some of the benefits are listed below:
In view of the above benefits of the fog computing model, an IoT application that produces high volume and velocity of data requires an extensive and dense network of devices that can take advantage of the fog computing paradigm. Examples of such applications are listed below:
This chapter discusses testing perspectives of three case studies viz. smart home, smart health, and smart transport, along with their limitations and future research directions. The reason behind this selection is that these three applications can be considered as the main founding needs of society. Agriculture is also one of the most important fundamental needs of society, and making it smart with the adoption of high‐end technology would greatly contribute to worldwide growth and prosperity. But due to time and space constraints, we will not describe this use of smart technology; it will be taken up in a future study.
In the era of a smart technology enabled environment, devices must interact with other devices or even human beings with the purpose to share system configuration. This may hamper the working of the application and may affect the end‐user experience. Hence, the software, being the soul of the smart system, must be reliable and robust, which can only be ensured by effectively testing the software. The testing perspectives and the approaches adopted by the industry and academia for various smart systems are presented in this section.
NTS is one of the testing service providers that provides validation of home area network (HAN) devices such as smart meters, smart door locks, light controls, thermostats, and smoke sensors. It tests the interoperability of appliances and reflects the energy consumption by various devices and thus helps in an effective energy management 3, 4. The testing tool also supports clients with self‐testing by simulating the functionality of the appliances. NTS has been designated by ZigBee Alliance to test wireless products for smart energy, and ZigBee Smart Energy is nominated by the US Department of Energy and the National Institute of Standards and Technology (NIST) as an initial interoperable standard for HAN. NTS also works for iControl Platforms to test its security and home automation commodities such as smart door locks, light controls, thermostats, and smoke sensors.
Corporate major players in the field of mobile phones manufacturing (such as Apple and MI) also provide smart home applications that help in the attainment of security, effective energy management, and automated detection of smoke or gas through mobile phone applications. Security of smart homes is ensured by setting up security sensor systems for doors and windows. Smoke or gas detectors can be turned on through mobile applications. In a similar way, smart light schedule and brightness can also be remotely controlled. Allion Smart Home provides testing and validation services, which support clients in the development, testing, and debugging of products for the three most important smart home environments – named as Cloud Service/Data Exchange,” “UI/APP,” and “End User Device” [5]. The lab established at Allion simulates a real home environment with three bedrooms, two living rooms, and two bathrooms including home items such as a sofa, TV cabinets, beds, desks, wardrobes, and so forth. Common appliances and electronics, such as a television, wireless speakers, computers (desktop and laptop), wireless LED lights, and so on have been installed in compartments with powerline wireless extenders, one‐in‐three wireless phones, and a microwave in the kitchen to introduce interference from other electrical products in the 2.4 GHz band. This is done to simulate behavior patterns and user habits in the real world [5].
eInfochips carries out performance testing for iOS and Android apps and redesigned the UI for Android and iOS platforms to improve the performance of home devices and to avoid inconsistency between iOS and Android platforms. Application response time is measured by using 24 × 7 performance evaluation tools and carries out bottleneck analysis to identify performance inefficiencies with the help of data flows and log files. Performance optimization techniques are implemented using cost–benefit analysis. Crash issues are resolved by utilizing detailed analysis and creating a crash log review. Code analysis is done with the help of SonarQube and XClarify tools [6].
UL has established the UL living lab in a 2500‐square‐foot fully furnished home situated near Silicon Valley campus and it thus enables testing of smart home devices in real‐world user scenarios and provides various benefits such as ecosystem integration, large‐scale interoperability, RF performance, and audio quality [7].
TUV is the third‐party testing provider that tests smart home products to ensure privacy of the data as per the guidelines of data protection regulations. Various types of tests such as device default settings, local communication testing for encrypted data, interoperability testing etc. are carried out to test the effectiveness of privacy of user data. Smart home devices are tested to certify their functionality and mechanical and electrical safety by testing the products such as motion sensors and smoke alarms. In addition, usability tests are also carried out for the smart home devices [8].
Smart Home Test platform established at VDE Institute conducts tests to evaluate and certify smart home network devices for compliance, faultless functionality, user data protection and interoperability [9].
The National Renewable Energy Laboratory (NREL) has devised a smart home test bed to simulate power distribution grid for industry, manufacturers, universities, and other government organizations. The NREL test bed includes the combination of powered hardware and software simulations. The smart home hardware comprises electric vehicle supply equipment (EVSE), home loads, a water heater, a thermostat, and an air conditioner, all powered (via red lines) by a photovoltaic inverter and an alternating current (AC) power amplifier, which emulates grid power. A high‐performance computer (HPC), Peregrine, has been utilized to execute advanced home energy management system (HEMS) optimization algorithms that simulates power distribution feeder, also uses weather and price data to determine control signals sent to simulated homes and to the smart home's hardware via the HEMS. The key component of a smart home test bed is a co‐simulation tool, integrated energy system model (IESM) that is responsible for managing the power system and home simulations, the HEMS algorithms, communications with the HEMS hardware, and a simulation of the smart home (using EnergyPlus) that runs on the hardware‐in‐the‐loop (HIL) control computer in the laboratory. The IESM also provides price signals as inputs to the HEMS, allowing users to evaluate how smart home technologies respond to different retail price structures [10]. Zipperer et al. [11] have also worked in this direction and developed a mechanism for electric energy management in the smart home. Cordopatri et al. [12] established test lab in the campus of the University of Calabria to experiment with various management systems for smart homes such as energy flow and comfort management systems. The main objective of the energy and comfort management system (ECMS) developed at the University of Calabria is to attain reduction in the cost and usage of energy along with improved comfort and safety of the smart home systems. Several authors have proposed similar kind of frameworks based on fuzzy logic, neural networks, and genetic algorithms [13–16]. Hu et al. [17] developed an open and smart home test bed named as SHEMS that can be used for educational purposes. The summary of these products is depicted in Table 15.1.
Table 15.1 Outline of the work done to test smart homes.
Authors/Company | Objective | Approach | Outcome |
National Technical Systems (NTS)[2, 3] | ZigBee Smart Energy Certification Testing for SimpleHomeNet Appliance |
|
Smart energy device testing; increase reliability and cut costs for consumers |
Allion Smart Home Testing Services [5] | Hardware development support, software apps validation and user experience optimization, cloud service validation, RF signal and interference validation and interoperability testing. | Allion carries functional testing and ensures that the products meet the specification and verification standards of the certification process; The lab established at Allion simulates a real home environment that includes simulation of users' habits and behavior patterns; Carries out testing for different products and test scenarios. | Certifies all 18 Wi‐Fi certification services |
eInfochops [6] | Performance testing; reliability and usability testing | SonarQube and XClarify tools are used for code analysis; Performance of the app is determined by using gap analysis between technical requirements and actual expected performance of the mobile app. Performance inefficiencies are resolved using bottleneck analysis. Mobile performance optimization techniques are realized using cost–benefit analysis. Resolution of crashes is done using detailed analysis. |
Mobile app performance optimization Mobile UI redesign Code review and performance testing expertise Better app reliability |
TUV Smart Home Testing and Certification [8] | Security, protected privacy and testing for user friendliness |
|
Certification named as Certipedia and Greater Transparency |
UL Living Lab[7] | Interoperability testing | 2500‐square‐foot fully furnished home to test products in a real home and in a real neighborhood | Testing real‐world user scenarios: out of the box experience; Physical installation; ecosystem integration; large‐scale interoperability; audio quality and RF performance |
VDE Smart Home Test Platform [9] | Interoperability, information security, functional safety, and data protection |
|
Conformity assessment; Certification Program" funded by Federal Ministry of Economics and Technology (BMWi) |
NREL Smart Home TestBed [10] | Energy‐efficiency testing |
|
Controllable, flexible, and fully integrated smart home test bed |
Zipperer et al. [11] | Electric energy management |
|
|
A. Cordopatri et al.[12] | Energy and comfort management system (ECMS) |
|
|
I. Dounis et al.[13] | Multi‐agent control system (MACS) | TRNSYS/MATLAB |
|
R. Baos et al. [14] | Review of the current state of the art in computational optimization methods applied to renewable and sustainable energy | Well‐defined visualization of the modern research advancements | |
J‐J. Wang et al. [15] | Review of multi‐criteria decision analysis (MCDA) methods | Energy decision‐making computed by the combination of weighted sum, priority setting, outranking, and fuzzy set methodology | Identification of MCDA method, and the aggregation methods for sustainable energy decision‐making |
T. Teich et al. [16] | Energy‐efficient smart home | Neural networks | Energy saving |
Q. Hu et al.[17] | Open and extensible model for energy conservation based on smart grids | Machine learning and pattern recognition algorithms | Smart home test bed named as SHEMS developed that can be used for educational purposes |
The main objective of the healthcare industry is to provide patients with quality healing services round the clock in a cost‐effective manner. The software industry enables the smooth functioning of the healthcare industry by providing software applications that assist in the functioning of various hospital operations and at the same also maintains the privacy of patients. Hence, crashing of an application would severely impact healthcare process and may also adversely affect the health of the patient. Therefore, testing of healthcare software is very essential as it ensures the quality and productivity of a healthcare service. The healthcare industry needs to follow strict regulatory and compliance norms, and it is bound to identify novel revenue generation strategies and to effectively utilize R&D budgets. This raises the need of software professionals to have thorough understanding of domain and industry regulations and standards. Significant work done in this direction is explained below and is portrayed in Table 15.2.
Table 15.2 Outline of the work done to test smart health.
Authors/Company | Objective | Approach | Outcome |
Virtusa COE [18] | Healthcare domain testing, user acceptance testing (UAT) optimization, ICD‐10 testing, and enterprise end‐to‐end testing | Business process management, customer experience management, enterprise information management, cloud, mobility, SAP |
|
MindfireSolution [19] | Conformance testing, interoperability testing, functional testing, security testing, platform testing, load and performance testing, system integration and interface testing and enterprise workflow testing | QTP, Selenium, Appium, and Robotium over several platforms |
|
QA Infotech [20] | Functional testing, database testing, performance testing, content QA testing and evelopment and implementation of QA and test strategies, performance and security tests |
|
Assurance of Security, privacy and mandated compliances in healthcare application tested by QAInfotech |
ALTEN Calsoft Labs' [21] |
|
|
|
Precise Testing Solution [22] |
Healthcare application testing in the domain of electronic medical records, patient survey solutions, quality and compliance solutions, enterprise content management, medical equipment software solution and compliance testing services | JMeter for load testing, ZAP proxy | Bugfree software |
ZenQ [23] | Functional/regression testing, usability testing, interoperability testing, mobile apps testing, conformance/certification testing, performance testing and security testing |
|
Assurance of quality, patient‐centric care, high efficiency and cost‐effectiveness
|
Testree [24] | Functional testing, integration testing, interoperability testing, security testing, device compatibility testing, selection of manual or automation testing methods, performance testing like load testing and scalability and compliance testing |
|
Comprehensive quality assurance
|
KiwiQA [25] | Compliance conformance testing, product consistency testing, platform testing and security testing | Test approach is
|
|
XBOSoft [26] |
|
|
|
Infoicon Technologies [27] | Interoperability testing, functional testing, security testing, load and performance testing, system integration testing and acceptance testing |
|
|
W3Softech [28] | Testing and QA services for healthcare and pharmaceuticals industry such as claims management testing, clinical decision support system (CDSS),healthcare billing software testing Personal health record and e‐prescribing,implanted application testingQA in clinical data management systems CRO workflow management system,testing support for regulatory requirements |
|
|
Prova [29] | Manual testing, PLM testing, and automation testing | Automation testing
|
Better quality products and services
|
Calpion [30] |
|
HP quality center (QC), Quick Test Professional (QTP) and HP ALM
|
|
Abstracta [31] | Automated functional testing, security testing and performance testing services |
|
|
360logica labs [32] |
|
|
|
Renate Löffler et al. [35] | Model‐based test‐case generation strategy | UML 2.0 | Developed model‐based approach for the specification of requirements followed by integration testing for healthcare applications |
Bastien et al. [36] | User‐based evaluation | KALDI, Morae, Noldus | Identification of open issues in usability testing |
R. Snelick [33] | Conformance testing | NIST HL7 v2 conformance test tools | Certification of EHR technologies |
P. Scott et al. [34] | Conformance testing | Schematron, mind‐mapping | Developed an openEHR archetype model for creating HL7 and IHE implementation artifacts |
Virtusa has established dedicated center of excellence that provides healthcare domain testing, user acceptance testing (UAT) optimization, ICD‐10 testing, and enterprise end‐to‐end testing [18]. Mindfiresolutions provides a manual as well as automated healthcare application testing services by using various tools such as QTP, Selenium, Appium, and Robotium over several platforms. The testing services offered are: conformance testing, interoperability testing, functional testing, security testing, platform testing, load and performance testing, system integration and interface testing, and enterprise workflow testing [19]. The healthcare testing services provided by QAInfotech include functional testing, database testing, performance testing, content QA testing and development and implementation of QA and test strategies. In addition, testing professionals also take care of HIPAA guidelines and carries out performance and security tests [20]. Cloud lab established by ALTEN Calsoft Labs' provides healthcare domain testing in the area of clinical systems, nonclinical systems, and specialized testing services. Clinical systems include EHR/EMR, hospital ERP, radiology information systems, imaging systems, and compliance‐related standards and guidelines such as HIPAA. Nonclinical system contains the modules of pharmacy, billing, and revenue cycle management. Specialized testing services comprise compatibility and localization, security testing, performance testing, legacy modernization and testing, mobile healthcare, BI/analytics, and cloud migration and testing [21]. Precise Testing Solution delivers healthcare application testing in the domain of electronic medical records, patient survey solutions, quality and compliance solutions, enterprise content management, medical equipment software solution and compliance testing services [22].
ZenQ helps healthcare organizations in attaining quality, efficiency, and cost‐effectiveness by providing specialized healthcare testing solutions in the area of electronic health records (EHRs) electronic medical records (EMRs), hospital management systems, healthcare data interoperability and messaging standards conformation, and mobile health. Testing services include functional/regression testing, usability testing, interoperability testing, mobile apps testing, conformance/certification testing, performance testing and security testing [23]. Testree offers a complete package of quality assurance and healthcare application testing that includes certification for automatic compliance of various standards, appropriate administration, and control of policy claims and benefits, patient and disease management, billing and reporting, etc. [24]. The healthcare testing services offered by KiwiQA encompass compliance conformance testing, product consistency testing, platform testing, and security testing [25].
XBOSoft makes the provision of testing services in the domain of healthcare and ensures the compliant working of electronic health records (EHR), automated drug dispensing machines, pharmacy management, EMAR, and EPCS with mobile apps. This is done by careful design of test cases that ensures test coverage, cross‐platform, multidevice, and multibrowser compatibility [26]. The lab setup at Infoicon Technologies Pvt. Ltd. dedicatedly provides the cost‐effective healthcare testing services covering the domain of pharmaceutical industry, clinical systems, healthcare startups, body fitness, dental care, physiotherapy, doctor consultation, and homeopathy. It provides multiple platforms for manual as well as automated testing services that include interoperability testing, functional testing, security testing, load and performance testing, system integration testing, and acceptance testing [27]. W3Softech offers agile‐based healthcare and pharmaceutical testing services [28].
In the similar way, Prova also provides cost‐effective software testing and QA services for the healthcare industry [29]. Calpion's offers convenient and fast‐testing framework that works for both web and mobile healthcare application by utilizing HP quality center (QC), quick test professional (QTP) and HP ALM [30]. Abstracta provides healthcare testing system for patient portals, medical imaging, and electronic health records (EHR) while adhering to the standards and regulations. It provides automated functional testing, security testing, and performance testing services [31]. The 360logica labs offers cost‐effective, reliable, and standard compliant healthcare software testing services. The testing services are in the area of hospitals, pharmaceutical and clinical labs, which include healthcare billing software testing, R&D software testing, and embedded application testing [32].
Löffler et al. [35] devised a model‐based test‐case generation strategy from use case scenarios described with their newly introduced formal specification language by extending UML2.0 sequence diagrams. Test models have been derived from specifications, which are then used to generate test cases corresponding to each and every flow in the test model. J.M.C. Bastien et al. [36] carried out user‐based evaluation for healthcare applications to assess the usability of the application by employing single user and paired‐user testing. In this approach, users are asked to carry out certain tasks, and performance of the users is noted such as task completion rate, types of error accorded, etc. to recognize certain design flaws that causes user errors. Based on these observations, design changes can be suggested to front‐end designers. Snelick [33] investigated conformance testing and the tools that are used to perform HL7 (Health Level Seven) v2‐based conformance testing for certification of EHR technologies. Scott et al. [34] demonstrated the development of conformance methods based on the professional standards. Table 15.2 summarizes work done in smart health.
The researchers at UMTRI carry out development and testing of intelligent transportation systems off‐the‐road to prevent collisions in passenger vehicles. Exhaustive study is carried out in the direction of automotive collision avoidance, in‐vehicle driver‐assistance and safety systems, and integrated technologies between the vehicle and the infrastructure [38]. Connected Vehicle Test Bed has been established in Michigan, Virginia, Florida, California, New York, and Arizona to facilitate a real environment where intersections, roadways, and vehicles are able to communicate through wireless connectivity by the US Department of Transportation (USDOT), and it comprises of a network of 50 roadside equipment (RSE) units installed along various segments of live interstate roadways, arterials, and signalized and unsignalized intersections, in Novi, Michigan. These RSEs communicate messages over 5.9 Ghz dedicated short‐range communication (DSRC). This test bed provisions testing of new hardware and software for the evolution in connected vehicle technology. Various types of tests (such as signal phase and timing (SPaT) communications; security system operations; and other connected vehicle applications, concepts, and equipment) can be successfully carried out for free. In addition, there is a provision of experts to carry out complex scenario tests. Also, there is no need to make any testing arrangements because of prior contracts between the local agencies and roadway operators. Test beds frequently undergo upgrades and enrichments to provision the changing requirements of users. Clients of Connected Vehicle Test Bed include Denso, Delphi, Hirschmann, Eaton, Argenia, Wayne State University, MET Labs, Ricardo, and University of North Texas [39].
The test lab instituted at IBS provides end‐to‐end software testing services to travel, transportation and logistics enterprises. It provides four types of testing services which includes Enterprise QA Automation Services, Product Acceptance Test Services, Managed Testing Services and NFR Testing services. Enterprise QA Automation Services provides automation to support DevOps environment, process automation to validate build to release quality, reusable frameworks for TTL customers and transformational models to support guaranteed outcome, Product Acceptance Test Services involves system Integration, final acceptance and UAT support, domain experts to validate business requirements, reusable assets for TDM (Test Data Management), automation, and performance and multivendor management for airlines' IT solutions testing. Managed Testing Services comprises consulting services for outsourcing, transition from incumbent vendors/captive organization, end‐to‐end testing from functional to acceptance test, and assured output/outcome model for delivery. NFR Testing services consist of performance benchmarking and capacity planning, SMAC, usability, security, performance covered, projects supported with dedicated lab facility and compliance, industry standards and frameworks in mobility and multitenancy/cloud [40].
ETSI worked in collaboration with Telecom Italia, ERTICO, the regional government, local highway authorities and port authority to launch the ITS test bed in Livorno. The test bed contains traffic lights, IoT sensors, cameras, variable message signs, and connectivity with a highway control center. RSUs and on‐board units within vehicles can be tested effectively by deploying it in the road sideways. Other ITS testing activities such as traffic sign violation, road hazards, intersections and collision warnings, and loading zones can also be carried out successfully [41].
Woo et al. [42] have designed a test bed to handle testing on various ITS and advanced driver assistance system (ADAS) technologies, such as adaptive cruise control (ACC), lane departure warning system (LDWS), cooperative intersection warning system, as well as rollover stability control (RSC) and electronic stability control (ESC). The test bed has been devised to meet the requirements of ISO/TC204 standards. The test bed for ITS encompass three tracks named as ITS high‐speed track, Cooperative vehicle‐infra test intersections, and Special test track. The main purpose of ITS high‐speed track is to test performance of ACC, LDWS, LKAS, etc. It has three lanes of high‐speed track of length equal to 1,360 m with maximum allowable speed of 204 km/h. The total length of Cooperative vehicle‐infra test intersections is 1,200 m and there are three intersections. The main objective is to test pedestrian protection and intersection safety. Special test track comprises of four lanes of test road with the total area of 490 × 35 m. It includes Belgian road, washboard road, cobblestone road, water splash shower tunnel, for example. Durability and reliability test are carried out it these tracks.
The government of Estonia plans to restructure its public transportation system by adopting autonomous vehicles and thus legalized testing of autonomous vehicles on national and local roads of the country. Rigorous efforts are put into developing a cyber‐risk management framework for autonomous vehicles in regular road and traffic conditions. The government has planned to create a fleet management system and integration of vehicles into the public transport system and the implementation of call‐to‐order bus stops [44].
Transit Windsor provides the development as well as testing services for intelligent transportation systems. The company has produced 10 buses furnished with an efficient, safer, and more user‐friendly system. It provides onboard voice and visual announcements on the display boards for the upcoming bus stop messages. It also stipulates real‐time Transit Windsor bus arrival information as well as route for the bus progress via the Internet [45].
Siphen has achieved an intelligent transportation system (ITS) product compliance with UBS II and ARAI testing. It is known for its rigorous testing procedure. It is working with the government of India to equip the country with ITS by providing 24 × 7 bus operation service with the features such as automatic vehicle location, vehicle health monitoring, and diagnostics. In addition, it is carrying out end‐to‐end testing as well as a certification process according to the timelines given by government authorities [46].
Anritsu provides ITS solutions for V2X, testing, and manufacturing in a very efficient manner with reduced test time and test cycles. Testing solutions are provided with the help of four components: MD8475A Signalling Tester, MS2830A Spectrum Analyzer, MS269xA series, and V2X 802.11p Message Evaluation Software. MD8475A Signalling Tester is similar in that it supports cellular as well as M2M standards. The services supported are eCall, IMS, VoLTE, WLAN off‐load tests, and call‐processing tests for vehicles. The testing tasks are easy, fast, and reliable due to GUI‐based SmartStudio software and supplied test sequences for automatic remote control of the GUI. Multimode terminals and all cellular standards, such as LTE (2×2 MIMO) and LTE‐Advanced (Carrier Aggregation) are well supported. SmartStudio GUI provisions easy setup of test environments and functional tests. It also carries out automated mobile terminal verification testing with the available test sequences. MS2830A Spectrum Analyzer is used for testing of 2G, 3G, LTE, and LTE‐Advanced signals on a vehicle‐to‐vehicle or vehicle‐to‐x test environment. To improve the product quality, capture and replay functions are compared with the real‐world effects with simulated designs and performance. The supported frequency range is 9 kHz to 26.5 GHz/43. MS269xA series units contain swept spectrum analysis, FFT signal analysis, and a precision digitizer function and are the latest high‐performance signal analyzers for next‐generation communication applications. It has One‐Box Tester with the addition of the signal generator option. Due to the support of batch capture measurements, analysis time gets faster [47].
Penta Security Systems has launched secured smart transportation with the secure data solution AutoCrypt, implemented on the connected vehicles in the three cities of South Korea. It has also established the second‐largest test bed named as K‐City to test and certify autonomous cars. Public key infrastructure and V2X security system has been implemented to ensure secure and encrypted communication between vehicle‐to‐vehicle and vehicle‐to‐infrastructure, as well as the security and encryption of roadside units [43].
Simulation‐based test bed has been developed at Georgia Institute of Technology by the School of Civil and Environmental Engineering, which can be used for fast assessment and incorporation of sensor and actuator systems in ITS. The test can also be used to study and examine various data networks architectural possibilities to support ITS applications. The test bed supports integrated parallel simulation ability and also involves interoperable simulations of transportation infrastructures, wired and wireless communication networks, and distributed computing applications. In addition, it possesses emulation ability that allows conducting live experiments with prototype hardware and software embedded into virtual transportation systems. The test bed incorporates data generated from sensors embedded in the vehicles (such as location, velocity, and acceleration etc.) functioning in the Atlanta metropolitan area. This data are also used for modeling and scenario development as well as validation of simulations [37]. The above stated work is summarized in Table 15.3.
Table 15.3 Outline of the work done to test smart transport.
Authors/Company | Objective | Approach | Outcome |
UMTRI[38] |
|
|
Vehicle safety |
US DOT Connected Vehicle Test Bed [39] |
|
The Test Bed operates as per the guidelines of latest IEEE 1609/802 and SAE J2735 standards
|
|
IBS Lab [40] |
|
|
Emphasis on the quality deliverables
|
ETSI Test Bed [41] | Testing activities such as traffic sign violation, road hazards, intersections and collision warnings, and loading zones |
|
Compliance with ETSI's ITS Release 1 standard and interoperability with radio equipment |
JW Woo et al. [42] |
|
|
As per the requirements of ISO/TC204 standards |
E‐Estonia [44] | To restructure the public transportation system using autonomous vehicles |
|
Provision of legal and cyber‐risk management framework for testing fully autonomous vehicles in regular road and traffic conditions |
Transit Windsor Testing Solutions [45] |
|
Vocal announcements are in synchronization with the messages displayed on the display signs inside the bus. |
|
Siphen [46] |
|
|
|
Anritsu Test Bed [47] |
Functional testing; mobile terminal verification testing; testing of 2G, 3G, LTE, and LTE‐advanced signals on a vehicle‐to‐vehicle or vehicle‐to‐x test environment |
Four components:MD8475A Signalling Tester, MS2830A Spectrum Analyzer, MS269xA series and V2X 802.11p Message Evaluation Software; GUI‐based SmartStudio software |
Helped in making testing of ITS systems convenient, reliable, and efficient |
Penta Security Systems K‐City Testbed [43] |
To carry out testing and certification of autonomous cars | AutoCrypt; Public key infrastructure and V2X security system | Reliable and secure system of ITS |
Georgia Institute of Technology [37] | Prompt assessment and assimilation of sensor and actuator systems in ITS |
|
Framework can be used for investigation and assessment of new mechanisms under virtual operating conditions before actual deployment in real environment of intelligent transportation systems (ITS) |
This section discusses the open issues and research directions from the perspectives of testing and future enhancements for smart technologies viz. smart home, smart health, and smart transport. Certain evaluation criteria have also been presented for the assessment of existing work and to ascertain limitations and research directions.
We have proposed the following set of criteria to evaluate the existing work in smart home test beds. The pertinence of these criteria is described in this section:
Based on the evaluation criteria already described, limitations and research directions as well as suggestions for the future work have been elucidated here and portrayed in Table 15.4. The first hindrance in acceptance of the smart home technology is that smart homes are vulnerable to hacking. Hence, a test bed should be established that takes into account cyber‐security measures to protect the smart home. The second hindrance is the high cost; measures should be taken to develop a technology that can be made available to users at a lower cost. Combinatorial testing strategy can be used to ensure the low price suggested by the pricing model that supports the pooling of distributed, dispersed resources in fog computing and the IoT. The third hindrance is the learning curve for non‐tech‐savvy people with smart home. Hence, usability testing should also be given topmost priority. Another most important factor that hinders the acceptance is lack of industry standardization, as use of proprietary technology can cause problems smart home users. Hence, conformance testing should also be given priority. Dependency on Internet connection should also be tackled, and reliability testing methodology specifically designed to suit the environment of the smart system need to be addressed.
Table 15.4 Summary of limitations and research directions for smart home.
Criteria | Research direction | Work | Limitations | Suggestions |
Energy efficiency testing | Verify reduction in energy consumption in the smart homes. | [3, 10, 11, 12, 13, 16, 17] |
|
Test beds are required to be established for the following objectives:
|
Reliability testing | Ensure the stability of the system under various specific tests. | [3] | ||
Functionality testing | Verify each function of the software application in conformance with the requirement specification. | [3, 5, 6, 8, 9, 10] | ||
Interoperability testing | Ensure interoperability among devices. | [3, 5, 6, 8, 9, 10, 7] | ||
Performance testing | Ensure software applications will perform well under their expected workload. | [5, 6] | ||
Usability testing | Evaluate a product or service by testing it with representative users. | [6, 8] | ||
Security testing | Check whether the application or the product is secured. | [8, 9] |
To assess the existing smart health test beds, the following set of criteria have been suggested:
Evaluation criteria described above helped in deducing the limitations and research directions in the existing smart health testing solutions. The same have been illustrated in this section and depicted in Table 15.5, along with research suggestions for the future work.
Table 15.5 Summary of limitations and research directions for smart health.
Criteria | Research directions | Work | Limitations | Suggestions |
Conformance testing | Ensure adherence to the standards. | [18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 30, 31, 32, 34, 35] |
|
Test beds need to be developed to carry out research in the following areas:
|
Platform Testing | Ensure application runs across all platforms. | [19, 25, 26] | ||
Interoperability testing | Assess whether applications (or software systems) can communicate with one another effectively and correctly. | [19, 21, 23, 24, 27, 30] | ||
Functionality testing | Verify each function of the software application in conformance with the requirement specification. | [18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32] | ||
Enterprise workflow testing | Check whether the expected activities are executed and workflow data properties have correct value. | [18, 19, 21, 22, 23, 26, 27, 28, 30, 31, 32] | ||
Performance testing | Ensure software applications will perform well under their expected workload. | [18, 19, 20, 21, 23, 24, 27, 28, 29, 30, 31] | ||
Usability testing | Evaluate a product or service by testing it with representative users. | [18, 19, 23, 26, 27, 36] | ||
Security testing | Check whether the application or the product is secured. | [19, 20, 21, 23, 24, 25, 26, 27, 31] | ||
Mobile app testing | Ensure applications worked well for handheld devices. | [21, 23, 27, 28, 29, 30, 31] |
It has been implied that there is a lack of effective methodology that provides a systematic way to manage the data collected from various wearable devices. To combat this challenge, big‐data, machine learning, and AI can be used. To ensure the attainment of the mentioned functionality, a test bed is required that executes blockchain‐based repeatable tests with massive data received from wearable devices such as smart watches, eyeglass displays, and electroluminescent clothing, for example.
Further, it has been found that despite the various benefits of smart healthcare, it is not well adopted and market growth is restrained. It may be due to the high cost of IoT infrastructure and data privacy and security apprehensions. This can be resolved by building confidence among various stakeholders, which can be brought into practice by executing security tests specifically designed to examine the cybersecurity measures taken up to address the above‐mentioned issue.
Another challenge is the management of connected devices and a lack of interoperability with EHR systems. This can be ensured by executing context‐aware testing techniques. Thus, context‐aware test case generation methodologies need to be worked out for smart health systems. To address the limitations associated with smart glasses (i.e. short battery life and inability to understand the medical terms of doctors by voice‐control system), context‐aware test data generation must be applied to ensure that the system will work. Blockchain technology should be reconnoitered to solve the problems of large‐scale data sharing, ensuring data privacy and security and transparency between patient and doctors and between various healthcare providers. In this case, blockchain‐based repeatable regression tests can be employed for assurance of the privacy and security of data shared between doctors and patients.
Genomics is a field that deals with genes editing and genomic sequencing in which robotics plays a major role. Such test beds that ensure the proper functioning of genomics would help patients to recover from diseases like central nervous system and infectious diseases. Thus, for this purpose an efficient testing strategy must be identified. Prospects of the utilization of virtual reality for rehabilitation in orthopedics need more exploration. Context‐aware test case design would strengthen confidence in the system.
Augmented reality should also be surveyed broadly so that it can be used effectively for gathering 3D data sets of a patient in real time using sensors like magnetic resonance imaging (MRI), ultrasound imaging, or CT scans. It should also be investigated for its use as a visualization tool during surgeries. Appropriate testing mechanisms need to be identified that work well in this direction. In addition, exploration of 5G applications for its use in the smart devices (such as wearable sensors) to monitor the health condition of patients is the need of the hour. To ensure the attainment of desired functionality, a comprehensive and customized testing strategy need to be devised. Also, a transparent pricing model is required to be implemented that ensures cost reduction of the associated IoT infrastructure by promoting the pooling of distributed, dispersed resources in fog computing and the Internet of Things. This also demands the establishment of test beds that make use of customized testing strategies to ensure the attainment of desired functionality of the ubiquitous system (such as smart home, smart health, and smart transport). Such test beds should be freely available to the research community to carry out extensive studies in this area.
Table 15.6 Summary of limitations and research directions for smart transport.
Criteria | Research Direction | Work | Limitations | Suggestions |
Privacy testing | Ensure privacy and security of transport devices and the associated data. | [37, 39, 43, 44] |
|
|
Energy efficiency testing | Maintain fuel efficiency of vehicles. | [37, 38, 47] | ||
Collision avoidance testing | Ensure the effectiveness of collision avoidance algorithm. | [38, 41] | ||
Autonomous vehicle testing | Validate self‐steering vehicle in real environment. | [38, 43, 44] | ||
Traffic congestion management | Test bed to assess traffic congestion management strategy. | [38, 41, 42, 44, 47] | ||
Connected vehicle technology | Validate connected vehicle technology. | [38, 39, 37, 44, 47] | ||
Compliance with standards | Comply with standards. | [39, 41, 42, 46] | ||
Reliability testing | Ensure robustness and resiliency of transport devices. | [38] | ||
Performance testing | Ensure performance of the devices. | [37, 42] | ||
Usability testing | Ensure user‐friendliness of mobile apps. | [37] | ||
Pollution Control testing | Verify functionality of roadside pollution monitoring equipment. | • | ||
Interopera‐bility testing | Ensure interoperability among devices and roadside infrastructure. | [41, 44, 43] |
The existing work toward the implementation of test beds for smart transport system has been evaluated as per the following set of verification criteria and the associated research directions along with limitation are provided in Table 15.6:
Although many corporate provides the testing solution for smart transport, inadequate work has been done in academia, and this requires special attention of researchers. In addition, several works discusses the importance of cyber‐physical systems in transportation, but no work has been found describing the novel testing methodology that verifies the security of smart transport vehicle. Similarly, numerous works have been found that discuss the importance of connected vehicle technology in reducing air pollution and improve efficiency but no test bed has been found that quantitatively measures the percentage level of air pollution reduction and up to what percentage travel experience gets enriched. Further, no case study has been discussed that empirically proves the benefits of the technology. Also, very few test beds have been developed that actively carry out testing of autonomous vehicles, and no work has been found that tests user friendliness of transportation systems. Test‐bed executing of the repeatable regression tests based on blockchain technology must be studied to address quality assurance issues of the smart transportation system.
Pollution‐monitoring devices are also required to be verified for effectiveness. No test bed has been suggested that works in this direction. Reliability is one of the most crucial feature that should be possessed by transport devices and related infrastructure; hence, there should be appropriate methodology that verifies the cybersecurity measures to ensure the resilience and robustness of the system. Only one research work has been found that works in this direction. Novel testing methodologies should be proposed for the comprehensive testing of collision avoidance algorithms, and the test bed should be designed to be portable so that it can be freely used by the research community.
Fog computing is a paradigm that can be successfully utilized to implement smart applications, as it overcomes the disadvantages associated with edge and cloud computing. The assurance of quality and reliability of fog‐based IOT application is very important before their release to the market as poor design may hamper the working of the application and affects the end‐user experience.
This chapter has surveyed testing perspectives of three cases studies (viz. smart home, smart health and smart transport), along with the elucidation of their objectives, approaches, and the achieved outcomes.
Software testing in the area of fog‐based IoT applications has great potential in future research toward verification and validation of reliability, better security from hacking, Internet connection independency, user‐friendliness, cost cutting, and industry standardization. Practitioners can create prototype ubiquitous testing environment for fog‐based smart applications using advanced testing strategies such as context‐aware test case generation, combinatorial testing and blockchain‐based regression testing to address the issues of quality assurance.
This area commemorates a great deal of success and recognition in the seeable future. However, as we have explained in this chapter, industry and academia need to jump on and grab the compelling challenges and risks associated with it. It will ensure favorable outcome for fog computing in smart technology in distant future. The apparent trends in this sphere include the materialization of standards, the inception of enhanced testing services by boosting and merging current compute, storage and network services, utilization of fog computing along with cloud to provide acceptable QoS and governance; the possibility of exponential growth in smart technology developers and operators, thus widening the horse race and innovation. The researchers and practitioners would find endless opportunities to invent solutions to address hindrances in smart technologies using fog computing.