8. Cooperative Video Provisioning in Mobile Wireless Environments

In the last years, the wireless Internet has received evergrowing attention resulted from the unprecedented and mass-market spread of mobile devices: smartphones, tablets, and netbooks are widely and increasingly exploited to access the Internet from everywhere, in a ubiquitous manner. Initially, mobile devices were primarily used to surf the web and access contents managed by traditional servers connected to the fixed Internet via wide-bandwidth links. In other words, users exploited their mobile devices similar to what they were used to do with their desktops, with the only but notable differences that, in the case of mobile devices, (a) users were not tied to fixed locations and (b) devices usually had limited hardware/software capabilities, e.g., making harder to render HTML pages designed for high-resolution and large displays.

However, mobile computing has quickly moved toward more powerful service provisioning scenarios where devices not only play the role of clients that access remote resources while in mobility, but also generate and provide interesting content to other nodes. Mobile nodes generating audio/video real-time streams are a notable example, where communication endpoints can even simultaneously play the roles of both content generators and consumers. However, in relatively traditional solutions, inter-node communication is still supported by infrastructure-based networking, and multimedia stream delivery can exploit special-purpose infrastructure components, e.g., proxy nodes that transcode audio/video streams to fit client constraints dynamically.

More recently, mobile environments are shifting toward novel and more challenging paradigms for communication and service provisioning, which aim at connecting wireless devices one another directly, without any constraint to be connected to the traditional Internet or to exploit support components deployed on either the infrastructure or server-side. In particular, nowadays cooperative mobile networks are attracting significant academic/industrial attention and are providing promising preliminary results: the specific property of cooperative mobile networks is that they are originated from the willingness of social interaction of people via impromptu interconnection of the mobile personal devices they carry (e.g., smartphones, PDAs, and iPADs) (Feeney et al. 2001). Services take advantage of inter-device cooperation to dispatch packets toward their destination, thus potentially enabling the full exploitation of all the potential of all the available wireless connectivity opportunities, for instance, by allowing resource connectivity sharing and immediate connectivity offers in regions with difficult coverage (Ferreira et al. 2010, Salameh et al. 2009, Wu et al. 2007). In our vision, cooperative mobile networks will make soon another relevant evolution step: they will support multi-hop application-oriented scenarios where cooperative trusted users will collaborate to invoke/provide services to the community, such as sharing locally stored JPEG pictures, delivering live multimedia, and offering underutilized flat-rate Internet connectivity. In other words, we envision a collaborative provisioning environment where mobile communications are fully enabled also in the absence of a networking infrastructure: Cooperating nodes can generate multimedia flows, route them toward their destination, and eventually collaborate to adapt them at runtime, e.g., in response to the detection of bandwidth degradation.

Let us point out from the beginning that, on the one hand, the user-centric nature of cooperative mobile networks relaxes the constraint of having infrastructure-based communication support (e.g., expensive cellular coverage anywhere). On the other hand, it naturally yields to very heterogeneous, uncoordinated, and dynamic networking environments where, for instance, any node can create its self-administered layer 2 links. In addition, to fully exploit the potential of these networks, nodes should be able to take advantage of different wireless interfaces simultaneously (multi-homing) to join/create multiple IP networks (via both ad hoc and infrastructure-based connectivity). These user-centric networks should be autonomously created, configured, and destroyed by collaborating users in a completely decentralized way and should be managed with very lightweight coordination (or without any coordination at all). Moreover, to be scalable, cooperative mobile networks call for the challenging issue of the effective management of end-to-end multi-hop communications with local and partial topology awareness.

Also due to the number of technical challenges (heterogeneity, lack of coordination, high dynamicity, and so on) to address to enable effective use of cooperative mobile networks, as better detailed in the following, up to now social collaboration has been limited to very simple applications, such as message/status sharing. However, because mobile devices are becoming general-purpose media players and personal data carriers, we strongly claim that there will be the need, in the near future, of supporting more complex resource sharing and providing more challenging collaborative applications. In this perspective, a crucial role will be played by collaborative streaming of live multimedia.

8.1.1 Application Case and Problem Statement

To practically clarify the targeted application case with an example, consider a hybrid scenario including the characteristics of both traditional wireless Internet and cooperative mobile networking. For instance, a student (Alice) is watching a soccer match on her smartphone via an HSDPA device (Figure 8.1). The quality of the visualized stream depends on many factors, ranging from client-side static capabilities, e.g., screen size and CPU/memory, to server-side dynamic state, e.g., number of clients requiring the stream, and client-to-server connectivity quality, e.g., wireless connectivity quality is greatly influenced by interferences and competing traffic due to concurrent access by multiple mobile clients (Bellavista et al. 2009). In order to achieve a quality level suitable for the final user, the multimedia player on Alice’s smartphone can negotiate the stream quality with the remote server at start-up, e.g., selecting the stream version with the best quality that fits client capabilities, and also during service provisioning, e.g., dynamically switching stream versions according to currently available bandwidth. In this way, Alice’s smartphone can receive the stream at a quality compatible with its low capabilities, while the same multimedia content is sent at full quality to desktop nodes that connect to the Internet via traditional ADSL. Moreover, desktop nodes can forward the stream to other nodes in the subnet, eventually transcoding and tailoring it at runtime to fit codec and hardware capabilities of nearby devices, ranging from high-definition UPnP-enabled TVs to smartphones connected to the local gateway via IEEE 802.11.

FIGURE 8.1
Multimedia stream delivery with differentiated quality.

In addition, Alice can decide to play an active collaborative role by working to provide online redistribution (namely recast) of the received stream to her friends’ devices in the neighborhood, even if they are not equipped with HSDPA. To this purpose, Alice’s smartphone starts streaming the match, possibly via multi-hop paths consisting of heterogeneous ad hoc links. Bob receives the match with automatically downscaled quality level (lower-frame frequency) because the path between Alice and Bob has limited bandwidth, e.g., due to the fact it consists of low-quality hops (Figure 8.2a). The stream is downscaled only if and when required, thus permitting to nodes connected to Alice via high-quality paths to receive the multimedia stream at full quality. To this purpose, nodes in the Alice-to-Bob path have to (a) collaboratively interact to dispatch packets along the path toward the destination and (b) actively monitor the path quality to tailor traversing streams depending on current connectivity capabilities. In the meanwhile, Carl joins the network and provides the same multimedia content at higher quality via its local DVB-T device (Figure 8.2b). Bob’s smartphone perceives that Carl’s netbook provides the same content at higher quality and via a better collaborative path; therefore, it switches streaming from Alice’s laptop to Carl’s netbook.

FIGURE 8.2
Collaborative content distribution/adaptation (a) and path reconfiguration (b).

This chapter will focus on this articulated and state-of-the-art scenario of cooperative video provisioning in mobile wireless environments. The primary idea is to exploit the spontaneous and opportunistic collaboration of mobile wireless devices, which cooperate to gather, monitor, and tailor multimedia content depending on dynamically changing capabilities of their communication channels. In particular, the objective of this chapter is twofold. On the one hand, it provides an insight of the state-of-the-art literature about collaborative mobile networking, ranging from static server-to-client interaction for the selection of the proper multimedia flow at service start-up, to more challenging forms of peer-to-peer cooperation involving dynamic downscaling/upscaling of multimedia flows at service provisioning time. On the other hand, it reports about the RAMP case study, demonstrating that node cooperation can properly and effectively achieve multimedia streaming adaptation, with minimum impact on computing/memory resources available at intermediary participating nodes.

The structure of the chapter is as follows. Section 8.2 presents how the traditional Internet has evolved toward a more dynamic scenario with ubiquitous wireless connectivity, eventually involving even direct communication among devices. In particular, it focuses on multimedia stream adaptation techniques suitable for these environments. Section 8.3 outlines the primary related work in the field, classifying state-of-the-art contributions along three main guidelines: (a) traditional end-to-end stream provisioning/adaptation that involves only server and client; (b) more articulated and distributed solutions based on proxies, with the objective of improving the quality perceived by final users; and (c) challenging proposals based on the dynamic interaction of peers along the path between stream providers and final users. Section 8.4 details our RAMP middleware solution, aiming at supporting the provisioning of multimedia content in collaborative mobile networks. Finally, Section 8.5 reports conclusive remarks and points out some hot topics, which are promising research directions for the near future.

8.2 Multimedia Streaming in Mobile Wireless Environments

The traditional Internet has already evolved to a mobile one characterized by handheld devices capable of receiving streams via wireless links, while content providers are still usually fixed and reachable via the traditional Internet. The widespread utilization of peer-to-peer multimedia services, e.g., audio/video communication over IP, calls for more complex provisioning environments where both streaming endpoints are connected via wireless links and mobile. Also the distinction between content providers and consumers is becoming blurred: the trend is that any node tends to be producer and consumer of multimedia content, e.g., by sharing user-generated multimedia content composed by text, pictures, videos, or a mix of them. As already sketched in the introduction, the upcoming communication trend goes beyond, toward more challenging scenarios where endpoints are connected via multi-hop wireless paths, not only exploiting special-purpose infrastructure equipment, e.g., IEEE 802.11s mesh routers, but also mobile nodes collaboratively cooperating to dispatch packets, e.g., as in our RAMP middleware for collaborative mobile networking (Bellavista et al. 2012).

In particular, we identify four main communication paradigms for mobile wireless streaming, spanning from the traditional wireless Internet to more challenging mobile environments based on peer-to-peer communications (see Figure 8.3):

1. Only client endpoints are connected via wireless hops (Figure 8.3a): This is the traditional wireless Internet environment, where mobile nodes (e.g., node A) access the Internet via one wireless hop toward an infrastructure of access points, e.g., by exploiting IEEE 802.11 or HSDPA interfaces. Typically, services are provided by fixed servers (e.g., node S) deployed on nodes with relatively high-computing capabilities and connected to the Internet via highly reliable and large-bandwidth wired links; clients are the only mobile entities and are connected to the Internet via wireless hops with low/medium connectivity quality. For instance, users can view videos on their mobile nodes connecting to YouTube and alike, as they are used to do from their desktop computers, even if with a reduced content resolution and screen size.

FIGURE 8.3
The proposed categorization of communication paradigms for mobile streaming: (a) only clients are wireless, (b) clients and servers are wireless, (c) also intermediary nodes are wireless, and (d) intermediary nodes are wireless, mobile, and collaborative.

2. Both endpoints use wireless hops (Figure 8.3b): The widespread diffusion of mobile devices has recently leveraged the adoption of applications where both endpoints of audio/video streams are connected via wireless hops. In addition, client and server roles may be blurred, since both nodes can offer and take advantage of services and resources in a peer-to-peer way. Several VoIP applications are a notable example in this category: both endpoints typically exploit wireless interfaces to connect to the Internet and to transmit/receive audio and video streams; there is no streaming server playing the specific role of content generator and deployed on a high performance and reliable node, as it was used in more traditional web-based servers for streaming download.

3. Also intermediary hops are wireless (Figure 8.3c), e.g., in wireless mesh networks (Camp and Knightly 2008): In the last years, the deployment of large wireless networks covering small-to-medium urban areas, e.g., university campuses and city downtowns, has moved the attention to the proper management of wireless connectivity among nodes via multi-hop paths composed of wireless links provided by mesh routers, eventually without any need of connecting to the Internet. For instance, mesh networks can be adopted to set up medium-scale remote surveillance systems, via proper deployment of wireless routers along the covered area and to collect video streams toward a control point. Mesh-based solutions make easier and less expensive the dynamic extension of the wireless coverage area, e.g., by adding or relocating wireless mesh routers.

4. Intermediary nodes are wireless and mobile (Figure 8.3d), e.g., collaborative mobile networks: In collaborative mobile networks, devices discover and interact with one another opportunistically and without any prior mutual knowledge, by exploiting any wireless opportunity currently available in a best-effort way (e.g., Wi-Fi or Bluetooth ad hoc links and UMTS infrastructure-based ones). In particular, social/group-related behavior and the ever-increasing willingness to share rich user-generated contents, also pertaining to the personal sphere, call for a user-centric communication paradigm shift (as shown also by recent standardization efforts for point-to-point Wi-Fi connectivity [Wi-Fi Alliance 2009]), where the ad hoc interconnection of mobile devices in direct visibility plays a central role. For instance, it should be used for conference/class streaming with attendees/students spread on multiple rooms or for DVB-T program redistribution by (the generally few) nodes equipped with DVB-T receivers, as rapidly sketched in Section 8.1.

Such scenarios require novel multimedia management/adaptation techniques, aware of specific characteristics of mobile/wireless environments (e.g., heterogeneous performance between wired and wireless subpaths) and dynamically exploiting the resources offered by intermediary nodes (e.g., in collaborative mobile networks, intermediary nodes cooperate not only to dispatch packets but also to dynamically tailor the traversing multimedia streams). In particular, we classify the techniques for multimedia content provisioning, monitoring, and adaptation in three main categories, depending on where streaming quality assessment/tailoring are performed: end-to-end; proxy-based (one single- and special-purpose intermediary); collaborative (several general-purpose intermediary nodes).

End-to-end adaptation techniques represent the basic and currently most adopted mechanism to adapt the multimedia content. Servers and/or clients are aware that the same content can be provided at different quality levels and interact to select the suitable quality to fit the currently available capabilities. In the simplest case, the final user has the opportunity to select among multiple choices, e.g., low, medium, and high quality; once a quality level is selected, the server delivers the content to the client, despite that client device characteristics and available bandwidth actually fit to the case for the whole streaming duration. For instance, a user can select a movie in HD format from her smartphone connected to the Internet via UMTS, thus almost certainly receiving the stream jerkily, if receiving it at all. Moreover, the same multimedia content can be statically tailored at different quality levels and stored on the server-side, e.g., by generating offline multiple files with different codecs and bitrates. Or it can be dynamically adapted by need and on the fly during service provisioning with per-client granularity, thus finely tuning the content in relation to client capabilities; this produces the nonnegligible overhead of possibly transcoding/adapting the same multimedia content several times.

Slightly more sophisticated solutions involve content distributors and clients in the quality selection process. On the one hand, content distributors take into consideration server-side information to provide the set of available choices. For instance, the streaming server can provide the list of available quality levels and related codecs, eventually avoiding the delivery of HD streams in the case of heavy load due to many concurrent requests from multiple users. On the other hand, clients rendering the multimedia content to final users can exploit local resource information in order to discard unsuitable choices, e.g., by comparing the characteristics of potentially available streams with the maximum resolution of the local display. In this case, the final user achieves the notable advantage of not requiring any knowledge about multimedia content formats and device hardware/software capabilities, delegating to the underlying system the proper selection of the most suitable quality level in a completely transparent way. At the same time, the server can increase the amount of concurrently served clients by adapting the set of provided streams in relation to its capabilities. However, in this case, once service distribution is started, it is not possible to modify the content quality anymore; for instance, in case of abrupt bandwidth fluctuations, the final user may perceive frequent service disruptions, requiring to manually and explicitly restart the streaming with lower quality.

Wireless environments are usually characterized by highly varying connectivity quality. For instance, user movements or interferences may cause frequent packet loss, by negatively affecting the quality perceived by final users. For this reason, it is not possible to consider only static characteristics at service initialization time. To this purpose, recent proposals for mobile wireless streaming aim at assessing connectivity quality and tailor multimedia content at service provisioning time. The most notable advantage is the capability of adapting content quality dynamically, by fully exploiting the currently available capabilities. At service request time, content distributor and client select the proper quality level, then they properly renegotiate content quality at runtime in relation to, e.g., increased/decreased server-side load, better/worse connectivity bandwidth, or client-side stream delay/jitter.

The dynamic reconfiguration of multimedia content is particularly effective when coupled with proxy-based solutions. In this case, end-to-end paths are typically characterized by a reliable, powerful wired subpath, e.g., from the server to the wired/wireless edge, and an unreliable and bandwidth-limited wireless one, e.g., from a UMTS base station to the client on a smartphone. Proxy-based solutions decouple the content delivery process by carefully considering this difference: proxies are special-purpose components specifically designed to support multimedia content delivery at the proper quality level. They are typically deployed on the infrastructure-side, gather the content from the server, and provide it to wireless clients considering the unreliable nature of their adjacent wireless links. For instance, proxies can retrieve multimedia content at full quality and then deliver it to clients of the same UMTS base station in a differentiated manner, considering that different smartphones may have different hardware/software capabilities and access the network with different bandwidth. In addition, proxies can eventually pre-fetch large chunks of multimedia content and/or transcode it in order to fit client capabilities. In this way, it is possible to achieve the notable advantage of lowering the load on servers because multimedia content is requested only once and delivered to multiple users by exploiting different codecs and bitrates.

In the case of collaborative mobile networks, each intermediary node can perform quality assessment and content tailoring similar to the proxies provided earlier, by achieving highly collaborative solutions. Proxies are usually deployed on special-purpose equipment with the main goal of supporting content delivery and adaptation. On the contrary, intermediary nodes of collaborative mobile networks share part of their computational/memory/connectivity resources to support content delivery/tailoring in a best-effort way. In collaborative environments, users are willing to cooperate to allow content dispatching, but they also require limiting local resource consumption, e.g., avoiding excessive consumption of local computing resources for multimedia content transcoding. At the same time, since the same path is split into multiple segments, each one corresponding to a couple of adjacent intermediary nodes, it is possible to monitor local subpaths with finer granularity if compared with proxy-based solutions. For instance, it is possible to tailor the multimedia content only in correspondence of a Bluetooth link with limited capabilities, while maintaining higher quality in other parts of the served path.

8.3 Related Work

This section classifies the state-of-the-art literature contributions about multimedia content adaptation for mobile/wireless streaming by taking into consideration the definitions and concepts presented in Section 8.2. Existing approaches are organized in three categories: end-to-end, based on quality selection performed at content distributor, typically choosing among few coarse-grained possibilities, eventually by following client indications; proxy-based, performing dynamic adaptation typically at the wired/wireless edge (Gene Cheung et al. 2005); and collaborative tailoring, where clients are more involved in quality assessment and eventually adapt multimedia streams in a collaborative and peer-to-peer way.

The remainder of the section focuses, for each category, on the related work considered as the most relevant and exemplar, with the main goal of reporting few but notable solutions that are also useful to fully understand the categorization that we have originally introduced in the previous section. For the sake of presentation clarity, in the following, all the reported solutions are associated with one single category in our classification (in relation to the solutions characteristics we deem more relevant), even if minor relationships with other categories may exist.

8.3.1 End-to-End Solutions

Traditional stream tailoring solutions for fixed and mobile multimedia are mainly based on end-to-end mechanisms, involving only the server offering the multimedia content and the client rendering it to the user.

Adaptive bitrate streaming describes a basic and effective mechanism to provide content adaptation via differentiated multimedia streams to HTTP clients, by avoiding asking final users for the explicit selection of the requested quality level (Adobe 2010). In particular, the client actively probes local computing resources and available bandwidth, and accordingly selects the multimedia content available on the server-side at the most suitable bitrate. In this way, adaptive bitrate streaming succeeds in fitting the provided quality to current computing/networking capabilities. Moreover, the adoption of HTTP as the transport protocol easily supports the delivery of both live and on-demand streams to very differentiated clients. However, bitrate options have to be statically decided, and modifications of video bitrate are possible only at the beginning of a multimedia fragment (temporal subset of the provided stream). There is no possibility of gracefully and dynamically increasing/decreasing multimedia bitrate at runtime, in a fine-grained way and only when actually needed.

In order to provide a more powerful and sophisticated mechanism to dynamically tailor multimedia content at provisioning time, research activities in scalable video coding (SVC) for H.264 have mainly focused on multilayered codecs, where the same multimedia resource can be provided to clients in the form of different subsets of layers (a base layer plus one or more enhancement ones) (Schwarz et al. 2007). In particular, temporal scalability approaches provide clients with a different set of frames by varying the desired frame rate; in spatial scalability, enhancement layers enable higher spatial resolution, e.g., by augmenting horizontal or vertical frame resolutions; in quality scalability, enhancement layers permit to improve the rendering process. SVC has the notable advantage of easily allowing fine-tuned adaptation of multimedia content on the fly, by overcoming many issues associated with solutions that exploit a small set of predefined quality levels, such as adaptive bitrate streaming. In particular, SVC permits to store multimedia content on the server-side in only one format: The same multimedia content is tailored, only at provisioning time, in relation to client requirements. Moreover, content adaptation can be performed only when actually required, thus overcoming limitations of coarse-grained per-fragment quality selection of adaptive bitrate streaming. Note that the great flexibility of SVC makes its adoption suitable not only for end-to-end solutions, but also for proxy-based and collaborative ones.

By focusing on mobile wireless environments, some relevant work has explored quality management via dynamic flow transformation. The main goal is to assess end-to-end quality in order to select the most suitable quality level. The work by Frojdh et al. (2006) specifically considers 2.5G and 3G cellular networks, by evaluating end-to-end paths based on information sent by the client. The main goal is to send the proper amount of multimedia content to the client in order to avoid both client-side buffer overflow (e.g., sending the stream at a bitrate higher than rendering capabilities) and underflow (e.g., sending the stream at a bitrate higher than the currently available bandwidth). To this purpose, the client provides the server with RTCP feedback about the free space available on the client-side buffer, the amount of lost packets, RTT, and the sequence numbers of received packets. Then, the server is in charge of dynamically selecting the proper quality and downscaling/upscaling the delivered multimedia stream.

The work by Zhi-Jin et al. (2005) is another example of multimedia content adaptation driven by the conditions of the buffer on the receiver side. In this case, content adaptation is performed by providing differentiated priority to different stream layers: The basic layer has greatest priority and is scheduled to be transmitted first, while enhancement layers are sent only if there is enough bandwidth.

The work by Guenkova-Luy et al. (2004) adopts a more general approach, which aims at facilitating the QoS negotiation process not only at service start-up, but also when notable events may vary end-to-end characteristics, such as vertical handovers. The proposed solution adopts SIP to transfer control data and XML to describe system characteristics and quality parameters. The proposed management process considers a wide set of parameters: quality perceived by the user, application-level stream characteristics such as frame rate and size, transport-level parameters such as media codec and network-access technology, and specific characteristics of end systems, e.g., memory and CPU.

Finally, the work by Wanghong Yuan et al. (2006) represents an interesting solution aiming at offering the best user experience possible by adapting not only the multimedia stream but also the behavior of the client at the hardware and operating system layers. The goal is to coordinate the adaptation process with the application, operating system, and hardware layers in order to improve user satisfaction while minimizing the imposed overhead. For instance, the proposed solution takes into consideration that processors can run at a selectable speed in a discrete set, that codec operations are usually long-living and CPU intensive, and that different video frames may have different computational complexity and different importance in the rendering process.

8.3.2 Proxy-Based Solutions

In mobile environments, the necessity of properly and differently managing the heterogeneity of performance in the wired and wireless links that usually compose paths between servers and clients has generally emerged. Traditional solutions have managed content delivery to mobile and wired clients in the same manner, e.g., simply considering the available bandwidth in the end-to-end path equal to its bottleneck, i.e., the wireless hop. However, the unreliable and ephemeral nature of wireless connectivity pushes for novel solutions explicitly considering the peculiarities of mobile wireless networks. In particular, proxy-based solutions represent the most spread and widely accepted architecture proposal for delivering multimedia streams with adaptation capabilities, by taking into primary consideration the specific differences and characteristics between the wired and the wireless subpaths.

In fact, proxies reside at the edge between wired and wireless path segments and actively interact with content providers/consumers to dynamically assess stream quality and tailor multimedia contents, e.g., via selective frame dropping. On the one hand, they take full advantage of reliable and powerful connectivity toward the Internet to get multimedia content from servers, possibly at full quality. On the other hand, they consider the intermittent and highly variable characteristics of the wireless subpath to properly and dynamically adapt delivered content, e.g., upscaling/downscaling video quality in relation to wireless interferences and the actually achievable throughput. For instance, a proxy can get a multimedia stream from a web server only once and then redistribute the same stream to multiple mobile devices with differentiated quality levels. In this way, it achieves the twofold benefit of lowering the traffic load on the server-side and providing the multimedia stream to users in a per-client fine-grained fashion.

To this purpose, Liu et al. (2010) propose components at the wired/wireless borders that manage multimedia streams to maximize quality while reducing the needed throughput over limited wireless links. Proxies divide streams into different layers, by grouping frames with the same priority; then, they can separately handle different layers, by dropping part of the streams with per-layer per-client granularity. Jeong et al. (2007) focus on home network environments, with the specific goal of transcoding multimedia content on residential gateways between high-quality and bandwidth-eager digital video format and the more efficient low-quality MPEG4 one. Thus, while Liu et al. (2010) and Jeong et al. (2007) have demonstrated their effectiveness in access point-based network infrastructures, their architecture are only partially distributed and only border nodes manage streams, e.g., depending on explicit client feedback.

Ji Shen et al. (2004) delineate an architecture composed of several proxies interacting the one with the other in order to provide multimedia content to wireless devices. The main purpose is to overcome typical issues associated with wireless communication, such as hardware/software heterogeneity of mobile devices, reduced capabilities (preventing from on-the-fly content transcoding on the client side), and intermittent connectivity. Proxies deployed close to wireless devices interact to negotiate stream quality on behalf of clients, eventually by buffering part of the content in the case of abrupt connectivity disruption; the buffered content can be delivered immediately once the wireless link has been recovered.

The SProxy project aims at increasing the quality perceived by final users while lowering the load on web servers that provide multimedia content (Chen et al. 2007). Since it is based on well-known protocols, i.e., RTSP, RTP, and HTTP, it is compatible with legacy clients and servers, i. e., clients require streams to the proxy in a transparent manner as if it were a standard web server. SProxy provides the capability of dividing multimedia content in several segments related to different time intervals: client requests are split in multiple subrequests, in order to retrieve only the multimedia content segments actually required by the client instead of the whole content, thus lowering the traffic imposed on servers. Moreover, SProxy implements prefetching techniques, thus tuning the traffic rate according to the dynamically detected bandwidth between proxy and server. Finally, the retrieved segments are stored at the proxy by adopting a popularity-based replacement policy, in order to exploit caching capabilities in an efficient way.

Park et al. (2006) propose a solution that is based on frame relevance discrimination and that drops frames in a selective way. It evaluates connectivity quality based on MAC/PHY parameters, thus tightly coupling its evaluation metric to the IEEE 802.11 protocol. Home gateways distribute multimedia content gathered from the Internet to local clients via RTP-MPEG2. The proposed solution differently manages RTP/MPEG2 packets in relation to their content, considering that MPEG2 video is composed of self-contained intra-coded (I) video frames including any information required for rendering them and predictive-coded/bidirectionally predictive-coded (P/B) frames relying on data included in other previous/following frames. In particular, it gives maximum priority to RTP control packets, then to audio RTP packets, to I frame packets, and finally to P/B frame packets. Note that RTP packets are possibly split at the proxy in order to contain only one type of frame at a time, e.g., to avoid transmitting TS packets (188B packets representing MPEG2 basic data unit; typically each RTP packet contains 7 TS packets) related to I and P frames together within the same RTP packet.

In the work by Burza et al. (2007), proxies assess bandwidth saturation based on the number of buffered frames that still wait to be sent to the client. In case of saturation, they lower stream bitrate by selectively dropping B and P frames in this order, thus according to their importance for the rendering process on the client. Instead, since I frames are crucial for the correct rendering of the multimedia stream, the proposed solution does not drop them but, in case of saturation, briefly delay their transmission. In fact, even if delayed, I frames can be very useful to decode the following P/B frames. Similar to the work by Park et al. (2006), the proposed solution actually changes how TS packets are encapsulated into RTP packets: It forces any RTP packet to contain at most one frame in order to simplify content-type determination and differentiated packet management. However, RTP packets frequently tend to have a payload of limited size (even less than the traditional 7 TS packets) and, therefore, the relative overhead due to RTP/UDP/IP headers is not negligible.

8.3.3 Collaborative Solutions

In the recent literature, novel streaming adaptation solutions that also support collaborative management of multimedia content, eventually based on peer-to-peer cooperation, are emerging. The goal is to enable content adaptation (quality reduction) only if and where actually needed, by keeping maximum quality otherwise. For instance, Jurca et al. (2007) provide a general overview of peer-to-peer systems applied to self-organized networks, with the main purpose of delivering multimedia content in a scalable way. In particular, Jurca et al. (2007) point out that intermediary nodes should actively cooperate not only to dispatch the provided streams, but also to gather context information related to their locality, to properly dispatch content along the most suitable path (with finer-grained and greater flexibility if compared with traditional end-to-end solutions), and eventually to adapt/filter the multimedia content.

Narayanan et al. (2007) exploit the availability of multiple relays close to the node that is providing multimedia content in a peer-to-peer way. Every device can play the role of relay; therefore, there is the need to support the dynamic discovery of relays in such an opportunistic network. Discovery should be scalable, because the number of available relays could be high, and should be based on proximity considerations, because closer relays generally provide multimedia content with higher quality, e.g., with lower latency.

Uhm et al. (2010) provide nearby devices with content adaptation features in a user-centric way. Even if the proposed solution is also proxy based and exploits special-purpose equipment for multimedia delivery, we position (Uhm et al. 2010) in the category of collaborative solutions because of the prevalence of its collaborative and opportunistic contributions. In fact, Uhm et al. (2010) propose a portable gateway that dynamically retrieves devices and sensors close to the user, interacts with them to gather their characteristics and profiles, and adapts/transcodes the provided multimedia content to fit user/device requirements. In particular, the portable gateway offers the notable advantages of, on the one hand, adapting its behavior in relation to the current location, and, on the other hand, having full knowledge of user profile/requirements, thus enabling the periodic reassessment of connectivity quality and dynamic readaptation of multimedia content to fit changing conditions. For instance, if the user moves from home to her car, the portable gateway actively interacts with dynamically discovered nearby devices to keep active sessions alive, by switching them from the home IEEE 802.11 network to the cellular-based HSDPA one.

Mukherjee et al. (2005) propose a tailoring mechanism suitable for arbitrary formats, based on a universal model fitting all scalable bit streams. The general solution is based on metadata, associated to the stream, that specify how it is possible to tailor the stream and which are the network/terminal constraints to support the real-time process of quality tailoring selection. Adaptation can be performed also at intermediary nodes; since network/terminal constraints/information are sent to the preceding adaptation entity, it is possible to deploy chains of adaptors, each one collecting feedback from the successive entity, either a client or another adaptation entity. In this manner, it is possible to optimize the overall end-to-end transmission efficiency, by adapting the stream only where actually required.

Ip et al. (2007) merge a proxy-based architecture with a peer-to-peer one, exploiting an inherently collaborative approach. In fact, several proxies, colocated near their mobile clients, collaboratively host multimedia streams in a partitioned and partially replicated way. The main purpose is to minimize bandwidth consumption by properly deploying middleware components close to either final users or popular stream sources. Thus, Ip et al. (2007) achieve the twofold objective of lowering transmission costs while providing greater reliability if compared with pure peer-to-peer solutions without any proxy support.

Hutter et al. (2005) couple SVC-based tailoring with a collaborative approach that distributes the burden of content adaptation among intermediate nodes. In particular, each adaptation node consists of three modules. The context aggregation module gathers context information provided by clients. The adaptation engine module exploits information provided by each client to select the proper quality level (local consideration), possibly different for each client. The context merging module considers together all the information provided by its clients (global consideration) and sends to the server (or another cooperating node) the quality level suitable for the client capable of receiving the best quality at the moment.

Mastronarde et al. (2007) exploit the multipath nature of mesh networks, together with the layered structure of SVC streams, to take full advantage of available connectivity resources. In particular, different layers of the same multimedia stream are sent as subflows toward the same destination along different paths, thus achieving a considerable quality gain. Nodes collaborate to monitor and assess wireless link conditions with the twofold goal of deciding which subflows should be admitted and of selecting which are the paths where the accepted subflows should be sent through.

Finally, the work by Hsien-Po Shiang and van der Schaar (2007) introduces the concept of “network horizon” in a wireless multi-hop path. Its purpose is to efficiently disseminate quality-related information within a given network scope, identified in terms of hop-distance. Intermediary nodes provide transmitting nodes with lightweight monitoring feedback, in order to limit the number of hops that monitoring data should go backward along the stream path. On the one hand, larger horizons allow deeper knowledge of global network conditions at the cost of imposing greater overhead; on the other hand, tighter horizons limit the monitoring awareness of the adaptation mechanism but permit to receive more up-to-date information, since control packets traverse only a limited set of intermediate nodes.

Just to summarize some crucial points analyzed in this related work section, it is evident that traditional end-to-end solutions cannot properly and fully take into consideration the specific characteristics of mobile wireless streaming. Proxy-based solutions certainly represent a notable improvement and evolution step in the related architectures of solution, since they permit to lower the load on servers while supporting fine-grained and per-client content adaptation. However, proxy solutions do not apply well to collaborative environments stemming from the impromptu interaction of mobile devices getting and providing connectivity in a peer-to-peer way. For this reason, collaborative solutions have recently emerged as the most promising approach to fully exploit resource sharing among nearby devices carried by users socially interacting and willing to collaborate. However, while first proposals have already delineated interesting solution guidelines and notable high-level models of general applicability, they lack in providing actual prototypes supporting the dynamic discovery and exploitation of shared resources in collaborative wireless environments suitable for the proper distribution and adaptation of multimedia content.

8.4 The RAMP Case Study

The previous section has clearly pointed out that node cooperation can greatly improve multimedia content delivery in multi-hop wireless networks stemming from the impromptu interaction of peer nodes. On the one hand, the overhead imposed on intermediary nodes can be properly lowered down by redistributing the same content toward multiple clients. On the other hand, final user satisfaction can be maximized by tailoring streams dynamically depending on specific requirements and capabilities of client devices. However, the reported related work has mostly privileged the proposal of high-level guidelines and analytical solutions, sometimes applied to simulation-based deployment environments (Hsien-Po Shiang and van der Schaar 2007, Hutter et al. 2005, Jurca et al. 2007, Mastronarde et al. 2007); few prototypes have been proposed in the related literature, and they tend to be specialized for some limited and technology-specific deployment environments (Narayanan et al. 2007, Uhm et al. 2010).

We claim that the RAMP middleware is a notable example of solution for collaborative recasting of live multimedia in multi-hop collaborative mobile networks, which overcomes the aforementioned limitations (Bellavista et al. 2010, Bellavista et al. 2012, Bellavista and Giannelli 2010). In particular, this section focuses on how node cooperation in RAMP can enable novel mechanisms for dynamic quality assessment and adaptation, for the support of multimedia tailoring only when and where actually needed, and for the distribution of tailoring load on multiple nodes in a fully distributed way. In particular, as better detailed in the following, the RAMP middleware exploits node collaboration to achieve multiple goals at different layers. RAMP collaborative nodes are as follows:

• Dispatch packets among nodes by exploiting middleware-layer routing facilities

• Recast multimedia flows at the application layer in order to avoid traffic duplication

• Assess stream quality and tailor multimedia content by considering both transport-layer features, e.g., packet delay and jitter, and application-layer characteristics, e.g., the dynamic bitrate variation of H.264 video streams

RAMP adopts many of the guidelines and principles proposed in Section 8.3.3, with the nonnegligible additional proof offering a prototype solution that is easily deployable on most widespread devices. Compared with the work presented in Section 8.3.3, our proposal aims to demonstrate that it is possible to support cooperative multimedia dispatching, monitoring, and tailoring in a lightweight way, also by exploiting off-the-shelf streaming components (i.e., the multiplatform VLC streamer/player for audio/video encoding) and by adopting middleware-based solutions that does not force to modify packet routing at the operating system level. In fact, we have already implemented and extensively validated the RAMP middleware for live streaming in collaborative mobile wireless networks on most common operating systems, i.e., Android, MS Windows XP/Vista/7, Linux Ubuntu/Debian, and MacOSX Snow Leopard. The prototype code is available at our website (http://lia.deis.unibo.it/Research/RAMP/). The goal is also to provide the researchers’ community with a practical contribution to facilitate collaborative networking experimentation and to leverage an emerging critical mass of application developers in the field. In the following, the chapter devotes some space to the presentation of the design guidelines of our original RAMP-based collaborative recasting solution for multimedia delivery and adaptation; this should work as a clear practical example of the general solution guidelines described earlier.

8.4.1 RAMP-Based Collaborative Multimedia Recasting

RAMP supports multimedia content dispatching and tailoring in mobile environments at the middleware layer, enabling to easily share resources and provide services in a collaborative manner. In particular, RAMP collaborative multimedia recasting works based on two primary ideas:

1. Dynamic exploitation of collaborating nodes to split end-to-end multimedia paths in different RAMP-managed segments (Figure 8.4)

2. Dynamic reconfiguration of clients to get their requested stream from their closest source, by autonomously switching among the available path segments (Figure 8.5)

In this way, RAMP frees the multimedia source from the burden of directly managing service provisioning to every client, by lowering the traffic load on it and on its neighbors. At the same time, RAMP has the goal of imposing minimum overhead on collaborating nodes that offer themselves to recast live multimedia content. Furthermore, RAMP manages path segments as if they were the original and complete stream path, e.g., by possibly tuning the multimedia quality of each segment differently, via temporary downscaling of video quality only in correspondence of low-performance links.

FIGURE 8.4
Active streams in case of two additional harvesters (a) without and (b) with splitter. Values over links indicate the number of per-hop active streams.

FIGURE 8.5
Active streams in the case of splitter activation (a) without and (b) with dynamic reconfiguration.

To clarify how RAMP nodes collaborate to effectively provide live multimedia to multiple receivers, let us consider again the case of DVB-T program recasting. In this scenario, we identify three different roles:

• Multimedia Dispenser, a RAMP node acting as streaming server because equipped with a DVB-T receiver, e.g., node D in Figure 8.4

• Dynamic Harvester, a RAMP client receiving the requested stream and rendering it to its user, e.g., node H

• Smart Splitter, an intermediary RAMP node that is aware of traversing streams and possibly offers them to clients as if it were a Multimedia Dispenser, e. g., node S

The users of the Multimedia Dispenser nodes can specify which multimedia flows they are willing to redistribute, e.g., a soccer match. Dynamic Harvesters look for the closest Multimedia Dispenser (or Smart Splitter) providing a specific match and possibly request it. The Smart Splitter nodes passively monitor their traversing traffic; thus, Smart Splitters can decide to split and offer traversing programs to other new clients via secondary stream segments, as if they were the real source (origin stream splitting into different segments).

If one adopts a trivial solution with no content recasting on intermediary nodes, the same multimedia flow must be replicated, even in the common case of delivery of the same multimedia content over shared links toward different destinations. Instead, the exploitation of splitters on intermediary nodes can relevantly reduce content delivery redundancy. For instance, in the example of Figure 8.4, the splitter reduces the impact of two new harvesters (H₃ and H₄) by dividing end-to-end streams into two segments (from D to S and from S to H_3/4), thus lowering the total amount of per-hop active streams from 10 to 6 (Figure 8.4a-right and 8.4b-right, respectively). In particular, this relevantly reduces the traffic load on both the dispenser and its neighbors.

Let us clarify that traditional IP multicast and broadcast mechanisms are unsuitable for these collaborative mobile scenarios. They would require the proactive configuration of operating system routing tables on intermediary nodes, updated and modified whenever the exploited network topology varies, with the consequent frequent and resource-consuming transmissions of both monitoring data and operating system-level reconfigurations. In addition, the RAMP middleware approach can enable multimedia management with per-segment granularity, with the possibility of finegrained quality tailoring, especially on the intermediaries close to low-performance links. Moreover, traditional IP multicast/broadcast cannot be easily used because, in our targeted environments, collaborative nodes typically reside in different and uncoordinated IP subnets.

In the previous lines, for the sake of clarity, we have described only the “static” behavior of our solution, exploited at receiver start-up. In addition to that, our proposal optimizes resource exploitation also by dynamically reconfiguring multimedia streams at runtime. For instance, if our approach were only static, in the case of a new Smart Splitter becoming active on an intermediary node, already active receivers would keep using their “old” Multimedia Dispenser and could not benefit from new Smart Splitter availability (Figure 8.5a); only new receivers could be able to take advantage of it (such as H₃ and H₄ in Figure 8.4b). On the contrary, to fully exploit dynamic node cooperation, RAMP enables Dynamic Harvesters and Smart Splitters to autonomously reconfigure/manage paths during provisioning. In particular, Dynamic Harvesters periodically look for other sources of requested multimedia programs: whenever they find a closer source along the activated path (either a Multimedia Dispenser or a Smart Splitter), they update their path to the new source. For instance, harvesters in Figure 8.5b can use S as their source.

8.4.2 Distributed and Collaborative Quality Tailoring

Cooperating nodes actively work to maximize the stream quality perceived by final users, by mimicking the behavior of components in a feedback-based self-regulating system (Figure 8.6):

• The input is the original stream

• The output is the tailored stream (via proper selective video frame dropping) sent by the sender to the receiver

• The monitor evaluates the received stream error (defined on the basis of percentage of lost packets and jitter) and tries to achieve a good trade-off between stream error and quality (e.g., video frame rate) by dynamically modifying the number of dropped frames

Before going into deeper details, let us note that there is a different tailor/monitor pair of middleware components for each path segment. In fact, while Multimedia Dispenser and Dynamic Harvester provide only tailoring and monitoring features, respectively, Smart Splitter includes both, the former on the sending side and the latter on the receiving one. Thus, we can evaluate the performance of each segment and tailor each stream separately. In addition, by increasing the number of splitters, the stream is divided into a larger set of shorter segments, thus enabling the desired granularity in estimating where bandwidth bottlenecks are and allowing quality downscaling only where required.

8.4.2.1 Per-Segment Quality Metric

We have experimentally found that user-received quality is affected mainly by two factors: the number of lost/unordered packets and the regularity of packet reception (Luan et al. 2010). While the former can be easily computed by monitoring the sequence number of received RTP headers, the correct determination of the latter deserves more attention because packet arrival regularity depends not only on path conditions (e.g., link saturation largely affects packet dispatching time) but also on packet creation time.

FIGURE 8.6
Per-segment quality tailoring.

In the simple case of constant bit rate (CBR) coding techniques, such as MPEG2, the generated amount of data per second is always the same. To estimate reception regularity, we have decided to consider the RAMP packet delay variation (rampPDV), defined as the variation of inter-packet interval times (similar to Demichelis and Chimento [2002], but considering RAMP RTP packets instead of IP ones). In particular, we define the normalized rampPDV as the ratio between rampPDV standard deviation and average RAMP inter-packet interval. However, in the more challenging case of variable bitrate coding (VBR) techniques, such as H.264, the desirable throughput varies at runtime, e.g., RTP packets have higher rates in scenes with more moving objects. Moreover, during transitions between motion/nonmotion (and vice versa) scenes, inter-RTP packet delivery time changes due to bitrate variations. To correctly estimate these cases as “regular” bitrate modifications, we define Stream Jitter (SJ) as follows:

$SJ = Normalized rampPDV - Normalized timestampPDV$ $SJ = Normalized rampPDV - Normalized timestampPDV$

(8.1)

i.e., the jitter computed for CBR coding minus the ratio among the standard deviation of RTP header timestampPDV and the average inter-packet RTP timestamps. In other words, we estimate RAMP arrival regularity by considering that sometimes RTP packets are deliberately created in a nonperiodic way. In case of CBR coding, normalized timestampPDV value is usually about 1/20 of normalized rampPDV, thus not influencing SJ (it is not zero because of small differences in packet creation times due to multithreading on collaborating nodes).

On the one hand, our experience with in-the-field prototyping and validation has suggested us to consider stream quality still fine only if the percentage of lost/unordered RTP packets (LP) is very low, e.g., less than 2%, nonnegligibly bad when LP does not guarantee stream continuity, e.g., LP greater than 8%, acceptable otherwise. On the other hand, we have experimentally verified that SJ is lower than 0.2 in lightly loaded network conditions, by ensuring very regular packet delivery and thus fine-quality perception. Instead, SJ quickly rises to values greater than 0.8 when the communication channel is heavily loaded, with packet traffic close to saturation bandwidth. Note that the adoption of the SJ indicator allows us to be independent of specific audio/video bitrates, codec techniques, and inter-packet interval, thus facilitating the general applicability of this metric to different collaborative mobile scenarios. In addition, it is worth noting that both LP and SJ are computed by exploiting only data that are already available in RTP packets, without any additional monitoring overhead and negative effect on the size of transmitted packets.

Our receiver (either Dynamic Harvester or Smart Splitter) monitors received packets in a very lightweight way in order to evaluate LP and SJ of the path segment between it and the source (either Smart Splitter or Multimedia Dispenser). In case of need and with a default period of 1.0 s, it notifies its streamer to increase/decrease the quality of the provided stream. In particular, it requests increasing/decreasing quality value of

$Δ Quality = (Δ LP * Δ SJ * 1 .5) - 1.0$ $Δ Quality = (Δ LP * Δ SJ * 1 .5) - 1.0$

(8.2)

where

$ΔLP = \frac{(1 - \tanh ((LP - 5) * 0.5))}{2}$ $ΔLP = \frac{(1 - \tanh ((LP - 5) * 0.5))}{2}$

(8.3)

$ΔSJ = \frac{(1 - \tanh ((SJ - 0.5) * 5))}{2}$ $ΔSJ = \frac{(1 - \tanh ((SJ - 0.5) * 5))}{2}$

(8.4)

In other words, if LP is lower than 2% and SJ is lower than 0.2, quality is increased by ≈ 0.5 (the available bandwidth permits to sustain higher video bitrates); if LP is greater than 8% or SJ is greater than 0.8, quality is decreased by ≈ 1.0 (network conditions call for decreasing video bitrate); and by adopting intermediate values otherwise (see Figure 8.7).

8.4.2.2 Per-Segment Quality Tailoring

Our multimedia streamer receives quality feedback by receivers and tailors the stream accordingly, with fine granularity. The solution guideline is to lower stream throughput selectively by dropping video frames (possibly losing perfect continuity when the supported collaborative path offers very low bandwidth), while preserving TS packets containing audio content. In other words, our solution does not drop RTP packets, but reduces their size discarding part of their payload, thus lowering the total bitrate. In particular, the streamer assigns each client with a quality value (QV) in the [0, 10] range (10 maximum quality, 0 minimum, default value at session startup = 10). Whenever a Dynamic Harvester sends a ΔQuality request, the sender increments/decrements QV accordingly (within the admitted value range). Thus, with the period of 1 s, our solution takes 20/10 s to move from worst/best quality to best/worst quality. In addition, QV is decreased by 2.0 if the stream source does not receive any ACK message from a given receiver for 2 s.

FIGURE 8.7
ΔQuality, depending on LP and SJ.

We classify video frames in two different groups, in relation to their role in the rendering process: self-determining (SD) and non-self-determining (NSD). SD frames are self-contained and include any information required for rendering (similar to JPEG images). On the contrary, NSD frames rely on data included in other frames (either SD or NSD). For instance, MPEG2 I frames are classified as SD ones, while MPEG2 P/B are NSD frames. As Figure 8.8 shows, depending on QV, the streamer drops video frames in a differentiated way and with different probabilities (the adopted strategy can be improved and extended with more sophisticated tailoring algorithms, outside of the scope of this chapter):

• If QV = 10.0 (maximum quality), no dropping.

• If $QV \geq 5 .0$ $QV \geq 5 .0$ (mid quality), it drops only NSD frames, with probability $1 - (QV - mid) / (\max - mid)$ $1 - (QV - mid) / (\max - mid)$ .

FIGURE 8.8
The simple strategy adopted for quality tailoring in RAMP.

• If $QV < 5.0$ $QV < 5.0$ , it drops every NSD frame and SD frames with probability $1 - (mid - QV) / (mid - min)$ $1 - (mid - QV) / (mid - min)$ .

• Exceptional handling—in the unusual situation of very low QV (lower than 1.0), it drops every frame and 90% of SD frames.

In other words, QV variations modify the dropping probability of SD/NSD video frames. Bounding QV variations to a maximum of two units per second permits to modify the video quality slowly, thus gracefully decreasing/increasing the quality perceived by final users without any abrupt disruption. At the same time, stream throughput varies gradually, permitting to the receiver quality monitor to assess whether the current bitrate fits the current channel capabilities. This quality adaptation strategy (with gradually varying quality modifications) smoothes bouncing effects and reduces useless quality adaptation efforts. In addition, as widely recognized, decreasing quality faster than increasing it (fast-decrease, slow-increase) permits to quickly react to bandwidth degradation and to properly tune bandwidth allocation in the case of multiple clients that perceive bandwidth growth concurrently (Allman et al. 2009). Finally, note that quality tailoring is performed with per-segment granularity: one sender can provide the same multimedia stream along different segments with different quality levels based on differentiated quality feedback from receivers.

8.4.3 Performance Evaluation

To present a rapid evaluation of our middleware performance, in the following we consider the simple basic case of one subpath composed of two segments, where a Multimedia Dispenser on node D provides a remote harvester (node H) with multimedia streams, while an intermediate node I dispatches packets between node D and node H. Node D–node I link is based on ad hoc IEEE 802.11g, while node I–node H exploits infrastructured IEEE 802.11b. This scenario represents a typical segmentation case of end-to-end multimedia streaming in spontaneous networking scenarios and permits to have a quantitative but simple overview of the management operations (and related performance) possibly performed in any path segment.

The employed video bitrate is 768 kbps (VBR/H.264 video codec, GOP size of 4), and the audio bitrate is 64 kbps (MPEG-1 Layer 1 audio codec), transmitted via RTP/MPEG-TS; RTP maximum size is 5 KB and, thus, each RTP packet includes at most 27 TS packets. The overall maximum throughput requested for each stream (audio/video bitrate plus RAMP/RTP/MPEG-TS overhead) is about 1050 kbps. Additional experimental results, e.g., based on other multimedia samples and exploiting the less challenging CBR/MPEG2 video codec, are available on our companion website http://lia.deis.unibo.it/Research/RAMP.

FIGURE 8.9
Node D outgoing throughput (a), received quality on node H with (b) and without (e) monitoring/tailoring, messages sent by harvester to dispenser (c), and quality value assigned by dispenser to harvester (d).

Figure 8.9 shows the significant performance improvements that our middleware can achieve in the case of limited bandwidth availability and with quality monitoring/tailoring activated on node H/node D, respectively (always an active harvester on node H). Figure 8.9a shows node D outgoing throughput without (interval 1) and with (interval 2) bandwidth limitation at 850 kbps. Figure 8.9b and e report the quality received by node H harvester in terms of SJ and LP, exploiting and not exploiting our monitoring/tailoring mechanism, respectively. Figure 8.9c illustrates the ΔQuality messages sent by receiver to sender. Figure 8.9d shows the QV assigned by dispenser to harvester. During interval 1, the RTP stream reaches node H steadily, as shown by SJ and LP values, which are rather low and stable. When bandwidth is limited to 850 kbps (interval 2), node H is notified of a quality degradation event (α, high LP value) and requires node D to greatly lower QV (β); usable SJ values are not available due to insufficient RTP packets received in this interval. After a brief interruption (31 nonconsecutive RTP packets lost in about 2.5 s), the harvester starts receiving the stream again.

Let us stress that now the exchanged RTP packets include only a small subset of TS packets, i.e., every audio and video SD frames and only part of video NSD frames because some of the latter have been discarded by our tailoring middleware (experimented latency of less than 0.03 ms per RTP packet in case of RTP 5088B size). Therefore, on the one hand, throughput is lowered enough to permit to send RTP packets without losing them; on the other hand, final users experience only a very short streaming interruption. After sending few negative ΔQuality messages, LP value gets better again, i.e., the stream bitrate goes below the bandwidth limitation. However, even if LP is zero, QV does not increase because it is limited by the relatively high value of SJ (γ); only when the video bitrate decreases under the bandwidth limitation, the SJ value goes down considerably (δ), by triggering positive ΔQuality messages.

Instead, without our middleware quality monitoring/tailoring, most RTP packets fail to reach node H, in particular in correspondence of H.264 higher bitrates, by producing severe service interruptions (see Figure 8.9e, high LP values for a long period). Let us stress that our bandwidth-shaping mechanism (based on Linux tc) drops packets with layer 2 MTU granularity (1500B): if at least one layer 2 fragment in an RTP packet is lost, the whole RTP packet is lost; for this reason, the percentage of received RTP packets is lower than expected. Then, when the bitrate of the VBR/H.264-based multimedia stream goes below the bandwidth limitation, RTP packets are correctly received (at t = 28.5).

To go into finer details about the performance of the parsing/tailoring mechanisms we have implemented, Table 8.1 and Figure 8.10 show their overhead depending on RTP packet size (set by Multimedia Dispenser prior to VLC transcoder activation at session startup). The reported experimental results have been measured by filtering out the 5% worst data, thus removing spurious measurements, which are generally due to local multithreading management and which made the performance trend less easily understandable.

Moreover, our packet parsing/tailoring mechanisms have demonstrated to be lightweight, with very limited overhead and good scalability while growing the number of simultaneous harvesters. In particular, parsing is applied only once for each program and only on Multimedia Dispensers; Smart Splitters and Dynamic Harvesters do not perform it. Instead, tailoring is performed on per-segment basis (additive overhead), on both Multimedia Dispensers and Smart Splitters. In particular, to correctly deliver an RTP packet, Multimedia Dispenser has the following constraint:

TABLE 8.1

Performance of Parsing/Tailoring Mechanisms

FIGURE 8.10
Inter-RTP interval (a), RTP parsing (b), and RTP tailoring (c) depending on RTP size.

$interRTP > parsing+ (tailoring * # segments)$ $interRTP > parsing+ (tailoring * # segments)$

In other words, the interval between successive RTP packets must be greater than parsing + tailoring overhead (the latter linearly depends on the number of served segments). Therefore, the maximum number of segments that our multimedia recasting application can serve is as follows:

$# segment < \frac{(interRTP - parsing)}{tailoring}$ $# segment < \frac{(interRTP - parsing)}{tailoring}$

By neglecting bandwidth limitation and considering a maximum RTP size of 5 kB (RAMP default value) for 5% filtering, in the worst case scenario of lower standard deviation for inter-RTP (26.417 = 43.316 -16.899) and upper standard deviation for parsing (2.861 = 1.559 + 1.302) and tailoring (0.036 = 0.028 + 0.08), each Multimedia Dispenser can serve at most 654 segments concurrently. Similar considerations apply to the Smart Splitter case, but with the notable advantage of having to take into account only tailoring overhead (no RTP packet parsing): the result is that Smart Splitter can manage at most 733 concurrent segments. These performance figures have been obtained for Intel Core2 Duo PCs with 2.00 GHz processor and 3 GB RAM; Smart Splitter role is expected to be played by nodes with less resource capabilities but, anyway, the figures provided earlier are definitely larger than what required in any realistic collaborative scenario. In addition, let us note that splitting/tailoring operations have demonstrated to be scarcely CPU-intensive, thus further ensuring that they can be effectively performed also on very resource-limited portable devices.

8.5 Conclusive Remarks

The widespread diffusion of wireless devices is pushing toward the pervasive adoption of mobile services accessed from everywhere and anytime. Multimedia streaming is both the key application in users’ demands and the most challenging service to support. On the one hand, users are willing to get on-demand multimedia content from “traditional” servers, such as web-based ones, and to generate live multimedia streams, e.g., audio/video peer-to-peer communications over IP. On the other hand, due to the inherently unreliability and limited performance of wireless links, it is very challenging to support high-quality multimedia streaming in mobile environments.

As pointed out in the chapter, the envisioned mobile deployment environments of the near future call for the adoption of powerful adaptation mechanisms coping with limited connectivity performance and ever-increasing heterogeneity of client devices. To this purpose, end-to-end solutions involve clients and servers to negotiate and properly select the stream quality that best fit the requirements of entities involved in content distribution, e.g., server load, client capabilities, and network available bandwidth. Quality negotiation can be performed either at session start-up or dynamically at provisioning time: in the former case, static constraints are taken into account, e.g., client display size/resolution; in the latter case, the increased complexity due to periodic monitoring and dynamic tailoring is largely counterbalanced by the highly valuable capability of finely tuning stream quality in a much more flexible manner. To simplify multimedia management in mobile environments, proxy-based solutions split paths between servers and clients in two segments, explicitly considering that wired and wireless segments have highly differentiated performance. The adoption of proxies permits to increase performance of legacy servers and clients, e.g., by transparently pre-fetching and caching multimedia content close to mobile devices in order to minimize delivery latency. Collaborative solutions go a step forward, distributing the burden of dynamic quality adaptation among multiple nodes; however, to limit the imposed overhead on collaborating nodes, only low-cost adaptation techniques should be exploited, e.g., avoiding to use CPU-intensive quality tailoring solutions for on-the-fly multimedia transcoding.

We envision future mobile scenarios characterized by increasing heterogeneity where the ever-growing availability of powerful mobile devices will be coupled with the spread of low-end devices. In fact, while most expensive smartphones and tablets will exhibit increasing computing capabilities, the forthcoming mobile ecosystem will probably comprise very heterogeneous devices, also including low-end market devices with limited resources, differing in terms of hardware (e.g., small/medium displays and low-consumption CPUs with reduced computing power), of software (e.g., different mobile platforms supporting different video codecs), and of connectivity, ranging from IEEE 802.11 interfaces to Wi-MAX/HSDPA ones. Moreover, mobile devices will be exploited more massively for audio/video communications, as a valuable alternative to traditional cellular-based ones. In addition to that, the increasing user expectations for multimedia contents, which have to be seamlessly delivered everywhere, will push the adoption of content adaptation techniques for dynamic stream tailoring. In fact, we claim that more traditional solutions based on the static definition and creation of a small set of versions, with differentiated quality, of the same multimedia contents will not fit the ever-increasing demand of different quality levels and the capability of dynamically switching among them in a fine-grained way.

Finally, we believe that user and node cooperation will play a relevant role in the near future. In fact, the widespread diffusion of mobile devices equipped with multiple wireless interfaces, coupled with the willingness of users to share their generated content, is paving the way to a growing market of pervasive and participatory applications. Many related technical challenges still need proper and effective solutions. Among them, there are some crucial technical aspects calling for further investigation: (a) the support of cross-layer-optimized cooperation for multimedia management in order to fulfill quality requirements and constraints while minimizing overhead on content generators, clients, and intermediary nodes; (b) the full exploitation of node cooperation not only to dispatch/adapt multimedia content but also to enable simultaneous multipath techniques and content prefetching/caching; and (c) the capability to assess user levels of collaboration in a completely decentralized way, with the main purpose of awarding most collaborative users and punishing too selfish ones, also via incentive-based mechanisms to stimulate node cooperation.

References

Adobe, HTTP dynamic streaming datasheet, http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/products/hds-dynamic-streaming/pdfs/hds_datasheet.pdf, 2010

Allman, M., V. Paxson, E. Blanton, TCP congestion control, http://tools.ietf.org/html/rfc5681, September 2009.

Bellavista, P., A. Corradi, C. Giannelli, Multi-hop multi-path cooperative connectivity guided by mobility, throughput, and energy awareness: A middleware approach, The Academy Publisher Journal of Software (SEUS 2008 special issue), 4(7), 644–653, September 2009.

Bellavista, P., A. Corradi, C. Giannelli, The real ad-hoc multi-hop peer-to-peer (RAMP) middleware: An easy-to-use support for spontaneous networking, IEEE Symposium on Computers and Communications (ISCC’10), Riccione-Rimini, Italy, June 2010.

Bellavista, P., A. Corradi, C. Giannelli, Middleware for differentiated quality in spontaneous networks, IEEE Pervasive Computing, 11(3), 64–75, March 2012.

Bellavista, P., C. Giannelli, Internet connectivity sharing in multi-path spontaneous networks: Comparing and integrating network- and application-layer approaches, Conference on Mobile Wireless Middle Ware, Operating Systems, and Applications (Mobilware 2010), Chicago, IL, pp.84–99, June–July 2010.

Burza, M., J. Kang, P. Van Der Stok, Adaptive streaming of MPEG-based audio/video content over wireless networks, Journal of Multimedia (JMM), 2(2), 17–27, April 2007.

Camp, J., E. Knightly, The IEEE 802.11s extended service set mesh networking standard, IEEE Communications Magazine, 46(8), 120–126, August 2008.

Chen, S., B. Shen, S. Wee, X. Zhang, SProxy: A caching infrastructure to support internet streaming, IEEE Transactions on Multimedia, 9(5), 1062–1072, August 2007.

Cheung, G., W. Tan, T. Yoshimura, Real-time video transport optimization using streaming agent over 3G wireless networks, IEEE Transactions on Multimedia, 7(4), 777–785, August 2005.

Demichelis, C., P. Chimento, IP packet delay variation metric for IP performance metrics (IPPM), IETF, RFC 3393, November 2002, http://www.ietf.org/rfc/rfc3393.txt

Feeney, L. M., B. Ahlgren, A. Westerlund, Spontaneous networking: An application oriented approach to ad hoc networking, IEEE Communications Magazine, 39(6), 176–181, June 2001.

Ferreira, L. S., M. D. De Amorim, L. Iannone, L. Berlemann, L. M. Correia, Opportunistic management of spontaneous and heterogeneous wireless mesh networks, IEEE Wireless Communications, 17(2), 41–46, April 2010.

Frojdh, P., U. Horn, M. Kampmann, A. Nohlgren, M. Westerlund, Adaptive streaming within the 3GPP packet-switched streaming service, IEEE Network, 20(2), 34–40, March–April 2006.

Guenkova-Luy, T., A.J. Kassler, D. Mandato, End-to-end quality-of-service coordination for mobile multimedia applications, IEEE Journal on Selected Areas in Communications, 22(5), 889–903, June 2004.

Hutter, A., P. Amon, G. Panis, E. Delfosse, M. Ransburg, H. Hellwagner, Automatic adaptation of streaming multimedia content in a dynamic and distributed environment, IEEE Conference on Image Processing (ICIP 2005), 3, 716–719, September 2005.

Ip, A.T.S., J. Liu, J.C.-S. Lui, COPACC: An architecture of cooperative proxy-client caching system for on-demand media streaming, IEEE Transactions on Parallel and Distributed Systems, 18(1), 70–83, January 2007.

Jeong, H.M., M.J. Lee, D.K. Lee, S.J. Kang, Design of home network gateway for real-time A/V streaming between IEEE1394 and ethernet, IEEE Transactions on Consumer Electronics, 53(2), 390–396, May 2007.

Jurca, D., J. Chakareski, J.-P. Wagner, P. Frossard, Enabling adaptive video streaming in P2P systems, IEEE Communications Magazine, 45(6), 108–114, June 2007.

Liu, Z., Z. Wu, P. Liu, H. Liu, Y. Wang, Layer bargaining: Multicast layered video over wireless networks, IEEE Journal on Selected Areas in Communications, 28(3), 445–455, April 2010.

Luan, T.H., L.X. Cai, X. Shen, Impact of network dynamics on user’s video quality: Analytical framework and QoS provision, IEEE Transactions on Multimedia, 12(1), 64–78, January 2010.

Mastronarde, N., D.S. Turaga, M. Van Der Schaar, Collaborative resource exchanges for peer-to-peer video streaming over wireless mesh networks, IEEE Journal on Selected Areas in Communications, 25(1), 108–118, January 2007.

Mukherjee, D., A. Said, S. Liu, A framework for fully format-independent adaptation of scalable bit streams, IEEE Transactions on Circuits and Systems for Video Technology, 15(10), 1280–1290, October 2005.

Narayanan, S.R., D. Braun, J. Buford, R.S. Fish, A.D. Gelman, A. Kaplan, R. Khandelwal, E. Shim, H. Yu, Peer-to-peer streaming for networked consumer electronics, IEEE Communications Magazine, 45(6), 124–131, June 2007.

Park, S., H. Yoon, J. Kim, A cross-layered network-adaptive HD video streaming in digital A/V home network: Channel monitoring and video rate adaptation, IEEE Transactions on Consumer Electronics, 52(4), 1245–1252, November 2006.

Salameh, H.B., M. Krunz, Channel access protocols for multihop opportunistic networks: Challenges and recent developments, IEEE Network, 23(4), 14–19, July–August 2009.

Schwarz, H., D. Marpe, T. Wiegand, Overview of the scalable video coding extension of the H.264/AVC standard, IEEE Transactions on Circuits and Systems for Video Technology, 17(9), 1103–1120, September 2007.

Shen, J., B. Han, M.-C. Yuen, W. Jia, End-to-end wireless multimedia transmission system, IEEE 60th Vehicular Technology Conference (VTC2004-Fall), Los Angeles, CA, 4, 2616–2620, September 2004.

Shiang, H.-P., M. van der Schaar, Informationally decentralized video streaming over multihop wireless networks, IEEE Transactions on Multimedia, 9(6), 1299–1313, October 2007.

Uhm, Y., M. Lee, J. Byun, Y. Kim, S. Park, Development of portable intelligent gateway system for ubiquitous entertainment and location-aware push services, IEEE Transactions on Consumer Electronics, 56(1), 70–78, February 2010.

Wang, Z.-J., S.-P. Chan, C.-W. Kok, Receiver-buffer-driven layered quality adaptation for multimedia streaming, Conference on Signals, Systems and Computers, Pacific Grove, CA, pp.1235–1239, October–November 2005.

Wi-Fi Alliance announces groundbreaking specification to support direct Wi-Fi connections between devices, Wi-Fi Alliance press release, Austin, TX, October 2009.

Wu, H., Y. Liu, Q. Zhang, Z.L. Zhang, SoftMAC: Layer 2.5 collaborative MAC for multimedia support in multihop wireless networks, IEEE Transactions on Mobile Computing, 6(1), 12–25, January 2007.

Yuan, W., K. Nahrstedt, S.V. Adve, D. L. Jones, R. H. Kravets, GRACE-1: Cross-layer adaptation for multimedia quality and battery energy, IEEE Transactions on Mobile Computing, 5(7), 799–815, July 2006.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.

Table of Contents for 8. Cooperative Video Provisioning in Mobile Wireless Environments

Create new playlist

Sign In

Sign Up

Table of Contents for
8. Cooperative Video Provisioning in Mobile Wireless Environments