During an adaptive streaming session of video content, an access latency is obtained by a client device (100A) running the adaptive streaming session in order to adapt the quality of the requested video content when necessary.
|
9. A method for use in a device configured for running at least one adaptive streaming session, the method comprising:
obtaining access latency information regarding an access latency between the device and a network access equipment by using an auxiliary connection, wherein said auxiliary connection is distinct from a current network connection being used for receiving segments of a multimedia content available at one or more qualities; and
adapting the adaptive streaming session running at said device according to the obtained access latency information by:
decreasing the quality in the case where the access latency is greater than a first value and is stable over time;
maintaining the quality in the case where the access latency is greater than the first value and is decreasing over time;
maintaining or increasing the quality in the case where access latency is smaller than a second value and stable or decreasing over time; and
decreasing the quality in the case where the access latency is smaller than the second value and is increasing over time.
1. A device configured to run at least one adaptive streaming session, the device comprising one or more processors configured to:
obtain access latency information regarding an access latency between the device and a network access equipment by using an auxiliary connection, wherein said auxiliary connection is distinct from a current network connection being used for receiving segments of a multimedia content available at one or more qualities; and
adapt the adaptive streaming session running at the device according to the obtained access latency information by:
decreasing the quality for one or more next segments of the multimedia content in the case where the access latency is greater than a first value and is stable over time;
maintaining the quality for one or more next segments of the multimedia content in the case where the access latency is greater than the first value and is decreasing over time;
maintaining or increasing for one or more next segments of the multimedia content the quality in the case where access latency is smaller than a second value and stable or decreasing over time; and
decreasing the quality for one or more next segments of the multimedia content in the case where the access latency is smaller than the second value and is increasing over time.
2. The device of
3. The device of
4. The device of
5. The device of
6. The device of
7. The device of
8. The device of
10. The method of
11. The method of
12. The method of
13. The method of
14. The method of
15. The method of
16. The method of
|
This application is the U.S. National Stage, under 35 U.S.C. § 371, of International Application No. PCT/EP2019/076349 filed Sep. 30, 2019, which claims the benefit of European Application No. 18306313.0 filed Oct. 5, 2018, the contents of which are incorporated herein by reference.
The present disclosure generally relates to the management of adaptive streaming sessions operated at a device belonging to a network.
This section is intended to introduce the reader to various aspects of art, which may be related to various aspects of the present disclosure that are described and/or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. Accordingly, it should be understood that these statements are to be read in this light, and not as admissions of prior art.
HTTP adaptive technologies are pushed by various stakeholders to allow provision of over the top audiovisual delivery in the Internet. Such technologies allow a client device to receive video in the form of small successive segments (a few seconds long), so called chunk. Each segment is requested through the HTTP protocol and may exist in different variants (so called representations), allowing the client device to choose at any time an appropriate bit rate matching the network and device constraints.
Among the HTTP adaptive streaming (HAS) protocols which are already used, the most famous are the HTTP Live Streaming (HLS) from Apple, the Silverlight Smooth Streaming (SSS) from Microsoft, the Adobe Dynamic Streaming (ADS) from Adobe and the Dynamic Adaptive Streaming over HTTP (DASH) developed by 3GPP within the SA4 group. These HTTP adaptive streaming existing techniques vary in the manifest file format, said manifest providing meta-data to describe the content options (bit rate, image dimensions, frame rate . . . ), the organization of the available representations of the content in segments, in the codecs supported, and in the content protection technologies.
In particular, when a client device wants to play an audio/video content, it first has to get such a manifest describing how this specific content can be obtained. This is done through HTTP by getting some ‘file’ from an URL. This manifest file lists the available representations of the content (in terms of bit rate and other properties) and, for each representation, the URLs that allow loading the content segments for each time slice. For Video on Demand (VoD), the entire description of the A/V content is provided, while for live content (e.g. TV content), the description covers only a short period of time and needs to be reloaded periodically to discover the new items when time passes.
Depending on its capabilities and the knowledge it has from the networking environment, the client device selects one representation (e.g. based on its bit rate) and loads the first segment(s) of content. It buffers a few segments to be able to cope with network impediments. Then, the A/V content is played from each received segments one after the other. At the same time, the client device measures the reception rate and may decide to select a higher or a lower bit rate. In such case, it just requests the next segment(s) from another representation. Every HTTP streaming technique is such that it is possible for the client device to keep a continuous playing while going from a segment with a given bit rate to the next segment with another bit rate. This way when the competing traffic on the network introduces variations on the rate at which the A/V content is received, the client device is able to react and adapt by selecting segments with a bit rate that allows maintaining the buffer filling to a secure level. Indeed, the client device tries to reach the highest possible bit rate to provide a good viewing quality to end user, while staying at a level where the rendering will not suffer from late reception of data causing macro-blocks or picture freezes.
HTTP streaming techniques (in particular their associated rate adaptation algorithm) have then in common the fact that they dynamically adapt to varying network conditions by adjusting the amount data they transfer. However, they generally take decisions disregarding the fact that other applications may use the network.
The present disclosure has been devised with the foregoing in mind.
According to one or more embodiments, there is provided a device configured to belong to a network comprising one or more devices, said device being able to run at least one ongoing adaptive streaming session relying on one current network connection for receiving segments of a multimedia content available at one or more qualities, said device comprising one or more processors configured for:
According to one or more embodiments, there is provided a method to be implemented at a device configured to belong to a network comprising one or more devices, said device being able to run at least one ongoing adaptive streaming session relying on one current network connection for receiving segments of multimedia content available at one or more qualities,
the method comprising:
According to one or more embodiments, there is provided a computer program product at least one of downloadable from a communication network and recorded on a non-transitory computer readable medium readable by at least one of computer and executable by a processor, comprising program code instructions for implementing a method to be implemented at a device configured to belong to a network comprising one or more devices, said device being able to run at least one ongoing adaptive streaming session relying on one current network connection for receiving segments of multimedia content available at one or more qualities, the method comprising:
According to one or more embodiments, there is provided a non-transitory program storage device, readable by a computer, tangibly embodying a program of instructions executable by the computer to perform a method to be implemented at a device configured to belong to a network comprising one or more devices, said device being able to run at least one ongoing adaptive streaming session relying on one current network connection for receiving segments of multimedia content available at one or more qualities, the method comprising:
The methods according to the one or more embodiments may be implemented in software on a programmable apparatus. They may be implemented solely in hardware or in software, or in a combination thereof.
Some processes implemented by elements of the one or more embodiments may be computer implemented. Accordingly, such elements may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as “circuit”, “module” or “system”. Furthermore, such elements may take the form of a computer program product embodied in any tangible medium of expression having computer usable program code embodied in the medium.
Since elements can be implemented in software, some aspects of the embodiments can be embodied as computer readable code for provision to a programmable apparatus on any suitable carrier medium. A tangible carrier medium may comprise a storage medium such as a floppy disk, a CD-ROM, a hard disk drive, a magnetic tape device or a solid state memory device and the like.
The one or more embodiments thus provide a computer-readable program comprising computer-executable instructions to enable a computer to perform above mentioned methods.
Certain aspects commensurate in scope with the disclosed embodiments are set forth below. It should be understood that these aspects are presented merely to provide the reader with a brief summary of certain forms the one or more embodiments might take and that these aspects are not intended to limit the scope of the disclosure. Indeed, the disclosure may encompass a variety of aspects that may not be set forth below.
The disclosure will be better understood and illustrated by means of the following embodiment and execution examples, in no way limitative, with reference to the appended figures on which:
Wherever possible, the same reference numerals will be used throughout the figures to refer to the same or like parts.
The following description illustrates some embodiments. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody some aspects of the embodiments and are included within their scope.
All examples and conditional language recited herein are intended for educational purposes to aid the reader in understanding the embodiments and are to be construed as being without limitation to such specifically recited examples and conditions.
Moreover, all statements herein reciting embodiments, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.
Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein represent conceptual views of illustrative circuitry embodying some aspects of the embodiments. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudocode, and the like represent various processes which may be substantially represented in computer readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
The functions of the various elements shown in the figures may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. Moreover, explicit use of the term “processor” or “controller” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (DSP) hardware, read only memory (ROM) for storing software, random access memory (RAM), and nonvolatile storage.
In the claims hereof, any element expressed as a means and/or module for performing a specified function is intended to encompass any way of performing that function including, for example, a) a combination of circuit elements that performs that function or b) software in any form, including, therefore, firmware, microcode or the like, combined with appropriate circuitry for executing that software to perform the function. It is thus regarded that any means that can provide those functionalities are equivalent to those shown herein.
In addition, it is to be understood that the figures and descriptions of the present disclosure have been simplified to illustrate elements that are relevant for a clear understanding of the present embodiments, while eliminating, for purposes of clarity, many other elements found in typical digital multimedia content delivery methods, devices and systems. However, because such elements are well known in the art, a detailed discussion of such elements is not provided herein. Embodiments herein are directed to all such variations and modifications known to those skilled in the art.
Embodiments are depicted with regard to an adaptive streaming environment to deliver a video content to a client device through a delivery network.
As shown in the exemplary embodiment of
The client device 100A (e.g. a set-top box as shown in
As shown in the example of
As an example, the client device 100A, 100B is a portable media device, a mobile phone, a tablet or a laptop, a head mounted device, a TV set, a set-top box or the like. Naturally, the client device 100A, 100B might not comprise a complete video player, but only some sub-elements such as the ones for demultiplexing and decoding the media content and might rely upon an external means to display the decoded content to the end user. In this case, the client device 100A, 100B is a HTTP Adaptive Streaming capable video decoder, such as the set-top box as shown in the
Besides, in an embodiment, the gateway 200 can be, for instance, a Digital Subscriber Line (DSL) gateway, providing an Internet broadband access to the local network 10 through the DSL technology. Of course, the gateway could be any type of broadband gateway such as cable, fiber or wireless.
As shown in
As shown in the embodiment of
As illustrated in
Thus, when a web session is run by a web browser operated by a second client device 100B belonging to the local network 10, data packets exchanged during the web navigation over a distinct network connection 500B can encounter the same buffers as the ones used for the network connection 500A supporting an adaptive streaming session (i.e. such buffers are shared between applications). Since known streaming algorithms disregard other applications (such a web browser) that may use the network, they tend to consume as much bandwidth as possible, impacting negatively web navigation for instance. Packets belonging to web navigation can be queued in buffers, further buffering packets associated with the streaming session(s), so that the delivery time of packets for web navigation increases leading to a degradation of web browsing.
In one embodiment, for instance to avoid degradation of web browsing at second client device 100B, the client device 100A operating an adaptive streaming session (e.g. thanks to its streaming controller 103 and/or player 104) can implement a method 700 (shown in
According to the method 700, in a step 701, an access latency (different from segment-download latency or playback latency (commonly used to ensure that segments are delivered on-time for playback)) is obtained by the client device 100A (e.g. via the latency determination controller 109) through an auxiliary network communication 500C, 500D distinct from the network communication 500A supporting the adaptive streaming session. In an embodiment, the auxiliary connection can further be independent from the communication supporting adaptive streaming. Such an independent auxiliary connection can be established when the client device and the gateway comprise an additional WiFi network interface, independent from the one used to support the adaptive streaming session.
In an embodiment, the access latency monitored on the auxiliary connection can be defined as the Round Trip Time (RTT) between:
To obtain such an access latency, the latency determination controller 109 can send a message (such as a single data packet) and measure the time for receiving a corresponding response (i.e. the round-trip time). Different protocol can support such an implementation such as ICMP, TCP, UDP, HTTP, HTTP/2, etc.
It should be understood that the access latency considered in the present disclosure differs from a latency for delivering a segment in a streaming session corresponding to the time spent between the request for a segment and the delivery of this segment.
In an embodiment, different types of access latency (i.e. latency to the web-server of the gateway, latency to a server of the network access point, latency to a router/CDN server on Internet) can be obtained at the same time t to provide more accurate insights on the source of a potential congestion to the adaption algorithm operated by the client device 100A (e.g. at the streaming controller 103 and/or the player 104).
While the latency determination controller 109 has been described outside of the player 104 (as shown in
In a further optional step 702, the client device 100A (e.g. thanks to the controller 109, the streaming controller 103, the player 104 and/or processor(s) 105) can determine a derivative of the network latency in order to assess a trend of its evolution (such as stable, decreasing, increasing). In an illustrative but non-limitative example, an estimation of the derivative of the network latency at time t can be obtained by the following equation:
wherein:
From the obtained derivative of the access latency, the three following cases can be derived:
For example, K can correspond to a changes tolerance threshold to prevent from overreacting to small changes of latency not related to the streaming session.
In one embodiment, in a step 703, the client device 100A (e.g. via its streaming controller 103, player 104 and/or processor(s) 105) can decide whether or not the video quality of the ongoing adaptive streaming session can be adapted (in a step 704) based on the obtained access latency. To that end, the obtained access latency can be compared to a latency threshold (e.g. 200 ms), so that:
In another embodiment, the client device 100A can decide whether or not the video quality of the ongoing adaptive streaming session can be adapted (in a step 704) based on the obtained access latency and its derivative, as follows:
In further aspect, the latency threshold, the first time threshold and second time threshold can be determined from the impact the latency may have on user's experience. As a variant, these thresholds can be determined based on information from the gateway regarding the application(s) run on the device(s) (e.g. the thresholds tend to be lower for gaming applications than for web applications). In that case, thresholds are dynamically adapted according to information received from the gateway, so as to improve, for instance, video quality of the streaming session when a web application is implemented on a further device in comparison with video quality of the streaming session when a gaming application is operated.
Step 703 can be implemented by considering either the current access latency (i.e. the access latency at a time t), or the current access latency with an history of the access latency (e.g. corresponding to the values of the access latency obtained from the beginning of the streaming session, or to the values of the access latency obtained for a previous time interval).
Then, in the step 704, when the quality restriction is required by step 703, the client device will adapt the quality of the video streaming session when requesting the next segment(s) of the multimedia content (i.e. one or more qualities (i.e. representations) available at the content server 300) could not be chosen anymore by the client device 100A).
In an additional aspect, to further stabilize the video quality of the adaptive streaming session in order to avoid quality oscillation, the next increase/decrease in quality resulting from the implementation of steps 703 and 704 (especially when the obtained access latency is high and stable, or low and increasing, or low and stable/decreasing) may optionally be forced to occur only after a timeout (e.g. 10 seconds). For example, when the access latency is low and stable/decreasing, then the client device 100A can be allowed to increase the video quality of the streaming session after the timeout.
In a further aspect or complement, the method 700 can further comprise an optional step (which can be implemented within the adaptation step 704) wherein the client device 100A can assess the effect of a quality increase during the adaptive streaming session resulting from the implementation of the method 700. To that end, the current access latency is compared with past access latency(ies) (e.g. the previous obtained access latency). When the current access latency exceeds the past access latency from a given value (e.g. which depends on the considered past access latency(ies)), the quality increase determined at step 703 is not allowed to prevent from impacting negatively the access latency.
In a further embodiment, to avoid access latency from becoming high again, video qualities leading to access latency increases (e.g. observed one or several times) can be blacklisted permanently or for a predetermined time period (e.g. 30 or 60 seconds).
Thus, according to the described embodiments, the quality of the content provided at the client device 100A via the adaptive streaming session can be penalized when the latency is high and stable to allow buffers to drain, so that subsequent connections, HTTP requests, etc. can benefit from the low buffer-level (e.g. the response is received quickly). For instance, web navigation by another client device 100B of the local network 10 can be less impacted by long running download flows from an adaptive video streaming session running on another device 100A of the local network 10. Long page load time, unresponsive pages due to partially loaded resources can be avoided. Thanks to the implementation of the described method 700, the link quality for future short network connections (such as web browsing) can be preserved. In addition, because buffering is reduced, adaptive streaming players can react faster to changing conditions, impacting the video quality positively.
More generally, in an embodiment, adapting the ongoing adaptive streaming session comprises selecting a new quality for one or more next segments of the multimedia content depending on the obtained access latency.
In an embodiment, adapting the ongoing adaptive streaming session comprises selecting a new quality for one or more next segments of the multimedia content depending on the obtained access latency along with an history of the access latency.
In an embodiment, adapting the ongoing adaptive streaming session comprises comparing the obtained latency to a default value.
In an embodiment, obtaining the access latency relies on sending at least one data packet towards the network equipment to measure a Round Trip Time.
In an embodiment, a trend for the access latency is further obtained.
In an embodiment, the trend of the access latency is obtained from a derivative of the access latency.
In an embodiment, a quality leading to an increase of the access latency is blacklisted up to the end of the adaptive streaming session or for a given period of time.
In an embodiment, adapting the ongoing adaptive streaming session is implemented after a timeout.
In an embodiment, the auxiliary connection is independent from the current network connection.
References disclosed in the description, the claims and the drawings may be provided independently or in any appropriate combination. Features may, where appropriate, be implemented in hardware, software, or a combination of the two.
Reference herein to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one implementation of the method and device described. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments necessarily mutually exclusive of other embodiments.
Reference numerals appearing in the claims are by way of illustration only and shall have no limiting effect on the scope of the claims.
Although certain embodiments only of the disclosure have been described herein, it will be understood by any person skilled in the art that other modifications, variations, and possibilities of the disclosure are possible. Such modifications, variations and possibilities are therefore to be considered as falling within the spirit and scope of the disclosure and hence forming part of the disclosure as herein described and/or exemplified.
The flowchart and/or block diagrams in the Figures illustrate the configuration, operation and functionality of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, or blocks may be executed in an alternative order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of the blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. While not explicitly described, the present embodiments may be employed in any combination or sub-combination.
Besides, it is to be appreciated that the use of any of the following “/”, “and/or”, and “at least one of”, for example, in the cases of “A/B”, “A and/or B” and “at least one of A and B”, is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of both options (A and B). As a further example, in the cases of “A, B, and/or C” and “at least one of A, B, and C”, such phrasing is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of the third listed option (C) only, or the selection of the first and the second listed options (A and B) only, or the selection of the first and third listed options (A and C) only, or the selection of the second and third listed options (B and C) only, or the selection of all three options (A and B and C). This may be extended, as is clear to one of ordinary skill in this and related arts, for as many items as are listed.
Gouache, Stephane, Le Scouarnec, Nicolas, Riviere, Marc
Patent | Priority | Assignee | Title |
Patent | Priority | Assignee | Title |
9473406, | Jun 10 2011 | OPTIMORPHIX, INC | On-demand adaptive bitrate management for streaming media over packet networks |
9549189, | Nov 06 2012 | Ittiam Systems (P) Ltd. | Method for media rate control in a video encoding system |
20070097852, | |||
20070217448, | |||
20080134258, | |||
20080267069, | |||
20120307885, | |||
20130219073, | |||
20130282918, | |||
20130326024, | |||
20150058448, | |||
20150248462, | |||
20150334153, | |||
20160191594, | |||
20160205164, | |||
20160261483, | |||
20160366488, | |||
20170126527, | |||
20170126765, | |||
20180191601, | |||
20180278664, | |||
20180295144, | |||
20190141193, | |||
20200196036, | |||
20210211490, | |||
CN102843579, | |||
CN103888846, | |||
CN106412701, | |||
EP3113442, | |||
GB2565766, | |||
WO2009106015, | |||
WO2011047335, | |||
WO2011153475, | |||
WO2012175448, | |||
WO2013048484, | |||
WO2014201068, | |||
WO2017054935, | |||
WO2017027714, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Sep 30 2019 | InterDigital Madison Patent Holdings, SAS | (assignment on the face of the patent) | / | |||
Dec 17 2019 | LE SCOUARNEC, NICOLAS | INTERDIGITAL CE PATENT HOLDINGS | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 055988 | /0929 | |
Dec 18 2019 | GOUACHE, STEPHANE | INTERDIGITAL CE PATENT HOLDINGS | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 055988 | /0929 | |
Jan 07 2020 | RIVIERE, MARC | INTERDIGITAL CE PATENT HOLDINGS | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 055988 | /0929 | |
Sep 23 2022 | INTERDIGITAL CE PATENT HOLDINGS, SAS | InterDigital Madison Patent Holdings, SAS | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 062275 | /0786 |
Date | Maintenance Fee Events |
Mar 31 2021 | BIG: Entity status set to Undiscounted (note the period is included in the code). |
Date | Maintenance Schedule |
Mar 19 2027 | 4 years fee payment window open |
Sep 19 2027 | 6 months grace period start (w surcharge) |
Mar 19 2028 | patent expiry (for year 4) |
Mar 19 2030 | 2 years to revive unintentionally abandoned end. (for year 4) |
Mar 19 2031 | 8 years fee payment window open |
Sep 19 2031 | 6 months grace period start (w surcharge) |
Mar 19 2032 | patent expiry (for year 8) |
Mar 19 2034 | 2 years to revive unintentionally abandoned end. (for year 8) |
Mar 19 2035 | 12 years fee payment window open |
Sep 19 2035 | 6 months grace period start (w surcharge) |
Mar 19 2036 | patent expiry (for year 12) |
Mar 19 2038 | 2 years to revive unintentionally abandoned end. (for year 12) |