systems and methods for streaming and playing back video having a variety of resolutions, frame rates, and/or sample aspect ratios, where the video streams are encoded at one of a number of maximum bit rate levels, in accordance with embodiments of the invention are disclosed. One embodiment includes a processor, and storage containing data relating combinations of resolution and frame rates to maximum bitrates, where a plurality of resolution and frame rates that are related to the same maximum bitrate form a maximum bitrate level. In addition, an encoding application configures the processor to encode a video stream as a plurality of video streams having different resolutions and frame rates, where the target maximum bitrate used during the encoding is selected based upon the maximum bitrate levels of the resolution and frame rate combinations indicated within the data relating combinations of resolution and frame rates to maximum bitrates.

Patent
   10931982
Priority
Aug 30 2011
Filed
Feb 12 2020
Issued
Feb 23 2021
Expiry
Aug 30 2032

TERM.DISCL.
Assg.orig
Entity
Large
4
588
currently ok
9. A method for encoding source content as a set of streams that are each stored in separate container files, each container files comprising a set of one or more alternative video streams, each having a frame rate and a bitrate, the method comprising:
identifying a plurality of sections of the source content;
identifying a plurality of resolution and frame rate combinations for encoding of video content;
determining an optimal target maximum bitrate for each of the received plurality of resolution and frame rate combinations, wherein an optimal target maximum bitrate is the lowest bitrate at which a highest video quality criterion is satisfied for a specific resolution and frame rate combination;
for each particular bitrate group of a plurality of bitrate groups:
identifying, based on a relationship between the optimal target maximum bitrates and the particular bitrate group, a set of resolution and frame rate combinations from the plurality of resolution and frame rate combinations;
for each particular section of the plurality of sections, encoding the particular section to produce a plurality of encodings of the particular section, wherein the plurality of encodings is encoded at one of the set of resolution and frame rate combinations and a bitrate associated with the particular bitrate group;
assembling the encodings of the plurality of sections for a given video stream from the plurality of alternative video streams a container file for each of the plurality of alternative video streams by:
assembling the encodings of the plurality of sections for a given video stream from the plurality of alternative video streams into a video container file; and
generating entries in a container index within the video container file indicating sizes of portions of the given video stream within the video container file; and
uploading the container files for each of the plurality of alternative video streams to a set of servers that form part of a content distribution system.
1. A content distribution system comprising:
a set of one or more encoding servers, wherein each server of the set of encoding servers comprises:
a non-volatile storage containing an encoder application; and
at least one processor;
wherein the encoder applications cause the set of encoding servers to encode source content as a set of streams that are each stored in separate container files, each container files comprising a set of one or more alternative video streams, each having a frame rate and a bitrate, by causing the processor to perform the steps of:
identifying a plurality of sections of the source content;
identifying a plurality of resolution and frame rate combinations for encoding of video content;
determining an optimal target maximum bitrate for each of the received plurality of resolution and frame rate combinations, wherein an optimal target maximum bitrate is the lowest bitrate at which a highest video quality criterion is satisfied for a specific resolution and frame rate combination;
for each particular bitrate group of a plurality of bitrate groups:
identifying, based on a relationship between the optimal target maximum bitrates and the particular bitrate group, a set of resolution and frame rate combinations from the plurality of resolution and frame rate combinations;
for each particular section of the plurality of sections, encoding the particular section to produce a plurality of encodings of the particular section, wherein the plurality of encodings is encoded at one of the set of resolution and frame rate combinations and a bitrate associated with the particular bitrate group;
assembling the encodings of the plurality of sections for a given video stream from the plurality of alternative video streams a container file for each of the plurality of alternative video streams by:
assembling the encodings of the plurality of sections for a given video stream from the plurality of alternative video streams into a video container file; and
generating entries in a container index within the video container file indicating sizes of portions of the given video stream within the video container file; and
uploading the container files for each of the plurality of alternative video streams to a set of servers that form part of a content distribution system.
2. The content distribution system of claim 1, wherein the optimal target maximum bitrates are determined using a formula.
3. The content distribution system of claim 1, wherein the optimal target maximum bitrate of a resolution and frame rate combination is determined using experimentation.
4. The content distribution system of claim 1, wherein identifying a set of resolution and frame rate combinations comprises identifying resolution and frame rate combinations with optimal target maximum bitrates within a predetermined percentage of the highest optimal target maximum bitrate of a resolution and frame rate combination within the particular bitrate group.
5. The content distribution system of claim 1, wherein the resolution and frame rate combinations within a bitrate group have optimal target maximum bitrates that are within a predetermined percentage of the optimal target maximum bitrate of at least one other resolution and frame rate combination within the maximum bitrate level.
6. The content distribution system of claim 1, wherein both a width and a height of each resolution of the plurality of resolution and frame rate combinations have an integer number of pixels.
7. The content distribution system of claim 1, wherein each resolution of the plurality of resolution and frame rate combinations has a same display aspect ratio.
8. The content distribution system of claim 7, wherein at least two resolutions of the plurality of resolution and frame rate combinations have different sample aspect ratios.
10. The method of claim 9, wherein the optimal target maximum bitrates are determined using a formula.
11. The method of claim 9, wherein the optimal target maximum bitrate of a resolution and frame rate combination is determined using experimentation.
12. The method of claim 9, wherein identifying a set of resolution and frame rate combinations comprises identifying resolution and frame rate combinations with optimal target maximum bitrates within a predetermined percentage of the highest optimal target maximum bitrate of a resolution and frame rate combination within the particular bitrate group.
13. The method of claim 9, wherein the resolution and frame rate combinations within a bitrate group have optimal target maximum bitrates that are within a predetermined percentage of the optimal target maximum bitrate of at least one other resolution and frame rate combination within the maximum bitrate level.
14. The method of claim 9, wherein both a width and a height of each resolution of the plurality of resolution and frame rate combinations have an integer number of pixels.
15. The method of claim 9, wherein each resolution of the plurality of resolution and frame rate combinations has a same display aspect ratio.
16. The method of claim 15, wherein at least two resolutions of the plurality of resolution and frame rate combinations have different sample aspect ratios.

The current application is a continuation of U.S. patent application Ser. No. 15/922,198, filed Mar. 15, 2018, which is a continuation of U.S. patent application Ser. No. 13/600,046, filed Aug. 30, 2012, which claims priority to U.S. Provisional Patent Application No. 61/529,201, filed Aug. 30, 2011, the disclosures of which are incorporated herein by reference.

The present invention relates to the delivery of multimedia content, specifically streaming video content encoded at a variety of recommended maximum bitrates optimized for a variety of scaled display resolutions and network connection maximum data rates of playback devices receiving the streams.

The term streaming media describes the playback of media on a playback device, where the media is stored on a server and continuously sent to the playback device over a network during playback. Typically, the playback device stores a sufficient quantity of media in a buffer at any given time during playback to prevent disruption of playback due to the playback device completing playback of all the buffered media prior to receipt of the next portion of media. Adaptive bitrate streaming or adaptive streaming involves detecting the present streaming conditions (e.g. the user's network bandwidth and CPU capacity) in real time and adjusting the quality of the streamed media accordingly.

In adaptive streaming systems, the source media is typically stored on a media server as a top level index file pointing to a number of alternate streams that contain the actual video and audio data. Each stream is typically stored in one or more container files. Different adaptive streaming solutions typically utilize different index and media containers. The Matroska container is a media container developed as an open standard project by the Matroska non-profit organization of Aussonne, France. The Matroska container is based upon Extensible Binary Meta Language (EBML), which is a binary derivative of the Extensible Markup Language (XML). Decoding of the Matroska container is supported by many consumer electronics (CE) devices. The DivX Plus file format developed by DivX, LLC of San Diego, Calif. utilizes an extension of the Matroska container format, including elements that are not specified within the Matroska format.

Systems and methods in accordance with embodiments of the invention stream and play back video having a variety of resolutions, frame rates, and/or sample aspect ratios, where the video streams are encoded at one of a limited number of maximum bit rate levels. One embodiment includes a processor, and storage containing an encoding application and data relating combinations of resolution and frame rates to maximum bitrates, where a plurality of resolution and frame rates that are related to the same maximum bitrate form a maximum bitrate level. In addition, the encoding application configures the processor to encode a video stream as a plurality of video streams having different resolutions and frame rates, where the target maximum bitrate used during the encoding of the video streams is selected based upon the maximum bitrate levels of the resolution and frame rate combinations indicated within the data relating combinations of resolution and frame rates to maximum bitrates.

In a further embodiment, the resolution and frame rates within a maximum bitrate level have optimal target maximum bitrates that are within a predetermined percentage of the highest optimal target maximum bitrate of a resolution and frame rate combination within the maximum bitrate level.

In another embodiment, the optimal target maximum bitrate of a resolution and frame rate combination is determined using a formula.

In a still further embodiment, the optimal target maximum bitrate of a resolution and frame rate combination is determined using experimentation.

In still another embodiment, the resolution and frame rates within a maximum bitrate level have optimal target maximum bitrates that are within a predetermined percentage of the optimal target maximum bitrate of at least one other resolution and frame rate combination within the maximum bitrate level.

In a yet further embodiment, the data relating combinations of resolution and frame rates to maximum bitrate levels relates the resolution and frame rates to maximum bitrates in the manner illustrated in FIG. 5.

Yet another embodiment includes a plurality of playback devices, where the playback devices have different display aspect ratios and utilize different types of network connections, and a streaming system comprising a plurality of servers on which a plurality of video streams are stored, where the plurality of video streams are encoded at different resolutions and frame rates. In addition, video streams stored on the streaming system are encoded using one of a plurality of maximum bitrate levels, where video streams having different resolutions and frame rates are encoded at the same maximum bitrate level, the playback devices are configured to support playback of video data encoded at maximum bitrates corresponding to a plurality of the maximum bitrate levels, the playback devices are configured to request video data from at least one video stream from the streaming system encoded at a maximum bitrate level supported by the playback device, and the playback devices are configured to playback video data encoded at a supported maximum bitrate level received from the streaming system.

A further embodiment again also includes a source encoder including storage containing data relating combinations of resolution and frame rates to maximum bitrate levels. In addition, the source encoder is configured to encode a video stream as a plurality of video streams having different resolutions and frame rates, where the target maximum bitrate used during the encoding of the video streams is selected based upon the maximum bitrate levels of the resolution and frame rate combinations indicated within the data relating combinations of resolution and frame rates to maximum bitrates levels.

In another embodiment again, the data relating combinations of resolution and frame rates to maximum bitrate levels relates the resolution and frame rates to maximum bitrate levels in the manner illustrated in FIG. 5.

A further additional embodiment includes a machine readable medium containing processor instructions, where execution of the instructions by a processor causes the processor to perform a process for determining a set of maximum bitrate levels for encoding video data for use in a streaming system including playback devices having different display aspect ratios and network connections. In addition, the process includes selecting a set of resolution and frame rate combinations, determining the optimal target maximum bitrate for encoding a stream of video data at each resolution and frame rate combination, grouping the resolution and frame rate combinations into maximum bitrate levels based upon the optimal target maximum bitrates of the resolution and frame rate combinations utilizing at least one grouping criterion, and determining a target maximum bitrate for the resolution and frame rate combinations in a maximum bitrate level.

In another additional embodiment, the optimal target maximum bitrate of a resolution and frame rate combination is determined using a formula.

In a still yet further embodiment, the optimal target maximum bitrate of a resolution and frame combination is determined using experimentation.

In still yet another embodiment, the at least one grouping criterion groups resolution and frame rates within a maximum bitrate level that have optimal target maximum bitrates that are within a predetermined percentage of the highest optimal target maximum bitrate of a resolution and frame rate combination within the maximum bitrate level.

In a still further embodiment again, the at least one grouping criterion groups resolution and frame rates within a maximum bitrate level that have optimal target maximum bitrates that are within a predetermined percentage of the optimal target maximum bitrate of at least one other resolution and frame rate combination within the maximum bitrate level.

In still another embodiment again, determining a target maximum bitrate for the resolution and frame rate combinations in a maximum bitrate level includes selecting the maximum optimal target bitrate of a resolution and frame rate combination within the maximum bitrate level.

In a still further additional embodiment, determining a target maximum bitrate for the resolution and frame rate combinations in a maximum bitrate level includes selecting the minimum optimal target bitrate of a resolution and frame rate combination within the maximum bitrate level.

In still another additional embodiment, determining a target maximum bitrate for the resolution and frame rate combinations in a maximum bitrate level includes selecting the average optimal target bitrate of the resolution and frame rate combinations within the maximum bitrate level.

A yet further embodiment again also includes rounding the average optimal target bitrate using a predetermined rounding function.

In yet another embodiment again, determining a target maximum bitrate for the resolution and frame rate combinations in a maximum bitrate level includes selecting one of a plurality known network data rates based upon the optimal target data rate of at least one of the resolution and frame rate combinations within the maximum bitrate level.

FIG. 1 is a network diagram of a level bitrate adaptive streaming system in accordance with an embodiment of the invention.

FIG. 2 conceptually illustrates a playback device configured to deliver video data encoded utilizing target maximum bitrate levels in accordance with an embodiment of the invention.

FIG. 3 is a flow chart illustrating a process for determining a set of bitrates given video data in accordance with an embodiment of the invention.

FIG. 4 is a flow chart illustrating a process for determining a set of encoding maximum bitrate levels in accordance with an embodiment of the invention.

FIG. 5 is a table illustrating an example of a set of resolutions with encoded bitrates matched to maximum profile bitrates in accordance with an embodiment of the invention.

FIG. 6 is a table illustrating an example of sets of resolutions for different aspect ratios, where the resolutions in each set are encoded so that each aspect ratio is encoded at a resolution falling within each of the maximum bitrate levels in accordance with an embodiment of the invention.

Turning now to the drawings, systems and methods for streaming and playing back video having a variety of resolutions, frame rates, and/or sample aspect ratios, where the video streams are encoded at one of a limited number of maximum bit rate levels, in accordance with embodiments of the invention are illustrated. Video data is typically encoded to achieve a target maximum bitrate. The quality of video encoded with a specific resolution, and frame rate typically does not improve appreciably beyond a specific maximum bitrate threshold. Beyond that threshold, increasing the resolution of the encoded video can increase video quality. A formula can be used to recommend an optimal target maximum bitrate at which to encode video content at a specific resolution and frame rate efficiently and with the highest video quality achievable at the selected resolution. Alternatively, the bitrate threshold at which video quality does not appreciably improve can be determined through testing. While it can be beneficial to determine the optimal maximum bitrate at which to encode a specific piece of video content to efficiently achieve the highest quality video at a specific resolution and frame rate, the delivery of a large number of streams of video content to a large variety of playback devices can be simplified by specifying a limited number of maximum bitrates at which video can be encoded irrespective of the optimal target maximum bitrate for a specific frame rate and resolution combination. The ability of a specific type of playback device to playback video encoded at a specific maximum bitrate is typically confirmed via testing. Therefore, the smaller the number of maximum bitrate levels supported by a streaming system, the simpler the process of testing each playback device to confirm its ability to playback video encoded at supported target maximum bitrates. Accordingly, streaming systems can utilize a set of maximum bitrate levels to simply the process of certifying playback devices for use within the streaming system. When video is encoded for distribution via the streaming system at a number of different resolutions and frame rates, the maximum bitrate levels appropriate to each of the specific resolution and frame combinations can be selected and used as the target maximum bitrates when encoding the streams of video data.

In several embodiments, the process of selecting a set of maximum bitrate levels to use when encoding video for distribution via a streaming system involves determining the most popular aspect ratios and frame rates for video content and then determining all of the streaming resolutions and frame rates at which the video content will be encoded. When the most likely resolution and frame rate combinations are identified, the optimal target maximum bitrates for each resolution and frame rate combination can be determined using an appropriate bitrate formula or testing. Bitrate formulas that can be utilized to determine a optimal target maximum bitrate at which to encode video at a specified resolution and frame rate are discussed in U.S. patent application Ser. No. 13/432,521 entitled “Systems and Methods for Encoding Alternative Streams of Video for Playback on Playback Devices having Predetermined Display Aspect Ratios and Network Connection Maximum Data Rates”, to Soroushian et al., filed Mar. 28, 2012 (the disclosure of which is incorporated by reference herein in its entirety). The optimal target maximum bitrates can then be grouped into levels and a maximum bitrate determined for each level.

In several embodiments, the resolution and frame rate combinations can be grouped into levels based upon their corresponding optimal target maximum bitrates so that all optimal target maximum bitrates within a group are within a predetermined percentage of the highest optimal target maximum bitrate within the group. In many embodiments, the resolution and frame rate combinations can be grouped into levels based upon their corresponding optimal target maximum bitrates so that each optimal target maximum bitrate within the group is within a predetermined percentage of the next highest optimal target maximum bitrate within the group. In other embodiments, any of a variety of grouping criteria can be utilized in accordance with embodiments of the invention including criteria that group resolution and frame rate combinations into a predetermined number of groups based upon their optimal target maximum bitrates.

The maximum bitrate that is assigned to the resolution and frame rate combinations at a level can be determined in any number of different ways including (but not limited to) the average bitrate of the optimal target maximum bitrates included in the group, the minimum bitrate of the optimal target maximum bitrates included in the level, and/or a typical network connection data rate that is close to one or more of the target maximum bitrates in the level. The maximum bitrate determined for each level can then be used to encode content at the resolution and frame rate combinations within the level. In several embodiments, the process of determining the maximum bitrate levels is performed in a first process and the maximum bit rate levels and the resolution and frame rate combinations that fall within each level can be provided to a source encoding system for use in the encoding of video content.

In many embodiments, the streaming system is an adaptive bitrate streaming system. Adaptive bitrate streaming systems can stream alternative streams of video data encoded at maximum bitrates. In order to provide the highest quality video experience independent of the network data rate, the adaptive bitrate streaming systems switch between the available streams of video data throughout the delivery of the video data according to a variety of factors, including, but not limited to, the available network data rate and video decoder performance. Systems and methods for switching between video streams during playback are disclosed in U.S. patent application Ser. No. 13/221,682 entitled “Systems and Methods for Adaptive Bitrate Streaming of Media Stored in Matroska Container Files Using Hypertext Transfer Protocol” to Braness et al., filed Aug. 30, 2011, the disclosure of which is incorporated by reference herein in its entirety. In adaptive bitrate streaming systems, a number of alternative streams are likely to be encoded having the same aspect ratio, but different resolutions. Systems and methods for selecting the resolutions at which to encode alternative streams of video for use in adaptive bitrate streaming systems are described in U.S. patent application Ser. No. 13/430,032 entitled “Selection of Resolutions for Seamless Resolution Switching of Multimedia Content” to Soroushian et al., filed Mar. 26, 2012 (the disclosure of which is incorporated by reference herein in its entirety). Although much of the discussion that follows references adaptive bitrate streaming systems, systems and methods in accordance with embodiments of the invention can be utilized in streaming systems that are non-adaptive. Systems and methods for streaming video at a plurality of resolution and frame rates using a smaller set of maximum bit rate levels in accordance with embodiments of the invention are discussed further below.

System Overview

A streaming system in accordance with an embodiment of the invention is illustrated in FIG. 1. The streaming system 100 includes a source encoder 106 configured to encode source video as a number of alternative video streams. Alternative streams are streams that encode the same media content in different ways. In many instances, alternative streams encode media content (such as but not limited to video) at different resolution and sample aspect ratio combinations and different maximum bitrates. In a number of embodiments, alternative video streams form sets of alternative video streams encoded with the same aspect ratio corresponding to the display aspect ratio of a set of playback devices. In many embodiments, the source encoder possesses a lookup table that indexes resolution and frame rate combinations to a predetermined set of maximum bitrate levels. The source encoder uses the lookup table to determine the target maximum bitrate at which to encode video at a specific resolution and frame rate combination.

In many embodiments, the streaming system is an adaptive bitrate streaming system and the source encoder encodes each alternative stream into a separate container file and generates a top level index file describing each of the streams contained within the container files. The top level index file and the container files are uploaded to an HTTP server 104. In other embodiments, the container files are uploaded to an HTTP server 104 and the top level index file is dynamically generated in response to a request for a specific piece of content by a playback device. In the illustrated embodiment, the source encoder is a server. In other embodiments, the source encoder can be any processing device including a processor and sufficient resources to perform the transcoding of source media (including but not limited to video, audio, and/or subtitles).

In the illustrated embodiment, playback devices include personal computers 110, CE players 108, and mobile phones 112. In other embodiments, playback devices can include consumer electronics devices such as DVD players, Blu-ray players, televisions, set top boxes, video game consoles, tablets, and other devices that are capable of connecting to a server via HTTP and playing back encoded media. In the illustrated embodiment, a variety of playback devices use HTTP or another appropriate stateless protocol to request portions of a top level index file and the container files via a network 102 such as the Internet. Prior to a playback device performing adaptive bitrate streaming using portions of media from alternative streams contained within the container files, a bandwidth probe can be performed by the playback device to determine available bandwidth. Once the bandwidth probe has been completed, the playback device can utilize data within the top level index including (but not limited to) the maximum bitrate of each of the available streams to determine the initial streams from which to commence requesting portions of encoded media as part of an adaptive streaming process.

Once playback of content from the initial set of streams commences, the playback device utilizes the top level index to perform adaptive bitrate streaming of the content in response to changes in streaming conditions. In many adaptive bitrate streaming systems, the playback device can progress through a series of operational phases in which the playback device responds differently in each phase to changes in the streaming conditions. In a number of embodiments, stability in streaming conditions or improving streaming conditions can result in a transition to a phase in which the playback device assumes stable operating conditions, buffers more content, and is less responsive to fluctuations in streaming conditions. In many embodiments, a deterioration in streaming conditions results in a stream switch to a set of streams at a lower resolution utilizing less bandwidth and resulting in the playback device transitioning to a phase in which the playback device assumes unstable operating conditions, buffers less content, and responds rapidly to variations in streaming conditions.

Source Encoders

Source encoders in accordance with embodiments of the invention are utilized to encode source content at any of a number of resolutions and frame rates at target maximum bitrates determined using a set of predetermined maximum bitrate levels. The basic architecture of a source encoder in accordance with an embodiment of the invention is illustrated in FIG. 2. The source encoder 150 includes a processor 152 in communication with memory 154. In the illustrated embodiment, the memory 154 includes an encoding application 156, video data 158, which can include (but is not limited to) source video data and encoded alternative streams of video data, and data describing the target maximum bitrates at which to encode a plurality of resolution and frame rate combinations 160. In many embodiments, the target maximum bitrates are selected from a set of predetermined maximum bitrate levels that includes a number of levels that is smaller than the number of resolution and frame rate combinations. In certain embodiments, a plurality of resolution and frame rate combinations is associated with each of a plurality of maximum bitrate levels. In several embodiments, the data concerning target maximum bitrate levels is a lookup table that is indexed by resolution and frame rate combinations and associates specific maximum bitrate levels with specific resolution and frame rate combinations. In other embodiments, any of a variety of data structures can be utilized to identify a target maximum bitrate from a predetermined set of target maximum bitrates to utilize in the encoding of a specific resolution and frame rate combination.

Although a specific architecture is illustrated in FIG. 2, any of a variety of architectures including architectures where the application and/or set of predetermined maximum bitrate levels are located on disk or some other form of storage and is loaded into memory at runtime can be utilized to implement a source encoder in accordance with an embodiment of the invention. Furthermore, any of a variety of system architectures including (but not limited) to the system architecture illustrated in FIG. 1 can be utilized to perform video delivery in accordance with embodiments of the invention. Systems and methods for determining a predetermined set of maximum bitrate levels at which to encode video at a plurality of resolution and frame combinations in accordance with embodiments of the invention are discussed further below.

Determining Target Maximum Bitrates

The initial step in determining a set of predetermined bitrates at which to encode video is determining the characteristics of the video that is likely to be streamed by a streaming system. Once the resolution and frame rate combinations of the video that is likely to be streamed are identified, the optimal target maximum bitrate of each resolution and frame rate combination can be calculated and the results used to determine a set of maximum bitrate levels at which video can be encoded for different resolution and frame rate combinations. The term optimal target maximum bitrate can be used to describe the maximum bitrate beyond which video quality at a specific resolution and frame rate does not appreciably increase. Stated another way, the optimal target maximum bitrate is the lowest bitrate at which the highest quality video can be achieved at a given resolution and frame rate. In using a term such as optimal target maximum bitrate, there is typically an acknowledgement that video quality is subjective. Therefore, there is no concept of a “true” optimal target maximum bitrate. The optimal target maximum bitrate is simply a value that can be determined through subjective experimentation. In many instances, observations concerning target maximum bitrates have resulted in the definition of functions that can be utilized to calculate an optimal target maximum bitrate based upon the resolution and frame rate. As is discussed further below, the technique for determining an optimal target maximum bitrate is not as important as the manner in which that maximum bitrates are utilized to select maximum bitrate levels.

A process for determining optimal target maximum bitrates for video streamed within an adaptive bitrate streaming system in accordance with embodiments of the invention is illustrated in FIG. 3. The process 200 commences by selecting 210 display aspect ratios and frame rates supported by the streaming system. The display aspect ratios and frame rates are typically determined based upon popular display aspect ratios and frame rates supported by playback devices that are likely to be used to stream video within the streaming system. Resolutions can be determined 212 based upon the selected display aspect ratios and, in many instances, based upon anticipated maximum data rates supported within the streaming system. Processes for determining resolutions at which to encode alternative video streams based upon the display aspect ratio of a playback device are described in U.S. patent application Ser. No. 13/430,032 entitled “Selection of Resolutions for Seamless Resolution Switching of Multimedia Content” to Soroushian et al., filed Mar. 26, 2012 (incorporated by reference above).

An optimal target maximum bitrate is determined (214) for each selected frame rate and resolution combination. In several embodiments, maximum bitrate targets are determined 214 utilizing a bitrate formula where the bitrate formula may utilize various aspects of the video data, including frame rate, resolution, and other pertinent information as input. Examples of appropriate bitrate formulas are described in U.S. patent application No. “Systems and Methods for Encoding Alternative Streams of Video for Playback on Playback Devices having Predetermined Display Aspect Ratios and Network Connection Maximum Data Rates”, to Soroushian et al., filed Mar. 28, 2012 (incorporated by reference above). In a number of embodiments, the determination (214) of the optimal target maximum bitrate for a specific resolution and frame rate combination may involve subjective or objective quality evaluation methods of the video data encoded at the target maximum bitrate. Once the optimal target maximum bitrates for each of the resolution and frame rate combinations is determined, a process can then be performed to identify a set of maximum bitrate can be selected based upon the optimal target maximum bitrates. Processes for determining a set of maximum bitrate levels based upon a set of optimal target maximum bitrates associated with specific resolution and frame rate combinations in accordance with embodiments of the invention are discussed below.

Generating Sets of Maximum Bitrate Levels

A process for determining a set of maximum bitrate levels based upon a set of optimal target maximum bitrates associated with specific resolution and frame rate combinations in accordance with an embodiment of the invention is illustrated in FIG. 4. The process 300 involves setting (310) the current bitrate to the highest optimal target maximum bitrate from a set of optimal target maximum bitrates. The level maximum bitrate is updated (312) utilizing the current bitrate. In many embodiments, the level maximum bitrate may be set to the average, median, minimum, or maximum combination of the bitrates in the current bitrate level. In a number of embodiments, the level average bitrate is a linear or non-linear combination of the bitrates in the current bitrate level. In other embodiments, the level maximum bitrate can be determined after all of the levels are identified.

The current bitrate is compared (314) to the next lower optimal target maximum bitrate in the input set of optimal target maximum bitrates. A decision (316) is then made. Given a threshold value, if the difference between the current bitrate and the next lower optimal target maximum bitrate does not exceed the threshold value, the current bitrate is included in the optimal maximum bitrate level. In other embodiments, any of a variety of criteria can be utilized to determine whether to include a specific resolution and frame rate combination in a level based upon the optimal target maximum bitrate for the resolution and frame combination. Accordingly, the current bitrate is set (318) to the next lower optimal target maximum bitrate and the process loops back to updating (312) the level maximum bitrate. In several embodiments, the threshold value is a percentage difference between consecutive bitrate levels, such as 10%. In several embodiments, the threshold value is a fixed value or is related in a non-linear manner to the value of the current bitrate. If the threshold value is exceeded, the level maximum bitrate is recorded (320) in the set of maximum bitrate levels. In several embodiments, the level bitrate level is rounded to the closest whole number. Another decision (322) is made. If target maximum bitrates remain to be considered, a new level bitrate is started (324), the current bitrate is set (318) to the next highest optimal target maximum bitrate, and the process continues updating (312) the level maximum bitrate of the new level. The process 300 is complete when all bitrates have been considered.

In a number of embodiments, the maximum bitrate levels and the resolutions and frame rates assigned to specific groups can be modified in accordance with the requirements of a specific application. For example, the optimal maximum bitrate at each level can be rounded down (or up) to the closest whole-number multiple of 10, 100 or 1000. Although a specific process is illustrated in FIG. 4, any of a variety of processes can be utilized including a process that groups resolution and frame rate combinations into a predetermined number of levels based upon one or more criteria including (but not limited to) maximizing the similarity of the optimal target maximum bitrates of the resolution and frame combinations at each level.

An example of a grouping of a set of maximum bitrate levels defined based on the resolution, and frame rate of encoded video in accordance with an embodiment of the invention is illustrated in FIG. 5.

For example, in many embodiments of the invention 1440×816 is a sub-resolution for video data having a primary resolution of 1920×816 at 24 frames per second with a sample aspect ratio of 4:3. Likewise, the resolution 1440×800 is a sub-resolution for video data having a primary resolution of 1920×800 at 24 frames per second with a sample aspect ratio of 4:3. The encoded maximum bitrate of the 1440×816 video is 3,557 Kbps and the encoded maximum bitrate of the 1440×800 video is 3,459 Kbps. In several embodiments, 1440×816 and 1440×800 are within a threshold value of each other and no other resolutions are within the threshold value. The average bitrate for these resolutions is 3,508 Kbps, rounded to 3,500 Kbps. In accordance with an embodiment of the invention, 3,500 Kbps will be one maximum bitrate in the set of maximum profile bitrates.

In order to stream video content at each maximum bitrate level, a variety of resolutions may be utilized depending on the sample aspect ratio and frame rate of the source content. An example profile of recommended resolutions based on the frame rate and primary aspect ratio of the video content for various maximum bitrates is illustrated in FIG. 6. In many embodiments of the invention, the same resolution may be utilized for multiple frame rates and sample aspect ratios in order to maximize the subjective and/or objective quality of the streamed video data.

Although the present invention has been described in certain specific aspects, many additional modifications and variations would be apparent to those skilled in the art. It is therefore to be understood that the present invention may be practiced otherwise than specifically described, including playback devices where the set of streaming switching conditions utilized by the playback device are continuously changing, without departing from the scope and spirit of the present invention. Thus, embodiments of the present invention should be considered in all respects as illustrative and not restrictive. Accordingly, the scope of the invention should be determined not by the embodiments illustrated, but by the appended claims and their equivalents.

Soroushian, Kourosh

Patent Priority Assignee Title
11457054, Aug 30 2011 DIVX, LLC Selection of resolutions for seamless resolution switching of multimedia content
11611785, Aug 30 2011 DIVX, LLC Systems and methods for encoding and streaming video encoded using a plurality of maximum bitrate levels
11638033, Jan 05 2011 DIVX, LLC Systems and methods for performing adaptive bitrate streaming
11785066, Dec 31 2012 DIVX, LLC Systems, methods, and media for controlling delivery of content
Patent Priority Assignee Title
10148989, Jun 15 2016 DivX CF Holdings LLC; DIVX, LLC Systems and methods for encoding video content
10645429, Aug 30 2011 DIVX, LLC Systems and methods for encoding and streaming video encoded using a plurality of maximum bitrate levels
10708587, Aug 30 2011 DivX CF Holdings LLC; DIVX, LLC Systems and methods for encoding alternative streams of video for playback on playback devices having predetermined display aspect ratios and network connection maximum data rates
10798143, Aug 30 2011 DivX CF Holdings LLC; DIVX, LLC Selection of resolutions for seamless resolution switching of multimedia content
3919474,
4009331, Dec 24 1974 Goldmark Communications Corporation Still picture program video recording composing and playback method and system
4694357, Apr 24 1985 Thomson-CSF Broadcast, Inc. Apparatus and method for video signal processing
4802170, Apr 29 1987 MATROX ELECTRONICS SYSTEMS LIMITED, 1055 ST REGIS BLVD , DORVAL, QUEBEC, H9P 2T4, CANADA Error disbursing format for digital information and method for organizing same
4964069, May 12 1987 International Business Machines Corporation Self adjusting video interface
4974260, Jun 02 1989 GLOBAL 360, INC Apparatus for identifying and correcting unrecognizable characters in optical character recognition machines
5119474, Jun 16 1989 LENOVO SINGAPORE PTE LTD Computer-based, audio/visual creation and presentation system and method
5274758, Jun 16 1989 LENOVO SINGAPORE PTE LTD Computer-based, audio/visual creation and presentation system and method
5361332, Oct 09 1990 CANON KABUSHIKI KAISHA, A CORPORATION OF JAPAN Method of commonly using font information for outputting information in a system having a plurality of information processing devices
5396497, Feb 26 1993 Sony Corporation Synchronization of audio/video information
5400401, Oct 30 1992 TECH 5 SAS System and method for transmitting a plurality of digital services
5404436, Nov 27 1991 HEWLETT-PACKARD DEVELOPMENT COMPANY, L P Computer method and apparatus for converting compressed characters for display in full size
5420801, Nov 13 1992 International Business Machines Corporation System and method for synchronization of multimedia streams
5420974, Oct 15 1992 International Business Machines Corporation Multimedia complex form creation, display and editing method apparatus
5471576, Nov 16 1992 International Business Machines Corp Audio/video synchronization for application programs
5487167, Dec 31 1991 LENOVO SINGAPORE PTE LTD Personal computer with generalized data streaming apparatus for multimedia devices
5533021, Feb 03 1995 International Business Machines Corporation Apparatus and method for segmentation and time synchronization of the transmission of multimedia data
5537408, Feb 03 1995 International Business Machines Corporation apparatus and method for segmentation and time synchronization of the transmission of multimedia data
5539908, Nov 24 1992 International Business Machines Corporation Dynamically linked and shared compression/decompression
5541662, Sep 30 1994 Intel Corporation Content programmer control of video and data display using associated data
5574785, May 31 1994 Fujitsu Limited Enciphered communication system
5583652, Apr 28 1994 IBM Corporation Synchronized, variable-speed playback of digitally recorded audio and video
5600721, Jul 30 1993 Sony Corporation Apparatus for scrambling a digital video signal
5621794, Dec 06 1994 Fujitsu Limited Moving-picture encipherment method and system, and enciphered moving-picture decipherment method and system
5627936, Dec 21 1995 Intel Corporation Apparatus and method for temporal indexing of multiple audio, video and data streams
5633472, Jun 21 1994 Microsoft Technology Licensing, LLC Method and system using pathnames to specify and access audio data at fidelity levels other than the level at which the data is stored
5642171, Jun 08 1994 Dell USA, L.P. Method and apparatus for synchronizing audio and video data streams in a multimedia system
5642338, Oct 08 1993 Matsushita Electric Industrial Co., Ltd. Information recording medium and apparatus and method for recording and reproducing information
5655117, Nov 18 1994 Alcatel Lucent Method and apparatus for indexing multimedia information streams
5664044, Apr 28 1994 International Business Machines Corporation Synchronized, variable-speed playback of digitally recorded audio and video
5675382, Apr 08 1996 LOGITECH EUROPE, S A Spatial compression and decompression for video
5675511, Dec 21 1995 Intel Corporation Apparatus and method for event tagging for multiple audio, video, and data streams
5684542, Dec 21 1993 Sony Corporation Video subtitle processing system
5717394, Feb 10 1993 Ricoh Corporation Method and apparatus for encoding and decoding data
5719786, Feb 03 1993 JPMORGAN CHASE BANK, N A , AS SUCCESSOR AGENT Digital media data stream network management system
5745643, Apr 06 1995 Kabushiki Kaisha Toshiba System for and method of reproducing playback data appropriately by the use of attribute information on the playback data
5751280, Dec 11 1995 RPX Corporation System and method for media stream synchronization with a base atom index file and an auxiliary atom index file
5763800, Aug 14 1995 CREATIVE TECHNOLOGY LTD Method and apparatus for formatting digital audio data
5765164, Dec 21 1995 Intel Corporation Apparatus and method for management of discontinuous segments of multiple audio, video, and data streams
5794018, Nov 24 1993 Intel Corporation System and method for synchronizing data streams
5805228, Aug 09 1996 Hewlett Packard Enterprise Development LP Video encoder/decoder system
5805700, Oct 15 1996 Intel Corporation Policy based selective encryption of compressed video data
5813010, Apr 14 1995 Kabushiki Kaisha Toshiba Information storage and information transmission media with parental control
5822524, Jul 21 1995 Infovalue Computing, Inc. System for just-in-time retrieval of multimedia files over computer networks by transmitting data packets at transmission rate determined by frame size
5828370, Jul 01 1996 OPEN TV, INC ; OPENTV, INC Video delivery system and method for displaying indexing slider bar on the subscriber video screen
5844575, Jun 27 1996 Intel Corporation Video compression interface
5848217, Aug 02 1995 Sony Corporation Subtitle encoding/decoding method and apparatus
5854873, Sep 29 1995 MATSUSHITA ELECTRIC INDUSTRIAL CO ,L TD Method and an apparatus for encoding a bitstream with plural possible searching reproduction paths information
5903261, Jun 20 1996 DATA TRANSLATION II, INC ; DATA TRANSLATION, INC , A DELAWARE CORPORATION Computer based video system
5907597, Aug 05 1994 SMART TONE, INC , A DELWARE CORPORATION Method and system for the secure communication of data
5907658, Aug 21 1995 Matsushita Electric Industrial Co., Ltd. Multimedia optical disk, reproduction apparatus and method for achieving variable scene development based on interactive control
5912710, Dec 18 1996 Kabushiki Kaisha Toshiba System and method for controlling a display of graphics data pixels on a video monitor having a different display aspect ratio than the pixel aspect ratio
5923869, Sep 29 1995 MATSUSHITA ELECTRIC INDUSTRIAL CO , LTD Method and an apparatus for reproducing bitstream having non-sequential system clock data seamlessly therebetween
5956729, Sep 06 1996 Google Technology Holdings LLC Multimedia file, supporting multiple instances of media types, and method for forming same
5959690, Feb 20 1996 Research Triangle Institute Method and apparatus for transitions and other special effects in digital motion video
5990955, Oct 03 1997 JNC OPPORTUNITY FUND, LTD Dual encoding/compression method and system for picture quality/data density enhancement
6002834, Feb 24 1995 HITACHI CONSUMER ELECTRONICS CO , LTD Optical disk having table relating sector address and time and optical disk reproducing apparatus
6005621, Dec 23 1996 AVAGO TECHNOLOGIES GENERAL IP SINGAPORE PTE LTD Multiple resolution video compression
6009237, Feb 24 1995 Hitachi Ltd. Optical disk and optical disk reproduction apparatus
6016381, Apr 14 1995 Kabushiki Kaisha Toshiba Apparatus for reproducing information
6031622, May 16 1996 MONOTYPE IMAGING INC Method and apparatus for font compression and decompression
6046778, Oct 29 1997 Matsushita Electric Industrial Co., Ltd. Apparatus for generating sub-picture units for subtitles and storage medium storing sub-picture unit generation program
6057832, Dec 02 1997 L-3 Communications Corporation Method and apparatus for video-on-demand with fast play capability
6065050, Jun 05 1996 Oracle America, Inc System and method for indexing between trick play and normal play video streams in a video delivery system
6079566, Apr 07 1997 TRUSTEES OF COLUMBIA UNIVERSITY IN THE CITY OF NEW YORK, THE System and method for processing object-based audiovisual information
6157410, May 17 1996 Sony Corporation Processing and display of images retrieved from digital still image files generated from digital moving images
6169242, Feb 02 1999 Microsoft Technology Licensing, LLC Track-based music performance architecture
6195388, Nov 27 1997 UNILOC 2017 LLC Apparatus and method for encoding multiple video programs
6204883, Dec 21 1993 Sony Corporation Video subtitle processing system
6246803, Dec 27 1998 KANSAS, THE UNIVERSITY OF Real-time feature-based video stream validation and distortion analysis system using color moments
6266483, Oct 12 1998 Panasonic Corporation Information recording medium, apparatus and method for recording or reproducing data thereof
6282320, Mar 19 1997 Sony Corporation Video data decoding apparatus and method and video signal reproduction apparatus and method
6308005, May 15 1998 Kabushiki Kaisha Toshiba Information recording method and information reproducing method
6320905, Jul 08 1998 Integrated Device Technology, inc Postprocessing system for removing blocking artifacts in block-based codecs
6330286, Jun 09 1999 HBC SOLUTIONS, INC Flow control, latency control, and bitrate conversions in a timing correction and frame synchronization apparatus
6351538, Oct 06 1998 AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE LIMITED Conditional access and copy protection scheme for MPEG encoded video data
6373803, Feb 07 2000 Kabushiki Kaisha Toshiba Stream data generation method and partial erase processing method
6374144, Dec 22 1998 Varian Semiconductor Equipment Associates, Inc Method and apparatus for controlling a system using hierarchical state machines
6395969, Jul 28 2000 MXWORKS, INC System and method for artistically integrating music and visual effects
6415031, Mar 12 1999 Comcast IP Holdings I, LLC Selective and renewable encryption for secure distribution of video on-demand
6430354, Dec 16 1997 MAXELL, LTD Methods of recording/reproducing moving image data and the devices using the methods
6445877, Nov 19 1998 MATSUSHITA ELECTRIC INDUSTRIAL CO , LTD Information recording medium, apparatus and method for recording or reproducing data thereof
6453115, Aug 31 2000 KEEN PERSONAL MEDIA, INC ; KEEN PERSONAL TECHNOLOGIES, INC Digital video recording system which generates an index data structure for displaying a video stream in trickplay mode
6453116, Feb 18 1999 Kabushiki Kaisha Toshiba Recording medium of stream data, and recording method and playback method of the same
6481012, Oct 27 1999 TIVO CORPORATION Picture-in-picture and multiple video streams using slice-based encoding
6504873, Jun 13 1997 Nokia Technologies Oy Filtering based on activities inside the video blocks and at their boundary
6512883, Jun 28 1997 Samsung Electronics Co., Ltd. Control apparatus and method for reproducing data from a digital video disk at high speed
6594699, Oct 10 1997 ESPIAL, INC System for capability based multimedia streaming over a network
6614843, Apr 15 1999 TIVO CORPORATION Stream indexing for delivery of interactive program guide
6654933, Sep 21 1999 ESPIAL, INC System and method for media stream indexing
6658056, Mar 30 1999 Sony Corporation; Sony Electronics INC Digital video decoding, buffering and frame-rate converting method and apparatus
6665835, Dec 23 1997 VERIZON LABORATORIES, INC Real time media journaler with a timing event coordinator
6671408, Jan 27 1999 XACTI CORPORATION Motion image reproducing apparatus
6690838, Nov 30 1998 Equator Technologies, Inc. Image processing circuit and method for reducing a difference between pixel values across an image boundary
6697568, Feb 08 1999 XACTI CORPORATION Motion image recording apparatus and digital camera
6724944, Mar 13 1997 Qualcomm Incorporated Adaptive filter
6725281, Jun 11 1999 Rovi Technologies Corporation Synchronization of controlled device state using state table and eventing in data-driven remote device control model
6751623, Jan 26 1998 AT&T Corp Flexible interchange of coded multimedia facilitating access and streaming
6807306, May 28 1999 FUJI XEROX CO , LTD ; Xerox Corporation Time-constrained keyframe selection method
6813437, Jan 14 1999 Kabushiki Kaisha Toshiba Digital video recording system and its recording medium
6819394, Sep 08 1998 Sharp Kabushiki Kaisha Time-varying image editing method and time-varying image editing device
6856997, Oct 27 2000 LG Electronics Inc. Apparatus and method for providing file structure for multimedia streaming service
6859496, May 29 1998 International Business Machines Corporation Adaptively encoding multiple streams of video data in parallel for multiplexing onto a constant bit rate channel
6871006, Jun 30 2000 EMC IP HOLDING COMPANY LLC Processing of MPEG encoded video for trick mode operation
6912513, Oct 29 1999 Sony Corporation; Sony Electronics Inc. Copy-protecting management using a user scrambling key
6917652, Jan 12 2000 LG Electronics, Inc. Device and method for decoding video signal
6920179, Nov 16 1999 AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE LIMITED Method and apparatus for video transmission over a heterogeneous network using progressive video coding
6931531, Sep 02 1998 MATSUSHITA ELECTRIC INDUSTRIAL CO , LTD Image object recording, compression, and encryption method and system
6944621, Apr 21 1999 Rovi Technologies Corporation System, method and article of manufacture for updating content stored on a portable storage medium
6944629, Sep 08 1998 Sharp Kabushiki Kaisha Method and device for managing multimedia file
6956901, May 29 1998 International Business Machines Corporation Control strategy for dynamically encoding multiple streams of video data in parallel for multiplexing onto a constant bit rate channel
6957350, Jan 30 1996 DOLBY LABORATORIES, INC ; Dolby Laboratories Licensing Corporation Encrypted and watermarked temporal and resolution layering in advanced television
6970564, Jul 13 1998 Sony Corporation Data multiplexing device, program distribution system, program transmission system, pay broadcast system, program transmission method, conditional access system, and data reception device
6983079, Sep 20 2001 Seiko Epson Corporation Reducing blocking and ringing artifacts in low-bit-rate coding
6985588, Oct 30 2000 LONGBEAM TECHNOLOGIES LLC System and method for using location identity to control access to digital information
6988144, Nov 18 1999 International Business Machines Corporation Packet scheduling system and method for multimedia data
7006757, Mar 17 1999 Kabushiki Kaisha Toshiba Recording method of stream data and data structure thereof
7007170, Mar 18 2003 GOOGLE LLC System, method, and apparatus for securely providing content viewable on a secure device
7020287, Sep 30 2002 Sony Corporation; Sony Electronics, Inc. Method and system for key insertion for stored encrypted content
7127155, Jul 07 1998 Kabushiki Kaisha Toshiba Information storage system capable of recording and playing back a plurality of still pictures
7151832, Nov 18 1999 International Business Machines Corporation Dynamic encryption and decryption of a stream of data
7188183, Aug 04 2000 Oracle International Corporation Maintaining state information in mobile applications
7209892, Dec 24 1998 Universal Music Group Electronic music/media distribution system
7212726, Sep 15 2000 HULU, LLC System and method of processing MPEG streams for file index insertion
7237061, Apr 17 2003 Intel Corporation Systems and methods for the efficient reading of data in a server system
7242772, Sep 07 2000 Monument Peak Ventures, LLC Encryption apparatus and method for synchronizing multiple encryption keys with a data stream
7274861, Apr 10 2003 Panasonic Corporation Information recording medium, and apparatus and method for recording information to information recording medium
7295673, Oct 23 2002 DivX CF Holdings LLC; DIVX, LLC Method and system for securing compressed digital video
7330875, Jun 15 1999 Microsoft Technology Licensing, LLC System and method for recording a presentation for on-demand viewing over a computer network
7340528, Mar 21 2001 ADAPTFLOW TECHNOLOGIES LLC Data referencing system
7349886, Mar 25 2005 GOOGLE LLC Securely relaying content using key chains
7352956, Mar 19 1999 Thomson Licensing Method for implementing trickplay modes in a data stream recorder
7356245, Jun 29 2001 Meta Platforms, Inc Methods to facilitate efficient transmission and playback of digital information
7366788, Jan 15 1998 Apple Inc Method and apparatus for media data transmission
7382879, Jul 23 2003 T-MOBILE INNOVATIONS LLC Digital rights management negotiation for streaming media over a network
7397853, Apr 29 2004 Mediatek Incorporation Adaptive de-blocking filtering apparatus and method for MPEG video decoder
7400679, Apr 29 2004 Mediatek Incorporation Adaptive de-blocking filtering apparatus and method for MPEG video decoder
7418132, Jul 01 2003 Nikon Corporation Signal processing apparatus, signal processing program and electronic camera
7457359, Sep 26 2001 INTERACT DEVICES, INC Systems, devices and methods for securely distributing highly-compressed multimedia content
7457415, Aug 20 1998 Akikaze Technologies, LLC Secure information distribution system utilizing information segment scrambling
7478325, Apr 22 2005 Microsoft Technology Licensing, LLC Methods for providing an accurate visual rendition of a text element formatted with an unavailable font
7493018, May 19 1999 Method for creating caption-based search information of moving picture data, searching and repeating playback of moving picture data based on said search information, and reproduction apparatus using said method
7499930, Jan 28 2004 Canon Kabushiki Kaisha Data processing apparatus, data processing method, and data processing program
7499938, Apr 21 1999 Rovi Technologies Corporation Controlling playback of content stored on a portable storage medium
7546641, Feb 13 2004 Microsoft Technology Licensing, LLC Conditional access to digital rights management conversion
7639921, Nov 20 2002 LG Electronics Inc. Recording medium having data structure for managing reproduction of still images recorded thereon and recording and reproducing methods and apparatuses
7640435, Mar 18 2003 GOOGLE LLC System, method, and apparatus for securely providing content viewable on a secure device
7711052, May 15 2000 Nokia Technologies Oy Video coding
7728878, Dec 17 2004 Mitsubishi Electric Research Laboratories, Inc Method and system for processing multiview videos for view synthesis using side information
7853980, Oct 31 2003 Sony Corporation; Sony Electronics Inc. Bi-directional indices for trick mode video-on-demand
7864186, Aug 07 2000 DIGIMEDIA TECH, LLC Device-specific content versioning
7869691, Sep 26 2002 KONNINKLIJKE PHILIPS ELECTRONICS, N V Apparatus for recording a main file and auxiliary files in a track on a record carrier
7945143, Jul 23 2001 Panasonic Corporation Information recording medium, and apparatus and method for recording information on information recording medium
8023562, Sep 07 2007 BEAMR IMAGING LTD Real-time video coding/decoding
8046453, Sep 20 2007 QURIO Holdings, Inc. Illustration supported P2P media content streaming
8054880, Dec 10 2004 BISON PATENT LICENSING, LLC Parallel rate control for digital video encoder with multi-processor architecture and picture-based look-ahead window
8131875, Nov 26 2007 Adobe Inc Device profile assignment based on device capabilities
8169916, Nov 23 2007 Media Melon, Inc. Multi-platform video delivery configuration
8225061, Jul 02 2009 Apple Inc.; Apple Inc Method and apparatus for protected content data processing
8233768, Nov 16 2007 DivX CF Holdings LLC; DIVX, LLC Hierarchical and reduced index structures for multimedia files
8243924, Jun 29 2007 GOOGLE LLC Progressive download or streaming of digital media securely through a localized container and communication protocol proxy
8249168, Nov 06 2008 Advanced Micro Devices, Inc.; Advanced Micro Devices, INC Multi-instance video encoder
8270473, Jun 12 2009 Microsoft Technology Licensing, LLC Motion based dynamic resolution multiple bit rate video encoding
8270819, Oct 31 2006 TIVO SOLUTIONS INC Performing trick play functions in a digital video recorder with efficient use of resources
8286213, May 13 2003 LG Electronics Inc. HTTP based video streaming apparatus and method in mobile communication system
8289338, Jun 06 2008 DivX CF Holdings LLC; DIVX, LLC Systems and methods for font file optimization for multimedia files
8311115, Jan 29 2009 Microsoft Technology Licensing, LLC Video encoding using previously calculated motion information
8312079, Sep 22 2009 THWAPR, INC Adaptive rendering for mobile media sharing
8321556, Jul 09 2007 CITIBANK, N A Method and system for collecting data on a wireless device
8369421, Nov 29 2001 Godo Kaisha IP Bridge 1 Coding distortion removal method by selectively filtering based on a pixel difference
8386621, Mar 12 2010 NETFLIX, INC. Parallel streaming
8396114, Jan 29 2009 Microsoft Technology Licensing, LLC Multiple bit rate video encoding using variable bit rate and dynamic resolution for adaptive video streaming
8456380, May 15 2008 LENOVO INTERNATIONAL LIMITED Processing computer graphics generated by a remote computer for streaming to a client computer
8649669, Jan 05 2011 DivX CF Holdings LLC; DIVX, LLC Systems and methods for performing smooth visual search of media encoded for adaptive bitrate streaming via hypertext transfer protocol using trick play streams
8683066, Aug 06 2007 ECHOSTAR TECHNOLOGIES L L C Apparatus, system, and method for multi-bitrate content streaming
8767825, Nov 30 2009 GOOGLE LLC Content-based adaptive video transcoding framework
8768984, Nov 06 2009 TELEFONAKTIEBOLAGET L M ERICSSON PUBL Media container file management
8782268, Jul 20 2010 Microsoft Technology Licensing, LLC Dynamic composition of media
8818171, Aug 30 2011 DivX CF Holdings LLC; DIVX, LLC Systems and methods for encoding alternative streams of video for playback on playback devices having predetermined display aspect ratios and network connection maximum data rates
8819116, Mar 08 2010 Amazon Technologies, Inc Providing services using a device capabilities service
8832297, Jun 29 2011 DivX CF Holdings LLC; DIVX, LLC Systems and methods for performing multiphase adaptive bitrate streaming
8849950, Apr 07 2011 Qualcomm Incorporated Network streaming of video data using byte range requests
8914534, Jan 05 2011 DivX CF Holdings LLC; DIVX, LLC Systems and methods for adaptive bitrate streaming of media stored in matroska container files using hypertext transfer protocol
9021119, Jan 06 2011 DivX CF Holdings LLC; DIVX, LLC Systems and methods for performing adaptive bitrate streaming based upon the delay of each stream and the channel rate
9025659, Jan 05 2011 DivX CF Holdings LLC; DIVX, LLC Systems and methods for encoding media including subtitles for adaptive bitrate streaming
9038116, Dec 28 2009 AKAMAI TECHNOLOGIES, INC Method and system for recording streams
9197944, Aug 23 2012 Disney Enterprises, Inc. Systems and methods for high availability HTTP streaming
9215466, Jan 31 2011 Apple Inc Joint frame rate and resolution adaptation
9344721, Jun 23 2011 Saab AB Best quality given a maximum bit rate
9350990, Feb 28 2013 NLD Holdings I, LLC Systems and methods of encoding multiple video streams with adaptive quantization for adaptive bitrate streaming
9357210, Feb 28 2013 NLD Holdings I, LLC Systems and methods of encoding multiple video streams for adaptive bitrate streaming
9467708, Aug 30 2011 DivX CF Holdings LLC; DIVX, LLC Selection of resolutions for seamless resolution switching of multimedia content
9510031, Aug 30 2011 DivX CF Holdings LLC; DIVX, LLC Systems and methods for encoding alternative streams of video for playback on playback devices having predetermined display aspect ratios and network connection maximum data rates
9571827, Jun 08 2012 Apple Inc Techniques for adaptive video streaming
9661049, Oct 29 2014 DLVR, INC. Systems and methods of providing interspersed manifest file specifications for adaptive streaming video
9712890, May 30 2013 DivX CF Holdings LLC; DIVX, LLC Network video streaming with trick play based on separate trick play files
9955195, Aug 30 2011 DivX CF Holdings LLC; DIVX, LLC Systems and methods for encoding and streaming video encoded using a plurality of maximum bitrate levels
20010021276,
20010030710,
20010052077,
20010052127,
20020034252,
20020048450,
20020051494,
20020062313,
20020067432,
20020076112,
20020087569,
20020091665,
20020093571,
20020094031,
20020110193,
20020118953,
20020120498,
20020135607,
20020141503,
20020143413,
20020143547,
20020147980,
20020151992,
20020154779,
20020161462,
20020164024,
20020169971,
20020180929,
20020184159,
20020191112,
20020191960,
20030001964,
20030002577,
20030002578,
20030005442,
20030035488,
20030044080,
20030053541,
20030063675,
20030077071,
20030078930,
20030093799,
20030123855,
20030128296,
20030133506,
20030135742,
20030142594,
20030142872,
20030152370,
20030165328,
20030185302,
20030185542,
20030206558,
20030206717,
20030216922,
20030231863,
20030231867,
20030236836,
20040001594,
20040006701,
20040021684,
20040022391,
20040025180,
20040028227,
20040037421,
20040047592,
20040047607,
20040047614,
20040052501,
20040071453,
20040076237,
20040081333,
20040081434,
20040093494,
20040101059,
20040107356,
20040114687,
20040117347,
20040136698,
20040143760,
20040146276,
20040150747,
20040208245,
20040213547,
20040217971,
20040255236,
20050013494,
20050015797,
20050038826,
20050052294,
20050055399,
20050063541,
20050076232,
20050089091,
20050144468,
20050157948,
20050177741,
20050180641,
20050193070,
20050193322,
20050196147,
20050207442,
20050207578,
20050210145,
20050243912,
20050265555,
20050273695,
20050275656,
20060013568,
20060015813,
20060039481,
20060072672,
20060078301,
20060093320,
20060126717,
20060129909,
20060165163,
20060168639,
20060173887,
20060181965,
20060182139,
20060235880,
20060245727,
20060259588,
20060263056,
20060267986,
20060274835,
20070005333,
20070024706,
20070031110,
20070047645,
20070047901,
20070053293,
20070067472,
20070083467,
20070083617,
20070086528,
20070140647,
20070154165,
20070168541,
20070168542,
20070177812,
20070180051,
20070180125,
20070239839,
20070292107,
20080030614,
20080052306,
20080086570,
20080101466,
20080101718,
20080104633,
20080120330,
20080120342,
20080126248,
20080137736,
20080137847,
20080137848,
20080192818,
20080196076,
20080232456,
20080253454,
20080256105,
20080263354,
20080266522,
20080279535,
20080310496,
20090010622,
20090013195,
20090031220,
20090037959,
20090060452,
20090066839,
20090077143,
20090106082,
20090116821,
20090132599,
20090132721,
20090150557,
20090169181,
20090178090,
20090201988,
20090226148,
20090249081,
20090282162,
20090293116,
20090300204,
20090303241,
20090307258,
20090307267,
20090310819,
20090313544,
20090313564,
20090328124,
20100002069,
20100040351,
20100094969,
20100095121,
20100111192,
20100142915,
20100146055,
20100158109,
20100189183,
20100226582,
20100228795,
20100259690,
20100262712,
20100278271,
20100284473,
20110010466,
20110022432,
20110055585,
20110058675,
20110080940,
20110082924,
20110096828,
20110099594,
20110103374,
20110126104,
20110126191,
20110129011,
20110135090,
20110142415,
20110145858,
20110150100,
20110153785,
20110164679,
20110170408,
20110173345,
20110179185,
20110197261,
20110239078,
20110246659,
20110246661,
20110268178,
20110280307,
20110296048,
20110302319,
20110305273,
20110310982,
20110314130,
20110314176,
20110314500,
20120005312,
20120023251,
20120042090,
20120047542,
20120072493,
20120093214,
20120105279,
20120110120,
20120167132,
20120170642,
20120170643,
20120170906,
20120170915,
20120173751,
20120177101,
20120179834,
20120203766,
20120269275,
20120278496,
20120281767,
20120307883,
20120311174,
20120331167,
20130007200,
20130013803,
20130044821,
20130046902,
20130051767,
20130051768,
20130054958,
20130055084,
20130058393,
20130061045,
20130080267,
20130091249,
20130095855,
20130169863,
20130191754,
20130196292,
20140003501,
20140003523,
20140140253,
20140149557,
20140211840,
20140241421,
20140250473,
20140355958,
20140359680,
20150104153,
20150229695,
20150288530,
20160127440,
20160134881,
20170026445,
20170041604,
20170366833,
20180278975,
20190182524,
AU2010202963,
CA2237293,
CN101461149,
CN102138327,
CN103858419,
CN103875248,
CN108989847,
CN109314784,
CN1221284,
CN1662952,
CN1723696,
CN1756359,
CN1787422,
EP1283640,
EP1335603,
EP1420580,
EP1453319,
EP1718074,
EP2180664,
EP2360923,
EP2661895,
EP3473005,
EP757484,
EP813167,
HK1195183,
HK1260329,
JP11328929,
JP2001043668,
JP2002170363,
JP2002218384,
JP2003250113,
JP2005027153,
JP2009508452,
JP2019526188,
JP8163488,
JP8287613,
KR100221423,
KR101928910,
KR101936142,
KR101981923,
KR1020020064888,
KR102020764,
KR102074148,
KR102086995,
KR102140339,
KR2002013664,
KR20040039852,
KR20060106250,
KR20140056317,
RE45052, Dec 08 2003 DivX CF Holdings LLC; DIVX, LLC File format for multiple track digital data
RU2328040,
WO1995015660,
WO2000049762,
WO2000049763,
WO2001031497,
WO2001050732,
WO2002001880,
WO2003047262,
WO2004012378,
WO2004054247,
WO2004097811,
WO2004100158,
WO2004102571,
WO2005008385,
WO2005015935,
WO2009006302,
WO2009065137,
WO2009109976,
WO20100150470,
WO2010060106,
WO2010111261,
WO2010122447,
WO2011053658,
WO2011059291,
WO2011087449,
WO2011093835,
WO2011101371,
WO2011102791,
WO2011103364,
WO20120094181,
WO20120094189,
WO2012094171,
WO2013033334,
WO2013033335,
WO2013033458,
WO2017218095,
/////
Executed onAssignorAssigneeConveyanceFrameReelDoc
Nov 14 2012SOROUSHIAN, KOUROSHDIVX, LLCASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS 0549380933 pdf
Nov 21 2013DIVX, LLCSONIC IP, INC ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS 0549380948 pdf
Feb 12 2018SONIC IP, INC DivX CF Holdings LLCASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS 0549380959 pdf
Feb 12 2018DivX CF Holdings LLCDIVX, LLCCHANGE OF NAME SEE DOCUMENT FOR DETAILS 0549380984 pdf
Feb 12 2020DIVX, LLC(assignment on the face of the patent)
Date Maintenance Fee Events
Feb 12 2020BIG: Entity status set to Undiscounted (note the period is included in the code).
Aug 07 2024M1551: Payment of Maintenance Fee, 4th Year, Large Entity.


Date Maintenance Schedule
Feb 23 20244 years fee payment window open
Aug 23 20246 months grace period start (w surcharge)
Feb 23 2025patent expiry (for year 4)
Feb 23 20272 years to revive unintentionally abandoned end. (for year 4)
Feb 23 20288 years fee payment window open
Aug 23 20286 months grace period start (w surcharge)
Feb 23 2029patent expiry (for year 8)
Feb 23 20312 years to revive unintentionally abandoned end. (for year 8)
Feb 23 203212 years fee payment window open
Aug 23 20326 months grace period start (w surcharge)
Feb 23 2033patent expiry (for year 12)
Feb 23 20352 years to revive unintentionally abandoned end. (for year 12)