A video encoding apparatus and a corresponding method for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal for the video signal, and quantizing an obtained orthogonal transformation coefficient by using a preset quantization step size so as to encode the coefficient. A prediction error power which is a power of the prediction error signal is computed. For input information such as the computed prediction error power, the preset quantization step size, and an upper limit of an amount of code generated for the encoding target area, it is determined whether or not an amount of code generated when performing quantization using the preset quantization step size exceeds the upper limit. An encoding process is changed based on a result of the determination.
|
5. A video encoding method for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal for the video signal, and quantizing an obtained orthogonal transformation coefficient by using a preset quantization step size so as to encode the coefficient, the method comprising:
a determination step that receives information indicative of a prediction error power, the preset quantization step size, and an upper limit of an amount of code generated for the encoding target area, and determines whether or not an amount of code generated when performing quantization using the preset quantization step size exceeds the upper limit; and
a change step that changes an encoding process based on a result of the determination,
wherein the determination step applies a permissive power for the prediction error power based on the upper limit and the preset quantization step size, and compares the permissive power with the prediction error power so as to determine whether or not the amount of code generated when performing the quantization using the preset quantization step size exceeds the upper limit.
1. A video encoding apparatus for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal for the video signal, and quantizing an obtained orthogonal transformation coefficient by using a preset quantization step size so as to encode the coefficient, the apparatus comprising:
a determination circuit that receives information indicative of a prediction error power, the preset quantization step size, and an upper limit of an amount of code generated for the encoding target area, and determines whether or not an amount of code generated when performing quantization using the preset quantization step size exceeds the upper limit; and
a change circuit that changes an encoding process based on a result of the determination by the determination circuit,
wherein the determination circuit applies a permissive power for the prediction error power based on the upper limit and the preset quantization step size, and compares the permissive power with the prediction error power so as to determine whether or not the amount of code generated when performing the quantization using the preset quantization step size exceeds the upper limit.
2. The video encoding apparatus in accordance with
the determination circuit applies the permissive power for the prediction error power by setting variables of a function, which are the upper limit and the quantization step size, to the values of the upper limit and the quantization step size, where the value of the function is the permissive power.
3. The video encoding apparatus in accordance with
the determination circuit applies the permissive power for the prediction error power by referring to a table in which a relationship between data values of the upper limit, the quantization step size, and the permissive power is defined.
4. A non-transitory computer-readable storage medium which stores a video encoding program by which a computer executes an operation for implementing the video encoding apparatus in accordance with
|
The present invention relates to a video encoding apparatus and a corresponding method for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal thereof, and quantizing an obtained orthogonal transformation coefficient by using a quantization step size so as to encode the coefficient, and also relates to a video encoding program used for implementing the video encoding apparatus and a storage medium which stores the program. In particular, the present invention relates to a video encoding apparatus and a corresponding method, which do not require re-encoding or encoding which handles two or more encoding modes and implement encoding which generates codes less than an upper limit amount of code, a video encoding program used for implementing the video encoding apparatus, and a storage medium which stores the program.
Priority is claimed on Japanese Patent Application No. 2007-185374, filed Jul. 17, 2007, the contents of which are incorporated herein by reference.
In H.264 as an international coding standard, the upper limit amount of code for one macroblock is determined (see, for example, Non-Patent Document 1).
Therefore, a video encoding apparatus based on H.264 should perform encoding in a manner such that the amount of generated code generated for one macroblock does not exceed the above upper limit amount.
In order to implement the above condition, the amount of generated code is measured after encoding, and if the measured amount exceeds the upper limit, encoding should be again performed with revised encoding conditions.
However, in such a method, the amount of computation or the processing time increases due to re-encoding with revised encoding conditions.
In a proposed method for solving the above problem, encoding processes (orthogonal transformation, quantization, information source encoding, and the like) corresponding to two or more encoding modes to which different encoding conditions are assigned are simultaneously executed, and one which produces an encoding result whose amount of generated code does not exceed the relevant upper limit is selected.
However, in such a method, encoding processes corresponding to two or more encoding modes having different encoding conditions should be simultaneously executed, and an encoding result whose amount of generated code does not exceed the upper limit is not always obtained.
Therefore, in order to reliably encode each macroblock of any input image with a number of bits less than an upper limit, H.264 employs a pulse code modulation (PCM) mode in which the pixel value is directly transmitted without compression (i.e., without quantization).
In a conventional technique using the above, as shown in
On the other hand, in comparison with a conventional encoding method using a coding table, an arithmetic coding method employed in H.264 has a feature such that the amount of code cannot be instantaneously measured.
Therefore, an excess over the upper limit number of bits may be detected after the processing of the next macroblock is started. In such a situation, a problem occurs in that there is a delay in a pipeline operation (i.e., parallel execution).
Accordingly, in a hardware device for performing a pipeline operation for macroblocks (as units), if an input image of a macroblock whose number of bits exceeds an upper limit is re-encoded in the above-described PCM mode, an additional memory is necessary for storing the input image until the encoding reaches the final stage.
Therefore, in a currently-proposed technique (see, for example, Non-Patent Document 2) relating to hardware devices for performing pipeline operation for macroblocks as units, when there is a macroblock whose number of bits exceeds an upper limit, not the input image of the macroblock but a local decoded image thereof in the relevant encoder is re-encoded in the PCM mode.
As described above, in a video encoding apparatus based on H.264, encoding should be performed in a manner such that the amount of code generated for one macroblock is within a specific upper limit. In order to implement this condition, the amount of generated code is measured after an encoding process, and if the amount of generated code exceeds a specific upper limit, re-encoding may be performed with revised encoding conditions.
However, in such a method, the amount of computation or the processing time increases due to re-encoding with revised encoding conditions.
In a proposed method for solving the above problem, encoding processes corresponding to two or more encoding modes to which different encoding conditions are assigned are simultaneously executed, and the one which produces an encoding result whose amount of generated code does not exceed the relevant upper limit is selected.
However, in such a method, encoding processes corresponding to two or more encoding modes having different encoding conditions should be simultaneously executed, and an encoding result whose amount of generated code does not exceed the upper limit is not always obtained.
Therefore, in the conventional technique as shown in the above-referenced
However, in the above conventional technique, even when the amount of generated code can be reduced in comparison with the re-encoding in the PCM mode, such a possibility is disregarded.
Furthermore, the arithmetic coding method employed in H.264 has a feature such that the amount of code cannot be instantaneously measured, and thus a processing delay occurs in a hardware device which executes a pipeline operation.
In light of the above circumstances, an object of the present invention is to provide a novel image encoding technique which does not require re-encoding or encoding corresponding to two or more encoding modes, and implements an encoding whose amount of generated code does not exceed an upper limit without awaiting a measured result of the amount of generated code.
In order to achieve the object, the present invention provides a video encoding apparatus for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal for the video signal, and quantizing an obtained orthogonal transformation coefficient by using a preset quantization step size so as to encode the coefficient. The apparatus comprises:
The above-described processing devices can also be implemented by a computer program. Such a computer program may be provided by storing it in an appropriate computer-readable storage medium, or by means of a network, and can be installed and operate on a control device such as a CPU so as to implement the present invention.
Generally, amount G of generated code and quantization step size Q have the following relationship:
G=X/Q
where X is a value depending on the input signal.
In addition, for the same quantization step size Q, there is a correlation between the amount G of generated code and power D of the input signal. Therefore, in the selection of the prediction mode used in the encoding, a mode for minimizing the prediction error power is selected.
In accordance with the above relationships, an approximate amount of generated code can be estimated.
In consideration of the above, a prediction error power which is a power of the prediction error signal (as an encoding target) is computed. Based on the computed prediction error power and the quantization step size to be used in the encoding, an amount of code generated when performing the quantization using the quantization step size to be used in the encoding is estimated. The estimated value is compared with the relevant upper limit of the amount of generated code, so that it can be determined whether or not the amount of code generated when performing quantization using the quantization step size to be used in the encoding exceeds the upper limit.
In the above determination process, the amount of generated code is directly estimated. However, the determination process is equivalent to a process for determining whether or not the prediction error power is within a permissive power range defined based on the upper limit of the amount of generated code.
Therefore, in the video encoding apparatus of the present invention, a permissive power for the prediction error power is computed based on the upper limit of the amount of generated code and the quantization step size to be used in the encoding, and the permissive power is compared with the computed prediction error power so as to determine whether or not the amount of code generated when performing the quantization using the quantization step size to be used in the encoding exceeds the upper limit.
The estimated value for the amount of generated code or the permissive power for the prediction error power, which is used in the determination process, can be easily computed by means of a function or a table.
That is, it is possible to estimate the amount of generated code by setting variables of a function, which are the prediction error power and the quantization step size, to the values of the prediction error power and the quantization step size, where the value of the function is the relevant amount of generated code. It is also possible to estimate the amount of generated code by referring to a table in which a relationship between data values of the prediction error power, the quantization step size, and the relevant amount of generated code is defined.
It is also possible to compute the permissive power for the prediction error power by setting variables of a function, which are the upper limit of the amount of generated code and the quantization step size, to the values of the upper limit and the quantization step size, where the value of the function is the permissive power for the prediction error power. It is also possible to compute the permissive power for the prediction error power by referring to a table in which a relationship between data values of the upper limit of the amount of generated code, the quantization step size, and the permissive power for the prediction error power is defined.
Strictly speaking, different encoding modes (prediction modes) have different overhead amounts of code or the like, and such a function or look-up table depends on the encoding mode. Therefore, it is preferable that such a function or look-up table is provided for each encoding mode, and one suitable for the encoding mode of the encoding target area is selected and used.
If it is determined by the above determination process that the amount of code generated when performing quantization using the quantization step size to be used in the encoding exceeds the upper limit, then (i) in a first example, a quantized value of the orthogonal transformation coefficient may not be encoded, but the video signal may be encoded without quantizing the video signal; and (ii) in a second example, a quantization step size may be obtained, which is computed based on the prediction error power and the upper limit of the amount of generated code and implements generation of the amount of code which does not exceed the upper limit, and the quantization step size may be switched from the quantization step size to be used in the encoding to the obtained quantization step size.
The computation of the quantization step size used in the above switching operation is implemented using an inverse function of the function which is used for the above-described estimation of the amount of generated code. Therefore, also in this case, the relevant quantization step size can be easily computed using a function or a table.
That is, it is possible to compute the quantization step size which implements generation of the amount of code which does not exceed the upper limit, by setting variables of a function, which are the prediction error power and the upper limit of the amount of generated code, to the values of the prediction error power and the upper limit, where the value of the function is the quantization step size which implements the generation of the amount of code which does not exceed the upper limit.
It is also possible to compute the quantization step size which implements generation of the amount of code which does not exceed the upper limit of the amount of generated code, by referring to a table in which a relationship between data values of the prediction error power, the upper limit, and the quantization step size which implements the generation of the amount of code which does not exceed the upper limit is defined.
Strictly speaking, different encoding modes (prediction modes) have different overhead amounts of code or the like, and such a function or look-up table depends on the encoding mode. Therefore, it is preferable that such a function or look-up table is provided for each encoding mode, and one suitable for the encoding mode of the encoding target area is selected and used.
As described above, the present invention can be applied to an apparatus for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal for the video signal, and quantizing an obtained orthogonal transformation coefficient by using a quantization step size so as to encode the coefficient. The present invention can implement encoding which generates codes less than an upper limit amount of code, without measuring the amount of generated code. Therefore, the present invention does not require re-encoding or encoding which handles two or more encoding modes and can implement the encoding which generates codes less than the upper limit amount of code.
Additionally, As the present invention can implement an encoding whose amount of generated code does not exceed an upper limit without awaiting a measured result of the amount of generated code, no processing delay occurs in a hardware device which executes a pipeline operation.
Below, the present invention will be explained in detail in accordance with embodiments thereof.
In
Similar to conventional video encoding apparatuses based on H.264, the part 10 as the H.264 video encoding apparatus includes a motion detector 100, a motion compensator 101, a frame memory 102, an interframe prediction mode determination unit 103, an intraframe prediction mode determination unit 104, a selector switch 105, a subtractor 106, an orthogonal transformer 107, a quantizer 108, a quantization controller 109, an inverse quantizer 110, an inverse orthogonal transformer 111, an adder 112, a loop filter 113, and an information source encoder 114. After the subtracter 106 generates a prediction error signal between a video signal of an encoding target macroblock and a predicted signal thereof, the orthogonal transformer 107 subjects the generated prediction error signal to orthogonal transformation. In accordance with the quantization step size set by the quantization controller 109, the quantizer 108 quantizes orthogonal transformation coefficients obtained by the orthogonal transformation. The information source encoder 114 subjects the quantized values to entropy encoding so as to encode the video signal.
The code amount estimator 20 receives an upper limit value of the amount of code generated for the relevant macroblock (i.e., an upper limit amount of code), the prediction error signal generated by the subtractor 106, and the quantization step size set by the quantization controller 109, and includes a prediction error (electric) power computation unit 200, a generated code amount estimator 201, and a code amount comparator 202.
The prediction error power computation unit 200 computes a prediction error power, which is a power of the prediction error signal generated by the subtractor 106.
Based on the prediction error power computed by the prediction error power computation unit 200 and the quantization step size set by the quantization controller 109, the generated code amount estimator 201 estimates an amount of code generated when quantizing the encoding target macroblock by the relevant quantization step size.
The code amount comparator 202 compares the estimated amount of generated code obtained by the generated code amount estimator 201 with the upper limit (defined in H.264) for the amount of code generated for the macroblock. If the estimated amount of generated code obtained by the generated code amount estimator 201 is greater than the upper limit for the amount of code generated for the macroblock, the code amount comparator 202 directs the selector switch 22 to switch the quantization step size supplied to the quantizer 108 from the quantization step size set by the quantization controller 109 to a quantization step size computed by the quantization step size computation unit 21. In contrast, if the estimated amount of generated code obtained by the generated code amount estimator 201 is smaller than or equal to the upper limit for the amount of code generated for the macroblock, the code amount comparator 202 directs the selector switch 22 to directly use the quantization step size set by the quantization controller 109 as the quantization step size supplied to the quantizer 108.
The quantization step size computation unit 21 receives the upper limit value of the amount of code generated for the relevant macroblock (i.e., the upper limit amount of code) and the prediction error signal generated by the subtractor 106, and includes a prediction error (electric) power computation unit 210 and a minimum quantization step size computation unit 211.
The prediction error power computation unit 210 computes a prediction error power, which is a power of the prediction error signal generated by the subtractor 106.
Based on the prediction error power computed by the prediction error power computation unit 210 and the upper limit of the amount of code generated for the macroblock, the minimum quantization step size computation unit 211 computes a quantization step size for implementing code amount generation which does not exceed the upper limit (i.e., a minimum quantization step size).
Based on the flowcharts, the operation of the video encoding apparatus in the present embodiment will be explained in detail.
As shown in the flowchart of
In the next step S12, it is determined whether or not the estimated amount of code generated for the relevant macroblock is greater than the upper limit defined therefor. If it is determined that the estimated amount is greater than the upper limit, the operation proceeds to step S13, where the quantization step size is changed. In the next step S14, encoding is performed using the newly-set quantization step size.
If it is determined in the determination of step S12 that the estimated amount is smaller than or equal to the upper limit, the operation directly proceeds to step S14 by skipping step S13, and encoding is performed using the currently-set quantization step size.
As shown in the flowchart of
In the above-explained step S13, as shown in the flowchart of
Accordingly, the video encoding apparatus shown in
Therefore, in accordance with the video encoding apparatus of the present embodiment, re-encoding or encoding corresponding to two or more encoding modes is unnecessary, and an encoding whose amount of generated code does not exceed an upper limit can be implemented without awaiting a measured result of the amount of generated code.
When employing the shown structure, the code amount estimator 20 receives an upper limit value of the amount of code generated for the relevant macroblock (i.e., an upper limit amount of code), the prediction error signal generated by the subtractor 106, and the quantization step size set by the quantization controller 109, and includes a prediction error power computation unit 200, a permissive prediction error power computation unit 203, and a prediction error comparator 204.
The prediction error power computation unit 200 computes a prediction error power, which is a power of the prediction error signal generated by the subtractor 106.
The permissive prediction error power computation unit 203 computes a permissive power of the prediction error power (i.e., permissive prediction error power) based on the upper limit of the amount of code generated for the macroblock and the quantization step size set by the quantization controller 109.
The prediction error comparator 204 compares the prediction error power computed by the prediction error power computation unit 200 with the permissive prediction error power computed by the permissive prediction error power computation unit 203. If the prediction error power computed by the prediction error power computation unit 200 is larger than the permissive prediction error power computed by the permissive prediction error power computation unit 203, the prediction error comparator 204 directs the selector switch 22 to switch the quantization step size supplied to the quantizer 108 from the quantization step size set by the quantization controller 109 to a quantization step size computed by the quantization step size computation unit 21. In contrast, if the prediction error power computed by the prediction error power computation unit 200 is smaller than or equal to the permissive prediction error power computed by the permissive prediction error power computation unit 203, the prediction error comparator 204 directs the selector switch 22 to directly use the quantization step size set by the quantization controller 109 as the quantization step size supplied to the quantizer 108.
Based on the flowcharts, the operation of the video encoding apparatus in this case will be explained in detail.
As shown in the flowchart of
In the next step S22, it is determined whether or not the prediction error power is larger than the permissive prediction error power. If it is determined that the prediction error power is larger than the permissive power, the operation proceeds to step S23, where the quantization step size is changed. In the next step S24, encoding is performed using the newly-set quantization step size.
Although it is not shown in the flowchart of
If it is determined in the determination of step S22 that the prediction error power is smaller than or equal to the permissive prediction error power, the operation directly proceeds to step S24 by skipping step S23, and encoding is performed using the currently-set quantization step size.
As shown in the flowchart of
In the above-explained step S23, as shown in the flowchart of
Accordingly, when the code amount estimator 20 has the structure shown in
Therefore, in accordance with the video encoding apparatus of the present embodiment, re-encoding or encoding corresponding to two or more encoding modes is unnecessary, and an encoding whose amount of generated code does not exceed an upper limit can be implemented without awaiting a measured result of the amount of generated code.
In
The PCM encoder 31 subjects the relevant video signal as an encoding target to PCM encoding, without performing quantization, and outputs the encoded data via the selector switch 32 to the information source encoder 114.
The code amount estimator 30 has a basic structure identical to that of the code amount estimator 20 in the embodiment shown in
That is, as shown in a flowchart of
The code amount estimator 30 may have the structure shown in
That is, as shown in a flowchart of
As described above, the video encoding apparatus of the present embodiment (see
Specifically, the code amount estimator 201 shown in
Strictly speaking, different prediction modes (encoding modes) have different overhead amounts of code or the like, and such a function or look-up table depends on the prediction mode. Therefore, it is preferable that such a function or look-up table is provided for each prediction mode, and one suitable for the prediction mode of the encoding target macroblock is selected and used.
That is, in a preferable example shown in
In another preferable example shown in
In another preferable example shown in
In another preferable example shown in
In another preferable example shown in
In another preferable example shown in
The present invention can be applied to a video encoding apparatus for applying orthogonal transformation to a prediction error signal between a video signal of an encoding target area and a predicted signal for the video signal, and quantizing an obtained orthogonal transformation coefficient by using a quantization step size so as to encode the coefficient. The present invention does not require re-encoding or encoding which handles two or more encoding modes and can implement encoding which generates codes less than an upper limit amount of code, without awaiting a measured result of the amount of generated code.
Shimizu, Atsushi, Nakajima, Yasuyuki
Patent | Priority | Assignee | Title |
Patent | Priority | Assignee | Title |
6404933, | Jun 02 1997 | NEC Corporation | Image encoding method and apparatus thereof |
6961375, | Feb 06 1997 | Sony Corporation | Picture coding device and method, picture transmitting device and method and recording medium |
6963608, | Oct 02 1998 | ARRIS ENTERPRISES LLC | Method and apparatus for providing rate control in a video encoder |
20020136297, | |||
20050036698, | |||
20070133892, | |||
CA2491522, | |||
JP11331850, | |||
JP200586249, | |||
JP2007158430, | |||
JP2007166039, | |||
JP9098427, | |||
KR1020070075585, | |||
RU2123769, | |||
RU2217882, | |||
RU2322770, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Jul 10 2008 | Nippon Telegraph and Telephone Corporation | (assignment on the face of the patent) | / | |||
Dec 14 2009 | SHIMIZU, ATSUSHI | Nippon Telegraph and Telephone Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 023729 | /0736 | |
Dec 14 2009 | NAKAJIMA, YASUYUKI | Nippon Telegraph and Telephone Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 023729 | /0736 |
Date | Maintenance Fee Events |
Aug 16 2019 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Aug 16 2023 | M1552: Payment of Maintenance Fee, 8th Year, Large Entity. |
Date | Maintenance Schedule |
Feb 23 2019 | 4 years fee payment window open |
Aug 23 2019 | 6 months grace period start (w surcharge) |
Feb 23 2020 | patent expiry (for year 4) |
Feb 23 2022 | 2 years to revive unintentionally abandoned end. (for year 4) |
Feb 23 2023 | 8 years fee payment window open |
Aug 23 2023 | 6 months grace period start (w surcharge) |
Feb 23 2024 | patent expiry (for year 8) |
Feb 23 2026 | 2 years to revive unintentionally abandoned end. (for year 8) |
Feb 23 2027 | 12 years fee payment window open |
Aug 23 2027 | 6 months grace period start (w surcharge) |
Feb 23 2028 | patent expiry (for year 12) |
Feb 23 2030 | 2 years to revive unintentionally abandoned end. (for year 12) |