An image coding apparatus which includes a frame memory selecting unit (35) for selecting, in response to a selection signal, an image to be continuously stored in a plurality of frame memories (9, 10) as a background image and storing the background image into the plurality of frame memories (9, 10), and a background motion compensating unit (14, 39) for performing motion compensating prediction corresponding to an input image based on the background image to generate a predicted image based on the motion compensating prediction, and an image decoding apparatus corresponding to the image coding apparatus.
|
0. 21. An image decoding apparatus, comprising:
frame memories for storing a plurality of decoded images;
motion compensation means for performing motion compensating prediction based on the decoded images stored in said frame memories to generate a motion compensated image;
decoding means for generating decoded images from the motion compensated image from said motion compensation means and a prediction error image;
an image storage controller for allocating a type of a reference image to be stored in one of said frame memories to continuously decoded image or a stationary background image based on a coding mode of the image to be decoded, which is extracted from encoded bitstream; and
background predicted image generation means for generating a background predicted image based on the stationary background image, wherein said image storage controller performs re-writing of image contents into said frame memories in response to a given control signal.
0. 1. An image coding apparatus, comprising:
frame memories for storing a plurality of decoded images;
motion compensating prediction means for performing motion compensating prediction corresponding to an input image based on the plurality of decoded images stored in said frame memories to produce a motion vector and for generating a predicted image based on the motion compensating prediction;
prediction error calculation means for calculating a difference between the predicted image generated by said motion compensating prediction means and the input image to calculate a prediction error image;
decoding means for generating the decoded images from the prediction error image calculated by said prediction error calculation means and the predicted image;
image storage controller for determining and outputting the coding mode of the image to be predicted according to an input control signal, and allocating the type of the reference image to be stored in one of said frame memories to continuously decoded image or the stationary background image based on the selected coding mode of the image to be predicted; and
background motion compensation means for performing motion compensating prediction corresponding to the input image based on the background image to generate a motion vector and generating a predicted image based on the motion compensating prediction, wherein said image storage controller performs re-writing of image contents into said frame memories in response to a given control signal.
0. 2. An image coding apparatus according to
0. 3. An image coding apparatus according to
0. 4. An image coding apparatus according to
0. 5. An image coding apparatus according to
0. 6. An image coding apparatus according to
0. 7. An image coding apparatus according to
0. 8. An image coding apparatus according to
0. 9. An image coding apparatus according to
0. 10. An image coding apparatus according to
0. 11. An image decoding apparatus, comprising:
frame memories for storing a plurality of decoded images;
motion compensation means for performing motion compensating prediction based on the decoded images stored in said frame memories to generate a motion compensated image;
decoding means for generating coded images from the motion compensated image from said motion compensation means and a prediction error image;
an image storage controller for allocating the type of the reference image to be stored in one of said frame memories to continuously decoded image or the stationary background image based on the coding mode of the image to be decoded, which is extracted from encoded bitstream; and
background predicted image generation means for generating a background predicted image based on the background image, wherein said image storage controller performs re-writing of image contents into said frame memories in response to a given control signal.
0. 12. An image decoding apparatus according to
0. 13. An image decoding apparatus according to
0. 14. An image decoding apparatus according to
0. 15. An image decoding apparatus according to
0. 16. An image coding apparatus according to
0. 17. An image coding apparatus according to
0. 18. An image coding apparatus according to
0. 19. An image coding apparatus according to
0. 20. An image coding/decoding apparatus, comprising:
an image coding apparatus, including,
image coding frame memories for storing a plurality of decoded images;
image coding motion compensating prediction means for performing motion compensating prediction corresponding to an input image based on the plurality of decoded images stored in said image coding frame memories to produce a motion vector and for generating a predicted image based on the motion compensating prediction;
image coding prediction error calculation means for calculating a difference between the predicted image generated by said image coding motion compensating prediction means and the input image to calculate a prediction error image;
first decoding means for generating the decoded images from the prediction error image calculated by said image coding prediction error calculation means and the predicted image;
an image coding image storage controller for determining and outputting the coding mode of the image to be predicted according to an input control signal, and allocating the type of the reference image to be stored in one of said image coding frame memories to continuously decoded image or the stationary background image based on the selected coding mode of the image to be predicted; and
image coding background motion compensation means for performing motion compensating prediction corresponding to the input image based on the background image to generate a motion vector and generating a predicted image based on the motion compensating prediction; and
an image decoding apparatus, including,
image decoding frame memories for storing a plurality of decoded images;
image decoding motion compensation means for performing motion compensating prediction based on the decoded images stored in said image decoding frame memories to generate a motion compensated image;
second decoding means for generating coded images from the motion compensated image from said image decoding motion compensation means and a prediction error image;
an image decoding image storage controller for allocating the type of the reference image to be stored in one of said image decoding frame memories to continuously decoded image or the stationary background image based on the coding mode of the image to be decoded, which is extracted from encoded bitstream; and
image decoding background predicted image generation means for generating a background predicted image based on the background image,
wherein said image decoding image storage controller performs re-writing of image contents into said image decoding frame memories in response to a given control signal.
|
where “median” is an operator for calculation of a median.
Similarly, for the second frame memory,
PMV(2)=MV(2)−median (MV1(2), MV2(2), MV3(2))
For calculation of a difference vector, in addition to the operation described above, a reference motion vector PMV(3) for the third frame memory may be calculated and variable length coded.
The information generation amount of motion vectors can be supplied in such a manner as described above.
Embodiment 9
In the decoding apparatus of the present embodiment 9, a difference vector 141 variable length decoded by the variable length decoding unit 22 is added to a reference vector by the variable length decoding unit 22 to calculate a motion vector 123.
The processing following it is the same as the operation of the decoding apparatus of the embodiment 2 shown in
Embodiment 10
While, in the coding apparatus of
Referring to
In the image coding apparatus of the present embodiment 10, such coding techniques are realized by using different picture types from one another. For example, since the object 1 exhibits a comparatively large amount of motion, the construction of picture types of
On the other hand, since the object 2 is an image which exhibits little motion, it is effective to use background prediction for it. Accordingly, the construction of
Operation will be described subsequently.
An input image 100 includes identification signals applied to individual objects in advance, and the identification signals are identified by the object distinguishing unit 42. The number of each of the thus identified objects is outputted as an object identification signal 138 from the object distinguishing unit 42.
The motion estimating unit 15 selects, from among the first frame memory group 43 and the second frame memory group 44, a frame memory which corresponds to the object of the subject of coding in accordance with the object identification signal 138, reads out a reference image from the selected frame memory and performs motion prediction.
Meanwhile, the motion compensation predicting unit 21 selects a frame memory corresponding to a predetermined object in accordance with a motion prediction mode 126 determined by the motion estimating unit 15 and generates a predicted image 115.
On the other hand, the frame memory selecting unit 35 writes a decoded image 108 into one of the frame memories of a predetermined one of the frame memory groups which corresponds to a predetermined object in accordance with the object identification signal 138.
Further, the object identification signal 138 is multiplexed together with other coding information by the multiplexing unit 45 and sent out as a multiplexed bit stream 139 to an external apparatus (not shown).
While, in the image coding apparatus of the present embodiment 10, the first and second memory groups are provided to realize the construction for switching of motion compensating prediction, for implementation of the hardware, a plurality of frame memories can be provided at a time by cutting a memory having a storage capacity for the plurality of frame memories based on internal addresses. As described above, with the image coding apparatus of the present embodiment 10, since a prediction structure which conforms with motion of an object can be taken, the overall prediction efficiency is improved.
Embodiment 11
A block diagram of an image decoding apparatus which corresponds to the image coding apparatus of the embodiment 10 shown in
Operation will be described subsequently.
In response to an object identification signal 138 demultiplexed by the demultiplexing unit 46, the motion compensating unit 23 reads out a reference image from one of frame memories of a predetermined frame memory group which corresponds to a predetermined object, and performs motion compensation corresponding to a prediction mode to generate a predicted image 115.
In the meantime, the frame memory selecting unit 35 writes a decoded image 108 into one of the frame memories of a predetermined frame memory group which corresponds to a predetermined object in accordance with the object identification signal 138. The other processing is similar to that of the image decoding apparatus of the embodiment 4 shown in
Embodiment 12
Subsequently, operation will be described.
An input image 100 includes identification signals applied to individual objects in advance, and the identification signals are identified by the object distinguishing unit 42. The number of each of the thus identified objects is outputted as an object identification signal 138 from the object distinguishing unit 42.
The motion estimating unit 15 selects, from among the first frame memory group 43, the second frame memory group 44 and the third frame memory group 49, a frame memory which corresponds to the object of the subject of coding in accordance with the object identification signal 138, reads out a reference image from the selected frame memory and performs motion prediction.
Meanwhile, the motion compensation predicting unit 21 selects a frame memory corresponding to a predetermined object in accordance with a motion prediction mode 126 determined by the motion estimating unit 15 and generates a predicted image 115.
On the other hand, the frame memory selecting unit 35 writes a decoded image 108 into one of the frame memories of a predetermined one of the frame memory groups which corresponds to a predetermined object in accordance with the object identification signal 138. Further, the object identification signal 138 is multiplexed together with other coding information by the multiplexing unit 45 and sent out as a multiplexed bit stream 139.
While, in the image coding apparatus of the present embodiment 12, the first, second and third memory groups are provided to realize the construction for switching of motion compensating prediction, for implementation of the hardware, a plurality of frame memories can be provided at a time by cutting a memory having a storage capacity for the plurality of frame memories based on internal addresses.
Embodiment 13
A block diagram of an image decoding apparatus corresponding to the image coding apparatus of the embodiment 12 shown in
Operation will be described subsequently.
In response to an object identification signal 138 demultiplexed by the demultiplexing unit 46, the motion compensating unit 23 reads out a reference image from one of frame memories of a predetermined frame memory group which corresponds to a predetermined object, and performs motion compensation corresponding to a prediction mode to generate a predicted image 115.
In the meantime, the frame memory selecting unit 35 writes a decoded image 108 into one of the frame memories of a predetermined frame memory group which corresponds to a predetermined object in accordance with the object identification signal 138.
The other processing is similar to that of the image decoding apparatus of the embodiment 11 shown in
Embodiment 14
The image coding apparatus such as embodiment 12 shown in
Further, where re-writing of image contents of a region, in which an object of a subject of coding is included, of a frame memory corresponding to the object in the third frame memory in which a decoded image of the object in the past is stored is performed after each certain interval of time or in response to a control signal from the outside, the writing operation into a frame memory in the second frame memory group in the foregoing description should be applied to the writing operation into a frame memory in the third frame memory group.
Also with a decoding apparatus which corresponds to the image coding apparatus such as embodiment 12 shown in
Embodiment 15
The image coding apparatus of the embodiment 10 shown in
For example, in the image coding apparatus of the embodiment 10 shown in
Further, the image coding apparatus of the embodiment 12 shown in
For example, in the image coding apparatus of the embodiment 12 shown in
As described above, since searching ranges for a motion vector are set separately from each other for a plurality of frame memory groups referred to by objects. for example, for an object which exhibits a comparatively small amount of motion, the information generation amount of motion vectors can be reduced by making the searching range for a motion vector narrow.
Embodiment 16
Subsequently, operation will be described.
The motion estimating unit 15 performs motion estimation of a current image 101 of an object of a subject of coding using an image in a frame memory corresponding to the object in one of the first frame memory group and the second frame memory group selected by motion estimation as a reference image to detect a motion vector 123.
Based on the motion vector 123, the differential vector generating unit 47 selects a candidate vector (MV1, MV2 or MV3 mentioned hereinabove) from among motion vectors of the object in the past stored in the differential vector generating unit 47 and outputs a difference vector 141 of the candidate vector from the motion vector 123. The difference vector 141 is coded into a variable length codeword by the variable length coding unit 17. Accordingly, the differential vector generating unit 47 has a memory function of holding motion vectors in the past separately for certain periods of time for the individual frame memory groups.
Embodiment 17
A block diagram of a decoding apparatus corresponding to the image coding apparatus of the embodiment 16 shown in
Subsequently, operation will be described.
In the image decoding apparatus of the present embodiment 17, a difference vector 141 variable length coded by the variable length decoding unit 22 is supplied to the motion vector adding unit 48, by which a candidate vector is selected from among motion vectors of an object in the past stored therein and added to the difference vector 141 to regenerate a motion vector 123.
The motion vector 123 is sent to the motion compensating unit 23. The motion compensating unit 23 receives the motion vector 123, reads out an image in the memory group 43 or 44 corresponding to the object in the frame memory group selected by the frame memory selecting unit 35 as a reference image, and outputs a predicted image 115. The other processing is similar to the operation of the image decoding apparatus of the embodiment 11 shown in
Embodiment 18
A construction of an image coding apparatus which includes a third frame memory group 49 in addition to the construction of the image coding apparatus of the embodiment 16 shown in
Subsequently, operation will be described.
The motion estimating unit 15 performs motion estimation of a current image 101 of an object of a subject of coding using an image in a frame memory corresponding to the object in one of the first frame memory group, the second frame memory group and the third frame memory group selected by motion estimation as a reference image to detect a motion vector 123.
Based on the motion vector 123, the differential vector generating unit 47 selects a candidate vector (MV1, MV2 or MV3 mentioned hereinabove) from among motion vectors of the object in the past stored in the differential vector generating unit 47 and outputs a difference vector 141 of the candidate vector from the motion vector 123. The difference vector 141 is coded into a variable length codeword by the variable length coding unit 17.
Also in this instance, the differential vector generating unit 47 has a memory function of holding motion vectors in the past separately for certain periods of time for the individual frame memory groups. Since the other processing is similar to the operation of the image coding apparatus of the embodiment 16 shown in
Embodiment 19
A construction of an image decoding apparatus corresponding to the image coding apparatus of the embodiment 18 shown in
Subsequently, operation will be described.
A difference vector 141 variable length coded by the variable length decoding unit 22 is supplied to the motion vector adding unit 48, by which a candidate vector is selected from among motion vectors of an object in the past stored therein and added to the difference vector 141 to regenerate a motion vector 123. The motion vector 123 is sent to the motion compensating unit 23. The motion compensating unit 23 reads out a reference image in a frame memory corresponding to the object in the selected frame memory group, and outputs a predicted image 115.
As described above, if a differential vector generating unit which has a memory function of holding a number of motion vectors, which is equal to the number of the frame memory groups, in the past separately for certain periods of time for the individual frame memory groups and calculates a difference vector between a detected motion vector and a candidate vector is provided, then the information generation amount of motion vectors can be suppressed.
As described above, with the image coding apparatus of the present invention, since a background image is stored and motion compensating prediction is performed using background prediction based on the stored background image, there is an effect that coding can be performed while keeping a high prediction efficiency without being influenced by a coding sequence.
Further, with the image coding apparatus and the image decoding apparatus of the present invention, since re-writing of image contents in the individual frame memories is performed in units of a picture after a certain interval of time or in response to a control signal from the outside, there is another effect that the image contents of the frame memories can always be kept to contents with which a high prediction efficiency in background prediction can be obtained.
Further, with the image coding apparatus and the image decoding apparatus of the present invention, since re-writing of the image contents of the individual frame memories is performed in units of a macroblock after a certain interval of time or in response to a control signal from the outside, there is a further effect that the image contents of the frame memories can always be kept to contents with which a high prediction efficiency in background prediction can be obtained with a finer level.
Further, with the image coding apparatus and the image decoding apparatus of the present invention, since the searching ranges for a motion vector to be used for motion estimation are variably set for the plurality of frame memories provided in the coding apparatus, for example, when motion is to be searched for from reference to a frame memory in which a screen which involves a comparatively small amount of motion is written, a comparatively short code can be given, and accordingly, there is a still further effect that the coding information amount of motion vectors can be reduced.
Further, with the image coding apparatus and the image decoding apparatus of the present invention, since the differential vector generating unit which has a memory function of holding a number of motion vectors, which is equal to the number of the frame memories, in the past separately for a certain period of time and calculates a difference vector between a detected motion vector and a candidate vector is provided, there is a yet further effect that the information generation amount of motion vectors can be suppressed.
Further, with the image coding apparatus and the image decoding apparatus of the present invention, since motion compensating prediction is performed using the plurality of frame memories for the individual objects which construct a screen, a prediction structure conforming to motion of the objects can be taken, and consequently, there is a yet further effect that the overall prediction efficiency is improved.
Further, with the image coding apparatus and the image decoding apparatus of the present invention, since only regions of the frame memories in the frame memory groups in which an object of a subject of coding is included are re-written after a certain interval of time or in response to an external control signal, there is a yet further effect that a high efficiency in background prediction can be maintained.
Further, with the image coding apparatus and the image decoding apparatus of the present invention, since the searching ranges for a motion vector are set separately for the plurality of frame memory groups referred to by an object, there is a yet further effect that, for example, for an object which exhibits a comparatively small amount of motion, the information generation amount of motion vectors can be reduced by making the searching range for a motion vector narrow.
Furthermore, with the image coding apparatus and the image decoding apparatus of the present invention, since the differential vector generating unit which has a memory function of holding a number of motion vectors, which is equal to the number of the frame memory groups, in the past separately for certain periods of time for the individual frame memory groups and calculates a difference vector between a detected motion vector and a candidate vector is provided, there is an additional effect that the information generation amount of motion vectors can be suppressed.
The invention being thus described, it will be obvious that the same may be varied in many ways. Such variations are not to be regarded as a departure from the spirit and scope of the invention, and all such modifications as would be obvious to one skilled in the art are intended to be included within the scope of the following claims.
Murakami, Tokumichi, Fukuhara, Takahiro, Asai, Kohtaro, Sekiguchi, Shunichi
Patent | Priority | Assignee | Title |
Patent | Priority | Assignee | Title |
4591909, | Apr 20 1983 | Nippon Telegraph & Telephone Corporation | Interframe coding method and apparatus therefor |
4689671, | Jun 27 1985 | NEC Corporation | Coding apparatus for moving object image |
4833535, | Feb 04 1987 | Kabushiki Kaisha Toshiba | Image transmission apparatus |
5367629, | Dec 18 1992 | SHAREVISION TECHNOLOGY, INC , A CORPORATION OF CA | Digital video compression system utilizing vector adaptive transform |
5436666, | May 21 1993 | HEATHWAY HOLDINGS, LLC | Limited-domain motion estimation/compensation for video encoding/decoding |
5592228, | Mar 04 1993 | Kabushiki Kaisha Toshiba | Video encoder using global motion estimation and polygonal patch motion estimation |
EP658053, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Apr 30 2004 | Mitsubishi Denki Kabushiki Kaisha | (assignment on the face of the patent) |
Date | Maintenance Fee Events |
Date | Maintenance Schedule |
Dec 24 2016 | 4 years fee payment window open |
Jun 24 2017 | 6 months grace period start (w surcharge) |
Dec 24 2017 | patent expiry (for year 4) |
Dec 24 2019 | 2 years to revive unintentionally abandoned end. (for year 4) |
Dec 24 2020 | 8 years fee payment window open |
Jun 24 2021 | 6 months grace period start (w surcharge) |
Dec 24 2021 | patent expiry (for year 8) |
Dec 24 2023 | 2 years to revive unintentionally abandoned end. (for year 8) |
Dec 24 2024 | 12 years fee payment window open |
Jun 24 2025 | 6 months grace period start (w surcharge) |
Dec 24 2025 | patent expiry (for year 12) |
Dec 24 2027 | 2 years to revive unintentionally abandoned end. (for year 12) |