The present invention relates to a method, apparatus and system for optimizing an expression tree (101,902,1102) for compositing an image. Such an expression tree (101,902, 1102) can comprise at least two nodes. Each node is either a graphical element (102,104) or image compositing operator ((103,104) and has a region of the image represented by the node (102,103,104). In the method, for at least one node in the tree, several steps are carried out. The region represented by the node (103,104) is compared to a region representation data structure, which is preferably a quadtree representation, corresponding to one or more regions represented by at least one other node. A determination is then made if the region represented by the node (102,103,104) is totally or partially obscured by the one or more regions. If the region represented by the node is at least partially or totally obscured, the expression tree (101,902,1102) is modified. Modifying the expression tree (101,902,1102) involves applying a clipping operator (58,59) to the node if the region represented by the node is partially obscured. If the node is totally obscured, either removing the node if the node is a graphical element (102, 104) or applying a predetermined set of node replacement rules in accordance with the image compositing operator if the node (103) is a image compositing operator.
|
17. An apparatus for optimising an expression tree, said expression tree for compositing an image and comprising at least three nodes, each said node of said tree being at least either a graphical element or a graphical operator, the apparatus comprising:
means for, comparing a first region of said node to a second region derived from at least one other node anywhere in said expression tree;
means for determining if said first region is totally or partially obscured by said second region; and
means for modifying the expression tree in the event that said first region is at least partially or totally obscured by said second region, to form an optimized expression tree in which an optimized part of said expression tree substantially represents unobscured portions of said first region.
24. An apparatus for optimizing an expression tree for compositing an image, said expression tree comprising a plurality of nodes, each said node being at least either a graphical element or a graphical operator, said apparatus comprising:
means for traversing the expression tree node by node;
means for determining at a current node if a first region of the image represented at said current node is obscured by a second region derived from at least one other node anywhere in said expression tree; and
means for modifying said expression tree in the event that said first region of said current node is partially or totally obscured by said second region to form an optimized expression tree in which an optimized part of said expression tree substantially represents unobsured unobscured portions of said first region.
1. A method of optimising an expression tree, said expression tree for compositing an image and comprising at least three nodes, each said node of said tree being at least either a graphical element or a graphical operator, the method comprising, for at least one node in said tree, the steps of:
comparing a first region of said node to a second region derived from at least one other node anywhere in said expression tree;
determining if said first region is totally or partially obscured by said second region; and
modifying the expression tree if said first region is at least partially or totally obscured by said second region, to form an optimised expression tree in which an optimised part of said expression tree substantially represents unobscured portions of said first region,
wherein said steps are performed by means of a programmed computer.
8. A method of optimising an expression tree for compositing an image, said expression tree comprising a plurality of nodes, each said node being at least either a graphical element or a graphical operator, said method comprising the steps of:
traversing the expression tree node by node; and
determining at a current node if a first region of the image represented at said current node is obscured by a second region derived from at least one other node anywhere in said expression tree, and modifying said expression tree in the event that said first region of said current node is partially or totally obscured by said second region, to form an optimised expression tree in which an optimised part of said expression tree substantially represents unobscured portions of said first region,
wherein said steps are performed by means of a programmed computer.
27. An apparatus for optimizing an expression tree for compositing an image, said expression tree comprising a plurality of nodes, each said node comprising at least either a graphical element or a graphical operator and having a region of the image represented by said node, said apparatus comprising:
means for traversing the expression tree node by node, said traversing means for each current node comprising a graphical operator, further comprising:
means for receiving a first region representation from a parent node;
means for passing to a first operand of said graphical operator a modified first region representation in accordance with a first predetermined modification rule for said operator;
means for returning to the graphical operator a second region representation of regions obscured by a sub-tree associated with the first operand;
means for passing to a second operand of said graphical operator a modified second region representation in accordance with a second predetermined modification rule for said operator;
means for returning to the graphical operator a third region representation of regions obscured by a sub-tree associated with the second operand; and
means for determining, in accordance with a set rule for said graphical operator, a final region representation to be returned to the parent to form an optimized expression tree in which said final region representation substantially represents an unobscured portion of said first region represented at the parent node a region within which said current node is capable of obscuring other nodes in said expression tree.
11. A method of optimising an expression tree for compositing an image, said expression tree comprising a plurality of nodes, each said node comprising:
at least either a graphical element or a graphical operator and having a region of the image represented by said node, said method comprising the steps of:
traversing the expression tree node by node and at each current node comprising a graphical operator applying the sub-steps of:
(i) receiving a first region representation from a parent node;
(ii) passing to a first operand of said graphical operator a modified first region representation in accordance with a first predetermined modification rule for said operator;
(iii) returning to the graphical operator a second region representation of regions obscured by a sub-tree associated with the first operand;
(iv) passing to a second operand of said graphical operator a modified second region representation in accordance with a second predetermined modification rule for said operator;
(v) returning to the graphical operator a third region representation of regions obscured by a sub-tree associated with the second operand; and
(vi) determining, in accordance will a set rule for said graphical operator, a final region representation to be returned to the parent node to form an optimised expression tree in which said final region representation substantially represents an unobscured portion of the first region represented at the parent node a region within which said current node is capable of obscuring other nodes in said expression tree,
wherein said steps are performed by means of a programmed computer.
2. The method as recited in
3. The method as recited in
removing the node; and
if the node has a parent node which has a graphical operator, selecting a node replacement rule from a predetermined set of node replacement rules in accordance with said graphical operator and applying said rule.
4. The method as recited in
if the parent node is an “over” graphical operator and the current node is at a left branch of the parent node, replacing the parent node with a right subtree of the parent node;
if the parent node is an “over” graphic graphical operator and the current node is at a right branch of the parent node, replacing the parent node with a left subtree of the parent node;
if the parent node is an “in” graphical operator, removing the parent node and any subtrees branching off the parent node;
if the parent node is a “ratop” graphical operator and the current node is at a left branch of the parent node, removing the parent node and any subtrees branching off the parent node;
if the parent node is a “ratop” graphical operator and the current node is at a right branch of the parent node, replacing the parent node with a left subtree of the parent node;
if the parent node is an “out” graphical operator and the current node is at a left branch of the parent node, removing the parent node and any subtrees branching off the parent node;
if the parent node is an “out” graphical operator and the current node is at a right branch of the parent node, replacing the parent node with a left subtree of the parent node;
if the parent node is a “plusC” graphical operator and the current node is at a left branch of the parent node, replacing the parent node with a right subtree of the parent node;
if the parent node is a “plusC” graphical operator and the current node is at a right branch of the parent node, replacing the parent node with a left subtree of the parent node;
if the parent node is a “plusW” or an “Xor” graphical operator and the current node is at a left branch of the parent node, replacing the parent node with a right subtree of the parent node; and
if the parent node is an “plusW” or an “Xor” graphical operator and the current node is at a right branch of the parent node, replacing the parent node with a left subtree of the parent node.
5. The method as recited in any one of
6. The method as recited in
7. The method as recited in
9. The method as recited in claim ti, wherein said modityinig modifying includes removing said current rode or replacing said current node with another node of the expression tree.
10. The method as recited in claim 7 8, wherein said modifying further includes clipping, or marking for clipping at a later time, said first region represented by said current node.
12. The method as recited in
(a) where the graphic operator is an “over” or a “plusC” operator, the final region representation to be returned to the parent node is determined from a union of the second region representation and the third region representation;
(b) where the graphic operator is an “in” operator, the final region representation to be returned to the parent node is determined from an intersection of the second region representation and the third region representation;
(c) where the graphic operator is a “ratop” operator, the final region representation to be returned to the parent node is the second region representation;
(d) where the graphic operator is an “out” operator, the final region representation to be returned to the parent node is determined from a difference of the second region representation and a region representation comprising at least a region represented by a bounding box of a node at a right subtree of the current node; and
(e) where the graphic operator is an “Xor” or a “plusW” operator the final region representation to be returned to the parent node is determined from a union of the second region representation less a region representation comprising at least a region represented by a bounding box of a node at a right subtree of the current node and the third region representation less a region representation containing a bounding box of a node at a right subtree of the current node.
13. The method as recited in
passing substantially the first region representation as the modified first region representation in the event that the graphical operator is an “over”, “in”, “ratop”, “plusC”, “plusW”, “Xor”, or “out” (visit left operand first)” or alike operators operator; and
if the graphical operator is an “out (visit right operand first)” operation, passing as the modified first region representation a union of the first region representation with the second region representation.
14. The method as recited in
passing substantially the first region representation as the modified second region representation in the event that the graphical operator is an “in”, “ratop”, “out”, “plusC”, “plusW”, or “Xor” or alike operators operator; and
in the event that the graphical operator is an “over” operator passing as the modified second region representation union of the first region representation with the second region representation.
15. The method as recited in any one of
16. The method as recited in
a right operand of an “over” operator and the “over” operator node does not need to return an image representation to its parent node;
a left operand of an “in”, “plusC”, “plusW” or “Xor” operator and said operator node does not need to return an image representation to its parent node;
a right operand of an “in”, “plusC”, “plusW” or “Xor” operator and said operator node does not need to return an image representation to its parent node;
a left operand of an “out” or “ratop” operator and said operator node does not need to return an image representation to its parent node;
a right operand of a “ratop” operator;
a root of the expression tree;
an operand of an image warp, affine transformation or convolution operator; and
an operand of a colour transformation that does not preserve opaqueness or if said transformation node does not need to return an image representation to its parent node.
18. The apparatus as recited in
19. The apparatus as recited in
means for removing the node; and
means for selecting a node replacement rule from a predetermined set of node replacement rules in accordance with said graphical operator and applying said rule if the node has a parent node which has a graphical operator and the node is totally obscured.
20. The apparatus as recited in
if the parent node is an “over” graphical operator and the current node is at a left branch of the parent node, replacing the parent node with a right subtree of the parent node;
if the parent node is an “over” graphic graphical operator and the current node is at a right branch of the parent node, replacing the parent node with a left subtree of the parent node;
if the parent node is an “in” graphical operator, removing the parent node and any subtrees branching off the parent node;
if the parent node is a “ratop” graphical operator and the current node is at a left branch of the parent node, removing the parent node and any subtrees branching off the parent node;
if the parent node is a “ratop” graphical operator and the current node is at a right branch of the parent node, replacing the parent node with a left subtree of the parent node;
if the parent node is an “out” graphical operator and the current node is at a left branch of the parent node, removing the parent node and any subtrees branching off the parent node;
if the parent node is an “out” graphical operator and the current node is at a right branch of the parent node, replacing the parent node with a left subtree of the parent node;
if the parent node is a “plusC” graphical operator and the current node is at a left branch of the parent node, replacing the parent node with a right subtree of the parent node;
if the parent node is a “plusC” graphical operator and the current node is at a right branch of the parent node, replacing the parent node with a left subtree of the parent node;
if the parent node is a “plusW” or an “Xor” graphical operator and the current node is at a left branch of the parent node, replacing the parent node with a right subtree of the parent node; and
if the parent node is a “plusW” or an “Xor” graphical operator and the current node is at a right branch of the parent node, replacing the parent node with a left subtree of the parent node.
21. The apparatus as recited in any one of
22. The apparatus as recited in
23. The apparatus as recited in
25. The apparatus as recited in
26. The apparatus as recited in
28. The apparatus as recited in
(a) where the graphic operator is an “over” or a “plusC” operator, the final region representation to be returned to the parent node is determined from a union of the second region representation and the third region representation;
(b) where the graphic operator is an “in” operator, the final region representation to be returned to the parent node is determined from an intersection of the second region representation and the third region representation;
(c) where the graphic operator is an a “ratop” operator, the final region representation to be returned to the parent node is the second region representation;
(d) where the graphic operator is an “out” operator, the final region representation to be returned to the parent node is determined from a difference of the second region representation and a region representation comprising at least a region represented by a bounding box of a node at a right subtree of the current node; and
(e) where the graphic operator is an “Xor” or a “plusW” operator the final region representation to be returned to the parent node is determined from a union of the second region representation less a region representation comprising at least a region represented by a bounding box of a node at a right subtree of the current node and the third region representation less a region representation containing a bounding box of a node at a right subtree of the current node.
29. The apparatus as recited in
passing substantially the first region representation as the modified first region representation in the event that the graphical operator is an “over”, “in”, “ratop”, “plusC”, “plusW”, “Xor”, or “out” (visit left operand first)” or alike operators operator; and
if the graphical operator is an “out (visit right operand first)” operation, passing as the modified first region representation a union of the first region representation with the second region representation.
30. The apparatus as recited in
passing substantially the first region representation as the modified second region representation in the event that the graphical operator is an “in”, “ratop”, “out”, “plusC”, “plusW”, or “Xor” or alike operators operator; and
in the event that the graphical operator is an “over” operator, passing as the modified second region representation a union of the first region representation with the second region representation.
31. The apparatus as recited in any one of
32. The apparatus as recited in
a right operand of an “over” operator and the “over” operator node does not need to return an image representation to its parent node;
a left operand of an “in”, “plusC”, “plusW” or “Xor” operator and said operator node does nor need to return an image representation to its parent node;
a right operand of an “in”, “plusC”, “plusW” or “Xor” operator and said operator node docs not need to return an image representation to its parent node;
a left operand of an “out” or “ratop” operator and said operator node does not need to return an image representation to its parent node;
a right operand of a “ratop” operator;
a root of the expression tree;
an operand of an image warp, affine transformation or convolution operator; and
an operand of a colour transformation that does not preserve opaqueness or if said transformation node does not need to return an imap representation to its parent node.
|
|||||||||||||||||
,
| function obscure(node, need_result, q0) |
| begin |
| case node's type begin |
| primitive → |
| if q0 obscures the node's bounding box then |
| return obscured. |
| else if q0 partially obscures the node's bounding box and |
| there is advantage in clipping the primitive (eg., it is an image, |
| edge blend, box, all, or path primitive) then |
| begin |
| Clip if the overhead of clipping is worth the saving in not gen- |
| erating and compositing the clipped pixels. |
| Obtain a clip path from q0. This clip path remains associated with |
| q0 while it exists, so that it is only ever created once. |
| Tag the node as requiring clipping to this path. |
| end |
| if need_result then |
| begin |
| if the primitive is a good obscurer (a large opaque image, box or all: |
| a large opaque path containing a single, simple, convex edge) |
| then |
| Construct a quadtree from the primitive's boundary. |
| return this quadtree. |
| else |
| return empty quadtree. |
| end |
| colour transformation → |
| if q0 obscures the node's bounding box then |
| return obscured. |
| else if q0 partially obscures the node's bounding box then |
| begin |
| Clip, as we expect the overhead of clipping to be worth the saving |
| in not transforming the clipped pixels. |
| Obtain a clip path from q0. This clip path remains associated with |
| q0 while it exists, so that it is only ever created once. |
| Tag the node as requiring clipping to this path. |
| end |
| Determine whether the transformation preserves opaqueness |
| (opacity 1 maps to opacity 1), and whether it preserves invisbility |
| (opacity 0 maps to opacity 0). |
| call obscure(node's operand, transformation preserves opaqueness |
| and need_result, q0), obtaining quadtree q1 if requested. If |
| opaqueness is not preserved, then we can't know what areas will |
| be obscured after the transformation is applied, so there is no |
| point asking for a quadtree. |
| if operand is obscured then |
| begin |
| Note that if the operand is said to be obscured, then it is only the |
| visible parts (opacity # 0) that are guaranteed to be obscured. |
| if transformation preserves invisibility then |
| return obscured. |
| else |
| begin |
| Determine what the transformation will transform invisible |
| (opacity = 0) to. |
| Replace this node by an “all” primitive of this colour/opacity. |
| if need_result then |
| return a quadtree constructed from the all's boundary. |
| return |
| end |
| end |
| if need_result then |
| if transformation preserves opaqueness then |
| return quadtree q1. |
| else |
| return empty quadtree. |
| affine transformation, image warp → |
| if q0 obscures the node's bounding box then |
| return obscured. |
| else if qO partially obscures the node's bounding box then |
| begin |
| Clip, as we expect the overhead of clipping to be worth the saving |
| in not generating and compositing the the clipped pixels. |
| Obtain a clip path from q0. This clip path remains associated with |
| q0 while it exists, so that it is only ever created once. |
| Tag the node as requiring clipping to this path. |
| end |
| call obscure(node's operand, false, empty quadtree). We cannot |
| pass q0 down the tree or accept a result unless we |
| inverse/transform the quadtrees through the transformation. |
| image convolution → |
| if q0 obscures the node's bounding box then |
| return obscured. |
| call obscure(node's operand, false, empty quadtree). |
| binary operator → |
| if q0 obscures the node's bounding box then |
| return obscured. |
| case node's operator begin |
| over → |
| call obscure(node's left operand, true, q0), obtaining area qL obscured |
| by left operand. |
| call obscure(node's right operand, need_result, if left operand |
| is obscured then q0 else q0∪qL), obtaining area qR, obscured |
| by right operand if need_result. |
| if left operand is obscured and right operand is obscured then |
| return obscured. |
| else if left operand is obscured then |
| begin |
| Replace this node with its right operand. |
| if need_result then |
| return qR. |
| end |
| else if right operand is obscured then |
| begin |
| Replace this node with its left operand. |
| if need_result then |
| return qL. |
| end |
| else |
| if need result then |
| return qL∪qR. |
| end |
| in → |
| call obscure(node's left operand, need_result, q0), obtaining area qL |
| obscured by left operand if need_result. |
| if left operand is obscured then |
| return obscured. |
| call obscure(node's right operand, need_result, q0), obtaining area qR |
| obscured by right operand if need_result. |
| if right operand is obscured then |
| return obscured. |
| if need result then |
| return qL∩qR. |
| out → |
| call obscure(node's right operand. true, q0), obtaining area qR |
| obscured by right operand. |
| call obscure(node's left operand, need_result, if right operand is |
| obscured then q0 else q0∪qR), obtaining area qL obscured by |
| left operand if need_result. |
| if left operand is obscured then |
| return obscured. |
| else if right operand is obscured then |
| begin |
| Replace this node with its left operand. |
| if need_ result then |
| return qL. |
| end |
| else |
| if need_result then |
| return qL-B(right operand). |
| end |
| ratop → |
| call obscure(node's left operand, need result, q0), obtaining area qL |
| obscured by left operand if need_result. |
| if left operand is obscured then |
| return obscured. |
| call obscure(node's right operand, false, q0). |
| if right operand is obscured then |
| Replace this node with its left operand. |
| if need_result then |
| return qL. |
| plusC → |
| call obscure(node's left operand, need_result, q0), obtaining area qL |
| obscured by left operand if need_result. |
| call obscure(node's right operand, need result, q0), obtaining area qR |
| obscured by right operand if need_result. |
| if left operand is obscured and right operand is obscured then |
| return obscured. |
| else if left operand is obscured then |
| begin |
| Replace this node with its right operand. |
| if need_result then |
| return qR. |
| end |
| else if right operand is obscured then |
| begin |
| Replace this node with its left operand. |
| if need_result then |
| return qL. |
| end |
| else |
| if need_result then |
| return qL↑qR. |
| end |
| plusW, Xor → |
| call obscure(node's left operand, need_result, q0), obtaining area qL |
| obscured by left operand if need_result. |
| call obscure(node's right operand, need_result, q0), obtaining area qR |
| obscured by right operand if need_result. |
| if left operand is obscured and right operand is obscured then |
| return obscured. |
| else if left operand is obscured then |
| begin |
| Replace this node with its right operand. |
| if need_result then |
| return qR. |
| end |
| else if right operand is obscured then |
| begin |
| Replace this node with its left operand. |
| if need_result then |
| return qL. |
| end |
| else |
| begin |
| if need_result then |
| return (qL-B(rightoperand)) ∪ (qR-B(leftoperand)). |
| end |
| end case binary operator |
| end case node type |
| end |
| Patent | Priority | Assignee | Title |
| 10032301, | Oct 06 2016 | International Business Machines Corporation | Rebuilding images based on historical image data |
| 10032302, | Oct 06 2016 | International Business Machines Corporation | Rebuilding images based on historical image data |
| 10169894, | Oct 06 2016 | International Business Machines Corporation | Rebuilding images based on historical image data |
| 10169896, | Oct 06 2016 | International Business Machines Corporation | Rebuilding images based on historical image data |
| 9892538, | Oct 06 2016 | International Business Machines Corporation | Rebuilding images based on historical image data |
| Patent | Priority | Assignee | Title |
| 5123084, | Dec 24 1987 | General Electric CGR S.A.; Octree Corporation | Method for the 3D display of octree-encoded objects and device for the application of this method |
| 5274718, | Sep 30 1991 | AT&T Corp | Image representation using tree-like structures |
| 5295236, | Mar 04 1991 | Adobe Systems Incorporated | Applying traps to a printed page specified in a page description language format |
| 5515487, | Apr 18 1994 | International Business Machines Corporation | Displaying partial graphs by expanding and collapsing nodes |
| 5579455, | Jul 30 1993 | Apple Inc | Rendering of 3D scenes on a display using hierarchical z-buffer visibility |
| 5600763, | Jul 21 1994 | Apple Inc | Error-bounded antialiased rendering of complex scenes |
| 5724494, | Jul 25 1994 | Canon Kabushiki Kaisha | Optimization method for the efficient production of images |
| 5745121, | Jul 25 1994 | Canon Kabushiki Kaisha | Methods and apparatus for optimizing the composition of graphical elements |
| 20020027563, | |||
| 20030118250, | |||
| 20050267908, | |||
| AU2336295, | |||
| EP528631, | |||
| EP694881, | |||
| EP69881, | |||
| JP8115413, |
| Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
| Feb 20 2003 | Canon Kabushiki Kaisha | (assignment on the face of the patent) |
| Date | Maintenance Fee Events |
| Feb 22 2012 | ASPN: Payor Number Assigned. |
| Jul 25 2012 | M1553: Payment of Maintenance Fee, 12th Year, Large Entity. |
| Date | Maintenance Schedule |
| Oct 18 2014 | 4 years fee payment window open |
| Apr 18 2015 | 6 months grace period start (w surcharge) |
| Oct 18 2015 | patent expiry (for year 4) |
| Oct 18 2017 | 2 years to revive unintentionally abandoned end. (for year 4) |
| Oct 18 2018 | 8 years fee payment window open |
| Apr 18 2019 | 6 months grace period start (w surcharge) |
| Oct 18 2019 | patent expiry (for year 8) |
| Oct 18 2021 | 2 years to revive unintentionally abandoned end. (for year 8) |
| Oct 18 2022 | 12 years fee payment window open |
| Apr 18 2023 | 6 months grace period start (w surcharge) |
| Oct 18 2023 | patent expiry (for year 12) |
| Oct 18 2025 | 2 years to revive unintentionally abandoned end. (for year 12) |