In order to efficiently backface cull rendering primitives during computer graphics rendering, it is important to be sure that the rendering primitives to be culled are guaranteed to be backfacing even if the primitives are moving or are undergoing defocus blur. Therefore, we derive conservative tests that determine if a moving and defocused triangle is backfacing over an entire time interval and over the area of a lens. In addition, we present tests for the special cases of only motion blur and only depth of field.
|
1. A method comprising:
determining, using a computer, the sign of a determinant of vertices on a triangle undergoing motion blur and depth of field, including determining the sign of a determinant of vertices of a triangle where, for each vertex, the depth of field effect can be represented by a function of two lens coordinates u and v and the vertex motion represented by a function of time; and
based on said sign, deciding whether to backface cull the triangle.
10. A non-transitory computer readable medium storing instructions to enable a computer to:
determine, using a computer, the sign of a determinant of vertices of a triangle undergoing motion blur and depth of field, including determining the sign of a determinant of vertices of a triangle where, for each vertex, the depth of field effect can be represented by a function of two lens coordinates u and v and the vertex motion represented by a function of time; and
based on said sign, decide whether to backface cull the triangle.
19. An apparatus comprising:
a processor to determine, using a computer, the sign of a determinant of vertices of a triangle undergoing motion blur and depth of field and, based on the sign, to decide whether to backface cull the triangle, said processor to determine the sign of a determinant of vertices of a triangle where, for each vertex, the depth of field effect can be represented by a function of two lens coordinates u and v and the vertex motion represented by a function of time; and
a memory coupled to said processor.
2. The method of
3. The method of
4. The method of
5. The method of
6. The method of
7. The method of
8. The method of
9. The method of
11. The medium of
12. The medium of
13. The medium of
14. The medium of
15. The medium of
16. The medium of
17. The medium of
18. The medium of
21. The apparatus of
22. The apparatus of
23. The apparatus of
24. The apparatus of
25. The apparatus of
26. The apparatus of
|
This relates to graphics processing and, particularly, to motion blur and depth of field rendering.
Motion blur is the effect where moving objects appear blurred when captured with long camera exposure times. Depth of field is the effect that a larger aperture has a shorter focus range, and objects out of focus appear blurred.
One of the most important culling tests is backface culling. Backface culling eliminates rasterization processing of objects that face away from the camera. A rendering primitive can be backface culled if its face normal makes an angle of more than 90 degrees with the camera view vector. By culling these rendering primitives, unnecessary processing may be reduced.
When rendering motion blur and depth of field, an excessive amount of inside or intersection tests can be generated, and therefore, accurate backface culling tests are needed also for these contexts. For moving triangles, where each vertex moves along a line in three dimensions, a moving triangle is commonly assumed to be backfacing over the entire time interval if the triangle is backfacing at the start of the motion and at the end.
A triangle where each vertex moves linearly in 3D can be backfacing (i.e. facing away from the camera) at the start of the motion (at t=0), then turn frontfacing, and then (again) backfacing at the end of the motion (at t=1). As a result, the assumption that a triangle whose vertices move linearly in time, is backfacing if the backfacing status is true at the start and end of the interval, is not always correct.
Assume that we have a moving triangle, where the vertices move linearly within a frame, from time t=0 to t=1. At t=0, we denote the vertices qi and at t=1 we call them ri. We work in clip space, using 2D homogeneous coordinates (2DH), so a vertex is defined as p=(px,py,pw). A linearly interpolated vertex is then expressed as:
pi(t)=(1−t)qi+tri. (1)
Given a moving triangle with vertices (p0(t),p1(t),p2(t)), we form the matrix:
where we have omitted the temporal dependence for readability. The triangle can be backface culled if det(M)<0, where the determinant is expressed as:
det(M)=p0·(p1×p2). (3)
Geometrically, this can be interpreted as a (scaled) signed volume computation of the tetrahedron spanned by the origin and the triangle. Thus, we want to determine if p0(t)·(p1(t)×p2(t))<0 for tε[0,1]. The cross product of two linearly moving vertices can be expanded as:
p1×p2=((1−t)q1+tr1)×((1−t)q2+tr2)=t2f+tg+h, (4)
where:
f=(r1−q1)×(r2−q2),
g=(r1−q1)×q2−(r2−q2)×q1,
h=q1×q2. (5)
Using this expression, we can derive the time-dependent determinant:
where:
a=(r0−q0)·f,
b=(r0−q0)g+q0·f,
c=(r0−q0)h+q0·g,
d=q0·h. (7)
Note that the coefficient d=q0·h=q0·(q1×q2) is the backface test for the triangle at t=0. Also, the value of the polynomial at t=1 is a+b+c+d=r0·(r1×r2), which, analogously, is the backface test at t=1. Finally, note that the expression for the coefficient:
a=(r0−q0)[(r1−q1)×(r2−q2)], (8)
is the determinant test for the motion vectors of the three vertices, and if they all lie in the same plane, the cubic term is zero, i.e., a=0. Thus, it is only when the motion vectors span a volume in 2DH that the determinant will be a cubic function.
We note that if the polynomial does not have any roots in tε[0,1] and d<0, then the triangle can be safely backface culled. Given that the backface function is below zero at t=0 and t=1, we can compute the local minimum and maximums of the cubic polynomial, and if we find a local maximum within the interval tε[0,1], we check the value of the cubic polynomial at this point. If it is below zero, we can safely backface cull the triangle. If the motion direction of the three vertices are parallel, the backfacing function is linear.
The coefficient a is the determinant of the triangle's three motion vectors, which are often small or near parallel. Therefore, directly computing the backfacing function on power form (Equation 6) can be numerically unstable. To alleviate this problem, we express the backfacing function on cubic Bernstein form:
with coefficients, bi, given by:
b0=q0·(q1×q2)
b1=⅓[q0·(q1×r2+r1×q2)+r0·(q1×q2)],
b2=⅓[r0·(q1×r2+r1×q2)+q0·(r1×r2)],
b3=r0·(r1×r2). (10)
Next, we exploit the convex hull property of the Bernstein basis, and simply check if any of the coefficients, bi, iε{0, 1, 2, 3}, are positive. This is a coarser test than testing against the true maximum of the cubic polynomial, but reduces the risk of numerical precision issues. Note that the test can be refined by applying de Casteljau steps to the coefficients and testing the generated coefficients.
A backface test for motion blur on Bernstein form, shown in
We sketch a practical implementation of the backface culling test for triangles where each vertex moves linearly in time in the pseudo code:
1
movingTriIsBackfacing(const TimeContTri& tri) {
2
b0 = det(M(t=0))
3
if b0 > 0
4
return false
5
b3 = det(M(t=1))
6
if b3 > 0
7
return false
8
compute b1
9
if b1 > 0
10
return false
11
compute b2
12
if b2 > 0
13
return false
14
return true
15
}
If the triangle vertex motion can be expressed as a polynomial, we can generalize the previous test. We express the motion of each triangle vertex as a Bézier curve of degree n in 2DH:
The backface test then becomes:
Note that this is a Bézier curve of degree 3n, where the control points are sums of scaled determinants of three control points, one from each of the three curves describing the vertex motion. A conservative backface test can again be derived by using the convex hull property. As expected, we obtain Equation 10 for the linear motion case, i.e., when n=1. A similar derivation can be performed for rational splines.
The test has potential to speed up motion blur rasterization of macro-sized triangles, or small triangles with large motion, where one expensive backface culling test can be amortized over a lot of saved inside tests. If the number of inside tests is small in relation to the overhead of the conservative backface culling test, it may be better to skip the conservative test entirely, as backfacing triangles will be correctly culled in the per-sample inside test. Depending on the workload, the test could be enabled when the triangle sizes or the motion surpasses a threshold value.
Depth of field is a shear in clip space. This shear can be represented by applying the matrix:
to the clip space coordinates of the triangle. H, I and J are constants given by the location of the focal plane, the camera aperture size and the near and far plane. The location on the lens is given by (u,v). Applying this matrix to a the three-dimensional homogeneous vertex (3DH), {circumflex over (p)}i=(pix,piypiz,piw), in clip space results in a sheared position, ŝi(u,v):
six=pix−H/Jupiz+Hupiw=pix+αiu,
siy=piy−I/Jvpiz+Ivpiw=piy+βiv,
siz=piz,
siw=piw (14)
where αi=H/Jpiz+Hpiw and βi=I/Jpiz+Ipiw are per-vertex values dependent on the piz and piw terms. To simplify notation below, we let si(u,v) and pi(u,v) denote the two-dimensional homogeneous vertices, consisting of the xyw components of ŝi(u,v) and {circumflex over (p)}i respectively, that is:
si(u,v)=pi+(αiu,βiv,0)=pi+li(u,v) (15)
Note that:
li×lj=uv(0,0,αiβj−αjβi)=0 (16)
since
The backfacing criterion then becomes:
The coefficients are given by:
a=α0(p1×p2)x+α1(p2×p0)x+α2(p0×p1)x,
b=β0(p1×p2)y+β1(p2×p0)y+β2(p0×p1)y,
c=p0·(p1×p2). (18)
We note that some expressions for a and b can be reused for the edge equation setup. As can be seen, the backfacing function for depth of field is a linear function in u and v (Equation 17). The triangle changes its facing when au+bv+c=0. Hence, the triangle will change its facing somewhere on the lens only if there is a solution to the following system of equations:
au+bv+c=0,
u2+v2<R2, (19)
where the last equation checks whether the lens position, (u,v), is inside a circular lens with radius, R. Geometrically, this is an intersection between a circle and a line, which has a solution only if:
Consequently, if we can prove that the above does not hold, there are no face changes when moving over the lens. Intuitively, the triangle will not change facing over the lens if the triangle's plane equation (in three dimensions) does not intersect with the shape of the lens.
Finally, note that if the triangle's face normal is aligned with the view vector, we have piz=pjz and piw=pjw, and therefore αi=αj and βi=βj, which leads to a=b=0. This means that facing will not change over the lens, as expected.
To determine if a static triangle can be backface culled, we start with computing the backface status at the at the middle of the lens (u=v=0),
c<0,
and
c2≧>R2(a2+b2) (21)
Otherwise, the triangle is front facing (block 54).
By multiplying the moving vertex in Equation 1 with the shear matrix, S, in Equation 13, the resulting vertex displacement from motion and depth of field, o(u,v,t), is obtained:
where αi(t)=H/Jpiz(t)+Hpiw(t) and βi(t)=I/Jpiz(t)+Ipiw(t) are linear functions in t.
The corresponding backface test from Equation 21 is now expressed as:
The coefficients a(t), b(t), and c(t) are cubic functions in t.
The triangle can be conservatively backface culled when:
c(t)<0,tε[0,1]
and
c2(t)>R2(a2(t)+b2(t)),tε[0,1]. (24)
Otherwise, the triangle is front facing (block 54).
A conservative test can be obtained by using interval arithmetic. We denote an interval as {circumflex over (x)}=[x,
where we used
If we approximate the lens with a square with side length R, we get a coarser test given by the equations:
au+bv+c=0,
|u|<R,|v|<R, (26)
which is a line against box test. Again, we start by testing whether ĉ<0, i.e., whether the triangle is backfacing on the center of the lens. Then, if the four corners of the (square) lens all are on the same side of the swept line, we can conservatively cull the triangle. These four conditions are given by:
max(±R(a±b)±c)<0. (27)
Note that the expressions a, b and c are cubic polynomials in t. Unlike the interval analysis test given for the circular lens, we can, for each of the four equations, compute the coefficients for a single cubic equation and bound it, therefore retaining the correlation between the power coefficients for tighter bounds. This may be more efficient for motion with large non-linear coefficients.
Next, we sketch a practical implementation of the backface culling test for motion blurred and defocused triangles. We first note that a coarse, but fast approximation of a2(t)+b2(t) is given by:
max(a2+b2)≦max(a2,ā2)+max(b2,
which essentially is a Manhattan distance approximation.
However, we will instead bound the cubic functions using first-order Taylor models, since the correlation in the linear terms can be preserved, which in turn can give tighter bounds. We also note that the t3 and t2 terms of the polynomial a(t) and b(t) are very close to zero for most triangles. This implies that the a(t), b(t) and c(t) terms in Equation 23 are approximately linear, but it also means that care must be taken to avoid precision issues. By using first-order Taylor models, we ensure stability when the t3 and t2 terms are small. For an arbitrary cubic polynomial, this is done as shown below:
k3t3+k2t2+k1t+k0≈k1t+k0+{circumflex over (r)}k, (29)
where {circumflex over (r)} is a remainder interval, which bounds the quadratic and cubic terms:
{circumflex over (r)}k=[−|k3|−|k2|,|k3|+|k2|]. (30)
We use this to conservatively express a2(t)+b2(t) as:
a2(t)+b2(t)≈(a1t+a0+{circumflex over (r)}a)2+(b1t+b0+{circumflex over (r)}b)2. (31)
The upper bound is given by:
a12+b12max(0,a0a1+b0b1)+a02+b02+
where
The final test is given by the conditions in Equation 34, where a2(t)+b2(t) is bounded using Equation 32.
Referring to
Otherwise, at block 68, the max[a(t)2+b(t)2] is computed and bounded, as indicated in block 68. Next, a test is done to determine if the minimum of c2 is greater than R2max[a(t)2+b(t)2], as indicated in block 70. If so, at diamond 72, the triangle is reported as front facing, as indicated in block 74 and, otherwise, it is reported as backfacing in block 76.
We will derive a backface culling test for motion blur where rasterization is done in screen space. For screen-space rasterization, a common backface test is given by the sign of the screen-space area of the triangle. A projective transform maps lines to lines, so the motion vectors are still lines after projection. However, the acceleration along the line is different due to perspective foreshortening. Let us define two edges of the projected triangle as:
Twice the signed area can now be expressed as:
Recall that each vertex is a function of t (Equation 1), which results in that the backface test in screen space is a cubic rational function in t. The triangle moves in a plane, but the vertex positions are no longer linearly interpolated in t and the triangle can change facing at most three times.
Furthermore, the magnitude of the denominator p0
On the other hand, if we assume linear motion in screen space, the area function becomes a quadratic polynomial, so even in this case, the moving triangle can be backfacing at t=0 and t=1, and still be frontfacing somewhere in between.
Our techniques apply to real-time and offline rendering, and to both stochastic point sampling and analytical visibility methods. The rendering errors introduced by the previous technique can easily be detected in extreme cases, but in the majority of cases, these generate reasonable images, where the error is hard to detect.
The computer system 130, shown in
In the case of a software implementation, the pertinent code may be stored in any suitable semiconductor, magnetic, or optical memory, including the main memory 132 or any available memory within the graphics processor. Thus, in one embodiment, the code to perform the sequences of
The graphics processing techniques described herein may be implemented in various hardware architectures. For example, graphics functionality may be integrated within a chipset. Alternatively, a discrete graphics processor may be used. As still another embodiment, the graphics functions may be implemented by a general purpose processor, including a multicore processor.
References throughout this specification to “one embodiment” or “an embodiment” mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one implementation encompassed within the present invention. Thus, appearances of the phrase “one embodiment” or “in an embodiment” are not necessarily referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be instituted in other suitable forms other than the particular embodiment illustrated and all such forms may be encompassed within the claims of the present application.
While the present invention has been described with respect to a limited number of embodiments, those skilled in the art will appreciate numerous modifications and variations therefrom. It is intended that the appended claims cover all such modifications and variations as fall within the true spirit and scope of this present invention.
Hasselgren, Jon N., Munkberg, Carl J., Toth, Robert M., Clarberg, Franz P., Salvi, Marco, Akenine-Möller, Tomas G., Pharr, Matt
Patent | Priority | Assignee | Title |
10235811, | Dec 29 2016 | Intel Corporation | Replicating primitives across multiple viewports |
10628910, | Sep 24 2018 | Intel Corporation | Vertex shader with primitive replication |
11087542, | Dec 29 2016 | Intel Corporation | Replicating primitives across multiple viewports |
Patent | Priority | Assignee | Title |
5929862, | Aug 05 1996 | HEWLETT-PACKARD DEVELOPMENT COMPANY, L P | Antialiasing system and method that minimize memory requirements and memory accesses by storing a reduced set of subsample information |
6057848, | Apr 08 1997 | AVAGO TECHNOLOGIES GENERAL IP SINGAPORE PTE LTD | System for rendering high order rational surface patches |
6559844, | May 05 1999 | ATI Technologies ULC | Method and apparatus for generating multiple views using a graphics engine |
7253816, | Mar 11 2002 | STMicroelectronics Ltd | Computer graphics acceleration method and apparatus for evaluating whether points are inside a triangle |
7952588, | Aug 03 2006 | Qualcomm Incorporated | Graphics processing unit with extended vertex cache |
20100097377, | |||
EP1542167, |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Sep 28 2010 | Intel Corporation | (assignment on the face of the patent) | / | |||
Sep 28 2010 | MUNKBERG, CARL J | Intel Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 025436 | /0823 | |
Sep 28 2010 | AKENINE-MOLLER, TOMAS G | Intel Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 025436 | /0823 | |
Sep 28 2010 | TOTH, ROBERT M | Intel Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 025436 | /0823 | |
Sep 28 2010 | HASSELGREN, JON N | Intel Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 025436 | /0823 | |
Sep 28 2010 | CLARBERG, FRANZ P | Intel Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 025436 | /0823 | |
Oct 22 2010 | PHARR, MATT | Intel Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 025436 | /0823 | |
Nov 24 2010 | SALVI, MARCO | Intel Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 025436 | /0823 |
Date | Maintenance Fee Events |
Oct 28 2013 | ASPN: Payor Number Assigned. |
May 04 2017 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
Jul 12 2021 | REM: Maintenance Fee Reminder Mailed. |
Dec 27 2021 | EXP: Patent Expired for Failure to Pay Maintenance Fees. |
Date | Maintenance Schedule |
Nov 19 2016 | 4 years fee payment window open |
May 19 2017 | 6 months grace period start (w surcharge) |
Nov 19 2017 | patent expiry (for year 4) |
Nov 19 2019 | 2 years to revive unintentionally abandoned end. (for year 4) |
Nov 19 2020 | 8 years fee payment window open |
May 19 2021 | 6 months grace period start (w surcharge) |
Nov 19 2021 | patent expiry (for year 8) |
Nov 19 2023 | 2 years to revive unintentionally abandoned end. (for year 8) |
Nov 19 2024 | 12 years fee payment window open |
May 19 2025 | 6 months grace period start (w surcharge) |
Nov 19 2025 | patent expiry (for year 12) |
Nov 19 2027 | 2 years to revive unintentionally abandoned end. (for year 12) |