A method and apparatus is described for producing a translucent image over a base image created on the display screen of a computer system by a selected first application program, and conducting image operations either on the base image created by the selected application program with reference to the translucent image produced, or conducting image operations on the translucent image with reference to the base image of the first application program. The first application program runs on a central processing unit (CPU) of a computer system to produce a base image, and another application program referred to as the overlay program is run to produce the translucent image such that portions of the base image which are overlapped by the overlay image are at least partially visible through the translucent image. There is also a mechanism for blending the first video data and the second video data to produce a blended image on the screen assembly.
|
0. 29. A method comprising:
producing a first image and a second image in memory, wherein at least a portion of the first image includes first blendable units, and at least a portion of the second image includes second blendable units, at least one of the first blendable units being blended with at least one of the second blendable units;
identifying updated first blendable units of the first image;
blending the updated first blendable units of the first image with the at least one of the second blendable units in a memory of an electronic device to yield a blended image; and
displaying the blended image on a display of the electronic device.
0. 37. A system comprising:
a memory configured to store a first image and a second image, wherein at least a portion of the first image includes first blendable units, and at least a portion of the second image includes second blendable units, at least one of the first blendable units being blended with at least one of the second blendable units;
a processor configured to identify updated first blendable units of the first image, and blend the updated first blendable units of the first image with the at least one of the second blendable units in the memory of an electronic device to yield a blended image;
a display configured to display the blended image.
0. 45. A device comprising:
a non-transitory computer-readable medium configured to store computer-executable instructions; and
computer-executable instructions stored on the non-transitory computer-readable medium that when executed are effective to cause a computer processor to:
produce a first image and a second image in memory, wherein at least a portion of the first image includes first blendable units, and at least a portion of the second image includes second blendable units, at least one of the first blendable units being blended with at least one of the second blendable units;
identify updated first blendable units of the first image;
blend the updated first blendable units of the first image with the at least one of the second blendable units in a memory of an electronic device to yield a blended image; and
display the blended image on a display of the electronic device.
0. 1. A method for establishing a translucent window having a translucent window background and a translucent window frame on a display screen of a computer system, comprising the steps of:
displaying a translucent window on a display screen such that a base window can be seen through said translucent window, and
conducting image operations on at least one of said translucent window and said base window.
0. 2. A method as recited in
0. 3. A method as recited in
0. 4. A method as recited in
0. 5. A method as recited in
0. 6. A method as recited in
0. 7. A method as recited in
0. 8. A method as recited in
0. 9. A method for displaying images on a display screen of a computer system, comprising the steps of:
displaying a base image on a display screen of a computer system; and
displaying a translucent image on said screen such that portions of said base image which are covered by said translucent image are at least partially visible through said translucent image.
0. 10. A method as recited in
0. 11. A method as recited in
0. 12. A method as recited in
0. 13. A method as recited in
0. 14. A method as recited in
0. 15. A method as recited in
0. 16. A method as recited in
0. 17. A method for displaying images on a display screen of a computer system comprising the steps of:
running an application program on the central processing unit (CPU) of a computer system to produce a base image on a display screen coupled to said CPU; and
running an overlay program on said CPU to produce a translucent image on said display screen such that portions of said base image are overlapped by said translucent image and are at least partially visible through said translucent image.
0. 18. A method as recited in
displaying a translucent image on said display screen;
intercepting screen inputs which contact the translucent image;
processing said intercepted screen inputs in said CPU; and
updating said application program based upon said processed screen inputs.
0. 19. A method as recited in
0. 20. A method as recited in
0. 21. A method as recited in
0. 22. A method as recited in
0. 23. A method of performing image operations in a computer system having a display screen, including the steps of:
presenting a first selected image with respect to which image operations are desired, and
producing a translucent image effective for overlapping at least a portion of said first selected image.
0. 24. A method according to
0. 25. A method according to
0. 26. A computer system comprising:
a central processing unit (CPU);
screen means for displaying images, said screen means being coupled to said CPU;
display means coupled to said screen means for displaying a translucent image on said screen means; and
means for conducting image operations on a region including the level of a translucent image produced by said display means and the level beneath the translucent image.
0. 27. A computer system according to
0. 28. A computer system according to
0. 30. The method of claim 29 further comprising:
activating the first image to receive inputs;
receiving inputs in the first image effective to require an update of the first blendable units.
0. 31. The method of claim 30 wherein the first image is overlaid by the second image.
0. 32. The method of claim 30 wherein the first image is overlaid by the second image while the first image is receiving inputs.
0. 33. The method of claim 30, wherein the first image is produced by a first application.
0. 34. The method of claim 33, wherein the received inputs are user inputs received via a user interface of an operating system and passed to the first application.
0. 35. The method of claim 29, wherein the blended image is comprised of a portion of pixel values contributed by the first blendable unit, and a portion of pixel values contributed by the second blendable unit.
0. 36. The method of claim 29, wherein the blended image appears as is the second image is over the first image, and appears as though the second image can be seen and seen through such that the portion of the first image can be seen through the second image.
0. 38. The system of claim 37, wherein the processor is futher configured to activate the first image to receive inputs, and to receive inputs in the first image effective to require an update of the first blendable units.
0. 39. The system of claim 38 wherein the first image is overlaid by the second image.
0. 40. The system of claim 38 wherein the first image is overlaid by the second image while the first image is receiving inputs.
0. 41. The system of claim 38, wherein the first image is produced by a first application executed by the processor.
0. 42. The system of claim 41, wherein the received inputs are user inputs received via a user interface of an operating system and passed to the first application.
0. 43. The system of claim 37, wherein the blended image is comprised of a portion of pixel values contributed by the first blendable unit, and a portion of pixel values contributed by the second blendable unit.
0. 44. The system of claim 37, wherein the blended image appears as is the second image is over the first image, and appears as though the second image can be seen and seen through such that the portion of the first image can be seen through the second image.
0. 46. The device of claim 45 further comprising:
activating the first image to receive inputs;
receiving inputs in the first image effective to require an update of the first blendable units.
0. 47. The device of claim 46 wherein the first image is overlaid by the second image.
0. 48. The device of claim 46 wherein the first image is overlaid by the second image while the first image is receiving inputs.
0. 49. The device of claim 46, wherein the first image is produced by a first application.
0. 50. The device of claim 49, wherein the received inputs are user inputs received via a user interface of an operating system and passed to the first application.
0. 51. The device of claim 45, wherein the blended image is comprised of a portion of pixel values contributed by the first blendable unit, and a portion of pixel values contributed by the second blendable unit.
0. 52. The device of claim 45, wherein the blended image appears as is the second image is over the first image, and appears as though the second image can be seen and seen through such that the portion of the first image can be seen through the second image.
|
In FIG. 22, the operating system, application program, overlay utility, system routines, etc., are shown in a somewhat hierarchical fashion. At the highest level is the operating system 1096 of the computer system 10 of FIG. 1. Running under the operating system 1096 is an application program 1098, such as the aforementioned AppleShare application program. Application program 1098, when it wants to open a window such as window 1044 of FIG. 18, calls a set of routines 1100 provided by the operating system 1096. More specifically, in the Macintosh operating system, application program 1098 calls a “New Window” routine 1102 which, in turn, calls a “Frame Rect” routine 1104. The Frame Rect routine uses a pointer table 1106 to call a “Shield Cursor” routine 1107 and a “Show Cursor” routine 1108. If the application program 1098 were running on system 1096 without the process 133 of the present invention, this would be the entirety of the calls to open up the window 1044 of FIG. 18. This process is extensively documented in the multi-volume reference set, Inside Macintosh, by C. Rose et al., Addison-Wesley Publishing Company, Inc., July 1988 and are well known to those skilled in the art of programming on the Macintosh operating system.
The implementation of computer implemented process 133 modifies this normal flow of routine calls in the following way. When the application program 1098 calls the New Window routine 1102 which calls the Frame Rect routine 1104, which attempts to call the Shield Cursor Routine, the Frame Rect routine 1104 instead calls a portion of the process of step 138 of FIG. 6B known as the Overlay Shield Cursor Patch 1110. This is accomplished by having the process 138 modify the pointer table 1106 such that when the Frame Rect routine 1104 is trying to call the Shield Cursor Routine 1107 it, instead, calls the Overlay Shield Cursor Patch 1110. After the Overlay Shield Cursor Patch 1110 completes its process, the Shield Cursor Routine 1107 is then called. As far as the Frame Rect routine 1104 is concerned, it does not know of the diversion of process control to the Overlay Shield Cursor Patch process 1110, and instead believes that it directly called the Shield Cursor Routine 1107.
The process step 138 of FIG. 6B similarly “tricks” the Frame Rect routine 1104 when it attempts to call the Show Cursor Routine 1108. In that instance, when the Frame Rect routine 1104 goes to the pointer table 1106 in an attempt to call the Show Cursor Routine 1108, process control is instead diverted to a process 1112 known as “Overlay Show Cursor Patch”. The Overlay Show Cursor Patch process 1112 interacts with a Blending Engine process 1114 to blend a first screen image 1116 generated by the Macintosh operating system and the application program, with a second image 1118 (in this case, the keyboard image) to form the blended image 1120. The operation of the Blending Engine will be discussed in greater detail subsequently. After the completion of the blending process of 1114, the Overlay Show Cursor Patch process 1112 turns over process control to the “Show Cursor Routine” process 1108. Again, as far as the Frame Rect routine 1104 is concerned, it made a direct call to the “Show Cursor Routine” 1108 and was ignorant of the diversion of the process control to the Overlay Show Cursor Patch 1112 and the Blending Engine 1114.
FIG. 23 illustrates an alternate embodiment of the present invention which has been optimized for screen-writing speed. While the process of FIG. 22 works very well, it requires that the entirety of the base screen 1116 be rewritten whenever the blended image 1120 is to be refreshed. The alternative process of FIG. 23 only refreshes the portions of the blended image that needs to be refreshed, thereby greatly increasing the writing speed to the screen 1040.
Much of the operation of the process illustrated in FIG. 23 is similar to that described in FIG. 22. An operating system 1172 supports an application program 1174 which, when it wants to open a window, calls a set of routines 1176 including a “New Window routine” 1178 and Frame Rect routine 1180. The Frame Rect routine 1180 then, as before, attempts to first call the Shield Cursor Routine 1182 first and then the Show Cursor Routine 1184. Again, as before, the pointer table is modified such that when the Frame Rect routine tries to call the Shield Cursor Routine 1182, it instead calls the Overlay Shield Cursor Patch 1186 of the present invention, and when the Frame Rect routine 1180 attempts to call the Show Cursor Routine 1184 it instead calls the Overlay Show Cursor Patch 1188. The Overlay Show Cursor Patch calls a Blending Engine 1190 which blends a partial base image 1192 with an overlay image 1194 to create a blended image 1196.
The system 1172, as part of its functioning, will make periodic calls to various system task processes 1198. The system task 1198 performs such functions as execute “Device Driver Code” and “Desk Accessory Code.” The process of the present invention opportunistically takes advantage of these periodic system task calls by modifying a pointer table 1200 to turn over process control to an Overlay System Task Patch 1202. This Overlay System Task Patch, along with the Overlay Shield Cursor Patch 1186, the Overlay Show Cursor Patch 1188, and the Blending Engine 1190 comprise the overlay utility 133 of FIGS. 6A and 6B in this second preferred embodiment.
FIG. 24 is used to illustrate the operation of the Blending Engine 1190 of FIG. 23 in greater detail. The process 138 of FIG. 6B remaps certain pages of VRAM to the RAM screen buffer when an overlay image contains objects that overlap these pages. The RAM overlay screen buffer 1194 is then merged with the RAM screen buffer 1192 in the Blending Engine 1190 by a process similar to that previously described and inserts the blended image into a “hole” 1204 of the VRAM screen buffer 1196. The portions 1206 and 1208 of the VRAM screen buffer remain the VRAM since the overlay image of the present invention does not overlap pages comprising these portions of the screen.
Since portions 1206 and 1208 are pages of VRAM screen buffer memory which are not overlapped, at least in part, by an overlay image of the present invention, these portions 1206 and 1208 can remain in VRAM screen buffer. VRAM screen buffer is much faster memory for video purposes than the RAM screen buffer 1192. Also, changes made to the RAM screen buffer 1192 or to the RAM overlay screen buffer 1194 that do not cause a change in portions 1206 and 1208 do not require that the system blend the portions 1206 and 1208. The combination of these factors substantially increase the blending speed of the VRAM screen buffer and therefore of the display on screen 1040.
FIGS. 25 and 26 are used to illustrate an alternate embodiment of the present invention wherein the blending of the base image and the overlay image are performed in the video driver hardware rather than within a computer implemented process on the CPU. In FIG. 25, a prior art video driver system of a Macintosh computer system is illustrated. In this prior art example, the video driver circuit 1302 is coupled to an address bus 1304 and a data bus 1306 connected to a Motorola 68030 microprocessor. The video driver circuit 1302 includes a color screen controller CSC 1307, and two banks of VRAM 1308 and 1310. The CSC 1307 produces LCD control and data on a bus 1312 which control a black and white or color liquid crystal display (LCD). For example, the video driver circuit 1302 can drive an Esher LCD circuit for a 640 by 400 bit display, with eight bits of information per pixel.
In FIG. 26, a modified video driver circuit 1302′ is coupled to the same Motorola 68030 address bus 1304 and data bus 1306, and includes the same CSC 1307, VRAM 1308, and VRAM 1310. However, the data and address connections have been modified as indicated. In this implementation, data from the screen buffer and the overlay screen buffer are input into the VRAM of modified video driver circuit 1302′, and combined therein to provide LCD control and blended data on the bus 1312. Again, the video driver circuit 1302′ can control a black and white or color LCD, except this time instead of having eight bits per pixel, there are four bits allocated to the base image and four bits allocated to the overlay image. A color look-up table (CLUT)—not shown—of CSC 1307 is loaded with 256 entries which detail each possible combination of bits from the 4 bit screen and the 4 bit overlay, and what the resultant blended value is. The color capability of the CSC 1307 is therefore no longer used for color look-up, and is instead used for the blending values. This technique makes it possible to use off-the-shelf integrated circuits, such as the CSC 1307 which is available from Chips & Technologies, Inc. of San Jose, Calif., to perform an entirely new operation.
In summary, the method of the invention includes establishing translucent images on a display screen including displaying a translucent images and conducting image operations enabled by the translucent image. Image operations can be any kind of operation conducted on an image or window. Drawing an image, placing an image, or for that matter modifying, moving, expanding, or changing an image or a window, are considered to be image operations. A reference image could be provided by a selected first application program. The translucent image could be produced by a selected second application program. The user is thus enabled to make sketches on the translucent image or window based upon what he or she sees on the base image produced by the first application program. This is made possible without any direct intervention in the operations of the first application program. In short, the features of the first application program are advantageously employed, without any modification of the first application program itself. The technical enablement of this cooperative screen is found in a feature of the invention according to which the second application program intercepts certain screen inputs of the first application program and uses them to supply the screen input needed as to the second application program.
The image operations enabled by the concurrent interoperability of the two applications can be implemented by user selected intervention at any of a number of screen operational levels. The base image or window is considered to operate at a lower level, or below the level of the translucent image or window. Thus, the translucent image or window is known as the “overlay” image or window. Typically, the cursor is active at the particular level at which the user can operate. In any case, according to the invention, it may be useful to operate at either the base level, i.e., the level of the base image or window, or at the translucent or overlay level. In other words, user input is permitted at either the base image or the translucent image. By a particular user input with respect to an image, the user implements a selected computer implemented process and the process receives screen inputs which contact or are otherwise associated with a particular window as the computer implemented process is effective for processing the screen inputs. These various inputs are controllable selectively by the user, in that users can take specific actions to determine which of the levels will be active for them. This can, for example, be accomplished by action of clicking or activating a pen or stylus or by another well known action users are considered capable of actuating. A particular window just opened is automatically active, as the newest window created or activated. Another window or image can be activated merely by user selection in positioning the cursor over the window or image and clicking on the mouse, trackball or another applicable interface device.
While this invention has been described in terms of several preferred embodiments, it is contemplated that many alterations, permutations, and equivalents will be apparent to those skilled in the art. It is therefore intended that the following appended claims be interpreted as including all such alterations, permutations, and equivalents as fall within the true spirit and scope of the present invention.
Gough, Michael L., Gilley, Thomas S., Venolia, Gina D., MacDougald, Joseph J., Robbins, Greg M., Hansen, Jr., Daniel J., Oswal, Abhay
Patent | Priority | Assignee | Title |
Patent | Priority | Assignee | Title |
4555775, | Oct 07 1982 | UNIX SYSTEMS LABORATORIES, INC | Dynamic generation and overlaying of graphic windows for multiple active program storage areas |
4686522, | Feb 19 1985 | International Business Machines Corporation | Method of editing graphic objects in an interactive draw graphic system using implicit editing actions |
4783648, | Jul 01 1985 | Hitachi, Ltd. | Display control system for multiwindow |
4823281, | Apr 30 1985 | IBM Corporation | Color graphic processor for performing logical operations |
4827253, | May 18 1987 | Discovision Associates | Video compositing using a software linear keyer |
4868765, | Jan 02 1986 | Texas Instruments Incorporated | Porthole window system for computer displays |
4914607, | Apr 09 1986 | Hitachi, Ltd. | Multi-screen display control system and its method |
4954970, | Apr 08 1988 | Video overlay image processing apparatus | |
4959803, | Jun 26 1987 | Sharp Kabushiki Kaisha | Display control system |
4974196, | Sep 21 1987 | Hitachi, Ltd. | Method of processing commands for cataloged procedure in multi-window system |
4992781, | Jul 17 1987 | SHARP KABUSHIKI KAISHA, A CORP OF JAPAN | Image synthesizer |
5119476, | Jun 22 1988 | Bull S.A. | Method for generating dialogue-windows visually displayed on a computer-screen and equipment for implementing this method |
5124691, | Jul 15 1988 | Sharp Kabushiki Kaisha | Picture information display device |
5157384, | Apr 28 1989 | International Business Machines Corporation; INTERNATIONAL BUSINESS MACHINES CORPORATION, A CORP OF NY | Advanced user interface |
5185808, | Jun 06 1991 | Eastman Kodak Company | Method for merging images |
5233686, | Sep 24 1991 | GENERAL DYNAMICS INFORMATION SYSTEMS, INC | Open systems software backplane architecture for federated execution of independent application programs |
5252951, | Apr 28 1989 | INTERNATIONAL BUSINESS MACHINES CORPORATION A CORP OF NEW YORK | Graphical user interface with gesture recognition in a multiapplication environment |
5260697, | Nov 13 1990 | WANG LABORATORIES, INC , A CORP OF COMMONWEALTH OF MA | Computer with separate display plane and user interface processor |
5265202, | Aug 28 1992 | International Business Machines Corporation | Method and system for accessing visually obscured data in a data processing system |
5283560, | Jun 25 1991 | HEWLETT-PACKARD DEVELOPMENT COMPANY, L P | Computer system and method for displaying images with superimposed partially transparent menus |
5283867, | Jun 16 1989 | International Business Machines | Digital image overlay system and method |
5307452, | Sep 21 1990 | PIXAR, A CORP OF CA | Method and apparatus for creating, manipulating and displaying images |
5313227, | Apr 15 1988 | International Business Machines Corporation | Graphic display system capable of cutting out partial images |
5313571, | Oct 17 1991 | FUJI XEROX CO , LTD | Apparatus for storing and displaying graphs |
5333255, | Jan 03 1991 | XEROX CORPORATION, STAMFORD, COUNTY OF FAIRFIELD, CONNECTICUT A CORP OF NEW YORK | Apparatus for displaying a plurality of two dimensional display regions on a display |
5351067, | Jul 22 1991 | MEDIATEK INC | Multi-source image real time mixing and anti-aliasing |
5367453, | Aug 02 1993 | Apple Computer, Inc | Method and apparatus for correcting words |
5398309, | May 17 1993 | Intel Corporation | Method and apparatus for generating composite images using multiple local masks |
5425137, | Jan 26 1993 | US JVC Corporation | System and method for processing images using computer-implemented software objects representing lenses |
5425141, | Dec 18 1991 | Sun Microsystems, Inc. | Managing display windows of inter-related applications using hollowed windows |
5463726, | Nov 20 1990 | International Business Machines Corporation | Method and apparatus for graphic accessing of multiple software applications |
5463728, | Mar 10 1993 | AT&T Corp. | Electronic circuits for the graphical display of overlapping windows with transparency |
5467441, | Jul 21 1993 | Xerox Corporation | Method for operating on objects in a first image using an object-based model data structure to produce a second contextual image having added, replaced or deleted objects |
5467443, | Sep 25 1991 | Adobe Systems Incorporated | System and method for automatically generating derived graphic elements |
5469540, | Jan 27 1993 | Apple Inc | Method and apparatus for generating and displaying multiple simultaneously-active windows |
5469541, | May 10 1990 | International Business Machines Corporation | Window specific control of overlay planes in a graphics display system |
5475812, | Sep 11 1992 | International Business Machines Corporation | Method and system for independent control of multiple windows in a graphics display system |
5491495, | Nov 13 1990 | 3M Innovative Properties Company | User interface having simulated devices |
5528738, | Oct 06 1993 | Microsoft Technology Licensing, LLC | Method and apparatus for antialiasing raster scanned, polygonal shaped images |
5581243, | Jun 04 1990 | MICROSLATE INC , A CANADIAN CORP | Method and apparatus for displaying simulated keyboards on touch-sensitive displays |
5581670, | Jul 21 1993 | JB PARTNERS, LLC | User interface having movable sheet with click-through tools |
5590265, | Jul 27 1992 | Canon Kabushiki Kaisha | System which can display multiwindows and its window dosplay method |
5596690, | Jul 21 1993 | Xerox Corporation | Method and apparatus for operating on an object-based model data structure to produce a second image in the spatial context of a first image |
5613050, | Jan 15 1993 | International Business Machines Corporation | Method and apparatus for reducing illumination calculations through efficient visibility determination |
5617114, | Jul 21 1993 | Xerox Corporation | User interface having click-through tools that can be composed with other tools |
5638501, | May 10 1993 | Apple Inc | Method and apparatus for displaying an overlay image |
5651107, | Dec 15 1992 | Sun Microsystems, Inc | Method and apparatus for presenting information in a display system using transparent windows |
5652851, | Jul 21 1993 | Xerox Corporation | User interface technique for producing a second image in the spatial context of a first image using a model-based operation |
5684939, | Jul 09 1993 | Microsoft Technology Licensing, LLC | Antialiased imaging with improved pixel supersampling |
5729704, | Jul 21 1993 | Xerox Corporation | User-directed method for operating on an object-based model data structure through a second contextual image |
5798752, | Jul 21 1993 | Xerox Corporation | User interface having simultaneously movable tools and cursor |
5818455, | Jul 21 1993 | Xerox Corporation | Method and apparatus for operating on the model data structure of an image to produce human perceptible output using a viewing operation region having explicit multiple regions |
5831615, | Sep 30 1994 | Intel Corporation | Method and apparatus for redrawing transparent windows |
5949432, | Sep 30 1993 | Apple Inc | Method and apparatus for providing translucent images on a computer display |
6072489, | May 10 1993 | Apple Inc | Method and apparatus for providing translucent images on a computer display |
7505046, | May 02 2000 | Adobe Inc | Preserving opaque-like rendering in transparent 2D graphics using knockout groups |
EP280582, | |||
EP635779, | |||
EP635780, | |||
JP1991288891, | |||
JP2114319, | |||
JP3288891, | |||
RE41922, | May 10 1993 | Apple Inc | Method and apparatus for providing translucent images on a computer display |
Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
Apr 30 2013 | Apple Inc. | (assignment on the face of the patent) | / |
Date | Maintenance Fee Events |
May 05 2015 | ASPN: Payor Number Assigned. |
Date | Maintenance Schedule |
Jul 28 2018 | 4 years fee payment window open |
Jan 28 2019 | 6 months grace period start (w surcharge) |
Jul 28 2019 | patent expiry (for year 4) |
Jul 28 2021 | 2 years to revive unintentionally abandoned end. (for year 4) |
Jul 28 2022 | 8 years fee payment window open |
Jan 28 2023 | 6 months grace period start (w surcharge) |
Jul 28 2023 | patent expiry (for year 8) |
Jul 28 2025 | 2 years to revive unintentionally abandoned end. (for year 8) |
Jul 28 2026 | 12 years fee payment window open |
Jan 28 2027 | 6 months grace period start (w surcharge) |
Jul 28 2027 | patent expiry (for year 12) |
Jul 28 2029 | 2 years to revive unintentionally abandoned end. (for year 12) |