Method and apparatus for providing translucent images on a computer display

Abstract

A method and apparatus is described for producing a translucent image over a base image created on the display screen of a computer system by a selected first application program, and conducting image operations either on the base image created by the selected application program with reference to the translucent image produced, or conducting image operations on the translucent image with reference to the base image of the first application program. The first application program runs on a central processing unit (CPU) of a computer system to produce a base image, and another application program referred to as the overlay program is run to produce the translucent image such that portions of the base image which are overlapped by the overlay image are at least partially visible through the translucent image. There is also a mechanism for blending the first video data and the second video data to produce a blended image on the screen assembly.

Description

CROSS-REFERENCE TO FIG. 18 is a view of a Macintosh computer screen showing a desktop, a window produced by an application program called “AppleShare” and a utility program known as “PenBoard”;By translucent, it is further meant that the lines of a-particular image can be seen, but that the spaces between the lines and the spaces around the lines can be seen through.

In FIG. 22, the operating system, application program, overlay utility, system routines, etc., are shown in a somewhat hierarchical fashion. At the highest level is the operating system 1096 of the computer system 10 of FIG. 1. Running under the operating system 1096 is an application program 1098, such as the aforementioned AppleShare application program. Application program 1098, when it wants to open a window such as window 1044 of FIG. 18, calls a set of routines 1100 provided by the operating system 1096. More specifically, in the Macintosh operating system, application program 1098 calls a “New Window” routine 1102 which, in turn, calls a “Frame Rect” routine 1104. The Frame Rect routine uses a pointer table 1106 to call a “Shield Cursor” routine 1107 and a “Show Cursor” routine 1108. If the application program 1098 were running on system 1096 without the process 133 of the present invention, this would be the entirety of the calls to open up the window 1044 of FIG. 18. This process is extensively documented in the multi-volume reference set, Inside Macintosh, by C. Rose et al., Addison-Wesley Publishing Company, Inc., July 1988 and are well known to those skilled in the art of programming on the Macintosh operating system.

The implementation of computer implemented process 133 modifies this normal flow of routine calls in the following way. When the application program 1098 calls the New Window routine 1102 which calls the Frame Rect routine 1104, which attempts to call the Shield Cursor Routine, the Frame Rect routine 1104 instead calls a portion of the process of step 138 of FIG. 6B known as the Overlay Shield Cursor Patch 1110. This is accomplished by having the process 138 modify the pointer table 1106 such that when the Frame Rect routine 1104 is trying to call the Shield Cursor Routine 1107 it, instead, calls the Overlay Shield Cursor Patch 1110. After the Overlay Shield Cursor Patch 1110 completes its process, the Shield Cursor Routine 1107 is then called. As far as the Frame Rect routine 1104 is concerned, it does not know of the diversion of process control to the Overlay Shield Cursor Patch process 1110, and instead believes that it directly called the Shield Cursor Routine 1107.

The process step 138 of FIG. 6B similarly “tricks” the Frame Rect routine 1104 when it attempts to call the Show Cursor Routine 1108. In that instance, when the Frame Rect routine 1104 goes to the pointer table 1106 in an attempt to call the Show Cursor Routine 1108, process control is instead diverted to a process 1112 known as “Overlay Show Cursor Patch”. The Overlay Show Cursor Patch process 1112 interacts with a Blending Engine process 1114 to blend a first screen image 1116 generated by the Macintosh operating system and the application program, with a second image 1118 (in this case, the keyboard image) to form the blended image 1120. The operation of the Blending Engine will be discussed in greater detail subsequently. After the completion of the blending process of 1114, the Overlay Show Cursor Patch process 1112 turns over process control to the “Show Cursor Routine” process 1108. Again, as far as the Frame Rect routine 1104 is concerned, it made a direct call to the “Show Cursor Routine” 1108 and was ignorant of the diversion of the process control to the Overlay Show Cursor Patch 1112 and the Blending Engine 1114.

FIG. 23 illustrates an alternate embodiment of the present invention which has been optimized for screen-writing speed. While the process of FIG. 22 works very well, it requires that the entirety of the base screen 1116 be rewritten whenever the blended image 1120 is to be refreshed. The alternative process of FIG. 23 only refreshes the portions of the blended image that needs to be refreshed, thereby greatly increasing the writing speed to the screen 1040.

Much of the operation of the process illustrated in FIG. 23 is similar to that described in FIG. 22. An operating system 1172 supports an application program 1174 which, when it wants to open a window, calls a set of routines 1176 including a “New Window routine” 1178 and Frame Rect routine 1180. The Frame Rect routine 1180 then, as before, attempts to first call the Shield Cursor Routine 1182 first and then the Show Cursor Routine 1184. Again, as before, the pointer table is modified such that when the Frame Rect routine tries to call the Shield Cursor Routine 1182, it instead calls the Overlay Shield Cursor Patch 1186 of the present invention, and when the Frame Rect routine 1180 attempts to call the Show Cursor Routine 1184 it instead calls the Overlay Show Cursor Patch 1188. The Overlay Show Cursor Patch calls a Blending Engine 1190 which blends a partial base image 1192 with an overlay image 1194 to create a blended image 1196.

The system 1172, as part of its functioning, will make periodic calls to various system task processes 1198. The system task 1198 performs such functions as execute “Device Driver Code” and “Desk Accessory Code.” The process of the present invention opportunistically takes advantage of these periodic system task calls by modifying a pointer table 1200 to turn over process control to an Overlay System Task Patch 1202. This Overlay System Task Patch, along with the Overlay Shield Cursor Patch 1186, the Overlay Show Cursor Patch 1188, and the Blending Engine 1190 comprise the overlay utility 133 of FIGS. 6A and 6B in this second preferred embodiment.

FIG. 24 is used to illustrate the operation of the Blending Engine 1190 of FIG. 23 in greater detail. The process 138 of FIG. 6B remaps certain pages of VRAM to the RAM screen buffer when an overlay image contains objects that overlap these pages. The RAM overlay screen buffer 1194 is then merged with the RAM screen buffer 1192 in the Blending Engine 1190 by a process similar to that previously described and inserts the blended image into a “hole” 1204 of the VRAM screen buffer 1196. The portions 1206 and 1208 of the VRAM screen buffer remain the VRAM since the overlay image of the present invention does not overlap pages comprising these portions of the screen.

Since portions 1206 and 1208 are pages of VRAM screen buffer memory which are not overlapped, at least in part, by an overlay image of the present invention, these portions 1206 and 1208 can remain in VRAM screen buffer. VRAM screen buffer is much faster memory for video purposes than the RAM screen buffer 1192. Also, changes made to the RAM screen buffer 1192 or to the RAM overlay screen buffer 1194 that do not cause a change in portions 1206 and 1208 do not require that the system blend the portions 1206 and 1208. The combination of these factors substantially increase the blending speed of the VRAM screen buffer and therefore of the display on screen 1040.

FIGS. 25 and 26 are used to illustrate an alternate embodiment of the present invention wherein the blending of the base image and the overlay image are performed in the video driver hardware rather than within a computer implemented process on the CPU. In FIG. 25, a prior art video driver system of a Macintosh computer system is illustrated. In this prior art example, the video driver circuit 1302 is coupled to an address bus 1304 and a data bus 1306 connected to a Motorola 68030 microprocessor. The video driver circuit 1302 includes a color screen controller CSC 1307, and two banks of VRAM 1308 and 1310. The CSC 1307 produces LCD control and data on a bus 1312 which control a black and white or color liquid crystal display (LCD). For example, the video driver circuit 1302 can drive an Esher LCD circuit for a 640 by 400 bit display, with eight bits of information per pixel.

In FIG. 26, a modified video driver circuit 1302′ is coupled to the same Motorola 68030 address bus 1304 and data bus 1306, and includes the same CSC 1307, VRAM 1308, and VRAM 1310. However, the data and address connections have been modified as indicated. In this implementation, data from the screen buffer and the overlay screen buffer are input into the VRAM of modified video driver circuit 1302′, and combined therein to provide LCD control and blended data on the bus 1312. Again, the video driver circuit 1302′ can control a black and white or color LCD, except this time instead of having eight bits per pixel, there are four bits allocated to the base image and four bits allocated to the overlay image. A color look-up table (CLUT)—not shown—of CSC 1307 is loaded with 256 entries which detail each possible combination of bits from the 4 bit screen and the 4 bit overlay, and what the resultant blended value is. The color capability of the CSC 1307 is therefore no longer used for color look-up, and is instead used for the blending values. This technique makes it possible to use off-the-shelf integrated circuits, such as the CSC 1307 which is available from Chips & Technologies, Inc. of San Jose, Calif., to perform an entirely new operation.

In summary, the method of the invention includes establishing translucent images on a display screen including displaying a translucent images and conducting image operations enabled by the translucent image. Image operations can be any kind of operation conducted on an image or window. Drawing an image, placing an image, or for that matter modifying, moving, expanding, or changing an image or a window, are considered to be image operations. A reference image could be provided by a selected first application program. The translucent image could be produced by a selected second application program. The user is thus enabled to make sketches on the translucent image or window based upon what he or she sees on the base image produced by the first application program. This is made possible without any direct intervention in the operations of the first application program. In short, the features of the first application program are advantageously employed, without any modification of the first application program itself. The technical enablement of this cooperative screen is found in a feature of the invention according to which the second application program intercepts certain screen inputs of the first application program and uses them to supply the screen input needed as to the second application program.

The image operations enabled by the concurrent interoperability of the two applications can be implemented by user selected intervention at any of a number of screen operational levels. The base image or window is considered to operate at a lower level, or below the level of the translucent image or window. Thus, the translucent image or window is known as the “overlay” image or window. Typically, the cursor is active at the particular level at which the user can operate. In any case, according to the invention, it may be useful to operate at either the base level, i.e., the level of the base image or window, or at the translucent or overlay level. In other words, user input is permitted at either the base image or the translucent image. By a particular user input with respect to an image, the user implements a selected computer implemented process and the process receives screen inputs which contact or are otherwise associated with a particular window as the computer implemented process is effective for processing the screen inputs. These various inputs are controllable selectively by the user, in that users can take specific actions to determine which of the levels will be active for them. This can, for example, be accomplished by action of clicking or activating a pen or stylus or by another well known action users are considered capable of actuating. A particular window just opened is automatically active, as the newest window created or activated. Another window or image can be activated merely by user selection in positioning the cursor over the window or image and clicking on the mouse, trackball or another applicable interface device.

While this invention has been described in terms of several preferred embodiments, it is contemplated that many alterations, permutations, and equivalents will be apparent to those skilled in the art. It is therefore intended that the following appended claims be interpreted as including all such alterations, permutations, and equivalents as fall within the true spirit and scope of the present invention.

Claims

1. A method for establishing a translucent window having a translucent window background and a translucent window frame on a display screen of a computer system, comprising the steps of:

displaying a translucent window on a display screen such that a base window can be seen through said translucent window, and

conducting image operations on at least one of said translucent window and said base window.

2. A method as recited in claim 1 wherein said base window is produced on said display screen by a first computer implemented process running on said computer system, and said translucent window is produced by a second computer implemented process running on said computer system.

3. A method as recited in claim 2 wherein said second computer implemented process receives screen inputs which are associated with said translucent window and processes said screen inputs.

4. A method as recited in claim 2, wherein said second computer implemented process receives screen inputs which are physically applied in connection with said translucent window and processes said screen inputs.

5. A method as recited in claim 2, wherein said second computer implemented process receives screen inputs which make image contact with said translucent window and processes said screen inputs.

6. A method as recited in claim 1 wherein image operations are conducted with respect to said translucent window.

7. A method as recited in claim 1 wherein image operations are conducted below said translucent window.

8. A method as recited in claim 1 wherein image operations are conducted in connection with windows referenced by a cursor indication.

9. A method for displaying images on a display screen of a computer system, comprising the steps of:

displaying a base image on a display screen of a computer system; and

displaying a translucent image on said screen such that portions of said base image which are covered by said translucent image are at least partially visible through said translucent image.

10. A method as recited in claim 9 wherein said base image and said translucent image are produced by independent computer implemented processes.

11. A method as recited in claim 9 wherein said base image is active to receive user inputs.

12. A method as recited in claim 9 wherein said translucent image is active to receive user inputs.

13. A method as recited in claim 12 wherein said translucent image is made active by user action.

14. A method as recited in claim 12 wherein said translucent image is made active by positioning the cursor at a portion of the translucent image and conducting a select action.

15. A method as recited in claim 12 wherein said translucent image is made active by clicking a mouse when the cursor is over a portion of the translucent image.

16. A method as recited in claim 9 wherein said translucent image and said base image are selectably active to receive user inputs.

17. A method for displaying images on a display screen of a computer system comprising the steps of:

running an application program on the central processing unit (CPU) of a computer system to produce a base image on a display screen coupled to said CPU; and

running an overlay program on said CPU to produce a translucent image on said display screen such that portions of said base image are overlapped by said translucent image and are at least partially visible through said translucent image.

18. A method as recited in claim 17 wherein said step of running an overlay program comprises the steps of:

displaying a translucent image on said display screen;

intercepting screen inputs which contact the translucent image;

processing said intercepted screen inputs in said CPU; and

updating said application program based upon said processed screen inputs.

19. A method as recited in claim 18 wherein said step of displaying a translucent image comprises the step of blending a translucent image with said base image.

20. A method as recited in claim 19 wherein said step of blending is accomplished in said CPU.

21. A method as recited in claim 19 wherein said step of blending is accomplished externally to said CPU.

22. A method as recited in claim 18 wherein said step of processing said intercepted screen inputs includes redirecting at least one page of memory within the memory management means of said computer system.

23. A method of performing image operations in a computer system having a display screen, including the steps of:

presenting a first selected image with respect to which image operations are desired, and

producing a translucent image effective for overlapping at least a portion of said first selected image.

24. A method according to claim 23 wherein said first selected image contains features of interest, and image operations are conducted on said translucent image with respect to said features of interest.

25. A method according to claim 23 wherein said translucent image contains features of interest, and image operations are conducted with respect to said first selected image based upon said features of interest.

26. A computer system comprising:

a central processing unit (CPU);

screen means for displaying images, said screen means being coupled to said CPU;

display means coupled to said screen means for displaying a translucent image on said screen means; and

means for conducting image operations on a region including the level of a translucent image produced by said display means and the level beneath the translucent image.

27. A computer system according to claim 26 wherein said means for conducting image operations performs image operations with reference to a translucent image on said screen means.

28. A computer system according to claim 26 wherein said means for conducting image operations performs image operations selectably with reference to a translucent image on said screen means and below the level of a translucent image on said screen means.

29. A method comprising:

producing a first image and a second image in memory, wherein at least a portion of the first image includes first blendable units, and at least a portion of the second image includes second blendable units, at least one of the first blendable units being blended with at least one of the second blendable units;

identifying updated first blendable units of the first image;

blending the updated first blendable units of the first image with the at least one of the second blendable units in a memory of an electronic device to yield a blended image; and

displaying the blended image on a display of the electronic device.

30. The method of claim 29 further comprising:

activating the first image to receive inputs;

receiving inputs in the first image effective to require an update of the first blendable units.

31. The method of claim 30 wherein the first image is overlaid by the second image.

32. The method of claim 30 wherein the first image is overlaid by the second image while the first image is receiving inputs.

33. The method of claim 30, wherein the first image is produced by a first application.

34. The method of claim 33, wherein the received inputs are user inputs received via a user interface of an operating system and passed to the first application.

35. The method of claim 29, wherein the blended image is comprised of a portion of pixel values contributed by the first blendable unit, and a portion of pixel values contributed by the second blendable unit.

36. The method of claim 29, wherein the blended image appears as is the second image is over the first image, and appears as though the second image can be seen and seen through such that the portion of the first image can be seen through the second image.

37. A system comprising:

a memory configured to store a first image and a second image, wherein at least a portion of the first image includes first blendable units, and at least a portion of the second image includes second blendable units, at least one of the first blendable units being blended with at least one of the second blendable units;

a processor configured to identify updated first blendable units of the first image, and blend the updated first blendable units of the first image with the at least one of the second blendable units in the memory of an electronic device to yield a blended image;

a display configured to display the blended image.

38. The system of claim 37, wherein the processor is futher configured to activate the first image to receive inputs, and to receive inputs in the first image effective to require an update of the first blendable units.

39. The system of claim 38 wherein the first image is overlaid by the second image.

40. The system of claim 38 wherein the first image is overlaid by the second image while the first image is receiving inputs.

41. The system of claim 38, wherein the first image is produced by a first application executed by the processor.

42. The system of claim 41, wherein the received inputs are user inputs received via a user interface of an operating system and passed to the first application.

43. The system of claim 37, wherein the blended image is comprised of a portion of pixel values contributed by the first blendable unit, and a portion of pixel values contributed by the second blendable unit.

44. The system of claim 37, wherein the blended image appears as is the second image is over the first image, and appears as though the second image can be seen and seen through such that the portion of the first image can be seen through the second image.

45. A device comprising:

a non-transitory computer-readable medium configured to store computer-executable instructions; and

computer-executable instructions stored on the non-transitory computer-readable medium that when executed are effective to cause a computer processor to:

produce a first image and a second image in memory, wherein at least a portion of the first image includes first blendable units, and at least a portion of the second image includes second blendable units, at least one of the first blendable units being blended with at least one of the second blendable units;

identify updated first blendable units of the first image;

blend the updated first blendable units of the first image with the at least one of the second blendable units in a memory of an electronic device to yield a blended image; and

display the blended image on a display of the electronic device.

46. The device of claim 45 further comprising:

activating the first image to receive inputs;

receiving inputs in the first image effective to require an update of the first blendable units.

47. The device of claim 46 wherein the first image is overlaid by the second image.

48. The device of claim 46 wherein the first image is overlaid by the second image while the first image is receiving inputs.

49. The device of claim 46, wherein the first image is produced by a first application.

50. The device of claim 49, wherein the received inputs are user inputs received via a user interface of an operating system and passed to the first application.

51. The device of claim 45, wherein the blended image is comprised of a portion of pixel values contributed by the first blendable unit, and a portion of pixel values contributed by the second blendable unit.

52. The device of claim 45, wherein the blended image appears as is the second image is over the first image, and appears as though the second image can be seen and seen through such that the portion of the first image can be seen through the second image.