Method and apparatus for providing translucent images on a computer display

Abstract

A method and apparatus is described for producing a translucent image over a base image created on the display screen of a computer system by a selected first application program, and conducting image operations either on the base image created by the selected application program with reference to the translucent image produced, or conducting image operations on the translucent image with reference to the base image of the first application program. The first application program runs on a central processing unit (CPU) of a computer system to produce a base image, and another application program referred to as the overlay program is run to produce the translucent image such that portions of the base image which are overlapped by the overlay image are at least partially visible through the translucent image. There is also a mechanism for blending the first video data and the second video data to produce a blended image on the screen assembly.

Description

CROSS-REFERENCE TO FIG. 18 is a view of a Macintosh computer screen showing a desktop, a window produced by an application program called “AppleShare” and a utility program known as “PenBoard”;By translucent, it is further meant that the lines of a-particular image can be seen, but that the spaces between the lines and the spaces around the lines can be seen through.

In FIG. 22, the operating system, application program, overlay utility, system routines, etc., are shown in a somewhat hierarchical fashion. At the highest level is the operating system 1096 of the computer system 10 of FIG. 1. Running under the operating system 1096 is an application program 1098, such as the aforementioned AppleShare application program. Application program 1098, when it wants to open a window such as window 1044 of FIG. 18, calls a set of routines 1100 provided by the operating system 1096. More specifically, in the Macintosh operating system, application program 1098 calls a “New Window” routine 1102 which, in turn, calls a “Frame Rect” routine 1104. The Frame Rect routine uses a pointer table 1106 to call a “Shield Cursor” routine 1107 and a “Show Cursor” routine 1108. If the application program 1098 were running on system 1096 without the process 133 of the present invention, this would be the entirety of the calls to open up the window 1044 of FIG. 18. This process is extensively documented in the multi-volume reference set, Inside Macintosh, by C. Rose et al., Addison-Wesley Publishing Company, Inc., July 1988 and are well known to those skilled in the art of programming on the Macintosh operating system.

The implementation of computer implemented process 133 modifies this normal flow of routine calls in the following way. When the application program 1098 calls the New Window routine 1102 which calls the Frame Rect routine 1104, which attempts to call the Shield Cursor Routine, the Frame Rect routine 1104 instead calls a portion of the process of step 138 of FIG. 6B known as the Overlay Shield Cursor Patch 1110. This is accomplished by having the process 138 modify the pointer table 1106 such that when the Frame Rect routine 1104 is trying to call the Shield Cursor Routine 1107 it, instead, calls the Overlay Shield Cursor Patch 1110. After the Overlay Shield Cursor Patch 1110 completes its process, the Shield Cursor Routine 1107 is then called. As far as the Frame Rect routine 1104 is concerned, it does not know of the diversion of process control to the Overlay Shield Cursor Patch process 1110, and instead believes that it directly called the Shield Cursor Routine 1107.

The process step 138 of FIG. 6B similarly “tricks” the Frame Rect routine 1104 when it attempts to call the Show Cursor Routine 1108. In that instance, when the Frame Rect routine 1104 goes to the pointer table 1106 in an attempt to call the Show Cursor Routine 1108, process control is instead diverted to a process 1112 known as “Overlay Show Cursor Patch”. The Overlay Show Cursor Patch process 1112 interacts with a Blending Engine process 1114 to blend a first screen image 1116 generated by the Macintosh operating system and the application program, with a second image 1118 (in this case, the keyboard image) to form the blended image 1120. The operation of the Blending Engine will be discussed in greater detail subsequently. After the completion of the blending process of 1114, the Overlay Show Cursor Patch process 1112 turns over process control to the “Show Cursor Routine” process 1108. Again, as far as the Frame Rect routine 1104 is concerned, it made a direct call to the “Show Cursor Routine” 1108 and was ignorant of the diversion of the process control to the Overlay Show Cursor Patch 1112 and the Blending Engine 1114.

FIG. 23 illustrates an alternate embodiment of the present invention which has been optimized for screen-writing speed. While the process of FIG. 22 works very well, it requires that the entirety of the base screen 1116 be rewritten whenever the blended image 1120 is to be refreshed. The alternative process of FIG. 23 only refreshes the portions of the blended image that needs to be refreshed, thereby greatly increasing the writing speed to the screen 1040.

Much of the operation of the process illustrated in FIG. 23 is similar to that described in FIG. 22. An operating system 1172 supports an application program 1174 which, when it wants to open a window, calls a set of routines 1176 including a “New Window routine” 1178 and Frame Rect routine 1180. The Frame Rect routine 1180 then, as before, attempts to first call the Shield Cursor Routine 1182 first and then the Show Cursor Routine 1184. Again, as before, the pointer table is modified such that when the Frame Rect routine tries to call the Shield Cursor Routine 1182, it instead calls the Overlay Shield Cursor Patch 1186 of the present invention, and when the Frame Rect routine 1180 attempts to call the Show Cursor Routine 1184 it instead calls the Overlay Show Cursor Patch 1188. The Overlay Show Cursor Patch calls a Blending Engine 1190 which blends a partial base image 1192 with an overlay image 1194 to create a blended image 1196.

The system 1172, as part of its functioning, will make periodic calls to various system task processes 1198. The system task 1198 performs such functions as execute “Device Driver Code” and “Desk Accessory Code.” The process of the present invention opportunistically takes advantage of these periodic system task calls by modifying a pointer table 1200 to turn over process control to an Overlay System Task Patch 1202. This Overlay System Task Patch, along with the Overlay Shield Cursor Patch 1186, the Overlay Show Cursor Patch 1188, and the Blending Engine 1190 comprise the overlay utility 133 of FIGS. 6A and 6B in this second preferred embodiment.

FIG. 24 is used to illustrate the operation of the Blending Engine 1190 of FIG. 23 in greater detail. The process 138 of FIG. 6B remaps certain pages of VRAM to the RAM screen buffer when an overlay image contains objects that overlap these pages. The RAM overlay screen buffer 1194 is then merged with the RAM screen buffer 1192 in the Blending Engine 1190 by a process similar to that previously described and inserts the blended image into a “hole” 1204 of the VRAM screen buffer 1196. The portions 1206 and 1208 of the VRAM screen buffer remain the VRAM since the overlay image of the present invention does not overlap pages comprising these portions of the screen.

Since portions 1206 and 1208 are pages of VRAM screen buffer memory which are not overlapped, at least in part, by an overlay image of the present invention, these portions 1206 and 1208 can remain in VRAM screen buffer. VRAM screen buffer is much faster memory for video purposes than the RAM screen buffer 1192. Also, changes made to the RAM screen buffer 1192 or to the RAM overlay screen buffer 1194 that do not cause a change in portions 1206 and 1208 do not require that the system blend the portions 1206 and 1208. The combination of these factors substantially increase the blending speed of the VRAM screen buffer and therefore of the display on screen 1040.

FIGS. 25 and 26 are used to illustrate an alternate embodiment of the present invention wherein the blending of the base image and the overlay image are performed in the video driver hardware rather than within a computer implemented process on the CPU. In FIG. 25, a prior art video driver system of a Macintosh computer system is illustrated. In this prior art example, the video driver circuit 1302 is coupled to an address bus 1304 and a data bus 1306 connected to a Motorola 68030 microprocessor. The video driver circuit 1302 includes a color screen controller CSC 1307, and two banks of VRAM 1308 and 1310. The CSC 1307 produces LCD control and data on a bus 1312 which control a black and white or color liquid crystal display (LCD). For example, the video driver circuit 1302 can drive an Esher LCD circuit for a 640 by 400 bit display, with eight bits of information per pixel.

In FIG. 26, a modified video driver circuit 1302′ is coupled to the same Motorola 68030 address bus 1304 and data bus 1306, and includes the same CSC 1307, VRAM 1308, and VRAM 1310. However, the data and address connections have been modified as indicated. In this implementation, data from the screen buffer and the overlay screen buffer are input into the VRAM of modified video driver circuit 1302′, and combined therein to provide LCD control and blended data on the bus 1312. Again, the video driver circuit 1302′ can control a black and white or color LCD, except this time instead of having eight bits per pixel, there are four bits allocated to the base image and four bits allocated to the overlay image. A color look-up table (CLUT)—not shown—of CSC 1307 is loaded with 256 entries which detail each possible combination of bits from the 4 bit screen and the 4 bit overlay, and what the resultant blended value is. The color capability of the CSC 1307 is therefore no longer used for color look-up, and is instead used for the blending values. This technique makes it possible to use off-the-shelf integrated circuits, such as the CSC 1307 which is available from Chips & Technologies, Inc. of San Jose, Calif., to perform an entirely new operation.

In summary, the method of the invention includes establishing translucent images on a display screen including displaying a translucent images and conducting image operations enabled by the translucent image. Image operations can be any kind of operation conducted on an image or window. Drawing an image, placing an image, or for that matter modifying, moving, expanding, or changing an image or a window, are considered to be image operations. A reference image could be provided by a selected first application program. The translucent image could be produced by a selected second application program. The user is thus enabled to make sketches on the translucent image or window based upon what he or she sees on the base image produced by the first application program. This is made possible without any direct intervention in the operations of the first application program. In short, the features of the first application program are advantageously employed, without any modification of the first application program itself. The technical enablement of this cooperative screen is found in a feature of the invention according to which the second application program intercepts certain screen inputs of the first application program and uses them to supply the screen input needed as to the second application program.

The image operations enabled by the concurrent interoperability of the two applications can be implemented by user selected intervention at any of a number of screen operational levels. The base image or window is considered to operate at a lower level, or below the level of the translucent image or window. Thus, the translucent image or window is known as the “overlay” image or window. Typically, the cursor is active at the particular level at which the user can operate. In any case, according to the invention, it may be useful to operate at either the base level, i.e., the level of the base image or window, or at the translucent or overlay level. In other words, user input is permitted at either the base image or the translucent image. By a particular user input with respect to an image, the user implements a selected computer implemented process and the process receives screen inputs which contact or are otherwise associated with a particular window as the computer implemented process is effective for processing the screen inputs. These various inputs are controllable selectively by the user, in that users can take specific actions to determine which of the levels will be active for them. This can, for example, be accomplished by action of clicking or activating a pen or stylus or by another well known action users are considered capable of actuating. A particular window just opened is automatically active, as the newest window created or activated. Another window or image can be activated merely by user selection in positioning the cursor over the window or image and clicking on the mouse, trackball or another applicable interface device.

While this invention has been described in terms of several preferred embodiments, it is contemplated that many alterations, permutations, and equivalents will be apparent to those skilled in the art. It is therefore intended that the following appended claims be interpreted as including all such alterations, permutations, and equivalents as fall within the true spirit and scope of the present invention.

Claims

1. A computer-implemented method for establishing a translucent window having a translucent window background and a translucent window frame on a display screen of a computer system, comprising the steps of:

displaying a base window on a display screen;

displaying a translucent window on a the display screen using an overlay program such that a the translucent window partially obscures the base window can in a manner the permits the base window to be seen through said the translucent window, and;

conducting image operations on at least one of said the translucent window and said the base window; and

updating the display of the at least one of the translucent window and the base window on the display screen based on the conducted image operations.

2. A The method as recited in claim 1 wherein said the base window is produced on said the display screen by a first computer implemented process running on said computer system, and said the translucent window is produced by a second computer implemented process running on said computer system.

3. A The method as recited in claim 2 wherein said the second computer implemented process receives screen inputs which are associated with said the translucent window and processes said the screen inputs.

4. A The method as recited in claim 2, wherein said the second computer implemented process receives screen inputs which are physically applied in connection with said the translucent window and processes said the screen inputs.

5. A The method as recited in claim 2, wherein said the second computer implemented process receives screen inputs which make image contact with said the translucent window and processes said the screen inputs.

6. A The method as recited in claim 1 wherein image operations are conducted with respect to said the translucent window.

7. A The method as recited in claim 1 wherein image operations are conducted below said the translucent window.

8. A The method as recited in claim 1 wherein image operations are conducted in connection with windows referenced by a cursor indication.

9. A method for displaying images on a display screen of a computer system, comprising the steps of:

displaying a base image on a display screen of a computer system; and

displaying a translucent image on said screen such that portions of said base image which are covered by said translucent image are at least partially visible through said translucent image.

10. A method as recited in claim 9 comprising:

storing a base image and a top image in one or more buffers; and

blending the base image and the top image in a second buffer such that, when displayed, portions of the base image which are covered by the top image are at least partially obscured, yet partially visible through the top image such that the top image appears translucent,

wherein said the base image and said translucent the top image are produced by independent computer implemented processes.;

conducting image operations on at least one of the top image and the base image; and

updating the display of the at least one of the translucent window and the base window on the display screen based on the conducted image operations.

11. A method as recited in claim 9 wherein said base image is active to receive user inputs.

12. A method as recited in claim 9 comprising:

displaying a base image made up of a plurality of pixels;

displaying a translucent image made up of a plurality of pixels on the screen such that the pixels making up a portion of the base image that is covered by the translucent image are blended with the pixels of the translucent image so that the covered portion of the base image is at least partially visible through the translucent image,

wherein said the translucent image is active to receive and receives user inputs effective to induce image operations on the translucent image; and

updating the translucent window on the screen based on the induced image operations.

13. A The method as recited in claim 12 wherein said the translucent image is made active by user action.

14. A The method as recited in claim 12 wherein said the translucent image is made active by positioning the cursor at a portion of the translucent image and conducting a select action.

15. A The method as recited in claim 12 wherein said the translucent image is made active by clicking a mouse when the cursor is over a portion of the translucent image.

16. A method as recited in claim 9 comprising:

displaying a base image; and

displaying a translucent image such that portions of the base image which are covered by the translucent image are blended with portions of the translucent image so that the portions of the base image are at least partially visible through the translucent image,

wherein said the translucent image and said the base image are selectably active to receive user inputs effective to induce image operations on the translucent image; and

updating at least one of the translucent window and the base window on the display screen based on the conducted image operations.

17. A method for displaying images on a display screen of a computer system comprising the steps of:

running an application program on the central processing unit (CPU) of a computer system to produce a base image on a display screen coupled to said CPU; and

running an overlay program on said CPU to produce a translucent image on said display screen such that portions of said base image are overlapped by said translucent image and are at least partially visible through said translucent image.

18. A method as recited in claim 17 comprising:

running an application program on one or more processors to produce a base image on a display; and

running an overlay program on the processor(s) to produce a translucent image on the display simultaneously with the display of the base image such that portions of the base image are overlapped by the translucent image and are partially obscured by the translucent image,

wherein said step of running an the overlay program comprises the steps of:

displaying a the translucent image on said the display screen;

intercepting screen inputs which contact for the translucent image, the screen inputs inputting image operations to the translucent image;

processing said the intercepted screen inputs in said CPU the one or more processors effective to conduct image operations on the translucent image; and

updating said the application program based upon said the processed screen inputs to reflect the image operations.

19. A The method as recited in claim 18 wherein said step of the displaying a translucent image comprises the step of blending a translucent image with said the base image.

20. A The method as recited in claim 19 wherein said step of the blending is accomplished in said the CPU.

21. A The method as recited in claim 19 wherein said step of the blending is accomplished externally to said the CPU.

22. A The method as recited in claim 18 wherein said step of the processing said the intercepted screen inputs includes redirecting at least one page of memory within the memory management means of said the computer system.

23. A method of performing image operations in a computer system having a display screen, including the steps of:

presenting a first selected image with respect to which image operations are desired, and

producing a translucent image effective for overlapping at least a portion of said first selected image.

24. A method according to claim 23 comprising:

presenting a first image with respect to which image operations are desired;

producing a second image effective for overlapping and obscuring at least a portion of the first selected image; and

transforming the second image into a translucent image so that the portions of the first image that are overlapped by the second image are only partially obscured by the second image,

wherein said the first selected image contains features of interest, and image operations are conducted on said the translucent image with respect to said the features of interest.

25. A method according to claim 23 comprising:

presenting a first image with respect to which image operations are desired;

producing a second image effective for overlapping and obscuring at least a portion of the first selected image; and

transforming the second image into a translucent image so that the portions of the first image that are overlapped by the second image are only partially obscured by the second image, wherein said the translucent image contains features of interest, and image operations are conducted with respect to said the first selected image based upon said the features of interest.

26. A computer system comprising:

a central processing unit (CPU);

screen means for displaying images, said screen means being coupled to said CPU;

display means coupled to said screen means for displaying a translucent image on said screen means; and

means for conducting image operations on a region including the level of a translucent image produced by said display means and the level beneath the translucent image.

27. A computer system according to claim 26 wherein said means for conducting image operations performs image operations with reference to a translucent image on said screen means.

28. A computer system according to claim 26 wherein said means for conducting image operations performs image operations selectably with reference to a translucent image on said screen means and below the level of a translucent image on said screen means.

29. A method comprising:

displaying a translucent window having a translucent window background and a translucent window frame on a display screen of an electronic device, such that a base window can be seen through the translucent window;

conducting image operations on at least one of the translucent window and the base window; and

updating the display of the at least one of the translucent window and the base window on the display screen based on the conducted image operations.

30. A method comprising:

running an application program on one or more processors of an electronic device to produce a base image; and

running an overlay program on the processor(s) to produce a translucent image such that portions of the base image are overlapped by the translucent image are blended with the translucent image and are at least partially visible through the translucent image,

wherein running an overlay program comprises:

providing the translucent image;

intercepting screen inputs which for the translucent image, the screen inputs inputting image operations to the translucent image;

processing the intercepted screen inputs in the processor(s) effective to conduct image operations on the translucent image; and

updating the application program based upon the processed screen inputs to reflect the image operations.

31. The method of claim 30, wherein the translucent image is a translucent keyboard.

32. The method of claim 31, wherein the image operations input characters selected by the screen inputs that contact the translucent keyboard.

33. The method of claim 18, wherein the translucent image is a translucent keyboard.

34. The method of claim 31, wherein the application program is updated with characters selected by the screen inputs that contact the translucent keyboard.

35. An electronic device implemented method comprising:

displaying a base window;

displaying a translucent window overlapping at least a portion of the base window, the portions of the translucent window and the base window which overlap being blended, whereby the contents of both windows are visible; and

receiving inputs into the translucent window effective to induce image operations on the translucent image and displaying a representation of the image operations in the base window.

36. The method of claim 35, wherein the translucent window contains a keyboard configured for receiving inputs into individually displayed keys, each key corresponding to a character, whereby a received input selects one of the characters to be displayed in the base window.