Method and an apparatus for determining a deviation between an actual direction of a launched projectile and a predetermined direction

Abstract

A system and method for relating radar data to an image to track the flight of a path of a projectile includes a radar, an imager providing an image of an area into which a projectile is to be launched, a controller receiving the image and data from the radar relative to at least a portion of a trajectory of the projectile, the controller generating trajectory data by correlating the radar data and the image to determine the portion of the trajectory relative to at least one feature in the image, and a display displaying the trajectory data.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No. 14/536,827, filed on Nov. 10, 2014, now U.S. Pat. No. 9,857,459, which is a continuation of U.S. application Ser. No. 13/303,889, filed on Nov. 23, 2011, now U.S. Pat. No. 8,912,945, which is a divisional of U.S. application Ser. No. 11/630,904, filed on Jul. 20, 2009, now U.S. Pat. No. 8,085,188, which is a National Phase entry of PCT Application No. PCT/DK2005/000456, filed on Jul. 1, 2005, which claims priority under 35 U.S.C. § 119(e) to U.S. Provisional Application No. 60/584,483, filed on Jul. 2, 2004, in the U.S. Patent and Trademark Office, the entire contents of all of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to the determination of a deviation between an actual direction and a desired direction of a launched projectile, such as a golf ball I another type of sports ball, or a projectile launched by a weapon.

More particularly, the desired direction is indicated using an image of a target area toward which the projectile is launched or is intended to be launched.

SUMMARY OF THE INVENTION

In a first aspect, the invention relates to a method of determining a deviation between an actual direction of a launched projectile and a predetermined direction, the method comprising:

• • providing an image representing a target area,
  • indicating a position in the image, the indicated position corresponding to a position in the target area,
  • determining the actual direction of the projectile,
  • determining a launch position of the projectile,
  • determining, from the launch position and the corresponding position, the predetermined direction, and
  • determining the deviation between the actual direction of the projectile and the predetermined direction.

In the present context, an image may be an actually taken image (taken using e.g. a camera) representing the target area, or it may represent more schematically the target area or parts therein, such as a number of elements present in the target area (trees, flags, houses or other identifiable elements).

The target area is normally an area, including all elements therein, as defined either by boundaries of the image or defined by elements in or at the area. Alternatively, the target area needs not be directly defined with boundaries but is merely an area surrounding or comprising the position corresponding to the indicated position.

The position in the target area normally is a position in or at the ground plane of the target area. This, however, is not required. Any position, such as on a vertical part in the target area (such as a target provided on a tree, a wall or the like) may be used as well as any elevated or lowered position (a roof or the like).

The correlation of the position in the image to the position in the target area will depend on the actual type of image and the angle from which the target area is represented in the image. Normally, there is a one-to-one correspondence between positions in the image and in the target area, when the image is of a type representing the target area when viewed from a certain position. Normally, this position will be the position, or a position close to, where the indication is performed in order to facilitate the selection or indication of the position in the image. Then, the image indicates the target area as seen from that position.

Alternatively, the image may represent the target area, such as seen from the air in the manner as maps are normally drawn.

It is noted that the projectile needs not hit or land in the target area and it needs not be aimed at the target area. The position indicated in the target area is in the desired direction, but the actual path desired may extend further or not as far as the target area.

Presently, the projectile may be any flying object, such as a sports ball (golf ball, base ball, foot ball, hand ball, basket ball), or a projectile launched by a weapon (hand gun, machine gun, missile, grenade) or the like.

The launch position may be a position from which the projectile was launched either by being hit (such as by a club, a bat, a hand or a foot), released from a hand then being thrown, launched from a launch pad, such as a barrel of a weapon, a tee fixing the ball before launch, or a surface on which the projectile rests before launch.

The launch position may be determined in a number of manners. One manner is to simply assume or dictate that the launch takes place at a predetermined position or within a predetermined area.

Another manner of providing the launch position is to have the launch position also within the image, where a plurality of images are taken, and a difference therein is identified corresponding to the projectile having been launched. This difference may be the projectile missing in a later image compared to one or more images taken before launch.

The actual direction of the projectile is the direction of the projectile while in flight. The direction may be derived from the actual path of the projectile. The direction may be determined in a predetermined plane, in that the direction may vary in a 3-D scenario (such as the path of a golf ball) but where the most interesting direction is that of the 3D path projected on to the ground plane.

In the present context, the deviation may be determined in any suitable manner. This deviation may be determined in a predetermined plane in which also the directions have been projected, if required. This is often a horizontal plane or a plane of the ground between the launch position and the position corresponding to the indicated position.

The deviation may be an angular deviation between the two directions, such as in a predetermined plane. The plane may be both horizontal and/or vertical, or deviations in multiple planes may be provided.

Alternatively or in addition, the deviation may be a distance, such as in a predetermined plane, between the actual direction and the corresponding position or between the predetermined direction and e.g. a landing point (see below) of the projectile.

In a preferred embodiment, step 1) comprises providing, with a camera, the image of the target area. The image may be provided as a video sequence, such as in real time, of the target area or may be a still image, such as an Image on a monitor or display, or represented as a hard copy.

Normally, the camera will be positioned close to a position where the position in the image is indicated. This indication may be an indication on a display or monitor, which may be a touch pad or other touch sensitive display means, whereby indication may be performed simply by touching the position in the image. Alternatively, a pointing device may be used, such as a computer mouse, an optical pen, or the like.

In general, a computer or other calculating means may be used for controlling the determination, the indication! the providing or taking of the image, the pointing, and the like.

In one embodiment, step 1) comprises providing an image comprising a plurality of predetermined positions, and step 2) comprises indicating one of the predetermined positions. In this manner, the correlation between positions in the image and the target area may be simpler and may merely be a table describing the correlation. In fact, the positions may have simple numbers, and the selecting of a position may be the selection or indication, such as on a keyboard, of a number.

A particular aspect of this embodiment is one wherein the target area comprises a plurality of predetermined elements, wherein step 1) comprises providing an image illustrating or identifying the plurality of predetermined elements at positions corresponding to positions of the elements in the target area, and wherein step 2) comprises indicating a position of one of the predetermined elements (or indicating the element in the image).

The above embodiment using the providing of a number for each position may equally well be used here.

These positions in the image then correspond to physical elements or demarcations in the target area. In this situation, the plurality of positions in the image and any corresponding physical means/elements of the target area may be sufficient for a user to be able to indicate a desired position, whereby the requirements to the image may be reduced. The mere structure and inter-relation between the individual elements and positions may be sufficient for the user to identify the desired element/position. Thus, the image may be a more simple or schematic and only (or at least) represent selected elements or all the elements corresponding to the positions in the image.

Also, this means that the same set-up may be used in order to provide a plurality of predetermined directions and to launch multiple projectiles and determine multiple deviations.

In the above embodiment, a plurality of elements is visible in the image and in the target area. Naturally, a single such element may also be used.

In this situation, the element at the position corresponding to the indicated position may have a visible part having a predetermined size. Then, step 2) could comprise:

• • a) determining, from the image, an angle to the indicated element and
  • b) determining, from the image, a distance from a predetermined position to the indicated element by correlating an extent of the visible part in the image with the predetermined size thereof, the corresponding position being determined on the basis of the angle and the distance.

Knowing the angle and the distance will enable a determination of the position of the element from the image alone. The width of the predetermined size in the image provides the distance measure. The visible part may be e.g. a part at the end of a stick or pole fixed to the ground or may be a predetermined distance from the ground to the element.

The predetermined point may be a center of the image or a point fixed in relation to the target area.

In an interesting embodiment, step 3) comprises determining a landing point of the projectile and the actual direction of the projectile is determined from the launch position and the landing point. Determination of the landing point may be performed in a number of manners. The presently preferred manner is described in the applicants co-pending PCT/DK2005/000336, which is incorporated herein by reference.

A preferred manner of actually determining the path or direction of the projectile is one wherein step 3) comprises:

• • receiving radiation transmitted by or reflected by the projectile while it is in flight and determining, from the received radiation, a direction of the projectile.

This may be a RADAR system or any other system using radiation (or sound) for determining the position/path/direction of a projectile.

Naturally, the radiation from the projectile may be emitted from the projectile or may be reflected there from. In the latter situation, the set-up may further comprise a step of providing radiation toward the projectile in order to have the projectile reflect this for it to be determined.

In this embodiment, step 4) may comprise determining the launch position from the radiation received. A number of manners exist for this. One manner is to determine the actual point in time of launch using e.g. a microphone, where the launch position may then be determined from the direction or path of the projectile.

In that situation, the positioning or position determination in the RADAR (or similar) system is preferably fixed or has a predetermined relation to the image or image generation in order for the launch position (and optionally landing position) to be related to the position identified in the image. This relation may be provided by simply fixing the image generation (if provided e.g. as a camera) to the RADAR system (or at least the radiation receiver(s)).

In a second aspect, the invention relates to an apparatus for determining a deviation between an actual direction of a launched projectile and a predetermined direction, the apparatus comprising:

• • means for providing an image representing a target area,

means for receiving an indication of a position in the image, the receiving means being adapted to correlate the indicated position to a corresponding position in the target area,

means for determining the actual direction of the projectile,

means for determining a launch position of the projectile,

means for determining, from the launch position and the corresponding position, the predetermined direction, and

means for determining the deviation between the actual direction of the projectile and the predetermined direction.

Preferably, the providing means comprise a display or monitor and a processing unit, the display or monitor being operatively connected to the processing unit and being adapted to provide the image, the receiving means preferably also comprising a pointing device also connected to the processing unit. This pointing device may be a computer mouse, an optical pen, or the like. Alternatively, the display/monitor may be a touch pad, which then also comprises the pointing device in that touching the display/monitor at a position will enter that position into the system.

In general, the means for determining the actual direction of the projectile may comprise means for determining a landing point of the projectile and means for determining, from the launch position and the landing point, the actual direction of the projectile.

As mentioned above, the providing means may comprise a camera for providing the image of the target area. This camera may be a still image camera or a video camera, digital cameras are widespread, but other types of cameras are equally useful.

In one embodiment, the providing means are adapted to provide the image comprising a plurality of predetermined positions, and the receiving means are adapted to receive an indication of one of the predetermined positions.

In addition to that or in combination with that, the target area preferably comprises a plurality of predetermined elements, wherein the providing means are adapted to provide an image illustrating or identifying the plurality of predetermined elements at positions corresponding to positions of the elements in the target area, and the receiving means are then adapted to receive an indication of a position of one of the predetermined elements. Thus, the positions could correspond to physical “targets” in the target area.

In that situation, the element at the position correlated to the indicated position could have a visible part having a predetermined size, and the receiving means could then comprise means for:

• • a) determining, from the indicated position in the image, an angle to the indicated element and

b) determining, from the image, a distance from a predetermined position to the indicated element by correlating an extent of the visible part in the image with the predetermined size thereof, the receiving means being adapted to determine the corresponding position on the basis of the angle and the distance.

The visible part may be a part at the end of a stick and may comprise a visible pattern, from which the boundaries of the visible part or from which dimensions of the visible part may be determined.

As mentioned above, the means for determining the actual direction of the projectile may comprise means for:

• • receiving radiation transmitted by or reflected by the projectile while it is in flight and
  • determining, from the received radiation, a direction of the projectile.

Thus, the means for determining the actual direction may be a RADAR system or another system adapted to determine a position/path/direction from received radiation (or sound).

The radiation may be emitted/transmitted by the projectile or it may be reflected. In the latter situation, the apparatus may further comprise means for providing radiation toward the projectile.

In this embodiment, the means for determining the launch position of the projectile may be adapted to determine the launch position from the radiation received.

In another embodiment, the means for determining the launch position of the projectile are adapted to assume that the launch is performed at a predetermined position. This is a simple manner of “obtaining” that information.

In the embodiment where the apparatus comprises the camera, it is desired that the determination of the launch position/landing position and/or the path/direction/position of the projectile are correlated to each other. This may be obtained when the camera and the radiation receiving means are fixed in relation to each other so that the radiation receiving means are adapted to receive radiation from the target area viewed by the camera.

In fact, the radiation receiving means may be directed toward the target area so that the fields of view of the radiation receiving means and the camera are either the same or one is comprised within the other—at least for the expected path or direction of the projectile.

A third aspect of the invention relates to an element for use in the above method or for use with the above apparatus, the element comprising a visible part having a predetermined size.

This visible part may be an element having any suitable shape, such as round, circular, square, which is visible from a distance. The element may have visible demarcations, such as boundaries, in order for the distance to the element to be determinable from the extent of these demarcations at that distance.

The visible part may be positioned a distance, such as a predetermined distance, from the ground and may be provided at (such as at the end of) a stick, a pole, or the like.

The visible part may be solid or may be a flag on which the demarcations are provided. The flag may then be kept in a stretched-out manner in order for the demarcations to be visible also in calm weather.

Additional aspects and/or advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects and advantages of the invention will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

FIG. 1 is a block diagram of the main components in the present invention.

FIG. 2 shows the positions of the different coordinate systems in a top view.

FIG. 3 is the process flow for selecting a target and getting ball flight data in the desired coordinate system.

FIG. 4 shows the image of a camera looking toward the target area from behind the launch position.

FIG. 5 shows an example of a physical layout of a target that can ease target identification.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Reference will now be made in detail to the present embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. The embodiments are described below in order to explain the present invention by referring to the figures.

In the preferred embodiment of the present invention the projectile is a golf ball being launched towards a target, the image capturing device is a camera 1 and the measurement equipment to determine the actual projectile path is a radar 2, as shown in FIG. 1. Both the image and the radar data are sent to a computer 3, which displays the image and radar data on a display 4, to ease the identification of a target in an image, a touch screen 5 is used.

Independent of the position and physical orientation of the radar, the golf ball trajectory data, and specifically the landing point, is desired in a coordinate system which has origo at the launch point 12 and with a horizontal reference line 11 that goes through the target 10. The hereby defined coordinate system is called the golfer coordinate system 15, as seen in FIG. 2.

The trajectory data is measured in the radar coordinate system 16. The image captured by the camera 8 is defined in the camera coordinate system 17.

Every golf shot will typically be launched from slightly different positions, and also the target will often change during a golf session. This means that the position and orientation of the golfer coordinate system 15 is in general different for every golf shot. This illustrates the need for an efficient way to automatically transfer measurement data from the radar coordinate system 16 into the golfer coordinate system 15.

Previous technologies define the target by mechanically adjusting the measurement system, e.g. the radar 2, to point directly at the target or in a predetermined offset from the target. In some technologies several predetermined targets positions are stored, to ease the selection between these predetermined targets. Anyhow, this means that whenever a new target, the position of which is not known a priori is selected, a new mechanical adjustment would be required.

The present embodiment of the invention does not require any mechanical adjustment as long as the desired target is in the field of view of the camera 1 and the radar 2. This field of view is typically between 10 to 30 degrees.

Previous techniques obtain the launch position by typically dictating the launch position to be within a predetermined very limited area, e.g. within a 20 cm diameter circle 2-4 m in front of the radar 2.

In the present embodiment of the invention, the launch position is directly measured by the radar 2, as being the first data point belonging to the golf ball flight. Other methods are outlined below.

The physical orientation and position of the camera 1 relative to the radar 2 must be known. This is most easily obtained by mounting the camera directly on or inside the radar, so the two units will be fixed relative to each other. The relative orientation and position is determined through a calibration procedure. When operating the system, the radar and camera units are assumed to be adjusted to be in level.

The procedure to select a new target in the present invention is outlined in FIG. 3. First a picture is taken in step 19, which is typically initiated by the user. The captured image is displayed and the user determines the position of the preferred target in the picture 21, if more targets are present. If the target is not in the field of view, it may be necessary to perform a rough alignment 27 of the camera and radar unit.

The image 28 presented to the user could look like that illustrated in FIG. 4. The horizontal line 29 in the image 28 represents the horizontal plane of the coordinate system 17, and the vertical line 30 represents the vertical plane at the reference horizontal angle in the same coordinate system 17. The center 31 of the camera is consequently reference line 9.

In the image, several targets 34, 35 will typically be displayed. To point out a specific target, the vertical and horizontal reference lines 32, 33 are adjusted to pin point the exact location of the desired target 33. This adjustment can be done manually by the user by means of a keyboard, touch panel, mouse or similar. In some cases, see below, the system can also automatically detect and center the reference lines 32, 33 to point at the desired target.

If the target has a predetermined shape or pattern, like in FIG. 5, the system can automatically center the vertical and horizontal reference lines 32, 33. If several targets are present in the target area, the targets might preferably be of different shapes, so that the system can distinguish between them. The user will in this case either indicate the target number to the system, or point out the approximate location of the target in the image.

In the above several methods for determining the angles to the target are outlined. To calculate the target position 10, the distance to the target is also required. There are several ways to determine this distance.

One method of determining the distance to the target is to simply have the user enter the distance, which of course requires that the user knows this.

Another method of determining the distance to the target is to measure the angular extent of a predetermined pattern of the target. Knowing the size of this predetermined pattern, it is straight forward to calculate the distance to the target. One predetermined target pattern 38, that can be used for this, is shown in FIG. 5, the angular dimension DH and/or DV can easily be determined from the image 28 by edge-detection techniques.

Once the angles and the distance to target have been determined, the target position can be calculated 22.

For each ball hit, the radar measures the ball flight 13. This includes measurements of the ball launch position 12 and the ball landing point 14.

An alternative way of measuring the ball launch position is by analyzing a sequence of images 28 taken before and after launch. In this way the launched ball 36 can be identified in the image 28. Hereby the vertical and horizontal angle to the launch position from the camera 8 can be determined. The distance from the camera to the launch position can be determined in at least two ways.

The first method measures the angular extent of the ball in the image 28 and compares this with the predetermined size of the ball. For most sports balls, this size is very well defined.

The second method assumes that the radar 2 is at a given height above the launch position. Then, by knowing the vertical angle to the launch position from above, the distance can be calculated directly using the simple trigonometric relation between the parameters.

By knowing the launch position 12 and the target position 10, the golfer coordinate system 15 is determined.

The radar will measure the ball flight 13 in the radar coordinate system 16. These data are then coordinate transferred 25 in to the golfer coordinate system 15 through a 3 dimensional translation and rotation.

Finally the ball flight data are presented to the user 26.

Although a few embodiments of the present invention have been shown and described, it would be appreciated by those skilled in the art that changes may be made in this embodiment without departing from the principles and spirit of the invention, the scope of which is defined in the claims and their equivalents.

Claims

1. A system for relating radar data to an image to track the flight of a projectile comprising:

a radar providing radar data of at least a portion of a trajectory of a projectile;

an imager providing an image of an area into which the projectile is to be launched, the image including a feature in the area;

a controller receiving the image from the imager and the radar data from the radar, the controller generating trajectory data of the projectile relative to the feature by correlating the radar data with the image; and

a display displaying the trajectory data.

2. The system according to claim 1, wherein the controller receives an identification of a target within the image, the controller identifying a target line connecting the target and a launch position of the projectile.

3. The system according to claim 2, wherein the controller alters the image to display the trajectory data within the image.

4. The system according to claim 3, wherein the trajectory data is displayed as numerical data.

5. The system according to claim 2, wherein the controller determines a deviation value between the portion of the trajectory and the target line, and wherein the display displays the deviation value.

6. The system according to claim 2, wherein the target is identified by input received by the controller from a user.

7. The system according to claim 6, wherein the user input is generated by interaction with the image.

8. The system according to claim 2, wherein the controller determines a distance from the radar to the target.

9. The system according to claim 8, wherein the controller determines the distance from the radar to the target based on a comparison of a known size of a feature in the image with a size in the image of the feature.

10. The system according to claim 9, wherein the controller determines a launch direction of the projectile indicating an initial direction of the projectile as it leaves the launch position.

11. The system according to claim 2, wherein the controller receives data corresponding to a distance from the imager to the target and determines based on the distance from the imager to the target and the data from the radar a deviation of the projectile between a flight path of the projectile and the target line.

12. The system according to claim 2, wherein the controller identifies a coordinate system based on a horizontal axis extending through the launch position and the target.

13. The system according to claim 2, wherein the controller identifies the launch position as a first data point of the projectile in motion.

14. The system according to claim 1, wherein the controller identifies a landing position of the projectile.

15. The system according to claim 1, wherein the controller displays the trajectory data while the projectile is in flight.

16. The system of claim 1, wherein the projectile is a sports ball.

17. A device for coordinating radar data and image data to determine the trajectory of a projectile, comprising:

a controller receiving an image, from an imager, of an area into which a projectile is to be launched, the image including a feature in the area, and radar data from a radar relative of at least a portion of a trajectory of the projectile, the controller generating trajectory data of the projectile relative to the feature by correlating the radar data with the image.

18. The device of claim 17, wherein the controller identifies a target within the image, the controller identifying a target line connecting the target and a launch position of the projectile.

19. The device of claim 18, wherein the controller alters the image to include the trajectory data within the image and to display the target line within the image.

20. The device of claim 18, wherein controller receives input identifying the target and identifies the target as an intersection of vertical and horizontal reference lines.

21. The device of claim 17, wherein the radar is positioned so that the portion of the trajectory is within a field of view of the radar and wherein the controller receives the image from an imager having a field of view including the target and the portion of the trajectory.

22. A method for coordinating radar data and image data to track a flight of a projectile comprising the steps of:

receiving by a controller an image of an area into which a projectile is to be launched, the image including a feature in the area, and radar data of at least a portion of a trajectory of a projectile traveling through a field of view represented by the image;

generating by the controller trajectory data of the projectile relative to the feature by correlating the radar data with the image; and

displaying the trajectory data.

23. The method according to claim 22, further comprising the step of identifying a target within the image and a launch point of the projectile and determining a deviation of the projectile from a path connecting the launch point to the target.

24. The method according to claim 23, wherein a distance from the launch position to the target is input by a user.

25. The method according to claim 23, wherein a distance to the target is determined by the controller based on a comparison of an angular extent of an element within the image and a known size of the element.

26. The method according to claim 22, wherein the controller correlates the radar data and the image based on data corresponding to a position of the imager relative to the radar.