A method for selecting a perceptually optimal hrtf in a database according to morphological parameters. A first database includes the HRTFs of subjects M, a second database includes the morphological parameters of the subjects, and a third database corresponds to a perceptual classification of the HRTFs. The N most relevant morphological parameters are sorted by correlating the second and third databases. A multidimensional space is created, which optimizes the spatial separation between the HRTFs according to the classification thereof in the third database to obtain an optimized space. An optimized projection model MPO is calculated for correlating K optimal morphological parameters with the corresponding position of the hrtf filters in the optimized space. For any user whose hrtf is not included in the database, at least one hrtf can be selected from the database BD1 according to the parameters K of the user and the optimized projection model MPO.
|
1. A method for selecting a perceptually optimal head-related transfer function (hrtf) in a database according to morphological parameters, comprising the steps of:
sorting, among all of the morphological parameters from a second database, the N most relevant morphological parameters by correlating the second database and a third database, wherein a first database comprises the HRTFs of a plurality of subjects, a second database comprises the morphological parameters of the subjects from the first database, and a third database corresponds to a perceptual classification of the HRTFs from the first database with respect to a judgment by the subjects performed using a listening test corresponding to the different HRTFs from the first database;
generating a multidimensional space whose dimensions result from a combination of hrtf components;
modifying rules for combining components to maximize a correlation between a spatial separation between the HRTFs and the classification of the HRTFs in the third database to obtain an optimized multidimensional space;
calculating an optimized projection model for correlating K sorted morphological parameters extracted from the second database with a corresponding position of the HRTFs in the optimized multidimensional space, the K extracted parameters maximizing the correlation between the optimized multidimensional space and a space produced by the optimized projection model;
measuring the K morphological parameters for a user not having an hrtf in the first database;
applying the previously calculated optimized projection model to extracted morphological parameters to obtain the user's projected position in the optimized multidimensional space; and
selecting at least one hrtf in a vicinity of the user's projection position in the optimized multidimensional space.
2. The method of
3. The method of
4. The method of
presenting a sound signal on which each of the HRTFs from the first database is applied to each subject, including the hrtf of said each subject, the sound signal being a broadband white noise with a short duration obtained by a Hanning window; and
rendering the sound signal at point positions along both trajectories presented in a sequence:
a circle in a horizontal plane, with elevation=0 degrees, in 30 degrees increments, the trajectory starting at 0 degrees azimuth and 0 degrees elevation, a path being repeated one time;
an arc in a median plane, with azimuth=0 degrees, from an elevation of −45 degrees to a front, up to −45 degrees to the back, through an elevation of 90 degrees, in 15 degrees increments; and
the sound path starting to the front at elevation −45 degrees, and continuing to the elevation to the back and then returning along the same path to the starting position.
5. The method of
generating sub-databases by dividing morphological values from the second database by morphological values of each subject from the second database to normalize a morphological data;
associating each sub-database with the classification from the third database for a corresponding subject;
applying a support vector machine method to obtain the morphological parameters ranked from highest to lowest as a function of a separation quality of each hrtf parameter according to a categorization in the third database.
6. The method of
converting the HRTFs into Directional Transfer Functions (DTFs) that contain only the portion of the HRTFs that have a directional dependence;
smoothing the DTFs;
pre-processing the DTFs;
transforming a data dimensionality to reduce or increase a number of dimensions, depending on the data used, as a result of the preprocessing step; and
when the data dimensionality is reduced:
performing a principal component analysis on the processed DTFs to obtain a score matrix representing an original data projected onto new axes; and
generating a multidimensional space from each column of the score matrix, representing a dimension of the multidimensional space; or
where the data dimensionality is increased:
generating the multidimensional space using multidimensional scaling;
evaluating an optimization level by a significance level of the spatial separation between the classifications from the third database;
repeating the steps of generating and evaluating with at least one of the following: different preprocessing parameters or by limiting the number of dimensions in the generated multidimensional space; and
keeping the multidimensional space with the most optimal optimization level.
7. The method of
8. The method of
9. The method of
the significance level of the spatial separation between the classifications in the third database, the significance level evaluated using an ANOVA test; or
calculating a percentage of HRTFs ranked in a highest category among ten closest HRTFs in the multidimensional space and comparing the percentage with an overall percentage of HRTFs ranked in the highest category in the third database for each subject using a student test.
10. The method of
calculating a projection model by multiple linear regressions between the optimized multidimensional space and the ranked morphological parameters to determine a position in the optimized multidimensional space based on the ranked morphological parameters from the second database;
evaluating a quality level of the projection model;
reducing the ranked morphological parameters to first K ranked morphological parameters;
repeating the steps of calculating the projection model and evaluating the quality level for each K, where K=1 to N, and for each subject, and removing said each subject's data from the first database and the second database; and
keeping an optimum K for which the quality level is the highest.
11. The method of
|
|||||||||||||||||||||||||||||||||||||||
This application is a §371 application from PCT/FR2011/050840 filed Apr. 12, 2011, which claims priority from French Patent Application No. 10 52767 filed April 12, each of which is incorporated herein by reference in its entirety.
The invention relates to a method for selecting HRTF filters in a database according to morphological parameters. The invention notably aims to ensure reliability in the HRTFs selected for a particular user.
The invention has a particularly advantageous application in the domain of binaural synthesis applications, which refers to the generation of spatialized sound for both ears. The invention therefore is used, for example, for teleconferencing, hearing aids, assistive listening devices for the visually impaired, 3D audio/video games, mobile phones, mobile audio players, virtual reality audio, and augmented reality.
Humans have the ability to decode directional information from an incident sound with an acoustic transfer function. The head, the outer ears, and the body of a listener transform the spectral information from a sound in the space by means of what is called the Head-Related Transfer Function (HRTF), which allows us to perceive our acoustic environment based on the position, distance, etc. of sound sources and therefore to locate them.
HRTF filters consist of a pair of filters (left and right) that describe the filtering of a sound source at a given position by the body. It is commonly accepted that a set of about 200 positions is adequate for describing all of the directions in the space a person perceives. These HRTF filters essentially depend on the morphology of the ear (size, dimensions of the internal cavities, etc.) and other physical parameters of the person's body.
In the remainder of this document, the term “HRTF” represents the filters for all of the HRTF-type positions for a given subject.
Using the HRTFs in an audio application that are the closest possible to the listener's HRTF filters can achieve high-quality rendering. Several studies in the literature demonstrate the benefit of so-called individualized HRTFs (for example, see the Moller et al. article “Binaural technique: do we need individual recordings?” published in the Journal of the Audio Engineering Society: 44, 451-469), especially in terms of accuracy in location tests.
HRTF filters can be obtained by taking measurements with microphones in the listener's ear, or even by digital simulation. Despite the quality of these methods, they are still very tedious, very expensive, and inadaptable to consumer applications.
Moreover, a known method described in the document WO-01/54453, provides for selecting, within a database, the closest HRTFs to those of the user. However, unlike the invention, such a method that is effective in terms of statistics does not use the perceptual quality of the selection of HRTFs as a validation criterion and therefore does not select the best possible HRTFs.
The novelty of the invention therefore lies in the fact that a perceptual assessment criterion based on a perceptual listening test is used to create an optimized HRTF multidimensional space and to select the most relevant morphological parameters. The invention also allows a predictive model to be developed that establishes a perceptually relevant correlation between the space and the morphological parameters.
For any user, the invention will allow the most appropriate HRTF included in a database to be selected using only measurements of morphological parameters.
The selected HRTF filter is strongly correlated with the spatial perception (and not just a mathematical calculation), which provides for great comfort and sound quality.
The invention therefore relates to a method for selecting a perceptually optimal HRTF in a database according to morphological parameters using:
wherein the method further uses
and wherein the method comprises the following steps:
According to an embodiment, in order to perform the perceptual classification, the subject has at least two choices (good or bad) in his judgment on at least one listening criterion for a sound corresponding to an HRTF.
According to an embodiment, the listening criterion is selected, for example, from among the accuracy of the defined sound path, the overall spatial quality, the front rendering quality (for sound objects that are located in front), and the separation of front/rear sources (ability to identify whether a sound object is located in front of or behind the listener).
According to an embodiment, to develop the third database:
According to an embodiment, in order to make the correlation between the second and the third database to obtain the sorted morphological parameters,
According to an embodiment, in order to create the optimized multidimensional space,
According to an embodiment, a critical band smoothing of the DTFs is performed according to the limits of the frequency resolution of the auditory system.
According to an embodiment, the pre-processing is performed using one of the following methods: frequency filtering, delimiting frequency ranges, extracting frequency peaks and valleys, or calculating a frequency alignment factor.
According to an embodiment, the optimization level is evaluated:
According to an embodiment, in order to calculate a projection model for correlating the N morphological parameters extracted from the second database with the corresponding position of the HRTFs in the optimized space:
According to an embodiment, in order to select at least one HRTF in the vicinity of the user's projection position in the optimized multidimensional space, the HRTF that is closest to the projection position in the optimized multidimensional space is chosen.
The invention will be better understood upon reading the following description and studying the figures that accompany it. These figures are provided for illustrative purposes only and are not limiting to the invention. They show:
Identical, similar, or analogous elements maintain the same reference number from one figure to the next.
For a plurality of subjects microphones are positioned in the subject's ears, and sound sources are scattered over various points in the space in order to determine the HRTFs for each subject. The morphological parameters are also measured for each subject. A first database BD1 contains the HRTFs, and a second database BD2 contains the morphological parameters for the associated subjects.
In our example, the HRTFs stored in the first database BD1 come from the public database from the LISTEN project. The data from the first M subjects in this database are used (in one example M=45). The LISTEN HRTF measurements were taken at positions in the space that correspond to elevation angles ranging from −45 degrees to 90 degrees by 15 degrees increments and azimuth angles starting at 0 degrees by 15 degrees increments. The azimuth increments were gradually increased for the elevation angles over 45 degrees in order to evenly sample the space, for a total of 187 positions.
As shown in
These morphological parameters, which are stored in the second database BD2, correspond to the HRTFs of the subjects.
Moreover, in a step E1, a third database BD3 is created containing the perceptual evaluation results from the listening test. For each subject, a test signal on which HRTFs from the database BD1 are applied is emitted.
In one example, the sound signal used for the test is a broadband white noise with a short duration, such as 0.23 seconds, obtained by a Hanning window,
Each subject has classified each of the HRTFs into one of the following three categories: excellent, fair, or poor. Excellent is considered to be the highest judgment category. These judgments are based on at least one criterion for listening to a sound corresponding to an HRTF. The criterion may selected from one of the following examples: the accuracy of the previously defined path, the overall spatial quality, the front rendering quality (for sound object that are located in front), and the separation of front/rear sources (ability to identify whether a sound object is located in front of or behind the listener).
As shown in
For that purpose, in a sub-step E2.1, the morphological data is normalized by creating sub-databases BD2i (i ranging from 1 to M, which is the number of subjects in the databases) by dividing the morphological values from the second database BD2 by the morphological values of each subject in the second database BD2[i]. With this normalization, the values represent the percentage of one subject's morphological parameter relative to another's.
Each sub-database BD2i is associated in a sub-step E2.2 with the classification in the third database of the corresponding subject BD3[i].
Then, in a sub-step E2.3, a feature selection method is applied in order to obtain the morphological parameters ranked from highest to lowest Pmc. This classification is based on their ability to separate the HRTFs according to their classification in the third database BD3.
The chosen method is a support vector machine (SVM) method. This method is based on the construction of a set of hyper-planes in a high-dimension space in order to classify the normalized data. With this method, the parameters have therefore been ranked from highest to lowest.
Two variables control the classification with SVM. The complexity value C, which controls the classification error tolerance in the analysis, introduces a penalty function. A null value of C indicates that the penalty function is not being taken into account, and a high value of C (endlessly increasing C) indicates that the penalty function is dominant. The epsilon value ε is the insensitivity value that sets the penalty function to zero if the data to be classified is at a distance of less than ε from the hyper-plane. The classification of the morphological parameters changes according to the different values of C and ε. Using this method where C=1 and ε=1×10−25, the first ten highest elements of the Pmc, ranked from highest to lowest, in our example, are: x11, x2, x8, d5, x3, d4, x12, d2, d1, and x6.
In a step E3, a multidimensional space EM is created whose dimensions result from a combination of components from the HRTF filters.
For that purpose, in a first step E3.1, the HRTFs are converted into what are called Directional Transfer Functions (DTFs) that contain only the portion of the HRTFs that have a directional dependence.
In a step E3.2, a critical band smoothing of the DTFs is performed according to the limits of the frequency resolution of the auditory system.
In a step E3.3, the DTFs are preprocessed using a method selected from among the following: frequency filtering, delimiting frequency ranges, extracting frequency peaks and valleys, or calculating a frequency alignment factor.
In a step E3.4, the data dimensionality is transformed in order to reduce or increase the number of dimensions, depending on the data used, which is the result of the step E3.3.
To reduce the data dimensionality, a principal component analysis (PCA) is performed on the processed DTFs in order to obtain a new data matrix (the scores) that represent the original data projected onto new axes (the principal components), and a space EM is created from each column of the score matrix, representing a dimension of the space EM.
To increase the data dimensionality, a multidimensional scaling (MDS) analysis is used on the processed DTFs, resulting in the space EM.
In a step E3.5, the optimization level is evaluated. In a first example, the optimization level is evaluated by the significance level of the spatial separation between the classifications from the third database BD3. In one example, the significant level is evaluated using the ANOVA test to check whether the value distribution averages were statistically different for each different number of dimensions.
In a second example, the percentage of HRTFs ranked in the highest category among the ten closest HRTFs in the space EM is calculated and this percentage is compared, using the Student test for example, with the overall percentage of HRTFs ranked in the high category in the third database for each subject.
The previous steps are repeated with different preprocessing parameters and/or by limiting the number of dimensions in the created space.
The space with the most optimal optimization level is kept. This space is the one in our examples with the highest significance level or the one in the second example with the number of ranked HRTFs in the highest category for the closest ten HRTFs is maximized.
Such kept space is the optimized multidimensional space EMO.
The purpose of the step E3.5 is to optimize the spatial separation between the HRTFs according to their classification in the third database BD3 in order to obtain an optimized space. Indeed, in the space EMO, for a subject at a given position, the HRTFs located in the area near this position will be considered as good for the subject, while the HRTFs that are distant from this position will be considered as bad.
In other words, the rules for combining HRTF components are changed in order to maximize the correlation between the spatial separation between the HRTFs and the classification of HRTFs in the third database BD3.
In a step E4, a projection model is calculated for correlating the N morphological parameters extracted from the second database BD2 with the position of the corresponding HRTFs in the optimized space EMO.
For that purpose, in a step E4.1, a projection model is calculated by multiple linear regressions between EMO and Pmc using the second database BD2 for the purpose of finding a position in the space EMO based on the ranked morphological parameters Pmc.
In a step E4.2, the quality level of the projection model is evaluated. This quality level is calculated using the same methods as were used in E3.5.
In a step E4.3, Pmc is reduced to the first K ranked morphological parameters, and the calculations of the model are repeated from the step E4.1 and the step E4.2 of measure of the quality for each K from K equals 1 to K equals N. Preferably, this calculation is repeated for each subject by removing the data of the subject from the first database BD1 and from the second database BD2 in the step E3.
The optimum K for which the quality level is the highest is kept. Therefore, the K extracted parameters maximize the correlation between the optimized multidimensional space EMO and the space produced by the projection model.
This provides an optimized projection model MPO.
In a step E5, at least one HRTF is selected in the database BD1 for any user that does not have a HRTF in the database.
For this purpose, in a sub-step E5.1, the user measures the previously identified K morphological parameters. For this purpose, the user takes a photo of his ear in a determined position, the K parameters being extracted by an image processing method.
In a step E5.2, the K parameters are injected as input from the previously calculated projection model MPO into the extracted morphological parameters in order to obtain the user's position in the optimized space EMO.
At least one HRTF (marked HRTF-S) is then selected in the vicinity of the user's projection position in the optimized space. In one example, the HRTF that is closest to the projection position is chosen.
Katz, Brian, Schönstein, David
| Patent | Priority | Assignee | Title |
| 10129684, | May 22 2015 | Microsoft Technology Licensing, LLC | Systems and methods for audio creation and delivery |
| 10187740, | Sep 23 2016 | Apple Inc | Producing headphone driver signals in a digital audio signal processing binaural rendering environment |
| 10306396, | Apr 19 2017 | United States of America as represented by the Secretary of the Air Force | Collaborative personalization of head-related transfer function |
| 10555105, | Dec 01 2015 | Orange | Successive decompositions of audio filters |
| 10945080, | Nov 18 2016 | STAGES LLC; STAGES PCS, LLC | Audio analysis and processing system |
| 11582562, | Dec 03 2019 | Oticon A/S; OTICON A S | Hearing system comprising a personalized beamformer |
| 11601764, | Nov 18 2016 | STAGES LLC | Audio analysis and processing system |
| 11689846, | Dec 05 2014 | STAGES LLC | Active noise control and customized audio system |
| 9508335, | Dec 05 2014 | STAGES LLC | Active noise control and customized audio system |
| 9544706, | Mar 23 2015 | Amazon Technologies, Inc | Customized head-related transfer functions |
| 9609436, | May 22 2015 | Microsoft Technology Licensing, LLC | Systems and methods for audio creation and delivery |
| 9654868, | Dec 05 2014 | STAGES LLC | Multi-channel multi-domain source identification and tracking |
| 9747367, | Dec 05 2014 | STAGES LLC | Communication system for establishing and providing preferred audio |
| 9774970, | Dec 05 2014 | STAGES LLC | Multi-channel multi-domain source identification and tracking |
| 9980042, | Nov 18 2016 | STAGES LLC; STAGES PCS, LLC | Beamformer direction of arrival and orientation analysis system |
| 9980075, | Nov 18 2016 | STAGES LLC; STAGES PCS, LLC | Audio source spatialization relative to orientation sensor and output |
| Patent | Priority | Assignee | Title |
| 5742689, | Jan 04 1996 | TUCKER, TIMOTHY J ; AMSOUTH BANK | Method and device for processing a multichannel signal for use with a headphone |
| 6996244, | Aug 06 1998 | Interval Licensing LLC | Estimation of head-related transfer functions for spatial sound representative |
| 7840019, | Aug 06 1998 | Interval Licensing LLC | Estimation of head-related transfer functions for spatial sound representation |
| 7921016, | Aug 03 2007 | Foxconn Technology Co., Ltd. | Method and device for providing 3D audio work |
| 8489371, | Feb 29 2008 | France Telecom | Method and device for determining transfer functions of the HRTF type |
| 20080137870, | |||
| 20080306720, | |||
| 20090034772, | |||
| WO154453, | |||
| WO2007048900, |
| Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
| Apr 12 2011 | ARKAMYS | (assignment on the face of the patent) | / | |||
| Apr 12 2011 | Centre National de la Recherche Scientifique | (assignment on the face of the patent) | / | |||
| Oct 11 2012 | SCHONSTEIN, DAVID | ARKAMYS | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 029202 | /0315 | |
| Oct 11 2012 | SCHONSTEIN, DAVID | Centre National de la Recherche Scientifique | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 029202 | /0315 | |
| Oct 12 2012 | KATZ, BRIAN | ARKAMYS | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 029202 | /0315 | |
| Oct 12 2012 | KATZ, BRIAN | Centre National de la Recherche Scientifique | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 029202 | /0315 |
| Date | Maintenance Fee Events |
| Jan 02 2018 | M2551: Payment of Maintenance Fee, 4th Yr, Small Entity. |
| Jan 04 2022 | M2552: Payment of Maintenance Fee, 8th Yr, Small Entity. |
| Date | Maintenance Schedule |
| Jul 01 2017 | 4 years fee payment window open |
| Jan 01 2018 | 6 months grace period start (w surcharge) |
| Jul 01 2018 | patent expiry (for year 4) |
| Jul 01 2020 | 2 years to revive unintentionally abandoned end. (for year 4) |
| Jul 01 2021 | 8 years fee payment window open |
| Jan 01 2022 | 6 months grace period start (w surcharge) |
| Jul 01 2022 | patent expiry (for year 8) |
| Jul 01 2024 | 2 years to revive unintentionally abandoned end. (for year 8) |
| Jul 01 2025 | 12 years fee payment window open |
| Jan 01 2026 | 6 months grace period start (w surcharge) |
| Jul 01 2026 | patent expiry (for year 12) |
| Jul 01 2028 | 2 years to revive unintentionally abandoned end. (for year 12) |