Audio Processing is among the most necessary utility domains of digital sign processing (DSP) and machine studying. Modeling acoustic environments is an important step in creating digital audio processing programs resembling: speech recognition, speech enhancement, acoustic echo cancellation, and so forth.
Acoustic environments are crammed with background noise that may have a number of sources. For instance, when sitting in a espresso store, strolling down the road, or driving your automotive, you hear sounds that may be thought-about as interference or background noise. Such interferences don’t essentially comply with the identical statistical mannequin, and therefore, a combination of fashions may be helpful in modeling them.
These statistical fashions will also be helpful in classifying acoustic environments into totally different classes, e.g., a quiet auditorium (class 1), or a barely noisier room with closed home windows (class 2), and a 3rd possibility with home windows open (class 3). In every case, the extent of background noise may be modeled utilizing a combination of noise sources, every occurring with a special likelihood and with a special acoustic degree.
One other utility of such fashions is within the simulation of acoustic noise in numerous environments primarily based on which DSP and machine studying options may be designed to unravel particular acoustic issues in sensible audio programs resembling interference cancellation, echo cancellation, speech recognition, speech enhancement, and so forth.
A easy statistical mannequin that may be helpful in such eventualities is the Gaussian Combination Mannequin (GMM) wherein every of the totally different noise sources is assumed to comply with a particular Gaussian distribution with a sure variance. All of the distributions may be assumed to have zero imply whereas nonetheless being sufficiently correct for this utility, as additionally proven on this article.
Every of the GMM distributions has its personal likelihood of contributing to the background noise. For instance, there may very well be a constant background noise that happens more often than not, whereas different sources may be intermittent, such because the noise coming via home windows, and so forth. All this needs to be thought-about in our statistical mannequin.
An instance of simulated GMM information over time (normalized to the sampling time) is proven within the determine under wherein there are two Gaussian noise sources, each of zero imply however with two totally different variances. On this instance, the decrease variance sign happens extra usually with 90% likelihood therefore the intermittent spikes within the generated information representing the sign with increased variance.
In different eventualities and relying on the appliance, it may very well be the opposite means round wherein the excessive variance noise sign happens extra usually (as will likely be proven in a later instance on this article). Python code used to generate and analyze GMM information may even be proven later on this article.
Turning to a extra formal modelling language, let’s assume that the background noise sign that’s collected (utilizing a high-quality microphone for instance) is modeled as realizations of unbiased and identically distributed (iid) random variables that comply with a GMM as proven under.
The modeling downside thus boils all the way down to estimating the mannequin parameters (i.e., p1, σ²1, and σ²2) utilizing the noticed information (iid). On this article, we will likely be utilizing the technique of moments (MoM) estimator for such objective.
To simplify issues additional, we are able to assume that the noise variances (σ²1 and σ²2) are recognized and that solely the blending parameter (p1) is to be estimated. The MoM estimator can be utilized to estimate multiple parameter (i.e., p1, σ²1, and σ²2) as proven in Chapter 9 of the e-book: “Statistical Sign Processing: Estimation Idea”, by Steven Kay. Nevertheless, on this instance, we’ll assume that solely p1 is unknown and to be estimated.
Since each gaussians within the GMM are zero imply, we’ll begin with the second second and attempt to get hold of the unknown parameter p1 as a operate of the second second as follows.
Observe that one other easy technique to acquire the moments of a random variable (e.g., second second or increased) is by utilizing the second producing operate (MGF). A superb textbook in likelihood principle that covers such subjects, and extra is: “Introduction to Chance for Information Science”, by Stanley H. Chan.
Earlier than continuing any additional, we wish to quantify this estimator when it comes to the basic properties of estimators resembling bias, variance, consistency, and so forth. We are going to confirm this later numerically with a Python instance.
Beginning with the estimator bias, we are able to present that the above estimator of p1 is certainly unbiased as follows.
We will then proceed to derive the variance of our estimator as follows.
It is usually clear from the above evaluation that the estimator is constant since it’s unbiased and in addition its variance decreases when the pattern measurement (N) will increase. We may even use the above method of the p1 estimator variance in our Python numerical instance (proven intimately later on this article) when evaluating principle with sensible numerical outcomes.
Now let’s introduce some Python code and do some enjoyable stuff!
First, we generate our information that follows a GMM with zero means and normal deviations equal to 2 and 10, respectively, as proven within the code under. On this instance, the blending parameter p1 = 0.2, and the pattern measurement of the info equals 1000.
# Import the Python libraries that we are going to want on this GMM instance
import matplotlib.pyplot as plt
import numpy as np
from scipy import stats
# GMM information era
mu = 0 # each gaussians in GMM are zero imply
sigma_1 = 2 # std dev of the primary gaussian
sigma_2 = 10 # std dev of the second gaussian
norm_params = np.array([[mu, sigma_1],
[mu, sigma_2]])
sample_size = 1000
p1 = 0.2 # likelihood that the info level comes from first gaussian
mixing_prob = [p1, (1-p1)]
# A stream of indices from which to decide on the element
GMM_idx = np.random.selection(len(mixing_prob), measurement=sample_size, change=True,
p=mixing_prob)
# GMM_data is the GMM pattern information
GMM_data = np.fromiter((stats.norm.rvs(*(norm_params[i])) for i in GMM_idx),
dtype=np.float64)Then we plot the histogram of the generated information versus the likelihood density operate as proven under. The determine reveals the contribution of each Gaussian densities within the general GMM, with every density scaled by its corresponding issue.
The Python code used to generate the above determine is proven under.
x1 = np.linspace(GMM_data.min(), GMM_data.max(), sample_size)
y1 = np.zeros_like(x1)
# GMM likelihood distribution
for (l, s), w in zip(norm_params, mixing_prob):
y1 += stats.norm.pdf(x1, loc=l, scale=s) * w
# Plot the GMM likelihood distribution versus the info histogram
fig1, ax = plt.subplots()
ax.hist(GMM_data, bins=50, density=True, label="GMM information histogram",
coloration = GRAY9)
ax.plot(x1, p1*stats.norm(loc=mu, scale=sigma_1).pdf(x1),
label="p1 × first PDF",coloration = GREEN1,linewidth=3.0)
ax.plot(x1, (1-p1)*stats.norm(loc=mu, scale=sigma_2).pdf(x1),
label="(1-p1) × second PDF",coloration = ORANGE1,linewidth=3.0)
ax.plot(x1, y1, label="GMM distribution (PDF)",coloration = BLUE2,linewidth=3.0)
ax.set_title("Information histogram vs. true distribution", fontsize=14, loc="left")
ax.set_xlabel('Information worth')
ax.set_ylabel('Chance')
ax.legend()
ax.grid()After that, we compute the estimate of the blending parameter p1 that we derived earlier utilizing MoM and which is proven right here once more under for reference.
The Python code used to compute the above equation utilizing our GMM pattern information is proven under.
# Estimate the blending parameter p1 from the pattern information utilizing MoM estimator
p1_hat = (sum(pow(x,2) for x in GMM_data) / len(GMM_data) - pow(sigma_2,2))
/(pow(sigma_1,2) - pow(sigma_2,2))As a way to correctly assess this estimator, we use Monte Carlo simulation by producing a number of realizations of the GMM information and estimate p1 for every realization as proven within the Python code under.
# Monte Carlo simulation of the MoM estimator
num_monte_carlo_iterations = 500
p1_est = np.zeros((num_monte_carlo_iterations,1))
sample_size = 1000
p1 = 0.2 # likelihood that the info level comes from first gaussian
mixing_prob = [p1, (1-p1)]
# A stream of indices from which to decide on the element
GMM_idx = np.random.selection(len(mixing_prob), measurement=sample_size, change=True,
p=mixing_prob)
for iteration in vary(num_monte_carlo_iterations):
sample_data = np.fromiter((stats.norm.rvs(*(norm_params[i])) for i in GMM_idx))
p1_est[iteration] = (sum(pow(x,2) for x in sample_data)/len(sample_data)
- pow(sigma_2,2))/(pow(sigma_1,2) - pow(sigma_2,2))Then, we test for the bias and variance of our estimator and evaluate to the theoretical outcomes that we derived earlier as proven under.
p1_est_mean = np.imply(p1_est)
p1_est_var = np.sum((p1_est-p1_est_mean)**2)/num_monte_carlo_iterations
p1_theoritical_var_num = 3*p1*pow(sigma_1,4) + 3*(1-p1)*pow(sigma_2,4)
- pow(p1*pow(sigma_1,2) + (1-p1)*pow(sigma_2,2),2)
p1_theoritical_var_den = sample_size*pow(sigma_1**2-sigma_2**2,2)
p1_theoritical_var = p1_theoritical_var_num/p1_theoritical_var_den
print('Pattern variance of MoM estimator of p1 = %.6f' % p1_est_var)
print('Theoretical variance of MoM estimator of p1 = %.6f' % p1_theoritical_var)
print('Imply of MoM estimator of p1 = %.6f' % p1_est_mean)
# Under are the outcomes of the above code
Pattern variance of MoM estimator of p1 = 0.001876
Theoretical variance of MoM estimator of p1 = 0.001897
Imply of MoM estimator of p1 = 0.205141We will observe from the above outcomes that the imply of the p1 estimate equals 0.2051 which may be very near the true parameter p1 = 0.2. This imply will get even nearer to the true parameter because the pattern measurement will increase. Thus, we now have numerically proven that the estimator is unbiased as confirmed by the theoretical outcomes accomplished earlier.
Furthermore, the pattern variance of the p1 estimator (0.001876) is sort of an identical to the theoretical variance (0.001897) which is gorgeous.
It’s all the time a contented second when principle matches follow!
All photos on this article, until in any other case famous, are by the writer.
