Evaluation#

class src.evaluation.EvaluationManager.EvaluationManager(dataset_manager: DatasetManager, experiment_files_dir: str, experiment: Experiment, use_representation_dir: bool = False, nb_digits: int = 5)#

Bases: object

Class that handles enhanced speech evaluation.

compute_composite_objective_eval_metrics(preds_dir: str, target_dir: str, preds_data_type: str, filename_prefix: str = '') → None#

Method computes composite metrics: CSIG, CBAK, COVL and segSNR.

Parameters:

  • preds_dir (str) -- Noisy/Clean/Enhanced audio data directory.

  • target_dir (str) -- Clean audio data directory.

  • preds_data_type (str) -- 'noisy' or 'enhanced' audio.

  • filename_prefix (bool) -- Enhanced audio data filename prefix.

compute_dnsmos_objective_eval_metrics(speech_dir: str, data_type: str, personalized_mos: bool = False) → None#

Method that computes DNSMOS scores: SIG, BAK, OVRL.

Parameters:

  • speech_dir (str) -- Noisy/Clean/Enhanced audio data directory.

  • data_type (str) -- 'noisy', 'clean' or 'enhanced' audio.

  • personalized_mos (bool) -- Boolean that indicate weather personalized MOS score is needed or regular.

compute_dnsmos_score(testset_dir: str, data_type: str, csv_path: str, personalized_MOS: bool, SAMPLING_RATE: int = 16000) → None#

Method that computes DNSMOS scores: SIG, BAK, OVRL.

Parameters:

  • testset_dir (str) -- Noisy/Clean/Enhanced audio data directory.

  • data_type (str) -- 'noisy', 'clean' or 'enhanced' audio.

  • csv_path (bool) -- Output csv file saving directory.

  • personalized_mos (bool) -- Boolean that indicate weather personalized MOS score is needed or regular.

  • SAMPLING_RATE (bool) -- Audio data sampling rate.

compute_evaluation_metrics() → None#

Method computes evaluation metrics.

compute_mean_dnsmos(rows: list, data_type: str) → None#

Method that computes DNSMOS scores: SIG, BAK, OVRL.

Parameters:

  • rows (list) -- List of audio data.

  • data_type (str) -- 'noisy', 'clean' or 'enhanced' audio.

compute_objective_eval_metrics(preds_dir: str, target_dir: str, preds_data_type: str, filename_prefix: str = '') → None#

Method computes metrics: PESQ, STOI, ESTOI, SI-SNR.

Parameters:

  • preds_dir (str) -- Noisy/Enhanced audio data directory.

  • target_dir (str) -- Clean audio data directory.

  • preds_data_type (str) -- 'noisy' or 'enhanced' audio.

  • filename_prefix (bool) -- Enhanced audio data filename prefix.

static pesq_fn(preds: Tensor, target: Tensor, fs: int = 16000, mode: str = 'wb')#

Method that computes PESQ score.

Parameters:

  • preds (torch.Tensor) -- 'noisy', or 'enhanced' audio.

  • target (torch.Tensor) -- 'clean' audio.

  • fs (int) -- Audio sampling frequency.

  • mode (str) -- 'wb' or 'nb'.

static si_sdr_fn(preds: Tensor, target: Tensor)#

Method that computes SI-SDR score.

Parameters:

  • preds (torch.Tensor) -- 'noisy', or 'enhanced' audio.

  • target (torch.Tensor) -- 'clean' audio.

static si_snr_fn(preds: Tensor, target: Tensor, epsilon: float = 1e-08)#

Method that computes SI-SNR score.

Parameters:

  • preds (torch.Tensor) -- 'noisy', or 'enhanced' audio.

  • target (torch.Tensor) -- 'clean' audio.

  • epsilon (float) -- A small value to avoid computation error.

static stoi_fn(preds: Tensor, target: Tensor, fs: int = 16000, extended: bool = False)#

Method that computes STOI/ESTOI score.

Parameters:

  • preds (torch.Tensor) -- 'noisy', or 'enhanced' audio.

  • target (torch.Tensor) -- 'clean' audio.

  • fs (int) -- Audio sampling frequency.

  • extended (str) -- Boolean that indicates weather to use STOI or ESTOI function.