WER weighted = mean word-level Levenshtein distance per line.

CER weighted = mean char-level Levenshtein distance per line.

public_score = 1 - 0.5 * (WER_weighted / 12 + CER_weighted / 55)

from rapidfuzz.distance import Levenshtein 
def public_score(y_true, y_pred):     
  assert len(y_true) == len(y_pred), "Inputs must have the same number of lines" 

  wer_weighted = sum(Levenshtein.distance(ref.split(), pred.split()) for ref, pred in zip(y_true, y_pred)) / len(y_true) 
     
  cer_weighted = sum(Levenshtein.distance(ref, pred) for ref, pred in zip(y_true, y_pred) ) / len(y_true)

  score = 1 - 0.5 * (wer_weighted / 12 + cer_weighted / 55)     
  
  return wer_weighted, cer_weighted, score

That's cool. If you look in the code of their Starter code, you will see a file: eval_metrics.py which have an import at the top

import os

import sys

import yaml

import pandas as pd

SCRIPT_DIR = os.path.dirname(__file__)

EVAL_DIR = os.path.abspath(os.path.join(SCRIPT_DIR, "..", "..", "evaluations"))

if EVAL_DIR not in sys.path:

sys.path.insert(0, EVAL_DIR)

from wer import compute_weighted_wer

from cer import compute_weighted_cer

....

It says go to the SCRIPT_DIR which is the current directory, go up twice and enter an evaluations folder. This evaluations folder likely contain the wer and cer modules being loaded above because an exhaustive search of the starter downloads yields nothing.

Can someone tell me where to find these evaluation files?