EBU Logo EBU Logo
0246B - Text variation from audio
published
0246B v1.0
Item image
TEXT VARIATION FROM AUDIO
Audio Track, HoH Caption
Hard of Hearing Captions
Caption blocks match speech
Location of incorrect captions
https://qc.ebu.io/items/0246B/versions/1-0-0/
Check
Report
Summary
EBU QC Id
0246B
Name
Text variation from audio
Status
published
Category
subjective
Lead editor
Eyal Lavi
Aliases
inaccurate subtitles inaccurate captions difference from speech
Tags
audio access service
Versions
Version Status Edition date Saved by Note
Details
Description
This Test reports instances of textual mismatch between audio and the temporally corresponding HoH captions. HoH captions may be edited or contain additional information text (such as action labels). The variation tolerance input variable defines a threshold for acceptable variation from the audio. Variation is measured using the Word Error Rate formula, applied to the duration of the audio and expressed as a decimal (not to be be confused with WER expressed as percentage for the whole file). The mismatched HoH captions tolerance input variable specifies a percentage of HoH caption blocks that exceeded the tolerated variation. Note that this Test does not check for missing HOH captions. It should be used alongside #0248B (Latest version).
Definition
Used in Checks, the system shall detect Hard of Hearing (HoH) captions where the text deviates from the temporally corresponding dialogue by more than the defined Text Variation tolerance, and whether the ratio of such detected HoH caption blocks exceeds the tolerated ratio of mismatched HoH caption blocks.

Used in Reports, the system shall report the HoH captions where text deviates from the temporally corresponding dialogue by more than the defined Text Variation tolerance, with their time range, and the percent of mismatched HoH captions.
References

  • Word Error Rate

    Remark: WER = (S + D + I) / Nr where Nr is the total words in the reference audio, S is the number of substituted words in the subtitle, D is the number of deleted words in the subtitle, I is the number of words inserted in the subtitle.

Examples

  • Input:
    "Service Channel Allocation" = 4a 1,2
    "CaptionsReference" = "foo.xml"
    "TextVariationTolerance" = 1%
    "MismatchedCaptionsRatioTolerance" = 10%

    Output:
    "CheckResult" = false
    "IncorrectCaptionsSegment" = {
    From: 00:00:03:21 to 00:00:04:01, IncorrectCaptionsText="How you? I'm dine, thanks."
    From: 00:00:07:00 to 00:00:08:20, IncorrectCaptionsText="We're gonna rock around the plot"
    }
    IncorrectCaptionsRatio = 14%

    Remark: Used as a Check. This test fails because more than 10% of the HoH captions exceed a Word Error Rate of 1%.

  • Input:
    "Service Channel Allocation" = 4a 1,2
    "CaptionsReference" = "foo.xml"
    "TextVariationTolerance" = 1%

    Output:
    "IncorrectCaptionsSegment" {
    From: 00:00:03:21 to 00:00:04:01, IncorrectCaptionsText="How you? I'm dine, thanks."
    From: 00:00:07:00 to 00:00:08:20, IncorrectCaptionsText="We're gonna rock around the plot"
    }
    "IncorrectCaptionsRatio" = 14%

    Remark: Used as a Report.

Input parameters
Name Default value Description Data model name Unit Locator
Output parameters
Name Alias Description Data model name Unit Locator
Test material
Title Version Download
Remarks
Remarks
User discussions
Title Author Last update