Inter-transcriber Agreement (ITA)

The Inter-transcriber Agreement (ITA) analysis compares two blind transcriptions on the IPA Actual tier against a shared IPA Target and reports per-word agreement together with Cohen's Kappa. Each row of the result tables corresponds to a single word.

The analysis is intended for reliability studies in which two transcribers independently transcribe the same recordings using Phon's blind transcription mode. ITA reports how often the two transcribers produced the same phones, both for all phones and split by consonants and vowels.

Data Preparation

Each record must contain at least two blind transcriptions on the IPA Actual tier sharing a common IPA Target. Blind transcriptions are produced using Phon's blind transcription mode, where each transcriber records their own IPA Actual transcription without seeing the others.

Proper phone alignment between the shared IPA Target and each transcriber's IPA Actual is required for accurate results. Words from records that do not meet the two-transcriber requirement, or whose alignment cannot be resolved, are listed in the Skipped table and excluded from scoring.

Running the Analysis

Select one or more sessions containing blind transcription data in the Project Manager, then choose the Inter-transcriber Agreement (ITA) analysis from the analysis menu. Confirm the participant and session selection and run the analysis to generate the report.

Calculations

Agreement is scored per word against the shared IPA Target. For each target phone the two transcribers either agree or disagree, and additional counts track epenthesis (inserted) phones and a two-by-two contingency of each transcriber against the target.

Glides (feature glide) are excluded from both per-class tables; they appear only in the all-phones table. At an epenthesis position, the inserted phone's class on each side determines which per-class table it lands in. When both transcribers inserted at the same column with different classes (one consonant, one vowel), T1's phone class is used.

Per-word Columns

Each scored word row reports the following columns:

  • T1, T2 — the two blind transcriptions of the word being compared.
  • A / D — audible-phone positions where the transcribers agreed / disagreed.
  • EA / ED — epenthesis positions where both added the same phone (EA) or only one added a phone (ED).
  • AA, DD, AD, DA — two-by-two contingency counts versus the target: both agreed, both disagreed, only T1 agreed, only T2 agreed.
  • ITA — (A + EA) / (A + D + EA + ED).
  • κ — Cohen's Kappa computed from the contingency counts.

A Total row is appended to each per-class table with grand totals pooled across all scored word rows, with ITA and κ recomputed from the pooled counts.

Report Outline

Five tables are produced. A sample table of contents is displayed below. Bold level elements are section headers while italic items are tables.

  • Inter-transcriber Agreement (ITA)

    • Readme

    • Summary — pooled totals across the three per-class tables (one row each for All Phones, Consonants, Vowels).

    • All Phones — per-word agreement scored over every audible target phone, with a pooled Total row.

    • Consonants — per-word agreement scored only over target phones with the consonant feature.

    • Vowels — per-word agreement scored only over target phones with the vowel feature.

    • Skipped — words excluded from scoring because the record had fewer than two blind transcriptions or the word could not be aligned.