The automatic annotation and analysis of speech

Phonemes and words segmentation

Definition

Data preparation

Data preparation: example

An IPU of “Corpus of Interactional Data”
An IPU of “Corpus of Interactional Data”

Expected result

extract palign

Phonemes and words segmentation: research approach of SPPAS

  1. text normalization
  2. phonetization (grapheme to phoneme conversion)
  3. alignment (speech segmentation)

All three tasks are fully-automatic, but each annotation result can be manually checked.

Text Normalization

Text Normalization: SPPAS approach

Text Normalization main steps

  1. Split:
    • use whitespace or characters to split the utterance into separated strings
  2. Replace symbols by their written form:
    • based on a lexicon
      • ° is replaced by degrees (English), degrés (French), grados (Spanish), gradi (Italian), mức độ (Vietnamese), 度 (Chinese), du (Chinese pinyin and Taiwanese)
      • ² is replaced by square (English), carré (French), quadrados (Spanish), quadrato (Italian), bình phương (Vietnamese), 平方 (Chinese), ping fang (Chinese pinyin)

Text Normalization main steps (continued)

  1. Segment into words:
    • fixes a set of rules to segment strings including punctuation marks
    • based on a lexicon and rules
      • aujourd’hui, c’est-à-dire
      • porte-monnaie, cet homme-là, voulez-vous
      • poudre d’escampette, trompe-l’oeil, rock’n roll

Text Normalization main steps

  1. Stick, i.e. concatenate strings into words
    • based on a dictionary with an optimization criteria: the longest matching algorithm
      • English: once_upon_a_time, game_over
      • French: pomme_de_terre, au_fur_et_à_mesure, tel_que
      • Chinese: 登记簿
  2. Convert numbers to their written form
    • 123
      • cent-vingt-trois (French), one-hundred-twenty-three (English), ciento-veintitres (Spanish)
  3. Lower the text
  4. Remove punctuation

Text Normalization of speech transcription

Text Normalization of speech transcription: example

Text Normalization: current languages

The better lexicon, the better automatic text normalization.

Text Normalization: Adding a new language

  1. add lexicons
  2. add the num2letter module

Example:

 Roxana Fung, Brigitte Bigi (2015).
 Automatic word segmentation for spoken Cantonese.
 In Oriental COCOSDA and Conference on Asian Spoken Language Research and Evaluation,
 pp. 196–201.
 
 

Text Normalization: reference

 Brigitte Bigi (2014).
 A Multilingual Text Normalization Approach.
 Human Language Technologies Challenges for Computer Science and Linguistics.
 LNAI 8387, Springer, Heidelberg. ISBN: 978-3-319-14120-6. Pages 515-526.
 
tokenization paper

Phonetization

Converting from written texts into actual sounds, for any language, cause several problems that have their origins in the relative lack of correspondence between the spelling of the lexical items and their sound contents.

Phonetization: SPPAS research approach

Phonetization: dictionary

dict eng

Phonetization of normalized speech transcription

Phonetization: example

Phonetization: current languages

The better dictionary, the better automatic phonetization.

Phonetization: reference

 Brigitte Bigi (2016).
 A phonetization approach for the forced-alignment task in SPPAS.
 Human Language Technologies Challenges for Computer Science and Linguistics.
 LNAI 9561, Springer, Heidelberg. 
 
phonetization paper

Alignment

Alignment

Alignment: current languages

Alignment: results of French

UBPA of French on read and spontaneous speech
UBPA of French on read and spontaneous speech
Manual vs automatic durations of vowels on conversational speech
Manual vs automatic durations of vowels on conversational speech

Alignment: references

 Brigitte Bigi (2012).
 The SPPAS participation to the Forced-Alignment task of Evalita 2011.
 B. Magnini et al. (Eds.): EVALITA 2012, LNAI 7689, pp. 312-321. Springer, Heidelberg.
 Brigitte Bigi (2014).
 The SPPAS participation to Evalita 2014.
 In Proceedings of the First Italian Conference on Computational Linguistics CLiC-it 2014
 and the Fourth International Workshop EVALITA 2014, Pisa, Italy.
 Brigitte Bigi (2014).
 Automatic Speech Segmentation of French: Corpus Adaptation.
 In 2nd Asian Pacific Corpus Linguistics Conference, pp. 32, Hong Kong.
 

Speech segmentation: main reference

 Brigitte Bigi, Christine Meunier (2018).
 Automatic speech segmentation of spontaneous speech.
 Revista de Estudos da Linguagem.
 International Thematic Issue: Speech Segmentation.
 Editors: Tommaso Raso, Heliana Mello, Plinio Barbosa,
 e - ISSN 2237-2083
 

You reached the end of this tutorial!

Click here to go back to SPPAS tutorials web page

Syllables segmentation

Data preparation

CM extract palign

Expected result

CM extract salign

Syllabification: SPPAS approach

Syllabification: SPPAS approach (continued)

Syllabification: SPPAS approach (continued)

Syllabification: resources

Syllabification of French: reference

 Brigitte Bigi, Christine Meunier, Irina Nesterenko and Roxane Bertrand (2010).
 Automatic detection of syllable boundaries in spontaneous speech.
 Language Resource and Evaluation Conference, pages 3285-3292, La Valetta, Malte.
 
syllabification fra paper

Syllabification of Italian: reference

 Brigitte Bigi and Caterina Petrone (2014).
 A generic tool for the automatic syllabification of Italian.
 In Proceedings of the First Italian Conference on Computational Linguistics CLiC-it 2014 and
 of the Fourth International Workshop EVALITA 2014, pp. 73–77, Pisa, Italy.
 
syllabification ita paper

Syllabification of Polish: reference

 Brigitte Bigi and Katarzyna Klessa (2015).
 Automatic Syllabification of Polish.
 In 7th Language and Technology Conference: Human Language Technologies as a Challenge for
 Computer Science and Linguistics, pp. 262–266, Poznan, Poland.
 
syllabification pol paper

You reached the end of this tutorial!

Click here to go back to SPPAS tutorials web page

Repetitions detection

Repetitions

Extract of Corpus of Interactional Data
 AB: ils voulaient qu'on fasse un feu d'artifice en fait dans un voy- un foyer un foyer catho
 un foyer de bonnes soeurs
 CM: un feu d'artifice
 AB: ah ouais
 CM: dans un foyer de bonnes soeurs
 CM: @
 

Repetitions

Other-Repetitions: result

AB CM repetition
AB:
CM:

Repetitions: reference

 Brigitte Bigi, Roxane Bertrand, Mathilde Guardiola (2014).
 Automatic Detection of Other-Repetition Occurrences: Application to French Conversational Speech.
 In Proceedings of the Ninth International Conference on Language Resources and Evaluation,
 pp. 836-842, Reykjavik, Iceland.
 
repetitions paper

You reached the end of this tutorial!

Click here to go back to SPPAS tutorials web page