Introduction

About the author

SPPAS is a scientific computer software package written and maintained by Brigitte Bigi
CNRS researcher at the Laboratoire Parole et Langage, in Aix-en-Provence, France

Research topics are related to:
1. Multimodal corpora: collect, annotate, analyze
2. Multilinguality

Corpus and annotation

Corpus linguistics is the study of language as expressed in samples (corpora) of “real world”.
Corpus annotation is a path to greater linguistic understanding.

Corpus annotation “can be defined as the practice of adding interpretative, linguistic information to an electronic corpus of spoken and/or written language data. ‘Annotation’ can also refer to the end-product of this process” (Leech, 1997).

Annotations

Annotation is not an end in itself: it is a basis for further analyses
Annotations should be time-synchronized. Temporal information makes it possible to describe simultaneous behaviors:
- of different levels in an utterance (e.g., prosody and locution)
- of different modalities (e.g. speech and gesture)
- of different speakers or extralinguistic events

Annotation software

Manual annotation
- Audacity, Praat, Elan, Anvil, Winpitch, AnnotationPro…
Automatic annotation:
- The current state-of-the-art in Computational Linguistics allows many annotation tasks to be semi- or fully-automated:
  - MarsaTag (morpho-syntax for French)
  - MediaPipe
  - …
Each annotation that can be done automatically must be done automatically!
Because revising is faster and easier than annotating… if the automatic system is “good enough”.

Before using any automatic annotation tool/software, it is important to consider its error rate (where applicable) and to estimate how those errors will affect the purpose for the annotated corpora.

SPPAS is an awarded Research Software

Research global approaches:
- methods as language-independent as possible;
- the possibility to adapt technologies to low-resourced languages.
Award “Accessit” Special Jury Prize - 2022:
- The Ministry of Higher Education, Research and Innovation presented the 1st Open Science Awards for Open Source Research Software.

Multi-Lingual approaches to the automatic annotation of speech

SPPAS is designed and developed to handle multiple language corpora and/or tasks with the same algorithms in the same software environment.
SPPAS emphasizes new practices in the methodology of tool developments:
- considering problems with a generic multilingual aspect,
- sharing resources,
- putting the end-users in control of their own computing.
Only the resources are language-specific, and the approach is based on the simplest resources possible.

Resources extend

Phoneticians are of crucial importance for resource development
- they can contribute to improve the resources used by automatic systems.
New versions are systematically released to the public and serve to benefit of the whole community.

Resources extend (continued)

Resources are distributed under the terms of a public license, so that SPPAS users:
- have free access to the application source code and the resources of the software they use,
- are free to share the software and resources with other people,
- are free to modify the software and resources,
- are free to publish their modified versions of the software and resources.

SPPAS: Main reference to cite

		Brigitte Bigi (2015).
		SPPAS - Multi-lingual Approaches to the Automatic Annotation of Speech.
		In "the Phonetician" - International Society of Phonetic Sciences,
		ISSN 0741-6164, Number 111-112 / 2015-I-II, pages 54-69.

You reached the end of this tutorial!

Click here to go back to SPPAS tutorials web page

Corpus creation methodology

The context

SPPAS was created in the scope of annotating automatically and exploring large multi-modal corpora
A workflow was defined while creating CID - Corpus of Interactional Data
It was then applied to create new corpora

Screenshots of 4 corpora (left to right): CID, GrenelleII, Aix MapTask, DVD — Screenshots of four corpora (left to right): CID, GrenelleII, Aix MapTask, DVD

The corpus creation workflow

Corpus creation and annotation: methodology

Step 1: Recording speech

One channel per speaker
Anechoic room, or an environment with no/low noise
Audio: for automatic annotation tools:
- Any uncompressed file format, commonly WAVE
- 16000Hz / 16bits is enough
Audio: for manual annotation tools:
- Any un-compressed file format
- 48000Hz / 16bits is of high quality

Step 2: Search for Inter-Pausal Units

Parameters to define manually:
- fix the minimum silence duration
- fix the minimum speech duration
As results:
- speech and silences are time-aligned automatically
- a manual verification is highly recommended

Step 3: Orthographic Transcription

Any transcription must follow a convention
In speech (particularly in spontaneous speech), many phonetic variations occur:
- some of these phonologically known variants are predictable
- but many others are still unpredictable (especially invented words, regional words or words borrowed from another language)
The orthographic transcription must be enriched:
- it must be a representation of what is “perceived” in the signal.

Step 4: Speech segmentation

In SPPAS, this problem is divided into three different tasks:
1. Text normalization
2. Phonetization
3. Alignment
Allows a fully automatic annotation, or a semi-automatic one
See the tutorial ‘Phonemes and words segmentation’ for details

Other steps:

On the basis of the time-aligned phonemes/tokens, other automatic annotations:
- Syllables
- Repetitions
- TGA
- …
Analysis:
- Filter data
- Statistics
- …

Interested in knowing more?

You can see a full presentation here if you are interested in knowing more. It was created in 2015 and wasn’t updated since! However, it is general enough and can be of great help:

You reached the end of this tutorial!

Click here to go back to SPPAS tutorials web page

Data preparation for automatic annotations

(Step 1) Recording Speech

Recording audio

The resolution of the capture devices (microphones, framerate, file format, software) has a determining influence on the quality of the corpus, and so on the annotations.
Lack of standardization means that fewer researchers will be able to work with those signals.

Recording audio: software tools

A short list of software tool we already tested and checked:
- audacity http://audacity.sourceforge.net/
- sox http://sox.sourceforge.net/

Recording speech: SPPAS requirements

wav and au audio file formats
an audio file is MONO: only one channel
good recording quality is expected
Example:

(Step 2) Inter-Pausal Units

IPUs = sounding segments

The orthographic transcription must be pre-segmented

Orthographic transcription into the IPUs

How to do it?

SPPAS can automatically perform the IPUs segmentation
- then manual verification is recommended
- see the related tutorial and/or the documentation for details
but IPUs will be segmented manually:
- if the audio signal is of poor quality;
- if there are more than one speaker recorded in the same channel.

(Step 3) Transcribing Speech

Orthographic transcription:

must include:
- Filled pauses
- Short pauses
- Repeats
- Truncated words
- Noises
- Laughter
should include:
- un-regular elisions
- un-regular liaisons
- specific pronunciations
may include
- all elisions

Orthographic transcription: SPPAS convention

truncated words, noted as a '-' at the end of the token string (an ex- example)
noises, noted by a ’*’
laughter, noted by a ’@’
short pauses, noted by a ’+’
elisions, mentioned in parenthesis
specific pronunciations, noted with brackets [example,eczap]
comments are noted inside braces or brackets without using comma {this} or [this and this]
liaisons, noted between ’=’ (an =n= example)
morphological variants with <like,lie ok>
proper name annotation, like $John S. Doe$

Transcription example 1 (Conversational speech)

Manual orthographic transcription:

donc + i- i(l) prend la è- recette et tout bon i(l) vé- i(l) dit bon [okay, k]

Automatically extracted standard orthograph:
- donc il prend la recette et tout bon il dit bon okay
Automatically extracted tokens:
- donc + i i prend la è recette et tout bon i vé i dit bon k

Transcription example 2 (Conversational speech)

Manual orthographic transcription:

ah mais justement c’était pour vous vendre bla bla bla bl(a) le mec i(l) te l’a emboucané + en plus i(l) lu(i) a [acheté,acheuté] le truc et le mec il est parti j(e) dis putain le mec i(l) voulait

Automatically extracted standard orthography:
- ah mais justement c’était pour vous vendre bla bla bla bla le mec il te l’a emboucané en plus il lui a acheté le truc et le mec il est parti je dis putain le mec il voulait
Automatically extracted tokens:
- ah mais justement c’était pour vous vendre bla bla bla bl le mec i te l’a emboucané + en plus i lu a acheuté le truc et le mec il est parti j dis putain le mec i voulait

Transcription example 3 (GrenelleII)

Manual orthographic transcription:

euh les apiculteurs + et notamment b- on ne sait pas très bien + quelle est la cause de mortalité des abeilles m(ais) enfin il y a quand même + euh peut-êt(r)e des attaques systémiques

Automatically extracted standard orthography:
- les apiculteurs et notamment on ne sait pas très bien quelle est la cause de mortalité des abeilles mais enfin il y a quand même peut-être des attaques systémiques
Automatically extracted faked orthography:
- euh les apiculteurs + et notamment b on ne sait pas très bien + quelle est la cause de mortalité des abeilles m enfin il y a quand même + euh peut-ête des attaques systémiques

Annotated files: recommendations

UTF-8 encoding only
only US-ASCII characters in file names (nor in the path obviously)
Supported file formats to open/save (software, extension):
- SPPAS: xra
- Praat: TextGrid, PitchTier, IntensityTier
- Elan: eaf
- AnnotationPro: antx
- HTK: lab, mlf
- Sclite: ctm, stm
- Phonedit: mrk
- Excel/OpenOffice/R/…: csv
- Subtitles: sub, srt
Supported file formats for import only (software, extension):
- Transcriber: trs
- Anvil: anvil
- Xtrans: tdf
- Audacity: txt

Supported file formats

Always remember…

Each step - from collecting the recordings to the analyses of the annotations, depends on the previous one, and:

Garbage in, garbage out

You reached the end of this tutorial!

Click here to go back to SPPAS tutorials web page