annotations.Phon package¶
Submodules¶
annotations.Phon.dagphon module¶
- filename
sppas.src.annotations.Phon.dagphon.py
- author
Brigitte Bigi
- contact
- summary
Direct Acyclic Graph for the phonetization of unknown entries.
- class annotations.Phon.dagphon.sppasDAGPhonetizer(variants=4)[source]¶
Bases:
object
DAG to phonetize unk.
- DAG2phon(graph, pron_graph)[source]¶
Convert a DAG into a dict, including all pronunciation variants.
- Parameters
graph –
pron_graph –
- Returns
- __init__(variants=4)[source]¶
Create a sppasDAGPhonetizer instance.
- Parameters
variants – (int) Maximum number of variants for phonetizations.
annotations.Phon.phonetize module¶
- filename
sppas.src.annotations.Phon.phonetize.py
- author
Brigitte Bigi
- contact
- summary
Phonetization of an utterance.
- class annotations.Phon.phonetize.sppasDictPhonetizer(pdict, maptable=None)[source]¶
Bases:
object
Dictionary-based automatic phonetization.
Grapheme-to-phoneme conversion is a complex task, for which a number of diverse solutions have been proposed. It is a structure prediction task; both the input and output are structured, consisting of sequences of letters and phonemes, respectively.
This phonetization system is entirely designed to handle multiple languages and/or tasks with the same algorithms and the same tools. Only resources are language-specific, and the approach is based on the simplest resources as possible: this automatic annotation is using a dictionary-based approach.
The dictionary can contain words with a set of pronunciations (the canonical one, and optionally some common reductions, etc). In this approach, it is then assumed that most of the words of the speech transcription and their phonetic variants are mentioned in the pronunciation dictionary. If a word is missing, our system is based on the idea that given enough examples it should be possible to predict the pronunciation of unseen words purely by analogy.
- __init__(pdict, maptable=None)[source]¶
Create a sppasDictPhonetizer instance.
- Parameters
pdict – (sppasDictPron) The pronunciation dictionary.
maptable – (Mapping) A mapping table for phones.
- get_phon_entry(entry)[source]¶
Return the phonetization of an entry.
Unknown entries are not automatically phonetized. This is a pure dictionary-based method.
- Parameters
entry – (str) The entry to be phonetized.
- Returns
A string with the phonetization of the given entry or
the unknown symbol.
- get_phon_tokens(tokens, phonunk=True)[source]¶
Return the phonetization of a list of tokens, with the status.
Unknown entries are automatically phonetized if phonunk is set to True.
- Parameters
tokens – (list) The list of tokens to be phonetized.
phonunk – (bool) Phonetize unknown words (or not).
TODO: EOT is not fully supported.
- Returns
A list with the tuple (token, phon, status).
- phonetize(utterance, phonunk=True, delimiter=' ')[source]¶
Return the phonetization of an utterance.
- Parameters
utterance – (str) The utterance string to be phonetized.
phonunk – (bool) Phonetize unknown words (or not).
delimiter – (char) The character to be used to separate entries
in the result and which was used in the given utterance.
- Returns
A string with the phonetization of the given utterance.
- set_dict(pron_dict)[source]¶
Set the pronunciation dictionary.
- Parameters
pron_dict – (sppasDictPron) The pronunciation dictionary.
annotations.Phon.phonunk module¶
- filename
sppas.src.annotations.Phon.phonunk.py
- author
Brigitte Bigi
- contact
- summary
Phonetization of an unknown entry.
- class annotations.Phon.phonunk.sppasPhonUnk(pron_dict)[source]¶
Bases:
object
Perform a dictionary-based phonetization for unknown entries.
Implements a language-independent algorithm to phonetize unknown tokens. The algorithm is based on the idea that given enough examples it should be possible to predict the pronunciation of unseen tokens purely by analogy. It consists in exploring the unknown token from left to right, then from right to left, and to find the longest strings in the dictionary. Since this algorithm uses the dictionary, the quality of such a phonetization strongly depends on this resource.
Example of use:
>>> d = { 'a':'a|aa', 'b':'b', 'c':'c|cc', 'abb':'abb', 'bac':'bac' } >>> p = sppasPhonUnk(d)
- __init__(pron_dict)[source]¶
Create a sppasPhonUnk instance.
- Parameters
pron_dict – (sppasPronDict) Dictionary of a set of tuples:
token=key, phon=value.
annotations.Phon.sppasphon module¶
- filename
sppas.src.annotations.Phon.sppasphon.py
- author
Brigitte Bigi
- contact
- summary
SPPAS integration of Phonetization automatic annotation.
- class annotations.Phon.sppasphon.sppasPhon(log=None)[source]¶
Bases:
annotations.baseannot.sppasBaseAnnotation
SPPAS integration of the Phonetization automatic annotation.
- __init__(log=None)[source]¶
Create a sppasPhon instance without any linguistic resources.
Log is used for a better communication of the annotation process and its results. If None, logs are redirected to the default logging system.
- Parameters
log – (sppasLog) Human-readable logs.
- convert(tier)[source]¶
Phonetize annotations of a tokenized tier.
- Parameters
tier – (Tier) the ortho transcription previously tokenized.
- Returns
(Tier) phonetized tier with name “Phones”
- fix_options(options)[source]¶
Fix all options.
Available options are:
phonunk
usesstdtokens
- Parameters
options – (sppasOption)
- get_inputs(input_files)[source]¶
Return the the tier with aligned tokens.
- Parameters
input_files – (list)
- Raise
NoTierInputError
- Returns
(sppasTier)
- load_resources(dict_filename=None, map_filename=None, **kwargs)[source]¶
Set the pronunciation dictionary and the mapping table.
- Parameters
dict_filename – (str) The pronunciation dictionary in HTK-ASCII
format with UTF-8 encoding.
- Parameters
map_filename – (str) is the filename of a mapping table. It is used to generate new pronunciations by mapping phonemes of the dict.
- run(input_files, output=None)[source]¶
Run the automatic annotation process on an input.
- Parameters
input_files – (list of str) Normalized text
output – (str) the output name
- Returns
(sppasTranscription)
Module contents¶
- filename
sppas.src.annotations.Phon.__init__.py
- author
Brigitte Bigi
- contact
- summary
Multilingual system for Phonetization.
Phonetization is the process of representing sounds with phonetic signs. There are two general ways to construct a phonetization process: dictionary based solutions which consist in storing a maximum of phonological knowledge in a lexicon and rule based systems with rules based on inference approaches or proposed by expert linguists.
A system based on a dictionary solution consists in storing a maximum of phonological knowledge in a lexicon. In this sense, this approach is language-independent unlike rule-based systems. The SPPAS phonetization of the orthographic transcription produces a phonetic transcription based on a phonetic dictionary. The phonetization is the equivalent of a sequence of dictionary look-ups.
It is assumed that all words of the speech transcription are mentioned in the pronunciation dictionary. Otherwise, SPPAS implements a language-independent algorithm to phonetize unknown words. This implementation is in its early stage. It consists in exploring the unknown word from left to right and to find the longuest strings in the dictionary. Since this algorithm uses the dictionary, the quality of such a phonetization will depend on this resource.
Actually, some words can correspond to several entries in the dictionary with various pronunciations. Unlike rule-based systems, in SPPAS the pronunciation is not supposed to be ``standard’’. Phonetic variants are proposed for the aligner to choose the phoneme string. The hypothesis is that the answer to the phonetization question is in the signal.
SPPAS can take as input a tokenized standard orthographic transcription and some enrichment only if the acoustic model includes them. For example, the French transcriptions can contain laugh (represented by the symbol ‘@’ in the transcription).
The SPPAS phonetization follows the conventions:
whitespace separate tokens,
minus separate phonemes,
pipes separate phonetic variants.
For details, read the following reference:
Brigitte Bigi (2016).A phonetization approach for the forced-alignment task in SPPAS.Human Language Technology. Challenges for Computer Science andLinguistics, LNAI 9561, Springer, pp. 515–526.
To summarize:¶
A phoneme is the smallest structural unit that distinguishes meaning in a language. Phonemes are not the physical segments themselves, but are cognitive abstractions or categorizations of them. On the other hand, phones refer to the instances of phonemes in the actual utterances - i.e. the physical segments.
Phonetization consists in searching the possible phones of the given utterance. In the approach implemented in this package, phonetic variants are included in the result.
- class annotations.Phon.sppasDictPhonetizer(pdict, maptable=None)[source]¶
Bases:
object
Dictionary-based automatic phonetization.
Grapheme-to-phoneme conversion is a complex task, for which a number of diverse solutions have been proposed. It is a structure prediction task; both the input and output are structured, consisting of sequences of letters and phonemes, respectively.
This phonetization system is entirely designed to handle multiple languages and/or tasks with the same algorithms and the same tools. Only resources are language-specific, and the approach is based on the simplest resources as possible: this automatic annotation is using a dictionary-based approach.
The dictionary can contain words with a set of pronunciations (the canonical one, and optionally some common reductions, etc). In this approach, it is then assumed that most of the words of the speech transcription and their phonetic variants are mentioned in the pronunciation dictionary. If a word is missing, our system is based on the idea that given enough examples it should be possible to predict the pronunciation of unseen words purely by analogy.
- __init__(pdict, maptable=None)[source]¶
Create a sppasDictPhonetizer instance.
- Parameters
pdict – (sppasDictPron) The pronunciation dictionary.
maptable – (Mapping) A mapping table for phones.
- get_phon_entry(entry)[source]¶
Return the phonetization of an entry.
Unknown entries are not automatically phonetized. This is a pure dictionary-based method.
- Parameters
entry – (str) The entry to be phonetized.
- Returns
A string with the phonetization of the given entry or
the unknown symbol.
- get_phon_tokens(tokens, phonunk=True)[source]¶
Return the phonetization of a list of tokens, with the status.
Unknown entries are automatically phonetized if phonunk is set to True.
- Parameters
tokens – (list) The list of tokens to be phonetized.
phonunk – (bool) Phonetize unknown words (or not).
TODO: EOT is not fully supported.
- Returns
A list with the tuple (token, phon, status).
- phonetize(utterance, phonunk=True, delimiter=' ')[source]¶
Return the phonetization of an utterance.
- Parameters
utterance – (str) The utterance string to be phonetized.
phonunk – (bool) Phonetize unknown words (or not).
delimiter – (char) The character to be used to separate entries
in the result and which was used in the given utterance.
- Returns
A string with the phonetization of the given utterance.
- set_dict(pron_dict)[source]¶
Set the pronunciation dictionary.
- Parameters
pron_dict – (sppasDictPron) The pronunciation dictionary.
- class annotations.Phon.sppasPhon(log=None)[source]¶
Bases:
annotations.baseannot.sppasBaseAnnotation
SPPAS integration of the Phonetization automatic annotation.
- __init__(log=None)[source]¶
Create a sppasPhon instance without any linguistic resources.
Log is used for a better communication of the annotation process and its results. If None, logs are redirected to the default logging system.
- Parameters
log – (sppasLog) Human-readable logs.
- convert(tier)[source]¶
Phonetize annotations of a tokenized tier.
- Parameters
tier – (Tier) the ortho transcription previously tokenized.
- Returns
(Tier) phonetized tier with name “Phones”
- fix_options(options)[source]¶
Fix all options.
Available options are:
phonunk
usesstdtokens
- Parameters
options – (sppasOption)
- get_inputs(input_files)[source]¶
Return the the tier with aligned tokens.
- Parameters
input_files – (list)
- Raise
NoTierInputError
- Returns
(sppasTier)
- load_resources(dict_filename=None, map_filename=None, **kwargs)[source]¶
Set the pronunciation dictionary and the mapping table.
- Parameters
dict_filename – (str) The pronunciation dictionary in HTK-ASCII
format with UTF-8 encoding.
- Parameters
map_filename – (str) is the filename of a mapping table. It is used to generate new pronunciations by mapping phonemes of the dict.
- run(input_files, output=None)[source]¶
Run the automatic annotation process on an input.
- Parameters
input_files – (list of str) Normalized text
output – (str) the output name
- Returns
(sppasTranscription)
- class annotations.Phon.sppasPhonUnk(pron_dict)[source]¶
Bases:
object
Perform a dictionary-based phonetization for unknown entries.
Implements a language-independent algorithm to phonetize unknown tokens. The algorithm is based on the idea that given enough examples it should be possible to predict the pronunciation of unseen tokens purely by analogy. It consists in exploring the unknown token from left to right, then from right to left, and to find the longest strings in the dictionary. Since this algorithm uses the dictionary, the quality of such a phonetization strongly depends on this resource.
Example of use:
>>> d = { 'a':'a|aa', 'b':'b', 'c':'c|cc', 'abb':'abb', 'bac':'bac' } >>> p = sppasPhonUnk(d)
- __init__(pron_dict)[source]¶
Create a sppasPhonUnk instance.
- Parameters
pron_dict – (sppasPronDict) Dictionary of a set of tuples:
token=key, phon=value.