SPPAS

Workspaces

Overview

A workspace is a manager of both:

a set of files, and
manually defined references.

Several workspaces can be created but only one at a time is enabled when using SPPAS.

SPPAS functionalities are using the currently enabled workspace.

Create and save a workspace

A workspace is described in a configuration file into the folder workspaces of the SPPAS package. This file is made of the name of the workspace followed by the extension .json. It includes all properties of a workspace that are described below.

This configuration file can be copied into any other place, and re-imported into SPPAS.

Workspace Files

A workspace is managing a set of files. Every time a file is added into the currently selected workspace, its root and its path are extracted and stored into a tree-like architecture.

The data are structured as follows:

a workspace contains a list of paths;
each path contains the list of roots sharing this path;
each root contains the list of filenames sharing this root.

For example, when adding the file SPPAS/samples/samples-eng/oriana1.wav into the workspace, the following next actions are performed:

the path SPPAS/samples/samples-eng/ is added;
the root SPPAS/samples/samples-eng/oriana1 is added into the path;
the filename SPPAS/samples/samples-eng/oriana1.wav is added into the root. Several properties of the file are also stored, including its size or the date of last modification.

When SPPAS is creating a new file - like when annotating automatically, its filename is automatically added into the workspace. For example, when the automatic annotation Fill in IPUs finished, the newly created file SPPAS/samples/samples-eng/oriana1.xra is added into the workspace. It results in the following actions:

the path SPPAS/samples/samples-eng/ is already existing so it’s not added;
the root SPPAS/samples/samples-eng/oriana1 is not added into the path because it’s also already existing;
the filename SPPAS/samples/samples-eng/oriana1.xra is added into the root.

All files sharing the same root are then stored at the same place in the tree because they are all related to the same recording. The only difference between files sharing the same root are:

the extension (.wav, .mp4, .mkv, .xra, .TextGrid, …)
the pattern (-ipus, -token, -phon, -palign, -lexm, -orepet, …)

Notice that a pattern:

must start by the character -;
must contain at least 1 character.

Thanks to this management of files, the patterns used by the automatic annotations are not fixed anymore. Both the input and output patterns of the annotations can be modified at the time of configuring the annotations.

The workspace is also managing the state of each of its items: checked or not for filenames, checked/at least one checked/not checked for roots or paths.

The main benefit of such workspace file management is to add all files of a corpus only once; it is then ready-to-use every time SPPAS is started.

Workspace References

The references are information which can be associated to files.

There are three different types of references: STANDALONE, SPEAKER and INTERACTION. Each of them corresponds to an annotation type. The annotations of both SPEAKER and INTERACTION types require associating two roots with these references.

Each reference can contain a list of attributes with an identifier key to define them, a value and its type, and a description. Here is an example of a reference Oriana of type SPEAKER with its list of attributes in the form key: values (type):

firstname: Oriana (str)
gender: female (str)
recorded_in: 2012 (int)
L1: eng (str)
L2: fra (str)

It is recommended to use only us-ascii characters and no whitespace for the key.

Associate files and references

Roots and references should be associated. In the previous example, it can allow declaring the list of files related to a given speaker. In that example, the reference Oriana should be associated to the roots: SPPAS/samples/samples-eng/oriana1 and SPPAS/samples/samples-eng/oriana2.

In the specific case of interactions, a reference of the interaction has to be created and associated to the roots of the filenames.

The definition of such references, and their link to the corresponding roots allow implementing an elaborated way to check files of a workspace. It’s very easy, for example, to check all files with file extension .wav of female gender, and more than 30 years old.

Command-Line user Interface

Definition

A command-line user interface (CLI) is a means of interacting with a computer program where the user issues commands to the program in the form of successive lines of text (command lines). Command-line interfaces provide a more concise and powerful means to control the program than the GUI - Graphical User Interface.

Operating system command line interfaces are called a command-line interpreter, command processor or shell. It displays a prompt, accept a command line typed by the user terminated by the Enter key, then execute the specified command and provide textual display of results or error messages. When a shell is active a program is typically invoked by typing its name followed by command-line arguments (if any).

Overview

SPPAS CLI programs are located in the sppas/bin folder of the package. All these programs are written with the programming language Python.

These program can be invoked directly with the python system command or preferably with the one of the SPPAS virtual environment: .sppaspyenv/Scripts/python.exe under Windows, or .sppaspyenv/bin/python under macOS/Linux.

To be more concise, we will use the term python in the rest of this document. However, it refers to one of the previously mentioned commands.

Usage

It is usual for a program to be able to display a brief summary of its parameters. Each program included in SPPAS provides its usage by using the option --help, like for example:


     prompt> python .\sppas\bin\trsconvert.py --help
     usage: trsconvert.py [files] [options]
 
     ... a program to export annotated files.
 
     options:
       -h, --help   show this help message and exit
       --quiet      Disable the verbosity
       --debug      Highest level of verbosity
 
     Files:
       -i file      Input annotated file name.
       -o file      Output annotated file name.
 
     Options:
       -n value     Number of a tier (use as many -n options as wanted).
                    Positive or negative value: 1=first tier, -1=last tier.
       -t tiername  Name of a tier (use as many -t options as wanted).
 
     This program is part of SPPAS version 4.14.
     Copyright (C) 2011-2023 Brigitte Bigi, CNRS.
     Contact the author at: contact@sppas.org

Arguments for input/output

In most of the programs, there is an option -i for the input file. There’s no specific constraint on this file name. For example, the following program will execute the Momel automatic annotation:


     python .\sppas\bin\momel.py -i .\samples\samples-eng\ENG_M15_ENG_T02.PitchTier

With this option -i, a name of an output file can be given with the option -o; if not, the main part of the result is printed on the standard output.

In several programs, an option -I can also be available to execute the program, and several files can be processed with this option. Moreover, there is some flexibility in file names with this option. SPPAS will search for the appropriate file from the given file name. For example, all the next commands will process the same:


     python .\sppas\bin\intsint.py -I .\samples\samples-eng\ENG_M15_ENG_T02.wav
     python .\sppas\bin\intsint.py -I .\samples\samples-eng\ENG_M15_ENG_T02.PitchTier
     python .\sppas\bin\intsint.py -I .\samples\samples-eng\ENG_M15_ENG_T02-momel.xra

With the option -I, the name of the output file is fixed and can't be changed. For example, the previous example will create a file with name .\samples\samples-eng\ENG_M15_ENG_T02-intsint.xra. An option -e allows to choose the extension of the file, .xra is the default one.

The options to manage input/output files can be summarized as follows:

Files (manual mode):
        -i file               An input file.
        -o file               Output file name (optionnal).
      Files (auto mode):
        -I file               Input file (append).
        -e .ext               Output file extension. One of: .xra .TextGrid .eaf
                              .csv .mrk .txt .stm .ctm .lab .mlf .sub .srt .antx
                              .arff .xrff

Explore the bin folder for the full list of available programs.

Graphical User Interface

Overview

The GUI feature is enabled only if wxPython was successfully installed during the setup.

Launch under Windows

Once the SPPAS package is opened in the File Explorer, double-click on the sppas.bat file.

In recent versions of Windows (e.g. 10+), the first time you try to run SPPAS you may get a window with title Windows protected your PC and the following message: Windows SmartScreen prevented an unrecognized app from starting. Running this app might put your PC at risk. More info. Click More info message and then Run anyway button. The file will run SPPAS like expected, and you will now no longer get a Windows protected your PC prompt when you run this specific file next time.

Launch under macOS

Once the SPPAS package is opened in a file explorer, double-click on the sppas.command file.

The first time you try to run SPPAS you may get a message: sppas.command can't be opened because it is from an unidentified developer.. Then, you can run SPPAS with a right click (alt-click) on sppas.command file. This time you will get a message: sppas.command is from an unidentified developer. Are you sure you want to open it? Then click on Open. It will now work every time you run it.

If Apple is protecting you more than expected, the last solution is to directly execute either chmod +x sppas.command; .sppaspyenv/bin/python sppas.command or .sppaspyenv/bin/python sppas in a Terminal application.

Launch under Linux

Once the SPPAS package is opened in the File Explorer, double-click the sppas.command file. If it does not work, you can directly execute .sppaspyenv/bin/python sppas in a Terminal.

The GUI of SPPAS

Both a main window and a Log window are opened. The main frame is made of a menubar at top, a main content in the middle and buttons for actions at bottom.

Most of the existing/incoming tutorials are explaining how to access features of SPPAS with this GUI. This chapter only summarizes some of them.

The main frame

The top menu allows opening various pages like Files, Annotate or Analyze. There are two ways to browse through the items of the menu: with the mouse or with the keyboard. To open a page with the mouse, click on the item button in the menubar. To open a page with the keyboard, under Windows, press CTRL and under macOS, press COMMAND. Click left-right arrows to browse through the pages. Use up-arrow to go to the 1st page and Down-arrow to the last page. Use CTRL+F/COMMAND+F to go to the Files page.

To change colors and fonts, click the Settings icon at bottom. These settings are saved in a file which is used every time SPPAS is executed.

The About action button allows displaying information about SPPAS: author, license, a link to the website, etc.

Menubar: Home page

The home page shows a welcome message and gives quick access to a set of external links. Clicking on a button opens the default web-browser of the OS to this link.

The links are pointing to:

the SPPAS website: https://sppas.org
this book
in-line tutorials
the F.A.Q.
the home page of the author, including the whole list of publications related to SPPAS: https://sppas.org/bigi/

Menubar: Files page

The Files page is managing SPPAS workspaces (see the previous section) and their content.

Workspaces:

A column at left contains a list of buttons in green color to perform actions related to the workspaces, then the list of already existing ones.

The possible actions to perform are:

Import from: select a workspace file (.wjson) somewhere on the computer and add it into the SPPAS workspaces directory.
Export to: save the currently enabled workspace somewhere in the computer.
Pin & Save: The currently selected workspace is saved. If the currently selected workspace is Blank, clicking this action is asking for a name and create a new workspace (Pin), then select it.
Rename: change the name of the currently selected workspace. Can't be applied on the Blank workspace.

The list of workspaces allows selecting a workspace among the existing ones or working with the Blank one (default).

Files:

The "files" area displays the filenames of the currently selected workspace, in a tree-like view, and a toolbar with action buttons:

Add: add new files into the workspace.
Remove checked: remove the checked files of the workspace.
Remove missing files of the workspace: The tree of paths/roots/files is explored and if a filename does not correspond to an existing file on the disk, it is removed of the workspace.
Add all existing files of the workspace: All files of the disk sharing the checked roots are added to the workspace tree.

References:

The References area with a blue color displays the tree of references of the currently selected workspace, and a toolbar with action buttons.

The actions to perform are:

Create: Open a dialog to ask for a name and the reference type among: STANDALONE, INTERACTION, SPEAKER. It then creates the corresponding reference and add it to the workspace.
Edit: Open a dialog to delete, add or modify the attributes of the checked references.
Delete: Definitively delete the checked references. There's no way to recover them if needed.

Column of action buttons

Between files and references, there’s a column with action buttons. The actions to perform are either related to files or to link files and references.

Edit checked files: open files with a text editor (under development)
Delete checked files: remove the checked files of the workspace and delete them of the disk. Actually, there are moved and renamed into the trash of SPPAS. To recover a mistaken deleted file, use your OS explorer.
Check some files matching...: It is a powerful way to check files matching some criteria: a given pattern, a given extension, a given reference, etc. For example, it is possible to check all TextGrid files of speakers older than 30.
Check/uncheck all files.
Associate the checked roots to the checked references
Dissociate the checked roots of the checked references

Menubar: Annotate page

This page allows annotating automatically files of the checked roots, of the currently enabled workspace.

The automatic annotation is a 5-steps process:

Choose the output file format. All the automatic annotations are creating files: choosing a file extension means to choose the format of the resulting file. Some formats are more informative (xra) than others, some are more sharable...
Fix the language. Some annotations are language-independent, so they don't need a specific language resource, like Search for IPUs or Face detection. But some annotations are requiring linguistic resources like a lexicon (Text Normalization), a dictionary (Phonetization), etc. This step allows choosing the language among the list of installed ones. Other languages can be added.
Select and configure annotations. Click any of the STANDALONE, SPEAKER or INTERACTION button. The page content is changing in order to select the annotations SPPAS will perform and to configure them. Click the top arrow to come back to the main page.
Perform the annotations. Click the Let’s go button to launch the automatic annotation process, and wait! Some annotations are performed very fast like Text Normalization or Phonetization; however, some are very slow like Face Detection (about 15 minutes for a video file of 1 minute).
A procedure outcome report is displayed and saved in the logs folder of the SPPAS package.

The previously saved reports are listed in the column at the right of the page. Click the button Show report to display the content of the checked one.

For new users: It is highly recommended to launch annotations on the given samples and/or the demo before trying to annotate your own data.

Menubar: Analyze page

It allows displaying a summary of each file: either audio or annotated files, and helping in their analysis. The following helpful actions can be performed:

verify if an audio file is compatible with the automatic annotations;
manage tiers: duplicate, remove, copy, move, ...
estimate distribution statistics on tiers;
create new tiers with the filter systems: either the simple or the relation filter. Click here for details on how to request your annotated data

Menubar: Editor page

This editor allows playing several media at a time synchronously.

It allows annotating manually and viewing files in a timeline. The features video must be enabled to play a video file and the feature audioplay to play an audio file.

Menubar: Convert page

The convert page allows exporting the checked annotated files of the workspace in any of the supported file format.

The displayed table indicates all the format capabilities, including:

Metadata: the file format supports metadata, i.e., it can save extra-information, not only annotations. If not, the metadata won’t be exported;
Multi tiers: the file format accepts to save more than one tier;
No tier: the file format supports to save a file without tiers;
Point: the file format can save tiers of type Point;
Interval: the file format can save tiers of type Interval.
Gaps: the file format supports gaps between annotations of a tier. If not, when exporting the file, SPPAS fills in the gaps by annotations with an empty label;
Overlaps: the file format supports overlaps between annotations of a tier. If not, when exporting the file, SPPAS will split annotations to create a sequence of non-overlapping ones.
Hierarchy: the format can deal with a hierarchy of tiers. If not, the hierarchy is not be saved.
Ctrl vocab: the format can save a controlled vocabulary of tiers. If not, the controlled vocabularies are not be saved.
Media: the format can save a reference to a media of tiers. If not, the media information are not saved.
Vagueness: the format supports a vagueness value of the localization. If not, the vagueness values are not saved.
Alt. loc: the format supports alternative annotation localizations. If not, the alternative localizations are not saved.
Alt. tag: the format supports alternative annotation tags. If not, when exporting the annotations, SPPAS is using a { | } system to parse the alternative tags and = for scores.

Menubar: Plugins page

Installing plugins is a useful solution to extend the features of SPPAS. Several plugins are available for download in the website. The plugins of SPPAS are installed in a folder with name sppas/plugins in the SPPAS package.

To install a new plugin, simply follow this workflow:

Create or download the plugin package - e.g., a zip file.
Execute SPPAS.
Click on Plugin then click on the Install button of the toolbar.
Browse to indicate the downloaded plugin package.
See the new plugin at bottom of the plugin's list.

To delete a plugin, click on the Delete button of the toolbar. Choose the plugin in the given list then click on the OK button. Notice that the plugin is definitively deleted of the disk.

To execute a plug-in, check the file(s) of your current workspace in the File page, then, click on the icon of the plug-in and follow instructions of the plugged program.

User interfaces

Workspaces

Overview

Create and save a workspace

Workspace Files

Workspace References

Associate files and references

Command-Line user Interface

Definition

Overview

Usage

Arguments for input/output

Graphical User Interface

Overview

Launch under Windows

Launch under macOS

Launch under Linux

The GUI of SPPAS

The main frame

Menubar: Home page

Menubar: Files page

Workspaces:

Files:

References:

Column of action buttons

Menubar: Annotate page

Menubar: Analyze page

Menubar: Editor page

Menubar: Convert page

Menubar: Plugins page