Smile¶

class opensmile.Smile(feature_set=FeatureSet.ComParE_2016, feature_level=FeatureLevel.Functionals, *, options=None, loglevel=2, logfile=None, sampling_rate=None, channels=0, mixdown=False, resample=False, segment=None, keep_nat=False, num_workers=1, multiprocessing=False, verbose=False)[source]¶

OpenSMILE feature extractor.

You can choose a pre-defined feature set by passing one of opensmile.FeatureSet

You can also provide a custom config file using the following template:

[componentInstances:cComponentManager]
instance[dataMemory].type=cDataMemory

;;; default source
\{\cm[source{?}:source include config]}

;;; add components reading from reader.dmLevel=wave

;;; combine features
[componentInstances:cComponentManager]
instance[funcconcat].type=cVectorConcat
[funcconcat:cVectorConcat]
reader.dmLevel = <feat-1>;<feat-2>;...
writer.dmLevel = features
includeSingleElementFields = 1

;;; default sink
\{\cm[sink{?}:include external sink]}

Note

The following arguments are not serialized:

keep_nat
loglevel
logfile
num_workers
multiprocessing
segment
verbose

For more information see section on hidden arguments.

Parameters

feature_set (str | FeatureSet) – default feature set or path to a custom config file
feature_level (str | FeatureLevel) – default feature level or level name if a custom config file is used
options (Optional[dict]) – dictionary with optional script parameters
loglevel (int) – log level (0-5), the higher the number the more log messages are given
logfile (Optional[str]) – if not None log messages will be stored to this file
sampling_rate (Optional[int]) – sampling rate in Hz. If None it will call process_func with the actual sampling rate of the signal.
channels (int | Sequence[int]) – channel selection, see audresample.remix()
mixdown (bool) – apply mono mix-down on selection
resample (bool) – if True enforces given sampling rate by resampling
segment (Optional[Segment]) – when a audinterface.Segment object is provided, it will be used to find a segmentation of the input signal. Afterwards processing is applied to each segment
keep_nat (bool) – if the end of segment is set to NaT do not replace with file duration in the result
num_workers (Optional[int]) – number of parallel jobs or 1 for sequential processing. If None will be set to the number of processors on the machine multiplied by 5 in case of multithreading and number of processors in case of multiprocessing
multiprocessing (bool) – use multiprocessing instead of multithreading
verbose (bool) – show debug messages

Examples: >>> sampling_rate = 16000 >>> signal = np.zeros(sampling_rate) >>> smile = Smile( … feature_set=FeatureSet.ComParE_2016, … feature_level=FeatureLevel.Functionals, … ) >>> smile.process_signal(signal, sampling_rate).audspec_lengthL1norm_sma_range start end 0 days 0 days 00:00:01 0.0 Name: audspec_lengthL1norm_sma_range, dtype: float32

call()¶

Smile.__call__(signal, sampling_rate)[source]¶

Apply processing to signal.

This function processes the signal without transforming the output into a pd.DataFrame. Instead it will return the raw processed signal. However, if channel selection, mixdown and/or resampling is enabled, the signal will be first remixed and resampled if the input sampling rate does not fit the expected sampling rate.

Parameters

signal (ndarray) – signal values
sampling_rate (int) – sampling rate in Hz

Return type

ndarray

Returns

Processed signal

Raises

RuntimeError – if sampling rates do not match
RuntimeError – if channel selection is invalid

arguments¶

Smile.arguments¶

Returns arguments that are serialized.

Returns: Dictionary of arguments and their values.
Raises: RuntimeError – if arguments are found that are not assigned to attributes of the same name

Examples

>>> import audobject.testing
>>> o = audobject.testing.TestObject('test', point=(1, 1))
>>> o.arguments
{'name': 'test', 'point': (1, 1)}

borrowed_arguments¶

Smile.borrowed_arguments¶

Returns borrowed arguments.

Returns: Dictionary with borrowed arguments.

column_names¶

Smile.column_names¶: Feature column names.

config_name¶

Smile.config_name¶: Return name of config file (without file extension).

config_path¶

Smile.config_path¶: Return file path of config file.

default_config_root¶

Smile.default_config_root¶: Return root directory with standard config files.

feature_level¶

Smile.feature_level¶: Standard feature level or sink level in custom config file.

feature_names¶

Smile.feature_names¶: Feature names.

feature_set¶

Smile.feature_set¶: Standard feature set or path to custom config file

from_dict()¶

static Smile.from_dict(d, root=None, **kwargs)¶

Return type: Object

from_yaml()¶

static Smile.from_yaml(path_or_stream, **kwargs)¶

Return type: Object

from_yaml_s()¶

static Smile.from_yaml_s(yaml_string, **kwargs)¶

Return type: Object

hidden_arguments¶

Smile.hidden_arguments¶

Returns hidden arguments.

Returns: List with names of hidden arguments.

hop_dur¶

Smile.hop_dur¶: Hop duration.

id¶

Smile.id¶

Object identifier.

The ID of an object ID is created from its non-hidden arguments.

Returns: object identifier

Examples

>>> class Foo(Object):
...    def __init__(self, bar: str):
...        self.bar = bar
>>> foo1 = Foo('I am unique!')
>>> foo1.id
'893df240-babe-d796-cdf1-c436171b7a96'
>>> foo2 = Foo('I am different!')
>>> foo2.id
'9303f2a5-bfc9-e5ff-0ffa-a9846e2d2190'
>>> foo3 = Foo('I am unique!')
>>> foo1.id == foo3.id
True

is_loaded_from_dict¶

Smile.is_loaded_from_dict¶

Check if object was loaded from a dictionary.

Returns True if object was initialized from a dictionary, e.g. after loading it from a YAML file.

Returns

True if object was loaded from a dictionary,: otherwise False

logfile¶

Smile.logfile¶: Log file

loglevel¶

Smile.loglevel¶: Log level

name¶

Smile.name¶: Name of the feature set.

num_channels¶

Smile.num_channels¶: Expected number of channels

num_features¶

Smile.num_features¶: Number of features.

options¶

Smile.options¶: Dictionary with options

params¶

Smile.params¶: Dictionary of parameters describing the feature set.

process¶

Smile.process¶: Processing object.

process_file()¶

Smile.process_file(file, *, start=None, end=None, root=None, process_func_args=None)¶

Extract features from an audio file.

Parameters

file (str) – file path
start (Union[float, int, str, Timedelta, None]) – start processing at this position. If value is a float or integer it is treated as seconds. See audinterface.utils.to_timedelta() for further options
end (Union[float, int, str, Timedelta, None]) – end processing at this position. If value is a float or integer it is treated as seconds. See audinterface.utils.to_timedelta() for further options
root (Optional[str]) – root folder to expand relative file path
process_func_args (Optional[Dict[str, Any]]) – (keyword) arguments passed on to the processing function. They will temporarily overwrite the ones stored in audinterface.Feature.process.process_func_args

Raises

RuntimeError – if sampling rates do not match
RuntimeError – if channel selection is invalid
RuntimeError – if multiple frames are returned, but win_dur is not set

Return type

DataFrame

process_files()¶

Smile.process_files(files, *, starts=None, ends=None, root=None, process_func_args=None)¶

Extract features for a list of files.

Parameters

files (Sequence[str]) – list of file paths
starts (Union[float, int, str, Timedelta, Sequence[Union[float, int, str, Timedelta]], None]) – segment start positions. Time values given as float or integers are treated as seconds. See audinterface.utils.to_timedelta() for further options. If a scalar is given, it is applied to all files
ends (Union[float, int, str, Timedelta, Sequence[Union[float, int, str, Timedelta]], None]) – segment end positions. Time values given as float or integers are treated as seconds. See audinterface.utils.to_timedelta() for further options. If a scalar is given, it is applied to all files
root (Optional[str]) – root folder to expand relative file paths
process_func_args (Optional[Dict[str, Any]]) – (keyword) arguments passed on to the processing function. They will temporarily overwrite the ones stored in audinterface.Feature.process.process_func_args

Raises

RuntimeError – if sampling rates do not match
RuntimeError – if channel selection is invalid
RuntimeError – if multiple frames are returned, but win_dur is not set

Return type

DataFrame

process_folder()¶

Smile.process_folder(root, *, filetype='wav', include_root=True, process_func_args=None)¶

Extract features from files in a folder.

Note

At the moment does not scan in sub-folders!

Parameters

root (str) – root folder
filetype (str) – file extension
include_root (bool) – if True the file paths are absolute in the index of the returned result
process_func_args (Optional[Dict[str, Any]]) – (keyword) arguments passed on to the processing function. They will temporarily overwrite the ones stored in audinterface.Feature.process.process_func_args

Raises

FileNotFoundError – if folder does not exist
RuntimeError – if sampling rates do not match
RuntimeError – if channel selection is invalid
RuntimeError – if multiple frames are returned, but win_dur is not set

Return type

DataFrame

process_func_applies_sliding_window¶

Smile.process_func_applies_sliding_window¶: Controls if processing function applies sliding window.

process_index()¶

Smile.process_index(index, *, preserve_index=False, root=None, cache_root=None, process_func_args=None)¶

Extract features from an index conform to audformat.

If cache_root is not None, a hash value is created from the index using audformat.utils.hash() and the result is stored as <cache_root>/<hash>.pkl. When called again with the same index, features will be read from the cached file.

Parameters

index (Index) – index with segment information
preserve_index (bool) – if True and audinterface.Feature.process.segment is None the returned index will be of same type as the original one, otherwise always a segmented index is returned
root (Optional[str]) – root folder to expand relative file paths
cache_root (Optional[str]) – cache folder (see description)
process_func_args (Optional[Dict[str, Any]]) – (keyword) arguments passed on to the processing function. They will temporarily overwrite the ones stored in audinterface.Feature.process.process_func_args

Raises

RuntimeError – if sampling rates do not match
RuntimeError – if channel selection is invalid
RuntimeError – if multiple frames are returned, but win_dur is not set
ValueError – if index is not conform to audformat

Return type

DataFrame

process_signal()¶

Smile.process_signal(signal, sampling_rate, *, file=None, start=None, end=None, process_func_args=None)¶

Extract features for an audio signal.

Note

If a file is given, the index of the returned frame has levels file, start and end. Otherwise, it consists only of start and end.

Parameters

signal (ndarray) – signal values
sampling_rate (int) – sampling rate in Hz
file (Optional[str]) – file path
start (Union[float, int, str, Timedelta, None]) – start processing at this position. If value is a float or integer it is treated as seconds. See audinterface.utils.to_timedelta() for further options
end (Union[float, int, str, Timedelta, None]) – end processing at this position. If value is a float or integer it is treated as seconds. See audinterface.utils.to_timedelta() for further options
process_func_args (Optional[Dict[str, Any]]) – (keyword) arguments passed on to the processing function. They will temporarily overwrite the ones stored in audinterface.Feature.process.process_func_args

Raises

RuntimeError – if sampling rates do not match
RuntimeError – if channel selection is invalid
RuntimeError – if dimension of extracted features is greater than three
RuntimeError – if feature extractor uses sliding window, but self.win_dur is not specified
RuntimeError – if number of features does not match number of feature names
RuntimeError – if multiple frames are returned, but win_dur is not set

Return type

DataFrame

process_signal_from_index()¶

Smile.process_signal_from_index(signal, sampling_rate, index, process_func_args=None)¶

Split a signal into segments and extract features for each segment.

Parameters

signal (ndarray) – signal values
sampling_rate (int) – sampling rate in Hz
index (MultiIndex) – a pandas.MultiIndex with two levels named start and end that hold start and end positions as pandas.Timedelta objects. See also audinterface.utils.signal_index()
process_func_args (Optional[Dict[str, Any]]) – (keyword) arguments passed on to the processing function. They will temporarily overwrite the ones stored in audinterface.Feature.process.process_func_args

Raises

RuntimeError – if sampling rates do not match
RuntimeError – if channel selection is invalid
RuntimeError – if multiple frames are returned, but win_dur is not set
ValueError – if index contains duplicates

Return type

DataFrame

resolvers¶

Smile.resolvers¶

Return resolvers.

Returns: Dictionary with resolvers.

short_id¶

Smile.short_id¶

Short object identifier.

The short ID consists of eight characters and is created from its non-hidden arguments.

Returns: short object identifier

Examples

>>> class Foo(Object):
...    def __init__(self, bar: str):
...        self.bar = bar
>>> foo1 = Foo('I am unique!')
>>> foo1.id
'893df240-babe-d796-cdf1-c436171b7a96'
>>> foo1.short_id
'171b7a96'
>>> foo2 = Foo('I am different!')
>>> foo2.short_id
'6e2d2190'
>>> foo3 = Foo('I am unique!')
>>> foo1.short_id == foo3.short_id
True

to_dict()¶

Smile.to_dict(*, include_version=True, flatten=False, root=None)¶

Converts object to a dictionary.

Includes items from audobject.Object.arguments. If an argument has a resolver, its value is encoded. Usually, the object can be re-instantiated using audobject.Object.from_dict(). However, if flatten=True, this is not possible.

Parameters

include_version (bool) – add version to class name
flatten (bool) – flatten the dictionary
root (Optional[str]) – if file is written to disk, set to target directory

Return type

Dict[str, Union[bool, datetime, dict, float, int, list, None, str]]

Returns

dictionary that represent the object

Examples

>>> import audobject.testing
>>> o = audobject.testing.TestObject('test', point=(1, 1))
>>> o.to_dict(include_version=False)
{'$audobject.core.testing.TestObject': {'name': 'test', 'point': [1, 1]}}
>>> o.to_dict(flatten=True)
{'name': 'test', 'point.0': 1, 'point.1': 1}

to_numpy()¶

Smile.to_numpy(frame)¶

Return feature values as a numpy array.

The returned numpy.ndarray has the original shape, i.e. (channels, features, time).

Parameters: frame (DataFrame) – feature frame
Return type: ndarray

to_yaml()¶

Smile.to_yaml(path_or_stream, *, include_version=True)¶

Save object to YAML file.

Parameters

path_or_stream (Union[str, IO]) – file path or stream
include_version (bool) – add version to class name

to_yaml_s()¶

Smile.to_yaml_s(*, include_version=True)¶

Convert object to YAML string.

Parameters: include_version (bool) – add version to class name
Return type: str
Returns: YAML string

Examples

>>> import audobject.testing
>>> o = audobject.testing.TestObject('test', point=(1, 1))
>>> print(o.to_yaml_s(include_version=False))
$audobject.core.testing.TestObject:
  name: test
  point:
  - 1
  - 1

verbose¶

Smile.verbose¶: Show debug messages.

win_dur¶

Smile.win_dur¶: Window duration.

Smile¶

__call__()¶

arguments¶

borrowed_arguments¶

column_names¶

config_name¶

config_path¶

default_config_root¶

feature_level¶

feature_names¶

feature_set¶

from_dict()¶

from_yaml()¶

from_yaml_s()¶

hidden_arguments¶

hop_dur¶

id¶

is_loaded_from_dict¶

logfile¶

loglevel¶

name¶

num_channels¶

num_features¶

options¶

params¶

process¶

process_file()¶

process_files()¶

process_folder()¶

process_func_applies_sliding_window¶

process_index()¶

process_signal()¶

process_signal_from_index()¶

resolvers¶

short_id¶

to_dict()¶

to_numpy()¶

to_yaml()¶

to_yaml_s()¶

verbose¶

win_dur¶

call()¶