Base

class audbackend.backend.Base(host, repository, *, authentication=None)

Backend base class.

Derive from this class to implement a new backend.

authentication

Base.authentication

Object used for authentication, e.g. username, password tuple.

checksum()

Base.checksum(path)

MD5 checksum for file on backend.

Parameters

path (str) – path to file on backend

Return type

str

Returns

MD5 checksum

Raises

• BackendError – if an error is raised on the backend, e.g. path does not exist

• ValueError – if path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

close()

Base.close()

Close connection to backend.

Raises

BackendError – if an error is raised on the backend

copy_file()

Base.copy_file(src_path, dst_path, *, validate=False, verbose=False)

Copy file on backend.

If dst_path exists and has a different checksum, it is overwritten. Otherwise, the operation is silently skipped.

If validate is set to True, a final check is performed to assert that src_path and dst_path have the same checksum. If it fails, dst_path is removed and an InterruptedError is raised.

Parameters

• src_path (str) – source path to file on backend

• dst_path (str) – destination path to file on backend

• validate (bool) – verify file was successfully copied

• verbose (bool) – show debug messages

Raises

• BackendError – if an error is raised on the backend

• InterruptedError – if validation fails

• ValueError – if src_path or dst_path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

create()

classmethod Base.create(host, repository, *, authentication=None)

Create repository.

Creates repository located at host on the backend.

Parameters

• host (str) – host address

• repository (str) – repository name

• authentication (Optional[object]) – object used for authentication, e.g. a tuple with username and password

Raises

BackendError – if an error is raised on the backend, e.g. repository exists already or cannot be created

date()

Base.date(path)

Last modification date of file on backend.

If the date cannot be determined, an empty string is returned.

Parameters

path (str) – path to file on backend

Return type

str

Returns

date in format 'yyyy-mm-dd'

Raises

• BackendError – if an error is raised on the backend, e.g. path does not exist

• ValueError – if path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

delete()

classmethod Base.delete(host, repository, *, authentication=None)

Delete repository.

Deletes repository located at host on the backend.

Parameters

• host (str) – host address

• repository (str) – repository name

• authentication (Optional[object]) – access token for possible authentication, e.g. username, password tuple

Raises

BackendError – if an error is raised on the backend, e.g. repository does not exist

exists()

Base.exists(path, *, suppress_backend_errors=False)

Check if file exists on backend.

Parameters

• path (str) – path to file on backend

• suppress_backend_errors (bool) – if set to True, silently catch errors raised on the backend and return False

Return type

bool

Returns

True if file exists

Raises

• BackendError – if suppress_backend_errors is False and an error is raised on the backend, e.g. due to a connection timeout

• ValueError – if path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• ValueError – if version is empty or does not match '[A-Za-z0-9._-]+'

• RuntimeError – if backend was not opened

get_archive()

Base.get_archive(src_path, dst_root, *, tmp_root=None, validate=False, verbose=False)

Get archive from backend and extract.

The archive type is derived from the extension of src_path. See audeer.extract_archive() for supported extensions.

If dst_root does not exist, it is created.

If validate is set to True, a final check is performed to assert that src_path and the retrieved archive have the same checksum. If it fails, the retrieved archive is removed and an InterruptedError is raised.

Parameters

• src_path (str) – path to archive on backend

• dst_root (str) – local destination directory

• tmp_root (Optional[str]) – directory under which archive is temporarily extracted. Defaults to temporary directory of system

• validate (bool) – verify archive was successfully retrieved from the backend

• verbose (bool) – show debug messages

Return type

list[str]

Returns

extracted files

Raises

• BackendError – if an error is raised on the backend, e.g. src_path does not exist

• FileNotFoundError – if tmp_root does not exist

• InterruptedError – if validation fails

• NotADirectoryError – if dst_root is not a directory

• PermissionError – if the user lacks write permissions for dst_path

• RuntimeError – if extension of src_path is not supported or src_path is a malformed archive

• ValueError – if src_path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

get_file()

Base.get_file(src_path, dst_path, *, validate=False, verbose=False)

Get file from backend.

If the folder of dst_path does not exist, it is created.

If dst_path exists with a different checksum, it is overwritten, Otherwise, the operation is silently skipped.

If validate is set to True, a final check is performed to assert that src_path and dst_path have the same checksum. If it fails, dst_path is removed and an InterruptedError is raised.

Parameters

• src_path (str) – path to file on backend

• dst_path (str) – destination path to local file

• validate (bool) – verify file was successfully retrieved from the backend

• verbose (bool) – show debug messages

Return type

str

Returns

full path to local file

Raises

• BackendError – if an error is raised on the backend, e.g. src_path does not exist

• InterruptedError – if validation fails

• IsADirectoryError – if dst_path points to an existing folder

• PermissionError – if the user lacks write permissions for dst_path

• ValueError – if src_path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

host

Base.host

Host path.

join()

Base.join(path, *paths)

Join to (sub-)path on backend.

Parameters

• path (str) – first part of path

• *paths – additional parts of path

Return type

str

Returns

path joined by Backend.sep

Raises

ValueError – if path contains invalid character or does not start with '/', or if joined path contains invalid character

ls()

Base.ls(path='/', *, pattern=None, suppress_backend_errors=False)

List files on backend.

Returns a sorted list of tuples with path and version. If a full path (e.g. /sub/file.ext) is provided, all versions of the path are returned. If a sub-path (e.g. /sub/) is provided, all files that start with the sub-path are returned. When path is set to '/' a (possibly empty) list with all files on the backend is returned.

Parameters

• path (str) – path or sub-path (if it ends with '/') on backend

• pattern (Optional[str]) – if not None, return only files matching the pattern string, see fnmatch.fnmatch()

• suppress_backend_errors (bool) – if set to True, silently catch errors raised on the backend and return an empty list

Return type

list[str]

Returns

list of tuples (path, version)

Raises

• BackendError – if suppress_backend_errors is False and an error is raised on the backend, e.g. path does not exist

• ValueError – if path does not start with '/' or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

move_file()

Base.move_file(src_path, dst_path, *, validate=False, verbose=False)

Move file on backend.

If dst_path exists and has a different checksum, it is overwritten. Otherwise, src_path is removed and the operation silently skipped.

If validate is set to True, a final check is performed to assert that src_path and dst_path have the same checksum. If it fails, dst_path is removed and an InterruptedError is raised. To ensure src_path still exists in this case it is first copied and only removed when the check has successfully passed.

Parameters

• src_path (str) – source path to file on backend

• dst_path (str) – destination path to file on backend

• validate (bool) – verify file was successfully moved

• verbose (bool) – show debug messages

Raises

• BackendError – if an error is raised on the backend

• InterruptedError – if validation fails

• ValueError – if src_path or dst_path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

open()

Base.open()

Open connection to backend.

Repository must exist, use audbackend.backend.Base.create() to create it. Finally, use audbackend.backend.Base.close() to close the connection. Instead of explicitly calling audbackend.backend.Base.open() and audbackend.backend.Base.close() it is good practice to use a with statement.

Raises

BackendError – if an error is raised on the backend, e.g. repository does not exist

opened

Base.opened

If a connection to the repository has been established.

owner()

Base.owner(path)

Owner of file on backend.

If the owner of the file cannot be determined, an empty string is returned.

Parameters

path (str) – path to file on backend

Return type

str

Returns

owner

Raises

• BackendError – if an error is raised on the backend, e.g. path does not exist

• ValueError – if path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

put_archive()

Base.put_archive(src_root, dst_path, *, files=None, tmp_root=None, validate=False, verbose=False)

Create archive and put on backend.

The archive type is derived from the extension of dst_path. See audeer.create_archive() for supported extensions.

The operation is silently skipped, if an archive with the same checksum already exists on the backend.

If validate is set to True, a final check is performed to assert that the local archive and dst_path have the same checksum. If it fails, dst_path is removed and an InterruptedError is raised.

Parameters

• src_root (str) – local root directory where files are located. By default, all files below src_root will be included into the archive. Use files to select specific files

• dst_path (str) – path to archive on backend

• files (Union[str, Sequence[str], None]) – file(s) to include into the archive. Must exist within src_root

• tmp_root (Optional[str]) – directory under which archive is temporarily created. Defaults to temporary directory of system

• validate (bool) – verify archive was successfully put on the backend

• verbose (bool) – show debug messages

Raises

• BackendError – if an error is raised on the backend

• FileNotFoundError – if src_root, tmp_root, or one or more files do not exist

• InterruptedError – if validation fails

• NotADirectoryError – if src_root is not a folder

• RuntimeError – if dst_path does not end with zip or tar.gz or a file in files is not below root

• ValueError – if dst_path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

put_file()

Base.put_file(src_path, dst_path, *, validate=False, verbose=False)

Put file on backend.

The operation is silently skipped, if a file with the same checksum already exists on the backend.

If validate is set to True, a final check is performed to assert that src_path and dst_path have the same checksum. If it fails, dst_path is removed and an InterruptedError is raised.

Parameters

• src_path (str) – path to local file

• dst_path (str) – path to file on backend

• validate (bool) – verify file was successfully put on the backend

• verbose (bool) – show debug messages

Raises

• BackendError – if an error is raised on the backend

• FileNotFoundError – if src_path does not exist

• InterruptedError – if validation fails

• IsADirectoryError – if src_path is a folder

• ValueError – if dst_path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

remove_file()

Base.remove_file(path)

Remove file from backend.

Parameters

path (str) – path to file on backend

Raises

• BackendError – if an error is raised on the backend, e.g. path does not exist

• ValueError – if path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

repository

Base.repository

Repository name.

sep

Base.sep

File separator on backend.

Returns: file separator

split()

Base.split(path)

Split path on backend into sub-path and basename.

Parameters

path (str) – path containing Backend.sep as separator

Return type

tuple[str, str]

Returns

tuple containing (root, basename)

Raises

ValueError – if path does not start with '/' or does not match '[A-Za-z0-9/._-]+'