Unversioned

class audbackend.interface.Unversioned(backend)

Interface for unversioned file access.

Use this interface if you don’t care about versioning. For every backend path exactly one file exists on the backend.

Parameters

backend (Base) – backend object

Examples

>>> file = "src.txt"
>>> backend = audbackend.backend.FileSystem("host", "repo")
>>> backend.open()
>>> interface = Unversioned(backend)
>>> interface.put_file(file, "/file.txt")
>>> interface.put_archive(".", "/sub/archive.zip", files=[file])
>>> interface.ls()
['/file.txt', '/sub/archive.zip']
>>> interface.get_file("/file.txt", "dst.txt")
'...dst.txt'

backend

Unversioned.backend

Backend object.

Returns

backend object

Examples

>>> interface.backend
audbackend.backend.FileSystem('host', 'repo')

checksum()

Unversioned.checksum(path)

MD5 checksum for file on backend.

Parameters

path (str) – path to file on backend

Return type

str

Returns

MD5 checksum

Raises

• BackendError – if an error is raised on the backend, e.g. path does not exist

• ValueError – if path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

Examples

>>> file = "src.txt"
>>> import audeer
>>> audeer.md5(file)
'd41d8cd98f00b204e9800998ecf8427e'
>>> interface.put_file(file, "/file.txt")
>>> interface.checksum("/file.txt")
'd41d8cd98f00b204e9800998ecf8427e'

copy_file()

Unversioned.copy_file(src_path, dst_path, *, validate=False, verbose=False)

Copy file on backend.

If dst_path exists and has a different checksum, it is overwritten. Otherwise, the operation is silently skipped.

If validate is set to True, a final check is performed to assert that src_path and dst_path have the same checksum. If it fails, dst_path is removed and an InterruptedError is raised.

Parameters

• src_path (str) – source path to file on backend

• dst_path (str) – destination path to file on backend

• validate (bool) – verify file was successfully copied

• verbose (bool) – show debug messages

Raises

• BackendError – if an error is raised on the backend

• InterruptedError – if validation fails

• ValueError – if src_path or dst_path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

Examples

>>> file = "src.txt"
>>> interface.put_file(file, "/file.txt")
>>> interface.exists("/copy.txt")
False
>>> interface.copy_file("/file.txt", "/copy.txt")
>>> interface.exists("/copy.txt")
True

date()

Unversioned.date(path)

Last modification date of file on backend.

If the date cannot be determined, an empty string is returned.

Parameters

path (str) – path to file on backend

Return type

str

Returns

date in format 'yyyy-mm-dd'

Raises

• BackendError – if an error is raised on the backend, e.g. path does not exist

• ValueError – if path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

Examples

>>> file = "src.txt"
>>> interface.put_file(file, "/file.txt")
>>> interface.date("/file.txt")
'1991-02-20'

exists()

Unversioned.exists(path, *, suppress_backend_errors=False)

Check if file exists on backend.

Parameters

• path (str) – path to file on backend

• suppress_backend_errors (bool) – if set to True, silently catch errors raised on the backend and return False

Return type

bool

Returns

True if file exists

Raises

• BackendError – if suppress_backend_errors is False and an error is raised on the backend, e.g. due to a connection timeout

• ValueError – if path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• ValueError – if version is empty or does not match '[A-Za-z0-9._-]+'

• RuntimeError – if backend was not opened

Examples

>>> file = "src.txt"
>>> interface.exists("/file.txt")
False
>>> interface.put_file(file, "/file.txt")
>>> interface.exists("/file.txt")
True

get_archive()

Unversioned.get_archive(src_path, dst_root, *, tmp_root=None, validate=False, verbose=False)

Get archive from backend and extract.

The archive type is derived from the extension of src_path. See audeer.extract_archive() for supported extensions.

If dst_root does not exist, it is created.

If validate is set to True, a final check is performed to assert that src_path and the retrieved archive have the same checksum. If it fails, the retrieved archive is removed and an InterruptedError is raised.

Parameters

• src_path (str) – path to archive on backend

• dst_root (str) – local destination directory

• tmp_root (Optional[str]) – directory under which archive is temporarily extracted. Defaults to temporary directory of system

• validate (bool) – verify archive was successfully retrieved from the backend

• verbose (bool) – show debug messages

Return type

List[str]

Returns

extracted files

Raises

• BackendError – if an error is raised on the backend, e.g. src_path does not exist

• FileNotFoundError – if tmp_root does not exist

• InterruptedError – if validation fails

• NotADirectoryError – if dst_root is not a directory

• PermissionError – if the user lacks write permissions for dst_path

• RuntimeError – if extension of src_path is not supported or src_path is a malformed archive

• ValueError – if src_path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

Examples

>>> file = "src.txt"
>>> interface.put_archive(".", "/sub/archive.zip", files=[file])
>>> os.remove(file)
>>> interface.get_archive("/sub/archive.zip", ".")
['src.txt']

get_file()

Unversioned.get_file(src_path, dst_path, *, validate=False, verbose=False)

Get file from backend.

If the folder of dst_path does not exist, it is created.

If dst_path exists with a different checksum, it is overwritten, Otherwise, the operation is silently skipped.

If validate is set to True, a final check is performed to assert that src_path and dst_path have the same checksum. If it fails, dst_path is removed and an InterruptedError is raised.

Parameters

• src_path (str) – path to file on backend

• dst_path (str) – destination path to local file

• validate (bool) – verify file was successfully retrieved from the backend

• verbose (bool) – show debug messages

Return type

str

Returns

full path to local file

Raises

• BackendError – if an error is raised on the backend, e.g. src_path does not exist

• InterruptedError – if validation fails

• IsADirectoryError – if dst_path points to an existing folder

• PermissionError – if the user lacks write permissions for dst_path

• ValueError – if src_path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

Examples

>>> file = "src.txt"
>>> interface.put_file(file, "/file.txt")
>>> os.path.exists("dst.txt")
False
>>> _ = interface.get_file("/file.txt", "dst.txt")
>>> os.path.exists("dst.txt")
True

host

Unversioned.host

Host path.

Returns: host path

Examples

>>> interface.host
'host'

join()

Unversioned.join(path, *paths)

Join to path on backend.

Parameters

• path (str) – first part of path

• *paths – additional parts of path

Return type

str

Returns

path joined by Backend.sep

Raises

ValueError – if path contains invalid character or does not start with '/', or if joined path contains invalid character

Examples

>>> interface.join("/", "file.txt")
'/file.txt'
>>> interface.join("/sub", "file.txt")
'/sub/file.txt'
>>> interface.join("//sub//", "/", "", None, "/file.txt")
'/sub/file.txt'

ls()

Unversioned.ls(path='/', *, pattern=None, suppress_backend_errors=False)

List files on backend.

Returns a sorted list of tuples with path and version. If a full path (e.g. /sub/file.ext) is provided, all versions of the path are returned. If a sub-path (e.g. /sub/) is provided, all files that start with the sub-path are returned. When path is set to '/' a (possibly empty) list with all files on the backend is returned.

Parameters

• path (str) – path or sub-path (if it ends with '/') on backend

• pattern (Optional[str]) – if not None, return only files matching the pattern string, see fnmatch.fnmatch()

• suppress_backend_errors (bool) – if set to True, silently catch errors raised on the backend and return an empty list

Return type

List[str]

Returns

list of tuples (path, version)

Raises

• BackendError – if suppress_backend_errors is False and an error is raised on the backend, e.g. path does not exist

• ValueError – if path does not start with '/' or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

Examples

>>> file = "src.txt"
>>> interface.put_file(file, "/file.txt")
>>> interface.put_archive(".", "/sub/archive.zip", files=[file])
>>> interface.ls()
['/file.txt', '/sub/archive.zip']
>>> interface.ls("/file.txt")
['/file.txt']
>>> interface.ls(pattern="*.txt")
['/file.txt']
>>> interface.ls(pattern="archive.*")
['/sub/archive.zip']
>>> interface.ls("/sub/")
['/sub/archive.zip']

move_file()

Unversioned.move_file(src_path, dst_path, *, validate=False, verbose=False)

Move file on backend.

If dst_path exists and has a different checksum, it is overwritten. Otherwise, src_path is removed and the operation silently skipped.

If validate is set to True, a final check is performed to assert that src_path and dst_path have the same checksum. If it fails, dst_path is removed and an InterruptedError is raised. To ensure src_path still exists in this case it is first copied and only removed when the check has successfully passed.

Parameters

• src_path (str) – source path to file on backend

• dst_path (str) – destination path to file on backend

• validate (bool) – verify file was successfully moved

• verbose (bool) – show debug messages

Raises

• BackendError – if an error is raised on the backend

• InterruptedError – if validation fails

• ValueError – if src_path or dst_path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

Examples

>>> file = "src.txt"
>>> interface.put_file(file, "/file.txt")
>>> interface.exists("/move.txt")
False
>>> interface.move_file("/file.txt", "/move.txt")
>>> interface.exists("/move.txt")
True
>>> interface.exists("/file.txt")
False

owner()

Unversioned.owner(path)

Owner of file on backend.

If the owner of the file cannot be determined, an empty string is returned.

Parameters

path (str) – path to file on backend

Return type

str

Returns

owner

Raises

• BackendError – if an error is raised on the backend, e.g. path does not exist

• ValueError – if path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

Examples

>>> file = "src.txt"
>>> interface.put_file(file, "/file.txt")
>>> interface.owner("/file.txt")
'doctest'

put_archive()

Unversioned.put_archive(src_root, dst_path, *, files=None, tmp_root=None, validate=False, verbose=False)

Create archive and put on backend.

The archive type is derived from the extension of dst_path. See audeer.create_archive() for supported extensions.

The operation is silently skipped, if an archive with the same checksum already exists on the backend.

If validate is set to True, a final check is performed to assert that the local archive and dst_path have the same checksum. If it fails, dst_path is removed and an InterruptedError is raised.

Parameters

• src_root (str) – local root directory where files are located. By default, all files below src_root will be included into the archive. Use files to select specific files

• dst_path (str) – path to archive on backend

• files (Union[str, Sequence[str], None]) – file(s) to include into the archive. Must exist within src_root

• tmp_root (Optional[str]) – directory under which archive is temporarily created. Defaults to temporary directory of system

• validate (bool) – verify archive was successfully put on the backend

• verbose (bool) – show debug messages

Raises

• BackendError – if an error is raised on the backend

• FileNotFoundError – if src_root, tmp_root, or one or more files do not exist

• InterruptedError – if validation fails

• NotADirectoryError – if src_root is not a folder

• RuntimeError – if dst_path does not end with zip or tar.gz or a file in files is not below root

• ValueError – if dst_path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

Examples

>>> file = "src.txt"
>>> interface.exists("/sub/archive.tar.gz")
False
>>> interface.put_archive(".", "/sub/archive.tar.gz")
>>> interface.exists("/sub/archive.tar.gz")
True

put_file()

Unversioned.put_file(src_path, dst_path, *, validate=False, verbose=False)

Put file on backend.

The operation is silently skipped, if a file with the same checksum already exists on the backend.

If validate is set to True, a final check is performed to assert that src_path and dst_path have the same checksum. If it fails, dst_path is removed and an InterruptedError is raised.

Parameters

• src_path (str) – path to local file

• dst_path (str) – path to file on backend

• validate (bool) – verify file was successfully put on the backend

• verbose (bool) – show debug messages

Raises

• BackendError – if an error is raised on the backend

• FileNotFoundError – if src_path does not exist

• InterruptedError – if validation fails

• IsADirectoryError – if src_path is a folder

• ValueError – if dst_path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

• RuntimeError – if backend was not opened

Examples

>>> file = "src.txt"
>>> interface.exists("/file.txt")
False
>>> interface.put_file(file, "/file.txt")
>>> interface.exists("/file.txt")
True

remove_file()

Unversioned.remove_file(path)

Remove file from backend.

Parameters

path (str) – path to file on backend

Raises

• BackendError – if an error is raised on the backend, e.g. path does not exist

• ValueError – if path does not start with '/', ends on '/', or does not match '[A-Za-z0-9/._-]+'

Examples

>>> file = "src.txt"
>>> interface.put_file(file, "/file.txt")
>>> interface.exists("/file.txt")
True
>>> interface.remove_file("/file.txt")
>>> interface.exists("/file.txt")
False

repository

Unversioned.repository

Repository name.

Returns

repository name

Examples

>>> interface.repository
'repo'

sep

Unversioned.sep

File separator on backend.

Returns

file separator

Examples

>>> interface.sep
'/'

split()

Unversioned.split(path)

Split path on backend into sub-path and basename.

Parameters

path (str) – path containing Backend.sep as separator

Return type

Tuple[str, str]

Returns

tuple containing (root, basename)

Raises

ValueError – if path does not start with '/' or does not match '[A-Za-z0-9/._-]+'

Examples

>>> interface.split("/")
('/', '')
>>> interface.split("/file.txt")
('/', 'file.txt')
>>> interface.split("/sub/")
('/sub/', '')
>>> interface.split("/sub//file.txt")
('/sub/', 'file.txt')