pixaris.data_loaders package

Submodules

pixaris.data_loaders.base module

class pixaris.data_loaders.base.DatasetLoader[source]

Bases: object

When implementing a new Dataset Loader, inherit from this one and implement all the abstract methods.

abstractmethod load_dataset() → Iterable[dict[str, any]][source]

pixaris.data_loaders.gcp module

class pixaris.data_loaders.gcp.GCPDatasetLoader(gcp_project_id: str, gcp_pixaris_bucket_name: str, project: str, dataset: str, eval_dir_local: str = 'local_experiment_inputs', force_download: bool = True)[source]

Bases: DatasetLoader

GCPDatasetLoader is a class for loading datasets from a Google Cloud Storage bucket. Upon initialisation, the dataset is downloaded to a local directory.

Parameters:

gcp_project_id (str) – The Google Cloud Platform project ID.
gcp_pixaris_bucket_name (str) – The name of the Google Cloud Storage bucket.
project (str) – The name of the project containing the evaluation set.
dataset (str) – The name of the evaluation set to download images for.
eval_dir_local (str) – The local directory where evaluation images will be saved. Defaults to “local_experiment_inputs”.
force_download (bool) – Whether to force download the images even if they already exist locally. Defaults to True.

load_dataset() → List[dict[str, List[dict[str, Image]]]][source]

Returns all images in the evaluation set as an iterator of dictionaries containing PIL Images.

Returns:: list of dicts containing data loaded from the bucket. The key will always be “pillow_images”. The value is a dict mapping node names to PIL Image objects. This dict has a key for each directory in the image_dirs list representing a Node Name.
Return type:: List[dict[str, List[dict[str, Image.Image]]]]:

pixaris.data_loaders.local module

class pixaris.data_loaders.local.LocalDatasetLoader(project: str, dataset: str, eval_dir_local: str = 'local_experiment_inputs')[source]