pixaris.utils package
Submodules
pixaris.utils.bigquery module
- pixaris.utils.bigquery.create_schema_from_dict(data_dict) list[SchemaField][source]
Creates a BigQuery schema from a dictionary of data.
- Parameters:
data_dict (dict) – A dictionary where keys are field names and values are field values.
- Returns:
A list of BigQuery SchemaField objects.
- Return type:
list[bigquery.SchemaField]
- pixaris.utils.bigquery.ensure_table_exists(table_ref: str, bigquery_input: dict, bigquery_client: Client)[source]
Ensures that the BigQuery table exists with the correct schema. :param table_ref: The reference to the BigQuery table. :type table_ref: str :param bigquery_input: The dictionary used to generate the schema. :type bigquery_input: dict :param bigquery_client: The BigQuery client. :type bigquery_client: bigquery.Client
pixaris.utils.hyperparameters module
- pixaris.utils.hyperparameters.expand_hyperparameters(params: list[dict])[source]
Expands a list of hyperparameters to a list of dictionaries, each containing a single value. This will be used to check if each possible hyperparameter is valid. Not using generate_hyperparameter_grid to save time and noch check hyperparameter multiple times
- Parameters:
params (list[dict]) – A list of dictionaries where each dictionary represents a hyperparameter with keys “node_name”, “input” and “value”. The “value” key should have a list of possible values for that hyperparameter.
- Returns:
list of dictionaries containing a single value. Length of the list is the sum of all values in the hyperparameters
- Return type:
list[dict]
- pixaris.utils.hyperparameters.generate_hyperparameter_grid(params: list[dict]) list[list[dict]][source]
Generates a grid of hyperparameter combinations.
- Parameters:
params (list[dict]) – A list of dictionaries where each dictionary represents a hyperparameter with keys “node_name”, “input” and “value”. The “value” key should have a list of possible values for that hyperparameter.
- Returns:
list of dictionaries containing a single value. Length of the list is the product of all values in the hyperparameters
- Return type:
list[dict]
pixaris.utils.merge_dicts module
- pixaris.utils.merge_dicts.merge_dicts(dict1: dict, dict2: dict) dict[source]
Merge two dictionaries recursively by looping through all keys and updating the value lists, so that the resulting dictionary contains all keys and values from both dictionaries.
- Parameters:
dict1 (dict) – The first dictionary to merge.
dict2 (dict) – The second dictionary to merge.
- Returns:
A new dictionary containing all keys and values from both dictionaries.
- Return type:
dict
pixaris.utils.retry module
- pixaris.utils.retry.retry(exceptions=<class 'Exception'>, tries=-1, delay=0, max_delay=None, backoff=1)[source]
Returns a retry decorator.
- Parameters:
exceptions (BaseException) – an exception or a tuple of exceptions to catch. default: Exception.
tries (int) – the maximum number of attempts. default: -1 (infinite).
delay (int) – initial delay between attempts. default: 0.
max_delay (int) – the maximum value of delay. default: None (no limit).
backoff (int) – multiplier applied to delay between attempts. default: 1 (no backoff).
- Returns:
a retry decorator.
- Return type:
decorator