digital_land.utils package

Submodules

digital_land.utils.add_data_utils module

digital_land.utils.add_data_utils.clear_log(collection_dir, endpoint)

digital_land.utils.add_data_utils.download_dataset(dataset, specification, cache_dir)

digital_land.utils.add_data_utils.get_column_field_summary(dataset, endpoint_resource_info, column_field_dir, converted_dir, specification_dir, pipeline_dir)

digital_land.utils.add_data_utils.get_entity_summary(endpoint_resource_info, output_path, pipeline, issue_dir, pipeline_dir)

digital_land.utils.add_data_utils.get_existing_endpoints_summary(endpoint_resource_info, collection, dataset)

digital_land.utils.add_data_utils.get_issue_summary(endpoint_resource_info, issue_dir, new_entities=None)

digital_land.utils.add_data_utils.get_provision_entities_from_duckdb(lookup_path, pipeline, endpoint_resource_info)

digital_land.utils.add_data_utils.get_transformed_entities(dataset_path, transformed_path)

Returns a Dataframe of entities from a dataset. It returns entities that have facts in the transformed file at transformed_path

digital_land.utils.add_data_utils.get_updated_entities_summary(original_entity_df, updated_entity_df)

This will return a summary of the differences between two dataframes of the same entities

digital_land.utils.add_data_utils.get_user_response(message)

digital_land.utils.add_data_utils.is_date_valid(date, date_type)

digital_land.utils.add_data_utils.is_url_valid(url, url_type)

digital_land.utils.add_data_utils.normalise_json(val)

Returns a sorted stringified json

digital_land.utils.add_endpoints_utils module

digital_land.utils.add_endpoints_utils.task_preprocess(ctx)

preparatory steps to tidy up previous runs, and populate the context :param ctx: :return:

digital_land.utils.gdal_utils module

digital_land.utils.gdal_utils.get_gdal_version()

Module contents