digital_land.expectations.checkpoints package
Submodules
digital_land.expectations.checkpoints.base module
- class digital_land.expectations.checkpoints.base.BaseCheckpoint
Bases:
ABC- abstract load()
filled in by child classes, ensures a config is loaded correctly should raise error if not
- abstract run()
- abstract save(output_dir, format='csv')
filled in by child classes, uses save functions to save the data. could add default behaviour at somepoint
digital_land.expectations.checkpoints.dataset module
- class digital_land.expectations.checkpoints.dataset.DatasetCheckpoint(dataset, file_path, organisations: Organisation)
Bases:
BaseCheckpoint- get_rule_orgs(rule: dict) list
for each rule we need to get a list of the organisations that the rule applies to this is a semi colon separated list of individual orgs, org datasets or org prefixes which are a key of the inputted dict
- Parameters:
rule (-) -- a single expectation rule
- load(rules)
given a set of rules this function loads them into the checkpoint for the dataset checkpoint we antiipates rules contain organisations with which expectations need parsing
- operation_factory(operation_string: str)
conevrts a string into an operation, available operations are specific to the checkpoint
- Args
operation: a string representing an operation
- parse_rule(rule, org=None) dict
turn a rule into an expectation given an org it will format text strings using jinja templating
- run()
run the set of expectations that have been loaded into the checkpoint results will be stoed in the log and can be saved used .save
- run_expectation(expectation) tuple
runs a given expectation returning the result, description and message a log can be provided to record this information
- save(output_dir: Path)
save the outputs as a file, the file is named based the the dataset and stored in the provided directory