digital_land.phase package
Submodules
digital_land.phase.combine module
- class digital_land.phase.combine.FactCombinePhase(issue_log=None, fields=[])
Bases:
Phasecombine a field value from multiple facts
- process(stream)
- digital_land.phase.combine.combine_geometries(wkts, precision=6)
digital_land.phase.concat module
digital_land.phase.convert module
- exception digital_land.phase.convert.ConversionError
Bases:
Exception
- class digital_land.phase.convert.ConvertPhase(path=None, dataset_resource_log=None, converted_resource_log=None, output_path=None)
Bases:
Phase- find_internal_path(input_path)
- process(stream=None)
- digital_land.phase.convert.convert_features_to_csv(input_path, output_path=None)
- digital_land.phase.convert.convert_json_to_csv(input_path, encoding, output_path=None)
- digital_land.phase.convert.detect_encoding(f)
- digital_land.phase.convert.detect_file_encoding(path)
- digital_land.phase.convert.execute(command, env=environ({'JAVA_HOME': '/usr/lib/jvm/temurin-11-jdk-amd64', 'LEIN_HOME': '/usr/local/lib/lein', 'GHCUP_INSTALL_BASE_PREFIX': '/usr/local', 'RUNNER_NAME': 'GitHub Actions 1000013608', 'GITHUB_REF_NAME': 'main', 'ACCEPT_EULA': 'Y', 'DOTNET_SKIP_FIRST_TIME_EXPERIENCE': '1', 'GITHUB_ENV': '/home/runner/work/_temp/_runner_file_commands/set_env_ac234097-43ef-4f4c-812c-295484eee4d6', 'GITHUB_REPOSITORY_ID': '244599917', 'RUNNER_TRACKING_ID': 'github_fb5475b4-f6ec-4b2c-8cac-76b0f0c6d5cf', 'NVM_DIR': '/home/runner/.nvm', 'GITHUB_REPOSITORY_OWNER': 'digital-land', 'JAVA_HOME_17_X64': '/usr/lib/jvm/temurin-17-jdk-amd64', 'GITHUB_EVENT_NAME': 'push', 'SGX_AESM_ADDR': '1', 'PWD': '/home/runner/work/digital-land-python/digital-land-python', 'GITHUB_ACTION_REPOSITORY': '', 'RUNNER_WORKSPACE': '/home/runner/work/digital-land-python', 'GITHUB_WORKFLOW_SHA': 'df068df5f825dce05ea16717108189f8e4a85953', 'RUNNER_OS': 'Linux', 'EDGEWEBDRIVER': '/usr/local/share/edge_driver', 'CI': 'true', 'LANG': 'C.UTF-8', 'RUNNER_TEMP': '/home/runner/work/_temp', 'ENABLE_RUNNER_TRACING': 'true', 'GITHUB_SERVER_URL': 'https://github.com', 'GITHUB_API_URL': 'https://api.github.com', 'MAKEFLAGS': '', 'CHROME_BIN': '/usr/bin/google-chrome', 'PKG_CONFIG_PATH': '/opt/hostedtoolcache/Python/3.9.23/x64/lib/pkgconfig', 'Python2_ROOT_DIR': '/opt/hostedtoolcache/Python/3.9.23/x64', 'GITHUB_SHA': 'df068df5f825dce05ea16717108189f8e4a85953', 'ANDROID_NDK_HOME': '/usr/local/lib/android/sdk/ndk/27.3.13750724', 'GITHUB_ACTOR_ID': '35256612', 'POWERSHELL_DISTRIBUTION_CHANNEL': 'GitHub-Actions-ubuntu22', 'SELENIUM_JAR_PATH': '/usr/share/java/selenium-server.jar', 'pythonLocation': '/opt/hostedtoolcache/Python/3.9.23/x64', 'MFLAGS': '', 'GOROOT_1_24_X64': '/opt/hostedtoolcache/go/1.24.6/x64', 'ANDROID_NDK_LATEST_HOME': '/usr/local/lib/android/sdk/ndk/27.3.13750724', 'GOROOT_1_23_X64': '/opt/hostedtoolcache/go/1.23.12/x64', 'BOOTSTRAP_HASKELL_NONINTERACTIVE': '1', 'JAVA_HOME_8_X64': '/usr/lib/jvm/temurin-8-jdk-amd64', 'PIPX_HOME': '/opt/pipx', 'GITHUB_RUN_ATTEMPT': '1', 'GITHUB_TRIGGERING_ACTOR': 'eveleighoj', 'DOTNET_NOLOGO': '1', 'DOTNET_MULTILEVEL_LOOKUP': '0', 'JAVA_HOME_11_X64': '/usr/lib/jvm/temurin-11-jdk-amd64', 'GITHUB_REF': 'refs/heads/main', 'GITHUB_RUN_NUMBER': '106', 'Python3_ROOT_DIR': '/opt/hostedtoolcache/Python/3.9.23/x64', 'ANDROID_SDK_ROOT': '/usr/local/lib/android/sdk', 'GITHUB_JOB': 'build', 'ANDROID_HOME': '/usr/local/lib/android/sdk', 'SHELL': '/bin/bash', 'LD_LIBRARY_PATH': '/opt/hostedtoolcache/Python/3.9.23/x64/lib', 'GITHUB_STATE': '/home/runner/work/_temp/_runner_file_commands/save_state_ac234097-43ef-4f4c-812c-295484eee4d6', 'SHLVL': '1', 'ANDROID_NDK': '/usr/local/lib/android/sdk/ndk/27.3.13750724', 'ImageVersion': '20250810.1.0', 'ACTIONS_ID_TOKEN_REQUEST_TOKEN': 'eyJhbGciOiJSUzI1NiIsImtpZCI6IjM4ODI2YjE3LTZhMzAtNWY5Yi1iMTY5LThiZWI4MjAyZjcyMyIsInR5cCI6IkpXVCIsIng1dCI6InlrTmFZNHFNX3RhNGsyVGdaT0NFWUxrY1lsQSJ9.eyJJZGVudGl0eVR5cGVDbGFpbSI6IlN5c3RlbTpTZXJ2aWNlSWRlbnRpdHkiLCJhYyI6Ilt7XCJTY29wZVwiOlwicmVmcy9oZWFkcy9tYWluXCIsXCJQZXJtaXNzaW9uXCI6M31dIiwiYWNzbCI6IjEwIiwiYXVkIjoidnNvOjQ3Njc5ZDcwLTZhMzgtNGViNS04MTQ4LTcxNWE5MWI3ZmVlMiIsImJpbGxpbmdfb3duZXJfaWQiOiJPX2tnRE9BcUJRV0EiLCJleHAiOjE3NTU3MjU3NjksImh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vd3MvMjAwOC8wNi9pZGVudGl0eS9jbGFpbXMvcHJpbWFyeXNpZCI6ImRkZGRkZGRkLWRkZGQtZGRkZC1kZGRkLWRkZGRkZGRkZGRkZCIsImh0dHA6Ly9zY2hlbWFzLnhtbHNvYXAub3JnL3dzLzIwMDUvMDUvaWRlbnRpdHkvY2xhaW1zL3NpZCI6ImRkZGRkZGRkLWRkZGQtZGRkZC1kZGRkLWRkZGRkZGRkZGRkZCIsImlhdCI6MTc1NTcwMzU2OSwiaXNzIjoiaHR0cHM6Ly90b2tlbi5hY3Rpb25zLmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImpvYl9pZCI6IjNiMWI4OTMyLTBkY2YtNWM4ZC05Y2U4LTJlNzE1YzVhOWZmMiIsIm5hbWVpZCI6ImRkZGRkZGRkLWRkZGQtZGRkZC1kZGRkLWRkZGRkZGRkZGRkZCIsIm5iZiI6MTc1NTcwMzI2OSwib2lkY19leHRyYSI6IntcImFjdG9yXCI6XCJldmVsZWlnaG9qXCIsXCJhY3Rvcl9pZFwiOlwiMzUyNTY2MTJcIixcImJhc2VfcmVmXCI6XCJcIixcImV2ZW50X25hbWVcIjpcInB1c2hcIixcImhlYWRfcmVmXCI6XCJcIixcImpvYl93b3JrZmxvd19yZWZcIjpcImRpZ2l0YWwtbGFuZC9kaWdpdGFsLWxhbmQtcHl0aG9uLy5naXRodWIvd29ya2Zsb3dzL2RlcGxveS1kb2N1bWVudGF0aW9uLnltbEByZWZzL2hlYWRzL21haW5cIixcImpvYl93b3JrZmxvd19zaGFcIjpcImRmMDY4ZGY1ZjgyNWRjZTA1ZWExNjcxNzEwODE4OWY4ZTRhODU5NTNcIixcInJlZlwiOlwicmVmcy9oZWFkcy9tYWluXCIsXCJyZWZfcHJvdGVjdGVkXCI6XCJ0cnVlXCIsXCJyZWZfdHlwZVwiOlwiYnJhbmNoXCIsXCJyZXBvc2l0b3J5XCI6XCJkaWdpdGFsLWxhbmQvZGlnaXRhbC1sYW5kLXB5dGhvblwiLFwicmVwb3NpdG9yeV9pZFwiOlwiMjQ0NTk5OTE3XCIsXCJyZXBvc2l0b3J5X293bmVyXCI6XCJkaWdpdGFsLWxhbmRcIixcInJlcG9zaXRvcnlfb3duZXJfaWRcIjpcIjQ0MDYwNzYwXCIsXCJyZXBvc2l0b3J5X3Zpc2liaWxpdHlcIjpcInB1YmxpY1wiLFwicnVuX2F0dGVtcHRcIjpcIjFcIixcInJ1bl9pZFwiOlwiMTcxMDI5MjUxMjBcIixcInJ1bl9udW1iZXJcIjpcIjEwNlwiLFwicnVubmVyX2Vudmlyb25tZW50XCI6XCJnaXRodWItaG9zdGVkXCIsXCJzaGFcIjpcImRmMDY4ZGY1ZjgyNWRjZTA1ZWExNjcxNzEwODE4OWY4ZTRhODU5NTNcIixcIndvcmtmbG93XCI6XCJEZXBsb3kgRG9jdW1lbnRhdGlvblwiLFwid29ya2Zsb3dfcmVmXCI6XCJkaWdpdGFsLWxhbmQvZGlnaXRhbC1sYW5kLXB5dGhvbi8uZ2l0aHViL3dvcmtmbG93cy9kZXBsb3ktZG9jdW1lbnRhdGlvbi55bWxAcmVmcy9oZWFkcy9tYWluXCIsXCJ3b3JrZmxvd19zaGFcIjpcImRmMDY4ZGY1ZjgyNWRjZTA1ZWExNjcxNzEwODE4OWY4ZTRhODU5NTNcIn0iLCJvaWRjX3N1YiI6InJlcG86ZGlnaXRhbC1sYW5kL2RpZ2l0YWwtbGFuZC1weXRob246cmVmOnJlZnMvaGVhZHMvbWFpbiIsIm9yY2hfaWQiOiI4ZDliNGI2NC1jMjhmLTRhOTUtOTJmNS1hMmI5ZmUyNTZhN2QuYnVpbGQuX19kZWZhdWx0Iiwib3duZXJfaWQiOiJPX2tnRE9BcUJRV0EiLCJwbGFuX2lkIjoiOGQ5YjRiNjQtYzI4Zi00YTk1LTkyZjUtYTJiOWZlMjU2YTdkIiwicmVwb3NpdG9yeV9pZCI6IjI0NDU5OTkxNyIsInJ1bl9pZCI6IjE3MTAyOTI1MTIwIiwicnVuX251bWJlciI6IjEwNiIsInJ1bl90eXBlIjoiZnVsbCIsInJ1bm5lcl9pZCI6IjEwMDAwMTM2MDgiLCJydW5uZXJfdHlwZSI6Imhvc3RlZCIsInNjcCI6IkFjdGlvbnMuUmVzdWx0czo4ZDliNGI2NC1jMjhmLTRhOTUtOTJmNS1hMmI5ZmUyNTZhN2Q6M2IxYjg5MzItMGRjZi01YzhkLTljZTgtMmU3MTVjNWE5ZmYyIEFjdGlvbnMuUnVubmVyOjhkOWI0YjY0LWMyOGYtNGE5NS05MmY1LWEyYjlmZTI1NmE3ZDozYjFiODkzMi0wZGNmLTVjOGQtOWNlOC0yZTcxNWM1YTlmZjIgQWN0aW9ucy5VcGxvYWRBcnRpZmFjdHM6OGQ5YjRiNjQtYzI4Zi00YTk1LTkyZjUtYTJiOWZlMjU2YTdkOjNiMWI4OTMyLTBkY2YtNWM4ZC05Y2U4LTJlNzE1YzVhOWZmMiBnZW5lcmF0ZV9pZF90b2tlbjo4ZDliNGI2NC1jMjhmLTRhOTUtOTJmNS1hMmI5ZmUyNTZhN2Q6M2IxYjg5MzItMGRjZi01YzhkLTljZTgtMmU3MTVjNWE5ZmYyIEFjdGlvbnMuR2VuZXJpY1JlYWQ6MDAwMDAwMDAtMDAwMC0wMDAwLTAwMDAtMDAwMDAwMDAwMDAwIiwic2hhIjoiZGYwNjhkZjVmODI1ZGNlMDVlYTE2NzE3MTA4MTg5ZjhlNGE4NTk1MyJ9.jYi7DDq6npGFB3oR7BXIEc7kt9sErYfJzvu8R2CEQu20bRotIJFztkMGRQ6bSZd6g91VhiDWiErYaRlyr3-lJV93zJdr99volbQHz_1IjPkgKdcY3dicAbPHI6pzB6n1v3AlTdvPydSHbx5hfjsHfMMOswgjZjbQmYQLxDVHt5-rSlS_BNl5eFxC_fPt8GrvfZqSmsnfoGTy873hLQ5xAQ37rMvuP5yr9oIDD_iCvI00fm-LAhgSr0Fn_4uqkbVRYkfjMGN7_fHlXiZEqVh1ZMnYptit9H9jx83yi66xRTxTCttlyDsaMBj66-CrEmOw58QUQZiTMbPBWgpmLhbrsA', 'PATH': '/opt/hostedtoolcache/Python/3.9.23/x64/bin:/opt/hostedtoolcache/Python/3.9.23/x64:/snap/bin:/home/runner/.local/bin:/opt/pipx_bin:/home/runner/.cargo/bin:/home/runner/.config/composer/vendor/bin:/usr/local/.ghcup/bin:/home/runner/.dotnet/tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin', 'LEIN_JAR': '/usr/local/lib/lein/self-installs/leiningen-2.11.2-standalone.jar', 'GITHUB_EVENT_PATH': '/home/runner/work/_temp/_github_workflow/event.json', 'GRADLE_HOME': '/usr/share/gradle-9.0.0', 'GITHUB_RETENTION_DAYS': '90', 'XDG_CONFIG_HOME': '/home/runner/.config', 'INVOCATION_ID': 'f3b9f9d0cafe4e06ad62613c1d00c665', 'GITHUB_ACTIONS': 'true', 'GITHUB_WORKFLOW_REF': 'digital-land/digital-land-python/.github/workflows/deploy-documentation.yml@refs/heads/main', 'ACTIONS_ID_TOKEN_REQUEST_URL': 'https://run-actions-3-azure-eastus.actions.githubusercontent.com/7//idtoken/8d9b4b64-c28f-4a95-92f5-a2b9fe256a7d/3b1b8932-0dcf-5c8d-9ce8-2e715c5a9ff2?api-version=2.0', 'CONDA': '/usr/share/miniconda', 'GITHUB_HEAD_REF': '', 'CHROMEWEBDRIVER': '/usr/local/share/chromedriver-linux64', 'PIPX_BIN_DIR': '/opt/pipx_bin', 'GITHUB_REF_PROTECTED': 'true', 'GITHUB_REF_TYPE': 'branch', 'LOGNAME': 'runner', 'GECKOWEBDRIVER': '/usr/local/share/gecko_driver', 'SYSTEMD_EXEC_PID': '1719', 'GITHUB_ACTION_REF': '', 'GITHUB_WORKSPACE': '/home/runner/work/digital-land-python/digital-land-python', 'USER': 'runner', 'SWIFT_PATH': '/usr/share/swift/usr/bin', 'GITHUB_STEP_SUMMARY': '/home/runner/work/_temp/_runner_file_commands/step_summary_ac234097-43ef-4f4c-812c-295484eee4d6', 'RUNNER_ARCH': 'X64', 'GITHUB_PATH': '/home/runner/work/_temp/_runner_file_commands/add_path_ac234097-43ef-4f4c-812c-295484eee4d6', 'RUNNER_TOOL_CACHE': '/opt/hostedtoolcache', '_': '/usr/bin/make', 'ANDROID_NDK_ROOT': '/usr/local/lib/android/sdk/ndk/27.3.13750724', 'XDG_RUNTIME_DIR': '/run/user/1001', 'AGENT_TOOLSDIRECTORY': '/opt/hostedtoolcache', 'GITHUB_REPOSITORY': 'digital-land/digital-land-python', 'VCPKG_INSTALLATION_ROOT': '/usr/local/share/vcpkg', 'JOURNAL_STREAM': '8:13022', 'GITHUB_ACTION': '__run_2', 'DEBIAN_FRONTEND': 'noninteractive', 'HOME': '/home/runner', 'GITHUB_GRAPHQL_URL': 'https://api.github.com/graphql', 'GITHUB_WORKFLOW': 'Deploy Documentation', 'GITHUB_RUN_ID': '17102925120', 'HOMEBREW_CLEANUP_PERIODIC_FULL_DAYS': '3650', 'GITHUB_ACTOR': 'eveleighoj', 'ANT_HOME': '/usr/share/ant', 'ImageOS': 'ubuntu22', 'Python_ROOT_DIR': '/opt/hostedtoolcache/Python/3.9.23/x64', 'ACTIONS_RUNNER_ACTION_ARCHIVE_CACHE': '/opt/actionarchivecache', 'GITHUB_OUTPUT': '/home/runner/work/_temp/_runner_file_commands/set_output_ac234097-43ef-4f4c-812c-295484eee4d6', 'GITHUB_BASE_REF': '', 'GITHUB_REPOSITORY_OWNER_ID': '44060760', 'GOROOT_1_22_X64': '/opt/hostedtoolcache/go/1.22.12/x64', 'AZURE_EXTENSION_DIR': '/opt/az/azcliextensions', 'RUNNER_ENVIRONMENT': 'github-hosted', 'HOMEBREW_NO_AUTO_UPDATE': '1', 'JAVA_HOME_21_X64': '/usr/lib/jvm/temurin-21-jdk-amd64', 'MAKELEVEL': '1', 'DOCUTILSCONFIG': '/home/runner/work/digital-land-python/digital-land-python/docs/docutils.conf'}))
- digital_land.phase.convert.load_csv(path, encoding='UTF-8', log=None)
- digital_land.phase.convert.read_csv(input_path, encoding='utf-8')
- digital_land.phase.convert.read_excel(path)
- digital_land.phase.convert.save_efficient_json_as_csv(output_path, columns, data)
digital_land.phase.default module
digital_land.phase.dump module
- class digital_land.phase.dump.DumpPhase(path=None, f=None, enabled=True)
Bases:
Phasedump stream lines to a file
- process(stream)
- digital_land.phase.dump.dump(stream, path=None, f=None)
digital_land.phase.factor module
- class digital_land.phase.factor.FactorPhase
Bases:
Phaseadd a fact hash identifier
- process(stream)
- digital_land.phase.factor.fact_hash(entity, field, value)
digital_land.phase.filter module
digital_land.phase.harmonise module
digital_land.phase.load module
digital_land.phase.lookup module
- class digital_land.phase.lookup.EntityLookupPhase(lookups={}, redirect_lookups={}, issue_log=None, operational_issue_log=None, entity_range=[])
Bases:
LookupPhase- entity_field = 'entity'
- process(stream)
- class digital_land.phase.lookup.FactLookupPhase(lookups={}, redirect_lookups={}, issue_log=None, odp_collections=[])
Bases:
LookupPhase- process(stream)
- class digital_land.phase.lookup.LookupPhase(lookups={}, redirect_lookups={}, issue_log=None, operational_issue_log=None, entity_range=[])
Bases:
Phaselookup entity numbers by CURIE
- build_reverse_lookups()
- check_associated_organisation(entity)
- entity_field = None
- get_entity(block)
- lookup(**kwargs)
- process(stream)
- redirect_entity(entity)
Given an entity number can check the redirect lookups to see if the entity has been removed or redirected.
- class digital_land.phase.lookup.PrintLookupPhase(lookups={}, redirect_lookups={})
Bases:
LookupPhase- process(stream)
- digital_land.phase.lookup.key(entry_number='', prefix='', reference='', organisation='')
- digital_land.phase.lookup.normalise(value)
digital_land.phase.map module
- class digital_land.phase.map.MapPhase(fieldnames, columns={}, log=None)
Bases:
Phaserename field names using the provided column map
- headers(fieldnames)
- log_headers(headers)
- process(stream)
- digital_land.phase.map.normalise(name)
digital_land.phase.migrate module
digital_land.phase.normalise module
- class digital_land.phase.normalise.NormalisePhase(skip_patterns=[], null_path=None)
Bases:
Phase- normalise_whitespace(row)
- null_path = '/home/runner/work/digital-land-python/digital-land-python/digital_land/phase/../patch/null.csv'
- null_patterns = []
- process(stream)
- skip(row)
- skip_patterns = []
- spaces = ' \n\r\t\x0c'
- strip_nulls(row)
digital_land.phase.organisation module
digital_land.phase.parse module
digital_land.phase.patch module
digital_land.phase.phase module
digital_land.phase.pivot module
digital_land.phase.prefix module
digital_land.phase.priority module
digital_land.phase.prune module
- class digital_land.phase.prune.EntityPrunePhase(issue_log=None, dataset_resource_log=None)
Bases:
Phaseremove entries with a missing entity
- process(stream)
digital_land.phase.reference module
- class digital_land.phase.reference.EntityReferencePhase(dataset=None, prefix=None, specification=None, issues=None)
Bases:
Phaseensure an entry has the prefix and reference fields
- process(stream)
- process_row(row)
- class digital_land.phase.reference.FactReferencePhase(field_typology_map=None, field_prefix_map=None, dataset=None, specification=None)
Bases:
EntityReferencePhaseensure a fact which is a reference has a prefix and reference field
- get_field_prefix(field_name)
- get_field_typology_name(field_name)
- process_row(row)
- digital_land.phase.reference.split_curie(value)
digital_land.phase.save module
- class digital_land.phase.save.SavePhase(path=None, f=None, fieldnames=None, enabled=True)
Bases:
Phasesave stream rows to a file
- process(stream)
- digital_land.phase.save.save(stream, path=None, fieldnames=None, f=None)