digital_land.phase package
Submodules
digital_land.phase.combine module
- class digital_land.phase.combine.FactCombinePhase(issue_log=None, fields=[])
Bases:
Phasecombine a field value from multiple facts
- process(stream)
- digital_land.phase.combine.combine_geometries(wkts, precision=6)
digital_land.phase.concat module
digital_land.phase.convert module
- exception digital_land.phase.convert.ConversionError
Bases:
Exception
- class digital_land.phase.convert.ConvertPhase(path=None, dataset_resource_log=None, converted_resource_log=None, output_path=None)
Bases:
Phase- find_internal_path(input_path)
- process(stream=None)
- digital_land.phase.convert.convert_features_to_csv(input_path, output_path=None)
- digital_land.phase.convert.convert_json_to_csv(input_path, encoding, output_path=None)
- digital_land.phase.convert.detect_encoding(f)
- digital_land.phase.convert.detect_file_encoding(path)
- digital_land.phase.convert.execute(command, env=environ({'JAVA_HOME': '/usr/lib/jvm/temurin-11-jdk-amd64', 'LEIN_HOME': '/usr/local/lib/lein', 'GHCUP_INSTALL_BASE_PREFIX': '/usr/local', 'RUNNER_NAME': 'GitHub Actions 1000028806', 'GITHUB_REF_NAME': 'main', 'ACCEPT_EULA': 'Y', 'DOTNET_SKIP_FIRST_TIME_EXPERIENCE': '1', 'GITHUB_ENV': '/home/runner/work/_temp/_runner_file_commands/set_env_f1fbb62c-8480-4c50-854f-fdfb47e54140', 'GITHUB_REPOSITORY_ID': '244599917', 'RUNNER_TRACKING_ID': 'github_06578217-e927-4e3a-913e-500c71ac37a0', 'NVM_DIR': '/home/runner/.nvm', 'GITHUB_REPOSITORY_OWNER': 'digital-land', 'JAVA_HOME_17_X64': '/usr/lib/jvm/temurin-17-jdk-amd64', 'GITHUB_EVENT_NAME': 'push', 'SGX_AESM_ADDR': '1', 'PWD': '/home/runner/work/digital-land-python/digital-land-python', 'GITHUB_ACTION_REPOSITORY': '', 'RUNNER_WORKSPACE': '/home/runner/work/digital-land-python', 'GITHUB_WORKFLOW_SHA': 'c724877fc5e832c67e840c7bd8740247152ff7c6', 'RUNNER_OS': 'Linux', 'ACTIONS_ORCHESTRATION_ID': '851acdd4-8279-4c35-8ff5-e0a23cfd12ad.build.__default', 'EDGEWEBDRIVER': '/usr/local/share/edge_driver', 'CI': 'true', 'LANG': 'C.UTF-8', 'RUNNER_TEMP': '/home/runner/work/_temp', 'ENABLE_RUNNER_TRACING': 'true', 'GITHUB_SERVER_URL': 'https://github.com', 'GITHUB_API_URL': 'https://api.github.com', 'MAKEFLAGS': '', 'CHROME_BIN': '/usr/bin/google-chrome', 'PKG_CONFIG_PATH': '/opt/hostedtoolcache/Python/3.9.25/x64/lib/pkgconfig', 'Python2_ROOT_DIR': '/opt/hostedtoolcache/Python/3.9.25/x64', 'GITHUB_SHA': 'c724877fc5e832c67e840c7bd8740247152ff7c6', 'ANDROID_NDK_HOME': '/usr/local/lib/android/sdk/ndk/27.3.13750724', 'GITHUB_ACTOR_ID': '255134076', 'POWERSHELL_DISTRIBUTION_CHANNEL': 'GitHub-Actions-ubuntu22', 'SELENIUM_JAR_PATH': '/usr/share/java/selenium-server.jar', 'pythonLocation': '/opt/hostedtoolcache/Python/3.9.25/x64', 'MFLAGS': '', 'GOROOT_1_24_X64': '/opt/hostedtoolcache/go/1.24.13/x64', 'PSModulePath': '/root/.local/share/powershell/Modules:/usr/local/share/powershell/Modules:/opt/microsoft/powershell/7/Modules:/usr/share/az_14.6.0', 'ANDROID_NDK_LATEST_HOME': '/usr/local/lib/android/sdk/ndk/29.0.14206865', 'GOROOT_1_23_X64': '/opt/hostedtoolcache/go/1.23.12/x64', 'BOOTSTRAP_HASKELL_NONINTERACTIVE': '1', 'JAVA_HOME_8_X64': '/usr/lib/jvm/temurin-8-jdk-amd64', 'PIPX_HOME': '/opt/pipx', 'GITHUB_RUN_ATTEMPT': '1', 'GITHUB_TRIGGERING_ACTOR': 'MBR-COG', 'DOTNET_NOLOGO': '1', 'DOTNET_MULTILEVEL_LOOKUP': '0', 'JAVA_HOME_11_X64': '/usr/lib/jvm/temurin-11-jdk-amd64', 'GITHUB_REF': 'refs/heads/main', 'GITHUB_RUN_NUMBER': '135', 'Python3_ROOT_DIR': '/opt/hostedtoolcache/Python/3.9.25/x64', 'ANDROID_SDK_ROOT': '/usr/local/lib/android/sdk', 'GITHUB_JOB': 'build', 'ANDROID_HOME': '/usr/local/lib/android/sdk', 'SHELL': '/bin/bash', 'LD_LIBRARY_PATH': '/opt/hostedtoolcache/Python/3.9.25/x64/lib', 'GITHUB_STATE': '/home/runner/work/_temp/_runner_file_commands/save_state_f1fbb62c-8480-4c50-854f-fdfb47e54140', 'SHLVL': '1', 'ANDROID_NDK': '/usr/local/lib/android/sdk/ndk/27.3.13750724', 'ImageVersion': '20260209.31.1', 'ACTIONS_ID_TOKEN_REQUEST_TOKEN': 'eyJhbGciOiJSUzI1NiIsImtpZCI6IjM4ODI2YjE3LTZhMzAtNWY5Yi1iMTY5LThiZWI4MjAyZjcyMyIsInR5cCI6IkpXVCIsIng1dCI6InlrTmFZNHFNX3RhNGsyVGdaT0NFWUxrY1lsQSJ9.eyJJZGVudGl0eVR5cGVDbGFpbSI6IlN5c3RlbTpTZXJ2aWNlSWRlbnRpdHkiLCJhYyI6Ilt7XCJTY29wZVwiOlwicmVmcy9oZWFkcy9tYWluXCIsXCJQZXJtaXNzaW9uXCI6M31dIiwiYWNzbCI6IjEwIiwiYXVkIjoidnNvOjQ3Njc5ZDcwLTZhMzgtNGViNS04MTQ4LTcxNWE5MWI3ZmVlMiIsImJpbGxpbmdfb3duZXJfaWQiOiJPX2tnRE9BcUJRV0EiLCJleHAiOjE3NzEzNjU5MjgsImh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vd3MvMjAwOC8wNi9pZGVudGl0eS9jbGFpbXMvcHJpbWFyeXNpZCI6ImRkZGRkZGRkLWRkZGQtZGRkZC1kZGRkLWRkZGRkZGRkZGRkZCIsImh0dHA6Ly9zY2hlbWFzLnhtbHNvYXAub3JnL3dzLzIwMDUvMDUvaWRlbnRpdHkvY2xhaW1zL3NpZCI6ImRkZGRkZGRkLWRkZGQtZGRkZC1kZGRkLWRkZGRkZGRkZGRkZCIsImlhdCI6MTc3MTM0MzcyOCwiaXNzIjoiaHR0cHM6Ly90b2tlbi5hY3Rpb25zLmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImpvYl9pZCI6IjRiMDQ0Nzk2LTFjMjItNWI0YS1iZTgyLWVlOTAwNjhjMDY0OSIsIm5hbWVpZCI6ImRkZGRkZGRkLWRkZGQtZGRkZC1kZGRkLWRkZGRkZGRkZGRkZCIsIm5iZiI6MTc3MTM0MzQyOCwib2lkY19leHRyYSI6IntcImFjdG9yXCI6XCJNQlItQ09HXCIsXCJhY3Rvcl9pZFwiOlwiMjU1MTM0MDc2XCIsXCJiYXNlX3JlZlwiOlwiXCIsXCJjaGVja19ydW5faWRcIjpcIjYzODg2ODY1MTYwXCIsXCJldmVudF9uYW1lXCI6XCJwdXNoXCIsXCJoZWFkX3JlZlwiOlwiXCIsXCJqb2Jfd29ya2Zsb3dfcmVmXCI6XCJkaWdpdGFsLWxhbmQvZGlnaXRhbC1sYW5kLXB5dGhvbi8uZ2l0aHViL3dvcmtmbG93cy9kZXBsb3ktZG9jdW1lbnRhdGlvbi55bWxAcmVmcy9oZWFkcy9tYWluXCIsXCJqb2Jfd29ya2Zsb3dfc2hhXCI6XCJjNzI0ODc3ZmM1ZTgzMmM2N2U4NDBjN2JkODc0MDI0NzE1MmZmN2M2XCIsXCJyZWZcIjpcInJlZnMvaGVhZHMvbWFpblwiLFwicmVmX3Byb3RlY3RlZFwiOlwidHJ1ZVwiLFwicmVmX3R5cGVcIjpcImJyYW5jaFwiLFwicmVwb3NpdG9yeVwiOlwiZGlnaXRhbC1sYW5kL2RpZ2l0YWwtbGFuZC1weXRob25cIixcInJlcG9zaXRvcnlfaWRcIjpcIjI0NDU5OTkxN1wiLFwicmVwb3NpdG9yeV9vd25lclwiOlwiZGlnaXRhbC1sYW5kXCIsXCJyZXBvc2l0b3J5X293bmVyX2lkXCI6XCI0NDA2MDc2MFwiLFwicmVwb3NpdG9yeV92aXNpYmlsaXR5XCI6XCJwdWJsaWNcIixcInJ1bl9hdHRlbXB0XCI6XCIxXCIsXCJydW5faWRcIjpcIjIyMTA1NDg2NDgzXCIsXCJydW5fbnVtYmVyXCI6XCIxMzVcIixcInJ1bm5lcl9lbnZpcm9ubWVudFwiOlwiZ2l0aHViLWhvc3RlZFwiLFwic2hhXCI6XCJjNzI0ODc3ZmM1ZTgzMmM2N2U4NDBjN2JkODc0MDI0NzE1MmZmN2M2XCIsXCJ3b3JrZmxvd1wiOlwiRGVwbG95IERvY3VtZW50YXRpb25cIixcIndvcmtmbG93X3JlZlwiOlwiZGlnaXRhbC1sYW5kL2RpZ2l0YWwtbGFuZC1weXRob24vLmdpdGh1Yi93b3JrZmxvd3MvZGVwbG95LWRvY3VtZW50YXRpb24ueW1sQHJlZnMvaGVhZHMvbWFpblwiLFwid29ya2Zsb3dfc2hhXCI6XCJjNzI0ODc3ZmM1ZTgzMmM2N2U4NDBjN2JkODc0MDI0NzE1MmZmN2M2XCJ9Iiwib2lkY19zdWIiOiJyZXBvOmRpZ2l0YWwtbGFuZC9kaWdpdGFsLWxhbmQtcHl0aG9uOnJlZjpyZWZzL2hlYWRzL21haW4iLCJvcmNoX2lkIjoiODUxYWNkZDQtODI3OS00YzM1LThmZjUtZTBhMjNjZmQxMmFkLmJ1aWxkLl9fZGVmYXVsdCIsIm93bmVyX2lkIjoiT19rZ0RPQXFCUVdBIiwicGxhbl9pZCI6Ijg1MWFjZGQ0LTgyNzktNGMzNS04ZmY1LWUwYTIzY2ZkMTJhZCIsInJlcG9zaXRvcnlfaWQiOiIyNDQ1OTk5MTciLCJydW5faWQiOiIyMjEwNTQ4NjQ4MyIsInJ1bl9udW1iZXIiOiIxMzUiLCJydW5fdHlwZSI6ImZ1bGwiLCJydW5uZXJfaWQiOiIxMDAwMDI4ODA2IiwicnVubmVyX3R5cGUiOiJob3N0ZWQiLCJzY3AiOiJBY3Rpb25zLlJlc3VsdHM6ODUxYWNkZDQtODI3OS00YzM1LThmZjUtZTBhMjNjZmQxMmFkOjRiMDQ0Nzk2LTFjMjItNWI0YS1iZTgyLWVlOTAwNjhjMDY0OSBBY3Rpb25zLlJ1bm5lcjo4NTFhY2RkNC04Mjc5LTRjMzUtOGZmNS1lMGEyM2NmZDEyYWQ6NGIwNDQ3OTYtMWMyMi01YjRhLWJlODItZWU5MDA2OGMwNjQ5IEFjdGlvbnMuVXBsb2FkQXJ0aWZhY3RzOjg1MWFjZGQ0LTgyNzktNGMzNS04ZmY1LWUwYTIzY2ZkMTJhZDo0YjA0NDc5Ni0xYzIyLTViNGEtYmU4Mi1lZTkwMDY4YzA2NDkgZ2VuZXJhdGVfaWRfdG9rZW46ODUxYWNkZDQtODI3OS00YzM1LThmZjUtZTBhMjNjZmQxMmFkOjRiMDQ0Nzk2LTFjMjItNWI0YS1iZTgyLWVlOTAwNjhjMDY0OSBBY3Rpb25zLkdlbmVyaWNSZWFkOjAwMDAwMDAwLTAwMDAtMDAwMC0wMDAwLTAwMDAwMDAwMDAwMCIsInNoYSI6ImM3MjQ4NzdmYzVlODMyYzY3ZTg0MGM3YmQ4NzQwMjQ3MTUyZmY3YzYiLCJ0cnVzdF90aWVyIjoiMSJ9.HCSpq82msZkWyUdVBf9IlBfSISeRLRD1OD96JdY9X2YPu3vv6MtzxfxUKY6Wk1GeQh_PsvDUwOQw2Z03x6GK5GlHvFU6VvwtsbJWnjczhpt0TP3akFwYVCk5iZc_0K_kGNLaoSocc09qDSeVI_r4w_oHIIIjPHirkLxNatHtu4fIF8oi0auNQq5EC3ijHdI3hS6J5Asihrhh4o4Yc1JeNMTv9APGlArICvlLR8VCtgIo8nPkkIOFSHimwibAQVKNTEAcy9TI72zJV66-Ru0ng7UBrdK8m26VTbruwsIP2MvLIxt-WGO1g9nVLn3_Okq6thVNzB5PTe2QzHJ5C0w3mg', 'PATH': '/opt/hostedtoolcache/Python/3.9.25/x64/bin:/opt/hostedtoolcache/Python/3.9.25/x64:/snap/bin:/home/runner/.local/bin:/opt/pipx_bin:/home/runner/.cargo/bin:/home/runner/.config/composer/vendor/bin:/usr/local/.ghcup/bin:/home/runner/.dotnet/tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin', 'LEIN_JAR': '/usr/local/lib/lein/self-installs/leiningen-2.12.0-standalone.jar', 'GITHUB_EVENT_PATH': '/home/runner/work/_temp/_github_workflow/event.json', 'GRADLE_HOME': '/usr/share/gradle-9.3.1', 'GITHUB_RETENTION_DAYS': '90', 'XDG_CONFIG_HOME': '/home/runner/.config', 'INVOCATION_ID': '8899f0f7e04a4850b75af10a17a6d04b', 'GITHUB_ACTIONS': 'true', 'GITHUB_WORKFLOW_REF': 'digital-land/digital-land-python/.github/workflows/deploy-documentation.yml@refs/heads/main', 'ACTIONS_ID_TOKEN_REQUEST_URL': 'https://run-actions-1-azure-eastus.actions.githubusercontent.com/7//idtoken/851acdd4-8279-4c35-8ff5-e0a23cfd12ad/4b044796-1c22-5b4a-be82-ee90068c0649?api-version=2.0', 'CONDA': '/usr/share/miniconda', 'GITHUB_HEAD_REF': '', 'CHROMEWEBDRIVER': '/usr/local/share/chromedriver-linux64', 'PIPX_BIN_DIR': '/opt/pipx_bin', 'GITHUB_REF_PROTECTED': 'true', 'GITHUB_REF_TYPE': 'branch', 'LOGNAME': 'runner', 'GECKOWEBDRIVER': '/usr/local/share/gecko_driver', 'SYSTEMD_EXEC_PID': '1717', 'GITHUB_ACTION_REF': '', 'GITHUB_WORKSPACE': '/home/runner/work/digital-land-python/digital-land-python', 'USER': 'runner', 'SWIFT_PATH': '/usr/share/swift/usr/bin', 'GITHUB_STEP_SUMMARY': '/home/runner/work/_temp/_runner_file_commands/step_summary_f1fbb62c-8480-4c50-854f-fdfb47e54140', 'RUNNER_ARCH': 'X64', 'GITHUB_PATH': '/home/runner/work/_temp/_runner_file_commands/add_path_f1fbb62c-8480-4c50-854f-fdfb47e54140', 'RUNNER_TOOL_CACHE': '/opt/hostedtoolcache', '_': '/usr/bin/make', 'ANDROID_NDK_ROOT': '/usr/local/lib/android/sdk/ndk/27.3.13750724', 'XDG_RUNTIME_DIR': '/run/user/1001', 'JAVA_HOME_25_X64': '/usr/lib/jvm/temurin-25-jdk-amd64', 'AGENT_TOOLSDIRECTORY': '/opt/hostedtoolcache', 'GITHUB_REPOSITORY': 'digital-land/digital-land-python', 'VCPKG_INSTALLATION_ROOT': '/usr/local/share/vcpkg', 'JOURNAL_STREAM': '8:13479', 'GITHUB_ACTION': '__run_2', 'DEBIAN_FRONTEND': 'noninteractive', 'HOME': '/home/runner', 'GITHUB_GRAPHQL_URL': 'https://api.github.com/graphql', 'GITHUB_WORKFLOW': 'Deploy Documentation', 'GITHUB_RUN_ID': '22105486483', 'HOMEBREW_CLEANUP_PERIODIC_FULL_DAYS': '3650', 'GITHUB_ACTOR': 'MBR-COG', 'ANT_HOME': '/usr/share/ant', 'ImageOS': 'ubuntu22', 'Python_ROOT_DIR': '/opt/hostedtoolcache/Python/3.9.25/x64', 'ACTIONS_RUNNER_ACTION_ARCHIVE_CACHE': '/opt/actionarchivecache', 'GITHUB_OUTPUT': '/home/runner/work/_temp/_runner_file_commands/set_output_f1fbb62c-8480-4c50-854f-fdfb47e54140', 'GITHUB_BASE_REF': '', 'USE_BAZEL_FALLBACK_VERSION': 'silent:', 'GITHUB_REPOSITORY_OWNER_ID': '44060760', 'GOROOT_1_22_X64': '/opt/hostedtoolcache/go/1.22.12/x64', 'AZURE_EXTENSION_DIR': '/opt/az/azcliextensions', 'RUNNER_ENVIRONMENT': 'github-hosted', 'GOROOT_1_25_X64': '/opt/hostedtoolcache/go/1.25.7/x64', 'HOMEBREW_NO_AUTO_UPDATE': '1', 'JAVA_HOME_21_X64': '/usr/lib/jvm/temurin-21-jdk-amd64', 'MAKELEVEL': '1', 'DOCUTILSCONFIG': '/home/runner/work/digital-land-python/digital-land-python/docs/docutils.conf'}))
- digital_land.phase.convert.load_csv(path, encoding='UTF-8', log=None)
- digital_land.phase.convert.read_csv(input_path, encoding='utf-8')
- digital_land.phase.convert.read_excel(path)
- digital_land.phase.convert.save_efficient_json_as_csv(output_path, columns, data)
digital_land.phase.default module
digital_land.phase.dump module
- class digital_land.phase.dump.DumpPhase(path=None, f=None, enabled=True)
Bases:
Phasedump stream lines to a file
- process(stream)
- digital_land.phase.dump.dump(stream, path=None, f=None)
digital_land.phase.factor module
- class digital_land.phase.factor.FactorPhase
Bases:
Phaseadd a fact hash identifier
- process(stream)
- digital_land.phase.factor.fact_hash(entity, field, value)
digital_land.phase.filter module
digital_land.phase.harmonise module
digital_land.phase.load module
digital_land.phase.lookup module
- class digital_land.phase.lookup.EntityLookupPhase(lookups={}, redirect_lookups={}, issue_log=None, operational_issue_log=None, entity_range=[])
Bases:
LookupPhase- entity_field = 'entity'
- process(stream)
- class digital_land.phase.lookup.FactLookupPhase(lookups={}, redirect_lookups={}, issue_log=None, odp_collections=[])
Bases:
LookupPhase- process(stream)
- class digital_land.phase.lookup.LookupPhase(lookups={}, redirect_lookups={}, issue_log=None, operational_issue_log=None, entity_range=[])
Bases:
Phaselookup entity numbers by CURIE
- build_reverse_lookups()
- check_associated_organisation(entity)
- entity_field = None
- get_entity(block)
- lookup(**kwargs)
- process(stream)
- redirect_entity(entity)
Given an entity number can check the redirect lookups to see if the entity has been removed or redirected.
- class digital_land.phase.lookup.PrintLookupPhase(lookups={}, redirect_lookups={})
Bases:
LookupPhase- process(stream)
- digital_land.phase.lookup.key(entry_number='', prefix='', reference='', organisation='')
- digital_land.phase.lookup.normalise(value)
digital_land.phase.map module
- class digital_land.phase.map.MapPhase(fieldnames, columns={}, log=None)
Bases:
Phaserename field names using the provided column map
- headers(fieldnames)
- log_headers(headers)
- process(stream)
- digital_land.phase.map.normalise(name)
digital_land.phase.migrate module
digital_land.phase.normalise module
- class digital_land.phase.normalise.NormalisePhase(skip_patterns=[])
Bases:
Phase- normalise_whitespace(row)
- null_path = '/home/runner/work/digital-land-python/digital-land-python/digital_land/phase/../patch/null.csv'
- null_patterns = []
- process(stream)
- skip(row)
- skip_patterns = []
- spaces = ' \n\r\t\x0c'
- strip_nulls(row)
digital_land.phase.organisation module
digital_land.phase.parse module
digital_land.phase.patch module
digital_land.phase.phase module
digital_land.phase.pivot module
digital_land.phase.prefix module
digital_land.phase.priority module
digital_land.phase.prune module
- class digital_land.phase.prune.EntityPrunePhase(issue_log=None, dataset_resource_log=None)
Bases:
Phaseremove entries with a missing entity
- process(stream)
digital_land.phase.reference module
- class digital_land.phase.reference.EntityReferencePhase(dataset=None, prefix=None, specification=None, issues=None)
Bases:
Phaseensure an entry has the prefix and reference fields
- process(stream)
- process_row(row)
- class digital_land.phase.reference.FactReferencePhase(field_typology_map=None, field_prefix_map=None, dataset=None, specification=None)
Bases:
EntityReferencePhaseensure a fact which is a reference has a prefix and reference field
- get_field_prefix(field_name)
- get_field_typology_name(field_name)
- process_row(row)
- digital_land.phase.reference.split_curie(value)
digital_land.phase.save module
- class digital_land.phase.save.SavePhase(path=None, f=None, fieldnames=None, enabled=True)
Bases:
Phasesave stream rows to a file
- process(stream)
- digital_land.phase.save.save(stream, path=None, fieldnames=None, f=None)