ETL for generating data in (simple) alternative encoding
Opened this issue · 0 comments
arbakker commented
INSPIRE addresses in alternative encoding have been generated using the following SQL query (assuming databases with normalized datamodel voor INSPIRE addresses) according to these transformation rules:
create schema oaf_poc_2021;
create view oaf_poc_2021.simple_inspire_ad as
select '' as alternativeIdentifier,
ad.validfrom,
ad.validto,
null as beginLifespanVersion,
null as endLifespanVersion,
'' as building,
tfname.thoroughfarename as component_ThoroughfareName,
pdesc.postaldescriptor as component_PostalDescriptor,
adaname.addressareaname as component_AddressAreaName,
'' as component_AdminUnitName_1,
'' as component_AdminUnitName_2,
'' as component_AdminUnitName_3,
'' as component_AdminUnitName_4,
'' as component_AdminUnitName_5,
'' as component_AdminUnitName_6,
ad.designator_type_two as locator_designator_addressNumber,
ad.designator_type_three as locator_designator_addressNumberExtension,
ad.designator_type_four locator_designator_addressNumber2ndExtension,
'unit level' as locator_level,
'http://inspire.ec.europa.eu/codelist/LocatorLevelValue/unitLevel' as locator_href,
'' as locator_designator_buildingIdentifier,
'' as locator_designator_buildingIdentifierPrefix,
'' as locator_designator_cornerAddress1stIdentifier,
'' as locator_designator_cornerAddress2ndIdentifier,
'' as locator_designator_entranceDoorIdentifier,
'' as locator_designator_floorIdentifier,
'' as locator_designator_kilometrePoint,
'' as locator_designator_postalDeliveryIdentifier,
'' as locator_designator_staircaseIdentifier,
'' as locator_designator_unitIdentifier,
'' as locator_name,
'' as parcel,
'' as parentAddress,
ad.geom as geom,
'entrance' as position_specification,
'http://inspire.ec.europa.eu/codelist/GeometrySpecificationValue/entrance' as position_specification_href,
'by administrator' as position_method,
'http://inspire.ec.europa.eu/codelist/GeometryMethodValue/byAdministrator' as position_method_href,
true as position_default,
'' as status,
'' as status_href
from inspire_ad
.address ad
INNER JOIN inspire_ad.thoroughfarename tfname on ad.thoroughfarename_id = tfname.localid
INNER JOIN inspire_ad.postaldescriptor pdesc on ad.postaldescriptor_id = pdesc.localid
INNER JOIN inspire_ad.addressareaname adaname on ad.addressareaname_id = adaname.localid
;
Using this view a GeoPackage can be generated with ogr2ogr
:
export PGPASSWORD="postgres"
export PGCONN="PG:dbname='pdok' host='my-db-host' port='5432` user='postgres'"; ogr2ogr -f GPKG data.gpkg "$PGCONN" -oo ACTIVE_SCHEMA=oaf_poc_2021 -sql "select * from simple_inspire_ad where geom &&
ST_MakeEnvelope(4.6445,52.9685,4.9466,53.2027,4258)" -nln simple_inspire_ad
Note that with the above command the data is still in its source projection, namely EPSG:4258
.
Cluster data on geohash for better usability when browsing data (features will be in same area when paging through features):
CREATE INDEX simple_inspire_ad_texel_geohash ON oaf_poc_2021.simple_inspire_ad_texel (ST_GeoHash(ST_Transform(geom,4326)));
CLUSTER oaf_poc_2021.simple_inspire_ad_texel USING simple_inspire_ad_texel_geohash;