D2.8.III.10 Data Specification on Population Distribution – Technical Guidelines

The application schemas included in this section are specified in UML, version 2.1. The spatial object types, their properties and associated types are shown in UML class diagrams.

NOTE For an overview of the UML notation, see Annex D in [ISO 19103].

The use of a common conceptual schema language (i.e. UML) allows for an automated processing of application schemas and the encoding, querying and updating of data based on the application schema – across different themes and different levels of detail.

The following important rules related to class inheritance and abstract classes are included in the IRs.

The use of UML conforms to ISO 19109 8.3 and ISO/TS 19103 with the exception that UML 2.1 instead of ISO/IEC 19501 is being used. The use of UML also conforms to ISO 19136 E.2.1.1.1-E.2.1.1.4.

NOTE ISO/TS 19103 and ISO 19109 specify a profile of UML to be used in conjunction with the ISO 19100 series. This includes in particular a list of stereotypes and basic types to be used in application schemas. ISO 19136 specifies a more restricted UML profile that allows for a direct encoding in XML Schema for data transfer purposes.

To model constraints on the spatial object types and their properties, in particular to express data/data set consistency rules, OCL (Object Constraint Language) is used as described in ISO/TS 19103, whenever possible. In addition, all constraints are described in the feature catalogue in English, too.

NOTE Since "void" is not a concept supported by OCL, OCL constraints cannot include expressions to test whether a value is a void value. Such constraints may only be expressed in natural language.

In the application schemas in this section several stereotypes are used that have been defined as part of a UML profile for use in INSPIRE [DS-D2.5]. These are explained in Table 1 below.

The IRs distinguish the following types of code lists.

The type of code list is represented in the UML model through the tagged value extensibility, which can take the following values:

NOTE This data specification may specify recommended values for some of the code lists of type (b), (c) and (d) (see section 5.2.4.3). These recommended values are specified in a dedicated Annex.

In addition, code lists can be hierarchical, as explained in Article 6(2) of the IRs.

The type of code list and whether it is hierarchical or not is also indicated in the feature catalogues.

Article 6(4) obliges data providers to use only values that are allowed according to the specification of the code list. The "allowed values according to the specification of the code list" are the values explicitly defined in the IRs plus (in the case of code lists of type (b), (c) and (d)) additional values defined by data providers.

For attributes whose type is a code list of type (b), (c) or (d) data providers may use additional values that are not defined in the IRs. Article 6(3) requires that such additional values and their definition be made available in a register. This enables users of the data to look up the meaning of the additional values used in a data set, and also facilitates the re-use of additional values by other data providers (potentially across Member States).

NOTE Guidelines for setting up registers for additional values and how to register additional values in these registers is still an open discussion point between Member States and the Commission.

For code lists of type (b), (c) and (d), this data specification may propose additional values as a recommendation (in a dedicated Annex). These values will be included in the INSPIRE code list register. This will facilitate and encourage the usage of the recommended values by data providers since the obligation to make additional values defined by data providers available in a register (see section 5.2.4.2) is already met.

NOTE For some code lists of type (d), no values may be specified in these Technical Guidelines. In these cases, any additional value defined by data providers may be used.

The following two types of code lists are distinguished in INSPIRE:

NOTE Where practicable, the INSPIRE code list register could also provide http URIs and labels for externally governed code lists.

For each code list, a tagged value called "vocabulary" is specified to define a URI identifying the values of the code list. For INSPIRE-governed code lists and externally governed code lists that do not have a persistent identifier, the URI is constructed following the pattern http://inspire.ec.europa.eu/codelist/<UpperCamelCaseName>;.

If the value is missing or empty, this indicates an empty code list. If no sub-classes are defined for this empty code list, this means that any code list may be used that meets the given definition.

An empty code list may also be used as a super-class for a number of specific code lists whose values may be used to specify the attribute value. If the sub-classes specified in the model represent all valid extensions to the empty code list, the subtyping relationship is qualified with the standard UML constraint "\{complete,disjoint}".

The application schema(s) use(s) the attributes "validFrom" and "validTo" to record the validity of the real-world phenomenon represented by a spatial object.

The attributes "validFrom" specifies the date and time at which the real-world phenomenon became valid in the real world. The attribute "validTo" specifies the date and time at which the real-world phenomenon is no longer valid in the real world.

Specific application schemas may give examples what "being valid" means for a specific real-world phenomenon represented by a spatial object.

NOTE The requirement expressed in the IR Requirement above will be included as constraints in the UML data models of all themes.

A statistical data distribution describes the structure of a "dissemination area" based on a set of measures of a phenomenon of interest. The notion of "population distribution" can be best understood with the following example. The picture represents the way the population is distributed within the surroundings of Bordeaux, here defined as a geographical rectangle. The surroundings of Bordeaux are our "area of Dissemination". This area is pictured with a set of red or white pixels which are actually small 200 m wide grid cells.

For each cell population counts were produced: the "value" is just a number of people. The pixels are white when population is zero and darken when the population count gets higher. But the whole set of pixels - the whole set of grid cells - provide a complete picture of the area.

The aim of the proposed data model is to provide capabilities of exchanging the data that is behind such pictures. In this example: the whole set of figures that are associated to each pixel that are necessary to build the global picture that will represent the area of dissemination. The model is not restricted just to one form of area of dissemination, it is not restricted to grid cells as elementary components of the area of dissemination - statistical units described vectorial (census blocks, admin units) are also supported - and it is not restricted to simple population counts. The model is a generic one to allow almost any statistical data to be exchanged.

One part of the model is a kind of description of metadata that is not covered by the ISO 19115 / 9 metadata standards and that should be added as a header in the dataset.

An example:

The domain describes the domain of statistical knowledge the value belongs to and the measure contains the short term of the phenomenon which is counted. The classifications describe the type of classification used to calculate or aggregate the values. The area of Dissemination describes the area for which the statistical data are available and / or the area selected by the user.

Temporal dimension of population distribution is considered in the three aspects: the actual period of measurement and the theoretical reference period of the measurement and the period the data remain valid.

The second part of the model contains the elementary data values and the description by dimensions. Each value is characterised by a certain combination of thematical and a spatial dimensions. This part applies as often as values are available in the area disseminated / requested.

NOTE This example is simplified and does not represent all attributes of the model.

The Population distribution – demography data model is defined in a very generic way, and nearly any statistical value describing a statistical measured phenomenon e.g. human population, can be considered a part of it. Use cases show the need for statistical variables as population at residence place, population at work place and population density. These statistics need to be provided on a more detailed territorial level than what is possible to access already through the Eurostat databases, preferable on statistical units like grids or census districts.

The requirements reflect the role of this theme in the INSPIRE context. The National Statistical Institutes should not be forced to provide INSPIRE services for all statistics covering human population, as building two parallel systems (regional statistics already provided by Eurostat and regional statistics provided by the member states through INSPIRE) will both be costly and time consuming.

An overview of the Population distribution - demography package and referenced packages is provided in Figure 1 below. The Population Distribution - Demography package refers to the Base Types package for Identifier data type and some general base types to be used in accordance with the GCM. In addition the StatisticalUnit feature type of Annex III theme Statistical units is referenced to support the indirect positioning of statistical data with respect to the location on the Earth.

The application schema for Population distribution - demography is shown in Figure 2 and described below.

The schema is structured with three types of classes: "StatisticalDistribution", "StatisticalValue" and "StatisticalValue.Dimensions" and two groups of code lists.

Feature Type: StatisticalDistribution

The StatisticalDistribution is defined as a feature type for practical reasons, to be able to generate the according GML. This type mainly acts as a header to keep general (meta-) information about the distribution and contains common properties of each component of the distribution.

To provide the INSPIREId is not mandatory. It may be useful for internal organisational reasons of the provider, if the query or result will be stored for use at a later time.

areaOfDissemination: The areas of dissemination is the part of the 2D world the StatisticalDataDistribution describes, represented as a surface.

The domain attribute provides information about the domain of statistical knowledge.

The naming of the domains is not harmonised over Europe. Each provider should use the naming according his existing structure.

The universe attribute provides information if the statistics cannot be assigned to domain and measure. This is the case if statistics are produced for a certain use case on user request and must be documented here.

Example: "Agriculture employment" may be calculated by dividing the persons employed in agriculture by the number of total persons aged 18 to 64 in employment

The attribute measure contains the value of the code list variable, which describes in short- term the phenomena for which data provided.

The attributes measurementUnit and MeasurementMethod support description of the kind of measurements provided by the distribution. The possible values are taken from the corresponding code lists UnitOfMeasure and statisticsMeasurementMethod.

According the common approach in INSPIRE to describe the quality of the content of a dataset, some data quality considerations appear through the notCountedProportion attribute that reports the relative amount of population known to be in the area of interest but not delivered in any of its spatial elements.

According to Regulation 1205/2008/EC and because the time dimension is an important component of statistical information, the model includes three attributes to address this issue:

periodOfReference

is the period when the data is supposed to give a picture of the area of interest..

In PD the reference period can be a period or a fixed date. In the last case one should use the same date for the beginning and the end.

periodOfValidity

In SU and PD the Validity is the period in which data remain relevant.

For PD in most cases this is forever. In that case the 2nd date is left empty.

periodOfMeasurement

This attribute is only useful in PD. The date / period the observation has been taken, the data was collected.

To facilitate a user driven selection of data sets / values for a differential update the attributes beginLifeSpanVersion and endLifeSpanVersion are provided. These attributes are used by a user who is searching regularly but expecting only those values which are changed since his last query, not all data values. (What has changed since my last enquiry?)

The attribute generalStatus is used to describe the status of the values for the complete dataset, like definitive, semi-definitive or provisional and preliminary or final (Examples from Eurostat publications).

The classification attribute can be used to represent a set of distributions with respect to the way of splitting the statistical values into different groups using one or several classifications of the individuals according to their characteristics. No classification may happen for the "Total" values.

The {ClassificationItemTypeConsistency} rule defines that all ClassificationItemTypes used to classify the value should be included into this list.

Data Type: StatisticalValue

The data type StatisticalValue supports the description of the individual values of the distributions: This description contains the reference to understand to which spatial element and to which classification items the value refers to. The elementary datum can be seen as something that can be located in an n-dimensional space comprising the classical two dimensional one, but also the dimensions relative to the classifications used. For example, a simple population count on each municipality or on one country will only need a reference to the two dimensional space viewed through the municipalities or country list. But the same population divided by gender will need an additional third dimension.

In the centre of this item is the value attribute, which contains the number statistically calculated.

In case the value cannot be provided (e.g. privacy, error, confidentiality) the reason for value absence should be expressed using the specialValue attribute. It is required that either a value or a special value is provided.

Values can be accompanied with flags and footnotes (comment attribute) as usual in the statistics community, giving any kind of information useful to describe particular situations.

Two additional attributes related to data quality gives hints about the accuracy of the measure: conventionallyLocatedProportion and approximatelyLocatedProportion. They are discussed in the quality section.

The attribute status is used for exceptions of a certain value to what was specified in the generalStatus information of the statisticalDistribution.

Because there is no harmonised concept how to use the correct codes of the attributes "specialValue" and "flags", the corresponding code list must be provided by the data provider. It is recommended to the statistical community to develop a harmonised concept of using special values, flags and status.

Additionally, for each StatisticalValue a time stamp (periodOfMeasurement) can be provided, to allow exceptions to what was specified in the general information.

Data Type StatisticalValue.Dimensions:

NOTE The association between Dimensions and the Core::StatisticalUnit will be established using the attribute "thematicallyID" instead of the "INSPIREID" (see below: Referencing data to statistical units").

According to the SDMX view on dimensions, all dimensions of the value (thematical and spatial) are summarized in a unique data type dimensions.

Note: The cardinality of the attribute dimensions is 1, because at least the geographical reference has to be provided.

In case an additional splitting / grouping of the statistical values are needed according to one or more classifications, the attribute classificationItem will be used.

Example: To describe a value representing residential population by gender, nationality and 5 year age classes three thematical classificationItems are used "male", "French", and "25-30", out of the domains of the code lists "Gender", "Nationality" and "age5Years".

Referencing data to statistical units:

All statistical data are spatially referenced (indirectly linked to a statistical unit), which is expressed by an common identifier, in SDMX called geographical dimension. In the geographical word (in GIS) these identifiers are used to identify the corresponding geometry and to join the tabulated data. These ID build the bridge (link) between the statistical and geographical world. Difficulties rise because different rules are used, when and how these identifiers have to change during an evolution.

A strong general requirement of the GCM in INSPIRE is, that an ID should be capable of uniquely and permanently identifying that with which it is associated [ISO 19135]. This is a rule which do not apply for geo-attributes (identifiers) used in the statistical community.

Therefore beside the INSPIRE ID an additional thematical identifier attribute is provided in the VectorStatisticalUnit feature type. This thematical ID can be used for linking PD and SU data following the recommendation of statistical unit data specification.

In the case the update rules for the identifier of a certain statistical unit follow the rules of the GCM resp. ISO [ISO 19135] without any exceptions, the INSPIRE identifier can be copied as thematical identifier.

Because the spatial object to which a thematical ID is linked (e.g. a municipality) can change without changing the thematical ID, some version information is requested. Typically and easy to use this is the reference year. This is necessary to avoid linking statistical with geometrical data of different reference years.

The update rules for the identifier should describe clearly the use of the version identifier. There are several reasons to create a new version of an ID but not a new statistical unit (e.g. the area has changed less than 3% or the name has changed.

EXAMPLE:

The German State of Brandenburg was classified in the NUTS 2003 regulation as one level 1 unit divided in 18 level 3 units; level2 was equal to level 1. In the regulation 2007 the level 2 was divided in 2 units and the level 3 units have been reassigned. In the NUTS regulation 2010 the two level 2 units are aggregated to one level 2 unit and the level 3 units reassigned again. To distinguish the definition until 2003 from those of 2011 which have the same thematical identifier, the version ID (eg. 1999 and 2010 has to be added.

The spatial reference of the data type StatisticalValue is the thematical Identifier of the Statistical Unit referenced. It is the value of a specific instance of the StatisticalUnit feature type.

The requirements and recommendations given in the INSPIRE data specification Statistical Units should be taken in consideration when aggregating statistical data to spatial units.

Class Code lists:

The feature type StatisticalDistribution uses two groups of code lists, the one for classifying the statistical value and others for the description of the variable, the measurement and the status.

Universe, Domain and Variable are used to identify the object, statistical measurement method describes how data are carried out and status explains the maturity of the dataset in the production chain. All attributes of this feature type are valid for the complete dataset.

Exceptions from the description above on value level can be described using attributes assigned to the data type "StatisticalValue". Code lists should be used for the attributes special value, flags and status.

The code list group classifications contain the classificationItems used to identify each value thematically. The numbers of classificationItems depend on the granularity needed and used to describe the phenomenon according the statistic provided. If no classificationItem is provided the value of the variables in the statistical units represents the total. All classifications used have to be shown in the attribute Classification of the feature type StatisticalDistribution.

There is no specific recommendation regarding consistency between spatial data sets. There should be a consistency between PD and SU objects. For each statistical unit referenced in the statistical data the geometry of the statistical unit has to be provided. It is recommended to use an identification of the statistical unit version which is used to aggregate the surveyed data.

The feature type StatisticalDataDistribution has an identifier attribute. Following the general INSPIRE rules (GCM) each object has an ID, but exceptions are possible for themes of Annex III. From the point of using the data interoperable, no identifier is needed for the theme Population Distribution. The ID may be used for disseminating statistical data, if the provider will save the data selection or query for a later additional use. In this case the user should be able to find the statistical data (or probably an updated version) using the INSPIREID.

The identifier shall consist of two parts: the namespace and a local id. The namespace is to uniquely identify a national registry wherein the identifier is registered, whereas the local id is to unique identifying an object within this namespace. The pragmatic approach to making it internationally unique is to add a prefix of the Member State identifier along with a theme specific identifier for the namespace.

Since INSPIRE objectives refer to data exchange only, the maintenance and management of unique

INSPIRE identifiers is out of INSPIRE scope, and is in the responsibility of the Member States.

If no ID is included some other data about the provider and base has to be provided. This should be the providing institution and if applicable the legislation.

This section lists definitions for feature types, data types and code lists that are defined in other application schemas. The section is purely informative and should help the reader understand the feature catalogue presented in the previous sections. For the normative documentation of these types, see the given references.

These Technical Guidelines propose to use the http URIs provided by the Open Geospatial Consortium as coordinate reference system identifiers (see identifiers for the default CRSs in the INSPIRE coordinate reference systems register). These are based on and redirect to the definition in the EPSG Geodetic Parameter Registry (http://www.epsg-registry.org/).

NOTE CRS identifiers may be used e.g. in:

This data specification proposes the use of GML as the default encoding, as recommended in sections 7.2 and 7.3 of [DS-D2.7]. GML is an XML encoding in compliance with ISO 19118, as required in Article 7(1). For details, see [ISO 19136], and in particular Annex E (UML-to-GML application schema encoding rules).

The following TG requirements need to be met in order to be conformant with GML encodings.

NOTE 1 Not all constraints defined in the application schemas can be mapped to XML. Therefore, the following requirement is necessary.

NOTE 2 The obligation to use only the allowed code list values specified for attributes and most of the constraints defined in the application schemas cannot be mapped to the XML sch. They can therefore not be enforced through schema validation. It may be possible to express some of these constraints using other schema or rule languages (e.g. Schematron), in order to enable automatic validation.

Name: Population distribution - demography GML Application Schema
Version: 4.0
Specification: D2.8.III.10 Data Specification on Population Distribution – Technical Guidelines
Character set: UTF-8

The xml schema document is available from INSPIRE website: https://inspire.ec.europa.eu/schemas/pd/4.0/PopulationDistributionDemography.xsd

There is no specific requirement about the encoding of the data in general. However, the target of the whole INSPIRE specifications is to allow GIS-like application to use the data. As it was written before population distribution objects are not spatial objects and will primarily come from production processes that are not very close to the world of GIS.
In the case of pure PD data, this data will be made available through a translation. It is strongly expected that the data will be first available using standards that are today in use in the world of statistics, such as SDMX. SDMX dataset structure is not very different from the GML one, so a general XSL tool will be provided to make the bridge between the two formats.
SU and PD data can be provided in the same OGC geowebservice.
In the case of mapping (WMS) or transmitting data in combination with geometry (WFS) the standard INSPIRE network service should be used.
Special Encoding is required to transmit gridded data. In that case, typically grid data formats like TIFF, GeoTIFF, JPEG2000 or GMLJP2 should be used in combination with necessary additional metadata dataset (GML/XML combination of Metadata in XML and the data file embedded). More information the reader can find in the INSPIRE document D2.7 Guidelines for Encoding 7.5.
For transmitting gridded data, an alternative may be the WCS Service.
Some National Statistical Institutes are testing the new offered OGC based Service GDAS/TJS the Table join service based on a standardized table format.