API Documentation

This repository enables the deployment of Geovation Catalyst NGD Wrapper resources on Amazon Web Services (AWS).

• A template for automatically deploying the API and all the associated required AWS resources can be found at our catalyst-deployment repository.
• For the python package underlying the functionality of this deployment, see catalyst-ngd-wrappers-python.
• For the correspond deployment tools for Azure, see catalyst-ngd-wrappers-azure.

Chart specifying valid orders to "chain" different wrapper functions/extensions together

• Subtitles specifying the naming convention and ordering for python function names, and the corresponding final component of API paths
• Eg. the api url which combines the feature limit exentension, and the multiple collections extension will finish .../items/limit-col?...
• The OAuth2 Manager is applied by default, if the appropriate environment variables are configured (see below)

graph TD
    X[Basic Request Wrapper] --> A[OAuth2 Manager, automatic]
    A --> B[Feature Limit Extension<br><em>limit</em>]
    A --> C[Multigeometry Search Extension<br><em>geom</em>]
    A --> D[Multiple Collections Extension<br><em>col</em>]
    B --> C
    B --> D
    C --> D

Loading

OS NGD API - Features

• All the Catalyst APIs extend the core functionality of GET request for items using the OS NGD API - Features
  • The endpoint for this API is https://api.os.uk/features/ngd/ofa/v1/collections/{collectionId}/items
  • Documentation for the API can be found on the Gitbook docs for the National Geographic Database (NGD)

Authentication

• API Key
  • Requests can be authenticated using all the OS NGD API - Features authentication methods (keys and access tokens). The methods are:
    • OAuth2 bearer token
    • HTTP Header key
    • HTTP Query Parameter key
• OAuth2 Environment Variables
  • If CLIENT_ID and CLIENT_SECRET are set as environment variables in the production environment, the API handles OAuth2 authentication automatically, generating and reusing access tokens until they expire.
  • If you use this method, we highly reccomend you enable API key authentication on all the endpoints methods to secure access. if you use our automatic deployment templates, this will be configured automatically. You can find/create keys under 'API Keys' on the API Gateway console. This API key must be passed to all API requests as the X-API-Key header.
  • You will also need to create a project on the OS DataHub, and ensure that OS NGD API - Features is added to the project. CLIENT_ID should be set as the Project API Key value, and CLIENT_SECRET should be set as the Project API Secret value. CLIENT_ID should be set as the Project API Key value, and CLIENT_SECRET should be set as the Project API Secret value.
  • The variables can be set under the Configuration tab of the Lambda function console.

Request Specifications

• Path Parameters
  • collectionId: str - Any collection available from the NGD, or "multi-collection" (for when col extension applied)
• Query Parameters inherited from OS NGD API - Features items request (refer to docs above for details)
  • key: str - OS DataHub project API Key
  • bbox: str
  • bbox-crs: str (xmin,ymin,xmax,ymax)
  • datetime: str (datetime)
  • filter: str
  • filter-crs: str
  • filter-lang: str
  • limit: int
    • This can exceed the usual cap of 100 when the limit extension is applied.
    • IMPORTANT: When used with geom and/or col exention, this limit applies per search area, per collection. The total number of features returned could therefore be much higher.
    • When used in conjunction with request-limit, the lower cap is applied.
    • NOTE: When used with geom extension, the number of features returned per geom could be lower than limit in some cases. This is because, if features overlap multiple search areas, duplicates are removed after the limit is reached.
  • offset: int - Not available when used with the limit extension
• Additional Catalyst Query Parameters
  • wkt: str (well-known text) - A means of searching a geometry for features. The search area(s) must be supplied in wkt. The function automatically composes the full INTERSECTS filter and adds it to the filter query parameter. Read how to write well-known text here. Make sure that filter-crs is set to the appropriate value.
  • use-latest-collection (bool, default False) - If True, it ensures that if a specific version of a collection is not supplied (eg. "bld-fts-building-2"), the latest version is used. If use-latest-collection=True but the given collection does include a version, the specified version is always used regardless.
  • request-limit: int (default 50) - the number of OS NGD Feature requests at which to cap the Catalyst request. Consider pricing.
    • IMPORTANT: When used with geom and/or col exention, this limit applies per search area, per collection. The total number of features returned could therefore be much higher.
    • When used in conjunction with limit, the lower cap is applied.
  • hierarchical-output: bool, default False - If True, then results are returned in a hierarchical structure of GeoJSONs according to collection and/or search area, when the col and geom extensions are applied respectively. If False, results are returned as a single GeoJSON.
    • geom and col extensions only.
  • collection: str
    • accepts multiple values as a list. eg: collection=bld-fts-building-4,bld-fts-buildingaccesslocation-1,trn-ntwk-street-1
    • col extension only.
  • authenticate: bool, default True - if True, then request authentication is handled automatically through environment variables (you must configure these in the AWS Lambda console).

Response Specifications

• Format GeoJSON by default. If the hierarchical-output=True, a hierarchical json containing separate GeoJSONs according to collection and/or search area number.

• Response Metadata:

  • Attributes from OS NGD API - Features items request (refer to docs above for details)
    • type: str
    • timeStamp: str (date-time) - Format "YYYY-MM-DDTHH:MM:SS.sssssssZ"
    • numberReturned: int
    • features: array of Feature (object)
    • links - This is absent if either limit extension is applied, or if hierarchical-output=False (if this attribute applies). This is because in these cases the GeoJSON(s) comprising the response do not represent a single NGD feature request.
  • Additional Catalyst attributes
    • numberOfReqeusts: int - The number of NGD items requests from which the final response is compiled
    • numberOfRequestsByCollection: dict[str: int] - The number of NGD items requests made, split by collection. Only included when col extension applied and hierarchical-output=False.
    • numberReturnedByCollection: dict[str: int] - The number of features returned, split by collection.

• Feature-Level Attributes

  • id: str (uuid) - OSID of the feature
  • collection: str - Collection the feature belongs to. This is an additional attribute supplied by catalyst
  • geometry: dict - List-like representation of the feature's geometry, and the geometry type
  • searchAreaNumber: int | list - The number of the search area where the feature is found. If a feature intersects multiple search areas, the numbers are given in a list. Only inclded when geom extension applied and hierarchical-output=False.
  • properties: dict - Non-spatial attribution associated with the feature
    • OS NGD attribution for each theme, collection, and feature type here
    • The collection name is added by catalyst
    • When the geom extension is applied, the searchAreaNumber value is also included
  • type: str - object type ("Feature")

• Response Codes

  For error responses (all except 200), the 'errorSource' attribute specifies whether the error was thrown from the OS NGD API, or from the surrounding Catalyst wrapper. Please also cross-reference with AWS API Gateway response types

  Response Code	Description
  200 $${\color{green}✓}$$	A list of features in a feature collection, or a hierarchical dictionary of feature collections.
  400 $${\color{red}✗}$$	The items request is not supported. Please refer to documentation for examples.
  401 $${\color{red}✗}$$	Missing or unsupported API key. Please ensure you include a key for a project that has OS NGD API - Features added.
  404 $${\color{red}✗}$$	A/the collection is not a supported. Please refer to the documentation for a list of supported collections.
  405 $${\color{red}✗}$$	The HTTP method requested is not supported. This endpoint only supports "GET" requests.
  406 $${\color{red}✗}$$	A request header value was not supported.
  414 $${\color{red}✗}$$	The request URI is too long. Please simplify the geometry filter and/or use simpler attribute filters.
  500 $${\color{red}✗}$$	Unexpected/unknown error.
  504 $${\color{red}✗}$$	The request has timed out. Please check the service availability dashboard: OS Data Hub Service Status.

Summary of Extensions

Extension	Extra Query Parameters	Description	Notes & Constaints
all of below	wkt, use-latest-collection	Basic wrapper for OS NGD API - Features, with some extra query parameters, and automatic OAuth2 authentication handling through environment variables.
limit	request-limit	Extends the maximum number of features returned above the default maximum 100 by looping through multiple OS NGD API - Features requests.	Default value of 50. This can be increased manually. When request-limit and limit are both supplied, the lower constraint is applied. When combined with geom and/or col, the limit applies per search area, per collection.
geom	hierarchical-output	An alternative means of returning OS NGD features for a search area which is a Multi-Geometry (MultiPoint, MultiLinestring, MultiPolygon, or GeometryCollection), which will in some cases improve speed, performance, and prevent the call from timing out. Each geometry is assigned a "searchAreaNumber", and each search area is searched in turn for features, with the search area numbers returned in feature properties, and as feature metadata.	When a feature overlaps with multiple search areas, it is returned once when hierarchical-output=Flalse, with searchAreaNumber returning a list. When hierarchial-output=True, the feature is repeated in the output for each search area.
col	hierarchical-output, collections	Enables multiple OS NGD collections to be searched at once. Each collection value supplied as query parameters is searched in turn for features, with the collection returned in feature properties, and as feature metadata.	{collectionId} path parameter must be "multi-collection". When combined with the geom extension, requests are subdivided into collections first, and then into search areas.

Available endpoints resulting from extension combinations

• Make sure to prefix the endpoints below with your AWS Host and Stage URL. If deployed through cataly-deployment, the Invoke URL can be found under the CloudFormation 'outputs' tab. Otherwise, it can be found under 'stages'>'Stage details' of the API resource in the API Gateway console.

• catalyst/latest-collections

• catalyst/latest-collections/{collectionId}

• catalyst/features/{collectionId}/items

• catalyst/features/{collectionId}/items/limit

• catalyst/features/{collectionId}/items/geom

• catalyst/features/multi-collection/items/col

• catalyst/features/{collectionId}/items/limit-geom

• catalyst/features/multi-collection/items/limit-col

• catalyst/features/multi-collection/items/geom-col

• catalyst/features/multi-collection/items/limit-geom-col

Testing with unit_tests.py

Once you have deployed the API, you can test the outputs using unit_tests.py.

python -m unittest unit_tests.py

You must make sure that the following environment variables are configured correctly in the testing environment. The root URL can be found from the API Gateway resource on the AWS Console, under 'Stages' > 'Invoke URL'. The Client ID can be found from your project on the OS DataHub (see Authentication docs above). The Gateway API key is only required if you have configured OS OAuth2 environment variables. It can be found/generated from the Function App resource under 'Functions' > 'App keys'.

• ROOT_URL = 'https://<your-rest-api-name>.execute-api.<your-aws-region>.amazonaws.com/<your-api-gateway-stage-name>'
• CLIENT_ID = <your-datahub-project-key>
• GATEWAY_API_KEY = <your-gateway-api-key>

If any issues arise in this process, then please raise an issue to let us know. We welcome any feedback.