AVEVA Data Hub Namespace Data Copy Python Sample

📢 Notice: Samples have been updated to reflect that they work on AVEVA Data Hub. The samples also work on OSIsoft Cloud Services unless otherwise noted.

Version: 1.1.5

Developed against Python 3.9.1.

Requirements

Python 3.7+
Register a Client-Credentials Client in your AVEVA Data Hub tenant and create a client secret to use in the configuration of this sample. (Video Walkthrough)
Install required modules: pip install -r requirements.txt

About this sample

This sample uses REST API calls to copy data views, assets, and streams from a source namespace to a destination namespace. Some common use cases are described in the Configuring the sample section. The steps are as follows

Copy the data view specified by its Id and add all referenced assets and streams to lists to be copied later
Add all streams referenced by assets found by the specified query and referenced by the previously found data view to a list to be copied later
Copy all streams found by the specified query and referenced by the previously found assets and data view
Copy all assets found by the specified query and referenced by the previously found data view
If testing, cleanup all created data views, assets, and streams

Configuring the sample

The sample is configured by modifying the files appsettings.py and appsettings.placeholder.json. Details on how to configure them can be found in the sections below. Before editing appsettings.placeholder.json, rename this file to appsettings.json. This repository's .gitignore rules should prevent the file from ever being checked in to any fork or branch, to ensure credentials are not compromised.

Configuring appsettings.py

The configurable parameters within appsettings.py are described in the table below. In addition, the common use cases for this sample and how to configure parameters are described in the following subsections.

Parameters	Description
stream_query	The query used to find streams to copy. This can be set to '*' to copy all streams or None to copy no streams (besides the underlying streams of any queried assets or the specified data view). Please refer to docs.osisoft.com for more information on searching for streams.
asset_query	The query used to find assets to copy. This can be set to '*' to copy all assets or None to copy no assets (besides the underlying assets of the specified data view). Please refer to docs.osisoft.com for more information on searching for assets.
data_view_id	The id of the data view to copy.
prefix	The prefix to apply to the Ids of all streams, assets, and dataviews along with all underlying types.
max_stream_count	The maximum number of streams to be returned by the stream query. The maximum value is 1,000.
max_asset_count	The maximum number of assets to be returned by the stream query. The maximum value is 1,000.
start_time	The start time of the data window to be transfered.
end_time	The end time of the data window to be transfered.
request_timeout	The time before a request time-out. If requests are timing-out try increasing this value.

Copying all streams

Set max_stream_count to 1,000 (if more than 1,000 streams are being copied, the copying will need to be done in batches)
Set stream_query to '*' (if copying in batches, this query will need to be modified)
Set asset_query to None
Set data_view_id to None

Copying all assets and all underlying streams

Set max_asset_count to 1,000 (if more than 1,000 assets are being copied, the copying will need to be done in batches)
Set stream_query to None
Set asset_query to '*' (if copying in batches, this query will need to be modified)
Set data_view_id to None

Copy a data view and all underlying assets and streams

Set stream_query to None
Set asset_query to None
Set data_view_id to the desire data_view_id

Configuring appsettings.json

AVEVA Data Hub is secured by obtaining tokens from its identity endpoint. Client credentials clients provide a client application identifier and an associated secret (or key) that are authenticated against the token endpoint. You must replace the placeholders in your appsettings.json file with the authentication-related values from your tenant and a client-credentials client created in your ADH tenant. There are separate configurations for both the source and destination namespaces; however, there can be overlap in some settings such as TenantId, ClientId, ClientSecret, and even NamespaceId if, for example, you want to migrate to a namespace in the same tenant.

[SourceConfiguration]
NamespaceId = REPLACE_WITH_CURRENT_NAMESPACE_ID
TenantId = REPLACE_WITH_CURRENT_TENANT_ID
ClientId = REPLACE_WITH_CURRENT_CLIENT_ID
ClientSecret = REPLACE_WITH_CURRENT_CLIENT_SECRET
Resource = https://uswe.datahub.connect.aveva.com
ApiVersion = v1

[DestinationConfiguration]
NamespaceId = REPLACE_WITH_NEW_NAMESPACE_ID
TenantId = REPLACE_WITH_NEW_TENANT_ID
ClientId = REPLACE_WITH_NEW_CLIENT_ID
ClientSecret = REPLACE_WITH_NEW_CLIENT_SECRET
Resource = https://uswe.datahub.connect.aveva.com
ApiVersion = v1

Running the sample

To run this example from the command line once the appsettings.json is configured, run

python program.py

Running the automated test

Note: The test will delete the Data View, Assets, and Streams that were created.

To test the sample, run

pip install pytest
python -m pytest test.py

Tested against Python 3.9.1

For the main ADH samples page ReadMe
For the main AVEVA samples page ReadMe

osisoft/sample-adh-namespace_data_copy-python