Using the PDS API

Before an API call to a PDS can be made, the connection between the members PDS and the connecting service must be authenticated. The details of the authentication process can be found here.

Dataset Types 🔗︎ click to copy

A Mydex member's stored data forms are divided into two categories: Datasets and Transactional Datasets. The distinction between these datasets is detailed in the Data Schema.

Return Format 🔗︎ click to copy

The Connection API can return a response to an API request in either JSON or XML. The formats are interchangeable and are chosen by putting the desired file extension in the API request (eg. .json or .xml). In the following examples we use JSON.

Multiple Dataset Instances 🔗︎ click to copy

Occasionally, a member will have more than one dataset instance in their PDS (eg. multiple home addresses). In this situation, a Connection API request will return all dataset instances that the member has granted you permission to access. These datasets are identified by the key instance_N, where N is the N^th dataset instance, starting from N=0.

Filtering Datasets 🔗︎ click to copy

Returning a whole dataset might not be practical or desirable. For example, if you want to return only the latest values within a dataset where changes have occurred since the last time this dataset was queried. This can be overcome by filtering the results. Filtering results is achieved by passing two additional query parameters in the request: filters[{field_name}][operator] and &filters[{field_name}][value]. For example, if you wanted to get dataset rows greater than a specific ID value, you would add parameters like this: &filters[id][operator]>= and &filters[id][value]=21, where 21 is the ID value you are filtering against. For more details on filtering plus example code, please see under the heading Filtering Datasets at the bottom of this page.

Connection API Request Differences 🔗︎ click to copy

The key differences when making a Connection API request for each dataset type are summarised in the table below.

	Dataset	Transactional Dataset
About	Multiple datasets with potentially multiple dataset instances	Multiple datasets with potentially multiple dataset instances
Availability	On Production (https://api.mydex.org) & Sandbox (https://sbx-api.mydex.org) servers	On Production & Sandbox servers
URL	Uses the `api/pds/pds` path structure	Uses the `api/pds/transaction` path structure
When Retrieving Data	The `uid` (`345` in the following examples) is part of the path structure	The `uid` is part of the path structure
	The dataset `instance` is not included in the request	The dataset `instance` must be included as a parameter (eg. `&instance=0`)
	The `dataset` is passed as parameter	The `dataset` is passed as parameter
When Adding Data	The `uid` is part of the path structure	The `uid` is passed as a parameter (eg. `?uid=345`)
	The dataset `instance` must be included as a parameter	The dataset `instance` must be included as a parameter
	The `dataset` must not be included in the data	The `dataset` must be included in the data and as a parameter
	Updates are made using HTTP PUT	Updates are made using HTTP POST
When Filtering Datasets	Results can be filtered by using one of the following operators: `('=', '<', '<=', '>', '>=', 'LIKE', 'IN', 'BETWEEN');`	Results can be filtered by using one of the following operators: `('=', '<', '<=', '>', '>=', 'LIKE', 'IN', 'BETWEEN');`
Working with assets (files)	See link to dedicated documentation

In addition to the above, Connection API requests should also include the key, api_key and source_type=connection values as parameters.

Examples 🔗︎ click to copy

Example Connection API requests for each specific dataset, and details of dataset fields can be found in the Data Schema. General examples for each dataset type can be found below.

Datasets 🔗︎ click to copy

Retrieving Data 🔗︎ click to copy

Example API Request 🔗︎ click to copy

https://sbx-api.mydex.org/api/pds/pds/345.json
    ?key=YfMPI2iSFcBDQ5CdlJVQeXIbIUD156s0
    &api_key=5FXfidz9zw6YQaO2kWh0B154zz6wafrV
    &con_id=345-1545
    &source_type=connection
    &dataset=ds_home
    &instance=0

Pastable:

https://sbx-api.mydex.org/api/pds/pds/345.json?key=5FXfidz9zw6YQaO2kWh0B154zz6wafrV&api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf&con_id=345-1545&source_type=connection&dataset=field_ds_home

Example Response 🔗︎ click to copy

"1": {
   "id": "1",
   "field_home_bedrooms": "3",
   "field_home_country": "United Kingdom",
   "field_home_county": "Fake County",
   "field_home_date_from": "1327398114",
   "field_home_date_to": "1611481314",
   "field_home_habitable_rooms": "2",
   "field_home_owner_street": "Fake Street",
   "field_home_owner_town": "Fake Town",
   "field_home_postcode": "G92 3KJ",
   "field_home_property_type": "House",
   "field_home_street": "18 Fake Street",
   "field_home_tenancy": "Lease Tenant",
},

Note: not all of the fields have been included in this example to improve readability.

Updating Data 🔗︎ click to copy

It is possible to update numerous fields in multiple dataset instances with your own data by issuing a PUT request to the Connection API.

Note: you can only update a record that exists. We recommend you first make a GET request to the dataset determine if the record exists. You will need the ID of the record you wish to update.

Example API Request 🔗︎ click to copy

As you saw in the previous example, the dummy member we created was given a dataset instance. We will now submit an API request to update select details in the dataset instance.

Many browsers don't natively support PUT, so the following example uses curl to make a PUT request. This example request has been restructured to improve readability.

curl
    -X PUT
    -H "Content-Type: application/json"
    -d '{
           "field_home_bedrooms": "2",
           "field_home_owner_name": "I. M. Landlord"
        }'
    'https://sbx-api.mydex.org/api/pds/pds/345.json?
        key=5FXfidz9zw6YQaO2kWh0B154zz6wafrV
        &api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf
        &con_id=345-1545
        &dataset=ds_home
        &source_type=connection
        &instance=0
        &id=1'

This request changes the field_home_bedrooms field value and adds the field_home_owner_name field value. Tables of all available datasets and details of all their fields can be found in the Data Schema.

This request can also be made via a proxy PHP script. An example of how the script could be written is shown below.

<?php

$user_id = '345';

$instance[0] = 0;

$data = array();
$data['field_ds_home'][$instance[0]]['field_home_bedrooms'] = '2';
$data['field_ds_home'][$instance[0]]['field_home_owner_name'] = 'I. M. Landlord';
$data = (is_array($data)) ? http_build_query($data) : $data;

$arguments_array = array(
  'key=5FXfidz9zw6YQaO2kWh0B154zz6wafrV',
  'api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf',
  'con_id=345-1545',
  'source_type=connection',
  'dataset=ds_home',
  'instance=0',
  'id=1'
);
$arguments = implode('&', $arguments_array);

$mydex_api_url = "https://sbx-api.mydex.org/api/pds/pds/" . $user_id . ".json?" . $arguments;

$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, $mydex_api_url);
curl_setopt($ch, CURLOPT_HTTPHEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PUT');
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);

$mydex_request = curl_exec($ch);
$mydex_request_decode = json_decode($mydex_request);

curl_close($ch);

?>

Example Response 🔗︎ click to copy

A successful update request via PUT will return a status code of 200 OK and echo the updated fields. For example, a response to the previous example will yield:

{
    "success": "1",
    "transaction": {
        "field_home_bedrooms": "2",
        "field_home_owner_name": "I. M. Landlord"
    }
}

If unsuccessful, an error will be returned with a suggestion on how to correct the API request.

Creating a new row 🔗︎ click to copy

If the PDS dataset has no records eg. returns an empty array when making a GET request to the API, you will need to create a row. This can be done using a POST request.

curl
    -X POST
    -H "Content-Type: application/json"
    -d '{
         "ds_home": {
            "field_home_bedrooms": "3",
            "field_home_country": "England",
            "field_home_county": "Essex",
            "field_home_date_from": "1327398114",
            "field_home_date_to": "1611481314",
            "field_home_num_residents": "4",
            "field_home_owner_country": "United Kingdom",
            "field_home_owner_county": "Anycounty",
            "field_home_owner_email_address": "landlord@landlord.com",
            "field_home_owner_name": "I. M. Landlord",
            "field_home_owner_postcode": "AB12 3CD"
         }
      }'
    'https://sbx-api.mydex.org/api/pds/pds/345.json?
        key=5FXfidz9zw6YQaO2kWh0B154zz6wafrV
        &api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf
        &con_id=345-1545
        &dataset=ds_home
        &source_type=connection
        &instance=0

If successful, the API will return the following response:

{
    "success": "1",
    "last_insert_id": "1"
}

Note: you will need to use the value of 'last_insert_id' when updating the dataset. For Metadata datasets this will normally always be '1'.

Transactional Datasets 🔗︎ click to copy

Retrieving Data 🔗︎ click to copy

Example API Request 🔗︎ click to copy

https://sbx-api.mydex.org/api/pds/transaction/345.json
   ?key=5FXfidz9zw6YQaO2kWh0B154zz6wafrV
   &api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf
   &con_id=345-1545
   &source_type=connection
   &dataset=ds_browsing_history
   &instance=0

Pastable:

https://sbx-api.mydex.org/api/pds/transaction/345.json?key=5FXfidz9zw6YQaO2kWh0B154zz6wafrV&api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf&con_id=345-1545&source_type=connection&dataset=ds_browsing_history&instance=0

Example Response 🔗︎ click to copy

{
   "1":{
      "id":"1",
      "ds_id":"1",
      "browsing_history_url":"http://www.youtube.com",
      "browsing_history_title":"YouTube",
      "browsing_history_visit_date":"1397550508"
   },
   "2":{
      "id":"2",
      "ds_id":"2",
      "browsing_history_url":"http://www.youtube.com",
      "browsing_history_title":"YouTube",
      "browsing_history_visit_date":"1397550512"
   }
}

Adding Data 🔗︎ click to copy

The key changes when adding a transactional dataset are outlined in the table above.

Example API Request 🔗︎ click to copy

curl
   -X POST
   -H "Content-Type: application/json"
   -d '{
         "ds_browsing_history": {
            "browsing_history_url": "http://www.youtube.com",
            "browsing_history_title": "YouTube",
            "browsing_history_visit_date": "1397550510"
         }
      }'
   'https://sbx-api.mydex.org/api/pds/transaction.json
      ?uid=345
      &key=YfMPI2iSFcBDQ5CdlJVQeXIbIUD156s0
      &api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf
      &con_id=345-1545
      &source_type=connection
      &dataset=ds_browsing_history
      &instance=0'

This request can also be made via a proxy PHP script. An example of how the script could be written is shown below.

<?php

$user_id = '345';

$instance[0] = 0;

$data = array();

$data['ds_browsing_history']['browsing_history_url'] = 'http://www.youtube.com';
$data['ds_browsing_history']['browsing_history_title'] = 'YouTube';
$data['ds_browsing_history']['browsing_history_visit_date'] = '1397550510';


$data = (is_array($data)) ? http_build_query($data) : $data;

$arguments_transactional_array = array(
'uid=' . $user_id,
'api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf',
'source_type=connection',
'con_id=345-1545',
'key=5FXfidz9zw6YQaO2kWh0B154zz6wafrV',
'instance=' . $instance[0],
'dataset=ds_browsing_history',
);
$arguments_transactional = implode('&', $arguments_transactional_array);

$mydex_api_url = "https://sbx-api.mydex.org/api/pds/transaction.json?" . $arguments_transactional;

$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, $mydex_api_url);
curl_setopt($ch, CURLOPT_HTTPHEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'POST');
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);

$mydex_request = curl_exec($ch);
$mydex_request_decode = json_decode($mydex_request);

curl_close($ch);

?>

Example Response 🔗︎ click to copy

A successful request will return a HTTP status code of 200 OK and the line {"success":1}. An unsuccessful request will return an HTTP error code and a message explaining the problem.

Adding Batch Data 🔗︎ click to copy

If you wish to batch-send multiple rows of data in a single request, then you can used a nested array of data in the request body. The raw JSON would look like this:

{
   "ds_energy_consumption": [
      {
         "created_timestamp": "1641909333",
         "field_energy_homeid": "1",
         "field_energy_reading_timestamp": "1648633780",
         "field_energy_reading_value": "1123.25",
         "field_energy_source_index_id": "1",
         "field_energy_source_name": "Kettle",
         "field_energy_source_type": "CAD",
         "updated_timestamp": "1641909333"
      },
      {
         "created_timestamp": "1641909333",
         "field_energy_homeid": "1",
         "field_energy_reading_timestamp": "1648633780",
         "field_energy_reading_value": "1153.50",
         "field_energy_source_index_id": "1",
         "field_energy_source_name": "Kettle",
         "field_energy_source_type": "CAD",
         "updated_timestamp": "1641909333"
      }
   ]
}

Example Batch API Request 🔗︎ click to copy

curl
   -X POST
   -H "Content-Type: application/json"
   -d '{
      "ds_energy_consumption": [
         {
            "created_timestamp": "1641909333",
            "field_energy_homeid": "1",
            "field_energy_reading_timestamp": "1648633780",
            "field_energy_reading_value": "1123.25",
            "field_energy_source_index_id": "1",
            "field_energy_source_name": "Kettle",
            "field_energy_source_type": "CAD",
            "updated_timestamp": "1641909333"
         },
         {
            "created_timestamp": "1641909333",
            "field_energy_homeid": "1",
            "field_energy_reading_timestamp": "1648633780",
            "field_energy_reading_value": "1153.50",
            "field_energy_source_index_id": "1",
            "field_energy_source_name": "Kettle",
            "field_energy_source_type": "CAD",
            "updated_timestamp": "1641909333"
         }
      ]
   }'
   'https://sbx-api.mydex.org/api/pds/transaction.json
      ?uid=345
      &key=YfMPI2iSFcBDQ5CdlJVQeXIbIUD156s0
      &api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf
      &con_id=345-1545
      &source_type=connection
      &dataset=ds_browsing_history
      &instance=0'

Note: there is a limit of 1000 rows of data per request.

The request is exactly the same as the one above except we now have a nested array in the data payload. The parameters and endpoints are the same as adding data.

Updating Data 🔗︎ click to copy

Example API Request 🔗︎ click to copy

curl
   -X PUT
   -H "Content-Type: application/json"
   -d '{
         "browsing_history_url": "http://www.youtube.com",
         "browsing_history_title": "YouTube",
         "browsing_history_visit_date": "1397550510"
      }'
   'https://sbx-api.mydex.org/api/pds/transaction/345.json
      &key=YfMPI2iSFcBDQ5CdlJVQeXIbIUD156s0
      &api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf
      &con_id=345-1545
      &source_type=connection
      &dataset=ds_browsing_history
      &instance=0
      &id=3'

This request can also be made via a proxy PHP script. An example of how the script could be written is shown below.

<?php

$user_id = '345';

$instance[0] = 0;

$data = array();

$data['browsing_history_url'] = 'http://www.youtube.com';
$data['browsing_history_title'] = 'YouTube';
$data['browsing_history_visit_date'] = '1397550510';


$data = (is_array($data)) ? http_build_query($data) : $data;

$arguments_transactional_array = array(
'uid=' . $user_id,
'api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf',
'source_type=connection',
'con_id=345-1545',
'key=5FXfidz9zw6YQaO2kWh0B154zz6wafrV',
'instance=' . $instance[0],
'dataset=ds_browsing_history',
'id=3'
);
$arguments_transactional = implode('&', $arguments_transactional_array);

$mydex_api_url = "https://sbx-api.mydex.org/api/pds/transaction/345.json?" . $arguments_transactional;

$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, $mydex_api_url);
curl_setopt($ch, CURLOPT_HTTPHEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PUT');
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);

$mydex_request = curl_exec($ch);
$mydex_request_decode = json_decode($mydex_request);

curl_close($ch);

?>

For more information on how to structure the data for transactional data sets, see the Data Schema.

Example Response 🔗︎ click to copy

A successful update request will return a HTTP status code of 200 OK and the line {"success":1}. An unsuccessful request will return an HTTP error code and a message explaining the problem.

Filtering Datasets 🔗︎ click to copy

The following code shows an example READ request filtered by ID with a value >= 21. For example, this could be the the value of the record returned from the previous query.

https://sbx-api.mydex.org/api/pds/transaction/345.json
   ?key=YfMPI2iSFcBDQ5CdlJVQeXIbIUD156s0
   &api_key=5FXfidz9zw6YQaO2kWh0B154zz6wafrV
   &con_id=345-1545
   &source_type=connection
   &instance=0
   &dataset=ds_energy_consumption
   &filters[id][operator]>=
   &filters[id][value]=21 // eg.the previous id

Note the last two parameters &filters[id][operator]>= and &filters[id][value]=21.

A typical request would look like the following:

{{PDS_API_URL}}/api/pds/transaction/3490.json?key=xxxxxxxxxxxxxxxxxxxxxxxxxxxx&instance=0&api_key=xxxxxxxxxxxxxxxxxxxxxx&con_id=3490-40212&source_type=connection&dataset=ds_energy_consumption&filters[id][operator]>=&filters[id][value]=5

Any of the following operators can be used to filter results:

('=', '<', '<=', '>', '>=', 'LIKE', 'IN', 'BETWEEN');

When using operators like BETWEEN (where we have two values) we need two value parameters e.g. &filters[{field_name}][value][] and &filters[{field_name}][value][] (Note the empty square brackets to define the array).

For example, if we wanted to get rows between 3 and 5, we'd have the operator &filters[id][operator]=BETWEEN and the two values: &filters[id][value][]=3 and &filters[id][value][]=5.

The request would look like this:

{{PDS_API_URL}}/api/pds/transaction/3490.json?key=xxxxxxxxxxxxxxxxxxxxxxxxxxxx&instance=0&api_key=xxxxxxxxxxxxxxxxxxxxxxxxxxxx&con_id=3490-40212&source_type=connection&dataset=ds_energy_consumption&filters[id][operator]=BETWEEN&filters[id][value][]=3&filters[id][value][]=5