Using the PDS API
Before an API call to a PDS can be made, the connection between the members PDS and the connecting service must be authenticated. The details of the authentication process can be found here.
Dataset Types
A Mydex member's stored data forms are divided into two categories: Datasets and Transactional Datasets. The distinction between these datasets is detailed in the Data Schema.
Return Format
The Connection API can return a response to an API request in either JSON or XML. The formats are interchangeable and are chosen by putting the desired file extension in the API request (eg. .json
or .xml
). In the following examples we use JSON.
Multiple Datasets
Connection API requests can be made to multiple datasets by using the +
operand between datasets. For instance, a member's home details and personal details can be retrieved by using the following parameter in the API request: dataset=field_ds_home+field_ds_personal_details
.
Multiple Dataset Instances
Occasionally, a member will have more than one dataset instance in their PDS (eg. multiple home addresses). In this situation, a Connection API request will return all dataset instances that the member has granted you permission to access. These datasets are identified by the key instance_N
, where N is the Nth dataset instance, starting from N=0.
Filtering Datasets
Returning a whole dataset might not be practical or desirable. For example, if you want to return only the latest values within a dataset where changes have occurred since the last time this dataset was queried.
This can be overcome by filtering the results. Filtering results is achieved by passing two additional query parameters in the request: filters[{field_name}][operator]
and &filters[{field_name}][value]
.
For example, if you wanted to get dataset rows greater than a specific ID value, you would add parameters like this: &filters[id][operator]>=
and &filters[id][value]=21
, where 21 is the ID value you are filtering against.
For more details on filtering plus example code, please see under the heading Filtering Datasets at the bottom of this page.
Connection API Request Differences
The key differences when making a Connection API request for each dataset type are summarised in the table below.
Dataset | Transactional Dataset | |
---|---|---|
About | Multiple datasets with potentially multiple dataset instances | Multiple datasets with potentially multiple dataset instances |
Availability | On Production (https://api.mydex.org) & Sandbox (https://sbx-api.mydex.org) servers | On Production & Sandbox servers |
URL | Uses the api/pds/pds path structure |
Uses the api/pds/transaction path structure |
When Retrieving Data | The uid (345 in the following examples) is part of the path structure |
The uid is part of the path structure |
The dataset instance is not included in the request |
The dataset instance must be included as a parameter (eg. &instance=0 ) |
|
The dataset is passed as parameter |
The dataset is passed as parameter |
|
When Adding Data | The uid is part of the path structure |
The uid is passed as a parameter (eg. ?uid=345 ) |
The dataset instance must be included as a parameter |
The dataset instance must be included as a parameter |
|
The dataset must not be included in the data |
The dataset must be included in the data and as a parameter |
|
Updates are made using HTTP PUT | Updates are made using HTTP POST | |
When Filtering Datasets | Results can be filtered by using one of the following operators: ('=', '<', '<=', '>', '>=', 'LIKE', 'IN', 'BETWEEN'); |
Results can be filtered by using one of the following operators: ('=', '<', '<=', '>', '>=', 'LIKE', 'IN', 'BETWEEN'); |
Working with assets (files) | See link to dedicated documentation |
In addition to the above, Connection API requests should also include the key
, api_key
and source_type=connection
values as parameters.
Examples
Example Connection API requests for each specific dataset, and details of dataset fields can be found in the Data Schema. General examples for each dataset type can be found below.
Datasets
Retrieving Data
Example API Request
https://sbx-api.mydex.org/api/pds/pds/345.json ?key=YfMPI2iSFcBDQ5CdlJVQeXIbIUD156s0 &api_key=5FXfidz9zw6YQaO2kWh0B154zz6wafrV &con_id=345-1545 &source_type=connection &dataset=ds_home &instance=0
Pastable:
https://sbx-api.mydex.org/api/pds/pds/345.json?key=5FXfidz9zw6YQaO2kWh0B154zz6wafrV&api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf&con_id=345-1545&source_type=connection&dataset=field_ds_home
Example Response
"1": { "id": "1", "field_home_bedrooms": "3", "field_home_country": "United Kingdom", "field_home_county": "Fake County", "field_home_date_from": "1327398114", "field_home_date_to": "1611481314", "field_home_habitable_rooms": "2", "field_home_owner_street": "Fake Street", "field_home_owner_town": "Fake Town", "field_home_postcode": "G92 3KJ", "field_home_property_type": "House", "field_home_street": "18 Fake Street", "field_home_tenancy": "Lease Tenant", },
Note: not all of the fields have been included in this example to improve readability.
Updating Data
It is possible to update numerous fields in multiple dataset instances with your own data by issuing a PUT request to the Connection API.
Note: you can only update a record that exists. We recommend you first make a GET request to the dataset determine if the record exists. You will need the ID of the record you wish to update.
Example API Request
As you saw in the previous example, the dummy member we created was given a dataset instance. We will now submit an API request to update select details in the dataset instance.
Many browsers don't natively support PUT, so the following example uses curl to make a PUT request. This example request has been restructured to improve readability.
curl -X PUT -H "Content-Type: application/json" -d '{ "field_home_bedrooms": "2", "field_home_owner_name": "I. M. Landlord" }' 'https://sbx-api.mydex.org/api/pds/pds/345.json? key=5FXfidz9zw6YQaO2kWh0B154zz6wafrV &api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf &con_id=345-1545 &dataset=ds_home &source_type=connection &instance=0 &id=1'
This request changes the field_home_bedrooms
field value and adds the field_home_owner_name
field value. Tables of all available datasets and details of all their fields can be found in the Data Schema.
This request can also be made via a proxy PHP script. An example of how the script could be written is shown below.
<?php $user_id = '345'; $instance[0] = 0; $data = array(); $data['field_ds_home'][$instance[0]]['field_home_bedrooms'] = '2'; $data['field_ds_home'][$instance[0]]['field_home_owner_name'] = 'I. M. Landlord'; $data = (is_array($data)) ? http_build_query($data) : $data; $arguments_array = array( 'key=5FXfidz9zw6YQaO2kWh0B154zz6wafrV', 'api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf', 'con_id=345-1545', 'source_type=connection', 'dataset=ds_home', 'instance=0', 'id=1' ); $arguments = implode('&', $arguments_array); $mydex_api_url = "https://sbx-api.mydex.org/api/pds/pds/" . $user_id . ".json?" . $arguments; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $mydex_api_url); curl_setopt($ch, CURLOPT_HTTPHEADER, 0); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PUT'); curl_setopt($ch, CURLOPT_POSTFIELDS, $data); $mydex_request = curl_exec($ch); $mydex_request_decode = json_decode($mydex_request); curl_close($ch); ?>
Example Response
A successful update request via PUT will return a status code of 200 OK and echo the updated fields. For example, a response to the previous example will yield:
{ "success": "1", "transaction": { "field_home_bedrooms": "2", "field_home_owner_name": "I. M. Landlord" } }
If unsuccessful, an error will be returned with a suggestion on how to correct the API request.
Creating a new row
If the PDS dataset has no records eg. returns an empty array when making a GET request to the API, you will need to create a row. This can be done using a POST request.
curl -X POST -H "Content-Type: application/json" -d '{ "ds_home": { "field_home_bedrooms": "3", "field_home_country": "England", "field_home_county": "Essex", "field_home_date_from": "1327398114", "field_home_date_to": "1611481314", "field_home_num_residents": "4", "field_home_owner_country": "United Kingdom", "field_home_owner_county": "Anycounty", "field_home_owner_email_address": "landlord@landlord.com", "field_home_owner_name": "I. M. Landlord", "field_home_owner_postcode": "AB12 3CD" } }' 'https://sbx-api.mydex.org/api/pds/pds/345.json? key=5FXfidz9zw6YQaO2kWh0B154zz6wafrV &api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf &con_id=345-1545 &dataset=ds_home &source_type=connection &instance=0
If successful, the API will return the following response:
{ "success": "1", "last_insert_id": "1" }
Note: you will need to use the value of 'last_insert_id' when updating the dataset. For Metadata datasets this will normally always be '1'.
Transactional Datasets
Retrieving Data
Example API Request
https://sbx-api.mydex.org/api/pds/transaction/345.json ?key=5FXfidz9zw6YQaO2kWh0B154zz6wafrV &api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf &con_id=345-1545 &source_type=connection &dataset=ds_browsing_history &instance=0
Pastable:
https://sbx-api.mydex.org/api/pds/transaction/345.json?key=5FXfidz9zw6YQaO2kWh0B154zz6wafrV&api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf&con_id=345-1545&source_type=connection&dataset=ds_browsing_history&instance=0
Example Response
{ "1":{ "id":"1", "ds_id":"1", "browsing_history_url":"http://www.youtube.com", "browsing_history_title":"YouTube", "browsing_history_visit_date":"1397550508" }, "2":{ "id":"2", "ds_id":"2", "browsing_history_url":"http://www.youtube.com", "browsing_history_title":"YouTube", "browsing_history_visit_date":"1397550512" } }
Adding Data
The key changes when adding a transactional dataset are outlined in the table above.
Example API Request
curl -X POST -H "Content-Type: application/json" -d '{ "ds_browsing_history": { "browsing_history_url": "http://www.youtube.com", "browsing_history_title": "YouTube", "browsing_history_visit_date": "1397550510" } }' 'https://sbx-api.mydex.org/api/pds/transaction.json ?uid=345 &key=YfMPI2iSFcBDQ5CdlJVQeXIbIUD156s0 &api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf &con_id=345-1545 &source_type=connection &dataset=ds_browsing_history &instance=0'
This request can also be made via a proxy PHP script. An example of how the script could be written is shown below.
<?php $user_id = '345'; $instance[0] = 0; $data = array(); $data['ds_browsing_history']['browsing_history_url'] = 'http://www.youtube.com'; $data['ds_browsing_history']['browsing_history_title'] = 'YouTube'; $data['ds_browsing_history']['browsing_history_visit_date'] = '1397550510'; $data = (is_array($data)) ? http_build_query($data) : $data; $arguments_transactional_array = array( 'uid=' . $user_id, 'api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf', 'source_type=connection', 'con_id=345-1545', 'key=5FXfidz9zw6YQaO2kWh0B154zz6wafrV', 'instance=' . $instance[0], 'dataset=ds_browsing_history', ); $arguments_transactional = implode('&', $arguments_transactional_array); $mydex_api_url = "https://sbx-api.mydex.org/api/pds/transaction.json?" . $arguments_transactional; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $mydex_api_url); curl_setopt($ch, CURLOPT_HTTPHEADER, 0); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'POST'); curl_setopt($ch, CURLOPT_POSTFIELDS, $data); $mydex_request = curl_exec($ch); $mydex_request_decode = json_decode($mydex_request); curl_close($ch); ?>
Example Response
A successful request will return a HTTP status code of 200 OK
and the line {"success":1}
. An unsuccessful request will return an HTTP error code and a message explaining the problem.
Adding Batch Data
If you wish to batch-send multiple rows of data in a single request, then you can used a nested array of data in the request body. The raw JSON would look like this:
{ "ds_energy_consumption": [ { "created_timestamp": "1641909333", "field_energy_homeid": "1", "field_energy_reading_timestamp": "1648633780", "field_energy_reading_value": "1123.25", "field_energy_source_index_id": "1", "field_energy_source_name": "Kettle", "field_energy_source_type": "CAD", "updated_timestamp": "1641909333" }, { "created_timestamp": "1641909333", "field_energy_homeid": "1", "field_energy_reading_timestamp": "1648633780", "field_energy_reading_value": "1153.50", "field_energy_source_index_id": "1", "field_energy_source_name": "Kettle", "field_energy_source_type": "CAD", "updated_timestamp": "1641909333" } ] }
Example Batch API Request
curl -X POST -H "Content-Type: application/json" -d '{ "ds_energy_consumption": [ { "created_timestamp": "1641909333", "field_energy_homeid": "1", "field_energy_reading_timestamp": "1648633780", "field_energy_reading_value": "1123.25", "field_energy_source_index_id": "1", "field_energy_source_name": "Kettle", "field_energy_source_type": "CAD", "updated_timestamp": "1641909333" }, { "created_timestamp": "1641909333", "field_energy_homeid": "1", "field_energy_reading_timestamp": "1648633780", "field_energy_reading_value": "1153.50", "field_energy_source_index_id": "1", "field_energy_source_name": "Kettle", "field_energy_source_type": "CAD", "updated_timestamp": "1641909333" } ] }' 'https://sbx-api.mydex.org/api/pds/transaction.json ?uid=345 &key=YfMPI2iSFcBDQ5CdlJVQeXIbIUD156s0 &api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf &con_id=345-1545 &source_type=connection &dataset=ds_browsing_history &instance=0'
Note: there is a limit of 1000 rows of data per request.
The request is exactly the same as the one above except we now have a nested array in the data payload. The parameters and endpoints are the same as adding data.
Updating Data
Example API Request
curl -X PUT -H "Content-Type: application/json" -d '{ "browsing_history_url": "http://www.youtube.com", "browsing_history_title": "YouTube", "browsing_history_visit_date": "1397550510" }' 'https://sbx-api.mydex.org/api/pds/transaction/345.json &key=YfMPI2iSFcBDQ5CdlJVQeXIbIUD156s0 &api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf &con_id=345-1545 &source_type=connection &dataset=ds_browsing_history &instance=0 &id=3'
This request can also be made via a proxy PHP script. An example of how the script could be written is shown below.
<?php $user_id = '345'; $instance[0] = 0; $data = array(); $data['browsing_history_url'] = 'http://www.youtube.com'; $data['browsing_history_title'] = 'YouTube'; $data['browsing_history_visit_date'] = '1397550510'; $data = (is_array($data)) ? http_build_query($data) : $data; $arguments_transactional_array = array( 'uid=' . $user_id, 'api_key=jPssYUMVVNaLaYPIbY7P9ESGzdksCOlf', 'source_type=connection', 'con_id=345-1545', 'key=5FXfidz9zw6YQaO2kWh0B154zz6wafrV', 'instance=' . $instance[0], 'dataset=ds_browsing_history', 'id=3' ); $arguments_transactional = implode('&', $arguments_transactional_array); $mydex_api_url = "https://sbx-api.mydex.org/api/pds/transaction/345.json?" . $arguments_transactional; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $mydex_api_url); curl_setopt($ch, CURLOPT_HTTPHEADER, 0); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PUT'); curl_setopt($ch, CURLOPT_POSTFIELDS, $data); $mydex_request = curl_exec($ch); $mydex_request_decode = json_decode($mydex_request); curl_close($ch); ?>
For more information on how to structure the data for transactional data sets, see the Data Schema.
Example Response
A successful update request will return a HTTP status code of 200 OK
and the line {"success":1}
. An unsuccessful request will return an HTTP error code and a message explaining the problem.
Filtering Datasets
The following code shows an example READ request filtered by ID with a value >= 21. For example, this could be the the value of the record returned from the previous query.
https://sbx-api.mydex.org/api/pds/transaction/345.json ?key=YfMPI2iSFcBDQ5CdlJVQeXIbIUD156s0 &api_key=5FXfidz9zw6YQaO2kWh0B154zz6wafrV &con_id=345-1545 &source_type=connection &instance=0 &dataset=ds_energy_consumption &filters[id][operator]>= &filters[id][value]=21 // eg.the previous id
Note the last two parameters &filters[id][operator]>=
and &filters[id][value]=21
.
A typical request would look like the following:
{{PDS_API_URL}}/api/pds/transaction/3490.json?key=xxxxxxxxxxxxxxxxxxxxxxxxxxxx&instance=0&api_key=xxxxxxxxxxxxxxxxxxxxxx&con_id=3490-40212&source_type=connection&dataset=ds_energy_consumption&filters[id][operator]>=&filters[id][value]=5
Any of the following operators can be used to filter results:
('=', '<', '<=', '>', '>=', 'LIKE', 'IN', 'BETWEEN');
When using operators like BETWEEN
(where we have two values) we need two value parameters e.g. &filters[{field_name}][value][]
and &filters[{field_name}][value][]
(Note the empty square brackets to define the array).
For example, if we wanted to get rows between 3 and 5, we'd have the operator &filters[id][operator]=BETWEEN
and the two values: &filters[id][value][]=3
and &filters[id][value][]=5
.
The request would look like this:
{{PDS_API_URL}}/api/pds/transaction/3490.json?key=xxxxxxxxxxxxxxxxxxxxxxxxxxxx&instance=0&api_key=xxxxxxxxxxxxxxxxxxxxxxxxxxxx&con_id=3490-40212&source_type=connection&dataset=ds_energy_consumption&filters[id][operator]=BETWEEN&filters[id][value][]=3&filters[id][value][]=5