Using the PDX API
Before an API call to a PDS can be made, the connection between the members PDS and the connecting service must be authenticated. The details of the authentication process can be found here.
Dataset Types
A Mydex member's stored data forms are divided into two categories: Metadata Datasets and Transactional Datasets. The distinction between these datasets is detailed in the Data Schema.
Return Format
The Personal Data eXchange API can return a response to an API request in either JSON or XML. The formats are interchangeable and are chosen by putting the desired file extension in the API request (eg. .json
or .xml
). In the following examples we use JSON.
Multiple Dataset Instances
Occasionally, a member will have more than one dataset instance in their PDS (eg. multiple home addresses). In this situation, a PDX API request will return all dataset instances that the member has granted you permission to access. These datasets are identified by the key instance_N
, where N is the Nth dataset instance, starting from N=0.
Filtering Datasets
Returning a whole dataset might not be practical or desirable. For example, if you want to return only the latest values within a dataset where changes have occurred since the last time this dataset was queried.
This can be overcome by filtering the results. Filtering results is achieved by passing two additional query parameters in the request: filters[{field_name}][operator]
and &filters[{field_name}][value]
.
For example, if you wanted to get dataset rows greater than a specific ID value, you would add parameters like this: &filters[id][operator]>=
and &filters[id][value]=21
, where 21 is the ID value you are filtering against.
For more details on filtering plus example code, please see under the heading Filtering Datasets at the bottom of this page.
PDX API Request Differences
The key differences when making a PDX API request for each dataset type are summarised in the table below.
Metadata Dataset | Transactional Dataset | |
---|---|---|
About | Multiple datasets with potentially multiple dataset instances | Multiple datasets with potentially multiple dataset instances |
Availability | On Production (https://api.mydex.org) & Sandbox (https://sbx-api.mydex.org) servers | On Production & Sandbox servers |
URL | Uses the api/pds/pds path structure |
Uses the api/pds/transaction path structure |
When Retrieving Data | The uid (345 in the following examples) is part of the path structure |
The uid is part of the path structure |
The dataset instance is not included in the request |
The dataset instance must be included as a parameter (eg. &instance=0 ) |
|
The dataset is passed as parameter |
The dataset is passed as parameter |
|
When Adding Data | The uid is part of the path structure |
The uid is passed as a parameter (eg. ?uid=345 ) |
The dataset instance must be included as a parameter |
The dataset instance must be included as a parameter |
|
The dataset must not be included in the data |
The dataset must be included in the data and as a parameter |
|
Updates are made using HTTP PUT | Updates are made using HTTP POST | |
When Filtering Datasets | Results can be filtered by using one of the following operators: ('=', '<', '<=', '>', '>=', 'LIKE', 'IN', 'BETWEEN'); |
Results can be filtered by using one of the following operators: ('=', '<', '<=', '>', '>=', 'LIKE', 'IN', 'BETWEEN'); |
Working with assets (files) | See link to dedicated documentation |
In addition to the above, PDX API requests should also include the key
and source_type=connection
values as parameters.
Remember: All requests to the PDX API must send a valid OAuth2.0 access token (scoped to mydex:pdx
) as the Authorization header, e.g: Authorization: Bearer [token]
. Contact us if you have not received OAuth2.0 credentials.
Error responses
Although Mydex strives to prevent such issues, there is always the chance that the PDX API may return an error when attempting to read or write to a PDS.
Typically, this occurs if there is a syntax error with your payload, and we recommend you double check the examples in this documentation to avoid such cases.
Common errors
An invalid dataset in the request may return an error such as:
{ "error": "2: Table does not exist" }
Or, for example, an incorrect connection ID:
{ "error": "No connection access record." }
Or an incorrect PDS:
{ "error": "PDS does not exist." }
Trying to access a dataset that your dedicated connection or member has not granted permission for your connection to access:
{ "error": "Permission denied (The user has not granted you access to this dataset or instance)" }
If the error is due to some other exception when reading from or writing to the PDS, a general error may look like this:
{ "error": "Problem writing to the PDS, request_id: 24c6331b-1f70-47ee-92a0-a7deb847a21d" }
Locking and contention on a PDS
A contention issue (multiple requests to the same PDS at the same time) or a backend high availability storage issue can, in rare cases, temporarily impede access to the PDS.
Most requests to a PDS - even large batch inserts - typically take less than 1 second to complete, so contention issues between simultaneous requests to the same PDS are rare.
Mydex's PDX API allows up to 40 seconds for a lock on the PDS to clear. If the lock is still not cleared after that time, the PDX API will back off and then retry (once only) the same request, up to another 40 seconds. This all happens as part of the one HTTP request your application has made. Please allow for a total of up to 180s for your request to time out at Mydex's side before attempting to re-send the request.
If the PDS is still locked after the retry that the PDX API itself performs as part of that same single request, the PDX API will return an error in the HTTP response like this:
{ "error": "The PDS was busy, please retry again later, request_id: 4c19129b-e135-4972-8809-76f88fed554a" }
Mydex recommends that subscribers only attempt to re-send the same payload if they were returned a 502/503 timeout error, or another HTTP response that contains one of the errors above. It is also recommended to wait and retry a few minutes after such an incident, to increase the likelihood of the issue having been addressed.
If you are not sure whether to re-send a request, consider first attempting a GET (retrieve) request to check the last row in the dataset. If the data is not that of your last request, it may be that you should re-send the request - however, be aware that other subscribers may have a data sharing agreement with the member and so may be the originators of the last row (in which case, your last row might be earlier in the list) - so this technique is not a guarantee that your data should be re-sent.
If you receive an error, are not sure what to do, and it contains a request_id in the error message, please feel free to contact Mydex and we can cross-reference the request ID in our logs to help you troubleshoot the issue.
Examples
Example PDX API requests for each specific dataset, and details of dataset fields can be found in the Data Schema. General examples for each dataset type can be found below.
Metadata Datasets
Retrieving Data
Retrieving data is performed via GET HTTP method requests to the PDX API.
Example API Requests
PHP example
Click here to view the PHP example
Python example
Click here to view the Python example
Postman example
Click here to view the Postman example
Example Response
"1": { "id": "1", "field_personal_email_address": "john@example.com", "field_personal_faname": "Citizen", "field_personal_fname": "John", "field_personal_gender": "Male", "field_personal_marital_status": "Married", "field_personal_mobile_phone": "+441234567890", }
Note: not all of the fields have been included in this example to improve readability.
Updating Data
Updating data for existing records is performed via PUT HTTP method requests to the PDX API.
Note: you can only update a record that exists. We recommend you first make a GET request to the dataset determine if the record exists. You will need the ID of the record you wish to update. If the record does not exist, you must first POST a record to the dataset.
Example API Requests
PHP example
Click here to view the PHP example
Python example
Click here to view the Python example
Postman example
Click here to view the Postman example
Example Response
A successful update request via PUT will return a status code of 200 OK and echo the updated fields. For example:
{ "success": "1", "transaction": { "field_personal_marital_status": "Divorced" } }
If unsuccessful, the 'success' attribute in the response will be '0'.
Creating a new row
If the PDS dataset has no records eg. returns an empty array when making a GET request to the API, you will need to create a row.
Adding an entirely new set of values for an empty dataset is performed via POST HTTP method requests to the PDX API.
Example API Requests
PHP example
Click here to view the PHP example
Python example
Click here to view the Python example
Postman example
Click here to view the Postman example
If successful, the API will return the following response:
{ "success": "1", "last_insert_id": "1" }
Note: you will need to use the value of 'last_insert_id' when updating the dataset. For Metadata datasets this will normally always be '1'.
Transactional Datasets
Retrieving Data
Retrieving rows of transactional data is performed via GET HTTP method requests to the PDX API.
Example API Requests
PHP example
Click here to view the PHP example
Python example
Click here to view the Python example
Postman example
Click here to view the Postman example
Example Response
{ "1":{ "id":"1", "browsing_history_url":"http://www.youtube.com", "browsing_history_title":"YouTube", "browsing_history_visit_date":"1397550508" }, "2":{ "id":"2", "browsing_history_url":"http://www.youtube.com", "browsing_history_title":"YouTube", "browsing_history_visit_date":"1397550512" } }
Adding Data
Adding a new row to a transactional dataset is performed via POST HTTP method requests to the PDX API.
The key changes when adding a transactional dataset are outlined in the table above.
Example API Requests
PHP example
Click here to view the PHP example
Python example
Click here to view the Python example
Postman example
Click here to view the Postman example
Example Response
A successful request will return a HTTP status code of 200 OK
and the line {"success":1}
. If unsuccessful, the 'success' attribute in the response will be '0'.
Adding Batch Data
If you wish to batch-send multiple rows of data in a single request, then you can POST a nested array of data in the request body. The raw JSON would look like this:
{ "ds_energy_consumption": [ { "created_timestamp": "1641909333", "field_energy_homeid": "1", "field_energy_reading_timestamp": "1648633780", "field_energy_reading_value": "1123.25", "field_energy_source_index_id": "1", "field_energy_source_name": "Kettle", "field_energy_source_type": "CAD", "updated_timestamp": "1641909333" }, { "created_timestamp": "1641909333", "field_energy_homeid": "1", "field_energy_reading_timestamp": "1648633780", "field_energy_reading_value": "1153.50", "field_energy_source_index_id": "1", "field_energy_source_name": "Kettle", "field_energy_source_type": "CAD", "updated_timestamp": "1641909333" } ] }
Example Batch API Requests
PHP example
Click here to view the PHP example
Python example
Click here to view the Python example
Postman example
Click here to view the Postman example
Note: there is a limit of 1000 rows of data per batch request.
The request is exactly the same as the one above except we now have a nested array in the data payload. The parameters and endpoints are the same as adding data.
Updating Data
Updating values for an existing row in a transactional dataset is performed via PUT HTTP method requests to the PDX API.
Example API Requests
PHP example
Click here to view the PHP example
Python example
Click here to view the Python example
Postman example
Click here to view the Postman example
For more information on how to structure the data for transactional data sets, see the Data Schema.
Example Response
A successful update request will return a HTTP status code of 200 OK
and the line {"success":1}
. If unsuccessful, the 'success' attribute in the response will be '0'.
Filtering Datasets
The following code shows an example READ request filtered by ID with a value >= 21. For example, this could be the the value of the record returned from the previous query.
https://sbx-api.mydex.org/api/pds/transaction/345.json ?key=YfMPI2iSFcBDQ5CdlJVQeXIbIUD156s0 &con_id=345-1545 &source_type=connection &instance=0 &dataset=ds_energy_consumption &filters[id][operator]>= &filters[id][value]=21 // eg.the previous id
Note the last two parameters &filters[id][operator]>=
and &filters[id][value]=21
.
A typical request would look like the following:
{{PDS_API_URL}}/api/pds/transaction/3490.json?key=xxxxxxxxxxxxxxxxxxxxxxxxxxxx&instance=0&con_id=3490-40212&source_type=connection&dataset=ds_energy_consumption&filters[id][operator]>=&filters[id][value]=5
Any of the following operators can be used to filter results:
('=', '<', '<=', '>', '>=', 'LIKE', 'IN', 'BETWEEN');
When using operators like BETWEEN
(where we have two values) we need two value parameters e.g. &filters[{field_name}][value][]
and &filters[{field_name}][value][]
(Note the empty square brackets to define the array).
For example, if we wanted to get rows between 3 and 5, we'd have the operator &filters[id][operator]=BETWEEN
and the two values: &filters[id][value][]=3
and &filters[id][value][]=5
.
The request would look like this:
{{PDS_API_URL}}/api/pds/transaction/3490.json?key=xxxxxxxxxxxxxxxxxxxxxxxxxxxx&instance=0&con_id=3490-40212&source_type=connection&dataset=ds_energy_consumption&filters[id][operator]=BETWEEN&filters[id][value][]=3&filters[id][value][]=5