Iteration 12 Holistic Feedback

[ElizabethArnold-DSB]

This CR has been created to simplify the raising of minor changes, such as text corrections or description clarifications, that are not really material to the standards but do have real impact of readability and clarity.

Please raise any such suggestions that you would like included in Maintenance Iteration 12 and the DSB will review them. If we believe the suggestion are material we will raise a dedicated CR for them.

[perlboy]

A reminder that was lost in MI11, can the "effective date" used for oldest-date and newest-date for Get Invoices be clarified as issueDate (similar to the clarifications for Get Banking Transactions). This was stated as the intent by @HemangCDR in the Implementation call 21 July.

[CDR-API-Stream]

Sorry we missed that one @perlboy, we'll pick it up in this iteration

[CDR-API-Stream]

It was noted via a Zendesk ticket that some of the statements in the FDO section of the standards user may and must without capitalisation. This should be rectified as a simple formatting update.

[CDR-API-Stream]

The Get Data Holder Brands link is incorrect and the anchor needs to be updated (i.e. #get-data-holder-brands).

Similarly, the Get Data Recipients link needs to point to the correct Register API anchor (i.e. #get-data-recipients).

[nils-work]

The items in the Register Dependency Schedule should be shown in ascending order as per the FDO table.

[nils-work]

There is a documentation typo in ResponseOpenIDProviderConfigMetadata, there are escaped quotes around private_key_jwt -

[nils-work]

Review the Future Dated Obligations table - consider removing or greying-out items past their due date.

[nils-work]

The Profile Scope Data Language link in the Future Dated Obligations table is currently broken and should point to Profile Scope and Standard Claims: Common.

[nils-work]

Some parameters in the following sections specify field types that are not aligned to the Common Field Types definitions -

• Get Data Holder Brands
• ClientRegistrationRequest
• RegistrationProperties
• ClientRegistration
• MetaPaginated

The details have been acknowledged in the Known Issues list since v1.13.0 (Register APIs use different field type definitions).

[nils-work]

Response Headers in the Energy APIs section (under Get Generic Plans for example) show none for most header Descriptions.

These could be aligned to the convention used in the Banking APIs Response Headers (Get Accounts for example).

[nils-work]

There is a typo 'ofways' in the Standards README -

[CDR-API-Stream]

AEMO have stated that the description of validFromDate in EnergyServicePoint structure needs to be updated to the following to be more clear:

• The start date from which this service point first became valid. For service points that became valid prior to 2 years, this date will be set to the request date minus 2 years.

[perlboy]

AEMO have stated that the description of validFromDate in EnergyServicePoint structure needs to be updated to the following to be more clear:

• The start date from which this service point first became valid. For service points that became valid prior to 2 years, this date will be set to the request date minus 2 years.

Doesn't this fundamentally alter the meaning of this field? Does this mean that service points that are older than 2 years will now have a validFromDate that is constantly incrementing forward? I get the reasoning for historical data load but for perpetuity, this seems flawed.

[perlboy]

1.19.0 was just released but introduced open-status into Energy Account Detail as well? I don't believe this was part of ConsumerDataStandardsAustralia/standards#260 ?

[OpalRussAEMO]

AEMO have stated that the description of validFromDate in EnergyServicePoint structure needs to be updated to the following to be more clear:

• The start date from which this service point first became valid. For service points that became valid prior to 2 years, this date will be set to the request date minus 2 years.

Doesn't this fundamentally alter the meaning of this field? Does this mean that service points that are older than 2 years will now have a validFromDate that is constantly incrementing forward? I get the reasoning for historical data load but for perpetuity, this seems flawed.

I agree that's an issue. For background, we raised a question to clarify how this field should be populated and what a valid service point is. We have performance concerns if we need to retrieve records that are up to 20 years old to determine the original date a service point was created (which may or may not be a defaulted date), as we need 2 years of standing data to support usage APIs it was suggested that we could default the date if the service point was created more than 2 years prior.

Looking at previous consultation on this, it seems to have been suggested that the validFromDate would be the latest start date for service point details. So essentially validFromDate would be the effective date of the current set of service point details and the lastUpdateDateTime would be the date on which they were modified last. We would be happy to align with this option if preferable.

" “vaildFromDate”: the latest start date of the constituent data sets.
“lastUpdateDateTime”: the latest time any of the constituent data sets has been changed – noting that this may be from a different data-set providing the “validFromDate”. "

[perlboy]

I agree that's an issue. For background, we raised a question to clarify how this field should be populated and what a valid service point is. We have performance concerns if we need to retrieve records that are up to 20 years old to determine the original date a service point was created (which may or may not be a defaulted date), as we need 2 years of standing data to support usage APIs it was suggested that we could default the date if the service point was created more than 2 years prior.

Hmmm, I would have thought this data would be in a materialised view and not retrieved live, if it is live it will fail an NFR very quickly in an O(n2) problem as demand increases. Ultimately this is an implementation concern, I don't think it's appropriate to be altering the Standard because this isn't being persisted by the source system.

As per above, I get if this was once off but the rolling forward thing means that most service points, cause most are older and unchanged >2 years, will essentially be considered to have a transient value, this could actually exacerbate load from Data Recipients.

At a minimum it basically guarantees that the range queries from a retailer won't align with what the Data Recipient is going to get back in this field.

Looking at previous consultation on this, it seems to have been suggested that the validFromDate would be the latest start date for service point details. So essentially validFromDate would be the effective date of the current set of service point details and the lastUpdateDateTime would be the date on which they were modified last. We would be happy to align with this option if preferable.
_" “vaildFromDate”: the latest start date of the constituent data sets. “lastUpdateDateTime”: the latest time any of the constituent data sets has been changed – noting that this may be from a different data-set providing the “validFromDate”.

So the above was feedback from AEMO at the time but the DSB response was:

We defer to the AEMO's definition of this field under MSATS as this field has a direct corollary in the MSATS data.

What's interesting with that statement is that I can't actually find validFromDate in MSATS definitions, I admit I didn't look very long but does it actually exist?

Ultimately, from an implementation perspective I guess it doesn't matter ("a date is a date") but from a Data Recipient perspective how do they find out how old a meter is? That seems like a useful data point that I presumed was the purpose of this field?

[OpalRussAEMO]

So the above was feedback from AEMO at the time but the DSB response was:

We defer to the AEMO's definition of this field under MSATS as this field has a direct corollary in the MSATS data.

What's interesting with that statement is that I can't actually find validFromDate in MSATS definitions, I admit I didn't look very long but does it actually exist?

Ultimately, from an implementation perspective I guess it doesn't matter ("a date is a date") but from a Data Recipient perspective how do they find out how old a meter is? That seems like a useful data point that I presumed was the purpose of this field?

There is no such field in MSATS. We have effective from/to and updated/created dates on all standing data records which tell you what date range the information is effective for and when it was created / updated. You would not be able to tell how old a meter is by us providing a single date for when the service point was created in MSATS. Meters change over time and won't align with dates for when the service point was created or updated. Also service points are created in advance of supply being connected or metering being installed so the date on which it was created doesn't mean anything to consumers.

[CDR-API-Stream]

1.19.0 was just released but introduced open-status into Energy Account Detail as well? I don't believe this was part of ConsumerDataStandardsAustralia/standards#260 ?

Thanks @perlboy this has been identified and corrected in the 1.19.0 release.

[CDR-API-Stream]

Doesn't this fundamentally alter the meaning of this field? Does this mean that service points that are older than 2 years will now have a validFromDate that is constantly incrementing forward? I get the reasoning for historical data load but for perpetuity, this seems flawed.

This is valid feedback.

Looking at previous consultation on this, it seems to have been suggested that the validFromDate would be the latest start date for service point details. So essentially validFromDate would be the effective date of the current set of service point details and the lastUpdateDateTime would be the date on which they were modified last. We would be happy to align with this option if preferable.

This would be a better approach as it aligns with the original intent of the field. The description can be updated as below:

• validFromDate: The latest start date from which the constituent data sets of this service point became valid

[CDR-API-Stream]

It was noted via a Zendesk ticket that some of the statements in the FDO section of the standards user may and must without capitalisation. This should be rectified as a simple formatting update.

This change has been staged for review: ConsumerDataStandardsAustralia/standards-staging@b56b952

[CDR-API-Stream]

The Get Data Holder Brands link is incorrect and the anchor needs to be updated (i.e. #get-data-holder-brands).

Similarly, the Get Data Recipients link needs to point to the correct Register API anchor (i.e. #get-data-recipients).

These changes have been staged for review: ConsumerDataStandardsAustralia/standards-staging@76168ea

[CDR-API-Stream]

The items in the Register Dependency Schedule should be shown in ascending order as per the FDO table.

These changes have been staged for review: ConsumerDataStandardsAustralia/standards-staging@6db91d1

[CDR-API-Stream]

There is a documentation typo in ResponseOpenIDProviderConfigMetadata, there are escaped quotes around private_key_jwt -

This change has been staged for review: ConsumerDataStandardsAustralia/standards-staging@e63e85e

[CDR-API-Stream]

Review the Future Dated Obligations table - consider removing or greying-out items past their due date.

This change has been staged for review: ConsumerDataStandardsAustralia/standards-staging@b56b952

[CDR-API-Stream]

The Profile Scope Data Language link in the Future Dated Obligations table is currently broken and should point to Profile Scope and Standard Claims: Common.

This change has been staged for review: ConsumerDataStandardsAustralia/standards-staging@a806d41

[CDR-API-Stream]

Some parameters in the following sections specify field types that are not aligned to the Common Field Types definitions -

• Get Data Holder Brands
• ClientRegistrationRequest
• RegistrationProperties
• ClientRegistration
• MetaPaginated

The details have been acknowledged in the Known Issues list since v1.13.0 (Register APIs use different field type definitions).

Given the size of this change and need for closer review, this issue has been converted to a change request: #546

[CDR-API-Stream]

Response Headers in the Energy APIs section (under Get Generic Plans for example) show none for most header Descriptions.

These could be aligned to the convention used in the Banking APIs Response Headers (Get Accounts for example).

This change has been staged for review: ConsumerDataStandardsAustralia/standards-staging@79303fa

[CDR-API-Stream]

There is a typo 'ofways' in the Standards README -

This fix has been staged for review: ConsumerDataStandardsAustralia/standards-staging@8713da2

[CDR-API-Stream]

A reminder that was lost in MI11, can the "effective date" used for oldest-date and newest-date for Get Invoices be clarified as issueDate (similar to the clarifications for Get Banking Transactions). This was stated as the intent by @HemangCDR in the Implementation call 21 July.

This fix has been staged for review: ConsumerDataStandardsAustralia/standards-staging@57c6043

[CDR-API-Stream]

Doesn't this fundamentally alter the meaning of this field? Does this mean that service points that are older than 2 years will now have a validFromDate that is constantly incrementing forward? I get the reasoning for historical data load but for perpetuity, this seems flawed.

This is valid feedback.

Looking at previous consultation on this, it seems to have been suggested that the validFromDate would be the latest start date for service point details. So essentially validFromDate would be the effective date of the current set of service point details and the lastUpdateDateTime would be the date on which they were modified last. We would be happy to align with this option if preferable.

This would be a better approach as it aligns with the original intent of the field. The description can be updated as below:

• validFromDate: The latest start date from which the constituent data sets of this service point became valid

This fix has been staged for review: ConsumerDataStandardsAustralia/standards-staging@8a53ea2