Non-Functional Requirements Consultative Group meeting minutes – 11 September 2024

Chair Introduction

Mark Verstege, the Chair of the Non-functional Requirement Consultative Group (NFR CG) welcomed everyone to the meeting and acknowledged the traditional owners of the lands.

The Chair noted that Jim Basey (Basiq), Harish Krishnamurthy (ANZ), Thomas Lu (Origin), Mark Wallis (Skript) and Jeff Wang (NAB) were apologies for this meeting. 

Minutes

The Chair thanked the group for their comments on the minutes from 14 August 2024 meeting. The Minutes were formally accepted and will be made publicly available on the Consumer Data Standards website.

Action Items                                                    

The Chair provided an update on Action Items as follows:

• The DSB met with Xero out of session, and they will provide details of the discussion with the group via GovTEAMS
• DSB to create a survey around data collection patterns – item still outstanding
• DSB to circulate asynchronous patterns for review –item still outstanding

Mapping of Problems, Opportunities and Key Findings

Hemang Rathod from the DSB outlined an action plan focusing on moving discussion towards delivering tangible results. The plan involves summarising the problems, findings and opportunities discussed in previous meetings and proposing a draft action plan for each identified problem area as follows:

• Unoptimised/inefficient API calls
• Problems with sharing large volumes of data
• Performance tiers and NFR settings and under-utilised TPS capacity
• Projected traffic increase

The DSB asked the group to provide input via the Miro board on the “what were the key issues based on the problems being faced (the why)” to ensure a comprehensive and agreed upon action plan moving forward.

Key points included: 

• The design of endpoints may have been focused on intraday data retrieval, but use cases lean towards both intraday and historical pattern requirements.
• Current patterns are repeating retrievals of the same request every day, which seems to be designed for occasional interrogations rather than ongoing data retrieval.
• Accounts with a large number of transactions, especially business accounts, result in a large volume of data, indicating that the design might have been more consumer-focused initially.
• The off-peak window (Low Traffic Period) for customer-not-present scenarios is small. Adjusting the High Traffic Period and Low Traffic Period designations could help spread load to other parts of the day to avoid ‘best effort support’ for unattended traffic.

The DSB asked the group to provide input via the Miro board on the “key findings opportunities that have been identified through the Consultative Group”. 

Key points included: 

• Design of endpoints might have been more focused on intraday data retrieval, which does not fully support the actual use cases that lean towards both intraday and historical pattern requirements. The current pattern of repeating retrievals of the same request every day was highlighted as an area for improvement.
• The need for event-driven notification patterns that would allow for real-time updates and actions based on data changes. This would support use cases where data triggers specific workflows or actions, such as sending notifications or triggering other behaviours based on data updates.

These insights suggest a need for adjustments in API design and usage patterns to better accommodate the actual use cases observed in the ecosystem, as well as the exploration of event-driven models to enhance the efficiency and responsiveness of data retrieval and processing. 

The DSB asked the group to provide input via the Miro board on the “opportunities”. 

Key points included: 

• The need for optimising API calls to reduce the number of unnecessary requests, especially for low-velocity data sets. Suggestions included introducing bulk arrangement APIs and event-driven notification patterns to notify ADRs when data changes, thereby reducing the need for constant polling.
• The need for event-driven notification patterns was emphasised to support real-time updates and actions based on data changes. This would help in reducing the frequency of API calls by notifying ADRs only when relevant data changes occur. 
• The complexity of accurately reflecting pending transactions in running balances.
• Shedding load during spikes by being more assertive in response strategies, such as using HTTP status codes to indicate when to back off, could help in spreading out the load more evenly throughout the day. 
• Opportunity to collect detailed resource data in bulk was discussed as a way to reduce the number of API calls needed for large data sets. 
• Providing guidance on optimal API usage was identified as an opportunity, with the caveat that it requires insight into specific use cases to be effective. 

The DSB conducted an activity with the group to map out the opportunities associated with the Action Plan on the Miro board. 

A summary of the Action Plan follows: 

Action Plan for Q3 2034:

• 429 Retry-After trial discussion

Action Plan for Q4 2024:

• Discussion on TPS threshold, last modified query/response fields and cursor-based pagination discussion
• CR for Last Modified Query Parameter: Introduce a change request to optimise API calls by allowing ADRs to query data based on the last modification date. 
• CR for Cursor Based Pagination: Implement cursor-based pagination to improve the efficiency of data retrieval and reduce the load on data holders.
• DSAC NFR CG presentation/report
• SteerCo paper

Action Plan for Q1 2025:

• Discussions around bulk transaction details API, change notification patterns and bulk arrangement APIs
• CR/DP on 429 Retry After Trial
• DP for Async API calls
• NP or guidance paper on optimal API usages
• DP for bulk transaction detail API

Action Plan for Q2 2025:

• Discuss Statement API
• DP for change notification patterns
• DP for event-driven data notifications
• DP for Bulk Arrangement API
• CR for introducing customer present header in SDH calls

Action Plan for Q3 2025:

• DP for statement API

Discussion of Outages – issues and considerations

This item was not addressed at meeting.

Consultation Workplan

This item was not addressed at meeting.

Meeting Schedule

The Chair noted that the next meeting is scheduled for Wednesday 9 October 2024.

Any Other Business

No further business was raised.

Closing and Next Steps

The Chair thanked the group for participating for attending the meeting.

The Chair recapped the follow up tasks as follows: 

• DSB to review and potentially revise the API call patterns to optimise efficiency.
• DSB to create a survey to gather feedback on data collection patterns and use cases from ADRs.
• DSB to distribute the design of the asynchronous pattern for data collection to the team.

The meeting closed at 12:00