Mojaloop Community Central: James Bush

The Mojaloop Foundation Publishes Its Policy on the Responsible Use of AI Tools in the Community

James Bush — Mon, 13 Apr 2026 13:33:14 +0000

The Mojaloop Foundation is pleased to announce the publication of its policy on the responsible use of AI tools within the Mojaloop community. As AI technologies become increasingly embedded in how we design, build, and operate software, it is important that their use aligns with the Foundation’s principles of openness, transparency, and trust.

The policy provides clear guidance on how community members may use AI tools in ways that support collaboration and maintain the integrity of the Mojaloop open source ecosystem. It covers areas such as the use of AI for documentation, code development, and participation in community discussions, with an emphasis on human accountability and transparency.

This policy has been reviewed and endorsed by the Community Council, reflecting a shared commitment across the community to adopt AI responsibly while continuing to encourage innovation and contribution.

Given the rapid pace of change in AI technologies and their applications, this policy will be reviewed and updated regularly to ensure it remains relevant and effective.

You can read the full policy here:
https://docs.mojaloop.io/community/standards/ai_policy.html

We welcome feedback from the community as we continue to evolve our approach in this important area.

Participation Tools Workstream Weekly Update 2024-10-03

James Bush — Thu, 03 Oct 2024 14:44:49 +0000

Notes from Participation Tools Worksteam Weekly Sync Call

RPi demo - 10tps sequential easily achieved with very basic sequential test from TTK with mock DFSP backend.
- This is addressing smaller DFSP use cases
Payment Manager adoption; there is good stuff there e.g. MCM, management API, onboarding UX etc…:
- We need a plan to close the gaps on docs + test coverage.
Range of solutions to cater to different “classes” of DFSP:
- Todo: Define these classes properly:
- Do they self host their CBS? Or use SaaS?
- Do they have IT skills to install and configure?
- Small:
  - Option1: low cost “server” hardware e.g. RPi or mini-itx etc…, on-prem, docker-compose+systemd, linux OS firewall
    - Minimal redundancy requirement, can accept some downtime.
    - Todo: figure out UI requirements and how to deliver them efficiently; pm4ml UI? Or something else?
    - Pros: one off cost
    - Cons: needs installation and setup.
    - Could be PISP only via OpenBanking SDK.
    - No need for CBS integration.
  - Option2: aggregator model
    - Do they have their own CBS, bank accounts etc…?
    - ITK hosted by an aggregator?
    - Pros: simple
    - Cons: recurring costs?
    - Possibly suits “micro” DFSPs.
    - Could be PISP only via OpenBanking SDK.
    - No need for CBS integration.
- Mid-range:
  - Option1: mid-level “server” hardware Some redundancy requirements, less willing to accept downtime.
    - Todo: Find out typical SLAs.
    - More “enterprise” type features for management and monitoring.
    - Possibly integration into existing IT management stack (metrics, observability, alterting)
    - Reporting features?...probably same across the board.
  - Option 2: 3rd party hosting of integration layer Cloud or on-prem
    - Typically requires VPN between integration layer and CBS/backends. Not “mojaloopy”.
    - VPN seen as a cost barrier for small DFSPs in Mojaloop principles….is it ok for larger DFSPs?
      - This is ripe for challenging!
    - Pros: simple
    - Cons: recurring cost
- Large:
  - Option1: high-end “server” hardware (possibly existing on-prem private/public/hybrid cloud)
    - High resilience, reliability, redundancy requirements. Strong need for enterprise grade management, monitoring, alerting features.
    - High security bar.
    - Scalable performance requirement.
  - Option2: 3rd party hosting of integration layer Cloud or on-prem
    - Strong need for enterprise grade management, monitoring, alerting features.
    - High security bar.
    - Scalable performance requirement.
- Common:
  - Fast iteration to make use of scheme enhancements (upgrades, more traffic etc…):
  - CI for integration dev work: change -> test -> release
- Common requirements:
  - “Simple” integration with backends:
    - Low-code?
    - Efficient testing and validation not just during initial setup but ongoing.
      - Dev, UAT, Production testing? CI?
  - Excellent security.
  - Low cost.
  - Efficient use of resources.
  - Ability to “see” what has happened (debugging, reporting)
  - Support???!!!

Please find all meeting notes in this doc: https://docs.google.com/document/d/1FCwkin-2JE4OwjVk9wb3xAjs6aDywgxTdGbg4tJJYsQ/edit?usp=sharing

Participation Tools Workstream Weekly Update

James Bush — Thu, 26 Sep 2024 14:34:41 +0000

Notes from Participation Tools Worksteam Weekly Sync Call

Discussion around PM4ML having enterprise features like RBAC for larger DFSPs e.g. MoMos:
- Scalable architecture: scale down has been addressed so far in the workstream.
White-labelling is probably desirable for adopters; they will want to tell DFSPs to use the “scheme connector”...not Mojaloop connector.
PB mentioned there are some security concerns with the proposed architecture
- JB asked what is missing.
CL: We need a hosting solution for projects like COMESA asap. These will be larger DFSPs with higher throughput and resilience/reliability requirements.
- PB: the pm4ml docker-compose on-prem option is out of date. Pm4ml IaC deployment is OK; this has been optimised recently to reduce cost.

Please find all meeting notes in this doc: https://docs.google.com/document/d/1FCwkin-2JE4OwjVk9wb3xAjs6aDywgxTdGbg4tJJYsQ/edit?usp=sharing

Participation Tools Workstream Weekly Update

James Bush — Thu, 19 Sep 2024 16:35:22 +0000

Notes from Participation Tools Worksteam Weekly Sync Call

ITK name is perhaps OK, but we should define its content/constituent parts and capabilities. Specifically it’s not just the mojaloop-connector.
MCM: vNext has some possible feature overlap with MCM, how do we resolve this?
- vNext should not interfere with the infra layer, as this would break layer separation. Therefore, it is safe to assume that MCM will still be applicable once vNext becomes RC.
- PM: MCM server and client should go through the adoption process asap as its features are very important for the Mojaloop product. This is fully in line with MLF mission: making it easier for people to use Mojaloop and participate in Mojaloop schemes. Possibly ask Paul Baker to take MCM through the adoption process?

Please find all meeting notes in this doc: https://docs.google.com/document/d/1FCwkin-2JE4OwjVk9wb3xAjs6aDywgxTdGbg4tJJYsQ/edit?usp=sharing

Mojaloop Foundation Seeking Engineering Contractors

James Bush — Mon, 16 Sep 2024 14:35:33 +0000

Dear Mojaloop Community,

The Mojaloop Foundation is currently recruiting for several contract engineering positions to perform development work on multiple areas of the Mojaloop codebase. These positions will be individual contracts on an initial one year basis. We would love to hear from you if you, or someone you know is interested!

Please note that we do not want to hear from agencies at this time.

Outline Job Specifications:

Staff Engineer
- Very senior software engineer with 10+ years' experience in the financial software domain as a technical leader in an API first product.
- Expert in:
  - Event driven and cloud native architectures.
  - Node.js/typescript, docker, k8s, kafka, SQL.
  - Deployment patterns for highly available and reliable software systems.
  - Highly asynchronous system design, engineering, performance tuning, debugging and operation.
  - Open-source software best practices.
  - Security best practices for financial software.
Test Automation and QA Engineer
- 5+ years test engineering experience in cloud native architectures, API testing, asynchronous APIs.
- Expert knowledge of software and system testing fundamentals, principles and best practices.
- Highly familiar with Node.js/typescript, docker, k8s, SQL.

Please contact James Bush at jbush@mojaloop.io or via Mojaloop Slack for more information.

DA Meeting Summary 2024-06-12 0900 UTC

James Bush — Tue, 18 Jun 2024 07:37:06 +0000

Quick recap

The team discussed the interface between the transfers bounded context and the accounts and balances bounded context, with a focus on the interface being 'chatty' and the potential benefits of combining accounts and balances and transfers in the same process. They also delved into the invariants Document, the significance of focusing on clear functional and non-functional requirements, and the role of a design authority in preserving the best possible design. Lastly, they explored potential changes to the reference architecture, the importance of design reviews, the introduction of versioning to the reference architecture, and the potential impact of refactoring existing knowledge on ongoing projects.

Next steps

James will continue a discussion in the DA to address his concerns about the potential performance bottleneck due to the network interface between the transfers and accounts bounded contexts in the reference architecture.
Sam will review the requirements and assumptions that led to the design of the reference architecture and provide clarity on them.
Sam will investigate versioning the reference architecture to address ambiguity and ensure it remains a living document that can evolve with feedback and improvements.

Invariants Document Review and Revisions

The team discussed the invariants document and an open issue related to it. Paul confirmed that previous comments had resolved the final open issue, and the team agreed on the proposed changes to the document, with Paul leading the review. A point of confusion regarding the wording of a statement in the project documentation was also addressed, with James committing to reword it. Lastly, James indicated his intention to ask for a re-review on the DA channel, which was agreed upon by the team without objections.

Refining Reference Architecture and Separation

James raised concerns about the reference architecture, particularly the interface between the transfers BC and the accounts and balances BC being 'chatty' and suggested that the liquidity check should be clearly indicated on the flow diagram. He also proposed that separating accounts and balances from transfers across a network connection could introduce unnecessary overhead. Paul agreed, suggesting that accounts and balances could be a library used by the transfers BC. It was confirmed by James that in the upcoming implementation, accounts and balances will be separated and will run in separate docker containers in a Kubernetes cluster. The team agreed that while the reference architecture doesn't mandate separation, it also doesn't explicitly require integration.

Focusing on Clear Requirements in Design

Karim emphasized the significance of focusing on clear functional and non-functional requirements before design, and the role of a design authority as a custodian of the best possible design. Paul and James clarified that they were discussing a theoretical reference architecture, not implementation. They also differentiated between a reference architecture, which should fulfil all requirements before implementation, and an implementation, which involves detailed system design. The team agreed on the need for open discussions to keep the documentation current and a true reference for the system's design.

Forum Discussion on vNext Architecture

Julie encouraged taking a step back to examine the forum for discussing a particular document amidst high emotions. Paul agreed to initiate discussions despite not having all relevant parties present. Karim raised concerns about the existing reference architecture in vNext and suggested that any changes should begin with identifying issues or requirements not currently fulfilled by the design, such as performance or availability issues. Paul agreed with Karim's approach, emphasizing the importance of seeking input from the original designers.

Reference Architecture Change Discussion

Paul initiated a discussion about whether to change the reference architecture and the potential consequences of such a change. Simeon proposed postponing the conversation due to the tense atmosphere and the need for more involvement, particularly from Michael Richards. However, Paul disagreed, believing the discussion should continue due to its significance for ongoing projects. James clarified that he was not proposing any immediate changes, but the team needed a better understanding of his concerns. The group agreed to continue the discussion while also seeking a better understanding of James's perspective.

Discussing Reference Architecture Risks

James emphasized that a potential change to the reference architecture, while not a showstopper for the vNext project, should be discussed as a long-term risk. He highlighted the importance of design reviews in mitigating potential issues. Paul agreed and clarified that the current vNext implementation doesn't need to align perfectly with the reference architecture, but should strive towards it. The team agreed that open discussion of potential issues and risks is crucial for the success of the project.

Introducing Versioning to Reference Architecture

Sam proposed to introduce versioning to the reference architecture to reduce ambiguity and clarify that it's a living document continuously under improvement. He suggested that the current workstream could focus on reference architecture version 1.0, with ongoing efforts for version 2.0. He also emphasized the need to align the implementation with the reference architecture, but noted that exceptions could be addressed through a dedicated process. Lastly, Sam highlighted the necessity to document the requirements and assumptions underlying the reference architecture. Paul acknowledged Sam's suggestions.

Refactoring Knowledge and Managing Risks

Paul and Julie discussed the refactoring of existing knowledge and the potential impact on ongoing projects. Paul emphasized the importance of not delaying necessary discussions due to external factors, while Julie expressed concern about the potential impact on existing projects. There was also a discussion about the importance of maintaining a clean separation of concerns in the project's architecture, and the potential risks of chatty interfaces. James reiterated the foundation's commitment to delivering the highest quality software with the lowest possible risk profile, given the critical nature of the financial infrastructure it serves.

Bounded Context Transfer and Implementation Strategies

Karim and James discussed the potential issues and implementation strategies for the transfer bounded context and accounts and balances bounded context. Karim expressed concerns about maintaining two separate bounded contexts and suggested the need for implementation guidelines to ensure optimal design choices. James and Paul proposed a system where transfers, accounts, and balances are kept separate but functionally connected, with Paul suggesting a mitigation strategy of deploying the system on the same node or cluster. The team agreed to continue the discussion in the following week's meeting.

Please note that this content was originally generated by AI but has been reviewed for accuracy by James Bush. Some inaccuracies may remain. Please contact the author if you wish to make corrections.

DA Meeting Summary 2024-04-24 0900 UTC

James Bush — Wed, 01 May 2024 10:23:44 +0000

Quick recap

The team discussed the responsibilities of the Design Authority (DA) in maintaining the reference architecture and focusing on performance characterization tests, the potential of modifying the system to allow version 7 UUIDs in resource identifier fields. They also addressed concerns about the security bounded context in the vNext implementation potentially not aligning with some TGB directives, the need for more detailed design documentation for the vNext implementation, and the decision-making process of the DA (Design Authority) in general.

Summary

DA Calls, JWS, and Reference Architecture

James clarified to Min that DA calls are not open to observers unless specifically invited, which was a misunderstanding on Min's part. James confirmed that Min could be invited to future discussions about the JWS certificate and the reference architecture by a DA member. The postponed discussion on the JWS certificate and reference architecture was rescheduled for the following week. The main discussion item was about the responsibilities of the Design Authority (DA) in maintaining the reference architecture. Paul suggested reviewing the current state of the documentation to identify gaps for improvement

Performance Characterization and UUID Bottleneck

Kalin presented the results of performance characterization tests on their system, highlighting that the goal was to ensure reasonable performance throughput with standard hardware. Paul clarified that this was not about achieving the highest transaction per second. Kalin then identified a significant bottleneck caused by the use of UUIDs as primary keys in several tables, which was degrading index performance and consuming memory. Kalin proposed potential solutions such as using a different version of UUID and implementing as binary(16) in table storage, and suggested exploring new UUID formats proposed, specifically designed for database use.

Analyzing UUID Performance and Simulation Issues

Kalin and James analyzed the performance of different versions of a UUID, focusing on versions 4 and 7. They found a significant drop in performance for version 4, attributing it to its use in a secondary index. The performance of version 7 was found to be stable over time due to its sequential nature. They also discussed an issue with their simulation testing involving multiple simulated DFSPs, where all simulators ran on the same VM, leading to highly sequential generated UUIDs. To address the performance degradation due to use of non-sequential UUIDs, they proposed to modify the validation code to only allow version 7 UUIDs and add validation to certain parts of the system.

Discussing UUID Modifications and Performance

The team discussed the potential of modifying the system to allow version 7 UUIDs, which would be less restrictive but would require approval from the Change Control Board. James highlighted the possible impact on performance and cost, suggesting that the decision should consider the system's bottom line. Paul Baker recalled a previous discussion about the uniqueness and independence of UUIDs and their importance for the system. Karim proposed an alternative approach where each participant generates a unique ID, suggesting that non-enforced uniqueness could be a solution to the performance issue. The team left with an open question about whether this alternative approach could be a viable solution.

UUIDs, API Limitations, and Pattern Checking

Karim and James discussed the issue of UUIDs not fitting certain fields in the ISO-20022 API and possible solutions, including the potential use of CUID in the next version of the Mojaloop API. They agreed to further investigate this matter in future Change Control Board meetings. The team also discussed the need for pattern checking and validation to ensure good performance, with Kalin proposing to relax the pattern on the quoting service and update the SDK to generate UUID version 7. No objections were raised to this proposal.

Reference Architecture and TGB Directives

James and Paul discussed the reference architecture, with Paul expressing concerns about the security bounded context not aligning with some TGB directives. James agreed, noting the lack of well-defined documentation and implementation guidelines for the reference architecture. He emphasized the need for a clear translation from the high-level map into requirements or guidelines for any implementation. Paul concurred, highlighting the importance of a ubiquitous language in the process. James also pointed out a possible omission in the terminology regarding the scheduling for timeouts and notifications, indicating this as an oversight.

Improving Mojaloop Project Documentation

James expressed his concerns about the current state of the Mojaloop documentation, highlighting the need for more detailed design and implementation documentation beyond the high-level reference architecture. He suggested using the beta version of the vNext implementation as a starting point for this documentation. Karim clarified James's concept of 'implementation architecture', emphasizing its focus on non-functional properties of the system and its bottom-up view. James agreed, noting the complexity of defining architecture at various levels. They both agreed on the need to specify these aspects further.

vNext Beta Codebase Diagrams and Tool Choice

James proposed creating a diagram to illustrate the major constructs within each bounded context of the Vx Beta codebase. Paul agreed to this idea and suggested that James should also consider the interface protocols. James further discussed documenting design and code review guidelines and to provide more descriptive information on tool choice. Paul emphasized the importance of tool choice, highlighting the need for enterprise support and maintained Helm charts. James agreed and noted the value of product input in the reference architecture space.

Next steps

Aung will invite Min to the vNext call next week to discuss the JWS certificates vs raw keys issue.
Paul Baker will lead a discussion on the responsibilities of the DA in maintaining the reference architecture, including reviewing the state of the documentation and identifying gaps.

Please note that this content was originally generated by AI but has been reviewed for accuracy by James Bush

DA Meeting Summary 2024-04-17 0900 UTC

James Bush — Wed, 17 Apr 2024 10:35:04 +0000

Quick recap

The team discussed the challenges and potential solutions for achieving high uptime and reliability for the Mojaloop software, with a focus on infrastructure design and failover strategies. They also debated the importance of clear documentation and the role of manufacturers in improving system statistics and reliability. Lastly, they explored the complexities of probability calculations, the need for software components' availability, and the invariance of the Mojaloop software.

Summary

Pull Requests, Work Streams, and API Issues

James reported that two of his three pull requests had been merged, with the last one pending. James also shared updates about the pull requests, specifically about the 'coordinated vulnerability disclosure policy' and 'cyber security architecture' which had been published after some revisions. James agreed to address Sam's point about the API gateway issue in the initial days of Mojaloop to avoid similar situations in the future.

Mojaloop Deployment and Availability Calculation

James and Paul discussed the deployment of Mojaloop with the infrastructure necessary for 5 nines of uptime. Paul questioned the definition of appropriate infrastructure and the calculation of the availability figure, given redundancy in multiple nodes. They agreed this was a challenging probabilistic calculation involving the reliability of hardware and software. James reiterated his intent to include in the statement that Mojaloop makes use of suitable architectural patterns across many layers of the platform to enable this level of uptime.

Uptime Issues and Probability Calculations

James and Paul discussed issues related to uptime and the display of certain information. Paul pointed out a discrepancy in the display he was seeing, which James initially couldn't explain but later attributed to a cached display in his browser. The group also delved into the complexities of probability calculations, specifically regarding the combination of probabilities and the potential for multiple outcomes. Paul shared his statistical background and highlighted the subjective nature of the calculations.

Statistical Data Presentation and Architectural Decisions

The team discussed the challenges of presenting statistical data, with James suggesting that the focus should be on the general solution and architecture, rather than specific numerical values. Paul agreed and proposed emphasizing the redundancy built into the system, which can be scaled up. They debated the use of the term "uptime" versus "reliability", with James suggesting the latter, and Paul advocating for the former as it is more commonly used in the industry. The team also touched upon the architectural decisions of the deployer and the need for flexibility to adapt to changing conditions.

Hardware Redundancy and Site Reliability Discussion

James and Paul discussed the complexity and significance of hardware redundancy and site reliability in their systems, with a focus on the challenges of calculating and improving these aspects. They highlighted the value of resources like Google's reliability engineering documents and the statistics published by Backblaze on the reliability of hard disks and solid-state drives. They also emphasized the role of manufacturers in responding to feedback from cloud providers to improve their statistics and reliability.

Failover Approach in Engineering Design

Paul, James, and Paul Makin discussed the engineering approach of designing for potential failures, a strategy known as 'failover'. They agreed this approach, which originated in hardware engineering, has now been adopted for software and infrastructure. James committed to updating a document to reflect this approach, with a focus on using industry-standard availability as an example rather than specific number calculations. The team also briefly touched on a point raised by Michael in a previous Slack message, but did not elaborate further.

Payment System High Availability and Reliability

Karim discussed the importance of high availability in the payment system, emphasizing the use of dual auto failover links to ensure 99.5% uptime. He explained that in the event of one link failing, the other is immediately available to prevent unplanned outages. Additionally, he highlighted the need for a reliable mechanism to handle planned outages for software updates, suggesting a design that minimizes the planned outage time and allows for updates without system downtime.

Mojaloop Uptime Goal and Design Discussion

Karim, James, Vijay, Sam, and Paul discussed the ambitious goal of achieving 99.999% uptime for the Mojaloop software. They recognized that while this figure was theoretically possible, it would depend on various factors and could be compromised in practice. James suggested updating the wording to reflect this and to focus more on design. Paul emphasized the importance of discussing design invariants instead of specific uptime numbers. The team agreed on the need to support infrastructure designs that could achieve 5 nines of uptime, but they debated whether to include a specific uptime percentage in their statements.

Software Component Availability and Upgrades

Karim emphasized the importance of software components' availability and the ability to upgrade them without causing a prolonged downtime. He advocated for an active-active or active-passive model for the software components to ensure no single point of failure. James agreed with Karim's points, and David's interaction with James indicated a shift in the discussion toward improving the language of the invariants document.

Mojaloop Software Invariants and High Availability

James, David, and Karim discussed the invariants of the Mojaloop software and its components. They debated the importance of high availability and uptime, and how to achieve this. The team agreed that the software should be deployed using appropriate infrastructure and processes, with a focus on preventing single points of failure. They also discussed the need for clearer documentation on the use of Kubernetes and other tools and technologies for high availability. The invariants of the software, which should remain true regardless of design or implementation changes, was emphasized as a key principle.

Minimizing Downtime During System Upgrades

The team, led by James, discussed the challenges and potential solutions for minimizing downtime during system upgrades. Karim highlighted the recent issue with the national instant payment system in Pakistan, where a planned upgrade caused a two-hour outage. Paul Makin pointed out that version upgrades inherently require some downtime, but suggested that this could be minimized by using active-secondary site configurations. James proposed focusing on targets and risk management for future planned upgrades, and suggested moving the discussion of the UUID version 7 to the next week's meeting.

Next steps

• James will update the wording in the invariants section to focus on design and architectural decisions that contribute to high availability, rather than specific uptime numbers.
• James will rephrase the paragraph on planned upgrades to target 0 downtime and discuss the risk and potential impact of any upgrade.
• Paul will prepare an agenda for the next meeting to discuss the UUID version 7.

Please note that this content was originally generated by AI but has been reviewed for accuracy by James Bush

DA Meeting Summary 2024-03-13 0900 UTC

James Bush — Wed, 13 Mar 2024 13:24:01 +0000

Quick Recap

The team reviewed the vNext Beta, highlighting several implementation issues. James
emphasized the importance of understanding the code related to the transfer process and the potential issues with having
only one instance of a service running concurrently. The team discussed the scalability of their system, the advantages
of using a distributed database, and the challenges in implementing the TigerBeetle system. They also discussed the
potential performance issues with the Central Ledger Service and the need to balance performance and reliability.

Summary

vNext Beta Review and Implementation Issues

James initiated the meeting and noted that Michael Richards was absent. The main focus of the meeting was a review of
the vNext Beta presented to the Mojaloop Foundation. James, Sam, and Paul had summarized their findings in a document on
the vNext Beta Review Slack channel (#vnext-beta-review), which had garnered significant comments from community
members. James highlighted several issues with the current ledger implementation in the vNext code base and planned to
bring these issues to the design authority for discussion. Karim inquired about the differences between the vNow and
vNext implementations, which James confirmed and planned to discuss further.

System Architecture and Implementation Issues

James provided an overview of the system architecture, focusing on the issues related to the transfers and accounts and
balances bounded contexts. He explained that incoming requests are processed in a specific way, with the transfers
context handling the business logic and the accounts and balances context updating the ledger and account balances.
However, James highlighted several problems with the implementation, such as single threading, lack of synchronized
access to account balances, and lack of atomicity in writing entries to the persistent store and flushing account
balances. He was explaining the code related to the transfer process, noting the absence of a duplicate check function,
and
emphasized the importance of understanding this code for system improvement.

Concurrency Issues in Service Instances Discussed

James highlighted the potential issues with having only one instance of a service running concurrently in the system.
He explained the problem with the account balance synchronization and the possibility of overspending if two instances
assume the available liquidity balance is higher than it actually is. James suggested that the system should have
multiple instances of each service that can handle any request, ensuring redundancy, scalability, and reliability. Aung
proposed a possible solution using Kafka partitioning, but James reiterated his concern about the affinity between DFSPs
and service instances, suggesting that multiple instances of each service able to deal with requests from any DFSP would
be a more desirable solution.

Instant Payment System Implementation

Karim and James discussed the implementation of an instant payment system. Karim shared that the system can process 2
million transactions, with 1 million going through their data center. He emphasized the importance of ensuring data
persistence before committing operations. The system relies on Oracle as a database, and any issues with data
persistence could lead to problems if services are switched to another data center. James highlighted that the system
always writes to persistent storage, but reads only happen from a local cache at runtime and how the prevents multiple
processes from acting concurrently. They agreed on the need to balance system
scalability with zero tolerance on potential data loss.

Scalability and Distributed DBMS Discussion

James and Karim expressed concerns about the scalability of their system, specifically the issue of affinity. James
suggested that having only one instance of a service for any particular DFSP was a
problem and should be discussed further. Paul agreed, highlighting the complexity of effectively creating a distributed
DBMS.
However, James proposed leveraging an existing distributed DBMS, such as TigerBeetle, to avoid developing their own.
The team decided to kick off a discussion to gather different views on this matter.

Distributed Databases for Data Safety and Redundancy

James discussed the advantages of using a distributed database. He explained that distributed databases allow for
increased processing throughput by distributing the work across multiple machines, which avoids reaching the processing
throughput limit of a single hardware node. He also emphasized the importance of replicating data across multiple
physical storage devices, especially for mission-critical data. This approach ensures data redundancy and minimizes the
risk of data loss due to storage device failures. He noted that distributed databases allow for data to be stored in
different physical modes, different failure boundaries, and even different data centers, providing an added layer of
safety. He concluded that distributed databases are highly desirable for Mojaloop as they offer data redundancy and can
ensure the
safety of data in multiple physical locations.

Non-Functional Requirements and Solutions for Financial Systems

James proposed potential solutions for future clusters and emphasized the significance of non-functional requirements
for financial systems, including regulatory approval. He mentioned an open PR discussing operational characteristics and
non-functional requirements for Mojaloop (Mojaloop Invariants: https://github.com/mojaloop/documentation/pull/428),
urging the team to review and decide on possible changes to the ledger
implementation. James also noted that Pedro was working on a TigerBeetle integration, but it was still in its early
stages. James stressed the importance of reaching consensus on the correct approach for
Mojaloop and repeated his requested for all DA members to review the invariants, particularly the section on operational
characteristics. He
encouraged team members to comment on any disagreements in the PR and aimed to resolve these issues within the next
week.

TigerBeetle System Implementation Challenges

James and Paul discussed the challenges in implementing the tiger beetle system, with a focus on synchronization issues
between processes and the need for locking mechanisms to prevent dirty reads and writes. James
suggested the use of thread synchronization code across processes and expressed his hope that Pedro would deliver a
solution that utilizes tiger beetle as the single store for ledges and account balances, with multiple instances running
due to its distributed cluster nature. Paul proposed having a MySql version as an interim solution until the tiger
beetle tool matures. Karim added that they currently have multiple instances due to the absence of a context boundary.

Performance vs Reliability: Central Ledger Service Challenges

The team discussed the potential performance issues with a shared atomic account balance, particularly in handling bulk
transfers and high-volume transactions. Karim emphasized the need for reliability
over performance, while James highlighted the limitations of a single instance of a process writing to a single instance
of a database. The discussion concluded without a definitive solution, acknowledging the need to balance performance and
reliability. James also initiated a discussion about the invariants topic in the DA channel on Slack, encouraging team
members to contribute their thoughts. The meeting ran over time, prompting
James to suggest extending the discussion in the next meeting, which the team agreed to.

Next steps

• James will share the code for the critical path for transfer prepare and transfer commit on DA core slack channel.
• Discuss the need for multiple instances of services in Mojaloop to ensure high availability and scalability.
• All DA members to review the open PR on operational characteristics and non-functional requirements for Mojaloop (
Mojaloop Invariants)
• Discuss the implications of the Open PR on the vNext ledger design
• Consider the use of TigerBeetle as a single store for ledgers and account balances
• Consider the use of MySQL as a stopgap until TigerBeetle becomes production-ready

Please note that this content was originally generated by AI but has been reviewed for accuracy by James Bush

Benefits of vNext

James Bush — Thu, 07 Sep 2023 12:55:25 +0000

During review of the vNext Alpha report document a request has been made to include a section on the benefits of the vNext codebase. This post is to open the discussion as to what content should be included in the report. Please comment on this post with your input.