Mojaloop Community Central

Michael Richards
Michael Richards

Posted on

Minutes of DA Meeting 2023-10-04

#da

Minutes of DA Meeting 2023-10-04

Attendees:

Apologies:

Absent:

Agenda:

  • Report on actions from last meeting
    • Where are we on currency conversion?
    • Post the DA charter somewhere public
  • Position handler batching (VK)
  • Adding security to the DA charter (TD)
  • Issue 107: Mojaloop security review (TD)
  • Issue 108: Design and strategy questions (TD)
  • DA involvement in API changes (PSB)
  • AOB

Minutes

  • Actions from previous meeting
    • Where are we on currency conversion?
    • OK to proceed with PoC.
    • We will revisit improvements starting at the convening
    • Post the DA charter somewhere public
    • Still open
  • Position handler batching
    • Link to documentation
    • PB: there was batching, but it didn't work.
    • VK picked this up and designed it.
    • Some initial test results
    • VK: existing batching only for prepare handler, this change applies to all messages
    • Kafka consumes a fixed number of messages, selected by participant currency ID
    • Processing speed depends on batch size: 1 op/sec start position; 100 ops/sec with batches of 10; 1000 ops/sec with batches of 100.
    • Performance on a single machine with position handler only.
    • One handler per account ID.
    • PSB: can we get a deadlock? VK: no. Each handler will deal with a separate account, there should be no competitive locks.
    • JB: if lock at the point of reading the balance then you could have multiple processors processing the same account.
    • SK: we are using Kafka to partition.
    • PSB: how do you assign where the fulfil has two accounts? SK: we don't have to do that. In fulfil, we only update the creditor account. In step 1, payer account subtracts from the position. In fulfil, payee account gets credited.
    • PSB: how do we assign messages to bins? PB: all actions are assigned to bins automatically
    • PSB: how do we manage technical failures?
    • JB: this doesn't work around the presence of a single account balance. PB: the results are so good that it's not a problem.
    • PSB: this is scale up, not scale out. PB: yes, we need to do a full test. VK: we did try scaling out to several instances on one node. Improvement was preserved.
    • PSB: how do you commit to Kafka when it fails? VK: another handler picks it up. PSB: that's not a good strategy, because it gets the batch again. You need something like a dead letter queue. PB: it processes the failure, then commits the message. VK: if there is batch level failure, then we don't commit the messages. JB: you can't choose which messages to process. PSB: you have to assume that you can only process the whole batch.
    • JB: also a resilience thing - you need a backup strategy if a process dies. VK: Kafka will manage that itself.
    • PSB: this is what we've done in vNext. If done well, it's great.
    • TD: how do we do full-scale performance tests? Why do we need to improve performance on vNow? JB: INFITX have a business case that they need to meet. TD: seems to trivialise the TGB statement that performance was sufficient.
    • Foley: in principle it all makes sense. How do we handle errors? We think that's OK, but we need to be sure. Opinion: would be great to see an E2E performance test with real circumstances and make a decision based on that.
    • PSB: So, we make PoC decisions based on the agreement that the performance is good enough, then we have a business need to improve performance for the PoCs? Were Infitx not part of the ones that said vNow's performance is enough?
    • SK: “We” who? Who said performance is good enough of which version ? Sorry it isn’t clear
  • Adding security to the DA charter (TD)
  • Issue 107: Mojaloop security review (TD)
  • Issue 108: Design and strategy questions (TD)
  • DA involvement in API changes (PSB)
  • AOB

Actions:

  • SK: liaise with performance workstream to review tasks for donation.

Latest comments (0)