Implementing request forwarding
Closed this issue · 4 comments
Are messages of type ForwardRequest deprecated in regard to the state machine? I notice that in this block of code located in state_machine.go
func (sm *StateMachine) step(source nodeID, msg *msgs.Msg) *ActionList {
actions := &ActionList{}
switch msg.Type.(type) {
case *msgs.Msg_RequestAck:
return actions.concat(sm.clientHashDisseminator.step(source, msg))
case *msgs.Msg_FetchRequest:
return actions.concat(sm.clientHashDisseminator.step(source, msg))
case *msgs.Msg_ForwardRequest:
return actions.concat(sm.clientHashDisseminator.step(source, msg))The case for a message of type ForwardRequest is dealt with yet the step method of clientHashDisseminator doesn't account for it instead raising an error. Furthermore this type of message only seems to produced by a replica i.e. it is never produced internally by the state machine, and has an accompanying comment that indicates so.
So in regard to request forwarding (of transactions to be ordered) is it only the execution of an action of type forward request that has to be added? As seen below
func (pi *WorkItems) AddStateMachineResults(actions *statemachine.ActionList) {
// First we'll handle everything that's not a network send
iter := actions.Iterator()
for action := iter.Next(); action != nil; action = iter.Next() {
switch t := action.Type.(type) {
case *state.Action_Send:
walDependent := false
// TODO, make sure this switch captures all the safe ones
switch t.Send.Msg.Type.(type) {
case *msgs.Msg_RequestAck:
case *msgs.Msg_Checkpoint:
case *msgs.Msg_FetchBatch:
case *msgs.Msg_ForwardBatch:
default:
walDependent = true
}
if walDependent {
pi.WALActions().PushBack(action)
} else {
pi.NetActions().PushBack(action)
}
case *state.Action_Hash:
pi.HashActions().PushBack(action)
case *state.Action_AppendWriteAhead:
pi.WALActions().PushBack(action)
case *state.Action_TruncateWriteAhead:
pi.WALActions().PushBack(action)
case *state.Action_Commit:
pi.AppActions().PushBack(action)
case *state.Action_Checkpoint:
pi.AppActions().PushBack(action)
case *state.Action_AllocatedRequest:
pi.ClientActions().PushBack(action)
case *state.Action_CorrectRequest:
pi.ClientActions().PushBack(action)
case *state.Action_StateApplied:
pi.ClientActions().PushBack(action)
// TODO, create replicas
case *state.Action_ForwardRequest:
// XXX address
case *state.Action_StateTransfer:
pi.AppActions().PushBack(action)
}
}
}I guess here the action would be delegated to the net processor to make use of the link provided in processorConfig but I'm not entirely sure. However it is clear that this action is not deprecated since it is produced by the state machine in response to a fetch request.
Hi Harry, thanks for you interest in MirBFT! The request forwarding is indeed not yet implemented. The current plan is to take request forwarding out of the state machine in a separate module that deals with client requests. Currently a ForwardRequest message is not produced by any part of the code, in or outside the state machine. Also, the are rather big changes pending for the state machine code in general, but those will take a bit of time. What I do expect in a few days (hopefully next week) is a refactoring of the node and processor, putting some more order in the different modules (the state machine implementing the actual protocol will be one of these modules) with clearer responsibilities of each of the modules. That should shed more light on how and where request forwarding is to be implemented. Stay tuned!
Hi Matej, thanks for the clarification! I have a few questions regarding the refactoring. In refactoring node and processor will the general design remain unchanged? i.e. the processor is multiple sub-processors which each deal with a category of action fed by the state machine and node is the entry point in which these processors (and state machine) are started?. This meaning that this change will segregate these categories of processor into packages/modules.
Secondly, do the pending state machine changes change the protocol configuration, so as to change what settings are passed to a node on startup? I ask as my colleagues and I are planning how we would integrate this library into Fabric (using the Raft implementation as a starting point). We've identified that with the exception of request forwarding and catches of the state machine that remain to be ironed out, an end user has to determine how they set up the clients of a node and implement the methods of the app interface in the processor package.
Of course if these change then our planning will have to as well, though that isn't to say that this is an issue! We understand that this library is a work in progress and our planning pre-emptive. If you find you want help with these changes then I and my colleagues would be happy to help. Regardless I eagerly await the coming changes!
Hi Harry,
even though the refactoring will probably remove the processor as a package altogether and move most of the functionality directly to the node, the high-level architecture will remain unchanged. The main aim of the refactoring is to clean things up and make the code easier to navigate. Exactly as you said, modules will be introduced for the WAL, networking, hashing, etc., and the Node will orchestrate the interaction between them, in principle the very same way as is done now, but cleaner. This should not have much impact on the outer interface of the Node though, which still remains the entry point of the library.
For the state machine and the configuration, this still needs to be ironed out, and I'd be glad to have a more detailed discussion about that (I contacted you on Rocket.chat regarding that). Yes, the library is work in progress at this stage, and we're flexible to discuss everything and try to accommodate things that would make your life easier using the library!
Hi Matej, Good to hear that the high-level architecture is finalised. I'd be happy to have a more detailed discussion about the state machine and its configuration on Rocket.chat!