GoshawkDB: Howto Build a Bank

A distributed, transactional,
fault-tolerant object store

Building a bank

The purpose of this howto is to demonstrate how to model data using GoshawkDB. The main point is to avoid thinking "how would I do this in an SQL database with tables?" and instead think "if my program never needed to be restarted and so was the data store, how would I model this?". The code examples in this howto are all in Go, using the Go client for GoshawkDB. Please make sure you've read the guide to the Go client.

The code in this Howto, along with the other Howtos are available in the examples repository.

In this tutorial we're going to model a bank which has customer accounts, and transfers can occur, moving money from one account to another. The history of all transfers involving an account will be available to the customer. Because GoshawkDB supports the strong-serializability isolation level, transactions that transfer funds from one account to another are guaranteed to be safe: money cannot go missing or be invented. In data stores that support only isolation levels weaker than strong-serializability, the same transactions may not be safe.

Our root object is going to be our bank. It will contain no value, but have references (pointers) to all the customer accounts the bank has. To create the bank, we just need to ensure that the root object is empty:

package bank

import (
	"encoding/json"
	"errors"
	"fmt"
	"goshawkdb.io/client"
	"time"
)

type Bank struct {
	conn   *client.Connection
	objRef client.ObjectRef
}

func CreateBank(conn *client.Connection) (*Bank, error) {
	result, _, err := conn.RunTransaction(func(txn *client.Txn) (interface{}, error) {
		roots, err := txn.GetRootObjects()
		if err != nil {
			return nil, err
		}
		rootObj, found := roots["myRoot1"]
		if !found {
			return nil, errors.New("No root 'myRoot1' found")
		}
		return rootObj, rootObj.Set([]byte{})
	})
	if err != nil {
		return nil, err
	}
	return &Bank{
		conn:   conn,
		objRef: result.(client.ObjectRef),
	}, nil
}

Note that we capture the ObjectRef of the root object in the Bank value. Thus if in the future we have a bank not located at the root then we won't need to change any code outside this function.

Customer accounts will have a value that contains an account number, a customer name and a current balance. A customer account will also have pointers to transfers that have occurred that either debit or credit the customer account. For simplicity, we'll use Go's JSON encoding, hence the earlier import of "encoding/json".

type Account struct {
	Bank   *Bank
	objRef client.ObjectRef
	*account
}

type account struct {
	Name          string
	AccountNumber uint
	Balance       int
}

func (b *Bank) AddAccount(name string) (*Account, error) {
	acc := &account{
		Name:    name,
		Balance: 0,
	}
	accObjRef, _, err := b.conn.RunTransaction(func(txn *client.Txn) (interface{}, error) {
		bankObj, err := txn.GetObject(b.objRef)
		if err != nil {
			return nil, err
		}
		accounts, err := bankObj.References()
		if err != nil {
			return nil, err
		}
		acc.AccountNumber = uint(len(accounts))
		accValue, err := json.Marshal(acc)
		if err != nil {
			return nil, err
		}
		accObjRef, err := txn.CreateObject(accValue)
		if err != nil {
			return nil, err
		}
		return accObjRef, bankObj.Set([]byte{}, append(accounts, accObjRef)...)
	})
	if err != nil {
		return nil, err
	}
	return &Account{
		Bank:    b,
		objRef:  accObjRef.(client.ObjectRef),
		account: acc,
	}, nil
}

The account number is just the current number of accounts at the time when we create the account: we're not supporting deleting accounts at the moment. We use the account type to facilitate the conversion to and from JSON, whilst the Account is the public type. The public Account type exposes the customer name and account number. This is only safe because we do not envisage either of these details changing: they are immutable. If they could change then it would not be safe to make these fields public, and instead you'd want to add methods that run transactions to retrieve the account details from GoshawkDB.

We also need to add a method to get an account by account number: when a customer comes into our bank we expect them to know their own account number. Because the account number is the index into the list of references from our root bank object, this is very simple.

func (b *Bank) GetAccount(accNum uint) (*Account, error) {
	acc := &account{}
	result, _, err := b.conn.RunTransaction(func(txn *client.Txn) (interface{}, error) {
		bankObj, err := txn.GetObject(b.objRef)
		if err != nil {
			return nil, err
		}
		if accounts, err := bankObj.References(); err != nil {
			return nil, err
		} else if accNum < uint(len(accounts)) {
			accObjRef := accounts[accNum]
			accValue, err := accObjRef.Value()
			if err != nil {
				return nil, err
			}
			return accObjRef, json.Unmarshal(accValue, acc)
		} else {
			return nil, fmt.Errorf("Unknown account number: %v", accNum)
		}
	})
	if err != nil {
		return nil, err
	}
	return &Account{
		Bank:    b,
		objRef:  result.(client.ObjectRef),
		account: acc,
	}, nil
}

We're going to support both cash deposits and transfers, but to keep it simple, a cash deposit is just a transfer which has no from account. So a transfer will have a value containing the date of the transfer, the amount transferred, and references to the account transferring from and to. Usefully, the Go Time type supports the JSON Marshaler interface. The error handling makes this a bit long, and there's a little duplication which could probably be factored out, but hopefully the following is clear, if slightly verbose:

type transfer struct {
	Time   time.Time
	Amount int
}

type Transfer struct {
	objRef client.ObjectRef
	From   *Account
	To     *Account
	*transfer
}

func (dest *Account) TransferFrom(src *Account, amount int) (*Transfer, error) {
	if src != nil {
		if src.AccountNumber == dest.AccountNumber {
			return nil, fmt.Errorf("Transfer is from and to the same account: %v", src.AccountNumber)
		}
		if !src.Bank.objRef.ReferencesSameAs(dest.Bank.objRef) {
			return nil, fmt.Errorf("Transfer is not within the same bank!")
		}
	}
	t := &transfer{Amount: amount}
	result, _, err := dest.Bank.conn.RunTransaction(func(txn *client.Txn) (interface{}, error) {
		t.Time = time.Now() // first, let's create the transfer object
		transferValue, err := json.Marshal(t)
		if err != nil {
			return nil, err
		}
		destAccObjRef, err := txn.GetObject(dest.objRef)
		if err != nil {
			return nil, err
		}

		// the transfer has at least a reference to the destination account
		transferReferences := []client.ObjectRef{destAccObjRef}
		transferObj, err := txn.CreateObject(transferValue, transferReferences...)
		if err != nil {
			return nil, err
		}

		destReferences, err := destAccObjRef.References() // now we must update the dest account
		if err != nil {
			return nil, err
		}
		// append our transfer to the list of destination transfers
		destReferences = append(destReferences, transferObj)
		destValue, err := destAccObjRef.Value()
		if err != nil {
			return nil, err
		}
		destAcc := &account{}
		if err = json.Unmarshal(destValue, destAcc); err != nil {
			return nil, err
		}
		destAcc.Balance += t.Amount // destination is credited the transfer amount
		destValue, err = json.Marshal(destAcc)
		if err != nil {
			return nil, err
		}
		if err = destAccObjRef.Set(destValue, destReferences...); err != nil {
			return nil, err
		}

		if src != nil { // if we have a src, we must update the source account
			srcAccObjRef, err := txn.GetObject(src.objRef)
			if err != nil {
				return nil, err
			}
			srcReferences, err := srcAccObjRef.References()
			if err != nil {
				return nil, err
			}
			// append our transfer to the list of source transfers
			srcReferences = append(srcReferences, transferObj)
			srcValue, err := srcAccObjRef.Value()
			if err != nil {
				return nil, err
			}
			srcAcc := &account{}
			if err = json.Unmarshal(srcValue, srcAcc); err != nil {
				return nil, err
			}
			srcAcc.Balance -= t.Amount // source is debited the transfer amount
			if srcAcc.Balance < 0 {
				// returning an error will abort the entire transaction.
				return nil, fmt.Errorf("Account %v has insufficient funds.", src.AccountNumber)
			}
			srcValue, err = json.Marshal(srcAcc)
			if err != nil {
				return nil, err
			}
			if err = srcAccObjRef.Set(srcValue, srcReferences...); err != nil {
				return nil, err
			}
			// there is a source so add a ref from the transfer to the source account
			transferReferences = append(transferReferences, srcAccObjRef)
			if err = transferObj.Set(transferValue, transferReferences...); err != nil {
				return nil, err
			}
		}

		return transferObj, nil
	})

	if err != nil {
		return nil, err
	}
	return &Transfer{
		objRef:   result.(client.ObjectRef),
		From:     src,
		To:       dest,
		transfer: t,
	}, nil
}

Finally, let's make the API of the bank a bit nicer. Here, we'll take advantage nested transaction too to compose some of our above APIs. By using nested transactions, we ensure that the entire operation is atomic and strongly-serialized with regards to all other operations going on in the bank at the same time.

func (b *Bank) CashDeposit(accNum uint, amount int) (*Transfer, error) {
	result, _, err := b.conn.RunTransaction(func(txn *client.Txn) (interface{}, error) {
		account, err := b.GetAccount(accNum)
		if err != nil {
			return nil, err
		}
		return account.TransferFrom(nil, amount)
	})
	if err != nil {
		return nil, err
	}
	return result.(*Transfer), nil
}

func (b *Bank) TransferBetweenAccounts(srcAccNum, destAccNum uint, amount int) (*Transfer, error) {
	result, _, err := b.conn.RunTransaction(func(txn *client.Txn) (interface{}, error) {
		srcAccount, err := b.GetAccount(srcAccNum)
		if err != nil {
			return nil, err
		}
		destAccount, err := b.GetAccount(destAccNum)
		if err != nil {
			return nil, err
		}
		return destAccount.TransferFrom(srcAccount, amount)
	})
	if err != nil {
		return nil, err
	}
	return result.(*Transfer), nil
}

Hopefully this has shown some ideas of how to model data in GoshawkDB. As is often the case, the error handling in Go adds a fair amount of noise, and there is probably a little refactoring to be done to shrink this down further. The final use of nested transactions is powerful as it shows how small and simple APIs can be combined and composed and not risk violating atomicity. At no point did we think about tables and whilst there is more code and complexity than if we were just using Go structs alone, the object model seems to be sensible and natural.

There are a couple of brief extensions you might like to try as exercises:

Currently we have an API to create a new empty bank, but we have no way of resuming a bank that we'd already created. Add an API that assumes that the bank is in the root object and returns the correct *Bank object.
Add a method to *Account called Activity that iterates through all the transfers involving the account (i.e. found in the account references) and returns a []*Transfer. You'll want to add methods to transfer to extract the destination and source objects from the transfer object. Then write a String method on *Transfer that helpfully formats the transfer, correctly distinguishing an inter-account transfer from a cash deposit.
Use the Collections Library to store accounts rather than just a straight list. Then you add support for removal too. You might want to switch to using some form of UUId as the account number.