A distributed, transactional,
fault-tolerant object store

Getting Started

Installing GoshawkDB server

Linux

Downloads are available for binaries in deb, rpm and tarball formats.

If you wish to compile from source then you'll need to install Go which is packaged in all major Linux distributions. You will also need to install LMDB:

  • Under Debian-based Linux systems, the packages to install are liblmdb0 and liblmdb-dev.
  • On RPM-based Linux systems, the packages are often called lmdb-libs and lmdb-devel.

With these compile-time dependencies installed, the server is go gettable so it should just be a matter of: 

> go get goshawkdb.io/server/cmd/goshawkdb

OS X

There is a Homebrew Tap available for GoshawkDB. You should be able to fetch and build with: 

> brew install goshawkdb/goshawkdb/goshawkdb-server

Windows

Go and LMDB also work under Windows. Currently I do not have access to Windows machines so cannot prepare builds of GoshawkDB for this platform, but I believe it should be possible to compile from source: you'll need to install Go and LMDB, but once that's done, the server is go gettable so it should just be a matter of: 

> go get goshawkdb.io/server/cmd/goshawkdb

Docker

There is a Docker image available for GoshawkDB.

Running

Once you have GoshawkDB installed, you should be able to start it: 

> goshawkdb
GoshawkDB 2016/01/10 09:52:26.467248 [goshawkdb]
GoshawkDB 2016/01/10 09:52:26.467334 No certificate supplied (missing -cert parameter). Use -gen-cluster-cert to create cluster certificate
>

Success! GoshawkDB is able to start up, but we've not supplied enough parameters, so it shut down again.

GoshawkDB is designed for use on SSDs or faster storage devices. It will perform poorly when storing data on traditional rotating hard drives.

Command line parameters

Adding a -h will show us some help: 

> goshawkdb -h
GoshawkDB 2016/01/10 09:42:00.286354 [goshawkdb -h]
Usage of goshawkdb:
  -cert Path
        Path to cluster certificate and key file
  -config Path
        Path to configuration file
  -dir Path
        Path to data directory
  -gen-client-cert
        Generate client certificate key pair
  -gen-cluster-cert
        Generate new cluster certificate key pair
  -port int
        Port to listen on (default 7894)
  -version
        Display version and exit
>

Certificates

GoshawkDB uses X509 certificates for authentication of both servers and clients. First we need to generate the cluster certificate and key pair. The same cluster certificate and key pair must be supplied to every node in the cluster. 

> goshawkdb -gen-cluster-cert > ./clusterCert.pem
GoshawkDB 2016/01/10 09:46:33.177542 [goshawkdb -gen-cluster-cert]
>

The file clusterCert.pem now contains the cluster certificate and key pair in PEM format. Clients are also authenticated using X509 certificates. To generate a suitable client certificate and key pair: 

> goshawkdb -cert ./clusterCert.pem -gen-client-cert > ./user1.pem
GoshawkDB 2016/01/10 09:50:11.982124 [goshawkdb -cert ./clusterCert.pem -gen-client-cert]
GoshawkDB 2016/01/10 09:50:11.986663 Fingerprint: 34b1c780a4756a5cbc4c69a111188f641eed20aa825cb0b2fecd31cf28d643cb
>

The file user1.pem now contains a client certificate and key pair in PEM format. The fingerprint of the certificate needs to be added to our configuration file to allow this certificate to be accepted.

Configuration

The configuration file is covered in detail in its own guide so for now we'll just create the simplest possible configuration file. Create a file called config.json and add the following to it: 

{
    "ClusterId": "MyFirstGoshawkDBCluster",
    "Version": 1,
    "Hosts": ["localhost"],
    "F": 0,
    "MaxRMCount": 5,
    "ClientCertificateFingerprints": {
        "34b1c780a4756a5cbc4c69a111188f641eed20aa825cb0b2fecd31cf28d643cb": {
            "myRoot": {
              "Read": true,
              "Write": true
            }
        }
    }
}

This specifies a cluster of just one node, which will have no ability to withstand any failure. The fingerprint of the client certificate we created above appears in the ClientCertificateFingerprints section. We specify there is one root object called myRoot and we specify that our one client account will be able to both read and write that root object.

Launch!

Finally, we'll want to tell GoshawkDB where it can write data to disk, and should we stop and restart GoshawkDB, where it will be able to recover and load data back in from disk. This is the -dir parameter. 

> goshawkdb -config ./config.json -cert ./clusterCert.pem -dir /tmp/goshawkdb
GoshawkDB 2016/05/01 10:46:40.050297 [goshawkdb -config ./config.json -cert ./clusterCert.pem -dir /tmp/goshawkdb]
GoshawkDB 2016/05/01 10:46:40.064531 Loaded 0 acceptors from disk
GoshawkDB 2016/05/01 10:46:40.064716 Loaded 0 proposers from disk
GoshawkDB 2016/05/01 10:46:40.065089 Topology: Ensuring local topology.
GoshawkDB 2016/05/01 10:46:40.070231 Topology: task completed.
GoshawkDB 2016/05/01 10:46:40.070314 Topology: Attempting to join cluster with configuration: Next Configuration:
 AllHosts: [];
 NewRMIds: [];
 SurvivingRMIds: [];
 LostRMIds: [];
 InstalledOnNew: false;
 BarrierReached1: [];
 BarrierReached2: [];
 Pending:;
 Configuration: Configuration{ClusterId: MyFirstGoshawkDBCluster, Version: 1, Hosts: [localhost:7894], F: 0, MaxRMCount: 5, NoSync: false, RMs: [], Removed: map[]}
GoshawkDB 2016/05/01 10:46:40.108593 Topology: Requesting help from existing cluster members for topology change.
GoshawkDB 2016/05/01 10:46:40.108609 Topology: task completed.
GoshawkDB 2016/05/01 10:46:40.108932 >==> We are localhost:7894 (RM:e4cabf7e) <==<

And it's up, ready for work! To shut down GoshawkDB, either Ctl-c it, or send it either SIGTERM or SIGINT.

GoshawkDB refers to nodes within the cluster by their RM Id. RM stands for Resource Manager which is a traditional term for the nodes that make up a database that you'll come across in lots of database literature. The RM Id is created when a GoshawkDB node is started for the first time and is then stored so that when GoshawkDB restarts, its RM Id doesn't change. It is just a random number: the RM Ids you see will be different to those here.

When a GoshawkDB node starts for the first time, it also stores a copy of the configuration. This is necessary so that it can detect when the configuration changes and can calculate what's changed. As we've successfully started GoshawkDB, if we shut it down and restart it, we can now leave off the -config parameter and GoshawkDB will use whatever configuration it took a copy of: 

> goshawkdb -cert ./clusterCert.pem -dir /tmp/goshawkdb
GoshawkDB 2016/05/01 10:48:58.736960 [goshawkdb -cert ./clusterCert.pem -dir /tmp/goshawkdb]
GoshawkDB 2016/05/01 10:48:58.741711 Loaded 0 acceptors from disk
GoshawkDB 2016/05/01 10:48:58.742119 Loaded 0 proposers from disk
GoshawkDB 2016/05/01 10:48:58.742557 Topology: Ensuring local topology.
GoshawkDB 2016/05/01 10:48:58.754004 Topology: task completed.
GoshawkDB 2016/05/01 10:48:58.755198 >==> We are localhost:7894 (RM:e4cabf7e) <==<

A bigger cluster

A GoshawkDB cluster has a minimum size determined by the F parameter in the configuration file. F is the number of failures that GoshawkDB will withstand: if no more than F nodes are unreachable then GoshawkDB will continue working just fine. If more than F nodes are unreachable then a transaction submitted to GoshawkDB may block (not return any answer) or error until enough nodes recover so that no more than F nodes are unreachable. The minimum number of nodes in a cluster is 2*F + 1. So if F = 1 then 2*F + 1 = 3. So let's create a cluster of three nodes and set F = 1.

For the time being, we're going to have all three nodes running on the same machine. This is not recommended: by sharing a disk they will suffer from poorer performance, but for the purposes of getting started, this doesn't matter. Change the config.json file to: 

{
    "ClusterId": "MySecondGoshawkDBCluster",
    "Version": 1,
    "Hosts": [
        "localhost:10001",
        "localhost:10002",
        "localhost:10003"
    ],
    "F": 1,
    "MaxRMCount": 5,
    "ClientCertificateFingerprints": {
        "34b1c780a4756a5cbc4c69a111188f641eed20aa825cb0b2fecd31cf28d643cb": {
            "myRoot": {
              "Read": true,
              "Write": true
            }
        }
    }
}

The intention here is to have our three nodes on different ports, that way they won't interfere with each other. To specify the port to listen on, use the -port command line parameter. We must also give each node a different location to write to disk. All three nodes must be provided with the same cluster certificate and key pair, and the same configuration when they start for the first time. They will connect to each other and verify they've all been configured the same way.

First node: 

> goshawkdb -cert ./clusterCert.pem -config ./config.json -dir /tmp/goshawkdb1 -port 10001
GoshawkDB 2016/05/01 10:57:30.205813 [goshawkdb -cert ./clusterCert.pem -config ./config.json -dir /tmp/goshawkdb1 -port 10001]
GoshawkDB 2016/05/01 10:57:30.218516 Loaded 0 acceptors from disk
GoshawkDB 2016/05/01 10:57:30.218909 Loaded 0 proposers from disk
GoshawkDB 2016/05/01 10:57:30.219124 Topology: Ensuring local topology.
GoshawkDB 2016/05/01 10:57:30.225774 Topology: task completed.
GoshawkDB 2016/05/01 10:57:30.225834 Topology: Attempting to join cluster with configuration: Next Configuration:
 AllHosts: [];
 NewRMIds: [];
 SurvivingRMIds: [];
 LostRMIds: [];
 InstalledOnNew: false;
 BarrierReached1: [];
 BarrierReached2: [];
 Pending:;
 Configuration: Configuration{ClusterId: MySecondGoshawkDBCluster, Version: 1, Hosts: [localhost:10001 localhost:10002 localhost:10003], F: 1, MaxRMCount: 5, NoSync: false, RMs: [], Removed: map[]}
GoshawkDB 2016/05/01 10:59:01.205650 Connection established to localhost:10002 (RM:25e17bfc)
GoshawkDB 2016/05/01 11:01:28.147197 Connection established to localhost:10003 (RM:8dfb10db)
GoshawkDB 2016/05/01 11:01:28.197996 Topology: Config transition to version 1 already in progress.
GoshawkDB 2016/05/01 11:01:28.198823 Topology: Requesting help from existing cluster members for topology change.
GoshawkDB 2016/05/01 11:01:28.198836 Topology: task completed.
GoshawkDB 2016/05/01 11:01:28.199142 >==> We are localhost:10001 (RM:6b8f34b0) <==<

Second node: 

> goshawkdb -cert ./clusterCert.pem -config ./config.json -dir /tmp/goshawkdb2 -port 10002
GoshawkDB 2016/05/01 10:59:01.164249 [goshawkdb -cert ./clusterCert.pem -config ./config.json -dir /tmp/goshawkdb2 -port 10002]
GoshawkDB 2016/05/01 10:59:01.176136 Loaded 0 acceptors from disk
GoshawkDB 2016/05/01 10:59:01.176326 Loaded 0 proposers from disk
GoshawkDB 2016/05/01 10:59:01.176734 Topology: Ensuring local topology.
GoshawkDB 2016/05/01 10:59:01.182387 Topology: task completed.
GoshawkDB 2016/05/01 10:59:01.182444 Topology: Attempting to join cluster with configuration: Next Configuration:
 AllHosts: [];
 NewRMIds: [];
 SurvivingRMIds: [];
 LostRMIds: [];
 InstalledOnNew: false;
 BarrierReached1: [];
 BarrierReached2: [];
 Pending:;
 Configuration: Configuration{ClusterId: MySecondGoshawkDBCluster, Version: 1, Hosts: [localhost:10001 localhost:10002 localhost:10003], F: 1, MaxRMCount: 5, NoSync: false, RMs: [], Removed: map[]}
GoshawkDB 2016/05/01 10:59:01.205650 Connection established to localhost:10001 (RM:6b8f34b0)
GoshawkDB 2016/05/01 11:01:28.143714 Connection established to localhost:10003 (RM:8dfb10db)
GoshawkDB 2016/05/01 11:01:28.144094 Topology: Config transition to version 1 already in progress.
GoshawkDB 2016/05/01 11:01:28.144210 Connection terminated
GoshawkDB 2016/05/01 11:01:28.179411 Topology: Requesting help from existing cluster members for topology change.
GoshawkDB 2016/05/01 11:01:28.179429 Topology: task completed.
GoshawkDB 2016/05/01 11:01:28.179711 >==> We are localhost:10002 (RM:25e17bfc) <==<

Third node: 

> goshawkdb -cert ./clusterCert.pem -config ./config.json -dir /tmp/goshawkdb3 -port 10003
GoshawkDB 2016/05/01 11:01:28.081993 [goshawkdb -cert ./clusterCert.pem -config ./config.json -dir /tmp/goshawkdb3 -port 10003]
GoshawkDB 2016/05/01 11:01:28.113301 Loaded 0 acceptors from disk
GoshawkDB 2016/05/01 11:01:28.113541 Loaded 0 proposers from disk
GoshawkDB 2016/05/01 11:01:28.113942 Topology: Ensuring local topology.
GoshawkDB 2016/05/01 11:01:28.119103 Topology: task completed.
GoshawkDB 2016/05/01 11:01:28.119213 Topology: Attempting to join cluster with configuration: Next Configuration:
 AllHosts: [];
 NewRMIds: [];
 SurvivingRMIds: [];
 LostRMIds: [];
 InstalledOnNew: false;
 BarrierReached1: [];
 BarrierReached2: [];
 Pending:;
 Configuration: Configuration{ClusterId: MySecondGoshawkDBCluster, Version: 1, Hosts: [localhost:10001 localhost:10002 localhost:10003], F: 1, MaxRMCount: 5, NoSync: false, RMs: [], Removed: map[]}
GoshawkDB 2016/05/01 11:01:28.143750 Connection established to localhost:10002 (RM:25e17bfc)
GoshawkDB 2016/05/01 11:01:28.145167 Topology: Config transition to version 1 already in progress.
GoshawkDB 2016/05/01 11:01:28.147184 Connection established to localhost:10001 (RM:6b8f34b0)
GoshawkDB 2016/05/01 11:01:28.198190 Topology: Config transition to version 1 already in progress.
GoshawkDB 2016/05/01 11:01:28.199006 Topology: Requesting help from existing cluster members for topology change.
GoshawkDB 2016/05/01 11:01:28.199016 Topology: task completed.
GoshawkDB 2016/05/01 11:01:28.199902 >==> We are localhost:10003 (RM:8dfb10db) <==<

And that's it: the cluster is fully connected and running.

Try killing one of the nodes and restarting it. See how the cluster reforms. Whenever a node becomes unreachable, other nodes will periodically attempt to reconnect to it and recover automatically. Try generating a new cluster certificate and key pair and (re)starting one of the nodes with the "wrong" certificate: the cluster will refuse to form.

Monitoring GoshawkDB

Currently, GoshawkDB has only limited forms of monitoring built in, but this is something that will be addressed in an upcoming release: see the road map. If you send it a SIGUSR1 then it will print out a detailed set of diagnostics of each of GoshawkDB's various subsystems. On an active busy node, this can be a large amount of information, but on an idle server it is not overwhelming. With the cluster we created above still running: 

> ps ax | grep [g]oshawk | grep 'port 10003'
22701 pts/10   Sl+    0:00 goshawkdb -cert ./clusterCert.pem -config ./config.json -dir /tmp/goshawkdb3 -port 10003
> kill -USR1 22701

On the terminal running the third node, something like the following should have appeared: 

GoshawkDB 2016/05/01 11:05:58.121559 System Status for RM:8dfb10db
 Configuration File: ./config.json
 Data Directory: /tmp/goshawkdb3
 Port: 10003
 Address: localhost:10003
 Boot Count: 2
 Current Topology: Topology{Configuration{ClusterId: MySecondGoshawkDBCluster, Version: 1, Hosts: [localhost:10001 localhost:10002 localhost:10003], F: 1, MaxRMCount: 5, NoSync: false, RMs: [RM:6b8f34b0 RM:25e17bfc RM:8dfb10db], Removed: map[]}, F+1: 2, 2F+1: 3, DBVersion: TxnId:0000000000000001-00000000-00000001-25e17bfc, Root: VarUUId:0000000000000000000000000000000125e17bfc@[0 1 2 3 4]}
 ServerConnectionSubscribers: 10
 TopologySubscribers: [4 4 4 3 1 0]
 Active Server RMIds: [RM:8dfb10db RM:25e17bfc RM:6b8f34b0]
 Active Server Connections: [localhost:10001 localhost:10002  localhost:10003]
 Desired Server Connections: [localhost:10001 localhost:10002]
  Connection to localhost:10001 (RM:6b8f34b0, 1)
  - Current State: ConnectionRun
  - IsServer? true
  - IsClient? false
  Connection to localhost:10002 (RM:25e17bfc, 1)
  - Current State: ConnectionRun
  - IsServer? true
  - IsClient? false
 Client Connection Count: 1
  LocalConnection
   SimpleTxnSubmitter: live TxnIds: []
  Vars
   Var Manager 0
   - Active Vars: 1
   - Callbacks: 0
   - Beater live? false
   - Roll allowed? true
    VarUUId:0000000000000000000000000000000000000000
    - Positions: unknown
    - CurFrame:
     VarUUId:0000000000000000000000000000000000000000 Frame TxnId:0000000000000001-00000000-00000001-25e17bfc (1) rVC:map[VarUUId:0000000000000000000000000000000000000000:3] (cached cap? false) w<nil>
     - Read Count: 0 [0 0 0 0]
     - Uncommitted Read Count: 0
     - Learnt future reads: 0
     - Write Count: 0 [0 0 0 0]
     - Uncommitted Write Count: 0
     - RW Present: false
     - Mask: VC:map[] (cached cap? false)
     - Current State: frameOpen
     - Locked? false
     - Roll scheduled/active? false/false
     - DescendentOnDisk? false
     - Child == nil? true
     - Parent == nil? true
    - Subscribers: 1
    - Idle? false
    - IsOnDisk? true
   Var Manager 1
   - Active Vars: 0
   - Callbacks: 0
   - Beater live? false
   - Roll allowed? true
   Var Manager 2
   - Active Vars: 0
   - Callbacks: 0
   - Beater live? false
   - Roll allowed? true
   Var Manager 3
   - Active Vars: 0
   - Callbacks: 0
   - Beater live? false
   - Roll allowed? true
  Proposers
   Proposer Manager 0
   Live proposers: 0
   Live proposals: 0
   Proposer Manager 1
   Live proposers: 0
   Live proposals: 0
   Proposer Manager 2
   Live proposers: 0
   Live proposals: 0
   Proposer Manager 3
   Live proposers: 0
   Live proposals: 0
  Acceptors
   Acceptor Manager 0
    - Live Instances: 0
    - Acceptors: 0
   Acceptor Manager 1
    - Live Instances: 0
    - Acceptors: 0
   Acceptor Manager 2
    - Live Instances: 0
    - Acceptors: 0
   Acceptor Manager 3
    - Live Instances: 0
    - Acceptors: 0
Status End

Because this cluster is currently idle, various subsystems of GoshawkDB are idle and so don't display anything. Whenever a GoshawkDB server is idle, it should return to a similar state to this, with no Proposers and no Acceptors. The first section of these diagnostics show connections to other servers and to clients and shows details about the configuration of the cluster.

If you encounter a problem and get in touch and it's thought there may be a bug in the server, it's very likely you'll be asked to provide these diagnostics.

In addition to responding to SIGUSR1, GoshawkDB also responds to SIGQUIT. When GoshawkDB receives a SIGQUIT it will print out the current stack trace of all Go routines.

What's next?

That's everything for creating and running a GoshawkDB cluster.

Copyright © 2015-2016 Matthew Sackman. | All rights reserved.