GoshawkDB: Java Client Guide

A distributed, transactional,
fault-tolerant object store

Java client

The Java client released artifacts are available from:

Maven Central; or
This server

Javadoc can be browsed from this server too.

Source is browsable or can be cloned using Mercurial:
hg clone https://src.goshawkdb.io/java-client

Source is also mirrored at github.

The code on this page, along with the Howtos are available in the examples repository.

The Collections Library is available for the Java client.

Some general notes:

It is recommended that you read the guide to the data-model in conjunction with this guide.
One connection to a server can do one transaction at a time. If you need to run multiple transactions concurrently then you need to use multiple connections.
As with any data store which supports transactions, given that the transaction can fail to commit, you need to ensure that your transaction function is side-effect free: it should not alter any state until after the transaction commits. The client will automatically re-run transaction functions whenever necessary.
Nested transactions are very useful for keeping your code clean: using nested transactions ensures that you don't need to keep track of whether you already have a transaction open somewhere in your stack. Nested transactions are implemented as a client-side feature: committing a nested transaction does not need any communication with the server so they have very little cost.

Connection life-cycle

The Java client authenticates to the server using X509 certificates. These certificates can be generated by the goshawkdb server, and this is demonstrated in the getting started guide. The client also optionally uses the public certificate of the server to verify the identity of the server to which it is connecting. The examples in this guide directly embed the client certificate and key, and the server certificate, in the source code. These require adjusting for your own installation. If you wish to instead store these certificates in an external file, methods are provided to load certificates and keys from InputStreams.

Connections are created via the ConnectionFactory. The ConnectionFactory contains a Netty EventLoopGroup. There is a constructor in case you wish to provide your own EventLoopGroup. To create a connection, you'll need to create and populate a Certs object which contains the cluster public certificate (to authenticate the server) and the client certificate and key pair (to allow the server to authenticate the client). The Certs object is designed to load the certificates and keys in the same format that they were created by GoshawkDB.

Connections support AutoClosable so they can be used in try-with-resources statements.

package io.goshawkdb.examples;

import java.io.StringReader;

import io.goshawkdb.client.Certs;
import io.goshawkdb.client.Connection;
import io.goshawkdb.client.ConnectionFactory;

public class Untitled1 {
    private static final String clusterCert = "-----BEGIN CERTIFICATE-----\n" +
            "MIIBxzCCAW2gAwIBAgIIQqu37k6KPOIwCgYIKoZIzj0EAwIwOjESMBAGA1UEChMJ\n" +
            "R29zaGF3a0RCMSQwIgYDVQQDExtDbHVzdGVyIENBIFJvb3QgQ2VydGlmaWNhdGUw\n" +
            "IBcNMTYwMTAzMDkwODE2WhgPMjIxNjAxMDMwOTA4MTZaMDoxEjAQBgNVBAoTCUdv\n" +
            "c2hhd2tEQjEkMCIGA1UEAxMbQ2x1c3RlciBDQSBSb290IENlcnRpZmljYXRlMFkw\n" +
            "EwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEjHBXt+0n477zVZHTsGgu9rLYzNz/WMLm\n" +
            "l7/KC5v2nx+RC9yfkyfBKq8jJk3KYoB/YJ7s8BH0T456/+nRQIUo7qNbMFkwDgYD\n" +
            "VR0PAQH/BAQDAgIEMA8GA1UdEwEB/wQFMAMBAf8wGQYDVR0OBBIEEL9sxrcr6QTw\n" +
            "wk5csm2ZcfgwGwYDVR0jBBQwEoAQv2zGtyvpBPDCTlyybZlx+DAKBggqhkjOPQQD\n" +
            "AgNIADBFAiAy9NW3zE1ACYDWcp+qeTjQOfEtED3c/LKIXhrbzg2N/QIhANLb4crz\n" +
            "9ENxIifhZcJ/S2lqf49xZZS91dLF4x5ApKci\n" +
            "-----END CERTIFICATE-----";

    private static final String clientKeyPair = "-----BEGIN CERTIFICATE-----\n" +
            "MIIBszCCAVmgAwIBAgIIfOmxD9dF8ZMwCgYIKoZIzj0EAwIwOjESMBAGA1UEChMJ\n" +
            "R29zaGF3a0RCMSQwIgYDVQQDExtDbHVzdGVyIENBIFJvb3QgQ2VydGlmaWNhdGUw\n" +
            "IBcNMTYwMTAzMDkwODUwWhgPMjIxNjAxMDMwOTA4NTBaMBQxEjAQBgNVBAoTCUdv\n" +
            "c2hhd2tEQjBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABFrAPcdlw5DWQmS9mCFX\n" +
            "FlD6R8ABaBf4LA821wVmPa9tiM6n8vRJvbmHuSjy8LwJJRRjo9GJq7KD6ZmsK9P9\n" +
            "sXijbTBrMA4GA1UdDwEB/wQEAwIHgDATBgNVHSUEDDAKBggrBgEFBQcDAjAMBgNV\n" +
            "HRMBAf8EAjAAMBkGA1UdDgQSBBBX9qcbG4ofUoUTHGwOgGvFMBsGA1UdIwQUMBKA\n" +
            "EL9sxrcr6QTwwk5csm2ZcfgwCgYIKoZIzj0EAwIDSAAwRQIgOK9PVJt7KdvDU/9v\n" +
            "z9gQI8JnVLZm+6gsh6ro9WnaZ8YCIQDXhjfQAWaUmJNTgKq3rLHiEbPS4Mxl7h7S\n" +
            "kbkX/2GIjg==\n" +
            "-----END CERTIFICATE-----\n" +
            "-----BEGIN EC PRIVATE KEY-----\n" +
            "MHcCAQEEIN9Mf6CzDgCs1EbzJqDK3+12wcr7Ua3Huz6qNhyXCrS1oAoGCCqGSM49\n" +
            "AwEHoUQDQgAEWsA9x2XDkNZCZL2YIVcWUPpHwAFoF/gsDzbXBWY9r22Izqfy9Em9\n" +
            "uYe5KPLwvAklFGOj0YmrsoPpmawr0/2xeA==\n" +
            "-----END EC PRIVATE KEY-----";

    public static void main(final String[] args) throws Exception {
        Certs certs = new Certs();
        certs.addClusterCertificate("myGoshawkDBCluster", clusterCert.getBytes());
        certs.parseClientPEM(new StringReader(clientKeyPair));
        ConnectionFactory cf = new ConnectionFactory();
        try (Connection conn = cf.connect(certs, "hostname:7894")) {
            // do some work
        } finally {
            cf.group.shutdownGracefully();
        }
    }
}

hostname:port may specify a hostname, an FQDN or an IP address. The port component is optional: if omitted the default port of 7894 is used. The clientKeyPair is used to authenticate the client. If the fingerprint of the client certificate does not exist in the cluster configuration, the client will not be authenticated and the connection will be terminated. The clusterCert is recommended but not required: if it is not provided, the client will not be able to verify the server to which it connections. The clusterCert is the public certificate part of the certificate and key pair generated with the -gen-cluster-cert command line parameter. The connect method will not return until either a connection is fully established and ready for use, or has failed. Similarly, close will not return until the connection is closed.

Running a transaction

Use the Connection.runTransaction method. This method automatically detects if a transaction is already in progress and will create nested transactions as necessary. There is no limit to how many nested transactions you can create. The TransactionFunction that you supply is the transaction function and will automatically be run as many times as necessary until it is either able to successfully commit or chooses to abort. The final values that the transaction function returns are returned to the caller of runTransaction along with the transaction Id.

package io.goshawkdb.examples;

import java.io.StringReader;

import io.goshawkdb.client.Certs;
import io.goshawkdb.client.Connection;
import io.goshawkdb.client.ConnectionFactory;

public class Untitled2 {
    private static final String clusterCert = "...";
    private static final String clientKeyPair = "...";

    public static void main(final String[] args) throws Exception {
        Certs certs = new Certs();
        certs.addClusterCertificate("myGoshawkDBCluster", clusterCert.getBytes());
        certs.parseClientPEM(new StringReader(clientKeyPair));
        ConnectionFactory cf = new ConnectionFactory();
        try (Connection conn = cf.connect(certs, "hostname")) {
            String res = conn.runTransaction(txn ->
                    "Hello World"
            ).result;
            System.out.println(res);
        } finally {
            cf.group.shutdownGracefully();
        }
    }
}

This will print out Hello World.

To abort a transaction, throw a TransactionAbortedException exception. If a transaction aborts for any reason, the cause can be found in the cause field of the TransactionResult object which is returned by runTransaction. To start a retry, call the Transaction.retry method.

Objects and references

GoshawkDB stores an object graph. Every object has a unique Id and objects can have references (pointers) to other objects. Because GoshawkDB uses Object Capabilities as a security mechanism, you access all objects through references (pointers) to objects. A reference contains not only the unique Id of the object to which it points, but also a capability which is granted on that object. This capability grants the ability to operate on the object: which you can read, or write, or both, or neither, the object and its references. Every object has a value, which is a byte array (though in the Java client this is exposed as a Java ByteBuffer). GoshawkDB never needs to inspect the value of objects so the value is completely opaque and you are free to use any serialization you wish. The only requirement is that if one object points to another, you must declare that.

When the client first connects to the server, the server informs the client of the object Ids of the various root objects to which the client has access. Initially, the root objects are the only objects that you can access. As soon as you access an object, you gain access to the objects that are referenced (pointed to), as per the restrictions of the capabilities on each reference. Thus you can navigate the object graph. There are no restrictions on the shape of the object graph: cycles are permitted as is aliasing - it is a full graph, not a tree (unlike, say JSON). Objects are loaded on demand silently in the background as you navigate.

package io.goshawkdb.examples;

import java.io.StringReader;
import java.nio.ByteBuffer;

import io.goshawkdb.client.Certs;
import io.goshawkdb.client.Connection;
import io.goshawkdb.client.ConnectionFactory;
import io.goshawkdb.client.GoshawkObjRef;
import io.goshawkdb.client.TransactionResult;

public class Untitled3 {
    private static final String clusterCert = "...";
    private static final String clientKeyPair = "...";

    public static void main(final String[] args) throws Exception {
        Certs certs = new Certs();
        certs.addClusterCertificate("myGoshawkDBCluster", clusterCert.getBytes());
        certs.parseClientPEM(new StringReader(clientKeyPair));
        ConnectionFactory cf = new ConnectionFactory();
        try (Connection conn = cf.connect(certs, "hostname")) {

            TransactionResult<String> outcome = conn.runTransaction(txn -> {
                GoshawkObjRef root = txn.getRoots().get("myRoot1");
                if (root == null) {
                    throw new RuntimeException("No root 'myRoot1' found");
                }
                root.set(ByteBuffer.wrap("Hello".getBytes()));
                return "success!";
            });
            System.out.println("" + outcome.result + ", " + outcome.cause);

            outcome = conn.runTransaction(txn -> {
                GoshawkObjRef root = txn.getRoots().get("myRoot1");
                if (root == null) {
                    throw new RuntimeException("No root 'myRoot1' found");
                }
                ByteBuffer val = root.getValue();
                byte[] ary = new byte[val.limit()];
                val.get(ary);
                return new String(ary);
            });
            System.out.println("Found: " + outcome.result + ", " + outcome.cause);

        } finally {
            cf.group.shutdownGracefully();
        }
    }
}

Creating objects and adding references to them is easy too:

package io.goshawkdb.examples;

import java.io.StringReader;
import java.nio.ByteBuffer;

import io.goshawkdb.client.Certs;
import io.goshawkdb.client.Connection;
import io.goshawkdb.client.ConnectionFactory;
import io.goshawkdb.client.GoshawkObjRef;
import io.goshawkdb.client.TransactionResult;

public class Untitled4 {
    private static final String clusterCert = "...";
    private static final String clientKeyPair = "...";

    public static void main(final String[] args) throws Exception {
        Certs certs = new Certs();
        certs.addClusterCertificate("myGoshawkDBCluster", clusterCert.getBytes());
        certs.parseClientPEM(new StringReader(clientKeyPair));
        ConnectionFactory cf = new ConnectionFactory();
        try (Connection conn = cf.connect(certs, "hostname")) {

            TransactionResult<String> outcome = conn.runTransaction(txn -> {
                GoshawkObjRef root = txn.getRoots().get("myRoot1");
                if (root == null) {
                    throw new RuntimeException("No root 'myRoot1' found");
                }
                GoshawkObjRef obj = txn.createObject(ByteBuffer.wrap("a new value for a new object".getBytes()));
                root.set(null, obj); // root now has a single reference to our new object
                return "success!";
            });
            System.out.println("" + outcome.result + ", " + outcome.cause);

            outcome = conn.runTransaction(txn -> {
                GoshawkObjRef root = txn.getRoots().get("myRoot1");
                if (root == null) {
                    throw new RuntimeException("No root 'myRoot1' found");
                }
                GoshawkObjRef[] objs = root.getReferences();
                ByteBuffer val = objs[0].getValue();
                byte[] ary = new byte[val.limit()];
                val.get(ary);
                return new String(ary);
            });
            System.out.println("Found: " + outcome.result + ", " + outcome.cause);

        } finally {
            cf.group.shutdownGracefully();
        }
    }
}

Retry

Retry is an incredibly powerful feature. Within a transaction, if you retry, what you're saying is: "Look at all the objects I've read from in this transaction. Put me to sleep, until any of those objects are modified. When they are modified, tell me about them and restart this transaction." Thus retry is a very flexible form of triggers: it allows you to get the server to push data out to you rather than having to poll for it.

The tests for the Java Client include a ping-pong test which uses retry to make each connection wait until it's their turn to modify the root object's value.

What's next?

That's it for introducing the client. The API is very small, though quite powerful.

There are some howtos available which feature longer worked examples. These are written in Go, but the Go client and Java client APIs are very similar.
The Java Client integration tests may be of interest.