If you’ve worked with me in the past, you know I’m obsessed with Event Sourcing as a software pattern. For those of you who don’t know about Event Sourcing, here’s my take on it (which is slightly different from the rest of the world…):

1. The source of truth are the events, nothing else.
2. State is derived from events (see #1)
3. Events have a specific order that they occur in, called “versioning”, but they cannot be guaranteed that they will be in the order they occur. This is important.
4. Events have a stream id, to help with serialization.
5. State can be memoized, called snapshots. However, if the meaning of past events change, the snapshot state will need to be recalculated.
6. Testing is easy: put events in, assert events out, assert state.

So who cares?

I’m unhappy with just about every implementation of event sourcing I’ve seen. They’re either too complex, don’t treat events as source of truth, or fail to be open source. This has led me to create my own implementation over the years, which I’ll be releasing, :soon:

As soon as I figure out a name for it.

About this mythical library

It’s written for nodejs and rethinkdb, though a future iteration may include crate (it’s very easy to swap out storage engines) or redis.

My first iteration was written for C#, which became unwieldy in a certain situation that I’ll explain later. I chose node because of JavaScript’s immense flexibility, though it’s possible golang would work as well. It’s very easy to write Actors, which can model business processes and real life.

class User extends LiveActor {
    constructor( id, injectionContainer ) {
        super( id, injectionContainer );
        
        // set initial state for all users
        this._state[ 'logged_in' ] = false;
    }
    
    /**
     * Login a user
     */
    DoLoginAttempt( password, browserFingerprint ) {
        if ( this._state[ 'too_many_tries' ] ) {
            return this.Fire( 'too_many_tries' );
        }
        
        if ( this._state[ 'password' ] === password ) {
            const token = this.generateToken();
            this.Fire( 'logged_in', {
                browserFingerprint,
                token
            } );
            return token;
        } else {
            return this.Fire( 'invalid_login', {
                browserFingerprint
            } );
        }
    }
    
    /* Event handlers follow */
    
    /*
     * Handle an invalid login attempt
     */
    invalid_login( data ) {
        // do logic
    }
}

What this means:

1. Automatic audit trail.
2. Often the event handlers are on the same screen as the actions.
3. Actors can stay resident, performing actions and reacting to events in the background, but that’s not necessary
4. Free concurrency. If I add lives to a player, it will be updated instantly across the cluster and all their devices currently logged in.
5. Concurrency protection. Two competing events cannot be applied at the same time, so, the failing event will automatically get reapplied to the state and broadcast.
6. If there’s a bug, I can reapply the events locally and watch the bug happen in realtime.

Testing is also really easy:

await Given( 'A user logs in for the first time', [
    {
        name: 'signup',
        data: {
            username: 'test',
            password: 'test'
        }
    }
] ).When( User, 'DoLogin', 'test', {} ).Then( [
    {
        name: 'logged_in',
        data: {
            browserFingerprint: {},
            token: '{string}'
        }
    }
] ).AndState( {
    logged_in: true,
    invalid_password_tries_in_last_period: 0
    // etc
} )

This makes TDD very, very easy.

Actors

What’s interesting about my implementation of Actors? Actors can share id’s with each other, a sort of polymorphism, if you will.

example:

const user = new User( 'userName', container );
await user.Load();
user.DoGetPayment( payment );

const player = new Player( 'userName', container );
await player.Load();
assert( player._state[ 'lives' ], 1 );

In this example, a user represents a physical user, who has a credit card and an id of ‘userName’. They make a payment. A player represents a player in a game, who may be interested in some of the user’s events, such as payments, so that it can gain lives. They represent the same physical thing, but are used in very different contexts. It’s important to note that both objects are extending the same base class, there is no inheritance at play here.

This allows the programmer to separate concerns, while not worrying too much about some other context’s concerns. This has some great power, at a slight cost to maintain. For example, in inventory tracking, you have inventory in the context of shipping, and inventory in the context of listing, picking, and receiving. In the context of shipping, you really don’t care what shelf it was on.

In a traditional CRUD application, these things might exist on different tables, with a join to bring it all together.

In this library, there’s several different kinds of Actors:

• Actor: These offer a simple, serial view of the world. They offer no concurrency protection or distributed effects.
• LiveActor: These offer distributed transactions across a cluster, in realtime. These are good for long running lifetimes. They handle concurrency very well.
• OneActor: These offer a singleton actor, with a single leader if involved in a cluster environment. A new leader will become elected within a few seconds if the leader goes away.
• TickActor: These offer distributed actors a sense of time. They fire a tick event with some arbitrary resolution. The resolution is “at-least”, meaning if you want 10 ticks to happen with a resolution of 1 second, then you can expect to wait “at-least” 10 seconds, but it may be more. In a distributed system, you’re unlikely to get better than that.

All actors are optimistic, except OneActor’s. Meaning an event is likely to be applied locally, even if it fails to apply across the whole cluster. It is guaranteed to be eventually applied, however. The exception is OneActor actors, observe:

const singleActor = new SingleActor( id, container );
await singleActor.Load();
singleActor.Fire( 'someAction', { data } );

someAction will not be applied locally. It will, however, be acknowledged and applied by the leader, which will then apply the event locally and across all instances simultaneously. If after a set period, the leader doesn’t fire the event, it will assume leadership and the old leader will step down, if it’s still around. In the edge case that more than one actor becomes leader at the same time, they will work together as leaders in the same way that LiveActors work until one of them steps down as leader (usually one or two events later). This makes them tolerant of network / logical partitions.

Pub/Sub

All actors have the capability to subscribe to other actors or even events with a given name. This allows using actors to handle processes, such as payments, business processes, and longer running transactions. For example:

class Game extends LiveActor {
    constructor( room, container, playerIds ) {
        super( room, container );
        
        playerIds.forEach( ( playerId ) => {
            this.ListenTo( playerId, ( event ) => {
                switch( event.name ) {
                    case 'player_exit':
                        // eject player from game
                        this.Fire( 'player_ejected', { playerId } );
                        break;
                    case 'player_death':
                        // respawn player
                        this.Fire( 'player_respawned', { playerId } );
                        break;
                }
            } );
        } );
    }
}

Release

You can find the documentation at https://withinboredom.github.io/omniscient-net/ and can install it with npm install --save omniscient-net…