Flux over the Wire

When doing real world web apps, there is a very common pattern: single source of truth shared by multiple clients. The best example of this is a chat: the message list, and the user list, must be readable and kept up to date for every connected client.

Lets assume for a moment that all your clients are React apps executed in the same browser window. Then its easy: just use Flux! All clients will update their components every time the stores representing the message list and the user list is updated. Message posting will just be an action dispatch, as well as users joining or leaving a room.

The problem is that all your clients are not in the same browser window. They actually are on different computers, on different hosts, and their only means of interacting with the server it through asynchronous communication (typically HTTP requests).

This seems like a hard problem. Or is it?

Flux is symmetrical

Flux is often presented as an asymmetrical pattern, like the following:

                +--------------------+
              v                    |
Action -> Dispatcher -> Store -> View

But there is another way to see, that keeps the core idea of flux of decoupling stores and actions, and that is completely symetrical:

                +----------------> onChange ---------------+
              |                                          v
Store Producer/Action Consumer             Store Consumer/Action Producer
      "SSoT" or "Dispatcher"                     "View" or "Component"
              ^                                          |
              +---------------- onDispatch --------------+

onChange/onDispatch is implementation detail

Once you see it this way, how onChange and onDispatch are actually triggered is implementation detail.

The whole purpose of one-way data flow is that everything is fire & forget from the point of view of the producer, whether it produces actions or updates. A component just fires an action dispatch. Maybe this will trigger a store update and they will get a feedback at some point, but the strength of Flux is that the component doesn't need to care. Its simply not its concern. Same goes for dispatchers.

In fact, to have flux working, you only need a guaranteed order, asynchronous communication mechanism so that:

  • Store that can notify their consumers when they are changed

  • Actions that can notify their consumers when they are dispatched

And thats about it. So you can use plain function callbacks. Or EventEmitter. Or Promise. Or generators. Or Websockets!

Flux over Websockets !?

So what about using Websockets as the communication channel for Flux?

It means that you can have your single source of truth on the server, and your components on the clients. Actions are Client to Server Websocket messages (or HTTP requests), and updates are Server to Client Websocket messages (or XHR stream packets). From the point of view of the server, a client is just an inbound action stream, and an outbound updates stream. From the point of view of the client, the server is just an inbound updates steam, and an outbound actions streams.

                +--------> JSON.stringify(changes) --------+
              |                                          v
Store Producer/Action Consumer             Store Consumer/Action Producer
      "SSoT" or "Dispatcher"                     "View" or "Component"
              ^                                          |
              +---- JSON.stringify(action, payload) -----+

  in the browser        Websocket frames     in the server

Component #A1 <---+
                  |
Component #A2 <---+-- SocketIOAdapter -+
                  |      Client A      |
Component #A3 <---+                    |
                                       +-> SSoT/Dispatcher
Component #B1 <---+                    |
                  |                    |
Component #B2 <---+-- SocketIOAdapter -+
                  |      Client B
Component #B3 <---+

Note that under this paradigm and thanks to the brilliant design of flux, their is no actually shared mutable state (and we all know that shared mutable state if the root of all evil © @floydophone).

For exactly the same reason that it is perfectly okay to have many components subscribe to the same stores and fire the same actions, then it is perfectly okay to have many clients subscribe to the same server and send actions to the same server.

How does it look like, code-wise? It seems complicated.

The code for the message list example will look roughly like that on the server:

  const server = new FluxServer(8080);
const messageList = server.Store('/messageList');
messageList.set('nextId', 0).commit();
const postMessage = server.Action('/postMessage')
.onDispatch(({ nickname, message }) => {
  messageList.set(messageList.get('nextId') + 1, { nickname, message })
  .set('nextId', messageList.get('nextId') + 1)
  .commit()
});

And the client will look like:

  const client = new FluxClient('http://localhost:8080');
const messageList = client.Store('/messageList');
const postMessage = client.Action('/postMessage');
const MessageList = React.createClass({
  getInitialState() {
    return { messageList: this.props.messageList.head, message: null };
  },

  componentDidMount() {
    messageList.onUpdate(({ head }) => this.setState({ messageList: head }));
  },

  submitMessage() {
    this.props.postMessage.dispatch({ message: this.state.message });
  },

  render() {
    return <div>
      <ul>Messages
        {messageList.map(({ message }, key) => 
          <li ...{key}><Message ...{ message } /></li>
        )}
      </ul>
      <input value={this.state.message} 
        onChange={({ target }) => this.setState({ message: target.value })} />
      <button onClick={this.submitMessage} />
    </div>;
  }
});

React.render(<MessageList ...{ messageList, postMessage } >);

Super simple!

How does it work?

Conceptually, its simple: we have a store which is in state A, we dispatch an action, this store transitions to state B. We just have to make sure every client is updated to reflect this new state.

We'll just apply the good old React strategy: diffing! So we diff state B and state A, so that since client is already in state A, then if we transmit this diff over the wire, and apply this diff on the client, then it will transition to state B. Fair enough.

But deep diffing objects is super slow! The real question is: can we make it fast enough? :)

Much like React does, here we'll also use an heuristic. We won't actually diff objects. We will rather record the mutations (or more accurately, the transitions between immutable states), and just replay them on the client. From an abstract point of view, its equivalent to diffing, its just so much faster.

Can we use it today?

As a matter of fact, I've been working on this for quite a while.

These implementations are not battle tested, and probably need some work before you can actually use it in production apps, but hey, they work. I've used them in demos and I fully intent to use them in production very soon.

Since it involves several very distinct part, I've splitted them into distinct modules:

  • Remutable, the Immutable JS wrapper that enables transitions and fast diffs/patchs/serialization.
  • Nexus Flux, my implementation of Flux that abstracts away how onChange and onDispatch is actually transmitted.
  • Nexus Flux socket.io adapter implements socket.io adapter for Nexus Flux and handles the communication between the server and then clients.

Free benefits:

  • Nexus Flux can be used as your local "traditional" flux backend using Promises. It also comes with an experimental WebWorker/postMessage backend that should allow you to defer complex action dispatch/store update logic off the main thread (eg. data crunching). I've not used it yet but this seems like a really good candidate to do fun stuff.
  • Nexus Flux socket.io is super cache friendly: stores "initial states" are exposed to HTTP GET, and each patch gets a unique version identifier that can be used to cachebust very precisely (without overcaching or undercaching).
  • You get to use React Nexus for free server-side rendering, SEO goodness, and stuff, but this will be the the topic of another blogpost :)