Wiring up Deploy for Websockets (part 1)

In the last few weeks we've introduced websockets into Deploy to improve our deployments. In the next couple of posts we'll delve into the how and why of this development.

In part 1 we'll be looking at the backend technology and setup, and in part 2 we'll journey to the frontend and see how everything hooks up.

Until this update, the largest load on our frontend servers was the polling for new log entries during a deployment. On average, every user watching a deployment would generate a poll request every 2 seconds, whether there were any new log entries or not. These requests were generating the majority of our HTTP traffic. Replacing these polling requests with persistent connections should reduce load on our frontend servers significantly, meaning we can serve up pages in the rest of the interface more quickly.

We'd tried a few development spikes in the past using pusher-style services, but found that making calls to an external service every time we generate a log entry to be far too slow. On top of this there are the obvious data security concerns of sending deployment logs to a third party.

For the backend we looked to Myxi, an in-house developed ruby library which we've used on a few smaller projects in the past. Myxi provides a simple interface to a message queue along with a flexible websocket server which can handle private subscriptions and user authentication. Although untested on a project of Deploy's size, Myxi utilises RabbitMQ for it's message queue which is well battle-tested and should have no trouble scaling.

We'll take a look at the code necessary to integrate Myxi into Deploy. The first thing to do is to set up an action to authenticate users, we use another in-house developed library called Authie for managing our user sessions, but it shouldn't be any more difficult for any other authentication framework. We simply find our user based on their session token and return a separate token for the websocket to use when subscribing to channels.

# lib/myxi_exchanges.rb
#
# Authenticate users based on their session token
Myxi::Action.add(:Authenticate) do |session, payload|
  auth_session = Authie::Session.active.find_by_token(payload['session_id'])

  if auth_session
    session.auth_object = auth_session.user
    session.send('Authenticated', :token => auth_session.token)
  else
    session.send('Error', :error => 'InvalidAuthentication')
  end
end

The first thing our frontend will do when it connects to the server is send an Authenticate request to establish the user's identity. Once that identity is confirmed, the frontend can subscribe to private channels, which will carry our data. We'll create one of these channels to send updates about a deployment's status.

# lib/myxi_exchanges.rb
#
# Exchange concerned with specific deployment events
Myxi::Exchange.add(:deployment) do |routing_key, user|
  deployment = Deployment.find_by_identifier(routing_key)

  user.is_a?(User) && deployment.is_a?(Deployment) && user.read?(deployment)
end

When the frontend subscribes to a channel, it specifies a channel name, (which will be deployment) and a routing key, which is specific to the deployment (in this case the UUID assigned to the deployment). The authentication block is passed the routing key and the user object set in the Authenticate action earlier. We can use these two pieces of information to determine if the authenticated user is allowed to subscribe to the channel.

Now our channel is in place, the final piece of the puzzle is to allow our deployment to send updates to the channel. We'll build a simple wrapper for future extension and error handling.

# lib/websocket_logger.rb
#
# Simple adapter to send messages to our websocket server
module WebsocketLogger
  class << self
    def deployment_log_entry(deployment, event, payload)
      log(:deployment, deployment.identifier, event, payload)
    end

    def log(exchange, routing_key, event, payload)
      Myxi.push_event(exchange, routing_key, event, payload)
    rescue Bunny::Exception, Timeout::Error => e
      Rails.logger.warn "#{e.class} #{e.message}"
      false
    end
  end
end

Now from our deployment we can simply call this adapter to send a message to the message queue:

WebsocketLogger.deployment_log_entry(self, 'status-change', status: self.status)

Add a rake task to start the websocket server as described in the Myxi documentation and your websocket server will be up and running.

Stay tuned for part two next week where we build the frontend.