Ruby
Article
By Glenn Goodrich

JSON Validation by Committee

By Glenn Goodrich

JSON API Documentation & Validation

comm

In my previous post, I went through the paces of creating a JSON Schema to describe the JSON that my service will accept and return. The process leans on the prmd gem to supply the initial JSON Schema templates, as well as validation and document generation. All in all, it’s a process that feels like it should be more automated. I went through a fair amount of pain to get the schema created, manually defining types and links in my text editor. At the end of the effort, I have a .json file that describes what my API will accept and return, as well as matching Markdown documentation to boot.

In that post, I made promise of something better. A land where writing unit tests that verify the JSON returned by my Rails API is possible. A land where the production application will reject requests that do not conform to our JSON schema. Does that sound like a wonderful place? It does to me.

This post will build that land, showing you how to feed the schema into tools provided by the committee gem, using those tools to verify tests and application requests.

The Accounts Application

I am using the same application as the prmd post. It’s a Rails API application that provided account information. It is not a complete app, as the examples here don’t require it. You can peruse the app to see the full schema. For today’s purposes, we simply need:

  • the RSpec tests to validate the JSON Schema.
  • requests to the application to be validated against the JSON Schema.

Setup

Check out the Gemfile for the gem’s I am using. The most important one is the committee gem, obviously.

Tests/Specs

OK, down to brass tax. I want to make sure that the AccountsController is returning appropriate responses based on my JSON Schema. Let’s look at the simplest link in the schema:

# In the "links" section of schema/schemata/account.json
{
  "description": "Info for existing account.",
   "href": "/accounts/{(%2Fschemata%2Faccount%23%2Fdefinitions%2Fidentity)}",
  "method": "GET",
  "rel": "self",
  "title": "Info",
  "targetSchema": {
    "type": [
      "object"
    ],
    "properties": {
      "data": {
        "type": [
          "object"
        ],
        "properties": {
          "id": {
            "$ref": "#/definitions/account/definitions/id"
          },
          "email": {
            "$ref": "#/definitions/account/definitions/email"
          }
        }
      }
    },
    "required": [
      "data"
    ]
  }
}

The /account/#{account.id} path is a GET request with a response that looks like:

{
  "data": {
    "id": 1234,
    "email": "someone@somewhere.com"
  }
}

Currently, AccountController#show looks like:

def show
  render json: current_account
end

(Note: current_account is a simple helper that finds the account based on params[:id])

Here is the start of a spec:

describe "GET #show" do
  # Using FactoryGirl
  let(:account) { create(:account, email: email, password: password) }
  let(:email) { Faker::Internet.email }
  let(:password) { Faker::Internet.password }

  it "conforms to schema" do
    get :show, id: account.id
    # HOW???
  end
end

How do we write a test that validates the response? This is what the response to a /accounts/#{account_id} looks like, currently:

{
  "id":3,
  "email":"adriana@rodriguez.net",
  "encrypted_password":"$2a$10$D6TkSSCRU4WHwVUzGsV.BeEoIRoBuJEZkuWCj.Ys4PlNJzvmoulzm",
  "password_salt":"$2a$10$D6TkSSCRU4WHwVUzGsV.Be","jti":null,
  "reset_password_token":null,
  "refresh_token":null,
  "reset_password_sent_at":null,
  "created_at":"2015-08-16T14:21:50.061Z",
  "updated_at":"2015-08-16T14:21:50.061Z"
  }

Clearly, this isn’t what we want. Luckily, the committee gem provides test methods in Committee::Test::Methods, which needs to be included into the spec:

RSpec.describe AccountsController, type: :controller do
  include Committee::Test::Methods
  ...

The method we’re most interested in for this spec is called assert_schema_conform. There is a mild amount of shoe-horning to get that matcher to work in Rails, as the method expects schema_path, last_request, and last_response to exist. The RSpec let syntax makes this simple:

describe "GET #show" do
  # Using FactoryGirl
  let(:account) { create(:account, email: email, password: password) }
  let(:email) { Faker::Internet.email }
  let(:password) { Faker::Internet.password }
  let(:schema_path) { "#{Rails.root}/schema/authentication-api.json" }
  let(:last_response) { response }
  let(:last_request) { request }

  it "conforms to schema" do
    get :show, id: account.id
    assert_schema_conform
  end
end

schema_path points to the JSON Schema file generated with prmd. last_response and last_request are conventions from Sinatra testing.

Running the specs (rspec) results in the following:

1) AccountsController GET #show when the token is valid conforms to schema
   Failure/Error: assert_schema_conform
   Committee::InvalidResponse:
     Invalid response.

     #: failed schema #/definitions/account/links/4/targetSchema: "data" wasn't supplied.
   # /Users/ggoodrich/.rvm/gems/ruby-2.2.2@sp-json-schema/gems/committee-1.9.1/lib/committee/response_validator.rb:37:in `call'
   # /Users/ggoodrich/.rvm/gems/ruby-2.2.2@sp-json-schema/gems/committee-1.9.1/lib/committee/test/methods.rb:23:in `assert_schema_conform'
   # ./spec/controllers/accounts_controller_spec.rb:40:in `block (4 levels) in <top (required)>'

BOOM! That’s what we want. The spec complains that the response does not include data. Let’s change how the account is serialized:

# app/controllers/accounts_controller.rb
...

def show
  account_props = {
    data: {
      id: current_account.id,
      email: current_account.email
    }
  }
  render json: account_props
end
...

This will work great, I am sure of it:

 1) AccountsController GET #show when the token is valid conforms to schema
    Failure/Error: assert_schema_conform
    Committee::InvalidResponse:
      Invalid response.

      #/data/id: failed schema #/definitions/account/links/4/targetSchema/properties/data/properties/id: 4 is not a string.
...

What the what??? OOOOH riiiight! I forgot that I defined the account ID as a UUID, which is a string:

...
"id": {
  "description": "unique identifier of account",
  "readOnly": true,
  "format": "uuid",
  "type": [
    "string"
  ]
},
...

We’re using UUIDs for the account identifier, but I didn’t implement it that way in the app. Since the JSON Schema was reviewed and agreed up on by the team, it is correct and the “source of truth”. We had a couple of situations in our efforts where the agreed upon schema was NOT what was being returned by the implementation. It probably saved us a couple of rounds of oops-fix-deploy, which made the whole effort worth it. Furthermore, it motivated the team to keep the schema in sync. </moraleOfStory>

Back to our test, after going through the steps to make id a UUID (this is harder than it should be for SQLite3, btw), the tests pass. Check the repo for the UUID steps.

Committee will also validate that the request path is in the schema links. As an example, if I change the JSON Schema GET account path to /account/{id} (I removed an ‘s’) and rerun the specs, the result is:

1) AccountsController GET #show when the token is valid conforms to schema
   Failure/Error: assert_schema_conform
   Committee::InvalidResponse:
     `GET /accounts/c0adbb04-7706-4da8-8c04-d82079b72287` undefined in schema.
...

Excellent. Committee will test for valid links (path and method) and response format. We’ve now realized one of the major benefits that Committee offers.

Live Request/Response Validation

Another item that Committee supplies is middleware that validates incoming requests against the JSON Schema. The middleware comes in two flavors: Committee::Middleware::RequestValidation and Committee::Middleware::ResponseValidation. Like other middlewares, they are added to the Rails configuration files:

#config/application.rb

module SpJsonSchemaRails
  class Application < Rails::Application
    ...
    schema_file = "#{Rails.root}/schema/authentication-api.json"
    if File.exists?(schema_file)
      config.middleware.use Committee::Middleware::RequestValidation, schema: JSON.parse(File.read(schema_file)), strict: true
      config.middleware.use Committee::Middleware::ResponseValidation, schema: JSON.parse(File.read(schema_file))
    end
    ...
  end
end

Here, there’s quick check that the schema file exists, then the middleware is engaged. Each middleware has a its own set of options that allow some fine tuning.

Request Validation

Committee::Middleware::RequestValidation takes the following options (straight from the README):

  • allow_form_params: Specifies that input can alternatively be specified as application/x-www-form-urlencoded parameters when possible. This won’t work for more complex schema validations.
  • allow_query_params: Specifies that query string parameters will be taken into consideration when doing validation (defaults to true).
  • check_content_type: Specifies that content_type should be verified according jsonschema definition. (defaults to true).
  • error_class: Specifies the class to use for formatting and outputting validation errors (defaults to Committee::ValidationError)
  • optimistic_json: Will attempt to parse JSON in the request body even without a Content-Type: application/json before falling back to other options (defaults to false).
  • prefix: Mounts the middleware to respond at a configured prefix.
  • raise: Raise an exception on error instead of responding with a generic error body (defaults to false).
  • strict: Puts the middleware into strict mode, meaning that paths which are not defined in the schema will be responded to with a 404 instead of being run (default to false).

Let’s use curl on our application to see how this works:

# This path doesn't exist and we're using 'strict: true'
curl http://localhost:3000/account
=> {"id":"not_found","message":"That request method and path combination isn't defined."}

# This one should work, right? :)
curl http://localhost:3000/accounts/953cff00-23ac-47dc-bb68-4b391e75aae7
==> {"id":"invalid_response","message":"Invalid response.\n\n#/data/id: failed schema #/definitions/account/links/4/targetSchema/properties/data/properties/id: 953CFF0023AC47DCBB684B391E75AAE7 is not a valid uuid."}

Whoa! What’s that? Seems like we have a discrepancy between what the specs are returning and what the application actually returns. Hmph. Well, let’s use this as a chance to point out the response validation working coughs.

OK, one little fix and it’ll work:

# app/controllers/accounts_controller.rb
...

def show
  account_props = {
    data: {
      id: current_account.id.to_s, # to_s makes the UUID not BELIKETHIS
      email: current_account.email
    }
  }
  render json: account_props
end
.

curl http://localhost:3000/accounts/953cff00-23ac-47dc-bb68-4b391e75aae7
=> {"data":{"id":"953cff00-23ac-47dc-bb68-4b391e75aae7","email":"email@email.com"}}

In a real app you should be using a serialization framework, like active_model_serializers or roar, so you’d likely avoid this issue.

The important takeaway here is that an invalid request won’t get to your controller. That is called failing fast and all the cool kids are doing it.

Response Validation

Committee::Middleware::ResponseValidation has the following options (again, straight from the README):

  • error_class: Specifies the class to use for formatting and outputting validation errors (defaults to Committee::ValidationError)
  • prefix: Mounts the middleware to respond at a configured prefix.
  • raise: Raise an exception on error instead of responding with a generic error body (defaults to false).
  • validate_errors: Also validate non-2xx responses (defaults to false)

The most interesting option here, to me, is error_class. This allows you to specify a class to use to return errors. We really wanted this option so our errors could be JSON API compliant.

We already saw an example of response validation working, so let’s move on to what else Committee brings to the party.

Stubbing a Server

One very cool feature of Committee is the ability to stub out a server based on the JSON Schema. If we add the following line to config/application.rb:

...other config...
use Committee::Middleware::Stub, schema: JSON.parse(File.read("#{Rails.root/schema/authentication-api.json}"))

and start the server, we can hit any of the links defined in our JSON schema without having to implement them. This is crazy cool.

For example, we haven’t touched the POST /account/session which is the “Sign In” link. The JSON schema takes an account parameter with email and password, and returns a token:

{
  "description": "Sign in (generate token)",
  "href": "/account/session",
  "method": "POST",
  "rel": "",
  "schema": {
    "properties": {
      "account": {
        "type": [
          "object"
        ],
        "properties": {
          "email": {
            "$ref": "#/definitions/account/definitions/email"
          },
          "password": {
            "type": [
              "string"
            ],
            "description": "The password"
          },
          "remember_me": {
            "type": [
              "boolean"
            ],
            "description": "True/false - generate refresh token (optional)"
          }
        }
      }
    },
    "type": [
      "object"
    ],
    "required": [
      "account"
    ]
  },
  "targetSchema": {
    "type": [
      "object"
    ],
    "properties": {
      "token": {
        "$ref": "#/definitions/account/definitions/token"
      }
    }
  },
  "title": "Sign In"
},
...

I can issue a curl request and get back the expected result:

curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST http://localhost:3000/account/session  -d '{"account": {"email": "test@test.com", "password":"password"}}'

HTTP/1.1 200 OK
Content-Type: application/json
Etag: W/"d36cc07cb95b45bc67964243f0c35795"
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: a86b8f12-6d83-4eb8-a750-01331e802243
X-Runtime: 0.002147
Server: WEBrick/1.3.1 (Ruby/2.2.2/2015-04-13)
Date: Sun, 16 Aug 2015 20:47:15 GMT
Content-Length: 546
Connection: Keep-Alive

{"token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzUxMiJ9.eyJkYXRhIjp7ImlkIjoiMTE0MzYiLCJ0eXBlIjoiYWNjb3VudHMiLCJhdHRyaWJ1dGVzIjp7ImVtYWlsIjoiZ2xlbm4uZ29vZHJpY2hAZ21haWwuY29tIn19LCJzdWIiOiJhY2NvdW50IiwiZXhwIjoxNDM3MjM0OTM0LCJpc3MiOiJVbmlxdWUgVVNBIiwiaWF0IjoxNDM3MTQ4NTM0LCJqdGkiOiI3ZmJiYTgzOS1kMGRiLTQwODItOTBmZC1kNmMwM2YwN2NmMWMifQ.SuAAhWPz_7VfJ2iyQpPEHjAnj_aZ-0-gI4uptFucWWflQnrYJl3Z17vAjypiQB_6io85Nuw7VK0Kz2_VHc7VHZwAjxMpzSvigzpUS4HHjSsDil8iYocVEFlnJWERooCOCjSB9R150Pje1DKB8fNeePUGbkCDH6QSk2BsBzT07yT-7zrTJ7kRlsJ-3Kw2GDnvSbb_k2ecX_rkeMeaMj3FmF3PDBNlkM"}

The response relies on `examples` being defined for each field in the JSON schema file.

Committee also provides a committee_stub executable that will launch a server based on the same schema file:

committee-stub -p 3000 schema/authentication-api.json

And, BANGO, working API server. How cool is that?

The obvious use case here is the ability to provide a “working” implementation of the server to other teams or designers while you work on the implementation.

Join the Committee

The committee gem provides some desperately needed functionality to any team creating API servers in Ruby. While today’s examples use Rails, both the prmd and committee gems were created for pliny, which is a Sinatra-based framework. If you are creating APIs, you need to start treating the JSON like it’s a grown-up part of the application. PRMD and Committee give you the tools to do just that.

If you have any thoughts or think I got something wrong, please let me know in the comments.

  • Fred@Bootstrap

    This is very useful. The one thing I always envied from XML over JSON was the ability to easily validate payloads with XML Schema. It seems that with commitee and prmd this can be just as easy in the JSON world.

    • ggsp

      Hey Fred. We were really happy to find it and the results have been great. All our APIs have docs and validation now. Thanks for the comment.

  • DmitriyNesteryuk

    The feature for stubbing api responses is very interesting. But, how does it work when you need a specific state? For example, you need confirmed/unconfirmed user for your tests or the response about failed confirmation because of the expired token.

  • panSarin

    If you test like that do you have to define other json for each endpoint, or comittee assert method looks for proper key in .json file depending on your request?

    I mean if you have links for show and for update f.e. will it search for JSON schema that containst /resource/{$ID} and method that you used in your request?

Recommended
Sponsors
Get the latest in Ruby, once a week, for free.