Dealing with Asynchronous APIs in Server-rendered React

September 21, 2017

If you’ve ever made a basic React app page, it probably suffered from poor SEO and performance issues on slower devices. You can add back traditional server-side rendering of web pages, typically with NodeJS, but this isn’t a straightforward process, especially with asynchronous APIs.

The two main benefits you get from rendering your code on the server are:

increased performance in load times
improving the flexibility of your SEO.

Remember that Google does wait for your JavaScript to load, so simple things like title content will change without issue. (I can’t speak for other search engines, though, or how reliable that is.)

In this post, I’ll discuss getting data from asynchronous APIs when using server-rendered React code. React code has the entire structure of the app built in JavaScript. This means that, unlike traditional MVC patterns with a controller, you don’t know what data you need until the app is rendered. With a framework like Create React App, you can quickly create a working app of very high quality, but it requires you to handle rendering only on the client. There’s a performance issue with this, as well as an SEO/data issue, where traditional templating engines you can alter the head as you see fit.

The Problem

React renders synchronously for the most part, so if you don’t have the data, you render a loading screen and wait for the data to come. This doesn’t work so well from the server, because you don’t know what you need until you’ve rendered, or you know what you need but you’ve already rendered.

Check out this stock-standard render method:

ReactDOM.render(
  <provider store={store}>
    <browserrouter>
      <app></app>
    </browserrouter>
  </provider>
, document.getElementById('root')
)

Issues:

It’s a DOM render looking for a root element. This doesn’t exist on my server, so we have to separate that.
We don’t have access to anything outside our main root element. We can’t set Facebook tags, title, description, various SEO tags, and we don’t have control over the rest of the DOM outside the element, especially the head.
We’re providing some state, but the server and client have different states. We need to consider how to handle that state (in this case, Redux).

So I’ve used two libraries here, and they’re pretty popular, so hopefully it carries over to the other libraries you’re using.

Redux: Storing state where your server and client are synced is a nightmare issue. It’s very costly, and usually leads to complex bugs. On the server side, ideally, you don’t want to do anything with Redux apart from just enough to get things working and rendering correctly. (You can still use it as normal; just set enough of the state to look like the client.) If you want to try, check out the various distributed systems guides as a starting point.

React-Router: FYI, this is the v4 version, which is what is installed by default, but it’s significantly different if you’ve got an older existing project. You need to make sure you handle your routing server side and client side and with v4 — and it’s very good at this.

After all, what if you need to make a database call? Suddenly this becomes a big issue, because it’s async and it’s inside your component. Of course, this isn’t a new issue: check it out on the official React repo.

You have to render in order to determine what dependencies you need — which need to be determined at runtime — and to fetch those dependencies before serving to your client.

Existing Solutions

Below, I’ll review the solutions that are currently on offer to solve this problem.

Next.js

Before we go anywhere, if you want production, server-side-rendered React code or universal app, Next.js] is where you want to go. It works, it’s clean, and it’s got Zeit backing it.

However, it’s opinionated, you have to use their toolchain, and the way they handle async data loading isn’t necessarily that flexible.

Check out this direct copy from the Next.js repo documentation:

import React from 'react'
export default class extends React.Component {
  static async getInitialProps ({ req }) {
    return req
      ? { userAgent: req.headers['user-agent'] }
      : { userAgent: navigator.userAgent }
  }
  render () {
    return <div>
      Hello World {this.props.userAgent}
    </div>
  }
}

getInitialProps is the key there, which returns a promise that resolves to an object that populates props, and only on a page. What’s great is that’s just built in to their toolchain: add it and it works, no work required!

So how do you get database data? You make an API call. You don’t want to? Well, that’s too bad. (Okay, so you can add custom things, but you have to fully implement it yourself.) If you think about this, though, it’s a very reasonable and, generally speaking, good practice, because otherwise, your client would still be making the same API call, and latency on your server is virtually negligible.

You’re also limited in what you have access to — pretty much just the request object; and again, this seems like good practice, because you don’t have access to your state, which would be different on your server versus client anyways. Oh, and in case you didn’t catch it before, it only works on top-level page components.

Redux Connect

Redux Connect is a very opinionated server-side renderer, with a decent philosophy, but if you don’t use all the tools they describe, this might not be for you. There’s a lot to this package, but it’s so complex and not yet upgraded to React Router v4. There’s a lot of setup to this, but let’s take the most important part, just to learn some lessons:

// 1. Connect your data, similar to react-redux @connect
@asyncConnect([{
  key: 'lunch',
  promise: ({ params, helpers }) => Promise.resolve({ id: 1, name: 'Borsch' })
}])
class App extends React.Component {
  render() {
    // 2. access data as props
    const lunch = this.props.lunch
    return (
      <div>{lunch.name}</div>
    )
  }
}

Decorators aren’t standard in JavaScript. They’re Stage 2 at the time of writing, so use at your discretion. It’s just another way of adding higher-order components. The idea is pretty simple: the key is for what to pass to your props, and then you have a list of promises, which resolve and are passed in. This seems pretty good. Perhaps an alternative is simply this:

@asyncConnect([{
  lunch: ({ params, helpers }) => Promise.resolve({ id: 1, name: 'Borsch' })
}])

That seems doable with JavaScript without too many issues.

react-frontload

The react-frontload repo doesn’t have a lot of documentation, or explanation, but perhaps the best understanding I could get was from the tests (such as this one)
and just reading the source code. When something is mounted, it’s added to a promise queue, and when that resolves, it’s served. What it does is pretty good, though it’s hard to recommend something that’s not well documented, maintained or used:

const App = () => (
  <frontload isServer >
    <component1 entityId='1' store={store}></component1>
  </frontload>
)

return frontloadServerRender(() => (
  render(<app></app>)
)).then((serverRenderedMarkup) => {
  console.log(serverRenderedMarkup)
})

Finding a Better Solution

None of the solutions above really resonated with the flexibility and simplicity I would expect from a library, so now I’ll present my own implementation. The goal is not to write a package, but for you to understand how to write your own package, for your use case.

The repo for this example solution is here.

Theory

The idea behind this is relatively straightforward, though it ends up being a fair bit of code. This is to give an overview of the ideas we’re discussing.

The server has to render the React code twice, and we’ll just use renderToString for that. We want to maintain a context between first and second renders. On our first render, we’re trying to get any API calls, promises and asynchronous actions out of the way. On our second render, we want to get all the data we acquired and put it back in our context, therefore rendering out our working page for distribution. This also means that the app code needs to perform actions (or not) based on the context, such as whether on the server or on the client, whether or not data is being fetched in either case.

Also, we can customize this however we want. In this case, we change the status code and head based on our context.

First Render

Inside your code, you need to know you’re working off the server or your browser, and ideally you want to have complex control over that. With React Router, you get a static context prop, which is great, so we’ll be using that. For now, we’ve just added a data object and the request data as we learned from Next.js. Our APIs are different between the server and the client, so you need to provide a server API, preferably with a similar interface as your client-side API:

const context = {data: {}, head: [], req, api}
const store = configureStore()
renderToString(
  <provider store={store}>
    <staticrouter location={req.url}
      context={context}
    >
      <app></app>
    </staticrouter>
  </provider>
)

Second Render

Right after your first render, we’ll just grab those pending promises and wait till those promises are done, then re-render, updating the context:

const keys = Object.keys(context.data)
const promises = keys.map(k=>context.data[k])
try {
  const resolved = await Promise.all(promises)
  resolved.forEach((r,i)=>context.data[keys[i]]=r)
} catch (err) {
  // Render a better page than that? or just send the original markup, let the front end handle it. Many options here
  return res.status(400).json({message: "Uhhh, some thing didn't work"})
}
const markup = renderToString(
  <provider store={store}>
    <staticrouter location={req.url}
      context={context}
    >
      <app></app>
    </staticrouter>
  </provider>
)

App

Quick jump away from our server to app code: in any of our components that have the router connection, we can now get that:

class FirstPage extends Component {
  async componentWillMount(){
    this.state = {text: 'loading'}

    this._handleData('firstPage')
  }
  async _handleData(key){
    const {staticContext} = this.props

    if (staticContext && staticContext.data[key]){
      const {text, data} = staticContext.data[key]
      this.setState({text, data})
      staticContext.head.push(
        <meta name="description" content={"Some description: "+text}/>
      )
    } else if (staticContext){
      staticContext.data[key] = this._getData()
    } else if (!staticContext && window.DATA[key]){
      const {text, data} = window.DATA[key]
      this.state = {...this.state, text, data}
      window.DATA[key] = null
    } else if (!staticContext) {
      const {text, data} = await this._getData()
      this.setState({text, data})
    }
  }
  async _getData(){
    const {staticContext} = this.props
    const myApi = staticContext ? staticContext.api : api
    const resp = await butter.post.list()
    const {data} = resp.data
    const {text} = await myApi.getMain()
    return {text, data}
  }
  render() {
    const text = this.state.text
    return (
      <div className='FirstPage'>
        {text}
      </div>
    )
  }
}

Wow, that’s a lot of complex code. At this stage, you probably want to take a more relay approach, where you separate your data fetch code in to another component.

This component is bookended by things you’re probably familiar with — a render step and a componentWillMount step. The four-stage if statement handles the different states — prefetch, post fetch, preserver render, post server render. We also add to the head after our data is loaded.

Finally, there’s a get data step. Ideally, your API and database have the same API, which makes execution the same. You’ll probably want to put these into an action in Thunk or Saga to make it more extensible.

Checkout the article “Server-Side React Rendering” and the repo React Server-side Rendering for more information. Remember, you still need to handle the state where your data isn’t loaded! You’ll only be doing a server render on first load, so you’ll be showing loading screens on subsequent pages.

Change `index.html` for adding data

We need to send any prefetched data as part of our page request, so we’ll add a script tag:

<script>
window.DATA = {data:{}} // It doesn't really matter what this is, just keep it valid and replaceable
</script>

Serving

Then we need to add it to our search and replace. However, HTML uses a very basic script tag finder, so you’ll need to base-64 encode it if you have script tags. Also, don’t forget about our head tags!

// earlier on
const headMarkup = context.head.map(h=>(
  renderToStaticMarkup(h)
)).join('')

// then render
const RenderedApp = htmlData.replace('{{SSR}}', markup)
  .replace('{{head}}', headMarkup)
  .replace('{data:{}}', JSON.stringify(new Buffer(JSON.stringify(context.data)).toString('base64')))
if (context.code)
  res.status(context.code)
res.send(RenderedApp)

We also handle status code changes — for example, for a 404 — so if you have a 404 page you can just do this:

class NoMatch extends Component {
  componentWillMount(){
    const {staticContext} = this.props
    if (staticContext){
      staticContext.code = 404
    }
  }
  render() {
    return (
      <div>
        Sorry, page not found
      </div>
    )
  }
}

Summary

If you’re not sure what you’re doing, just use Next.js. It’s designed for server-side rendering and universal applications, or if you want the flexibility of doing everything manually, just the way you want. An example might include if you have data fetching in sub-components rather than at the page level.

Hopefully this article has helped you on your way! Don’t forget to checkout the GitHub repo for a working implementation.