Universal React Rendering: How We Rebuilt Sitepoint.com was peer reviewed by Stuart Mitchell, Matt Burnett and Joan Yin. Thanks to all of SitePoint’s peer reviewers for making SitePoint content the best it can be!
SitePoint has a lot of great content and it’s not just articles, so when we recently redesigned the SitePoint home page one of the goals was to do a better job of showing off other content like books, courses and videos from SitePoint Premium, and discussions from our forums. That posed a challenge as it meant pulling data from multiple sources outside of our main WordPress app.
In our development team, we really enjoy using React. The component model, and the pattern of having “data stores” in JavaScript, work really well for us and we find it results in a code base we enjoy working on.
We wanted to use these familiar tools but it was not going to be acceptable to send a mostly-empty homepage to users and then have them wait for all the different data sources to load and then render. So we knew what we wanted: our WordPress site to serve server-rendered content written in React. It would also be great if users could enjoy our site without being required to have JavaScript enabled, or should something cause our JavaScript to break.
Universal React Rendering and PHP
WordPress is PHP, React is JavaScript – so how do we do both server-side? Our solution was to create a Node.js proxy server. All requests come into this Node server and are passed straight on to our WordPress site. The proxy then inspects the returned page and does the React rendering before sending it on to the browser. The result is less a case of a full server-side app and more a case of basic pages from WordPress with a series of components that can be rendered/upgraded via JavaScript on the client or server.
While that idea sounds simple enough, and we found plenty of examples of rendering full pages via React, we found relatively few on how to upgrade an existing response. The tools we chose for this were:
- node-http-proxy, a full-featured HTTP proxy for Node.js
- and Harmon, middleware for node-http-proxy to modify the remote website response with trumpet
The best way to describe this process is with an example. Here’s a simple demo app. If you visit the /_src URL you can see the full source code, or check out the GitHub repo.
server/target.jsis a very simple HTTP server that always returns the same basic HTMLserver/proxy.jsimplementsnode-http-proxyforwarding all requests to the target serverserver/basicHarmon.jsprovides an Express middleware using Harmon to rewrite any<header></header>tags with some new, fancier contentserver/express.jscreates the Express server that uses the proxy and Harmon middleware
When you visit the app you should see the result you are served has the fancy header.
At this stage we only have a trivial example of replacing text in a header tag, but in the following sections I will show how this same approach can be used to render our React components. This approach gives us great flexibility. The Node.js app needs no knowledge of how WordPress works. It could just as easily be applied in front of a Rails app or anything else.
Components on SitePoint.com
As I mentioned we really enjoy working with React and the way we already use it on SitePoint stood us in good stead for the move to the server side.
If you have used React before then the following code block should look familiar.
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8" />
<title>Hello React!</title>
<script src="build/react.js"></script>
<script src="build/react-dom.js"></script>
<script src="https://unpkg.com/babel-core@5.8.38/browser.min.js"></script>
</head>
<body>
<div id="example"></div>
<script type="text/babel">
ReactDOM.render(
<h1>Hello, world!</h1>,
document.getElementById('example')
);
</script>
</body>
</html>
This comes straight from the React “getting started” page and shows how to render a single component into a page. What I see less often in examples is the use of multiple components in a page. Your site does not need to be a full single-page app in order to use multiple components. You are free to add multiple React mounting points and we think it’s a great pattern that shouldn’t be discounted. Here’s a trivial example:
See the Pen React Hello, Goodbye World by BradDenver (@BradDenver) on CodePen.
On SitePoint.com we use custom tags rather than element IDs and because it’s a pattern we use over and over again we have some helper functions to do that for us. First, we have the render() function that registers the custom tag with the browser and then locates all instances of that tag in the document.
function render (tag, Comp) {
document.createElement(tag);
const nodes = Array.from(document.getElementsByTagName(tag));
nodes.map((node, i) => renderNode(tag, Comp, node, i));
return Comp;
}
Second, we have renderNode() which performs the actual React render for each node after converting the nodes attributes to a props object.
function renderNode (tag, Comp, node, i) {
let attrs = Array.prototype.slice.call(node.attributes);
let props = {
key: `${ tag }-${ i }`,
};
attrs.map((attr) => props[attr.name] = attr.value);
if (!!props.class) {
props.className = props.class;
delete props.class;
}
ReactDOM.render(
<Comp { ...props }/>,
node
);
}
In this pen you can see multiple instances of a custom tag being rendered as React components with unique properties. The React app is a series of possible component entry points, not just one. We scan the DOM for recognized tags and React render into each. We have successfully used this technique for a while on the client side of SitePoint.com.
What’s great is that with some minor changes the same pattern can also be used server side.
const hasDocument = typeof document !== "undefined";
function render (tag, Comp) {
if (hasDocument) {
document.createElement(tag);
const nodes = Array.from(document.getElementsByTagName(tag));
nodes.map((node, i) => renderNode(tag, Comp, node, i));
} else {
__tags = [...__tags, {
query: tag,
func: (node, req) => serverRenderNode(tag, Comp, node, req)
}];
}
return Comp;
}
Above we see the updated render function and below is the server variant of renderNode.
let __tags = [];
let __id = 0;
function serverRenderNode (tag, Comp, node, req) {
let props = node.getAttributes();
__id++;
if (!!props.class) {
props.className = props.class;
delete props.class;
}
const nodeStream = node.createStream({outer: false});
try {
const html = ReactDOMServer.renderToString(
<Comp { ...props }/>
);
nodeStream.end(html);
} catch (err) {
nodeStream.end();
console.log("Rendered tag failed", tag, err);
};
}
__tags is now an array of query tags and render functions that Harmon can use to apply server rendering to the node proxy response. Here’s an update to the simple demo app and repo.
Again, if you visit the _src url you can see the full source code.
app/components/holds our React componentsapp/tools/holds our render functions and our hasDocument helperapp/index.jsinitialises our appserver/tagsHarmon.jsis the new middleware that renders the React components into the response html
Tada Data
Basic components have been useful thus far but it’s time to introduce data handling. On SitePoint.com we have used multiple Flux and Redux variations in the past but, as our data stores are fairly simple, MobX looked to be a good fit for us. MobX stores let us isolate fetch and parse logic for a single data domain in one spot.
MobX false start
First off, we create a basic posts store to hold a collection of posts. It will fetch the posts from an API on creation and then refresh at a set period. This article is not going to cover MobX in detail (see Matt Ruby’s article, How to Manage Your JavaScript Application State with MobX) but here are some quick points:
observable– signals a value that may change over timeaction– anything that modifies observable staterunInAction– something that asynchronously modifies observable stateobserver– will automatically run when some observable changes
In stores/index.js we create a stores observable object to wrap all the individual stores and give us a single observable state atom. To use this data we create a new PostList component to render out the list of posts. This component pulls the individual store it wants out into its state. Running the updated app (repo) all looks good until the post store updates again and we see:
Warning: forceUpdate(...): Can only update a mounting component. This usually means you called forceUpdate()
outside componentWillMount() on the server. This is a no-op. Please check the code for the PostList component.
Oh no!
Store or state
To fix this and ensure that all renders for a single page request use the same state (we could have multiple components using the posts store) we create a state snapshot, currentState. When running server-side we store a reference to currentState against the request object. Client-side, we can continue to use the observable store itself.
This is done via the following method:
export var storeOrState = (req) => {
if (hasDocument || typeof req === "undefined") {
return stores;
} else {
if (typeof req.SITEPOINT_state === "undefined") req.SITEPOINT_state = currentState;
return req.SITEPOINT_state;
}
}
We also want to move the logic for attaching the store to the component. We move this into our renderNode and serverRenderNode functions . They now check components for a propsFn allowing them to modify the props they will receive.
if (typeof Comp.propsFn === "function") props = {
...props,
...Comp.propsFn(props, __id, req),
};
We can now use this hook and the storeOrState function to add to the components props.
Here’s an update to the demo app and repo. Again if you visit the _src URL you can see the full source code.
Store Factory
We now have our app in a working state again but there are still improvements to be made.
Currently, we have to create all our stores at app startup or the calls to storeOrState() will fail. That means that all those stores would fire up client-side and possibly load data we don’t need for the current page or user.
To address this first we modify storeOrState to take a key to determine which individual store to retrieve and a factory function to create that store if it doesn’t already exist.
Second, we create a helper function postsStoreFor to house the logic of creating a posts store factory and then use the new version of storeOrState(). The component can now use that helper rather than reaching into storeOrState() directly.
Lastly, we modify the creation of our stores object so it stays empty client-side until sub-stores are needed. Server-side we create a few different types of post stores at start up so they are ready and primed with data if we need them.
Here’s the updated to the demo app and repo.
Client Data
Now we have our server-side data sorted out but client-side we have a problem. The client is starting with no data so it repeats all the data fetching that was done server side. We need to send the client the data it needs but preferably no more.
First we attach a list of stores used for any request in or storeOrState function.
if (typeof req.SITEPOINT_stores === "undefined") req.SITEPOINT_stores = [];
req.SITEPOINT_stores.push(key);
Next we create helper function storesForReq to reduce the full state snapshot to just the stores that the request is interested in.
export var storesForReq = (req) => {
if(typeof req.SITEPOINT_stores === "undefined") return {};
let reqStores = Array.from(new Set(req.SITEPOINT_stores))
.reduce((acc, key) => {
_set(acc, key, sn(key, req.SITEPOINT_state));
return acc;
}, {});
return reqStores;
}
Then we create a Harmon tag selector and function to render that into the DOM.
export default {
query: "client-initial-state",
func: (node, req) => {
node.createWriteStream({outer: true}).end(
`<script> window.INITIAL_STATE = ${JSON.stringify(storesForReq(req))}; </script>`
);
},
};
We add that to the knownTags function and now we can add our data to any page with a <client-initial-state> tag. If you check demo app you will see that only posts data for “Normal” and “Cats” posts has been included even though the server has fetched data for “Dogs” and “Chickens” too. The last thing to do here is update the Posts store to use the initial client data if it’s available client side.
Heres the updated to the demo app and repo.
Render Static
The demo is in pretty good shape now but there’s something going on that we also noticed on SitePoint.com: many of our components don’t change after the initial render so why send the client the HTML, render logic, and render data if they only ever use the HTML?
Our components already know how to add the stores they need to their props so it’s actually quite easy to for them to check if they have the required data to be fully rendered server-side. We add another static method to the component class.
static shouldServerRenderStatic(props, req) {
const { store: { loading, posts, type } } = props;
const hasCompleteData = posts.length && !loading;
if (hasCompleteData) deregisterStore(`posts.${type}`, req);
return hasCompleteData;
}
This method simply checks if the component props contain complete data and returns true or false. It also uses a new helper deregisterStore() so we don’t send data to the client for components that were able to be server render. serverRenderNode() is also updated with two new variables that make use of the components shouldServerRenderStatic method.
const shouldServerRenderStatic = (typeof Comp.shouldServerRenderStatic === "function")
? Comp.shouldServerRenderStatic(props, req)
: false;
const renderTo = (shouldServerRenderStatic)
? "renderToStaticMarkup"
: "renderToString";
Using this, ReactDOMServer.renderToString now becomes ReactDOMServer[renderTo] but more interestingly node.createStream({outer: false}) becomes node.createStream({outer: shouldServerRenderStatic}).
The outer option passed to node.createStream determines if the entire node is replaced when it’s true, or else just the node’s content is replaced. This is really handy because by setting it to true our custom tag is removed from the HTML and replaced with its rendered content. That means that when the client-side code runs looking for tags to turn into React components they aren’t there and so it is just treated like any other HTML.
On our production code, we also use webpack code splitting so the client doesn’t even load the JavaScript for the components if they aren’t needed client side but that’s beyond the scope of this article. The webpack code splitting docs cover it fairly well.
Here’s the final demo app and repo. As always if you visit the _src URL you can see the full source code.
The Finished Product
We are really happy with how our solution turned out. Firstly, we think SitePoint.com now does a lot better job of showing you our great content. Secondly, the technical solution is performant and maintainable.
It is also really flexible. At any time we can completely shut down our Node.js proxy and the site will continue to work as normal with some rendering done on the client side. When the proxy is running then users can enjoy our site without being required to have JavaScript enabled.
We haven’t had to change or remove any of the features we get from WordPress and we could use the same approach in front of other sites that might be built on Rails or anything else.
Brad DenverAfter wrangling code in one form or another for the best part of a decade Brad joined SitePoint as a Senior Developer in 2015. He likes his functions pure and his life full of side effects.
